Transcribed
RAG on Azure — Simply Explained
Jul 16, 2026 ·
13m 19s
Download and listen anywhere
Download your favorite episodes and enjoy them, wherever you are! Sign up or log in now to access offline listening.
RAG on Azure — Simply Explained
Description
Large language models do not know your company's data and their knowledge is frozen in time. This Simply Explained episode shows developers, architects and IT pros how retrieval-augmented generation on...
show more
Large language models do not know your company's data and their knowledge is frozen in time. This Simply Explained episode shows developers, architects and IT pros how retrieval-augmented generation on Azure solves that with Azure AI Search, Azure OpenAI and Azure AI Foundry, and how to start building it the right way.
In this episode
Key takeaways
Worth quoting
“Ragh isn't a new model. It's not a bigger or better GPT. It's a different approach entirely.” – Mirko Peters (00:42)
“The LLM never sees your raw data. It only sees what the retrieval system hands it. It's not a drawback, so it's a security feature. Your sensitive documents stay in your own storage.” – Mirko Peters (04:08)
Explore more on M365.FM
Follow and rate M365.FM in your podcast app, and join the free M365 community at m365con.net.
Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-microsoft-365-azure-cloud-power-platform-ai--6704921/support.
show less
In this episode
- 00:00 – What RAG is and why it matters
- 01:36 – The two problems LLMs have
- 03:05 – How retrieval-augmented generation works
- 04:50 – Azure AI Search: indexing and searching
- 06:37 – Azure OpenAI and Azure AI Foundry
- 09:03 – Classic RAG vs. agentic RAG
- 10:28 – End-to-end example
- 12:10 – How to start building RAG on Azure
Key takeaways
- RAG first retrieves relevant content from your own data and passes it to the model with the question, so answers are current and grounded instead of guessed from training memory.
- On Azure, Azure AI Search indexes and retrieves chunks with vector or hybrid search, Azure OpenAI generates the answer, and Azure AI Foundry wires the agent together.
- Start with your data, get retrieval right before adding the LLM, and choose agentic RAG only for complex questions where accuracy is worth the extra tokens and latency.
Worth quoting
“Ragh isn't a new model. It's not a bigger or better GPT. It's a different approach entirely.” – Mirko Peters (00:42)
“The LLM never sees your raw data. It only sees what the retrieval system hands it. It's not a drawback, so it's a security feature. Your sensitive documents stay in your own storage.” – Mirko Peters (04:08)
Explore more on M365.FM
- Episode page: the full episode with chapters, quotes and FAQ on building RAG on Azure
- Retrieval-augmented generation (RAG) – the core pattern of the episode: retrieve your own content first, then let the model answer
- Azure AI Search – the retrieval layer that chunks, embeds and indexes your documents and finds the best matches
- Azure OpenAI Service – provides the GPT model that generates grounded answers from the retrieved context
- Related episode: Vector Databases - Simply Explained
Follow and rate M365.FM in your podcast app, and join the free M365 community at m365con.net.
Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-microsoft-365-azure-cloud-power-platform-ai--6704921/support.
Information
| Author | Mirko Peters M365, Azure & AI |
| Organization | m365 FM |
| Website | www.m365.fm |
| Tags |
Copyright 2026 - Spreaker Inc. an iHeartMedia Company
Comments