This document describes the Agentic Retrieval-Augmented Generation (Agentic RAG) pattern, an advanced AI agent design where Large Language Models (LLMs) autonomously plan retrieval strategies, iteratively refine queries, and self-correct until achieving satisfactory results. Unlike traditional RAG implementations that follow a fixed retrieve-then-read sequence, Agentic RAG agents actively own their reasoning process through looped interactions with retrieval tools.
For foundational tool invocation mechanisms that enable Agentic RAG, see Tool Use Pattern. For agents that decompose complex tasks into subtasks, see Planning Pattern. For production deployment considerations including observability of agentic retrieval loops, see Observability and Evaluation.
Sources: 05-agentic-rag/README.md5-16 04-tool-use/README.md27-30
Agentic RAG distinguishes itself through the agent's ability to own its reasoning process. Rather than executing a pre-scripted retrieval workflow, the agent dynamically decides:
The agent operates in a "maker-checker" style loop where each retrieval result is evaluated before proceeding. If results are inadequate, the agent reformulates the query, tries alternative retrieval methods, or invokes additional tools autonomously.
Key Characteristics:
| Aspect | Traditional RAG | Agentic RAG |
|---|---|---|
| Query Execution | Single retrieval call | Iterative refinement loop |
| Tool Selection | Pre-defined by developer | Dynamically chosen by agent |
| Quality Control | Manual prompt engineering | Autonomous self-correction |
| Failure Handling | Returns low-quality results | Re-queries or tries alternative tools |
Sources: 05-agentic-rag/README.md31-56 05-agentic-rag/README.md58-66
The core execution pattern consists of a continuous loop where the agent evaluates retrieval quality and decides next actions:
Title: Agentic RAG Core Loop
Loop Components:
SearchPlugin.retrieve_documents) to gather context 05-agentic-rag/README.md65Sources: 05-agentic-rag/README.md43-70 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb115-124
Agentic RAG systems maintain a registry of retrieval tools that the agent can dynamically invoke. In the Microsoft Agent Framework and Semantic Kernel, this is achieved through plugins.
Title: Tool Selection and Code Entity Mapping
Implementation Pattern:
The SearchPlugin class demonstrates tool encapsulation with multiple retrieval capabilities:
retrieve_documents(query: str): Executes semantic search against Azure AI Search index using SearchClient 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb115-124build_augmented_prompt(query: str, retrieval_context: str): Constructs augmented prompts with specific instructions for self-correction 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb104-113The agent receives function schemas through the @kernel_function decorator mechanism, enabling autonomous tool discovery and invocation 03-agentic-design-patterns/README.md55-56
Sources: 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb94-155 04-tool-use/README.md45-52
Agentic RAG implements robust self-correction through multiple strategies:
When retrieval results are inadequate, the agent autonomously rewrites queries. The build_augmented_prompt function explicitly instructs the agent:
"First review the retrieved context, if this does not answer the query, try calling an available plugin functions that might give you an answer." 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb108-111
The agent maintains awareness of multiple retrieval tools and can switch strategies mid-execution. For example, if retrieve_documents returns insufficient results, the agent may invoke get_destination_temperature to supplement context 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb133-154
Title: Self-Correction Sequence
Sources: 05-agentic-rag/README.md72-82 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb104-124
Agentic RAG systems maintain conversation state using ChatHistoryAgentThread, enabling the agent to track previous retrieval attempts and avoid repetitive loops 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb249
| Component | Purpose | Implementation |
|---|---|---|
ChatHistoryAgentThread | Persists conversation context across turns | thread = ChatHistoryAgentThread() 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb249 |
invoke_stream | Executes agent logic while maintaining thread state | agent.invoke_stream(thread=thread) 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb272 |
FunctionCallContent | Tracks which tools the model decided to call | Captured in stream processing 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb280-285 |
Sources: 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb249-325 05-agentic-rag/README.md66-70
The implementation uses ChatCompletionAgent with plugin-based tool integration and streaming invocation.
Title: Agentic RAG Code Structure
Key Implementation Patterns:
SearchPlugin and WeatherInfoPlugin to the agent's plugin collection 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb225-230SearchClient with endpoint and index name 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb172-180agent.invoke_stream to handle FunctionCallContent and FunctionResultContent 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb272-302Sources: 05-agentic-rag/code_samples/05-semantic-kernel-azuresearch.ipynb1-387 02-explore-agentic-frameworks/README.md398-551
Agentic RAG systems implement defensive mechanisms against common failure patterns:
| Failure Mode | Description | Mitigation Strategy |
|---|---|---|
| Infinite Retrieval Loops | Agent repeatedly queries without convergence | Turn limiting or max iteration constraints 06-building-trustworthy-agents/README.md130-131 |
| Query Hallucination | Agent generates invalid queries for structured databases | Input filters and validation checks 06-building-trustworthy-agents/README.md130 |
| Retrieval Service Failures | Azure AI Search or external API unavailable | Implement fallbacks or retries 10-ai-agents-production/README.md52-53 |
Sources: 05-agentic-rag/README.md72-82 06-building-trustworthy-agents/README.md120-154 10-ai-agents-production/README.md52-53
Sources: 05-agentic-rag/README.md93-100 05-agentic-rag/README.md49-56
To build trustworthy Agentic RAG systems:
Sources: 05-agentic-rag/README.md101-116 03-agentic-design-patterns/README.md73-88 06-building-trustworthy-agents/README.md1-206
Refresh this wiki