AI
RAG
Vector Database
LLM
Python
Ask anything about this article
Hi! I've read this article.
What would you like to know?
@farhan
Basic RAG (retrieve chunks → stuff into prompt → generate) has a ceiling. If you've built a RAG system and hit accuracy issues, hallucinations, or irrelevant retrievals — this guide is for you.
Most basic RAG pipelines fail because:
Combine keyword search (BM25) with vector search, then rerank:
pythonfrom sentence_transformers import CrossEncoder
# Step 1: Hybrid retrieval
bm25_results = bm25_search(query, top_k=20)
vector_results = vector_search(query, top_k=20)
# Step 2: Merge and deduplicate
candidates = merge_results(bm25_results, vector_results)
# Step 3: Cross-encoder reranking
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-12-v2")
scores = reranker.predict([(query, doc.text) for doc in candidates])
# Step 4: Take top results
reranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
final_context = [doc for doc, _ in reranked[:5]]
Microsoft's GraphRAG builds a knowledge graph from your documents:
python# 1. Extract entities and relationships using LLM
entities = extract_entities(document)
# Output: [("OpenAI", "COMPANY"), ("GPT-4", "MODEL"), ...]
# 2. Build knowledge graph
graph.add_edge("OpenAI", "GPT-4", relation="CREATED")
graph.add_edge("GPT-4", "Transformer", relation="BASED_ON")
# 3. Community detection for summarization
communities = detect_communities(graph)
# 4. Query using graph traversal + vector search
results = graph_rag_query(
"What models has OpenAI created?",
graph=graph,
vector_store=vector_store
)
Let the LLM decide what to search for and when:
pythonclass AgenticRAG:
def answer(self, question: str) -> str:
plan = self.llm.plan_retrieval(question)
context = []
for step in plan:
if step.type == "SEARCH":
results = self.search(step.query)
context.extend(results)
elif step.type == "FILTER":
context = self.filter(context, step.criteria)
elif step.type == "SYNTHESIZE":
partial = self.llm.synthesize(context)
context = [partial]
return self.llm.generate_final(question, context)
Instead of fixed-size chunks, create semantically meaningful chunks:
python# Add document-level context to each chunk
def contextual_chunk(document: str, chunks: list) -> list:
doc_summary = llm.summarize(document)
enhanced_chunks = []
for chunk in chunks:
enhanced = f"""
Document Context: {doc_summary}
Section: {chunk.section_title}
Content: {chunk.text}
"""
enhanced_chunks.append(enhanced)
return enhanced_chunks
| Component | Basic RAG | Advanced RAG |
|---|---|---|
| Chunking | Fixed 512 tokens | Semantic + contextual |
| Search | Vector only | Hybrid + reranking |
| Retrieval | Single query | Multi-query + agentic |
| Context | Raw chunks | Summarized + graph |
| Evaluation | None | RAGAS metrics |
Use RAGAS framework for evaluation:
Advanced RAG isn't about one technique — it's about combining the right techniques for your specific use case.