Vector Search Can't Answer 'What Are the Main Themes?' — GraphRAG Builds a Knowledge Graph That Can

GraphRAG (Microsoft) extracts entities and relationships from your corpus into a knowledge graph, generates community summaries, and uses those summaries to answer synthesis questions that stumped vector search.

What you'll be able to do after this:

• Index a document corpus into a knowledge graph with five CLI commands
• Query it with global synthesis questions ("what are the main themes?") and local entity lookups ("who is X and what do they do?")
• Know when GraphRAG's higher index cost is worth paying versus sticking with plain vector search

Resource: GraphRAG: The Most Incredible RAG Strategy Revealed (Mervin Praison, July 2024) — 11-minute beginner walkthrough; or the Official Quickstart for the current v1.0 CLI.

---

Why vector RAG fails on synthesis questions

Standard RAG retrieves the passages most similar to your query. That works when the answer lives in a single passage — "What is the return policy?" retrieves the return-policy chunk. It fails completely for "What are the three main themes across all 200 documents?" — no single chunk contains that answer, so cosine similarity has nothing to return.

GraphRAG (Microsoft Research, open-sourced July 2024) solves this with a structured knowledge-graph index built at ingest time.

The four-stage GraphRAG pipeline

Stage 1 — Chunk. Documents split into overlapping TextUnits (~300 tokens each, configurable).

Stage 2 — Extract (expensive). An LLM reads every TextUnit and extracts named entities (people, orgs, concepts) plus relationships between them, each with a description. When the same entity appears across chunks, descriptions are merged into one canonical summary.

Stage 3 — Cluster. The entity-relationship graph is clustered using the Leiden algorithm into a hierarchy — tight topic clusters at the fine level, broad thematic communities at the coarse level.

Stage 4 — Summarize. Each community gets an LLM-generated summary report at each hierarchy level. These reports are the artifacts that power global queries.

Five CLI commands to a working index

pip install graphrag

mkdir my_graphrag && cd my_graphrag
graphrag init          # creates .env, settings.yaml, input/ directory

# Edit .env: add GRAPHRAG_API_KEY=<your-openai-key>
# Drop your .txt documents into input/, then:

graphrag index         # extracts entities, clusters, summarizes — builds the graph

Two query modes

# Global: decomposes query → fans out to ALL community summaries → synthesizes
# Best for: "what are the themes?", "compare the key arguments", "what patterns exist?"
graphrag query "What are the main themes across all documents?"

# Local: finds relevant entities → traverses graph edges → fetches context + summaries
# Best for: "who is X?", "what is the relationship between X and Y?"
graphrag query "Who is the main antagonist and what are their motivations?" --method local

Cost tradeoff and the budget option

GraphRAG indexing is expensive. LLM extraction runs over every chunk; community summarization runs over every cluster level. A ~30K-word document costs ~$0.34 with GPT-4-Turbo. Large enterprise corpora can run hundreds or thousands of dollars.

LazyGraphRAG (the budget option): uses NLP co-occurrence instead of LLM extraction at index time — same indexing cost as plain vector RAG. LLM is only called at query time. Reaches roughly 90% of full GraphRAG quality at a fraction of the cost. Enable with --method lazy during graphrag index.

When to use GraphRAG vs vector RAG

Use case	Better choice
"What are the main themes / trends / patterns?"	GraphRAG global mode
"Summarize this entire corpus"	GraphRAG global mode
"Who is X and what is their relationship to Y?"	GraphRAG local mode
Direct factual recall ("what is X?")	Vector RAG
Latency-sensitive (< 1s)	Vector RAG
Tight budget, no synthesis needs	Vector RAG

Sources: Microsoft GraphRAG — GitHub · GraphRAG Quickstart — Official Docs · From Local to Global: A GraphRAG Approach to Query-Focused Summarization — arXiv · LazyGraphRAG — Microsoft Research Blog