GraphRAG with Qdrant and Neo4j: Architecture, Cost & Team Planning

LLM — The Extractor and Reasoner

The LLM does two things at different stages:

Ingestion time: It reads raw text and extracts a structured ontology — entities become nodes, relationships become edges. In the Qdrant demo, GPT-4o processed 25 sentences and identified relationship types like works_at, collaborates_on, mentors, reports_to, and transferred_from. Each entity gets a UUID that persists across both databases.

Query time: It receives the assembled graph context (nodes + edges as readable triples) and generates the final answer by reasoning over the relationship chain.

Qdrant — The Semantic Entry Point

Qdrant stores vector embeddings of the raw text chunks. At query time, it converts the user’s question into an embedding and performs cosine similarity search to find the top-5 most relevant entries. Critically, each vector entry carries a payload with the UUID of its corresponding Neo4j node.

Qdrant’s job is narrowing — it filters millions of entities down to the handful most semantically relevant to the query. Without this step, graph traversal would be unbounded and expensive.

Neo4j — The Relationship Expander

Neo4j stores the knowledge graph: nodes with attributes, edges with typed relationships. When Qdrant returns entity IDs, Neo4j runs a Cypher query to expand the subgraph — fetching 1-hop and 2-hop neighbors along with their relationship types.

Neo4j’s job is connecting — it traces paths that vector search alone would miss. Bob and New York have low cosine similarity, but Neo4j finds Bob → collaborates_on → Carol → works_at → New York, a two-hop path that answers the question.

The Flow End-to-End

Ingestion (one-time):

Query (per request):

The three systems are complementary: the LLM understands language, Qdrant finds relevant starting points, and Neo4j reveals hidden connections. Remove any one and the system degrades — no LLM means no extraction or reasoning, no Qdrant means unbounded graph traversal, no Neo4j means you’re back to flat chunk retrieval.

Ingestion Pipeline

Raw data flows through two parallel tracks:

1. Ontology extraction — An LLM (GPT-4o) analyzes text and extracts entities (nodes) and relationships (edges), outputting structured JSON via Pydantic models
2. Vector embeddings — An embedding model (text-embedding-3-small) converts text chunks into 1,536-dimensional vectors

Both tracks share unique UUIDs that link Neo4j nodes to Qdrant vector entries, enabling cross-referencing during retrieval.

Retrieval Pipeline

Query processing follows a three-stage cascade:

1. Vector search — Qdrant returns top-5 entity IDs via cosine similarity
2. Graph expansion — Neo4j fetches 1-hop + 2-hop neighbors with relationship types
3. Context assembly — Nodes and edges formatted as readable triples for LLM reasoning

The retriever uses QdrantNeo4jRetriever from the neo4j_graphrag package, which handles the dual-database query coordination automatically.

Key Integration Points

Entity Extraction with Structured Output

The LLM extracts graph components using a strict JSON schema enforced by Pydantic:

1
class Single(BaseModel):
2
    node: str
3
    target_node: str
4
    relationship: str
5

6
class GraphComponents(BaseModel):
7
    graph: list[Single]

The system prompt instructs the LLM to extract ALL relationships including implicit ones. Response format is set to json_object for reliable parsing. Each extracted entity receives a UUID that persists across both databases.

Neo4j Ingestion Pattern

Nodes are created with explicit IDs, then relationships are established via MATCH + CREATE:

1
# Create nodes
2
session.run(
3
    "CREATE (n:Entity {id: $id, name: $name})",
4
    id=node_id, name=name
5
)
6

7
# Create relationships
8
session.run(
9
    "MATCH (a:Entity {id: $source_id}), (b:Entity {id: $target_id}) "
10
    "CREATE (a)-[:RELATIONSHIP {type: $type}]->(b)",
11
    source_id=rel["source"], target_id=rel["target"], type=rel["type"]
12
)

Graph Context Retrieval

The subgraph fetch uses a UNION query to capture both 1-hop and 2-hop relationships:

1
MATCH (e:Entity)-[r1]-(n1)-[r2]-(n2)
2
WHERE e.id IN $entity_ids
3
RETURN e, r1, n1 AS related, r2, n2
4
UNION
5
MATCH (e:Entity)-[r]-(related)
6
WHERE e.id IN $entity_ids
7
RETURN e, r, related, NULL, NULL

This pattern ensures that even if a 2-hop path doesn’t exist, the 1-hop relationship is still returned. The formatted context becomes a list of readable triples for the LLM.

Qdrant Collection Setup

1
client.create_collection(
2
    collection_name="graphRAGstoreds",
3
    vectors_config=models.VectorParams(
4
        size=1536,  # text-embedding-3-small dimension
5
        distance=models.Distance.COSINE
6
    )
7
)

Collections store vectors with payload containing the Neo4j entity ID for cross-referencing.

The Cost Cliff: $33,000 to$33

Microsoft Research documented a dramatic cost reduction in GraphRAG indexing. In early 2024, indexing a 5GB legal dataset cost $33,000** in LLM processing alone. By mid-2025, optimized pipelines reduced this to **$33 — a 99.9% decrease.

This was achieved through:

• Smaller, specialized models for entity extraction (gpt-4o-mini instead of gpt-4)
• Batch processing optimizations
• Reduced prompt sizes and token counts
• Better chunking strategies that minimize LLM calls

Indexing Cost Breakdown

For a 100-document corpus (500 chunks average):

• Entity extraction: 500 LLM calls × ~$0.01 =$5.00 (gpt-4o-mini)
• Embeddings: 500 calls × ~$0.00002 =$0.01
• Total indexing: ~$5.01

Query-Time Costs

Each query triggers multiple LLM interactions:

At scale (10,000 queries/month):

• Embeddings: 10,000 × $0.00002 =$0.20
• Completions: 10,000 × $0.01 =$100.00 (gpt-4o)
• Total: ~$100.20/month

Infrastructure Costs

Total Cost of Ownership by Scale

Cost Optimization Strategies

Failed to render: SSR render timeout

Core Roles

Skills Matrix

Team Size by Project Scale

Hiring Priorities

1. Graph Database Engineer — This is the scarcest skill. Most RAG teams have never worked with property graphs or Cypher. This role defines the knowledge graph schema and optimizes traversal queries.
2. LLM Engineer with extraction focus — Not just prompt writing, but structured output parsing, error handling for malformed LLM responses, and cost optimization.
3. Backend Engineer — Standard role, but needs familiarity with dual-database architectures and async retrieval patterns.

Training Path for Existing Teams

For teams already running vector-based RAG:

LLM-Dependent Graph Construction

The knowledge graph quality depends entirely on the LLM’s extraction accuracy. Errors propagate: a misidentified relationship creates a wrong edge that affects all subsequent queries traversing that path. Mitigation strategies include:

• Using lower-temperature settings (0.0-0.2) for deterministic extraction
• Post-processing validation rules (e.g., relationship type constraints)
• Periodic graph audits and manual corrections

Cypher Translation Errors

When using Text2Cypher (natural language to Cypher query generation), the LLM may produce syntactically valid but semantically wrong queries. The Qdrant + Neo4j approach partially mitigates this by using fixed Cypher templates rather than dynamic generation.

Scalability Limits

Graph traversal cost grows with neighborhood size. A highly connected entity (e.g., “New York” in a corporate dataset) may have hundreds of 2-hop neighbors, bloating the context window. Solutions include:

• Relationship type filtering (only traverse relevant edge types)
• Community detection to pre-cluster related entities
• Query-specific depth limits

Maintenance Overhead

Every data update requires re-indexing both the vector store and graph database. Unlike pure vector RAG where you simply upsert new embeddings, GraphRAG needs to reconcile entity changes, update relationships, and maintain ID consistency across both systems.