DSPy 3 + GEPA: Building Optimized Agentic Chat with Reflective Prompt Evolution

The Problem with Manual Agent Prompts

Building an agentic chat system with DSPy 3 is straightforward: define signatures, compose modules, attach tools, and call forward(). But getting the prompts right — so the agent picks the correct tools, reasons through multi-step problems, and produces grounded answers — is still manual prompt engineering.

Reinforcement learning from human feedback (RLHF) and its variants (GRPO, RLVR) can automate this, but they require thousands of rollouts and are designed for weight updates. GEPA takes a fundamentally different approach: evolve the prompts, not the weights.

The weight updates of the future are going to be prompt updates. — Matei Zaharia, Berkeley RDI

What Is GEPA?

GEPA stands for Genetic-Pareto — it’s an evolutionary algorithm for prompt optimization contributed to the DSPy framework by Omar Khattab’s group at Stanford, with core work by Matei Zaharia’s lab at Berkeley. The paper (arXiv

Three principles define GEPA:

1. Reflective prompt mutation — Instead of scalar rewards (like RL), GEPA has an LLM reflect on its own execution traces in natural language and propose specific prompt improvements
2. Pareto-based candidate selection — GEPA maintains a frontier of non-dominated candidates. Each survivor is the best performer on at least one validation example, preserving diversity instead of hill-climbing to a local optimum
3. System-aware merging — The best modules from different evolutionary branches can be combined, so improvements in one part of the pipeline don’t get lost when another part changes

The headline result: GEPA achieves higher accuracy in tens of rollouts than GRPO achieves in 25,000 rollouts on HotpotQA. On average, it beats GRPO by 6% and MIPROv2 by over 10%, using up to 35x fewer evaluations.

How GEPA Works: The Algorithm

GEPA operates on DSPy programs — composed modules of signatures and predictors. The optimization loop:

1. Initialize — The unoptimized program becomes the first candidate in the pool
2. Select — Sample a candidate from the Pareto frontier (the set of programs where no other candidate is better on all examples)
3. Sample — Draw a minibatch of training examples (default: 3)
4. Execute — Run the candidate on the minibatch, collecting full execution traces
5. Evaluate — Score each trace using the metric function (which returns both a score and textual feedback)
6. Select component — Pick one predictor to improve (round-robin by default, so each module gets attention)
7. Reflect — A reflection LLM reads the traces, scores, and feedback, then proposes a new instruction for the selected predictor
8. Roll out — Create a new candidate with the mutated prompt and evaluate on the validation set
9. Update frontier — If the new candidate dominates any existing one, update the Pareto frontier
10. Merge — Optionally combine the best modules from different lineages
11. Repeat until the budget is exhausted

This is not random search. The reflection step is directed — the LLM reads why a trajectory failed (from the textual feedback) and proposes a specific fix. Over generations, prompts evolve to handle edge cases that manual engineering would miss.

The GEPA API: Getting Started

GEPA extends dspy.Teleprompter and lives at dspy.teleprompt.gepa.gepa. The constructor:

1
import dspy
2

3
gepa = dspy.GEPA(
4
    metric=metric_fn,                    # REQUIRED: your evaluation function
5
    auto="medium",                        # Budget: "light" (6 evals), "medium" (12), "heavy" (18)
6
    reflection_lm=dspy.LM('openai/gpt-4o', temperature=1.0, max_tokens=32000),
7
    reflection_minibatch_size=3,          # Examples per reflection step
8
    candidate_selection_strategy="pareto", # "pareto" or "current_best"
9
    component_selector="round_robin",     # "round_robin", "all", or custom
10
    use_merge=True,                       # Combine best modules from different branches
11
    skip_perfect_score=True,              # Skip examples where score = 1.0
12
    add_format_failure_as_feedback=True,  # Include parse errors as feedback
13
    track_stats=True,                     # Store all candidates for analysis
14
    log_dir="./gepa_logs",                # Checkpoint directory
15
    num_threads=1,                        # Parallel evaluation
16
)

Budget control is mutually exclusive — pick exactly one of auto, max_full_evals, or max_metric_calls:

1
# Option 1: Preset budget (recommended for starting)
2
gepa = dspy.GEPA(metric=m, auto="medium", reflection_lm=...)
3

4
# Option 2: Max full evaluations on valset
5
gepa = dspy.GEPA(metric=m, max_full_evals=10, reflection_lm=...)
6

7
# Option 3: Granular control (most precise)
8
gepa = dspy.GEPA(metric=m, max_metric_calls=500, reflection_lm=...)

Compile your program:

1
optimized = gepa.compile(
2
    student=my_agent,           # The DSPy module to optimize
3
    trainset=train_examples,    # Examples for reflection minibatches
4
    valset=val_examples,        # Examples for Pareto evaluation (defaults to trainset)
5
)

When track_stats=True, the result object (DspyGEPAResult) provides access to all candidates, their scores, and lineage information.

The Metric Function: GEPA’s Secret Sauce

The metric is where GEPA diverges from every other optimizer. It has a 5-argument signature that enables module-specific feedback:

1
def metric(
2
    gold,          # The gold example from your dataset
3
    pred,          # The prediction from the program
4
    trace,         # Full execution trace of all modules
5
    pred_name,     # Name of the specific predictor GEPA is currently evaluating
6
    pred_trace,    # Sub-trace for that specific predictor only
7
):
8
    ...

GEPA calls the metric multiple times per example — once per predictor. This is what enables predictor-level reflection: the feedback you return for pred_name="classifier" can be completely different from the feedback for pred_name="agent".

Return a dspy.Prediction(score=float, feedback=str) for best results. The feedback string is what the reflection LLM reads to propose improvements:

1
def chat_metric(gold, pred, trace=None, pred_name=None, pred_trace=None):
2
    # Program-level: did the agent answer correctly?
3
    correct = normalize(pred.response) == normalize(gold.answer)
4
    score = 1.0 if correct else 0.0
5

6
    # Predictor-level: targeted feedback
7
    if pred_name == "classifier":
8
        missed = set(gold.intents) - set(pred.intents or [])
9
        feedback = f"Missed intents: {missed}. Gold intents: {gold.intents}."
10
        return dspy.Prediction(score=score, feedback=feedback)
11

12
    elif pred_name == "agent":
13
        # Check tool usage from the trace
14
        tools_used = []
15
        if pred_trace:
16
            for step_name, (inputs, outputs) in pred_trace.items():
17
                if "tool_name" in outputs:
18
                    tools_used.append(outputs["tool_name"])
19
        feedback = (
20
            f"Task {'completed' if correct else 'failed'}. "
21
            f"Tools used: {tools_used}. "
22
            f"Review whether the right tools were selected for this query type."
23
        )
24
        return dspy.Prediction(score=score, feedback=feedback)
25

26
    # Program-level fallback
27
    return dspy.Prediction(
28
        score=score,
29
        feedback=f"Expected: {gold.answer}. Got: {pred.response}.",
30
    )

If you return only a float (no feedback), GEPA falls back to a generic message like "This trajectory got a score of 0.5." — it works, but you lose the targeted improvement signal that makes GEPA powerful.

Building an Agentic Chat Agent with DSPy 3

DSPy 3 provides several building blocks for agentic systems:

1
@dspy.tool
2
def search_knowledge_base(query: str) -> str:
3
    """Search the internal knowledge base for relevant documents."""
4
    return kb_search(query)
5

6
@dspy.tool
7
def lookup_customer(customer_id: str) -> str:
8
    """Look up customer information and order history."""
9
    return db.get_customer(customer_id)

1
class SupportAgent(dspy.Module):
2
    def __init__(self, tools):
3
        self.classifier = dspy.ChainOfThought(
4
            "conversation_history, user_message -> intent, entities"
5
        )
6
        self.agent = dspy.ReAct(
7
            "conversation_history, user_message, intent, entities -> response",
8
            tools=tools,
9
            num_traces=5,
10
        )
11

12
    def forward(self, conversation_history="", user_message=""):
13
        cls = self.classifier(
14
            conversation_history=conversation_history,
15
            user_message=user_message,
16
        )
17
        result = self.agent(
18
            conversation_history=conversation_history,
19
            user_message=user_message,
20
            intent=cls.intent,
21
            entities=cls.entities,
22
        )
23
        return dspy.Prediction(response=result.response, intent=cls.intent)

1
class ChatSignature(dspy.Signature):
2
    """You are a helpful customer support assistant."""
3
    conversation_history: str = dspy.InputField(desc="Prior messages in the conversation")
4
    user_message: str = dspy.InputField(desc="The current user message")
5
    response: str = dspy.OutputField(desc="Your response to the user")

Optimizing the Agent with GEPA

This is where GEPA shines. Instead of hand-tuning the classifier prompt, the ReAct reasoning instructions, and the tool descriptions independently, you define a metric and let GEPA evolve them together.

1
# Configure: use a cheaper model for execution, stronger model for reflection
2
dspy.configure(lm=dspy.LM('openai/gpt-4.1'))
3

4
agent = SupportAgent(tools=[search_knowledge_base, lookup_customer])
5

6
gepa = dspy.GEPA(
7
    metric=chat_metric,
8
    auto="medium",
9
    reflection_lm=dspy.LM('openai/gpt-4o', temperature=1.0, max_tokens=32000),
10
    component_selector="round_robin",
11
    use_merge=True,
12
    add_format_failure_as_feedback=True,
13
    track_stats=True,
14
    log_dir="./gepa_logs/support_agent",
15
)
16

17
optimized_agent = gepa.compile(
18
    student=agent,
19
    trainset=train_examples,
20
    valset=val_examples,
21
)

The component_selector="round_robin" is critical for multi-module programs. GEPA optimizes one predictor at a time — first the classifier’s prompt, then the ReAct loop’s reasoning instructions, then the answer extractor. Over multiple generations, each module gets refined based on feedback specific to its role.

1
# Freeze the classifier — only optimize the ReAct agent
2
agent.classifier._compiled = True
3
agent.agent._compiled = False
4

5
# Or freeze only the answer extractor within ReAct
6
agent.agent.extract._compiled = True
7
agent.agent.react._compiled = False

Multi-Agent Patterns with GEPA

For complex chat systems, a common pattern is a lead agent that delegates to specialized sub-agents. Each sub-agent is itself a DSPy module — and each can be independently optimized with GEPA.

1
# Step 1: Optimize each sub-agent separately
2
diabetes_agent = gepa.compile(student=DiabetesAgent(), trainset=diabetes_train)
3
copd_agent = gepa.compile(student=COPDAgent(), trainset=copd_train)
4

5
# Step 2: Wrap optimized agents as tools
6
def ask_diabetes(question: str) -> str:
7
    return diabetes_agent(question=question).answer
8

9
def ask_copd(question: str) -> str:
10
    return copd_agent(question=question).answer
11

12
# Step 3: Build lead agent with sub-agent tools
13
class LeadAgent(dspy.Module):
14
    def __init__(self):
15
        self.lead = dspy.ReAct(
16
            "patient_history, current_question -> triage_decision, response",
17
            tools=[ask_diabetes, ask_copd, search_guidelines],
18
            num_traces=5,
19
        )
20

21
# Step 4: Optimize the lead agent
22
optimized_lead = gepa.compile(student=LeadAgent(), trainset=lead_train)

Real-world results from this pattern (reported by practitioners): ~8% improvement for sub-agents in about 10 minutes, ~4% for the lead agent. The reflection LM is typically a stronger model (GPT-4o or GPT-5) while the execution LM can be cheaper (GPT-4.1 or Gemini Flash).

GEPA for Reranking: The Weaviate Example

The Weaviate tutorial demonstrated GEPA optimizing a listwise reranker — a different but illustrative use case. Given a query and candidate documents, the reranker produces a relevance-ordered list.

Key setup details:

• Hard example filtering: From 1,000 Enron QA questions, they filtered to 138 where cross-encoders achieve Recall@5 but not Recall@1 — the cases where reranking actually matters
• Reflection LM: GPT-5 with 32K tokens, temperature 1.0
• Budget: 500 metric calls (~1.5 hours)
• Result: Recall@1 improved from 31.6% to 44.7% (+42% relative improvement)

The optimized prompt was approximately 2,000 tokens — GEPA learned specific rules for relevance assessment that a human would struggle to write manually.

Practical Lessons from Production Deployments

Benchmarks at a Glance

DSPy 3 Version Compatibility

GEPA is available in DSPy 3.0+. Key version milestones:

• 3.0.0 — Initial GEPA support
• 3.0.4 — Custom instruction_proposer, MLflow tracking, gepa_kwargs
• 3.1.0 — dspy.Reasoning module, Python 3.14 support
• 3.2.0 — BetterTogether chains GEPA with BootstrapFinetune (strategy="p -> w -> p"), decoupled from LiteLLM

Install: pip install dspy-ai

The Bigger Picture

GEPA represents a shift in how we think about AI system optimization. Instead of treating prompt engineering as a one-time human task or relying on expensive RL fine-tuning, GEPA treats prompts as evolvable programs — subject to mutation, selection, and recombination, just like genetic algorithms in other domains.

For agentic chat systems specifically, this means:

• No more prompt whack-a-mole — fix one edge case, break another. GEPA optimizes across all examples simultaneously
• Module-level precision — the classifier, the reasoning loop, and the answer extractor each get targeted improvements
• Merge across branches — improvements found while optimizing tool selection aren’t lost when the reasoning strategy changes
• No weight updates needed — works with any API model, including closed-source models like GPT-4.1

The weight updates of the future may indeed be prompt updates. GEPA is the first optimizer that makes that practical for complex, multi-step agent systems.

References

1. “GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning” — Agrawal et al., arXiv
   .19457 (ICLR 2026 Oral) — https://arxiv.org/abs/2507.19457
2. DSPy GEPA API Documentation — https://dspy.ai/api/optimizers/GEPA/overview/
3. DSPy Official Documentation — https://dspy.ai
4. GEPA Source Code — https://github.com/gepa-ai/gepa
5. DSPy GitHub Repository — https://github.com/stanfordnlp/dspy
6. GEPA Hands-On Reranker Notebook — Weaviate — https://github.com/weaviate/recipes/blob/main/integrations/llm-agent-frameworks/dspy/GEPA-Hands-On-Reranker.ipynb
7. Building and Optimizing Multi-Agent RAG Systems with DSPy and GEPA — Isaac Kargar, Medium — https://kargarisaac.medium.com/building-and-optimizing-multi-agent-rag-systems-with-dspy-and-gepa-2b88b5838ce2
8. Experiences from Building Enterprise Agents with DSPy and GEPA — slavozard, Bear Blog — https://slavozard.bearblog.dev/experiences-from-building-enterprise-agents-with-dspy-and-gepa/
9. Learning DSPy 3: Working with Optimizers — The Data Quarry — https://thedataquarry.com/blog/learning-dspy-3-working-with-optimizers
10. DSPy GEPA Reranker — Weaviate (YouTube) — https://www.youtube.com/watch?v=H4o7h6ZbA4o
11. Reflective Optimization of Agents with GEPA and DSPy — Matei Zaharia, Berkeley RDI (YouTube) — https://www.youtube.com/watch?v=rrtxyZ4Vnv8
12. DSPy 3 + GEPA: The Most Advanced RAG Framework Yet — Gao Dalie (YouTube) — https://www.youtube.com/watch?v=RyQNqzuAVcs
13. DSPy Tools Documentation — https://dspy.ai/learn/programming/tools/

This article was written by Hermes (glm-5-turbo | zai), based on content from Matei Zaharia — Reflective Optimization of Agents with GEPA and DSPy, DSPy 3 + GEPA by Gao Dalie, and GEPA Hands-On Reranker by Weaviate.