🏗️ The Three-Stage Architecture: DEMONSTRATE-SEARCH-PREDICT
At the heart of DSPy lies the three-stage architecture that originated from the Demonstrate-Search-Predict research paper. This architecture provides a systematic way to structure complex reasoning tasks.
DEMONSTRATE
Learn from examples and build task understanding
# Define what the task does
class TaskSignature(dspy.Signature):
input_field: str = dspy.InputField()
output_field: str = dspy.OutputField()
# Examples provide demonstrations
trainset = [
dspy.Example(input_field="Example 1", output_field="Output 1"),
dspy.Example(input_field="Example 2", output_field="Output 2"),
]SEARCH
Retrieve and synthesize information from multiple sources
class SearchModule(dspy.Module):
def __init__(self):
self.retrieve = dspy.Retrieve(k=5)
self.select = dspy.Predict("documents, query -> relevant_docs")
def forward(self, query):
docs = self.retrieve(query).passages
return self.select(documents=docs, query=query)PREDICT
Generate final outputs based on gathered evidence
class PredictModule(dspy.Module):
def __init__(self):
self.generate = dspy.ChainOfThought("context, query -> answer")
def forward(self, context, query):
result = self.generate(context=context, query=query)
return result.answerBenefits of Three-Stage Architecture
- Composability: Each stage can be optimized independently
- Transparency: Clear separation of concerns
- Flexibility: Different strategies can be swapped
- Debugging: Issues can be isolated to specific stages
🔄 Key Differences
Imperative vs. Declarative
Prompting (Imperative)
# You tell the model HOW to do it
prompt = """
First, read the context carefully.
Then, identify the key information.
Next, formulate an answer.
Finally, provide your response in one sentence.
Context: {context}
Question: {question}
"""DSPy (Declarative)
# You tell the model WHAT to do
class AnswerQuestion(dspy.Signature):
"""Answer questions based on context."""
context: str = dspy.InputField()
question: str = dspy.InputField()
answer: str = dspy.OutputField(desc="concise answer")DSPy figures out the HOW!
Manual vs. Automatic
Prompting: Manual optimization
# Try different prompts manually
prompts = [
"Answer: {question}",
"Provide a clear answer to: {question}",
"Question: {question}\nAnswer:",
]
for prompt in prompts:
result = test(prompt) # Manual testingDSPy: Automatic optimization
# Define your program
program = dspy.ChainOfThought(AnswerQuestion)
# Optimize automatically
optimizer = BootstrapFewShot(metric=accuracy)
optimized_program = optimizer.compile(program, trainset=data)Static vs. Composable
Prompting: Static, monolithic
# One big prompt for the entire task
mega_prompt = """
Step 1: Extract entities from the text
Step 2: Classify each entity
Step 3: Summarize the entities
Step 4: Generate final output
Text: {text}
"""DSPy: Modular, composable
# Separate, reusable components
class Pipeline(dspy.Module):
def __init__(self):
self.extract = dspy.Predict("text -> entities")
self.classify = dspy.Predict("entities -> categories")
self.summarize = dspy.Predict("categories -> summary")
def forward(self, text):
entities = self.extract(text=text).entities
categories = self.classify(entities=entities).categories
return self.summarize(categories=categories).summary✅ Benefits of the Programming Paradigm
Modularity
Break complex tasks into simple components:
# Each component is independent and testable
extract = dspy.Predict("text -> entities")
classify = dspy.Predict("entities -> categories")
generate = dspy.Predict("categories -> summary")Reusability
Create once, use everywhere:
# Define a reusable QA signature
class QA(dspy.Signature):
context: str = dspy.InputField()
question: str = dspy.InputField()
answer: str = dspy.OutputField()
# Use in different contexts
basic_qa = dspy.Predict(QA)
reasoning_qa = dspy.ChainOfThought(QA)Testability
Test components independently:
def test_entity_extraction():
extractor = dspy.Predict("text -> entities")
result = extractor(text="Apple released iPhone in 2007")
assert "Apple" in result.entities
assert "iPhone" in result.entitiesAutomatic Optimization
Improve systematically:
def accuracy_metric(example, prediction):
return prediction.answer == example.answer
optimizer = BootstrapFewShot(metric=accuracy_metric)
optimized = optimizer.compile(MyProgram(), trainset=data)Maintainability
Changes are localized and manageable:
# Change one signature
class ImprovedQA(dspy.Signature):
"""Better QA with sources."""
context: str = dspy.InputField()
question: str = dspy.InputField()
answer: str = dspy.OutputField()
sources: list[str] = dspy.OutputField() # Added field
# All modules using this signature adapt automatically📊 Paradigm Comparison
| Aspect | Prompting | Programming (DSPy) |
|---|---|---|
| Approach | Imperative ("how") | Declarative ("what") |
| Optimization | Manual trial & error | Automatic from data |
| Composition | Difficult | Natural |
| Maintainability | Poor for complex | Good |
| Scalability | Struggles | Excels |
| Best for | Simple, one-off tasks | Complex, evolving systems |
💡 Analogy: Assembly vs. High-Level Languages
The prompting → programming shift is like assembly → high-level languages:
Assembly (Manual Prompting)
; Direct, detailed control
MOV AX, 5
ADD AX, 3
MOV result, AX- Maximum control
- Tedious for complex tasks
- Hard to maintain
High-Level Language (DSPy)
# Abstract, declarative
result = 5 + 3- Easier to write and understand
- Better for complex systems
- Compiler handles optimization
Similarly, DSPy abstracts away prompt engineering!