Debugging & Tracing

Introduction

Debugging AI applications is tricky due to non-deterministic outputs. This section covers how to inspect module state, trace execution flow, and monitor performance.

Common Debugging Challenges

• Non-deterministic Outputs: The same input might yield different results.
• Hidden Complexity: Logic within optimizers and LMs is often opaque.
• Token Usage: Costs can spiral without careful monitoring.

Building a DSPy Debugger

A custom debugger class can help log and inspect execution:

class DSPyDebugger:
    def trace(self, module, args, result):
        log_entry = {
            "module": module,
            "args": args,
            "result": result
        }
        self.history.append(log_entry)
        self._print(log_entry)

Function Tracing with Decorators

Automatically trace forward passes in your modules:

def trace_function(func):
    def wrapper(*args, **kwargs):
        print(f"Executing {func.__name__}...")
        result = func(*args, **kwargs)
        print(f"Result: {result}")
        return result
    return wrapper

class TracedModule(dspy.Module):
    @trace_function
    def forward(self, x):
        return self.predict(input=x)

Token Usage & Cost Tracking

Monitor costs by tracking token usage:

import tiktoken

class TokenTracker:
    def count_tokens(self, text):
        return len(self.encoding.encode(text))
        
    def estimate_cost(self):
        # Calculate based on current pricing...
        pass