Extreme Few-Shot Learning

The "10 Examples" Challenge

In many real-world scenarios, you don't have thousands of labeled examples. You might have ten. Extreme few-shot learning focuses on squeezing every drop of signal from this tiny dataset.

Core Strategies

• Prompt Optimization: Tuning the prompt instruction is often more effective than fine-tuning weights on small data.
• Data Amplification: Generating synthetic variations (paraphrases, counterfactuals) to expand the training set.
• Bootstrap Few-Shot: Using the model itself to generate demonstrations, filtering for high-confidence outputs.

Implementation Example

Here's how you might set up a trainer for this scenario:

class ExtremeFewShotTrainer:
    def train_with_10_examples(self, signature, examples):
        # 1. Analyze input/output patterns in the 10 examples
        analysis = self._analyze_examples(examples)
        
        # 2. Optimize prompt using those insights
        final_program = self._prompt_optimization_training(signature, analysis)
        
        # 3. Compile with BootstrapFewShot (using limited demos)
        optimizer = dspy.BootstrapFewShot(metric=my_metric, max_bootstrapped_demos=3)
        return optimizer.compile(final_program, trainset=examples)

Best Practices

When data is this scarce, quality control is paramount. Ensure your 10 examples are diverse, perfectly accurate, and cover edge cases. Rely heavily on Chain of Thought to guide the model's reasoning, as it can compensate for the lack of training data.