Joint search over instructions and example sets.
- Example sets: per predictor,
num_candidateschoices: no demonstrations, then bootstrapped sets from the teacher role. - Instructions: per predictor, the original instruction plus
num_candidates - 1proposals from the proposal role, grounded in the predictor's fields, training examples, and a bootstrapped demonstration. - Search: a categorical TPE over
<name>.instructionand<name>.demos. Each trial scores a training minibatch; minibatch observations guide the sampler but never become the returned program. Everyminibatch_full_eval_stepstrials, and at the end, the best unpromoted choice by mean minibatch score gets a full validation pass. Only full, complete, eligible validation results can become the incumbent.
The baseline (original instructions, no demonstrations) is evaluated on the full validation set first. Teacher, proposal, and target calls share one ceiling.
Options: :num_candidates (default 4), :num_trials (default 12),
:minibatch_size (default 4), :minibatch_full_eval_steps (default 4),
:n_startup (default 4), :evidence_examples (default 3), and
BootstrapFewShot options (:accept_score required).
Differences from DSPy: minibatches come from training data; there is no automatic configuration preset, dataset summarizer, or program-code summary; proposals are grounded in the signature and examples.