Skip to content

dspy.BootstrapFewShot

dspy.BootstrapFewShot(metric=None, metric_threshold=None, teacher_settings: Optional[Dict] = None, max_bootstrapped_demos=4, max_labeled_demos=16, max_rounds=1, max_errors=5)

Bases: Teleprompter

A Teleprompter class that composes a set of demos/examples to go into a predictor's prompt. These demos come from a combination of labeled examples in the training set, and bootstrapped demos.

Parameters:

Name Type Description Default
metric

Callable A function that compares an expected value and predicted value, outputting the result of that comparison.

None
metric_threshold

optional float, default None If the metric yields a numerical value, then check it against this threshold when deciding whether or not to accept a bootstrap example.

 None
teacher_settings Optional[Dict]

dict, optional Settings for the teacher model.

 None
max_bootstrapped_demos

int, default 4 Maximum number of bootstrapped demonstrations to include

 4
max_labeled_demos

int, default 16 Maximum number of labeled demonstrations to include.

 16
max_rounds

int, default 1 Number of iterations to attempt generating the required bootstrap examples. If unsuccessful after max_rounds, the program ends.

 1
max_errors

int, default 5 Maximum number of errors until program ends.

 5
Source code in dspy/teleprompt/bootstrap.py 
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
def __init__(
    self,
    metric=None,
    metric_threshold=None,
    teacher_settings: Optional[Dict]=None,
    max_bootstrapped_demos=4,
    max_labeled_demos=16,
    max_rounds=1,
    max_errors=5,
):
    """
    A Teleprompter class that composes a set of demos/examples to go into a predictor's prompt.
    These demos come from a combination of labeled examples in the training set, and bootstrapped demos.

    Args:
        metric: Callable
            A function that compares an expected value and predicted value, outputting the result of that comparison. 
        metric_threshold: optional float, default `None`
            If the metric yields a numerical value, then check it against this threshold when
            deciding whether or not to accept a bootstrap example.
        teacher_settings: dict, optional
            Settings for the `teacher` model.
        max_bootstrapped_demos: int, default 4
            Maximum number of bootstrapped demonstrations to include
        max_labeled_demos: int, default 16
            Maximum number of labeled demonstrations to include.
        max_rounds: int, default 1
            Number of iterations to attempt generating the required bootstrap examples. If unsuccessful after `max_rounds`, the program ends.
        max_errors: int, default 5
            Maximum number of errors until program ends.
    """
    self.metric = metric
    self.metric_threshold = metric_threshold
    self.teacher_settings = {} if teacher_settings is None else teacher_settings

    self.max_bootstrapped_demos = max_bootstrapped_demos
    self.max_labeled_demos = max_labeled_demos
    self.max_rounds = max_rounds
    self.max_errors = max_errors
    self.error_count = 0
    self.error_lock = threading.Lock()

Functions

compile(student, *, teacher=None, trainset)

Source code in dspy/teleprompt/bootstrap.py 
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
def compile(self, student, *, teacher=None, trainset):
    self.trainset = trainset

    self._prepare_student_and_teacher(student, teacher)
    self._prepare_predictor_mappings()
    self._bootstrap()

    self.student = self._train()
    self.student._compiled = True

    # set assert_failures and suggest_failures as attributes of student w/ value 0
    self.student._assert_failures = 0
    self.student._suggest_failures = 0

    return self.student