Skip to content

dspy.BootstrapFewShot

dspy.BootstrapFewShot(metric=None, metric_threshold=None, teacher_settings: Optional[Dict] = None, max_bootstrapped_demos=4, max_labeled_demos=16, max_rounds=1, max_errors=5)

Bases: Teleprompter

A Teleprompter class that composes a set of demos/examples to go into a predictor's prompt. These demos come from a combination of labeled examples in the training set, and bootstrapped demos.

Parameters:

Name Type Description Default
metric Callable

A function that compares an expected value and predicted value, outputting the result of that comparison.

 None
metric_threshold float

If the metric yields a numerical value, then check it against this threshold when deciding whether or not to accept a bootstrap example. Defaults to None.

 None
teacher_settings dict

Settings for the teacher model. Defaults to None.

 None
max_bootstrapped_demos int

Maximum number of bootstrapped demonstrations to include. Defaults to 4.

 4
max_labeled_demos int

Maximum number of labeled demonstrations to include. Defaults to 16.

 16
max_rounds int

Number of iterations to attempt generating the required bootstrap examples. If unsuccessful after max_rounds, the program ends. Defaults to 1.

 1
max_errors int

Maximum number of errors until program ends. Defaults to 5.

 5
Source code in dspy/teleprompt/bootstrap.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
def __init__(
    self,
    metric=None,
    metric_threshold=None,
    teacher_settings: Optional[Dict] = None,
    max_bootstrapped_demos=4,
    max_labeled_demos=16,
    max_rounds=1,
    max_errors=5,
):
    """A Teleprompter class that composes a set of demos/examples to go into a predictor's prompt.
    These demos come from a combination of labeled examples in the training set, and bootstrapped demos.

    Args:
        metric (Callable): A function that compares an expected value and predicted value,
            outputting the result of that comparison.
        metric_threshold (float, optional): If the metric yields a numerical value, then check it
            against this threshold when deciding whether or not to accept a bootstrap example.
            Defaults to None.
        teacher_settings (dict, optional): Settings for the `teacher` model.
            Defaults to None.
        max_bootstrapped_demos (int): Maximum number of bootstrapped demonstrations to include.
            Defaults to 4.
        max_labeled_demos (int): Maximum number of labeled demonstrations to include.
            Defaults to 16.
        max_rounds (int): Number of iterations to attempt generating the required bootstrap
            examples. If unsuccessful after `max_rounds`, the program ends. Defaults to 1.
        max_errors (int): Maximum number of errors until program ends. Defaults to 5.
    """
    self.metric = metric
    self.metric_threshold = metric_threshold
    self.teacher_settings = {} if teacher_settings is None else teacher_settings

    self.max_bootstrapped_demos = max_bootstrapped_demos
    self.max_labeled_demos = max_labeled_demos
    self.max_rounds = max_rounds
    self.max_errors = max_errors
    self.error_count = 0
    self.error_lock = threading.Lock()

Functions

compile(student, *, teacher=None, trainset)

Source code in dspy/teleprompt/bootstrap.py 
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def compile(self, student, *, teacher=None, trainset):
    self.trainset = trainset

    self._prepare_student_and_teacher(student, teacher)
    self._prepare_predictor_mappings()
    self._bootstrap()

    self.student = self._train()
    self.student._compiled = True

    # set assert_failures and suggest_failures as attributes of student w/ value 0
    self.student._assert_failures = 0
    self.student._suggest_failures = 0

    return self.student