Skip to content

dspy.evaluate.SemanticF1

dspy.evaluate.SemanticF1(threshold=0.66, decompositional=False)

Bases: Module

Source code in dspy/evaluate/auto_evaluation.py 
39
40
41
42
43
44
45
def __init__(self, threshold=0.66, decompositional=False):
    self.threshold = threshold

    if decompositional:
        self.module = dspy.ChainOfThought(DecompositionalSemanticRecallPrecision)
    else:
        self.module = dspy.ChainOfThought(SemanticRecallPrecision)

Functions

forward(example, pred, trace=None)

Source code in dspy/evaluate/auto_evaluation.py 
47
48
49
50
51
def forward(self, example, pred, trace=None):
    scores = self.module(question=example.question, ground_truth=example.response, system_response=pred.response)
    score = f1_score(scores.precision, scores.recall)

    return score if trace is None else score >= self.threshold