Skip to content

dspy.evaluate.CompleteAndGrounded

dspy.evaluate.CompleteAndGrounded(threshold=0.66)

Bases: Module

Source code in dspy/evaluate/auto_evaluation.py 
90
91
92
93
def __init__(self, threshold=0.66):
    self.threshold = threshold
    self.completeness_module = dspy.ChainOfThought(AnswerCompleteness)
    self.groundedness_module = dspy.ChainOfThought(AnswerGroundedness)

Functions

forward(example, pred, trace=None)

Source code in dspy/evaluate/auto_evaluation.py 
 95
 96
 97
 98
 99
100
def forward(self, example, pred, trace=None):
    completeness = self.completeness_module(question=example.question, ground_truth=example.response, system_response=pred.response)
    groundedness = self.groundedness_module(question=example.question, retrieved_context=pred.context, system_response=pred.response)
    score = f1_score(groundedness.groundedness, completeness.completeness)

    return score if trace is None else score >= self.threshold