Skip to main content

evaluation.mint.tasks.reasoning

ReasoningTask Objects​

class ReasoningTask(Task)

extract_answer​

def extract_answer(solution: str) -> str | None

Extract the answer from the given solution.

compare_w_digits​

def compare_w_digits(reference: str, answer: str) -> bool

Compare the reference and answer with digits.

MultipleChoiceTask Objects​

class MultipleChoiceTask(Task)

Subclass of Task for multiple choice tasks.

TheoremqaTask Objects​

class TheoremqaTask(Task)

extract_answer​

def extract_answer(solution: str) -> Any

Extract the answer from the given solution.

success​

def success(solution: str) -> bool

This checks whether the given solution can complete the current task.