跳到主要内容

evaluation.mint.tasks.reasoning

ReasoningTask Objects

class ReasoningTask(Task)

extract_answer

def extract_answer(solution: str) -> str | None

Extract the answer from the given solution.

compare_w_digits

def compare_w_digits(reference: str, answer: str) -> bool

Compare the reference and answer with digits.

MultipleChoiceTask Objects

class MultipleChoiceTask(Task)

Subclass of Task for multiple choice tasks.

TheoremqaTask Objects

class TheoremqaTask(Task)

extract_answer

def extract_answer(solution: str) -> Any

Extract the answer from the given solution.

success

def success(solution: str) -> bool

This checks whether the given solution can complete the current task.