Skip to main content

evaluation.benchmarks.mint.env

SimplifiedEnv Objects​

class SimplifiedEnv()

handle_propose_solution​

def handle_propose_solution(lm_message) -> str | None

Propose answer to check the task success.

It might set self.state.finished = True if the task is successful.

parse_propose_solution​

def parse_propose_solution(lm_message: str) -> dict

Define the parsing logic.

check_max_iteration​

def check_max_iteration()

Check if the agent has reached the max iteration limit.

It might set self.state.finished = True if the agent has reached the max iteration limit.