Skip to main content

evaluation.utils.shared

update_progress

def update_progress(result: EvalOutput, pbar: tqdm, output_fp: TextIO)

Update the progress bar and write the result to the output file.

assert_and_raise

def assert_and_raise(condition: bool, msg: str)

Raise an EvalException if the condition is not met.

This will be used in conjunction with _process_instance_wrapper to handle retries. An EvalException should trigger a retry.

reset_logger_for_multiprocessing

def reset_logger_for_multiprocessing(logger: logging.Logger, instance_id: str,
log_dir: str)

Reset the logger for multiprocessing.

Save logs to a separate file for each process, instead of trying to write to the same file/console from multiple processes.