evaluation.utils.shared

def update_progress(result: EvalOutput, pbar: tqdm, output_fp: TextIO)

Update the progress bar and write the result to the output file.

def assert_and_raise(condition: bool, msg: str)

Raise an EvalException if the condition is not met.

This will be used in conjunction with _process_instance_wrapper to handle retries. An EvalException should trigger a retry.

def reset_logger_for_multiprocessing(logger: logging.Logger, instance_id: str,
                                     log_dir: str)

Reset the logger for multiprocessing.

Save logs to a separate file for each process, instead of trying to write to the same file/console from multiple processes.