Skip to main content

evaluation.integration_tests.tests.base

BaseIntegrationTest Objects​

class BaseIntegrationTest(ABC)

Base class for integration tests.

initialize_runtime​

@classmethod
@abstractmethod
def initialize_runtime(cls, runtime: Runtime) -> None

Initialize the runtime for the test to run.

verify_result​

@classmethod
@abstractmethod
def verify_result(cls, runtime: Runtime, histories: list[Event]) -> TestResult

Verify the result of the test.

This method will be called after the agent performs the task on the runtime.