Skip to main content

evaluation.regression.conftest

agents​

def agents()

Retrieves a list of available agents.

Returns:

A list of agent names.

test_cases_dir​

@pytest.fixture(scope='session')
def test_cases_dir()

Fixture that provides the directory path for test cases.

Returns:

The directory path for test cases.

task_file​

@pytest.fixture
def task_file(test_cases_dir, request)

Fixture that provides the path to the task file for a test case.

Arguments:

  • test_cases_dir - The directory path for test cases.
  • request - The pytest request object.

Returns:

The path to the task file for the test case.

workspace_dir​

@pytest.fixture
def workspace_dir(test_cases_dir, request)

Fixture that provides the workspace directory for a test case.

Arguments:

  • test_cases_dir - The directory path for test cases.
  • request - The pytest request object.

Returns:

The workspace directory for the test case.

model​

@pytest.fixture
def model(request)

Fixture that provides the model name.

Arguments:

  • request - The pytest request object.

Returns:

The model name, defaulting to "gpt-3.5-turbo".

run_test_case​

@pytest.fixture
def run_test_case(test_cases_dir, workspace_dir, request)

Fixture that provides a function to run a test case.

Arguments:

  • test_cases_dir - The directory path for test cases.
  • workspace_dir - The workspace directory for the test case.
  • request - The pytest request object.

Returns:

A function that runs a test case for a given agent and case.

pytest_configure​

def pytest_configure(config)

Configuration hook for pytest.

Arguments:

  • config - The pytest configuration object.