Testing

Ash uses pytest with async support for testing.

Running Tests

Run all tests:

uv run pytest

Run with verbose output:

uv run pytest -v

Run specific test file:

uv run pytest tests/unit/test_config.py

Run specific test:

uv run pytest tests/unit/test_config.py::test_load_config

Test Organization

tests/
├── conftest.py       # Shared fixtures
├── unit/             # Unit tests
│   ├── test_config.py
│   ├── test_llm.py
│   ├── test_memory.py
│   └── test_tools.py
└── integration/      # Integration tests
    └── test_agent.py

Writing Tests

Async Tests

Use pytest-asyncio:

import pytest

@pytest.mark.asyncio
async def test_async_function():
    result = await some_async_function()
    assert result == expected

Fixtures

Common fixtures in conftest.py:

import pytest
from ash.config import AshConfig

@pytest.fixture
def config():
    return AshConfig(
        models={"default": {...}},
    )

@pytest.fixture
async def memory_store(config):
    store = MemoryStore(config)
    await store.connect()
    yield store
    await store.close()

Mocking

Mock external services:

from unittest.mock import AsyncMock, patch

@pytest.mark.asyncio
async def test_with_mock():
    with patch("ash.llm.anthropic.AsyncAnthropic") as mock:
        mock.return_value.messages.create = AsyncMock(
            return_value=mock_response
        )
        result = await provider.complete(messages)
        assert result.content == expected

Coverage

Run with coverage:

uv run pytest --cov=ash --cov-report=html

View report at htmlcov/index.html.

Test Categories

Unit Tests

Test individual components in isolation:

def test_config_validation():
    config = AshConfig(models={...})
    assert config.default_model.provider == "anthropic"

Integration Tests

Test component interactions:

@pytest.mark.asyncio
async def test_agent_tool_execution():
    agent = Agent(config, tools)
    response = await agent.process("run ls")
    assert "file.txt" in response

CI Integration

Tests run on every PR via GitHub Actions:

- run: uv run pytest --cov-report=xml
- uses: codecov/codecov-action@v4