mirror of
https://github.com/Routstr/routstr-core.git
synced 2026-08-10 19:16:31 +00:00
Jeroen UbbinkandClaude Fable 5
eaf74edbba
Cached prompt tokens were billed at the full input rate whenever a vendor's usage dialect or cache pricing was unknown, overcharging DeepSeek topups ~5-10x on agentic workloads (hits are 10x cheaper upstream) and silently mispricing OpenAI cached reads and Anthropic cache writes the same way. Two root causes, two fixes: - Usage dialects: DeepSeek reports prompt_cache_hit_tokens / prompt_cache_miss_tokens, which billing never parsed. Usage normalization now lives in payment/usage.py as a union parser over the known, non-colliding dialects (OpenAI prompt_tokens_details, Anthropic additive cache fields, DeepSeek hit/miss), producing one canonical NormalizedUsage. Providers expose it as an overridable BaseUpstreamProvider.normalize_usage hook — the escape hatch for future vendors whose fields genuinely conflict — and every settlement call site passes the provider's result through, so calculate_cost holds no vendor knowledge of its own. - Cache rates: the OpenRouter model feed omits input_cache_read/-write for most DeepSeek models (and e.g. openai/gpt-4o), so billing fell back to the full input rate. Missing rates are now backfilled from litellm's bundled cost map before the provider fee is applied; the input-rate fallback remains only as the documented last resort. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FastAPI Async Unit Tests
This directory contains async unit tests for the Routstr proxy FastAPI application.
Installation
First, ensure you have the development dependencies installed:
uv pip install -e ".[dev]"
Running Tests
To run all tests:
pytest
To run tests with coverage:
pytest --cov=routstr --cov-report=html
To run specific test files:
pytest tests/test_main.py
pytest tests/test_models.py
pytest tests/test_proxy.py
To run only async tests:
pytest -m asyncio
Test Structure
conftest.py- Pytest fixtures and configurationtest_main.py- Tests for main app endpointstest_account.py- Tests for wallet/account management endpointstest_proxy.py- Tests for the proxy functionality with mocked upstreamtest_models.py- Tests for model pricing and data structures
Key Fixtures
async_client- Async HTTP client for testing FastAPI endpointstest_session- In-memory SQLite database session for teststest_api_key- Pre-configured API key with balanceapi_key_with_balance- API key with sufficient balance for proxy tests
Environment Variables
The tests automatically set up required environment variables in conftest.py. No manual configuration needed.
Writing New Tests
- Use
@pytest.mark.asynciofor async tests - Use the provided fixtures for database and client access
- Mock external dependencies (like upstream API calls)
- Test both success and error cases
- Verify database state changes when applicable