feat: Add comprehensive test suite analysis report

Co-authored-by: db2002dominic <db2002dominic@gmail.com>

TEST_SUITE_ANALYSIS_REPORT.md

@@ -0,0 +1,804 @@
+# Comprehensive Test Suite Analysis Report
+
+**Date:** 2024  
+**Target Audience:** Senior Pytest Engineer  
+**Purpose:** Deep analysis of test suite effectiveness, outdated tests, and missing coverage
+
+---
+
+## Executive Summary
+
+This report provides a comprehensive analysis of the routstr test suite, evaluating:
+- **Test Relevance**: Whether tests match current codebase implementation
+- **Test Effectiveness**: Whether tests provide meaningful coverage
+- **Missing Coverage**: Critical areas lacking tests
+- **Outdated Tests**: Tests that no longer reflect codebase behavior
+
+**Key Findings:**
+- ✅ Strong integration test coverage for wallet operations
+- ⚠️ Several unit tests may be testing implementation details rather than behavior
+- ❌ Missing tests for critical upstream provider functionality
+- ❌ Missing tests for cost calculation edge cases
+- ⚠️ Some tests have become outdated due to codebase refactoring
+
+---
+
+## 1. Unit Tests Analysis
+
+### 1.1 `test_algorithm.py` ✅ **GOOD**
+
+**Status:** Well-maintained and relevant
+
+**Coverage:**
+- `calculate_model_cost_score()` - ✅ Comprehensive
+- `get_provider_penalty()` - ✅ Covers OpenRouter penalty logic
+- `should_prefer_model()` - ✅ Tests alias matching, cost comparison, provider penalties
+
+**Assessment:** These tests correctly validate the model prioritization algorithm. The tests are focused on behavior, not implementation details.
+
+**Recommendations:**
+- ✅ Keep as-is
+- Consider adding test for `create_model_mappings()` function (currently untested at unit level)
+
+---
+
+### 1.2 `test_fee_consistency.py` ⚠️ **PARTIALLY OUTDATED**
+
+**Status:** Tests implementation details that may have changed
+
+**Issues:**
+1. **Tests `_model_to_row_payload()` directly** - This is a private function. Tests should focus on public API behavior.
+2. **Tests fee application logic** - The comment says "Fees are now applied per-provider when reading from the database" but tests don't verify this behavior.
+3. **Missing tests for fee application** - No tests verify that fees are correctly applied when reading models from database.
+
+**Current Test Focus:**
+- ✅ Pricing stored without fees (correct)
+- ✅ Payload structure validation (implementation detail)
+- ✅ Original model not mutated (implementation detail)
+
+**Missing:**
+- ❌ Tests for `_row_to_model()` with `apply_provider_fee=True`
+- ❌ Tests verifying provider fees are applied correctly
+- ❌ Tests for fee calculation edge cases
+
+**Recommendations:**
+- Keep existing tests but add integration tests for fee application
+- Consider testing through public API (`get_model_by_id()`, `list_models()`) rather than private functions
+
+---
+
+### 1.3 `test_image_tokens.py` ✅ **EXCELLENT**
+
+**Status:** Comprehensive and well-written
+
+**Coverage:**
+- ✅ Low detail image tokens (always 85)
+- ✅ High detail token calculation
+- ✅ Auto detail behavior
+- ✅ Image dimension extraction
+- ✅ Multiple images
+- ✅ Invalid image handling
+- ✅ Base64 encoding
+- ✅ `input_image` type support
+
+**Assessment:** These tests thoroughly cover image token estimation logic.
+
+**Recommendations:**
+- ✅ Keep as-is
+- Consider adding test for very large images (edge case)
+
+---
+
+### 1.4 `test_payment_helpers.py` ⚠️ **NEEDS IMPROVEMENT**
+
+**Status:** Tests exist but coverage is incomplete
+
+**Current Coverage:**
+- ✅ `get_max_cost_for_model()` - Basic cases covered
+- ✅ Tolerance percentage handling
+- ✅ Fixed pricing mode
+
+**Missing Critical Tests:**
+- ❌ `calculate_discounted_max_cost()` - **NOT TESTED** (critical function)
+- ❌ `check_token_balance()` - **NOT TESTED** (used in proxy)
+- ❌ `estimate_tokens()` - **NOT TESTED** (used for pre-flight checks)
+- ❌ `create_error_response()` - **NOT TESTED** (error formatting)
+
+**Issues:**
+- Tests mock database heavily, making them brittle
+- No tests for error paths in `get_max_cost_for_model()`
+- Missing tests for provider fee application in max cost calculation
+
+**Recommendations:**
+- Add tests for all missing helper functions
+- Add edge case tests (zero pricing, negative values, etc.)
+- Consider integration tests for `get_max_cost_for_model()` with real database
+
+---
+
+### 1.5 `test_settings.py` ✅ **GOOD**
+
+**Status:** Appropriate coverage for settings service
+
+**Coverage:**
+- ✅ Settings initialization from environment
+- ✅ Database precedence over environment
+- ✅ Settings persistence
+
+**Assessment:** Tests cover the core settings functionality adequately.
+
+**Recommendations:**
+- ✅ Keep as-is
+- Consider adding test for settings validation
+
+---
+
+### 1.6 `test_wallet.py` ⚠️ **NEEDS REVIEW**
+
+**Status:** Tests exist but may not reflect current implementation
+
+**Current Coverage:**
+- ✅ `get_balance()` - Basic test
+- ✅ `recieve_token()` - Valid token handling
+- ✅ `send_token()` - Basic test
+- ✅ `credit_balance()` - Tests atomic update
+
+**Potential Issues:**
+1. **`test_recieve_token_untrusted_mint()`** - Tests `swap_to_primary_mint()` but this may not be the current behavior
+2. **Heavy mocking** - Tests mock `Wallet.with_db()` which may not reflect real behavior
+3. **Missing tests** - No tests for:
+   - `swap_to_primary_mint()` directly
+   - `get_wallet()` function
+   - `fetch_all_balances()`
+   - `periodic_payout()` (only integration test exists)
+   - `send_to_lnurl()` (only integration test exists)
+
+**Recommendations:**
+- Review if `swap_to_primary_mint()` is still used
+- Add integration tests for wallet functions instead of heavy mocking
+- Add tests for missing functions
+
+---
+
+## 2. Integration Tests Analysis
+
+### 2.1 `test_background_tasks.py` ⚠️ **PARTIALLY OUTDATED**
+
+**Status:** Some tests are skipped or incomplete
+
+**Issues:**
+1. **`test_periodic_payout()`** - Skipped with reason "Database setup issues"
+   - Comment says "periodic_payout is currently not implemented (just logs warning)"
+   - **Action Required:** Either implement the function or remove the test
+
+2. **`test_calculates_payouts_accurately()`** - Skipped
+   - **Action Required:** Fix or remove
+
+3. **`TestTaskInteractions`** - Many tests skipped
+   - Tests for task interactions are marked as "Complex timing and concurrency tests"
+   - **Action Required:** Either implement proper async test patterns or document why these are skipped
+
+**Good Tests:**
+- ✅ `test_updates_model_prices_periodically()` - Tests pricing update logic
+- ✅ `test_handles_provider_api_failures()` - Tests error handling
+- ✅ `test_processes_pending_refunds()` - Tests refund processing
+
+**Recommendations:**
+- Remove or fix skipped tests
+- Implement `periodic_payout()` or document why it's not implemented
+- Add tests for `refresh_model_maps_periodically()` (currently untested)
+
+---
+
+### 2.2 `test_database_consistency.py` ✅ **EXCELLENT**
+
+**Status:** Comprehensive database consistency tests
+
+**Coverage:**
+- ✅ Transaction atomicity
+- ✅ Concurrent operations
+- ✅ Data integrity constraints
+- ✅ Performance characteristics
+
+**Assessment:** These tests provide excellent coverage for database operations.
+
+**Recommendations:**
+- ✅ Keep as-is
+- Consider adding test for reserved_balance consistency (see `test_reserved_balance_negative.py`)
+
+---
+
+### 2.3 `test_error_handling_edge_cases.py` ✅ **GOOD**
+
+**Status:** Comprehensive error handling tests
+
+**Coverage:**
+- ✅ Network failure scenarios
+- ✅ Invalid input handling
+- ✅ Resource exhaustion
+- ✅ Recovery scenarios
+- ✅ Edge case combinations
+
+**Assessment:** Good coverage of error scenarios.
+
+**Recommendations:**
+- ✅ Keep as-is
+- Consider adding tests for upstream provider-specific errors
+
+---
+
+### 2.4 `test_general_info_endpoints.py` ✅ **GOOD**
+
+**Status:** Well-structured endpoint tests
+
+**Coverage:**
+- ✅ Root endpoint (`/v1/info`)
+- ✅ Models endpoint (`/v1/models`)
+- ✅ Performance requirements
+- ✅ Response structure validation
+
+**Assessment:** Good coverage of public info endpoints.
+
+**Recommendations:**
+- ✅ Keep as-is
+
+---
+
+### 2.5 `test_provider_management.py` ✅ **GOOD**
+
+**Status:** Comprehensive provider discovery tests
+
+**Coverage:**
+- ✅ Provider listing
+- ✅ Health check integration
+- ✅ Nostr relay failures
+- ✅ Malformed URLs
+- ✅ Performance requirements
+
+**Assessment:** Good coverage of provider discovery functionality.
+
+**Recommendations:**
+- ✅ Keep as-is
+- Consider adding tests for provider caching behavior
+
+---
+
+### 2.6 `test_proxy_get_endpoints.py` ⚠️ **NEEDS IMPROVEMENT**
+
+**Status:** Basic coverage but missing critical scenarios
+
+**Current Coverage:**
+- ✅ Valid API key requests
+- ✅ Header forwarding
+- ✅ Unauthorized access
+
+**Missing Critical Tests:**
+- ❌ **Cost calculation verification** - No tests verify that costs are calculated correctly for GET requests
+- ❌ **Balance deduction** - Tests check database changes but don't verify exact cost calculation
+- ❌ **Reserved balance handling** - No tests for reserved balance during GET requests
+- ❌ **Model-specific pricing** - No tests for different models having different costs
+- ❌ **Streaming GET responses** - No tests for streaming GET endpoints
+- ❌ **Error propagation** - Limited tests for upstream error handling
+
+**Recommendations:**
+- Add tests verifying exact cost calculation for GET requests
+- Add tests for reserved balance behavior
+- Add tests for model-specific pricing
+- Add tests for streaming responses
+
+---
+
+### 2.7 `test_proxy_post_endpoints.py` ⚠️ **NEEDS IMPROVEMENT**
+
+**Status:** Good coverage but missing critical scenarios
+
+**Current Coverage:**
+- ✅ JSON payload forwarding
+- ✅ Streaming responses
+- ✅ Non-streaming responses
+- ✅ Error handling
+
+**Missing Critical Tests:**
+- ❌ **Token usage calculation** - No tests verify that token counts are correctly extracted and used for cost calculation
+- ❌ **Partial streaming failures** - Tests exist but don't verify cost calculation for partial responses
+- ❌ **Reserved balance during streaming** - No tests for reserved balance behavior during long-running streams
+- ❌ **Cost calculation with image tokens** - No tests for requests with images
+- ❌ **Cost calculation with web_search** - No tests for web search enabled requests
+- ❌ **Cost calculation with internal_reasoning** - No tests for reasoning tokens
+- ❌ **Max cost enforcement** - No tests verify that max_cost limits are enforced
+- ❌ **Tolerance percentage** - No tests for tolerance percentage in cost calculation
+
+**Recommendations:**
+- Add comprehensive tests for cost calculation with various token types
+- Add tests for reserved balance during streaming
+- Add tests for max cost enforcement
+- Add tests for tolerance percentage
+
+---
+
+### 2.8 `test_wallet_authentication.py` ✅ **GOOD**
+
+**Status:** Comprehensive authentication tests
+
+**Coverage:**
+- ✅ API key generation
+- ✅ Invalid token handling
+- ✅ Duplicate token handling
+- ✅ Authorization header validation
+
+**Assessment:** Good coverage of authentication flow.
+
+**Recommendations:**
+- ✅ Keep as-is
+
+---
+
+### 2.9 `test_wallet_information.py` ✅ **GOOD**
+
+**Status:** Good coverage of wallet info endpoints
+
+**Coverage:**
+- ✅ Wallet endpoint
+- ✅ Wallet info endpoint
+- ✅ Unauthorized access
+- ✅ Zero balance handling
+- ✅ Expired key behavior
+
+**Assessment:** Good coverage.
+
+**Recommendations:**
+- ✅ Keep as-is
+
+---
+
+### 2.10 `test_wallet_refund.py` ⚠️ **PARTIALLY OUTDATED**
+
+**Status:** Some tests are skipped or incomplete
+
+**Issues:**
+1. **`test_refund_with_lightning_address()`** - Skipped with reason "Lightning address refund functionality not implemented"
+   - **Action Required:** Either implement or remove test
+
+2. **`test_partial_refund_not_supported()`** - Test name suggests partial refunds aren't supported, but comment says "The refund_balance function supports it, but the endpoint doesn't expose it"
+   - **Action Required:** Clarify if partial refunds should be supported
+
+**Good Tests:**
+- ✅ Full balance refund
+- ✅ Zero balance handling
+- ✅ Token format validation
+
+**Recommendations:**
+- Implement Lightning address refund or remove test
+- Add test for refund with reserved balance
+- Add test for concurrent refund requests
+
+---
+
+### 2.11 `test_wallet_topup.py` ✅ **GOOD**
+
+**Status:** Comprehensive topup tests
+
+**Coverage:**
+- ✅ Valid token topup
+- ✅ Multiple denominations
+- ✅ Invalid tokens
+- ✅ Spent tokens
+- ✅ Concurrent topups
+
+**Assessment:** Good coverage.
+
+**Recommendations:**
+- ✅ Keep as-is
+
+---
+
+### 2.12 `test_reserved_balance_negative.py` ⚠️ **REVEALS BUG**
+
+**Status:** Test reveals that reserved balance CAN go negative
+
+**Critical Finding:**
+```python
+# Current implementation allows negative reserved balance
+assert test_key.reserved_balance == -100
+assert test_key.total_requests == -1
+```
+
+**Issue:** The test documents that `revert_pay_for_request()` allows reserved balance to go negative, which is likely a bug.
+
+**Recommendations:**
+- **URGENT:** Fix `revert_pay_for_request()` to prevent negative reserved balance
+- Update test to verify reserved balance never goes negative
+- Add integration test for concurrent requests with reserved balance
+
+---
+
+### 2.13 `test_example.py` ✅ **GOOD**
+
+**Status:** Example/template tests
+
+**Coverage:**
+- ✅ Infrastructure setup
+- ✅ Full wallet flow
+- ✅ Error handling
+- ✅ Performance requirements
+
+**Assessment:** Good example tests.
+
+**Recommendations:**
+- ✅ Keep as-is
+
+---
+
+## 3. Missing Test Coverage
+
+### 3.1 Critical Missing Tests
+
+#### 3.1.1 Upstream Provider Tests ❌ **CRITICAL**
+
+**Missing:**
+- ❌ No unit tests for `BaseUpstreamProvider` methods
+- ❌ No tests for provider-specific implementations (OpenAI, Anthropic, etc.)
+- ❌ No tests for `forward_request()` method
+- ❌ No tests for `handle_streaming_chat_completion()`
+- ❌ No tests for `handle_non_streaming_chat_completion()`
+- ❌ No tests for `prepare_headers()`, `prepare_params()`, `prepare_request_body()`
+- ❌ No tests for `transform_model_name()`
+- ❌ No tests for `_apply_provider_fee_to_model()`
+- ❌ No tests for error mapping (`map_upstream_error_response()`)
+- ❌ No tests for `fetch_models()` for each provider type
+
+**Impact:** High - Upstream providers are core functionality
+
+**Recommendations:**
+- Create `tests/unit/test_upstream_base.py`
+- Create `tests/unit/test_upstream_openai.py`
+- Create `tests/unit/test_upstream_anthropic.py`
+- Create `tests/integration/test_upstream_providers.py`
+
+---
+
+#### 3.1.2 Cost Calculation Tests ❌ **CRITICAL**
+
+**Missing:**
+- ❌ No unit tests for `calculate_cost()` function
+- ❌ No tests for `CostData`, `MaxCostData`, `CostDataError` models
+- ❌ No tests for cost calculation with missing usage data
+- ❌ No tests for cost calculation with invalid model
+- ❌ No tests for cost calculation with zero tokens
+- ❌ No tests for cost calculation edge cases (very large token counts)
+- ❌ No tests for cost calculation with fixed pricing vs model-based pricing
+
+**Impact:** High - Cost calculation is critical for billing
+
+**Recommendations:**
+- Create `tests/unit/test_cost_calculation.py`
+- Add comprehensive tests for all cost calculation scenarios
+
+---
+
+#### 3.1.3 Payment Helpers Tests ❌ **CRITICAL**
+
+**Missing:**
+- ❌ `calculate_discounted_max_cost()` - **NOT TESTED**
+- ❌ `check_token_balance()` - **NOT TESTED**
+- ❌ `estimate_tokens()` - **NOT TESTED**
+- ❌ `create_error_response()` - **NOT TESTED**
+
+**Impact:** High - These functions are used in critical paths
+
+**Recommendations:**
+- Add tests for all missing helper functions
+- Add edge case tests
+
+---
+
+#### 3.1.4 Algorithm Integration Tests ❌ **HIGH PRIORITY**
+
+**Missing:**
+- ❌ No integration tests for `create_model_mappings()`
+- ❌ No tests for model override behavior
+- ❌ No tests for disabled model handling
+- ❌ No tests for provider fee application in mappings
+- ❌ No tests for alias resolution
+
+**Impact:** High - Model selection is core functionality
+
+**Recommendations:**
+- Create `tests/integration/test_model_mappings.py`
+- Test full model mapping creation with multiple providers
+- Test override behavior
+- Test disabled model filtering
+
+---
+
+#### 3.1.5 Price Update Tests ⚠️ **MEDIUM PRIORITY**
+
+**Missing:**
+- ❌ No unit tests for `_fetch_btc_usd_price()`
+- ❌ No tests for individual exchange APIs (`_kraken_btc_usd`, `_coinbase_btc_usd`, `_binance_btc_usdt`)
+- ❌ No tests for price update failure handling
+- ❌ No tests for `update_prices_periodically()` (only integration test exists)
+
+**Impact:** Medium - Price updates affect pricing but have fallbacks
+
+**Recommendations:**
+- Create `tests/unit/test_price.py`
+- Mock exchange APIs and test failure scenarios
+- Test price aggregation logic
+
+---
+
+#### 3.1.6 LNURL Tests ❌ **MEDIUM PRIORITY**
+
+**Missing:**
+- ❌ No tests for `parse_lightning_invoice_amount()`
+- ❌ No tests for `decode_lnurl()`
+- ❌ No tests for `get_lnurl_data()`
+- ❌ No tests for `get_lnurl_invoice()`
+- ❌ No tests for `raw_send_to_lnurl()`
+
+**Impact:** Medium - LNURL is used for refunds
+
+**Recommendations:**
+- Create `tests/unit/test_lnurl.py`
+- Mock LNURL endpoints
+- Test error scenarios
+
+---
+
+#### 3.1.7 NIP-91 Tests ❌ **LOW PRIORITY**
+
+**Missing:**
+- ❌ No tests for `create_nip91_event()`
+- ❌ No tests for `query_nip91_events()`
+- ❌ No tests for `discover_onion_url_from_tor()`
+- ❌ No tests for event parsing
+
+**Impact:** Low - NIP-91 is for provider discovery
+
+**Recommendations:**
+- Create `tests/unit/test_nip91.py` if NIP-91 becomes more critical
+
+---
+
+### 3.2 Edge Cases Missing Tests
+
+#### 3.2.1 Reserved Balance Edge Cases ⚠️ **HIGH PRIORITY**
+
+**Missing:**
+- ❌ Concurrent requests with reserved balance
+- ❌ Reserved balance with insufficient balance
+- ❌ Reserved balance rollback on errors
+- ❌ Reserved balance with multiple concurrent streams
+
+**Recommendations:**
+- Add comprehensive reserved balance tests
+- Fix the bug where reserved balance can go negative
+
+---
+
+#### 3.2.2 Cost Calculation Edge Cases ⚠️ **HIGH PRIORITY**
+
+**Missing:**
+- ❌ Cost calculation with image tokens
+- ❌ Cost calculation with web_search enabled
+- ❌ Cost calculation with internal_reasoning tokens
+- ❌ Cost calculation with max_cost limits
+- ❌ Cost calculation with tolerance percentage
+- ❌ Cost calculation for streaming responses (partial charges)
+
+**Recommendations:**
+- Add comprehensive cost calculation tests
+- Test all token types
+- Test edge cases
+
+---
+
+#### 3.2.3 Model Selection Edge Cases ⚠️ **MEDIUM PRIORITY**
+
+**Missing:**
+- ❌ Model selection with identical costs
+- ❌ Model selection with provider fees
+- ❌ Model selection with disabled models
+- ❌ Model selection with overrides
+- ❌ Model selection with missing models
+
+**Recommendations:**
+- Add integration tests for model selection
+- Test edge cases
+
+---
+
+## 4. Outdated or Problematic Tests
+
+### 4.1 Tests That May Be Outdated
+
+#### 4.1.1 `test_wallet.py::test_recieve_token_untrusted_mint()`
+
+**Issue:** Tests `swap_to_primary_mint()` behavior, but this may not be current behavior
+
+**Action:** Verify if `swap_to_primary_mint()` is still used in the codebase
+
+---
+
+#### 4.1.2 `test_background_tasks.py::test_periodic_payout()`
+
+**Issue:** Test is skipped because `periodic_payout()` is not implemented
+
+**Action:** Either implement `periodic_payout()` or remove the test
+
+---
+
+#### 4.1.3 `test_wallet_refund.py::test_refund_with_lightning_address()`
+
+**Issue:** Test is skipped because Lightning address refund is not implemented
+
+**Action:** Either implement Lightning address refund or remove the test
+
+---
+
+### 4.2 Tests That Test Implementation Details
+
+#### 4.2.1 `test_fee_consistency.py`
+
+**Issue:** Tests private function `_model_to_row_payload()` directly
+
+**Recommendation:** Test through public API instead
+
+---
+
+#### 4.2.2 `test_wallet.py`
+
+**Issue:** Heavy mocking of `Wallet.with_db()` may not reflect real behavior
+
+**Recommendation:** Use integration tests instead
+
+---
+
+## 5. Test Quality Issues
+
+### 5.1 Skipped Tests
+
+**Count:** 5+ skipped tests
+
+**Issues:**
+- Tests marked as skipped without clear path to fix
+- Some skipped tests document unimplemented features
+
+**Recommendations:**
+- Review all skipped tests
+- Either fix them or remove them
+- Document why features are not implemented
+
+---
+
+### 5.2 Brittle Tests
+
+**Issues:**
+- Heavy mocking makes tests brittle
+- Tests that depend on implementation details
+- Tests that mock database heavily
+
+**Recommendations:**
+- Prefer integration tests over heavy mocking
+- Test behavior, not implementation
+- Use real database for integration tests
+
+---
+
+### 5.3 Missing Assertions
+
+**Issues:**
+- Some tests don't verify expected behavior
+- Tests that only check status codes without verifying data
+
+**Recommendations:**
+- Add comprehensive assertions
+- Verify data correctness, not just status codes
+
+---
+
+## 6. Recommendations Summary
+
+### 6.1 Immediate Actions (High Priority)
+
+1. **Fix Reserved Balance Bug** ⚠️ **CRITICAL**
+   - Fix `revert_pay_for_request()` to prevent negative reserved balance
+   - Add comprehensive reserved balance tests
+
+2. **Add Cost Calculation Tests** ❌ **CRITICAL**
+   - Create `tests/unit/test_cost_calculation.py`
+   - Test all cost calculation scenarios
+   - Test edge cases
+
+3. **Add Upstream Provider Tests** ❌ **CRITICAL**
+   - Create unit tests for `BaseUpstreamProvider`
+   - Create provider-specific tests
+   - Test error handling
+
+4. **Add Payment Helper Tests** ❌ **CRITICAL**
+   - Test `calculate_discounted_max_cost()`
+   - Test `check_token_balance()`
+   - Test `estimate_tokens()`
+
+5. **Review Skipped Tests** ⚠️ **HIGH**
+   - Fix or remove skipped tests
+   - Document unimplemented features
+
+---
+
+### 6.2 Short-term Actions (Medium Priority)
+
+1. **Add Model Mapping Integration Tests**
+2. **Add Price Update Tests**
+3. **Add LNURL Tests**
+4. **Improve Proxy Endpoint Tests** (cost calculation, reserved balance)
+5. **Add Edge Case Tests** (reserved balance, cost calculation)
+
+---
+
+### 6.3 Long-term Actions (Low Priority)
+
+1. **Refactor Tests** - Reduce mocking, prefer integration tests
+2. **Add NIP-91 Tests** - If NIP-91 becomes more critical
+3. **Improve Test Documentation** - Better docstrings and comments
+4. **Add Performance Tests** - For critical paths
+
+---
+
+## 7. Test Coverage Summary
+
+### 7.1 Well-Tested Areas ✅
+
+- Wallet operations (topup, refund, info)
+- Authentication and API key generation
+- Database consistency
+- Error handling
+- Image token calculation
+- Model prioritization algorithm (unit level)
+- Provider discovery
+
+### 7.2 Under-Tested Areas ⚠️
+
+- Cost calculation
+- Reserved balance handling
+- Upstream provider functionality
+- Payment helpers
+- Model mappings (integration)
+- Price updates
+
+### 7.3 Missing Tests ❌
+
+- Cost calculation unit tests
+- Upstream provider unit tests
+- Payment helper tests (`calculate_discounted_max_cost`, `check_token_balance`, etc.)
+- Model mapping integration tests
+- Price update unit tests
+- LNURL tests
+- Reserved balance edge cases
+- Cost calculation edge cases (images, web_search, reasoning)
+
+---
+
+## 8. Conclusion
+
+The test suite has **good coverage** for wallet operations, authentication, and database consistency. However, there are **critical gaps** in:
+
+1. **Cost calculation** - No unit tests for core billing logic
+2. **Upstream providers** - No tests for provider functionality
+3. **Payment helpers** - Several critical functions untested
+4. **Reserved balance** - Bug exists and needs fixing + tests
+
+**Priority Actions:**
+1. Fix reserved balance bug (can go negative)
+2. Add cost calculation tests
+3. Add upstream provider tests
+4. Review and fix skipped tests
+
+The test suite is **functional but incomplete**. With the recommended additions, it would provide comprehensive coverage of the codebase.
+
+---
+
+**Report Generated:** 2024  
+**Codebase Version:** Current  
+**Test Suite Version:** Current