Compare commits

..
Author SHA1 Message Date
9qeklajc 46bbba7027 resolve reivews 2026-07-07 14:29:39 +02:00
9qeklajc e2aa02307b explicit code for error report & update doc 2026-07-05 23:05:53 +02:00
9qeklajc 66bc1260a4 refactor: align bearer redemption errors with shared X-Cashu taxonomy
Route the Authorization: Bearer cashu… redemption failures through the
same failure taxonomy the X-Cashu path already uses (token_already_spent /
invalid_token / mint_error / cashu_error, carried in the error "type"),
with matching statuses, so both redemption paths agree on the
classification for a given failure class instead of introducing a third
parallel code.

- "already spent" -> token_already_spent (400)
- fee/melt failures -> mint_error (422)
- invalid/undecodable token -> invalid_token (400, was 401)
- other expected wallet errors -> cashu_error (400)
- unexpected faults still -> internal_error (500)

Also align the msats<=0 defense-in-depth envelope and note it is now
practically unreachable (credit_balance rejects non-positive amounts).
2026-07-05 17:14:30 +02:00
9qeklajc 9dc4c979b8 fix: tighten redemption error mapping and stop X-Cashu raw-text leak
- Anchor the broad invalid/decode buckets to co-occur with "token" so
  internal faults (e.g. "Invalid isoformat string") fall through to 500
  instead of masquerading as a 401 token error.
- Map the "estimate fees"/"exceed token amount" wallet errors to the
  specific "too small to cover swap fees" message instead of the generic
  bucket.
- Stop the X-Cashu redemption path from interpolating raw mint/exception
  text into mint_error/cashu_error responses (both duplicated blocks).
- Add tests for the fee-gap mapping and the anchored gate.
2026-07-05 16:53:54 +02:00
9qeklajc 7f49ba1771 fix: cast HTTPException detail in tests for mypy 2026-07-02 00:57:31 +02:00
9qeklajc 949dc433f1 fix: return sanitized, specific errors when redeeming an incoming Cashu token fails 2026-07-02 00:47:56 +02:00
9qeklajcandGitHub b7fcf000af Merge pull request #577 from jeroenubbink/refactor/provider-identity
refactor(upstream): resolve a provider's own row by primary key
2026-07-01 22:11:29 +02:00
9qeklajcandGitHub a1af1383e1 Merge pull request #514 from Routstr/add-provider-indentifier
add provider slug
2026-07-01 17:46:13 +02:00
9qeklajc 5bedbd129f use file path 2026-07-01 17:08:28 +02:00
9qeklajc f588147b41 use slug base 2026-07-01 17:01:32 +02:00
9qeklajc 17bc949597 add test 2026-07-01 16:53:52 +02:00
9qeklajc 7705656016 resolve review comment 2026-07-01 16:40:31 +02:00
Jeroen UbbinkandClaude Opus 4.8 434283c58d refactor(upstream): resolve a provider's own row by primary key
A live upstream provider re-finds its own database row in two places —
PPQ.AI's insufficient-balance self-disable and the base
refresh_models_cache — with WHERE base_url == self.base_url AND
api_key == self.api_key. That uses a rotatable secret as a self-handle:
if the row's key rotates under a live object, it can no longer find
itself.

Carry the row's primary key on the instance as db_id, stamped centrally
by from_db_row via a _build_from_row construction hook that subclasses
override, and look the row up with session.get(UpstreamProviderRow,
db_id). This also closes a latent gap where providers built outside the
init path (auto-topup) never received db_id.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 12:53:31 +02:00
9qeklajc 9fcf870e3f add provider slug 2026-06-29 23:30:09 +02:00
9qeklajcandGitHub 782b233d40 Merge pull request #573 from Routstr/fix-deepseek-calculation
update-pricing
2026-06-29 22:29:42 +02:00
9qeklajc 0c7373675f solidify logic 2026-06-27 22:53:35 +02:00
9qeklajc a33ea5da07 update-pricing 2026-06-27 16:01:45 +02:00
9qeklajcandGitHub 38d7075652 Merge pull request #567 from Routstr/feat-prune-dead-api-keys
feat: prune dead zero-balance API keys with background janitor
2026-06-25 14:21:20 +02:00
9qeklajcandGitHub ed27ad0dea Merge pull request #569 from jeroenubbink/docs/admin-ui-serving
docs(ui): document dev/build/serve workflow; make missing-bundle warning actionable
2026-06-25 14:20:34 +02:00
Jeroen UbbinkandClaude Opus 4.8 6b1b4285fa docs(ui): document dev/build/serve workflow; make missing-bundle warning actionable
The ui/README.md was still stock create-next-app boilerplate. Replace it with the
real story: the UI is a Next.js static export served by FastAPI from ui_out/, the
two-process dev loop (backend :8000 + `make ui-dev` :3000 with hot reload, auto-
targeting :8000), the build/integration commands, and the cross-origin CORS note.

Also make the "ui_out not found" startup warning actionable — it now points to
`make ui-build` / `make ui-dev` instead of silently saying it skipped serving.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 10:54:26 +02:00
9qeklajcandGitHub 2131cfc7f9 Merge pull request #566 from Routstr/fix-reservation-overrun-charge
fix: always charge balance on reservation overrun even after stale sweep
2026-06-24 22:38:28 +02:00
9qeklajc d6d9af91e4 refactor: drop dead rowcount conditional in overrun finalize
WHERE guard now matches on hashed_key, so the UPDATE always affects one
row and result.rowcount is always truthy. Remove the conditional and its
unreachable else branch.
2026-06-24 22:21:50 +02:00
9qeklajcandGitHub cbd97e8e3a Merge pull request #549 from jeroenubbink/fix/swap-fee-retry
fix: retry foreign mint swaps with observed fees instead of trusting fee_reserve
2026-06-23 22:42:26 +02:00
Jeroen UbbinkandClaude Opus 4.8 e4b26dd27b fix: require shortfall text for generic 11000 melt errors
11000 is nutshell's generic, unregistered TransactionError covering many
unrelated failures, so trusting the code alone made any 11000 retryable.
Gate the generic code on the 'not enough inputs' detail text; keep trusting
the registered 11005 (TransactionUnbalanced) on the code alone. Behavior is
unchanged for every existing case; a bare non-shortfall 11000 now surfaces
immediately instead of burning the retry budget.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 09:38:04 +02:00
9qeklajc 4d6e2cdf59 fix: always charge balance on reservation overrun even after stale sweep 2026-06-21 15:03:50 +02:00
9qeklajcandGitHub d079f45ba3 Merge pull request #564 from Routstr/fix-ratelimit-upstream-error-handling
better rate limit forwarding
2026-06-21 14:13:57 +02:00
9qeklajcandGitHub d08f07d6fb Merge pull request #565 from Routstr/tool-support
make openrouter default to provider supporting tool use
2026-06-21 14:06:20 +02:00
9qeklajc af0796689f make openrouter default to provider supporting tool use 2026-06-20 23:55:03 +02:00
9qeklajc f408785409 better rate limit forwarding 2026-06-20 20:03:40 +02:00
Jeroen UbbinkandClaude Fable 5 eb6dc5189c test: pin the verbatim production error from issue #468 as retryable
The reported error carries cashu-py's "could not pay invoice" wrapper
around the mint detail; the classifier must still find the code and the
shortfall inside it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:08:38 +02:00
Jeroen UbbinkandClaude Fable 5 cbdbe4b899 test: cover all branches of the foreign-mint swap path
Every statement and branch of the topup path (recieve_token,
swap_to_primary_mint and its helpers, credit_balance) is now covered,
including the previously untested mint-failure/recovery block: recovery
must never credit proofs the wallet does not hold, and post-melt failures
must propagate unwrapped since the foreign proofs are already spent.

On testing internals: orchestration and side effects are tested only
through the public functions, with mocks at the external mint boundary
(the cashu Wallet object) — no internal code is ever mocked. The two pure
leaf helpers (_melt_insufficient_shortfall, _net_minted_amount) are tested
directly instead: their input spaces (seven error formats across mint
implementations, sat/msat conversions both ways) would cost ~30 lines of
quote/melt mock choreography per case through the swap loop, for the same
assertion a parametrize row makes. The same-mint branch of
_calculate_swap_amount is unreachable from public callers today; its test
documents the contract for the planned pre-flight fee estimation, which
will call it directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:08:38 +02:00
Jeroen UbbinkandClaude Fable 5 7870db9af1 fix: retry foreign mint swaps with observed fees instead of trusting fee_reserve
A foreign mint's melt fee_reserve is a non-binding estimate (NUT-05): the
mint may demand more on the real quote or reject the melt outright (e.g.
mint.cubabitcoin.org charging input fees beyond its quote). Instead of
padding the estimate with a safety buffer that strands funds at the
foreign mint on every swap, retry the mint-quote/melt-quote/melt cycle
(max 3 attempts) with the amount recomputed from the fees the mint
actually demands.

Melt failures are classified by the NUT-00 error code (11005 registered
TransactionUnbalanced as sent by cdk, 11000 nutshell's generic
TransactionError): fee-related rejections retry, others (e.g. 20004
Lightning payment failed) surface immediately. Nutshell's Provided/needed
amounts refine the retry step when present; otherwise shrink by 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:08:38 +02:00
51 changed files with 4650 additions and 610 deletions
+87 -20
View File
@@ -113,42 +113,109 @@ All errors follow a consistent JSON structure:
**Status:** 402
**Resolution:** Top up API key balance
#### Invalid Token
### Cashu Token Redemption Errors
These errors are returned when a Cashu token you pay with cannot be redeemed.
They apply to every endpoint that accepts a token:
- **Per-request payment** via the `X-Cashu` header (chat completions + Responses API).
- **API key top-up** via `POST /v1/wallet/topup`.
- **Minting an API key** from a token sent in `Authorization: Bearer <cashu-token>`.
All three share one classifier, so the same failure yields the same HTTP status
and sanitized message everywhere. Structured error envelopes (`X-Cashu` and
`Authorization: Bearer <cashu-token>`) also expose the same `type` and `code`
branch on `type` (or `code` for finer granularity). `POST /v1/wallet/topup`
keeps its existing plain-string `detail` envelope, so branch on status there.
| `type` | Status | `code` | Retryable | Meaning |
|--------|--------|--------|-----------|---------|
| `token_already_spent` | 400 | `cashu_token_already_spent` | No | The token was already redeemed. |
| `invalid_token` | 400 | `invalid_cashu_token` | No | The token is malformed or cannot be decoded. |
| `mint_error` | 422 | `cashu_token_swap_fees_exceed_amount` | No | Token value is too small to cover the mint's swap/melt fees. |
| `mint_error` | 422 | `cashu_foreign_mint_swap_failed` | No | Swapping the token from a foreign mint to the primary mint failed. |
| `mint_unreachable` | 503 | `cashu_mint_unreachable` | **Yes** | The mint could not be reached (DNS failure, refused/reset connection, timeout). The token is fine — retry once the mint recovers. |
| `cashu_error` | 400 | `cashu_token_redemption_failed` | No | The token could not be redeemed for another expected reason. |
| `cashu_error` | 400 | `cashu_token_zero_value` | No | The token redeemed to zero (empty/dust token, or value fully consumed by fees). |
| `token_consumed` | 500 | `cashu_token_consumed` | No | The token was **spent** (melted/redeemed) but crediting it then failed. Do not retry — the token is gone; contact support to reconcile. |
| `api_error` | 500 | `internal_error` | Maybe | Unexpected server-side fault during redemption. |
!!! important "Retry only `mint_unreachable`"
Only `mint_unreachable` (503) means the same token will work again later —
everything else is a permanent property of the token and must not be
blindly retried. Use exponential backoff for the 503. In particular, a
`token_consumed` 500 means the mint already spent the token, so a retry
would fail as `token_already_spent`.
#### Mint Unreachable (retryable)
```json
{
"error": {
"type": "payment_error",
"message": "Invalid Cashu token",
"code": "invalid_token",
"details": {
"reason": "Token already spent"
}
"type": "mint_unreachable",
"message": "Cashu mint is unreachable",
"code": "cashu_mint_unreachable"
}
}
```
**Status:** 400
**Resolution:** Use a valid, unspent token
**Status:** 503
#### Mint Unavailable
**Resolution:** The token is valid — the mint is temporarily down. Retry with
backoff, or pay with a token from a different mint.
#### Token Already Spent
```json
{
"error": {
"type": "payment_error",
"message": "Cannot connect to Cashu mint",
"code": "mint_unavailable",
"details": {
"mint_url": "https://mint.example.com",
"retry_after": 60
}
"type": "token_already_spent",
"message": "Cashu token already spent",
"code": "cashu_token_already_spent"
}
}
```
**Status:** 503
**Resolution:** Try again later or use different mint
**Status:** 400
**Resolution:** Use a fresh, unspent token. Do not retry with the same token.
#### Response envelope differs by endpoint
The `error` object above is identical everywhere, but the surrounding envelope
depends on how you paid:
- **`X-Cashu` header payments** (chat + Responses API) return the object at the
top level, alongside a `request_id`:
```json
{
"error": { "type": "mint_unreachable", "message": "Cashu mint is unreachable", "code": "cashu_mint_unreachable" },
"request_id": "req-abc123"
}
```
The original token is echoed back in the `X-Cashu` **response header only when
it is still spendable** (e.g. `mint_unreachable`, `invalid_cashu_token`, fee
errors) so you can recover/retry it. It is **not** echoed for spent/consumed
tokens (`cashu_token_already_spent`, `cashu_token_consumed`,
`cashu_token_zero_value`, `internal_error`) — retrying those can never succeed.
- **`Authorization: Bearer <cashu-token>`** (API key minting) wraps it in
FastAPI's `detail` field:
```json
{ "detail": { "error": { "type": "mint_unreachable", "message": "Cashu mint is unreachable", "code": "cashu_mint_unreachable" } } }
```
- **`POST /v1/wallet/topup`** returns a plain string message under `detail` —
it carries the shared HTTP **status** and **message** (e.g. `503` for an
unreachable mint) but not the structured `type`/`code`, so branch on the
status code here:
```json
{ "detail": "Cashu mint is unreachable" }
```
### Validation Errors
@@ -347,7 +414,7 @@ class ErrorHandler:
'rate_limit',
'upstream_timeout',
'model_overloaded',
'mint_unavailable'
'cashu_mint_unreachable'
}
# Errors requiring user action
@@ -0,0 +1,88 @@
"""add slug to upstream_providers
Revision ID: c6d7e8f9a0b1
Revises: b5e7c9d1f3a2
Create Date: 2026-06-29 00:00:00.000000
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
from routstr.core.provider_slugs import provider_slug_base, provider_slug_candidate
revision = "c6d7e8f9a0b1"
down_revision = "b5e7c9d1f3a2"
branch_labels = None
depends_on = None
def _allocate_backfill_slug(provider_type: str, reserved_slugs: set[str]) -> str:
base = provider_slug_base(provider_type)
suffix_number = 1
while True:
candidate = provider_slug_candidate(base, suffix_number)
if candidate not in reserved_slugs:
reserved_slugs.add(candidate)
return candidate
suffix_number += 1
def _backfill_provider_slugs(conn: sa.Connection) -> None:
existing_rows = conn.execute(
sa.text(
"SELECT slug FROM upstream_providers "
"WHERE slug IS NOT NULL AND slug != ''"
)
)
reserved_slugs = {str(row.slug).lower() for row in existing_rows}
rows_to_backfill = conn.execute(
sa.text(
"SELECT id, provider_type FROM upstream_providers "
"WHERE slug IS NULL OR slug = '' "
"ORDER BY id"
)
)
for row in rows_to_backfill:
slug = _allocate_backfill_slug(str(row.provider_type), reserved_slugs)
conn.execute(
sa.text("UPDATE upstream_providers SET slug = :slug WHERE id = :id"),
{"slug": slug, "id": row.id},
)
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = {c["name"] for c in inspector.get_columns("upstream_providers")}
if "slug" not in columns:
op.add_column(
"upstream_providers",
sa.Column("slug", sa.String(), nullable=True),
)
_backfill_provider_slugs(conn)
existing_indexes = {idx["name"] for idx in inspector.get_indexes("upstream_providers")}
if "ix_upstream_providers_slug" not in existing_indexes:
op.create_index(
"ix_upstream_providers_slug",
"upstream_providers",
["slug"],
unique=True,
)
def downgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_indexes = {idx["name"] for idx in inspector.get_indexes("upstream_providers")}
if "ix_upstream_providers_slug" in existing_indexes:
op.drop_index("ix_upstream_providers_slug", table_name="upstream_providers")
columns = {c["name"] for c in inspector.get_columns("upstream_providers")}
if "slug" in columns:
op.drop_column("upstream_providers", "slug")
+110 -60
View File
@@ -20,7 +20,11 @@ from .payment.cost_calculation import (
MaxCostData,
calculate_cost,
)
from .wallet import credit_balance, deserialize_token_from_string
from .wallet import (
classify_redemption_error,
credit_balance,
deserialize_token_from_string,
)
logger = get_logger(__name__)
payments_logger = get_logger("routstr.payments")
@@ -78,6 +82,37 @@ async def check_and_reset_limit(key: ApiKey, session: AsyncSession) -> bool:
return False
def redemption_error_to_http_exception(error: Exception) -> HTTPException:
"""Map a Cashu token redemption failure to a sanitized client-facing error.
Thin wrapper over the shared :func:`classify_redemption_error` so the bearer
path stays identical to the X-Cashu and top-up paths.
"""
classified = classify_redemption_error(error)
if classified is None:
return HTTPException(
status_code=500,
detail={
"error": {
"message": "Internal error during token redemption",
"type": "api_error",
"code": "internal_error",
}
},
)
error_type, status_code, message, error_code = classified
return HTTPException(
status_code=status_code,
detail={
"error": {
"message": message,
"type": error_type,
"code": error_code,
}
},
)
async def validate_bearer_key(
bearer_key: str,
session: AsyncSession,
@@ -216,7 +251,17 @@ async def validate_bearer_key(
try:
hashed_key = hashlib.sha256(bearer_key.encode()).hexdigest()
token_obj = deserialize_token_from_string(bearer_key)
try:
token_obj = deserialize_token_from_string(bearer_key)
except Exception as decode_error:
# A malformed token is a bad token (400 invalid_cashu_token via
# the shared taxonomy), not an auth failure (401) — otherwise it
# would fall through to the generic "Invalid API key" handler.
raise redemption_error_to_http_exception(
ValueError(
f"Invalid Cashu token: could not decode token ({decode_error})"
)
) from decode_error
logger.debug(
"Generated token hash", extra={"hash_preview": hashed_key[:16] + "..."}
)
@@ -334,19 +379,32 @@ async def validate_bearer_key(
"error_type": type(credit_error).__name__,
},
)
raise credit_error
await session.rollback()
raise redemption_error_to_http_exception(credit_error) from credit_error
if msats <= 0:
logger.error(
"Token redemption returned zero or negative amount",
extra={"msats": msats, "key_hash": hashed_key[:8] + "..."},
)
# Defense-in-depth: credit_balance now refuses to commit on a
# zero/negative redemption, but if a row was nonetheless
# persisted, drop it so we never leave an orphan zero-balance key.
# Defense-in-depth: credit_balance already raises
# ValueError("Redeemed token amount must be positive…") before
# returning (wallet.py), so this branch is only reachable if a
# zero/negative row was somehow persisted; drop it so we never
# leave an orphan zero-balance key. Reuse the shared taxonomy
# (cashu_error) so the envelope matches the mapper above.
await session.delete(new_key)
await session.commit()
raise Exception("Token redemption failed")
raise HTTPException(
status_code=400,
detail={
"error": {
"message": "Failed to redeem Cashu token: token yielded no value",
"type": "cashu_error",
"code": "cashu_token_zero_value",
}
},
)
await session.refresh(new_key)
await session.commit()
@@ -379,7 +437,7 @@ async def validate_bearer_key(
status_code=401,
detail={
"error": {
"message": f"Invalid or expired Cashu key: {str(e)}",
"message": "Invalid or expired Cashu key",
"type": "invalid_request_error",
"code": "invalid_api_key",
}
@@ -1021,32 +1079,37 @@ async def adjust_payment_for_tokens(
# actual cost exceeded discounted reservation (due to tolerance_percentage)
if cost_difference > 0:
# Always release the reservation and charge min(actual_cost, balance).
# Using a CASE expression makes this a single atomic UPDATE — no
# multi-level fallback needed and balance can never go negative.
# CASE expressions keep this atomic and safe even when the
# stale-reservation sweeper has already released the reservation.
chargeable = case(
(col(ApiKey.balance) >= total_cost_msats, total_cost_msats),
else_=col(ApiKey.balance),
)
overrun_safe_reserved = case(
(
col(ApiKey.reserved_balance) >= deducted_max_cost,
col(ApiKey.reserved_balance) - deducted_max_cost,
),
else_=0,
)
finalize_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == billing_key.hashed_key)
.where(col(ApiKey.reserved_balance) >= deducted_max_cost)
.values(
reserved_balance=col(ApiKey.reserved_balance) - deducted_max_cost,
reserved_balance=overrun_safe_reserved,
balance=col(ApiKey.balance) - chargeable,
total_spent=col(ApiKey.total_spent) + chargeable,
)
)
result = await session.exec(finalize_stmt) # type: ignore[call-overload]
await session.exec(finalize_stmt) # type: ignore[call-overload]
if billing_key.hashed_key != key.hashed_key:
child_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == key.hashed_key)
.where(col(ApiKey.reserved_balance) >= deducted_max_cost)
.values(
reserved_balance=col(ApiKey.reserved_balance) - deducted_max_cost,
reserved_balance=overrun_safe_reserved,
total_spent=col(ApiKey.total_spent) + min(billing_key.balance, total_cost_msats),
)
)
@@ -1054,51 +1117,38 @@ async def adjust_payment_for_tokens(
await session.commit()
if result.rowcount:
await session.refresh(billing_key)
if billing_key.hashed_key != key.hashed_key:
await session.refresh(key)
cost.total_msats = total_cost_msats
logger.info(
"Finalized payment with additional charge",
extra={
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"charged_amount": total_cost_msats,
"new_balance": billing_key.balance,
"model": model,
},
)
await _accumulate_fee(total_cost_msats)
payments_logger.info(
"FINALIZE",
extra={
"event": "finalize",
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"model": model,
"cost_reserved": deducted_max_cost,
"cost_charged": total_cost_msats,
"input_tokens": cost.input_tokens,
"output_tokens": cost.output_tokens,
"balance": billing_key.balance,
"reserved_balance": billing_key.reserved_balance,
"total_spent": billing_key.total_spent,
"finalize_type": "overrun",
},
)
else:
# Guard fired: reservation was already released by a concurrent
# finalization for this key. Nothing left to do.
logger.warning(
"Finalization skipped - reservation already released",
extra={
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"attempted_charge": total_cost_msats,
"model": model,
},
)
await session.refresh(billing_key)
if billing_key.hashed_key != key.hashed_key:
await session.refresh(key)
cost.total_msats = total_cost_msats
logger.info(
"Finalized payment with additional charge",
extra={
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"charged_amount": total_cost_msats,
"new_balance": billing_key.balance,
"model": model,
},
)
await _accumulate_fee(total_cost_msats)
payments_logger.info(
"FINALIZE",
extra={
"event": "finalize",
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"model": model,
"cost_reserved": deducted_max_cost,
"cost_charged": total_cost_msats,
"input_tokens": cost.input_tokens,
"output_tokens": cost.output_tokens,
"balance": billing_key.balance,
"reserved_balance": billing_key.reserved_balance,
"total_spent": billing_key.total_spent,
"finalize_type": "overrun",
},
)
else:
# Refund some of the base cost
refund = abs(cost_difference)
+22 -31
View File
@@ -20,7 +20,14 @@ from .core.db import (
from .core.logging import get_logger
from .core.settings import settings
from .lightning import lightning_router
from .wallet import credit_balance, recieve_token, send_to_lnurl, send_token
from .wallet import (
classify_redemption_error,
credit_balance,
is_mint_connection_error,
recieve_token,
send_to_lnurl,
send_token,
)
router = APIRouter()
balance_router = APIRouter(prefix="/v1/balance")
@@ -156,30 +163,18 @@ async def topup_wallet_endpoint(
raise HTTPException(status_code=400, detail="Invalid token format")
try:
amount_msats = await credit_balance(cashu_token, billing_key, session)
except ValueError as e:
error_msg = str(e)
if "already spent" in error_msg.lower():
raise HTTPException(status_code=400, detail="Token already spent")
elif "invalid" in error_msg.lower() or "decode" in error_msg.lower():
raise HTTPException(status_code=400, detail="Invalid token format")
elif "insufficient" in error_msg.lower() or "melt fee" in error_msg.lower():
raise HTTPException(
status_code=400,
detail=f"Token value is too small to cover swap fees. {error_msg}",
)
elif "failed to melt" in error_msg.lower():
raise HTTPException(
status_code=400,
detail=f"Failed to swap foreign mint token. {error_msg}",
)
else:
raise HTTPException(status_code=400, detail=f"Failed to redeem token: {error_msg}")
except Exception as e:
logger.error(
"topup_wallet_endpoint: unhandled error",
extra={"error": str(e), "error_type": type(e).__name__},
)
raise HTTPException(status_code=500, detail="Internal server error")
# Shared taxonomy so top-up matches the bearer/X-Cashu paths (503 for an
# unreachable mint, 422 for fee/swap failures, 400 for token faults).
classified = classify_redemption_error(e)
if classified is None:
logger.error(
"topup_wallet_endpoint: unhandled error",
extra={"error": str(e), "error_type": type(e).__name__},
)
raise HTTPException(status_code=500, detail="Internal server error")
_type, status_code, message, _code = classified
raise HTTPException(status_code=status_code, detail=message)
return {"msats": amount_msats}
@@ -441,14 +436,10 @@ async def refund_wallet_endpoint(
"has_refund_address": bool(key.refund_address),
},
)
if (
"mint" in error_msg.lower()
or "connection" in error_msg.lower()
or "ConnectError" in str(type(e))
):
raise HTTPException(status_code=503, detail=f"Mint service unavailable: {error_msg}")
if is_mint_connection_error(e):
raise HTTPException(status_code=503, detail="Mint service unavailable")
else:
raise HTTPException(status_code=500, detail=f"Refund failed: {error_msg}")
raise HTTPException(status_code=500, detail="Refund failed")
await _refund_cache_set(bearer_value, result)
+219 -127
View File
@@ -1,5 +1,6 @@
import asyncio
import json
import re
import secrets
from datetime import datetime, timezone
from pathlib import Path
@@ -8,6 +9,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request
from pydantic import BaseModel, RootModel
from pydantic.v1 import ValidationError as PydanticValidationError
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from ..payment.models import _row_to_model, list_models
from ..proxy import refresh_model_maps, reinitialize_upstreams
@@ -29,6 +31,7 @@ from .db import (
)
from .log_manager import log_manager
from .logging import get_logger
from .provider_slugs import allocate_unique_provider_slug
from .settings import SettingsService, settings
logger = get_logger(__name__)
@@ -456,19 +459,18 @@ class ModelCreate(BaseModel):
dependencies=[Depends(require_admin_api)],
)
async def upsert_provider_model(
provider_id: int, payload: ModelCreate
provider_id: str, payload: ModelCreate
) -> dict[str, object]:
print(payload)
logger.info(
f"UPSERT_PROVIDER_MODEL called: provider_id={provider_id}, model_id={payload.id}"
)
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
# Try to get existing model
existing_row = await session.get(ModelRow, (payload.id, provider_id))
existing_row = await session.get(ModelRow, (payload.id, provider_pk))
if existing_row:
# Update existing model
@@ -524,7 +526,7 @@ async def upsert_provider_model(
alias_ids=(
json.dumps(payload.alias_ids) if payload.alias_ids else None
),
upstream_provider_id=provider_id,
upstream_provider_id=provider_pk,
enabled=payload.enabled,
forwarded_model_id=payload.forwarded_model_id or payload.id,
)
@@ -543,7 +545,7 @@ async def upsert_provider_model(
dependencies=[Depends(require_admin_api)],
)
async def update_provider_model_legacy(
provider_id: int, model_id: str, payload: ModelCreate
provider_id: str, model_id: str, payload: ModelCreate
) -> dict[str, object]:
"""Legacy PATCH endpoint - redirects to upsert POST endpoint for backward compatibility."""
logger.info(
@@ -556,13 +558,12 @@ async def update_provider_model_legacy(
"/api/upstream-providers/{provider_id}/models/{model_id:path}",
dependencies=[Depends(require_admin_api)],
)
async def get_provider_model(provider_id: int, model_id: str) -> dict[str, object]:
async def get_provider_model(provider_id: str, model_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
row = await session.get(ModelRow, (model_id, provider_id))
row = await session.get(ModelRow, (model_id, provider_pk))
if not row:
raise HTTPException(
status_code=404, detail="Model not found for this provider"
@@ -576,9 +577,11 @@ async def get_provider_model(provider_id: int, model_id: str) -> dict[str, objec
"/api/upstream-providers/{provider_id}/models/{model_id:path}",
dependencies=[Depends(require_admin_api)],
)
async def delete_provider_model(provider_id: int, model_id: str) -> dict[str, object]:
async def delete_provider_model(provider_id: str, model_id: str) -> dict[str, object]:
async with create_session() as session:
row = await session.get(ModelRow, (model_id, provider_id))
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
row = await session.get(ModelRow, (model_id, provider_pk))
if not row:
raise HTTPException(
status_code=404, detail="Model not found for this provider"
@@ -593,10 +596,12 @@ async def delete_provider_model(provider_id: int, model_id: str) -> dict[str, ob
"/api/upstream-providers/{provider_id}/models",
dependencies=[Depends(require_admin_api)],
)
async def delete_all_provider_models(provider_id: int) -> dict[str, object]:
async def delete_all_provider_models(provider_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
result = await session.exec(
select(ModelRow).where(ModelRow.upstream_provider_id == provider_id)
select(ModelRow).where(ModelRow.upstream_provider_id == provider_pk)
) # type: ignore
rows = result.all()
for row in rows:
@@ -615,7 +620,7 @@ class BatchOverrideRequest(BaseModel):
dependencies=[Depends(require_admin_api)],
)
async def batch_override_provider_models(
provider_id: int, payload: BatchOverrideRequest
provider_id: str, payload: BatchOverrideRequest
) -> dict[str, object]:
"""Batch override models for a specific provider."""
logger.info(
@@ -623,15 +628,14 @@ async def batch_override_provider_models(
)
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
overridden_count = 0
for model_data in payload.models:
# Try to get existing model regardless of whether it's enabled or not
existing_row = await session.get(ModelRow, (model_data.id, provider_id))
existing_row = await session.get(ModelRow, (model_data.id, provider_pk))
if existing_row:
# Update existing
@@ -685,7 +689,7 @@ async def batch_override_provider_models(
if model_data.alias_ids
else None
),
upstream_provider_id=provider_id,
upstream_provider_id=provider_pk,
enabled=model_data.enabled,
)
session.add(row)
@@ -702,6 +706,85 @@ async def batch_override_provider_models(
}
_SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9-]{1,62}[a-z0-9]$")
def _validate_slug(value: str) -> str:
candidate = value.strip().lower()
if not _SLUG_PATTERN.fullmatch(candidate):
raise HTTPException(
status_code=400,
detail=(
"slug must be 3-64 chars, lowercase letters/digits/hyphens, "
"and may not start or end with a hyphen"
),
)
if candidate.isdigit():
raise HTTPException(
status_code=400,
detail="slug must not be all digits",
)
return candidate
async def _ensure_unique_slug(
session: AsyncSession, slug: str, exclude_id: int | None = None
) -> None:
stmt = select(UpstreamProviderRow).where(UpstreamProviderRow.slug == slug)
result = await session.exec(stmt)
existing = result.first()
if existing and existing.id != exclude_id:
raise HTTPException(
status_code=409,
detail="Provider with this slug already exists",
)
async def _get_upstream_provider_by_ref(
session: AsyncSession, provider_ref: str
) -> UpstreamProviderRow:
if provider_ref.isdigit():
provider = await session.get(UpstreamProviderRow, int(provider_ref))
else:
slug = _validate_slug(provider_ref)
result = await session.exec(
select(UpstreamProviderRow).where(UpstreamProviderRow.slug == slug)
)
provider = result.first()
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
return provider
def _provider_pk(provider: UpstreamProviderRow) -> int:
if provider.id is None:
raise HTTPException(status_code=500, detail="Provider has no database id")
return provider.id
def _serialize_provider(
provider: UpstreamProviderRow, redact_api_key: bool = True
) -> dict[str, object]:
return {
"id": provider.id,
"slug": provider.slug,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]"
if (redact_api_key and provider.api_key)
else provider.api_key
if not redact_api_key
else "",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": json.loads(provider.provider_settings)
if provider.provider_settings
else None,
}
class UpstreamProviderCreate(BaseModel):
provider_type: str
base_url: str
@@ -710,6 +793,7 @@ class UpstreamProviderCreate(BaseModel):
enabled: bool = True
provider_fee: float = 1.01
provider_settings: dict | None = None
slug: str | None = None
class UpstreamProviderUpdate(BaseModel):
@@ -720,6 +804,50 @@ class UpstreamProviderUpdate(BaseModel):
enabled: bool | None = None
provider_fee: float | None = None
provider_settings: dict | None = None
slug: str | None = None
class UpstreamProviderUpdateBySlug(BaseModel):
slug: str
new_slug: str | None = None
provider_type: str | None = None
base_url: str | None = None
api_key: str | None = None
api_version: str | None = None
enabled: bool | None = None
provider_fee: float | None = None
provider_settings: dict | None = None
async def _apply_provider_update(
session: AsyncSession,
provider: UpstreamProviderRow,
payload: UpstreamProviderUpdate,
new_slug: str | None = None,
) -> None:
if new_slug is not None:
validated = _validate_slug(new_slug)
await _ensure_unique_slug(session, validated, exclude_id=provider.id)
provider.slug = validated
if payload.provider_type is not None:
provider.provider_type = payload.provider_type
if payload.base_url is not None:
provider.base_url = payload.base_url
if payload.api_key is not None:
provider.api_key = payload.api_key
if payload.api_version is not None:
provider.api_version = payload.api_version
if payload.enabled is not None:
provider.enabled = payload.enabled
if payload.provider_fee is not None:
provider.provider_fee = payload.provider_fee
if payload.provider_settings is not None:
provider.provider_settings = json.dumps(payload.provider_settings)
session.add(provider)
await session.commit()
await session.refresh(provider)
@admin_router.get("/api/upstream-providers", dependencies=[Depends(require_admin_api)])
@@ -727,21 +855,7 @@ async def get_upstream_providers() -> list[dict[str, object]]:
async with create_session() as session:
result = await session.exec(select(UpstreamProviderRow))
providers = result.all()
return [
{
"id": p.id,
"provider_type": p.provider_type,
"base_url": p.base_url,
"api_key": "[REDACTED]" if p.api_key else "",
"api_version": p.api_version,
"enabled": p.enabled,
"provider_fee": p.provider_fee,
"provider_settings": json.loads(p.provider_settings)
if p.provider_settings
else None,
}
for p in providers
]
return [_serialize_provider(p) for p in providers]
@admin_router.post("/api/upstream-providers", dependencies=[Depends(require_admin_api)])
@@ -761,7 +875,14 @@ async def create_upstream_provider(
detail="Provider with this base URL and API key already exists",
)
if payload.slug:
slug = _validate_slug(payload.slug)
await _ensure_unique_slug(session, slug)
else:
slug = await allocate_unique_provider_slug(session, payload.provider_type)
provider = UpstreamProviderRow(
slug=slug,
provider_type=payload.provider_type,
base_url=payload.base_url,
api_key=payload.api_key,
@@ -778,99 +899,81 @@ async def create_upstream_provider(
await reinitialize_upstreams()
await refresh_model_maps()
return {
"id": provider.id,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": payload.provider_settings,
}
return _serialize_provider(provider)
@admin_router.get(
"/api/upstream-providers/{provider_id}", dependencies=[Depends(require_admin_api)]
)
async def get_upstream_provider(provider_id: int) -> dict[str, object]:
async def get_upstream_provider(provider_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
return {
"id": provider.id,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]" if provider.api_key else "",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": json.loads(provider.provider_settings)
if provider.provider_settings
else None,
}
provider = await _get_upstream_provider_by_ref(session, provider_id)
return _serialize_provider(provider)
@admin_router.patch(
"/api/upstream-providers/{provider_id}", dependencies=[Depends(require_admin_api)]
)
async def update_upstream_provider(
provider_id: int, payload: UpstreamProviderUpdate
provider_id: str, payload: UpstreamProviderUpdate
) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
if payload.provider_type is not None:
provider.provider_type = payload.provider_type
if payload.base_url is not None:
provider.base_url = payload.base_url
if payload.api_key is not None:
provider.api_key = payload.api_key
if payload.api_version is not None:
provider.api_version = payload.api_version
if payload.enabled is not None:
provider.enabled = payload.enabled
if payload.provider_fee is not None:
provider.provider_fee = payload.provider_fee
if payload.provider_settings is not None:
provider.provider_settings = json.dumps(payload.provider_settings)
session.add(provider)
await session.commit()
await session.refresh(provider)
await _apply_provider_update(session, provider, payload, new_slug=payload.slug)
await reinitialize_upstreams()
await refresh_model_maps()
return {
"id": provider.id,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": json.loads(provider.provider_settings)
if provider.provider_settings
else None,
}
return _serialize_provider(provider)
@admin_router.patch(
"/api/upstream-providers", dependencies=[Depends(require_admin_api)]
)
async def update_upstream_provider_by_slug(
payload: UpstreamProviderUpdateBySlug,
) -> dict[str, object]:
lookup = _validate_slug(payload.slug)
async with create_session() as session:
result = await session.exec(
select(UpstreamProviderRow).where(
UpstreamProviderRow.slug == lookup
)
)
provider = result.first()
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
update_payload = UpstreamProviderUpdate(
provider_type=payload.provider_type,
base_url=payload.base_url,
api_key=payload.api_key,
api_version=payload.api_version,
enabled=payload.enabled,
provider_fee=payload.provider_fee,
provider_settings=payload.provider_settings,
)
await _apply_provider_update(
session, provider, update_payload, new_slug=payload.new_slug
)
await reinitialize_upstreams()
await refresh_model_maps()
return _serialize_provider(provider)
@admin_router.delete(
"/api/upstream-providers/{provider_id}", dependencies=[Depends(require_admin_api)]
)
async def delete_upstream_provider(provider_id: int) -> dict[str, object]:
async def delete_upstream_provider(provider_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
deleted_id = _provider_pk(provider)
await session.delete(provider)
await session.commit()
await reinitialize_upstreams()
await refresh_model_maps()
return {"ok": True, "deleted_id": provider_id}
return {"ok": True, "deleted_id": deleted_id}
@admin_router.get("/api/provider-types", dependencies=[Depends(require_admin_api)])
@@ -885,17 +988,16 @@ async def get_provider_types() -> list[dict[str, object]]:
"/api/upstream-providers/{provider_id}/models",
dependencies=[Depends(require_admin_api)],
)
async def get_provider_models(provider_id: int) -> dict[str, object]:
async def get_provider_models(provider_id: str) -> dict[str, object]:
from ..upstream.helpers import _instantiate_provider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
db_models = await list_models(
session=session,
upstream_id=provider_id,
upstream_id=provider_pk,
include_disabled=True,
apply_fees=False,
)
@@ -985,13 +1087,11 @@ class TopupTokenRequest(BaseModel):
dependencies=[Depends(require_admin_api)],
)
async def topup_provider_with_token(
provider_id: int, payload: TopupTokenRequest
provider_id: str, payload: TopupTokenRequest
) -> dict:
"""Redeem a Cashu token for an upstream provider."""
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
import httpx
@@ -1022,15 +1122,13 @@ async def topup_provider_with_token(
dependencies=[Depends(require_admin_api)],
)
async def initiate_provider_topup(
provider_id: int, payload: TopupRequest
provider_id: str, payload: TopupRequest
) -> dict[str, object]:
"""Initiate a Lightning Network top-up for the upstream provider account."""
from ..upstream.helpers import _instantiate_provider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
try:
logger.info(
@@ -1150,15 +1248,13 @@ async def initiate_provider_topup(
"/api/upstream-providers/{provider_id}/topup/{invoice_id}/status",
dependencies=[Depends(require_admin_api)],
)
async def check_topup_status(provider_id: int, invoice_id: str) -> dict[str, object]:
async def check_topup_status(provider_id: str, invoice_id: str) -> dict[str, object]:
"""Check the status of a Lightning Network top-up invoice."""
from ..upstream.helpers import _instantiate_provider
from ..upstream.ppqai import PPQAIUpstreamProvider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
# For Routstr providers, proxy the status check
if provider.provider_type == "routstr":
@@ -1205,14 +1301,12 @@ async def check_topup_status(provider_id: int, invoice_id: str) -> dict[str, obj
"/api/upstream-providers/{provider_id}/balance",
dependencies=[Depends(require_admin_api)],
)
async def get_provider_balance(provider_id: int) -> dict[str, object]:
async def get_provider_balance(provider_id: str) -> dict[str, object]:
"""Get the current balance for an upstream provider account."""
from ..upstream.helpers import _instantiate_provider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
# For Routstr providers, proxy the balance check
if provider.provider_type == "routstr":
@@ -1591,15 +1685,13 @@ async def get_lightning_invoices_api(
"/api/upstream-providers/{provider_id}/routstr/refund",
dependencies=[Depends(require_admin_api)],
)
async def refund_routstr_provider_balance(provider_id: int) -> dict[str, object]:
async def refund_routstr_provider_balance(provider_id: str) -> dict[str, object]:
"""Refund balance from an upstream Routstr provider back to the local wallet."""
from ..upstream.helpers import _instantiate_provider
from ..upstream.routstr import RoutstrUpstreamProvider
async with create_session() as session:
provider_row = await session.get(UpstreamProviderRow, provider_id)
if not provider_row:
raise HTTPException(status_code=404, detail="Provider not found")
provider_row = await _get_upstream_provider_by_ref(session, provider_id)
if provider_row.provider_type != "routstr":
raise HTTPException(
+6
View File
@@ -319,6 +319,12 @@ class UpstreamProviderRow(SQLModel, table=True): # type: ignore
),
)
id: int | None = Field(default=None, primary_key=True)
slug: str | None = Field(
default=None,
unique=True,
index=True,
description="Stable external slug used for updates via API key.",
)
provider_type: str = Field(
description="Provider type: custom, openai, anthropic, azure, openrouter, etc."
)
+17 -2
View File
@@ -7,11 +7,26 @@ logger = get_logger(__name__)
class UpstreamError(Exception):
"""Exception raised when an upstream provider fails."""
"""Exception raised when an upstream provider fails.
def __init__(self, message: str, status_code: int = 502):
``code`` carries a stable, machine-readable classification (e.g.
``UPSTREAM_RATE_LIMIT``) so callers can distinguish failure kinds without
string-matching the message. ``details`` holds optional structured,
redaction-safe context. Both default to ``None`` for backwards
compatibility.
"""
def __init__(
self,
message: str,
status_code: int = 502,
code: str | None = None,
details: dict[str, object] | None = None,
):
self.message = message
self.status_code = status_code
self.code = code
self.details = details
super().__init__(message)
+44
View File
@@ -51,6 +51,8 @@ from pythonjsonlogger import jsonlogger
from rich.console import Console
from rich.logging import RichHandler
from .redaction import redact_obj, redact_org_ids
# Only use RichHandler when stdout is a real TTY. In non-TTY contexts
# (docker logs, pipes, CI) Rich pads every line to width and wraps long
# records, producing visually-empty trailing whitespace and split records.
@@ -180,6 +182,37 @@ class RequestIdFilter(logging.Filter):
return True
# Standard ``LogRecord`` attributes that are never user-supplied ``extra``
# fields; skipped when redacting structured extras (``msg``/``message`` are
# handled separately above).
_NON_EXTRA_RECORD_ATTRS = frozenset(
{
"name",
"msg",
"args",
"levelname",
"levelno",
"pathname",
"filename",
"module",
"exc_info",
"exc_text",
"stack_info",
"lineno",
"funcName",
"created",
"msecs",
"relativeCreated",
"thread",
"threadName",
"processName",
"process",
"taskName",
"message",
}
)
class SecurityFilter(logging.Filter):
"""Filter to remove sensitive information from logs."""
@@ -203,6 +236,7 @@ class SecurityFilter(logging.Filter):
"""Filter out sensitive information from log records."""
try:
message = record.getMessage()
message = redact_org_ids(message)
standalone_patterns = [
r"Bearer\s+([a-zA-Z0-9_\-\.]{10,})", # Bearer token (must be 10 characters or more to reduce false-positives)
r"cashu[A-Z]+([a-zA-Z0-9_\-\.=/+]+)", # Cashu tokens
@@ -224,6 +258,16 @@ class SecurityFilter(logging.Filter):
record.msg = message
record.args = ()
# Structured `extra={...}` fields are emitted by the JSON formatter
# straight from the record dict and never pass through the message
# formatting above. Redact organization IDs from any string-valued
# extra so they cannot leak via structured logs.
for attr, value in list(record.__dict__.items()):
if attr in _NON_EXTRA_RECORD_ATTRS:
continue
if isinstance(value, (str, dict, list, tuple)):
record.__dict__[attr] = redact_obj(value)
except Exception:
pass
+4 -1
View File
@@ -379,7 +379,10 @@ if UI_DIST_PATH.exists() and UI_DIST_PATH.is_dir():
else:
logger.warning(
f"UI dist directory not found at {UI_DIST_PATH}, skipping static file serving"
"UI dist directory not found at %s; serving API only. Run `make ui-build` "
"to build the static UI served from here, or `make ui-dev` for the Next.js "
"dev server with hot reload on :3000 (it targets this backend on :8000).",
UI_DIST_PATH,
)
@app.get("/", include_in_schema=False)
+65
View File
@@ -0,0 +1,65 @@
from __future__ import annotations
import re
from itertools import count
from typing import Collection
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from .db import UpstreamProviderRow
_SLUG_BASE_PATTERN = re.compile(r"[^a-z0-9]+")
_MAX_SLUG_LENGTH = 64
def provider_slug_base(provider_type: str) -> str:
"""Return a deterministic slug base for a provider type."""
base = _SLUG_BASE_PATTERN.sub("-", provider_type.lower()).strip("-")
if not base:
base = "provider"
elif base.isdigit():
base = f"provider-{base}"
elif len(base) < 3:
base = f"{base}-provider"
if len(base) > _MAX_SLUG_LENGTH:
base = base[:_MAX_SLUG_LENGTH].rstrip("-") or "provider"
return base
def provider_slug_candidate(base: str, suffix_number: int) -> str:
if suffix_number == 1:
return base
suffix = f"-{suffix_number}"
max_base_length = _MAX_SLUG_LENGTH - len(suffix)
return f"{base[:max_base_length].rstrip('-')}{suffix}"
async def allocate_unique_provider_slug(
session: AsyncSession,
provider_type: str,
reserved_slugs: Collection[str] = (),
) -> str:
"""Allocate a stable, deterministic provider slug.
The first provider of a type gets ``openai``; later collisions get
``openai-2``, ``openai-3``, etc. ``reserved_slugs`` covers rows staged in
memory but not flushed yet, such as settings/env seeding.
"""
base = provider_slug_base(provider_type)
reserved = {slug.lower() for slug in reserved_slugs}
for suffix_number in count(1):
candidate = provider_slug_candidate(base, suffix_number)
if candidate in reserved:
continue
result = await session.exec(
select(UpstreamProviderRow).where(UpstreamProviderRow.slug == candidate)
)
if result.first() is None:
return candidate
raise RuntimeError("unreachable")
+51
View File
@@ -0,0 +1,51 @@
"""Redaction helpers for sensitive provider identifiers.
Single source of truth for stripping account-scoped identifiers (e.g. OpenAI
organization IDs) from any text before it is logged, returned to a caller, or
written to an audit entry.
"""
from __future__ import annotations
import re
from typing import Any
# OpenAI-style organization identifiers look like ``org-<base62>``. Require at
# least 6 trailing chars so the already-redacted literal ``org-[REDACTED]`` is
# never re-matched (``[`` is not in the character class).
_ORG_ID_PATTERN = re.compile(r"\borg-[A-Za-z0-9]{6,}\b")
ORG_ID_PLACEHOLDER = "org-[REDACTED]"
def redact_org_ids(text: str) -> str:
"""Replace OpenAI-style organization IDs with ``org-[REDACTED]``.
Args:
text: Arbitrary text that may embed an ``org-*`` identifier.
Returns:
The text with every organization ID replaced. Non-string input is
returned unchanged after coercion to ``str``.
"""
if not text:
return text
return _ORG_ID_PATTERN.sub(ORG_ID_PLACEHOLDER, text)
def redact_obj(obj: Any) -> Any:
"""Recursively redact organization IDs in arbitrary nested structures.
Strings are redacted in place; dicts and lists/tuples are walked so that
identifiers nested inside structured payloads (e.g. log ``extra`` fields or
error ``details``) are also stripped. Other types are returned unchanged.
"""
if isinstance(obj, str):
return redact_org_ids(obj)
if isinstance(obj, dict):
return {key: redact_obj(value) for key, value in obj.items()}
if isinstance(obj, list):
return [redact_obj(value) for value in obj]
if isinstance(obj, tuple):
return tuple(redact_obj(value) for value in obj)
return obj
+36 -6
View File
@@ -11,6 +11,8 @@ from PIL import Image
from sqlmodel.ext.asyncio.session import AsyncSession
from ..core import get_logger
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
from ..core.settings import settings
from ..wallet import deserialize_token_from_string
@@ -418,16 +420,27 @@ def create_error_response(
status_code: int,
request: Request,
token: str | None = None,
code: str | int | None = None,
details: dict[str, object] | None = None,
) -> Response:
"""Create a standardized error response."""
"""Create a standardized error response.
``code`` is a stable, machine-readable classification (e.g.
``UPSTREAM_RATE_LIMIT``); when omitted it defaults to the HTTP status code
for backwards compatibility. ``details`` carries optional structured,
redaction-safe context.
"""
error_obj: dict[str, object] = {
"message": redact_org_ids(message),
"type": error_type,
"code": code if code is not None else status_code,
}
if details is not None:
error_obj["details"] = details
return Response(
content=json.dumps(
{
"error": {
"message": message,
"type": error_type,
"code": status_code,
},
"error": error_obj,
"request_id": getattr(request.state, "request_id", "unknown"),
}
),
@@ -435,3 +448,20 @@ def create_error_response(
media_type="application/json",
headers={"X-Cashu": token} if token else {},
)
def create_upstream_error_response(
error: UpstreamError,
request: Request,
fallback_status: int = 502,
) -> Response:
"""Build an error response from an :class:`UpstreamError`, preserving its
structured ``code``, ``details``, and original ``status_code``."""
return create_error_response(
"upstream_error",
str(error),
error.status_code or fallback_status,
request=request,
code=getattr(error, "code", None),
details=getattr(error, "details", None),
)
+38 -5
View File
@@ -94,7 +94,10 @@ def backfill_cache_pricing(model_id: str, pricing: Pricing) -> Pricing:
cache reads (DeepSeek hits are 10x cheaper) and undercharges Anthropic
cache writes (1.25x). litellm ships per-model USD rates keyed by the exact
OpenRouter id (deepseek/deepseek-chat) or by the bare model name
(gpt-4o, claude-sonnet-4-5), so both spellings are tried.
(gpt-4o, claude-sonnet-4-5), so both spellings are tried. litellm keys are
lowercase, but a generic upstream may report a mixed-case id
(``deepseek-ai/DeepSeek-V4-Flash``); an exact match is attempted first, then
a case-insensitive fallback so such ids still resolve.
Rates already present (e.g. provided by OpenRouter) are authoritative and
never overwritten. Unknown models are returned unchanged.
@@ -106,12 +109,26 @@ def backfill_cache_pricing(model_id: str, pricing: Pricing) -> Pricing:
import litellm
candidates = (model_id, model_id.split("/", 1)[-1])
info: dict | None = None
for key in (model_id, model_id.split("/", 1)[-1]):
for key in candidates:
candidate = litellm.model_cost.get(key)
if isinstance(candidate, dict):
info = candidate
break
if info is None:
# Case-insensitive fallback: a mixed-case upstream id (e.g.
# ``deepseek-ai/DeepSeek-V4-Flash``) won't match litellm's lowercase
# keys exactly. Build a lowercased index once and retry.
lowered = {c.lower() for c in candidates}
for key, candidate in litellm.model_cost.items():
if (
isinstance(key, str)
and key.lower() in lowered
and isinstance(candidate, dict)
):
info = candidate
break
if info is None:
return pricing
@@ -215,13 +232,29 @@ def _row_to_model(
)
top_provider_dict = json.loads(row.top_provider) if row.top_provider else None
if apply_provider_fee and isinstance(pricing, dict):
pricing = {k: float(v) * provider_fee for k, v in pricing.items()}
if isinstance(pricing, dict) and float(pricing.get("request", 0.0)) <= 0.0:
pricing["request"] = max(pricing.get("request", 0.0), 0.0)
parsed_pricing = Pricing.parse_obj(pricing)
# Fill missing cache-read/write rates from litellm's cost map BEFORE applying
# the provider fee, so they carry the same markup as every other component.
# DB-stored override pricing (e.g. generic providers) omits cache rates;
# without this, ``_row_to_model`` bills cache reads at the full input rate —
# the ``_apply_provider_fee_to_model`` path backfills, but the override path
# used for admin-configured providers did not.
#
# Key on ``forwarded_model_id`` (the actual upstream model name litellm
# prices) when set: an alias row (id="local-alias",
# forwarded_model_id="deepseek-v4-flash") would otherwise look up the alias
# and miss the cache rate.
pricing_model_id = getattr(row, "forwarded_model_id", None) or row.id
parsed_pricing = backfill_cache_pricing(pricing_model_id, parsed_pricing)
if apply_provider_fee:
parsed_pricing = Pricing.parse_obj(
{k: float(v) * provider_fee for k, v in parsed_pricing.dict().items()}
)
model = Model(
id=row.id,
name=row.name,
+7 -13
View File
@@ -24,6 +24,7 @@ from .payment.helpers import (
calculate_discounted_max_cost,
check_token_balance,
create_error_response,
create_upstream_error_response,
get_max_cost_for_model,
)
from .payment.models import Model
@@ -211,9 +212,7 @@ async def proxy(
e,
)
if i == len(all_upstreams) - 1:
last_error_response = create_error_response(
"upstream_error", str(e), 502, request=request
)
last_error_response = create_upstream_error_response(e, request)
continue
return last_error_response or create_error_response(
"upstream_error", "All upstreams failed", 502, request=request
@@ -283,11 +282,10 @@ async def proxy(
last_error = e
continue
if last_error is not None:
return create_upstream_error_response(last_error, request)
return create_error_response(
"upstream_error",
str(last_error) if last_error else "All upstreams failed",
502,
request=request,
"upstream_error", "All upstreams failed", 502, request=request
)
elif auth := headers.get("authorization", None):
@@ -343,9 +341,7 @@ async def proxy(
except UpstreamError as e:
logger.warning(f"Upstream {upstream.provider_type} failed (GET): {e}")
if i == len(upstreams) - 1:
last_error_response = create_error_response(
"upstream_error", str(e), 502, request=request
)
last_error_response = create_upstream_error_response(e, request)
continue
return last_error_response or create_error_response(
"upstream_error", "All upstreams failed", 502, request=request
@@ -525,9 +521,7 @@ async def proxy(
# If this was the last provider
if i == len(upstreams) - 1:
await revert_pay_for_request(key, session, max_cost_for_model)
return create_error_response(
"upstream_error", str(e), 502, request=request
)
return create_upstream_error_response(e, request)
# Otherwise loop continues to next provider
continue
+1 -1
View File
@@ -24,7 +24,7 @@ class AnthropicUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "AnthropicUpstreamProvider":
return cls(
+2
View File
@@ -97,6 +97,8 @@ async def _check_and_topup(row: UpstreamProviderRow) -> None:
# Instantiate provider and check balance
provider = RoutstrUpstreamProvider.from_db_row(row)
if provider is None:
return
balance = await provider.get_balance()
if balance is None:
+1 -1
View File
@@ -38,7 +38,7 @@ class AzureUpstreamProvider(BaseUpstreamProvider):
self.api_version = api_version
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "AzureUpstreamProvider | None":
if not provider_row.api_version:
+187 -80
View File
@@ -2,16 +2,16 @@ from __future__ import annotations
import asyncio
import json
import math
import traceback
import uuid
from collections.abc import AsyncGenerator, AsyncIterator, Iterator
from typing import Any, Mapping, cast
from typing import Any, Mapping, Self, cast
import httpx
from fastapi import BackgroundTasks, HTTPException, Request
from fastapi.responses import Response, StreamingResponse
from pydantic.v1 import BaseModel
from sqlmodel import select
from ..auth import adjust_payment_for_tokens
from ..core import get_logger
@@ -23,6 +23,7 @@ from ..core.db import (
store_cashu_transaction,
)
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
from ..payment.cost_calculation import (
CostData,
CostDataError,
@@ -39,7 +40,12 @@ from ..payment.models import (
list_models,
)
from ..payment.price import sats_usd_price
from ..wallet import recieve_token, send_token
from ..wallet import (
SPENT_TOKEN_CODES,
classify_redemption_error,
recieve_token,
send_token,
)
from . import messages_dispatch
from .cache_breakpoints import (
inject_anthropic_cache_breakpoints,
@@ -47,6 +53,7 @@ from .cache_breakpoints import (
)
from .count_tokens import count_tokens_locally
from .litellm_routing import detect_litellm_prefix
from .rate_limit import UPSTREAM_RATE_LIMIT, classify_rate_limit
logger = get_logger(__name__)
@@ -89,6 +96,11 @@ class BaseUpstreamProvider:
base_url: str
api_key: str
provider_fee: float = 1.05
# Primary key of the ``upstream_providers`` row this instance was built
# from. Set by ``from_db_row`` so a live provider can re-find its own row by
# stable identity instead of its rotatable ``api_key``. ``None`` for
# instances not sourced from a row.
db_id: int | None = None
_models_cache: list[Model] = []
_models_by_id: dict[str, Model] = {}
@@ -103,6 +115,7 @@ class BaseUpstreamProvider:
self.base_url = base_url
self.api_key = api_key
self.provider_fee = provider_fee
self.db_id = None
self._models_cache = []
self._models_by_id = {}
@@ -120,10 +133,13 @@ class BaseUpstreamProvider:
return detect_litellm_prefix(self.base_url)
@classmethod
def from_db_row(
cls, provider_row: "UpstreamProviderRow"
) -> "BaseUpstreamProvider | None":
"""Factory method to instantiate provider from database row.
def from_db_row(cls, provider_row: "UpstreamProviderRow") -> "Self | None":
"""Instantiate a provider from a database row, carrying its identity.
Construction itself is delegated to the ``_build_from_row`` hook (which
subclasses override to match their constructor); this wrapper stamps the
row's primary key onto the instance as ``db_id`` so the provider can
later re-find its own row by identity rather than by its ``api_key``.
Args:
provider_row: Database row containing provider configuration
@@ -131,6 +147,19 @@ class BaseUpstreamProvider:
Returns:
Instantiated provider or None if instantiation fails
"""
provider = cls._build_from_row(provider_row)
if provider is not None:
provider.db_id = provider_row.id
return provider
@classmethod
def _build_from_row(cls, provider_row: "UpstreamProviderRow") -> "Self | None":
"""Construct the provider instance from a row (no identity stamping).
Overridden by subclasses whose constructors differ from the base
``(base_url, api_key, provider_fee)`` shape. Callers should use
``from_db_row`` instead, which also attaches ``db_id``.
"""
return cls(
base_url=provider_row.base_url,
api_key=provider_row.api_key,
@@ -582,7 +611,7 @@ class BaseUpstreamProvider:
preview = body_bytes.decode("utf-8", errors="ignore").strip()
if preview:
message = preview[:500]
return message, upstream_code
return redact_org_ids(message), upstream_code
async def on_upstream_error_redirect(
self, status_code: int, error_message: str
@@ -625,10 +654,23 @@ class BaseUpstreamProvider:
body_bytes = b""
body_read_error = f"{type(exc).__name__}: {exc}"
# ``message`` is already redacted by ``_extract_upstream_error_message``;
# the raw body preview is redacted here before it reaches logs or the
# forwarded envelope so provider account identifiers never leak.
message, upstream_code = self._extract_upstream_error_message(body_bytes)
body_preview = body_bytes.decode("utf-8", errors="ignore").strip()[:500]
body_preview = redact_org_ids(
body_bytes.decode("utf-8", errors="ignore").strip()[:500]
)
is_json_body = _is_json_content_type(content_type)
# Classify upstream rate-limit failures into a stable, structured error.
rate_limit = classify_rate_limit(status_code, message, headers)
error_code: str | int = upstream_code or status_code
error_details: dict[str, object] | None = None
if rate_limit is not None:
error_code = UPSTREAM_RATE_LIMIT
error_details = rate_limit.as_details()
logger.warning(
"Upstream %s returned %s for model=%s path=%s: %s",
self.provider_type,
@@ -642,6 +684,7 @@ class BaseUpstreamProvider:
"model": model_id or "unknown",
"upstream_status": status_code,
"upstream_code": upstream_code,
"error_code": error_code,
"upstream_content_type": content_type,
"upstream_request_id": upstream_request_id,
"message_preview": message[:200],
@@ -676,13 +719,41 @@ class BaseUpstreamProvider:
):
headers.pop(header_name, None)
# Propagate a usable retry hint to the caller when the upstream supplied
# one but did not echo a ``Retry-After`` header. RFC 7231 delta-seconds
# is an integer, so round sub-second hints up to a usable ``1``.
if (
rate_limit is not None
and rate_limit.retry_after_seconds is not None
and "retry-after" not in {k.lower() for k in headers}
):
headers["Retry-After"] = str(max(1, math.ceil(rate_limit.retry_after_seconds)))
if is_json_body:
if not content_type:
headers.pop("content-type", None)
headers.pop("Content-Type", None)
media_type = content_type or None
# Re-serialise the body with organization IDs stripped. The narrow
# ``org-*`` regex preserves the surrounding JSON structure.
redacted_text = redact_org_ids(body_bytes.decode("utf-8", errors="ignore"))
redacted_body = redacted_text.encode()
# Surface the stable rate-limit classification on the forwarded
# body so callers can switch on ``error.code`` without parsing the
# provider-specific message. Fall back to the redacted bytes if the
# body is not a JSON object with an ``error`` mapping.
if rate_limit is not None:
try:
parsed = json.loads(redacted_text)
err = parsed.get("error") if isinstance(parsed, dict) else None
if isinstance(err, dict):
err["code"] = UPSTREAM_RATE_LIMIT
err["details"] = error_details
redacted_body = json.dumps(parsed).encode()
except (ValueError, AttributeError):
pass
return Response(
content=body_bytes,
content=redacted_body,
status_code=status_code,
headers=headers,
media_type=media_type,
@@ -693,15 +764,18 @@ class BaseUpstreamProvider:
for header_name in ("content-type", "Content-Type"):
headers.pop(header_name, None)
error_obj: dict[str, object] = {
"message": message or "Upstream returned a non-JSON error response",
"type": "upstream_error",
"code": error_code,
"upstream_status": status_code,
"upstream_content_type": content_type or None,
"upstream_body_preview": body_preview or None,
}
if error_details is not None:
error_obj["details"] = error_details
envelope = {
"error": {
"message": message or "Upstream returned a non-JSON error response",
"type": "upstream_error",
"code": upstream_code or status_code,
"upstream_status": status_code,
"upstream_content_type": content_type or None,
"upstream_body_preview": body_preview or None,
},
"error": error_obj,
"request_id": getattr(request.state, "request_id", None),
}
@@ -2521,9 +2595,16 @@ class BaseUpstreamProvider:
body_bytes = await response.aread()
except Exception:
body_bytes = b""
body_preview = body_bytes.decode(
"utf-8", errors="ignore"
).strip()[:500]
# Redact provider account identifiers before the body text
# reaches logs or the raised error.
body_preview = redact_org_ids(
body_bytes.decode("utf-8", errors="ignore").strip()[:500]
)
rate_limit = classify_rate_limit(
response.status_code,
body_preview,
dict(response.headers),
)
logger.error(
"Upstream %s returned %s for model=%s path=%s: %s",
self.provider_type,
@@ -2535,6 +2616,7 @@ class BaseUpstreamProvider:
"provider": self.provider_type,
"model": original_model_id or "unknown",
"status_code": response.status_code,
"error_code": rate_limit.code if rate_limit else None,
"reason_phrase": response.reason_phrase,
"path": path,
"body_preview": body_preview,
@@ -2547,6 +2629,8 @@ class BaseUpstreamProvider:
f"for model {original_model_id or 'unknown'}: "
f"{body_preview[:200] or '<empty>'}",
status_code=response.status_code,
code=rate_limit.code if rate_limit else None,
details=rate_limit.as_details() if rate_limit else None,
)
try:
@@ -2843,9 +2927,16 @@ class BaseUpstreamProvider:
body_bytes = await response.aread()
except Exception:
body_bytes = b""
body_preview = body_bytes.decode(
"utf-8", errors="ignore"
).strip()[:500]
# Redact provider account identifiers before the body text
# reaches logs or the raised error.
body_preview = redact_org_ids(
body_bytes.decode("utf-8", errors="ignore").strip()[:500]
)
rate_limit = classify_rate_limit(
response.status_code,
body_preview,
dict(response.headers),
)
logger.error(
"Upstream %s returned %s for model=%s path=%s: %s",
self.provider_type,
@@ -2857,6 +2948,7 @@ class BaseUpstreamProvider:
"provider": self.provider_type,
"model": original_model_id or "unknown",
"status_code": response.status_code,
"error_code": rate_limit.code if rate_limit else None,
"path": path,
"body_preview": body_preview,
},
@@ -2868,6 +2960,8 @@ class BaseUpstreamProvider:
f"for model {original_model_id or 'unknown'}: "
f"{body_preview[:200] or '<empty>'}",
status_code=response.status_code,
code=rate_limit.code if rate_limit else None,
details=rate_limit.as_details() if rate_limit else None,
)
try:
@@ -3892,9 +3986,19 @@ class BaseUpstreamProvider:
},
)
redeemed = False
try:
headers = dict(request.headers)
amount, unit, mint = await recieve_token(x_cashu_token)
# Reject a zero/negative redemption (empty/dust token, or a value
# fully consumed by fees) before marking the token redeemed, so it
# classifies as cashu_token_zero_value like the bearer/top-up paths
# rather than being forwarded as a free request.
if amount <= 0:
raise ValueError(
f"Redeemed token amount must be positive, got {amount} {unit}"
)
redeemed = True
headers = self.prepare_headers(dict(request.headers))
request_id = getattr(request.state, "request_id", None)
@@ -3938,40 +4042,37 @@ class BaseUpstreamProvider:
},
)
# Use same error handling as regular X-Cashu
if "already spent" in error_message.lower():
# Post-redemption the token is spent; a forwarding failure must not
# be reported as a retryable redemption error (see handle_x_cashu).
if redeemed:
return create_error_response(
"token_already_spent",
"The provided CASHU token has already been spent",
400,
"upstream_error",
"Payment succeeded but the upstream request failed",
502,
request=request,
token=x_cashu_token,
code="upstream_request_failed",
)
if "invalid token" in error_message.lower():
classified = classify_redemption_error(e)
if classified is None:
return create_error_response(
"invalid_token",
"The provided CASHU token is invalid",
400,
"api_error",
"Internal error during token redemption",
500,
request=request,
token=x_cashu_token,
code="internal_error",
)
if "mint error" in error_message.lower():
return create_error_response(
"mint_error",
f"CASHU mint error: {error_message}",
422,
request=request,
token=x_cashu_token,
)
error_type, status_code, message, error_code = classified
# Echo the token back only when it is still spendable, so clients
# can recover it; a spent/consumed token is never re-offered.
echo_token = None if error_code in SPENT_TOKEN_CODES else x_cashu_token
return create_error_response(
"cashu_error",
f"CASHU token processing failed: {error_message}",
400,
error_type,
message,
status_code,
request=request,
token=x_cashu_token,
token=echo_token,
code=error_code,
)
async def forward_x_cashu_responses_request(
@@ -4561,9 +4662,19 @@ class BaseUpstreamProvider:
},
)
redeemed = False
try:
headers = dict(request.headers)
amount, unit, mint = await recieve_token(x_cashu_token)
# Reject a zero/negative redemption (empty/dust token, or a value
# fully consumed by fees) before marking the token redeemed, so it
# classifies as cashu_token_zero_value like the bearer/top-up paths
# rather than being forwarded as a free request.
if amount <= 0:
raise ValueError(
f"Redeemed token amount must be positive, got {amount} {unit}"
)
redeemed = True
headers = self.prepare_headers(dict(request.headers))
request_id = getattr(request.state, "request_id", None)
@@ -4607,39 +4718,38 @@ class BaseUpstreamProvider:
},
)
if "already spent" in error_message.lower():
# Once redeemed the token is spent, so a later forwarding failure
# must not surface as a retryable mint_unreachable (spent-token retry
# bait). Redemption classification only applies while not redeemed.
if redeemed:
return create_error_response(
"token_already_spent",
"The provided CASHU token has already been spent",
400,
"upstream_error",
"Payment succeeded but the upstream request failed",
502,
request=request,
token=x_cashu_token,
code="upstream_request_failed",
)
if "invalid token" in error_message.lower():
classified = classify_redemption_error(e)
if classified is None:
return create_error_response(
"invalid_token",
"The provided CASHU token is invalid",
400,
"api_error",
"Internal error during token redemption",
500,
request=request,
token=x_cashu_token,
code="internal_error",
)
if "mint error" in error_message.lower():
return create_error_response(
"mint_error",
f"CASHU mint error: {error_message}",
422,
request=request,
token=x_cashu_token,
)
error_type, status_code, message, error_code = classified
# Echo the token back only when it is still spendable, so clients
# can recover it; a spent/consumed token is never re-offered.
echo_token = None if error_code in SPENT_TOKEN_CODES else x_cashu_token
return create_error_response(
"cashu_error",
f"CASHU token processing failed: {error_message}",
400,
error_type,
message,
status_code,
request=request,
token=x_cashu_token,
token=echo_token,
code=error_code,
)
def _apply_provider_fee_to_model(self, model: Model) -> Model:
@@ -4810,14 +4920,11 @@ class BaseUpstreamProvider:
"""Refresh the in-memory models cache from upstream API."""
try:
async with create_session() as session:
stmt = select(UpstreamProviderRow).where(
UpstreamProviderRow.base_url == self.base_url,
UpstreamProviderRow.api_key == self.api_key,
provider = (
await session.get(UpstreamProviderRow, self.db_id)
if self.db_id is not None
else None
)
result = await session.exec(stmt)
# .first() returns the object or None if not found
provider = result.first()
if not provider or not provider.id:
raise HTTPException(status_code=404, detail="Provider not found")
+16 -9
View File
@@ -3,10 +3,15 @@
litellm's bundled cost map does not yet ship ``deepseek-v4-flash`` /
``deepseek-v4-pro``. Without an entry, ``backfill_cache_pricing`` cannot find a
``cache_read_input_token_cost`` and cache reads fall back to the full input
rate a ~60% overcharge on cache hits (DeepSeek hits are ~0.2x input).
rate a large overcharge on cache hits (DeepSeek V4 hits are ~0.008-0.02x
input, i.e. cached tokens cost 50-120x less than regular input).
This module injects the missing entries into ``litellm.model_cost`` at startup
so the existing backfill path resolves them. Rates mirror the open upstream PR
so the existing backfill path resolves them. Rates mirror the canonical
``deepseek`` provider entries now in litellm's ``model_prices`` map
(``input_cost_per_token`` is the cache-*miss* rate;
``cache_read_input_token_cost`` is the cache-*hit* rate), sourced from
https://api-docs.deepseek.com/quick_start/pricing via
https://github.com/BerriAI/litellm/pull/26380 (issue
https://github.com/BerriAI/litellm/issues/30430).
@@ -22,21 +27,23 @@ from ..core import get_logger
logger = get_logger(__name__)
# USD per token. Source: BerriAI/litellm PR #26380.
# USD per token. Mirrors the canonical ``deepseek`` provider entries in
# litellm's model_prices map (source: DeepSeek API pricing docs). Keep these in
# sync with ``litellm.model_cost["deepseek/deepseek-v4-*"]``.
_DEEPSEEK_V4_RATES: dict[str, dict[str, float]] = {
"deepseek-v4-flash": {
"input_cost_per_token": 1.4e-07,
"output_cost_per_token": 2.8e-07,
"cache_read_input_token_cost": 2.8e-08,
"cache_read_input_token_cost": 2.8e-09,
"cache_creation_input_token_cost": 0.0,
"input_cost_per_token_cache_hit": 2.8e-08,
"input_cost_per_token_cache_hit": 2.8e-09,
},
"deepseek-v4-pro": {
"input_cost_per_token": 1.74e-06,
"output_cost_per_token": 3.48e-06,
"cache_read_input_token_cost": 1.4e-07,
"input_cost_per_token": 4.35e-07,
"output_cost_per_token": 8.7e-07,
"cache_read_input_token_cost": 3.625e-09,
"cache_creation_input_token_cost": 0.0,
"input_cost_per_token_cache_hit": 1.4e-07,
"input_cost_per_token_cache_hit": 3.625e-09,
},
}
+1 -1
View File
@@ -20,7 +20,7 @@ class FireworksUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "FireworksUpstreamProvider":
return cls(
+1 -1
View File
@@ -51,7 +51,7 @@ class GeminiUpstreamProvider(BaseUpstreamProvider):
return self._client
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "GeminiUpstreamProvider":
return cls(
+1 -1
View File
@@ -45,7 +45,7 @@ class GenericUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "GenericUpstreamProvider":
return cls(
+1 -1
View File
@@ -20,7 +20,7 @@ class GroqUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(cls, provider_row: "UpstreamProviderRow") -> "GroqUpstreamProvider":
def _build_from_row(cls, provider_row: "UpstreamProviderRow") -> "GroqUpstreamProvider":
return cls(
api_key=provider_row.api_key,
provider_fee=provider_row.provider_fee,
+24 -7
View File
@@ -12,6 +12,7 @@ from sqlmodel import select
from ..core import get_logger
from ..core.db import AsyncSession, ModelRow, UpstreamProviderRow, create_session
from ..core.provider_slugs import allocate_unique_provider_slug
from ..payment.models import Model
from .base import BaseUpstreamProvider
@@ -215,9 +216,6 @@ async def init_upstreams() -> list[BaseUpstreamProvider]:
provider = _instantiate_provider(provider_row)
if provider:
# Keep provider DB id on runtime instance so model mapping can
# bind DB overrides to the correct upstream.
setattr(provider, "db_id", provider_row.id)
await provider.refresh_models_cache()
logger.debug(
f"Initialized {provider_row.provider_type} provider",
@@ -250,6 +248,7 @@ async def _seed_providers_from_settings(
providers_to_add: list[UpstreamProviderRow] = []
seeded_provider_keys: set[tuple[str, str]] = set()
reserved_slugs: set[str] = set()
provider_classes_by_type = {
cls.provider_type: cls
@@ -279,8 +278,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, provider_type, reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type=provider_type,
base_url=base_url,
api_key=api_key,
@@ -299,8 +303,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, "ollama", reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type="ollama",
base_url=ollama_base_url,
api_key=ollama_api_key,
@@ -320,8 +329,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, "azure", reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type="azure",
base_url=base_url,
api_key=api_key,
@@ -342,8 +356,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, "custom", reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type="custom",
base_url=base_url,
api_key=api_key,
@@ -356,7 +375,7 @@ async def _seed_providers_from_settings(
session.add(provider)
logger.info(
f"Seeding {provider.provider_type} provider", # type: ignore[str-format]
extra={"base_url": provider.base_url},
extra={"base_url": provider.base_url, "slug": provider.slug},
)
@@ -391,9 +410,7 @@ def _instantiate_provider(
return provider
if provider_row.provider_type == "custom":
return BaseUpstreamProvider(
provider_row.base_url, provider_row.api_key, provider_row.provider_fee
)
return BaseUpstreamProvider.from_db_row(provider_row)
logger.error(
f"Unknown provider type: {provider_row.provider_type}",
+18 -4
View File
@@ -29,7 +29,9 @@ import litellm
from ..core import get_logger
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
from ..payment.models import Model
from .rate_limit import classify_rate_limit
logger = get_logger(__name__)
@@ -505,23 +507,33 @@ async def dispatch_anthropic_messages(
try:
result = await litellm.anthropic.messages.acreate(**kwargs)
except Exception as exc:
exc_message = getattr(exc, "message", None) or str(exc) or repr(exc)
raw_message = getattr(exc, "message", None) or str(exc) or repr(exc)
# Redact provider account identifiers before the message reaches logs
# or the surfaced error.
exc_message = redact_org_ids(raw_message)
exc_status = getattr(exc, "status_code", None)
exc_response = getattr(exc, "response", None)
response_text = None
if exc_response is not None:
try:
response_text = getattr(exc_response, "text", str(exc_response))
response_text = redact_org_ids(
getattr(exc_response, "text", str(exc_response))
)
except Exception:
response_text = "<unreadable>"
status_for_classify = exc_status if isinstance(exc_status, int) else 502
rate_limit = classify_rate_limit(
status_for_classify, exc_message, getattr(exc, "headers", None)
)
logger.error(
"litellm dispatch failed",
extra={
"error": exc_message,
"error_type": type(exc).__name__,
"status_code": exc_status,
"error_code": rate_limit.code if rate_limit else None,
"llm_provider": getattr(exc, "llm_provider", None),
"body": getattr(exc, "body", None),
"body": redact_org_ids(str(getattr(exc, "body", "") or "")) or None,
"response_text": response_text,
"model": litellm_model,
"api_base": base_url,
@@ -529,7 +541,9 @@ async def dispatch_anthropic_messages(
)
raise UpstreamError(
f"Upstream error via litellm: {exc_message}",
status_code=exc_status if isinstance(exc_status, int) else 502,
status_code=status_for_classify,
code=rate_limit.code if rate_limit else None,
details=rate_limit.as_details() if rate_limit else None,
) from exc
if not client_stream and hasattr(result, "__aiter__"):
+1 -1
View File
@@ -43,7 +43,7 @@ class OllamaUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "OllamaUpstreamProvider":
return cls(
+1 -1
View File
@@ -20,7 +20,7 @@ class OpenAIUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "OpenAIUpstreamProvider":
return cls(
+34 -1
View File
@@ -1,3 +1,4 @@
import json
from typing import TYPE_CHECKING
import httpx
@@ -18,6 +19,38 @@ class OpenRouterUpstreamProvider(BaseUpstreamProvider):
supports_anthropic_messages = True
litellm_provider_prefix = "openrouter/"
def prepare_request_body(
self, body: bytes | None, model_obj: Model
) -> bytes | None:
"""Set provider.require_parameters on tool-use requests.
Without it OpenRouter can route a tool call to an endpoint that doesn't
support function calling and 404 with "No endpoints found that support
tool use". We leave a client-supplied value untouched.
"""
body = super().prepare_request_body(body, model_obj)
if not body:
return body
try:
data = json.loads(body)
except json.JSONDecodeError:
return body
if not isinstance(data, dict) or not data.get("tools"):
return body
provider = data.get("provider")
if not isinstance(provider, dict):
provider = {}
if "require_parameters" in provider:
return body
provider["require_parameters"] = True
data["provider"] = provider
return json.dumps(data).encode()
def _apply_provider_field(self, response_json: object) -> None:
"""Stamp the ``provider`` field for OpenRouter responses.
@@ -57,7 +90,7 @@ class OpenRouterUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "OpenRouterUpstreamProvider":
return cls(
+1 -1
View File
@@ -23,7 +23,7 @@ class PerplexityUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "PerplexityUpstreamProvider":
return cls(
+5 -8
View File
@@ -46,7 +46,7 @@ class PPQAIUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "PPQAIUpstreamProvider":
return cls(
@@ -229,17 +229,14 @@ class PPQAIUpstreamProvider(BaseUpstreamProvider):
f"Disabling PPQ.AI provider ({self.base_url}) due to insufficient balance",
extra={"error": error_message},
)
from sqlmodel import select
from ..core.db import UpstreamProviderRow, create_session
async with create_session() as session:
statement = select(UpstreamProviderRow).where(
UpstreamProviderRow.base_url == self.base_url,
UpstreamProviderRow.api_key == self.api_key,
provider = (
await session.get(UpstreamProviderRow, self.db_id)
if self.db_id is not None
else None
)
result = await session.exec(statement)
provider = result.first()
if provider:
provider.enabled = False
+136
View File
@@ -0,0 +1,136 @@
"""Detection and parsing of upstream provider rate-limit errors.
Upstream OpenAI-compatible providers signal rate limits via HTTP 429 and/or a
human-readable message such as::
Rate limit reached for gpt-5.5-2026-04-23 (for limit gpt-5.5) in organization
org-XXXX on tokens per min (TPM): Limit 180000000, Used 180000000,
Requested 8929. Please try again in 2ms.
This module classifies those failures into a stable :data:`UPSTREAM_RATE_LIMIT`
code and extracts useful debugging fields. All retained text is redacted of
organization IDs first.
"""
from __future__ import annotations
import re
from dataclasses import asdict, dataclass
from ..core.redaction import redact_org_ids
# Stable error code callers can switch on to distinguish upstream rate limits
# from generic request failures. The literal value matches the identifier named
# in issue #555 ("UPSTREAM_RATE_LIMIT") so the public API contract is exact.
UPSTREAM_RATE_LIMIT = "UPSTREAM_RATE_LIMIT"
# Message fragments that indicate a rate-limit even when the status code is not
# 429 (some providers wrap it in a 400/500 envelope).
_RATE_LIMIT_MARKERS = (
"rate limit reached",
"rate_limit_exceeded",
"rate limit exceeded",
"too many requests",
)
_MODEL_RE = re.compile(r"Rate limit reached for ([^\s(]+)", re.IGNORECASE)
_LIMIT_NAME_RE = re.compile(r"\(for limit ([^)]+)\)", re.IGNORECASE)
_METRIC_RE = re.compile(r"on ([a-z ]+\((?:TPM|RPM|TPD|RPD|IPM)\))", re.IGNORECASE)
_LIMIT_RE = re.compile(r"Limit (\d+)", re.IGNORECASE)
_USED_RE = re.compile(r"Used (\d+)", re.IGNORECASE)
_REQUESTED_RE = re.compile(r"Requested (\d+)", re.IGNORECASE)
_RETRY_RE = re.compile(r"try again in ([\d.]+)\s*(ms|s)", re.IGNORECASE)
@dataclass
class RateLimitInfo:
"""Structured, redaction-safe view of an upstream rate-limit error."""
code: str
message: str
model: str | None = None
limit_name: str | None = None
metric: str | None = None
limit: int | None = None
used: int | None = None
requested: int | None = None
retry_after_seconds: float | None = None
def as_details(self) -> dict[str, object]:
"""Return a JSON-serialisable dict for embedding in an error envelope."""
return {k: v for k, v in asdict(self).items() if v is not None}
def _looks_like_rate_limit(status_code: int, message: str) -> bool:
if status_code == 429:
return True
lowered = message.lower()
return any(marker in lowered for marker in _RATE_LIMIT_MARKERS)
def _parse_retry_after_header(headers: dict[str, str] | None) -> float | None:
"""Parse a ``Retry-After`` header (delta-seconds form) into seconds."""
if not headers:
return None
raw = headers.get("retry-after") or headers.get("Retry-After")
if raw is None:
return None
try:
return float(str(raw).strip())
except (TypeError, ValueError):
return None
def _int_or_none(match: re.Match[str] | None) -> int | None:
if match is None:
return None
try:
return int(match.group(1))
except (TypeError, ValueError):
return None
def classify_rate_limit(
status_code: int,
message: str,
headers: dict[str, str] | None = None,
) -> RateLimitInfo | None:
"""Classify an upstream error as a rate-limit and extract its fields.
Args:
status_code: HTTP status code from the upstream response.
message: Upstream error message (may contain sensitive identifiers).
headers: Optional upstream response headers, used for ``Retry-After``.
Returns:
A :class:`RateLimitInfo` when the error is a rate-limit, else ``None``.
"""
message = message or ""
if not _looks_like_rate_limit(status_code, message):
return None
redacted = redact_org_ids(message)
model_match = _MODEL_RE.search(redacted)
metric_match = _METRIC_RE.search(redacted)
retry_after = _parse_retry_after_header(headers)
if retry_after is None:
retry_match = _RETRY_RE.search(redacted)
if retry_match is not None:
value = float(retry_match.group(1))
retry_after = value / 1000.0 if retry_match.group(2).lower() == "ms" else value
limit_name_match = _LIMIT_NAME_RE.search(redacted)
return RateLimitInfo(
code=UPSTREAM_RATE_LIMIT,
message=redacted,
model=model_match.group(1) if model_match else None,
limit_name=limit_name_match.group(1).strip() if limit_name_match else None,
metric=metric_match.group(1).strip() if metric_match else None,
limit=_int_or_none(_LIMIT_RE.search(redacted)),
used=_int_or_none(_USED_RE.search(redacted)),
requested=_int_or_none(_REQUESTED_RE.search(redacted)),
retry_after_seconds=retry_after,
)
+1 -1
View File
@@ -55,7 +55,7 @@ class RoutstrUpstreamProvider(BaseUpstreamProvider):
return path.lstrip("/")
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "RoutstrUpstreamProvider":
import json
+1 -1
View File
@@ -21,7 +21,7 @@ class XAIUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(cls, provider_row: "UpstreamProviderRow") -> "XAIUpstreamProvider":
def _build_from_row(cls, provider_row: "UpstreamProviderRow") -> "XAIUpstreamProvider":
return cls(
api_key=provider_row.api_key,
provider_fee=provider_row.provider_fee,
+361 -82
View File
@@ -1,8 +1,11 @@
import asyncio
import re
import socket
import time
import typing
from typing import TypedDict
import httpx
from cashu.core.base import Proof, Token
from cashu.core.mint_info import MintInfo as _CashuMintInfo
from cashu.wallet.helpers import deserialize_token_from_string
@@ -30,6 +33,149 @@ _CashuMintInfo.model_rebuild(force=True)
logger = get_logger(__name__)
class MintConnectionError(Exception):
"""The mint could not be reached (network transport failure).
Maps to a 503, not a 4xx: the token is fine, the mint is just unavailable.
"""
class TokenConsumedError(Exception):
"""A failure that happened AFTER the token's proofs were spent (melt
succeeded, or redemption already returned) e.g. minting on the primary
mint or the DB credit then failed.
Non-retryable: the same token will not work again. Seals the cause chain so
a transport error underneath is never re-surfaced as a retryable
mint_unreachable.
"""
# httpx base classes cover their subclasses. HTTPStatusError is excluded on
# purpose — that means the mint answered, just with an error status.
_TRANSPORT_EXC_TYPES: tuple[type[BaseException], ...] = (
httpx.NetworkError,
httpx.TimeoutException,
ConnectionError, # refused/reset/aborted
socket.gaierror, # DNS failure
asyncio.TimeoutError,
)
def is_mint_connection_error(error: BaseException) -> bool:
"""True if ``error`` (or anything in its cause/context chain) is a mint
transport failure. Walks the chain because some sites re-raise transport
errors wrapped in ValueError/MintConnectionError; matches on TYPE, not text.
"""
seen: set[int] = set()
current: BaseException | None = error
while current is not None and id(current) not in seen:
seen.add(id(current))
if isinstance(current, TokenConsumedError):
# Sealed: the token was already spent, so whatever transport error
# sits underneath must not make this look retryable.
return False
if isinstance(current, MintConnectionError):
return True
if isinstance(current, _TRANSPORT_EXC_TYPES):
return True
current = current.__cause__ or current.__context__
return False
# Redemption ``code`` values whose token is spent/consumed/unusable — the
# X-Cashu path must NOT echo the original token for these (echoing invites a
# retry with a token that can never succeed again).
SPENT_TOKEN_CODES: frozenset[str] = frozenset(
{
"cashu_token_already_spent",
"cashu_token_consumed",
"cashu_token_zero_value",
"internal_error",
}
)
def classify_redemption_error(
error: Exception,
) -> tuple[str, int, str, str] | None:
"""Map a token-redemption failure to ``(type, status, message, code)``.
Single source of truth for every endpoint that redeems a token (bearer,
X-Cashu, top-up) so the same failure yields the same taxonomy everywhere.
``type`` and ``code`` are stable client contract; ``message`` is sanitized
(raw error text stays in logs). Returns None for an unclassified internal
fault the caller emits a generic 500.
"""
if isinstance(error, TokenConsumedError):
return (
"token_consumed",
500,
"Token was redeemed but could not be credited; do not retry",
"cashu_token_consumed",
)
if is_mint_connection_error(error):
return (
"mint_unreachable",
503,
"Cashu mint is unreachable",
"cashu_mint_unreachable",
)
lowered = str(error).lower()
if "already spent" in lowered:
return (
"token_already_spent",
400,
"Cashu token already spent",
"cashu_token_already_spent",
)
if (
"insufficient" in lowered
or "melt fee" in lowered
or "exceed token amount" in lowered
or "estimate fees" in lowered
):
return (
"mint_error",
422,
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
)
if "failed to melt" in lowered:
return (
"mint_error",
422,
"Failed to swap token from foreign mint",
"cashu_foreign_mint_swap_failed",
)
if ("invalid" in lowered or "decode" in lowered) and "token" in lowered:
# Anchored to "token" so internal faults whose text merely contains
# "invalid"/"decode" fall through to the 500 branch, not a token error.
return (
"invalid_token",
400,
"Invalid Cashu token",
"invalid_cashu_token",
)
if "must be positive" in lowered or "yielded no value" in lowered:
# Redeemed to <= 0 (empty/dust token, or value fully consumed by fees).
# Consumed, so non-retryable, but its own code — not the generic bucket.
return (
"cashu_error",
400,
"Failed to redeem Cashu token: token yielded no value",
"cashu_token_zero_value",
)
if isinstance(error, ValueError):
return (
"cashu_error",
400,
"Failed to redeem Cashu token",
"cashu_token_redemption_failed",
)
return None
async def get_balance(unit: str) -> int:
wallet = await get_wallet(settings.primary_mint, unit)
return wallet.available_balance.amount
@@ -129,6 +275,72 @@ async def send_token(amount: int, unit: str, mint_url: str | None = None) -> str
return token
# A foreign mint's fee_reserve is a non-binding estimate (NUT-05): the mint may
# demand more when re-quoting or at melt execution. Instead of padding the
# estimate with a safety buffer (which strands the margin at the foreign mint
# on every swap), the swap retries with the amount recomputed from the fees the
# mint actually demands, up to this many attempts.
_MAX_SWAP_ATTEMPTS = 3
_MINT_ERROR_CODE_RE = re.compile(r"\(Code: (\d+)\)")
_MELT_SHORTFALL_RE = re.compile(r"Provided: (\d+), needed: (\d+)")
# Insufficient-melt-inputs failures differ across mint implementations. 11005 is
# the registered "Transaction is not balanced" code (cdk), specific enough to
# trust on the code alone. 11000 is nutshell's generic, unregistered
# TransactionError covering many unrelated failures, so it only counts as a fee
# shortfall alongside the "not enough inputs" detail text. With no code suffix at
# all, that same text is the only signal.
def _net_minted_amount(amount_msat: int, token_unit: str, fees: int) -> int:
"""
Convert the token value minus fees (given in the token unit) into an
amount in the primary mint's unit.
"""
fee_msat = fees * 1000 if token_unit == "sat" else fees
remaining_msat = amount_msat - fee_msat
if settings.primary_mint_unit == "sat":
return int(remaining_msat // 1000)
return int(remaining_msat)
def _melt_insufficient_shortfall(error: Exception) -> int | None:
"""
Classify a melt failure: return the observed shortfall (in the token unit)
when the mint rejected the inputs as insufficient, or None when the failure
is unrelated to fees and must not be retried (e.g. a Lightning payment
failure, where a smaller invoice would not help).
Cashu errors carry no structured amounts (NUT-00 defines only detail/code,
flattened to "Mint Error: <detail> (Code: <code>)" by cashu-py), so the
classification uses the code and the shortfall must be inferred: the
"Provided: X, needed: Y" amounts are nutshell-specific free text and only
refine the shortfall when present; otherwise shrink one unit at a time.
"""
message = str(error)
code_match = _MINT_ERROR_CODE_RE.search(message)
code = code_match.group(1) if code_match is not None else None
has_shortfall_text = "not enough inputs" in message.lower()
match code:
case "11005": # registered TransactionUnbalanced: trust the code
pass
case "11000" if has_shortfall_text: # generic nutshell error: needs the text
pass
case None if has_shortfall_text: # no code suffix: text is the only signal
pass
case _: # other codes, a bare 11000, or no signal: must not retry
return None
amounts = _MELT_SHORTFALL_RE.search(message)
if amounts is not None:
provided, needed = int(amounts.group(1)), int(amounts.group(2))
if needed > provided:
return needed - provided
return 1
async def _calculate_swap_amount(
amount_msat: int,
token_unit: str,
@@ -167,26 +379,18 @@ async def _calculate_swap_amount(
fee_reserve = dummy_melt_quote.fee_reserve
input_fees = token_wallet.get_fees_for_proofs(proofs)
if token_unit == "sat":
fee_msat = (fee_reserve + input_fees) * 1000
else:
fee_msat = fee_reserve + input_fees
amount_msat_after_fee = amount_msat - fee_msat
if settings.primary_mint_unit == "sat":
minted_amount = int(amount_msat_after_fee // 1000)
else:
minted_amount = int(amount_msat_after_fee)
total_fees = fee_reserve + input_fees
minted_amount = _net_minted_amount(amount_msat, token_unit, total_fees)
if minted_amount <= 0:
raise ValueError(f"Fees ({fee_reserve + input_fees} {token_unit}) exceed token amount")
raise ValueError(f"Fees ({total_fees} {token_unit}) exceed token amount")
logger.info(
"swap_to_primary_mint: fee estimation result",
extra={
"token_amount_sat": amount_msat // 1000,
"estimated_fee_sat": fee_msat // 1000,
"estimated_fee": total_fees,
"estimated_fee_unit": token_unit,
"input_fees": input_fees,
"minted_amount": minted_amount,
"minted_unit": settings.primary_mint_unit,
@@ -199,6 +403,8 @@ async def _calculate_swap_amount(
"swap_to_primary_mint: fee estimation failed",
extra={"error": str(e)},
)
if is_mint_connection_error(e):
raise MintConnectionError("Cashu mint is unreachable") from e
raise ValueError(f"Failed to estimate fees: {e}") from e
@@ -251,67 +457,123 @@ async def swap_to_primary_mint(
token_obj.proofs,
)
mint_quote = await primary_wallet.request_mint(minted_amount)
logger.info(
"swap_to_primary_mint: mint quote received",
extra={"mint_quote_id": mint_quote.quote},
)
# The estimate above is non-binding: the mint may demand a higher fee on the
# real quote or reject the melt outright. Retry the quote/melt cycle with the
# amount recomputed from the fees the mint actually demands.
observed_extra_fee = 0
attempt = 0
while True:
attempt += 1
mint_quote = await primary_wallet.request_mint(minted_amount)
logger.info(
"swap_to_primary_mint: mint quote received",
extra={"mint_quote_id": mint_quote.quote, "attempt": attempt},
)
melt_quote = await token_wallet.melt_quote(mint_quote.request)
input_fees = token_wallet.get_fees_for_proofs(token_obj.proofs)
total_needed = melt_quote.amount + melt_quote.fee_reserve + input_fees
logger.info(
"swap_to_primary_mint: melt quote received",
extra={
"melt_quote_id": melt_quote.quote,
"melt_amount": melt_quote.amount,
"melt_fee_reserve": melt_quote.fee_reserve,
"input_fees": input_fees,
"total_needed": total_needed,
"token_amount": token_amount,
},
)
if total_needed > token_amount:
logger.warning(
"swap_to_primary_mint: insufficient token amount for melt fees",
melt_quote = await token_wallet.melt_quote(mint_quote.request)
input_fees = token_wallet.get_fees_for_proofs(token_obj.proofs)
total_needed = melt_quote.amount + melt_quote.fee_reserve + input_fees
logger.info(
"swap_to_primary_mint: melt quote received",
extra={
"token_amount": token_amount,
"melt_quote_id": melt_quote.quote,
"melt_amount": melt_quote.amount,
"melt_fee_reserve": melt_quote.fee_reserve,
"input_fees": input_fees,
"total_needed": total_needed,
"shortfall": total_needed - token_amount,
"token_amount": token_amount,
"attempt": attempt,
},
)
raise ValueError(
f"Token amount ({token_amount} {token_obj.unit}) is insufficient to cover "
f"melt fees. Needed: {total_needed} {token_obj.unit} "
f"(amount: {melt_quote.amount} + fee: {melt_quote.fee_reserve} + input_fees: {input_fees})"
)
try:
_ = await token_wallet.melt(
proofs=token_obj.proofs,
invoice=mint_quote.request,
fee_reserve_sat=melt_quote.fee_reserve,
quote_id=melt_quote.quote,
)
except Exception as e:
logger.error(
"swap_to_primary_mint: melt failed",
extra={
"error": str(e),
"error_type": type(e).__name__,
"foreign_mint": token_obj.mint,
"token_amount": token_amount,
"melt_quote_id": melt_quote.quote,
"total_needed": total_needed,
},
)
raise ValueError(
f"Failed to melt token from foreign mint {token_obj.mint}: {e}"
) from e
if total_needed > token_amount:
recomputed = _net_minted_amount(
amount_msat,
token_obj.unit,
melt_quote.fee_reserve + input_fees + observed_extra_fee,
)
if attempt >= _MAX_SWAP_ATTEMPTS or recomputed <= 0:
logger.warning(
"swap_to_primary_mint: insufficient token amount for melt fees",
extra={
"token_amount": token_amount,
"melt_amount": melt_quote.amount,
"melt_fee_reserve": melt_quote.fee_reserve,
"input_fees": input_fees,
"total_needed": total_needed,
"shortfall": total_needed - token_amount,
"attempts": attempt,
},
)
raise ValueError(
f"Token amount ({token_amount} {token_obj.unit}) is insufficient to cover "
f"melt fees. Needed: {total_needed} {token_obj.unit} "
f"(amount: {melt_quote.amount} + fee: {melt_quote.fee_reserve} + input_fees: {input_fees})"
)
logger.warning(
"swap_to_primary_mint: melt quote exceeds token amount, retrying",
extra={
"total_needed": total_needed,
"token_amount": token_amount,
"retry_minted_amount": recomputed,
"attempt": attempt,
},
)
minted_amount = recomputed
continue
try:
_ = await token_wallet.melt(
proofs=token_obj.proofs,
invoice=mint_quote.request,
fee_reserve_sat=melt_quote.fee_reserve,
quote_id=melt_quote.quote,
)
except Exception as e:
# A down mint won't fix itself by retrying with a smaller amount.
if is_mint_connection_error(e):
logger.error(
"swap_to_primary_mint: melt failed — mint unreachable",
extra={"error": str(e), "foreign_mint": token_obj.mint},
)
raise MintConnectionError("Cashu mint is unreachable") from e
shortfall = _melt_insufficient_shortfall(e)
recomputed = 0
if shortfall is not None:
observed_extra_fee += shortfall
recomputed = _net_minted_amount(
amount_msat,
token_obj.unit,
melt_quote.fee_reserve + input_fees + observed_extra_fee,
)
if shortfall is None or attempt >= _MAX_SWAP_ATTEMPTS or recomputed <= 0:
logger.error(
"swap_to_primary_mint: melt failed",
extra={
"error": str(e),
"error_type": type(e).__name__,
"foreign_mint": token_obj.mint,
"token_amount": token_amount,
"melt_quote_id": melt_quote.quote,
"total_needed": total_needed,
"attempts": attempt,
},
)
raise ValueError(
f"Failed to melt token from foreign mint {token_obj.mint}: {e}"
) from e
logger.warning(
"swap_to_primary_mint: mint demanded more than quoted at melt, retrying",
extra={
"shortfall": shortfall,
"retry_minted_amount": recomputed,
"attempt": attempt,
},
)
minted_amount = recomputed
continue
break
logger.info(
"swap_to_primary_mint: melt succeeded, minting on primary",
@@ -350,21 +612,21 @@ async def swap_to_primary_mint(
# Recovery scan ran but did NOT restore the orphaned proofs
# (mint reports them as spent — they're stuck). Refuse to
# credit the API key balance for proofs we don't actually hold.
raise ValueError(
raise TokenConsumedError(
f"Swap recovery failed: mint signed outputs but proofs are "
f"unrecoverable (mint reports them spent). "
f"Expected {minted_amount}, recovered {balance_gained}. "
f"Local wallet DB ('.wallet/') state is corrupted — "
f"the counter for keyset is stuck at a bad index range."
)
except ValueError:
except TokenConsumedError:
raise
except Exception as recovery_err:
logger.error(
"swap_to_primary_mint: recovery failed",
extra={"error": str(recovery_err)},
)
raise ValueError(
raise TokenConsumedError(
f"Mint on primary failed and recovery unsuccessful: {e}"
) from e
else:
@@ -377,7 +639,10 @@ async def swap_to_primary_mint(
"mint_quote_id": mint_quote.quote,
},
)
raise
# Foreign proofs already melted (spent) — non-retryable.
raise TokenConsumedError(
"Mint on primary failed after successful melt"
) from e
logger.info(
"swap_to_primary_mint: completed successfully",
@@ -435,19 +700,33 @@ async def credit_balance(
extra={"old_balance": key.balance, "credit_amount": amount},
)
# Use atomic SQL UPDATE to prevent race conditions during concurrent topups
stmt = (
update(db.ApiKey)
.where(col(db.ApiKey.hashed_key) == key.hashed_key)
.values(balance=(db.ApiKey.balance) + amount)
)
result = await session.exec(stmt) # type: ignore[call-overload]
# If pruning removed this key after redemption, do not commit a no-op
# balance update and pretend the top-up succeeded.
if (getattr(result, "rowcount", 0) or 0) == 0:
raise ValueError("API key disappeared before credit could be recorded")
await session.commit()
await session.refresh(key)
# The token is already redeemed (spent) here, so any crediting failure
# is post-redemption and non-retryable — surface it as TokenConsumedError
# (a key that vanished mid-flight, or an unexpected DB fault), never a
# retryable/token-error taxonomy.
try:
# Atomic UPDATE to prevent race conditions during concurrent topups.
stmt = (
update(db.ApiKey)
.where(col(db.ApiKey.hashed_key) == key.hashed_key)
.values(balance=(db.ApiKey.balance) + amount)
)
result = await session.exec(stmt) # type: ignore[call-overload]
# If pruning removed this key after redemption, do not commit a no-op
# balance update and pretend the top-up succeeded.
if (getattr(result, "rowcount", 0) or 0) == 0:
raise TokenConsumedError(
"Token redeemed but the API key disappeared before the "
"credit could be recorded"
)
await session.commit()
await session.refresh(key)
except TokenConsumedError:
raise
except Exception as db_error:
raise TokenConsumedError(
"Token redeemed but crediting the balance failed"
) from db_error
logger.info(
"credit_balance: Balance updated successfully",
@@ -0,0 +1,146 @@
"""Regression tests for charging after stale reservation cleanup."""
import time
import uuid
from unittest.mock import patch
import pytest
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.db import ApiKey
from routstr.payment.cost_calculation import CostData
def _make_key(balance: int, reserved: int) -> ApiKey:
return ApiKey(
hashed_key=f"test_{uuid.uuid4().hex}",
balance=balance,
reserved_balance=reserved,
total_spent=0,
total_requests=1,
)
def _cost_data(total_msats: int) -> CostData:
return CostData(
base_msats=0,
input_msats=total_msats // 2,
output_msats=total_msats - total_msats // 2,
total_msats=total_msats,
total_usd=0.0,
input_tokens=100,
output_tokens=100,
)
@pytest.mark.asyncio
async def test_overrun_charges_after_reservation_swept(
integration_session: AsyncSession,
) -> None:
"""Overrun finalize must charge even when the reservation was already released."""
from routstr.auth import adjust_payment_for_tokens
deducted_max_cost = 990 # discounted reservation
actual_token_cost = 1000 # actual cost overruns the reservation
# Sweeper has zeroed reserved_balance but left balance untouched.
key = _make_key(balance=1000, reserved=0)
integration_session.add(key)
await integration_session.commit()
response_data = {
"model": "test-model",
"usage": {"prompt_tokens": 100, "completion_tokens": 100},
}
with patch(
"routstr.auth.calculate_cost",
return_value=_cost_data(actual_token_cost),
):
await adjust_payment_for_tokens(
key, response_data, integration_session, deducted_max_cost
)
await integration_session.refresh(key)
assert key.total_spent == actual_token_cost, (
f"Request was not billed (total_spent={key.total_spent}) — free response bug"
)
assert key.balance == 1000 - actual_token_cost, (
f"Balance not charged: {key.balance}"
)
assert key.balance >= 0
assert key.reserved_balance == 0
@pytest.mark.asyncio
async def test_free_response_path_closed_end_to_end(
integration_session: AsyncSession,
patched_db_engine: None,
) -> None:
"""A reservation released by the real sweeper must not yield a free response."""
from routstr.auth import adjust_payment_for_tokens, pay_for_request
from routstr.core.db import create_session, release_stale_reservations
deducted_max_cost = 990
actual_token_cost = 1000
key_hash = f"test_sweep_{uuid.uuid4().hex}"
async with create_session() as session:
session.add(
ApiKey(
hashed_key=key_hash,
balance=1000,
reserved_balance=0,
total_spent=0,
total_requests=0,
)
)
await session.commit()
# Reserve the request, then backdate reserved_at so the sweeper treats it as
# stale (simulates a stream that outlived stale_reservation_timeout_seconds).
async with create_session() as session:
key = await session.get(ApiKey, key_hash)
assert key is not None
await pay_for_request(key, deducted_max_cost, session)
await session.refresh(key)
assert key.reserved_balance == deducted_max_cost
key.reserved_at = int(time.time()) - 10_000
session.add(key)
await session.commit()
# Sweeper releases the stale reservation without charging.
async with create_session() as session:
released = await release_stale_reservations(session, max_age_seconds=300)
assert released == 1
async with create_session() as session:
key = await session.get(ApiKey, key_hash)
assert key is not None
assert key.reserved_balance == 0, "Precondition: sweeper zeroed the reservation"
response_data = {
"model": "test-model",
"usage": {"prompt_tokens": 100, "completion_tokens": 100},
}
with patch(
"routstr.auth.calculate_cost",
return_value=_cost_data(actual_token_cost),
):
await adjust_payment_for_tokens(
key, response_data, session, deducted_max_cost
)
async with create_session() as session:
final = await session.get(ApiKey, key_hash)
assert final is not None
assert final.total_spent == actual_token_cost, (
f"Free response: total_spent={final.total_spent}, expected {actual_token_cost}"
)
assert final.balance == 1000 - actual_token_cost, (
f"Balance not charged after sweep: {final.balance}"
)
assert final.balance >= 0
assert final.reserved_balance == 0
@@ -0,0 +1,128 @@
"""A live upstream provider resolves its OWN database row by stable identity
(its primary key), not by its mutable/secret ``api_key``.
Today ``from_db_row`` drops ``provider_row.id`` and the two self-referential
paths PPQ.AI's insufficient-balance self-disable and the base
``refresh_models_cache`` re-find their own row with
``WHERE base_url == self.base_url AND api_key == self.api_key``. That uses a
rotatable secret as a self-handle: the moment the row's key changes underneath a
live object (a rotation racing an in-flight request), the object can no longer
find itself. These tests pin the invariant that a provider looks itself up by
identity, so the lookup survives a key change (and, later, key encryption).
"""
from unittest.mock import AsyncMock, patch
import pytest
from routstr.core.db import UpstreamProviderRow
from routstr.upstream.ppqai import PPQAIUpstreamProvider
@pytest.mark.integration
@pytest.mark.asyncio
async def test_provider_object_carries_its_persistent_identity(
integration_session: object,
patched_db_engine: None,
) -> None:
"""``from_db_row`` gives the in-memory object its row's identity (``db_id``)."""
row = UpstreamProviderRow(
provider_type="ppqai",
base_url="https://api.ppq.ai",
api_key="sk-original",
enabled=True,
provider_fee=1.0,
)
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
await integration_session.refresh(row) # type: ignore[attr-defined]
provider = PPQAIUpstreamProvider.from_db_row(row)
assert provider is not None
assert provider.db_id == row.id
@pytest.mark.integration
@pytest.mark.asyncio
async def test_self_disable_targets_own_row_after_key_rotation(
integration_session: object,
patched_db_engine: None,
) -> None:
"""PPQ.AI self-disable must disable *its* row even after the key rotated.
RED (current): the object holds the pre-rotation key, so the
``(base_url, api_key)`` lookup misses the row the provider is never
disabled. GREEN: lookup by ``id`` finds it and disables it.
"""
row = UpstreamProviderRow(
provider_type="ppqai",
base_url="https://api.ppq.ai",
api_key="sk-original",
enabled=True,
provider_fee=1.0,
)
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
await integration_session.refresh(row) # type: ignore[attr-defined]
provider = PPQAIUpstreamProvider.from_db_row(row) # captures sk-original
assert provider is not None
# Key is rotated in the DB while `provider` is still live.
row.api_key = "sk-rotated"
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
with patch("routstr.proxy.reinitialize_upstreams", new=AsyncMock()):
await provider.on_upstream_error_redirect(402, "Insufficient balance")
await integration_session.refresh(row) # type: ignore[attr-defined]
assert row.enabled is False
@pytest.mark.integration
@pytest.mark.asyncio
async def test_refresh_models_cache_finds_own_row_after_key_rotation(
integration_session: object,
patched_db_engine: None,
) -> None:
"""``refresh_models_cache`` must resolve its own row after a key rotation.
``refresh_models_cache`` swallows every exception (it only logs), so the
observable proof it found its row is that it reaches ``list_models`` which
is called with the row's ``id`` only *after* the row is resolved. RED
(current): the stale-key ``(base_url, api_key)`` lookup returns nothing, the
method raises ``404`` internally and returns before ``list_models`` is ever
called. GREEN: lookup by ``id`` finds the row and ``list_models`` runs for
that ``id``.
"""
row = UpstreamProviderRow(
provider_type="ppqai",
base_url="https://api.ppq.ai",
api_key="sk-original",
enabled=True,
provider_fee=1.0,
)
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
await integration_session.refresh(row) # type: ignore[attr-defined]
row_id = row.id
provider = PPQAIUpstreamProvider.from_db_row(row) # captures sk-original
assert provider is not None
row.api_key = "sk-rotated"
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
list_models_mock = AsyncMock(return_value=[])
with (
patch.object(provider, "fetch_models", new=AsyncMock(return_value=[])),
patch("routstr.upstream.base.list_models", new=list_models_mock),
):
await provider.refresh_models_cache()
list_models_mock.assert_awaited_once()
assert list_models_mock.await_args is not None
assert list_models_mock.await_args.kwargs["upstream_id"] == row_id
+194
View File
@@ -0,0 +1,194 @@
"""
Integration tests for reactive swap fee retries via the wallet topup endpoint.
Foreign-mint tokens are swapped to the primary mint using the foreign mint's
melt quote, whose fee_reserve is a non-binding estimate (NUT-05): the mint may
demand more when re-quoting or at melt execution. These tests cover the
endpoint behaviour in those cases:
1. The mint demands one sat more at melt time than every quote reported
(the mint.cubabitcoin.org incident): the swap retries with a smaller
invoice and the topup succeeds, crediting the recomputed amount.
2. The real melt quote reports a higher fee_reserve than the estimate: the
swap re-quotes from the observed fee and the topup succeeds.
3. The mint escalates its fee demands on every attempt: the retry budget is
exhausted and the endpoint returns 400 with a clear error (never 500),
without ever executing a melt.
"""
from collections.abc import Callable
from unittest.mock import AsyncMock, Mock, patch
import pytest
from httpx import AsyncClient, Response
from routstr.core.settings import settings
# Captured at collection time, before the integration_app fixture replaces it
# with the testmint stub that bypasses swapping (see conftest.py).
from routstr.wallet import recieve_token as _real_recieve_token
PRIMARY_MINT = "http://primary:3338"
def _make_swap_mocks(
token_amount: int,
fee_reserves: list[int],
input_fees: int = 0,
mint_url: str = "http://foreign-mint:3338",
) -> tuple[Mock, Mock, Mock]:
"""Return (token, token_wallet, primary_wallet) mocks that act like a mint.
Mint quotes pass the requested amount through their ``request`` field and
melt quotes echo that amount back, so the mocks stay consistent for
whatever amounts the implementation requests. ``fee_reserves`` supplies the
fee_reserve of each successive melt quote (the first serves the estimation
pass); requesting more quotes than provided fails the test.
"""
mock_token = Mock()
mock_token.mint = mint_url
mock_token.unit = "sat"
mock_token.amount = token_amount
mock_token.keysets = ["keyset1"]
mock_token.proofs = [Mock(amount=token_amount)]
mock_token_wallet = Mock()
mock_token_wallet.load_mint = AsyncMock()
mock_token_wallet.load_proofs = AsyncMock()
mock_token_wallet.get_fees_for_proofs = Mock(return_value=input_fees)
mock_primary_wallet = Mock()
mock_primary_wallet.load_mint = AsyncMock()
mock_primary_wallet.load_proofs = AsyncMock()
mock_primary_wallet.available_balance = Mock(amount=0)
mock_primary_wallet.mint = AsyncMock(return_value=Mock())
fees = iter(fee_reserves)
def _next_fee() -> int:
try:
return next(fees)
except StopIteration:
raise AssertionError(
"more melt quotes requested than fee_reserves provided"
) from None
mock_primary_wallet.request_mint = AsyncMock(
side_effect=lambda amount: Mock(quote=f"mint_quote_{amount}", request=amount)
)
mock_token_wallet.melt_quote = AsyncMock(
side_effect=lambda invoice: Mock(
quote=f"melt_quote_{invoice}", amount=invoice, fee_reserve=_next_fee()
)
)
mock_token_wallet.melt = AsyncMock(return_value=Mock())
return mock_token, mock_token_wallet, mock_primary_wallet
def _wallet_router(primary_wallet: Mock, token_wallet: Mock) -> Callable[..., Mock]:
"""Route get_wallet calls to the primary or foreign wallet mock by URL."""
def fake_get_wallet(mint_url: str, unit: str = "sat", load: bool = True) -> Mock:
return primary_wallet if mint_url == PRIMARY_MINT else token_wallet
return fake_get_wallet
async def _post_topup(
client: AsyncClient,
mock_token: Mock,
token_wallet: Mock,
primary_wallet: Mock,
) -> Response:
"""POST /v1/wallet/topup with the swap layer mocked at the mint boundary.
The conftest's testmint stub for recieve_token is swapped back for the
real implementation so the request exercises the actual swap path.
"""
with patch("routstr.wallet.recieve_token", _real_recieve_token):
with patch(
"routstr.wallet.deserialize_token_from_string", return_value=mock_token
):
with patch(
"routstr.wallet.get_wallet",
side_effect=_wallet_router(primary_wallet, token_wallet),
):
with patch.object(settings, "primary_mint", PRIMARY_MINT):
with patch.object(settings, "primary_mint_unit", "sat"):
with patch.object(settings, "cashu_mints", [PRIMARY_MINT]):
return await client.post(
"/v1/wallet/topup",
params={"cashu_token": "cashuAtest_foreign_token"},
)
@pytest.mark.integration
@pytest.mark.asyncio
async def test_topup_retries_when_melt_demands_more_than_quoted(
authenticated_client: AsyncClient,
) -> None:
"""A 179-sat token where every quote reports fee_reserve=1 but the mint
rejects the first melt demanding 180. The retry shrinks the invoice to 177
and the topup credits 177 sats (177_000 msats)."""
mock_token, token_wallet, primary_wallet = _make_swap_mocks(
179, fee_reserves=[1, 1, 1], mint_url="http://mint.cubabitcoin.org"
)
token_wallet.melt.side_effect = [
Exception(
"Mint Error: not enough inputs provided for melt. "
"Provided: 179, needed: 180 (Code: 11000)"
),
Mock(),
]
response = await _post_topup(
authenticated_client, mock_token, token_wallet, primary_wallet
)
assert response.status_code == 200
assert response.json()["msats"] == 177_000
assert token_wallet.melt.call_count == 2
@pytest.mark.integration
@pytest.mark.asyncio
async def test_topup_retries_when_quote_fee_exceeds_estimate(
authenticated_client: AsyncClient,
) -> None:
"""A 1000-sat token estimated at fee 20, but the real quote demands 23.
The retry recomputes 1000 - 23 = 977, which fits, and the topup credits
977 sats (977_000 msats) with a single melt."""
mock_token, token_wallet, primary_wallet = _make_swap_mocks(
1000, fee_reserves=[20, 23, 23]
)
response = await _post_topup(
authenticated_client, mock_token, token_wallet, primary_wallet
)
assert response.status_code == 200
assert response.json()["msats"] == 977_000
assert token_wallet.melt.call_count == 1
@pytest.mark.integration
@pytest.mark.asyncio
async def test_topup_returns_422_when_retries_exhausted(
authenticated_client: AsyncClient,
) -> None:
"""A mint that escalates fee_reserve on every re-quote (1 → 10 → 25 → 50)
exhausts the retry budget: clean 422 mint_error/too-small taxonomy, melt
never executed."""
mock_token, token_wallet, primary_wallet = _make_swap_mocks(
1000, fee_reserves=[1, 10, 25, 50]
)
response = await _post_topup(
authenticated_client, mock_token, token_wallet, primary_wallet
)
assert response.status_code == 422
assert "too small to cover swap fees" in response.json()["detail"]
assert token_wallet.melt_quote.call_count == 4 # estimation + 3 attempts
token_wallet.melt.assert_not_called()
+13 -14
View File
@@ -12,10 +12,7 @@ from sqlmodel import select
from routstr.core.db import ApiKey
from .utils import (
CashuTokenGenerator,
ResponseValidator,
)
from .utils import ResponseValidator
@pytest.mark.integration
@@ -79,29 +76,31 @@ async def test_api_key_generation_invalid_token(
# Capture initial state
await db_snapshot.capture()
# Test various invalid tokens
invalid_tokens = [
CashuTokenGenerator.generate_invalid_token(), # Malformed token
"not-a-cashu-token", # Wrong format
"cashuA", # Empty token
"cashuA" + "x" * 1000, # Invalid base64
# Non-Cashu bearer values are invalid API keys (401). Malformed values that
# look like Cashu tokens use the shared Cashu taxonomy (400 invalid_token).
invalid_tokens: list[tuple[str, int, str | None]] = [
("not-a-cashu-token", 401, None),
("sk-not-a-real-api-key", 401, None),
("cashuA", 400, "invalid_cashu_token"),
("cashuA" + "x" * 1000, 400, "invalid_cashu_token"),
]
for invalid_token in invalid_tokens:
for invalid_token, expected_status, expected_code in invalid_tokens:
integration_client.headers["Authorization"] = f"Bearer {invalid_token}"
response = await integration_client.get("/v1/wallet/info")
# Should fail with 401
assert response.status_code == 401, (
assert response.status_code == expected_status, (
f"Token {invalid_token[:20]}... should be invalid"
)
# Validate error response
validator = ResponseValidator()
error_validation = validator.validate_error_response(
response, expected_status=401, expected_error_key="detail"
response, expected_status=expected_status, expected_error_key="detail"
)
assert error_validation["valid"]
if expected_code is not None:
assert response.json()["detail"]["error"]["code"] == expected_code
# Verify no database changes
diff = await db_snapshot.diff()
+8 -14
View File
@@ -14,6 +14,7 @@ from httpx import AsyncClient
from sqlmodel import select
from routstr.core.db import ApiKey, CashuTransaction
from routstr.wallet import MintConnectionError
@pytest.mark.integration
@@ -503,26 +504,19 @@ async def test_mint_unavailability_handling(
# The global mock in conftest.py is already in place,
# so we need to temporarily modify it
from unittest.mock import patch
raw_error = "Mint unavailable: Connection refused"
# Make the send_token method raise an exception
# Make the send_token method raise a typed mint connection exception.
with patch(
"routstr.balance.send_token",
side_effect=Exception("Mint unavailable: Connection refused"),
side_effect=MintConnectionError(raw_error),
):
# The exception should propagate as a 503 error (Service Unavailable)
# But we need to handle it properly
try:
response = await authenticated_client.post("/v1/wallet/refund")
# If we get here, check the status code
assert response.status_code == 503
assert "Mint service unavailable" in response.json()["detail"]
except Exception as e:
# If the exception propagates, that's also a failure scenario
assert "Mint unavailable" in str(e)
response = await authenticated_client.post("/v1/wallet/refund")
assert response.status_code == 503
assert response.json()["detail"] == "Mint service unavailable"
assert raw_error not in response.text
# Balance should remain unchanged (transaction should roll back)
# Note: Current implementation might not handle this perfectly
wallet_response = await authenticated_client.get("/v1/wallet/")
assert wallet_response.status_code == 200
assert wallet_response.json()["balance"] == 10_000_000
+235 -1
View File
@@ -1,8 +1,9 @@
import hashlib
from types import SimpleNamespace
from typing import AsyncGenerator
from typing import AsyncGenerator, cast
from unittest.mock import AsyncMock, patch
import httpx
import pytest
from fastapi import HTTPException
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
@@ -12,6 +13,19 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.auth import validate_bearer_key
from routstr.core.db import ApiKey
from routstr.wallet import MintConnectionError
def _value_error_wrapping_transport() -> ValueError:
"""A ValueError re-raised ``from`` a real httpx transport error, mirroring
``wallet.py`` wrapping a connection failure. The sanitized classifier must
still see the mint-unreachable signal through the ``__cause__`` chain."""
try:
raise httpx.ConnectError("All connection attempts failed")
except httpx.ConnectError as exc:
err = ValueError("Failed to estimate fees: connection failed")
err.__cause__ = exc
return err
def _make_engine() -> AsyncEngine:
@@ -57,3 +71,223 @@ async def test_failed_first_cashu_redemption_rolls_back_empty_api_key(
await validate_bearer_key(token, session)
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.parametrize(
("error", "expected_status", "expected_type", "expected_message", "expected_code"),
[
(
ValueError("Mint Error: Token already spent. (Code: 11001)"),
400,
"token_already_spent",
"Cashu token already spent",
"cashu_token_already_spent",
),
(
# Raw httpx transport error propagated unwrapped from cashu.
httpx.ConnectError("All connection attempts failed"),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
# Typed error raised by wallet.py at a wrap site.
MintConnectionError("connect to http://mint:3338 refused"),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
# ValueError wrapping the httpx error in its __cause__ chain.
_value_error_wrapping_transport(),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
# asyncio.TimeoutError is builtin TimeoutError on 3.11+.
TimeoutError("Timed out connecting to Cashu mint http://mint:3338"),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
ValueError(
"Token amount (5 sat) is insufficient to cover melt fees. "
"Needed: 7 sat (amount: 5 + fee: 1 + input_fees: 1)"
),
422,
"mint_error",
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
),
(
ValueError(
"Failed to estimate fees: Fees (7 sat) exceed token amount (5 sat)"
),
422,
"mint_error",
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
),
(
ValueError(
"Failed to melt token from foreign mint http://foreign:3338: boom"
),
422,
"mint_error",
"Failed to swap token from foreign mint",
"cashu_foreign_mint_swap_failed",
),
(
ValueError("could not decode token"),
400,
"invalid_token",
"Invalid Cashu token",
"invalid_cashu_token",
),
(
ValueError("some unexpected wallet condition"),
400,
"cashu_error",
"Failed to redeem Cashu token",
"cashu_token_redemption_failed",
),
(
ValueError("Redeemed token amount must be positive, got 0 msats"),
400,
"cashu_error",
"Failed to redeem Cashu token: token yielded no value",
"cashu_token_zero_value",
),
],
)
@pytest.mark.asyncio
async def test_redemption_failure_returns_sanitized_error(
session: AsyncSession,
error: Exception,
expected_status: int,
expected_type: str,
expected_message: str,
expected_code: str,
) -> None:
"""Redemption failures reuse the shared X-Cashu taxonomy (carried in
``type``), expose stable sanitized messages and granular machine-readable
``code`` values, and leave no orphan ApiKey row."""
token = "cashuAredemption_fails_with_specific_error"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(side_effect=error),
),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == expected_status
detail = cast(dict[str, dict[str, object]], exc_info.value.detail)
error_detail = detail["error"]
assert error_detail["type"] == expected_type
assert error_detail["code"] == expected_code
assert error_detail["message"] == expected_message
assert str(error) not in cast(str, error_detail["message"])
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.asyncio
async def test_unexpected_redemption_error_returns_internal_error(
session: AsyncSession,
) -> None:
"""Unexpected (non-wallet) failures surface as generic 500s without
leaking internal details, instead of masquerading as token errors."""
token = "cashuAredemption_fails_with_internal_error"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(side_effect=RuntimeError("db exploded at /var/lib/secret")),
),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == 500
detail = cast(dict[str, dict[str, str]], exc_info.value.detail)
error_detail = detail["error"]
assert error_detail["code"] == "internal_error"
assert "/var/lib/secret" not in error_detail["message"]
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.asyncio
async def test_internal_error_with_invalid_keyword_does_not_masquerade(
session: AsyncSession,
) -> None:
"""A non-wallet fault whose text merely contains "invalid" (but not
"token") must fall through to a generic 500, not a 401 token error.
Guards the anchored `"invalid"/"decode"` + `"token"` gate against stdlib/
driver strings like "Invalid isoformat string" leaking as token errors."""
token = "cashuAinternal_fault_mentions_invalid"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(
side_effect=RuntimeError("Invalid isoformat string: '2020-13-99'")
),
),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == 500
detail = cast(dict[str, dict[str, str]], exc_info.value.detail)
assert detail["error"]["code"] == "internal_error"
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.asyncio
async def test_malformed_cashu_token_returns_400_invalid_token(
session: AsyncSession,
) -> None:
"""A malformed 'cashu...' token that fails to decode maps to 400
invalid_cashu_token (shared taxonomy), not the generic 401 invalid_api_key."""
token = "cashuAthis_is_not_a_valid_token"
with patch(
"routstr.auth.deserialize_token_from_string",
side_effect=ValueError("unable to decode token: bad base64"),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == 400
detail = cast(dict[str, dict[str, str]], exc_info.value.detail)
assert detail["error"]["type"] == "invalid_token"
assert detail["error"]["code"] == "invalid_cashu_token"
# Raw decoder text must not leak to the client.
assert "base64" not in detail["error"]["message"]
+230 -3
View File
@@ -1,12 +1,13 @@
import json
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from fastapi.responses import JSONResponse
from routstr.balance import refund_wallet_endpoint
from routstr.balance import refund_wallet_endpoint, topup_wallet_endpoint
from routstr.core.db import ApiKey, CashuTransaction
from routstr.wallet import credit_balance
from routstr.wallet import MintConnectionError, credit_balance
def _make_cashu_tx(
@@ -339,7 +340,10 @@ async def test_apikey_refund_restores_balance_on_mint_failure() -> None:
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.send_token", AsyncMock(side_effect=Exception("mint down"))),
patch(
"routstr.balance.send_token",
AsyncMock(side_effect=MintConnectionError("raw mint outage detail")),
),
patch("routstr.balance.store_cashu_transaction", AsyncMock()),
patch("routstr.balance._refund_cache_get", AsyncMock(return_value=None)),
patch("routstr.balance._refund_cache_set", AsyncMock()),
@@ -353,10 +357,46 @@ async def test_apikey_refund_restores_balance_on_mint_failure() -> None:
)
assert exc_info.value.status_code == 503
assert exc_info.value.detail == "Mint service unavailable"
assert "raw mint outage detail" not in exc_info.value.detail
# Verify two exec calls: debit + restore
assert session.exec.await_count == 2
@pytest.mark.asyncio
async def test_apikey_refund_generic_failure_is_sanitized_500() -> None:
"""Unexpected send-side failures restore balance without leaking exception text."""
from fastapi import HTTPException
key = _make_api_key(balance=5000, refund_currency="sat")
raw_error = "database secret token raw-mint-response"
session = MagicMock()
session.get = AsyncMock(return_value=key)
session.exec = AsyncMock(side_effect=[_update_result(1), _update_result(1)])
session.commit = AsyncMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.send_token", AsyncMock(side_effect=RuntimeError(raw_error))),
patch("routstr.balance.store_cashu_transaction", AsyncMock()),
patch("routstr.balance._refund_cache_get", AsyncMock(return_value=None)),
patch("routstr.balance._refund_cache_set", AsyncMock()),
patch("routstr.balance.logger"),
):
with pytest.raises(HTTPException) as exc_info:
await refund_wallet_endpoint(
authorization="Bearer sk-testhash",
x_cashu=None,
session=session,
)
assert exc_info.value.status_code == 500
assert exc_info.value.detail == "Refund failed"
assert raw_error not in exc_info.value.detail
assert session.exec.await_count == 2
# ---------------------------------------------------------------------------
# no-create guarantee: fresh Cashu/unknown sk- tokens must not create API keys
# ---------------------------------------------------------------------------
@@ -401,3 +441,190 @@ async def test_refund_unknown_sk_bearer_returns_401() -> None:
assert exc_info.value.status_code == 401
session.get.assert_awaited_once()
# --- Topup redemption error taxonomy (POST /v1/wallet/topup) ------------------
@pytest.mark.asyncio
@pytest.mark.parametrize(
"error",
[
httpx.ConnectError("All connection attempts failed"),
MintConnectionError("connect to mint refused"),
TimeoutError("timed out connecting to mint"),
],
)
async def test_topup_mint_unreachable_returns_503(error: Exception) -> None:
"""A down mint must surface 503 (retryable), not 400 or 500 — the token is
fine, so the client should retry once the mint recovers."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.credit_balance", AsyncMock(side_effect=error)),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 503
assert exc_info.value.detail == "Cashu mint is unreachable"
@pytest.mark.asyncio
async def test_topup_already_spent_still_returns_400() -> None:
"""Regression: the mint-unreachable short-circuit must not swallow the
existing ValueError substring buckets."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(side_effect=ValueError("Token already spent")),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 400
assert exc_info.value.detail == "Cashu token already spent"
@pytest.mark.asyncio
async def test_topup_zero_value_returns_400_zero_value_message() -> None:
"""A dust/zero redemption maps to the documented zero-value message, not the
generic redemption-failed one."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(
side_effect=ValueError("Redeemed token amount must be positive, got 0 msats")
),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 400
assert exc_info.value.detail == "Failed to redeem Cashu token: token yielded no value"
@pytest.mark.asyncio
async def test_topup_token_consumed_returns_500() -> None:
"""A post-redemption crediting failure (token spent) is a non-retryable 500,
not a 4xx that invites a retry."""
from fastapi import HTTPException
from routstr.wallet import TokenConsumedError
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(side_effect=TokenConsumedError("credit failed")),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 500
assert exc_info.value.detail == (
"Token was redeemed but could not be credited; do not retry"
)
@pytest.mark.asyncio
@pytest.mark.parametrize(
("error", "expected_status", "expected_detail"),
[
(
ValueError(
"Failed to estimate fees: Fees (7 sat) exceed token amount (5 sat)"
),
422,
"Token value is too small to cover swap fees",
),
(
ValueError(
"Token amount (5 sat) is insufficient to cover melt fees."
),
422,
"Token value is too small to cover swap fees",
),
(
ValueError("Failed to melt token from foreign mint http://m: boom"),
422,
"Failed to swap token from foreign mint",
),
],
)
async def test_topup_fee_and_swap_failures_return_422(
error: Exception, expected_status: int, expected_detail: str
) -> None:
"""Fee/swap failures map to 422 (shared taxonomy), matching the bearer and
X-Cashu paths previously top-up flattened these to 400."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.credit_balance", AsyncMock(side_effect=error)),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == expected_status
assert exc_info.value.detail == expected_detail
@pytest.mark.asyncio
async def test_topup_unexpected_non_valueerror_returns_500() -> None:
"""A non-ValueError, non-transport fault is an internal error (500), not a
sanitized 400 the merged except must preserve this."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(side_effect=RuntimeError("db exploded")),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 500
assert exc_info.value.detail == "Internal server error"
+102
View File
@@ -81,6 +81,21 @@ def test_backfill_strips_vendor_prefix_for_litellm_lookup() -> None:
assert result.input_cache_read == expected
def test_backfill_case_insensitive_lookup() -> None:
"""A generic upstream may report a mixed-case id
(deepseek-ai/DeepSeek-V4-Flash); litellm keys are lowercase. The
case-insensitive fallback still resolves the cache rate."""
pricing = Pricing(prompt=1.4e-07, completion=2.8e-07)
result = backfill_cache_pricing("deepseek-ai/DeepSeek-V4-Flash", pricing)
expected = litellm.model_cost["deepseek-v4-flash"][
"cache_read_input_token_cost"
]
assert result.input_cache_read == expected
assert result.input_cache_read < pricing.prompt # sanity: it's a discount
def test_backfill_fills_cache_write_rate() -> None:
"""Anthropic cache writes cost more than input (1.25x); billing them at
the input rate undercharges. litellm carries the write rate."""
@@ -136,6 +151,93 @@ def test_provider_fee_applies_to_backfilled_cache_rates() -> None:
assert adjusted.pricing.prompt == pytest.approx(2.8e-07 * 2.0)
def test_row_to_model_backfills_cache_rate() -> None:
"""The DB-override path (admin-configured providers, e.g. a generic
upstream) stores pricing without cache rates. ``_row_to_model`` must
backfill them from litellm just like ``_apply_provider_fee_to_model``,
otherwise cache reads bill at the full input rate."""
import json
from routstr.core.db import ModelRow
from routstr.payment.models import _row_to_model
row = ModelRow(
id="deepseek-v4-flash",
name="deepseek-v4-flash",
created=0,
description="",
context_length=1000000,
architecture=json.dumps(
{
"modality": "text",
"input_modalities": ["text"],
"output_modalities": ["text"],
"tokenizer": "unknown",
"instruct_type": None,
}
),
# Stored pricing omits input_cache_read (generic provider never sets it).
pricing=json.dumps({"prompt": 1.4e-07, "completion": 2.8e-07}),
enabled=True,
upstream_provider_id=1,
)
with patch(
"routstr.payment.models.sats_usd_price", return_value=5.0e-5
):
model = _row_to_model(row, apply_provider_fee=True, provider_fee=1.0)
litellm_read = litellm.model_cost["deepseek-v4-flash"][
"cache_read_input_token_cost"
]
assert model.pricing.input_cache_read == pytest.approx(litellm_read)
assert model.pricing.input_cache_read < model.pricing.prompt # a discount
assert model.sats_pricing is not None
assert model.sats_pricing.input_cache_read > 0
def test_row_to_model_backfills_via_forwarded_model_id() -> None:
"""An alias row (id != forwarded_model_id) must backfill cache rates from
the *forwarded* model name the real upstream model litellm prices
not the alias id, which litellm doesn't know."""
import json
from routstr.core.db import ModelRow
from routstr.payment.models import _row_to_model
row = ModelRow(
id="local-alias", # litellm has no such key
name="local-alias",
created=0,
description="",
context_length=1000000,
architecture=json.dumps(
{
"modality": "text",
"input_modalities": ["text"],
"output_modalities": ["text"],
"tokenizer": "unknown",
"instruct_type": None,
}
),
pricing=json.dumps({"prompt": 1.4e-07, "completion": 2.8e-07}),
enabled=True,
upstream_provider_id=1,
forwarded_model_id="deepseek-v4-flash",
)
with patch(
"routstr.payment.models.sats_usd_price", return_value=5.0e-5
):
model = _row_to_model(row, apply_provider_fee=True, provider_fee=1.0)
litellm_read = litellm.model_cost["deepseek-v4-flash"][
"cache_read_input_token_cost"
]
assert model.pricing.input_cache_read == pytest.approx(litellm_read)
assert model.pricing.input_cache_read < model.pricing.prompt
# ============================================================================
# calculate_cost — cached tokens billed at cache rates
# ============================================================================
@@ -11,6 +11,7 @@ import os
from typing import Any, AsyncIterator
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from fastapi.responses import Response, StreamingResponse
@@ -21,6 +22,7 @@ from routstr.core.db import ApiKey # noqa: E402
from routstr.payment.cost_calculation import CostData # noqa: E402
from routstr.payment.models import Architecture, Model, Pricing # noqa: E402
from routstr.upstream.base import BaseUpstreamProvider # noqa: E402
from routstr.wallet import MintConnectionError, TokenConsumedError # noqa: E402
# ---------------------------------------------------------------------------
# Fixtures
@@ -1308,3 +1310,320 @@ async def test_dispatch_uses_url_detected_prefix_for_fireworks_custom_row() -> N
"fireworks_ai/accounts/fireworks/models/glm-5"
)
assert captured_kwargs["api_base"] == "https://api.fireworks.ai/inference/v1"
# ---------------------------------------------------------------------------
# X-Cashu redemption error taxonomy (unreachable mint + string codes)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
@pytest.mark.parametrize(
"handler_name",
["handle_x_cashu", "handle_x_cashu_responses"],
)
@pytest.mark.parametrize(
"error",
[
httpx.ConnectError("All connection attempts failed"),
MintConnectionError("Cashu mint is unreachable"),
TimeoutError("timed out connecting to mint"),
],
)
async def test_x_cashu_mint_unreachable_returns_503(
handler_name: str, error: Exception
) -> None:
"""Both X-Cashu entrypoints classify a down mint as 503 mint_unreachable,
not a generic 400 cashu_error."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token", new=AsyncMock(side_effect=error)
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 503
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "mint_unreachable"
assert body["error"]["message"] == "Cashu mint is unreachable"
assert body["error"]["code"] == "cashu_mint_unreachable"
if str(error) != body["error"]["message"]:
assert str(error) not in body["error"]["message"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
"handler_name",
["handle_x_cashu", "handle_x_cashu_responses"],
)
@pytest.mark.parametrize(
(
"error",
"expected_status",
"expected_type",
"expected_message",
"expected_code",
),
[
(
ValueError("Mint Error: Token already spent"),
400,
"token_already_spent",
"Cashu token already spent",
"cashu_token_already_spent",
),
(
ValueError("invalid token: could not decode"),
400,
"invalid_token",
"Invalid Cashu token",
"invalid_cashu_token",
),
(
# Fee/swap failures now map to a granular 422 on the X-Cashu path,
# matching the bearer path (previously flattened to 400).
ValueError(
"Failed to estimate fees: Fees (7 sat) exceed token amount (5 sat)"
),
422,
"mint_error",
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
),
(
ValueError("Failed to melt token from foreign mint http://m: boom"),
422,
"mint_error",
"Failed to swap token from foreign mint",
"cashu_foreign_mint_swap_failed",
),
(
ValueError("some unexpected wallet condition"),
400,
"cashu_error",
"Failed to redeem Cashu token",
"cashu_token_redemption_failed",
),
(
# Non-ValueError faults are internal errors (500), not token errors.
RuntimeError("db exploded"),
500,
"api_error",
"Internal error during token redemption",
"internal_error",
),
],
)
async def test_x_cashu_error_code_is_stable_string(
handler_name: str,
error: Exception,
expected_status: int,
expected_type: str,
expected_message: str,
expected_code: str,
) -> None:
"""X-Cashu emits a stable string ``code`` on every branch, matching the
bearer path's taxonomy instead of an int HTTP status."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token", new=AsyncMock(side_effect=error)
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == expected_status
body = json.loads(bytes(response.body))
assert body["error"]["type"] == expected_type
assert body["error"]["message"] == expected_message
assert body["error"]["code"] == expected_code
assert isinstance(body["error"]["code"], str)
if str(error) != expected_message:
assert str(error) not in body["error"]["message"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
("handler_name", "forward_attr"),
[
("handle_x_cashu", "forward_x_cashu_request"),
("handle_x_cashu_responses", "forward_x_cashu_responses_request"),
],
)
async def test_x_cashu_transport_error_after_redemption_is_not_retryable(
handler_name: str, forward_attr: str
) -> None:
"""A transport failure while forwarding (after the token is spent) maps to
502 upstream_error, never a retryable cashu_mint_unreachable."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with (
patch(
"routstr.upstream.base.recieve_token",
new=AsyncMock(return_value=(5_000, "sat", "https://mint")),
),
patch("routstr.upstream.base.store_cashu_transaction", new=AsyncMock()),
patch.object(
provider,
forward_attr,
new=AsyncMock(side_effect=httpx.ConnectError("upstream down")),
),
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 502
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "upstream_error"
assert body["error"]["code"] != "cashu_mint_unreachable"
@pytest.mark.asyncio
@pytest.mark.parametrize(
"handler_name",
["handle_x_cashu", "handle_x_cashu_responses"],
)
async def test_x_cashu_token_consumed_returns_500_and_no_echo(
handler_name: str,
) -> None:
"""A post-redemption failure (token spent, crediting/minting failed) is a
non-retryable 500 token_consumed and must NOT echo the spent token back."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token",
new=AsyncMock(side_effect=TokenConsumedError("credit failed")),
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 500
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "token_consumed"
assert body["error"]["code"] == "cashu_token_consumed"
assert "X-Cashu" not in response.headers
@pytest.mark.asyncio
@pytest.mark.parametrize(
("handler_name", "error", "echoed"),
[
# Spent token: must NOT be echoed.
("handle_x_cashu", ValueError("Token already spent"), False),
("handle_x_cashu_responses", ValueError("Token already spent"), False),
# Unspent but unreachable mint: echo so the client can retry the token.
("handle_x_cashu", MintConnectionError("mint down"), True),
("handle_x_cashu_responses", MintConnectionError("mint down"), True),
# Consumed token (post-redemption): must NOT be echoed.
("handle_x_cashu", TokenConsumedError("credit failed"), False),
("handle_x_cashu_responses", TokenConsumedError("credit failed"), False),
],
)
async def test_x_cashu_echoes_token_only_when_recoverable(
handler_name: str, error: Exception, echoed: bool
) -> None:
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token", new=AsyncMock(side_effect=error)
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
if echoed:
assert response.headers.get("X-Cashu") == "cashuAtoken"
else:
assert "X-Cashu" not in response.headers
@pytest.mark.asyncio
@pytest.mark.parametrize(
("handler_name", "forward_attr"),
[
("handle_x_cashu", "forward_x_cashu_request"),
("handle_x_cashu_responses", "forward_x_cashu_responses_request"),
],
)
@pytest.mark.parametrize("amount", [0, -5])
async def test_x_cashu_zero_value_rejected_not_forwarded(
handler_name: str, forward_attr: str, amount: int
) -> None:
"""A token that redeems to <= 0 must be rejected as cashu_token_zero_value
(400) and NEVER forwarded as a free request the X-Cashu path lacked the
guard that credit_balance has."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with (
patch(
"routstr.upstream.base.recieve_token",
new=AsyncMock(return_value=(amount, "sat", "https://mint")),
),
patch("routstr.upstream.base.store_cashu_transaction", new=AsyncMock()),
patch.object(
provider,
forward_attr,
new=AsyncMock(side_effect=AssertionError("must not forward a zero-value token")),
),
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 400
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "cashu_error"
assert (
body["error"]["message"]
== "Failed to redeem Cashu token: token yielded no value"
)
assert body["error"]["code"] == "cashu_token_zero_value"
# Spent-to-zero token must not be echoed back for retry.
assert "X-Cashu" not in response.headers
@@ -0,0 +1,55 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sqlalchemy as sa
_MIGRATION_PATH = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "versions"
/ "c6d7e8f9a0b1_add_slug_to_upstream_providers.py"
)
_spec = importlib.util.spec_from_file_location("provider_slug_migration", _MIGRATION_PATH)
assert _spec is not None and _spec.loader is not None
migration = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(migration)
def test_slug_migration_backfill_uses_api_safe_deterministic_slugs() -> None:
engine = sa.create_engine("sqlite:///:memory:")
with engine.begin() as conn:
conn.execute(
sa.text(
"CREATE TABLE upstream_providers ("
"id INTEGER PRIMARY KEY, "
"provider_type VARCHAR NOT NULL, "
"slug VARCHAR NULL"
")"
)
)
conn.execute(
sa.text(
"INSERT INTO upstream_providers (id, provider_type, slug) VALUES "
"(1, 'OpenAI Compatible', NULL), "
"(2, 'OpenAI Compatible', ''), "
"(3, '123', NULL), "
"(4, 'x', NULL), "
"(5, 'anthropic', 'anthropic')"
)
)
migration._backfill_provider_slugs(conn)
rows = conn.execute(
sa.text("SELECT id, slug FROM upstream_providers ORDER BY id")
).all()
assert rows == [
(1, "openai-compatible"),
(2, "openai-compatible-2"),
(3, "provider-123"),
(4, "x-provider"),
(5, "anthropic"),
]
+144
View File
@@ -0,0 +1,144 @@
from __future__ import annotations
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel import SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.admin import _get_upstream_provider_by_ref
from routstr.core.db import UpstreamProviderRow
from routstr.core.provider_slugs import (
allocate_unique_provider_slug,
provider_slug_base,
)
from routstr.upstream.helpers import _seed_providers_from_settings
@pytest.mark.asyncio
async def test_allocate_unique_provider_slug_is_deterministic_with_suffixes() -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
async with AsyncSession(engine) as session:
session.add(
UpstreamProviderRow(
slug="openai",
provider_type="openai",
base_url="https://api.openai.com/v1",
api_key="key-1",
)
)
await session.commit()
assert await allocate_unique_provider_slug(session, "openai") == "openai-2"
assert (
await allocate_unique_provider_slug(session, "openai", {"openai-2"})
== "openai-3"
)
await engine.dispose()
def test_provider_slug_base_sanitizes_provider_type() -> None:
assert provider_slug_base("OpenAI Compatible") == "openai-compatible"
assert provider_slug_base("!!!") == "provider"
assert provider_slug_base("AI") == "ai-provider"
assert provider_slug_base("123") == "provider-123"
@pytest.mark.asyncio
async def test_provider_ref_lookup_accepts_existing_numeric_ids_and_slugs() -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
async with AsyncSession(engine) as session:
provider = UpstreamProviderRow(
slug="openai",
provider_type="openai",
base_url="https://api.openai.com/v1",
api_key="key-1",
)
session.add(provider)
await session.commit()
await session.refresh(provider)
by_id = await _get_upstream_provider_by_ref(session, str(provider.id))
by_slug = await _get_upstream_provider_by_ref(session, "openai")
assert by_id.id == provider.id
assert by_slug.id == provider.id
await engine.dispose()
@pytest.mark.asyncio
async def test_seed_providers_from_settings_sets_deterministic_slug(
monkeypatch: pytest.MonkeyPatch,
) -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
monkeypatch.setenv("OPENAI_API_KEY", "seeded-openai-key")
class SettingsStub:
chat_completions_api_version: str | None = None
upstream_base_url: str | None = None
upstream_api_key: str = ""
async with AsyncSession(engine) as session:
await _seed_providers_from_settings(session, SettingsStub()) # type: ignore[arg-type]
await session.commit()
result = await session.exec(select(UpstreamProviderRow))
providers: list[UpstreamProviderRow] = list(result.all())
assert [(p.provider_type, p.slug) for p in providers] == [("openai", "openai")]
await engine.dispose()
@pytest.mark.asyncio
async def test_seed_providers_from_settings_keeps_slug_stable_on_reseed(
monkeypatch: pytest.MonkeyPatch,
) -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
monkeypatch.setenv("OPENAI_API_KEY", "seeded-openai-key")
class SettingsStub:
chat_completions_api_version: str | None = None
upstream_base_url: str | None = None
upstream_api_key: str = ""
async with AsyncSession(engine) as session:
session.add(
UpstreamProviderRow(
slug="openai",
provider_type="openai",
base_url="https://example.invalid/v1",
api_key="other-key",
)
)
await session.commit()
await _seed_providers_from_settings(session, SettingsStub()) # type: ignore[arg-type]
await session.commit()
await _seed_providers_from_settings(session, SettingsStub()) # type: ignore[arg-type]
await session.commit()
result = await session.exec(
select(UpstreamProviderRow).order_by(UpstreamProviderRow.slug)
)
providers: list[UpstreamProviderRow] = list(result.all())
assert [(p.provider_type, p.slug) for p in providers] == [
("openai", "openai"),
("openai", "openai-2"),
]
await engine.dispose()
+95
View File
@@ -0,0 +1,95 @@
import json
import os
os.environ.setdefault("UPSTREAM_BASE_URL", "http://test")
os.environ.setdefault("UPSTREAM_API_KEY", "test")
os.environ.setdefault("LIGHTNING_ADDRESS", "test@stm.to")
from routstr.upstream import GenericUpstreamProvider
from routstr.upstream.openrouter import OpenRouterUpstreamProvider
def _model(model_id: str = "openai/gpt-4o"): # type: ignore[no-untyped-def]
from routstr.payment.models import Architecture, Model, Pricing
return Model(
id=model_id,
name=model_id,
created=0,
description="",
context_length=128000,
architecture=Architecture(
modality="text->text",
input_modalities=["text"],
output_modalities=["text"],
tokenizer="GPT",
instruct_type=None,
),
pricing=Pricing(prompt=0.0, completion=0.0),
)
def _tool_body() -> dict:
return {
"model": "openai/gpt-4o",
"messages": [{"role": "user", "content": "What's the weather?"}],
"tools": [
{
"type": "function",
"function": {"name": "get_weather", "parameters": {}},
}
],
}
def _prepare(provider, body: dict) -> dict: # type: ignore[no-untyped-def]
out = provider.prepare_request_body(json.dumps(body).encode(), _model())
assert out is not None
return json.loads(out)
def test_injects_require_parameters_for_tool_request() -> None:
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), _tool_body())
assert data["provider"]["require_parameters"] is True
def test_generic_provider_on_openrouter_url_is_left_alone() -> None:
# Only OpenRouterUpstreamProvider injects; a generic provider pointed at the
# same base URL doesn't.
provider = GenericUpstreamProvider(base_url="https://openrouter.ai/api/v1")
data = _prepare(provider, _tool_body())
assert "provider" not in data
def test_no_injection_without_tools() -> None:
body = {"model": "openai/gpt-4o", "messages": [{"role": "user", "content": "hi"}]}
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert "provider" not in data
def test_empty_tools_list_does_not_inject() -> None:
body = _tool_body()
body["tools"] = []
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert "provider" not in data
def test_direct_provider_does_not_inject() -> None:
provider = GenericUpstreamProvider(base_url="https://api.openai.com/v1")
data = _prepare(provider, _tool_body())
assert "provider" not in data
def test_keeps_client_set_require_parameters() -> None:
body = _tool_body()
body["provider"] = {"require_parameters": False}
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert data["provider"]["require_parameters"] is False
def test_preserves_other_provider_fields() -> None:
body = _tool_body()
body["provider"] = {"order": ["openai", "azure"]}
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert data["provider"]["order"] == ["openai", "azure"]
assert data["provider"]["require_parameters"] is True
+396
View File
@@ -0,0 +1,396 @@
"""Tests for upstream rate-limit detection, classification, and org-ID redaction.
Covers issue #555: upstream OpenAI-compatible providers return rate-limit
errors that embed a sensitive organization ID. The proxy must classify these
distinctly (``UPSTREAM_RATE_LIMIT``), preserve useful debugging fields, and
never emit a raw ``org-*`` identifier in logs, errors, or returned bodies.
"""
from __future__ import annotations
import json
from typing import Any
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import httpx
import pytest
from routstr.core.redaction import redact_org_ids
from routstr.upstream.base import BaseUpstreamProvider
from routstr.upstream.rate_limit import (
UPSTREAM_RATE_LIMIT,
RateLimitInfo,
classify_rate_limit,
)
# The exact scenario from the issue, with a realistic (fake) org identifier.
RAW_ORG_ID = "org-abc123XYZ456def"
RATE_LIMIT_MESSAGE = (
f"Rate limit reached for gpt-5.5-2026-04-23 (for limit gpt-5.5) in "
f"organization {RAW_ORG_ID} on tokens per min (TPM): Limit 180000000, "
f"Used 180000000, Requested 8929. Please try again in 2ms. Visit "
f"https://platform.openai.com/account/rate-limits to learn more."
)
def _make_request(request_id: str = "req-123") -> Mock:
request = Mock(spec=["method", "state"])
request.method = "POST"
request.state = Mock()
request.state.request_id = request_id
return request
def _make_upstream_response(
*,
body: bytes,
status_code: int = 429,
content_type: str | None = "application/json",
extra_headers: dict[str, str] | None = None,
) -> httpx.Response:
headers: dict[str, str] = {}
if content_type is not None:
headers["content-type"] = content_type
if extra_headers:
headers.update(extra_headers)
return httpx.Response(status_code=status_code, headers=headers, content=body)
@pytest.fixture
def provider() -> BaseUpstreamProvider:
return BaseUpstreamProvider(
base_url="https://privateprovider.xyz", api_key="k", provider_fee=1.0
)
# --------------------------------------------------------------------------- #
# Redaction
# --------------------------------------------------------------------------- #
def test_redact_org_ids_replaces_identifier() -> None:
assert RAW_ORG_ID not in redact_org_ids(RATE_LIMIT_MESSAGE)
assert "org-[REDACTED]" in redact_org_ids(RATE_LIMIT_MESSAGE)
def test_redact_org_ids_is_idempotent() -> None:
once = redact_org_ids(RATE_LIMIT_MESSAGE)
assert redact_org_ids(once) == once
def test_redact_org_ids_leaves_unrelated_text() -> None:
assert redact_org_ids("organize the org-chart") == "organize the org-chart"
assert redact_org_ids("") == ""
# --------------------------------------------------------------------------- #
# Classification
# --------------------------------------------------------------------------- #
def test_classify_exact_scenario() -> None:
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE)
assert isinstance(info, RateLimitInfo)
assert info.code == UPSTREAM_RATE_LIMIT
assert info.model == "gpt-5.5-2026-04-23"
assert info.limit_name == "gpt-5.5"
assert info.metric == "tokens per min (TPM)"
assert info.limit == 180000000
assert info.used == 180000000
assert info.requested == 8929
assert info.retry_after_seconds == pytest.approx(0.002)
# Redaction-safe: no raw org id survives into the structured view.
assert RAW_ORG_ID not in info.message
assert RAW_ORG_ID not in json.dumps(info.as_details())
def test_classify_by_status_code_without_marker() -> None:
info = classify_rate_limit(429, "slow down")
assert info is not None
assert info.code == UPSTREAM_RATE_LIMIT
def test_classify_by_message_marker_without_429() -> None:
info = classify_rate_limit(400, "rate_limit_exceeded for this key")
assert info is not None
def test_retry_after_header_takes_precedence() -> None:
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE, {"Retry-After": "12"})
assert info is not None
assert info.retry_after_seconds == pytest.approx(12.0)
def test_non_rate_limit_error_is_not_classified() -> None:
assert classify_rate_limit(400, "invalid request: missing field 'model'") is None
assert classify_rate_limit(500, "internal server error") is None
# --------------------------------------------------------------------------- #
# forward_upstream_error_response integration
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_json_rate_limit_body_is_redacted_and_forwarded(
provider: BaseUpstreamProvider,
) -> None:
body = json.dumps(
{"error": {"message": RATE_LIMIT_MESSAGE, "type": "rate_limit_exceeded"}}
).encode()
upstream = _make_upstream_response(body=body, status_code=429)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
assert response.status_code == 429
raw = bytes(response.body).decode()
# No raw organization id may survive in the forwarded body.
assert RAW_ORG_ID not in raw
assert "org-[REDACTED]" in raw
# Body remains valid JSON; the original type is preserved while a stable
# rate-limit code is injected so callers can switch on it.
payload: dict[str, Any] = json.loads(raw)
assert payload["error"]["type"] == "rate_limit_exceeded"
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["model"] == "gpt-5.5-2026-04-23"
# A retry hint extracted from the message is surfaced as a header.
assert "retry-after" in {k.lower() for k in response.headers}
@pytest.mark.asyncio
async def test_non_json_rate_limit_envelope_uses_stable_code(
provider: BaseUpstreamProvider,
) -> None:
upstream = _make_upstream_response(
body=RATE_LIMIT_MESSAGE.encode(),
status_code=429,
content_type="text/plain",
)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["model"] == "gpt-5.5-2026-04-23"
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
@pytest.mark.asyncio
async def test_non_rate_limit_json_error_unchanged(
provider: BaseUpstreamProvider,
) -> None:
body = json.dumps(
{"error": {"message": "missing field 'model'", "type": "invalid_request"}}
).encode()
upstream = _make_upstream_response(body=body, status_code=400)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
assert response.status_code == 400
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "invalid_request"
assert "retry-after" not in {k.lower() for k in response.headers}
# --------------------------------------------------------------------------- #
# UpstreamError -> proxy response (preserves code/details/status)
# --------------------------------------------------------------------------- #
def test_create_upstream_error_response_preserves_structure() -> None:
from routstr.core.exceptions import UpstreamError
from routstr.payment.helpers import create_upstream_error_response
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE)
assert info is not None
err = UpstreamError(
f"Upstream error via litellm: {RATE_LIMIT_MESSAGE}",
status_code=429,
code=info.code,
details=info.as_details(),
)
response = create_upstream_error_response(err, _make_request())
# Original upstream status is preserved (not flattened to 502).
assert response.status_code == 429
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "upstream_error"
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["requested"] == 8929
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
def test_generic_upstream_error_still_defaults_to_502() -> None:
from routstr.core.exceptions import UpstreamError
from routstr.payment.helpers import create_upstream_error_response
err = UpstreamError("connection refused") # status_code defaults to 502
response = create_upstream_error_response(err, _make_request())
assert response.status_code == 502
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "upstream_error"
assert payload["error"]["code"] == 502
assert "details" not in payload["error"]
# --------------------------------------------------------------------------- #
# Structured log-extra redaction
# --------------------------------------------------------------------------- #
def test_security_filter_redacts_org_id_in_extra() -> None:
import logging
from routstr.core.logging import SecurityFilter
record = logging.LogRecord(
name="test",
level=logging.ERROR,
pathname=__file__,
lineno=1,
msg="upstream failed",
args=(),
exc_info=None,
)
# Simulate an ``extra={"body_preview": ...}`` field carrying an org id.
setattr(record, "body_preview", RATE_LIMIT_MESSAGE)
assert SecurityFilter().filter(record) is True
redacted: str = getattr(record, "body_preview")
assert RAW_ORG_ID not in redacted
assert "org-[REDACTED]" in redacted
def test_security_filter_redacts_org_id_in_nested_extra() -> None:
import logging
from routstr.core.logging import SecurityFilter
record = logging.LogRecord(
name="test",
level=logging.ERROR,
pathname=__file__,
lineno=1,
msg="upstream failed",
args=(),
exc_info=None,
)
# Nested structures: dict containing a list containing the org id.
setattr(record, "body", {"error": {"messages": [RATE_LIMIT_MESSAGE]}})
assert SecurityFilter().filter(record) is True
serialized = json.dumps(getattr(record, "body"))
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
# --------------------------------------------------------------------------- #
# 5xx-wrapped rate limit through forward_upstream_error_response
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_5xx_wrapped_rate_limit_is_classified(
provider: BaseUpstreamProvider,
) -> None:
# Some providers wrap a rate-limit in a 5xx envelope; classification must
# key off the message marker, not only the 429 status.
body = json.dumps({"error": {"message": RATE_LIMIT_MESSAGE}}).encode()
upstream = _make_upstream_response(body=body, status_code=500)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
assert response.status_code == 500
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
# --------------------------------------------------------------------------- #
# Real proxy loop: structured error surfaced + reservation reverted once
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_proxy_loop_surfaces_rate_limit_and_reverts_once() -> None:
from routstr import proxy as proxy_module
from routstr.core.db import ApiKey
from routstr.core.exceptions import UpstreamError
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE)
assert info is not None
key = ApiKey(hashed_key="rlkey", balance=10_000)
request = MagicMock()
request.method = "POST"
request.headers = {"authorization": "Bearer sk-rlkey"}
request.body = AsyncMock(return_value=b'{"model": "test-model"}')
request.state = MagicMock()
request.state.request_id = "req-rl"
upstream = MagicMock()
upstream.provider_type = "test"
upstream.prepare_headers = MagicMock(side_effect=lambda h: h)
upstream.forward_request = AsyncMock(
side_effect=UpstreamError(
f"Upstream error via litellm: {RATE_LIMIT_MESSAGE}",
status_code=429,
code=info.code,
details=info.as_details(),
)
)
session = MagicMock()
revert_mock = AsyncMock(return_value=True)
with (
patch.object(proxy_module, "get_model_instance", return_value=MagicMock()),
patch.object(proxy_module, "get_provider_for_model", return_value=[upstream]),
patch.object(
proxy_module, "get_max_cost_for_model", AsyncMock(return_value=1_000)
),
patch.object(
proxy_module,
"calculate_discounted_max_cost",
AsyncMock(return_value=1_000),
),
patch.object(proxy_module, "check_token_balance", MagicMock()),
patch.object(
proxy_module, "get_bearer_token_key", AsyncMock(return_value=key)
),
patch.object(proxy_module, "pay_for_request", AsyncMock(return_value=1_000)),
patch.object(proxy_module, "revert_pay_for_request", revert_mock),
):
response = await proxy_module.proxy(
request, "v1/chat/completions", session=session
)
# Original 429 status and the stable code/details survive to the client.
assert response.status_code == 429
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "upstream_error"
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["requested"] == 8929
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
# Single upstream failed -> reservation reverted exactly once (no double-charge).
revert_mock.assert_awaited_once_with(key, session, 1_000)
+942 -87
View File
File diff suppressed because it is too large Load Diff
+32 -24
View File
@@ -1,36 +1,44 @@
This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).
# Routstr node admin UI
## Getting Started
A [Next.js](https://nextjs.org) app (App Router, **static export**) that provides the
admin dashboard for a `routstr-core` node: login, settings, providers, balances,
transactions, usage, and logs.
First, run the development server:
There is no separate web server in production. `next build` produces a fully static
export (`next.config.ts` sets `output: 'export'`), and the FastAPI backend serves it
directly from `../ui_out/` (see `routstr/core/main.py`). So the UI and the API are
served from the **same origin** in production.
```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```
## Developing the UI (hot reload)
Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
The everyday loop runs two processes side by side — you do **not** rebuild the static
export while developing:
You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.
1. Start the backend on `:8000` — from the repo root: `make docker-up` (or
`uvicorn routstr.core.main:app --reload`).
2. Start the Next.js dev server on `:3000` — from the repo root: `make ui-dev`
(or `cd ui && pnpm dev`). Edits hot-reload instantly.
This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.
Open http://localhost:3000. With no `NEXT_PUBLIC_API_URL` set, the UI falls back to
`http://127.0.0.1:8000` in development (see `lib/api/services/configuration.ts`), so it
talks to the local backend out of the box.
## Learn More
Because dev is cross-origin (`:3000``:8000`), it relies on the backend's CORS
allowing the UI origin. The default `cors_origins` is `["*"]`; if you tighten CORS,
keep `http://localhost:3000` allowed for development.
To learn more about Next.js, take a look at the following resources:
## Building the integrated/static UI (what production serves)
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
To produce the bundle that FastAPI serves from `../ui_out/`:
You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!
- `make ui-build` — builds with local Node/pnpm (`scripts/build-ui.sh`), then moves
`ui/out/*` to `../ui_out/`.
- `make ui-build-docker` — same, but inside Docker (no local Node needed).
## Deploy on Vercel
`NEXT_PUBLIC_*` variables are read from the repo-root `.env` at build time and baked in.
For a same-origin deployment leave `NEXT_PUBLIC_API_URL` empty (relative paths); the UI
uses `window.location.origin` at runtime. After building, start the backend and open
http://localhost:8000 — the dashboard is served at `/` and `/admin`.
The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.
If `../ui_out/` does not exist, the backend logs a warning at startup and serves the API
only (hitting a UI route returns a small JSON fallback instead of the dashboard).
+20
View File
@@ -118,6 +118,26 @@ export function ProviderFormFields({
/>
)}
<div className='grid gap-2'>
<Label htmlFor={`${idPrefix}slug`}>
Slug {mode === 'create' ? '(optional, auto-generated)' : ''}
</Label>
<Input
id={`${idPrefix}slug`}
value={formData.slug || ''}
onChange={(e) =>
setFormData((prev) => ({
...prev,
slug: e.target.value || undefined,
}))
}
placeholder='e.g. openai-prod'
/>
<p className='text-muted-foreground text-xs'>
Stable external key used to update this provider via the admin API.
</p>
</div>
<div className='grid gap-2'>
<Label htmlFor={`${idPrefix}base_url`}>Base URL</Label>
<Input
+3
View File
@@ -14,6 +14,7 @@ export const ProviderTypeSchema = z.object({
export const UpstreamProviderSchema = z.object({
id: z.number(),
slug: z.string().nullable().optional(),
provider_type: z.string(),
base_url: z.string(),
api_key: z.string().optional(),
@@ -31,6 +32,7 @@ export const CreateUpstreamProviderSchema = z.object({
enabled: z.boolean().default(true),
provider_fee: z.number().optional(),
provider_settings: z.record(z.string(), z.any()).nullable().optional(),
slug: z.string().optional(),
});
export const UpdateUpstreamProviderSchema = z.object({
@@ -41,6 +43,7 @@ export const UpdateUpstreamProviderSchema = z.object({
enabled: z.boolean().optional(),
provider_fee: z.number().optional(),
provider_settings: z.record(z.string(), z.any()).nullable().optional(),
slug: z.string().optional(),
});
export const AdminModelPricingSchema = z.object({