Add TinfoilUpstreamProvider that uses inference.tinfoil.sh as a direct
EHBP upstream. Routstr acts as a blind relay: it forwards the opaque
encrypted body to the Tinfoil enclave without ever seeing plaintext,
and bills from the X-Tinfoil-Usage-Metrics response header.
- New routstr/upstream/tinfoil.py: fetches models from public
GET /v1/models, parses Tinfoil pricing into standard Model/Pricing
schema, supports_ehbp=True, proxies /attestation to atc.tinfoil.sh
- Updated routstr/upstream/ehbp.py:
- parse_tinfoil_usage_metrics() parses prompt=N,completion=N header
- _resolve_ehbp_target_url() honors X-Tinfoil-Enclave-Url from SDK
- _strip_proxy_headers() removes proxy-only headers before forwarding
- _compute_ehbp_actual_cost() converts usage to msats via calculate_cost
- forward_ehbp_request() finalizes with exact token cost when usage
header is present (non-streaming), falls back to max-cost otherwise
- forward_ehbp_x_cashu_request() computes refund from actual cost
when usage is available
- Updated routstr/proxy.py: forward /attestation and /.well-known/ paths
without model/cost/auth lookups
- Updated routstr/upstream/__init__.py and helpers.py: register and
auto-seed Tinfoil provider from TINFOIL_API_KEY env var
- Updated .env.example with TINFOIL_API_KEY
- 22 new unit tests in tests/unit/test_tinfoil_integration.py
- Updated docs/tinfoil-direct-integration.md and docs/ehbp-proxy-support.md
with implementation status and billing behavior table
- Add supports_ehbp + get_ehbp_forwarding_target hooks to BaseUpstreamProvider,
keeping base.py minimal (no large forwarding methods)
- Create routstr/upstream/ehbp.py with forward_ehbp_request and
forward_ehbp_x_cashu_request helpers, plus max-cost finalization
- PPQAIUpstreamProvider: set supports_ehbp = True, implement target to
/private/v1/... with X-Private-Model header
- proxy.py: filter to EHBP-capable providers, route to ehbp helpers
- Fix bearer EHBP payment: reserve upfront, finalize max cost on success
- Fix X-Cashu EHBP: refund full token on failure, refund excess on success
- Update docs/ehbp-proxy-support.md to reflect new architecture
The same-mint shortcut in swap_to_primary_mint did a same-mint
split(include_fees=True) — which burns the mint's NUT-02 per-proof input
fee — but returned the full token amount, over-crediting the user (the
same bug already fixed for the trusted-mint receive path). It also
skipped DLEQ verification that the trusted path performs.
Extract the shared same-mint redeem into _redeem_same_mint (load mint,
verify DLEQ, split, credit amount - input_fees) and delegate from both
recieve_token and the shortcut, so the two paths can't drift again. This
shortcut is reachable when PRIMARY_MINT_URL is set outside CASHU_MINTS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The comment described "21 proofs @ 100 ppk" arithmetic, but the mocked
token has one proof and get_fees_for_proofs is hard-mocked to 3, so the
math wasn't exercised. Describe what the mock actually does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
recieve_token swaps the incoming proofs at the same mint with
include_fees=True (paying the mint's NUT-02 per-proof input fee) but credited
the full face value. On every topup from a fee-charging trusted mint, routstr
over-credited the user by the fee and its own wallet drifted toward insolvency.
Subtract get_fees_for_proofs(proofs) from the credited amount, mirroring the
foreign-mint swap path which already accounts for it. Adds a fee-charging
trusted-mint unit test (credited == face - input_fee).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve pay_for_request conflict: keep main's atomic child-key
balance_limit guard and post-rowcount-check ordering, and stamp
reserved_at on both billing and child reservations.
CI runs 'uv run mypy .' while the Makefile only checked routstr/, so test
files could pass locally and fail the pipeline. lint, type-check and
ci-lint now check everything; --ignore-missing-imports is dropped since
the CI invocation passes without it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The session was needed when model pricing lived in the DB (73d3613) and has
been dead since pricing moved to the in-memory model map (0da08fb), yet every
caller was still obliged to supply one. get_x_cashu_cost even opened a DB
session per x-cashu request solely to feed it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cached prompt tokens were billed at the full input rate whenever a vendor's
usage dialect or cache pricing was unknown, overcharging DeepSeek topups
~5-10x on agentic workloads (hits are 10x cheaper upstream) and silently
mispricing OpenAI cached reads and Anthropic cache writes the same way.
Two root causes, two fixes:
- Usage dialects: DeepSeek reports prompt_cache_hit_tokens /
prompt_cache_miss_tokens, which billing never parsed. Usage normalization
now lives in payment/usage.py as a union parser over the known,
non-colliding dialects (OpenAI prompt_tokens_details, Anthropic additive
cache fields, DeepSeek hit/miss), producing one canonical NormalizedUsage.
Providers expose it as an overridable BaseUpstreamProvider.normalize_usage
hook — the escape hatch for future vendors whose fields genuinely
conflict — and every settlement call site passes the provider's result
through, so calculate_cost holds no vendor knowledge of its own.
- Cache rates: the OpenRouter model feed omits input_cache_read/-write for
most DeepSeek models (and e.g. openai/gpt-4o), so billing fell back to the
full input rate. Missing rates are now backfilled from litellm's bundled
cost map before the provider fee is applied; the input-rate fallback
remains only as the documented last resort.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Python pre-check on balance_limit provided no concurrency guarantee —
two concurrent requests could both pass the check on stale in-memory state
and both proceed to reserve, exceeding the limit.
Add the balance_limit guard to the child key UPDATE's WHERE clause so the
enforcement is atomic. The HTTP 402 is raised without an explicit rollback:
since session.commit() was never called, the uncommitted billing key update
is discarded when the session context manager closes on exception exit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the fragile `re.split(b"data: ")` streaming parser with a
buffered, event-delimited parser in both the chat-completion and
Responses-API streamers. Events are accumulated until the SSE blank-line
delimiter, so parsing is independent of network chunk boundaries.
Fixes:
- OpenRouter `: OPENROUTER PROCESSING` keepalive comments no longer leak
to clients as `data: : ...` (the `Unexpected token ':'` client crash).
- JSON payloads split across TCP reads are reassembled before parsing.
- CRLF framing (Gemini native alt=sse) handled.
- Combined content+usage chunks (Gemini thinking models over the
OpenAI-compat endpoint) forward content once and still report usage in
the cost trailer, instead of dropping the assistant message.
- Multi-line non-JSON `data` blocks are re-prefixed per line so they stay
valid SSE framing for the client.
Adds tests/unit/test_streaming_sse_providers.py driving the real
generator against per-provider on-the-wire framing, and switches the
integration token-mint fallback to secrets.token_hex to kill a
PRNG/clock collision flake.
LightningInvoice had no columns for balance_limit, balance_limit_reset,
or validity_date. SQLModel silently dropped these constructor kwargs, so
create_api_key_from_invoice always produced an unconstrained key.
Add the three columns to LightningInvoice with a migration, and wire them
through to the ApiKey in create_api_key_from_invoice, matching the pattern
already used in the child key creation path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous in-memory deduction (key.balance -= cost) was a read-modify-
write on stale state, allowing two concurrent create_child_key() calls to
both pass the balance check and both succeed, effectively charging the
parent only once for two child keys.
Replace with an atomic UPDATE ... WHERE balance - reserved_balance >= cost
and check rowcount, matching the pattern already used in pay_for_request.
Also adds a concurrent integration test that reproduces the race and
confirms the fix holds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
redshift
9qeklajcandGitHub
Jeroen UbbinkandClaude Opus 4.8