The secret-key docs still said ROUTSTR_SECRET_KEY was required and that
the node would not start without it. Update the README, .env.example, and
the provider docs (configuration, quickstart, deployment) to the current
behaviour: the key is optional; when unset the node generates one beside
the database, so it persists on the same volume as the data, and prints a
one-time back-it-up notice; set it explicitly to manage the key yourself.
Switch the generation and reset snippets to `uv run python`, and add the
containerised reset variant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tinfoil PR #385 added model=<name> to the X-Tinfoil-Usage-Metrics
header/trailer. This commit uses that field for accurate billing.
Changes in routstr/upstream/ehbp.py:
- parse_tinfoil_usage_metrics(): extract the model= field as a string
alongside the existing token counts (previously silently discarded
because int() failed on it).
- _build_cost_info(): accept optional actual_model parameter propagated
through to callers when a real mismatch is detected.
- _compute_ehbp_actual_cost(): compare the served model against
model_obj.forwarded_model_id (the expected upstream ID) rather than
model_obj.id (the client-facing alias). This prevents spurious
mismatches when a node runner maps e.g. tinfoil-glm-5-2 -> glm-5-2
and the header correctly reports glm-5-2. On a genuine mismatch
(failover to a different upstream model), look up the actual model's
pricing via get_model_instance() (forwarded_model_id values are
registered as routable aliases in the global model map).
- forward_ehbp_request() / forward_ehbp_x_cashu_request(): use the
actual served model for payment finalization and logging when a
mismatch is detected.
Tests: 6 new scenarios (alias match, real mismatch with alias, unknown
model fallback, old-format compat, cache token details + model), plus
forwarded_model_id set on all existing mock model objects to keep them
passing. All 49 Tinfoil/EHBP unit tests pass.
Tinfoil's proxy server guide only requires /attestation (GET) and
/v1/chat/completions + /v1/responses (POST). The .well-known/ path was
never requested by Tinfoil and was incorrectly added to:
- _API_PATH_PREFIXES (prefix gate)
- the unauthenticated GET bypass branch
- the Tinfoil integration docs
Remove it from all three.
EHBP response bodies are opaque encrypted blobs, so cost cannot be injected
into JSON like the normal proxy flow. Instead, surface cost as response headers:
X-Routstr-Cost-Msats — total msats charged (bearer + x-cashu)
X-Routstr-Cost-Usd — USD equivalent (bearer only)
X-Routstr-Input-Cost-Msats — msats attributed to input tokens
X-Routstr-Output-Cost-Msats— msats attributed to output tokens
- Refactor _compute_ehbp_actual_cost from int→dict to carry full cost breakdown
- Add _build_cost_info() and _inject_cost_response_headers() helpers
- Capture adjust_payment_for_tokens return in bearer path (was discarded)
- Update CORS expose_headers, docs, and unit tests
Add TinfoilUpstreamProvider that uses inference.tinfoil.sh as a direct
EHBP upstream. Routstr acts as a blind relay: it forwards the opaque
encrypted body to the Tinfoil enclave without ever seeing plaintext,
and bills from the X-Tinfoil-Usage-Metrics response header.
- New routstr/upstream/tinfoil.py: fetches models from public
GET /v1/models, parses Tinfoil pricing into standard Model/Pricing
schema, supports_ehbp=True, proxies /attestation to atc.tinfoil.sh
- Updated routstr/upstream/ehbp.py:
- parse_tinfoil_usage_metrics() parses prompt=N,completion=N header
- _resolve_ehbp_target_url() honors X-Tinfoil-Enclave-Url from SDK
- _strip_proxy_headers() removes proxy-only headers before forwarding
- _compute_ehbp_actual_cost() converts usage to msats via calculate_cost
- forward_ehbp_request() finalizes with exact token cost when usage
header is present (non-streaming), falls back to max-cost otherwise
- forward_ehbp_x_cashu_request() computes refund from actual cost
when usage is available
- Updated routstr/proxy.py: forward /attestation and /.well-known/ paths
without model/cost/auth lookups
- Updated routstr/upstream/__init__.py and helpers.py: register and
auto-seed Tinfoil provider from TINFOIL_API_KEY env var
- Updated .env.example with TINFOIL_API_KEY
- 22 new unit tests in tests/unit/test_tinfoil_integration.py
- Updated docs/tinfoil-direct-integration.md and docs/ehbp-proxy-support.md
with implementation status and billing behavior table
- Add supports_ehbp + get_ehbp_forwarding_target hooks to BaseUpstreamProvider,
keeping base.py minimal (no large forwarding methods)
- Create routstr/upstream/ehbp.py with forward_ehbp_request and
forward_ehbp_x_cashu_request helpers, plus max-cost finalization
- PPQAIUpstreamProvider: set supports_ehbp = True, implement target to
/private/v1/... with X-Private-Model header
- proxy.py: filter to EHBP-capable providers, route to ehbp helpers
- Fix bearer EHBP payment: reserve upfront, finalize max cost on success
- Fix X-Cashu EHBP: refund full token on failure, refund excess on success
- Update docs/ehbp-proxy-support.md to reflect new architecture
Jeroen UbbinkandClaude Opus 4.8
redshift
9qeklajc
Bilthon
Evan Yang
Shroominic
9qeklajc
Cursor Agentanddb2002dominic