Provider scoping (items 1/2/6):
- Key visibility maps on (model_id.lower(), upstream_provider_id), matching
refresh_model_maps, so a disable/override row on one provider never leaks
onto another provider's model, and matching is case-insensitive.
Data safety (items 3/5):
- Degraded OpenRouter fetches (network error, 429, non-200, bad payload)
return None (unknown) instead of []; a provider whose path set is unknown
keeps its previously persisted rows instead of being wiped.
- Endpoint payload parsing moved fully inside try, with a list guard, so
endpoints:null or non-list shapes are swallowed as documented.
- refresh with an empty live upstream list is a no-op; the unfiltered
DELETE in the prune path is gone (prune now keys off enabled DB rows).
Hot path (items 4/12/14):
- Persist uses chunked bulk INSERTs (one statement per 500 rows) instead of
per-row ORM adds; redundant ix_model_paths_model_id index dropped.
- Read routes filter in SQL instead of materializing the whole table, and
output ordering is deterministic (public id + path), independent of rowid.
- Visibility no longer rebuilds fully priced Model objects per override row;
it reads id/forwarded_model_id/canonical_slug straight off ModelRow.
Path/id contract (items 7/8/9/11):
- discovery_path_for_subprovider/discovery_base_paths hooks on
BaseUpstreamProvider, overridden by OpenRouterUpstreamProvider, mirror
_apply_provider_field so discovery and response stamping cannot drift
(openrouter:OpenRouter now correctly maps to unknown).
- openrouter_author_slug falls back to a slash-containing forwarded_model_id,
so admin-created alias rows are discoverable.
- public_model_id splits on the first slash, same as get_base_model_id, so
discovery ids can be sent to chat completions verbatim.
Lifecycle (items 10/13):
- ENABLE_MODEL_PATHS_REFRESH kill switch; interval and flag re-read every
loop iteration, and the task idles (not exits) while disabled.
- First 429 latches and aborts the remaining fan-out for the cycle; a
per-cycle cache dedupes fetches across providers sharing a base URL.
- refresh_model_maps prunes paths of disabled/deleted providers so admin
mutations take effect immediately; rows carry updated_at and both
endpoints expose it.
Tests (item 15) rewritten through the public refresh entry point with
transport-level httpx.MockTransport fakes, FK enforcement on, and coverage
for the periodic loop. Migration re-chained onto 9c4d8e2f1a6b.
The admin password is generated and logged on first start and the nsec is
set from the admin UI; ADMIN_PASSWORD/NSEC in .env are only a legacy seed.
Update the README, quickstart, configuration, and deployment docs to match,
and drop the unused ADMIN_KEY environment variable.
Several compose/.env examples mounted /app/data but left DATABASE_URL at the
relative default, so the database — and the master key file generated beside it —
landed off the persisted volume and would be lost on a container recreate. Point
DATABASE_URL inside /app/data in those examples and list routstr_secret.key in
the persistence table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The secret-key docs still said ROUTSTR_SECRET_KEY was required and that
the node would not start without it. Update the README, .env.example, and
the provider docs (configuration, quickstart, deployment) to the current
behaviour: the key is optional; when unset the node generates one beside
the database, so it persists on the same volume as the data, and prints a
one-time back-it-up notice; set it explicitly to manage the key yourself.
Switch the generation and reset snippets to `uv run python`, and add the
containerised reset variant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tinfoil PR #385 added model=<name> to the X-Tinfoil-Usage-Metrics
header/trailer. This commit uses that field for accurate billing.
Changes in routstr/upstream/ehbp.py:
- parse_tinfoil_usage_metrics(): extract the model= field as a string
alongside the existing token counts (previously silently discarded
because int() failed on it).
- _build_cost_info(): accept optional actual_model parameter propagated
through to callers when a real mismatch is detected.
- _compute_ehbp_actual_cost(): compare the served model against
model_obj.forwarded_model_id (the expected upstream ID) rather than
model_obj.id (the client-facing alias). This prevents spurious
mismatches when a node runner maps e.g. tinfoil-glm-5-2 -> glm-5-2
and the header correctly reports glm-5-2. On a genuine mismatch
(failover to a different upstream model), look up the actual model's
pricing via get_model_instance() (forwarded_model_id values are
registered as routable aliases in the global model map).
- forward_ehbp_request() / forward_ehbp_x_cashu_request(): use the
actual served model for payment finalization and logging when a
mismatch is detected.
Tests: 6 new scenarios (alias match, real mismatch with alias, unknown
model fallback, old-format compat, cache token details + model), plus
forwarded_model_id set on all existing mock model objects to keep them
passing. All 49 Tinfoil/EHBP unit tests pass.
Tinfoil's proxy server guide only requires /attestation (GET) and
/v1/chat/completions + /v1/responses (POST). The .well-known/ path was
never requested by Tinfoil and was incorrectly added to:
- _API_PATH_PREFIXES (prefix gate)
- the unauthenticated GET bypass branch
- the Tinfoil integration docs
Remove it from all three.
EHBP response bodies are opaque encrypted blobs, so cost cannot be injected
into JSON like the normal proxy flow. Instead, surface cost as response headers:
X-Routstr-Cost-Msats — total msats charged (bearer + x-cashu)
X-Routstr-Cost-Usd — USD equivalent (bearer only)
X-Routstr-Input-Cost-Msats — msats attributed to input tokens
X-Routstr-Output-Cost-Msats— msats attributed to output tokens
- Refactor _compute_ehbp_actual_cost from int→dict to carry full cost breakdown
- Add _build_cost_info() and _inject_cost_response_headers() helpers
- Capture adjust_payment_for_tokens return in bearer path (was discarded)
- Update CORS expose_headers, docs, and unit tests
Add TinfoilUpstreamProvider that uses inference.tinfoil.sh as a direct
EHBP upstream. Routstr acts as a blind relay: it forwards the opaque
encrypted body to the Tinfoil enclave without ever seeing plaintext,
and bills from the X-Tinfoil-Usage-Metrics response header.
- New routstr/upstream/tinfoil.py: fetches models from public
GET /v1/models, parses Tinfoil pricing into standard Model/Pricing
schema, supports_ehbp=True, proxies /attestation to atc.tinfoil.sh
- Updated routstr/upstream/ehbp.py:
- parse_tinfoil_usage_metrics() parses prompt=N,completion=N header
- _resolve_ehbp_target_url() honors X-Tinfoil-Enclave-Url from SDK
- _strip_proxy_headers() removes proxy-only headers before forwarding
- _compute_ehbp_actual_cost() converts usage to msats via calculate_cost
- forward_ehbp_request() finalizes with exact token cost when usage
header is present (non-streaming), falls back to max-cost otherwise
- forward_ehbp_x_cashu_request() computes refund from actual cost
when usage is available
- Updated routstr/proxy.py: forward /attestation and /.well-known/ paths
without model/cost/auth lookups
- Updated routstr/upstream/__init__.py and helpers.py: register and
auto-seed Tinfoil provider from TINFOIL_API_KEY env var
- Updated .env.example with TINFOIL_API_KEY
- 22 new unit tests in tests/unit/test_tinfoil_integration.py
- Updated docs/tinfoil-direct-integration.md and docs/ehbp-proxy-support.md
with implementation status and billing behavior table
- Add supports_ehbp + get_ehbp_forwarding_target hooks to BaseUpstreamProvider,
keeping base.py minimal (no large forwarding methods)
- Create routstr/upstream/ehbp.py with forward_ehbp_request and
forward_ehbp_x_cashu_request helpers, plus max-cost finalization
- PPQAIUpstreamProvider: set supports_ehbp = True, implement target to
/private/v1/... with X-Private-Model header
- proxy.py: filter to EHBP-capable providers, route to ehbp helpers
- Fix bearer EHBP payment: reserve upfront, finalize max cost on success
- Fix X-Cashu EHBP: refund full token on failure, refund excess on success
- Update docs/ehbp-proxy-support.md to reflect new architecture
9qeklajc
Jeroen Ubbink
redshift
Bilthon
Evan Yang
Shroominic
9qeklajc