upstream_api_key was added to SECRET_FIELDS, so it was stripped from every
blob write — but unlike nsec, nothing migrates it into encrypted storage. A
node carrying it only in the DB blob would load it into memory once, rewrite
the blob without it, and lose it on the next restart, breaking upstream auth.
It is node-scoped config that really belongs on a provider, not a vault
secret, and it has no encrypted home yet. Remove it from SECRET_FIELDS so it
stays in the blob exactly as before; redaction on read and ignore-on-write in
the admin settings endpoint are unchanged. Encrypting it is follow-up work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Provide an offline recovery command that sets a new admin password directly in
the Secret store, for operators locked out of the admin UI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an admin endpoint to set, rotate and clear the nsec, authenticate against
the stored password hash, and redact secret values (nsec shown as [REDACTED])
in settings responses. Wire the admin UI to the new endpoint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist and load admin password and nsec from the encrypted Secret store on
boot: generate a temporary admin password on first run (logged once), encrypt
a provided nsec, and fail fast if a stored nsec cannot be decrypted with the
current key. Stop clobbering live secret settings with empty env values.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce a singleton Secret model holding the admin password hash and the
Fernet-encrypted nsec, with a hand-written migration for the secrets table.
Add suite-wide pytest config pinning a valid ROUTSTR_SECRET_KEY.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Encrypt/decrypt secrets at rest with Fernet keyed by ROUTSTR_SECRET_KEY,
and hash/verify admin passwords with scrypt. Self-contained helpers with
no consumers yet.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make model_obj/provider_fee required (still nullable) on
adjust_payment_for_tokens, get_x_cashu_cost and the private pricing
helpers so a call site that fails to thread the served candidate is a
type error instead of a silent fallback to alias-map re-derivation.
calculate_cost keeps its defaults as the one documented fallback seam.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Admission and reservation were sized once to the best-ranked candidate's
max cost while settlement bills the candidate that actually serves, so a
failover to a pricier candidate could settle far beyond the admitted
envelope and consume balance reserved by other in-flight requests. Before
trying a fallback candidate, raise the reservation to its own envelope;
reject candidates the key cannot cover, exactly as admission would have
had they been ranked first.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The failover loop resolved a single Model for the request and reused it
for every provider: a fallback provider was asked to serve the routing
winner's model id and billed at the winner's pricing and fee. The alias
map now keeps (model, provider) candidate pairs, the proxy rebinds both
per attempt, and forwarding, max-cost echo, and settlement all use the
candidate actually being tried. On a failover serve the response's
model field now names the serving candidate's id.
The unified candidate lookup also applies the version-suffix strip
(-YYYYMMDD) that model resolution already had, so version-suffixed
requests no longer resolve a model yet 400 with "no provider found".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two tests entered patch("routstr.auth.calculate_cost", ...) inside
concurrently gathered tasks. Interleaved patch exits restore in the
wrong order, leaving the mock permanently installed for every later
test in the session. Hoist the patch around the gather.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The USD-cost path (and the litellm pricing fallback) resolved the
provider fee via get_provider_for_model(model_id)[0] — the best-ranked
provider for the alias, not the one that served. Settlement callers in
the upstream handlers now pass their own provider_fee through
adjust_payment_for_tokens / get_x_cashu_cost into calculate_cost; the
string-derived fallback remains for callers without a serving provider.
Configured model pricing is unaffected (the fee is already baked into
cached pricing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
X-Cashu handlers do not rewrite the upstream's echoed model string, so
get_x_cashu_cost previously priced whatever wire name the upstream
reported — the most collapse-prone alias lookup of all. The routed Model
is now threaded from forward_x_cashu_request through the chat and
Responses handler chains (and the litellm messages path) into
get_x_cashu_cost, so cost and refund are computed from the model that
actually served.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Settlement previously re-derived pricing from the response's model string
through the alias map, which resolves to the best-ranked candidate for
that alias — not necessarily the provider/model that actually served the
request. adjust_payment_for_tokens and calculate_cost now accept the
routed Model and bill its pricing directly; the string lookup remains as
a warning fallback for callers without routed identity (e.g. the generic
streaming finalizer).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PPQ.AI (BYOK) requests were billed at ~5% of their true cost because
_resolve_usd_cost fell through to usage.cost (a small BYOK routing fee)
instead of using cost_details.upstream_inference_cost (the real inference
cost). The proxy operator absorbed the inference cost.
The fix adds a BYOK-specific branch in _resolve_usd_cost: when is_byok is
true and cost_details.upstream_inference_cost is present, bill
upstream_inference_cost + byok_fee — what PPQ actually deducts from the
balance. Non-BYOK providers (e.g. OpenRouter) are unaffected because their
usage.cost already equals upstream_inference_cost.
Regression tests mirror the live glm-5.2-fast request from GitHub issue #615,
asserting the corrected billing (940,274 msats vs the old 45,202 msats — a
20.8× undercharge).
Closes#615
- Case-insensitive comparison of served model vs forwarded_model_id
(matches get_model_instance's lowercasing semantics)
- Suppress spurious mismatch when get_model_instance resolves back to
the same model (e.g. date-versioned glm-5-2-20260415 -> glm-5-2)
- Preserve unknown-model warning only when lookup returns None
- Tests: case-insensitive match + date-versioned alias resolution
Tinfoil PR #385 added model=<name> to the X-Tinfoil-Usage-Metrics
header/trailer. This commit uses that field for accurate billing.
Changes in routstr/upstream/ehbp.py:
- parse_tinfoil_usage_metrics(): extract the model= field as a string
alongside the existing token counts (previously silently discarded
because int() failed on it).
- _build_cost_info(): accept optional actual_model parameter propagated
through to callers when a real mismatch is detected.
- _compute_ehbp_actual_cost(): compare the served model against
model_obj.forwarded_model_id (the expected upstream ID) rather than
model_obj.id (the client-facing alias). This prevents spurious
mismatches when a node runner maps e.g. tinfoil-glm-5-2 -> glm-5-2
and the header correctly reports glm-5-2. On a genuine mismatch
(failover to a different upstream model), look up the actual model's
pricing via get_model_instance() (forwarded_model_id values are
registered as routable aliases in the global model map).
- forward_ehbp_request() / forward_ehbp_x_cashu_request(): use the
actual served model for payment finalization and logging when a
mismatch is detected.
Tests: 6 new scenarios (alias match, real mismatch with alias, unknown
model fallback, old-format compat, cache token details + model), plus
forwarded_model_id set on all existing mock model objects to keep them
passing. All 49 Tinfoil/EHBP unit tests pass.
The prepare_request_body override in OpenRouterUpstreamProvider injected
provider.require_parameters=true on tool-use requests. This is no longer
needed. Removed the override, the now-unused json import, and the
corresponding test suite.
The X-Cashu refund endpoint raised 404 "Refund not found" when the
"in" transaction existed with a request_id but the "out" (refund)
transaction had not been written yet. This is a timing race: the
endpoint is polled while the upstream request is still in flight,
before send_refund() has minted and stored the refund token.
The 404 was indistinguishable from a genuinely-missing refund, so
clients had no signal that retrying would succeed — leading to
stranded refunds when clients gave up.
Replace the third 404 branch with 425 Too Early + Retry-After: 2.
The two earlier 404 branches (no "in" row, no request_id) remain 404
since those genuinely mean no refund will ever exist.
Also adds debug logging on all three not-found/pending branches so
the race is no longer invisible to operators (middleware does not log
request headers).
Adds unit tests for the new 425 pending path and the no-request_id
404 path.
Refs: refund-race-condition
Jeroen UbbinkandClaude Opus 4.8
9qeklajcandGitHub
redshift