laantungir/routstr-core
1
0
Fork 0
mirror of https://github.com/Routstr/routstr-core.git synced 2026-10-05 12:28:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(typesafe): simplify catalog assembly, seed versioned ids, log fetch failures, drop session report doc

Browse Source
This commit is contained in:
9qeklajc
2026-09-20 15:57:55 +02:00
parent 4f8f2bb1e0
commit 303d323aeb
4 changed files with 131 additions and 276 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/SYSTEMONE_TYPESAFE.md
-161
View File
@@ -1,161 +0,0 @@
# TypeSafe `/v1/systemone` support — changes & test report
**Repo:** `~/projects/routstr-core` **Branch:** `feat/systemone-typesafe` (cut from `origin/main` @ `c1b610d4`)
**Date:** 2026-09-19 **Status:** code complete, tests green, **not deployed** (no container restart)
---
## 1. What was added
Support for TypeSafe's System One decision endpoint — `POST /v1/systemone`
(`{state, model, questions}` → `{model, answers, usage}`) — as a first-class
Routstr endpoint with a dedicated upstream provider.
| File | Change |
|---|---|
| `routstr/upstream/typesafe.py` | **new** — `TypeSafeUpstreamProvider`: fixed base URL `https://api.typesafe.ai/v1`, `fetch_models()` maps TypeSafe's pricing-less `GET /v1/models` onto priced `Model` objects ($0.042/M input, $0 output, 64k context, `text->decisions`), catalog failures fail closed |
| `routstr/upstream/__init__.py` | provider registered → appears in the admin UI provider-type dropdown |
| `routstr/proxy.py` | `systemone` added to `_ALLOWED_ENDPOINTS` (POST only) |
| `routstr/upstream/base.py` | **two settlement fixes** (see below) |
| `routstr/upstream/helpers.py` | `TYPESAFE_API_KEY` env seeding (empty provider table only) |
| `docs/api/overview.md`, `docs/api/endpoints.md` | endpoint documented |
### The two `base.py` fixes matter most
1. `_x_cashu_path_has_settlement_handler` — now admits `systemone`, so ecash
payments settle and refund the delta instead of being rejected with
`x_cashu_unsupported_endpoint`.
2. `forward_request`'s response-settlement branch — now includes
`systemone`. **Without this the endpoint served free**: the request fell
through to the generic streaming path, whose `_finalize_generic_streaming_payment`
releases the reservation *without charging* (usage never read).
---
## 2. Test results
Both runs executed against the working tree on the branch above.
### Targeted suites — 16/16 pass
```
pytest tests/unit/test_typesafe_integration.py tests/integration/test_systemone.py
→ 16 passed
```
Covers: allowlist admit/refuse (incl. `systemonedump`, `v1/systemone/secret`,
traversal spellings, GET/DELETE), x-cashu gate, provider metadata,
`fetch_models` pricing + error handling, and an end-to-end proxied request that
asserts the upstream hop is exactly `https://api.typesafe.ai/v1/systemone` and
that billing settles from usage (1000 input × 0.001 sats = 1000 msats, output
free) — plus an X-Cashu settle-and-refund case.
### Full unit suite
```
pytest tests/unit
→ 1584 passed, 9 failed
```
### Full integration suite
```
pytest tests/integration -m "not requires_docker"
→ 492 passed, 13 skipped, 0 failed
```
### Lint / types
```
ruff check routstr tests → All checks passed!
mypy routstr/upstream/typesafe.py → only a pre-existing error in routstr/nostr/discovery.py
```
---
## 3. The 9 unit failures are pre-existing (proven)
Failing: `test_cashu_httpx_compat.py` (3), `test_fee_payout_migration.py` (3),
`test_mint_url_migration.py` (1), `test_provider_id_migration.py` (2).
Verified by stashing the branch (`git stash push -u`) and re-running those four
files against pristine `origin/main`:
```
→ 9 failed, 10 passed
```
Identical failures with none of this work applied. Causes:
* **httpx compat (3)** — installed httpx no longer accepts the `proxies=` kwarg
the shim probes for; version drift, unrelated to this branch.
* **migration tests (6)** — the tests shell out to `alembic upgrade`, whose
subprocess imports the app, whose file logger opens
`logs/app_2026-09-19.log` — a **root-owned file created by the running
Docker container**, unwritable by the `debian` user → `PermissionError` →
alembic exits non-zero. Environment artifact, not code.
---
## 4. Environment note (how to reproduce these runs)
Running pytest **from the repo directory** currently fails at import:
```
ValueError: Unable to configure handler 'file'
← PermissionError: .../logs/app_2026-09-19.log
```
The running container (root) owns today's log file. The runs above therefore
used a scratch cwd so the logger writes elsewhere, with the repo on
`PYTHONPATH`:
```bash
mkdir -p /tmp/routstr-test && cd /tmp/routstr-test
PYTHONPATH=$HOME/projects/routstr-core \
$HOME/projects/routstr-core/.venv/bin/python -m pytest \
$HOME/projects/routstr-core/tests/integration -m "not requires_docker" -q
```
To run in-repo instead (`make test-unit`), either run as root, or move today's
root-owned log aside first — the container keeps writing to its open file
descriptor, so `mv` is safe and lossless:
```bash
mv logs/app_$(date -u +%F).log logs/app_$(date -u +%F).log.root-owned
```
Also note `nostr-sdk==0.45.1` was installed into `.venv` during this work (it
was missing, and blocks every import of the package).
---
## 5. Enabling it on the node (when you add the API key)
The provider table is non-empty on the live node, so env seeding won't fire —
add it explicitly:
1. Admin UI → Providers → *TypeSafe* (base URL is fixed to
`https://api.typesafe.ai/v1`), paste your `api.typesafe.ai` key.
2. Or: `POST /admin/api/upstream-providers` with
`{"provider_type": "typesafe", "base_url": "https://api.typesafe.ai/v1", "api_key": "<key>"}`.
3. `jev-latest` / `jev-preview` are then catalogued automatically with the
built-in rates; override the model row if TypeSafe changes pricing.
4. Client call: `POST {node}/v1/systemone` with `{state, model, questions}`.
Containers were **not** restarted and nothing was deployed.
---
## 6. Caveats / not yet verified
* **No live call has been made** — no TypeSafe API key was available during
this work, so the upstream contract is implemented from
`docs.typesafe.ai` and the OpenRouter model metadata. Specifically
unverified against the live API: the exact `GET /v1/models` envelope
(code accepts `{"models": [...]}` and a bare list; entries keyed by `name`
or `id`) and the `release_date` format.
* The `usage` shape (`{input_tokens, output_tokens}`) is already the canonical
billing shape, so settlement needed no dialect handling.
* Not covered by tests: TypeSafe's `529 Overloaded` status (treated as a
generic 5xx; single-provider failover has nothing to fall back to).
docs/api/endpoints.md
+23 -1
View File
@@ -231,7 +231,7 @@ POST /v1/systemone
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `model` | string | Yes | - | TypeSafe model or alias (e.g. `jev-latest`) |
| `model` | string | Yes | - | TypeSafe alias (`jev-latest`, `jev-preview`) or versioned id (`jev-1.13.0`) |
| `state` | string/object/array | Yes | - | Content to evaluate |
| `questions` | map<string, Question> | Yes | - | Typed questions; answers keyed identically |
@@ -256,6 +256,28 @@ POST /v1/systemone
Billing is input-token based (output tokens are free on Jev); the response's
`usage` is the settlement seam, exactly like embeddings.
**Notes:**
- The response `model` echoes the id you requested (e.g. `jev-latest`), not the
resolved build (`jev-1.13.0`) TypeSafe returns. Routstr also adds its standard
`id`, `cost`, `metadata.routstr` and `usage.*_msats` fields.
- TypeSafe's `GET /v1/models` lists aliases only; the node additionally seeds
the known versioned ids so they can be requested directly.
- TypeSafe answers `429 Too Many Requests` and `529 Overloaded` when throttled.
Both are forwarded as upstream errors; retry with exponential backoff.
**Enabling the provider:**
1. Admin UI → Providers → *TypeSafe* (base URL is fixed to
`https://api.typesafe.ai/v1`), paste your `api.typesafe.ai` key. Or
`POST /admin/api/upstream-providers` with
`{"provider_type": "typesafe", "api_key": "<key>"}`.
2. On a node with an empty provider table, setting `TYPESAFE_API_KEY` seeds
the provider automatically.
3. `jev-latest`, `jev-preview` and `jev-1.13.0` are catalogued with the
published rate ($0.042 per million input tokens, output free). Override the
model row if TypeSafe changes pricing.
## Images (Coming Soon)
### Create Image
routstr/upstream/typesafe.py
+77 -110
View File
@@ -1,59 +1,91 @@
"""Upstream provider for the TypeSafe System One API.
TypeSafe serves "System One" decision models (Jev) at a dedicated
``POST /v1/systemone`` endpoint: the request carries a ``state`` and a map of
typed ``questions`` (noul / choice / score), and the response returns one
``answers`` entry per question plus a flat ``usage`` object
(``{"input_tokens": n, "output_tokens": n}``). That usage shape is exactly
what :func:`routstr.payment.usage.normalize_usage` already parses, so billing
needs no dialect handling — the provider's job is catalog assembly (TypeSafe's
``GET /v1/models`` lists model names but no prices) and standard forwarding.
Rates below are USD per token, matching the prices TypeSafe publishes at
https://docs.typesafe.ai/models (input is charged; output tokens are free).
Because TypeSafe's model listing does not carry pricing, the operator's DB
model row is authoritative whenever one exists; these rates seed the row.
``POST /v1/systemone`` takes ``{state, model, questions}`` and returns
``{model, answers, usage}``. The ``usage`` shape (``input_tokens`` /
``output_tokens``) is what :func:`routstr.payment.usage.normalize_usage`
already parses, so billing needs no dialect handling. This provider only
assembles the catalog: ``GET /v1/models`` lists names without prices.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from datetime import datetime
from typing import TYPE_CHECKING, Any
import httpx
from ..core.logging import get_logger
from ..payment.models import Architecture, Model, Pricing, TopProvider
from .base import BaseUpstreamProvider
if TYPE_CHECKING:
from ..core.db import UpstreamProviderRow
logger = get_logger(__name__)
# USD per token, keyed by the exact `model` value TypeSafe accepts. Aliases
# (jev-latest -> jev-1.13.0) resolve upstream, so one entry per alias keeps
# every advertised id priced even if the alias target moves.
_TYPESAFE_RATES: dict[str, tuple[float, float]] = {
"jev-latest": (0.042 / 1_000_000, 0.0),
"jev-preview": (0.042 / 1_000_000, 0.0),
}
# USD per token (https://docs.typesafe.ai/models). Output is free.
_INPUT_RATE_USD = 0.042 / 1_000_000
_OUTPUT_RATE_USD = 0.0
_DEFAULT_RATE = (0.042 / 1_000_000, 0.0)
# The listing returns aliases only; versioned ids are accepted but unlisted.
_VERSIONED_MODEL_IDS = ("jev-1.13.0",)
# Jev ingests state once and evaluates all questions against it in parallel.
# The 64k budget covers state + all questions combined; 32k applies to state +
# the single longest question. 64k is the safe reservation context.
_TYPESAFE_CONTEXT_LENGTH = 64_000
_CONTEXT_LENGTH = 64_000
_MODELS_TIMEOUT_SECONDS = 30.0
def _parse_release_date(value: object) -> int:
if not isinstance(value, str) or not value:
return 0
try:
return int(datetime.fromisoformat(value.replace("Z", "+00:00")).timestamp())
except ValueError:
return 0
def _build_model(name: str, entry: dict[str, Any] | None = None) -> Model:
entry = entry or {}
description = entry.get("description")
if not isinstance(description, str) or not description:
description = f"TypeSafe System One model {name}"
return Model(
id=name,
name=name,
created=_parse_release_date(entry.get("release_date")),
description=description,
context_length=_CONTEXT_LENGTH,
architecture=Architecture(
modality="text->decisions",
input_modalities=["text"],
output_modalities=["decisions"],
tokenizer="Other",
instruct_type=None,
),
pricing=Pricing(prompt=_INPUT_RATE_USD, completion=_OUTPUT_RATE_USD),
top_provider=TopProvider(context_length=_CONTEXT_LENGTH),
)
def _models_from_listing(data: object) -> list[Model]:
entries = data.get("models", []) if isinstance(data, dict) else data
if not isinstance(entries, list):
entries = []
models: dict[str, Model] = {}
for entry in entries:
name = entry.get("name") if isinstance(entry, dict) else None
if isinstance(name, str) and name and name not in models:
models[name] = _build_model(name, entry)
for name in _VERSIONED_MODEL_IDS:
if name not in models:
models[name] = _build_model(name)
return list(models.values())
class TypeSafeUpstreamProvider(BaseUpstreamProvider):
"""Upstream provider for the TypeSafe System One decision API.
TypeSafe exposes one evaluation endpoint (``POST /v1/systemone``) shared
by every model; the request's ``model`` field selects which one answers.
The generic chat-forwarding machinery in the base class handles the
request/response plumbing unchanged — the model id sits in the top-level
``model`` field like any OpenAI-compatible API, and the response's
``usage`` is already in the canonical billing shape.
"""
"""Upstream provider for the TypeSafe System One decision API."""
provider_type = "typesafe"
default_base_url = "https://api.typesafe.ai/v1"
@@ -89,88 +121,23 @@ class TypeSafeUpstreamProvider(BaseUpstreamProvider):
}
def transform_model_name(self, model_id: str) -> str:
"""Strip a ``typesafe/`` prefix if one was used to namespace the id."""
return model_id.removeprefix("typesafe/")
async def fetch_models(self) -> list[Model]:
"""Fetch the model list TypeSafe's account can call.
``GET /v1/models`` requires the provider's API key and returns
``{"models": [{"name", "description", "release_date"}, ...]}`` —
aliases only, with no pricing or context fields. Prices come from the
table above; the operator's DB model row overrides this pricing
whenever one exists.
"""
"""Fetch the catalog; return an empty list on failure so init never breaks."""
url = f"{self.base_url}/models"
headers = {"Authorization": f"Bearer {self.api_key}"}
try:
async with httpx.AsyncClient(timeout=30.0) as client:
async with httpx.AsyncClient(timeout=_MODELS_TIMEOUT_SECONDS) as client:
response = await client.get(url, headers=headers)
response.raise_for_status()
data = response.json()
entries = data.get("models", []) if isinstance(data, dict) else data
if not isinstance(entries, list):
return []
models: list[Model] = []
seen: set[str] = set()
for entry in entries:
if not isinstance(entry, dict):
continue
name = entry.get("name")
if not isinstance(name, str) or not name or name in seen:
continue
seen.add(name)
rate = _TYPESAFE_RATES.get(name, _DEFAULT_RATE)
# An unlisted model is priced at the default Jev rate, but a
# missing entry in the rate table is still imported enabled:
# TypeSafe only lists models the account may call, and the
# operator's DB row overrides this pricing anyway.
created = 0
release_date = entry.get("release_date")
if isinstance(release_date, str):
try:
from datetime import datetime
created = int(
datetime.fromisoformat(
release_date.replace("Z", "+00:00")
).timestamp()
)
except ValueError:
created = 0
description = entry.get("description")
if not isinstance(description, str) or not description:
description = f"TypeSafe System One model {name}"
models.append(
Model(
id=name,
name=name,
created=created,
description=description,
context_length=_TYPESAFE_CONTEXT_LENGTH,
architecture=Architecture(
modality="text->decisions",
input_modalities=["text"],
output_modalities=["decisions"],
tokenizer="Other",
instruct_type=None,
),
pricing=Pricing(
prompt=rate[0],
completion=rate[1],
),
top_provider=TopProvider(
context_length=_TYPESAFE_CONTEXT_LENGTH,
),
)
)
return models
except Exception:
# Catalog fetch failures (bad key, outage) must not break provider
# initialization; cached/DB models keep serving.
except Exception as exc:
logger.warning(
"Failed to fetch TypeSafe model catalog",
extra={"url": url, "error": str(exc)},
exc_info=True,
)
return []
return _models_from_listing(data)
tests/unit/test_typesafe_integration.py
+31 -4
View File
@@ -110,7 +110,7 @@ async def test_fetch_models_prices_the_listing() -> None:
with _patch_client(_mock_models_response(payload)):
models = await provider.fetch_models()
assert [m.id for m in models] == ["jev-latest", "jev-preview"]
assert [m.id for m in models] == ["jev-latest", "jev-preview", "jev-1.13.0"]
for model in models:
# Input priced, output free; rates usable (zero is a real price).
assert model.pricing.prompt == pytest.approx(0.042 / 1_000_000)
@@ -128,10 +128,14 @@ async def test_fetch_models_handles_error() -> None:
side_effect=RuntimeError("upstream down")
)
with _patch_client(mock_response):
with (
_patch_client(mock_response),
patch("routstr.upstream.typesafe.logger") as mock_logger,
):
models = await provider.fetch_models()
assert models == []
mock_logger.warning.assert_called_once()
@pytest.mark.asyncio
@@ -147,5 +151,28 @@ async def test_fetch_models_defaults_unknown_model_rates() -> None:
with _patch_client(_mock_models_response(payload)):
models = await provider.fetch_models()
assert len(models) == 1
assert models[0].pricing.prompt == pytest.approx(0.042 / 1_000_000)
by_id = {m.id: m for m in models}
assert "jev-2.0" in by_id
assert by_id["jev-2.0"].pricing.prompt == pytest.approx(0.042 / 1_000_000)
assert by_id["jev-2.0"].created == 0
@pytest.mark.asyncio
async def test_fetch_models_does_not_duplicate_listed_versioned_id() -> None:
provider = TypeSafeUpstreamProvider(api_key="test")
payload = {
"models": [
{
"name": "jev-1.13.0",
"description": "Jev 1.13",
"release_date": "2026-09-17T00:00:00Z",
}
]
}
with _patch_client(_mock_models_response(payload)):
models = await provider.fetch_models()
assert [m.id for m in models] == ["jev-1.13.0"]
assert models[0].description == "Jev 1.13"
assert models[0].created > 0