mirror of
https://github.com/Routstr/routstr-core.git
synced 2026-10-06 12:38:23 +00:00
9qeklajc
9e9c2bde57
Replace the fragile `re.split(b"data: ")` streaming parser with a buffered, event-delimited parser in both the chat-completion and Responses-API streamers. Events are accumulated until the SSE blank-line delimiter, so parsing is independent of network chunk boundaries. Fixes: - OpenRouter `: OPENROUTER PROCESSING` keepalive comments no longer leak to clients as `data: : ...` (the `Unexpected token ':'` client crash). - JSON payloads split across TCP reads are reassembled before parsing. - CRLF framing (Gemini native alt=sse) handled. - Combined content+usage chunks (Gemini thinking models over the OpenAI-compat endpoint) forward content once and still report usage in the cost trailer, instead of dropping the assistant message. - Multi-line non-JSON `data` blocks are re-prefixed per line so they stay valid SSE framing for the client. Adds tests/unit/test_streaming_sse_providers.py driving the real generator against per-provider on-the-wire framing, and switches the integration token-mint fallback to secrets.token_hex to kill a PRNG/clock collision flake.
Routstr Payment Proxy
Routstr is a decentralized protocol for permissionless, private, and censorship-resistant AI inference. It combines Nostr for discovery and Cashu for private Bitcoin micropayments.
This repo contains Routstr Core: a FastAPI-based reverse proxy that sits in front of OpenAI-compatible APIs and handles pay-per-request billing.
Start Here
- Overview: https://docs.routstr.com/overview/
- Provider Guide: https://docs.routstr.com/provider/quickstart/
- User Guide: https://docs.routstr.com/user-guide/introduction/
Basic Usage
If you are a user/developer, you just point an OpenAI-compatible SDK at a Routstr node and pay with a Cashu token.
OpenAI SDK
from openai import OpenAI
client = OpenAI(
base_url="https://api.routstr.com/v1",
api_key="cashuBo2FteCJodHRwczovL21...",
)
response = client.chat.completions.create(
model="gpt-5-nano",
messages=[{"role": "user", "content": "hello"}],
)
print(response.choices[0].message.content)
cURL
curl https://api.routstr.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "x-cashu: cashuBo2FteCJodHRwczovL21..." \
-d '{
"model": "gpt-5-nano",
"messages": [{"role": "user", "content": "hello"}]
}'
Quick Start (Docker)
If you are a node runner, start a Routstr Core instance using Docker Compose:
-
Prepare your
.env:ADMIN_PASSWORD=mysecretpassword NAME="My AI Node" DESCRIPTION="Fast access to models" NSEC=yournsec RECEIVE_LN_ADDRESS=yourname@wallet.com -
Start the services:
docker compose up -d -
Configure: Open http://localhost:8000/admin/ to connect your AI providers and set pricing.
For full instructions, see the Provider Quick Start Guide.
Development
make setup
cp .env.example .env
fastapi run routstr