Compare commits

...
22 Commits
Author SHA1 Message Date
9qeklajc ec80f39e64 Merge branch 'fix-x-cashu-misscalculation' into refresh-models
# Conflicts:
#	routstr/core/admin.py
2025-10-15 22:10:42 +02:00
9qeklajc 4327f542a0 test 2025-10-15 20:23:00 +02:00
9qeklajc b4eac33542 clean up 2025-10-14 00:16:16 +02:00
9qeklajc 9911ce9dcd fix different max cost calculation 2025-10-14 00:02:04 +02:00
Shroominic c56a06f3df fix tests 2025-10-04 20:49:53 +08:00
Shroominic 325abad736 ruff fmt 2025-10-04 20:49:45 +08:00
Shroominic f8090f5c35 dontations endpoint 2025-10-02 14:55:02 +08:00
shroominicandGitHub b2e5da4b46 Merge pull request #180 from Routstr/sh1ftred-patch-1
Fixed the bug where refund amount is below 1 sat for a sat mint
2025-09-23 12:40:43 +01:00
redshiftandGitHub 729f00caa2 Fixed the bug where refund amount is below 1 sat for a sat mint 2025-09-22 16:55:16 +00:00
Shroominic 54d7a5a247 fix periodic payouts 2025-09-22 17:54:28 +01:00
redshiftandGitHub ecbe3158c0 Merge pull request #179 from Routstr/503-mint-service-bug
Fixed the 503 mint service unavailable bug
2025-09-22 16:27:36 +00:00
redshiftandGitHub bac50f2150 Fixed the 503 mint service unavailable bug
This flag fetch all keysets from the db into the wallet, which is wrong given that we're initiating it for a mint. And load_mint sets the first keyset from all the keysets from the db, irrespective of the unit and the mint. 

Fixes #175
2025-09-22 16:25:02 +00:00
Shroominic 150f3084c1 remove need for ADMIN_PASSWORD, added in initial setup 2025-09-22 16:40:48 +01:00
Shroominic d4e1715613 improved upstream error message handling 2025-09-22 16:36:52 +01:00
Shroominic c2a26a3c00 v1/providers redirect to v1/providers/ 2025-09-22 16:36:35 +01:00
Shroominic 8e0ae6cbc8 filter out openrouter spam models 2025-09-22 13:46:07 +01:00
Shroominic 18a055ee2c add models.json batch paste 2025-09-22 12:58:53 +01:00
Shroominic c5289364e4 models admin dashboard 2025-09-22 12:47:39 +01:00
Shroominic 5fb583be55 v0.1.4-dev 2025-09-22 11:11:36 +01:00
shroominicandGitHub 4bf02226dc Merge pull request #172 from Routstr/fix-max-tokens-string-int
fix max-tokens coming in as string
2025-09-16 13:05:39 +01:00
Shroominic c8f4e7d3e5 fix max-tokens coming in as string 2025-09-16 13:00:38 +01:00
shroominicandGitHub 3d4a8e0e14 Merge pull request #169 from Routstr/dev
## Routstr v0.1.3 — 2025-09-15

### Highlights

- **DB-backed settings with live updates**: Env vars seed on first run; DB is the source of truth (except `DATABASE_URL`). Manage via Admin UI or `PATCH /admin/api/settings`.
- **Models moved to database** with background sats-pricing computation and optional periodic refresh.
- **Smarter max-cost reservation**: Discount based on prompt tokens and `max_tokens` using `TOLERANCE_PERCENTAGE`, with a `MIN_REQUEST_MSAT` floor.
- **Faster startup and provider discovery**; `.onion` endpoints use Tor proxy by default.

### Breaking changes

- **Configuration is DB-first** after bootstrap. Env changes won’t auto-apply (besides `DATABASE_URL`).
- **Models endpoint**: Use `/v1/models` for canonical model info; `/v1/info` keeps an empty `models` field for back-compat.
- **Legacy pricing envs** are auto-mapped but deprecated:
  - `MODEL_BASED_PRICING` → `!FIXED_PRICING`
  - `COST_PER_REQUEST` → `FIXED_COST_PER_REQUEST`
  - `COST_PER_1K_*` → `FIXED_PER_1K_*`

### Added

- **SettingsService** with env→computed→DB merge; runtime edits; app metadata (name/description) applied to OpenAPI.
- **Automatic migrations** on startup; new tables: `settings`, `models`.
- **Per-model sats pricing** and per-request maximums computed in the background.
- **Pricing controls**: `FIXED_PRICING`, `FIXED_COST_PER_REQUEST`, optional per-1k token overrides, `MIN_REQUEST_MSAT`.
- **Azure chat support**: Optional `CHAT_COMPLETIONS_API_VERSION` adds `api-version` to `/chat/completions`.
- **Derive `NPUB` from `NSEC`** during bootstrap if not provided.

### Changed / Improvements

- **Proxy header hardening**: Strip sensitive headers; replace Authorization with upstream key when configured.
- **Discovery**: Read `RELAYS` from settings; Tor proxy defaults for `.onion` via `TOR_PROXY_URL`; faster announcement fetching.
- **Admin**: Improved auth/log UX; fixed log search; show mint balances in dashboard.
- **Performance**: Optimized startup; lazy DB info loading.

### Fixed

- **Refunds**: Correct msat→sat conversion; configurable refund cache TTL via `REFUND_CACHE_TTL_SECONDS`; better error mapping for mint outages.
- **Balances**: Reserved balance reporting and negative-reserved edge cases.
- **Tests**: Updated to patch settings and use fixed-pricing flags.

### Upgrade notes (from v0.1.2)

1. Ensure `DATABASE_URL`, `UPSTREAM_BASE_URL`, and `ADMIN_PASSWORD` are set. Optionally set `UPSTREAM_API_KEY`, `RELAYS`, `CASHU_MINTS`, `TOR_PROXY_URL`.
2. Start the service; automatic Alembic migrations will run (`settings`, `models` tables).
3. Configure settings via Admin UI (`/admin`) or `PATCH /admin/api/settings`. Env changes after first run won’t apply automatically.
4. Prefer new pricing envs or a `models.json` at `MODELS_PATH`. Legacy envs are mapped but will be removed in a future release.
5. Optional tuning: `TOLERANCE_PERCENTAGE`, `MIN_REQUEST_MSAT`, `PRICING_REFRESH_INTERVAL_SECONDS`, `MODELS_REFRESH_INTERVAL_SECONDS`, `ENABLE_*_REFRESH`.
6. For Azure, set `CHAT_COMPLETIONS_API_VERSION`.

### References

- Configuration: `docs/getting-started/configuration.md`
- API: `docs/api/endpoints.md`
- Custom pricing: `docs/advanced/custom-pricing.md`

### Stats and credits

- 42 files changed, +1819/−832 lines.
- Merges: #170 token-discount-max-cost, #171 move-models-to-db.
- Contributors: Shroominic.
2025-09-16 12:43:54 +01:00
17 changed files with 1375 additions and 107 deletions
+1 -1
View File
@@ -347,7 +347,7 @@ GET /health
Response:
{
"status": "healthy",
"version": "0.1.3",
"version": "0.1.4",
"timestamp": "2024-01-01T00:00:00Z",
"checks": {
"database": "ok",
+1 -1
View File
@@ -348,7 +348,7 @@ Project metadata and dependencies:
```toml
[project]
name = "routstr"
version = "0.1.3"
version = "0.1.4"
dependencies = [
"fastapi[standard]>=0.115",
"sqlmodel>=0.0.24",
+1 -1
View File
@@ -67,7 +67,7 @@ You should see:
{
"name": "ARoutstrNode",
"description": "A Routstr Node",
"version": "0.1.3",
"version": "0.1.4",
"npub": "",
"mints": ["https://mint.minibits.cash/Bitcoin"],
"models": {...}
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "routstr"
version = "0.1.3"
version = "0.1.4"
description = "Payment proxy for your LLM endpoint using cashu and nostr."
readme = "README.md"
requires-python = ">=3.11"
+26 -10
View File
@@ -8,12 +8,15 @@ from pydantic import BaseModel
from .auth import validate_bearer_key
from .core.db import ApiKey, AsyncSession, get_session
from .core.logging import get_logger
from .core.settings import settings
from .wallet import credit_balance, send_to_lnurl, send_token
from .wallet import credit_balance, recieve_token, send_to_lnurl, send_token
router = APIRouter()
balance_router = APIRouter(prefix="/v1/balance")
logger = get_logger(__name__)
async def get_key_from_header(
authorization: Annotated[str, Header(...)],
@@ -152,14 +155,19 @@ async def refund_wallet_endpoint(
key: ApiKey = await validate_bearer_key(bearer_value, session)
remaining_balance_msats: int = key.balance
if remaining_balance_msats <= 0:
if key.refund_currency == "sat":
remaining_balance = remaining_balance_msats // 1000
else:
remaining_balance = remaining_balance_msats
if remaining_balance_msats > 0 and remaining_balance <= 0:
raise HTTPException(status_code=400, detail="Balance too small to refund")
elif remaining_balance <= 0:
raise HTTPException(status_code=400, detail="No balance to refund")
# Perform refund operation first, before modifying balance
try:
if key.refund_address:
if key.refund_currency == "sat":
remaining_balance = remaining_balance_msats // 1000
from .core.settings import settings as global_settings
await send_to_lnurl(
@@ -170,14 +178,9 @@ async def refund_wallet_endpoint(
)
result = {"recipient": key.refund_address}
else:
refund_amount = (
remaining_balance_msats // 1000
if key.refund_currency == "sat"
else remaining_balance_msats
)
refund_currency = key.refund_currency or "sat"
token = await send_token(
refund_amount, refund_currency, key.refund_mint_url
remaining_balance, refund_currency, key.refund_mint_url
)
result = {"token": token}
@@ -210,6 +213,19 @@ async def refund_wallet_endpoint(
return result
@router.post("/donate")
async def donate(token: str, ref: str | None = None) -> str:
try:
amount, unit, _ = await recieve_token(token)
if ref:
logger.info(
"donation received", extra={"ref": ref, "amount": amount, "unit": unit}
)
return "Thanks!"
except Exception:
return "Invalid token."
@router.api_route(
"/{path:path}",
methods=["GET", "POST", "PUT", "DELETE"],
+919 -4
View File
@@ -8,6 +8,8 @@ from fastapi.responses import HTMLResponse
from pydantic import BaseModel
from sqlmodel import select
from ..payment.models import Model, get_model_by_id, list_models
from ..payment.models import sync_models_with_api
from ..wallet import (
fetch_all_balances,
get_proofs_per_mint_and_unit,
@@ -15,7 +17,7 @@ from ..wallet import (
send_token,
slow_filter_spend_proofs,
)
from .db import ApiKey, create_session
from .db import ApiKey, ModelRow, create_session
from .logging import get_logger
from .settings import SettingsService, settings
@@ -163,6 +165,28 @@ async def update_settings(request: Request, update: SettingsUpdate) -> dict:
return data
class SetupRequest(BaseModel):
password: str
@admin_router.post("/api/setup")
async def initial_setup(request: Request, payload: SetupRequest) -> dict[str, object]:
try:
current = SettingsService.get()
except Exception:
current = settings
if getattr(current, "admin_password", ""):
raise HTTPException(status_code=409, detail="Admin password already set")
pw = (payload.password or "").strip()
if len(pw) < 8:
raise HTTPException(
status_code=400, detail="Password must be at least 8 characters"
)
async with create_session() as session:
await SettingsService.update({"admin_password": pw}, session)
return {"ok": True}
class WithdrawRequest(BaseModel):
amount: int
mint_url: str | None = None
@@ -205,6 +229,67 @@ def login_form() -> str:
"""
def setup_form() -> str:
return """<!DOCTYPE html>
<html>
<head>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; display: flex; justify-content: center; align-items: center; height: 100vh; background: #f5f7fa; }
.setup-card { background: white; padding: 2.5rem; border-radius: 12px; box-shadow: 0 10px 25px rgba(0,0,0,0.1); width: 360px; }
h2 { margin-bottom: 1.25rem; color: #1a202c; text-align: center; }
p { color: #4a5568; font-size: 0.95rem; margin-bottom: 1rem; text-align: center; }
input[type="password"] { width: 100%; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px; font-size: 16px; transition: border 0.2s; }
input[type="password"]:focus { outline: none; border-color: #4299e1; }
button { width: 100%; padding: 12px; margin-top: 1rem; background: #4299e1; color: white; border: none; border-radius: 6px; font-size: 16px; font-weight: 600; cursor: pointer; transition: all 0.2s; }
button:hover { background: #3182ce; transform: translateY(-1px); box-shadow: 0 4px 6px rgba(0,0,0,0.1); }
.error { color: #e53e3e; margin-top: 10px; text-align: center; }
</style>
<script>
async function handleSetupSubmit(e) {
e.preventDefault();
const pw = document.getElementById('password').value;
const pw2 = document.getElementById('password2').value;
const err = document.getElementById('error');
err.textContent = '';
if (pw.length < 8) { err.textContent = 'Password must be at least 8 characters'; return; }
if (pw !== pw2) { err.textContent = 'Passwords do not match'; return; }
try {
const resp = await fetch('/admin/api/setup', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ password: pw })
});
if (!resp.ok) {
let msg = 'Failed to set password';
try { const j = await resp.json(); if (j && j.detail) msg = typeof j.detail === 'string' ? j.detail : JSON.stringify(j.detail); } catch(_) {}
throw new Error(msg);
}
document.cookie = `admin_password=${pw}; path=/; max-age=86400; samesite=lax`;
window.location.replace('/admin');
} catch (e) {
err.textContent = e.message || String(e);
}
}
</script>
</head>
<body>
<div class="setup-card">
<h2>🔧 Initial Admin Setup</h2>
<p>Create a secure password for your admin dashboard.</p>
<form onsubmit="handleSetupSubmit(event)">
<input type="password" id="password" placeholder="New Password" required autofocus>
<input type="password" id="password2" placeholder="Confirm Password" required style="margin-top:10px;">
<button type="submit">Set Password</button>
<div id="error" class="error"></div>
</form>
</div>
</body>
</html>
"""
def info(content: str) -> str:
return f"""<!DOCTYPE html>
<html>
@@ -232,7 +317,7 @@ def admin_auth() -> str:
except Exception:
admin_pw = os.getenv("ADMIN_PASSWORD", "")
if admin_pw == "":
return info("Please set a secure ADMIN_PASSWORD= in your ENV variables.")
return setup_form()
else:
return login_form()
@@ -417,6 +502,64 @@ async def dashboard(request: Request) -> str:
window.location.href = `/admin/logs/${requestId}`;
}
function openSyncModelsModal() {
const modal = document.getElementById('sync-models-modal');
modal.style.display = 'block';
}
function closeSyncModelsModal() {
const modal = document.getElementById('sync-models-modal');
modal.style.display = 'none';
}
async function syncModels() {
const deleteRemoved = document.getElementById('delete-removed-models').checked;
const button = document.getElementById('sync-models-btn');
const resultDiv = document.getElementById('sync-models-result');
button.disabled = true;
button.textContent = 'Syncing...';
resultDiv.style.display = 'none';
try {
const response = await fetch('/admin/api/sync_models', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
credentials: 'same-origin',
body: JSON.stringify({
delete_removed: deleteRemoved
})
});
if (response.ok) {
const data = await response.json();
resultDiv.innerHTML = `<strong>✅ Success!</strong><br>${data.message}`;
resultDiv.style.display = 'block';
resultDiv.style.backgroundColor = '#d4edda';
resultDiv.style.borderColor = '#c3e6cb';
resultDiv.style.color = '#155724';
} else {
const errorData = await response.json();
resultDiv.innerHTML = `<strong>❌ Error:</strong><br>${errorData.detail || 'Unknown error'}`;
resultDiv.style.display = 'block';
resultDiv.style.backgroundColor = '#f8d7da';
resultDiv.style.borderColor = '#f5c6cb';
resultDiv.style.color = '#721c24';
}
} catch (error) {
resultDiv.innerHTML = `<strong>❌ Error:</strong><br>${error.message}`;
resultDiv.style.display = 'block';
resultDiv.style.backgroundColor = '#f8d7da';
resultDiv.style.borderColor = '#f5c6cb';
resultDiv.style.color = '#721c24';
} finally {
button.disabled = false;
button.textContent = 'Sync Models';
}
}
async function openSettingsModal() {
const modal = document.getElementById('settings-modal');
const textarea = document.getElementById('settings-json');
@@ -498,12 +641,15 @@ async def dashboard(request: Request) -> str:
const withdrawModal = document.getElementById('withdraw-modal');
const investigateModal = document.getElementById('investigate-modal');
const settingsModal = document.getElementById('settings-modal');
const syncModelsModal = document.getElementById('sync-models-modal');
if (event.target == withdrawModal) {
closeWithdrawModal();
} else if (event.target == investigateModal) {
closeInvestigateModal();
} else if (event.target == settingsModal) {
closeSettingsModal();
} else if (event.target == syncModelsModal) {
closeSyncModelsModal();
}
}
</script>
@@ -529,9 +675,35 @@ async def dashboard(request: Request) -> str:
<button class="investigate-btn" onclick="openInvestigateModal()">
🔍 Investigate Logs
</button>
<button onclick="window.location.href='/admin/models'">
🧩 Edit Models
</button>
<button onclick="openSettingsModal()">
⚙️ Settings
</button>
<button onclick="openSyncModelsModal()">
🔄 Sync Models
</button>
<div id="sync-models-modal" class="modal">
<div class="modal-content">
<span class="close" onclick="closeSyncModelsModal()">&times;</span>
<h3>Sync Models from API</h3>
<p>This will fetch the latest models from OpenRouter and update the database.</p>
<div style="margin: 15px 0;">
<label style="display: flex; align-items: center; gap: 10px; cursor: pointer;">
<input type="checkbox" id="delete-removed-models" style="width: auto; margin: 0;">
<span>Delete models that are no longer available</span>
</label>
<p style="font-size: 0.85rem; color: #718096; margin-top: 5px; margin-left: 30px;">
⚠️ Warning: This will permanently remove models that are no longer in the API.
</p>
</div>
<div id="sync-models-result" style="display: none; padding: 12px; border-radius: 6px; margin: 15px 0; border: 1px solid;"></div>
<button id="sync-models-btn" onclick="syncModels()">🔄 Sync Models</button>
<button onclick="closeSyncModelsModal()" style="background-color: #718096;">Cancel</button>
</div>
</div>
<div id="withdraw-modal" class="modal">
<div class="modal-content">
@@ -721,7 +893,6 @@ async def view_logs(request: Request, request_id: str) -> str:
async def withdraw(
request: Request, withdraw_request: WithdrawRequest
) -> dict[str, str]:
# Get wallet and check balance
from .settings import settings as global_settings
wallet = await get_wallet(
@@ -749,6 +920,750 @@ async def withdraw(
)
return {"token": token}
class SyncModelsRequest(BaseModel):
delete_removed: bool = False
@admin_router.post("/api/sync_models", dependencies=[Depends(require_admin_api)])
async def sync_models(request: Request, sync_request: SyncModelsRequest) -> dict:
try:
src = settings.source or None
source_filter = src if src and src.strip() else None
except Exception:
source_filter = None
logger.info(
"Manual models sync triggered",
extra={"delete_removed": sync_request.delete_removed},
)
counts = await sync_models_with_api(
source_filter=source_filter, delete_removed=sync_request.delete_removed
)
return {
"success": True,
"inserted": counts["inserted"],
"updated": counts["updated"],
"deleted": counts["deleted"],
"message": f"Synced: {counts['inserted']} inserted, {counts['updated']} updated, {counts['deleted']} deleted",
}
DASHBOARD_MODELS_JS: str = """<!--html-->
<script>
let modelsList = [];
let currentQuery = '';
let editModelCreated = 0;
async function fetchModels() {
const tableBody = document.getElementById('models-tbody');
tableBody.innerHTML = '<tr><td colspan="1" style="color:#718096;">Loading…</td></tr>';
try {
const resp = await fetch('/admin/api/models', { credentials: 'same-origin' });
if (!resp.ok) throw new Error('HTTP ' + resp.status);
const data = await resp.json();
modelsList = data;
renderModelsTable();
} catch (e) {
tableBody.innerHTML = '<tr><td colspan="1" style="color:#e53e3e;">Failed to load models: ' + e.message + '</td></tr>';
}
}
function renderModelsTable() {
const tableBody = document.getElementById('models-tbody');
if (!Array.isArray(modelsList) || !modelsList.length) {
tableBody.innerHTML = '<tr><td colspan="1" style="color:#718096;">No models found</td></tr>';
return;
}
const q = (currentQuery || '').trim().toLowerCase();
const items = q ? modelsList.filter(m => {
const id = (m.id || '').toLowerCase();
return id.includes(q);
}) : modelsList;
if (!items.length) {
tableBody.innerHTML = '<tr><td colspan="1" style="color:#718096;">No models match your search</td></tr>';
return;
}
const rows = items.map(m => `
<tr>
<td>
<div style="display:flex; align-items:center; justify-content:space-between; gap:12px;">
<span style="font-family:monospace; word-break: break-all;">${m.id}</span>
<span>
<button onclick=\"openModelEditor('${m.id}')\">Edit</button>
<button onclick=\"deleteModel(event, '${m.id}')\" style=\"background:#e53e3e;\">Delete</button>
</span>
</div>
</td>
</tr>
`).join('');
tableBody.innerHTML = rows;
}
function handleSearch(query) {
currentQuery = String(query || '');
renderModelsTable();
}
async function deleteModel(ev, modelId) {
if (!confirm('Are you sure you want to delete model: ' + modelId + '?')) return;
const btn = ev && ev.currentTarget ? ev.currentTarget : null;
if (btn) { btn.disabled = true; btn.textContent = 'Deleting…'; }
const errorBox = document.getElementById('models-error');
if (errorBox) { errorBox.style.display = 'none'; errorBox.textContent = ''; }
try {
const resp = await fetch('/admin/api/models/' + encodeURIComponent(modelId), {
method: 'DELETE',
credentials: 'same-origin'
});
if (!resp.ok) {
let errText = 'Failed to delete model';
try { const err = await resp.json(); if (err && err.detail) errText = typeof err.detail === 'string' ? err.detail : JSON.stringify(err.detail); } catch (_) {}
throw new Error(errText);
}
await fetchModels();
} catch (e) {
if (errorBox) { errorBox.style.display = 'block'; errorBox.textContent = e.message; }
else { alert(e.message); }
} finally {
if (btn) { btn.disabled = false; btn.textContent = 'Delete'; }
}
}
async function deleteAllModels() {
if (!confirm('Are you absolutely sure you want to delete ALL models?')) return;
const errorBox = document.getElementById('models-error');
if (errorBox) { errorBox.style.display = 'none'; errorBox.textContent = ''; }
try {
const resp = await fetch('/admin/api/models', { method: 'DELETE', credentials: 'same-origin' });
if (!resp.ok) {
let errText = 'Failed to delete all models';
try { const err = await resp.json(); if (err && err.detail) errText = typeof err.detail === 'string' ? err.detail : JSON.stringify(err.detail); } catch (_) {}
throw new Error(errText);
}
await fetchModels();
} catch (e) {
if (errorBox) { errorBox.style.display = 'block'; errorBox.textContent = e.message; }
else { alert(e.message); }
}
}
async function openModelEditor(modelId) {
const modal = document.getElementById('model-edit-modal');
const errorBox = document.getElementById('model-error');
errorBox.style.display = 'none';
errorBox.textContent = '';
document.getElementById('model-id').value = modelId;
try {
const resp = await fetch('/admin/api/models/' + encodeURIComponent(modelId), { credentials: 'same-origin' });
if (!resp.ok) throw new Error('HTTP ' + resp.status);
const m = await resp.json();
document.getElementById('model-name').value = m.name || '';
document.getElementById('model-description').value = m.description || '';
editModelCreated = m.created || Math.floor(Date.now()/1000);
document.getElementById('model-context').value = m.context_length || 0;
document.getElementById('model-architecture').value = JSON.stringify(m.architecture || {
modality: '',
input_modalities: [],
output_modalities: [],
tokenizer: '',
instruct_type: null
}, null, 2);
const pricingObj = m.pricing || {
prompt: 0.0,
completion: 0.0,
request: 0.0,
image: 0.0,
web_search: 0.0,
internal_reasoning: 0.0
};
delete pricingObj.max_prompt_cost;
delete pricingObj.max_completion_cost;
delete pricingObj.max_cost;
document.getElementById('model-pricing').value = JSON.stringify(pricingObj, null, 2);
document.getElementById('model-per-request-limits').value = m.per_request_limits ? JSON.stringify(m.per_request_limits, null, 2) : '';
document.getElementById('model-top-provider').value = m.top_provider ? JSON.stringify(m.top_provider, null, 2) : '';
} catch (e) {
errorBox.style.display = 'block';
errorBox.textContent = 'Failed to load model: ' + e.message;
}
modal.style.display = 'block';
}
function closeModelEditor() {
const modal = document.getElementById('model-edit-modal');
modal.style.display = 'none';
}
async function saveModel() {
const modelId = document.getElementById('model-id').value;
const errorBox = document.getElementById('model-error');
errorBox.style.display = 'none';
errorBox.style.color = '#e53e3e';
let payload = {};
try {
const name = document.getElementById('model-name').value;
const description = document.getElementById('model-description').value;
const contextLength = parseInt(document.getElementById('model-context').value) || 0;
const architecture = JSON.parse(document.getElementById('model-architecture').value || '{}');
const pricing = JSON.parse(document.getElementById('model-pricing').value || '{}');
const perReqLimitsStr = document.getElementById('model-per-request-limits').value.trim();
const topProviderStr = document.getElementById('model-top-provider').value.trim();
payload = {
id: modelId,
name: name,
description: description,
created: editModelCreated,
context_length: contextLength,
architecture: architecture,
pricing: pricing,
per_request_limits: perReqLimitsStr === '' ? null : JSON.parse(perReqLimitsStr),
top_provider: topProviderStr === '' ? null : JSON.parse(topProviderStr)
};
} catch (e) {
errorBox.style.display = 'block';
errorBox.textContent = 'Invalid input: ' + e.message;
return;
}
const saveBtn = document.getElementById('model-save-btn');
saveBtn.disabled = true;
saveBtn.textContent = 'Saving…';
try {
const resp = await fetch('/admin/api/models/' + encodeURIComponent(modelId), {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify(payload)
});
if (!resp.ok) {
let errText = 'Failed to save model';
try { const err = await resp.json(); if (err && err.detail) errText = typeof err.detail === 'string' ? err.detail : JSON.stringify(err.detail); } catch (_) {}
throw new Error(errText);
}
await resp.json();
closeModelEditor();
fetchModels();
} catch (e) {
errorBox.style.display = 'block';
errorBox.textContent = e.message;
} finally {
saveBtn.disabled = false;
saveBtn.textContent = 'Save';
}
}
async function deleteAllModels() {
if (!confirm('Are you absolutely sure you want to delete ALL models?')) return;
const errorBox = document.getElementById('models-error');
if (errorBox) { errorBox.style.display = 'none'; errorBox.textContent = ''; }
try {
const resp = await fetch('/admin/api/models', { method: 'DELETE', credentials: 'same-origin' });
if (!resp.ok) {
let errText = 'Failed to delete all models';
try { const err = await resp.json(); if (err && err.detail) errText = typeof err.detail === 'string' ? err.detail : JSON.stringify(err.detail); } catch (_) {}
throw new Error(errText);
}
await fetchModels();
} catch (e) {
if (errorBox) { errorBox.style.display = 'block'; errorBox.textContent = e.message; }
else { alert(e.message); }
}
}
window.addEventListener('DOMContentLoaded', fetchModels);
window.onclick = function(event) {
const modal = document.getElementById('model-edit-modal');
if (event.target == modal) {
closeModelEditor();
}
const createModal = document.getElementById('model-create-modal');
if (event.target == createModal) {
closeCreateModel();
}
const batchModal = document.getElementById('model-batch-modal');
if (event.target == batchModal) {
closeBatchModal();
}
}
function openCreateModel() {
const modal = document.getElementById('model-create-modal');
const err = document.getElementById('model-create-error');
err.style.display = 'none';
err.textContent = '';
const defaults = {
architecture: {
modality: 'text',
input_modalities: ['text'],
output_modalities: ['text'],
tokenizer: '',
instruct_type: null
},
pricing: {
prompt: 0.0,
completion: 0.0,
request: 0.0,
image: 0.0,
web_search: 0.0,
internal_reasoning: 0.0
}
};
document.getElementById('create-id').value = '';
document.getElementById('create-name').value = '';
document.getElementById('create-description').value = '';
document.getElementById('create-context').value = 0;
document.getElementById('create-architecture').value = JSON.stringify(defaults.architecture, null, 2);
document.getElementById('create-pricing').value = JSON.stringify(defaults.pricing, null, 2);
document.getElementById('create-per-request-limits').value = '';
document.getElementById('create-top-provider').value = '';
modal.style.display = 'block';
}
function closeCreateModel() {
const modal = document.getElementById('model-create-modal');
modal.style.display = 'none';
}
async function createModel() {
const err = document.getElementById('model-create-error');
err.style.display = 'none';
err.textContent = '';
const btn = document.getElementById('model-create-btn');
btn.disabled = true;
btn.textContent = 'Creating…';
try {
const id = document.getElementById('create-id').value.trim();
const name = document.getElementById('create-name').value.trim();
const description = document.getElementById('create-description').value.trim();
const contextStr = document.getElementById('create-context').value.trim();
const architectureStr = document.getElementById('create-architecture').value.trim();
const pricingStr = document.getElementById('create-pricing').value.trim();
const perReqLimitsStr = document.getElementById('create-per-request-limits').value.trim();
const topProviderStr = document.getElementById('create-top-provider').value.trim();
if (!id) throw new Error('ID is required');
if (!name) throw new Error('Name is required');
if (!description) throw new Error('Description is required');
const created = Math.floor(Date.now()/1000);
if (!contextStr) throw new Error('Context length is required');
const context_length = parseInt(contextStr);
if (!architectureStr) throw new Error('Architecture JSON is required');
const architecture = JSON.parse(architectureStr);
if (!pricingStr) throw new Error('Pricing JSON is required');
const pricing = JSON.parse(pricingStr);
const payload = {
id: id,
name: name,
description: description,
created: created,
context_length: context_length,
architecture: architecture,
pricing: pricing,
per_request_limits: perReqLimitsStr ? JSON.parse(perReqLimitsStr) : null,
top_provider: topProviderStr ? JSON.parse(topProviderStr) : null
};
const resp = await fetch('/admin/api/models', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify(payload)
});
if (!resp.ok) {
let errText = 'Failed to create model';
try { const e = await resp.json(); if (e && e.detail) errText = typeof e.detail === 'string' ? e.detail : JSON.stringify(e.detail); } catch(_) {}
throw new Error(errText);
}
closeCreateModel();
await fetchModels();
} catch (e) {
err.style.display = 'block';
err.textContent = e.message || String(e);
} finally {
btn.disabled = false;
btn.textContent = ' Create';
}
}
function openBatchModal() {
const modal = document.getElementById('model-batch-modal');
if (!modal) { alert('Batch modal not found'); return; }
const err = document.getElementById('batch-error');
if (err) { err.style.display = 'none'; err.textContent = ''; }
const textarea = document.getElementById('batch-json');
if (textarea && !textarea.value.trim()) {
const sample = {
models: [
{
id: 'provider/model-id',
name: 'Model Name',
description: 'Description',
created: Math.floor(Date.now()/1000),
context_length: 0,
architecture: { modality: 'text', input_modalities: ['text'], output_modalities: ['text'], tokenizer: '', instruct_type: null },
pricing: { prompt: 0.0, completion: 0.0, request: 0.0, image: 0.0, web_search: 0.0, internal_reasoning: 0.0 },
per_request_limits: null,
top_provider: null
}
]
};
textarea.value = JSON.stringify(sample, null, 2);
}
modal.style.display = 'block';
}
function closeBatchModal() {
const modal = document.getElementById('model-batch-modal');
if (modal) modal.style.display = 'none';
}
async function performBatchAdd() {
const textarea = document.getElementById('batch-json');
const err = document.getElementById('batch-error');
const btn = document.getElementById('batch-submit-btn');
if (err) { err.style.display = 'none'; err.textContent = ''; }
if (btn) { btn.disabled = true; btn.textContent = 'Adding…'; }
try {
if (!textarea) throw new Error('Input not found');
const data = JSON.parse(textarea.value);
if (!data || !Array.isArray(data.models) || data.models.length === 0) {
throw new Error('Payload must include a non-empty "models" array');
}
const resp = await fetch('/admin/api/models/batch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify(data)
});
if (!resp.ok) {
let errText = 'Failed to add models';
try { const e = await resp.json(); if (e && e.detail) errText = typeof e.detail === 'string' ? e.detail : JSON.stringify(e.detail); } catch(_) {}
throw new Error(errText);
}
closeBatchModal();
await fetchModels();
} catch (e) {
if (err) { err.style.display = 'block'; err.textContent = e.message || String(e); }
else { alert(e.message || String(e)); }
} finally {
if (btn) { btn.disabled = false; btn.textContent = ' Add Models'; }
}
}
</script>
"""
def models_page() -> str:
return (
f"""<!DOCTYPE html>
<html>
<head>
<style>{DASHBOARD_CSS}</style>
{DASHBOARD_MODELS_JS}
</head>
"""
+ """<!--html-->
<body>
<a href="/admin" class="back-btn">← Back to Dashboard</a>
<h1>Models</h1>
<div class="balance-card">
<h2>Models Table</h2>
<div style="display:flex; gap:10px; align-items:center; margin: 8px 0 12px;">
<input type="text" id="models-search" placeholder="Search by id" oninput="handleSearch(this.value)" style="flex:1; padding: 10px; border: 2px solid #e2e8f0; border-radius: 6px;">
<button onclick="openCreateModel()"> Create Model</button>
</div>
<div id="models-error" style="display:none; margin: 8px 0; color:#e53e3e;"></div>
<table>
<thead>
<tr>
<th>ID</th>
</tr>
</thead>
<tbody id="models-tbody">
<tr><td colspan="1" style="color:#718096;">Loading…</td></tr>
</tbody>
</table>
<div style="margin-top: 12px; display:flex; justify-content:flex-end;">
<button onclick="deleteAllModels()" style="background:#e53e3e;">🗑️ Delete All</button>
<button onclick="openBatchModal()" style="background:#4a5568;">📥 Batch Add</button>
</div>
</div>
<div id="model-edit-modal" class="modal">
<div class="modal-content" style="max-width: 720px;">
<span class="close" onclick="closeModelEditor()">&times;</span>
<h3>Edit Model: <span id="model-id" style="font-family:monospace;"></span></h3>
<div id="model-error" style="display:none; margin: 10px 0; color:#e53e3e;"></div>
<label>ID</label>
<input type="text" id="model-id" placeholder="model-id" disabled>
<label>Name</label>
<input type="text" id="model-name" placeholder="Name">
<label>Description</label>
<input type="text" id="model-description" placeholder="Description">
<label>Context Length</label>
<input type="number" id="model-context" min="0" placeholder="Context length">
<h4 style="margin-top:10px;">Architecture (JSON)</h4>
<textarea id="model-architecture" style="width:100%; min-height: 160px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<h4 style="margin-top:10px;">Pricing (JSON)</h4>
<textarea id="model-pricing" style="width:100%; min-height: 160px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<h4 style="margin-top:10px;">Per Request Limits (JSON, optional) — leave blank to clear</h4>
<textarea id="model-per-request-limits" style="width:100%; min-height: 120px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<h4 style="margin-top:10px;">Top Provider (JSON, optional) — leave blank to clear</h4>
<textarea id="model-top-provider" style="width:100%; min-height: 120px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<div style="margin-top: 12px; display: flex; gap: 10px;">
<button id="model-save-btn" onclick="saveModel()">💾 Save</button>
<button onclick="closeModelEditor()" style="background-color: #718096;">Cancel</button>
</div>
</div>
</div>
<div id="model-create-modal" class="modal">
<div class="modal-content" style="max-width: 720px;">
<span class="close" onclick="closeCreateModel()">&times;</span>
<h3>Create Model</h3>
<div id="model-create-error" style="display:none; margin: 10px 0; color:#e53e3e;"></div>
<label>ID</label>
<input type="text" id="create-id" placeholder="model-id">
<label>Name</label>
<input type="text" id="create-name" placeholder="Name">
<label>Description</label>
<input type="text" id="create-description" placeholder="Description">
<label>Context Length</label>
<input type="number" id="create-context" min="0" placeholder="Context length">
<h4 style="margin-top:10px;">Architecture (JSON)</h4>
<textarea id="create-architecture" style="width:100%; min-height: 160px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<h4 style="margin-top:10px;">Pricing (JSON)</h4>
<textarea id="create-pricing" style="width:100%; min-height: 160px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<h4 style="margin-top:10px;">Per Request Limits (JSON, optional)</h4>
<textarea id="create-per-request-limits" style="width:100%; min-height: 120px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<h4 style="margin-top:10px;">Top Provider (JSON, optional)</h4>
<textarea id="create-top-provider" style="width:100%; min-height: 120px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<div style="margin-top: 12px; display: flex; gap: 10px;">
<button id="model-create-btn" onclick="createModel()"> Create</button>
<button onclick="closeCreateModel()" style="background-color: #718096;">Cancel</button>
</div>
</div>
</div>
<div id="model-batch-modal" class="modal">
<div class="modal-content" style="max-width: 840px;">
<span class="close" onclick="closeBatchModal()">&times;</span>
<h3>Batch Add Models</h3>
<p style="font-size: 0.9rem; color: #718096; margin: 6px 0 10px;">Paste JSON in the format like models.example.json</p>
<div id="batch-error" style="display:none; margin: 10px 0; color:#e53e3e;"></div>
<textarea id="batch-json" style="width:100%; min-height: 320px; font-family: 'Monaco', monospace; font-size: 13px; background:#f8fafc; color:#2d3748; padding: 12px; border: 2px solid #e2e8f0; border-radius: 6px;"></textarea>
<div style="margin-top: 12px; display:flex; gap:10px;">
<button id="batch-submit-btn" onclick="performBatchAdd()"> Add Models</button>
<button onclick="closeBatchModal()" style="background-color:#718096;">Cancel</button>
</div>
</div>
</div>
</body>
</html>
"""
)
@admin_router.get("/models", response_class=HTMLResponse)
async def admin_models(request: Request) -> str:
if is_admin_authenticated(request):
return models_page()
return admin_auth()
@admin_router.get("/api/models", dependencies=[Depends(require_admin_api)])
async def get_models_admin_api(request: Request) -> list[dict[str, object]]:
items = await list_models()
return [m.dict() for m in items] # type: ignore
@admin_router.post("/api/models", dependencies=[Depends(require_admin_api)])
async def create_model_admin_api(payload: Model) -> dict[str, object]:
async with create_session() as session:
exists = await session.get(ModelRow, payload.id)
if exists:
raise HTTPException(
status_code=409, detail="Model with this ID already exists"
)
pricing_dict = payload.pricing.dict()
for k in ("max_prompt_cost", "max_completion_cost", "max_cost"):
pricing_dict.pop(k, None)
row = ModelRow(
id=payload.id,
name=payload.name,
description=payload.description,
created=int(payload.created),
context_length=int(payload.context_length),
architecture=json.dumps(payload.architecture.dict()),
pricing=json.dumps(pricing_dict),
sats_pricing=None,
per_request_limits=(
json.dumps(payload.per_request_limits)
if payload.per_request_limits is not None
else None
),
top_provider=(
json.dumps(payload.top_provider.dict())
if payload.top_provider
else None
),
)
session.add(row)
await session.commit()
created_model = await get_model_by_id(payload.id)
return created_model.dict() if created_model else {"id": payload.id} # type: ignore
@admin_router.post("/api/models/batch", dependencies=[Depends(require_admin_api)])
async def batch_create_models(payload: dict[str, object]) -> dict[str, int]:
models = payload.get("models")
if not isinstance(models, list) or not models:
raise HTTPException(
status_code=400, detail="Payload must include non-empty 'models' array"
)
created = 0
skipped = 0
async with create_session() as session:
for m in models:
try:
model = Model(**m) # type: ignore[arg-type]
except Exception:
skipped += 1
continue
exists = await session.get(ModelRow, model.id)
if exists:
skipped += 1
continue
pricing_dict = model.pricing.dict()
for k in ("max_prompt_cost", "max_completion_cost", "max_cost"):
pricing_dict.pop(k, None)
row = ModelRow(
id=model.id,
name=model.name,
description=model.description,
created=int(model.created),
context_length=int(model.context_length),
architecture=json.dumps(model.architecture.dict()),
pricing=json.dumps(pricing_dict),
sats_pricing=None,
per_request_limits=(
json.dumps(model.per_request_limits)
if model.per_request_limits is not None
else None
),
top_provider=(
json.dumps(model.top_provider.dict())
if model.top_provider
else None
),
)
session.add(row)
created += 1
if created:
await session.commit()
return {"created": created, "skipped": skipped}
@admin_router.get(
"/api/models/{model_id:path}", dependencies=[Depends(require_admin_api)]
)
async def get_model_admin_api(model_id: str) -> dict[str, object]:
model = await get_model_by_id(model_id)
if not model:
raise HTTPException(status_code=404, detail="Model not found")
return model.dict() # type: ignore
@admin_router.patch(
"/api/models/{model_id:path}", dependencies=[Depends(require_admin_api)]
)
async def update_model_admin_api(model_id: str, payload: Model) -> dict[str, object]:
if payload.id != model_id:
raise HTTPException(status_code=400, detail="Path id does not match payload id")
async with create_session() as session:
row = await session.get(ModelRow, model_id)
if not row:
raise HTTPException(status_code=404, detail="Model not found")
row.name = payload.name
row.description = payload.description
row.created = int(payload.created)
row.context_length = int(payload.context_length)
row.architecture = json.dumps(payload.architecture.dict())
pricing_dict = payload.pricing.dict()
for k in ("max_prompt_cost", "max_completion_cost", "max_cost"):
pricing_dict.pop(k, None)
row.pricing = json.dumps(pricing_dict)
row.sats_pricing = None
row.per_request_limits = (
json.dumps(payload.per_request_limits)
if payload.per_request_limits is not None
else None
)
row.top_provider = (
json.dumps(payload.top_provider.dict()) if payload.top_provider else None
)
session.add(row)
await session.commit()
updated = await get_model_by_id(model_id)
if not updated:
raise HTTPException(status_code=404, detail="Model not found after update")
return updated.dict() # type: ignore
@admin_router.delete(
"/api/models/{model_id:path}", dependencies=[Depends(require_admin_api)]
)
async def delete_model_admin_api(model_id: str) -> dict[str, object]:
async with create_session() as session:
row = await session.get(ModelRow, model_id)
if not row:
raise HTTPException(status_code=404, detail="Model not found")
await session.delete(row)
await session.commit()
return {"ok": True, "deleted_id": model_id}
@admin_router.delete("/api/models", dependencies=[Depends(require_admin_api)])
async def delete_all_models_admin_api() -> dict[str, object]:
async with create_session() as session:
result = await session.exec(select(ModelRow)) # type: ignore
rows = result.all()
for row in rows:
await session.delete(row) # type: ignore
await session.commit()
return {"ok": True, "deleted": "all"}
DASHBOARD_CSS: str = """
* { margin: 0; padding: 0; box-sizing: border-box; }
@@ -785,7 +1700,7 @@ button:disabled { background: #a0aec0; cursor: not-allowed; transform: none; }
.copy-btn { background: #38a169; padding: 6px 12px; font-size: 14px; }
.copy-btn:hover { background: #2f855a; }
.modal { display: none; position: fixed; z-index: 1000; left: 0; top: 0; width: 100%; height: 100%; background: rgba(0,0,0,0.5); backdrop-filter: blur(4px); }
.modal-content { background: white; margin: 10% auto; padding: 2rem; width: 90%; max-width: 400px; border-radius: 12px; box-shadow: 0 20px 25px -5px rgba(0,0,0,0.1); animation: slideIn 0.3s ease; }
.modal-content { background: white; margin: 5% auto; padding: 0.75rem 1rem 2.25rem; width: 90%; max-width: 720px; max-height: 85vh; overflow-y: auto; border-radius: 12px; box-shadow: 0 20px 25px -5px rgba(0,0,0,0.1); animation: slideIn 0.3s ease; }
@keyframes slideIn { from { transform: translateY(-20px); opacity: 0; } to { transform: translateY(0); opacity: 1; } }
.close { color: #a0aec0; float: right; font-size: 28px; font-weight: bold; cursor: pointer; margin: -10px -10px 0 0; }
.close:hover { color: #2d3748; }
+7 -3
View File
@@ -30,7 +30,7 @@ from .settings import settings as global_settings
setup_logging()
logger = get_logger(__name__)
__version__ = "0.1.3"
__version__ = "0.1.4-dev"
@asynccontextmanager
@@ -67,8 +67,7 @@ async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
await ensure_models_bootstrapped()
pricing_task = asyncio.create_task(update_sats_pricing())
if global_settings.models_refresh_interval_seconds > 0:
models_refresh_task = asyncio.create_task(refresh_models_periodically())
models_refresh_task = asyncio.create_task(refresh_models_periodically())
payout_task = asyncio.create_task(periodic_payout())
nip91_task = asyncio.create_task(announce_provider())
providers_task = asyncio.create_task(providers_cache_refresher())
@@ -158,6 +157,11 @@ async def admin_redirect() -> RedirectResponse:
return RedirectResponse("/admin/")
@app.get("/v1/providers")
async def providers() -> RedirectResponse:
return RedirectResponse("/v1/providers/")
app.include_router(models_router)
app.include_router(admin_router)
app.include_router(balance_router)
+2 -1
View File
@@ -64,10 +64,11 @@ class Settings(BaseSettings):
default=120, env="PRICING_REFRESH_INTERVAL_SECONDS"
)
models_refresh_interval_seconds: int = Field(
default=0, env="MODELS_REFRESH_INTERVAL_SECONDS"
default=30, env="MODELS_REFRESH_INTERVAL_SECONDS"
)
enable_pricing_refresh: bool = Field(default=True, env="ENABLE_PRICING_REFRESH")
enable_models_refresh: bool = Field(default=True, env="ENABLE_MODELS_REFRESH")
delete_removed_models: bool = Field(default=False, env="DELETE_REMOVED_MODELS")
refund_cache_ttl_seconds: int = Field(default=3600, env="REFUND_CACHE_TTL_SECONDS")
# Logging
+31 -18
View File
@@ -11,7 +11,7 @@ from ..core import get_logger
from ..core.db import ModelRow
from ..core.settings import settings
from ..wallet import deserialize_token_from_string
from .models import Pricing
from .models import Pricing, compute_effective_max_cost_msats
logger = get_logger(__name__)
@@ -72,7 +72,8 @@ def check_token_balance(headers: dict, body: dict, max_cost_for_model: int) -> N
token_obj.amount if token_obj.unit == "msat" else token_obj.amount * 1000
)
if max_cost_for_model > amount_msat:
fee_buffer = 60
if max_cost_for_model > amount_msat + fee_buffer:
raise HTTPException(
status_code=413,
detail={
@@ -133,16 +134,17 @@ async def get_max_cost_for_model(
row = await session.get(ModelRow, model)
if row and row.sats_pricing:
try:
sats = Pricing(**json.loads(row.sats_pricing)) # type: ignore
max_cost = sats.max_cost * 1000 * (1 - settings.tolerance_percentage / 100)
logger.debug(
"Found model-specific max cost",
extra={"model": model, "max_cost_msats": max_cost},
)
calculated_msats = int(max_cost)
return max(settings.min_request_msat, calculated_msats)
sats_dict = json.loads(row.sats_pricing)
except Exception:
pass
sats_dict = None
if isinstance(sats_dict, dict):
effective_msats = compute_effective_max_cost_msats(sats_dict)
if effective_msats > 0:
logger.debug(
"Found model-specific max cost",
extra={"model": model, "max_cost_msats": effective_msats},
)
return effective_msats
logger.warning(
"Model pricing not found, using fixed cost",
@@ -183,14 +185,25 @@ async def calculate_discounted_max_cost(
else:
adjusted = adjusted + math.ceil(-estimated_prompt_delta_sats * 1000)
if max_tokens := body.get("max_tokens"):
estimated_completion_delta_sats = (
max_completion_allowed_sats - max_tokens * model_pricing.completion
)
if estimated_completion_delta_sats >= 0:
adjusted = adjusted - math.floor(estimated_completion_delta_sats * 1000)
max_tokens_raw = body.get("max_tokens", None)
if max_tokens_raw is not None:
try:
max_tokens_int = int(max_tokens_raw)
except (TypeError, ValueError):
logger.warning(
"Invalid max_tokens; ignoring in cost adjustment",
extra={"max_tokens": str(max_tokens_raw)[:64], "model": model},
)
else:
adjusted = adjusted + math.ceil(-estimated_completion_delta_sats * 1000)
estimated_completion_delta_sats = (
max_completion_allowed_sats - max_tokens_int * model_pricing.completion
)
if estimated_completion_delta_sats >= 0:
adjusted = adjusted - math.floor(estimated_completion_delta_sats * 1000)
else:
adjusted = adjusted + math.ceil(-estimated_completion_delta_sats * 1000)
adjusted = min(max_cost_for_model, adjusted)
logger.debug(
"Discounted max cost computed",
+14 -1
View File
@@ -230,7 +230,11 @@ async def get_lnurl_invoice(
async def raw_send_to_lnurl(
wallet: Wallet, proofs: list[Proof], lnurl: str, unit: str
wallet: Wallet,
proofs: list[Proof],
lnurl: str,
unit: str,
amount: int | None = None,
) -> int:
"""Send funds to an LNURL address.
@@ -255,6 +259,11 @@ async def raw_send_to_lnurl(
paid = await wallet.send_to_lnurl("user@getalby.com", 50, unit="usd")
"""
total_balance = sum(proof.amount for proof in proofs)
if amount and total_balance < amount:
raise ValueError("Amount to send is higher than available proofs.")
else:
assert isinstance(amount, int)
total_balance = amount
lnurl_data = await get_lnurl_data(lnurl)
if unit == "sat":
@@ -285,6 +294,10 @@ async def raw_send_to_lnurl(
melt_quote_resp = await wallet.melt_quote(
invoice=bolt11_invoice, amount_msat=final_amount
)
if amount:
proofs, _ = await wallet.select_to_send(proofs, amount, set_reserved=True)
_ = await wallet.melt(
proofs=proofs,
invoice=bolt11_invoice,
+149 -30
View File
@@ -58,6 +58,39 @@ class Model(BaseModel):
top_provider: TopProvider | None = None
def compute_effective_max_cost_msats(
pricing: Pricing | dict | None,
) -> int:
if pricing is None:
try:
return max(1, int(settings.min_request_msat))
except Exception:
return 1
pricing_obj = (
pricing if isinstance(pricing, Pricing) else Pricing.parse_obj(pricing) # type: ignore[arg-type]
)
try:
tolerance = float(getattr(settings, "tolerance_percentage", 0.0))
except Exception:
tolerance = 0.0
tolerance_factor = max(0.0, 1.0 - tolerance / 100.0)
try:
min_request_msat = max(1, int(settings.min_request_msat))
except Exception:
min_request_msat = 1
base_msats = int(float(pricing_obj.max_cost or 0.0) * 1000.0 * tolerance_factor)
if base_msats <= 0:
return min_request_msat
return max(min_request_msat, base_msats)
def fetch_openrouter_models(source_filter: str | None = None) -> list[dict]:
"""Fetches model information from OpenRouter API."""
base_url = "https://openrouter.ai/api/v1"
@@ -83,6 +116,9 @@ def fetch_openrouter_models(source_filter: str | None = None) -> list[dict]:
"(free)" in model.get("name", "")
or model_id == "openrouter/auto"
or model_id == "google/gemini-2.5-pro-exp-03-25"
or model_id == "opengvlab/internvl3-78b"
or model_id == "openrouter/sonoma-dusk-alpha"
or model_id == "openrouter/sonoma-sky-alpha"
):
continue
@@ -94,6 +130,14 @@ def fetch_openrouter_models(source_filter: str | None = None) -> list[dict]:
return []
def is_openrouter_upstream() -> bool:
try:
base = (settings.upstream_base_url or "").strip().rstrip("/")
except Exception:
return False
return base.lower() == "https://openrouter.ai/api/v1"
def load_models() -> list[Model]:
"""Load model definitions from a JSON file or auto-generate from OpenRouter API.
@@ -119,7 +163,13 @@ def load_models() -> list[Model]:
logger.error(f"Error loading models from {models_path}: {e}")
# Fall through to auto-generation
# Auto-generate models from OpenRouter API
# Only auto-generate from OpenRouter when upstream is OpenRouter
if not is_openrouter_upstream():
logger.info(
"Skipping auto-generation from OpenRouter because upstream_base_url is not https://openrouter.ai/api/v1"
)
return []
logger.info("Auto-generating models from OpenRouter API")
try:
source_filter = settings.source or None
@@ -160,6 +210,10 @@ def _row_to_model(row: ModelRow) -> Model:
except Exception:
pass
if isinstance(sats_pricing, dict):
effective_msats = compute_effective_max_cost_msats(sats_pricing)
sats_pricing["max_cost"] = effective_msats / 1000.0
return Model(
id=row.id,
name=row.name,
@@ -240,7 +294,7 @@ async def ensure_models_bootstrapped() -> None:
except Exception as e:
logger.error(f"Error loading models from {models_path}: {e}")
if not models_to_insert:
if not models_to_insert and is_openrouter_upstream():
logger.info("Bootstrapping models from OpenRouter API")
source_filter = None
try:
@@ -249,6 +303,10 @@ async def ensure_models_bootstrapped() -> None:
except Exception:
pass
models_to_insert = fetch_openrouter_models(source_filter=source_filter)
elif not models_to_insert:
logger.info(
"No models.json found and upstream is not OpenRouter; skipping bootstrap"
)
for m in models_to_insert:
try:
@@ -379,17 +437,87 @@ async def update_sats_pricing() -> None:
break
async def sync_models_with_api(
source_filter: str | None = None, delete_removed: bool = False
) -> dict[str, int]:
"""Fetch models from OpenRouter and sync with database.
Args:
source_filter: Optional source filter (e.g., 'anthropic')
delete_removed: If True, delete models that no longer exist in API
Returns:
Dict with counts: inserted, updated, deleted
"""
models = fetch_openrouter_models(source_filter=source_filter)
if not models:
return {"inserted": 0, "updated": 0, "deleted": 0}
async with create_session() as s:
result = await s.exec(select(ModelRow)) # type: ignore
existing_rows = {row.id: row for row in result.all()}
fetched_ids = set()
inserted = 0
updated = 0
for m in models:
try:
model = Model(**m) # type: ignore
except Exception:
continue
fetched_ids.add(model.id)
payload = _model_to_row_payload(model)
if model.id not in existing_rows:
try:
s.add(ModelRow(**payload)) # type: ignore
inserted += 1
except Exception:
pass
else:
existing_row = existing_rows[model.id]
changed = False
for key, value in payload.items():
if getattr(existing_row, key) != value:
setattr(existing_row, key, value)
changed = True
if changed:
s.add(existing_row)
updated += 1
deleted = 0
if delete_removed:
for existing_id in existing_rows:
if existing_id not in fetched_ids:
row_to_delete = existing_rows[existing_id]
await s.delete(row_to_delete)
deleted += 1
if inserted or updated or deleted:
await s.commit()
return {"inserted": inserted, "updated": updated, "deleted": deleted}
async def refresh_models_periodically() -> None:
"""Background task: periodically fetch OpenRouter models and insert new ones.
"""Background task: periodically fetch OpenRouter models and sync with database.
- Respects optional SOURCE filter from settings
- Does not overwrite existing rows
- Updates existing models with new information
- Optionally deletes models no longer in API (if settings.delete_removed_models)
- Sleeps according to settings.models_refresh_interval_seconds; disabled when 0
"""
interval = getattr(settings, "models_refresh_interval_seconds", 0)
if not interval or interval <= 0:
return
# Only refresh from OpenRouter when upstream is OpenRouter
if not is_openrouter_upstream():
logger.info("Skipping models refresh: upstream_base_url is not OpenRouter")
return
while True:
try:
try:
@@ -403,33 +531,24 @@ async def refresh_models_periodically() -> None:
except Exception:
source_filter = None
models = fetch_openrouter_models(source_filter=source_filter)
if not models:
await asyncio.sleep(interval)
continue
try:
delete_removed = getattr(settings, "delete_removed_models", False)
except Exception:
delete_removed = False
async with create_session() as s:
result = await s.exec(select(ModelRow.id)) # type: ignore
existing_ids = {
row[0] if isinstance(row, tuple) else row for row in result.all()
}
inserted = 0
for m in models:
try:
model = Model(**m) # type: ignore
except Exception:
continue
if model.id in existing_ids:
continue
payload = _model_to_row_payload(model)
try:
s.add(ModelRow(**payload)) # type: ignore
except Exception:
pass
inserted += 1
if inserted:
await s.commit()
logger.info(f"Inserted {inserted} new models from OpenRouter")
counts = await sync_models_with_api(
source_filter=source_filter, delete_removed=delete_removed
)
if counts["inserted"] or counts["updated"] or counts["deleted"]:
logger.info(
"Models synced",
extra={
"inserted": counts["inserted"],
"updated": counts["updated"],
"deleted": counts["deleted"],
},
)
except asyncio.CancelledError:
break
except Exception as e:
+117 -10
View File
@@ -30,6 +30,104 @@ logger = get_logger(__name__)
proxy_router = APIRouter()
def _extract_upstream_error_message(body_bytes: bytes) -> tuple[str, str | None]:
"""Extract a human-friendly message and optional upstream error code from a response body."""
message: str = "Upstream request failed"
upstream_code: str | None = None
if not body_bytes:
return message, upstream_code
try:
data = json.loads(body_bytes)
if isinstance(data, dict):
err = data.get("error")
if isinstance(err, dict):
raw_msg = err.get("message") or err.get("detail") or err.get("error")
if isinstance(raw_msg, (str, int, float)):
message = str(raw_msg)
upstream_code_raw = err.get("code") or err.get("type")
if isinstance(upstream_code_raw, (str, int, float)):
upstream_code = str(upstream_code_raw)
elif "message" in data and isinstance(data["message"], (str, int, float)):
message = str(data["message"]) # type: ignore[arg-type]
elif "detail" in data and isinstance(data["detail"], (str, int, float)):
message = str(data["detail"]) # type: ignore[arg-type]
except Exception:
preview = body_bytes.decode("utf-8", errors="ignore").strip()
if preview:
message = preview[:500]
return message, upstream_code
async def map_upstream_error_response(
request: Request,
path: str,
upstream_response: httpx.Response,
) -> Response:
"""Map upstream non-200 responses to standardized error responses.
- Known cases are mapped to friendly messages and appropriate status codes
- Unknown errors are converted to a generic 502
"""
status_code = upstream_response.status_code
headers = dict(upstream_response.headers)
content_type = headers.get("content-type", "")
try:
body_bytes = await upstream_response.aread()
except Exception:
body_bytes = b""
message, upstream_code = _extract_upstream_error_message(body_bytes)
lowered_message = message.lower()
lowered_code = (upstream_code or "").lower()
error_type = "upstream_error"
mapped_status = 502
# Specific mappings
if status_code in (400, 422):
error_type = "invalid_request_error"
mapped_status = 400
elif status_code in (401, 403):
error_type = "upstream_auth_error"
mapped_status = 502
elif status_code == 404:
# Many providers return 404 for unknown models or routes
if path.endswith("chat/completions"):
error_type = "invalid_model"
mapped_status = 400
if not message or message == "Upstream request failed":
message = "Requested model is not available upstream"
elif "model" in lowered_message or "model" in lowered_code:
error_type = "invalid_model"
mapped_status = 400
if not message or message == "Upstream request failed":
message = "Requested model is not available upstream"
else:
error_type = "upstream_error"
mapped_status = 502
elif status_code == 429:
error_type = "rate_limit_exceeded"
mapped_status = 429
elif status_code >= 500:
error_type = "upstream_error"
mapped_status = 502
# Include upstream content type hint in logs for diagnostics
logger.debug(
"Mapped upstream error",
extra={
"path": path,
"upstream_status": status_code,
"mapped_status": mapped_status,
"error_type": error_type,
"upstream_content_type": content_type,
"message_preview": message[:200],
},
)
return create_error_response(error_type, message, mapped_status, request=request)
async def handle_streaming_chat_completion(
response: httpx.Response, key: ApiKey, max_cost_for_model: int
) -> StreamingResponse:
@@ -362,6 +460,17 @@ async def forward_to_upstream(
},
)
# Map and return errors immediately to provide clear messages
if response.status_code != 200:
try:
mapped_error = await map_upstream_error_response(
request, path, response
)
finally:
await response.aclose()
await client.aclose()
return mapped_error
# For chat completions, we need to handle token-based pricing
if path.endswith("chat/completions"):
# Check if client requested streaming
@@ -655,18 +764,10 @@ async def proxy(
"upstream_headers": response.headers
if hasattr(response, "headers")
else None,
"upstream_response": response.body
if hasattr(response, "body")
else None,
},
)
request_id = (
request.state.request_id if hasattr(request.state, "request_id") else None
)
raise HTTPException(
status_code=502,
detail=f"Upstream request failed, please contact support with request id: {request_id}",
)
# Return the mapped error response generated earlier rather than masking with 502
return response
return response
@@ -786,6 +887,12 @@ async def forward_get_to_upstream(
"GET request forwarded successfully",
extra={"path": path, "status_code": response.status_code},
)
if response.status_code != 200:
try:
mapped = await map_upstream_error_response(request, path, response)
finally:
await response.aclose()
return mapped
return StreamingResponse(
response.aiter_bytes(),
+7 -5
View File
@@ -152,9 +152,7 @@ async def get_wallet(mint_url: str, unit: str = "sat", load: bool = True) -> Wal
global _wallets
id = f"{mint_url}_{unit}"
if id not in _wallets:
_wallets[id] = await Wallet.with_db(
mint_url, db=".wallet", load_all_keysets=True, unit=unit
)
_wallets[id] = await Wallet.with_db(mint_url, db=".wallet", unit=unit)
if load:
await _wallets[id].load_mint()
@@ -299,7 +297,7 @@ async def periodic_payout() -> None:
logger.error("RECEIVE_LN_ADDRESS is not set, skipping payout")
return
while True:
await asyncio.sleep(60 * 5)
await asyncio.sleep(60 * 15)
try:
async with db.create_session() as session:
for mint_url in settings.cashu_mints:
@@ -319,7 +317,11 @@ async def periodic_payout() -> None:
min_amount = 210 if unit == "sat" else 210000
if available_balance > min_amount:
amount_received = await raw_send_to_lnurl(
wallet, proofs, settings.receive_ln_address, unit
wallet,
proofs,
settings.receive_ln_address,
unit,
amount=available_balance,
)
logger.info(
"Payout sent successfully",
+1
View File
@@ -63,6 +63,7 @@ else:
# Set test environment variables before importing the app
os.environ.update(test_env)
os.environ.pop("ADMIN_PASSWORD", None)
from routstr.core.db import ApiKey, get_session # noqa: E402
from routstr.core.main import app, lifespan # noqa: E402
@@ -271,36 +271,23 @@ async def test_models_endpoint_accept_headers(integration_client: AsyncClient) -
async def test_admin_endpoint_unauthenticated(
integration_client: AsyncClient, db_snapshot: Any
) -> None:
"""Test GET /admin/ endpoint without authentication"""
# Capture initial database state
"""Test GET /admin/ endpoint without authentication shows setup form"""
await db_snapshot.capture()
response = await integration_client.get("/admin/")
# Should return 200 with login form (not 401/403)
assert response.status_code == 200
assert "text/html" in response.headers["content-type"]
# Response should be HTML
html_content = response.text
assert "<!DOCTYPE html>" in html_content
assert "<html>" in html_content
assert "<form" in html_content
assert 'type="password"' in html_content
assert "password" in html_content.lower()
assert "<script>" in html_content or "<script " in html_content
assert "Initial Admin Setup" in html_content or "setup" in html_content.lower()
# Either shows login form or message about setting ADMIN_PASSWORD
if "ADMIN_PASSWORD" in html_content:
# When ADMIN_PASSWORD is not set, it shows a message
assert "Please set a secure ADMIN_PASSWORD" in html_content
else:
# When ADMIN_PASSWORD is set, it shows a login form
assert "<form" in html_content
assert 'type="password"' in html_content
assert "password" in html_content.lower()
assert "login" in html_content.lower()
# Should have JavaScript for form handling
assert "<script>" in html_content or "<script " in html_content
# Verify no database state changes
diff = await db_snapshot.diff()
assert len(diff["api_keys"]["added"]) == 0
assert len(diff["api_keys"]["removed"]) == 0
+91 -1
View File
@@ -1,12 +1,20 @@
import os
from unittest.mock import AsyncMock, Mock, patch
import pytest
from fastapi import HTTPException
# Set required env vars before importing
os.environ["UPSTREAM_BASE_URL"] = "http://test"
os.environ["UPSTREAM_API_KEY"] = "test"
from routstr.core.settings import settings # noqa: E402
from routstr.payment.helpers import get_max_cost_for_model # noqa: E402
from routstr.payment.helpers import ( # noqa: E402
calculate_discounted_max_cost,
check_token_balance,
get_max_cost_for_model,
)
from routstr.payment.models import Pricing # noqa: E402
async def test_get_max_cost_for_model_known() -> None:
@@ -73,3 +81,85 @@ async def test_get_max_cost_for_model_tolerance() -> None:
with patch.object(settings, "tolerance_percentage", 10):
cost = await get_max_cost_for_model("gpt-4", session=mock_session)
assert cost == 450000 # 500 sats * 1000 * 0.9 = 450000
async def test_calculate_discounted_max_cost_no_session() -> None:
with patch.object(settings, "fixed_pricing", False):
result = await calculate_discounted_max_cost(123, {}, session=None)
assert result == 123
async def test_calculate_discounted_max_cost_clamped(
monkeypatch: pytest.MonkeyPatch,
) -> None:
mock_session = AsyncMock()
mock_exec_result = Mock()
mock_exec_result.all = Mock(return_value=[("gpt-4",)])
mock_session.exec.return_value = mock_exec_result
pricing = {
"prompt": 0.0,
"completion": 0.0,
"request": 0.0,
"image": 0.0,
"web_search": 0.0,
"internal_reasoning": 0.0,
"max_prompt_cost": 100.0,
"max_completion_cost": 200.0,
"max_cost": 300.0,
}
async def mock_get_model_cost_info(*args, **kwargs): # type: ignore
return Pricing(**pricing)
monkeypatch.setattr(
"routstr.payment.helpers.get_model_cost_info", mock_get_model_cost_info
)
with patch.object(settings, "fixed_pricing", False):
with patch.object(settings, "tolerance_percentage", 0):
result = await calculate_discounted_max_cost(
320000,
{"model": "gpt-4", "max_tokens": 10, "messages": ["one"]},
session=mock_session,
)
assert 0 <= result <= 320000
async def test_check_token_balance_with_fee_buffer(
monkeypatch: pytest.MonkeyPatch,
) -> None:
class Token:
unit = "msat"
amount = 320400
def mock_deserialize(_value: str) -> Token:
return Token()
monkeypatch.setattr(
"routstr.payment.helpers.deserialize_token_from_string", mock_deserialize
)
headers = {"x-cashu": "token"}
body = {"model": "gpt-4"}
check_token_balance(headers, body, max_cost_for_model=320450)
def test_check_token_balance_insufficient(monkeypatch: pytest.MonkeyPatch) -> None:
class Token:
unit = "msat"
amount = 320200
def mock_deserialize(_value: str) -> Token:
return Token()
monkeypatch.setattr(
"routstr.payment.helpers.deserialize_token_from_string", mock_deserialize
)
headers = {"x-cashu": "token"}
body = {"model": "gpt-4"}
with pytest.raises(HTTPException) as exc:
check_token_balance(headers, body, max_cost_for_model=320450)
assert exc.value.status_code == 413
detail = exc.value.detail # type: ignore[assignment]
assert isinstance(detail, dict)
assert detail.get("type") == "minimum_balance_required"
Generated
+1 -1
View File
@@ -1783,7 +1783,7 @@ wheels = [
[[package]]
name = "routstr"
version = "0.1.3"
version = "0.1.4"
source = { editable = "." }
dependencies = [
{ name = "aiosqlite" },