laantungir/routstrd
1
0
Fork 0
mirror of https://github.com/Routstr/routstrd.git synced 2026-10-05 12:28:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(proxy): inject max_completion_tokens instead of max_tokens

Browse Source 
When the client omits an output-token cap, routstrd injects one so the
SDK prices against a bounded completion budget instead of the provider's
worst-case max_completion_cost. It previously injected the legacy
max_tokens field, which is deprecated in the OpenAI chat-completions
spec and rejected by reasoning models.

Inject max_completion_tokens (the OpenAI-standard field) instead, and
only when neither max_completion_tokens nor max_tokens is already
present, so an explicit client cap is never overridden. The Responses
API path is unchanged (max_output_tokens).
This commit is contained in:
redshift
2026-09-07 14:17:52 +02:00
parent a2639f3bc6
commit c36de22cfd
1 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/daemon/http/index.ts
+8 -3
View File
@@ -1712,15 +1712,20 @@ export function createDaemonRequestHandler(deps: {
// limit. Without this, the SDK prices at the provider's worst-case
// max_completion_cost, which varies widely across providers (2.3× for
// kimi-k3) and balloons during provider failover. Chat/completions use
// max_tokens; the OpenAI Responses API uses max_output_tokens.
// max_completion_tokens (the OpenAI-standard field; legacy max_tokens is
// deprecated and rejected by reasoning models); the Responses API uses
// max_output_tokens.
if (deps.maxTokens > 0) {
const isResponsesPath = url.pathname.includes("/responses");
if (isResponsesPath) {
if (typeof bodyObj.max_output_tokens !== "number") {
bodyObj.max_output_tokens = deps.maxTokens;
}
} else if (typeof bodyObj.max_tokens !== "number") {
bodyObj.max_tokens = deps.maxTokens;
} else if (
typeof bodyObj.max_completion_tokens !== "number" &&
typeof bodyObj.max_tokens !== "number"
) {
bodyObj.max_completion_tokens = deps.maxTokens;
}
}