This commit is contained in:
redshift
2026-03-28 22:36:13 +00:00
parent e03342d119
commit 327e4999b4
3 changed files with 227 additions and 4 deletions
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "routstrd",
"version": "0.1.0",
"version": "0.1.1",
"module": "src/index.ts",
"type": "module",
"private": false,
@@ -12,7 +12,7 @@
"monitor": "bun src/index.ts monitor",
"lint": "tsc --noEmit",
"test": "bun test",
"build": "bun build src/index.ts --target=bun --outfile=dist/index.js",
"build": "mkdir -p dist/daemon && bun build src/index.ts --target=bun --outfile=dist/index.js && bun build src/daemon/index.ts --target=bun --outfile=dist/daemon/index.js",
"prepublishOnly": "bun run build"
},
"devDependencies": {
+2 -2
View File
@@ -1,7 +1,7 @@
import { LOG_FILE } from "./utils/config";
import { logger } from "./utils/logger";
import { existsSync, mkdirSync } from "fs";
import { dirname, join } from "path";
import { dirname } from "path";
export async function startDaemon(
options: { port?: string; provider?: string } = {},
@@ -41,7 +41,7 @@ export async function startDaemon(
// Use shell redirection to append stdout/stderr to log file
// Bun.file() overwrites, so we need shell >> for appending
const daemonScript = `${import.meta.dir}/daemon/index.ts`;
const daemonScript = new URL("./daemon/index.js", import.meta.url).pathname;
const shellCmd = `bun run "${daemonScript}" ${args.map(a => `'${a}'`).join(" ")} >> "${LOG_FILE}" 2>&1`;
const proc = Bun.spawn(["sh", "-c", shellCmd], {
+223
View File
@@ -0,0 +1,223 @@
# Report: why `POST /v1/messages` on port 8008 returns chat-completions chunks while port 8009 preserves Messages API format
Date: 2026-03-28
## Summary
I tested the same request against both local daemons using `scripts/test-direct-local.ts` semantics (`POST /v1/messages`, `stream: true`).
- `localhost:8009` preserves the Anthropic/OpenAI Messages-style stream as expected:
- `message_start`
- `content_block_start`
- `content_block_delta`
- `message_delta`
- `message_stop`
- `localhost:8008` returns OpenAI chat-completions streaming chunks instead:
- `object: "chat.completion.chunk"`
- `choices[].delta`
This is **not** because the SDK itself is incapable of preserving `/v1/messages`.
It happens because the two daemons call the SDK differently.
---
## What I tested
### 8008 (`../routstrd/`)
Listener:
- `../routstrd/src/daemon/index.ts`
- request handler in `../routstrd/src/daemon/http/index.ts`
Observed result for `POST http://localhost:8008/v1/messages`:
- response `content-type: text/event-stream`
- SSE payload is OpenAI chat-completions chunks (`chat.completion.chunk`)
### 8009 (`routstr-chat/scripts/routstr-daemon.ts`)
Listener:
- `scripts/routstr-daemon.ts`
Observed result for `POST http://localhost:8009/v1/messages`:
- response `content-type: text/event-stream`
- SSE payload preserves Messages API events (`message_start`, `content_block_delta`, etc.)
---
## Direct cause
### 8009 forwards the incoming path to the SDK
In `routstr-chat/scripts/routstr-daemon.ts`, the request is routed with:
```ts
await routeRequestsToNodeResponse({
modelId,
requestBody,
path: url.pathname,
headers: forwardedHeaders,
...
});
```
Because it passes:
```ts
path: url.pathname
```
an incoming request to `/v1/messages` stays `/v1/messages` all the way through the SDK and upstream provider routing.
### 8008 does **not** forward the incoming path
In `../routstrd/src/daemon/http/index.ts`, the request is routed with:
```ts
const response = await routeRequests({
modelId,
requestBody,
forcedProvider,
headers: incomingHeaders,
walletAdapter: deps.walletAdapter,
storageAdapter: deps.storageAdapter,
providerRegistry: deps.providerRegistry,
discoveryAdapter: deps.discoveryAdapter,
modelManager: deps.modelManager,
debugLevel: "DEBUG",
mode: deps.mode,
usageTrackingDriver: deps.usageTrackingDriver,
sdkStore: deps.store,
});
```
Notice: **no `path` is passed**.
In the SDK, `routeRequests()` defaults the path to:
```ts
path = "/v1/chat/completions"
```
from:
- `routstr-chat/sdk/routeRequests.ts`
So even when the client calls:
```http
POST /v1/messages
```
on port 8008, the daemon internally re-routes it as:
```http
POST /v1/chat/completions
```
That is why the upstream/provider response is converted into chat-completions format.
---
## Why the behavior differs even though both are built on the same SDK
Both daemons use the same SDK primitives, but:
- **8009** uses `routeRequestsToNodeResponse(...)` and explicitly passes `path: url.pathname`
- **8008** uses `routeRequests(...)` and relies on the SDK default path, which is `/v1/chat/completions`
So the format difference is caused by **daemon integration code**, not by a provider-specific quirk and not by an unavoidable SDK conversion.
---
## Important implementation detail
The SDK helper itself documents this default:
- `routstr-chat/sdk/routeRequests.ts`
```ts
/** Optional: API path (defaults to /v1/chat/completions) */
```
and in `resolveRouteRequestContext(...)`:
```ts
path = "/v1/chat/completions"
```
Therefore any caller that omits `path` will get chat-completions semantics by default.
---
## Evidence from runtime tests
### Port 8008
Observed streamed chunks included:
- `object: "chat.completion.chunk"`
- `choices[0].delta.content`
- final `[DONE]`
### Port 8009
Observed streamed events included:
- `event: message_start`
- `event: content_block_start`
- `event: content_block_delta`
- `event: message_delta`
- `event: message_stop`
- final `[DONE]`
This matches the code-path difference above.
---
## Recommended fix
In `../routstrd/src/daemon/http/index.ts`, pass the incoming path through to the SDK:
```ts
const response = await routeRequests({
modelId,
requestBody,
path: url.pathname,
forcedProvider,
headers: incomingHeaders,
walletAdapter: deps.walletAdapter,
storageAdapter: deps.storageAdapter,
providerRegistry: deps.providerRegistry,
discoveryAdapter: deps.discoveryAdapter,
modelManager: deps.modelManager,
debugLevel: "DEBUG",
mode: deps.mode,
usageTrackingDriver: deps.usageTrackingDriver,
sdkStore: deps.store,
});
```
This should make `POST /v1/messages` on port 8008 preserve the Messages API format, matching port 8009.
---
## Secondary note
Port 8008 currently uses `routeRequests(...)` and then manually streams the returned response body to `res`.
Port 8009 uses `routeRequestsToNodeResponse(...)` directly.
That difference is probably not the root cause here.
The root cause is specifically that **8008 drops the incoming request path and falls back to the SDK default of `/v1/chat/completions`**.
---
## Conclusion
The reason `localhost:8008/v1/messages` appears to "convert to chat completions" is:
1. `../routstrd` receives `/v1/messages`
2. its handler calls `routeRequests(...)` without `path`
3. the SDK defaults `path` to `/v1/chat/completions`
4. the upstream request is therefore made against chat completions
5. the stream returned is chat-completions SSE, not Messages API SSE
Port 8009 works because it explicitly forwards `url.pathname` into the SDK call.