routstrd/docs/linking-sessions-to-request-logs.md

Linking pi sessions to request/response logs

When requestResponseLogging is enabled, the daemon records every proxied upstream call as two files that share one id:

• requests/<id>.json — method, url, baseUrl, redacted headers, request body
• responses/<id>.jsonl — one JSON event per line: response_start, chunk, end / error

The id is minted in src/daemon/request-response-log-sink.ts (makeId()): the request's arrival time with :/. swapped for -, plus 8 hex chars, e.g. 2026-09-30T09-22-35-288Z-60c8e8cf. The request's id and the response's requestLogId are the same string, so the request and response logs are paired by filename stem and never need matching against each other.

Separately, pi records its own transcript per working directory. The two are written by different processes, so the session has no log id and the logs have no session id. This document explains the two keys that recover the link exactly, and how to run the mapper on redtop.

TL;DR

# on redtop
bun scripts/link-session-logs.ts <session.jsonl> /home/user/.routstrd/reqRes

Every assistant turn prints the response log it produced. The request log is the same stem under requests/.

Where the data lives

	redtop (ssh redtop)	default
request/response logs	/home/user/.routstrd/reqRes	~/.routstrd/request-response-logs
log root on disk	requestResponseLogging.dir in ~/.routstrd/config.json	REQUEST_RESPONSE_LOGS_DIR (src/utils/config.ts)
pi sessions	/home/user/.pi/agent/sessions/--<cwd>--/*.jsonl	~/.pi/agent/sessions/…
pi provider config	/home/user/.pi/agent/models.json (providers.routstr)	~/.pi/agent/models.json

On redtop the log root is set explicitly:

// ~/.routstrd/config.json
"requestResponseLogging": { "enabled": true, "dir": "/home/user/.routstrd/reqRes" }

pi reaches the daemon through the routstr provider that routstrd clients add pi-agent writes into models.json (src/integrations/pi.ts, configPath: ~/.pi/agent/models.json). So the sessions and the logs on redtop are two views of the same traffic.

Logs are stored uncompressed on redtop (requests/*.json, responses/*.jsonl). Elsewhere they may be brotli-compressed (.jsonl.br); the mapper handles both.

The join keys

1. responseId ⇄ response stream id (primary, exact)

Each assistant message in a pi session stores the upstream response id:

// session .jsonl
{ "type": "message",
  "timestamp": "2026-09-30T09:22:36.852Z",
  "message": { "role": "assistant", "provider": "routstr",
               "responseId": "chatcmpl-b5e868d7b747dd8be5a2fbc0b1313507", … } }

The response log replays the raw upstream SSE stream inside its chunk events, so the same id appears verbatim:

// responses/<id>.jsonl
{ "requestLogId": "2026-09-30T09-22-35-288Z-60c8e8cf", "type": "chunk",
  "text": "data: {\"id\":\"chatcmpl-b5e868d7b747dd8be5a2fbc0b1313507\", …}\n\n" }

The id prefix depends on which upstream provider answered — chatcmpl-… for Venice, gen-… for OpenRouter — but it is always present and always identical to responseId. Match on chunk.text, never on the raw file: the SSE text is JSON-escaped in the log, so grep '"id":"' misses it.

2. request ⇄ response (exact, by id)

requests/<id>.json and responses/<id>.jsonl share the filename stem (id == requestLogId). A response hit therefore yields its request for free.

3. trigger timestamp ≈ request timestamp (secondary)

The request log's timestamp is when the daemon received the call, which is the instant pi fired it — within ~10–30 ms of the triggering session message (the user turn or tool result), and within a few ms of the assistant record's epoch-ms message.timestamp. Useful as a sanity check or fallback, but not needed once key 1 works.

Running the mapper

The mapper is scripts/link-session-logs.ts (bun, no extra deps):

bun run link-logs -- <session.jsonl> [logsDir] [--all] [--json]

• logsDir defaults to $ROUTSTRD_LOG_DIR, else ~/.routstrd/request-response-logs.
• --all scans every response log instead of only the session's UTC day (use when a session spans midnight or logs were rotated).
• --json prints a machine-readable table with both the requests/ and responses/ paths per turn.

On redtop the script must exist in the redtop checkout. Copy it over now (or git pull once the change is pushed):

scp scripts/link-session-logs.ts \
    redtop:~/projects/routstr_main/routstrd/scripts/

bun is on PATH in an interactive fish session on redtop (~/.bun/bin); from a non-interactive shell, add it explicitly:

ssh redtop 'export PATH=$HOME/.bun/bin:$PATH; \
  bun ~/projects/routstr_main/routstrd/scripts/link-session-logs.ts \
      ~/.pi/agent/sessions/--home-user-projects-routstr_main-routstr-core--/<session>.jsonl \
      ~/.routstrd/reqRes'

To work from the Mac against redtop's logs, either run it there over SSH (as above) or pull the logs first:

rsync -a redtop:/home/user/.routstrd/reqRes/ ~/.routstrd/reqRes-redtop/
bun scripts/link-session-logs.ts <session.jsonl> ~/.routstrd/reqRes-redtop

Finding the session for a given directory

pi slugs the message's cwd into the session directory name by dropping the leading / and replacing the rest with -, wrapped in --:

/home/user/projects/routstr_main/routstr-core
  -> ~/.pi/agent/sessions/--home-user-projects-routstr_main-routstr-core--/

Pick the file you want inside that directory (or ls -t … | head -1 for the latest).

Verified

Against this repo's own history the link is 1:1 and exact:

• 2026-09-30T09-21-41-962Z_01a0f19e…jsonl → 12/12 assistant turns matched (provider: Venice, chatcmpl-…).
• 2026-09-15T19-49-27-933Z_01a0a69e…jsonl → 8/8 (OpenRouter, gen-…).
• 2026-08-16T10-51-25-080Z_01a00a32…jsonl → 51 of 55 turns carry a responseId; all 51 matched (compressed .br logs).
• On redtop, 2026-10-01T07-55-38-825Z_01a0f676…jsonl → 2/2.

Caveats

• Retries and aborted calls leave orphan logs. A single assistant turn can produce more than one request/response pair (transient scp-style failures, provider retries), and some logs belong to sessions outside the one you are looking at. Drive the join from the session, never assume the counts match.
• No responseId = no exact link. The key exists in modern pi session files (see the 2026-06-* / 2026-08-* sessions above). If a session omits it, fall back to the trigger-timestamp window.
• Don't match on the system/developer prompt. The session stores it as sections (preamble + project context); the request log stores it as one flattened developer message. They are not byte-identical.
• Content can quote an id. Tool output that happens to contain an id string can make a stem look like it matches an unrelated turn. The mapper only reads chunk.text, which is model output, keeping this rare.
• Logging may start mid-history. If requestResponseLogging was enabled after a session began, earlier turns simply have no logs.