docs(quic+nestsClient): post-implementation status + audio-rooms completion plan

Two new module-local plan docs (per CLAUDE.md's "plans live in the owning
module" rule) and a sweep of stale inline phase references.

quic/plans/2026-04-26-quic-stack-status.md:
  Post-mortem of the original docs/plans/2026-04-22 plan. Documents
  what shipped vs what was estimated, the actual package layout (~8.5k
  LoC, 39 test files, 5 audit rounds), the crypto delegation surface
  (Quartz only — no BouncyCastle, no JNI), interop verification status
  (aioquic + picoquic; nests not yet), and known deferred items
  (STREAM retransmit, Initial-key discard, etc.).

nestsClient/plans/2026-04-26-audio-rooms-completion.md:
  Punch list to ship audio rooms end-to-end:
    M1 Listener wire-up in Amethyst UI
    M2 Multi-speaker audience UX
    M3 Foreground service for backgrounded playback
    M4 Manual interop pass against nostrnests.com
    M5 MoQ publisher path (ANNOUNCE / TrackPublisher)
    M6 Capture → encode → publish pipeline
    M7 NestsSpeaker API
    M8 App polish (reconnect, leave cleanup)
    M9 Foreground service for speakers
  ~6 weeks for full audio rooms; ~2 weeks for listener-only MVP.

Inline doc cleanup:
  * Removed "Phase 3a/3c-1/3c-2/3c-3" / "Phase B/C/D-K/L" references
    from active code; replaced with "today" or pointers to the
    completion plan
  * Removed "Kwik-based stub" references; QuicWebTransportFactory and
    surrounding docs now describe :quic as the production path
  * TlsClient header reflects non-null certificateValidator + the
    JdkCertificateValidator / PermissiveCertificateValidator split
  * SendBuffer header documents the best-effort no-retransmit mode
    explicitly (was hidden behind a "Phase L will fix this" note)
  * MoqMessage / MoqObject / MoqSession reflect listener-side as
    shipped + publisher-side as Phase M5

CLAUDE.md:
  * Module list now includes :quic and :nestsClient (was 5 modules,
    now 7)
  * Architecture diagram + sharing philosophy explain what each new
    module owns

No production behaviour changes; doc + comment-only edits. Tests green.

https://claude.ai/code/session_01EC1tfXfap8k8GyKvrxkxZx
This commit is contained in:
Claude
2026-04-26 01:38:48 +00:00
parent 7f05fd6e2a
commit 4338e5e6c4
14 changed files with 529 additions and 45 deletions
+24 -7
View File
@@ -3,13 +3,17 @@
## Project Overview
Amethyst is a Nostr Client for Android that was made for Android-only and has been slowly switching
over to a Kotlin Multiplatform project. This project has 5 main modules: `quartz`, `commons`,
`amethyst`, `desktopApp`, and `cli`. Quartz should contain implementations of Nostr specifications
and utilities to help implement them. Commons stores shared code between Amethyst Android
(`amethyst`) and Amethyst Desktop (`desktopApp`). The Desktop App is designed to be mouse first and
so uses a completely different screen and navigation architecture while sharing the back end
components with the android counterpart. `cli` ships `amy`, a non-interactive JVM command-line
client that drives the same `quartz` + `commons` code — used by humans, agents, and interop tests.
over to a Kotlin Multiplatform project. The main modules are: `quartz`, `commons`, `amethyst`,
`desktopApp`, `cli`, plus the audio-rooms transport stack `quic` + `nestsClient`. Quartz should
contain implementations of Nostr specifications and utilities to help implement them. Commons stores
shared code between Amethyst Android (`amethyst`) and Amethyst Desktop (`desktopApp`). The Desktop
App is designed to be mouse first and so uses a completely different screen and navigation
architecture while sharing the back end components with the android counterpart. `cli` ships `amy`,
a non-interactive JVM command-line client that drives the same `quartz` + `commons` code — used by
humans, agents, and interop tests. `quic` is a from-scratch pure-Kotlin QUIC v1 + HTTP/3 +
WebTransport client (no JNI, no BouncyCastle), built because no Android-compatible Java QUIC library
exists. `nestsClient` runs the MoQ-transport audio-room protocol on top of `:quic` for the NIP-53
audio-rooms feature.
## Architecture
@@ -26,6 +30,15 @@ amethyst/
│ ├── commonMain/ # Shared composables, icons, state
│ ├── androidMain/ # Android-specific UI utilities
│ └── jvmMain/ # Desktop-specific UI utilities
├── quic/ # Pure-Kotlin QUIC v1 + HTTP/3 + WebTransport (audio-rooms transport)
│ └── src/
│ ├── commonMain/ # Protocol, frame/packet codecs, TLS state machine
│ ├── jvmAndroid/ # JCA-backed AEAD + UDP socket actuals
│ └── commonTest/ # RFC vector + adversarial tests
├── nestsClient/ # MoQ-transport audio-room client on top of :quic
│ └── src/
│ ├── commonMain/ # MoQ session, NestsListener, audio glue
│ └── jvmAndroid/ # Opus encode/decode, AudioRecord/AudioTrack
├── desktopApp/ # Desktop JVM application (layouts, navigation)
├── amethyst/ # Android app (layouts, navigation)
├── cli/ # Amy — non-interactive CLI (JVM only, no Compose)
@@ -35,6 +48,10 @@ amethyst/
**Sharing Philosophy:**
- `quartz/` = Nostr business logic, protocol, data (no UI)
- `commons/` = Shared UI components, icons, composables, flows and ViewModels
- `quic/` = Transport library (QUIC + HTTP/3 + WebTransport); reusable for any
KMP project that needs MoQ. Has no Android-framework dependencies.
- `nestsClient/` = MoQ + audio-rooms client; takes `:quic` as transport,
Quartz for crypto, `MediaCodec` / `AudioRecord` / `AudioTrack` for audio.
- `amethyst/` & `desktopApp/` = Platform-native layouts and navigation
- `cli/` = Thin assembly layer over `quartz/` + `commons/` (no new logic allowed)
@@ -0,0 +1,246 @@
# Audio rooms — completion plan (2026-04-26)
What's left between today's code and shippable audio rooms in Amethyst.
## Where we are
The transport stack is **done** and audited
([quic/plans/2026-04-26-quic-stack-status.md](../../quic/plans/2026-04-26-quic-stack-status.md)).
On top of it, `:nestsClient` already has:
- HTTP control plane (`NestsClient.resolveRoom` — NIP-98 auth → room info)
- WebTransport adapter (`QuicWebTransportFactory` wires `:quic` into the
`WebTransportSession` interface)
- MoQ session — listener side: `MoqSession.client(...)` + `setup()` +
`subscribe(namespace, trackName, filter)` + control + datagram pumps
- Opus decode + audio playback chain: `MediaCodecOpusDecoder`,
`AudioTrackPlayer`, `AudioRoomPlayer`
- `NestsListener` API + `connectNestsListener` orchestration
- Audio capture primitives (`AudioRecordCapture`, `MediaCodecOpusEncoder`)
exist but are not wired into a publisher path
Amethyst's `audiorooms/` UI parses NIP-53 events and renders rooms +
participant chips. It does NOT call `NestsListener` — there's no Connect
button, no audio output, no mute control wired.
So the punch list is: app-side wiring → manual interop validation → speaker
path → backgrounding & polish.
## Phase M1 — Listener-only MVP (1 week)
**Goal:** open a real audio room from the Amethyst UI, hear one speaker.
- Wire `connectNestsListener` into a `RememberRoomConnection` composable in
`amethyst/.../audiorooms/room/`. Lifecycle tied to `DisposableEffect`;
cancels on screen exit.
- Surface `NestsListenerState` in the UI:
- `Idle` / `Connecting` → show a spinner or chip "Connecting…"
- `Connected` → show "Audio connected" chip + auto-subscribe to the host's
speaker track (NIP-53 room's `p` tag with role `host`)
- `Failed(reason, cause)` → toast / inline message
- `AudioRoomPlayer` per subscription. Per the audio-rooms NIP draft
(`docs/plans/2026-04-22-nip-audio-rooms-draft.md`) one speaker = one track
name = `<speaker-pubkey-hex>`; one `AudioRoomPlayer` per speaker.
- Mute toggle drives `AudioPlayer.setVolume(0f / 1f)` on the active player.
Mute at the player keeps the network running so unmute is instant.
- Backed by an `AudioRoomViewModel` in `commons/.../viewmodels/` so desktop
can reuse the orchestration once it gets WT.
Tests:
- Manual: connect to `nostrnests.com`, open a known room, hear audio.
- Unit: `AudioRoomViewModel` state-flow transitions on
`NestsListenerState` updates.
## Phase M2 — Multi-speaker + audience UX (3 days)
- Subscribe to every `host` + `speaker` `p` tag, not just the first one.
Mix at the audio side (Android `AudioTrack` accepts multiple writers if
we use one shared track + downmix; cleaner: one `AudioTrack` per
subscription and let the OS mix).
- Show per-speaker level meters (if the encoder exposes RMS) or just a
speaking indicator driven by "objects received in last 200 ms".
- React to NIP-53 room event updates: a new speaker added to `p`
open a subscription; a speaker removed → close one.
## Phase M3 — Foreground service (2 days)
- Android `MediaSessionService` with a media-style notification so
playback continues when the app backgrounds.
- Stop the service on:
- screen exit AND no other audio-room-screen is alive
- user dismisses the notification
- underlying `NestsListener` enters `Failed` or `Closed`
- Permission shim: `RECORD_AUDIO` is NOT needed for listener-only.
## Phase M4 — Manual interop pass against `nostrnests.com` (3 days)
This is the proof-of-life step before any speaker work.
- Build a debug build with the listener flow above.
- Open one of the long-running test rooms hosted by nests.
- Confirm: connect succeeds; SUBSCRIBE_OK arrives; OBJECT_DATAGRAMs
decode through MediaCodec into audible audio.
- Anything that surfaces here goes into a follow-up audit / fix pass on
`:quic` or `:nestsClient`. We expect one or two issues — protocol drafts
drift, and we've only verified against aioquic, not a real MoQ relay.
- Capture a packet trace if anything fails so we can compare on-the-wire
bytes against a known-working JS client.
## Phase M5 — Speaker path: MoQ publisher (1 week)
The big one. `MoqSession` only does subscribe today; it needs ANNOUNCE +
OBJECT emission.
Required MoQ messages to encode + decode:
| Message | Direction | Status |
|---|---|---|
| ANNOUNCE | client → server | not implemented |
| ANNOUNCE_OK / ANNOUNCE_ERROR | server → client | decode + match-by-namespace |
| ANNOUNCE_CANCEL | server → client | decode + signal publisher to stop |
| UNANNOUNCE | client → server | encode |
| SUBSCRIBE | server → client (we're publisher) | accept + map to our track sink |
| SUBSCRIBE_OK / SUBSCRIBE_ERROR | client → server | encode |
| SUBSCRIBE_DONE | client → server | encode on track end |
| OBJECT_DATAGRAM (publish-side) | client → server | encode + emit |
API we need on `MoqSession`:
```kotlin
suspend fun announce(
namespace: TrackNamespace,
parameters: List<TrackParameter> = emptyList(),
): AnnounceHandle
interface AnnounceHandle {
/** New publisher per track name we serve under this namespace. */
suspend fun openTrack(name: ByteArray): TrackPublisher
/** Stop announcing; sends UNANNOUNCE + closes any open track publishers. */
suspend fun unannounce()
}
interface TrackPublisher {
/** Push one OBJECT_DATAGRAM. group/objectId are managed internally
* per the audio-rooms NIP. */
suspend fun send(payload: ByteArray)
suspend fun close()
}
```
Internal additions:
- `pendingAnnounces` keyed by namespace, like the existing
`pendingSubscribes`
- inbound-SUBSCRIBE routing: when the server SUBSCRIBEs, we look up the
publisher by namespace+name and start delivering its objects with the
server-assigned subscribeId/trackAlias
- group/object id management: monotonic group per
`TrackPublisher`, object id zero-reset per group; reflect this in the
emitted `OBJECT_DATAGRAM` header
Tests:
- `MoqSession` unit tests for ANNOUNCE round-trip via `FakeWebTransport`
- Integration: a publisher sends 100 Opus-shaped payloads through to a
matching subscriber, all received with intact group/object ids
## Phase M6 — Capture → encode → publish (3 days)
The inverse of `AudioRoomPlayer`:
- `AudioCaptureSource` (commonMain interface) with platform actuals on
`AudioRecordCapture` (Android) and a desktop one later
- `AudioRoomBroadcaster` orchestrates: pull PCM frames from the capture →
feed `MediaCodecOpusEncoder` → push the resulting Opus packet into
`TrackPublisher.send`
- `RECORD_AUDIO` permission gate — surface on first-tap of the talk button
- Push-to-talk vs always-on toggle: at the API level, just `start()` /
`stop()` on the broadcaster; the UI decides
## Phase M7 — `NestsSpeaker` API (2 days)
Mirror of `NestsListener` for hosts/speakers:
```kotlin
interface NestsSpeaker {
val state: StateFlow<NestsSpeakerState>
suspend fun startBroadcasting(): BroadcastHandle
suspend fun close()
}
interface BroadcastHandle {
suspend fun setMuted(muted: Boolean)
suspend fun close()
}
```
Same `connectNestsSpeaker` orchestration as `connectNestsListener` but the
post-`setup` step is `announce(...)` instead of `subscribe(...)`.
UI:
- Talk button only enabled when our pubkey is in the room's `p` tags with
role `host` or `speaker`
- "Live" indicator while broadcasting, level meter from the encoder
- Mute / unmute drives `BroadcastHandle.setMuted`
## Phase M8 — App polish (3-5 days)
- Connection-recovery: `NestsListener` exposes `reconnect()`; the screen
retries on `Failed` after a short backoff
- Room-leave cleanup: on screen exit, send UNSUBSCRIBE + UNANNOUNCE before
closing the WT session (audit-4 / 5 already wired the
`WtCloseSession` capsule emit on `close()`)
- Surface server `peerGoawayProtocolError` and the various
`NestsListenerState.Failed` reasons as user-readable messages
- iOS: stub everything in `iosMain` with `expect`s that error cleanly until
iOS audio capture/playback land
## Phase M9 — Backgrounding for speakers (2 days)
Different from M3 because capture has stricter Android rules:
- Foreground service type `microphone` (Android 14+ requires this)
- Notification with prominent "Speaking" indicator + mute action
## Out of scope for this plan
- **Recording / saving** room audio.
- **Server-mixed audio.** Each speaker is a separate track per the NIP
draft; mixing is client-side.
- **Video.** We support audio only.
- **Accessibility transcription.**
- **Desktop audio capture** (until Compose Desktop has a stable
`AudioInput` API; today's options are JNA-heavy).
## Timeline
| Phase | Days | Cumulative |
|---|---|---|
| M1 Listener wire-up | 5 | 5 |
| M2 Multi-speaker | 3 | 8 |
| M3 Foreground listener | 2 | 10 |
| M4 Real-server interop | 3 | 13 |
| M5 MoQ publisher | 5 | 18 |
| M6 Capture + encode | 3 | 21 |
| M7 NestsSpeaker | 2 | 23 |
| M8 Polish | 4 | 27 |
| M9 Foreground speaker | 2 | 29 |
**6 weeks** to ship full audio rooms (listener + speaker + Android polish).
**2 weeks** to ship listener-only (M1+M3+M4) which is the 95% case for
audience members.
## Stop conditions
- **M4 reveals the QUIC stack can't reach `nostrnests.com`** — drop into a
protocol-comparison pass (likely a draft-version mismatch or a small
framing bug). Up to 1 wk of `:quic` adjustment, otherwise we ship behind
a feature flag and chase interop async.
- **MediaCodec Opus is missing on a target device.** Android 10+ ships the
decoder; for older devices we'd need a software Opus, which is out of
scope.
## Pointers
- QUIC stack status: `quic/plans/2026-04-26-quic-stack-status.md`
- Audio-rooms NIP draft: `docs/plans/2026-04-22-nip-audio-rooms-draft.md`
- Original (frozen) QUIC plan: `docs/plans/2026-04-22-pure-kotlin-quic-webtransport-plan.md`
- Existing listener entry point: `nestsClient/src/commonMain/kotlin/com/vitorpamplona/nestsclient/NestsListener.kt`
- App-side audio-room screen: `amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/audiorooms/`
@@ -23,12 +23,15 @@ package com.vitorpamplona.nestsclient
import com.vitorpamplona.quartz.nip01Core.signers.NostrSigner
/**
* High-level entry point for talking to a nests-compatible audio-room backend.
* HTTP control-plane entry point for talking to a nests-compatible
* audio-room backend. Resolves a room's MoQ endpoint + bearer token via
* NIP-98 auth — that's the only HTTP step before the WebTransport / MoQ
* session takes over.
*
* Phase 3a only exposes the HTTP control plane — resolving a room's MoQ
* endpoint + token via NIP-98 auth. Phase 3b will add the WebTransport/MoQ
* transport on top, keeping this interface stable so audio-room callers only
* depend on [resolveRoom] for the control-plane step.
* The full connect orchestration (HTTP → WebTransport → MoQ → audio) lives
* in [NestsListener] / `connectNestsListener`; this interface stays
* narrowly focused on the control plane so testing the audio path doesn't
* require an HTTP fake.
*/
interface NestsClient {
/**
@@ -74,7 +74,7 @@ sealed class NestsListenerState {
/** Calling `<service>/<roomId>` to obtain the MoQ endpoint + token. */
ResolvingRoom,
/** Opening the WebTransport (Kwik QUIC + Extended CONNECT). */
/** Opening the WebTransport ([:quic] + Extended CONNECT). */
OpeningTransport,
/** Running the MoQ CLIENT_SETUP / SERVER_SETUP exchange. */
@@ -27,8 +27,11 @@ package com.vitorpamplona.nestsclient.moq
*
* message_type (varint) | message_length (varint) | payload...
*
* This phase (3c-1) covers only the setup handshake. SUBSCRIBE / ANNOUNCE /
* OBJECT messages arrive in Phase 3c-2.
* Listener-side messages (CLIENT/SERVER_SETUP, SUBSCRIBE, SUBSCRIBE_OK /
* SUBSCRIBE_ERROR, UNSUBSCRIBE) are implemented. Publisher-side messages
* (ANNOUNCE / ANNOUNCE_OK / SUBSCRIBE-receiving / SUBSCRIBE_DONE) are not
* yet implemented; see `nestsClient/plans/2026-04-26-audio-rooms-completion.md`
* (Phase M5).
*/
sealed class MoqMessage {
abstract val type: MoqMessageType
@@ -164,7 +167,7 @@ enum class SubscribeFilter(
/**
* SUBSCRIBE (0x03): client asks a publisher to forward objects belonging to a
* (namespace, track) pair. Phase 3c-2 supports only the LatestGroup /
* (namespace, track) pair. Today the codec supports only the LatestGroup /
* LatestObject filters — absolute-range variants add extra wire fields the
* codec will grow in a follow-up if nests ever needs them.
*/
@@ -182,7 +185,7 @@ data class Subscribe(
init {
require(filter == SubscribeFilter.LatestGroup || filter == SubscribeFilter.LatestObject) {
"Phase 3c-2 only supports LatestGroup / LatestObject filters, got $filter"
"only LatestGroup / LatestObject filters supported, got $filter"
}
require(subscriberPriority in 0..255) { "subscriber_priority must fit in a byte" }
require(groupOrder in 0..255) { "group_order must fit in a byte" }
@@ -32,7 +32,9 @@ package com.vitorpamplona.nestsclient.moq
* 2. STREAM_HEADER_SUBGROUP — multiple objects per uni stream, reliable.
* 3. FETCH_HEADER — historical objects over a bidi stream.
*
* Phase 3c-2 covers only (1). Stream-delivered objects arrive in Phase 3c-3.
* Today the listener path implements only (1) — OBJECT_DATAGRAM — which is
* what nests uses for live audio. (2) and (3) are reserved for future
* stream-delivered media; see the audio-rooms completion plan.
*/
data class MoqObject(
val trackAlias: Long,
@@ -345,9 +345,11 @@ class MoqSession private constructor(
}
else -> {
// Other control messages (SETUP echoes, future ANNOUNCE/etc.)
// are silently dropped at this layer; Phase 3c-3 only needs the
// subscribe lifecycle.
// Other control messages (echoed SETUP, future ANNOUNCE +
// SUBSCRIBE-receiving for the publisher path, etc.) are
// silently dropped — the listener path only needs the
// subscribe lifecycle. Publisher-side routing is Phase M5
// in nestsClient/plans/2026-04-26-audio-rooms-completion.md.
}
}
}
@@ -32,8 +32,8 @@ import kotlinx.coroutines.sync.withLock
*
* A pair of fakes is connected via [pair] — anything written on one side is
* delivered on the other. This deliberately simulates *success* semantics
* only (no packet loss, no congestion); the real Kwik-backed transport will
* exercise those codepaths separately.
* only (no packet loss, no congestion); the real `:quic`-backed transport
* exercises those codepaths via its own pipe + interop tests.
*
* [incomingDatagrams] and [FakeBidiStream.incoming] use [receiveAsFlow]
* semantics: a `take(1)` / `first()` followed by a long-running `collect`
@@ -26,13 +26,14 @@ import kotlinx.coroutines.flow.Flow
* Platform-agnostic WebTransport session, as produced by a successful Extended
* CONNECT (RFC 9220) handshake.
*
* The MoQ layer (Phase 3c) talks to this interface; the real Kwik-based
* implementation sits behind [WebTransportFactory] in jvmAndroid. Keeping this
* abstract lets us:
* - unit-test the MoQ framing layer with an in-memory fake,
* - swap transport implementations (Cronet on Android, a browser-backed
* WebView bridge as a contingency, Kwik on JVM desktop) without touching
* audio/UI code.
* The MoQ layer talks to this interface; the production implementation is
* [com.vitorpamplona.nestsclient.transport.QuicWebTransportFactory] which
* sits on top of the pure-Kotlin `:quic` stack. Keeping this abstract lets
* us:
* - unit-test the MoQ framing layer with [FakeWebTransport],
* - swap transport implementations (a different QUIC backend, or a
* browser-backed bridge as a contingency) without touching audio/UI
* code.
*
* Lifecycle: the session is opened via [WebTransportFactory.connect] and must
* be closed with [close] to release the underlying QUIC connection.
@@ -91,7 +92,7 @@ interface WebTransportWriteStream {
*
* [authority] is the `host:port` of the WT server, [path] is the URL path
* (nests defaults to `/moq`), and [bearerToken] is typically the token
* returned by the `/api/v1/nests/<roomId>` HTTP call in Phase 3a.
* returned by the `/api/v1/nests/<roomId>` HTTP call (see [NestsClient]).
*/
interface WebTransportFactory {
suspend fun connect(
@@ -42,8 +42,10 @@ import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.flow
/**
* Pure-Kotlin WebTransport over QUIC v1, drop-in replacement for the
* Kwik-based stub. This is the realisation of every layer in :quic.
* Pure-Kotlin WebTransport over QUIC v1, sitting on top of every layer in
* `:quic`. The historical alternative was a Kwik-based JNI binding; that
* was rejected because no Android-compatible native classifier ships, so
* we wrote `:quic` from scratch.
*
* Lifecycle on [connect]:
* 1. Open a UDP socket connected to (authority host, authority port).
+201
View File
@@ -0,0 +1,201 @@
# QUIC + WebTransport stack — current state (2026-04-26)
This document is the post-implementation snapshot of `:quic`. It supersedes
the original [pure-kotlin QUIC + WebTransport plan](../../docs/plans/2026-04-22-pure-kotlin-quic-webtransport-plan.md)
which was written before any code shipped and is now historical.
## TL;DR
`:quic` is a self-contained Kotlin-Multiplatform module that speaks QUIC v1
+ HTTP/3 + WebTransport against real-world servers (aioquic + picoquic
verified; nestsClient/MoQ on top). It uses Quartz crypto primitives only —
no BouncyCastle, no JNI. ~8.5k lines of production code, ~5k lines of
tests, 39 test files, five rounds of parallel audit + fix passes.
## What shipped vs the original plan
| Phase | Original estimate | Actual | Notes |
|---|---|---|---|
| A. Foundations | 1 wk | done | KMP module, UdpSocket on `jvmAndroid`, Varint migrated from nestsClient |
| B. TLS 1.3 | 3 wk | done | RFC 8446 client state machine, X25519 ECDHE, RFC 8448 §3 vectors pass bit-for-bit |
| C. Initial + Handshake packets | 2 wk | done | RFC 9001 §A.2/§A.3 vectors pass; ChaCha20 per §A.5 |
| D. 1-RTT + STREAM | 1 wk | done | Stream offset reassembly, FIN, fuzzed |
| E. ACK + flow control | 1 wk | done | MAX_DATA / MAX_STREAM_DATA / MAX_STREAMS routing + writer enforcement |
| F. Loss recovery + congestion control | 1 wk | partial | PTO timer for handshake retries; **no retransmit-on-loss in steady state** (out of scope — see "deferred" below) |
| G. Datagram extension | ½ wk | done | RFC 9221 frames + bounded incoming queue |
| H. Connection lifecycle | 1 wk | done | CONNECTION_CLOSE, idle timeout, draining/closing, idempotent driver close |
| I. HTTP/3 | 2 wk | done | Control stream + SETTINGS + GOAWAY (with id-regression check) + duplicate-id rejection |
| J. QPACK | 2 wk | done | Static-table + Huffman + integer codec (RFC 7541 §B + RFC 9204 §B.1 vectors) |
| K. Extended CONNECT + WT | 1 wk | done | Stream-type prefixes, HTTP Datagram + WT_CLOSE_SESSION capsule |
| L. Interop + hardening | 2 wk | done + much more | Live interop against aioquic Docker; **5 rounds of audit + fix** beyond the plan |
The original plan estimated 1719 weeks. We ran the full sequence plus five
unscheduled audit rounds. Every audit found real bugs; the suite is what
caught them on regression.
## What's actually in the module
```
quic/
├── plans/ ← module-local design docs
└── src/
├── commonMain/kotlin/com/vitorpamplona/quic/
│ ├── Buffer.kt ← QuicReader / QuicWriter
│ ├── Varint.kt ← RFC 9000 §16
│ ├── connection/ ← QuicConnection orchestrator + Driver
│ │ ├── QuicConnection.kt (≈ 600 lines, the hub)
│ │ ├── QuicConnectionDriver.kt (read/send loops + close)
│ │ ├── QuicConnectionParser.kt (feedDatagram + dispatchFrames)
│ │ ├── QuicConnectionWriter.kt (drainOutbound + flow-control updates)
│ │ ├── PacketProtection.kt + builder
│ │ ├── PacketNumberSpace.kt
│ │ ├── ConnectionId.kt + TransportParameters.kt
│ │ └── EncryptionLevel.kt + LevelState.kt
│ ├── crypto/ ← Aead, header protection, HKDF helpers, AesEcbHeaderProtection,
│ │ ChaCha20HeaderProtection, ChaCha20Poly1305Aead, InitialSecrets,
│ │ PlatformAesOneBlock, PlatformChaCha20Block (expect),
│ │ bestAes128GcmAead (expect)
│ ├── frame/ ← Frame.kt sealed hierarchy + FrameFuzzerTest target
│ │ includes RESET_STREAM / STOP_SENDING / NEW_TOKEN
│ ├── http3/ ← Http3FrameReader + Http3Settings + frame types
│ ├── packet/ ← LongHeaderPacket, ShortHeaderPacket, RetryPacket, peekHeader
│ ├── qpack/ ← QpackDecoder, QpackEncoder, QpackHuffman, QpackInteger,
│ │ QpackStaticTable
│ ├── recovery/ ← AckTracker (with ack-eliciting gating)
│ ├── stream/ ← QuicStream, ReceiveBuffer (with FIN-fully-read), SendBuffer, StreamId
│ ├── tls/ ← TlsClient state machine + ClientHello/ServerHello/EE/Cert/CV/Finished
│ │ codecs, TlsKeySchedule, TlsTranscriptHash (incremental), TlsConstants,
│ │ PermissiveCertificateValidator, TlsRunningSha256 (expect)
│ ├── transport/ ← UdpSocket (expect)
│ └── webtransport/ ← QuicWebTransportFactory + QuicWebTransportSessionState +
│ WtPeerStreamDemux + WtCapsule + WtDatagram + ExtendedConnect
└── jvmAndroid/kotlin/com/vitorpamplona/quic/
├── crypto/JcaAesGcmAead.kt ← cached JCA Cipher per direction with IV-reuse fallback
├── crypto/PlatformCrypto.kt ← actuals
├── tls/JdkCertificateValidator.kt ← system-trust-store chain validation + RSA-PSS / ECDSA / Ed25519
├── tls/TlsRunningSha256.kt ← MessageDigest.clone()-based incremental hash
└── transport/UdpSocket.kt ← DatagramChannel + suspend wrapper
```
## Crypto surface
Quartz primitives only:
| Primitive | Source |
|---|---|
| AES-128-GCM | `JcaAesGcmAead` (jvmAndroid) — cached `Cipher` per direction; `Aes128Gcm` singleton (commonMain) for non-hot paths |
| ChaCha20-Poly1305 | Quartz `ChaCha20Poly1305` (commonMain pure-Kotlin) wrapped in `ChaCha20Poly1305Aead` |
| HKDF-Extract / Expand-Label | Quartz `Hkdf` + `MacInstance`; thin RFC 8446 §7.1 helper in `crypto/HkdfHelpers.kt` |
| SHA-256 (one-shot) | Quartz `sha256(...)` |
| SHA-256 (incremental, for transcript) | `TlsRunningSha256` (expect/actual; jvmAndroid wraps `MessageDigest.clone()`) |
| X25519 ECDHE | Quartz `X25519` |
| Ed25519 | Quartz `Ed25519` (only inside the JVM cert validator path) |
| AES-ECB (one block, for header protection) | `Cipher.getInstance("AES/ECB/NoPadding")` (jvmAndroid only) |
| ChaCha20 keystream (header protection) | Quartz `ChaCha20Core.chaCha20Xor` |
| SecureRandom | Quartz `RandomInstance` |
X.509 chain validation, hostname verification, signature verification all
delegate to JDK `TrustManagerFactory` / `Signature.getInstance(...)`.
`CertificateValidator` is a non-null typed parameter — tests pass an
explicit `PermissiveCertificateValidator`; production passes
`JdkCertificateValidator`.
## What we deliberately don't do
- **QUIC server role.** Client-only.
- **0-RTT / session resumption.** No PSK extension offered; an arriving
ServerFinished without prior Certificate is hard-failed.
- **Connection migration / preferred address / multiple paths.** `NEW_CONNECTION_ID`
and `PATH_*` frames decode (so peers don't break us) but aren't acted on.
- **Path MTU discovery.** Fixed 1200-byte ceiling per RFC 9000 §14.
- **HTTP/3 server push.**
- **QPACK dynamic-table inserts on the encoder.** We send literal-only;
decoder accepts dynamic-table indexed lines.
- **ECN / anti-amplification limits.** We're a client.
- **Retransmit-on-loss in steady state.** `SendBuffer.takeChunk` releases
bytes to the wire and doesn't retain them. The handshake survives via the
`Driver.sendLoop` PTO path which re-pulls from CRYPTO send buffers; for
STREAM data, a real loss event truncates the stream silently. This is
acceptable for MoQ (DATAGRAM-mode audio, plus stream usage is
control-plane only) but would be the first item to add for general use.
- **TLS Key-Update / NewSessionTicket.** Detected and refused (KeyUpdate
fails the connection rather than silently desynchronising).
## Verified interop
- **aioquic** (Python): `quic-interop-runner`-style Docker setup; full
handshake + Extended CONNECT + h3 datagram round-trip.
- **picoquic** (C): Docker image, lightweight HTTP/3 GET.
- **In-memory pipe** (`InMemoryQuicPipe`, modeled on Cloudflare quiche's
`Pipe`) drives both sides of the handshake in one JVM for fast tests
without sockets.
What's NOT verified: a live nests/MoQ audio-room exchange end to end. That
gates on the audio-rooms completion plan
([nestsClient/plans/2026-04-26-audio-rooms-completion.md](../../nestsClient/plans/2026-04-26-audio-rooms-completion.md)).
## Audit summary
| Round | Focus | Findings | Status |
|---|---|---|---|
| 1 | Initial review (pre-interop) | 6 critical correctness/security bugs | all fixed |
| 2 | TLS hardening + lifecycle | hangs + TLS edge cases | all fixed |
| 3 | Performance + concurrency | cipher caching, polling, transcript O(n²) | all fixed |
| 4 | Core + TLS + perf + coverage gaps (4 parallel agents) | ~30 items including 4 CRITICAL interop blockers | all fixed; comprehensive regression tests added |
| 5 | Regression check + concurrency-specific (2 parallel agents) | 1 CRITICAL ackEliciting regression I'd just introduced + WT scope leak + others | all fixed |
Every fix carries an inline `audit-N #M` reference comment so the regression
test → fix → comment chain is auditable. The whole audit corpus is in the
git log (commits whose subject starts with `fix(quic):` or `perf(quic):`).
## Test inventory
Roughly grouped:
- **RFC vectors:** RFC 8448 §3 (TLS handshake), RFC 9001 §A.1A.5 (Initial
encrypt/decrypt, Retry, ChaCha20, server-side HP), RFC 9204 §B.1 (QPACK),
RFC 7541 (Huffman).
- **End-to-end pipe tests:** `InMemoryQuicPipeTest`, `CoalescedPacketSkipTest`,
`ReceiveLimitEnforcementTest`, `PeerStreamLimitTest`, `FrameRoutingTest`,
`AckElicitingFramesTest`.
- **Adversarial:** `FrameFuzzerTest`, `HostilePacketInputTest`,
`TlsSecurityPropertiesTest`, `HelloRetryRequestTest`.
- **Crypto:** `JcaAesGcmAeadTest`, `ChaCha20Poly1305AeadTest`,
`TlsTranscriptHashTest`.
- **WT / HTTP/3:** `CapsuleReaderTest`, `WtPeerStreamDemuxTest`,
`WtFramingTest`.
- **Recovery:** `AckTrackerCoalescedTest`, `AckTrackerGatingTest`.
- **Interop:** `InteropRunner` (jvmTest, drives a real socket against a
Dockerised aioquic; opt-in, not in CI).
## Known limitations / deferred work
These are the items future audit rounds keep flagging that we've
consciously not tackled — all confined to the steady-state path that audio
rooms don't exercise heavily:
1. **No STREAM retransmit on loss** (audit-4 #10). Acceptable for MoQ
datagram audio; would block any heavy stream-based use. ~1 wk to add.
2. **`SendBuffer` doesn't retain bytes until ACK.** Same scope as #1.
3. **No Initial / Handshake key discard.** RFC 9000 §17.2.2 / RFC 9001 §4.9
require dropping these after handshake completes; we hold them
indefinitely. Memory leak per long session.
4. **No path validation for `NEW_CONNECTION_ID`.** We don't migrate.
5. **Stateless reset detection.** Stateless-reset packets look like
corruption to us.
6. **`AckTracker.purgeBelow` threshold semantics.** Pre-existing bug:
purges based on peer's largestAcknowledged of OUR outbound PNs, but
purges OUR inbound PN tracker. Causes range-list bloat, not correctness
failure.
7. **Driver direct unit tests** require turning `UdpSocket` from `expect
class` into an interface so the test side can stub. The driver is
covered indirectly by every pipe-based test plus the live interop
runner.
## Pointers
- Original (frozen) plan: `docs/plans/2026-04-22-pure-kotlin-quic-webtransport-plan.md`
- Audio-rooms NIP draft: `docs/plans/2026-04-22-nip-audio-rooms-draft.md`
- Completion plan: `nestsClient/plans/2026-04-26-audio-rooms-completion.md`
- Live interop runner: `quic/src/jvmTest/.../interop/InteropRunner.kt`
- Audit history: `git log --grep='audit' -- quic/`
@@ -33,8 +33,11 @@ import com.vitorpamplona.quic.tls.deriveQuicKeys
* TLS traffic secret + TLS cipher-suite identifier. The QUIC labels in
* RFC 9001 §5.1 (`quic key`, `quic iv`, `quic hp`) drive the expansion.
*
* For Phase BK we only support TLS_AES_128_GCM_SHA256 (16/12/16) and
* TLS_CHACHA20_POLY1305_SHA256 — the SHA-256 suites; nests speaks both.
* Supports the two SHA-256-keyed cipher suites required for QUIC v1
* interop with nests / aioquic / picoquic: TLS_AES_128_GCM_SHA256
* (16/12/16) and TLS_CHACHA20_POLY1305_SHA256. AES-256-GCM-SHA384 is
* omitted — Quartz's primitives don't ship SHA-384 and no interop target
* we've encountered requires it.
*/
fun packetProtectionFromSecret(
cipherSuite: Int,
@@ -27,14 +27,16 @@ package com.vitorpamplona.quic.stream
* Application code [enqueue]s payload bytes; the connection's send loop
* [takeChunk]s as much as it can fit in the next packet, given the
* remaining packet budget and stream-level / connection-level flow control
* credit. Sent bytes stay in the buffer until [acknowledge] (Phase F adds
* retransmission of lost bytes; for v1 we just trust the receiver and
* release on send).
* credit.
*
* For Phase D-K we run a "best effort" mode: bytes are released from the
* **Best-effort mode (no STREAM retransmit):** bytes are released from the
* buffer the moment they're handed off, on the assumption that the
* underlying network is stable. Phase L adds retransmit-on-loss (using the
* same send buffer that retains until ACK).
* underlying network is stable. A real loss event silently truncates the
* stream. Acceptable for MoQ over QUIC (audio rooms use OBJECT_DATAGRAM,
* which is loss-tolerant; STREAM is control-plane only). See the deferred
* items in `quic/plans/2026-04-26-quic-stack-status.md` — adding
* retain-until-ACK + retransmit is the first thing to add for general
* STREAM-heavy use.
*/
class SendBuffer {
/**
@@ -41,10 +41,12 @@ import com.vitorpamplona.quic.QuicWriter
* 2. After ServerHello arrives → install Handshake keys both directions.
* 3. After server Finished decoded → install 1-RTT (application) keys both directions.
*
* For Phase B we **do not yet validate the certificate chain or the
* CertificateVerify signature**. That's wired in during Phase C/L when we
* have a real server to talk to. We DO compute and verify the server
* Finished MAC.
* Certificate chain validation + CertificateVerify signature verification
* are delegated to [certificateValidator] (`JdkCertificateValidator` in
* production; `PermissiveCertificateValidator` for in-process tests). The
* validator parameter is non-null — passing `null` was a silent-MITM
* hazard and was removed in round-4 of the audit. We also compute and
* verify the server Finished MAC ourselves.
*/
class TlsClient(
val serverName: String,