Carries the four security fixes forward. Three resolutions were not mechanical,
because this line has moved under them:
The handshake reaper is async here too, so it can close the transport
connection it used to forget. That meant porting the change onto the sans-IO
refactor rather than taking maint's text: check_timeouts, cleanup_stale_connection
and drive_handshake_timeouts all become async, and the call sites in the rx loop's
instrumented tick body and in the supervisor's stale sweep gain their awaits.
maint's lifecycle.rs no longer exists on this line; its call site is in
lifecycle/mod.rs.
The keygen guard keeps both changes: this line's note about keys stranded at the
old macOS and FreeBSD default directory, and maint's switch from exists() to
symlink_metadata() so a dangling symlink cannot slip past the overwrite refusal.
The onion transport's first-frame deadline is NOT carried. maint reads its frames
inline, while this line has moved Tor and Nym onto a shared proxied receive loop,
so the fix would have to be implemented against that shared loop and would touch
Nym with it. Doing that inside a merge resolution, on a security deadline, with no
review, is the wrong place for it. The TCP half of the deadline is present and
covers the listener that carries the 256-slot default; the onion listener with its
64-slot cap is left as filed follow-up work on this line and next.
The test frame builder lives in transport::framing here rather than in
transport::tcp::stream.
The dependency refresh landed with its own entry; the gateway DNS validation,
the key-write hardening, the inbound TCP slot deadline and the action pinning
did not. Each entry states what the defect was, what closed it, and where a
behaviour changed: the DNS rcode relay, the fips.pub mode policy, and the fact
that the TCP deadline covers the first frame only and leaves a peer that sends
one frame then goes silent still holding a slot.
Not one action reference in this repository was pinned. Every uses: line named
a mutable tag, and one named a branch. That includes the jobs holding the AUR
deploy key, the jobs with release write scope, and the packaging jobs that run
with a signing key in the environment, so whoever controls one of those action
repositories could repoint a tag into a job holding our credentials.
Sixty-two of the sixty-six references are now full commit SHAs with the
original tag kept as a trailing comment, so a reader can still tell which
release a pin is. Each SHA was resolved from the upstream peeled tag. Four
references are left unpinned and justified in one place rather than silently:
two actions select the tool they install from the ref name itself, so a bare
SHA hands them a hex string where a toolchain name belongs and the step fails.
Pinning those means moving the selection into with:, which changes what
resolves, and that is a separate decision from pinning.
A guard enforces the form on every sweep, wired into the parity job and the
local runner beside the existing checkers. It accepts only owner/repo@40-hex
with a mandatory trailing comment, treats an unreadable tree as exit 2 rather
than as a pass, and its header names what it does not cover: the actions that
pinned actions themselves invoke, the pip and cargo installs that are version
pinned at best, and anything fetched at run time.
The sharper hole was not the tags. The OpenWrt packaging workflow fetched a
helper binary straight from a release URL with no verification, in two jobs
that hold a signing key, which is code execution from a third-party host into a
credentialed job and needs nobody to retag anything. That download now goes
through a shared script with per-architecture pinned SHA-256 constants,
modelled on the zig block already in that workflow. Upstream publishes no
checksum document, so the provenance comment records the asset URL and the date
the hashes were taken by downloading rather than pretending they were verified
against a published sum.
An accepted TCP socket took an inbound slot before a single byte was read. The
cap is tested at accept, the pool insert and the counter bump follow with no
read in between, and the frame reader's two read_exact calls carry no deadline.
So an unauthenticated remote held a slot by connecting and sending nothing, and
since pool keys are ip:port, N sockets from one address are N slots rather than
one. At the 256 default that locks out inbound peering for as long as the
attacker keeps the sockets open.
The first frame on an inbound connection now has a deadline. It is a module
constant rather than a config key: this branch takes no new operator-facing
surface, and a knob is not needed to fix a missing bound. The onion listener
has the identical accept-then-count ordering and gets the same treatment; it
was not in the original report.
The second half is that nothing reclaimed a slot once taken. The node-layer
handshake reaper tore down session state but never closed the transport
connection, so a peer that sent a real msg1 and then stalled was forgotten by
the node while its socket, its pool entry and its slot lived on. The reaper now
closes the transport connection too. Closing twice is safe: every
close_connection implementation guards on removing the entry from its pool, and
the connectionless ones are no-ops, so the handshake paths that already close
and then drop a link are undisturbed.
What this does not close, and it should be said plainly rather than discovered
later: the deadline covers the first frame only. A peer that sends one
well-formed frame and then goes silent still holds its slot, and so does one
that completes msg1 and stalls beyond the reaper's reach. Closing those needs a
rolling idle deadline, which interacts with heartbeats being per peer rather
than per link and is a larger decision than this change.
The gateway's DNS forwarder accepted whatever datagram arrived on its upstream
socket as the answer. It reused the client's own transaction ID in the upstream
query, bound that socket to a wildcard address, received with a call that
discards the sender, never compared the response ID or the question section
against what it asked, and never checked that the returned address was inside
the mesh prefix. The address it extracted is installed as a DNAT rule that
carries no interface constraint, so a forged answer redirected traffic rather
than merely poisoning a lookup.
Four changes close it. The upstream query now carries a freshly drawn random
transaction ID rather than the client's. The upstream socket is connected to
the resolver before use, so the kernel drops datagrams from anyone else. A
parsed response must be a response, carry the same ID, carry exactly one
question matching the qname and qclass that were sent, and be for AAAA;
anything else is discarded and the receive continues against the original
deadline instead of accepting the first datagram to arrive. The extracted
address goes through the validating address parser rather than a comment
asserting the prefix byte, and a non-mesh answer is refused before any pool
allocation, so no mapping event is emitted and no rule is installed.
The validation sits before the rcode check, which changes one behaviour worth
naming: an upstream that answers FORMERR or REFUSED with an empty question
section no longer has that rcode relayed to the client and gets SERVFAIL
instead. Checking after the rcode would let a forged NXDOMAIN through, so the
placement is deliberate.
Connecting the socket also fixes the dead-upstream half of the availability
problem in the same loop, since a connected socket surfaces ECONNREFUSED
immediately instead of stalling to the five second timeout. The serve loop
still handles one query at a time; that half is untouched here.
The tests drive real queries through a fake upstream: a foreign source
injecting a well-formed answer, a wrong transaction ID, a wrong question, a
non-mesh address, and the healthy path as an over-rejection guard. Each was
checked by reverting the corresponding fix and confirming the intended test
reds alone.
Every private key in the tree is written by one function, and it opened the
path with create+truncate and no O_NOFOLLOW, so a symlink pre-planted at the
key path was followed and its target overwritten. The mode was supplied only
through open(2)'s mode argument, which the kernel applies when it creates the
file and ignores otherwise, so a fips.key that already existed at 0644 stayed
0644 after every rewrite. That half needs no attacker: one chmod, or a restore
that did not preserve modes, leaves the key readable forever.
Both writers now go through a shared open helper that carries O_NOFOLLOW and,
for the private key only, applies the mode to the open descriptor before any
secret bytes are written. The public key keeps its create-time mode instead,
because forcing it would reopen an operator-tightened fips.pub on every start.
A refused open is classified by inspecting the path rather than the errno,
since O_NOFOLLOW reports a symlinked final component differently across the
Unix variants this module compiles for.
Six write results that were discarded now report. The sharpest was in
persistent mode: a failed key write fell through to an ephemeral identity with
no message at all, so a node could silently change npub on every start while
its config asked for a stable one. An ephemeral start over an existing key file
now warns before it overwrites, naming the path and the setting that would have
preserved the identity, which is the warning the key generation tool has always
given and the daemon never did. Existence is tested with symlink_metadata
rather than exists, because a dangling symlink reports false from the latter
while still being a file the write acts on.
The persistent read path warns when it finds a key file whose mode is looser
than 0600 or which is a symlink. It does not repair either: the daemon does not
own a file it did not create.
Windows is a named coverage gap in both writers' docs. There is no mode to
enforce and no O_NOFOLLOW; the file inherits the parent directory's ACLs.
The lockfile pinned nostr 0.44.3 and nostr-relay-pool 0.44.1, both yanked
and both carrying advisories that reach this code. The relay-pool ones are
the reason this is worth doing promptly: RUSTSEC-2026-0224 and -0232 are a
verification-cache bypass and the processing of unverified relay events,
and the path they land on is how a node learns peer adverts, which it
consumes without verifying anything itself. RUSTSEC-2026-0231 sits on the
same path, and -0216 and -0227 reach NIP-44 decryption of relay-supplied
content. The other advisories in the set cover APIs this code never calls.
nostr moves to 0.44.8 and nostr-relay-pool to 0.44.3. Both fixed floors are
inside 0.44, so the existing requirements already admitted them and nothing
in the source changed. The refresh is taken over the whole lockfile rather
than the two crates alone, because a targeted bump leaves RUSTSEC-2026-0204
in crossbeam-epoch open and four yanked crates in the tree for no gain.
cargo audit reports no vulnerability now, against twelve before. The four
remaining warnings cannot be closed by a version move: instant and paste are
unmaintained, lru 0.16.4 is unsound, and nostr-relay-pool is itself marked
unmaintained as of RUSTSEC-2026-0243, which the dependency strategy needs to
answer separately.
Everything the maint line contributed is already here by merge, so this
adds what only master carries: the 802.11s mesh backhaul and the open
!FIPS access SSID for OpenWrt, the Android-ready core with its app-owned
TUN seam, the bounded shutdown drain and its new node.drain_timeout_secs,
the node health states with start-time and runtime child-exit detection,
and the connected UDP socket errors that now name the failing syscall.
Zero transports up becoming fatal is the entry an upgrader most needs:
a node that previously came up serving nothing now exits.
The sans-IO migration and the module relocations are deliberately absent.
They are the bulk of the cycle by commit count and none of them changes
behaviour, so the config key renames they produced are the only part a
reader can observe, and those were already recorded.
Bug fixes for defects introduced on master since 0.4.0 are left out as
well, since no released version ever carried them: the permanent Degraded
latch from watching the wrong Nostr handle, and the open-discovery sweep
log lost in the peering refactor.
The Unreleased section carried the msg1 metering, the macOS path fixes,
the traversal clock and the hop-limit change, but not the work that
landed alongside them. Adds the rekey config validation, the three tick
body performance changes, the richer socket adoption log line, the
traversal relay filtering, the log write panic, and a Security entry for
the FSP session address binding, which is the one an operator most needs
to see before deciding when to upgrade.
Left out deliberately: the CI and test harness commits, which are the
bulk of the cycle by count and none of which changes shipped behaviour,
and the zig download integrity fix, which hardens how the OpenWrt
packages are built rather than what they contain. The 0.4.1 section sets
that precedent by carrying no CI entries.
fips-gateway builds its subscriber the same way the daemon did, with the
default internal-error reporting that sends a failed log write to
`eprintln!`. Both shipped units set StandardOutput=journal and
StandardError=journal, so one full disk fails both sinks together, which
is the same precondition as on the daemon side.
What dies here is a spawned task rather than a thread. The DNS resolver,
the control accept loop and the pool tick are spawned and then not looked
at again: their handles are first touched at shutdown, as `let _ =
task.await`, which discards the JoinError. A task lost this way leaves
the process running and reporting healthy with mesh name resolution or
lease expiry and NAT cleanup stopped, and nothing recording that it
stopped.
Whether each task reaches a log site while the disk is full is not
enumerated here; the resolver logs on exactly the error paths a full disk
makes likely.
The subscriber is built with the default internal-error reporting, which
sends a failed log write to `eprintln!`. That macro panics when stderr also
fails, so on a full disk a single WARN can unwind whichever thread emitted
it. The shipped macOS plist points StandardOutPath and StandardErrorPath at
the same unrotated file and systemd routes both to journald, so the two
sinks fail together rather than independently.
A crypto worker is the case that matters. It logs a WARN on send
backpressure, and a worker that dies takes its share of the peer space with
it permanently: dispatch keeps hashing peers onto a channel nobody is
reading. The panic message is discarded along the same broken path, so
nothing records why.
Turning off internal-error reporting closes that path without touching
dispatch or the deliberate blocking backpressure on a full worker channel.
It does not make worker death survivable, which is a separate concern.
The six methods connecting the live peer map to the routing core had no
tests. Replacing the bloom predicate with one returning true, which disables
filtering on the forwarding path entirely, left the suite green at 1718
passed, and the two tests whose names promise to cover it passed as well.
The new tests assert on the real adapter through find_next_hop, which builds
it internally and cannot be handed a mock. Two fixtures make them
discriminate: one places two peers at equal distance and equal cost so the
tie-break is the only thing separating them, and one places three peers at
distinct distances so the winner depends on the coordinate values returned
rather than merely on their presence.
Each of the six methods was broken in turn and the intended test confirmed to
fail. The break that motivated this now fails three tests where it failed
none.
The two misleading tests keep their assertions and gain a note saying what
they do not discriminate and which test does. One of them cannot be made to
discriminate without a production change: under the broken predicate it
returns the same peer by a different branch, and the caller sees no branch.
The OpenWrt cross-compile fetched zig through `curl | sudo tar xJ`, so a
short read reached tar as a truncated archive and failed the build with
"Unexpected EOF in archive". A pipe leaves nowhere to check the bytes, and
curl's own --retry does not cover it: exit 18 is not in its transient set.
Download to a staging directory first, verify a pinned SHA-256, then
extract. Each architecture now sets its hash on the same case branch that
sets its name, so an architecture cannot be added without one, and a guard
fails with the jq recipe for deriving it if the hash is ever empty. Three
attempts with 10s and 20s backoff, matching the retry idiom already in this
workflow, and an early exit when two attempts return identical bytes, since
a stable mismatch is a wrong pin rather than a bad transfer.
The step also gains `set -euo pipefail` and a trap that removes the staging
directory on every exit path. It previously ran under the default shell
without pipefail, so a failure inside the pipe could be masked by tar.
The hashes come from ziglang.org's download index and were checked against
the bytes of both tarballs. That is integrity, not authenticity: index and
archive share an origin, and upstream publishes no detached sums.
Routing currently collects peer addresses and eligible candidates into temporary vectors before selecting the best next hop.
Visit borrowed peers and fuse eligibility checks with the cost, distance, and address comparison so the hot path no longer allocates or clones coordinates.
`io.rs` had grown to 1322 lines in which almost nothing was actually
generic: two mutually exclusive `platform` modules, plus a third module
for the connected-socket fast path, with Linux and macOS diverging
repeatedly inside the Unix one rather than at any module boundary. The
only genuinely shared items were the doc header, four imports and the
re-exports.
Turn it into a directory that says which is which. `io/unix.rs` holds
everything both Unix targets share: socket creation and adoption, buffer
sizing, the synchronous calls, and the `AsyncFd` wrapper. The three
points where behaviour really differs move to `io/linux.rs` and
`io/macos.rs` behind identical signatures, so the shared file selects a
`sys` module once at the top and its bodies carry no `cfg` at all:
- `enable_drop_counting` — `SO_RXQ_OVFL` on Linux, nothing on Darwin
- `CMSG_BUF_SIZE` / `parse_drops` — the ancillary control buffer and its
reader, which only Linux populates
- `recv_batch` — `recvmmsg` against `recvmsg_x`
`io/unix_other.rs` supplies the same three for the Unix targets that are
neither: FreeBSD, and Android, which is `target_os = "android"` and is
the one of the two that CI lints. Naming them is the point — this arm
was previously spelled `not(target_os = "linux")` and was only correct
because of a `cfg` two files away.
`io/windows.rs` stays a whole separate backend over
`tokio::net::UdpSocket`. It shares no implementation with the Unix side,
so it exports the same two type names and nothing else.
The connected-socket fast path becomes `io/connected/`, and this is the
part that was split across two trees. Constructing the fd lived in
`io.rs`; the handle that adopts it and the drain thread that must
accompany it lived under `peer/`, which they had no reason to — that
module imported nothing from `peer`, reached only into
`transport::udp`, and pointed at it three times in its own docs to
explain itself. All three pieces are now siblings: `fd.rs`, `socket.rs`
and `drain.rs`. The four platform differences in fd construction — the
`socket(2)` type flags, the follow-up fd-flag call, the service-type
tuning and the buffer-size strategy — join the same `sys` seam as the
receive code, which is what makes this the right parent for them.
`sockopts_macos.rs` moves in beside them as `io/macos_sockopts.rs`: only
this module ever consumed it, so it drops from `pub(crate)` to a private
child, and its file-wide `dead_code` allowance for the unreferenced
service-type table stays confined to the constants it was written for.
No behaviour change. Every call sequence, error path and syscall order
is preserved, including the order in which the connected socket applies
its options and which of those calls propagate errors rather than being
best-effort. The crate-facing names are unchanged except for the two
handle types, which move with the module and are re-exported as
`transport::udp::{ConnectedPeerSocket, PeerRecvDrain}` beside the
`open_connected_fd` that was already there.
Connected peer drains currently issue one recv syscall for each queued datagram on macOS even though the wildcard UDP path already uses recvmsg_x.
Reuse the Darwin batch ABI for connected sockets, preserving partial-batch and EINTR behavior while requesting up to 32 datagrams per receive call. Add a connected-socket burst test that crosses the batch boundary.
Adds FreeBSD as a supported platform. The daemon, fipsctl, TUN datapath and
DNS integration build and run there, with a native pkg and an rc.d service.
The one piece of genuinely new datapath logic is the TUN framing. FreeBSD's
tun rejects every non-IPv4 packet with EAFNOSUPPORT unless TUNSIFHEAD is set,
so nothing IPv6 can be sent at all; with it set, every frame carries a 4-byte
network-order address-family prefix the way macOS utun does. The ioctl is
issued at device creation and the prefix is stripped on read, which gives
callers the same raw-IP contract as Linux and macOS. A frame carrying only
the header reads as zero bytes and the reader loops treat it as nothing to
do. The address family is now taken from libc rather than hardcoded, because
AF_INET6 is 30 on Darwin and 28 on FreeBSD.
The reader shutdown path, the writer's address-family header and the
supervisor's shutdown pipe were all macOS-only and are now shared with
FreeBSD, since neither platform wakes a blocked read when the interface goes
down. Linux continues to rely on interface deletion.
mdns-sd moves from 0.19 to 0.20 for socket-pktinfo 0.4.1, the first release
that builds on FreeBSD, which uses IP_RECVDSTADDR and IP_RECVIF instead of
Linux-style IP_PKTINFO. This is the only change here that affects every
platform rather than just the new one.
The config, ACL, hosts and keygen path constants now treat FreeBSD the same
as macOS, since both install under /usr/local/etc/fips. Those constants
arrived separately on maint and are merged here rather than duplicated: the
predicates widen to cover FreeBSD, the platform-gated tests widen with them,
and keygen keeps reading the shared SYSTEM_CONFIG_DIR constant rather than
reintroducing a literal.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
The macOS packaging installs config under /usr/local/etc/fips, wired through
the launchd plist and build-pkg.sh, but the default-path constants and the
config search path were hardcoded to /etc/fips for all Unix. On macOS the
daemon and fipsctl therefore looked in a directory that does not exist: the
ACL and host-map loaders hit their NotFound no-op arm and returned empty
state, so a populated peers.deny reported effective_mode "default_open" with
enforcement inactive, and host-file aliases went unloaded, with no error.
The peers.allow, peers.deny and hosts defaults now follow the platform's
packaging, and fipsctl keygen writes its identity there too. The config
search path keeps probing /etc/fips first and adds /usr/local/etc/fips after
it, so an existing install keeps working across the upgrade and the packaged
file still wins over a stale leftover. Both the macOS search-path entry and
the keygen output directory read one SYSTEM_CONFIG_DIR constant, so they
cannot drift apart. At startup the daemon warns once about hosts, peers.allow
or peers.deny stranded at the old location; the config file is deliberately
excluded, since both directories stay on the search path and a config left
behind is still read.
The control-socket snapshot tests repoint the ACL reloader at non-existent
paths under the temp dir, so the snapshot no longer reflects whatever ACL
files happen to exist on the machine running the tests.
Platform-gated unit tests pin both layouts, so a future refactor cannot
silently drift either one. Linux and Windows behavior is unchanged.
Adding a second system config directory moves the directory the daemon
derives the identity key path from, since that comes from whichever config
file loaded last. A host carrying fips.yaml at both locations would have
resolved fips.key to the new directory, found none, and under persistent
generated a fresh identity, silently changing its npub, routing address and
mesh IPv6 with no migration path. The daemon now adopts a key stranded at the
legacy path and warns to move it rather than generating one. The fallback is
confined to keys resolved from the system config directory, so a run using
./fips.yaml or a user config is never redirected to a system key.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
Carries up the FSP session address-to-key binding fix.
The only conflict was the test module's import line: master reads the FSP
message types from proto::fsp and proto::link, where maint still reads them
from protocol. Resolved to master's paths, with SessionMsg3 taken from
proto::fsp alongside SessionAck.
The responder recorded a session under the source address carried in the
datagram without ever checking that address against the static key the
Noise handshake had just authenticated. A peer could therefore complete a
genuine handshake while claiming another node's address, and the identity
cache, the session map and the address the IPv6 shim reconstructs on
delivery would all attribute its traffic to the node it named.
Derive the address from the authenticated key at the point the key first
becomes available in msg3, and reject the handshake when it does not match
the claimed source. The entry has already been removed by that point, so
returning drops the half-open session and neither the identity nor the
session is recorded.
The rekey responder path needed its own check rather than inheriting that
one. It returns before the initial path's code is reached, and it never
read the peer's static key at all, so a rekey could complete under an
established session with a different key than the one that opened it. It
now requires the key to be unchanged, which is the stronger comparison
available there, and abandons the rekey while keeping the existing session
intact on mismatch. Tearing the session down instead would have handed an
attacker a way to kill established sessions.
Both comparisons are on x-only keys. A stored peer key may carry a
synthesized even parity because npubs encode no parity, while the
handshake learns the true point, so comparing full keys would reject
roughly half of legitimate peers on every rekey.
Both rejections are counted separately in the session reject statistics.
The tests drive real Noise handshakes through the datagram entry point and
construct the mismatch rather than asserting the comparison exists.
The guard that stops a consumer naming the shared mutable test image had
three ways through it, so a regression could walk back in past something
that looked like it was watching.
It matched only the literal fips-test:latest, but docker resolves a bare
untagged fips-test to exactly that, so image: fips-test reintroduced the
defect verbatim and passed. The match now catches the untagged form while
still allowing the run-scoped tag and the documented FIPS_TEST_IMAGE
fallback.
It did not look at the build context at all, so reverting a build.context
to the shared testing/docker directory passed. That half of the original
defect is now guarded the same way as the image name, matching a path
whose last component is docker rather than exempting bind mounts and
per-run directories by name.
Its allowlist was whole-file, and the comment claiming none of those files
was reachable from a CI run was false for two of them. Entries are now
scoped to the specific reference that is justified, so a new shared-tag
reference anywhere in a formerly exempt file is still caught, and the
comment says plainly which files CI does reach and why their particular
references are safe. The guard's own exemption is bounded by asserting it
starts no docker command.
git ls-files failure is now checked rather than discarded, so a sweep that
could not run exits 2 instead of reporting a clean tree.
Each hole was reproduced against both the old and new guard: all three are
red now and were green before. The healthy tree still passes, and the
legitimate forms do not false-positive.
Master-line half of the same correction made on the maintenance line. The header
comments on the connected-socket module inside the UDP io layer and on the
per-peer drain thread both said the fast path was not yet wired into the
encrypt-worker dispatch site, and that a follow-up would do it. It has been
wired at both ends for some time: the rx loop calls
activate_connected_udp_sessions, which spawns PeerRecvDrain. A reader deciding
whether the connected-UDP send path is live got the wrong answer from the first
comment they met.
Both modules already document themselves, so the forecast comments are deleted
rather than reworded, and the dead_code allowances they justified go with them.
The bare allowance on the connected-socket handle, which carried no comment at
all, goes too. None of the three was suppressing anything: clippy with
-D warnings passes with all of them removed, so besides being unjustified they
would have hidden whatever went dead next in those modules. The item-level
allowance on local_addr is a different attribute and is left alone.
Its header said the tuning was dormant, and that the module was kept visible on
Linux so clippy would not lose track of it. Both claims are false:
apply_udp_socket_tuning is called from the connected-peer socket opener under
cfg(macos), and the module declaration is itself gated to macOS, so it is not
compiled on Linux at all.
Unlike the two sibling modules, this allowance stays, because here it is doing
real work: the module defines the whole NET_SERVICE_TYPE table for reference and
selects only OAM, which leaves eight constants deliberately unreferenced. The
comment now says so, which is the justification the attribute was missing.
The header comments on the connected-peer socket and on its drain thread both
said the fast path was not yet wired into the node tick, and that a follow-up
would do it. It has been wired at both ends for some time: the rx loop calls
activate_connected_udp_sessions, which spawns PeerRecvDrain. A reader deciding
whether the connected-UDP send path is live got the wrong answer from the first
comment they met.
Both files already carry module docs describing what they are, so the forecast
comments are deleted rather than reworded. The dead_code allowances they
justified go with them: clippy with -D warnings passes without either, so they
were suppressing nothing, and while they sat there they would also have hidden
whatever went dead next in those modules.
The traversal adoption log line moved from src/node/lifecycle.rs to
src/node/lifecycle/mod.rs on this line, which git saw as a modify/delete
rather than a rename, so the change was re-applied by hand at the new
path and the stray maint-path file dropped. Both fields it reads are
present on this line's BootstrapHandoffResult in src/nostr/handoff.rs.
The "Adopted NAT traversal socket" line carried only the peer npub, which
left two questions unanswerable from a capture.
An operator cannot join a host socket table against our adoption events
without the local address, so there was no way to confirm which socket a
stuck connected-UDP activation was contending with. And an adopted
transport inherits accept_connections from the configured UDP transport,
so it can admit peers beyond the one it was punched for: without the
transport id, several peers sharing one adopted transport and several
separate adopted transports produce identical log output.
Both fields already existed on BootstrapHandoffResult, so this only binds
the value the match arm was discarding.
The established-link msg1 metering is re-authored against this line's
handshake handler rather than taken from the merge. The two versions of
src/node/handlers/handshake.rs differ by roughly 1437 lines from the
earlier decomposition, so the conflict hunks spanned whole divergent
bodies, including this line's restructured dual-initiation handling,
and resolving them hunk by hunk would have spliced the two structures
together. The file was taken from this line and the change re-applied
by hand; the resulting diff against the pre-merge tip is the metering
change and nothing else.
The rate limiter itself needed no adaptation: src/node/rate_limit.rs
was byte-identical on both lines, so the guard, the second bucket and
the derivation merged in unchanged.
Eighteen explicit pending-slot releases in handle_msg1 are replaced by
a single guard held for the function's scope. Every one of the
eighteen either preceded a return or ended a match arm, and the
success path released at the end of the function, so the guard is
behaviour-equivalent. One ordering difference: the slot is now
released at function exit rather than immediately before each arm's
reject-stat bump. Nothing outside the handler reads the pending count
between those two points, so it is unobservable in tree, but it is an
ordering change rather than a pure refactor.
Four tests are ported from the other line, one with its wire-module
import path adjusted for this line's layout.
Known coverage gap: no test distinguishes the guard being held from
the guard being released at acquire time. Rebinding `_slot` to a bare
`_` leaves the whole suite green and draws no clippy warning, so the
invariant rests on the binding name. Noted at the site.
open_connected_fd returned a bare last_os_error() from both the bind
and the connect path, and its caller wrapped both identically, so a
field report of "Address already in use" could not be attributed to
either. The two have entirely different causes: on Linux a UDP
connect(2) to a 4-tuple another socket already holds returns
EADDRINUSE, which is not the same fault as bind refusing the local
address. A node at roughly 245 peers is emitting this three times a
second across nine peers and the report cannot be diagnosed as it
stands.
Name the syscall and the address it was operating on in each error,
the local address for bind and the peer address for connect. The
address is what makes the next step possible: the identity that can
collide is the resolved SocketAddr, not the configured transport
address, and nothing in the log carried it on the failure path.
This is diagnosis only and fixes nothing. The unbounded per-tick
retry, the process-global failure counter that hides it, and the
sockets that outlive their peers are separate changes.
Note that the resolved peer address now appears at warn level on the
failure path, alongside the peer's node address, where previously it
appeared only in the success-path debug line.
raw_os_error() on the returned error is now None, since wrapping
produces a custom error. No caller reads it: the sole consumer
flattens the error to a string, and the errno text survives in the
message. Recorded as a known gap rather than a defect.
The msg1 rate limiter ran before the established-peer carve-out, so a
rekey or restart msg1 arriving on an existing link was refused on
exactly the same terms as a stranger's first packet and the carve-out
below it never applied to the traffic it was written for. On a node
with many peers this refuses a large share of ordinary maintenance
traffic: a field node at roughly 245 peers refused 8753 msg1 in 25
minutes, and 159 of the 201 distinct sources were peers it already
held sessions with.
Classify the source before metering it, and give established-link msg1
its own token bucket instead of a bypass. A bypass was rejected
deliberately: the limiter is global precisely because UDP sources are
spoofable, and an established-peer exemption is by construction keyed
on source address, so metering the exempted class keeps that property
where bypassing discards it.
The bucket is derived from settings the operator already sets, burst
from max_peers and rate from max_peers, the rekey interval and the
resend budget, so raising the peer limit sizes it automatically rather
than leaving a constant nobody revisits. Both parameters can be set
explicitly; an explicit zero burst or non-positive rate is rejected at
config validation, because it would refuse every rekey msg1 from an
established peer rather than disabling the limit.
Split out the established-link test as its own predicate so the
rate-limit classifier and the accept_connections gate cannot drift,
and convert the limiter's pending slot to a guard released on drop.
The slot was previously acquired in one place and released explicitly
at eighteen exit paths; once some paths stop acquiring one, any path
that still released one would have freed a slot belonging to a
different in-flight handshake, lifting effective concurrency above the
configured maximum with no counter moving and no log firing.
The "Msg1 rate limited" line now reports which limb refused, the
pending count or the token bucket, which it did not distinguish
before.
Classification costs an O(peers) scan on every inbound msg1 including
refused ones, where the previous order refused at O(1). The scan is
only needed because addr_to_link is keyed on the unresolved dial
address; correcting that keying reduces this to a single O(1) lookup.
Recorded at the predicate.
The reap runs two selectors, one on the CI label and one on the
compose project, and each flag narrows only its own. So --run-id
alone leaves the project sweep broad and --project-prefix alone
leaves the label sweep broad, and either way the reap still destroys
every concurrent run on the host. Both single-flag forms look scoped
and are not.
The usage text asserted otherwise. It documented --run-id as leaving
other runs alone and --project-prefix as scoping the reap to a single
run, and both claims were false for the same reason. A caller passing
--run-id alone, on the strength of that line, force-removed the
containers of three concurrent runs on 2026-07-29, and its own
comment recorded the belief that it was scoped.
Rather than ask every caller to remember the pair, close the two
half-scoped states here. --run-id now derives the matching project
prefix when none is given, built from the existing base so the two
cannot drift, and an explicit --project-prefix still wins.
--project-prefix without --run-id is refused outright, because there
is nothing to derive a label scope from and the quiet failure is
someone else's run disappearing.
Passing neither flag is untouched: that is the deliberate "reap
everything" form a manual cleanup wants.
Verified against a decoy container labelled as another run: the
scoped reap leaves it up, and reproducing the old broad project sweep
removes it, so the check discriminates rather than passing because
there was nothing to reap. An earlier version of that check ran when
no CI containers existed at all and passed without proving anything.
Each peer must be sent the union of all other peers' inbound filters,
excluding its own contribution. Building that per recipient rebuilt
the whole peer-filter map and re-ORed it once for every peer, so a
tick that announced to R peers did R kilobyte-scale map builds and
R by T merges. At 240 peers this was 20.6 ms per tick, roughly half
the tick body, and it was steady work rather than a tail: the
per-interval maximum had a median of 34.5 ms.
Replace it with a prefix and suffix union sweep that produces every
target's filter in one pass, T merges instead of R by T. The packet
path gets the same treatment, since marking changed peers had the
identical shape once per inbound announce.
The result is exactly equal, not approximately. Merging is a bytewise
OR, so regrouping the unions cannot change the outcome, and a filter
whose size does not match is rejected before any byte is touched, at
every merge site in both the old and new arrangement. Every
accumulator here is default-sized, so an odd-sized peer filter is
skipped in the new code exactly where it was skipped in the old.
Cadence, the debounce, the sequence rule and the fill-ratio cap are
untouched. A sequence number is still drawn after the debounce
re-check and before encoding, so a suppressed peer still consumes
none and an encode failure still burns one.
Known trade-off, measured rather than assumed: the sweep does its
full O(T) work regardless of how many peers are ready, so a tick that
announces to only one or two peers now costs about twice what it did.
Break-even is around three ready peers, and the saving above that
grows without bound. The marking path is a pure win, since it always
targets every peer.
The per-tick stats snapshot ran a bech32 encode for every tracked
peer, and for the common mesh peer — one with no hosts-file entry and
no configured alias — it ran a second one, because the display-name
fallback chain bottoms out in the same encode. At 240 peers that was
14.1 ms per tick, a third of the tick body and its second largest
cost, all of it recomputing values that cannot change.
Cache the npub and the shortened npub on the peer at construction. An
npub is a pure function of the peer's public key, and the identity is
never mutated after construction: there is no setter, no identity_mut,
and no assignment to the field anywhere in the tree, so the cache
cannot go stale.
The display name itself is deliberately NOT cached. Two of its inputs
do mutate at runtime — the alias map and the host map, the latter
reloaded on this same tick — so a resolved name stored on the peer
would go stale on an alias change or a hosts reload. Only the
immutable component is memoized.
Tests cover both constructors, that the cached npub matches the
identity, and that the display name still tracks an alias change. The
memoization itself is asserted by pointer stability rather than by
timing, so it is deterministic under load. Three deliberate breaks
were each caught by exactly one test: re-deriving instead of
memoizing, populating one constructor's cache from the wrong source,
and reordering the display-name fallback so it stops honoring aliases.
Every node reported state: degraded permanently. The liveness probe
polled connect_task, which wraps a single Client::connect() call. That
call only spawns a per-relay background connection task and returns,
so the handle finished moments after start on a perfectly healthy
node, the supervisor saw a child exit, and the node latched Degraded
for the rest of its life.
Nothing behaved differently, because every consumer of NodeState
treats Degraded the same as Running. What was lost is the signal: a
genuine degradation was indistinguishable from the permanent false
one.
Watch the three service loops that cannot return by design instead —
the inbound notify loop, the advert publisher, and the refresh ticker.
Each is an unconditional loop, so a finished handle means a panic or
an abort, which is unrecoverable and matches the one-way ChildExited
latch in the supervisor. connect_task and relay_startup_task stay
deliberately unwatched, and the doc comment now says why, since
watching either reproduces this bug exactly.
The tests install task handles directly and check both directions:
that a finished connect_task alongside three live loops reports
healthy, which is the production configuration a few hundred
milliseconds after start, and that each loop dying on its own reports
degraded. Reverting to the old predicate reds four of five; replacing
the predicate with a constant false also reds four of five, so the
fix cannot pass by never reporting degraded at all.
process_pending_retries runs inline on the node's 1-second rx-loop
tick. For each due peer it awaited a Nostr relay fetch carrying a
2-second timeout, and discarded the result. With up to sixteen due
peers in one tick body, the timeouts stack: field profiling measured
single 2.00 s stalls as the common case and a worst tick of 12.4 s
against a 1 s period, with every other rx-loop arm delayed behind it
by as much as 4.2 s.
Spawn the refetch instead of awaiting it, matching the pattern the
failure arm of the same loop already uses thirty lines below. The dial
now uses whatever advert is cached at that moment and the refreshed
one lands for the next retry of that peer. Since retries are
backoff-paced, that defers the benefit by one backoff interval rather
than losing it, and the result was already being discarded, so nothing
downstream read it.
The test drives four due peers whose refetches all hang against a
local listener that accepts and never speaks, so the fetch burns its
full timeout with no network egress. Awaited, the call takes 8.0 s;
spawned, it returns in milliseconds. It also asserts every due peer
was still attempted and rescheduled, so a version that skipped the
dial entirely cannot pass it.
A traversal signal is addressed to a merge of the peer's NIP-17 inbox
relays, the relays its advert nominates for signaling, and our own DM
relays. The client pool is built once at startup from our configured
relays and never added to, and send_event_to rejects the entire send
with "relay not found" if any single URL in the list is outside that
pool, before contacting anything. So one relay we are not configured
with, anywhere in that merge, killed the whole attempt -- including the
sends to relays we do share and that would have carried the signal.
In an open-mode window on a public node this made discovery
non-functional: 309 traversal attempts, 290 explicit failures, zero
successes, every failure on "relay not found". Configured peers were
unaffected because they run a matching relay set.
Filter the merged list down to relays the pool holds before sending.
Our own DM relays are always in the merge and always in the pool, so
the result is empty only when no DM relay is configured at all, which
is already a total failure. Comparison is on the normalized RelayUrl
rather than the raw string, because that is how the pool is keyed --
a raw comparison would discard a configured relay spelled with a
trailing slash or a different host case, which is the same defect in
a quieter form.
Merge and filter are one synchronous function so the decision can be
exercised without a relay client. The pool is read via all_relays(),
which is the set send_event_to validates against; relays() is filtered
by service flags and would be narrower.
Two smaller fixes ride along. The responder now resolves its relays
before binding a socket and running STUN, instead of spending a STUN
round trip and holding an offer slot only to discover it has nowhere
to answer. And it gained the empty-list guard the initiator already
had, which gives BootstrapError::MissingRelays a condition it can
reach for the first time -- it was unreachable, since the merge always
appended our own DM relays.
A measurement run needs the .deb built with a non-default feature, and the
script had no way to express that: its argument loop took only --target,
--version and --no-build, and nothing forwarded a feature list or read one
from the environment. --features <list> now forwards to cargo-deb's native
-F.
The marking is the half that is easy to skip and matters more. The
auto-derived dev Version is built from the crate version, the commit date
and the sha, none of which change when a feature is enabled, so an
instrumented package and a default package of the same commit carried
byte-identical versions. Two consequences, and the second is worse than
the first: the node offers no way to tell which one it is running, and
reverting is an install of a version already present, which no-ops
silently and leaves the instrumented binary in place. That is precisely
the failure the per-commit version was introduced to prevent, reappearing
one level down. The Version now carries a +<features> marker, folded to
dots since underscores and commas are not legal there.
The marker sorts above the unmarked build, checked with dpkg
--compare-versions rather than assumed, so installing a feature build is
an upgrade and reverting is a downgrade: revert with dpkg -i, not apt
install. The ~dev ordering below a tagged release is preserved.
--features is refused together with --no-build, which would stamp the
marker onto whatever binaries happened to be sitting in target/ while
claiming the features had reached them.
Verified end to end rather than by reading: a real --features profiling
build produces a package whose fipsctl accepts the profile subcommand and
whose daemon carries the capture-file header text that only exists under
the feature gate. The release packaging path is untouched, shown by
diffing the cargo invocation the old and new scripts produce across five
argument shapes including the --version/--no-build pair the packaging
workflow uses; all five are identical.
Authored on master rather than maint, which takes bug fixes and CI or
tooling changes but not new capability, so the three copies of this
script now differ by design.
The mesh rehearsal could not exercise the write-error stop at all, and
that is not a gap in the rehearsal. Removing the sink file leaves the
writer's descriptor valid, so writes keep succeeding into the unlinked
inode and the capture runs on; mounting a tiny filesystem inside the test
container is refused outright. So the terminal state added for a failing
sink had no coverage from either direction.
Split one flush cycle out of the writer loop as a function returning what
it decided, so the loop owns the waiting and the state transition while
the decision is testable on its own. A sink that fails every write now
drives the error outcome directly, and a working sink is asserted to
continue so the first test cannot pass for a writer that always stops.
The residual gap is recorded at the test: what is covered is that an
error from the sink produces the error outcome rather than the cap
outcome, not that a real full disk reaches that branch.
The rehearsal itself was otherwise clean on a live three-node mesh. The
header carries node, build, platform, tick period and the reading
caveats; every step reports every interval; stopping returns in 53 ms
rather than waiting out the flush interval; an idle status reports no
path and no bytes; arming twice is refused naming the active file; an
unwritable directory fails the command; and the node kept serving
throughout. On an idle node the entry gap sits at one tick period and
arm starvation reads about 1.4 ms, which is small but not zero, so the
measurement is live rather than degenerate.
The tick arm runs twenty-six housekeeping steps in sequence on the one
runtime thread, and is polled last, so anything slow in it holds up
inbound packets, TUN traffic and control commands behind it. Field
evidence says that happens for over a second at a time, but the
attribution behind that is two months old and predates the
connect-on-send gate, the control read isolation, and the peer lifecycle
rework. This measures it rather than continuing to reason about it.
Per step it records exact count, max and total into fixed static
counters; a dedicated writer thread drains them every ten seconds to a
TSV under /var/log/fips, one file per capture, capped at 32 MB. Nothing
accumulates: the counters are swapped to zero each interval and the
thread holds no history. Arming is `fipsctl profile tick on`, served in
the control accept task so the toggle cannot queue behind the very
behaviour it measures, and it does not survive a restart.
The whole thing is behind a Cargo feature that is off by default,
because the risk worth eliminating is the twenty-six edited call sites
in the hot loop. With the feature off the macro expands to the bare
expression, which makes the default build's neutrality something you
read off the generated code rather than something a benchmark fails to
disprove.
The measurement that matters is how late each tick is against the
deadline it was scheduled for, since the arm is polled last and that
lateness is the delay. Two earlier designs derived it from the interval
between entries and both under-reported: the schedule is fixed, so a
steady delay leaves every gap exactly one period and any gap-derived
figure reads zero under precisely the sustained overload this is meant
to find. The interval hands back its own deadline, so the delay is now a
subtraction with no model behind it, and a test drives three late ticks
at a constant gap to keep it that way.
CI gains a default-features clippy and a feature-on build and test on
both runners, closing the gap left by clippy already running with all
features.
The NAT lab was the last suite pinning fixed IPv4 subnets, so two overlapping
runs collided on the wan and shared-lan bridges.
Each run now claims a free /24 for each bridge, scanning candidates and
advancing on an overlap while still failing fast on any other network-create
error. The claim exports NAT_WAN_PREFIX and NAT_LAN_PREFIX, and every routable
address in the compose file and the suite scripts derives from them, with
defaults that render exactly what the lab used before. The router-side LANs are
deliberately left pinned: they live inside per-container network namespaces,
never become docker networks, and cannot collide.
An external-network overlay lets the suites attach to the networks the run
already claimed instead of creating their own. Two of the three suite scripts
had no overlay hook, so they would have requested the claimed range a second
time; both now have one.
Host veth names are scoped by a short token rather than the full run id, which
overruns the fifteen-character interface-name limit at the default run-id
length, and the cleanup reaper's pattern is widened to match the new shape.
Networks are released inline on every exit path rather than in a trap, since a
trap written inside a shell function replaces the script-level handler and
would disable the whole run's teardown.
The NAT lab wrote its generated node configs to one shared directory, unlike
the static and firewall labs which already scope theirs by run. The directory
is bind-mounted by compose and read back by the suite scripts after the
containers are up, so two overlapping runs let the second run's generator
overwrite the npubs the first is about to ping.
Scope the directory with FIPS_CI_NAME_SUFFIX at all three places that have to
agree: the generator's output path, the ten compose bind-mounts, and the
CONFIG_DIR the suite scripts read back from. An unset suffix renders the plain
path the lab has always used, so a bare invocation and the GitHub matrix are
unaffected. The run teardown removes the per-run directory alongside the static
and firewall ones, and the gitignore is widened to cover the suffixed form.
Rendering the compose file with every profile and no suffix set reproduces
today's exact ten paths; setting a suffix moves all ten.
This lands on its own, ahead of the network and address work, so that a failure
of the two-overlapping-runs acceptance test can be bisected between the config
fix and the address conversion.
Carries today's maint batch: the local CI image-scoping work, which gives
each run its own build context and test image tag instead of writing the
shared mutable one, and the retirement of the bloom-storm chaos scenario.
Both apply unchanged here — the compose files, suite lists and matrix legs
they touch are identical on the two lines, so no branch adaptation was
needed. Parity stays symmetric across runners at 21 legs a side on this
branch and 24 on maint, the gap being the three rekey Docker suites maint
keeps by design.
The scenario guarded a real regression: a mid-chain tree update that
changed neither root nor depth leaking downstream as a sustained bloom
announce storm. But it was never once run against the regressed binary,
and its per-node ceiling was inferred from a post-mortem harness that no
longer exists in the tree. On the only surviving regressed measurement
the tail node's rate scales to roughly 7 sends per 30s, well under the
scenario's ceiling of 40, so it was never established that the assertion
could fire on its own bug class. The ceiling is also uniform per node,
calibrated against the flap target that is legitimately busy rather than
against the tail where the storm actually shows.
This removes coverage rather than relocating it, unlike the two earlier
retirements above it, and the comment in ci-local.sh records that gap
explicitly. The scenario, its README, the link_swap sim primitive and the
mesh-lab dispatch all stay on disk, so it remains runnable by hand.
Parity holds at 24 legs a side. The parity guard was break-checked by
re-adding the GitHub leg alone, which correctly reported the asymmetry.
Johnathan Corgan
Martti MalmiandJohnathan Corgan
fr34akyandJohnathan Corgan
redshiftandJohnathan Corgan