The reap runs two selectors, one on the CI label and one on the
compose project, and each flag narrows only its own. So --run-id
alone leaves the project sweep broad and --project-prefix alone
leaves the label sweep broad, and either way the reap still destroys
every concurrent run on the host. Both single-flag forms look scoped
and are not.
The usage text asserted otherwise. It documented --run-id as leaving
other runs alone and --project-prefix as scoping the reap to a single
run, and both claims were false for the same reason. A caller passing
--run-id alone, on the strength of that line, force-removed the
containers of three concurrent runs on 2026-07-29, and its own
comment recorded the belief that it was scoped.
Rather than ask every caller to remember the pair, close the two
half-scoped states here. --run-id now derives the matching project
prefix when none is given, built from the existing base so the two
cannot drift, and an explicit --project-prefix still wins.
--project-prefix without --run-id is refused outright, because there
is nothing to derive a label scope from and the quiet failure is
someone else's run disappearing.
Passing neither flag is untouched: that is the deliberate "reap
everything" form a manual cleanup wants.
Verified against a decoy container labelled as another run: the
scoped reap leaves it up, and reproducing the old broad project sweep
removes it, so the check discriminates rather than passing because
there was nothing to reap. An earlier version of that check ran when
no CI containers existed at all and passed without proving anything.
Each peer must be sent the union of all other peers' inbound filters,
excluding its own contribution. Building that per recipient rebuilt
the whole peer-filter map and re-ORed it once for every peer, so a
tick that announced to R peers did R kilobyte-scale map builds and
R by T merges. At 240 peers this was 20.6 ms per tick, roughly half
the tick body, and it was steady work rather than a tail: the
per-interval maximum had a median of 34.5 ms.
Replace it with a prefix and suffix union sweep that produces every
target's filter in one pass, T merges instead of R by T. The packet
path gets the same treatment, since marking changed peers had the
identical shape once per inbound announce.
The result is exactly equal, not approximately. Merging is a bytewise
OR, so regrouping the unions cannot change the outcome, and a filter
whose size does not match is rejected before any byte is touched, at
every merge site in both the old and new arrangement. Every
accumulator here is default-sized, so an odd-sized peer filter is
skipped in the new code exactly where it was skipped in the old.
Cadence, the debounce, the sequence rule and the fill-ratio cap are
untouched. A sequence number is still drawn after the debounce
re-check and before encoding, so a suppressed peer still consumes
none and an encode failure still burns one.
Known trade-off, measured rather than assumed: the sweep does its
full O(T) work regardless of how many peers are ready, so a tick that
announces to only one or two peers now costs about twice what it did.
Break-even is around three ready peers, and the saving above that
grows without bound. The marking path is a pure win, since it always
targets every peer.
The per-tick stats snapshot ran a bech32 encode for every tracked
peer, and for the common mesh peer — one with no hosts-file entry and
no configured alias — it ran a second one, because the display-name
fallback chain bottoms out in the same encode. At 240 peers that was
14.1 ms per tick, a third of the tick body and its second largest
cost, all of it recomputing values that cannot change.
Cache the npub and the shortened npub on the peer at construction. An
npub is a pure function of the peer's public key, and the identity is
never mutated after construction: there is no setter, no identity_mut,
and no assignment to the field anywhere in the tree, so the cache
cannot go stale.
The display name itself is deliberately NOT cached. Two of its inputs
do mutate at runtime — the alias map and the host map, the latter
reloaded on this same tick — so a resolved name stored on the peer
would go stale on an alias change or a hosts reload. Only the
immutable component is memoized.
Tests cover both constructors, that the cached npub matches the
identity, and that the display name still tracks an alias change. The
memoization itself is asserted by pointer stability rather than by
timing, so it is deterministic under load. Three deliberate breaks
were each caught by exactly one test: re-deriving instead of
memoizing, populating one constructor's cache from the wrong source,
and reordering the display-name fallback so it stops honoring aliases.
Every node reported state: degraded permanently. The liveness probe
polled connect_task, which wraps a single Client::connect() call. That
call only spawns a per-relay background connection task and returns,
so the handle finished moments after start on a perfectly healthy
node, the supervisor saw a child exit, and the node latched Degraded
for the rest of its life.
Nothing behaved differently, because every consumer of NodeState
treats Degraded the same as Running. What was lost is the signal: a
genuine degradation was indistinguishable from the permanent false
one.
Watch the three service loops that cannot return by design instead —
the inbound notify loop, the advert publisher, and the refresh ticker.
Each is an unconditional loop, so a finished handle means a panic or
an abort, which is unrecoverable and matches the one-way ChildExited
latch in the supervisor. connect_task and relay_startup_task stay
deliberately unwatched, and the doc comment now says why, since
watching either reproduces this bug exactly.
The tests install task handles directly and check both directions:
that a finished connect_task alongside three live loops reports
healthy, which is the production configuration a few hundred
milliseconds after start, and that each loop dying on its own reports
degraded. Reverting to the old predicate reds four of five; replacing
the predicate with a constant false also reds four of five, so the
fix cannot pass by never reporting degraded at all.
process_pending_retries runs inline on the node's 1-second rx-loop
tick. For each due peer it awaited a Nostr relay fetch carrying a
2-second timeout, and discarded the result. With up to sixteen due
peers in one tick body, the timeouts stack: field profiling measured
single 2.00 s stalls as the common case and a worst tick of 12.4 s
against a 1 s period, with every other rx-loop arm delayed behind it
by as much as 4.2 s.
Spawn the refetch instead of awaiting it, matching the pattern the
failure arm of the same loop already uses thirty lines below. The dial
now uses whatever advert is cached at that moment and the refreshed
one lands for the next retry of that peer. Since retries are
backoff-paced, that defers the benefit by one backoff interval rather
than losing it, and the result was already being discarded, so nothing
downstream read it.
The test drives four due peers whose refetches all hang against a
local listener that accepts and never speaks, so the fetch burns its
full timeout with no network egress. Awaited, the call takes 8.0 s;
spawned, it returns in milliseconds. It also asserts every due peer
was still attempted and rescheduled, so a version that skipped the
dial entirely cannot pass it.
A traversal signal is addressed to a merge of the peer's NIP-17 inbox
relays, the relays its advert nominates for signaling, and our own DM
relays. The client pool is built once at startup from our configured
relays and never added to, and send_event_to rejects the entire send
with "relay not found" if any single URL in the list is outside that
pool, before contacting anything. So one relay we are not configured
with, anywhere in that merge, killed the whole attempt -- including the
sends to relays we do share and that would have carried the signal.
In an open-mode window on a public node this made discovery
non-functional: 309 traversal attempts, 290 explicit failures, zero
successes, every failure on "relay not found". Configured peers were
unaffected because they run a matching relay set.
Filter the merged list down to relays the pool holds before sending.
Our own DM relays are always in the merge and always in the pool, so
the result is empty only when no DM relay is configured at all, which
is already a total failure. Comparison is on the normalized RelayUrl
rather than the raw string, because that is how the pool is keyed --
a raw comparison would discard a configured relay spelled with a
trailing slash or a different host case, which is the same defect in
a quieter form.
Merge and filter are one synchronous function so the decision can be
exercised without a relay client. The pool is read via all_relays(),
which is the set send_event_to validates against; relays() is filtered
by service flags and would be narrower.
Two smaller fixes ride along. The responder now resolves its relays
before binding a socket and running STUN, instead of spending a STUN
round trip and holding an offer slot only to discover it has nowhere
to answer. And it gained the empty-list guard the initiator already
had, which gives BootstrapError::MissingRelays a condition it can
reach for the first time -- it was unreachable, since the merge always
appended our own DM relays.
A measurement run needs the .deb built with a non-default feature, and the
script had no way to express that: its argument loop took only --target,
--version and --no-build, and nothing forwarded a feature list or read one
from the environment. --features <list> now forwards to cargo-deb's native
-F.
The marking is the half that is easy to skip and matters more. The
auto-derived dev Version is built from the crate version, the commit date
and the sha, none of which change when a feature is enabled, so an
instrumented package and a default package of the same commit carried
byte-identical versions. Two consequences, and the second is worse than
the first: the node offers no way to tell which one it is running, and
reverting is an install of a version already present, which no-ops
silently and leaves the instrumented binary in place. That is precisely
the failure the per-commit version was introduced to prevent, reappearing
one level down. The Version now carries a +<features> marker, folded to
dots since underscores and commas are not legal there.
The marker sorts above the unmarked build, checked with dpkg
--compare-versions rather than assumed, so installing a feature build is
an upgrade and reverting is a downgrade: revert with dpkg -i, not apt
install. The ~dev ordering below a tagged release is preserved.
--features is refused together with --no-build, which would stamp the
marker onto whatever binaries happened to be sitting in target/ while
claiming the features had reached them.
Verified end to end rather than by reading: a real --features profiling
build produces a package whose fipsctl accepts the profile subcommand and
whose daemon carries the capture-file header text that only exists under
the feature gate. The release packaging path is untouched, shown by
diffing the cargo invocation the old and new scripts produce across five
argument shapes including the --version/--no-build pair the packaging
workflow uses; all five are identical.
Authored on master rather than maint, which takes bug fixes and CI or
tooling changes but not new capability, so the three copies of this
script now differ by design.
Reconciles the tick profiler with this line's extra tick step. `next`
drives `resend_pending_fmp_rekey_msg3` from the tick arm and the master
line does not, so the step table gains a variant here and the arm gains
one more instrumented call.
The pinned row-count test is what caught it: it failed on the merge
rather than letting the emitted table and the call sites drift apart
silently, which is the failure mode it was written for.
The mesh rehearsal could not exercise the write-error stop at all, and
that is not a gap in the rehearsal. Removing the sink file leaves the
writer's descriptor valid, so writes keep succeeding into the unlinked
inode and the capture runs on; mounting a tiny filesystem inside the test
container is refused outright. So the terminal state added for a failing
sink had no coverage from either direction.
Split one flush cycle out of the writer loop as a function returning what
it decided, so the loop owns the waiting and the state transition while
the decision is testable on its own. A sink that fails every write now
drives the error outcome directly, and a working sink is asserted to
continue so the first test cannot pass for a writer that always stops.
The residual gap is recorded at the test: what is covered is that an
error from the sink produces the error outcome rather than the cap
outcome, not that a real full disk reaches that branch.
The rehearsal itself was otherwise clean on a live three-node mesh. The
header carries node, build, platform, tick period and the reading
caveats; every step reports every interval; stopping returns in 53 ms
rather than waiting out the flush interval; an idle status reports no
path and no bytes; arming twice is refused naming the active file; an
unwritable directory fails the command; and the node kept serving
throughout. On an idle node the entry gap sits at one tick period and
arm starvation reads about 1.4 ms, which is small but not zero, so the
measurement is live rather than degenerate.
The tick arm runs twenty-six housekeeping steps in sequence on the one
runtime thread, and is polled last, so anything slow in it holds up
inbound packets, TUN traffic and control commands behind it. Field
evidence says that happens for over a second at a time, but the
attribution behind that is two months old and predates the
connect-on-send gate, the control read isolation, and the peer lifecycle
rework. This measures it rather than continuing to reason about it.
Per step it records exact count, max and total into fixed static
counters; a dedicated writer thread drains them every ten seconds to a
TSV under /var/log/fips, one file per capture, capped at 32 MB. Nothing
accumulates: the counters are swapped to zero each interval and the
thread holds no history. Arming is `fipsctl profile tick on`, served in
the control accept task so the toggle cannot queue behind the very
behaviour it measures, and it does not survive a restart.
The whole thing is behind a Cargo feature that is off by default,
because the risk worth eliminating is the twenty-six edited call sites
in the hot loop. With the feature off the macro expands to the bare
expression, which makes the default build's neutrality something you
read off the generated code rather than something a benchmark fails to
disprove.
The measurement that matters is how late each tick is against the
deadline it was scheduled for, since the arm is polled last and that
lateness is the delay. Two earlier designs derived it from the interval
between entries and both under-reported: the schedule is fixed, so a
steady delay leaves every gap exactly one period and any gap-derived
figure reads zero under precisely the sustained overload this is meant
to find. The interval hands back its own deadline, so the delay is now a
subtraction with no model behind it, and a test drives three late ticks
at a constant gap to keep it that way.
CI gains a default-features clippy and a feature-on build and test on
both runners, closing the gap left by clippy already running with all
features.
The NAT lab was the last suite pinning fixed IPv4 subnets, so two overlapping
runs collided on the wan and shared-lan bridges.
Each run now claims a free /24 for each bridge, scanning candidates and
advancing on an overlap while still failing fast on any other network-create
error. The claim exports NAT_WAN_PREFIX and NAT_LAN_PREFIX, and every routable
address in the compose file and the suite scripts derives from them, with
defaults that render exactly what the lab used before. The router-side LANs are
deliberately left pinned: they live inside per-container network namespaces,
never become docker networks, and cannot collide.
An external-network overlay lets the suites attach to the networks the run
already claimed instead of creating their own. Two of the three suite scripts
had no overlay hook, so they would have requested the claimed range a second
time; both now have one.
Host veth names are scoped by a short token rather than the full run id, which
overruns the fifteen-character interface-name limit at the default run-id
length, and the cleanup reaper's pattern is widened to match the new shape.
Networks are released inline on every exit path rather than in a trap, since a
trap written inside a shell function replaces the script-level handler and
would disable the whole run's teardown.
The NAT lab wrote its generated node configs to one shared directory, unlike
the static and firewall labs which already scope theirs by run. The directory
is bind-mounted by compose and read back by the suite scripts after the
containers are up, so two overlapping runs let the second run's generator
overwrite the npubs the first is about to ping.
Scope the directory with FIPS_CI_NAME_SUFFIX at all three places that have to
agree: the generator's output path, the ten compose bind-mounts, and the
CONFIG_DIR the suite scripts read back from. An unset suffix renders the plain
path the lab has always used, so a bare invocation and the GitHub matrix are
unaffected. The run teardown removes the per-run directory alongside the static
and firewall ones, and the gitignore is widened to cover the suffixed form.
Rendering the compose file with every profile and no suffix set reproduces
today's exact ten paths; setting a suffix moves all ten.
This lands on its own, ahead of the network and address work, so that a failure
of the two-overlapping-runs acceptance test can be bisected between the config
fix and the address conversion.
Windows unit tests have been reding on next at a rekey-drain assertion that
reads like a defect in the drain logic. The drain logic is correct; the test
never reached the state it asserts on.
All three test-only backdating seams shared the fallback
checked_sub(age).unwrap_or_else(Instant::now). When the requested backdate
falls outside the monotonic clock's representable range, that assigns the
present instant, moving the timestamp forward rather than backward. A fresh
Windows CI runner has been up for less than the 600 seconds the drain test
asked for, so the window looked brand new, check_rekey correctly declined to
release the demoted session, and the assertion fired against innocent code.
The seams now route through a shared helper that panics naming the field, the
requested age and the real cause. The drain test asks for 30 seconds, three
times the ten-second window it needs to clear, rather than sixty times it.
Both properties were established by breaking them. Backdating by five seconds,
inside the window, reds the test, so it still depends on expiry rather than
passing vacuously at the smaller margin. Forcing the subtraction to fail
produces the new panic and its explanation rather than a silent reset. Full
cargo test --lib green at 1742, fmt and clippy clean.
No step ran on Windows, so the platform mechanism is inferred from the split
between runners and confirmed only when that job goes green on this tip.
Carries today's maint batch up: the local CI image-scoping work, which
gives each run its own build context and test image tag instead of
writing the shared mutable one, and the retirement of the bloom-storm
chaos scenario.
Resolved as the earlier per-commit merges did, in the two files where
this line has diverged — the node test module list and the static suite
compose file. Parity holds at 21 legs a side, matching master.
Carries today's maint batch: the local CI image-scoping work, which gives
each run its own build context and test image tag instead of writing the
shared mutable one, and the retirement of the bloom-storm chaos scenario.
Both apply unchanged here — the compose files, suite lists and matrix legs
they touch are identical on the two lines, so no branch adaptation was
needed. Parity stays symmetric across runners at 21 legs a side on this
branch and 24 on maint, the gap being the three rekey Docker suites maint
keeps by design.
The scenario guarded a real regression: a mid-chain tree update that
changed neither root nor depth leaking downstream as a sustained bloom
announce storm. But it was never once run against the regressed binary,
and its per-node ceiling was inferred from a post-mortem harness that no
longer exists in the tree. On the only surviving regressed measurement
the tail node's rate scales to roughly 7 sends per 30s, well under the
scenario's ceiling of 40, so it was never established that the assertion
could fire on its own bug class. The ceiling is also uniform per node,
calibrated against the flap target that is legitimately busy rather than
against the tail where the storm actually shows.
This removes coverage rather than relocating it, unlike the two earlier
retirements above it, and the comment in ci-local.sh records that gap
explicitly. The scenario, its README, the link_swap sim primitive and the
mesh-lab dispatch all stay on disk, so it remains runnable by hand.
Parity holds at 24 legs a side. The parity guard was break-checked by
re-adding the GitHub leg alone, which correctly reported the asymmetry.
The run built per-run images and then retagged them to fips-test:latest as
a compatibility bridge for the consumers that had never been migrated. The
bridge was the defect: while it existed, two concurrent runs shared one
binding, so a suite could start containers from the other run's binaries
and the verdict was recorded against a commit whose code never ran. It
fails silently by construction — the run is green either way, and nothing
compares a running container's binary against the commit under test.
Every consumer now reads the run's image, so the retag is deleted rather
than kept. That is the point of deleting it: a consumer that was missed
fails loudly instead of quietly resolving whichever run wrote the tag last.
Two consequences handled here. The cleanup script ran ip(8) inside
fips-test:latest to reap simulation interfaces, and nothing writes that tag
any more, so it now takes the caller's image, then the run's, then any
surviving test image — the last of which is what keeps an unscoped --reap
working, since that path execs before the run identity is exported. And a
guard checks statically that nothing names the shared tag, because on a
host with a hand-built copy lying around a reintroduction would run green;
it is break-checked against a reintroduced compose consumer and a
reintroduced default. Both runners gate on it, as they do the other guards.
Scoping the image tag was never sufficient on its own. Every build read one
unscoped directory in the working tree, into which each run copies the
binaries it just built, so two concurrent runs raced on the contents of the
context as well as on the name of the result — and a run could produce a
correctly-per-run-tagged image built from the other run's binaries.
The run now copies the context's tracked files into its own directory,
installs its binaries there, and builds from it, exporting the path so every
other consumer follows. Deliberately not carried over: a previous run's
binaries, since inheriting them is the failure this prevents. The path is
absolute because compose resolves a relative build context against the
compose file's own directory rather than the working directory, which was
measured rather than assumed.
The chaos entry script gated the whole simulation on a binary in the shared
directory by literal path, so it moves in this same commit: left behind, it
would have failed on a clean checkout and, worse on a host with leftovers,
passed while reading a binary that was not the one under test.
Teardown removes the directory on red runs as well as green, since it holds
only reproducible content and is never the evidence of a failure. The
worker's SIGKILL runs no trap, so the cleanup script also sweeps contexts
left by a preempted run, and the ignore rule keeps a concurrent run from
showing up as untracked working-tree noise.
The runner rebuilt the shared tag from the shared build context at the
start of every scenario, unconditionally. Local CI runs chaos scenarios in
parallel, so that was several concurrent builds into one name inside a
single run, before any second run is considered, and it silently replaced
whatever image the harness had already built and handed over.
It now uses the caller's image when one is named, and asserts the image is
present rather than building a substitute, because under a harness a miss
means something upstream is broken and building would hide it. A bare run
still builds the shared tag exactly as before, reading the run's own build
context when one is set.
Only the static family read FIPS_TEST_IMAGE; the firewall, ACL, nat,
sidecar and both Tor compose files named the shared mutable tag directly,
which is why the local CI runner has to retag its per-run image to that
name. They now take the same defaulted form the static file already uses,
so a harness run resolves the image it built and a bare hand run still
resolves the shared tag exactly as before.
Checked by rendering each file through `docker compose config` with the
variables set: every service resolves to the supplied name and none is
left naming the shared tag.
Both suites ran `docker compose up -d --build` unconditionally, and their
compose files carry both a build context and an image name, so the flag
rebuilt and retagged the shared test image from the shared build context on
every run. Their --skip-build flag did not cover it: it guarded only the
Rust build above. Under a harness that has already built the image and
handed it over, that rebuild can only replace the binaries under test with
whatever the shared context currently holds.
The nat scripts had the mirror-image problem. Their image guard built a
replacement whenever the image was missing, which is right by hand and
wrong under a harness: when the caller names an image it built, a miss
means something upstream is broken, and manufacturing a substitute hides
that. They now fail loudly in that case and keep building only on the hand
path. The mesh-lab guard reads the same variable and its build hint moves
to the script that still produces the shared tag.
The root-count ceiling of 4 was set from six runs whose observed maximum
was 3. Fourteen runs that recorded a baseline verdict put the real
distribution at 1 to 5 roots, so the ceiling sat at roughly the 93rd
percentile of the scenario's own variance: one sample in fourteen
exceeded it and four sat at or above it. It therefore reddened a share of
runs whatever the daemon did, which is a calibration fault rather than a
convergence one. Every sample ran the file's fixed seed, so the spread is
container timing and not a differing chaos schedule.
The ceiling is now 6, one step beyond the observed maximum, and the
parented floor moves to its complement at 4. That still fails a mesh
where seven or more of ten nodes are islanded, which is the collapse case
the assertion exists to catch; driving evaluate_baseline directly
confirms all fourteen samples pass and 7, 8 and 10 roots still fail.
min_sessions is left at 10 against an observed minimum of 12. It has
never fired, but the comment now records the margin as thin so a future
failure there is read as calibration first.
All three required root or CAP_NET_RAW, neither runner passes --ignored,
and so none had ever executed in CI. Before deleting them I checked what
each covered and whether anything else covered it:
- The two-node handshake is covered by the ethernet-only chaos scenario,
whose baseline assertion cannot pass unless handshakes complete over
AF_PACKET.
- Tree convergence and root election by smallest NodeAddr are covered by
ethernet-only's single-root assertion and by the in-process
spanning-tree tests respectively.
- The mixed-transport coexistence property is covered twice over by
test_tcp_mixed_transport_coexistence and test_ble_mixed_transport, which
call the same verify_tree_convergence_components helper with the same
two-component shape and both run today.
So the only thing running them would have added is an AF_PACKET variant of
a property already proven on two other transports.
The counts confirm nothing running was lost: 1392 tests pass before and
after, and the ignored count falls from 7 to 4.
The ethernet-only comment block cited these tests as the reason its
assertion was the only Ethernet coverage in CI. It now records the sharper
fact found while reading them: that coverage is control plane only. Both
Ethernet scenarios disable traffic, so no datagram crosses an Ethernet
link anywhere, which leaves the frame length field that trims NIC minimum
frame padding ahead of AEAD verification unexercised. Deleting these tests
does not widen that gap, because the test named for data exchange did not
exchange any.
The shared analyzer matched two strings for one event: "Peer promoted to
active", emitted at info, and "Outbound handshake completed", emitted at
debug. Both come from handle_msg2 on the same call path for the same
outbound promotion, so any run at debug level counted every outbound
promotion twice.
Measured rather than argued. On the archived debug-level churn-mixed run
of 2026-07-25, the node logs carry 20 of each string; the old matcher
reported 40 promotions and the new one reports 20. The neighbouring
counters are unmoved at 13 sessions and 33 parent switches.
Info-level runs are unaffected, which was checked rather than assumed: on
the tcp-mesh run of the same date the debug string occurs zero times, and
the old and new matchers both report 7 against 7 raw occurrences. Three
scenarios run at debug, one of them gating, so this corrects the promotion
figure in their artifacts. No assertion reads the counter, so no suite's
verdict changes.
The earlier fix that introduced this added the live string alongside the
dead one where it needed to replace it.
The tcp-chain profile has never been runnable. ping-test.sh dispatches on
chain and mesh only, and the string tcp-chain has never appeared in that
script in its history, so invoking the profile has always fallen through
to the unknown-profile branch: silently asserting nothing before that
branch was made to fail, and exiting 2 since. Neither runner referenced
it either.
What it would have covered is already covered. The chaos tcp-mesh
scenario runs in local CI and gives TCP a discriminating gate: n04's only
edges are TCP, so if the transport were broken n04 could not parent and
the scenario's baseline assertion would fail. It also carries a pure-TCP
two-hop path, n01 to n04 to n05, under netem and link flaps. The only
residual tcp-chain would have added is a mesh with no UDP present
anywhere, which is not worth a fixture that has never run.
Remove the topology, its three compose services, and the documentation
rows. Every profile the static compose file still defines is now
exercised by a suite.
Also correct the chaos README, which has listed tcp-only and tcp-chain in
its transport table since those two scenarios were deleted in 9e63b42.
Neither has existed for months.
The mesh-public profile ran in neither runner: ci-local's static suite
list carries only static-mesh and static-chain, and the GitHub matrix has
only mesh and chain. It also added no coverage over static-mesh.
ping-test.sh and iperf-test.sh branched mesh and mesh-public together and
exercised the same 20 directed pairs among node-a through node-e, and no
script referenced the external node at all. The convergence waits even
used mesh's peer counts rather than mesh-public's, so the extra link to
the public node was never counted, let alone asserted.
Running it would therefore have added a dependency on a live internet
host (test-us01.fips.network) in exchange for zero additional assertions.
Remove the topology, its five compose services, the script branches that
aliased it to mesh, and the documentation rows. An invocation using the
old profile name now fails on ping-test.sh's unknown-profile guard
instead of silently behaving as mesh.
The config generator's external-node support (external_ip,
is_external_node) stays. It has no consumer now, but it is woven into the
config path every remaining topology uses, so removing it would put the
gating suites at risk for no present gain.
wait_for_peers reads the connected-peer count through a pipeline ending in
`|| echo 0`, so a container that never answers contributes 0. That is safe
against a floor of 1 or more, where 0 reads as "not converged yet" and the wait
eventually times out, and it is unsafe against a floor of 0, where the first
read from a dead container satisfies the wait and the caller proceeds as though
convergence had been observed.
No caller passes 0 today; every one passes 1 or more, and the single variable
minimum is advisory. Rather than harden the reader, reject the input: a floor
below 1 is now an error. That makes the shape unreachable instead of repairing
instances of it, and it costs a future caller nothing except a clear message
saying that asserting "exactly zero peers" needs a reader that distinguishes no
answer from zero, which this floor is not.
wait_for_links goes with it. It had no caller anywhere in the tree on any
branch, and it carried the identical fallback, so the only thing it could do was
hand the hazard to whoever called it first. An uncalled helper cannot be wrong
today, which is exactly why it was the risk worth removing rather than the one
worth keeping for symmetry.
The hermetic gate suite gains a case covering both directions, with docker
stubbed so it stays container-free: a floor of 0 is refused with a message
naming why, and the same unreachable container with a floor of 1 still polls its
full budget and times out. Verified by removing the guard and confirming exactly
the three zero-floor assertions red while both floor-of-one controls stay green.
wait_for_peers_exact read the connected-peer count through a pipeline ending in
`|| echo 0`, so a container that never answered and a daemon that answered zero
produced the same value. That is only harmless while every caller expects a
non-zero count, which is true here today and is the reason this copy was left
alone when the acl suite's copy was fixed. It leaves the trap armed for whoever
adds the first caller expecting zero: the check would be satisfied on its first
iteration without the property it exists to verify ever being observed.
The read moves into its own function that returns the empty string when the
container does not answer, and the caller treats empty as "no answer" rather
than as a count. A run that never gets an answer now fails saying so, distinctly
from one that answered the wrong count, and the diagnostic peer dump runs only
on the latter, since dumping from a container that cannot answer prints a docker
error rather than evidence.
This is the shape the acl-allowlist suite already uses. The two copies had
diverged on whether a silent container counts as an answer, which is the kind of
disagreement that decides a security assertion in whichever file is read last.
Checked by construction rather than by a green run: driving the old form against
an absent container with an expected count of zero returns success on the first
iteration, and the new form fails; with an expected count of one, the shape both
real callers use, both forms still fail, and a genuine zero from a live daemon
is still reported as zero.
The compose pinned 172.32.0.0/24 with a per-container ipv4_address, so two
concurrent runs asked docker for the same address space and the second failed
with a pool-overlap error. Request no subnet and let docker assign one from the
daemon pool. Peers address each other by the docker hostname the compose already
sets (host-a, host-b) rather than by literal IP; docker's embedded DNS is
per-network, so the same hostname in two runs resolves inside each run's own
subnet. Nothing this suite asserts on moves as a result: its checks run over the
fips0 overlay, whose addresses are derived from node npubs and are independent of
docker addressing, and the packaged nftables ruleset matches on interface rather
than on any address.
The generated config directory moves under the run suffix for the same reason it
did in the acl suite, and the run teardown gains the matching removal so a run
no longer leaves its directory behind.
Case (b) also wrote curl's output to a fixed path under /tmp. Two concurrent
runs shared that one file, and either run's cleanup landing between the other's
write and read left an empty read, failing the http_code check for a reason
having nothing to do with the firewall. It uses mktemp now.
As in the acl suite, the floating subnet removes one of the two obstacles to
concurrent runs. The compose project name is still fixed; the local CI runner
scopes it externally, a bare hand run does not, and the comment at the site says
so rather than claiming the file is self-sufficient.
The compose pinned 172.31.0.0/24 with a per-container ipv4_address, so two
concurrent runs of this suite asked docker for the same address space and the
second failed with a pool-overlap error. Request no subnet instead and let
docker assign one from the daemon pool, which leaves nothing for two runs to
contend for. Peers now address each other by the docker hostname the compose
already sets (host-a through host-f) rather than by literal IP, and docker's
embedded DNS is per-network, so the same hostname in two runs resolves inside
each run's own subnet.
The generated config directory moves under the run suffix in the same change,
because it has to: the generator does rm -rf on it, so a shared directory means
one run deletes the fixtures another is still using, which fails silently rather
than loudly at bring-up. Scoping it requires the generator output path, all five
bind-mount sources per service, and the gitignore entry together; scoping only
the generator leaves the compose reading the unscoped path.
The floating subnet removes one of the two obstacles to running this suite
twice at once, not both: the compose project name is still fixed, and nothing
scopes it for this suite. Recorded at the site so the comment does not claim
more than the change delivers.
Everything this suite asserted is now covered in-process over loopback by
mixed_profile_nodes_converge_and_forward: mixed Full, NonRouting and Leaf
convergence, the peer degrees, every direct reachability pair, and the
Leaf-to-Full pair that routes through the Full node between them. The
in-process test covers a superset, asserting that multi-hop path in both
directions where the Docker suite covered one.
Retired on evidence that the multi-hop assertion discriminates rather than on a
green run: reverting the leaf root-election gate reds it in roughly a third of
runs, with the leaf reporting itself as tree root. That was the property the
suite was being kept for, and it is the property now demonstrated elsewhere.
Removed from the local runner's suite list, its runner function, the default
flow and the single-suite dispatch, and from the workflow matrix and its five
steps. Parity reports 22 legs a side, one fewer than before and equal across
the two runners. The suite's script, topology and compose profile stay on disk
and it remains runnable by hand; the retirement is recorded with the others in
the runner's deliberately-not-run block.
The rekey tests configured a one-second interval to put the timer arm within
reach, then backdated the session two minutes so the jitter draw could not
decide the outcome. That worked, but it asks for a configuration the daemon now
refuses to build: an interval at or below the jitter bound saturates to zero on
a negative draw and rekeys on sight for roughly half of sessions, which
validation rejects.
Thirty seconds clears the bound and leaves the existing backdate a margin of
seventy-five seconds against the worst draw, so the tests keep the property
they were relying on. The helper that drives these handshakes now records why
the value cannot simply be made smaller again.
Surfaced by merging the deployed line up, which is the first point at which
these tests met the new validation.
Clean merge, no conflicts. The hop-limit helper corrections and the rekey
config validation arrive from the deployed line and apply here unchanged.
Checked rather than assumed, since this branch changed the same subsystem in
the same session: next keeps its own split of the rekey cadence, where the
config flag rides into the core as RekeyCfg::initiate and check_rekey no longer
returns early, and the incoming validation of the same config block sits
alongside it without overlap.
Both maint changes carry over unchanged. The hop-limit helpers follow the
module rename from src/protocol/link.rs to src/proto/link.rs, applied by
rename detection and verified at the renamed path rather than assumed; the old
path is not recreated. The rekey config validation and its jitter-constant
import apply as written, the node module exposing the constant on this branch
as well.
The only conflict was the unreleased changelog, where master's own Changed
entries and the incoming hop-limit entry were both correct for their branch.
Resolved as the union, with the incoming entry appended to master's Changed
list.
The in-process mixed-profile test asserted convergence, peer degrees and every
direct pair, but deliberately left out the Leaf-to-Full pair that routes
through the Full node between them: a leaf holding a small enough address used
to self-elect as tree root and partition the mesh, so the assertion would have
been flaky. That defect is fixed, so assert the pair here, in both directions.
The leaf's peer degree is already pinned at one in the same test, so neither
direction can be satisfied by a direct link.
Failures in this test are root-election partitions keyed on the random
per-node address ordering, which are undiagnosable from a bare assertion
message. Each of the three failure paths now reports every node's address,
elected root and self-rooted flag.
Reverting the self-election gate reds the new pair in roughly a third of runs
with the leaf reporting itself as root, and the assertion is clean over
repeated runs with the gate in place.
Config validation said nothing about the rekey block, so two settings that
disable rekey in appearance and hammer it in practice were accepted silently.
after_messages of zero makes the message-count arm true on every poll, since
the trigger tests the counter against it with a greater-or-equal. It reads like
a way to switch the arm off and does the opposite.
after_secs at or below the per-session jitter is the same trap on the timer
arm. Each session offsets the interval by a random value drawn from plus or
minus the jitter bound, so a smaller interval saturates to zero on a negative
draw and rekeys on sight, for roughly half of sessions and not the other half.
The rule is expressed against the jitter constant rather than its current
value, so it tracks if the bound ever moves.
Both are checked whether or not rekey is enabled, so turning it on later cannot
surface a configuration error at a surprising moment. Neither has an upper
bound: a very large value is the established way to disable one arm of the
trigger and stays legal.
The forwarding path was corrected to full IP semantics: local delivery is not
hop-limit gated, and forwarding decrements first and drops at zero. Two helpers
on SessionDatagram were left implementing the old rule. decrement_ttl still
checked before decrementing, and can_forward still answered true at a hop limit
of one, where the decrement leaves zero and the datagram is dropped. Neither is
called anywhere today, which is precisely why they are worth correcting rather
than leaving as an invitation to reintroduce the defect.
The TtlExhausted reject doc described behaviour that no longer exists. The
reject is charged for a transit arrival at one as well as at zero, and is never
charged for a datagram addressed to this node, whose delivery is decided ahead
of the test.
Both helpers are pinned by tests at the boundary, which fail against the
previous implementations.
The responder half of the establish decision was gated on this node's own
node.rekey.enabled. With the rekey now declared in the msg3 negotiation
payload, that flag was the only thing that could divert a msg3 whose marker
matched the session we hold, and it diverted it to a msg2 resend while the
initiator had already installed its pending session and would cut over on its
own timer regardless. A pair configured with the flag true on one end and
false on the other therefore parted company at the initiator's cutover and
carried no traffic in either direction until the link-dead timer.
Drop the flag from that gate. The arm now turns on the sender's declaration
matching the session we hold, which is the only signal that was ever
authoritative for the decision, and the snapshot field it read is gone since
every initiator-side use reads the config directly.
Removing the gate exposed a second reading of the same flag. check_rekey
returned early when rekey was disabled, and the drain-expiry arm of the rekey
poll is the only thing that releases a demoted session and its index. A node
that does not initiate had never accepted a rekey before, so it never
accumulated a drain; now that it does, the early return would pin the previous
session and its allocator index forever. The flag rides into the core as
RekeyCfg::initiate instead: the polled cutover and the trigger are gated on it,
the drain is not, and the shell runs the cadence unconditionally.
The cutover arm has to stay gated. The peer snapshot carries no rekey role, so
a pending session stored as responder satisfies that arm exactly as one we
initiated does; ungating it would move a non-initiating node's cutover off the
peer's k-bit flip and onto the poll.
Covered by a two-ended test asserting the pair converges on the same session
indices under an asymmetric setting, a node-level test that a non-initiating
responder actually releases the demoted session and index once the drain window
expires, and a core test pinning which arms a non-initiating snapshot yields.
The existing msg2-resend test reached its arm only through the removed gate and
now reaches it through the health conjunct instead.
The doc comment merged up from the maintenance line lists the map's live
readers, and on this branch there is one more: the anonymous-beacon dial
dedup in poll_transport_discovery, which has no counterpart there.
It reads the map with an address supplied by transport discovery, and the
shared-media dial paths resolve before registering, so it compares a key
written in the same form it reads and is not exposed to the mismatch the
comment warns about.
Carries the address-to-link map's corrected doc comment. The only
conflict was the adjacent line naming the discovery config keys, which
this line renamed to node.lookup.* and the maintenance line did not; kept
this branch's names.
The doc said the map "enables dispatching incoming packets to the right
connection before authentication completes." It does not: find_link_by_addr
has no callers outside its own tests on any branch, and encrypted frames
dispatch by session index. Its live readers are the msg1 admission fast
path and the duplicate-inbound-handshake check.
The comment mattered because the map has a trap that invites exactly the
misuse it was advertising. An outbound dial registers the literal
configured address string, which may be a hostname, while an inbound
packet carries the resolved form; TransportAddr compares byte-wise, so
the two never match and a lookup keyed on an inbound address returns "no
such link" for every hostname-configured peer rather than failing. The
entry is also single-valued per key, so an inbound handshake overwrites
an outbound dial's entry for the same address.
Both current readers survive this because each compares a key written in
the same form it reads. That is a property of those two call sites and
not of the map, so the comment now says so, and says not to key a
peer-identity question on it.
Documentation only; no behaviour changes.
Carries the rekey counter-arm boundary test and the removal of the
closure-based trigger test. Applied cleanly with no adaptation: the
trigger predicate is byte-identical on both lines, and the claim the new
test rests on was re-measured here rather than assumed to carry — widening
the bound to `p.counter >= 1` reds exactly one test on this line too.
The send-counter arm of the rekey trigger had no test that could fail for
the reason it existed. The integration suites never reached it: they
inject after_messages = 65536 against a ping-driven workload, so every
rekey they observe is time-triggered. The unit test that looked like the
guard reproduced the trigger's OR predicate as a local closure and
asserted against its own copy, so deleting the arm from the real
predicate left it green.
That is the coverage hole that let the data-plane drop across a
message-count-triggered rekey survive: the defect is precisely a rekey of
a young session, which only the counter arm can produce.
Replaces the existing at-threshold check with a boundary test driving
poll_rekey, holding the time arm off so the counter is provably what
decides. The silence-below half is the part that is new, and it earns its
place by measurement: widening the bound to `p.counter >= 1` reds exactly
one test in the library suite, this one. Deletion, narrowing to `>` and
inversion are caught here too, though the time arm's negative tests
already catch a bound removed altogether.
Removes the closure-based test rather than repairing it. Its one
assertion over real code, that a fresh session entry's jitter lies within
the symmetric bound, is already covered over 100 samples elsewhere.
Note this leaves the deployed maintenance line untested on the same arm:
its trigger is inline in an async method that reads the clock directly,
with no pure function to drive, so the same test cannot be written there
without the sans-IO seam this line already has.
Carries the in-process rekey continuity test and the retirement of the
three rekey Docker suites onto the XX line.
The continuity test was written against master's IK handshake, where the
responder decided rekey-versus-fresh-dial from session age and the test
had to backdate both sessions past a 30s acceptance gate. Here the
responder classifies from the rekey marker the initiator declares in its
msg3 negotiation TLV and reads no clock at all, so the backdating is
dropped: it would age the session past nothing, and leaving it in would
tell a reader a gate exists that does not.
Checked that the test still discriminates rather than merely observing a
session index change. Suppressing the declaration so the responder falls
back to the cross-connection path reds the post-cutover assertion in 8 of
16 runs, which is the address-ordering coin flip the original data-plane
drop had; the healthy tree is 0 of 16 red. The 30s backdate makes no
difference to either rate, which is what confirms it was vestigial here.
The suite wiring keeps mixed-profile, which master does not have, and
drops the three rekey suites from the local suite list, the runner
functions, the default flow, the --only dispatch and the GitHub matrix
together, so check-ci-parity.sh stays green at 23 legs a side. The
retirement note is retargeted at this branch's coverage: the establish
decision tests live in src/proto/fmp/tests/core.rs and src/peer/machine.rs
here, not in the establish_chartests.rs file master keeps.
The rekey marker's cross-connection arm declined to swap while a rekey of
ours was in flight, reasoning that the swap rewrites the session and both
indices while touching no rekey state, so it would orphan that rekey. The
reasoning was one-sided and the arm was wrong: this is half of a two-sided
tie-break whose other half reads only whether a peer exists and the address
ordering, and cannot see our rekey. Declining here while the peer's outbound
half swaps anyway leaves the two ends on four distinct indices with neither
holding a pending session, dead both directions until the link-dead timer.
Measured against the pre-marker tree, which converges: it holds the crossing
dial as pending and adopts it at cutover.
The orphaned-rekey hazard is real but belongs to the executor. The swap now
abandons the rekey it displaces and frees that index first, exactly as the
rekey-responder path already does when it loses the dual-rekey tie-break, so
the classifier can take the arm unconditionally.
Two further failure paths of the marker are tightened. A malformed marker
tore the handshake down without freeing the index msg1 allocated, and that
arm sits ahead of the ACL gate, so any peer able to complete a msg3 could
grow the allocator set one entry at a time. A rekey with no session index to
declare silently omitted the marker, which puts a real rekey back on the
cross-connection path; it is unreachable today and is now asserted rather
than skipped.
On the test side, the marker's wire contract gains coverage: the accessor
round-trips, an unknown TLV field leaves it alone in both directions, and a
well-formed marker of the wrong length is an error rather than an absence,
which is the case the container codec cannot see and the whole reason the
accessor is fallible.
Two node-level tests now assert both ends rather than one. A rekey fired by
message count on a zero-age session is the mechanism behind the datagram
drops, and nothing exercised it: it fails on the pre-marker tree and passes
here. The mutual-dial test gains an index-pair assertion, since every
per-end predicate it checked holds when the ends have diverged.
One existing test had stopped reaching its own arm. Driving a bare second
handshake into a rekey-disabled peer now lands on the cross-connection arm,
which that flag does not gate, and every assertion held there too; it is
retargeted onto a declared rekey, which is the real route to the duplicate
arm, and asserts the session is left alone so the two arms are told apart.
Documentation that described the removed session-age floor is rewritten
around the declaration.
A rekey and a fresh dial both arrive as a new msg1 on a new link, so the
responder has to tell them apart at msg3. It cannot: the distinguishing
fact is internal to the initiator and nothing on the wire carries it.
Session age stood in for it and could not do the job - a
message-count-triggered rekey fires on a young session, so a real rekey
landed below the floor, was resolved as a cross-connection, and left the
two ends of the link on different session indices.
Carry the answer instead. The initiator adds a negotiation TLV to its
rekey msg3 naming the session it replaces, using the index the receiver
allocated so the receiver can match it against its own. The shell
resolves that into a three-valued claim - none, matching, mismatched -
and the classifier branches on it.
A mismatched marker resends msg2 rather than rejecting: the initiator has
already installed its pending session and will cut over on its own timer,
and the reject path sends nothing back, so tearing down would strand the
link in the way this defect already does. A fresh dial arriving while our
own rekey is in flight also resends, because the cross-connection swap
rewrites both indices and touches nothing else, which would leave that
rekey pointing at a session the counterpart no longer holds. The age
floor was excluding that case incidentally.
The floor and the session-age snapshot field are gone with it, so the
establish snapshot no longer reads the clock at all.
Both node-level rekey tests now drive a real rekey through the
initiator's own trigger. They previously reached the rekey arms by
backdating a session and re-handshaking, which tested the age proxy this
replaces; one of them was reaching its assertions through the duplicate
arm rather than the arm it names. Age them well past the trigger, since
the jitter is +/-15s and a smaller margin makes firing depend on the draw.
Incomplete: no TLV round-trip test, no two-ended convergence test, no
rekey-crossed-with-fresh-dial case, and no integration run yet.
The static suites hardcoded RUST_LOG=info, so raising the log level for a
node under test meant editing the compose file. Take it from the
environment with info as the default, matching the passthrough the same
block already uses for the worker-pool overrides.
Rendering is unchanged when RUST_LOG is unset, which is how CI invokes it.
A leaf-profile node that held the smallest NodeAddr would self-elect as
tree root, but its peers refuse a non-full node as a parent, so it formed
an isolated second root and partitioned the mesh: a multi-hop session from
the leaf to a non-adjacent full node then failed because the far node could
not route a handshake reply back into the leaf's separate coordinate tree.
Gate tree-state self-election on a new self_is_leaf flag, set from the node
profile at construction. A leaf now attaches under its full upstream and
holds that subtree's coordinate for its own routing. That coordinate's
self < root would be rejected by the root-min wire check, but a leaf never
announces it; it reaches peers only via the coordinates carried on the
leaf's session frames, and the upstream already advertises the leaf in its
bloom filter, so the far node routes back through the upstream.
Also fix Node::with_identity to derive the node profile, leaf-only flag,
and bloom state from the config, matching Node::new; it previously
hardcoded the full profile and silently dropped a configured leaf or
non-routing profile.
Covered by sans-IO decision tests (leaf does not self-elect; leaf keeps its
coordinate under a larger-rooted parent) and an end-to-end multi-hop
regression, each confirmed to fail if the gate is reverted.
Brings up a Full/Full/NonRouting/Leaf mesh matching the mixed-profile
Docker topology, asserts each node peers to the degree its role and
topology dictate (A=3, B=2, C=2, D=1), and verifies end-to-end data
between every directly reachable pair (Full-Full, Full-NonRouting,
Full-Leaf). Adds run_tree_test_with_profiles, a per-node-profile sibling
of run_tree_test.
The Leaf-to-Full multi-hop the Docker suite also checks is deliberately
left out for now: a Leaf node's multi-hop session initiation on the XX
line is not yet reliable, so the Docker mixed-profile suite is retained
for that path rather than replaced with a flaky assertion.
The three rekey integration suites (rekey, rekey-accept-off,
rekey-outbound-only) are dropped from both runners. Their coverage now
lives in fast, deterministic in-process tests:
- rekey timing and choreography (trigger, K-bit cutover, drain, jitter,
guards) in the sans-IO poll_rekey tests (proto/fsp, proto/fmp);
- data-plane continuity across a real cutover in
rekey_cutover_preserves_data_plane (a real IK rekey over loopback);
- the accept-off dual-init regression and the udp.outbound_only rekey
loop in the should_admit_msg1 and dual-init characterization tests.
Drop them from both runners in lockstep so the parity guard stays green,
and record the retirement in the deliberately-not-run block with a pointer
to the standalone runner. The rekey-test.sh script stays on disk.
Drives a real IK rekey handshake to K-bit cutover between two loopback
nodes and asserts an encrypted datagram sent after the cutover decodes on
the new session, with no spurious peer teardown. The rekey is forced
deterministically: rekey.after_messages = 1 crosses the initiator's
trigger on the first sent datagram, and both sessions are backdated past
the responder's 30s rekey-acceptance gate, so no wall clock is read.
The cutover is proven to actually occur (the live session index changes),
which is what makes the continuity assertion meaningful rather than
vacuous. Adds a make_test_node_with_config loopback helper for setting the
rekey thresholds. The rekey timing and choreography decisions themselves
are already covered exhaustively by the sans-IO poll_rekey tests.
The inbound max_peers early-gate in handle_msg1 (silent-drop a Msg1 from
a net-new identity at saturation, send no Msg2, admit no peer) is
unit-tested over a real UDP socket by
handle_msg1_silent_drops_at_cap_for_new_peer in src/node/tests/unit.rs:
it saturates a node, sends a Msg1 from a fresh identity, and polls the
sender socket to assert no Msg2 comes back — the same wire-observable
discriminator the Docker suite's tcpdump used — with a sibling test for
the existing-peer bypass. That is a deterministic superset of the Docker
packet-capture assertion.
Drop it from both runners in lockstep (its *_SUITES array, run function,
default-flow loop, --only dispatch arm, and the GitHub matrix leg with
its steps) so the parity guard stays green, and record the retirement in
the deliberately-not-run block with a pointer to the standalone runner.
The admission-cap-test.sh script stays on disk and runs by hand.
The ACL admission decision is exhaustively unit-tested per npub over real
loaded allow/deny files (src/node/acl.rs test module: allow-match-wins,
allowlist-miss falls through, deny-only, deny-all, allow_all override,
deny-after-allowlist-miss), and the inbound and outbound handshake-
admission paths are covered in-process over the loopback transport
(src/node/tests/acl.rs: inbound msg1 denial, outbound denial, reload).
The Docker suite's only unique coverage was admission over a real UDP
transport, which every other real-transport suite already exercises, and
its operator-facing log assertion.
Drop it from both runners in lockstep (its *_SUITES array, run function,
default-flow call, --only dispatch arm, and the GitHub matrix leg with
its steps) so the parity guard stays green, and record the retirement in
the deliberately-not-run block with a pointer to the standalone runner.
The testing/acl-allowlist/ suite stays on disk and runs by hand.
smoke-10 was a no-stressor 10-node tree-convergence sanity check (netem
off, no ping). Its subject, spanning-tree convergence and root election,
is now covered in-process, faster and deterministically, by the loopback
spanning-tree harness (src/node/tests/spanning_tree.rs: ring, star,
chain, 100-node and disconnected-component convergence) plus end-to-end
datagram delivery (src/node/tests/forwarding.rs). Real-UDP convergence
smoke still runs via static-mesh and the other scenarios' baseline
assertions, so no Docker coverage is lost.
Drop it from both runners in lockstep (the CHAOS_SUITES list and the
GitHub chaos matrix) so the parity guard stays green, delete the scenario
YAML, and update the chaos README.
Bring the six retired cost-based parent-selection chaos scenarios up to
next, resolving the ci-local.sh suite-list conflict (next carries the extra
mixed-profile and admission-cap suites). next already has the equivalent
sans-IO cost and transport-drop coverage, so src is unchanged; this takes
only the scenario removals and the CI-list, README and comment updates.
Bring up the retirement of the six cost-based parent-selection chaos
scenarios (cost-reeval, cost-avoidance, cost-stability, depth-vs-cost,
mixed-technology, bottleneck-parent), which tested a decision the Docker
harness could not exercise reliably.
master already carries the equivalent sans-IO coverage in
src/proto/stp/tests (effective-depth cost selection, hysteresis, cost
degradation) and its transport-drop tests, so this keeps master's src
unchanged and takes only the scenario removals and the CI-list, README and
scenario-comment updates from maint.
The cost-selection chaos scenarios (cost-reeval, cost-avoidance,
cost-stability, depth-vs-cost, mixed-technology, bottleneck-parent) tested
TreeState::evaluate_parent's decision logic through a Docker mesh that could
not exercise it reliably: the tree roots at whichever node holds the smallest
NodeAddr, MMP link costs take several measurement windows to settle, and the
parent hold-down plus hysteresis timing all confound the outcome. A
deterministic link-cost flap still produced zero periodic parent switches in a
full run.
Replace those six scenarios with deterministic unit tests in src/tree/tests.rs
that drive evaluate_parent directly: cheaper-link selection at equal depth,
switch-on-cost-change, hysteresis suppressing a marginal change while allowing
a significant one, and the depth-versus-cost effective-depth tradeoff. Each is
constructed so that breaking the cost or hysteresis logic makes it fail.
The congestion kernel-drop signal (SO_RXQ_OVFL) cannot be provoked
deterministically in Docker: a fresh daemon reader keeps up with
container-speed traffic, so the socket receive queue never overflows (an
unshaped run with a 4 KB buffer and heavy traffic recorded zero drops on every
node). Extract the drop-detection edge -- read the cumulative counter, fire an
event only on the transition into a new drop burst -- into
TransportDropState::observe_drops and unit-test it directly. congestion-stress
keeps its ECN and MMP congestion-signal assertions, which do need the real
shaped bottleneck queue.
Remove the retired scenarios from both CI runners and update the chaos README.
gateway-lan pinned 172.20.1.0/24 and fd02::/64, so two concurrent local CI
runs collided on "Pool overlaps" at network creation. The IPv4 subnet has no
consumer -- the whole gateway LAN path is IPv6 -- so drop it and let docker
auto-assign. The IPv6 side cannot float, because the LAN clients' resolv.conf
pins the gateway's address as their nameserver and that must be a literal
known before they start, so claim a free /64 per run and thread the prefix
through the compose address pins, a generated resolv.conf, and the test via a
single exported variable.
The claim and the external network ship as a harness-only compose overlay
applied by run_gateway; the base compose keeps a normal gateway-lan network,
so the GitHub matrix and any standalone bring-up are unaffected. With no claim
every address renders exactly as before.
The boot check piped `systemctl is-system-running --wait` into `grep -qE
'running|degraded'`, but the script runs `set -o pipefail` and is-system-running
exits non-zero for `degraded`. So the pipeline failed on a degraded system even
though grep matched, and the wait looped to its timeout. The older distros reach
`running` (exit 0) and passed; debian:trixie and ubuntu:26.04 reach `degraded`
because a unit that cannot run in a container (systemd-modules-load) fails, so
they timed out at every ceiling -- which is why the earlier timeout increase did
nothing.
Capture the state string and test it directly instead of trusting the pipeline
exit, so a degraded-but-booted system is accepted as intended. Validated: all
five distros pass, including the two that previously failed.
cost-avoidance and mixed-technology decide a parent by measured link cost
(etx * (1 + srtt_ms/100)). Under the local CI's 4-way-parallel chaos the fast
fiber link's measured srtt spikes from host scheduling and can exceed the
Bluetooth link's, flipping the choice and reddening a run that proves nothing
about the daemon. The old margin was ~0.4 cost units, which a ~24ms one-sided
scheduling blip closes.
Widen the Bluetooth delay to 150-250ms so the margin dwarfs any plausible
one-sided spike. The link still establishes well within the handshake budget
(a ~500ms RTT against a 30s stale-handshake window), so the losing candidate
is still a real, established peer rather than an absent one.
Add a netem mutation exclude_edges option and use it in mixed-technology to
pin the two links n08 chooses between, so a random degradation can't flip the
asserted comparison either. An unknown excluded edge is a hard error, since a
silent no-op would reintroduce the flakiness it exists to remove.
Validated by running both scenarios repeatedly under heavier-than-CI
concurrent load: the parent choice holds every time.
Two concurrent ci-local runs could not both run chaos. Each child's subnet came
from its position in the suite list, so both runs walked 10.30.0 through
10.30.12 and requested identical ranges; whichever reached the daemon first won
and the other died with a pool overlap. A run index or hashed offset would only
make that unlikely, and it is precisely the failure being removed.
The simulator now claims its range: attempt-create on a candidate, advance on
docker's own overlap error, fail loudly on anything else. Docker's address pool
becomes the arbiter, so an overlap is impossible rather than improbable. The
claim lives in the sim rather than in ci-local because only the process that
creates the network can advance on conflict, and because it fixes the bare
chaos.sh path too, which a ci-local-only fix would have left broken.
The generated compose now declares the network external over the claimed one,
and teardown releases the range from both paths, including the setup-failed
path where a run that fell over after claiming still holds one.
Ordering matters here and is not obvious: node IPs derive from the subnet
during topology generation, and traffic shaping keys its filters on those
addresses, so the claim happens before the topology exists. Claiming later
would give a network on one range and filters on another, which does not fail
at bring-up and instead leaves the shaping matching nothing.
--subnet survives as an explicit pin for when a known range is wanted, and now
fails loudly if that range is taken rather than silently overlapping. ci-local
no longer passes it.
Validated by running two instances of the same scenario deliberately
concurrently: they claimed 10.30.1.0/24 and 10.30.0.0/24, both exited 0 with
their assertions passing, and both released their networks. The negative
control holds too: with a squatter on a pinned range the run aborts with a
message naming the range rather than proceeding.
With the root pinned, n08's test subject is assertable for the first time: its
two candidates sit at equal depth with fiber-grade default netem on one side and
Bluetooth on the other, which is exactly the preference this scenario exists to
test, and it takes the fiber parent in four of four runs. Encoded as written
rather than calibrated from the runs, since "should pick n03" is the spec and
the runs merely confirm it is met.
n06 is deliberately absent, and its documented criterion is corrected rather
than left standing. The header called n03 a fiber parent for n06; the netem
policies in the same file say that link is WiFi and the other candidate is
Bluetooth, so n06 chooses between two impaired links. Its only fiber-grade link
reaches a node at depth 3 that is never a good parent. There was never a reason
to expect it to prefer n03, and the two-of-four split observed is correct
behaviour: in a run where it took the Bluetooth peer, the fiber-labelled
candidate measured a link cost of 3 against the other's 1, so the daemon chose
the cheaper effective depth as it should.
Validated by replaying the assertion against all four archived runs, then
breaking it three ways to confirm it can fail, then one live run for the wiring.
One of the breaks expects n06 to take n03 and reports that it chose n02, which
is the direct demonstration of why n06 is not encoded.
cost-reeval records that its subject now fires in two runs of three, against
zero of five when it was structurally impossible, and why that rate is still not
one an assertion can rest on.
congestion-stress said the transport drop-detection path "is exercised by
nothing". Widening the scan beyond this one scenario finds it firing in six
others, so the path is live and what is dead is this scenario's ability to
provoke it. Records the arithmetic that explains why: at the 1 Mbps cap the
receive queue needs tens of milliseconds of reader stall to overflow, and the
ingress policer discards the same traffic a layer below the socket.
mixed-technology said the implementation does not do what its criteria say.
It does. The criteria were conditioned on a tree rooted at n01 while the mesh
rooted at n09, under which n08's choice was settled by depth before link
technology could matter. The archived parent distributions are relabelled as
describing the old root so nobody calibrates against them.
cost-reeval now records why it had no assertion about its own subject: its
designated subject was the root, so the parent switch it exists to observe
could not occur. The historical logs show the switch on n04 in 13 of 14 runs
when the root still wandered, and only on n01 in the five runs before the fix.
Each header now names what remains to be done and where it is tracked, rather
than leaving a disproved conclusion in place for the next reader to inherit.
For a value-returning helper the trailing echo is the return mechanism, not a
log call. The hazard is the same either way -- it fixes the exit status at zero
-- but calling it a log call misdescribes four of the sites the gate now finds.
Found while fixing count_log_pattern: the gate reported that function clean
even though it ends in echo and both callers consume its status. Its call-site
patterns only matched direct forms -- if fn, while fn, fn ||, fn && -- so a
status consumed through command substitution was invisible, because the line
begins with the variable rather than the function name.
That is not an exotic form. It is how a shell function returns a value, and it
is precisely the shape of the swallowed-failure family this gate exists to
catch, so the gate was blind to a large part of its own stated class.
Three patterns added for the assignment, if-guarded and test-expression forms.
The extension finds four real instances, all value-returning helpers whose
trailing echo made their exit status unconditionally zero, and each now carries
an explicit return 0. Also corrects the finding message, which described the
trailing command as a log call; for these it is the return mechanism, and the
hazard is that it fixes the status either way.
Validated by breaking what it guards: a probe reintroducing the defect shape
behind a command substitution is reported and exits 1, where before the
extension it would have passed.
Every chaos scenario draws n01 at the top of its topology, and until now that
held in three of thirteen. The mesh roots itself at the numerically smallest
NodeAddr, which is a hash of the node's public key and bears no relation to the
node numbering, so which node ended up as root was effectively arbitrary.
The consequences were not cosmetic. cost-reeval rooted at n04, its own
designated test subject, so that node had no parent and the periodic parent
switch the scenario exists to observe could not occur at all. mixed-technology
rooted at n09, which put its two documented parent criteria out of reach and
made correct cost-based selection look like a defect.
Identities are still derived from the mesh name exactly as before and are still
deterministic. What changes is which node id holds which one: they are now
assigned in NodeAddr order, so n01 holds the smallest and is the root. Verified
against a model of the daemon's own derivation that reproduces the previously
observed root for every scenario and n01's address byte for byte; all twelve
pinned scenarios now root at n01.
smoke-10 deliberately opts out via pin_root: false so that root election from an
arbitrary key distribution stays exercised somewhere. Its assertion is a
convergence floor and is root-agnostic, which is why it is the cheapest home for
that. The new key is rejected when non-boolean, and a near-miss spelling is
rejected as unknown; both checked.
Not yet established: the trees themselves change, so the parent-dependent
assertions in bottleneck-parent, cost-avoidance and cost-stability need
re-deriving against live runs. Those are held until the in-flight CI finishes,
because a chaos run rebuilds the shared fips-test image that run is using.
Four sites where a check could report a verdict it had not established, or
detect a failure and then not turn it red.
assert_no_panic in both NAT suites read `docker logs ... || true`, so a
container that could not be read produced empty output, matched no panic
pattern, and returned success. The assertion's failure mode was
indistinguishable from its success condition. It now reports that absence of
panics is not established.
deb-install's apt capture discarded the exit status, so a failed docker exec
gave an empty capture that matched neither error pattern and reached the pass
branch. The status is now kept and checked before the output is inspected, with
the output still printed on failure.
wait_for_systemd printed a warning and returned success on timeout, so every
check after it read a system that may not have started its units. It now returns
non-zero and the caller abandons that distro leg rather than testing an
unstarted system.
interop's copy of count_log_pattern carried the same defect fixed in rekey: a
node whose logs could not be read contributed zero to eight expect-zero
assertions. Fixed the same way, and its consumer now reports the unreadable case
rather than comparing a sentinel against zero.
Each validated by breaking what it guards and by confirming the healthy path is
unchanged: an absent container now fails each check where it previously passed,
a readable panic-free container still passes, a real panic is still caught, and
a genuinely successful install still passes.
count_log_pattern summed a per-node `docker logs | grep -c ... || true`, so a
node the harness could not read contributed zero. The six assert_zero_count
callers are negative health assertions -- no panics, no ERROR lines, no AEAD
decrypt failures, no rekey msg2 failures -- and a contributed zero reads as
clean, so one unreadable node silently weakened the assertion and all of them
voided it. This is the family the harness-fallback issue was raised to high
priority for, and it is the one remaining audit residual that can make a green
rekey run lie about protocol code.
The read now fails the count rather than degrading it, printing a sentinel that
names the container. Both callers split the declaration from the assignment,
because `local c=$(fn)` takes local's exit status and discards the function's --
which is how the original defect stayed invisible.
Validated by breaking what it guards rather than by observing green: against
absent containers the old reader returns 0 and assert_zero_count passes
vacuously, while the new one returns rc=1 and reports a failure. Checked the
other direction too, since a fix that reds a legitimately clean run is no use: a
genuine zero across readable nodes still returns 0, and the sum is unchanged at
2+1+0 for a shimmed three-node read.
A bash function returns its last command's status, so one ending in a log
call returns 0 whatever it did, and a caller written as `if func` or
`func || fail` has a gate that cannot fire. This tree found two in one
day: run_chaos ended in record, making every chaos row unconditionally
green since 2026-03-09, and build_fips_for_e2e ended in log, letting five
end-to-end legs test the previous commit's binary and report green. Both
were repaired one at a time. This is the rule that catches the next one.
What it enforces is a contract, not a bug hunt: a function whose status a
caller tests must end in an explicit return rather than leaving its
success value to whatever the last log call produced. That distinction is
deliberate and worth stating, because all three instances found in the
tree were benign — each failure path already returned early, so the
trailing echo reported a real success. Reading the function is the only
way to know that, and the next edit that puts an unguarded command before
the final log converts the benign shape into the defect with nothing to
notice. An explicit return costs a line and makes the class unreachable.
The three are given one here.
Scoped on the call sites rather than the definitions, the same way
check-log-strings.py scopes on what a grep reads rather than what its file
mentions: 315 functions scanned, 42 end in a log call, and only the ones
whose status something consumes are reported. A function that ends by
reporting and is called for its output is idiomatic and is left alone.
Without that scoping the finding list would be 42 long and would stop
being read.
Validated by construction rather than by a green run: injecting a copy of
build_fips_for_e2e's exact shape — unguarded docker cp, then a log as the
last statement, with a `|| { ...; return 1; }` caller — makes the checker
report it and exit 1.
Wired into both runners beside the parity and log-string checks.
mmp_focused_pane_indicator checked that the unfocused Link MMP title is
not cyan with assert_ne! over fg_at. fg_at is find(..)? mapped to the
cell's foreground, so it returns None for a title that was never drawn,
and None != Some(Cyan). The assertion therefore passed just as happily
when the pane was missing entirely as when it was present and unstyled.
Assert presence first, so the colour check means "not highlighted"
rather than "not there".
Demonstrated rather than argued. Renaming the pane title in mmp.rs so
"Link MMP" is never rendered leaves the original assertion passing, and
makes the new one fail with the message it was given. Both files restored
after.
This is the only instance of the shape: it is the sole assert_ne! over
fg_at or find in the fipstop tree, and the only assert_ne! in snapshots.rs
at all.
Quartet clean: fmt, build, clippy --all-targets -D warnings, test --lib
at 1376 passed / 0 failed / 7 ignored.
The UDP recv microbenchmark carried a bare #[ignore] while its sibling in
the link module carries a self-describing one. Match it, so the reason
appears wherever the test is listed rather than only in the doc comment.
Record why [profile.ci] retries = 2 is not applied locally. The two gates
disagree on purpose: the hosted runner retries a flaky test twice, the
local sweep fails on the first failure. It exists for shared-runner packet
loss, a property of that environment rather than of the code, and applying
it locally would suppress a real local flake — a failure that only
reproduces under load is a robustness bug to fix, not to retry past.
Keeping the local sweep strict is what makes it the sharper gate. An
undocumented asymmetry is indistinguishable from an oversight, which is
why this is written down rather than left to be rediscovered.
The cost is stated rather than hidden: a test that fails once and passes
on retry is reported green with no separate signal, so a genuine
intermittent failure can be absorbed. If that starts mattering the fix is
to surface retried-but-passed tests, not to drop the retries.
Quartet clean: fmt, build --workspace, clippy --all-targets -D warnings,
and test --lib at 1376 passed / 0 failed / 7 ignored.
Two suites could report a clean pass for work that did not run.
stun-faults skips Phase 2 when tc netem is unavailable, announcing it
several hundred lines above the result and then printing a bare
"stun-faults-test passed". The verdict line now carries the count and
names each skipped phase, and a run in which every phase was skipped
fails outright, since it tested nothing.
deb-install defines a skip() helper and a SKIP counter, prints "N
skipped" in its summary, and calls skip() from nowhere, so the number can
only ever be 0. Its exit tested FAIL alone, meaning a skip could not have
failed the run even once something did set the counter. Gate on SKIP too.
That changes no current outcome, which is the reason to do it now rather
than later: the machinery and the reported number already existed, so the
first skip path added would have printed "N skipped" beside a zero exit
and read as coverage.
Verified by driving the verdict logic at zero, one and three skips: clean
pass, pass with the skip named, and a non-zero exit when nothing ran.
ecn-ab-test.sh was a -test.sh with no path to a failing result: it
asserts nothing, applies no threshold, and nothing invokes it. It also
could not have worked. It read a fixed sim-results/ecn-ab-on/ path while
the runner has written timestamped directories since 2026-03-20, and not
one ecn-ab result directory exists on disk, so it has found neither input
for months and the "+10.2% recv throughput" figure the README carried is
not reproducible from anything available.
Rename it to ecn-ab-compare.sh, fix the path resolution to glob the
timestamped directories, and stop swallowing a failed simulation with
|| true so a broken run cannot feed the comparison.
Deliberately not adding the threshold assertion that would make it
gateable. That needs a calibration corpus, and none exists precisely
because the tool has never produced a kept result; a number invented now
would assert a guess. The prerequisite is a calibration run set and a
decision about what ECN is expected to deliver, which is a protocol
question. Written in the script header rather than left implied.
Add the exclusion block to ci-local.sh naming every suite and scenario
neither runner runs, with the reason for each: interop, boringtun,
iperf-test, this comparison tool, mesh-lab, and the four chaos scenarios
outside CHAOS_SUITES. Until now "not in the suite list" was
indistinguishable from "forgotten".
wait_until_connected decides whether every static suite proceeds or
gives up, so a regression in it turns those suites' verdicts into noise.
Its unit tests existed and were invoked by nothing.
Wire them into ci-local.sh beside the parity and log-string checks,
above the mode branches so --only and --test-only are gated too, and add
the matching step to the ci-parity job in ci.yml. Putting them in one
runner only would create exactly the drift check-ci-parity.sh exists to
catch, and neither placement is visible to that checker since this is not
a matrix suite.
Verified by breaking what it guards rather than by observing green:
setting wait_until_connected's default near-converged slack to 0 fails
case4, records FAIL wait-converge, and exits the sweep 1. Restored after.
They pass, which nothing had established before — 13 assertions, all
green. They also take about 45 seconds rather than the "a few seconds"
the header claimed, and nothing had contradicted that claim because no
runner had ever invoked it. Header corrected with the measurement.
The orphan sweep this called for is done and there are now no
zero-reference *-test.sh files under testing/. It also settles a
disagreement recorded between the audit and the task file: ecn-ab-test.sh
is invoked by nothing, its one reference being a README description
rather than a driver, so the audit was right and the earlier sweep wrong
to call it a documented manual tool. interop-test.sh does have a driver
in interop-stress.sh and iperf-test.sh one in iperf-compare-refs.sh, so
those two were correctly classified.
The baseline floors added for this scenario were calibrated from archived
runs, and those runs are all the 10-node 120-second variant ci-local
invokes as "churn-mixed --nodes 10 --duration 120". The file itself
defaults to 20 nodes and 600 seconds, and --nodes sed-patches num_nodes
in a copy before the scenario loads, so the gating run and a bare
chaos.sh run are different scenarios. Nothing at the site said so.
The floors hold for both, but only because the gating run is the smaller
one. Retuning them against a bare 20-node run, which clears them easily
at 20 answering, 1 root, 19 parented and 69 sessions, would put them past
what the 10-node run reaches and turn CI red while a manual run stayed
green. Say that where someone retuning them will read it.
Confirmed by running the gating invocation directly: 10 answered, 2
roots, 8 parented, 18 sessions, all inside the ranges the floors were
built from. churn-mixed is the only scenario CI overrides this way; the
other twelve run their files as written.
Every section of ping-test.sh is guarded by the profile name, so an
unrecognised profile fell through all of them and the script exited 0
having run no assertion. A typo in the caller's argument produced a green
run that tested nothing. Give it an else branch that names the valid
profiles and exits 2.
No existing leg changes behaviour: the hosted runner gates the ping step
on matrix.type == 'static', which only static-mesh and static-chain
carry, and ci-local passes only the two names in STATIC_SUITES. Worth
noting while confirming that, though: ping-test.sh accepts a mesh-public
profile that neither runner ever passes.
Also record why the convergence waits are `|| true` — they are settling
delays rather than assertions, and the directed-pair pings are what
decides the run — and add admission-cap to the suite list in the
ci-local header, which ran it without listing it.
Five scenarios named by the audit carried no assertions at all, so a run
in which the mesh never formed exited 0 and reported green. Add a
baseline assertion covering how many nodes answered, how many distinct
roots they agreed on, how many took a parent, and optionally how many
sessions were established, and apply it to those five plus the two cost
scenarios left unasserted by the previous commit.
For ethernet-only, ethernet-mesh, tcp-mesh, smoke-10, mixed-technology
and depth-vs-cost the values are not calibrated: one root and N-1
parented nodes is what a spanning tree is, and all provably-completed
archived runs of each show exactly that. churn-mixed is different and
says so at the site. A scenario that stops and starts nodes on purpose
does not hold a single tree, and its six completed runs end with two or
three roots and seven or eight of ten nodes parented, so its floors sit
one step outside the observed range. That leaves them catching a mesh
that collapsed rather than one that churned, which is the most a sample
of six supports.
This gives Ethernet transport its only assertion anywhere in CI. The
three tests in src/node/tests/ethernet.rs are ignored for requiring
CAP_NET_RAW and neither runner passes --ignored, so until now nothing
exercised that transport with a verdict attached.
The floor is deliberately weak and deliberately not a substitute: it says
the mesh formed, not that it formed the tree the scenario describes.
Where those differ the scenario now says so in its own comments.
Four scenarios exist to test cost-based parent selection and each named
its expected outcome in a comment that nothing read. Add a tree_parents
assertion mapping a node to the parent it must have in the final tree
snapshot, and encode it for the two scenarios whose stated outcome the
implementation meets: cost-avoidance (n04 takes the fiber n03) and
bottleneck-parent (n06 takes the fiber n03, n09 keeps its only parent
n05). Both hold in all six provably-completed archived runs.
The other two are left unasserted on purpose, and each for a different
reason worth keeping distinct.
mixed-technology says n06 and n08 should both pick the fiber parent n03.
They do not. Across the five completed archived runs n06 picks n10 three
times and n08 picks n04 four times. Encoding the criterion as written
would red the scenario most runs, and encoding the observed behaviour
would bless something nobody specified, so the disagreement is recorded
at the site as an open question. n06 preferring n10 may well be correct
and the comment stale, since n10 reaches it by fiber too.
depth-vs-cost is different: its validation line does not name an outcome
at all, saying only that the choice "reflects the actual cost tradeoff",
which either answer satisfies. The corpus shows both occurring under one
seed, three runs to two. That scenario needs a protocol decision about
which parent is correct before it can have an assertion.
Compare parents by address resolved from the snapshot's own
my_node_addr, and fail rather than skip when a node is absent from the
snapshot or still claims to be its own root. Those two states are the
common ones in the older corpus and both produce the same "no match" a
wrong parent does, so only separating them keeps a harness problem from
reading as a routing verdict.
The scenario listed four success criteria "verified via post-run
congestion snapshot". Nothing read the snapshot, so the scenario could
not fail on any of them.
Add a congestion_signals assertion taking a floor on the number of nodes
reporting each counter, and encode three of the four. The floors are one
node each because that is what the criteria say; tightening them to the
counts recently observed would assert something nobody wrote down.
The fourth criterion is not encoded, and that is the finding rather than
an omission. No node has reported a non-zero kernel_drop_events in any of
the 182 archived runs, so asserting it would red the scenario
permanently. It is recorded at the site as an unmet criterion and a
coverage gap: the transport drop-detection path the scenario names as its
second signal is exercised by nothing.
Two things about the corpus are worth carrying, both recorded in the
scenario. Only six archived runs carry a status.txt and are therefore
provably completed; all six meet the three encoded criteria with five to
nine nodes reporting each signal. The 176 older runs meet none of them,
and they are not invalid samples: each reached teardown far enough to
write an analysis.txt, and ECN landed before all but one of them. What
changed on 2026-07-22 is not established, since neither the scenario nor
netem.py, traffic.py or control.py has been touched.
Count the nodes reporting a signal rather than the magnitude of any one
counter, since a single node with a large count would satisfy a magnitude
test while proving the signal never propagated. A missing snapshot fails
rather than reading as an absence of congestion.
The simulation exit ladder reported panics and failed assertions but said
nothing about ERROR-level log lines, so a run in which every node errored
on every line still exited 0.
Add a max_errors assertion and apply it to every scenario by default
rather than having each one opt in. This is a floor on what a green run
means rather than a property of an individual scenario, and a scenario
that has to ask for the floor is one that can forget to.
The default ceiling of 0 is what the archived corpus supports: across
2416 result directories no node log contains an ERROR-level line, while
the WARN counter extracted by the same code path ranges from 0 to 1135,
so the counter is known to discriminate rather than merely known to read
zero. A scenario that legitimately induces errors raises the ceiling in
its own YAML and says there why.
Reject rust_log: off alongside it, since that is the one level that would
leave the ceiling counting zero whatever the mesh did.
The scenario's success criterion existed only as a comment: count
"Parent switched" in n04's log, expect at most 5. Nothing read it, and
the scenario declared no assertions at all, so its exit code could only
report that the mesh came up and nothing crashed.
Add a max_parent_switches assertion and wire that criterion to it.
The assertion takes an optional node scope, which is the part that
matters. The existing sibling counts mesh-wide, and a criterion written
about one node's log is a different quantity from the sum over every
node: across archived runs of this scenario n04 is 1 or 2 while the
mesh-wide total ranges 3 to 6. Asserting the sum against a per-node
threshold would check something other than what was specified, and at
this threshold would also have failed several runs that were fine.
Counting nothing must not read as stability. A node id absent from the
topology fails explicitly rather than matching zero log lines and
sailing under the ceiling, and a scenario that declares a parent-switch
assertion while setting a log level that suppresses the events it counts
is rejected at load rather than passing vacuously after a full run. The
parse rejects a mistyped key, a missing, negative, boolean or
non-integer ceiling, and a node written empty or as a non-string -- that
last one because YAML renders a bare `node:` as null, which would
silently revert to the mesh-wide count.
The scenario comment now records what the threshold is worth against
the 11 completed archived runs rather than a single sample: 5 is well
above anything observed, the daemon's own parent hold-down caps
switches near 6 per run, and nearly every switch lands during initial
tree formation before any mutation fires. So this catches only
near-pathological reparenting. Tightening to 3 would make it a real
detector, but that changes the stated criterion rather than fixing it,
so it is left as a decision.
Verified against a live run: the assertion evaluates n04 at 2 against
mesh-wide 4 and passes, and with the ceiling temporarily set to 0 the
scenario exits 3, the assertion-failure code, which it previously could
not reach.
The mapping-reclaimed check expects zero mappings, and read the count
with `r.get('data',{}).get('mappings',[])`. An error response carries
no data field, so it parsed to zero and satisfied the check without the
gateway having answered at all. The expected value and the failure
value were the same number, which is the shape that lets an assertion
pass without asking anything.
Require the key to exist and be a list, and exit non-zero otherwise, so
the existing fallback turns an unanswered or malformed response into a
failure. The gateway always emits the key on success, including for an
empty set, so a genuine zero still reads as zero; confirmed against the
running gateway, where the reclamation check passes.
The earlier show_mappings poll shares the parse and is left alone: it
waits for a positive two, which no error response can produce. The
hazard was already documented there, and this is the call site that was
exposed to it.
Case (a) sends an unallowed inbound request that the ruleset should
drop, and accepted any non-zero curl status as proof. A drop produces
no RST, so curl can only hit its deadline and exit 28; connection
refused, an unroutable address or a missing listener all fail too, with
different codes, and the check counted those as a blocked connection.
Require 28 exactly, so the assertion distinguishes silently dropped
from failed for some other reason. Confirmed against the harness: the
real run returns 28.
The baseline check matched `counter packets`, which any counter rule
satisfies whatever its verdict, including one that accepts. Require the
rendered drop rule instead.
The drop-counter read had the same weakness plus a worse one: it took
field three of the first line mentioning a counter, which is the packet
count only when the line begins with `counter`. On a rule such as
`tcp dport 9 counter packets 42 bytes 3000 drop` it printed the port
number. That shape is reachable, because the shipped ruleset includes
the operator drop-in directory ahead of the trailing default deny and a
drop-in may add its own counted drop. Extract by position within the
matched text, and take the last match rather than the first, since case
(a) falls through to the default deny that the ruleset emits as the
final rule.
Note in place at the baseline check that it still only proves a counted
drop rule exists somewhere in the table, not that it is the trailing
one; asserting rule position is a larger change than this warrants.
The sample output in the README is updated to the new wording.
The four rejection checks were independent greps over the whole log:
two npubs, a context and a decision. Together they proved only that
node-a mentioned each npub somewhere, rejected somebody on an inbound
handshake, and rejected somebody by denylist. Nothing tied a rejection
to a named peer, and node-a lists both denied peers as auto_connect
peers, so it logs their npubs on the outbound connect path whether or
not any rejection ever happened. The actual message was never matched.
Replace them with a helper that requires every given string on ONE
line, and assert per denied peer: the real message, that peer's npub,
and the denylist decision together. Strings match in any order by
chaining fixed-string greps over the surviving lines, so the assertion
does not depend on how the log formatter orders a message and its
fields. A grep over empty input yields the empty string rather than
anything a caller could mistake for a match, so an unreadable container
times out and fails instead of passing.
Two limits are written at the call sites rather than left implied. The
decision conjunct discriminates nothing, since the two allowing
variants return early and denylist is the only value that can reach
that warning. And node-a authorizes before dialing, so it emits a
fully formed rejection line for each denied peer on the outbound path;
these assertions are satisfiable without the inbound check running at
all, and it is the peer-count assertions that would catch that.
Verified against the running harness: all three find a real matching
line. The defect shape they replace was exercised offline first, with
the three strings spread across three lines, where the old checks
passed and the new ones fail.
The traversal clock cached a Unix millisecond value and an Instant at
first use, then served every later call by advancing the cached value
with the monotonic elapsed time. A monotonic clock does not tick while
the host is suspended, so once a machine had slept the returned value
trailed real time by the sleep duration for the rest of the process
lifetime, and never re-synced.
Almost everything that clock feeds is an absolute timestamp. The NIP-40
expiration tags on adverts and traversal signals, and the issuedAt and
expiresAt fields of offers and answers, are all computed as now plus a
TTL; the freshness and cache-pruning paths compare it against a
peer-authored, signed created_at. Once the host had slept longer than
signal_ttl_secs, every offer was published already expired, relays
dropped it, and the initiator timed out waiting for an answer with
traversal broken until a restart.
Read the wall clock on every call instead. This also removes a
mismatch inside the traversal failure-state map, which was written
here from the cached clock but written and read from the node
lifecycle with the real one.
Not platform-specific: monotonic clocks exclude suspended time on
Linux and Windows as well, so this affected any host that suspends.
A laptop is simply where a process lives long enough across a sleep
to notice.
The interval-shaped consumers hold up under a clock step. A forward
step, which is what a resume produces, saturates the punch start delay
to zero, and the attempt's own bounds are monotonic deadlines that are
unaffected. A backward step lengthens that delay and can cost one punch
attempt, which retries. Early eviction from the replay window cannot
admit a replay under the shipped defaults, because the freshness window
a replayed offer must also satisfy is strictly narrower than the replay
window itself.
The added test pins the contract and fires on a host that has genuinely
suspended, but it is not a regression guard for this defect: nothing
reachable from a unit test can simulate a suspend, so on a machine that
has not slept the old implementation passes it too. Its comment says
so rather than leaving a false sense of coverage.
Reported in https://github.com/jmcorgan/fips/issues/128
Access layer for phones and laptops to reach FIPS routers on OpenWrt,
stacked on the 802.11s mesh backhaul from #123. Squashed from five
commits by Arjen (Origami74); their original messages follow.
* feat(openwrt): open !FIPS access SSID — fips-ap-setup helper, default transport binding, how-to
Client access layer for phones and laptops: every FIPS router
broadcasts the same open SSID ('!FIPS' — the leading '!' sorts it to
the top of alphabetically ordered network pickers), forming one
standard ESS. Clients save it once and roam between all FIPS routers
natively, with FIPS's Noise IK handshake as the only security layer:
- fips-ap-setup: opt-in UCI helper that creates the 'fips-ap0' open AP
(encryption none — security type must be uniform across routers or
clients treat the ESS as different saved networks), an isolated
network with a static ULA /64, RA-only odhcpd addressing (stateless
SLAAC, no DHCP — the minimum that satisfies Android's provisioning
check; no internet by design, so phones keep cellular as default
route), and a locked-down fips_ap firewall zone (no path to br-lan
or the WAN; only ICMPv6, mDNS, and the FIPS transports reachable).
'remove' subcommand undoes it. Radio setup stays opt-in; a package
must not commandeer radios on install.
- fips.yaml: ship 'ap0'/'ap1' Ethernet-transport entries commented out
(matching the 802.11s mesh backhaul) so a stock install that never
creates fips-ap* logs no per-boot "interface missing" bind warning;
fips-ap-setup uncomments the matching block when it creates the
interface and re-comments it on remove.
- Regression test: extend shipped_openwrt_config_parses to assert the
ap0/ap1 entries ship commented out and still parse once uncommented,
alongside mesh0/mesh1.
- Packaging: install the helper in ipk/apk/buildroot (three synced
copies), extend CI structural checks and shellcheck targets.
- docs/how-to/set-up-open-access-ssid.md: full guide, including the
one-time 'no internet, stay connected' acceptance (stored per SSID,
covers every FIPS router) and the security-type-uniformity
constraint.
* docs(openwrt): correct open-SSID/mesh security framing — peering is open by design
The fips-ap-setup/fips-mesh-setup comments and both how-tos claimed a
stranger "cannot pass the FIPS handshake" / "their frames die at the
handshake" / the handshake surface "drops them". That is wrong: FIPS
peer admission is open. An inbound handshake from any net-new identity
is promoted (node::handlers::handshake::promote_connection), gated only
by the daemon's max-peers cap — there is no allowlist, no PSK, and the
AuthChallenge path is not wired to admission. The Noise IK handshake
provides authentication (no impersonation of another identity, no MITM),
not authorization.
Restate the model accurately in all five places: a stranger on the open
SSID (or the open mesh) can associate AND form a FIPS peer link — that
is the point of open access. Containment is the isolated fips_ap zone
(no path to br-lan or the WAN) plus the max-peers cap, not the
handshake. Clarify that AP client isolation is an L2 control only: a
peered stranger is an overlay peer like any other, so the FIPS overlay,
not L2, is the trust boundary between clients.
* feat(openwrt): serve DHCPv4 on the access SSID from a fixed roamable subnet
RA-only addressing satisfied Android's provisioning check but left
anything expecting IPv4 with a self-assigned address and a "no IP"
complaint. dnsmasq now leases out of 10.21.<N>.0/24 (N = radio index;
prefix echoes FIPS port 2121), deliberately identical on every router:
a roaming client keeps its lease across the ESS, and dnsmasq's
authoritative mode — the OpenWrt default, pinned by the helper — ACKs
the renew a foreign router never issued. Lease collisions across
routers surface as a NAK on renew and the client re-DHCPs.
The dhcp section's 'dhcpv4 server' is read by both dnsmasq (default
images) and odhcpd (only with maindhcp), so either arrangement serves.
A DHCPv4/udp-67 accept rule joins the fips_ap zone; DHCPv6 stays off,
the ULA RA stays as-is, and nothing depends on an upstream. The zone
remains isolated — no forwardings, no internet.
* feat(openwrt): enable mDNS rendezvous from fips-ap-setup
Phone FIPS apps cannot open raw-Ethernet sockets, so DNS-SD is how they
find the router's daemon — but node.rendezvous.lan defaults to off. Ship
the lan block commented in fips.yaml (consistent with the apN transport
entries) and have fips-ap-setup uncomment it when creating the access
SSID. The awk match is scoped to node.rendezvous because
transports.ethernet carries a 'lan' entry at the same indent. The switch
is daemon-wide, so 'remove' deliberately leaves it on rather than guess
whether other transports rely on it.
* fix(openwrt): bind UDP dual-stack [::]:2121 so access-SSID clients reach it
The shipped router config bound the UDP transport "0.0.0.0:2121" (IPv4
wildcard) while the mDNS LAN advert announces every interface address,
including the router's IPv6 link-local — which phones on the !FIPS
access SSID rightly prefer (their cellular default route swallows v4,
and fd00::/8 is captured by the Myco mesh TUN). Result: the client's
Noise msg1 arrives on an unbound v6 port and is silently lost; the
handshake resends and times out.
Symptom chain (observed on-device): mDNS resolve OK, platform push OK,
"Sent Noise handshake message 1" to [fe80::…%N]:2121, four resends, no
reply, 30 s stale-timeout.
OpenWrt is Linux (bindv6only=0), so "[::]" accepts IPv4 via v4-mapped
addresses too — nothing is lost. packaging/common is deliberately left
on "0.0.0.0" for now: Windows defaults IPV6_V6ONLY=1, where "[::]"
would drop v4 instead.
wait_for_peers_exact fell back to 0 when it could not read a container's peer
count. Two of its six call sites are the ACL denial checks, which expect
exactly 0 connected peers — so a failed docker exec satisfied them on the
first iteration and the isolation property the suite exists to prove was
never observed. Confirmed against a container that does not exist: the old
reader returns "0" and the expect-zero comparison passes.
The reader now returns the empty string when the daemon does not answer,
which no numeric comparison can satisfy, and the timeout message says which
of the two happened — never answered, or answered the wrong number. Same
shape as admission-cap-test.sh's read_peer_count.
A test that greps the daemon's log for a message the daemon stopped emitting
does not fail. It stops observing, and an assertion built on it — especially
one expecting a count of zero — then passes because nothing can be seen
rather than because nothing happened. Several findings have come from that
one class, so it is now checked mechanically.
testing/check-log-strings.py extracts the strings test code matches against
daemon log text and requires each to exist in src/. It reads four shapes:
python `"..." in line` tests, the first argument of the shell log helpers,
bash associative-array pattern tables, and greps whose input is daemon log
output. Patterns carry regex syntax, so each is reduced to the longest
literal run of every alternation branch, with escaping resolved in the same
pass — `\.` is a literal dot and `.` is a wildcard, and conflating them would
let a pattern match text that is not there.
Scoping is by what a grep READS, not by what its file mentions. A suite greps
its own analyzer output, fipsctl JSON, Tor's log and ping output, and none of
those have to correspond to a string in src/; scoping on the file produced 34
false positives. Strings that legitimately do not come from src/ are named in
ALLOWED with a reason, so the exceptions are reviewable rather than invisible.
It found six dead strings, three of which were not previously known: a second
copy of "Excessive decrypt failures" in the interop pattern table, a dead
"Handshake error" branch inside an alternation whose other branches still
matched, and "bootstrap failed" in the NAT suite. All six are corrected to
what the daemon emits, or dropped where the surviving branch already covers
the case. Verified by breaking it two ways and confirming it reports.
Also here, since it is the same failure shape one layer down: the shared log
library returned an empty string for a container whose logs it could not
read, so analysing a missing container reported no panics, no errors and no
sessions, and exited 0 — indistinguishable from a clean run. It now reports
which containers could not be read and raises, matching the fix already
landed in the chaos copy.
Giving the mixed-profile services a run-scoped container_name broke this
script's two ping sites, which assemble the name as "fips-$from" rather than
writing it out. A grep for the literal name finds neither, which is the same
trap the earlier naming sweep hit on the other static scripts.
The failure was loud — link convergence passed 3/2/2/1 and then every ping
failed, because docker exec could not find the container — but it is worth
noting that these names were unsuffixed until now, so this suite was never
safe to run twice on one host.
The static compose's mixed-profile services are next-only, so the floating
subnet arrived without them: four services kept ipv4_address pins on a
network that no longer declares a subnet, which docker rejects outright at
`up`. They now float with the rest, take the per-run config directory, and
carry the run name suffix their container_name was missing.
Phase 3 of the admission-cap test conflicted for a real reason rather than a
textual one. This line asserts size-agnostically — sustained inbound from
each denied peer plus "was never promoted" — because the XX handshake
message sizes differ from IK and may change again. That assertion is kept;
what comes across from maint is the address handling underneath it, which
matters here for the same reason: with the subnet floating, a restarted
container can change address, and matching on a single one silently
under-counts the very evidence the assertion rests on.
Two local CI runs on one host both asked docker for 172.20.0.0/24 and the
second lost its whole static family to "Pool overlaps". Docker honours a
fixed subnet request verbatim, so the only robust fix is to stop making one:
fips-net now requests no subnet and docker assigns from its own pool, which
cannot hand the same range to two runs.
That means node addresses are not known before `up`, so peers address each
other by container hostname instead. The generator emits node-<id>, or the
topology's docker_host where the compose hostname differs — only the gateway
profile, whose services are gw-*. External peers keep the address the
topology gives them, since it is not ours to assign. The resolv.conf mount
stays: dnsmasq is what forwards these names to docker's resolver and .fips
to the daemon, so removing it would take out every .fips assertion.
generated-configs is now per-run as well. A shared directory let two runs
overwrite each other's node configs, which the subnet collision had been
hiding by killing runs before that window opened. The generator, the compose
bind mounts and env_file, the six scripts that read it, and teardown all
follow FIPS_CI_NAME_SUFFIX; unset, every path renders as before. Teardown
keeps the directory after a failed run, where it is the evidence of what the
failing nodes were configured with.
Three things this exposed that were wrong independently:
admission-cap built its tcpdump patterns from the topology file's docker_ip
literals. Floating the subnet makes those match nothing, which would have
left its expect-zero "no Msg2 leaked" assertion passing because it could no
longer see anything at all. It now reads addresses from the running
containers. Restarting the denied peers together also made them swap
addresses, so each peer's counts were really the pair's total; they are
restarted one at a time now, and a check fails the suite outright if two
denied peers ever share an address, because per-peer attribution is
impossible once they do.
Attribute lookups in the generator used a fixed ten-line window and read the
next node's fields when a node omitted an attribute. An external node
followed by an internal one was classified as internal, which under hostname
peering would emit a name that resolves nowhere. Lookups are bounded to the
node's own block; generated output is byte-identical for all eight
topologies.
The rekey outbound-only variant used to rewrite peer addresses to hostnames
to set up its scenario. The generator now does that everywhere, so the
rewrite matched nothing and was silently doing no work. It asserts the
premise instead, and fails if a numeric address ever reappears.
Verified by running three instances of this compose at once — tcp-chain plus
two independent meshes — which drew 10.128.2/3/4.0/24 with no overlap while
both meshes passed ping-test 20/20 over the real .fips path. tcp-chain is
run by neither CI runner, so it was checked by hand: chain peer counts 1/2/1
and multi-hop .fips reachable both directions over TCP.
gateway-lan still pins its own IPv4 and fd02:: ranges and is unchanged here,
so the gateway profile is not yet concurrency-safe.
The parity guard now checks that every suite the local runner can dispatch has a backing suite array, so a suite dispatched without one is visible to nobody. On next, mixed-profile was dispatched by a bare call with no array, so the guard reported it as present on the GitHub matrix and missing from the local runner even though the local runner does run it.
This adds MIXED_PROFILE_SUITES, dispatches through it in the same idiom the admission-cap suite uses so the array is the single source of truth for whether the suite runs, and lists it in the suite listing. It exists only on next, so it is authored here directly rather than merged up.
Four of these are places the parity guard could stop covering something without saying so, which is the failure mode it exists to prevent. Removing the deb-install suite array left it green, because the dispatch cross-check compared against a hardcoded name rather than the array it was meant to read. An arm-shaped line at an unexpected indent, or one written in quotes, was dropped silently by the arm pattern; unexpected indents are now a hard error rather than a quiet skip, and quoted arms parse. A matrix leg with a type of chaos or deb-install but no scenario key raised a Python traceback instead of reporting a problem, and a leg carrying a scenario but no suite key was skipped entirely even though scenario is now the identity for those legs.
The e2e builder no longer folds docker create's stderr into the container id. Docker prints warnings on success as well as failure, so a platform-mismatch warning would have become part of the id and made every later reference to that container fail, turning a working extraction into a spurious red.
Also corrects the comment about registering a new chaos assertion type, which named only one of the two key sets that actually need it.
The guard that keeps "local green" and "GitHub green" meaning the same thing was invoked by nothing. Its only entry point was an exec behind an explicit --check-parity flag, which by exec-ing could not be combined with an actual run, and no workflow step called it at all. A guard nobody runs is a guard that does not exist, which is how the mixed-profile divergence on next survived unnoticed.
It now runs as the first stage of every local run and as its own job on GitHub. Locally it reports through the same result-recording path as every other stage, so a divergence sets the run's exit status instead of scrolling past. It is deliberately placed above the mode branches, so a divergence also fails --only, --test-only and --build-only runs: whichever subset was asked for, the claim that a local result means what a GitHub result means is what has broken. The GitHub side installs pyyaml explicitly rather than assuming the runner image carries it.
--check-parity keeps working exactly as before. Three comment blocks that described the old folded comparison are corrected here, in the workflow, the local runner and the testing README; they were spread across three files and only two mention the guard by name, so the third is best found by searching for what it claims rather than for the script.
The guard collapsed all thirteen chaos legs to the token "chaos" and all five deb-install legs to "deb-install", so sixteen of the thirty-four integration legs were invisible to it. Deleting twelve chaos legs and four deb-install legs from a copy of the workflow left it still reporting "CI parity OK". The fold existed because the guard compared the cosmetic suite name, which really does differ between the runners; both sides have carried a directly comparable scenario field all along, so the fix is to compare that instead, along with the chaos flags.
The local suite set is now discovered by sweeping ci-local.sh for suite arrays rather than reading a hardcoded list of variable names, and the deb-install distro list is read from the suite's own script. A suite dispatched with no backing array is invisible to either approach, so every run_suite arm is now checked to have one and the guard names any that does not. That is not hypothetical: next dispatches its mixed-profile suite without an array, and the guard could only report it as missing from the local runner without being able to say why.
The guard also now refuses to run rather than failing obscurely when python3 or pyyaml is absent, since a parity check that cannot run must not look like one that passed. Verified by breaking each thing it guards: legs deleted from the workflow, drifted chaos flags, a fabricated suite array, and a dispatch arm whose array was removed. That last case initially did not fire because the arm pattern was pinned to the wrong indentation, which the test caught.
The scenario loader read every section and member through raw.get with a default, so a mistyped key was silently ignored and the block it belonged to stayed default-constructed while the YAML still looked correct on the page. Writing "assertion:" for "assertions:" disarmed a scenario's only assertions and left it unable to return a non-zero exit code; "link_flap:" for "link_flaps:" turned off the churn the scenario existed to inject, and a calm mesh then ran under the name of a chaos test.
Keys are now checked at load time, at the top level and inside every section with a fixed schema, including the netem policy bodies, the {min, max} ranges and the assertion bodies themselves. The error names the offending key, the section it appeared in, and the keys that section does understand. Three mappings are deliberately exempt because their keys are names the author chooses rather than a schema, and two sub-trees are passed through whole; all five are listed at the key sets with the reason.
Adding an assertion type now means registering it, so the next person to add one gets a rejection rather than a silent no-op. That coupling is the point and it is written down where they will see it. Verified against the seventeen scenarios in the tree, all of which still load, and against fixtures for each failure mode: a top-level typo, a nested typo, a typo inside an assertion body, and a section left empty, which previously raised an unrelated AttributeError from the next line of the loader.
The e2e scenarios build fips and fips-gateway in a builder image and extract them with two docker cp calls whose stderr went to /dev/null and whose exit status was never tested. Because build_fips_for_e2e ended in a log call it returned 0 on every path, so the caller's guard could not fire. A failed extraction left the previous run's binaries in the cache, they satisfied the caller's executable check, and the five e2e legs then exercised the previous commit's code and reported green.
The extraction now deletes the cached binaries before it starts, which is what makes the stale case impossible rather than merely detected, and it checks each docker cp independently, reports the failure with docker's own message, verifies each extracted file is non-empty and executable, and returns an explicit status. This mirrors what the deb-install suite already does.
Confirmed by inducing the failure rather than by observing a green run. With both extractions broken and a stale binary in place, the old code logged "Cached fips (26936 bytes)" and went on to build the runtime image around it; the new code reports both docker errors, deletes the stale binaries and fails the suite. Breaking either extraction on its own is caught separately, and an unmodified run passes 7 of 7.
The chaos simulation was equally unable to report a failure on this line: scenarios collided on globally-scoped container names and a shared config directory, the runner swallowed whatever they raised, and the harness discarded their exit codes before anything read them. The merged commits scope names and directories per scenario, give an aborted run its own exit code and let it reach the summary, stop a failed scenario harvesting whichever containers hold its names, and keep the daemon's reason for a failure instead of discarding it.
Testing only. ci-local.sh merged automatically; run_chaos and ci_teardown come across byte for byte and this line's own mixed-profile handling is untouched, the two having never overlapped.
The chaos simulation could not report a failure on this line either: scenarios collided on globally-scoped container names and a shared config directory, the runner swallowed whatever they raised, and the harness discarded their exit codes before anything read them. The five commits scope the names and directories per scenario, give an aborted run its own exit code and let that code reach the summary, stop a failed scenario harvesting whichever containers happen to hold its names, and keep the daemon's reason for a failure instead of throwing it away.
Testing only, and the merge is textual: every chaos file matches maint byte for byte, and the ci-local divergence on this line does not touch the changed regions.
Compose commands run with their output captured and with check set, so a failure raised a CalledProcessError and nothing ever looked at what the daemon had said. That exception reports the argv and the exit status and nothing else, so every scenario that died bringing its containers up left a runner log saying only that docker compose returned non-zero exit status 1, and the reason was captured and then discarded. In the run that prompted this work twelve scenarios failed that way and why they failed can no longer be established at all. Log the captured stderr and stdout before the error goes anywhere, keeping the tail of each so a verbose failure cannot fill the log file. A command that times out raises before its exit status is ever read, so that path records whatever it had emitted too, rather than leaving the same hole one shape over. Reproduced against both of the shapes this is meant to catch: an unusable network prefix now records the daemon's parse error, and a name already held by another container now records the conflict and names the container holding it. The success path logs nothing new, and a completed scenario's runner log is unchanged line for line.
The raising behaviour is deliberately untouched, since the abort flag and the exit code that came with it depend on it. The check is applied here rather than by subprocess so that a caller who passed check false gets the same record: both such callers are the compose down in teardown, which today can leave a mesh behind without saying so.
The same omission was costing the veth path its reasons. Failing ip commands had their stderr logged at debug, which the runner does not emit unless asked for verbose output, so a failed pair creation reached the log as nothing more than the names of the two interfaces it could not create. Raise that to a warning. The deletes issued to clear a stale pair are expected to fail and still say nothing.
Teardown runs from a finally, so it also runs after a failed setup, and it was guarded only on the topology and the compose file, both of which are set well before any container exists. A scenario that died bringing its containers up therefore ran the whole harvest anyway: final snapshots, docker logs, the analysis, the assertions and the metadata, all addressing containers by names that are global to the host. What it left behind was not an empty directory an investigator would notice but a full and plausible one, in the recorded case ten node logs and an analysis reporting two promotions and two parent switches, every byte of it from a different scenario's mesh. Gate the harvest on whether the containers ever started, and leave the compose down outside that gate so a partly successful start still gets cleaned up.
Record the outcome in a status file in every result directory, naming the run as completed, interrupted, aborted, setup-failed or teardown-failed, alongside the scenario, the seed and the container names it used. With the harvest gated, the presence of an analysis file is now itself proof that the scenario's own mesh existed, and the status file says which of the several ways a run can end applies. A run cut short by a signal keeps the existing exit codes rather than gaining one of its own: what it collected before stopping is real and still worth reporting, the status file records that it was truncated, and every wrapper already reports a Ctrl-C of its own.
Log collection never looked at the status of docker logs. It concatenated stdout and stderr unconditionally, so collecting from containers that were not there wrote the daemon's "No such container" reply into each node log and analysed the result as a mesh with no panics, no errors and no sessions, which reads exactly like a clean run and exits zero. Check the return code, and treat a harvest that cannot read every container, or that reads none, as a failed teardown. A teardown that raises now also reports on the same footing as a run that never started, instead of escaping past the exit codes as a bare traceback and being read as a malformed command line.
The simulation caught every exception a scenario raised, logged it, and exited 0, so a scenario that died during setup was recorded as a pass. Local CI discarded the exit code in any case: run_chaos ends in a call whose own last statement is an echo, so it returned 0 whatever the scenario did, and the parallel launcher's wait therefore always succeeded and always recorded a pass. Between them no chaos failure of any kind could turn a local run red, and none could since the script was written.
Carry the abort out of the run and give it its own exit code, ordered ahead of the two content codes so a run that never produced a mesh cannot report that mesh's panic and assertion counts. Return the scenario's status from run_chaos so the launcher sees it. The documented exit codes were stale before this change and are rewritten in full, including that the argument parser rejects a malformed command line with the same code that reports panics.
Expect scenarios that have been reporting green to start reporting red. In the last recorded local run twelve of the thirteen scenarios never got past starting their containers, and all thirteen were counted as passes; those that now execute have not run for months, and their assertions have never been evaluated against a mesh of their own.
Ethernet edges get a veth pair created in the host namespace and then moved into the containers, named from the node ids alone. ethernet-mesh and ethernet-only both connect n01 to n04 and always run together, so one tore down the other's live link. Hash the scenario suffix to four hex characters and insert those after the vh prefix; 15 characters is far too few to carry the suffix itself.
The names are now run-specific and never regenerated, so the delete-before-create no longer reclaims an interface an earlier run abandoned. ci-cleanup.sh takes over, deriving the names from the suffixes a run's teardown hands it. It is the first thing that script removes that is neither a docker object nor labelled, so an unscoped reap can sever a running bare simulation's links; testing/README.md and the --reap help say so.
Chaos scenarios run four at a time under local CI, but every one of them claimed the container names fips-node-nNN and wrote its generated configs and compose file to the same generated-configs/sim directory. Container names are global in Docker and are not scoped by the compose project, so concurrent scenarios collided on both, and a scenario could start containers from a compose file another had overwritten.
Thread the existing FIPS_CI_NAME_SUFFIX into the simulation. run_chaos narrows the run-wide suffix to the scenario, and the sim reads it once when the topology is built, applying it to the container names and to the config directory basename. The compose template renders the name through the topology accessor instead of duplicating the literal, so one expression produces every chaos container name.
The suffix is empty when the variable is unset, so a bare chaos.sh run and the hosted CI jobs render byte-identical names and paths. Verified by rendering every scenario's compose file before and after with the variable unset and diffing.
Both branches carry the same correction, authored separately because the
sans-IO relocation moved the logic into a core that returns an outcome rather
than acting inline. Git resolved the file move as a rename and offered to merge
the two versions textually, which would have been wrong in both files.
Resolved by keeping master's re-expression wholesale for the handler and its
tests, and taking the changelog entry from maint. Verified rather than assumed:
the resulting tree is byte identical to the reviewed and CI-green master commit
across every code path, and the merge contributes nothing but the fifteen line
changelog entry. Since the code is unchanged from a tree that already passed the
full local suite, that result still stands and no re-run is owed.
Nothing was lost from maint's side. Its tests are written against a file layout
that does not exist here, and master carries equivalent coverage written against
its own shape, including the multi-node chain and the cache warming cases that
the maint version does not have.
The same correction as on the maintenance line, re-expressed rather than merged,
because the sans-IO relocation moved this logic into a synchronous core that
returns an outcome instead of acting inline.
Delivery to the addressed node is no longer gated on the hop limit, and the
decrement happens before the drop decision, so a datagram that would leave with
zero is not sent. The reachable radius does not change: the old behavior refused
to deliver at zero but was willing to send at zero, and the new behavior is the
mirror of that, so the two cancel.
The shell above the core needed changing too, and that was the real work. It
carried two hop-limit predicates of its own, both justified by comments citing the
ordering that existed before the refactor. The first gated coordinate cache
warming, which this fix requires to be unconditional; left alone it would have
suppressed warming twice over, once for a datagram addressed here and now
delivered, and once for a transit datagram dropped for hop limit whose plaintext
coordinates are still perfectly usable. The second gated next hop resolution,
which carries a cache touch side effect, and now mirrors the core's
would-leave-zero rule instead of the old one. Neither divergence would have
surfaced as a test failure.
That last point is worth stating plainly, because it nearly went the other way.
Restoring the warming gate on top of the corrected core passes the entire suite
without a single failure. The tests could not see the seam between the core rule
and the shell's copy of it. So this adds coverage that can: a three node chain
pinning the decrement across a real hop, which is the only thing exercising the
shell predicates composed with the core, and two warming tests at a zero hop limit
covering both the delivered and the dropped case. Each was checked by putting the
specific defect it targets back and confirming the test fails.
One existing test that claimed to cover hop limit behavior asserted nothing
whatsoever. It asserts now, but those assertions do not discriminate this change,
since the case it exercises behaves identically under both rules. They guard
against the test going vacuous again, not against this defect, and the comment
says so.
The forwarding path tested the hop limit before testing whether the datagram was
addressed to this node, and decremented only on the forwarding branch. Two
consequences followed. A datagram addressed here that arrived with a zero hop
limit was dropped rather than delivered. And a forwarder receiving a transit
datagram with one hop left transmitted it with zero for the next hop to discard,
wasting a transmission on every expiring datagram.
Delivery to the addressed node is no longer gated on the hop limit, and the
decrement now happens before the drop decision, so a datagram that would leave
with zero is not sent. Saturating subtraction folds the two arrivals that cannot
be transmitted, already exhausted and last hop, into a single test.
The reachable radius does not change, which is worth stating because it is easy
to assume otherwise. The old behavior refused to deliver at zero but was willing
to send at zero, and the new behavior is the mirror of that, so the two cancel: a
path of h links still delivers for any source value of h or more. What this
actually buys is the wasted transmission, the exhaustion counter charging at the
node that makes the decision rather than the one after it, conformance with the
specified semantics, and one real gain during a rolling upgrade, where an
unupgraded forwarder feeding an upgraded destination delivers one hop further
than either version does on its own.
Coordinate cache warming moved ahead of both decisions, so a transit datagram
later dropped for hop limit still contributes its plaintext coordinates. The only
arrivals this newly warms from are those with a zero hop limit, and every insert
they can make is already achievable with a value of one, so it grants nothing
that was not already available and adds no memory growth the cap does not bound.
Four tests added: delivery at zero to this node, the transit drop at one, the
transit pass at two, and the one per hop decrement across a three node chain,
which brackets the emitted value from both sides since it is only observable
through what the next hop does. One existing test was renamed because its
destination was this node, so it never exercised the transit path its name
claimed, and it asserted nothing at all.
Under Noise XX the peer static key is learned during the handshake rather
than pinned in advance, and neither of the two paths that learn one was
checking it against what we already knew. That let an attacker who could
observe and inject on path substitute their own identity, on both a fresh
dial and an established link.
On a fresh dial we recorded who we meant to reach and then overwrote it
with whoever answered, without ever comparing the two. Promotion, the ACL
check and the peer registry all then ran on the answering identity, so an
attacker who raced the real peer to msg2 became the peer and the intended
node was never reached. On an established link, a rekey msg2 was matched
to its peer only by the session index we had put in the cleartext msg1
header, so anyone who saw that header could answer with their own static
and take the link over at the cutover.
Both are now compared before anything is committed. The dial-time
expectation moves into its own field with no setter, so the handshake
cannot overwrite it the way it used to; the identity learned from msg2
keeps landing where it always did. Anonymous dials still promote whoever
answers, which is what shared-media discovery means, and that branch is
chosen locally when we dial rather than from anything on the wire, so it
cannot be reached as an exemption. Both comparisons are decisions made in
the synchronous core alongside the existing classifiers.
The gates sit ahead of every mutation, not merely ahead of the session
install, so a forged msg2 no longer poisons the recorded peer epoch and
never earns a msg3. A rejected dial is rescheduled from the dial-time
expectation rather than the answering identity, so refusing an impostor
does not silently retire a configured peer from the dial schedule.
Only this branch was exposed. The released lines use Noise IK, where the
initiator pins the responder static before it dials and a wrong key fails
the AEAD outright.
No wire change: both gates compare a value we already learn against one
we already hold, and legitimate handshakes behave exactly as before.
Six tests cover it, driving the real cadence and dial paths with a third
node answering the intercepted message. Each was checked to fail when the
decision is forced to accept. Local CI 37/37.
master carries only maint's 0.4.2-dev version bump linkage; next keeps its
own 0.6.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
maint carries only its 0.4.2-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
Bump the version to 0.4.2-dev and update the status badge and prose. The
0.4.x patch line continues; v0.4.1 remains the shipped release it points at.
Follows the shape of the v0.4.0 rollover commit. No maint reset was needed
this time: v0.4.1 was cut from maint, so the tag is already on this line and
it simply continues.
Same split as the maint-to-master merge: version identity stays with the
receiving branch, project state flows up.
Kept next's: Cargo.toml and Cargo.lock at 0.6.0-dev, the status badge, and
the paragraph identifying next as the wire-format-breaking line that will not
interoperate with v0.2.x, v0.3.x, or v0.4.x peers. Only the shipped-release
pointer inside it moved from v0.4.0 to v0.4.1.
The changelog conflict was additive rather than competing, and resolving it
either way would have lost real content. Next's [Unreleased] Fixed section
carries the XX rekey divergence and dual-initiation work; master brought the
[0.4.1] section. Git could not tell these were adjacent rather than rival, so
both were kept in order, with next's Breaking block and its own [Unreleased]
entries untouched.
Took from master: the bloom FPR default change and its duplicate-definition
fix, the docs describing them, the v0.4.1 release notes, the v0.4.0 date
correction, and the root RELEASE-NOTES.md mirror.
Checked before merging that no incoming content names the Noise handshake
pattern, since next is XX where master is IK. The two "Noise IK" strings on
master both predate v0.4.0 and next already carries its own wording for them,
so nothing needed rewording here.
Quartet green: 1698 tests passed, clippy clean with -D warnings.
Carries the v0.4.1 content up the one-way branch flow. The release itself
belongs to maint, so the parts of this merge that identify a version are
resolved in master's favor and the parts that describe the project's state
are taken from maint.
Kept master's: Cargo.toml and Cargo.lock at 0.5.0-dev, the README status
badge, and the "FIPS is at v0.5.0-dev on the master branch" line. A patch
release consumes no minor version and master's development line is
unaffected by it.
Took from maint: the bloom antipoison FPR default change and the
duplicate-definition fix it rests on, the two docs that describe them, the
new v0.4.1 release notes, the correction to the v0.4.0 release date, and the
[0.4.1] changelog section, which slots below master's own [Unreleased] and
above [0.4.0]. The root RELEASE-NOTES.md mirror moves to v0.4.1 because it
tracks the latest shipped release, which also clears the stale provisional
date it had been carrying.
One line needed splitting rather than choosing: the README said "v0.4.0 has
shipped" inside the paragraph that identifies master as 0.5.0-dev. The
version identity stays master's and the shipped-release pointer moves to
v0.4.1.
Quartet green on the result: 1645 tests passed, clippy clean with -D warnings.
Drop the -dev suffix from the package version and refresh the lockfile.
The version is single-sourced from CARGO_PKG_VERSION, so build.rs and
src/version.rs need no change.
Also fix the archived v0.4.0 release notes, which still read
"2026-06-21 (provisional)" while the changelog records the actual
release date of 2026-06-27. The date was confirmed in the changelog at
the time but not in the two release-notes files that carry it, so the
shipped notes have been showing a wrong date and the word provisional
since that release. The root mirror is regenerated for v0.4.1, so only
the archived copy needed correcting.
Back-fill the changelog for the six user-facing changes on this line
since v0.4.0: the antipoison FPR cap default, the bloom probe and
secp256k1 context performance work, the coordinate-cache and path-MTU
fixes, and the removed parent_switched counter. The Unreleased section
was empty, so all six entries are new.
Write the v0.4.1 release notes and mirror them to the root copy. The
notes lead with the FPR cap change and state plainly that this is the
second raise of that default in two releases, that it buys headroom
rather than fixing the underlying fixed-filter constraint, and that the
structural remedy is the v2 filter work. They also flag the rolling
upgrade window, where an upgraded and a not-yet-upgraded node can
disagree about mesh size, and point consumers of parent_switched at
parent_switches.
Bump the README status badge and the roadmap prose together so the two
agree.
The inbound FilterAnnounce cap at 0.10 rejects aggregates that are
legitimately near their operating ceiling, before the network reaches
the fixed-filter capacity limit. Raise the default to 0.20, which
corresponds to fill 0.7248 at k=5, about 2,114 entries on the 1 KB
filter (Swamidass-Baldi).
Also remove the duplicate default definitions in BloomConfig. Each
default was written twice, once in impl Default and once in the serde
default function, with nothing enforcing that they agree, so a config
file that omits the key took a different path from one that sets it.
impl Default now delegates to the serde default functions, leaving a
single source of truth.
The isolation loop and the failure-log dump both built container names
without the run suffix, so under a suffixed run they addressed containers
that do not exist.
The consequence was worse than the visible failure. Two of the three
checks in that loop assert a ping FAILS, and a ping into a non-existent
container fails for the wrong reason - so both passed vacuously, and the
security assertion the loop exists for was silently a no-op. Only the
third check, which expects a ping to succeed, noticed anything was wrong.
Add a guard that fails loudly when the container is absent, so a future
naming slip cannot quietly turn these back into no-ops rather than
surfacing as one confusing failure among two false passes.
These sites were missed because the suite passes when run by hand: with no
suffix set the unsuffixed names are correct, and the defect only appears
under the automation that sets one. Verified this time with the suffix
set, which is the condition that matters: 18 of 18, container names
resolving to real containers.
Making the sidecar network name unique per run exposed a latent problem
rather than causing a new one. With a fixed name, compose found any
existing network and reused it, which quietly masked the fact that the
address range was hardcoded. With a unique name there is nothing to
reuse, so the range is requested every time and the run fails outright if
anything already holds it.
Claim the range instead of assuming it. The suite now tries to create its
network on a candidate range and, if the daemon reports an overlap, moves
to the next candidate. The daemon's own address pool becomes the arbiter,
so two concurrent runs cannot land on the same range at all - as opposed
to a range derived from the run identifier, which only makes a collision
unlikely, and a collision is the precise failure being avoided.
The node addresses are derived from whichever range is claimed rather than
written alongside it, including the gateway address the isolation check
probes. Moved to a range that nothing in the tree claims and that sits
outside the daemon's default pool.
Only an overlap is worth advancing on. Any other creation failure is real
and is reported with the daemon's own message, rather than being retried
sixty-four times and buried - a lesson from a failure elsewhere in the
harness this week that was undiagnosable because its error output was
discarded.
All three nodes now attach to the pre-created network as external, so
teardown removes it explicitly.
Verified: three concurrent claims take three distinct ranges; a
non-overlap error fails fast rather than looping; the suite passes 18/18;
and the claimed gateway address is confirmed pingable when not firewalled,
so the isolation assertions still mean something.
Conflict in the admission-cap suite: this line's copy is the XX variant,
with a different phase 3 and an extra phase the other line does not have.
Took the polled final read from the merge, kept this variant's more
specific failure message and its enforcement-evidence phase.
Also templated two container references that phase and the peer-list read
own. They are unique to this line, so the pass that introduced the per-run
name suffix never saw them, and this suite would have looked up containers
under names that no longer exist once a run sets the suffix.
The suite red-ed with an empty final peer_count while every substantive
assertion passed: both denied peers showed inbound handshake attempts and
zero outbound responses, which is the gate this suite exists to verify.
Two pre-existing defects combined. The final count was sampled once, in
the same second the load driver issues its last round of peer restarts, so
the read can land on a daemon busy with those restarts and return nothing.
And the fallback meant to cover that could never fire, because the shell
applies it to the last stage of the pipeline, which succeeds on empty
input - so an unanswered query was indistinguishable from a genuine zero,
and reported as a cap violation.
Poll the final read for a short window instead, and give the two cases
distinct output. A real regression still fails, just after the retry
window, since the loop exits early only on the expected value. Extract the
read the two phases share rather than leaving them to drift.
Two runs on one host destroyed each other's containers, producing
mid-test "No such container" failures that look like real defects. The
automated builder runs a full local CI on the same box every few minutes,
so the machine is contended almost always and this has red-ed both a hand
run and an automated gate.
Two independent causes are fixed here. Container names were hardcoded, and
docker names are global rather than scoped by compose project, so two runs
collided on the same name; every name now takes an optional suffix that
the harness sets from the run id. And the cleanup sweep matched a label
shared by every run, so one run's teardown force-removed another's
containers; resources now also carry a per-run label and the sweep can be
narrowed to it.
A third hazard turned up that was not in the original report: the sidecar
suite passes explicit compose project names, which override the run-scoped
project and put it outside the shared prefix entirely. Its project names,
network, and derived container references are now scoped too.
Both are default-off. With the suffix unset, names render exactly as they
do today and a bare compose invocation is unchanged, which is what keeps
the hosted CI and the documentation correct. A cleanup run with no run id
still reaps everything, which is what a manual "clear the box" wants.
The literal-name sweep was not sufficient: six scripts build container
names dynamically from node labels, and two suites create their own
containers outside compose. Those are handled at their construction sites.
Verified: syntax check on all modified scripts; compose validation on
every modified file with the suffix both set and unset; and a synthetic
two-run reproduction that shows the old cleanup destroying a bystander run
and the new one leaving it alone.
Known gap: subnets are still hardcoded, so two concurrent full runs will
still collide on address-pool overlap. That fix reaches into topology
configs, chaos scenarios, diagrams and production source, so it is left
for its own change rather than half-done here.
Conflict in the FMP core doc pass: this line has no inbound establish
view at all, so the block that master edited does not exist here and the
merge saw a modify-versus-absent conflict. Resolved by keeping this
line's side - the establish view is master-line only and tracks the
handshake-pattern split, so its absence here is correct rather than an
omission to repair.
The lifecycle view correction and the comment rewrites in the node
module carry across unchanged.
The lifecycle view's doc described the shell as implementing it over a
`connections` map that no longer exists. The establish view named that map
too, and was doubly wrong: both of its snapshot builders read only
`peers`. Rewrite both to describe the maps actually read.
Several comments in the node module carried shorthand from private
working notes - bare parenthetical labels, and one reference to an
internal tracker item - none of which mean anything to a reader of this
repo. Rewrite them so each clause states its own reasoning inline. The
paragraph above the tracker reference already carried the rationale, so
that one is simply dropped.
Comment-only; no behavior change.
The rekey suites' Phase 4 polled until the initiator-cutover count
reached its value at phase entry plus two, while Phase 6 asserts an
absolute minimum of four. On an unloaded host the ping phases can
outrun the jittered first rekey cycle, so Phase 4 can begin with a
single cutover on the books, release at three, and leave Phase 6 to
lose a sub-second race against the remaining first-cycle cutovers.
A rekey-outbound-only run failed exactly this way: five counted
cutover lines by t=32.6s of container life, with the Phase-6 count
snapshot at t~32.4s seeing three.
Floor the Phase-4 wait target at four so a released wait guarantees
the Phase-6 assertion — the docker-logs count is monotone over a
container's lifetime. A genuine rekey stall still times out the
bounded wait and fails Phase 6 with the real count.
Also drop the second-cycle framing from the comment, the phase
banner, and the assertion description: with rekey timers resetting at
each cutover a second cycle cannot begin inside this test's window,
so the counted events are first-cycle cutovers spread across the
topology's links, emitted once or twice per completed link rekey
depending on which side's promotion path logs them.
Two properties that the connection-state consolidation depends on had
no test on this line, because the pattern here is Noise XX and the
natural place to assert them under IK is msg1 -- which under XX carries
only ephemeral keys and so proves neither.
The first is that a learned peer identity lands on the surviving
carrier, the control machine's own connection state, rather than
staying locked inside the Noise handle. Every reader that names a peer
mid-handshake goes there: the promotion hand-off reads it to decide who
is being admitted, and the stale-connection sweep reads it to decide
whether a reaped leg is retried or torn down. XX learns identity once
per role at two different messages, so the guard asserts both -- the
initiator learning the responder from msg2, and the responder learning
the initiator from msg3. The initiator half starts from a deliberately
wrong dial-time expectation, so a missing write cannot pass by leaving
a pre-seeded value in place.
The second is that a peer the access list turns away leaves nothing
behind. Under XX the inbound admission gate cannot sit at msg1, since
no static key has crossed the wire yet, so it sits at msg3 -- by which
point the responder has built a control machine, allocated a session
index, opened a link, and completed the Noise session. All of that has
to come back down on the denial. A machine left registered is
unreachable by every teardown path and holds a peering-budget slot
forever; an index left mapped misroutes the next frame to arrive on it.
Both guards were confirmed to fail when the write they pin is dropped
and when the teardown they pin is skipped. Tests only; no production
behavior changes.
Brings the handshake-leg deletion onto the XX line: the leg struct
and src/peer/connection.rs are gone, the Noise handles and crypto
methods now live on PeerMachine, and every leg accessor reads the
machine's own ConnectionState.
The incoming branch speaks Noise IK, this one speaks XX, so the crypto
methods could not be taken as they arrived. Git added the IK versions
to machine.rs with no conflict marker, and complete_handshake_msg3
-- which XX needs and IK has no counterpart for -- was absent
from the merged tree entirely. All four are re-expressed here: the
incoming structure (the Option<HandshakeCrypto> handle, the self.conn
reads, the borrow scoping that hoists carrier writes out of the leg
borrow) carrying this line's XX bodies. start_handshake is the one
that mattered most: its signature is identical on both lines, so it
compiled silently, and left alone it would have emitted an IK msg1
through the two-argument new_initiator and panicked every anonymous
dial on an expect for an identity XX does not have at that point. It
now uses the one-argument XX form and no expect.
handle_msg1 keeps this line's flow, not the incoming one's. The two
functions share a name but not a shape -- the admission gate moved
out of msg1 into msg3 when the line went to XX, and identity is
unknown until msg3. The machine is built above the crypto because
it now drives it, but it stays a local until the late insert, so a
rejected msg1 still leaves no registry trace and the drop-the-local
error arms are unchanged. Parking it at SentMsg2 after the index is
allocated needed a transition the birth constructor could not provide,
so that constructor now delegates to it.
Node::start_handshake is re-expressed the same way and for the same
reason: keeping the machine a local until all fallible setup has
succeeded preserves the existing error arms exactly, with no machine
disposal to add.
The two leg-to-carrier mirror blocks collapse. With one carrier
the writes they mirrored are self-assignment. Their content
is preserved: the completion touch comes from the relocated
complete_handshake, on the same clock, and the negotiated profile
from process_fmp_negotiation, which now takes the machine directly
and writes through set_conn_peer_profile. Worth recording, because
the collapse was justified on a narrower claim that does not hold:
complete_handshake writes only the touch. It never wrote the profile
on either line. The two writes land on the same carrier at the same
point, so the collapse is neutral, but it is discharged jointly by
two methods rather than by complete_handshake alone.
Three inbound tests are dropped rather than carried. Each asserts
a property at msg1 that XX does not have there: an ACL decision, a
no-registry-trace guard on that decision, and an identity learn on the
carrier. Under XX msg1 is ephemeral-only, so none of the three has a
landing site at that step. A comment at each drop site records where
the property actually lives on this line. One of them was a deletion
here that the incoming branch had merely modified, so its removal
is this line's own prior decision rather than a new one. The same is
true of the craft_and_send_msg1 helper and of HandshakeSeed::inbound,
both left without callers once those tests went.
Also carried: peer_actions.rs had no conflict and no incoming change,
but three sites reached deleted Node accessors and are repointed at
the machine; the EstablishView snapshot is not resurrected, as this
line has never had it; and the connection-state doc keeps this line's
XX wording with the registry-independence invariant grafted in.
The per-peer control machine has absorbed every field the pending
connection carried. What remained was a struct holding two Noise
handles beside a duplicate copy of bookkeeping nobody read. Replace it
with a small carrier for the two handles and delete the type.
Presence of that carrier, not the state of the handles inside it, is
what marks a machine as mid-handshake. The distinction is essential
rather than stylistic: a failed handshake drops its initiation handle
and is deliberately retained so the stale sweep can reclaim it, and a
completed one has its session taken before disposal. Deriving presence
from the handles would make both invisible to the sweep, the
connection count, and the peering budget at once, leaking the slot
permanently. A test drives an empty carrier past every presence
predicate and then detaches it, so a future edit cannot quietly couple
the two.
The remote startup epoch now comes from the surviving carrier, which
the handshake operations already wrote at the same two points with the
same value. The paired writes onto the pending connection's own
bookkeeping had no readers left and are gone.
The handshake-phase surface leaves the public API: it was public by
accident rather than design, and the machine behind it is crate
internal. Callers outside the crate that need a view of pending
handshakes go through the operator queries, which are unchanged.
ConnectionState::inbound_with_transport loses its last non-test caller
with the inbound seed and is marked test-only.
The pending connection and the per-peer control machine have carried
duplicate copies of the handshake-phase fields since the machine gained
its own connection state. Read them from the machine and drop the
connection's projections.
The peer identity is the sharp one. The connection learned it from msg1
and the machine's carrier did not, so the two views genuinely disagreed
for inbound connections until the Noise operations moved onto the
machine and began recording each result on both. That is now in place,
so every reader can take the machine's copy unchanged: the establish
snapshot, the promotion sweep for competing connections, the dial and
path in-progress checks, the peering observations and per-peer in-flight
budget, and the stale-connection sweep's retry address.
Also repointed: our_index, their_index, transport_id, started_at, the
stored handshake message bytes, and the idle-timeout check. Two
duplicate index writes on the connection are dropped, both immediately
preceded by the machine-side write of the same value. Reads that
previously came off a connection detached just before its machine was
disposed now capture the machine's value first.
Test seeding follows the establish paths: the seed builders write
our_index to the carrier, which promotion now reads. A new test pins
the inbound identity learn on the carrier and the retry address a
failed inbound connection reports to the sweep, so a silent regression
to a blank identity cannot pass.
ConnectionState::duration loses its last non-test caller with the
connection accessor and is marked test-only.
These three had no counterpart on the control machine, so readers still
reached them through the pending connection. Add machine-side
accessors and repoint every reader, then drop the connection's.
The peer address needed its writes lifted, not just its reads
repointed: the machine's copy was never written. It is now written at
each of the points the connection's copy was — the inbound seed, the
dial, message-2 completion, and the two paths that seed a machine from
a pre-built connection — so promotion and the resend path read a value
with the same provenance at the same time as before.
Link and direction need no lift. Both machine constructors already
seed them from the same arguments the connection is built with, so an
outbound machine carries outbound state and an inbound one inbound,
and the machine's link always equals the connection's. The handshake
operations' direction guards read the machine's copy for the same
reason.
The stale-connection sweep's teardown log and its resend path both now
take the transport and address from the machine, and each keeps its own
check that a pending connection is still attached rather than relying
on the caller to have established it.
Add a test pinning link, direction, and address on the two shapes that
seed a carrier independently — the dial and an accepted message 1. The
cross-connection winner reads both values from a carrier one of those
two already seeded.
The pending connection drove its own Noise handshake while the control
machine held it, so the crypto and the state it produces lived on the
carrier that is going away. Move the six operations onto the machine:
starting and completing an initiation, processing an inbound
initiation, taking the session, testing for one, and dropping the
handle on failure. Bodies are unchanged apart from reaching the
handles through the attached connection.
Each operation now records its results on both carriers. The learned
identity, the remote epoch, and the activity stamp are written to the
machine's own bookkeeping at the same point and with the same value as
they are written to the connection's. The connection's copies still
have readers until those reads are repointed, so both have to be
written; the machine's copy of the learned identity was previously
never populated for an inbound connection, which is why an inbound
pending row showed no expected peer.
Driving the handshake from the machine means the machine has to exist
before the crypto runs. On the inbound path it is built above the
message-1 processing and still kept local, so a rejected message
leaves no registry entry and allocates no index. On the outbound path
it already existed from the dial, so it simply takes the connection
before the index allocation, and both failure arms unwind it as they
did.
Completing message 2 no longer mirrors the activity stamp separately,
since the completion itself now writes both carriers at the point the
mirror was approximating.
Three tests cover what the compiler cannot. A connection whose
handshake failed holds neither Noise handle yet must stay visible to
the stale-connection sweep, or every failed connection would leak and
hold a peering-budget slot forever. A message 1 rejected by the crypto
or by the ACL must leave no machine, no index, and the same rejection
count as before. A dial whose message-1 preparation fails must unwind
the machine registered at dial time; that test drives the
index-allocation failure rather than a crypto failure, which is the
arm this change actually widens, since the allocation now happens with
the connection already attached.
`Node::connections()` yielded the pending connection itself, so its
consumers reached the handshake-phase fields through that value. The
pending connection is being folded into the control machine, and the
fields will move off it, so yield the machine (keyed by its link) and
let each consumer reach what it needs from there.
Membership is unchanged: the iterator still selects machines that
carry a pending connection, which is the predicate `connection_count`
and the stale-connection sweep already use. Every consumer takes the
same value from the same place, one hop further out, and no field
changes carrier here.
The method is now internal to the crate. It yields the control
machine, which is not part of the published surface, and nothing
outside the crate iterates pending connections — operator views of
the handshake phase go through the query surface instead.
`promote_connection` detached the pending connection and then
interleaved reads off that detached value with three separate lookups
of the same control machine. Gather the machine-side fields
(`their_index`, `transport_id`, link stats) in the single `get_mut`
that takes the connection, so the machine is borrowed once.
Behaviour is unchanged. The connection is still taken before anything
is validated, so a rejected promotion leaves the machine with no
pending connection, and the prelude only gathers options: the checks
below it still report the first missing field in the same order
(`our_index`, `their_index`, `transport_id`, `source_addr`).
Link stats move to the prelude, ahead of those checks. The value is
the same either way: they live on the surviving carrier rather than
the detached connection, and nothing between the two points touches
the machine map. The lookup could not fail at the old site either,
since the function had already reached it through a successful lookup
on the same key, so dropping the defaulting arm changes nothing.
Add a test covering the error order and the detach, driving promotion
with connections missing each required field in turn plus every later
one, so an implementation that validated during the prelude would
report the wrong field. Test seeding grows a variant that lets the
caller shape the seed.
Tests built a free-standing PeerConnection, mutated it, and handed it to
Node::add_connection by value. None of those sites survives the removal of
PeerConnection, so converting them afterwards would mean one enormous commit
that cannot be reviewed honestly. Convert them now, while the struct still
exists and the conversion can be validated against a green tree.
Adds a cfg(test) Node::seed_handshake_machine plus a HandshakeSeed builder,
and rewrites make_completed_connection (now seed_completed_connection) and
the twenty inline builders onto it. add_connection keeps its body and loses
its test callers.
The builder's carrier seeding is a verbatim copy of add_connection's: the two
conditional writes for their_index and transport_id, then set_leg, through the
same entry().or_insert_with() so an existing leg-less machine keeps its
constructor-side fields. Nothing else reaches the carrier -- our_index,
source_addr, post-construction started_at and the stored handshake bytes stay
leg-only. Seeding more than that would let these tests observe a carrier
richer than production's and keep passing even if a later production
write-lift were missed.
The Noise exchange now runs on the already-seeded leg rather than before the
hand-over. That is neutral because the only read of expected_identity is
guarded by is_outbound, and no crypto method allocates a session index.
Also adds a compile-time check that PeerAction is Clone + Eq, which is what
keeps a runtime handle from being smuggled into an action payload.
Forward-merge the state-carrier collapse (connection start/activity timestamps,
peer-identity telemetry, handshake resend buffers, index/transport bookkeeping,
and the session-index shadow all moved onto the peer machine's own connection
state) onto the XX handshake.
Next-specific resolutions folded into the merge: the responder learns its transport
id and peer index at msg1 and they are written directly onto the machine there (the
XX machine has no msg1 step, so the promote read would otherwise be unset); the
negotiated peer profile is mirrored onto the machine carrier at both the initiator
(msg2) and responder (msg3) negotiation sites; and the outbound carrier writes land
on both the identified and anonymous dial paths. The leg's transport/index/buffer
copies remain as dead duplicates until the leg dissolves; the machine copy is the
sole live source.
The session index was carried three ways: on the leg, on the peer machine's own
connection-state copy, and on a separate machine-owned shadow field. Populate the
machine's connection copy on the outbound path (it was written only on the inbound
and cross-connection-swap paths before), then delete the shadow field and route its
readers, the decrypt-registration lifecycle, and the rekey cutover through the
single carrier. The shadow and the connection copy diverged only on paths that are
unreachable or dormant in production (a rekey cutover with no newly allocated index,
the swap winner's later reads, and the never-dispatched timeout/disconnect events),
so the merge is byte-identical for every reachable path. Rewrote the outbound-promote
index tests and the dial-time comment for the single-carrier contract.
The connection leg's their_index, transport_id and link_stats duplicated the peer
machine's own connection-state copy. Route every production reader through the
machine copy: promotion, the stale-connection reaper, the outbound msg1 resend, and
the in-use / connecting-path checks. Seed the inbound machine's transport_id from the
msg1 packet (the same source the leg used), matching the outbound dial seed, so the
hard-required promote read never sees a missing transport. link_stats is a
never-mutated zero seed, so its leg accessors are removed outright; the leg
their_index/transport_id getters/setters remain only for the test-only connection
builder. Also delete the write-only next_resend_at_ms field from the connection
state (the resend deadline is carried by the machine-armed retransmit timer, not this
field). Byte-identical.
The connection leg's stored msg1/msg2 handshake-resend buffers duplicated the
peer machine's own connection-state copy. Write the machine copy at the outbound
msg1-prep and the two inbound authorize sites, then source the retransmit
resend-bytes reader, send_stored_msg1, and the pending tier of find_stored_msg2
from the machine. The post-promote active-peer msg2 copy is written from the same
wire bytes, so the pending tier now matching after promotion is value-identical.
Byte-identical resend behavior.
The expected_peer connection-row field read the peer identity off the leg;
source it from the machine's own connection-state copy instead, matching the
started_at/last_activity treatment. The machine carrier holds the same identity
as the leg for every connection that appears in the view: outbound legs seed
both from the dialed identity, and inbound legs never rest in the connections
view (the machine takes the leg at promotion), so their machine copy is
unobserved. The leg accessor stays for the crypto-extraction path and retry
decisions. Byte-identical.
The connection leg's started_at and last_activity duplicated the peer
machine's own connection-state copy. Make the machine copy the sole live
telemetry and timeout carrier: re-stamp it with the leg's provenance when
the leg is attached at handshake start (the msg1-prep clock, not the earlier
dial-time constructor value) and mirror the completion touch, then delete the
leg's last_activity/touch accessors and source the connection-row projection,
show_connections, and the timeout readers from the machine. Byte-identical
for all normal paths.
Re-express the handshake-state carrier collapse onto the XX code: the leg's
handshake_state field is deleted and the displayed state is derived from the
peer machine's phase, with failure carried on the machine (a send_failed flag
that preserves the handshake phase) rather than on the leg. The next projection
maps the SentMsg2 responder phase to received_msg1 and the anonymous-dial
Discovered phase to sent_msg1; the three initiator send-failure sites carry
failure via send_failed. Telemetry strings, wire bytes, index allocation, and
stale-connection reaping are byte-identical to next.
The connection leg's HandshakeState field duplicated the peer machine's
handshake phase. Delete it and derive the displayed handshake state string
from the machine's PeerState, looked up by link id (mirroring the resend
count). Move the failure signal onto the machine: a send_failed flag that
preserves retransmit eligibility (the machine stays in its handshake phase),
alongside the existing Failed state. The leg's crypto self-gates now guard on
Noise handle presence instead of the deleted phase field, and mark_failed
only drops the handle. Telemetry strings, wire bytes, index allocation, and
the stale-connection reaping are byte-identical for all normal paths.
handle_msg3 now routes on the InboundDecision returned by the machine's
inbound_msg3 call, instead of computing establish_inbound in the shell and
handling the terminal arms inline. The DualRekeyWon and ResendMsg2 arms'
index free moves to the machine's FreeIndex action (run through the
executor); the ResendMsg2 stored-msg2 resend stays inline. The pre-decision
reject gates and the proceed-arm bodies are unchanged.
Behavior-neutral: the same establish_inbound decision over the same snapshot
and wire, evaluated once in the machine, with a byte-identical free-count on
every arm and no change to wire sends, machine state, or telemetry.
on_inbound_msg3 becomes inbound_msg3, returning the InboundDecision
alongside the machine-phase actions so a caller can route on it; the step
arm delegates and drops the decision. The signature is unchanged, so
step-driven callers are untouched.
Also drop two emissions that are dead on the live path: the ResendMsg2
arm's SendHandshake (the stored-msg2 resend is a driver mechanism the
executor would otherwise reframe) and the dormant inbound-msg1 arm's
retransmit timer. No live behavior changes.
Carries the inbound establish-decision reshape from the IK line. Next's XX
inbound surface already computes that decision on its own machine, so the
merge records the shared history without changing next's tree.
handle_msg1 now creates one local machine before classification and routes
on the InboundDecision it returns, instead of calling establish_inbound in
the shell and re-deriving the decision inside each arm. The promote and
restart arms drop their own per-arm machine construction and the redundant
InboundMsg1 step; their phase-1 actions come from the single call. The
effect-bearing arm bodies (rekey-respond, duplicate resend, reject
bookkeeping, and the promote/restart tails) are unchanged.
Behavior-neutral: the same establish_inbound evaluation over the same
snapshot and wire, done once in the machine method rather than once in the
shell, with byte-identical arm bodies and no change to index allocation,
wire sends, machine state, or telemetry.
Make the FIPS core build and run as an embedded Android library. The host
app owns the TUN (e.g. an Android VpnService) and FIPS performs no
system-TUN or CAP_NET_ADMIN operations.
Squashed from the following changes:
- gate desktop transports/TUN by target_os, not features: a plain
`cargo build` now compiles for every target with no flags. Ethernet (raw
AF_PACKET / BPF) is gated to linux/macos, so Android (target_os =
"android", not "linux") self-excludes it as Windows already did; real
system-TUN ops are gated per linux/macos and Android gets a no-op stub;
the ipi6_ifindex cast handles it being i32 on Android vs u32 on macOS. No
Cargo features are introduced; desktop builds are unchanged.
- app-owned TUN seam: Node::enable_app_owned_tun() lets an embedder that
owns the TUN fd exchange IPv6 packet bytes with FIPS over channels
instead of FIPS creating a system TUN device. It returns (app_outbound_tx,
app_inbound_rx): the embedder pushes packets read from its fd into the
outbound sender (app -> mesh) and pulls packets destined for its fd from
the inbound receiver (mesh -> app). start() gates system-TUN creation on
tun_tx being unset, so with the channels pre-installed it skips device
creation and does no system-TUN ops; both directions reuse the existing
inbound-shim and run_rx_loop wiring. Packets entering via app_outbound_tx
bypass handle_tun_packet, so the embedder must push only fd00::/8-destined
packets and clamp TCP MSS on outbound SYNs; the rustdoc and the
IPv6-adapter design doc spell this out.
- keep the android target warning-clean so the cross-compile check passes
clippy -D warnings.
- add an Android cross-compile CI check: cross-compile the library for
aarch64-linux-android via cargo-ndk and run clippy -D warnings. Android
ships as an embedded library (the host app owns the TUN), so there is no
daemon binary to package; this is a check job, not a packaging one.
- docs: list Android as a supported platform.
Tests: app_owned_tun_seam_wires_channels covers the channel round-trip and
the Active state; start_skips_system_tun_when_app_owned runs start() and
asserts no named system device is created.
The inbound msg1 machine arm re-derived the establish decision for
its own action selection while the shell matched on a separately
computed copy. Expose the arm as a method returning the decision
alongside the actions so a driver can route on it directly; the
step dispatch delegates and keeps its action-only shape.
The dead rekey-respond arm is stripped to decision-only and its
helper deleted: it allocated from the real index allocator, wrote
the rekey shadow index, emitted a wrong-framed rekey send, stamped
the dampening clock, and flipped state, none of which happens on
the live path, where the inline respond body owns all effects. The
resend-msg2 arm's send emission is stripped the same way; the
decision already carries the stored bytes.
send_stored_msg1 marked the embedded leg failed by writing it
directly from the shell. Route the write through a new
HandshakeSendFailed machine event instead: the machine marks its
leg so the stale-connection sweep reclaims it, without leaving the
handshaking state, so retransmit eligibility survives the window
between the failed send and the sweep exactly as before.
send_stored_msg1 gains a now_ms parameter threaded from the action
executor for the step call; the failure arm itself ignores it.
handle_msg2 no longer pre-computes establish_outbound alongside the
machine's own evaluation of the same snapshot. The shell now builds
the snapshot, steps the persistent outbound machine once at the
decision point, and routes on what comes back: the promote action
vector drives promotion through the executor as before, and the
ResolveCrossConnection decision selects the inline swap/keep
resolution bodies, which are unchanged.
The machine step and its defensive transient-rebuild move up from
the promote arm to the decision point; the cross-connection path
keeps its take-leg-then-dispose ordering with the step preceding
both. Comment prose at the touched sites refreshed to describe the
single-decision-site shape.
The machine's on_msg2 cross-connection arms crystallized state and
emitted FreeIndex/RegisterDecryptSession actions that duplicate the
inline shell resolution, but they are unreachable on the live path
(the shell removes the machine before running the swap/keep bodies).
Making them live in that shape would double-free the outbound index
through the executor.
Strip both arms to a single new ResolveCrossConnection { swap }
action: a decision conveyed to the driver, not an effect. The shell
intercepts it and runs the inline resolution, which owns all effects
permanently. The action executor gets a defensive unreachable arm.
The Promote arm and set_their_index are unchanged.
Carries the leg-embed storage move: PeerMachine.leg replaces the
Node.connections map. Hand-resolved onto the XX surfaces keeping this
line's semantics with the new storage: the msg3 termination arms and
executor swap/rekey-responder teardowns keep their exact cleanup sets
and index-free behavior; the msg2 cross-connection extract takes the
leg before the unconditional machine dispose; msg1 and anonymous leg
births embed the leg at machine insertion; the rekey-vs-establish gate
tests leg-absence. The merge also drops the now-redundant explicit
machine inserts next to the seeded test seam, whose auto-merged
combination clobbered leg-carrying machines.
The pending-handshake PeerConnection map and the per-peer control
machine map were parallel LinkId-keyed structures whose keysets must
stay coherent by hand. With every leg now born with a machine, the leg
becomes storage inside its machine (leg: Option<PeerConnection>, pure
storage the machine never reads or drives) and Node loses the
connections field; every access routes through the machine.
The non-mechanical lowerings, each argued at the site: the
rekey-vs-establish gate in handle_msg2 tests leg-absence (an established
peer's machine stays keyed by its link, so machine-presence would
misclassify every rekey msg2 as a fresh establish); the
connecting-predicates, peering observation, and handshake-slot budget
iterate machines-with-legs so connect-window machines (leg not yet
born) are excluded exactly as before and never double-counted against
their pending-connect slot; the cross-connection extract takes the leg
before disposing the machine; the stale reaper takes the leg and leaves
the machine untouched when none is present, matching the old early
return. The map-coherence debug check keeps its machine-has-carrier
direction with the embedded leg as a carrier; the leg-to-machine
direction is now true by construction and its gate const is gone.
connection_count() counts machines with legs; the connections()
iterator, the test seams, and the control-socket connection rows are
re-implemented over the embedded legs with unchanged output.
The anonymous-discovery dial paths drove no control machine: the leg was
born bare in start_handshake and a throwaway machine appeared only at
msg2. Anonymous legs now birth a persistent identity-less machine
alongside the leg (new_outbound widened to Option<PeerIdentity>, backed
by the existing identity-less outbound connection state), and handle_msg2
crystallizes the learned identity onto it before stepping. The msg2
missing-machine arm becomes defensive recovery (debug-assert + insert,
matching the IK line), and the self-connect drop arm disposes the
machine it previously leaked — reachable from identified dials too,
since the handshake completion overwrites the expected identity without
comparing it to the dial-time expectation. The self-connect arm's
index/link/pending-entry leak is pre-existing and stays as-is.
With every leg now born with a machine, the leg-to-machine direction of
the debug coherence check switches on for this line. The add_connection
test seam seeds direction-faithful machines (an outbound-anonymous
seeded connection previously got an inbound-shaped one), and the
hand-rolled dial sites in tests seed dial machines mirroring
production. New tests pin the identity-less birth, the msg2 identity
crystallization through promote, and the self-connect disposal.
The inbound control machine was a msg3-time throwaway: handle_msg3 built
a transient, stepped it once for the decision, and discarded it, with
promote_connection birthing a fresh established() machine. Every
handshake leg now gets one persistent machine for its whole life:
- handle_msg1 births the machine parked in the sent-msg2 phase alongside
the window leg (msg1 crypto and the msg2 build/send stay inline; the
msg2-send-failure cleanup disposes it).
- handle_msg3 steps that persistent machine instead of a transient. The
decision path is unchanged: the msg3 handler never reads machine state
and overwrites the identity-plane fields from the wire outcome before
dispatching, so every arm's decision and actions are byte-identical.
- promote_connection stops rebirthing: the executor feeds the promotion
result back into the machine (the shape the IK line already uses) and
the machine crystallizes to Established in place.
- Every path that tears down an inbound window leg now disposes its
machine through remove_peer_machine: the seven inline msg3 termination
arms, the six executor swap/rekey-responder consumption sites, both
promote-failure arms, the single-peer leaf reject, the losing side of
a cross-connection, and the msg1 send-failure cleanup. The
promote-failure disposal also fixes a pre-existing leak: an outbound
leg whose promotion fails had its connections entry consumed before
the error, so the stale reaper could never reach its dial machine.
Index-free behavior is unchanged at every site. Two test helpers that
hand-roll outbound dials now insert the dial machine production
inserts, and new tests pin the machine's birth phase, its post-promote
crystallized state, and disposal on the failure and rekey-responder
arms.
Carries the debug-build peer-map coherence check. The leg-to-machine
direction is gated OFF on this line (const flipped in the merge): the
inbound msg1-to-msg3 window legs and anonymous-discovery outbound legs
legitimately run machine-less here, so only the machine-to-carrier
leak-tripwire direction asserts until every leg is born with a
persistent machine.
Assert, once per tick in debug builds, that the peer control-machine map
and its carriers stay coherent: every control machine has a live
handshake leg, an active peer, or a pending connect (a machine with none
is a leak the stale reaper can never see), and every handshake leg has a
machine. The second direction sits behind a file-local const so a branch
where handshake-window legs legitimately run machine-less can gate it
off without weakening the leak tripwire.
Teach the add_connection test seam to seed a control machine for the leg
it inserts (derived from the connection's direction and identity), and
remove_connection to dispose it, so the seam-built topologies satisfy
the check. Three unit tests: seam coherence, coherence through a real
promotion, and the orphaned-machine panic.
Carries the dead-code deletion (PeerSlot enum, PeerConnection resend
API) and the module-doc liveness corrections. Hand-resolved the doc
conflicts with next-appropriate wording: the executor module doc names
the live msg3 decision-machine path (PromoteToActive /
SwapToInboundSession / RekeyRespondTrigger) while keeping the still-true
msg1-inline dormancy note; the peer_machines field doc reflects next's
promote-time established() births; the machine module doc drops the
stale unwired claims, including the provisional-and-unwired note on the
msg3 trigger variants which are live.
Several module and field docs still described the per-peer machine as
unwired shadow scaffolding. It has been live for some time: machines are
inserted at dial and inbound msg1, stepped by the handshake handlers and
the rekey-cadence and liveness-reap routers, and the executor's
SwapSendState/CompleteDrain/InvalidateSendState arms are the authoritative
paths (the inline bodies survive only as debug-assert release fallbacks).
Rewrite those docs to the current truth while keeping the still-true
dormancy facts: PeerEvent::Timeout and PeerEvent::Tick are never
dispatched in production, retransmit fires on the machine-armed deadline
while the timeout reaper keys on timer presence with the config
threshold, and the remaining inert executor stubs are SendRekey,
SendLinkMessage, and the connected-UDP arms. Drop the stale
allow(dead_code) on the peer_machines field.
PeerSlot (and its entire impl surface) was referenced only by its own
module tests and the lib.rs re-export; peer storage has always used the
separate connections/peers maps. PeerConnection's resend_count/
next_resend_at_ms/record_resend delegations had no production callers:
the live resend counter is machine-sourced (connection_resend_count reads
the per-peer machine), and the FSP session layer uses SessionEntry's own
methods. ConnectionState::next_resend_at_ms is now test-only (its
remaining callers are the fmp state unit tests) and marked cfg(test).
The PeerSlot unit tests go with the enum; test_resend_count_tracking is
dropped because the delegation target's schedule arithmetic is already
covered by the fmp resend_bookkeeping test.
Fold in a next-only wording fix for the OpenWrt 802.11s mesh backhaul:
next's FMP link handshake is Noise XX, not the Noise IK on master, so the
fips-mesh-setup header and the how-to now name it pattern-agnostically
("the Noise handshake"), matching the shipped fips.yaml comment and
staying accurate on this branch.
Router-to-router radio backhaul over an open 802.11s mesh interface,
with FIPS providing all encryption (Noise IK), authentication, and
routing on top of bare L2 neighbor links. The mesh runs OPEN with
mesh_fwding 0 — SAE would duplicate the Noise layer and force ath10k
raw mode, and FIPS is the routing layer — so the Noise handshake is the
real auth/encryption boundary and FIPS's spanning tree does the routing.
fips-mesh-setup: an opt-in UCI helper that creates a per-radio
mesh-point interface (radio0 -> fips-mesh0, radio1 -> fips-mesh1;
trailing-digit derivation with a free-index fallback and a collision
guard). Radio setup stays opt-in — a package must not commandeer radios
on install. 'remove' takes an optional radio and otherwise removes all
instances. Dual-band routers get one instance per radio; FIPS treats the
two backhaul paths as failover, not multipath: it keeps one active link
per peer (cross-connection resolution picks a single winner), and the
second band stands by, re-establishing the peer after keepalive timeout —
traffic never uses both bands at once.
fips.yaml ships the mesh0/mesh1 Ethernet-transport entries commented
out, so a stock install that never creates fips-mesh* logs no per-boot
"interface missing" bind warning. fips-mesh-setup uncomments the matching
meshN block when it creates the interface and re-comments it on remove,
so the flash-and-drop-in flow needs no manual config edit. The file is
rewritten 0600-first (it may hold an inline nsec) via an atomic replace.
Two field-found silent non-peering causes are surfaced by the helper and
the guide:
- Same channel: mesh points only peer on a shared channel, and 'auto'
lets each radio pick its own. The helper prints the radio's
band/channel and warns loudly on 'auto' with the exact uci command to
pin one; the how-to gains an ordered no-peers triage (channel mismatch,
on-air scan check, DFS CAC wait, regdomain).
- STA channel capture: a client (sta) interface drags the whole radio to
its upstream AP's channel, so a mesh pinned elsewhere never joins and
does not recover until the STA disconnects. The helper warns when the
target radio carries a STA; the guide documents the incompatibility of a
roaming uplink with a fixed-channel mesh on the same radio.
Both the create and remove paths run 'wifi reload', which briefly drops
every client AP on all radios; the how-to sets that expectation.
Regression test: the shipped OpenWrt fips.yaml must parse via the real
Config deserializer in both states — as shipped (mesh inactive) and after
the uncomment the helper performs.
Packaging: the helper is installed across the ipk/apk/buildroot paths
(three synced copies), with the CI structural checks and shellcheck
targets extended to cover it. Full guide in
docs/how-to/set-up-80211s-mesh-backhaul.md.
Phase 4 of the rekey suite waited for only one more initiator cutover
(guaranteeing three) while Phase 6 asserts at least four. The fourth
cutover then had to land in the brief window between the wait returning
and the Phase 6 log snapshot, so host load could push it past the window
and fail the run even though every cutover completed correctly. Wait for
two more cutovers, the full second rekey cycle, matching the Phase 6
threshold, so the asserted count is guaranteed before it is checked.
Re-express the rekey-msg2 and cross-connection observation feeds onto
the XX cores. The rekey-msg2 observation lands inside the block that
installs the pending session (where XX also generates msg3), so it fires
only when the install actually ran. The cross-connection observation
sits at the outbound resolution arms and targets the promoted peer's
machine; the inbound session-swap effects already own their executor
path here and are untouched. Crypto effect bodies are byte-unchanged.
Feed the per-peer machine two observation events so its control/rekey
shadow state stays coherent with the inline crypto-session installs,
without moving those installs. The crypto effect bodies are unchanged.
On rekey-msg2 completion, feed the machine the responder index on the
success path only (after the pending session is installed); the abandon
path feeds nothing.
On cross-connection resolution, feed the machine the swap outcome: on a
swap it records the new outbound and responder indices to mirror the
session replacement, and on a keep it does nothing. The observation
targets the promoted peer's machine, not the outbound leg whose machine
is removed as the block is entered.
Both observation handlers emit no action; nothing consumes the updated
shadow state yet.
Move the two Noise-XX inbound establish effects — the cross-connection
session swap and the rekey-respond session install — out of the inline
handle_msg3 arms and into their per-peer machine executor actions
(SwapToInboundSession, RekeyRespondTrigger). The inbound establish
decision now falls through to the shared machine-drive arm, matching the
promote/restart arms.
Carry the session index on the InboundMsg3 event and seed it into the
transient machine before dispatch. The swap actions build their index
from the machine's connection state, which is unset on a fresh inbound
transient; without the seed the actions would emit empty and the swap
would silently no-op while leaking the allocated index.
Give the inbound reject and msg2-resend machine arms their terminal
Failed state and index free, matching the inline arms that own those
paths, so a later timer cannot free the index or report loss for a peer
whose handshake has already resolved.
Preserve exactly: the three distinct index frees and their paired index
and pending-outbound map removals; the link-teardown asymmetry (the
cross-connection path drops the address mapping, the rekey-respond path
keeps it); the session take-then-install ordering; and every reject kind
on the error sub-paths.
Fold the handshake timer lifecycle (msg1 retransmit + timeout/stale
reap) through the per-peer machine, re-expressed onto the XX handshake
surface.
- Route peer-machine removal through remove_peer_machine so each link's
timer store is dropped together with its machine (choke-point).
- Home the msg1-retransmit decision on the machine-armed retransmit
timer; advance the resend counter only on send success and relocate
the operator-visible resend count onto the machine.
- Reap outbound handshake timeouts via a presence-scan over the machine
HandshakeTimeout timer, reading the threshold from config each tick so
the reap stays neutral for any handshake_timeout_secs.
Cancel both dial-armed handshake timers on outbound promote so a
promoted machine carries no stale timer entry. Preserve the guard that
suppresses a msg1 resend at a peer already promoted via the inbound
cross-connection path.
The XX inbound HandshakeTimeout presence-scan is neutral only because
the inbound establish path sends msg2 inline and never dispatches the
inbound machine event, so no inbound leg ever populates a
HandshakeTimeout timer. A future change that drives inbound establish
through the machine must re-verify the reap equivalence for inbound
legs.
Move the outbound handshake-timeout reap off the unconditional check_timeouts
scan onto the machine-armed HandshakeTimeout timer. A new drive_handshake_timeouts
(run before the retransmit drive, so a timed-out leg is reaped rather than resent
on the same tick) reaps the outbound legs that carry a HandshakeTimeout timer and
have idle-timed-out this tick.
The timer's presence selects the leg (only outbound legs arm one; IK inbound arms
none); the reap threshold is the shell is_timed_out(now, config) predicate, not
the timer's stored deadline. The machine arms the timer from a hardcoded constant
at dial, which is not authoritative for an operator-tuned handshake_timeout_secs,
so reading the threshold from config each tick keeps the reap neutral for any
timeout value and on the last_activity clock exactly as before.
check_timeouts keeps reaping everything else: failed connections (all of them,
promptly) and the idle-timeout of legs without a machine timer (inbound legs, and
any machine-less connection). Total coverage is unchanged. check_timeouts runs
before the timer drive, so a failed outbound leg is reaped there first, its timers
dropped, and the timeout drive never double-fires on it.
Split drive_peer_timers into the timeout drive and the (byte-identical) retransmit
drive. The dormant machine timeout handler is not dispatched, so the session index
is freed once, by cleanup_stale_connection. A regression test drives a timed-out
outbound leg to reap; the existing check_timeouts tests continue to anchor the
residual sweep. Test count 1631 to 1632.
Move the outbound msg1 resend off the unconditional tick function onto the
machine-armed retransmit timer. The per-peer machine already arms a
HandshakeRetransmit deadline at dial; a new drive_peer_timers fires the due ones
(kind-filtered to the retransmit timer) and homes the resend counter on the
machine, where the operator-visible count now reads from.
The resend decision reads the same operator config as before (interval, backoff,
max), the wire bytes and transport target still come from the shell connection,
and the pure core computes the backoff schedule. As with the deleted
resend_pending_handshakes, the count and reschedule advance only on a successful
send: a failed send neither advances the count nor marks the connection failed,
it just retries on the next tick. The handshake-timeout timer stays on the legacy
check_timeouts path, and the rekey/liveness timers keep their own shell drivers,
so drive_peer_timers deliberately fires only the retransmit kind.
The show_connections resend count is relocated to read from the machine (the
counter's new home) via connection_resend_count; machine-less and inbound
connections report 0, matching what the shell connection reported before. Delete
resend_pending_handshakes and resend_candidates (the latter also from the
LifecycleView trait, its only user). The resend unit test is re-expressed against
the machine + timer path, covering the due check and the record-on-success
semantics.
Known limitation: the first resend interval is armed from the machine's hardcoded
1000ms constant, which equals the config default. Under an operator override of
handshake_resend_interval_ms the first resend diverges from the pre-change
dial+interval; subsequent resends and the cap remain config-driven. Neutralizing
the first-resend override needs the interval threaded into the sync core (the
shell arm would be clobbered by the machine's own timer arming at dial), deferred
to when the connection and machine entities merge. Test count unchanged at 1631.
The control machine already emits SetTimer/CancelTimer actions when it arms the
handshake retransmit and timeout deadlines, but their executor arms were a
single no-op stub and the machine's Timeout event was never dispatched -- the
real work still runs on the legacy tick (check_timeouts,
resend_pending_handshakes).
Add the storage the time-as-input driver will read: a peer_timers map keyed by
LinkId then TimerKind, holding each armed timer's absolute deadline. The SetTimer
arm now inserts (overwrite = reschedule) and CancelTimer removes; the outbound
msg2 promote cancels the two dial-armed handshake timers, since the machine
survives promotion and the entries would otherwise linger in the store.
This store is a shadow: it is written and cleared but no driver reads it yet, so
behavior is unchanged. The legacy tick stays authoritative until the driver that
feeds Timeout and deletes the overlapping tick paths lands.
Route every machine removal through a new remove_peer_machine(link) choke-point
that drops the timer store alongside the machine, replacing the twelve direct
peer_machines.remove sites so no armed timer outlives its machine. TimerKind
gains Hash (to key the store) and Ord (for deterministic driver collection); it
is internal and never serialized. Test count unchanged at 1631.
Bring the outbound-handshake-through-the-machine series onto the next-branch
Noise-XX cores. Re-expressed rather than transcribed: next has no outbound
decision core, so the promote drives PeerEvent::OutboundMsg2 (not the IK Msg2
snapshot event) and cross-connection resolution stays inline. Hybrid provenance
-- identified outbound legs persist the control machine at dial and drive
through it; anonymous-discovery legs (identity unknown until XX msg2) retain the
inline transient-at-msg2 path. ReportLost{kind} routing, the connection-oriented
dial via the machine (OpenTransport + TransportConnected), and the prepare/send
msg1 split all carried over.
Behavior-neutral on next: the XX msg2 body (Noise completion, FMP, ACL, msg3) is
untouched; anonymous legs are unchanged; the dial-persisted machine promotes
with our_index unset.
Route the connection-oriented (TCP/Tor) outbound path through the peer state
machine, matching how the connectionless path already works.
initiate_connection's oriented branch now drives PeerEvent::Dial with
connection_oriented=true; the machine parks in Connecting and emits
OpenTransport, whose executor arm performs the non-blocking transport.connect
and pushes the PendingConnect. When the connect resolves, poll_pending_connects
prepares msg1 in the shell and then drives PeerEvent::TransportConnected, which
sends msg1 via the machine's SendHandshake arm.
The msg1 prepare (index allocation, Noise leaf, wire arming) MUST run in the
shell before the TransportConnected drive: send_stored_msg1 only transmits an
already-armed wire, so a drive-only path would silently send nothing. The
machine's our_index stays unset; the connect-failure path keeps its direct
handshake-timeout handling (TransportFailed stays dormant). The now-unused
Node::start_handshake helper is removed.
Behavior-neutral: same transport.connect, same PendingConnect, same msg1 send
and failure teardown as the removed inline path -- only the driver changes from
inline shell code to the state machine.
Move the peer_machines insert for an outbound leg out of the tail of
prepare_outbound_msg1 to a single shared site in initiate_connection, before
the connection-oriented / connectionless fork. This lets the connection-oriented
path find the machine at dial time (so it can be driven through the connect
handshake) without prepare_outbound_msg1 -- which for that path runs after the
connect completes -- clobbering an in-progress machine back to Discovered.
Because the machine now exists before the fallible dial steps, add
peer_machines.remove to every failure path in the widened dial window: the
index-allocation and Noise-leaf failures in prepare_outbound_msg1, the oriented
transport.connect failure, and both poll_pending_connects teardown arms
(handshake-start failure and async connect failure). remove_link does not touch
peer_machines, so these explicit removes are required.
Behavior-neutral: the connectionless path still drives the machine to
Handshaking and sends msg1 identically; the machine's our_index stays unset
(no spurious UnregisterDecryptSession on a later inbound restart); a failed dial
leaves peer_machines empty for that link exactly as before; and no
connection-oriented machine drive is wired here.
The connection-oriented (TCP/Tor) outbound connect->handshake path had no
`cargo test --lib` coverage; it was exercised only by the opt-in Tor
integration suites, and the TCP node tests bypass it via a manual
connectionless handshake helper. Add three unit tests:
- a machine-level test driving Dial{connection_oriented:true} -> Connecting
(emitting only OpenTransport, no msg1) -> TransportConnected ->
Handshaking{SentMsg1}, asserting the exact OpenTransport and
SendHandshake+SetTimer action vectors that start_outbound_handshake emits
(its oriented reach via on_transport_connected was previously untested; the
connectionless reach via on_dial was already covered);
- two node-level tests over a real loopback TcpTransport: a successful connect
that reaches start_handshake (observed via a pending_outbound entry), and a
connect to a closed port that routes through the failure arm (link torn down,
no msg1 dispatched).
Tests only; no production change.
Split the outbound handshake setup out of start_handshake into
prepare_outbound_msg1 (allocate the index, run the Noise leaf, frame and arm
msg1 -- the fallible steps, returning an error the caller propagates) and
send_stored_msg1 (transmit the armed wire). A connectionless dial now runs
prepare in the shell, then drives the control machine, whose SendHandshake
action sends the wire via the executor. Connection-oriented dials keep calling
start_handshake, now prepare followed by the send inline.
The dial event gains a connection_oriented flag so the machine's on_dial sends
msg1 immediately for connectionless transports (no connect step) instead of
opening a transport first. Behavior is unchanged: the same index, Noise leaf,
wire bytes, maps, and send-error handling as before, with index-allocation and
Noise failures still propagated synchronously before the send. A regression
test covers that a promote from the post-dial handshaking state is identical to
the former discovered-state promote.
Create and persist the per-peer control machine when an outbound handshake is
dialed, keyed by its link, instead of building a transient at msg2. The msg2
completion path now looks up that persisted machine to drive the promote,
falling back to a transient only if none is present (e.g. a direct-seeded test).
The machine parks in the Discovered state until promotion and is inert to the
liveness reap and rekey cadence while unpromoted, since it is absent from the
peers map. It is removed on every path that ends the outbound leg without
promoting -- the stale-connection reaper, the msg2 authorization-failure arm,
and the cross-connection resolution block -- mirroring the connection's own
lifetime so no dangling machine survives.
Its session index is deliberately left unset on the machine (the shell owns the
index on its connection), so a later inbound restart does not emit a spurious
decrypt-session unregister. A regression test covers that invariant.
Add a LostKind discriminator to the ReportLost action so the executor can
route an un-promoted handshake failure to the connected-guarded reconnect
reflex (note_handshake_timeout) and an established peer's link-death to the
unconditional one (note_link_dead), instead of collapsing every loss to
note_link_dead.
The two loss producers dispatched today, the liveness reap and the
inbound-restart-then-promote arm, both keep the link-dead routing, so
behavior is unchanged. The handshake-timeout and dial-failure producers are
tagged accordingly but stay dormant until their events are dispatched.
On two inbound msg3 outcomes the msg1-allocated inbound session index
was dropped without returning it to the allocator: losing the dual-rekey
tie-break (we keep our in-progress rekey and drop their msg3), and a
duplicate handshake that resolves to a msg2 resend. Under repeated hits
this slowly leaks index space. Free the index on both arms, matching the
sibling reject arms.
Add a test-only helper to backdate a peer's session-established instant,
and two regression tests that drive a real second handshake into each
arm and assert the index is returned to the allocator (both fail if the
free is removed).
Remove the now-stale `#[allow(dead_code)]` on `advance_peer_machine`
and `execute_peer_actions`; both are live (called from the link-dead
reap, rekey cadence routing, and the handshake establish sites).
In `route_rekey_cadence`, reuse the ambient context timestamp for the
machine step instead of taking a second clock sample; the extra
sample's only consumer was the currently-inert cutover drain timer, so
the change is behavior-neutral.
Forward-merge the action-contract round-trip test. This line's action type
carries two extra establish-trigger variants (SwapToInboundSession,
RekeyRespondTrigger), so the merge resolution extends the test's sample set
and its wildcard-free exhaustiveness match to cover all of them — the test
proves every variant, including the two XX-only triggers, round-trips
unchanged through the async channel.
Prove the per-peer machine's action type is a runtime-agnostic message
contract: it is Send + Sync + 'static, and every variant round-trips
unchanged through a single-threaded async channel (a current-thread runtime,
sender and receiver on one thread). A wildcard-free match over the variants
makes any future action a compile error here until it is added to the sample,
so a variant that embedded a runtime handle could not slip through unproven.
Forward-merge the promote-failure warning target pin. The peer-action
executor diverges between the two lines (IK vs XX establish arms), so the
pin was applied to this line's own two promote-Err warns during the merge
resolution: outbound "Failed to promote connection" and inbound "Failed to
promote inbound connection" now emit under fips::node::handlers::handshake,
matching the sibling max-peers reject log already pinned to that target.
The two promote-failure warn! sites in the peer-action executor emit under
this module's own tracing target rather than the handshake target the rest of
the establish-path logging uses. Pin both (outbound "Failed to promote
connection" and inbound "Failed to promote inbound connection") to
fips::node::handlers::handshake, matching the sibling max-peers reject log and
keeping all establish-path diagnostics grouped under one target regardless of
which module physically emits them.
Comments across the per-peer machine, executor, lifecycle supervisor,
and peering reconciler carried internal rollout labels and design-note
section references. Rewrite them to describe the code's behavior
directly. Comment-text only; no code changes.
Route the outbound side's msg2-completion promote through the per-peer
state machine and its action executor, mirroring how the net-new inbound
promote is driven. A transient outbound machine emits the promote action;
the executor performs the promotion and, on success only, clears the
pending-outbound entry (threaded in through the action context). The
outbound cross-connection path stays inline as before.
The debug log for rejecting an over-cap inbound connection is emitted from
the peer-action executor, under that module's tracing target. Nodes that
raise only the handshake module to debug (leaving other modules at info) no
longer surfaced the rejection, so cap-enforcement events were invisible in
their logs. Re-pin the log to the handshake target, matching the adjacent
promoting-inbound log, so the rejection stays visible under a
handshake=debug filter regardless of the executor module's own level.
Route XX inbound establishment through the per-peer machine: handle_msg3 hands
the establish decision to a transient machine and drives the net-new promote
(Promote + RestartThenPromote) through the executor's PromoteToActive action,
which transcribes the shared promote block (promote_connection plus the
msg2/tree-announce/bloom follow-ups). The four non-promote decisions (Reject,
ResendMsg2, CrossConnect, RekeyRespond) stay inline byte-for-byte; the two
session-swap arms are deferred to a later non-neutral step.
The decrypt-worker register stays inside promote_connection (not relocated),
no promotion-resolved feedback is fed back, and the transient machine is never
inserted into peer_machines (the persistent machine is created once inside
promote_connection). Behavior matches the inline path exactly, and operator
diagnostics are preserved: the relocated promoting-inbound log keeps its
handshake tracing target, and the epoch-mismatch restart breadcrumb is
re-emitted.
Route the initiator rekey cutover and drain-completion through the
per-peer machine and executor instead of running them inline in
check_rekey. The batch poll_rekey and its per-peer snapshots stay
shell-side and unchanged, so the cross-peer phase ordering that governs
the shared index-allocator sequence is preserved; the machine only
consumes each decided action in that order.
The routed cutover and drain reproduce the inline bodies exactly (the
executor's swap-send-state and complete-drain arms), including the
info-level cutover log and the decrypt-worker re-register. Initiating a
rekey stays inline; the machine is fed a rekey-initiated observation
afterward to keep its control state coherent. A guard falls back to the
inline bodies if a machine is somehow absent.
Route the liveness reap through the per-peer machine: on a link-dead
timeout, the mmp reap advances the peer's machine with a link-dead event
and executes the resulting actions, instead of calling remove_active_peer
and note_link_dead inline.
The effect is unchanged — the machine emits InvalidateSendState then
ReportLost, which the executor maps back to remove_active_peer(peer)
then note_link_dead(peer, now), the same two calls in the same order.
The reap log stays shell-side so it keeps the handlers::mmp target, and
a guard falls back to the inline path if a machine is somehow absent.
Insert a per-peer machine into peer_machines at each promote site and
remove it on teardown, so every established peer has exactly one machine
entry keyed by its link id. Nothing drives the machine yet — the entry
is inert — but this correspondence is what the rekey and liveness-reap
folds depend on.
Inserts pair with the two promote_connection arms (normal promote and
cross-connection-won); removes pair with remove_active_peer and the
cross-connection loser teardown. A new PeerMachine::established
constructor parks the machine in the post-handshake Established state.
Behavior is unchanged: nothing reads the map on a live path.
Add the executor that consumes the per-peer machine's actions, plus the
peer_machines map that homes the machines on the node. Both are shadow:
nothing populates the map or invokes the executor yet.
The rekey-cutover, drain, liveness-reap, and index-plane action bodies
are ported to reproduce the next-line handlers exactly (the cutover log
matches check_rekey's info-level line, kept under the handlers::rekey
target so operator and test log filters still see it). The establishment
actions (promote, cross-connect swap, rekey-respond) are deferred stubs
until the inbound establish path is wired.
Re-derive the per-peer FMP lifecycle state machine against the next-line
XX establishment cores, as an unwired module. It ports the master-line
machine's structure but adapts it to the XX handshake:
- inbound establishment splits into a msg1 event (allocate the index,
send msg2, defer) and a msg3 classify event (identity crystallizes,
establish_inbound runs), matching XX's identity-at-msg3 timing;
- the inbound decision handles all six InboundDecision variants including
the XX-only CrossConnect;
- the outbound establish_outbound path and the two-phase authorize are
dropped (no analog on the XX line);
- session-carrying establish effects (cross-connect swap, rekey-respond
pending session) are represented as plain-data trigger actions; the
session surgery is done shell-side when the machine is wired.
Nothing drives the machine yet; it is dead code pending the executor and
handler wiring. Unit tests cover the XX dispatch: inbound establish to
Established, dual-init tie-break, cross-connect at msg3, rekey cutover
key-consistency, and liveness reporting.
Forward-merge the per-peer FMP decomposition from the master line onto
the next line. The establishment and rekey machine-drive on refactor-node
is built on the IK establishment cores (establish_outbound, identity
learned at msg1) that the next line replaced with the XX msg3 model, so
it cannot be carried textually; it is re-derived against next's XX cores
in follow-on commits on this branch.
This merge commit carries only the two changes that are neutral and
next-compatible as-is:
- OwnedFd hygiene for the connected-UDP socket (RawFd -> OwnedFd, drop the
hand-rolled unsafe Drop; OwnedFd's own Drop closes the descriptor).
- The two-tier send-state boundary: the send-critical subset of ActivePeer
is regrouped into a PeerSendState struct, with accessors kept stable so
no caller changes. next's XX establishment, rekey, and liveness-reap
behavior is unchanged (verified: exact field partition, identical
accessor signatures and semantics, identical init values, 3-arg mmp
construction).
The per-peer machine, its action executor, and the peer_machines map are
not carried here; they are re-derived against next's XX establishment
surface in subsequent commits on this branch.
Route each link-dead peer that the tick sweep's plan_heartbeats decides
to reap through the per-peer machine and executor, replacing the inline
reap body in check_link_heartbeats. The batch decision, the liveness
snapshots (read from the hot-path-written receive clock), and the
heartbeat-send arm stay shell-side and byte-unchanged; the machine only
consumes the decided LinkDeadSuspected, tearing the peer down via
remove_active_peer and reporting the loss to the reconciler exactly as
before, on the same tick with the same wall-clock timestamp. The reap
log stays shell-side.
The machine's link-dead handler no longer emits a decrypt-session
unregister keyed by its shadow index (the full peer teardown already
unregisters the real index; the shadow could have drifted to a reused
index), and its guard now covers the Established state a freshly
promoted peer sits in.
Handshake-timeout, retransmit, and stale-connection cleanup stay inline:
they act on pre-promotion legs that have no machine, and the loss reflex
they use differs from the link-dead one.
Route each Cutover and Drain that the shell-side batch poll_rekey decides
through the per-peer machine and the executor, replacing the inline
effect bodies in check_rekey. The batch decision and its per-peer
snapshots stay shell-side and byte-unchanged: poll_rekey phase-groups all
cutovers, then all drains, then all initiations across the peer set, and
that ordering governs the shared index allocator's free-then-allocate
sequence that appears on the wire, so the machine only consumes the
already-decided actions (a new RekeyConsume event) without re-deciding.
InitiateRekey stays inline (its Noise msg1 build is a shell-side leaf)
with a RekeyInitiated observation feeding the machine so its control
state stays coherent for the next tick's cutover.
The cutover and drain logs, which relocated into the executor in the
prior commit, are pinned back to the fips::node::handlers::rekey tracing
target so they stay visible under the operator's module log filter.
Also clears the machine's shadow draining_index on drain so a later
cross-connection resolution cannot double-free the already-freed index.
Prepare the establish executor to drive FMP rekey by activating the
SwapSendState action (initiator K-bit cutover via cutover_to_new_session,
with the gated decrypt-worker re-registration) and adding a CompleteDrain
action (erase the drained previous session: free its index, drop its
peers_by_index entry, unregister its decrypt session). Both reproduce the
current inline rekey.rs cutover and drain bodies. The machine's drain
mapping now emits CompleteDrain, using the real drained index rather than
a shadow copy.
Unwired: nothing drives the machine's rekey path yet (live rekey still
runs inline), so these arms are unreachable and the change is
behavior-neutral. The cadence fold that routes cutover and drain through
the machine follows.
Move register_decrypt_worker_session out of promote_connection into the
executor's PromoteToActive handler, gated on a promoted or
cross-connection-won result. Every live promote now flows through that
one executor path, so registration still fires exactly once at the same
synchronous point; the direct test callers of promote_connection spawn
no worker pool, so the call was already a no-op for them.
Add the cross-connection loser-link teardown (close the losing
transport, remove its link, re-point addr_to_link at the winner) to the
executor as a guarded follow-up. It is unreachable on the current driven
establish paths, which only promote net-new peers, and asserts so, but
keeps the executor complete for when that case is driven.
Remove the now-dead drive_promote_to_active and ConnAction::PromoteToActive.
Cut the net-new outbound handshake completion (a received msg2 that
promotes a fresh outbound leg to a new peer) over to the per-peer
control machine, mirroring the inbound cutover. handle_msg2 still runs
the msg2 prologue, the ACL check, and the cross-connection swap/keep
arms inline; for the net-new promote it now builds a transient machine,
steps it, and drives promote_connection through the executor. The
session index was already allocated at dial, so there is no two-phase
authorize here.
The wire, index sequence, and peer registry state after promote are
byte-neutral. To keep the promote-failure path neutral, the executor's
cleanup now distinguishes inbound from outbound: an outbound promote
failure records the reject only, matching the prior handler, rather
than the inbound path's link and index teardown.
Cross-connection swap/keep, rekey-msg2, and the dial path stay inline;
the loser-link surgery for the currently unreachable driven
cross-connection case lands with the register relocation next.
Cut the restart handshake (an inbound msg1 from a peer that reconnected
with a new epoch) over to the per-peer control machine, completing the
inbound establish cutover. The old peer is torn down and the fresh leg
promotes through the same two-step authorize-then-allocate path as the
net-new case: the machine's first step emits the old-peer teardown
(invalidate send-state, report loss), the shell interposes the ACL
check, and the second step allocates the new index and sends msg2. The
old index is freed before the new one is allocated and the msg2 wire
bytes are unchanged, so the sequence stays byte-neutral.
With restart driven through the machine, the shared inline establish
tail that only the restart arm reached is deleted.
Also bounds the peer_machines map (remove_active_peer now drops the
peer's machine entry) and restores the msg2-send, promote, and
index-allocation failure warnings the cutover had dropped.
Cut the net-new inbound handshake (a fresh msg1 that promotes to a new
peer, plus the at-capacity reject) over from the inline handle_msg1 logic
to the per-peer control machine. handle_msg1 still classifies via
establish_inbound and still owns the Noise wire step, the late ACL check,
and the promote_connection registry surgery; for the net-new path it now
builds the machine, steps it, and executes the returned actions.
Authorization is interposed between two machine steps so the session
index is allocated only after the ACL check passes: a rejected or
unauthorized msg1 consumes no index, matching the prior order exactly.
The msg2 wire bytes, the index-allocation sequence, and the reject
metrics are all byte-neutral. Restart, resend, rekey-respond, and the
other reject arms stay inline unchanged; they move to the machine once
outbound establish is cut over and every promoted peer has a machine.
Also fills in the executor's send-failure and promote-failure cleanup so
a mid-establish error tears the leg down and frees its index exactly as
before.
Add Node.peer_machines (a LinkId-keyed map from the stable link handle to
the per-peer control machine) as the home for the machines, and a new
dataplane/peer_actions.rs holding execute_peer_actions / advance_peer_machine:
the executor that maps each PeerAction the machine emits to its shell call —
frame and send a handshake via build_msg2, drive promote_connection and feed
the PromotionResult back through the machine, tear a peer down via
remove_active_peer, free session indices, report loss via note_link_dead.
Actions for the rekey, connected-UDP, and timer paths are stubbed with notes
for the commits that fold those mechanisms in.
Unwired: nothing drives the machine yet — no live handler path calls the
executor and peer_machines is never populated — so this is behavior-neutral;
the inbound and outbound establish paths still run their existing inline
logic. The executor is cut over path-by-path in the following commits.
Draw the control/published-send-state boundary inside ActivePeer by
grouping the send-critical fields — the three epoch session slots
{current, previous, pending}, the K-bit flag and session-start, the
transport target, the connected-UDP handles, and the hot counters —
into a new co-located PeerSendState struct. The control-tier fields
(identity, connectivity, declaration/ancestry, filter and tree-announce
groups, remote_epoch, the rekey-negotiation sub-machine, and the rest)
stay on ActivePeer.
Behavior-neutral: a pure field regrouping. Every accessor signature is
unchanged (bodies now read/write self.send.*), so the hot path and the
handlers are byte-untouched; both K-bit cutovers still rotate the three
slots atomically with the same control-tier updates. No Arc/ArcSwap —
the fields are co-located and read by plain borrow; publishing behind a
shared cell is later plumbing for a sharded data plane.
Introduce src/peer/machine.rs: a sans-IO per-peer control FSM that
consolidates the scattered handshake/rekey/timeout driver logic now
spread across node/handlers. The machine is a pure reducer —
step(event, now, index_allocator) -> [action] — that reuses the
existing FMP decision cores (establish_inbound/establish_outbound/
cross_connection_winner/poll_*) rather than reimplementing any
decision, and returns runtime-agnostic actions the driver executes.
Control-tier state only; the published send-state boundary and the
driver wiring land in following commits. The machine is terminal at
Closed — re-dial is the reconciler's, so it holds no cross-attempt
retry state.
Includes eight unit tests: inbound and outbound establish, N:1
identity crystallization, the dual-initiation tie-break,
restart-override, rekey initiator cutover, the data-plane-owned
responder cutover boundary, and liveness -> link-dead -> report-lost.
Unwired — nothing calls it yet.
Replace the hand-rolled RawFd field and the unsafe Drop on
ConnectedPeerSocket with an OwnedFd, whose own drop glue closes the
fd. from_fd now stores the OwnedFd it already receives instead of
stripping ownership through into_raw_fd, and the manual libc::close is
gone, shrinking the unsafe surface.
Behavior-neutral: the fd still closes exactly once at last-Arc-drop and
as_raw_fd returns the same underlying fd while the socket is alive.
Wire exit detection for the four directly-observable optional children so
the supervisor FSM's ChildExited edge fires at runtime. A runtime
child-liveness mpsc channel carries a Child on exit: the DNS task and the
two TUN threads self-report when their body returns, and a 2s poll monitor
reports mDNS and Nostr via new is_finished accessors. The rx_loop gains a
select arm that steps the FSM and republishes health (Degraded, since a
running node always has at least one transport up). Transports and worker
pools expose no runtime-exit signal and are left for a follow-up.
Adds LanRendezvous::is_finished and NostrRendezvous::is_finished; the
latter treats a shutdown-taken connect_task as finished so the monitor
terminates after a stop instead of polling forever.
Add the sans-IO half of runtime child-liveness monitoring: a ChildExited
event and an on_child_exited handler that routes a runtime task or thread
exit the same way a start failure does. An optional child exiting degrades
the node (Degraded, with the child recorded in the health reasons); the
last transport exiting publishes Failed. There is no restart, and the
handler is inert outside Running (startup, drain, and teardown own their
own child bookkeeping via the pending/up sets). Failed here is a published
health signal only, not a teardown.
The start and runtime paths share the health classification, extracted
from resolve_start_health into classify_health.
The producer that emits the event (the exit-detection wiring) lands in a
following commit; the event variant carries a temporary allow(dead_code)
until then.
The overlay-discovery cutover to the sans-IO reconciler dropped the
operator-facing "open-discovery sweep complete" summary, including its
per-reason skip breakdown, because the sweep logic moved into the
log-free core. Have the core accumulate the enqueue/skip tally as it
reconciles and return it as data; the driver adds the two values only it
holds (the raw cache size and the self-advert filter) and emits the
summary, reproducing the old startup-vs-per-tick summarize gate and the
budget-zero debug path.
The self-advert precedence matches the pre-cutover sweep: a stale or
self-configured own advert is attributed to the age/configured buckets,
not skipped_self.
Behavior-neutral: the tally is pure side-counting; no dial or enqueue
decision changes.
Forward-merge the peering homeostatic reconciler onto the XX/v2 handshake
line. The three scattered peering mechanisms (auto-connect and retry,
overlay discovery, and opportunistic transport-neighbor growth) are now
unified in the sans-IO reconciler on both lines.
Hand-resolved on the first-contact surface: the reconciler opportunistic
layer emits a connect for anonymous, identity-unknown transport legs
unconditionally (bypassing the connect budget and per-peer cap, gated only
by the addr_to_link dedup), reproducing the XX dial-before-identity-known
behavior; named legs route through the reconciler with inputs identical to
the master line. handle_msg1/msg3 keep the XX path; the peer-loss reflexes
use the relocated wrappers. Behavior-neutral versus the pre-merge next line.
Cut the last scattered peering mechanism — opportunistic growth from
transport-neighbor beacons and LAN mDNS — over to the reconciler's
opportunistic layer via a gate-checked reconcile_opportunistic wrapper,
one call per tick slot (transport and LAN stay separate slots so their
per-tick budget and per-peer cap are not shared). The driver keeps the
beacon/mDNS I/O and the path-granular prefilters that read live state
(self, fresh-enough-to-skip, connecting-on-path) and executes the emitted
Connect intents; the core owns the connected/budget/per-peer-cap
decisions. A first-wins per-peer dedup on the LAN path reproduces the old
inline once-per-peer dial now that the snapshot core cannot observe the
intra-tick connecting feedback.
Delete the now-dead discovery_connect_budget helper. With this, all three
scattered peering mechanisms (auto-connect and retry, overlay discovery,
neighbor growth) are unified in one sans-IO reconciler. Behavior-neutral;
unit test count unchanged (1607).
Cut the overlay (Nostr open-discovery) enqueue over to the sans-IO
reconciler. A gate-checked reconcile_overlay wrapper runs the overlay
layer alone at the discovery tick slot; the monolithic reconcile would
re-fire the always-on retry-dial and double the per-tick dial cap, so the
overlay slot must call only its layer. The driver builds the candidate
pool from the overlay advert cache (self excluded), the cooldown and
configured-npub sets, and the startup/steady max-age, then feeds the
reconciler; the emitted enqueue set drives the per-enqueue identity-cache
and alias pre-seed exactly as before. Enqueued entries are dialed at the
retry slot, preserving the two-phase cadence.
Delete the now-dead enqueue-budget and expiry helpers. Behavior-neutral:
same candidates enqueued in the same order with the same budget, cap, and
expiry. Opportunistic transport-neighbor growth remains imperative and is
cut over next. Unit test count unchanged (1607).
Route the auto-connect floor and the connection-retry mechanism through
the sans-IO reconciler instead of the imperative Node methods. A thin
peering driver (note_handshake_timeout / note_link_dead) centralizes the
reflex path with the gate guard and the already-connected check; the
per-tick retry-dial and the startup floor build reconciler inputs and
execute the emitted Connect intents. The three imperative retry methods
are removed and their call sites rerouted.
Wire the drain and startup gates. Entering a bounded drain now clears the
retry schedule and suppresses the peer-loss reconnect reflex (the drain
window runs under the Suspended gate), so the drain no longer fights a
reconnect for the peers it just closed. The startup floor runs under an
explicit Reconciling gate at the existing peer-connect seam, preserving
the current dial position.
Behavior-neutral except the intended drain change. Overlay discovery and
opportunistic transport-neighbor growth remain imperative for now; they
observe the relocated retry schedule and are cut over next. Unit tests
migrated to the driver API with assertions unchanged (1607).
Move retry_pending and pending_connects from Node into the Peering owner
struct (Node.peering), the home introduced with the reconciler core.
Pure mechanical relocation: the two collections now live behind
self.peering, with roughly seventy accessor sites repointed. No decision
logic is touched and the unit-test count is unchanged (1607). The
imperative peering methods still run; the reconciler is wired in the
following commit.
Introduce PeeringReconciler, the synchronous decision core for peer
desired-state, with its input/action vocabulary (Gate, Budget, Observed,
Candidate, DiscoveryPools, Policy, PeeringAction) and the Peering owner
struct. reconcile() computes connect/retry intents across four layers:
the auto-connect mandatory floor, the ceiling-only overlay enqueue, the
opportunistic transport-neighbor growth, and the node.limits ceiling
enforced inline. It mirrors the supervisor sans-IO shape: no I/O, no
clock reads (time enters as a parameter), no runtime handles.
The core owns its own cross-attempt retry schedule so escalating backoff
survives the fresh connection created per re-dial. Gate-guarded reflexes
(on_handshake_timeout / on_link_dead) reproduce the current backoff math
and no-op while draining. Overlay is strictly ceiling-only (no peer-count
set-point). Disconnect is defined but never emitted (no shedding).
Unwired here: the driver still runs the imperative peering methods and
Node holds the live retry map. 11 unit tests drive the pure core with
synthetic inputs. No behavior change.
Homing the retry schedule under peering/ lengthened the RetryState
module path, pushing the retry_state_iter return-type line past the
100-column max_width. Rewrap to satisfy rustfmt. No behavior change.
Create src/node/peering/ as the home for the peer desired-state
(homeostatic reconciler) concept and move the cross-attempt connection
retry schedule into it: RetryState plus schedule_retry /
schedule_reconnect / process_pending_retries.
Mechanical relocation only. The methods remain inherent on Node; the
retry_count must persist across re-dials (a fresh connection is created
each attempt), which is why the schedule belongs in the peering home
rather than on a per-connection type. The one-level-deeper module path
requires pub(super) -> pub(in crate::node) to preserve the prior scope,
plus module-path fixups at the reference sites. No behavior change; unit
test count unchanged (1596).
Determine node health at start completion instead of unconditionally
reaching Running. Zero transports up is now Failed (fatal): start()
tears down cleanly and returns an error, and the daemon exits. Any
configured optional child that failed to start - a transport beyond the
first, Nostr, mDNS, TUN, DNS, or a worker pool - leaves the node
Degraded but serving, with an operator warning naming what failed. All
configured children up is Full. A child the node was never asked to run
does not count against health.
The published NodeState gains Degraded and Failed variants, both visible
via control queries; Degraded is operational, Failed is not. The
lifecycle FSM gains the health states plus the PublishState action that
drives them - a health fork cannot be a single direct state write, which
is why the earlier commits deferred it to here.
Runtime child-exit health re-evaluation (a running child dying) is a
separate liveness-monitoring mechanism left for a follow-up; this commit
is start-time health only.
Add an operator-visible Draining phase on daemon shutdown. On the
shutdown signal the node broadcasts Disconnect to all peers, then keeps
serving for a bounded window - up to node.drain_timeout_secs (default 2s),
exiting early once all peers are gone - before tearing down. This lets
in-flight traffic settle and peers observe the disconnect before the
transports close, rather than the previous immediate teardown.
The lifecycle FSM gains a Draining state plus Drain/DrainDeadlineElapsed
events; the run loop observes the shutdown signal and transitions to
draining in place - one continuous loop, so the channel receivers are
never destructively cancelled. The published NodeState gains a Draining
variant, visible via control queries during the window. The immediate
stop() path used by tests and non-daemon callers is unchanged: it still
tears down immediately with no drain wait.
The reconciler-gate actions the drain emits are no-ops until the peering
reconciler lands and consumes them.
Introduce a sans-IO lifecycle supervisor: a synchronous step(event) ->
[action] state machine (SupervisorFsm) that authors the substrate-child
spawn and teardown order, plus an owner struct (Supervisor) holding the
substrate runtime fields and embedding the FSM. The substrate-lifecycle
fields leave Node's flat list into the owner - state, packet_tx, the TUN
reader/writer handles + shutdown fd + channels, the DNS task + identity
channel, the Nostr/LAN rendezvous drivers, and the encrypt/decrypt worker
pools; the dataplane keeps packet_rx.
start()/stop() become the driver executing the FSM's SpawnChild/StopChild
actions: same children, same order, same warn/debug-and-continue on
optional failures, same logs, same NodeState transitions. Behavior- and
wire-neutral. The only determinism change is that transports now tear
down in ascending-id order, previously nondeterministic HashMap iteration.
The bounded Draining phase and the Running{Full|Degraded} health split
land as separate follow-on commits.
Forward-merge the dataplane/session concept-home reorganization onto
the next line. The moved files carry next's (XX-line) content; the one
hand-resolved surface is src/peer/active.rs, where the source-location
citation sits in different surrounding text than on the master line —
resolved by keeping next's body and applying the same node/session.rs
-> node/session/mod.rs citation update.
Reorganize the node module tree by concept rather than by
message-handling verb, as the first step of the node runtime
decomposition. Pure relocation: no wire, config, metric, or log
semantics change; the lib test count is unchanged (1577 passed).
Moves (git mv, 100% rename similarity):
- handlers/{forwarding,rx_loop,connected_udp,dispatch,encrypted}.rs
-> node/dataplane/ — the whole RX hot path (the select! run loop,
transit/local forwarding, the link-message router, the RX decrypt
path with responder K-bit cutover + roam writes, and connected-UDP
fast-path activation) now lives in one home.
- node/session.rs -> node/session/mod.rs — establishes the session
concept home for the data/state types. The message-behavior file
handlers/session.rs stays put for now (folds in with the later FSP
session step).
The IK/XX-divergent establishment files (handlers/{handshake,rekey,
timeout}.rs) and the deferred-home files (handlers/{mmp,lookup}.rs)
deliberately stay in handlers/, to move once rather than twice.
Every module is reached through impl Node methods, so no call site or
re-export shim was needed. Updated in lockstep with the moves: the
module_path!-derived tracing targets in the two mesh-lab compose-trace
overlays, a structural test's include_str! source path, doc-comments
in proto/routing and the mesh-lab docs, and the stale source-location
citations (node/handlers/{forwarding,rx_loop,encrypted}.rs and
node/session.rs) in doc-comments and the discovery design doc.
Forward-merge the fipstop routing-stats label flip and the three fixes
carried up from maint: drop the redundant parent_switched counter, keep
the tighter path_mtu on a LookupResponse, and reuse one shared secp256k1
context. Clean auto-merge into next's post-refactor structure.
Forward-merge three maint fixes, hand-relocated into master's
post-sans-IO / discovery-to-lookup structure:
- drop the redundant TreeMetrics parent_switched counter (keep
parent_switches), reconciled across the refactored tree/mmp sites and
the spanning-tree test reads
- keep the tighter path_mtu when applying a LookupResponse, now in
handlers/lookup.rs after the discovery module rename
- reuse one shared secp256k1 context in the identity module
Every sign/verify/key-derive site built a fresh context via
Secp256k1::new(), which allocates a Secp256k1<All> and runs
randomization/blinding table setup on each call. Introduce one
crate-wide LazyLock<Secp256k1<All>> and reuse it across the local, peer,
and auth sites (and their tests). Behavior-neutral: identical secp256k1
API calls, only the context lifetime changes, and the shared All context
still performs the standard construction-time blinding.
An originator handling a LookupResponse unconditionally overwrote the
cached path_mtu_lookup entry, so a looser (larger) estimate in a later
response could clobber a tighter value already learned from a reactive
MtuExceeded or PathMtuNotification. Read-and-compare before writing and
keep the minimum, so a looser discovery estimate no longer loosens the
clamp. Add a regression test.
parent_switched was incremented on the line immediately before
parent_switches at every site and never independently, so the two
counters were always identical. Drop parent_switched from TreeMetrics,
its snapshot, TreeStatsSnapshot, the show_tree fixture, and the fipstop
render, keeping parent_switches as the sole counter.
The code on next is currently v1 after the v1.5 removal during the
sans-IO refactor, but this doc still described the abandoned v1.5 design
as implemented, which is misleading. Add an interim status banner
pointing at v2 as the pending target until the doc is rewritten to v2.
The routing-stats pane's Discovery Requests/Responses sections show the FMP
overlay coordinate-lookup counters. Rename the section labels and the nested
JSON key they read from "discovery" to "lookup" to match the metric family's
canonical name. The daemon dual-emits both keys, so this reads the current
name and no longer depends on the deprecated "discovery" alias.
Forward-merge the transport neighbor-beacon rename (ethernet + BLE
discovery -> neighbor module/buffer rename; ethernet config discovery
flag -> listen with serde alias). Hand-resolved the module files against
next's identity-out-of-beacon beacon rewrite by starting from next's
bodies and re-applying the rename by identifier, so next's beacon logic
(no-arg build_beacon, bool parse_beacon, computed BEACON_SIZE, one-arg
add_peer, deleted BLE add_peer_with_pubkey) is preserved unchanged.
Rename the Ethernet per-interface config flag from discovery to listen,
so the receive/transmit toggle pair reads as the symmetric announce
(transmit) / listen (receive) neighbor-beacon vocabulary. The old
discovery: key is still accepted via a serde alias, so deployed configs
load unchanged; to_yaml re-emits it under the canonical listen: name.
Marked deprecated for removal at the v2 cutover.
Updates the config field + accessor, the transport listen_enabled local,
the one struct-literal test consumer, the chaos sim config generator,
packaged fips.yaml examples, and the classified operator-facing docs
(ethernet neighbor-beacon subsystem prose; the generic Transport
discovery capability prose is left unchanged). Adds a compat parse test
asserting the legacy alias, the new key, and that deny_unknown_fields
still rejects unknown keys. Behavior-neutral.
Mirror the ethernet neighbor rename in the BLE transport: the internal
peer-detection buffer becomes NeighborBuffer (from DiscoveryBuffer) and
the module is renamed discovery -> neighbor. The BlueR/bluez API terms
(DiscoveryFilter, set_discovery_filter) and the BLE advertise/scan
mechanism vocabulary are unchanged, as is BleConfig. Behavior-neutral.
Rename the ethernet link-local neighbor-beacon subsystem from the
overloaded "discovery" vocabulary to a neighbor umbrella. The beacon
frame vocabulary is retained (build_beacon/parse_beacon/BEACON_SIZE/
FRAME_TYPE_BEACON/FRAME_TYPE_DATA); only the subsystem and buffer
identifiers change: DiscoveryBuffer becomes NeighborBuffer,
discovery_buffer becomes neighbor_buffer, and DISCOVERY_VERSION becomes
BEACON_VERSION (wire value 0x01 unchanged). Config toggles are left for
a follow-up commit. Behavior-neutral.
Brings the connected-UDP fast-path plane relocation (ConnectedPeerSocket/
PeerRecvDrain into peer/connected_udp; udp keeps open_connected_fd in io.rs)
and the darwin_sockopts -> sockopts_macos rename onto the next line. Clean
three-way auto-merge; behavior-neutral.
The module is gated target_os="macos" (not the broader Darwin/iOS family),
so the name now tracks the cfg and matches the *_macos.rs file convention
(io_macos.rs). Drops the redundant inner #![cfg(target_os="macos")] (the
decl gate already covers it) and corrects a stale top comment that claimed
the module stays visible on Linux — it is macos-decl-gated, so it never was.
Mechanical rename; no logic change.
ConnectedPeerSocket and PeerRecvDrain are node/peer-side logic: the Transport
trait never touches them, ActivePeer stores them, and node's encrypt worker
drives them. They only happened to live under transport/udp. Relocate the
handle types to a new src/peer/connected_udp/ module (socket.rs + drain.rs),
which the node handler and encrypt worker reach via node -> peer (no new edge;
a node home would have forced a peer -> node cycle).
The udp transport keeps only the kernel-construction seam: open_connected_fd,
now folded into udp/io.rs (the byte-layer home) behind a linux/macos-gated
submodule and re-exported as transport::udp::open_connected_fd. The node
handler builds the fd through it and adopts it via ConnectedPeerSocket::from_fd.
Behavior-neutral: no wire/config/metric/log change; the per-packet hot path
(bare-RawFd send_batch_gso/raw) is untouched. Preserves the Arc multi-owner
contract, the drop-drain-before-socket ordering, and the drain's detach-on-Drop
deadlock avoidance. Reconciles the old cfg(unix)/any(linux,macos) double-gate
onto the single any(linux,macos) predicate.
Pulls the connected-UDP socket construction (socket/REUSEADDR/REUSEPORT/
BUFFORCE/bind/connect + darwin tuning) out of ConnectedPeerSocket::open into
a pub(crate) open_connected_fd returning an OwnedFd; open() now delegates and
adopts the fd. Error paths still close the fd via the temporary's Drop; the
success tail transfers ownership through OwnedFd. Behavior identical. Prepares
the fast-path handle types to move to the peer module while the socket
construction stays with the udp transport.
Moves the inline TcpConnection/ConnectingEntry structs, the Direction
enum, and the ConnectionPool/ConnectingPool type aliases out of the
1147-line tcp/mod.rs into a dedicated pool.rs, matching the canonical
per-transport layout. Struct fields are pub(crate) so mod.rs can still
construct and read them across the module boundary. Pure relocation; no
logic change.
Moves parse_mac_string into ethernet/addr.rs and re-exports it at the
ethernet module root so the ethernet::parse_mac_string path stays
byte-identical for its one external consumer (zero churn there). Its
unit tests stay in mod.rs, calling through the re-export. Pure
relocation; no logic change.
Normalizes the ethernet transport onto the canonical byte-layer name
(io.rs), including the per-OS files io_linux.rs/io_macos.rs and their
#[path] wiring. Pure file rename plus module-path and doc-comment
updates; no logic, wire, config, metric, or log change.
Normalizes the udp transport onto the canonical per-transport module
layout (ble is the reference: io/pool/addr/stats). Pure file rename plus
module-path updates at the use sites; no logic, wire, config, metric, or
log change. Behavior identical.
The Nym transport was a near-clone of the Tor transport's outbound
SOCKS5 path. Extract the shared logic into src/transport/socks5/ so both
transports drive one implementation instead of two maintained copies:
- share one SOCKS5 mock server between the tor and nym tests
- extract the common send/receive/connect counters into a shared
ProxiedStatsBase (snapshot structs and emitted metrics unchanged)
- add a shared Socks5Dialer that collapses all six connect variants;
the sole dialing difference (tor's per-destination circuit-isolation
auth vs nym's no-auth) is an enum on the dialer
- route both tor and nym dialing through the shared dialer
- share the proxied connection pool, generic over a per-connection meta
type that carries tor's inbound/outbound direction counting (nym uses
the unit type)
- share the proxied receive loop
Behavior-neutral: no wire-format, config-key, emitted-metric, log, or
error-variant change. Tor keeps its control-port, inbound/onion, and
directory surface and its own address validation.
The FMP frame-boundary reader lived in transport/tcp/stream.rs, but the
Tor and Nym transports both reached across module boundaries to
'use crate::transport::tcp::stream::read_fmp_packet', a layering smell:
the reader is a shared stream-framing utility, not a TCP-private one.
Move the file to transport/framing.rs and repoint the tcp/tor/nym use
sites, removing the tor->tcp and nym->tcp dependencies. Behavior
unchanged.
TcpStats and TorStats each carried an identical pair of pool_inbound/
pool_outbound atomics plus the same five record_pool_* / pool_inbound_count
methods over them. Move the counter logic into a shared PoolCounters
struct (new transport/stats_common.rs) embedded as a 'pool' field in both.
The public record_pool_* methods stay as thin delegators so all call
sites and the flat pool_inbound/pool_outbound snapshot fields are
unchanged; only the duplicated atomic bookkeeping is now single-sourced.
EthernetStatsSnapshot derived only Clone/Debug/Default, unlike every
other transport's *StatsSnapshot which also derives Serialize. That gap
forced a hand-rolled serde_json::json!{} arm in TransportHandle::
transport_stats() that re-listed all ten fields by hand. Add the
Serialize derive and collapse the arm to the same one-line
serde_json::to_value(...) form the other transports use. The emitted
JSON keys and values are unchanged.
Measures the per-forwarded-packet cost of routing candidate assembly
(routing_candidates over a synthetic RoutingView) against a zero-alloc
reference across 8/32/128/256 peers, with per-call allocation counts via
a counting allocator. Criterion harness; no production code change.
Forward-merges the bloom SHA-256-once fix from maint. The bloom filter was
relocated to proto/bloom/ by the sans-IO refactor, so the fix applied cleanly
to proto/bloom/core.rs; the behavior-neutral test was re-homed into
proto/bloom/tests/core.rs (maint carried it in the pre-split bloom/tests.rs).
BloomFilter derived all k hash functions from one SHA-256 digest but
recomputed that digest inside the per-function loop, so every insert and
contains ran SHA-256 hash_count times (5x at the default) over the same
bytes. Hoist the digest out of the loop: base_hashes() computes it once
and returns (h1, h2), and bit_index() derives each of the k indices with
the same (h1 + k*h2) mod m arithmetic. Bit-for-bit identical output; this
is the hottest path in packet forwarding and mesh-size estimation.
Adds a test pinning the bit indices against the double-hashing formula
recomputed independently, proving the refactor is behavior-neutral.
Forward-merge the master-line FMP-establish relocations onto next: the FMP
link wire codec into proto/fmp/wire.rs and PromotionResult into
proto/fmp/core.rs. The wire move was hand-resolved against next's wire
format (msg3/Msg3Header/build_msg3, FMP_VERSION=1, no spin-bit flag), so
next's behavior is unchanged. Behavior-neutral; full lib suite green at
next baseline.
Move the PromotionResult enum (and its impl) out of peer::mod and into
proto/fmp/core.rs, alongside the cross_connection_winner tie-break helper
that was relocated the same way. This is FMP connection-lifecycle result
vocabulary, so it belongs in the FMP subsystem home rather than the peer
module.
Behavior-neutral pure type relocation: consumers import it from
crate::proto::fmp, and the crate-root public path crate::PromotionResult
is preserved via a re-export in lib.rs (mirroring cross_connection_winner).
Full lib suite green at baseline.
Move the FMP mesh-layer wire format (common prefix, encrypted/msg1/msg2
headers, and the build_*/inner-header codec fns) out of node/wire.rs and
into proto/fmp/wire.rs, so the whole FMP wire surface lives with its
subsystem, matching the proto/fsp/wire.rs layout. The wire module becomes
pub(crate) mod wire; callers reach it via crate::proto::fmp::wire.
Behavior-neutral: pure relocation plus import-path rewrites across the
node/peer consumers; no logic change. Full lib suite green at baseline.
Carve the nostr rendezvous engine's decision logic out of the async
NostrRendezvous driver into synchronous, clock-injected cores, following
the failure_state.rs pattern: sync state behind std::sync::Mutex, time
passed in as now_ms, no .await and no I/O in the core. The driver performs
all relay/socket I/O and executes the returned plans.
- AdvertMachine (src/nostr/advert.rs): advert publish/cache/fetch/prune
decision logic; the driver executes a returned PublishPlan.
- TraversalMachine (src/nostr/traversal_machine.rs): the engine-scoped
cross-session decision state -- in-flight-initiator dedup, the dual-init
responder suppression election, and the replay/seen-session cache.
- classify_punch_packet (src/nostr/traversal.rs): a pure classifier for
the punch recv loop's packet branch table.
The driver keeps all I/O, the NAT punch send cadence, the offer-slot
admission semaphore, and the pending-answer oneshot routing. Behavior-
neutral: no wire, config-key, or metric change. Adds unit coverage for
the advert plans, election ordering, replay eviction, initiator dedup,
and punch classification.
Forward-merge the src/nostr / src/mdns rendezvous reorganization
(relocation out of src/discovery/, the Discovery->Rendezvous rename, the
RendezvousDriver consolidation, and the trace-target reference updates)
onto the next branch. The sole conflict was in the LAN poll method's
doc-comment and signature: kept next's XX-handshake wording and applied
the lan_rendezvous rename. Merged tree builds clean; next lib suite 1583
passing, fmt/clippy clean.
Moving the nostr rendezvous engine to src/nostr/ (and mDNS to src/mdns/)
changed the module-path-derived tracing targets from
fips::discovery::nostr::* to fips::nostr::*. Update the RUST_LOG filters
in the NAT and mesh-lab test compose files and the resolve-peers-via-nostr
tutorial's debug recipe to the new targets so trace configs and the
documented journal-watch command keep emitting the intended lines.
Without this, the stun-faults suite's Phase-0 pre-flight (which greps the
daemon journal for the debug-level "STUN observation succeeded" /
"traversal: initiator STUN observed" lines) saw those lines suppressed —
the daemon behaved correctly, but the stale RUST_LOG target hid the
evidence. Test-harness and docs only; no source or behavior change.
Pull the rendezvous driver state and the movable driver logic out of the
Node struct into the src/nostr home. A new RendezvousDriver owns the
engine handle and the four bookkeeping fields (traversal start time,
startup-sweep latch, adopted bootstrap-transport set and their npubs)
that previously sat loose on Node; Node holds a single driver field and
reaches the same data through thin accessors, including the rx-loop
hot-path protocol-mismatch hook.
The advert build/refresh, the via_nostr fallback-address resolve, the
overlay-endpoint-to-PeerAddress mapping, and the bootstrap request move
onto the driver, taking their Node inputs explicitly (a transport-
endpoint snapshot for advert building) rather than reading Node fields
directly. Transport/connection-table-bound work (traversal adoption,
bootstrap-transport cleanup, the open-discovery sweep, and the outbound
budget calculators) stays on Node as thin glue that calls into the
driver.
Behavior-neutral relocation: statements moved verbatim, no logic, wire,
config-key, or metric changes. lifecycle.rs shrinks ~230 lines. cargo
fmt/build/clippy clean; lib suite 1547 passing (baseline unchanged).
Relocate the overlay peer-rendezvous subsystem out of the overloaded
src/discovery/ tree into two focused, independent homes: src/nostr/
(relay-mediated overlay endpoint advertise/resolve/auto-mesh plus NAT
traversal) and src/mdns/ (link-local DNS-SD rendezvous). The two
subsystems are independent, so they get separate homes rather than
sharing one.
Drop the ambiguous "Discovery" stem from their identifiers in favor of
"Rendezvous": NostrDiscovery -> NostrRendezvous, LanDiscovery ->
LanRendezvous, and the matching config, policy, field, and method names.
The former src/discovery.rs handoff types (EstablishedTraversal,
BootstrapHandoffResult, the punch-packet helpers) fold into
src/nostr/handoff and stay reachable via the crate-root re-exports.
Pure relocation and rename: no logic, wire-format, config-key, metric,
or tracing-target changes. The operator-facing node.rendezvous.nostr.*
and node.rendezvous.lan.* config keys and the fips-overlay-v1 advert
namespace are byte-identical. cargo fmt/build/clippy clean; lib test
suite 1547 passing (baseline unchanged).
Forward-merge the mesh-lookup "discovery" disambiguation onto the next
line: the lookup handler module and Node.lookup field renames, the lookup
metric types, the dual-emitted lookup/discovery metric family, and the
node.discovery -> node.lookup/node.rendezvous config split with its
deprecation shim.
Conflicts resolved by keeping next's XX handshake path (the inbound
responder stores pending_inbound for msg3 rather than promoting on msg1)
and renaming its reset_discovery_backoff call sites to reset_lookup_backoff;
the CHANGELOG Unreleased entries from both lines are combined.
The identifier "discovery" named three unrelated subsystems; the FMP
overlay coordinate-lookup subsystem is now consistently "lookup". This
finishes the concept-#1 rename across the shell, config, and metric
layers left after the earlier proto-layer rename:
- Handler module node::handlers::discovery -> node::handlers::lookup, and
reset_discovery_backoff -> reset_lookup_backoff.
- The Node lookup-engine field Node.discovery -> Node.lookup, renamed by
resolved binding so the metrics().discovery and node.discovery config
paths are left untouched.
- The lookup metric types DiscoveryMetrics -> LookupMetrics,
DiscoveryStatsSnapshot -> LookupStatsSnapshot, and Metrics.discovery ->
Metrics.lookup.
Two surfaces cross a stability boundary and ship behind a compatibility
window, both marked in-code for removal at the v2 cutover:
- The control-socket metric family is dual-emitted under both "discovery"
(deprecated alias) and "lookup" so existing dashboards keep working.
- The node.discovery.* config table is split into node.lookup.* (mesh
lookup scalars) and node.rendezvous.* (nostr/lan peer rendezvous).
NodeConfig does not deny unknown fields, so a naive rename would make a
deployed node.discovery: block deserialize into nothing and silently
revert every setting to default. A deprecated all-Option
DiscoveryConfigCompat field captures a legacy block and a new post-parse
Config::normalize_deprecated_keys pass folds it into the new tables with
a one-time deprecation warning.
Flip the packaged fips.yaml templates to the new keys, add legacy/new/
scalar compat parse tests, and record the split and deprecations in the
CHANGELOG. Behavior-neutral; fmt/clippy clean, lib suite green.
The previous portability fix bounded the powi test's drift from std::powi
at an absolute ULP count, but Windows/MSVC drift grows with the exponent
(3 ULP by exp=14), so no fixed ULP bound is portable.
Replace the ULP oracle with a loose relative-tolerance sanity sweep
(1e-11), which any sane libm clears by ~1000x regardless of exponent
while still catching a grossly wrong impl. Add a golden-bit pin for the
exp=14 case that exposed the growth. The powi function is unchanged; the
golden-bit determinism pins already passed on Windows.
The powi guard test asserted our square-and-multiply result was
bit-for-bit equal to f64::powi, which failed the Windows unit-test job by
one ULP. f64::powi is not portably bit-stable: Linux and macOS lower it
to compiler-rt's __powidf2, but Windows/MSVC rounds differently.
Our square-and-multiply is pure IEEE-754 f64 multiplication and is
therefore deterministic across every platform, which is the property that
actually matters for mesh nodes to agree on bloom FPR and backoff timing
regardless of OS. Replace the std bit-equality assertion with golden-bit
pins on our own output (locking that cross-platform determinism) plus a
bounded-drift sanity sweep against std::powi. The powi function itself is
unchanged.
Forward-merge the mesh discovery module rename (directory, exported types, and
internal lookup-stem terminology) onto the next wire-format line. next's
divergent lookup wire and core content is preserved under the new proto/lookup
path, with the same terminology sweep applied to its next-only lines.
Rename src/proto/discovery to src/proto/lookup and bring the module's naming
onto the lookup stem, matching its concept: a mesh lookup of a node's
coordinates from its pubkey, sent into the mesh as a bloom-filter-guided
multicast request that returns a unicast response with the coordinates.
- the module directory and its declaration (proto::discovery -> proto::lookup)
- exported types: DiscoveryAction -> LookupAction, DiscoveryBackoff ->
LookupBackoff, DiscoveryForwardRateLimiter -> LookupForwardRateLimiter,
MAX_RECENT_DISCOVERY_REQUESTS -> MAX_RECENT_LOOKUP_REQUESTS, and the Discovery
state struct -> Lookup
- internal terminology: doc comments, the "overlay-lookup" phrasing, the
empty_discovery/suppressing_discovery test helpers, and the disc
parameter/variable all take the lookup names
- the already-lookup-named wire types (LookupRequest/LookupResponse) are unchanged
Behavior-neutral: no wire bytes or decision logic change. Two references are
intentionally kept as "discovery": the still-named node::handlers::discovery
shell module and the node.discovery.* config keys, which belong to the broader
disambiguation of the shell, config, and metric surfaces still to come.
Bring the proto tree closer to a std+alloc-only posture, behavior unchanged on
every decision path:
- Sweep the remaining std::fmt and std::collections imports over to core::fmt
and alloc::collections across the wire codecs and their tests. Subsystem files
that shadow the core name with a child core module use the leading-colon
::core::fmt form. Imports only.
- Replace the two f64::powi calls (bloom false-positive-rate, FMP backoff timer)
with a shared core-only square-and-multiply helper in proto/math, bit-identical
to std::powi (a guard test pins this against every exponent the codecs reach),
and route the diagnostic estimated-count natural log through libm::log. The FPR
reject decision and the backoff timer are bit-for-bit unchanged; only the
debug-only count estimate may differ by at most one ULP.
Introduce a shared proto/codec module with a cursor Reader (short reads fail with
MessageTooShort { expected: position + needed, got: total }) and an append Writer,
and adopt them across the seven subsystem wire codecs, replacing the repetitive
manual slicing, try_into, and from_le_bytes extraction. Each existing length check
maps to the reader (an up-front minimum becomes require at position zero, so the
per-field expected values, including the tree-announce expected 99, are reproduced
exactly); the bloom exact-length check stays explicit since it also rejects
over-long payloads. Encoded bytes and decode decisions are unchanged.
Replace Malformed(String) with a no_std-clean set: Malformed(&static str)
for the static decode diagnostics, BadSizeClass { got, max } for the two
bloom size-class checks, and typed sources BadCoord(CoordError) /
BadBloom(BloomError) for the coordinate and bloom construction failures. Add a
dedicated CoordError so proto/coord no longer depends upward on stp TreeError,
resolving the temporary inversion from the coordinate relocation. Drop the
thiserror derive from the four proto error types (Error, TreeError, BloomError,
CoordError) in favor of hand-rolled core::fmt::Display and core::error::Error
impls. thiserror remains in use elsewhere in the crate. Wire decode decisions
are unchanged; only the error type and its diagnostic text change.
Hoist the two line-for-line-identical per-destination minimum-interval limiters
(discovery forward, routing error) into a shared PerAddrRateLimiter, and fold
the duplicated exponential-backoff math (discovery originator, FMP retry) into a
shared backoff_ms helper, both in a new proto/rate_limit module. The subsystem
limiters become thin delegating newtypes so should_forward/should_send and the
backoff call conventions are preserved. Behavior is byte-identical.
Two behavior-neutral relocations, wire bytes unchanged:
- Move the ParentDeclaration type and its impls out of state.rs into a dedicated
declaration.rs, re-exported at the same path, and relocate the inline wire.rs
test module into tests/wire.rs alongside the other stp unit tests (reusing the
shared test helpers). wire.rs drops to non-test code only.
- Move TreeCoordinate, CoordEntry, and the coordinate wire codec out of
proto/stp into a shared proto/coord module (a peer of proto/link), re-exported
from proto::stp so every existing import path keeps resolving unchanged.
Coordinate is a shared addressing primitive with many non-stp consumers, so it
no longer belongs under the spanning-tree subsystem. The codec carries a
documented temporary dependency on stp::TreeError until a dedicated CoordError
is introduced.
evaluate_parent no longer reads the injected clock: it returns a ParentEval of
Mandatory, Discretionary, or None, and the flap/hold-down veto is applied at the
edge via a new TreeState::is_switch_suppressed(now_ms), gating only the
discretionary arm. classify_announce/classify_periodic take the pre-computed
switch_suppressed bool instead of now_ms, so the whole classify ladder is
clock-free; the shell callers (tree announce/periodic re-eval, MMP first-RTT
re-eval, handle_parent_lost) compute the veto verdict. The no-coords parent
case stays discretionary (veto-gated) exactly as before. Behavior unchanged;
the veto tests now assert the moved responsibility.
Reorganize the MMP subsystem, behavior unchanged throughout:
- Restore the pre-migration role split: move SenderState into sender.rs,
ReceiverState (with its GapTracker helper) into receiver.rs, MmpMetrics and
RrLog into metrics.rs, and PathMtuState into path_mtu.rs, leaving the Mmp
aggregate plus the peer/session state in state.rs (1339 to 196 lines). Home
the module constants in a new limits.rs and re-export them at the same paths.
Pure code-motion with module rewiring; visibility unchanged.
- Dissolve the 97-line src/mmp shell, which held only MmpConfig and the
monotonic mono_ms() clock: move MmpConfig into config/node.rs (re-exported as
crate::config::MmpConfig), move mono_ms() into a new top-level src/time.rs
shell time seam, repoint all callers, and delete src/mmp/. Also correct stale
src/mmp/ doc-comment path labels to proto/mmp/ where they name the protocol
primitives.
Two behavior-neutral discovery cleanups:
- Move MAX_RECENT_DISCOVERY_REQUESTS out of the node discovery handler into the
discovery subsystem limits module and re-export it, keeping the two use sites
unchanged. Same value and semantics; only its home moves.
- Remove the ambient rand draw from LookupRequest by deleting generate() and
having the shell draw the random request_id and pass it into the existing
new() constructor. Same per-request u64 draw, now at the shell, leaving the
discovery codec free of ambient RNG reads.
The false-positive-rate primitive (fill ratio raised to the hash count) was
inlined at three sites. Hoist it into a BloomFilter::fpr() method and call it
from all three; the threshold/policy logic stays at the call sites. Behavior
and result bytes are unchanged.
Forward-merge the sans-IO cleanup series (bloom fpr, discovery const/RNG-injection,
mmp state decomposition, mmp shell dissolution, STP clock-free classify core, STP
declaration split, shared proto/coord relocation, shared rate limiter/backoff,
typed proto errors dropping thiserror, shared bounds-checked codec reader/writer,
core/alloc import sweep, no_std-shaped filter math) onto the next wire-format line.
Reconciled master structure against the next wire semantics: the mmp role-module
split adopts next slim report format (spin-bit stays dropped, sender/receiver
build next reports), the discovery and fmp codecs keep next TLV and profile
negotiation while moving to the typed error, and the parent-eval handler keeps the
Full/Leaf profile filter under the new ParentEval/is_switch_suppressed seam.
Forward-merge the FSP session-protocol sans-IO migration and the src/protocol
teardown from the master-refactor line onto next. Converge proto/fsp across
both lines (core.rs byte-identical), keeping next's born-on-next residue: the
additive SessionSetup/SessionAck wire variants, the spin-bit-less unit
FspInnerFlags, and the XX handshake shell residue (negotiation-payload
piggyback, initiator identity check, min-size length checks) in
handlers/session.rs. src/protocol is removed on next as well; ProtocolError,
LinkMessageType, and SessionDatagram reach their proto/ homes.
Migrate the FSP end-to-end session subsystem into src/proto/fsp/ following the
established sans-IO shape, and retire the src/protocol grab-bag now that FSP was
its last occupant.
Relocate the FSP session wire (node/session_wire.rs plus the FSP message types
from protocol/session.rs) into proto/fsp/wire.rs. Hoist the pure decision logic
into proto/fsp/core.rs over plain-data SessionSnapshots returning an ordered
FspAction list the shell drives: session-rekey policy, msg3-resend
classification, post-decrypt epoch reaction, setup/dual-init tie-break,
coords/path-MTU emit-policy, bounded pending-queue, and IPv6 ECN. The
crypto-owning SessionEntry stays shell-side in node/session.rs (matching the FMP
ActivePeer pattern); proto/fsp is wire + core + limits only, with no proto->noise
dependency and no crypto.
Move the coords helpers to proto/stp/ (they serialize TreeCoordinate), and split
SessionMessageType: the encrypted-inner 0x10-0x1F variants stay in proto/fsp/wire.rs
while the 0x20-0x2F routing signals become a new RoutingSignalType in
proto/routing/wire.rs. Migrate the session-MMP shell adapter, which continues to
drive proto/mmp/.
Retire src/protocol: LinkMessageType and SessionDatagram move to a new shared
proto/link.rs, ProtocolError becomes proto::Error (relocated verbatim), the
deprecated MessageType alias and the unimported PROTOCOL_VERSION are dropped, and
src/protocol/ is deleted along with its lib.rs module declaration.
Behavior-neutral: wire bytes unchanged, oracle tests pass unedited except
mod-path relocation; adds rekey/epoch characterization tests and pure
poll/emit-policy core tests.
Forward-merge the master-line bloom relocation into the next line,
discarding next's v1.5 bloom draft (RLE codec, XOR-diff delta,
FilterNack/0x21, adaptive sizing) and converging both lines onto the
identical proto/bloom v1-sans-IO module. This is a deliberate,
temporary wire regression on the next line: the v2 bloom is rebuilt
fresh, sans-IO from day one, on both lines later. Nothing outside the
bloom feature depended on the v1.5-specific surface. proto/bloom is now
byte-identical across both integration lines.
Migrate the v1 bloom filter subsystem into src/proto/bloom/, matching the
discovery/routing/fmp/mmp/stp reference layout and completing the proto/
relocation series for the data/wire subsystems.
- wire.rs: the FilterAnnounce (0x20) codec, moved from protocol/filter.rs
- core.rs: the pure BloomFilter algorithm (hash/insert/contains/merge/
as_bytes/from_bytes/estimated_count), moved from bloom/filter.rs
- state.rs: BloomState (per-peer inbound store, compute_outgoing_filter,
the injected-clock send debounce), moved from bloom/state.rs
- limits.rs: the v1 sizing constants
- mod.rs: module wiring + the BloomError enum
- tests/: the unit suite split by target (core/state/wire), no inline tests
no_std+alloc hygiene: core::fmt over std::fmt, the tracing dependency
dropped from the pure filter, and std collections replaced with
BTreeMap/BTreeSet (NodeAddr: Ord) for deterministic iteration. The pure
filter combination stays a BloomState method; the two irreducible shell
gathers (peer_inbound_filters, build_filter_announce) remain in the async
shell. Wire bytes and observable behavior are unchanged; full local CI
green (36/36) including the bloom-storm chaos gate.
Forward-merge the STP sans-IO migration. The classify / proto-stp structure,
clock injection, crypto field-partition, and BTree collections come from the
master line; next's non-full/leaf parent-candidacy skip is preserved by passing
self.non_full_peers() at the four shell call sites (handle_tree_announce,
check_periodic_parent_reeval, the MMP first-RTT re-eval, and the greedy-tree
find_next_hop fallback) instead of an empty set, with non_full_peers returning a
BTreeSet. next's four non_full_peers skip tests fold into proto/stp/tests.
Migrate the full non-async spanning-tree surface into proto/stp/, mirroring the
discovery/routing/fmp/mmp conversions. The classification ladder (parent-switch /
self-root / loop-drop / ancestry-update / periodic-rebroadcast / parent-lost) moves
out of the async node handlers into a pure Stp classify layer returning a
TreeDecision the shell drives, with effect ordering and per-arm invalidation
preserved verbatim. src/tree/ relocates wholesale: TreeState + ParentDeclaration data
+ coordinates into proto/stp/{state,coordinate}, the flap-dampening / hold-down
cluster into a FlapDampener in limits.rs, and the wire codec into wire.rs. The clock
is injected as u64 (wall-clock secs for the escaping declaration timestamp, monotonic
ms for the dampening timers via mmp::mono_ms); declaration crypto is field-partitioned
so sign/verify/hash run in the shell while the in-core modules carry data +
signing_bytes only. Peer maps/sets move to BTree; core/state/coordinate/limits are
core+alloc clean, with wire.rs the one std-tethered file. Behavior-neutral:
characterization tests added for the handler decision arms; convergence suite and
ci-local (36/36) green.
Forward-merge the master-side sans-IO refactor (discovery migration + no_std
reductions, collapsed to one commit) into the next-side branch. The discovery
decision core meets next's FMP profile/LookupRequest delta here: next's four
transit-forward predicates (Leaf no-forward, Full-profile, min_mtu, tree/fallback)
are folded into the pure core planners via an extended RoutingView seam
(node_is_leaf / peer_is_full / peer_meets_mtu, new ForwardOutcome::LeafNoForward),
and next's v2 LookupRequest API delta (origin_coords removed, tlv_entries added)
is reconciled across the discovery test tree, with next's four TLV wire tests
ported into the relocated wire test module.
The master-side branch now also carries the no_std+alloc reductions (BTreeMap,
alloc::sync::Arc, backoff-reset log moved to the shell, extern crate alloc),
which come through cleanly on the pilot-only core files.
Validated: cargo fmt / clippy (-D warnings) / test --lib all green, including
next's own transit MTU-pruning integration tests driving the refactored core
through the full shell path.
Migrate the FMP discovery decision logic out of the async handlers into
synchronous, runtime-agnostic sans-IO state machines owned by the protocol
structs, with I/O pushed to the edges. Pulls the full decision surface into a
pure core (backoff, rate-limit, planners, response routing), consolidates the
tests into a per-module tree with a shared crate testutil, and injects a u64
wall-clock so the core is free of Instant and std time.
Also brings the module toward no_std+alloc: the four discovery maps use
alloc::collections::BTreeMap (HashMap's RandomState is std-only), Arc is spelled
alloc::sync::Arc, the backoff-reset log lives in the shell (the core returns the
cleared count so observability stays out of the pure core), and the crate root
names alloc directly. The one remaining tether is ProtocolError's
std::error::Error coupling in the wire codec.
First subsystem of the broader sans-IO refactor; establishes the extraction
patterns and conventions carried forward to the remaining protocols.
handle_peer_removal_tree_cleanup reparents or self-roots the node when
its parent link drops, but omitted the coordinate-cache invalidation that
every other position-change path performs. Cached entries for downstream
destinations kept the node's now-stale coordinate prefix, and because
find_next_hop refreshes the TTL on every routing access, an actively
routed stale entry never self-expired — corrected only by a fresh insert.
Mirror the loop-detection branch: inside the parent-loss changed block,
invalidate both classes — invalidate_via_node (reparent) and
invalidate_other_roots (self-root). Add regression tests for a parent-link
removal that reparents (via-node entry dropped, same-root sibling
preserved) and one that self-roots (both via-node and stale old-root
entries dropped).
Add a "Running CI locally" section to testing/README.md covering the
ci-local.sh orchestrator: the per-run FIPS_CI_RUN_ID override and how it
scopes compose projects, image tags, and per-chaos-child subnets so
simultaneous runs on one host never collide; the cancellation exit codes
(130/143 vs 0/1) and how a preempting worker interprets them; and the
label-based cleanup via --reap / ci-cleanup.sh.
A CI worker may preempt an in-flight ci-local.sh run (SIGTERM, then SIGKILL
after a grace period) to restart on a newer commit. For that kill to be safe,
the script must clean up after itself and never let a dying run collide with
its restart. It previously had no signal handling, shared the default compose
project name across runs, and tore down each suite only at the suite end.
- Derive a per-run id (honoring FIPS_CI_RUN_ID, else short-sha+random) and
namespace every docker resource to it: a fipsci_<run>_<suite> compose project
per suite and per parallel chaos child, and per-run image tags
(fips-test:<run>, fips-test-app:<run>) retagged to :latest only after both
builds succeed so :latest never points at a half-built image.
- Install a bounded, idempotent teardown trap on SIGTERM/SIGINT (+ EXIT): reap
parallel chaos children, then force-remove this run's docker resources via
the new ci-cleanup.sh, wrapped in timeout so a stuck down cannot wedge it.
- Exit 143 (SIGTERM) / 130 (SIGINT), distinct from 0 (pass) / 1 (failed), so a
preempting worker tells a cancelled run from a real failure.
- Add ci-cleanup.sh (also ci-local.sh --reap): force-removes leftover CI
resources by the com.corganlabs.fips-ci=1 label and the fipsci_ project
prefix, robust to however a prior run died.
- Label every per-suite docker resource so the label sweep reaps it after a
SIGKILL regardless of network name: direct docker run/network resources, the
sidecar compose services, and every per-suite compose network (acl-allowlist,
boringtun, firewall, nat, static, both tor suites, and the chaos generator
template). Parametrize the static/sidecar compose image refs so the per-run
tags are honored.
- Give each parallel chaos child a unique /24 from 10.30.x (a new --subnet
override on the sim CLI, assigned per-child in ci-local.sh) so parallel
children never collide on a shared docker subnet, and a chaos net can never
span a fixed-subnet suite (sidecar/static in 172.20.x). 10.30.x sits outside
docker's default-address-pool range, so an auto-assigned net cannot land on
it either; node IPs derive from the subnet, so no scenario config changes.
Replace the fixed post-first-rekey and post-second-rekey settle sleeps in
the rekey integration test, and the fixed-deadline baseline wait in the
interop test, with the deterministic wait_until_connected progress-aware
polling helper already used for the initial baseline convergence.
Each converted site fails fast with structured diagnostics when the mesh
is stuck and extends its deadline only while pairwise reachability is
still climbing, removing the wall-clock settle windows that produced
intermittent connectivity failures after a rekey.
Brings the v0.4.0 release line up to date on next without disturbing
the in-flight wire-format work; next keeps its own 0.6.0-dev version.
Linkage merge so later forward-merges land cleanly.
maint carries only its 0.4.1-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
Forward-merge the v0.4.0 content finalize from master. The only master
commit merged touches the CHANGELOG, release notes, and README badge.
CHANGELOG reconciliation: adopt the regrouped, stamped [0.4.0] release
section verbatim from master (a frozen release record, identical across
branches), keep next's top-of-file Breaking block, and retain next's
XX-handshake / XX-rekey items in [Unreleased] as 0.5.0 work (the XX path
does not exist on master, which is IK). Release notes adopt master's
refreshed versions. README stays at v0.5.0-dev on next (master's badge
bump to v0.4.0 is master-line only).
Claude-Session: https://claude.ai/code/session_01A2pYfSypNmmG4HyHwZuLex
Forward-merge the v0.4.0 pre-release content (dependency refresh, CI
deb-install + AUR-build legs, docs refresh, doc-comment fixes, and the
Phase 4 source-content squash: CHANGELOG, release notes, README,
fips.yaml, Cargo.toml metadata, reference docs) up the one-way flow.
Conflict fixups (keep next's identity, fold in master's improvements):
- Version: keep next's 0.5.0-dev (Cargo.toml/lock).
- README status: keep next's v0.5.0-dev / wire-format-breaking framing
and the Breaking-section pointer; fold in the Nym transport and the
"global, public test mesh of thousands of nodes" description.
- docs/reference/cli-fips.md: keep next's 0.5.0-dev version example.
- CHANGELOG: keep next's XX-handshake admission entry (no early cap gate
on XX) and drop master's IK early-cap-at-handle_msg1 entry, which
describes IK-only behavior that does not apply on next; take master's
OR-union mesh-size rewrite (next carries the OR-union code); restore
the Tor connect_refused and MMP receiver-report entries that the
Fixed-section consolidation would otherwise have dropped.
- lifecycle.rs: keep the mDNS/LAN handshake doc-comment as Noise XX
(next unifies on XX), not master's XX-to-IK correction.
Quartet green on the merged tree: fmt, build, clippy -D warnings, and
cargo test --lib (1434 passed).
Carries the Nym mixnet transport + single-container demo.
Forward-merge adaptation: the Nym end-to-end SOCKS5 send/recv test was
authored against master's IK FMP framing (msg1 wire size 114, version
nibble 0). next uses the XX handshake and FMP v1, so the TCP read path
rejected the test frame (UnknownVersion / HandshakeSizeMismatch),
timing the test out. Adapted build_msg1_frame to next's framing (wire
size 41, ver=1/phase=1 header byte). next-only; master keeps its IK
form.
Forward-merge the off-loop control read-snapshot plane (every show_* read
query served from published snapshots; show_* removed from the rx_loop).
Adapt the per-entity MMP projection to next's MMP model, which no longer
carries a spin bit: drop the spin_bit_initiator row field, its tick-side
population, and the spin_bit_role output, so the off-loop show_mmp render
stays byte-identical to next's metrics-only on-loop handler.
Brings the OR-union mesh-size estimator fix and the per-peer / capacity-cap
log-level demotions forward to next.
Conflict resolution: kept next's XX handshake structure and re-applied the
intent at next's own sites. compute_mesh_size auto-merged (next already
carries the config() accessor). The K-bit cutover promotion log demotion
applied to next's richer index-bearing form in encrypted.rs.
master's log changes lived in handle_msg1's IK-only identity/promotion
blocks, which have no XX equivalent there (next defers identity and
promotion to handle_msg3). next's equivalents were demoted in place:
"Connection promoted to active peer" and "Peer restart detected (epoch
mismatch)" info -> debug, and the duplicate "Inbound peer promoted to
active" removed. The two XX-only restart variants ("during FMP rekey" and
"during promotion") were also demoted info -> debug so the restart-detected
class is uniformly at debug on next.
Two next-line entries had been left describing states that a later commit
reverted. Collapse the XX max_peers early-gate add/remove churn into its
net final state: on the XX handshake an over-cap inbound connection is
rejected solely by the late promotion check (no early gate), logged at
debug. Replace the stale "rekey jitter disabled on next" note with the
actual state: jitter is re-enabled (REKEY_JITTER_SECS = 15) and the three
XX rekey-path divergence defects that had blocked it are fixed
(authenticated-decrypt cutover instead of bare K-bit, msg3 retransmission
with per-link rekey serialization, and a jitter-aware handle_msg3
session-age partition). Merged-in IK-line entries are left untouched.
At the inbound peer cap the XX FMP handshake had two cap checks: an early
gate in handle_msg3 and the late promote_connection check. On XX the
peer's identity is not known until the third handshake message, by which
point all three messages have already crossed the wire, so the early gate
saved no wire bytes and governed exactly the same net-new-peer set as the
late check (known and pending-outbound peers return earlier via the
cross-connection paths). Remove the early gate and rely on the promotion
check, and log the resulting MaxPeersExceeded rejection at debug instead
of warn so a cap'd node under sustained inbound pressure does not emit
WARN spam for expected policy rejections.
Re-enable the two previously-ignored inbound-cap unit tests, rewritten to
drive a full XX three-message handshake and assert the path-agnostic
invariant (no promotion over cap, peer count unchanged, no
connection/link/index leak) plus the known-peer reconnect bypass.
Update the admission-cap integration suite's enforcement-event
discriminator to match: drop the removed early-gate string and count the
new debug-level promotion-cap rejection.
Re-enables the rekey timer jitter on the XX FMP rekey path
(REKEY_JITTER_SECS 0 -> 15), which had been disabled because it produced
reproducible post-rekey routing loss (~50% Phase-5 ping failure) with no
crypto errors. The failures were session divergence: under jitter the two
directions of a link rekey close together in time, and three distinct
defects in the FMP rekey state machine could leave the two endpoints
committed to different Noise sessions, starving the receiver until the 30s
heartbeat dead-timer tore the link down (tree parent loss -> routing
failure). All three are fixed here.
1. Promote on authenticated decrypt, not the bare K-bit. The K-bit-flip
handler promoted whatever pending session existed the instant the header
bit flipped; under interleaved rekeys that could be a stale pending from
an earlier epoch. Trial-decrypt the inbound frame against the pending
session and promote only if it authenticates, mirroring the FSP cutover
discipline; deliver that plaintext through the canonical path and leave
the pending untouched otherwise.
2. Retransmit FMP rekey msg3 until confirmed. FMP sent msg3 once; a lost
datagram left the responder without the new session. Retain the msg3
payload and resend over the existing link until a peer frame
authenticates against the pending or post-cutover current session,
abandoning after the configured handshake-resend budget (mirrors FSP).
Also serialize per-link rekeys: do not start a new rekey while one awaits
cutover or is still retransmitting msg3.
3. Partition the handle_msg3 paths by rekey age. An inbound msg3 on a
different link took the initial-handshake cross-connection tie-breaker
when the session was under a fixed 30s old, otherwise the rekey
responder. A rekey resets the session-age clock, so under jitter a
rekey-aged session is frequently under 30s and its concurrent rekey msg3
was swallowed by the cross-connection branch, which discarded the peer's
rekey session with no pending slot while the peer cut over to it anyway.
Bound the cross-connection branch by the same jitter-aware age floor the
responder uses, so the two paths partition with no overlap.
Verified at jitter=15: rekey integration suite 70/70 across repeated runs
locally and on GitHub CI, rekey-accept-off 71/71, rekey-outbound-only
75/75; lib 1369/0, clippy and fmt clean. At zero jitter the acceptance
floor equals the previous 30s constant, so default-cadence behavior is
unchanged.
Bring the immutable-state single-store change onto the Noise XX line.
The shared NodeContext is now the sole store; this merge applies the
next-only adaptations the master-side change couldn't carry:
- Remove node_profile from the Node struct (next-only field) so it lives
solely in NodeContext; migrate its readers (tree/bloom/discovery/
handshake negotiation) onto the node_profile() accessor. The two FMP
negotiation sites hoist node_profile() into a local to avoid borrowing
&self while a connection is mutably borrowed.
- leaf_only sets both is_leaf_only and node_profile via the context swap.
- Preserve the XX handshake/rekey structure (no identity-in-msg1; XX/XK
initiator/responder constructors) while applying the startup_epoch()
accessor migration.
- Tests: route profile selection through a make_test_node_with_profile
helper (profile is immutable, set via Config flags) instead of poking
the removed field.
cargo test --lib 1369/0; clippy -D warnings and release build clean.
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, node profile, capability limits).
Source the immutable config and identity reads across the receive hot
path, the XX handshake/session/rekey state machines, and the discovery,
tree, bloom, retry, and lifecycle modules through the context accessors.
Includes the bloom delta/full/NACK/resize and byte-total counters in the
registry. The Node fields and the context are rebuilt in lockstep at every
mutation site.
Bring master's runtime peer-list refresh, opt-in mDNS LAN discovery, and
the receive-path reject-reason / reloadable-config refactor into the XX
handshake integration branch. The peer-restart-epoch detection and
stale-FSP-session teardown authored against Noise IK are re-authored onto
the Noise XX rekey path: complete_rekey_msg2 now also surfaces the remote
startup epoch, and the stale session is cleared once the XX rekey
completes (after msg3 is sent). Connect-budget and path-refresh work is
threaded through the XX anonymous-discovery branch.
The admission-cap suite asserted IK wire sizes (inbound Msg1 len 84,
outbound Msg2 len 104) to verify the inbound max_peers cap. On the XX
handshake those sizes differ and Msg2 is always sent before identity is
known, so the IK discriminator matched nothing and the suite failed
despite the cap working correctly.
Replace the wire-size discriminator with a protocol-agnostic cap-holds
invariant: each denied peer must keep re-initiating handshakes
(sustained inbound, size-agnostic), no denied peer may be promoted to an
active session, the node must log cap-enforcement events, and the capped
node must hold exactly max_peers. Raise node-c's handshake logging to
debug in the mesh profile so the early-gate drop is visible alongside the
late-check rejection.
Also record in the changelog that for peers the node also dials, the cap
is enforced by the retained late check rather than the early msg3 drop.
Adapt the typed RejectReason coverage to the Noise XX handshake: wire the
msg1/msg2/msg3 state-machine rejection sites in handlers/handshake.rs and
the rekey-initiator outbound sites in handlers/rekey.rs, and add the
XX-specific HandshakeStats counters. The shared RejectReason scaffold and
the branch-agnostic clusters come from the merged refactor-hotpath work;
this commit carries only the next-side delta.
Drop an internal tracker reference from the doc-comments of the
should_admit_msg1 and udp.outbound_only rekey regression tests; the
prose now describes the scenario directly.
The msg3-processing-failure cleanup read our_index from the connection
after removing it from the connections map, so the lookup always returned
None and the allocated index was never released, slowly leaking index
slots across failed XX handshakes. Capture the index before the remove,
matching the idiom used in the other cleanup paths in this file.
The XX leaf-nonrouting rejection test shares the localhost-UDP rcvbuf
overflow failure mode of the other large-network tests under parallel
cargo test --lib load. Mark it #[ignore] for the default run; it stays
runnable with --ignored or --test-threads=1.
Carries the inbound max_peers admission-cap silent-drop work from
master across the IK→XX structural boundary.
On master (Noise IK), the gate fires at handle_msg1 after the peer's
identity is extracted from the IK msg1 payload, before Msg2 is built
and sent. Wire savings of Msg2 (~104 B) plus the responder crypto
per cap-denied attempt motivate the gate placement.
On next (Noise XX), peer identity is not learned until msg3, by
which point Msg1, Msg2, and Msg3 have all crossed the wire. The IK
gate's wire-savings rationale does not apply on XX. The equivalent
gate is placed in handle_msg3 after the peer's static-key + signature
verification (identity now known) and before promote_connection
(ActivePeer construction, peers_by_index insert, link transition).
The value on XX is local CPU / allocation savings and cleaner peer-
side semantics: no fake-promotion whose subsequent data frames fail
decryption on this side, replaced by a silent drop the peer's
existing auto-reconnect backoff handles cleanly.
Bypass logic is the same as the IK version: known-active peers
(reconnect / cross-connection / restart) and pending-outbound peers
skip the cap so admitting them doesn't grow peers.len(). The late
cap check inside promote_connection is intentionally retained as
defense-in-depth.
Cleanup ordering for the gate's silent-drop path corrects a latent
issue in the adjacent msg3-processing-failure path: our_index is
captured before the connection is removed so the allocator slot is
actually freed (the pre-existing path's get-after-remove returns
None, leaking the index).
The two IK-shaped unit tests at handle_msg1 are marked #[ignore] on
this branch with reason. Their wire-bytes discriminator ("Msg2 must
NOT be sent at cap") is structurally false on XX, and the XX-shaped
equivalents need full three-message-handshake test scaffolding that
isn't authored here. XX-adapted unit + integration test coverage for
the handle_msg3 gate is a follow-up.
CHANGELOG [Unreleased] entry adjusted to describe the handle_msg3
placement on this branch (preserving the master-side framing on the
mainline-merge half of the entry would have been misleading on this
branch).
Three conflicts auto-resolved: the structural handshake.rs span
(~253 lines, IK identity-tied logic discarded on this branch's XX
msg1 path), and three textual ci-local.sh conflicts (admission-cap
suite list / dispatch added alongside the existing mixed-profile
suite list / dispatch).
Forward-merges PR #91 (off-task encrypt/decrypt worker pools, GSO,
connected UDP), the macOS #102 package-integrity fix, and the AUR
#98 fips-dns fix from master.
Six textual conflicts resolved: wire/noise imports, the K-bit-flip
and rekey-cutover handlers, ActivePeer fields, and an advert test
fixture. One semantic conflict: PR #91's worker-pool code on master
referenced FLAG_SP and MmpPeerState.spin_bit, which next removed in
4aded9a2 (spin bit superseded by the MMP receiver-report timestamp
echo; FMP flags bit 2 reclaimed). The worker acceleration and its
fmp_flags round-trip test are kept; the spin-bit references are
dropped to honor next's wire-format change.
Set `REKEY_JITTER_SECS` to 0 on next. The symmetric per-session
jitter mechanism added on maint/master works cleanly on the IK
FMP rekey path on those branches, but on next's XX FMP rekey
path it produces reproducible post-cutover routing-convergence
failures: ~50% Phase 5 per-pair-ping loss in the rekey integration
suite across multiple runs, with all crypto, transport, and
link-state log assertions still green (no decrypt failures, no
panics, no link teardowns, no rekey msg2 failures, no FSP
decryption failures during rekey).
The XX rekey path itself is sound — pre-merge it had been passing
18/18 across six consecutive CI runs. Isolation experiment
confirms the trigger: with REKEY_JITTER_SECS=0, the same suite
on next passes 70/70 cleanly with FMP cutover counts (18-21)
in line with master's baseline (24). With the constant at 15
and the test suite's 35s nominal rekey interval, FMP cutover
counts climb to 27-40 and Phase 5 fails consistently.
Two attempted fixes to absorb the jitter-induced state churn on
the XX path (post-msg3 addr_to_link restore; optimistic dampening
at msg1 receipt) were tried and reverted — neither restored
Phase 5 reliability. Diagnosis of the underlying XX cutover
state-cleanup behaviour that fails to absorb variable-interval
rekeys is deferred. The const and surrounding draw/redraw
machinery (peer::active::ActivePeer, node::session::SessionEntry,
draw_rekey_jitter at session.rs:20, redraws at the four cutover
sites) are kept in place; flipping the const back to a positive
value is the re-enable path once the investigation lands.
CHANGELOG entry under `### Changed`.
The two rekey-jitter unit tests added on the IK-rekey branches
construct a HandshakeState via:
HandshakeState::new_initiator(keypair, remote_pubkey)
On next, the XX rewrite changed the initiator constructor to take
only the local keypair:
HandshakeState::new_initiator(keypair)
XX does not learn the responder's static pubkey until msg2, so
the IK-shaped signature does not apply. The merge from master
brought the test bodies verbatim; this drops the second argument
in both call sites so the tests compile against the XX API. The
test semantics (rekey-jitter range and mean checks) are unchanged
and orthogonal to the handshake-pattern choice.
Same shape as 8a6477b adapting the bootstrap-handoff tests to
XX's three-way handshake.
The `mixed-profile` integration suite (Full / NonRouting / Leaf
node-role mix introduced by the forklift-upgrade work) is wired
into `testing/ci-local.sh` but was silently absent from the GitHub
Actions integration matrix on next. Regressions to node-role
behavior could land via GitHub CI without the suite running.
Adds a matrix entry plus the matching step block (mirroring the
rekey pattern: generate configs + inject config + compose up +
test + log-collect on failure + compose down always). master
correctly omits the suite from both ci-local and ci.yml since
mixed-profile is forklift-only, so this change never forward-merges
to master.
The dual-initiation tie-breaker in the rekey arm of handle_msg3 (and
the FSP analogue in handle_session_setup) only fired when
rekey_in_progress was true. With Noise IK (one-message rekey on
master/maint) that is sufficient: msg1 IS the rekey, so both sides
being mid-handshake is the only state where the race can fire.
With Noise XX (three-message rekey on next), set_pending_session runs
when the initiator has processed msg2 and sent msg3, and that call
clears rekey_in_progress. Both sides' set_pending_session can run
before either peer's msg3 has landed. Each side then receives the
peer's msg3 in the post-pending state with rekey_in_progress=false,
falls into the "drop because pending_new_session is set" guard, and
discards the peer's handshake. Each side commits its own initiator
session at K-bit cutover. The two sessions use different Noise key
material, so the link breaks asymmetrically after cutover.
Unify both checks into a single tie-breaker that fires when either
rekey_in_progress() OR pending_new_session().is_some() is true. The
existing smaller-NodeAddr rule applies uniformly; abandon_rekey()
already clears both states and returns whichever index needs
freeing.
Mirrored to the FSP rekey msg1 path in handle_session_setup for the
same race shape.
Logging:
- The previously-silent drop in the rekey arm of handle_msg3 logs at
info as a tie-break decision rather than as a silent drop.
Pending_new_session is added as a log field on the tie-break
win/lose lines so a reader can distinguish which of the two race
states fired.
- Cutover-complete and K-bit-flip log lines gained our_addr and
their_addr fields so the two endpoints' logs of the same handshake
can be correlated.
- "Pending session set, awaiting K-bit cutover" lines remain at
debug per the existing convention of info-for-cutover-completion-
only.
Verification: under the un-fixed handler, both sides log
"rekey-msg3 drop: pending_new_session already set" within 242
microseconds of each other, with rekey_in_progress=false and
different pending session indices. Failure pattern matches: six of
twenty pairs FAILED post-rekey, all involving the single-peer node.
Six consecutive post-fix CI attempts on the same test green across
rekey, rekey-accept-off, and rekey-outbound-only suites.
Marker merge to record the master dev-line (now v0.4.0-dev, with
maint's v0.3.1-dev open already incorporated) as known on next
without taking master's tree (next is on v0.5.0-dev). Future
forward-merges from master to next land cleanly on top of this base.
Master is now the v0.4.0-dev line; next moves up to v0.5.0-dev for
the wire-format-breaking work staged here.
- Cargo: 0.4.0-dev → 0.5.0-dev
- CHANGELOG Breaking-section header retargeted to v0.5.0; baseline
bumped from v0.2.x to v0.3.x peers (master is now the v0.3.1-dev
patch line and v0.4.0-dev minor line, so v0.5.0 breaks against
v0.3.x, not v0.2.x)
- README: badge v0.4.0--dev → v0.5.0--dev, three Status & Roadmap
prose references retargeted
- docs/reference/cli-fips.md: example version 0.4.0-dev → 0.5.0-dev
Brings v0.3.0 release content forward into the next-branch
development line, preserves next's own version state, and updates
operator-facing version references to v0.4.0-dev.
Kept from master:
- docs/releases/release-notes-v0.2.1.md and -v0.3.0.md (cumulative
archive of shipped releases)
- RELEASE-NOTES.md root mirror at v0.3.0 (tracks the most recent
shipped release; will be replaced when the v0.4.0 release cycle
begins on this line)
- CHANGELOG entries: [0.3.0] and [0.2.1] sections inserted under
the existing Breaking and [Unreleased] sections
- All code, config, test, and documentation updates from master
(openwrt yaml resync, doc-config IP placeholders, etc.)
Kept from next (resolved against master's release-prep changes):
- Cargo.toml / Cargo.lock at 0.4.0-dev (next's package version)
- CHANGELOG ## Breaking section (next-specific v0.4.0 wire-format
breaking work) and the empty ## [Unreleased] block for future
v0.4.0 non-breaking work
Updated to v0.4.0-dev for consistency with Cargo.toml:
- README badge (v0.3.0--dev -> v0.4.0--dev) and status-section
prose, rewritten to describe v0.4.0 wire-format-breaking work
on this branch (Noise XX unification, FMP node profiles,
slimmer MMP, extensible bloom-filter encoding) instead of the
v0.3.0 testing-and-polishing narrative that applied while
v0.3.0 was unreleased
- docs/reference/cli-fips.md example version string
Forward-merge of 12 master commits past the previous merge
(823b830, master @ 18019bb): dep-audit bumps (rand,
clap, tun, rtnetlink, windows-service, plus the bump-safe
lockfile batch), bloom-storm chaos scenario, control-socket
resolver consolidation, gateway dns.listen default change,
OpenWrt ipk README refresh, gateway tutorial review, Ethernet
MTU rustdoc fix, dead session-variant drop, CHANGELOG prep.
Conflict resolution:
- CHANGELOG.md: both bullets kept under [Unreleased] / Fixed.
Master's spanning-tree internal-path-propagation fix precedes
next's tree-ancestry-test determinism entry and the
responder-Disconnect XX-handshake entry.
- src/protocol/session.rs: kept next's SessionSetup/SessionAck
variants and rustdoc. Master's drop of those variants suits
v0.3.0's FSP phase-byte dispatch but is undone by next's
v0.4.0 wire format, which retains the variants and uses the
inner msg_type byte for handshake identification.
- src/transport/ethernet/mod.rs: kept next's "interface MTU - 4"
comment. Master corrected the v0.3.0 3-byte rustdoc; next
redesigned the framing to a 4-byte header (type/flags/length)
for shared-media beacons, so master's correction does not
apply to next's format.
Auto-merged cleanly: Cargo.toml (next's 0.4.0-dev + the new dep
pins from master), Cargo.lock, all gateway docs,
docs/reference/configuration.md, packaging files,
.github/workflows/ci.yml, testing/chaos/sim/* and
testing/ci-local.sh (bloom-storm additions), src/config/*.
Local verification: cargo build --release, cargo test (1252
passed, 4 ignored), cargo clippy -D warnings, cargo fmt --check
all green.
Forward-merge of the docs-overhaul squash (5abf9a9) and top-level
README rewrite (18019bb). Conflict resolution:
- README.md: master's rewritten feature lists adopted, with the encryption
bullets reflecting next's two-layer Noise XX (replacing the IK/XK pair
master describes for v0.3.0).
- 6 design/reference markdown files (fips-bloom-filters.md, fips-mesh-layer.md,
fips-mesh-operation.md, fips-session-layer.md, fips-transport-layer.md,
reference/wire-formats.md): master's reorg taken, next's protocol details
preserved (XX handshake naming, bloom v2 RLE/delta wire format,
v2 LookupRequest sizing).
- fips-intro.md modify/delete: accepted master's split into
fips-architecture.md / fips-concepts.md / fips-prior-work.md, then
re-applied next's IK/XK -> XX transition and spin-bit removal across
the relevant split files. Same pass swept docs/reference/security.md,
docs/design/fips-mmp.md, docs/design/fips-security.md,
docs/design/fips-nostr-discovery.md,
docs/design/port-advertisement-and-nat-traversal.md,
docs/how-to/enable-nostr-discovery.md, and the affected tutorials so
no IK/XK or spin-bit prose remains in current-state docs.
- Diagram path conflicts: noise-ik-msg{1,2}.svg removed (IK is gone);
noise-xx-msg{1,2,3}.svg moved from docs/design/diagrams/ to
docs/reference/diagrams/ to match master's diagram reorg. The
wire-formats.md image references resolve correctly to the new path.
Local verification: cargo build --release, cargo test (1265 passed,
4 ignored), cargo clippy -D warnings, cargo fmt --check all green.
Bump the default Nostr-discovery advert namespace from
`fips-overlay-v1` to `fips-overlay-v1-next` on next. Master
continues to publish under `fips-overlay-v1`.
Background: next runs FMP-v1 (Noise XX, msg1 33 bytes) which is
wire-incompatible with master's FMP-v0. Until now both branches
defaulted to the same Nostr advert namespace, so a stock
next-branch daemon's open-discovery sweep would happily pick up
master peers' adverts (and vice versa), succeed at the UDP punch,
adopt the socket, and fail every FMP handshake at the version-gate.
The per-peer-mismatch cooldown introduced on master is the safety
net for any case that slips past this default; the namespace
separation is the structural answer.
Three sites updated:
- `src/discovery/nostr/types.rs` `ADVERT_IDENTIFIER` const
documents why the value is branch-specific.
- `src/config/node.rs` `default_app()` matches.
- `src/discovery/nostr/tests.rs` and the
`testing/nat/scripts/nostr-relay-test.sh` malformed-advert
fixture publish under the new namespace so test harnesses see
the same adverts a real daemon would.
Operators who need cross-branch discovery during a coordinated
rolling upgrade can override `node.discovery.nostr.app` in
fips.yaml back to `fips-overlay-v1`.
The control-query show_bloom snapshot, baselined on master, mismatches
the next-branch output because the bloom filter v2 work on next added
six new stats fields (deltas_sent, full_sends, nacks_received,
nacks_sent, size_changes, total_compressed_bytes, total_raw_bytes)
and dropped non_v1. The snapshot test correctly flagged this as
schema drift.
The drift is intentional (next's bloom v2 is a deliberate schema
expansion, not a regression), so the right action per the test's own
guidance is to delete the fixture and let the test regenerate it
against next's actual output.
No production-code change.
Forward-merges the squashed per-destination TCP MSS clamping work
(master `ae60743`) into next.
Auto-merge of source files clean. Two pre-existing test assertions
on the next-side discovery tests were updated as part of merge
resolution to reflect the target-edge MTU fold introduced by the
merged-in commit:
- test_transit_forwards_when_mtu_sufficient: 1400 → 1280 = min(target-edge 1280, transit 1400)
- test_response_path_mtu_four_node_chain: 1350 → 1280 = min(target-edge 1280, transits 1350+1500)
Resulting next tree is bit-for-bit identical to the pre-squash next
state previously verified by full local CI sweep (29 + 1 next-only
suites pass on `6a8b519` in 23m 41s).
After the master→next merge brought PR #53's bootstrap-handoff socket
adoption onto next's XX three-message handshake, three integration jobs
(rekey, nat-cone, nat-lan) regressed: the FMP handshake completed but
post-handshake encrypted frames silently dropped — packets_recv stayed
at 0 and link-dead timeout fired every 30s.
Root cause: in symmetric bootstrap-handoff both sides initiate XX in
parallel, running two concurrent handshakes (each side's outbound paired
with the peer's inbound). Each side's handle_msg2 (immediate response
to its own outbound msg1) ran before the peer's outbound msg3 arrived;
peers.contains_key was false, so the "Normal path" promoted the outbound
connection. When the peer's msg3 then arrived, peers contained the peer
at the same epoch and the request fell through to the
"duplicate handshake from same epoch" branch, which tore down the inbound
link without applying the cross-connection tie-breaker. Both sides ended
up keeping their own outbound session whose Noise key material pairs with
the peer's discarded inbound, so each side's their_index pointed at the
peer's inbound session index that was never registered in peers_by_index.
handle_msg2 already has the symmetric handler for the inverse ordering
(msg3-then-msg2). Add the missing simultaneous-init handler in
handle_msg3: when the existing peer is on a different link from this
msg3's pending_inbound link and the session is fresh (<30s, so this
isn't a rekey), apply cross_connection_winner with this_is_outbound=false.
The larger-node side swaps to the inbound session via replace_session and
updates peers_by_index from outbound_idx to inbound_idx; the smaller-node
side keeps its outbound session and frees the inbound's allocated index.
Both sides converge on the same Noise session pair.
Verified: cargo test --lib (1144 passed), cargo fmt + clippy clean,
nat-cone + nat-lan + symmetric NAT scenarios pass, rekey integration
test 70/70 pairs across all phases.
The two bootstrap-handoff tests were written against the IK two-message
handshake where peer promotion happens after msg2. Under the noise-XX
three-message handshake on this branch, identity is only revealed in
msg3, so peer promotion happens one message later: the responder learns
the initiator's identity in msg3 and only then promotes the peer.
`test_adopted_udp_traversal_completes_handshake`: Previous version ran
two `run_rx_loop` iterations (one per node) and asserted both peers
had been promoted. That works for IK but not XX: after node_a sends
msg1 from `adopt_established_traversal`, node_b's first rx_loop pass
generates msg2 (revealing node_b's identity to node_a), node_a's next
rx_loop pass generates msg3 (revealing node_a's identity to node_b),
and only then does node_b promote node_a. A third rx_loop pass on
node_b is needed.
Restructured the test to use direct `handle_msg{1,2,3}` dispatch via
`packet_rx.recv()` rather than `run_rx_loop` + select cancellation,
matching the pattern already used by
`test_third_peer_can_handshake_via_adopted_transport_socket`. This
sidesteps the issue that `run_rx_loop` does `packet_rx.take()` and a
cancelled future never returns it, so a second rx_loop call fails
with `NotStarted`.
`test_third_peer_can_handshake_via_adopted_transport_socket`: Added
explicit msg3 dispatch on both Alice/Bob and Bob/Colin sub-handshakes
so the responder side actually receives the initiator's identity
before the test asserts peer promotion.
Under Noise XX the responder only sees the initiator's static key
in msg3, so the inbound-handshake ACL check cannot fire until msg3
has already been processed. By then the initiator has received
msg2, passed its own outbound ACL check, and promoted its side of
the peering. Silently tearing down the responder's state left the
initiator as a "connected" peer with no traffic until the
link-dead timeout fired tens of seconds later; the acl-allowlist
integration test on the next branch timed out at the 5s
convergence check on blocked nodes.
The responder now sends an encrypted Disconnect on the
freshly-completed Noise session before tearing down. The initiator
decodes it via the existing handle_disconnect path and cleans up
the zombie peer within one RTT. DisconnectReason::Other is used
instead of SecurityViolation so the wire payload does not name the
ACL mechanism; the behavioural signature (explicit reject right
after handshake) is the only observable leak.
Supporting changes:
- Add send_encrypted_link_message_raw in node/mod.rs that operates
on a raw NoiseSession + indices without going through self.peers,
since the peer has not yet been promoted at the reject site.
- Declare the acl test module in node/tests/mod.rs. It had been
orphaned since PR #50 so none of its tests ran; two of them no
longer compiled against the current next branch. Fixed the two
and replaced test_inbound_msg1_denied_by_acl (asserts IK-era
behaviour that cannot happen under XX) with
test_inbound_msg3_denied_triggers_disconnect, a full end-to-end
UDP test covering both the responder cleanup and the initiator's
Disconnect-driven cleanup.
Exposed by the rustfmt toolchain-pin fix now that rustfmt runs under
the pinned 1.94.1 toolchain in CI. Mechanical formatting changes:
- src/node/mod.rs: collapse EthernetTransport::new let-binding onto
one line (fits within width limit)
- src/node/tests/handshake.rs: reorder imports (super::* after
specific import) and expand two assert_eq! calls to multi-line form
The log analysis matched "proof verified, caching route" but the
actual log message is "Discovery succeeded, proof verified, route
cached", causing discovery succeeded to show 0 in all sim runs.
Replace fixed 1KB bloom filters with variable-size filters (512B-32KB)
that adapt to each node's position in the spanning tree.
Core changes:
- Internal storage: Vec<u8> → Vec<u64> for word-level operations
- Delta compression: XOR diff with word-level RLE, sequence-based
NACK protocol for full retransmit recovery
- Size conversion: fold (large→small) and duplicate (small→large)
with auto-converting merge for mixed-size filter combination
- Native-size storage: peer filters stored at advertised size for
full-resolution routing queries, converted only for outgoing filter
- Adaptive sizing: outgoing fill ratio drives step-up/step-down
between size classes with hysteresis (20%/5% thresholds)
- Filter size decoupled from FMP negotiation: announced dynamically
in filter updates, bit 7 and TLV field 1 removed from handshake
Wire format: FilterAnnounce gains flags byte (delta bit), base_seq
field, RLE-compressed payload. New FilterNack message (0x21) for
out-of-sequence delta recovery.
Redesign shared-media transport framing now that XX replaces IK:
- Ethernet: unified 4-byte header [type][flags][length:2 LE] for all
frame types, effective MTU now if_mtu - 4
- Beacons: strip 32-byte pubkey (34 → 5 bytes), identity learned from
XX handshake msg2/msg3 instead of beacon
- BLE: remove pre-handshake pubkey_exchange() and cross-probe
tie-breaker, both unnecessary with XX
- Discovery: support anonymous connections (pubkey_hint: None) with
address-based dedup for shared-media transports
- Post-handshake identity hooks in handle_msg2/msg3 for self-detection
and future allow/deny list filtering (IDEA-0047)
Drop origin_coords from LookupRequest — unused since reverse-path
routing became primary. Saves 2+16*depth bytes per request.
Wire up min_mtu: populated from TUN MTU config (default 1280) at
origination. Transit nodes skip peers whose link MTU is below the
request's min_mtu requirement. path_mtu on LookupResponse was
already wired (transit min() applied).
Add TLV extension to LookupRequest (after min_mtu) and
LookupResponse (after proof). Uses same TlvEntry format as
negotiation. Transit nodes forward TLV bytes verbatim.
- Delete PROTOCOL_NAME_IK, PROTOCOL_NAME_XK and all IK/XK methods
- Remove Ik/Xk variants from NoisePattern enum
- Rename XX methods to drop xx_ prefix (sole pattern)
- Rename HandshakeMessageType variants from NoiseIKMsg1/NoiseIKMsg2
to Msg1/Msg2, add Msg3 variant (0x03) for XX 3-message flow
- Fix doc comments with correct XX message sizes and descriptions
- Fix stale SessionSetup/SessionAck handshake payload size comments
Replace the 3-message XK handshake with XX for FSP session establishment.
XX requires no prior knowledge of the peer's static key — the responder's
identity is revealed in msg2, the initiator's in msg3.
Key changes:
- session.rs: XX initiator/responder, post-handshake identity verification
using x-only key comparison (parity-independent for npub compatibility),
negotiation payload in msg2/msg3 (FSP version [0,0], features=0)
- Rekey: switched from XK to XX for FSP rekey handshake
- timeout.rs: suppress msg1 resends when target peer is already promoted,
preventing cross-connection session mismatch from duplicate handshakes
- Test template: discovery backoff 3s and handshake timeout 10s for
faster convergence in integration tests
- Integration test timeouts restored to 45s (ping) and 60s (rekey)
Squashed commits:
- Switch FSP handshake from Noise XK to XX
- Fix integration test convergence by reducing discovery backoff
- Fix cross-connection session mismatch from msg1 resend
- Fix FSP identity verification parity mismatch
Replace the 2-message IK handshake with a 3-message XX handshake for
FMP link establishment. XX requires no prior knowledge of the peer's
static key — both identities are revealed during the handshake
(responder in msg2, initiator in msg3). This is the foundation for
the forklift upgrade that enables rolling protocol upgrades.
Changes:
- Noise XX state machine alongside IK/XK (8 unit tests)
- Protocol negotiation payload codec: format byte, packed version
min/max, 64-bit feature bitfield, TLV extensions (11 unit tests)
- FMP wire format version 0→1, msg3 header/builder, TCP stream framing
- FMP handshake switched to XX: PeerConnection 3-message flow,
handle_msg1 simplified (no identity), handle_msg2 sends msg3 and
promotes initiator, new handle_msg3 promotes responder with
restart/rekey/cross-connection detection
- Rekey handshake switched to XX with negotiation payload hash chain
fix (decrypt-and-discard in complete_rekey_msg2/msg3)
- Negotiation payload in msg2/msg3 (FMP version [1,1], features=0)
- Debug logging for handshake promotion paths
- Integration test convergence timeouts adjusted for extra round-trip
Squashed commits:
- Add Noise XX state machine alongside IK/XK
- Add protocol negotiation payload codec
- FMP wire format prep: version 1, msg3 header support
- Switch FMP handshake from Noise IK to XX
- Increase convergence timeouts for XX 3-message handshake
- Fix negotiation hash chain desync in rekey handshake
| [persistent-identity.md](persistent-identity.md) | Provision a stable Nostr keypair so the node keeps the same npub across restarts |
| [host-aliases.md](host-aliases.md) | Use shortnames (`test-us01.fips`, `my-laptop.fips`) instead of full npubs by editing `/etc/fips/hosts` or setting peer aliases |
| [set-up-bluetooth-peer.md](set-up-bluetooth-peer.md) | Configure a Bluetooth Low Energy peer link |
| [set-up-80211s-mesh-backhaul.md](set-up-80211s-mesh-backhaul.md) | Link OpenWrt FIPS routers over an open 802.11s radio backhaul (FIPS provides encryption, authentication, and routing) |
| [set-up-open-access-ssid.md](set-up-open-access-ssid.md) | Broadcast the open `!FIPS` access SSID so phones and laptops roam onto the mesh (one ESS: save once, roam every FIPS router) |
| [diagnose-mtu-issues.md](diagnose-mtu-issues.md) | Triage MTU-shaped failures and rule out their imposters (bufferbloat, transport saturation) |
| `profile tick on` | `profile_tick_on` | Create the capture file and start recording. Fails if a capture is already running (naming the active file) or if the directory cannot be written. |
| `profile tick off` | `profile_tick_off` | Stop the capture. The writer is woken immediately, drains once more and is joined, so the command returns promptly. Succeeds, reporting nothing active, when no capture is running. |
| `profile tick status` | `profile_tick_status` | Report `idle`, `running`, `stopped_by_cap` or `stopped_by_error`, plus the active path, bytes written, flush interval and byte cap. |
`profile tick on` options:
| Flag | Argument | Default | Description |
| ---- | -------- | ------- | ----------- |
| `--dir` | directory path | `/var/log/fips` | Where to write the capture. Created if absent. Use it to profile a non-root `cargo run`, or on a platform whose log root differs. |
One file is written per capture, named `profile-<UTC timestamp>.tsv`.
It opens with a `#`-prefixed header block (node npub, build version,
@@ -104,7 +104,8 @@ to the highest-priority config file for operator visibility, even in ephemeral m
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.leaf_only` | bool | `false` | Leaf-only mode: node does not forward traffic or participate in routing |
| `node.disable_routing` | bool | `false` | Non-routing mode: participates in spanning tree but does not forward transit traffic or send bloom filters |
| `node.leaf_only` | bool | `false` | Leaf mode: single upstream peer, no tree/bloom/transit participation. Implies `disable_routing: true` |
| `node.bloom.max_inbound_fpr` | f64 | `0.10` | Antipoison cap: reject inbound `FilterAnnounce` frames whose advertised false-positive rate exceeds this value. Valid range `(0.0, 1.0)`. The default `0.10` corresponds to fill 0.631 at k=5 (≈1,630 entries on the 1 KB filter); a saturated/poisoned filter is still ~100% FPR and rejected |
| `node.bloom.max_inbound_fpr` | f64 | `0.20` | Antipoison cap: reject inbound `FilterAnnounce` frames whose advertised false-positive rate exceeds this value. Valid range `(0.0, 1.0)`. The default `0.20` corresponds to fill 0.7248 at k=5 (≈2,114 entries on the 1 KB filter); a saturated/poisoned filter is still ~100% FPR and rejected |
Bloom filter size (1 KB), hash count (5), and size classes are protocol
constants and not configurable.
@@ -304,7 +305,7 @@ stays set for all subsequent hops to the destination.
### Rekey (`node.rekey.*`)
Controls periodic Noise rekey for forward secrecy. When enabled, both FMP
(link-layer IK) and FSP (session-layer XK) sessions perform fresh Diffie-Hellman
(link-layer XX) and FSP (session-layer XX) sessions perform fresh Diffie-Hellman
key exchanges after a time or message count threshold, whichever comes first.
A 10-second drain window keeps the old session active for decryption during
cutover.
@@ -338,7 +339,7 @@ Metrics Measurement Protocol for per-peer link measurement. See
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.mmp.mode` | string | `"full"` | Operating mode: `full` (sender + receiver reports), `lightweight` (receiver reports only), or `minimal` (spin bit + CE echo only, no reports) |
| `profile_tick_on` | `dir` (optional directory path; default `/var/log/fips`) | Creates the capture file, publishes its path, and starts the writer thread. `data`: `state`, `path`, `interval_secs`, `byte_cap`. Errors if a capture is already running (naming the active file) or the directory is unwritable. |
| `profile_tick_off` | — | Stops the capture, drains once more, joins the writer. `data`: `state`, `stopped`, `stopped_by_cap`, `stopped_by_error`, `path`, `bytes`. |
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Johnathan Corgan
ArjenandGitHub
