laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-08-01 12:36:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: laantungir/fips:v0.4.1
Branches Tags
laantungir/fips:master laantungir/fips:feat/platform-peer-queue laantungir/fips:next laantungir/fips:maint laantungir/fips:ble-v2
laantungir/fips:v0.4.1 laantungir/fips:v0.4.0 laantungir/fips:v0.4.0-rc2 laantungir/fips:v0.4.0-rc1 laantungir/fips:v0.3.0 laantungir/fips:v0.2.1-rel laantungir/fips:v0.2.1 laantungir/fips:v0.2.0-rel laantungir/fips:untagged-837413e700b3aaa62052 laantungir/fips:v0.1.0
..
compare: laantungir/fips:next
Branches Tags
laantungir/fips:feat/platform-peer-queue laantungir/fips:next laantungir/fips:master laantungir/fips:maint laantungir/fips:ble-v2
laantungir/fips:v0.4.1 laantungir/fips:v0.4.0 laantungir/fips:v0.4.0-rc2 laantungir/fips:v0.4.0-rc1 laantungir/fips:v0.3.0 laantungir/fips:v0.2.1-rel laantungir/fips:v0.2.1 laantungir/fips:v0.2.0-rel laantungir/fips:untagged-837413e700b3aaa62052 laantungir/fips:v0.1.0
462 Commits
v0.4.1 .. next
Author SHA1 Message Date
Johnathan Corgan 069d450983 Merge branch 'master' into next 2026-07-29 02:24:38 +00:00
Johnathan Corgan 83c4e800a5 Merge branch 'maint' 
# Conflicts:
#	src/node/retry.rs
 2026-07-29 02:17:34 +00:00
Johnathan Corgan 13a98ae702 Make a half-scoped reap impossible in ci-cleanup.sh 
The reap runs two selectors, one on the CI label and one on the
compose project, and each flag narrows only its own. So --run-id
alone leaves the project sweep broad and --project-prefix alone
leaves the label sweep broad, and either way the reap still destroys
every concurrent run on the host. Both single-flag forms look scoped
and are not.

The usage text asserted otherwise. It documented --run-id as leaving
other runs alone and --project-prefix as scoping the reap to a single
run, and both claims were false for the same reason. A caller passing
--run-id alone, on the strength of that line, force-removed the
containers of three concurrent runs on 2026-07-29, and its own
comment recorded the belief that it was scoped.

Rather than ask every caller to remember the pair, close the two
half-scoped states here. --run-id now derives the matching project
prefix when none is given, built from the existing base so the two
cannot drift, and an explicit --project-prefix still wins.
--project-prefix without --run-id is refused outright, because there
is nothing to derive a label scope from and the quiet failure is
someone else's run disappearing.

Passing neither flag is untouched: that is the deliberate "reap
everything" form a manual cleanup wants.

Verified against a decoy container labelled as another run: the
scoped reap leaves it up, and reproducing the old broad project sweep
removes it, so the check discriminates rather than passing because
there was nothing to reap. An earlier version of that check ran when
no CI containers existed at all and passed without proving anything.
 2026-07-29 02:09:59 +00:00
Johnathan Corgan 77ed64bb88 Compute every peer's outgoing bloom filter in one sweep 
Each peer must be sent the union of all other peers' inbound filters,
excluding its own contribution. Building that per recipient rebuilt
the whole peer-filter map and re-ORed it once for every peer, so a
tick that announced to R peers did R kilobyte-scale map builds and
R by T merges. At 240 peers this was 20.6 ms per tick, roughly half
the tick body, and it was steady work rather than a tail: the
per-interval maximum had a median of 34.5 ms.

Replace it with a prefix and suffix union sweep that produces every
target's filter in one pass, T merges instead of R by T. The packet
path gets the same treatment, since marking changed peers had the
identical shape once per inbound announce.

The result is exactly equal, not approximately. Merging is a bytewise
OR, so regrouping the unions cannot change the outcome, and a filter
whose size does not match is rejected before any byte is touched, at
every merge site in both the old and new arrangement. Every
accumulator here is default-sized, so an odd-sized peer filter is
skipped in the new code exactly where it was skipped in the old.

Cadence, the debounce, the sequence rule and the fill-ratio cap are
untouched. A sequence number is still drawn after the debounce
re-check and before encoding, so a suppressed peer still consumes
none and an encode failure still burns one.

Known trade-off, measured rather than assumed: the sweep does its
full O(T) work regardless of how many peers are ready, so a tick that
announces to only one or two peers now costs about twice what it did.
Break-even is around three ready peers, and the saving above that
grows without bound. The marking path is a pure win, since it always
targets every peer.
 2026-07-29 01:45:12 +00:00
Johnathan Corgan bf2e7a0892 Derive each peer's npub once instead of once per tick 
The per-tick stats snapshot ran a bech32 encode for every tracked
peer, and for the common mesh peer — one with no hosts-file entry and
no configured alias — it ran a second one, because the display-name
fallback chain bottoms out in the same encode. At 240 peers that was
14.1 ms per tick, a third of the tick body and its second largest
cost, all of it recomputing values that cannot change.

Cache the npub and the shortened npub on the peer at construction. An
npub is a pure function of the peer's public key, and the identity is
never mutated after construction: there is no setter, no identity_mut,
and no assignment to the field anywhere in the tree, so the cache
cannot go stale.

The display name itself is deliberately NOT cached. Two of its inputs
do mutate at runtime — the alias map and the host map, the latter
reloaded on this same tick — so a resolved name stored on the peer
would go stale on an alias change or a hosts reload. Only the
immutable component is memoized.

Tests cover both constructors, that the cached npub matches the
identity, and that the display name still tracks an alias change. The
memoization itself is asserted by pointer stability rather than by
timing, so it is deterministic under load. Three deliberate breaks
were each caught by exactly one test: re-deriving instead of
memoizing, populating one constructor's cache from the wrong source,
and reordering the display-name fallback so it stops honoring aliases.
 2026-07-29 01:32:48 +00:00
Johnathan Corgan 2045b51f5d Watch the Nostr loops that never return, not the one that does 
Every node reported state: degraded permanently. The liveness probe
polled connect_task, which wraps a single Client::connect() call. That
call only spawns a per-relay background connection task and returns,
so the handle finished moments after start on a perfectly healthy
node, the supervisor saw a child exit, and the node latched Degraded
for the rest of its life.

Nothing behaved differently, because every consumer of NodeState
treats Degraded the same as Running. What was lost is the signal: a
genuine degradation was indistinguishable from the permanent false
one.

Watch the three service loops that cannot return by design instead —
the inbound notify loop, the advert publisher, and the refresh ticker.
Each is an unconditional loop, so a finished handle means a panic or
an abort, which is unrecoverable and matches the one-way ChildExited
latch in the supervisor. connect_task and relay_startup_task stay
deliberately unwatched, and the doc comment now says why, since
watching either reproduces this bug exactly.

The tests install task handles directly and check both directions:
that a finished connect_task alongside three live loops reports
healthy, which is the production configuration a few hundred
milliseconds after start, and that each loop dying on its own reports
degraded. Reverting to the old predicate reds four of five; replacing
the predicate with a constant false also reds four of five, so the
fix cannot pass by never reporting degraded at all.
 2026-07-29 01:31:13 +00:00
Johnathan Corgan d6457fa74f Stop awaiting the advert refetch on the retry tick 
process_pending_retries runs inline on the node's 1-second rx-loop
tick. For each due peer it awaited a Nostr relay fetch carrying a
2-second timeout, and discarded the result. With up to sixteen due
peers in one tick body, the timeouts stack: field profiling measured
single 2.00 s stalls as the common case and a worst tick of 12.4 s
against a 1 s period, with every other rx-loop arm delayed behind it
by as much as 4.2 s.

Spawn the refetch instead of awaiting it, matching the pattern the
failure arm of the same loop already uses thirty lines below. The dial
now uses whatever advert is cached at that moment and the refreshed
one lands for the next retry of that peer. Since retries are
backoff-paced, that defers the benefit by one backoff interval rather
than losing it, and the result was already being discarded, so nothing
downstream read it.

The test drives four due peers whose refetches all hang against a
local listener that accepts and never speaks, so the fetch burns its
full timeout with no network egress. Awaited, the call takes 8.0 s;
spawned, it returns in milliseconds. It also asserts every due peer
was still attempted and rescheduled, so a version that skipped the
dial entirely cannot pass it.
 2026-07-29 01:27:49 +00:00
Johnathan Corgan 22e71b55a3 Merge branch 'master' into next 2026-07-29 00:36:24 +00:00
Johnathan Corgan 3c4f003c91 Merge branch 'maint' 
# Conflicts:
#	src/nostr/runtime.rs
#	src/nostr/tests.rs
 2026-07-29 00:36:19 +00:00
Johnathan Corgan 7d0a110f4e Send traversal signals only to relays the client pool actually holds 
A traversal signal is addressed to a merge of the peer's NIP-17 inbox
relays, the relays its advert nominates for signaling, and our own DM
relays. The client pool is built once at startup from our configured
relays and never added to, and send_event_to rejects the entire send
with "relay not found" if any single URL in the list is outside that
pool, before contacting anything. So one relay we are not configured
with, anywhere in that merge, killed the whole attempt -- including the
sends to relays we do share and that would have carried the signal.

In an open-mode window on a public node this made discovery
non-functional: 309 traversal attempts, 290 explicit failures, zero
successes, every failure on "relay not found". Configured peers were
unaffected because they run a matching relay set.

Filter the merged list down to relays the pool holds before sending.
Our own DM relays are always in the merge and always in the pool, so
the result is empty only when no DM relay is configured at all, which
is already a total failure. Comparison is on the normalized RelayUrl
rather than the raw string, because that is how the pool is keyed --
a raw comparison would discard a configured relay spelled with a
trailing slash or a different host case, which is the same defect in
a quieter form.

Merge and filter are one synchronous function so the decision can be
exercised without a relay client. The pool is read via all_relays(),
which is the set send_event_to validates against; relays() is filtered
by service flags and would be narrower.

Two smaller fixes ride along. The responder now resolves its relays
before binding a socket and running STUN, instead of spending a STUN
round trip and holding an offer slot only to discover it has nowhere
to answer. And it gained the empty-list guard the initiator already
had, which gives BootstrapError::MissingRelays a condition it can
reach for the first time -- it was unreachable, since the merge always
appended our own DM relays.
 2026-07-29 00:31:36 +00:00
Johnathan Corgan e8800c3a34 Merge branch 'master' 2026-07-28 17:40:34 +00:00
Johnathan Corgan 2e8fb60970 Let build-deb.sh build with Cargo features, and mark the version when it does 
A measurement run needs the .deb built with a non-default feature, and the
script had no way to express that: its argument loop took only --target,
--version and --no-build, and nothing forwarded a feature list or read one
from the environment. --features <list> now forwards to cargo-deb's native
-F.

The marking is the half that is easy to skip and matters more. The
auto-derived dev Version is built from the crate version, the commit date
and the sha, none of which change when a feature is enabled, so an
instrumented package and a default package of the same commit carried
byte-identical versions. Two consequences, and the second is worse than
the first: the node offers no way to tell which one it is running, and
reverting is an install of a version already present, which no-ops
silently and leaves the instrumented binary in place. That is precisely
the failure the per-commit version was introduced to prevent, reappearing
one level down. The Version now carries a +<features> marker, folded to
dots since underscores and commas are not legal there.

The marker sorts above the unmarked build, checked with dpkg
--compare-versions rather than assumed, so installing a feature build is
an upgrade and reverting is a downgrade: revert with dpkg -i, not apt
install. The ~dev ordering below a tagged release is preserved.

--features is refused together with --no-build, which would stamp the
marker onto whatever binaries happened to be sitting in target/ while
claiming the features had reached them.

Verified end to end rather than by reading: a real --features profiling
build produces a package whose fipsctl accepts the profile subcommand and
whose daemon carries the capture-file header text that only exists under
the feature gate. The release packaging path is untouched, shown by
diffing the cargo invocation the old and new scripts produce across five
argument shapes including the --version/--no-build pair the packaging
workflow uses; all five are identical.

Authored on master rather than maint, which takes bug fixes and CI or
tooling changes but not new capability, so the three copies of this
script now differ by design.
 2026-07-28 17:40:28 +00:00
Johnathan Corgan 60fb3af359 Merge branch 'master' 
Reconciles the tick profiler with this line's extra tick step. `next`
drives `resend_pending_fmp_rekey_msg3` from the tick arm and the master
line does not, so the step table gains a variant here and the arm gains
one more instrumented call.

The pinned row-count test is what caught it: it failed on the merge
rather than letting the emitted table and the call sites drift apart
silently, which is the failure mode it was written for.
 2026-07-28 01:39:47 +00:00
Johnathan Corgan 8162b7d9fc Cover the capture writer's failure path and split its flush cycle 
The mesh rehearsal could not exercise the write-error stop at all, and
that is not a gap in the rehearsal. Removing the sink file leaves the
writer's descriptor valid, so writes keep succeeding into the unlinked
inode and the capture runs on; mounting a tiny filesystem inside the test
container is refused outright. So the terminal state added for a failing
sink had no coverage from either direction.

Split one flush cycle out of the writer loop as a function returning what
it decided, so the loop owns the waiting and the state transition while
the decision is testable on its own. A sink that fails every write now
drives the error outcome directly, and a working sink is asserted to
continue so the first test cannot pass for a writer that always stops.
The residual gap is recorded at the test: what is covered is that an
error from the sink produces the error outcome rather than the cap
outcome, not that a real full disk reaches that branch.

The rehearsal itself was otherwise clean on a live three-node mesh. The
header carries node, build, platform, tick period and the reading
caveats; every step reports every interval; stopping returns in 53 ms
rather than waiting out the flush interval; an idle status reports no
path and no bytes; arming twice is refused naming the active file; an
unwritable directory fails the command; and the node kept serving
throughout. On an idle node the entry gap sits at one tick period and
arm starvation reads about 1.4 ms, which is small but not zero, so the
measurement is live rather than degenerate.
 2026-07-28 01:30:11 +00:00
Johnathan Corgan 7493153a89 Add a feature-gated profiler for the rx-loop maintenance tick 
The tick arm runs twenty-six housekeeping steps in sequence on the one
runtime thread, and is polled last, so anything slow in it holds up
inbound packets, TUN traffic and control commands behind it. Field
evidence says that happens for over a second at a time, but the
attribution behind that is two months old and predates the
connect-on-send gate, the control read isolation, and the peer lifecycle
rework. This measures it rather than continuing to reason about it.

Per step it records exact count, max and total into fixed static
counters; a dedicated writer thread drains them every ten seconds to a
TSV under /var/log/fips, one file per capture, capped at 32 MB. Nothing
accumulates: the counters are swapped to zero each interval and the
thread holds no history. Arming is `fipsctl profile tick on`, served in
the control accept task so the toggle cannot queue behind the very
behaviour it measures, and it does not survive a restart.

The whole thing is behind a Cargo feature that is off by default,
because the risk worth eliminating is the twenty-six edited call sites
in the hot loop. With the feature off the macro expands to the bare
expression, which makes the default build's neutrality something you
read off the generated code rather than something a benchmark fails to
disprove.

The measurement that matters is how late each tick is against the
deadline it was scheduled for, since the arm is polled last and that
lateness is the delay. Two earlier designs derived it from the interval
between entries and both under-reported: the schedule is fixed, so a
steady delay leaves every gap exactly one period and any gap-derived
figure reads zero under precisely the sustained overload this is meant
to find. The interval hands back its own deadline, so the delay is now a
subtraction with no model behind it, and a test drives three late ticks
at a constant gap to keep it that way.

CI gains a default-features clippy and a feature-on build and test on
both runners, closing the gap left by clippy already running with all
features.
 2026-07-28 00:56:54 +00:00
Johnathan Corgan 746218b6bb Merge branch 'master' into next 2026-07-27 17:57:36 +00:00
Johnathan Corgan 52dc21726a Merge branch 'maint' 2026-07-27 17:57:21 +00:00
Johnathan Corgan 2fdc831ce3 Claim the NAT lab's two bridges per run and derive every address from them 
The NAT lab was the last suite pinning fixed IPv4 subnets, so two overlapping
runs collided on the wan and shared-lan bridges.

Each run now claims a free /24 for each bridge, scanning candidates and
advancing on an overlap while still failing fast on any other network-create
error. The claim exports NAT_WAN_PREFIX and NAT_LAN_PREFIX, and every routable
address in the compose file and the suite scripts derives from them, with
defaults that render exactly what the lab used before. The router-side LANs are
deliberately left pinned: they live inside per-container network namespaces,
never become docker networks, and cannot collide.

An external-network overlay lets the suites attach to the networks the run
already claimed instead of creating their own. Two of the three suite scripts
had no overlay hook, so they would have requested the claimed range a second
time; both now have one.

Host veth names are scoped by a short token rather than the full run id, which
overruns the fifteen-character interface-name limit at the default run-id
length, and the cleanup reaper's pattern is widened to match the new shape.

Networks are released inline on every exit path rather than in a trap, since a
trap written inside a shell function replaces the script-level handler and
would disable the whole run's teardown.
 2026-07-27 14:34:46 +00:00
Johnathan Corgan fbff52d85e Give the NAT lab per-run generated configs 
The NAT lab wrote its generated node configs to one shared directory, unlike
the static and firewall labs which already scope theirs by run. The directory
is bind-mounted by compose and read back by the suite scripts after the
containers are up, so two overlapping runs let the second run's generator
overwrite the npubs the first is about to ping.

Scope the directory with FIPS_CI_NAME_SUFFIX at all three places that have to
agree: the generator's output path, the ten compose bind-mounts, and the
CONFIG_DIR the suite scripts read back from. An unset suffix renders the plain
path the lab has always used, so a bare invocation and the GitHub matrix are
unaffected. The run teardown removes the per-run directory alongside the static
and firewall ones, and the gitignore is widened to cover the suffixed form.

Rendering the compose file with every profile and no suffix set reproduces
today's exact ten paths; setting a suffix moves all ten.

This lands on its own, ahead of the network and address work, so that a failure
of the two-overlapping-runs acceptance test can be bisected between the config
fix and the address conversion.
 2026-07-27 14:34:33 +00:00
Johnathan Corgan 540532de44 Make the test backdating seams fail loudly instead of moving time forward 
Windows unit tests have been reding on next at a rekey-drain assertion that
reads like a defect in the drain logic. The drain logic is correct; the test
never reached the state it asserts on.

All three test-only backdating seams shared the fallback
checked_sub(age).unwrap_or_else(Instant::now). When the requested backdate
falls outside the monotonic clock's representable range, that assigns the
present instant, moving the timestamp forward rather than backward. A fresh
Windows CI runner has been up for less than the 600 seconds the drain test
asked for, so the window looked brand new, check_rekey correctly declined to
release the demoted session, and the assertion fired against innocent code.

The seams now route through a shared helper that panics naming the field, the
requested age and the real cause. The drain test asks for 30 seconds, three
times the ten-second window it needs to clear, rather than sixty times it.

Both properties were established by breaking them. Backdating by five seconds,
inside the window, reds the test, so it still depends on expiry rather than
passing vacuously at the smaller margin. Forcing the subtraction to fail
produces the new panic and its explanation rather than a silent reset. Full
cargo test --lib green at 1742, fmt and clippy clean.

No step ran on Windows, so the platform mechanism is inferred from the split
between runners and confirmed only when that job goes green on this tip.
 2026-07-26 18:50:53 +00:00
Johnathan Corgan b6c1977e16 Merge branch 'master' into next 
Carries today's maint batch up: the local CI image-scoping work, which
gives each run its own build context and test image tag instead of
writing the shared mutable one, and the retirement of the bloom-storm
chaos scenario.

Resolved as the earlier per-commit merges did, in the two files where
this line has diverged — the node test module list and the static suite
compose file. Parity holds at 21 legs a side, matching master.
 2026-07-26 17:17:32 +00:00
Johnathan Corgan 76d7e43d60 Merge branch 'maint' 
Carries today's maint batch: the local CI image-scoping work, which gives
each run its own build context and test image tag instead of writing the
shared mutable one, and the retirement of the bloom-storm chaos scenario.

Both apply unchanged here — the compose files, suite lists and matrix legs
they touch are identical on the two lines, so no branch adaptation was
needed. Parity stays symmetric across runners at 21 legs a side on this
branch and 24 on maint, the gap being the three rekey Docker suites maint
keeps by design.
 2026-07-26 17:16:26 +00:00
Johnathan Corgan 93800a503e Retire the bloom-storm chaos scenario from both CI runners 
The scenario guarded a real regression: a mid-chain tree update that
changed neither root nor depth leaking downstream as a sustained bloom
announce storm. But it was never once run against the regressed binary,
and its per-node ceiling was inferred from a post-mortem harness that no
longer exists in the tree. On the only surviving regressed measurement
the tail node's rate scales to roughly 7 sends per 30s, well under the
scenario's ceiling of 40, so it was never established that the assertion
could fire on its own bug class. The ceiling is also uniform per node,
calibrated against the flap target that is legitimately busy rather than
against the tail where the storm actually shows.

This removes coverage rather than relocating it, unlike the two earlier
retirements above it, and the comment in ci-local.sh records that gap
explicitly. The scenario, its README, the link_swap sim primitive and the
mesh-lab dispatch all stay on disk, so it remains runnable by hand.

Parity holds at 24 legs a side. The parity guard was break-checked by
re-adding the GitHub leg alone, which correctly reported the asymmetry.
 2026-07-26 16:51:23 +00:00
Johnathan Corgan cdda660f10 Stop local CI writing the shared mutable test image tag 
The run built per-run images and then retagged them to fips-test:latest as
a compatibility bridge for the consumers that had never been migrated. The
bridge was the defect: while it existed, two concurrent runs shared one
binding, so a suite could start containers from the other run's binaries
and the verdict was recorded against a commit whose code never ran. It
fails silently by construction — the run is green either way, and nothing
compares a running container's binary against the commit under test.

Every consumer now reads the run's image, so the retag is deleted rather
than kept. That is the point of deleting it: a consumer that was missed
fails loudly instead of quietly resolving whichever run wrote the tag last.

Two consequences handled here. The cleanup script ran ip(8) inside
fips-test:latest to reap simulation interfaces, and nothing writes that tag
any more, so it now takes the caller's image, then the run's, then any
surviving test image — the last of which is what keeps an unscoped --reap
working, since that path execs before the run identity is exported. And a
guard checks statically that nothing names the shared tag, because on a
host with a hand-built copy lying around a reintroduction would run green;
it is break-checked against a reintroduced compose consumer and a
reintroduced default. Both runners gate on it, as they do the other guards.
 2026-07-26 15:47:22 +00:00
Johnathan Corgan 49163befd5 Give each local CI run its own docker build context 
Scoping the image tag was never sufficient on its own. Every build read one
unscoped directory in the working tree, into which each run copies the
binaries it just built, so two concurrent runs raced on the contents of the
context as well as on the name of the result — and a run could produce a
correctly-per-run-tagged image built from the other run's binaries.

The run now copies the context's tracked files into its own directory,
installs its binaries there, and builds from it, exporting the path so every
other consumer follows. Deliberately not carried over: a previous run's
binaries, since inheriting them is the failure this prevents. The path is
absolute because compose resolves a relative build context against the
compose file's own directory rather than the working directory, which was
measured rather than assumed.

The chaos entry script gated the whole simulation on a binary in the shared
directory by literal path, so it moves in this same commit: left behind, it
would have failed on a clean checkout and, worse on a host with leftovers,
passed while reading a binary that was not the one under test.

Teardown removes the directory on red runs as well as green, since it holds
only reproducible content and is never the evidence of a failure. The
worker's SIGKILL runs no trap, so the cleanup script also sweeps contexts
left by a preempted run, and the ignore rule keeps a concurrent run from
showing up as untracked working-tree noise.
 2026-07-26 15:42:48 +00:00
Johnathan Corgan af847c68b5 Stop the chaos simulation rebuilding the shared test image per scenario 
The runner rebuilt the shared tag from the shared build context at the
start of every scenario, unconditionally. Local CI runs chaos scenarios in
parallel, so that was several concurrent builds into one name inside a
single run, before any second run is considered, and it silently replaced
whatever image the harness had already built and handed over.

It now uses the caller's image when one is named, and asserts the image is
present rather than building a substitute, because under a harness a miss
means something upstream is broken and building would hide it. A bare run
still builds the shared tag exactly as before, reading the run's own build
context when one is set.
 2026-07-26 15:40:03 +00:00
Johnathan Corgan a718cef8ce Have every suite compose file name the caller's test image 
Only the static family read FIPS_TEST_IMAGE; the firewall, ACL, nat,
sidecar and both Tor compose files named the shared mutable tag directly,
which is why the local CI runner has to retag its per-run image to that
name. They now take the same defaulted form the static file already uses,
so a harness run resolves the image it built and a bare hand run still
resolves the shared tag exactly as before.

Checked by rendering each file through `docker compose config` with the
variables set: every service resolves to the supplied name and none is
left naming the shared tag.
 2026-07-26 15:38:28 +00:00
Johnathan Corgan cbbdf2c13c Stop the firewall and ACL suites rebuilding an image their caller supplied 
Both suites ran `docker compose up -d --build` unconditionally, and their
compose files carry both a build context and an image name, so the flag
rebuilt and retagged the shared test image from the shared build context on
every run. Their --skip-build flag did not cover it: it guarded only the
Rust build above. Under a harness that has already built the image and
handed it over, that rebuild can only replace the binaries under test with
whatever the shared context currently holds.

The nat scripts had the mirror-image problem. Their image guard built a
replacement whenever the image was missing, which is right by hand and
wrong under a harness: when the caller names an image it built, a miss
means something upstream is broken, and manufacturing a substitute hides
that. They now fail loudly in that case and keep building only on the hand
path. The mesh-lab guard reads the same variable and its build hint moves
to the script that still produces the shared tag.
 2026-07-26 15:37:50 +00:00
Johnathan Corgan 24bf4d46d4 Widen the churn-mixed baseline bounds to sit outside the scenario's own spread 
The root-count ceiling of 4 was set from six runs whose observed maximum
was 3. Fourteen runs that recorded a baseline verdict put the real
distribution at 1 to 5 roots, so the ceiling sat at roughly the 93rd
percentile of the scenario's own variance: one sample in fourteen
exceeded it and four sat at or above it. It therefore reddened a share of
runs whatever the daemon did, which is a calibration fault rather than a
convergence one. Every sample ran the file's fixed seed, so the spread is
container timing and not a differing chaos schedule.

The ceiling is now 6, one step beyond the observed maximum, and the
parented floor moves to its complement at 4. That still fails a mesh
where seven or more of ten nodes are islanded, which is the collapse case
the assertion exists to catch; driving evaluate_baseline directly
confirms all fourteen samples pass and 7, 8 and 10 roots still fail.

min_sessions is left at 10 against an observed minimum of 12. It has
never fired, but the comment now records the margin as thin so a future
failure there is read as calibration first.
 2026-07-26 15:22:43 +00:00
Johnathan Corgan cf129cde77 Retire the three ignored Ethernet tests, which covered nothing unique 
All three required root or CAP_NET_RAW, neither runner passes --ignored,
and so none had ever executed in CI. Before deleting them I checked what
each covered and whether anything else covered it:

- The two-node handshake is covered by the ethernet-only chaos scenario,
  whose baseline assertion cannot pass unless handshakes complete over
  AF_PACKET.
- Tree convergence and root election by smallest NodeAddr are covered by
  ethernet-only's single-root assertion and by the in-process
  spanning-tree tests respectively.
- The mixed-transport coexistence property is covered twice over by
  test_tcp_mixed_transport_coexistence and test_ble_mixed_transport, which
  call the same verify_tree_convergence_components helper with the same
  two-component shape and both run today.

So the only thing running them would have added is an AF_PACKET variant of
a property already proven on two other transports.

The counts confirm nothing running was lost: 1392 tests pass before and
after, and the ignored count falls from 7 to 4.

The ethernet-only comment block cited these tests as the reason its
assertion was the only Ethernet coverage in CI. It now records the sharper
fact found while reading them: that coverage is control plane only. Both
Ethernet scenarios disable traffic, so no datagram crosses an Ethernet
link anywhere, which leaves the frame length field that trims NIC minimum
frame padding ahead of AEAD verification unexercised. Deleting these tests
does not widen that gap, because the test named for data exchange did not
exchange any.
 2026-07-26 14:41:47 +00:00
Johnathan Corgan 4315328f2b Stop the log analyzer double-counting outbound peer promotions 
The shared analyzer matched two strings for one event: "Peer promoted to
active", emitted at info, and "Outbound handshake completed", emitted at
debug. Both come from handle_msg2 on the same call path for the same
outbound promotion, so any run at debug level counted every outbound
promotion twice.

Measured rather than argued. On the archived debug-level churn-mixed run
of 2026-07-25, the node logs carry 20 of each string; the old matcher
reported 40 promotions and the new one reports 20. The neighbouring
counters are unmoved at 13 sessions and 33 parent switches.

Info-level runs are unaffected, which was checked rather than assumed: on
the tcp-mesh run of the same date the debug string occurs zero times, and
the old and new matchers both report 7 against 7 raw occurrences. Three
scenarios run at debug, one of them gating, so this corrects the promotion
figure in their artifacts. No assertion reads the counter, so no suite's
verdict changes.

The earlier fix that introduced this added the live string alongside the
dead one where it needed to replace it.
 2026-07-26 14:24:44 +00:00
Johnathan Corgan 44e04eb2e2 Retire the tcp-chain static test topology 
The tcp-chain profile has never been runnable. ping-test.sh dispatches on
chain and mesh only, and the string tcp-chain has never appeared in that
script in its history, so invoking the profile has always fallen through
to the unknown-profile branch: silently asserting nothing before that
branch was made to fail, and exiting 2 since. Neither runner referenced
it either.

What it would have covered is already covered. The chaos tcp-mesh
scenario runs in local CI and gives TCP a discriminating gate: n04's only
edges are TCP, so if the transport were broken n04 could not parent and
the scenario's baseline assertion would fail. It also carries a pure-TCP
two-hop path, n01 to n04 to n05, under netem and link flaps. The only
residual tcp-chain would have added is a mesh with no UDP present
anywhere, which is not worth a fixture that has never run.

Remove the topology, its three compose services, and the documentation
rows. Every profile the static compose file still defines is now
exercised by a suite.

Also correct the chaos README, which has listed tcp-only and tcp-chain in
its transport table since those two scenarios were deleted in 9e63b42.
Neither has existed for months.
 2026-07-26 14:02:12 +00:00
Johnathan Corgan fa49dc1210 Retire the mesh-public static test topology 
The mesh-public profile ran in neither runner: ci-local's static suite
list carries only static-mesh and static-chain, and the GitHub matrix has
only mesh and chain. It also added no coverage over static-mesh.
ping-test.sh and iperf-test.sh branched mesh and mesh-public together and
exercised the same 20 directed pairs among node-a through node-e, and no
script referenced the external node at all. The convergence waits even
used mesh's peer counts rather than mesh-public's, so the extra link to
the public node was never counted, let alone asserted.

Running it would therefore have added a dependency on a live internet
host (test-us01.fips.network) in exchange for zero additional assertions.
Remove the topology, its five compose services, the script branches that
aliased it to mesh, and the documentation rows. An invocation using the
old profile name now fails on ping-test.sh's unknown-profile guard
instead of silently behaving as mesh.

The config generator's external-node support (external_ip,
is_external_node) stays. It has no consumer now, but it is woven into the
config path every remaining topology uses, so removing it would put the
gating suites at risk for no present gain.
 2026-07-26 13:54:14 +00:00
Johnathan Corgan 0f3f716142 Merge branch 'master' into next 2026-07-25 19:16:01 +00:00
Johnathan Corgan d94300a4f4 Merge branch 'maint' 2026-07-25 19:15:54 +00:00
Johnathan Corgan 0eea7dae13 Make a zero-peer floor unreachable in the shared convergence wait 
wait_for_peers reads the connected-peer count through a pipeline ending in
`|| echo 0`, so a container that never answers contributes 0. That is safe
against a floor of 1 or more, where 0 reads as "not converged yet" and the wait
eventually times out, and it is unsafe against a floor of 0, where the first
read from a dead container satisfies the wait and the caller proceeds as though
convergence had been observed.

No caller passes 0 today; every one passes 1 or more, and the single variable
minimum is advisory. Rather than harden the reader, reject the input: a floor
below 1 is now an error. That makes the shape unreachable instead of repairing
instances of it, and it costs a future caller nothing except a clear message
saying that asserting "exactly zero peers" needs a reader that distinguishes no
answer from zero, which this floor is not.

wait_for_links goes with it. It had no caller anywhere in the tree on any
branch, and it carried the identical fallback, so the only thing it could do was
hand the hazard to whoever called it first. An uncalled helper cannot be wrong
today, which is exactly why it was the risk worth removing rather than the one
worth keeping for symmetry.

The hermetic gate suite gains a case covering both directions, with docker
stubbed so it stays container-free: a floor of 0 is refused with a message
naming why, and the same unreachable container with a floor of 1 still polls its
full budget and times out. Verified by removing the guard and confirming exactly
the three zero-floor assertions red while both floor-of-one controls stay green.
 2026-07-25 19:15:31 +00:00
Johnathan Corgan 1077fd6a7d Stop the firewall peer reader turning a silent container into a count of zero 
wait_for_peers_exact read the connected-peer count through a pipeline ending in
`|| echo 0`, so a container that never answered and a daemon that answered zero
produced the same value. That is only harmless while every caller expects a
non-zero count, which is true here today and is the reason this copy was left
alone when the acl suite's copy was fixed. It leaves the trap armed for whoever
adds the first caller expecting zero: the check would be satisfied on its first
iteration without the property it exists to verify ever being observed.

The read moves into its own function that returns the empty string when the
container does not answer, and the caller treats empty as "no answer" rather
than as a count. A run that never gets an answer now fails saying so, distinctly
from one that answered the wrong count, and the diagnostic peer dump runs only
on the latter, since dumping from a container that cannot answer prints a docker
error rather than evidence.

This is the shape the acl-allowlist suite already uses. The two copies had
diverged on whether a silent container counts as an answer, which is the kind of
disagreement that decides a security assertion in whichever file is read last.

Checked by construction rather than by a green run: driving the old form against
an absent container with an expected count of zero returns success on the first
iteration, and the new form fails; with an expected count of one, the shape both
real callers use, both forms still fail, and a genuine zero from a live daemon
is still reported as zero.
 2026-07-25 19:02:15 +00:00
Johnathan Corgan 0e42c789be Let the firewall suite float its docker subnet 
The compose pinned 172.32.0.0/24 with a per-container ipv4_address, so two
concurrent runs asked docker for the same address space and the second failed
with a pool-overlap error. Request no subnet and let docker assign one from the
daemon pool. Peers address each other by the docker hostname the compose already
sets (host-a, host-b) rather than by literal IP; docker's embedded DNS is
per-network, so the same hostname in two runs resolves inside each run's own
subnet. Nothing this suite asserts on moves as a result: its checks run over the
fips0 overlay, whose addresses are derived from node npubs and are independent of
docker addressing, and the packaged nftables ruleset matches on interface rather
than on any address.

The generated config directory moves under the run suffix for the same reason it
did in the acl suite, and the run teardown gains the matching removal so a run
no longer leaves its directory behind.

Case (b) also wrote curl's output to a fixed path under /tmp. Two concurrent
runs shared that one file, and either run's cleanup landing between the other's
write and read left an empty read, failing the http_code check for a reason
having nothing to do with the firewall. It uses mktemp now.

As in the acl suite, the floating subnet removes one of the two obstacles to
concurrent runs. The compose project name is still fixed; the local CI runner
scopes it externally, a bare hand run does not, and the comment at the site says
so rather than claiming the file is self-sufficient.
 2026-07-25 19:01:56 +00:00
Johnathan Corgan 611f045d33 Let the acl-allowlist suite float its docker subnet 
The compose pinned 172.31.0.0/24 with a per-container ipv4_address, so two
concurrent runs of this suite asked docker for the same address space and the
second failed with a pool-overlap error. Request no subnet instead and let
docker assign one from the daemon pool, which leaves nothing for two runs to
contend for. Peers now address each other by the docker hostname the compose
already sets (host-a through host-f) rather than by literal IP, and docker's
embedded DNS is per-network, so the same hostname in two runs resolves inside
each run's own subnet.

The generated config directory moves under the run suffix in the same change,
because it has to: the generator does rm -rf on it, so a shared directory means
one run deletes the fixtures another is still using, which fails silently rather
than loudly at bring-up. Scoping it requires the generator output path, all five
bind-mount sources per service, and the gitignore entry together; scoping only
the generator leaves the compose reading the unscoped path.

The floating subnet removes one of the two obstacles to running this suite
twice at once, not both: the compose project name is still fixed, and nothing
scopes it for this suite. Recorded at the site so the comment does not claim
more than the change delivers.
 2026-07-25 19:01:16 +00:00
Johnathan Corgan 679c9aaeb2 Retire the Docker mixed-profile suite in favour of the in-process test 
Everything this suite asserted is now covered in-process over loopback by
mixed_profile_nodes_converge_and_forward: mixed Full, NonRouting and Leaf
convergence, the peer degrees, every direct reachability pair, and the
Leaf-to-Full pair that routes through the Full node between them. The
in-process test covers a superset, asserting that multi-hop path in both
directions where the Docker suite covered one.

Retired on evidence that the multi-hop assertion discriminates rather than on a
green run: reverting the leaf root-election gate reds it in roughly a third of
runs, with the leaf reporting itself as tree root. That was the property the
suite was being kept for, and it is the property now demonstrated elsewhere.

Removed from the local runner's suite list, its runner function, the default
flow and the single-suite dispatch, and from the workflow matrix and its five
steps. Parity reports 22 legs a side, one fewer than before and equal across
the two runners. The suite's script, topology and compose profile stay on disk
and it remains runnable by hand; the retirement is recorded with the others in
the runner's deliberately-not-run block.
 2026-07-25 17:36:47 +00:00
Johnathan Corgan da57a57150 Raise the rekey interval in the handshake tests above the validation floor 
The rekey tests configured a one-second interval to put the timer arm within
reach, then backdated the session two minutes so the jitter draw could not
decide the outcome. That worked, but it asks for a configuration the daemon now
refuses to build: an interval at or below the jitter bound saturates to zero on
a negative draw and rekeys on sight for roughly half of sessions, which
validation rejects.

Thirty seconds clears the bound and leaves the existing backdate a margin of
seventy-five seconds against the worst draw, so the tests keep the property
they were relying on. The helper that drives these handshakes now records why
the value cannot simply be made smaller again.

Surfaced by merging the deployed line up, which is the first point at which
these tests met the new validation.
 2026-07-25 17:24:07 +00:00
Johnathan Corgan e30f3b7ddb Merge branch 'master' into next 
Clean merge, no conflicts. The hop-limit helper corrections and the rekey
config validation arrive from the deployed line and apply here unchanged.

Checked rather than assumed, since this branch changed the same subsystem in
the same session: next keeps its own split of the rekey cadence, where the
config flag rides into the core as RekeyCfg::initiate and check_rekey no longer
returns early, and the incoming validation of the same config block sits
alongside it without overlap.
 2026-07-25 17:17:23 +00:00
Johnathan Corgan 66d5be2bfb Merge branch 'maint' into master 
Both maint changes carry over unchanged. The hop-limit helpers follow the
module rename from src/protocol/link.rs to src/proto/link.rs, applied by
rename detection and verified at the renamed path rather than assumed; the old
path is not recreated. The rekey config validation and its jitter-constant
import apply as written, the node module exposing the constant on this branch
as well.

The only conflict was the unreleased changelog, where master's own Changed
entries and the incoming hop-limit entry were both correct for their branch.
Resolved as the union, with the incoming entry appended to master's Changed
list.
 2026-07-25 17:14:30 +00:00
Johnathan Corgan 026a9c6abf Cover the mixed-profile multi-hop path in the in-process mesh test 
The in-process mixed-profile test asserted convergence, peer degrees and every
direct pair, but deliberately left out the Leaf-to-Full pair that routes
through the Full node between them: a leaf holding a small enough address used
to self-elect as tree root and partition the mesh, so the assertion would have
been flaky. That defect is fixed, so assert the pair here, in both directions.
The leaf's peer degree is already pinned at one in the same test, so neither
direction can be satisfied by a direct link.

Failures in this test are root-election partitions keyed on the random
per-node address ordering, which are undiagnosable from a bare assertion
message. Each of the three failure paths now reports every node's address,
elected root and self-rooted flag.

Reverting the self-election gate reds the new pair in roughly a third of runs
with the leaf reporting itself as root, and the assertion is clean over
repeated runs with the gate in place.
 2026-07-25 17:10:44 +00:00
Johnathan Corgan 0cb5574077 Reject rekey settings that fire the trigger immediately and forever 
Config validation said nothing about the rekey block, so two settings that
disable rekey in appearance and hammer it in practice were accepted silently.

after_messages of zero makes the message-count arm true on every poll, since
the trigger tests the counter against it with a greater-or-equal. It reads like
a way to switch the arm off and does the opposite.

after_secs at or below the per-session jitter is the same trap on the timer
arm. Each session offsets the interval by a random value drawn from plus or
minus the jitter bound, so a smaller interval saturates to zero on a negative
draw and rekeys on sight, for roughly half of sessions and not the other half.
The rule is expressed against the jitter constant rather than its current
value, so it tracks if the bound ever moves.

Both are checked whether or not rekey is enabled, so turning it on later cannot
surface a configuration error at a surprising moment. Neither has an upper
bound: a very large value is the established way to disable one arm of the
trigger and stays legal.
 2026-07-25 17:07:57 +00:00
Johnathan Corgan 989ae65fc5 Align the datagram hop-limit helpers with the forwarder's semantics 
The forwarding path was corrected to full IP semantics: local delivery is not
hop-limit gated, and forwarding decrements first and drops at zero. Two helpers
on SessionDatagram were left implementing the old rule. decrement_ttl still
checked before decrementing, and can_forward still answered true at a hop limit
of one, where the decrement leaves zero and the datagram is dropped. Neither is
called anywhere today, which is precisely why they are worth correcting rather
than leaving as an invitation to reintroduce the defect.

The TtlExhausted reject doc described behaviour that no longer exists. The
reject is charged for a transit arrival at one as well as at zero, and is never
charged for a datagram addressed to this node, whose delivery is decided ahead
of the test.

Both helpers are pinned by tests at the boundary, which fail against the
previous implementations.
 2026-07-25 17:05:20 +00:00
Johnathan Corgan 3d390fbafd Make rekey.enabled mean "initiate rekeys" and nothing else 
The responder half of the establish decision was gated on this node's own
node.rekey.enabled. With the rekey now declared in the msg3 negotiation
payload, that flag was the only thing that could divert a msg3 whose marker
matched the session we hold, and it diverted it to a msg2 resend while the
initiator had already installed its pending session and would cut over on its
own timer regardless. A pair configured with the flag true on one end and
false on the other therefore parted company at the initiator's cutover and
carried no traffic in either direction until the link-dead timer.

Drop the flag from that gate. The arm now turns on the sender's declaration
matching the session we hold, which is the only signal that was ever
authoritative for the decision, and the snapshot field it read is gone since
every initiator-side use reads the config directly.

Removing the gate exposed a second reading of the same flag. check_rekey
returned early when rekey was disabled, and the drain-expiry arm of the rekey
poll is the only thing that releases a demoted session and its index. A node
that does not initiate had never accepted a rekey before, so it never
accumulated a drain; now that it does, the early return would pin the previous
session and its allocator index forever. The flag rides into the core as
RekeyCfg::initiate instead: the polled cutover and the trigger are gated on it,
the drain is not, and the shell runs the cadence unconditionally.

The cutover arm has to stay gated. The peer snapshot carries no rekey role, so
a pending session stored as responder satisfies that arm exactly as one we
initiated does; ungating it would move a non-initiating node's cutover off the
peer's k-bit flip and onto the poll.

Covered by a two-ended test asserting the pair converges on the same session
indices under an asymmetric setting, a node-level test that a non-initiating
responder actually releases the demoted session and index once the drain window
expires, and a core test pinning which arms a non-initiating snapshot yields.
The existing msg2-resend test reached its arm only through the removed gate and
now reaches it through the health conjunct instead.
 2026-07-25 17:05:00 +00:00
Johnathan Corgan 30e1dcb989 Name this branch's third reader of the address-to-link map 
The doc comment merged up from the maintenance line lists the map's live
readers, and on this branch there is one more: the anonymous-beacon dial
dedup in poll_transport_discovery, which has no counterpart there.

It reads the map with an address supplied by transport discovery, and the
shared-media dial paths resolve before registering, so it compares a key
written in the same form it reads and is not exposed to the mismatch the
comment warns about.
 2026-07-25 01:56:10 +00:00
Johnathan Corgan 7505b86a5d Merge branch 'maint' 
Carries the address-to-link map's corrected doc comment. The only
conflict was the adjacent line naming the discovery config keys, which
this line renamed to node.lookup.* and the maintenance line did not; kept
this branch's names.
 2026-07-25 01:53:15 +00:00
Johnathan Corgan b920200569 Merge branch 'master' into next 2026-07-25 01:53:15 +00:00
Johnathan Corgan 1eabe08575 Correct what the address-to-link map claims to be for 
The doc said the map "enables dispatching incoming packets to the right
connection before authentication completes." It does not: find_link_by_addr
has no callers outside its own tests on any branch, and encrypted frames
dispatch by session index. Its live readers are the msg1 admission fast
path and the duplicate-inbound-handshake check.

The comment mattered because the map has a trap that invites exactly the
misuse it was advertising. An outbound dial registers the literal
configured address string, which may be a hostname, while an inbound
packet carries the resolved form; TransportAddr compares byte-wise, so
the two never match and a lookup keyed on an inbound address returns "no
such link" for every hostname-configured peer rather than failing. The
entry is also single-valued per key, so an inbound handshake overwrites
an outbound dial's entry for the same address.

Both current readers survive this because each compares a key written in
the same form it reads. That is a property of those two call sites and
not of the map, so the comment now says so, and says not to key a
peer-identity question on it.

Documentation only; no behaviour changes.
 2026-07-25 01:50:40 +00:00
Johnathan Corgan 4a448cbe21 Merge branch 'master' 
Carries the rekey counter-arm boundary test and the removal of the
closure-based trigger test. Applied cleanly with no adaptation: the
trigger predicate is byte-identical on both lines, and the claim the new
test rests on was re-measured here rather than assumed to carry — widening
the bound to `p.counter >= 1` reds exactly one test on this line too.
 2026-07-25 00:40:14 +00:00
Johnathan Corgan b29908ba8a Pin the rekey counter arm's boundary against the real trigger 
The send-counter arm of the rekey trigger had no test that could fail for
the reason it existed. The integration suites never reached it: they
inject after_messages = 65536 against a ping-driven workload, so every
rekey they observe is time-triggered. The unit test that looked like the
guard reproduced the trigger's OR predicate as a local closure and
asserted against its own copy, so deleting the arm from the real
predicate left it green.

That is the coverage hole that let the data-plane drop across a
message-count-triggered rekey survive: the defect is precisely a rekey of
a young session, which only the counter arm can produce.

Replaces the existing at-threshold check with a boundary test driving
poll_rekey, holding the time arm off so the counter is provably what
decides. The silence-below half is the part that is new, and it earns its
place by measurement: widening the bound to `p.counter >= 1` reds exactly
one test in the library suite, this one. Deletion, narrowing to `>` and
inversion are caught here too, though the time arm's negative tests
already catch a bound removed altogether.

Removes the closure-based test rather than repairing it. Its one
assertion over real code, that a fresh session entry's jitter lies within
the symmetric bound, is already covered over 100 samples elsewhere.

Note this leaves the deployed maintenance line untested on the same arm:
its trigger is inline in an async method that reads the clock directly,
with no pure function to drive, so the same test cannot be written there
without the sans-IO seam this line already has.
 2026-07-25 00:35:16 +00:00
Johnathan Corgan 537f0b32c7 Merge branch 'master' 
Carries the in-process rekey continuity test and the retirement of the
three rekey Docker suites onto the XX line.

The continuity test was written against master's IK handshake, where the
responder decided rekey-versus-fresh-dial from session age and the test
had to backdate both sessions past a 30s acceptance gate. Here the
responder classifies from the rekey marker the initiator declares in its
msg3 negotiation TLV and reads no clock at all, so the backdating is
dropped: it would age the session past nothing, and leaving it in would
tell a reader a gate exists that does not.

Checked that the test still discriminates rather than merely observing a
session index change. Suppressing the declaration so the responder falls
back to the cross-connection path reds the post-cutover assertion in 8 of
16 runs, which is the address-ordering coin flip the original data-plane
drop had; the healthy tree is 0 of 16 red. The 30s backdate makes no
difference to either rate, which is what confirms it was vestigial here.

The suite wiring keeps mixed-profile, which master does not have, and
drops the three rekey suites from the local suite list, the runner
functions, the default flow, the --only dispatch and the GitHub matrix
together, so check-ci-parity.sh stays green at 23 legs a side. The
retirement note is retargeted at this branch's coverage: the establish
decision tests live in src/proto/fmp/tests/core.rs and src/peer/machine.rs
here, not in the establish_chartests.rs file master keeps.
 2026-07-24 23:58:42 +00:00
Johnathan Corgan aa204cf729 Swap on a dial that crosses a rekey, and cover the marker's two ends 
The rekey marker's cross-connection arm declined to swap while a rekey of
ours was in flight, reasoning that the swap rewrites the session and both
indices while touching no rekey state, so it would orphan that rekey. The
reasoning was one-sided and the arm was wrong: this is half of a two-sided
tie-break whose other half reads only whether a peer exists and the address
ordering, and cannot see our rekey. Declining here while the peer's outbound
half swaps anyway leaves the two ends on four distinct indices with neither
holding a pending session, dead both directions until the link-dead timer.
Measured against the pre-marker tree, which converges: it holds the crossing
dial as pending and adopts it at cutover.

The orphaned-rekey hazard is real but belongs to the executor. The swap now
abandons the rekey it displaces and frees that index first, exactly as the
rekey-responder path already does when it loses the dual-rekey tie-break, so
the classifier can take the arm unconditionally.

Two further failure paths of the marker are tightened. A malformed marker
tore the handshake down without freeing the index msg1 allocated, and that
arm sits ahead of the ACL gate, so any peer able to complete a msg3 could
grow the allocator set one entry at a time. A rekey with no session index to
declare silently omitted the marker, which puts a real rekey back on the
cross-connection path; it is unreachable today and is now asserted rather
than skipped.

On the test side, the marker's wire contract gains coverage: the accessor
round-trips, an unknown TLV field leaves it alone in both directions, and a
well-formed marker of the wrong length is an error rather than an absence,
which is the case the container codec cannot see and the whole reason the
accessor is fallible.

Two node-level tests now assert both ends rather than one. A rekey fired by
message count on a zero-age session is the mechanism behind the datagram
drops, and nothing exercised it: it fails on the pre-marker tree and passes
here. The mutual-dial test gains an index-pair assertion, since every
per-end predicate it checked holds when the ends have diverged.

One existing test had stopped reaching its own arm. Driving a bare second
handshake into a rekey-disabled peer now lands on the cross-connection arm,
which that flag does not gate, and every assertion held there too; it is
retargeted onto a declared rekey, which is the real route to the duplicate
arm, and asserts the session is left alone so the two arms are told apart.

Documentation that described the removed session-age floor is rewritten
around the declaration.
 2026-07-24 21:48:49 +00:00
Johnathan Corgan 911a000d85 Declare a rekey explicitly in msg3 instead of inferring it from session age 
A rekey and a fresh dial both arrive as a new msg1 on a new link, so the
responder has to tell them apart at msg3. It cannot: the distinguishing
fact is internal to the initiator and nothing on the wire carries it.
Session age stood in for it and could not do the job - a
message-count-triggered rekey fires on a young session, so a real rekey
landed below the floor, was resolved as a cross-connection, and left the
two ends of the link on different session indices.

Carry the answer instead. The initiator adds a negotiation TLV to its
rekey msg3 naming the session it replaces, using the index the receiver
allocated so the receiver can match it against its own. The shell
resolves that into a three-valued claim - none, matching, mismatched -
and the classifier branches on it.

A mismatched marker resends msg2 rather than rejecting: the initiator has
already installed its pending session and will cut over on its own timer,
and the reject path sends nothing back, so tearing down would strand the
link in the way this defect already does. A fresh dial arriving while our
own rekey is in flight also resends, because the cross-connection swap
rewrites both indices and touches nothing else, which would leave that
rekey pointing at a session the counterpart no longer holds. The age
floor was excluding that case incidentally.

The floor and the session-age snapshot field are gone with it, so the
establish snapshot no longer reads the clock at all.

Both node-level rekey tests now drive a real rekey through the
initiator's own trigger. They previously reached the rekey arms by
backdating a session and re-handshaking, which tested the age proxy this
replaces; one of them was reaching its assertions through the duplicate
arm rather than the arm it names. Age them well past the trigger, since
the jitter is +/-15s and a smaller margin makes firing depend on the draw.

Incomplete: no TLV round-trip test, no two-ended convergence test, no
rekey-crossed-with-fresh-dial case, and no integration run yet.
 2026-07-24 20:27:55 +00:00
Johnathan Corgan 2223763dab Let the host shell override RUST_LOG for the static test harness 
The static suites hardcoded RUST_LOG=info, so raising the log level for a
node under test meant editing the compose file. Take it from the
environment with info as the default, matching the passthrough the same
block already uses for the worker-pool overrides.

Rendering is unchanged when RUST_LOG is unset, which is how CI invokes it.
 2026-07-24 19:11:27 +00:00
Johnathan Corgan 694d03375b Stop a leaf node from self-electing as tree root 
A leaf-profile node that held the smallest NodeAddr would self-elect as
tree root, but its peers refuse a non-full node as a parent, so it formed
an isolated second root and partitioned the mesh: a multi-hop session from
the leaf to a non-adjacent full node then failed because the far node could
not route a handshake reply back into the leaf's separate coordinate tree.

Gate tree-state self-election on a new self_is_leaf flag, set from the node
profile at construction. A leaf now attaches under its full upstream and
holds that subtree's coordinate for its own routing. That coordinate's
self < root would be rejected by the root-min wire check, but a leaf never
announces it; it reaches peers only via the coordinates carried on the
leaf's session frames, and the upstream already advertises the leaf in its
bloom filter, so the far node routes back through the upstream.

Also fix Node::with_identity to derive the node profile, leaf-only flag,
and bloom state from the config, matching Node::new; it previously
hardcoded the full profile and silently dropped a configured leaf or
non-routing profile.

Covered by sans-IO decision tests (leaf does not self-elect; leaf keeps its
coordinate under a larger-rooted parent) and an end-to-end multi-hop
regression, each confirmed to fail if the gate is reverted.
 2026-07-24 14:42:08 +00:00
Johnathan Corgan bcfe9270bb Add an in-process mixed-profile lifecycle test over loopback 
Brings up a Full/Full/NonRouting/Leaf mesh matching the mixed-profile
Docker topology, asserts each node peers to the degree its role and
topology dictate (A=3, B=2, C=2, D=1), and verifies end-to-end data
between every directly reachable pair (Full-Full, Full-NonRouting,
Full-Leaf). Adds run_tree_test_with_profiles, a per-node-profile sibling
of run_tree_test.

The Leaf-to-Full multi-hop the Docker suite also checks is deliberately
left out for now: a Leaf node's multi-hop session initiation on the XX
line is not yet reliable, so the Docker mixed-profile suite is retained
for that path rather than replaced with a flaky assertion.
 2026-07-24 05:02:05 +00:00
Johnathan Corgan 9213cce6c4 Retire the rekey Docker suites; timing, continuity and variants are in-process 
The three rekey integration suites (rekey, rekey-accept-off,
rekey-outbound-only) are dropped from both runners. Their coverage now
lives in fast, deterministic in-process tests:

- rekey timing and choreography (trigger, K-bit cutover, drain, jitter,
  guards) in the sans-IO poll_rekey tests (proto/fsp, proto/fmp);
- data-plane continuity across a real cutover in
  rekey_cutover_preserves_data_plane (a real IK rekey over loopback);
- the accept-off dual-init regression and the udp.outbound_only rekey
  loop in the should_admit_msg1 and dual-init characterization tests.

Drop them from both runners in lockstep so the parity guard stays green,
and record the retirement in the deliberately-not-run block with a pointer
to the standalone runner. The rekey-test.sh script stays on disk.
 2026-07-24 03:51:13 +00:00
Johnathan Corgan f44de56fd8 Add an in-process test that a rekey cutover preserves the data plane 
Drives a real IK rekey handshake to K-bit cutover between two loopback
nodes and asserts an encrypted datagram sent after the cutover decodes on
the new session, with no spurious peer teardown. The rekey is forced
deterministically: rekey.after_messages = 1 crosses the initiator's
trigger on the first sent datagram, and both sessions are backdated past
the responder's 30s rekey-acceptance gate, so no wall clock is read.

The cutover is proven to actually occur (the live session index changes),
which is what makes the continuity assertion meaningful rather than
vacuous. Adds a make_test_node_with_config loopback helper for setting the
rekey thresholds. The rekey timing and choreography decisions themselves
are already covered exhaustively by the sans-IO poll_rekey tests.
 2026-07-24 03:39:47 +00:00
Johnathan Corgan e1543d201a Merge branch 'master' into next 
# Conflicts:
#	.github/workflows/ci.yml
#	testing/ci-local.sh
 2026-07-24 02:45:37 +00:00
Johnathan Corgan 768df453b7 Merge branch 'maint' 2026-07-24 02:40:19 +00:00
Johnathan Corgan 9846e85705 Retire the admission-cap Docker suite; the inbound gate is proven in-process 
The inbound max_peers early-gate in handle_msg1 (silent-drop a Msg1 from
a net-new identity at saturation, send no Msg2, admit no peer) is
unit-tested over a real UDP socket by
handle_msg1_silent_drops_at_cap_for_new_peer in src/node/tests/unit.rs:
it saturates a node, sends a Msg1 from a fresh identity, and polls the
sender socket to assert no Msg2 comes back — the same wire-observable
discriminator the Docker suite's tcpdump used — with a sibling test for
the existing-peer bypass. That is a deterministic superset of the Docker
packet-capture assertion.

Drop it from both runners in lockstep (its *_SUITES array, run function,
default-flow loop, --only dispatch arm, and the GitHub matrix leg with
its steps) so the parity guard stays green, and record the retirement in
the deliberately-not-run block with a pointer to the standalone runner.
The admission-cap-test.sh script stays on disk and runs by hand.
 2026-07-24 02:31:43 +00:00
Johnathan Corgan 41ce64ba82 Retire the acl-allowlist Docker suite; the decision is proven in-process 
The ACL admission decision is exhaustively unit-tested per npub over real
loaded allow/deny files (src/node/acl.rs test module: allow-match-wins,
allowlist-miss falls through, deny-only, deny-all, allow_all override,
deny-after-allowlist-miss), and the inbound and outbound handshake-
admission paths are covered in-process over the loopback transport
(src/node/tests/acl.rs: inbound msg1 denial, outbound denial, reload).
The Docker suite's only unique coverage was admission over a real UDP
transport, which every other real-transport suite already exercises, and
its operator-facing log assertion.

Drop it from both runners in lockstep (its *_SUITES array, run function,
default-flow call, --only dispatch arm, and the GitHub matrix leg with
its steps) so the parity guard stays green, and record the retirement in
the deliberately-not-run block with a pointer to the standalone runner.
The testing/acl-allowlist/ suite stays on disk and runs by hand.
 2026-07-24 02:29:38 +00:00
Johnathan Corgan 7cbe1d3d4e Retire the smoke-10 chaos scenario; convergence is covered in-process 
smoke-10 was a no-stressor 10-node tree-convergence sanity check (netem
off, no ping). Its subject, spanning-tree convergence and root election,
is now covered in-process, faster and deterministically, by the loopback
spanning-tree harness (src/node/tests/spanning_tree.rs: ring, star,
chain, 100-node and disconnected-component convergence) plus end-to-end
datagram delivery (src/node/tests/forwarding.rs). Real-UDP convergence
smoke still runs via static-mesh and the other scenarios' baseline
assertions, so no Docker coverage is lost.

Drop it from both runners in lockstep (the CHAOS_SUITES list and the
GitHub chaos matrix) so the parity guard stays green, delete the scenario
YAML, and update the chaos README.
 2026-07-24 01:43:20 +00:00
Johnathan Corgan 28421c4b0e Merge master: retire cost/congestion chaos scenarios 
Bring the six retired cost-based parent-selection chaos scenarios up to
next, resolving the ci-local.sh suite-list conflict (next carries the extra
mixed-profile and admission-cap suites). next already has the equivalent
sans-IO cost and transport-drop coverage, so src is unchanged; this takes
only the scenario removals and the CI-list, README and comment updates.
 2026-07-23 23:46:26 +00:00
Johnathan Corgan 08547e8dff Merge maint: retire cost/congestion chaos scenarios 
Bring up the retirement of the six cost-based parent-selection chaos
scenarios (cost-reeval, cost-avoidance, cost-stability, depth-vs-cost,
mixed-technology, bottleneck-parent), which tested a decision the Docker
harness could not exercise reliably.

master already carries the equivalent sans-IO coverage in
src/proto/stp/tests (effective-depth cost selection, hysteresis, cost
degradation) and its transport-drop tests, so this keeps master's src
unchanged and takes only the scenario removals and the CI-list, README and
scenario-comment updates from maint.
 2026-07-23 23:44:31 +00:00
Johnathan Corgan 08a226fb63 Test cost-based parent selection and kernel-drop detection as unit tests 
The cost-selection chaos scenarios (cost-reeval, cost-avoidance,
cost-stability, depth-vs-cost, mixed-technology, bottleneck-parent) tested
TreeState::evaluate_parent's decision logic through a Docker mesh that could
not exercise it reliably: the tree roots at whichever node holds the smallest
NodeAddr, MMP link costs take several measurement windows to settle, and the
parent hold-down plus hysteresis timing all confound the outcome. A
deterministic link-cost flap still produced zero periodic parent switches in a
full run.

Replace those six scenarios with deterministic unit tests in src/tree/tests.rs
that drive evaluate_parent directly: cheaper-link selection at equal depth,
switch-on-cost-change, hysteresis suppressing a marginal change while allowing
a significant one, and the depth-versus-cost effective-depth tradeoff. Each is
constructed so that breaking the cost or hysteresis logic makes it fail.

The congestion kernel-drop signal (SO_RXQ_OVFL) cannot be provoked
deterministically in Docker: a fresh daemon reader keeps up with
container-speed traffic, so the socket receive queue never overflows (an
unshaped run with a 4 KB buffer and heavy traffic recorded zero drops on every
node). Extract the drop-detection edge -- read the cumulative counter, fire an
event only on the transition into a new drop burst -- into
TransportDropState::observe_drops and unit-test it directly. congestion-stress
keeps its ECN and MMP congestion-signal assertions, which do need the real
shaped bottleneck queue.

Remove the retired scenarios from both CI runners and update the chaos README.
 2026-07-23 23:33:01 +00:00
Johnathan Corgan 45406292aa Merge branch 'master' into next 2026-07-23 19:52:08 +00:00
Johnathan Corgan abf2f9ba48 Merge branch 'maint' 2026-07-23 19:51:58 +00:00
Johnathan Corgan be5deee814 Make the gateway integration suite safe for concurrent CI runs 
gateway-lan pinned 172.20.1.0/24 and fd02::/64, so two concurrent local CI
runs collided on "Pool overlaps" at network creation. The IPv4 subnet has no
consumer -- the whole gateway LAN path is IPv6 -- so drop it and let docker
auto-assign. The IPv6 side cannot float, because the LAN clients' resolv.conf
pins the gateway's address as their nameserver and that must be a literal
known before they start, so claim a free /64 per run and thread the prefix
through the compose address pins, a generated resolv.conf, and the test via a
single exported variable.

The claim and the external network ship as a harness-only compose overlay
applied by run_gateway; the base compose keeps a normal gateway-lan network,
so the GitHub matrix and any standalone bring-up are unaffected. With no claim
every address renders exactly as before.
 2026-07-23 19:50:17 +00:00
Johnathan Corgan bb9bca4d83 Accept a degraded systemd as a booted one in the deb-install boot wait 
The boot check piped `systemctl is-system-running --wait` into `grep -qE
'running|degraded'`, but the script runs `set -o pipefail` and is-system-running
exits non-zero for `degraded`. So the pipeline failed on a degraded system even
though grep matched, and the wait looped to its timeout. The older distros reach
`running` (exit 0) and passed; debian:trixie and ubuntu:26.04 reach `degraded`
because a unit that cannot run in a container (systemd-modules-load) fails, so
they timed out at every ceiling -- which is why the earlier timeout increase did
nothing.

Capture the state string and test it directly instead of trusting the pipeline
exit, so a degraded-but-booted system is accepted as intended. Validated: all
five distros pass, including the two that previously failed.
 2026-07-23 19:50:07 +00:00
Johnathan Corgan 2bc0345174 Make the cost-based chaos assertions reliable under parallel CI load 
cost-avoidance and mixed-technology decide a parent by measured link cost
(etx * (1 + srtt_ms/100)). Under the local CI's 4-way-parallel chaos the fast
fiber link's measured srtt spikes from host scheduling and can exceed the
Bluetooth link's, flipping the choice and reddening a run that proves nothing
about the daemon. The old margin was ~0.4 cost units, which a ~24ms one-sided
scheduling blip closes.

Widen the Bluetooth delay to 150-250ms so the margin dwarfs any plausible
one-sided spike. The link still establishes well within the handshake budget
(a ~500ms RTT against a 30s stale-handshake window), so the losing candidate
is still a real, established peer rather than an absent one.

Add a netem mutation exclude_edges option and use it in mixed-technology to
pin the two links n08 chooses between, so a random degradation can't flip the
asserted comparison either. An unknown excluded edge is a hard error, since a
silent no-op would reintroduce the flakiness it exists to remove.

Validated by running both scenarios repeatedly under heavier-than-CI
concurrent load: the parent choice holds every time.
 2026-07-23 18:58:49 +00:00
Johnathan Corgan 9b7a27f219 Claim the chaos network range instead of deriving it, ending the collision 
Two concurrent ci-local runs could not both run chaos. Each child's subnet came
from its position in the suite list, so both runs walked 10.30.0 through
10.30.12 and requested identical ranges; whichever reached the daemon first won
and the other died with a pool overlap. A run index or hashed offset would only
make that unlikely, and it is precisely the failure being removed.

The simulator now claims its range: attempt-create on a candidate, advance on
docker's own overlap error, fail loudly on anything else. Docker's address pool
becomes the arbiter, so an overlap is impossible rather than improbable. The
claim lives in the sim rather than in ci-local because only the process that
creates the network can advance on conflict, and because it fixes the bare
chaos.sh path too, which a ci-local-only fix would have left broken.

The generated compose now declares the network external over the claimed one,
and teardown releases the range from both paths, including the setup-failed
path where a run that fell over after claiming still holds one.

Ordering matters here and is not obvious: node IPs derive from the subnet
during topology generation, and traffic shaping keys its filters on those
addresses, so the claim happens before the topology exists. Claiming later
would give a network on one range and filters on another, which does not fail
at bring-up and instead leaves the shaping matching nothing.

--subnet survives as an explicit pin for when a known range is wanted, and now
fails loudly if that range is taken rather than silently overlapping. ci-local
no longer passes it.

Validated by running two instances of the same scenario deliberately
concurrently: they claimed 10.30.1.0/24 and 10.30.0.0/24, both exited 0 with
their assertions passing, and both released their networks. The negative
control holds too: with a squatter on a pinned range the run aborts with a
message naming the range rather than proceeding.
 2026-07-23 17:09:07 +00:00
Johnathan Corgan f478f51afe Assert mixed-technology's sound criterion, and correct the one that never was 
With the root pinned, n08's test subject is assertable for the first time: its
two candidates sit at equal depth with fiber-grade default netem on one side and
Bluetooth on the other, which is exactly the preference this scenario exists to
test, and it takes the fiber parent in four of four runs. Encoded as written
rather than calibrated from the runs, since "should pick n03" is the spec and
the runs merely confirm it is met.

n06 is deliberately absent, and its documented criterion is corrected rather
than left standing. The header called n03 a fiber parent for n06; the netem
policies in the same file say that link is WiFi and the other candidate is
Bluetooth, so n06 chooses between two impaired links. Its only fiber-grade link
reaches a node at depth 3 that is never a good parent. There was never a reason
to expect it to prefer n03, and the two-of-four split observed is correct
behaviour: in a run where it took the Bluetooth peer, the fiber-labelled
candidate measured a link cost of 3 against the other's 1, so the daemon chose
the cheaper effective depth as it should.

Validated by replaying the assertion against all four archived runs, then
breaking it three ways to confirm it can fail, then one live run for the wiring.
One of the breaks expects n06 to take n03 and reports that it chose n02, which
is the direct demonstration of why n06 is not encoded.

cost-reeval records that its subject now fires in two runs of three, against
zero of five when it was structurally impossible, and why that rate is still not
one an assertion can rest on.
 2026-07-23 17:04:23 +00:00
Johnathan Corgan 9d92cfeab4 Correct three scenario headers that documented conclusions since disproved 
congestion-stress said the transport drop-detection path "is exercised by
nothing". Widening the scan beyond this one scenario finds it firing in six
others, so the path is live and what is dead is this scenario's ability to
provoke it. Records the arithmetic that explains why: at the 1 Mbps cap the
receive queue needs tens of milliseconds of reader stall to overflow, and the
ingress policer discards the same traffic a layer below the socket.

mixed-technology said the implementation does not do what its criteria say.
It does. The criteria were conditioned on a tree rooted at n01 while the mesh
rooted at n09, under which n08's choice was settled by depth before link
technology could matter. The archived parent distributions are relabelled as
describing the old root so nobody calibrates against them.

cost-reeval now records why it had no assertion about its own subject: its
designated subject was the root, so the parent switch it exists to observe
could not occur. The historical logs show the switch on n04 in 13 of 14 runs
when the root still wandered, and only on n01 in the five runs before the fix.

Each header now names what remains to be done and where it is tracked, rather
than leaving a disproved conclusion in place for the next reader to inherit.
 2026-07-23 15:47:28 +00:00
Johnathan Corgan 6319c7f577 Describe the trailing command accurately in the gate's finding message 
For a value-returning helper the trailing echo is the return mechanism, not a
log call. The hazard is the same either way -- it fixes the exit status at zero
-- but calling it a log call misdescribes four of the sites the gate now finds.
 2026-07-23 15:46:04 +00:00
Johnathan Corgan d4a2504f99 Teach the trailing-log gate to see command-substitution call sites 
Found while fixing count_log_pattern: the gate reported that function clean
even though it ends in echo and both callers consume its status. Its call-site
patterns only matched direct forms -- if fn, while fn, fn ||, fn && -- so a
status consumed through command substitution was invisible, because the line
begins with the variable rather than the function name.

That is not an exotic form. It is how a shell function returns a value, and it
is precisely the shape of the swallowed-failure family this gate exists to
catch, so the gate was blind to a large part of its own stated class.

Three patterns added for the assignment, if-guarded and test-expression forms.
The extension finds four real instances, all value-returning helpers whose
trailing echo made their exit status unconditionally zero, and each now carries
an explicit return 0. Also corrects the finding message, which described the
trailing command as a log call; for these it is the return mechanism, and the
hazard is that it fixes the status either way.

Validated by breaking what it guards: a probe reintroducing the defect shape
behind a command substitution is reported and exits 1, where before the
extension it would have passed.
 2026-07-23 15:45:37 +00:00
Johnathan Corgan bf173d8d98 Pin the chaos mesh root to n01 so scenario diagrams describe the real tree 
Every chaos scenario draws n01 at the top of its topology, and until now that
held in three of thirteen. The mesh roots itself at the numerically smallest
NodeAddr, which is a hash of the node's public key and bears no relation to the
node numbering, so which node ended up as root was effectively arbitrary.

The consequences were not cosmetic. cost-reeval rooted at n04, its own
designated test subject, so that node had no parent and the periodic parent
switch the scenario exists to observe could not occur at all. mixed-technology
rooted at n09, which put its two documented parent criteria out of reach and
made correct cost-based selection look like a defect.

Identities are still derived from the mesh name exactly as before and are still
deterministic. What changes is which node id holds which one: they are now
assigned in NodeAddr order, so n01 holds the smallest and is the root. Verified
against a model of the daemon's own derivation that reproduces the previously
observed root for every scenario and n01's address byte for byte; all twelve
pinned scenarios now root at n01.

smoke-10 deliberately opts out via pin_root: false so that root election from an
arbitrary key distribution stays exercised somewhere. Its assertion is a
convergence floor and is root-agnostic, which is why it is the cheapest home for
that. The new key is rejected when non-boolean, and a near-miss spelling is
rejected as unknown; both checked.

Not yet established: the trees themselves change, so the parent-dependent
assertions in bottleneck-parent, cost-avoidance and cost-stability need
re-deriving against live runs. Those are held until the in-flight CI finishes,
because a chaos run rebuilds the shared fips-test image that run is using.
 2026-07-23 15:43:10 +00:00
Johnathan Corgan 34a2561af7 Close the remaining "caught" and "produces red" harness holes 
Four sites where a check could report a verdict it had not established, or
detect a failure and then not turn it red.

assert_no_panic in both NAT suites read `docker logs ... || true`, so a
container that could not be read produced empty output, matched no panic
pattern, and returned success. The assertion's failure mode was
indistinguishable from its success condition. It now reports that absence of
panics is not established.

deb-install's apt capture discarded the exit status, so a failed docker exec
gave an empty capture that matched neither error pattern and reached the pass
branch. The status is now kept and checked before the output is inspected, with
the output still printed on failure.

wait_for_systemd printed a warning and returned success on timeout, so every
check after it read a system that may not have started its units. It now returns
non-zero and the caller abandons that distro leg rather than testing an
unstarted system.

interop's copy of count_log_pattern carried the same defect fixed in rekey: a
node whose logs could not be read contributed zero to eight expect-zero
assertions. Fixed the same way, and its consumer now reports the unreadable case
rather than comparing a sentinel against zero.

Each validated by breaking what it guards and by confirming the healthy path is
unchanged: an absent container now fails each check where it previously passed,
a readable panic-free container still passes, a real panic is still caught, and
a genuinely successful install still passes.
 2026-07-23 15:36:59 +00:00
Johnathan Corgan c8a0ac5fca Stop a node whose logs cannot be read from counting as a clean node 
count_log_pattern summed a per-node `docker logs | grep -c ... || true`, so a
node the harness could not read contributed zero. The six assert_zero_count
callers are negative health assertions -- no panics, no ERROR lines, no AEAD
decrypt failures, no rekey msg2 failures -- and a contributed zero reads as
clean, so one unreadable node silently weakened the assertion and all of them
voided it. This is the family the harness-fallback issue was raised to high
priority for, and it is the one remaining audit residual that can make a green
rekey run lie about protocol code.

The read now fails the count rather than degrading it, printing a sentinel that
names the container. Both callers split the declaration from the assignment,
because `local c=$(fn)` takes local's exit status and discards the function's --
which is how the original defect stayed invisible.

Validated by breaking what it guards rather than by observing green: against
absent containers the old reader returns 0 and assert_zero_count passes
vacuously, while the new one returns rc=1 and reports a failure. Checked the
other direction too, since a fix that reds a legitimately clean run is no use: a
genuine zero across readable nodes still returns 0, and the sum is unchanged at
2+1+0 for a shimmed three-node read.
 2026-07-23 15:30:01 +00:00
Johnathan Corgan e6d99d62c4 Merge branch 'master' into next 
# Conflicts:
#	testing/ci-local.sh
 2026-07-23 14:13:24 +00:00
Johnathan Corgan c58149b0b5 Merge branch 'maint' 2026-07-23 14:09:43 +00:00
Johnathan Corgan 55331a80b5 Gate on shell functions whose exit status is a log call's 
A bash function returns its last command's status, so one ending in a log
call returns 0 whatever it did, and a caller written as `if func` or
`func || fail` has a gate that cannot fire. This tree found two in one
day: run_chaos ended in record, making every chaos row unconditionally
green since 2026-03-09, and build_fips_for_e2e ended in log, letting five
end-to-end legs test the previous commit's binary and report green. Both
were repaired one at a time. This is the rule that catches the next one.

What it enforces is a contract, not a bug hunt: a function whose status a
caller tests must end in an explicit return rather than leaving its
success value to whatever the last log call produced. That distinction is
deliberate and worth stating, because all three instances found in the
tree were benign — each failure path already returned early, so the
trailing echo reported a real success. Reading the function is the only
way to know that, and the next edit that puts an unguarded command before
the final log converts the benign shape into the defect with nothing to
notice. An explicit return costs a line and makes the class unreachable.
The three are given one here.

Scoped on the call sites rather than the definitions, the same way
check-log-strings.py scopes on what a grep reads rather than what its file
mentions: 315 functions scanned, 42 end in a log call, and only the ones
whose status something consumes are reported. A function that ends by
reporting and is called for its output is idiomatic and is left alone.
Without that scoping the finding list would be 42 long and would stop
being read.

Validated by construction rather than by a green run: injecting a copy of
build_fips_for_e2e's exact shape — unguarded docker cp, then a log as the
last statement, with a `|| { ...; return 1; }` caller — makes the checker
report it and exit 1.

Wired into both runners beside the parity and log-string checks.
 2026-07-23 14:09:04 +00:00
Johnathan Corgan 3181861341 Assert the Link MMP pane exists before asserting its colour 
mmp_focused_pane_indicator checked that the unfocused Link MMP title is
not cyan with assert_ne! over fg_at. fg_at is find(..)? mapped to the
cell's foreground, so it returns None for a title that was never drawn,
and None != Some(Cyan). The assertion therefore passed just as happily
when the pane was missing entirely as when it was present and unstyled.

Assert presence first, so the colour check means "not highlighted"
rather than "not there".

Demonstrated rather than argued. Renaming the pane title in mmp.rs so
"Link MMP" is never rendered leaves the original assertion passing, and
makes the new one fail with the message it was given. Both files restored
after.

This is the only instance of the shape: it is the sole assert_ne! over
fg_at or find in the fipstop tree, and the only assert_ne! in snapshots.rs
at all.

Quartet clean: fmt, build, clippy --all-targets -D warnings, test --lib
at 1376 passed / 0 failed / 7 ignored.
 2026-07-23 14:00:49 +00:00
Johnathan Corgan 0f27bdbd2c Give the UDP microbench a reason and record the retry asymmetry 
The UDP recv microbenchmark carried a bare #[ignore] while its sibling in
the link module carries a self-describing one. Match it, so the reason
appears wherever the test is listed rather than only in the doc comment.

Record why [profile.ci] retries = 2 is not applied locally. The two gates
disagree on purpose: the hosted runner retries a flaky test twice, the
local sweep fails on the first failure. It exists for shared-runner packet
loss, a property of that environment rather than of the code, and applying
it locally would suppress a real local flake — a failure that only
reproduces under load is a robustness bug to fix, not to retry past.
Keeping the local sweep strict is what makes it the sharper gate. An
undocumented asymmetry is indistinguishable from an oversight, which is
why this is written down rather than left to be rediscovered.

The cost is stated rather than hidden: a test that fails once and passes
on retry is reported green with no separate signal, so a genuine
intermittent failure can be absorbed. If that starts mattering the fix is
to surface retried-but-passed tests, not to drop the retries.

Quartet clean: fmt, build --workspace, clippy --all-targets -D warnings,
and test --lib at 1376 passed / 0 failed / 7 ignored.
 2026-07-23 13:48:11 +00:00
Johnathan Corgan d3eaad543d Make a skipped check visible in the verdict rather than in scrollback 
Two suites could report a clean pass for work that did not run.

stun-faults skips Phase 2 when tc netem is unavailable, announcing it
several hundred lines above the result and then printing a bare
"stun-faults-test passed". The verdict line now carries the count and
names each skipped phase, and a run in which every phase was skipped
fails outright, since it tested nothing.

deb-install defines a skip() helper and a SKIP counter, prints "N
skipped" in its summary, and calls skip() from nowhere, so the number can
only ever be 0. Its exit tested FAIL alone, meaning a skip could not have
failed the run even once something did set the counter. Gate on SKIP too.
That changes no current outcome, which is the reason to do it now rather
than later: the machinery and the reported number already existed, so the
first skip path added would have printed "N skipped" beside a zero exit
and read as coverage.

Verified by driving the verdict logic at zero, one and three skips: clean
pass, pass with the skip named, and a non-zero exit when nothing ran.
 2026-07-23 13:44:29 +00:00
Johnathan Corgan 2a6039995d Retire the ECN A/B test framing and record why suites are excluded 
ecn-ab-test.sh was a -test.sh with no path to a failing result: it
asserts nothing, applies no threshold, and nothing invokes it. It also
could not have worked. It read a fixed sim-results/ecn-ab-on/ path while
the runner has written timestamped directories since 2026-03-20, and not
one ecn-ab result directory exists on disk, so it has found neither input
for months and the "+10.2% recv throughput" figure the README carried is
not reproducible from anything available.

Rename it to ecn-ab-compare.sh, fix the path resolution to glob the
timestamped directories, and stop swallowing a failed simulation with
|| true so a broken run cannot feed the comparison.

Deliberately not adding the threshold assertion that would make it
gateable. That needs a calibration corpus, and none exists precisely
because the tool has never produced a kept result; a number invented now
would assert a guess. The prerequisite is a calibration run set and a
decision about what ECN is expected to deliver, which is a protocol
question. Written in the script header rather than left implied.

Add the exclusion block to ci-local.sh naming every suite and scenario
neither runner runs, with the reason for each: interop, boringtun,
iperf-test, this comparison tool, mesh-lab, and the four chaos scenarios
outside CHAOS_SUITES. Until now "not in the suite list" was
indistinguishable from "forgotten".
 2026-07-23 13:40:53 +00:00
Johnathan Corgan 1f149bda32 Run the convergence-gate unit tests in both CI runners 
wait_until_connected decides whether every static suite proceeds or
gives up, so a regression in it turns those suites' verdicts into noise.
Its unit tests existed and were invoked by nothing.

Wire them into ci-local.sh beside the parity and log-string checks,
above the mode branches so --only and --test-only are gated too, and add
the matching step to the ci-parity job in ci.yml. Putting them in one
runner only would create exactly the drift check-ci-parity.sh exists to
catch, and neither placement is visible to that checker since this is not
a matrix suite.

Verified by breaking what it guards rather than by observing green:
setting wait_until_connected's default near-converged slack to 0 fails
case4, records FAIL wait-converge, and exits the sweep 1. Restored after.

They pass, which nothing had established before — 13 assertions, all
green. They also take about 45 seconds rather than the "a few seconds"
the header claimed, and nothing had contradicted that claim because no
runner had ever invoked it. Header corrected with the measurement.

The orphan sweep this called for is done and there are now no
zero-reference *-test.sh files under testing/. It also settles a
disagreement recorded between the audit and the task file: ecn-ab-test.sh
is invoked by nothing, its one reference being a README description
rather than a driver, so the audit was right and the earlier sweep wrong
to call it a documented manual tool. interop-test.sh does have a driver
in interop-stress.sh and iperf-test.sh one in iperf-compare-refs.sh, so
those two were correctly classified.
 2026-07-23 13:38:25 +00:00
Johnathan Corgan 38a60c61da Record that churn-mixed's floors describe the invocation CI overrides 
The baseline floors added for this scenario were calibrated from archived
runs, and those runs are all the 10-node 120-second variant ci-local
invokes as "churn-mixed --nodes 10 --duration 120". The file itself
defaults to 20 nodes and 600 seconds, and --nodes sed-patches num_nodes
in a copy before the scenario loads, so the gating run and a bare
chaos.sh run are different scenarios. Nothing at the site said so.

The floors hold for both, but only because the gating run is the smaller
one. Retuning them against a bare 20-node run, which clears them easily
at 20 answering, 1 root, 19 parented and 69 sessions, would put them past
what the 10-node run reaches and turn CI red while a manual run stayed
green. Say that where someone retuning them will read it.

Confirmed by running the gating invocation directly: 10 answered, 2
roots, 8 parented, 18 sessions, all inside the ranges the floors were
built from. churn-mixed is the only scenario CI overrides this way; the
other twelve run their files as written.
 2026-07-23 07:27:16 +00:00
Johnathan Corgan 3d0a388511 Fail the ping test on an unknown topology profile 
Every section of ping-test.sh is guarded by the profile name, so an
unrecognised profile fell through all of them and the script exited 0
having run no assertion. A typo in the caller's argument produced a green
run that tested nothing. Give it an else branch that names the valid
profiles and exits 2.

No existing leg changes behaviour: the hosted runner gates the ping step
on matrix.type == 'static', which only static-mesh and static-chain
carry, and ci-local passes only the two names in STATIC_SUITES. Worth
noting while confirming that, though: ping-test.sh accepts a mesh-public
profile that neither runner ever passes.

Also record why the convergence waits are `|| true` — they are settling
delays rather than assertions, and the directed-pair pings are what
decides the run — and add admission-cap to the suite list in the
ci-local header, which ran it without listing it.
 2026-07-23 07:13:32 +00:00
Johnathan Corgan 5d3a3d7cad Give the assertion-free chaos scenarios a convergence floor 
Five scenarios named by the audit carried no assertions at all, so a run
in which the mesh never formed exited 0 and reported green. Add a
baseline assertion covering how many nodes answered, how many distinct
roots they agreed on, how many took a parent, and optionally how many
sessions were established, and apply it to those five plus the two cost
scenarios left unasserted by the previous commit.

For ethernet-only, ethernet-mesh, tcp-mesh, smoke-10, mixed-technology
and depth-vs-cost the values are not calibrated: one root and N-1
parented nodes is what a spanning tree is, and all provably-completed
archived runs of each show exactly that. churn-mixed is different and
says so at the site. A scenario that stops and starts nodes on purpose
does not hold a single tree, and its six completed runs end with two or
three roots and seven or eight of ten nodes parented, so its floors sit
one step outside the observed range. That leaves them catching a mesh
that collapsed rather than one that churned, which is the most a sample
of six supports.

This gives Ethernet transport its only assertion anywhere in CI. The
three tests in src/node/tests/ethernet.rs are ignored for requiring
CAP_NET_RAW and neither runner passes --ignored, so until now nothing
exercised that transport with a verdict attached.

The floor is deliberately weak and deliberately not a substitute: it says
the mesh formed, not that it formed the tree the scenario describes.
Where those differ the scenario now says so in its own comments.
 2026-07-23 07:09:36 +00:00
Johnathan Corgan e9ca741e53 Assert parent selection where the cost scenarios actually specify it 
Four scenarios exist to test cost-based parent selection and each named
its expected outcome in a comment that nothing read. Add a tree_parents
assertion mapping a node to the parent it must have in the final tree
snapshot, and encode it for the two scenarios whose stated outcome the
implementation meets: cost-avoidance (n04 takes the fiber n03) and
bottleneck-parent (n06 takes the fiber n03, n09 keeps its only parent
n05). Both hold in all six provably-completed archived runs.

The other two are left unasserted on purpose, and each for a different
reason worth keeping distinct.

mixed-technology says n06 and n08 should both pick the fiber parent n03.
They do not. Across the five completed archived runs n06 picks n10 three
times and n08 picks n04 four times. Encoding the criterion as written
would red the scenario most runs, and encoding the observed behaviour
would bless something nobody specified, so the disagreement is recorded
at the site as an open question. n06 preferring n10 may well be correct
and the comment stale, since n10 reaches it by fiber too.

depth-vs-cost is different: its validation line does not name an outcome
at all, saying only that the choice "reflects the actual cost tradeoff",
which either answer satisfies. The corpus shows both occurring under one
seed, three runs to two. That scenario needs a protocol decision about
which parent is correct before it can have an assertion.

Compare parents by address resolved from the snapshot's own
my_node_addr, and fail rather than skip when a node is absent from the
snapshot or still claims to be its own root. Those two states are the
common ones in the older corpus and both produce the same "no match" a
wrong parent does, so only separating them keeps a harness problem from
reading as a routing verdict.
 2026-07-23 07:05:50 +00:00
Johnathan Corgan 5a11cf091d Give congestion-stress the assertions its criteria described 
The scenario listed four success criteria "verified via post-run
congestion snapshot". Nothing read the snapshot, so the scenario could
not fail on any of them.

Add a congestion_signals assertion taking a floor on the number of nodes
reporting each counter, and encode three of the four. The floors are one
node each because that is what the criteria say; tightening them to the
counts recently observed would assert something nobody wrote down.

The fourth criterion is not encoded, and that is the finding rather than
an omission. No node has reported a non-zero kernel_drop_events in any of
the 182 archived runs, so asserting it would red the scenario
permanently. It is recorded at the site as an unmet criterion and a
coverage gap: the transport drop-detection path the scenario names as its
second signal is exercised by nothing.

Two things about the corpus are worth carrying, both recorded in the
scenario. Only six archived runs carry a status.txt and are therefore
provably completed; all six meet the three encoded criteria with five to
nine nodes reporting each signal. The 176 older runs meet none of them,
and they are not invalid samples: each reached teardown far enough to
write an analysis.txt, and ECN landed before all but one of them. What
changed on 2026-07-22 is not established, since neither the scenario nor
netem.py, traffic.py or control.py has been touched.

Count the nodes reporting a signal rather than the magnitude of any one
counter, since a single node with a large count would satisfy a magnitude
test while proving the signal never propagated. A missing snapshot fails
rather than reading as an absence of congestion.
 2026-07-23 06:59:21 +00:00
Johnathan Corgan aab149e215 Fail a chaos run whose nodes logged errors 
The simulation exit ladder reported panics and failed assertions but said
nothing about ERROR-level log lines, so a run in which every node errored
on every line still exited 0.

Add a max_errors assertion and apply it to every scenario by default
rather than having each one opt in. This is a floor on what a green run
means rather than a property of an individual scenario, and a scenario
that has to ask for the floor is one that can forget to.

The default ceiling of 0 is what the archived corpus supports: across
2416 result directories no node log contains an ERROR-level line, while
the WARN counter extracted by the same code path ranges from 0 to 1135,
so the counter is known to discriminate rather than merely known to read
zero. A scenario that legitimately induces errors raises the ceiling in
its own YAML and says there why.

Reject rust_log: off alongside it, since that is the one level that would
leave the ceiling counting zero whatever the mesh did.
 2026-07-23 06:53:56 +00:00
Johnathan Corgan 53ec14c2d5 Merge branch 'master' 2026-07-23 05:29:07 +00:00
Johnathan Corgan bc9bef717e Merge branch 'maint' 2026-07-23 05:29:07 +00:00
Johnathan Corgan 73e6917341 Give cost-stability the parent-switch ceiling its criterion described 
The scenario's success criterion existed only as a comment: count
"Parent switched" in n04's log, expect at most 5. Nothing read it, and
the scenario declared no assertions at all, so its exit code could only
report that the mesh came up and nothing crashed.

Add a max_parent_switches assertion and wire that criterion to it.

The assertion takes an optional node scope, which is the part that
matters. The existing sibling counts mesh-wide, and a criterion written
about one node's log is a different quantity from the sum over every
node: across archived runs of this scenario n04 is 1 or 2 while the
mesh-wide total ranges 3 to 6. Asserting the sum against a per-node
threshold would check something other than what was specified, and at
this threshold would also have failed several runs that were fine.

Counting nothing must not read as stability. A node id absent from the
topology fails explicitly rather than matching zero log lines and
sailing under the ceiling, and a scenario that declares a parent-switch
assertion while setting a log level that suppresses the events it counts
is rejected at load rather than passing vacuously after a full run. The
parse rejects a mistyped key, a missing, negative, boolean or
non-integer ceiling, and a node written empty or as a non-string -- that
last one because YAML renders a bare `node:` as null, which would
silently revert to the mesh-wide count.

The scenario comment now records what the threshold is worth against
the 11 completed archived runs rather than a single sample: 5 is well
above anything observed, the daemon's own parent hold-down caps
switches near 6 per run, and nearly every switch lands during initial
tree formation before any mutation fires. So this catches only
near-pathological reparenting. Tightening to 3 would make it a real
detector, but that changes the stated criterion rather than fixing it,
so it is left as a decision.

Verified against a live run: the assertion evaluates n04 at 2 against
mesh-wide 4 and passes, and with the ceiling temporarily set to 0 the
scenario exits 3, the assertion-failure code, which it previously could
not reach.
 2026-07-23 05:19:36 +00:00
Johnathan Corgan c805a4492e Merge branch 'master' 2026-07-23 04:16:51 +00:00
Johnathan Corgan 8d7053fa72 Merge branch 'maint' 2026-07-23 04:16:51 +00:00
Johnathan Corgan 300deb0476 Reject an unanswered show_mappings instead of reading it as zero 
The mapping-reclaimed check expects zero mappings, and read the count
with `r.get('data',{}).get('mappings',[])`. An error response carries
no data field, so it parsed to zero and satisfied the check without the
gateway having answered at all. The expected value and the failure
value were the same number, which is the shape that lets an assertion
pass without asking anything.

Require the key to exist and be a list, and exit non-zero otherwise, so
the existing fallback turns an unanswered or malformed response into a
failure. The gateway always emits the key on success, including for an
empty set, so a genuine zero still reads as zero; confirmed against the
running gateway, where the reclamation check passes.

The earlier show_mappings poll shares the parse and is left alone: it
waits for a positive two, which no error response can produce. The
hazard was already documented there, and this is the call site that was
exposed to it.
 2026-07-23 04:16:42 +00:00
Johnathan Corgan 700ac581ee Require a timeout and a drop verdict in the firewall assertions 
Case (a) sends an unallowed inbound request that the ruleset should
drop, and accepted any non-zero curl status as proof. A drop produces
no RST, so curl can only hit its deadline and exit 28; connection
refused, an unroutable address or a missing listener all fail too, with
different codes, and the check counted those as a blocked connection.
Require 28 exactly, so the assertion distinguishes silently dropped
from failed for some other reason. Confirmed against the harness: the
real run returns 28.

The baseline check matched `counter packets`, which any counter rule
satisfies whatever its verdict, including one that accepts. Require the
rendered drop rule instead.

The drop-counter read had the same weakness plus a worse one: it took
field three of the first line mentioning a counter, which is the packet
count only when the line begins with `counter`. On a rule such as
`tcp dport 9 counter packets 42 bytes 3000 drop` it printed the port
number. That shape is reachable, because the shipped ruleset includes
the operator drop-in directory ahead of the trailing default deny and a
drop-in may add its own counted drop. Extract by position within the
matched text, and take the last match rather than the first, since case
(a) falls through to the default deny that the ruleset emits as the
final rule.

Note in place at the baseline check that it still only proves a counted
drop rule exists somewhere in the table, not that it is the trailing
one; asserting rule position is a larger change than this warrants.

The sample output in the README is updated to the new wording.
 2026-07-23 04:16:32 +00:00
Johnathan Corgan 98560594cf Prove ACL rejection per peer by matching a single log line 
The four rejection checks were independent greps over the whole log:
two npubs, a context and a decision. Together they proved only that
node-a mentioned each npub somewhere, rejected somebody on an inbound
handshake, and rejected somebody by denylist. Nothing tied a rejection
to a named peer, and node-a lists both denied peers as auto_connect
peers, so it logs their npubs on the outbound connect path whether or
not any rejection ever happened. The actual message was never matched.

Replace them with a helper that requires every given string on ONE
line, and assert per denied peer: the real message, that peer's npub,
and the denylist decision together. Strings match in any order by
chaining fixed-string greps over the surviving lines, so the assertion
does not depend on how the log formatter orders a message and its
fields. A grep over empty input yields the empty string rather than
anything a caller could mistake for a match, so an unreadable container
times out and fails instead of passing.

Two limits are written at the call sites rather than left implied. The
decision conjunct discriminates nothing, since the two allowing
variants return early and denylist is the only value that can reach
that warning. And node-a authorizes before dialing, so it emits a
fully formed rejection line for each denied peer on the outbound path;
these assertions are satisfiable without the inbound check running at
all, and it is the peer-count assertions that would catch that.

Verified against the running harness: all three find a real matching
line. The defect shape they replace was exercised offline first, with
the three strings spread across three lines, where the old checks
passed and the new ones fail.
 2026-07-23 04:16:16 +00:00
Johnathan Corgan 719b587920 Merge branch 'master' 
# Conflicts:
#	CHANGELOG.md
 2026-07-23 03:24:23 +00:00
Johnathan Corgan fc8ebd5a06 Merge branch 'maint' 2026-07-23 03:23:18 +00:00
Johnathan Corgan ebf34ae712 Read the wall clock directly for Nostr traversal timestamps 
The traversal clock cached a Unix millisecond value and an Instant at
first use, then served every later call by advancing the cached value
with the monotonic elapsed time. A monotonic clock does not tick while
the host is suspended, so once a machine had slept the returned value
trailed real time by the sleep duration for the rest of the process
lifetime, and never re-synced.

Almost everything that clock feeds is an absolute timestamp. The NIP-40
expiration tags on adverts and traversal signals, and the issuedAt and
expiresAt fields of offers and answers, are all computed as now plus a
TTL; the freshness and cache-pruning paths compare it against a
peer-authored, signed created_at. Once the host had slept longer than
signal_ttl_secs, every offer was published already expired, relays
dropped it, and the initiator timed out waiting for an answer with
traversal broken until a restart.

Read the wall clock on every call instead. This also removes a
mismatch inside the traversal failure-state map, which was written
here from the cached clock but written and read from the node
lifecycle with the real one.

Not platform-specific: monotonic clocks exclude suspended time on
Linux and Windows as well, so this affected any host that suspends.
A laptop is simply where a process lives long enough across a sleep
to notice.

The interval-shaped consumers hold up under a clock step. A forward
step, which is what a resume produces, saturates the punch start delay
to zero, and the attempt's own bounds are monotonic deadlines that are
unaffected. A backward step lengthens that delay and can cost one punch
attempt, which retries. Early eviction from the replay window cannot
admit a replay under the shipped defaults, because the freshness window
a replayed offer must also satisfy is strictly narrower than the replay
window itself.

The added test pins the contract and fires on a host that has genuinely
suspended, but it is not a regression guard for this defect: nothing
reachable from a unit test can simulate a suspend, so on a machine that
has not slept the old implementation passes it too. Its comment says
so rather than leaving a false sense of coverage.

Reported in https://github.com/jmcorgan/fips/issues/128
 2026-07-23 03:23:00 +00:00
Johnathan Corgan 32b40221b0 Merge branch 'master' 2026-07-23 03:03:05 +00:00
Johnathan Corgan a1222d6d75 Merge the open !FIPS access SSID layer from GitHub 2026-07-23 03:02:54 +00:00
ArjenandGitHub f624013b83 feat(openwrt): open !FIPS access SSID layer (#126) 
Access layer for phones and laptops to reach FIPS routers on OpenWrt,
stacked on the 802.11s mesh backhaul from #123. Squashed from five
commits by Arjen (Origami74); their original messages follow.

* feat(openwrt): open !FIPS access SSID — fips-ap-setup helper, default transport binding, how-to

  Client access layer for phones and laptops: every FIPS router
  broadcasts the same open SSID ('!FIPS' — the leading '!' sorts it to
  the top of alphabetically ordered network pickers), forming one
  standard ESS. Clients save it once and roam between all FIPS routers
  natively, with FIPS's Noise IK handshake as the only security layer:

  - fips-ap-setup: opt-in UCI helper that creates the 'fips-ap0' open AP
    (encryption none — security type must be uniform across routers or
    clients treat the ESS as different saved networks), an isolated
    network with a static ULA /64, RA-only odhcpd addressing (stateless
    SLAAC, no DHCP — the minimum that satisfies Android's provisioning
    check; no internet by design, so phones keep cellular as default
    route), and a locked-down fips_ap firewall zone (no path to br-lan
    or the WAN; only ICMPv6, mDNS, and the FIPS transports reachable).
    'remove' subcommand undoes it. Radio setup stays opt-in; a package
    must not commandeer radios on install.
  - fips.yaml: ship 'ap0'/'ap1' Ethernet-transport entries commented out
    (matching the 802.11s mesh backhaul) so a stock install that never
    creates fips-ap* logs no per-boot "interface missing" bind warning;
    fips-ap-setup uncomments the matching block when it creates the
    interface and re-comments it on remove.
  - Regression test: extend shipped_openwrt_config_parses to assert the
    ap0/ap1 entries ship commented out and still parse once uncommented,
    alongside mesh0/mesh1.
  - Packaging: install the helper in ipk/apk/buildroot (three synced
    copies), extend CI structural checks and shellcheck targets.
  - docs/how-to/set-up-open-access-ssid.md: full guide, including the
    one-time 'no internet, stay connected' acceptance (stored per SSID,
    covers every FIPS router) and the security-type-uniformity
    constraint.

* docs(openwrt): correct open-SSID/mesh security framing — peering is open by design

  The fips-ap-setup/fips-mesh-setup comments and both how-tos claimed a
  stranger "cannot pass the FIPS handshake" / "their frames die at the
  handshake" / the handshake surface "drops them". That is wrong: FIPS
  peer admission is open. An inbound handshake from any net-new identity
  is promoted (node::handlers::handshake::promote_connection), gated only
  by the daemon's max-peers cap — there is no allowlist, no PSK, and the
  AuthChallenge path is not wired to admission. The Noise IK handshake
  provides authentication (no impersonation of another identity, no MITM),
  not authorization.

  Restate the model accurately in all five places: a stranger on the open
  SSID (or the open mesh) can associate AND form a FIPS peer link — that
  is the point of open access. Containment is the isolated fips_ap zone
  (no path to br-lan or the WAN) plus the max-peers cap, not the
  handshake. Clarify that AP client isolation is an L2 control only: a
  peered stranger is an overlay peer like any other, so the FIPS overlay,
  not L2, is the trust boundary between clients.

* feat(openwrt): serve DHCPv4 on the access SSID from a fixed roamable subnet

  RA-only addressing satisfied Android's provisioning check but left
  anything expecting IPv4 with a self-assigned address and a "no IP"
  complaint. dnsmasq now leases out of 10.21.<N>.0/24 (N = radio index;
  prefix echoes FIPS port 2121), deliberately identical on every router:
  a roaming client keeps its lease across the ESS, and dnsmasq's
  authoritative mode — the OpenWrt default, pinned by the helper — ACKs
  the renew a foreign router never issued. Lease collisions across
  routers surface as a NAK on renew and the client re-DHCPs.

  The dhcp section's 'dhcpv4 server' is read by both dnsmasq (default
  images) and odhcpd (only with maindhcp), so either arrangement serves.
  A DHCPv4/udp-67 accept rule joins the fips_ap zone; DHCPv6 stays off,
  the ULA RA stays as-is, and nothing depends on an upstream. The zone
  remains isolated — no forwardings, no internet.

* feat(openwrt): enable mDNS rendezvous from fips-ap-setup

  Phone FIPS apps cannot open raw-Ethernet sockets, so DNS-SD is how they
  find the router's daemon — but node.rendezvous.lan defaults to off. Ship
  the lan block commented in fips.yaml (consistent with the apN transport
  entries) and have fips-ap-setup uncomment it when creating the access
  SSID. The awk match is scoped to node.rendezvous because
  transports.ethernet carries a 'lan' entry at the same indent. The switch
  is daemon-wide, so 'remove' deliberately leaves it on rather than guess
  whether other transports rely on it.

* fix(openwrt): bind UDP dual-stack [::]:2121 so access-SSID clients reach it

  The shipped router config bound the UDP transport "0.0.0.0:2121" (IPv4
  wildcard) while the mDNS LAN advert announces every interface address,
  including the router's IPv6 link-local — which phones on the !FIPS
  access SSID rightly prefer (their cellular default route swallows v4,
  and fd00::/8 is captured by the Myco mesh TUN). Result: the client's
  Noise msg1 arrives on an unbound v6 port and is silently lost; the
  handshake resends and times out.

  Symptom chain (observed on-device): mDNS resolve OK, platform push OK,
  "Sent Noise handshake message 1" to [fe80::…%N]:2121, four resends, no
  reply, 30 s stale-timeout.

  OpenWrt is Linux (bindv6only=0), so "[::]" accepts IPv4 via v4-mapped
  addresses too — nothing is lost. packaging/common is deliberately left
  on "0.0.0.0" for now: Windows defaults IPV6_V6ONLY=1, where "[::]"
  would drop v4 instead.
 2026-07-22 20:02:20 -07:00
Johnathan Corgan 308e2ce3b3 Merge branch 'master' into next 2026-07-23 02:42:58 +00:00
Johnathan Corgan 16b1bc2c5c Merge branch 'maint' 2026-07-23 02:42:48 +00:00
Johnathan Corgan 291c4312dc Stop the ACL peer-count reader from turning "no answer" into zero 
wait_for_peers_exact fell back to 0 when it could not read a container's peer
count. Two of its six call sites are the ACL denial checks, which expect
exactly 0 connected peers — so a failed docker exec satisfied them on the
first iteration and the isolation property the suite exists to prove was
never observed. Confirmed against a container that does not exist: the old
reader returns "0" and the expect-zero comparison passes.

The reader now returns the empty string when the daemon does not answer,
which no numeric comparison can satisfy, and the timeout message says which
of the two happened — never answered, or answered the wrong number. Same
shape as admission-cap-test.sh's read_peer_count.
 2026-07-23 02:42:36 +00:00
Johnathan Corgan d21f818659 Check that log strings tests match on are still ones the daemon emits 
A test that greps the daemon's log for a message the daemon stopped emitting
does not fail. It stops observing, and an assertion built on it — especially
one expecting a count of zero — then passes because nothing can be seen
rather than because nothing happened. Several findings have come from that
one class, so it is now checked mechanically.

testing/check-log-strings.py extracts the strings test code matches against
daemon log text and requires each to exist in src/. It reads four shapes:
python `"..." in line` tests, the first argument of the shell log helpers,
bash associative-array pattern tables, and greps whose input is daemon log
output. Patterns carry regex syntax, so each is reduced to the longest
literal run of every alternation branch, with escaping resolved in the same
pass — `\.` is a literal dot and `.` is a wildcard, and conflating them would
let a pattern match text that is not there.

Scoping is by what a grep READS, not by what its file mentions. A suite greps
its own analyzer output, fipsctl JSON, Tor's log and ping output, and none of
those have to correspond to a string in src/; scoping on the file produced 34
false positives. Strings that legitimately do not come from src/ are named in
ALLOWED with a reason, so the exceptions are reviewable rather than invisible.

It found six dead strings, three of which were not previously known: a second
copy of "Excessive decrypt failures" in the interop pattern table, a dead
"Handshake error" branch inside an alternation whose other branches still
matched, and "bootstrap failed" in the NAT suite. All six are corrected to
what the daemon emits, or dropped where the surviving branch already covers
the case. Verified by breaking it two ways and confirming it reports.

Also here, since it is the same failure shape one layer down: the shared log
library returned an empty string for a container whose logs it could not
read, so analysing a missing container reported no panics, no errors and no
sessions, and exited 0 — indistinguishable from a clean run. It now reports
which containers could not be read and raises, matching the fix already
landed in the chaos copy.
 2026-07-23 02:42:25 +00:00
Johnathan Corgan 64f68fbc77 Suffix the mixed-profile test's container names where it builds them 
Giving the mixed-profile services a run-scoped container_name broke this
script's two ping sites, which assemble the name as "fips-$from" rather than
writing it out. A grep for the literal name finds neither, which is the same
trap the earlier naming sweep hit on the other static scripts.

The failure was loud — link convergence passed 3/2/2/1 and then every ping
failed, because docker exec could not find the container — but it is worth
noting that these names were unsuffixed until now, so this suite was never
safe to run twice on one host.
 2026-07-23 02:20:00 +00:00
Johnathan Corgan e9cb3f20a5 Merge branch 'master' into next 2026-07-23 02:09:27 +00:00
Johnathan Corgan 630a04c432 Merge branch 'maint' 
The static compose's mixed-profile services are next-only, so the floating
subnet arrived without them: four services kept ipv4_address pins on a
network that no longer declares a subnet, which docker rejects outright at
`up`. They now float with the rest, take the per-run config directory, and
carry the run name suffix their container_name was missing.

Phase 3 of the admission-cap test conflicted for a real reason rather than a
textual one. This line asserts size-agnostically — sustained inbound from
each denied peer plus "was never promoted" — because the XX handshake
message sizes differ from IK and may change again. That assertion is kept;
what comes across from maint is the address handling underneath it, which
matters here for the same reason: with the subnet floating, a restarted
container can change address, and matching on a single one silently
under-counts the very evidence the assertion rests on.
 2026-07-23 02:08:52 +00:00
Johnathan Corgan fbd0dba870 Merge branch 'maint' 2026-07-23 02:06:06 +00:00
Johnathan Corgan 6c52b0e01e Let the static test network float so concurrent CI runs cannot collide 
Two local CI runs on one host both asked docker for 172.20.0.0/24 and the
second lost its whole static family to "Pool overlaps". Docker honours a
fixed subnet request verbatim, so the only robust fix is to stop making one:
fips-net now requests no subnet and docker assigns from its own pool, which
cannot hand the same range to two runs.

That means node addresses are not known before `up`, so peers address each
other by container hostname instead. The generator emits node-<id>, or the
topology's docker_host where the compose hostname differs — only the gateway
profile, whose services are gw-*. External peers keep the address the
topology gives them, since it is not ours to assign. The resolv.conf mount
stays: dnsmasq is what forwards these names to docker's resolver and .fips
to the daemon, so removing it would take out every .fips assertion.

generated-configs is now per-run as well. A shared directory let two runs
overwrite each other's node configs, which the subnet collision had been
hiding by killing runs before that window opened. The generator, the compose
bind mounts and env_file, the six scripts that read it, and teardown all
follow FIPS_CI_NAME_SUFFIX; unset, every path renders as before. Teardown
keeps the directory after a failed run, where it is the evidence of what the
failing nodes were configured with.

Three things this exposed that were wrong independently:

admission-cap built its tcpdump patterns from the topology file's docker_ip
literals. Floating the subnet makes those match nothing, which would have
left its expect-zero "no Msg2 leaked" assertion passing because it could no
longer see anything at all. It now reads addresses from the running
containers. Restarting the denied peers together also made them swap
addresses, so each peer's counts were really the pair's total; they are
restarted one at a time now, and a check fails the suite outright if two
denied peers ever share an address, because per-peer attribution is
impossible once they do.

Attribute lookups in the generator used a fixed ten-line window and read the
next node's fields when a node omitted an attribute. An external node
followed by an internal one was classified as internal, which under hostname
peering would emit a name that resolves nowhere. Lookups are bounded to the
node's own block; generated output is byte-identical for all eight
topologies.

The rekey outbound-only variant used to rewrite peer addresses to hostnames
to set up its scenario. The generator now does that everywhere, so the
rewrite matched nothing and was silently doing no work. It asserts the
premise instead, and fails if a numeric address ever reappears.

Verified by running three instances of this compose at once — tcp-chain plus
two independent meshes — which drew 10.128.2/3/4.0/24 with no overlap while
both meshes passed ping-test 20/20 over the real .fips path. tcp-chain is
run by neither CI runner, so it was checked by hand: chain peer counts 1/2/1
and multi-hop .fips reachable both directions over TCP.

gateway-lan still pins its own IPv4 and fd02:: ranges and is unchanged here,
so the gateway profile is not yet concurrency-safe.
 2026-07-23 02:05:56 +00:00
Johnathan Corgan 5851c7f36f Give the next-only mixed-profile suite a suite array 
The parity guard now checks that every suite the local runner can dispatch has a backing suite array, so a suite dispatched without one is visible to nobody. On next, mixed-profile was dispatched by a bare call with no array, so the guard reported it as present on the GitHub matrix and missing from the local runner even though the local runner does run it.

This adds MIXED_PROFILE_SUITES, dispatches through it in the same idiom the admission-cap suite uses so the array is the single source of truth for whether the suite runs, and lists it in the suite listing. It exists only on next, so it is authored here directly rather than merged up.
 2026-07-22 23:18:08 +00:00
Johnathan Corgan d8ded7000f Merge branch 'master' into next 2026-07-22 23:17:43 +00:00
Johnathan Corgan c2d6283bd6 Merge branch 'maint' 2026-07-22 22:57:33 +00:00
Johnathan Corgan 428773490f Close the gaps an adversarial review found in the new CI guards 
Four of these are places the parity guard could stop covering something without saying so, which is the failure mode it exists to prevent. Removing the deb-install suite array left it green, because the dispatch cross-check compared against a hardcoded name rather than the array it was meant to read. An arm-shaped line at an unexpected indent, or one written in quotes, was dropped silently by the arm pattern; unexpected indents are now a hard error rather than a quiet skip, and quoted arms parse. A matrix leg with a type of chaos or deb-install but no scenario key raised a Python traceback instead of reporting a problem, and a leg carrying a scenario but no suite key was skipped entirely even though scenario is now the identity for those legs.

The e2e builder no longer folds docker create's stderr into the container id. Docker prints warnings on success as well as failure, so a platform-mismatch warning would have become part of the id and made every later reference to that container fail, turning a working extraction into a spurious red.

Also corrects the comment about registering a new chaos assertion type, which named only one of the two key sets that actually need it.
 2026-07-22 22:41:24 +00:00
Johnathan Corgan 56f00058d4 Run the CI parity guard in both runners 
The guard that keeps "local green" and "GitHub green" meaning the same thing was invoked by nothing. Its only entry point was an exec behind an explicit --check-parity flag, which by exec-ing could not be combined with an actual run, and no workflow step called it at all. A guard nobody runs is a guard that does not exist, which is how the mixed-profile divergence on next survived unnoticed.

It now runs as the first stage of every local run and as its own job on GitHub. Locally it reports through the same result-recording path as every other stage, so a divergence sets the run's exit status instead of scrolling past. It is deliberately placed above the mode branches, so a divergence also fails --only, --test-only and --build-only runs: whichever subset was asked for, the claim that a local result means what a GitHub result means is what has broken. The GitHub side installs pyyaml explicitly rather than assuming the runner image carries it.

--check-parity keeps working exactly as before. Three comment blocks that described the old folded comparison are corrected here, in the workflow, the local runner and the testing README; they were spread across three files and only two mention the guard by name, so the third is best found by searching for what it claims rather than for the script.
 2026-07-22 22:03:08 +00:00
Johnathan Corgan e578a11b7a Compare every CI leg in the parity guard, not a folded token 
The guard collapsed all thirteen chaos legs to the token "chaos" and all five deb-install legs to "deb-install", so sixteen of the thirty-four integration legs were invisible to it. Deleting twelve chaos legs and four deb-install legs from a copy of the workflow left it still reporting "CI parity OK". The fold existed because the guard compared the cosmetic suite name, which really does differ between the runners; both sides have carried a directly comparable scenario field all along, so the fix is to compare that instead, along with the chaos flags.

The local suite set is now discovered by sweeping ci-local.sh for suite arrays rather than reading a hardcoded list of variable names, and the deb-install distro list is read from the suite's own script. A suite dispatched with no backing array is invisible to either approach, so every run_suite arm is now checked to have one and the guard names any that does not. That is not hypothetical: next dispatches its mixed-profile suite without an array, and the guard could only report it as missing from the local runner without being able to say why.

The guard also now refuses to run rather than failing obscurely when python3 or pyyaml is absent, since a parity check that cannot run must not look like one that passed. Verified by breaking each thing it guards: legs deleted from the workflow, drifted chaos flags, a fabricated suite array, and a dispatch arm whose array was removed. That last case initially did not fire because the arm pattern was pinned to the wrong indentation, which the test caught.
 2026-07-22 21:59:28 +00:00
Johnathan Corgan 99cb0a51fd Reject unknown keys when loading a chaos scenario 
The scenario loader read every section and member through raw.get with a default, so a mistyped key was silently ignored and the block it belonged to stayed default-constructed while the YAML still looked correct on the page. Writing "assertion:" for "assertions:" disarmed a scenario's only assertions and left it unable to return a non-zero exit code; "link_flap:" for "link_flaps:" turned off the churn the scenario existed to inject, and a calm mesh then ran under the name of a chaos test.

Keys are now checked at load time, at the top level and inside every section with a fixed schema, including the netem policy bodies, the {min, max} ranges and the assertion bodies themselves. The error names the offending key, the section it appeared in, and the keys that section does understand. Three mappings are deliberately exempt because their keys are names the author chooses rather than a schema, and two sub-trees are passed through whole; all five are listed at the key sets with the reason.

Adding an assertion type now means registering it, so the next person to add one gets a rejection rather than a silent no-op. That coupling is the point and it is written down where they will see it. Verified against the seventeen scenarios in the tree, all of which still load, and against fixtures for each failure mode: a top-level typo, a nested typo, a typo inside an assertion body, and a section left empty, which previously raised an unrelated AttributeError from the next line of the loader.
 2026-07-22 21:56:41 +00:00
Johnathan Corgan e9ca16f3a1 Stop the DNS resolver e2e tests certifying a binary that was never built 
The e2e scenarios build fips and fips-gateway in a builder image and extract them with two docker cp calls whose stderr went to /dev/null and whose exit status was never tested. Because build_fips_for_e2e ended in a log call it returned 0 on every path, so the caller's guard could not fire. A failed extraction left the previous run's binaries in the cache, they satisfied the caller's executable check, and the five e2e legs then exercised the previous commit's code and reported green.

The extraction now deletes the cached binaries before it starts, which is what makes the stale case impossible rather than merely detected, and it checks each docker cp independently, reports the failure with docker's own message, verifies each extracted file is non-empty and executable, and returns an explicit status. This mirrors what the deb-install suite already does.

Confirmed by inducing the failure rather than by observing a green run. With both extractions broken and a stale binary in place, the old code logged "Cached fips (26936 bytes)" and went on to build the runtime image around it; the new code reports both docker errors, deletes the stale binaries and fails the suite. Breaking either extraction on its own is caught separately, and an unmodified run passes 7 of 7.
 2026-07-22 21:54:37 +00:00
Johnathan Corgan b56ec14d14 Merge master into next, bringing the chaos suite's failure reporting with it 
The chaos simulation was equally unable to report a failure on this line: scenarios collided on globally-scoped container names and a shared config directory, the runner swallowed whatever they raised, and the harness discarded their exit codes before anything read them. The merged commits scope names and directories per scenario, give an aborted run its own exit code and let it reach the summary, stop a failed scenario harvesting whichever containers hold its names, and keep the daemon's reason for a failure instead of discarding it.

Testing only. ci-local.sh merged automatically; run_chaos and ci_teardown come across byte for byte and this line's own mixed-profile handling is untouched, the two having never overlapped.
 2026-07-22 10:09:13 +00:00
Johnathan Corgan 946df54d0f Merge maint into master, bringing the chaos suite's failure reporting with it 
The chaos simulation could not report a failure on this line either: scenarios collided on globally-scoped container names and a shared config directory, the runner swallowed whatever they raised, and the harness discarded their exit codes before anything read them. The five commits scope the names and directories per scenario, give an aborted run its own exit code and let that code reach the summary, stop a failed scenario harvesting whichever containers happen to hold its names, and keep the daemon's reason for a failure instead of throwing it away.

Testing only, and the merge is textual: every chaos file matches maint byte for byte, and the ci-local divergence on this line does not touch the changed regions.
 2026-07-22 09:33:58 +00:00
Johnathan Corgan ab750f60b3 Record why a chaos docker command failed 
Compose commands run with their output captured and with check set, so a failure raised a CalledProcessError and nothing ever looked at what the daemon had said. That exception reports the argv and the exit status and nothing else, so every scenario that died bringing its containers up left a runner log saying only that docker compose returned non-zero exit status 1, and the reason was captured and then discarded. In the run that prompted this work twelve scenarios failed that way and why they failed can no longer be established at all. Log the captured stderr and stdout before the error goes anywhere, keeping the tail of each so a verbose failure cannot fill the log file. A command that times out raises before its exit status is ever read, so that path records whatever it had emitted too, rather than leaving the same hole one shape over. Reproduced against both of the shapes this is meant to catch: an unusable network prefix now records the daemon's parse error, and a name already held by another container now records the conflict and names the container holding it. The success path logs nothing new, and a completed scenario's runner log is unchanged line for line.

The raising behaviour is deliberately untouched, since the abort flag and the exit code that came with it depend on it. The check is applied here rather than by subprocess so that a caller who passed check false gets the same record: both such callers are the compose down in teardown, which today can leave a mesh behind without saying so.

The same omission was costing the veth path its reasons. Failing ip commands had their stderr logged at debug, which the runner does not emit unless asked for verbose output, so a failed pair creation reached the log as nothing more than the names of the two interfaces it could not create. Raise that to a warning. The deletes issued to clear a stale pair are expected to fail and still say nothing.
 2026-07-22 08:26:59 +00:00
Johnathan Corgan 226c6994d3 Harvest chaos results only from a mesh that actually started 
Teardown runs from a finally, so it also runs after a failed setup, and it was guarded only on the topology and the compose file, both of which are set well before any container exists. A scenario that died bringing its containers up therefore ran the whole harvest anyway: final snapshots, docker logs, the analysis, the assertions and the metadata, all addressing containers by names that are global to the host. What it left behind was not an empty directory an investigator would notice but a full and plausible one, in the recorded case ten node logs and an analysis reporting two promotions and two parent switches, every byte of it from a different scenario's mesh. Gate the harvest on whether the containers ever started, and leave the compose down outside that gate so a partly successful start still gets cleaned up.

Record the outcome in a status file in every result directory, naming the run as completed, interrupted, aborted, setup-failed or teardown-failed, alongside the scenario, the seed and the container names it used. With the harvest gated, the presence of an analysis file is now itself proof that the scenario's own mesh existed, and the status file says which of the several ways a run can end applies. A run cut short by a signal keeps the existing exit codes rather than gaining one of its own: what it collected before stopping is real and still worth reporting, the status file records that it was truncated, and every wrapper already reports a Ctrl-C of its own.

Log collection never looked at the status of docker logs. It concatenated stdout and stderr unconditionally, so collecting from containers that were not there wrote the daemon's "No such container" reply into each node log and analysed the result as a mesh with no panics, no errors and no sessions, which reads exactly like a clean run and exits zero. Check the return code, and treat a harvest that cannot read every container, or that reads none, as a failed teardown. A teardown that raises now also reports on the same footing as a run that never started, instead of escaping past the exit codes as a bare traceback and being read as a malformed command line.
 2026-07-22 08:02:38 +00:00
Johnathan Corgan 4f55a281ac Let a failed chaos scenario fail the run 
The simulation caught every exception a scenario raised, logged it, and exited 0, so a scenario that died during setup was recorded as a pass. Local CI discarded the exit code in any case: run_chaos ends in a call whose own last statement is an echo, so it returned 0 whatever the scenario did, and the parallel launcher's wait therefore always succeeded and always recorded a pass. Between them no chaos failure of any kind could turn a local run red, and none could since the script was written.

Carry the abort out of the run and give it its own exit code, ordered ahead of the two content codes so a run that never produced a mesh cannot report that mesh's panic and assertion counts. Return the scenario's status from run_chaos so the launcher sees it. The documented exit codes were stale before this change and are rewritten in full, including that the argument parser rejects a malformed command line with the same code that reports panics.

Expect scenarios that have been reporting green to start reporting red. In the last recorded local run twelve of the thirteen scenarios never got past starting their containers, and all thirteen were counted as passes; those that now execute have not run for months, and their assertions have never been evaluated against a mesh of their own.
 2026-07-22 07:37:48 +00:00
Johnathan Corgan 2ef36c0071 Scope chaos host interface names per scenario and reap orphaned pairs 
Ethernet edges get a veth pair created in the host namespace and then moved into the containers, named from the node ids alone. ethernet-mesh and ethernet-only both connect n01 to n04 and always run together, so one tore down the other's live link. Hash the scenario suffix to four hex characters and insert those after the vh prefix; 15 characters is far too few to carry the suffix itself.

The names are now run-specific and never regenerated, so the delete-before-create no longer reclaims an interface an earlier run abandoned. ci-cleanup.sh takes over, deriving the names from the suffixes a run's teardown hands it. It is the first thing that script removes that is neither a docker object nor labelled, so an unscoped reap can sever a running bare simulation's links; testing/README.md and the --reap help say so.
 2026-07-22 07:35:11 +00:00
Johnathan Corgan 5be7c6d0cb Give each chaos scenario its own container names and config directory 
Chaos scenarios run four at a time under local CI, but every one of them claimed the container names fips-node-nNN and wrote its generated configs and compose file to the same generated-configs/sim directory. Container names are global in Docker and are not scoped by the compose project, so concurrent scenarios collided on both, and a scenario could start containers from a compose file another had overwritten.

Thread the existing FIPS_CI_NAME_SUFFIX into the simulation. run_chaos narrows the run-wide suffix to the scenario, and the sim reads it once when the topology is built, applying it to the container names and to the config directory basename. The compose template renders the name through the topology accessor instead of duplicating the literal, so one expression produces every chaos container name.

The suffix is empty when the variable is unset, so a bare chaos.sh run and the hosted CI jobs render byte-identical names and paths. Verified by rendering every scenario's compose file before and after with the variable unset and diffing.
 2026-07-22 07:35:11 +00:00
Johnathan Corgan bd3b639570 Merge branch 'master' into next 
# Conflicts:
#	CHANGELOG.md
 2026-07-22 03:46:15 +00:00
Johnathan Corgan 29923cf676 Merge maint into master, reconciling the two hop limit implementations 
Both branches carry the same correction, authored separately because the
sans-IO relocation moved the logic into a core that returns an outcome rather
than acting inline. Git resolved the file move as a rename and offered to merge
the two versions textually, which would have been wrong in both files.

Resolved by keeping master's re-expression wholesale for the handler and its
tests, and taking the changelog entry from maint. Verified rather than assumed:
the resulting tree is byte identical to the reviewed and CI-green master commit
across every code path, and the merge contributes nothing but the fifteen line
changelog entry. Since the code is unchanged from a tree that already passed the
full local suite, that result still stands and no re-run is owed.

Nothing was lost from maint's side. Its tests are written against a file layout
that does not exist here, and master carries equivalent coverage written against
its own shape, including the multi-node chain and the cache warming cases that
the maint version does not have.
 2026-07-22 02:59:44 +00:00
Johnathan Corgan 34431209f5 Follow IP semantics for the session datagram hop limit on the refactored path 
The same correction as on the maintenance line, re-expressed rather than merged,
because the sans-IO relocation moved this logic into a synchronous core that
returns an outcome instead of acting inline.

Delivery to the addressed node is no longer gated on the hop limit, and the
decrement happens before the drop decision, so a datagram that would leave with
zero is not sent. The reachable radius does not change: the old behavior refused
to deliver at zero but was willing to send at zero, and the new behavior is the
mirror of that, so the two cancel.

The shell above the core needed changing too, and that was the real work. It
carried two hop-limit predicates of its own, both justified by comments citing the
ordering that existed before the refactor. The first gated coordinate cache
warming, which this fix requires to be unconditional; left alone it would have
suppressed warming twice over, once for a datagram addressed here and now
delivered, and once for a transit datagram dropped for hop limit whose plaintext
coordinates are still perfectly usable. The second gated next hop resolution,
which carries a cache touch side effect, and now mirrors the core's
would-leave-zero rule instead of the old one. Neither divergence would have
surfaced as a test failure.

That last point is worth stating plainly, because it nearly went the other way.
Restoring the warming gate on top of the corrected core passes the entire suite
without a single failure. The tests could not see the seam between the core rule
and the shell's copy of it. So this adds coverage that can: a three node chain
pinning the decrement across a real hop, which is the only thing exercising the
shell predicates composed with the core, and two warming tests at a zero hop limit
covering both the delivered and the dropped case. Each was checked by putting the
specific defect it targets back and confirming the test fails.

One existing test that claimed to cover hop limit behavior asserted nothing
whatsoever. It asserts now, but those assertions do not discriminate this change,
since the case it exercises behaves identically under both rules. They guard
against the test going vacuous again, not against this defect, and the comment
says so.
 2026-07-22 02:32:34 +00:00
Johnathan Corgan 8ca2362e6c Follow IP semantics for the session datagram hop limit 
The forwarding path tested the hop limit before testing whether the datagram was
addressed to this node, and decremented only on the forwarding branch. Two
consequences followed. A datagram addressed here that arrived with a zero hop
limit was dropped rather than delivered. And a forwarder receiving a transit
datagram with one hop left transmitted it with zero for the next hop to discard,
wasting a transmission on every expiring datagram.

Delivery to the addressed node is no longer gated on the hop limit, and the
decrement now happens before the drop decision, so a datagram that would leave
with zero is not sent. Saturating subtraction folds the two arrivals that cannot
be transmitted, already exhausted and last hop, into a single test.

The reachable radius does not change, which is worth stating because it is easy
to assume otherwise. The old behavior refused to deliver at zero but was willing
to send at zero, and the new behavior is the mirror of that, so the two cancel: a
path of h links still delivers for any source value of h or more. What this
actually buys is the wasted transmission, the exhaustion counter charging at the
node that makes the decision rather than the one after it, conformance with the
specified semantics, and one real gain during a rolling upgrade, where an
unupgraded forwarder feeding an upgraded destination delivers one hop further
than either version does on its own.

Coordinate cache warming moved ahead of both decisions, so a transit datagram
later dropped for hop limit still contributes its plaintext coordinates. The only
arrivals this newly warms from are those with a zero hop limit, and every insert
they can make is already achievable with a value of one, so it grants nothing
that was not already available and adds no memory growth the cap does not bound.

Four tests added: delivery at zero to this node, the transit drop at one, the
transit pass at two, and the one per hop decrement across a three node chain,
which brackets the emitted value from both sides since it is only observable
through what the next hop does. One existing test was renamed because its
destination was this node, so it never exercised the transit path its name
claimed, and it asserted nothing at all.
 2026-07-22 01:53:53 +00:00
Johnathan Corgan feb1e02e9c Verify the peer static on both FMP handshake paths, not just at rekey 
Under Noise XX the peer static key is learned during the handshake rather
than pinned in advance, and neither of the two paths that learn one was
checking it against what we already knew. That let an attacker who could
observe and inject on path substitute their own identity, on both a fresh
dial and an established link.

On a fresh dial we recorded who we meant to reach and then overwrote it
with whoever answered, without ever comparing the two. Promotion, the ACL
check and the peer registry all then ran on the answering identity, so an
attacker who raced the real peer to msg2 became the peer and the intended
node was never reached. On an established link, a rekey msg2 was matched
to its peer only by the session index we had put in the cleartext msg1
header, so anyone who saw that header could answer with their own static
and take the link over at the cutover.

Both are now compared before anything is committed. The dial-time
expectation moves into its own field with no setter, so the handshake
cannot overwrite it the way it used to; the identity learned from msg2
keeps landing where it always did. Anonymous dials still promote whoever
answers, which is what shared-media discovery means, and that branch is
chosen locally when we dial rather than from anything on the wire, so it
cannot be reached as an exemption. Both comparisons are decisions made in
the synchronous core alongside the existing classifiers.

The gates sit ahead of every mutation, not merely ahead of the session
install, so a forged msg2 no longer poisons the recorded peer epoch and
never earns a msg3. A rejected dial is rescheduled from the dial-time
expectation rather than the answering identity, so refusing an impostor
does not silently retire a configured peer from the dial schedule.

Only this branch was exposed. The released lines use Noise IK, where the
initiator pins the responder static before it dials and a wrong key fails
the AEAD outright.

No wire change: both gates compare a value we already learn against one
we already hold, and legitimate handshakes behave exactly as before.

Six tests cover it, driving the real cadence and dial paths with a third
node answering the intercepted message. Each was checked to fail when the
decision is forced to accept. Local CI 37/37.
 2026-07-21 19:40:50 +00:00
Johnathan Corgan deb5ded8eb Merge master into next after the v0.4.1 rollover 
master carries only maint's 0.4.2-dev version bump linkage; next keeps its
own 0.6.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
 2026-07-19 19:06:16 +00:00
Johnathan Corgan 80c956a6fd Merge maint into master after the v0.4.1 rollover 
maint carries only its 0.4.2-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
 2026-07-19 19:06:08 +00:00
Johnathan Corgan 75d7077880 Open 0.4.2-dev cycle on maint after v0.4.1 release 
Bump the version to 0.4.2-dev and update the status badge and prose. The
0.4.x patch line continues; v0.4.1 remains the shipped release it points at.

Follows the shape of the v0.4.0 rollover commit. No maint reset was needed
this time: v0.4.1 was cut from maint, so the tag is already on this line and
it simply continues.
 2026-07-19 19:05:13 +00:00
Johnathan Corgan d0a0824768 Merge master into next after the v0.4.1 release 
Same split as the maint-to-master merge: version identity stays with the
receiving branch, project state flows up.

Kept next's: Cargo.toml and Cargo.lock at 0.6.0-dev, the status badge, and
the paragraph identifying next as the wire-format-breaking line that will not
interoperate with v0.2.x, v0.3.x, or v0.4.x peers. Only the shipped-release
pointer inside it moved from v0.4.0 to v0.4.1.

The changelog conflict was additive rather than competing, and resolving it
either way would have lost real content. Next's [Unreleased] Fixed section
carries the XX rekey divergence and dual-initiation work; master brought the
[0.4.1] section. Git could not tell these were adjacent rather than rival, so
both were kept in order, with next's Breaking block and its own [Unreleased]
entries untouched.

Took from master: the bloom FPR default change and its duplicate-definition
fix, the docs describing them, the v0.4.1 release notes, the v0.4.0 date
correction, and the root RELEASE-NOTES.md mirror.

Checked before merging that no incoming content names the Noise handshake
pattern, since next is XX where master is IK. The two "Noise IK" strings on
master both predate v0.4.0 and next already carries its own wording for them,
so nothing needed rewording here.

Quartet green: 1698 tests passed, clippy clean with -D warnings.
 2026-07-19 19:03:40 +00:00
Johnathan Corgan a47ddbd5a5 Merge maint into master after the v0.4.1 release 
Carries the v0.4.1 content up the one-way branch flow. The release itself
belongs to maint, so the parts of this merge that identify a version are
resolved in master's favor and the parts that describe the project's state
are taken from maint.

Kept master's: Cargo.toml and Cargo.lock at 0.5.0-dev, the README status
badge, and the "FIPS is at v0.5.0-dev on the master branch" line. A patch
release consumes no minor version and master's development line is
unaffected by it.

Took from maint: the bloom antipoison FPR default change and the
duplicate-definition fix it rests on, the two docs that describe them, the
new v0.4.1 release notes, the correction to the v0.4.0 release date, and the
[0.4.1] changelog section, which slots below master's own [Unreleased] and
above [0.4.0]. The root RELEASE-NOTES.md mirror moves to v0.4.1 because it
tracks the latest shipped release, which also clears the stale provisional
date it had been carrying.

One line needed splitting rather than choosing: the README said "v0.4.0 has
shipped" inside the paragraph that identifies master as 0.5.0-dev. The
version identity stays master's and the shipped-release pointer moves to
v0.4.1.

Quartet green on the result: 1645 tests passed, clippy clean with -D warnings.
 2026-07-19 18:57:52 +00:00
Johnathan Corgan 7089a2949b Merge branch 'master' into next 2026-07-19 07:40:07 +00:00
Johnathan Corgan ea74cd7e58 Merge branch 'maint' 2026-07-19 07:40:07 +00:00
Johnathan Corgan 0dea111fab Merge branch 'master' into next 2026-07-19 07:17:26 +00:00
Johnathan Corgan 37adb13d5b Merge branch 'maint' 2026-07-19 07:17:26 +00:00
Johnathan Corgan 564b31a914 Merge branch 'master' into next 
Conflict in the admission-cap suite: this line's copy is the XX variant,
with a different phase 3 and an extra phase the other line does not have.
Took the polled final read from the merge, kept this variant's more
specific failure message and its enforcement-evidence phase.

Also templated two container references that phase and the peer-list read
own. They are unique to this line, so the pass that introduced the per-run
name suffix never saw them, and this suite would have looked up containers
under names that no longer exist once a run sets the suffix.
 2026-07-19 06:48:17 +00:00
Johnathan Corgan 93a5b71728 Merge branch 'maint' 2026-07-19 06:46:55 +00:00
Johnathan Corgan f1879b8fe9 Merge branch 'master' into next 2026-07-19 06:45:39 +00:00
Johnathan Corgan 4ff7de4d81 Merge branch 'maint' 2026-07-19 06:45:18 +00:00
Johnathan Corgan afdae094dc Merge branch 'master' into next 
Conflict in the FMP core doc pass: this line has no inbound establish
view at all, so the block that master edited does not exist here and the
merge saw a modify-versus-absent conflict. Resolved by keeping this
line's side - the establish view is master-line only and tracks the
handshake-pattern split, so its absence here is correct rather than an
omission to repair.

The lifecycle view correction and the comment rewrites in the node
module carry across unchanged.
 2026-07-19 06:15:21 +00:00
Johnathan Corgan 0c3d9a0b73 Correct stale view-trait docs and drop internal shorthand from comments 
The lifecycle view's doc described the shell as implementing it over a
`connections` map that no longer exists. The establish view named that map
too, and was doubly wrong: both of its snapshot builders read only
`peers`. Rewrite both to describe the maps actually read.

Several comments in the node module carried shorthand from private
working notes - bare parenthetical labels, and one reference to an
internal tracker item - none of which mean anything to a reader of this
repo. Rewrite them so each clause states its own reasoning inline. The
paragraph above the tracker reference already carried the rationale, so
that one is simply dropped.

Comment-only; no behavior change.
 2026-07-19 06:11:40 +00:00
Johnathan Corgan 60b1d27c0d Merge branch 'refactor-node' into refactor-node-next 2026-07-19 04:25:36 +00:00
Johnathan Corgan 281ed132f1 Merge branch 'master' into refactor-node 2026-07-19 04:25:36 +00:00
Johnathan Corgan 6e54161022 Merge branch 'next' into refactor-node-next 2026-07-19 04:25:36 +00:00
Johnathan Corgan c5492f4572 Merge branch 'maint' 2026-07-19 04:25:25 +00:00
Johnathan Corgan cb96a601c2 Merge branch 'master' into next 2026-07-19 04:25:25 +00:00
Johnathan Corgan 26bd1ada5b tests: guard the identity carrier and the inbound denial teardown 
Two properties that the connection-state consolidation depends on had
no test on this line, because the pattern here is Noise XX and the
natural place to assert them under IK is msg1 -- which under XX carries
only ephemeral keys and so proves neither.

The first is that a learned peer identity lands on the surviving
carrier, the control machine's own connection state, rather than
staying locked inside the Noise handle. Every reader that names a peer
mid-handshake goes there: the promotion hand-off reads it to decide who
is being admitted, and the stale-connection sweep reads it to decide
whether a reaped leg is retried or torn down. XX learns identity once
per role at two different messages, so the guard asserts both -- the
initiator learning the responder from msg2, and the responder learning
the initiator from msg3. The initiator half starts from a deliberately
wrong dial-time expectation, so a missing write cannot pass by leaving
a pre-seeded value in place.

The second is that a peer the access list turns away leaves nothing
behind. Under XX the inbound admission gate cannot sit at msg1, since
no static key has crossed the wire yet, so it sits at msg3 -- by which
point the responder has built a control machine, allocated a session
index, opened a link, and completed the Noise session. All of that has
to come back down on the denial. A machine left registered is
unreachable by every teardown path and holds a peering-budget slot
forever; an index left mapped misroutes the next frame to arrive on it.

Both guards were confirmed to fail when the write they pin is dropped
and when the teardown they pin is skipped. Tests only; no production
behavior changes.
 2026-07-19 03:30:12 +00:00
Johnathan Corgan b0cef7a221 Merge branch 'refactor-node' into refactor-node-next 
Brings the handshake-leg deletion onto the XX line: the leg struct
and src/peer/connection.rs are gone, the Noise handles and crypto
methods now live on PeerMachine, and every leg accessor reads the
machine's own ConnectionState.

The incoming branch speaks Noise IK, this one speaks XX, so the crypto
methods could not be taken as they arrived. Git added the IK versions
to machine.rs with no conflict marker, and complete_handshake_msg3
-- which XX needs and IK has no counterpart for -- was absent
from the merged tree entirely. All four are re-expressed here: the
incoming structure (the Option<HandshakeCrypto> handle, the self.conn
reads, the borrow scoping that hoists carrier writes out of the leg
borrow) carrying this line's XX bodies. start_handshake is the one
that mattered most: its signature is identical on both lines, so it
compiled silently, and left alone it would have emitted an IK msg1
through the two-argument new_initiator and panicked every anonymous
dial on an expect for an identity XX does not have at that point. It
now uses the one-argument XX form and no expect.

handle_msg1 keeps this line's flow, not the incoming one's. The two
functions share a name but not a shape -- the admission gate moved
out of msg1 into msg3 when the line went to XX, and identity is
unknown until msg3. The machine is built above the crypto because
it now drives it, but it stays a local until the late insert, so a
rejected msg1 still leaves no registry trace and the drop-the-local
error arms are unchanged. Parking it at SentMsg2 after the index is
allocated needed a transition the birth constructor could not provide,
so that constructor now delegates to it.

Node::start_handshake is re-expressed the same way and for the same
reason: keeping the machine a local until all fallible setup has
succeeded preserves the existing error arms exactly, with no machine
disposal to add.

The two leg-to-carrier mirror blocks collapse. With one carrier
the writes they mirrored are self-assignment. Their content
is preserved: the completion touch comes from the relocated
complete_handshake, on the same clock, and the negotiated profile
from process_fmp_negotiation, which now takes the machine directly
and writes through set_conn_peer_profile.  Worth recording, because
the collapse was justified on a narrower claim that does not hold:
complete_handshake writes only the touch. It never wrote the profile
on either line. The two writes land on the same carrier at the same
point, so the collapse is neutral, but it is discharged jointly by
two methods rather than by complete_handshake alone.

Three inbound tests are dropped rather than carried. Each asserts
a property at msg1 that XX does not have there: an ACL decision, a
no-registry-trace guard on that decision, and an identity learn on the
carrier. Under XX msg1 is ephemeral-only, so none of the three has a
landing site at that step. A comment at each drop site records where
the property actually lives on this line. One of them was a deletion
here that the incoming branch had merely modified, so its removal
is this line's own prior decision rather than a new one. The same is
true of the craft_and_send_msg1 helper and of HandshakeSeed::inbound,
both left without callers once those tests went.

Also carried: peer_actions.rs had no conflict and no incoming change,
but three sites reached deleted Node accessors and are repointed at
the machine; the EstablishView snapshot is not resurrected, as this
line has never had it; and the connection-state doc keeps this line's
XX wording with the registry-independence invariant grafted in.
 2026-07-19 02:42:24 +00:00
Johnathan Corgan 7fe1d75637 peer: delete the pending-connection type, leaving the control machine whole 
The per-peer control machine has absorbed every field the pending
connection carried. What remained was a struct holding two Noise
handles beside a duplicate copy of bookkeeping nobody read. Replace it
with a small carrier for the two handles and delete the type.

Presence of that carrier, not the state of the handles inside it, is
what marks a machine as mid-handshake. The distinction is essential
rather than stylistic: a failed handshake drops its initiation handle
and is deliberately retained so the stale sweep can reclaim it, and a
completed one has its session taken before disposal. Deriving presence
from the handles would make both invisible to the sweep, the
connection count, and the peering budget at once, leaking the slot
permanently. A test drives an empty carrier past every presence
predicate and then detaches it, so a future edit cannot quietly couple
the two.

The remote startup epoch now comes from the surviving carrier, which
the handshake operations already wrote at the same two points with the
same value. The paired writes onto the pending connection's own
bookkeeping had no readers left and are gone.

The handshake-phase surface leaves the public API: it was public by
accident rather than design, and the machine behind it is crate
internal. Callers outside the crate that need a view of pending
handshakes go through the operator queries, which are unchanged.

ConnectionState::inbound_with_transport loses its last non-test caller
with the inbound seed and is marked test-only.
 2026-07-19 01:18:05 +00:00
Johnathan Corgan e21e09d7e6 peer: source the handshake identity and index family from the control machine 
The pending connection and the per-peer control machine have carried
duplicate copies of the handshake-phase fields since the machine gained
its own connection state. Read them from the machine and drop the
connection's projections.

The peer identity is the sharp one. The connection learned it from msg1
and the machine's carrier did not, so the two views genuinely disagreed
for inbound connections until the Noise operations moved onto the
machine and began recording each result on both. That is now in place,
so every reader can take the machine's copy unchanged: the establish
snapshot, the promotion sweep for competing connections, the dial and
path in-progress checks, the peering observations and per-peer in-flight
budget, and the stale-connection sweep's retry address.

Also repointed: our_index, their_index, transport_id, started_at, the
stored handshake message bytes, and the idle-timeout check. Two
duplicate index writes on the connection are dropped, both immediately
preceded by the machine-side write of the same value. Reads that
previously came off a connection detached just before its machine was
disposed now capture the machine's value first.

Test seeding follows the establish paths: the seed builders write
our_index to the carrier, which promotion now reads. A new test pins
the inbound identity learn on the carrier and the retry address a
failed inbound connection reports to the sweep, so a silent regression
to a blank identity cannot pass.

ConnectionState::duration loses its last non-test caller with the
connection accessor and is marked test-only.
 2026-07-19 00:25:17 +00:00
Johnathan Corgan e7537929ba node: move link, direction, and peer address onto the control machine 
These three had no counterpart on the control machine, so readers still
reached them through the pending connection. Add machine-side
accessors and repoint every reader, then drop the connection's.

The peer address needed its writes lifted, not just its reads
repointed: the machine's copy was never written. It is now written at
each of the points the connection's copy was — the inbound seed, the
dial, message-2 completion, and the two paths that seed a machine from
a pre-built connection — so promotion and the resend path read a value
with the same provenance at the same time as before.

Link and direction need no lift. Both machine constructors already
seed them from the same arguments the connection is built with, so an
outbound machine carries outbound state and an inbound one inbound,
and the machine's link always equals the connection's. The handshake
operations' direction guards read the machine's copy for the same
reason.

The stale-connection sweep's teardown log and its resend path both now
take the transport and address from the machine, and each keeps its own
check that a pending connection is still attached rather than relying
on the caller to have established it.

Add a test pinning link, direction, and address on the two shapes that
seed a carrier independently — the dial and an accepted message 1. The
cross-connection winner reads both values from a carrier one of those
two already seeded.
 2026-07-18 23:17:21 +00:00
Johnathan Corgan 347cbe60bd peer: move the Noise handshake operations onto the control machine 
The pending connection drove its own Noise handshake while the control
machine held it, so the crypto and the state it produces lived on the
carrier that is going away. Move the six operations onto the machine:
starting and completing an initiation, processing an inbound
initiation, taking the session, testing for one, and dropping the
handle on failure. Bodies are unchanged apart from reaching the
handles through the attached connection.

Each operation now records its results on both carriers. The learned
identity, the remote epoch, and the activity stamp are written to the
machine's own bookkeeping at the same point and with the same value as
they are written to the connection's. The connection's copies still
have readers until those reads are repointed, so both have to be
written; the machine's copy of the learned identity was previously
never populated for an inbound connection, which is why an inbound
pending row showed no expected peer.

Driving the handshake from the machine means the machine has to exist
before the crypto runs. On the inbound path it is built above the
message-1 processing and still kept local, so a rejected message
leaves no registry entry and allocates no index. On the outbound path
it already existed from the dial, so it simply takes the connection
before the index allocation, and both failure arms unwind it as they
did.

Completing message 2 no longer mirrors the activity stamp separately,
since the completion itself now writes both carriers at the point the
mirror was approximating.

Three tests cover what the compiler cannot. A connection whose
handshake failed holds neither Noise handle yet must stay visible to
the stale-connection sweep, or every failed connection would leak and
hold a peering-budget slot forever. A message 1 rejected by the crypto
or by the ACL must leave no machine, no index, and the same rejection
count as before. A dial whose message-1 preparation fails must unwind
the machine registered at dial time; that test drives the
index-allocation failure rather than a crypto failure, which is the
arm this change actually widens, since the allocation now happens with
the connection already attached.
 2026-07-18 22:55:38 +00:00
Johnathan Corgan 11ec16777c node: yield the control machine when iterating pending connections 
`Node::connections()` yielded the pending connection itself, so its
consumers reached the handshake-phase fields through that value. The
pending connection is being folded into the control machine, and the
fields will move off it, so yield the machine (keyed by its link) and
let each consumer reach what it needs from there.

Membership is unchanged: the iterator still selects machines that
carry a pending connection, which is the predicate `connection_count`
and the stale-connection sweep already use. Every consumer takes the
same value from the same place, one hop further out, and no field
changes carrier here.

The method is now internal to the crate. It yields the control
machine, which is not part of the published surface, and nothing
outside the crate iterates pending connections — operator views of
the handshake phase go through the query surface instead.
 2026-07-18 22:06:46 +00:00
Johnathan Corgan 5b09e22956 node: read promotion's carrier fields in one borrow 
`promote_connection` detached the pending connection and then
interleaved reads off that detached value with three separate lookups
of the same control machine. Gather the machine-side fields
(`their_index`, `transport_id`, link stats) in the single `get_mut`
that takes the connection, so the machine is borrowed once.

Behaviour is unchanged. The connection is still taken before anything
is validated, so a rejected promotion leaves the machine with no
pending connection, and the prelude only gathers options: the checks
below it still report the first missing field in the same order
(`our_index`, `their_index`, `transport_id`, `source_addr`).

Link stats move to the prelude, ahead of those checks. The value is
the same either way: they live on the surviving carrier rather than
the detached connection, and nothing between the two points touches
the machine map. The lookup could not fail at the old site either,
since the function had already reached it through a successful lookup
on the same key, so dropping the defaulting arm changes nothing.

Add a test covering the error order and the detach, driving promotion
with connections missing each required field in turn plus every later
one, so an implementation that validated during the prelude would
report the wrong field. Test seeding grows a variant that lets the
caller shape the seed.
 2026-07-18 21:45:09 +00:00
Johnathan Corgan a382b17931 node: seed control machines directly in tests, ahead of removing the leg 
Tests built a free-standing PeerConnection, mutated it, and handed it to
Node::add_connection by value. None of those sites survives the removal of
PeerConnection, so converting them afterwards would mean one enormous commit
that cannot be reviewed honestly. Convert them now, while the struct still
exists and the conversion can be validated against a green tree.

Adds a cfg(test) Node::seed_handshake_machine plus a HandshakeSeed builder,
and rewrites make_completed_connection (now seed_completed_connection) and
the twenty inline builders onto it. add_connection keeps its body and loses
its test callers.

The builder's carrier seeding is a verbatim copy of add_connection's: the two
conditional writes for their_index and transport_id, then set_leg, through the
same entry().or_insert_with() so an existing leg-less machine keeps its
constructor-side fields. Nothing else reaches the carrier -- our_index,
source_addr, post-construction started_at and the stored handshake bytes stay
leg-only. Seeding more than that would let these tests observe a carrier
richer than production's and keep passing even if a later production
write-lift were missed.

The Noise exchange now runs on the already-seeded leg rather than before the
hand-over. That is neutral because the only read of expected_identity is
guarded by is_outbound, and no crypto method allocates a session index.

Also adds a compile-time check that PeerAction is Clone + Eq, which is what
keeps a runtime handle from being smuggled into an action payload.
 2026-07-18 21:24:03 +00:00
Johnathan Corgan a3c53a3b3c Merge branch 'refactor-node' into refactor-node-next 
Re-establish ancestry after both branches independently merged their
respective mainline.
 2026-07-18 20:42:45 +00:00
Johnathan Corgan 1e6b4cb918 Merge branch 'next' into refactor-node-next 
Bring the Android-ready core (target_os gating and app-owned TUN seam)
onto this branch so subsequent work builds on the current mainline tree.
 2026-07-18 20:42:45 +00:00
Johnathan Corgan a90049d3a1 Merge branch 'master' into refactor-node 
Bring the Android-ready core (target_os gating and app-owned TUN seam)
onto this branch so subsequent work builds on the current mainline tree.
 2026-07-18 20:38:56 +00:00
Johnathan Corgan df52ce459c Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the state-carrier collapse (connection start/activity timestamps,
peer-identity telemetry, handshake resend buffers, index/transport bookkeeping,
and the session-index shadow all moved onto the peer machine's own connection
state) onto the XX handshake.

Next-specific resolutions folded into the merge: the responder learns its transport
id and peer index at msg1 and they are written directly onto the machine there (the
XX machine has no msg1 step, so the promote read would otherwise be unset); the
negotiated peer profile is mirrored onto the machine carrier at both the initiator
(msg2) and responder (msg3) negotiation sites; and the outbound carrier writes land
on both the identified and anonymous dial paths. The leg's transport/index/buffer
copies remain as dead duplicates until the leg dissolves; the machine copy is the
sole live source.
 2026-07-18 18:24:44 +00:00
Johnathan Corgan 791b35c221 node: make the peer machine's connection state the sole session-index carrier 
The session index was carried three ways: on the leg, on the peer machine's own
connection-state copy, and on a separate machine-owned shadow field. Populate the
machine's connection copy on the outbound path (it was written only on the inbound
and cross-connection-swap paths before), then delete the shadow field and route its
readers, the decrypt-registration lifecycle, and the rekey cutover through the
single carrier. The shadow and the connection copy diverged only on paths that are
unreachable or dormant in production (a rekey cutover with no newly allocated index,
the swap winner's later reads, and the never-dispatched timeout/disconnect events),
so the merge is byte-identical for every reachable path. Rewrote the outbound-promote
index tests and the dial-time comment for the single-carrier contract.
 2026-07-18 17:19:44 +00:00
Johnathan Corgan 6a80790742 node: source connection index/transport/stats from the peer machine 
The connection leg's their_index, transport_id and link_stats duplicated the peer
machine's own connection-state copy. Route every production reader through the
machine copy: promotion, the stale-connection reaper, the outbound msg1 resend, and
the in-use / connecting-path checks. Seed the inbound machine's transport_id from the
msg1 packet (the same source the leg used), matching the outbound dial seed, so the
hard-required promote read never sees a missing transport. link_stats is a
never-mutated zero seed, so its leg accessors are removed outright; the leg
their_index/transport_id getters/setters remain only for the test-only connection
builder. Also delete the write-only next_resend_at_ms field from the connection
state (the resend deadline is carried by the machine-armed retransmit timer, not this
field). Byte-identical.
 2026-07-18 16:43:22 +00:00
Johnathan Corgan 60b8acf716 node: source handshake resend buffers from the peer machine 
The connection leg's stored msg1/msg2 handshake-resend buffers duplicated the
peer machine's own connection-state copy. Write the machine copy at the outbound
msg1-prep and the two inbound authorize sites, then source the retransmit
resend-bytes reader, send_stored_msg1, and the pending tier of find_stored_msg2
from the machine. The post-promote active-peer msg2 copy is written from the same
wire bytes, so the pending tier now matching after promotion is value-identical.
Byte-identical resend behavior.
 2026-07-18 16:00:15 +00:00
Johnathan Corgan 4fc295d90a node: source connection peer-identity telemetry from the peer machine 
The expected_peer connection-row field read the peer identity off the leg;
source it from the machine's own connection-state copy instead, matching the
started_at/last_activity treatment. The machine carrier holds the same identity
as the leg for every connection that appears in the view: outbound legs seed
both from the dialed identity, and inbound legs never rest in the connections
view (the machine takes the leg at promotion), so their machine copy is
unobserved. The leg accessor stays for the crypto-extraction path and retry
decisions. Byte-identical.
 2026-07-18 15:32:04 +00:00
Johnathan Corgan b3f2018fce node: source connection start/activity timestamps from the peer machine 
The connection leg's started_at and last_activity duplicated the peer
machine's own connection-state copy. Make the machine copy the sole live
telemetry and timeout carrier: re-stamp it with the leg's provenance when
the leg is attached at handshake start (the msg1-prep clock, not the earlier
dial-time constructor value) and mirror the completion touch, then delete the
leg's last_activity/touch accessors and source the connection-row projection,
show_connections, and the timeout readers from the machine. Byte-identical
for all normal paths.
 2026-07-18 15:11:43 +00:00
Johnathan Corgan 91ba660213 Merge branch 'refactor-node' into refactor-node-next 
Re-express the handshake-state carrier collapse onto the XX code: the leg's
handshake_state field is deleted and the displayed state is derived from the
peer machine's phase, with failure carried on the machine (a send_failed flag
that preserves the handshake phase) rather than on the leg. The next projection
maps the SentMsg2 responder phase to received_msg1 and the anonymous-dial
Discovered phase to sent_msg1; the three initiator send-failure sites carry
failure via send_failed. Telemetry strings, wire bytes, index allocation, and
stale-connection reaping are byte-identical to next.
 2026-07-18 04:08:26 +00:00
Johnathan Corgan 7ea5718270 Merge branch 'master' into next 2026-07-18 02:57:30 +00:00
Johnathan Corgan 56bbc81a40 node: derive handshake state from the peer machine, delete the leg field 
The connection leg's HandshakeState field duplicated the peer machine's
handshake phase. Delete it and derive the displayed handshake state string
from the machine's PeerState, looked up by link id (mirroring the resend
count). Move the failure signal onto the machine: a send_failed flag that
preserves retransmit eligibility (the machine stays in its handshake phase),
alongside the existing Failed state. The leg's crypto self-gates now guard on
Noise handle presence instead of the deleted phase field, and mark_failed
only drops the handle. Telemetry strings, wire bytes, index allocation, and
the stale-connection reaping are byte-identical for all normal paths.
 2026-07-18 02:26:39 +00:00
Johnathan Corgan f7fa257349 node: classify inbound msg3 via a single machine decision site 
handle_msg3 now routes on the InboundDecision returned by the machine's
inbound_msg3 call, instead of computing establish_inbound in the shell and
handling the terminal arms inline. The DualRekeyWon and ResendMsg2 arms'
index free moves to the machine's FreeIndex action (run through the
executor); the ResendMsg2 stored-msg2 resend stays inline. The pre-decision
reject gates and the proceed-arm bodies are unchanged.

Behavior-neutral: the same establish_inbound decision over the same snapshot
and wire, evaluated once in the machine, with a byte-identical free-count on
every arm and no change to wire sends, machine state, or telemetry.
 2026-07-17 21:02:05 +00:00
Johnathan Corgan 7e5619ad32 peer: return the inbound msg3 decision from the machine arm 
on_inbound_msg3 becomes inbound_msg3, returning the InboundDecision
alongside the machine-phase actions so a caller can route on it; the step
arm delegates and drops the decision. The signature is unchanged, so
step-driven callers are untouched.

Also drop two emissions that are dead on the live path: the ResendMsg2
arm's SendHandshake (the stored-msg2 resend is a driver mechanism the
executor would otherwise reframe) and the dormant inbound-msg1 arm's
retransmit timer. No live behavior changes.
 2026-07-17 20:43:08 +00:00
Johnathan Corgan b1adbe3af7 Merge branch 'refactor-node' into refactor-node-next 
Carries the inbound establish-decision reshape from the IK line. Next's XX
inbound surface already computes that decision on its own machine, so the
merge records the shared history without changing next's tree.
 2026-07-17 20:19:33 +00:00
Johnathan Corgan b38f8c6ffb node: classify inbound msg1 via a single machine decision site 
handle_msg1 now creates one local machine before classification and routes
on the InboundDecision it returns, instead of calling establish_inbound in
the shell and re-deriving the decision inside each arm. The promote and
restart arms drop their own per-arm machine construction and the redundant
InboundMsg1 step; their phase-1 actions come from the single call. The
effect-bearing arm bodies (rekey-respond, duplicate resend, reject
bookkeeping, and the promote/restart tails) are unchanged.

Behavior-neutral: the same establish_inbound evaluation over the same
snapshot and wire, done once in the machine method rather than once in the
shell, with byte-identical arm bodies and no change to index allocation,
wire sends, machine state, or telemetry.
 2026-07-17 19:17:25 +00:00
ArjenandJohnathan Corgan cf62cff5f4 feat: Android-ready core: target_os gating and app-owned TUN seam 
Make the FIPS core build and run as an embedded Android library. The host
app owns the TUN (e.g. an Android VpnService) and FIPS performs no
system-TUN or CAP_NET_ADMIN operations.

Squashed from the following changes:

- gate desktop transports/TUN by target_os, not features: a plain
  `cargo build` now compiles for every target with no flags. Ethernet (raw
  AF_PACKET / BPF) is gated to linux/macos, so Android (target_os =
  "android", not "linux") self-excludes it as Windows already did; real
  system-TUN ops are gated per linux/macos and Android gets a no-op stub;
  the ipi6_ifindex cast handles it being i32 on Android vs u32 on macOS. No
  Cargo features are introduced; desktop builds are unchanged.

- app-owned TUN seam: Node::enable_app_owned_tun() lets an embedder that
  owns the TUN fd exchange IPv6 packet bytes with FIPS over channels
  instead of FIPS creating a system TUN device. It returns (app_outbound_tx,
  app_inbound_rx): the embedder pushes packets read from its fd into the
  outbound sender (app -> mesh) and pulls packets destined for its fd from
  the inbound receiver (mesh -> app). start() gates system-TUN creation on
  tun_tx being unset, so with the channels pre-installed it skips device
  creation and does no system-TUN ops; both directions reuse the existing
  inbound-shim and run_rx_loop wiring. Packets entering via app_outbound_tx
  bypass handle_tun_packet, so the embedder must push only fd00::/8-destined
  packets and clamp TCP MSS on outbound SYNs; the rustdoc and the
  IPv6-adapter design doc spell this out.

- keep the android target warning-clean so the cross-compile check passes
  clippy -D warnings.

- add an Android cross-compile CI check: cross-compile the library for
  aarch64-linux-android via cargo-ndk and run clippy -D warnings. Android
  ships as an embedded library (the host app owns the TUN), so there is no
  daemon binary to package; this is a check job, not a packaging one.

- docs: list Android as a supported platform.

Tests: app_owned_tun_seam_wires_channels covers the channel round-trip and
the Active state; start_skips_system_tun_when_app_owned runs start() and
asserts no named system device is created.
 2026-07-17 18:25:55 +00:00
Johnathan Corgan 7fe3388f2f peer: return the inbound establish decision from the machine arm 
The inbound msg1 machine arm re-derived the establish decision for
its own action selection while the shell matched on a separately
computed copy. Expose the arm as a method returning the decision
alongside the actions so a driver can route on it directly; the
step dispatch delegates and keeps its action-only shape.

The dead rekey-respond arm is stripped to decision-only and its
helper deleted: it allocated from the real index allocator, wrote
the rekey shadow index, emitted a wrong-framed rekey send, stamped
the dampening clock, and flipped state, none of which happens on
the live path, where the inline respond body owns all effects. The
resend-msg2 arm's send emission is stripped the same way; the
decision already carries the stored bytes.
 2026-07-17 13:47:50 +00:00
Johnathan Corgan f65cd07048 Merge branch 'refactor-node' into refactor-node-next 2026-07-17 07:43:36 +00:00
Johnathan Corgan 252d16fab9 node: feed msg1 send failure to the peer machine as an event 
send_stored_msg1 marked the embedded leg failed by writing it
directly from the shell. Route the write through a new
HandshakeSendFailed machine event instead: the machine marks its
leg so the stale-connection sweep reclaims it, without leaving the
handshaking state, so retransmit eligibility survives the window
between the failed send and the sweep exactly as before.

send_stored_msg1 gains a now_ms parameter threaded from the action
executor for the step call; the failure arm itself ignores it.
 2026-07-17 07:12:40 +00:00
Johnathan Corgan 94d7b91244 node: route the outbound msg2 establish decision through the machine 
handle_msg2 no longer pre-computes establish_outbound alongside the
machine's own evaluation of the same snapshot. The shell now builds
the snapshot, steps the persistent outbound machine once at the
decision point, and routes on what comes back: the promote action
vector drives promotion through the executor as before, and the
ResolveCrossConnection decision selects the inline swap/keep
resolution bodies, which are unchanged.

The machine step and its defensive transient-rebuild move up from
the promote arm to the decision point; the cross-connection path
keeps its take-leg-then-dispose ordering with the step preceding
both. Comment prose at the touched sites refreshed to describe the
single-decision-site shape.
 2026-07-17 06:59:30 +00:00
Johnathan Corgan 7790eb86bd peer: make the outbound msg2 swap/keep machine arms decision-only 
The machine's on_msg2 cross-connection arms crystallized state and
emitted FreeIndex/RegisterDecryptSession actions that duplicate the
inline shell resolution, but they are unreachable on the live path
(the shell removes the machine before running the swap/keep bodies).
Making them live in that shape would double-free the outbound index
through the executor.

Strip both arms to a single new ResolveCrossConnection { swap }
action: a decision conveyed to the driver, not an effect. The shell
intercepts it and runs the inline resolution, which owns all effects
permanently. The action executor gets a defensive unreachable arm.
The Promote arm and set_their_index are unchanged.
 2026-07-17 06:44:09 +00:00
Johnathan Corgan 4fa565e528 Merge branch 'refactor-node' into refactor-node-next 
Carries the leg-embed storage move: PeerMachine.leg replaces the
Node.connections map. Hand-resolved onto the XX surfaces keeping this
line's semantics with the new storage: the msg3 termination arms and
executor swap/rekey-responder teardowns keep their exact cleanup sets
and index-free behavior; the msg2 cross-connection extract takes the
leg before the unconditional machine dispose; msg1 and anonymous leg
births embed the leg at machine insertion; the rekey-vs-establish gate
tests leg-absence. The merge also drops the now-redundant explicit
machine inserts next to the seeded test seam, whose auto-merged
combination clobbered leg-carrying machines.
 2026-07-17 05:07:36 +00:00
Johnathan Corgan cf1c957336 node: embed the handshake leg in the peer machine, drop the connections map 
The pending-handshake PeerConnection map and the per-peer control
machine map were parallel LinkId-keyed structures whose keysets must
stay coherent by hand. With every leg now born with a machine, the leg
becomes storage inside its machine (leg: Option<PeerConnection>, pure
storage the machine never reads or drives) and Node loses the
connections field; every access routes through the machine.

The non-mechanical lowerings, each argued at the site: the
rekey-vs-establish gate in handle_msg2 tests leg-absence (an established
peer's machine stays keyed by its link, so machine-presence would
misclassify every rekey msg2 as a fresh establish); the
connecting-predicates, peering observation, and handshake-slot budget
iterate machines-with-legs so connect-window machines (leg not yet
born) are excluded exactly as before and never double-counted against
their pending-connect slot; the cross-connection extract takes the leg
before disposing the machine; the stale reaper takes the leg and leaves
the machine untouched when none is present, matching the old early
return. The map-coherence debug check keeps its machine-has-carrier
direction with the embedded leg as a carrier; the leg-to-machine
direction is now true by construction and its gate const is gone.

connection_count() counts machines with legs; the connections()
iterator, the test seams, and the control-socket connection rows are
re-implemented over the embedded legs with unchanged output.
 2026-07-17 04:27:53 +00:00
Johnathan Corgan ba3c552dce node: give anonymous outbound legs a persistent control machine 
The anonymous-discovery dial paths drove no control machine: the leg was
born bare in start_handshake and a throwaway machine appeared only at
msg2. Anonymous legs now birth a persistent identity-less machine
alongside the leg (new_outbound widened to Option<PeerIdentity>, backed
by the existing identity-less outbound connection state), and handle_msg2
crystallizes the learned identity onto it before stepping. The msg2
missing-machine arm becomes defensive recovery (debug-assert + insert,
matching the IK line), and the self-connect drop arm disposes the
machine it previously leaked — reachable from identified dials too,
since the handshake completion overwrites the expected identity without
comparing it to the dial-time expectation. The self-connect arm's
index/link/pending-entry leak is pre-existing and stays as-is.

With every leg now born with a machine, the leg-to-machine direction of
the debug coherence check switches on for this line. The add_connection
test seam seeds direction-faithful machines (an outbound-anonymous
seeded connection previously got an inbound-shaped one), and the
hand-rolled dial sites in tests seed dial machines mirroring
production. New tests pin the identity-less birth, the msg2 identity
crystallization through promote, and the self-connect disposal.
 2026-07-17 03:25:28 +00:00
Johnathan Corgan d6785f2970 node: make the inbound handshake machine persistent across its leg's life 
The inbound control machine was a msg3-time throwaway: handle_msg3 built
a transient, stepped it once for the decision, and discarded it, with
promote_connection birthing a fresh established() machine. Every
handshake leg now gets one persistent machine for its whole life:

- handle_msg1 births the machine parked in the sent-msg2 phase alongside
  the window leg (msg1 crypto and the msg2 build/send stay inline; the
  msg2-send-failure cleanup disposes it).
- handle_msg3 steps that persistent machine instead of a transient. The
  decision path is unchanged: the msg3 handler never reads machine state
  and overwrites the identity-plane fields from the wire outcome before
  dispatching, so every arm's decision and actions are byte-identical.
- promote_connection stops rebirthing: the executor feeds the promotion
  result back into the machine (the shape the IK line already uses) and
  the machine crystallizes to Established in place.
- Every path that tears down an inbound window leg now disposes its
  machine through remove_peer_machine: the seven inline msg3 termination
  arms, the six executor swap/rekey-responder consumption sites, both
  promote-failure arms, the single-peer leaf reject, the losing side of
  a cross-connection, and the msg1 send-failure cleanup. The
  promote-failure disposal also fixes a pre-existing leak: an outbound
  leg whose promotion fails had its connections entry consumed before
  the error, so the stale reaper could never reach its dial machine.

Index-free behavior is unchanged at every site. Two test helpers that
hand-roll outbound dials now insert the dial machine production
inserts, and new tests pin the machine's birth phase, its post-promote
crystallized state, and disposal on the failure and rekey-responder
arms.
 2026-07-17 02:54:00 +00:00
Johnathan Corgan 3f9de1adcd Merge branch 'refactor-node' into refactor-node-next 
Carries the debug-build peer-map coherence check. The leg-to-machine
direction is gated OFF on this line (const flipped in the merge): the
inbound msg1-to-msg3 window legs and anonymous-discovery outbound legs
legitimately run machine-less here, so only the machine-to-carrier
leak-tripwire direction asserts until every leg is born with a
persistent machine.
 2026-07-17 02:13:31 +00:00
Johnathan Corgan 5ccd95cf3f node: add a debug-build peer-map coherence check 
Assert, once per tick in debug builds, that the peer control-machine map
and its carriers stay coherent: every control machine has a live
handshake leg, an active peer, or a pending connect (a machine with none
is a leak the stale reaper can never see), and every handshake leg has a
machine. The second direction sits behind a file-local const so a branch
where handshake-window legs legitimately run machine-less can gate it
off without weakening the leak tripwire.

Teach the add_connection test seam to seed a control machine for the leg
it inserts (derived from the connection's direction and identity), and
remove_connection to dispose it, so the seam-built topologies satisfy
the check. Three unit tests: seam coherence, coherence through a real
promotion, and the orphaned-machine panic.
 2026-07-17 02:09:28 +00:00
Johnathan Corgan 4779a3970d Merge branch 'refactor-node' into refactor-node-next 
Carries the dead-code deletion (PeerSlot enum, PeerConnection resend
API) and the module-doc liveness corrections. Hand-resolved the doc
conflicts with next-appropriate wording: the executor module doc names
the live msg3 decision-machine path (PromoteToActive /
SwapToInboundSession / RekeyRespondTrigger) while keeping the still-true
msg1-inline dormancy note; the peer_machines field doc reflects next's
promote-time established() births; the machine module doc drops the
stale unwired claims, including the provisional-and-unwired note on the
msg3 trigger variants which are live.
 2026-07-17 01:41:21 +00:00
Johnathan Corgan 119b85d28e node: correct stale liveness claims in machine, executor, and timer docs 
Several module and field docs still described the per-peer machine as
unwired shadow scaffolding. It has been live for some time: machines are
inserted at dial and inbound msg1, stepped by the handshake handlers and
the rekey-cadence and liveness-reap routers, and the executor's
SwapSendState/CompleteDrain/InvalidateSendState arms are the authoritative
paths (the inline bodies survive only as debug-assert release fallbacks).
Rewrite those docs to the current truth while keeping the still-true
dormancy facts: PeerEvent::Timeout and PeerEvent::Tick are never
dispatched in production, retransmit fires on the machine-armed deadline
while the timeout reaper keys on timer presence with the config
threshold, and the remaining inert executor stubs are SendRekey,
SendLinkMessage, and the connected-UDP arms. Drop the stale
allow(dead_code) on the peer_machines field.
 2026-07-17 01:33:06 +00:00
Johnathan Corgan 74245e80ac peer: remove the dead PeerSlot enum and PeerConnection resend API 
PeerSlot (and its entire impl surface) was referenced only by its own
module tests and the lib.rs re-export; peer storage has always used the
separate connections/peers maps. PeerConnection's resend_count/
next_resend_at_ms/record_resend delegations had no production callers:
the live resend counter is machine-sourced (connection_resend_count reads
the per-peer machine), and the FSP session layer uses SessionEntry's own
methods. ConnectionState::next_resend_at_ms is now test-only (its
remaining callers are the fmp state unit tests) and marked cfg(test).

The PeerSlot unit tests go with the enum; test_resend_count_tracking is
dropped because the delegation target's schedule arithmetic is already
covered by the fmp resend_bookkeeping test.
 2026-07-17 01:32:49 +00:00
Johnathan Corgan 5a8da01f24 Merge branch 'master' into next 
Fold in a next-only wording fix for the OpenWrt 802.11s mesh backhaul:
next's FMP link handshake is Noise XX, not the Noise IK on master, so the
fips-mesh-setup header and the how-to now name it pattern-agnostically
("the Noise handshake"), matching the shipped fips.yaml comment and
staying accurate on this branch.
 2026-07-16 05:56:36 +00:00
Johnathan Corgan e42598a86e Merge branch 'maint' into master 2026-07-16 05:55:20 +00:00
ArjenandJohnathan Corgan 054d17aac5 feat(openwrt): 802.11s open-mesh backhaul support 
Router-to-router radio backhaul over an open 802.11s mesh interface,
with FIPS providing all encryption (Noise IK), authentication, and
routing on top of bare L2 neighbor links. The mesh runs OPEN with
mesh_fwding 0 — SAE would duplicate the Noise layer and force ath10k
raw mode, and FIPS is the routing layer — so the Noise handshake is the
real auth/encryption boundary and FIPS's spanning tree does the routing.

fips-mesh-setup: an opt-in UCI helper that creates a per-radio
mesh-point interface (radio0 -> fips-mesh0, radio1 -> fips-mesh1;
trailing-digit derivation with a free-index fallback and a collision
guard). Radio setup stays opt-in — a package must not commandeer radios
on install. 'remove' takes an optional radio and otherwise removes all
instances. Dual-band routers get one instance per radio; FIPS treats the
two backhaul paths as failover, not multipath: it keeps one active link
per peer (cross-connection resolution picks a single winner), and the
second band stands by, re-establishing the peer after keepalive timeout —
traffic never uses both bands at once.

fips.yaml ships the mesh0/mesh1 Ethernet-transport entries commented
out, so a stock install that never creates fips-mesh* logs no per-boot
"interface missing" bind warning. fips-mesh-setup uncomments the matching
meshN block when it creates the interface and re-comments it on remove,
so the flash-and-drop-in flow needs no manual config edit. The file is
rewritten 0600-first (it may hold an inline nsec) via an atomic replace.

Two field-found silent non-peering causes are surfaced by the helper and
the guide:

- Same channel: mesh points only peer on a shared channel, and 'auto'
  lets each radio pick its own. The helper prints the radio's
  band/channel and warns loudly on 'auto' with the exact uci command to
  pin one; the how-to gains an ordered no-peers triage (channel mismatch,
  on-air scan check, DFS CAC wait, regdomain).
- STA channel capture: a client (sta) interface drags the whole radio to
  its upstream AP's channel, so a mesh pinned elsewhere never joins and
  does not recover until the STA disconnects. The helper warns when the
  target radio carries a STA; the guide documents the incompatibility of a
  roaming uplink with a fixed-channel mesh on the same radio.

Both the create and remove paths run 'wifi reload', which briefly drops
every client AP on all radios; the how-to sets that expectation.

Regression test: the shipped OpenWrt fips.yaml must parse via the real
Config deserializer in both states — as shipped (mesh inactive) and after
the uncomment the helper performs.

Packaging: the helper is installed across the ipk/apk/buildroot paths
(three synced copies), with the CI structural checks and shellcheck
targets extended to cover it. Full guide in
docs/how-to/set-up-80211s-mesh-backhaul.md.
 2026-07-16 05:48:04 +00:00
Johnathan Corgan 252b77683e Merge branch 'refactor-node' into refactor-node-next 
Re-express the rekey-msg2 and cross-connection observation feeds onto
the XX cores. The rekey-msg2 observation lands inside the block that
installs the pending session (where XX also generates msg3), so it fires
only when the install actually ran. The cross-connection observation
sits at the outbound resolution arms and targets the promoted peer's
machine; the inbound session-swap effects already own their executor
path here and are untouched. Crypto effect bodies are byte-unchanged.
 2026-07-16 00:04:31 +00:00
Johnathan Corgan 87399795f8 node: observe rekey-msg2 and cross-connection resolution in the per-peer machine 
Feed the per-peer machine two observation events so its control/rekey
shadow state stays coherent with the inline crypto-session installs,
without moving those installs. The crypto effect bodies are unchanged.

On rekey-msg2 completion, feed the machine the responder index on the
success path only (after the pending session is installed); the abandon
path feeds nothing.

On cross-connection resolution, feed the machine the swap outcome: on a
swap it records the new outbound and responder indices to mirror the
session replacement, and on a keep it does nothing. The observation
targets the promoted peer's machine, not the outbound leg whose machine
is removed as the block is entered.

Both observation handlers emit no action; nothing consumes the updated
shadow state yet.
 2026-07-15 23:50:18 +00:00
Johnathan Corgan 2fad6be5e6 node: execute inbound session-swap effects through the per-peer machine 
Move the two Noise-XX inbound establish effects — the cross-connection
session swap and the rekey-respond session install — out of the inline
handle_msg3 arms and into their per-peer machine executor actions
(SwapToInboundSession, RekeyRespondTrigger). The inbound establish
decision now falls through to the shared machine-drive arm, matching the
promote/restart arms.

Carry the session index on the InboundMsg3 event and seed it into the
transient machine before dispatch. The swap actions build their index
from the machine's connection state, which is unset on a fresh inbound
transient; without the seed the actions would emit empty and the swap
would silently no-op while leaking the allocated index.

Give the inbound reject and msg2-resend machine arms their terminal
Failed state and index free, matching the inline arms that own those
paths, so a later timer cannot free the index or report loss for a peer
whose handshake has already resolved.

Preserve exactly: the three distinct index frees and their paired index
and pending-outbound map removals; the link-teardown asymmetry (the
cross-connection path drops the address mapping, the rekey-respond path
keeps it); the session take-then-install ordering; and every reject kind
on the error sub-paths.
 2026-07-15 23:12:48 +00:00
Johnathan Corgan efbfb32e07 Merge branch 'refactor-node' into refactor-node-next 
Fold the handshake timer lifecycle (msg1 retransmit + timeout/stale
reap) through the per-peer machine, re-expressed onto the XX handshake
surface.

- Route peer-machine removal through remove_peer_machine so each link's
  timer store is dropped together with its machine (choke-point).
- Home the msg1-retransmit decision on the machine-armed retransmit
  timer; advance the resend counter only on send success and relocate
  the operator-visible resend count onto the machine.
- Reap outbound handshake timeouts via a presence-scan over the machine
  HandshakeTimeout timer, reading the threshold from config each tick so
  the reap stays neutral for any handshake_timeout_secs.

Cancel both dial-armed handshake timers on outbound promote so a
promoted machine carries no stale timer entry. Preserve the guard that
suppresses a msg1 resend at a peer already promoted via the inbound
cross-connection path.

The XX inbound HandshakeTimeout presence-scan is neutral only because
the inbound establish path sends msg2 inline and never dispatches the
inbound machine event, so no inbound leg ever populates a
HandshakeTimeout timer. A future change that drives inbound establish
through the machine must re-verify the reap equivalence for inbound
legs.
 2026-07-15 21:38:52 +00:00
Johnathan Corgan 765819f52b node: reap the handshake timeout through the per-peer machine timer 
Move the outbound handshake-timeout reap off the unconditional check_timeouts
scan onto the machine-armed HandshakeTimeout timer. A new drive_handshake_timeouts
(run before the retransmit drive, so a timed-out leg is reaped rather than resent
on the same tick) reaps the outbound legs that carry a HandshakeTimeout timer and
have idle-timed-out this tick.

The timer's presence selects the leg (only outbound legs arm one; IK inbound arms
none); the reap threshold is the shell is_timed_out(now, config) predicate, not
the timer's stored deadline. The machine arms the timer from a hardcoded constant
at dial, which is not authoritative for an operator-tuned handshake_timeout_secs,
so reading the threshold from config each tick keeps the reap neutral for any
timeout value and on the last_activity clock exactly as before.

check_timeouts keeps reaping everything else: failed connections (all of them,
promptly) and the idle-timeout of legs without a machine timer (inbound legs, and
any machine-less connection). Total coverage is unchanged. check_timeouts runs
before the timer drive, so a failed outbound leg is reaped there first, its timers
dropped, and the timeout drive never double-fires on it.

Split drive_peer_timers into the timeout drive and the (byte-identical) retransmit
drive. The dormant machine timeout handler is not dispatched, so the session index
is freed once, by cleanup_stale_connection. A regression test drives a timed-out
outbound leg to reap; the existing check_timeouts tests continue to anchor the
residual sweep. Test count 1631 to 1632.
 2026-07-15 20:23:49 +00:00
Johnathan Corgan 5021197f5c node: drive the handshake msg1 resend from the per-peer machine timer 
Move the outbound msg1 resend off the unconditional tick function onto the
machine-armed retransmit timer. The per-peer machine already arms a
HandshakeRetransmit deadline at dial; a new drive_peer_timers fires the due ones
(kind-filtered to the retransmit timer) and homes the resend counter on the
machine, where the operator-visible count now reads from.

The resend decision reads the same operator config as before (interval, backoff,
max), the wire bytes and transport target still come from the shell connection,
and the pure core computes the backoff schedule. As with the deleted
resend_pending_handshakes, the count and reschedule advance only on a successful
send: a failed send neither advances the count nor marks the connection failed,
it just retries on the next tick. The handshake-timeout timer stays on the legacy
check_timeouts path, and the rekey/liveness timers keep their own shell drivers,
so drive_peer_timers deliberately fires only the retransmit kind.

The show_connections resend count is relocated to read from the machine (the
counter's new home) via connection_resend_count; machine-less and inbound
connections report 0, matching what the shell connection reported before. Delete
resend_pending_handshakes and resend_candidates (the latter also from the
LifecycleView trait, its only user). The resend unit test is re-expressed against
the machine + timer path, covering the due check and the record-on-success
semantics.

Known limitation: the first resend interval is armed from the machine's hardcoded
1000ms constant, which equals the config default. Under an operator override of
handshake_resend_interval_ms the first resend diverges from the pre-change
dial+interval; subsequent resends and the cap remain config-driven. Neutralizing
the first-resend override needs the interval threaded into the sync core (the
shell arm would be clobbered by the machine's own timer arming at dial), deferred
to when the connection and machine entities merge. Test count unchanged at 1631.
 2026-07-15 19:38:49 +00:00
Johnathan Corgan 31f5a8c1b7 peer: add a per-peer timer store populated by the machine timer actions 
The control machine already emits SetTimer/CancelTimer actions when it arms the
handshake retransmit and timeout deadlines, but their executor arms were a
single no-op stub and the machine's Timeout event was never dispatched -- the
real work still runs on the legacy tick (check_timeouts,
resend_pending_handshakes).

Add the storage the time-as-input driver will read: a peer_timers map keyed by
LinkId then TimerKind, holding each armed timer's absolute deadline. The SetTimer
arm now inserts (overwrite = reschedule) and CancelTimer removes; the outbound
msg2 promote cancels the two dial-armed handshake timers, since the machine
survives promotion and the entries would otherwise linger in the store.

This store is a shadow: it is written and cleared but no driver reads it yet, so
behavior is unchanged. The legacy tick stays authoritative until the driver that
feeds Timeout and deletes the overlapping tick paths lands.

Route every machine removal through a new remove_peer_machine(link) choke-point
that drops the timer store alongside the machine, replacing the twelve direct
peer_machines.remove sites so no armed timer outlives its machine. TimerKind
gains Hash (to key the store) and Ord (for deterministic driver collection); it
is internal and never serialized. Test count unchanged at 1631.
 2026-07-15 18:49:50 +00:00
Johnathan Corgan 8abcd6ff8c Merge branch 'refactor-node' into refactor-node-next 
Bring the outbound-handshake-through-the-machine series onto the next-branch
Noise-XX cores. Re-expressed rather than transcribed: next has no outbound
decision core, so the promote drives PeerEvent::OutboundMsg2 (not the IK Msg2
snapshot event) and cross-connection resolution stays inline. Hybrid provenance
-- identified outbound legs persist the control machine at dial and drive
through it; anonymous-discovery legs (identity unknown until XX msg2) retain the
inline transient-at-msg2 path. ReportLost{kind} routing, the connection-oriented
dial via the machine (OpenTransport + TransportConnected), and the prepare/send
msg1 split all carried over.

Behavior-neutral on next: the XX msg2 body (Noise completion, FMP, ACL, msg3) is
untouched; anonymous legs are unchanged; the dial-persisted machine promotes
with our_index unset.
 2026-07-15 15:52:29 +00:00
Johnathan Corgan 3e7ca90212 peer: drive the connection-oriented outbound dial through the machine 
Route the connection-oriented (TCP/Tor) outbound path through the peer state
machine, matching how the connectionless path already works.
initiate_connection's oriented branch now drives PeerEvent::Dial with
connection_oriented=true; the machine parks in Connecting and emits
OpenTransport, whose executor arm performs the non-blocking transport.connect
and pushes the PendingConnect. When the connect resolves, poll_pending_connects
prepares msg1 in the shell and then drives PeerEvent::TransportConnected, which
sends msg1 via the machine's SendHandshake arm.

The msg1 prepare (index allocation, Noise leaf, wire arming) MUST run in the
shell before the TransportConnected drive: send_stored_msg1 only transmits an
already-armed wire, so a drive-only path would silently send nothing. The
machine's our_index stays unset; the connect-failure path keeps its direct
handshake-timeout handling (TransportFailed stays dormant). The now-unused
Node::start_handshake helper is removed.

Behavior-neutral: same transport.connect, same PendingConnect, same msg1 send
and failure teardown as the removed inline path -- only the driver changes from
inline shell code to the state machine.
 2026-07-15 14:40:58 +00:00
Johnathan Corgan 9588c50063 peer: persist the outbound control machine at dial, not at msg1 prepare 
Move the peer_machines insert for an outbound leg out of the tail of
prepare_outbound_msg1 to a single shared site in initiate_connection, before
the connection-oriented / connectionless fork. This lets the connection-oriented
path find the machine at dial time (so it can be driven through the connect
handshake) without prepare_outbound_msg1 -- which for that path runs after the
connect completes -- clobbering an in-progress machine back to Discovered.

Because the machine now exists before the fallible dial steps, add
peer_machines.remove to every failure path in the widened dial window: the
index-allocation and Noise-leaf failures in prepare_outbound_msg1, the oriented
transport.connect failure, and both poll_pending_connects teardown arms
(handshake-start failure and async connect failure). remove_link does not touch
peer_machines, so these explicit removes are required.

Behavior-neutral: the connectionless path still drives the machine to
Handshaking and sends msg1 identically; the machine's our_index stays unset
(no spurious UnregisterDecryptSession on a later inbound restart); a failed dial
leaves peer_machines empty for that link exactly as before; and no
connection-oriented machine drive is wired here.
 2026-07-15 14:12:49 +00:00
Johnathan Corgan f698da50b6 peer: add unit coverage for the connection-oriented outbound dial path 
The connection-oriented (TCP/Tor) outbound connect->handshake path had no
`cargo test --lib` coverage; it was exercised only by the opt-in Tor
integration suites, and the TCP node tests bypass it via a manual
connectionless handshake helper. Add three unit tests:

- a machine-level test driving Dial{connection_oriented:true} -> Connecting
  (emitting only OpenTransport, no msg1) -> TransportConnected ->
  Handshaking{SentMsg1}, asserting the exact OpenTransport and
  SendHandshake+SetTimer action vectors that start_outbound_handshake emits
  (its oriented reach via on_transport_connected was previously untested; the
  connectionless reach via on_dial was already covered);

- two node-level tests over a real loopback TcpTransport: a successful connect
  that reaches start_handshake (observed via a pending_outbound entry), and a
  connect to a closed port that routes through the failure arm (link torn down,
  no msg1 dispatched).

Tests only; no production change.
 2026-07-15 13:52:34 +00:00
Johnathan Corgan 1f765cfd8f peer: drive the connectionless outbound msg1 send through the machine 
Split the outbound handshake setup out of start_handshake into
prepare_outbound_msg1 (allocate the index, run the Noise leaf, frame and arm
msg1 -- the fallible steps, returning an error the caller propagates) and
send_stored_msg1 (transmit the armed wire). A connectionless dial now runs
prepare in the shell, then drives the control machine, whose SendHandshake
action sends the wire via the executor. Connection-oriented dials keep calling
start_handshake, now prepare followed by the send inline.

The dial event gains a connection_oriented flag so the machine's on_dial sends
msg1 immediately for connectionless transports (no connect step) instead of
opening a transport first. Behavior is unchanged: the same index, Noise leaf,
wire bytes, maps, and send-error handling as before, with index-allocation and
Noise failures still propagated synchronously before the send. A regression
test covers that a promote from the post-dial handshaking state is identical to
the former discovered-state promote.
 2026-07-15 06:15:21 +00:00
Johnathan Corgan 3b99a416ad peer: persist the outbound control machine at dial 
Create and persist the per-peer control machine when an outbound handshake is
dialed, keyed by its link, instead of building a transient at msg2. The msg2
completion path now looks up that persisted machine to drive the promote,
falling back to a transient only if none is present (e.g. a direct-seeded test).

The machine parks in the Discovered state until promotion and is inert to the
liveness reap and rekey cadence while unpromoted, since it is absent from the
peers map. It is removed on every path that ends the outbound leg without
promoting -- the stale-connection reaper, the msg2 authorization-failure arm,
and the cross-connection resolution block -- mirroring the connection's own
lifetime so no dangling machine survives.

Its session index is deliberately left unset on the machine (the shell owns the
index on its connection), so a later inbound restart does not emit a spurious
decrypt-session unregister. A regression test covers that invariant.
 2026-07-15 04:06:44 +00:00
Johnathan Corgan e064c96df3 peer: route peer-loss reports by kind 
Add a LostKind discriminator to the ReportLost action so the executor can
route an un-promoted handshake failure to the connected-guarded reconnect
reflex (note_handshake_timeout) and an established peer's link-death to the
unconditional one (note_link_dead), instead of collapsing every loss to
note_link_dead.

The two loss producers dispatched today, the liveness reap and the
inbound-restart-then-promote arm, both keep the link-dead routing, so
behavior is unchanged. The handshake-timeout and dial-failure producers are
tagged accordingly but stay dormant until their events are dispatched.
 2026-07-15 03:37:31 +00:00
Johnathan Corgan 4620d9f82e Merge branch 'refactor-node' into refactor-node-next 
# Conflicts:
#	src/node/dataplane/peer_actions.rs
 2026-07-15 01:30:14 +00:00
Johnathan Corgan 7e4358782e node: free the inbound session index on the msg3 reject and resend paths 
On two inbound msg3 outcomes the msg1-allocated inbound session index
was dropped without returning it to the allocator: losing the dual-rekey
tie-break (we keep our in-progress rekey and drop their msg3), and a
duplicate handshake that resolves to a msg2 resend. Under repeated hits
this slowly leaks index space. Free the index on both arms, matching the
sibling reject arms.

Add a test-only helper to backdate a peer's session-established instant,
and two regression tests that drive a real second handshake into each
arm and assert the index is returned to the allocator (both fail if the
free is removed).
 2026-07-15 01:26:54 +00:00
Johnathan Corgan a70c725e48 node: drop stale dead-code allows and a redundant clock read 
Remove the now-stale `#[allow(dead_code)]` on `advance_peer_machine`
and `execute_peer_actions`; both are live (called from the link-dead
reap, rekey cadence routing, and the handshake establish sites).

In `route_rekey_cadence`, reuse the ambient context timestamp for the
machine step instead of taking a second clock sample; the extra
sample's only consumer was the currently-inert cutover drain timer, so
the change is behavior-neutral.
 2026-07-15 00:56:58 +00:00
Johnathan Corgan 21b93f4116 Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the action-contract round-trip test. This line's action type
carries two extra establish-trigger variants (SwapToInboundSession,
RekeyRespondTrigger), so the merge resolution extends the test's sample set
and its wildcard-free exhaustiveness match to cover all of them — the test
proves every variant, including the two XX-only triggers, round-trips
unchanged through the async channel.
 2026-07-14 23:52:43 +00:00
Johnathan Corgan c8077967cd peer: add a round-trip contract test for the action message type 
Prove the per-peer machine's action type is a runtime-agnostic message
contract: it is Send + Sync + 'static, and every variant round-trips
unchanged through a single-threaded async channel (a current-thread runtime,
sender and receiver on one thread). A wildcard-free match over the variants
makes any future action a compile error here until it is added to the sample,
so a variant that embedded a runtime handle could not slip through unproven.
 2026-07-14 23:36:18 +00:00
Johnathan Corgan a6f7fc2d2f Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the promote-failure warning target pin. The peer-action
executor diverges between the two lines (IK vs XX establish arms), so the
pin was applied to this line's own two promote-Err warns during the merge
resolution: outbound "Failed to promote connection" and inbound "Failed to
promote inbound connection" now emit under fips::node::handlers::handshake,
matching the sibling max-peers reject log already pinned to that target.
 2026-07-14 23:29:34 +00:00
Johnathan Corgan 5dfa571908 node: pin promote-failure warnings to the handshake tracing target 
The two promote-failure warn! sites in the peer-action executor emit under
this module's own tracing target rather than the handshake target the rest of
the establish-path logging uses. Pin both (outbound "Failed to promote
connection" and inbound "Failed to promote inbound connection") to
fips::node::handlers::handshake, matching the sibling max-peers reject log and
keeping all establish-path diagnostics grouped under one target regardless of
which module physically emits them.
 2026-07-14 23:23:54 +00:00
Johnathan Corgan 22a5f1f807 Merge branch 'refactor-node' into refactor-node-next 
# Conflicts:
#	src/node/dataplane/dispatch.rs
#	src/node/dataplane/peer_actions.rs
#	src/node/handlers/handshake.rs
#	src/node/handlers/mmp.rs
#	src/node/handlers/rekey.rs
#	src/node/mod.rs
#	src/peer/machine.rs
 2026-07-14 01:40:01 +00:00
Johnathan Corgan 6bebca88ac node: rewrite planning-phase locators out of source comments 
Comments across the per-peer machine, executor, lifecycle supervisor,
and peering reconciler carried internal rollout labels and design-note
section references. Rewrite them to describe the code's behavior
directly. Comment-text only; no code changes.
 2026-07-14 01:38:52 +00:00
Johnathan Corgan 1e934e7603 node: drive net-new outbound promote through the peer machine 
Route the outbound side's msg2-completion promote through the per-peer
state machine and its action executor, mirroring how the net-new inbound
promote is driven. A transient outbound machine emits the promote action;
the executor performs the promotion and, on success only, clears the
pending-outbound entry (threaded in through the action context). The
outbound cross-connection path stays inline as before.
 2026-07-13 23:26:30 +00:00
Johnathan Corgan 40d1a5bca7 node: pin the max-peers inbound-reject log to the handshake tracing target 
The debug log for rejecting an over-cap inbound connection is emitted from
the peer-action executor, under that module's tracing target. Nodes that
raise only the handshake module to debug (leaving other modules at info) no
longer surfaced the rejection, so cap-enforcement events were invisible in
their logs. Re-pin the log to the handshake target, matching the adjacent
promoting-inbound log, so the rejection stays visible under a
handshake=debug filter regardless of the executor module's own level.
 2026-07-13 23:24:56 +00:00
Johnathan Corgan cde1af2d1b node: drive inbound establish decision and net-new promote through the peer machine 
Route XX inbound establishment through the per-peer machine: handle_msg3 hands
the establish decision to a transient machine and drives the net-new promote
(Promote + RestartThenPromote) through the executor's PromoteToActive action,
which transcribes the shared promote block (promote_connection plus the
msg2/tree-announce/bloom follow-ups). The four non-promote decisions (Reject,
ResendMsg2, CrossConnect, RekeyRespond) stay inline byte-for-byte; the two
session-swap arms are deferred to a later non-neutral step.

The decrypt-worker register stays inside promote_connection (not relocated),
no promotion-resolved feedback is fed back, and the transient machine is never
inserted into peer_machines (the persistent machine is created once inside
promote_connection). Behavior matches the inline path exactly, and operator
diagnostics are preserved: the relocated promoting-inbound log keeps its
handshake tracing target, and the epoch-mismatch restart breadcrumb is
re-emitted.
 2026-07-13 22:26:19 +00:00
Johnathan Corgan 09d0d41639 node: drive rekey cadence cutover and drain through the peer machine 
Route the initiator rekey cutover and drain-completion through the
per-peer machine and executor instead of running them inline in
check_rekey. The batch poll_rekey and its per-peer snapshots stay
shell-side and unchanged, so the cross-peer phase ordering that governs
the shared index-allocator sequence is preserved; the machine only
consumes each decided action in that order.

The routed cutover and drain reproduce the inline bodies exactly (the
executor's swap-send-state and complete-drain arms), including the
info-level cutover log and the decrypt-worker re-register. Initiating a
rekey stays inline; the machine is fed a rekey-initiated observation
afterward to keep its control state coherent. A guard falls back to the
inline bodies if a machine is somehow absent.
 2026-07-13 21:29:50 +00:00
Johnathan Corgan c15e63e101 node: drive the link-dead peer reap through the per-peer machine 
Route the liveness reap through the per-peer machine: on a link-dead
timeout, the mmp reap advances the peer's machine with a link-dead event
and executes the resulting actions, instead of calling remove_active_peer
and note_link_dead inline.

The effect is unchanged — the machine emits InvalidateSendState then
ReportLost, which the executor maps back to remove_active_peer(peer)
then note_link_dead(peer, now), the same two calls in the same order.
The reap log stays shell-side so it keeps the handlers::mmp target, and
a guard falls back to the inline path if a machine is somehow absent.
 2026-07-13 20:52:58 +00:00
Johnathan Corgan 5498b2a6a3 node: populate the per-peer machine map at establishment 
Insert a per-peer machine into peer_machines at each promote site and
remove it on teardown, so every established peer has exactly one machine
entry keyed by its link id. Nothing drives the machine yet — the entry
is inert — but this correspondence is what the rekey and liveness-reap
folds depend on.

Inserts pair with the two promote_connection arms (normal promote and
cross-connection-won); removes pair with remove_active_peer and the
cross-connection loser teardown. A new PeerMachine::established
constructor parks the machine in the post-handshake Established state.
Behavior is unchanged: nothing reads the map on a live path.
 2026-07-13 20:33:29 +00:00
Johnathan Corgan 42de582ef0 node/dataplane: add the per-peer machine action executor (shadow) 
Add the executor that consumes the per-peer machine's actions, plus the
peer_machines map that homes the machines on the node. Both are shadow:
nothing populates the map or invokes the executor yet.

The rekey-cutover, drain, liveness-reap, and index-plane action bodies
are ported to reproduce the next-line handlers exactly (the cutover log
matches check_rekey's info-level line, kept under the handlers::rekey
target so operator and test log filters still see it). The establishment
actions (promote, cross-connect swap, rekey-respond) are deferred stubs
until the inbound establish path is wired.
 2026-07-13 20:13:16 +00:00
Johnathan Corgan 0eaae58dc8 peer: add the per-peer FMP control state machine (XX, unwired) 
Re-derive the per-peer FMP lifecycle state machine against the next-line
XX establishment cores, as an unwired module. It ports the master-line
machine's structure but adapts it to the XX handshake:

- inbound establishment splits into a msg1 event (allocate the index,
  send msg2, defer) and a msg3 classify event (identity crystallizes,
  establish_inbound runs), matching XX's identity-at-msg3 timing;
- the inbound decision handles all six InboundDecision variants including
  the XX-only CrossConnect;
- the outbound establish_outbound path and the two-phase authorize are
  dropped (no analog on the XX line);
- session-carrying establish effects (cross-connect swap, rekey-respond
  pending session) are represented as plain-data trigger actions; the
  session surgery is done shell-side when the machine is wired.

Nothing drives the machine yet; it is dead code pending the executor and
handler wiring. Unit tests cover the XX dispatch: inbound establish to
Established, dual-init tie-break, cross-connect at msg3, rekey cutover
key-consistency, and liveness reporting.
 2026-07-13 19:47:56 +00:00
Johnathan Corgan 49677cc426 Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the per-peer FMP decomposition from the master line onto
the next line. The establishment and rekey machine-drive on refactor-node
is built on the IK establishment cores (establish_outbound, identity
learned at msg1) that the next line replaced with the XX msg3 model, so
it cannot be carried textually; it is re-derived against next's XX cores
in follow-on commits on this branch.

This merge commit carries only the two changes that are neutral and
next-compatible as-is:

- OwnedFd hygiene for the connected-UDP socket (RawFd -> OwnedFd, drop the
  hand-rolled unsafe Drop; OwnedFd's own Drop closes the descriptor).
- The two-tier send-state boundary: the send-critical subset of ActivePeer
  is regrouped into a PeerSendState struct, with accessors kept stable so
  no caller changes. next's XX establishment, rekey, and liveness-reap
  behavior is unchanged (verified: exact field partition, identical
  accessor signatures and semantics, identical init values, 3-arg mmp
  construction).

The per-peer machine, its action executor, and the peer_machines map are
not carried here; they are re-derived against next's XX establishment
surface in subsequent commits on this branch.
 2026-07-13 18:56:41 +00:00
Johnathan Corgan 5d5da69a5b node: drive the link-dead peer reap through the per-peer machine 
Route each link-dead peer that the tick sweep's plan_heartbeats decides
to reap through the per-peer machine and executor, replacing the inline
reap body in check_link_heartbeats. The batch decision, the liveness
snapshots (read from the hot-path-written receive clock), and the
heartbeat-send arm stay shell-side and byte-unchanged; the machine only
consumes the decided LinkDeadSuspected, tearing the peer down via
remove_active_peer and reporting the loss to the reconciler exactly as
before, on the same tick with the same wall-clock timestamp. The reap
log stays shell-side.

The machine's link-dead handler no longer emits a decrypt-session
unregister keyed by its shadow index (the full peer teardown already
unregisters the real index; the shadow could have drifted to a reused
index), and its guard now covers the Established state a freshly
promoted peer sits in.

Handshake-timeout, retransmit, and stale-connection cleanup stay inline:
they act on pre-promotion legs that have no machine, and the loss reflex
they use differs from the link-dead one.
 2026-07-13 17:08:35 +00:00
Johnathan Corgan e05b868cf8 node: drive rekey cadence cutover and drain through the peer machine 
Route each Cutover and Drain that the shell-side batch poll_rekey decides
through the per-peer machine and the executor, replacing the inline
effect bodies in check_rekey. The batch decision and its per-peer
snapshots stay shell-side and byte-unchanged: poll_rekey phase-groups all
cutovers, then all drains, then all initiations across the peer set, and
that ordering governs the shared index allocator's free-then-allocate
sequence that appears on the wire, so the machine only consumes the
already-decided actions (a new RekeyConsume event) without re-deciding.
InitiateRekey stays inline (its Noise msg1 build is a shell-side leaf)
with a RekeyInitiated observation feeding the machine so its control
state stays coherent for the next tick's cutover.

The cutover and drain logs, which relocated into the executor in the
prior commit, are pinned back to the fips::node::handlers::rekey tracing
target so they stay visible under the operator's module log filter.

Also clears the machine's shadow draining_index on drain so a later
cross-connection resolution cannot double-free the already-freed index.
 2026-07-13 16:20:45 +00:00
Johnathan Corgan 0ebd1b44c0 node: add the initiator-cutover and drain executor actions (unwired) 
Prepare the establish executor to drive FMP rekey by activating the
SwapSendState action (initiator K-bit cutover via cutover_to_new_session,
with the gated decrypt-worker re-registration) and adding a CompleteDrain
action (erase the drained previous session: free its index, drop its
peers_by_index entry, unregister its decrypt session). Both reproduce the
current inline rekey.rs cutover and drain bodies. The machine's drain
mapping now emits CompleteDrain, using the real drained index rather than
a shadow copy.

Unwired: nothing drives the machine's rekey path yet (live rekey still
runs inline), so these arms are unreachable and the change is
behavior-neutral. The cadence fold that routes cutover and drain through
the machine follows.
 2026-07-13 14:51:11 +00:00
Johnathan Corgan 800cfb23e3 node: relocate decrypt-session registration to the establish executor 
Move register_decrypt_worker_session out of promote_connection into the
executor's PromoteToActive handler, gated on a promoted or
cross-connection-won result. Every live promote now flows through that
one executor path, so registration still fires exactly once at the same
synchronous point; the direct test callers of promote_connection spawn
no worker pool, so the call was already a no-op for them.

Add the cross-connection loser-link teardown (close the losing
transport, remove its link, re-point addr_to_link at the winner) to the
executor as a guarded follow-up. It is unreachable on the current driven
establish paths, which only promote net-new peers, and asserts so, but
keeps the executor complete for when that case is driven.

Remove the now-dead drive_promote_to_active and ConnAction::PromoteToActive.
 2026-07-13 14:20:56 +00:00
Johnathan Corgan e9112cc1bb node: drive net-new outbound establish through the per-peer machine 
Cut the net-new outbound handshake completion (a received msg2 that
promotes a fresh outbound leg to a new peer) over to the per-peer
control machine, mirroring the inbound cutover. handle_msg2 still runs
the msg2 prologue, the ACL check, and the cross-connection swap/keep
arms inline; for the net-new promote it now builds a transient machine,
steps it, and drives promote_connection through the executor. The
session index was already allocated at dial, so there is no two-phase
authorize here.

The wire, index sequence, and peer registry state after promote are
byte-neutral. To keep the promote-failure path neutral, the executor's
cleanup now distinguishes inbound from outbound: an outbound promote
failure records the reject only, matching the prior handler, rather
than the inbound path's link and index teardown.

Cross-connection swap/keep, rekey-msg2, and the dial path stay inline;
the loser-link surgery for the currently unreachable driven
cross-connection case lands with the register relocation next.
 2026-07-13 13:42:08 +00:00
Johnathan Corgan c80a7fdea5 node: drive restart inbound establish through the per-peer machine 
Cut the restart handshake (an inbound msg1 from a peer that reconnected
with a new epoch) over to the per-peer control machine, completing the
inbound establish cutover. The old peer is torn down and the fresh leg
promotes through the same two-step authorize-then-allocate path as the
net-new case: the machine's first step emits the old-peer teardown
(invalidate send-state, report loss), the shell interposes the ACL
check, and the second step allocates the new index and sends msg2. The
old index is freed before the new one is allocated and the msg2 wire
bytes are unchanged, so the sequence stays byte-neutral.

With restart driven through the machine, the shared inline establish
tail that only the restart arm reached is deleted.

Also bounds the peer_machines map (remove_active_peer now drops the
peer's machine entry) and restores the msg2-send, promote, and
index-allocation failure warnings the cutover had dropped.
 2026-07-13 12:46:23 +00:00
Johnathan Corgan 0bf031dd32 node: drive net-new inbound establish through the per-peer machine 
Cut the net-new inbound handshake (a fresh msg1 that promotes to a new
peer, plus the at-capacity reject) over from the inline handle_msg1 logic
to the per-peer control machine. handle_msg1 still classifies via
establish_inbound and still owns the Noise wire step, the late ACL check,
and the promote_connection registry surgery; for the net-new path it now
builds the machine, steps it, and executes the returned actions.

Authorization is interposed between two machine steps so the session
index is allocated only after the ACL check passes: a rejected or
unauthorized msg1 consumes no index, matching the prior order exactly.
The msg2 wire bytes, the index-allocation sequence, and the reject
metrics are all byte-neutral. Restart, resend, rekey-respond, and the
other reject arms stay inline unchanged; they move to the machine once
outbound establish is cut over and every promoted peer has a machine.

Also fills in the executor's send-failure and promote-failure cleanup so
a mid-establish error tears the leg down and frees its index exactly as
before.
 2026-07-13 12:05:17 +00:00
Johnathan Corgan 4a0584a5e9 node/dataplane: add the per-peer machine home and action executor (unwired) 
Add Node.peer_machines (a LinkId-keyed map from the stable link handle to
the per-peer control machine) as the home for the machines, and a new
dataplane/peer_actions.rs holding execute_peer_actions / advance_peer_machine:
the executor that maps each PeerAction the machine emits to its shell call —
frame and send a handshake via build_msg2, drive promote_connection and feed
the PromotionResult back through the machine, tear a peer down via
remove_active_peer, free session indices, report loss via note_link_dead.
Actions for the rekey, connected-UDP, and timer paths are stubbed with notes
for the commits that fold those mechanisms in.

Unwired: nothing drives the machine yet — no live handler path calls the
executor and peer_machines is never populated — so this is behavior-neutral;
the inbound and outbound establish paths still run their existing inline
logic. The executor is cut over path-by-path in the following commits.
 2026-07-13 11:11:20 +00:00
Johnathan Corgan 59155df4e3 peer: split ActivePeer send-state into PeerSendState (two-tier boundary) 
Draw the control/published-send-state boundary inside ActivePeer by
grouping the send-critical fields — the three epoch session slots
{current, previous, pending}, the K-bit flag and session-start, the
transport target, the connected-UDP handles, and the hot counters —
into a new co-located PeerSendState struct. The control-tier fields
(identity, connectivity, declaration/ancestry, filter and tree-announce
groups, remote_epoch, the rekey-negotiation sub-machine, and the rest)
stay on ActivePeer.

Behavior-neutral: a pure field regrouping. Every accessor signature is
unchanged (bodies now read/write self.send.*), so the hot path and the
handlers are byte-untouched; both K-bit cutovers still rotate the three
slots atomically with the same control-tier updates. No Arc/ArcSwap —
the fields are co-located and read by plain borrow; publishing behind a
shared cell is later plumbing for a sharded data plane.
 2026-07-13 10:36:15 +00:00
Johnathan Corgan fcaee74ec0 peer: add the per-peer FMP control state machine (unwired) 
Introduce src/peer/machine.rs: a sans-IO per-peer control FSM that
consolidates the scattered handshake/rekey/timeout driver logic now
spread across node/handlers. The machine is a pure reducer —
step(event, now, index_allocator) -> [action] — that reuses the
existing FMP decision cores (establish_inbound/establish_outbound/
cross_connection_winner/poll_*) rather than reimplementing any
decision, and returns runtime-agnostic actions the driver executes.

Control-tier state only; the published send-state boundary and the
driver wiring land in following commits. The machine is terminal at
Closed — re-dial is the reconciler's, so it holds no cross-attempt
retry state.

Includes eight unit tests: inbound and outbound establish, N:1
identity crystallization, the dual-initiation tie-break,
restart-override, rekey initiator cutover, the data-plane-owned
responder cutover boundary, and liveness -> link-dead -> report-lost.
Unwired — nothing calls it yet.
 2026-07-13 09:58:45 +00:00
Johnathan Corgan 56e3d56c25 peer/connected_udp: own the connected-socket fd with OwnedFd 
Replace the hand-rolled RawFd field and the unsafe Drop on
ConnectedPeerSocket with an OwnedFd, whose own drop glue closes the
fd. from_fd now stores the OwnedFd it already receives instead of
stripping ownership through into_raw_fd, and the manual libc::close is
gone, shrinking the unsafe surface.

Behavior-neutral: the fd still closes exactly once at last-Arc-drop and
as_raw_fd returns the same underlying fd while the socket is alive.
 2026-07-13 09:20:04 +00:00
Johnathan Corgan d58bd083fa Merge branch 'refactor-node' into refactor-node-next 2026-07-13 08:49:05 +00:00
Johnathan Corgan 7b0590f70e node/lifecycle: wire the runtime child-exit producer 
Wire exit detection for the four directly-observable optional children so
the supervisor FSM's ChildExited edge fires at runtime. A runtime
child-liveness mpsc channel carries a Child on exit: the DNS task and the
two TUN threads self-report when their body returns, and a 2s poll monitor
reports mDNS and Nostr via new is_finished accessors. The rx_loop gains a
select arm that steps the FSM and republishes health (Degraded, since a
running node always has at least one transport up). Transports and worker
pools expose no runtime-exit signal and are left for a follow-up.

Adds LanRendezvous::is_finished and NostrRendezvous::is_finished; the
latter treats a shutdown-taken connect_task as finished so the monitor
terminates after a stop instead of polling forever.
 2026-07-13 08:48:51 +00:00
Johnathan Corgan c08888afe6 Merge branch 'refactor-node' into refactor-node-next 2026-07-13 08:25:56 +00:00
Johnathan Corgan b93a127623 node/lifecycle: add runtime ChildExited health routing to the supervisor FSM 
Add the sans-IO half of runtime child-liveness monitoring: a ChildExited
event and an on_child_exited handler that routes a runtime task or thread
exit the same way a start failure does. An optional child exiting degrades
the node (Degraded, with the child recorded in the health reasons); the
last transport exiting publishes Failed. There is no restart, and the
handler is inert outside Running (startup, drain, and teardown own their
own child bookkeeping via the pending/up sets). Failed here is a published
health signal only, not a teardown.

The start and runtime paths share the health classification, extracted
from resolve_start_health into classify_health.

The producer that emits the event (the exit-detection wiring) lands in a
following commit; the event variant carries a temporary allow(dead_code)
until then.
 2026-07-13 08:25:45 +00:00
Johnathan Corgan fa3f88b433 Merge branch 'refactor-node' into refactor-node-next 2026-07-13 08:00:54 +00:00
Johnathan Corgan 85a4983dbe node/peering: restore the open-discovery sweep summary log 
The overlay-discovery cutover to the sans-IO reconciler dropped the
operator-facing "open-discovery sweep complete" summary, including its
per-reason skip breakdown, because the sweep logic moved into the
log-free core. Have the core accumulate the enqueue/skip tally as it
reconciles and return it as data; the driver adds the two values only it
holds (the raw cache size and the self-advert filter) and emits the
summary, reproducing the old startup-vs-per-tick summarize gate and the
budget-zero debug path.

The self-advert precedence matches the pre-cutover sweep: a stale or
self-configured own advert is attributed to the age/configured buckets,
not skipped_self.

Behavior-neutral: the tally is pure side-counting; no dial or enqueue
decision changes.
 2026-07-13 08:00:46 +00:00
Johnathan Corgan b7c3d8935c Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the peering homeostatic reconciler onto the XX/v2 handshake
line. The three scattered peering mechanisms (auto-connect and retry,
overlay discovery, and opportunistic transport-neighbor growth) are now
unified in the sans-IO reconciler on both lines.

Hand-resolved on the first-contact surface: the reconciler opportunistic
layer emits a connect for anonymous, identity-unknown transport legs
unconditionally (bypassing the connect budget and per-peer cap, gated only
by the addr_to_link dedup), reproducing the XX dial-before-identity-known
behavior; named legs route through the reconciler with inputs identical to
the master line. handle_msg1/msg3 keep the XX path; the peer-loss reflexes
use the relocated wrappers. Behavior-neutral versus the pre-merge next line.
 2026-07-13 06:40:05 +00:00
Johnathan Corgan 5090ab7851 node/peering: drive opportunistic transport-neighbor growth through the reconciler 
Cut the last scattered peering mechanism — opportunistic growth from
transport-neighbor beacons and LAN mDNS — over to the reconciler's
opportunistic layer via a gate-checked reconcile_opportunistic wrapper,
one call per tick slot (transport and LAN stay separate slots so their
per-tick budget and per-peer cap are not shared). The driver keeps the
beacon/mDNS I/O and the path-granular prefilters that read live state
(self, fresh-enough-to-skip, connecting-on-path) and executes the emitted
Connect intents; the core owns the connected/budget/per-peer-cap
decisions. A first-wins per-peer dedup on the LAN path reproduces the old
inline once-per-peer dial now that the snapshot core cannot observe the
intra-tick connecting feedback.

Delete the now-dead discovery_connect_budget helper. With this, all three
scattered peering mechanisms (auto-connect and retry, overlay discovery,
neighbor growth) are unified in one sans-IO reconciler. Behavior-neutral;
unit test count unchanged (1607).
 2026-07-13 06:02:21 +00:00
Johnathan Corgan 03ced618ce node/peering: drive Nostr open-discovery through the reconciler overlay layer 
Cut the overlay (Nostr open-discovery) enqueue over to the sans-IO
reconciler. A gate-checked reconcile_overlay wrapper runs the overlay
layer alone at the discovery tick slot; the monolithic reconcile would
re-fire the always-on retry-dial and double the per-tick dial cap, so the
overlay slot must call only its layer. The driver builds the candidate
pool from the overlay advert cache (self excluded), the cooldown and
configured-npub sets, and the startup/steady max-age, then feeds the
reconciler; the emitted enqueue set drives the per-enqueue identity-cache
and alias pre-seed exactly as before. Enqueued entries are dialed at the
retry slot, preserving the two-phase cadence.

Delete the now-dead enqueue-budget and expiry helpers. Behavior-neutral:
same candidates enqueued in the same order with the same budget, cap, and
expiry. Opportunistic transport-neighbor growth remains imperative and is
cut over next. Unit test count unchanged (1607).
 2026-07-13 05:34:28 +00:00
Johnathan Corgan bf81f422ea node/peering: drive the mandatory-floor and retry mechanism through the reconciler 
Route the auto-connect floor and the connection-retry mechanism through
the sans-IO reconciler instead of the imperative Node methods. A thin
peering driver (note_handshake_timeout / note_link_dead) centralizes the
reflex path with the gate guard and the already-connected check; the
per-tick retry-dial and the startup floor build reconciler inputs and
execute the emitted Connect intents. The three imperative retry methods
are removed and their call sites rerouted.

Wire the drain and startup gates. Entering a bounded drain now clears the
retry schedule and suppresses the peer-loss reconnect reflex (the drain
window runs under the Suspended gate), so the drain no longer fights a
reconnect for the peers it just closed. The startup floor runs under an
explicit Reconciling gate at the existing peer-connect seam, preserving
the current dial position.

Behavior-neutral except the intended drain change. Overlay discovery and
opportunistic transport-neighbor growth remain imperative for now; they
observe the relocated retry schedule and are cut over next. Unit tests
migrated to the driver API with assertions unchanged (1607).
 2026-07-13 05:00:10 +00:00
Johnathan Corgan a0cf593580 node: relocate the peering fields off Node into the Peering owner 
Move retry_pending and pending_connects from Node into the Peering owner
struct (Node.peering), the home introduced with the reconciler core.
Pure mechanical relocation: the two collections now live behind
self.peering, with roughly seventy accessor sites repointed. No decision
logic is touched and the unit-test count is unchanged (1607). The
imperative peering methods still run; the reconciler is wired in the
following commit.
 2026-07-13 04:24:34 +00:00
Johnathan Corgan 5d08d27d3c node/peering: add the sans-IO peering reconciler core (unwired) 
Introduce PeeringReconciler, the synchronous decision core for peer
desired-state, with its input/action vocabulary (Gate, Budget, Observed,
Candidate, DiscoveryPools, Policy, PeeringAction) and the Peering owner
struct. reconcile() computes connect/retry intents across four layers:
the auto-connect mandatory floor, the ceiling-only overlay enqueue, the
opportunistic transport-neighbor growth, and the node.limits ceiling
enforced inline. It mirrors the supervisor sans-IO shape: no I/O, no
clock reads (time enters as a parameter), no runtime handles.

The core owns its own cross-attempt retry schedule so escalating backoff
survives the fresh connection created per re-dial. Gate-guarded reflexes
(on_handshake_timeout / on_link_dead) reproduce the current backoff math
and no-op while draining. Overlay is strictly ceiling-only (no peer-count
set-point). Disconnect is defined but never emitted (no shedding).

Unwired here: the driver still runs the imperative peering methods and
Node holds the live retry map. 11 unit tests drive the pure core with
synthetic inputs. No behavior change.
 2026-07-13 04:10:36 +00:00
Johnathan Corgan b676c9d83a node: rewrap over-width retry_state_iter signature 
Homing the retry schedule under peering/ lengthened the RetryState
module path, pushing the retry_state_iter return-type line past the
100-column max_width. Rewrap to satisfy rustfmt. No behavior change.
 2026-07-13 04:00:41 +00:00
Johnathan Corgan a45eefb58a node: home the peering concept; relocate the connection-retry schedule 
Create src/node/peering/ as the home for the peer desired-state
(homeostatic reconciler) concept and move the cross-attempt connection
retry schedule into it: RetryState plus schedule_retry /
schedule_reconnect / process_pending_retries.

Mechanical relocation only. The methods remain inherent on Node; the
retry_count must persist across re-dials (a fresh connection is created
each attempt), which is why the schedule belongs in the peering home
rather than on a per-connection type. The one-level-deeper module path
requires pub(super) -> pub(in crate::node) to preserve the prior scope,
plus module-path fixups at the reference sites. No behavior change; unit
test count unchanged (1596).
 2026-07-13 03:30:52 +00:00
Johnathan Corgan 79f79cf6ac Merge branch 'refactor-node' into refactor-node-next 2026-07-13 00:50:40 +00:00
Johnathan Corgan d61d189572 node: split Running into Full/Degraded, add Failed health state 
Determine node health at start completion instead of unconditionally
reaching Running. Zero transports up is now Failed (fatal): start()
tears down cleanly and returns an error, and the daemon exits. Any
configured optional child that failed to start - a transport beyond the
first, Nostr, mDNS, TUN, DNS, or a worker pool - leaves the node
Degraded but serving, with an operator warning naming what failed. All
configured children up is Full. A child the node was never asked to run
does not count against health.

The published NodeState gains Degraded and Failed variants, both visible
via control queries; Degraded is operational, Failed is not. The
lifecycle FSM gains the health states plus the PublishState action that
drives them - a health fork cannot be a single direct state write, which
is why the earlier commits deferred it to here.

Runtime child-exit health re-evaluation (a running child dying) is a
separate liveness-monitoring mechanism left for a follow-up; this commit
is start-time health only.
 2026-07-13 00:46:31 +00:00
Johnathan Corgan d6ca632251 node: add bounded graceful-shutdown drain phase 
Add an operator-visible Draining phase on daemon shutdown. On the
shutdown signal the node broadcasts Disconnect to all peers, then keeps
serving for a bounded window - up to node.drain_timeout_secs (default 2s),
exiting early once all peers are gone - before tearing down. This lets
in-flight traffic settle and peers observe the disconnect before the
transports close, rather than the previous immediate teardown.

The lifecycle FSM gains a Draining state plus Drain/DrainDeadlineElapsed
events; the run loop observes the shutdown signal and transitions to
draining in place - one continuous loop, so the channel receivers are
never destructively cancelled. The published NodeState gains a Draining
variant, visible via control queries during the window. The immediate
stop() path used by tests and non-daemon callers is unchanged: it still
tears down immediately with no drain wait.

The reconciler-gate actions the drain emits are no-ops until the peering
reconciler lands and consumes them.
 2026-07-12 23:11:28 +00:00
Johnathan Corgan 6c5fd3f4b0 node: extract lifecycle supervisor FSM, migrate substrate fields (behavior-neutral) 
Introduce a sans-IO lifecycle supervisor: a synchronous step(event) ->
[action] state machine (SupervisorFsm) that authors the substrate-child
spawn and teardown order, plus an owner struct (Supervisor) holding the
substrate runtime fields and embedding the FSM. The substrate-lifecycle
fields leave Node's flat list into the owner - state, packet_tx, the TUN
reader/writer handles + shutdown fd + channels, the DNS task + identity
channel, the Nostr/LAN rendezvous drivers, and the encrypt/decrypt worker
pools; the dataplane keeps packet_rx.

start()/stop() become the driver executing the FSM's SpawnChild/StopChild
actions: same children, same order, same warn/debug-and-continue on
optional failures, same logs, same NodeState transitions. Behavior- and
wire-neutral. The only determinism change is that transports now tear
down in ascending-id order, previously nondeterministic HashMap iteration.

The bounded Draining phase and the Running{Full|Degraded} health split
land as separate follow-on commits.
 2026-07-12 21:03:46 +00:00
Johnathan Corgan de72bf76ea Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the dataplane/session concept-home reorganization onto
the next line. The moved files carry next's (XX-line) content; the one
hand-resolved surface is src/peer/active.rs, where the source-location
citation sits in different surrounding text than on the master line —
resolved by keeping next's body and applying the same node/session.rs
-> node/session/mod.rs citation update.
 2026-07-12 19:24:07 +00:00
Johnathan Corgan 434b9726aa node: establish dataplane/ and session/ concept homes (behavior-neutral) 
Reorganize the node module tree by concept rather than by
message-handling verb, as the first step of the node runtime
decomposition. Pure relocation: no wire, config, metric, or log
semantics change; the lib test count is unchanged (1577 passed).

Moves (git mv, 100% rename similarity):
- handlers/{forwarding,rx_loop,connected_udp,dispatch,encrypted}.rs
  -> node/dataplane/ — the whole RX hot path (the select! run loop,
  transit/local forwarding, the link-message router, the RX decrypt
  path with responder K-bit cutover + roam writes, and connected-UDP
  fast-path activation) now lives in one home.
- node/session.rs -> node/session/mod.rs — establishes the session
  concept home for the data/state types. The message-behavior file
  handlers/session.rs stays put for now (folds in with the later FSP
  session step).

The IK/XX-divergent establishment files (handlers/{handshake,rekey,
timeout}.rs) and the deferred-home files (handlers/{mmp,lookup}.rs)
deliberately stay in handlers/, to move once rather than twice.

Every module is reached through impl Node methods, so no call site or
re-export shim was needed. Updated in lockstep with the moves: the
module_path!-derived tracing targets in the two mesh-lab compose-trace
overlays, a structural test's include_str! source path, doc-comments
in proto/routing and the mesh-lab docs, and the stale source-location
citations (node/handlers/{forwarding,rx_loop,encrypted}.rs and
node/session.rs) in doc-comments and the discovery design doc.
 2026-07-12 19:22:08 +00:00
Johnathan Corgan a7de830b8d Merge branch 'master' into next 
Forward-merge the fipstop routing-stats label flip and the three fixes
carried up from maint: drop the redundant parent_switched counter, keep
the tighter path_mtu on a LookupResponse, and reuse one shared secp256k1
context. Clean auto-merge into next's post-refactor structure.
 2026-07-12 16:55:50 +00:00
Johnathan Corgan 26d70ebb59 Merge branch 'maint' into master 
Forward-merge three maint fixes, hand-relocated into master's
post-sans-IO / discovery-to-lookup structure:
- drop the redundant TreeMetrics parent_switched counter (keep
  parent_switches), reconciled across the refactored tree/mmp sites and
  the spanning-tree test reads
- keep the tighter path_mtu when applying a LookupResponse, now in
  handlers/lookup.rs after the discovery module rename
- reuse one shared secp256k1 context in the identity module
 2026-07-12 16:51:01 +00:00
Johnathan Corgan 6f8590cd92 docs: flag the bloom-filter design doc as describing the retired v1.5 
The code on next is currently v1 after the v1.5 removal during the
sans-IO refactor, but this doc still described the abandoned v1.5 design
as implemented, which is misleading. Add an interim status banner
pointing at v2 as the pending target until the doc is rewritten to v2.
 2026-07-12 15:36:54 +00:00
Johnathan Corgan 9b46b6fa85 fipstop: rename routing-stats "Discovery" labels to "Lookup" 
The routing-stats pane's Discovery Requests/Responses sections show the FMP
overlay coordinate-lookup counters. Rename the section labels and the nested
JSON key they read from "discovery" to "lookup" to match the metric family's
canonical name. The daemon dual-emits both keys, so this reads the current
name and no longer depends on the deprecated "discovery" alias.
 2026-07-12 01:52:54 +00:00
Johnathan Corgan 033957f23d Merge branch 'refactor-transport' into refactor-next 
Forward-merge the transport neighbor-beacon rename (ethernet + BLE
discovery -> neighbor module/buffer rename; ethernet config discovery
flag -> listen with serde alias). Hand-resolved the module files against
next's identity-out-of-beacon beacon rewrite by starting from next's
bodies and re-applying the rename by identifier, so next's beacon logic
(no-arg build_beacon, bool parse_beacon, computed BEACON_SIZE, one-arg
add_peer, deleted BLE add_peer_with_pubkey) is preserved unchanged.
 2026-07-11 23:20:50 +00:00
Johnathan Corgan cbc089b820 transport/ethernet: rename config discovery flag to listen 
Rename the Ethernet per-interface config flag from discovery to listen,
so the receive/transmit toggle pair reads as the symmetric announce
(transmit) / listen (receive) neighbor-beacon vocabulary. The old
discovery: key is still accepted via a serde alias, so deployed configs
load unchanged; to_yaml re-emits it under the canonical listen: name.
Marked deprecated for removal at the v2 cutover.

Updates the config field + accessor, the transport listen_enabled local,
the one struct-literal test consumer, the chaos sim config generator,
packaged fips.yaml examples, and the classified operator-facing docs
(ethernet neighbor-beacon subsystem prose; the generic Transport
discovery capability prose is left unchanged). Adds a compat parse test
asserting the legacy alias, the new key, and that deny_unknown_fields
still rejects unknown keys. Behavior-neutral.
 2026-07-11 23:13:28 +00:00
Johnathan Corgan 4c95be0000 transport/ble: rename discovery module to neighbor 
Mirror the ethernet neighbor rename in the BLE transport: the internal
peer-detection buffer becomes NeighborBuffer (from DiscoveryBuffer) and
the module is renamed discovery -> neighbor. The BlueR/bluez API terms
(DiscoveryFilter, set_discovery_filter) and the BLE advertise/scan
mechanism vocabulary are unchanged, as is BleConfig. Behavior-neutral.
 2026-07-11 23:01:48 +00:00
Johnathan Corgan e362ab67a6 transport/ethernet: rename discovery module to neighbor 
Rename the ethernet link-local neighbor-beacon subsystem from the
overloaded "discovery" vocabulary to a neighbor umbrella. The beacon
frame vocabulary is retained (build_beacon/parse_beacon/BEACON_SIZE/
FRAME_TYPE_BEACON/FRAME_TYPE_DATA); only the subsystem and buffer
identifiers change: DiscoveryBuffer becomes NeighborBuffer,
discovery_buffer becomes neighbor_buffer, and DISCOVERY_VERSION becomes
BEACON_VERSION (wire value 0x01 unchanged). Config toggles are left for
a follow-up commit. Behavior-neutral.
 2026-07-11 22:56:05 +00:00
Johnathan Corgan 3f214c8f63 Merge branch 'refactor-transport' into refactor-next 
Brings the connected-UDP fast-path plane relocation (ConnectedPeerSocket/
PeerRecvDrain into peer/connected_udp; udp keeps open_connected_fd in io.rs)
and the darwin_sockopts -> sockopts_macos rename onto the next line. Clean
three-way auto-merge; behavior-neutral.
 2026-07-11 21:42:25 +00:00
Johnathan Corgan 6c9f55ea80 transport: rename darwin_sockopts to sockopts_macos 
The module is gated target_os="macos" (not the broader Darwin/iOS family),
so the name now tracks the cfg and matches the *_macos.rs file convention
(io_macos.rs). Drops the redundant inner #![cfg(target_os="macos")] (the
decl gate already covers it) and corrects a stale top comment that claimed
the module stays visible on Linux — it is macos-decl-gated, so it never was.
Mechanical rename; no logic change.
 2026-07-11 21:37:00 +00:00
Johnathan Corgan 89a31fd555 transport: move the connected-UDP fast-path handles into the peer module 
ConnectedPeerSocket and PeerRecvDrain are node/peer-side logic: the Transport
trait never touches them, ActivePeer stores them, and node's encrypt worker
drives them. They only happened to live under transport/udp. Relocate the
handle types to a new src/peer/connected_udp/ module (socket.rs + drain.rs),
which the node handler and encrypt worker reach via node -> peer (no new edge;
a node home would have forced a peer -> node cycle).

The udp transport keeps only the kernel-construction seam: open_connected_fd,
now folded into udp/io.rs (the byte-layer home) behind a linux/macos-gated
submodule and re-exported as transport::udp::open_connected_fd. The node
handler builds the fd through it and adopts it via ConnectedPeerSocket::from_fd.

Behavior-neutral: no wire/config/metric/log change; the per-packet hot path
(bare-RawFd send_batch_gso/raw) is untouched. Preserves the Arc multi-owner
contract, the drop-drain-before-socket ordering, and the drain's detach-on-Drop
deadlock avoidance. Reconciles the old cfg(unix)/any(linux,macos) double-gate
onto the single any(linux,macos) predicate.
 2026-07-11 21:30:36 +00:00
Johnathan Corgan ab0a46f2c0 transport: extract udp open_connected_fd syscall body into a standalone fn 
Pulls the connected-UDP socket construction (socket/REUSEADDR/REUSEPORT/
BUFFORCE/bind/connect + darwin tuning) out of ConnectedPeerSocket::open into
a pub(crate) open_connected_fd returning an OwnedFd; open() now delegates and
adopts the fd. Error paths still close the fd via the temporary's Drop; the
success tail transfers ownership through OwnedFd. Behavior identical. Prepares
the fast-path handle types to move to the peer module while the socket
construction stays with the udp transport.
 2026-07-11 21:08:29 +00:00
Johnathan Corgan ccd47bd742 Merge branch 'refactor-transport' into refactor-next 
Brings the canonical per-transport layout normalization (udp/ethernet
byte-layer renamed to io, ethernet addr.rs home, tcp pool.rs extraction)
onto the next line. Clean three-way auto-merge; behavior-neutral.
 2026-07-11 20:32:52 +00:00
Johnathan Corgan e839aead7a transport: extract tcp connection-pool types into pool.rs 
Moves the inline TcpConnection/ConnectingEntry structs, the Direction
enum, and the ConnectionPool/ConnectingPool type aliases out of the
1147-line tcp/mod.rs into a dedicated pool.rs, matching the canonical
per-transport layout. Struct fields are pub(crate) so mod.rs can still
construct and read them across the module boundary. Pure relocation; no
logic change.
 2026-07-11 20:26:42 +00:00
Johnathan Corgan 6d6889d0f6 transport: give ethernet a canonical addr.rs home for MAC parsing 
Moves parse_mac_string into ethernet/addr.rs and re-exports it at the
ethernet module root so the ethernet::parse_mac_string path stays
byte-identical for its one external consumer (zero churn there). Its
unit tests stay in mod.rs, calling through the re-export. Pure
relocation; no logic change.
 2026-07-11 20:19:08 +00:00
Johnathan Corgan 196d9492da transport: rename ethernet byte-layer module from socket to io 
Normalizes the ethernet transport onto the canonical byte-layer name
(io.rs), including the per-OS files io_linux.rs/io_macos.rs and their
#[path] wiring. Pure file rename plus module-path and doc-comment
updates; no logic, wire, config, metric, or log change.
 2026-07-11 20:14:37 +00:00
Johnathan Corgan 0f2e91b479 transport: rename udp byte-layer module from socket to io 
Normalizes the udp transport onto the canonical per-transport module
layout (ble is the reference: io/pool/addr/stats). Pure file rename plus
module-path updates at the use sites; no logic, wire, config, metric, or
log change. Behavior identical.
 2026-07-11 20:10:19 +00:00
Johnathan Corgan 775c7ae4c0 Merge branch 'refactor-transport' into refactor-next 2026-07-11 18:46:00 +00:00
Johnathan Corgan 32475d859e transport: consolidate tor and nym SOCKS5 dialing into a shared module 
The Nym transport was a near-clone of the Tor transport's outbound
SOCKS5 path. Extract the shared logic into src/transport/socks5/ so both
transports drive one implementation instead of two maintained copies:

- share one SOCKS5 mock server between the tor and nym tests
- extract the common send/receive/connect counters into a shared
  ProxiedStatsBase (snapshot structs and emitted metrics unchanged)
- add a shared Socks5Dialer that collapses all six connect variants;
  the sole dialing difference (tor's per-destination circuit-isolation
  auth vs nym's no-auth) is an enum on the dialer
- route both tor and nym dialing through the shared dialer
- share the proxied connection pool, generic over a per-connection meta
  type that carries tor's inbound/outbound direction counting (nym uses
  the unit type)
- share the proxied receive loop

Behavior-neutral: no wire-format, config-key, emitted-metric, log, or
error-variant change. Tor keeps its control-port, inbound/onion, and
directory surface and its own address validation.
 2026-07-11 18:45:41 +00:00
Johnathan Corgan 14edddc14a Merge branch 'master' into next 2026-07-11 05:05:22 +00:00
Johnathan Corgan f2e6b8befb transport: promote FMP stream framing to transport::framing 
The FMP frame-boundary reader lived in transport/tcp/stream.rs, but the
Tor and Nym transports both reached across module boundaries to
'use crate::transport::tcp::stream::read_fmp_packet', a layering smell:
the reader is a shared stream-framing utility, not a TCP-private one.
Move the file to transport/framing.rs and repoint the tcp/tor/nym use
sites, removing the tor->tcp and nym->tcp dependencies. Behavior
unchanged.
 2026-07-11 04:32:38 +00:00
Johnathan Corgan 1aacdfa086 transport: share connection-pool counters via PoolCounters 
TcpStats and TorStats each carried an identical pair of pool_inbound/
pool_outbound atomics plus the same five record_pool_* / pool_inbound_count
methods over them. Move the counter logic into a shared PoolCounters
struct (new transport/stats_common.rs) embedded as a 'pool' field in both.
The public record_pool_* methods stay as thin delegators so all call
sites and the flat pool_inbound/pool_outbound snapshot fields are
unchanged; only the duplicated atomic bookkeeping is now single-sourced.
 2026-07-11 04:30:19 +00:00
Johnathan Corgan a7dfe47663 transport: derive Serialize for EthernetStatsSnapshot 
EthernetStatsSnapshot derived only Clone/Debug/Default, unlike every
other transport's *StatsSnapshot which also derives Serialize. That gap
forced a hand-rolled serde_json::json!{} arm in TransportHandle::
transport_stats() that re-listed all ten fields by hand. Add the
Serialize derive and collapse the arm to the same one-line
serde_json::to_value(...) form the other transports use. The emitted
JSON keys and values are unchanged.
 2026-07-11 04:27:50 +00:00
Johnathan Corgan 81baeebf22 Merge branch 'master' into next 2026-07-11 02:15:24 +00:00
Johnathan Corgan 8aab71af86 bench: add routing next-hop microbench 
Measures the per-forwarded-packet cost of routing candidate assembly
(routing_candidates over a synthetic RoutingView) against a zero-alloc
reference across 8/32/128/256 peers, with per-call allocation counts via
a counting allocator. Criterion harness; no production code change.
 2026-07-11 01:48:34 +00:00
Johnathan Corgan 2cffc10520 Merge branch 'maint' (bloom single-digest double-hashing) 
Forward-merges the bloom SHA-256-once fix from maint. The bloom filter was
relocated to proto/bloom/ by the sans-IO refactor, so the fix applied cleanly
to proto/bloom/core.rs; the behavior-neutral test was re-homed into
proto/bloom/tests/core.rs (maint carried it in the pre-split bloom/tests.rs).
 2026-07-11 01:32:48 +00:00
Johnathan Corgan b9eb725e5a Merge FMP wire + PromotionResult relocations into next 
Forward-merge the master-line FMP-establish relocations onto next: the FMP
link wire codec into proto/fmp/wire.rs and PromotionResult into
proto/fmp/core.rs. The wire move was hand-resolved against next's wire
format (msg3/Msg3Header/build_msg3, FMP_VERSION=1, no spin-bit flag), so
next's behavior is unchanged. Behavior-neutral; full lib suite green at
next baseline.
 2026-07-10 17:34:49 +00:00
Johnathan Corgan 1c1ed0d939 Relocate PromotionResult into proto/fmp 
Move the PromotionResult enum (and its impl) out of peer::mod and into
proto/fmp/core.rs, alongside the cross_connection_winner tie-break helper
that was relocated the same way. This is FMP connection-lifecycle result
vocabulary, so it belongs in the FMP subsystem home rather than the peer
module.

Behavior-neutral pure type relocation: consumers import it from
crate::proto::fmp, and the crate-root public path crate::PromotionResult
is preserved via a re-export in lib.rs (mirroring cross_connection_winner).
Full lib suite green at baseline.
 2026-07-10 16:52:18 +00:00
Johnathan Corgan 1c41f73931 Relocate FMP link wire codec into proto/fmp 
Move the FMP mesh-layer wire format (common prefix, encrypted/msg1/msg2
headers, and the build_*/inner-header codec fns) out of node/wire.rs and
into proto/fmp/wire.rs, so the whole FMP wire surface lives with its
subsystem, matching the proto/fsp/wire.rs layout. The wire module becomes
pub(crate) mod wire; callers reach it via crate::proto::fmp::wire.

Behavior-neutral: pure relocation plus import-path rewrites across the
node/peer consumers; no logic change. Full lib suite green at baseline.
 2026-07-10 16:42:17 +00:00
Johnathan Corgan f1ff410fd2 Merge the nostr sans-IO state-machine refactor into the next line 
Forward-merge the AdvertMachine / TraversalMachine / classify_punch_packet
sans-IO extraction. Clean auto-merge; the next-line XX-handshake runtime.rs
deltas sit outside the traversal/replay/election sites.
 2026-07-10 03:28:20 +00:00
Johnathan Corgan 39ad4d2e67 Extract the nostr rendezvous decision logic into sans-IO state machines 
Carve the nostr rendezvous engine's decision logic out of the async
NostrRendezvous driver into synchronous, clock-injected cores, following
the failure_state.rs pattern: sync state behind std::sync::Mutex, time
passed in as now_ms, no .await and no I/O in the core. The driver performs
all relay/socket I/O and executes the returned plans.

- AdvertMachine (src/nostr/advert.rs): advert publish/cache/fetch/prune
  decision logic; the driver executes a returned PublishPlan.
- TraversalMachine (src/nostr/traversal_machine.rs): the engine-scoped
  cross-session decision state -- in-flight-initiator dedup, the dual-init
  responder suppression election, and the replay/seen-session cache.
- classify_punch_packet (src/nostr/traversal.rs): a pure classifier for
  the punch recv loop's packet branch table.

The driver keeps all I/O, the NAT punch send cadence, the offer-slot
admission semaphore, and the pending-answer oneshot routing. Behavior-
neutral: no wire, config-key, or metric change. Adds unit coverage for
the advert plans, election ordering, replay eviction, initiator dedup,
and punch classification.
 2026-07-10 03:28:11 +00:00
Johnathan Corgan 6954ebbf34 Merge the nostr rendezvous reorg into the next line 
Forward-merge the src/nostr / src/mdns rendezvous reorganization
(relocation out of src/discovery/, the Discovery->Rendezvous rename, the
RendezvousDriver consolidation, and the trace-target reference updates)
onto the next branch. The sole conflict was in the LAN poll method's
doc-comment and signature: kept next's XX-handshake wording and applied
the lan_rendezvous rename. Merged tree builds clean; next lib suite 1583
passing, fmt/clippy clean.
 2026-07-09 23:48:53 +00:00
Johnathan Corgan 4d2504f59d Update trace-target references for the relocated rendezvous modules 
Moving the nostr rendezvous engine to src/nostr/ (and mDNS to src/mdns/)
changed the module-path-derived tracing targets from
fips::discovery::nostr::* to fips::nostr::*. Update the RUST_LOG filters
in the NAT and mesh-lab test compose files and the resolve-peers-via-nostr
tutorial's debug recipe to the new targets so trace configs and the
documented journal-watch command keep emitting the intended lines.

Without this, the stun-faults suite's Phase-0 pre-flight (which greps the
daemon journal for the debug-level "STUN observation succeeded" /
"traversal: initiator STUN observed" lines) saw those lines suppressed —
the daemon behaved correctly, but the stale RUST_LOG target hid the
evidence. Test-harness and docs only; no source or behavior change.
 2026-07-09 23:20:28 +00:00
Johnathan Corgan 1208f6a5c2 Consolidate the nostr rendezvous driver into src/nostr 
Pull the rendezvous driver state and the movable driver logic out of the
Node struct into the src/nostr home. A new RendezvousDriver owns the
engine handle and the four bookkeeping fields (traversal start time,
startup-sweep latch, adopted bootstrap-transport set and their npubs)
that previously sat loose on Node; Node holds a single driver field and
reaches the same data through thin accessors, including the rx-loop
hot-path protocol-mismatch hook.

The advert build/refresh, the via_nostr fallback-address resolve, the
overlay-endpoint-to-PeerAddress mapping, and the bootstrap request move
onto the driver, taking their Node inputs explicitly (a transport-
endpoint snapshot for advert building) rather than reading Node fields
directly. Transport/connection-table-bound work (traversal adoption,
bootstrap-transport cleanup, the open-discovery sweep, and the outbound
budget calculators) stays on Node as thin glue that calls into the
driver.

Behavior-neutral relocation: statements moved verbatim, no logic, wire,
config-key, or metric changes. lifecycle.rs shrinks ~230 lines. cargo
fmt/build/clippy clean; lib suite 1547 passing (baseline unchanged).
 2026-07-09 22:13:56 +00:00
Johnathan Corgan 3f80530cc5 Move nostr peer rendezvous and mDNS into dedicated module homes 
Relocate the overlay peer-rendezvous subsystem out of the overloaded
src/discovery/ tree into two focused, independent homes: src/nostr/
(relay-mediated overlay endpoint advertise/resolve/auto-mesh plus NAT
traversal) and src/mdns/ (link-local DNS-SD rendezvous). The two
subsystems are independent, so they get separate homes rather than
sharing one.

Drop the ambiguous "Discovery" stem from their identifiers in favor of
"Rendezvous": NostrDiscovery -> NostrRendezvous, LanDiscovery ->
LanRendezvous, and the matching config, policy, field, and method names.
The former src/discovery.rs handoff types (EstablishedTraversal,
BootstrapHandoffResult, the punch-packet helpers) fold into
src/nostr/handoff and stay reachable via the crate-root re-exports.

Pure relocation and rename: no logic, wire-format, config-key, metric,
or tracing-target changes. The operator-facing node.rendezvous.nostr.*
and node.rendezvous.lan.* config keys and the fips-overlay-v1 advert
namespace are byte-identical. cargo fmt/build/clippy clean; lib test
suite 1547 passing (baseline unchanged).
 2026-07-09 21:51:54 +00:00
Johnathan Corgan d2331ecb78 Merge refactor-sans-io: disambiguate the "discovery" name 
Forward-merge the mesh-lookup "discovery" disambiguation onto the next
line: the lookup handler module and Node.lookup field renames, the lookup
metric types, the dual-emitted lookup/discovery metric family, and the
node.discovery -> node.lookup/node.rendezvous config split with its
deprecation shim.

Conflicts resolved by keeping next's XX handshake path (the inbound
responder stores pending_inbound for msg3 rather than promoting on msg1)
and renaming its reset_discovery_backoff call sites to reset_lookup_backoff;
the CHANGELOG Unreleased entries from both lines are combined.
 2026-07-09 15:43:57 +00:00
Johnathan Corgan 3b401a0cbd Disambiguate the mesh-lookup "discovery" name across the source tree 
The identifier "discovery" named three unrelated subsystems; the FMP
overlay coordinate-lookup subsystem is now consistently "lookup". This
finishes the concept-#1 rename across the shell, config, and metric
layers left after the earlier proto-layer rename:

- Handler module node::handlers::discovery -> node::handlers::lookup, and
  reset_discovery_backoff -> reset_lookup_backoff.
- The Node lookup-engine field Node.discovery -> Node.lookup, renamed by
  resolved binding so the metrics().discovery and node.discovery config
  paths are left untouched.
- The lookup metric types DiscoveryMetrics -> LookupMetrics,
  DiscoveryStatsSnapshot -> LookupStatsSnapshot, and Metrics.discovery ->
  Metrics.lookup.

Two surfaces cross a stability boundary and ship behind a compatibility
window, both marked in-code for removal at the v2 cutover:

- The control-socket metric family is dual-emitted under both "discovery"
  (deprecated alias) and "lookup" so existing dashboards keep working.
- The node.discovery.* config table is split into node.lookup.* (mesh
  lookup scalars) and node.rendezvous.* (nostr/lan peer rendezvous).
  NodeConfig does not deny unknown fields, so a naive rename would make a
  deployed node.discovery: block deserialize into nothing and silently
  revert every setting to default. A deprecated all-Option
  DiscoveryConfigCompat field captures a legacy block and a new post-parse
  Config::normalize_deprecated_keys pass folds it into the new tables with
  a one-time deprecation warning.

Flip the packaged fips.yaml templates to the new keys, add legacy/new/
scalar compat parse tests, and record the split and deprecations in the
CHANGELOG. Behavior-neutral; fmt/clippy clean, lib suite green.
 2026-07-09 15:43:31 +00:00
Johnathan Corgan 1ec7a4c680 Merge refactor-sans-io: relative-tolerance powi test drift into the next line 2026-07-09 01:30:28 +00:00
Johnathan Corgan 0b2212e1e8 proto: bound powi test drift by relative tolerance, not ULP 
The previous portability fix bounded the powi test's drift from std::powi
at an absolute ULP count, but Windows/MSVC drift grows with the exponent
(3 ULP by exp=14), so no fixed ULP bound is portable.

Replace the ULP oracle with a loose relative-tolerance sanity sweep
(1e-11), which any sane libm clears by ~1000x regardless of exponent
while still catching a grossly wrong impl. Add a golden-bit pin for the
exp=14 case that exposed the growth. The powi function is unchanged; the
golden-bit determinism pins already passed on Windows.
 2026-07-09 01:30:18 +00:00
Johnathan Corgan 0c948bd707 Merge refactor-sans-io: portable powi guard test into the next line 2026-07-09 01:20:00 +00:00
Johnathan Corgan b2ce7cd3c8 proto: make the powi guard test portable across platforms 
The powi guard test asserted our square-and-multiply result was
bit-for-bit equal to f64::powi, which failed the Windows unit-test job by
one ULP. f64::powi is not portably bit-stable: Linux and macOS lower it
to compiler-rt's __powidf2, but Windows/MSVC rounds differently.

Our square-and-multiply is pure IEEE-754 f64 multiplication and is
therefore deterministic across every platform, which is the property that
actually matters for mesh nodes to agree on bloom FPR and backoff timing
regardless of OS. Replace the std bit-equality assertion with golden-bit
pins on our own output (locking that cross-platform determinism) plus a
bounded-drift sanity sweep against std::powi. The powi function itself is
unchanged.
 2026-07-09 01:19:49 +00:00
Johnathan Corgan c5d48e5ef4 Merge refactor-sans-io: rename proto/discovery to proto/lookup into the next line 
Forward-merge the mesh discovery module rename (directory, exported types, and
internal lookup-stem terminology) onto the next wire-format line. next's
divergent lookup wire and core content is preserved under the new proto/lookup
path, with the same terminology sweep applied to its next-only lines.
 2026-07-08 21:54:24 +00:00
Johnathan Corgan e3e03f6a5d proto: rename the mesh discovery subsystem module from discovery to lookup 
Rename src/proto/discovery to src/proto/lookup and bring the module's naming
onto the lookup stem, matching its concept: a mesh lookup of a node's
coordinates from its pubkey, sent into the mesh as a bloom-filter-guided
multicast request that returns a unicast response with the coordinates.

- the module directory and its declaration (proto::discovery -> proto::lookup)
- exported types: DiscoveryAction -> LookupAction, DiscoveryBackoff ->
  LookupBackoff, DiscoveryForwardRateLimiter -> LookupForwardRateLimiter,
  MAX_RECENT_DISCOVERY_REQUESTS -> MAX_RECENT_LOOKUP_REQUESTS, and the Discovery
  state struct -> Lookup
- internal terminology: doc comments, the "overlay-lookup" phrasing, the
  empty_discovery/suppressing_discovery test helpers, and the disc
  parameter/variable all take the lookup names
- the already-lookup-named wire types (LookupRequest/LookupResponse) are unchanged

Behavior-neutral: no wire bytes or decision logic change. Two references are
intentionally kept as "discovery": the still-named node::handlers::discovery
shell module and the node.discovery.* config keys, which belong to the broader
disambiguation of the shell, config, and metric surfaces still to come.
 2026-07-08 21:54:24 +00:00
Johnathan Corgan 2b009196b5 proto: no_std hygiene for the fmt/alloc imports and the filter math 
Bring the proto tree closer to a std+alloc-only posture, behavior unchanged on
every decision path:

- Sweep the remaining std::fmt and std::collections imports over to core::fmt
  and alloc::collections across the wire codecs and their tests. Subsystem files
  that shadow the core name with a child core module use the leading-colon
  ::core::fmt form. Imports only.

- Replace the two f64::powi calls (bloom false-positive-rate, FMP backoff timer)
  with a shared core-only square-and-multiply helper in proto/math, bit-identical
  to std::powi (a guard test pins this against every exponent the codecs reach),
  and route the diagnostic estimated-count natural log through libm::log. The FPR
  reject decision and the backoff timer are bit-for-bit unchanged; only the
  debug-only count estimate may differ by at most one ULP.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan 5d13090d8f proto: add a bounds-checked byte reader/writer and adopt it across the wire codecs 
Introduce a shared proto/codec module with a cursor Reader (short reads fail with
MessageTooShort { expected: position + needed, got: total }) and an append Writer,
and adopt them across the seven subsystem wire codecs, replacing the repetitive
manual slicing, try_into, and from_le_bytes extraction. Each existing length check
maps to the reader (an up-front minimum becomes require at position zero, so the
per-field expected values, including the tree-announce expected 99, are reproduced
exactly); the bloom exact-length check stays explicit since it also rejects
over-long payloads. Encoded bytes and decode decisions are unchanged.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan 6538731176 proto: replace the string Malformed error with typed variants and drop thiserror 
Replace Malformed(String) with a no_std-clean set: Malformed(&static str)
for the static decode diagnostics, BadSizeClass { got, max } for the two
bloom size-class checks, and typed sources BadCoord(CoordError) /
BadBloom(BloomError) for the coordinate and bloom construction failures. Add a
dedicated CoordError so proto/coord no longer depends upward on stp TreeError,
resolving the temporary inversion from the coordinate relocation. Drop the
thiserror derive from the four proto error types (Error, TreeError, BloomError,
CoordError) in favor of hand-rolled core::fmt::Display and core::error::Error
impls. thiserror remains in use elsewhere in the crate. Wire decode decisions
are unchanged; only the error type and its diagnostic text change.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan 9697026c81 proto: share a per-address rate limiter and backoff helper across subsystems 
Hoist the two line-for-line-identical per-destination minimum-interval limiters
(discovery forward, routing error) into a shared PerAddrRateLimiter, and fold
the duplicated exponential-backoff math (discovery originator, FMP retry) into a
shared backoff_ms helper, both in a new proto/rate_limit module. The subsystem
limiters become thin delegating newtypes so should_forward/should_send and the
backoff call conventions are preserved. Behavior is byte-identical.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan a2400d823f proto/stp: extract ParentDeclaration and relocate the coordinate type to a shared proto/coord module 
Two behavior-neutral relocations, wire bytes unchanged:

- Move the ParentDeclaration type and its impls out of state.rs into a dedicated
  declaration.rs, re-exported at the same path, and relocate the inline wire.rs
  test module into tests/wire.rs alongside the other stp unit tests (reusing the
  shared test helpers). wire.rs drops to non-test code only.

- Move TreeCoordinate, CoordEntry, and the coordinate wire codec out of
  proto/stp into a shared proto/coord module (a peer of proto/link), re-exported
  from proto::stp so every existing import path keeps resolving unchanged.
  Coordinate is a shared addressing primitive with many non-stp consumers, so it
  no longer belongs under the spanning-tree subsystem. The codec carries a
  documented temporary dependency on stp::TreeError until a dedicated CoordError
  is introduced.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan dc9334e725 proto/stp: hoist the parent flap/hold-down veto to the shell for a clock-free classify core 
evaluate_parent no longer reads the injected clock: it returns a ParentEval of
Mandatory, Discretionary, or None, and the flap/hold-down veto is applied at the
edge via a new TreeState::is_switch_suppressed(now_ms), gating only the
discretionary arm. classify_announce/classify_periodic take the pre-computed
switch_suppressed bool instead of now_ms, so the whole classify ladder is
clock-free; the shell callers (tree announce/periodic re-eval, MMP first-RTT
re-eval, handle_parent_lost) compute the veto verdict. The no-coords parent
case stays discretionary (veto-gated) exactly as before. Behavior unchanged;
the veto tests now assert the moved responsibility.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan 309a91d293 proto/mmp: split state into per-role modules and dissolve the vestigial mmp shell 
Reorganize the MMP subsystem, behavior unchanged throughout:

- Restore the pre-migration role split: move SenderState into sender.rs,
  ReceiverState (with its GapTracker helper) into receiver.rs, MmpMetrics and
  RrLog into metrics.rs, and PathMtuState into path_mtu.rs, leaving the Mmp
  aggregate plus the peer/session state in state.rs (1339 to 196 lines). Home
  the module constants in a new limits.rs and re-export them at the same paths.
  Pure code-motion with module rewiring; visibility unchanged.

- Dissolve the 97-line src/mmp shell, which held only MmpConfig and the
  monotonic mono_ms() clock: move MmpConfig into config/node.rs (re-exported as
  crate::config::MmpConfig), move mono_ms() into a new top-level src/time.rs
  shell time seam, repoint all callers, and delete src/mmp/. Also correct stale
  src/mmp/ doc-comment path labels to proto/mmp/ where they name the protocol
  primitives.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan c1ddbf053c proto/discovery: home the recent-request cap and inject request_id from the shell 
Two behavior-neutral discovery cleanups:

- Move MAX_RECENT_DISCOVERY_REQUESTS out of the node discovery handler into the
  discovery subsystem limits module and re-export it, keeping the two use sites
  unchanged. Same value and semantics; only its home moves.

- Remove the ambient rand draw from LookupRequest by deleting generate() and
  having the shell draw the random request_id and pass it into the existing
  new() constructor. Same per-request u64 draw, now at the shell, leaving the
  discovery codec free of ambient RNG reads.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan b53db662c3 proto/bloom: add BloomFilter::fpr() and use it for the false-positive-rate 
The false-positive-rate primitive (fill ratio raised to the hash count) was
inlined at three sites. Hoist it into a BloomFilter::fpr() method and call it
from all three; the threshold/policy logic stays at the call sites. Behavior
and result bytes are unchanged.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan c0b2822d56 Merge refactor-sans-io: sans-IO proto cleanup into the next line 
Forward-merge the sans-IO cleanup series (bloom fpr, discovery const/RNG-injection,
mmp state decomposition, mmp shell dissolution, STP clock-free classify core, STP
declaration split, shared proto/coord relocation, shared rate limiter/backoff,
typed proto errors dropping thiserror, shared bounds-checked codec reader/writer,
core/alloc import sweep, no_std-shaped filter math) onto the next wire-format line.

Reconciled master structure against the next wire semantics: the mmp role-module
split adopts next slim report format (spin-bit stays dropped, sender/receiver
build next reports), the discovery and fmp codecs keep next TLV and profile
negotiation while moving to the typed error, and the parent-eval handler keeps the
Full/Leaf profile filter under the new ParentEval/is_switch_suppressed seam.
 2026-07-08 17:52:24 +00:00
Johnathan Corgan fff1d6285a Merge refactor-sans-io: FSP sans-IO extraction and protocol teardown 
Forward-merge the FSP session-protocol sans-IO migration and the src/protocol
teardown from the master-refactor line onto next. Converge proto/fsp across
both lines (core.rs byte-identical), keeping next's born-on-next residue: the
additive SessionSetup/SessionAck wire variants, the spin-bit-less unit
FspInnerFlags, and the XX handshake shell residue (negotiation-payload
piggyback, initiator identity check, min-size length checks) in
handlers/session.rs. src/protocol is removed on next as well; ProtocolError,
LinkMessageType, and SessionDatagram reach their proto/ homes.
 2026-07-08 10:18:18 +00:00
Johnathan Corgan 4ad5940114 proto/fsp: extract FSP session protocol into sans-IO layout, retire src/protocol 
Migrate the FSP end-to-end session subsystem into src/proto/fsp/ following the
established sans-IO shape, and retire the src/protocol grab-bag now that FSP was
its last occupant.

Relocate the FSP session wire (node/session_wire.rs plus the FSP message types
from protocol/session.rs) into proto/fsp/wire.rs. Hoist the pure decision logic
into proto/fsp/core.rs over plain-data SessionSnapshots returning an ordered
FspAction list the shell drives: session-rekey policy, msg3-resend
classification, post-decrypt epoch reaction, setup/dual-init tie-break,
coords/path-MTU emit-policy, bounded pending-queue, and IPv6 ECN. The
crypto-owning SessionEntry stays shell-side in node/session.rs (matching the FMP
ActivePeer pattern); proto/fsp is wire + core + limits only, with no proto->noise
dependency and no crypto.

Move the coords helpers to proto/stp/ (they serialize TreeCoordinate), and split
SessionMessageType: the encrypted-inner 0x10-0x1F variants stay in proto/fsp/wire.rs
while the 0x20-0x2F routing signals become a new RoutingSignalType in
proto/routing/wire.rs. Migrate the session-MMP shell adapter, which continues to
drive proto/mmp/.

Retire src/protocol: LinkMessageType and SessionDatagram move to a new shared
proto/link.rs, ProtocolError becomes proto::Error (relocated verbatim), the
deprecated MessageType alias and the unimported PROTOCOL_VERSION are dropped, and
src/protocol/ is deleted along with its lib.rs module declaration.

Behavior-neutral: wire bytes unchanged, oracle tests pass unedited except
mod-path relocation; adds rekey/epoch characterization tests and pure
poll/emit-policy core tests.
 2026-07-08 04:57:00 +00:00
Johnathan Corgan 45f65e7ca1 Merge refactor-sans-io: converge next bloom onto v1-sans-IO 
Forward-merge the master-line bloom relocation into the next line,
discarding next's v1.5 bloom draft (RLE codec, XOR-diff delta,
FilterNack/0x21, adaptive sizing) and converging both lines onto the
identical proto/bloom v1-sans-IO module. This is a deliberate,
temporary wire regression on the next line: the v2 bloom is rebuilt
fresh, sans-IO from day one, on both lines later. Nothing outside the
bloom feature depended on the v1.5-specific surface. proto/bloom is now
byte-identical across both integration lines.
 2026-07-07 23:22:22 +00:00
Johnathan Corgan 4ed674ea8b proto/bloom: relocate bloom filter into sans-IO layout 
Migrate the v1 bloom filter subsystem into src/proto/bloom/, matching the
discovery/routing/fmp/mmp/stp reference layout and completing the proto/
relocation series for the data/wire subsystems.

- wire.rs: the FilterAnnounce (0x20) codec, moved from protocol/filter.rs
- core.rs: the pure BloomFilter algorithm (hash/insert/contains/merge/
  as_bytes/from_bytes/estimated_count), moved from bloom/filter.rs
- state.rs: BloomState (per-peer inbound store, compute_outgoing_filter,
  the injected-clock send debounce), moved from bloom/state.rs
- limits.rs: the v1 sizing constants
- mod.rs: module wiring + the BloomError enum
- tests/: the unit suite split by target (core/state/wire), no inline tests

no_std+alloc hygiene: core::fmt over std::fmt, the tracing dependency
dropped from the pure filter, and std collections replaced with
BTreeMap/BTreeSet (NodeAddr: Ord) for deterministic iteration. The pure
filter combination stays a BloomState method; the two irreducible shell
gathers (peer_inbound_filters, build_filter_announce) remain in the async
shell. Wire bytes and observable behavior are unchanged; full local CI
green (36/36) including the bloom-storm chaos gate.
 2026-07-07 23:10:12 +00:00
Johnathan Corgan 6a028064ef Merge refactor-sans-io: STP sans-IO spanning-tree on the next line 
Forward-merge the STP sans-IO migration. The classify / proto-stp structure,
clock injection, crypto field-partition, and BTree collections come from the
master line; next's non-full/leaf parent-candidacy skip is preserved by passing
self.non_full_peers() at the four shell call sites (handle_tree_announce,
check_periodic_parent_reeval, the MMP first-RTT re-eval, and the greedy-tree
find_next_hop fallback) instead of an empty set, with non_full_peers returning a
BTreeSet. next's four non_full_peers skip tests fold into proto/stp/tests.
 2026-07-07 17:22:25 +00:00
Johnathan Corgan a67801099d proto/stp: sans-IO spanning-tree state machine 
Migrate the full non-async spanning-tree surface into proto/stp/, mirroring the
discovery/routing/fmp/mmp conversions. The classification ladder (parent-switch /
self-root / loop-drop / ancestry-update / periodic-rebroadcast / parent-lost) moves
out of the async node handlers into a pure Stp classify layer returning a
TreeDecision the shell drives, with effect ordering and per-arm invalidation
preserved verbatim. src/tree/ relocates wholesale: TreeState + ParentDeclaration data
+ coordinates into proto/stp/{state,coordinate}, the flap-dampening / hold-down
cluster into a FlapDampener in limits.rs, and the wire codec into wire.rs. The clock
is injected as u64 (wall-clock secs for the escaping declaration timestamp, monotonic
ms for the dampening timers via mmp::mono_ms); declaration crypto is field-partitioned
so sign/verify/hash run in the shell while the in-core modules carry data +
signing_bytes only. Peer maps/sets move to BTree; core/state/coordinate/limits are
core+alloc clean, with wire.rs the one std-tethered file. Behavior-neutral:
characterization tests added for the handler decision arms; convergence suite and
ci-local (36/36) green.
 2026-07-07 17:07:33 +00:00
Johnathan Corgan 584edfac20 Merge refactor-sans-io: MMP sans-IO reporting on the next line 2026-07-07 11:30:40 +00:00
Johnathan Corgan 50a595a0ed proto/mmp: sans-IO metrics-reporting state machine 2026-07-07 09:24:17 +00:00
Johnathan Corgan 3b46bbe17d Merge refactor-sans-io: FMP sans-IO connection-lifecycle on the next line 2026-07-07 07:32:02 +00:00
Johnathan Corgan 4802792e38 proto/fmp: sans-IO connection-lifecycle state machine 2026-07-07 06:20:02 +00:00
Johnathan Corgan 78eadbcd65 Merge refactor-sans-io: routing sans-IO on the next line 2026-07-07 04:29:58 +00:00
Johnathan Corgan 9ea57b483a proto/routing: sans-IO transit + hop-selection state machine 2026-07-07 04:12:05 +00:00
Johnathan Corgan 74ab6d58ba Merge refactor-sans-io: reconcile discovery sans-IO core with the FMP v2 delta 
Forward-merge the master-side sans-IO refactor (discovery migration + no_std
reductions, collapsed to one commit) into the next-side branch. The discovery
decision core meets next's FMP profile/LookupRequest delta here: next's four
transit-forward predicates (Leaf no-forward, Full-profile, min_mtu, tree/fallback)
are folded into the pure core planners via an extended RoutingView seam
(node_is_leaf / peer_is_full / peer_meets_mtu, new ForwardOutcome::LeafNoForward),
and next's v2 LookupRequest API delta (origin_coords removed, tlv_entries added)
is reconciled across the discovery test tree, with next's four TLV wire tests
ported into the relocated wire test module.

The master-side branch now also carries the no_std+alloc reductions (BTreeMap,
alloc::sync::Arc, backoff-reset log moved to the shell, extern crate alloc),
which come through cleanly on the pilot-only core files.

Validated: cargo fmt / clippy (-D warnings) / test --lib all green, including
next's own transit MTU-pruning integration tests driving the refactored core
through the full shell path.
 2026-07-05 22:01:06 +00:00
Johnathan Corgan e03b206f62 proto/discovery: sans-IO state-machine migration + no_std reductions 
Migrate the FMP discovery decision logic out of the async handlers into
synchronous, runtime-agnostic sans-IO state machines owned by the protocol
structs, with I/O pushed to the edges. Pulls the full decision surface into a
pure core (backoff, rate-limit, planners, response routing), consolidates the
tests into a per-module tree with a shared crate testutil, and injects a u64
wall-clock so the core is free of Instant and std time.

Also brings the module toward no_std+alloc: the four discovery maps use
alloc::collections::BTreeMap (HashMap's RandomState is std-only), Arc is spelled
alloc::sync::Arc, the backoff-reset log lives in the shell (the core returns the
cleared count so observability stays out of the pure core), and the crate root
names alloc directly. The one remaining tether is ProtocolError's
std::error::Error coupling in the wire codec.

First subsystem of the broader sans-IO refactor; establishes the extraction
patterns and conventions carried forward to the remaining protocols.
 2026-07-05 21:59:21 +00:00
Johnathan Corgan bfc1cd780f Merge branch 'master' into next 2026-07-02 22:03:52 +00:00
Johnathan Corgan 1dbfefc9d0 Merge branch 'maint' 2026-07-02 22:03:41 +00:00
Johnathan Corgan ceb60c9976 Merge branch 'master' into next 2026-06-29 16:47:43 +00:00
Johnathan Corgan 793f844448 Merge branch 'maint' 2026-06-29 16:47:43 +00:00
Johnathan Corgan f7a0f906fa Merge branch 'master' into next 2026-06-29 14:23:57 +00:00
Johnathan Corgan 243bd7985a Merge branch 'maint' 2026-06-29 14:23:57 +00:00
Johnathan Corgan 56bdea153b Merge master into next after the v0.4.0 rollover 
Brings the v0.4.0 release line up to date on next without disturbing
the in-flight wire-format work; next keeps its own 0.6.0-dev version.
Linkage merge so later forward-merges land cleanly.
 2026-06-27 18:35:51 +00:00
Johnathan Corgan 30c5808e09 Merge maint into master after the v0.4.0 rollover 
maint carries only its 0.4.1-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
 2026-06-27 18:35:41 +00:00
Johnathan Corgan 330c6aa9af Open 0.6.0-dev cycle on next after v0.4.0 release 
Bump the version to 0.6.0-dev and update the status badge, and correct
the 0.4.0 changelog heading to the actual release date.
 2026-06-27 18:30:26 +00:00
Johnathan Corgan 3c9a629ad4 Open 0.5.0-dev cycle on master after v0.4.0 release 
Bump the version to 0.5.0-dev and update the status badge.
 2026-06-27 18:28:52 +00:00
Johnathan Corgan c8c2ae96eb Merge branch 'master' into next (openwrt .apk WAN-port default for DSA boards) 2026-06-26 03:42:23 +00:00
Johnathan Corgan 5bc42b4deb Merge branch 'master' into next (OpenWrt Blossom upload resilience) 2026-06-21 14:40:50 +00:00
Johnathan Corgan e305e1c4e4 Merge branch 'master' into next (v0.4.0 release content) 
Forward-merge the v0.4.0 content finalize from master. The only master
commit merged touches the CHANGELOG, release notes, and README badge.

CHANGELOG reconciliation: adopt the regrouped, stamped [0.4.0] release
section verbatim from master (a frozen release record, identical across
branches), keep next's top-of-file Breaking block, and retain next's
XX-handshake / XX-rekey items in [Unreleased] as 0.5.0 work (the XX path
does not exist on master, which is IK). Release notes adopt master's
refreshed versions. README stays at v0.5.0-dev on next (master's badge
bump to v0.4.0 is master-line only).

Claude-Session: https://claude.ai/code/session_01A2pYfSypNmmG4HyHwZuLex
 2026-06-21 14:02:23 +00:00
Johnathan Corgan d59e4d18f9 Merge branch 'master' into next 2026-06-20 14:58:16 +00:00
Johnathan Corgan 5b767827bb Merge branch 'master' into next 2026-06-18 15:38:10 +00:00
Johnathan Corgan 82ba8e656f Merge branch 'master' into next 2026-06-17 18:31:13 +00:00
Johnathan Corgan db7ad32db8 Merge branch 'master' into next 2026-06-17 16:43:18 +00:00
Johnathan Corgan ec123107ef Merge branch 'master' into next 2026-06-17 15:40:54 +00:00
Johnathan Corgan 86ec7a37e8 Merge branch 'master' into next 2026-06-14 18:54:23 +00:00
Johnathan Corgan 784e22ab1f Merge branch 'master' into next 
Forward-merge the v0.4.0 pre-release content (dependency refresh, CI
deb-install + AUR-build legs, docs refresh, doc-comment fixes, and the
Phase 4 source-content squash: CHANGELOG, release notes, README,
fips.yaml, Cargo.toml metadata, reference docs) up the one-way flow.

Conflict fixups (keep next's identity, fold in master's improvements):
- Version: keep next's 0.5.0-dev (Cargo.toml/lock).
- README status: keep next's v0.5.0-dev / wire-format-breaking framing
  and the Breaking-section pointer; fold in the Nym transport and the
  "global, public test mesh of thousands of nodes" description.
- docs/reference/cli-fips.md: keep next's 0.5.0-dev version example.
- CHANGELOG: keep next's XX-handshake admission entry (no early cap gate
  on XX) and drop master's IK early-cap-at-handle_msg1 entry, which
  describes IK-only behavior that does not apply on next; take master's
  OR-union mesh-size rewrite (next carries the OR-union code); restore
  the Tor connect_refused and MMP receiver-report entries that the
  Fixed-section consolidation would otherwise have dropped.
- lifecycle.rs: keep the mDNS/LAN handshake doc-comment as Noise XX
  (next unifies on XX), not master's XX-to-IK correction.

Quartet green on the merged tree: fmt, build, clippy -D warnings, and
cargo test --lib (1434 passed).
 2026-06-14 18:24:32 +00:00
Johnathan Corgan 0e5f7c90f4 Merge branch 'master' into next 2026-06-13 23:16:48 +00:00
Johnathan Corgan d97c07fd8d Merge branch 'master' into next 
Carries the Nym mixnet transport + single-container demo.

Forward-merge adaptation: the Nym end-to-end SOCKS5 send/recv test was
authored against master's IK FMP framing (msg1 wire size 114, version
nibble 0). next uses the XX handshake and FMP v1, so the TCP read path
rejected the test frame (UnknownVersion / HandshakeSizeMismatch),
timing the test out. Adapted build_msg1_frame to next's framing (wire
size 41, ver=1/phase=1 header byte). next-only; master keeps its IK
form.
 2026-06-13 19:44:44 +00:00
Johnathan Corgan 65d44ab59a Merge master into next: fipstop SSH/tmux garble fix 2026-06-13 02:55:02 +00:00
Johnathan Corgan 0aada5c156 Merge master into next: demote rekey-abort logs to debug 2026-06-13 01:37:55 +00:00
Johnathan Corgan 2ea215f480 Merge branch 'master' into next: fipstop TUI overhaul and disconnect notify-peer fix 2026-06-12 23:53:54 +00:00
Johnathan Corgan 63b1821542 Merge branch 'master' into next: control read-snapshot plane 
Forward-merge the off-loop control read-snapshot plane (every show_* read
query served from published snapshots; show_* removed from the rx_loop).

Adapt the per-entity MMP projection to next's MMP model, which no longer
carries a spin bit: drop the spin_bit_initiator row field, its tick-side
population, and the spin_bit_role output, so the off-loop show_mmp render
stays byte-identical to next's metrics-only on-loop handler.
 2026-06-10 17:57:22 +00:00
Johnathan Corgan cefdc549f7 Merge branch 'master' into next 2026-06-10 02:43:47 +00:00
Johnathan Corgan 906e01b388 Merge branch 'master' into next 2026-06-10 00:05:23 +00:00
Johnathan Corgan 6aded6e365 Merge branch 'master' into next 2026-06-09 11:33:51 +00:00
Johnathan Corgan 8985ea66db Merge branch 'master' into next 2026-06-08 20:26:30 +00:00
Johnathan Corgan 878c40c91f Merge master into next (single-uplink-leaf tree-attachment re-push fix; macOS resolver ::1 fix) 2026-06-08 18:19:10 +00:00
Johnathan Corgan aed64f6d80 Merge master into next (MMP stale-report rejection; convergence-gate near-budget hold) 2026-06-08 00:19:37 +00:00
Johnathan Corgan 9ce4f2e908 Merge master into next (mesh-size OR-union estimate; log-level hygiene) 
Brings the OR-union mesh-size estimator fix and the per-peer / capacity-cap
log-level demotions forward to next.

Conflict resolution: kept next's XX handshake structure and re-applied the
intent at next's own sites. compute_mesh_size auto-merged (next already
carries the config() accessor). The K-bit cutover promotion log demotion
applied to next's richer index-bearing form in encrypted.rs.

master's log changes lived in handle_msg1's IK-only identity/promotion
blocks, which have no XX equivalent there (next defers identity and
promotion to handle_msg3). next's equivalents were demoted in place:
"Connection promoted to active peer" and "Peer restart detected (epoch
mismatch)" info -> debug, and the duplicate "Inbound peer promoted to
active" removed. The two XX-only restart variants ("during FMP rekey" and
"during promotion") were also demoted info -> debug so the restart-detected
class is uniformly at debug on next.
 2026-06-06 19:32:44 +00:00
Johnathan Corgan a3b9954797 Merge master into next (libclang build-prereq doc + transport mutex-poison recovery) 
# Conflicts:
#	src/transport/ble/discovery.rs
#	src/transport/ethernet/discovery.rs
 2026-06-06 13:40:28 +00:00
Johnathan Corgan 84c690cecd docs: reconcile the next changelog with superseded XX-line work 
Two next-line entries had been left describing states that a later commit
reverted. Collapse the XX max_peers early-gate add/remove churn into its
net final state: on the XX handshake an over-cap inbound connection is
rejected solely by the late promotion check (no early gate), logged at
debug. Replace the stale "rekey jitter disabled on next" note with the
actual state: jitter is re-enabled (REKEY_JITTER_SECS = 15) and the three
XX rekey-path divergence defects that had blocked it are fixed
(authenticated-decrypt cutover instead of bare K-bit, msg3 retransmission
with per-link rekey serialization, and a jitter-aware handle_msg3
session-age partition). Merged-in IK-line entries are left untouched.
 2026-06-05 21:44:37 +00:00
Johnathan Corgan 95b475d59b Merge branch 'master' into next 2026-06-05 21:30:14 +00:00
Johnathan Corgan 1c28c2a736 node: enforce the inbound max_peers cap solely at promotion on XX 
At the inbound peer cap the XX FMP handshake had two cap checks: an early
gate in handle_msg3 and the late promote_connection check. On XX the
peer's identity is not known until the third handshake message, by which
point all three messages have already crossed the wire, so the early gate
saved no wire bytes and governed exactly the same net-new-peer set as the
late check (known and pending-outbound peers return earlier via the
cross-connection paths). Remove the early gate and rely on the promotion
check, and log the resulting MaxPeersExceeded rejection at debug instead
of warn so a cap'd node under sustained inbound pressure does not emit
WARN spam for expected policy rejections.

Re-enable the two previously-ignored inbound-cap unit tests, rewritten to
drive a full XX three-message handshake and assert the path-agnostic
invariant (no promotion over cap, peer count unchanged, no
connection/link/index leak) plus the known-peer reconnect bypass.

Update the admission-cap integration suite's enforcement-event
discriminator to match: drop the removed early-gate string and count the
new debug-level promotion-cap rejection.
 2026-06-05 18:52:37 +00:00
Johnathan Corgan 66db3e14a7 Merge branch 'master' into next 2026-06-05 17:17:14 +00:00
Johnathan Corgan 2bc3a505f6 Merge branch 'master' into next 2026-06-05 04:47:00 +00:00
Johnathan Corgan 97f496afc4 Merge master into next (FMP cutover authenticate-before-promote; next already carries the XX form, no code change) 2026-06-04 21:44:36 +00:00
Johnathan Corgan 40c06b28e5 Merge master into next: FMP rekey msg1 resend cap + rekey-aware link-dead heartbeat (with next msg3-budget suppression clause) 2026-06-04 18:52:30 +00:00
Johnathan Corgan ea3e7f8c21 fmp: fix jitter × XX rekey session divergence, re-enable rekey jitter 
Re-enables the rekey timer jitter on the XX FMP rekey path
(REKEY_JITTER_SECS 0 -> 15), which had been disabled because it produced
reproducible post-rekey routing loss (~50% Phase-5 ping failure) with no
crypto errors. The failures were session divergence: under jitter the two
directions of a link rekey close together in time, and three distinct
defects in the FMP rekey state machine could leave the two endpoints
committed to different Noise sessions, starving the receiver until the 30s
heartbeat dead-timer tore the link down (tree parent loss -> routing
failure). All three are fixed here.

1. Promote on authenticated decrypt, not the bare K-bit. The K-bit-flip
   handler promoted whatever pending session existed the instant the header
   bit flipped; under interleaved rekeys that could be a stale pending from
   an earlier epoch. Trial-decrypt the inbound frame against the pending
   session and promote only if it authenticates, mirroring the FSP cutover
   discipline; deliver that plaintext through the canonical path and leave
   the pending untouched otherwise.

2. Retransmit FMP rekey msg3 until confirmed. FMP sent msg3 once; a lost
   datagram left the responder without the new session. Retain the msg3
   payload and resend over the existing link until a peer frame
   authenticates against the pending or post-cutover current session,
   abandoning after the configured handshake-resend budget (mirrors FSP).
   Also serialize per-link rekeys: do not start a new rekey while one awaits
   cutover or is still retransmitting msg3.

3. Partition the handle_msg3 paths by rekey age. An inbound msg3 on a
   different link took the initial-handshake cross-connection tie-breaker
   when the session was under a fixed 30s old, otherwise the rekey
   responder. A rekey resets the session-age clock, so under jitter a
   rekey-aged session is frequently under 30s and its concurrent rekey msg3
   was swallowed by the cross-connection branch, which discarded the peer's
   rekey session with no pending slot while the peer cut over to it anyway.
   Bound the cross-connection branch by the same jitter-aware age floor the
   responder uses, so the two paths partition with no overlap.

Verified at jitter=15: rekey integration suite 70/70 across repeated runs
locally and on GitHub CI, rekey-accept-off 71/71, rekey-outbound-only
75/75; lib 1369/0, clippy and fmt clean. At zero jitter the acceptance
floor equals the previous 30s constant, so default-cadence behavior is
unchanged.
 2026-06-03 13:43:11 +00:00
Johnathan Corgan 3bf49de01f Merge sole-store context change into the next-side 
Bring the immutable-state single-store change onto the Noise XX line.
The shared NodeContext is now the sole store; this merge applies the
next-only adaptations the master-side change couldn't carry:

- Remove node_profile from the Node struct (next-only field) so it lives
  solely in NodeContext; migrate its readers (tree/bloom/discovery/
  handshake negotiation) onto the node_profile() accessor. The two FMP
  negotiation sites hoist node_profile() into a local to avoid borrowing
  &self while a connection is mutably borrowed.
- leaf_only sets both is_leaf_only and node_profile via the context swap.
- Preserve the XX handshake/rekey structure (no identity-in-msg1; XX/XK
  initiator/responder constructors) while applying the startup_epoch()
  accessor migration.
- Tests: route profile selection through a make_test_node_with_profile
  helper (profile is immutable, set via Config flags) instead of poking
  the removed field.

cargo test --lib 1369/0; clippy -D warnings and release build clean.
 2026-06-02 17:30:15 +00:00
Johnathan Corgan e181df0ac4 node: extract immutable state into a shared context and atomic metric registry 
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, node profile, capability limits).
Source the immutable config and identity reads across the receive hot
path, the XX handshake/session/rekey state machines, and the discovery,
tree, bloom, retry, and lifecycle modules through the context accessors.
Includes the bloom delta/full/NACK/resize and byte-total counters in the
registry. The Node fields and the context are rebuilt in lockstep at every
mutation site.
 2026-06-02 13:05:03 +00:00
Johnathan Corgan 88e48fdc68 Merge the fast-path refactor into the next line 2026-06-02 13:05:03 +00:00
Johnathan Corgan 3b27a1604d Merge branch 'master' into next 2026-05-30 20:03:13 +00:00
Johnathan Corgan 1914b20485 Merge branch 'next' into refactor-hotpath-next 2026-05-30 02:42:32 +00:00
Johnathan Corgan bbd4de9e76 Merge master into refactor-hotpath-next 
Bring master's runtime peer-list refresh, opt-in mDNS LAN discovery, and
the receive-path reject-reason / reloadable-config refactor into the XX
handshake integration branch. The peer-restart-epoch detection and
stale-FSP-session teardown authored against Noise IK are re-authored onto
the Noise XX rekey path: complete_rekey_msg2 now also surfaces the remote
startup epoch, and the stale session is cleared once the XX rekey
completes (after msg3 is sent). Connect-budget and path-refresh work is
threaded through the XX anonymous-discovery branch.
 2026-05-30 02:42:01 +00:00
Johnathan Corgan fccd760ad7 Merge branch 'refactor-hotpath' into refactor-hotpath-next 
# Conflicts:
#	src/node/tests/acl.rs
 2026-05-29 02:38:26 +00:00
Johnathan Corgan a965b1ec67 Merge branch 'refactor-hotpath' into refactor-hotpath-next 2026-05-29 01:23:41 +00:00
Johnathan Corgan b643eb3a89 testing: port admission-cap integration test to the Noise XX handshake 
The admission-cap suite asserted IK wire sizes (inbound Msg1 len 84,
outbound Msg2 len 104) to verify the inbound max_peers cap. On the XX
handshake those sizes differ and Msg2 is always sent before identity is
known, so the IK discriminator matched nothing and the suite failed
despite the cap working correctly.

Replace the wire-size discriminator with a protocol-agnostic cap-holds
invariant: each denied peer must keep re-initiating handshakes
(sustained inbound, size-agnostic), no denied peer may be promoted to an
active session, the node must log cap-enforcement events, and the capped
node must hold exactly max_peers. Raise node-c's handshake logging to
debug in the mesh profile so the early-gate drop is visible alongside the
late-check rejection.

Also record in the changelog that for peers the node also dials, the cap
is enforced by the retained late check rather than the early msg3 drop.
 2026-05-29 00:20:29 +00:00
Johnathan Corgan e1ae261eb2 node: wire the next-side XX handshake and rekey rejection sites 
Adapt the typed RejectReason coverage to the Noise XX handshake: wire the
msg1/msg2/msg3 state-machine rejection sites in handlers/handshake.rs and
the rekey-initiator outbound sites in handlers/rekey.rs, and add the
XX-specific HandshakeStats counters. The shared RejectReason scaffold and
the branch-agnostic clusters come from the merged refactor-hotpath work;
this commit carries only the next-side delta.
 2026-05-28 21:21:45 +00:00
Johnathan Corgan 939e2cbd4f Merge branch 'refactor-hotpath' into refactor-hotpath-next 
# Conflicts:
#	src/node/bloom.rs
#	src/node/handlers/handshake.rs
 2026-05-28 21:21:12 +00:00
Johnathan Corgan 726461c193 node/tests: reword handshake regression-test doc-comments 
Drop an internal tracker reference from the doc-comments of the
should_admit_msg1 and udp.outbound_only rekey regression tests; the
prose now describes the scenario directly.
 2026-05-28 21:13:26 +00:00
Johnathan Corgan cc417a7bef node: free the connection index when XX msg3 processing fails 
The msg3-processing-failure cleanup read our_index from the connection
after removing it from the connections map, so the lookup always returned
None and the allocated index was never released, slowly leaking index
slots across failed XX handshakes. Capture the index before the remove,
matching the idiom used in the other cleanup paths in this file.
 2026-05-28 21:13:26 +00:00
Johnathan Corgan 8e72326ec2 node/tests: mark leaf-nonrouting handshake test as parallel-load flake-class 
The XX leaf-nonrouting rejection test shares the localhost-UDP rcvbuf
overflow failure mode of the other large-network tests under parallel
cargo test --lib load. Mark it #[ignore] for the default run; it stays
runnable with --ignored or --test-threads=1.
 2026-05-28 20:19:50 +00:00
Johnathan Corgan a6e4481c1f Merge branch 'master' into next 2026-05-28 20:19:19 +00:00
Johnathan Corgan 0a84a25db1 Merge master into next (inbound admission cap silent-drop; XX-adapted gate at handle_msg3) 
Carries the inbound max_peers admission-cap silent-drop work from
master across the IK→XX structural boundary.

On master (Noise IK), the gate fires at handle_msg1 after the peer's
identity is extracted from the IK msg1 payload, before Msg2 is built
and sent. Wire savings of Msg2 (~104 B) plus the responder crypto
per cap-denied attempt motivate the gate placement.

On next (Noise XX), peer identity is not learned until msg3, by
which point Msg1, Msg2, and Msg3 have all crossed the wire. The IK
gate's wire-savings rationale does not apply on XX. The equivalent
gate is placed in handle_msg3 after the peer's static-key + signature
verification (identity now known) and before promote_connection
(ActivePeer construction, peers_by_index insert, link transition).
The value on XX is local CPU / allocation savings and cleaner peer-
side semantics: no fake-promotion whose subsequent data frames fail
decryption on this side, replaced by a silent drop the peer's
existing auto-reconnect backoff handles cleanly.

Bypass logic is the same as the IK version: known-active peers
(reconnect / cross-connection / restart) and pending-outbound peers
skip the cap so admitting them doesn't grow peers.len(). The late
cap check inside promote_connection is intentionally retained as
defense-in-depth.

Cleanup ordering for the gate's silent-drop path corrects a latent
issue in the adjacent msg3-processing-failure path: our_index is
captured before the connection is removed so the allocator slot is
actually freed (the pre-existing path's get-after-remove returns
None, leaking the index).

The two IK-shaped unit tests at handle_msg1 are marked #[ignore] on
this branch with reason. Their wire-bytes discriminator ("Msg2 must
NOT be sent at cap") is structurally false on XX, and the XX-shaped
equivalents need full three-message-handshake test scaffolding that
isn't authored here. XX-adapted unit + integration test coverage for
the handle_msg3 gate is a follow-up.

CHANGELOG [Unreleased] entry adjusted to describe the handle_msg3
placement on this branch (preserving the master-side framing on the
mainline-merge half of the entry would have been misleading on this
branch).

Three conflicts auto-resolved: the structural handshake.rs span
(~253 lines, IK identity-tied logic discarded on this branch's XX
msg1 path), and three textual ci-local.sh conflicts (admission-cap
suite list / dispatch added alongside the existing mixed-profile
suite list / dispatch).
 2026-05-27 17:01:59 +00:00
Johnathan Corgan 05124367db Merge master into next (outbound admission gate + mesh-size parent skip) 2026-05-26 17:31:47 +00:00
Johnathan Corgan df342f268a Merge master into next (periodic TreeAnnounce re-broadcast + harness) 2026-05-26 15:11:40 +00:00
Johnathan Corgan 61ac4c6a32 Merge master into next (cross-conn-won decrypt-worker unregister fix) 2026-05-25 17:23:04 +00:00
Johnathan Corgan 49735980b8 Merge master into next (cross-init tiebreaker + tick-arm connect-on-send fix) 2026-05-25 05:13:00 +00:00
Johnathan Corgan 09df743485 Merge master into next (interop harness, mesh-lab harness, forwarding debug log) 2026-05-23 14:48:46 +00:00
Johnathan Corgan 60c42daa82 Merge master into next (FSP rekey overlapping-epoch, drain-erase fix) 2026-05-23 02:06:04 +00:00
Johnathan Corgan d6729086de Merge master into next (data-plane perf overhaul, packaging fixes) 
Forward-merges PR #91 (off-task encrypt/decrypt worker pools, GSO,
connected UDP), the macOS #102 package-integrity fix, and the AUR
#98 fips-dns fix from master.

Six textual conflicts resolved: wire/noise imports, the K-bit-flip
and rekey-cutover handlers, ActivePeer fields, and an advert test
fixture. One semantic conflict: PR #91's worker-pool code on master
referenced FLAG_SP and MmpPeerState.spin_bit, which next removed in
4aded9a2 (spin bit superseded by the MMP receiver-report timestamp
echo; FMP flags bit 2 reclaimed). The worker acceleration and its
fmp_flags round-trip test are kept; the spin-bit references are
dropped to honor next's wire-format change.
 2026-05-21 01:54:47 +00:00
Johnathan Corgan d1b2f31a3b Merge master into next (rekey-test ping retry, platform-warning cleanup, CHANGELOG backfills) 
# Conflicts:
#	CHANGELOG.md
 2026-05-18 01:47:51 +00:00
Johnathan Corgan c80b3eb0e3 Merge master into next (coord cache surgical invalidation) 
# Conflicts:
#	CHANGELOG.md
 2026-05-17 01:12:44 +00:00
Johnathan Corgan 6bb17be8b8 rekey: disable timer jitter on next pending XX-path investigation 
Set `REKEY_JITTER_SECS` to 0 on next. The symmetric per-session
jitter mechanism added on maint/master works cleanly on the IK
FMP rekey path on those branches, but on next's XX FMP rekey
path it produces reproducible post-cutover routing-convergence
failures: ~50% Phase 5 per-pair-ping loss in the rekey integration
suite across multiple runs, with all crypto, transport, and
link-state log assertions still green (no decrypt failures, no
panics, no link teardowns, no rekey msg2 failures, no FSP
decryption failures during rekey).

The XX rekey path itself is sound — pre-merge it had been passing
18/18 across six consecutive CI runs. Isolation experiment
confirms the trigger: with REKEY_JITTER_SECS=0, the same suite
on next passes 70/70 cleanly with FMP cutover counts (18-21)
in line with master's baseline (24). With the constant at 15
and the test suite's 35s nominal rekey interval, FMP cutover
counts climb to 27-40 and Phase 5 fails consistently.

Two attempted fixes to absorb the jitter-induced state churn on
the XX path (post-msg3 addr_to_link restore; optimistic dampening
at msg1 receipt) were tried and reverted — neither restored
Phase 5 reliability. Diagnosis of the underlying XX cutover
state-cleanup behaviour that fails to absorb variable-interval
rekeys is deferred. The const and surrounding draw/redraw
machinery (peer::active::ActivePeer, node::session::SessionEntry,
draw_rekey_jitter at session.rs:20, redraws at the four cutover
sites) are kept in place; flipping the const back to a positive
value is the re-enable path once the investigation lands.

CHANGELOG entry under `### Changed`.
 2026-05-14 21:16:43 +00:00
Johnathan Corgan a014296639 node/tests: adapt rekey-jitter tests to XX HandshakeState signature 
The two rekey-jitter unit tests added on the IK-rekey branches
construct a HandshakeState via:

  HandshakeState::new_initiator(keypair, remote_pubkey)

On next, the XX rewrite changed the initiator constructor to take
only the local keypair:

  HandshakeState::new_initiator(keypair)

XX does not learn the responder's static pubkey until msg2, so
the IK-shaped signature does not apply. The merge from master
brought the test bodies verbatim; this drops the second argument
in both call sites so the tests compile against the XX API. The
test semantics (rekey-jitter range and mean checks) are unchanged
and orthogonal to the handshake-pattern choice.

Same shape as 8a6477b adapting the bootstrap-handoff tests to
XX's three-way handshake.
 2026-05-14 18:59:14 +00:00
Johnathan Corgan 74b08b9b6f Merge master into next (rekey jitter, Phase 5 settle, acl-allowlist, CI concurrency) 
# Conflicts:
#	CHANGELOG.md
 2026-05-14 18:54:05 +00:00
Johnathan Corgan 418c9d61e0 ci: add mixed-profile suite to integration matrix on next 
The `mixed-profile` integration suite (Full / NonRouting / Leaf
node-role mix introduced by the forklift-upgrade work) is wired
into `testing/ci-local.sh` but was silently absent from the GitHub
Actions integration matrix on next. Regressions to node-role
behavior could land via GitHub CI without the suite running.

Adds a matrix entry plus the matching step block (mirroring the
rekey pattern: generate configs + inject config + compose up +
test + log-collect on failure + compose down always). master
correctly omits the suite from both ci-local and ci.yml since
mixed-profile is forklift-only, so this change never forward-merges
to master.
 2026-05-14 15:58:28 +00:00
Johnathan Corgan 358a06be06 Merge master into next (MTU, rekey baseline, CONTRIBUTING) 
# Conflicts:
#	CHANGELOG.md
 2026-05-14 00:02:08 +00:00
Johnathan Corgan 183ad148de Fix XX rekey dual-init race: tie-break also when pending_new_session set 
The dual-initiation tie-breaker in the rekey arm of handle_msg3 (and
the FSP analogue in handle_session_setup) only fired when
rekey_in_progress was true. With Noise IK (one-message rekey on
master/maint) that is sufficient: msg1 IS the rekey, so both sides
being mid-handshake is the only state where the race can fire.

With Noise XX (three-message rekey on next), set_pending_session runs
when the initiator has processed msg2 and sent msg3, and that call
clears rekey_in_progress. Both sides' set_pending_session can run
before either peer's msg3 has landed. Each side then receives the
peer's msg3 in the post-pending state with rekey_in_progress=false,
falls into the "drop because pending_new_session is set" guard, and
discards the peer's handshake. Each side commits its own initiator
session at K-bit cutover. The two sessions use different Noise key
material, so the link breaks asymmetrically after cutover.

Unify both checks into a single tie-breaker that fires when either
rekey_in_progress() OR pending_new_session().is_some() is true. The
existing smaller-NodeAddr rule applies uniformly; abandon_rekey()
already clears both states and returns whichever index needs
freeing.

Mirrored to the FSP rekey msg1 path in handle_session_setup for the
same race shape.

Logging:

- The previously-silent drop in the rekey arm of handle_msg3 logs at
  info as a tie-break decision rather than as a silent drop.
  Pending_new_session is added as a log field on the tie-break
  win/lose lines so a reader can distinguish which of the two race
  states fired.
- Cutover-complete and K-bit-flip log lines gained our_addr and
  their_addr fields so the two endpoints' logs of the same handshake
  can be correlated.
- "Pending session set, awaiting K-bit cutover" lines remain at
  debug per the existing convention of info-for-cutover-completion-
  only.

Verification: under the un-fixed handler, both sides log
"rekey-msg3 drop: pending_new_session already set" within 242
microseconds of each other, with rekey_in_progress=false and
different pending session indices. Failure pattern matches: six of
twenty pairs FAILED post-rekey, all involving the single-peer node.
Six consecutive post-fix CI attempts on the same test green across
rekey, rekey-accept-off, and rekey-outbound-only suites.
 2026-05-13 23:58:10 +00:00
Johnathan Corgan 195b3bad01 Merge master into next (AUR publish fix) 2026-05-12 17:57:25 +00:00
Johnathan Corgan 212c45bd37 Merge master into next after v0.4.0-dev open 
Marker merge to record the master dev-line (now v0.4.0-dev, with
maint's v0.3.1-dev open already incorporated) as known on next
without taking master's tree (next is on v0.5.0-dev). Future
forward-merges from master to next land cleanly on top of this base.
 2026-05-12 13:47:29 +00:00
Johnathan Corgan b0e9627a77 chore: retarget next from v0.4.0-dev to v0.5.0-dev 
Master is now the v0.4.0-dev line; next moves up to v0.5.0-dev for
the wire-format-breaking work staged here.

- Cargo: 0.4.0-dev → 0.5.0-dev
- CHANGELOG Breaking-section header retargeted to v0.5.0; baseline
  bumped from v0.2.x to v0.3.x peers (master is now the v0.3.1-dev
  patch line and v0.4.0-dev minor line, so v0.5.0 breaks against
  v0.3.x, not v0.2.x)
- README: badge v0.4.0--dev → v0.5.0--dev, three Status & Roadmap
  prose references retargeted
- docs/reference/cli-fips.md: example version 0.4.0-dev → 0.5.0-dev
 2026-05-12 13:41:21 +00:00
Johnathan Corgan 7865557f8a Merge master into next after v0.3.0 release 
Brings v0.3.0 release content forward into the next-branch
development line, preserves next's own version state, and updates
operator-facing version references to v0.4.0-dev.

Kept from master:
- docs/releases/release-notes-v0.2.1.md and -v0.3.0.md (cumulative
  archive of shipped releases)
- RELEASE-NOTES.md root mirror at v0.3.0 (tracks the most recent
  shipped release; will be replaced when the v0.4.0 release cycle
  begins on this line)
- CHANGELOG entries: [0.3.0] and [0.2.1] sections inserted under
  the existing Breaking and [Unreleased] sections
- All code, config, test, and documentation updates from master
  (openwrt yaml resync, doc-config IP placeholders, etc.)

Kept from next (resolved against master's release-prep changes):
- Cargo.toml / Cargo.lock at 0.4.0-dev (next's package version)
- CHANGELOG ## Breaking section (next-specific v0.4.0 wire-format
  breaking work) and the empty ## [Unreleased] block for future
  v0.4.0 non-breaking work

Updated to v0.4.0-dev for consistency with Cargo.toml:
- README badge (v0.3.0--dev -> v0.4.0--dev) and status-section
  prose, rewritten to describe v0.4.0 wire-format-breaking work
  on this branch (Noise XX unification, FMP node profiles,
  slimmer MMP, extensible bloom-filter encoding) instead of the
  v0.3.0 testing-and-polishing narrative that applied while
  v0.3.0 was unreleased
- docs/reference/cli-fips.md example version string
 2026-05-11 19:00:48 +00:00
Johnathan Corgan 7807516b8d Merge branch 'master' into next 2026-05-10 22:04:29 +00:00
Johnathan Corgan e5071d74fb Merge branch 'master' into next 2026-05-10 17:02:33 +00:00
Johnathan Corgan 831ce5f741 Merge branch 'master' into next 2026-05-10 02:16:49 +00:00
Johnathan Corgan dd65951244 Merge branch 'master' into next 
# Conflicts:
#	src/node/tests/bootstrap.rs
 2026-05-09 23:22:32 +00:00
Johnathan Corgan c22b50c147 Merge branch 'master' into next 2026-05-09 18:45:05 +00:00
Johnathan Corgan 19b921ec5d Merge branch 'master' into next 2026-05-08 21:42:16 +00:00
Johnathan Corgan 45a8a4cd9b Merge branch 'master' into next 
Forward-merge of 12 master commits past the previous merge
(823b830, master @ 18019bb): dep-audit bumps (rand,
clap, tun, rtnetlink, windows-service, plus the bump-safe
lockfile batch), bloom-storm chaos scenario, control-socket
resolver consolidation, gateway dns.listen default change,
OpenWrt ipk README refresh, gateway tutorial review, Ethernet
MTU rustdoc fix, dead session-variant drop, CHANGELOG prep.

Conflict resolution:

- CHANGELOG.md: both bullets kept under [Unreleased] / Fixed.
  Master's spanning-tree internal-path-propagation fix precedes
  next's tree-ancestry-test determinism entry and the
  responder-Disconnect XX-handshake entry.
- src/protocol/session.rs: kept next's SessionSetup/SessionAck
  variants and rustdoc. Master's drop of those variants suits
  v0.3.0's FSP phase-byte dispatch but is undone by next's
  v0.4.0 wire format, which retains the variants and uses the
  inner msg_type byte for handshake identification.
- src/transport/ethernet/mod.rs: kept next's "interface MTU - 4"
  comment. Master corrected the v0.3.0 3-byte rustdoc; next
  redesigned the framing to a 4-byte header (type/flags/length)
  for shared-media beacons, so master's correction does not
  apply to next's format.

Auto-merged cleanly: Cargo.toml (next's 0.4.0-dev + the new dep
pins from master), Cargo.lock, all gateway docs,
docs/reference/configuration.md, packaging files,
.github/workflows/ci.yml, testing/chaos/sim/* and
testing/ci-local.sh (bloom-storm additions), src/config/*.

Local verification: cargo build --release, cargo test (1252
passed, 4 ignored), cargo clippy -D warnings, cargo fmt --check
all green.
 2026-05-08 18:47:13 +00:00
Johnathan Corgan 823b830289 Merge branch 'master' into next 
Forward-merge of the docs-overhaul squash (5abf9a9) and top-level
README rewrite (18019bb). Conflict resolution:

- README.md: master's rewritten feature lists adopted, with the encryption
  bullets reflecting next's two-layer Noise XX (replacing the IK/XK pair
  master describes for v0.3.0).
- 6 design/reference markdown files (fips-bloom-filters.md, fips-mesh-layer.md,
  fips-mesh-operation.md, fips-session-layer.md, fips-transport-layer.md,
  reference/wire-formats.md): master's reorg taken, next's protocol details
  preserved (XX handshake naming, bloom v2 RLE/delta wire format,
  v2 LookupRequest sizing).
- fips-intro.md modify/delete: accepted master's split into
  fips-architecture.md / fips-concepts.md / fips-prior-work.md, then
  re-applied next's IK/XK -> XX transition and spin-bit removal across
  the relevant split files. Same pass swept docs/reference/security.md,
  docs/design/fips-mmp.md, docs/design/fips-security.md,
  docs/design/fips-nostr-discovery.md,
  docs/design/port-advertisement-and-nat-traversal.md,
  docs/how-to/enable-nostr-discovery.md, and the affected tutorials so
  no IK/XK or spin-bit prose remains in current-state docs.
- Diagram path conflicts: noise-ik-msg{1,2}.svg removed (IK is gone);
  noise-xx-msg{1,2,3}.svg moved from docs/design/diagrams/ to
  docs/reference/diagrams/ to match master's diagram reorg. The
  wire-formats.md image references resolve correctly to the new path.

Local verification: cargo build --release, cargo test (1265 passed,
4 ignored), cargo clippy -D warnings, cargo fmt --check all green.
 2026-05-08 13:45:04 +00:00
Johnathan Corgan 933e9ee6ee Merge branch 'master' into next 
# Conflicts:
#	src/node/bloom.rs
 2026-05-07 14:31:11 +00:00
Johnathan Corgan fb834dc2ca nostr: separate next-branch advert namespace by default 
Bump the default Nostr-discovery advert namespace from
`fips-overlay-v1` to `fips-overlay-v1-next` on next. Master
continues to publish under `fips-overlay-v1`.

Background: next runs FMP-v1 (Noise XX, msg1 33 bytes) which is
wire-incompatible with master's FMP-v0. Until now both branches
defaulted to the same Nostr advert namespace, so a stock
next-branch daemon's open-discovery sweep would happily pick up
master peers' adverts (and vice versa), succeed at the UDP punch,
adopt the socket, and fail every FMP handshake at the version-gate.
The per-peer-mismatch cooldown introduced on master is the safety
net for any case that slips past this default; the namespace
separation is the structural answer.

Three sites updated:

- `src/discovery/nostr/types.rs` `ADVERT_IDENTIFIER` const
  documents why the value is branch-specific.
- `src/config/node.rs` `default_app()` matches.
- `src/discovery/nostr/tests.rs` and the
  `testing/nat/scripts/nostr-relay-test.sh` malformed-advert
  fixture publish under the new namespace so test harnesses see
  the same adverts a real daemon would.

Operators who need cross-branch discovery during a coordinated
rolling upgrade can override `node.discovery.nostr.app` in
fips.yaml back to `fips-overlay-v1`.
 2026-05-06 15:09:16 +00:00
Johnathan Corgan acb750c0d2 Merge branch 'master' into next 2026-05-06 15:03:27 +00:00
Johnathan Corgan 5a80582b0b Merge branch 'master' into next 2026-05-05 14:49:51 +00:00
Johnathan Corgan 60a4353661 Merge branch 'master' into next 2026-05-04 14:02:09 +00:00
Johnathan Corgan 3cbb667655 Regenerate show_bloom snapshot for next-branch bloom v2 stats 
The control-query show_bloom snapshot, baselined on master, mismatches
the next-branch output because the bloom filter v2 work on next added
six new stats fields (deltas_sent, full_sends, nacks_received,
nacks_sent, size_changes, total_compressed_bytes, total_raw_bytes)
and dropped non_v1. The snapshot test correctly flagged this as
schema drift.

The drift is intentional (next's bloom v2 is a deliberate schema
expansion, not a regression), so the right action per the test's own
guidance is to delete the fixture and let the test regenerate it
against next's actual output.

No production-code change.
 2026-05-03 23:51:32 +00:00
Johnathan Corgan 4457587e5f Merge branch 'master' into next 
# Conflicts:
#	src/node/tests/discovery.rs
#	testing/ci-local.sh
 2026-05-03 23:47:16 +00:00
Johnathan Corgan 1c8b8d55e8 Merge branch 'master' into next 2026-05-03 12:32:49 +00:00
Johnathan Corgan b02eaf2c5b Merge branch 'master' into next 
Forward-merges the squashed per-destination TCP MSS clamping work
(master `ae60743`) into next.

Auto-merge of source files clean. Two pre-existing test assertions
on the next-side discovery tests were updated as part of merge
resolution to reflect the target-edge MTU fold introduced by the
merged-in commit:

- test_transit_forwards_when_mtu_sufficient: 1400 → 1280 = min(target-edge 1280, transit 1400)
- test_response_path_mtu_four_node_chain: 1350 → 1280 = min(target-edge 1280, transits 1350+1500)

Resulting next tree is bit-for-bit identical to the pre-squash next
state previously verified by full local CI sweep (29 + 1 next-only
suites pass on `6a8b519` in 23m 41s).
 2026-05-02 17:39:26 +00:00
Johnathan Corgan d457b6d5f6 Merge branch 'master' into next 2026-05-02 02:45:10 +00:00
Johnathan Corgan e7fc4f7d10 Merge branch 'master' into next 2026-05-02 02:14:36 +00:00
Johnathan Corgan 5317735a91 Merge branch 'master' into next 
# Conflicts:
#	src/node/tests/handshake.rs
#	testing/ci-local.sh
 2026-04-30 14:05:59 +00:00
Johnathan Corgan 9c35c7e55d Merge branch 'master' into next 2026-04-29 12:59:10 +00:00
Johnathan Corgan 15753cdc20 Resolve simultaneous-init cross-connection in XX handle_msg3 
After the master→next merge brought PR #53's bootstrap-handoff socket
adoption onto next's XX three-message handshake, three integration jobs
(rekey, nat-cone, nat-lan) regressed: the FMP handshake completed but
post-handshake encrypted frames silently dropped — packets_recv stayed
at 0 and link-dead timeout fired every 30s.

Root cause: in symmetric bootstrap-handoff both sides initiate XX in
parallel, running two concurrent handshakes (each side's outbound paired
with the peer's inbound). Each side's handle_msg2 (immediate response
to its own outbound msg1) ran before the peer's outbound msg3 arrived;
peers.contains_key was false, so the "Normal path" promoted the outbound
connection. When the peer's msg3 then arrived, peers contained the peer
at the same epoch and the request fell through to the
"duplicate handshake from same epoch" branch, which tore down the inbound
link without applying the cross-connection tie-breaker. Both sides ended
up keeping their own outbound session whose Noise key material pairs with
the peer's discarded inbound, so each side's their_index pointed at the
peer's inbound session index that was never registered in peers_by_index.

handle_msg2 already has the symmetric handler for the inverse ordering
(msg3-then-msg2). Add the missing simultaneous-init handler in
handle_msg3: when the existing peer is on a different link from this
msg3's pending_inbound link and the session is fresh (<30s, so this
isn't a rekey), apply cross_connection_winner with this_is_outbound=false.
The larger-node side swaps to the inbound session via replace_session and
updates peers_by_index from outbound_idx to inbound_idx; the smaller-node
side keeps its outbound session and frees the inbound's allocated index.
Both sides converge on the same Noise session pair.

Verified: cargo test --lib (1144 passed), cargo fmt + clippy clean,
nat-cone + nat-lan + symmetric NAT scenarios pass, rekey integration
test 70/70 pairs across all phases.
 2026-04-27 19:39:00 +00:00
Johnathan Corgan 8a6477b7ff Adapt bootstrap-handoff tests to XX three-way handshake 
The two bootstrap-handoff tests were written against the IK two-message
handshake where peer promotion happens after msg2. Under the noise-XX
three-message handshake on this branch, identity is only revealed in
msg3, so peer promotion happens one message later: the responder learns
the initiator's identity in msg3 and only then promotes the peer.

`test_adopted_udp_traversal_completes_handshake`: Previous version ran
two `run_rx_loop` iterations (one per node) and asserted both peers
had been promoted. That works for IK but not XX: after node_a sends
msg1 from `adopt_established_traversal`, node_b's first rx_loop pass
generates msg2 (revealing node_b's identity to node_a), node_a's next
rx_loop pass generates msg3 (revealing node_a's identity to node_b),
and only then does node_b promote node_a. A third rx_loop pass on
node_b is needed.

Restructured the test to use direct `handle_msg{1,2,3}` dispatch via
`packet_rx.recv()` rather than `run_rx_loop` + select cancellation,
matching the pattern already used by
`test_third_peer_can_handshake_via_adopted_transport_socket`. This
sidesteps the issue that `run_rx_loop` does `packet_rx.take()` and a
cancelled future never returns it, so a second rx_loop call fails
with `NotStarted`.

`test_third_peer_can_handshake_via_adopted_transport_socket`: Added
explicit msg3 dispatch on both Alice/Bob and Bob/Colin sub-handshakes
so the responder side actually receives the initiator's identity
before the test asserts peer promotion.
 2026-04-27 17:28:54 +00:00
Johnathan Corgan 5a5dbe36da Merge branch 'master' into next 
# Conflicts:
#	src/node/lifecycle.rs
#	testing/static/scripts/rekey-test.sh
 2026-04-27 16:10:11 +00:00
Johnathan Corgan 64c38b2e26 Merge branch 'master' into next 2026-04-22 03:43:20 +00:00
Johnathan Corgan 3edca4a84f Notify initiator when inbound ACL rejects a Noise XX handshake 
Under Noise XX the responder only sees the initiator's static key
in msg3, so the inbound-handshake ACL check cannot fire until msg3
has already been processed. By then the initiator has received
msg2, passed its own outbound ACL check, and promoted its side of
the peering. Silently tearing down the responder's state left the
initiator as a "connected" peer with no traffic until the
link-dead timeout fired tens of seconds later; the acl-allowlist
integration test on the next branch timed out at the 5s
convergence check on blocked nodes.

The responder now sends an encrypted Disconnect on the
freshly-completed Noise session before tearing down. The initiator
decodes it via the existing handle_disconnect path and cleans up
the zombie peer within one RTT. DisconnectReason::Other is used
instead of SecurityViolation so the wire payload does not name the
ACL mechanism; the behavioural signature (explicit reject right
after handshake) is the only observable leak.

Supporting changes:

- Add send_encrypted_link_message_raw in node/mod.rs that operates
  on a raw NoiseSession + indices without going through self.peers,
  since the peer has not yet been promoted at the reject site.
- Declare the acl test module in node/tests/mod.rs. It had been
  orphaned since PR #50 so none of its tests ran; two of them no
  longer compiled against the current next branch. Fixed the two
  and replaced test_inbound_msg1_denied_by_acl (asserts IK-era
  behaviour that cannot happen under XX) with
  test_inbound_msg3_denied_triggers_disconnect, a full end-to-end
  UDP test covering both the responder cleanup and the initiator's
  Disconnect-driven cleanup.
 2026-04-22 02:51:17 +00:00
Johnathan Corgan bcb87165fa Merge branch 'master' into next 2026-04-22 01:22:54 +00:00
Johnathan Corgan e5e6054229 Merge branch 'master' into next 
# Conflicts:
#	src/bloom/filter.rs
#	src/node/bloom.rs
 2026-04-21 19:42:35 +00:00
Johnathan Corgan 14d2a4f2df Merge branch 'master' into next 
Integrates PR #50 (peer ACL enforcement) into the XX handshake
architecture. ACL enforcement points adapted for XX's deferred
identity learning: InboundHandshake check moves from handle_msg1
to handle_msg3 (responder learns initiator identity), OutboundHandshake
check remains in handle_msg2 (initiator learns responder identity).
Borrow scopes restructured to release connection borrows before
authorize_peer calls.
 2026-04-16 06:11:03 +00:00
Johnathan Corgan fd9a8d132d Merge branch 'master' into next 2026-04-15 17:10:08 +00:00
Johnathan Corgan dab0ed4854 Merge branch 'master' into next 
# Conflicts:
#	testing/ci-local.sh
 2026-04-15 06:35:32 +00:00
Johnathan Corgan 2382c1438f Merge branch 'master' into next 2026-04-14 15:27:12 +00:00
Johnathan Corgan 942cc83878 Merge branch 'master' into next 2026-04-14 14:07:43 +00:00
Johnathan Corgan c2133a0f29 Merge branch 'master' into next 2026-04-14 11:45:59 +00:00
Johnathan Corgan 76a7147b17 Merge branch 'master' into next 2026-04-14 09:34:05 +00:00
Johnathan Corgan 9e93b0b224 Merge branch 'master' into next 2026-04-13 06:37:23 +00:00
Johnathan Corgan 9f539ce3c5 Apply cargo fmt to node module and handshake tests 
Exposed by the rustfmt toolchain-pin fix now that rustfmt runs under
the pinned 1.94.1 toolchain in CI. Mechanical formatting changes:

- src/node/mod.rs: collapse EthernetTransport::new let-binding onto
  one line (fits within width limit)
- src/node/tests/handshake.rs: reorder imports (super::* after
  specific import) and expand two assert_eq! calls to multi-line form
 2026-04-12 09:14:27 +00:00
Johnathan Corgan 5d898490ac Merge branch 'master' into next 2026-04-12 08:39:19 +00:00
Johnathan Corgan fe6f9aaa78 Merge branch 'master' into next 
# Conflicts:
#	src/node/mod.rs
 2026-04-11 17:51:24 +00:00
Johnathan Corgan 44667d9cc2 Update design docs and diagrams for Noise XX and forklift changes 
Update CHANGELOG Breaking section with all v0.4.0 wire format
changes. Comprehensive update to fips-wire-formats.md (handshake,
MMP, FilterAnnounce, discovery, size tables).

Update fips-mesh-layer.md, fips-session-layer.md, fips-intro.md
for IK/XK→XX transition and spin bit removal. Document
disable_routing and leaf_only profile config in
fips-configuration.md. Update fips-transport-layer.md for 4-byte
beacon and unified header. Update fips-bloom-filters.md for delta
compression and variable sizing. Update fips-mesh-operation.md for
node profiles and discovery min_mtu.

Create/update 8 SVG diagrams (XX msg1/2/3, handshake flow, session
setup/ack/msg3, filter-announce, sender/receiver report). Update
README.md Noise pattern references.
 2026-04-11 13:21:47 +00:00
Johnathan Corgan 10122d7d87 Production hardening: unwrap safety, 14 new tests, diagnostics 
Harden unwrap() calls in handler hot paths (handshake, encrypted,
rekey, session handlers) with proper error propagation.

Add 14 tests: profile rejection (6), MMP forward-compat (4),
discovery min_mtu pruning (3), XX duplicate msg1 dedup (1).

Add NodeProfile Display impl with structured tracing for profile
mismatch logging. Expose bloom compression diagnostics
(total_compressed_bytes, total_raw_bytes) in control socket.

Add module documentation for XX identity timing, profile decision
tree, and bloom codec strategy.
 2026-04-11 13:14:17 +00:00
Johnathan Corgan 8b5f1e349f Update .git-blame-ignore-revs with noise-xx format commit 2026-04-11 08:16:01 +00:00
Johnathan Corgan 34f840a77b Apply rustfmt to noise-xx-only code 2026-04-11 08:16:01 +00:00
Johnathan Corgan 59b7ea765b Fix stale discovery success regex in log analyzer 
The log analysis matched "proof verified, caching route" but the
actual log message is "Discovery succeeded, proof verified, route
cached", causing discovery succeeded to show 0 in all sim runs.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan eb8479eb20 Heterogeneous bloom filters: delta compression, variable sizing, adaptive heuristic 
Replace fixed 1KB bloom filters with variable-size filters (512B-32KB)
that adapt to each node's position in the spanning tree.

Core changes:
- Internal storage: Vec<u8> → Vec<u64> for word-level operations
- Delta compression: XOR diff with word-level RLE, sequence-based
  NACK protocol for full retransmit recovery
- Size conversion: fold (large→small) and duplicate (small→large)
  with auto-converting merge for mixed-size filter combination
- Native-size storage: peer filters stored at advertised size for
  full-resolution routing queries, converted only for outgoing filter
- Adaptive sizing: outgoing fill ratio drives step-up/step-down
  between size classes with hysteresis (20%/5% thresholds)
- Filter size decoupled from FMP negotiation: announced dynamically
  in filter updates, bit 7 and TLV field 1 removed from handshake

Wire format: FilterAnnounce gains flags byte (delta bit), base_seq
field, RLE-compressed payload. New FilterNack message (0x21) for
out-of-sequence delta recovery.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 8162d3c237 Minimal shared-media beacons: 4-byte header, strip pubkey, remove BLE exchange 
Redesign shared-media transport framing now that XX replaces IK:

- Ethernet: unified 4-byte header [type][flags][length:2 LE] for all
  frame types, effective MTU now if_mtu - 4
- Beacons: strip 32-byte pubkey (34 → 5 bytes), identity learned from
  XX handshake msg2/msg3 instead of beacon
- BLE: remove pre-handshake pubkey_exchange() and cross-probe
  tie-breaker, both unnecessary with XX
- Discovery: support anonymous connections (pubkey_hint: None) with
  address-based dedup for shared-media transports
- Post-handshake identity hooks in handle_msg2/msg3 for self-detection
  and future allow/deny list filtering (IDEA-0047)
 2026-04-11 08:16:01 +00:00
Johnathan Corgan e06015dd43 Discovery wire format cleanup 
Drop origin_coords from LookupRequest — unused since reverse-path
routing became primary. Saves 2+16*depth bytes per request.

Wire up min_mtu: populated from TUN MTU config (default 1280) at
origination. Transit nodes skip peers whose link MTU is below the
request's min_mtu requirement. path_mtu on LookupResponse was
already wired (transit min() applied).

Add TLV extension to LookupRequest (after min_mtu) and
LookupResponse (after proof). Uses same TlvEntry format as
negotiation. Transit nodes forward TLV bytes verbatim.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 4aded9a238 Remove spin bit, slim down MMP reports with extensibility header 
Delete SpinBitState, FLAG_SP (FMP bit 2), and FspInnerFlags.spin_bit.
Spin bit superseded by MMP receiver report timestamp echo for RTT.
Reclaims FMP flags bit 2 and FSP inner flags bit 0.

SenderReport: 48 -> 20 bytes. Three fields: interval_packets_sent,
interval_bytes_sent, cumulative_packets_sent.

ReceiverReport: 68 -> 54 bytes. Ten decision-driving and diagnostic
fields retained; removed max_burst_loss, mean_burst_loss,
interval_packets_recv, interval_bytes_recv.

Both report types use extensibility header: repurposed 3 reserved
bytes as [format_version:1][total_length:2 LE]. Decoders skip
unknown trailing bytes for forward compatibility. Session-layer
reports updated to match.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan f4278f1dd7 Add FMP node profile negotiation with mixed-profile integration tests 
NodeProfile enum (Full/NonRouting/Leaf) with FMP feature bitfield:
bits 0-2 profile, bits 3-6 MMP wants/provides, bit 7 bloom filter
size negotiable. Bloom size TLV always sent with min=max=1KB.

Config mapping: leaf_only -> Leaf, disable_routing -> NonRouting.
Profile and agreed bloom size stored on PeerConnection and
ActivePeer. Handshake msg2/msg3 carry FMP negotiation payload
with profile validation and bloom size agreement.

MMP report sending gated by profile wants/provides. Parent
selection and routing constraints skip non-full peers. One-way
bloom filters for non-routing peers (F inserts N as dependent).
Leaf mode: single-peer enforcement, suppress tree announces and
discovery forwarding.

Mixed-profile integration test: A(Full) + B(Full) + C(NonRouting)
+ D(Leaf) with 9 connectivity assertions.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 8357200b0e Remove IK/XK dead code, rename XX methods as sole pattern 
- Delete PROTOCOL_NAME_IK, PROTOCOL_NAME_XK and all IK/XK methods
- Remove Ik/Xk variants from NoisePattern enum
- Rename XX methods to drop xx_ prefix (sole pattern)
- Rename HandshakeMessageType variants from NoiseIKMsg1/NoiseIKMsg2
  to Msg1/Msg2, add Msg3 variant (0x03) for XX 3-message flow
- Fix doc comments with correct XX message sizes and descriptions
- Fix stale SessionSetup/SessionAck handshake payload size comments
 2026-04-11 08:16:01 +00:00
Johnathan Corgan ae0f791dbc Switch FSP handshake from Noise XK to XX 
Replace the 3-message XK handshake with XX for FSP session establishment.
XX requires no prior knowledge of the peer's static key — the responder's
identity is revealed in msg2, the initiator's in msg3.

Key changes:
- session.rs: XX initiator/responder, post-handshake identity verification
  using x-only key comparison (parity-independent for npub compatibility),
  negotiation payload in msg2/msg3 (FSP version [0,0], features=0)
- Rekey: switched from XK to XX for FSP rekey handshake
- timeout.rs: suppress msg1 resends when target peer is already promoted,
  preventing cross-connection session mismatch from duplicate handshakes
- Test template: discovery backoff 3s and handshake timeout 10s for
  faster convergence in integration tests
- Integration test timeouts restored to 45s (ping) and 60s (rekey)

Squashed commits:
- Switch FSP handshake from Noise XK to XX
- Fix integration test convergence by reducing discovery backoff
- Fix cross-connection session mismatch from msg1 resend
- Fix FSP identity verification parity mismatch
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 179689d6f2 Switch FMP handshake from Noise IK to XX with version negotiation 
Replace the 2-message IK handshake with a 3-message XX handshake for
FMP link establishment. XX requires no prior knowledge of the peer's
static key — both identities are revealed during the handshake
(responder in msg2, initiator in msg3). This is the foundation for
the forklift upgrade that enables rolling protocol upgrades.

Changes:
- Noise XX state machine alongside IK/XK (8 unit tests)
- Protocol negotiation payload codec: format byte, packed version
  min/max, 64-bit feature bitfield, TLV extensions (11 unit tests)
- FMP wire format version 0→1, msg3 header/builder, TCP stream framing
- FMP handshake switched to XX: PeerConnection 3-message flow,
  handle_msg1 simplified (no identity), handle_msg2 sends msg3 and
  promotes initiator, new handle_msg3 promotes responder with
  restart/rekey/cross-connection detection
- Rekey handshake switched to XX with negotiation payload hash chain
  fix (decrypt-and-discard in complete_rekey_msg2/msg3)
- Negotiation payload in msg2/msg3 (FMP version [1,1], features=0)
- Debug logging for handshake promotion paths
- Integration test convergence timeouts adjusted for extra round-trip

Squashed commits:
- Add Noise XX state machine alongside IK/XK
- Add protocol negotiation payload codec
- FMP wire format prep: version 1, msg3 header support
- Switch FMP handshake from Noise IK to XX
- Increase convergence timeouts for XX 3-message handshake
- Fix negotiation hash chain desync in rekey handshake
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 9ccaae5044 Create next branch for breaking changes (0.4.0-dev) 
- Bump version to 0.4.0-dev
- Add Breaking section to CHANGELOG above Unreleased
 2026-04-11 08:16:01 +00:00
382 changed files with 56553 additions and 24544 deletions
Expand all files Collapse all files
.config/nextest.toml
+17
View File
@@ -5,6 +5,23 @@ junit = { path = "junit.xml" }
# occasional msg1 under burst load even with the per-edge repair loop.
# Allow a retry rather than failing the whole CI run on a single
# dropped packet.
#
# Deliberately scoped to [profile.ci] and NOT applied locally, which makes
# the two gates disagree: the hosted runner retries a flaky test twice, the
# local sweep fails on the first failure. The asymmetry is intended and this
# is the record of why, since an undocumented one is indistinguishable from
# an oversight.
#
# It is here for shared-runner packet loss, a property of the hosted
# environment and not of the code. Applying it locally would suppress a real
# local flake, and a failure that only reproduces under load is a robustness
# bug to fix rather than to retry past. Keeping the local sweep strict is
# what makes it the sharper of the two gates.
#
# The cost, stated rather than hidden: a test that fails once and passes on
# retry is reported green here with no separate signal, so a genuine
# intermittent failure can be absorbed. If that starts mattering, the fix is
# to surface retried-but-passed tests, not to drop the retries.
retries = 2
[test-groups]
.git-blame-ignore-revs
+3
View File
@@ -3,3 +3,6 @@
# rustfmt master-only code
e9da598f8ab13de5dea3a1496531d675af6a0b94
# rustfmt noise-xx-only code
fef3d011ab290743a13ae3f9b287ad4ebaa1713b
.github/workflows/ci.yml
+88 -196
View File
@@ -38,12 +38,12 @@ env:
# unreliable on GitHub-hosted runners.
# tor-directory — same; live Tor dependency.
#
# Granularity-only differences (same coverage, different matrix shape —
# NOT a divergence):
# deb-install — split here into per-distro legs (debian12/debian13/
# ubuntu22/ubuntu24/ubuntu26) for parallelism; local runs the
# same distro set in one suite.
# dns-resolver — single leg here; runs all scenarios (same as local).
# The two runners express the same work in different matrix shapes, and the
# parity guard compares through that shape rather than around it: chaos legs
# are compared per scenario (and per flag) via their `scenario:` field,
# deb-install legs per distro. The one leg still compared at leg granularity
# is dns-resolver — a single leg here, running all of its scenarios
# internally, exactly as the local suite does.
# ─────────────────────────────────────────────────────────────────────────────
# ─────────────────────────────────────────────────────────────────────────────
@@ -52,6 +52,29 @@ env:
# Builds on Linux x86_64, Linux aarch64, and macOS.
# ─────────────────────────────────────────────────────────────────────────────
jobs:
ci-parity:
name: CI parity
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install Python deps
run: pip3 install --quiet pyyaml
- name: Check local and GitHub runners cover the same work
run: bash testing/check-ci-parity.sh
- name: Check test log matchers against the strings src/ emits
run: python3 testing/check-log-strings.py
- name: Check no tested function's exit status is a log call's
run: python3 testing/check-trailing-log.py
- name: Check nothing resolves the shared mutable test image
run: bash testing/check-image-scoping.sh
# Hermetic: synthetic ping functions, no containers, ~45s. Lives beside
# the other two so both runners gate on it identically — putting it in
# only one would create exactly the drift check-ci-parity.sh exists to
# catch, and it is invisible to that checker either way since it is not
# a matrix suite.
- name: Run convergence-gate unit tests
run: bash testing/lib/wait-converge-test.sh
fmt:
name: Format check
runs-on: ubuntu-latest
@@ -87,6 +110,59 @@ jobs:
restore-keys: |
${{ runner.os }}-cargo-
- run: cargo clippy --all-targets --all-features -- -D warnings
# An optional feature means two source trees, and --all-features lints
# only one of them. The default build is what ships, so lint it
# explicitly: without this stage, code that compiles only with
# `profiling` enabled would pass CI while breaking every release build.
# Mirrored in testing/ci-local.sh — check-ci-parity.sh compares
# integration suites only and will not catch a stage added to one runner
# and not the other.
- name: Clippy (default features)
run: cargo clippy --all-targets -- -D warnings
- name: Build with the tick-body profiler enabled
run: cargo build --workspace --features profiling
# ───────────────────────────────────────────────────────────────────────────
# Android cross-check
#
# FIPS runs on Android as an embedded library — the host app owns the TUN
# (an Android VpnService), so there are no daemon binaries to package, unlike
# the desktop targets. This job only cross-compiles the library for the
# android target to guard the android-only cfg paths (and the `not(android)`
# exclusions) from silently bit-rotting; nothing else in CI compiles them.
# cargo-ndk wires the NDK toolchain, which is required even for a check
# because `ring` compiles C at build time.
# ───────────────────────────────────────────────────────────────────────────
android-check:
name: Android cross-check (aarch64)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install Rust toolchain (+ Android target)
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
target: aarch64-linux-android
components: clippy
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v5
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-android-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Install cargo-ndk
uses: taiki-e/install-action@v2
with:
tool: cargo-ndk
- name: Clippy the library for Android
run: |
export ANDROID_NDK_HOME="${ANDROID_NDK_HOME:-$ANDROID_NDK_LATEST_HOME}"
cargo ndk -t arm64-v8a clippy --lib -- -D warnings
build:
name: Build (${{ matrix.os }})
@@ -228,6 +304,12 @@ jobs:
check_name: Unit Tests Summary
fail_on_failure: false
# The `profiling` feature adds a module, a recorder and a writer thread
# that the default-feature run above never compiles, so its own tests do
# not execute there. Mirrored in testing/ci-local.sh.
- name: Run library tests with the tick-body profiler enabled
run: cargo test --lib --features profiling
# ─────────────────────────────────────────────────────────────────────────────
# Job 2b – Unit tests (macOS)
# ─────────────────────────────────────────────────────────────────────────────
@@ -351,22 +433,6 @@ jobs:
- suite: static-chain
type: static
topology: chain
# ── Rekey integration test ──────────────────────────────────────────
- suite: rekey
type: rekey
topology: rekey
- suite: rekey-accept-off
type: rekey-accept-off
topology: rekey-accept-off
- suite: rekey-outbound-only
type: rekey-outbound-only
topology: rekey-outbound-only
# ── Inbound max_peers admission-cap test ───────────────────────
- suite: admission-cap
type: admission-cap
topology: mesh
- suite: acl-allowlist
type: acl-allowlist
# ── Firewall baseline (fips0 nftables default-deny) ────────────
- suite: firewall
type: firewall
@@ -375,9 +441,6 @@ jobs:
type: gateway
topology: gateway
# ── Chaos / stochastic scenarios ───────────────────────────────────
- suite: chaos-smoke-10
type: chaos
scenario: smoke-10
- suite: churn-mixed-10
type: chaos
scenario: churn-mixed
@@ -391,30 +454,9 @@ jobs:
- suite: tcp-mesh
type: chaos
scenario: tcp-mesh
- suite: bottleneck-parent
type: chaos
scenario: bottleneck-parent
- suite: cost-avoidance
type: chaos
scenario: cost-avoidance
- suite: cost-reeval
type: chaos
scenario: cost-reeval
- suite: cost-stability
type: chaos
scenario: cost-stability
- suite: depth-vs-cost
type: chaos
scenario: depth-vs-cost
- suite: mixed-technology
type: chaos
scenario: mixed-technology
- suite: congestion-stress
type: chaos
scenario: congestion-stress
- suite: bloom-storm
type: chaos
scenario: bloom-storm
# ── Sidecar deployment ──────────────────────────────────────────
- suite: sidecar
type: sidecar
@@ -527,120 +569,6 @@ jobs:
docker compose -f testing/static/docker-compose.yml \
--profile ${{ matrix.topology }} down --volumes --remove-orphans
# ── Rekey integration test ──────────────────────────────────────────────
- name: Generate and inject configs (rekey)
if: matrix.type == 'rekey'
run: |
bash testing/static/scripts/generate-configs.sh rekey
bash testing/static/scripts/rekey-test.sh inject-config
- name: Start containers (rekey)
if: matrix.type == 'rekey'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey up -d
- name: Run rekey test
if: matrix.type == 'rekey'
run: bash testing/static/scripts/rekey-test.sh
- name: Collect logs on failure (rekey)
if: matrix.type == 'rekey' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey logs --no-color
- name: Stop containers (rekey)
if: matrix.type == 'rekey' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey down --volumes --remove-orphans
# ── Rekey + accept_connections=false variant ──────────────────────────
- name: Generate and inject configs (rekey-accept-off)
if: matrix.type == 'rekey-accept-off'
env:
REKEY_TOPOLOGY: rekey-accept-off
REKEY_ACCEPT_OFF_NODES: b
run: |
bash testing/static/scripts/generate-configs.sh rekey-accept-off
bash testing/static/scripts/rekey-test.sh inject-config
- name: Start containers (rekey-accept-off)
if: matrix.type == 'rekey-accept-off'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-accept-off up -d
- name: Run rekey test (accept-off variant)
if: matrix.type == 'rekey-accept-off'
env:
REKEY_TOPOLOGY: rekey-accept-off
REKEY_ACCEPT_OFF_NODES: b
run: bash testing/static/scripts/rekey-test.sh
- name: Collect logs on failure (rekey-accept-off)
if: matrix.type == 'rekey-accept-off' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-accept-off logs --no-color | tail -300
- name: Stop containers (rekey-accept-off)
if: matrix.type == 'rekey-accept-off' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-accept-off down --volumes --remove-orphans
# ── Rekey + udp.outbound_only=true variant ─────────────────────────────
- name: Generate and inject configs (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only'
env:
REKEY_TOPOLOGY: rekey-outbound-only
REKEY_OUTBOUND_ONLY_NODES: b
run: |
bash testing/static/scripts/generate-configs.sh rekey-outbound-only
bash testing/static/scripts/rekey-test.sh inject-config
- name: Start containers (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-outbound-only up -d
- name: Run rekey test (outbound-only variant)
if: matrix.type == 'rekey-outbound-only'
env:
REKEY_TOPOLOGY: rekey-outbound-only
REKEY_OUTBOUND_ONLY_NODES: b
run: bash testing/static/scripts/rekey-test.sh
- name: Collect logs on failure (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-outbound-only logs --no-color | tail -300
- name: Stop containers (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-outbound-only down --volumes --remove-orphans
# ── ACL allowlist integration test ─────────────────────────────────────
- name: Run ACL allowlist integration test
if: matrix.type == 'acl-allowlist'
run: bash testing/acl-allowlist/test.sh --skip-build --keep-up
- name: Collect logs on failure (acl-allowlist)
if: matrix.type == 'acl-allowlist' && failure()
run: |
docker compose -f testing/acl-allowlist/docker-compose.yml logs --no-color
- name: Stop containers (acl-allowlist)
if: matrix.type == 'acl-allowlist' && always()
run: |
docker compose -f testing/acl-allowlist/docker-compose.yml down --volumes --remove-orphans
# ── Firewall baseline integration test ─────────────────────────────────
- name: Run firewall baseline integration test
if: matrix.type == 'firewall'
@@ -771,42 +699,6 @@ jobs:
docker compose -f testing/static/docker-compose.yml \
--profile gateway down --volumes --remove-orphans
# ── Inbound max_peers admission-cap integration test ────────────────
# Lowers node.max_peers on one mesh node and asserts the inbound cap
# holds under sustained retry pressure: denied peers keep retrying but
# are never promoted to an active session. The admission-cap-test.sh
# assertions are tailored per link-layer handshake variant; the leg
# itself is uniform. Static-style harness on the shared mesh profile.
- name: Generate configs (admission-cap)
if: matrix.type == 'admission-cap'
run: bash testing/static/scripts/generate-configs.sh mesh
- name: Inject admission-cap config (admission-cap)
if: matrix.type == 'admission-cap'
run: bash testing/static/scripts/admission-cap-test.sh inject-config
- name: Start containers (admission-cap)
if: matrix.type == 'admission-cap'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile mesh up -d
- name: Run admission-cap test
if: matrix.type == 'admission-cap'
run: bash testing/static/scripts/admission-cap-test.sh
- name: Collect logs on failure (admission-cap)
if: matrix.type == 'admission-cap' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile mesh logs --no-color | tail -300
- name: Stop containers (admission-cap)
if: matrix.type == 'admission-cap' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile mesh down --volumes --remove-orphans
# ── Real-deb install integration ────────────────────────────────────
# The deb-install harness builds its own .deb from source in a
# cargo-deb builder image; the pre-built Linux binary from the
.github/workflows/package-openwrt.yml
+5
View File
@@ -285,6 +285,8 @@ jobs:
"$FILES_DIR/etc/fips/firewall.sh"
"$FILES_DIR/etc/hotplug.d/net/99-fips"
"$FILES_DIR/etc/uci-defaults/90-fips-setup"
"$FILES_DIR/usr/bin/fips-mesh-setup"
"$FILES_DIR/usr/bin/fips-ap-setup"
)
fail=0
for f in "${TARGETS[@]}"; do
@@ -404,6 +406,8 @@ jobs:
./usr/bin/fipsctl
./usr/bin/fipstop
./usr/bin/fips-gateway
./usr/bin/fips-mesh-setup
./usr/bin/fips-ap-setup
./etc/init.d/fips
./etc/init.d/fips-gateway
./etc/fips/fips.yaml
@@ -717,6 +721,7 @@ jobs:
for path in \
usr/bin/fips usr/bin/fipsctl usr/bin/fipstop usr/bin/fips-gateway \
usr/bin/fips-mesh-setup usr/bin/fips-ap-setup \
etc/init.d/fips etc/init.d/fips-gateway \
etc/fips/fips.yaml etc/fips/firewall.sh etc/dnsmasq.d/fips.conf \
etc/sysctl.d/fips-gateway.conf etc/sysctl.d/fips-bridge.conf \
.gitignore
+5
View File
@@ -33,6 +33,11 @@ __pycache__/
*.egg-info/
*.egg
# Per-run build contexts created by testing/ci-local.sh. Its teardown normally
# removes them, but the CI worker's SIGKILL runs no trap, so one can survive a
# preempted run; ci-cleanup.sh sweeps the survivors.
/testing/docker-*/
# Runtime artifacts from running fips in-tree during local testing.
# Root-anchored so legitimately-tracked fips.yaml under packaging/ and
# examples/ stays included.
CHANGELOG.md
+252
View File
@@ -5,14 +5,266 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## Breaking
Wire-format breaking changes for v0.6.0. All nodes in a mesh must
run the same major version — these changes are not backward compatible
with v0.4.x or earlier peers.
### Changed
#### Noise XX Handshake (FMP and FSP)
- FMP link handshake switched from Noise IK (2 messages) to Noise XX
(3 messages). Neither side requires prior knowledge of the peer's
static key. Responder identity revealed in msg2, initiator in msg3.
FMP wire version incremented to 1.
- FSP session handshake switched from Noise XK to Noise XX. Same
3-message flow with post-handshake identity verification using
x-only key comparison (parity-independent for npub compatibility).
- Protocol negotiation payload added to XX msg2/msg3 for both layers:
format byte, packed version min/max, 64-bit feature bitfield, and
forward-compatible TLV extensions. Enables rolling protocol upgrades
in future releases.
- FMP msg1 reduced from 106 to 33 bytes (ephemeral key only, no
encrypted static key or DH products).
#### FMP Node Profiles
- Node profile enum (Full, NonRouting, Leaf) advertised in FMP feature
bitfield bits 0-2. At least one side of a link must be Full.
- MMP report flow gated by wants/provides bits (bits 3-6): reports
only sent when the sender can provide and the receiver wants them.
- Non-routing nodes receive bloom filters (one-way) but do not send
them; the full peer inserts their identity as a leaf dependent.
- Leaf nodes enforce single-peer constraint with no tree, bloom, or
transit participation.
#### MMP Report Format
- Spin bit removed. Reclaims FMP flags bit 2 and FSP inner flags
bit 0. Superseded by MMP receiver report timestamp echo for RTT.
- SenderReport reduced from 48 to 20 bytes (3 fields: interval
packets/bytes sent, cumulative packets sent).
- ReceiverReport reduced from 68 to 54 bytes (10 fields retained;
removed max/mean burst loss and interval recv counters).
- Both report types use extensibility header: `[format_version:1]
[total_length:2 LE]` replacing reserved bytes. Decoders skip
unknown trailing bytes for forward compatibility.
#### Discovery Wire Format
- Dropped `origin_coords` from LookupRequest (saves 2 + 16*depth
bytes per request). Reverse-path routing via `recent_requests` is
the primary response mechanism.
- `min_mtu` field wired up in LookupRequest: transit nodes skip peers
whose link MTU is below the request's minimum.
- TLV extension section added to LookupRequest and LookupResponse
after fixed fields. Transit nodes forward TLV bytes verbatim.
#### Nostr-Discovery Advert Namespace
- Default Nostr-discovery advert namespace bumped from
`fips-overlay-v1` to `fips-overlay-v1-next` on the `next` branch.
Master continues to publish under `fips-overlay-v1`. Effect: a
stock `next`-branch daemon's open-discovery sweep no longer
discovers `master` peers, and vice versa — eliminating the
cross-version retraversal storms that arise when both sides
punched a UDP socket via Nostr but cannot complete an FMP
handshake. Operators who genuinely want cross-branch reach (e.g.
during a coordinated rolling upgrade) can override per-daemon
via `node.discovery.nostr.app` in `fips.yaml`. The
`protocol_mismatch_cooldown_secs` defense-in-depth on master is
the safety net against any peer that bypasses this default
(config override, future fork, static-peer config).
#### Shared-Media Beacons
- Ethernet frame header unified to 4 bytes `[type][flags][length:2
LE]` for all frame types. Beacons reduced from 34 to 5 bytes
(pubkey stripped — identity learned from XX handshake).
- BLE pre-handshake pubkey exchange removed. Cross-probe tie-breaker
eliminated (unnecessary with XX).
#### Bloom Filter Wire Format
- FilterAnnounce gains flags byte (delta bit), `base_seq` field, and
RLE-compressed payload for XOR-diff delta compression.
- New FilterNack message type (0x21) for out-of-sequence delta
recovery (triggers full retransmit).
- Filter size decoupled from FMP negotiation: announced dynamically in
filter updates. Bit 7 and TLV field 1 removed from handshake.
- Variable filter sizes (512 bytes to 32 KB) with adaptive sizing
based on outgoing fill ratio (step-up at 20%, step-down at 5%).
## [Unreleased]
### Added
- The receive-path `RejectReason` classification (shipped in 0.4.0) is
additionally wired into the Noise XX handshake cluster
(msg1/msg2/msg3) and the rekey-initiator outbound sites on `next`.
- An optional tick-body profiler behind the new `profiling` Cargo feature,
**off by default**. When enabled, `fipsctl profile tick on [--dir PATH]` /
`off` / `status` starts and stops a capture at runtime with no restart. Each
capture writes one tab-separated file (default `/var/log/fips`, capped at
32 MB) carrying, per ten-second interval, the exact count, max and total for
every step of the rx-loop tick arm, the whole-tick span, and gauges for ticks,
peer count, the gap between successive tick-arm entries and the resulting
arm-starvation delay. With the feature off the instrumentation macro is a pure
pass-through, so a default build contains no timing code on the tick path.
`LogsDirectory=fips` was added to the packaged systemd units so the capture
directory is created and cleaned up declaratively.
- `packaging/debian/build-deb.sh --features <list>` builds the `.deb` with a
Cargo feature list, which is how an instrumented package is produced for a
measurement run. The auto-derived dev Version gains a matching `+<features>`
marker, so a feature build and a default build of the same commit are no
longer indistinguishable: without it the two carry byte-identical versions,
an install of one over the other is an apt no-op, and the running node offers
no way to tell which one it has. The marker sorts above the unmarked build, so
installing a feature build is an upgrade and reverting to the default build is
a downgrade — revert with `dpkg -i` rather than `apt install`. `--features` is
refused together with `--no-build`, which would stamp the marker onto binaries
the features never reached.
### Changed
- Rekey timer jitter is enabled on next's XX FMP rekey path
(`REKEY_JITTER_SECS = 15` at `src/node/mod.rs`), matching the
IK-line behavior on maint/master. It had been temporarily set to
`0` on next because variable-interval rekeys exposed three XX
rekey-path defects that left the two endpoints on divergent Noise
sessions; those defects are fixed (see `### Fixed`), so the
per-session signed jitter over `[-15, +15]` seconds is restored.
`node.rekey.after_secs` is the nominal interval rather than a floor;
mean is preserved.
- On the XX FMP handshake, an over-cap inbound connection is rejected
solely by the late `promote_connection` check, and the resulting
`MaxPeersExceeded` rejection is logged at debug rather than warn so a
saturated node under sustained inbound pressure does not emit WARN
spam for these expected policy rejections. There is no early cap gate:
on XX the peer's identity is not known until the third handshake
message, by which point Msg1, Msg2, and Msg3 have all crossed the
wire, so an early gate would save no wire bytes and would govern
exactly the same net-new-peer set as the late check. The known-peer /
cross-connection bypass — which also covers peers the node is itself
dialing, e.g. configured `auto_connect` peers — is handled by that
late check, since those peers return earlier via the cross-connection
paths and are not subject to the cap.
- The Ethernet transport's per-interface `discovery` flag was renamed to
`listen` (`transports.ethernet.*`) to match the symmetric `announce`
(transmit) / `listen` (receive) neighbor-beacon vocabulary. The old
`discovery:` key is still accepted via a serde alias, so deployed configs
continue to load unchanged; `Config::to_yaml()` re-emits it under the
canonical `listen:` name. Update your `fips.yaml` to `listen:`.
- The mesh-lookup control-metrics family is now emitted under the key
`lookup` in `fipsctl stats metrics` and `show routing`. The former key
`discovery` is still emitted as a deprecated alias carrying identical
counters; update dashboards and alerts to read `lookup`.
- The overloaded `node.discovery.*` config table was split into
`node.lookup.*` (mesh-lookup scalars: `ttl`, `attempt_timeouts_secs`,
`recent_expiry_secs`, `backoff_base_secs`, `backoff_max_secs`,
`forward_min_interval_secs`) and `node.rendezvous.*` (peer rendezvous:
`nostr.*`, `lan.*`). A deployed `node.discovery:` block still loads and is
folded into the new tables with a one-time deprecation warning; migrate your
`fips.yaml` to the new keys.
- `SessionDatagram::decrement_ttl` and `SessionDatagram::can_forward` now match
the forwarder's IP hop-limit semantics: `decrement_ttl` decrements first and
reports false when the result is zero, and `can_forward` is true only at a
TTL of 2 or more.
### Deprecated
- The `discovery` metric-family key (control-socket JSON). It is dual-emitted
alongside the new `lookup` key during a migration window and will be removed.
Migrate dashboards/alerts from `discovery.*` to `lookup.*`.
- The `node.discovery.*` config table. Its keys were split into `node.lookup.*`
(mesh-lookup) and `node.rendezvous.*` (peer rendezvous). A legacy
`node.discovery:` block still applies for now with a deprecation warning and
will be removed; migrate to `node.lookup.*` / `node.rendezvous.*`.
- The Ethernet `transports.ethernet.discovery` flag, renamed to
`transports.ethernet.listen`. The old key is still accepted via a serde
alias and will be removed at the v2 cutover; migrate to `listen`.
### Fixed
- Nostr NAT traversal no longer breaks after the host suspends. The traversal
clock cached a Unix timestamp once at startup and advanced it with a
monotonic `Instant`, which does not tick while a machine is asleep, so after
a suspend the daemon's idea of the time trailed real time by the suspend
duration for the rest of the process lifetime. Every NIP-40 expiration it
computed was therefore published already in the past: relays dropped the
offers as expired, the initiator logged a signal timeout waiting for an
answer, and traversal stayed broken until the daemon was restarted. The
clock now reads the wall clock on every call. This is not macOS-specific,
though a laptop that sleeps is where it is easiest to hit; any host that
suspends or hibernates was affected. Reported in
[#128](https://github.com/jmcorgan/fips/issues/128).
- XX FMP rekey no longer diverges under timer jitter, which unblocked
re-enabling the rekey jitter on next (`REKEY_JITTER_SECS = 15`; see
`### Changed`). With jitter the two directions of a link rekey close
together in time, and three defects specific to the XX three-message
rekey state machine could each leave the endpoints committed to
different Noise sessions — silent session divergence that starved the
receiver into ~50% post-rekey ping loss and a 30-second heartbeat
link-dead teardown (tree parent loss, routing failure) while every
crypto, transport, and link-state gate stayed green. All three are
fixed:
- The K-bit-flip handler promoted whatever pending session existed the
instant the header bit flipped, which under interleaved rekeys could
be a stale pending from an earlier epoch. It now trial-decrypts the
inbound frame against the pending session and promotes only on an
authenticated decrypt, delivering that plaintext through the
canonical path and leaving the pending untouched otherwise — the
same cutover discipline used on FSP.
- The FMP rekey msg3 was sent once, so a lost datagram left the
responder without the new session. The msg3 payload is now retained
and retransmitted over the existing link until a peer frame
authenticates against the pending or post-cutover current session,
abandoning after the configured handshake-resend budget. Per-link
rekeys are also serialized: a new rekey does not start while one
awaits cutover or is still retransmitting msg3.
- The `handle_msg3` cross-connection and rekey-responder paths were
partitioned by a fixed 30-second session-age threshold, but a rekey
resets the session-age clock, so under jitter a rekey-aged msg3 was
frequently under 30 seconds and got swallowed by the
initial-handshake cross-connection branch, which discarded the
peer's rekey session with no pending slot while the peer cut over to
it anyway. The cross-connection branch is now bounded by the same
jitter-aware session-age floor the rekey responder uses, so the two
paths partition with no overlap. At zero jitter the floor equals the
previous 30-second constant, so default-cadence behavior is
unchanged.
- XX rekey dual-initiation race that broke six pair-directions
post-rekey when both endpoints initiated rekey simultaneously.
The `handle_msg3` tie-breaker only fired when `rekey_in_progress`
was still true, but XX's three-message handshake lets both sides
clear that flag (via `set_pending_session`) before either's msg3
lands. The drop-on-pending-session guard then silently discarded
the peer's msg3, each side cut over to its own initiator session,
and the link broke asymmetrically. The tie-breaker now also fires
when `pending_new_session().is_some()`, applying the same
smaller-NodeAddr resolution rule. Mirrored to the FSP rekey msg1
path for symmetry.
- `SessionDatagram` hop-limit handling now follows IP semantics. Delivery to
the addressed node is no longer TTL-gated, and a forwarder decrements before
deciding rather than after, so a datagram that would leave with a TTL of zero
is dropped instead of transmitted. Previously the TTL check ran ahead of the
local-delivery test, so a datagram addressed to this node that arrived with
TTL 0 was dropped, and a forwarder receiving a transit datagram at TTL 1
transmitted it at TTL 0 for the next hop to discard, wasting one transmission
per expiring datagram. The reachable radius is unchanged, because the two
behaviors compensated exactly: a path of `h` links still delivers for any
source TTL of `h` or more. During a rolling upgrade, an unupgraded forwarder
feeding an upgraded destination delivers one hop further than either version
does on its own; no version mix delivers less far. The `TtlExhausted` reject
counter now charges at the node that makes the decision rather than at the
hop after it.
## [0.4.1] - 2026-07-19
### Changed
Cargo.lock
Generated
+2 -1
View File
@@ -1074,7 +1074,7 @@ checksum = "9844ddc3a6e533d62bba727eb6c28b5d360921d5175e9ff0f1e621a5c590a4d5"
[[package]]
name = "fips"
version = "0.4.1"
version = "0.6.0-dev"
dependencies = [
"arc-swap",
"bech32",
@@ -1087,6 +1087,7 @@ dependencies = [
"hex",
"hkdf",
"libc",
"libm",
"mdns-sd",
"nostr",
"nostr-sdk",
Cargo.toml
+15 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "fips"
version = "0.4.1"
version = "0.6.0-dev"
edition = "2024"
description = "A distributed, decentralized network routing protocol for mesh nodes connecting over arbitrary transports"
license = "MIT"
@@ -11,12 +11,21 @@ readme = "README.md"
keywords = ["mesh", "p2p", "decentralized", "overlay-network", "nostr"]
categories = ["network-programming", "command-line-utilities", "cryptography"]
[features]
default = []
# Tick-body profiler (`src/instr`). Off by default: enabling it edits 26 call
# sites in the rx loop's hot tick arm, and only a compile-time gate makes the
# default build's neutrality a property of the generated code rather than of a
# runtime check. Build with `--features profiling` for a measurement run.
profiling = []
[dependencies]
ratatui = "0.30"
secp256k1 = { version = "0.30", features = ["rand", "global-context"] }
sha2 = "0.10"
hkdf = "0.12"
ring = "0.17"
libm = "0.2"
rand = "0.10.1"
crossbeam-channel = "0.5"
thiserror = "2.0"
@@ -108,3 +117,8 @@ path = "src/bin/fips-gateway.rs"
[[bin]]
name = "fipstop"
path = "src/bin/fipstop/main.rs"
[[bench]]
name = "routing_next_hop"
path = "benches/routing_next_hop.rs"
harness = false
README.md
+29 -25
View File
@@ -3,7 +3,7 @@
![banner](docs/logos/fips_banner.png)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-1.85%2B-orange.svg)](https://www.rust-lang.org/)
[![Status](https://img.shields.io/badge/status-v0.4.1-green.svg)](#status--roadmap)
[![Status](https://img.shields.io/badge/status-v0.6.0--dev-green.svg)](#status--roadmap)
A self-organizing encrypted mesh network built on Nostr identities,
capable of operating over arbitrary transports without central
@@ -43,9 +43,9 @@ same way it would on a local network.
- **Multi-transport.** UDP, TCP, Ethernet, Tor, Nym, and Bluetooth
(BLE L2CAP) ship today; transports compose on a single mesh and a
node may run several at once.
- **Two-layer encryption.** Noise IK between peers (hop-by-hop) and
Noise XK between mesh endpoints (independent end-to-end), with
periodic rekey for forward secrecy.
- **Two-layer encryption.** Noise XX both hop-by-hop (peer links)
and end-to-end (mesh sessions), with periodic rekey for forward
secrecy and protocol negotiation in the handshake.
- **Nostr-native identity.** secp256k1 / schnorr keypairs as node
addresses; self-generated, no registration, no central authority.
- **IPv6 adapter.** A TUN interface maps each remote npub to an
@@ -112,17 +112,19 @@ tutorial progression starting at
cargo build --release
```
Requires Rust 1.94.1+ (edition 2024). Linux, macOS, and Windows are
supported; transport availability varies by platform.
Requires Rust 1.94.1+ (edition 2024). Linux, macOS, and Windows run as
standalone daemons; Android is supported as an embedded library (the host
app owns the TUN, e.g. a `VpnService`). Transport availability varies by
platform.
| Transport | Linux | macOS | Windows | OpenWrt |
|-----------|:-----:|:-----:|:-------:|:-------:|
| UDP | ✅ | ✅ | ✅ | ✅ |
| TCP | ✅ | ✅ | ✅ | ✅ |
| Ethernet | ✅ | ✅ | ❌ | ✅ |
| Tor | ✅ | ✅ | ✅ | ✅ |
| Nym | ✅ | ✅ | ✅ | ❌ |
| BLE | ✅ | ❌ | ❌ | ❌ |
| Transport | Linux | macOS | Windows | Android | OpenWrt |
|-----------|:-----:|:-----:|:-------:|:-------:|:-------:|
| UDP | ✅ | ✅ | ✅ | ✅ | ✅ |
| TCP | ✅ | ✅ | ✅ | ✅ | ✅ |
| Ethernet | ✅ | ✅ | ❌ | ❌ | ✅ |
| Tor | ✅ | ✅ | ✅ | ❌ | ✅ |
| Nym | ✅ | ✅ | ✅ | ❌ | ❌ |
| BLE | ✅ | ❌ | ❌ | ❌ | ❌ |
On Linux, a source build requires `libclang` — the LAN gateway's
nftables bindings are generated by `bindgen` at build time, which
@@ -210,24 +212,26 @@ testing/ Docker-based integration test harnesses + chaos simulation
## Status & roadmap
FIPS is at **v0.4.1** on the `maint` branch.
[v0.4.1](https://github.com/jmcorgan/fips/releases/tag/v0.4.1) has
shipped; this line carries patch-level fixes for the 0.4.x series. The
core protocol works end-to-end over
FIPS is at **v0.6.0-dev** on the `next` branch.
[v0.4.1](https://github.com/jmcorgan/fips/releases/tag/v0.4.1)
has shipped from `master`; this development line carries
wire-format-breaking work for v0.6.0 — unified Noise XX handshake
at both layers, FMP node profiles, slimmer MMP reports, and an
extensible bloom-filter encoding — that will not interoperate with
v0.2.x, v0.3.x, or v0.4.x peers. The core protocol works end-to-end over
UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test
mesh of thousands of nodes. v0.4.0 added the Nym mixnet transport and
mDNS LAN discovery alongside the existing Nostr-mediated peer discovery,
UDP NAT traversal, peer ACL, and packaging hardening. New wire-format work
continues to be staged on the `next` branch for the subsequent
release line.
mesh of thousands of nodes. See the CHANGELOG `## Breaking` section for the
full list of v0.6.0 wire-format changes in flight.
### What works today
- Spanning-tree construction with greedy coordinate routing.
- Bloom-filter-guided destination discovery (no flooding,
single-path with retry).
- Two-layer Noise encryption (IK at the link, XK at the session)
with periodic hitless rekey for forward secrecy at both layers.
- Two-layer Noise XX encryption (hop-by-hop at the link layer and
end-to-end at the session layer) with periodic hitless rekey for
forward secrecy at both layers and protocol negotiation in the
handshake.
- Persistent or ephemeral node identity with key-file management.
- IPv6 TUN adapter with built-in `.fips` DNS resolver and
multi-backend auto-configuration (systemd dns-delegate,
benches/routing_next_hop.rs
+365
View File
@@ -0,0 +1,365 @@
//! Micro-benchmark quantifying the per-forwarded-packet heap-allocation cost
//! of the routing next-hop candidate-assembly path.
//!
//! `find_next_hop` runs once per forwarded data packet. Its sans-IO core
//! assembles a `Vec<Candidate>` by enumerating every peer through the
//! `RoutingView` seam: `peer_addrs()` materializes a `Vec<NodeAddr>` of all
//! peers, the survivors are snapshotted (each cloning its `TreeCoordinate`),
//! and the result is collected into a second `Vec`. This bench measures that
//! per-call allocation against a fused zero-alloc reference that iterates the
//! peer map directly and borrows coordinates instead of cloning.
//!
//! Visibility caveat: the production `routing_candidates` / `select_best_candidate`
//! / `RoutingView` / `Candidate` are `pub(crate)` (src/proto/routing/core.rs)
//! and are not re-exported at the crate root, so an external bench crate cannot
//! name them. Rather than change production visibility, this file reproduces
//! that path verbatim over the real public `NodeAddr` / `TreeCoordinate` /
//! `CoordEntry` / `BloomFilter` types with the same iterator chain and the same
//! `HashMap`-backed view the shell uses (src/node/mod.rs NodeRoutingView). The
//! allocation behavior is therefore identical to production by construction;
//! only the symbol identity differs.
use std::alloc::{GlobalAlloc, Layout, System};
use std::collections::HashMap;
use std::hint::black_box;
use std::sync::atomic::{AtomicUsize, Ordering};
use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
use fips::{BloomFilter, NodeAddr, TreeCoordinate};
// ---------------------------------------------------------------------------
// Counting global allocator: bumps a process-global counter on every heap
// allocation operation (alloc / alloc_zeroed / realloc). Sampled tightly and
// single-threaded in `report_allocs` so no unrelated allocations are captured.
// ---------------------------------------------------------------------------
struct CountingAlloc;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);
unsafe impl GlobalAlloc for CountingAlloc {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
ALLOCS.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc(layout) }
}
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
unsafe { System.dealloc(ptr, layout) }
}
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
ALLOCS.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc_zeroed(layout) }
}
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
ALLOCS.fetch_add(1, Ordering::Relaxed);
unsafe { System.realloc(ptr, layout, new_size) }
}
}
#[global_allocator]
static GLOBAL: CountingAlloc = CountingAlloc;
const PEER_COUNTS: [usize; 4] = [8, 32, 128, 256];
/// Fraction of peers whose bloom filter reports the destination reachable.
const REACH_NUMERATOR: usize = 1;
const REACH_DENOMINATOR: usize = 2;
/// Tree depth for synthetic coordinates (self..root), a realistic mesh depth.
const COORD_DEPTH: usize = 8;
// ---------------------------------------------------------------------------
// Reproduction of the pub(crate) routing seam (src/proto/routing/core.rs).
// ---------------------------------------------------------------------------
trait RoutingView {
fn peer_addrs(&self) -> Vec<NodeAddr>;
fn peer_may_reach(&self, peer: &NodeAddr, dest: &NodeAddr) -> bool;
fn peer_can_send(&self, peer: &NodeAddr) -> bool;
fn peer_link_cost(&self, peer: &NodeAddr) -> f64;
fn peer_coords(&self, peer: &NodeAddr) -> Option<TreeCoordinate>;
}
struct Candidate {
addr: NodeAddr,
can_send: bool,
link_cost: f64,
coords: Option<TreeCoordinate>,
}
/// Verbatim from `routing::routing_candidates` (core.rs). Allocates the
/// `peer_addrs` Vec, clones each survivor's coords, and collects into a Vec.
fn routing_candidates(rv: &impl RoutingView, dest: &NodeAddr) -> Vec<Candidate> {
rv.peer_addrs()
.into_iter()
.filter(|peer| rv.peer_may_reach(peer, dest))
.map(|peer| Candidate {
can_send: rv.peer_can_send(&peer),
link_cost: rv.peer_link_cost(&peer),
coords: rv.peer_coords(&peer),
addr: peer,
})
.collect()
}
/// Verbatim from `routing::select_best_candidate` (core.rs). Pure, no alloc.
fn select_best_candidate(
candidates: &[Candidate],
dest_coords: &TreeCoordinate,
my_coords: &TreeCoordinate,
) -> Option<NodeAddr> {
let my_distance = my_coords.distance_to(dest_coords);
let mut best: Option<(&Candidate, f64, usize)> = None;
for candidate in candidates {
if !candidate.can_send {
continue;
}
let cost = candidate.link_cost;
let dist = candidate
.coords
.as_ref()
.map(|pc| pc.distance_to(dest_coords))
.unwrap_or(usize::MAX);
if dist >= my_distance {
continue;
}
let dominated = match &best {
None => true,
Some((_, best_cost, best_dist)) => {
cost < *best_cost
|| (cost == *best_cost && dist < *best_dist)
|| (cost == *best_cost
&& dist == *best_dist
&& candidate.addr < best.as_ref().unwrap().0.addr)
}
};
if dominated {
best = Some((candidate, cost, dist));
}
}
best.map(|(candidate, _, _)| candidate.addr)
}
// ---------------------------------------------------------------------------
// Bench-local view, HashMap-backed exactly like src/node/mod.rs NodeRoutingView.
// ---------------------------------------------------------------------------
struct BenchPeer {
bloom: BloomFilter,
can_send: bool,
link_cost: f64,
}
struct BenchView {
peers: HashMap<NodeAddr, BenchPeer>,
coords: HashMap<NodeAddr, TreeCoordinate>,
}
impl RoutingView for BenchView {
fn peer_addrs(&self) -> Vec<NodeAddr> {
self.peers.keys().copied().collect()
}
fn peer_may_reach(&self, peer: &NodeAddr, dest: &NodeAddr) -> bool {
self.peers.get(peer).is_some_and(|p| p.bloom.contains(dest))
}
fn peer_can_send(&self, peer: &NodeAddr) -> bool {
self.peers.get(peer).is_some_and(|p| p.can_send)
}
fn peer_link_cost(&self, peer: &NodeAddr) -> f64 {
self.peers.get(peer).map_or(f64::INFINITY, |p| p.link_cost)
}
fn peer_coords(&self, peer: &NodeAddr) -> Option<TreeCoordinate> {
self.coords.get(peer).cloned()
}
}
/// Zero-alloc reference: what an iterator/visitor seam would do. Iterates the
/// peer map directly, fuses the may_reach + can_send filters, borrows coords
/// instead of cloning, and tracks the best hop inline. No Vec, no coord clone.
fn resolve_next_hop_zeroalloc(
view: &BenchView,
dest: &NodeAddr,
dest_coords: &TreeCoordinate,
my_coords: &TreeCoordinate,
) -> Option<NodeAddr> {
let my_distance = my_coords.distance_to(dest_coords);
let mut best: Option<(NodeAddr, f64, usize)> = None;
for (addr, peer) in &view.peers {
if !peer.bloom.contains(dest) {
continue;
}
if !peer.can_send {
continue;
}
let cost = peer.link_cost;
let dist = view
.coords
.get(addr)
.map(|pc| pc.distance_to(dest_coords))
.unwrap_or(usize::MAX);
if dist >= my_distance {
continue;
}
let dominated = match &best {
None => true,
Some((best_addr, best_cost, best_dist)) => {
cost < *best_cost
|| (cost == *best_cost && dist < *best_dist)
|| (cost == *best_cost && dist == *best_dist && *addr < *best_addr)
}
};
if dominated {
best = Some((*addr, cost, dist));
}
}
best.map(|(addr, _, _)| addr)
}
// ---------------------------------------------------------------------------
// Scenario construction.
// ---------------------------------------------------------------------------
fn addr(tag: u8, i: u16) -> NodeAddr {
let mut b = [0u8; 16];
b[0] = tag;
b[1..3].copy_from_slice(&i.to_le_bytes());
NodeAddr::from_bytes(b)
}
/// A depth-`COORD_DEPTH` coordinate whose leaf is `leaf`, sharing a fixed
/// interior path and root with `shared_tag`. Peers built with the dest's
/// shared_tag sit close to the destination (distance 2); a distinct shared_tag
/// sits far (near the root), modeling our own position.
fn coord(leaf: NodeAddr, shared_tag: u8) -> TreeCoordinate {
let mut path = Vec::with_capacity(COORD_DEPTH);
path.push(leaf);
for level in 1..(COORD_DEPTH - 1) {
path.push(addr(shared_tag, level as u16));
}
path.push(addr(9, 0)); // common root
TreeCoordinate::from_addrs(path).expect("valid coord path")
}
struct Scenario {
view: BenchView,
dest: NodeAddr,
dest_coords: TreeCoordinate,
my_coords: TreeCoordinate,
}
impl Scenario {
fn new(n: usize) -> Self {
let dest = addr(2, 0);
// Destination path uses interior tag 4; peers reuse tag 4 so survivors
// are close to the destination. Our own coords use tag 5 (far).
let dest_coords = coord(dest, 4);
let my_coords = coord(addr(6, 0), 5);
let mut peers = HashMap::new();
let mut coords = HashMap::new();
for i in 0..n {
let paddr = addr(1, i as u16);
let mut bloom = BloomFilter::new();
// Realistic fill: a handful of unrelated reachable addrs.
for f in 0..4u16 {
bloom.insert(&addr(7, i as u16 * 4 + f));
}
// A controlled fraction advertise the destination as reachable.
if (i % REACH_DENOMINATOR) < REACH_NUMERATOR {
bloom.insert(&dest);
}
peers.insert(
paddr,
BenchPeer {
bloom,
can_send: true,
link_cost: 1.0 + (i as f64) * 0.01,
},
);
// Peers share the destination's interior path (tag 4) → close.
coords.insert(paddr, coord(paddr, 4));
}
Self {
view: BenchView { peers, coords },
dest,
dest_coords,
my_coords,
}
}
fn survivors(&self) -> usize {
self.view
.peers
.values()
.filter(|p| p.bloom.contains(&self.dest))
.count()
}
}
// ---------------------------------------------------------------------------
// Allocation-per-call report (printed once, before criterion timing).
// ---------------------------------------------------------------------------
fn count_allocs<T>(iters: usize, mut f: impl FnMut() -> T) -> f64 {
for _ in 0..8 {
black_box(f());
}
let start = ALLOCS.load(Ordering::Relaxed);
for _ in 0..iters {
black_box(f());
}
let end = ALLOCS.load(Ordering::Relaxed);
(end - start) as f64 / iters as f64
}
fn report_allocs() {
const ITERS: usize = 2000;
println!("\n=== allocations per call (heap alloc ops: alloc+alloc_zeroed+realloc) ===");
println!(
"{:>6} {:>10} {:>16} {:>16}",
"peers", "survivors", "current/call", "zero-alloc/call"
);
for &n in &PEER_COUNTS {
let s = Scenario::new(n);
let survivors = s.survivors();
let current = count_allocs(ITERS, || {
let cands = routing_candidates(&s.view, &s.dest);
select_best_candidate(&cands, &s.dest_coords, &s.my_coords)
});
let zero = count_allocs(ITERS, || {
resolve_next_hop_zeroalloc(&s.view, &s.dest, &s.dest_coords, &s.my_coords)
});
println!("{n:>6} {survivors:>10} {current:>16.2} {zero:>16.2}");
}
println!();
}
fn bench_next_hop(c: &mut Criterion) {
report_allocs();
let mut group = c.benchmark_group("find_next_hop");
for &n in &PEER_COUNTS {
let scenario = Scenario::new(n);
group.bench_with_input(BenchmarkId::new("current_alloc", n), &n, |b, _| {
b.iter(|| {
let cands = routing_candidates(&scenario.view, &scenario.dest);
black_box(select_best_candidate(
&cands,
&scenario.dest_coords,
&scenario.my_coords,
))
});
});
group.bench_with_input(BenchmarkId::new("zero_alloc_ref", n), &n, |b, _| {
b.iter(|| {
black_box(resolve_next_hop_zeroalloc(
&scenario.view,
&scenario.dest,
&scenario.dest_coords,
&scenario.my_coords,
))
});
});
}
group.finish();
}
criterion_group! {
name = benches;
config = Criterion::default().sample_size(50);
targets = bench_next_hop
}
criterion_main!(benches);
docs/design/fips-architecture.md
+15 -17
View File
@@ -44,7 +44,7 @@ See [fips-transport-layer.md](fips-transport-layer.md) for the
transport layer specification.
**FIPS Mesh Protocol (FMP)**: Manages peer connections, authenticates
peers via Noise IK handshakes, and encrypts all traffic on each link.
peers via Noise XX handshakes, and encrypts all traffic on each link.
FMP is where the mesh organizes itself — nodes exchange spanning tree
announcements and bloom filters with their direct peers, and FMP
makes forwarding decisions for transit traffic. FMP provides
@@ -116,8 +116,8 @@ three are deterministically derived from the same keypair.
![Identity Derivation](diagrams/fips-identity-derivation.svg)
The pubkey is the node's cryptographic identity, used in Noise
handshakes for both link encryption (IK) and session encryption (XK).
The pubkey is the node's cryptographic identity, used in Noise XX
handshakes for both link encryption and session encryption.
It is never exposed beyond the endpoints of an encrypted channel. The node_addr, a one-way
SHA-256 hash truncated to 16 bytes, serves as the routing identifier
in packet headers and bloom filters. Intermediate routers see only
@@ -156,31 +156,29 @@ FIPS uses independent encryption at two protocol layers:
| Layer | Scope | Pattern | Purpose |
| ----- | ----- | ------- | ------- |
| **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link |
| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints |
| **FMP (Mesh)** | Hop-by-hop | Noise XX | Encrypt all traffic on each peer link |
| **FSP (Session)** | End-to-end | Noise XX | Encrypt application payload between endpoints |
### Link Layer (Hop-by-Hop)
When two nodes establish a direct connection, they perform a [Noise
IK](https://noiseprotocol.org/) handshake. This authenticates both
XX](https://noiseprotocol.org/) handshake. This authenticates both
parties and establishes symmetric keys for encrypting all traffic on
that link. Every packet between direct peers is encrypted — gossip
messages, routing queries, and forwarded session datagrams alike.
The IK pattern is used because outbound connections know the peer's
npub from configuration, while inbound connections learn the
initiator's identity from the first handshake message.
Neither side requires prior knowledge of the other's static key —
both identities are revealed during the three-message handshake,
along with a protocol negotiation payload that enables rolling
upgrades.
### Session Layer (End-to-End)
FIPS establishes end-to-end encrypted sessions between any two
communicating nodes using Noise XK, regardless of how many hops
separate them. The initiator knows the destination's npub (required
for XK's pre-message); the responder learns the initiator's identity
from the third handshake message. Unlike the link-layer IK pattern
where the initiator's identity is revealed in msg1, XK delays
identity disclosure until msg3, providing stronger initiator identity
protection for traffic traversing untrusted intermediate nodes.
communicating nodes using Noise XX, regardless of how many hops
separate them. The same three-message XX pattern is used at both
layers — neither side reveals its identity until msg2 (responder) or
msg3 (initiator), providing mutual identity protection for traffic
traversing untrusted intermediate nodes.
A packet from A to D through intermediate nodes B and C:
docs/design/fips-bloom-filters.md
+43 -35
View File
@@ -1,5 +1,12 @@
# FIPS Bloom Filters
> **Status (2026-07-12):** This document still describes the **abandoned v1.5**
> bloom design. The code on `next` is presently plain **v1** — the v1.5
> implementation was removed during the sans-IO refactor and replaced with the
> v1 filter. The target design is **v2** (per the settled v2 bloom protocol
> specification), which is **pending implementation**. Until v2 lands, treat any
> "Implemented" wording below as the retired v1.5 work, not the shipped code.
This document describes the bloom filter data structures, parameters, and
mathematical properties used by FIPS for reachability-based candidate
selection. It is a supporting reference — for how bloom filters fit into
@@ -255,29 +262,30 @@ for i in 0..hash_count:
return true // Maybe present (possible false positive)
```
Where `filter_bits = 8 × (512 << size_class)` — 8,192 for v1.
Where `filter_bits = 8 × (512 << size_class)` — 8,192 for size_class 1 (default).
## Wire Format
The FilterAnnounce byte layout (`msg_type 0x20`, sequence, hash_count,
size_class, filter_bits) lives in
The FilterAnnounce byte layout (`msg_type 0x20`, flags, sequence,
base_seq, size_class, compressed payload) lives in
[../reference/wire-formats.md](../reference/wire-formats.md). The
v1 plaintext payload is 1,035 bytes (11-byte header + 1,024-byte
filter); link encryption adds 36 bytes of FMP framing (16-byte outer
header + 4-byte inner timestamp + 16-byte AEAD tag), bringing the
on-the-wire size to roughly 1,071 bytes before the underlying
transport's per-packet overhead.
v2 payload uses RLE-compressed full filters and XOR deltas
(`[count:2 LE][word:8 LE]` runs) so steady-state announcements are
typically a few dozen bytes; a `FilterNack` (msg_type 0x21) requests
full retransmission when a sequence gap is detected. Link encryption
adds 36 bytes of FMP framing (16-byte outer header + 4-byte inner
timestamp + 16-byte AEAD tag) on top of the compressed payload.
## Scale and Size Classes
### v1 Scale Limits
### Scale Limits
Coordinate-based tree distance checking ensures correct routing decisions
at all network sizes — bloom filters are an optimization that narrows the
set of peers considered, not a correctness requirement. As filters
saturate, routing still works; it just evaluates more candidates per hop.
With the v1 mandatory 1 KB filter (size_class 1):
With the default 1 KB filter (size_class 1):
- **Small networks (< 1,000 nodes)**: Both upward and downward filters
are highly accurate (worst-case FPR < 1%). Filters effectively narrow
@@ -298,33 +306,29 @@ With the v1 mandatory 1 KB filter (size_class 1):
### Size Class Table
| size_class | Bytes | Bits | Status |
| ---------- | ----- | ---- | ------ |
| 0 | 512 | 4,096 | Reserved |
| 1 | 1,024 | 8,192 | **v1 (MUST use)** |
| 2 | 2,048 | 16,384 | Reserved |
| 3 | 4,096 | 32,768 | Reserved |
| size_class | Bytes | Bits | Notes |
| ---------- | ----- | ---- | ----- |
| 0 | 512 | 4,096 | Minimum |
| 1 | 1,024 | 8,192 | Default |
| 2 | 2,048 | 16,384 | |
| 3 | 4,096 | 32,768 | |
| 4 | 8,192 | 65,536 | |
| 5 | 16,384 | 131,072 | |
| 6 | 32,768 | 262,144 | Maximum |
FMP v1 mandates size_class = 1. Nodes MUST use size_class = 1 and MUST
reject FilterAnnounce messages with any other size_class. The size_class
field is reserved in the wire format to support future protocol versions
with larger default filter sizes.
### Adaptive Sizing
### Scaling Strategy
Filter size is a node property, not a link property. Each node selects
its own size class based on outgoing filter fill ratio: step up above
~20%, step down below ~5%, with hysteresis to prevent oscillation.
Nodes near the root of the spanning tree — which carry larger combined
filters — naturally upsize, while leaf and edge nodes stay small.
The 1 KB filter becomes a practical limitation beyond ~2,000 nodes. The
size class mechanism provides the path forward: future FMP versions may
use larger default filters (size_class 2 or 3) to support larger networks
while remaining compatible with constrained nodes through folding.
Size_class 2 (2 KB, 16,384 bits) would roughly double the practical
network size limit.
The envisioned approach is that hub nodes near the root — which carry the
largest downward filters — would use larger size classes, while leaf nodes
and resource-constrained nodes continue with smaller filters. A node
receiving a filter larger than its own size class folds it down locally.
The mechanism by which heterogeneous filter sizes propagate through
the tree is a future design direction not specified in v1.
When a node receives a filter at a different size class than its own,
it converts on receipt: larger filters are folded down, smaller filters
are expanded via bit duplication. Routing queries use the peer's filter
at its native (advertised) size for full resolution; conversion happens
only when building the node's own outgoing filter.
### Folding
@@ -378,7 +382,7 @@ as described above.
| Feature | Status |
| ------- | ------ |
| 1 KB bloom filter (size_class 1) | **Implemented** |
| Variable-size bloom filters (512 B – 32 KB) | **Implemented** |
| 5 hash functions | **Implemented** |
| Split-horizon filter computation | **Implemented** |
| Tree-only merge propagation | **Implemented** |
@@ -386,6 +390,10 @@ as described above.
| Per-peer filter maintenance | **Implemented** |
| Event-driven updates | **Implemented** |
| 500ms rate limiting | **Implemented** |
| Delta compression (XOR diff + RLE) | **Implemented** |
| FilterNack sequence recovery | **Implemented** |
| Adaptive sizing (fill-ratio heuristic) | **Implemented** |
| Fold/duplicate size conversion | **Implemented** |
| FilterAnnounce gossip (all peers) | **Implemented** |
| Filter cardinality logging | **Implemented** |
| Mesh size estimation (OR-union of peer filters) | **Implemented** |
docs/design/fips-ipv6-adapter.md
+28
View File
@@ -302,6 +302,34 @@ alternative — running under a dedicated unprivileged service
account with the capability granted on the binary — see
[../how-to/run-as-unprivileged-user.md](../how-to/run-as-unprivileged-user.md).
### App-Owned TUN (embedded hosts)
On platforms where FIPS is embedded rather than run as a daemon — notably
Android, where the `VpnService` owns the TUN fd and the app has no
`CAP_NET_ADMIN` — FIPS does not create `fips0` itself. Instead the embedder owns
the fd and exchanges IPv6 packet bytes with FIPS over channels.
`Node::enable_app_owned_tun()` sets this up. It is called after `Node::new` and
before `start()` (and before the node is moved into a background task), mirroring
`control_read_handle()`, and returns two app-side channel ends:
- **app → mesh** — the embedder pushes IPv6 packets read from its fd into
`app_outbound_tx`. These are drained by `run_rx_loop` into `handle_tun_outbound`
and routed exactly as the Reader Thread's output would be.
- **mesh → app** — inbound mesh traffic on port 256 is reconstructed and written
to the node's `tun_tx` (the same sink the Writer Thread reads); the embedder
pulls from `app_inbound_rx` and writes to its fd.
With the channels installed, `start()` skips system-TUN creation (it gates on
`tun_tx` being unset), so FIPS does no `CAP_NET_ADMIN` operations.
Because packets enter via `app_outbound_tx` rather than the Reader Thread, they
**bypass `handle_tun_packet`** — the `fd00::/8` destination filter, the ICMPv6
Destination Unreachable for off-mesh dests (see [Reader Thread](#reader-thread)),
and the [TUN-Side TCP MSS Clamping](#tun-side-tcp-mss-clamping). The embedder is
therefore responsible for routing only `fd00::/8` to its TUN (so only mesh-bound
packets arrive) and for clamping TCP MSS on outbound SYNs.
## Implementation Status
| Feature | Status |
docs/design/fips-mesh-layer.md
+52 -39
View File
@@ -9,7 +9,7 @@ the mesh self-organizes, and where forwarding decisions are made.
FMP manages direct peer connections over transports. When a transport delivers
a datagram from an unknown address, FMP authenticates the sender through a
Noise IK handshake, establishing a cryptographic link. Once authenticated, the
Noise XX handshake, establishing a cryptographic link. Once authenticated, the
link carries all inter-peer communication: spanning tree gossip, bloom filter
updates, coordinate discovery, and forwarded session datagrams — all encrypted
per-hop.
@@ -82,7 +82,7 @@ for the encrypted frame wrapper: 16-byte outer header + 5-byte inner header +
### Connection Lifecycle
For connection-oriented transports, the transport must establish the underlying
connection before FMP can begin the Noise IK handshake. For connectionless
connection before FMP can begin the Noise XX handshake. For connectionless
transports, datagrams can flow immediately.
### Endpoint Discovery (Optional)
@@ -94,46 +94,52 @@ through configuration.
## Peer Authentication
### Noise IK Handshake
### Noise XX Handshake
Every peer connection begins with a Noise IK handshake that mutually
Every peer connection begins with a Noise XX handshake that mutually
authenticates both parties and establishes symmetric keys for link encryption.
The IK pattern is chosen because:
The XX pattern is chosen because:
- The **initiator** knows the responder's static public key from configuration
or discovery, and sends their own static key encrypted in the first message
- The **responder** learns the initiator's identity from the first message,
then responds with their own ephemeral key
- **Neither side** requires prior knowledge of the other's static public key
- The **responder** reveals its identity in msg2; the **initiator** reveals
its identity in msg3
- A protocol negotiation payload (version range, feature bitfield, TLV
extensions) is appended to msg2 and msg3, enabling rolling protocol
upgrades without additional round-trips
After the two-message handshake completes, both parties share symmetric
session keys derived from four DH operations (es, ss, ee, se). The handshake
provides mutual authentication, forward secrecy, and identity hiding for the
initiator.
After the three-message handshake completes, both parties share symmetric
session keys derived from three DH operations (ee, es, se). The handshake
provides mutual authentication, forward secrecy, and identity hiding for
both parties until they choose to reveal.
### Epoch Exchange and Peer Restart Detection
Both IK handshake messages carry an encrypted epoch payload — an 8-byte
random value generated once at node startup:
XX handshake messages msg2 and msg3 carry an encrypted epoch payload — an
8-byte random value generated once at node startup:
- **msg1**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) +
encrypted epoch (24 bytes) = 106 bytes total
- **msg2**: Ephemeral key (33 bytes) + encrypted epoch (24 bytes) = 57 bytes
total
- **msg1**: Ephemeral key only (33 bytes) — no identity or epoch
- **msg2**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) +
encrypted epoch (24 bytes) = 106 bytes base, plus negotiation payload
- **msg3**: Encrypted static key (49 bytes) + encrypted epoch (24 bytes)
= 73 bytes base, plus negotiation payload
The encrypted epoch (EPOCH_ENCRYPTED_SIZE = 24 bytes) consists of the
8-byte epoch value plus a 16-byte AEAD tag.
On reconnection, each peer compares the received epoch with the previously
stored epoch for that peer. An epoch mismatch indicates the peer has
restarted (generated a new epoch), triggering full link re-establishment
rather than treating the handshake as a simple reconnection. This prevents
stale session state from persisting across restarts.
Because msg1 carries no identity, restart detection is deferred: the
initiator checks the responder's epoch after msg2, and the responder
checks the initiator's epoch after msg3. An epoch mismatch indicates the
peer has restarted, triggering full link re-establishment rather than
treating the handshake as a simple reconnection. This prevents stale
session state from persisting across restarts.
### Identity Binding
The Noise handshake binds the link to the peer's cryptographic identity. After
handshake completion:
The Noise handshake binds the link to the peer's cryptographic identity.
With XX, identity confirmation happens at different points: the initiator
learns the responder's identity from msg2, and the responder learns the
initiator's identity from msg3. After handshake completion:
- The peer's public key (FIPS identity) is confirmed
- The node_addr is computed from the public key (SHA-256, truncated to 128 bits)
@@ -143,8 +149,12 @@ handshake completion:
### Reconnection
When a Noise IK msg1 arrives from a peer that already has an authenticated
link, FMP accepts the new handshake alongside the existing session. If the new
When a Noise XX msg1 arrives from an address that already has an
authenticated link, FMP accepts the new handshake alongside the existing
session. With XX, the peer's identity is not known at msg1 time — FMP can
only detect the duplicate by transport address. Identity-based checks
(restart detection, rekey recognition, cross-connection resolution) are
deferred to msg3 when the initiator's identity is revealed. If the new
handshake completes successfully, it replaces the old session. This handles
legitimate reconnection (network change, process restart, NAT rebinding)
without disrupting ongoing traffic until the new session is confirmed.
@@ -163,7 +173,7 @@ The auto-reconnect path:
3. If eligible, the peer is fed into the retry system with unlimited retries
and exponential backoff (same base interval and max backoff as startup
retries, configured via `node.retry.*`)
4. On each retry tick, a fresh Noise IK handshake is initiated toward the
4. On each retry tick, a fresh Noise XX handshake is initiated toward the
peer's configured transport addresses
Auto-reconnect only applies to peers in the static peer list with
@@ -173,8 +183,8 @@ config) is responsible for re-establishing the link.
### Handshake Message Retry
Both link-layer (Noise IK msg1/msg2) and session-layer (SessionSetup/
SessionAck) handshakes use message-level retry with exponential backoff
Both link-layer (Noise XX msg1/msg2/msg3) and session-layer (SessionSetup/
SessionAck/SessionMsg3) handshakes use message-level retry with exponential backoff
within the handshake timeout window. This handles packet loss on the
underlying transport without waiting for the full handshake timeout to
expire.
@@ -380,13 +390,14 @@ configuration tree is documented in
### Mechanism
A rekey reuses the Noise IK pattern of the initial handshake, but the two
messages travel over the existing link as ordinary encrypted FMP frames
rather than as plaintext bootstrap packets. The initiator builds a fresh
`HandshakeState`, generates msg1, and sends it through the current session;
the responder consumes msg1, builds msg2, and replies. After both sides
have exchanged messages and finalised the new keys, traffic transitions
from the old session to the new one.
A rekey reuses the Noise XX pattern of the initial handshake, but the
three messages travel over the existing link as ordinary encrypted FMP
frames rather than as plaintext bootstrap packets. The initiator builds
a fresh `HandshakeState`, generates msg1, and sends it through the
current session; the responder consumes msg1, builds msg2, and replies;
the initiator then sends msg3. After both sides have exchanged messages
and finalised the new keys, traffic transitions from the old session to
the new one.
Cutover is signalled in-band by the **K-bit** in the FMP flags byte. Each
side starts emitting frames under the new session with K set; on receipt
@@ -547,7 +558,9 @@ an attacker sends invalid packets to elicit responses.
| Feature | Status |
| ------- | ------ |
| Noise IK handshake (with epoch) | **Implemented** |
| Noise XX handshake (with epoch and negotiation) | **Implemented** |
| Protocol negotiation (version + features + TLV) | **Implemented** |
| Node profiles (Full, NonRouting, Leaf) | **Implemented** |
| Peer restart detection (epoch mismatch) | **Implemented** |
| Link encryption (ChaCha20-Poly1305) | **Implemented** |
| Index-based session dispatch | **Implemented** |
docs/design/fips-mesh-operation.md
+20 -13
View File
@@ -182,8 +182,8 @@ The source creates a LookupRequest containing:
- **request_id**: Unique identifier for deduplication
- **target**: The node_addr being sought
- **origin**: The requester's node_addr
- **origin_coords**: The requester's current tree coordinates (so the
response can route back)
- **min_mtu**: Minimum transport MTU the origin requires (transit nodes
skip peers whose link MTU is below this)
- **TTL**: Bounds the forwarding radius
### Bloom-Guided Tree Routing
@@ -269,8 +269,8 @@ the primary mechanism: each transit node looks up the `request_id` in its
`recent_requests` table to find the peer that forwarded the original request,
and sends the response back through that peer. This ensures the response
follows the same path as the request. Greedy tree routing toward the
`origin_coords` is used only as a fallback if the reverse-path entry has
expired.
greedy tree routing toward the origin's coordinates is used only as a
fallback if the reverse-path entry has expired.
**Response-forwarded flag**: Each `recent_requests` entry tracks whether a
response has already been forwarded for that `request_id`. If a second
@@ -472,16 +472,23 @@ When traffic resumes:
3. Coordinates: discovery may be needed if cache has expired
4. SessionSetup re-warms transit caches on the new path
## Leaf-Only Operation *(under development)*
## Node Profiles
Leaf-only operation is an optimization for resource-constrained nodes
(sensors, battery-powered devices). The core infrastructure exists (config
flag, node constructor, bloom filter support) but is not yet enabled in
normal operation.
Nodes advertise a profile during FMP negotiation (bits 0-2 of the feature
bitfield): **Full** (default), **NonRouting**, or **Leaf**. At least one
side of a link must be Full. Config mapping: `disable_routing: true` →
NonRouting, `leaf_only: true` → Leaf.
### Concept
### Non-Routing Nodes
A leaf-only node connects to a single upstream peer that handles all routing
A non-routing node participates in the spanning tree but does not forward
transit traffic or send bloom filters. Its full peer inserts the
non-routing node's identity as a leaf dependent. MMP report flow is gated
by wants/provides bits negotiated during the handshake.
### Leaf Nodes
A leaf node connects to a single upstream peer that handles all routing
on its behalf:
- **No bloom filter storage or processing**: The upstream peer includes the
@@ -504,7 +511,7 @@ The upstream peer:
Even as a leaf-only node, it still:
- Maintains its own Noise IK link session with the upstream peer (FMP layer)
- Maintains its own Noise XX link session with the upstream peer (FMP layer)
- Can establish end-to-end FSP sessions with arbitrary destinations
- Has its own identity (npub, node_addr)
@@ -564,7 +571,7 @@ recovery).
| Discovery originator backoff | **Implemented** |
| Discovery transit-side rate limiting | **Implemented** |
| Discovery response-forwarded dedup | **Implemented** |
| Leaf-only operation | Under development |
| Node profiles (Full, NonRouting, Leaf) | **Implemented** |
| Link cost in parent selection (ETX) | **Implemented** |
| Link cost in candidate ranking | **Implemented** |
docs/design/fips-mmp.md
+6 -16
View File
@@ -68,7 +68,7 @@ MMP supports three modes:
| ---- | ----------------- | ----------------- |
| **Full** (default) | SenderReport + ReceiverReport | All metrics including RTT, loss, jitter, goodput, OWD trend |
| **Lightweight** | ReceiverReport only | Loss (from counter gaps), jitter, OWD trend. No RTT. |
| **Minimal** | None | Spin bit and CE echo flags only. No computed metrics. |
| **Minimal** | None | CE echo flags only. No computed metrics. |
The mode is configured per layer (`node.mmp.mode` and
`node.session_mmp.mode`).
@@ -88,28 +88,18 @@ The session-layer bounds are higher because session reports are
encrypted and forwarded through every transit link, so bandwidth cost
is proportional to path length.
## Spin Bit and RTT
## RTT Measurement
The SP (spin bit) flag in the FMP inner header follows the QUIC spin
bit pattern: reflected on receive, toggled on send when the reflected
value matches the last sent value. The spin bit state machine runs
for TX reflection, but **RTT samples from the spin bit are
discarded**. In a mesh protocol where frames are sent irregularly
(tree announces, bloom filters, MMP reports on different timers),
inter-frame processing delays inflate spin bit RTT measurements
unpredictably. Timestamp-echo from ReceiverReports (with dwell-time
compensation) is the sole SRTT source.
SRTT is derived exclusively from timestamp-echo in ReceiverReports
with dwell-time compensation, applied via the Jacobson/Karels
algorithm (RFC 6298, α = 1/8). This is the sole SRTT source at both
layers.
Duplicate or regressed ReceiverReports are ignored before any RTT, loss,
goodput, or ETX update. If receiver-side dwell time exceeds the wire
field, the report keeps its counters but sends a zero timestamp echo so
the sender cannot form an invalid RTT sample.
The spin bit lives in the link-layer FMP inner header, so this
mechanism applies to link-layer MMP only. Session-layer MMP carries
its spin bit in the FSP encrypted inner header but uses it the same
way: reflected for diagnostic visibility, not used for SRTT.
## ECN Congestion Signaling
The CE (Congestion Experienced) flag (bit 1 in the FMP flags byte)
docs/design/fips-nostr-discovery.md
+4 -4
View File
@@ -273,7 +273,7 @@ carrying the session id, the punch socket, and the learned remote
address. `adopt_established_traversal()` in the node lifecycle takes
the socket, registers it with the UDP transport layer as a new
transport instance, and calls `initiate_connection()` with the peer's
FIPS identity as the expected remote. FMP's Noise IK handshake runs on
FIPS identity as the expected remote. FMP's Noise XX handshake runs on
the same socket — there is no "promote link" step between punch and
handshake; the punch socket *is* the FMP socket.
@@ -382,7 +382,7 @@ semaphore and replay-cache layers downstream.
non-default `app` value to scope visibility.
- **Nothing about discovery bypasses FMP.** A successful punch yields
a UDP socket with a claimed remote identity. That identity is not
trusted until FMP's Noise IK handshake completes. A peer whose
trusted until FMP's Noise XX handshake completes. A peer whose
advert says "I am npub X at 1.2.3.4:5678" but whose FMP handshake
presents a different static key is rejected at the mesh layer.
@@ -477,7 +477,7 @@ The TXT record carries three keys (`src/discovery/lan/mod.rs:47-55`):
Once per node tick, the node drains browser events and acts on them in
`poll_lan_discovery()` (`src/node/lifecycle.rs:907`, called from
`src/node/handlers/rx_loop.rs:266`). For each discovered peer it finds
`src/node/dataplane/rx_loop.rs:266`). For each discovered peer it finds
a UDP transport whose family matches the peer address, parses the
`npub` into a `PeerIdentity`, skips peers it is already connected to or
currently connecting to, and otherwise initiates a connection.
@@ -584,7 +584,7 @@ beyond the shared scope fallback.
- [fips-transport-layer.md](fips-transport-layer.md) — UDP, TCP, and
Tor transport mechanics; the punch socket is adopted as a normal
UDP transport after handoff.
- [fips-mesh-layer.md](fips-mesh-layer.md) — FMP Noise IK handshake
- [fips-mesh-layer.md](fips-mesh-layer.md) — FMP Noise XX handshake
that runs on the adopted socket.
- [port-advertisement-and-nat-traversal.md](port-advertisement-and-nat-traversal.md)
— generic protocol reference (event tags, NIP usage, on-the-wire
docs/design/fips-prior-work.md
+17 -29
View File
@@ -80,27 +80,24 @@ patterns.
## Noise Protocol Framework
FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/)
at both protocol layers, with different handshake patterns chosen for
each layer's threat model. FMP link encryption uses **Noise IK**,
providing mutual authentication with a single round trip where the
initiator knows the responder's static key in advance.
[WireGuard](https://www.wireguard.com/) uses the same IK base pattern
(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP
session encryption uses **Noise XK**, the same pattern used by the
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md),
where the initiator's static key is transmitted in a third message
rather than the first. XK provides stronger initiator identity hiding
at the cost of an additional round trip — a worthwhile tradeoff for
session-layer traffic that traverses untrusted intermediate nodes. At
the link layer, where both peers are configured and directly
connected, IK's single round trip is preferred.
at both protocol layers with the **Noise XX** handshake pattern. XX
requires no prior knowledge of the peer's static key — both
identities are revealed during a three-message handshake (responder
in msg2, initiator in msg3). This enables anonymous peer discovery on
shared-media transports and allows a protocol negotiation payload to
be exchanged alongside the handshake, supporting rolling protocol
upgrades without extra round trips.
[WireGuard](https://www.wireguard.com/) uses the related IK pattern
for VPN tunnels;
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md)
uses XK for transport encryption.
Specific Noise references and adapted constructions:
- Perrin, T. ["The Noise Protocol Framework"](https://noiseprotocol.org/noise.html).
Revision 34, 2018. *Framework for building crypto protocols using
Diffie-Hellman key agreement and AEAD ciphers. FSP uses the XK
handshake pattern.*
Diffie-Hellman key agreement and AEAD ciphers. FIPS uses the XX
handshake pattern at both layers.*
- Donenfeld, J.A. ["WireGuard: Next Generation Kernel Network Tunnel"](https://www.wireguard.com/papers/wireguard.pdf).
NDSS 2017. *Transport-independent cryptographic sessions bound to
@@ -156,14 +153,6 @@ computation used in TCP for retransmission timeout calculation since
1988. MMP derives RTT from timestamp-echo in ReceiverReports with
dwell-time compensation, rather than from packet round-trips.
The spin bit in the FMP frame header follows the
[QUIC](https://www.rfc-editor.org/rfc/rfc9000) spin bit
([RFC 9312](https://www.rfc-editor.org/rfc/rfc9312)) — a single bit
that alternates each round trip, enabling passive latency measurement.
FIPS implements the spin bit state machine but relies on
timestamp-echo for SRTT, as irregular mesh traffic makes spin bit RTT
unreliable.
The Expected Transmission Count (ETX) metric, computed from
bidirectional delivery ratios, was introduced by
[De Couto et al. (2003)](https://pdos.csail.mit.edu/papers/grid:mobicom03/paper.pdf)
@@ -325,11 +314,10 @@ The protocol builds on these foundations and adds several new elements:
| [HIP](https://en.wikipedia.org/wiki/Host_Identity_Protocol) | identity-as-address |
| [Babel](https://www.irif.fr/~jch/software/babel/) | split-horizon, ETX |
| [RIP](https://en.wikipedia.org/wiki/Routing_Information_Protocol) | split-horizon |
| [Noise Framework](https://noiseprotocol.org/) | FMP IK, FSP XK |
| [WireGuard](https://www.wireguard.com/) | IK pattern, receiver-index dispatch, identity-bound sessions |
| [Lightning BOLT #8](https://github.com/lightning/bolts/blob/master/08-transport.md) | XK pattern |
| [QUIC (RFC 9000)](https://www.rfc-editor.org/rfc/rfc9000) | spin bit, transport design |
| [QUIC Spin Bit (RFC 9312)](https://www.rfc-editor.org/rfc/rfc9312) | passive RTT measurement |
| [Noise Framework](https://noiseprotocol.org/) | FMP and FSP XX handshakes |
| [WireGuard](https://www.wireguard.com/) | receiver-index dispatch, identity-bound sessions |
| [Lightning BOLT #8](https://github.com/lightning/bolts/blob/master/08-transport.md) | comparison reference |
| [QUIC (RFC 9000)](https://www.rfc-editor.org/rfc/rfc9000) | transport design |
| [RTCP (RFC 3550)](https://www.rfc-editor.org/rfc/rfc3550) | sender/receiver report structure, jitter algorithm |
| [TCP SRTT/RTO (RFC 6298)](https://www.rfc-editor.org/rfc/rfc6298) | Jacobson/Karels SRTT |
| [ECN (RFC 3168)](https://www.rfc-editor.org/rfc/rfc3168) | CE echo |
docs/design/fips-security.md
+3 -3
View File
@@ -17,8 +17,8 @@ you can deliver packets to your `fips0` address — your direct peers
forward traffic from non-peer mesh nodes onto your `fips0` the same
way any router forwards transit traffic. Identity on the mesh is the
originating node's npub — the FMP link layer authenticates direct
peers with Noise IK and the FSP session layer authenticates session
endpoints with Noise XK — but identity is **not** authorization.
peers with Noise XX and the FSP session layer authenticates session
endpoints with Noise XX — but identity is **not** authorization.
Knowing who sent a packet does not, by itself, decide whether the
local host should accept it.
@@ -149,7 +149,7 @@ explicitly not:
originating mesh node's npub is allowed to use that service. That
is the application's responsibility (e.g., an `authorized_keys`
file for SSH, an ACL in the application's configuration).
- **ACL on the mesh handshake.** The FMP Noise IK handshake
- **ACL on the mesh handshake.** The FMP Noise XX handshake
authenticates the peer's npub and, on both inbound and outbound
paths, consults the peer ACL (`peers.allow` / `peers.deny`) before
promoting the connection. The ACL evaluates in TCP-Wrappers order:
docs/design/fips-session-layer.md
+71 -48
View File
@@ -120,25 +120,29 @@ node, FMP delivers it to FSP for session-layer processing.
Sessions are established on demand when the first datagram needs to be sent to
a destination with no existing session.
FSP uses Noise XK for session key agreement (Noise Protocol Framework;
Perrin 2018). The initiator knows the destination's npub (required for
XK's pre-message `s` token); the responder learns the initiator's
identity from msg3 (not msg1, unlike IK at the link layer). This
provides stronger initiator identity hiding — the initiator's static
key is encrypted under the established shared secret rather than under
only the responder's static key.
FSP uses Noise XX for session key agreement (Noise Protocol Framework;
Perrin 2018). Neither side requires prior knowledge of the other's
static key — both identities are revealed during the handshake
(responder in msg2, initiator in msg3). An optional protocol negotiation
payload may be appended to msg2/msg3 (omitted for rekey handshakes).
The handshake is a three-message flow carried in SessionSetup, SessionAck,
and SessionMsg3:
1. **Initiator** sends SessionSetup containing Noise XK msg1 (ephemeral key
1. **Initiator** sends SessionSetup containing Noise XX msg1 (ephemeral key
only) and both parties' tree coordinates
2. **Responder** processes msg1, sends SessionAck containing Noise XK msg2
(ephemeral key + encrypted epoch) and both parties' tree coordinates.
The responder transitions to AwaitingMsg3 state.
3. **Initiator** processes msg2, sends SessionMsg3 containing the encrypted
static key and encrypted epoch. Both parties derive identical symmetric
session keys and the session is established.
2. **Responder** processes msg1, sends SessionAck containing Noise XX msg2
(ephemeral key + encrypted static key + encrypted epoch) and both
parties' tree coordinates. The responder transitions to AwaitingMsg3
state.
3. **Initiator** processes msg2 (learning the responder's identity), sends
SessionMsg3 containing its encrypted static key and encrypted epoch.
The responder learns the initiator's identity from msg3. Both parties
derive identical symmetric session keys and the session is established.
Post-handshake identity verification uses x-only key comparison
(parity-independent) to confirm the revealed identity matches the
expected npub.
Each side's epoch (an 8-byte random value generated at startup) is
exchanged encrypted in msg2 and msg3. On subsequent handshakes, an epoch
@@ -219,29 +223,30 @@ than network addresses. A session survives:
## End-to-End Encryption
### Noise XK Pattern
### Noise XX Pattern
FSP uses Noise XK for session encryption, distinct from the Noise IK
pattern used at the link layer. The full Noise descriptor is
`Noise_XK_secp256k1_ChaChaPoly_SHA256`.
FSP uses the same Noise XX pattern as the link layer (FMP). The full
Noise descriptor is `Noise_XX_secp256k1_ChaChaPoly_SHA256`.
The XK pattern (pre-message: `← s`):
The XX pattern (no pre-message):
- **msg1** (`→ e, es`): Initiator sends ephemeral key only. The initiator's
static identity is not revealed in this message.
- **msg2** (`← e, ee`): Responder sends ephemeral key and encrypted epoch.
- **msg1** (`→ e`): Initiator sends ephemeral key only. No identity
disclosed, no DH with static keys.
- **msg2** (`← e, ee, s, es`): Responder sends ephemeral key, encrypted
static key, and encrypted epoch. The initiator learns the responder's
identity.
- **msg3** (`→ s, se`): Initiator sends encrypted static key and encrypted
epoch. Both parties now share identical session keys.
epoch. The responder learns the initiator's identity. Both parties now
share identical session keys.
After the handshake, Noise produces two directional symmetric keys
(`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent data.
(`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent
data.
The XK pattern requires the initiator to know the responder's static key
in advance (the `← s` pre-message), which is satisfied by the discovery
or DNS lookup that precedes session establishment. In exchange, XK
provides stronger initiator identity protection than IK — the initiator's
static key is encrypted under the full shared secret (after three DH
operations) rather than under only the responder's static key.
XX requires no prior knowledge of the peer's static key. The initiator
still needs the destination's npub to address the SessionSetup, but the
Noise handshake itself does not depend on it — identity is verified
post-handshake by comparing the revealed key against the expected npub.
### Cryptographic Primitives
@@ -251,26 +256,27 @@ primitive table shared with the link layer.
### secp256k1 Parity Normalization
Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity). The
Noise XK pre-message mixes the responder's static key as a 33-byte compressed
key, and the default secp256k1 ECDH hash includes a parity-dependent version
byte.
Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity).
When the Noise XX handshake reveals a peer's static key via
`public_key().serialize()`, the key has its actual parity (0x02 or 0x03
prefix). The default secp256k1 ECDH hash also includes a parity-dependent
version byte.
Both operations are normalized to be parity-independent: the pre-message hash
uses even parity (`0x02` prefix), and ECDH hashes only the x-coordinate of the
result point. This ensures handshakes succeed regardless of the responder's
actual key parity.
Both operations are normalized to be parity-independent: ECDH hashes only
the x-coordinate of the result point, and post-handshake identity
verification uses `x_only_public_key()` to strip parity before comparing
against the expected npub. This ensures handshakes and identity checks
succeed regardless of key parity.
### Privacy Note
Noise XK provides stronger initiator identity protection than IK. In XK, the
initiator's static key is encrypted in msg3 under the full shared secret
(derived from three DH operations), so an attacker who compromises only the
responder's nsec cannot decrypt the initiator's identity from captured
handshake messages (they would also need the responder's ephemeral key).
This is the primary reason FSP uses XK rather than IK — session-layer
traffic traverses untrusted intermediate nodes, making initiator identity
protection more valuable than at the link layer.
Noise XX provides mutual identity protection — both the initiator's and
responder's static keys are encrypted under the evolving shared secret
(derived from DH operations completed in earlier messages). An attacker
who compromises only one side's nsec cannot decrypt the other side's
identity from captured handshake messages without also obtaining the
corresponding ephemeral key. Since session-layer traffic traverses
untrusted intermediate nodes, this mutual identity hiding is valuable.
### Data Packet Authentication
@@ -461,7 +467,7 @@ reactive PMTUD mechanism.
| Feature | Status |
| ------- | ------ |
| Session establishment (Noise XK) | **Implemented** |
| Session establishment (Noise XX) | **Implemented** |
| Peer restart detection (epoch exchange) | **Implemented** |
| MtuExceeded handling | **Implemented** |
| End-to-end encryption (ChaCha20-Poly1305) | **Implemented** |
@@ -497,7 +503,7 @@ reactive PMTUD mechanism.
- [fips-mmp.md](fips-mmp.md) — Metrics Measurement Protocol (link + session)
- [fips-mtu.md](fips-mtu.md) — Path MTU model (PathMtuNotification,
MtuExceeded, hysteresis)
- [fips-prior-work.md](fips-prior-work.md) — Noise XK, WireGuard,
- [fips-prior-work.md](fips-prior-work.md) — Noise XX, WireGuard,
DTLS replay window, IKEv2 simultaneous initiation, hybrid coordinate
warmup citations
- [../reference/wire-formats.md](../reference/wire-formats.md) — Wire
@@ -507,6 +513,23 @@ reactive PMTUD mechanism.
### External References
- Perrin, T. ["The Noise Protocol Framework"](https://noiseprotocol.org/noise.html).
Revision 34, 2018. *Framework for building crypto protocols using Diffie-Hellman
key agreement and AEAD ciphers. FSP uses the XX handshake pattern.*
- Donenfeld, J.A. ["WireGuard: Next Generation Kernel Network Tunnel"](https://www.wireguard.com/papers/wireguard.pdf).
NDSS 2017. *Transport-independent cryptographic sessions bound to identity keys
rather than network addresses; AEAD-only authentication model.*
- Rescorla, E., Modadugu, N. [RFC 6347](https://datatracker.ietf.org/doc/html/rfc6347):
"Datagram Transport Layer Security Version 1.2". 2012. *Explicit sequence numbers
with sliding bitmap window for replay protection over unreliable transports.*
- Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., Kivinen, T.
[RFC 7296](https://datatracker.ietf.org/doc/html/rfc7296):
"Internet Key Exchange Protocol Version 2 (IKEv2)". 2014. *Simultaneous
initiation resolution (§2.8) and INITIAL_CONTACT peer restart detection (§2.4).*
- Mogul, J., Deering, S. [RFC 1191](https://datatracker.ietf.org/doc/html/rfc1191):
"Path MTU Discovery". 1990. *End-to-end path MTU discovery; FSP adapts this for
overlay networks using transit-node min() propagation.*
docs/design/fips-spanning-tree.md
+1 -1
View File
@@ -171,7 +171,7 @@ When a node receives a TreeAnnounce from peer P:
1. **Validate version**: Reject if version ≠ 0x01
2. **Verify signature**: Check P's declaration signature using P's known
public key (established during Noise IK handshake)
public key (established during Noise XX handshake)
3. **Verify identity**: Confirm the declaration's node_addr matches the
sender's known identity
4. **Check freshness**: If `sequence ≤ stored sequence for P`, discard
docs/design/fips-transport-layer.md
+23 -25
View File
@@ -18,7 +18,7 @@ to the FIPS Mesh Protocol (FMP) above.
The transport layer deals exclusively in **transport addresses** — IP:port
or hostname:port addresses, MAC addresses, .onion identifiers, radio device addresses. These are
opaque to every layer above FMP. The mapping from transport address to FIPS
identity happens at the link layer after the Noise IK link handshake completes.
identity happens at the link layer after the Noise XX link handshake completes.
The word "peer" belongs to the link layer and above; the transport layer
knows only about remote endpoints identified by transport addresses.
@@ -71,7 +71,7 @@ forwarding and LookupResponse transit annotation.
For connection-oriented transports, manage the underlying connection: TCP
handshake, Tor circuit establishment, BLE pairing. FMP cannot begin
the Noise IK link handshake until the transport-layer connection is
the Noise XX link handshake until the transport-layer connection is
established.
Connection-oriented transports expose a non-blocking connect interface.
@@ -154,7 +154,7 @@ and duplication at the routing layer.
**Connection model**: Connectionless transports (UDP, raw Ethernet) allow
immediate datagram exchange. Connection-oriented transports (TCP, Tor, BLE)
require connection setup before FMP can begin the Noise IK link handshake,
require connection setup before FMP can begin the Noise XX link handshake,
adding startup latency.
**Stream vs. datagram**: Datagram transports have natural packet boundaries.
@@ -262,7 +262,7 @@ UDP (1500 vs 1472 MTU).
- **No IP dependency**: Operates below the IP layer. Nodes on the same
Ethernet segment can communicate without IP addresses or routing
infrastructure
- **Broadcast discovery**: Nodes discover each other via periodic beacon
- **Broadcast neighbor detection**: Nodes discover each other via periodic beacon
broadcasts on the shared medium, with no static peer configuration required
- **Higher MTU**: Standard Ethernet frames carry 1500 bytes of payload,
yielding an effective FIPS MTU of 1499 after the frame type prefix
@@ -276,32 +276,30 @@ EtherType 0x2121. SOCK_DGRAM mode
lets the kernel handle Ethernet header construction and parsing — the
transport deals only with payloads and MAC addresses.
Data frames use a 3-byte header: a 1-byte frame type (`0x00`) followed by
a 2-byte little-endian payload length. The length field allows the receiver
to trim Ethernet minimum-frame padding that would otherwise corrupt AEAD
verification. Beacon frames (`0x01`) use only the 1-byte type prefix
(fixed 34-byte payload). Beacons and data share the same EtherType and
socket.
All frames use a unified 4-byte header: `[type:1][flags:1][length:2 LE]`.
The length field allows the receiver to trim Ethernet minimum-frame padding
that would otherwise corrupt AEAD verification. Frame types: `0x00` (data),
`0x01` (beacon). Beacons and data share the same EtherType and socket.
| Property | Value |
| -------- | ----- |
| EtherType | 0x2121 |
| Socket type | AF_PACKET SOCK_DGRAM |
| Data frame header | `[type:1][length:2 LE][payload]` |
| Beacon frame header | `[type:1][payload]` (fixed 34 bytes) |
| Effective MTU | Interface MTU - 3 (typically 1497) |
| Frame header | `[type:1][flags:1][length:2 LE][payload]` |
| Effective MTU | Interface MTU - 4 (typically 1496) |
| Addressing | 6-byte MAC address |
| Platform | Linux only (`CAP_NET_RAW` required) |
### Beacon Discovery
### Neighbor Beacons
Ethernet nodes discover peers via broadcast beacons sent to
ff:ff:ff:ff:ff:ff. Each beacon is a 34-byte frame containing the sender's
x-only public key. Receiving nodes extract the MAC source address from the
frame and the public key from the payload, then report the discovered peer
to FMP.
ff:ff:ff:ff:ff:ff. Beacons are minimal 5-byte frames (4-byte header +
1-byte beacon type) — no public key is included. The peer's identity is
learned from the Noise XX handshake after the connection is established.
Receiving nodes extract the MAC source address from the frame and report
the discovered address to FMP.
Four configuration flags control discovery behavior — `discovery`
Four configuration flags control neighbor behavior — `listen`
(listen for beacons), `announce` (broadcast beacons), `auto_connect`
(initiate handshakes to discovered peers), and `accept_connections`
(accept inbound handshakes). The flag table and per-flag defaults
@@ -310,13 +308,13 @@ under `transports.ethernet.*`.
A typical discoverable node sets `announce`, `auto_connect`, and
`accept_connections` all true. A passive listener uses just
`discovery: true` to observe the network without announcing itself.
`listen: true` to observe the network without announcing itself.
### WiFi Compatibility
WiFi interfaces in infrastructure (managed) mode work transparently for
unicast — the mac80211 subsystem handles frame translation between 802.11
and 802.3. Broadcast beacon discovery is unreliable in managed mode because
and 802.3. Broadcast neighbor detection is unreliable in managed mode because
access points commonly isolate clients from each other's broadcast traffic.
Startup logging:
@@ -384,7 +382,7 @@ every tick, `poll_pending_connects()` calls `connection_state(addr)` to
check progress. When the transport reports `Connected`, the completed
connection is promoted to the established pool (stream split into
read/write halves, per-connection receive task spawned), and the node
initiates the Noise IK link handshake. If the transport reports `Failed`,
initiates the Noise XX link handshake. If the transport reports `Failed`,
the node schedules a retry with exponential backoff.
As a fallback, `send(addr, data)` still performs synchronous
@@ -422,7 +420,7 @@ socket is created).
The Tor transport routes FIPS traffic through the Tor network, hiding
a node's IP address from its peers. A node behind Tor connects outbound
through a local Tor SOCKS5 proxy; the remote peer sees the Tor exit
node's IP, not the initiator's. After the Noise IK handshake, the remote
node's IP, not the initiator's. After the Noise XX handshake, the remote
peer knows the initiator's FIPS identity (npub) but not its network
location.
@@ -503,7 +501,7 @@ The inbound accept loop mirrors the TCP transport's pattern: accept
connection, configure socket (TCP_NODELAY, keepalive), spawn a
per-connection receive loop using the shared FMP stream reader. Inbound
connections arrive from `127.0.0.1` (Tor daemon's local forwarding); peer
identity is resolved during the Noise IK handshake, not from the transport
identity is resolved during the Noise XX handshake, not from the transport
address.
Configuration requires coordinating `torrc` and `fips.yaml`. The
@@ -895,7 +893,7 @@ transitions through `Starting` to `Up` (operational). `stop()` moves to
| --------- | ------ | ----- |
| UDP/IP | **Implemented** | Primary transport, AsyncFd/recvmsg, SO_RXQ_OVFL kernel drop detection |
| TCP/IP | **Implemented** | FMP header-based framing, non-blocking connect, per-connection MSS MTU |
| Ethernet | **Implemented** | AF_PACKET SOCK_DGRAM, EtherType 0x2121, beacon discovery, Linux only |
| Ethernet | **Implemented** | AF_PACKET SOCK_DGRAM, EtherType 0x2121, neighbor beacons, Linux only |
| WiFi | **Implemented** (via Ethernet transport, infrastructure mode) | mac80211 translates 802.11↔802.3; broadcast beacons unreliable through APs |
| Tor | **Implemented** | Outbound SOCKS5, inbound via onion service, .onion and clearnet addressing |
| Nym | **Implemented** | Outbound-only SOCKS5 through nym-socks5-client, mixnet anonymity, IP/hostname addressing |
docs/design/port-advertisement-and-nat-traversal.md
+1 -1
View File
@@ -457,7 +457,7 @@ authentication or encryption**. The application layer is responsible
for establishing its own security on the punched channel — for
example, a Noise Protocol handshake keyed from the Nostr identity,
or an application-specific authenticated-encryption layer. FIPS
runs its FMP Noise IK handshake immediately after adoption; the
runs its FMP Noise XX handshake immediately after adoption; the
identity proven by the Noise handshake is the same Nostr pubkey
that signed the inner offer/answer rumour, so a man-in-the-middle on
the relay cannot impersonate the responder.
docs/how-to/README.md
+2
View File
@@ -25,4 +25,6 @@ X" to "X is done".
| [persistent-identity.md](persistent-identity.md) | Provision a stable Nostr keypair so the node keeps the same npub across restarts |
| [host-aliases.md](host-aliases.md) | Use shortnames (`test-us01.fips`, `my-laptop.fips`) instead of full npubs by editing `/etc/fips/hosts` or setting peer aliases |
| [set-up-bluetooth-peer.md](set-up-bluetooth-peer.md) | Configure a Bluetooth Low Energy peer link |
| [set-up-80211s-mesh-backhaul.md](set-up-80211s-mesh-backhaul.md) | Link OpenWrt FIPS routers over an open 802.11s radio backhaul (FIPS provides encryption, authentication, and routing) |
| [set-up-open-access-ssid.md](set-up-open-access-ssid.md) | Broadcast the open `!FIPS` access SSID so phones and laptops roam onto the mesh (one ESS: save once, roam every FIPS router) |
| [diagnose-mtu-issues.md](diagnose-mtu-issues.md) | Triage MTU-shaped failures and rule out their imposters (bufferbloat, transport saturation) |
docs/how-to/enable-nostr-discovery.md
+1 -1
View File
@@ -197,7 +197,7 @@ What this achieves: the node publishes a `udp:nat` endpoint plus its
signaling relays in the advert. When either side initiates, an
encrypted offer is sealed to the peer's npub, a matching answer
comes back, and both sides punch at the negotiated time. On success,
the punch socket is adopted as an FMP UDP transport and Noise IK
the punch socket is adopted as an FMP UDP transport and Noise XX
proceeds normally.
> **Validation:** `advertise_on_nostr: true` with `public: false` on
docs/how-to/set-up-80211s-mesh-backhaul.md
+249
View File
@@ -0,0 +1,249 @@
# Set Up an 802.11s Mesh Backhaul (OpenWrt)
Link FIPS routers over radio — no cables, no APs, no shared
infrastructure — by running the Ethernet transport on an open 802.11s
mesh interface. The radio layer provides nothing but L2 frames to
direct neighbors; FIPS provides everything else: encryption and
authentication (the Noise handshake), peer discovery (Ethernet beacons), and
routing (the spanning tree).
For the transport design, see
[../design/fips-transport-layer.md](../design/fips-transport-layer.md).
For all `transports.ethernet.*` configuration keys, see
[../reference/configuration.md](../reference/configuration.md).
## Why open, why forwarding off
Two deliberate choices distinguish this from a stock 802.11s setup:
- **`encryption none`** — the mesh is open on purpose. Every FIPS peer
link is already authenticated and encrypted by the Noise
handshake, so SAE at L2 would duplicate that work, add a shared
credential to provision across routers, and (on ath10k) force the
firmware into its slower raw Tx/Rx mode. A stranger can form an
802.11s peering with your router *and* a FIPS peer link on top of it —
the same open model as mDNS and BLE discovery, where the advert is
only a hint and the handshake authenticates each link (no
impersonation, no MITM) rather than gating who may peer. Admission is
open up to the daemon's max-peers cap. What you concede: any nearby
radio can peer and reach the FIPS overlay surface; L2 metadata (MAC
addresses, frame sizes) is visible in the air; a hostile radio can
burn airtime — all inherent to an open radio link.
- **`mesh_fwding 0`** — disables 802.11s's own HWMP routing so each
mesh link is a plain neighbor link. FIPS is the routing layer; two
routing layers would fight, and broadcast discovery beacons would
flood the whole mesh instead of reaching direct neighbors only.
The interface is **not** bridged into `br-lan` — the FIPS Ethernet
transport binds it directly.
## When to use
- Two or more OpenWrt FIPS routers within radio range of each other,
where running cable is impractical.
- You want the mesh segment to keep working with zero shared
credentials or per-site configuration ("flash and drop in").
It is **not** for connecting phones or laptops — client devices
cannot join an 802.11s mesh. They enter the mesh through a normal AP
on the same router (see constraints below), or over BLE.
## Requirements
- OpenWrt 22.03+ with the FIPS package installed.
- A radio whose driver supports mesh point interfaces. Check with:
```sh
iw list | grep -A 10 "Supported interface modes" | grep "mesh point"
```
The mainstream OpenWrt chips (ath9k, ath10k, mt76) all qualify.
- Ideally a dual- or tri-band router, so one band can be dedicated to
the backhaul (see constraints).
## Step 1 — create the mesh interface(s)
On **each** router, run the helper once per radio you want in the
backhaul:
```sh
fips-mesh-setup radio1
```
This creates an open 802.11s interface with mesh ID `fips-mesh` and
HWMP forwarding off, attaches it to an unmanaged netifd interface (no
IP configuration — none is needed), uncomments the matching `meshN`
transport entry in `/etc/fips/fips.yaml` (see Step 2), and reloads the
radio. Interfaces are named by radio index: `radio0` → `fips-mesh0`,
`radio1` → `fips-mesh1`. Pass a second argument to use a different
mesh ID.
Note: the helper runs `wifi reload`, which re-applies the whole
wireless config and so briefly drops every client AP on all radios for
a few seconds. `fips-mesh-setup remove` reloads the same way. Expect
the blip if clients are connected.
On dual-band routers, meshing **both** bands is worth it: 2.4 GHz
reaches further at lower rates, 5 GHz carries more over shorter
links. Note this is **failover, not multipath**: FIPS keeps one
active link per peer, so traffic uses one band at a time — the other
is a standby that re-establishes the peer if the active link dies
(detection via keepalive timeout, so a cutover takes seconds, not
milliseconds):
```sh
fips-mesh-setup radio0
fips-mesh-setup radio1
```
**Pin the same channel on every backhaul router, per band.** Mesh
points only peer on the same channel, and the mesh inherits whatever
the radio is set to — with `channel 'auto'` (the default on many
devices) each router picks its own and the mesh silently never forms.
The script prints the radio's current band and channel and warns on
`auto`:
```sh
uci set wireless.radio1.channel='36'
uci commit wireless && wifi reload
```
Prefer a non-DFS channel (36–48 on 5 GHz): on DFS channels the radio
must wait ~60 s in CAC before transmitting after every reload.
Equivalent manual UCI (per radio), if you prefer to see what it does:
```sh
uci batch <<'EOF'
set wireless.fips_mesh_radio1=wifi-iface
set wireless.fips_mesh_radio1.device='radio1'
set wireless.fips_mesh_radio1.mode='mesh'
set wireless.fips_mesh_radio1.mesh_id='fips-mesh'
set wireless.fips_mesh_radio1.encryption='none'
set wireless.fips_mesh_radio1.mesh_fwding='0'
set wireless.fips_mesh_radio1.ifname='fips-mesh1'
set wireless.fips_mesh_radio1.network='fips_mesh_radio1'
set network.fips_mesh_radio1=interface
set network.fips_mesh_radio1.proto='none'
EOF
uci commit
wifi reload
```
## Step 2 — check the FIPS transport binding
The `fips.yaml` shipped in the OpenWrt package carries one transport
entry per radio, but **commented out** — so a stock install that never
runs this helper logs no per-boot "interface missing" warning.
`fips-mesh-setup` uncommented the matching `meshN` entry in Step 1, so
there is normally nothing to do here. If you maintain your own config
(or ran the manual UCI above instead of the helper), make sure the
entries are present and uncommented:
```yaml
transports:
ethernet:
mesh0:
interface: "fips-mesh0"
discovery: true
announce: true
auto_connect: true
accept_connections: true
mesh1:
interface: "fips-mesh1"
discovery: true
announce: true
auto_connect: true
accept_connections: true
```
## Step 3 — restart the daemon (order matters)
```sh
/etc/init.d/fips restart
```
Restart fips **after** the mesh interface is up. A transport whose
interface is missing at startup is logged and skipped, not retried —
so if the daemon comes up before the radio, the mesh transport stays
dead until the next restart. (An interface that *vanishes and
returns* after startup is recovered automatically; only the missing-
at-startup case needs this ordering.)
## Verify
L2 first — the 802.11s peering, with a second configured router in
range:
```sh
iw dev fips-mesh0 station dump
```
You should see one station entry per neighbor router, with signal
levels. No entries means a radio problem, not a FIPS problem — triage
in this order:
1. **Channel mismatch** (the most common cause): compare
`iw dev fips-mesh0 info` on both routers — mesh ID *and* channel
must match exactly.
2. **The mesh interface never joined** — `iw dev fips-meshX info`
shows `type mesh point` but **no channel line**, and `station dump`
is empty. Usual cause: a client (`sta`) interface on the same
radio. A STA must follow its upstream AP's channel, the whole
radio follows the STA, and a mesh pinned to a different channel
silently stays down. Check for a STA sharing the radio
(`iw dev`, look for `type managed` on the same phy), compare
`iw dev <sta-iface> info | grep channel`, and re-pin the mesh
channel to match — on every backhaul router.
3. **Is the other router transmitting at all?**
```sh
iw dev fips-mesh0 scan | grep -i -B4 "MESH ID"
```
Its mesh ID visible → transmission works, peering is failing
(mesh ID typo, or one side has encryption set). Nothing visible →
check `wifi status` on the other router, remember the ~60 s DFS
CAC wait, and confirm the country code is set
(`uci get wireless.radio1.country`) — an unset regdomain can
block channels entirely.
4. `logread | grep -iE "mesh|fips-mesh0"` on both sides.
Then the FIPS layer on top:
```sh
logread | grep -i beacon # beacons flowing on the new transport
fipsctl show peers # neighbor authenticated and connected
fipsctl show links # link on the 'ethernet' transport
```
Discovery is automatic: each node beacons its pubkey every few
seconds, and `auto_connect` initiates the Noise handshake on first
sight.
## Constraints
- **Airtime is shared per radio.** All virtual interfaces on one
radio (AP + mesh) share one channel, and multi-hop forwarding on a
single radio roughly halves throughput per hop. On dual/tri-band
hardware, dedicate one band to `fips-mesh0` and serve clients on
the others.
- **AP + mesh coexistence is driver-dependent.** It works on the
mainstream chips (this is the standard Freifunk/Gluon setup), but
check `iw list` under "valid interface combinations" for your
hardware.
- **Clients can't join.** Phones and laptops reach the mesh through
the router's normal AP or via BLE — never through the 802.11s
interface.
- **Radio links are lossy.** A neighbor at the edge of range will
form an 802.11s peering yet deliver a fraction of its frames.
Expect link-quality effects that don't exist on wired Ethernet.
- **A client (STA) uplink on the same radio owns the channel.** The
STA must follow whatever channel its upstream AP uses; every other
interface on that radio follows the STA. A mesh pinned to a
different channel silently never joins, and it does **not** recover
when the STA disconnects — a `wifi reload` (plus a fips restart) is
needed. A *roaming* uplink (travel-router / hotspot-chasing setups)
is fundamentally incompatible with a fixed-channel mesh on the same
radio: dedicate the mesh to the radio the STA never uses, and treat
any mesh sharing a STA radio as best-effort.
docs/how-to/set-up-open-access-ssid.md
+267
View File
@@ -0,0 +1,267 @@
# Set Up the Open FIPS Access SSID (OpenWrt)
Give phones and laptops a way in: every FIPS router broadcasts the
same open SSID — `!FIPS` — from its access radio. Same SSID + unique
BSSIDs is one standard ESS, so a client saves the network once and
roams between all FIPS routers natively, with no per-router setup and
no shared credentials (the Freifunk model). The leading `!` sorts the
network to the top of alphabetically ordered pickers (iOS, desktop
OSes — Android sorts by signal strength) and is part of the name:
SSIDs match byte-for-byte or not at all. The radio layer provides
nothing but open L2 to the nearest router; FIPS provides everything
else: encryption and authentication (Noise IK), discovery
(mDNS/Ethernet beacons), and mobility (the overlay identity survives
roaming, so no 802.11r or L2 tricks are needed).
This is the *access* layer — how clients reach FIPS routers. For the
router-to-router *backhaul*, see
[set-up-80211s-mesh-backhaul.md](set-up-80211s-mesh-backhaul.md).
For all `transports.ethernet.*` configuration keys, see
[../reference/configuration.md](../reference/configuration.md).
## Why open, why this addressing
Three deliberate choices distinguish this from a stock guest network:
- **`encryption none`** — the SSID is open on purpose, and it *must*
be. Clients key a saved network on SSID **plus security type**: if
one router used a PSK and another OWE, the same `FIPS` name would be
three different saved networks and roaming would break. Open is the
only security type that needs zero provisioning, and OWE is left out
for now for exactly this uniformity reason (OWE-transition mode is
inconsistent across client vendors). Every FIPS peer link is already
authenticated and encrypted by the Noise IK handshake. A stranger
can associate *and* form a FIPS peer link — that is the point of open
access; the handshake authenticates each link (no impersonation, no
MITM) but does not gate who may peer, and admission is open up to the
daemon's max-peers cap. What confines a hostile peer is the isolated
`fips_ap` zone (no path to br-lan or the WAN — see below), not the
handshake. What you concede: any nearby device can reach the FIPS
overlay surface (handshake, discovery, lookup, routing) and peer with
the router; L2 metadata is visible in the air; a hostile radio can
burn airtime — all inherent to an open radio link.
- **DHCPv4 from a fixed subnet, plus IPv6 router advertisements.**
dnsmasq leases IPv4 out of `10.21.<N>.0/24` (`N` = the radio index;
the prefix echoes FIPS port 2121). The subnet is deliberately
**identical on every router**: a roaming phone keeps its lease
across routers, and dnsmasq's authoritative mode (the OpenWrt
default, pinned by the helper) ACKs a renew the new router never
issued. odhcpd additionally announces a ULA prefix (`fd..`-range)
for stateless SLAAC; DHCPv6 stays off. FIPS itself only needs
link-local + mDNS, but Android's provisioning check requires an RA
or a DHCP offer and *disconnects* with neither, and plain laptops
expect a real IPv4 address. Works with or without an upstream —
nothing here depends on the WAN. The IPv6 side stays per-router and
disposable; in all cases the FIPS overlay identity, not the IP, is
the mobility anchor.
- **Isolated interface** — its own network and firewall zone, with no
path to `br-lan` and no forwarding to the WAN. Inbound traffic is
rejected except DHCPv4, ICMPv6 (SLAAC itself), mDNS, and the FIPS
transport ports; the raw-Ethernet transport (EtherType 0x2121) is
not IP and never traverses the firewall. AP client isolation is on, so clients
cannot reach each other at L2 — two FIPS phones on one router still
reach each other through the router at the overlay layer.
## The "no internet" behavior (expected, one-time acceptance)
The network intentionally provides **no internet**. On first connect,
a phone's validation probe fails and it asks whether to stay on a
network without internet access — choose **stay connected** and
**don't ask again**. That choice is stored per SSID, so accepting it
once covers every FIPS router anywhere.
After that, the network is marked "connected, no internet"
(unvalidated) and the phone keeps **cellular as its default route**
while staying associated — normal apps never notice the FIPS network
exists. FIPS apps bind their sockets to the Wi-Fi network explicitly,
so mesh traffic flows over Wi-Fi while everything else uses cellular.
## When to use
- Any FIPS router that should serve phones and laptops directly, not
just peer with other routers.
- You want clients to roam between FIPS routers with zero per-router
or per-site configuration.
It is the complement of the 802.11s backhaul: the backhaul links
routers (clients cannot join it), the access SSID admits clients.
Both can share a radio, at an airtime cost (see constraints).
## Requirements
- OpenWrt 22.03+ with the FIPS package installed (fw4; dnsmasq and
odhcpd are part of the default images).
- Any radio — AP mode needs no special driver support.
## Step 1 — create the access point(s)
On **each** router, run the helper once per radio that should serve
clients:
```sh
fips-ap-setup radio0
```
This creates an open AP with SSID `!FIPS` and client isolation, an
isolated network with `10.21.<N>.1/24` and a static ULA `/64`, a
DHCPv4 + RA dhcp config (dnsmasq leases, SLAAC, no DHCPv6), and a
locked-down `fips_ap` firewall zone — then reloads the radio. Interfaces are named by radio index: `radio0` →
`fips-ap0`, `radio1` → `fips-ap1`. Pass a second argument to use a
different SSID — but the SSID, like the security type, must be
identical on **all** routers or clients will treat them as separate
networks and stop roaming.
On dual-band routers, run it for both radios so clients can pick
either band:
```sh
fips-ap-setup radio0
fips-ap-setup radio1
```
**Channels are free per router.** Unlike the mesh backhaul, there is
no same-channel constraint — clients scan when they roam — so leave
each router on whatever channel suits its RF environment.
Equivalent manual UCI (per radio), if you prefer to see what it does
(`fdxx:...` stands for a `/64` out of the router's ULA prefix):
```sh
uci batch <<'EOF'
set wireless.fips_ap_radio0=wifi-iface
set wireless.fips_ap_radio0.device='radio0'
set wireless.fips_ap_radio0.mode='ap'
set wireless.fips_ap_radio0.ssid='!FIPS'
set wireless.fips_ap_radio0.encryption='none'
set wireless.fips_ap_radio0.isolate='1'
set wireless.fips_ap_radio0.ifname='fips-ap0'
set wireless.fips_ap_radio0.network='fips_ap_radio0'
set network.fips_ap_radio0=interface
set network.fips_ap_radio0.proto='static'
set network.fips_ap_radio0.ipaddr='10.21.0.1'
set network.fips_ap_radio0.netmask='255.255.255.0'
set network.fips_ap_radio0.ip6addr='fdxx:xxxx:xxxx:fa00::1/64'
set dhcp.fips_ap_radio0=dhcp
set dhcp.fips_ap_radio0.interface='fips_ap_radio0'
set dhcp.fips_ap_radio0.ra='server'
set dhcp.fips_ap_radio0.ra_default='2'
set dhcp.fips_ap_radio0.dhcpv6='disabled'
set dhcp.fips_ap_radio0.dhcpv4='server'
set dhcp.fips_ap_radio0.start='10'
set dhcp.fips_ap_radio0.limit='200'
EOF
uci commit
wifi reload
```
plus the `fips_ap` firewall zone (input/forward REJECT, no
forwardings, ACCEPT rules for DHCPv4/UDP 67, ICMPv6, UDP 5353/2121,
TCP 8443).
## Step 2 — check the FIPS transport binding
The `fips.yaml` shipped in the OpenWrt package carries one transport
entry per access interface, but **commented out** — so a stock install
that never runs this helper logs no per-boot "interface missing"
warning. `fips-ap-setup` uncommented the matching `apN` entry in Step 1,
and also enabled `node.rendezvous.lan` (the daemon's mDNS/DNS-SD
rendezvous — phone FIPS apps cannot see raw-Ethernet beacons, so mDNS
is how they find the daemon; the switch is daemon-wide and stays on if
you later remove the AP). So there is normally nothing to do here. If
you maintain your own config (or ran the manual UCI above instead of
the helper), make sure both are present and uncommented:
```yaml
node:
rendezvous:
lan:
enabled: true
```
```yaml
transports:
ethernet:
ap0:
interface: "fips-ap0"
discovery: true
announce: true
auto_connect: true
accept_connections: true
ap1:
interface: "fips-ap1"
discovery: true
announce: true
auto_connect: true
accept_connections: true
```
## Step 3 — restart the daemon (order matters)
```sh
/etc/init.d/fips restart
```
Restart fips **after** the AP interface is up. A transport whose
interface is missing at startup is logged and skipped, not retried —
so if the daemon comes up before the radio, the access transport
stays dead until the next restart. (An interface that *vanishes and
returns* after startup is recovered automatically; only the missing-
at-startup case needs this ordering.)
## Verify
L2 and addressing first, with a phone or laptop connected to `!FIPS`:
```sh
iw dev fips-ap0 station dump # one entry per associated client
ip addr show dev fips-ap0 # 10.21.0.1/24 and the fd..::1/64
cat /tmp/dhcp.leases # one lease per connected client
```
No station entries means a radio problem; an association that drops
after ~30 s usually means the client never got an address — check
`logread | grep -e dnsmasq -e odhcpd` and that the
`dhcp.fips_ap_radio0` section survived
(`uci show dhcp | grep fips_ap`).
Then the FIPS layer on top, for a client running FIPS:
```sh
logread | grep -i beacon # beacons flowing on the new transport
fipsctl show peers # client authenticated and connected
```
On the phone itself: the network shows "connected, no internet" and
stays associated — that is the designed steady state, not an error.
## Constraints
- **SSID and security type must be uniform across ALL routers.**
One router with a PSK (or OWE) under the same name splits the ESS
into different saved networks and silently breaks roaming. Never
"harden" a single router.
- **Airtime is shared per radio.** An access AP and a mesh backhaul
on the same radio share one channel. On dual/tri-band hardware,
dedicate a band to the backhaul and serve clients on the others.
- **Strangers can associate and peer — by design.** Open access means
any nearby device can complete the Noise handshake and become a FIPS
peer (up to the max-peers cap); the handshake authenticates each link,
it does not restrict who joins. They reach only the FIPS overlay
surface — the isolated zone gives no path to br-lan or the WAN. Do not
add forwardings to the `fips_ap` zone: that would turn the open SSID
into a hotspot and hand the isolation away.
- **Roaming is client-driven.** Clients decide when to hop BSSIDs
(standard ESS behavior); the IPv4 lease survives the hop (same
subnet everywhere), the SLAAC address renumbers, and FIPS sessions
ride through because the overlay identity is the anchor. Expect a
brief L2 gap during the hop, as on any ESS without 802.11r.
- **The `10.21.<N>.0/24` convention must hold everywhere.** Lease
survival depends on every router serving the same subnet from the
same radio index — the helper guarantees this; don't hand-pick
per-router subnets. Two routers can lease the same address to two
different clients; after a roam the conflict is caught (dnsmasq
NAKs a renew for an address in use) and the client re-DHCPs. If a
laptop is *also* wired to a LAN that really uses `10.21.<N>.0/24`,
its routing table will conflict — a corner case worth knowing, not
designing around: the zone forwards nowhere, so the FIPS side never
reaches beyond the router either way.
docs/reference/cli-fips.md
+1 -1
View File
@@ -28,7 +28,7 @@ controlled through the standard service control manager.
| Flag | Argument | Description |
| ---- | -------- | ----------- |
| `-c`, `--config` | `FILE` | Use `FILE` as the configuration. Skips the default search paths. |
| `-V` | — | Print the short version (e.g. `0.4.0 (rev abcdef1)`). |
| `-V` | — | Print the short version (e.g. `0.5.0-dev (rev abcdef1)`). |
| `--version` | — | Print the long version: short version plus build target triple. |
| `-h`, `--help` | — | Print usage and exit. |
| `--install-service` | — | (Windows only) Install `fips` as a Windows service. Requires Administrator. |
docs/reference/cli-fipsctl.md
+52
View File
@@ -115,6 +115,58 @@ Tell the daemon to drop a peer link.
| -------- | ----------- |
| `peer` | npub (bech32) or hostname from `/etc/fips/hosts`. |
### `profile tick <on|off|status>`
> **Reading the output.** Step durations are wall clock measured across `await`
> points, not CPU time: a step that waits on I/O accrues that wait, and other
> tasks may run inside the span. That is the intended measure for head-of-line
> delay, and it means a large step is not necessarily an expensive one.
> `arm_starvation` is measured directly as the entry time minus the deadline
> the interval scheduled that tick for. It is not derived from
> `tick_entry_gap`, which carries no starvation signal on its own: under a
> steady delay every gap is exactly one tick period.
Start, stop and inspect a capture of the rx-loop tick body. **Present
only when both `fipsctl` and the daemon are built with
`--features profiling`**; the feature is off by default, so a stock
package does not carry this subcommand and a stock daemon reports
`profile_tick_*` as an unknown command.
| Subcommand | Control-socket command | Description |
| ---------- | ---------------------- | ----------- |
| `profile tick on` | `profile_tick_on` | Create the capture file and start recording. Fails if a capture is already running (naming the active file) or if the directory cannot be written. |
| `profile tick off` | `profile_tick_off` | Stop the capture. The writer is woken immediately, drains once more and is joined, so the command returns promptly. Succeeds, reporting nothing active, when no capture is running. |
| `profile tick status` | `profile_tick_status` | Report `idle`, `running`, `stopped_by_cap` or `stopped_by_error`, plus the active path, bytes written, flush interval and byte cap. |
`profile tick on` options:
| Flag | Argument | Default | Description |
| ---- | -------- | ------- | ----------- |
| `--dir` | directory path | `/var/log/fips` | Where to write the capture. Created if absent. Use it to profile a non-root `cargo run`, or on a platform whose log root differs. |
One file is written per capture, named `profile-<UTC timestamp>.tsv`.
It opens with a `#`-prefixed header block (node npub, build version,
platform, configured tick period, flush interval, byte cap, start
time), then a tab-separated column header, then one row per measured
step per flush interval:
```text
ts_unix kind domain name count max total unit
```
`kind` is `step` for a timed span and `gauge` for a sampled scalar, so
a gauge value never lands under a duration column; `unit` names the
unit of `max` and `total` for that row. Every step present in the build
gets a row every interval, including zero-count rows. Gauges cover
ticks per interval, peer count, the wall gap between successive
tick-arm entries, and the arm-starvation delay, which is measured
against the deadline the tick was scheduled for rather than derived
from the gap.
A capture stops itself on reaching 32 MB, appending a `#` line saying
so; `profile tick status` then reports `stopped_by_cap` until the next
`on` or `off` clears it.
## Exit Codes
| Code | Meaning |
docs/reference/configuration.md
+12 -10
View File
@@ -104,7 +104,8 @@ to the highest-priority config file for operator visibility, even in ephemeral m
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.leaf_only` | bool | `false` | Leaf-only mode: node does not forward traffic or participate in routing |
| `node.disable_routing` | bool | `false` | Non-routing mode: participates in spanning tree but does not forward transit traffic or send bloom filters |
| `node.leaf_only` | bool | `false` | Leaf mode: single upstream peer, no tree/bloom/transit participation. Implies `disable_routing: true` |
| `node.tick_interval_secs` | u64 | `1` | Periodic maintenance tick interval (retry checks, timeout cleanup, tree refresh) |
| `node.base_rtt_ms` | u64 | `100` | Initial RTT estimate for new links before measurements converge |
| `node.heartbeat_interval_secs` | u64 | `10` | Heartbeat send interval per peer for liveness detection |
@@ -124,7 +125,7 @@ Controls capacity for connections, peers, and links.
### Rate Limiting (`node.rate_limit.*`)
Handshake rate limiting protects against DoS on the Noise IK handshake path.
Handshake rate limiting protects against DoS on the Noise XX handshake path.
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
@@ -304,7 +305,7 @@ stays set for all subsequent hops to the destination.
### Rekey (`node.rekey.*`)
Controls periodic Noise rekey for forward secrecy. When enabled, both FMP
(link-layer IK) and FSP (session-layer XK) sessions perform fresh Diffie-Hellman
(link-layer XX) and FSP (session-layer XX) sessions perform fresh Diffie-Hellman
key exchanges after a time or message count threshold, whichever comes first.
A 10-second drain window keeps the old session active for decryption during
cutover.
@@ -338,7 +339,7 @@ Metrics Measurement Protocol for per-peer link measurement. See
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.mmp.mode` | string | `"full"` | Operating mode: `full` (sender + receiver reports), `lightweight` (receiver reports only), or `minimal` (spin bit + CE echo only, no reports) |
| `node.mmp.mode` | string | `"full"` | Operating mode: `full` (sender + receiver reports), `lightweight` (receiver reports only), or `minimal` (CE echo only, no reports) |
| `node.mmp.log_interval_secs` | u64 | `30` | Periodic operator log interval for link metrics |
| `node.mmp.owd_window_size` | usize | `32` | One-way delay trend ring buffer size |
@@ -436,7 +437,7 @@ Requires `CAP_NET_RAW` or running as root. Linux only.
| `mtu` | u16 | *(auto)* | Override MTU. Default: interface MTU minus 3 (for frame type + length prefix) |
| `recv_buf_size` | usize | `2097152` | Socket receive buffer size in bytes (2 MB) |
| `send_buf_size` | usize | `2097152` | Socket send buffer size in bytes (2 MB) |
| `discovery` | bool | `true` | Listen for discovery beacons from other nodes |
| `listen` | bool | `true` | Listen for neighbor beacons from other nodes |
| `announce` | bool | `false` | Broadcast announcement beacons on the LAN |
| `auto_connect` | bool | `false` | Auto-connect to discovered peers |
| `accept_connections` | bool | `false` | Accept incoming connection attempts from discovered peers |
@@ -450,7 +451,7 @@ transports:
ethernet:
lan:
interface: "eth0"
discovery: true
listen: true
announce: true
backbone:
interface: "eth1"
@@ -458,7 +459,7 @@ transports:
```
Each named instance operates independently with its own socket and
discovery state. The instance name is used in log messages and the
neighbor state. The instance name is used in log messages and the
`name()` method on the Transport trait.
### TCP (`transports.tcp.*`)
@@ -840,7 +841,7 @@ peers:
### Mixed UDP + Ethernet Example
A node bridging internet peers (UDP) and a local Ethernet segment with
beacon discovery:
neighbor beacons:
```yaml
node:
@@ -856,7 +857,7 @@ transports:
mtu: 1472
ethernet:
interface: "eth0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
@@ -899,6 +900,7 @@ node:
identity:
nsec: null # secret key in nsec or hex (null = depends on persistent)
persistent: false # true = load/save fips.key; false = ephemeral each start
disable_routing: false
leaf_only: false
tick_interval_secs: 1
base_rtt_ms: 100
@@ -999,7 +1001,7 @@ transports:
# mtu: null # null = interface MTU - 3 (typically 1497)
# recv_buf_size: 2097152 # 2 MB
# send_buf_size: 2097152 # 2 MB
# discovery: true # listen for beacons
# listen: true # listen for beacons
# announce: false # broadcast beacons
# auto_connect: false # connect to discovered peers
# accept_connections: false # accept inbound handshakes
docs/reference/control-socket.md
+15
View File
@@ -159,6 +159,21 @@ not reproduced here to avoid duplicating the source.
Both commands run on the daemon's main task and may block briefly
while the node mutates its state.
#### Profiler toggle (`--features profiling` builds only)
| Command | Params | Behaviour |
| ------- | ------ | --------- |
| `profile_tick_on` | `dir` (optional directory path; default `/var/log/fips`) | Creates the capture file, publishes its path, and starts the writer thread. `data`: `state`, `path`, `interval_secs`, `byte_cap`. Errors if a capture is already running (naming the active file) or the directory is unwritable. |
| `profile_tick_off` | — | Stops the capture, drains once more, joins the writer. `data`: `state`, `stopped`, `stopped_by_cap`, `stopped_by_error`, `path`, `bytes`. |
| `profile_tick_status` | — | `data`: `state` (`idle` / `running` / `stopped_by_cap` / `stopped_by_error`), `path`, `bytes`, `byte_cap`, `interval_secs`. |
Unlike `connect` and `disconnect`, these three are served in the
control accept task rather than on the daemon's main task. All of their
state is process statics and none of them needs `&mut Node`, so
routing them through the main loop would only make the toggle queue
behind the tick body it exists to measure. They are absent from a
default build, where the daemon answers them as unknown commands.
## Gateway Command Catalog
`fips-gateway` exposes a separate control socket with its own command
docs/reference/diagrams/filter-announce.svg
+34 -25
View File
@@ -1,43 +1,52 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 270" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 374" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="270" fill="#1a1a2e" rx="4"/>
<rect width="580" height="374" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FilterAnnounce (0x20) — 11 + filter bytes</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FilterAnnounce (0x20) &#8212; 19-byte header + RLE payload</text>
<!-- Row 0 (0–3): msg_type(1) + sequence starts -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<!-- Row 0 (0-1): msg_type + flags -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;1</text>
<rect x="55" y="36" width="130" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="120" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">0x20</text>
<text x="120" y="56" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="74" fill="#8ab4f8" text-anchor="middle" font-size="10">0x20</text>
<rect x="185" y="36" width="390" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="380" y="66" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sequence</text>
<rect x="185" y="36" width="390" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="380" y="56" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="380" y="74" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte &#8212; bit 0: delta (XOR diff)</text>
<!-- Row 1 (4–8): sequence continued -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–8</text>
<!-- Row 1 (2-9): sequence -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">2&#8211;9</text>
<rect x="55" y="88" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="118" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE — monotonic counter</text>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sequence</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE &#8212; per-peer monotonic counter</text>
<!-- Row 2 (9–10): hash_count + size_class -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">9–10</text>
<!-- Row 2 (10-17): base_seq -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">10&#8211;17</text>
<rect x="55" y="140" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="185" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">hash_count</text>
<text x="185" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte</text>
<rect x="55" y="140" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">base_seq</text>
<text x="315" y="180" fill="#d8a0f8" text-anchor="middle" font-size="10">8 bytes LE &#8212; reference sequence for delta (0 if full)</text>
<rect x="315" y="140" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">size_class</text>
<text x="445" y="180" fill="#d8a0f8" text-anchor="middle" font-size="10">1 byte</text>
<!-- Row 3 (18): size_class -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">18</text>
<!-- Row 3 (11–): filter_bits -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">11–</text>
<rect x="55" y="192" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">size_class</text>
<text x="315" y="232" fill="#8af8f8" text-anchor="middle" font-size="10">1 byte &#8212; filter size = 512 &#171; size_class bytes</text>
<rect x="55" y="198" width="520" height="40" fill="#1f1f3a" stroke="#4ad9d9" stroke-width="1" stroke-dasharray="4,3" rx="3"/>
<text x="315" y="222" fill="#8af8f8" text-anchor="middle" font-size="11">filter_bits (variable, 512 &lt;&lt; size_class bytes)</text>
<!-- Row 4 (19-): compressed_payload -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">19&#8211;</text>
<rect x="55" y="250" width="520" height="40" fill="#1f1f3a" stroke="#4a9090" stroke-width="1" stroke-dasharray="4,3" rx="3"/>
<text x="315" y="274" fill="#8ad8d8" text-anchor="middle" font-size="11">compressed_payload (RLE: [count:2 LE][word:8 LE] per run)</text>
<!-- Explanation -->
<rect x="55" y="304" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="324" fill="#888" text-anchor="middle" font-size="10">delta: XOR diff of current vs last-sent filter &#8212; full: raw filter words</text>
<!-- Total -->
<text x="310" y="258" fill="#777" font-size="10" text-anchor="middle">v1 payload: 1,035 bytes (11 header + 1,024 filter)</text>
<text x="310" y="362" fill="#777" font-size="10" text-anchor="middle">19-byte header + variable compressed payload</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.6 KiB

After

Width:  |  Height:  |  Size: 3.3 KiB

docs/reference/diagrams/handshake-flow.svg
+43 -34
View File
@@ -1,26 +1,23 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 620 500" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 620 620" font-family="monospace" font-size="13">
<defs>
<marker id="arrowR" markerWidth="10" markerHeight="8" refX="9" refY="4" orient="auto" markerUnits="userSpaceOnUse">
<polygon points="0,0 10,4 0,8" fill="#e0e0e0"/>
</marker>
<marker id="arrowL" markerWidth="10" markerHeight="8" refX="1" refY="4" orient="auto" markerUnits="userSpaceOnUse">
<polygon points="10,0 0,4 10,8" fill="#e0e0e0"/>
</marker>
</defs>
<!-- Background -->
<rect width="620" height="500" fill="#1a1a2e" rx="4"/>
<rect width="620" height="620" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FMP Handshake Flow (Noise IK)</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FMP Handshake Flow (Noise XX)</text>
<!-- Column headers -->
<text x="100" y="54" fill="#8ab4f8" text-anchor="middle" font-size="12" font-weight="bold">Initiator</text>
<text x="520" y="54" fill="#8ab4f8" text-anchor="middle" font-size="12" font-weight="bold">Responder</text>
<!-- Lifelines -->
<line x1="100" y1="62" x2="100" y2="480" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<line x1="520" y1="62" x2="520" y2="480" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<line x1="100" y1="62" x2="100" y2="600" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<line x1="520" y1="62" x2="520" y2="600" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<!-- Step 1: Initiator prepares -->
<text x="20" y="88" fill="#8af8c8" font-size="10">generates sender_idx</text>
@@ -28,39 +25,51 @@
<!-- Arrow 1: msg1 (initiator -> responder) -->
<line x1="100" y1="120" x2="520" y2="120" stroke="#4a90d9" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="130" y="128" width="360" height="24" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1" rx="3"/>
<text x="310" y="144" fill="#e0e0e0" text-anchor="middle" font-size="10">[0x01|flags=0|len] | sender_idx | noise_msg1</text>
<text x="310" y="166" fill="#666" text-anchor="middle" font-size="9">phase 0x1 — 114 bytes</text>
<rect x="145" y="128" width="330" height="24" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1" rx="3"/>
<text x="310" y="144" fill="#e0e0e0" text-anchor="middle" font-size="10">[0x11|flags=0|len] | sender_idx | noise_msg1</text>
<text x="310" y="166" fill="#666" text-anchor="middle" font-size="9">phase 0x1 &#8212; 41 bytes &#8212; pattern: &#8594; e</text>
<!-- Step 2: Responder processes (right-aligned to stay in bounds) -->
<text x="600" y="192" fill="#8af8c8" font-size="10" text-anchor="end">validates msg1</text>
<text x="600" y="206" fill="#8af8c8" font-size="10" text-anchor="end">learns initiator's static key</text>
<text x="600" y="220" fill="#8af8c8" font-size="10" text-anchor="end">generates sender_idx</text>
<text x="600" y="234" fill="#8af8c8" font-size="10" text-anchor="end">generates ephemeral keypair</text>
<!-- Step 2: Responder processes msg1 -->
<text x="600" y="192" fill="#8af8c8" font-size="10" text-anchor="end">validates msg1 (ephemeral only)</text>
<text x="600" y="206" fill="#8af8c8" font-size="10" text-anchor="end">generates sender_idx</text>
<text x="600" y="220" fill="#8af8c8" font-size="10" text-anchor="end">generates ephemeral keypair</text>
<!-- Arrow 2: msg2 (responder -> initiator) -->
<!-- Draw line right-to-left so orient="auto" points the arrowhead left -->
<line x1="520" y1="256" x2="100" y2="256" stroke="#4ad99a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="130" y="264" width="360" height="24" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1" rx="3"/>
<text x="310" y="280" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x02|flags=0|len] | sender_idx | receiver_idx | noise_msg2</text>
<text x="310" y="300" fill="#666" text-anchor="middle" font-size="9">phase 0x2 — 69 bytes</text>
<line x1="520" y1="244" x2="100" y2="244" stroke="#4ad99a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="252" width="390" height="24" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1" rx="3"/>
<text x="310" y="268" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x12|flags=0|len] | sender_idx | receiver_idx | noise_msg2 + negotiation</text>
<text x="310" y="288" fill="#666" text-anchor="middle" font-size="9">phase 0x2 &#8212; 144 bytes &#8212; pattern: &#8592; e, ee, s, es</text>
<text x="600" y="300" fill="#d8a0f8" font-size="9" text-anchor="end">responder identity revealed</text>
<!-- Step 3: Initiator completes -->
<text x="20" y="324" fill="#8af8c8" font-size="10">validates msg2</text>
<text x="20" y="338" fill="#8af8c8" font-size="10">derives session keys</text>
<!-- Step 3: Initiator processes msg2, learns responder identity -->
<text x="20" y="322" fill="#8af8c8" font-size="10">validates msg2</text>
<text x="20" y="336" fill="#8af8c8" font-size="10">learns responder identity</text>
<text x="20" y="350" fill="#8af8c8" font-size="10">checks epoch (restart detection)</text>
<!-- Arrow 3: msg3 (initiator -> responder) -->
<line x1="100" y1="368" x2="520" y2="368" stroke="#d94a6a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="376" width="390" height="24" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1" rx="3"/>
<text x="310" y="392" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x13|flags=0|len] | sender_idx | receiver_idx | noise_msg3 + neg</text>
<text x="310" y="412" fill="#666" text-anchor="middle" font-size="9">phase 0x3 &#8212; 111 bytes &#8212; pattern: &#8594; s, se</text>
<text x="20" y="424" fill="#d8a0f8" font-size="9">initiator identity revealed</text>
<!-- Step 4: Responder processes msg3, learns initiator identity -->
<text x="600" y="440" fill="#8af8c8" font-size="10" text-anchor="end">validates msg3</text>
<text x="600" y="454" fill="#8af8c8" font-size="10" text-anchor="end">learns initiator identity</text>
<text x="600" y="468" fill="#8af8c8" font-size="10" text-anchor="end">checks epoch, derives session keys</text>
<!-- Handshake complete separator -->
<line x1="30" y1="360" x2="230" y2="360" stroke="#d9904a" stroke-width="1.5"/>
<text x="310" y="364" fill="#d9904a" text-anchor="middle" font-size="11" font-weight="bold">HANDSHAKE COMPLETE</text>
<line x1="390" y1="360" x2="590" y2="360" stroke="#d9904a" stroke-width="1.5"/>
<line x1="30" y1="488" x2="230" y2="488" stroke="#d9904a" stroke-width="1.5"/>
<text x="310" y="492" fill="#d9904a" text-anchor="middle" font-size="11" font-weight="bold">HANDSHAKE COMPLETE</text>
<line x1="390" y1="488" x2="590" y2="488" stroke="#d9904a" stroke-width="1.5"/>
<!-- Arrow 3: first encrypted frame -->
<text x="20" y="392" fill="#777" font-size="10">first encrypted frame:</text>
<line x1="100" y1="406" x2="520" y2="406" stroke="#d94a6a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="414" width="390" height="24" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1" rx="3"/>
<text x="310" y="430" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x00|flags|len] | receiver_idx | counter=0 | ciphertext+tag</text>
<text x="310" y="450" fill="#666" text-anchor="middle" font-size="9">phase 0x0 — established frame</text>
<!-- Arrow 4: first encrypted frame -->
<text x="20" y="516" fill="#777" font-size="10">first encrypted frame:</text>
<line x1="100" y1="530" x2="520" y2="530" stroke="#d9b04a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="538" width="390" height="24" fill="#4a3d2d" stroke="#d9b04a" stroke-width="1" rx="3"/>
<text x="310" y="554" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x00|flags|len] | receiver_idx | counter=0 | ciphertext+tag</text>
<text x="310" y="574" fill="#666" text-anchor="middle" font-size="9">phase 0x0 &#8212; established frame</text>
<!-- Legend -->
<text x="310" y="478" fill="#555" text-anchor="middle" font-size="9">Both parties hold identical symmetric keys. Epoch exchange enables restart detection.</text>
<text x="310" y="600" fill="#555" text-anchor="middle" font-size="9">Both parties hold identical symmetric keys. Epoch + negotiation exchanged in msg2/msg3.</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.2 KiB

After

Width:  |  Height:  |  Size: 5.1 KiB

docs/reference/diagrams/noise-ik-msg1.svg
-67
View File
@@ -1,67 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 426" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="426" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise IK Message 1 — phase 0x1 (114 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
<text x="102" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="150" y="36" width="95" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="197" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">phase</text>
<text x="197" y="78" fill="#8af8c8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="245" y="36" width="130" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="310" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="310" y="78" fill="#f8c88a" text-anchor="middle" font-size="10">1 byte</text>
<rect x="375" y="36" width="200" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="475" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">payload_len</text>
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx (bytes 4-7) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–7</text>
<rect x="55" y="88" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="315" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise IK msg1 label -->
<text x="10" y="250" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 250)">Noise IK msg1 (106 bytes)</text>
<line x1="18" y1="140" x2="18" y2="348" stroke="#555" stroke-width="1"/>
<line x1="18" y1="140" x2="23" y2="140" stroke="#555" stroke-width="1"/>
<line x1="18" y1="348" x2="23" y2="348" stroke="#555" stroke-width="1"/>
<!-- Rows 2-3: ephemeral_pubkey (bytes 8-40, 33 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8–40</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="170" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<rect x="55" y="192" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="222" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Rows 4-5: encrypted_static (bytes 41-89, 49 bytes) -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">41–89</text>
<rect x="55" y="244" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="274" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_static</text>
<rect x="55" y="296" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="326" fill="#f88aaa" text-anchor="middle" font-size="10">49 bytes (static key 33 + AEAD tag 16)</text>
<!-- Row 6: encrypted_epoch (bytes 90-113, 24 bytes) -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">90–113</text>
<rect x="55" y="348" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="372" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="390" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<!-- Total -->
<text x="310" y="418" fill="#777" font-size="10" text-anchor="middle">total: 114 bytes · payload_len = 110 · pattern: e, es, s, ss</text>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 4.3 KiB

docs/reference/diagrams/noise-ik-msg2.svg → docs/reference/diagrams/noise-xx-msg1.svg
+11 -33
View File
@@ -1,12 +1,12 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 374" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 218" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="374" fill="#1a1a2e" rx="4"/>
<rect width="580" height="218" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise IK Message 2 — phase 0x2 (69 bytes)</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise XX Message 1 &#8212; phase 0x1 (41 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
@@ -25,41 +25,19 @@
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx (bytes 4-7) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–7</text>
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="315" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 2: receiver_idx (bytes 8-11) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8–11</text>
<!-- Row 2: ephemeral_pubkey (bytes 8-40, 33 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8&#8211;40</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">receiver_idx</text>
<text x="315" y="182" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise IK msg2 bracket -->
<text x="10" y="270" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 270)">Noise IK msg2 (57 bytes)</text>
<line x1="18" y1="192" x2="18" y2="348" stroke="#555" stroke-width="1"/>
<line x1="18" y1="192" x2="23" y2="192" stroke="#555" stroke-width="1"/>
<line x1="18" y1="348" x2="23" y2="348" stroke="#555" stroke-width="1"/>
<!-- Rows 3-4: ephemeral_pubkey (bytes 12-44, 33 bytes) -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">12–44</text>
<rect x="55" y="192" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="222" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<rect x="55" y="244" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="274" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Row 5: encrypted_epoch (bytes 45-68, 24 bytes) -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">45–68</text>
<rect x="55" y="296" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="320" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="338" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<text x="315" y="182" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Total -->
<text x="310" y="366" fill="#777" font-size="10" text-anchor="middle">total: 69 bytes · payload_len = 65 · pattern: e, ee, se</text>
<text x="310" y="210" fill="#777" font-size="10" text-anchor="middle">total: 41 bytes &#183; payload_len = 37 &#183; pattern: &#8594; e</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.1 KiB

After

Width:  |  Height:  |  Size: 2.7 KiB

docs/reference/diagrams/noise-xx-msg2.svg
+82
View File
@@ -0,0 +1,82 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 478" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="478" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise XX Message 2 &#8212; phase 0x2 (118&#8211;144 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
<text x="102" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="150" y="36" width="95" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="197" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">phase</text>
<text x="197" y="78" fill="#8af8c8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="245" y="36" width="130" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="310" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="310" y="78" fill="#f8c88a" text-anchor="middle" font-size="10">1 byte</text>
<rect x="375" y="36" width="200" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="475" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">payload_len</text>
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx (bytes 4-7) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="185" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- receiver_idx (bytes 8-11) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end"/>
<rect x="315" y="88" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="445" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">receiver_idx</text>
<text x="445" y="130" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise XX msg2 bracket -->
<text x="10" y="290" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 290)">Noise XX msg2 (106 bytes base)</text>
<line x1="18" y1="140" x2="18" y2="400" stroke="#555" stroke-width="1"/>
<line x1="18" y1="140" x2="23" y2="140" stroke="#555" stroke-width="1"/>
<line x1="18" y1="400" x2="23" y2="400" stroke="#555" stroke-width="1"/>
<!-- Row 2: ephemeral_pubkey (bytes 12-44, 33 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12&#8211;44</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<text x="315" y="182" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Row 3: encrypted_static (bytes 45-93, 49 bytes) -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">45&#8211;93</text>
<rect x="55" y="192" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="216" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_static</text>
<text x="315" y="234" fill="#f88aaa" text-anchor="middle" font-size="10">49 bytes (static key 33 + AEAD tag 16)</text>
<!-- Row 4: encrypted_epoch (bytes 94-117, 24 bytes) -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">94&#8211;117</text>
<rect x="55" y="244" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="268" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="286" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<!-- Row 5: negotiation payload (variable, optional) -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">118+</text>
<rect x="55" y="296" width="520" height="52" fill="#2d4a4a" stroke="#4a9090" stroke-width="1.5" rx="3" stroke-dasharray="4,3"/>
<text x="315" y="320" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted negotiation payload</text>
<text x="315" y="338" fill="#8ad8d8" text-anchor="middle" font-size="10">26 bytes typical (10 payload + 16 AEAD tag)</text>
<!-- Row 6: AEAD explanation -->
<rect x="55" y="362" width="520" height="38" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="386" fill="#888" text-anchor="middle" font-size="10">negotiation appended via encrypt_payload() &#8212; extends Noise hash chain</text>
<!-- Total -->
<text x="310" y="425" fill="#777" font-size="10" text-anchor="middle">base: 118 bytes &#183; with FMP negotiation: 144 bytes</text>
<text x="310" y="442" fill="#777" font-size="10" text-anchor="middle">pattern: &#8592; e, ee, s, es</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 5.4 KiB

docs/reference/diagrams/noise-xx-msg3.svg
+72
View File
@@ -0,0 +1,72 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 426" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="426" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise XX Message 3 &#8212; phase 0x3 (85&#8211;111 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
<text x="102" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="150" y="36" width="95" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="197" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">phase</text>
<text x="197" y="78" fill="#8af8c8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="245" y="36" width="130" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="310" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="310" y="78" fill="#f8c88a" text-anchor="middle" font-size="10">1 byte</text>
<rect x="375" y="36" width="200" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="475" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">payload_len</text>
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx + receiver_idx (bytes 4-11) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;11</text>
<rect x="55" y="88" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="185" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="315" y="88" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="445" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">receiver_idx</text>
<text x="445" y="130" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise XX msg3 bracket -->
<text x="10" y="240" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 240)">Noise XX msg3 (73 bytes base)</text>
<line x1="18" y1="140" x2="18" y2="348" stroke="#555" stroke-width="1"/>
<line x1="18" y1="140" x2="23" y2="140" stroke="#555" stroke-width="1"/>
<line x1="18" y1="348" x2="23" y2="348" stroke="#555" stroke-width="1"/>
<!-- Row 2: encrypted_static (bytes 12-60, 49 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12&#8211;60</text>
<rect x="55" y="140" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_static</text>
<text x="315" y="182" fill="#f88aaa" text-anchor="middle" font-size="10">49 bytes (static key 33 + AEAD tag 16)</text>
<!-- Row 3: encrypted_epoch (bytes 61-84, 24 bytes) -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">61&#8211;84</text>
<rect x="55" y="192" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="216" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="234" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<!-- Row 4: negotiation payload (variable, optional) -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">85+</text>
<rect x="55" y="244" width="520" height="52" fill="#2d4a4a" stroke="#4a9090" stroke-width="1.5" rx="3" stroke-dasharray="4,3"/>
<text x="315" y="268" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted negotiation payload</text>
<text x="315" y="286" fill="#8ad8d8" text-anchor="middle" font-size="10">26 bytes typical (10 payload + 16 AEAD tag)</text>
<!-- Row 5: explanation -->
<rect x="55" y="310" width="520" height="38" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="334" fill="#888" text-anchor="middle" font-size="10">negotiation appended via encrypt_payload() &#8212; extends Noise hash chain</text>
<!-- Total -->
<text x="310" y="378" fill="#777" font-size="10" text-anchor="middle">base: 85 bytes &#183; with FMP negotiation: 111 bytes</text>
<text x="310" y="395" fill="#777" font-size="10" text-anchor="middle">pattern: &#8594; s, se</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.8 KiB

docs/reference/diagrams/receiver-report.svg
+58 -67
View File
@@ -3,104 +3,95 @@
<rect width="580" height="634" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">ReceiverReport (0x02) — 68 bytes</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">ReceiverReport (0x02) &#8212; 54 bytes</text>
<!-- Row 0 (0–3): msg_type + reserved -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<!-- Row 0 (0-3): msg_type + format_version + total_length -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="130" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="120" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">0x02</text>
<text x="120" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">msg_type</text>
<text x="120" y="74" fill="#8ab4f8" text-anchor="middle" font-size="10">0x02</text>
<rect x="185" y="36" width="390" height="52" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="380" y="66" fill="#666" text-anchor="middle" font-size="12">reserved (3 bytes zero)</text>
<rect x="185" y="36" width="130" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="250" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">fmt_ver</text>
<text x="250" y="74" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte</text>
<!-- Row 1 (4–11): highest_counter -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–11</text>
<rect x="315" y="36" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">total_length</text>
<text x="445" y="74" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE (= 50)</text>
<!-- Row 1 (4-7): timestamp_echo -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">highest_counter</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">timestamp_echo</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">4 bytes LE (echoed sender timestamp for RTT)</text>
<!-- Row 2 (12–19): cumulative_packets_recv -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12–19</text>
<!-- Row 2 (8-9): dwell_time -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8&#8211;9</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_recv</text>
<text x="315" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">dwell_time</text>
<text x="315" y="180" fill="#8af8f8" text-anchor="middle" font-size="10">2 bytes LE (ms between receive and echo)</text>
<!-- Row 3 (20–27): cumulative_bytes_recv -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">20–27</text>
<!-- Row 3 (10-17): highest_counter -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">10&#8211;17</text>
<rect x="55" y="192" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_bytes_recv</text>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">highest_counter</text>
<text x="315" y="232" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<!-- Row 4 (28–33): timestamp_echo + dwell_time -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">28–33</text>
<!-- Row 4 (18-25): cumulative_packets_recv -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">18&#8211;25</text>
<rect x="55" y="244" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="185" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">timestamp_echo</text>
<text x="185" y="284" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="55" y="244" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_recv</text>
<text x="315" y="284" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="315" y="244" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="445" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">dwell_time</text>
<text x="445" y="284" fill="#8af8f8" text-anchor="middle" font-size="10">2 bytes LE ms</text>
<!-- Row 5 (26-33): cumulative_bytes_recv -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">26&#8211;33</text>
<!-- Row 5 (34–39): max_burst_loss + mean_burst_loss + reserved -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">34–39</text>
<rect x="55" y="296" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="318" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_bytes_recv</text>
<text x="315" y="336" fill="#8af8f8" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="55" y="296" width="173" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="141" y="318" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">max_burst</text>
<text x="141" y="336" fill="#f88aaa" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 6 (34-41): jitter + ecn_ce_count -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">34&#8211;41</text>
<rect x="228" y="296" width="174" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="318" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">mean_burst</text>
<text x="315" y="336" fill="#f88aaa" text-anchor="middle" font-size="10">2 bytes u8.8</text>
<rect x="402" y="296" width="173" height="52" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="488" y="322" fill="#666" text-anchor="middle" font-size="10">reserved</text>
<!-- Row 6 (40–47): jitter + ecn_ce_count -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">40–47</text>
<rect x="55" y="348" width="260" height="52" fill="#4a3d2d" stroke="#d9b04a" stroke-width="1.5" rx="3"/>
<rect x="55" y="348" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="185" y="370" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">jitter</text>
<text x="185" y="388" fill="#f8d88a" text-anchor="middle" font-size="10">4 bytes LE µs</text>
<text x="185" y="388" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE (&#181;s)</text>
<rect x="315" y="348" width="260" height="52" fill="#4a3d2d" stroke="#d9b04a" stroke-width="1.5" rx="3"/>
<rect x="315" y="348" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="445" y="370" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ecn_ce_count</text>
<text x="445" y="388" fill="#f8d88a" text-anchor="middle" font-size="10">4 bytes LE</text>
<text x="445" y="388" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 7 (48–55): owd_trend + burst_loss_count -->
<text x="50" y="426" fill="#666" font-size="10" text-anchor="end">48–55</text>
<!-- Row 7 (42-49): owd_trend + burst_loss_count -->
<text x="50" y="426" fill="#666" font-size="10" text-anchor="end">42&#8211;49</text>
<rect x="55" y="400" width="260" height="52" fill="#3d4a2d" stroke="#8ad94a" stroke-width="1.5" rx="3"/>
<rect x="55" y="400" width="260" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="185" y="422" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">owd_trend</text>
<text x="185" y="440" fill="#c8f88a" text-anchor="middle" font-size="10">4 bytes i32 LE µs/s</text>
<text x="185" y="440" fill="#f8c88a" text-anchor="middle" font-size="10">i32 LE (&#181;s/s)</text>
<rect x="315" y="400" width="260" height="52" fill="#3d4a2d" stroke="#8ad94a" stroke-width="1.5" rx="3"/>
<rect x="315" y="400" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="445" y="422" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">burst_loss_count</text>
<text x="445" y="440" fill="#c8f88a" text-anchor="middle" font-size="10">4 bytes LE</text>
<text x="445" y="440" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 8 (56–63): cumulative_reorder_count + interval_packets_recv -->
<text x="50" y="478" fill="#666" font-size="10" text-anchor="end">56–63</text>
<!-- Row 8 (50-53): cumulative_reorder_count -->
<text x="50" y="478" fill="#666" font-size="10" text-anchor="end">50&#8211;53</text>
<rect x="55" y="452" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="474" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">reorder_count</text>
<text x="185" y="492" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE cumulative</text>
<rect x="55" y="452" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="474" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_reorder_count</text>
<text x="315" y="492" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="315" y="452" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="445" y="474" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">interval_pkts_recv</text>
<text x="445" y="492" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Forward compat note -->
<rect x="55" y="518" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="538" fill="#888" text-anchor="middle" font-size="10">decoders skip trailing bytes beyond total_length (forward compatibility)</text>
<!-- Row 9 (64–67): interval_bytes_recv -->
<text x="50" y="530" fill="#666" font-size="10" text-anchor="end">64–67</text>
<rect x="55" y="504" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="526" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">interval_bytes_recv</text>
<text x="185" y="544" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Removed fields note -->
<rect x="55" y="558" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="578" fill="#666" text-anchor="middle" font-size="10">removed from v0.2: max_burst_loss, mean_burst_loss, interval_packets/bytes_recv</text>
<!-- Total -->
<text x="310" y="586" fill="#777" font-size="10" text-anchor="middle">total: 68 bytes</text>
<text x="310" y="618" fill="#777" font-size="10" text-anchor="middle">total: 54 bytes &#183; format_version = 0 &#183; total_length = 50</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 6.8 KiB

After

Width:  |  Height:  |  Size: 6.3 KiB

docs/reference/diagrams/sender-report.svg
+32 -48
View File
@@ -1,66 +1,50 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 478" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 322" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="478" fill="#1a1a2e" rx="4"/>
<rect width="580" height="322" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SenderReport (0x01) — 48 bytes</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SenderReport (0x01) &#8212; 20 bytes</text>
<!-- Row 0 (0–3): msg_type + reserved -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<!-- Row 0 (0-3): msg_type + format_version + total_length -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="130" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="120" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">0x01</text>
<text x="120" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">msg_type</text>
<text x="120" y="74" fill="#8ab4f8" text-anchor="middle" font-size="10">0x01</text>
<rect x="185" y="36" width="390" height="52" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="380" y="66" fill="#666" text-anchor="middle" font-size="12">reserved (3 bytes zero)</text>
<rect x="185" y="36" width="130" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="250" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">fmt_ver</text>
<text x="250" y="74" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte</text>
<!-- Row 1 (4–11): interval_start_counter -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–11</text>
<rect x="315" y="36" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">total_length</text>
<text x="445" y="74" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE (= 16)</text>
<!-- Row 1 (4-7): interval_packets_sent -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_start_counter</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_packets_sent</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 2 (12–19): interval_end_counter -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12–19</text>
<!-- Row 2 (8-11): interval_bytes_sent -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8&#8211;11</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_end_counter</text>
<text x="315" y="180" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_bytes_sent</text>
<text x="315" y="180" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 3 (20–27): interval_start_timestamp + interval_end_timestamp -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">20–27</text>
<!-- Row 3 (12-19): cumulative_packets_sent -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">12&#8211;19</text>
<rect x="55" y="192" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="185" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_start_ts</text>
<text x="185" y="232" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="55" y="192" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_sent</text>
<text x="315" y="232" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="315" y="192" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_end_ts</text>
<text x="445" y="232" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 4 (28–31): interval_bytes_sent -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">28–31</text>
<rect x="55" y="244" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_bytes_sent</text>
<text x="315" y="284" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 5 (32–39): cumulative_packets_sent -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">32–39</text>
<rect x="55" y="296" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="318" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_sent</text>
<text x="315" y="336" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<!-- Row 6 (40–47): cumulative_bytes_sent -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">40–47</text>
<rect x="55" y="348" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="370" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_bytes_sent</text>
<text x="315" y="388" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<!-- Forward compat note -->
<rect x="55" y="258" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="278" fill="#888" text-anchor="middle" font-size="10">decoders skip trailing bytes beyond total_length (forward compatibility)</text>
<!-- Total -->
<text x="310" y="422" fill="#777" font-size="10" text-anchor="middle">total: 48 bytes</text>
<text x="310" y="314" fill="#777" font-size="10" text-anchor="middle">total: 20 bytes &#183; format_version = 0 &#183; total_length = 16</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.1 KiB

After

Width:  |  Height:  |  Size: 3.2 KiB

docs/reference/diagrams/session-ack.svg
+3 -3
View File
@@ -3,7 +3,7 @@
<rect width="580" height="322" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionAck (phase 0x2) — Noise XK msg2</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionAck (phase 0x2) &#8212; Noise XX msg2</text>
<!-- Row 0 (0–3): FSP prefix -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
@@ -40,8 +40,8 @@
<rect x="185" y="232" width="390" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="380" y="254" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">handshake_payload</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XK msg2 (57 bytes — ephemeral + epoch)</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XX msg2 (106+ bytes &#8212; ephemeral + static + epoch)</text>
<!-- Total -->
<text x="310" y="308" fill="#777" font-size="10" text-anchor="middle">typical ~190 bytes (depth-dependent, carries both endpoints' coords)</text>
<text x="310" y="308" fill="#777" font-size="10" text-anchor="middle">typical ~240 bytes (depth-dependent, carries both endpoints' coords)</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.9 KiB

After

Width:  |  Height:  |  Size: 2.9 KiB

docs/reference/diagrams/session-msg3.svg
+2 -2
View File
@@ -3,7 +3,7 @@
<rect width="580" height="244" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionMsg3 (phase 0x3) — Noise XK msg3</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionMsg3 (phase 0x3) &#8212; Noise XX msg3</text>
<!-- Row 0 (0–3): FSP prefix -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
@@ -28,7 +28,7 @@
<rect x="55" y="140" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">handshake_payload</text>
<text x="315" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XK msg3 (73 bytes — encrypted static + encrypted epoch)</text>
<text x="315" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XX msg3 (73+ bytes &#8212; encrypted static + epoch + optional negotiation)</text>
<!-- Total -->
<text x="310" y="218" fill="#777" font-size="10" text-anchor="middle">~80 bytes · no coordinates (both endpoints already have them)</text>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.2 KiB

After

Width:  |  Height:  |  Size: 2.2 KiB

docs/reference/diagrams/session-setup.svg
+2 -2
View File
@@ -3,7 +3,7 @@
<rect width="580" height="322" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionSetup (phase 0x1) — Noise XK msg1</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionSetup (phase 0x1) &#8212; Noise XX msg1</text>
<!-- Row 0 (0–3): FSP prefix -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
@@ -40,7 +40,7 @@
<rect x="185" y="232" width="390" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="380" y="254" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">handshake_payload</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XK msg1 (33 bytes — ephemeral key)</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XX msg1 (33 bytes &#8212; ephemeral key)</text>
<!-- Total -->
<text x="310" y="308" fill="#777" font-size="10" text-anchor="middle">typical ~170 bytes (depth-dependent)</text>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.9 KiB

After

Width:  |  Height:  |  Size: 2.9 KiB

docs/reference/security.md
+7 -7
View File
@@ -55,14 +55,14 @@ idempotent).
| Component | Choice | Where Used |
| --------- | ------ | ---------- |
| Curve | secp256k1 | FMP IK, FSP XK, Schnorr signatures |
| Diffie-Hellman | ECDH on secp256k1 (x-only normalized) | Noise IK, Noise XK |
| Curve | secp256k1 | FMP XX, FSP XX, Schnorr signatures |
| Diffie-Hellman | ECDH on secp256k1 (x-only normalized) | Noise XX (both layers) |
| AEAD | ChaCha20-Poly1305 | FMP link encryption, FSP session encryption |
| Hash | SHA-256 | NodeAddr derivation, Noise transcript |
| Key derivation | HKDF-SHA256 | Noise key schedule |
| Signatures | secp256k1 Schnorr | TreeAnnounce, LookupResponse proof, Nostr adverts |
| Noise pattern (link) | `Noise_IK_secp256k1_ChaChaPoly_SHA256` | FMP link layer (IK with epoch payload) |
| Noise pattern (session) | `Noise_XK_secp256k1_ChaChaPoly_SHA256` | FSP session layer (XK with epoch payload) |
| Noise pattern (link) | `Noise_XX_secp256k1_ChaChaPoly_SHA256` | FMP link layer (XX with negotiation payload) |
| Noise pattern (session) | `Noise_XX_secp256k1_ChaChaPoly_SHA256` | FSP session layer (XX with negotiation payload) |
These choices align with the Nostr cryptographic stack
(secp256k1 + ChaCha20-Poly1305 + SHA-256) and the NIP-44 encrypted
@@ -101,7 +101,7 @@ and FSP layers.
Mesh-level ACL files at `/etc/fips/peers.allow` and
`/etc/fips/peers.deny` give the operator allowlist/blocklist control
over which npubs may complete the FMP Noise IK link handshake.
over which npubs may complete the FMP Noise XX link handshake.
File format:
@@ -209,7 +209,7 @@ for the metadata-privacy model and the rejection of onion routing.
| --------- | --------------- | ------------ | ------ |
| UDP | None until `bind_addr` set | `0.0.0.0:2121` typical | Operator sets `transports.udp.bind_addr` |
| TCP | None until `bind_addr` set | None — outbound-only without bind | Operator sets `transports.tcp.bind_addr` |
| Ethernet | Listens on configured interface (raw `AF_PACKET`) | EtherType 0x2121 on selected interface | Per-flag `discovery`, `announce`, `auto_connect`, `accept_connections` |
| Ethernet | Listens on configured interface (raw `AF_PACKET`) | EtherType 0x2121 on selected interface | Per-flag `listen`, `announce`, `auto_connect`, `accept_connections` |
| Tor | None until `directory_service` configured | `127.0.0.1:8443` (loopback only) | Operator sets `transports.tor.directory_service` and configures `HiddenServiceDir` in `torrc` |
| BLE | Off by default | n/a | Operator enables `transports.ble.*` |
| Nostr discovery | Off by default | n/a (relay client, not a listener) | Operator sets `node.discovery.nostr.enabled: true` |
@@ -228,7 +228,7 @@ way to restrict inbound traffic on `fips0`. See
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) — FMP
link encryption, replay protection, rate limiting
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— FSP end-to-end encryption, Noise XK, replay window
— FSP end-to-end encryption, Noise XX, replay window
- [../how-to/enable-mesh-firewall.md](../how-to/enable-mesh-firewall.md)
— operator activation and drop-in recipes
- [configuration.md](configuration.md) — full `node.rekey.*`,
docs/reference/wire-formats.md
+220 -122
View File
@@ -24,7 +24,7 @@ The FMP link layer defines the following message types, dispatched by the
Handshake messages travel before encryption is established and are identified
by the FMP common-prefix `phase` field rather than a `msg_type` byte
(phase 0x1 = Noise IK msg1, phase 0x2 = Noise IK msg2).
(phase 0x1 = Noise XX msg1, phase 0x2 = Noise XX msg2, phase 0x3 = Noise XX msg3).
## Packet Type Summary
@@ -33,8 +33,8 @@ A higher-level summary that includes typical sizes and forwarding category:
| Message | Typical Size | When | Forwarded? |
| ------- | ------------ | ---- | ---------- |
| TreeAnnounce | Variable (depth-dependent) | Topology changes | No (peer-to-peer) |
| FilterAnnounce | ~1 KB | Topology changes | No (peer-to-peer) |
| LookupRequest | ~300 bytes | First contact, recovery | Yes (bloom-guided tree) |
| FilterAnnounce | variable (RLE compressed) | Topology changes | No (peer-to-peer) |
| LookupRequest | 44 bytes + TLV | First contact, recovery | Yes (bloom-guided tree) |
| LookupResponse | ~400 bytes | Response to discovery | Yes (reverse-path) |
| SessionDatagram + SessionSetup | ~232–402 bytes | Session establishment | Yes (routed) |
| SessionDatagram + SessionAck | ~170 bytes | Session confirmation | Yes (routed) |
@@ -63,14 +63,14 @@ stream; the common prefix `payload_len` field provides this framing
directly. TCP and Tor share a common stream reader (`tcp/stream.rs`)
that implements this framing.
**Ethernet data frame header.** The Ethernet transport prepends a 3-byte
header before the FMP payload on data frames: a 1-byte frame type
(`0x00`) followed by a 2-byte little-endian payload length. The length
field allows the receiver to trim Ethernet minimum-frame padding that
would otherwise corrupt AEAD verification. Beacon frames (`0x01`) have
no length field (fixed 34-byte payload). These bytes are consumed by the
transport layer and are not visible to FMP. The effective MTU for FMP is
the interface MTU minus three bytes (typically 1497).
**Ethernet frame header.** The Ethernet transport prepends a 4-byte
unified header before the payload: `[type:1][flags:1][length:2 LE]`.
The length field allows the receiver to trim Ethernet minimum-frame
padding that would otherwise corrupt AEAD verification. Frame types:
`0x00` (data), `0x01` (beacon). Beacons are 5 bytes total (4-byte
header + 1-byte beacon type). These bytes are consumed by the
transport layer and are not visible to FMP. The effective MTU for FMP
is the interface MTU minus four bytes (typically 1496).
## Link-Layer Formats
@@ -84,7 +84,7 @@ length.
| Field | Size | Description |
| ----- | ---- | ----------- |
| version | 4 bits (high) | Protocol version. Currently 0x0 |
| version | 4 bits (high) | Protocol version. Currently 0x1 |
| phase | 4 bits (low) | Session lifecycle phase (see table) |
| flags | 1 byte | Per-packet signal flags (zero during handshake) |
| payload_len | 2 bytes LE | Length of payload after phase-specific header, excluding AEAD tag |
@@ -94,8 +94,9 @@ length.
| Phase | Type | Description |
| ----- | ---- | ----------- |
| 0x0 | Established frame | Post-handshake encrypted traffic |
| 0x1 | Noise IK msg1 | Handshake initiation |
| 0x2 | Noise IK msg2 | Handshake response |
| 0x1 | Noise XX msg1 | Handshake initiation (ephemeral only) |
| 0x2 | Noise XX msg2 | Handshake response (responder identity + negotiation) |
| 0x3 | Noise XX msg3 | Handshake completion (initiator identity + negotiation) |
### Flags (Established Phase Only)
@@ -103,10 +104,9 @@ length.
| --- | ---- | ----------- |
| 0 | K (key epoch) | Selects active key during rekeying |
| 1 | CE | Congestion Experienced echo |
| 2 | SP (spin bit) | RTT measurement |
| 3-7 | — | Reserved (must be zero) |
| 2-7 | — | Reserved (must be zero) |
Flags must be zero in handshake packets (phase 0x1 and 0x2).
Flags must be zero in handshake packets (phase 0x1, 0x2, and 0x3).
### Established Frame (phase 0x0)
@@ -119,7 +119,7 @@ encrypted link-layer message.
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=0, flags, payload_len |
| common prefix | 4 bytes | ver=1, phase=0, flags, payload_len |
| receiver_idx | 4 bytes LE | Session index for O(1) lookup |
| counter | 8 bytes LE | Monotonic nonce, used as AEAD nonce and for replay detection |
@@ -147,67 +147,141 @@ the 1-byte message type and message-specific fields.
| Type | Message | Description |
| ---- | ------- | ----------- |
| 0x00 | SessionDatagram | Encapsulated session-layer payload for forwarding |
| 0x01 | SenderReport | MMP sender-side metrics report (48 bytes) |
| 0x02 | ReceiverReport | MMP receiver-side metrics report (68 bytes) |
| 0x01 | SenderReport | MMP sender-side metrics report (20 bytes) |
| 0x02 | ReceiverReport | MMP receiver-side metrics report (54 bytes) |
| 0x10 | TreeAnnounce | Spanning tree state announcement |
| 0x20 | FilterAnnounce | Bloom filter reachability update |
| 0x21 | FilterNack | Bloom filter delta NACK (retransmit request) |
| 0x30 | LookupRequest | Coordinate discovery request |
| 0x31 | LookupResponse | Coordinate discovery response |
| 0x50 | Disconnect | Orderly link teardown |
| 0x51 | Heartbeat | Link liveness probe |
### Noise IK Message 1 (phase 0x1)
### Noise XX Message 1 (phase 0x1)
Handshake initiation from connecting party.
Handshake initiation from connecting party. The initiator sends only its
ephemeral key — neither side's static identity is revealed in msg1.
![Noise IK message 1](diagrams/noise-ik-msg1.svg)
![Noise XX message 1](diagrams/noise-xx-msg1.svg)
Common prefix: ver=0, phase=0x1, flags=0, payload_len=110 (4 + 106).
Common prefix: ver=1, phase=0x1, flags=0, payload_len=37 (4 + 33).
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=1, flags=0, payload_len |
| common prefix | 4 bytes | ver=1, phase=1, flags=0, payload_len |
| sender_idx | 4 bytes LE | Initiator's session index (becomes receiver's `receiver_idx`) |
| noise_msg1 | 106 bytes | Noise IK first message |
| noise_msg1 | 33 bytes | Noise XX first message |
**Noise msg1 breakdown** (106 bytes):
**Noise msg1 breakdown** (33 bytes):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Initiator's ephemeral key (compressed secp256k1) |
| 33 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `-> e, es, s, ss` with epoch payload
Noise pattern: `-> e`
### Noise IK Message 2 (phase 0x2)
**Total wire size**: 41 bytes (4 prefix + 4 sender_idx + 33 noise).
Handshake response from responder.
### Noise XX Message 2 (phase 0x2)
![Noise IK message 2](diagrams/noise-ik-msg2.svg)
Handshake response from responder. The responder reveals its identity
(ephemeral key, encrypted static key, and encrypted epoch) plus an
optional protocol negotiation payload.
Common prefix: ver=0, phase=0x2, flags=0, payload_len=65 (4 + 4 + 57).
![Noise XX message 2](diagrams/noise-xx-msg2.svg)
Common prefix: ver=1, phase=0x2, flags=0, payload_len varies.
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=2, flags=0, payload_len |
| common prefix | 4 bytes | ver=1, phase=2, flags=0, payload_len |
| sender_idx | 4 bytes LE | Responder's session index |
| receiver_idx | 4 bytes LE | Echo of initiator's sender_idx from msg1 |
| noise_msg2 | 57 bytes | Noise IK second message |
| noise_msg2 | 106+ bytes | Noise XX second message (variable with negotiation) |
**Noise msg2 breakdown** (57 bytes):
**Noise msg2 breakdown** (106 bytes base):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Responder's ephemeral key (compressed secp256k1) |
| 33 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
| 33 | encrypted_static | 49 bytes | Responder's static key (33) + AEAD tag (16) |
| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `<- e, ee, se` with epoch payload
Noise pattern: `<- e, ee, s, es` with epoch and negotiation payload
After msg2, both parties derive identical symmetric session keys. The
encrypted epoch in msg1 and msg2 enables peer restart detection — if a
peer's epoch changes, the other side knows it restarted and must
re-establish the link.
**Negotiation payload** (variable, appended via `encrypt_payload()`):
encrypted negotiation bytes + AEAD tag. Minimum 26 bytes (10 payload +
16 tag) when present. See [Protocol Negotiation Payload](#protocol-negotiation-payload).
**Total wire size**: 118 bytes minimum (without negotiation), 144 bytes
typical (with FMP negotiation: 118 + 26).
After msg2, the responder's identity is known to the initiator.
### Noise XX Message 3 (phase 0x3)
Handshake completion from initiator. The initiator reveals its encrypted
static identity and epoch, plus an optional negotiation payload.
![Noise XX message 3](diagrams/noise-xx-msg3.svg)
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=1, phase=3, flags=0, payload_len |
| sender_idx | 4 bytes LE | Echo of initiator's sender_idx |
| receiver_idx | 4 bytes LE | Echo of responder's sender_idx from msg2 |
| noise_msg3 | 73+ bytes | Noise XX third message (variable with negotiation) |
**Noise msg3 breakdown** (73 bytes base):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
| 49 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `-> s, se` with epoch and negotiation payload
**Negotiation payload** (variable, appended via `encrypt_payload()`):
same format as msg2. Minimum 26 bytes when present.
**Total wire size**: 85 bytes minimum (without negotiation), 111 bytes
typical (with FMP negotiation: 85 + 26).
After msg3, both parties derive identical symmetric session keys and
have exchanged negotiation payloads. The encrypted epoch in msg2 and
msg3 enables peer restart detection — if a peer's epoch changes, the
other side knows it restarted and must re-establish the link.
### Protocol Negotiation Payload
Appended to XX msg2 and msg3 via Noise `encrypt_payload()` — extends
the Noise hash chain so negotiation data is authenticated alongside the
handshake transcript.
**Wire format**: `[format:1][versions:1][features:8][TLV entries...]`
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | format | 1 byte | Must be 0 |
| 1 | versions | 1 byte | High nibble = version_min, low nibble = version_max |
| 2 | features | 8 bytes LE | 64-bit feature bitfield |
| 10 | tlv_entries | variable | TLV extensions: `[field_num:2 LE][length:2 LE][value:N]` per entry |
**Minimum size**: 10 bytes (no TLV entries). With AEAD tag: 26 bytes on
the wire.
**FMP feature bits**:
| Bits | Name | Description |
| ---- | ---- | ----------- |
| 0-2 | Node profile | Full (0), NonRouting (1), Leaf (2) |
| 3-6 | MMP wants/provides | MMP capability negotiation |
| 7 | Bloom | Bloom filter capability |
| 8-63 | — | Reserved |
**Critical**: `decrypt_payload()` MUST be called even if the result is
discarded, to maintain hash chain consistency between the two Noise
transport channels.
### Index Semantics
@@ -269,28 +343,49 @@ includes self)
### FilterAnnounce (0x20)
Bloom filter reachability update, exchanged between direct peers only.
Supports both full sends and delta (XOR diff) updates with RLE
compression.
![FilterAnnounce](diagrams/filter-announce.svg)
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | msg_type | 1 byte | 0x20 |
| 1 | sequence | 8 bytes LE | Monotonic counter for freshness |
| 9 | hash_count | 1 byte | Number of hash functions (5 in v1) |
| 10 | size_class | 1 byte | Filter size: `512 << size_class` bytes |
| 11 | filter_bits | variable | Bloom filter bit array |
| 1 | flags | 1 byte | Bit 0: delta (XOR diff), bits 1-7 reserved |
| 2 | sequence | 8 bytes LE | Monotonic counter, per-peer (only increments on actual send) |
| 10 | base_seq | 8 bytes LE | Sequence of reference filter for delta (0 if full send) |
| 18 | size_class | 1 byte | Filter size: `512 << size_class` bytes |
| 19 | compressed_payload | variable | RLE-encoded filter or XOR diff |
**RLE format**: each run = `[count:2 LE][word:8 LE]` (10 bytes per run).
Sparse XOR diffs compress to very few runs.
**Size class table**:
| size_class | Bytes | Bits | Status |
| ---------- | ----- | ---- | ------ |
| 0 | 512 | 4,096 | Reserved |
| 1 | 1,024 | 8,192 | **v1 (MUST use)** |
| 2 | 2,048 | 16,384 | Reserved |
| 3 | 4,096 | 32,768 | Reserved |
| size_class | Bytes | Bits | Notes |
| ---------- | ----- | ---- | ----- |
| 0 | 512 | 4,096 | Minimum |
| 1 | 1,024 | 8,192 | Default |
| 2 | 2,048 | 16,384 | |
| 3 | 4,096 | 32,768 | |
| 4 | 8,192 | 65,536 | |
| 5 | 16,384 | 131,072 | |
| 6 | 32,768 | 262,144 | Maximum |
**v1 payload**: 1,035 bytes (11 header + 1,024 filter).
With link overhead: 1,072 bytes.
**Size**: variable (19-byte header + RLE-compressed payload).
### FilterNack (0x21)
Request full filter retransmission. Sent when a node receives an
out-of-sequence delta update (sequence gap detected), triggering a
full retransmit from the sender.
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | msg_type | 1 byte | 0x21 |
| 1 | expected_seq | 8 bytes LE | Sequence number the receiver expected |
**Total**: 9 bytes.
### LookupRequest (0x30)
@@ -310,16 +405,10 @@ restructuring.
| 25 | origin | 16 bytes | Requester's NodeAddr |
| 41 | ttl | 1 byte | Remaining hops (default 64) |
| 42 | min_mtu | 2 bytes LE | Minimum transport MTU the origin requires (0 = no requirement) |
| 44 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 46 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
| 44 | tlv_entries | variable | TLV extensions, forwarded verbatim by transit |
**Size**: `46 + (n x 16)` bytes, where n = origin depth + 1
| Origin Depth | Payload |
| ------------ | ------- |
| 3 | 110 bytes |
| 5 | 142 bytes |
| 10 | 222 bytes |
**Size**: 44 bytes fixed + TLV entries. TLV format:
`[field_num:2 LE][length:2 LE][value:N]` per entry.
### LookupResponse (0x31)
@@ -337,11 +426,12 @@ requester via the transit nodes that forwarded the request.
| 27 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 29 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
| 29 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target \|\| target_coords)` |
| 93 + 16n | tlv_entries | variable | TLV extensions, forwarded verbatim by transit |
**Size**: `93 + (n x 16)` bytes
**Size**: `93 + (n x 16)` bytes + TLV entries
| Target Depth | Payload |
| ------------ | ------- |
| Target Depth | Payload (no TLV) |
| ------------ | ---------------- |
| 3 | 141 bytes |
| 5 | 173 bytes |
| 10 | 253 bytes |
@@ -414,16 +504,16 @@ Sent by the frame sender to provide interval-based transmission statistics.
| Offset | Field | Size | Encoding |
| ------ | ----- | ---- | -------- |
| 0 | msg_type | 1 | `0x01` |
| 1 | reserved | 3 | Zero |
| 4 | interval_start_counter | 8 | u64 LE — first counter in this interval |
| 12 | interval_end_counter | 8 | u64 LE — last counter in this interval |
| 20 | interval_start_timestamp | 4 | u32 LE — timestamp at interval start |
| 24 | interval_end_timestamp | 4 | u32 LE — timestamp at interval end |
| 28 | interval_bytes_sent | 4 | u32 LE — payload bytes sent in interval |
| 32 | cumulative_packets_sent | 8 | u64 LE — total packets sent on this link |
| 40 | cumulative_bytes_sent | 8 | u64 LE — total bytes sent on this link |
| 1 | format_version | 1 | 0 (current) |
| 2 | total_length | 2 | u16 LE — 16 (v0 payload size) |
| 4 | interval_packets_sent | 4 | u32 LE — packets sent in interval |
| 8 | interval_bytes_sent | 4 | u32 LE — payload bytes sent in interval |
| 12 | cumulative_packets_sent | 8 | u64 LE — total packets sent on this link |
**Total: 48 bytes.**
**Total: 20 bytes.**
Decoders skip trailing bytes beyond `total_length` for forward
compatibility with future format versions.
### ReceiverReport (0x02)
@@ -434,24 +524,26 @@ Sent by the frame receiver to provide loss, jitter, and timing feedback.
| Offset | Field | Size | Encoding |
| ------ | ----- | ---- | -------- |
| 0 | msg_type | 1 | `0x02` |
| 1 | reserved | 3 | Zero |
| 4 | highest_counter | 8 | u64 LE — highest counter value received |
| 12 | cumulative_packets_recv | 8 | u64 LE — total packets received |
| 20 | cumulative_bytes_recv | 8 | u64 LE — total bytes received |
| 28 | timestamp_echo | 4 | u32 LE — echoed sender timestamp for RTT |
| 32 | dwell_time | 2 | u16 LE — time between receive and echo (ms) |
| 34 | max_burst_loss | 2 | u16 LE — largest loss burst in interval |
| 36 | mean_burst_loss | 2 | u16 LE — mean burst length (u8.8 fixed-point) |
| 38 | reserved | 2 | Zero |
| 40 | jitter | 4 | u32 LE — interarrival jitter (microseconds) |
| 44 | ecn_ce_count | 4 | u32 LE — cumulative ECN-CE marked packets |
| 48 | owd_trend | 4 | i32 LE — one-way delay trend (µs/s, signed) |
| 52 | burst_loss_count | 4 | u32 LE — number of loss bursts in interval |
| 56 | cumulative_reorder_count | 4 | u32 LE — total reordered packets |
| 60 | interval_packets_recv | 4 | u32 LE — packets received in interval |
| 64 | interval_bytes_recv | 4 | u32 LE — bytes received in interval |
| 1 | format_version | 1 | 0 (current) |
| 2 | total_length | 2 | u16 LE — 50 (v0 payload size) |
| 4 | timestamp_echo | 4 | u32 LE — echoed sender timestamp for RTT |
| 8 | dwell_time | 2 | u16 LE — time between receive and echo (ms) |
| 10 | highest_counter | 8 | u64 LE — highest counter value received |
| 18 | cumulative_packets_recv | 8 | u64 LE — total packets received |
| 26 | cumulative_bytes_recv | 8 | u64 LE — total bytes received |
| 34 | jitter | 4 | u32 LE — interarrival jitter (microseconds) |
| 38 | ecn_ce_count | 4 | u32 LE — cumulative ECN-CE marked packets |
| 42 | owd_trend | 4 | i32 LE — one-way delay trend (µs/s, signed) |
| 46 | burst_loss_count | 4 | u32 LE — number of loss bursts in interval |
| 50 | cumulative_reorder_count | 4 | u32 LE — total reordered packets |
**Total: 68 bytes.**
**Total: 54 bytes.**
Fields removed from v0.2: `max_burst_loss`, `mean_burst_loss`,
`interval_packets_recv`, `interval_bytes_recv`.
Decoders skip trailing bytes beyond `total_length` for forward
compatibility with future format versions.
## Session-Layer Message Formats
@@ -473,9 +565,9 @@ protocol version, session lifecycle phase, per-packet flags, and payload length.
| Phase | Type | Description |
| ----- | ---- | ----------- |
| 0x0 | Established | Post-handshake encrypted traffic or plaintext error signals |
| 0x1 | Handshake msg1 | SessionSetup (Noise XK msg1) |
| 0x2 | Handshake msg2 | SessionAck (Noise XK msg2) |
| 0x3 | Handshake msg3 | SessionMsg3 (Noise XK msg3) |
| 0x1 | Handshake msg1 | SessionSetup (Noise XX msg1) |
| 0x2 | Handshake msg2 | SessionAck (Noise XX msg2) |
| 0x3 | Handshake msg3 | SessionMsg3 (Noise XX msg3) |
### FSP Flags (Established Phase Only)
@@ -518,7 +610,7 @@ Transit nodes parse the CP flag and extract coordinates without decryption.
| ----- | ---- | ----------- |
| timestamp | 4 bytes LE | Session-relative milliseconds (u32) |
| msg_type | 1 byte | Session-layer message type |
| inner_flags | 1 byte | Bit 0: SP (spin bit for RTT measurement) |
| inner_flags | 1 byte | Reserved (must be zero) |
After the inner header, the remaining plaintext is the message-type-specific
body.
@@ -563,8 +655,8 @@ encrypted inner header.
### SessionSetup (phase 0x1)
Establishes a session and warms transit coordinate caches. Contains the
first message of the Noise XK handshake (ephemeral key only — the
initiator's static identity is not revealed until msg3).
first message of the Noise XX handshake (ephemeral key only — neither
side's static identity is revealed until msg2/msg3).
SessionSetup, SessionAck, and SessionMsg3 are identified by the **phase**
field in the FSP common prefix (0x1, 0x2, 0x3), not by a message type
@@ -585,12 +677,13 @@ Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len.
| ... | dest_coords_count | 2 bytes LE | Number of dest coordinate entries |
| ... | dest_coords | 16 x m bytes | Destination's ancestry |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise XK msg1 (33 bytes — ephemeral key only) |
| ... | handshake_payload | variable | Noise XX msg1 (33 bytes — ephemeral key only) |
### SessionAck (phase 0x2)
Second message of the Noise XK handshake. The responder sends its
ephemeral key and encrypted epoch.
Second message of the Noise XX handshake. The responder reveals its
identity (ephemeral key, encrypted static key, and encrypted epoch).
Optional FSP negotiation payload may be appended (omitted for rekey).
Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
![SessionAck](diagrams/session-ack.svg)
@@ -605,12 +698,13 @@ Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
| ... | dest_coords_count | 2 bytes LE | Number of initiator coordinate entries |
| ... | dest_coords | 16 x m bytes | Initiator's ancestry (for return-path cache warming) |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise XK msg2 (57 bytes — ephemeral key + encrypted epoch) |
| ... | handshake_payload | variable | Noise XX msg2 (106+ bytes — ephemeral + encrypted static + encrypted epoch + optional negotiation) |
### SessionMsg3 (phase 0x3)
Third and final message of the Noise XK handshake. The initiator reveals
its encrypted static identity and epoch. After msg3, both parties derive
Third and final message of the Noise XX handshake. The initiator reveals
its encrypted static identity and epoch. Optional FSP negotiation payload
may be appended (omitted for rekey). After msg3, both parties derive
identical symmetric session keys and the session is established.
Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len.
@@ -622,9 +716,9 @@ Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len.
| ------ | ----- | ---- | ----------- |
| 0 | flags | 1 byte | Reserved |
| 1 | handshake_len | 2 bytes LE | Noise payload length |
| 3 | handshake_payload | variable | Noise XK msg3 (73 bytes — encrypted static + encrypted epoch) |
| 3 | handshake_payload | variable | Noise XX msg3 (73+ bytes — encrypted static + encrypted epoch + optional negotiation) |
**Noise XK msg3 breakdown** (73 bytes):
**Noise XX msg3 breakdown** (73 bytes base):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
@@ -873,29 +967,33 @@ endpoint session keys).
## Size Summary
### FMP Handshake Messages (Noise IK)
### FMP Handshake Messages (Noise XX)
| Message | Raw Noise | Wire Frame |
| ------- | --------- | ---------- |
| IK msg1 (ephemeral + encrypted static + encrypted epoch) | 106 bytes | 114 bytes |
| IK msg2 (ephemeral + encrypted epoch) | 57 bytes | 69 bytes |
| XX msg1 (ephemeral only) | 33 bytes | 41 bytes |
| XX msg2 (ephemeral + encrypted static + epoch) | 106 bytes | 144 bytes (with negotiation) |
| XX msg3 (encrypted static + epoch) | 73 bytes | 111 bytes (with negotiation) |
### FSP Handshake Messages (Noise XK)
### FSP Handshake Messages (Noise XX)
| Message | Raw Noise | Notes |
| ------- | --------- | ----- |
| XK msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup |
| XK msg2 (ephemeral + encrypted epoch) | 57 bytes | Carried in SessionAck |
| XK msg3 (encrypted static + encrypted epoch) | 73 bytes | Carried in SessionMsg3 |
| XX msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup |
| XX msg2 (ephemeral + encrypted static + epoch) | 106 bytes | Carried in SessionAck |
| XX msg3 (encrypted static + epoch) | 73 bytes | Carried in SessionMsg3 |
### Link-Layer Messages (inside encrypted frame)
| Message | Size | Notes |
| ------- | ---- | ----- |
| TreeAnnounce | 100 + 32n bytes | n = depth + 1 |
| FilterAnnounce | 1,035 bytes | v1 (1KB filter) |
| LookupRequest | 46 + 16n bytes | n = origin depth + 1 |
| LookupResponse | 93 + 16n bytes | n = target depth + 1 |
| FilterAnnounce | variable | 19-byte header + RLE-compressed payload |
| FilterNack | 9 bytes | |
| LookupRequest | 44 bytes + TLV | Fixed (no longer depth-dependent) |
| LookupResponse | 93 + 16n bytes + TLV | n = target depth + 1 |
| SenderReport | 20 bytes | Extensibility header |
| ReceiverReport | 54 bytes | Extensibility header |
| SessionDatagram | 36 + payload bytes | Fixed 36-byte header |
| Disconnect | 2 bytes | |
@@ -903,13 +1001,13 @@ endpoint session keys).
| Message | Typical Size | Notes |
| ------- | ------------ | ----- |
| SessionSetup | ~170 bytes | Depth-dependent (XK msg1 = 33 bytes) |
| SessionAck | ~190 bytes | Depth-dependent, carries both endpoints' coords (XK msg2 = 57 bytes) |
| SessionMsg3 | ~80 bytes | Fixed (XK msg3 = 73 bytes, no coords) |
| SessionSetup | ~170 bytes | Depth-dependent (XX msg1 = 33 bytes) |
| SessionAck | ~240 bytes | Depth-dependent, carries both endpoints' coords (XX msg2 = 106+ bytes) |
| SessionMsg3 | ~80 bytes | Fixed (XX msg3 = 73+ bytes, no coords) |
| Data (minimal) | 12 + 6 + 4 + payload + 16 bytes | Steady state (port header included) |
| Data (with coords) | 12 + ~130 + 6 + 4 + payload + 16 bytes | Warmup/recovery (port header included) |
| SenderReport | 12 + 6 + 46 + 16 bytes | MMP metrics |
| ReceiverReport | 12 + 6 + 66 + 16 bytes | MMP metrics |
| SenderReport | 12 + 6 + 20 + 16 bytes | MMP metrics |
| ReceiverReport | 12 + 6 + 54 + 16 bytes | MMP metrics |
| PathMtuNotification | 12 + 6 + 2 + 16 bytes | MTU signal |
| CoordsWarmup | 12 + coords + 6 + 16 bytes | Standalone warmup (empty body) |
| CoordsRequired | 38 bytes | Fixed (prefix + msg_type + body) |
docs/tutorials/advertise-your-node.md
+1 -1
View File
@@ -282,7 +282,7 @@ an entry for `test-us03` (the open-discovery test mesh node).
It will have `connectivity` active and its own
`transport_addr`. This peering appeared without you
configuring anything — the test-mesh open-discovery node saw
your advert, dialed the endpoint, and Noise IK established
your advert, dialed the endpoint, and Noise XX established
the link.
If no inbound peers appear, that's not necessarily a failure
docs/tutorials/ground-up-mesh.md
+10 -10
View File
@@ -47,7 +47,7 @@ Two machines, each running `fips`, joined by a physical Ethernet
link. After the worked example:
- The two daemons have discovered each other via L2 beacons on
the link, peered over Noise IK, and brought up an FMP link.
the link, peered over Noise XX, and brought up an FMP link.
- Each `fips0` adapter has a routable mesh address; each can
ping the other by `<npub>.fips`.
- Nothing between the two machines speaks IP. The link carries
@@ -71,7 +71,7 @@ supplies the rest:
- **Addressing**: the `fips0` adapter takes an `fd97:...` ULA
derived from the npub. No DHCP. No SLAAC. The address is
cryptographically tied to the identity.
- **Discovery**: each daemon broadcasts a small beacon on the
- **Neighbor detection**: each daemon broadcasts a small beacon on the
link advertising its npub; the other daemon's listener picks
it up and dials in over the same link.
- **Routing**: the FIPS mesh layer builds its own spanning tree
@@ -177,7 +177,7 @@ different interface names — that is normal.
Edit `/etc/fips/fips.yaml` on **both** nodes. Under
`transports:`, add an `ethernet:` block. The key settings are
the four discovery flags — both nodes must opt in to all four,
the four neighbor flags — both nodes must opt in to all four,
and they default to off:
```yaml
@@ -185,7 +185,7 @@ transports:
ethernet:
interface: "<eth>" # the name from Step 1
announce: true # broadcast our beacon on the link
discovery: true # listen for beacons (default; shown for clarity)
listen: true # listen for beacons (default; shown for clarity)
auto_connect: true # dial peers we discover
accept_connections: true # accept dial-ins from peers we discover
```
@@ -194,7 +194,7 @@ Each flag does one thing:
- `announce: true` — emit a small beacon every
`beacon_interval_secs` (default 30s) carrying our npub.
- `discovery: true` — listen for incoming beacons; populate a
- `listen: true` — listen for incoming beacons; populate a
candidate-peer list keyed by source MAC and observed npub.
- `auto_connect: true` — when we see a beacon from an npub
we have not yet peered with, initiate the outbound Noise
@@ -218,7 +218,7 @@ is "all four flags on both ends."
> lan:
> interface: "eth0"
> announce: true
> discovery: true
> listen: true
> auto_connect: true
> accept_connections: true
> dongle:
@@ -227,7 +227,7 @@ is "all four flags on both ends."
> # ...
> ```
>
> Each named instance runs its own socket and discovery state.
> Each named instance runs its own socket and neighbor state.
> A single ground-up link only needs the flat form shown
> first; named instances become useful when the same node
> bridges multiple physical segments.
@@ -390,7 +390,7 @@ What you do need on the AP side:
networks and "secure" enterprise APs ship with it on.
When client isolation is on, the AP refuses to forward
station-to-station frames — the broadcast beacons never
arrive at the other node, and discovery fails silently.
arrive at the other node, and neighbor detection fails silently.
If beacons aren't crossing, this is the first thing to
check.
@@ -401,7 +401,7 @@ adapter name.
### Bluetooth LE (experimental but works)
BLE is a separate transport (`transports.ble.*`) with its own
discovery model — L2CAP advertisements rather than raw L2
neighbor-detection model — L2CAP advertisements rather than raw L2
broadcasts. The shape of the tutorial is the same (advertise +
scan + auto-connect + accept), but the prerequisites are
different: BlueZ, `bluetoothd`, an HCI adapter, and the
@@ -424,7 +424,7 @@ Windows builds skip it.
a radio link), `CAP_NET_RAW`, and a few config flags on each
end are sufficient. The mesh supplies its own identity,
addressing, discovery, and routing.
- **Discovery is a four-flag opt-in.** `announce`, `discovery`,
- **Neighbor detection is a four-flag opt-in.** `announce`, `listen`,
`auto_connect`, and `accept_connections` each control one
thing; both ends must agree before a link will form.
- **The two modes coexist.** Overlay peers and ground-up peers
docs/tutorials/ipv6-adapter-walkthrough.md
+2 -2
View File
@@ -206,8 +206,8 @@ metrics" to "I understand why each one moves the way it does":
in `show sessions` means: the proactive forward-path field, the
reactive `MtuExceeded` mechanism, the hysteresis on increase.
- [../design/fips-architecture.md](../design/fips-architecture.md)
— the two-layer encryption model: link-layer Noise IK over
each hop, end-to-end Noise XK over the session.
— the two-layer encryption model: link-layer Noise XX over
each hop, end-to-end Noise XX over the session.
## What you've learned
docs/tutorials/join-the-test-mesh.md
+6 -6
View File
@@ -27,7 +27,7 @@ UDP/2121, and is reachable from any network that permits arbitrary
outbound UDP.
> **Peer vs. node.** In FIPS terminology, a *peer* is a node
> you have a direct link to — same Noise IK handshake, same
> you have a direct link to — same Noise XX handshake, same
> transport socket. A *node* is any participant on the mesh,
> whether you peer with it directly or reach it through one or
> more hops via your peer's connections. Peering is a local
@@ -85,7 +85,7 @@ peers:
What each field does:
- `npub` — the canonical Nostr public key of `test-us01`. This is
who your daemon will mutually authenticate with over Noise IK.
who your daemon will mutually authenticate with over Noise XX.
- `alias` — a short name your daemon will use when referring to
this peer in logs and `fipsctl show peers` output. Optional.
- `addresses` — one or more transport endpoints. UDP on the
@@ -111,7 +111,7 @@ Within a few seconds you should see lines mentioning:
- An outbound connection attempt to `test-us01` or
`test-us01.fips.network:2121`
- A handshake completion (a "Noise IK link handshake complete"
- A handshake completion (a "Noise XX link handshake complete"
style line, or "peer authenticated" with the test-us01 npub)
- An MMP / link metrics entry naming `test-us01`
@@ -203,7 +203,7 @@ public test mesh, with reach to every node that mesh routes you
to. You have seen:
- **Identity.** Your daemon's ephemeral keypair authenticated to
`test-us01` over Noise IK without either side trusting anyone in
`test-us01` over Noise XX without either side trusting anyone in
advance.
- **Transports.** A UDP socket on your host carries
authenticated, encrypted mesh frames to your peer. No central
@@ -317,8 +317,8 @@ For "what just happened, in detail":
- [../design/fips-architecture.md](../design/fips-architecture.md) —
the protocol stack and the two-layer encryption model.
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) —
Noise IK link encryption, hop-by-hop forwarding.
Noise XX link encryption, hop-by-hop forwarding.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— end-to-end Noise XK, session lifecycle.
— end-to-end Noise XX, session lifecycle.
- [../design/fips-ipv6-adapter.md](../design/fips-ipv6-adapter.md) —
the TUN, the local DNS responder, MTU enforcement.
docs/tutorials/persistent-identity.md
+1 -1
View File
@@ -44,7 +44,7 @@ literal sense. Several things derive from it:
- Your `fd97:...` mesh address — derived from the public key.
- Your `<npub>.fips` DNS name — the npub itself with `.fips`
appended.
- Every authenticated connection — Noise IK at the mesh layer,
- Every authenticated connection — Noise XX at the mesh layer,
XK at the session layer, both prove you hold the matching
secret key.
docs/tutorials/reach-mesh-services.md
+2 -2
View File
@@ -245,9 +245,9 @@ For "what's actually in those packets":
- [../design/fips-architecture.md](../design/fips-architecture.md)
— the protocol stack and the two-layer encryption model.
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) —
Noise IK link encryption, hop-by-hop forwarding.
Noise XX link encryption, hop-by-hop forwarding.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— end-to-end Noise XK between source and destination.
— end-to-end Noise XX between source and destination.
For the trace-it-yourself version of the path you just
exercised, see
docs/tutorials/resolve-peers-via-nostr.md
+1 -1
View File
@@ -178,7 +178,7 @@ The resolution itself happens at debug-log level, so you will
not see it in the default-level journal. The user-facing way to
confirm everything worked is `fipsctl show peers` in the next
step. (To watch the resolution in the journal, run the daemon
manually with `RUST_LOG=fips::discovery::nostr=debug`; not
manually with `RUST_LOG=fips::nostr=debug`; not
necessary for this tutorial.)
## Step 5: Verify the resolved endpoint
packaging/common/fips.yaml
+18 -9
View File
@@ -10,12 +10,21 @@ node:
#
# Or set an explicit key (overrides persistent):
# nsec: "nsec1..."
discovery:
# Optional Nostr-mediated overlay endpoint discovery.
# Mesh-lookup protocol (node.lookup.*): the overlay coordinate-lookup engine
# (mesh address -> coordinates). Defaults shown; uncomment to override.
# lookup:
# ttl: 64
# attempt_timeouts_secs: [1, 2, 4, 8]
# recent_expiry_secs: 10
# backoff_base_secs: 0
# backoff_max_secs: 0
# forward_min_interval_secs: 2
rendezvous:
# Optional Nostr-mediated overlay endpoint rendezvous.
# nostr:
# enabled: true
# policy: configured_only # disabled | configured_only | open
# open_discovery_max_pending: 64 # caps queued open-discovery retries
# open_discovery_max_pending: 64 # caps queued open-rendezvous retries
# app: "fips-overlay-v1"
# advertise: true
# advert_relays:
@@ -34,17 +43,17 @@ node:
# - "stun:stun.cloudflare.com:3478"
# - "stun:global.stun.twilio.com:3478"
#
# Optional mDNS-based LAN discovery for sub-second same-LAN pairing.
# Optional mDNS-based LAN rendezvous for sub-second same-LAN pairing.
# Opt-in (default false): default-off avoids a per-LAN identity
# broadcast on nodes that have deliberately disabled other discovery
# broadcast on nodes that have deliberately disabled other rendezvous
# channels, and avoids any multicast surprise on upgrade. Requires an
# operational UDP transport (the advertised port is the one peers dial).
# lan:
# enabled: false
# # Optional application/network scope carried in the LAN-only TXT
# # record. Browsers that set a scope ignore adverts for other scopes.
# # Kept separate from the Nostr discovery `app` tag so relay-visible
# # adverts can stay generic while LAN discovery stays per-private-network.
# # Kept separate from the Nostr rendezvous `app` tag so relay-visible
# # adverts can stay generic while LAN rendezvous stays per-private-network.
# # scope: "lab-floor-3"
# # Advanced: overrides the mDNS service type. Leave unset in normal
# # use — only needed to run multiple isolated services on one
@@ -89,7 +98,7 @@ transports:
# Ethernet transport — uncomment and set your interface name.
# ethernet:
# interface: "eth0"
# discovery: true
# listen: true
# announce: true
# auto_connect: true
# accept_connections: true
@@ -147,5 +156,5 @@ peers: []
# - transport: udp
# addr: "test-us01.fips.network:2121" # IP or hostname (e.g., "peer.example.com:2121")
# - transport: udp
# addr: "nat" # Use node.discovery.nostr for Nostr/STUN hole punching
# addr: "nat" # Use node.rendezvous.nostr for Nostr/STUN hole punching
# connect_policy: auto_connect
packaging/debian/build-deb.sh
+44 -2
View File
@@ -2,6 +2,7 @@
# Build a .deb package for FIPS using cargo-deb.
#
# Usage: ./build-deb.sh [--target <triple>] [--version <version>] [--no-build]
# [--features <list>]
#
# Prerequisites: cargo-deb (install with: cargo install cargo-deb)
# Output: deploy/fips_<version>_<arch>.deb
@@ -19,6 +20,9 @@ Options:
--target <triple> Rust target triple to build/package
--version <version> Override Debian package version
--no-build Package existing binaries without running cargo build
--features <list> Cargo features to build with (comma-separated). Marks the
auto-derived Version so the package is distinguishable
from a default build of the same commit.
-h, --help Show this help
EOF
}
@@ -26,6 +30,7 @@ EOF
TARGET_TRIPLE=""
VERSION_OVERRIDE=""
NO_BUILD=0
FEATURES=""
while [[ $# -gt 0 ]]; do
case "$1" in
@@ -41,6 +46,10 @@ while [[ $# -gt 0 ]]; do
NO_BUILD=1
shift
;;
--features)
FEATURES="${2:?missing value for --features}"
shift 2
;;
-h|--help)
usage
exit 0
@@ -53,6 +62,16 @@ while [[ $# -gt 0 ]]; do
esac
done
# A feature build that skips the build step would stamp a feature-marked Version
# onto whatever binaries already sit in target/, which is the one outcome the
# marking exists to prevent. Refuse rather than emit a package that misdescribes
# itself.
if [[ -n "${FEATURES}" && "${NO_BUILD}" -eq 1 ]]; then
echo "--features cannot be combined with --no-build: the features would not" >&2
echo "reach the binaries, but the Version would claim they had." >&2
exit 1
fi
cd "${PROJECT_ROOT}"
# Ensure cargo-deb is available
@@ -81,13 +100,33 @@ if [[ -z "${VERSION_OVERRIDE}" ]]; then
if [[ -n "$(git status --porcelain 2>/dev/null)" ]]; then
DIRTY_SUFFIX=".dirty"
fi
# Debian Version: <upstream>~dev+git<YYYYMMDD>.<sha>[.dirty]-1
# A feature build of a given commit is a different package from the
# default build of that same commit, but nothing else in this version
# says so: the crate version, the date and the sha are all identical.
# Without a marker the two are byte-identical versions, so installing
# one over the other is an apt no-op (the very failure the per-commit
# version above exists to prevent) and the node offers no way to tell
# which one it is running. Underscores and commas are not legal in a
# Debian version, so the feature list is folded to dots.
FEATURE_SUFFIX=""
if [[ -n "${FEATURES}" ]]; then
FEATURE_SUFFIX="+$(printf '%s' "${FEATURES}" | tr -c 'a-zA-Z0-9.' '.')"
fi
# Debian Version: <upstream>~dev+git<YYYYMMDD>.<sha>[.dirty][+<features>]-1
# The "~" makes every dev build sort BEFORE the eventual tagged
# release; the date+sha makes consecutive dev builds compare as
# different versions; the trailing "-1" is the Debian revision.
VERSION_OVERRIDE="${BASE_VERSION}~dev+git${GIT_DATE}.${GIT_SHA}${DIRTY_SUFFIX}-1"
# The feature suffix sorts ABOVE the unsuffixed build, so installing a
# feature build is an upgrade and reverting to the default build is a
# downgrade — which apt refuses without being told to, and `dpkg -i`
# performs. Revert with `dpkg -i`, not `apt install`.
VERSION_OVERRIDE="${BASE_VERSION}~dev+git${GIT_DATE}.${GIT_SHA}${DIRTY_SUFFIX}${FEATURE_SUFFIX}-1"
echo "Auto-derived dev Version: ${VERSION_OVERRIDE}"
fi
elif [[ -n "${FEATURES}" ]]; then
echo "Warning: --version was given with --features, so the Version carries no" >&2
echo "feature marker and this package is indistinguishable from a default" >&2
echo "build of the same commit. Mark it yourself if that matters." >&2
fi
# Build the .deb package
@@ -105,6 +144,9 @@ fi
if [[ "${NO_BUILD}" -eq 1 ]]; then
cargo_args+=(--no-build)
fi
if [[ -n "${FEATURES}" ]]; then
cargo_args+=(--features "${FEATURES}")
fi
cargo "${cargo_args[@]}"
# Move output to deploy/
packaging/debian/fips.service
+5
View File
@@ -19,6 +19,11 @@ RestartSec=5
RuntimeDirectory=fips
RuntimeDirectoryMode=0750
# Log directory (/var/log/fips/), where the built-in tick-body profiler writes
# its capture files. Declared so systemd creates it on start and removes it on
# purge; the daemon runs as root and already has access without it.
LogsDirectory=fips
# Security hardening (daemon runs as root for TUN and raw sockets)
ProtectHome=yes
PrivateTmp=yes
packaging/openwrt-apk/build-apk.sh
+2
View File
@@ -182,6 +182,8 @@ install -m 0755 "$RELEASE_DIR/fips" "$STAGE_DIR/usr/bin/fips"
install -m 0755 "$RELEASE_DIR/fipsctl" "$STAGE_DIR/usr/bin/fipsctl"
install -m 0755 "$RELEASE_DIR/fipstop" "$STAGE_DIR/usr/bin/fipstop"
install -m 0755 "$RELEASE_DIR/fips-gateway" "$STAGE_DIR/usr/bin/fips-gateway"
install -m 0755 "$FILES_DIR/usr/bin/fips-mesh-setup" "$STAGE_DIR/usr/bin/fips-mesh-setup"
install -m 0755 "$FILES_DIR/usr/bin/fips-ap-setup" "$STAGE_DIR/usr/bin/fips-ap-setup"
install -d "$STAGE_DIR/etc/init.d"
install -m 0755 "$FILES_DIR/etc/init.d/fips" "$STAGE_DIR/etc/init.d/fips"
packaging/openwrt-ipk/Makefile
+6
View File
@@ -96,6 +96,12 @@ define Package/fips/install
$(INSTALL_BIN) $(RUST_RELEASE_DIR)/fipstop $(1)/usr/bin/fipstop
$(INSTALL_BIN) $(RUST_RELEASE_DIR)/fips-gateway $(1)/usr/bin/fips-gateway
# 802.11s mesh backhaul setup helper
$(INSTALL_BIN) $(CURDIR)/files/usr/bin/fips-mesh-setup $(1)/usr/bin/fips-mesh-setup
# Open "FIPS" access SSID setup helper
$(INSTALL_BIN) $(CURDIR)/files/usr/bin/fips-ap-setup $(1)/usr/bin/fips-ap-setup
# procd init script
$(INSTALL_DIR) $(1)/etc/init.d
$(INSTALL_BIN) $(CURDIR)/files/etc/init.d/fips $(1)/etc/init.d/fips
packaging/openwrt-ipk/README.md
+1
View File
@@ -14,6 +14,7 @@ For ad-hoc deployment without the build system, see
| `/usr/bin/fipsctl` | CLI control tool (`fipsctl show peers`, `fipsctl show links`, …) |
| `/usr/bin/fipstop` | Live TUI dashboard |
| `/usr/bin/fips-gateway` | Outbound LAN gateway service (not started by default) |
| `/usr/bin/fips-mesh-setup` | Opt-in helper — creates an open 802.11s mesh interface for router↔router backhaul |
| `/etc/init.d/fips` | procd service for the daemon (auto-start, crash respawn) |
| `/etc/init.d/fips-gateway` | procd service for the gateway (disabled by default) |
| `/etc/fips/fips.yaml` | Node configuration (edit before first start) |
packaging/openwrt-ipk/build-ipk.sh
+2
View File
@@ -161,6 +161,8 @@ install -m 0755 "$RELEASE_DIR/fips" "$DATA_DIR/usr/bin/fips"
install -m 0755 "$RELEASE_DIR/fipsctl" "$DATA_DIR/usr/bin/fipsctl"
install -m 0755 "$RELEASE_DIR/fipstop" "$DATA_DIR/usr/bin/fipstop"
install -m 0755 "$RELEASE_DIR/fips-gateway" "$DATA_DIR/usr/bin/fips-gateway"
install -m 0755 "$FILES_DIR/usr/bin/fips-mesh-setup" "$DATA_DIR/usr/bin/fips-mesh-setup"
install -m 0755 "$FILES_DIR/usr/bin/fips-ap-setup" "$DATA_DIR/usr/bin/fips-ap-setup"
install -d "$DATA_DIR/etc/init.d"
install -m 0755 "$FILES_DIR/etc/init.d/fips" "$DATA_DIR/etc/init.d/fips"
packaging/openwrt-ipk/files/etc/fips/fips.yaml
+79 -8
View File
@@ -10,12 +10,21 @@ node:
#
# Or set an explicit key (overrides persistent):
# nsec: "nsec1..."
discovery:
# Optional Nostr-mediated overlay endpoint discovery.
# Mesh-lookup protocol (node.lookup.*): the overlay coordinate-lookup engine
# (mesh address -> coordinates). Defaults shown; uncomment to override.
# lookup:
# ttl: 64
# attempt_timeouts_secs: [1, 2, 4, 8]
# recent_expiry_secs: 10
# backoff_base_secs: 0
# backoff_max_secs: 0
# forward_min_interval_secs: 2
rendezvous:
# Optional Nostr-mediated overlay endpoint rendezvous.
# nostr:
# enabled: true
# policy: configured_only # disabled | configured_only | open
# open_discovery_max_pending: 64 # caps queued open-discovery retries
# open_discovery_max_pending: 64 # caps queued open-rendezvous retries
# app: "fips-overlay-v1"
# advertise: true
# advert_relays:
@@ -34,6 +43,14 @@ node:
# - "stun:stun.cloudflare.com:3478"
# - "stun:global.stun.twilio.com:3478"
# mDNS/DNS-SD peer rendezvous on the local link. Ships commented (the
# daemon default is off); 'fips-ap-setup' uncomments it when creating
# the access SSID — phone FIPS apps cannot see raw-Ethernet beacons,
# so mDNS is how they find this router's daemon. Daemon-wide switch,
# left enabled on 'fips-ap-setup remove'.
# lan:
# enabled: true
tun:
enabled: true
name: fips0
@@ -55,7 +72,11 @@ dns:
transports:
udp:
bind_addr: "0.0.0.0:2121"
# Dual-stack wildcard, not "0.0.0.0": access-SSID clients (phones) learn
# this node's addresses from the mDNS advert and prefer the IPv6
# link-local — a v4-only bind silently drops their Noise msg1.
# OpenWrt is Linux (bindv6only=0), so "[::]" accepts v4 too.
bind_addr: "[::]:2121"
# advertise_on_nostr: true
# public: false # false => advertise udp:nat; true => advertise bound host:port
# accept_connections: true # default; refuse inbound msg1 when false
@@ -74,23 +95,73 @@ transports:
ethernet:
wan:
interface: "eth0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
wwan:
interface: "phy0-sta0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
lan:
interface: "br-lan"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
# 802.11s mesh backhaul between FIPS routers. These entries ship
# commented out so a stock install that never creates fips-mesh*
# logs no per-boot "interface missing" bind warning. Running
# 'fips-mesh-setup <radio>' creates the interface AND uncomments the
# matching block here (once per radio; radio0 -> fips-mesh0, radio1 ->
# fips-mesh1); 'fips-mesh-setup remove' re-comments it. Restart fips
# after — a transport whose interface is missing at startup is skipped,
# not retried. Dual-band routers can mesh on both bands at once —
# failover, not multipath: FIPS keeps one active link per peer, the
# other band stands by. The mesh runs OPEN (no SAE) with 802.11s
# forwarding off: FIPS's Noise handshake is the encryption and
# authentication, and FIPS is the routing layer. See
# docs/how-to/set-up-80211s-mesh-backhaul.md.
# mesh0:
# interface: "fips-mesh0"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# mesh1:
# interface: "fips-mesh1"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# Open "!FIPS" access SSID for phones and laptops running FIPS. These
# entries ship commented out so a stock install that never creates
# fips-ap* logs no per-boot "interface missing" bind warning. Running
# 'fips-ap-setup <radio>' creates the interface AND uncomments the
# matching block here (once per radio; radio0 -> fips-ap0, radio1 ->
# fips-ap1); 'fips-ap-setup remove' re-comments it. Restart fips after
# — a transport whose interface is missing at startup is skipped, not
# retried. The SSID is OPEN and isolated on purpose: FIPS's Noise
# handshake is the only security layer, and associated clients reach
# nothing but the FIPS handshake surface. See
# docs/how-to/set-up-open-access-ssid.md.
# ap0:
# interface: "fips-ap0"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# ap1:
# interface: "fips-ap1"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# Bluetooth Low Energy transport — requires BlueZ and the 'ble' feature.
# ble:
# adapter: "hci0"
@@ -121,5 +192,5 @@ peers: []
# - transport: udp
# addr: "test-us01.fips.network:2121" # IP or hostname (e.g., "peer.example.com:2121")
# - transport: udp
# addr: "nat" # Use node.discovery.nostr for Nostr/STUN hole punching
# addr: "nat" # Use node.rendezvous.nostr for Nostr/STUN hole punching
# connect_policy: auto_connect
packaging/openwrt-ipk/files/usr/bin/fips-ap-setup
Executable
+412
View File
@@ -0,0 +1,412 @@
#!/bin/sh
# fips-ap-setup — configure the open "FIPS" access SSID for phones/laptops.
#
# Usage:
# fips-ap-setup <radio> [ssid] e.g. fips-ap-setup radio0
# fips-ap-setup remove [radio] no radio: remove all instances
#
# Creates an open AP on the given radio so client devices running FIPS can
# reach the router. Every FIPS router broadcasts the SAME SSID ("!FIPS" by
# default — the leading '!' sorts it to the top of alphabetically ordered
# network pickers): same SSID + unique BSSIDs is one standard ESS, so a
# phone saves the network once and roams between all FIPS routers natively.
#
# - encryption 'none' — the AP is OPEN on purpose. FIPS's Noise IK
# handshake authenticates and encrypts everything above the radio, and
# the security type must be uniform across ALL routers anyway: clients
# key a saved network on SSID + security type, so one router with a PSK
# splits the ESS into a different saved network. A stranger can
# associate AND form a FIPS peer link — that is the point of open
# access. The Noise handshake authenticates each link (no
# impersonation of another identity, no MITM); it does NOT gate who
# may peer. Admission is open up to the daemon's max-peers cap; the
# firewall zone below is what confines every client to the FIPS
# overlay (no path to br-lan or the WAN).
# - DHCPv4 + RA IPv6 — dnsmasq serves DHCPv4 from a FIXED subnet,
# 10.21.<N>.0/24 (echoes FIPS port 2121), identical on every router:
# a roaming phone keeps its lease across routers, and dnsmasq's
# authoritative mode (the OpenWrt default) ACKs the renew a foreign
# router never issued. odhcpd additionally announces a ULA prefix in
# router advertisements (stateless SLAAC); DHCPv6 stays off. FIPS
# itself only needs link-local + mDNS, but client provisioning checks
# (Android disconnects without an RA or a DHCP offer) and plain
# laptops both want a real address. The network provides no internet,
# so phones mark it unvalidated and keep cellular as the default
# route while staying associated.
# - ISOLATED — own network and firewall zone: no path to
# br-lan, no forwarding to the WAN, and AP client isolation on.
# Associated clients reach only the FIPS handshake surface.
#
# Interfaces are named per radio index (radio0 -> fips-ap0, radio1 ->
# fips-ap1). Unlike the 802.11s backhaul there is NO same-channel
# constraint — clients scan when they roam, so every router picks its
# access channels freely.
#
# The shipped /etc/fips/fips.yaml carries 'ap0' and 'ap1' entries under
# 'transports.ethernet' bound to these names, but commented out — a stock
# install that never creates fips-ap* then logs no bind warning. This
# helper uncomments the matching entry when it creates an interface and
# re-comments it on remove, so the daemon binds the transport without a
# manual config edit. It also uncomments the node.rendezvous.lan block
# (mDNS/DNS-SD — how phone FIPS apps discover the daemon); that switch is
# daemon-wide and stays on at remove. After an interface is up, restart
# fips.
# See docs/how-to/set-up-open-access-ssid.md for the full guide.
DEFAULT_SSID="!FIPS"
CONFIG="/etc/fips/fips.yaml"
# Replace $CONFIG with the rewritten $CONFIG.tmp. Force mode 0600 first: the
# package installs fips.yaml 0600 (it may hold an inline 'nsec' private key),
# and a fresh tmp file would otherwise land world-readable after the move.
ap_config_write() {
chmod 600 "$CONFIG.tmp" && mv "$CONFIG.tmp" "$CONFIG"
}
# Uncomment the 'ap<idx>' transports.ethernet block in $CONFIG (created by
# 'fips-ap-setup'). Reversible with ap_config_disable. Returns:
# 0 enabled (or already active) 1 no config file 2 no such block
ap_config_enable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ ap$idx:" "$CONFIG" && return 0
grep -q "^ # ap$idx:" "$CONFIG" || return 2
awk -v idx="$idx" '
$0 ~ ("^ # ap" idx ":[ \t]*$") { blk = 1; sub(/^ # /, " "); print; next }
blk && /^ # / { sub(/^ # /, " "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && ap_config_write
}
# Uncomment the 'lan' block under node.rendezvous in $CONFIG — the daemon's
# mDNS/DNS-SD responder+browser. Phone FIPS apps cannot open raw-Ethernet
# sockets, so mDNS is how they find this router's daemon. The match is
# scoped to node.rendezvous: transports.ethernet also has a 'lan' entry at
# the same indent. Daemon-wide switch — enabled here, deliberately NOT
# re-commented on remove (other transports use it once on). Returns:
# 0 enabled (or already active) 1 no config file 2 no such block
lan_rendezvous_enable() {
[ -f "$CONFIG" ] || return 1
state="$(awk '
/^[A-Za-z_]/ { top = $1 }
top == "node:" && /^ [A-Za-z_]/ { sec = $1 }
top == "node:" && sec == "rendezvous:" && /^ lan:[ \t]*$/ { print "active"; exit }
top == "node:" && sec == "rendezvous:" && /^ # lan:[ \t]*$/ { print "commented"; exit }
' "$CONFIG")"
case "$state" in
active) return 0 ;;
commented) ;;
*) return 2 ;;
esac
awk '
/^[A-Za-z_]/ { top = $1 }
top == "node:" && /^ [A-Za-z_]/ { sec = $1 }
top == "node:" && sec == "rendezvous:" && $0 ~ /^ # lan:[ \t]*$/ { blk = 1; sub(/^ # /, " "); print; next }
blk && /^ # / { sub(/^ # /, " "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && ap_config_write
}
# Re-comment the 'ap<idx>' block so the daemon stops binding it (and stops
# warning about the now-missing interface). Inverse of ap_config_enable.
ap_config_disable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ ap$idx:" "$CONFIG" || return 0
awk -v idx="$idx" '
$0 ~ ("^ ap" idx ":[ \t]*$") { blk = 1; sub(/^ /, " # "); print; next }
blk && /^ / { sub(/^ /, " # "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && ap_config_write
}
usage() {
echo "Usage: fips-ap-setup <radio> [ssid]" >&2
echo " fips-ap-setup remove [radio]" >&2
echo "Radios on this device:" >&2
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\([^.]*\)=wifi-device$/ \1/p" >&2
exit 1
}
# List the UCI section names of fips-managed access-point wifi-ifaces.
ap_sections() {
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\(fips_ap[^.=]*\)=wifi-iface$/\1/p"
}
# ---------------------------------------------------------------------------
# remove [radio] — delete the wireless, network, dhcp, and firewall sections
# created below
# ---------------------------------------------------------------------------
if [ "$1" = "remove" ]; then
if [ -n "$2" ]; then
SECTIONS="fips_ap_$(printf '%s' "$2" | tr -c 'a-zA-Z0-9_' '_')"
else
SECTIONS="$(ap_sections)"
fi
[ -n "$SECTIONS" ] || {
echo "No fips access-point instances configured."
exit 0
}
for section in $SECTIONS; do
ifname="$(uci -q get "wireless.$section.ifname")"
uci -q delete "wireless.$section"
uci -q delete "network.$section"
uci -q delete "dhcp.$section"
uci -q del_list "firewall.fips_ap.network=$section"
# Re-comment the matching ap<N> transport in fips.yaml so the
# daemon stops warning about the interface we just removed.
idx="$(printf '%s' "$ifname" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$idx" ] && ap_config_disable "$idx"
echo "Removed ${ifname:-$section}."
done
# Drop the shared zone and its rules once the last instance is gone.
if [ -z "$(uci -q get firewall.fips_ap.network)" ]; then
uci -q delete firewall.fips_ap
uci -q delete firewall.fips_ap_icmpv6
uci -q delete firewall.fips_ap_dhcpv4
uci -q delete firewall.fips_ap_mdns
uci -q delete firewall.fips_ap_fips_udp
uci -q delete firewall.fips_ap_fips_tcp
fi
uci commit wireless
uci commit network
uci commit dhcp
uci commit firewall
wifi reload
/etc/init.d/dnsmasq reload
/etc/init.d/odhcpd reload
/etc/init.d/firewall reload
echo "Restart fips: /etc/init.d/fips restart"
exit 0
fi
RADIO="$1"
SSID="${2:-$DEFAULT_SSID}"
[ -n "$RADIO" ] || usage
if [ "$(uci -q get "wireless.$RADIO")" != "wifi-device" ]; then
echo "Error: '$RADIO' is not a wifi-device in /etc/config/wireless." >&2
usage
fi
# One instance per radio: section fips_ap_<radio>, netdev fips-ap<N>
# where N is the radio's trailing index (radio0 -> fips-ap0). For radios
# named without a trailing number, fall back to the first free index.
SECTION="fips_ap_$(printf '%s' "$RADIO" | tr -c 'a-zA-Z0-9_' '_')"
IDX="$(printf '%s' "$RADIO" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$IDX" ] || IDX="$(printf '%s' "$RADIO" | sed -n 's/^\([0-9]\{1,\}\)$/\1/p')"
if [ -z "$IDX" ]; then
IDX=0
while uci show wireless 2>/dev/null | grep -q "\.ifname='fips-ap$IDX'"; do
IDX=$((IDX + 1))
done
fi
AP_IFNAME="fips-ap$IDX"
# Refuse a name collision from another radio's instance (e.g. two radios
# whose names end in the same digit) rather than silently hijacking it.
OWNER="$(uci show wireless 2>/dev/null \
| sed -n "s/^wireless\.\(fips_ap[^.=]*\)\.ifname='$AP_IFNAME'$/\1/p")"
if [ -n "$OWNER" ] && [ "$OWNER" != "$SECTION" ]; then
echo "Error: $AP_IFNAME is already used by section '$OWNER'." >&2
echo "Remove it first: fips-ap-setup remove" >&2
exit 1
fi
# ---------------------------------------------------------------------------
# Wireless: open AP with client isolation. Clients of the same AP cannot
# exchange L2 frames directly — two FIPS phones on one router still reach
# each other through the router at the overlay layer. Isolation is an L2
# control only: a stranger who peers is an overlay peer like any other, so
# the FIPS overlay (not L2) is the trust boundary between clients.
# ---------------------------------------------------------------------------
uci -q delete "wireless.$SECTION"
uci set "wireless.$SECTION=wifi-iface"
uci set "wireless.$SECTION.device=$RADIO"
uci set "wireless.$SECTION.mode=ap"
uci set "wireless.$SECTION.ssid=$SSID"
uci set "wireless.$SECTION.encryption=none"
uci set "wireless.$SECTION.isolate=1"
uci set "wireless.$SECTION.ifname=$AP_IFNAME"
uci set "wireless.$SECTION.network=$SECTION"
# Radios ship disabled on fresh OpenWrt installs; a disabled radio would
# leave the AP down with no error anywhere visible.
if [ "$(uci -q get "wireless.$RADIO.disabled")" = "1" ]; then
echo "Note: enabling $RADIO (was disabled)."
uci -q delete "wireless.$RADIO.disabled"
fi
CHANNEL="$(uci -q get "wireless.$RADIO.channel")"
BAND="$(uci -q get "wireless.$RADIO.band")"
# ---------------------------------------------------------------------------
# Network: IPv4 from the fixed convention 10.21.<IDX>.1/24 — deterministic,
# so every router serving the same radio index lands on the same subnet and
# a roaming client's lease stays valid. IPv6 is a static ULA /64 so odhcpd
# has a prefix to announce; that space is per-router and disposable — a
# roaming phone SLAACs a fresh address on each router, and the FIPS overlay
# identity (not the IP) is the mobility anchor. The ULA is derived from the
# router's global ULA prefix; a re-run keeps the address already configured.
# ---------------------------------------------------------------------------
AP_ADDR="$(uci -q get "network.$SECTION.ip6addr")"
case "$AP_ADDR" in
fd*) ;; # keep the existing address on re-run
*)
ULA_BASE=""
ULA_PREFIX="$(uci -q get network.globals.ula_prefix)"
case "$ULA_PREFIX" in
fd*::/48) ULA_BASE="${ULA_PREFIX%::/48}" ;;
esac
if [ -z "$ULA_BASE" ]; then
HEX="$(head -c 5 /dev/urandom | hexdump -e '5/1 "%02x"')"
ULA_BASE="fd$(printf '%s' "$HEX" | cut -c1-2):$(printf '%s' "$HEX" | cut -c3-6):$(printf '%s' "$HEX" | cut -c7-10)"
echo "Note: no usable ULA prefix in network.globals — generated $ULA_BASE::/48 for this AP."
fi
# 64000 = 0xfa00 — high subnet IDs keep clear of br-lan's low
# ip6assign allocations from the same ULA prefix.
AP_ADDR="$ULA_BASE:$(printf '%04x' $((64000 + IDX)))::1/64"
;;
esac
AP_ADDR4="10.21.$IDX.1"
uci -q delete "network.$SECTION"
uci set "network.$SECTION=interface"
uci set "network.$SECTION.proto=static"
uci set "network.$SECTION.ipaddr=$AP_ADDR4"
uci set "network.$SECTION.netmask=255.255.255.0"
uci set "network.$SECTION.ip6addr=$AP_ADDR"
# ---------------------------------------------------------------------------
# DHCP/RA: dnsmasq DHCPv4 leases out of 10.21.<IDX>.0/24, plus router
# advertisements for the ULA (stateless SLAAC, no DHCPv6). ra_default '2'
# announces a default router even without an upstream default route:
# Android's provisioning wants address + route + DNS, and its validation
# probe then fails by design (no internet), so the phone keeps cellular as
# the default route. 'dhcpv4 server' is read by BOTH dnsmasq (the default
# DHCPv4 server) and odhcpd (serves v4 only when odhcpd.maindhcp is set),
# so either arrangement hands out leases.
# ---------------------------------------------------------------------------
uci -q delete "dhcp.$SECTION"
uci set "dhcp.$SECTION=dhcp"
uci set "dhcp.$SECTION.interface=$SECTION"
uci set "dhcp.$SECTION.ra=server"
uci set "dhcp.$SECTION.ra_default=2"
uci set "dhcp.$SECTION.dhcpv6=disabled"
uci set "dhcp.$SECTION.dhcpv4=server"
uci set "dhcp.$SECTION.start=10"
uci set "dhcp.$SECTION.limit=200"
# Authoritative is the OpenWrt default, but roaming correctness depends on
# it (a foreign router must ACK a lease it never issued), so pin it.
[ -n "$(uci -q get dhcp.@dnsmasq[0])" ] && uci set dhcp.@dnsmasq[0].authoritative=1
# ---------------------------------------------------------------------------
# Firewall: one shared 'fips_ap' zone for all instances. Everything is
# rejected except what a FIPS client needs — DHCPv4 (addressing), ICMPv6
# (SLAAC itself), mDNS (discovery), and the FIPS UDP/TCP transports (the
# handshake surface).
# The raw-Ethernet transport (EtherType 0x2121) is not IP and never
# traverses the firewall. No forwardings exist, so there is no path to
# br-lan or the WAN.
# ---------------------------------------------------------------------------
if [ "$(uci -q get firewall.fips_ap)" != "zone" ]; then
uci set firewall.fips_ap=zone
fi
uci set firewall.fips_ap.name=fips_ap
uci set firewall.fips_ap.input=REJECT
uci set firewall.fips_ap.output=ACCEPT
uci set firewall.fips_ap.forward=REJECT
uci -q del_list "firewall.fips_ap.network=$SECTION"
uci add_list "firewall.fips_ap.network=$SECTION"
# ap_rule <section-suffix> <name> <proto> [dest_port]
ap_rule() {
rule="firewall.fips_ap_$1"
uci -q delete "$rule"
uci set "$rule=rule"
uci set "$rule.name=$2"
uci set "$rule.src=fips_ap"
uci set "$rule.proto=$3"
uci set "$rule.target=ACCEPT"
[ -z "${4:-}" ] || uci set "$rule.dest_port=$4"
}
ap_rule icmpv6 "FIPS-AP-ICMPv6" icmp
uci set firewall.fips_ap_icmpv6.family=ipv6
ap_rule dhcpv4 "FIPS-AP-DHCPv4" udp 67
uci set firewall.fips_ap_dhcpv4.family=ipv4
ap_rule mdns "FIPS-AP-mDNS" udp 5353
ap_rule fips_udp "FIPS-AP-FIPS-UDP" udp 2121
ap_rule fips_tcp "FIPS-AP-FIPS-TCP" tcp 8443
uci commit wireless
uci commit network
uci commit dhcp
uci commit firewall
wifi reload
/etc/init.d/dnsmasq reload
/etc/init.d/odhcpd reload
/etc/init.d/firewall reload
# Enable the matching ap<N> transport in the shipped fips.yaml (it ships
# commented out). Tailor the restart hint to what we could do.
ap_config_enable "$IDX"
case $? in
0) TRANSPORT_NOTE="The ap$IDX transport in $CONFIG that binds '$AP_IFNAME' is
now uncommented and enabled." ;;
1) TRANSPORT_NOTE="No $CONFIG found — add a transports.ethernet entry binding
interface '$AP_IFNAME' by hand." ;;
*) TRANSPORT_NOTE="No 'ap$IDX' entry in $CONFIG — add a transports.ethernet
entry binding interface '$AP_IFNAME' by hand (copy the ap0 block)." ;;
esac
# Phones discover the daemon via mDNS, not raw-Ethernet beacons — make sure
# the daemon-wide mDNS rendezvous is on.
if lan_rendezvous_enable; then
MDNS_NOTE="node.rendezvous.lan (mDNS) is enabled — phone FIPS apps
discover this router via DNS-SD."
else
MDNS_NOTE="Could not enable mDNS in $CONFIG — set
'node.rendezvous.lan.enabled: true' by hand; phone FIPS apps rely
on it to discover this router."
fi
cat <<EOF
Created open access SSID '$SSID' as $AP_IFNAME on $RADIO \
(band ${BAND:-?}, channel ${CHANNEL:-auto}).
DHCPv4 on $AP_ADDR4/24 and RA IPv6 on $AP_ADDR — no internet,
isolated from br-lan and the WAN.
ALL FIPS routers must broadcast this SSID with the same security type
(open) — phones then save it once and roam between routers as one
network. The 10.21.$IDX.0/24 subnet is the same on every router on
purpose: leases survive roaming. Unlike the mesh backhaul, channels
are free per router. On a dual-band router, run fips-ap-setup for the
other radio too so clients can pick either band.
On first connect a phone warns that the network has no internet —
choose "stay connected" and "don't ask again". That choice is stored
per SSID, so it covers every FIPS router.
Next steps:
1. $TRANSPORT_NOTE
2. $MDNS_NOTE
Restart the daemon AFTER the interface is up — a transport whose
interface is missing at startup is skipped, not retried:
/etc/init.d/fips restart
3. Associate a phone or laptop running FIPS and verify:
iw dev $AP_IFNAME station dump
and the FIPS link on top of it:
fipsctl show peers
Run 'fips-ap-setup remove' to undo all instances, or
'fips-ap-setup remove $RADIO' for just this one.
EOF
packaging/openwrt-ipk/files/usr/bin/fips-mesh-setup
Executable
+261
View File
@@ -0,0 +1,261 @@
#!/bin/sh
# fips-mesh-setup — configure open 802.11s mesh interfaces for FIPS backhaul.
#
# Usage:
# fips-mesh-setup <radio> [mesh-id] e.g. fips-mesh-setup radio1
# fips-mesh-setup remove [radio] no radio: remove all instances
#
# Creates a mesh-point interface on the given radio and leaves everything
# above L2 to FIPS. Run once per radio: dual-band routers can mesh on both
# bands at once (2.4 GHz reaches further, 5 GHz carries more). Note this is
# failover, not multipath — FIPS keeps one active link per peer; the other
# band stands by and reconnects the peer if the active link dies.
#
# - encryption 'none' — the mesh is OPEN on purpose. FIPS's Noise
# handshake authenticates and encrypts every peer link, so SAE would
# only duplicate that (and on ath10k it forces the slower raw Tx/Rx
# firmware mode). A stranger can form an 802.11s peering AND a FIPS
# peer link — the Noise handshake authenticates each link (no
# impersonation of another identity, no MITM), it does not gate who
# may peer. Admission is open up to the daemon's max-peers cap.
# - mesh_fwding '0' — disables 802.11s HWMP forwarding so each mesh
# link is a plain L2 neighbor link. FIPS is the routing layer; two
# routing layers would fight.
#
# Interfaces are named per radio index (radio0 -> fips-mesh0, radio1 ->
# fips-mesh1) and are intentionally NOT bridged into br-lan: the FIPS
# Ethernet transport binds each directly and runs discovery beacons over it.
#
# The shipped /etc/fips/fips.yaml carries 'mesh0' and 'mesh1' entries under
# 'transports.ethernet' bound to these names, but commented out — a stock
# install that never creates fips-mesh* then logs no bind warning. This
# helper uncomments the matching entry when it creates an interface and
# re-comments it on remove, so the daemon binds the transport without a
# manual config edit. After an interface is up, restart fips.
# See docs/how-to/set-up-80211s-mesh-backhaul.md for the full guide.
DEFAULT_MESH_ID="fips-mesh"
CONFIG="/etc/fips/fips.yaml"
# Replace $CONFIG with the rewritten $CONFIG.tmp. Force mode 0600 first: the
# package installs fips.yaml 0600 (it may hold an inline 'nsec' private key),
# and a fresh tmp file would otherwise land world-readable after the move.
mesh_config_write() {
chmod 600 "$CONFIG.tmp" && mv "$CONFIG.tmp" "$CONFIG"
}
# Uncomment the 'mesh<idx>' transports.ethernet block in $CONFIG (created by
# 'fips-mesh-setup'). Reversible with mesh_config_disable. Returns:
# 0 enabled (or already active) 1 no config file 2 no such block
mesh_config_enable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ mesh$idx:" "$CONFIG" && return 0
grep -q "^ # mesh$idx:" "$CONFIG" || return 2
awk -v idx="$idx" '
$0 ~ ("^ # mesh" idx ":[ \t]*$") { blk = 1; sub(/^ # /, " "); print; next }
blk && /^ # / { sub(/^ # /, " "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && mesh_config_write
}
# Re-comment the 'mesh<idx>' block so the daemon stops binding it (and stops
# warning about the now-missing interface). Inverse of mesh_config_enable.
mesh_config_disable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ mesh$idx:" "$CONFIG" || return 0
awk -v idx="$idx" '
$0 ~ ("^ mesh" idx ":[ \t]*$") { blk = 1; sub(/^ /, " # "); print; next }
blk && /^ / { sub(/^ /, " # "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && mesh_config_write
}
usage() {
echo "Usage: fips-mesh-setup <radio> [mesh-id]" >&2
echo " fips-mesh-setup remove [radio]" >&2
echo "Radios on this device:" >&2
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\([^.]*\)=wifi-device$/ \1/p" >&2
exit 1
}
# List the UCI section names of fips-managed mesh wifi-ifaces.
mesh_sections() {
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\(fips_mesh[^.=]*\)=wifi-iface$/\1/p"
}
# ---------------------------------------------------------------------------
# remove [radio] — delete the wireless and network sections created below
# ---------------------------------------------------------------------------
if [ "$1" = "remove" ]; then
if [ -n "$2" ]; then
SECTIONS="fips_mesh_$(printf '%s' "$2" | tr -c 'a-zA-Z0-9_' '_')"
else
SECTIONS="$(mesh_sections)"
fi
[ -n "$SECTIONS" ] || {
echo "No fips mesh instances configured."
exit 0
}
for section in $SECTIONS; do
ifname="$(uci -q get "wireless.$section.ifname")"
uci -q delete "wireless.$section"
uci -q delete "network.$section"
# Re-comment the matching mesh<N> transport in fips.yaml so the
# daemon stops warning about the interface we just removed.
idx="$(printf '%s' "$ifname" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$idx" ] && mesh_config_disable "$idx"
echo "Removed ${ifname:-$section}."
done
uci commit wireless
uci commit network
# 'wifi reload' re-applies the whole wireless config, so it briefly drops
# every client AP on all radios (a few seconds) — expected on remove.
wifi reload
echo "Restart fips: /etc/init.d/fips restart"
exit 0
fi
RADIO="$1"
MESH_ID="${2:-$DEFAULT_MESH_ID}"
[ -n "$RADIO" ] || usage
if [ "$(uci -q get "wireless.$RADIO")" != "wifi-device" ]; then
echo "Error: '$RADIO' is not a wifi-device in /etc/config/wireless." >&2
usage
fi
# One instance per radio: section fips_mesh_<radio>, netdev fips-mesh<N>
# where N is the radio's trailing index (radio0 -> fips-mesh0). For radios
# named without a trailing number, fall back to the first free index.
SECTION="fips_mesh_$(printf '%s' "$RADIO" | tr -c 'a-zA-Z0-9_' '_')"
IDX="$(printf '%s' "$RADIO" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$IDX" ] || IDX="$(printf '%s' "$RADIO" | sed -n 's/^\([0-9]\{1,\}\)$/\1/p')"
if [ -z "$IDX" ]; then
IDX=0
while uci show wireless 2>/dev/null | grep -q "\.ifname='fips-mesh$IDX'"; do
IDX=$((IDX + 1))
done
fi
MESH_IFNAME="fips-mesh$IDX"
# Refuse a name collision from another radio's instance (e.g. two radios
# whose names end in the same digit) rather than silently hijacking it.
OWNER="$(uci show wireless 2>/dev/null \
| sed -n "s/^wireless\.\(fips_mesh[^.=]*\)\.ifname='$MESH_IFNAME'$/\1/p")"
if [ -n "$OWNER" ] && [ "$OWNER" != "$SECTION" ]; then
echo "Error: $MESH_IFNAME is already used by section '$OWNER'." >&2
echo "Remove it first: fips-mesh-setup remove" >&2
exit 1
fi
# ---------------------------------------------------------------------------
# Driver capability check (advisory — config below is harmless either way)
# ---------------------------------------------------------------------------
if command -v iw >/dev/null 2>&1; then
if ! iw list 2>/dev/null | grep -q "\* mesh point"; then
echo "Warning: no radio on this device advertises 'mesh point' support" >&2
echo "(iw list | grep 'mesh point'). The interface may fail to come up." >&2
fi
fi
# ---------------------------------------------------------------------------
# Wireless: open 802.11s mesh point, HWMP forwarding off
# ---------------------------------------------------------------------------
uci -q delete "wireless.$SECTION"
uci set "wireless.$SECTION=wifi-iface"
uci set "wireless.$SECTION.device=$RADIO"
uci set "wireless.$SECTION.mode=mesh"
uci set "wireless.$SECTION.mesh_id=$MESH_ID"
uci set "wireless.$SECTION.encryption=none"
uci set "wireless.$SECTION.mesh_fwding=0"
uci set "wireless.$SECTION.ifname=$MESH_IFNAME"
uci set "wireless.$SECTION.network=$SECTION"
# Radios ship disabled on fresh OpenWrt installs; a disabled radio would
# leave the mesh interface down with no error anywhere visible.
if [ "$(uci -q get "wireless.$RADIO.disabled")" = "1" ]; then
echo "Note: enabling $RADIO (was disabled)."
uci -q delete "wireless.$RADIO.disabled"
fi
# The mesh inherits the radio's channel, and mesh points only peer on the
# same channel. 'auto' lets each router pick its own — the classic silent
# non-peering cause — so surface the setting loudly.
CHANNEL="$(uci -q get "wireless.$RADIO.channel")"
BAND="$(uci -q get "wireless.$RADIO.band")"
if [ -z "$CHANNEL" ] || [ "$CHANNEL" = "auto" ]; then
echo "Warning: $RADIO channel is '${CHANNEL:-unset}' — each router may" >&2
echo "auto-select a different channel and mesh points only peer on the" >&2
echo "same one. Pin the same channel on every backhaul router, e.g.:" >&2
echo " uci set wireless.$RADIO.channel='36' && uci commit wireless && wifi reload" >&2
fi
# A client (sta) interface on the same radio follows its upstream AP's
# channel and drags every other interface with it — a mesh pinned to a
# different channel silently never joins, and does not recover when the
# STA disconnects.
for s in $(uci show wireless 2>/dev/null | sed -n "s/^wireless\.\([^.]*\)\.mode='sta'$/\1/p"); do
if [ "$(uci -q get "wireless.$s.device")" = "$RADIO" ]; then
echo "Warning: $RADIO also carries client interface '$s' (mode 'sta')." >&2
echo "The whole radio follows that STA's upstream channel — a mesh" >&2
echo "pinned to a different channel stays down silently. Align the" >&2
echo "mesh channel with the upstream AP, or put the mesh on a radio" >&2
echo "without a STA (a roaming uplink is incompatible with a" >&2
echo "fixed-channel mesh on the same radio)." >&2
fi
done
# ---------------------------------------------------------------------------
# Network: unmanaged interface so netifd brings the netdev up. No IP config —
# the FIPS Ethernet transport speaks raw frames on it.
# ---------------------------------------------------------------------------
uci -q delete "network.$SECTION"
uci set "network.$SECTION=interface"
uci set "network.$SECTION.proto=none"
uci commit wireless
uci commit network
# 'wifi reload' re-applies the whole wireless config, so it briefly drops
# every client AP on all radios (a few seconds) — expected when adding a mesh.
wifi reload
# Enable the matching mesh<N> transport in the shipped fips.yaml (it ships
# commented out). Tailor the restart hint to what we could do.
mesh_config_enable "$IDX"
case $? in
0) TRANSPORT_NOTE="The mesh$IDX transport in $CONFIG that binds '$MESH_IFNAME' is
now uncommented and enabled." ;;
1) TRANSPORT_NOTE="No $CONFIG found — add a transports.ethernet entry binding
interface '$MESH_IFNAME' by hand." ;;
*) TRANSPORT_NOTE="No 'mesh$IDX' entry in $CONFIG — add a transports.ethernet
entry binding interface '$MESH_IFNAME' by hand (copy the mesh0 block)." ;;
esac
cat <<EOF
Created open 802.11s mesh '$MESH_ID' as $MESH_IFNAME on $RADIO \
(band ${BAND:-?}, channel ${CHANNEL:-auto}).
ALL routers in this backhaul must share this mesh ID AND channel
(per band). On a dual-band router, run fips-mesh-setup for the other
radio too — second band is a standby path (failover, not multipath).
Next steps:
1. $TRANSPORT_NOTE
Restart the daemon AFTER the interface is up — a transport whose
interface is missing at startup is skipped, not retried:
/etc/init.d/fips restart
2. Verify L2 peering with a second FIPS router in range:
iw dev $MESH_IFNAME station dump
and the FIPS link on top of it:
fipsctl show peers
Run 'fips-mesh-setup remove' to undo all instances, or
'fips-mesh-setup remove $RADIO' for just this one.
EOF
packaging/systemd/README.install.md
+1 -1
View File
@@ -68,7 +68,7 @@ and set the interface name:
transports:
ethernet:
interface: "eth0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
packaging/systemd/fips.service
+5
View File
@@ -14,6 +14,11 @@ RestartSec=5
RuntimeDirectory=fips
RuntimeDirectoryMode=0750
# Log directory (/var/log/fips/), where the built-in tick-body profiler writes
# its capture files. Declared so systemd creates it on start and removes it on
# purge; the daemon runs as root and already has access without it.
LogsDirectory=fips
# Security hardening (daemon runs as root for TUN and raw sockets)
ProtectHome=yes
PrivateTmp=yes
src/bin/fips.rs
+14 -17
View File
@@ -8,7 +8,7 @@ use fips::config::{IdentitySource, resolve_identity};
use fips::version;
use fips::{Config, Node};
use std::path::PathBuf;
use tracing::{debug, error, info, warn};
use tracing::{debug, error, info};
use tracing_subscriber::{EnvFilter, fmt};
/// FIPS mesh network daemon
@@ -157,26 +157,23 @@ async fn run_daemon(
info!("FIPS running");
// Run the RX event loop until shutdown signal.
// stop() drops the packet channel, causing run_rx_loop to exit.
tokio::select! {
result = node.run_rx_loop() => {
match result {
Ok(()) => info!("RX loop exited"),
Err(e) => error!("RX loop error: {}", e),
}
}
_ = shutdown_signal => {
info!("Shutdown signal received");
}
// Serve until the shutdown signal, then drain in place before returning.
// The rx loop observes the signal directly, so its channels are never
// destructively cancelled — they live in the loop's locals across serve and
// drain, and are dropped only on clean exit (after which teardown does not
// need them). On the signal the loop broadcasts a shutdown Disconnect and
// waits (bounded by node.drain_timeout_secs) for peers to clear.
match node.run_rx_loop_with_shutdown(shutdown_signal).await {
Ok(()) => info!("RX loop exited"),
Err(e) => error!("RX loop error: {}", e),
}
info!("FIPS shutting down");
// Stop the node (shuts down transports, TUN, I/O threads)
if let Err(e) = node.stop().await {
warn!("Error during shutdown: {}", e);
}
// Close the drain window (if the loop drained) and tear down. A drained
// loop tears down without re-broadcasting; a loop that exited some other
// way falls back to the immediate stop().
node.finish_shutdown().await;
info!("FIPS shutdown complete");
}
src/bin/fipsctl.rs
+45
View File
@@ -76,6 +76,37 @@ enum Commands {
#[command(subcommand)]
what: StatsCommands,
},
/// Control the built-in profiler (requires a `--features profiling` build)
#[cfg(feature = "profiling")]
Profile {
#[command(subcommand)]
what: ProfileCommands,
},
}
#[cfg(feature = "profiling")]
#[derive(Subcommand, Debug)]
enum ProfileCommands {
/// Profile the rx-loop tick body
Tick {
#[command(subcommand)]
action: ProfileTickAction,
},
}
#[cfg(feature = "profiling")]
#[derive(Subcommand, Debug)]
enum ProfileTickAction {
/// Start a capture
On {
/// Directory for the capture file (default /var/log/fips)
#[arg(long)]
dir: Option<PathBuf>,
},
/// Stop the running capture
Off,
/// Report capture state
Status,
}
#[derive(Subcommand, Debug)]
@@ -471,6 +502,20 @@ fn main() {
build_command("show_stats_history", params)
}
},
#[cfg(feature = "profiling")]
Commands::Profile { what } => match what {
ProfileCommands::Tick { action } => match action {
ProfileTickAction::On { dir } => match dir {
Some(dir) => build_command(
"profile_tick_on",
serde_json::json!({"dir": dir.display().to_string()}),
),
None => build_query("profile_tick_on"),
},
ProfileTickAction::Off => build_query("profile_tick_off"),
ProfileTickAction::Status => build_query("profile_tick_status"),
},
},
Commands::Keygen { .. } => unreachable!(),
};
src/bin/fipstop/ui/bloom.rs
+20 -1
View File
@@ -121,7 +121,6 @@ fn draw_stats(frame: &mut Frame, data: &serde_json::Value, scroll: u16, focused:
&helpers::nested_u64(data, "stats", "decode_error"),
),
helpers::kv_line("Invalid", &helpers::nested_u64(data, "stats", "invalid")),
helpers::kv_line("Non-V1", &helpers::nested_u64(data, "stats", "non_v1")),
helpers::kv_line(
"Unknown Peer",
&helpers::nested_u64(data, "stats", "unknown_peer"),
@@ -130,6 +129,26 @@ fn draw_stats(frame: &mut Frame, data: &serde_json::Value, scroll: u16, focused:
Line::from(""),
helpers::section_header("Outbound"),
helpers::kv_line("Sent", &helpers::nested_u64(data, "stats", "sent")),
helpers::kv_line(
"Full Sends",
&helpers::nested_u64(data, "stats", "full_sends"),
),
helpers::kv_line(
"Deltas Sent",
&helpers::nested_u64(data, "stats", "deltas_sent"),
),
helpers::kv_line(
"NACKs Sent",
&helpers::nested_u64(data, "stats", "nacks_sent"),
),
helpers::kv_line(
"NACKs Received",
&helpers::nested_u64(data, "stats", "nacks_received"),
),
helpers::kv_line(
"Size Changes",
&helpers::nested_u64(data, "stats", "size_changes"),
),
helpers::kv_line(
"Debounce Suppressed",
&helpers::nested_u64(data, "stats", "debounce_suppressed"),
src/bin/fipstop/ui/routing.rs
+22 -22
View File
@@ -134,8 +134,8 @@ fn draw_routing_stats(
let cols =
Layout::horizontal([Constraint::Percentage(50), Constraint::Percentage(50)]).split(inner);
// Shorthand for a nested counter value (e.g. discovery.req_received).
let disc = |key: &str| helpers::nested_u64(data, "discovery", key);
// Shorthand for a nested counter value (e.g. lookup.req_received).
let lookup = |key: &str| helpers::nested_u64(data, "lookup", key);
let err = |key: &str| helpers::nested_u64(data, "error_signals", key);
let cong = |key: &str| helpers::nested_u64(data, "congestion", key);
@@ -174,32 +174,32 @@ fn draw_routing_stats(
));
left.push(Line::from(""));
left.extend(section(
"Discovery Requests",
"Lookup Requests",
&[
("Received", disc("req_received")),
("Forwarded", disc("req_forwarded")),
("Initiated", disc("req_initiated")),
("Deduplicated", disc("req_deduplicated")),
("Target Is Us", disc("req_target_is_us")),
("Duplicate", disc("req_duplicate")),
("Bloom Miss", disc("req_bloom_miss")),
("Backoff Suppressed", disc("req_backoff_suppressed")),
("Fwd Rate Limited", disc("req_forward_rate_limited")),
("TTL Exhausted", disc("req_ttl_exhausted")),
("Decode Error", disc("req_decode_error")),
("Received", lookup("req_received")),
("Forwarded", lookup("req_forwarded")),
("Initiated", lookup("req_initiated")),
("Deduplicated", lookup("req_deduplicated")),
("Target Is Us", lookup("req_target_is_us")),
("Duplicate", lookup("req_duplicate")),
("Bloom Miss", lookup("req_bloom_miss")),
("Backoff Suppressed", lookup("req_backoff_suppressed")),
("Fwd Rate Limited", lookup("req_forward_rate_limited")),
("TTL Exhausted", lookup("req_ttl_exhausted")),
("Decode Error", lookup("req_decode_error")),
],
));
left.push(Line::from(""));
left.extend(section(
"Discovery Responses",
"Lookup Responses",
&[
("Received", disc("resp_received")),
("Accepted", disc("resp_accepted")),
("Forwarded", disc("resp_forwarded")),
("Timed Out", disc("resp_timed_out")),
("Identity Miss", disc("resp_identity_miss")),
("Proof Failed", disc("resp_proof_failed")),
("Decode Error", disc("resp_decode_error")),
("Received", lookup("resp_received")),
("Accepted", lookup("resp_accepted")),
("Forwarded", lookup("resp_forwarded")),
("Timed Out", lookup("resp_timed_out")),
("Identity Miss", lookup("resp_identity_miss")),
("Proof Failed", lookup("resp_proof_failed")),
("Decode Error", lookup("resp_decode_error")),
],
));
src/bin/fipstop/ui/snapshots.rs
+9
View File
@@ -1189,6 +1189,15 @@ fn mmp_focused_pane_indicator() {
});
// The focused Session MMP title is cyan; the unfocused Link MMP title is not.
assert_eq!(testkit::fg_at(&buf, "Session MMP"), Some(Color::Cyan));
// Presence before colour. `fg_at` is `find(..)?` mapped to the cell's fg, so
// it returns None for a title that was never drawn, and None != Some(Cyan) --
// meaning the assertion below passed when the Link MMP pane was missing
// entirely. Asserting it is on screen first is what makes the next line read
// "not highlighted" rather than "not there".
assert!(
testkit::find(&buf, "Link MMP").is_some(),
"the unfocused Link MMP title should still be rendered"
);
assert_ne!(testkit::fg_at(&buf, "Link MMP"), Some(Color::Cyan));
}
src/bloom/mod.rs
-60
View File
@@ -1,60 +0,0 @@
//! Bloom Filter Implementation
//!
//! 1KB Bloom filters for reachability in FIPS routing. Each node
//! maintains filters that summarize which destinations are reachable
//! through each peer, enabling efficient routing decisions without
//! global network knowledge.
//!
//! ## v1 Parameters
//!
//! - Size: 1 KB (8,192 bits) - sized for actual ~400-800 entry occupancy
//! - Hash functions: k=5 - optimal at ~1,200 entries, good for 800-1,600
//! - Bandwidth: 1 KB/announce (75% reduction from original 4KB design)
//!
//! These parameters are right-sized for typical network occupancy of
//! ~250-800 entries per node.
mod filter;
mod state;
use thiserror::Error;
pub use filter::BloomFilter;
pub use state::BloomState;
/// Default filter size in bits (1KB = 8,192 bits).
///
/// Sized for ~800-1,600 entries. FPR ~0.05% at 400 entries, ~0.9% at 800.
/// This is v1 protocol default (size_class=1).
pub const DEFAULT_FILTER_SIZE_BITS: usize = 8192;
/// Default filter size in bytes (1KB).
pub const DEFAULT_FILTER_SIZE_BYTES: usize = DEFAULT_FILTER_SIZE_BITS / 8;
/// Default number of hash functions.
///
/// k=5 is optimal at ~1,200 entries and a good compromise for 800-1,600.
/// At 400 entries: FPR ~0.05%. At 800 entries: FPR ~0.9%.
pub const DEFAULT_HASH_COUNT: u8 = 5;
/// Size class for v1 protocol (1 KB filters).
pub const V1_SIZE_CLASS: u8 = 1;
/// Filter sizes by size_class: bytes = 512 << size_class
pub const SIZE_CLASS_BYTES: [usize; 4] = [512, 1024, 2048, 4096];
/// Errors related to Bloom filter operations.
#[derive(Debug, Error)]
pub enum BloomError {
#[error("invalid filter size: expected {expected} bits, got {got}")]
InvalidSize { expected: usize, got: usize },
#[error("filter size must be a multiple of 8, got {0}")]
SizeNotByteAligned(usize),
#[error("hash count must be positive")]
ZeroHashCount,
}
#[cfg(test)]
mod tests;
src/cache/coord_cache.rs
Vendored
+1 -1
View File
@@ -9,7 +9,7 @@ use std::collections::HashMap;
use super::CacheStats;
use super::entry::CacheEntry;
use crate::NodeAddr;
use crate::tree::TreeCoordinate;
use crate::proto::stp::TreeCoordinate;
/// Default maximum entries in coordinate cache.
pub const DEFAULT_COORD_CACHE_SIZE: usize = 50_000;
src/cache/entry.rs
Vendored
+1 -1
View File
@@ -1,6 +1,6 @@
//! Cache entry with TTL and LRU tracking.
use crate::tree::TreeCoordinate;
use crate::proto::stp::TreeCoordinate;
/// A cached coordinate entry.
#[derive(Clone, Debug)]
src/config/mod.rs
+334 -31
View File
@@ -24,6 +24,7 @@ mod node;
mod peer;
mod transport;
use crate::node::REKEY_JITTER_SECS;
use crate::upper::config::{DnsConfig, TunConfig};
use crate::{Identity, IdentityError};
use serde::{Deserialize, Serialize};
@@ -33,9 +34,9 @@ use thiserror::Error;
#[cfg(target_os = "linux")]
pub use gateway::{ConntrackConfig, GatewayConfig, GatewayDnsConfig, PortForward, Proto};
pub use node::{
BloomConfig, BuffersConfig, CacheConfig, ControlConfig, DiscoveryConfig, LimitsConfig,
NodeConfig, NostrDiscoveryConfig, NostrDiscoveryPolicy, RateLimitConfig, RekeyConfig,
RetryConfig, SessionConfig, SessionMmpConfig, TreeConfig,
BloomConfig, BuffersConfig, CacheConfig, ControlConfig, LimitsConfig, LookupConfig, MmpConfig,
NodeConfig, NostrRendezvousConfig, NostrRendezvousPolicy, RateLimitConfig, RekeyConfig,
RendezvousConfig, RetryConfig, SessionConfig, SessionMmpConfig, TreeConfig,
};
pub use peer::{ConnectPolicy, PeerAddress, PeerConfig};
pub use transport::{
@@ -489,10 +490,59 @@ impl Config {
source: e,
})?;
serde_yaml::from_str(&contents).map_err(|e| ConfigError::ParseYaml {
path: path.to_path_buf(),
source: e,
})
let mut config: Config =
serde_yaml::from_str(&contents).map_err(|e| ConfigError::ParseYaml {
path: path.to_path_buf(),
source: e,
})?;
config.normalize_deprecated_keys();
Ok(config)
}
/// COMPAT (drop at the v2 cutover): fold a deprecated `node.discovery:`
/// block into the `node.lookup.*` (mesh-lookup scalars) and
/// `node.rendezvous.*` (nostr/LAN peer rendezvous) tables that replaced it.
///
/// Runs at every deserialize boundary (see `load_file`). A present legacy
/// field fills the corresponding new-table field, so a config that predates
/// the split keeps behaving identically. When a legacy block is seen, a
/// one-time deprecation warning names the old→new key moves. Exposed to the
/// crate so config tests that deserialize directly can invoke it.
pub(crate) fn normalize_deprecated_keys(&mut self) {
let Some(compat) = self.node.discovery.take() else {
return;
};
tracing::warn!(
target: "fips::config",
"`node.discovery.*` is deprecated and will be removed: mesh-lookup \
scalars moved to `node.lookup.*`, and peer-rendezvous keys moved to \
`node.rendezvous.nostr.*` / `node.rendezvous.lan.*`. Please migrate; \
a legacy `node.discovery` block still applies for now."
);
if let Some(v) = compat.ttl {
self.node.lookup.ttl = v;
}
if let Some(v) = compat.attempt_timeouts_secs {
self.node.lookup.attempt_timeouts_secs = v;
}
if let Some(v) = compat.recent_expiry_secs {
self.node.lookup.recent_expiry_secs = v;
}
if let Some(v) = compat.backoff_base_secs {
self.node.lookup.backoff_base_secs = v;
}
if let Some(v) = compat.backoff_max_secs {
self.node.lookup.backoff_max_secs = v;
}
if let Some(v) = compat.forward_min_interval_secs {
self.node.lookup.forward_min_interval_secs = v;
}
if let Some(v) = compat.nostr {
self.node.rendezvous.nostr = v;
}
if let Some(v) = compat.lan {
self.node.rendezvous.lan = v;
}
}
/// Get the standard search paths in priority order (lowest to highest).
@@ -588,6 +638,21 @@ impl Config {
self.node.leaf_only
}
/// Derive the node profile from config.
///
/// leaf_only → Leaf (implies non-routing),
/// disable_routing → NonRouting,
/// otherwise → Full.
pub fn node_profile(&self) -> crate::proto::fmp::NodeProfile {
if self.node.leaf_only {
crate::proto::fmp::NodeProfile::Leaf
} else if self.node.disable_routing {
crate::proto::fmp::NodeProfile::NonRouting
} else {
crate::proto::fmp::NodeProfile::Full
}
}
/// Get the configured peers.
pub fn peers(&self) -> &[PeerConfig] {
&self.peers
@@ -600,7 +665,7 @@ impl Config {
/// Validate cross-field configuration invariants.
pub fn validate(&self) -> Result<(), ConfigError> {
let nostr = &self.node.discovery.nostr;
let nostr = &self.node.rendezvous.nostr;
let any_transport_advertises_on_nostr = self
.transports
@@ -620,13 +685,13 @@ impl Config {
if any_transport_advertises_on_nostr && !nostr.enabled {
return Err(ConfigError::Validation(
"at least one transport has `advertise_on_nostr = true`, but `node.discovery.nostr.enabled` is false".to_string(),
"at least one transport has `advertise_on_nostr = true`, but `node.rendezvous.nostr.enabled` is false".to_string(),
));
}
if self.peers.iter().any(|peer| peer.via_nostr) && !nostr.enabled {
return Err(ConfigError::Validation(
"at least one peer has `via_nostr = true`, but `node.discovery.nostr.enabled` is false".to_string(),
"at least one peer has `via_nostr = true`, but `node.rendezvous.nostr.enabled` is false".to_string(),
));
}
@@ -648,12 +713,12 @@ impl Config {
if nostr.enabled && has_nat_udp_advert {
if nostr.dm_relays.is_empty() {
return Err(ConfigError::Validation(
"NAT UDP advert publishing requires `node.discovery.nostr.dm_relays` to be non-empty".to_string(),
"NAT UDP advert publishing requires `node.rendezvous.nostr.dm_relays` to be non-empty".to_string(),
));
}
if nostr.stun_servers.is_empty() {
return Err(ConfigError::Validation(
"NAT UDP advert publishing requires `node.discovery.nostr.stun_servers` to be non-empty".to_string(),
"NAT UDP advert publishing requires `node.rendezvous.nostr.stun_servers` to be non-empty".to_string(),
));
}
}
@@ -686,6 +751,32 @@ impl Config {
}
}
// Reject rekey triggers that fire immediately and forever. Both
// arms are checked regardless of `node.rekey.enabled` so that
// turning rekey on later cannot surface a config error at a
// surprising moment. There is deliberately no upper bound:
// u64::MAX is the idiom for disabling one arm of the trigger.
let rekey = &self.node.rekey;
if rekey.after_messages == 0 {
return Err(ConfigError::Validation(
"`node.rekey.after_messages` must be at least 1; 0 fires the message-count trigger on every poll instead of disabling it. \
Use a very large value to effectively disable the message-count trigger."
.to_string(),
));
}
let jitter_secs = REKEY_JITTER_SECS.unsigned_abs();
if rekey.after_secs <= jitter_secs {
return Err(ConfigError::Validation(format!(
"`node.rekey.after_secs` is {}, but must be greater than the per-session rekey jitter of {jitter_secs}s; \
each session offsets the interval by a random value in [-{jitter_secs}, +{jitter_secs}] seconds, so a smaller interval saturates to zero \
and rekeys on sight for roughly half of sessions. \
Use a very large value to effectively disable the timer trigger.",
rekey.after_secs
)));
}
Ok(())
}
@@ -721,6 +812,84 @@ node:
assert!(config.has_identity());
}
/// The fips.yaml shipped in the OpenWrt package must keep parsing as the
/// config schema evolves. Both the 802.11s mesh backhaul entries
/// (docs/how-to/set-up-80211s-mesh-backhaul.md) and the open-access SSID
/// entries (docs/how-to/set-up-open-access-ssid.md) ship commented out —
/// one per radio, so dual-band routers can run either on both bands — so
/// a stock install that never creates fips-mesh*/fips-ap* logs no
/// per-boot bind warning; `fips-mesh-setup`/`fips-ap-setup` uncomment the
/// matching block when they create the interface. Verify both states
/// parse: as shipped (both inactive), and after the uncomment the helpers
/// perform.
#[test]
fn shipped_openwrt_config_parses() {
let yaml = include_str!("../../packaging/openwrt-ipk/files/etc/fips/fips.yaml");
// As shipped: parses, and the mesh/ap entries are commented out (a
// running daemon binds no fips-mesh*/fips-ap* transport, no warning).
let config: Config = serde_yaml::from_str(yaml).expect("shipped OpenWrt fips.yaml");
for name in ["mesh0", "mesh1", "ap0", "ap1"] {
assert!(
!config
.transports
.ethernet
.iter()
.any(|(n, _)| n == Some(name)),
"{name} must ship commented out, not active, in fips.yaml"
);
}
// What `fips-mesh-setup`/`fips-ap-setup` produce: uncomment each
// block, which must still parse into a transport bound to the right
// netdev.
let uncommented =
uncomment_transport_blocks(&uncomment_transport_blocks(yaml, "mesh"), "ap");
let config: Config = serde_yaml::from_str(&uncommented)
.expect("fips.yaml with mesh and ap transports uncommented");
for (name, interface) in [
("mesh0", "fips-mesh0"),
("mesh1", "fips-mesh1"),
("ap0", "fips-ap0"),
("ap1", "fips-ap1"),
] {
assert!(
config
.transports
.ethernet
.iter()
.any(|(n, eth)| n == Some(name) && eth.interface == interface),
"{name} entry missing after uncommenting shipped fips.yaml"
);
}
}
/// Mirror the setup helpers' block uncomment: strip the ` # ` prefix
/// from each `# <prefix><N>:` header and its ` # ` continuation
/// lines, leaving every other comment untouched.
fn uncomment_transport_blocks(yaml: &str, prefix: &str) -> String {
let header = format!(" # {prefix}");
let mut out = String::new();
let mut in_block = false;
for line in yaml.lines() {
let is_header = line
.strip_prefix(&header)
.and_then(|r| r.strip_suffix(':'))
.is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()));
if is_header {
in_block = true;
out.push_str(&line.replacen(" # ", " ", 1));
} else if in_block && line.starts_with(" # ") {
out.push_str(&line.replacen(" # ", " ", 1));
} else {
in_block = false;
out.push_str(line);
}
out.push('\n');
}
out
}
#[test]
fn test_parse_yaml_with_hex() {
let yaml = r#"
@@ -1261,7 +1430,9 @@ peers:
}
#[test]
fn test_parse_nostr_discovery_config() {
fn test_parse_legacy_discovery_nostr_config_compat() {
// COMPAT (drop at the v2 cutover): a deprecated `node.discovery.nostr`
// block must fold into `node.rendezvous.nostr` via normalize.
let yaml = r#"
node:
discovery:
@@ -1285,26 +1456,27 @@ peers:
- transport: udp
addr: "nat"
"#;
let config: Config = serde_yaml::from_str(yaml).unwrap();
assert!(config.node.discovery.nostr.enabled);
assert!(!config.node.discovery.nostr.advertise);
assert_eq!(config.node.discovery.nostr.app, "fips.nat.test.v1");
assert_eq!(config.node.discovery.nostr.signal_ttl_secs, 45);
let mut config: Config = serde_yaml::from_str(yaml).unwrap();
config.normalize_deprecated_keys();
assert!(config.node.rendezvous.nostr.enabled);
assert!(!config.node.rendezvous.nostr.advertise);
assert_eq!(config.node.rendezvous.nostr.app, "fips.nat.test.v1");
assert_eq!(config.node.rendezvous.nostr.signal_ttl_secs, 45);
assert_eq!(
config.node.discovery.nostr.policy,
NostrDiscoveryPolicy::ConfiguredOnly
config.node.rendezvous.nostr.policy,
NostrRendezvousPolicy::ConfiguredOnly
);
assert_eq!(config.node.discovery.nostr.open_discovery_max_pending, 12);
assert_eq!(config.node.rendezvous.nostr.open_discovery_max_pending, 12);
assert_eq!(
config.node.discovery.nostr.advert_relays,
config.node.rendezvous.nostr.advert_relays,
vec!["wss://relay-a.example".to_string()]
);
assert_eq!(
config.node.discovery.nostr.dm_relays,
config.node.rendezvous.nostr.dm_relays,
vec!["wss://relay-b.example".to_string()]
);
assert_eq!(
config.node.discovery.nostr.stun_servers,
config.node.rendezvous.nostr.stun_servers,
vec!["stun:stun.example.org:3478".to_string()]
);
assert_eq!(
@@ -1314,6 +1486,55 @@ peers:
assert!(config.peers[0].via_nostr);
}
#[test]
fn test_parse_lookup_and_rendezvous_new_keys() {
// The post-split keys parse directly, with no deprecated block and no
// normalize warning.
let yaml = r#"
node:
lookup:
ttl: 7
attempt_timeouts_secs: [3, 6]
forward_min_interval_secs: 9
rendezvous:
nostr:
enabled: true
app: "fips.new.keys.v1"
"#;
let mut config: Config = serde_yaml::from_str(yaml).unwrap();
config.normalize_deprecated_keys();
assert_eq!(config.node.lookup.ttl, 7);
assert_eq!(config.node.lookup.attempt_timeouts_secs, vec![3, 6]);
assert_eq!(config.node.lookup.forward_min_interval_secs, 9);
// Unset scalar keeps its default.
assert_eq!(config.node.lookup.recent_expiry_secs, 10);
assert!(config.node.rendezvous.nostr.enabled);
assert_eq!(config.node.rendezvous.nostr.app, "fips.new.keys.v1");
assert!(config.node.discovery.is_none());
}
#[test]
fn test_legacy_discovery_lookup_scalars_compat() {
// COMPAT (drop at the v2 cutover): legacy `node.discovery` mesh-lookup
// scalars must fold into `node.lookup`; unset keys keep their defaults.
let yaml = r#"
node:
discovery:
ttl: 5
backoff_base_secs: 4
backoff_max_secs: 30
"#;
let mut config: Config = serde_yaml::from_str(yaml).unwrap();
config.normalize_deprecated_keys();
assert_eq!(config.node.lookup.ttl, 5);
assert_eq!(config.node.lookup.backoff_base_secs, 4);
assert_eq!(config.node.lookup.backoff_max_secs, 30);
// Unset legacy scalar leaves the new-table default intact.
assert_eq!(config.node.lookup.attempt_timeouts_secs, vec![1, 2, 4, 8]);
// The compat block is consumed by normalize.
assert!(config.node.discovery.is_none());
}
#[test]
fn test_validate_transport_advert_requires_nostr_enabled() {
let mut config = Config::default();
@@ -1321,13 +1542,14 @@ peers:
advertise_on_nostr: Some(true),
..Default::default()
});
config.node.discovery.nostr.enabled = false;
config.node.rendezvous.nostr.enabled = false;
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("advertise_on_nostr"));
}
#[test]
#[allow(clippy::field_reassign_with_default)]
fn test_validate_peer_via_nostr_requires_nostr_enabled() {
let mut config = Config {
peers: vec![PeerConfig {
@@ -1337,13 +1559,14 @@ peers:
}],
..Default::default()
};
config.node.discovery.nostr.enabled = false;
config.node.rendezvous.nostr.enabled = false;
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("via_nostr"));
}
#[test]
#[allow(clippy::field_reassign_with_default)]
fn test_validate_peer_addresses_required_unless_via_nostr() {
// Empty addresses + via_nostr=false → error.
let mut config = Config {
@@ -1358,7 +1581,7 @@ peers:
// Empty addresses + via_nostr=true + nostr.enabled=true → ok.
config.peers[0].via_nostr = true;
config.node.discovery.nostr.enabled = true;
config.node.rendezvous.nostr.enabled = true;
config
.validate()
.expect("via_nostr should allow empty addresses");
@@ -1367,8 +1590,8 @@ peers:
#[test]
fn test_validate_nat_udp_advert_requires_relays_and_stun() {
let mut config = Config::default();
config.node.discovery.nostr.enabled = true;
config.node.discovery.nostr.dm_relays.clear();
config.node.rendezvous.nostr.enabled = true;
config.node.rendezvous.nostr.dm_relays.clear();
config.transports.udp = TransportInstances::Single(UdpConfig {
advertise_on_nostr: Some(true),
public: Some(false),
@@ -1378,8 +1601,8 @@ peers:
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("dm_relays"));
config.node.discovery.nostr.dm_relays = vec!["wss://relay.example".to_string()];
config.node.discovery.nostr.stun_servers.clear();
config.node.rendezvous.nostr.dm_relays = vec!["wss://relay.example".to_string()];
config.node.rendezvous.nostr.stun_servers.clear();
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("stun_servers"));
}
@@ -1459,6 +1682,86 @@ peers:
.expect("outbound_only should be exempt from the loopback check");
}
#[test]
fn test_validate_default_rekey_settings_ok() {
Config::default()
.validate()
.expect("shipped default rekey settings must validate");
}
#[test]
fn test_validate_rekey_after_messages_zero_rejected() {
let mut config = Config::default();
config.node.rekey.after_messages = 0;
let err = config.validate().expect_err("validation should fail");
let msg = err.to_string();
assert!(msg.contains("after_messages"), "got: {msg}");
}
#[test]
fn test_validate_rekey_after_messages_one_accepted() {
let mut config = Config::default();
config.node.rekey.after_messages = 1;
config
.validate()
.expect("after_messages = 1 rekeys every message, which is wasteful but well defined");
}
#[test]
fn test_validate_rekey_after_secs_at_or_below_jitter_rejected() {
let jitter = REKEY_JITTER_SECS.unsigned_abs();
for after_secs in [0, 1, jitter - 1, jitter] {
let mut config = Config::default();
config.node.rekey.after_secs = after_secs;
match config.validate() {
Err(e) => assert!(e.to_string().contains("after_secs"), "got: {e}"),
Ok(()) => panic!("after_secs = {after_secs} should be rejected"),
}
}
}
#[test]
fn test_validate_rekey_after_secs_just_above_jitter_accepted() {
let mut config = Config::default();
config.node.rekey.after_secs = REKEY_JITTER_SECS.unsigned_abs() + 1;
config
.validate()
.expect("one second above the jitter bound leaves a non-zero effective interval");
}
#[test]
fn test_validate_rekey_unbounded_values_accepted() {
let mut config = Config::default();
config.node.rekey.after_secs = u64::MAX;
config.node.rekey.after_messages = u64::MAX;
config
.validate()
.expect("u64::MAX disables an arm of the trigger and must stay legal");
}
#[test]
fn test_validate_rekey_checked_even_when_disabled() {
let mut config = Config::default();
config.node.rekey.enabled = false;
config.node.rekey.after_messages = 0;
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("after_messages"));
let mut config = Config::default();
config.node.rekey.enabled = false;
config.node.rekey.after_secs = REKEY_JITTER_SECS.unsigned_abs();
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("after_secs"));
}
#[test]
fn test_outbound_only_forces_ephemeral_bind() {
let cfg = UdpConfig {
src/config/node.rs
+238 -76
View File
@@ -7,7 +7,7 @@
use serde::{Deserialize, Serialize};
use super::IdentityConfig;
use crate::mmp::{DEFAULT_LOG_INTERVAL_SECS, DEFAULT_OWD_WINDOW_SIZE, MmpConfig, MmpMode};
use crate::proto::mmp::{DEFAULT_LOG_INTERVAL_SECS, DEFAULT_OWD_WINDOW_SIZE, MmpMode};
// ============================================================================
// Node Configuration Subsections
@@ -186,48 +186,42 @@ impl CacheConfig {
}
}
/// Discovery protocol (`node.discovery.*`).
/// Mesh-lookup protocol (`node.lookup.*`): the overlay coordinate-lookup
/// engine (address → coordinates). The peer-rendezvous keys that used to
/// share this table (`nostr`/`lan`) now live under [`RendezvousConfig`]
/// (`node.rendezvous.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DiscoveryConfig {
/// Hop limit for LookupRequest flood (`node.discovery.ttl`).
#[serde(default = "DiscoveryConfig::default_ttl")]
pub struct LookupConfig {
/// Hop limit for LookupRequest flood (`node.lookup.ttl`).
#[serde(default = "LookupConfig::default_ttl")]
pub ttl: u8,
/// Per-attempt timeouts in seconds (`node.discovery.attempt_timeouts_secs`).
/// Per-attempt timeouts in seconds (`node.lookup.attempt_timeouts_secs`).
/// Each entry is the time to wait for a response before sending the next
/// LookupRequest (with a fresh request_id). Sequence length determines the
/// total number of attempts before declaring the destination unreachable.
/// Default `[1, 2, 4, 8]` gives 4 attempts and a 15s total budget.
#[serde(default = "DiscoveryConfig::default_attempt_timeouts_secs")]
#[serde(default = "LookupConfig::default_attempt_timeouts_secs")]
pub attempt_timeouts_secs: Vec<u64>,
/// Dedup cache expiry in seconds (`node.discovery.recent_expiry_secs`).
#[serde(default = "DiscoveryConfig::default_recent_expiry_secs")]
/// Dedup cache expiry in seconds (`node.lookup.recent_expiry_secs`).
#[serde(default = "LookupConfig::default_recent_expiry_secs")]
pub recent_expiry_secs: u64,
/// Base backoff after lookup failure in seconds (`node.discovery.backoff_base_secs`).
/// Base backoff after lookup failure in seconds (`node.lookup.backoff_base_secs`).
/// Doubles per consecutive failure up to `backoff_max_secs`. Defaults to 0
/// (no post-failure suppression); the per-attempt sequence in
/// `attempt_timeouts_secs` provides the only retry pacing.
#[serde(default = "DiscoveryConfig::default_backoff_base_secs")]
#[serde(default = "LookupConfig::default_backoff_base_secs")]
pub backoff_base_secs: u64,
/// Maximum backoff cap in seconds (`node.discovery.backoff_max_secs`).
#[serde(default = "DiscoveryConfig::default_backoff_max_secs")]
/// Maximum backoff cap in seconds (`node.lookup.backoff_max_secs`).
#[serde(default = "LookupConfig::default_backoff_max_secs")]
pub backoff_max_secs: u64,
/// Minimum interval between forwarded lookups for the same target in seconds
/// (`node.discovery.forward_min_interval_secs`).
/// (`node.lookup.forward_min_interval_secs`).
/// Defense-in-depth against misbehaving nodes.
#[serde(default = "DiscoveryConfig::default_forward_min_interval_secs")]
#[serde(default = "LookupConfig::default_forward_min_interval_secs")]
pub forward_min_interval_secs: u64,
/// Nostr-mediated overlay endpoint discovery.
#[serde(default = "DiscoveryConfig::default_nostr")]
pub nostr: NostrDiscoveryConfig,
/// mDNS / DNS-SD peer discovery on the local link. Identity surface
/// is a strict subset of what `nostr.advertise` already publishes
/// publicly, so there's no marginal privacy cost; the latency win
/// for same-LAN peers is large (sub-second pairing, no relay).
#[serde(default = "DiscoveryConfig::default_lan")]
pub lan: crate::discovery::lan::LanDiscoveryConfig,
}
impl Default for DiscoveryConfig {
impl Default for LookupConfig {
fn default() -> Self {
Self {
ttl: 64,
@@ -236,13 +230,11 @@ impl Default for DiscoveryConfig {
backoff_base_secs: 0,
backoff_max_secs: 0,
forward_min_interval_secs: 2,
nostr: NostrDiscoveryConfig::default(),
lan: crate::discovery::lan::LanDiscoveryConfig::default(),
}
}
}
impl DiscoveryConfig {
impl LookupConfig {
fn default_ttl() -> u8 {
64
}
@@ -261,12 +253,45 @@ impl DiscoveryConfig {
fn default_forward_min_interval_secs() -> u64 {
2
}
fn default_nostr() -> NostrDiscoveryConfig {
NostrDiscoveryConfig::default()
}
fn default_lan() -> crate::discovery::lan::LanDiscoveryConfig {
crate::discovery::lan::LanDiscoveryConfig::default()
}
}
/// Peer rendezvous (`node.rendezvous.*`): how the node finds peers to connect
/// to at all — Nostr-mediated overlay endpoints and mDNS/DNS-SD on the local
/// link. Distinct from mesh lookup ([`LookupConfig`]), which finds coordinates
/// for an already-known mesh address.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct RendezvousConfig {
/// Nostr-mediated overlay endpoint rendezvous (`node.rendezvous.nostr.*`).
#[serde(default)]
pub nostr: NostrRendezvousConfig,
/// mDNS / DNS-SD peer rendezvous on the local link (`node.rendezvous.lan.*`).
/// Identity surface is a strict subset of what `nostr.advertise` already
/// publishes publicly, so there's no marginal privacy cost; the latency
/// win for same-LAN peers is large (sub-second pairing, no relay).
#[serde(default)]
pub lan: crate::mdns::LanRendezvousConfig,
}
/// COMPAT (drop at the v2 cutover): a deprecated legacy `node.discovery:` block.
///
/// The `node.discovery.*` table was split into `node.lookup.*` (mesh-lookup
/// scalars) and `node.rendezvous.*` (nostr/LAN peer rendezvous). Because
/// `NodeConfig` does not deny unknown fields, a still-deployed `node.discovery:`
/// block would otherwise deserialize into nothing and silently revert every
/// lookup/rendezvous setting to its default. This all-`Option` mirror captures
/// it so [`Config::normalize_deprecated_keys`] can fold it into the new tables
/// with a one-time deprecation warning; unset legacy keys stay `None` and leave
/// the new-table defaults intact.
#[derive(Debug, Clone, Deserialize)]
pub(crate) struct DiscoveryConfigCompat {
pub ttl: Option<u8>,
pub attempt_timeouts_secs: Option<Vec<u64>>,
pub recent_expiry_secs: Option<u64>,
pub backoff_base_secs: Option<u64>,
pub backoff_max_secs: Option<u64>,
pub forward_min_interval_secs: Option<u64>,
pub nostr: Option<NostrRendezvousConfig>,
pub lan: Option<crate::mdns::LanRendezvousConfig>,
}
/// Nostr advert discovery policy.
@@ -278,33 +303,33 @@ impl DiscoveryConfig {
/// - `open`: also consider adverts for non-configured peers
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum NostrDiscoveryPolicy {
pub enum NostrRendezvousPolicy {
Disabled,
#[default]
ConfiguredOnly,
Open,
}
/// Nostr-mediated overlay endpoint discovery (`node.discovery.nostr.*`).
/// Nostr-mediated overlay endpoint discovery (`node.rendezvous.nostr.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct NostrDiscoveryConfig {
pub struct NostrRendezvousConfig {
/// Enable Nostr-signaled traversal bootstrap.
#[serde(default)]
pub enabled: bool,
/// Publish service advertisements so remote peers can bootstrap inbound.
#[serde(default = "NostrDiscoveryConfig::default_advertise")]
#[serde(default = "NostrRendezvousConfig::default_advertise")]
pub advertise: bool,
/// Relay URLs used for service advertisements.
#[serde(default = "NostrDiscoveryConfig::default_advert_relays")]
#[serde(default = "NostrRendezvousConfig::default_advert_relays")]
pub advert_relays: Vec<String>,
/// Relay URLs used for encrypted signaling events.
#[serde(default = "NostrDiscoveryConfig::default_dm_relays")]
#[serde(default = "NostrRendezvousConfig::default_dm_relays")]
pub dm_relays: Vec<String>,
/// STUN servers used for local reflexive address discovery.
/// Outbound observation uses only this local list; peer-advertised STUN
/// values are informational and are not treated as egress targets.
#[serde(default = "NostrDiscoveryConfig::default_stun_servers")]
#[serde(default = "NostrRendezvousConfig::default_stun_servers")]
pub stun_servers: Vec<String>,
/// Whether to advertise local (RFC 1918 / ULA) interface addresses as
/// host candidates in the traversal offer.
@@ -318,85 +343,85 @@ pub struct NostrDiscoveryConfig {
#[serde(default)]
pub share_local_candidates: bool,
/// Traversal application namespace and advert identifier suffix.
#[serde(default = "NostrDiscoveryConfig::default_app")]
#[serde(default = "NostrRendezvousConfig::default_app")]
pub app: String,
/// Signaling TTL in seconds.
#[serde(default = "NostrDiscoveryConfig::default_signal_ttl_secs")]
#[serde(default = "NostrRendezvousConfig::default_signal_ttl_secs")]
pub signal_ttl_secs: u64,
/// Policy for advert-derived endpoint discovery.
#[serde(default)]
pub policy: NostrDiscoveryPolicy,
pub policy: NostrRendezvousPolicy,
/// Max number of open-discovery peers queued for outbound retry/connection
/// at once. Prevents unbounded queue growth from ambient advert traffic.
#[serde(default = "NostrDiscoveryConfig::default_open_discovery_max_pending")]
#[serde(default = "NostrRendezvousConfig::default_open_discovery_max_pending")]
pub open_discovery_max_pending: usize,
/// Max concurrent inbound traversal offers processed at once.
/// Acts as a rate limit against offer spam from relays.
#[serde(default = "NostrDiscoveryConfig::default_max_concurrent_incoming_offers")]
#[serde(default = "NostrRendezvousConfig::default_max_concurrent_incoming_offers")]
pub max_concurrent_incoming_offers: usize,
/// Max cached overlay adverts retained from relay traffic.
/// Bounds memory under ambient advert volume.
#[serde(default = "NostrDiscoveryConfig::default_advert_cache_max_entries")]
#[serde(default = "NostrRendezvousConfig::default_advert_cache_max_entries")]
pub advert_cache_max_entries: usize,
/// Max seen-session IDs retained for replay detection.
/// Oldest entries are evicted when the cap is exceeded.
#[serde(default = "NostrDiscoveryConfig::default_seen_sessions_max_entries")]
#[serde(default = "NostrRendezvousConfig::default_seen_sessions_max_entries")]
pub seen_sessions_max_entries: usize,
/// Overall punch attempt timeout in seconds.
#[serde(default = "NostrDiscoveryConfig::default_attempt_timeout_secs")]
#[serde(default = "NostrRendezvousConfig::default_attempt_timeout_secs")]
pub attempt_timeout_secs: u64,
/// Replay tracking retention window in seconds.
#[serde(default = "NostrDiscoveryConfig::default_replay_window_secs")]
#[serde(default = "NostrRendezvousConfig::default_replay_window_secs")]
pub replay_window_secs: u64,
/// Delay before punch traffic starts.
#[serde(default = "NostrDiscoveryConfig::default_punch_start_delay_ms")]
#[serde(default = "NostrRendezvousConfig::default_punch_start_delay_ms")]
pub punch_start_delay_ms: u64,
/// Interval between punch packets.
#[serde(default = "NostrDiscoveryConfig::default_punch_interval_ms")]
#[serde(default = "NostrRendezvousConfig::default_punch_interval_ms")]
pub punch_interval_ms: u64,
/// How long to keep punching before failure.
#[serde(default = "NostrDiscoveryConfig::default_punch_duration_ms")]
#[serde(default = "NostrRendezvousConfig::default_punch_duration_ms")]
pub punch_duration_ms: u64,
/// Advert TTL in seconds.
#[serde(default = "NostrDiscoveryConfig::default_advert_ttl_secs")]
#[serde(default = "NostrRendezvousConfig::default_advert_ttl_secs")]
pub advert_ttl_secs: u64,
/// How often adverts are refreshed in seconds.
#[serde(default = "NostrDiscoveryConfig::default_advert_refresh_secs")]
#[serde(default = "NostrRendezvousConfig::default_advert_refresh_secs")]
pub advert_refresh_secs: u64,
/// Settle delay in seconds after Nostr discovery starts before the
/// one-shot startup sweep of cached adverts runs. Allows the relay
/// subscription backlog to populate the in-memory advert cache.
/// Only used under `policy: open`. Default: 5.
#[serde(default = "NostrDiscoveryConfig::default_startup_sweep_delay_secs")]
#[serde(default = "NostrRendezvousConfig::default_startup_sweep_delay_secs")]
pub startup_sweep_delay_secs: u64,
/// Maximum age in seconds for cached adverts considered by the
/// one-shot startup sweep. Adverts whose `created_at` is older than
/// `now - startup_sweep_max_age_secs` are skipped. Only used under
/// `policy: open`. Default: 3600 (1 hour).
#[serde(default = "NostrDiscoveryConfig::default_startup_sweep_max_age_secs")]
#[serde(default = "NostrRendezvousConfig::default_startup_sweep_max_age_secs")]
pub startup_sweep_max_age_secs: u64,
/// Number of consecutive NAT-traversal failures against a peer before
/// an extended cooldown is applied to throttle further offer publishes.
/// At this threshold the daemon also actively re-fetches the peer's
/// advert from `advert_relays` to evict cache entries for peers that
/// have gone away. Default: 5.
#[serde(default = "NostrDiscoveryConfig::default_failure_streak_threshold")]
#[serde(default = "NostrRendezvousConfig::default_failure_streak_threshold")]
pub failure_streak_threshold: u32,
/// Cooldown applied to a peer once `failure_streak_threshold` is hit.
/// Suppresses both open-discovery sweep enqueues and per-attempt
/// retry firings until elapsed. Default: 1800 (30 minutes).
#[serde(default = "NostrDiscoveryConfig::default_extended_cooldown_secs")]
#[serde(default = "NostrRendezvousConfig::default_extended_cooldown_secs")]
pub extended_cooldown_secs: u64,
/// Minimum interval between `NAT traversal failed` WARN log lines for
/// the same peer. Subsequent failures inside the window log at DEBUG.
/// Reduces log spam on public-test nodes with many cache-learned
/// peers. Default: 300 (5 minutes).
#[serde(default = "NostrDiscoveryConfig::default_warn_log_interval_secs")]
#[serde(default = "NostrRendezvousConfig::default_warn_log_interval_secs")]
pub warn_log_interval_secs: u64,
/// Maximum entries retained in the per-npub failure-state map.
/// Bounds memory under high cache turnover. Oldest entries (by last
/// failure time) evicted when the cap is exceeded. Default: 4096.
#[serde(default = "NostrDiscoveryConfig::default_failure_state_max_entries")]
#[serde(default = "NostrRendezvousConfig::default_failure_state_max_entries")]
pub failure_state_max_entries: usize,
/// Cooldown applied after observing a fatal protocol mismatch on a
/// Nostr-adopted bootstrap transport (e.g. `Unknown FMP version`
@@ -404,11 +429,11 @@ pub struct NostrDiscoveryConfig {
/// of `extended_cooldown_secs` and much longer because the mismatch
/// is structural — re-traversing the peer is wasted effort until one
/// side upgrades. Default: 86400 (24 hours).
#[serde(default = "NostrDiscoveryConfig::default_protocol_mismatch_cooldown_secs")]
#[serde(default = "NostrRendezvousConfig::default_protocol_mismatch_cooldown_secs")]
pub protocol_mismatch_cooldown_secs: u64,
}
impl Default for NostrDiscoveryConfig {
impl Default for NostrRendezvousConfig {
fn default() -> Self {
Self {
enabled: false,
@@ -419,7 +444,7 @@ impl Default for NostrDiscoveryConfig {
share_local_candidates: false,
app: Self::default_app(),
signal_ttl_secs: Self::default_signal_ttl_secs(),
policy: NostrDiscoveryPolicy::default(),
policy: NostrRendezvousPolicy::default(),
open_discovery_max_pending: Self::default_open_discovery_max_pending(),
max_concurrent_incoming_offers: Self::default_max_concurrent_incoming_offers(),
advert_cache_max_entries: Self::default_advert_cache_max_entries(),
@@ -442,7 +467,7 @@ impl Default for NostrDiscoveryConfig {
}
}
impl NostrDiscoveryConfig {
impl NostrRendezvousConfig {
fn default_advertise() -> bool {
true
}
@@ -472,7 +497,11 @@ impl NostrDiscoveryConfig {
}
fn default_app() -> String {
"fips-overlay-v1".to_string()
// Branch-specific default. `next` runs FMP-v1 which is wire-
// incompatible with `master`'s FMP-v0, so the two namespaces
// separate the discovery overlays by default — operators who
// want cross-branch discovery can override here.
"fips-overlay-v1-next".to_string()
}
fn default_signal_ttl_secs() -> u64 {
@@ -726,6 +755,41 @@ impl SessionConfig {
}
}
/// MMP configuration (`node.mmp.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MmpConfig {
/// Operating mode (`node.mmp.mode`).
#[serde(default)]
pub mode: MmpMode,
/// Periodic operator log interval in seconds (`node.mmp.log_interval_secs`).
#[serde(default = "MmpConfig::default_log_interval_secs")]
pub log_interval_secs: u64,
/// OWD trend ring buffer size (`node.mmp.owd_window_size`).
#[serde(default = "MmpConfig::default_owd_window_size")]
pub owd_window_size: usize,
}
impl Default for MmpConfig {
fn default() -> Self {
Self {
mode: MmpMode::default(),
log_interval_secs: DEFAULT_LOG_INTERVAL_SECS,
owd_window_size: DEFAULT_OWD_WINDOW_SIZE,
}
}
}
impl MmpConfig {
fn default_log_interval_secs() -> u64 {
DEFAULT_LOG_INTERVAL_SECS
}
fn default_owd_window_size() -> usize {
DEFAULT_OWD_WINDOW_SIZE
}
}
/// Session-layer Metrics Measurement Protocol (`node.session_mmp.*`).
///
/// Separate from link-layer `node.mmp.*` to allow independent mode/interval
@@ -949,7 +1013,18 @@ pub struct NodeConfig {
#[serde(default)]
pub identity: IdentityConfig,
/// Non-routing mode (`node.disable_routing`).
///
/// Tree participation and one-way bloom receipt, but no transit
/// forwarding or bloom combination/propagation. Overridden by
/// `leaf_only` (leaf implies non-routing).
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub disable_routing: bool,
/// Leaf-only mode (`node.leaf_only`).
///
/// Single upstream peer, no tree/bloom/transit. Implies
/// `disable_routing`.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub leaf_only: bool,
@@ -970,6 +1045,19 @@ pub struct NodeConfig {
#[serde(default = "NodeConfig::default_link_dead_timeout_secs")]
pub link_dead_timeout_secs: u64,
/// Graceful-shutdown drain deadline in seconds (`node.drain_timeout_secs`).
/// The bounded `Draining` phase broadcasts a shutdown `Disconnect` and then
/// waits up to this long for peers to clear before tearing down, early-
/// exiting as soon as all peers are gone. `None` selects the 2-second
/// default (see [`NodeConfig::drain_timeout`]).
///
/// Kept `Option` deliberately: `NodeConfig` has no `deny_unknown_fields`, so
/// a naive non-`Option` add with a `default` fn would silently rewrite the
/// value into deployed configs on the next serialize. The `Option` +
/// `skip_serializing_if` keeps absent configs absent.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub drain_timeout_secs: Option<u64>,
/// Resource limits (`node.limits.*`).
#[serde(default)]
pub limits: LimitsConfig,
@@ -986,9 +1074,19 @@ pub struct NodeConfig {
#[serde(default)]
pub cache: CacheConfig,
/// Discovery protocol (`node.discovery.*`).
/// Mesh-lookup protocol (`node.lookup.*`).
#[serde(default)]
pub discovery: DiscoveryConfig,
pub lookup: LookupConfig,
/// Peer rendezvous (`node.rendezvous.*`).
#[serde(default)]
pub rendezvous: RendezvousConfig,
/// COMPAT (drop at the v2 cutover): a deprecated legacy `node.discovery:`
/// block, folded into `lookup`/`rendezvous` by
/// [`Config::normalize_deprecated_keys`]. Never re-serialized.
#[serde(default, skip_serializing)]
pub(crate) discovery: Option<DiscoveryConfigCompat>,
/// Spanning tree (`node.tree.*`).
#[serde(default)]
@@ -1036,16 +1134,20 @@ impl Default for NodeConfig {
fn default() -> Self {
Self {
identity: IdentityConfig::default(),
disable_routing: false,
leaf_only: false,
tick_interval_secs: 1,
base_rtt_ms: 100,
heartbeat_interval_secs: 10,
link_dead_timeout_secs: 30,
drain_timeout_secs: None,
limits: LimitsConfig::default(),
rate_limit: RateLimitConfig::default(),
retry: RetryConfig::default(),
cache: CacheConfig::default(),
discovery: DiscoveryConfig::default(),
lookup: LookupConfig::default(),
rendezvous: RendezvousConfig::default(),
discovery: None,
tree: TreeConfig::default(),
bloom: BloomConfig::default(),
session: SessionConfig::default(),
@@ -1089,12 +1191,72 @@ impl NodeConfig {
fn default_link_dead_timeout_secs() -> u64 {
30
}
/// Graceful-shutdown drain deadline as a `Duration`.
///
/// Returns the configured `drain_timeout_secs`, or the 2-second default
/// when unset. Used by the daemon's bounded `Draining` phase.
pub fn drain_timeout(&self) -> std::time::Duration {
std::time::Duration::from_secs(self.drain_timeout_secs.unwrap_or(2))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_config_default() {
let config = MmpConfig::default();
assert_eq!(config.mode, MmpMode::Full);
assert_eq!(config.log_interval_secs, 30);
assert_eq!(config.owd_window_size, 32);
}
#[test]
fn test_config_yaml_parse() {
let yaml = r#"
mode: lightweight
log_interval_secs: 60
owd_window_size: 48
"#;
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Lightweight);
assert_eq!(config.log_interval_secs, 60);
assert_eq!(config.owd_window_size, 48);
}
#[test]
fn test_config_yaml_partial() {
let yaml = "mode: minimal";
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Minimal);
assert_eq!(config.log_interval_secs, DEFAULT_LOG_INTERVAL_SECS);
assert_eq!(config.owd_window_size, DEFAULT_OWD_WINDOW_SIZE);
}
#[test]
fn test_drain_timeout_default_and_override() {
// Unset → the 2-second default.
let c = NodeConfig::default();
assert_eq!(c.drain_timeout_secs, None);
assert_eq!(c.drain_timeout(), std::time::Duration::from_secs(2));
// Explicit override is honored.
let c2 = NodeConfig {
drain_timeout_secs: Some(10),
..NodeConfig::default()
};
assert_eq!(c2.drain_timeout(), std::time::Duration::from_secs(10));
// A zero override is a valid (immediate) drain, not the default.
let c3 = NodeConfig {
drain_timeout_secs: Some(0),
..NodeConfig::default()
};
assert_eq!(c3.drain_timeout(), std::time::Duration::from_secs(0));
}
#[test]
fn test_ecn_config_defaults() {
let c = EcnConfig::default();
@@ -1123,27 +1285,27 @@ mod tests {
}
#[test]
fn test_nostr_discovery_startup_sweep_defaults() {
let c = NostrDiscoveryConfig::default();
fn test_nostr_rendezvous_startup_sweep_defaults() {
let c = NostrRendezvousConfig::default();
assert_eq!(c.startup_sweep_delay_secs, 5);
assert_eq!(c.startup_sweep_max_age_secs, 3_600);
}
#[test]
fn test_nostr_discovery_startup_sweep_yaml_override() {
fn test_nostr_rendezvous_startup_sweep_yaml_override() {
let yaml = "enabled: true\npolicy: open\nstartup_sweep_delay_secs: 10\nstartup_sweep_max_age_secs: 1800\n";
let c: NostrDiscoveryConfig = serde_yaml::from_str(yaml).unwrap();
let c: NostrRendezvousConfig = serde_yaml::from_str(yaml).unwrap();
assert!(c.enabled);
assert_eq!(c.policy, NostrDiscoveryPolicy::Open);
assert_eq!(c.policy, NostrRendezvousPolicy::Open);
assert_eq!(c.startup_sweep_delay_secs, 10);
assert_eq!(c.startup_sweep_max_age_secs, 1_800);
}
#[test]
fn test_nostr_discovery_startup_sweep_partial_yaml_uses_defaults() {
fn test_nostr_rendezvous_startup_sweep_partial_yaml_uses_defaults() {
// Only override delay; max_age should fall back to default.
let yaml = "enabled: true\nstartup_sweep_delay_secs: 30\n";
let c: NostrDiscoveryConfig = serde_yaml::from_str(yaml).unwrap();
let c: NostrRendezvousConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(c.startup_sweep_delay_secs, 30);
assert_eq!(c.startup_sweep_max_age_secs, 3_600);
}
src/config/transport.rs
+25 -6
View File
@@ -282,9 +282,10 @@ pub struct EthernetConfig {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub send_buf_size: Option<usize>,
/// Listen for discovery beacons from other nodes. Default: true.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub discovery: Option<bool>,
/// Listen for neighbor beacons from other nodes. Default: true.
/// (Renamed from `discovery`; the old key is still accepted.)
#[serde(default, alias = "discovery", skip_serializing_if = "Option::is_none")]
pub listen: Option<bool>,
/// Broadcast announcement beacons on the LAN. Default: false.
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -319,9 +320,9 @@ impl EthernetConfig {
self.send_buf_size.unwrap_or(DEFAULT_ETHERNET_SEND_BUF)
}
/// Whether to listen for discovery beacons. Default: true.
pub fn discovery(&self) -> bool {
self.discovery.unwrap_or(true)
/// Whether to listen for neighbor beacons. Default: true.
pub fn listen(&self) -> bool {
self.listen.unwrap_or(true)
}
/// Whether to broadcast announcement beacons. Default: false.
@@ -1046,4 +1047,22 @@ mod tests {
assert_eq!(parse_bind_port("[::]:443"), Some(443));
assert_eq!(parse_bind_port("not-a-socket-addr"), None);
}
#[test]
fn ethernet_listen_accepts_legacy_discovery_alias_and_rejects_unknown() {
// (a) The legacy `discovery:` key is still accepted via serde alias.
let legacy: EthernetConfig =
serde_yaml::from_str("interface: eth0\ndiscovery: true\n").unwrap();
assert_eq!(legacy.listen, Some(true));
// (b) The new canonical `listen:` key parses into the renamed field.
let renamed: EthernetConfig =
serde_yaml::from_str("interface: eth0\nlisten: true\n").unwrap();
assert_eq!(renamed.listen, Some(true));
// (c) `deny_unknown_fields` still rejects an unknown ethernet key.
let bogus: Result<EthernetConfig, _> =
serde_yaml::from_str("interface: eth0\nbogus: true\n");
assert!(bogus.is_err());
}
}
src/control/queries.rs
+81 -59
View File
@@ -236,7 +236,7 @@ pub fn show_peers(node: &Node) -> Value {
// Per-npub Nostr-traversal failure-state snapshot, indexed by npub
// for O(1) per-peer lookup. Empty if Nostr discovery is disabled.
let nostr_state: std::collections::HashMap<String, _> = node
.nostr_discovery_handle()
.nostr_rendezvous_handle()
.map(|d| {
d.failure_state_snapshot()
.into_iter()
@@ -988,56 +988,70 @@ pub(crate) fn show_bloom_from_handle(handle: &super::read_handle::ControlReadHan
/// `show_mmp` — MMP metrics summary.
pub fn show_mmp(node: &Node) -> Value {
// Link-layer MMP per peer
let peers: Vec<Value> = node.peers().filter_map(|peer| {
let mmp = peer.mmp()?;
let addr = *peer.node_addr();
let metrics = &mmp.metrics;
let peers: Vec<Value> = node
.peers()
.filter_map(|peer| {
let mmp = peer.mmp()?;
let addr = *peer.node_addr();
let metrics = &mmp.metrics;
let mut link_layer = json!({
"loss_rate": metrics.loss_rate(),
"etx": metrics.etx,
"goodput_bps": metrics.goodput_bps,
"spin_bit_role": if mmp.spin_bit.is_initiator() { "initiator" } else { "responder" },
});
let mut link_layer = json!({
"loss_rate": metrics.loss_rate(),
"etx": metrics.etx,
"goodput_bps": metrics.goodput_bps,
});
if let Some(smoothed_loss) = metrics.smoothed_loss() {
link_layer["smoothed_loss"] = json!(smoothed_loss);
}
if let Some(smoothed_etx) = metrics.smoothed_etx() {
link_layer["smoothed_etx"] = json!(smoothed_etx);
}
if let Some(srtt) = metrics.srtt_ms() {
link_layer["srtt_ms"] = json!(srtt);
if let Some(setx) = metrics.smoothed_etx() {
link_layer["lqi"] = json!(setx * (1.0 + srtt / 100.0));
if let Some(smoothed_loss) = metrics.smoothed_loss() {
link_layer["smoothed_loss"] = json!(smoothed_loss);
}
if let Some(smoothed_etx) = metrics.smoothed_etx() {
link_layer["smoothed_etx"] = json!(smoothed_etx);
}
if let Some(srtt) = metrics.srtt_ms() {
link_layer["srtt_ms"] = json!(srtt);
if let Some(setx) = metrics.smoothed_etx() {
link_layer["lqi"] = json!(setx * (1.0 + srtt / 100.0));
}
}
}
// Trend indicators
if metrics.rtt_trend.initialized() {
link_layer["rtt_trend"] = json!(trend_label(metrics.rtt_trend.short(), metrics.rtt_trend.long()));
}
if metrics.loss_trend.initialized() {
link_layer["loss_trend"] = json!(trend_label(metrics.loss_trend.short(), metrics.loss_trend.long()));
}
if metrics.goodput_trend.initialized() {
link_layer["goodput_trend"] = json!(trend_label(metrics.goodput_trend.short(), metrics.goodput_trend.long()));
}
if metrics.jitter_trend.initialized() {
link_layer["jitter_trend"] = json!(trend_label(metrics.jitter_trend.short(), metrics.jitter_trend.long()));
}
// Trend indicators
if metrics.rtt_trend.initialized() {
link_layer["rtt_trend"] = json!(trend_label(
metrics.rtt_trend.short(),
metrics.rtt_trend.long()
));
}
if metrics.loss_trend.initialized() {
link_layer["loss_trend"] = json!(trend_label(
metrics.loss_trend.short(),
metrics.loss_trend.long()
));
}
if metrics.goodput_trend.initialized() {
link_layer["goodput_trend"] = json!(trend_label(
metrics.goodput_trend.short(),
metrics.goodput_trend.long()
));
}
if metrics.jitter_trend.initialized() {
link_layer["jitter_trend"] = json!(trend_label(
metrics.jitter_trend.short(),
metrics.jitter_trend.long()
));
}
link_layer["delivery_ratio_forward"] = json!(metrics.delivery_ratio_forward);
link_layer["delivery_ratio_reverse"] = json!(metrics.delivery_ratio_reverse);
link_layer["ecn_ce_count"] = json!(metrics.last_ecn_ce_count());
link_layer["delivery_ratio_forward"] = json!(metrics.delivery_ratio_forward);
link_layer["delivery_ratio_reverse"] = json!(metrics.delivery_ratio_reverse);
link_layer["ecn_ce_count"] = json!(metrics.last_ecn_ce_count());
Some(json!({
"peer": hex::encode(addr.as_bytes()),
"display_name": node.peer_display_name(&addr),
"mode": format!("{}", mmp.mode()),
"link_layer": link_layer,
}))
}).collect();
Some(json!({
"peer": hex::encode(addr.as_bytes()),
"display_name": node.peer_display_name(&addr),
"mode": format!("{}", mmp.mode()),
"link_layer": link_layer,
}))
})
.collect();
// Session-layer MMP
let sessions: Vec<Value> = node
@@ -1116,7 +1130,6 @@ pub(crate) fn show_mmp_from_handle(handle: &super::read_handle::ControlReadHandl
"loss_rate": peer.loss_rate,
"etx": peer.etx,
"goodput_bps": peer.goodput_bps,
"spin_bit_role": if peer.spin_bit_initiator { "initiator" } else { "responder" },
});
if let Some(smoothed_loss) = peer.smoothed_loss {
@@ -1303,17 +1316,18 @@ pub fn show_connections(node: &Node) -> Value {
let now = now_ms();
let connections: Vec<Value> = node
.connections()
.map(|conn| {
.map(|(_, machine)| {
let link_id = machine.link_id();
let mut conn_json = json!({
"link_id": conn.link_id().as_u64(),
"direction": format!("{}", conn.direction()),
"handshake_state": format!("{}", conn.handshake_state()),
"started_at_ms": conn.started_at(),
"idle_ms": now.saturating_sub(conn.last_activity()),
"resend_count": conn.resend_count(),
"link_id": link_id.as_u64(),
"direction": format!("{}", machine.conn_direction()),
"handshake_state": node.connection_handshake_state(link_id),
"started_at_ms": node.connection_started_at(link_id),
"idle_ms": now.saturating_sub(node.connection_last_activity(link_id)),
"resend_count": node.connection_resend_count(link_id),
});
if let Some(identity) = conn.expected_identity() {
if let Some(identity) = node.connection_expected_identity(link_id) {
conn_json["expected_peer"] = json!(identity.npub());
}
@@ -1487,7 +1501,9 @@ pub fn show_routing(node: &Node) -> Value {
"recent_requests": node.recent_request_count(),
"retries": retries,
"forwarding": serde_json::to_value(metrics.forwarding.snapshot()).unwrap_or_default(),
"discovery": serde_json::to_value(metrics.discovery.snapshot()).unwrap_or_default(),
// COMPAT: `discovery` is the deprecated alias for `lookup`; drop at the v2 cutover.
"discovery": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"lookup": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"error_signals": serde_json::to_value(metrics.errors.snapshot()).unwrap_or_default(),
"congestion": serde_json::to_value(metrics.congestion.snapshot()).unwrap_or_default(),
})
@@ -1543,7 +1559,9 @@ pub(crate) fn show_routing_from_handle(handle: &super::read_handle::ControlReadH
"recent_requests": view.recent_requests,
"retries": retries,
"forwarding": serde_json::to_value(metrics.forwarding.snapshot()).unwrap_or_default(),
"discovery": serde_json::to_value(metrics.discovery.snapshot()).unwrap_or_default(),
// COMPAT: `discovery` is the deprecated alias for `lookup`; drop at the v2 cutover.
"discovery": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"lookup": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"error_signals": serde_json::to_value(metrics.errors.snapshot()).unwrap_or_default(),
"congestion": serde_json::to_value(metrics.congestion.snapshot()).unwrap_or_default(),
})
@@ -2328,7 +2346,9 @@ pub(crate) fn show_metrics_from_handle(handle: &super::read_handle::ControlReadH
let m = handle.metrics();
json!({
"forwarding": m.forwarding.snapshot(),
"discovery": m.discovery.snapshot(),
// COMPAT: `discovery` is the deprecated alias for `lookup`; drop at the v2 cutover.
"discovery": m.lookup.snapshot(),
"lookup": m.lookup.snapshot(),
"tree": m.tree.snapshot(),
"bloom": m.bloom.snapshot(),
"congestion": m.congestion.snapshot(),
@@ -2761,12 +2781,12 @@ mod tests {
/// Structural confirmation that the rx_loop no longer dispatches `show_*`:
/// the rx_loop source carries no `queries::dispatch` call and no
/// `starts_with("show_")` routing branch. Reads the committed source of
/// `src/node/handlers/rx_loop.rs` and asserts both markers are absent. This
/// `src/node/dataplane/rx_loop.rs` and asserts both markers are absent. This
/// is the milestone's "remove `show_*` from the data-plane dispatch path"
/// invariant, guarded against regression.
#[test]
fn rx_loop_has_no_show_dispatch() {
let src = include_str!("../node/handlers/rx_loop.rs");
let src = include_str!("../node/dataplane/rx_loop.rs");
assert!(
!src.contains("queries::dispatch"),
"rx_loop must not call queries::dispatch (show_* served off-loop)"
@@ -2794,7 +2814,9 @@ mod tests {
let expected_families = [
("forwarding", "received_packets"),
// `discovery` is the deprecated dual-emit alias for `lookup`; drop at the v2 cutover.
("discovery", "req_received"),
("lookup", "req_received"),
("tree", "accepted"),
("bloom", "accepted"),
("congestion", "ce_forwarded"),
src/control/read_handle.rs
+31
View File
@@ -118,10 +118,41 @@ impl ControlReadHandle {
/// Cutover queries (R1) read only `NodeContext` / `MetricsRegistry` (the state
/// the read handle already bundles) plus host-OS facts (`/proc`, nftables), so
/// they render entirely in the control task without touching `Node`.
///
/// **It now also carries mutating commands**, namely the `profile_tick_*`
/// family under the `profiling` feature. They are served here rather than on
/// the rx_loop deliberately: all of their state is process statics, they need
/// no `&mut Node`, and routing them through the loop would make the toggle
/// queue behind the very behavior it exists to measure.
pub(crate) fn snapshot_dispatch(request: &Request, handle: &ControlReadHandle) -> Option<Response> {
use crate::control::queries;
match request.command.as_str() {
// Tick-body profiler toggle. Present only in a `--features profiling`
// build; otherwise these fall through to the rx_loop dispatch, which
// reports them as unknown commands.
#[cfg(feature = "profiling")]
"profile_tick_on" => {
let dir = request
.params
.as_ref()
.and_then(|p| p.get("dir"))
.and_then(|v| v.as_str());
let context = handle.context();
let npub = context.identity.npub();
let period = context.config.node.tick_interval_secs;
Some(match crate::instr::capture::start(dir, &npub, period) {
Ok(value) => Response::ok(value),
Err(e) => Response::error(e),
})
}
#[cfg(feature = "profiling")]
"profile_tick_off" => Some(match crate::instr::capture::stop() {
Ok(value) => Response::ok(value),
Err(e) => Response::error(e),
}),
#[cfg(feature = "profiling")]
"profile_tick_status" => Some(Response::ok(crate::instr::capture::status())),
"show_listening_sockets" => Some(Response::ok(
queries::show_listening_sockets_from_handle(handle),
)),
src/control/snapshot.rs
-1
View File
@@ -692,7 +692,6 @@ pub(crate) struct MmpPeerRow {
pub loss_rate: f64,
pub etx: f64,
pub goodput_bps: f64,
pub spin_bit_initiator: bool,
pub smoothed_loss: Option<f64>,
pub smoothed_etx: Option<f64>,
pub srtt_ms: Option<f64>,
src/control/snapshots/show_bloom.json
+7
View File
@@ -10,13 +10,20 @@
"accepted": 0,
"debounce_suppressed": 0,
"decode_error": 0,
"deltas_sent": 0,
"fill_exceeded": 0,
"full_sends": 0,
"invalid": 0,
"nacks_received": 0,
"nacks_sent": 0,
"non_v1": 0,
"received": 0,
"send_failed": 0,
"sent": 0,
"size_changes": 0,
"stale": 0,
"total_compressed_bytes": 0,
"total_raw_bytes": 0,
"unknown_peer": 0
},
"uptree_estimated_count": null,
src/control/snapshots/show_routing.json
+24
View File
@@ -63,6 +63,30 @@
"ttl_exhausted_packets": 0
},
"identity_cache_entries": 0,
"lookup": {
"req_backoff_suppressed": 0,
"req_bloom_miss": 0,
"req_decode_error": 0,
"req_dedup_cache_full": 0,
"req_deduplicated": 0,
"req_duplicate": 0,
"req_fallback_forwarded": 0,
"req_forward_rate_limited": 0,
"req_forwarded": 0,
"req_initiated": 0,
"req_no_tree_peer": 0,
"req_received": 0,
"req_target_is_us": 0,
"req_ttl_exhausted": 0,
"resp_accepted": 0,
"resp_decode_error": 0,
"resp_forwarded": 0,
"resp_identity_miss": 0,
"resp_no_route": 0,
"resp_proof_failed": 0,
"resp_received": 0,
"resp_timed_out": 0
},
"pending_lookups": [],
"pending_tun_destinations": 0,
"pending_tun_packets": 0,
src/instr/capture.rs
+418
View File
@@ -0,0 +1,418 @@
//! Capture lifecycle: the arm/disarm state machine, the sink file, and the
//! `fipsctl`-facing operations.
//!
//! The toggle — not the writer — creates and opens the sink and publishes its
//! path, so an unwritable directory fails the `on` command loudly instead of
//! being discovered later by a background thread with nobody to report to.
//!
//! Capture state is a single atomic state machine (`Idle`, `Running`,
//! `StoppedByCap`) transitioned by `compare_exchange`. Every accepted control
//! connection is served by its own spawned task, so two simultaneous `on`
//! requests are genuinely concurrent and must not both create a writer.
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use std::sync::Mutex;
use std::sync::atomic::{AtomicBool, AtomicU8, AtomicU64, Ordering};
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use super::recorder;
use super::writer;
/// Default sink directory. Overridable per capture with `--dir`.
pub(crate) const DEFAULT_DIR: &str = "/var/log/fips";
/// Writer flush interval.
pub(crate) const INTERVAL: Duration = Duration::from_secs(10);
/// Size at which a capture stops itself. Reaching it stops the capture rather
/// than rotating: the point of a capture is a bounded, self-describing window.
pub(crate) const BYTE_CAP: u64 = 32 * 1024 * 1024;
pub(crate) const IDLE: u8 = 0;
pub(crate) const RUNNING: u8 = 1;
pub(crate) const STOPPED_BY_CAP: u8 = 2;
/// The writer could not write and stopped itself. Distinct from a cap stop:
/// a capture that died on a full disk produced a truncated window, and calling
/// that "stopped_by_cap" tells the operator it ran to its limit when it did
/// not. The trailer line explaining it goes to the same failing file, so the
/// state is the only signal that survives.
pub(crate) const STOPPED_BY_ERROR: u8 = 3;
static STATE: AtomicU8 = AtomicU8::new(IDLE);
static GATE: AtomicBool = AtomicBool::new(false);
static BYTES: AtomicU64 = AtomicU64::new(0);
static ACTIVE_PATH: Mutex<Option<PathBuf>> = Mutex::new(None);
static WRITER: Mutex<Option<writer::Handle>> = Mutex::new(None);
/// The per-tick gate. One relaxed load per tick when the feature is compiled in
/// and no capture is running.
#[inline]
pub(crate) fn gate() -> bool {
GATE.load(Ordering::Relaxed)
}
pub(crate) fn bytes_written() -> u64 {
BYTES.load(Ordering::Relaxed)
}
pub(crate) fn add_bytes(n: u64) -> u64 {
BYTES.fetch_add(n, Ordering::Relaxed) + n
}
fn active_path() -> Option<PathBuf> {
ACTIVE_PATH
.lock()
.unwrap_or_else(|e| e.into_inner())
.clone()
}
fn path_display() -> String {
active_path()
.map(|p| p.display().to_string())
.unwrap_or_else(|| "<none>".to_string())
}
fn state_name(state: u8) -> &'static str {
match state {
RUNNING => "running",
STOPPED_BY_CAP => "stopped_by_cap",
STOPPED_BY_ERROR => "stopped_by_error",
_ => "idle",
}
}
/// Called by the writer when it stops itself. `terminal` is `STOPPED_BY_CAP`
/// or `STOPPED_BY_ERROR`. Returns true if this call is the one that stopped it.
pub(crate) fn mark_stopped(terminal: u8) -> bool {
debug_assert!(terminal == STOPPED_BY_CAP || terminal == STOPPED_BY_ERROR);
GATE.store(false, Ordering::Relaxed);
STATE
.compare_exchange(RUNNING, terminal, Ordering::AcqRel, Ordering::Acquire)
.is_ok()
}
/// Join the writer thread, if one exists. Never called while holding another
/// lock the writer might want.
fn reap() {
let handle = WRITER.lock().unwrap_or_else(|e| e.into_inner()).take();
if let Some(handle) = handle {
handle.stop_and_join();
}
}
/// Arm a capture.
///
/// Opens the sink first and only then starts the writer, so a bad `--dir` is
/// reported to the caller rather than logged into the void.
pub(crate) fn start(
dir: Option<&str>,
node_npub: &str,
tick_period_secs: u64,
) -> Result<serde_json::Value, String> {
claim()?;
match open_sink(dir, node_npub, tick_period_secs) {
Ok((file, path, header_len)) => {
recorder::reset();
BYTES.store(header_len, Ordering::Relaxed);
match writer::spawn(file) {
Ok(handle) => {
*WRITER.lock().unwrap_or_else(|e| e.into_inner()) = Some(handle);
*ACTIVE_PATH.lock().unwrap_or_else(|e| e.into_inner()) = Some(path.clone());
GATE.store(true, Ordering::Release);
Ok(serde_json::json!({
"state": "running",
"path": path.display().to_string(),
"interval_secs": INTERVAL.as_secs(),
"byte_cap": BYTE_CAP,
}))
}
Err(e) => {
let _ = std::fs::remove_file(&path);
BYTES.store(0, Ordering::Relaxed);
STATE.store(IDLE, Ordering::Release);
Err(format!("cannot start profile writer thread: {e}"))
}
}
}
Err(e) => {
BYTES.store(0, Ordering::Relaxed);
STATE.store(IDLE, Ordering::Release);
Err(e)
}
}
}
/// Take the capture slot, reaping a cap-stopped predecessor if that is what is
/// in the way.
fn claim() -> Result<(), String> {
match STATE.compare_exchange(IDLE, RUNNING, Ordering::AcqRel, Ordering::Acquire) {
Ok(_) => Ok(()),
Err(RUNNING) => Err(format!("capture already running: {}", path_display())),
Err(stopped @ (STOPPED_BY_CAP | STOPPED_BY_ERROR)) => {
reap();
*ACTIVE_PATH.lock().unwrap_or_else(|e| e.into_inner()) = None;
STATE
.compare_exchange(stopped, RUNNING, Ordering::AcqRel, Ordering::Acquire)
.map(|_| ())
.map_err(|_| "capture state changed concurrently; retry".to_string())
}
Err(_) => Err("capture in an unexpected state".to_string()),
}
}
/// Disarm the capture. Succeeds when nothing is running, reporting so.
pub(crate) fn stop() -> Result<serde_json::Value, String> {
let previous = STATE.load(Ordering::Acquire);
if previous == IDLE {
return Ok(serde_json::json!({"state": "idle", "stopped": false}));
}
GATE.store(false, Ordering::Release);
// The writer wakes on the stop message rather than after the interval, so
// this join returns promptly instead of parking the caller for up to one
// flush interval.
reap();
let path = path_display();
*ACTIVE_PATH.lock().unwrap_or_else(|e| e.into_inner()) = None;
let bytes = bytes_written();
// Clear the counter with the slot: a later `status` while idle must not
// report the previous capture's byte total as though a capture were live.
BYTES.store(0, Ordering::Relaxed);
STATE.store(IDLE, Ordering::Release);
Ok(serde_json::json!({
"state": "idle",
"stopped": true,
"stopped_by_cap": previous == STOPPED_BY_CAP,
"stopped_by_error": previous == STOPPED_BY_ERROR,
"path": path,
"bytes": bytes,
}))
}
/// Report capture state. Distinguishes all four states.
pub(crate) fn status() -> serde_json::Value {
let state = STATE.load(Ordering::Acquire);
serde_json::json!({
"state": state_name(state),
"path": active_path().map(|p| p.display().to_string()),
"bytes": bytes_written(),
"byte_cap": BYTE_CAP,
"interval_secs": INTERVAL.as_secs(),
})
}
/// Stop and reap at daemon teardown. Idempotent.
pub(crate) fn shutdown() {
if STATE.load(Ordering::Acquire) != IDLE {
let _ = stop();
}
}
/// Create the sink file and write its header block. Returns the open file, its
/// path, and the number of header bytes written.
fn open_sink(
dir: Option<&str>,
node_npub: &str,
tick_period_secs: u64,
) -> Result<(File, PathBuf, u64), String> {
let dir = PathBuf::from(dir.unwrap_or(DEFAULT_DIR));
std::fs::create_dir_all(&dir)
.map_err(|e| format!("cannot use profile directory {}: {e}", dir.display()))?;
let start_unix = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let path = dir.join(format!("profile-{}.tsv", compact_utc(start_unix)));
let mut file = File::create(&path)
.map_err(|e| format!("cannot create profile file {}: {e}", path.display()))?;
let header = format!(
"# fips tick profile\n\
# node\t{node}\n\
# build\t{build}\n\
# platform\t{platform}\n\
# tick_period_secs\t{period}\n\
# interval_secs\t{interval}\n\
# byte_cap\t{cap}\n\
# start_utc\t{start_utc}\n\
# start_unix\t{start_unix}\n\
# NOTE\tstep durations are WALL CLOCK across await points, not CPU time:\n\
# NOTE\ta step that awaits I/O accrues the wait, and other tasks may run\n\
# NOTE\tinside that span. That is the intended measure for head-of-line\n\
# NOTE\tdelay; do not read a large step as CPU cost.\n\
# NOTE\tarm_starvation is measured directly as (entry time - the deadline\n\
# NOTE\tthe interval scheduled the tick for). It is NOT derived from\n\
# NOTE\ttick_entry_gap, which carries no starvation signal by itself:\n\
# NOTE\tunder a steady delay every gap is exactly one tick period.\n\
ts_unix\tkind\tdomain\tname\tcount\tmax\ttotal\tunit\n",
node = node_npub,
build = crate::version::short_version(),
platform = std::env::consts::OS,
period = tick_period_secs,
interval = INTERVAL.as_secs(),
cap = BYTE_CAP,
start_utc = iso_utc(start_unix),
start_unix = start_unix,
);
file.write_all(header.as_bytes())
.map_err(|e| format!("cannot write profile header to {}: {e}", path.display()))?;
Ok((file, path, header.len() as u64))
}
/// Break a Unix timestamp into UTC `(year, month, day, hour, minute, second)`.
///
/// Hinnant's `civil_from_days`, era-based. No date crate is in the dependency
/// set and one filename stamp does not justify adding one.
fn utc_parts(unix: u64) -> (i64, u32, u32, u32, u32, u32) {
let days = (unix / 86_400) as i64;
let secs = unix % 86_400;
let z = days + 719_468;
let era = z.div_euclid(146_097);
let doe = z.rem_euclid(146_097);
let yoe = (doe - doe / 1_460 + doe / 36_524 - doe / 146_096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = (doy - (153 * mp + 2) / 5 + 1) as u32;
let m = (if mp < 10 { mp + 3 } else { mp - 9 }) as u32;
let y = if m <= 2 { y + 1 } else { y };
(
y,
m,
d,
(secs / 3_600) as u32,
((secs % 3_600) / 60) as u32,
(secs % 60) as u32,
)
}
/// `20260727T191500Z` — filename-safe.
fn compact_utc(unix: u64) -> String {
let (y, mo, d, h, mi, s) = utc_parts(unix);
format!("{y:04}{mo:02}{d:02}T{h:02}{mi:02}{s:02}Z")
}
/// `2026-07-27T19:15:00Z` — for the header block.
fn iso_utc(unix: u64) -> String {
let (y, mo, d, h, mi, s) = utc_parts(unix);
format!("{y:04}-{mo:02}-{d:02}T{h:02}:{mi:02}:{s:02}Z")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn utc_parts_matches_known_instants() {
assert_eq!(utc_parts(0), (1970, 1, 1, 0, 0, 0));
assert_eq!(utc_parts(946_684_800), (2000, 1, 1, 0, 0, 0));
// 2026-07-27T19:15:00Z
assert_eq!(utc_parts(1_785_179_700), (2026, 7, 27, 19, 15, 0));
// Leap day.
assert_eq!(utc_parts(1_709_164_800), (2024, 2, 29, 0, 0, 0));
}
#[test]
fn stamps_render_expected_shapes() {
assert_eq!(compact_utc(1_785_179_700), "20260727T191500Z");
assert_eq!(iso_utc(1_785_179_700), "2026-07-27T19:15:00Z");
}
// The lock these tests take is shared with the recorder tests, which
// mutate the same statics. See `crate::instr::test_serial`.
#[test]
fn capture_round_trip_writes_header_and_rows() {
let _guard = crate::instr::test_serial();
let dir = tempfile::tempdir().expect("tempdir");
let dir_str = dir.path().to_str().unwrap().to_string();
let started = start(Some(&dir_str), "npub1test", 1).expect("start");
assert_eq!(started["state"], "running");
assert!(gate(), "gate must be armed while running");
let path = PathBuf::from(started["path"].as_str().unwrap());
// A second `on` is refused while one is running, and names the file.
let refused = start(Some(&dir_str), "npub1test", 1).unwrap_err();
assert!(refused.contains(&path.display().to_string()), "{refused}");
// Feed one observation so the drained rows are not all zero.
recorder::record(
recorder::Domain::Tick,
recorder::Step::WholeTick,
Duration::from_millis(7),
);
// Stopping wakes the writer immediately; it drains once more and joins.
let stopped = stop().expect("stop");
assert_eq!(stopped["stopped"], true);
assert_eq!(stopped["stopped_by_cap"], false);
assert!(!gate(), "gate must be clear after stop");
let text = std::fs::read_to_string(&path).expect("read capture");
assert!(text.starts_with("# fips tick profile\n"), "{text}");
assert!(text.contains("# node\tnpub1test\n"), "{text}");
assert!(
text.contains("ts_unix\tkind\tdomain\tname\tcount\tmax\ttotal\tunit\n"),
"{text}"
);
// The final drain emitted one row per emitted step, plus the gauges.
let rows: Vec<&str> = text
.lines()
.filter(|l| l.starts_with(|c: char| c.is_ascii_digit()))
.collect();
let expected_steps = recorder::STEPS.iter().filter(|s| s.emitted()).count();
assert_eq!(rows.len(), expected_steps + recorder::N_GAUGES);
// The 7 ms observation above is in the whole-tick row, converted to
// microseconds. Bounds rather than equality: the gate is process-wide,
// so a node under test elsewhere in this binary may have ticked into
// the same capture window.
let whole_tick = rows
.iter()
.find(|r| r.contains("\tstep\ttick\twhole_tick\t"))
.expect("whole_tick row");
let fields: Vec<&str> = whole_tick.split('\t').collect();
assert_eq!(fields.last(), Some(&"us"), "{whole_tick}");
assert!(
fields[4].parse::<u64>().unwrap() >= 1,
"count: {whole_tick}"
);
assert!(
fields[5].parse::<u64>().unwrap() >= 7_000,
"max: {whole_tick}"
);
assert!(
rows.iter()
.any(|r| r.contains("\tgauge\ttick\tarm_starvation\t")),
"{text}"
);
// A stop with nothing running is not an error.
let again = stop().expect("second stop");
assert_eq!(again["stopped"], false);
}
#[test]
fn start_fails_loudly_on_an_unwritable_directory() {
let _guard = crate::instr::test_serial();
let err = start(Some("/proc/fips-profile-should-not-exist"), "npub1test", 1)
.expect_err("must fail");
assert!(err.contains("profile directory"), "{err}");
// The failed attempt must leave the slot free for the next try.
assert_eq!(STATE.load(Ordering::Acquire), IDLE);
assert!(!gate());
}
#[test]
fn status_reports_the_bounds_it_is_enforcing() {
let _guard = crate::instr::test_serial();
let value = status();
assert_eq!(value["byte_cap"], BYTE_CAP);
assert_eq!(value["interval_secs"], INTERVAL.as_secs());
}
}
src/instr/mod.rs
+171
View File
@@ -0,0 +1,171 @@
//! Tick-body instrumentation.
//!
//! A purpose-built, feature-gated profiler for the rx-loop tick arm. It exists
//! to answer one question with field data: which subsystem step dominates the
//! tick body, and how long does the tick arm wait behind the other `select!`
//! arms before it runs at all.
//!
//! # Shape
//!
//! - Everything that costs anything at runtime is behind the `profiling` Cargo
//! feature, which is **off by default**. The default build's neutrality is a
//! property of the generated code, not of a runtime check.
//! - The instrumentation macro is defined twice, once per feature state. The
//! feature-off definition is a pure pass-through: it expands to the measured
//! expression and nothing else, so no timing code exists in a default build.
//! - The always-present surface — [`gate`], [`tick_entry`], [`tick_gauges`],
//! [`shutdown`] — exists in both feature states because the call sites in
//! `rx_loop.rs` and the lifecycle teardown must compile either way. Their
//! feature-off forms are empty (and [`gate`] is a `const fn` returning
//! `false`), so they cost nothing.
//! - The module is named `instr` rather than `profiling` so that it sorts
//! before `node` in `lib.rs`'s alphabetical module list: a `#[macro_use]`
//! module must be declared before the modules that use its macros.
//!
//! # Data model
//!
//! Domain above step: [`Domain`] carries exactly one variant today
//! (`Domain::Tick`). The primitive, the recorder, the writer and the `fipsctl`
//! surface all take a domain, so adding a data-path domain later is additive.
//! No second domain is declared until something records into it.
//!
//! Per (domain, step) the recorder keeps an exact count, max and total in fixed
//! static `AtomicU64` arrays — no histogram, no accumulation, a fixed footprint
//! regardless of run length. Gauges (ticks per interval, peer count, and the
//! arm-starvation figures) live in a parallel array and are emitted with an
//! explicit row kind so a gauge value never lands under a duration column.
#[cfg(feature = "profiling")]
pub(crate) mod capture;
#[cfg(feature = "profiling")]
mod recorder;
#[cfg(feature = "profiling")]
mod writer;
#[cfg(feature = "profiling")]
pub(crate) use recorder::{Domain, Step, now, record};
// ---------------------------------------------------------------------------
// The macro pair.
//
// Every path in the body is `$crate::`-qualified. `macro_rules!` bodies are not
// path-hygienic: an unqualified `Instant::now()` or `record(..)` would resolve
// at the *call site* (`rx_loop.rs`), where neither name is in scope. Importing
// them there is worse still, because the imports would be unused in the
// feature-off build and red it under `-D warnings`.
//
// `$e` is evaluated exactly once in both forms, which is what makes nesting the
// whole-tick span around the per-step spans safe.
// ---------------------------------------------------------------------------
/// Time `$e` as one step of `$domain`, when `$on` is true.
///
/// `$on` is the per-tick gate hoist: the enable flag is read once at the top of
/// the tick arm into a local, and that local is passed explicitly to every
/// invocation, because macro hygiene makes a call-site local invisible inside
/// the macro body.
#[cfg(feature = "profiling")]
macro_rules! instr_step {
($on:expr, $domain:expr, $step:expr, $e:expr) => {{
let t0 = if $on {
Some($crate::instr::now())
} else {
None
};
let r = $e;
if let Some(t) = t0 {
$crate::instr::record($domain, $step, t.elapsed());
}
r
}};
}
/// Feature-off form: a pure pass-through. The expansion contains no clock read,
/// no counter update and no reference to the recorder — only the measured
/// expression, plus a discard of the gate local so it is not unused.
#[cfg(not(feature = "profiling"))]
macro_rules! instr_step {
($on:expr, $domain:expr, $step:expr, $e:expr) => {{
let _ = &$on;
$e
}};
}
// ---------------------------------------------------------------------------
// Always-present surface.
// ---------------------------------------------------------------------------
/// Whether a capture is armed. Read **once per tick** into a local that is then
/// passed to each `instr_step!` invocation, so the feature-on-but-idle cost of
/// the whole tick arm is a single relaxed load.
#[cfg(feature = "profiling")]
#[inline]
pub(crate) fn gate() -> bool {
capture::gate()
}
/// Feature-off gate: a `const fn` returning `false`, so the whole tick arm
/// folds to the uninstrumented sequence at compile time.
#[cfg(not(feature = "profiling"))]
#[inline]
pub(crate) const fn gate() -> bool {
false
}
/// Record how late this tick-arm entry is against its scheduled deadline.
///
/// The arm is polled **last** under `biased;`, so its lateness is the time it
/// spent waiting behind the packet, TUN and control arms. `tokio::time::
/// interval::tick` returns the deadline it was scheduled for, so this is a
/// direct subtraction rather than a model. Two earlier designs derived it from
/// the inter-entry gap instead and both under-reported: one by the previous
/// body, the other by reporting only the first difference of the delay, so a
/// sustained stall read as zero. The inter-entry gap is still recorded as its
/// own gauge, but it carries no starvation signal on its own.
#[cfg(feature = "profiling")]
#[inline]
pub(crate) fn tick_entry(on: bool, deadline: std::time::Instant, now: std::time::Instant) {
recorder::tick_entry(on, deadline, now);
}
#[cfg(not(feature = "profiling"))]
#[inline]
pub(crate) fn tick_entry(_on: bool, _deadline: std::time::Instant, _now: std::time::Instant) {}
/// Sample the per-tick gauges taken from node state.
#[cfg(feature = "profiling")]
#[inline]
pub(crate) fn tick_gauges(on: bool, peers: u64) {
recorder::tick_gauges(on, peers);
}
#[cfg(not(feature = "profiling"))]
#[inline]
pub(crate) fn tick_gauges(_on: bool, _peers: u64) {}
/// Stop and reap any running capture at daemon teardown. Idempotent.
#[cfg(feature = "profiling")]
pub(crate) fn shutdown() {
capture::shutdown();
}
#[cfg(not(feature = "profiling"))]
pub(crate) fn shutdown() {}
/// One serialization lock for every test in this module tree.
///
/// The recorder counters and the capture state machine are the *same* process
/// statics: `capture::start` calls `recorder::reset`, and `capture::stop`
/// drains every slot. Two suites with their own locks therefore do not
/// serialize against each other, and the feature-on stage runs tests as
/// threads in one process, so a capture round-trip can zero the counters a
/// recorder test is mid-way through asserting on. One lock for both.
#[cfg(all(test, feature = "profiling"))]
pub(crate) static TEST_SERIAL: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// Take the shared test lock, recovering from a poisoned mutex so one failing
/// test does not cascade into every other one.
#[cfg(all(test, feature = "profiling"))]
pub(crate) fn test_serial() -> std::sync::MutexGuard<'static, ()> {
TEST_SERIAL.lock().unwrap_or_else(|e| e.into_inner())
}
src/instr/recorder.rs
+487
View File
@@ -0,0 +1,487 @@
//! Fixed-footprint recorder: exact count / max / total per (domain, step).
//!
//! All state is process statics, not `Node` state, because the `fipsctl`
//! handler that arms and disarms a capture runs in the control accept task and
//! has no `&Node` — that is the whole point of serving it off-loop, so it
//! cannot queue behind the behavior it is measuring.
//!
//! The writer thread is the only reader. It takes each interval's figures with
//! `swap(0)`, so there are no "previous value" arrays to carry and the counters
//! are per-interval by construction.
use std::sync::LazyLock;
use std::sync::atomic::{AtomicU64, Ordering::Relaxed};
use std::time::{Duration, Instant};
/// Measurement domain. Structural only: one variant today.
///
/// A data-path domain is deliberately **not** declared until something records
/// into it. What generalizes here is the enum, the counter table and the
/// writer; the per-tick gate hoist does not, so a data-path domain will need
/// its own gate strategy.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[repr(usize)]
pub(crate) enum Domain {
Tick = 0,
}
pub(crate) const N_DOMAINS: usize = 1;
pub(crate) const DOMAINS: [Domain; N_DOMAINS] = [Domain::Tick];
impl Domain {
pub(crate) const fn name(self) -> &'static str {
match self {
Domain::Tick => "tick",
}
}
}
/// One measured step of the rx-loop tick arm, in call order, plus the
/// whole-body span.
///
/// `as usize` indexes the counter arrays, so the discriminants are dense and
/// `WholeTick` is last (it defines `N_STEPS`). Variants are declared
/// unconditionally — see [`Step::emitted`] for how the two platform- and
/// profile-conditional steps are kept out of the emitted table.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[repr(usize)]
pub(crate) enum Step {
CheckTimeouts = 0,
ReloadPeerAcl,
ReloadHostMap,
PollPendingConnects,
PollNostrRendezvous,
PollLanRendezvous,
DrivePeerTimers,
ResendPendingRekeys,
/// `next`-only: the FMP rekey msg3 resend driver has no master-line
/// counterpart, so this variant exists on this line alone.
ResendPendingFmpRekeyMsg3,
ResendPendingSessionHandshakes,
ResendPendingSessionMsg3,
PurgeIdleSessions,
ProcessPendingRetries,
CheckTreeState,
CheckBloomState,
ComputeMeshSize,
RecordStatsHistory,
CheckMmpReports,
CheckSessionMmpReports,
CheckLinkHeartbeats,
CheckRekey,
CheckSessionRekey,
CheckPendingLookups,
PollTransportDiscovery,
SampleTransportCongestion,
ActivateConnectedUdpSessions,
DebugAssertPeerMapsCoherent,
/// The whole tick-arm body, from before `check_timeouts` to after the last
/// step. Composes safely with the per-step spans because the macro
/// evaluates its measured expression exactly once.
WholeTick,
}
pub(crate) const N_STEPS: usize = Step::WholeTick as usize + 1;
/// Every step, in emission order. Index `i` of this table is `STEPS[i] as
/// usize`; `steps_table_is_dense` asserts it.
pub(crate) const STEPS: [Step; N_STEPS] = [
Step::CheckTimeouts,
Step::ReloadPeerAcl,
Step::ReloadHostMap,
Step::PollPendingConnects,
Step::PollNostrRendezvous,
Step::PollLanRendezvous,
Step::DrivePeerTimers,
Step::ResendPendingRekeys,
Step::ResendPendingFmpRekeyMsg3,
Step::ResendPendingSessionHandshakes,
Step::ResendPendingSessionMsg3,
Step::PurgeIdleSessions,
Step::ProcessPendingRetries,
Step::CheckTreeState,
Step::CheckBloomState,
Step::ComputeMeshSize,
Step::RecordStatsHistory,
Step::CheckMmpReports,
Step::CheckSessionMmpReports,
Step::CheckLinkHeartbeats,
Step::CheckRekey,
Step::CheckSessionRekey,
Step::CheckPendingLookups,
Step::PollTransportDiscovery,
Step::SampleTransportCongestion,
Step::ActivateConnectedUdpSessions,
Step::DebugAssertPeerMapsCoherent,
Step::WholeTick,
];
impl Step {
pub(crate) const fn name(self) -> &'static str {
match self {
Step::CheckTimeouts => "check_timeouts",
Step::ReloadPeerAcl => "reload_peer_acl",
Step::ReloadHostMap => "reload_host_map",
Step::PollPendingConnects => "poll_pending_connects",
Step::PollNostrRendezvous => "poll_nostr_rendezvous",
Step::PollLanRendezvous => "poll_lan_rendezvous",
Step::DrivePeerTimers => "drive_peer_timers",
Step::ResendPendingRekeys => "resend_pending_rekeys",
Step::ResendPendingFmpRekeyMsg3 => "resend_pending_fmp_rekey_msg3",
Step::ResendPendingSessionHandshakes => "resend_pending_session_handshakes",
Step::ResendPendingSessionMsg3 => "resend_pending_session_msg3",
Step::PurgeIdleSessions => "purge_idle_sessions",
Step::ProcessPendingRetries => "process_pending_retries",
Step::CheckTreeState => "check_tree_state",
Step::CheckBloomState => "check_bloom_state",
Step::ComputeMeshSize => "compute_mesh_size",
Step::RecordStatsHistory => "record_stats_history",
Step::CheckMmpReports => "check_mmp_reports",
Step::CheckSessionMmpReports => "check_session_mmp_reports",
Step::CheckLinkHeartbeats => "check_link_heartbeats",
Step::CheckRekey => "check_rekey",
Step::CheckSessionRekey => "check_session_rekey",
Step::CheckPendingLookups => "check_pending_lookups",
Step::PollTransportDiscovery => "poll_transport_discovery",
Step::SampleTransportCongestion => "sample_transport_congestion",
Step::ActivateConnectedUdpSessions => "activate_connected_udp_sessions",
Step::DebugAssertPeerMapsCoherent => "debug_assert_peer_maps_coherent",
Step::WholeTick => "whole_tick",
}
}
/// Whether this step gets a row in this build.
///
/// Two steps are conditionally compiled at their call sites. Emitting a row
/// for them in a build where the call site does not exist would publish a
/// count that is structurally zero forever, which reads as "this step never
/// runs" rather than "this step is not in this build". The predicates below
/// are the same `cfg` expressions that gate the call sites in
/// `node::dataplane::rx_loop`; keep them in step.
pub(crate) const fn emitted(self) -> bool {
match self {
Step::ActivateConnectedUdpSessions => {
cfg!(any(target_os = "linux", target_os = "macos"))
}
Step::DebugAssertPeerMapsCoherent => cfg!(debug_assertions),
_ => true,
}
}
}
/// A scalar sampled once per tick, as opposed to a duration.
///
/// Gauges carry their own row kind and their own unit in the output so a gauge
/// value can never be read as a duration.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[repr(usize)]
pub(crate) enum Gauge {
Ticks = 0,
Peers,
TickGap,
ArmStarvation,
}
pub(crate) const N_GAUGES: usize = Gauge::ArmStarvation as usize + 1;
pub(crate) const GAUGES: [Gauge; N_GAUGES] = [
Gauge::Ticks,
Gauge::Peers,
Gauge::TickGap,
Gauge::ArmStarvation,
];
impl Gauge {
pub(crate) const fn name(self) -> &'static str {
match self {
Gauge::Ticks => "ticks",
Gauge::Peers => "peers",
Gauge::TickGap => "tick_entry_gap",
Gauge::ArmStarvation => "arm_starvation",
}
}
/// Unit of the `max` and `total` columns for this gauge.
pub(crate) const fn unit(self) -> &'static str {
match self {
Gauge::Ticks => "ticks",
Gauge::Peers => "peers",
Gauge::TickGap | Gauge::ArmStarvation => "us",
}
}
/// Whether the gauge's stored values are nanosecond durations that the
/// writer converts to microseconds.
pub(crate) const fn is_duration(self) -> bool {
matches!(self, Gauge::TickGap | Gauge::ArmStarvation)
}
}
const N_SLOTS: usize = N_DOMAINS * N_STEPS;
static COUNT: [AtomicU64; N_SLOTS] = [const { AtomicU64::new(0) }; N_SLOTS];
static MAX_NS: [AtomicU64; N_SLOTS] = [const { AtomicU64::new(0) }; N_SLOTS];
static TOTAL_NS: [AtomicU64; N_SLOTS] = [const { AtomicU64::new(0) }; N_SLOTS];
static G_COUNT: [AtomicU64; N_GAUGES] = [const { AtomicU64::new(0) }; N_GAUGES];
static G_MAX: [AtomicU64; N_GAUGES] = [const { AtomicU64::new(0) }; N_GAUGES];
static G_TOTAL: [AtomicU64; N_GAUGES] = [const { AtomicU64::new(0) }; N_GAUGES];
/// Monotonic baseline so tick-arm entry times fit in an atomic. Offset by one
/// on store so that zero can mean "no previous entry".
static BASE: LazyLock<Instant> = LazyLock::new(Instant::now);
static PREV_ENTRY_NS: AtomicU64 = AtomicU64::new(0);
#[inline]
const fn slot(domain: Domain, step: Step) -> usize {
(domain as usize * N_STEPS) + step as usize
}
/// Read the clock for a step span.
#[inline]
pub(crate) fn now() -> Instant {
Instant::now()
}
/// Record one observation of `step`.
#[inline]
pub(crate) fn record(domain: Domain, step: Step, elapsed: Duration) {
let ns = elapsed.as_nanos() as u64;
let idx = slot(domain, step);
COUNT[idx].fetch_add(1, Relaxed);
TOTAL_NS[idx].fetch_add(ns, Relaxed);
MAX_NS[idx].fetch_max(ns, Relaxed);
}
#[inline]
fn record_gauge(gauge: Gauge, value: u64) {
let idx = gauge as usize;
G_COUNT[idx].fetch_add(1, Relaxed);
G_TOTAL[idx].fetch_add(value, Relaxed);
G_MAX[idx].fetch_max(value, Relaxed);
}
/// Sample the inter-entry gap and the measured arm-starvation delay.
pub(crate) fn tick_entry(on: bool, deadline: Instant, now: Instant) {
if !on {
return;
}
// Offset by one so that a stored zero unambiguously means "no previous
// entry", even for an entry that lands on the baseline instant.
let stamp = BASE.elapsed().as_nanos() as u64 + 1;
let late = now.saturating_duration_since(deadline).as_nanos() as u64;
tick_entry_at(stamp, late);
}
/// The clock-free half of [`tick_entry`]: both times are inputs, so the
/// arithmetic can be driven with synthetic stamps in a test.
///
/// `late_ns` is how far past its scheduled deadline this entry was, measured
/// directly rather than derived. Two earlier designs derived it from the
/// inter-entry gap and were both wrong: subtracting the previous body
/// understated it by exactly the body, and subtracting `max(period, body)`
/// reported the *first difference* of the delay, so a sustained stall — the
/// overload regime this measurement exists to characterize — read as zero
/// forever. `tokio::time::interval::tick` hands back the deadline it was
/// scheduled for, so the delay is a subtraction with no model behind it.
fn tick_entry_at(stamp: u64, late_ns: u64) {
let prev = PREV_ENTRY_NS.swap(stamp, Relaxed);
record_gauge(Gauge::Ticks, 1);
record_gauge(Gauge::ArmStarvation, late_ns);
if prev == 0 {
// First entry of this capture: there is no previous entry to measure a
// gap against, and the idle interval before arming is not a gap. The
// lateness above does not depend on a previous entry, so it still counts.
return;
}
record_gauge(Gauge::TickGap, stamp.saturating_sub(prev));
}
/// Sample the gauges that come from node state.
pub(crate) fn tick_gauges(on: bool, peers: u64) {
if !on {
return;
}
record_gauge(Gauge::Peers, peers);
}
/// Take (and clear) this interval's figures for one step: count, max ns, total
/// ns. Called only by the writer thread.
pub(crate) fn take_step(domain: Domain, step: Step) -> (u64, u64, u64) {
let idx = slot(domain, step);
(
COUNT[idx].swap(0, Relaxed),
MAX_NS[idx].swap(0, Relaxed),
TOTAL_NS[idx].swap(0, Relaxed),
)
}
/// Take (and clear) this interval's figures for one gauge.
pub(crate) fn take_gauge(gauge: Gauge) -> (u64, u64, u64) {
let idx = gauge as usize;
(
G_COUNT[idx].swap(0, Relaxed),
G_MAX[idx].swap(0, Relaxed),
G_TOTAL[idx].swap(0, Relaxed),
)
}
/// Zero every counter so a capture starts from a clean slate.
pub(crate) fn reset() {
for i in 0..N_SLOTS {
COUNT[i].store(0, Relaxed);
MAX_NS[i].store(0, Relaxed);
TOTAL_NS[i].store(0, Relaxed);
}
for i in 0..N_GAUGES {
G_COUNT[i].store(0, Relaxed);
G_MAX[i].store(0, Relaxed);
G_TOTAL[i].store(0, Relaxed);
}
PREV_ENTRY_NS.store(0, Relaxed);
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::MutexGuard;
/// Shared with the capture tests: they mutate the same statics. See
/// `crate::instr::test_serial`.
fn serial() -> MutexGuard<'static, ()> {
crate::instr::test_serial()
}
#[test]
fn steps_table_is_dense() {
for (i, step) in STEPS.iter().enumerate() {
assert_eq!(*step as usize, i, "step {} is out of order", step.name());
}
assert_eq!(STEPS.len(), N_STEPS);
}
#[test]
fn gauges_table_is_dense() {
for (i, gauge) in GAUGES.iter().enumerate() {
assert_eq!(*gauge as usize, i, "gauge {} is out of order", gauge.name());
}
assert_eq!(GAUGES.len(), N_GAUGES);
}
#[test]
fn step_names_are_unique() {
let mut names: Vec<&str> = STEPS.iter().map(|s| s.name()).collect();
names.sort_unstable();
let before = names.len();
names.dedup();
assert_eq!(before, names.len(), "duplicate step name");
}
#[test]
fn emitted_row_count_matches_build() {
let emitted = STEPS.iter().filter(|s| s.emitted()).count();
// 25 unconditional subsystem steps on this line (24 shared with the
// master line, plus `resend_pending_fmp_rekey_msg3`, which exists only
// here) + the whole-tick span, plus the two conditionally-compiled
// steps where this build has them. The count is pinned deliberately: it
// is what caught the extra step when the master-line instrumentation
// was merged up, rather than letting the tables silently disagree.
let mut expected = 26;
if cfg!(any(target_os = "linux", target_os = "macos")) {
expected += 1;
}
if cfg!(debug_assertions) {
expected += 1;
}
assert_eq!(emitted, expected);
}
#[test]
fn record_accumulates_count_max_and_total() {
let _guard = serial();
reset();
record(Domain::Tick, Step::CheckRekey, Duration::from_nanos(10));
record(Domain::Tick, Step::CheckRekey, Duration::from_nanos(30));
let (count, max, total) = take_step(Domain::Tick, Step::CheckRekey);
assert_eq!((count, max, total), (2, 30, 40));
// Taking clears the slot.
assert_eq!(take_step(Domain::Tick, Step::CheckRekey), (0, 0, 0));
}
#[test]
fn starvation_is_the_measured_lateness_of_the_entry() {
let _guard = serial();
reset();
// The interval hands back the deadline it was scheduled for, so the
// delay is `now - deadline` and nothing is derived from the period, the
// previous entry, or the previous body.
PREV_ENTRY_NS.store(1_000_000_000, Relaxed);
tick_entry_at(1_100_000_000, 50_000_000);
assert_eq!(take_gauge(Gauge::TickGap), (1, 100_000_000, 100_000_000));
assert_eq!(
take_gauge(Gauge::ArmStarvation),
(1, 50_000_000, 50_000_000)
);
assert_eq!(take_gauge(Gauge::Ticks).0, 1);
reset();
}
#[test]
fn sustained_lateness_is_reported_on_every_tick() {
let _guard = serial();
reset();
// The regime the two earlier designs both hid. Three consecutive entries
// each 50 ms past their deadline, one period apart, i.e. the arm waiting
// a constant amount behind the other select arms every round. The gaps
// are all exactly one period, so any formula derived from the
// inter-entry gap reports zero here; measured lateness reports 50 ms
// three times, which is the truth.
PREV_ENTRY_NS.store(1_000_000_000, Relaxed);
tick_entry_at(1_050_000_000, 50_000_000);
tick_entry_at(1_100_000_000, 50_000_000);
tick_entry_at(1_150_000_000, 50_000_000);
let (count, max, total) = take_gauge(Gauge::ArmStarvation);
assert_eq!(count, 3);
assert_eq!(max, 50_000_000);
assert_eq!(total, 150_000_000);
// ...and the gap alone carries no signal about it: every gap is one
// period, exactly as it would be on a perfectly healthy node.
assert_eq!(take_gauge(Gauge::TickGap), (3, 50_000_000, 150_000_000));
reset();
}
#[test]
fn first_entry_of_a_capture_records_no_gap_but_still_records_lateness() {
let _guard = serial();
reset();
tick_entry_at(500, 7_000_000);
assert_eq!(take_gauge(Gauge::Ticks).0, 1);
assert_eq!(take_gauge(Gauge::TickGap), (0, 0, 0));
// Lateness does not depend on a previous entry, so the first tick of a
// capture still contributes one.
assert_eq!(take_gauge(Gauge::ArmStarvation), (1, 7_000_000, 7_000_000));
reset();
}
#[test]
fn an_on_schedule_entry_reports_no_starvation() {
let _guard = serial();
reset();
PREV_ENTRY_NS.store(1_000_000_000, Relaxed);
tick_entry_at(1_050_000_000, 0);
assert_eq!(take_gauge(Gauge::ArmStarvation), (1, 0, 0));
assert_eq!(take_gauge(Gauge::TickGap), (1, 50_000_000, 50_000_000));
reset();
}
#[test]
fn gate_off_records_nothing() {
let _guard = serial();
reset();
let t = Instant::now();
tick_entry(false, t, t);
tick_gauges(false, 42);
assert_eq!(take_gauge(Gauge::Ticks), (0, 0, 0));
assert_eq!(take_gauge(Gauge::Peers), (0, 0, 0));
}
}
src/instr/writer.rs
+218
View File
@@ -0,0 +1,218 @@
//! The capture writer: a dedicated, named OS thread with an explicit
//! lifecycle.
//!
//! There is no worker-thread lifecycle in this codebase to copy — the crypto
//! worker pools are never torn down and drop their join handles at spawn — so
//! this one is designed here.
//!
//! Two properties matter:
//!
//! - **It is an OS thread, not a spawned task.** The runtime is
//! `current_thread`, so file I/O on a task would run on the rx loop's own
//! thread and stall every `select!` arm, including the one being measured.
//! - **It waits on a channel with a timeout, not on a sleep.** `recv_timeout`
//! returns immediately when the toggle sends stop, so `off` performs a final
//! drain and joins promptly instead of parking the caller for up to a full
//! flush interval.
//!
//! The thread is created lazily when a capture starts, so a node that never
//! arms one never has the thread.
use std::fs::File;
use std::io::Write;
use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender};
use std::thread::{self, JoinHandle};
use std::time::{SystemTime, UNIX_EPOCH};
use super::capture::{self, BYTE_CAP, INTERVAL};
use super::recorder::{self, DOMAINS, GAUGES, STEPS};
/// Owner-side handle to the writer thread.
pub(crate) struct Handle {
stop_tx: Sender<()>,
join: JoinHandle<()>,
}
impl Handle {
/// Wake the writer, let it drain once more, and join it.
pub(crate) fn stop_and_join(self) {
// A send error means the thread already exited (cap stop); joining is
// still correct and returns at once.
let _ = self.stop_tx.send(());
let _ = self.join.join();
}
}
/// Start the writer thread on an already-open sink.
pub(crate) fn spawn(file: File) -> std::io::Result<Handle> {
let (stop_tx, stop_rx) = mpsc::channel();
let join = thread::Builder::new()
.name("fips-profile".to_string())
.spawn(move || run(file, stop_rx))?;
Ok(Handle { stop_tx, join })
}
/// What one flush cycle decided. Separated from [`run`] so the terminal paths
/// can be driven in a test with a failing sink: the loop below owns the waiting
/// and the state transition, this owns the decision.
#[derive(Debug, PartialEq, Eq)]
enum Cycle {
Continue,
CapReached,
WriteFailed,
}
/// Drain one interval into the sink and decide whether the capture goes on.
fn flush_cycle<W: Write>(file: &mut W) -> Cycle {
if flush(file).is_err() {
// The sink is gone or full; stop rather than spinning on a broken file
// for the rest of the run.
let _ = note(file, "capture stopped: write error");
return Cycle::WriteFailed;
}
if capture::bytes_written() >= BYTE_CAP {
let _ = note(
file,
&format!("capture stopped: byte cap {BYTE_CAP} reached"),
);
let _ = file.flush();
return Cycle::CapReached;
}
Cycle::Continue
}
fn run<W: Write>(mut file: W, stop_rx: Receiver<()>) {
loop {
match stop_rx.recv_timeout(INTERVAL) {
// Stop requested, or the owner went away: final drain, then exit.
Ok(()) | Err(RecvTimeoutError::Disconnected) => {
let _ = flush(&mut file);
let _ = file.flush();
return;
}
Err(RecvTimeoutError::Timeout) => match flush_cycle(&mut file) {
Cycle::Continue => {}
Cycle::WriteFailed => {
// The trailer went to the same failing file, so it is not a
// signal that survives. `stop` and a subsequent `on` both
// clear the state without surfacing it, so an operator would
// otherwise never learn the window was truncated.
tracing::warn!(
target: "fips::instr",
"profile capture stopped: write error on the sink"
);
capture::mark_stopped(capture::STOPPED_BY_ERROR);
return;
}
Cycle::CapReached => {
capture::mark_stopped(capture::STOPPED_BY_CAP);
return;
}
},
}
}
}
/// Append a `#`-prefixed trailer line.
fn note<W: Write>(file: &mut W, text: &str) -> std::io::Result<()> {
let line = format!("# {text}\n");
file.write_all(line.as_bytes())?;
capture::add_bytes(line.len() as u64);
Ok(())
}
/// Emit one interval: every step of every domain, then the gauges.
///
/// Every emitted step gets a row every interval, including zero-count rows, so
/// "this step did not run" is visible rather than absent. The two steps whose
/// call sites are conditionally compiled are excluded in builds that do not
/// have them, so no row is structurally zero forever.
fn flush<W: Write>(file: &mut W) -> std::io::Result<()> {
let ts = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let mut out = String::with_capacity(4096);
for domain in DOMAINS {
for step in STEPS {
if !step.emitted() {
continue;
}
let (count, max_ns, total_ns) = recorder::take_step(domain, step);
out.push_str(&format!(
"{ts}\tstep\t{domain}\t{name}\t{count}\t{max}\t{total}\tus\n",
domain = domain.name(),
name = step.name(),
max = max_ns / 1_000,
total = total_ns / 1_000,
));
}
}
// Gauges carry the tick domain today; the row kind and the unit column keep
// them distinguishable from the duration rows above.
for gauge in GAUGES {
let (count, mut max, mut total) = recorder::take_gauge(gauge);
if gauge.is_duration() {
max /= 1_000;
total /= 1_000;
}
out.push_str(&format!(
"{ts}\tgauge\t{domain}\t{name}\t{count}\t{max}\t{total}\t{unit}\n",
domain = recorder::Domain::Tick.name(),
name = gauge.name(),
unit = gauge.unit(),
));
}
file.write_all(out.as_bytes())?;
capture::add_bytes(out.len() as u64);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// A sink that fails every write, so the writer's error path is driven by a
/// real `Err` rather than asserted about.
struct AlwaysFails;
impl Write for AlwaysFails {
fn write(&mut self, _buf: &[u8]) -> std::io::Result<usize> {
Err(std::io::Error::new(
std::io::ErrorKind::StorageFull,
"no space left on device",
))
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
/// A failing sink ends the capture as a write error, which `run` turns into
/// `STOPPED_BY_ERROR` — not into a byte-cap stop. The distinction is what
/// tells an operator that a window is truncated rather than complete.
///
/// **Coverage note.** This drives the decision, not the filesystem
/// condition. The mesh rehearsal cannot produce one: removing the file
/// leaves the writer's descriptor valid and writes keep succeeding into the
/// unlinked inode, and mounting a tiny filesystem inside the test container
/// is refused. So "a real ENOSPC reaches this branch" stays unexercised;
/// what is covered is that an `Err` from the sink produces the error
/// outcome and not the cap outcome.
#[test]
fn a_failing_sink_ends_the_cycle_as_an_error_not_a_cap() {
let _guard = crate::instr::test_serial();
assert_eq!(flush_cycle(&mut AlwaysFails), Cycle::WriteFailed);
}
/// The healthy path must not be reported as either terminal state, or the
/// test above would pass for a writer that always stops.
#[test]
fn a_working_sink_continues() {
let _guard = crate::instr::test_serial();
let mut sink: Vec<u8> = Vec::new();
assert_eq!(flush_cycle(&mut sink), Cycle::Continue);
}
}
src/lib.rs
+56 -20
View File
@@ -3,22 +3,34 @@
//! A distributed, decentralized network routing protocol for mesh nodes
//! connecting over arbitrary transports.
pub mod bloom;
// Name the `alloc` crate directly so the sans-IO protocol cores can spell their
// heap-type imports in `no_std`-forward form (`alloc::sync::Arc`,
// `alloc::collections::BTreeMap`). The crate remains `std`; this only reduces the
// distance to extracting the pure cores into a `no_std` crate later.
extern crate alloc;
pub mod cache;
pub mod config;
pub mod control;
pub mod discovery;
#[cfg(target_os = "linux")]
pub mod gateway;
pub mod identity;
pub mod mmp;
// Declared before `node` (and named to sort there) because it carries
// `#[macro_use]`: the tick instrumentation macro must be in scope for the
// modules that follow.
#[macro_use]
pub(crate) mod instr;
pub mod mdns;
pub mod node;
pub mod noise;
pub mod nostr;
pub mod peer;
pub mod perf_profile;
pub mod protocol;
pub(crate) mod proto;
#[cfg(test)]
pub(crate) mod testutil;
mod time;
pub mod transport;
pub mod tree;
pub mod upper;
pub mod utils;
pub mod version;
@@ -33,14 +45,16 @@ pub use identity::{
pub use config::{Config, ConfigError, IdentityConfig, NymConfig, TorConfig, UdpConfig};
pub use upper::config::{DnsConfig, TunConfig};
// Re-export discovery types
pub use discovery::{BootstrapHandoffResult, EstablishedTraversal};
// Re-export nostr rendezvous handoff types
pub use nostr::{BootstrapHandoffResult, EstablishedTraversal, is_punch_packet};
// Re-export tree types
pub use tree::{CoordEntry, ParentDeclaration, TreeCoordinate, TreeError, TreeState};
// Re-export tree types (relocated from tree:: to proto::stp)
pub use proto::stp::{
CoordEntry, CoordError, ParentDeclaration, TreeCoordinate, TreeError, TreeState,
};
// Re-export bloom filter types
pub use bloom::{BloomError, BloomFilter, BloomState};
// Re-export bloom filter types (relocated from bloom:: to proto::bloom)
pub use proto::bloom::{BloomError, BloomFilter, BloomState};
// Re-export transport types
pub use transport::udp::UdpTransport;
@@ -50,21 +64,43 @@ pub use transport::{
TransportState, TransportType, packet_channel,
};
// Re-export protocol types
pub use protocol::{
CoordsRequired, FilterAnnounce, HandshakeMessageType, LinkMessageType, LookupRequest,
LookupResponse, PathBroken, ProtocolError, SessionAck, SessionDatagram, SessionFlags,
SessionMessageType, SessionSetup, TreeAnnounce,
// Re-export link-layer types (relocated from protocol:: to proto::link)
pub use proto::link::{LinkMessageType, SessionDatagram};
// Re-export the shared protocol error (relocated from protocol:: to proto::Error)
pub use proto::Error;
// Re-export FSP session wire types (relocated from protocol:: to proto::fsp)
pub use proto::fsp::{SessionAck, SessionFlags, SessionMessageType, SessionSetup};
// Re-export STP wire types (relocated from protocol:: to proto::stp)
pub use proto::stp::TreeAnnounce;
// Re-export bloom wire types (relocated from protocol:: to proto::bloom)
pub use proto::bloom::FilterAnnounce;
// Re-export discovery wire types (relocated from protocol:: to proto::lookup)
pub use proto::lookup::{LookupRequest, LookupResponse};
// Re-export routing wire types (relocated from protocol:: to proto::routing)
pub use proto::routing::{
COORDS_REQUIRED_SIZE, CoordsRequired, MTU_EXCEEDED_SIZE, MtuExceeded, PathBroken,
};
// Re-export FMP link-framing wire type (relocated from protocol:: to proto::fmp)
pub use proto::fmp::HandshakeMessageType;
// Re-export FMP negotiation wire types (relocated from protocol:: to proto::fmp)
pub use proto::fmp::{NegotiationPayload, NodeProfile, TlvEntry};
// Re-export cache types
pub use cache::{CacheEntry, CacheError, CacheStats, CoordCache};
// Re-export FMP tie-break helper and promotion result (relocated from peer:: to proto::fmp)
pub use proto::fmp::{PromotionResult, cross_connection_winner};
// Re-export peer types
pub use peer::{
ActivePeer, ConnectivityState, HandshakeState, PeerConnection, PeerError, PeerSlot,
PromotionResult, cross_connection_winner,
};
pub use peer::{ActivePeer, ConnectivityState, PeerError};
// Re-export node types
pub use node::{Node, NodeError, NodeState, UpdatePeersOutcome};
src/discovery/lan/mod.rs → src/mdns/mod.rs
+28 -19
View File
@@ -54,8 +54,12 @@ pub const TXT_KEY_SCOPE: &str = "scope";
/// `PROTOCOL_VERSION`).
pub const TXT_KEY_VERSION: &str = "v";
/// FIPS protocol version advertised in the mDNS TXT `v` key. Kept in sync
/// with the Nostr rendezvous `PROTOCOL_VERSION` (same value, `"1"`).
const TXT_PROTOCOL_VERSION: &str = "1";
#[derive(Debug, Error)]
pub enum LanDiscoveryError {
pub enum LanRendezvousError {
#[error("mDNS daemon init failed: {0}")]
Daemon(String),
#[error("mDNS register failed: {0}")]
@@ -79,7 +83,7 @@ pub struct LanDiscoveredPeer {
pub observed_at: Instant,
}
/// Browser-side events surfaced by `LanDiscovery::drain_events`.
/// Browser-side events surfaced by `LanRendezvous::drain_events`.
#[derive(Debug, Clone)]
pub enum LanEvent {
Discovered(LanDiscoveredPeer),
@@ -87,17 +91,17 @@ pub enum LanEvent {
/// Runtime configuration for the mDNS responder + browser.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct LanDiscoveryConfig {
pub struct LanRendezvousConfig {
/// Master switch. Default: `false` — LAN discovery is opt-in. Operators
/// who want sub-second same-LAN pairing enable it via
/// `node.discovery.lan.enabled: true`. Default-off avoids reintroducing
/// `node.rendezvous.lan.enabled: true`. Default-off avoids reintroducing
/// a per-LAN identity broadcast on nodes that have deliberately disabled
/// other discovery channels, and avoids any multicast surprise on upgrade.
#[serde(default = "LanDiscoveryConfig::default_enabled")]
#[serde(default = "LanRendezvousConfig::default_enabled")]
pub enabled: bool,
/// Overridable service type, primarily so integration tests can run
/// multiple isolated services on the same loopback interface.
#[serde(default = "LanDiscoveryConfig::default_service_type")]
#[serde(default = "LanRendezvousConfig::default_service_type")]
pub service_type: String,
/// Optional application/network scope carried in the LAN-only TXT
/// record. Browsers that set a scope ignore adverts for other scopes.
@@ -109,7 +113,7 @@ pub struct LanDiscoveryConfig {
pub scope: Option<String>,
}
impl Default for LanDiscoveryConfig {
impl Default for LanRendezvousConfig {
fn default() -> Self {
Self {
enabled: Self::default_enabled(),
@@ -119,7 +123,7 @@ impl Default for LanDiscoveryConfig {
}
}
impl LanDiscoveryConfig {
impl LanRendezvousConfig {
fn default_enabled() -> bool {
false
}
@@ -129,7 +133,7 @@ impl LanDiscoveryConfig {
}
/// Running mDNS responder + browser bound to the node's UDP advert port.
pub struct LanDiscovery {
pub struct LanRendezvous {
daemon: ServiceDaemon,
own_npub: String,
instance_fullname: String,
@@ -137,7 +141,12 @@ pub struct LanDiscovery {
event_pump: tokio::task::JoinHandle<()>,
}
impl LanDiscovery {
impl LanRendezvous {
/// Whether the mDNS event-pump task has exited (runtime liveness).
pub fn is_finished(&self) -> bool {
self.event_pump.is_finished()
}
/// Start the mDNS responder and browser.
///
/// `advertised_port` is the UDP port the operational UDP transport
@@ -148,16 +157,16 @@ impl LanDiscovery {
identity: &Identity,
scope: Option<String>,
advertised_port: u16,
config: LanDiscoveryConfig,
) -> Result<Arc<Self>, LanDiscoveryError> {
config: LanRendezvousConfig,
) -> Result<Arc<Self>, LanRendezvousError> {
if !config.enabled {
return Err(LanDiscoveryError::Disabled);
return Err(LanRendezvousError::Disabled);
}
if advertised_port == 0 {
return Err(LanDiscoveryError::NoAdvertisedPort);
return Err(LanRendezvousError::NoAdvertisedPort);
}
let daemon = ServiceDaemon::new().map_err(|e| LanDiscoveryError::Daemon(e.to_string()))?;
let daemon = ServiceDaemon::new().map_err(|e| LanRendezvousError::Daemon(e.to_string()))?;
let npub = identity.npub();
// mDNS DNS labels are capped at 63 bytes. 16 bech32 chars of npub
@@ -176,7 +185,7 @@ impl LanDiscovery {
}
props.insert(
TXT_KEY_VERSION.to_string(),
super::nostr::PROTOCOL_VERSION.to_string(),
TXT_PROTOCOL_VERSION.to_string(),
);
// host_ipv4 is set to "127.0.0.1" *and* enable_addr_auto() is
@@ -193,18 +202,18 @@ impl LanDiscovery {
advertised_port,
Some(props),
)
.map_err(|e| LanDiscoveryError::Register(e.to_string()))?
.map_err(|e| LanRendezvousError::Register(e.to_string()))?
.enable_addr_auto();
let instance_fullname = service_info.get_fullname().to_string();
daemon
.register(service_info)
.map_err(|e| LanDiscoveryError::Register(e.to_string()))?;
.map_err(|e| LanRendezvousError::Register(e.to_string()))?;
let browse_rx = daemon
.browse(&config.service_type)
.map_err(|e| LanDiscoveryError::Browse(e.to_string()))?;
.map_err(|e| LanRendezvousError::Browse(e.to_string()))?;
let (events_tx, events_rx) = tokio::sync::mpsc::unbounded_channel();
let own_npub = npub.clone();
src/discovery/lan/tests.rs → src/mdns/tests.rs
+12 -12
View File
@@ -4,7 +4,7 @@ use std::time::Duration;
use crate::Identity;
use mdns_sd::ScopedIp;
use super::{LanDiscovery, LanDiscoveryConfig, LanEvent};
use super::{LanEvent, LanRendezvous, LanRendezvousConfig};
/// Distinct service type per test run so concurrent cargo-test workers
/// on the same machine don't cross-feed each other's adverts via the
@@ -15,8 +15,8 @@ fn isolated_service_type(tag: &str) -> String {
format!("_fipstest-{tag}-{rand:08x}._udp.local.")
}
fn config_for(service_type: String) -> LanDiscoveryConfig {
LanDiscoveryConfig {
fn config_for(service_type: String) -> LanRendezvousConfig {
LanRendezvousConfig {
enabled: true,
service_type,
scope: None,
@@ -47,7 +47,7 @@ fn non_link_local_ipv6_advert_is_preserved() {
}
async fn wait_for_peer(
discovery: &LanDiscovery,
discovery: &LanRendezvous,
expected_npub: &str,
timeout: Duration,
) -> Option<super::LanDiscoveredPeer> {
@@ -64,7 +64,7 @@ async fn wait_for_peer(
None
}
/// Two LanDiscovery instances on isolated service types — `a` browses
/// Two LanRendezvous instances on isolated service types — `a` browses
/// only its own type and never sees `b`, and vice versa. Sanity check
/// that the scope-isolation defense works (we'd lose isolation if mdns-
/// sd ever leaked across service types).
@@ -76,7 +76,7 @@ async fn isolated_service_types_do_not_cross_feed() {
let service_a = isolated_service_type("isolated-a");
let service_b = isolated_service_type("isolated-b");
let lan_a = LanDiscovery::start(
let lan_a = LanRendezvous::start(
&identity_a,
Some("scope-x".to_string()),
61001,
@@ -84,7 +84,7 @@ async fn isolated_service_types_do_not_cross_feed() {
)
.await
.expect("start a");
let lan_b = LanDiscovery::start(
let lan_b = LanRendezvous::start(
&identity_b,
Some("scope-x".to_string()),
61002,
@@ -118,7 +118,7 @@ async fn isolated_service_types_do_not_cross_feed() {
assert!(!saw_a_from_b, "isolated service types must not cross-feed");
}
/// Two LanDiscovery instances on the same service type and the same
/// Two LanRendezvous instances on the same service type and the same
/// scope: each should observe the other's advert within a few seconds.
/// Exercises the responder + browser + TXT plumbing end-to-end.
///
@@ -136,7 +136,7 @@ async fn matched_scope_peers_observe_each_other() {
let service = isolated_service_type("matched");
let lan_a = LanDiscovery::start(
let lan_a = LanRendezvous::start(
&identity_a,
Some("scope-shared".to_string()),
61101,
@@ -144,7 +144,7 @@ async fn matched_scope_peers_observe_each_other() {
)
.await
.expect("start a");
let lan_b = LanDiscovery::start(
let lan_b = LanRendezvous::start(
&identity_b,
Some("scope-shared".to_string()),
61102,
@@ -181,7 +181,7 @@ async fn cross_scope_advert_is_filtered() {
let service = isolated_service_type("cross-scope");
let lan_a = LanDiscovery::start(
let lan_a = LanRendezvous::start(
&identity_a,
Some("scope-a".to_string()),
61201,
@@ -189,7 +189,7 @@ async fn cross_scope_advert_is_filtered() {
)
.await
.expect("start a");
let lan_b = LanDiscovery::start(
let lan_b = LanRendezvous::start(
&identity_b,
Some("scope-b".to_string()),
61202,
src/mmp/algorithms.rs
-526
View File
@@ -1,526 +0,0 @@
//! MMP algorithmic building blocks.
//!
//! Pure computational types with no dependency on peer or node state.
//! Each is independently testable.
use std::collections::VecDeque;
use std::time::Instant;
use crate::mmp::{EWMA_LONG_ALPHA, EWMA_SHORT_ALPHA};
// ============================================================================
// Jitter Estimator (RFC 3550 §6.4.1)
// ============================================================================
/// Interarrival jitter estimator using RFC 3550 algorithm.
///
/// Maintains a smoothed jitter estimate (α = 1/16) from the absolute
/// difference in one-way transit times between consecutive frames.
/// Uses integer arithmetic scaled by 16 to avoid floating-point.
pub struct JitterEstimator {
/// Scaled jitter estimate (×16 for integer arithmetic).
jitter_q4: i64,
}
impl JitterEstimator {
pub fn new() -> Self {
Self { jitter_q4: 0 }
}
/// Update with transit time delta between consecutive frames.
///
/// `transit_delta` = (R_i - R_{i-1}) - (S_i - S_{i-1}) in microseconds.
pub fn update(&mut self, transit_delta: i32) {
// RFC 3550: J = J + (1/16)(|D(i)| - J)
// Scaled: J_q4 = J_q4 + (|D| - J_q4/16)
// = J_q4 + |D| - J_q4 >> 4
let abs_d = (transit_delta as i64).unsigned_abs() as i64;
self.jitter_q4 += abs_d - (self.jitter_q4 >> 4);
}
/// Current jitter estimate in microseconds.
pub fn jitter_us(&self) -> u32 {
(self.jitter_q4 >> 4) as u32
}
}
impl Default for JitterEstimator {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// SRTT Estimator (Jacobson, RFC 6298)
// ============================================================================
/// Smoothed RTT estimator using Jacobson's algorithm.
///
/// SRTT and RTTVAR are maintained in microseconds using integer arithmetic.
pub struct SrttEstimator {
/// Smoothed RTT (microseconds).
srtt_us: i64,
/// RTT variance (microseconds).
rttvar_us: i64,
/// Whether the first sample has been applied.
initialized: bool,
}
impl SrttEstimator {
pub fn new() -> Self {
Self {
srtt_us: 0,
rttvar_us: 0,
initialized: false,
}
}
/// Feed an RTT sample in microseconds.
pub fn update(&mut self, rtt_us: i64) {
if !self.initialized {
// RFC 6298 §2.2: first measurement
self.srtt_us = rtt_us;
self.rttvar_us = rtt_us / 2;
self.initialized = true;
} else {
// RFC 6298 §2.3:
// RTTVAR = (1 - β) * RTTVAR + β * |SRTT - R'| β = 1/4
// SRTT = (1 - α) * SRTT + α * R' α = 1/8
let err = (self.srtt_us - rtt_us).abs();
self.rttvar_us = self.rttvar_us - (self.rttvar_us >> 2) + (err >> 2);
self.srtt_us = self.srtt_us - (self.srtt_us >> 3) + (rtt_us >> 3);
}
}
pub fn srtt_us(&self) -> i64 {
self.srtt_us
}
pub fn rttvar_us(&self) -> i64 {
self.rttvar_us
}
pub fn initialized(&self) -> bool {
self.initialized
}
/// Retransmission timeout = SRTT + max(4 * RTTVAR, 1s), floored at 1s.
pub fn rto_us(&self) -> i64 {
let rto = self.srtt_us + (self.rttvar_us << 2).max(1_000_000);
rto.max(1_000_000)
}
}
impl Default for SrttEstimator {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Dual EWMA Trend Detector
// ============================================================================
/// Dual EWMA for trend detection on a single metric.
///
/// Short-term (α=1/4) tracks recent conditions; long-term (α=1/32)
/// establishes a stable baseline. Divergence indicates trend direction.
pub struct DualEwma {
short: f64,
long: f64,
initialized: bool,
}
impl DualEwma {
pub fn new() -> Self {
Self {
short: 0.0,
long: 0.0,
initialized: false,
}
}
pub fn update(&mut self, sample: f64) {
if !self.initialized {
self.short = sample;
self.long = sample;
self.initialized = true;
} else {
self.short += EWMA_SHORT_ALPHA * (sample - self.short);
self.long += EWMA_LONG_ALPHA * (sample - self.long);
}
}
pub fn short(&self) -> f64 {
self.short
}
pub fn long(&self) -> f64 {
self.long
}
pub fn initialized(&self) -> bool {
self.initialized
}
}
impl Default for DualEwma {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// One-Way Delay Trend Detector
// ============================================================================
/// OWD trend detector using linear regression over a ring buffer.
///
/// Stores (sequence, owd_us) samples and computes the slope via
/// least-squares regression. The slope (µs/s) indicates whether
/// queuing delay is increasing (congestion) or stable.
pub struct OwdTrendDetector {
samples: VecDeque<(u32, i64)>,
capacity: usize,
}
impl OwdTrendDetector {
pub fn new(capacity: usize) -> Self {
Self {
samples: VecDeque::with_capacity(capacity),
capacity,
}
}
/// Clear all samples, keeping the same capacity.
pub fn clear(&mut self) {
self.samples.clear();
}
/// Add an OWD sample.
///
/// `seq` is a monotonic sequence number (e.g., truncated frame counter).
/// `owd_us` is the relative one-way delay in microseconds (R_i - S_i).
pub fn push(&mut self, seq: u32, owd_us: i64) {
if self.samples.len() == self.capacity {
self.samples.pop_front();
}
self.samples.push_back((seq, owd_us));
}
/// Compute the OWD trend as a slope in µs/second.
///
/// Uses simple linear regression: slope = Σ((x-x̄)(y-ȳ)) / Σ((x-x̄)²)
/// where x = sequence number and y = owd_us.
///
/// Returns 0 if fewer than 2 samples.
pub fn trend_us_per_sec(&self) -> i32 {
let n = self.samples.len();
if n < 2 {
return 0;
}
let n_f = n as f64;
let sum_x: f64 = self.samples.iter().map(|(s, _)| *s as f64).sum();
let sum_y: f64 = self.samples.iter().map(|(_, y)| *y as f64).sum();
let mean_x = sum_x / n_f;
let mean_y = sum_y / n_f;
let mut num = 0.0;
let mut den = 0.0;
for &(x, y) in &self.samples {
let dx = x as f64 - mean_x;
let dy = y as f64 - mean_y;
num += dx * dy;
den += dx * dx;
}
if den.abs() < f64::EPSILON {
return 0;
}
// slope is in µs/packet. Convert to µs/second assuming ~1ms inter-packet
// spacing as a rough estimate. The raw slope per packet is more useful
// for trend detection than an absolute rate, but the wire format specifies
// µs/s. We report the raw per-packet slope scaled by 1000.
let slope_per_packet = num / den;
(slope_per_packet * 1000.0) as i32
}
pub fn len(&self) -> usize {
self.samples.len()
}
pub fn is_empty(&self) -> bool {
self.samples.is_empty()
}
}
// ============================================================================
// ETX
// ============================================================================
/// Compute Expected Transmission Count from bidirectional delivery ratios.
///
/// ETX = 1 / (d_f × d_r) where d_f and d_r are forward and reverse
/// delivery probabilities (1.0 = perfect, 0.0 = no delivery).
///
/// Clamped to [1.0, 100.0].
pub fn compute_etx(d_forward: f64, d_reverse: f64) -> f64 {
let product = d_forward * d_reverse;
if product <= 0.0 {
return 100.0;
}
(1.0 / product).clamp(1.0, 100.0)
}
// ============================================================================
// Spin Bit
// ============================================================================
/// Spin bit state for passive RTT estimation.
///
/// Uses asymmetric roles (initiator/responder) per the MMP design:
/// - **Initiator**: flips spin value on each received frame; measures RTT
/// from edge-to-edge intervals.
/// - **Responder**: copies received spin bit into outgoing frames, with a
/// counter guard to filter reordered frames.
pub struct SpinBitState {
is_initiator: bool,
current_value: bool,
/// Highest counter observed with a spin edge (responder guard).
highest_counter_for_spin: u64,
/// Time of last spin edge (initiator only, for RTT measurement).
last_edge_time: Option<Instant>,
}
impl SpinBitState {
pub fn new(is_initiator: bool) -> Self {
Self {
is_initiator,
current_value: false,
highest_counter_for_spin: 0,
last_edge_time: None,
}
}
/// Check if this is the spin bit initiator.
pub fn is_initiator(&self) -> bool {
self.is_initiator
}
/// Get the spin bit value to set on an outgoing frame.
pub fn tx_bit(&self) -> bool {
self.current_value
}
/// Process a received frame's spin bit.
///
/// Returns an RTT sample duration if an edge was detected (initiator only).
pub fn rx_observe(
&mut self,
received_bit: bool,
counter: u64,
now: Instant,
) -> Option<std::time::Duration> {
if self.is_initiator {
// Initiator: when the reflected bit matches what we sent,
// that completes a round trip. Record the edge time, then
// flip for the next cycle.
if received_bit == self.current_value {
let rtt = self.last_edge_time.map(|t| now.duration_since(t));
self.last_edge_time = Some(now);
self.current_value = !self.current_value;
rtt
} else {
None
}
} else {
// Responder: copy received bit, but only if counter is higher
// (reordering guard)
if counter > self.highest_counter_for_spin {
self.highest_counter_for_spin = counter;
self.current_value = received_bit;
}
None
}
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_jitter_zero_input() {
let mut j = JitterEstimator::new();
j.update(0);
assert_eq!(j.jitter_us(), 0);
}
#[test]
fn test_jitter_convergence() {
let mut j = JitterEstimator::new();
// Feed constant transit delta of 1000µs
for _ in 0..200 {
j.update(1000);
}
// Should converge near 1000µs
let jitter = j.jitter_us();
assert!(
jitter > 900 && jitter < 1100,
"jitter={jitter}, expected ~1000"
);
}
#[test]
fn test_srtt_first_sample() {
let mut s = SrttEstimator::new();
s.update(10_000); // 10ms
assert_eq!(s.srtt_us(), 10_000);
assert_eq!(s.rttvar_us(), 5_000);
assert!(s.initialized());
}
#[test]
fn test_srtt_convergence() {
let mut s = SrttEstimator::new();
// Feed constant 50ms RTT
for _ in 0..100 {
s.update(50_000);
}
let srtt = s.srtt_us();
assert!((srtt - 50_000).abs() < 1000, "srtt={srtt}, expected ~50000");
}
#[test]
fn test_dual_ewma_initialization() {
let mut e = DualEwma::new();
assert!(!e.initialized());
e.update(100.0);
assert!(e.initialized());
assert_eq!(e.short(), 100.0);
assert_eq!(e.long(), 100.0);
}
#[test]
fn test_dual_ewma_short_tracks_faster() {
let mut e = DualEwma::new();
// Initialize at 0
e.update(0.0);
// Jump to 100
for _ in 0..20 {
e.update(100.0);
}
// Short should be closer to 100 than long
assert!(
e.short() > e.long(),
"short={} long={}",
e.short(),
e.long()
);
}
#[test]
fn test_owd_trend_flat() {
let mut d = OwdTrendDetector::new(32);
for i in 0..20 {
d.push(i, 5000); // constant OWD
}
let trend = d.trend_us_per_sec();
assert_eq!(trend, 0, "flat OWD should have zero trend");
}
#[test]
fn test_owd_trend_increasing() {
let mut d = OwdTrendDetector::new(32);
for i in 0..20 {
d.push(i, 5000 + (i as i64) * 100); // increasing by 100µs per packet
}
let trend = d.trend_us_per_sec();
assert!(
trend > 0,
"increasing OWD should have positive trend, got {trend}"
);
}
#[test]
fn test_owd_trend_insufficient_samples() {
let mut d = OwdTrendDetector::new(32);
d.push(0, 5000);
assert_eq!(d.trend_us_per_sec(), 0);
}
#[test]
fn test_etx_perfect_link() {
assert!((compute_etx(1.0, 1.0) - 1.0).abs() < f64::EPSILON);
}
#[test]
fn test_etx_lossy_link() {
// 10% forward loss, 5% reverse loss
let etx = compute_etx(0.9, 0.95);
assert!(etx > 1.0 && etx < 2.0, "etx={etx}");
}
#[test]
fn test_etx_zero_delivery() {
assert_eq!(compute_etx(0.0, 1.0), 100.0);
assert_eq!(compute_etx(1.0, 0.0), 100.0);
}
#[test]
fn test_spin_bit_initiator_rtt() {
let mut initiator = SpinBitState::new(true);
let mut responder = SpinBitState::new(false);
let t0 = Instant::now();
let t1 = t0 + std::time::Duration::from_millis(10);
let t2 = t0 + std::time::Duration::from_millis(20);
// Initiator sends with spin=false (initial)
let bit_to_send = initiator.tx_bit();
assert!(!bit_to_send);
// Responder receives, copies bit
responder.rx_observe(bit_to_send, 1, t0);
assert!(!responder.tx_bit());
// Responder sends back, initiator receives
let resp_bit = responder.tx_bit();
let rtt1 = initiator.rx_observe(resp_bit, 2, t1);
// First edge: no previous edge to compare
assert!(rtt1.is_none());
// Now initiator's spin flipped to true
let bit2 = initiator.tx_bit();
assert!(bit2);
// Responder receives new bit
responder.rx_observe(bit2, 3, t1);
assert!(responder.tx_bit());
// Responder sends back, initiator receives
let resp_bit2 = responder.tx_bit();
let rtt2 = initiator.rx_observe(resp_bit2, 4, t2);
// Second edge: should produce an RTT sample
assert!(rtt2.is_some());
}
#[test]
fn test_spin_bit_responder_counter_guard() {
let mut responder = SpinBitState::new(false);
// Receive counter=5 with spin=true
responder.rx_observe(true, 5, Instant::now());
assert!(responder.tx_bit());
// Reordered packet with counter=3 and spin=false should be ignored
responder.rx_observe(false, 3, Instant::now());
assert!(responder.tx_bit()); // unchanged
}
}
src/mmp/metrics.rs
-556
View File
@@ -1,556 +0,0 @@
//! MMP derived metrics.
//!
//! `MmpMetrics` processes incoming ReceiverReports (from our peer) and
//! maintains derived metrics: SRTT, loss rate, goodput, ETX, and dual
//! EWMA trend indicators. Updated by the sender side when it receives
//! a ReceiverReport about its own traffic.
use crate::mmp::algorithms::{DualEwma, SrttEstimator, compute_etx};
use crate::mmp::report::ReceiverReport;
use std::time::Instant;
use tracing::trace;
/// Derived MMP metrics, updated from incoming ReceiverReports.
///
/// This lives on the sender side: when we receive a ReceiverReport from
/// our peer describing what they observed about our traffic, we process
/// it here to compute RTT, loss, goodput, and trend indicators.
pub struct MmpMetrics {
/// Smoothed RTT from timestamp echo.
pub srtt: SrttEstimator,
/// Dual EWMA trend detectors.
pub rtt_trend: DualEwma,
pub loss_trend: DualEwma,
pub goodput_trend: DualEwma,
pub jitter_trend: DualEwma,
pub etx_trend: DualEwma,
/// Forward delivery ratio (what fraction of our frames the peer received).
pub delivery_ratio_forward: f64,
/// Reverse delivery ratio (set when we compute from our own receiver state).
pub delivery_ratio_reverse: f64,
/// ETX computed from bidirectional delivery ratios.
pub etx: f64,
/// Smoothed goodput in bytes/sec (forward direction: what the peer received from us).
pub goodput_bps: f64,
// --- State for delta computation ---
/// Previous ReceiverReport's cumulative counters (for computing interval deltas).
prev_rr_cum_packets: u64,
prev_rr_cum_bytes: u64,
prev_rr_highest_counter: u64,
prev_rr_ecn_ce: u32,
prev_rr_reorder: u32,
/// Time of previous ReceiverReport (for goodput rate computation).
prev_rr_time: Option<Instant>,
/// Whether we have a previous ReceiverReport for delta computation.
has_prev_rr: bool,
// --- State for reverse delivery ratio delta computation ---
/// Previous reverse-side cumulative packets received (our receiver state).
prev_reverse_packets: u64,
/// Previous reverse-side highest counter (our receiver state).
prev_reverse_highest: u64,
/// Whether we have a previous reverse-side snapshot for delta computation.
has_prev_reverse: bool,
}
impl MmpMetrics {
/// Reset state derived from ReceiverReport counters for rekey cutover.
///
/// The new session starts with counter 0, so the prev_rr deltas must
/// be reset to avoid computing bogus loss/goodput from the counter
/// discontinuity. RTT (SRTT) is preserved since it remains valid.
pub fn reset_for_rekey(&mut self) {
self.prev_rr_cum_packets = 0;
self.prev_rr_cum_bytes = 0;
self.prev_rr_highest_counter = 0;
self.prev_rr_ecn_ce = 0;
self.prev_rr_reorder = 0;
self.prev_rr_time = None;
self.has_prev_rr = false;
self.delivery_ratio_forward = 1.0;
self.prev_reverse_packets = 0;
self.prev_reverse_highest = 0;
self.has_prev_reverse = false;
// Keep srtt, etx, trends, goodput_bps — they'll refresh from data
}
pub fn new() -> Self {
Self {
srtt: SrttEstimator::new(),
rtt_trend: DualEwma::new(),
loss_trend: DualEwma::new(),
goodput_trend: DualEwma::new(),
jitter_trend: DualEwma::new(),
etx_trend: DualEwma::new(),
delivery_ratio_forward: 1.0,
delivery_ratio_reverse: 1.0,
etx: 1.0,
goodput_bps: 0.0,
prev_rr_cum_packets: 0,
prev_rr_cum_bytes: 0,
prev_rr_highest_counter: 0,
prev_rr_ecn_ce: 0,
prev_rr_reorder: 0,
prev_rr_time: None,
has_prev_rr: false,
prev_reverse_packets: 0,
prev_reverse_highest: 0,
has_prev_reverse: false,
}
}
/// Process an incoming ReceiverReport (from the peer about our traffic).
///
/// `our_timestamp_ms` is the current session-relative time in ms (for RTT).
/// `now` is the current monotonic time (for goodput rate computation).
///
/// Returns `true` if this report produced the first SRTT measurement
/// (transition from uninitialized to initialized).
pub fn process_receiver_report(
&mut self,
rr: &ReceiverReport,
our_timestamp_ms: u32,
now: Instant,
) -> bool {
let had_srtt = self.srtt.initialized();
if self.has_prev_rr {
let counters_regressed = rr.highest_counter < self.prev_rr_highest_counter
|| rr.cumulative_packets_recv < self.prev_rr_cum_packets
|| rr.cumulative_bytes_recv < self.prev_rr_cum_bytes
|| rr.ecn_ce_count < self.prev_rr_ecn_ce
|| rr.cumulative_reorder_count < self.prev_rr_reorder;
let duplicate_counters = rr.highest_counter == self.prev_rr_highest_counter
&& rr.cumulative_packets_recv == self.prev_rr_cum_packets
&& rr.cumulative_bytes_recv == self.prev_rr_cum_bytes
&& rr.ecn_ce_count == self.prev_rr_ecn_ce
&& rr.cumulative_reorder_count == self.prev_rr_reorder;
// Safe to drop: reports are only built after interval data, so
// a fresh report always advances at least one cumulative counter.
if counters_regressed || duplicate_counters {
trace!(
highest_counter = rr.highest_counter,
prev_highest_counter = self.prev_rr_highest_counter,
cumulative_packets_recv = rr.cumulative_packets_recv,
prev_cumulative_packets_recv = self.prev_rr_cum_packets,
cumulative_bytes_recv = rr.cumulative_bytes_recv,
prev_cumulative_bytes_recv = self.prev_rr_cum_bytes,
"Ignoring stale MMP ReceiverReport"
);
return false;
}
}
// --- RTT from timestamp echo ---
// RTT = now - echoed_timestamp - dwell_time
if rr.timestamp_echo > 0 {
let echo_ms = rr.timestamp_echo;
let dwell_ms = u32::from(rr.dwell_time);
let rtt_sample_ms = echo_ms
.checked_add(dwell_ms)
.and_then(|send_done_ms| our_timestamp_ms.checked_sub(send_done_ms));
match rtt_sample_ms {
Some(rtt_ms) if rtt_ms > 0 => {
let rtt_us = (rtt_ms as i64) * 1000;
trace!(
our_ts = our_timestamp_ms,
echo = echo_ms,
dwell = dwell_ms,
rtt_ms = rtt_ms,
srtt_ms = self.srtt.srtt_us() as f64 / 1000.0,
"RTT sample from timestamp echo"
);
self.srtt.update(rtt_us);
self.rtt_trend.update(rtt_us as f64);
}
_ => {
trace!(
our_ts = our_timestamp_ms,
echo = echo_ms,
dwell = dwell_ms,
"Ignoring invalid MMP RTT sample"
);
}
}
}
// --- Loss rate from cumulative counters ---
// Delta: frames the peer should have received vs. actually received
if self.has_prev_rr {
let counter_span = rr
.highest_counter
.saturating_sub(self.prev_rr_highest_counter);
let packets_delta = rr
.cumulative_packets_recv
.saturating_sub(self.prev_rr_cum_packets);
if counter_span > 0 {
let delivery = (packets_delta as f64) / (counter_span as f64);
self.delivery_ratio_forward = delivery.clamp(0.0, 1.0);
let loss_rate = 1.0 - self.delivery_ratio_forward;
self.loss_trend.update(loss_rate);
self.etx = compute_etx(self.delivery_ratio_forward, self.delivery_ratio_reverse);
self.etx_trend.update(self.etx);
}
}
// --- Goodput from cumulative bytes + time delta ---
if self.has_prev_rr {
let bytes_delta = rr
.cumulative_bytes_recv
.saturating_sub(self.prev_rr_cum_bytes);
self.goodput_trend.update(bytes_delta as f64);
// Compute bytes/sec if we have a time reference
if let Some(prev_time) = self.prev_rr_time {
let elapsed = now.duration_since(prev_time);
let secs = elapsed.as_secs_f64();
if secs > 0.0 {
let bps = bytes_delta as f64 / secs;
// EWMA smoothing: α = 1/4
if self.goodput_bps == 0.0 {
self.goodput_bps = bps;
} else {
self.goodput_bps += (bps - self.goodput_bps) * 0.25;
}
}
}
}
// --- Jitter trend ---
self.jitter_trend.update(rr.jitter as f64);
// --- Save for next delta ---
self.prev_rr_cum_packets = rr.cumulative_packets_recv;
self.prev_rr_cum_bytes = rr.cumulative_bytes_recv;
self.prev_rr_highest_counter = rr.highest_counter;
self.prev_rr_ecn_ce = rr.ecn_ce_count;
self.prev_rr_reorder = rr.cumulative_reorder_count;
self.prev_rr_time = Some(now);
self.has_prev_rr = true;
!had_srtt && self.srtt.initialized()
}
/// Update the reverse delivery ratio from our own receiver state.
///
/// Computes a per-interval delta (same as forward ratio) rather than
/// a lifetime cumulative ratio, so ETX responds to recent conditions.
pub fn update_reverse_delivery(&mut self, our_recv_packets: u64, peer_highest: u64) {
if self.has_prev_reverse {
let counter_span = peer_highest.saturating_sub(self.prev_reverse_highest);
let packets_delta = our_recv_packets.saturating_sub(self.prev_reverse_packets);
if counter_span > 0 {
let delivery = (packets_delta as f64) / (counter_span as f64);
self.delivery_ratio_reverse = delivery.clamp(0.0, 1.0);
self.etx = compute_etx(self.delivery_ratio_forward, self.delivery_ratio_reverse);
self.etx_trend.update(self.etx);
}
}
self.prev_reverse_packets = our_recv_packets;
self.prev_reverse_highest = peer_highest;
self.has_prev_reverse = true;
}
/// Current smoothed RTT in milliseconds, or `None` if not yet measured.
pub fn srtt_ms(&self) -> Option<f64> {
if self.srtt.initialized() {
Some(self.srtt.srtt_us() as f64 / 1000.0)
} else {
None
}
}
/// Current loss rate (0.0 = no loss, 1.0 = total loss).
pub fn loss_rate(&self) -> f64 {
1.0 - self.delivery_ratio_forward
}
/// Smoothed loss rate (long-term EWMA), or `None` if not yet initialized.
pub fn smoothed_loss(&self) -> Option<f64> {
if self.loss_trend.initialized() {
Some(self.loss_trend.long())
} else {
None
}
}
/// Smoothed ETX (long-term EWMA), or `None` if not yet initialized.
pub fn smoothed_etx(&self) -> Option<f64> {
if self.etx_trend.initialized() {
Some(self.etx_trend.long())
} else {
None
}
}
/// Current smoothed goodput in bytes/sec, or 0 if not yet measured.
pub fn goodput_bps(&self) -> f64 {
self.goodput_bps
}
/// Cumulative ECN CE count from the most recent ReceiverReport.
pub fn last_ecn_ce_count(&self) -> u32 {
self.prev_rr_ecn_ce
}
}
impl Default for MmpMetrics {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
use std::time::Duration;
fn make_rr(
highest_counter: u64,
cum_packets: u64,
cum_bytes: u64,
timestamp_echo: u32,
dwell: u16,
jitter: u32,
) -> ReceiverReport {
ReceiverReport {
highest_counter,
cumulative_packets_recv: cum_packets,
cumulative_bytes_recv: cum_bytes,
timestamp_echo,
dwell_time: dwell,
max_burst_loss: 0,
mean_burst_loss: 0,
jitter,
ecn_ce_count: 0,
owd_trend: 0,
burst_loss_count: 0,
cumulative_reorder_count: 0,
interval_packets_recv: 0,
interval_bytes_recv: 0,
}
}
#[test]
fn test_rtt_from_echo() {
let mut m = MmpMetrics::new();
let now = Instant::now();
// Peer echoes timestamp 1000ms, dwell=5ms, our current time=1050ms
let rr = make_rr(10, 10, 5000, 1000, 5, 0);
m.process_receiver_report(&rr, 1050, now);
assert!(m.srtt.initialized());
// RTT = 1050 - 1000 - 5 = 45ms
let srtt_ms = m.srtt_ms().unwrap();
assert!((srtt_ms - 45.0).abs() < 1.0, "srtt={srtt_ms}, expected ~45");
}
#[test]
fn test_ignores_duplicate_receiver_report_after_valid_sample() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
let rr1 = make_rr(10, 10, 5_000, 1_000, 5, 0);
m.process_receiver_report(&rr1, 1_050, t0);
let rr2 = make_rr(20, 18, 14_000, 1_100, 5, 0);
m.process_receiver_report(&rr2, 1_150, t0 + Duration::from_secs(1));
let baseline_srtt_ms = m.srtt_ms().unwrap();
let baseline_loss = m.loss_rate();
let baseline_goodput = m.goodput_bps();
assert!(baseline_loss > 0.0);
assert!(baseline_goodput > 0.0);
// A duplicate of the same counters arriving later would be a 4.895s
// RTT sample if accepted. It is stale and must not move metrics.
m.process_receiver_report(&rr2, 6_000, t0 + Duration::from_secs(5));
assert_eq!(m.srtt_ms().unwrap(), baseline_srtt_ms);
assert_eq!(m.loss_rate(), baseline_loss);
assert_eq!(m.goodput_bps(), baseline_goodput);
}
#[test]
fn test_ignores_out_of_order_receiver_report_after_valid_sample() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let valid_rr = make_rr(20, 20, 10000, 1000, 5, 0);
m.process_receiver_report(&valid_rr, 1050, now);
let baseline_srtt_ms = m.srtt_ms().unwrap();
let old_rr = make_rr(10, 10, 5000, 1000, 0, 0);
m.process_receiver_report(&old_rr, 6000, now + Duration::from_secs(5));
let srtt_ms = m.srtt_ms().unwrap();
assert_eq!(srtt_ms, baseline_srtt_ms);
}
#[test]
fn test_ignores_wrapped_rtt_sample() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let wrapped_rr = make_rr(10, 10, 5000, u32::MAX - 10, 20, 0);
m.process_receiver_report(&wrapped_rr, 15, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_ignores_future_rtt_sample() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let future_rr = make_rr(10, 10, 5_000, 2_000, 5, 0);
m.process_receiver_report(&future_rr, 1_000, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_loss_rate_computation() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
// First report: baseline
let rr1 = make_rr(100, 100, 50000, 0, 0, 0);
m.process_receiver_report(&rr1, 0, t0);
// Second report: 200 counters sent, 190 received (5% loss)
let rr2 = make_rr(300, 290, 145000, 0, 0, 0);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
let loss = m.loss_rate();
assert!((loss - 0.05).abs() < 0.01, "loss={loss}, expected ~0.05");
}
#[test]
fn test_etx_updates() {
let mut m = MmpMetrics::new();
assert_eq!(m.etx, 1.0); // initial: perfect
// Simulate some loss via forward ratio
m.delivery_ratio_forward = 0.9;
// First call establishes the baseline (no ETX update yet)
m.update_reverse_delivery(100, 100);
assert_eq!(m.etx, 1.0); // still perfect — baseline only
// Second call: 190 of 200 frames received (5% loss)
m.update_reverse_delivery(290, 300);
assert!(m.etx > 1.0);
assert!(m.etx < 2.0);
}
#[test]
fn test_no_rtt_without_echo() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let rr = make_rr(10, 10, 5000, 0, 0, 0);
m.process_receiver_report(&rr, 1000, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_jitter_trend() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
let rr1 = make_rr(10, 10, 5000, 0, 0, 100);
m.process_receiver_report(&rr1, 0, t0);
let rr2 = make_rr(20, 20, 10000, 0, 0, 500);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
assert!(m.jitter_trend.initialized());
// Short-term should be closer to 500 than long-term
assert!(m.jitter_trend.short() > m.jitter_trend.long());
}
#[test]
fn test_goodput_bps() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
// First report: baseline (50KB received)
let rr1 = make_rr(100, 100, 50_000, 0, 0, 0);
m.process_receiver_report(&rr1, 0, t0);
assert_eq!(m.goodput_bps(), 0.0); // no rate yet (first report)
// Second report 1s later: 150KB total (100KB delta in 1s = 100KB/s)
let rr2 = make_rr(300, 290, 150_000, 0, 0, 0);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
assert!(
m.goodput_bps() > 90_000.0,
"goodput={}, expected ~100000",
m.goodput_bps()
);
assert!(
m.goodput_bps() < 110_000.0,
"goodput={}, expected ~100000",
m.goodput_bps()
);
}
#[test]
fn test_reverse_delivery_delta() {
let mut m = MmpMetrics::new();
// First call: baseline only, no ratio update
m.update_reverse_delivery(100, 100);
assert_eq!(m.delivery_ratio_reverse, 1.0); // unchanged from default
// Second call: perfect delivery (200 new frames, all received)
m.update_reverse_delivery(300, 300);
assert!((m.delivery_ratio_reverse - 1.0).abs() < 0.001);
// Third call: 50% loss (100 frames sent, 50 received)
m.update_reverse_delivery(350, 400);
assert!(
(m.delivery_ratio_reverse - 0.5).abs() < 0.001,
"reverse={}, expected 0.5",
m.delivery_ratio_reverse
);
}
#[test]
fn test_reverse_delivery_rekey_reset() {
let mut m = MmpMetrics::new();
// Establish baseline and one measurement
m.update_reverse_delivery(100, 100);
m.update_reverse_delivery(300, 300);
assert!((m.delivery_ratio_reverse - 1.0).abs() < 0.001);
// Rekey resets reverse state
m.reset_for_rekey();
// First call after rekey: baseline only
m.update_reverse_delivery(50, 50);
// delivery_ratio_reverse was reset to 1.0 by reset_for_rekey's
// clearing of delivery_ratio_forward; reverse is not explicitly
// reset — but the delta state is, so next call computes fresh.
assert_eq!(m.delivery_ratio_reverse, 1.0);
// Second call after rekey: 80% delivery
m.update_reverse_delivery(90, 100);
assert!(
(m.delivery_ratio_reverse - 0.8).abs() < 0.001,
"reverse={}, expected 0.8",
m.delivery_ratio_reverse
);
}
}
src/mmp/mod.rs
-555
View File
@@ -1,555 +0,0 @@
//! Metrics Measurement Protocol (MMP) — link-layer instantiation.
//!
//! Measures link quality between adjacent peers: RTT, loss, jitter,
//! throughput, one-way delay trend, and ETX. Operates on the per-frame
//! hooks (counter, timestamp, flags) introduced by the FMP wire format
//! revision.
//!
//! Three operating modes trade measurement fidelity for overhead:
//! - **Full**: sender + receiver reports at RTT-adaptive intervals
//! - **Lightweight**: receiver reports only (infer loss from counters)
//! - **Minimal**: spin bit + CE echo only, no reports
use serde::{Deserialize, Serialize};
use std::fmt::{self, Debug};
use std::time::{Duration, Instant};
// Sub-modules
pub mod algorithms;
pub mod metrics;
pub mod receiver;
pub mod report;
pub mod sender;
// Re-exports
pub use algorithms::{
DualEwma, JitterEstimator, OwdTrendDetector, SpinBitState, SrttEstimator, compute_etx,
};
pub use metrics::MmpMetrics;
pub use receiver::ReceiverState;
pub use report::{ReceiverReport, SenderReport};
pub use sender::SenderState;
// Session-layer re-exports
// MmpSessionState and PathMtuState are defined in this file
// ============================================================================
// Constants
// ============================================================================
/// SenderReport body size (after msg_type byte): 3 reserved + 44 payload = 47.
pub const SENDER_REPORT_BODY_SIZE: usize = 47;
/// ReceiverReport body size (after msg_type byte): 3 reserved + 64 payload = 67.
pub const RECEIVER_REPORT_BODY_SIZE: usize = 67;
/// SenderReport total wire size including inner header: 5 + 47 = 52.
pub const SENDER_REPORT_WIRE_SIZE: usize = 52;
/// ReceiverReport total wire size including inner header: 5 + 67 = 72.
pub const RECEIVER_REPORT_WIRE_SIZE: usize = 72;
// --- EWMA parameters (as shift amounts for integer arithmetic) ---
/// Jitter EWMA: α = 1/16 (RFC 3550 §6.4.1).
pub const JITTER_ALPHA_SHIFT: u32 = 4;
/// SRTT: α = 1/8 (Jacobson, RFC 6298).
pub const SRTT_ALPHA_SHIFT: u32 = 3;
/// RTTVAR: β = 1/4 (Jacobson, RFC 6298).
pub const RTTVAR_BETA_SHIFT: u32 = 2;
/// Dual EWMA short-term: α = 1/4.
pub const EWMA_SHORT_ALPHA: f64 = 0.25;
/// Dual EWMA long-term: α = 1/32.
pub const EWMA_LONG_ALPHA: f64 = 1.0 / 32.0;
// --- Timing defaults (milliseconds) ---
/// Default report interval before SRTT is available (cold start).
pub const DEFAULT_COLD_START_INTERVAL_MS: u64 = 200;
/// Minimum report interval (SRTT clamp floor).
///
/// Raised from 100ms to 1000ms: parent re-evaluation runs every 60s,
/// so 60 samples/cycle is more than sufficient for EWMA convergence (~10).
/// The cold-start phase uses `DEFAULT_COLD_START_INTERVAL_MS` (200ms) for
/// fast initial SRTT convergence before transitioning to this floor.
pub const MIN_REPORT_INTERVAL_MS: u64 = 1_000;
/// Maximum report interval (SRTT clamp ceiling).
pub const MAX_REPORT_INTERVAL_MS: u64 = 5_000;
/// Number of SRTT samples before transitioning from cold-start to normal floor.
///
/// During cold-start, report intervals use `DEFAULT_COLD_START_INTERVAL_MS` as
/// the floor to gather SRTT samples quickly. After this many updates, the floor
/// switches to `MIN_REPORT_INTERVAL_MS`.
pub const COLD_START_SAMPLES: u32 = 5;
/// Default OWD ring buffer capacity.
pub const DEFAULT_OWD_WINDOW_SIZE: usize = 32;
/// Default operator log interval in seconds.
pub const DEFAULT_LOG_INTERVAL_SECS: u64 = 30;
// --- Session-layer timing defaults ---
// Session reports are routed end-to-end (bandwidth cost on every transit link),
// so intervals are higher than link-layer.
/// Session-layer minimum report interval.
pub const MIN_SESSION_REPORT_INTERVAL_MS: u64 = 500;
/// Session-layer maximum report interval.
pub const MAX_SESSION_REPORT_INTERVAL_MS: u64 = 10_000;
/// Session-layer cold-start report interval (before SRTT is available).
pub const SESSION_COLD_START_INTERVAL_MS: u64 = 1_000;
// ============================================================================
// Operating Mode
// ============================================================================
/// MMP operating mode.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum MmpMode {
/// Sender + receiver reports at RTT-adaptive intervals. Maximum fidelity.
#[default]
Full,
/// Receiver reports only. Loss inferred from counter gaps.
Lightweight,
/// Spin bit + CE echo only. No reports exchanged.
Minimal,
}
impl fmt::Display for MmpMode {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
MmpMode::Full => write!(f, "full"),
MmpMode::Lightweight => write!(f, "lightweight"),
MmpMode::Minimal => write!(f, "minimal"),
}
}
}
// ============================================================================
// Configuration
// ============================================================================
/// MMP configuration (`node.mmp.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MmpConfig {
/// Operating mode (`node.mmp.mode`).
#[serde(default)]
pub mode: MmpMode,
/// Periodic operator log interval in seconds (`node.mmp.log_interval_secs`).
#[serde(default = "MmpConfig::default_log_interval_secs")]
pub log_interval_secs: u64,
/// OWD trend ring buffer size (`node.mmp.owd_window_size`).
#[serde(default = "MmpConfig::default_owd_window_size")]
pub owd_window_size: usize,
}
impl Default for MmpConfig {
fn default() -> Self {
Self {
mode: MmpMode::default(),
log_interval_secs: DEFAULT_LOG_INTERVAL_SECS,
owd_window_size: DEFAULT_OWD_WINDOW_SIZE,
}
}
}
impl MmpConfig {
fn default_log_interval_secs() -> u64 {
DEFAULT_LOG_INTERVAL_SECS
}
fn default_owd_window_size() -> usize {
DEFAULT_OWD_WINDOW_SIZE
}
}
// ============================================================================
// Per-Peer MMP State
// ============================================================================
/// Combined MMP state for a single peer link.
///
/// Wraps sender, receiver, metrics, and spin bit state. One instance
/// per `ActivePeer`.
pub struct MmpPeerState {
pub sender: SenderState,
pub receiver: ReceiverState,
pub metrics: MmpMetrics,
pub spin_bit: SpinBitState,
mode: MmpMode,
log_interval: Duration,
last_log_time: Option<Instant>,
}
impl MmpPeerState {
/// Create MMP state for a new peer link.
///
/// `is_initiator`: true if this node initiated the Noise handshake
/// (determines spin bit role).
pub fn new(config: &MmpConfig, is_initiator: bool) -> Self {
Self {
sender: SenderState::new(),
receiver: ReceiverState::new(config.owd_window_size),
metrics: MmpMetrics::new(),
spin_bit: SpinBitState::new(is_initiator),
mode: config.mode,
log_interval: Duration::from_secs(config.log_interval_secs),
last_log_time: None,
}
}
/// Reset counter-dependent state for rekey cutover.
pub fn reset_for_rekey(&mut self, now: Instant) {
self.receiver.reset_for_rekey(now);
self.metrics.reset_for_rekey();
}
/// Current operating mode.
pub fn mode(&self) -> MmpMode {
self.mode
}
/// Check if it's time to emit a periodic metrics log.
pub fn should_log(&self, now: Instant) -> bool {
match self.last_log_time {
None => true,
Some(last) => now.duration_since(last) >= self.log_interval,
}
}
/// Mark that a periodic log was emitted.
pub fn mark_logged(&mut self, now: Instant) {
self.last_log_time = Some(now);
}
}
// ============================================================================
// Per-Session MMP State (session-layer instantiation)
// ============================================================================
/// Combined MMP state for a single end-to-end session.
///
/// Wraps sender, receiver, metrics, spin bit, and path MTU state.
/// One instance per established `SessionEntry`.
pub struct MmpSessionState {
pub sender: SenderState,
pub receiver: ReceiverState,
pub metrics: MmpMetrics,
pub spin_bit: SpinBitState,
mode: MmpMode,
log_interval: Duration,
last_log_time: Option<Instant>,
pub path_mtu: PathMtuState,
}
impl MmpSessionState {
/// Create MMP state for a new session.
///
/// `is_initiator`: true if this node initiated the Noise handshake
/// (determines spin bit role).
pub fn new(config: &crate::config::SessionMmpConfig, is_initiator: bool) -> Self {
Self {
sender: SenderState::new_with_cold_start(SESSION_COLD_START_INTERVAL_MS),
receiver: ReceiverState::new_with_cold_start(
config.owd_window_size,
SESSION_COLD_START_INTERVAL_MS,
),
metrics: MmpMetrics::new(),
spin_bit: SpinBitState::new(is_initiator),
mode: config.mode,
log_interval: Duration::from_secs(config.log_interval_secs),
last_log_time: None,
path_mtu: PathMtuState::new(),
}
}
/// Reset counter-dependent state for rekey cutover.
pub fn reset_for_rekey(&mut self, now: Instant) {
self.receiver.reset_for_rekey(now);
self.metrics.reset_for_rekey();
}
/// Current operating mode.
pub fn mode(&self) -> MmpMode {
self.mode
}
/// Check if it's time to emit a periodic metrics log.
pub fn should_log(&self, now: Instant) -> bool {
match self.last_log_time {
None => true,
Some(last) => now.duration_since(last) >= self.log_interval,
}
}
/// Mark that a periodic log was emitted.
pub fn mark_logged(&mut self, now: Instant) {
self.last_log_time = Some(now);
}
}
impl Debug for MmpSessionState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("MmpSessionState")
.field("mode", &self.mode)
.field("path_mtu", &self.path_mtu.current_mtu())
.finish_non_exhaustive()
}
}
// ============================================================================
// Path MTU State (session-layer only)
// ============================================================================
/// Path MTU tracking for a single session.
///
/// Destination side: observes `path_mtu` from incoming SessionDatagram envelopes
/// and generates PathMtuNotification messages back to the source.
///
/// Source side: applies received PathMtuNotification to limit outbound datagram
/// size. Decrease is immediate; increase requires 3 consecutive notifications.
pub struct PathMtuState {
/// Current effective path MTU (what we use for sending).
current_mtu: u16,
/// Last observed path MTU from incoming datagrams (destination-side).
last_observed_mtu: u16,
/// Whether the observed MTU has changed since the last notification.
observed_changed: bool,
/// Last time a PathMtuNotification was sent.
last_notification_time: Option<Instant>,
/// Notification interval: max(10s, 5 * SRTT). Default 10s.
notification_interval: Duration,
/// For source-side increase tracking: consecutive higher-value notifications.
consecutive_increase_count: u8,
/// Time of the first notification in the current increase sequence.
first_increase_time: Option<Instant>,
/// The MTU value being proposed for increase.
pending_increase_mtu: u16,
}
impl PathMtuState {
/// Create path MTU state with no initial measurement.
pub fn new() -> Self {
Self {
current_mtu: u16::MAX,
last_observed_mtu: u16::MAX,
observed_changed: false,
last_notification_time: None,
notification_interval: Duration::from_secs(10),
consecutive_increase_count: 0,
first_increase_time: None,
pending_increase_mtu: 0,
}
}
/// Current effective path MTU (source-side, for sending).
pub fn current_mtu(&self) -> u16 {
self.current_mtu
}
/// Last observed incoming path MTU (destination-side).
pub fn last_observed_mtu(&self) -> u16 {
self.last_observed_mtu
}
/// Update notification interval from SRTT: max(10s, 5 * SRTT).
pub fn update_interval_from_srtt(&mut self, srtt_ms: f64) {
let five_srtt = Duration::from_millis((srtt_ms * 5.0) as u64);
self.notification_interval = five_srtt.max(Duration::from_secs(10));
}
/// Seed source-side current_mtu from outbound transport MTU.
///
/// Called on each send. Only decreases (never increases) the current_mtu
/// so the destination's PathMtuNotification can still raise it later.
/// Ensures current_mtu doesn't stay at u16::MAX before any notification
/// arrives from the destination.
pub fn seed_source_mtu(&mut self, outbound_mtu: u16) {
if outbound_mtu < self.current_mtu {
self.current_mtu = outbound_mtu;
}
}
// --- Destination side ---
/// Observe the path_mtu from an incoming SessionDatagram envelope.
///
/// Called on the destination (receiver) side for every session message.
pub fn observe_incoming_mtu(&mut self, path_mtu: u16) {
if path_mtu != self.last_observed_mtu {
self.observed_changed = true;
self.last_observed_mtu = path_mtu;
}
}
/// Check if a PathMtuNotification should be sent.
///
/// Send on first measurement, on decrease (immediate), or periodic
/// confirmation at the notification interval.
pub fn should_send_notification(&self, now: Instant) -> bool {
if self.last_observed_mtu == u16::MAX {
return false; // No measurement yet
}
match self.last_notification_time {
None => true, // First measurement
Some(last) => {
// Immediate on decrease
if self.observed_changed && self.last_observed_mtu < self.current_mtu {
return true;
}
// Periodic confirmation
now.duration_since(last) >= self.notification_interval
}
}
}
/// Build a PathMtuNotification from current state.
///
/// Returns the path_mtu value to send. Caller handles encoding.
pub fn build_notification(&mut self, now: Instant) -> Option<u16> {
if self.last_observed_mtu == u16::MAX {
return None;
}
self.last_notification_time = Some(now);
self.observed_changed = false;
Some(self.last_observed_mtu)
}
// --- Source side ---
/// Apply a received PathMtuNotification.
///
/// - Decrease: immediate (take the lower value).
/// - Increase: require 3 consecutive notifications with the same higher
/// value, spanning at least 2 * notification_interval.
///
/// Returns `true` if the effective MTU changed.
pub fn apply_notification(&mut self, reported_mtu: u16, now: Instant) -> bool {
if reported_mtu < self.current_mtu {
// Decrease: immediate
self.current_mtu = reported_mtu;
self.consecutive_increase_count = 0;
self.first_increase_time = None;
return true;
}
if reported_mtu > self.current_mtu {
// Increase: track consecutive notifications
if reported_mtu == self.pending_increase_mtu {
self.consecutive_increase_count += 1;
} else {
// Different value: reset sequence
self.pending_increase_mtu = reported_mtu;
self.consecutive_increase_count = 1;
self.first_increase_time = Some(now);
}
// Accept increase after 3 consecutive spanning 2 * interval
if self.consecutive_increase_count >= 3
&& let Some(first_time) = self.first_increase_time
{
let required = self.notification_interval * 2;
if now.duration_since(first_time) >= required {
self.current_mtu = reported_mtu;
self.consecutive_increase_count = 0;
self.first_increase_time = None;
return true;
}
}
}
// No change (equal or increase not yet confirmed)
false
}
}
impl Default for PathMtuState {
fn default() -> Self {
Self::new()
}
}
impl Debug for MmpPeerState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("MmpPeerState")
.field("mode", &self.mode)
.finish_non_exhaustive()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_mode_default() {
assert_eq!(MmpMode::default(), MmpMode::Full);
}
#[test]
fn test_mode_display() {
assert_eq!(MmpMode::Full.to_string(), "full");
assert_eq!(MmpMode::Lightweight.to_string(), "lightweight");
assert_eq!(MmpMode::Minimal.to_string(), "minimal");
}
#[test]
fn test_mode_serde_roundtrip() {
let yaml = "full";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Full);
let yaml = "lightweight";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Lightweight);
let yaml = "minimal";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Minimal);
}
#[test]
fn test_config_default() {
let config = MmpConfig::default();
assert_eq!(config.mode, MmpMode::Full);
assert_eq!(config.log_interval_secs, 30);
assert_eq!(config.owd_window_size, 32);
}
#[test]
fn test_config_yaml_parse() {
let yaml = r#"
mode: lightweight
log_interval_secs: 60
owd_window_size: 48
"#;
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Lightweight);
assert_eq!(config.log_interval_secs, 60);
assert_eq!(config.owd_window_size, 48);
}
#[test]
fn test_config_yaml_partial() {
let yaml = "mode: minimal";
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Minimal);
assert_eq!(config.log_interval_secs, DEFAULT_LOG_INTERVAL_SECS);
assert_eq!(config.owd_window_size, DEFAULT_OWD_WINDOW_SIZE);
}
}
src/mmp/report.rs
-385
View File
@@ -1,385 +0,0 @@
//! MMP report wire format: SenderReport and ReceiverReport.
//!
//! Serialization and deserialization for the two report types exchanged
//! between link-layer peers. Wire format follows the MMP design doc.
use crate::protocol::ProtocolError;
// ============================================================================
// SenderReport (msg_type 0x01, 48-byte body including type byte)
// ============================================================================
/// Link-layer sender report.
///
/// Wire layout (48 bytes total, sent as link message):
/// ```text
/// [0] msg_type = 0x01
/// [1-3] reserved (zero)
/// [4-11] interval_start_counter: u64 LE
/// [12-19] interval_end_counter: u64 LE
/// [20-23] interval_start_timestamp: u32 LE
/// [24-27] interval_end_timestamp: u32 LE
/// [28-31] interval_bytes_sent: u32 LE
/// [32-39] cumulative_packets_sent: u64 LE
/// [40-47] cumulative_bytes_sent: u64 LE
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SenderReport {
pub interval_start_counter: u64,
pub interval_end_counter: u64,
pub interval_start_timestamp: u32,
pub interval_end_timestamp: u32,
pub interval_bytes_sent: u32,
pub cumulative_packets_sent: u64,
pub cumulative_bytes_sent: u64,
}
/// ReceiverReport (msg_type 0x02, 68-byte body including type byte)
///
/// Wire layout (68 bytes total, sent as link message):
/// ```text
/// [0] msg_type = 0x02
/// [1-3] reserved (zero)
/// [4-11] highest_counter: u64 LE
/// [12-19] cumulative_packets_recv: u64 LE
/// [20-27] cumulative_bytes_recv: u64 LE
/// [28-31] timestamp_echo: u32 LE
/// [32-33] dwell_time: u16 LE
/// [34-35] max_burst_loss: u16 LE
/// [36-37] mean_burst_loss: u16 LE (u8.8 fixed-point)
/// [38-39] reserved: u16 LE
/// [40-43] jitter: u32 LE (microseconds)
/// [44-47] ecn_ce_count: u32 LE
/// [48-51] owd_trend: i32 LE (µs/s)
/// [52-55] burst_loss_count: u32 LE
/// [56-59] cumulative_reorder_count: u32 LE
/// [60-63] interval_packets_recv: u32 LE
/// [64-67] interval_bytes_recv: u32 LE
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ReceiverReport {
pub highest_counter: u64,
pub cumulative_packets_recv: u64,
pub cumulative_bytes_recv: u64,
pub timestamp_echo: u32,
pub dwell_time: u16,
pub max_burst_loss: u16,
pub mean_burst_loss: u16,
pub jitter: u32,
pub ecn_ce_count: u32,
pub owd_trend: i32,
pub burst_loss_count: u32,
pub cumulative_reorder_count: u32,
pub interval_packets_recv: u32,
pub interval_bytes_recv: u32,
}
// Encode/decode will be implemented in Step 2.
impl SenderReport {
/// Encode to wire format (48 bytes: msg_type + 3 reserved + 44 payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(48);
buf.push(0x01); // msg_type
buf.extend_from_slice(&[0u8; 3]); // reserved
buf.extend_from_slice(&self.interval_start_counter.to_le_bytes());
buf.extend_from_slice(&self.interval_end_counter.to_le_bytes());
buf.extend_from_slice(&self.interval_start_timestamp.to_le_bytes());
buf.extend_from_slice(&self.interval_end_timestamp.to_le_bytes());
buf.extend_from_slice(&self.interval_bytes_sent.to_le_bytes());
buf.extend_from_slice(&self.cumulative_packets_sent.to_le_bytes());
buf.extend_from_slice(&self.cumulative_bytes_sent.to_le_bytes());
buf
}
/// Decode from payload after msg_type byte has been consumed.
///
/// `payload` starts at the reserved bytes (offset 1 in the wire format).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
if payload.len() < 47 {
return Err(ProtocolError::MessageTooShort {
expected: 47,
got: payload.len(),
});
}
// Skip 3 reserved bytes
let p = &payload[3..];
Ok(Self {
interval_start_counter: u64::from_le_bytes(p[0..8].try_into().unwrap()),
interval_end_counter: u64::from_le_bytes(p[8..16].try_into().unwrap()),
interval_start_timestamp: u32::from_le_bytes(p[16..20].try_into().unwrap()),
interval_end_timestamp: u32::from_le_bytes(p[20..24].try_into().unwrap()),
interval_bytes_sent: u32::from_le_bytes(p[24..28].try_into().unwrap()),
cumulative_packets_sent: u64::from_le_bytes(p[28..36].try_into().unwrap()),
cumulative_bytes_sent: u64::from_le_bytes(p[36..44].try_into().unwrap()),
})
}
}
impl ReceiverReport {
/// Encode to wire format (68 bytes: msg_type + 3 reserved + 64 payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(68);
buf.push(0x02); // msg_type
buf.extend_from_slice(&[0u8; 3]); // reserved
buf.extend_from_slice(&self.highest_counter.to_le_bytes());
buf.extend_from_slice(&self.cumulative_packets_recv.to_le_bytes());
buf.extend_from_slice(&self.cumulative_bytes_recv.to_le_bytes());
buf.extend_from_slice(&self.timestamp_echo.to_le_bytes());
buf.extend_from_slice(&self.dwell_time.to_le_bytes());
buf.extend_from_slice(&self.max_burst_loss.to_le_bytes());
buf.extend_from_slice(&self.mean_burst_loss.to_le_bytes());
buf.extend_from_slice(&[0u8; 2]); // reserved
buf.extend_from_slice(&self.jitter.to_le_bytes());
buf.extend_from_slice(&self.ecn_ce_count.to_le_bytes());
buf.extend_from_slice(&self.owd_trend.to_le_bytes());
buf.extend_from_slice(&self.burst_loss_count.to_le_bytes());
buf.extend_from_slice(&self.cumulative_reorder_count.to_le_bytes());
buf.extend_from_slice(&self.interval_packets_recv.to_le_bytes());
buf.extend_from_slice(&self.interval_bytes_recv.to_le_bytes());
buf
}
/// Decode from payload after msg_type byte has been consumed.
///
/// `payload` starts at the reserved bytes (offset 1 in the wire format).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
if payload.len() < 67 {
return Err(ProtocolError::MessageTooShort {
expected: 67,
got: payload.len(),
});
}
// Skip 3 reserved bytes
let p = &payload[3..];
Ok(Self {
highest_counter: u64::from_le_bytes(p[0..8].try_into().unwrap()),
cumulative_packets_recv: u64::from_le_bytes(p[8..16].try_into().unwrap()),
cumulative_bytes_recv: u64::from_le_bytes(p[16..24].try_into().unwrap()),
timestamp_echo: u32::from_le_bytes(p[24..28].try_into().unwrap()),
dwell_time: u16::from_le_bytes(p[28..30].try_into().unwrap()),
max_burst_loss: u16::from_le_bytes(p[30..32].try_into().unwrap()),
mean_burst_loss: u16::from_le_bytes(p[32..34].try_into().unwrap()),
// skip 2 reserved bytes at p[34..36]
jitter: u32::from_le_bytes(p[36..40].try_into().unwrap()),
ecn_ce_count: u32::from_le_bytes(p[40..44].try_into().unwrap()),
owd_trend: i32::from_le_bytes(p[44..48].try_into().unwrap()),
burst_loss_count: u32::from_le_bytes(p[48..52].try_into().unwrap()),
cumulative_reorder_count: u32::from_le_bytes(p[52..56].try_into().unwrap()),
interval_packets_recv: u32::from_le_bytes(p[56..60].try_into().unwrap()),
interval_bytes_recv: u32::from_le_bytes(p[60..64].try_into().unwrap()),
})
}
}
// ============================================================================
// Conversions between link-layer and session-layer report types
// ============================================================================
use crate::protocol::{SessionReceiverReport, SessionSenderReport};
impl From<&SenderReport> for SessionSenderReport {
fn from(r: &SenderReport) -> Self {
Self {
interval_start_counter: r.interval_start_counter,
interval_end_counter: r.interval_end_counter,
interval_start_timestamp: r.interval_start_timestamp,
interval_end_timestamp: r.interval_end_timestamp,
interval_bytes_sent: r.interval_bytes_sent,
cumulative_packets_sent: r.cumulative_packets_sent,
cumulative_bytes_sent: r.cumulative_bytes_sent,
}
}
}
impl From<&SessionSenderReport> for SenderReport {
fn from(r: &SessionSenderReport) -> Self {
Self {
interval_start_counter: r.interval_start_counter,
interval_end_counter: r.interval_end_counter,
interval_start_timestamp: r.interval_start_timestamp,
interval_end_timestamp: r.interval_end_timestamp,
interval_bytes_sent: r.interval_bytes_sent,
cumulative_packets_sent: r.cumulative_packets_sent,
cumulative_bytes_sent: r.cumulative_bytes_sent,
}
}
}
impl From<&ReceiverReport> for SessionReceiverReport {
fn from(r: &ReceiverReport) -> Self {
Self {
highest_counter: r.highest_counter,
cumulative_packets_recv: r.cumulative_packets_recv,
cumulative_bytes_recv: r.cumulative_bytes_recv,
timestamp_echo: r.timestamp_echo,
dwell_time: r.dwell_time,
max_burst_loss: r.max_burst_loss,
mean_burst_loss: r.mean_burst_loss,
jitter: r.jitter,
ecn_ce_count: r.ecn_ce_count,
owd_trend: r.owd_trend,
burst_loss_count: r.burst_loss_count,
cumulative_reorder_count: r.cumulative_reorder_count,
interval_packets_recv: r.interval_packets_recv,
interval_bytes_recv: r.interval_bytes_recv,
}
}
}
impl From<&SessionReceiverReport> for ReceiverReport {
fn from(r: &SessionReceiverReport) -> Self {
Self {
highest_counter: r.highest_counter,
cumulative_packets_recv: r.cumulative_packets_recv,
cumulative_bytes_recv: r.cumulative_bytes_recv,
timestamp_echo: r.timestamp_echo,
dwell_time: r.dwell_time,
max_burst_loss: r.max_burst_loss,
mean_burst_loss: r.mean_burst_loss,
jitter: r.jitter,
ecn_ce_count: r.ecn_ce_count,
owd_trend: r.owd_trend,
burst_loss_count: r.burst_loss_count,
cumulative_reorder_count: r.cumulative_reorder_count,
interval_packets_recv: r.interval_packets_recv,
interval_bytes_recv: r.interval_bytes_recv,
}
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
fn sample_sender_report() -> SenderReport {
SenderReport {
interval_start_counter: 100,
interval_end_counter: 200,
interval_start_timestamp: 5000,
interval_end_timestamp: 6000,
interval_bytes_sent: 50_000,
cumulative_packets_sent: 10_000,
cumulative_bytes_sent: 5_000_000,
}
}
fn sample_receiver_report() -> ReceiverReport {
ReceiverReport {
highest_counter: 195,
cumulative_packets_recv: 9_500,
cumulative_bytes_recv: 4_750_000,
timestamp_echo: 5900,
dwell_time: 5,
max_burst_loss: 3,
mean_burst_loss: 384, // 1.5 in u8.8
jitter: 1200,
ecn_ce_count: 0,
owd_trend: -50,
burst_loss_count: 2,
cumulative_reorder_count: 10,
interval_packets_recv: 95,
interval_bytes_recv: 47_500,
}
}
#[test]
fn test_sender_report_encode_size() {
let sr = sample_sender_report();
let encoded = sr.encode();
assert_eq!(encoded.len(), 48);
assert_eq!(encoded[0], 0x01); // msg_type
}
#[test]
fn test_sender_report_roundtrip() {
let sr = sample_sender_report();
let encoded = sr.encode();
// decode expects payload after msg_type
let decoded = SenderReport::decode(&encoded[1..]).unwrap();
assert_eq!(sr, decoded);
}
#[test]
fn test_sender_report_too_short() {
let result = SenderReport::decode(&[0u8; 10]);
assert!(result.is_err());
}
#[test]
fn test_receiver_report_encode_size() {
let rr = sample_receiver_report();
let encoded = rr.encode();
assert_eq!(encoded.len(), 68);
assert_eq!(encoded[0], 0x02); // msg_type
}
#[test]
fn test_receiver_report_roundtrip() {
let rr = sample_receiver_report();
let encoded = rr.encode();
// decode expects payload after msg_type
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(rr, decoded);
}
#[test]
fn test_receiver_report_too_short() {
let result = ReceiverReport::decode(&[0u8; 10]);
assert!(result.is_err());
}
#[test]
fn test_sender_report_zero_values() {
let sr = SenderReport {
interval_start_counter: 0,
interval_end_counter: 0,
interval_start_timestamp: 0,
interval_end_timestamp: 0,
interval_bytes_sent: 0,
cumulative_packets_sent: 0,
cumulative_bytes_sent: 0,
};
let encoded = sr.encode();
let decoded = SenderReport::decode(&encoded[1..]).unwrap();
assert_eq!(sr, decoded);
}
#[test]
fn test_receiver_report_max_values() {
let rr = ReceiverReport {
highest_counter: u64::MAX,
cumulative_packets_recv: u64::MAX,
cumulative_bytes_recv: u64::MAX,
timestamp_echo: u32::MAX,
dwell_time: u16::MAX,
max_burst_loss: u16::MAX,
mean_burst_loss: u16::MAX,
jitter: u32::MAX,
ecn_ce_count: u32::MAX,
owd_trend: i32::MAX,
burst_loss_count: u32::MAX,
cumulative_reorder_count: u32::MAX,
interval_packets_recv: u32::MAX,
interval_bytes_recv: u32::MAX,
};
let encoded = rr.encode();
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(rr, decoded);
}
#[test]
fn test_receiver_report_negative_owd_trend() {
let rr = ReceiverReport {
owd_trend: -12345,
..sample_receiver_report()
};
let encoded = rr.encode();
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(decoded.owd_trend, -12345);
}
}
src/mmp/sender.rs
-418
View File
@@ -1,418 +0,0 @@
//! MMP sender state machine.
//!
//! Tracks what this node has sent to a specific peer and produces
//! SenderReport messages on demand. One `SenderState` per active peer.
use std::time::{Duration, Instant};
use crate::mmp::report::SenderReport;
use crate::mmp::{
COLD_START_SAMPLES, DEFAULT_COLD_START_INTERVAL_MS, MAX_REPORT_INTERVAL_MS,
MIN_REPORT_INTERVAL_MS,
};
/// Per-peer sender-side MMP state.
///
/// Records cumulative and interval counters for every frame transmitted
/// to this peer. Produces `SenderReport` snapshots on demand.
pub struct SenderState {
// --- Cumulative (lifetime) ---
cumulative_packets_sent: u64,
cumulative_bytes_sent: u64,
// --- Current interval ---
interval_start_counter: u64,
interval_start_timestamp: u32,
interval_bytes_sent: u32,
/// Counter of the most recently sent frame.
last_counter: u64,
/// Timestamp of the most recently sent frame.
last_timestamp: u32,
/// Whether any frames have been sent in the current interval.
interval_has_data: bool,
// --- Report timing ---
last_report_time: Option<Instant>,
report_interval: Duration,
// --- Send failure backoff ---
/// Consecutive send failure count for backoff calculation.
consecutive_send_failures: u32,
// --- Cold-start tracking ---
/// Number of SRTT-based interval updates received.
srtt_sample_count: u32,
}
impl SenderState {
pub fn new() -> Self {
Self::new_with_cold_start(DEFAULT_COLD_START_INTERVAL_MS)
}
/// Create with a custom cold-start interval (ms).
///
/// Used by session-layer MMP which needs a longer initial interval
/// since reports consume bandwidth on every transit link.
pub fn new_with_cold_start(cold_start_ms: u64) -> Self {
Self {
cumulative_packets_sent: 0,
cumulative_bytes_sent: 0,
interval_start_counter: 0,
interval_start_timestamp: 0,
interval_bytes_sent: 0,
last_counter: 0,
last_timestamp: 0,
interval_has_data: false,
last_report_time: None,
report_interval: Duration::from_millis(cold_start_ms),
consecutive_send_failures: 0,
srtt_sample_count: 0,
}
}
/// Record a frame sent to this peer.
///
/// Called on the TX path for every encrypted link message.
/// `counter` is the AEAD nonce/counter, `timestamp` is the inner header
/// session-relative timestamp (ms), `bytes` is the wire payload size.
pub fn record_sent(&mut self, counter: u64, timestamp: u32, bytes: usize) {
if !self.interval_has_data {
self.interval_start_counter = counter;
self.interval_start_timestamp = timestamp;
self.interval_has_data = true;
}
self.last_counter = counter;
self.last_timestamp = timestamp;
self.interval_bytes_sent = self.interval_bytes_sent.saturating_add(bytes as u32);
self.cumulative_packets_sent += 1;
self.cumulative_bytes_sent += bytes as u64;
}
/// Build a SenderReport from current state and reset the interval.
///
/// Returns `None` if no frames have been sent since the last report.
pub fn build_report(&mut self, now: Instant) -> Option<SenderReport> {
if !self.interval_has_data {
return None;
}
let report = SenderReport {
interval_start_counter: self.interval_start_counter,
interval_end_counter: self.last_counter,
interval_start_timestamp: self.interval_start_timestamp,
interval_end_timestamp: self.last_timestamp,
interval_bytes_sent: self.interval_bytes_sent,
cumulative_packets_sent: self.cumulative_packets_sent,
cumulative_bytes_sent: self.cumulative_bytes_sent,
};
// Reset interval
self.interval_has_data = false;
self.interval_bytes_sent = 0;
self.last_report_time = Some(now);
Some(report)
}
/// Check if it's time to send a report.
///
/// When consecutive send failures have occurred, the effective interval
/// is multiplied by an exponential backoff factor (2^failures, capped at 32×).
pub fn should_send_report(&self, now: Instant) -> bool {
if !self.interval_has_data {
return false;
}
match self.last_report_time {
None => true, // Never sent a report — send immediately
Some(last) => {
let effective = self
.report_interval
.mul_f64(self.send_failure_backoff_multiplier());
now.duration_since(last) >= effective
}
}
}
/// Record a send failure. Returns the new consecutive failure count.
pub fn record_send_failure(&mut self) -> u32 {
self.consecutive_send_failures += 1;
self.consecutive_send_failures
}
/// Record a successful send. Returns the previous failure count (for summary logging).
pub fn record_send_success(&mut self) -> u32 {
let prev = self.consecutive_send_failures;
self.consecutive_send_failures = 0;
prev
}
/// Get the backoff multiplier based on consecutive failures.
///
/// Returns 1.0 for no failures, 2.0 for 1 failure, 4.0 for 2, ...
/// capped at 32.0 (5 failures).
pub fn send_failure_backoff_multiplier(&self) -> f64 {
if self.consecutive_send_failures == 0 {
1.0
} else {
2.0_f64.powi(self.consecutive_send_failures.min(5) as i32)
}
}
/// Update the report interval based on SRTT (link-layer defaults).
///
/// Sender reports at 2× SRTT clamped to [floor, MAX]. During cold-start
/// (first `COLD_START_SAMPLES` updates), the floor is the cold-start
/// interval (200ms) for fast SRTT convergence. After that, it rises to
/// `MIN_REPORT_INTERVAL_MS` (1000ms) for steady-state efficiency.
pub fn update_report_interval_from_srtt(&mut self, srtt_us: i64) {
self.srtt_sample_count = self.srtt_sample_count.saturating_add(1);
let floor = if self.srtt_sample_count <= COLD_START_SAMPLES {
DEFAULT_COLD_START_INTERVAL_MS
} else {
MIN_REPORT_INTERVAL_MS
};
self.update_report_interval_with_bounds(srtt_us, floor, MAX_REPORT_INTERVAL_MS);
}
/// Update the report interval based on SRTT with custom bounds.
///
/// Used by session-layer MMP which needs higher clamp values since
/// each report consumes bandwidth on every transit link.
pub fn update_report_interval_with_bounds(&mut self, srtt_us: i64, min_ms: u64, max_ms: u64) {
if srtt_us <= 0 {
return;
}
let interval_us = (srtt_us * 2) as u64;
let interval_ms = (interval_us / 1000).clamp(min_ms, max_ms);
self.report_interval = Duration::from_millis(interval_ms);
}
// --- Accessors ---
pub fn cumulative_packets_sent(&self) -> u64 {
self.cumulative_packets_sent
}
pub fn cumulative_bytes_sent(&self) -> u64 {
self.cumulative_bytes_sent
}
pub fn report_interval(&self) -> Duration {
self.report_interval
}
pub fn consecutive_send_failures(&self) -> u32 {
self.consecutive_send_failures
}
}
impl Default for SenderState {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_sender_state() {
let s = SenderState::new();
assert_eq!(s.cumulative_packets_sent(), 0);
assert_eq!(s.cumulative_bytes_sent(), 0);
}
#[test]
fn test_record_sent() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
s.record_sent(2, 200, 600);
assert_eq!(s.cumulative_packets_sent(), 2);
assert_eq!(s.cumulative_bytes_sent(), 1100);
}
#[test]
fn test_build_report_empty() {
let mut s = SenderState::new();
assert!(s.build_report(Instant::now()).is_none());
}
#[test]
fn test_build_report() {
let mut s = SenderState::new();
s.record_sent(10, 1000, 500);
s.record_sent(11, 1100, 600);
s.record_sent(12, 1200, 400);
let report = s.build_report(Instant::now()).unwrap();
assert_eq!(report.interval_start_counter, 10);
assert_eq!(report.interval_end_counter, 12);
assert_eq!(report.interval_start_timestamp, 1000);
assert_eq!(report.interval_end_timestamp, 1200);
assert_eq!(report.interval_bytes_sent, 1500);
assert_eq!(report.cumulative_packets_sent, 3);
assert_eq!(report.cumulative_bytes_sent, 1500);
}
#[test]
fn test_build_report_resets_interval() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
let _ = s.build_report(Instant::now());
// Second report with no new data returns None
assert!(s.build_report(Instant::now()).is_none());
// New data starts a fresh interval
s.record_sent(2, 200, 300);
let report = s.build_report(Instant::now()).unwrap();
assert_eq!(report.interval_start_counter, 2);
assert_eq!(report.interval_bytes_sent, 300);
// Cumulative continues
assert_eq!(report.cumulative_packets_sent, 2);
assert_eq!(report.cumulative_bytes_sent, 800);
}
#[test]
fn test_should_send_report_no_data() {
let s = SenderState::new();
assert!(!s.should_send_report(Instant::now()));
}
#[test]
fn test_should_send_report_first_time() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
assert!(s.should_send_report(Instant::now()));
}
#[test]
fn test_should_send_report_respects_interval() {
let mut s = SenderState::new();
let t0 = Instant::now();
s.record_sent(1, 100, 500);
let _ = s.build_report(t0);
s.record_sent(2, 200, 500);
// Immediately after report — should not send
assert!(!s.should_send_report(t0));
// After interval elapses
let t1 = t0 + s.report_interval() + Duration::from_millis(1);
assert!(s.should_send_report(t1));
}
#[test]
fn test_update_report_interval_cold_start() {
let mut s = SenderState::new();
// During cold-start, floor is 200ms (DEFAULT_COLD_START_INTERVAL_MS)
// 50ms RTT → 100ms sender interval (2× SRTT), clamped to cold-start floor 200ms
s.update_report_interval_from_srtt(50_000);
assert_eq!(s.report_interval(), Duration::from_millis(200));
// 500ms RTT → 1000ms sender interval (above cold-start floor)
s.update_report_interval_from_srtt(500_000);
assert_eq!(s.report_interval(), Duration::from_millis(1000));
}
#[test]
fn test_update_report_interval_after_cold_start() {
let mut s = SenderState::new();
// Burn through cold-start samples (COLD_START_SAMPLES = 5)
for _ in 0..COLD_START_SAMPLES {
s.update_report_interval_from_srtt(500_000);
}
// 6th sample: now in steady state, floor is MIN_REPORT_INTERVAL_MS (1000ms)
// 50ms RTT → 100ms sender interval (2× SRTT), clamped to 1000ms
s.update_report_interval_from_srtt(50_000);
assert_eq!(
s.report_interval(),
Duration::from_millis(MIN_REPORT_INTERVAL_MS)
);
// 3s RTT → 6s, clamped to max 5s
s.update_report_interval_from_srtt(3_000_000);
assert_eq!(
s.report_interval(),
Duration::from_millis(MAX_REPORT_INTERVAL_MS)
);
}
#[test]
fn test_backoff_multiplier_progression() {
let mut s = SenderState::new();
// No failures → multiplier 1.0
assert_eq!(s.send_failure_backoff_multiplier(), 1.0);
assert_eq!(s.consecutive_send_failures(), 0);
// Progressive failures: 2^1, 2^2, 2^3, 2^4, 2^5
let expected = [2.0, 4.0, 8.0, 16.0, 32.0];
for (i, &exp) in expected.iter().enumerate() {
let count = s.record_send_failure();
assert_eq!(count, (i + 1) as u32);
assert_eq!(s.send_failure_backoff_multiplier(), exp);
}
// Beyond 5 failures: stays capped at 32.0
s.record_send_failure(); // 6th
assert_eq!(s.send_failure_backoff_multiplier(), 32.0);
s.record_send_failure(); // 7th
assert_eq!(s.send_failure_backoff_multiplier(), 32.0);
}
#[test]
fn test_backoff_reset_on_success() {
let mut s = SenderState::new();
// Accumulate failures
s.record_send_failure();
s.record_send_failure();
s.record_send_failure();
assert_eq!(s.consecutive_send_failures(), 3);
assert_eq!(s.send_failure_backoff_multiplier(), 8.0);
// Success resets and returns previous count
let prev = s.record_send_success();
assert_eq!(prev, 3);
assert_eq!(s.consecutive_send_failures(), 0);
assert_eq!(s.send_failure_backoff_multiplier(), 1.0);
}
#[test]
fn test_backoff_success_with_no_prior_failures() {
let mut s = SenderState::new();
// Success with no failures returns 0
let prev = s.record_send_success();
assert_eq!(prev, 0);
assert_eq!(s.consecutive_send_failures(), 0);
}
#[test]
fn test_should_send_report_respects_backoff() {
let mut s = SenderState::new();
let t0 = Instant::now();
s.record_sent(1, 100, 500);
let _ = s.build_report(t0);
// Record a failure: multiplier becomes 2.0
s.record_send_failure();
s.record_sent(2, 200, 500);
// At 1× interval: should NOT send (backoff requires 2×)
let t1 = t0 + s.report_interval() + Duration::from_millis(1);
assert!(!s.should_send_report(t1));
// At 2× interval: should send
let t2 = t0 + s.report_interval() * 2 + Duration::from_millis(1);
assert!(s.should_send_report(t2));
}
}
src/node/bloom.rs
+31 -23
View File
@@ -4,12 +4,12 @@
//! including debounced propagation to peers.
use crate::NodeAddr;
use crate::bloom::BloomFilter;
use crate::protocol::FilterAnnounce;
use crate::proto::bloom::BloomFilter;
use crate::proto::bloom::FilterAnnounce;
use super::reject::BloomReject;
use super::{Node, NodeError};
use std::collections::HashMap;
use std::collections::BTreeMap;
use tracing::{debug, warn};
impl Node {
@@ -17,8 +17,8 @@ impl Node {
///
/// Returns a map of (peer_node_addr -> filter) for peers that
/// have sent us a FilterAnnounce.
pub(super) fn peer_inbound_filters(&self) -> HashMap<NodeAddr, BloomFilter> {
let mut filters = HashMap::new();
pub(super) fn peer_inbound_filters(&self) -> BTreeMap<NodeAddr, BloomFilter> {
let mut filters = BTreeMap::new();
for (addr, peer) in &self.peers {
if self.is_tree_peer(addr)
&& let Some(filter) = peer.inbound_filter()
@@ -29,27 +29,19 @@ impl Node {
filters
}
/// Build a FilterAnnounce for a specific peer.
///
/// The outgoing filter excludes the destination peer's own filter
/// to prevent routing loops (don't tell a peer about destinations
/// reachable only through them).
fn build_filter_announce(&mut self, exclude_peer: &NodeAddr) -> FilterAnnounce {
let peer_filters = self.peer_inbound_filters();
let filter = self
.bloom_state
.compute_outgoing_filter(exclude_peer, &peer_filters);
let sequence = self.bloom_state.next_sequence();
FilterAnnounce::new(filter, sequence)
}
/// Send a FilterAnnounce to a specific peer, respecting debounce.
///
/// `filter` is the outgoing filter for this peer, already computed
/// with the destination peer's own contribution excluded to prevent
/// routing loops (don't tell a peer about destinations reachable
/// only through them).
///
/// If the peer is rate-limited, the update stays pending for
/// delivery on the next tick cycle.
pub(super) async fn send_filter_announce_to_peer(
&mut self,
peer_addr: &NodeAddr,
filter: BloomFilter,
) -> Result<(), NodeError> {
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
@@ -64,7 +56,7 @@ impl Node {
}
// Build and encode
let announce = self.build_filter_announce(peer_addr);
let announce = FilterAnnounce::new(filter, self.bloom_state.next_sequence());
let sent_filter = announce.filter.clone();
let encoded = announce.encode().map_err(|e| NodeError::SendFailed {
node_addr: *peer_addr,
@@ -87,7 +79,7 @@ impl Node {
// operator to see one clear message, not spam.
let max_fpr = self.config().node.bloom.max_inbound_fpr;
let out_fill = sent_filter.fill_ratio();
let out_fpr = out_fill.powi(sent_filter.hash_count() as i32);
let out_fpr = sent_filter.fpr();
if out_fpr > max_fpr {
let now = std::time::Instant::now();
let should_warn = self
@@ -142,8 +134,24 @@ impl Node {
.copied()
.collect();
if ready.is_empty() {
return;
}
// One snapshot and one union pass for the whole ready set. The
// send path never mutates peer inbound filters or the tree state,
// and the rx loop holds `&mut self` across the awaits, so the
// snapshot cannot go stale mid-loop.
let peer_filters = self.peer_inbound_filters();
let mut outgoing = self
.bloom_state
.compute_outgoing_filters(&ready, &peer_filters);
for peer_addr in ready {
if let Err(e) = self.send_filter_announce_to_peer(&peer_addr).await {
let Some(filter) = outgoing.remove(&peer_addr) else {
continue;
};
if let Err(e) = self.send_filter_announce_to_peer(&peer_addr, filter).await {
debug!(
peer = %self.peer_display_name(&peer_addr),
error = %e,
@@ -213,7 +221,7 @@ impl Node {
// to wipe a victim's contribution to aggregation.
let max_fpr = self.config().node.bloom.max_inbound_fpr;
let fill = announce.filter.fill_ratio();
let fpr = fill.powi(announce.filter.hash_count() as i32);
let fpr = announce.filter.fpr();
if fpr > max_fpr {
self.metrics()
.bloom
src/node/context.rs
+6
View File
@@ -15,6 +15,7 @@
use std::sync::Arc;
use crate::proto::fmp::NodeProfile;
use crate::{Config, Identity};
/// Effectively-immutable `Node` state, shared via `Arc<NodeContext>`.
@@ -36,6 +37,9 @@ pub(crate) struct NodeContext {
/// Whether this is a leaf-only node.
pub is_leaf_only: bool,
/// This node's routing profile (Full, NonRouting, Leaf).
pub node_profile: NodeProfile,
/// Maximum connections (0 = unlimited).
pub max_connections: usize,
@@ -55,6 +59,7 @@ impl NodeContext {
startup_epoch: [u8; 8],
started_at: std::time::Instant,
is_leaf_only: bool,
node_profile: NodeProfile,
max_connections: usize,
max_peers: usize,
max_links: usize,
@@ -65,6 +70,7 @@ impl NodeContext {
startup_epoch,
started_at,
is_leaf_only,
node_profile,
max_connections,
max_peers,
max_links,
src/node/handlers/connected_udp.rs → src/node/dataplane/connected_udp.rs
+18 -16
View File
@@ -45,12 +45,10 @@ impl Node {
/// (e.g. only non-UDP transports). Enabled on Linux and macOS:
/// both kernels route a matching peer 5-tuple to the connected
/// socket when it shares the wildcard listen port via SO_REUSEPORT.
/// Only compiled on Linux/macOS — the sole caller (the rx_loop tick) is
/// gated the same way, so on other targets (android) there is nothing to do.
#[cfg(any(target_os = "linux", target_os = "macos"))]
pub(in crate::node) async fn activate_connected_udp_sessions(&mut self) {
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
{
// No-op on platforms without the connected-UDP fast path.
}
#[cfg(any(target_os = "linux", target_os = "macos"))]
{
if !connected_udp_enabled() {
return;
@@ -142,20 +140,24 @@ impl Node {
(peer_sa, local, recv_buf, send_buf, tx)
};
// Open the connected socket on the kernel side.
let socket = std::sync::Arc::new(
crate::transport::udp::connected_peer::ConnectedPeerSocket::open(
local_addr,
peer_socket_addr,
recv_buf,
send_buf,
)
.map_err(|e| format!("ConnectedPeerSocket::open: {e}"))?,
);
// Open the connected socket on the kernel side, then adopt the
// fd into the owning handle.
let owned = crate::transport::udp::open_connected_fd(
local_addr,
peer_socket_addr,
recv_buf,
send_buf,
)
.map_err(|e| format!("open_connected_fd: {e}"))?;
let socket = std::sync::Arc::new(crate::peer::connected_udp::ConnectedPeerSocket::from_fd(
owned,
peer_socket_addr,
local_addr,
));
// Spawn the drain thread. It feeds `packet_tx` exactly like
// the wildcard listen socket — rx_loop dispatches identically.
let drain = crate::transport::udp::peer_drain::PeerRecvDrain::spawn(
let drain = crate::peer::connected_udp::PeerRecvDrain::spawn(
socket.clone(),
transport_id,
peer_socket_addr,
src/node/handlers/dispatch.rs → src/node/dataplane/dispatch.rs
+9 -3
View File
@@ -73,7 +73,7 @@ impl Node {
/// entries — other removal paths (link-dead, decrypt failure, peer
/// restart) all schedule reconnect.
pub(in crate::node) fn handle_disconnect(&mut self, from: &NodeAddr, payload: &[u8]) {
let disconnect = match crate::protocol::Disconnect::decode(payload) {
let disconnect = match crate::proto::fmp::Disconnect::decode(payload) {
Ok(msg) => msg,
Err(e) => {
debug!(from = %self.peer_display_name(from), error = %e, "Malformed disconnect message");
@@ -93,7 +93,7 @@ impl Node {
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
self.schedule_reconnect(addr, now_ms);
self.note_link_dead(addr, now_ms);
}
/// Remove an active peer and clean up all associated state.
@@ -187,6 +187,11 @@ impl Node {
// Remove link and address mapping
self.remove_link(&link_id);
// Drop this peer's inert machine, keyed by the link_id
// derived above (a peer's link_id is immutable, so the key never
// moved). Keeps peers <-> peer_machines in exact correspondence on
// teardown. NEUTRAL: nothing reads peer_machines yet.
self.remove_peer_machine(link_id);
if let Some(transport_id) = transport_id {
self.cleanup_bootstrap_transport_if_unused(transport_id);
}
@@ -201,7 +206,8 @@ impl Node {
}
}
// Bloom filter cleanup: clear state for removed peer, mark all remaining peers
// Bloom filter cleanup: remove dependent (non-routing/leaf peers), clear state
self.bloom_state.remove_leaf_dependent(node_addr);
self.bloom_state.remove_peer_state(node_addr);
let remaining_peers: Vec<NodeAddr> = self.peers.keys().copied().collect();
self.bloom_state.mark_all_updates_needed(remaining_peers);
src/node/handlers/encrypted.rs → src/node/dataplane/encrypted.rs
+55 -45
View File
@@ -1,10 +1,9 @@
//! Encrypted frame handling (hot path).
use crate::node::Node;
use crate::node::wire::{EncryptedHeader, FLAG_CE, FLAG_KEY_EPOCH, FLAG_SP, strip_inner_header};
use crate::noise::NoiseError;
use crate::proto::fmp::wire::{EncryptedHeader, FLAG_CE, FLAG_KEY_EPOCH, strip_inner_header};
use crate::transport::ReceivedPacket;
use std::time::Instant;
use tracing::{debug, trace, warn};
/// Force-remove a peer after this many consecutive decryption failures.
@@ -58,15 +57,11 @@ impl Node {
// actually belongs to rekey N+1. Promoting on the bare bit then
// installs the WRONG Noise session as current — the two endpoints
// diverge, every subsequent frame fails AEAD on the far side, the
// receiver starves, and the link is declared dead at the heartbeat
// timeout (routing failure, green crypto). This mirrors the FSP fix
// (node/session.rs / node/handlers/session.rs): the authenticated
// decrypt, not the header bit, is the cutover signal. Trial-decrypt
// the frame against `pending` first; only promote if it
// authenticates. On success the same frame is delivered via
// `process_authentic_fmp_plaintext` and we return — it must not
// fall through to a second decrypt, which would be rejected as a
// replay (the trial-decrypt already advanced `pending`'s window).
// receiver starves, and the link is declared dead at the 30s
// heartbeat timeout (Phase-5 routing failure, green crypto). This
// mirrors the FSP fix (node/session.rs): the authenticated decrypt,
// not the header bit, is the cutover signal. Trial-decrypt the
// frame against `pending` first; only promote if it authenticates.
{
let Some(peer) = self.peers.get(&node_addr) else {
return;
@@ -77,12 +72,11 @@ impl Node {
if k_bit_flipped {
let ciphertext = &packet.data[header.ciphertext_offset()..];
let display_name = self.peer_display_name(&node_addr);
let our_addr = *self.identity().node_addr();
let Some(peer) = self.peers.get_mut(&node_addr) else {
return;
};
// Authenticate the frame against the pending session.
// Trial-decrypt mutates `pending`'s replay window only on
// success, so a failed trial leaves it untouched.
let pending_plaintext = peer.pending_new_session_mut().and_then(|pending| {
pending
.decrypt_with_replay_check_and_aad(
@@ -94,19 +88,27 @@ impl Node {
});
if let Some(plaintext) = pending_plaintext {
let pending_our = peer.pending_our_index();
let pending_their = peer.pending_their_index();
debug!(
peer = %display_name,
our_addr = %our_addr,
their_addr = %node_addr,
pending_our_index = ?pending_our,
pending_their_index = ?pending_their,
"Peer new-epoch frame authenticated, K-bit flip promoting new session"
);
// The peer authenticated a frame on the new epoch, so it
// derived the new session (it received our rekey msg3).
// If we are the rekey initiator still retransmitting msg3,
// stop — the responder is confirmed. (No-op for the
// responder side, which never retained a msg3 payload.)
peer.clear_rekey_msg3_payload();
// The trial-decrypt already advanced the pending
// session's replay window; `handle_peer_kbit_flip`
// moves that same session object to `current`, so no
// re-decrypt.
// session's replay window; handle_peer_kbit_flip moves
// that same session object to current, so no re-decrypt.
let did_flip = peer.handle_peer_kbit_flip().is_some();
if did_flip {
// New index was pre-registered in peers_by_index
// during msg1 handling (handshake.rs). Verify,
// don't duplicate.
debug_assert!(
peer.transport_id().is_some()
&& peer.our_index().is_some()
@@ -117,13 +119,8 @@ impl Node {
"peers_by_index should contain pre-registered new index after K-bit flip"
);
}
// Re-register the (now-promoted) session with the
// decrypt worker: cache_key = (transport_id, our_index)
// changed at the flip, so the old worker entry is
// stranded and every packet on the new session would
// miss the worker's HashMap lookup. Without this,
// throughput drops back to the inline-decrypt path
// after each rekey.
// Re-register the promoted session with the decrypt
// worker (cache_key changed at the flip).
#[cfg(unix)]
if did_flip {
self.register_decrypt_worker_session(&node_addr);
@@ -133,7 +130,6 @@ impl Node {
// canonical post-decrypt path, then return — it must
// not fall through to a second decrypt attempt.
let ce_flag = header.flags & FLAG_CE != 0;
let sp_flag = header.flags & FLAG_SP != 0;
self.process_authentic_fmp_plaintext(
&node_addr,
packet.transport_id,
@@ -142,7 +138,6 @@ impl Node {
packet.data.len(),
header.counter,
ce_flag,
sp_flag,
&plaintext,
)
.await;
@@ -171,7 +166,7 @@ impl Node {
#[cfg(unix)]
{
let cache_key = (packet.transport_id, header.receiver_idx.as_u32());
if let Some(workers) = self.decrypt_workers.as_ref().cloned()
if let Some(workers) = self.supervisor.decrypt_workers.as_ref().cloned()
&& self.decrypt_registered_sessions.contains(&cache_key)
{
let job = crate::node::decrypt_worker::DecryptJob {
@@ -195,7 +190,9 @@ impl Node {
// Decrypt: try current session first, then previous (drain fallback)
let ciphertext = &packet.data[header.ciphertext_offset()..];
let plaintext = {
let peer = self.peers.get_mut(&node_addr).unwrap();
let Some(peer) = self.peers.get_mut(&node_addr) else {
return;
};
let session = match peer.noise_session_mut() {
Some(s) => s,
None => {
@@ -259,20 +256,25 @@ impl Node {
};
// MMP per-frame processing and statistics
let now = Instant::now();
let now_ms = crate::time::mono_ms();
let ce_flag = header.flags & FLAG_CE != 0;
let sp_flag = header.flags & FLAG_SP != 0;
if let Some(peer) = self.peers.get_mut(&node_addr) {
// Initiator-side msg3 confirm (see process_authentic_fmp_plaintext):
// a frame authenticated against post-cutover `current` (no pending)
// proves the responder reached the new epoch. Inline-decrypt path
// mirror of the worker-bounce confirm.
if peer.rekey_msg3_payload().is_some() && peer.pending_new_session().is_none() {
peer.clear_rekey_msg3_payload();
}
if let Some(mmp) = peer.mmp_mut() {
mmp.receiver.record_recv(
header.counter,
timestamp,
packet.data.len(),
ce_flag,
now,
now_ms,
);
let _spin_rtt = mmp.spin_bit.rx_observe(sp_flag, header.counter, now);
}
peer.set_current_addr(packet.transport_id, packet.remote_addr.clone());
peer.link_stats_mut()
@@ -328,8 +330,8 @@ impl Node {
/// Canonical post-FMP-decrypt side-effect site. Used by both the
/// inline rx_loop decrypt path and the decrypt-worker bounce path
/// so the per-peer bookkeeping (stats, MMP, spin-bit RTT, ECN
/// propagation, address-rotation handling, link-message dispatch)
/// so the per-peer bookkeeping (stats, MMP, ECN propagation,
/// address-rotation handling, link-message dispatch)
/// happens in exactly one place.
#[allow(clippy::too_many_arguments)]
pub(in crate::node) async fn process_authentic_fmp_plaintext(
@@ -341,7 +343,6 @@ impl Node {
packet_len: usize,
fmp_counter: u64,
ce_flag: bool,
sp_flag: bool,
fmp_plaintext: &[u8],
) {
const INNER_TIMESTAMP_LEN: usize = 4;
@@ -355,18 +356,29 @@ impl Node {
} else {
return;
};
let now = Instant::now();
let now_ms = crate::time::mono_ms();
let mut address_changed = false;
if let Some(peer) = self.peers.get_mut(node_addr) {
peer.reset_decrypt_failures();
// If we are the rekey initiator that already cut over on its
// own timer (no `pending`, `current` is the new session) but
// still retain a msg3 retransmission payload, an authenticated
// peer frame here decrypts against the post-cutover `current`
// session — proof the responder reached the new epoch. Stop
// retransmitting. Mirrors the FSP Current-slot confirm in
// handle_encrypted_session_msg. Works in both the inline and
// worker-bounce paths since both funnel through here, and the
// only session registered for the new index is the new one.
if peer.rekey_msg3_payload().is_some() && peer.pending_new_session().is_none() {
peer.clear_rekey_msg3_payload();
}
address_changed = peer.set_current_addr(transport_id, remote_addr.clone());
peer.link_stats_mut()
.record_recv(packet_len, packet_timestamp_ms);
peer.touch(packet_timestamp_ms);
if let Some(mmp) = peer.mmp_mut() {
mmp.receiver
.record_recv(fmp_counter, inner_ts, packet_len, ce_flag, now);
let _spin_rtt = mmp.spin_bit.rx_observe(sp_flag, fmp_counter, now);
.record_recv(fmp_counter, inner_ts, packet_len, ce_flag, now_ms);
}
}
// Address rotation invalidates the per-peer connect()-ed UDP
@@ -393,7 +405,6 @@ impl Node {
fallback: crate::node::decrypt_worker::DecryptFallback,
) {
let ce_flag = fallback.fmp_flags & FLAG_CE != 0;
let sp_flag = fallback.fmp_flags & FLAG_SP != 0;
let plaintext = &fallback.packet_data[fallback.fmp_plaintext_offset
..fallback.fmp_plaintext_offset + fallback.fmp_plaintext_len];
self.process_authentic_fmp_plaintext(
@@ -404,7 +415,6 @@ impl Node {
fallback.packet_len,
fallback.fmp_counter,
ce_flag,
sp_flag,
plaintext,
)
.await;
@@ -450,7 +460,7 @@ impl Node {
/// black-hole the session.
#[cfg(unix)]
pub(in crate::node) fn register_decrypt_worker_session(&mut self, node_addr: &crate::NodeAddr) {
let Some(workers) = self.decrypt_workers.as_ref().cloned() else {
let Some(workers) = self.supervisor.decrypt_workers.as_ref().cloned() else {
return;
};
let (cache_key, state) = {
@@ -491,7 +501,7 @@ impl Node {
&mut self,
cache_key: (crate::transport::TransportId, u32),
) {
if let Some(workers) = self.decrypt_workers.as_ref() {
if let Some(workers) = self.supervisor.decrypt_workers.as_ref() {
workers.unregister_session(cache_key);
}
self.decrypt_registered_sessions.remove(&cache_key);
@@ -533,7 +543,7 @@ impl Node {
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
self.schedule_reconnect(addr, now_ms);
self.note_link_dead(addr, now_ms);
}
}
}
src/node/handlers/forwarding.rs → src/node/dataplane/forwarding.rs
+215 -150
View File
@@ -1,21 +1,22 @@
//! SessionDatagram forwarding handler.
//!
//! Handles incoming SessionDatagram (0x00) link messages: decodes the
//! envelope, enforces hop limits, performs coordinate cache warming from
//! plaintext session-layer headers, routes to the next hop or delivers
//! locally, and generates error signals on routing failure.
//! envelope, performs coordinate cache warming from plaintext session-layer
//! headers, pre-resolves the next hop for forwardable transit datagrams, and
//! drives the routing core's outcome — local delivery when the datagram is
//! addressed to this node, the transit hop-limit drop, the forward, or the
//! error signal generated on routing failure.
use crate::NodeAddr;
use crate::node::reject::ForwardingReject;
use crate::node::session_wire::{
use crate::node::{Node, NodeError, NodeRoutingView};
use crate::proto::fsp::wire::{
FSP_COMMON_PREFIX_SIZE, FSP_HEADER_SIZE, FSP_PHASE_ESTABLISHED, FSP_PHASE_MSG1, FSP_PHASE_MSG2,
FspCommonPrefix, parse_encrypted_coords,
};
use crate::node::{Node, NodeError};
use crate::protocol::{
CoordsRequired, MtuExceeded, PathBroken, SessionAck, SessionDatagram, SessionDatagramRef,
SessionSetup,
};
use crate::proto::fsp::{SessionAck, SessionSetup};
use crate::proto::link::{SessionDatagram, SessionDatagramRef};
use crate::proto::routing::{DropReason, NextHop, RouteAction, RouteOutcome};
use std::time::{Duration, Instant};
use tracing::{debug, warn};
@@ -43,123 +44,169 @@ impl Node {
}
};
// TTL enforcement: decrement for forwarding and drop only if the
// received datagram was already exhausted.
if datagram_ref.ttl == 0 {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::TtlExhausted, payload.len());
debug!(
src = %datagram_ref.src_addr,
dest = %datagram_ref.dest_addr,
"SessionDatagram TTL exhausted, dropping"
);
return;
}
let forwarded_ttl = datagram_ref.ttl - 1;
let my_addr = *self.node_addr();
// Coordinate cache warming from plaintext session-layer headers
// Coordinate cache warming from plaintext session-layer headers. Runs
// ahead of both the delivery and the TTL decisions the core makes: the
// coords a peer put on the wire are equally valid whichever way those
// go, and the only arrivals this newly warms from are those with an
// exhausted TTL, whose every insert is already achievable at TTL 1.
self.try_warm_coord_cache_ref(&datagram_ref);
// Local delivery: dispatch to session layer handlers without
// materializing an owned SessionDatagram payload Vec.
if datagram_ref.dest_addr == *self.node_addr() {
self.metrics().forwarding.record_delivered(payload.len());
self.handle_session_payload(
&datagram_ref.src_addr,
datagram_ref.payload,
datagram_ref.path_mtu,
incoming_ce,
)
.await;
return;
}
// Pre-resolve the next hop only for datagrams the core can actually
// forward: not locally destined, and carrying a TTL that survives the
// decrement (`ttl > 1` — the shell-side mirror of the core's
// would-leave-zero drop). This keeps `find_next_hop`'s coord-cache
// LRU-touch side effect scoped to genuine forwards, as it was when the
// TTL test ran inline ahead of it. Warming above has already run, so
// the resolution observes freshly cached coords.
let next_hop = if datagram_ref.dest_addr != my_addr && datagram_ref.ttl > 1 {
self.resolve_next_hop(&datagram_ref.dest_addr)
} else {
None
};
let mut datagram = datagram_ref.into_owned();
datagram.ttl = forwarded_ttl;
// Read local congestion once and reuse it for both the CE decision
// (via the view) and the congestion metric/log below, keeping
// `detect_congestion` the single source of truth.
let congested = next_hop
.as_ref()
.map(|nh| self.detect_congestion(&nh.addr))
.unwrap_or(false);
// Find next hop toward destination
let next_hop_addr = match self.find_next_hop(&datagram.dest_addr) {
Some(peer) => *peer.node_addr(),
None => {
// Borrow the routing tables disjointly from `&mut self.routing` for
// the pure decision, then release both before driving the outcome.
let outcome = {
let view = NodeRoutingView {
coord_cache: &self.coord_cache,
peers: &self.peers,
tree_state: &self.tree_state,
congested,
};
self.routing
.route(&datagram_ref, &my_addr, incoming_ce, next_hop, &view)
};
match outcome {
RouteOutcome::Drop {
reason: DropReason::TtlExhausted,
} => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::TtlExhausted, payload.len());
debug!(
src = %datagram_ref.src_addr,
dest = %datagram_ref.dest_addr,
ttl = datagram_ref.ttl,
"SessionDatagram TTL exhausted, dropping"
);
}
RouteOutcome::DeliverLocal => {
// Local delivery: dispatch to session layer handlers without
// materializing an owned SessionDatagram payload Vec.
self.metrics().forwarding.record_delivered(payload.len());
self.handle_session_payload(
&datagram_ref.src_addr,
datagram_ref.payload,
datagram_ref.path_mtu,
incoming_ce,
)
.await;
}
RouteOutcome::NoRoute => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::NoRoute, payload.len());
let original = datagram_ref.into_owned();
debug!(
src = %self.peer_display_name(&datagram.src_addr),
dest = %self.peer_display_name(&datagram.dest_addr),
src = %self.peer_display_name(&original.src_addr),
dest = %self.peer_display_name(&original.dest_addr),
bytes = payload.len(),
"Dropping transit SessionDatagram: no route to destination"
);
self.send_routing_error(&datagram).await;
return;
self.send_routing_error(&original).await;
}
};
RouteOutcome::Forward {
next_hop,
bytes,
outgoing_ce,
} => {
let dest = datagram_ref.dest_addr;
// Apply path_mtu min() from the outgoing link's transport MTU
if let Some(peer) = self.peers.get(&next_hop_addr)
// ECN CE relay: congestion was detected locally above; emit the
// metric and rate-limited log at the transit chokepoint.
if congested {
self.metrics().congestion.congestion_detected.inc();
let now = Instant::now();
let should_log = self
.last_congestion_log
.map(|t| now.duration_since(t) >= Duration::from_secs(5))
.unwrap_or(true);
if should_log {
self.last_congestion_log = Some(now);
debug!(next_hop = %next_hop, "Congestion detected, CE flag set on forwarded packet");
}
}
match self
.send_encrypted_link_message_with_ce(&next_hop, &bytes, outgoing_ce)
.await
{
Err(NodeError::MtuExceeded { mtu, .. }) => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::MtuExceeded, payload.len());
self.send_mtu_exceeded_error(dest, datagram_ref.src_addr, mtu)
.await;
}
Err(e) => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::SendError, payload.len());
debug!(
next_hop = %next_hop,
dest = %dest,
error = %e,
"Failed to forward SessionDatagram"
);
}
Ok(()) => {
self.metrics().forwarding.record_forwarded(bytes.len());
// Classify this transit forward by route class (partition
// of forwarded_packets). Done here, at the data-plane
// chokepoint, so the error-signal routing callers of
// find_next_hop are excluded.
let class = self.classify_forward(&dest, &next_hop);
self.metrics().forwarding.record_route_class(class);
if outgoing_ce {
self.metrics().congestion.ce_forwarded.inc();
}
}
}
}
}
}
/// Resolve the next hop toward `dest` into its address plus the outgoing
/// link's transport MTU. Returns `None` when there is no route.
///
/// The MTU defaults to `u16::MAX` (a no-op min-fold) when the peer's
/// transport is not resolvable, matching the pre-refactor inline behavior
/// where the MTU `if let` chain simply did not fire.
fn resolve_next_hop(&mut self, dest: &NodeAddr) -> Option<NextHop> {
let addr = *self.find_next_hop(dest)?.node_addr();
let link_mtu = if let Some(peer) = self.peers.get(&addr)
&& let Some(tid) = peer.transport_id()
&& let Some(transport) = self.transports.get(&tid)
{
if let Some(addr) = peer.current_addr() {
datagram.path_mtu = datagram.path_mtu.min(transport.link_mtu(addr));
} else {
datagram.path_mtu = datagram.path_mtu.min(transport.mtu());
}
}
// ECN CE relay: propagate incoming CE and detect local congestion
let local_congestion = self.detect_congestion(&next_hop_addr);
let outgoing_ce = incoming_ce || local_congestion;
if local_congestion {
self.metrics().congestion.congestion_detected.inc();
let now = Instant::now();
let should_log = self
.last_congestion_log
.map(|t| now.duration_since(t) >= Duration::from_secs(5))
.unwrap_or(true);
if should_log {
self.last_congestion_log = Some(now);
debug!(next_hop = %next_hop_addr, "Congestion detected, CE flag set on forwarded packet");
}
}
// Forward: re-encode (includes 0x00 type byte) and send
let encoded = datagram.encode();
if let Err(e) = self
.send_encrypted_link_message_with_ce(&next_hop_addr, &encoded, outgoing_ce)
.await
{
match e {
NodeError::MtuExceeded { mtu, .. } => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::MtuExceeded, payload.len());
self.send_mtu_exceeded_error(&datagram, mtu).await;
}
_ => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::SendError, payload.len());
debug!(
next_hop = %next_hop_addr,
dest = %datagram.dest_addr,
error = %e,
"Failed to forward SessionDatagram"
);
}
match peer.current_addr() {
Some(link_addr) => transport.link_mtu(link_addr),
None => transport.mtu(),
}
} else {
self.metrics().forwarding.record_forwarded(encoded.len());
// Classify this transit forward by route class (partition of
// forwarded_packets). Done here, at the data-plane chokepoint, so
// the error-signal routing callers of find_next_hop are excluded.
let class = self.classify_forward(&datagram.dest_addr, &next_hop_addr);
self.metrics().forwarding.record_route_class(class);
if outgoing_ce {
self.metrics().congestion.ce_forwarded.inc();
}
}
u16::MAX
};
Some(NextHop { addr, link_mtu })
}
/// Attempt to warm the coordinate cache from session-layer payload headers.
@@ -260,35 +307,41 @@ impl Node {
/// If we can't route the error back to the source either, drop silently.
/// No cascading errors.
async fn send_routing_error(&mut self, original: &SessionDatagram) {
// Rate limit: one error signal per destination per 100ms
if !self
.routing_error_rate_limiter
.should_send(&original.dest_addr)
{
return;
}
let my_addr = *self.node_addr();
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
let default_ttl = self.config().node.session.default_ttl;
let error_payload =
if let Some(coords) = self.coord_cache().get(&original.dest_addr, now_ms) {
let coords = coords.clone();
PathBroken::new(original.dest_addr, my_addr)
.with_last_coords(coords)
.encode()
} else {
CoordsRequired::new(original.dest_addr, my_addr).encode()
// Pure decision: rate-limit gate + PathBroken/CoordsRequired choice +
// error-PDU encode. Borrow the routing tables disjointly from
// `&mut self.routing`, then release them before the reverse-hop lookup.
let action = {
let view = NodeRoutingView {
coord_cache: &self.coord_cache,
peers: &self.peers,
tree_state: &self.tree_state,
congested: false,
};
self.routing.synth_routing_error(
&original.dest_addr,
&original.src_addr,
&my_addr,
&view,
now_ms,
default_ttl,
)
};
let RouteAction::SendError { toward, bytes } = match action {
Some(action) => action,
// Rate limited: drop silently. No cascading errors.
None => return,
};
let error_dg = SessionDatagram::new(my_addr, original.src_addr, error_payload)
.with_ttl(self.config().node.session.default_ttl);
let next_hop_addr = match self.find_next_hop(&original.src_addr) {
// Resolve the reverse link hop only now, after the gate passed, so
// `find_next_hop`'s coord-cache touch keeps its pre-refactor scope.
let next_hop_addr = match self.find_next_hop(&toward) {
Some(peer) => *peer.node_addr(),
None => {
debug!(
@@ -300,9 +353,8 @@ impl Node {
}
};
let encoded = error_dg.encode();
if let Err(e) = self
.send_encrypted_link_message(&next_hop_addr, &encoded)
.send_encrypted_link_message(&next_hop_addr, &bytes)
.await
{
debug!(
@@ -324,37 +376,50 @@ impl Node {
/// Called when `send_encrypted_link_message()` fails with
/// `NodeError::MtuExceeded` during forwarding. The signal tells the
/// source the bottleneck MTU so it can immediately reduce its path MTU.
async fn send_mtu_exceeded_error(&mut self, original: &SessionDatagram, bottleneck_mtu: u16) {
// Rate limit: reuse routing_error_rate_limiter keyed on dest_addr
if !self
.routing_error_rate_limiter
.should_send(&original.dest_addr)
{
return;
}
///
/// `dest` is the failed datagram's destination (rate-limit key); `toward`
/// is its source, where the signal is routed back.
async fn send_mtu_exceeded_error(
&mut self,
dest: NodeAddr,
toward: NodeAddr,
bottleneck_mtu: u16,
) {
let my_addr = *self.node_addr();
let now_ms = Self::now_ms();
let default_ttl = self.config().node.session.default_ttl;
let error_payload = MtuExceeded::new(original.dest_addr, my_addr, bottleneck_mtu).encode();
// Pure decision: rate-limit gate + MtuExceeded PDU + encode.
let action = self.routing.synth_mtu_exceeded(
&dest,
&toward,
&my_addr,
bottleneck_mtu,
now_ms,
default_ttl,
);
let RouteAction::SendError { toward, bytes } = match action {
Some(action) => action,
// Rate limited: drop silently. No cascading errors.
None => return,
};
let error_dg = SessionDatagram::new(my_addr, original.src_addr, error_payload)
.with_ttl(self.config().node.session.default_ttl);
let next_hop_addr = match self.find_next_hop(&original.src_addr) {
// Resolve the reverse link hop only now, after the gate passed, so
// `find_next_hop`'s coord-cache touch keeps its pre-refactor scope.
let next_hop_addr = match self.find_next_hop(&toward) {
Some(peer) => *peer.node_addr(),
None => {
debug!(
src = %original.src_addr,
dest = %original.dest_addr,
src = %toward,
dest = %dest,
"Cannot route MtuExceeded signal back to source, dropping"
);
return;
}
};
let encoded = error_dg.encode();
if let Err(e) = self
.send_encrypted_link_message(&next_hop_addr, &encoded)
.send_encrypted_link_message(&next_hop_addr, &bytes)
.await
{
debug!(
@@ -364,8 +429,8 @@ impl Node {
);
} else {
debug!(
original_dest = %original.dest_addr,
error_dest = %original.src_addr,
original_dest = %dest,
error_dest = %toward,
bottleneck_mtu,
"Sent MtuExceeded error signal"
);
src/node/dataplane/mod.rs
+18
View File
@@ -0,0 +1,18 @@
//! Data plane: the RX `select!` loop and the per-packet forwarding path.
//!
//! Holds the whole hot path in one home: the `select!` run loop
//! (`rx_loop`), transit/local datagram forwarding (`forwarding`), the
//! link-message router (`dispatch`), the RX decrypt path including responder
//! K-bit cutover and address-roam writes (`encrypted`), and the per-peer
//! connected-UDP fast-path socket activation (`connected_udp`). Each module
//! contributes `impl Node` methods driven by the run loop.
#[cfg(unix)]
pub(crate) mod connected_udp;
mod dispatch;
mod encrypted;
mod forwarding;
mod peer_actions;
mod rx_loop;
pub(in crate::node) use peer_actions::PeerActionCtx;
src/node/dataplane/peer_actions.rs
+829
View File
@@ -0,0 +1,829 @@
//! Executor for the per-peer control machine's [`PeerAction`]s.
//!
//! The per-peer FSM in [`crate::peer::machine`] is a sans-IO reducer: it decides
//! *what* must happen and returns a `Vec<PeerAction>`; this module is the *doing*
//! half — the thin driver that maps each action onto the exact shell call it
//! stands for (`build_msg2` + `transport.send`, `promote_connection`,
//! `remove_active_peer`, `index_allocator.free`, `note_link_dead`, …).
//!
//! ## Progressive cutover
//!
//! The executor is wired incrementally. Live today: the outbound msg2 promote
//! (`handle_msg2` looks up the dial-persisted machine), the connectionless
//! outbound msg1 send (`SendHandshake` with `their_index == None` →
//! `send_stored_msg1`, driven from `initiate_connection`), the
//! connection-oriented dial (`OpenTransport` performs the non-blocking
//! `transport.connect`; `TransportConnected` drives the connect-resolution msg1
//! send from `poll_pending_connects`), the rekey cadence (`check_rekey` →
//! `route_rekey_cadence` → `RekeyConsume`, driving the `SwapSendState` and
//! `CompleteDrain` arms), and the liveness reap (`route_link_dead` →
//! `LinkDeadSuspected`, driving `InvalidateSendState` → `remove_active_peer`).
//!
//! Inbound msg1 is not machine-driven here: `handle_msg1` builds and sends
//! msg2 inline (persisting the leg's machine parked at `SentMsg2` alongside),
//! so `PeerEvent::InboundMsg1` is never dispatched and the `SendHandshake`
//! `their_index == Some` (msg2) branch stays dormant. Inbound msg3 IS
//! machine-driven: `handle_msg3` steps the leg's persistent machine and this
//! executor performs its verdict (`PromoteToActive`, `SwapToInboundSession`,
//! `RekeyRespondTrigger`), disposing the machine on every path that consumes
//! the leg without promoting it.
//!
//! The genuine inert stubs remaining are `SendRekey`, `SendLinkMessage`, and
//! the connected-UDP arms. `RegisterDecryptSession` is a deliberate no-op —
//! see its arm for the note.
//!
//! The timer arms (`SetTimer`/`CancelTimer`) populate/clear the per-peer timer
//! store (`peer_timers`). The `HandshakeRetransmit` and `HandshakeTimeout`
//! deadlines are read and fired by `drive_peer_timers` (the handshake resend +
//! reap home). The rekey/liveness kinds are still SHADOW — driven by their own
//! shell drivers — so populating them stays behavior-neutral.
use crate::PeerIdentity;
use crate::node::reject::{HandshakeReject, RejectReason};
use crate::node::{Node, NodeError};
use crate::peer::machine::{LostKind, PeerAction, PeerEvent};
use crate::proto::fmp::PromotionResult;
use crate::proto::fmp::wire::build_msg2;
use crate::transport::{LinkId, TransportAddr, TransportId};
use crate::utils::index::SessionIndex;
use std::collections::VecDeque;
use tracing::{debug, info, trace, warn};
/// Ambient shell facts a [`PeerAction`] executor needs that the machine's
/// runtime-agnostic action payloads deliberately omit (verified identity,
/// transport target, the msg2 framing indices, the promotion timestamp).
///
/// Unlike a machine event/action payload this is **executor-side**, so it may
/// hold real values resolved from the wire context (cf. `handle_msg3`'s
/// `wire`/`packet` locals and `promote_connection`'s ambient args). It is built
/// fresh per driven step by the caller at cutover time.
#[allow(dead_code)]
pub(in crate::node) struct PeerActionCtx {
/// The authenticated peer identity (`PromoteToActive` / `InvalidateSendState`
/// / `SwapSendState` resolve their `NodeAddr` from this).
pub(in crate::node) verified_identity: PeerIdentity,
/// The transport the exchange is happening over (msg2 send target, decrypt
/// cache-key transport half).
pub(in crate::node) transport_id: TransportId,
/// The peer's wire address (msg2 send target).
pub(in crate::node) remote_addr: TransportAddr,
/// Our session index for this exchange (msg2 framing sender_idx).
pub(in crate::node) our_index: Option<SessionIndex>,
/// The peer's session index for this exchange (msg2 framing receiver_idx).
pub(in crate::node) their_index: Option<SessionIndex>,
/// The wire timestamp driving this step (promotion ts / loss-report clock).
pub(in crate::node) now_ms: u64,
/// Establish direction for this exchange. `false` = inbound, `true` =
/// outbound. `PromoteToActive` reads this to pick the direction-specific
/// promote tail: the outbound branch logs a `Peer promoted to active` line
/// and clears `pending_outbound`, and its promote-Err cleanup is warn-only
/// (no link/index teardown), unlike the inbound branch.
pub(in crate::node) is_outbound: bool,
/// The `pending_outbound` key for an outbound promote, cleared on success.
/// `Some` only on the outbound driven step (the map entry keyed by the wire
/// `receiver_idx`); `None` on the inbound and maintenance paths, which have
/// no `pending_outbound` entry to clear.
pub(in crate::node) pending_outbound_key: Option<(TransportId, u32)>,
}
impl Node {
/// Advance the machine for `link` by one event and execute the resulting
/// actions.
///
/// The borrow structure the whole seam turns on: the machine needs
/// `&mut IndexAllocator` as a synchronous capability *while it is itself
/// borrowed mutably out of `peer_machines`*. `peer_machines` and
/// `index_allocator` are **distinct `Node` fields**, so the collect below is a
/// disjoint two-field borrow the checker accepts; once the actions are
/// collected both borrows drop and the executor runs against `&mut self`.
pub(in crate::node) async fn advance_peer_machine(
&mut self,
link: LinkId,
event: PeerEvent,
now: u64,
ambient: &PeerActionCtx,
) {
let actions = match self.peer_machines.get_mut(&link) {
// Disjoint field borrow: `self.peer_machines` (the map entry) and
// `self.index_allocator` (the capability) are separate fields.
Some(machine) => machine.step(event, now, &mut self.index_allocator),
None => return,
};
self.execute_peer_actions(link, ambient, actions).await;
}
/// Map each [`PeerAction`] onto its shell call.
///
/// The worklist is a `VecDeque` rather than self-recursion so the async
/// executor stays a single flat future (no boxing) and the emitted order is
/// preserved. `PromoteToActive` (deferred) will feed its resolution back into
/// the machine and fold follow-up actions onto this same queue.
pub(in crate::node) async fn execute_peer_actions(
&mut self,
link: LinkId,
ambient: &PeerActionCtx,
actions: Vec<PeerAction>,
) {
let mut queue: VecDeque<PeerAction> = actions.into();
while let Some(action) = queue.pop_front() {
match action {
PeerAction::OpenTransport {
transport_id,
remote_addr,
} => {
// Outbound connection-oriented dial. `initiate_connection`'s
// oriented branch drove the machine to `Connecting`, which
// emitted this action. Perform the non-blocking
// `transport.connect` and, on success, push the
// `PendingConnect` for `poll_pending_connects` to resolve. On
// connect error, tear down the dial-window state (link,
// reverse map, control machine) and abort the queue — the
// executor-local mirror of the old inline
// `initiate_connection` connect+push.
if let Some(transport) = self.transports.get(&transport_id) {
match transport.connect(&remote_addr).await {
Ok(()) => {
debug!(
transport_id = %transport_id,
remote_addr = %remote_addr,
link_id = %link,
"Transport connect initiated (non-blocking)"
);
self.peering
.pending_connects
.push(crate::node::PendingConnect {
link_id: link,
transport_id,
remote_addr,
peer_identity: Some(ambient.verified_identity),
});
}
Err(_e) => {
self.links.remove(&link);
self.addr_to_link.remove(&(transport_id, remote_addr));
self.remove_peer_machine(link);
return;
}
}
}
}
PeerAction::SendHandshake { bytes } => {
// Two outbound directions share this action, discriminated by
// `their_index`:
// msg2 (`their_index == Some`): the machine payload is the
// UNFRAMED Noise msg2; frame it with our/their index
// (`build_msg2`) and send.
// msg1 (`their_index == None`): a fresh outbound handshake;
// the machine's empty payload is ignored — the shell already
// allocated the index, ran the Noise leaf, and armed the
// wire at dial (`prepare_outbound_msg1`); this just sends the
// stored wire (see `send_stored_msg1`).
if let (Some(sender_idx), Some(receiver_idx)) =
(ambient.our_index, ambient.their_index)
{
let frame = build_msg2(sender_idx, receiver_idx, &bytes);
// Surface the send Result. A missing transport skips
// the send and continues (mirrors `handle_msg1`'s
// `if let Some(transport)` guard); a send *error* runs the
// pre-refactor msg2-send-failure cleanup (`handle_msg1`
// L494-503) and ABORTS the remaining queue so the queued
// `PromoteToActive` never runs.
let send_err = match self.transports.get(&ambient.transport_id) {
Some(transport) => {
transport.send(&ambient.remote_addr, &frame).await.err()
}
None => None,
};
if let Some(e) = send_err {
// Restored pre-refactor msg2-send-failure warn!
// (`handle_msg1` L665): the send error text is surfaced
// at the executor point where the failure is now handled.
warn!(link_id = %link, error = %e, "Failed to send msg2");
self.links.remove(&link);
self.addr_to_link
.remove(&(ambient.transport_id, ambient.remote_addr.clone()));
if let Some(idx) = ambient.our_index {
let _ = self.index_allocator.free(idx);
}
self.remove_peer_machine(link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
return;
}
} else {
// msg1: the shell already allocated the index, ran the
// Noise leaf, and armed the wire on the connection at dial
// (`prepare_outbound_msg1`); send the stored wire. The
// machine's empty payload is ignored.
let _ = bytes;
self.send_stored_msg1(
link,
ambient.transport_id,
&ambient.remote_addr,
ambient.now_ms,
)
.await;
}
}
PeerAction::SendRekey { .. } => {
// Rekey msg framing (`build_msg2(our_new_index, …)`) + send.
// Rekey fold is out of scope here.
}
PeerAction::SendLinkMessage { .. } => {
// Encrypt + send a link-control frame (heartbeat / filter /
// tree / disconnect). Data-plane-owned; out of scope here.
}
PeerAction::PromoteToActive { link: promote_link } => {
// Establish promote, driven through the machine. Transcribes
// `handle_msg3`'s shared inbound promote block verbatim, adapted
// to the executor's ambient context. One XX-specific choice vs
// the IK-lineage executor: the decrypt-worker register stays
// INSIDE `promote_connection` (NOT relocated here) — re-
// registering would double-register. The promoted leg's machine
// (msg1-born inbound, dial-born outbound) survives the promotion
// and is crystallized in place by the `PromotionResolved`
// feedback fed back after the Ok handling below.
// Capture msg2 BEFORE `promote_connection` removes the pending
// connection, so a duplicate msg1 can be answered with it. Only
// the inbound promote answers a duplicate inbound msg1; the
// outbound side has no stored msg2 to resend.
let wire_msg2 = if ambient.is_outbound {
None
} else {
self.peer_machines
.get(&promote_link)
.and_then(|m| m.conn_handshake_msg2().map(|b| b.to_vec()))
};
if ambient.is_outbound {
debug!(
// Relocated from `handlers/handshake.rs`: pin the target
// so it stays visible under the harness's
// `fips::node::handlers::handshake=debug` filter.
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(ambient.verified_identity.node_addr()),
link_id = %promote_link,
"handle_msg2: promoting outbound, peers_has_key={}",
self.peers.contains_key(ambient.verified_identity.node_addr()),
);
} else {
debug!(
// Relocated from `handlers/handshake.rs`: pin the target
// so it stays visible under the harness's
// `fips::node::handlers::handshake=debug` filter.
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(ambient.verified_identity.node_addr()),
link_id = %promote_link,
our_index = ?ambient.our_index,
"handle_msg3: promoting inbound, peers_has_key={}",
self.peers.contains_key(ambient.verified_identity.node_addr()),
);
}
let promote_result = self.promote_connection(
promote_link,
ambient.verified_identity,
ambient.now_ms,
);
match &promote_result {
Ok(PromotionResult::Promoted(node_addr)) => {
let node_addr = *node_addr;
if ambient.is_outbound {
// The outbound promote logs a second line here in
// addition to `promote_connection`'s "Connection
// promoted to active peer". Pin the target so the
// relocated line keeps the module it filtered under.
info!(
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(&node_addr),
"Peer promoted to active"
);
} else {
// Store msg2 on peer for resend on duplicate msg1
if let (Some(peer), Some(msg2)) =
(self.peers.get_mut(&node_addr), wire_msg2)
{
peer.set_handshake_msg2(msg2);
}
// Promotion is logged once by `promote_connection`
// ("Connection promoted to active peer"); no separate
// inbound-path line.
}
// Send initial tree announce to new peer
if let Err(e) = self.send_tree_announce_to_peer(&node_addr).await {
debug!(peer = %self.peer_display_name(&node_addr), error = %e, "Failed to send initial TreeAnnounce");
}
// Schedule filter announce (sent on next tick via debounce)
self.bloom_state.mark_update_needed(node_addr);
self.reset_lookup_backoff();
// Clear the pending outbound entry on promote success
// only; a failed promote leaves it for the stale-
// connection reaper.
if let Some(k) = ambient.pending_outbound_key {
self.pending_outbound.remove(&k);
}
}
Ok(PromotionResult::CrossConnectionWon {
loser_link_id,
node_addr,
}) => {
let (loser_link_id, node_addr) = (*loser_link_id, *node_addr);
// UNREACHABLE on driven XX establish paths: `Promote`
// and `RestartThenPromote` (which removes the old peer
// first) both imply no existing peer at promote time, so
// `promote_connection` returns `Promoted`. Body kept
// byte-equivalent to next so a future path that drives a
// cross-connection through the executor trips the assert.
debug_assert!(
false,
"executor CrossConnectionWon is unreachable on driven \
XX inbound establish paths"
);
// Store msg2 on peer for resend on duplicate msg1
if let (Some(peer), Some(msg2)) =
(self.peers.get_mut(&node_addr), wire_msg2)
{
peer.set_handshake_msg2(msg2);
}
// Close the losing TCP connection (no-op for connectionless)
if let Some(loser_link) = self.links.get(&loser_link_id) {
let loser_tid = loser_link.transport_id();
let loser_addr = loser_link.remote_addr().clone();
if let Some(transport) = self.transports.get(&loser_tid) {
transport.close_connection(&loser_addr).await;
}
}
// Clean up the losing connection's link
self.remove_link(&loser_link_id);
debug!(
peer = %self.peer_display_name(&node_addr),
loser_link_id = %loser_link_id,
"Inbound cross-connection won, loser link cleaned up"
);
if let Err(e) = self.send_tree_announce_to_peer(&node_addr).await {
debug!(peer = %self.peer_display_name(&node_addr), error = %e, "Failed to send initial TreeAnnounce");
}
self.bloom_state.mark_update_needed(node_addr);
self.reset_lookup_backoff();
}
Ok(PromotionResult::CrossConnectionLost { winner_link_id }) => {
let winner_link_id = *winner_link_id;
// UNREACHABLE on driven XX establish paths (see the Won
// arm). Body kept byte-equivalent to next; uses the
// ambient transport/addr in place of next's `packet.*`.
debug_assert!(
false,
"executor CrossConnectionLost is unreachable on driven \
XX inbound establish paths"
);
// Close the losing TCP connection (no-op for connectionless)
if let Some(transport) = self.transports.get(&ambient.transport_id) {
transport.close_connection(&ambient.remote_addr).await;
}
// This connection lost — clean up its link
self.remove_link(&promote_link);
// Restore addr_to_link for the winner's link
self.addr_to_link.insert(
(ambient.transport_id, ambient.remote_addr.clone()),
winner_link_id,
);
debug!(
winner_link_id = %winner_link_id,
"Inbound cross-connection lost, keeping existing"
);
}
Err(e) if ambient.is_outbound => {
// The outbound promote-failure path is warn-only: it
// records the reject but performs no link/index teardown
// and leaves the `pending_outbound` entry for the stale-
// connection reaper. The leg's machine must go with the
// leg, though: `promote_connection` took the pending
// connection off the machine before erring, so the reaper
// (which sweeps the machines' embedded legs) can never
// reach this link's machine — dropping it here is the
// only disposal point.
warn!(
target: "fips::node::handlers::handshake",
link_id = %promote_link,
error = %e,
"Failed to promote connection"
);
self.remove_peer_machine(promote_link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
}
Err(e) => {
// A max_peers rejection is expected policy, not a fault —
// log it at debug to avoid WARN spam when a cap'd node is
// under sustained inbound pressure. Other promotion
// failures remain at warn.
if matches!(e, NodeError::MaxPeersExceeded { .. }) {
debug!(
// Emit under the handshake target (same as the
// "promoting inbound" line above) so this stays
// visible wherever inbound handshake events are
// logged at debug, independent of this module's
// own log level.
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(ambient.verified_identity.node_addr()),
max = self.max_peers(),
"Rejecting inbound connection at max_peers cap (no promotion)"
);
} else {
warn!(
target: "fips::node::handlers::handshake",
link_id = %promote_link,
error = %e,
"Failed to promote inbound connection"
);
}
// Clean up on promotion failure. promote_connection
// already freed our_index in its MaxPeersExceeded path;
// freeing again here is benign (IndexAllocator::free is a
// HashSet::remove, the second call returns Err(NotFound)
// and is ignored).
self.remove_link(&promote_link);
if let Some(idx) = ambient.our_index {
let _ = self.index_allocator.free(idx);
}
self.remove_peer_machine(promote_link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
}
}
// Feed the promotion outcome back into the surviving machine
// and fold the follow-up actions onto the worklist (the
// `Promoted` arm crystallizes the machine in place and emits
// `RegisterDecryptSession`, a redundant no-op here — see its
// arm). Unconditional across Ok variants; on
// `CrossConnectionLost` the losing leg's machine was disposed
// inside `promote_connection`, so the lookup misses and no
// machine-side index free can double the inline one. Disjoint
// field borrow again.
if let Ok(result) = promote_result {
let follow = match self.peer_machines.get_mut(&promote_link) {
Some(machine) => machine.step(
PeerEvent::PromotionResolved { result },
ambient.now_ms,
&mut self.index_allocator,
),
None => Vec::new(),
};
queue.extend(follow);
}
}
PeerAction::ResolveCrossConnection { .. } => {
// A decision token, not an effect: the outbound msg2
// handler intercepts it and runs the inline swap/keep
// resolution itself, so it must never reach the executor.
debug_assert!(
false,
"ResolveCrossConnection is intercepted by the msg2 \
handler and must never reach the executor"
);
}
PeerAction::SwapSendState { .. } => {
// Initiator cutover: the live authoritative rekey-cadence
// path, routed here from `check_rekey` via
// `route_rekey_cadence` → `PeerEvent::RekeyConsume`; the
// inline body survives only as `cutover_peer_inline`, a
// debug-assert release fallback. `addr` is resolved
// from the ambient verified identity (as `InvalidateSendState`
// does). The decrypt re-register folds HERE, gated on
// `did_cutover` — the generic `RegisterDecryptSession` arm stays a
// no-op so a promote never double-registers.
let node_addr = *ambient.verified_identity.node_addr();
let did_cutover = if let Some(peer) = self.peers.get_mut(&node_addr) {
if let Some(_old_our_index) = peer.cutover_to_new_session() {
// New index was pre-registered in peers_by_index
// during msg2 handling (handshake.rs).
debug_assert!(
peer.transport_id().is_some()
&& peer.our_index().is_some()
&& self.peers_by_index.contains_key(&(
peer.transport_id().unwrap(),
peer.our_index().unwrap().as_u32()
)),
"peers_by_index should contain pre-registered new index after cutover"
);
let our_index = peer.our_index();
let their_index = peer.their_index();
info!(
// Pin the target to the pre-refactor module: this
// cutover log relocated from handlers/rekey.rs into
// the executor, but operators (and the test harness)
// filter it under fips::node::handlers::rekey.
// Keeping the target preserves the observable log
// contract (level + fields match next's rekey.rs).
target: "fips::node::handlers::rekey",
peer = %self.peer_display_name(&node_addr),
our_addr = %self.identity().node_addr(),
their_addr = %node_addr,
our_index = ?our_index,
their_index = ?their_index,
"Rekey cutover complete (initiator), K-bit flipped"
);
true
} else {
false
}
} else {
false
};
// Re-register the new session with the decrypt worker — the
// cache_key (transport_id, our_index) just changed, so the old
// worker entry is stale and every packet on the new session
// would miss the worker's HashMap lookup.
#[cfg(unix)]
if did_cutover {
self.register_decrypt_worker_session(&node_addr);
}
#[cfg(not(unix))]
let _ = did_cutover;
}
PeerAction::CompleteDrain { peer: node_addr } => {
// Drain completion: the live authoritative rekey-cadence
// path, routed here from `check_rekey` via
// `route_rekey_cadence` → `PeerEvent::RekeyConsume`; the
// inline body survives only as `drain_peer_inline`, a
// debug-assert release fallback. Reached in either rekey
// role — a responder's cutover demotes a session the same
// way an initiator's does — including on a node whose own
// trigger is off. Extract the real previous
// index + transport_id under the peer borrow, drop the
// borrow, then run the cache_key cleanup (which takes
// &mut self for unregister_decrypt_worker_session).
let drained = self.peers.get_mut(&node_addr).and_then(|peer| {
peer.complete_drain().map(|idx| (idx, peer.transport_id()))
});
if let Some((old_our_index, transport_id)) = drained {
if let Some(tid) = transport_id {
let cache_key = (tid, old_our_index.as_u32());
self.peers_by_index.remove(&cache_key);
#[cfg(unix)]
self.unregister_decrypt_worker_session(cache_key);
}
let _ = self.index_allocator.free(old_our_index);
trace!(
// Pin to the pre-refactor module (see the cutover log
// above) so the relocated drain log stays visible under
// the operator's fips::node::handlers::rekey filter.
target: "fips::node::handlers::rekey",
peer = %self.peer_display_name(&node_addr),
old_index = %old_our_index,
"Drain complete, previous session erased"
);
}
}
PeerAction::InvalidateSendState => {
// The FULL teardown. `remove_active_peer` frees the four index
// slots (current/rekey/pending/previous), drops
// `peers_by_index`, unregisters the decrypt worker, removes the
// FSP `sessions` entry and `pending_tun_packets`. The machine
// emits NO `FreeIndex` for those slots, so there is no
// double-free.
self.remove_active_peer(ambient.verified_identity.node_addr());
}
PeerAction::SwapToInboundSession {
peer,
our_index,
our_inbound_wins,
} => {
// Simultaneous-init cross-connection resolved at msg3 (msg2-then-
// msg3 ordering): apply the same tie-breaker the inverse ordering
// uses so both sides converge on a single Noise session pair.
let their_index = ambient
.their_index
.expect("cross-connection swap carries the peer session index");
if our_inbound_wins {
// Larger node side: swap to the inbound session so it pairs
// with the peer's kept outbound session.
let inbound_session = match self
.peer_machines
.get_mut(&link)
.and_then(|m| m.take_session())
{
Some(s) => s,
None => {
self.remove_link(&link);
self.remove_peer_machine(link);
self.stats_mut().record_reject(RejectReason::Handshake(
HandshakeReject::BadState,
));
return;
}
};
// A rekey of ours in flight is now stale: the session it
// was negotiated against is about to be replaced, and
// `replace_session` rewrites the session and both indices
// while touching no rekey state. Abandoning here is what
// lets the classifier take this arm unconditionally —
// declining the swap instead would desynchronize us from
// the peer's outbound half, which cannot see our rekey.
// Same clearing the rekey-responder arm does on its
// `abandon_first` path.
if let Some(peer_ref) = self.peers.get_mut(&peer)
&& let Some(idx) = peer_ref.abandon_rekey()
{
if let Some(tid) = peer_ref.transport_id() {
self.peers_by_index.remove(&(tid, idx.as_u32()));
self.pending_outbound.remove(&(tid, idx.as_u32()));
}
let _ = self.index_allocator.free(idx);
}
if let Some(peer_ref) = self.peers.get_mut(&peer) {
let old_our_index =
peer_ref.replace_session(inbound_session, our_index, their_index);
let Some(transport_id) = peer_ref.transport_id() else {
self.remove_link(&link);
self.remove_peer_machine(link);
self.stats_mut().record_reject(RejectReason::Handshake(
HandshakeReject::BadState,
));
return;
};
if let Some(old_idx) = old_our_index {
self.peers_by_index
.remove(&(transport_id, old_idx.as_u32()));
let _ = self.index_allocator.free(old_idx);
}
self.peers_by_index
.insert((transport_id, our_index.as_u32()), peer);
debug!(
peer = %self.peer_display_name(&peer),
new_our_index = %our_index,
new_their_index = %their_index,
"Simultaneous-init (msg3): swapped to inbound session (our inbound wins)"
);
}
} else {
// Smaller node side: keep the existing outbound session, drop
// the inbound leg's allocated index.
let _ = self.index_allocator.free(our_index);
debug!(
peer = %self.peer_display_name(&peer),
"Simultaneous-init (msg3): keeping outbound session (our outbound wins)"
);
}
// Both branches tear down the temporary inbound link fully
// (including its `addr_to_link` mapping) via `remove_link`,
// disposing the leg's machine (and its embedded connection)
// with it.
self.remove_link(&link);
self.remove_peer_machine(link);
return;
}
PeerAction::RekeyRespondTrigger {
peer,
our_index,
abandon_first,
} => {
// Rekey-responder resolved at msg3: store the new session as
// pending on the existing peer, awaiting the K-bit cutover.
let their_index = ambient
.their_index
.expect("rekey-responder trigger carries the peer session index");
if abandon_first {
// We lose the dual-rekey tie-break (larger addr): abandon our
// own rekey/pending and fall through as responder.
// `abandon_rekey` clears both the in-progress flag and any
// pending session state, returning whichever index needs
// freeing.
info!(
peer = %self.peer_display_name(&peer),
our_addr = %self.identity().node_addr(),
their_addr = %peer,
"rekey-msg3 tie-break: we lose (larger addr), abandon ours"
);
if let Some(peer_ref) = self.peers.get_mut(&peer)
&& let Some(idx) = peer_ref.abandon_rekey()
{
if let Some(tid) = peer_ref.transport_id() {
self.peers_by_index.remove(&(tid, idx.as_u32()));
self.pending_outbound.remove(&(tid, idx.as_u32()));
}
let _ = self.index_allocator.free(idx);
}
}
// Rekey: process as responder, store new session as pending.
let noise_session = {
let Some(machine) = self.peer_machines.get_mut(&link) else {
warn!(link_id = %link, "Connection removed during rekey msg3 processing");
self.links.remove(&link);
self.remove_peer_machine(link);
self.stats_mut().record_reject(RejectReason::Handshake(
HandshakeReject::UnknownConnection,
));
return;
};
machine.take_session()
};
let our_new_index = our_index;
let noise_session = match noise_session {
Some(s) => s,
None => {
warn!("Rekey msg3: no session from handshake");
self.links.remove(&link);
self.remove_peer_machine(link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
return;
}
};
// Store pending session on the existing peer
if let Some(peer_ref) = self.peers.get_mut(&peer) {
peer_ref.set_pending_session(noise_session, our_new_index, their_index);
peer_ref.record_peer_rekey();
}
// Register new index in peers_by_index
self.peers_by_index
.insert((ambient.transport_id, our_new_index.as_u32()), peer);
// Clean up: remove the temporary link and the leg's machine
// (dropping its embedded connection; the established peer
// keeps its own machine, keyed by its own link). Do NOT
// remove addr_to_link — the entry must remain pointing to
// the original link so the established peer stays routable,
// so this uses the bare `links.remove` rather than the full
// `remove_link`.
self.links.remove(&link);
self.remove_peer_machine(link);
debug!(
peer = %self.peer_display_name(&peer),
our_addr = %self.identity().node_addr(),
new_our_index = %our_new_index,
new_their_index = %their_index,
"rekey-msg3 responder: pending session set, awaiting K-bit cutover"
);
return;
}
PeerAction::RegisterDecryptSession { index } => {
let _ = index;
// No-op by design. The rekey-cutover decrypt-worker register
// relocates into the driven `SwapSendState` site above (gated
// on `did_cutover`); the establish-promote register stays INSIDE
// `promote_connection`, so `PromoteToActive` does not re-register
// either. This machine-emitted action is redundant with both;
// kept as an inert no-op (rather than removing the emission) so
// the machine's action sequence and unit tests stay unchanged.
// The keyed-by-NodeAddr register does not need the machine's
// `index` payload.
}
PeerAction::UnregisterDecryptSession { index } => {
// Executor supplies `transport_id` from ambient; keyed by
// (tid, index) like `remove_active_peer` / the rekey drain path.
#[cfg(unix)]
self.unregister_decrypt_worker_session((ambient.transport_id, index.as_u32()));
#[cfg(not(unix))]
let _ = index;
}
PeerAction::FreeIndex { index } => {
let _ = self.index_allocator.free(index);
}
PeerAction::ActivateConnectedUdp | PeerAction::TeardownConnectedUdp => {
// Connected-UDP plane ownership (`connected_udp.rs`). Out of
// scope for now.
}
PeerAction::SetTimer { kind, at_ms } => {
// Populate the per-peer timer store (overwrite = reschedule).
// The `HandshakeRetransmit` and `HandshakeTimeout` deadlines
// are read + fired by `drive_peer_timers`. Rekey/liveness kinds
// are still SHADOW here — they keep their own shell drivers —
// so populating them stays behavior-neutral.
self.peer_timers
.entry(link)
.or_default()
.insert(kind, at_ms);
}
PeerAction::CancelTimer { kind } => {
if let Some(timers) = self.peer_timers.get_mut(&link) {
timers.remove(&kind);
}
}
PeerAction::ReportLost { peer, kind } => {
// The single loss token, routed to the reconciler reflex the
// `kind` names: an un-promoted handshake attempt takes the
// connected-guarded `note_handshake_timeout` (`driver.rs:28`),
// an established peer's link-death takes the unconditional
// `note_link_dead` (`driver.rs:48`).
match kind {
LostKind::HandshakeTimeout => {
self.note_handshake_timeout(peer, ambient.now_ms);
}
LostKind::LinkDead => {
self.note_link_dead(peer, ambient.now_ms);
}
}
}
}
}
}
}
src/node/handlers/rx_loop.rs → src/node/dataplane/rx_loop.rs
+200 -46
View File
@@ -1,10 +1,11 @@
//! RX event loop and packet dispatch.
use crate::control::{ControlSocket, commands};
use crate::node::wire::{
COMMON_PREFIX_SIZE, CommonPrefix, FMP_VERSION, PHASE_ESTABLISHED, PHASE_MSG1, PHASE_MSG2,
};
use crate::node::{Node, NodeError};
use crate::proto::fmp::wire::{
COMMON_PREFIX_SIZE, CommonPrefix, FMP_VERSION, PHASE_ESTABLISHED, PHASE_MSG1, PHASE_MSG2,
PHASE_MSG3,
};
use crate::transport::ReceivedPacket;
use std::time::Duration;
use tracing::{debug, info, warn};
@@ -29,6 +30,7 @@ impl Node {
/// - Phase 0x0: Encrypted frame (session data)
/// - Phase 0x1: Handshake message 1 (initiator -> responder)
/// - Phase 0x2: Handshake message 2 (responder -> initiator)
/// - Phase 0x3: Handshake message 3 (initiator -> responder, XX completion)
///
/// Also processes outbound IPv6 packets from the TUN reader for session
/// encapsulation and routing through the mesh.
@@ -42,12 +44,42 @@ impl Node {
/// This method takes ownership of the packet_rx channel and runs
/// until the channel is closed (typically when stop() is called).
pub async fn run_rx_loop(&mut self) -> Result<(), NodeError> {
// No shutdown observer → today's infinite loop, byte-identical. All
// existing callers/tests use this; `pending()` never fires, so the
// shutdown/deadline arms below stay permanently disabled.
self.run_rx_loop_with_shutdown(std::future::pending()).await
}
/// The rx event loop, which serves until `shutdown` fires and then drains
/// **in place** before returning.
///
/// The channel receivers are moved into this frame's locals and live across
/// both serve and drain, so — unlike a `select!`-cancelled loop — they are
/// never destructively dropped mid-flight; they are released only on clean
/// exit, after which teardown does not need them.
///
/// - While serving (`drain_deadline == None`) the loop is behaviorally
/// identical to before: the shutdown arm, the deadline arm, and the
/// peers-empty early-exit are all guarded off, so the hot per-packet path
/// and the `biased` order of the real arms are unchanged.
/// - When `shutdown` fires, the loop calls [`Node::enter_drain`] once
/// (broadcast Disconnect, gate the reconciler off) and arms the bounded
/// deadline, then keeps servicing inbound/tick/peer-removal until all
/// peers clear or the deadline elapses, then returns. The caller
/// ([`Node::finish_shutdown`]) closes the window and tears down.
pub async fn run_rx_loop_with_shutdown(
&mut self,
shutdown: impl std::future::Future<Output = ()>,
) -> Result<(), NodeError> {
tokio::pin!(shutdown);
// `None` = serving; `Some(deadline)` = draining (bounded window).
let mut drain_deadline: Option<tokio::time::Instant> = None;
let mut packet_rx = self.packet_rx.take().ok_or(NodeError::NotStarted)?;
// Take the TUN outbound receiver, or create a dummy channel that never
// produces messages (when TUN is disabled). Holding the sender prevents
// the channel from closing.
let (mut tun_outbound_rx, _tun_guard) = match self.tun_outbound_rx.take() {
let (mut tun_outbound_rx, _tun_guard) = match self.supervisor.tun_outbound_rx.take() {
Some(rx) => (rx, None),
None => {
let (tx, rx) = tokio::sync::mpsc::channel(1);
@@ -57,7 +89,7 @@ impl Node {
// Take the DNS identity receiver, or create a dummy channel (when DNS
// is disabled). Same pattern as TUN outbound.
let (mut dns_identity_rx, _dns_guard) = match self.dns_identity_rx.take() {
let (mut dns_identity_rx, _dns_guard) = match self.supervisor.dns_identity_rx.take() {
Some(rx) => (rx, None),
None => {
let (tx, rx) = tokio::sync::mpsc::channel(1);
@@ -65,8 +97,20 @@ impl Node {
}
};
let mut tick =
tokio::time::interval(Duration::from_secs(self.config().node.tick_interval_secs));
// Take the runtime child-liveness receiver, or a dummy channel (when the
// node was seeded straight into Running without a start()). Holding the
// dummy sender in the guard keeps the channel open. Same pattern as TUN
// outbound / DNS identity.
let (mut child_exit_rx, _child_exit_guard) = match self.child_exit_rx.take() {
Some(rx) => (rx, None),
None => {
let (tx, rx) = tokio::sync::mpsc::channel(1);
(rx, Some(tx))
}
};
let tick_period = Duration::from_secs(self.config().node.tick_interval_secs);
let mut tick = tokio::time::interval(tick_period);
// Set up control socket channel
let (control_tx, mut control_rx) =
@@ -122,6 +166,13 @@ impl Node {
crate::perf_profile::maybe_spawn_reporter();
loop {
// Bounded drain mode: break as soon as all peers have cleared. In
// normal mode (`None`) this short-circuits before touching
// `self.peers`, so the loop is byte-identical.
if drain_deadline.is_some() && self.peers.is_empty() {
info!("Drain complete: all peers cleared, ending drain loop");
break;
}
tokio::select! {
biased;
// Decrypt-worker fallback drains FIRST. Under sustained
@@ -214,6 +265,27 @@ impl Node {
}
}
}
// Runtime child-liveness. Placed AFTER `packet_rx` so the hot
// inbound path keeps its `biased` priority. A directly-observable
// child (TUN threads, DNS/mDNS/Nostr) exited on its own; feed the
// FSM, which republishes health (Degraded here — a Running node
// always has ≥1 transport up). `on_child_exited` only ever emits
// `PublishState`; other variants are ignored defensively.
maybe_child = child_exit_rx.recv() => {
if let Some(child) = maybe_child {
let actions = self
.supervisor
.fsm
.step(crate::node::lifecycle::supervisor::Event::ChildExited { child });
for action in actions {
if let crate::node::lifecycle::supervisor::Action::PublishState(ns) =
action
{
self.supervisor.state = ns;
}
}
}
}
Some(ipv6_packet) = tun_outbound_rx.recv() => {
self.handle_tun_outbound(ipv6_packet).await;
let mut drained = 0;
@@ -249,41 +321,116 @@ impl Node {
).await;
let _ = response_tx.send(response);
}
_ = tick.tick() => {
self.check_timeouts();
let now_ms = Self::now_ms();
self.reload_peer_acl().await;
// The host map hot-reloads on the same tick as the ACL. It
// is polled separately from `reload_peer_acl` because the
// ACL's embedded alias reloader and this snapshot are
// distinct resources; the `path_mtu_lookup` cache and the
// `nostr_discovery` subsystem are deliberately excluded
// from `Reloadable` since neither reloads from a backing
// file (see `node::reloadable`).
self.reload_host_map().await;
self.poll_pending_connects().await;
self.poll_nostr_discovery().await;
self.poll_lan_discovery().await;
self.resend_pending_handshakes(now_ms).await;
self.resend_pending_rekeys(now_ms).await;
self.resend_pending_session_handshakes(now_ms).await;
self.resend_pending_session_msg3(now_ms).await;
self.purge_idle_sessions(now_ms);
self.process_pending_retries(now_ms).await;
self.check_tree_state().await;
self.check_bloom_state().await;
self.compute_mesh_size();
self.record_stats_history();
self.check_mmp_reports().await;
self.check_session_mmp_reports().await;
self.check_link_heartbeats().await;
self.check_rekey().await;
self.check_session_rekey().await;
self.check_pending_lookups(now_ms).await;
self.poll_transport_discovery().await;
self.sample_transport_congestion();
#[cfg(any(target_os = "linux", target_os = "macos"))]
self.activate_connected_udp_sessions().await;
deadline = tick.tick() => {
// Tick-body instrumentation. The gate is read ONCE per tick
// into `instr_on`, which is then passed explicitly to every
// `instr_step!` invocation — macro hygiene makes a call-site
// local invisible inside the macro body. With the
// `profiling` feature off, `gate()` is a `const fn`
// returning false and the macro is a pure pass-through, so
// the whole arm compiles to the uninstrumented sequence.
//
// `tick_entry` records how late this entry is against the
// deadline the interval scheduled it for. That is the
// measurement this instrumentation exists for: the arm is
// polled LAST under `biased;`, so the
// lateness IS the time it spent waiting behind the packet,
// TUN and control arms. `tick()` hands back its scheduled
// deadline, so this is a subtraction rather than a model.
// The whole-tick span below measures the body alone.
let instr_on = crate::instr::gate();
crate::instr::tick_entry(instr_on, deadline.into_std(), std::time::Instant::now());
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::WholeTick, {
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckTimeouts,
self.check_timeouts());
let now_ms = Self::now_ms();
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ReloadPeerAcl,
self.reload_peer_acl().await);
// The host map hot-reloads on the same tick as the ACL. It
// is polled separately from `reload_peer_acl` because the
// ACL's embedded alias reloader and this snapshot are
// distinct resources; the `path_mtu_lookup` cache and the
// `nostr_rendezvous` subsystem are deliberately excluded
// from `Reloadable` since neither reloads from a backing
// file (see `node::reloadable`).
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ReloadHostMap,
self.reload_host_map().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollPendingConnects,
self.poll_pending_connects().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollNostrRendezvous,
self.poll_nostr_rendezvous().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollLanRendezvous,
self.poll_lan_rendezvous().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::DrivePeerTimers,
self.drive_peer_timers(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingRekeys,
self.resend_pending_rekeys(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingFmpRekeyMsg3,
self.resend_pending_fmp_rekey_msg3(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingSessionHandshakes,
self.resend_pending_session_handshakes(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingSessionMsg3,
self.resend_pending_session_msg3(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PurgeIdleSessions,
self.purge_idle_sessions(now_ms));
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ProcessPendingRetries,
self.process_pending_retries(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckTreeState,
self.check_tree_state().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckBloomState,
self.check_bloom_state().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ComputeMeshSize,
self.compute_mesh_size());
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::RecordStatsHistory,
self.record_stats_history());
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckMmpReports,
self.check_mmp_reports().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckSessionMmpReports,
self.check_session_mmp_reports().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckLinkHeartbeats,
self.check_link_heartbeats().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckRekey,
self.check_rekey().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckSessionRekey,
self.check_session_rekey().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckPendingLookups,
self.check_pending_lookups(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollTransportDiscovery,
self.poll_transport_discovery().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::SampleTransportCongestion,
self.sample_transport_congestion());
#[cfg(any(target_os = "linux", target_os = "macos"))]
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ActivateConnectedUdpSessions,
self.activate_connected_udp_sessions().await);
// Debug-build sweep of the peer-lifecycle map invariant
// (leaked machines / machine-less legs); two map scans,
// compiled out of release builds.
#[cfg(debug_assertions)]
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::DebugAssertPeerMapsCoherent,
self.debug_assert_peer_maps_coherent());
});
crate::instr::tick_gauges(instr_on, self.peers.len() as u64);
}
// Shutdown signal → enter the bounded drain in place, ONCE.
// Gated on `is_none()` so it only fires while serving; after
// entering drain the arm is disabled (the completed signal is
// never polled again) and the deadline arm below bounds the
// window. Placed after the real arms so their `biased` priority
// is unchanged, and inert while serving with `pending()`.
_ = &mut shutdown, if drain_deadline.is_none() => {
self.enter_drain().await;
drain_deadline =
Some(tokio::time::Instant::now() + self.config().node.drain_timeout());
}
// Bounded drain deadline (drain mode only). Placed LAST so the
// `biased` priority of the normal arms is unchanged, and gated
// on `is_some()` so in normal mode the branch is disabled — the
// future is created but never polled and never fires.
_ = tokio::time::sleep_until(
drain_deadline.unwrap_or_else(tokio::time::Instant::now)
), if drain_deadline.is_some() => {
info!("Drain deadline elapsed, ending drain loop");
break;
}
}
}
@@ -319,12 +466,16 @@ impl Node {
// though no msg1/msg2 exchange can ever succeed. Bump the
// discovery-layer cooldown to the long protocol-mismatch
// window and emit a single WARN per fresh observation.
if self.bootstrap_transports.contains(&packet.transport_id)
if self
.supervisor
.nostr_rendezvous
.is_bootstrap_transport(&packet.transport_id)
&& let Some(npub) = self
.bootstrap_transport_npubs
.get(&packet.transport_id)
.supervisor
.nostr_rendezvous
.bootstrap_transport_npub(&packet.transport_id)
.cloned()
&& let Some(handle) = self.nostr_discovery_handle()
&& let Some(handle) = self.nostr_rendezvous_handle()
{
let now_ms = Self::now_ms();
let cooldown_secs = handle.protocol_mismatch_cooldown_secs();
@@ -352,6 +503,9 @@ impl Node {
PHASE_MSG2 => {
self.handle_msg2(packet).await;
}
PHASE_MSG3 => {
self.handle_msg3(packet).await;
}
_ => {
debug!(
phase = prefix.phase,

Some files were not shown because too many files have changed in this diff Show More