laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-07-31 20:16:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: laantungir/fips:maint
Branches Tags
laantungir/fips:master laantungir/fips:feat/platform-peer-queue laantungir/fips:next laantungir/fips:maint laantungir/fips:ble-v2
laantungir/fips:v0.4.1 laantungir/fips:v0.4.0 laantungir/fips:v0.4.0-rc2 laantungir/fips:v0.4.0-rc1 laantungir/fips:v0.3.0 laantungir/fips:v0.2.1-rel laantungir/fips:v0.2.1 laantungir/fips:v0.2.0-rel laantungir/fips:untagged-837413e700b3aaa62052 laantungir/fips:v0.1.0
...
compare: laantungir/fips:next
Branches Tags
laantungir/fips:feat/platform-peer-queue laantungir/fips:next laantungir/fips:master laantungir/fips:maint laantungir/fips:ble-v2
laantungir/fips:v0.4.1 laantungir/fips:v0.4.0 laantungir/fips:v0.4.0-rc2 laantungir/fips:v0.4.0-rc1 laantungir/fips:v0.3.0 laantungir/fips:v0.2.1-rel laantungir/fips:v0.2.1 laantungir/fips:v0.2.0-rel laantungir/fips:untagged-837413e700b3aaa62052 laantungir/fips:v0.1.0
390 Commits
maint ... next
Author SHA1 Message Date
Johnathan Corgan 069d450983 Merge branch 'master' into next 2026-07-29 02:24:38 +00:00
Johnathan Corgan 83c4e800a5 Merge branch 'maint' 
# Conflicts:
#	src/node/retry.rs
 2026-07-29 02:17:34 +00:00
Johnathan Corgan 2045b51f5d Watch the Nostr loops that never return, not the one that does 
Every node reported state: degraded permanently. The liveness probe
polled connect_task, which wraps a single Client::connect() call. That
call only spawns a per-relay background connection task and returns,
so the handle finished moments after start on a perfectly healthy
node, the supervisor saw a child exit, and the node latched Degraded
for the rest of its life.

Nothing behaved differently, because every consumer of NodeState
treats Degraded the same as Running. What was lost is the signal: a
genuine degradation was indistinguishable from the permanent false
one.

Watch the three service loops that cannot return by design instead —
the inbound notify loop, the advert publisher, and the refresh ticker.
Each is an unconditional loop, so a finished handle means a panic or
an abort, which is unrecoverable and matches the one-way ChildExited
latch in the supervisor. connect_task and relay_startup_task stay
deliberately unwatched, and the doc comment now says why, since
watching either reproduces this bug exactly.

The tests install task handles directly and check both directions:
that a finished connect_task alongside three live loops reports
healthy, which is the production configuration a few hundred
milliseconds after start, and that each loop dying on its own reports
degraded. Reverting to the old predicate reds four of five; replacing
the predicate with a constant false also reds four of five, so the
fix cannot pass by never reporting degraded at all.
 2026-07-29 01:31:13 +00:00
Johnathan Corgan 22e71b55a3 Merge branch 'master' into next 2026-07-29 00:36:24 +00:00
Johnathan Corgan 3c4f003c91 Merge branch 'maint' 
# Conflicts:
#	src/nostr/runtime.rs
#	src/nostr/tests.rs
 2026-07-29 00:36:19 +00:00
Johnathan Corgan e8800c3a34 Merge branch 'master' 2026-07-28 17:40:34 +00:00
Johnathan Corgan 2e8fb60970 Let build-deb.sh build with Cargo features, and mark the version when it does 
A measurement run needs the .deb built with a non-default feature, and the
script had no way to express that: its argument loop took only --target,
--version and --no-build, and nothing forwarded a feature list or read one
from the environment. --features <list> now forwards to cargo-deb's native
-F.

The marking is the half that is easy to skip and matters more. The
auto-derived dev Version is built from the crate version, the commit date
and the sha, none of which change when a feature is enabled, so an
instrumented package and a default package of the same commit carried
byte-identical versions. Two consequences, and the second is worse than
the first: the node offers no way to tell which one it is running, and
reverting is an install of a version already present, which no-ops
silently and leaves the instrumented binary in place. That is precisely
the failure the per-commit version was introduced to prevent, reappearing
one level down. The Version now carries a +<features> marker, folded to
dots since underscores and commas are not legal there.

The marker sorts above the unmarked build, checked with dpkg
--compare-versions rather than assumed, so installing a feature build is
an upgrade and reverting is a downgrade: revert with dpkg -i, not apt
install. The ~dev ordering below a tagged release is preserved.

--features is refused together with --no-build, which would stamp the
marker onto whatever binaries happened to be sitting in target/ while
claiming the features had reached them.

Verified end to end rather than by reading: a real --features profiling
build produces a package whose fipsctl accepts the profile subcommand and
whose daemon carries the capture-file header text that only exists under
the feature gate. The release packaging path is untouched, shown by
diffing the cargo invocation the old and new scripts produce across five
argument shapes including the --version/--no-build pair the packaging
workflow uses; all five are identical.

Authored on master rather than maint, which takes bug fixes and CI or
tooling changes but not new capability, so the three copies of this
script now differ by design.
 2026-07-28 17:40:28 +00:00
Johnathan Corgan 60fb3af359 Merge branch 'master' 
Reconciles the tick profiler with this line's extra tick step. `next`
drives `resend_pending_fmp_rekey_msg3` from the tick arm and the master
line does not, so the step table gains a variant here and the arm gains
one more instrumented call.

The pinned row-count test is what caught it: it failed on the merge
rather than letting the emitted table and the call sites drift apart
silently, which is the failure mode it was written for.
 2026-07-28 01:39:47 +00:00
Johnathan Corgan 8162b7d9fc Cover the capture writer's failure path and split its flush cycle 
The mesh rehearsal could not exercise the write-error stop at all, and
that is not a gap in the rehearsal. Removing the sink file leaves the
writer's descriptor valid, so writes keep succeeding into the unlinked
inode and the capture runs on; mounting a tiny filesystem inside the test
container is refused outright. So the terminal state added for a failing
sink had no coverage from either direction.

Split one flush cycle out of the writer loop as a function returning what
it decided, so the loop owns the waiting and the state transition while
the decision is testable on its own. A sink that fails every write now
drives the error outcome directly, and a working sink is asserted to
continue so the first test cannot pass for a writer that always stops.
The residual gap is recorded at the test: what is covered is that an
error from the sink produces the error outcome rather than the cap
outcome, not that a real full disk reaches that branch.

The rehearsal itself was otherwise clean on a live three-node mesh. The
header carries node, build, platform, tick period and the reading
caveats; every step reports every interval; stopping returns in 53 ms
rather than waiting out the flush interval; an idle status reports no
path and no bytes; arming twice is refused naming the active file; an
unwritable directory fails the command; and the node kept serving
throughout. On an idle node the entry gap sits at one tick period and
arm starvation reads about 1.4 ms, which is small but not zero, so the
measurement is live rather than degenerate.
 2026-07-28 01:30:11 +00:00
Johnathan Corgan 7493153a89 Add a feature-gated profiler for the rx-loop maintenance tick 
The tick arm runs twenty-six housekeeping steps in sequence on the one
runtime thread, and is polled last, so anything slow in it holds up
inbound packets, TUN traffic and control commands behind it. Field
evidence says that happens for over a second at a time, but the
attribution behind that is two months old and predates the
connect-on-send gate, the control read isolation, and the peer lifecycle
rework. This measures it rather than continuing to reason about it.

Per step it records exact count, max and total into fixed static
counters; a dedicated writer thread drains them every ten seconds to a
TSV under /var/log/fips, one file per capture, capped at 32 MB. Nothing
accumulates: the counters are swapped to zero each interval and the
thread holds no history. Arming is `fipsctl profile tick on`, served in
the control accept task so the toggle cannot queue behind the very
behaviour it measures, and it does not survive a restart.

The whole thing is behind a Cargo feature that is off by default,
because the risk worth eliminating is the twenty-six edited call sites
in the hot loop. With the feature off the macro expands to the bare
expression, which makes the default build's neutrality something you
read off the generated code rather than something a benchmark fails to
disprove.

The measurement that matters is how late each tick is against the
deadline it was scheduled for, since the arm is polled last and that
lateness is the delay. Two earlier designs derived it from the interval
between entries and both under-reported: the schedule is fixed, so a
steady delay leaves every gap exactly one period and any gap-derived
figure reads zero under precisely the sustained overload this is meant
to find. The interval hands back its own deadline, so the delay is now a
subtraction with no model behind it, and a test drives three late ticks
at a constant gap to keep it that way.

CI gains a default-features clippy and a feature-on build and test on
both runners, closing the gap left by clippy already running with all
features.
 2026-07-28 00:56:54 +00:00
Johnathan Corgan 746218b6bb Merge branch 'master' into next 2026-07-27 17:57:36 +00:00
Johnathan Corgan 52dc21726a Merge branch 'maint' 2026-07-27 17:57:21 +00:00
Johnathan Corgan 540532de44 Make the test backdating seams fail loudly instead of moving time forward 
Windows unit tests have been reding on next at a rekey-drain assertion that
reads like a defect in the drain logic. The drain logic is correct; the test
never reached the state it asserts on.

All three test-only backdating seams shared the fallback
checked_sub(age).unwrap_or_else(Instant::now). When the requested backdate
falls outside the monotonic clock's representable range, that assigns the
present instant, moving the timestamp forward rather than backward. A fresh
Windows CI runner has been up for less than the 600 seconds the drain test
asked for, so the window looked brand new, check_rekey correctly declined to
release the demoted session, and the assertion fired against innocent code.

The seams now route through a shared helper that panics naming the field, the
requested age and the real cause. The drain test asks for 30 seconds, three
times the ten-second window it needs to clear, rather than sixty times it.

Both properties were established by breaking them. Backdating by five seconds,
inside the window, reds the test, so it still depends on expiry rather than
passing vacuously at the smaller margin. Forcing the subtraction to fail
produces the new panic and its explanation rather than a silent reset. Full
cargo test --lib green at 1742, fmt and clippy clean.

No step ran on Windows, so the platform mechanism is inferred from the split
between runners and confirmed only when that job goes green on this tip.
 2026-07-26 18:50:53 +00:00
Johnathan Corgan b6c1977e16 Merge branch 'master' into next 
Carries today's maint batch up: the local CI image-scoping work, which
gives each run its own build context and test image tag instead of
writing the shared mutable one, and the retirement of the bloom-storm
chaos scenario.

Resolved as the earlier per-commit merges did, in the two files where
this line has diverged — the node test module list and the static suite
compose file. Parity holds at 21 legs a side, matching master.
 2026-07-26 17:17:32 +00:00
Johnathan Corgan 76d7e43d60 Merge branch 'maint' 
Carries today's maint batch: the local CI image-scoping work, which gives
each run its own build context and test image tag instead of writing the
shared mutable one, and the retirement of the bloom-storm chaos scenario.

Both apply unchanged here — the compose files, suite lists and matrix legs
they touch are identical on the two lines, so no branch adaptation was
needed. Parity stays symmetric across runners at 21 legs a side on this
branch and 24 on maint, the gap being the three rekey Docker suites maint
keeps by design.
 2026-07-26 17:16:26 +00:00
Johnathan Corgan 0f3f716142 Merge branch 'master' into next 2026-07-25 19:16:01 +00:00
Johnathan Corgan d94300a4f4 Merge branch 'maint' 2026-07-25 19:15:54 +00:00
Johnathan Corgan 679c9aaeb2 Retire the Docker mixed-profile suite in favour of the in-process test 
Everything this suite asserted is now covered in-process over loopback by
mixed_profile_nodes_converge_and_forward: mixed Full, NonRouting and Leaf
convergence, the peer degrees, every direct reachability pair, and the
Leaf-to-Full pair that routes through the Full node between them. The
in-process test covers a superset, asserting that multi-hop path in both
directions where the Docker suite covered one.

Retired on evidence that the multi-hop assertion discriminates rather than on a
green run: reverting the leaf root-election gate reds it in roughly a third of
runs, with the leaf reporting itself as tree root. That was the property the
suite was being kept for, and it is the property now demonstrated elsewhere.

Removed from the local runner's suite list, its runner function, the default
flow and the single-suite dispatch, and from the workflow matrix and its five
steps. Parity reports 22 legs a side, one fewer than before and equal across
the two runners. The suite's script, topology and compose profile stay on disk
and it remains runnable by hand; the retirement is recorded with the others in
the runner's deliberately-not-run block.
 2026-07-25 17:36:47 +00:00
Johnathan Corgan da57a57150 Raise the rekey interval in the handshake tests above the validation floor 
The rekey tests configured a one-second interval to put the timer arm within
reach, then backdated the session two minutes so the jitter draw could not
decide the outcome. That worked, but it asks for a configuration the daemon now
refuses to build: an interval at or below the jitter bound saturates to zero on
a negative draw and rekeys on sight for roughly half of sessions, which
validation rejects.

Thirty seconds clears the bound and leaves the existing backdate a margin of
seventy-five seconds against the worst draw, so the tests keep the property
they were relying on. The helper that drives these handshakes now records why
the value cannot simply be made smaller again.

Surfaced by merging the deployed line up, which is the first point at which
these tests met the new validation.
 2026-07-25 17:24:07 +00:00
Johnathan Corgan e30f3b7ddb Merge branch 'master' into next 
Clean merge, no conflicts. The hop-limit helper corrections and the rekey
config validation arrive from the deployed line and apply here unchanged.

Checked rather than assumed, since this branch changed the same subsystem in
the same session: next keeps its own split of the rekey cadence, where the
config flag rides into the core as RekeyCfg::initiate and check_rekey no longer
returns early, and the incoming validation of the same config block sits
alongside it without overlap.
 2026-07-25 17:17:23 +00:00
Johnathan Corgan 66d5be2bfb Merge branch 'maint' into master 
Both maint changes carry over unchanged. The hop-limit helpers follow the
module rename from src/protocol/link.rs to src/proto/link.rs, applied by
rename detection and verified at the renamed path rather than assumed; the old
path is not recreated. The rekey config validation and its jitter-constant
import apply as written, the node module exposing the constant on this branch
as well.

The only conflict was the unreleased changelog, where master's own Changed
entries and the incoming hop-limit entry were both correct for their branch.
Resolved as the union, with the incoming entry appended to master's Changed
list.
 2026-07-25 17:14:30 +00:00
Johnathan Corgan 026a9c6abf Cover the mixed-profile multi-hop path in the in-process mesh test 
The in-process mixed-profile test asserted convergence, peer degrees and every
direct pair, but deliberately left out the Leaf-to-Full pair that routes
through the Full node between them: a leaf holding a small enough address used
to self-elect as tree root and partition the mesh, so the assertion would have
been flaky. That defect is fixed, so assert the pair here, in both directions.
The leaf's peer degree is already pinned at one in the same test, so neither
direction can be satisfied by a direct link.

Failures in this test are root-election partitions keyed on the random
per-node address ordering, which are undiagnosable from a bare assertion
message. Each of the three failure paths now reports every node's address,
elected root and self-rooted flag.

Reverting the self-election gate reds the new pair in roughly a third of runs
with the leaf reporting itself as root, and the assertion is clean over
repeated runs with the gate in place.
 2026-07-25 17:10:44 +00:00
Johnathan Corgan 3d390fbafd Make rekey.enabled mean "initiate rekeys" and nothing else 
The responder half of the establish decision was gated on this node's own
node.rekey.enabled. With the rekey now declared in the msg3 negotiation
payload, that flag was the only thing that could divert a msg3 whose marker
matched the session we hold, and it diverted it to a msg2 resend while the
initiator had already installed its pending session and would cut over on its
own timer regardless. A pair configured with the flag true on one end and
false on the other therefore parted company at the initiator's cutover and
carried no traffic in either direction until the link-dead timer.

Drop the flag from that gate. The arm now turns on the sender's declaration
matching the session we hold, which is the only signal that was ever
authoritative for the decision, and the snapshot field it read is gone since
every initiator-side use reads the config directly.

Removing the gate exposed a second reading of the same flag. check_rekey
returned early when rekey was disabled, and the drain-expiry arm of the rekey
poll is the only thing that releases a demoted session and its index. A node
that does not initiate had never accepted a rekey before, so it never
accumulated a drain; now that it does, the early return would pin the previous
session and its allocator index forever. The flag rides into the core as
RekeyCfg::initiate instead: the polled cutover and the trigger are gated on it,
the drain is not, and the shell runs the cadence unconditionally.

The cutover arm has to stay gated. The peer snapshot carries no rekey role, so
a pending session stored as responder satisfies that arm exactly as one we
initiated does; ungating it would move a non-initiating node's cutover off the
peer's k-bit flip and onto the poll.

Covered by a two-ended test asserting the pair converges on the same session
indices under an asymmetric setting, a node-level test that a non-initiating
responder actually releases the demoted session and index once the drain window
expires, and a core test pinning which arms a non-initiating snapshot yields.
The existing msg2-resend test reached its arm only through the removed gate and
now reaches it through the health conjunct instead.
 2026-07-25 17:05:00 +00:00
Johnathan Corgan 30e1dcb989 Name this branch's third reader of the address-to-link map 
The doc comment merged up from the maintenance line lists the map's live
readers, and on this branch there is one more: the anonymous-beacon dial
dedup in poll_transport_discovery, which has no counterpart there.

It reads the map with an address supplied by transport discovery, and the
shared-media dial paths resolve before registering, so it compares a key
written in the same form it reads and is not exposed to the mismatch the
comment warns about.
 2026-07-25 01:56:10 +00:00
Johnathan Corgan 7505b86a5d Merge branch 'maint' 
Carries the address-to-link map's corrected doc comment. The only
conflict was the adjacent line naming the discovery config keys, which
this line renamed to node.lookup.* and the maintenance line did not; kept
this branch's names.
 2026-07-25 01:53:15 +00:00
Johnathan Corgan b920200569 Merge branch 'master' into next 2026-07-25 01:53:15 +00:00
Johnathan Corgan 4a448cbe21 Merge branch 'master' 
Carries the rekey counter-arm boundary test and the removal of the
closure-based trigger test. Applied cleanly with no adaptation: the
trigger predicate is byte-identical on both lines, and the claim the new
test rests on was re-measured here rather than assumed to carry — widening
the bound to `p.counter >= 1` reds exactly one test on this line too.
 2026-07-25 00:40:14 +00:00
Johnathan Corgan b29908ba8a Pin the rekey counter arm's boundary against the real trigger 
The send-counter arm of the rekey trigger had no test that could fail for
the reason it existed. The integration suites never reached it: they
inject after_messages = 65536 against a ping-driven workload, so every
rekey they observe is time-triggered. The unit test that looked like the
guard reproduced the trigger's OR predicate as a local closure and
asserted against its own copy, so deleting the arm from the real
predicate left it green.

That is the coverage hole that let the data-plane drop across a
message-count-triggered rekey survive: the defect is precisely a rekey of
a young session, which only the counter arm can produce.

Replaces the existing at-threshold check with a boundary test driving
poll_rekey, holding the time arm off so the counter is provably what
decides. The silence-below half is the part that is new, and it earns its
place by measurement: widening the bound to `p.counter >= 1` reds exactly
one test in the library suite, this one. Deletion, narrowing to `>` and
inversion are caught here too, though the time arm's negative tests
already catch a bound removed altogether.

Removes the closure-based test rather than repairing it. Its one
assertion over real code, that a fresh session entry's jitter lies within
the symmetric bound, is already covered over 100 samples elsewhere.

Note this leaves the deployed maintenance line untested on the same arm:
its trigger is inline in an async method that reads the clock directly,
with no pure function to drive, so the same test cannot be written there
without the sans-IO seam this line already has.
 2026-07-25 00:35:16 +00:00
Johnathan Corgan 537f0b32c7 Merge branch 'master' 
Carries the in-process rekey continuity test and the retirement of the
three rekey Docker suites onto the XX line.

The continuity test was written against master's IK handshake, where the
responder decided rekey-versus-fresh-dial from session age and the test
had to backdate both sessions past a 30s acceptance gate. Here the
responder classifies from the rekey marker the initiator declares in its
msg3 negotiation TLV and reads no clock at all, so the backdating is
dropped: it would age the session past nothing, and leaving it in would
tell a reader a gate exists that does not.

Checked that the test still discriminates rather than merely observing a
session index change. Suppressing the declaration so the responder falls
back to the cross-connection path reds the post-cutover assertion in 8 of
16 runs, which is the address-ordering coin flip the original data-plane
drop had; the healthy tree is 0 of 16 red. The 30s backdate makes no
difference to either rate, which is what confirms it was vestigial here.

The suite wiring keeps mixed-profile, which master does not have, and
drops the three rekey suites from the local suite list, the runner
functions, the default flow, the --only dispatch and the GitHub matrix
together, so check-ci-parity.sh stays green at 23 legs a side. The
retirement note is retargeted at this branch's coverage: the establish
decision tests live in src/proto/fmp/tests/core.rs and src/peer/machine.rs
here, not in the establish_chartests.rs file master keeps.
 2026-07-24 23:58:42 +00:00
Johnathan Corgan aa204cf729 Swap on a dial that crosses a rekey, and cover the marker's two ends 
The rekey marker's cross-connection arm declined to swap while a rekey of
ours was in flight, reasoning that the swap rewrites the session and both
indices while touching no rekey state, so it would orphan that rekey. The
reasoning was one-sided and the arm was wrong: this is half of a two-sided
tie-break whose other half reads only whether a peer exists and the address
ordering, and cannot see our rekey. Declining here while the peer's outbound
half swaps anyway leaves the two ends on four distinct indices with neither
holding a pending session, dead both directions until the link-dead timer.
Measured against the pre-marker tree, which converges: it holds the crossing
dial as pending and adopts it at cutover.

The orphaned-rekey hazard is real but belongs to the executor. The swap now
abandons the rekey it displaces and frees that index first, exactly as the
rekey-responder path already does when it loses the dual-rekey tie-break, so
the classifier can take the arm unconditionally.

Two further failure paths of the marker are tightened. A malformed marker
tore the handshake down without freeing the index msg1 allocated, and that
arm sits ahead of the ACL gate, so any peer able to complete a msg3 could
grow the allocator set one entry at a time. A rekey with no session index to
declare silently omitted the marker, which puts a real rekey back on the
cross-connection path; it is unreachable today and is now asserted rather
than skipped.

On the test side, the marker's wire contract gains coverage: the accessor
round-trips, an unknown TLV field leaves it alone in both directions, and a
well-formed marker of the wrong length is an error rather than an absence,
which is the case the container codec cannot see and the whole reason the
accessor is fallible.

Two node-level tests now assert both ends rather than one. A rekey fired by
message count on a zero-age session is the mechanism behind the datagram
drops, and nothing exercised it: it fails on the pre-marker tree and passes
here. The mutual-dial test gains an index-pair assertion, since every
per-end predicate it checked holds when the ends have diverged.

One existing test had stopped reaching its own arm. Driving a bare second
handshake into a rekey-disabled peer now lands on the cross-connection arm,
which that flag does not gate, and every assertion held there too; it is
retargeted onto a declared rekey, which is the real route to the duplicate
arm, and asserts the session is left alone so the two arms are told apart.

Documentation that described the removed session-age floor is rewritten
around the declaration.
 2026-07-24 21:48:49 +00:00
Johnathan Corgan 911a000d85 Declare a rekey explicitly in msg3 instead of inferring it from session age 
A rekey and a fresh dial both arrive as a new msg1 on a new link, so the
responder has to tell them apart at msg3. It cannot: the distinguishing
fact is internal to the initiator and nothing on the wire carries it.
Session age stood in for it and could not do the job - a
message-count-triggered rekey fires on a young session, so a real rekey
landed below the floor, was resolved as a cross-connection, and left the
two ends of the link on different session indices.

Carry the answer instead. The initiator adds a negotiation TLV to its
rekey msg3 naming the session it replaces, using the index the receiver
allocated so the receiver can match it against its own. The shell
resolves that into a three-valued claim - none, matching, mismatched -
and the classifier branches on it.

A mismatched marker resends msg2 rather than rejecting: the initiator has
already installed its pending session and will cut over on its own timer,
and the reject path sends nothing back, so tearing down would strand the
link in the way this defect already does. A fresh dial arriving while our
own rekey is in flight also resends, because the cross-connection swap
rewrites both indices and touches nothing else, which would leave that
rekey pointing at a session the counterpart no longer holds. The age
floor was excluding that case incidentally.

The floor and the session-age snapshot field are gone with it, so the
establish snapshot no longer reads the clock at all.

Both node-level rekey tests now drive a real rekey through the
initiator's own trigger. They previously reached the rekey arms by
backdating a session and re-handshaking, which tested the age proxy this
replaces; one of them was reaching its assertions through the duplicate
arm rather than the arm it names. Age them well past the trigger, since
the jitter is +/-15s and a smaller margin makes firing depend on the draw.

Incomplete: no TLV round-trip test, no two-ended convergence test, no
rekey-crossed-with-fresh-dial case, and no integration run yet.
 2026-07-24 20:27:55 +00:00
Johnathan Corgan 2223763dab Let the host shell override RUST_LOG for the static test harness 
The static suites hardcoded RUST_LOG=info, so raising the log level for a
node under test meant editing the compose file. Take it from the
environment with info as the default, matching the passthrough the same
block already uses for the worker-pool overrides.

Rendering is unchanged when RUST_LOG is unset, which is how CI invokes it.
 2026-07-24 19:11:27 +00:00
Johnathan Corgan 694d03375b Stop a leaf node from self-electing as tree root 
A leaf-profile node that held the smallest NodeAddr would self-elect as
tree root, but its peers refuse a non-full node as a parent, so it formed
an isolated second root and partitioned the mesh: a multi-hop session from
the leaf to a non-adjacent full node then failed because the far node could
not route a handshake reply back into the leaf's separate coordinate tree.

Gate tree-state self-election on a new self_is_leaf flag, set from the node
profile at construction. A leaf now attaches under its full upstream and
holds that subtree's coordinate for its own routing. That coordinate's
self < root would be rejected by the root-min wire check, but a leaf never
announces it; it reaches peers only via the coordinates carried on the
leaf's session frames, and the upstream already advertises the leaf in its
bloom filter, so the far node routes back through the upstream.

Also fix Node::with_identity to derive the node profile, leaf-only flag,
and bloom state from the config, matching Node::new; it previously
hardcoded the full profile and silently dropped a configured leaf or
non-routing profile.

Covered by sans-IO decision tests (leaf does not self-elect; leaf keeps its
coordinate under a larger-rooted parent) and an end-to-end multi-hop
regression, each confirmed to fail if the gate is reverted.
 2026-07-24 14:42:08 +00:00
Johnathan Corgan bcfe9270bb Add an in-process mixed-profile lifecycle test over loopback 
Brings up a Full/Full/NonRouting/Leaf mesh matching the mixed-profile
Docker topology, asserts each node peers to the degree its role and
topology dictate (A=3, B=2, C=2, D=1), and verifies end-to-end data
between every directly reachable pair (Full-Full, Full-NonRouting,
Full-Leaf). Adds run_tree_test_with_profiles, a per-node-profile sibling
of run_tree_test.

The Leaf-to-Full multi-hop the Docker suite also checks is deliberately
left out for now: a Leaf node's multi-hop session initiation on the XX
line is not yet reliable, so the Docker mixed-profile suite is retained
for that path rather than replaced with a flaky assertion.
 2026-07-24 05:02:05 +00:00
Johnathan Corgan 9213cce6c4 Retire the rekey Docker suites; timing, continuity and variants are in-process 
The three rekey integration suites (rekey, rekey-accept-off,
rekey-outbound-only) are dropped from both runners. Their coverage now
lives in fast, deterministic in-process tests:

- rekey timing and choreography (trigger, K-bit cutover, drain, jitter,
  guards) in the sans-IO poll_rekey tests (proto/fsp, proto/fmp);
- data-plane continuity across a real cutover in
  rekey_cutover_preserves_data_plane (a real IK rekey over loopback);
- the accept-off dual-init regression and the udp.outbound_only rekey
  loop in the should_admit_msg1 and dual-init characterization tests.

Drop them from both runners in lockstep so the parity guard stays green,
and record the retirement in the deliberately-not-run block with a pointer
to the standalone runner. The rekey-test.sh script stays on disk.
 2026-07-24 03:51:13 +00:00
Johnathan Corgan f44de56fd8 Add an in-process test that a rekey cutover preserves the data plane 
Drives a real IK rekey handshake to K-bit cutover between two loopback
nodes and asserts an encrypted datagram sent after the cutover decodes on
the new session, with no spurious peer teardown. The rekey is forced
deterministically: rekey.after_messages = 1 crosses the initiator's
trigger on the first sent datagram, and both sessions are backdated past
the responder's 30s rekey-acceptance gate, so no wall clock is read.

The cutover is proven to actually occur (the live session index changes),
which is what makes the continuity assertion meaningful rather than
vacuous. Adds a make_test_node_with_config loopback helper for setting the
rekey thresholds. The rekey timing and choreography decisions themselves
are already covered exhaustively by the sans-IO poll_rekey tests.
 2026-07-24 03:39:47 +00:00
Johnathan Corgan e1543d201a Merge branch 'master' into next 
# Conflicts:
#	.github/workflows/ci.yml
#	testing/ci-local.sh
 2026-07-24 02:45:37 +00:00
Johnathan Corgan 768df453b7 Merge branch 'maint' 2026-07-24 02:40:19 +00:00
Johnathan Corgan 28421c4b0e Merge master: retire cost/congestion chaos scenarios 
Bring the six retired cost-based parent-selection chaos scenarios up to
next, resolving the ci-local.sh suite-list conflict (next carries the extra
mixed-profile and admission-cap suites). next already has the equivalent
sans-IO cost and transport-drop coverage, so src is unchanged; this takes
only the scenario removals and the CI-list, README and comment updates.
 2026-07-23 23:46:26 +00:00
Johnathan Corgan 08547e8dff Merge maint: retire cost/congestion chaos scenarios 
Bring up the retirement of the six cost-based parent-selection chaos
scenarios (cost-reeval, cost-avoidance, cost-stability, depth-vs-cost,
mixed-technology, bottleneck-parent), which tested a decision the Docker
harness could not exercise reliably.

master already carries the equivalent sans-IO coverage in
src/proto/stp/tests (effective-depth cost selection, hysteresis, cost
degradation) and its transport-drop tests, so this keeps master's src
unchanged and takes only the scenario removals and the CI-list, README and
scenario-comment updates from maint.
 2026-07-23 23:44:31 +00:00
Johnathan Corgan 45406292aa Merge branch 'master' into next 2026-07-23 19:52:08 +00:00
Johnathan Corgan abf2f9ba48 Merge branch 'maint' 2026-07-23 19:51:58 +00:00
Johnathan Corgan e6d99d62c4 Merge branch 'master' into next 
# Conflicts:
#	testing/ci-local.sh
 2026-07-23 14:13:24 +00:00
Johnathan Corgan c58149b0b5 Merge branch 'maint' 2026-07-23 14:09:43 +00:00
Johnathan Corgan 53ec14c2d5 Merge branch 'master' 2026-07-23 05:29:07 +00:00
Johnathan Corgan bc9bef717e Merge branch 'maint' 2026-07-23 05:29:07 +00:00
Johnathan Corgan c805a4492e Merge branch 'master' 2026-07-23 04:16:51 +00:00
Johnathan Corgan 8d7053fa72 Merge branch 'maint' 2026-07-23 04:16:51 +00:00
Johnathan Corgan 719b587920 Merge branch 'master' 
# Conflicts:
#	CHANGELOG.md
 2026-07-23 03:24:23 +00:00
Johnathan Corgan fc8ebd5a06 Merge branch 'maint' 2026-07-23 03:23:18 +00:00
Johnathan Corgan 32b40221b0 Merge branch 'master' 2026-07-23 03:03:05 +00:00
Johnathan Corgan a1222d6d75 Merge the open !FIPS access SSID layer from GitHub 2026-07-23 03:02:54 +00:00
ArjenandGitHub f624013b83 feat(openwrt): open !FIPS access SSID layer (#126) 
Access layer for phones and laptops to reach FIPS routers on OpenWrt,
stacked on the 802.11s mesh backhaul from #123. Squashed from five
commits by Arjen (Origami74); their original messages follow.

* feat(openwrt): open !FIPS access SSID — fips-ap-setup helper, default transport binding, how-to

  Client access layer for phones and laptops: every FIPS router
  broadcasts the same open SSID ('!FIPS' — the leading '!' sorts it to
  the top of alphabetically ordered network pickers), forming one
  standard ESS. Clients save it once and roam between all FIPS routers
  natively, with FIPS's Noise IK handshake as the only security layer:

  - fips-ap-setup: opt-in UCI helper that creates the 'fips-ap0' open AP
    (encryption none — security type must be uniform across routers or
    clients treat the ESS as different saved networks), an isolated
    network with a static ULA /64, RA-only odhcpd addressing (stateless
    SLAAC, no DHCP — the minimum that satisfies Android's provisioning
    check; no internet by design, so phones keep cellular as default
    route), and a locked-down fips_ap firewall zone (no path to br-lan
    or the WAN; only ICMPv6, mDNS, and the FIPS transports reachable).
    'remove' subcommand undoes it. Radio setup stays opt-in; a package
    must not commandeer radios on install.
  - fips.yaml: ship 'ap0'/'ap1' Ethernet-transport entries commented out
    (matching the 802.11s mesh backhaul) so a stock install that never
    creates fips-ap* logs no per-boot "interface missing" bind warning;
    fips-ap-setup uncomments the matching block when it creates the
    interface and re-comments it on remove.
  - Regression test: extend shipped_openwrt_config_parses to assert the
    ap0/ap1 entries ship commented out and still parse once uncommented,
    alongside mesh0/mesh1.
  - Packaging: install the helper in ipk/apk/buildroot (three synced
    copies), extend CI structural checks and shellcheck targets.
  - docs/how-to/set-up-open-access-ssid.md: full guide, including the
    one-time 'no internet, stay connected' acceptance (stored per SSID,
    covers every FIPS router) and the security-type-uniformity
    constraint.

* docs(openwrt): correct open-SSID/mesh security framing — peering is open by design

  The fips-ap-setup/fips-mesh-setup comments and both how-tos claimed a
  stranger "cannot pass the FIPS handshake" / "their frames die at the
  handshake" / the handshake surface "drops them". That is wrong: FIPS
  peer admission is open. An inbound handshake from any net-new identity
  is promoted (node::handlers::handshake::promote_connection), gated only
  by the daemon's max-peers cap — there is no allowlist, no PSK, and the
  AuthChallenge path is not wired to admission. The Noise IK handshake
  provides authentication (no impersonation of another identity, no MITM),
  not authorization.

  Restate the model accurately in all five places: a stranger on the open
  SSID (or the open mesh) can associate AND form a FIPS peer link — that
  is the point of open access. Containment is the isolated fips_ap zone
  (no path to br-lan or the WAN) plus the max-peers cap, not the
  handshake. Clarify that AP client isolation is an L2 control only: a
  peered stranger is an overlay peer like any other, so the FIPS overlay,
  not L2, is the trust boundary between clients.

* feat(openwrt): serve DHCPv4 on the access SSID from a fixed roamable subnet

  RA-only addressing satisfied Android's provisioning check but left
  anything expecting IPv4 with a self-assigned address and a "no IP"
  complaint. dnsmasq now leases out of 10.21.<N>.0/24 (N = radio index;
  prefix echoes FIPS port 2121), deliberately identical on every router:
  a roaming client keeps its lease across the ESS, and dnsmasq's
  authoritative mode — the OpenWrt default, pinned by the helper — ACKs
  the renew a foreign router never issued. Lease collisions across
  routers surface as a NAK on renew and the client re-DHCPs.

  The dhcp section's 'dhcpv4 server' is read by both dnsmasq (default
  images) and odhcpd (only with maindhcp), so either arrangement serves.
  A DHCPv4/udp-67 accept rule joins the fips_ap zone; DHCPv6 stays off,
  the ULA RA stays as-is, and nothing depends on an upstream. The zone
  remains isolated — no forwardings, no internet.

* feat(openwrt): enable mDNS rendezvous from fips-ap-setup

  Phone FIPS apps cannot open raw-Ethernet sockets, so DNS-SD is how they
  find the router's daemon — but node.rendezvous.lan defaults to off. Ship
  the lan block commented in fips.yaml (consistent with the apN transport
  entries) and have fips-ap-setup uncomment it when creating the access
  SSID. The awk match is scoped to node.rendezvous because
  transports.ethernet carries a 'lan' entry at the same indent. The switch
  is daemon-wide, so 'remove' deliberately leaves it on rather than guess
  whether other transports rely on it.

* fix(openwrt): bind UDP dual-stack [::]:2121 so access-SSID clients reach it

  The shipped router config bound the UDP transport "0.0.0.0:2121" (IPv4
  wildcard) while the mDNS LAN advert announces every interface address,
  including the router's IPv6 link-local — which phones on the !FIPS
  access SSID rightly prefer (their cellular default route swallows v4,
  and fd00::/8 is captured by the Myco mesh TUN). Result: the client's
  Noise msg1 arrives on an unbound v6 port and is silently lost; the
  handshake resends and times out.

  Symptom chain (observed on-device): mDNS resolve OK, platform push OK,
  "Sent Noise handshake message 1" to [fe80::…%N]:2121, four resends, no
  reply, 30 s stale-timeout.

  OpenWrt is Linux (bindv6only=0), so "[::]" accepts IPv4 via v4-mapped
  addresses too — nothing is lost. packaging/common is deliberately left
  on "0.0.0.0" for now: Windows defaults IPV6_V6ONLY=1, where "[::]"
  would drop v4 instead.
 2026-07-22 20:02:20 -07:00
Johnathan Corgan 308e2ce3b3 Merge branch 'master' into next 2026-07-23 02:42:58 +00:00
Johnathan Corgan 16b1bc2c5c Merge branch 'maint' 2026-07-23 02:42:48 +00:00
Johnathan Corgan 64f68fbc77 Suffix the mixed-profile test's container names where it builds them 
Giving the mixed-profile services a run-scoped container_name broke this
script's two ping sites, which assemble the name as "fips-$from" rather than
writing it out. A grep for the literal name finds neither, which is the same
trap the earlier naming sweep hit on the other static scripts.

The failure was loud — link convergence passed 3/2/2/1 and then every ping
failed, because docker exec could not find the container — but it is worth
noting that these names were unsuffixed until now, so this suite was never
safe to run twice on one host.
 2026-07-23 02:20:00 +00:00
Johnathan Corgan e9cb3f20a5 Merge branch 'master' into next 2026-07-23 02:09:27 +00:00
Johnathan Corgan 630a04c432 Merge branch 'maint' 
The static compose's mixed-profile services are next-only, so the floating
subnet arrived without them: four services kept ipv4_address pins on a
network that no longer declares a subnet, which docker rejects outright at
`up`. They now float with the rest, take the per-run config directory, and
carry the run name suffix their container_name was missing.

Phase 3 of the admission-cap test conflicted for a real reason rather than a
textual one. This line asserts size-agnostically — sustained inbound from
each denied peer plus "was never promoted" — because the XX handshake
message sizes differ from IK and may change again. That assertion is kept;
what comes across from maint is the address handling underneath it, which
matters here for the same reason: with the subnet floating, a restarted
container can change address, and matching on a single one silently
under-counts the very evidence the assertion rests on.
 2026-07-23 02:08:52 +00:00
Johnathan Corgan fbd0dba870 Merge branch 'maint' 2026-07-23 02:06:06 +00:00
Johnathan Corgan 5851c7f36f Give the next-only mixed-profile suite a suite array 
The parity guard now checks that every suite the local runner can dispatch has a backing suite array, so a suite dispatched without one is visible to nobody. On next, mixed-profile was dispatched by a bare call with no array, so the guard reported it as present on the GitHub matrix and missing from the local runner even though the local runner does run it.

This adds MIXED_PROFILE_SUITES, dispatches through it in the same idiom the admission-cap suite uses so the array is the single source of truth for whether the suite runs, and lists it in the suite listing. It exists only on next, so it is authored here directly rather than merged up.
 2026-07-22 23:18:08 +00:00
Johnathan Corgan d8ded7000f Merge branch 'master' into next 2026-07-22 23:17:43 +00:00
Johnathan Corgan c2d6283bd6 Merge branch 'maint' 2026-07-22 22:57:33 +00:00
Johnathan Corgan b56ec14d14 Merge master into next, bringing the chaos suite's failure reporting with it 
The chaos simulation was equally unable to report a failure on this line: scenarios collided on globally-scoped container names and a shared config directory, the runner swallowed whatever they raised, and the harness discarded their exit codes before anything read them. The merged commits scope names and directories per scenario, give an aborted run its own exit code and let it reach the summary, stop a failed scenario harvesting whichever containers hold its names, and keep the daemon's reason for a failure instead of discarding it.

Testing only. ci-local.sh merged automatically; run_chaos and ci_teardown come across byte for byte and this line's own mixed-profile handling is untouched, the two having never overlapped.
 2026-07-22 10:09:13 +00:00
Johnathan Corgan 946df54d0f Merge maint into master, bringing the chaos suite's failure reporting with it 
The chaos simulation could not report a failure on this line either: scenarios collided on globally-scoped container names and a shared config directory, the runner swallowed whatever they raised, and the harness discarded their exit codes before anything read them. The five commits scope the names and directories per scenario, give an aborted run its own exit code and let that code reach the summary, stop a failed scenario harvesting whichever containers happen to hold its names, and keep the daemon's reason for a failure instead of throwing it away.

Testing only, and the merge is textual: every chaos file matches maint byte for byte, and the ci-local divergence on this line does not touch the changed regions.
 2026-07-22 09:33:58 +00:00
Johnathan Corgan bd3b639570 Merge branch 'master' into next 
# Conflicts:
#	CHANGELOG.md
 2026-07-22 03:46:15 +00:00
Johnathan Corgan 29923cf676 Merge maint into master, reconciling the two hop limit implementations 
Both branches carry the same correction, authored separately because the
sans-IO relocation moved the logic into a core that returns an outcome rather
than acting inline. Git resolved the file move as a rename and offered to merge
the two versions textually, which would have been wrong in both files.

Resolved by keeping master's re-expression wholesale for the handler and its
tests, and taking the changelog entry from maint. Verified rather than assumed:
the resulting tree is byte identical to the reviewed and CI-green master commit
across every code path, and the merge contributes nothing but the fifteen line
changelog entry. Since the code is unchanged from a tree that already passed the
full local suite, that result still stands and no re-run is owed.

Nothing was lost from maint's side. Its tests are written against a file layout
that does not exist here, and master carries equivalent coverage written against
its own shape, including the multi-node chain and the cache warming cases that
the maint version does not have.
 2026-07-22 02:59:44 +00:00
Johnathan Corgan 34431209f5 Follow IP semantics for the session datagram hop limit on the refactored path 
The same correction as on the maintenance line, re-expressed rather than merged,
because the sans-IO relocation moved this logic into a synchronous core that
returns an outcome instead of acting inline.

Delivery to the addressed node is no longer gated on the hop limit, and the
decrement happens before the drop decision, so a datagram that would leave with
zero is not sent. The reachable radius does not change: the old behavior refused
to deliver at zero but was willing to send at zero, and the new behavior is the
mirror of that, so the two cancel.

The shell above the core needed changing too, and that was the real work. It
carried two hop-limit predicates of its own, both justified by comments citing the
ordering that existed before the refactor. The first gated coordinate cache
warming, which this fix requires to be unconditional; left alone it would have
suppressed warming twice over, once for a datagram addressed here and now
delivered, and once for a transit datagram dropped for hop limit whose plaintext
coordinates are still perfectly usable. The second gated next hop resolution,
which carries a cache touch side effect, and now mirrors the core's
would-leave-zero rule instead of the old one. Neither divergence would have
surfaced as a test failure.

That last point is worth stating plainly, because it nearly went the other way.
Restoring the warming gate on top of the corrected core passes the entire suite
without a single failure. The tests could not see the seam between the core rule
and the shell's copy of it. So this adds coverage that can: a three node chain
pinning the decrement across a real hop, which is the only thing exercising the
shell predicates composed with the core, and two warming tests at a zero hop limit
covering both the delivered and the dropped case. Each was checked by putting the
specific defect it targets back and confirming the test fails.

One existing test that claimed to cover hop limit behavior asserted nothing
whatsoever. It asserts now, but those assertions do not discriminate this change,
since the case it exercises behaves identically under both rules. They guard
against the test going vacuous again, not against this defect, and the comment
says so.
 2026-07-22 02:32:34 +00:00
Johnathan Corgan feb1e02e9c Verify the peer static on both FMP handshake paths, not just at rekey 
Under Noise XX the peer static key is learned during the handshake rather
than pinned in advance, and neither of the two paths that learn one was
checking it against what we already knew. That let an attacker who could
observe and inject on path substitute their own identity, on both a fresh
dial and an established link.

On a fresh dial we recorded who we meant to reach and then overwrote it
with whoever answered, without ever comparing the two. Promotion, the ACL
check and the peer registry all then ran on the answering identity, so an
attacker who raced the real peer to msg2 became the peer and the intended
node was never reached. On an established link, a rekey msg2 was matched
to its peer only by the session index we had put in the cleartext msg1
header, so anyone who saw that header could answer with their own static
and take the link over at the cutover.

Both are now compared before anything is committed. The dial-time
expectation moves into its own field with no setter, so the handshake
cannot overwrite it the way it used to; the identity learned from msg2
keeps landing where it always did. Anonymous dials still promote whoever
answers, which is what shared-media discovery means, and that branch is
chosen locally when we dial rather than from anything on the wire, so it
cannot be reached as an exemption. Both comparisons are decisions made in
the synchronous core alongside the existing classifiers.

The gates sit ahead of every mutation, not merely ahead of the session
install, so a forged msg2 no longer poisons the recorded peer epoch and
never earns a msg3. A rejected dial is rescheduled from the dial-time
expectation rather than the answering identity, so refusing an impostor
does not silently retire a configured peer from the dial schedule.

Only this branch was exposed. The released lines use Noise IK, where the
initiator pins the responder static before it dials and a wrong key fails
the AEAD outright.

No wire change: both gates compare a value we already learn against one
we already hold, and legitimate handshakes behave exactly as before.

Six tests cover it, driving the real cadence and dial paths with a third
node answering the intercepted message. Each was checked to fail when the
decision is forced to accept. Local CI 37/37.
 2026-07-21 19:40:50 +00:00
Johnathan Corgan deb5ded8eb Merge master into next after the v0.4.1 rollover 
master carries only maint's 0.4.2-dev version bump linkage; next keeps its
own 0.6.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
 2026-07-19 19:06:16 +00:00
Johnathan Corgan 80c956a6fd Merge maint into master after the v0.4.1 rollover 
maint carries only its 0.4.2-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
 2026-07-19 19:06:08 +00:00
Johnathan Corgan d0a0824768 Merge master into next after the v0.4.1 release 
Same split as the maint-to-master merge: version identity stays with the
receiving branch, project state flows up.

Kept next's: Cargo.toml and Cargo.lock at 0.6.0-dev, the status badge, and
the paragraph identifying next as the wire-format-breaking line that will not
interoperate with v0.2.x, v0.3.x, or v0.4.x peers. Only the shipped-release
pointer inside it moved from v0.4.0 to v0.4.1.

The changelog conflict was additive rather than competing, and resolving it
either way would have lost real content. Next's [Unreleased] Fixed section
carries the XX rekey divergence and dual-initiation work; master brought the
[0.4.1] section. Git could not tell these were adjacent rather than rival, so
both were kept in order, with next's Breaking block and its own [Unreleased]
entries untouched.

Took from master: the bloom FPR default change and its duplicate-definition
fix, the docs describing them, the v0.4.1 release notes, the v0.4.0 date
correction, and the root RELEASE-NOTES.md mirror.

Checked before merging that no incoming content names the Noise handshake
pattern, since next is XX where master is IK. The two "Noise IK" strings on
master both predate v0.4.0 and next already carries its own wording for them,
so nothing needed rewording here.

Quartet green: 1698 tests passed, clippy clean with -D warnings.
 2026-07-19 19:03:40 +00:00
Johnathan Corgan a47ddbd5a5 Merge maint into master after the v0.4.1 release 
Carries the v0.4.1 content up the one-way branch flow. The release itself
belongs to maint, so the parts of this merge that identify a version are
resolved in master's favor and the parts that describe the project's state
are taken from maint.

Kept master's: Cargo.toml and Cargo.lock at 0.5.0-dev, the README status
badge, and the "FIPS is at v0.5.0-dev on the master branch" line. A patch
release consumes no minor version and master's development line is
unaffected by it.

Took from maint: the bloom antipoison FPR default change and the
duplicate-definition fix it rests on, the two docs that describe them, the
new v0.4.1 release notes, the correction to the v0.4.0 release date, and the
[0.4.1] changelog section, which slots below master's own [Unreleased] and
above [0.4.0]. The root RELEASE-NOTES.md mirror moves to v0.4.1 because it
tracks the latest shipped release, which also clears the stale provisional
date it had been carrying.

One line needed splitting rather than choosing: the README said "v0.4.0 has
shipped" inside the paragraph that identifies master as 0.5.0-dev. The
version identity stays master's and the shipped-release pointer moves to
v0.4.1.

Quartet green on the result: 1645 tests passed, clippy clean with -D warnings.
 2026-07-19 18:57:52 +00:00
Johnathan Corgan 7089a2949b Merge branch 'master' into next 2026-07-19 07:40:07 +00:00
Johnathan Corgan ea74cd7e58 Merge branch 'maint' 2026-07-19 07:40:07 +00:00
Johnathan Corgan 0dea111fab Merge branch 'master' into next 2026-07-19 07:17:26 +00:00
Johnathan Corgan 37adb13d5b Merge branch 'maint' 2026-07-19 07:17:26 +00:00
Johnathan Corgan 564b31a914 Merge branch 'master' into next 
Conflict in the admission-cap suite: this line's copy is the XX variant,
with a different phase 3 and an extra phase the other line does not have.
Took the polled final read from the merge, kept this variant's more
specific failure message and its enforcement-evidence phase.

Also templated two container references that phase and the peer-list read
own. They are unique to this line, so the pass that introduced the per-run
name suffix never saw them, and this suite would have looked up containers
under names that no longer exist once a run sets the suffix.
 2026-07-19 06:48:17 +00:00
Johnathan Corgan 93a5b71728 Merge branch 'maint' 2026-07-19 06:46:55 +00:00
Johnathan Corgan f1879b8fe9 Merge branch 'master' into next 2026-07-19 06:45:39 +00:00
Johnathan Corgan 4ff7de4d81 Merge branch 'maint' 2026-07-19 06:45:18 +00:00
Johnathan Corgan afdae094dc Merge branch 'master' into next 
Conflict in the FMP core doc pass: this line has no inbound establish
view at all, so the block that master edited does not exist here and the
merge saw a modify-versus-absent conflict. Resolved by keeping this
line's side - the establish view is master-line only and tracks the
handshake-pattern split, so its absence here is correct rather than an
omission to repair.

The lifecycle view correction and the comment rewrites in the node
module carry across unchanged.
 2026-07-19 06:15:21 +00:00
Johnathan Corgan 0c3d9a0b73 Correct stale view-trait docs and drop internal shorthand from comments 
The lifecycle view's doc described the shell as implementing it over a
`connections` map that no longer exists. The establish view named that map
too, and was doubly wrong: both of its snapshot builders read only
`peers`. Rewrite both to describe the maps actually read.

Several comments in the node module carried shorthand from private
working notes - bare parenthetical labels, and one reference to an
internal tracker item - none of which mean anything to a reader of this
repo. Rewrite them so each clause states its own reasoning inline. The
paragraph above the tracker reference already carried the rationale, so
that one is simply dropped.

Comment-only; no behavior change.
 2026-07-19 06:11:40 +00:00
Johnathan Corgan 60b1d27c0d Merge branch 'refactor-node' into refactor-node-next 2026-07-19 04:25:36 +00:00
Johnathan Corgan 281ed132f1 Merge branch 'master' into refactor-node 2026-07-19 04:25:36 +00:00
Johnathan Corgan 6e54161022 Merge branch 'next' into refactor-node-next 2026-07-19 04:25:36 +00:00
Johnathan Corgan c5492f4572 Merge branch 'maint' 2026-07-19 04:25:25 +00:00
Johnathan Corgan cb96a601c2 Merge branch 'master' into next 2026-07-19 04:25:25 +00:00
Johnathan Corgan 26bd1ada5b tests: guard the identity carrier and the inbound denial teardown 
Two properties that the connection-state consolidation depends on had
no test on this line, because the pattern here is Noise XX and the
natural place to assert them under IK is msg1 -- which under XX carries
only ephemeral keys and so proves neither.

The first is that a learned peer identity lands on the surviving
carrier, the control machine's own connection state, rather than
staying locked inside the Noise handle. Every reader that names a peer
mid-handshake goes there: the promotion hand-off reads it to decide who
is being admitted, and the stale-connection sweep reads it to decide
whether a reaped leg is retried or torn down. XX learns identity once
per role at two different messages, so the guard asserts both -- the
initiator learning the responder from msg2, and the responder learning
the initiator from msg3. The initiator half starts from a deliberately
wrong dial-time expectation, so a missing write cannot pass by leaving
a pre-seeded value in place.

The second is that a peer the access list turns away leaves nothing
behind. Under XX the inbound admission gate cannot sit at msg1, since
no static key has crossed the wire yet, so it sits at msg3 -- by which
point the responder has built a control machine, allocated a session
index, opened a link, and completed the Noise session. All of that has
to come back down on the denial. A machine left registered is
unreachable by every teardown path and holds a peering-budget slot
forever; an index left mapped misroutes the next frame to arrive on it.

Both guards were confirmed to fail when the write they pin is dropped
and when the teardown they pin is skipped. Tests only; no production
behavior changes.
 2026-07-19 03:30:12 +00:00
Johnathan Corgan b0cef7a221 Merge branch 'refactor-node' into refactor-node-next 
Brings the handshake-leg deletion onto the XX line: the leg struct
and src/peer/connection.rs are gone, the Noise handles and crypto
methods now live on PeerMachine, and every leg accessor reads the
machine's own ConnectionState.

The incoming branch speaks Noise IK, this one speaks XX, so the crypto
methods could not be taken as they arrived. Git added the IK versions
to machine.rs with no conflict marker, and complete_handshake_msg3
-- which XX needs and IK has no counterpart for -- was absent
from the merged tree entirely. All four are re-expressed here: the
incoming structure (the Option<HandshakeCrypto> handle, the self.conn
reads, the borrow scoping that hoists carrier writes out of the leg
borrow) carrying this line's XX bodies. start_handshake is the one
that mattered most: its signature is identical on both lines, so it
compiled silently, and left alone it would have emitted an IK msg1
through the two-argument new_initiator and panicked every anonymous
dial on an expect for an identity XX does not have at that point. It
now uses the one-argument XX form and no expect.

handle_msg1 keeps this line's flow, not the incoming one's. The two
functions share a name but not a shape -- the admission gate moved
out of msg1 into msg3 when the line went to XX, and identity is
unknown until msg3. The machine is built above the crypto because
it now drives it, but it stays a local until the late insert, so a
rejected msg1 still leaves no registry trace and the drop-the-local
error arms are unchanged. Parking it at SentMsg2 after the index is
allocated needed a transition the birth constructor could not provide,
so that constructor now delegates to it.

Node::start_handshake is re-expressed the same way and for the same
reason: keeping the machine a local until all fallible setup has
succeeded preserves the existing error arms exactly, with no machine
disposal to add.

The two leg-to-carrier mirror blocks collapse. With one carrier
the writes they mirrored are self-assignment. Their content
is preserved: the completion touch comes from the relocated
complete_handshake, on the same clock, and the negotiated profile
from process_fmp_negotiation, which now takes the machine directly
and writes through set_conn_peer_profile.  Worth recording, because
the collapse was justified on a narrower claim that does not hold:
complete_handshake writes only the touch. It never wrote the profile
on either line. The two writes land on the same carrier at the same
point, so the collapse is neutral, but it is discharged jointly by
two methods rather than by complete_handshake alone.

Three inbound tests are dropped rather than carried. Each asserts
a property at msg1 that XX does not have there: an ACL decision, a
no-registry-trace guard on that decision, and an identity learn on the
carrier. Under XX msg1 is ephemeral-only, so none of the three has a
landing site at that step. A comment at each drop site records where
the property actually lives on this line. One of them was a deletion
here that the incoming branch had merely modified, so its removal
is this line's own prior decision rather than a new one. The same is
true of the craft_and_send_msg1 helper and of HandshakeSeed::inbound,
both left without callers once those tests went.

Also carried: peer_actions.rs had no conflict and no incoming change,
but three sites reached deleted Node accessors and are repointed at
the machine; the EstablishView snapshot is not resurrected, as this
line has never had it; and the connection-state doc keeps this line's
XX wording with the registry-independence invariant grafted in.
 2026-07-19 02:42:24 +00:00
Johnathan Corgan 7fe1d75637 peer: delete the pending-connection type, leaving the control machine whole 
The per-peer control machine has absorbed every field the pending
connection carried. What remained was a struct holding two Noise
handles beside a duplicate copy of bookkeeping nobody read. Replace it
with a small carrier for the two handles and delete the type.

Presence of that carrier, not the state of the handles inside it, is
what marks a machine as mid-handshake. The distinction is essential
rather than stylistic: a failed handshake drops its initiation handle
and is deliberately retained so the stale sweep can reclaim it, and a
completed one has its session taken before disposal. Deriving presence
from the handles would make both invisible to the sweep, the
connection count, and the peering budget at once, leaking the slot
permanently. A test drives an empty carrier past every presence
predicate and then detaches it, so a future edit cannot quietly couple
the two.

The remote startup epoch now comes from the surviving carrier, which
the handshake operations already wrote at the same two points with the
same value. The paired writes onto the pending connection's own
bookkeeping had no readers left and are gone.

The handshake-phase surface leaves the public API: it was public by
accident rather than design, and the machine behind it is crate
internal. Callers outside the crate that need a view of pending
handshakes go through the operator queries, which are unchanged.

ConnectionState::inbound_with_transport loses its last non-test caller
with the inbound seed and is marked test-only.
 2026-07-19 01:18:05 +00:00
Johnathan Corgan e21e09d7e6 peer: source the handshake identity and index family from the control machine 
The pending connection and the per-peer control machine have carried
duplicate copies of the handshake-phase fields since the machine gained
its own connection state. Read them from the machine and drop the
connection's projections.

The peer identity is the sharp one. The connection learned it from msg1
and the machine's carrier did not, so the two views genuinely disagreed
for inbound connections until the Noise operations moved onto the
machine and began recording each result on both. That is now in place,
so every reader can take the machine's copy unchanged: the establish
snapshot, the promotion sweep for competing connections, the dial and
path in-progress checks, the peering observations and per-peer in-flight
budget, and the stale-connection sweep's retry address.

Also repointed: our_index, their_index, transport_id, started_at, the
stored handshake message bytes, and the idle-timeout check. Two
duplicate index writes on the connection are dropped, both immediately
preceded by the machine-side write of the same value. Reads that
previously came off a connection detached just before its machine was
disposed now capture the machine's value first.

Test seeding follows the establish paths: the seed builders write
our_index to the carrier, which promotion now reads. A new test pins
the inbound identity learn on the carrier and the retry address a
failed inbound connection reports to the sweep, so a silent regression
to a blank identity cannot pass.

ConnectionState::duration loses its last non-test caller with the
connection accessor and is marked test-only.
 2026-07-19 00:25:17 +00:00
Johnathan Corgan e7537929ba node: move link, direction, and peer address onto the control machine 
These three had no counterpart on the control machine, so readers still
reached them through the pending connection. Add machine-side
accessors and repoint every reader, then drop the connection's.

The peer address needed its writes lifted, not just its reads
repointed: the machine's copy was never written. It is now written at
each of the points the connection's copy was — the inbound seed, the
dial, message-2 completion, and the two paths that seed a machine from
a pre-built connection — so promotion and the resend path read a value
with the same provenance at the same time as before.

Link and direction need no lift. Both machine constructors already
seed them from the same arguments the connection is built with, so an
outbound machine carries outbound state and an inbound one inbound,
and the machine's link always equals the connection's. The handshake
operations' direction guards read the machine's copy for the same
reason.

The stale-connection sweep's teardown log and its resend path both now
take the transport and address from the machine, and each keeps its own
check that a pending connection is still attached rather than relying
on the caller to have established it.

Add a test pinning link, direction, and address on the two shapes that
seed a carrier independently — the dial and an accepted message 1. The
cross-connection winner reads both values from a carrier one of those
two already seeded.
 2026-07-18 23:17:21 +00:00
Johnathan Corgan 347cbe60bd peer: move the Noise handshake operations onto the control machine 
The pending connection drove its own Noise handshake while the control
machine held it, so the crypto and the state it produces lived on the
carrier that is going away. Move the six operations onto the machine:
starting and completing an initiation, processing an inbound
initiation, taking the session, testing for one, and dropping the
handle on failure. Bodies are unchanged apart from reaching the
handles through the attached connection.

Each operation now records its results on both carriers. The learned
identity, the remote epoch, and the activity stamp are written to the
machine's own bookkeeping at the same point and with the same value as
they are written to the connection's. The connection's copies still
have readers until those reads are repointed, so both have to be
written; the machine's copy of the learned identity was previously
never populated for an inbound connection, which is why an inbound
pending row showed no expected peer.

Driving the handshake from the machine means the machine has to exist
before the crypto runs. On the inbound path it is built above the
message-1 processing and still kept local, so a rejected message
leaves no registry entry and allocates no index. On the outbound path
it already existed from the dial, so it simply takes the connection
before the index allocation, and both failure arms unwind it as they
did.

Completing message 2 no longer mirrors the activity stamp separately,
since the completion itself now writes both carriers at the point the
mirror was approximating.

Three tests cover what the compiler cannot. A connection whose
handshake failed holds neither Noise handle yet must stay visible to
the stale-connection sweep, or every failed connection would leak and
hold a peering-budget slot forever. A message 1 rejected by the crypto
or by the ACL must leave no machine, no index, and the same rejection
count as before. A dial whose message-1 preparation fails must unwind
the machine registered at dial time; that test drives the
index-allocation failure rather than a crypto failure, which is the
arm this change actually widens, since the allocation now happens with
the connection already attached.
 2026-07-18 22:55:38 +00:00
Johnathan Corgan 11ec16777c node: yield the control machine when iterating pending connections 
`Node::connections()` yielded the pending connection itself, so its
consumers reached the handshake-phase fields through that value. The
pending connection is being folded into the control machine, and the
fields will move off it, so yield the machine (keyed by its link) and
let each consumer reach what it needs from there.

Membership is unchanged: the iterator still selects machines that
carry a pending connection, which is the predicate `connection_count`
and the stale-connection sweep already use. Every consumer takes the
same value from the same place, one hop further out, and no field
changes carrier here.

The method is now internal to the crate. It yields the control
machine, which is not part of the published surface, and nothing
outside the crate iterates pending connections — operator views of
the handshake phase go through the query surface instead.
 2026-07-18 22:06:46 +00:00
Johnathan Corgan 5b09e22956 node: read promotion's carrier fields in one borrow 
`promote_connection` detached the pending connection and then
interleaved reads off that detached value with three separate lookups
of the same control machine. Gather the machine-side fields
(`their_index`, `transport_id`, link stats) in the single `get_mut`
that takes the connection, so the machine is borrowed once.

Behaviour is unchanged. The connection is still taken before anything
is validated, so a rejected promotion leaves the machine with no
pending connection, and the prelude only gathers options: the checks
below it still report the first missing field in the same order
(`our_index`, `their_index`, `transport_id`, `source_addr`).

Link stats move to the prelude, ahead of those checks. The value is
the same either way: they live on the surviving carrier rather than
the detached connection, and nothing between the two points touches
the machine map. The lookup could not fail at the old site either,
since the function had already reached it through a successful lookup
on the same key, so dropping the defaulting arm changes nothing.

Add a test covering the error order and the detach, driving promotion
with connections missing each required field in turn plus every later
one, so an implementation that validated during the prelude would
report the wrong field. Test seeding grows a variant that lets the
caller shape the seed.
 2026-07-18 21:45:09 +00:00
Johnathan Corgan a382b17931 node: seed control machines directly in tests, ahead of removing the leg 
Tests built a free-standing PeerConnection, mutated it, and handed it to
Node::add_connection by value. None of those sites survives the removal of
PeerConnection, so converting them afterwards would mean one enormous commit
that cannot be reviewed honestly. Convert them now, while the struct still
exists and the conversion can be validated against a green tree.

Adds a cfg(test) Node::seed_handshake_machine plus a HandshakeSeed builder,
and rewrites make_completed_connection (now seed_completed_connection) and
the twenty inline builders onto it. add_connection keeps its body and loses
its test callers.

The builder's carrier seeding is a verbatim copy of add_connection's: the two
conditional writes for their_index and transport_id, then set_leg, through the
same entry().or_insert_with() so an existing leg-less machine keeps its
constructor-side fields. Nothing else reaches the carrier -- our_index,
source_addr, post-construction started_at and the stored handshake bytes stay
leg-only. Seeding more than that would let these tests observe a carrier
richer than production's and keep passing even if a later production
write-lift were missed.

The Noise exchange now runs on the already-seeded leg rather than before the
hand-over. That is neutral because the only read of expected_identity is
guarded by is_outbound, and no crypto method allocates a session index.

Also adds a compile-time check that PeerAction is Clone + Eq, which is what
keeps a runtime handle from being smuggled into an action payload.
 2026-07-18 21:24:03 +00:00
Johnathan Corgan a3c53a3b3c Merge branch 'refactor-node' into refactor-node-next 
Re-establish ancestry after both branches independently merged their
respective mainline.
 2026-07-18 20:42:45 +00:00
Johnathan Corgan 1e6b4cb918 Merge branch 'next' into refactor-node-next 
Bring the Android-ready core (target_os gating and app-owned TUN seam)
onto this branch so subsequent work builds on the current mainline tree.
 2026-07-18 20:42:45 +00:00
Johnathan Corgan a90049d3a1 Merge branch 'master' into refactor-node 
Bring the Android-ready core (target_os gating and app-owned TUN seam)
onto this branch so subsequent work builds on the current mainline tree.
 2026-07-18 20:38:56 +00:00
Johnathan Corgan df52ce459c Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the state-carrier collapse (connection start/activity timestamps,
peer-identity telemetry, handshake resend buffers, index/transport bookkeeping,
and the session-index shadow all moved onto the peer machine's own connection
state) onto the XX handshake.

Next-specific resolutions folded into the merge: the responder learns its transport
id and peer index at msg1 and they are written directly onto the machine there (the
XX machine has no msg1 step, so the promote read would otherwise be unset); the
negotiated peer profile is mirrored onto the machine carrier at both the initiator
(msg2) and responder (msg3) negotiation sites; and the outbound carrier writes land
on both the identified and anonymous dial paths. The leg's transport/index/buffer
copies remain as dead duplicates until the leg dissolves; the machine copy is the
sole live source.
 2026-07-18 18:24:44 +00:00
Johnathan Corgan 791b35c221 node: make the peer machine's connection state the sole session-index carrier 
The session index was carried three ways: on the leg, on the peer machine's own
connection-state copy, and on a separate machine-owned shadow field. Populate the
machine's connection copy on the outbound path (it was written only on the inbound
and cross-connection-swap paths before), then delete the shadow field and route its
readers, the decrypt-registration lifecycle, and the rekey cutover through the
single carrier. The shadow and the connection copy diverged only on paths that are
unreachable or dormant in production (a rekey cutover with no newly allocated index,
the swap winner's later reads, and the never-dispatched timeout/disconnect events),
so the merge is byte-identical for every reachable path. Rewrote the outbound-promote
index tests and the dial-time comment for the single-carrier contract.
 2026-07-18 17:19:44 +00:00
Johnathan Corgan 6a80790742 node: source connection index/transport/stats from the peer machine 
The connection leg's their_index, transport_id and link_stats duplicated the peer
machine's own connection-state copy. Route every production reader through the
machine copy: promotion, the stale-connection reaper, the outbound msg1 resend, and
the in-use / connecting-path checks. Seed the inbound machine's transport_id from the
msg1 packet (the same source the leg used), matching the outbound dial seed, so the
hard-required promote read never sees a missing transport. link_stats is a
never-mutated zero seed, so its leg accessors are removed outright; the leg
their_index/transport_id getters/setters remain only for the test-only connection
builder. Also delete the write-only next_resend_at_ms field from the connection
state (the resend deadline is carried by the machine-armed retransmit timer, not this
field). Byte-identical.
 2026-07-18 16:43:22 +00:00
Johnathan Corgan 60b8acf716 node: source handshake resend buffers from the peer machine 
The connection leg's stored msg1/msg2 handshake-resend buffers duplicated the
peer machine's own connection-state copy. Write the machine copy at the outbound
msg1-prep and the two inbound authorize sites, then source the retransmit
resend-bytes reader, send_stored_msg1, and the pending tier of find_stored_msg2
from the machine. The post-promote active-peer msg2 copy is written from the same
wire bytes, so the pending tier now matching after promotion is value-identical.
Byte-identical resend behavior.
 2026-07-18 16:00:15 +00:00
Johnathan Corgan 4fc295d90a node: source connection peer-identity telemetry from the peer machine 
The expected_peer connection-row field read the peer identity off the leg;
source it from the machine's own connection-state copy instead, matching the
started_at/last_activity treatment. The machine carrier holds the same identity
as the leg for every connection that appears in the view: outbound legs seed
both from the dialed identity, and inbound legs never rest in the connections
view (the machine takes the leg at promotion), so their machine copy is
unobserved. The leg accessor stays for the crypto-extraction path and retry
decisions. Byte-identical.
 2026-07-18 15:32:04 +00:00
Johnathan Corgan b3f2018fce node: source connection start/activity timestamps from the peer machine 
The connection leg's started_at and last_activity duplicated the peer
machine's own connection-state copy. Make the machine copy the sole live
telemetry and timeout carrier: re-stamp it with the leg's provenance when
the leg is attached at handshake start (the msg1-prep clock, not the earlier
dial-time constructor value) and mirror the completion touch, then delete the
leg's last_activity/touch accessors and source the connection-row projection,
show_connections, and the timeout readers from the machine. Byte-identical
for all normal paths.
 2026-07-18 15:11:43 +00:00
Johnathan Corgan 91ba660213 Merge branch 'refactor-node' into refactor-node-next 
Re-express the handshake-state carrier collapse onto the XX code: the leg's
handshake_state field is deleted and the displayed state is derived from the
peer machine's phase, with failure carried on the machine (a send_failed flag
that preserves the handshake phase) rather than on the leg. The next projection
maps the SentMsg2 responder phase to received_msg1 and the anonymous-dial
Discovered phase to sent_msg1; the three initiator send-failure sites carry
failure via send_failed. Telemetry strings, wire bytes, index allocation, and
stale-connection reaping are byte-identical to next.
 2026-07-18 04:08:26 +00:00
Johnathan Corgan 7ea5718270 Merge branch 'master' into next 2026-07-18 02:57:30 +00:00
Johnathan Corgan 56bbc81a40 node: derive handshake state from the peer machine, delete the leg field 
The connection leg's HandshakeState field duplicated the peer machine's
handshake phase. Delete it and derive the displayed handshake state string
from the machine's PeerState, looked up by link id (mirroring the resend
count). Move the failure signal onto the machine: a send_failed flag that
preserves retransmit eligibility (the machine stays in its handshake phase),
alongside the existing Failed state. The leg's crypto self-gates now guard on
Noise handle presence instead of the deleted phase field, and mark_failed
only drops the handle. Telemetry strings, wire bytes, index allocation, and
the stale-connection reaping are byte-identical for all normal paths.
 2026-07-18 02:26:39 +00:00
Johnathan Corgan f7fa257349 node: classify inbound msg3 via a single machine decision site 
handle_msg3 now routes on the InboundDecision returned by the machine's
inbound_msg3 call, instead of computing establish_inbound in the shell and
handling the terminal arms inline. The DualRekeyWon and ResendMsg2 arms'
index free moves to the machine's FreeIndex action (run through the
executor); the ResendMsg2 stored-msg2 resend stays inline. The pre-decision
reject gates and the proceed-arm bodies are unchanged.

Behavior-neutral: the same establish_inbound decision over the same snapshot
and wire, evaluated once in the machine, with a byte-identical free-count on
every arm and no change to wire sends, machine state, or telemetry.
 2026-07-17 21:02:05 +00:00
Johnathan Corgan 7e5619ad32 peer: return the inbound msg3 decision from the machine arm 
on_inbound_msg3 becomes inbound_msg3, returning the InboundDecision
alongside the machine-phase actions so a caller can route on it; the step
arm delegates and drops the decision. The signature is unchanged, so
step-driven callers are untouched.

Also drop two emissions that are dead on the live path: the ResendMsg2
arm's SendHandshake (the stored-msg2 resend is a driver mechanism the
executor would otherwise reframe) and the dormant inbound-msg1 arm's
retransmit timer. No live behavior changes.
 2026-07-17 20:43:08 +00:00
Johnathan Corgan b1adbe3af7 Merge branch 'refactor-node' into refactor-node-next 
Carries the inbound establish-decision reshape from the IK line. Next's XX
inbound surface already computes that decision on its own machine, so the
merge records the shared history without changing next's tree.
 2026-07-17 20:19:33 +00:00
Johnathan Corgan b38f8c6ffb node: classify inbound msg1 via a single machine decision site 
handle_msg1 now creates one local machine before classification and routes
on the InboundDecision it returns, instead of calling establish_inbound in
the shell and re-deriving the decision inside each arm. The promote and
restart arms drop their own per-arm machine construction and the redundant
InboundMsg1 step; their phase-1 actions come from the single call. The
effect-bearing arm bodies (rekey-respond, duplicate resend, reject
bookkeeping, and the promote/restart tails) are unchanged.

Behavior-neutral: the same establish_inbound evaluation over the same
snapshot and wire, done once in the machine method rather than once in the
shell, with byte-identical arm bodies and no change to index allocation,
wire sends, machine state, or telemetry.
 2026-07-17 19:17:25 +00:00
ArjenandJohnathan Corgan cf62cff5f4 feat: Android-ready core: target_os gating and app-owned TUN seam 
Make the FIPS core build and run as an embedded Android library. The host
app owns the TUN (e.g. an Android VpnService) and FIPS performs no
system-TUN or CAP_NET_ADMIN operations.

Squashed from the following changes:

- gate desktop transports/TUN by target_os, not features: a plain
  `cargo build` now compiles for every target with no flags. Ethernet (raw
  AF_PACKET / BPF) is gated to linux/macos, so Android (target_os =
  "android", not "linux") self-excludes it as Windows already did; real
  system-TUN ops are gated per linux/macos and Android gets a no-op stub;
  the ipi6_ifindex cast handles it being i32 on Android vs u32 on macOS. No
  Cargo features are introduced; desktop builds are unchanged.

- app-owned TUN seam: Node::enable_app_owned_tun() lets an embedder that
  owns the TUN fd exchange IPv6 packet bytes with FIPS over channels
  instead of FIPS creating a system TUN device. It returns (app_outbound_tx,
  app_inbound_rx): the embedder pushes packets read from its fd into the
  outbound sender (app -> mesh) and pulls packets destined for its fd from
  the inbound receiver (mesh -> app). start() gates system-TUN creation on
  tun_tx being unset, so with the channels pre-installed it skips device
  creation and does no system-TUN ops; both directions reuse the existing
  inbound-shim and run_rx_loop wiring. Packets entering via app_outbound_tx
  bypass handle_tun_packet, so the embedder must push only fd00::/8-destined
  packets and clamp TCP MSS on outbound SYNs; the rustdoc and the
  IPv6-adapter design doc spell this out.

- keep the android target warning-clean so the cross-compile check passes
  clippy -D warnings.

- add an Android cross-compile CI check: cross-compile the library for
  aarch64-linux-android via cargo-ndk and run clippy -D warnings. Android
  ships as an embedded library (the host app owns the TUN), so there is no
  daemon binary to package; this is a check job, not a packaging one.

- docs: list Android as a supported platform.

Tests: app_owned_tun_seam_wires_channels covers the channel round-trip and
the Active state; start_skips_system_tun_when_app_owned runs start() and
asserts no named system device is created.
 2026-07-17 18:25:55 +00:00
Johnathan Corgan 7fe3388f2f peer: return the inbound establish decision from the machine arm 
The inbound msg1 machine arm re-derived the establish decision for
its own action selection while the shell matched on a separately
computed copy. Expose the arm as a method returning the decision
alongside the actions so a driver can route on it directly; the
step dispatch delegates and keeps its action-only shape.

The dead rekey-respond arm is stripped to decision-only and its
helper deleted: it allocated from the real index allocator, wrote
the rekey shadow index, emitted a wrong-framed rekey send, stamped
the dampening clock, and flipped state, none of which happens on
the live path, where the inline respond body owns all effects. The
resend-msg2 arm's send emission is stripped the same way; the
decision already carries the stored bytes.
 2026-07-17 13:47:50 +00:00
Johnathan Corgan f65cd07048 Merge branch 'refactor-node' into refactor-node-next 2026-07-17 07:43:36 +00:00
Johnathan Corgan 252d16fab9 node: feed msg1 send failure to the peer machine as an event 
send_stored_msg1 marked the embedded leg failed by writing it
directly from the shell. Route the write through a new
HandshakeSendFailed machine event instead: the machine marks its
leg so the stale-connection sweep reclaims it, without leaving the
handshaking state, so retransmit eligibility survives the window
between the failed send and the sweep exactly as before.

send_stored_msg1 gains a now_ms parameter threaded from the action
executor for the step call; the failure arm itself ignores it.
 2026-07-17 07:12:40 +00:00
Johnathan Corgan 94d7b91244 node: route the outbound msg2 establish decision through the machine 
handle_msg2 no longer pre-computes establish_outbound alongside the
machine's own evaluation of the same snapshot. The shell now builds
the snapshot, steps the persistent outbound machine once at the
decision point, and routes on what comes back: the promote action
vector drives promotion through the executor as before, and the
ResolveCrossConnection decision selects the inline swap/keep
resolution bodies, which are unchanged.

The machine step and its defensive transient-rebuild move up from
the promote arm to the decision point; the cross-connection path
keeps its take-leg-then-dispose ordering with the step preceding
both. Comment prose at the touched sites refreshed to describe the
single-decision-site shape.
 2026-07-17 06:59:30 +00:00
Johnathan Corgan 7790eb86bd peer: make the outbound msg2 swap/keep machine arms decision-only 
The machine's on_msg2 cross-connection arms crystallized state and
emitted FreeIndex/RegisterDecryptSession actions that duplicate the
inline shell resolution, but they are unreachable on the live path
(the shell removes the machine before running the swap/keep bodies).
Making them live in that shape would double-free the outbound index
through the executor.

Strip both arms to a single new ResolveCrossConnection { swap }
action: a decision conveyed to the driver, not an effect. The shell
intercepts it and runs the inline resolution, which owns all effects
permanently. The action executor gets a defensive unreachable arm.
The Promote arm and set_their_index are unchanged.
 2026-07-17 06:44:09 +00:00
Johnathan Corgan 4fa565e528 Merge branch 'refactor-node' into refactor-node-next 
Carries the leg-embed storage move: PeerMachine.leg replaces the
Node.connections map. Hand-resolved onto the XX surfaces keeping this
line's semantics with the new storage: the msg3 termination arms and
executor swap/rekey-responder teardowns keep their exact cleanup sets
and index-free behavior; the msg2 cross-connection extract takes the
leg before the unconditional machine dispose; msg1 and anonymous leg
births embed the leg at machine insertion; the rekey-vs-establish gate
tests leg-absence. The merge also drops the now-redundant explicit
machine inserts next to the seeded test seam, whose auto-merged
combination clobbered leg-carrying machines.
 2026-07-17 05:07:36 +00:00
Johnathan Corgan cf1c957336 node: embed the handshake leg in the peer machine, drop the connections map 
The pending-handshake PeerConnection map and the per-peer control
machine map were parallel LinkId-keyed structures whose keysets must
stay coherent by hand. With every leg now born with a machine, the leg
becomes storage inside its machine (leg: Option<PeerConnection>, pure
storage the machine never reads or drives) and Node loses the
connections field; every access routes through the machine.

The non-mechanical lowerings, each argued at the site: the
rekey-vs-establish gate in handle_msg2 tests leg-absence (an established
peer's machine stays keyed by its link, so machine-presence would
misclassify every rekey msg2 as a fresh establish); the
connecting-predicates, peering observation, and handshake-slot budget
iterate machines-with-legs so connect-window machines (leg not yet
born) are excluded exactly as before and never double-counted against
their pending-connect slot; the cross-connection extract takes the leg
before disposing the machine; the stale reaper takes the leg and leaves
the machine untouched when none is present, matching the old early
return. The map-coherence debug check keeps its machine-has-carrier
direction with the embedded leg as a carrier; the leg-to-machine
direction is now true by construction and its gate const is gone.

connection_count() counts machines with legs; the connections()
iterator, the test seams, and the control-socket connection rows are
re-implemented over the embedded legs with unchanged output.
 2026-07-17 04:27:53 +00:00
Johnathan Corgan ba3c552dce node: give anonymous outbound legs a persistent control machine 
The anonymous-discovery dial paths drove no control machine: the leg was
born bare in start_handshake and a throwaway machine appeared only at
msg2. Anonymous legs now birth a persistent identity-less machine
alongside the leg (new_outbound widened to Option<PeerIdentity>, backed
by the existing identity-less outbound connection state), and handle_msg2
crystallizes the learned identity onto it before stepping. The msg2
missing-machine arm becomes defensive recovery (debug-assert + insert,
matching the IK line), and the self-connect drop arm disposes the
machine it previously leaked — reachable from identified dials too,
since the handshake completion overwrites the expected identity without
comparing it to the dial-time expectation. The self-connect arm's
index/link/pending-entry leak is pre-existing and stays as-is.

With every leg now born with a machine, the leg-to-machine direction of
the debug coherence check switches on for this line. The add_connection
test seam seeds direction-faithful machines (an outbound-anonymous
seeded connection previously got an inbound-shaped one), and the
hand-rolled dial sites in tests seed dial machines mirroring
production. New tests pin the identity-less birth, the msg2 identity
crystallization through promote, and the self-connect disposal.
 2026-07-17 03:25:28 +00:00
Johnathan Corgan d6785f2970 node: make the inbound handshake machine persistent across its leg's life 
The inbound control machine was a msg3-time throwaway: handle_msg3 built
a transient, stepped it once for the decision, and discarded it, with
promote_connection birthing a fresh established() machine. Every
handshake leg now gets one persistent machine for its whole life:

- handle_msg1 births the machine parked in the sent-msg2 phase alongside
  the window leg (msg1 crypto and the msg2 build/send stay inline; the
  msg2-send-failure cleanup disposes it).
- handle_msg3 steps that persistent machine instead of a transient. The
  decision path is unchanged: the msg3 handler never reads machine state
  and overwrites the identity-plane fields from the wire outcome before
  dispatching, so every arm's decision and actions are byte-identical.
- promote_connection stops rebirthing: the executor feeds the promotion
  result back into the machine (the shape the IK line already uses) and
  the machine crystallizes to Established in place.
- Every path that tears down an inbound window leg now disposes its
  machine through remove_peer_machine: the seven inline msg3 termination
  arms, the six executor swap/rekey-responder consumption sites, both
  promote-failure arms, the single-peer leaf reject, the losing side of
  a cross-connection, and the msg1 send-failure cleanup. The
  promote-failure disposal also fixes a pre-existing leak: an outbound
  leg whose promotion fails had its connections entry consumed before
  the error, so the stale reaper could never reach its dial machine.

Index-free behavior is unchanged at every site. Two test helpers that
hand-roll outbound dials now insert the dial machine production
inserts, and new tests pin the machine's birth phase, its post-promote
crystallized state, and disposal on the failure and rekey-responder
arms.
 2026-07-17 02:54:00 +00:00
Johnathan Corgan 3f9de1adcd Merge branch 'refactor-node' into refactor-node-next 
Carries the debug-build peer-map coherence check. The leg-to-machine
direction is gated OFF on this line (const flipped in the merge): the
inbound msg1-to-msg3 window legs and anonymous-discovery outbound legs
legitimately run machine-less here, so only the machine-to-carrier
leak-tripwire direction asserts until every leg is born with a
persistent machine.
 2026-07-17 02:13:31 +00:00
Johnathan Corgan 5ccd95cf3f node: add a debug-build peer-map coherence check 
Assert, once per tick in debug builds, that the peer control-machine map
and its carriers stay coherent: every control machine has a live
handshake leg, an active peer, or a pending connect (a machine with none
is a leak the stale reaper can never see), and every handshake leg has a
machine. The second direction sits behind a file-local const so a branch
where handshake-window legs legitimately run machine-less can gate it
off without weakening the leak tripwire.

Teach the add_connection test seam to seed a control machine for the leg
it inserts (derived from the connection's direction and identity), and
remove_connection to dispose it, so the seam-built topologies satisfy
the check. Three unit tests: seam coherence, coherence through a real
promotion, and the orphaned-machine panic.
 2026-07-17 02:09:28 +00:00
Johnathan Corgan 4779a3970d Merge branch 'refactor-node' into refactor-node-next 
Carries the dead-code deletion (PeerSlot enum, PeerConnection resend
API) and the module-doc liveness corrections. Hand-resolved the doc
conflicts with next-appropriate wording: the executor module doc names
the live msg3 decision-machine path (PromoteToActive /
SwapToInboundSession / RekeyRespondTrigger) while keeping the still-true
msg1-inline dormancy note; the peer_machines field doc reflects next's
promote-time established() births; the machine module doc drops the
stale unwired claims, including the provisional-and-unwired note on the
msg3 trigger variants which are live.
 2026-07-17 01:41:21 +00:00
Johnathan Corgan 119b85d28e node: correct stale liveness claims in machine, executor, and timer docs 
Several module and field docs still described the per-peer machine as
unwired shadow scaffolding. It has been live for some time: machines are
inserted at dial and inbound msg1, stepped by the handshake handlers and
the rekey-cadence and liveness-reap routers, and the executor's
SwapSendState/CompleteDrain/InvalidateSendState arms are the authoritative
paths (the inline bodies survive only as debug-assert release fallbacks).
Rewrite those docs to the current truth while keeping the still-true
dormancy facts: PeerEvent::Timeout and PeerEvent::Tick are never
dispatched in production, retransmit fires on the machine-armed deadline
while the timeout reaper keys on timer presence with the config
threshold, and the remaining inert executor stubs are SendRekey,
SendLinkMessage, and the connected-UDP arms. Drop the stale
allow(dead_code) on the peer_machines field.
 2026-07-17 01:33:06 +00:00
Johnathan Corgan 74245e80ac peer: remove the dead PeerSlot enum and PeerConnection resend API 
PeerSlot (and its entire impl surface) was referenced only by its own
module tests and the lib.rs re-export; peer storage has always used the
separate connections/peers maps. PeerConnection's resend_count/
next_resend_at_ms/record_resend delegations had no production callers:
the live resend counter is machine-sourced (connection_resend_count reads
the per-peer machine), and the FSP session layer uses SessionEntry's own
methods. ConnectionState::next_resend_at_ms is now test-only (its
remaining callers are the fmp state unit tests) and marked cfg(test).

The PeerSlot unit tests go with the enum; test_resend_count_tracking is
dropped because the delegation target's schedule arithmetic is already
covered by the fmp resend_bookkeeping test.
 2026-07-17 01:32:49 +00:00
Johnathan Corgan 5a8da01f24 Merge branch 'master' into next 
Fold in a next-only wording fix for the OpenWrt 802.11s mesh backhaul:
next's FMP link handshake is Noise XX, not the Noise IK on master, so the
fips-mesh-setup header and the how-to now name it pattern-agnostically
("the Noise handshake"), matching the shipped fips.yaml comment and
staying accurate on this branch.
 2026-07-16 05:56:36 +00:00
Johnathan Corgan e42598a86e Merge branch 'maint' into master 2026-07-16 05:55:20 +00:00
ArjenandJohnathan Corgan 054d17aac5 feat(openwrt): 802.11s open-mesh backhaul support 
Router-to-router radio backhaul over an open 802.11s mesh interface,
with FIPS providing all encryption (Noise IK), authentication, and
routing on top of bare L2 neighbor links. The mesh runs OPEN with
mesh_fwding 0 — SAE would duplicate the Noise layer and force ath10k
raw mode, and FIPS is the routing layer — so the Noise handshake is the
real auth/encryption boundary and FIPS's spanning tree does the routing.

fips-mesh-setup: an opt-in UCI helper that creates a per-radio
mesh-point interface (radio0 -> fips-mesh0, radio1 -> fips-mesh1;
trailing-digit derivation with a free-index fallback and a collision
guard). Radio setup stays opt-in — a package must not commandeer radios
on install. 'remove' takes an optional radio and otherwise removes all
instances. Dual-band routers get one instance per radio; FIPS treats the
two backhaul paths as failover, not multipath: it keeps one active link
per peer (cross-connection resolution picks a single winner), and the
second band stands by, re-establishing the peer after keepalive timeout —
traffic never uses both bands at once.

fips.yaml ships the mesh0/mesh1 Ethernet-transport entries commented
out, so a stock install that never creates fips-mesh* logs no per-boot
"interface missing" bind warning. fips-mesh-setup uncomments the matching
meshN block when it creates the interface and re-comments it on remove,
so the flash-and-drop-in flow needs no manual config edit. The file is
rewritten 0600-first (it may hold an inline nsec) via an atomic replace.

Two field-found silent non-peering causes are surfaced by the helper and
the guide:

- Same channel: mesh points only peer on a shared channel, and 'auto'
  lets each radio pick its own. The helper prints the radio's
  band/channel and warns loudly on 'auto' with the exact uci command to
  pin one; the how-to gains an ordered no-peers triage (channel mismatch,
  on-air scan check, DFS CAC wait, regdomain).
- STA channel capture: a client (sta) interface drags the whole radio to
  its upstream AP's channel, so a mesh pinned elsewhere never joins and
  does not recover until the STA disconnects. The helper warns when the
  target radio carries a STA; the guide documents the incompatibility of a
  roaming uplink with a fixed-channel mesh on the same radio.

Both the create and remove paths run 'wifi reload', which briefly drops
every client AP on all radios; the how-to sets that expectation.

Regression test: the shipped OpenWrt fips.yaml must parse via the real
Config deserializer in both states — as shipped (mesh inactive) and after
the uncomment the helper performs.

Packaging: the helper is installed across the ipk/apk/buildroot paths
(three synced copies), with the CI structural checks and shellcheck
targets extended to cover it. Full guide in
docs/how-to/set-up-80211s-mesh-backhaul.md.
 2026-07-16 05:48:04 +00:00
Johnathan Corgan 252b77683e Merge branch 'refactor-node' into refactor-node-next 
Re-express the rekey-msg2 and cross-connection observation feeds onto
the XX cores. The rekey-msg2 observation lands inside the block that
installs the pending session (where XX also generates msg3), so it fires
only when the install actually ran. The cross-connection observation
sits at the outbound resolution arms and targets the promoted peer's
machine; the inbound session-swap effects already own their executor
path here and are untouched. Crypto effect bodies are byte-unchanged.
 2026-07-16 00:04:31 +00:00
Johnathan Corgan 87399795f8 node: observe rekey-msg2 and cross-connection resolution in the per-peer machine 
Feed the per-peer machine two observation events so its control/rekey
shadow state stays coherent with the inline crypto-session installs,
without moving those installs. The crypto effect bodies are unchanged.

On rekey-msg2 completion, feed the machine the responder index on the
success path only (after the pending session is installed); the abandon
path feeds nothing.

On cross-connection resolution, feed the machine the swap outcome: on a
swap it records the new outbound and responder indices to mirror the
session replacement, and on a keep it does nothing. The observation
targets the promoted peer's machine, not the outbound leg whose machine
is removed as the block is entered.

Both observation handlers emit no action; nothing consumes the updated
shadow state yet.
 2026-07-15 23:50:18 +00:00
Johnathan Corgan 2fad6be5e6 node: execute inbound session-swap effects through the per-peer machine 
Move the two Noise-XX inbound establish effects — the cross-connection
session swap and the rekey-respond session install — out of the inline
handle_msg3 arms and into their per-peer machine executor actions
(SwapToInboundSession, RekeyRespondTrigger). The inbound establish
decision now falls through to the shared machine-drive arm, matching the
promote/restart arms.

Carry the session index on the InboundMsg3 event and seed it into the
transient machine before dispatch. The swap actions build their index
from the machine's connection state, which is unset on a fresh inbound
transient; without the seed the actions would emit empty and the swap
would silently no-op while leaking the allocated index.

Give the inbound reject and msg2-resend machine arms their terminal
Failed state and index free, matching the inline arms that own those
paths, so a later timer cannot free the index or report loss for a peer
whose handshake has already resolved.

Preserve exactly: the three distinct index frees and their paired index
and pending-outbound map removals; the link-teardown asymmetry (the
cross-connection path drops the address mapping, the rekey-respond path
keeps it); the session take-then-install ordering; and every reject kind
on the error sub-paths.
 2026-07-15 23:12:48 +00:00
Johnathan Corgan efbfb32e07 Merge branch 'refactor-node' into refactor-node-next 
Fold the handshake timer lifecycle (msg1 retransmit + timeout/stale
reap) through the per-peer machine, re-expressed onto the XX handshake
surface.

- Route peer-machine removal through remove_peer_machine so each link's
  timer store is dropped together with its machine (choke-point).
- Home the msg1-retransmit decision on the machine-armed retransmit
  timer; advance the resend counter only on send success and relocate
  the operator-visible resend count onto the machine.
- Reap outbound handshake timeouts via a presence-scan over the machine
  HandshakeTimeout timer, reading the threshold from config each tick so
  the reap stays neutral for any handshake_timeout_secs.

Cancel both dial-armed handshake timers on outbound promote so a
promoted machine carries no stale timer entry. Preserve the guard that
suppresses a msg1 resend at a peer already promoted via the inbound
cross-connection path.

The XX inbound HandshakeTimeout presence-scan is neutral only because
the inbound establish path sends msg2 inline and never dispatches the
inbound machine event, so no inbound leg ever populates a
HandshakeTimeout timer. A future change that drives inbound establish
through the machine must re-verify the reap equivalence for inbound
legs.
 2026-07-15 21:38:52 +00:00
Johnathan Corgan 765819f52b node: reap the handshake timeout through the per-peer machine timer 
Move the outbound handshake-timeout reap off the unconditional check_timeouts
scan onto the machine-armed HandshakeTimeout timer. A new drive_handshake_timeouts
(run before the retransmit drive, so a timed-out leg is reaped rather than resent
on the same tick) reaps the outbound legs that carry a HandshakeTimeout timer and
have idle-timed-out this tick.

The timer's presence selects the leg (only outbound legs arm one; IK inbound arms
none); the reap threshold is the shell is_timed_out(now, config) predicate, not
the timer's stored deadline. The machine arms the timer from a hardcoded constant
at dial, which is not authoritative for an operator-tuned handshake_timeout_secs,
so reading the threshold from config each tick keeps the reap neutral for any
timeout value and on the last_activity clock exactly as before.

check_timeouts keeps reaping everything else: failed connections (all of them,
promptly) and the idle-timeout of legs without a machine timer (inbound legs, and
any machine-less connection). Total coverage is unchanged. check_timeouts runs
before the timer drive, so a failed outbound leg is reaped there first, its timers
dropped, and the timeout drive never double-fires on it.

Split drive_peer_timers into the timeout drive and the (byte-identical) retransmit
drive. The dormant machine timeout handler is not dispatched, so the session index
is freed once, by cleanup_stale_connection. A regression test drives a timed-out
outbound leg to reap; the existing check_timeouts tests continue to anchor the
residual sweep. Test count 1631 to 1632.
 2026-07-15 20:23:49 +00:00
Johnathan Corgan 5021197f5c node: drive the handshake msg1 resend from the per-peer machine timer 
Move the outbound msg1 resend off the unconditional tick function onto the
machine-armed retransmit timer. The per-peer machine already arms a
HandshakeRetransmit deadline at dial; a new drive_peer_timers fires the due ones
(kind-filtered to the retransmit timer) and homes the resend counter on the
machine, where the operator-visible count now reads from.

The resend decision reads the same operator config as before (interval, backoff,
max), the wire bytes and transport target still come from the shell connection,
and the pure core computes the backoff schedule. As with the deleted
resend_pending_handshakes, the count and reschedule advance only on a successful
send: a failed send neither advances the count nor marks the connection failed,
it just retries on the next tick. The handshake-timeout timer stays on the legacy
check_timeouts path, and the rekey/liveness timers keep their own shell drivers,
so drive_peer_timers deliberately fires only the retransmit kind.

The show_connections resend count is relocated to read from the machine (the
counter's new home) via connection_resend_count; machine-less and inbound
connections report 0, matching what the shell connection reported before. Delete
resend_pending_handshakes and resend_candidates (the latter also from the
LifecycleView trait, its only user). The resend unit test is re-expressed against
the machine + timer path, covering the due check and the record-on-success
semantics.

Known limitation: the first resend interval is armed from the machine's hardcoded
1000ms constant, which equals the config default. Under an operator override of
handshake_resend_interval_ms the first resend diverges from the pre-change
dial+interval; subsequent resends and the cap remain config-driven. Neutralizing
the first-resend override needs the interval threaded into the sync core (the
shell arm would be clobbered by the machine's own timer arming at dial), deferred
to when the connection and machine entities merge. Test count unchanged at 1631.
 2026-07-15 19:38:49 +00:00
Johnathan Corgan 31f5a8c1b7 peer: add a per-peer timer store populated by the machine timer actions 
The control machine already emits SetTimer/CancelTimer actions when it arms the
handshake retransmit and timeout deadlines, but their executor arms were a
single no-op stub and the machine's Timeout event was never dispatched -- the
real work still runs on the legacy tick (check_timeouts,
resend_pending_handshakes).

Add the storage the time-as-input driver will read: a peer_timers map keyed by
LinkId then TimerKind, holding each armed timer's absolute deadline. The SetTimer
arm now inserts (overwrite = reschedule) and CancelTimer removes; the outbound
msg2 promote cancels the two dial-armed handshake timers, since the machine
survives promotion and the entries would otherwise linger in the store.

This store is a shadow: it is written and cleared but no driver reads it yet, so
behavior is unchanged. The legacy tick stays authoritative until the driver that
feeds Timeout and deletes the overlapping tick paths lands.

Route every machine removal through a new remove_peer_machine(link) choke-point
that drops the timer store alongside the machine, replacing the twelve direct
peer_machines.remove sites so no armed timer outlives its machine. TimerKind
gains Hash (to key the store) and Ord (for deterministic driver collection); it
is internal and never serialized. Test count unchanged at 1631.
 2026-07-15 18:49:50 +00:00
Johnathan Corgan 8abcd6ff8c Merge branch 'refactor-node' into refactor-node-next 
Bring the outbound-handshake-through-the-machine series onto the next-branch
Noise-XX cores. Re-expressed rather than transcribed: next has no outbound
decision core, so the promote drives PeerEvent::OutboundMsg2 (not the IK Msg2
snapshot event) and cross-connection resolution stays inline. Hybrid provenance
-- identified outbound legs persist the control machine at dial and drive
through it; anonymous-discovery legs (identity unknown until XX msg2) retain the
inline transient-at-msg2 path. ReportLost{kind} routing, the connection-oriented
dial via the machine (OpenTransport + TransportConnected), and the prepare/send
msg1 split all carried over.

Behavior-neutral on next: the XX msg2 body (Noise completion, FMP, ACL, msg3) is
untouched; anonymous legs are unchanged; the dial-persisted machine promotes
with our_index unset.
 2026-07-15 15:52:29 +00:00
Johnathan Corgan 3e7ca90212 peer: drive the connection-oriented outbound dial through the machine 
Route the connection-oriented (TCP/Tor) outbound path through the peer state
machine, matching how the connectionless path already works.
initiate_connection's oriented branch now drives PeerEvent::Dial with
connection_oriented=true; the machine parks in Connecting and emits
OpenTransport, whose executor arm performs the non-blocking transport.connect
and pushes the PendingConnect. When the connect resolves, poll_pending_connects
prepares msg1 in the shell and then drives PeerEvent::TransportConnected, which
sends msg1 via the machine's SendHandshake arm.

The msg1 prepare (index allocation, Noise leaf, wire arming) MUST run in the
shell before the TransportConnected drive: send_stored_msg1 only transmits an
already-armed wire, so a drive-only path would silently send nothing. The
machine's our_index stays unset; the connect-failure path keeps its direct
handshake-timeout handling (TransportFailed stays dormant). The now-unused
Node::start_handshake helper is removed.

Behavior-neutral: same transport.connect, same PendingConnect, same msg1 send
and failure teardown as the removed inline path -- only the driver changes from
inline shell code to the state machine.
 2026-07-15 14:40:58 +00:00
Johnathan Corgan 9588c50063 peer: persist the outbound control machine at dial, not at msg1 prepare 
Move the peer_machines insert for an outbound leg out of the tail of
prepare_outbound_msg1 to a single shared site in initiate_connection, before
the connection-oriented / connectionless fork. This lets the connection-oriented
path find the machine at dial time (so it can be driven through the connect
handshake) without prepare_outbound_msg1 -- which for that path runs after the
connect completes -- clobbering an in-progress machine back to Discovered.

Because the machine now exists before the fallible dial steps, add
peer_machines.remove to every failure path in the widened dial window: the
index-allocation and Noise-leaf failures in prepare_outbound_msg1, the oriented
transport.connect failure, and both poll_pending_connects teardown arms
(handshake-start failure and async connect failure). remove_link does not touch
peer_machines, so these explicit removes are required.

Behavior-neutral: the connectionless path still drives the machine to
Handshaking and sends msg1 identically; the machine's our_index stays unset
(no spurious UnregisterDecryptSession on a later inbound restart); a failed dial
leaves peer_machines empty for that link exactly as before; and no
connection-oriented machine drive is wired here.
 2026-07-15 14:12:49 +00:00
Johnathan Corgan f698da50b6 peer: add unit coverage for the connection-oriented outbound dial path 
The connection-oriented (TCP/Tor) outbound connect->handshake path had no
`cargo test --lib` coverage; it was exercised only by the opt-in Tor
integration suites, and the TCP node tests bypass it via a manual
connectionless handshake helper. Add three unit tests:

- a machine-level test driving Dial{connection_oriented:true} -> Connecting
  (emitting only OpenTransport, no msg1) -> TransportConnected ->
  Handshaking{SentMsg1}, asserting the exact OpenTransport and
  SendHandshake+SetTimer action vectors that start_outbound_handshake emits
  (its oriented reach via on_transport_connected was previously untested; the
  connectionless reach via on_dial was already covered);

- two node-level tests over a real loopback TcpTransport: a successful connect
  that reaches start_handshake (observed via a pending_outbound entry), and a
  connect to a closed port that routes through the failure arm (link torn down,
  no msg1 dispatched).

Tests only; no production change.
 2026-07-15 13:52:34 +00:00
Johnathan Corgan 1f765cfd8f peer: drive the connectionless outbound msg1 send through the machine 
Split the outbound handshake setup out of start_handshake into
prepare_outbound_msg1 (allocate the index, run the Noise leaf, frame and arm
msg1 -- the fallible steps, returning an error the caller propagates) and
send_stored_msg1 (transmit the armed wire). A connectionless dial now runs
prepare in the shell, then drives the control machine, whose SendHandshake
action sends the wire via the executor. Connection-oriented dials keep calling
start_handshake, now prepare followed by the send inline.

The dial event gains a connection_oriented flag so the machine's on_dial sends
msg1 immediately for connectionless transports (no connect step) instead of
opening a transport first. Behavior is unchanged: the same index, Noise leaf,
wire bytes, maps, and send-error handling as before, with index-allocation and
Noise failures still propagated synchronously before the send. A regression
test covers that a promote from the post-dial handshaking state is identical to
the former discovered-state promote.
 2026-07-15 06:15:21 +00:00
Johnathan Corgan 3b99a416ad peer: persist the outbound control machine at dial 
Create and persist the per-peer control machine when an outbound handshake is
dialed, keyed by its link, instead of building a transient at msg2. The msg2
completion path now looks up that persisted machine to drive the promote,
falling back to a transient only if none is present (e.g. a direct-seeded test).

The machine parks in the Discovered state until promotion and is inert to the
liveness reap and rekey cadence while unpromoted, since it is absent from the
peers map. It is removed on every path that ends the outbound leg without
promoting -- the stale-connection reaper, the msg2 authorization-failure arm,
and the cross-connection resolution block -- mirroring the connection's own
lifetime so no dangling machine survives.

Its session index is deliberately left unset on the machine (the shell owns the
index on its connection), so a later inbound restart does not emit a spurious
decrypt-session unregister. A regression test covers that invariant.
 2026-07-15 04:06:44 +00:00
Johnathan Corgan e064c96df3 peer: route peer-loss reports by kind 
Add a LostKind discriminator to the ReportLost action so the executor can
route an un-promoted handshake failure to the connected-guarded reconnect
reflex (note_handshake_timeout) and an established peer's link-death to the
unconditional one (note_link_dead), instead of collapsing every loss to
note_link_dead.

The two loss producers dispatched today, the liveness reap and the
inbound-restart-then-promote arm, both keep the link-dead routing, so
behavior is unchanged. The handshake-timeout and dial-failure producers are
tagged accordingly but stay dormant until their events are dispatched.
 2026-07-15 03:37:31 +00:00
Johnathan Corgan 4620d9f82e Merge branch 'refactor-node' into refactor-node-next 
# Conflicts:
#	src/node/dataplane/peer_actions.rs
 2026-07-15 01:30:14 +00:00
Johnathan Corgan 7e4358782e node: free the inbound session index on the msg3 reject and resend paths 
On two inbound msg3 outcomes the msg1-allocated inbound session index
was dropped without returning it to the allocator: losing the dual-rekey
tie-break (we keep our in-progress rekey and drop their msg3), and a
duplicate handshake that resolves to a msg2 resend. Under repeated hits
this slowly leaks index space. Free the index on both arms, matching the
sibling reject arms.

Add a test-only helper to backdate a peer's session-established instant,
and two regression tests that drive a real second handshake into each
arm and assert the index is returned to the allocator (both fail if the
free is removed).
 2026-07-15 01:26:54 +00:00
Johnathan Corgan a70c725e48 node: drop stale dead-code allows and a redundant clock read 
Remove the now-stale `#[allow(dead_code)]` on `advance_peer_machine`
and `execute_peer_actions`; both are live (called from the link-dead
reap, rekey cadence routing, and the handshake establish sites).

In `route_rekey_cadence`, reuse the ambient context timestamp for the
machine step instead of taking a second clock sample; the extra
sample's only consumer was the currently-inert cutover drain timer, so
the change is behavior-neutral.
 2026-07-15 00:56:58 +00:00
Johnathan Corgan 21b93f4116 Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the action-contract round-trip test. This line's action type
carries two extra establish-trigger variants (SwapToInboundSession,
RekeyRespondTrigger), so the merge resolution extends the test's sample set
and its wildcard-free exhaustiveness match to cover all of them — the test
proves every variant, including the two XX-only triggers, round-trips
unchanged through the async channel.
 2026-07-14 23:52:43 +00:00
Johnathan Corgan c8077967cd peer: add a round-trip contract test for the action message type 
Prove the per-peer machine's action type is a runtime-agnostic message
contract: it is Send + Sync + 'static, and every variant round-trips
unchanged through a single-threaded async channel (a current-thread runtime,
sender and receiver on one thread). A wildcard-free match over the variants
makes any future action a compile error here until it is added to the sample,
so a variant that embedded a runtime handle could not slip through unproven.
 2026-07-14 23:36:18 +00:00
Johnathan Corgan a6f7fc2d2f Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the promote-failure warning target pin. The peer-action
executor diverges between the two lines (IK vs XX establish arms), so the
pin was applied to this line's own two promote-Err warns during the merge
resolution: outbound "Failed to promote connection" and inbound "Failed to
promote inbound connection" now emit under fips::node::handlers::handshake,
matching the sibling max-peers reject log already pinned to that target.
 2026-07-14 23:29:34 +00:00
Johnathan Corgan 5dfa571908 node: pin promote-failure warnings to the handshake tracing target 
The two promote-failure warn! sites in the peer-action executor emit under
this module's own tracing target rather than the handshake target the rest of
the establish-path logging uses. Pin both (outbound "Failed to promote
connection" and inbound "Failed to promote inbound connection") to
fips::node::handlers::handshake, matching the sibling max-peers reject log and
keeping all establish-path diagnostics grouped under one target regardless of
which module physically emits them.
 2026-07-14 23:23:54 +00:00
Johnathan Corgan 22a5f1f807 Merge branch 'refactor-node' into refactor-node-next 
# Conflicts:
#	src/node/dataplane/dispatch.rs
#	src/node/dataplane/peer_actions.rs
#	src/node/handlers/handshake.rs
#	src/node/handlers/mmp.rs
#	src/node/handlers/rekey.rs
#	src/node/mod.rs
#	src/peer/machine.rs
 2026-07-14 01:40:01 +00:00
Johnathan Corgan 6bebca88ac node: rewrite planning-phase locators out of source comments 
Comments across the per-peer machine, executor, lifecycle supervisor,
and peering reconciler carried internal rollout labels and design-note
section references. Rewrite them to describe the code's behavior
directly. Comment-text only; no code changes.
 2026-07-14 01:38:52 +00:00
Johnathan Corgan 1e934e7603 node: drive net-new outbound promote through the peer machine 
Route the outbound side's msg2-completion promote through the per-peer
state machine and its action executor, mirroring how the net-new inbound
promote is driven. A transient outbound machine emits the promote action;
the executor performs the promotion and, on success only, clears the
pending-outbound entry (threaded in through the action context). The
outbound cross-connection path stays inline as before.
 2026-07-13 23:26:30 +00:00
Johnathan Corgan 40d1a5bca7 node: pin the max-peers inbound-reject log to the handshake tracing target 
The debug log for rejecting an over-cap inbound connection is emitted from
the peer-action executor, under that module's tracing target. Nodes that
raise only the handshake module to debug (leaving other modules at info) no
longer surfaced the rejection, so cap-enforcement events were invisible in
their logs. Re-pin the log to the handshake target, matching the adjacent
promoting-inbound log, so the rejection stays visible under a
handshake=debug filter regardless of the executor module's own level.
 2026-07-13 23:24:56 +00:00
Johnathan Corgan cde1af2d1b node: drive inbound establish decision and net-new promote through the peer machine 
Route XX inbound establishment through the per-peer machine: handle_msg3 hands
the establish decision to a transient machine and drives the net-new promote
(Promote + RestartThenPromote) through the executor's PromoteToActive action,
which transcribes the shared promote block (promote_connection plus the
msg2/tree-announce/bloom follow-ups). The four non-promote decisions (Reject,
ResendMsg2, CrossConnect, RekeyRespond) stay inline byte-for-byte; the two
session-swap arms are deferred to a later non-neutral step.

The decrypt-worker register stays inside promote_connection (not relocated),
no promotion-resolved feedback is fed back, and the transient machine is never
inserted into peer_machines (the persistent machine is created once inside
promote_connection). Behavior matches the inline path exactly, and operator
diagnostics are preserved: the relocated promoting-inbound log keeps its
handshake tracing target, and the epoch-mismatch restart breadcrumb is
re-emitted.
 2026-07-13 22:26:19 +00:00
Johnathan Corgan 09d0d41639 node: drive rekey cadence cutover and drain through the peer machine 
Route the initiator rekey cutover and drain-completion through the
per-peer machine and executor instead of running them inline in
check_rekey. The batch poll_rekey and its per-peer snapshots stay
shell-side and unchanged, so the cross-peer phase ordering that governs
the shared index-allocator sequence is preserved; the machine only
consumes each decided action in that order.

The routed cutover and drain reproduce the inline bodies exactly (the
executor's swap-send-state and complete-drain arms), including the
info-level cutover log and the decrypt-worker re-register. Initiating a
rekey stays inline; the machine is fed a rekey-initiated observation
afterward to keep its control state coherent. A guard falls back to the
inline bodies if a machine is somehow absent.
 2026-07-13 21:29:50 +00:00
Johnathan Corgan c15e63e101 node: drive the link-dead peer reap through the per-peer machine 
Route the liveness reap through the per-peer machine: on a link-dead
timeout, the mmp reap advances the peer's machine with a link-dead event
and executes the resulting actions, instead of calling remove_active_peer
and note_link_dead inline.

The effect is unchanged — the machine emits InvalidateSendState then
ReportLost, which the executor maps back to remove_active_peer(peer)
then note_link_dead(peer, now), the same two calls in the same order.
The reap log stays shell-side so it keeps the handlers::mmp target, and
a guard falls back to the inline path if a machine is somehow absent.
 2026-07-13 20:52:58 +00:00
Johnathan Corgan 5498b2a6a3 node: populate the per-peer machine map at establishment 
Insert a per-peer machine into peer_machines at each promote site and
remove it on teardown, so every established peer has exactly one machine
entry keyed by its link id. Nothing drives the machine yet — the entry
is inert — but this correspondence is what the rekey and liveness-reap
folds depend on.

Inserts pair with the two promote_connection arms (normal promote and
cross-connection-won); removes pair with remove_active_peer and the
cross-connection loser teardown. A new PeerMachine::established
constructor parks the machine in the post-handshake Established state.
Behavior is unchanged: nothing reads the map on a live path.
 2026-07-13 20:33:29 +00:00
Johnathan Corgan 42de582ef0 node/dataplane: add the per-peer machine action executor (shadow) 
Add the executor that consumes the per-peer machine's actions, plus the
peer_machines map that homes the machines on the node. Both are shadow:
nothing populates the map or invokes the executor yet.

The rekey-cutover, drain, liveness-reap, and index-plane action bodies
are ported to reproduce the next-line handlers exactly (the cutover log
matches check_rekey's info-level line, kept under the handlers::rekey
target so operator and test log filters still see it). The establishment
actions (promote, cross-connect swap, rekey-respond) are deferred stubs
until the inbound establish path is wired.
 2026-07-13 20:13:16 +00:00
Johnathan Corgan 0eaae58dc8 peer: add the per-peer FMP control state machine (XX, unwired) 
Re-derive the per-peer FMP lifecycle state machine against the next-line
XX establishment cores, as an unwired module. It ports the master-line
machine's structure but adapts it to the XX handshake:

- inbound establishment splits into a msg1 event (allocate the index,
  send msg2, defer) and a msg3 classify event (identity crystallizes,
  establish_inbound runs), matching XX's identity-at-msg3 timing;
- the inbound decision handles all six InboundDecision variants including
  the XX-only CrossConnect;
- the outbound establish_outbound path and the two-phase authorize are
  dropped (no analog on the XX line);
- session-carrying establish effects (cross-connect swap, rekey-respond
  pending session) are represented as plain-data trigger actions; the
  session surgery is done shell-side when the machine is wired.

Nothing drives the machine yet; it is dead code pending the executor and
handler wiring. Unit tests cover the XX dispatch: inbound establish to
Established, dual-init tie-break, cross-connect at msg3, rekey cutover
key-consistency, and liveness reporting.
 2026-07-13 19:47:56 +00:00
Johnathan Corgan 49677cc426 Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the per-peer FMP decomposition from the master line onto
the next line. The establishment and rekey machine-drive on refactor-node
is built on the IK establishment cores (establish_outbound, identity
learned at msg1) that the next line replaced with the XX msg3 model, so
it cannot be carried textually; it is re-derived against next's XX cores
in follow-on commits on this branch.

This merge commit carries only the two changes that are neutral and
next-compatible as-is:

- OwnedFd hygiene for the connected-UDP socket (RawFd -> OwnedFd, drop the
  hand-rolled unsafe Drop; OwnedFd's own Drop closes the descriptor).
- The two-tier send-state boundary: the send-critical subset of ActivePeer
  is regrouped into a PeerSendState struct, with accessors kept stable so
  no caller changes. next's XX establishment, rekey, and liveness-reap
  behavior is unchanged (verified: exact field partition, identical
  accessor signatures and semantics, identical init values, 3-arg mmp
  construction).

The per-peer machine, its action executor, and the peer_machines map are
not carried here; they are re-derived against next's XX establishment
surface in subsequent commits on this branch.
 2026-07-13 18:56:41 +00:00
Johnathan Corgan 5d5da69a5b node: drive the link-dead peer reap through the per-peer machine 
Route each link-dead peer that the tick sweep's plan_heartbeats decides
to reap through the per-peer machine and executor, replacing the inline
reap body in check_link_heartbeats. The batch decision, the liveness
snapshots (read from the hot-path-written receive clock), and the
heartbeat-send arm stay shell-side and byte-unchanged; the machine only
consumes the decided LinkDeadSuspected, tearing the peer down via
remove_active_peer and reporting the loss to the reconciler exactly as
before, on the same tick with the same wall-clock timestamp. The reap
log stays shell-side.

The machine's link-dead handler no longer emits a decrypt-session
unregister keyed by its shadow index (the full peer teardown already
unregisters the real index; the shadow could have drifted to a reused
index), and its guard now covers the Established state a freshly
promoted peer sits in.

Handshake-timeout, retransmit, and stale-connection cleanup stay inline:
they act on pre-promotion legs that have no machine, and the loss reflex
they use differs from the link-dead one.
 2026-07-13 17:08:35 +00:00
Johnathan Corgan e05b868cf8 node: drive rekey cadence cutover and drain through the peer machine 
Route each Cutover and Drain that the shell-side batch poll_rekey decides
through the per-peer machine and the executor, replacing the inline
effect bodies in check_rekey. The batch decision and its per-peer
snapshots stay shell-side and byte-unchanged: poll_rekey phase-groups all
cutovers, then all drains, then all initiations across the peer set, and
that ordering governs the shared index allocator's free-then-allocate
sequence that appears on the wire, so the machine only consumes the
already-decided actions (a new RekeyConsume event) without re-deciding.
InitiateRekey stays inline (its Noise msg1 build is a shell-side leaf)
with a RekeyInitiated observation feeding the machine so its control
state stays coherent for the next tick's cutover.

The cutover and drain logs, which relocated into the executor in the
prior commit, are pinned back to the fips::node::handlers::rekey tracing
target so they stay visible under the operator's module log filter.

Also clears the machine's shadow draining_index on drain so a later
cross-connection resolution cannot double-free the already-freed index.
 2026-07-13 16:20:45 +00:00
Johnathan Corgan 0ebd1b44c0 node: add the initiator-cutover and drain executor actions (unwired) 
Prepare the establish executor to drive FMP rekey by activating the
SwapSendState action (initiator K-bit cutover via cutover_to_new_session,
with the gated decrypt-worker re-registration) and adding a CompleteDrain
action (erase the drained previous session: free its index, drop its
peers_by_index entry, unregister its decrypt session). Both reproduce the
current inline rekey.rs cutover and drain bodies. The machine's drain
mapping now emits CompleteDrain, using the real drained index rather than
a shadow copy.

Unwired: nothing drives the machine's rekey path yet (live rekey still
runs inline), so these arms are unreachable and the change is
behavior-neutral. The cadence fold that routes cutover and drain through
the machine follows.
 2026-07-13 14:51:11 +00:00
Johnathan Corgan 800cfb23e3 node: relocate decrypt-session registration to the establish executor 
Move register_decrypt_worker_session out of promote_connection into the
executor's PromoteToActive handler, gated on a promoted or
cross-connection-won result. Every live promote now flows through that
one executor path, so registration still fires exactly once at the same
synchronous point; the direct test callers of promote_connection spawn
no worker pool, so the call was already a no-op for them.

Add the cross-connection loser-link teardown (close the losing
transport, remove its link, re-point addr_to_link at the winner) to the
executor as a guarded follow-up. It is unreachable on the current driven
establish paths, which only promote net-new peers, and asserts so, but
keeps the executor complete for when that case is driven.

Remove the now-dead drive_promote_to_active and ConnAction::PromoteToActive.
 2026-07-13 14:20:56 +00:00
Johnathan Corgan e9112cc1bb node: drive net-new outbound establish through the per-peer machine 
Cut the net-new outbound handshake completion (a received msg2 that
promotes a fresh outbound leg to a new peer) over to the per-peer
control machine, mirroring the inbound cutover. handle_msg2 still runs
the msg2 prologue, the ACL check, and the cross-connection swap/keep
arms inline; for the net-new promote it now builds a transient machine,
steps it, and drives promote_connection through the executor. The
session index was already allocated at dial, so there is no two-phase
authorize here.

The wire, index sequence, and peer registry state after promote are
byte-neutral. To keep the promote-failure path neutral, the executor's
cleanup now distinguishes inbound from outbound: an outbound promote
failure records the reject only, matching the prior handler, rather
than the inbound path's link and index teardown.

Cross-connection swap/keep, rekey-msg2, and the dial path stay inline;
the loser-link surgery for the currently unreachable driven
cross-connection case lands with the register relocation next.
 2026-07-13 13:42:08 +00:00
Johnathan Corgan c80a7fdea5 node: drive restart inbound establish through the per-peer machine 
Cut the restart handshake (an inbound msg1 from a peer that reconnected
with a new epoch) over to the per-peer control machine, completing the
inbound establish cutover. The old peer is torn down and the fresh leg
promotes through the same two-step authorize-then-allocate path as the
net-new case: the machine's first step emits the old-peer teardown
(invalidate send-state, report loss), the shell interposes the ACL
check, and the second step allocates the new index and sends msg2. The
old index is freed before the new one is allocated and the msg2 wire
bytes are unchanged, so the sequence stays byte-neutral.

With restart driven through the machine, the shared inline establish
tail that only the restart arm reached is deleted.

Also bounds the peer_machines map (remove_active_peer now drops the
peer's machine entry) and restores the msg2-send, promote, and
index-allocation failure warnings the cutover had dropped.
 2026-07-13 12:46:23 +00:00
Johnathan Corgan 0bf031dd32 node: drive net-new inbound establish through the per-peer machine 
Cut the net-new inbound handshake (a fresh msg1 that promotes to a new
peer, plus the at-capacity reject) over from the inline handle_msg1 logic
to the per-peer control machine. handle_msg1 still classifies via
establish_inbound and still owns the Noise wire step, the late ACL check,
and the promote_connection registry surgery; for the net-new path it now
builds the machine, steps it, and executes the returned actions.

Authorization is interposed between two machine steps so the session
index is allocated only after the ACL check passes: a rejected or
unauthorized msg1 consumes no index, matching the prior order exactly.
The msg2 wire bytes, the index-allocation sequence, and the reject
metrics are all byte-neutral. Restart, resend, rekey-respond, and the
other reject arms stay inline unchanged; they move to the machine once
outbound establish is cut over and every promoted peer has a machine.

Also fills in the executor's send-failure and promote-failure cleanup so
a mid-establish error tears the leg down and frees its index exactly as
before.
 2026-07-13 12:05:17 +00:00
Johnathan Corgan 4a0584a5e9 node/dataplane: add the per-peer machine home and action executor (unwired) 
Add Node.peer_machines (a LinkId-keyed map from the stable link handle to
the per-peer control machine) as the home for the machines, and a new
dataplane/peer_actions.rs holding execute_peer_actions / advance_peer_machine:
the executor that maps each PeerAction the machine emits to its shell call —
frame and send a handshake via build_msg2, drive promote_connection and feed
the PromotionResult back through the machine, tear a peer down via
remove_active_peer, free session indices, report loss via note_link_dead.
Actions for the rekey, connected-UDP, and timer paths are stubbed with notes
for the commits that fold those mechanisms in.

Unwired: nothing drives the machine yet — no live handler path calls the
executor and peer_machines is never populated — so this is behavior-neutral;
the inbound and outbound establish paths still run their existing inline
logic. The executor is cut over path-by-path in the following commits.
 2026-07-13 11:11:20 +00:00
Johnathan Corgan 59155df4e3 peer: split ActivePeer send-state into PeerSendState (two-tier boundary) 
Draw the control/published-send-state boundary inside ActivePeer by
grouping the send-critical fields — the three epoch session slots
{current, previous, pending}, the K-bit flag and session-start, the
transport target, the connected-UDP handles, and the hot counters —
into a new co-located PeerSendState struct. The control-tier fields
(identity, connectivity, declaration/ancestry, filter and tree-announce
groups, remote_epoch, the rekey-negotiation sub-machine, and the rest)
stay on ActivePeer.

Behavior-neutral: a pure field regrouping. Every accessor signature is
unchanged (bodies now read/write self.send.*), so the hot path and the
handlers are byte-untouched; both K-bit cutovers still rotate the three
slots atomically with the same control-tier updates. No Arc/ArcSwap —
the fields are co-located and read by plain borrow; publishing behind a
shared cell is later plumbing for a sharded data plane.
 2026-07-13 10:36:15 +00:00
Johnathan Corgan fcaee74ec0 peer: add the per-peer FMP control state machine (unwired) 
Introduce src/peer/machine.rs: a sans-IO per-peer control FSM that
consolidates the scattered handshake/rekey/timeout driver logic now
spread across node/handlers. The machine is a pure reducer —
step(event, now, index_allocator) -> [action] — that reuses the
existing FMP decision cores (establish_inbound/establish_outbound/
cross_connection_winner/poll_*) rather than reimplementing any
decision, and returns runtime-agnostic actions the driver executes.

Control-tier state only; the published send-state boundary and the
driver wiring land in following commits. The machine is terminal at
Closed — re-dial is the reconciler's, so it holds no cross-attempt
retry state.

Includes eight unit tests: inbound and outbound establish, N:1
identity crystallization, the dual-initiation tie-break,
restart-override, rekey initiator cutover, the data-plane-owned
responder cutover boundary, and liveness -> link-dead -> report-lost.
Unwired — nothing calls it yet.
 2026-07-13 09:58:45 +00:00
Johnathan Corgan 56e3d56c25 peer/connected_udp: own the connected-socket fd with OwnedFd 
Replace the hand-rolled RawFd field and the unsafe Drop on
ConnectedPeerSocket with an OwnedFd, whose own drop glue closes the
fd. from_fd now stores the OwnedFd it already receives instead of
stripping ownership through into_raw_fd, and the manual libc::close is
gone, shrinking the unsafe surface.

Behavior-neutral: the fd still closes exactly once at last-Arc-drop and
as_raw_fd returns the same underlying fd while the socket is alive.
 2026-07-13 09:20:04 +00:00
Johnathan Corgan d58bd083fa Merge branch 'refactor-node' into refactor-node-next 2026-07-13 08:49:05 +00:00
Johnathan Corgan 7b0590f70e node/lifecycle: wire the runtime child-exit producer 
Wire exit detection for the four directly-observable optional children so
the supervisor FSM's ChildExited edge fires at runtime. A runtime
child-liveness mpsc channel carries a Child on exit: the DNS task and the
two TUN threads self-report when their body returns, and a 2s poll monitor
reports mDNS and Nostr via new is_finished accessors. The rx_loop gains a
select arm that steps the FSM and republishes health (Degraded, since a
running node always has at least one transport up). Transports and worker
pools expose no runtime-exit signal and are left for a follow-up.

Adds LanRendezvous::is_finished and NostrRendezvous::is_finished; the
latter treats a shutdown-taken connect_task as finished so the monitor
terminates after a stop instead of polling forever.
 2026-07-13 08:48:51 +00:00
Johnathan Corgan c08888afe6 Merge branch 'refactor-node' into refactor-node-next 2026-07-13 08:25:56 +00:00
Johnathan Corgan b93a127623 node/lifecycle: add runtime ChildExited health routing to the supervisor FSM 
Add the sans-IO half of runtime child-liveness monitoring: a ChildExited
event and an on_child_exited handler that routes a runtime task or thread
exit the same way a start failure does. An optional child exiting degrades
the node (Degraded, with the child recorded in the health reasons); the
last transport exiting publishes Failed. There is no restart, and the
handler is inert outside Running (startup, drain, and teardown own their
own child bookkeeping via the pending/up sets). Failed here is a published
health signal only, not a teardown.

The start and runtime paths share the health classification, extracted
from resolve_start_health into classify_health.

The producer that emits the event (the exit-detection wiring) lands in a
following commit; the event variant carries a temporary allow(dead_code)
until then.
 2026-07-13 08:25:45 +00:00
Johnathan Corgan fa3f88b433 Merge branch 'refactor-node' into refactor-node-next 2026-07-13 08:00:54 +00:00
Johnathan Corgan 85a4983dbe node/peering: restore the open-discovery sweep summary log 
The overlay-discovery cutover to the sans-IO reconciler dropped the
operator-facing "open-discovery sweep complete" summary, including its
per-reason skip breakdown, because the sweep logic moved into the
log-free core. Have the core accumulate the enqueue/skip tally as it
reconciles and return it as data; the driver adds the two values only it
holds (the raw cache size and the self-advert filter) and emits the
summary, reproducing the old startup-vs-per-tick summarize gate and the
budget-zero debug path.

The self-advert precedence matches the pre-cutover sweep: a stale or
self-configured own advert is attributed to the age/configured buckets,
not skipped_self.

Behavior-neutral: the tally is pure side-counting; no dial or enqueue
decision changes.
 2026-07-13 08:00:46 +00:00
Johnathan Corgan b7c3d8935c Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the peering homeostatic reconciler onto the XX/v2 handshake
line. The three scattered peering mechanisms (auto-connect and retry,
overlay discovery, and opportunistic transport-neighbor growth) are now
unified in the sans-IO reconciler on both lines.

Hand-resolved on the first-contact surface: the reconciler opportunistic
layer emits a connect for anonymous, identity-unknown transport legs
unconditionally (bypassing the connect budget and per-peer cap, gated only
by the addr_to_link dedup), reproducing the XX dial-before-identity-known
behavior; named legs route through the reconciler with inputs identical to
the master line. handle_msg1/msg3 keep the XX path; the peer-loss reflexes
use the relocated wrappers. Behavior-neutral versus the pre-merge next line.
 2026-07-13 06:40:05 +00:00
Johnathan Corgan 5090ab7851 node/peering: drive opportunistic transport-neighbor growth through the reconciler 
Cut the last scattered peering mechanism — opportunistic growth from
transport-neighbor beacons and LAN mDNS — over to the reconciler's
opportunistic layer via a gate-checked reconcile_opportunistic wrapper,
one call per tick slot (transport and LAN stay separate slots so their
per-tick budget and per-peer cap are not shared). The driver keeps the
beacon/mDNS I/O and the path-granular prefilters that read live state
(self, fresh-enough-to-skip, connecting-on-path) and executes the emitted
Connect intents; the core owns the connected/budget/per-peer-cap
decisions. A first-wins per-peer dedup on the LAN path reproduces the old
inline once-per-peer dial now that the snapshot core cannot observe the
intra-tick connecting feedback.

Delete the now-dead discovery_connect_budget helper. With this, all three
scattered peering mechanisms (auto-connect and retry, overlay discovery,
neighbor growth) are unified in one sans-IO reconciler. Behavior-neutral;
unit test count unchanged (1607).
 2026-07-13 06:02:21 +00:00
Johnathan Corgan 03ced618ce node/peering: drive Nostr open-discovery through the reconciler overlay layer 
Cut the overlay (Nostr open-discovery) enqueue over to the sans-IO
reconciler. A gate-checked reconcile_overlay wrapper runs the overlay
layer alone at the discovery tick slot; the monolithic reconcile would
re-fire the always-on retry-dial and double the per-tick dial cap, so the
overlay slot must call only its layer. The driver builds the candidate
pool from the overlay advert cache (self excluded), the cooldown and
configured-npub sets, and the startup/steady max-age, then feeds the
reconciler; the emitted enqueue set drives the per-enqueue identity-cache
and alias pre-seed exactly as before. Enqueued entries are dialed at the
retry slot, preserving the two-phase cadence.

Delete the now-dead enqueue-budget and expiry helpers. Behavior-neutral:
same candidates enqueued in the same order with the same budget, cap, and
expiry. Opportunistic transport-neighbor growth remains imperative and is
cut over next. Unit test count unchanged (1607).
 2026-07-13 05:34:28 +00:00
Johnathan Corgan bf81f422ea node/peering: drive the mandatory-floor and retry mechanism through the reconciler 
Route the auto-connect floor and the connection-retry mechanism through
the sans-IO reconciler instead of the imperative Node methods. A thin
peering driver (note_handshake_timeout / note_link_dead) centralizes the
reflex path with the gate guard and the already-connected check; the
per-tick retry-dial and the startup floor build reconciler inputs and
execute the emitted Connect intents. The three imperative retry methods
are removed and their call sites rerouted.

Wire the drain and startup gates. Entering a bounded drain now clears the
retry schedule and suppresses the peer-loss reconnect reflex (the drain
window runs under the Suspended gate), so the drain no longer fights a
reconnect for the peers it just closed. The startup floor runs under an
explicit Reconciling gate at the existing peer-connect seam, preserving
the current dial position.

Behavior-neutral except the intended drain change. Overlay discovery and
opportunistic transport-neighbor growth remain imperative for now; they
observe the relocated retry schedule and are cut over next. Unit tests
migrated to the driver API with assertions unchanged (1607).
 2026-07-13 05:00:10 +00:00
Johnathan Corgan a0cf593580 node: relocate the peering fields off Node into the Peering owner 
Move retry_pending and pending_connects from Node into the Peering owner
struct (Node.peering), the home introduced with the reconciler core.
Pure mechanical relocation: the two collections now live behind
self.peering, with roughly seventy accessor sites repointed. No decision
logic is touched and the unit-test count is unchanged (1607). The
imperative peering methods still run; the reconciler is wired in the
following commit.
 2026-07-13 04:24:34 +00:00
Johnathan Corgan 5d08d27d3c node/peering: add the sans-IO peering reconciler core (unwired) 
Introduce PeeringReconciler, the synchronous decision core for peer
desired-state, with its input/action vocabulary (Gate, Budget, Observed,
Candidate, DiscoveryPools, Policy, PeeringAction) and the Peering owner
struct. reconcile() computes connect/retry intents across four layers:
the auto-connect mandatory floor, the ceiling-only overlay enqueue, the
opportunistic transport-neighbor growth, and the node.limits ceiling
enforced inline. It mirrors the supervisor sans-IO shape: no I/O, no
clock reads (time enters as a parameter), no runtime handles.

The core owns its own cross-attempt retry schedule so escalating backoff
survives the fresh connection created per re-dial. Gate-guarded reflexes
(on_handshake_timeout / on_link_dead) reproduce the current backoff math
and no-op while draining. Overlay is strictly ceiling-only (no peer-count
set-point). Disconnect is defined but never emitted (no shedding).

Unwired here: the driver still runs the imperative peering methods and
Node holds the live retry map. 11 unit tests drive the pure core with
synthetic inputs. No behavior change.
 2026-07-13 04:10:36 +00:00
Johnathan Corgan b676c9d83a node: rewrap over-width retry_state_iter signature 
Homing the retry schedule under peering/ lengthened the RetryState
module path, pushing the retry_state_iter return-type line past the
100-column max_width. Rewrap to satisfy rustfmt. No behavior change.
 2026-07-13 04:00:41 +00:00
Johnathan Corgan a45eefb58a node: home the peering concept; relocate the connection-retry schedule 
Create src/node/peering/ as the home for the peer desired-state
(homeostatic reconciler) concept and move the cross-attempt connection
retry schedule into it: RetryState plus schedule_retry /
schedule_reconnect / process_pending_retries.

Mechanical relocation only. The methods remain inherent on Node; the
retry_count must persist across re-dials (a fresh connection is created
each attempt), which is why the schedule belongs in the peering home
rather than on a per-connection type. The one-level-deeper module path
requires pub(super) -> pub(in crate::node) to preserve the prior scope,
plus module-path fixups at the reference sites. No behavior change; unit
test count unchanged (1596).
 2026-07-13 03:30:52 +00:00
Johnathan Corgan 79f79cf6ac Merge branch 'refactor-node' into refactor-node-next 2026-07-13 00:50:40 +00:00
Johnathan Corgan d61d189572 node: split Running into Full/Degraded, add Failed health state 
Determine node health at start completion instead of unconditionally
reaching Running. Zero transports up is now Failed (fatal): start()
tears down cleanly and returns an error, and the daemon exits. Any
configured optional child that failed to start - a transport beyond the
first, Nostr, mDNS, TUN, DNS, or a worker pool - leaves the node
Degraded but serving, with an operator warning naming what failed. All
configured children up is Full. A child the node was never asked to run
does not count against health.

The published NodeState gains Degraded and Failed variants, both visible
via control queries; Degraded is operational, Failed is not. The
lifecycle FSM gains the health states plus the PublishState action that
drives them - a health fork cannot be a single direct state write, which
is why the earlier commits deferred it to here.

Runtime child-exit health re-evaluation (a running child dying) is a
separate liveness-monitoring mechanism left for a follow-up; this commit
is start-time health only.
 2026-07-13 00:46:31 +00:00
Johnathan Corgan d6ca632251 node: add bounded graceful-shutdown drain phase 
Add an operator-visible Draining phase on daemon shutdown. On the
shutdown signal the node broadcasts Disconnect to all peers, then keeps
serving for a bounded window - up to node.drain_timeout_secs (default 2s),
exiting early once all peers are gone - before tearing down. This lets
in-flight traffic settle and peers observe the disconnect before the
transports close, rather than the previous immediate teardown.

The lifecycle FSM gains a Draining state plus Drain/DrainDeadlineElapsed
events; the run loop observes the shutdown signal and transitions to
draining in place - one continuous loop, so the channel receivers are
never destructively cancelled. The published NodeState gains a Draining
variant, visible via control queries during the window. The immediate
stop() path used by tests and non-daemon callers is unchanged: it still
tears down immediately with no drain wait.

The reconciler-gate actions the drain emits are no-ops until the peering
reconciler lands and consumes them.
 2026-07-12 23:11:28 +00:00
Johnathan Corgan 6c5fd3f4b0 node: extract lifecycle supervisor FSM, migrate substrate fields (behavior-neutral) 
Introduce a sans-IO lifecycle supervisor: a synchronous step(event) ->
[action] state machine (SupervisorFsm) that authors the substrate-child
spawn and teardown order, plus an owner struct (Supervisor) holding the
substrate runtime fields and embedding the FSM. The substrate-lifecycle
fields leave Node's flat list into the owner - state, packet_tx, the TUN
reader/writer handles + shutdown fd + channels, the DNS task + identity
channel, the Nostr/LAN rendezvous drivers, and the encrypt/decrypt worker
pools; the dataplane keeps packet_rx.

start()/stop() become the driver executing the FSM's SpawnChild/StopChild
actions: same children, same order, same warn/debug-and-continue on
optional failures, same logs, same NodeState transitions. Behavior- and
wire-neutral. The only determinism change is that transports now tear
down in ascending-id order, previously nondeterministic HashMap iteration.

The bounded Draining phase and the Running{Full|Degraded} health split
land as separate follow-on commits.
 2026-07-12 21:03:46 +00:00
Johnathan Corgan de72bf76ea Merge branch 'refactor-node' into refactor-node-next 
Forward-merge the dataplane/session concept-home reorganization onto
the next line. The moved files carry next's (XX-line) content; the one
hand-resolved surface is src/peer/active.rs, where the source-location
citation sits in different surrounding text than on the master line —
resolved by keeping next's body and applying the same node/session.rs
-> node/session/mod.rs citation update.
 2026-07-12 19:24:07 +00:00
Johnathan Corgan 434b9726aa node: establish dataplane/ and session/ concept homes (behavior-neutral) 
Reorganize the node module tree by concept rather than by
message-handling verb, as the first step of the node runtime
decomposition. Pure relocation: no wire, config, metric, or log
semantics change; the lib test count is unchanged (1577 passed).

Moves (git mv, 100% rename similarity):
- handlers/{forwarding,rx_loop,connected_udp,dispatch,encrypted}.rs
  -> node/dataplane/ — the whole RX hot path (the select! run loop,
  transit/local forwarding, the link-message router, the RX decrypt
  path with responder K-bit cutover + roam writes, and connected-UDP
  fast-path activation) now lives in one home.
- node/session.rs -> node/session/mod.rs — establishes the session
  concept home for the data/state types. The message-behavior file
  handlers/session.rs stays put for now (folds in with the later FSP
  session step).

The IK/XX-divergent establishment files (handlers/{handshake,rekey,
timeout}.rs) and the deferred-home files (handlers/{mmp,lookup}.rs)
deliberately stay in handlers/, to move once rather than twice.

Every module is reached through impl Node methods, so no call site or
re-export shim was needed. Updated in lockstep with the moves: the
module_path!-derived tracing targets in the two mesh-lab compose-trace
overlays, a structural test's include_str! source path, doc-comments
in proto/routing and the mesh-lab docs, and the stale source-location
citations (node/handlers/{forwarding,rx_loop,encrypted}.rs and
node/session.rs) in doc-comments and the discovery design doc.
 2026-07-12 19:22:08 +00:00
Johnathan Corgan a7de830b8d Merge branch 'master' into next 
Forward-merge the fipstop routing-stats label flip and the three fixes
carried up from maint: drop the redundant parent_switched counter, keep
the tighter path_mtu on a LookupResponse, and reuse one shared secp256k1
context. Clean auto-merge into next's post-refactor structure.
 2026-07-12 16:55:50 +00:00
Johnathan Corgan 26d70ebb59 Merge branch 'maint' into master 
Forward-merge three maint fixes, hand-relocated into master's
post-sans-IO / discovery-to-lookup structure:
- drop the redundant TreeMetrics parent_switched counter (keep
  parent_switches), reconciled across the refactored tree/mmp sites and
  the spanning-tree test reads
- keep the tighter path_mtu when applying a LookupResponse, now in
  handlers/lookup.rs after the discovery module rename
- reuse one shared secp256k1 context in the identity module
 2026-07-12 16:51:01 +00:00
Johnathan Corgan 6f8590cd92 docs: flag the bloom-filter design doc as describing the retired v1.5 
The code on next is currently v1 after the v1.5 removal during the
sans-IO refactor, but this doc still described the abandoned v1.5 design
as implemented, which is misleading. Add an interim status banner
pointing at v2 as the pending target until the doc is rewritten to v2.
 2026-07-12 15:36:54 +00:00
Johnathan Corgan 9b46b6fa85 fipstop: rename routing-stats "Discovery" labels to "Lookup" 
The routing-stats pane's Discovery Requests/Responses sections show the FMP
overlay coordinate-lookup counters. Rename the section labels and the nested
JSON key they read from "discovery" to "lookup" to match the metric family's
canonical name. The daemon dual-emits both keys, so this reads the current
name and no longer depends on the deprecated "discovery" alias.
 2026-07-12 01:52:54 +00:00
Johnathan Corgan 033957f23d Merge branch 'refactor-transport' into refactor-next 
Forward-merge the transport neighbor-beacon rename (ethernet + BLE
discovery -> neighbor module/buffer rename; ethernet config discovery
flag -> listen with serde alias). Hand-resolved the module files against
next's identity-out-of-beacon beacon rewrite by starting from next's
bodies and re-applying the rename by identifier, so next's beacon logic
(no-arg build_beacon, bool parse_beacon, computed BEACON_SIZE, one-arg
add_peer, deleted BLE add_peer_with_pubkey) is preserved unchanged.
 2026-07-11 23:20:50 +00:00
Johnathan Corgan cbc089b820 transport/ethernet: rename config discovery flag to listen 
Rename the Ethernet per-interface config flag from discovery to listen,
so the receive/transmit toggle pair reads as the symmetric announce
(transmit) / listen (receive) neighbor-beacon vocabulary. The old
discovery: key is still accepted via a serde alias, so deployed configs
load unchanged; to_yaml re-emits it under the canonical listen: name.
Marked deprecated for removal at the v2 cutover.

Updates the config field + accessor, the transport listen_enabled local,
the one struct-literal test consumer, the chaos sim config generator,
packaged fips.yaml examples, and the classified operator-facing docs
(ethernet neighbor-beacon subsystem prose; the generic Transport
discovery capability prose is left unchanged). Adds a compat parse test
asserting the legacy alias, the new key, and that deny_unknown_fields
still rejects unknown keys. Behavior-neutral.
 2026-07-11 23:13:28 +00:00
Johnathan Corgan 4c95be0000 transport/ble: rename discovery module to neighbor 
Mirror the ethernet neighbor rename in the BLE transport: the internal
peer-detection buffer becomes NeighborBuffer (from DiscoveryBuffer) and
the module is renamed discovery -> neighbor. The BlueR/bluez API terms
(DiscoveryFilter, set_discovery_filter) and the BLE advertise/scan
mechanism vocabulary are unchanged, as is BleConfig. Behavior-neutral.
 2026-07-11 23:01:48 +00:00
Johnathan Corgan e362ab67a6 transport/ethernet: rename discovery module to neighbor 
Rename the ethernet link-local neighbor-beacon subsystem from the
overloaded "discovery" vocabulary to a neighbor umbrella. The beacon
frame vocabulary is retained (build_beacon/parse_beacon/BEACON_SIZE/
FRAME_TYPE_BEACON/FRAME_TYPE_DATA); only the subsystem and buffer
identifiers change: DiscoveryBuffer becomes NeighborBuffer,
discovery_buffer becomes neighbor_buffer, and DISCOVERY_VERSION becomes
BEACON_VERSION (wire value 0x01 unchanged). Config toggles are left for
a follow-up commit. Behavior-neutral.
 2026-07-11 22:56:05 +00:00
Johnathan Corgan 3f214c8f63 Merge branch 'refactor-transport' into refactor-next 
Brings the connected-UDP fast-path plane relocation (ConnectedPeerSocket/
PeerRecvDrain into peer/connected_udp; udp keeps open_connected_fd in io.rs)
and the darwin_sockopts -> sockopts_macos rename onto the next line. Clean
three-way auto-merge; behavior-neutral.
 2026-07-11 21:42:25 +00:00
Johnathan Corgan 6c9f55ea80 transport: rename darwin_sockopts to sockopts_macos 
The module is gated target_os="macos" (not the broader Darwin/iOS family),
so the name now tracks the cfg and matches the *_macos.rs file convention
(io_macos.rs). Drops the redundant inner #![cfg(target_os="macos")] (the
decl gate already covers it) and corrects a stale top comment that claimed
the module stays visible on Linux — it is macos-decl-gated, so it never was.
Mechanical rename; no logic change.
 2026-07-11 21:37:00 +00:00
Johnathan Corgan 89a31fd555 transport: move the connected-UDP fast-path handles into the peer module 
ConnectedPeerSocket and PeerRecvDrain are node/peer-side logic: the Transport
trait never touches them, ActivePeer stores them, and node's encrypt worker
drives them. They only happened to live under transport/udp. Relocate the
handle types to a new src/peer/connected_udp/ module (socket.rs + drain.rs),
which the node handler and encrypt worker reach via node -> peer (no new edge;
a node home would have forced a peer -> node cycle).

The udp transport keeps only the kernel-construction seam: open_connected_fd,
now folded into udp/io.rs (the byte-layer home) behind a linux/macos-gated
submodule and re-exported as transport::udp::open_connected_fd. The node
handler builds the fd through it and adopts it via ConnectedPeerSocket::from_fd.

Behavior-neutral: no wire/config/metric/log change; the per-packet hot path
(bare-RawFd send_batch_gso/raw) is untouched. Preserves the Arc multi-owner
contract, the drop-drain-before-socket ordering, and the drain's detach-on-Drop
deadlock avoidance. Reconciles the old cfg(unix)/any(linux,macos) double-gate
onto the single any(linux,macos) predicate.
 2026-07-11 21:30:36 +00:00
Johnathan Corgan ab0a46f2c0 transport: extract udp open_connected_fd syscall body into a standalone fn 
Pulls the connected-UDP socket construction (socket/REUSEADDR/REUSEPORT/
BUFFORCE/bind/connect + darwin tuning) out of ConnectedPeerSocket::open into
a pub(crate) open_connected_fd returning an OwnedFd; open() now delegates and
adopts the fd. Error paths still close the fd via the temporary's Drop; the
success tail transfers ownership through OwnedFd. Behavior identical. Prepares
the fast-path handle types to move to the peer module while the socket
construction stays with the udp transport.
 2026-07-11 21:08:29 +00:00
Johnathan Corgan ccd47bd742 Merge branch 'refactor-transport' into refactor-next 
Brings the canonical per-transport layout normalization (udp/ethernet
byte-layer renamed to io, ethernet addr.rs home, tcp pool.rs extraction)
onto the next line. Clean three-way auto-merge; behavior-neutral.
 2026-07-11 20:32:52 +00:00
Johnathan Corgan e839aead7a transport: extract tcp connection-pool types into pool.rs 
Moves the inline TcpConnection/ConnectingEntry structs, the Direction
enum, and the ConnectionPool/ConnectingPool type aliases out of the
1147-line tcp/mod.rs into a dedicated pool.rs, matching the canonical
per-transport layout. Struct fields are pub(crate) so mod.rs can still
construct and read them across the module boundary. Pure relocation; no
logic change.
 2026-07-11 20:26:42 +00:00
Johnathan Corgan 6d6889d0f6 transport: give ethernet a canonical addr.rs home for MAC parsing 
Moves parse_mac_string into ethernet/addr.rs and re-exports it at the
ethernet module root so the ethernet::parse_mac_string path stays
byte-identical for its one external consumer (zero churn there). Its
unit tests stay in mod.rs, calling through the re-export. Pure
relocation; no logic change.
 2026-07-11 20:19:08 +00:00
Johnathan Corgan 196d9492da transport: rename ethernet byte-layer module from socket to io 
Normalizes the ethernet transport onto the canonical byte-layer name
(io.rs), including the per-OS files io_linux.rs/io_macos.rs and their
#[path] wiring. Pure file rename plus module-path and doc-comment
updates; no logic, wire, config, metric, or log change.
 2026-07-11 20:14:37 +00:00
Johnathan Corgan 0f2e91b479 transport: rename udp byte-layer module from socket to io 
Normalizes the udp transport onto the canonical per-transport module
layout (ble is the reference: io/pool/addr/stats). Pure file rename plus
module-path updates at the use sites; no logic, wire, config, metric, or
log change. Behavior identical.
 2026-07-11 20:10:19 +00:00
Johnathan Corgan 775c7ae4c0 Merge branch 'refactor-transport' into refactor-next 2026-07-11 18:46:00 +00:00
Johnathan Corgan 32475d859e transport: consolidate tor and nym SOCKS5 dialing into a shared module 
The Nym transport was a near-clone of the Tor transport's outbound
SOCKS5 path. Extract the shared logic into src/transport/socks5/ so both
transports drive one implementation instead of two maintained copies:

- share one SOCKS5 mock server between the tor and nym tests
- extract the common send/receive/connect counters into a shared
  ProxiedStatsBase (snapshot structs and emitted metrics unchanged)
- add a shared Socks5Dialer that collapses all six connect variants;
  the sole dialing difference (tor's per-destination circuit-isolation
  auth vs nym's no-auth) is an enum on the dialer
- route both tor and nym dialing through the shared dialer
- share the proxied connection pool, generic over a per-connection meta
  type that carries tor's inbound/outbound direction counting (nym uses
  the unit type)
- share the proxied receive loop

Behavior-neutral: no wire-format, config-key, emitted-metric, log, or
error-variant change. Tor keeps its control-port, inbound/onion, and
directory surface and its own address validation.
 2026-07-11 18:45:41 +00:00
Johnathan Corgan 14edddc14a Merge branch 'master' into next 2026-07-11 05:05:22 +00:00
Johnathan Corgan f2e6b8befb transport: promote FMP stream framing to transport::framing 
The FMP frame-boundary reader lived in transport/tcp/stream.rs, but the
Tor and Nym transports both reached across module boundaries to
'use crate::transport::tcp::stream::read_fmp_packet', a layering smell:
the reader is a shared stream-framing utility, not a TCP-private one.
Move the file to transport/framing.rs and repoint the tcp/tor/nym use
sites, removing the tor->tcp and nym->tcp dependencies. Behavior
unchanged.
 2026-07-11 04:32:38 +00:00
Johnathan Corgan 1aacdfa086 transport: share connection-pool counters via PoolCounters 
TcpStats and TorStats each carried an identical pair of pool_inbound/
pool_outbound atomics plus the same five record_pool_* / pool_inbound_count
methods over them. Move the counter logic into a shared PoolCounters
struct (new transport/stats_common.rs) embedded as a 'pool' field in both.
The public record_pool_* methods stay as thin delegators so all call
sites and the flat pool_inbound/pool_outbound snapshot fields are
unchanged; only the duplicated atomic bookkeeping is now single-sourced.
 2026-07-11 04:30:19 +00:00
Johnathan Corgan a7dfe47663 transport: derive Serialize for EthernetStatsSnapshot 
EthernetStatsSnapshot derived only Clone/Debug/Default, unlike every
other transport's *StatsSnapshot which also derives Serialize. That gap
forced a hand-rolled serde_json::json!{} arm in TransportHandle::
transport_stats() that re-listed all ten fields by hand. Add the
Serialize derive and collapse the arm to the same one-line
serde_json::to_value(...) form the other transports use. The emitted
JSON keys and values are unchanged.
 2026-07-11 04:27:50 +00:00
Johnathan Corgan 81baeebf22 Merge branch 'master' into next 2026-07-11 02:15:24 +00:00
Johnathan Corgan 8aab71af86 bench: add routing next-hop microbench 
Measures the per-forwarded-packet cost of routing candidate assembly
(routing_candidates over a synthetic RoutingView) against a zero-alloc
reference across 8/32/128/256 peers, with per-call allocation counts via
a counting allocator. Criterion harness; no production code change.
 2026-07-11 01:48:34 +00:00
Johnathan Corgan 2cffc10520 Merge branch 'maint' (bloom single-digest double-hashing) 
Forward-merges the bloom SHA-256-once fix from maint. The bloom filter was
relocated to proto/bloom/ by the sans-IO refactor, so the fix applied cleanly
to proto/bloom/core.rs; the behavior-neutral test was re-homed into
proto/bloom/tests/core.rs (maint carried it in the pre-split bloom/tests.rs).
 2026-07-11 01:32:48 +00:00
Johnathan Corgan b9eb725e5a Merge FMP wire + PromotionResult relocations into next 
Forward-merge the master-line FMP-establish relocations onto next: the FMP
link wire codec into proto/fmp/wire.rs and PromotionResult into
proto/fmp/core.rs. The wire move was hand-resolved against next's wire
format (msg3/Msg3Header/build_msg3, FMP_VERSION=1, no spin-bit flag), so
next's behavior is unchanged. Behavior-neutral; full lib suite green at
next baseline.
 2026-07-10 17:34:49 +00:00
Johnathan Corgan 1c1ed0d939 Relocate PromotionResult into proto/fmp 
Move the PromotionResult enum (and its impl) out of peer::mod and into
proto/fmp/core.rs, alongside the cross_connection_winner tie-break helper
that was relocated the same way. This is FMP connection-lifecycle result
vocabulary, so it belongs in the FMP subsystem home rather than the peer
module.

Behavior-neutral pure type relocation: consumers import it from
crate::proto::fmp, and the crate-root public path crate::PromotionResult
is preserved via a re-export in lib.rs (mirroring cross_connection_winner).
Full lib suite green at baseline.
 2026-07-10 16:52:18 +00:00
Johnathan Corgan 1c41f73931 Relocate FMP link wire codec into proto/fmp 
Move the FMP mesh-layer wire format (common prefix, encrypted/msg1/msg2
headers, and the build_*/inner-header codec fns) out of node/wire.rs and
into proto/fmp/wire.rs, so the whole FMP wire surface lives with its
subsystem, matching the proto/fsp/wire.rs layout. The wire module becomes
pub(crate) mod wire; callers reach it via crate::proto::fmp::wire.

Behavior-neutral: pure relocation plus import-path rewrites across the
node/peer consumers; no logic change. Full lib suite green at baseline.
 2026-07-10 16:42:17 +00:00
Johnathan Corgan f1ff410fd2 Merge the nostr sans-IO state-machine refactor into the next line 
Forward-merge the AdvertMachine / TraversalMachine / classify_punch_packet
sans-IO extraction. Clean auto-merge; the next-line XX-handshake runtime.rs
deltas sit outside the traversal/replay/election sites.
 2026-07-10 03:28:20 +00:00
Johnathan Corgan 39ad4d2e67 Extract the nostr rendezvous decision logic into sans-IO state machines 
Carve the nostr rendezvous engine's decision logic out of the async
NostrRendezvous driver into synchronous, clock-injected cores, following
the failure_state.rs pattern: sync state behind std::sync::Mutex, time
passed in as now_ms, no .await and no I/O in the core. The driver performs
all relay/socket I/O and executes the returned plans.

- AdvertMachine (src/nostr/advert.rs): advert publish/cache/fetch/prune
  decision logic; the driver executes a returned PublishPlan.
- TraversalMachine (src/nostr/traversal_machine.rs): the engine-scoped
  cross-session decision state -- in-flight-initiator dedup, the dual-init
  responder suppression election, and the replay/seen-session cache.
- classify_punch_packet (src/nostr/traversal.rs): a pure classifier for
  the punch recv loop's packet branch table.

The driver keeps all I/O, the NAT punch send cadence, the offer-slot
admission semaphore, and the pending-answer oneshot routing. Behavior-
neutral: no wire, config-key, or metric change. Adds unit coverage for
the advert plans, election ordering, replay eviction, initiator dedup,
and punch classification.
 2026-07-10 03:28:11 +00:00
Johnathan Corgan 6954ebbf34 Merge the nostr rendezvous reorg into the next line 
Forward-merge the src/nostr / src/mdns rendezvous reorganization
(relocation out of src/discovery/, the Discovery->Rendezvous rename, the
RendezvousDriver consolidation, and the trace-target reference updates)
onto the next branch. The sole conflict was in the LAN poll method's
doc-comment and signature: kept next's XX-handshake wording and applied
the lan_rendezvous rename. Merged tree builds clean; next lib suite 1583
passing, fmt/clippy clean.
 2026-07-09 23:48:53 +00:00
Johnathan Corgan 4d2504f59d Update trace-target references for the relocated rendezvous modules 
Moving the nostr rendezvous engine to src/nostr/ (and mDNS to src/mdns/)
changed the module-path-derived tracing targets from
fips::discovery::nostr::* to fips::nostr::*. Update the RUST_LOG filters
in the NAT and mesh-lab test compose files and the resolve-peers-via-nostr
tutorial's debug recipe to the new targets so trace configs and the
documented journal-watch command keep emitting the intended lines.

Without this, the stun-faults suite's Phase-0 pre-flight (which greps the
daemon journal for the debug-level "STUN observation succeeded" /
"traversal: initiator STUN observed" lines) saw those lines suppressed —
the daemon behaved correctly, but the stale RUST_LOG target hid the
evidence. Test-harness and docs only; no source or behavior change.
 2026-07-09 23:20:28 +00:00
Johnathan Corgan 1208f6a5c2 Consolidate the nostr rendezvous driver into src/nostr 
Pull the rendezvous driver state and the movable driver logic out of the
Node struct into the src/nostr home. A new RendezvousDriver owns the
engine handle and the four bookkeeping fields (traversal start time,
startup-sweep latch, adopted bootstrap-transport set and their npubs)
that previously sat loose on Node; Node holds a single driver field and
reaches the same data through thin accessors, including the rx-loop
hot-path protocol-mismatch hook.

The advert build/refresh, the via_nostr fallback-address resolve, the
overlay-endpoint-to-PeerAddress mapping, and the bootstrap request move
onto the driver, taking their Node inputs explicitly (a transport-
endpoint snapshot for advert building) rather than reading Node fields
directly. Transport/connection-table-bound work (traversal adoption,
bootstrap-transport cleanup, the open-discovery sweep, and the outbound
budget calculators) stays on Node as thin glue that calls into the
driver.

Behavior-neutral relocation: statements moved verbatim, no logic, wire,
config-key, or metric changes. lifecycle.rs shrinks ~230 lines. cargo
fmt/build/clippy clean; lib suite 1547 passing (baseline unchanged).
 2026-07-09 22:13:56 +00:00
Johnathan Corgan 3f80530cc5 Move nostr peer rendezvous and mDNS into dedicated module homes 
Relocate the overlay peer-rendezvous subsystem out of the overloaded
src/discovery/ tree into two focused, independent homes: src/nostr/
(relay-mediated overlay endpoint advertise/resolve/auto-mesh plus NAT
traversal) and src/mdns/ (link-local DNS-SD rendezvous). The two
subsystems are independent, so they get separate homes rather than
sharing one.

Drop the ambiguous "Discovery" stem from their identifiers in favor of
"Rendezvous": NostrDiscovery -> NostrRendezvous, LanDiscovery ->
LanRendezvous, and the matching config, policy, field, and method names.
The former src/discovery.rs handoff types (EstablishedTraversal,
BootstrapHandoffResult, the punch-packet helpers) fold into
src/nostr/handoff and stay reachable via the crate-root re-exports.

Pure relocation and rename: no logic, wire-format, config-key, metric,
or tracing-target changes. The operator-facing node.rendezvous.nostr.*
and node.rendezvous.lan.* config keys and the fips-overlay-v1 advert
namespace are byte-identical. cargo fmt/build/clippy clean; lib test
suite 1547 passing (baseline unchanged).
 2026-07-09 21:51:54 +00:00
Johnathan Corgan d2331ecb78 Merge refactor-sans-io: disambiguate the "discovery" name 
Forward-merge the mesh-lookup "discovery" disambiguation onto the next
line: the lookup handler module and Node.lookup field renames, the lookup
metric types, the dual-emitted lookup/discovery metric family, and the
node.discovery -> node.lookup/node.rendezvous config split with its
deprecation shim.

Conflicts resolved by keeping next's XX handshake path (the inbound
responder stores pending_inbound for msg3 rather than promoting on msg1)
and renaming its reset_discovery_backoff call sites to reset_lookup_backoff;
the CHANGELOG Unreleased entries from both lines are combined.
 2026-07-09 15:43:57 +00:00
Johnathan Corgan 3b401a0cbd Disambiguate the mesh-lookup "discovery" name across the source tree 
The identifier "discovery" named three unrelated subsystems; the FMP
overlay coordinate-lookup subsystem is now consistently "lookup". This
finishes the concept-#1 rename across the shell, config, and metric
layers left after the earlier proto-layer rename:

- Handler module node::handlers::discovery -> node::handlers::lookup, and
  reset_discovery_backoff -> reset_lookup_backoff.
- The Node lookup-engine field Node.discovery -> Node.lookup, renamed by
  resolved binding so the metrics().discovery and node.discovery config
  paths are left untouched.
- The lookup metric types DiscoveryMetrics -> LookupMetrics,
  DiscoveryStatsSnapshot -> LookupStatsSnapshot, and Metrics.discovery ->
  Metrics.lookup.

Two surfaces cross a stability boundary and ship behind a compatibility
window, both marked in-code for removal at the v2 cutover:

- The control-socket metric family is dual-emitted under both "discovery"
  (deprecated alias) and "lookup" so existing dashboards keep working.
- The node.discovery.* config table is split into node.lookup.* (mesh
  lookup scalars) and node.rendezvous.* (nostr/lan peer rendezvous).
  NodeConfig does not deny unknown fields, so a naive rename would make a
  deployed node.discovery: block deserialize into nothing and silently
  revert every setting to default. A deprecated all-Option
  DiscoveryConfigCompat field captures a legacy block and a new post-parse
  Config::normalize_deprecated_keys pass folds it into the new tables with
  a one-time deprecation warning.

Flip the packaged fips.yaml templates to the new keys, add legacy/new/
scalar compat parse tests, and record the split and deprecations in the
CHANGELOG. Behavior-neutral; fmt/clippy clean, lib suite green.
 2026-07-09 15:43:31 +00:00
Johnathan Corgan 1ec7a4c680 Merge refactor-sans-io: relative-tolerance powi test drift into the next line 2026-07-09 01:30:28 +00:00
Johnathan Corgan 0b2212e1e8 proto: bound powi test drift by relative tolerance, not ULP 
The previous portability fix bounded the powi test's drift from std::powi
at an absolute ULP count, but Windows/MSVC drift grows with the exponent
(3 ULP by exp=14), so no fixed ULP bound is portable.

Replace the ULP oracle with a loose relative-tolerance sanity sweep
(1e-11), which any sane libm clears by ~1000x regardless of exponent
while still catching a grossly wrong impl. Add a golden-bit pin for the
exp=14 case that exposed the growth. The powi function is unchanged; the
golden-bit determinism pins already passed on Windows.
 2026-07-09 01:30:18 +00:00
Johnathan Corgan 0c948bd707 Merge refactor-sans-io: portable powi guard test into the next line 2026-07-09 01:20:00 +00:00
Johnathan Corgan b2ce7cd3c8 proto: make the powi guard test portable across platforms 
The powi guard test asserted our square-and-multiply result was
bit-for-bit equal to f64::powi, which failed the Windows unit-test job by
one ULP. f64::powi is not portably bit-stable: Linux and macOS lower it
to compiler-rt's __powidf2, but Windows/MSVC rounds differently.

Our square-and-multiply is pure IEEE-754 f64 multiplication and is
therefore deterministic across every platform, which is the property that
actually matters for mesh nodes to agree on bloom FPR and backoff timing
regardless of OS. Replace the std bit-equality assertion with golden-bit
pins on our own output (locking that cross-platform determinism) plus a
bounded-drift sanity sweep against std::powi. The powi function itself is
unchanged.
 2026-07-09 01:19:49 +00:00
Johnathan Corgan c5d48e5ef4 Merge refactor-sans-io: rename proto/discovery to proto/lookup into the next line 
Forward-merge the mesh discovery module rename (directory, exported types, and
internal lookup-stem terminology) onto the next wire-format line. next's
divergent lookup wire and core content is preserved under the new proto/lookup
path, with the same terminology sweep applied to its next-only lines.
 2026-07-08 21:54:24 +00:00
Johnathan Corgan e3e03f6a5d proto: rename the mesh discovery subsystem module from discovery to lookup 
Rename src/proto/discovery to src/proto/lookup and bring the module's naming
onto the lookup stem, matching its concept: a mesh lookup of a node's
coordinates from its pubkey, sent into the mesh as a bloom-filter-guided
multicast request that returns a unicast response with the coordinates.

- the module directory and its declaration (proto::discovery -> proto::lookup)
- exported types: DiscoveryAction -> LookupAction, DiscoveryBackoff ->
  LookupBackoff, DiscoveryForwardRateLimiter -> LookupForwardRateLimiter,
  MAX_RECENT_DISCOVERY_REQUESTS -> MAX_RECENT_LOOKUP_REQUESTS, and the Discovery
  state struct -> Lookup
- internal terminology: doc comments, the "overlay-lookup" phrasing, the
  empty_discovery/suppressing_discovery test helpers, and the disc
  parameter/variable all take the lookup names
- the already-lookup-named wire types (LookupRequest/LookupResponse) are unchanged

Behavior-neutral: no wire bytes or decision logic change. Two references are
intentionally kept as "discovery": the still-named node::handlers::discovery
shell module and the node.discovery.* config keys, which belong to the broader
disambiguation of the shell, config, and metric surfaces still to come.
 2026-07-08 21:54:24 +00:00
Johnathan Corgan 2b009196b5 proto: no_std hygiene for the fmt/alloc imports and the filter math 
Bring the proto tree closer to a std+alloc-only posture, behavior unchanged on
every decision path:

- Sweep the remaining std::fmt and std::collections imports over to core::fmt
  and alloc::collections across the wire codecs and their tests. Subsystem files
  that shadow the core name with a child core module use the leading-colon
  ::core::fmt form. Imports only.

- Replace the two f64::powi calls (bloom false-positive-rate, FMP backoff timer)
  with a shared core-only square-and-multiply helper in proto/math, bit-identical
  to std::powi (a guard test pins this against every exponent the codecs reach),
  and route the diagnostic estimated-count natural log through libm::log. The FPR
  reject decision and the backoff timer are bit-for-bit unchanged; only the
  debug-only count estimate may differ by at most one ULP.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan 5d13090d8f proto: add a bounds-checked byte reader/writer and adopt it across the wire codecs 
Introduce a shared proto/codec module with a cursor Reader (short reads fail with
MessageTooShort { expected: position + needed, got: total }) and an append Writer,
and adopt them across the seven subsystem wire codecs, replacing the repetitive
manual slicing, try_into, and from_le_bytes extraction. Each existing length check
maps to the reader (an up-front minimum becomes require at position zero, so the
per-field expected values, including the tree-announce expected 99, are reproduced
exactly); the bloom exact-length check stays explicit since it also rejects
over-long payloads. Encoded bytes and decode decisions are unchanged.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan 6538731176 proto: replace the string Malformed error with typed variants and drop thiserror 
Replace Malformed(String) with a no_std-clean set: Malformed(&static str)
for the static decode diagnostics, BadSizeClass { got, max } for the two
bloom size-class checks, and typed sources BadCoord(CoordError) /
BadBloom(BloomError) for the coordinate and bloom construction failures. Add a
dedicated CoordError so proto/coord no longer depends upward on stp TreeError,
resolving the temporary inversion from the coordinate relocation. Drop the
thiserror derive from the four proto error types (Error, TreeError, BloomError,
CoordError) in favor of hand-rolled core::fmt::Display and core::error::Error
impls. thiserror remains in use elsewhere in the crate. Wire decode decisions
are unchanged; only the error type and its diagnostic text change.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan 9697026c81 proto: share a per-address rate limiter and backoff helper across subsystems 
Hoist the two line-for-line-identical per-destination minimum-interval limiters
(discovery forward, routing error) into a shared PerAddrRateLimiter, and fold
the duplicated exponential-backoff math (discovery originator, FMP retry) into a
shared backoff_ms helper, both in a new proto/rate_limit module. The subsystem
limiters become thin delegating newtypes so should_forward/should_send and the
backoff call conventions are preserved. Behavior is byte-identical.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan a2400d823f proto/stp: extract ParentDeclaration and relocate the coordinate type to a shared proto/coord module 
Two behavior-neutral relocations, wire bytes unchanged:

- Move the ParentDeclaration type and its impls out of state.rs into a dedicated
  declaration.rs, re-exported at the same path, and relocate the inline wire.rs
  test module into tests/wire.rs alongside the other stp unit tests (reusing the
  shared test helpers). wire.rs drops to non-test code only.

- Move TreeCoordinate, CoordEntry, and the coordinate wire codec out of
  proto/stp into a shared proto/coord module (a peer of proto/link), re-exported
  from proto::stp so every existing import path keeps resolving unchanged.
  Coordinate is a shared addressing primitive with many non-stp consumers, so it
  no longer belongs under the spanning-tree subsystem. The codec carries a
  documented temporary dependency on stp::TreeError until a dedicated CoordError
  is introduced.
 2026-07-08 19:00:25 +00:00
Johnathan Corgan dc9334e725 proto/stp: hoist the parent flap/hold-down veto to the shell for a clock-free classify core 
evaluate_parent no longer reads the injected clock: it returns a ParentEval of
Mandatory, Discretionary, or None, and the flap/hold-down veto is applied at the
edge via a new TreeState::is_switch_suppressed(now_ms), gating only the
discretionary arm. classify_announce/classify_periodic take the pre-computed
switch_suppressed bool instead of now_ms, so the whole classify ladder is
clock-free; the shell callers (tree announce/periodic re-eval, MMP first-RTT
re-eval, handle_parent_lost) compute the veto verdict. The no-coords parent
case stays discretionary (veto-gated) exactly as before. Behavior unchanged;
the veto tests now assert the moved responsibility.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan 309a91d293 proto/mmp: split state into per-role modules and dissolve the vestigial mmp shell 
Reorganize the MMP subsystem, behavior unchanged throughout:

- Restore the pre-migration role split: move SenderState into sender.rs,
  ReceiverState (with its GapTracker helper) into receiver.rs, MmpMetrics and
  RrLog into metrics.rs, and PathMtuState into path_mtu.rs, leaving the Mmp
  aggregate plus the peer/session state in state.rs (1339 to 196 lines). Home
  the module constants in a new limits.rs and re-export them at the same paths.
  Pure code-motion with module rewiring; visibility unchanged.

- Dissolve the 97-line src/mmp shell, which held only MmpConfig and the
  monotonic mono_ms() clock: move MmpConfig into config/node.rs (re-exported as
  crate::config::MmpConfig), move mono_ms() into a new top-level src/time.rs
  shell time seam, repoint all callers, and delete src/mmp/. Also correct stale
  src/mmp/ doc-comment path labels to proto/mmp/ where they name the protocol
  primitives.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan c1ddbf053c proto/discovery: home the recent-request cap and inject request_id from the shell 
Two behavior-neutral discovery cleanups:

- Move MAX_RECENT_DISCOVERY_REQUESTS out of the node discovery handler into the
  discovery subsystem limits module and re-export it, keeping the two use sites
  unchanged. Same value and semantics; only its home moves.

- Remove the ambient rand draw from LookupRequest by deleting generate() and
  having the shell draw the random request_id and pass it into the existing
  new() constructor. Same per-request u64 draw, now at the shell, leaving the
  discovery codec free of ambient RNG reads.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan b53db662c3 proto/bloom: add BloomFilter::fpr() and use it for the false-positive-rate 
The false-positive-rate primitive (fill ratio raised to the hash count) was
inlined at three sites. Hoist it into a BloomFilter::fpr() method and call it
from all three; the threshold/policy logic stays at the call sites. Behavior
and result bytes are unchanged.
 2026-07-08 19:00:24 +00:00
Johnathan Corgan c0b2822d56 Merge refactor-sans-io: sans-IO proto cleanup into the next line 
Forward-merge the sans-IO cleanup series (bloom fpr, discovery const/RNG-injection,
mmp state decomposition, mmp shell dissolution, STP clock-free classify core, STP
declaration split, shared proto/coord relocation, shared rate limiter/backoff,
typed proto errors dropping thiserror, shared bounds-checked codec reader/writer,
core/alloc import sweep, no_std-shaped filter math) onto the next wire-format line.

Reconciled master structure against the next wire semantics: the mmp role-module
split adopts next slim report format (spin-bit stays dropped, sender/receiver
build next reports), the discovery and fmp codecs keep next TLV and profile
negotiation while moving to the typed error, and the parent-eval handler keeps the
Full/Leaf profile filter under the new ParentEval/is_switch_suppressed seam.
 2026-07-08 17:52:24 +00:00
Johnathan Corgan fff1d6285a Merge refactor-sans-io: FSP sans-IO extraction and protocol teardown 
Forward-merge the FSP session-protocol sans-IO migration and the src/protocol
teardown from the master-refactor line onto next. Converge proto/fsp across
both lines (core.rs byte-identical), keeping next's born-on-next residue: the
additive SessionSetup/SessionAck wire variants, the spin-bit-less unit
FspInnerFlags, and the XX handshake shell residue (negotiation-payload
piggyback, initiator identity check, min-size length checks) in
handlers/session.rs. src/protocol is removed on next as well; ProtocolError,
LinkMessageType, and SessionDatagram reach their proto/ homes.
 2026-07-08 10:18:18 +00:00
Johnathan Corgan 4ad5940114 proto/fsp: extract FSP session protocol into sans-IO layout, retire src/protocol 
Migrate the FSP end-to-end session subsystem into src/proto/fsp/ following the
established sans-IO shape, and retire the src/protocol grab-bag now that FSP was
its last occupant.

Relocate the FSP session wire (node/session_wire.rs plus the FSP message types
from protocol/session.rs) into proto/fsp/wire.rs. Hoist the pure decision logic
into proto/fsp/core.rs over plain-data SessionSnapshots returning an ordered
FspAction list the shell drives: session-rekey policy, msg3-resend
classification, post-decrypt epoch reaction, setup/dual-init tie-break,
coords/path-MTU emit-policy, bounded pending-queue, and IPv6 ECN. The
crypto-owning SessionEntry stays shell-side in node/session.rs (matching the FMP
ActivePeer pattern); proto/fsp is wire + core + limits only, with no proto->noise
dependency and no crypto.

Move the coords helpers to proto/stp/ (they serialize TreeCoordinate), and split
SessionMessageType: the encrypted-inner 0x10-0x1F variants stay in proto/fsp/wire.rs
while the 0x20-0x2F routing signals become a new RoutingSignalType in
proto/routing/wire.rs. Migrate the session-MMP shell adapter, which continues to
drive proto/mmp/.

Retire src/protocol: LinkMessageType and SessionDatagram move to a new shared
proto/link.rs, ProtocolError becomes proto::Error (relocated verbatim), the
deprecated MessageType alias and the unimported PROTOCOL_VERSION are dropped, and
src/protocol/ is deleted along with its lib.rs module declaration.

Behavior-neutral: wire bytes unchanged, oracle tests pass unedited except
mod-path relocation; adds rekey/epoch characterization tests and pure
poll/emit-policy core tests.
 2026-07-08 04:57:00 +00:00
Johnathan Corgan 45f65e7ca1 Merge refactor-sans-io: converge next bloom onto v1-sans-IO 
Forward-merge the master-line bloom relocation into the next line,
discarding next's v1.5 bloom draft (RLE codec, XOR-diff delta,
FilterNack/0x21, adaptive sizing) and converging both lines onto the
identical proto/bloom v1-sans-IO module. This is a deliberate,
temporary wire regression on the next line: the v2 bloom is rebuilt
fresh, sans-IO from day one, on both lines later. Nothing outside the
bloom feature depended on the v1.5-specific surface. proto/bloom is now
byte-identical across both integration lines.
 2026-07-07 23:22:22 +00:00
Johnathan Corgan 4ed674ea8b proto/bloom: relocate bloom filter into sans-IO layout 
Migrate the v1 bloom filter subsystem into src/proto/bloom/, matching the
discovery/routing/fmp/mmp/stp reference layout and completing the proto/
relocation series for the data/wire subsystems.

- wire.rs: the FilterAnnounce (0x20) codec, moved from protocol/filter.rs
- core.rs: the pure BloomFilter algorithm (hash/insert/contains/merge/
  as_bytes/from_bytes/estimated_count), moved from bloom/filter.rs
- state.rs: BloomState (per-peer inbound store, compute_outgoing_filter,
  the injected-clock send debounce), moved from bloom/state.rs
- limits.rs: the v1 sizing constants
- mod.rs: module wiring + the BloomError enum
- tests/: the unit suite split by target (core/state/wire), no inline tests

no_std+alloc hygiene: core::fmt over std::fmt, the tracing dependency
dropped from the pure filter, and std collections replaced with
BTreeMap/BTreeSet (NodeAddr: Ord) for deterministic iteration. The pure
filter combination stays a BloomState method; the two irreducible shell
gathers (peer_inbound_filters, build_filter_announce) remain in the async
shell. Wire bytes and observable behavior are unchanged; full local CI
green (36/36) including the bloom-storm chaos gate.
 2026-07-07 23:10:12 +00:00
Johnathan Corgan 6a028064ef Merge refactor-sans-io: STP sans-IO spanning-tree on the next line 
Forward-merge the STP sans-IO migration. The classify / proto-stp structure,
clock injection, crypto field-partition, and BTree collections come from the
master line; next's non-full/leaf parent-candidacy skip is preserved by passing
self.non_full_peers() at the four shell call sites (handle_tree_announce,
check_periodic_parent_reeval, the MMP first-RTT re-eval, and the greedy-tree
find_next_hop fallback) instead of an empty set, with non_full_peers returning a
BTreeSet. next's four non_full_peers skip tests fold into proto/stp/tests.
 2026-07-07 17:22:25 +00:00
Johnathan Corgan a67801099d proto/stp: sans-IO spanning-tree state machine 
Migrate the full non-async spanning-tree surface into proto/stp/, mirroring the
discovery/routing/fmp/mmp conversions. The classification ladder (parent-switch /
self-root / loop-drop / ancestry-update / periodic-rebroadcast / parent-lost) moves
out of the async node handlers into a pure Stp classify layer returning a
TreeDecision the shell drives, with effect ordering and per-arm invalidation
preserved verbatim. src/tree/ relocates wholesale: TreeState + ParentDeclaration data
+ coordinates into proto/stp/{state,coordinate}, the flap-dampening / hold-down
cluster into a FlapDampener in limits.rs, and the wire codec into wire.rs. The clock
is injected as u64 (wall-clock secs for the escaping declaration timestamp, monotonic
ms for the dampening timers via mmp::mono_ms); declaration crypto is field-partitioned
so sign/verify/hash run in the shell while the in-core modules carry data +
signing_bytes only. Peer maps/sets move to BTree; core/state/coordinate/limits are
core+alloc clean, with wire.rs the one std-tethered file. Behavior-neutral:
characterization tests added for the handler decision arms; convergence suite and
ci-local (36/36) green.
 2026-07-07 17:07:33 +00:00
Johnathan Corgan 584edfac20 Merge refactor-sans-io: MMP sans-IO reporting on the next line 2026-07-07 11:30:40 +00:00
Johnathan Corgan 50a595a0ed proto/mmp: sans-IO metrics-reporting state machine 2026-07-07 09:24:17 +00:00
Johnathan Corgan 3b46bbe17d Merge refactor-sans-io: FMP sans-IO connection-lifecycle on the next line 2026-07-07 07:32:02 +00:00
Johnathan Corgan 4802792e38 proto/fmp: sans-IO connection-lifecycle state machine 2026-07-07 06:20:02 +00:00
Johnathan Corgan 78eadbcd65 Merge refactor-sans-io: routing sans-IO on the next line 2026-07-07 04:29:58 +00:00
Johnathan Corgan 9ea57b483a proto/routing: sans-IO transit + hop-selection state machine 2026-07-07 04:12:05 +00:00
Johnathan Corgan 74ab6d58ba Merge refactor-sans-io: reconcile discovery sans-IO core with the FMP v2 delta 
Forward-merge the master-side sans-IO refactor (discovery migration + no_std
reductions, collapsed to one commit) into the next-side branch. The discovery
decision core meets next's FMP profile/LookupRequest delta here: next's four
transit-forward predicates (Leaf no-forward, Full-profile, min_mtu, tree/fallback)
are folded into the pure core planners via an extended RoutingView seam
(node_is_leaf / peer_is_full / peer_meets_mtu, new ForwardOutcome::LeafNoForward),
and next's v2 LookupRequest API delta (origin_coords removed, tlv_entries added)
is reconciled across the discovery test tree, with next's four TLV wire tests
ported into the relocated wire test module.

The master-side branch now also carries the no_std+alloc reductions (BTreeMap,
alloc::sync::Arc, backoff-reset log moved to the shell, extern crate alloc),
which come through cleanly on the pilot-only core files.

Validated: cargo fmt / clippy (-D warnings) / test --lib all green, including
next's own transit MTU-pruning integration tests driving the refactored core
through the full shell path.
 2026-07-05 22:01:06 +00:00
Johnathan Corgan e03b206f62 proto/discovery: sans-IO state-machine migration + no_std reductions 
Migrate the FMP discovery decision logic out of the async handlers into
synchronous, runtime-agnostic sans-IO state machines owned by the protocol
structs, with I/O pushed to the edges. Pulls the full decision surface into a
pure core (backoff, rate-limit, planners, response routing), consolidates the
tests into a per-module tree with a shared crate testutil, and injects a u64
wall-clock so the core is free of Instant and std time.

Also brings the module toward no_std+alloc: the four discovery maps use
alloc::collections::BTreeMap (HashMap's RandomState is std-only), Arc is spelled
alloc::sync::Arc, the backoff-reset log lives in the shell (the core returns the
cleared count so observability stays out of the pure core), and the crate root
names alloc directly. The one remaining tether is ProtocolError's
std::error::Error coupling in the wire codec.

First subsystem of the broader sans-IO refactor; establishes the extraction
patterns and conventions carried forward to the remaining protocols.
 2026-07-05 21:59:21 +00:00
Johnathan Corgan bfc1cd780f Merge branch 'master' into next 2026-07-02 22:03:52 +00:00
Johnathan Corgan 1dbfefc9d0 Merge branch 'maint' 2026-07-02 22:03:41 +00:00
Johnathan Corgan ceb60c9976 Merge branch 'master' into next 2026-06-29 16:47:43 +00:00
Johnathan Corgan 793f844448 Merge branch 'maint' 2026-06-29 16:47:43 +00:00
Johnathan Corgan f7a0f906fa Merge branch 'master' into next 2026-06-29 14:23:57 +00:00
Johnathan Corgan 243bd7985a Merge branch 'maint' 2026-06-29 14:23:57 +00:00
Johnathan Corgan 56bdea153b Merge master into next after the v0.4.0 rollover 
Brings the v0.4.0 release line up to date on next without disturbing
the in-flight wire-format work; next keeps its own 0.6.0-dev version.
Linkage merge so later forward-merges land cleanly.
 2026-06-27 18:35:51 +00:00
Johnathan Corgan 30c5808e09 Merge maint into master after the v0.4.0 rollover 
maint carries only its 0.4.1-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
 2026-06-27 18:35:41 +00:00
Johnathan Corgan 330c6aa9af Open 0.6.0-dev cycle on next after v0.4.0 release 
Bump the version to 0.6.0-dev and update the status badge, and correct
the 0.4.0 changelog heading to the actual release date.
 2026-06-27 18:30:26 +00:00
Johnathan Corgan 3c9a629ad4 Open 0.5.0-dev cycle on master after v0.4.0 release 
Bump the version to 0.5.0-dev and update the status badge.
 2026-06-27 18:28:52 +00:00
Johnathan Corgan c8c2ae96eb Merge branch 'master' into next (openwrt .apk WAN-port default for DSA boards) 2026-06-26 03:42:23 +00:00
Johnathan Corgan 5bc42b4deb Merge branch 'master' into next (OpenWrt Blossom upload resilience) 2026-06-21 14:40:50 +00:00
Johnathan Corgan e305e1c4e4 Merge branch 'master' into next (v0.4.0 release content) 
Forward-merge the v0.4.0 content finalize from master. The only master
commit merged touches the CHANGELOG, release notes, and README badge.

CHANGELOG reconciliation: adopt the regrouped, stamped [0.4.0] release
section verbatim from master (a frozen release record, identical across
branches), keep next's top-of-file Breaking block, and retain next's
XX-handshake / XX-rekey items in [Unreleased] as 0.5.0 work (the XX path
does not exist on master, which is IK). Release notes adopt master's
refreshed versions. README stays at v0.5.0-dev on next (master's badge
bump to v0.4.0 is master-line only).

Claude-Session: https://claude.ai/code/session_01A2pYfSypNmmG4HyHwZuLex
 2026-06-21 14:02:23 +00:00
Johnathan Corgan d59e4d18f9 Merge branch 'master' into next 2026-06-20 14:58:16 +00:00
Johnathan Corgan 5b767827bb Merge branch 'master' into next 2026-06-18 15:38:10 +00:00
Johnathan Corgan 82ba8e656f Merge branch 'master' into next 2026-06-17 18:31:13 +00:00
Johnathan Corgan db7ad32db8 Merge branch 'master' into next 2026-06-17 16:43:18 +00:00
Johnathan Corgan ec123107ef Merge branch 'master' into next 2026-06-17 15:40:54 +00:00
Johnathan Corgan 86ec7a37e8 Merge branch 'master' into next 2026-06-14 18:54:23 +00:00
Johnathan Corgan 784e22ab1f Merge branch 'master' into next 
Forward-merge the v0.4.0 pre-release content (dependency refresh, CI
deb-install + AUR-build legs, docs refresh, doc-comment fixes, and the
Phase 4 source-content squash: CHANGELOG, release notes, README,
fips.yaml, Cargo.toml metadata, reference docs) up the one-way flow.

Conflict fixups (keep next's identity, fold in master's improvements):
- Version: keep next's 0.5.0-dev (Cargo.toml/lock).
- README status: keep next's v0.5.0-dev / wire-format-breaking framing
  and the Breaking-section pointer; fold in the Nym transport and the
  "global, public test mesh of thousands of nodes" description.
- docs/reference/cli-fips.md: keep next's 0.5.0-dev version example.
- CHANGELOG: keep next's XX-handshake admission entry (no early cap gate
  on XX) and drop master's IK early-cap-at-handle_msg1 entry, which
  describes IK-only behavior that does not apply on next; take master's
  OR-union mesh-size rewrite (next carries the OR-union code); restore
  the Tor connect_refused and MMP receiver-report entries that the
  Fixed-section consolidation would otherwise have dropped.
- lifecycle.rs: keep the mDNS/LAN handshake doc-comment as Noise XX
  (next unifies on XX), not master's XX-to-IK correction.

Quartet green on the merged tree: fmt, build, clippy -D warnings, and
cargo test --lib (1434 passed).
 2026-06-14 18:24:32 +00:00
Johnathan Corgan 0e5f7c90f4 Merge branch 'master' into next 2026-06-13 23:16:48 +00:00
Johnathan Corgan d97c07fd8d Merge branch 'master' into next 
Carries the Nym mixnet transport + single-container demo.

Forward-merge adaptation: the Nym end-to-end SOCKS5 send/recv test was
authored against master's IK FMP framing (msg1 wire size 114, version
nibble 0). next uses the XX handshake and FMP v1, so the TCP read path
rejected the test frame (UnknownVersion / HandshakeSizeMismatch),
timing the test out. Adapted build_msg1_frame to next's framing (wire
size 41, ver=1/phase=1 header byte). next-only; master keeps its IK
form.
 2026-06-13 19:44:44 +00:00
Johnathan Corgan 65d44ab59a Merge master into next: fipstop SSH/tmux garble fix 2026-06-13 02:55:02 +00:00
Johnathan Corgan 0aada5c156 Merge master into next: demote rekey-abort logs to debug 2026-06-13 01:37:55 +00:00
Johnathan Corgan 2ea215f480 Merge branch 'master' into next: fipstop TUI overhaul and disconnect notify-peer fix 2026-06-12 23:53:54 +00:00
Johnathan Corgan 63b1821542 Merge branch 'master' into next: control read-snapshot plane 
Forward-merge the off-loop control read-snapshot plane (every show_* read
query served from published snapshots; show_* removed from the rx_loop).

Adapt the per-entity MMP projection to next's MMP model, which no longer
carries a spin bit: drop the spin_bit_initiator row field, its tick-side
population, and the spin_bit_role output, so the off-loop show_mmp render
stays byte-identical to next's metrics-only on-loop handler.
 2026-06-10 17:57:22 +00:00
Johnathan Corgan cefdc549f7 Merge branch 'master' into next 2026-06-10 02:43:47 +00:00
Johnathan Corgan 906e01b388 Merge branch 'master' into next 2026-06-10 00:05:23 +00:00
Johnathan Corgan 6aded6e365 Merge branch 'master' into next 2026-06-09 11:33:51 +00:00
Johnathan Corgan 8985ea66db Merge branch 'master' into next 2026-06-08 20:26:30 +00:00
Johnathan Corgan 878c40c91f Merge master into next (single-uplink-leaf tree-attachment re-push fix; macOS resolver ::1 fix) 2026-06-08 18:19:10 +00:00
Johnathan Corgan aed64f6d80 Merge master into next (MMP stale-report rejection; convergence-gate near-budget hold) 2026-06-08 00:19:37 +00:00
Johnathan Corgan 9ce4f2e908 Merge master into next (mesh-size OR-union estimate; log-level hygiene) 
Brings the OR-union mesh-size estimator fix and the per-peer / capacity-cap
log-level demotions forward to next.

Conflict resolution: kept next's XX handshake structure and re-applied the
intent at next's own sites. compute_mesh_size auto-merged (next already
carries the config() accessor). The K-bit cutover promotion log demotion
applied to next's richer index-bearing form in encrypted.rs.

master's log changes lived in handle_msg1's IK-only identity/promotion
blocks, which have no XX equivalent there (next defers identity and
promotion to handle_msg3). next's equivalents were demoted in place:
"Connection promoted to active peer" and "Peer restart detected (epoch
mismatch)" info -> debug, and the duplicate "Inbound peer promoted to
active" removed. The two XX-only restart variants ("during FMP rekey" and
"during promotion") were also demoted info -> debug so the restart-detected
class is uniformly at debug on next.
 2026-06-06 19:32:44 +00:00
Johnathan Corgan a3b9954797 Merge master into next (libclang build-prereq doc + transport mutex-poison recovery) 
# Conflicts:
#	src/transport/ble/discovery.rs
#	src/transport/ethernet/discovery.rs
 2026-06-06 13:40:28 +00:00
Johnathan Corgan 84c690cecd docs: reconcile the next changelog with superseded XX-line work 
Two next-line entries had been left describing states that a later commit
reverted. Collapse the XX max_peers early-gate add/remove churn into its
net final state: on the XX handshake an over-cap inbound connection is
rejected solely by the late promotion check (no early gate), logged at
debug. Replace the stale "rekey jitter disabled on next" note with the
actual state: jitter is re-enabled (REKEY_JITTER_SECS = 15) and the three
XX rekey-path divergence defects that had blocked it are fixed
(authenticated-decrypt cutover instead of bare K-bit, msg3 retransmission
with per-link rekey serialization, and a jitter-aware handle_msg3
session-age partition). Merged-in IK-line entries are left untouched.
 2026-06-05 21:44:37 +00:00
Johnathan Corgan 95b475d59b Merge branch 'master' into next 2026-06-05 21:30:14 +00:00
Johnathan Corgan 1c28c2a736 node: enforce the inbound max_peers cap solely at promotion on XX 
At the inbound peer cap the XX FMP handshake had two cap checks: an early
gate in handle_msg3 and the late promote_connection check. On XX the
peer's identity is not known until the third handshake message, by which
point all three messages have already crossed the wire, so the early gate
saved no wire bytes and governed exactly the same net-new-peer set as the
late check (known and pending-outbound peers return earlier via the
cross-connection paths). Remove the early gate and rely on the promotion
check, and log the resulting MaxPeersExceeded rejection at debug instead
of warn so a cap'd node under sustained inbound pressure does not emit
WARN spam for expected policy rejections.

Re-enable the two previously-ignored inbound-cap unit tests, rewritten to
drive a full XX three-message handshake and assert the path-agnostic
invariant (no promotion over cap, peer count unchanged, no
connection/link/index leak) plus the known-peer reconnect bypass.

Update the admission-cap integration suite's enforcement-event
discriminator to match: drop the removed early-gate string and count the
new debug-level promotion-cap rejection.
 2026-06-05 18:52:37 +00:00
Johnathan Corgan 66db3e14a7 Merge branch 'master' into next 2026-06-05 17:17:14 +00:00
Johnathan Corgan 2bc3a505f6 Merge branch 'master' into next 2026-06-05 04:47:00 +00:00
Johnathan Corgan 97f496afc4 Merge master into next (FMP cutover authenticate-before-promote; next already carries the XX form, no code change) 2026-06-04 21:44:36 +00:00
Johnathan Corgan 40c06b28e5 Merge master into next: FMP rekey msg1 resend cap + rekey-aware link-dead heartbeat (with next msg3-budget suppression clause) 2026-06-04 18:52:30 +00:00
Johnathan Corgan ea3e7f8c21 fmp: fix jitter × XX rekey session divergence, re-enable rekey jitter 
Re-enables the rekey timer jitter on the XX FMP rekey path
(REKEY_JITTER_SECS 0 -> 15), which had been disabled because it produced
reproducible post-rekey routing loss (~50% Phase-5 ping failure) with no
crypto errors. The failures were session divergence: under jitter the two
directions of a link rekey close together in time, and three distinct
defects in the FMP rekey state machine could leave the two endpoints
committed to different Noise sessions, starving the receiver until the 30s
heartbeat dead-timer tore the link down (tree parent loss -> routing
failure). All three are fixed here.

1. Promote on authenticated decrypt, not the bare K-bit. The K-bit-flip
   handler promoted whatever pending session existed the instant the header
   bit flipped; under interleaved rekeys that could be a stale pending from
   an earlier epoch. Trial-decrypt the inbound frame against the pending
   session and promote only if it authenticates, mirroring the FSP cutover
   discipline; deliver that plaintext through the canonical path and leave
   the pending untouched otherwise.

2. Retransmit FMP rekey msg3 until confirmed. FMP sent msg3 once; a lost
   datagram left the responder without the new session. Retain the msg3
   payload and resend over the existing link until a peer frame
   authenticates against the pending or post-cutover current session,
   abandoning after the configured handshake-resend budget (mirrors FSP).
   Also serialize per-link rekeys: do not start a new rekey while one awaits
   cutover or is still retransmitting msg3.

3. Partition the handle_msg3 paths by rekey age. An inbound msg3 on a
   different link took the initial-handshake cross-connection tie-breaker
   when the session was under a fixed 30s old, otherwise the rekey
   responder. A rekey resets the session-age clock, so under jitter a
   rekey-aged session is frequently under 30s and its concurrent rekey msg3
   was swallowed by the cross-connection branch, which discarded the peer's
   rekey session with no pending slot while the peer cut over to it anyway.
   Bound the cross-connection branch by the same jitter-aware age floor the
   responder uses, so the two paths partition with no overlap.

Verified at jitter=15: rekey integration suite 70/70 across repeated runs
locally and on GitHub CI, rekey-accept-off 71/71, rekey-outbound-only
75/75; lib 1369/0, clippy and fmt clean. At zero jitter the acceptance
floor equals the previous 30s constant, so default-cadence behavior is
unchanged.
 2026-06-03 13:43:11 +00:00
Johnathan Corgan 3bf49de01f Merge sole-store context change into the next-side 
Bring the immutable-state single-store change onto the Noise XX line.
The shared NodeContext is now the sole store; this merge applies the
next-only adaptations the master-side change couldn't carry:

- Remove node_profile from the Node struct (next-only field) so it lives
  solely in NodeContext; migrate its readers (tree/bloom/discovery/
  handshake negotiation) onto the node_profile() accessor. The two FMP
  negotiation sites hoist node_profile() into a local to avoid borrowing
  &self while a connection is mutably borrowed.
- leaf_only sets both is_leaf_only and node_profile via the context swap.
- Preserve the XX handshake/rekey structure (no identity-in-msg1; XX/XK
  initiator/responder constructors) while applying the startup_epoch()
  accessor migration.
- Tests: route profile selection through a make_test_node_with_profile
  helper (profile is immutable, set via Config flags) instead of poking
  the removed field.

cargo test --lib 1369/0; clippy -D warnings and release build clean.
 2026-06-02 17:30:15 +00:00
Johnathan Corgan e181df0ac4 node: extract immutable state into a shared context and atomic metric registry 
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, node profile, capability limits).
Source the immutable config and identity reads across the receive hot
path, the XX handshake/session/rekey state machines, and the discovery,
tree, bloom, retry, and lifecycle modules through the context accessors.
Includes the bloom delta/full/NACK/resize and byte-total counters in the
registry. The Node fields and the context are rebuilt in lockstep at every
mutation site.
 2026-06-02 13:05:03 +00:00
Johnathan Corgan 88e48fdc68 Merge the fast-path refactor into the next line 2026-06-02 13:05:03 +00:00
Johnathan Corgan 3b27a1604d Merge branch 'master' into next 2026-05-30 20:03:13 +00:00
Johnathan Corgan 1914b20485 Merge branch 'next' into refactor-hotpath-next 2026-05-30 02:42:32 +00:00
Johnathan Corgan bbd4de9e76 Merge master into refactor-hotpath-next 
Bring master's runtime peer-list refresh, opt-in mDNS LAN discovery, and
the receive-path reject-reason / reloadable-config refactor into the XX
handshake integration branch. The peer-restart-epoch detection and
stale-FSP-session teardown authored against Noise IK are re-authored onto
the Noise XX rekey path: complete_rekey_msg2 now also surfaces the remote
startup epoch, and the stale session is cleared once the XX rekey
completes (after msg3 is sent). Connect-budget and path-refresh work is
threaded through the XX anonymous-discovery branch.
 2026-05-30 02:42:01 +00:00
Johnathan Corgan fccd760ad7 Merge branch 'refactor-hotpath' into refactor-hotpath-next 
# Conflicts:
#	src/node/tests/acl.rs
 2026-05-29 02:38:26 +00:00
Johnathan Corgan a965b1ec67 Merge branch 'refactor-hotpath' into refactor-hotpath-next 2026-05-29 01:23:41 +00:00
Johnathan Corgan b643eb3a89 testing: port admission-cap integration test to the Noise XX handshake 
The admission-cap suite asserted IK wire sizes (inbound Msg1 len 84,
outbound Msg2 len 104) to verify the inbound max_peers cap. On the XX
handshake those sizes differ and Msg2 is always sent before identity is
known, so the IK discriminator matched nothing and the suite failed
despite the cap working correctly.

Replace the wire-size discriminator with a protocol-agnostic cap-holds
invariant: each denied peer must keep re-initiating handshakes
(sustained inbound, size-agnostic), no denied peer may be promoted to an
active session, the node must log cap-enforcement events, and the capped
node must hold exactly max_peers. Raise node-c's handshake logging to
debug in the mesh profile so the early-gate drop is visible alongside the
late-check rejection.

Also record in the changelog that for peers the node also dials, the cap
is enforced by the retained late check rather than the early msg3 drop.
 2026-05-29 00:20:29 +00:00
Johnathan Corgan e1ae261eb2 node: wire the next-side XX handshake and rekey rejection sites 
Adapt the typed RejectReason coverage to the Noise XX handshake: wire the
msg1/msg2/msg3 state-machine rejection sites in handlers/handshake.rs and
the rekey-initiator outbound sites in handlers/rekey.rs, and add the
XX-specific HandshakeStats counters. The shared RejectReason scaffold and
the branch-agnostic clusters come from the merged refactor-hotpath work;
this commit carries only the next-side delta.
 2026-05-28 21:21:45 +00:00
Johnathan Corgan 939e2cbd4f Merge branch 'refactor-hotpath' into refactor-hotpath-next 
# Conflicts:
#	src/node/bloom.rs
#	src/node/handlers/handshake.rs
 2026-05-28 21:21:12 +00:00
Johnathan Corgan 726461c193 node/tests: reword handshake regression-test doc-comments 
Drop an internal tracker reference from the doc-comments of the
should_admit_msg1 and udp.outbound_only rekey regression tests; the
prose now describes the scenario directly.
 2026-05-28 21:13:26 +00:00
Johnathan Corgan cc417a7bef node: free the connection index when XX msg3 processing fails 
The msg3-processing-failure cleanup read our_index from the connection
after removing it from the connections map, so the lookup always returned
None and the allocated index was never released, slowly leaking index
slots across failed XX handshakes. Capture the index before the remove,
matching the idiom used in the other cleanup paths in this file.
 2026-05-28 21:13:26 +00:00
Johnathan Corgan 8e72326ec2 node/tests: mark leaf-nonrouting handshake test as parallel-load flake-class 
The XX leaf-nonrouting rejection test shares the localhost-UDP rcvbuf
overflow failure mode of the other large-network tests under parallel
cargo test --lib load. Mark it #[ignore] for the default run; it stays
runnable with --ignored or --test-threads=1.
 2026-05-28 20:19:50 +00:00
Johnathan Corgan a6e4481c1f Merge branch 'master' into next 2026-05-28 20:19:19 +00:00
Johnathan Corgan 0a84a25db1 Merge master into next (inbound admission cap silent-drop; XX-adapted gate at handle_msg3) 
Carries the inbound max_peers admission-cap silent-drop work from
master across the IK→XX structural boundary.

On master (Noise IK), the gate fires at handle_msg1 after the peer's
identity is extracted from the IK msg1 payload, before Msg2 is built
and sent. Wire savings of Msg2 (~104 B) plus the responder crypto
per cap-denied attempt motivate the gate placement.

On next (Noise XX), peer identity is not learned until msg3, by
which point Msg1, Msg2, and Msg3 have all crossed the wire. The IK
gate's wire-savings rationale does not apply on XX. The equivalent
gate is placed in handle_msg3 after the peer's static-key + signature
verification (identity now known) and before promote_connection
(ActivePeer construction, peers_by_index insert, link transition).
The value on XX is local CPU / allocation savings and cleaner peer-
side semantics: no fake-promotion whose subsequent data frames fail
decryption on this side, replaced by a silent drop the peer's
existing auto-reconnect backoff handles cleanly.

Bypass logic is the same as the IK version: known-active peers
(reconnect / cross-connection / restart) and pending-outbound peers
skip the cap so admitting them doesn't grow peers.len(). The late
cap check inside promote_connection is intentionally retained as
defense-in-depth.

Cleanup ordering for the gate's silent-drop path corrects a latent
issue in the adjacent msg3-processing-failure path: our_index is
captured before the connection is removed so the allocator slot is
actually freed (the pre-existing path's get-after-remove returns
None, leaking the index).

The two IK-shaped unit tests at handle_msg1 are marked #[ignore] on
this branch with reason. Their wire-bytes discriminator ("Msg2 must
NOT be sent at cap") is structurally false on XX, and the XX-shaped
equivalents need full three-message-handshake test scaffolding that
isn't authored here. XX-adapted unit + integration test coverage for
the handle_msg3 gate is a follow-up.

CHANGELOG [Unreleased] entry adjusted to describe the handle_msg3
placement on this branch (preserving the master-side framing on the
mainline-merge half of the entry would have been misleading on this
branch).

Three conflicts auto-resolved: the structural handshake.rs span
(~253 lines, IK identity-tied logic discarded on this branch's XX
msg1 path), and three textual ci-local.sh conflicts (admission-cap
suite list / dispatch added alongside the existing mixed-profile
suite list / dispatch).
 2026-05-27 17:01:59 +00:00
Johnathan Corgan 05124367db Merge master into next (outbound admission gate + mesh-size parent skip) 2026-05-26 17:31:47 +00:00
Johnathan Corgan df342f268a Merge master into next (periodic TreeAnnounce re-broadcast + harness) 2026-05-26 15:11:40 +00:00
Johnathan Corgan 61ac4c6a32 Merge master into next (cross-conn-won decrypt-worker unregister fix) 2026-05-25 17:23:04 +00:00
Johnathan Corgan 49735980b8 Merge master into next (cross-init tiebreaker + tick-arm connect-on-send fix) 2026-05-25 05:13:00 +00:00
Johnathan Corgan 09df743485 Merge master into next (interop harness, mesh-lab harness, forwarding debug log) 2026-05-23 14:48:46 +00:00
Johnathan Corgan 60c42daa82 Merge master into next (FSP rekey overlapping-epoch, drain-erase fix) 2026-05-23 02:06:04 +00:00
Johnathan Corgan d6729086de Merge master into next (data-plane perf overhaul, packaging fixes) 
Forward-merges PR #91 (off-task encrypt/decrypt worker pools, GSO,
connected UDP), the macOS #102 package-integrity fix, and the AUR
#98 fips-dns fix from master.

Six textual conflicts resolved: wire/noise imports, the K-bit-flip
and rekey-cutover handlers, ActivePeer fields, and an advert test
fixture. One semantic conflict: PR #91's worker-pool code on master
referenced FLAG_SP and MmpPeerState.spin_bit, which next removed in
4aded9a2 (spin bit superseded by the MMP receiver-report timestamp
echo; FMP flags bit 2 reclaimed). The worker acceleration and its
fmp_flags round-trip test are kept; the spin-bit references are
dropped to honor next's wire-format change.
 2026-05-21 01:54:47 +00:00
Johnathan Corgan d1b2f31a3b Merge master into next (rekey-test ping retry, platform-warning cleanup, CHANGELOG backfills) 
# Conflicts:
#	CHANGELOG.md
 2026-05-18 01:47:51 +00:00
Johnathan Corgan c80b3eb0e3 Merge master into next (coord cache surgical invalidation) 
# Conflicts:
#	CHANGELOG.md
 2026-05-17 01:12:44 +00:00
Johnathan Corgan 6bb17be8b8 rekey: disable timer jitter on next pending XX-path investigation 
Set `REKEY_JITTER_SECS` to 0 on next. The symmetric per-session
jitter mechanism added on maint/master works cleanly on the IK
FMP rekey path on those branches, but on next's XX FMP rekey
path it produces reproducible post-cutover routing-convergence
failures: ~50% Phase 5 per-pair-ping loss in the rekey integration
suite across multiple runs, with all crypto, transport, and
link-state log assertions still green (no decrypt failures, no
panics, no link teardowns, no rekey msg2 failures, no FSP
decryption failures during rekey).

The XX rekey path itself is sound — pre-merge it had been passing
18/18 across six consecutive CI runs. Isolation experiment
confirms the trigger: with REKEY_JITTER_SECS=0, the same suite
on next passes 70/70 cleanly with FMP cutover counts (18-21)
in line with master's baseline (24). With the constant at 15
and the test suite's 35s nominal rekey interval, FMP cutover
counts climb to 27-40 and Phase 5 fails consistently.

Two attempted fixes to absorb the jitter-induced state churn on
the XX path (post-msg3 addr_to_link restore; optimistic dampening
at msg1 receipt) were tried and reverted — neither restored
Phase 5 reliability. Diagnosis of the underlying XX cutover
state-cleanup behaviour that fails to absorb variable-interval
rekeys is deferred. The const and surrounding draw/redraw
machinery (peer::active::ActivePeer, node::session::SessionEntry,
draw_rekey_jitter at session.rs:20, redraws at the four cutover
sites) are kept in place; flipping the const back to a positive
value is the re-enable path once the investigation lands.

CHANGELOG entry under `### Changed`.
 2026-05-14 21:16:43 +00:00
Johnathan Corgan a014296639 node/tests: adapt rekey-jitter tests to XX HandshakeState signature 
The two rekey-jitter unit tests added on the IK-rekey branches
construct a HandshakeState via:

  HandshakeState::new_initiator(keypair, remote_pubkey)

On next, the XX rewrite changed the initiator constructor to take
only the local keypair:

  HandshakeState::new_initiator(keypair)

XX does not learn the responder's static pubkey until msg2, so
the IK-shaped signature does not apply. The merge from master
brought the test bodies verbatim; this drops the second argument
in both call sites so the tests compile against the XX API. The
test semantics (rekey-jitter range and mean checks) are unchanged
and orthogonal to the handshake-pattern choice.

Same shape as 8a6477b adapting the bootstrap-handoff tests to
XX's three-way handshake.
 2026-05-14 18:59:14 +00:00
Johnathan Corgan 74b08b9b6f Merge master into next (rekey jitter, Phase 5 settle, acl-allowlist, CI concurrency) 
# Conflicts:
#	CHANGELOG.md
 2026-05-14 18:54:05 +00:00
Johnathan Corgan 418c9d61e0 ci: add mixed-profile suite to integration matrix on next 
The `mixed-profile` integration suite (Full / NonRouting / Leaf
node-role mix introduced by the forklift-upgrade work) is wired
into `testing/ci-local.sh` but was silently absent from the GitHub
Actions integration matrix on next. Regressions to node-role
behavior could land via GitHub CI without the suite running.

Adds a matrix entry plus the matching step block (mirroring the
rekey pattern: generate configs + inject config + compose up +
test + log-collect on failure + compose down always). master
correctly omits the suite from both ci-local and ci.yml since
mixed-profile is forklift-only, so this change never forward-merges
to master.
 2026-05-14 15:58:28 +00:00
Johnathan Corgan 358a06be06 Merge master into next (MTU, rekey baseline, CONTRIBUTING) 
# Conflicts:
#	CHANGELOG.md
 2026-05-14 00:02:08 +00:00
Johnathan Corgan 183ad148de Fix XX rekey dual-init race: tie-break also when pending_new_session set 
The dual-initiation tie-breaker in the rekey arm of handle_msg3 (and
the FSP analogue in handle_session_setup) only fired when
rekey_in_progress was true. With Noise IK (one-message rekey on
master/maint) that is sufficient: msg1 IS the rekey, so both sides
being mid-handshake is the only state where the race can fire.

With Noise XX (three-message rekey on next), set_pending_session runs
when the initiator has processed msg2 and sent msg3, and that call
clears rekey_in_progress. Both sides' set_pending_session can run
before either peer's msg3 has landed. Each side then receives the
peer's msg3 in the post-pending state with rekey_in_progress=false,
falls into the "drop because pending_new_session is set" guard, and
discards the peer's handshake. Each side commits its own initiator
session at K-bit cutover. The two sessions use different Noise key
material, so the link breaks asymmetrically after cutover.

Unify both checks into a single tie-breaker that fires when either
rekey_in_progress() OR pending_new_session().is_some() is true. The
existing smaller-NodeAddr rule applies uniformly; abandon_rekey()
already clears both states and returns whichever index needs
freeing.

Mirrored to the FSP rekey msg1 path in handle_session_setup for the
same race shape.

Logging:

- The previously-silent drop in the rekey arm of handle_msg3 logs at
  info as a tie-break decision rather than as a silent drop.
  Pending_new_session is added as a log field on the tie-break
  win/lose lines so a reader can distinguish which of the two race
  states fired.
- Cutover-complete and K-bit-flip log lines gained our_addr and
  their_addr fields so the two endpoints' logs of the same handshake
  can be correlated.
- "Pending session set, awaiting K-bit cutover" lines remain at
  debug per the existing convention of info-for-cutover-completion-
  only.

Verification: under the un-fixed handler, both sides log
"rekey-msg3 drop: pending_new_session already set" within 242
microseconds of each other, with rekey_in_progress=false and
different pending session indices. Failure pattern matches: six of
twenty pairs FAILED post-rekey, all involving the single-peer node.
Six consecutive post-fix CI attempts on the same test green across
rekey, rekey-accept-off, and rekey-outbound-only suites.
 2026-05-13 23:58:10 +00:00
Johnathan Corgan 195b3bad01 Merge master into next (AUR publish fix) 2026-05-12 17:57:25 +00:00
Johnathan Corgan 212c45bd37 Merge master into next after v0.4.0-dev open 
Marker merge to record the master dev-line (now v0.4.0-dev, with
maint's v0.3.1-dev open already incorporated) as known on next
without taking master's tree (next is on v0.5.0-dev). Future
forward-merges from master to next land cleanly on top of this base.
 2026-05-12 13:47:29 +00:00
Johnathan Corgan b0e9627a77 chore: retarget next from v0.4.0-dev to v0.5.0-dev 
Master is now the v0.4.0-dev line; next moves up to v0.5.0-dev for
the wire-format-breaking work staged here.

- Cargo: 0.4.0-dev → 0.5.0-dev
- CHANGELOG Breaking-section header retargeted to v0.5.0; baseline
  bumped from v0.2.x to v0.3.x peers (master is now the v0.3.1-dev
  patch line and v0.4.0-dev minor line, so v0.5.0 breaks against
  v0.3.x, not v0.2.x)
- README: badge v0.4.0--dev → v0.5.0--dev, three Status & Roadmap
  prose references retargeted
- docs/reference/cli-fips.md: example version 0.4.0-dev → 0.5.0-dev
 2026-05-12 13:41:21 +00:00
Johnathan Corgan 7865557f8a Merge master into next after v0.3.0 release 
Brings v0.3.0 release content forward into the next-branch
development line, preserves next's own version state, and updates
operator-facing version references to v0.4.0-dev.

Kept from master:
- docs/releases/release-notes-v0.2.1.md and -v0.3.0.md (cumulative
  archive of shipped releases)
- RELEASE-NOTES.md root mirror at v0.3.0 (tracks the most recent
  shipped release; will be replaced when the v0.4.0 release cycle
  begins on this line)
- CHANGELOG entries: [0.3.0] and [0.2.1] sections inserted under
  the existing Breaking and [Unreleased] sections
- All code, config, test, and documentation updates from master
  (openwrt yaml resync, doc-config IP placeholders, etc.)

Kept from next (resolved against master's release-prep changes):
- Cargo.toml / Cargo.lock at 0.4.0-dev (next's package version)
- CHANGELOG ## Breaking section (next-specific v0.4.0 wire-format
  breaking work) and the empty ## [Unreleased] block for future
  v0.4.0 non-breaking work

Updated to v0.4.0-dev for consistency with Cargo.toml:
- README badge (v0.3.0--dev -> v0.4.0--dev) and status-section
  prose, rewritten to describe v0.4.0 wire-format-breaking work
  on this branch (Noise XX unification, FMP node profiles,
  slimmer MMP, extensible bloom-filter encoding) instead of the
  v0.3.0 testing-and-polishing narrative that applied while
  v0.3.0 was unreleased
- docs/reference/cli-fips.md example version string
 2026-05-11 19:00:48 +00:00
Johnathan Corgan 7807516b8d Merge branch 'master' into next 2026-05-10 22:04:29 +00:00
Johnathan Corgan e5071d74fb Merge branch 'master' into next 2026-05-10 17:02:33 +00:00
Johnathan Corgan 831ce5f741 Merge branch 'master' into next 2026-05-10 02:16:49 +00:00
Johnathan Corgan dd65951244 Merge branch 'master' into next 
# Conflicts:
#	src/node/tests/bootstrap.rs
 2026-05-09 23:22:32 +00:00
Johnathan Corgan c22b50c147 Merge branch 'master' into next 2026-05-09 18:45:05 +00:00
Johnathan Corgan 19b921ec5d Merge branch 'master' into next 2026-05-08 21:42:16 +00:00
Johnathan Corgan 45a8a4cd9b Merge branch 'master' into next 
Forward-merge of 12 master commits past the previous merge
(823b830, master @ 18019bb): dep-audit bumps (rand,
clap, tun, rtnetlink, windows-service, plus the bump-safe
lockfile batch), bloom-storm chaos scenario, control-socket
resolver consolidation, gateway dns.listen default change,
OpenWrt ipk README refresh, gateway tutorial review, Ethernet
MTU rustdoc fix, dead session-variant drop, CHANGELOG prep.

Conflict resolution:

- CHANGELOG.md: both bullets kept under [Unreleased] / Fixed.
  Master's spanning-tree internal-path-propagation fix precedes
  next's tree-ancestry-test determinism entry and the
  responder-Disconnect XX-handshake entry.
- src/protocol/session.rs: kept next's SessionSetup/SessionAck
  variants and rustdoc. Master's drop of those variants suits
  v0.3.0's FSP phase-byte dispatch but is undone by next's
  v0.4.0 wire format, which retains the variants and uses the
  inner msg_type byte for handshake identification.
- src/transport/ethernet/mod.rs: kept next's "interface MTU - 4"
  comment. Master corrected the v0.3.0 3-byte rustdoc; next
  redesigned the framing to a 4-byte header (type/flags/length)
  for shared-media beacons, so master's correction does not
  apply to next's format.

Auto-merged cleanly: Cargo.toml (next's 0.4.0-dev + the new dep
pins from master), Cargo.lock, all gateway docs,
docs/reference/configuration.md, packaging files,
.github/workflows/ci.yml, testing/chaos/sim/* and
testing/ci-local.sh (bloom-storm additions), src/config/*.

Local verification: cargo build --release, cargo test (1252
passed, 4 ignored), cargo clippy -D warnings, cargo fmt --check
all green.
 2026-05-08 18:47:13 +00:00
Johnathan Corgan 823b830289 Merge branch 'master' into next 
Forward-merge of the docs-overhaul squash (5abf9a9) and top-level
README rewrite (18019bb). Conflict resolution:

- README.md: master's rewritten feature lists adopted, with the encryption
  bullets reflecting next's two-layer Noise XX (replacing the IK/XK pair
  master describes for v0.3.0).
- 6 design/reference markdown files (fips-bloom-filters.md, fips-mesh-layer.md,
  fips-mesh-operation.md, fips-session-layer.md, fips-transport-layer.md,
  reference/wire-formats.md): master's reorg taken, next's protocol details
  preserved (XX handshake naming, bloom v2 RLE/delta wire format,
  v2 LookupRequest sizing).
- fips-intro.md modify/delete: accepted master's split into
  fips-architecture.md / fips-concepts.md / fips-prior-work.md, then
  re-applied next's IK/XK -> XX transition and spin-bit removal across
  the relevant split files. Same pass swept docs/reference/security.md,
  docs/design/fips-mmp.md, docs/design/fips-security.md,
  docs/design/fips-nostr-discovery.md,
  docs/design/port-advertisement-and-nat-traversal.md,
  docs/how-to/enable-nostr-discovery.md, and the affected tutorials so
  no IK/XK or spin-bit prose remains in current-state docs.
- Diagram path conflicts: noise-ik-msg{1,2}.svg removed (IK is gone);
  noise-xx-msg{1,2,3}.svg moved from docs/design/diagrams/ to
  docs/reference/diagrams/ to match master's diagram reorg. The
  wire-formats.md image references resolve correctly to the new path.

Local verification: cargo build --release, cargo test (1265 passed,
4 ignored), cargo clippy -D warnings, cargo fmt --check all green.
 2026-05-08 13:45:04 +00:00
Johnathan Corgan 933e9ee6ee Merge branch 'master' into next 
# Conflicts:
#	src/node/bloom.rs
 2026-05-07 14:31:11 +00:00
Johnathan Corgan fb834dc2ca nostr: separate next-branch advert namespace by default 
Bump the default Nostr-discovery advert namespace from
`fips-overlay-v1` to `fips-overlay-v1-next` on next. Master
continues to publish under `fips-overlay-v1`.

Background: next runs FMP-v1 (Noise XX, msg1 33 bytes) which is
wire-incompatible with master's FMP-v0. Until now both branches
defaulted to the same Nostr advert namespace, so a stock
next-branch daemon's open-discovery sweep would happily pick up
master peers' adverts (and vice versa), succeed at the UDP punch,
adopt the socket, and fail every FMP handshake at the version-gate.
The per-peer-mismatch cooldown introduced on master is the safety
net for any case that slips past this default; the namespace
separation is the structural answer.

Three sites updated:

- `src/discovery/nostr/types.rs` `ADVERT_IDENTIFIER` const
  documents why the value is branch-specific.
- `src/config/node.rs` `default_app()` matches.
- `src/discovery/nostr/tests.rs` and the
  `testing/nat/scripts/nostr-relay-test.sh` malformed-advert
  fixture publish under the new namespace so test harnesses see
  the same adverts a real daemon would.

Operators who need cross-branch discovery during a coordinated
rolling upgrade can override `node.discovery.nostr.app` in
fips.yaml back to `fips-overlay-v1`.
 2026-05-06 15:09:16 +00:00
Johnathan Corgan acb750c0d2 Merge branch 'master' into next 2026-05-06 15:03:27 +00:00
Johnathan Corgan 5a80582b0b Merge branch 'master' into next 2026-05-05 14:49:51 +00:00
Johnathan Corgan 60a4353661 Merge branch 'master' into next 2026-05-04 14:02:09 +00:00
Johnathan Corgan 3cbb667655 Regenerate show_bloom snapshot for next-branch bloom v2 stats 
The control-query show_bloom snapshot, baselined on master, mismatches
the next-branch output because the bloom filter v2 work on next added
six new stats fields (deltas_sent, full_sends, nacks_received,
nacks_sent, size_changes, total_compressed_bytes, total_raw_bytes)
and dropped non_v1. The snapshot test correctly flagged this as
schema drift.

The drift is intentional (next's bloom v2 is a deliberate schema
expansion, not a regression), so the right action per the test's own
guidance is to delete the fixture and let the test regenerate it
against next's actual output.

No production-code change.
 2026-05-03 23:51:32 +00:00
Johnathan Corgan 4457587e5f Merge branch 'master' into next 
# Conflicts:
#	src/node/tests/discovery.rs
#	testing/ci-local.sh
 2026-05-03 23:47:16 +00:00
Johnathan Corgan 1c8b8d55e8 Merge branch 'master' into next 2026-05-03 12:32:49 +00:00
Johnathan Corgan b02eaf2c5b Merge branch 'master' into next 
Forward-merges the squashed per-destination TCP MSS clamping work
(master `ae60743`) into next.

Auto-merge of source files clean. Two pre-existing test assertions
on the next-side discovery tests were updated as part of merge
resolution to reflect the target-edge MTU fold introduced by the
merged-in commit:

- test_transit_forwards_when_mtu_sufficient: 1400 → 1280 = min(target-edge 1280, transit 1400)
- test_response_path_mtu_four_node_chain: 1350 → 1280 = min(target-edge 1280, transits 1350+1500)

Resulting next tree is bit-for-bit identical to the pre-squash next
state previously verified by full local CI sweep (29 + 1 next-only
suites pass on `6a8b519` in 23m 41s).
 2026-05-02 17:39:26 +00:00
Johnathan Corgan d457b6d5f6 Merge branch 'master' into next 2026-05-02 02:45:10 +00:00
Johnathan Corgan e7fc4f7d10 Merge branch 'master' into next 2026-05-02 02:14:36 +00:00
Johnathan Corgan 5317735a91 Merge branch 'master' into next 
# Conflicts:
#	src/node/tests/handshake.rs
#	testing/ci-local.sh
 2026-04-30 14:05:59 +00:00
Johnathan Corgan 9c35c7e55d Merge branch 'master' into next 2026-04-29 12:59:10 +00:00
Johnathan Corgan 15753cdc20 Resolve simultaneous-init cross-connection in XX handle_msg3 
After the master→next merge brought PR #53's bootstrap-handoff socket
adoption onto next's XX three-message handshake, three integration jobs
(rekey, nat-cone, nat-lan) regressed: the FMP handshake completed but
post-handshake encrypted frames silently dropped — packets_recv stayed
at 0 and link-dead timeout fired every 30s.

Root cause: in symmetric bootstrap-handoff both sides initiate XX in
parallel, running two concurrent handshakes (each side's outbound paired
with the peer's inbound). Each side's handle_msg2 (immediate response
to its own outbound msg1) ran before the peer's outbound msg3 arrived;
peers.contains_key was false, so the "Normal path" promoted the outbound
connection. When the peer's msg3 then arrived, peers contained the peer
at the same epoch and the request fell through to the
"duplicate handshake from same epoch" branch, which tore down the inbound
link without applying the cross-connection tie-breaker. Both sides ended
up keeping their own outbound session whose Noise key material pairs with
the peer's discarded inbound, so each side's their_index pointed at the
peer's inbound session index that was never registered in peers_by_index.

handle_msg2 already has the symmetric handler for the inverse ordering
(msg3-then-msg2). Add the missing simultaneous-init handler in
handle_msg3: when the existing peer is on a different link from this
msg3's pending_inbound link and the session is fresh (<30s, so this
isn't a rekey), apply cross_connection_winner with this_is_outbound=false.
The larger-node side swaps to the inbound session via replace_session and
updates peers_by_index from outbound_idx to inbound_idx; the smaller-node
side keeps its outbound session and frees the inbound's allocated index.
Both sides converge on the same Noise session pair.

Verified: cargo test --lib (1144 passed), cargo fmt + clippy clean,
nat-cone + nat-lan + symmetric NAT scenarios pass, rekey integration
test 70/70 pairs across all phases.
 2026-04-27 19:39:00 +00:00
Johnathan Corgan 8a6477b7ff Adapt bootstrap-handoff tests to XX three-way handshake 
The two bootstrap-handoff tests were written against the IK two-message
handshake where peer promotion happens after msg2. Under the noise-XX
three-message handshake on this branch, identity is only revealed in
msg3, so peer promotion happens one message later: the responder learns
the initiator's identity in msg3 and only then promotes the peer.

`test_adopted_udp_traversal_completes_handshake`: Previous version ran
two `run_rx_loop` iterations (one per node) and asserted both peers
had been promoted. That works for IK but not XX: after node_a sends
msg1 from `adopt_established_traversal`, node_b's first rx_loop pass
generates msg2 (revealing node_b's identity to node_a), node_a's next
rx_loop pass generates msg3 (revealing node_a's identity to node_b),
and only then does node_b promote node_a. A third rx_loop pass on
node_b is needed.

Restructured the test to use direct `handle_msg{1,2,3}` dispatch via
`packet_rx.recv()` rather than `run_rx_loop` + select cancellation,
matching the pattern already used by
`test_third_peer_can_handshake_via_adopted_transport_socket`. This
sidesteps the issue that `run_rx_loop` does `packet_rx.take()` and a
cancelled future never returns it, so a second rx_loop call fails
with `NotStarted`.

`test_third_peer_can_handshake_via_adopted_transport_socket`: Added
explicit msg3 dispatch on both Alice/Bob and Bob/Colin sub-handshakes
so the responder side actually receives the initiator's identity
before the test asserts peer promotion.
 2026-04-27 17:28:54 +00:00
Johnathan Corgan 5a5dbe36da Merge branch 'master' into next 
# Conflicts:
#	src/node/lifecycle.rs
#	testing/static/scripts/rekey-test.sh
 2026-04-27 16:10:11 +00:00
Johnathan Corgan 64c38b2e26 Merge branch 'master' into next 2026-04-22 03:43:20 +00:00
Johnathan Corgan 3edca4a84f Notify initiator when inbound ACL rejects a Noise XX handshake 
Under Noise XX the responder only sees the initiator's static key
in msg3, so the inbound-handshake ACL check cannot fire until msg3
has already been processed. By then the initiator has received
msg2, passed its own outbound ACL check, and promoted its side of
the peering. Silently tearing down the responder's state left the
initiator as a "connected" peer with no traffic until the
link-dead timeout fired tens of seconds later; the acl-allowlist
integration test on the next branch timed out at the 5s
convergence check on blocked nodes.

The responder now sends an encrypted Disconnect on the
freshly-completed Noise session before tearing down. The initiator
decodes it via the existing handle_disconnect path and cleans up
the zombie peer within one RTT. DisconnectReason::Other is used
instead of SecurityViolation so the wire payload does not name the
ACL mechanism; the behavioural signature (explicit reject right
after handshake) is the only observable leak.

Supporting changes:

- Add send_encrypted_link_message_raw in node/mod.rs that operates
  on a raw NoiseSession + indices without going through self.peers,
  since the peer has not yet been promoted at the reject site.
- Declare the acl test module in node/tests/mod.rs. It had been
  orphaned since PR #50 so none of its tests ran; two of them no
  longer compiled against the current next branch. Fixed the two
  and replaced test_inbound_msg1_denied_by_acl (asserts IK-era
  behaviour that cannot happen under XX) with
  test_inbound_msg3_denied_triggers_disconnect, a full end-to-end
  UDP test covering both the responder cleanup and the initiator's
  Disconnect-driven cleanup.
 2026-04-22 02:51:17 +00:00
Johnathan Corgan bcb87165fa Merge branch 'master' into next 2026-04-22 01:22:54 +00:00
Johnathan Corgan e5e6054229 Merge branch 'master' into next 
# Conflicts:
#	src/bloom/filter.rs
#	src/node/bloom.rs
 2026-04-21 19:42:35 +00:00
Johnathan Corgan 14d2a4f2df Merge branch 'master' into next 
Integrates PR #50 (peer ACL enforcement) into the XX handshake
architecture. ACL enforcement points adapted for XX's deferred
identity learning: InboundHandshake check moves from handle_msg1
to handle_msg3 (responder learns initiator identity), OutboundHandshake
check remains in handle_msg2 (initiator learns responder identity).
Borrow scopes restructured to release connection borrows before
authorize_peer calls.
 2026-04-16 06:11:03 +00:00
Johnathan Corgan fd9a8d132d Merge branch 'master' into next 2026-04-15 17:10:08 +00:00
Johnathan Corgan dab0ed4854 Merge branch 'master' into next 
# Conflicts:
#	testing/ci-local.sh
 2026-04-15 06:35:32 +00:00
Johnathan Corgan 2382c1438f Merge branch 'master' into next 2026-04-14 15:27:12 +00:00
Johnathan Corgan 942cc83878 Merge branch 'master' into next 2026-04-14 14:07:43 +00:00
Johnathan Corgan c2133a0f29 Merge branch 'master' into next 2026-04-14 11:45:59 +00:00
Johnathan Corgan 76a7147b17 Merge branch 'master' into next 2026-04-14 09:34:05 +00:00
Johnathan Corgan 9e93b0b224 Merge branch 'master' into next 2026-04-13 06:37:23 +00:00
Johnathan Corgan 9f539ce3c5 Apply cargo fmt to node module and handshake tests 
Exposed by the rustfmt toolchain-pin fix now that rustfmt runs under
the pinned 1.94.1 toolchain in CI. Mechanical formatting changes:

- src/node/mod.rs: collapse EthernetTransport::new let-binding onto
  one line (fits within width limit)
- src/node/tests/handshake.rs: reorder imports (super::* after
  specific import) and expand two assert_eq! calls to multi-line form
 2026-04-12 09:14:27 +00:00
Johnathan Corgan 5d898490ac Merge branch 'master' into next 2026-04-12 08:39:19 +00:00
Johnathan Corgan fe6f9aaa78 Merge branch 'master' into next 
# Conflicts:
#	src/node/mod.rs
 2026-04-11 17:51:24 +00:00
Johnathan Corgan 44667d9cc2 Update design docs and diagrams for Noise XX and forklift changes 
Update CHANGELOG Breaking section with all v0.4.0 wire format
changes. Comprehensive update to fips-wire-formats.md (handshake,
MMP, FilterAnnounce, discovery, size tables).

Update fips-mesh-layer.md, fips-session-layer.md, fips-intro.md
for IK/XK→XX transition and spin bit removal. Document
disable_routing and leaf_only profile config in
fips-configuration.md. Update fips-transport-layer.md for 4-byte
beacon and unified header. Update fips-bloom-filters.md for delta
compression and variable sizing. Update fips-mesh-operation.md for
node profiles and discovery min_mtu.

Create/update 8 SVG diagrams (XX msg1/2/3, handshake flow, session
setup/ack/msg3, filter-announce, sender/receiver report). Update
README.md Noise pattern references.
 2026-04-11 13:21:47 +00:00
Johnathan Corgan 10122d7d87 Production hardening: unwrap safety, 14 new tests, diagnostics 
Harden unwrap() calls in handler hot paths (handshake, encrypted,
rekey, session handlers) with proper error propagation.

Add 14 tests: profile rejection (6), MMP forward-compat (4),
discovery min_mtu pruning (3), XX duplicate msg1 dedup (1).

Add NodeProfile Display impl with structured tracing for profile
mismatch logging. Expose bloom compression diagnostics
(total_compressed_bytes, total_raw_bytes) in control socket.

Add module documentation for XX identity timing, profile decision
tree, and bloom codec strategy.
 2026-04-11 13:14:17 +00:00
Johnathan Corgan 8b5f1e349f Update .git-blame-ignore-revs with noise-xx format commit 2026-04-11 08:16:01 +00:00
Johnathan Corgan 34f840a77b Apply rustfmt to noise-xx-only code 2026-04-11 08:16:01 +00:00
Johnathan Corgan 59b7ea765b Fix stale discovery success regex in log analyzer 
The log analysis matched "proof verified, caching route" but the
actual log message is "Discovery succeeded, proof verified, route
cached", causing discovery succeeded to show 0 in all sim runs.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan eb8479eb20 Heterogeneous bloom filters: delta compression, variable sizing, adaptive heuristic 
Replace fixed 1KB bloom filters with variable-size filters (512B-32KB)
that adapt to each node's position in the spanning tree.

Core changes:
- Internal storage: Vec<u8> → Vec<u64> for word-level operations
- Delta compression: XOR diff with word-level RLE, sequence-based
  NACK protocol for full retransmit recovery
- Size conversion: fold (large→small) and duplicate (small→large)
  with auto-converting merge for mixed-size filter combination
- Native-size storage: peer filters stored at advertised size for
  full-resolution routing queries, converted only for outgoing filter
- Adaptive sizing: outgoing fill ratio drives step-up/step-down
  between size classes with hysteresis (20%/5% thresholds)
- Filter size decoupled from FMP negotiation: announced dynamically
  in filter updates, bit 7 and TLV field 1 removed from handshake

Wire format: FilterAnnounce gains flags byte (delta bit), base_seq
field, RLE-compressed payload. New FilterNack message (0x21) for
out-of-sequence delta recovery.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 8162d3c237 Minimal shared-media beacons: 4-byte header, strip pubkey, remove BLE exchange 
Redesign shared-media transport framing now that XX replaces IK:

- Ethernet: unified 4-byte header [type][flags][length:2 LE] for all
  frame types, effective MTU now if_mtu - 4
- Beacons: strip 32-byte pubkey (34 → 5 bytes), identity learned from
  XX handshake msg2/msg3 instead of beacon
- BLE: remove pre-handshake pubkey_exchange() and cross-probe
  tie-breaker, both unnecessary with XX
- Discovery: support anonymous connections (pubkey_hint: None) with
  address-based dedup for shared-media transports
- Post-handshake identity hooks in handle_msg2/msg3 for self-detection
  and future allow/deny list filtering (IDEA-0047)
 2026-04-11 08:16:01 +00:00
Johnathan Corgan e06015dd43 Discovery wire format cleanup 
Drop origin_coords from LookupRequest — unused since reverse-path
routing became primary. Saves 2+16*depth bytes per request.

Wire up min_mtu: populated from TUN MTU config (default 1280) at
origination. Transit nodes skip peers whose link MTU is below the
request's min_mtu requirement. path_mtu on LookupResponse was
already wired (transit min() applied).

Add TLV extension to LookupRequest (after min_mtu) and
LookupResponse (after proof). Uses same TlvEntry format as
negotiation. Transit nodes forward TLV bytes verbatim.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 4aded9a238 Remove spin bit, slim down MMP reports with extensibility header 
Delete SpinBitState, FLAG_SP (FMP bit 2), and FspInnerFlags.spin_bit.
Spin bit superseded by MMP receiver report timestamp echo for RTT.
Reclaims FMP flags bit 2 and FSP inner flags bit 0.

SenderReport: 48 -> 20 bytes. Three fields: interval_packets_sent,
interval_bytes_sent, cumulative_packets_sent.

ReceiverReport: 68 -> 54 bytes. Ten decision-driving and diagnostic
fields retained; removed max_burst_loss, mean_burst_loss,
interval_packets_recv, interval_bytes_recv.

Both report types use extensibility header: repurposed 3 reserved
bytes as [format_version:1][total_length:2 LE]. Decoders skip
unknown trailing bytes for forward compatibility. Session-layer
reports updated to match.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan f4278f1dd7 Add FMP node profile negotiation with mixed-profile integration tests 
NodeProfile enum (Full/NonRouting/Leaf) with FMP feature bitfield:
bits 0-2 profile, bits 3-6 MMP wants/provides, bit 7 bloom filter
size negotiable. Bloom size TLV always sent with min=max=1KB.

Config mapping: leaf_only -> Leaf, disable_routing -> NonRouting.
Profile and agreed bloom size stored on PeerConnection and
ActivePeer. Handshake msg2/msg3 carry FMP negotiation payload
with profile validation and bloom size agreement.

MMP report sending gated by profile wants/provides. Parent
selection and routing constraints skip non-full peers. One-way
bloom filters for non-routing peers (F inserts N as dependent).
Leaf mode: single-peer enforcement, suppress tree announces and
discovery forwarding.

Mixed-profile integration test: A(Full) + B(Full) + C(NonRouting)
+ D(Leaf) with 9 connectivity assertions.
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 8357200b0e Remove IK/XK dead code, rename XX methods as sole pattern 
- Delete PROTOCOL_NAME_IK, PROTOCOL_NAME_XK and all IK/XK methods
- Remove Ik/Xk variants from NoisePattern enum
- Rename XX methods to drop xx_ prefix (sole pattern)
- Rename HandshakeMessageType variants from NoiseIKMsg1/NoiseIKMsg2
  to Msg1/Msg2, add Msg3 variant (0x03) for XX 3-message flow
- Fix doc comments with correct XX message sizes and descriptions
- Fix stale SessionSetup/SessionAck handshake payload size comments
 2026-04-11 08:16:01 +00:00
Johnathan Corgan ae0f791dbc Switch FSP handshake from Noise XK to XX 
Replace the 3-message XK handshake with XX for FSP session establishment.
XX requires no prior knowledge of the peer's static key — the responder's
identity is revealed in msg2, the initiator's in msg3.

Key changes:
- session.rs: XX initiator/responder, post-handshake identity verification
  using x-only key comparison (parity-independent for npub compatibility),
  negotiation payload in msg2/msg3 (FSP version [0,0], features=0)
- Rekey: switched from XK to XX for FSP rekey handshake
- timeout.rs: suppress msg1 resends when target peer is already promoted,
  preventing cross-connection session mismatch from duplicate handshakes
- Test template: discovery backoff 3s and handshake timeout 10s for
  faster convergence in integration tests
- Integration test timeouts restored to 45s (ping) and 60s (rekey)

Squashed commits:
- Switch FSP handshake from Noise XK to XX
- Fix integration test convergence by reducing discovery backoff
- Fix cross-connection session mismatch from msg1 resend
- Fix FSP identity verification parity mismatch
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 179689d6f2 Switch FMP handshake from Noise IK to XX with version negotiation 
Replace the 2-message IK handshake with a 3-message XX handshake for
FMP link establishment. XX requires no prior knowledge of the peer's
static key — both identities are revealed during the handshake
(responder in msg2, initiator in msg3). This is the foundation for
the forklift upgrade that enables rolling protocol upgrades.

Changes:
- Noise XX state machine alongside IK/XK (8 unit tests)
- Protocol negotiation payload codec: format byte, packed version
  min/max, 64-bit feature bitfield, TLV extensions (11 unit tests)
- FMP wire format version 0→1, msg3 header/builder, TCP stream framing
- FMP handshake switched to XX: PeerConnection 3-message flow,
  handle_msg1 simplified (no identity), handle_msg2 sends msg3 and
  promotes initiator, new handle_msg3 promotes responder with
  restart/rekey/cross-connection detection
- Rekey handshake switched to XX with negotiation payload hash chain
  fix (decrypt-and-discard in complete_rekey_msg2/msg3)
- Negotiation payload in msg2/msg3 (FMP version [1,1], features=0)
- Debug logging for handshake promotion paths
- Integration test convergence timeouts adjusted for extra round-trip

Squashed commits:
- Add Noise XX state machine alongside IK/XK
- Add protocol negotiation payload codec
- FMP wire format prep: version 1, msg3 header support
- Switch FMP handshake from Noise IK to XX
- Increase convergence timeouts for XX 3-message handshake
- Fix negotiation hash chain desync in rekey handshake
 2026-04-11 08:16:01 +00:00
Johnathan Corgan 9ccaae5044 Create next branch for breaking changes (0.4.0-dev) 
- Bump version to 0.4.0-dev
- Add Breaking section to CHANGELOG above Unreleased
 2026-04-11 08:16:01 +00:00
304 changed files with 50987 additions and 23045 deletions
Expand all files Collapse all files
.git-blame-ignore-revs
+3
View File
@@ -3,3 +3,6 @@
# rustfmt master-only code
e9da598f8ab13de5dea3a1496531d675af6a0b94
# rustfmt noise-xx-only code
fef3d011ab290743a13ae3f9b287ad4ebaa1713b
.github/workflows/ci.yml
+59 -109
View File
@@ -110,6 +110,59 @@ jobs:
restore-keys: |
${{ runner.os }}-cargo-
- run: cargo clippy --all-targets --all-features -- -D warnings
# An optional feature means two source trees, and --all-features lints
# only one of them. The default build is what ships, so lint it
# explicitly: without this stage, code that compiles only with
# `profiling` enabled would pass CI while breaking every release build.
# Mirrored in testing/ci-local.sh — check-ci-parity.sh compares
# integration suites only and will not catch a stage added to one runner
# and not the other.
- name: Clippy (default features)
run: cargo clippy --all-targets -- -D warnings
- name: Build with the tick-body profiler enabled
run: cargo build --workspace --features profiling
# ───────────────────────────────────────────────────────────────────────────
# Android cross-check
#
# FIPS runs on Android as an embedded library — the host app owns the TUN
# (an Android VpnService), so there are no daemon binaries to package, unlike
# the desktop targets. This job only cross-compiles the library for the
# android target to guard the android-only cfg paths (and the `not(android)`
# exclusions) from silently bit-rotting; nothing else in CI compiles them.
# cargo-ndk wires the NDK toolchain, which is required even for a check
# because `ring` compiles C at build time.
# ───────────────────────────────────────────────────────────────────────────
android-check:
name: Android cross-check (aarch64)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install Rust toolchain (+ Android target)
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
target: aarch64-linux-android
components: clippy
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v5
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-android-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Install cargo-ndk
uses: taiki-e/install-action@v2
with:
tool: cargo-ndk
- name: Clippy the library for Android
run: |
export ANDROID_NDK_HOME="${ANDROID_NDK_HOME:-$ANDROID_NDK_LATEST_HOME}"
cargo ndk -t arm64-v8a clippy --lib -- -D warnings
build:
name: Build (${{ matrix.os }})
@@ -251,6 +304,12 @@ jobs:
check_name: Unit Tests Summary
fail_on_failure: false
# The `profiling` feature adds a module, a recorder and a writer thread
# that the default-feature run above never compiles, so its own tests do
# not execute there. Mirrored in testing/ci-local.sh.
- name: Run library tests with the tick-body profiler enabled
run: cargo test --lib --features profiling
# ─────────────────────────────────────────────────────────────────────────────
# Job 2b – Unit tests (macOS)
# ─────────────────────────────────────────────────────────────────────────────
@@ -374,16 +433,6 @@ jobs:
- suite: static-chain
type: static
topology: chain
# ── Rekey integration test ──────────────────────────────────────────
- suite: rekey
type: rekey
topology: rekey
- suite: rekey-accept-off
type: rekey-accept-off
topology: rekey-accept-off
- suite: rekey-outbound-only
type: rekey-outbound-only
topology: rekey-outbound-only
# ── Firewall baseline (fips0 nftables default-deny) ────────────
- suite: firewall
type: firewall
@@ -520,105 +569,6 @@ jobs:
docker compose -f testing/static/docker-compose.yml \
--profile ${{ matrix.topology }} down --volumes --remove-orphans
# ── Rekey integration test ──────────────────────────────────────────────
- name: Generate and inject configs (rekey)
if: matrix.type == 'rekey'
run: |
bash testing/static/scripts/generate-configs.sh rekey
bash testing/static/scripts/rekey-test.sh inject-config
- name: Start containers (rekey)
if: matrix.type == 'rekey'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey up -d
- name: Run rekey test
if: matrix.type == 'rekey'
run: bash testing/static/scripts/rekey-test.sh
- name: Collect logs on failure (rekey)
if: matrix.type == 'rekey' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey logs --no-color
- name: Stop containers (rekey)
if: matrix.type == 'rekey' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey down --volumes --remove-orphans
# ── Rekey + accept_connections=false variant ──────────────────────────
- name: Generate and inject configs (rekey-accept-off)
if: matrix.type == 'rekey-accept-off'
env:
REKEY_TOPOLOGY: rekey-accept-off
REKEY_ACCEPT_OFF_NODES: b
run: |
bash testing/static/scripts/generate-configs.sh rekey-accept-off
bash testing/static/scripts/rekey-test.sh inject-config
- name: Start containers (rekey-accept-off)
if: matrix.type == 'rekey-accept-off'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-accept-off up -d
- name: Run rekey test (accept-off variant)
if: matrix.type == 'rekey-accept-off'
env:
REKEY_TOPOLOGY: rekey-accept-off
REKEY_ACCEPT_OFF_NODES: b
run: bash testing/static/scripts/rekey-test.sh
- name: Collect logs on failure (rekey-accept-off)
if: matrix.type == 'rekey-accept-off' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-accept-off logs --no-color | tail -300
- name: Stop containers (rekey-accept-off)
if: matrix.type == 'rekey-accept-off' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-accept-off down --volumes --remove-orphans
# ── Rekey + udp.outbound_only=true variant ─────────────────────────────
- name: Generate and inject configs (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only'
env:
REKEY_TOPOLOGY: rekey-outbound-only
REKEY_OUTBOUND_ONLY_NODES: b
run: |
bash testing/static/scripts/generate-configs.sh rekey-outbound-only
bash testing/static/scripts/rekey-test.sh inject-config
- name: Start containers (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-outbound-only up -d
- name: Run rekey test (outbound-only variant)
if: matrix.type == 'rekey-outbound-only'
env:
REKEY_TOPOLOGY: rekey-outbound-only
REKEY_OUTBOUND_ONLY_NODES: b
run: bash testing/static/scripts/rekey-test.sh
- name: Collect logs on failure (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-outbound-only logs --no-color | tail -300
- name: Stop containers (rekey-outbound-only)
if: matrix.type == 'rekey-outbound-only' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile rekey-outbound-only down --volumes --remove-orphans
# ── Firewall baseline integration test ─────────────────────────────────
- name: Run firewall baseline integration test
if: matrix.type == 'firewall'
.github/workflows/package-openwrt.yml
+5
View File
@@ -285,6 +285,8 @@ jobs:
"$FILES_DIR/etc/fips/firewall.sh"
"$FILES_DIR/etc/hotplug.d/net/99-fips"
"$FILES_DIR/etc/uci-defaults/90-fips-setup"
"$FILES_DIR/usr/bin/fips-mesh-setup"
"$FILES_DIR/usr/bin/fips-ap-setup"
)
fail=0
for f in "${TARGETS[@]}"; do
@@ -404,6 +406,8 @@ jobs:
./usr/bin/fipsctl
./usr/bin/fipstop
./usr/bin/fips-gateway
./usr/bin/fips-mesh-setup
./usr/bin/fips-ap-setup
./etc/init.d/fips
./etc/init.d/fips-gateway
./etc/fips/fips.yaml
@@ -717,6 +721,7 @@ jobs:
for path in \
usr/bin/fips usr/bin/fipsctl usr/bin/fipstop usr/bin/fips-gateway \
usr/bin/fips-mesh-setup usr/bin/fips-ap-setup \
etc/init.d/fips etc/init.d/fips-gateway \
etc/fips/fips.yaml etc/fips/firewall.sh etc/dnsmasq.d/fips.conf \
etc/sysctl.d/fips-gateway.conf etc/sysctl.d/fips-bridge.conf \
CHANGELOG.md
+219
View File
@@ -5,17 +5,190 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## Breaking
Wire-format breaking changes for v0.6.0. All nodes in a mesh must
run the same major version — these changes are not backward compatible
with v0.4.x or earlier peers.
### Changed
#### Noise XX Handshake (FMP and FSP)
- FMP link handshake switched from Noise IK (2 messages) to Noise XX
(3 messages). Neither side requires prior knowledge of the peer's
static key. Responder identity revealed in msg2, initiator in msg3.
FMP wire version incremented to 1.
- FSP session handshake switched from Noise XK to Noise XX. Same
3-message flow with post-handshake identity verification using
x-only key comparison (parity-independent for npub compatibility).
- Protocol negotiation payload added to XX msg2/msg3 for both layers:
format byte, packed version min/max, 64-bit feature bitfield, and
forward-compatible TLV extensions. Enables rolling protocol upgrades
in future releases.
- FMP msg1 reduced from 106 to 33 bytes (ephemeral key only, no
encrypted static key or DH products).
#### FMP Node Profiles
- Node profile enum (Full, NonRouting, Leaf) advertised in FMP feature
bitfield bits 0-2. At least one side of a link must be Full.
- MMP report flow gated by wants/provides bits (bits 3-6): reports
only sent when the sender can provide and the receiver wants them.
- Non-routing nodes receive bloom filters (one-way) but do not send
them; the full peer inserts their identity as a leaf dependent.
- Leaf nodes enforce single-peer constraint with no tree, bloom, or
transit participation.
#### MMP Report Format
- Spin bit removed. Reclaims FMP flags bit 2 and FSP inner flags
bit 0. Superseded by MMP receiver report timestamp echo for RTT.
- SenderReport reduced from 48 to 20 bytes (3 fields: interval
packets/bytes sent, cumulative packets sent).
- ReceiverReport reduced from 68 to 54 bytes (10 fields retained;
removed max/mean burst loss and interval recv counters).
- Both report types use extensibility header: `[format_version:1]
[total_length:2 LE]` replacing reserved bytes. Decoders skip
unknown trailing bytes for forward compatibility.
#### Discovery Wire Format
- Dropped `origin_coords` from LookupRequest (saves 2 + 16*depth
bytes per request). Reverse-path routing via `recent_requests` is
the primary response mechanism.
- `min_mtu` field wired up in LookupRequest: transit nodes skip peers
whose link MTU is below the request's minimum.
- TLV extension section added to LookupRequest and LookupResponse
after fixed fields. Transit nodes forward TLV bytes verbatim.
#### Nostr-Discovery Advert Namespace
- Default Nostr-discovery advert namespace bumped from
`fips-overlay-v1` to `fips-overlay-v1-next` on the `next` branch.
Master continues to publish under `fips-overlay-v1`. Effect: a
stock `next`-branch daemon's open-discovery sweep no longer
discovers `master` peers, and vice versa — eliminating the
cross-version retraversal storms that arise when both sides
punched a UDP socket via Nostr but cannot complete an FMP
handshake. Operators who genuinely want cross-branch reach (e.g.
during a coordinated rolling upgrade) can override per-daemon
via `node.discovery.nostr.app` in `fips.yaml`. The
`protocol_mismatch_cooldown_secs` defense-in-depth on master is
the safety net against any peer that bypasses this default
(config override, future fork, static-peer config).
#### Shared-Media Beacons
- Ethernet frame header unified to 4 bytes `[type][flags][length:2
LE]` for all frame types. Beacons reduced from 34 to 5 bytes
(pubkey stripped — identity learned from XX handshake).
- BLE pre-handshake pubkey exchange removed. Cross-probe tie-breaker
eliminated (unnecessary with XX).
#### Bloom Filter Wire Format
- FilterAnnounce gains flags byte (delta bit), `base_seq` field, and
RLE-compressed payload for XOR-diff delta compression.
- New FilterNack message type (0x21) for out-of-sequence delta
recovery (triggers full retransmit).
- Filter size decoupled from FMP negotiation: announced dynamically in
filter updates. Bit 7 and TLV field 1 removed from handshake.
- Variable filter sizes (512 bytes to 32 KB) with adaptive sizing
based on outgoing fill ratio (step-up at 20%, step-down at 5%).
## [Unreleased]
### Added
- The receive-path `RejectReason` classification (shipped in 0.4.0) is
additionally wired into the Noise XX handshake cluster
(msg1/msg2/msg3) and the rekey-initiator outbound sites on `next`.
- An optional tick-body profiler behind the new `profiling` Cargo feature,
**off by default**. When enabled, `fipsctl profile tick on [--dir PATH]` /
`off` / `status` starts and stops a capture at runtime with no restart. Each
capture writes one tab-separated file (default `/var/log/fips`, capped at
32 MB) carrying, per ten-second interval, the exact count, max and total for
every step of the rx-loop tick arm, the whole-tick span, and gauges for ticks,
peer count, the gap between successive tick-arm entries and the resulting
arm-starvation delay. With the feature off the instrumentation macro is a pure
pass-through, so a default build contains no timing code on the tick path.
`LogsDirectory=fips` was added to the packaged systemd units so the capture
directory is created and cleaned up declaratively.
- `packaging/debian/build-deb.sh --features <list>` builds the `.deb` with a
Cargo feature list, which is how an instrumented package is produced for a
measurement run. The auto-derived dev Version gains a matching `+<features>`
marker, so a feature build and a default build of the same commit are no
longer indistinguishable: without it the two carry byte-identical versions,
an install of one over the other is an apt no-op, and the running node offers
no way to tell which one it has. The marker sorts above the unmarked build, so
installing a feature build is an upgrade and reverting to the default build is
a downgrade — revert with `dpkg -i` rather than `apt install`. `--features` is
refused together with `--no-build`, which would stamp the marker onto binaries
the features never reached.
### Changed
- Rekey timer jitter is enabled on next's XX FMP rekey path
(`REKEY_JITTER_SECS = 15` at `src/node/mod.rs`), matching the
IK-line behavior on maint/master. It had been temporarily set to
`0` on next because variable-interval rekeys exposed three XX
rekey-path defects that left the two endpoints on divergent Noise
sessions; those defects are fixed (see `### Fixed`), so the
per-session signed jitter over `[-15, +15]` seconds is restored.
`node.rekey.after_secs` is the nominal interval rather than a floor;
mean is preserved.
- On the XX FMP handshake, an over-cap inbound connection is rejected
solely by the late `promote_connection` check, and the resulting
`MaxPeersExceeded` rejection is logged at debug rather than warn so a
saturated node under sustained inbound pressure does not emit WARN
spam for these expected policy rejections. There is no early cap gate:
on XX the peer's identity is not known until the third handshake
message, by which point Msg1, Msg2, and Msg3 have all crossed the
wire, so an early gate would save no wire bytes and would govern
exactly the same net-new-peer set as the late check. The known-peer /
cross-connection bypass — which also covers peers the node is itself
dialing, e.g. configured `auto_connect` peers — is handled by that
late check, since those peers return earlier via the cross-connection
paths and are not subject to the cap.
- The Ethernet transport's per-interface `discovery` flag was renamed to
`listen` (`transports.ethernet.*`) to match the symmetric `announce`
(transmit) / `listen` (receive) neighbor-beacon vocabulary. The old
`discovery:` key is still accepted via a serde alias, so deployed configs
continue to load unchanged; `Config::to_yaml()` re-emits it under the
canonical `listen:` name. Update your `fips.yaml` to `listen:`.
- The mesh-lookup control-metrics family is now emitted under the key
`lookup` in `fipsctl stats metrics` and `show routing`. The former key
`discovery` is still emitted as a deprecated alias carrying identical
counters; update dashboards and alerts to read `lookup`.
- The overloaded `node.discovery.*` config table was split into
`node.lookup.*` (mesh-lookup scalars: `ttl`, `attempt_timeouts_secs`,
`recent_expiry_secs`, `backoff_base_secs`, `backoff_max_secs`,
`forward_min_interval_secs`) and `node.rendezvous.*` (peer rendezvous:
`nostr.*`, `lan.*`). A deployed `node.discovery:` block still loads and is
folded into the new tables with a one-time deprecation warning; migrate your
`fips.yaml` to the new keys.
- `SessionDatagram::decrement_ttl` and `SessionDatagram::can_forward` now match
the forwarder's IP hop-limit semantics: `decrement_ttl` decrements first and
reports false when the result is zero, and `can_forward` is true only at a
TTL of 2 or more.
### Deprecated
- The `discovery` metric-family key (control-socket JSON). It is dual-emitted
alongside the new `lookup` key during a migration window and will be removed.
Migrate dashboards/alerts from `discovery.*` to `lookup.*`.
- The `node.discovery.*` config table. Its keys were split into `node.lookup.*`
(mesh-lookup) and `node.rendezvous.*` (peer rendezvous). A legacy
`node.discovery:` block still applies for now with a deprecation warning and
will be removed; migrate to `node.lookup.*` / `node.rendezvous.*`.
- The Ethernet `transports.ethernet.discovery` flag, renamed to
`transports.ethernet.listen`. The old key is still accepted via a serde
alias and will be removed at the v2 cutover; migrate to `listen`.
### Fixed
- Nostr NAT traversal no longer breaks after the host suspends. The traversal
@@ -31,6 +204,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
suspends or hibernates was affected. Reported in
[#128](https://github.com/jmcorgan/fips/issues/128).
- XX FMP rekey no longer diverges under timer jitter, which unblocked
re-enabling the rekey jitter on next (`REKEY_JITTER_SECS = 15`; see
`### Changed`). With jitter the two directions of a link rekey close
together in time, and three defects specific to the XX three-message
rekey state machine could each leave the endpoints committed to
different Noise sessions — silent session divergence that starved the
receiver into ~50% post-rekey ping loss and a 30-second heartbeat
link-dead teardown (tree parent loss, routing failure) while every
crypto, transport, and link-state gate stayed green. All three are
fixed:
- The K-bit-flip handler promoted whatever pending session existed the
instant the header bit flipped, which under interleaved rekeys could
be a stale pending from an earlier epoch. It now trial-decrypts the
inbound frame against the pending session and promotes only on an
authenticated decrypt, delivering that plaintext through the
canonical path and leaving the pending untouched otherwise — the
same cutover discipline used on FSP.
- The FMP rekey msg3 was sent once, so a lost datagram left the
responder without the new session. The msg3 payload is now retained
and retransmitted over the existing link until a peer frame
authenticates against the pending or post-cutover current session,
abandoning after the configured handshake-resend budget. Per-link
rekeys are also serialized: a new rekey does not start while one
awaits cutover or is still retransmitting msg3.
- The `handle_msg3` cross-connection and rekey-responder paths were
partitioned by a fixed 30-second session-age threshold, but a rekey
resets the session-age clock, so under jitter a rekey-aged msg3 was
frequently under 30 seconds and got swallowed by the
initial-handshake cross-connection branch, which discarded the
peer's rekey session with no pending slot while the peer cut over to
it anyway. The cross-connection branch is now bounded by the same
jitter-aware session-age floor the rekey responder uses, so the two
paths partition with no overlap. At zero jitter the floor equals the
previous 30-second constant, so default-cadence behavior is
unchanged.
- XX rekey dual-initiation race that broke six pair-directions
post-rekey when both endpoints initiated rekey simultaneously.
The `handle_msg3` tie-breaker only fired when `rekey_in_progress`
was still true, but XX's three-message handshake lets both sides
clear that flag (via `set_pending_session`) before either's msg3
lands. The drop-on-pending-session guard then silently discarded
the peer's msg3, each side cut over to its own initiator session,
and the link broke asymmetrically. The tie-breaker now also fires
when `pending_new_session().is_some()`, applying the same
smaller-NodeAddr resolution rule. Mirrored to the FSP rekey msg1
path for symmetry.
- `SessionDatagram` hop-limit handling now follows IP semantics. Delivery to
the addressed node is no longer TTL-gated, and a forwarder decrements before
deciding rather than after, so a datagram that would leave with a TTL of zero
Cargo.lock
Generated
+2 -1
View File
@@ -1074,7 +1074,7 @@ checksum = "9844ddc3a6e533d62bba727eb6c28b5d360921d5175e9ff0f1e621a5c590a4d5"
[[package]]
name = "fips"
version = "0.4.2-dev"
version = "0.6.0-dev"
dependencies = [
"arc-swap",
"bech32",
@@ -1087,6 +1087,7 @@ dependencies = [
"hex",
"hkdf",
"libc",
"libm",
"mdns-sd",
"nostr",
"nostr-sdk",
Cargo.toml
+15 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "fips"
version = "0.4.2-dev"
version = "0.6.0-dev"
edition = "2024"
description = "A distributed, decentralized network routing protocol for mesh nodes connecting over arbitrary transports"
license = "MIT"
@@ -11,12 +11,21 @@ readme = "README.md"
keywords = ["mesh", "p2p", "decentralized", "overlay-network", "nostr"]
categories = ["network-programming", "command-line-utilities", "cryptography"]
[features]
default = []
# Tick-body profiler (`src/instr`). Off by default: enabling it edits 26 call
# sites in the rx loop's hot tick arm, and only a compile-time gate makes the
# default build's neutrality a property of the generated code rather than of a
# runtime check. Build with `--features profiling` for a measurement run.
profiling = []
[dependencies]
ratatui = "0.30"
secp256k1 = { version = "0.30", features = ["rand", "global-context"] }
sha2 = "0.10"
hkdf = "0.12"
ring = "0.17"
libm = "0.2"
rand = "0.10.1"
crossbeam-channel = "0.5"
thiserror = "2.0"
@@ -108,3 +117,8 @@ path = "src/bin/fips-gateway.rs"
[[bin]]
name = "fipstop"
path = "src/bin/fipstop/main.rs"
[[bench]]
name = "routing_next_hop"
path = "benches/routing_next_hop.rs"
harness = false
README.md
+29 -25
View File
@@ -3,7 +3,7 @@
![banner](docs/logos/fips_banner.png)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-1.85%2B-orange.svg)](https://www.rust-lang.org/)
[![Status](https://img.shields.io/badge/status-v0.4.2--dev-green.svg)](#status--roadmap)
[![Status](https://img.shields.io/badge/status-v0.6.0--dev-green.svg)](#status--roadmap)
A self-organizing encrypted mesh network built on Nostr identities,
capable of operating over arbitrary transports without central
@@ -43,9 +43,9 @@ same way it would on a local network.
- **Multi-transport.** UDP, TCP, Ethernet, Tor, Nym, and Bluetooth
(BLE L2CAP) ship today; transports compose on a single mesh and a
node may run several at once.
- **Two-layer encryption.** Noise IK between peers (hop-by-hop) and
Noise XK between mesh endpoints (independent end-to-end), with
periodic rekey for forward secrecy.
- **Two-layer encryption.** Noise XX both hop-by-hop (peer links)
and end-to-end (mesh sessions), with periodic rekey for forward
secrecy and protocol negotiation in the handshake.
- **Nostr-native identity.** secp256k1 / schnorr keypairs as node
addresses; self-generated, no registration, no central authority.
- **IPv6 adapter.** A TUN interface maps each remote npub to an
@@ -112,17 +112,19 @@ tutorial progression starting at
cargo build --release
```
Requires Rust 1.94.1+ (edition 2024). Linux, macOS, and Windows are
supported; transport availability varies by platform.
Requires Rust 1.94.1+ (edition 2024). Linux, macOS, and Windows run as
standalone daemons; Android is supported as an embedded library (the host
app owns the TUN, e.g. a `VpnService`). Transport availability varies by
platform.
| Transport | Linux | macOS | Windows | OpenWrt |
|-----------|:-----:|:-----:|:-------:|:-------:|
| UDP | ✅ | ✅ | ✅ | ✅ |
| TCP | ✅ | ✅ | ✅ | ✅ |
| Ethernet | ✅ | ✅ | ❌ | ✅ |
| Tor | ✅ | ✅ | ✅ | ✅ |
| Nym | ✅ | ✅ | ✅ | ❌ |
| BLE | ✅ | ❌ | ❌ | ❌ |
| Transport | Linux | macOS | Windows | Android | OpenWrt |
|-----------|:-----:|:-----:|:-------:|:-------:|:-------:|
| UDP | ✅ | ✅ | ✅ | ✅ | ✅ |
| TCP | ✅ | ✅ | ✅ | ✅ | ✅ |
| Ethernet | ✅ | ✅ | ❌ | ❌ | ✅ |
| Tor | ✅ | ✅ | ✅ | ❌ | ✅ |
| Nym | ✅ | ✅ | ✅ | ❌ | ❌ |
| BLE | ✅ | ❌ | ❌ | ❌ | ❌ |
On Linux, a source build requires `libclang` — the LAN gateway's
nftables bindings are generated by `bindgen` at build time, which
@@ -210,24 +212,26 @@ testing/ Docker-based integration test harnesses + chaos simulation
## Status & roadmap
FIPS is at **v0.4.2-dev** on the `maint` branch.
[v0.4.1](https://github.com/jmcorgan/fips/releases/tag/v0.4.1) has
shipped; this line carries patch-level fixes for the 0.4.x series. The
core protocol works end-to-end over
FIPS is at **v0.6.0-dev** on the `next` branch.
[v0.4.1](https://github.com/jmcorgan/fips/releases/tag/v0.4.1)
has shipped from `master`; this development line carries
wire-format-breaking work for v0.6.0 — unified Noise XX handshake
at both layers, FMP node profiles, slimmer MMP reports, and an
extensible bloom-filter encoding — that will not interoperate with
v0.2.x, v0.3.x, or v0.4.x peers. The core protocol works end-to-end over
UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test
mesh of thousands of nodes. v0.4.0 added the Nym mixnet transport and
mDNS LAN discovery alongside the existing Nostr-mediated peer discovery,
UDP NAT traversal, peer ACL, and packaging hardening. New wire-format work
continues to be staged on the `next` branch for the subsequent
release line.
mesh of thousands of nodes. See the CHANGELOG `## Breaking` section for the
full list of v0.6.0 wire-format changes in flight.
### What works today
- Spanning-tree construction with greedy coordinate routing.
- Bloom-filter-guided destination discovery (no flooding,
single-path with retry).
- Two-layer Noise encryption (IK at the link, XK at the session)
with periodic hitless rekey for forward secrecy at both layers.
- Two-layer Noise XX encryption (hop-by-hop at the link layer and
end-to-end at the session layer) with periodic hitless rekey for
forward secrecy at both layers and protocol negotiation in the
handshake.
- Persistent or ephemeral node identity with key-file management.
- IPv6 TUN adapter with built-in `.fips` DNS resolver and
multi-backend auto-configuration (systemd dns-delegate,
benches/routing_next_hop.rs
+365
View File
@@ -0,0 +1,365 @@
//! Micro-benchmark quantifying the per-forwarded-packet heap-allocation cost
//! of the routing next-hop candidate-assembly path.
//!
//! `find_next_hop` runs once per forwarded data packet. Its sans-IO core
//! assembles a `Vec<Candidate>` by enumerating every peer through the
//! `RoutingView` seam: `peer_addrs()` materializes a `Vec<NodeAddr>` of all
//! peers, the survivors are snapshotted (each cloning its `TreeCoordinate`),
//! and the result is collected into a second `Vec`. This bench measures that
//! per-call allocation against a fused zero-alloc reference that iterates the
//! peer map directly and borrows coordinates instead of cloning.
//!
//! Visibility caveat: the production `routing_candidates` / `select_best_candidate`
//! / `RoutingView` / `Candidate` are `pub(crate)` (src/proto/routing/core.rs)
//! and are not re-exported at the crate root, so an external bench crate cannot
//! name them. Rather than change production visibility, this file reproduces
//! that path verbatim over the real public `NodeAddr` / `TreeCoordinate` /
//! `CoordEntry` / `BloomFilter` types with the same iterator chain and the same
//! `HashMap`-backed view the shell uses (src/node/mod.rs NodeRoutingView). The
//! allocation behavior is therefore identical to production by construction;
//! only the symbol identity differs.
use std::alloc::{GlobalAlloc, Layout, System};
use std::collections::HashMap;
use std::hint::black_box;
use std::sync::atomic::{AtomicUsize, Ordering};
use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
use fips::{BloomFilter, NodeAddr, TreeCoordinate};
// ---------------------------------------------------------------------------
// Counting global allocator: bumps a process-global counter on every heap
// allocation operation (alloc / alloc_zeroed / realloc). Sampled tightly and
// single-threaded in `report_allocs` so no unrelated allocations are captured.
// ---------------------------------------------------------------------------
struct CountingAlloc;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);
unsafe impl GlobalAlloc for CountingAlloc {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
ALLOCS.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc(layout) }
}
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
unsafe { System.dealloc(ptr, layout) }
}
unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
ALLOCS.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc_zeroed(layout) }
}
unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
ALLOCS.fetch_add(1, Ordering::Relaxed);
unsafe { System.realloc(ptr, layout, new_size) }
}
}
#[global_allocator]
static GLOBAL: CountingAlloc = CountingAlloc;
const PEER_COUNTS: [usize; 4] = [8, 32, 128, 256];
/// Fraction of peers whose bloom filter reports the destination reachable.
const REACH_NUMERATOR: usize = 1;
const REACH_DENOMINATOR: usize = 2;
/// Tree depth for synthetic coordinates (self..root), a realistic mesh depth.
const COORD_DEPTH: usize = 8;
// ---------------------------------------------------------------------------
// Reproduction of the pub(crate) routing seam (src/proto/routing/core.rs).
// ---------------------------------------------------------------------------
trait RoutingView {
fn peer_addrs(&self) -> Vec<NodeAddr>;
fn peer_may_reach(&self, peer: &NodeAddr, dest: &NodeAddr) -> bool;
fn peer_can_send(&self, peer: &NodeAddr) -> bool;
fn peer_link_cost(&self, peer: &NodeAddr) -> f64;
fn peer_coords(&self, peer: &NodeAddr) -> Option<TreeCoordinate>;
}
struct Candidate {
addr: NodeAddr,
can_send: bool,
link_cost: f64,
coords: Option<TreeCoordinate>,
}
/// Verbatim from `routing::routing_candidates` (core.rs). Allocates the
/// `peer_addrs` Vec, clones each survivor's coords, and collects into a Vec.
fn routing_candidates(rv: &impl RoutingView, dest: &NodeAddr) -> Vec<Candidate> {
rv.peer_addrs()
.into_iter()
.filter(|peer| rv.peer_may_reach(peer, dest))
.map(|peer| Candidate {
can_send: rv.peer_can_send(&peer),
link_cost: rv.peer_link_cost(&peer),
coords: rv.peer_coords(&peer),
addr: peer,
})
.collect()
}
/// Verbatim from `routing::select_best_candidate` (core.rs). Pure, no alloc.
fn select_best_candidate(
candidates: &[Candidate],
dest_coords: &TreeCoordinate,
my_coords: &TreeCoordinate,
) -> Option<NodeAddr> {
let my_distance = my_coords.distance_to(dest_coords);
let mut best: Option<(&Candidate, f64, usize)> = None;
for candidate in candidates {
if !candidate.can_send {
continue;
}
let cost = candidate.link_cost;
let dist = candidate
.coords
.as_ref()
.map(|pc| pc.distance_to(dest_coords))
.unwrap_or(usize::MAX);
if dist >= my_distance {
continue;
}
let dominated = match &best {
None => true,
Some((_, best_cost, best_dist)) => {
cost < *best_cost
|| (cost == *best_cost && dist < *best_dist)
|| (cost == *best_cost
&& dist == *best_dist
&& candidate.addr < best.as_ref().unwrap().0.addr)
}
};
if dominated {
best = Some((candidate, cost, dist));
}
}
best.map(|(candidate, _, _)| candidate.addr)
}
// ---------------------------------------------------------------------------
// Bench-local view, HashMap-backed exactly like src/node/mod.rs NodeRoutingView.
// ---------------------------------------------------------------------------
struct BenchPeer {
bloom: BloomFilter,
can_send: bool,
link_cost: f64,
}
struct BenchView {
peers: HashMap<NodeAddr, BenchPeer>,
coords: HashMap<NodeAddr, TreeCoordinate>,
}
impl RoutingView for BenchView {
fn peer_addrs(&self) -> Vec<NodeAddr> {
self.peers.keys().copied().collect()
}
fn peer_may_reach(&self, peer: &NodeAddr, dest: &NodeAddr) -> bool {
self.peers.get(peer).is_some_and(|p| p.bloom.contains(dest))
}
fn peer_can_send(&self, peer: &NodeAddr) -> bool {
self.peers.get(peer).is_some_and(|p| p.can_send)
}
fn peer_link_cost(&self, peer: &NodeAddr) -> f64 {
self.peers.get(peer).map_or(f64::INFINITY, |p| p.link_cost)
}
fn peer_coords(&self, peer: &NodeAddr) -> Option<TreeCoordinate> {
self.coords.get(peer).cloned()
}
}
/// Zero-alloc reference: what an iterator/visitor seam would do. Iterates the
/// peer map directly, fuses the may_reach + can_send filters, borrows coords
/// instead of cloning, and tracks the best hop inline. No Vec, no coord clone.
fn resolve_next_hop_zeroalloc(
view: &BenchView,
dest: &NodeAddr,
dest_coords: &TreeCoordinate,
my_coords: &TreeCoordinate,
) -> Option<NodeAddr> {
let my_distance = my_coords.distance_to(dest_coords);
let mut best: Option<(NodeAddr, f64, usize)> = None;
for (addr, peer) in &view.peers {
if !peer.bloom.contains(dest) {
continue;
}
if !peer.can_send {
continue;
}
let cost = peer.link_cost;
let dist = view
.coords
.get(addr)
.map(|pc| pc.distance_to(dest_coords))
.unwrap_or(usize::MAX);
if dist >= my_distance {
continue;
}
let dominated = match &best {
None => true,
Some((best_addr, best_cost, best_dist)) => {
cost < *best_cost
|| (cost == *best_cost && dist < *best_dist)
|| (cost == *best_cost && dist == *best_dist && *addr < *best_addr)
}
};
if dominated {
best = Some((*addr, cost, dist));
}
}
best.map(|(addr, _, _)| addr)
}
// ---------------------------------------------------------------------------
// Scenario construction.
// ---------------------------------------------------------------------------
fn addr(tag: u8, i: u16) -> NodeAddr {
let mut b = [0u8; 16];
b[0] = tag;
b[1..3].copy_from_slice(&i.to_le_bytes());
NodeAddr::from_bytes(b)
}
/// A depth-`COORD_DEPTH` coordinate whose leaf is `leaf`, sharing a fixed
/// interior path and root with `shared_tag`. Peers built with the dest's
/// shared_tag sit close to the destination (distance 2); a distinct shared_tag
/// sits far (near the root), modeling our own position.
fn coord(leaf: NodeAddr, shared_tag: u8) -> TreeCoordinate {
let mut path = Vec::with_capacity(COORD_DEPTH);
path.push(leaf);
for level in 1..(COORD_DEPTH - 1) {
path.push(addr(shared_tag, level as u16));
}
path.push(addr(9, 0)); // common root
TreeCoordinate::from_addrs(path).expect("valid coord path")
}
struct Scenario {
view: BenchView,
dest: NodeAddr,
dest_coords: TreeCoordinate,
my_coords: TreeCoordinate,
}
impl Scenario {
fn new(n: usize) -> Self {
let dest = addr(2, 0);
// Destination path uses interior tag 4; peers reuse tag 4 so survivors
// are close to the destination. Our own coords use tag 5 (far).
let dest_coords = coord(dest, 4);
let my_coords = coord(addr(6, 0), 5);
let mut peers = HashMap::new();
let mut coords = HashMap::new();
for i in 0..n {
let paddr = addr(1, i as u16);
let mut bloom = BloomFilter::new();
// Realistic fill: a handful of unrelated reachable addrs.
for f in 0..4u16 {
bloom.insert(&addr(7, i as u16 * 4 + f));
}
// A controlled fraction advertise the destination as reachable.
if (i % REACH_DENOMINATOR) < REACH_NUMERATOR {
bloom.insert(&dest);
}
peers.insert(
paddr,
BenchPeer {
bloom,
can_send: true,
link_cost: 1.0 + (i as f64) * 0.01,
},
);
// Peers share the destination's interior path (tag 4) → close.
coords.insert(paddr, coord(paddr, 4));
}
Self {
view: BenchView { peers, coords },
dest,
dest_coords,
my_coords,
}
}
fn survivors(&self) -> usize {
self.view
.peers
.values()
.filter(|p| p.bloom.contains(&self.dest))
.count()
}
}
// ---------------------------------------------------------------------------
// Allocation-per-call report (printed once, before criterion timing).
// ---------------------------------------------------------------------------
fn count_allocs<T>(iters: usize, mut f: impl FnMut() -> T) -> f64 {
for _ in 0..8 {
black_box(f());
}
let start = ALLOCS.load(Ordering::Relaxed);
for _ in 0..iters {
black_box(f());
}
let end = ALLOCS.load(Ordering::Relaxed);
(end - start) as f64 / iters as f64
}
fn report_allocs() {
const ITERS: usize = 2000;
println!("\n=== allocations per call (heap alloc ops: alloc+alloc_zeroed+realloc) ===");
println!(
"{:>6} {:>10} {:>16} {:>16}",
"peers", "survivors", "current/call", "zero-alloc/call"
);
for &n in &PEER_COUNTS {
let s = Scenario::new(n);
let survivors = s.survivors();
let current = count_allocs(ITERS, || {
let cands = routing_candidates(&s.view, &s.dest);
select_best_candidate(&cands, &s.dest_coords, &s.my_coords)
});
let zero = count_allocs(ITERS, || {
resolve_next_hop_zeroalloc(&s.view, &s.dest, &s.dest_coords, &s.my_coords)
});
println!("{n:>6} {survivors:>10} {current:>16.2} {zero:>16.2}");
}
println!();
}
fn bench_next_hop(c: &mut Criterion) {
report_allocs();
let mut group = c.benchmark_group("find_next_hop");
for &n in &PEER_COUNTS {
let scenario = Scenario::new(n);
group.bench_with_input(BenchmarkId::new("current_alloc", n), &n, |b, _| {
b.iter(|| {
let cands = routing_candidates(&scenario.view, &scenario.dest);
black_box(select_best_candidate(
&cands,
&scenario.dest_coords,
&scenario.my_coords,
))
});
});
group.bench_with_input(BenchmarkId::new("zero_alloc_ref", n), &n, |b, _| {
b.iter(|| {
black_box(resolve_next_hop_zeroalloc(
&scenario.view,
&scenario.dest,
&scenario.dest_coords,
&scenario.my_coords,
))
});
});
}
group.finish();
}
criterion_group! {
name = benches;
config = Criterion::default().sample_size(50);
targets = bench_next_hop
}
criterion_main!(benches);
docs/design/fips-architecture.md
+15 -17
View File
@@ -44,7 +44,7 @@ See [fips-transport-layer.md](fips-transport-layer.md) for the
transport layer specification.
**FIPS Mesh Protocol (FMP)**: Manages peer connections, authenticates
peers via Noise IK handshakes, and encrypts all traffic on each link.
peers via Noise XX handshakes, and encrypts all traffic on each link.
FMP is where the mesh organizes itself — nodes exchange spanning tree
announcements and bloom filters with their direct peers, and FMP
makes forwarding decisions for transit traffic. FMP provides
@@ -116,8 +116,8 @@ three are deterministically derived from the same keypair.
![Identity Derivation](diagrams/fips-identity-derivation.svg)
The pubkey is the node's cryptographic identity, used in Noise
handshakes for both link encryption (IK) and session encryption (XK).
The pubkey is the node's cryptographic identity, used in Noise XX
handshakes for both link encryption and session encryption.
It is never exposed beyond the endpoints of an encrypted channel. The node_addr, a one-way
SHA-256 hash truncated to 16 bytes, serves as the routing identifier
in packet headers and bloom filters. Intermediate routers see only
@@ -156,31 +156,29 @@ FIPS uses independent encryption at two protocol layers:
| Layer | Scope | Pattern | Purpose |
| ----- | ----- | ------- | ------- |
| **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link |
| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints |
| **FMP (Mesh)** | Hop-by-hop | Noise XX | Encrypt all traffic on each peer link |
| **FSP (Session)** | End-to-end | Noise XX | Encrypt application payload between endpoints |
### Link Layer (Hop-by-Hop)
When two nodes establish a direct connection, they perform a [Noise
IK](https://noiseprotocol.org/) handshake. This authenticates both
XX](https://noiseprotocol.org/) handshake. This authenticates both
parties and establishes symmetric keys for encrypting all traffic on
that link. Every packet between direct peers is encrypted — gossip
messages, routing queries, and forwarded session datagrams alike.
The IK pattern is used because outbound connections know the peer's
npub from configuration, while inbound connections learn the
initiator's identity from the first handshake message.
Neither side requires prior knowledge of the other's static key —
both identities are revealed during the three-message handshake,
along with a protocol negotiation payload that enables rolling
upgrades.
### Session Layer (End-to-End)
FIPS establishes end-to-end encrypted sessions between any two
communicating nodes using Noise XK, regardless of how many hops
separate them. The initiator knows the destination's npub (required
for XK's pre-message); the responder learns the initiator's identity
from the third handshake message. Unlike the link-layer IK pattern
where the initiator's identity is revealed in msg1, XK delays
identity disclosure until msg3, providing stronger initiator identity
protection for traffic traversing untrusted intermediate nodes.
communicating nodes using Noise XX, regardless of how many hops
separate them. The same three-message XX pattern is used at both
layers — neither side reveals its identity until msg2 (responder) or
msg3 (initiator), providing mutual identity protection for traffic
traversing untrusted intermediate nodes.
A packet from A to D through intermediate nodes B and C:
docs/design/fips-bloom-filters.md
+43 -35
View File
@@ -1,5 +1,12 @@
# FIPS Bloom Filters
> **Status (2026-07-12):** This document still describes the **abandoned v1.5**
> bloom design. The code on `next` is presently plain **v1** — the v1.5
> implementation was removed during the sans-IO refactor and replaced with the
> v1 filter. The target design is **v2** (per the settled v2 bloom protocol
> specification), which is **pending implementation**. Until v2 lands, treat any
> "Implemented" wording below as the retired v1.5 work, not the shipped code.
This document describes the bloom filter data structures, parameters, and
mathematical properties used by FIPS for reachability-based candidate
selection. It is a supporting reference — for how bloom filters fit into
@@ -255,29 +262,30 @@ for i in 0..hash_count:
return true // Maybe present (possible false positive)
```
Where `filter_bits = 8 × (512 << size_class)` — 8,192 for v1.
Where `filter_bits = 8 × (512 << size_class)` — 8,192 for size_class 1 (default).
## Wire Format
The FilterAnnounce byte layout (`msg_type 0x20`, sequence, hash_count,
size_class, filter_bits) lives in
The FilterAnnounce byte layout (`msg_type 0x20`, flags, sequence,
base_seq, size_class, compressed payload) lives in
[../reference/wire-formats.md](../reference/wire-formats.md). The
v1 plaintext payload is 1,035 bytes (11-byte header + 1,024-byte
filter); link encryption adds 36 bytes of FMP framing (16-byte outer
header + 4-byte inner timestamp + 16-byte AEAD tag), bringing the
on-the-wire size to roughly 1,071 bytes before the underlying
transport's per-packet overhead.
v2 payload uses RLE-compressed full filters and XOR deltas
(`[count:2 LE][word:8 LE]` runs) so steady-state announcements are
typically a few dozen bytes; a `FilterNack` (msg_type 0x21) requests
full retransmission when a sequence gap is detected. Link encryption
adds 36 bytes of FMP framing (16-byte outer header + 4-byte inner
timestamp + 16-byte AEAD tag) on top of the compressed payload.
## Scale and Size Classes
### v1 Scale Limits
### Scale Limits
Coordinate-based tree distance checking ensures correct routing decisions
at all network sizes — bloom filters are an optimization that narrows the
set of peers considered, not a correctness requirement. As filters
saturate, routing still works; it just evaluates more candidates per hop.
With the v1 mandatory 1 KB filter (size_class 1):
With the default 1 KB filter (size_class 1):
- **Small networks (< 1,000 nodes)**: Both upward and downward filters
are highly accurate (worst-case FPR < 1%). Filters effectively narrow
@@ -298,33 +306,29 @@ With the v1 mandatory 1 KB filter (size_class 1):
### Size Class Table
| size_class | Bytes | Bits | Status |
| ---------- | ----- | ---- | ------ |
| 0 | 512 | 4,096 | Reserved |
| 1 | 1,024 | 8,192 | **v1 (MUST use)** |
| 2 | 2,048 | 16,384 | Reserved |
| 3 | 4,096 | 32,768 | Reserved |
| size_class | Bytes | Bits | Notes |
| ---------- | ----- | ---- | ----- |
| 0 | 512 | 4,096 | Minimum |
| 1 | 1,024 | 8,192 | Default |
| 2 | 2,048 | 16,384 | |
| 3 | 4,096 | 32,768 | |
| 4 | 8,192 | 65,536 | |
| 5 | 16,384 | 131,072 | |
| 6 | 32,768 | 262,144 | Maximum |
FMP v1 mandates size_class = 1. Nodes MUST use size_class = 1 and MUST
reject FilterAnnounce messages with any other size_class. The size_class
field is reserved in the wire format to support future protocol versions
with larger default filter sizes.
### Adaptive Sizing
### Scaling Strategy
Filter size is a node property, not a link property. Each node selects
its own size class based on outgoing filter fill ratio: step up above
~20%, step down below ~5%, with hysteresis to prevent oscillation.
Nodes near the root of the spanning tree — which carry larger combined
filters — naturally upsize, while leaf and edge nodes stay small.
The 1 KB filter becomes a practical limitation beyond ~2,000 nodes. The
size class mechanism provides the path forward: future FMP versions may
use larger default filters (size_class 2 or 3) to support larger networks
while remaining compatible with constrained nodes through folding.
Size_class 2 (2 KB, 16,384 bits) would roughly double the practical
network size limit.
The envisioned approach is that hub nodes near the root — which carry the
largest downward filters — would use larger size classes, while leaf nodes
and resource-constrained nodes continue with smaller filters. A node
receiving a filter larger than its own size class folds it down locally.
The mechanism by which heterogeneous filter sizes propagate through
the tree is a future design direction not specified in v1.
When a node receives a filter at a different size class than its own,
it converts on receipt: larger filters are folded down, smaller filters
are expanded via bit duplication. Routing queries use the peer's filter
at its native (advertised) size for full resolution; conversion happens
only when building the node's own outgoing filter.
### Folding
@@ -378,7 +382,7 @@ as described above.
| Feature | Status |
| ------- | ------ |
| 1 KB bloom filter (size_class 1) | **Implemented** |
| Variable-size bloom filters (512 B – 32 KB) | **Implemented** |
| 5 hash functions | **Implemented** |
| Split-horizon filter computation | **Implemented** |
| Tree-only merge propagation | **Implemented** |
@@ -386,6 +390,10 @@ as described above.
| Per-peer filter maintenance | **Implemented** |
| Event-driven updates | **Implemented** |
| 500ms rate limiting | **Implemented** |
| Delta compression (XOR diff + RLE) | **Implemented** |
| FilterNack sequence recovery | **Implemented** |
| Adaptive sizing (fill-ratio heuristic) | **Implemented** |
| Fold/duplicate size conversion | **Implemented** |
| FilterAnnounce gossip (all peers) | **Implemented** |
| Filter cardinality logging | **Implemented** |
| Mesh size estimation (OR-union of peer filters) | **Implemented** |
docs/design/fips-ipv6-adapter.md
+28
View File
@@ -302,6 +302,34 @@ alternative — running under a dedicated unprivileged service
account with the capability granted on the binary — see
[../how-to/run-as-unprivileged-user.md](../how-to/run-as-unprivileged-user.md).
### App-Owned TUN (embedded hosts)
On platforms where FIPS is embedded rather than run as a daemon — notably
Android, where the `VpnService` owns the TUN fd and the app has no
`CAP_NET_ADMIN` — FIPS does not create `fips0` itself. Instead the embedder owns
the fd and exchanges IPv6 packet bytes with FIPS over channels.
`Node::enable_app_owned_tun()` sets this up. It is called after `Node::new` and
before `start()` (and before the node is moved into a background task), mirroring
`control_read_handle()`, and returns two app-side channel ends:
- **app → mesh** — the embedder pushes IPv6 packets read from its fd into
`app_outbound_tx`. These are drained by `run_rx_loop` into `handle_tun_outbound`
and routed exactly as the Reader Thread's output would be.
- **mesh → app** — inbound mesh traffic on port 256 is reconstructed and written
to the node's `tun_tx` (the same sink the Writer Thread reads); the embedder
pulls from `app_inbound_rx` and writes to its fd.
With the channels installed, `start()` skips system-TUN creation (it gates on
`tun_tx` being unset), so FIPS does no `CAP_NET_ADMIN` operations.
Because packets enter via `app_outbound_tx` rather than the Reader Thread, they
**bypass `handle_tun_packet`** — the `fd00::/8` destination filter, the ICMPv6
Destination Unreachable for off-mesh dests (see [Reader Thread](#reader-thread)),
and the [TUN-Side TCP MSS Clamping](#tun-side-tcp-mss-clamping). The embedder is
therefore responsible for routing only `fd00::/8` to its TUN (so only mesh-bound
packets arrive) and for clamping TCP MSS on outbound SYNs.
## Implementation Status
| Feature | Status |
docs/design/fips-mesh-layer.md
+52 -39
View File
@@ -9,7 +9,7 @@ the mesh self-organizes, and where forwarding decisions are made.
FMP manages direct peer connections over transports. When a transport delivers
a datagram from an unknown address, FMP authenticates the sender through a
Noise IK handshake, establishing a cryptographic link. Once authenticated, the
Noise XX handshake, establishing a cryptographic link. Once authenticated, the
link carries all inter-peer communication: spanning tree gossip, bloom filter
updates, coordinate discovery, and forwarded session datagrams — all encrypted
per-hop.
@@ -82,7 +82,7 @@ for the encrypted frame wrapper: 16-byte outer header + 5-byte inner header +
### Connection Lifecycle
For connection-oriented transports, the transport must establish the underlying
connection before FMP can begin the Noise IK handshake. For connectionless
connection before FMP can begin the Noise XX handshake. For connectionless
transports, datagrams can flow immediately.
### Endpoint Discovery (Optional)
@@ -94,46 +94,52 @@ through configuration.
## Peer Authentication
### Noise IK Handshake
### Noise XX Handshake
Every peer connection begins with a Noise IK handshake that mutually
Every peer connection begins with a Noise XX handshake that mutually
authenticates both parties and establishes symmetric keys for link encryption.
The IK pattern is chosen because:
The XX pattern is chosen because:
- The **initiator** knows the responder's static public key from configuration
or discovery, and sends their own static key encrypted in the first message
- The **responder** learns the initiator's identity from the first message,
then responds with their own ephemeral key
- **Neither side** requires prior knowledge of the other's static public key
- The **responder** reveals its identity in msg2; the **initiator** reveals
its identity in msg3
- A protocol negotiation payload (version range, feature bitfield, TLV
extensions) is appended to msg2 and msg3, enabling rolling protocol
upgrades without additional round-trips
After the two-message handshake completes, both parties share symmetric
session keys derived from four DH operations (es, ss, ee, se). The handshake
provides mutual authentication, forward secrecy, and identity hiding for the
initiator.
After the three-message handshake completes, both parties share symmetric
session keys derived from three DH operations (ee, es, se). The handshake
provides mutual authentication, forward secrecy, and identity hiding for
both parties until they choose to reveal.
### Epoch Exchange and Peer Restart Detection
Both IK handshake messages carry an encrypted epoch payload — an 8-byte
random value generated once at node startup:
XX handshake messages msg2 and msg3 carry an encrypted epoch payload — an
8-byte random value generated once at node startup:
- **msg1**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) +
encrypted epoch (24 bytes) = 106 bytes total
- **msg2**: Ephemeral key (33 bytes) + encrypted epoch (24 bytes) = 57 bytes
total
- **msg1**: Ephemeral key only (33 bytes) — no identity or epoch
- **msg2**: Ephemeral key (33 bytes) + encrypted static key (49 bytes) +
encrypted epoch (24 bytes) = 106 bytes base, plus negotiation payload
- **msg3**: Encrypted static key (49 bytes) + encrypted epoch (24 bytes)
= 73 bytes base, plus negotiation payload
The encrypted epoch (EPOCH_ENCRYPTED_SIZE = 24 bytes) consists of the
8-byte epoch value plus a 16-byte AEAD tag.
On reconnection, each peer compares the received epoch with the previously
stored epoch for that peer. An epoch mismatch indicates the peer has
restarted (generated a new epoch), triggering full link re-establishment
rather than treating the handshake as a simple reconnection. This prevents
stale session state from persisting across restarts.
Because msg1 carries no identity, restart detection is deferred: the
initiator checks the responder's epoch after msg2, and the responder
checks the initiator's epoch after msg3. An epoch mismatch indicates the
peer has restarted, triggering full link re-establishment rather than
treating the handshake as a simple reconnection. This prevents stale
session state from persisting across restarts.
### Identity Binding
The Noise handshake binds the link to the peer's cryptographic identity. After
handshake completion:
The Noise handshake binds the link to the peer's cryptographic identity.
With XX, identity confirmation happens at different points: the initiator
learns the responder's identity from msg2, and the responder learns the
initiator's identity from msg3. After handshake completion:
- The peer's public key (FIPS identity) is confirmed
- The node_addr is computed from the public key (SHA-256, truncated to 128 bits)
@@ -143,8 +149,12 @@ handshake completion:
### Reconnection
When a Noise IK msg1 arrives from a peer that already has an authenticated
link, FMP accepts the new handshake alongside the existing session. If the new
When a Noise XX msg1 arrives from an address that already has an
authenticated link, FMP accepts the new handshake alongside the existing
session. With XX, the peer's identity is not known at msg1 time — FMP can
only detect the duplicate by transport address. Identity-based checks
(restart detection, rekey recognition, cross-connection resolution) are
deferred to msg3 when the initiator's identity is revealed. If the new
handshake completes successfully, it replaces the old session. This handles
legitimate reconnection (network change, process restart, NAT rebinding)
without disrupting ongoing traffic until the new session is confirmed.
@@ -163,7 +173,7 @@ The auto-reconnect path:
3. If eligible, the peer is fed into the retry system with unlimited retries
and exponential backoff (same base interval and max backoff as startup
retries, configured via `node.retry.*`)
4. On each retry tick, a fresh Noise IK handshake is initiated toward the
4. On each retry tick, a fresh Noise XX handshake is initiated toward the
peer's configured transport addresses
Auto-reconnect only applies to peers in the static peer list with
@@ -173,8 +183,8 @@ config) is responsible for re-establishing the link.
### Handshake Message Retry
Both link-layer (Noise IK msg1/msg2) and session-layer (SessionSetup/
SessionAck) handshakes use message-level retry with exponential backoff
Both link-layer (Noise XX msg1/msg2/msg3) and session-layer (SessionSetup/
SessionAck/SessionMsg3) handshakes use message-level retry with exponential backoff
within the handshake timeout window. This handles packet loss on the
underlying transport without waiting for the full handshake timeout to
expire.
@@ -380,13 +390,14 @@ configuration tree is documented in
### Mechanism
A rekey reuses the Noise IK pattern of the initial handshake, but the two
messages travel over the existing link as ordinary encrypted FMP frames
rather than as plaintext bootstrap packets. The initiator builds a fresh
`HandshakeState`, generates msg1, and sends it through the current session;
the responder consumes msg1, builds msg2, and replies. After both sides
have exchanged messages and finalised the new keys, traffic transitions
from the old session to the new one.
A rekey reuses the Noise XX pattern of the initial handshake, but the
three messages travel over the existing link as ordinary encrypted FMP
frames rather than as plaintext bootstrap packets. The initiator builds
a fresh `HandshakeState`, generates msg1, and sends it through the
current session; the responder consumes msg1, builds msg2, and replies;
the initiator then sends msg3. After both sides have exchanged messages
and finalised the new keys, traffic transitions from the old session to
the new one.
Cutover is signalled in-band by the **K-bit** in the FMP flags byte. Each
side starts emitting frames under the new session with K set; on receipt
@@ -547,7 +558,9 @@ an attacker sends invalid packets to elicit responses.
| Feature | Status |
| ------- | ------ |
| Noise IK handshake (with epoch) | **Implemented** |
| Noise XX handshake (with epoch and negotiation) | **Implemented** |
| Protocol negotiation (version + features + TLV) | **Implemented** |
| Node profiles (Full, NonRouting, Leaf) | **Implemented** |
| Peer restart detection (epoch mismatch) | **Implemented** |
| Link encryption (ChaCha20-Poly1305) | **Implemented** |
| Index-based session dispatch | **Implemented** |
docs/design/fips-mesh-operation.md
+20 -13
View File
@@ -182,8 +182,8 @@ The source creates a LookupRequest containing:
- **request_id**: Unique identifier for deduplication
- **target**: The node_addr being sought
- **origin**: The requester's node_addr
- **origin_coords**: The requester's current tree coordinates (so the
response can route back)
- **min_mtu**: Minimum transport MTU the origin requires (transit nodes
skip peers whose link MTU is below this)
- **TTL**: Bounds the forwarding radius
### Bloom-Guided Tree Routing
@@ -269,8 +269,8 @@ the primary mechanism: each transit node looks up the `request_id` in its
`recent_requests` table to find the peer that forwarded the original request,
and sends the response back through that peer. This ensures the response
follows the same path as the request. Greedy tree routing toward the
`origin_coords` is used only as a fallback if the reverse-path entry has
expired.
greedy tree routing toward the origin's coordinates is used only as a
fallback if the reverse-path entry has expired.
**Response-forwarded flag**: Each `recent_requests` entry tracks whether a
response has already been forwarded for that `request_id`. If a second
@@ -472,16 +472,23 @@ When traffic resumes:
3. Coordinates: discovery may be needed if cache has expired
4. SessionSetup re-warms transit caches on the new path
## Leaf-Only Operation *(under development)*
## Node Profiles
Leaf-only operation is an optimization for resource-constrained nodes
(sensors, battery-powered devices). The core infrastructure exists (config
flag, node constructor, bloom filter support) but is not yet enabled in
normal operation.
Nodes advertise a profile during FMP negotiation (bits 0-2 of the feature
bitfield): **Full** (default), **NonRouting**, or **Leaf**. At least one
side of a link must be Full. Config mapping: `disable_routing: true` →
NonRouting, `leaf_only: true` → Leaf.
### Concept
### Non-Routing Nodes
A leaf-only node connects to a single upstream peer that handles all routing
A non-routing node participates in the spanning tree but does not forward
transit traffic or send bloom filters. Its full peer inserts the
non-routing node's identity as a leaf dependent. MMP report flow is gated
by wants/provides bits negotiated during the handshake.
### Leaf Nodes
A leaf node connects to a single upstream peer that handles all routing
on its behalf:
- **No bloom filter storage or processing**: The upstream peer includes the
@@ -504,7 +511,7 @@ The upstream peer:
Even as a leaf-only node, it still:
- Maintains its own Noise IK link session with the upstream peer (FMP layer)
- Maintains its own Noise XX link session with the upstream peer (FMP layer)
- Can establish end-to-end FSP sessions with arbitrary destinations
- Has its own identity (npub, node_addr)
@@ -564,7 +571,7 @@ recovery).
| Discovery originator backoff | **Implemented** |
| Discovery transit-side rate limiting | **Implemented** |
| Discovery response-forwarded dedup | **Implemented** |
| Leaf-only operation | Under development |
| Node profiles (Full, NonRouting, Leaf) | **Implemented** |
| Link cost in parent selection (ETX) | **Implemented** |
| Link cost in candidate ranking | **Implemented** |
docs/design/fips-mmp.md
+6 -16
View File
@@ -68,7 +68,7 @@ MMP supports three modes:
| ---- | ----------------- | ----------------- |
| **Full** (default) | SenderReport + ReceiverReport | All metrics including RTT, loss, jitter, goodput, OWD trend |
| **Lightweight** | ReceiverReport only | Loss (from counter gaps), jitter, OWD trend. No RTT. |
| **Minimal** | None | Spin bit and CE echo flags only. No computed metrics. |
| **Minimal** | None | CE echo flags only. No computed metrics. |
The mode is configured per layer (`node.mmp.mode` and
`node.session_mmp.mode`).
@@ -88,28 +88,18 @@ The session-layer bounds are higher because session reports are
encrypted and forwarded through every transit link, so bandwidth cost
is proportional to path length.
## Spin Bit and RTT
## RTT Measurement
The SP (spin bit) flag in the FMP inner header follows the QUIC spin
bit pattern: reflected on receive, toggled on send when the reflected
value matches the last sent value. The spin bit state machine runs
for TX reflection, but **RTT samples from the spin bit are
discarded**. In a mesh protocol where frames are sent irregularly
(tree announces, bloom filters, MMP reports on different timers),
inter-frame processing delays inflate spin bit RTT measurements
unpredictably. Timestamp-echo from ReceiverReports (with dwell-time
compensation) is the sole SRTT source.
SRTT is derived exclusively from timestamp-echo in ReceiverReports
with dwell-time compensation, applied via the Jacobson/Karels
algorithm (RFC 6298, α = 1/8). This is the sole SRTT source at both
layers.
Duplicate or regressed ReceiverReports are ignored before any RTT, loss,
goodput, or ETX update. If receiver-side dwell time exceeds the wire
field, the report keeps its counters but sends a zero timestamp echo so
the sender cannot form an invalid RTT sample.
The spin bit lives in the link-layer FMP inner header, so this
mechanism applies to link-layer MMP only. Session-layer MMP carries
its spin bit in the FSP encrypted inner header but uses it the same
way: reflected for diagnostic visibility, not used for SRTT.
## ECN Congestion Signaling
The CE (Congestion Experienced) flag (bit 1 in the FMP flags byte)
docs/design/fips-nostr-discovery.md
+4 -4
View File
@@ -273,7 +273,7 @@ carrying the session id, the punch socket, and the learned remote
address. `adopt_established_traversal()` in the node lifecycle takes
the socket, registers it with the UDP transport layer as a new
transport instance, and calls `initiate_connection()` with the peer's
FIPS identity as the expected remote. FMP's Noise IK handshake runs on
FIPS identity as the expected remote. FMP's Noise XX handshake runs on
the same socket — there is no "promote link" step between punch and
handshake; the punch socket *is* the FMP socket.
@@ -382,7 +382,7 @@ semaphore and replay-cache layers downstream.
non-default `app` value to scope visibility.
- **Nothing about discovery bypasses FMP.** A successful punch yields
a UDP socket with a claimed remote identity. That identity is not
trusted until FMP's Noise IK handshake completes. A peer whose
trusted until FMP's Noise XX handshake completes. A peer whose
advert says "I am npub X at 1.2.3.4:5678" but whose FMP handshake
presents a different static key is rejected at the mesh layer.
@@ -477,7 +477,7 @@ The TXT record carries three keys (`src/discovery/lan/mod.rs:47-55`):
Once per node tick, the node drains browser events and acts on them in
`poll_lan_discovery()` (`src/node/lifecycle.rs:907`, called from
`src/node/handlers/rx_loop.rs:266`). For each discovered peer it finds
`src/node/dataplane/rx_loop.rs:266`). For each discovered peer it finds
a UDP transport whose family matches the peer address, parses the
`npub` into a `PeerIdentity`, skips peers it is already connected to or
currently connecting to, and otherwise initiates a connection.
@@ -584,7 +584,7 @@ beyond the shared scope fallback.
- [fips-transport-layer.md](fips-transport-layer.md) — UDP, TCP, and
Tor transport mechanics; the punch socket is adopted as a normal
UDP transport after handoff.
- [fips-mesh-layer.md](fips-mesh-layer.md) — FMP Noise IK handshake
- [fips-mesh-layer.md](fips-mesh-layer.md) — FMP Noise XX handshake
that runs on the adopted socket.
- [port-advertisement-and-nat-traversal.md](port-advertisement-and-nat-traversal.md)
— generic protocol reference (event tags, NIP usage, on-the-wire
docs/design/fips-prior-work.md
+17 -29
View File
@@ -80,27 +80,24 @@ patterns.
## Noise Protocol Framework
FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/)
at both protocol layers, with different handshake patterns chosen for
each layer's threat model. FMP link encryption uses **Noise IK**,
providing mutual authentication with a single round trip where the
initiator knows the responder's static key in advance.
[WireGuard](https://www.wireguard.com/) uses the same IK base pattern
(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP
session encryption uses **Noise XK**, the same pattern used by the
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md),
where the initiator's static key is transmitted in a third message
rather than the first. XK provides stronger initiator identity hiding
at the cost of an additional round trip — a worthwhile tradeoff for
session-layer traffic that traverses untrusted intermediate nodes. At
the link layer, where both peers are configured and directly
connected, IK's single round trip is preferred.
at both protocol layers with the **Noise XX** handshake pattern. XX
requires no prior knowledge of the peer's static key — both
identities are revealed during a three-message handshake (responder
in msg2, initiator in msg3). This enables anonymous peer discovery on
shared-media transports and allows a protocol negotiation payload to
be exchanged alongside the handshake, supporting rolling protocol
upgrades without extra round trips.
[WireGuard](https://www.wireguard.com/) uses the related IK pattern
for VPN tunnels;
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md)
uses XK for transport encryption.
Specific Noise references and adapted constructions:
- Perrin, T. ["The Noise Protocol Framework"](https://noiseprotocol.org/noise.html).
Revision 34, 2018. *Framework for building crypto protocols using
Diffie-Hellman key agreement and AEAD ciphers. FSP uses the XK
handshake pattern.*
Diffie-Hellman key agreement and AEAD ciphers. FIPS uses the XX
handshake pattern at both layers.*
- Donenfeld, J.A. ["WireGuard: Next Generation Kernel Network Tunnel"](https://www.wireguard.com/papers/wireguard.pdf).
NDSS 2017. *Transport-independent cryptographic sessions bound to
@@ -156,14 +153,6 @@ computation used in TCP for retransmission timeout calculation since
1988. MMP derives RTT from timestamp-echo in ReceiverReports with
dwell-time compensation, rather than from packet round-trips.
The spin bit in the FMP frame header follows the
[QUIC](https://www.rfc-editor.org/rfc/rfc9000) spin bit
([RFC 9312](https://www.rfc-editor.org/rfc/rfc9312)) — a single bit
that alternates each round trip, enabling passive latency measurement.
FIPS implements the spin bit state machine but relies on
timestamp-echo for SRTT, as irregular mesh traffic makes spin bit RTT
unreliable.
The Expected Transmission Count (ETX) metric, computed from
bidirectional delivery ratios, was introduced by
[De Couto et al. (2003)](https://pdos.csail.mit.edu/papers/grid:mobicom03/paper.pdf)
@@ -325,11 +314,10 @@ The protocol builds on these foundations and adds several new elements:
| [HIP](https://en.wikipedia.org/wiki/Host_Identity_Protocol) | identity-as-address |
| [Babel](https://www.irif.fr/~jch/software/babel/) | split-horizon, ETX |
| [RIP](https://en.wikipedia.org/wiki/Routing_Information_Protocol) | split-horizon |
| [Noise Framework](https://noiseprotocol.org/) | FMP IK, FSP XK |
| [WireGuard](https://www.wireguard.com/) | IK pattern, receiver-index dispatch, identity-bound sessions |
| [Lightning BOLT #8](https://github.com/lightning/bolts/blob/master/08-transport.md) | XK pattern |
| [QUIC (RFC 9000)](https://www.rfc-editor.org/rfc/rfc9000) | spin bit, transport design |
| [QUIC Spin Bit (RFC 9312)](https://www.rfc-editor.org/rfc/rfc9312) | passive RTT measurement |
| [Noise Framework](https://noiseprotocol.org/) | FMP and FSP XX handshakes |
| [WireGuard](https://www.wireguard.com/) | receiver-index dispatch, identity-bound sessions |
| [Lightning BOLT #8](https://github.com/lightning/bolts/blob/master/08-transport.md) | comparison reference |
| [QUIC (RFC 9000)](https://www.rfc-editor.org/rfc/rfc9000) | transport design |
| [RTCP (RFC 3550)](https://www.rfc-editor.org/rfc/rfc3550) | sender/receiver report structure, jitter algorithm |
| [TCP SRTT/RTO (RFC 6298)](https://www.rfc-editor.org/rfc/rfc6298) | Jacobson/Karels SRTT |
| [ECN (RFC 3168)](https://www.rfc-editor.org/rfc/rfc3168) | CE echo |
docs/design/fips-security.md
+3 -3
View File
@@ -17,8 +17,8 @@ you can deliver packets to your `fips0` address — your direct peers
forward traffic from non-peer mesh nodes onto your `fips0` the same
way any router forwards transit traffic. Identity on the mesh is the
originating node's npub — the FMP link layer authenticates direct
peers with Noise IK and the FSP session layer authenticates session
endpoints with Noise XK — but identity is **not** authorization.
peers with Noise XX and the FSP session layer authenticates session
endpoints with Noise XX — but identity is **not** authorization.
Knowing who sent a packet does not, by itself, decide whether the
local host should accept it.
@@ -149,7 +149,7 @@ explicitly not:
originating mesh node's npub is allowed to use that service. That
is the application's responsibility (e.g., an `authorized_keys`
file for SSH, an ACL in the application's configuration).
- **ACL on the mesh handshake.** The FMP Noise IK handshake
- **ACL on the mesh handshake.** The FMP Noise XX handshake
authenticates the peer's npub and, on both inbound and outbound
paths, consults the peer ACL (`peers.allow` / `peers.deny`) before
promoting the connection. The ACL evaluates in TCP-Wrappers order:
docs/design/fips-session-layer.md
+71 -48
View File
@@ -120,25 +120,29 @@ node, FMP delivers it to FSP for session-layer processing.
Sessions are established on demand when the first datagram needs to be sent to
a destination with no existing session.
FSP uses Noise XK for session key agreement (Noise Protocol Framework;
Perrin 2018). The initiator knows the destination's npub (required for
XK's pre-message `s` token); the responder learns the initiator's
identity from msg3 (not msg1, unlike IK at the link layer). This
provides stronger initiator identity hiding — the initiator's static
key is encrypted under the established shared secret rather than under
only the responder's static key.
FSP uses Noise XX for session key agreement (Noise Protocol Framework;
Perrin 2018). Neither side requires prior knowledge of the other's
static key — both identities are revealed during the handshake
(responder in msg2, initiator in msg3). An optional protocol negotiation
payload may be appended to msg2/msg3 (omitted for rekey handshakes).
The handshake is a three-message flow carried in SessionSetup, SessionAck,
and SessionMsg3:
1. **Initiator** sends SessionSetup containing Noise XK msg1 (ephemeral key
1. **Initiator** sends SessionSetup containing Noise XX msg1 (ephemeral key
only) and both parties' tree coordinates
2. **Responder** processes msg1, sends SessionAck containing Noise XK msg2
(ephemeral key + encrypted epoch) and both parties' tree coordinates.
The responder transitions to AwaitingMsg3 state.
3. **Initiator** processes msg2, sends SessionMsg3 containing the encrypted
static key and encrypted epoch. Both parties derive identical symmetric
session keys and the session is established.
2. **Responder** processes msg1, sends SessionAck containing Noise XX msg2
(ephemeral key + encrypted static key + encrypted epoch) and both
parties' tree coordinates. The responder transitions to AwaitingMsg3
state.
3. **Initiator** processes msg2 (learning the responder's identity), sends
SessionMsg3 containing its encrypted static key and encrypted epoch.
The responder learns the initiator's identity from msg3. Both parties
derive identical symmetric session keys and the session is established.
Post-handshake identity verification uses x-only key comparison
(parity-independent) to confirm the revealed identity matches the
expected npub.
Each side's epoch (an 8-byte random value generated at startup) is
exchanged encrypted in msg2 and msg3. On subsequent handshakes, an epoch
@@ -219,29 +223,30 @@ than network addresses. A session survives:
## End-to-End Encryption
### Noise XK Pattern
### Noise XX Pattern
FSP uses Noise XK for session encryption, distinct from the Noise IK
pattern used at the link layer. The full Noise descriptor is
`Noise_XK_secp256k1_ChaChaPoly_SHA256`.
FSP uses the same Noise XX pattern as the link layer (FMP). The full
Noise descriptor is `Noise_XX_secp256k1_ChaChaPoly_SHA256`.
The XK pattern (pre-message: `← s`):
The XX pattern (no pre-message):
- **msg1** (`→ e, es`): Initiator sends ephemeral key only. The initiator's
static identity is not revealed in this message.
- **msg2** (`← e, ee`): Responder sends ephemeral key and encrypted epoch.
- **msg1** (`→ e`): Initiator sends ephemeral key only. No identity
disclosed, no DH with static keys.
- **msg2** (`← e, ee, s, es`): Responder sends ephemeral key, encrypted
static key, and encrypted epoch. The initiator learns the responder's
identity.
- **msg3** (`→ s, se`): Initiator sends encrypted static key and encrypted
epoch. Both parties now share identical session keys.
epoch. The responder learns the initiator's identity. Both parties now
share identical session keys.
After the handshake, Noise produces two directional symmetric keys
(`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent data.
(`send_key`, `recv_key`) used with ChaCha20-Poly1305 for all subsequent
data.
The XK pattern requires the initiator to know the responder's static key
in advance (the `← s` pre-message), which is satisfied by the discovery
or DNS lookup that precedes session establishment. In exchange, XK
provides stronger initiator identity protection than IK — the initiator's
static key is encrypted under the full shared secret (after three DH
operations) rather than under only the responder's static key.
XX requires no prior knowledge of the peer's static key. The initiator
still needs the destination's npub to address the SessionSetup, but the
Noise handshake itself does not depend on it — identity is verified
post-handshake by comparing the revealed key against the expected npub.
### Cryptographic Primitives
@@ -251,26 +256,27 @@ primitive table shared with the link layer.
### secp256k1 Parity Normalization
Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity). The
Noise XK pre-message mixes the responder's static key as a 33-byte compressed
key, and the default secp256k1 ECDH hash includes a parity-dependent version
byte.
Nostr npubs encode x-only public keys (32 bytes, no y-coordinate parity).
When the Noise XX handshake reveals a peer's static key via
`public_key().serialize()`, the key has its actual parity (0x02 or 0x03
prefix). The default secp256k1 ECDH hash also includes a parity-dependent
version byte.
Both operations are normalized to be parity-independent: the pre-message hash
uses even parity (`0x02` prefix), and ECDH hashes only the x-coordinate of the
result point. This ensures handshakes succeed regardless of the responder's
actual key parity.
Both operations are normalized to be parity-independent: ECDH hashes only
the x-coordinate of the result point, and post-handshake identity
verification uses `x_only_public_key()` to strip parity before comparing
against the expected npub. This ensures handshakes and identity checks
succeed regardless of key parity.
### Privacy Note
Noise XK provides stronger initiator identity protection than IK. In XK, the
initiator's static key is encrypted in msg3 under the full shared secret
(derived from three DH operations), so an attacker who compromises only the
responder's nsec cannot decrypt the initiator's identity from captured
handshake messages (they would also need the responder's ephemeral key).
This is the primary reason FSP uses XK rather than IK — session-layer
traffic traverses untrusted intermediate nodes, making initiator identity
protection more valuable than at the link layer.
Noise XX provides mutual identity protection — both the initiator's and
responder's static keys are encrypted under the evolving shared secret
(derived from DH operations completed in earlier messages). An attacker
who compromises only one side's nsec cannot decrypt the other side's
identity from captured handshake messages without also obtaining the
corresponding ephemeral key. Since session-layer traffic traverses
untrusted intermediate nodes, this mutual identity hiding is valuable.
### Data Packet Authentication
@@ -461,7 +467,7 @@ reactive PMTUD mechanism.
| Feature | Status |
| ------- | ------ |
| Session establishment (Noise XK) | **Implemented** |
| Session establishment (Noise XX) | **Implemented** |
| Peer restart detection (epoch exchange) | **Implemented** |
| MtuExceeded handling | **Implemented** |
| End-to-end encryption (ChaCha20-Poly1305) | **Implemented** |
@@ -497,7 +503,7 @@ reactive PMTUD mechanism.
- [fips-mmp.md](fips-mmp.md) — Metrics Measurement Protocol (link + session)
- [fips-mtu.md](fips-mtu.md) — Path MTU model (PathMtuNotification,
MtuExceeded, hysteresis)
- [fips-prior-work.md](fips-prior-work.md) — Noise XK, WireGuard,
- [fips-prior-work.md](fips-prior-work.md) — Noise XX, WireGuard,
DTLS replay window, IKEv2 simultaneous initiation, hybrid coordinate
warmup citations
- [../reference/wire-formats.md](../reference/wire-formats.md) — Wire
@@ -507,6 +513,23 @@ reactive PMTUD mechanism.
### External References
- Perrin, T. ["The Noise Protocol Framework"](https://noiseprotocol.org/noise.html).
Revision 34, 2018. *Framework for building crypto protocols using Diffie-Hellman
key agreement and AEAD ciphers. FSP uses the XX handshake pattern.*
- Donenfeld, J.A. ["WireGuard: Next Generation Kernel Network Tunnel"](https://www.wireguard.com/papers/wireguard.pdf).
NDSS 2017. *Transport-independent cryptographic sessions bound to identity keys
rather than network addresses; AEAD-only authentication model.*
- Rescorla, E., Modadugu, N. [RFC 6347](https://datatracker.ietf.org/doc/html/rfc6347):
"Datagram Transport Layer Security Version 1.2". 2012. *Explicit sequence numbers
with sliding bitmap window for replay protection over unreliable transports.*
- Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., Kivinen, T.
[RFC 7296](https://datatracker.ietf.org/doc/html/rfc7296):
"Internet Key Exchange Protocol Version 2 (IKEv2)". 2014. *Simultaneous
initiation resolution (§2.8) and INITIAL_CONTACT peer restart detection (§2.4).*
- Mogul, J., Deering, S. [RFC 1191](https://datatracker.ietf.org/doc/html/rfc1191):
"Path MTU Discovery". 1990. *End-to-end path MTU discovery; FSP adapts this for
overlay networks using transit-node min() propagation.*
docs/design/fips-spanning-tree.md
+1 -1
View File
@@ -171,7 +171,7 @@ When a node receives a TreeAnnounce from peer P:
1. **Validate version**: Reject if version ≠ 0x01
2. **Verify signature**: Check P's declaration signature using P's known
public key (established during Noise IK handshake)
public key (established during Noise XX handshake)
3. **Verify identity**: Confirm the declaration's node_addr matches the
sender's known identity
4. **Check freshness**: If `sequence ≤ stored sequence for P`, discard
docs/design/fips-transport-layer.md
+23 -25
View File
@@ -18,7 +18,7 @@ to the FIPS Mesh Protocol (FMP) above.
The transport layer deals exclusively in **transport addresses** — IP:port
or hostname:port addresses, MAC addresses, .onion identifiers, radio device addresses. These are
opaque to every layer above FMP. The mapping from transport address to FIPS
identity happens at the link layer after the Noise IK link handshake completes.
identity happens at the link layer after the Noise XX link handshake completes.
The word "peer" belongs to the link layer and above; the transport layer
knows only about remote endpoints identified by transport addresses.
@@ -71,7 +71,7 @@ forwarding and LookupResponse transit annotation.
For connection-oriented transports, manage the underlying connection: TCP
handshake, Tor circuit establishment, BLE pairing. FMP cannot begin
the Noise IK link handshake until the transport-layer connection is
the Noise XX link handshake until the transport-layer connection is
established.
Connection-oriented transports expose a non-blocking connect interface.
@@ -154,7 +154,7 @@ and duplication at the routing layer.
**Connection model**: Connectionless transports (UDP, raw Ethernet) allow
immediate datagram exchange. Connection-oriented transports (TCP, Tor, BLE)
require connection setup before FMP can begin the Noise IK link handshake,
require connection setup before FMP can begin the Noise XX link handshake,
adding startup latency.
**Stream vs. datagram**: Datagram transports have natural packet boundaries.
@@ -262,7 +262,7 @@ UDP (1500 vs 1472 MTU).
- **No IP dependency**: Operates below the IP layer. Nodes on the same
Ethernet segment can communicate without IP addresses or routing
infrastructure
- **Broadcast discovery**: Nodes discover each other via periodic beacon
- **Broadcast neighbor detection**: Nodes discover each other via periodic beacon
broadcasts on the shared medium, with no static peer configuration required
- **Higher MTU**: Standard Ethernet frames carry 1500 bytes of payload,
yielding an effective FIPS MTU of 1499 after the frame type prefix
@@ -276,32 +276,30 @@ EtherType 0x2121. SOCK_DGRAM mode
lets the kernel handle Ethernet header construction and parsing — the
transport deals only with payloads and MAC addresses.
Data frames use a 3-byte header: a 1-byte frame type (`0x00`) followed by
a 2-byte little-endian payload length. The length field allows the receiver
to trim Ethernet minimum-frame padding that would otherwise corrupt AEAD
verification. Beacon frames (`0x01`) use only the 1-byte type prefix
(fixed 34-byte payload). Beacons and data share the same EtherType and
socket.
All frames use a unified 4-byte header: `[type:1][flags:1][length:2 LE]`.
The length field allows the receiver to trim Ethernet minimum-frame padding
that would otherwise corrupt AEAD verification. Frame types: `0x00` (data),
`0x01` (beacon). Beacons and data share the same EtherType and socket.
| Property | Value |
| -------- | ----- |
| EtherType | 0x2121 |
| Socket type | AF_PACKET SOCK_DGRAM |
| Data frame header | `[type:1][length:2 LE][payload]` |
| Beacon frame header | `[type:1][payload]` (fixed 34 bytes) |
| Effective MTU | Interface MTU - 3 (typically 1497) |
| Frame header | `[type:1][flags:1][length:2 LE][payload]` |
| Effective MTU | Interface MTU - 4 (typically 1496) |
| Addressing | 6-byte MAC address |
| Platform | Linux only (`CAP_NET_RAW` required) |
### Beacon Discovery
### Neighbor Beacons
Ethernet nodes discover peers via broadcast beacons sent to
ff:ff:ff:ff:ff:ff. Each beacon is a 34-byte frame containing the sender's
x-only public key. Receiving nodes extract the MAC source address from the
frame and the public key from the payload, then report the discovered peer
to FMP.
ff:ff:ff:ff:ff:ff. Beacons are minimal 5-byte frames (4-byte header +
1-byte beacon type) — no public key is included. The peer's identity is
learned from the Noise XX handshake after the connection is established.
Receiving nodes extract the MAC source address from the frame and report
the discovered address to FMP.
Four configuration flags control discovery behavior — `discovery`
Four configuration flags control neighbor behavior — `listen`
(listen for beacons), `announce` (broadcast beacons), `auto_connect`
(initiate handshakes to discovered peers), and `accept_connections`
(accept inbound handshakes). The flag table and per-flag defaults
@@ -310,13 +308,13 @@ under `transports.ethernet.*`.
A typical discoverable node sets `announce`, `auto_connect`, and
`accept_connections` all true. A passive listener uses just
`discovery: true` to observe the network without announcing itself.
`listen: true` to observe the network without announcing itself.
### WiFi Compatibility
WiFi interfaces in infrastructure (managed) mode work transparently for
unicast — the mac80211 subsystem handles frame translation between 802.11
and 802.3. Broadcast beacon discovery is unreliable in managed mode because
and 802.3. Broadcast neighbor detection is unreliable in managed mode because
access points commonly isolate clients from each other's broadcast traffic.
Startup logging:
@@ -384,7 +382,7 @@ every tick, `poll_pending_connects()` calls `connection_state(addr)` to
check progress. When the transport reports `Connected`, the completed
connection is promoted to the established pool (stream split into
read/write halves, per-connection receive task spawned), and the node
initiates the Noise IK link handshake. If the transport reports `Failed`,
initiates the Noise XX link handshake. If the transport reports `Failed`,
the node schedules a retry with exponential backoff.
As a fallback, `send(addr, data)` still performs synchronous
@@ -422,7 +420,7 @@ socket is created).
The Tor transport routes FIPS traffic through the Tor network, hiding
a node's IP address from its peers. A node behind Tor connects outbound
through a local Tor SOCKS5 proxy; the remote peer sees the Tor exit
node's IP, not the initiator's. After the Noise IK handshake, the remote
node's IP, not the initiator's. After the Noise XX handshake, the remote
peer knows the initiator's FIPS identity (npub) but not its network
location.
@@ -503,7 +501,7 @@ The inbound accept loop mirrors the TCP transport's pattern: accept
connection, configure socket (TCP_NODELAY, keepalive), spawn a
per-connection receive loop using the shared FMP stream reader. Inbound
connections arrive from `127.0.0.1` (Tor daemon's local forwarding); peer
identity is resolved during the Noise IK handshake, not from the transport
identity is resolved during the Noise XX handshake, not from the transport
address.
Configuration requires coordinating `torrc` and `fips.yaml`. The
@@ -895,7 +893,7 @@ transitions through `Starting` to `Up` (operational). `stop()` moves to
| --------- | ------ | ----- |
| UDP/IP | **Implemented** | Primary transport, AsyncFd/recvmsg, SO_RXQ_OVFL kernel drop detection |
| TCP/IP | **Implemented** | FMP header-based framing, non-blocking connect, per-connection MSS MTU |
| Ethernet | **Implemented** | AF_PACKET SOCK_DGRAM, EtherType 0x2121, beacon discovery, Linux only |
| Ethernet | **Implemented** | AF_PACKET SOCK_DGRAM, EtherType 0x2121, neighbor beacons, Linux only |
| WiFi | **Implemented** (via Ethernet transport, infrastructure mode) | mac80211 translates 802.11↔802.3; broadcast beacons unreliable through APs |
| Tor | **Implemented** | Outbound SOCKS5, inbound via onion service, .onion and clearnet addressing |
| Nym | **Implemented** | Outbound-only SOCKS5 through nym-socks5-client, mixnet anonymity, IP/hostname addressing |
docs/design/port-advertisement-and-nat-traversal.md
+1 -1
View File
@@ -457,7 +457,7 @@ authentication or encryption**. The application layer is responsible
for establishing its own security on the punched channel — for
example, a Noise Protocol handshake keyed from the Nostr identity,
or an application-specific authenticated-encryption layer. FIPS
runs its FMP Noise IK handshake immediately after adoption; the
runs its FMP Noise XX handshake immediately after adoption; the
identity proven by the Noise handshake is the same Nostr pubkey
that signed the inner offer/answer rumour, so a man-in-the-middle on
the relay cannot impersonate the responder.
docs/how-to/README.md
+2
View File
@@ -25,4 +25,6 @@ X" to "X is done".
| [persistent-identity.md](persistent-identity.md) | Provision a stable Nostr keypair so the node keeps the same npub across restarts |
| [host-aliases.md](host-aliases.md) | Use shortnames (`test-us01.fips`, `my-laptop.fips`) instead of full npubs by editing `/etc/fips/hosts` or setting peer aliases |
| [set-up-bluetooth-peer.md](set-up-bluetooth-peer.md) | Configure a Bluetooth Low Energy peer link |
| [set-up-80211s-mesh-backhaul.md](set-up-80211s-mesh-backhaul.md) | Link OpenWrt FIPS routers over an open 802.11s radio backhaul (FIPS provides encryption, authentication, and routing) |
| [set-up-open-access-ssid.md](set-up-open-access-ssid.md) | Broadcast the open `!FIPS` access SSID so phones and laptops roam onto the mesh (one ESS: save once, roam every FIPS router) |
| [diagnose-mtu-issues.md](diagnose-mtu-issues.md) | Triage MTU-shaped failures and rule out their imposters (bufferbloat, transport saturation) |
docs/how-to/enable-nostr-discovery.md
+1 -1
View File
@@ -197,7 +197,7 @@ What this achieves: the node publishes a `udp:nat` endpoint plus its
signaling relays in the advert. When either side initiates, an
encrypted offer is sealed to the peer's npub, a matching answer
comes back, and both sides punch at the negotiated time. On success,
the punch socket is adopted as an FMP UDP transport and Noise IK
the punch socket is adopted as an FMP UDP transport and Noise XX
proceeds normally.
> **Validation:** `advertise_on_nostr: true` with `public: false` on
docs/how-to/set-up-80211s-mesh-backhaul.md
+249
View File
@@ -0,0 +1,249 @@
# Set Up an 802.11s Mesh Backhaul (OpenWrt)
Link FIPS routers over radio — no cables, no APs, no shared
infrastructure — by running the Ethernet transport on an open 802.11s
mesh interface. The radio layer provides nothing but L2 frames to
direct neighbors; FIPS provides everything else: encryption and
authentication (the Noise handshake), peer discovery (Ethernet beacons), and
routing (the spanning tree).
For the transport design, see
[../design/fips-transport-layer.md](../design/fips-transport-layer.md).
For all `transports.ethernet.*` configuration keys, see
[../reference/configuration.md](../reference/configuration.md).
## Why open, why forwarding off
Two deliberate choices distinguish this from a stock 802.11s setup:
- **`encryption none`** — the mesh is open on purpose. Every FIPS peer
link is already authenticated and encrypted by the Noise
handshake, so SAE at L2 would duplicate that work, add a shared
credential to provision across routers, and (on ath10k) force the
firmware into its slower raw Tx/Rx mode. A stranger can form an
802.11s peering with your router *and* a FIPS peer link on top of it —
the same open model as mDNS and BLE discovery, where the advert is
only a hint and the handshake authenticates each link (no
impersonation, no MITM) rather than gating who may peer. Admission is
open up to the daemon's max-peers cap. What you concede: any nearby
radio can peer and reach the FIPS overlay surface; L2 metadata (MAC
addresses, frame sizes) is visible in the air; a hostile radio can
burn airtime — all inherent to an open radio link.
- **`mesh_fwding 0`** — disables 802.11s's own HWMP routing so each
mesh link is a plain neighbor link. FIPS is the routing layer; two
routing layers would fight, and broadcast discovery beacons would
flood the whole mesh instead of reaching direct neighbors only.
The interface is **not** bridged into `br-lan` — the FIPS Ethernet
transport binds it directly.
## When to use
- Two or more OpenWrt FIPS routers within radio range of each other,
where running cable is impractical.
- You want the mesh segment to keep working with zero shared
credentials or per-site configuration ("flash and drop in").
It is **not** for connecting phones or laptops — client devices
cannot join an 802.11s mesh. They enter the mesh through a normal AP
on the same router (see constraints below), or over BLE.
## Requirements
- OpenWrt 22.03+ with the FIPS package installed.
- A radio whose driver supports mesh point interfaces. Check with:
```sh
iw list | grep -A 10 "Supported interface modes" | grep "mesh point"
```
The mainstream OpenWrt chips (ath9k, ath10k, mt76) all qualify.
- Ideally a dual- or tri-band router, so one band can be dedicated to
the backhaul (see constraints).
## Step 1 — create the mesh interface(s)
On **each** router, run the helper once per radio you want in the
backhaul:
```sh
fips-mesh-setup radio1
```
This creates an open 802.11s interface with mesh ID `fips-mesh` and
HWMP forwarding off, attaches it to an unmanaged netifd interface (no
IP configuration — none is needed), uncomments the matching `meshN`
transport entry in `/etc/fips/fips.yaml` (see Step 2), and reloads the
radio. Interfaces are named by radio index: `radio0` → `fips-mesh0`,
`radio1` → `fips-mesh1`. Pass a second argument to use a different
mesh ID.
Note: the helper runs `wifi reload`, which re-applies the whole
wireless config and so briefly drops every client AP on all radios for
a few seconds. `fips-mesh-setup remove` reloads the same way. Expect
the blip if clients are connected.
On dual-band routers, meshing **both** bands is worth it: 2.4 GHz
reaches further at lower rates, 5 GHz carries more over shorter
links. Note this is **failover, not multipath**: FIPS keeps one
active link per peer, so traffic uses one band at a time — the other
is a standby that re-establishes the peer if the active link dies
(detection via keepalive timeout, so a cutover takes seconds, not
milliseconds):
```sh
fips-mesh-setup radio0
fips-mesh-setup radio1
```
**Pin the same channel on every backhaul router, per band.** Mesh
points only peer on the same channel, and the mesh inherits whatever
the radio is set to — with `channel 'auto'` (the default on many
devices) each router picks its own and the mesh silently never forms.
The script prints the radio's current band and channel and warns on
`auto`:
```sh
uci set wireless.radio1.channel='36'
uci commit wireless && wifi reload
```
Prefer a non-DFS channel (36–48 on 5 GHz): on DFS channels the radio
must wait ~60 s in CAC before transmitting after every reload.
Equivalent manual UCI (per radio), if you prefer to see what it does:
```sh
uci batch <<'EOF'
set wireless.fips_mesh_radio1=wifi-iface
set wireless.fips_mesh_radio1.device='radio1'
set wireless.fips_mesh_radio1.mode='mesh'
set wireless.fips_mesh_radio1.mesh_id='fips-mesh'
set wireless.fips_mesh_radio1.encryption='none'
set wireless.fips_mesh_radio1.mesh_fwding='0'
set wireless.fips_mesh_radio1.ifname='fips-mesh1'
set wireless.fips_mesh_radio1.network='fips_mesh_radio1'
set network.fips_mesh_radio1=interface
set network.fips_mesh_radio1.proto='none'
EOF
uci commit
wifi reload
```
## Step 2 — check the FIPS transport binding
The `fips.yaml` shipped in the OpenWrt package carries one transport
entry per radio, but **commented out** — so a stock install that never
runs this helper logs no per-boot "interface missing" warning.
`fips-mesh-setup` uncommented the matching `meshN` entry in Step 1, so
there is normally nothing to do here. If you maintain your own config
(or ran the manual UCI above instead of the helper), make sure the
entries are present and uncommented:
```yaml
transports:
ethernet:
mesh0:
interface: "fips-mesh0"
discovery: true
announce: true
auto_connect: true
accept_connections: true
mesh1:
interface: "fips-mesh1"
discovery: true
announce: true
auto_connect: true
accept_connections: true
```
## Step 3 — restart the daemon (order matters)
```sh
/etc/init.d/fips restart
```
Restart fips **after** the mesh interface is up. A transport whose
interface is missing at startup is logged and skipped, not retried —
so if the daemon comes up before the radio, the mesh transport stays
dead until the next restart. (An interface that *vanishes and
returns* after startup is recovered automatically; only the missing-
at-startup case needs this ordering.)
## Verify
L2 first — the 802.11s peering, with a second configured router in
range:
```sh
iw dev fips-mesh0 station dump
```
You should see one station entry per neighbor router, with signal
levels. No entries means a radio problem, not a FIPS problem — triage
in this order:
1. **Channel mismatch** (the most common cause): compare
`iw dev fips-mesh0 info` on both routers — mesh ID *and* channel
must match exactly.
2. **The mesh interface never joined** — `iw dev fips-meshX info`
shows `type mesh point` but **no channel line**, and `station dump`
is empty. Usual cause: a client (`sta`) interface on the same
radio. A STA must follow its upstream AP's channel, the whole
radio follows the STA, and a mesh pinned to a different channel
silently stays down. Check for a STA sharing the radio
(`iw dev`, look for `type managed` on the same phy), compare
`iw dev <sta-iface> info | grep channel`, and re-pin the mesh
channel to match — on every backhaul router.
3. **Is the other router transmitting at all?**
```sh
iw dev fips-mesh0 scan | grep -i -B4 "MESH ID"
```
Its mesh ID visible → transmission works, peering is failing
(mesh ID typo, or one side has encryption set). Nothing visible →
check `wifi status` on the other router, remember the ~60 s DFS
CAC wait, and confirm the country code is set
(`uci get wireless.radio1.country`) — an unset regdomain can
block channels entirely.
4. `logread | grep -iE "mesh|fips-mesh0"` on both sides.
Then the FIPS layer on top:
```sh
logread | grep -i beacon # beacons flowing on the new transport
fipsctl show peers # neighbor authenticated and connected
fipsctl show links # link on the 'ethernet' transport
```
Discovery is automatic: each node beacons its pubkey every few
seconds, and `auto_connect` initiates the Noise handshake on first
sight.
## Constraints
- **Airtime is shared per radio.** All virtual interfaces on one
radio (AP + mesh) share one channel, and multi-hop forwarding on a
single radio roughly halves throughput per hop. On dual/tri-band
hardware, dedicate one band to `fips-mesh0` and serve clients on
the others.
- **AP + mesh coexistence is driver-dependent.** It works on the
mainstream chips (this is the standard Freifunk/Gluon setup), but
check `iw list` under "valid interface combinations" for your
hardware.
- **Clients can't join.** Phones and laptops reach the mesh through
the router's normal AP or via BLE — never through the 802.11s
interface.
- **Radio links are lossy.** A neighbor at the edge of range will
form an 802.11s peering yet deliver a fraction of its frames.
Expect link-quality effects that don't exist on wired Ethernet.
- **A client (STA) uplink on the same radio owns the channel.** The
STA must follow whatever channel its upstream AP uses; every other
interface on that radio follows the STA. A mesh pinned to a
different channel silently never joins, and it does **not** recover
when the STA disconnects — a `wifi reload` (plus a fips restart) is
needed. A *roaming* uplink (travel-router / hotspot-chasing setups)
is fundamentally incompatible with a fixed-channel mesh on the same
radio: dedicate the mesh to the radio the STA never uses, and treat
any mesh sharing a STA radio as best-effort.
docs/how-to/set-up-open-access-ssid.md
+267
View File
@@ -0,0 +1,267 @@
# Set Up the Open FIPS Access SSID (OpenWrt)
Give phones and laptops a way in: every FIPS router broadcasts the
same open SSID — `!FIPS` — from its access radio. Same SSID + unique
BSSIDs is one standard ESS, so a client saves the network once and
roams between all FIPS routers natively, with no per-router setup and
no shared credentials (the Freifunk model). The leading `!` sorts the
network to the top of alphabetically ordered pickers (iOS, desktop
OSes — Android sorts by signal strength) and is part of the name:
SSIDs match byte-for-byte or not at all. The radio layer provides
nothing but open L2 to the nearest router; FIPS provides everything
else: encryption and authentication (Noise IK), discovery
(mDNS/Ethernet beacons), and mobility (the overlay identity survives
roaming, so no 802.11r or L2 tricks are needed).
This is the *access* layer — how clients reach FIPS routers. For the
router-to-router *backhaul*, see
[set-up-80211s-mesh-backhaul.md](set-up-80211s-mesh-backhaul.md).
For all `transports.ethernet.*` configuration keys, see
[../reference/configuration.md](../reference/configuration.md).
## Why open, why this addressing
Three deliberate choices distinguish this from a stock guest network:
- **`encryption none`** — the SSID is open on purpose, and it *must*
be. Clients key a saved network on SSID **plus security type**: if
one router used a PSK and another OWE, the same `FIPS` name would be
three different saved networks and roaming would break. Open is the
only security type that needs zero provisioning, and OWE is left out
for now for exactly this uniformity reason (OWE-transition mode is
inconsistent across client vendors). Every FIPS peer link is already
authenticated and encrypted by the Noise IK handshake. A stranger
can associate *and* form a FIPS peer link — that is the point of open
access; the handshake authenticates each link (no impersonation, no
MITM) but does not gate who may peer, and admission is open up to the
daemon's max-peers cap. What confines a hostile peer is the isolated
`fips_ap` zone (no path to br-lan or the WAN — see below), not the
handshake. What you concede: any nearby device can reach the FIPS
overlay surface (handshake, discovery, lookup, routing) and peer with
the router; L2 metadata is visible in the air; a hostile radio can
burn airtime — all inherent to an open radio link.
- **DHCPv4 from a fixed subnet, plus IPv6 router advertisements.**
dnsmasq leases IPv4 out of `10.21.<N>.0/24` (`N` = the radio index;
the prefix echoes FIPS port 2121). The subnet is deliberately
**identical on every router**: a roaming phone keeps its lease
across routers, and dnsmasq's authoritative mode (the OpenWrt
default, pinned by the helper) ACKs a renew the new router never
issued. odhcpd additionally announces a ULA prefix (`fd..`-range)
for stateless SLAAC; DHCPv6 stays off. FIPS itself only needs
link-local + mDNS, but Android's provisioning check requires an RA
or a DHCP offer and *disconnects* with neither, and plain laptops
expect a real IPv4 address. Works with or without an upstream —
nothing here depends on the WAN. The IPv6 side stays per-router and
disposable; in all cases the FIPS overlay identity, not the IP, is
the mobility anchor.
- **Isolated interface** — its own network and firewall zone, with no
path to `br-lan` and no forwarding to the WAN. Inbound traffic is
rejected except DHCPv4, ICMPv6 (SLAAC itself), mDNS, and the FIPS
transport ports; the raw-Ethernet transport (EtherType 0x2121) is
not IP and never traverses the firewall. AP client isolation is on, so clients
cannot reach each other at L2 — two FIPS phones on one router still
reach each other through the router at the overlay layer.
## The "no internet" behavior (expected, one-time acceptance)
The network intentionally provides **no internet**. On first connect,
a phone's validation probe fails and it asks whether to stay on a
network without internet access — choose **stay connected** and
**don't ask again**. That choice is stored per SSID, so accepting it
once covers every FIPS router anywhere.
After that, the network is marked "connected, no internet"
(unvalidated) and the phone keeps **cellular as its default route**
while staying associated — normal apps never notice the FIPS network
exists. FIPS apps bind their sockets to the Wi-Fi network explicitly,
so mesh traffic flows over Wi-Fi while everything else uses cellular.
## When to use
- Any FIPS router that should serve phones and laptops directly, not
just peer with other routers.
- You want clients to roam between FIPS routers with zero per-router
or per-site configuration.
It is the complement of the 802.11s backhaul: the backhaul links
routers (clients cannot join it), the access SSID admits clients.
Both can share a radio, at an airtime cost (see constraints).
## Requirements
- OpenWrt 22.03+ with the FIPS package installed (fw4; dnsmasq and
odhcpd are part of the default images).
- Any radio — AP mode needs no special driver support.
## Step 1 — create the access point(s)
On **each** router, run the helper once per radio that should serve
clients:
```sh
fips-ap-setup radio0
```
This creates an open AP with SSID `!FIPS` and client isolation, an
isolated network with `10.21.<N>.1/24` and a static ULA `/64`, a
DHCPv4 + RA dhcp config (dnsmasq leases, SLAAC, no DHCPv6), and a
locked-down `fips_ap` firewall zone — then reloads the radio. Interfaces are named by radio index: `radio0` →
`fips-ap0`, `radio1` → `fips-ap1`. Pass a second argument to use a
different SSID — but the SSID, like the security type, must be
identical on **all** routers or clients will treat them as separate
networks and stop roaming.
On dual-band routers, run it for both radios so clients can pick
either band:
```sh
fips-ap-setup radio0
fips-ap-setup radio1
```
**Channels are free per router.** Unlike the mesh backhaul, there is
no same-channel constraint — clients scan when they roam — so leave
each router on whatever channel suits its RF environment.
Equivalent manual UCI (per radio), if you prefer to see what it does
(`fdxx:...` stands for a `/64` out of the router's ULA prefix):
```sh
uci batch <<'EOF'
set wireless.fips_ap_radio0=wifi-iface
set wireless.fips_ap_radio0.device='radio0'
set wireless.fips_ap_radio0.mode='ap'
set wireless.fips_ap_radio0.ssid='!FIPS'
set wireless.fips_ap_radio0.encryption='none'
set wireless.fips_ap_radio0.isolate='1'
set wireless.fips_ap_radio0.ifname='fips-ap0'
set wireless.fips_ap_radio0.network='fips_ap_radio0'
set network.fips_ap_radio0=interface
set network.fips_ap_radio0.proto='static'
set network.fips_ap_radio0.ipaddr='10.21.0.1'
set network.fips_ap_radio0.netmask='255.255.255.0'
set network.fips_ap_radio0.ip6addr='fdxx:xxxx:xxxx:fa00::1/64'
set dhcp.fips_ap_radio0=dhcp
set dhcp.fips_ap_radio0.interface='fips_ap_radio0'
set dhcp.fips_ap_radio0.ra='server'
set dhcp.fips_ap_radio0.ra_default='2'
set dhcp.fips_ap_radio0.dhcpv6='disabled'
set dhcp.fips_ap_radio0.dhcpv4='server'
set dhcp.fips_ap_radio0.start='10'
set dhcp.fips_ap_radio0.limit='200'
EOF
uci commit
wifi reload
```
plus the `fips_ap` firewall zone (input/forward REJECT, no
forwardings, ACCEPT rules for DHCPv4/UDP 67, ICMPv6, UDP 5353/2121,
TCP 8443).
## Step 2 — check the FIPS transport binding
The `fips.yaml` shipped in the OpenWrt package carries one transport
entry per access interface, but **commented out** — so a stock install
that never runs this helper logs no per-boot "interface missing"
warning. `fips-ap-setup` uncommented the matching `apN` entry in Step 1,
and also enabled `node.rendezvous.lan` (the daemon's mDNS/DNS-SD
rendezvous — phone FIPS apps cannot see raw-Ethernet beacons, so mDNS
is how they find the daemon; the switch is daemon-wide and stays on if
you later remove the AP). So there is normally nothing to do here. If
you maintain your own config (or ran the manual UCI above instead of
the helper), make sure both are present and uncommented:
```yaml
node:
rendezvous:
lan:
enabled: true
```
```yaml
transports:
ethernet:
ap0:
interface: "fips-ap0"
discovery: true
announce: true
auto_connect: true
accept_connections: true
ap1:
interface: "fips-ap1"
discovery: true
announce: true
auto_connect: true
accept_connections: true
```
## Step 3 — restart the daemon (order matters)
```sh
/etc/init.d/fips restart
```
Restart fips **after** the AP interface is up. A transport whose
interface is missing at startup is logged and skipped, not retried —
so if the daemon comes up before the radio, the access transport
stays dead until the next restart. (An interface that *vanishes and
returns* after startup is recovered automatically; only the missing-
at-startup case needs this ordering.)
## Verify
L2 and addressing first, with a phone or laptop connected to `!FIPS`:
```sh
iw dev fips-ap0 station dump # one entry per associated client
ip addr show dev fips-ap0 # 10.21.0.1/24 and the fd..::1/64
cat /tmp/dhcp.leases # one lease per connected client
```
No station entries means a radio problem; an association that drops
after ~30 s usually means the client never got an address — check
`logread | grep -e dnsmasq -e odhcpd` and that the
`dhcp.fips_ap_radio0` section survived
(`uci show dhcp | grep fips_ap`).
Then the FIPS layer on top, for a client running FIPS:
```sh
logread | grep -i beacon # beacons flowing on the new transport
fipsctl show peers # client authenticated and connected
```
On the phone itself: the network shows "connected, no internet" and
stays associated — that is the designed steady state, not an error.
## Constraints
- **SSID and security type must be uniform across ALL routers.**
One router with a PSK (or OWE) under the same name splits the ESS
into different saved networks and silently breaks roaming. Never
"harden" a single router.
- **Airtime is shared per radio.** An access AP and a mesh backhaul
on the same radio share one channel. On dual/tri-band hardware,
dedicate a band to the backhaul and serve clients on the others.
- **Strangers can associate and peer — by design.** Open access means
any nearby device can complete the Noise handshake and become a FIPS
peer (up to the max-peers cap); the handshake authenticates each link,
it does not restrict who joins. They reach only the FIPS overlay
surface — the isolated zone gives no path to br-lan or the WAN. Do not
add forwardings to the `fips_ap` zone: that would turn the open SSID
into a hotspot and hand the isolation away.
- **Roaming is client-driven.** Clients decide when to hop BSSIDs
(standard ESS behavior); the IPv4 lease survives the hop (same
subnet everywhere), the SLAAC address renumbers, and FIPS sessions
ride through because the overlay identity is the anchor. Expect a
brief L2 gap during the hop, as on any ESS without 802.11r.
- **The `10.21.<N>.0/24` convention must hold everywhere.** Lease
survival depends on every router serving the same subnet from the
same radio index — the helper guarantees this; don't hand-pick
per-router subnets. Two routers can lease the same address to two
different clients; after a roam the conflict is caught (dnsmasq
NAKs a renew for an address in use) and the client re-DHCPs. If a
laptop is *also* wired to a LAN that really uses `10.21.<N>.0/24`,
its routing table will conflict — a corner case worth knowing, not
designing around: the zone forwards nowhere, so the FIPS side never
reaches beyond the router either way.
docs/reference/cli-fips.md
+1 -1
View File
@@ -28,7 +28,7 @@ controlled through the standard service control manager.
| Flag | Argument | Description |
| ---- | -------- | ----------- |
| `-c`, `--config` | `FILE` | Use `FILE` as the configuration. Skips the default search paths. |
| `-V` | — | Print the short version (e.g. `0.4.0 (rev abcdef1)`). |
| `-V` | — | Print the short version (e.g. `0.5.0-dev (rev abcdef1)`). |
| `--version` | — | Print the long version: short version plus build target triple. |
| `-h`, `--help` | — | Print usage and exit. |
| `--install-service` | — | (Windows only) Install `fips` as a Windows service. Requires Administrator. |
docs/reference/cli-fipsctl.md
+52
View File
@@ -115,6 +115,58 @@ Tell the daemon to drop a peer link.
| -------- | ----------- |
| `peer` | npub (bech32) or hostname from `/etc/fips/hosts`. |
### `profile tick <on|off|status>`
> **Reading the output.** Step durations are wall clock measured across `await`
> points, not CPU time: a step that waits on I/O accrues that wait, and other
> tasks may run inside the span. That is the intended measure for head-of-line
> delay, and it means a large step is not necessarily an expensive one.
> `arm_starvation` is measured directly as the entry time minus the deadline
> the interval scheduled that tick for. It is not derived from
> `tick_entry_gap`, which carries no starvation signal on its own: under a
> steady delay every gap is exactly one tick period.
Start, stop and inspect a capture of the rx-loop tick body. **Present
only when both `fipsctl` and the daemon are built with
`--features profiling`**; the feature is off by default, so a stock
package does not carry this subcommand and a stock daemon reports
`profile_tick_*` as an unknown command.
| Subcommand | Control-socket command | Description |
| ---------- | ---------------------- | ----------- |
| `profile tick on` | `profile_tick_on` | Create the capture file and start recording. Fails if a capture is already running (naming the active file) or if the directory cannot be written. |
| `profile tick off` | `profile_tick_off` | Stop the capture. The writer is woken immediately, drains once more and is joined, so the command returns promptly. Succeeds, reporting nothing active, when no capture is running. |
| `profile tick status` | `profile_tick_status` | Report `idle`, `running`, `stopped_by_cap` or `stopped_by_error`, plus the active path, bytes written, flush interval and byte cap. |
`profile tick on` options:
| Flag | Argument | Default | Description |
| ---- | -------- | ------- | ----------- |
| `--dir` | directory path | `/var/log/fips` | Where to write the capture. Created if absent. Use it to profile a non-root `cargo run`, or on a platform whose log root differs. |
One file is written per capture, named `profile-<UTC timestamp>.tsv`.
It opens with a `#`-prefixed header block (node npub, build version,
platform, configured tick period, flush interval, byte cap, start
time), then a tab-separated column header, then one row per measured
step per flush interval:
```text
ts_unix kind domain name count max total unit
```
`kind` is `step` for a timed span and `gauge` for a sampled scalar, so
a gauge value never lands under a duration column; `unit` names the
unit of `max` and `total` for that row. Every step present in the build
gets a row every interval, including zero-count rows. Gauges cover
ticks per interval, peer count, the wall gap between successive
tick-arm entries, and the arm-starvation delay, which is measured
against the deadline the tick was scheduled for rather than derived
from the gap.
A capture stops itself on reaching 32 MB, appending a `#` line saying
so; `profile tick status` then reports `stopped_by_cap` until the next
`on` or `off` clears it.
## Exit Codes
| Code | Meaning |
docs/reference/configuration.md
+12 -10
View File
@@ -104,7 +104,8 @@ to the highest-priority config file for operator visibility, even in ephemeral m
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.leaf_only` | bool | `false` | Leaf-only mode: node does not forward traffic or participate in routing |
| `node.disable_routing` | bool | `false` | Non-routing mode: participates in spanning tree but does not forward transit traffic or send bloom filters |
| `node.leaf_only` | bool | `false` | Leaf mode: single upstream peer, no tree/bloom/transit participation. Implies `disable_routing: true` |
| `node.tick_interval_secs` | u64 | `1` | Periodic maintenance tick interval (retry checks, timeout cleanup, tree refresh) |
| `node.base_rtt_ms` | u64 | `100` | Initial RTT estimate for new links before measurements converge |
| `node.heartbeat_interval_secs` | u64 | `10` | Heartbeat send interval per peer for liveness detection |
@@ -124,7 +125,7 @@ Controls capacity for connections, peers, and links.
### Rate Limiting (`node.rate_limit.*`)
Handshake rate limiting protects against DoS on the Noise IK handshake path.
Handshake rate limiting protects against DoS on the Noise XX handshake path.
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
@@ -304,7 +305,7 @@ stays set for all subsequent hops to the destination.
### Rekey (`node.rekey.*`)
Controls periodic Noise rekey for forward secrecy. When enabled, both FMP
(link-layer IK) and FSP (session-layer XK) sessions perform fresh Diffie-Hellman
(link-layer XX) and FSP (session-layer XX) sessions perform fresh Diffie-Hellman
key exchanges after a time or message count threshold, whichever comes first.
A 10-second drain window keeps the old session active for decryption during
cutover.
@@ -338,7 +339,7 @@ Metrics Measurement Protocol for per-peer link measurement. See
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `node.mmp.mode` | string | `"full"` | Operating mode: `full` (sender + receiver reports), `lightweight` (receiver reports only), or `minimal` (spin bit + CE echo only, no reports) |
| `node.mmp.mode` | string | `"full"` | Operating mode: `full` (sender + receiver reports), `lightweight` (receiver reports only), or `minimal` (CE echo only, no reports) |
| `node.mmp.log_interval_secs` | u64 | `30` | Periodic operator log interval for link metrics |
| `node.mmp.owd_window_size` | usize | `32` | One-way delay trend ring buffer size |
@@ -436,7 +437,7 @@ Requires `CAP_NET_RAW` or running as root. Linux only.
| `mtu` | u16 | *(auto)* | Override MTU. Default: interface MTU minus 3 (for frame type + length prefix) |
| `recv_buf_size` | usize | `2097152` | Socket receive buffer size in bytes (2 MB) |
| `send_buf_size` | usize | `2097152` | Socket send buffer size in bytes (2 MB) |
| `discovery` | bool | `true` | Listen for discovery beacons from other nodes |
| `listen` | bool | `true` | Listen for neighbor beacons from other nodes |
| `announce` | bool | `false` | Broadcast announcement beacons on the LAN |
| `auto_connect` | bool | `false` | Auto-connect to discovered peers |
| `accept_connections` | bool | `false` | Accept incoming connection attempts from discovered peers |
@@ -450,7 +451,7 @@ transports:
ethernet:
lan:
interface: "eth0"
discovery: true
listen: true
announce: true
backbone:
interface: "eth1"
@@ -458,7 +459,7 @@ transports:
```
Each named instance operates independently with its own socket and
discovery state. The instance name is used in log messages and the
neighbor state. The instance name is used in log messages and the
`name()` method on the Transport trait.
### TCP (`transports.tcp.*`)
@@ -840,7 +841,7 @@ peers:
### Mixed UDP + Ethernet Example
A node bridging internet peers (UDP) and a local Ethernet segment with
beacon discovery:
neighbor beacons:
```yaml
node:
@@ -856,7 +857,7 @@ transports:
mtu: 1472
ethernet:
interface: "eth0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
@@ -899,6 +900,7 @@ node:
identity:
nsec: null # secret key in nsec or hex (null = depends on persistent)
persistent: false # true = load/save fips.key; false = ephemeral each start
disable_routing: false
leaf_only: false
tick_interval_secs: 1
base_rtt_ms: 100
@@ -999,7 +1001,7 @@ transports:
# mtu: null # null = interface MTU - 3 (typically 1497)
# recv_buf_size: 2097152 # 2 MB
# send_buf_size: 2097152 # 2 MB
# discovery: true # listen for beacons
# listen: true # listen for beacons
# announce: false # broadcast beacons
# auto_connect: false # connect to discovered peers
# accept_connections: false # accept inbound handshakes
docs/reference/control-socket.md
+15
View File
@@ -159,6 +159,21 @@ not reproduced here to avoid duplicating the source.
Both commands run on the daemon's main task and may block briefly
while the node mutates its state.
#### Profiler toggle (`--features profiling` builds only)
| Command | Params | Behaviour |
| ------- | ------ | --------- |
| `profile_tick_on` | `dir` (optional directory path; default `/var/log/fips`) | Creates the capture file, publishes its path, and starts the writer thread. `data`: `state`, `path`, `interval_secs`, `byte_cap`. Errors if a capture is already running (naming the active file) or the directory is unwritable. |
| `profile_tick_off` | — | Stops the capture, drains once more, joins the writer. `data`: `state`, `stopped`, `stopped_by_cap`, `stopped_by_error`, `path`, `bytes`. |
| `profile_tick_status` | — | `data`: `state` (`idle` / `running` / `stopped_by_cap` / `stopped_by_error`), `path`, `bytes`, `byte_cap`, `interval_secs`. |
Unlike `connect` and `disconnect`, these three are served in the
control accept task rather than on the daemon's main task. All of their
state is process statics and none of them needs `&mut Node`, so
routing them through the main loop would only make the toggle queue
behind the tick body it exists to measure. They are absent from a
default build, where the daemon answers them as unknown commands.
## Gateway Command Catalog
`fips-gateway` exposes a separate control socket with its own command
docs/reference/diagrams/filter-announce.svg
+34 -25
View File
@@ -1,43 +1,52 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 270" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 374" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="270" fill="#1a1a2e" rx="4"/>
<rect width="580" height="374" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FilterAnnounce (0x20) — 11 + filter bytes</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FilterAnnounce (0x20) &#8212; 19-byte header + RLE payload</text>
<!-- Row 0 (0–3): msg_type(1) + sequence starts -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<!-- Row 0 (0-1): msg_type + flags -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;1</text>
<rect x="55" y="36" width="130" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="120" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">0x20</text>
<text x="120" y="56" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="74" fill="#8ab4f8" text-anchor="middle" font-size="10">0x20</text>
<rect x="185" y="36" width="390" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="380" y="66" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sequence</text>
<rect x="185" y="36" width="390" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="380" y="56" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="380" y="74" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte &#8212; bit 0: delta (XOR diff)</text>
<!-- Row 1 (4–8): sequence continued -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–8</text>
<!-- Row 1 (2-9): sequence -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">2&#8211;9</text>
<rect x="55" y="88" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="118" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE — monotonic counter</text>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sequence</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE &#8212; per-peer monotonic counter</text>
<!-- Row 2 (9–10): hash_count + size_class -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">9–10</text>
<!-- Row 2 (10-17): base_seq -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">10&#8211;17</text>
<rect x="55" y="140" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="185" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">hash_count</text>
<text x="185" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte</text>
<rect x="55" y="140" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">base_seq</text>
<text x="315" y="180" fill="#d8a0f8" text-anchor="middle" font-size="10">8 bytes LE &#8212; reference sequence for delta (0 if full)</text>
<rect x="315" y="140" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">size_class</text>
<text x="445" y="180" fill="#d8a0f8" text-anchor="middle" font-size="10">1 byte</text>
<!-- Row 3 (18): size_class -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">18</text>
<!-- Row 3 (11–): filter_bits -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">11–</text>
<rect x="55" y="192" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">size_class</text>
<text x="315" y="232" fill="#8af8f8" text-anchor="middle" font-size="10">1 byte &#8212; filter size = 512 &#171; size_class bytes</text>
<rect x="55" y="198" width="520" height="40" fill="#1f1f3a" stroke="#4ad9d9" stroke-width="1" stroke-dasharray="4,3" rx="3"/>
<text x="315" y="222" fill="#8af8f8" text-anchor="middle" font-size="11">filter_bits (variable, 512 &lt;&lt; size_class bytes)</text>
<!-- Row 4 (19-): compressed_payload -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">19&#8211;</text>
<rect x="55" y="250" width="520" height="40" fill="#1f1f3a" stroke="#4a9090" stroke-width="1" stroke-dasharray="4,3" rx="3"/>
<text x="315" y="274" fill="#8ad8d8" text-anchor="middle" font-size="11">compressed_payload (RLE: [count:2 LE][word:8 LE] per run)</text>
<!-- Explanation -->
<rect x="55" y="304" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="324" fill="#888" text-anchor="middle" font-size="10">delta: XOR diff of current vs last-sent filter &#8212; full: raw filter words</text>
<!-- Total -->
<text x="310" y="258" fill="#777" font-size="10" text-anchor="middle">v1 payload: 1,035 bytes (11 header + 1,024 filter)</text>
<text x="310" y="362" fill="#777" font-size="10" text-anchor="middle">19-byte header + variable compressed payload</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.6 KiB

After

Width:  |  Height:  |  Size: 3.3 KiB

docs/reference/diagrams/handshake-flow.svg
+43 -34
View File
@@ -1,26 +1,23 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 620 500" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 620 620" font-family="monospace" font-size="13">
<defs>
<marker id="arrowR" markerWidth="10" markerHeight="8" refX="9" refY="4" orient="auto" markerUnits="userSpaceOnUse">
<polygon points="0,0 10,4 0,8" fill="#e0e0e0"/>
</marker>
<marker id="arrowL" markerWidth="10" markerHeight="8" refX="1" refY="4" orient="auto" markerUnits="userSpaceOnUse">
<polygon points="10,0 0,4 10,8" fill="#e0e0e0"/>
</marker>
</defs>
<!-- Background -->
<rect width="620" height="500" fill="#1a1a2e" rx="4"/>
<rect width="620" height="620" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FMP Handshake Flow (Noise IK)</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">FMP Handshake Flow (Noise XX)</text>
<!-- Column headers -->
<text x="100" y="54" fill="#8ab4f8" text-anchor="middle" font-size="12" font-weight="bold">Initiator</text>
<text x="520" y="54" fill="#8ab4f8" text-anchor="middle" font-size="12" font-weight="bold">Responder</text>
<!-- Lifelines -->
<line x1="100" y1="62" x2="100" y2="480" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<line x1="520" y1="62" x2="520" y2="480" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<line x1="100" y1="62" x2="100" y2="600" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<line x1="520" y1="62" x2="520" y2="600" stroke="#333" stroke-width="1" stroke-dasharray="4,4"/>
<!-- Step 1: Initiator prepares -->
<text x="20" y="88" fill="#8af8c8" font-size="10">generates sender_idx</text>
@@ -28,39 +25,51 @@
<!-- Arrow 1: msg1 (initiator -> responder) -->
<line x1="100" y1="120" x2="520" y2="120" stroke="#4a90d9" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="130" y="128" width="360" height="24" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1" rx="3"/>
<text x="310" y="144" fill="#e0e0e0" text-anchor="middle" font-size="10">[0x01|flags=0|len] | sender_idx | noise_msg1</text>
<text x="310" y="166" fill="#666" text-anchor="middle" font-size="9">phase 0x1 — 114 bytes</text>
<rect x="145" y="128" width="330" height="24" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1" rx="3"/>
<text x="310" y="144" fill="#e0e0e0" text-anchor="middle" font-size="10">[0x11|flags=0|len] | sender_idx | noise_msg1</text>
<text x="310" y="166" fill="#666" text-anchor="middle" font-size="9">phase 0x1 &#8212; 41 bytes &#8212; pattern: &#8594; e</text>
<!-- Step 2: Responder processes (right-aligned to stay in bounds) -->
<text x="600" y="192" fill="#8af8c8" font-size="10" text-anchor="end">validates msg1</text>
<text x="600" y="206" fill="#8af8c8" font-size="10" text-anchor="end">learns initiator's static key</text>
<text x="600" y="220" fill="#8af8c8" font-size="10" text-anchor="end">generates sender_idx</text>
<text x="600" y="234" fill="#8af8c8" font-size="10" text-anchor="end">generates ephemeral keypair</text>
<!-- Step 2: Responder processes msg1 -->
<text x="600" y="192" fill="#8af8c8" font-size="10" text-anchor="end">validates msg1 (ephemeral only)</text>
<text x="600" y="206" fill="#8af8c8" font-size="10" text-anchor="end">generates sender_idx</text>
<text x="600" y="220" fill="#8af8c8" font-size="10" text-anchor="end">generates ephemeral keypair</text>
<!-- Arrow 2: msg2 (responder -> initiator) -->
<!-- Draw line right-to-left so orient="auto" points the arrowhead left -->
<line x1="520" y1="256" x2="100" y2="256" stroke="#4ad99a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="130" y="264" width="360" height="24" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1" rx="3"/>
<text x="310" y="280" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x02|flags=0|len] | sender_idx | receiver_idx | noise_msg2</text>
<text x="310" y="300" fill="#666" text-anchor="middle" font-size="9">phase 0x2 — 69 bytes</text>
<line x1="520" y1="244" x2="100" y2="244" stroke="#4ad99a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="252" width="390" height="24" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1" rx="3"/>
<text x="310" y="268" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x12|flags=0|len] | sender_idx | receiver_idx | noise_msg2 + negotiation</text>
<text x="310" y="288" fill="#666" text-anchor="middle" font-size="9">phase 0x2 &#8212; 144 bytes &#8212; pattern: &#8592; e, ee, s, es</text>
<text x="600" y="300" fill="#d8a0f8" font-size="9" text-anchor="end">responder identity revealed</text>
<!-- Step 3: Initiator completes -->
<text x="20" y="324" fill="#8af8c8" font-size="10">validates msg2</text>
<text x="20" y="338" fill="#8af8c8" font-size="10">derives session keys</text>
<!-- Step 3: Initiator processes msg2, learns responder identity -->
<text x="20" y="322" fill="#8af8c8" font-size="10">validates msg2</text>
<text x="20" y="336" fill="#8af8c8" font-size="10">learns responder identity</text>
<text x="20" y="350" fill="#8af8c8" font-size="10">checks epoch (restart detection)</text>
<!-- Arrow 3: msg3 (initiator -> responder) -->
<line x1="100" y1="368" x2="520" y2="368" stroke="#d94a6a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="376" width="390" height="24" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1" rx="3"/>
<text x="310" y="392" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x13|flags=0|len] | sender_idx | receiver_idx | noise_msg3 + neg</text>
<text x="310" y="412" fill="#666" text-anchor="middle" font-size="9">phase 0x3 &#8212; 111 bytes &#8212; pattern: &#8594; s, se</text>
<text x="20" y="424" fill="#d8a0f8" font-size="9">initiator identity revealed</text>
<!-- Step 4: Responder processes msg3, learns initiator identity -->
<text x="600" y="440" fill="#8af8c8" font-size="10" text-anchor="end">validates msg3</text>
<text x="600" y="454" fill="#8af8c8" font-size="10" text-anchor="end">learns initiator identity</text>
<text x="600" y="468" fill="#8af8c8" font-size="10" text-anchor="end">checks epoch, derives session keys</text>
<!-- Handshake complete separator -->
<line x1="30" y1="360" x2="230" y2="360" stroke="#d9904a" stroke-width="1.5"/>
<text x="310" y="364" fill="#d9904a" text-anchor="middle" font-size="11" font-weight="bold">HANDSHAKE COMPLETE</text>
<line x1="390" y1="360" x2="590" y2="360" stroke="#d9904a" stroke-width="1.5"/>
<line x1="30" y1="488" x2="230" y2="488" stroke="#d9904a" stroke-width="1.5"/>
<text x="310" y="492" fill="#d9904a" text-anchor="middle" font-size="11" font-weight="bold">HANDSHAKE COMPLETE</text>
<line x1="390" y1="488" x2="590" y2="488" stroke="#d9904a" stroke-width="1.5"/>
<!-- Arrow 3: first encrypted frame -->
<text x="20" y="392" fill="#777" font-size="10">first encrypted frame:</text>
<line x1="100" y1="406" x2="520" y2="406" stroke="#d94a6a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="414" width="390" height="24" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1" rx="3"/>
<text x="310" y="430" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x00|flags|len] | receiver_idx | counter=0 | ciphertext+tag</text>
<text x="310" y="450" fill="#666" text-anchor="middle" font-size="9">phase 0x0 — established frame</text>
<!-- Arrow 4: first encrypted frame -->
<text x="20" y="516" fill="#777" font-size="10">first encrypted frame:</text>
<line x1="100" y1="530" x2="520" y2="530" stroke="#d9b04a" stroke-width="1.5" marker-end="url(#arrowR)"/>
<rect x="115" y="538" width="390" height="24" fill="#4a3d2d" stroke="#d9b04a" stroke-width="1" rx="3"/>
<text x="310" y="554" fill="#e0e0e0" text-anchor="middle" font-size="9">[0x00|flags|len] | receiver_idx | counter=0 | ciphertext+tag</text>
<text x="310" y="574" fill="#666" text-anchor="middle" font-size="9">phase 0x0 &#8212; established frame</text>
<!-- Legend -->
<text x="310" y="478" fill="#555" text-anchor="middle" font-size="9">Both parties hold identical symmetric keys. Epoch exchange enables restart detection.</text>
<text x="310" y="600" fill="#555" text-anchor="middle" font-size="9">Both parties hold identical symmetric keys. Epoch + negotiation exchanged in msg2/msg3.</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.2 KiB

After

Width:  |  Height:  |  Size: 5.1 KiB

docs/reference/diagrams/noise-ik-msg1.svg
-67
View File
@@ -1,67 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 426" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="426" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise IK Message 1 — phase 0x1 (114 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
<text x="102" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="150" y="36" width="95" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="197" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">phase</text>
<text x="197" y="78" fill="#8af8c8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="245" y="36" width="130" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="310" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="310" y="78" fill="#f8c88a" text-anchor="middle" font-size="10">1 byte</text>
<rect x="375" y="36" width="200" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="475" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">payload_len</text>
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx (bytes 4-7) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–7</text>
<rect x="55" y="88" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="315" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise IK msg1 label -->
<text x="10" y="250" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 250)">Noise IK msg1 (106 bytes)</text>
<line x1="18" y1="140" x2="18" y2="348" stroke="#555" stroke-width="1"/>
<line x1="18" y1="140" x2="23" y2="140" stroke="#555" stroke-width="1"/>
<line x1="18" y1="348" x2="23" y2="348" stroke="#555" stroke-width="1"/>
<!-- Rows 2-3: ephemeral_pubkey (bytes 8-40, 33 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8–40</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="170" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<rect x="55" y="192" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="222" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Rows 4-5: encrypted_static (bytes 41-89, 49 bytes) -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">41–89</text>
<rect x="55" y="244" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="274" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_static</text>
<rect x="55" y="296" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="326" fill="#f88aaa" text-anchor="middle" font-size="10">49 bytes (static key 33 + AEAD tag 16)</text>
<!-- Row 6: encrypted_epoch (bytes 90-113, 24 bytes) -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">90–113</text>
<rect x="55" y="348" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="372" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="390" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<!-- Total -->
<text x="310" y="418" fill="#777" font-size="10" text-anchor="middle">total: 114 bytes · payload_len = 110 · pattern: e, es, s, ss</text>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 4.3 KiB

docs/reference/diagrams/noise-ik-msg2.svg → docs/reference/diagrams/noise-xx-msg1.svg
+11 -33
View File
@@ -1,12 +1,12 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 374" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 218" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="374" fill="#1a1a2e" rx="4"/>
<rect width="580" height="218" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise IK Message 2 — phase 0x2 (69 bytes)</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise XX Message 1 &#8212; phase 0x1 (41 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
@@ -25,41 +25,19 @@
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx (bytes 4-7) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–7</text>
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="315" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 2: receiver_idx (bytes 8-11) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8–11</text>
<!-- Row 2: ephemeral_pubkey (bytes 8-40, 33 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8&#8211;40</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">receiver_idx</text>
<text x="315" y="182" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise IK msg2 bracket -->
<text x="10" y="270" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 270)">Noise IK msg2 (57 bytes)</text>
<line x1="18" y1="192" x2="18" y2="348" stroke="#555" stroke-width="1"/>
<line x1="18" y1="192" x2="23" y2="192" stroke="#555" stroke-width="1"/>
<line x1="18" y1="348" x2="23" y2="348" stroke="#555" stroke-width="1"/>
<!-- Rows 3-4: ephemeral_pubkey (bytes 12-44, 33 bytes) -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">12–44</text>
<rect x="55" y="192" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="222" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<rect x="55" y="244" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="274" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Row 5: encrypted_epoch (bytes 45-68, 24 bytes) -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">45–68</text>
<rect x="55" y="296" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="320" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="338" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<text x="315" y="182" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Total -->
<text x="310" y="366" fill="#777" font-size="10" text-anchor="middle">total: 69 bytes · payload_len = 65 · pattern: e, ee, se</text>
<text x="310" y="210" fill="#777" font-size="10" text-anchor="middle">total: 41 bytes &#183; payload_len = 37 &#183; pattern: &#8594; e</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.1 KiB

After

Width:  |  Height:  |  Size: 2.7 KiB

docs/reference/diagrams/noise-xx-msg2.svg
+82
View File
@@ -0,0 +1,82 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 478" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="478" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise XX Message 2 &#8212; phase 0x2 (118&#8211;144 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
<text x="102" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="150" y="36" width="95" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="197" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">phase</text>
<text x="197" y="78" fill="#8af8c8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="245" y="36" width="130" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="310" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="310" y="78" fill="#f8c88a" text-anchor="middle" font-size="10">1 byte</text>
<rect x="375" y="36" width="200" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="475" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">payload_len</text>
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx (bytes 4-7) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="185" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- receiver_idx (bytes 8-11) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end"/>
<rect x="315" y="88" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="445" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">receiver_idx</text>
<text x="445" y="130" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise XX msg2 bracket -->
<text x="10" y="290" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 290)">Noise XX msg2 (106 bytes base)</text>
<line x1="18" y1="140" x2="18" y2="400" stroke="#555" stroke-width="1"/>
<line x1="18" y1="140" x2="23" y2="140" stroke="#555" stroke-width="1"/>
<line x1="18" y1="400" x2="23" y2="400" stroke="#555" stroke-width="1"/>
<!-- Row 2: ephemeral_pubkey (bytes 12-44, 33 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12&#8211;44</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ephemeral_pubkey</text>
<text x="315" y="182" fill="#f8c88a" text-anchor="middle" font-size="10">33 bytes (compressed secp256k1)</text>
<!-- Row 3: encrypted_static (bytes 45-93, 49 bytes) -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">45&#8211;93</text>
<rect x="55" y="192" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="216" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_static</text>
<text x="315" y="234" fill="#f88aaa" text-anchor="middle" font-size="10">49 bytes (static key 33 + AEAD tag 16)</text>
<!-- Row 4: encrypted_epoch (bytes 94-117, 24 bytes) -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">94&#8211;117</text>
<rect x="55" y="244" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="268" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="286" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<!-- Row 5: negotiation payload (variable, optional) -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">118+</text>
<rect x="55" y="296" width="520" height="52" fill="#2d4a4a" stroke="#4a9090" stroke-width="1.5" rx="3" stroke-dasharray="4,3"/>
<text x="315" y="320" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted negotiation payload</text>
<text x="315" y="338" fill="#8ad8d8" text-anchor="middle" font-size="10">26 bytes typical (10 payload + 16 AEAD tag)</text>
<!-- Row 6: AEAD explanation -->
<rect x="55" y="362" width="520" height="38" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="386" fill="#888" text-anchor="middle" font-size="10">negotiation appended via encrypt_payload() &#8212; extends Noise hash chain</text>
<!-- Total -->
<text x="310" y="425" fill="#777" font-size="10" text-anchor="middle">base: 118 bytes &#183; with FMP negotiation: 144 bytes</text>
<text x="310" y="442" fill="#777" font-size="10" text-anchor="middle">pattern: &#8592; e, ee, s, es</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 5.4 KiB

docs/reference/diagrams/noise-xx-msg3.svg
+72
View File
@@ -0,0 +1,72 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 426" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="426" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">Noise XX Message 3 &#8212; phase 0x3 (85&#8211;111 bytes)</text>
<!-- Row 0: Common prefix (bytes 0-3) -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="95" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="102" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ver</text>
<text x="102" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="150" y="36" width="95" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="197" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">phase</text>
<text x="197" y="78" fill="#8af8c8" text-anchor="middle" font-size="10">4 bits</text>
<rect x="245" y="36" width="130" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="310" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">flags</text>
<text x="310" y="78" fill="#f8c88a" text-anchor="middle" font-size="10">1 byte</text>
<rect x="375" y="36" width="200" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="475" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">payload_len</text>
<text x="475" y="78" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 1: sender_idx + receiver_idx (bytes 4-11) -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;11</text>
<rect x="55" y="88" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">sender_idx</text>
<text x="185" y="130" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="315" y="88" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="445" y="112" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">receiver_idx</text>
<text x="445" y="130" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Noise XX msg3 bracket -->
<text x="10" y="240" fill="#777" font-size="10" text-anchor="middle" transform="rotate(-90, 10, 240)">Noise XX msg3 (73 bytes base)</text>
<line x1="18" y1="140" x2="18" y2="348" stroke="#555" stroke-width="1"/>
<line x1="18" y1="140" x2="23" y2="140" stroke="#555" stroke-width="1"/>
<line x1="18" y1="348" x2="23" y2="348" stroke="#555" stroke-width="1"/>
<!-- Row 2: encrypted_static (bytes 12-60, 49 bytes) -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12&#8211;60</text>
<rect x="55" y="140" width="520" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="164" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_static</text>
<text x="315" y="182" fill="#f88aaa" text-anchor="middle" font-size="10">49 bytes (static key 33 + AEAD tag 16)</text>
<!-- Row 3: encrypted_epoch (bytes 61-84, 24 bytes) -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">61&#8211;84</text>
<rect x="55" y="192" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="216" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted_epoch</text>
<text x="315" y="234" fill="#d8a0f8" text-anchor="middle" font-size="10">24 bytes (epoch 8 + AEAD tag 16)</text>
<!-- Row 4: negotiation payload (variable, optional) -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">85+</text>
<rect x="55" y="244" width="520" height="52" fill="#2d4a4a" stroke="#4a9090" stroke-width="1.5" rx="3" stroke-dasharray="4,3"/>
<text x="315" y="268" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">encrypted negotiation payload</text>
<text x="315" y="286" fill="#8ad8d8" text-anchor="middle" font-size="10">26 bytes typical (10 payload + 16 AEAD tag)</text>
<!-- Row 5: explanation -->
<rect x="55" y="310" width="520" height="38" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="334" fill="#888" text-anchor="middle" font-size="10">negotiation appended via encrypt_payload() &#8212; extends Noise hash chain</text>
<!-- Total -->
<text x="310" y="378" fill="#777" font-size="10" text-anchor="middle">base: 85 bytes &#183; with FMP negotiation: 111 bytes</text>
<text x="310" y="395" fill="#777" font-size="10" text-anchor="middle">pattern: &#8594; s, se</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.8 KiB

docs/reference/diagrams/receiver-report.svg
+58 -67
View File
@@ -3,104 +3,95 @@
<rect width="580" height="634" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">ReceiverReport (0x02) — 68 bytes</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">ReceiverReport (0x02) &#8212; 54 bytes</text>
<!-- Row 0 (0–3): msg_type + reserved -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<!-- Row 0 (0-3): msg_type + format_version + total_length -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="130" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="120" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">0x02</text>
<text x="120" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">msg_type</text>
<text x="120" y="74" fill="#8ab4f8" text-anchor="middle" font-size="10">0x02</text>
<rect x="185" y="36" width="390" height="52" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="380" y="66" fill="#666" text-anchor="middle" font-size="12">reserved (3 bytes zero)</text>
<rect x="185" y="36" width="130" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="250" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">fmt_ver</text>
<text x="250" y="74" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte</text>
<!-- Row 1 (4–11): highest_counter -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–11</text>
<rect x="315" y="36" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">total_length</text>
<text x="445" y="74" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE (= 50)</text>
<!-- Row 1 (4-7): timestamp_echo -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">highest_counter</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">timestamp_echo</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">4 bytes LE (echoed sender timestamp for RTT)</text>
<!-- Row 2 (12–19): cumulative_packets_recv -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12–19</text>
<!-- Row 2 (8-9): dwell_time -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8&#8211;9</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_recv</text>
<text x="315" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">dwell_time</text>
<text x="315" y="180" fill="#8af8f8" text-anchor="middle" font-size="10">2 bytes LE (ms between receive and echo)</text>
<!-- Row 3 (20–27): cumulative_bytes_recv -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">20–27</text>
<!-- Row 3 (10-17): highest_counter -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">10&#8211;17</text>
<rect x="55" y="192" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_bytes_recv</text>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">highest_counter</text>
<text x="315" y="232" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<!-- Row 4 (28–33): timestamp_echo + dwell_time -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">28–33</text>
<!-- Row 4 (18-25): cumulative_packets_recv -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">18&#8211;25</text>
<rect x="55" y="244" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="185" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">timestamp_echo</text>
<text x="185" y="284" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="55" y="244" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_recv</text>
<text x="315" y="284" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="315" y="244" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="445" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">dwell_time</text>
<text x="445" y="284" fill="#8af8f8" text-anchor="middle" font-size="10">2 bytes LE ms</text>
<!-- Row 5 (26-33): cumulative_bytes_recv -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">26&#8211;33</text>
<!-- Row 5 (34–39): max_burst_loss + mean_burst_loss + reserved -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">34–39</text>
<rect x="55" y="296" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="318" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_bytes_recv</text>
<text x="315" y="336" fill="#8af8f8" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="55" y="296" width="173" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="141" y="318" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">max_burst</text>
<text x="141" y="336" fill="#f88aaa" text-anchor="middle" font-size="10">2 bytes LE</text>
<!-- Row 6 (34-41): jitter + ecn_ce_count -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">34&#8211;41</text>
<rect x="228" y="296" width="174" height="52" fill="#5a2d3d" stroke="#d94a6a" stroke-width="1.5" rx="3"/>
<text x="315" y="318" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">mean_burst</text>
<text x="315" y="336" fill="#f88aaa" text-anchor="middle" font-size="10">2 bytes u8.8</text>
<rect x="402" y="296" width="173" height="52" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="488" y="322" fill="#666" text-anchor="middle" font-size="10">reserved</text>
<!-- Row 6 (40–47): jitter + ecn_ce_count -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">40–47</text>
<rect x="55" y="348" width="260" height="52" fill="#4a3d2d" stroke="#d9b04a" stroke-width="1.5" rx="3"/>
<rect x="55" y="348" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="185" y="370" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">jitter</text>
<text x="185" y="388" fill="#f8d88a" text-anchor="middle" font-size="10">4 bytes LE µs</text>
<text x="185" y="388" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE (&#181;s)</text>
<rect x="315" y="348" width="260" height="52" fill="#4a3d2d" stroke="#d9b04a" stroke-width="1.5" rx="3"/>
<rect x="315" y="348" width="260" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="445" y="370" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">ecn_ce_count</text>
<text x="445" y="388" fill="#f8d88a" text-anchor="middle" font-size="10">4 bytes LE</text>
<text x="445" y="388" fill="#8af8c8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 7 (48–55): owd_trend + burst_loss_count -->
<text x="50" y="426" fill="#666" font-size="10" text-anchor="end">48–55</text>
<!-- Row 7 (42-49): owd_trend + burst_loss_count -->
<text x="50" y="426" fill="#666" font-size="10" text-anchor="end">42&#8211;49</text>
<rect x="55" y="400" width="260" height="52" fill="#3d4a2d" stroke="#8ad94a" stroke-width="1.5" rx="3"/>
<rect x="55" y="400" width="260" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="185" y="422" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">owd_trend</text>
<text x="185" y="440" fill="#c8f88a" text-anchor="middle" font-size="10">4 bytes i32 LE µs/s</text>
<text x="185" y="440" fill="#f8c88a" text-anchor="middle" font-size="10">i32 LE (&#181;s/s)</text>
<rect x="315" y="400" width="260" height="52" fill="#3d4a2d" stroke="#8ad94a" stroke-width="1.5" rx="3"/>
<rect x="315" y="400" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="445" y="422" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">burst_loss_count</text>
<text x="445" y="440" fill="#c8f88a" text-anchor="middle" font-size="10">4 bytes LE</text>
<text x="445" y="440" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 8 (56–63): cumulative_reorder_count + interval_packets_recv -->
<text x="50" y="478" fill="#666" font-size="10" text-anchor="end">56–63</text>
<!-- Row 8 (50-53): cumulative_reorder_count -->
<text x="50" y="478" fill="#666" font-size="10" text-anchor="end">50&#8211;53</text>
<rect x="55" y="452" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="474" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">reorder_count</text>
<text x="185" y="492" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE cumulative</text>
<rect x="55" y="452" width="520" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="315" y="474" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_reorder_count</text>
<text x="315" y="492" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="315" y="452" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="445" y="474" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">interval_pkts_recv</text>
<text x="445" y="492" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Forward compat note -->
<rect x="55" y="518" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="538" fill="#888" text-anchor="middle" font-size="10">decoders skip trailing bytes beyond total_length (forward compatibility)</text>
<!-- Row 9 (64–67): interval_bytes_recv -->
<text x="50" y="530" fill="#666" font-size="10" text-anchor="end">64–67</text>
<rect x="55" y="504" width="260" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="185" y="526" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">interval_bytes_recv</text>
<text x="185" y="544" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Removed fields note -->
<rect x="55" y="558" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="578" fill="#666" text-anchor="middle" font-size="10">removed from v0.2: max_burst_loss, mean_burst_loss, interval_packets/bytes_recv</text>
<!-- Total -->
<text x="310" y="586" fill="#777" font-size="10" text-anchor="middle">total: 68 bytes</text>
<text x="310" y="618" fill="#777" font-size="10" text-anchor="middle">total: 54 bytes &#183; format_version = 0 &#183; total_length = 50</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 6.8 KiB

After

Width:  |  Height:  |  Size: 6.3 KiB

docs/reference/diagrams/sender-report.svg
+32 -48
View File
@@ -1,66 +1,50 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 478" font-family="monospace" font-size="13">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 580 322" font-family="monospace" font-size="13">
<!-- Background -->
<rect width="580" height="478" fill="#1a1a2e" rx="4"/>
<rect width="580" height="322" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SenderReport (0x01) — 48 bytes</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SenderReport (0x01) &#8212; 20 bytes</text>
<!-- Row 0 (0–3): msg_type + reserved -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
<!-- Row 0 (0-3): msg_type + format_version + total_length -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0&#8211;3</text>
<rect x="55" y="36" width="130" height="52" fill="#2d4a7a" stroke="#4a90d9" stroke-width="1.5" rx="3"/>
<text x="120" y="60" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">msg_type</text>
<text x="120" y="78" fill="#8ab4f8" text-anchor="middle" font-size="10">0x01</text>
<text x="120" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">msg_type</text>
<text x="120" y="74" fill="#8ab4f8" text-anchor="middle" font-size="10">0x01</text>
<rect x="185" y="36" width="390" height="52" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="380" y="66" fill="#666" text-anchor="middle" font-size="12">reserved (3 bytes zero)</text>
<rect x="185" y="36" width="130" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="250" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">fmt_ver</text>
<text x="250" y="74" fill="#8af8c8" text-anchor="middle" font-size="10">1 byte</text>
<!-- Row 1 (4–11): interval_start_counter -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4–11</text>
<rect x="315" y="36" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="56" fill="#e0e0e0" text-anchor="middle" font-size="11" font-weight="bold">total_length</text>
<text x="445" y="74" fill="#d8a0f8" text-anchor="middle" font-size="10">2 bytes LE (= 16)</text>
<!-- Row 1 (4-7): interval_packets_sent -->
<text x="50" y="114" fill="#666" font-size="10" text-anchor="end">4&#8211;7</text>
<rect x="55" y="88" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_start_counter</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<text x="315" y="110" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_packets_sent</text>
<text x="315" y="128" fill="#f8c88a" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 2 (12–19): interval_end_counter -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">12–19</text>
<!-- Row 2 (8-11): interval_bytes_sent -->
<text x="50" y="166" fill="#666" font-size="10" text-anchor="end">8&#8211;11</text>
<rect x="55" y="140" width="520" height="52" fill="#5a3d2d" stroke="#d9904a" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_end_counter</text>
<text x="315" y="180" fill="#f8c88a" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="55" y="140" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_bytes_sent</text>
<text x="315" y="180" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 3 (20–27): interval_start_timestamp + interval_end_timestamp -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">20–27</text>
<!-- Row 3 (12-19): cumulative_packets_sent -->
<text x="50" y="218" fill="#666" font-size="10" text-anchor="end">12&#8211;19</text>
<rect x="55" y="192" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="185" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_start_ts</text>
<text x="185" y="232" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<rect x="55" y="192" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_sent</text>
<text x="315" y="232" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<rect x="315" y="192" width="260" height="52" fill="#4a2d5a" stroke="#b04ad9" stroke-width="1.5" rx="3"/>
<text x="445" y="214" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_end_ts</text>
<text x="445" y="232" fill="#d8a0f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 4 (28–31): interval_bytes_sent -->
<text x="50" y="270" fill="#666" font-size="10" text-anchor="end">28–31</text>
<rect x="55" y="244" width="520" height="52" fill="#2d5a5a" stroke="#4ad9d9" stroke-width="1.5" rx="3"/>
<text x="315" y="266" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">interval_bytes_sent</text>
<text x="315" y="284" fill="#8af8f8" text-anchor="middle" font-size="10">4 bytes LE</text>
<!-- Row 5 (32–39): cumulative_packets_sent -->
<text x="50" y="322" fill="#666" font-size="10" text-anchor="end">32–39</text>
<rect x="55" y="296" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="318" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_packets_sent</text>
<text x="315" y="336" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<!-- Row 6 (40–47): cumulative_bytes_sent -->
<text x="50" y="374" fill="#666" font-size="10" text-anchor="end">40–47</text>
<rect x="55" y="348" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="370" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">cumulative_bytes_sent</text>
<text x="315" y="388" fill="#8af8c8" text-anchor="middle" font-size="10">8 bytes LE</text>
<!-- Forward compat note -->
<rect x="55" y="258" width="520" height="32" fill="#1f1f3a" stroke="#444" stroke-width="1" rx="3"/>
<text x="315" y="278" fill="#888" text-anchor="middle" font-size="10">decoders skip trailing bytes beyond total_length (forward compatibility)</text>
<!-- Total -->
<text x="310" y="422" fill="#777" font-size="10" text-anchor="middle">total: 48 bytes</text>
<text x="310" y="314" fill="#777" font-size="10" text-anchor="middle">total: 20 bytes &#183; format_version = 0 &#183; total_length = 16</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.1 KiB

After

Width:  |  Height:  |  Size: 3.2 KiB

docs/reference/diagrams/session-ack.svg
+3 -3
View File
@@ -3,7 +3,7 @@
<rect width="580" height="322" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionAck (phase 0x2) — Noise XK msg2</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionAck (phase 0x2) &#8212; Noise XX msg2</text>
<!-- Row 0 (0–3): FSP prefix -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
@@ -40,8 +40,8 @@
<rect x="185" y="232" width="390" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="380" y="254" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">handshake_payload</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XK msg2 (57 bytes — ephemeral + epoch)</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XX msg2 (106+ bytes &#8212; ephemeral + static + epoch)</text>
<!-- Total -->
<text x="310" y="308" fill="#777" font-size="10" text-anchor="middle">typical ~190 bytes (depth-dependent, carries both endpoints' coords)</text>
<text x="310" y="308" fill="#777" font-size="10" text-anchor="middle">typical ~240 bytes (depth-dependent, carries both endpoints' coords)</text>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.9 KiB

After

Width:  |  Height:  |  Size: 2.9 KiB

docs/reference/diagrams/session-msg3.svg
+2 -2
View File
@@ -3,7 +3,7 @@
<rect width="580" height="244" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionMsg3 (phase 0x3) — Noise XK msg3</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionMsg3 (phase 0x3) &#8212; Noise XX msg3</text>
<!-- Row 0 (0–3): FSP prefix -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
@@ -28,7 +28,7 @@
<rect x="55" y="140" width="520" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="315" y="162" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">handshake_payload</text>
<text x="315" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XK msg3 (73 bytes — encrypted static + encrypted epoch)</text>
<text x="315" y="180" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XX msg3 (73+ bytes &#8212; encrypted static + epoch + optional negotiation)</text>
<!-- Total -->
<text x="310" y="218" fill="#777" font-size="10" text-anchor="middle">~80 bytes · no coordinates (both endpoints already have them)</text>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.2 KiB

After

Width:  |  Height:  |  Size: 2.2 KiB

docs/reference/diagrams/session-setup.svg
+2 -2
View File
@@ -3,7 +3,7 @@
<rect width="580" height="322" fill="#1a1a2e" rx="4"/>
<!-- Title -->
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionSetup (phase 0x1) — Noise XK msg1</text>
<text x="310" y="26" fill="#e0e0e0" text-anchor="middle" font-size="14" font-weight="bold">SessionSetup (phase 0x1) &#8212; Noise XX msg1</text>
<!-- Row 0 (0–3): FSP prefix -->
<text x="50" y="62" fill="#666" font-size="10" text-anchor="end">0–3</text>
@@ -40,7 +40,7 @@
<rect x="185" y="232" width="390" height="52" fill="#2d5a4a" stroke="#4ad99a" stroke-width="1.5" rx="3"/>
<text x="380" y="254" fill="#e0e0e0" text-anchor="middle" font-size="12" font-weight="bold">handshake_payload</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XK msg1 (33 bytes — ephemeral key)</text>
<text x="380" y="272" fill="#8af8c8" text-anchor="middle" font-size="10">Noise XX msg1 (33 bytes &#8212; ephemeral key)</text>
<!-- Total -->
<text x="310" y="308" fill="#777" font-size="10" text-anchor="middle">typical ~170 bytes (depth-dependent)</text>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.9 KiB

After

Width:  |  Height:  |  Size: 2.9 KiB

docs/reference/security.md
+7 -7
View File
@@ -55,14 +55,14 @@ idempotent).
| Component | Choice | Where Used |
| --------- | ------ | ---------- |
| Curve | secp256k1 | FMP IK, FSP XK, Schnorr signatures |
| Diffie-Hellman | ECDH on secp256k1 (x-only normalized) | Noise IK, Noise XK |
| Curve | secp256k1 | FMP XX, FSP XX, Schnorr signatures |
| Diffie-Hellman | ECDH on secp256k1 (x-only normalized) | Noise XX (both layers) |
| AEAD | ChaCha20-Poly1305 | FMP link encryption, FSP session encryption |
| Hash | SHA-256 | NodeAddr derivation, Noise transcript |
| Key derivation | HKDF-SHA256 | Noise key schedule |
| Signatures | secp256k1 Schnorr | TreeAnnounce, LookupResponse proof, Nostr adverts |
| Noise pattern (link) | `Noise_IK_secp256k1_ChaChaPoly_SHA256` | FMP link layer (IK with epoch payload) |
| Noise pattern (session) | `Noise_XK_secp256k1_ChaChaPoly_SHA256` | FSP session layer (XK with epoch payload) |
| Noise pattern (link) | `Noise_XX_secp256k1_ChaChaPoly_SHA256` | FMP link layer (XX with negotiation payload) |
| Noise pattern (session) | `Noise_XX_secp256k1_ChaChaPoly_SHA256` | FSP session layer (XX with negotiation payload) |
These choices align with the Nostr cryptographic stack
(secp256k1 + ChaCha20-Poly1305 + SHA-256) and the NIP-44 encrypted
@@ -101,7 +101,7 @@ and FSP layers.
Mesh-level ACL files at `/etc/fips/peers.allow` and
`/etc/fips/peers.deny` give the operator allowlist/blocklist control
over which npubs may complete the FMP Noise IK link handshake.
over which npubs may complete the FMP Noise XX link handshake.
File format:
@@ -209,7 +209,7 @@ for the metadata-privacy model and the rejection of onion routing.
| --------- | --------------- | ------------ | ------ |
| UDP | None until `bind_addr` set | `0.0.0.0:2121` typical | Operator sets `transports.udp.bind_addr` |
| TCP | None until `bind_addr` set | None — outbound-only without bind | Operator sets `transports.tcp.bind_addr` |
| Ethernet | Listens on configured interface (raw `AF_PACKET`) | EtherType 0x2121 on selected interface | Per-flag `discovery`, `announce`, `auto_connect`, `accept_connections` |
| Ethernet | Listens on configured interface (raw `AF_PACKET`) | EtherType 0x2121 on selected interface | Per-flag `listen`, `announce`, `auto_connect`, `accept_connections` |
| Tor | None until `directory_service` configured | `127.0.0.1:8443` (loopback only) | Operator sets `transports.tor.directory_service` and configures `HiddenServiceDir` in `torrc` |
| BLE | Off by default | n/a | Operator enables `transports.ble.*` |
| Nostr discovery | Off by default | n/a (relay client, not a listener) | Operator sets `node.discovery.nostr.enabled: true` |
@@ -228,7 +228,7 @@ way to restrict inbound traffic on `fips0`. See
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) — FMP
link encryption, replay protection, rate limiting
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— FSP end-to-end encryption, Noise XK, replay window
— FSP end-to-end encryption, Noise XX, replay window
- [../how-to/enable-mesh-firewall.md](../how-to/enable-mesh-firewall.md)
— operator activation and drop-in recipes
- [configuration.md](configuration.md) — full `node.rekey.*`,
docs/reference/wire-formats.md
+220 -122
View File
@@ -24,7 +24,7 @@ The FMP link layer defines the following message types, dispatched by the
Handshake messages travel before encryption is established and are identified
by the FMP common-prefix `phase` field rather than a `msg_type` byte
(phase 0x1 = Noise IK msg1, phase 0x2 = Noise IK msg2).
(phase 0x1 = Noise XX msg1, phase 0x2 = Noise XX msg2, phase 0x3 = Noise XX msg3).
## Packet Type Summary
@@ -33,8 +33,8 @@ A higher-level summary that includes typical sizes and forwarding category:
| Message | Typical Size | When | Forwarded? |
| ------- | ------------ | ---- | ---------- |
| TreeAnnounce | Variable (depth-dependent) | Topology changes | No (peer-to-peer) |
| FilterAnnounce | ~1 KB | Topology changes | No (peer-to-peer) |
| LookupRequest | ~300 bytes | First contact, recovery | Yes (bloom-guided tree) |
| FilterAnnounce | variable (RLE compressed) | Topology changes | No (peer-to-peer) |
| LookupRequest | 44 bytes + TLV | First contact, recovery | Yes (bloom-guided tree) |
| LookupResponse | ~400 bytes | Response to discovery | Yes (reverse-path) |
| SessionDatagram + SessionSetup | ~232–402 bytes | Session establishment | Yes (routed) |
| SessionDatagram + SessionAck | ~170 bytes | Session confirmation | Yes (routed) |
@@ -63,14 +63,14 @@ stream; the common prefix `payload_len` field provides this framing
directly. TCP and Tor share a common stream reader (`tcp/stream.rs`)
that implements this framing.
**Ethernet data frame header.** The Ethernet transport prepends a 3-byte
header before the FMP payload on data frames: a 1-byte frame type
(`0x00`) followed by a 2-byte little-endian payload length. The length
field allows the receiver to trim Ethernet minimum-frame padding that
would otherwise corrupt AEAD verification. Beacon frames (`0x01`) have
no length field (fixed 34-byte payload). These bytes are consumed by the
transport layer and are not visible to FMP. The effective MTU for FMP is
the interface MTU minus three bytes (typically 1497).
**Ethernet frame header.** The Ethernet transport prepends a 4-byte
unified header before the payload: `[type:1][flags:1][length:2 LE]`.
The length field allows the receiver to trim Ethernet minimum-frame
padding that would otherwise corrupt AEAD verification. Frame types:
`0x00` (data), `0x01` (beacon). Beacons are 5 bytes total (4-byte
header + 1-byte beacon type). These bytes are consumed by the
transport layer and are not visible to FMP. The effective MTU for FMP
is the interface MTU minus four bytes (typically 1496).
## Link-Layer Formats
@@ -84,7 +84,7 @@ length.
| Field | Size | Description |
| ----- | ---- | ----------- |
| version | 4 bits (high) | Protocol version. Currently 0x0 |
| version | 4 bits (high) | Protocol version. Currently 0x1 |
| phase | 4 bits (low) | Session lifecycle phase (see table) |
| flags | 1 byte | Per-packet signal flags (zero during handshake) |
| payload_len | 2 bytes LE | Length of payload after phase-specific header, excluding AEAD tag |
@@ -94,8 +94,9 @@ length.
| Phase | Type | Description |
| ----- | ---- | ----------- |
| 0x0 | Established frame | Post-handshake encrypted traffic |
| 0x1 | Noise IK msg1 | Handshake initiation |
| 0x2 | Noise IK msg2 | Handshake response |
| 0x1 | Noise XX msg1 | Handshake initiation (ephemeral only) |
| 0x2 | Noise XX msg2 | Handshake response (responder identity + negotiation) |
| 0x3 | Noise XX msg3 | Handshake completion (initiator identity + negotiation) |
### Flags (Established Phase Only)
@@ -103,10 +104,9 @@ length.
| --- | ---- | ----------- |
| 0 | K (key epoch) | Selects active key during rekeying |
| 1 | CE | Congestion Experienced echo |
| 2 | SP (spin bit) | RTT measurement |
| 3-7 | — | Reserved (must be zero) |
| 2-7 | — | Reserved (must be zero) |
Flags must be zero in handshake packets (phase 0x1 and 0x2).
Flags must be zero in handshake packets (phase 0x1, 0x2, and 0x3).
### Established Frame (phase 0x0)
@@ -119,7 +119,7 @@ encrypted link-layer message.
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=0, flags, payload_len |
| common prefix | 4 bytes | ver=1, phase=0, flags, payload_len |
| receiver_idx | 4 bytes LE | Session index for O(1) lookup |
| counter | 8 bytes LE | Monotonic nonce, used as AEAD nonce and for replay detection |
@@ -147,67 +147,141 @@ the 1-byte message type and message-specific fields.
| Type | Message | Description |
| ---- | ------- | ----------- |
| 0x00 | SessionDatagram | Encapsulated session-layer payload for forwarding |
| 0x01 | SenderReport | MMP sender-side metrics report (48 bytes) |
| 0x02 | ReceiverReport | MMP receiver-side metrics report (68 bytes) |
| 0x01 | SenderReport | MMP sender-side metrics report (20 bytes) |
| 0x02 | ReceiverReport | MMP receiver-side metrics report (54 bytes) |
| 0x10 | TreeAnnounce | Spanning tree state announcement |
| 0x20 | FilterAnnounce | Bloom filter reachability update |
| 0x21 | FilterNack | Bloom filter delta NACK (retransmit request) |
| 0x30 | LookupRequest | Coordinate discovery request |
| 0x31 | LookupResponse | Coordinate discovery response |
| 0x50 | Disconnect | Orderly link teardown |
| 0x51 | Heartbeat | Link liveness probe |
### Noise IK Message 1 (phase 0x1)
### Noise XX Message 1 (phase 0x1)
Handshake initiation from connecting party.
Handshake initiation from connecting party. The initiator sends only its
ephemeral key — neither side's static identity is revealed in msg1.
![Noise IK message 1](diagrams/noise-ik-msg1.svg)
![Noise XX message 1](diagrams/noise-xx-msg1.svg)
Common prefix: ver=0, phase=0x1, flags=0, payload_len=110 (4 + 106).
Common prefix: ver=1, phase=0x1, flags=0, payload_len=37 (4 + 33).
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=1, flags=0, payload_len |
| common prefix | 4 bytes | ver=1, phase=1, flags=0, payload_len |
| sender_idx | 4 bytes LE | Initiator's session index (becomes receiver's `receiver_idx`) |
| noise_msg1 | 106 bytes | Noise IK first message |
| noise_msg1 | 33 bytes | Noise XX first message |
**Noise msg1 breakdown** (106 bytes):
**Noise msg1 breakdown** (33 bytes):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Initiator's ephemeral key (compressed secp256k1) |
| 33 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `-> e, es, s, ss` with epoch payload
Noise pattern: `-> e`
### Noise IK Message 2 (phase 0x2)
**Total wire size**: 41 bytes (4 prefix + 4 sender_idx + 33 noise).
Handshake response from responder.
### Noise XX Message 2 (phase 0x2)
![Noise IK message 2](diagrams/noise-ik-msg2.svg)
Handshake response from responder. The responder reveals its identity
(ephemeral key, encrypted static key, and encrypted epoch) plus an
optional protocol negotiation payload.
Common prefix: ver=0, phase=0x2, flags=0, payload_len=65 (4 + 4 + 57).
![Noise XX message 2](diagrams/noise-xx-msg2.svg)
Common prefix: ver=1, phase=0x2, flags=0, payload_len varies.
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=0, phase=2, flags=0, payload_len |
| common prefix | 4 bytes | ver=1, phase=2, flags=0, payload_len |
| sender_idx | 4 bytes LE | Responder's session index |
| receiver_idx | 4 bytes LE | Echo of initiator's sender_idx from msg1 |
| noise_msg2 | 57 bytes | Noise IK second message |
| noise_msg2 | 106+ bytes | Noise XX second message (variable with negotiation) |
**Noise msg2 breakdown** (57 bytes):
**Noise msg2 breakdown** (106 bytes base):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Responder's ephemeral key (compressed secp256k1) |
| 33 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
| 33 | encrypted_static | 49 bytes | Responder's static key (33) + AEAD tag (16) |
| 82 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `<- e, ee, se` with epoch payload
Noise pattern: `<- e, ee, s, es` with epoch and negotiation payload
After msg2, both parties derive identical symmetric session keys. The
encrypted epoch in msg1 and msg2 enables peer restart detection — if a
peer's epoch changes, the other side knows it restarted and must
re-establish the link.
**Negotiation payload** (variable, appended via `encrypt_payload()`):
encrypted negotiation bytes + AEAD tag. Minimum 26 bytes (10 payload +
16 tag) when present. See [Protocol Negotiation Payload](#protocol-negotiation-payload).
**Total wire size**: 118 bytes minimum (without negotiation), 144 bytes
typical (with FMP negotiation: 118 + 26).
After msg2, the responder's identity is known to the initiator.
### Noise XX Message 3 (phase 0x3)
Handshake completion from initiator. The initiator reveals its encrypted
static identity and epoch, plus an optional negotiation payload.
![Noise XX message 3](diagrams/noise-xx-msg3.svg)
| Field | Size | Description |
| ----- | ---- | ----------- |
| common prefix | 4 bytes | ver=1, phase=3, flags=0, payload_len |
| sender_idx | 4 bytes LE | Echo of initiator's sender_idx |
| receiver_idx | 4 bytes LE | Echo of responder's sender_idx from msg2 |
| noise_msg3 | 73+ bytes | Noise XX third message (variable with negotiation) |
**Noise msg3 breakdown** (73 bytes base):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | encrypted_static | 49 bytes | Initiator's static key (33) + AEAD tag (16) |
| 49 | encrypted_epoch | 24 bytes | Startup epoch (8) + AEAD tag (16) |
Noise pattern: `-> s, se` with epoch and negotiation payload
**Negotiation payload** (variable, appended via `encrypt_payload()`):
same format as msg2. Minimum 26 bytes when present.
**Total wire size**: 85 bytes minimum (without negotiation), 111 bytes
typical (with FMP negotiation: 85 + 26).
After msg3, both parties derive identical symmetric session keys and
have exchanged negotiation payloads. The encrypted epoch in msg2 and
msg3 enables peer restart detection — if a peer's epoch changes, the
other side knows it restarted and must re-establish the link.
### Protocol Negotiation Payload
Appended to XX msg2 and msg3 via Noise `encrypt_payload()` — extends
the Noise hash chain so negotiation data is authenticated alongside the
handshake transcript.
**Wire format**: `[format:1][versions:1][features:8][TLV entries...]`
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | format | 1 byte | Must be 0 |
| 1 | versions | 1 byte | High nibble = version_min, low nibble = version_max |
| 2 | features | 8 bytes LE | 64-bit feature bitfield |
| 10 | tlv_entries | variable | TLV extensions: `[field_num:2 LE][length:2 LE][value:N]` per entry |
**Minimum size**: 10 bytes (no TLV entries). With AEAD tag: 26 bytes on
the wire.
**FMP feature bits**:
| Bits | Name | Description |
| ---- | ---- | ----------- |
| 0-2 | Node profile | Full (0), NonRouting (1), Leaf (2) |
| 3-6 | MMP wants/provides | MMP capability negotiation |
| 7 | Bloom | Bloom filter capability |
| 8-63 | — | Reserved |
**Critical**: `decrypt_payload()` MUST be called even if the result is
discarded, to maintain hash chain consistency between the two Noise
transport channels.
### Index Semantics
@@ -269,28 +343,49 @@ includes self)
### FilterAnnounce (0x20)
Bloom filter reachability update, exchanged between direct peers only.
Supports both full sends and delta (XOR diff) updates with RLE
compression.
![FilterAnnounce](diagrams/filter-announce.svg)
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | msg_type | 1 byte | 0x20 |
| 1 | sequence | 8 bytes LE | Monotonic counter for freshness |
| 9 | hash_count | 1 byte | Number of hash functions (5 in v1) |
| 10 | size_class | 1 byte | Filter size: `512 << size_class` bytes |
| 11 | filter_bits | variable | Bloom filter bit array |
| 1 | flags | 1 byte | Bit 0: delta (XOR diff), bits 1-7 reserved |
| 2 | sequence | 8 bytes LE | Monotonic counter, per-peer (only increments on actual send) |
| 10 | base_seq | 8 bytes LE | Sequence of reference filter for delta (0 if full send) |
| 18 | size_class | 1 byte | Filter size: `512 << size_class` bytes |
| 19 | compressed_payload | variable | RLE-encoded filter or XOR diff |
**RLE format**: each run = `[count:2 LE][word:8 LE]` (10 bytes per run).
Sparse XOR diffs compress to very few runs.
**Size class table**:
| size_class | Bytes | Bits | Status |
| ---------- | ----- | ---- | ------ |
| 0 | 512 | 4,096 | Reserved |
| 1 | 1,024 | 8,192 | **v1 (MUST use)** |
| 2 | 2,048 | 16,384 | Reserved |
| 3 | 4,096 | 32,768 | Reserved |
| size_class | Bytes | Bits | Notes |
| ---------- | ----- | ---- | ----- |
| 0 | 512 | 4,096 | Minimum |
| 1 | 1,024 | 8,192 | Default |
| 2 | 2,048 | 16,384 | |
| 3 | 4,096 | 32,768 | |
| 4 | 8,192 | 65,536 | |
| 5 | 16,384 | 131,072 | |
| 6 | 32,768 | 262,144 | Maximum |
**v1 payload**: 1,035 bytes (11 header + 1,024 filter).
With link overhead: 1,072 bytes.
**Size**: variable (19-byte header + RLE-compressed payload).
### FilterNack (0x21)
Request full filter retransmission. Sent when a node receives an
out-of-sequence delta update (sequence gap detected), triggering a
full retransmit from the sender.
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | msg_type | 1 byte | 0x21 |
| 1 | expected_seq | 8 bytes LE | Sequence number the receiver expected |
**Total**: 9 bytes.
### LookupRequest (0x30)
@@ -310,16 +405,10 @@ restructuring.
| 25 | origin | 16 bytes | Requester's NodeAddr |
| 41 | ttl | 1 byte | Remaining hops (default 64) |
| 42 | min_mtu | 2 bytes LE | Minimum transport MTU the origin requires (0 = no requirement) |
| 44 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 46 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
| 44 | tlv_entries | variable | TLV extensions, forwarded verbatim by transit |
**Size**: `46 + (n x 16)` bytes, where n = origin depth + 1
| Origin Depth | Payload |
| ------------ | ------- |
| 3 | 110 bytes |
| 5 | 142 bytes |
| 10 | 222 bytes |
**Size**: 44 bytes fixed + TLV entries. TLV format:
`[field_num:2 LE][length:2 LE][value:N]` per entry.
### LookupResponse (0x31)
@@ -337,11 +426,12 @@ requester via the transit nodes that forwarded the request.
| 27 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 29 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
| 29 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target \|\| target_coords)` |
| 93 + 16n | tlv_entries | variable | TLV extensions, forwarded verbatim by transit |
**Size**: `93 + (n x 16)` bytes
**Size**: `93 + (n x 16)` bytes + TLV entries
| Target Depth | Payload |
| ------------ | ------- |
| Target Depth | Payload (no TLV) |
| ------------ | ---------------- |
| 3 | 141 bytes |
| 5 | 173 bytes |
| 10 | 253 bytes |
@@ -414,16 +504,16 @@ Sent by the frame sender to provide interval-based transmission statistics.
| Offset | Field | Size | Encoding |
| ------ | ----- | ---- | -------- |
| 0 | msg_type | 1 | `0x01` |
| 1 | reserved | 3 | Zero |
| 4 | interval_start_counter | 8 | u64 LE — first counter in this interval |
| 12 | interval_end_counter | 8 | u64 LE — last counter in this interval |
| 20 | interval_start_timestamp | 4 | u32 LE — timestamp at interval start |
| 24 | interval_end_timestamp | 4 | u32 LE — timestamp at interval end |
| 28 | interval_bytes_sent | 4 | u32 LE — payload bytes sent in interval |
| 32 | cumulative_packets_sent | 8 | u64 LE — total packets sent on this link |
| 40 | cumulative_bytes_sent | 8 | u64 LE — total bytes sent on this link |
| 1 | format_version | 1 | 0 (current) |
| 2 | total_length | 2 | u16 LE — 16 (v0 payload size) |
| 4 | interval_packets_sent | 4 | u32 LE — packets sent in interval |
| 8 | interval_bytes_sent | 4 | u32 LE — payload bytes sent in interval |
| 12 | cumulative_packets_sent | 8 | u64 LE — total packets sent on this link |
**Total: 48 bytes.**
**Total: 20 bytes.**
Decoders skip trailing bytes beyond `total_length` for forward
compatibility with future format versions.
### ReceiverReport (0x02)
@@ -434,24 +524,26 @@ Sent by the frame receiver to provide loss, jitter, and timing feedback.
| Offset | Field | Size | Encoding |
| ------ | ----- | ---- | -------- |
| 0 | msg_type | 1 | `0x02` |
| 1 | reserved | 3 | Zero |
| 4 | highest_counter | 8 | u64 LE — highest counter value received |
| 12 | cumulative_packets_recv | 8 | u64 LE — total packets received |
| 20 | cumulative_bytes_recv | 8 | u64 LE — total bytes received |
| 28 | timestamp_echo | 4 | u32 LE — echoed sender timestamp for RTT |
| 32 | dwell_time | 2 | u16 LE — time between receive and echo (ms) |
| 34 | max_burst_loss | 2 | u16 LE — largest loss burst in interval |
| 36 | mean_burst_loss | 2 | u16 LE — mean burst length (u8.8 fixed-point) |
| 38 | reserved | 2 | Zero |
| 40 | jitter | 4 | u32 LE — interarrival jitter (microseconds) |
| 44 | ecn_ce_count | 4 | u32 LE — cumulative ECN-CE marked packets |
| 48 | owd_trend | 4 | i32 LE — one-way delay trend (µs/s, signed) |
| 52 | burst_loss_count | 4 | u32 LE — number of loss bursts in interval |
| 56 | cumulative_reorder_count | 4 | u32 LE — total reordered packets |
| 60 | interval_packets_recv | 4 | u32 LE — packets received in interval |
| 64 | interval_bytes_recv | 4 | u32 LE — bytes received in interval |
| 1 | format_version | 1 | 0 (current) |
| 2 | total_length | 2 | u16 LE — 50 (v0 payload size) |
| 4 | timestamp_echo | 4 | u32 LE — echoed sender timestamp for RTT |
| 8 | dwell_time | 2 | u16 LE — time between receive and echo (ms) |
| 10 | highest_counter | 8 | u64 LE — highest counter value received |
| 18 | cumulative_packets_recv | 8 | u64 LE — total packets received |
| 26 | cumulative_bytes_recv | 8 | u64 LE — total bytes received |
| 34 | jitter | 4 | u32 LE — interarrival jitter (microseconds) |
| 38 | ecn_ce_count | 4 | u32 LE — cumulative ECN-CE marked packets |
| 42 | owd_trend | 4 | i32 LE — one-way delay trend (µs/s, signed) |
| 46 | burst_loss_count | 4 | u32 LE — number of loss bursts in interval |
| 50 | cumulative_reorder_count | 4 | u32 LE — total reordered packets |
**Total: 68 bytes.**
**Total: 54 bytes.**
Fields removed from v0.2: `max_burst_loss`, `mean_burst_loss`,
`interval_packets_recv`, `interval_bytes_recv`.
Decoders skip trailing bytes beyond `total_length` for forward
compatibility with future format versions.
## Session-Layer Message Formats
@@ -473,9 +565,9 @@ protocol version, session lifecycle phase, per-packet flags, and payload length.
| Phase | Type | Description |
| ----- | ---- | ----------- |
| 0x0 | Established | Post-handshake encrypted traffic or plaintext error signals |
| 0x1 | Handshake msg1 | SessionSetup (Noise XK msg1) |
| 0x2 | Handshake msg2 | SessionAck (Noise XK msg2) |
| 0x3 | Handshake msg3 | SessionMsg3 (Noise XK msg3) |
| 0x1 | Handshake msg1 | SessionSetup (Noise XX msg1) |
| 0x2 | Handshake msg2 | SessionAck (Noise XX msg2) |
| 0x3 | Handshake msg3 | SessionMsg3 (Noise XX msg3) |
### FSP Flags (Established Phase Only)
@@ -518,7 +610,7 @@ Transit nodes parse the CP flag and extract coordinates without decryption.
| ----- | ---- | ----------- |
| timestamp | 4 bytes LE | Session-relative milliseconds (u32) |
| msg_type | 1 byte | Session-layer message type |
| inner_flags | 1 byte | Bit 0: SP (spin bit for RTT measurement) |
| inner_flags | 1 byte | Reserved (must be zero) |
After the inner header, the remaining plaintext is the message-type-specific
body.
@@ -563,8 +655,8 @@ encrypted inner header.
### SessionSetup (phase 0x1)
Establishes a session and warms transit coordinate caches. Contains the
first message of the Noise XK handshake (ephemeral key only — the
initiator's static identity is not revealed until msg3).
first message of the Noise XX handshake (ephemeral key only — neither
side's static identity is revealed until msg2/msg3).
SessionSetup, SessionAck, and SessionMsg3 are identified by the **phase**
field in the FSP common prefix (0x1, 0x2, 0x3), not by a message type
@@ -585,12 +677,13 @@ Encoded with FSP prefix: ver=0, phase=0x1, flags=0, payload_len.
| ... | dest_coords_count | 2 bytes LE | Number of dest coordinate entries |
| ... | dest_coords | 16 x m bytes | Destination's ancestry |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise XK msg1 (33 bytes — ephemeral key only) |
| ... | handshake_payload | variable | Noise XX msg1 (33 bytes — ephemeral key only) |
### SessionAck (phase 0x2)
Second message of the Noise XK handshake. The responder sends its
ephemeral key and encrypted epoch.
Second message of the Noise XX handshake. The responder reveals its
identity (ephemeral key, encrypted static key, and encrypted epoch).
Optional FSP negotiation payload may be appended (omitted for rekey).
Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
![SessionAck](diagrams/session-ack.svg)
@@ -605,12 +698,13 @@ Encoded with FSP prefix: ver=0, phase=0x2, flags=0, payload_len.
| ... | dest_coords_count | 2 bytes LE | Number of initiator coordinate entries |
| ... | dest_coords | 16 x m bytes | Initiator's ancestry (for return-path cache warming) |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise XK msg2 (57 bytes — ephemeral key + encrypted epoch) |
| ... | handshake_payload | variable | Noise XX msg2 (106+ bytes — ephemeral + encrypted static + encrypted epoch + optional negotiation) |
### SessionMsg3 (phase 0x3)
Third and final message of the Noise XK handshake. The initiator reveals
its encrypted static identity and epoch. After msg3, both parties derive
Third and final message of the Noise XX handshake. The initiator reveals
its encrypted static identity and epoch. Optional FSP negotiation payload
may be appended (omitted for rekey). After msg3, both parties derive
identical symmetric session keys and the session is established.
Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len.
@@ -622,9 +716,9 @@ Encoded with FSP prefix: ver=0, phase=0x3, flags=0, payload_len.
| ------ | ----- | ---- | ----------- |
| 0 | flags | 1 byte | Reserved |
| 1 | handshake_len | 2 bytes LE | Noise payload length |
| 3 | handshake_payload | variable | Noise XK msg3 (73 bytes — encrypted static + encrypted epoch) |
| 3 | handshake_payload | variable | Noise XX msg3 (73+ bytes — encrypted static + encrypted epoch + optional negotiation) |
**Noise XK msg3 breakdown** (73 bytes):
**Noise XX msg3 breakdown** (73 bytes base):
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
@@ -873,29 +967,33 @@ endpoint session keys).
## Size Summary
### FMP Handshake Messages (Noise IK)
### FMP Handshake Messages (Noise XX)
| Message | Raw Noise | Wire Frame |
| ------- | --------- | ---------- |
| IK msg1 (ephemeral + encrypted static + encrypted epoch) | 106 bytes | 114 bytes |
| IK msg2 (ephemeral + encrypted epoch) | 57 bytes | 69 bytes |
| XX msg1 (ephemeral only) | 33 bytes | 41 bytes |
| XX msg2 (ephemeral + encrypted static + epoch) | 106 bytes | 144 bytes (with negotiation) |
| XX msg3 (encrypted static + epoch) | 73 bytes | 111 bytes (with negotiation) |
### FSP Handshake Messages (Noise XK)
### FSP Handshake Messages (Noise XX)
| Message | Raw Noise | Notes |
| ------- | --------- | ----- |
| XK msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup |
| XK msg2 (ephemeral + encrypted epoch) | 57 bytes | Carried in SessionAck |
| XK msg3 (encrypted static + encrypted epoch) | 73 bytes | Carried in SessionMsg3 |
| XX msg1 (ephemeral only) | 33 bytes | Carried in SessionSetup |
| XX msg2 (ephemeral + encrypted static + epoch) | 106 bytes | Carried in SessionAck |
| XX msg3 (encrypted static + epoch) | 73 bytes | Carried in SessionMsg3 |
### Link-Layer Messages (inside encrypted frame)
| Message | Size | Notes |
| ------- | ---- | ----- |
| TreeAnnounce | 100 + 32n bytes | n = depth + 1 |
| FilterAnnounce | 1,035 bytes | v1 (1KB filter) |
| LookupRequest | 46 + 16n bytes | n = origin depth + 1 |
| LookupResponse | 93 + 16n bytes | n = target depth + 1 |
| FilterAnnounce | variable | 19-byte header + RLE-compressed payload |
| FilterNack | 9 bytes | |
| LookupRequest | 44 bytes + TLV | Fixed (no longer depth-dependent) |
| LookupResponse | 93 + 16n bytes + TLV | n = target depth + 1 |
| SenderReport | 20 bytes | Extensibility header |
| ReceiverReport | 54 bytes | Extensibility header |
| SessionDatagram | 36 + payload bytes | Fixed 36-byte header |
| Disconnect | 2 bytes | |
@@ -903,13 +1001,13 @@ endpoint session keys).
| Message | Typical Size | Notes |
| ------- | ------------ | ----- |
| SessionSetup | ~170 bytes | Depth-dependent (XK msg1 = 33 bytes) |
| SessionAck | ~190 bytes | Depth-dependent, carries both endpoints' coords (XK msg2 = 57 bytes) |
| SessionMsg3 | ~80 bytes | Fixed (XK msg3 = 73 bytes, no coords) |
| SessionSetup | ~170 bytes | Depth-dependent (XX msg1 = 33 bytes) |
| SessionAck | ~240 bytes | Depth-dependent, carries both endpoints' coords (XX msg2 = 106+ bytes) |
| SessionMsg3 | ~80 bytes | Fixed (XX msg3 = 73+ bytes, no coords) |
| Data (minimal) | 12 + 6 + 4 + payload + 16 bytes | Steady state (port header included) |
| Data (with coords) | 12 + ~130 + 6 + 4 + payload + 16 bytes | Warmup/recovery (port header included) |
| SenderReport | 12 + 6 + 46 + 16 bytes | MMP metrics |
| ReceiverReport | 12 + 6 + 66 + 16 bytes | MMP metrics |
| SenderReport | 12 + 6 + 20 + 16 bytes | MMP metrics |
| ReceiverReport | 12 + 6 + 54 + 16 bytes | MMP metrics |
| PathMtuNotification | 12 + 6 + 2 + 16 bytes | MTU signal |
| CoordsWarmup | 12 + coords + 6 + 16 bytes | Standalone warmup (empty body) |
| CoordsRequired | 38 bytes | Fixed (prefix + msg_type + body) |
docs/tutorials/advertise-your-node.md
+1 -1
View File
@@ -282,7 +282,7 @@ an entry for `test-us03` (the open-discovery test mesh node).
It will have `connectivity` active and its own
`transport_addr`. This peering appeared without you
configuring anything — the test-mesh open-discovery node saw
your advert, dialed the endpoint, and Noise IK established
your advert, dialed the endpoint, and Noise XX established
the link.
If no inbound peers appear, that's not necessarily a failure
docs/tutorials/ground-up-mesh.md
+10 -10
View File
@@ -47,7 +47,7 @@ Two machines, each running `fips`, joined by a physical Ethernet
link. After the worked example:
- The two daemons have discovered each other via L2 beacons on
the link, peered over Noise IK, and brought up an FMP link.
the link, peered over Noise XX, and brought up an FMP link.
- Each `fips0` adapter has a routable mesh address; each can
ping the other by `<npub>.fips`.
- Nothing between the two machines speaks IP. The link carries
@@ -71,7 +71,7 @@ supplies the rest:
- **Addressing**: the `fips0` adapter takes an `fd97:...` ULA
derived from the npub. No DHCP. No SLAAC. The address is
cryptographically tied to the identity.
- **Discovery**: each daemon broadcasts a small beacon on the
- **Neighbor detection**: each daemon broadcasts a small beacon on the
link advertising its npub; the other daemon's listener picks
it up and dials in over the same link.
- **Routing**: the FIPS mesh layer builds its own spanning tree
@@ -177,7 +177,7 @@ different interface names — that is normal.
Edit `/etc/fips/fips.yaml` on **both** nodes. Under
`transports:`, add an `ethernet:` block. The key settings are
the four discovery flags — both nodes must opt in to all four,
the four neighbor flags — both nodes must opt in to all four,
and they default to off:
```yaml
@@ -185,7 +185,7 @@ transports:
ethernet:
interface: "<eth>" # the name from Step 1
announce: true # broadcast our beacon on the link
discovery: true # listen for beacons (default; shown for clarity)
listen: true # listen for beacons (default; shown for clarity)
auto_connect: true # dial peers we discover
accept_connections: true # accept dial-ins from peers we discover
```
@@ -194,7 +194,7 @@ Each flag does one thing:
- `announce: true` — emit a small beacon every
`beacon_interval_secs` (default 30s) carrying our npub.
- `discovery: true` — listen for incoming beacons; populate a
- `listen: true` — listen for incoming beacons; populate a
candidate-peer list keyed by source MAC and observed npub.
- `auto_connect: true` — when we see a beacon from an npub
we have not yet peered with, initiate the outbound Noise
@@ -218,7 +218,7 @@ is "all four flags on both ends."
> lan:
> interface: "eth0"
> announce: true
> discovery: true
> listen: true
> auto_connect: true
> accept_connections: true
> dongle:
@@ -227,7 +227,7 @@ is "all four flags on both ends."
> # ...
> ```
>
> Each named instance runs its own socket and discovery state.
> Each named instance runs its own socket and neighbor state.
> A single ground-up link only needs the flat form shown
> first; named instances become useful when the same node
> bridges multiple physical segments.
@@ -390,7 +390,7 @@ What you do need on the AP side:
networks and "secure" enterprise APs ship with it on.
When client isolation is on, the AP refuses to forward
station-to-station frames — the broadcast beacons never
arrive at the other node, and discovery fails silently.
arrive at the other node, and neighbor detection fails silently.
If beacons aren't crossing, this is the first thing to
check.
@@ -401,7 +401,7 @@ adapter name.
### Bluetooth LE (experimental but works)
BLE is a separate transport (`transports.ble.*`) with its own
discovery model — L2CAP advertisements rather than raw L2
neighbor-detection model — L2CAP advertisements rather than raw L2
broadcasts. The shape of the tutorial is the same (advertise +
scan + auto-connect + accept), but the prerequisites are
different: BlueZ, `bluetoothd`, an HCI adapter, and the
@@ -424,7 +424,7 @@ Windows builds skip it.
a radio link), `CAP_NET_RAW`, and a few config flags on each
end are sufficient. The mesh supplies its own identity,
addressing, discovery, and routing.
- **Discovery is a four-flag opt-in.** `announce`, `discovery`,
- **Neighbor detection is a four-flag opt-in.** `announce`, `listen`,
`auto_connect`, and `accept_connections` each control one
thing; both ends must agree before a link will form.
- **The two modes coexist.** Overlay peers and ground-up peers
docs/tutorials/ipv6-adapter-walkthrough.md
+2 -2
View File
@@ -206,8 +206,8 @@ metrics" to "I understand why each one moves the way it does":
in `show sessions` means: the proactive forward-path field, the
reactive `MtuExceeded` mechanism, the hysteresis on increase.
- [../design/fips-architecture.md](../design/fips-architecture.md)
— the two-layer encryption model: link-layer Noise IK over
each hop, end-to-end Noise XK over the session.
— the two-layer encryption model: link-layer Noise XX over
each hop, end-to-end Noise XX over the session.
## What you've learned
docs/tutorials/join-the-test-mesh.md
+6 -6
View File
@@ -27,7 +27,7 @@ UDP/2121, and is reachable from any network that permits arbitrary
outbound UDP.
> **Peer vs. node.** In FIPS terminology, a *peer* is a node
> you have a direct link to — same Noise IK handshake, same
> you have a direct link to — same Noise XX handshake, same
> transport socket. A *node* is any participant on the mesh,
> whether you peer with it directly or reach it through one or
> more hops via your peer's connections. Peering is a local
@@ -85,7 +85,7 @@ peers:
What each field does:
- `npub` — the canonical Nostr public key of `test-us01`. This is
who your daemon will mutually authenticate with over Noise IK.
who your daemon will mutually authenticate with over Noise XX.
- `alias` — a short name your daemon will use when referring to
this peer in logs and `fipsctl show peers` output. Optional.
- `addresses` — one or more transport endpoints. UDP on the
@@ -111,7 +111,7 @@ Within a few seconds you should see lines mentioning:
- An outbound connection attempt to `test-us01` or
`test-us01.fips.network:2121`
- A handshake completion (a "Noise IK link handshake complete"
- A handshake completion (a "Noise XX link handshake complete"
style line, or "peer authenticated" with the test-us01 npub)
- An MMP / link metrics entry naming `test-us01`
@@ -203,7 +203,7 @@ public test mesh, with reach to every node that mesh routes you
to. You have seen:
- **Identity.** Your daemon's ephemeral keypair authenticated to
`test-us01` over Noise IK without either side trusting anyone in
`test-us01` over Noise XX without either side trusting anyone in
advance.
- **Transports.** A UDP socket on your host carries
authenticated, encrypted mesh frames to your peer. No central
@@ -317,8 +317,8 @@ For "what just happened, in detail":
- [../design/fips-architecture.md](../design/fips-architecture.md) —
the protocol stack and the two-layer encryption model.
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) —
Noise IK link encryption, hop-by-hop forwarding.
Noise XX link encryption, hop-by-hop forwarding.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— end-to-end Noise XK, session lifecycle.
— end-to-end Noise XX, session lifecycle.
- [../design/fips-ipv6-adapter.md](../design/fips-ipv6-adapter.md) —
the TUN, the local DNS responder, MTU enforcement.
docs/tutorials/persistent-identity.md
+1 -1
View File
@@ -44,7 +44,7 @@ literal sense. Several things derive from it:
- Your `fd97:...` mesh address — derived from the public key.
- Your `<npub>.fips` DNS name — the npub itself with `.fips`
appended.
- Every authenticated connection — Noise IK at the mesh layer,
- Every authenticated connection — Noise XX at the mesh layer,
XK at the session layer, both prove you hold the matching
secret key.
docs/tutorials/reach-mesh-services.md
+2 -2
View File
@@ -245,9 +245,9 @@ For "what's actually in those packets":
- [../design/fips-architecture.md](../design/fips-architecture.md)
— the protocol stack and the two-layer encryption model.
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) —
Noise IK link encryption, hop-by-hop forwarding.
Noise XX link encryption, hop-by-hop forwarding.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— end-to-end Noise XK between source and destination.
— end-to-end Noise XX between source and destination.
For the trace-it-yourself version of the path you just
exercised, see
docs/tutorials/resolve-peers-via-nostr.md
+1 -1
View File
@@ -178,7 +178,7 @@ The resolution itself happens at debug-log level, so you will
not see it in the default-level journal. The user-facing way to
confirm everything worked is `fipsctl show peers` in the next
step. (To watch the resolution in the journal, run the daemon
manually with `RUST_LOG=fips::discovery::nostr=debug`; not
manually with `RUST_LOG=fips::nostr=debug`; not
necessary for this tutorial.)
## Step 5: Verify the resolved endpoint
packaging/common/fips.yaml
+18 -9
View File
@@ -10,12 +10,21 @@ node:
#
# Or set an explicit key (overrides persistent):
# nsec: "nsec1..."
discovery:
# Optional Nostr-mediated overlay endpoint discovery.
# Mesh-lookup protocol (node.lookup.*): the overlay coordinate-lookup engine
# (mesh address -> coordinates). Defaults shown; uncomment to override.
# lookup:
# ttl: 64
# attempt_timeouts_secs: [1, 2, 4, 8]
# recent_expiry_secs: 10
# backoff_base_secs: 0
# backoff_max_secs: 0
# forward_min_interval_secs: 2
rendezvous:
# Optional Nostr-mediated overlay endpoint rendezvous.
# nostr:
# enabled: true
# policy: configured_only # disabled | configured_only | open
# open_discovery_max_pending: 64 # caps queued open-discovery retries
# open_discovery_max_pending: 64 # caps queued open-rendezvous retries
# app: "fips-overlay-v1"
# advertise: true
# advert_relays:
@@ -34,17 +43,17 @@ node:
# - "stun:stun.cloudflare.com:3478"
# - "stun:global.stun.twilio.com:3478"
#
# Optional mDNS-based LAN discovery for sub-second same-LAN pairing.
# Optional mDNS-based LAN rendezvous for sub-second same-LAN pairing.
# Opt-in (default false): default-off avoids a per-LAN identity
# broadcast on nodes that have deliberately disabled other discovery
# broadcast on nodes that have deliberately disabled other rendezvous
# channels, and avoids any multicast surprise on upgrade. Requires an
# operational UDP transport (the advertised port is the one peers dial).
# lan:
# enabled: false
# # Optional application/network scope carried in the LAN-only TXT
# # record. Browsers that set a scope ignore adverts for other scopes.
# # Kept separate from the Nostr discovery `app` tag so relay-visible
# # adverts can stay generic while LAN discovery stays per-private-network.
# # Kept separate from the Nostr rendezvous `app` tag so relay-visible
# # adverts can stay generic while LAN rendezvous stays per-private-network.
# # scope: "lab-floor-3"
# # Advanced: overrides the mDNS service type. Leave unset in normal
# # use — only needed to run multiple isolated services on one
@@ -89,7 +98,7 @@ transports:
# Ethernet transport — uncomment and set your interface name.
# ethernet:
# interface: "eth0"
# discovery: true
# listen: true
# announce: true
# auto_connect: true
# accept_connections: true
@@ -147,5 +156,5 @@ peers: []
# - transport: udp
# addr: "test-us01.fips.network:2121" # IP or hostname (e.g., "peer.example.com:2121")
# - transport: udp
# addr: "nat" # Use node.discovery.nostr for Nostr/STUN hole punching
# addr: "nat" # Use node.rendezvous.nostr for Nostr/STUN hole punching
# connect_policy: auto_connect
packaging/debian/build-deb.sh
+44 -2
View File
@@ -2,6 +2,7 @@
# Build a .deb package for FIPS using cargo-deb.
#
# Usage: ./build-deb.sh [--target <triple>] [--version <version>] [--no-build]
# [--features <list>]
#
# Prerequisites: cargo-deb (install with: cargo install cargo-deb)
# Output: deploy/fips_<version>_<arch>.deb
@@ -19,6 +20,9 @@ Options:
--target <triple> Rust target triple to build/package
--version <version> Override Debian package version
--no-build Package existing binaries without running cargo build
--features <list> Cargo features to build with (comma-separated). Marks the
auto-derived Version so the package is distinguishable
from a default build of the same commit.
-h, --help Show this help
EOF
}
@@ -26,6 +30,7 @@ EOF
TARGET_TRIPLE=""
VERSION_OVERRIDE=""
NO_BUILD=0
FEATURES=""
while [[ $# -gt 0 ]]; do
case "$1" in
@@ -41,6 +46,10 @@ while [[ $# -gt 0 ]]; do
NO_BUILD=1
shift
;;
--features)
FEATURES="${2:?missing value for --features}"
shift 2
;;
-h|--help)
usage
exit 0
@@ -53,6 +62,16 @@ while [[ $# -gt 0 ]]; do
esac
done
# A feature build that skips the build step would stamp a feature-marked Version
# onto whatever binaries already sit in target/, which is the one outcome the
# marking exists to prevent. Refuse rather than emit a package that misdescribes
# itself.
if [[ -n "${FEATURES}" && "${NO_BUILD}" -eq 1 ]]; then
echo "--features cannot be combined with --no-build: the features would not" >&2
echo "reach the binaries, but the Version would claim they had." >&2
exit 1
fi
cd "${PROJECT_ROOT}"
# Ensure cargo-deb is available
@@ -81,13 +100,33 @@ if [[ -z "${VERSION_OVERRIDE}" ]]; then
if [[ -n "$(git status --porcelain 2>/dev/null)" ]]; then
DIRTY_SUFFIX=".dirty"
fi
# Debian Version: <upstream>~dev+git<YYYYMMDD>.<sha>[.dirty]-1
# A feature build of a given commit is a different package from the
# default build of that same commit, but nothing else in this version
# says so: the crate version, the date and the sha are all identical.
# Without a marker the two are byte-identical versions, so installing
# one over the other is an apt no-op (the very failure the per-commit
# version above exists to prevent) and the node offers no way to tell
# which one it is running. Underscores and commas are not legal in a
# Debian version, so the feature list is folded to dots.
FEATURE_SUFFIX=""
if [[ -n "${FEATURES}" ]]; then
FEATURE_SUFFIX="+$(printf '%s' "${FEATURES}" | tr -c 'a-zA-Z0-9.' '.')"
fi
# Debian Version: <upstream>~dev+git<YYYYMMDD>.<sha>[.dirty][+<features>]-1
# The "~" makes every dev build sort BEFORE the eventual tagged
# release; the date+sha makes consecutive dev builds compare as
# different versions; the trailing "-1" is the Debian revision.
VERSION_OVERRIDE="${BASE_VERSION}~dev+git${GIT_DATE}.${GIT_SHA}${DIRTY_SUFFIX}-1"
# The feature suffix sorts ABOVE the unsuffixed build, so installing a
# feature build is an upgrade and reverting to the default build is a
# downgrade — which apt refuses without being told to, and `dpkg -i`
# performs. Revert with `dpkg -i`, not `apt install`.
VERSION_OVERRIDE="${BASE_VERSION}~dev+git${GIT_DATE}.${GIT_SHA}${DIRTY_SUFFIX}${FEATURE_SUFFIX}-1"
echo "Auto-derived dev Version: ${VERSION_OVERRIDE}"
fi
elif [[ -n "${FEATURES}" ]]; then
echo "Warning: --version was given with --features, so the Version carries no" >&2
echo "feature marker and this package is indistinguishable from a default" >&2
echo "build of the same commit. Mark it yourself if that matters." >&2
fi
# Build the .deb package
@@ -105,6 +144,9 @@ fi
if [[ "${NO_BUILD}" -eq 1 ]]; then
cargo_args+=(--no-build)
fi
if [[ -n "${FEATURES}" ]]; then
cargo_args+=(--features "${FEATURES}")
fi
cargo "${cargo_args[@]}"
# Move output to deploy/
packaging/debian/fips.service
+5
View File
@@ -19,6 +19,11 @@ RestartSec=5
RuntimeDirectory=fips
RuntimeDirectoryMode=0750
# Log directory (/var/log/fips/), where the built-in tick-body profiler writes
# its capture files. Declared so systemd creates it on start and removes it on
# purge; the daemon runs as root and already has access without it.
LogsDirectory=fips
# Security hardening (daemon runs as root for TUN and raw sockets)
ProtectHome=yes
PrivateTmp=yes
packaging/openwrt-apk/build-apk.sh
+2
View File
@@ -182,6 +182,8 @@ install -m 0755 "$RELEASE_DIR/fips" "$STAGE_DIR/usr/bin/fips"
install -m 0755 "$RELEASE_DIR/fipsctl" "$STAGE_DIR/usr/bin/fipsctl"
install -m 0755 "$RELEASE_DIR/fipstop" "$STAGE_DIR/usr/bin/fipstop"
install -m 0755 "$RELEASE_DIR/fips-gateway" "$STAGE_DIR/usr/bin/fips-gateway"
install -m 0755 "$FILES_DIR/usr/bin/fips-mesh-setup" "$STAGE_DIR/usr/bin/fips-mesh-setup"
install -m 0755 "$FILES_DIR/usr/bin/fips-ap-setup" "$STAGE_DIR/usr/bin/fips-ap-setup"
install -d "$STAGE_DIR/etc/init.d"
install -m 0755 "$FILES_DIR/etc/init.d/fips" "$STAGE_DIR/etc/init.d/fips"
packaging/openwrt-ipk/Makefile
+6
View File
@@ -96,6 +96,12 @@ define Package/fips/install
$(INSTALL_BIN) $(RUST_RELEASE_DIR)/fipstop $(1)/usr/bin/fipstop
$(INSTALL_BIN) $(RUST_RELEASE_DIR)/fips-gateway $(1)/usr/bin/fips-gateway
# 802.11s mesh backhaul setup helper
$(INSTALL_BIN) $(CURDIR)/files/usr/bin/fips-mesh-setup $(1)/usr/bin/fips-mesh-setup
# Open "FIPS" access SSID setup helper
$(INSTALL_BIN) $(CURDIR)/files/usr/bin/fips-ap-setup $(1)/usr/bin/fips-ap-setup
# procd init script
$(INSTALL_DIR) $(1)/etc/init.d
$(INSTALL_BIN) $(CURDIR)/files/etc/init.d/fips $(1)/etc/init.d/fips
packaging/openwrt-ipk/README.md
+1
View File
@@ -14,6 +14,7 @@ For ad-hoc deployment without the build system, see
| `/usr/bin/fipsctl` | CLI control tool (`fipsctl show peers`, `fipsctl show links`, …) |
| `/usr/bin/fipstop` | Live TUI dashboard |
| `/usr/bin/fips-gateway` | Outbound LAN gateway service (not started by default) |
| `/usr/bin/fips-mesh-setup` | Opt-in helper — creates an open 802.11s mesh interface for router↔router backhaul |
| `/etc/init.d/fips` | procd service for the daemon (auto-start, crash respawn) |
| `/etc/init.d/fips-gateway` | procd service for the gateway (disabled by default) |
| `/etc/fips/fips.yaml` | Node configuration (edit before first start) |
packaging/openwrt-ipk/build-ipk.sh
+2
View File
@@ -161,6 +161,8 @@ install -m 0755 "$RELEASE_DIR/fips" "$DATA_DIR/usr/bin/fips"
install -m 0755 "$RELEASE_DIR/fipsctl" "$DATA_DIR/usr/bin/fipsctl"
install -m 0755 "$RELEASE_DIR/fipstop" "$DATA_DIR/usr/bin/fipstop"
install -m 0755 "$RELEASE_DIR/fips-gateway" "$DATA_DIR/usr/bin/fips-gateway"
install -m 0755 "$FILES_DIR/usr/bin/fips-mesh-setup" "$DATA_DIR/usr/bin/fips-mesh-setup"
install -m 0755 "$FILES_DIR/usr/bin/fips-ap-setup" "$DATA_DIR/usr/bin/fips-ap-setup"
install -d "$DATA_DIR/etc/init.d"
install -m 0755 "$FILES_DIR/etc/init.d/fips" "$DATA_DIR/etc/init.d/fips"
packaging/openwrt-ipk/files/etc/fips/fips.yaml
+79 -8
View File
@@ -10,12 +10,21 @@ node:
#
# Or set an explicit key (overrides persistent):
# nsec: "nsec1..."
discovery:
# Optional Nostr-mediated overlay endpoint discovery.
# Mesh-lookup protocol (node.lookup.*): the overlay coordinate-lookup engine
# (mesh address -> coordinates). Defaults shown; uncomment to override.
# lookup:
# ttl: 64
# attempt_timeouts_secs: [1, 2, 4, 8]
# recent_expiry_secs: 10
# backoff_base_secs: 0
# backoff_max_secs: 0
# forward_min_interval_secs: 2
rendezvous:
# Optional Nostr-mediated overlay endpoint rendezvous.
# nostr:
# enabled: true
# policy: configured_only # disabled | configured_only | open
# open_discovery_max_pending: 64 # caps queued open-discovery retries
# open_discovery_max_pending: 64 # caps queued open-rendezvous retries
# app: "fips-overlay-v1"
# advertise: true
# advert_relays:
@@ -34,6 +43,14 @@ node:
# - "stun:stun.cloudflare.com:3478"
# - "stun:global.stun.twilio.com:3478"
# mDNS/DNS-SD peer rendezvous on the local link. Ships commented (the
# daemon default is off); 'fips-ap-setup' uncomments it when creating
# the access SSID — phone FIPS apps cannot see raw-Ethernet beacons,
# so mDNS is how they find this router's daemon. Daemon-wide switch,
# left enabled on 'fips-ap-setup remove'.
# lan:
# enabled: true
tun:
enabled: true
name: fips0
@@ -55,7 +72,11 @@ dns:
transports:
udp:
bind_addr: "0.0.0.0:2121"
# Dual-stack wildcard, not "0.0.0.0": access-SSID clients (phones) learn
# this node's addresses from the mDNS advert and prefer the IPv6
# link-local — a v4-only bind silently drops their Noise msg1.
# OpenWrt is Linux (bindv6only=0), so "[::]" accepts v4 too.
bind_addr: "[::]:2121"
# advertise_on_nostr: true
# public: false # false => advertise udp:nat; true => advertise bound host:port
# accept_connections: true # default; refuse inbound msg1 when false
@@ -74,23 +95,73 @@ transports:
ethernet:
wan:
interface: "eth0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
wwan:
interface: "phy0-sta0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
lan:
interface: "br-lan"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
# 802.11s mesh backhaul between FIPS routers. These entries ship
# commented out so a stock install that never creates fips-mesh*
# logs no per-boot "interface missing" bind warning. Running
# 'fips-mesh-setup <radio>' creates the interface AND uncomments the
# matching block here (once per radio; radio0 -> fips-mesh0, radio1 ->
# fips-mesh1); 'fips-mesh-setup remove' re-comments it. Restart fips
# after — a transport whose interface is missing at startup is skipped,
# not retried. Dual-band routers can mesh on both bands at once —
# failover, not multipath: FIPS keeps one active link per peer, the
# other band stands by. The mesh runs OPEN (no SAE) with 802.11s
# forwarding off: FIPS's Noise handshake is the encryption and
# authentication, and FIPS is the routing layer. See
# docs/how-to/set-up-80211s-mesh-backhaul.md.
# mesh0:
# interface: "fips-mesh0"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# mesh1:
# interface: "fips-mesh1"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# Open "!FIPS" access SSID for phones and laptops running FIPS. These
# entries ship commented out so a stock install that never creates
# fips-ap* logs no per-boot "interface missing" bind warning. Running
# 'fips-ap-setup <radio>' creates the interface AND uncomments the
# matching block here (once per radio; radio0 -> fips-ap0, radio1 ->
# fips-ap1); 'fips-ap-setup remove' re-comments it. Restart fips after
# — a transport whose interface is missing at startup is skipped, not
# retried. The SSID is OPEN and isolated on purpose: FIPS's Noise
# handshake is the only security layer, and associated clients reach
# nothing but the FIPS handshake surface. See
# docs/how-to/set-up-open-access-ssid.md.
# ap0:
# interface: "fips-ap0"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# ap1:
# interface: "fips-ap1"
# discovery: true
# announce: true
# auto_connect: true
# accept_connections: true
# Bluetooth Low Energy transport — requires BlueZ and the 'ble' feature.
# ble:
# adapter: "hci0"
@@ -121,5 +192,5 @@ peers: []
# - transport: udp
# addr: "test-us01.fips.network:2121" # IP or hostname (e.g., "peer.example.com:2121")
# - transport: udp
# addr: "nat" # Use node.discovery.nostr for Nostr/STUN hole punching
# addr: "nat" # Use node.rendezvous.nostr for Nostr/STUN hole punching
# connect_policy: auto_connect
packaging/openwrt-ipk/files/usr/bin/fips-ap-setup
Executable
+412
View File
@@ -0,0 +1,412 @@
#!/bin/sh
# fips-ap-setup — configure the open "FIPS" access SSID for phones/laptops.
#
# Usage:
# fips-ap-setup <radio> [ssid] e.g. fips-ap-setup radio0
# fips-ap-setup remove [radio] no radio: remove all instances
#
# Creates an open AP on the given radio so client devices running FIPS can
# reach the router. Every FIPS router broadcasts the SAME SSID ("!FIPS" by
# default — the leading '!' sorts it to the top of alphabetically ordered
# network pickers): same SSID + unique BSSIDs is one standard ESS, so a
# phone saves the network once and roams between all FIPS routers natively.
#
# - encryption 'none' — the AP is OPEN on purpose. FIPS's Noise IK
# handshake authenticates and encrypts everything above the radio, and
# the security type must be uniform across ALL routers anyway: clients
# key a saved network on SSID + security type, so one router with a PSK
# splits the ESS into a different saved network. A stranger can
# associate AND form a FIPS peer link — that is the point of open
# access. The Noise handshake authenticates each link (no
# impersonation of another identity, no MITM); it does NOT gate who
# may peer. Admission is open up to the daemon's max-peers cap; the
# firewall zone below is what confines every client to the FIPS
# overlay (no path to br-lan or the WAN).
# - DHCPv4 + RA IPv6 — dnsmasq serves DHCPv4 from a FIXED subnet,
# 10.21.<N>.0/24 (echoes FIPS port 2121), identical on every router:
# a roaming phone keeps its lease across routers, and dnsmasq's
# authoritative mode (the OpenWrt default) ACKs the renew a foreign
# router never issued. odhcpd additionally announces a ULA prefix in
# router advertisements (stateless SLAAC); DHCPv6 stays off. FIPS
# itself only needs link-local + mDNS, but client provisioning checks
# (Android disconnects without an RA or a DHCP offer) and plain
# laptops both want a real address. The network provides no internet,
# so phones mark it unvalidated and keep cellular as the default
# route while staying associated.
# - ISOLATED — own network and firewall zone: no path to
# br-lan, no forwarding to the WAN, and AP client isolation on.
# Associated clients reach only the FIPS handshake surface.
#
# Interfaces are named per radio index (radio0 -> fips-ap0, radio1 ->
# fips-ap1). Unlike the 802.11s backhaul there is NO same-channel
# constraint — clients scan when they roam, so every router picks its
# access channels freely.
#
# The shipped /etc/fips/fips.yaml carries 'ap0' and 'ap1' entries under
# 'transports.ethernet' bound to these names, but commented out — a stock
# install that never creates fips-ap* then logs no bind warning. This
# helper uncomments the matching entry when it creates an interface and
# re-comments it on remove, so the daemon binds the transport without a
# manual config edit. It also uncomments the node.rendezvous.lan block
# (mDNS/DNS-SD — how phone FIPS apps discover the daemon); that switch is
# daemon-wide and stays on at remove. After an interface is up, restart
# fips.
# See docs/how-to/set-up-open-access-ssid.md for the full guide.
DEFAULT_SSID="!FIPS"
CONFIG="/etc/fips/fips.yaml"
# Replace $CONFIG with the rewritten $CONFIG.tmp. Force mode 0600 first: the
# package installs fips.yaml 0600 (it may hold an inline 'nsec' private key),
# and a fresh tmp file would otherwise land world-readable after the move.
ap_config_write() {
chmod 600 "$CONFIG.tmp" && mv "$CONFIG.tmp" "$CONFIG"
}
# Uncomment the 'ap<idx>' transports.ethernet block in $CONFIG (created by
# 'fips-ap-setup'). Reversible with ap_config_disable. Returns:
# 0 enabled (or already active) 1 no config file 2 no such block
ap_config_enable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ ap$idx:" "$CONFIG" && return 0
grep -q "^ # ap$idx:" "$CONFIG" || return 2
awk -v idx="$idx" '
$0 ~ ("^ # ap" idx ":[ \t]*$") { blk = 1; sub(/^ # /, " "); print; next }
blk && /^ # / { sub(/^ # /, " "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && ap_config_write
}
# Uncomment the 'lan' block under node.rendezvous in $CONFIG — the daemon's
# mDNS/DNS-SD responder+browser. Phone FIPS apps cannot open raw-Ethernet
# sockets, so mDNS is how they find this router's daemon. The match is
# scoped to node.rendezvous: transports.ethernet also has a 'lan' entry at
# the same indent. Daemon-wide switch — enabled here, deliberately NOT
# re-commented on remove (other transports use it once on). Returns:
# 0 enabled (or already active) 1 no config file 2 no such block
lan_rendezvous_enable() {
[ -f "$CONFIG" ] || return 1
state="$(awk '
/^[A-Za-z_]/ { top = $1 }
top == "node:" && /^ [A-Za-z_]/ { sec = $1 }
top == "node:" && sec == "rendezvous:" && /^ lan:[ \t]*$/ { print "active"; exit }
top == "node:" && sec == "rendezvous:" && /^ # lan:[ \t]*$/ { print "commented"; exit }
' "$CONFIG")"
case "$state" in
active) return 0 ;;
commented) ;;
*) return 2 ;;
esac
awk '
/^[A-Za-z_]/ { top = $1 }
top == "node:" && /^ [A-Za-z_]/ { sec = $1 }
top == "node:" && sec == "rendezvous:" && $0 ~ /^ # lan:[ \t]*$/ { blk = 1; sub(/^ # /, " "); print; next }
blk && /^ # / { sub(/^ # /, " "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && ap_config_write
}
# Re-comment the 'ap<idx>' block so the daemon stops binding it (and stops
# warning about the now-missing interface). Inverse of ap_config_enable.
ap_config_disable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ ap$idx:" "$CONFIG" || return 0
awk -v idx="$idx" '
$0 ~ ("^ ap" idx ":[ \t]*$") { blk = 1; sub(/^ /, " # "); print; next }
blk && /^ / { sub(/^ /, " # "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && ap_config_write
}
usage() {
echo "Usage: fips-ap-setup <radio> [ssid]" >&2
echo " fips-ap-setup remove [radio]" >&2
echo "Radios on this device:" >&2
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\([^.]*\)=wifi-device$/ \1/p" >&2
exit 1
}
# List the UCI section names of fips-managed access-point wifi-ifaces.
ap_sections() {
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\(fips_ap[^.=]*\)=wifi-iface$/\1/p"
}
# ---------------------------------------------------------------------------
# remove [radio] — delete the wireless, network, dhcp, and firewall sections
# created below
# ---------------------------------------------------------------------------
if [ "$1" = "remove" ]; then
if [ -n "$2" ]; then
SECTIONS="fips_ap_$(printf '%s' "$2" | tr -c 'a-zA-Z0-9_' '_')"
else
SECTIONS="$(ap_sections)"
fi
[ -n "$SECTIONS" ] || {
echo "No fips access-point instances configured."
exit 0
}
for section in $SECTIONS; do
ifname="$(uci -q get "wireless.$section.ifname")"
uci -q delete "wireless.$section"
uci -q delete "network.$section"
uci -q delete "dhcp.$section"
uci -q del_list "firewall.fips_ap.network=$section"
# Re-comment the matching ap<N> transport in fips.yaml so the
# daemon stops warning about the interface we just removed.
idx="$(printf '%s' "$ifname" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$idx" ] && ap_config_disable "$idx"
echo "Removed ${ifname:-$section}."
done
# Drop the shared zone and its rules once the last instance is gone.
if [ -z "$(uci -q get firewall.fips_ap.network)" ]; then
uci -q delete firewall.fips_ap
uci -q delete firewall.fips_ap_icmpv6
uci -q delete firewall.fips_ap_dhcpv4
uci -q delete firewall.fips_ap_mdns
uci -q delete firewall.fips_ap_fips_udp
uci -q delete firewall.fips_ap_fips_tcp
fi
uci commit wireless
uci commit network
uci commit dhcp
uci commit firewall
wifi reload
/etc/init.d/dnsmasq reload
/etc/init.d/odhcpd reload
/etc/init.d/firewall reload
echo "Restart fips: /etc/init.d/fips restart"
exit 0
fi
RADIO="$1"
SSID="${2:-$DEFAULT_SSID}"
[ -n "$RADIO" ] || usage
if [ "$(uci -q get "wireless.$RADIO")" != "wifi-device" ]; then
echo "Error: '$RADIO' is not a wifi-device in /etc/config/wireless." >&2
usage
fi
# One instance per radio: section fips_ap_<radio>, netdev fips-ap<N>
# where N is the radio's trailing index (radio0 -> fips-ap0). For radios
# named without a trailing number, fall back to the first free index.
SECTION="fips_ap_$(printf '%s' "$RADIO" | tr -c 'a-zA-Z0-9_' '_')"
IDX="$(printf '%s' "$RADIO" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$IDX" ] || IDX="$(printf '%s' "$RADIO" | sed -n 's/^\([0-9]\{1,\}\)$/\1/p')"
if [ -z "$IDX" ]; then
IDX=0
while uci show wireless 2>/dev/null | grep -q "\.ifname='fips-ap$IDX'"; do
IDX=$((IDX + 1))
done
fi
AP_IFNAME="fips-ap$IDX"
# Refuse a name collision from another radio's instance (e.g. two radios
# whose names end in the same digit) rather than silently hijacking it.
OWNER="$(uci show wireless 2>/dev/null \
| sed -n "s/^wireless\.\(fips_ap[^.=]*\)\.ifname='$AP_IFNAME'$/\1/p")"
if [ -n "$OWNER" ] && [ "$OWNER" != "$SECTION" ]; then
echo "Error: $AP_IFNAME is already used by section '$OWNER'." >&2
echo "Remove it first: fips-ap-setup remove" >&2
exit 1
fi
# ---------------------------------------------------------------------------
# Wireless: open AP with client isolation. Clients of the same AP cannot
# exchange L2 frames directly — two FIPS phones on one router still reach
# each other through the router at the overlay layer. Isolation is an L2
# control only: a stranger who peers is an overlay peer like any other, so
# the FIPS overlay (not L2) is the trust boundary between clients.
# ---------------------------------------------------------------------------
uci -q delete "wireless.$SECTION"
uci set "wireless.$SECTION=wifi-iface"
uci set "wireless.$SECTION.device=$RADIO"
uci set "wireless.$SECTION.mode=ap"
uci set "wireless.$SECTION.ssid=$SSID"
uci set "wireless.$SECTION.encryption=none"
uci set "wireless.$SECTION.isolate=1"
uci set "wireless.$SECTION.ifname=$AP_IFNAME"
uci set "wireless.$SECTION.network=$SECTION"
# Radios ship disabled on fresh OpenWrt installs; a disabled radio would
# leave the AP down with no error anywhere visible.
if [ "$(uci -q get "wireless.$RADIO.disabled")" = "1" ]; then
echo "Note: enabling $RADIO (was disabled)."
uci -q delete "wireless.$RADIO.disabled"
fi
CHANNEL="$(uci -q get "wireless.$RADIO.channel")"
BAND="$(uci -q get "wireless.$RADIO.band")"
# ---------------------------------------------------------------------------
# Network: IPv4 from the fixed convention 10.21.<IDX>.1/24 — deterministic,
# so every router serving the same radio index lands on the same subnet and
# a roaming client's lease stays valid. IPv6 is a static ULA /64 so odhcpd
# has a prefix to announce; that space is per-router and disposable — a
# roaming phone SLAACs a fresh address on each router, and the FIPS overlay
# identity (not the IP) is the mobility anchor. The ULA is derived from the
# router's global ULA prefix; a re-run keeps the address already configured.
# ---------------------------------------------------------------------------
AP_ADDR="$(uci -q get "network.$SECTION.ip6addr")"
case "$AP_ADDR" in
fd*) ;; # keep the existing address on re-run
*)
ULA_BASE=""
ULA_PREFIX="$(uci -q get network.globals.ula_prefix)"
case "$ULA_PREFIX" in
fd*::/48) ULA_BASE="${ULA_PREFIX%::/48}" ;;
esac
if [ -z "$ULA_BASE" ]; then
HEX="$(head -c 5 /dev/urandom | hexdump -e '5/1 "%02x"')"
ULA_BASE="fd$(printf '%s' "$HEX" | cut -c1-2):$(printf '%s' "$HEX" | cut -c3-6):$(printf '%s' "$HEX" | cut -c7-10)"
echo "Note: no usable ULA prefix in network.globals — generated $ULA_BASE::/48 for this AP."
fi
# 64000 = 0xfa00 — high subnet IDs keep clear of br-lan's low
# ip6assign allocations from the same ULA prefix.
AP_ADDR="$ULA_BASE:$(printf '%04x' $((64000 + IDX)))::1/64"
;;
esac
AP_ADDR4="10.21.$IDX.1"
uci -q delete "network.$SECTION"
uci set "network.$SECTION=interface"
uci set "network.$SECTION.proto=static"
uci set "network.$SECTION.ipaddr=$AP_ADDR4"
uci set "network.$SECTION.netmask=255.255.255.0"
uci set "network.$SECTION.ip6addr=$AP_ADDR"
# ---------------------------------------------------------------------------
# DHCP/RA: dnsmasq DHCPv4 leases out of 10.21.<IDX>.0/24, plus router
# advertisements for the ULA (stateless SLAAC, no DHCPv6). ra_default '2'
# announces a default router even without an upstream default route:
# Android's provisioning wants address + route + DNS, and its validation
# probe then fails by design (no internet), so the phone keeps cellular as
# the default route. 'dhcpv4 server' is read by BOTH dnsmasq (the default
# DHCPv4 server) and odhcpd (serves v4 only when odhcpd.maindhcp is set),
# so either arrangement hands out leases.
# ---------------------------------------------------------------------------
uci -q delete "dhcp.$SECTION"
uci set "dhcp.$SECTION=dhcp"
uci set "dhcp.$SECTION.interface=$SECTION"
uci set "dhcp.$SECTION.ra=server"
uci set "dhcp.$SECTION.ra_default=2"
uci set "dhcp.$SECTION.dhcpv6=disabled"
uci set "dhcp.$SECTION.dhcpv4=server"
uci set "dhcp.$SECTION.start=10"
uci set "dhcp.$SECTION.limit=200"
# Authoritative is the OpenWrt default, but roaming correctness depends on
# it (a foreign router must ACK a lease it never issued), so pin it.
[ -n "$(uci -q get dhcp.@dnsmasq[0])" ] && uci set dhcp.@dnsmasq[0].authoritative=1
# ---------------------------------------------------------------------------
# Firewall: one shared 'fips_ap' zone for all instances. Everything is
# rejected except what a FIPS client needs — DHCPv4 (addressing), ICMPv6
# (SLAAC itself), mDNS (discovery), and the FIPS UDP/TCP transports (the
# handshake surface).
# The raw-Ethernet transport (EtherType 0x2121) is not IP and never
# traverses the firewall. No forwardings exist, so there is no path to
# br-lan or the WAN.
# ---------------------------------------------------------------------------
if [ "$(uci -q get firewall.fips_ap)" != "zone" ]; then
uci set firewall.fips_ap=zone
fi
uci set firewall.fips_ap.name=fips_ap
uci set firewall.fips_ap.input=REJECT
uci set firewall.fips_ap.output=ACCEPT
uci set firewall.fips_ap.forward=REJECT
uci -q del_list "firewall.fips_ap.network=$SECTION"
uci add_list "firewall.fips_ap.network=$SECTION"
# ap_rule <section-suffix> <name> <proto> [dest_port]
ap_rule() {
rule="firewall.fips_ap_$1"
uci -q delete "$rule"
uci set "$rule=rule"
uci set "$rule.name=$2"
uci set "$rule.src=fips_ap"
uci set "$rule.proto=$3"
uci set "$rule.target=ACCEPT"
[ -z "${4:-}" ] || uci set "$rule.dest_port=$4"
}
ap_rule icmpv6 "FIPS-AP-ICMPv6" icmp
uci set firewall.fips_ap_icmpv6.family=ipv6
ap_rule dhcpv4 "FIPS-AP-DHCPv4" udp 67
uci set firewall.fips_ap_dhcpv4.family=ipv4
ap_rule mdns "FIPS-AP-mDNS" udp 5353
ap_rule fips_udp "FIPS-AP-FIPS-UDP" udp 2121
ap_rule fips_tcp "FIPS-AP-FIPS-TCP" tcp 8443
uci commit wireless
uci commit network
uci commit dhcp
uci commit firewall
wifi reload
/etc/init.d/dnsmasq reload
/etc/init.d/odhcpd reload
/etc/init.d/firewall reload
# Enable the matching ap<N> transport in the shipped fips.yaml (it ships
# commented out). Tailor the restart hint to what we could do.
ap_config_enable "$IDX"
case $? in
0) TRANSPORT_NOTE="The ap$IDX transport in $CONFIG that binds '$AP_IFNAME' is
now uncommented and enabled." ;;
1) TRANSPORT_NOTE="No $CONFIG found — add a transports.ethernet entry binding
interface '$AP_IFNAME' by hand." ;;
*) TRANSPORT_NOTE="No 'ap$IDX' entry in $CONFIG — add a transports.ethernet
entry binding interface '$AP_IFNAME' by hand (copy the ap0 block)." ;;
esac
# Phones discover the daemon via mDNS, not raw-Ethernet beacons — make sure
# the daemon-wide mDNS rendezvous is on.
if lan_rendezvous_enable; then
MDNS_NOTE="node.rendezvous.lan (mDNS) is enabled — phone FIPS apps
discover this router via DNS-SD."
else
MDNS_NOTE="Could not enable mDNS in $CONFIG — set
'node.rendezvous.lan.enabled: true' by hand; phone FIPS apps rely
on it to discover this router."
fi
cat <<EOF
Created open access SSID '$SSID' as $AP_IFNAME on $RADIO \
(band ${BAND:-?}, channel ${CHANNEL:-auto}).
DHCPv4 on $AP_ADDR4/24 and RA IPv6 on $AP_ADDR — no internet,
isolated from br-lan and the WAN.
ALL FIPS routers must broadcast this SSID with the same security type
(open) — phones then save it once and roam between routers as one
network. The 10.21.$IDX.0/24 subnet is the same on every router on
purpose: leases survive roaming. Unlike the mesh backhaul, channels
are free per router. On a dual-band router, run fips-ap-setup for the
other radio too so clients can pick either band.
On first connect a phone warns that the network has no internet —
choose "stay connected" and "don't ask again". That choice is stored
per SSID, so it covers every FIPS router.
Next steps:
1. $TRANSPORT_NOTE
2. $MDNS_NOTE
Restart the daemon AFTER the interface is up — a transport whose
interface is missing at startup is skipped, not retried:
/etc/init.d/fips restart
3. Associate a phone or laptop running FIPS and verify:
iw dev $AP_IFNAME station dump
and the FIPS link on top of it:
fipsctl show peers
Run 'fips-ap-setup remove' to undo all instances, or
'fips-ap-setup remove $RADIO' for just this one.
EOF
packaging/openwrt-ipk/files/usr/bin/fips-mesh-setup
Executable
+261
View File
@@ -0,0 +1,261 @@
#!/bin/sh
# fips-mesh-setup — configure open 802.11s mesh interfaces for FIPS backhaul.
#
# Usage:
# fips-mesh-setup <radio> [mesh-id] e.g. fips-mesh-setup radio1
# fips-mesh-setup remove [radio] no radio: remove all instances
#
# Creates a mesh-point interface on the given radio and leaves everything
# above L2 to FIPS. Run once per radio: dual-band routers can mesh on both
# bands at once (2.4 GHz reaches further, 5 GHz carries more). Note this is
# failover, not multipath — FIPS keeps one active link per peer; the other
# band stands by and reconnects the peer if the active link dies.
#
# - encryption 'none' — the mesh is OPEN on purpose. FIPS's Noise
# handshake authenticates and encrypts every peer link, so SAE would
# only duplicate that (and on ath10k it forces the slower raw Tx/Rx
# firmware mode). A stranger can form an 802.11s peering AND a FIPS
# peer link — the Noise handshake authenticates each link (no
# impersonation of another identity, no MITM), it does not gate who
# may peer. Admission is open up to the daemon's max-peers cap.
# - mesh_fwding '0' — disables 802.11s HWMP forwarding so each mesh
# link is a plain L2 neighbor link. FIPS is the routing layer; two
# routing layers would fight.
#
# Interfaces are named per radio index (radio0 -> fips-mesh0, radio1 ->
# fips-mesh1) and are intentionally NOT bridged into br-lan: the FIPS
# Ethernet transport binds each directly and runs discovery beacons over it.
#
# The shipped /etc/fips/fips.yaml carries 'mesh0' and 'mesh1' entries under
# 'transports.ethernet' bound to these names, but commented out — a stock
# install that never creates fips-mesh* then logs no bind warning. This
# helper uncomments the matching entry when it creates an interface and
# re-comments it on remove, so the daemon binds the transport without a
# manual config edit. After an interface is up, restart fips.
# See docs/how-to/set-up-80211s-mesh-backhaul.md for the full guide.
DEFAULT_MESH_ID="fips-mesh"
CONFIG="/etc/fips/fips.yaml"
# Replace $CONFIG with the rewritten $CONFIG.tmp. Force mode 0600 first: the
# package installs fips.yaml 0600 (it may hold an inline 'nsec' private key),
# and a fresh tmp file would otherwise land world-readable after the move.
mesh_config_write() {
chmod 600 "$CONFIG.tmp" && mv "$CONFIG.tmp" "$CONFIG"
}
# Uncomment the 'mesh<idx>' transports.ethernet block in $CONFIG (created by
# 'fips-mesh-setup'). Reversible with mesh_config_disable. Returns:
# 0 enabled (or already active) 1 no config file 2 no such block
mesh_config_enable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ mesh$idx:" "$CONFIG" && return 0
grep -q "^ # mesh$idx:" "$CONFIG" || return 2
awk -v idx="$idx" '
$0 ~ ("^ # mesh" idx ":[ \t]*$") { blk = 1; sub(/^ # /, " "); print; next }
blk && /^ # / { sub(/^ # /, " "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && mesh_config_write
}
# Re-comment the 'mesh<idx>' block so the daemon stops binding it (and stops
# warning about the now-missing interface). Inverse of mesh_config_enable.
mesh_config_disable() {
idx="$1"
[ -f "$CONFIG" ] || return 1
grep -q "^ mesh$idx:" "$CONFIG" || return 0
awk -v idx="$idx" '
$0 ~ ("^ mesh" idx ":[ \t]*$") { blk = 1; sub(/^ /, " # "); print; next }
blk && /^ / { sub(/^ /, " # "); print; next }
{ blk = 0; print }
' "$CONFIG" > "$CONFIG.tmp" && mesh_config_write
}
usage() {
echo "Usage: fips-mesh-setup <radio> [mesh-id]" >&2
echo " fips-mesh-setup remove [radio]" >&2
echo "Radios on this device:" >&2
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\([^.]*\)=wifi-device$/ \1/p" >&2
exit 1
}
# List the UCI section names of fips-managed mesh wifi-ifaces.
mesh_sections() {
uci show wireless 2>/dev/null | sed -n "s/^wireless\.\(fips_mesh[^.=]*\)=wifi-iface$/\1/p"
}
# ---------------------------------------------------------------------------
# remove [radio] — delete the wireless and network sections created below
# ---------------------------------------------------------------------------
if [ "$1" = "remove" ]; then
if [ -n "$2" ]; then
SECTIONS="fips_mesh_$(printf '%s' "$2" | tr -c 'a-zA-Z0-9_' '_')"
else
SECTIONS="$(mesh_sections)"
fi
[ -n "$SECTIONS" ] || {
echo "No fips mesh instances configured."
exit 0
}
for section in $SECTIONS; do
ifname="$(uci -q get "wireless.$section.ifname")"
uci -q delete "wireless.$section"
uci -q delete "network.$section"
# Re-comment the matching mesh<N> transport in fips.yaml so the
# daemon stops warning about the interface we just removed.
idx="$(printf '%s' "$ifname" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$idx" ] && mesh_config_disable "$idx"
echo "Removed ${ifname:-$section}."
done
uci commit wireless
uci commit network
# 'wifi reload' re-applies the whole wireless config, so it briefly drops
# every client AP on all radios (a few seconds) — expected on remove.
wifi reload
echo "Restart fips: /etc/init.d/fips restart"
exit 0
fi
RADIO="$1"
MESH_ID="${2:-$DEFAULT_MESH_ID}"
[ -n "$RADIO" ] || usage
if [ "$(uci -q get "wireless.$RADIO")" != "wifi-device" ]; then
echo "Error: '$RADIO' is not a wifi-device in /etc/config/wireless." >&2
usage
fi
# One instance per radio: section fips_mesh_<radio>, netdev fips-mesh<N>
# where N is the radio's trailing index (radio0 -> fips-mesh0). For radios
# named without a trailing number, fall back to the first free index.
SECTION="fips_mesh_$(printf '%s' "$RADIO" | tr -c 'a-zA-Z0-9_' '_')"
IDX="$(printf '%s' "$RADIO" | sed -n 's/.*[^0-9]\([0-9]\{1,\}\)$/\1/p')"
[ -n "$IDX" ] || IDX="$(printf '%s' "$RADIO" | sed -n 's/^\([0-9]\{1,\}\)$/\1/p')"
if [ -z "$IDX" ]; then
IDX=0
while uci show wireless 2>/dev/null | grep -q "\.ifname='fips-mesh$IDX'"; do
IDX=$((IDX + 1))
done
fi
MESH_IFNAME="fips-mesh$IDX"
# Refuse a name collision from another radio's instance (e.g. two radios
# whose names end in the same digit) rather than silently hijacking it.
OWNER="$(uci show wireless 2>/dev/null \
| sed -n "s/^wireless\.\(fips_mesh[^.=]*\)\.ifname='$MESH_IFNAME'$/\1/p")"
if [ -n "$OWNER" ] && [ "$OWNER" != "$SECTION" ]; then
echo "Error: $MESH_IFNAME is already used by section '$OWNER'." >&2
echo "Remove it first: fips-mesh-setup remove" >&2
exit 1
fi
# ---------------------------------------------------------------------------
# Driver capability check (advisory — config below is harmless either way)
# ---------------------------------------------------------------------------
if command -v iw >/dev/null 2>&1; then
if ! iw list 2>/dev/null | grep -q "\* mesh point"; then
echo "Warning: no radio on this device advertises 'mesh point' support" >&2
echo "(iw list | grep 'mesh point'). The interface may fail to come up." >&2
fi
fi
# ---------------------------------------------------------------------------
# Wireless: open 802.11s mesh point, HWMP forwarding off
# ---------------------------------------------------------------------------
uci -q delete "wireless.$SECTION"
uci set "wireless.$SECTION=wifi-iface"
uci set "wireless.$SECTION.device=$RADIO"
uci set "wireless.$SECTION.mode=mesh"
uci set "wireless.$SECTION.mesh_id=$MESH_ID"
uci set "wireless.$SECTION.encryption=none"
uci set "wireless.$SECTION.mesh_fwding=0"
uci set "wireless.$SECTION.ifname=$MESH_IFNAME"
uci set "wireless.$SECTION.network=$SECTION"
# Radios ship disabled on fresh OpenWrt installs; a disabled radio would
# leave the mesh interface down with no error anywhere visible.
if [ "$(uci -q get "wireless.$RADIO.disabled")" = "1" ]; then
echo "Note: enabling $RADIO (was disabled)."
uci -q delete "wireless.$RADIO.disabled"
fi
# The mesh inherits the radio's channel, and mesh points only peer on the
# same channel. 'auto' lets each router pick its own — the classic silent
# non-peering cause — so surface the setting loudly.
CHANNEL="$(uci -q get "wireless.$RADIO.channel")"
BAND="$(uci -q get "wireless.$RADIO.band")"
if [ -z "$CHANNEL" ] || [ "$CHANNEL" = "auto" ]; then
echo "Warning: $RADIO channel is '${CHANNEL:-unset}' — each router may" >&2
echo "auto-select a different channel and mesh points only peer on the" >&2
echo "same one. Pin the same channel on every backhaul router, e.g.:" >&2
echo " uci set wireless.$RADIO.channel='36' && uci commit wireless && wifi reload" >&2
fi
# A client (sta) interface on the same radio follows its upstream AP's
# channel and drags every other interface with it — a mesh pinned to a
# different channel silently never joins, and does not recover when the
# STA disconnects.
for s in $(uci show wireless 2>/dev/null | sed -n "s/^wireless\.\([^.]*\)\.mode='sta'$/\1/p"); do
if [ "$(uci -q get "wireless.$s.device")" = "$RADIO" ]; then
echo "Warning: $RADIO also carries client interface '$s' (mode 'sta')." >&2
echo "The whole radio follows that STA's upstream channel — a mesh" >&2
echo "pinned to a different channel stays down silently. Align the" >&2
echo "mesh channel with the upstream AP, or put the mesh on a radio" >&2
echo "without a STA (a roaming uplink is incompatible with a" >&2
echo "fixed-channel mesh on the same radio)." >&2
fi
done
# ---------------------------------------------------------------------------
# Network: unmanaged interface so netifd brings the netdev up. No IP config —
# the FIPS Ethernet transport speaks raw frames on it.
# ---------------------------------------------------------------------------
uci -q delete "network.$SECTION"
uci set "network.$SECTION=interface"
uci set "network.$SECTION.proto=none"
uci commit wireless
uci commit network
# 'wifi reload' re-applies the whole wireless config, so it briefly drops
# every client AP on all radios (a few seconds) — expected when adding a mesh.
wifi reload
# Enable the matching mesh<N> transport in the shipped fips.yaml (it ships
# commented out). Tailor the restart hint to what we could do.
mesh_config_enable "$IDX"
case $? in
0) TRANSPORT_NOTE="The mesh$IDX transport in $CONFIG that binds '$MESH_IFNAME' is
now uncommented and enabled." ;;
1) TRANSPORT_NOTE="No $CONFIG found — add a transports.ethernet entry binding
interface '$MESH_IFNAME' by hand." ;;
*) TRANSPORT_NOTE="No 'mesh$IDX' entry in $CONFIG — add a transports.ethernet
entry binding interface '$MESH_IFNAME' by hand (copy the mesh0 block)." ;;
esac
cat <<EOF
Created open 802.11s mesh '$MESH_ID' as $MESH_IFNAME on $RADIO \
(band ${BAND:-?}, channel ${CHANNEL:-auto}).
ALL routers in this backhaul must share this mesh ID AND channel
(per band). On a dual-band router, run fips-mesh-setup for the other
radio too — second band is a standby path (failover, not multipath).
Next steps:
1. $TRANSPORT_NOTE
Restart the daemon AFTER the interface is up — a transport whose
interface is missing at startup is skipped, not retried:
/etc/init.d/fips restart
2. Verify L2 peering with a second FIPS router in range:
iw dev $MESH_IFNAME station dump
and the FIPS link on top of it:
fipsctl show peers
Run 'fips-mesh-setup remove' to undo all instances, or
'fips-mesh-setup remove $RADIO' for just this one.
EOF
packaging/systemd/README.install.md
+1 -1
View File
@@ -68,7 +68,7 @@ and set the interface name:
transports:
ethernet:
interface: "eth0"
discovery: true
listen: true
announce: true
auto_connect: true
accept_connections: true
packaging/systemd/fips.service
+5
View File
@@ -14,6 +14,11 @@ RestartSec=5
RuntimeDirectory=fips
RuntimeDirectoryMode=0750
# Log directory (/var/log/fips/), where the built-in tick-body profiler writes
# its capture files. Declared so systemd creates it on start and removes it on
# purge; the daemon runs as root and already has access without it.
LogsDirectory=fips
# Security hardening (daemon runs as root for TUN and raw sockets)
ProtectHome=yes
PrivateTmp=yes
src/bin/fips.rs
+14 -17
View File
@@ -8,7 +8,7 @@ use fips::config::{IdentitySource, resolve_identity};
use fips::version;
use fips::{Config, Node};
use std::path::PathBuf;
use tracing::{debug, error, info, warn};
use tracing::{debug, error, info};
use tracing_subscriber::{EnvFilter, fmt};
/// FIPS mesh network daemon
@@ -157,26 +157,23 @@ async fn run_daemon(
info!("FIPS running");
// Run the RX event loop until shutdown signal.
// stop() drops the packet channel, causing run_rx_loop to exit.
tokio::select! {
result = node.run_rx_loop() => {
match result {
Ok(()) => info!("RX loop exited"),
Err(e) => error!("RX loop error: {}", e),
}
}
_ = shutdown_signal => {
info!("Shutdown signal received");
}
// Serve until the shutdown signal, then drain in place before returning.
// The rx loop observes the signal directly, so its channels are never
// destructively cancelled — they live in the loop's locals across serve and
// drain, and are dropped only on clean exit (after which teardown does not
// need them). On the signal the loop broadcasts a shutdown Disconnect and
// waits (bounded by node.drain_timeout_secs) for peers to clear.
match node.run_rx_loop_with_shutdown(shutdown_signal).await {
Ok(()) => info!("RX loop exited"),
Err(e) => error!("RX loop error: {}", e),
}
info!("FIPS shutting down");
// Stop the node (shuts down transports, TUN, I/O threads)
if let Err(e) = node.stop().await {
warn!("Error during shutdown: {}", e);
}
// Close the drain window (if the loop drained) and tear down. A drained
// loop tears down without re-broadcasting; a loop that exited some other
// way falls back to the immediate stop().
node.finish_shutdown().await;
info!("FIPS shutdown complete");
}
src/bin/fipsctl.rs
+45
View File
@@ -76,6 +76,37 @@ enum Commands {
#[command(subcommand)]
what: StatsCommands,
},
/// Control the built-in profiler (requires a `--features profiling` build)
#[cfg(feature = "profiling")]
Profile {
#[command(subcommand)]
what: ProfileCommands,
},
}
#[cfg(feature = "profiling")]
#[derive(Subcommand, Debug)]
enum ProfileCommands {
/// Profile the rx-loop tick body
Tick {
#[command(subcommand)]
action: ProfileTickAction,
},
}
#[cfg(feature = "profiling")]
#[derive(Subcommand, Debug)]
enum ProfileTickAction {
/// Start a capture
On {
/// Directory for the capture file (default /var/log/fips)
#[arg(long)]
dir: Option<PathBuf>,
},
/// Stop the running capture
Off,
/// Report capture state
Status,
}
#[derive(Subcommand, Debug)]
@@ -471,6 +502,20 @@ fn main() {
build_command("show_stats_history", params)
}
},
#[cfg(feature = "profiling")]
Commands::Profile { what } => match what {
ProfileCommands::Tick { action } => match action {
ProfileTickAction::On { dir } => match dir {
Some(dir) => build_command(
"profile_tick_on",
serde_json::json!({"dir": dir.display().to_string()}),
),
None => build_query("profile_tick_on"),
},
ProfileTickAction::Off => build_query("profile_tick_off"),
ProfileTickAction::Status => build_query("profile_tick_status"),
},
},
Commands::Keygen { .. } => unreachable!(),
};
src/bin/fipstop/ui/bloom.rs
+20 -1
View File
@@ -121,7 +121,6 @@ fn draw_stats(frame: &mut Frame, data: &serde_json::Value, scroll: u16, focused:
&helpers::nested_u64(data, "stats", "decode_error"),
),
helpers::kv_line("Invalid", &helpers::nested_u64(data, "stats", "invalid")),
helpers::kv_line("Non-V1", &helpers::nested_u64(data, "stats", "non_v1")),
helpers::kv_line(
"Unknown Peer",
&helpers::nested_u64(data, "stats", "unknown_peer"),
@@ -130,6 +129,26 @@ fn draw_stats(frame: &mut Frame, data: &serde_json::Value, scroll: u16, focused:
Line::from(""),
helpers::section_header("Outbound"),
helpers::kv_line("Sent", &helpers::nested_u64(data, "stats", "sent")),
helpers::kv_line(
"Full Sends",
&helpers::nested_u64(data, "stats", "full_sends"),
),
helpers::kv_line(
"Deltas Sent",
&helpers::nested_u64(data, "stats", "deltas_sent"),
),
helpers::kv_line(
"NACKs Sent",
&helpers::nested_u64(data, "stats", "nacks_sent"),
),
helpers::kv_line(
"NACKs Received",
&helpers::nested_u64(data, "stats", "nacks_received"),
),
helpers::kv_line(
"Size Changes",
&helpers::nested_u64(data, "stats", "size_changes"),
),
helpers::kv_line(
"Debounce Suppressed",
&helpers::nested_u64(data, "stats", "debounce_suppressed"),
src/bin/fipstop/ui/routing.rs
+22 -22
View File
@@ -134,8 +134,8 @@ fn draw_routing_stats(
let cols =
Layout::horizontal([Constraint::Percentage(50), Constraint::Percentage(50)]).split(inner);
// Shorthand for a nested counter value (e.g. discovery.req_received).
let disc = |key: &str| helpers::nested_u64(data, "discovery", key);
// Shorthand for a nested counter value (e.g. lookup.req_received).
let lookup = |key: &str| helpers::nested_u64(data, "lookup", key);
let err = |key: &str| helpers::nested_u64(data, "error_signals", key);
let cong = |key: &str| helpers::nested_u64(data, "congestion", key);
@@ -174,32 +174,32 @@ fn draw_routing_stats(
));
left.push(Line::from(""));
left.extend(section(
"Discovery Requests",
"Lookup Requests",
&[
("Received", disc("req_received")),
("Forwarded", disc("req_forwarded")),
("Initiated", disc("req_initiated")),
("Deduplicated", disc("req_deduplicated")),
("Target Is Us", disc("req_target_is_us")),
("Duplicate", disc("req_duplicate")),
("Bloom Miss", disc("req_bloom_miss")),
("Backoff Suppressed", disc("req_backoff_suppressed")),
("Fwd Rate Limited", disc("req_forward_rate_limited")),
("TTL Exhausted", disc("req_ttl_exhausted")),
("Decode Error", disc("req_decode_error")),
("Received", lookup("req_received")),
("Forwarded", lookup("req_forwarded")),
("Initiated", lookup("req_initiated")),
("Deduplicated", lookup("req_deduplicated")),
("Target Is Us", lookup("req_target_is_us")),
("Duplicate", lookup("req_duplicate")),
("Bloom Miss", lookup("req_bloom_miss")),
("Backoff Suppressed", lookup("req_backoff_suppressed")),
("Fwd Rate Limited", lookup("req_forward_rate_limited")),
("TTL Exhausted", lookup("req_ttl_exhausted")),
("Decode Error", lookup("req_decode_error")),
],
));
left.push(Line::from(""));
left.extend(section(
"Discovery Responses",
"Lookup Responses",
&[
("Received", disc("resp_received")),
("Accepted", disc("resp_accepted")),
("Forwarded", disc("resp_forwarded")),
("Timed Out", disc("resp_timed_out")),
("Identity Miss", disc("resp_identity_miss")),
("Proof Failed", disc("resp_proof_failed")),
("Decode Error", disc("resp_decode_error")),
("Received", lookup("resp_received")),
("Accepted", lookup("resp_accepted")),
("Forwarded", lookup("resp_forwarded")),
("Timed Out", lookup("resp_timed_out")),
("Identity Miss", lookup("resp_identity_miss")),
("Proof Failed", lookup("resp_proof_failed")),
("Decode Error", lookup("resp_decode_error")),
],
));
src/bloom/mod.rs
-60
View File
@@ -1,60 +0,0 @@
//! Bloom Filter Implementation
//!
//! 1KB Bloom filters for reachability in FIPS routing. Each node
//! maintains filters that summarize which destinations are reachable
//! through each peer, enabling efficient routing decisions without
//! global network knowledge.
//!
//! ## v1 Parameters
//!
//! - Size: 1 KB (8,192 bits) - sized for actual ~400-800 entry occupancy
//! - Hash functions: k=5 - optimal at ~1,200 entries, good for 800-1,600
//! - Bandwidth: 1 KB/announce (75% reduction from original 4KB design)
//!
//! These parameters are right-sized for typical network occupancy of
//! ~250-800 entries per node.
mod filter;
mod state;
use thiserror::Error;
pub use filter::BloomFilter;
pub use state::BloomState;
/// Default filter size in bits (1KB = 8,192 bits).
///
/// Sized for ~800-1,600 entries. FPR ~0.05% at 400 entries, ~0.9% at 800.
/// This is v1 protocol default (size_class=1).
pub const DEFAULT_FILTER_SIZE_BITS: usize = 8192;
/// Default filter size in bytes (1KB).
pub const DEFAULT_FILTER_SIZE_BYTES: usize = DEFAULT_FILTER_SIZE_BITS / 8;
/// Default number of hash functions.
///
/// k=5 is optimal at ~1,200 entries and a good compromise for 800-1,600.
/// At 400 entries: FPR ~0.05%. At 800 entries: FPR ~0.9%.
pub const DEFAULT_HASH_COUNT: u8 = 5;
/// Size class for v1 protocol (1 KB filters).
pub const V1_SIZE_CLASS: u8 = 1;
/// Filter sizes by size_class: bytes = 512 << size_class
pub const SIZE_CLASS_BYTES: [usize; 4] = [512, 1024, 2048, 4096];
/// Errors related to Bloom filter operations.
#[derive(Debug, Error)]
pub enum BloomError {
#[error("invalid filter size: expected {expected} bits, got {got}")]
InvalidSize { expected: usize, got: usize },
#[error("filter size must be a multiple of 8, got {0}")]
SizeNotByteAligned(usize),
#[error("hash count must be positive")]
ZeroHashCount,
}
#[cfg(test)]
mod tests;
src/cache/coord_cache.rs
Vendored
+1 -1
View File
@@ -9,7 +9,7 @@ use std::collections::HashMap;
use super::CacheStats;
use super::entry::CacheEntry;
use crate::NodeAddr;
use crate::tree::TreeCoordinate;
use crate::proto::stp::TreeCoordinate;
/// Default maximum entries in coordinate cache.
pub const DEFAULT_COORD_CACHE_SIZE: usize = 50_000;
src/cache/entry.rs
Vendored
+1 -1
View File
@@ -1,6 +1,6 @@
//! Cache entry with TTL and LRU tracking.
use crate::tree::TreeCoordinate;
use crate::proto::stp::TreeCoordinate;
/// A cached coordinate entry.
#[derive(Clone, Debug)]
src/config/mod.rs
+227 -31
View File
@@ -34,9 +34,9 @@ use thiserror::Error;
#[cfg(target_os = "linux")]
pub use gateway::{ConntrackConfig, GatewayConfig, GatewayDnsConfig, PortForward, Proto};
pub use node::{
BloomConfig, BuffersConfig, CacheConfig, ControlConfig, DiscoveryConfig, LimitsConfig,
NodeConfig, NostrDiscoveryConfig, NostrDiscoveryPolicy, RateLimitConfig, RekeyConfig,
RetryConfig, SessionConfig, SessionMmpConfig, TreeConfig,
BloomConfig, BuffersConfig, CacheConfig, ControlConfig, LimitsConfig, LookupConfig, MmpConfig,
NodeConfig, NostrRendezvousConfig, NostrRendezvousPolicy, RateLimitConfig, RekeyConfig,
RendezvousConfig, RetryConfig, SessionConfig, SessionMmpConfig, TreeConfig,
};
pub use peer::{ConnectPolicy, PeerAddress, PeerConfig};
pub use transport::{
@@ -490,10 +490,59 @@ impl Config {
source: e,
})?;
serde_yaml::from_str(&contents).map_err(|e| ConfigError::ParseYaml {
path: path.to_path_buf(),
source: e,
})
let mut config: Config =
serde_yaml::from_str(&contents).map_err(|e| ConfigError::ParseYaml {
path: path.to_path_buf(),
source: e,
})?;
config.normalize_deprecated_keys();
Ok(config)
}
/// COMPAT (drop at the v2 cutover): fold a deprecated `node.discovery:`
/// block into the `node.lookup.*` (mesh-lookup scalars) and
/// `node.rendezvous.*` (nostr/LAN peer rendezvous) tables that replaced it.
///
/// Runs at every deserialize boundary (see `load_file`). A present legacy
/// field fills the corresponding new-table field, so a config that predates
/// the split keeps behaving identically. When a legacy block is seen, a
/// one-time deprecation warning names the old→new key moves. Exposed to the
/// crate so config tests that deserialize directly can invoke it.
pub(crate) fn normalize_deprecated_keys(&mut self) {
let Some(compat) = self.node.discovery.take() else {
return;
};
tracing::warn!(
target: "fips::config",
"`node.discovery.*` is deprecated and will be removed: mesh-lookup \
scalars moved to `node.lookup.*`, and peer-rendezvous keys moved to \
`node.rendezvous.nostr.*` / `node.rendezvous.lan.*`. Please migrate; \
a legacy `node.discovery` block still applies for now."
);
if let Some(v) = compat.ttl {
self.node.lookup.ttl = v;
}
if let Some(v) = compat.attempt_timeouts_secs {
self.node.lookup.attempt_timeouts_secs = v;
}
if let Some(v) = compat.recent_expiry_secs {
self.node.lookup.recent_expiry_secs = v;
}
if let Some(v) = compat.backoff_base_secs {
self.node.lookup.backoff_base_secs = v;
}
if let Some(v) = compat.backoff_max_secs {
self.node.lookup.backoff_max_secs = v;
}
if let Some(v) = compat.forward_min_interval_secs {
self.node.lookup.forward_min_interval_secs = v;
}
if let Some(v) = compat.nostr {
self.node.rendezvous.nostr = v;
}
if let Some(v) = compat.lan {
self.node.rendezvous.lan = v;
}
}
/// Get the standard search paths in priority order (lowest to highest).
@@ -589,6 +638,21 @@ impl Config {
self.node.leaf_only
}
/// Derive the node profile from config.
///
/// leaf_only → Leaf (implies non-routing),
/// disable_routing → NonRouting,
/// otherwise → Full.
pub fn node_profile(&self) -> crate::proto::fmp::NodeProfile {
if self.node.leaf_only {
crate::proto::fmp::NodeProfile::Leaf
} else if self.node.disable_routing {
crate::proto::fmp::NodeProfile::NonRouting
} else {
crate::proto::fmp::NodeProfile::Full
}
}
/// Get the configured peers.
pub fn peers(&self) -> &[PeerConfig] {
&self.peers
@@ -601,7 +665,7 @@ impl Config {
/// Validate cross-field configuration invariants.
pub fn validate(&self) -> Result<(), ConfigError> {
let nostr = &self.node.discovery.nostr;
let nostr = &self.node.rendezvous.nostr;
let any_transport_advertises_on_nostr = self
.transports
@@ -621,13 +685,13 @@ impl Config {
if any_transport_advertises_on_nostr && !nostr.enabled {
return Err(ConfigError::Validation(
"at least one transport has `advertise_on_nostr = true`, but `node.discovery.nostr.enabled` is false".to_string(),
"at least one transport has `advertise_on_nostr = true`, but `node.rendezvous.nostr.enabled` is false".to_string(),
));
}
if self.peers.iter().any(|peer| peer.via_nostr) && !nostr.enabled {
return Err(ConfigError::Validation(
"at least one peer has `via_nostr = true`, but `node.discovery.nostr.enabled` is false".to_string(),
"at least one peer has `via_nostr = true`, but `node.rendezvous.nostr.enabled` is false".to_string(),
));
}
@@ -649,12 +713,12 @@ impl Config {
if nostr.enabled && has_nat_udp_advert {
if nostr.dm_relays.is_empty() {
return Err(ConfigError::Validation(
"NAT UDP advert publishing requires `node.discovery.nostr.dm_relays` to be non-empty".to_string(),
"NAT UDP advert publishing requires `node.rendezvous.nostr.dm_relays` to be non-empty".to_string(),
));
}
if nostr.stun_servers.is_empty() {
return Err(ConfigError::Validation(
"NAT UDP advert publishing requires `node.discovery.nostr.stun_servers` to be non-empty".to_string(),
"NAT UDP advert publishing requires `node.rendezvous.nostr.stun_servers` to be non-empty".to_string(),
));
}
}
@@ -748,6 +812,84 @@ node:
assert!(config.has_identity());
}
/// The fips.yaml shipped in the OpenWrt package must keep parsing as the
/// config schema evolves. Both the 802.11s mesh backhaul entries
/// (docs/how-to/set-up-80211s-mesh-backhaul.md) and the open-access SSID
/// entries (docs/how-to/set-up-open-access-ssid.md) ship commented out —
/// one per radio, so dual-band routers can run either on both bands — so
/// a stock install that never creates fips-mesh*/fips-ap* logs no
/// per-boot bind warning; `fips-mesh-setup`/`fips-ap-setup` uncomment the
/// matching block when they create the interface. Verify both states
/// parse: as shipped (both inactive), and after the uncomment the helpers
/// perform.
#[test]
fn shipped_openwrt_config_parses() {
let yaml = include_str!("../../packaging/openwrt-ipk/files/etc/fips/fips.yaml");
// As shipped: parses, and the mesh/ap entries are commented out (a
// running daemon binds no fips-mesh*/fips-ap* transport, no warning).
let config: Config = serde_yaml::from_str(yaml).expect("shipped OpenWrt fips.yaml");
for name in ["mesh0", "mesh1", "ap0", "ap1"] {
assert!(
!config
.transports
.ethernet
.iter()
.any(|(n, _)| n == Some(name)),
"{name} must ship commented out, not active, in fips.yaml"
);
}
// What `fips-mesh-setup`/`fips-ap-setup` produce: uncomment each
// block, which must still parse into a transport bound to the right
// netdev.
let uncommented =
uncomment_transport_blocks(&uncomment_transport_blocks(yaml, "mesh"), "ap");
let config: Config = serde_yaml::from_str(&uncommented)
.expect("fips.yaml with mesh and ap transports uncommented");
for (name, interface) in [
("mesh0", "fips-mesh0"),
("mesh1", "fips-mesh1"),
("ap0", "fips-ap0"),
("ap1", "fips-ap1"),
] {
assert!(
config
.transports
.ethernet
.iter()
.any(|(n, eth)| n == Some(name) && eth.interface == interface),
"{name} entry missing after uncommenting shipped fips.yaml"
);
}
}
/// Mirror the setup helpers' block uncomment: strip the ` # ` prefix
/// from each `# <prefix><N>:` header and its ` # ` continuation
/// lines, leaving every other comment untouched.
fn uncomment_transport_blocks(yaml: &str, prefix: &str) -> String {
let header = format!(" # {prefix}");
let mut out = String::new();
let mut in_block = false;
for line in yaml.lines() {
let is_header = line
.strip_prefix(&header)
.and_then(|r| r.strip_suffix(':'))
.is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()));
if is_header {
in_block = true;
out.push_str(&line.replacen(" # ", " ", 1));
} else if in_block && line.starts_with(" # ") {
out.push_str(&line.replacen(" # ", " ", 1));
} else {
in_block = false;
out.push_str(line);
}
out.push('\n');
}
out
}
#[test]
fn test_parse_yaml_with_hex() {
let yaml = r#"
@@ -1288,7 +1430,9 @@ peers:
}
#[test]
fn test_parse_nostr_discovery_config() {
fn test_parse_legacy_discovery_nostr_config_compat() {
// COMPAT (drop at the v2 cutover): a deprecated `node.discovery.nostr`
// block must fold into `node.rendezvous.nostr` via normalize.
let yaml = r#"
node:
discovery:
@@ -1312,26 +1456,27 @@ peers:
- transport: udp
addr: "nat"
"#;
let config: Config = serde_yaml::from_str(yaml).unwrap();
assert!(config.node.discovery.nostr.enabled);
assert!(!config.node.discovery.nostr.advertise);
assert_eq!(config.node.discovery.nostr.app, "fips.nat.test.v1");
assert_eq!(config.node.discovery.nostr.signal_ttl_secs, 45);
let mut config: Config = serde_yaml::from_str(yaml).unwrap();
config.normalize_deprecated_keys();
assert!(config.node.rendezvous.nostr.enabled);
assert!(!config.node.rendezvous.nostr.advertise);
assert_eq!(config.node.rendezvous.nostr.app, "fips.nat.test.v1");
assert_eq!(config.node.rendezvous.nostr.signal_ttl_secs, 45);
assert_eq!(
config.node.discovery.nostr.policy,
NostrDiscoveryPolicy::ConfiguredOnly
config.node.rendezvous.nostr.policy,
NostrRendezvousPolicy::ConfiguredOnly
);
assert_eq!(config.node.discovery.nostr.open_discovery_max_pending, 12);
assert_eq!(config.node.rendezvous.nostr.open_discovery_max_pending, 12);
assert_eq!(
config.node.discovery.nostr.advert_relays,
config.node.rendezvous.nostr.advert_relays,
vec!["wss://relay-a.example".to_string()]
);
assert_eq!(
config.node.discovery.nostr.dm_relays,
config.node.rendezvous.nostr.dm_relays,
vec!["wss://relay-b.example".to_string()]
);
assert_eq!(
config.node.discovery.nostr.stun_servers,
config.node.rendezvous.nostr.stun_servers,
vec!["stun:stun.example.org:3478".to_string()]
);
assert_eq!(
@@ -1341,6 +1486,55 @@ peers:
assert!(config.peers[0].via_nostr);
}
#[test]
fn test_parse_lookup_and_rendezvous_new_keys() {
// The post-split keys parse directly, with no deprecated block and no
// normalize warning.
let yaml = r#"
node:
lookup:
ttl: 7
attempt_timeouts_secs: [3, 6]
forward_min_interval_secs: 9
rendezvous:
nostr:
enabled: true
app: "fips.new.keys.v1"
"#;
let mut config: Config = serde_yaml::from_str(yaml).unwrap();
config.normalize_deprecated_keys();
assert_eq!(config.node.lookup.ttl, 7);
assert_eq!(config.node.lookup.attempt_timeouts_secs, vec![3, 6]);
assert_eq!(config.node.lookup.forward_min_interval_secs, 9);
// Unset scalar keeps its default.
assert_eq!(config.node.lookup.recent_expiry_secs, 10);
assert!(config.node.rendezvous.nostr.enabled);
assert_eq!(config.node.rendezvous.nostr.app, "fips.new.keys.v1");
assert!(config.node.discovery.is_none());
}
#[test]
fn test_legacy_discovery_lookup_scalars_compat() {
// COMPAT (drop at the v2 cutover): legacy `node.discovery` mesh-lookup
// scalars must fold into `node.lookup`; unset keys keep their defaults.
let yaml = r#"
node:
discovery:
ttl: 5
backoff_base_secs: 4
backoff_max_secs: 30
"#;
let mut config: Config = serde_yaml::from_str(yaml).unwrap();
config.normalize_deprecated_keys();
assert_eq!(config.node.lookup.ttl, 5);
assert_eq!(config.node.lookup.backoff_base_secs, 4);
assert_eq!(config.node.lookup.backoff_max_secs, 30);
// Unset legacy scalar leaves the new-table default intact.
assert_eq!(config.node.lookup.attempt_timeouts_secs, vec![1, 2, 4, 8]);
// The compat block is consumed by normalize.
assert!(config.node.discovery.is_none());
}
#[test]
fn test_validate_transport_advert_requires_nostr_enabled() {
let mut config = Config::default();
@@ -1348,13 +1542,14 @@ peers:
advertise_on_nostr: Some(true),
..Default::default()
});
config.node.discovery.nostr.enabled = false;
config.node.rendezvous.nostr.enabled = false;
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("advertise_on_nostr"));
}
#[test]
#[allow(clippy::field_reassign_with_default)]
fn test_validate_peer_via_nostr_requires_nostr_enabled() {
let mut config = Config {
peers: vec![PeerConfig {
@@ -1364,13 +1559,14 @@ peers:
}],
..Default::default()
};
config.node.discovery.nostr.enabled = false;
config.node.rendezvous.nostr.enabled = false;
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("via_nostr"));
}
#[test]
#[allow(clippy::field_reassign_with_default)]
fn test_validate_peer_addresses_required_unless_via_nostr() {
// Empty addresses + via_nostr=false → error.
let mut config = Config {
@@ -1385,7 +1581,7 @@ peers:
// Empty addresses + via_nostr=true + nostr.enabled=true → ok.
config.peers[0].via_nostr = true;
config.node.discovery.nostr.enabled = true;
config.node.rendezvous.nostr.enabled = true;
config
.validate()
.expect("via_nostr should allow empty addresses");
@@ -1394,8 +1590,8 @@ peers:
#[test]
fn test_validate_nat_udp_advert_requires_relays_and_stun() {
let mut config = Config::default();
config.node.discovery.nostr.enabled = true;
config.node.discovery.nostr.dm_relays.clear();
config.node.rendezvous.nostr.enabled = true;
config.node.rendezvous.nostr.dm_relays.clear();
config.transports.udp = TransportInstances::Single(UdpConfig {
advertise_on_nostr: Some(true),
public: Some(false),
@@ -1405,8 +1601,8 @@ peers:
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("dm_relays"));
config.node.discovery.nostr.dm_relays = vec!["wss://relay.example".to_string()];
config.node.discovery.nostr.stun_servers.clear();
config.node.rendezvous.nostr.dm_relays = vec!["wss://relay.example".to_string()];
config.node.rendezvous.nostr.stun_servers.clear();
let err = config.validate().expect_err("validation should fail");
assert!(err.to_string().contains("stun_servers"));
}
src/config/node.rs
+238 -76
View File
@@ -7,7 +7,7 @@
use serde::{Deserialize, Serialize};
use super::IdentityConfig;
use crate::mmp::{DEFAULT_LOG_INTERVAL_SECS, DEFAULT_OWD_WINDOW_SIZE, MmpConfig, MmpMode};
use crate::proto::mmp::{DEFAULT_LOG_INTERVAL_SECS, DEFAULT_OWD_WINDOW_SIZE, MmpMode};
// ============================================================================
// Node Configuration Subsections
@@ -186,48 +186,42 @@ impl CacheConfig {
}
}
/// Discovery protocol (`node.discovery.*`).
/// Mesh-lookup protocol (`node.lookup.*`): the overlay coordinate-lookup
/// engine (address → coordinates). The peer-rendezvous keys that used to
/// share this table (`nostr`/`lan`) now live under [`RendezvousConfig`]
/// (`node.rendezvous.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DiscoveryConfig {
/// Hop limit for LookupRequest flood (`node.discovery.ttl`).
#[serde(default = "DiscoveryConfig::default_ttl")]
pub struct LookupConfig {
/// Hop limit for LookupRequest flood (`node.lookup.ttl`).
#[serde(default = "LookupConfig::default_ttl")]
pub ttl: u8,
/// Per-attempt timeouts in seconds (`node.discovery.attempt_timeouts_secs`).
/// Per-attempt timeouts in seconds (`node.lookup.attempt_timeouts_secs`).
/// Each entry is the time to wait for a response before sending the next
/// LookupRequest (with a fresh request_id). Sequence length determines the
/// total number of attempts before declaring the destination unreachable.
/// Default `[1, 2, 4, 8]` gives 4 attempts and a 15s total budget.
#[serde(default = "DiscoveryConfig::default_attempt_timeouts_secs")]
#[serde(default = "LookupConfig::default_attempt_timeouts_secs")]
pub attempt_timeouts_secs: Vec<u64>,
/// Dedup cache expiry in seconds (`node.discovery.recent_expiry_secs`).
#[serde(default = "DiscoveryConfig::default_recent_expiry_secs")]
/// Dedup cache expiry in seconds (`node.lookup.recent_expiry_secs`).
#[serde(default = "LookupConfig::default_recent_expiry_secs")]
pub recent_expiry_secs: u64,
/// Base backoff after lookup failure in seconds (`node.discovery.backoff_base_secs`).
/// Base backoff after lookup failure in seconds (`node.lookup.backoff_base_secs`).
/// Doubles per consecutive failure up to `backoff_max_secs`. Defaults to 0
/// (no post-failure suppression); the per-attempt sequence in
/// `attempt_timeouts_secs` provides the only retry pacing.
#[serde(default = "DiscoveryConfig::default_backoff_base_secs")]
#[serde(default = "LookupConfig::default_backoff_base_secs")]
pub backoff_base_secs: u64,
/// Maximum backoff cap in seconds (`node.discovery.backoff_max_secs`).
#[serde(default = "DiscoveryConfig::default_backoff_max_secs")]
/// Maximum backoff cap in seconds (`node.lookup.backoff_max_secs`).
#[serde(default = "LookupConfig::default_backoff_max_secs")]
pub backoff_max_secs: u64,
/// Minimum interval between forwarded lookups for the same target in seconds
/// (`node.discovery.forward_min_interval_secs`).
/// (`node.lookup.forward_min_interval_secs`).
/// Defense-in-depth against misbehaving nodes.
#[serde(default = "DiscoveryConfig::default_forward_min_interval_secs")]
#[serde(default = "LookupConfig::default_forward_min_interval_secs")]
pub forward_min_interval_secs: u64,
/// Nostr-mediated overlay endpoint discovery.
#[serde(default = "DiscoveryConfig::default_nostr")]
pub nostr: NostrDiscoveryConfig,
/// mDNS / DNS-SD peer discovery on the local link. Identity surface
/// is a strict subset of what `nostr.advertise` already publishes
/// publicly, so there's no marginal privacy cost; the latency win
/// for same-LAN peers is large (sub-second pairing, no relay).
#[serde(default = "DiscoveryConfig::default_lan")]
pub lan: crate::discovery::lan::LanDiscoveryConfig,
}
impl Default for DiscoveryConfig {
impl Default for LookupConfig {
fn default() -> Self {
Self {
ttl: 64,
@@ -236,13 +230,11 @@ impl Default for DiscoveryConfig {
backoff_base_secs: 0,
backoff_max_secs: 0,
forward_min_interval_secs: 2,
nostr: NostrDiscoveryConfig::default(),
lan: crate::discovery::lan::LanDiscoveryConfig::default(),
}
}
}
impl DiscoveryConfig {
impl LookupConfig {
fn default_ttl() -> u8 {
64
}
@@ -261,12 +253,45 @@ impl DiscoveryConfig {
fn default_forward_min_interval_secs() -> u64 {
2
}
fn default_nostr() -> NostrDiscoveryConfig {
NostrDiscoveryConfig::default()
}
fn default_lan() -> crate::discovery::lan::LanDiscoveryConfig {
crate::discovery::lan::LanDiscoveryConfig::default()
}
}
/// Peer rendezvous (`node.rendezvous.*`): how the node finds peers to connect
/// to at all — Nostr-mediated overlay endpoints and mDNS/DNS-SD on the local
/// link. Distinct from mesh lookup ([`LookupConfig`]), which finds coordinates
/// for an already-known mesh address.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct RendezvousConfig {
/// Nostr-mediated overlay endpoint rendezvous (`node.rendezvous.nostr.*`).
#[serde(default)]
pub nostr: NostrRendezvousConfig,
/// mDNS / DNS-SD peer rendezvous on the local link (`node.rendezvous.lan.*`).
/// Identity surface is a strict subset of what `nostr.advertise` already
/// publishes publicly, so there's no marginal privacy cost; the latency
/// win for same-LAN peers is large (sub-second pairing, no relay).
#[serde(default)]
pub lan: crate::mdns::LanRendezvousConfig,
}
/// COMPAT (drop at the v2 cutover): a deprecated legacy `node.discovery:` block.
///
/// The `node.discovery.*` table was split into `node.lookup.*` (mesh-lookup
/// scalars) and `node.rendezvous.*` (nostr/LAN peer rendezvous). Because
/// `NodeConfig` does not deny unknown fields, a still-deployed `node.discovery:`
/// block would otherwise deserialize into nothing and silently revert every
/// lookup/rendezvous setting to its default. This all-`Option` mirror captures
/// it so [`Config::normalize_deprecated_keys`] can fold it into the new tables
/// with a one-time deprecation warning; unset legacy keys stay `None` and leave
/// the new-table defaults intact.
#[derive(Debug, Clone, Deserialize)]
pub(crate) struct DiscoveryConfigCompat {
pub ttl: Option<u8>,
pub attempt_timeouts_secs: Option<Vec<u64>>,
pub recent_expiry_secs: Option<u64>,
pub backoff_base_secs: Option<u64>,
pub backoff_max_secs: Option<u64>,
pub forward_min_interval_secs: Option<u64>,
pub nostr: Option<NostrRendezvousConfig>,
pub lan: Option<crate::mdns::LanRendezvousConfig>,
}
/// Nostr advert discovery policy.
@@ -278,33 +303,33 @@ impl DiscoveryConfig {
/// - `open`: also consider adverts for non-configured peers
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum NostrDiscoveryPolicy {
pub enum NostrRendezvousPolicy {
Disabled,
#[default]
ConfiguredOnly,
Open,
}
/// Nostr-mediated overlay endpoint discovery (`node.discovery.nostr.*`).
/// Nostr-mediated overlay endpoint discovery (`node.rendezvous.nostr.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct NostrDiscoveryConfig {
pub struct NostrRendezvousConfig {
/// Enable Nostr-signaled traversal bootstrap.
#[serde(default)]
pub enabled: bool,
/// Publish service advertisements so remote peers can bootstrap inbound.
#[serde(default = "NostrDiscoveryConfig::default_advertise")]
#[serde(default = "NostrRendezvousConfig::default_advertise")]
pub advertise: bool,
/// Relay URLs used for service advertisements.
#[serde(default = "NostrDiscoveryConfig::default_advert_relays")]
#[serde(default = "NostrRendezvousConfig::default_advert_relays")]
pub advert_relays: Vec<String>,
/// Relay URLs used for encrypted signaling events.
#[serde(default = "NostrDiscoveryConfig::default_dm_relays")]
#[serde(default = "NostrRendezvousConfig::default_dm_relays")]
pub dm_relays: Vec<String>,
/// STUN servers used for local reflexive address discovery.
/// Outbound observation uses only this local list; peer-advertised STUN
/// values are informational and are not treated as egress targets.
#[serde(default = "NostrDiscoveryConfig::default_stun_servers")]
#[serde(default = "NostrRendezvousConfig::default_stun_servers")]
pub stun_servers: Vec<String>,
/// Whether to advertise local (RFC 1918 / ULA) interface addresses as
/// host candidates in the traversal offer.
@@ -318,85 +343,85 @@ pub struct NostrDiscoveryConfig {
#[serde(default)]
pub share_local_candidates: bool,
/// Traversal application namespace and advert identifier suffix.
#[serde(default = "NostrDiscoveryConfig::default_app")]
#[serde(default = "NostrRendezvousConfig::default_app")]
pub app: String,
/// Signaling TTL in seconds.
#[serde(default = "NostrDiscoveryConfig::default_signal_ttl_secs")]
#[serde(default = "NostrRendezvousConfig::default_signal_ttl_secs")]
pub signal_ttl_secs: u64,
/// Policy for advert-derived endpoint discovery.
#[serde(default)]
pub policy: NostrDiscoveryPolicy,
pub policy: NostrRendezvousPolicy,
/// Max number of open-discovery peers queued for outbound retry/connection
/// at once. Prevents unbounded queue growth from ambient advert traffic.
#[serde(default = "NostrDiscoveryConfig::default_open_discovery_max_pending")]
#[serde(default = "NostrRendezvousConfig::default_open_discovery_max_pending")]
pub open_discovery_max_pending: usize,
/// Max concurrent inbound traversal offers processed at once.
/// Acts as a rate limit against offer spam from relays.
#[serde(default = "NostrDiscoveryConfig::default_max_concurrent_incoming_offers")]
#[serde(default = "NostrRendezvousConfig::default_max_concurrent_incoming_offers")]
pub max_concurrent_incoming_offers: usize,
/// Max cached overlay adverts retained from relay traffic.
/// Bounds memory under ambient advert volume.
#[serde(default = "NostrDiscoveryConfig::default_advert_cache_max_entries")]
#[serde(default = "NostrRendezvousConfig::default_advert_cache_max_entries")]
pub advert_cache_max_entries: usize,
/// Max seen-session IDs retained for replay detection.
/// Oldest entries are evicted when the cap is exceeded.
#[serde(default = "NostrDiscoveryConfig::default_seen_sessions_max_entries")]
#[serde(default = "NostrRendezvousConfig::default_seen_sessions_max_entries")]
pub seen_sessions_max_entries: usize,
/// Overall punch attempt timeout in seconds.
#[serde(default = "NostrDiscoveryConfig::default_attempt_timeout_secs")]
#[serde(default = "NostrRendezvousConfig::default_attempt_timeout_secs")]
pub attempt_timeout_secs: u64,
/// Replay tracking retention window in seconds.
#[serde(default = "NostrDiscoveryConfig::default_replay_window_secs")]
#[serde(default = "NostrRendezvousConfig::default_replay_window_secs")]
pub replay_window_secs: u64,
/// Delay before punch traffic starts.
#[serde(default = "NostrDiscoveryConfig::default_punch_start_delay_ms")]
#[serde(default = "NostrRendezvousConfig::default_punch_start_delay_ms")]
pub punch_start_delay_ms: u64,
/// Interval between punch packets.
#[serde(default = "NostrDiscoveryConfig::default_punch_interval_ms")]
#[serde(default = "NostrRendezvousConfig::default_punch_interval_ms")]
pub punch_interval_ms: u64,
/// How long to keep punching before failure.
#[serde(default = "NostrDiscoveryConfig::default_punch_duration_ms")]
#[serde(default = "NostrRendezvousConfig::default_punch_duration_ms")]
pub punch_duration_ms: u64,
/// Advert TTL in seconds.
#[serde(default = "NostrDiscoveryConfig::default_advert_ttl_secs")]
#[serde(default = "NostrRendezvousConfig::default_advert_ttl_secs")]
pub advert_ttl_secs: u64,
/// How often adverts are refreshed in seconds.
#[serde(default = "NostrDiscoveryConfig::default_advert_refresh_secs")]
#[serde(default = "NostrRendezvousConfig::default_advert_refresh_secs")]
pub advert_refresh_secs: u64,
/// Settle delay in seconds after Nostr discovery starts before the
/// one-shot startup sweep of cached adverts runs. Allows the relay
/// subscription backlog to populate the in-memory advert cache.
/// Only used under `policy: open`. Default: 5.
#[serde(default = "NostrDiscoveryConfig::default_startup_sweep_delay_secs")]
#[serde(default = "NostrRendezvousConfig::default_startup_sweep_delay_secs")]
pub startup_sweep_delay_secs: u64,
/// Maximum age in seconds for cached adverts considered by the
/// one-shot startup sweep. Adverts whose `created_at` is older than
/// `now - startup_sweep_max_age_secs` are skipped. Only used under
/// `policy: open`. Default: 3600 (1 hour).
#[serde(default = "NostrDiscoveryConfig::default_startup_sweep_max_age_secs")]
#[serde(default = "NostrRendezvousConfig::default_startup_sweep_max_age_secs")]
pub startup_sweep_max_age_secs: u64,
/// Number of consecutive NAT-traversal failures against a peer before
/// an extended cooldown is applied to throttle further offer publishes.
/// At this threshold the daemon also actively re-fetches the peer's
/// advert from `advert_relays` to evict cache entries for peers that
/// have gone away. Default: 5.
#[serde(default = "NostrDiscoveryConfig::default_failure_streak_threshold")]
#[serde(default = "NostrRendezvousConfig::default_failure_streak_threshold")]
pub failure_streak_threshold: u32,
/// Cooldown applied to a peer once `failure_streak_threshold` is hit.
/// Suppresses both open-discovery sweep enqueues and per-attempt
/// retry firings until elapsed. Default: 1800 (30 minutes).
#[serde(default = "NostrDiscoveryConfig::default_extended_cooldown_secs")]
#[serde(default = "NostrRendezvousConfig::default_extended_cooldown_secs")]
pub extended_cooldown_secs: u64,
/// Minimum interval between `NAT traversal failed` WARN log lines for
/// the same peer. Subsequent failures inside the window log at DEBUG.
/// Reduces log spam on public-test nodes with many cache-learned
/// peers. Default: 300 (5 minutes).
#[serde(default = "NostrDiscoveryConfig::default_warn_log_interval_secs")]
#[serde(default = "NostrRendezvousConfig::default_warn_log_interval_secs")]
pub warn_log_interval_secs: u64,
/// Maximum entries retained in the per-npub failure-state map.
/// Bounds memory under high cache turnover. Oldest entries (by last
/// failure time) evicted when the cap is exceeded. Default: 4096.
#[serde(default = "NostrDiscoveryConfig::default_failure_state_max_entries")]
#[serde(default = "NostrRendezvousConfig::default_failure_state_max_entries")]
pub failure_state_max_entries: usize,
/// Cooldown applied after observing a fatal protocol mismatch on a
/// Nostr-adopted bootstrap transport (e.g. `Unknown FMP version`
@@ -404,11 +429,11 @@ pub struct NostrDiscoveryConfig {
/// of `extended_cooldown_secs` and much longer because the mismatch
/// is structural — re-traversing the peer is wasted effort until one
/// side upgrades. Default: 86400 (24 hours).
#[serde(default = "NostrDiscoveryConfig::default_protocol_mismatch_cooldown_secs")]
#[serde(default = "NostrRendezvousConfig::default_protocol_mismatch_cooldown_secs")]
pub protocol_mismatch_cooldown_secs: u64,
}
impl Default for NostrDiscoveryConfig {
impl Default for NostrRendezvousConfig {
fn default() -> Self {
Self {
enabled: false,
@@ -419,7 +444,7 @@ impl Default for NostrDiscoveryConfig {
share_local_candidates: false,
app: Self::default_app(),
signal_ttl_secs: Self::default_signal_ttl_secs(),
policy: NostrDiscoveryPolicy::default(),
policy: NostrRendezvousPolicy::default(),
open_discovery_max_pending: Self::default_open_discovery_max_pending(),
max_concurrent_incoming_offers: Self::default_max_concurrent_incoming_offers(),
advert_cache_max_entries: Self::default_advert_cache_max_entries(),
@@ -442,7 +467,7 @@ impl Default for NostrDiscoveryConfig {
}
}
impl NostrDiscoveryConfig {
impl NostrRendezvousConfig {
fn default_advertise() -> bool {
true
}
@@ -472,7 +497,11 @@ impl NostrDiscoveryConfig {
}
fn default_app() -> String {
"fips-overlay-v1".to_string()
// Branch-specific default. `next` runs FMP-v1 which is wire-
// incompatible with `master`'s FMP-v0, so the two namespaces
// separate the discovery overlays by default — operators who
// want cross-branch discovery can override here.
"fips-overlay-v1-next".to_string()
}
fn default_signal_ttl_secs() -> u64 {
@@ -726,6 +755,41 @@ impl SessionConfig {
}
}
/// MMP configuration (`node.mmp.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MmpConfig {
/// Operating mode (`node.mmp.mode`).
#[serde(default)]
pub mode: MmpMode,
/// Periodic operator log interval in seconds (`node.mmp.log_interval_secs`).
#[serde(default = "MmpConfig::default_log_interval_secs")]
pub log_interval_secs: u64,
/// OWD trend ring buffer size (`node.mmp.owd_window_size`).
#[serde(default = "MmpConfig::default_owd_window_size")]
pub owd_window_size: usize,
}
impl Default for MmpConfig {
fn default() -> Self {
Self {
mode: MmpMode::default(),
log_interval_secs: DEFAULT_LOG_INTERVAL_SECS,
owd_window_size: DEFAULT_OWD_WINDOW_SIZE,
}
}
}
impl MmpConfig {
fn default_log_interval_secs() -> u64 {
DEFAULT_LOG_INTERVAL_SECS
}
fn default_owd_window_size() -> usize {
DEFAULT_OWD_WINDOW_SIZE
}
}
/// Session-layer Metrics Measurement Protocol (`node.session_mmp.*`).
///
/// Separate from link-layer `node.mmp.*` to allow independent mode/interval
@@ -949,7 +1013,18 @@ pub struct NodeConfig {
#[serde(default)]
pub identity: IdentityConfig,
/// Non-routing mode (`node.disable_routing`).
///
/// Tree participation and one-way bloom receipt, but no transit
/// forwarding or bloom combination/propagation. Overridden by
/// `leaf_only` (leaf implies non-routing).
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub disable_routing: bool,
/// Leaf-only mode (`node.leaf_only`).
///
/// Single upstream peer, no tree/bloom/transit. Implies
/// `disable_routing`.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub leaf_only: bool,
@@ -970,6 +1045,19 @@ pub struct NodeConfig {
#[serde(default = "NodeConfig::default_link_dead_timeout_secs")]
pub link_dead_timeout_secs: u64,
/// Graceful-shutdown drain deadline in seconds (`node.drain_timeout_secs`).
/// The bounded `Draining` phase broadcasts a shutdown `Disconnect` and then
/// waits up to this long for peers to clear before tearing down, early-
/// exiting as soon as all peers are gone. `None` selects the 2-second
/// default (see [`NodeConfig::drain_timeout`]).
///
/// Kept `Option` deliberately: `NodeConfig` has no `deny_unknown_fields`, so
/// a naive non-`Option` add with a `default` fn would silently rewrite the
/// value into deployed configs on the next serialize. The `Option` +
/// `skip_serializing_if` keeps absent configs absent.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub drain_timeout_secs: Option<u64>,
/// Resource limits (`node.limits.*`).
#[serde(default)]
pub limits: LimitsConfig,
@@ -986,9 +1074,19 @@ pub struct NodeConfig {
#[serde(default)]
pub cache: CacheConfig,
/// Discovery protocol (`node.discovery.*`).
/// Mesh-lookup protocol (`node.lookup.*`).
#[serde(default)]
pub discovery: DiscoveryConfig,
pub lookup: LookupConfig,
/// Peer rendezvous (`node.rendezvous.*`).
#[serde(default)]
pub rendezvous: RendezvousConfig,
/// COMPAT (drop at the v2 cutover): a deprecated legacy `node.discovery:`
/// block, folded into `lookup`/`rendezvous` by
/// [`Config::normalize_deprecated_keys`]. Never re-serialized.
#[serde(default, skip_serializing)]
pub(crate) discovery: Option<DiscoveryConfigCompat>,
/// Spanning tree (`node.tree.*`).
#[serde(default)]
@@ -1036,16 +1134,20 @@ impl Default for NodeConfig {
fn default() -> Self {
Self {
identity: IdentityConfig::default(),
disable_routing: false,
leaf_only: false,
tick_interval_secs: 1,
base_rtt_ms: 100,
heartbeat_interval_secs: 10,
link_dead_timeout_secs: 30,
drain_timeout_secs: None,
limits: LimitsConfig::default(),
rate_limit: RateLimitConfig::default(),
retry: RetryConfig::default(),
cache: CacheConfig::default(),
discovery: DiscoveryConfig::default(),
lookup: LookupConfig::default(),
rendezvous: RendezvousConfig::default(),
discovery: None,
tree: TreeConfig::default(),
bloom: BloomConfig::default(),
session: SessionConfig::default(),
@@ -1089,12 +1191,72 @@ impl NodeConfig {
fn default_link_dead_timeout_secs() -> u64 {
30
}
/// Graceful-shutdown drain deadline as a `Duration`.
///
/// Returns the configured `drain_timeout_secs`, or the 2-second default
/// when unset. Used by the daemon's bounded `Draining` phase.
pub fn drain_timeout(&self) -> std::time::Duration {
std::time::Duration::from_secs(self.drain_timeout_secs.unwrap_or(2))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_config_default() {
let config = MmpConfig::default();
assert_eq!(config.mode, MmpMode::Full);
assert_eq!(config.log_interval_secs, 30);
assert_eq!(config.owd_window_size, 32);
}
#[test]
fn test_config_yaml_parse() {
let yaml = r#"
mode: lightweight
log_interval_secs: 60
owd_window_size: 48
"#;
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Lightweight);
assert_eq!(config.log_interval_secs, 60);
assert_eq!(config.owd_window_size, 48);
}
#[test]
fn test_config_yaml_partial() {
let yaml = "mode: minimal";
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Minimal);
assert_eq!(config.log_interval_secs, DEFAULT_LOG_INTERVAL_SECS);
assert_eq!(config.owd_window_size, DEFAULT_OWD_WINDOW_SIZE);
}
#[test]
fn test_drain_timeout_default_and_override() {
// Unset → the 2-second default.
let c = NodeConfig::default();
assert_eq!(c.drain_timeout_secs, None);
assert_eq!(c.drain_timeout(), std::time::Duration::from_secs(2));
// Explicit override is honored.
let c2 = NodeConfig {
drain_timeout_secs: Some(10),
..NodeConfig::default()
};
assert_eq!(c2.drain_timeout(), std::time::Duration::from_secs(10));
// A zero override is a valid (immediate) drain, not the default.
let c3 = NodeConfig {
drain_timeout_secs: Some(0),
..NodeConfig::default()
};
assert_eq!(c3.drain_timeout(), std::time::Duration::from_secs(0));
}
#[test]
fn test_ecn_config_defaults() {
let c = EcnConfig::default();
@@ -1123,27 +1285,27 @@ mod tests {
}
#[test]
fn test_nostr_discovery_startup_sweep_defaults() {
let c = NostrDiscoveryConfig::default();
fn test_nostr_rendezvous_startup_sweep_defaults() {
let c = NostrRendezvousConfig::default();
assert_eq!(c.startup_sweep_delay_secs, 5);
assert_eq!(c.startup_sweep_max_age_secs, 3_600);
}
#[test]
fn test_nostr_discovery_startup_sweep_yaml_override() {
fn test_nostr_rendezvous_startup_sweep_yaml_override() {
let yaml = "enabled: true\npolicy: open\nstartup_sweep_delay_secs: 10\nstartup_sweep_max_age_secs: 1800\n";
let c: NostrDiscoveryConfig = serde_yaml::from_str(yaml).unwrap();
let c: NostrRendezvousConfig = serde_yaml::from_str(yaml).unwrap();
assert!(c.enabled);
assert_eq!(c.policy, NostrDiscoveryPolicy::Open);
assert_eq!(c.policy, NostrRendezvousPolicy::Open);
assert_eq!(c.startup_sweep_delay_secs, 10);
assert_eq!(c.startup_sweep_max_age_secs, 1_800);
}
#[test]
fn test_nostr_discovery_startup_sweep_partial_yaml_uses_defaults() {
fn test_nostr_rendezvous_startup_sweep_partial_yaml_uses_defaults() {
// Only override delay; max_age should fall back to default.
let yaml = "enabled: true\nstartup_sweep_delay_secs: 30\n";
let c: NostrDiscoveryConfig = serde_yaml::from_str(yaml).unwrap();
let c: NostrRendezvousConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(c.startup_sweep_delay_secs, 30);
assert_eq!(c.startup_sweep_max_age_secs, 3_600);
}
src/config/transport.rs
+25 -6
View File
@@ -282,9 +282,10 @@ pub struct EthernetConfig {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub send_buf_size: Option<usize>,
/// Listen for discovery beacons from other nodes. Default: true.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub discovery: Option<bool>,
/// Listen for neighbor beacons from other nodes. Default: true.
/// (Renamed from `discovery`; the old key is still accepted.)
#[serde(default, alias = "discovery", skip_serializing_if = "Option::is_none")]
pub listen: Option<bool>,
/// Broadcast announcement beacons on the LAN. Default: false.
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -319,9 +320,9 @@ impl EthernetConfig {
self.send_buf_size.unwrap_or(DEFAULT_ETHERNET_SEND_BUF)
}
/// Whether to listen for discovery beacons. Default: true.
pub fn discovery(&self) -> bool {
self.discovery.unwrap_or(true)
/// Whether to listen for neighbor beacons. Default: true.
pub fn listen(&self) -> bool {
self.listen.unwrap_or(true)
}
/// Whether to broadcast announcement beacons. Default: false.
@@ -1046,4 +1047,22 @@ mod tests {
assert_eq!(parse_bind_port("[::]:443"), Some(443));
assert_eq!(parse_bind_port("not-a-socket-addr"), None);
}
#[test]
fn ethernet_listen_accepts_legacy_discovery_alias_and_rejects_unknown() {
// (a) The legacy `discovery:` key is still accepted via serde alias.
let legacy: EthernetConfig =
serde_yaml::from_str("interface: eth0\ndiscovery: true\n").unwrap();
assert_eq!(legacy.listen, Some(true));
// (b) The new canonical `listen:` key parses into the renamed field.
let renamed: EthernetConfig =
serde_yaml::from_str("interface: eth0\nlisten: true\n").unwrap();
assert_eq!(renamed.listen, Some(true));
// (c) `deny_unknown_fields` still rejects an unknown ethernet key.
let bogus: Result<EthernetConfig, _> =
serde_yaml::from_str("interface: eth0\nbogus: true\n");
assert!(bogus.is_err());
}
}
src/control/queries.rs
+81 -59
View File
@@ -236,7 +236,7 @@ pub fn show_peers(node: &Node) -> Value {
// Per-npub Nostr-traversal failure-state snapshot, indexed by npub
// for O(1) per-peer lookup. Empty if Nostr discovery is disabled.
let nostr_state: std::collections::HashMap<String, _> = node
.nostr_discovery_handle()
.nostr_rendezvous_handle()
.map(|d| {
d.failure_state_snapshot()
.into_iter()
@@ -988,56 +988,70 @@ pub(crate) fn show_bloom_from_handle(handle: &super::read_handle::ControlReadHan
/// `show_mmp` — MMP metrics summary.
pub fn show_mmp(node: &Node) -> Value {
// Link-layer MMP per peer
let peers: Vec<Value> = node.peers().filter_map(|peer| {
let mmp = peer.mmp()?;
let addr = *peer.node_addr();
let metrics = &mmp.metrics;
let peers: Vec<Value> = node
.peers()
.filter_map(|peer| {
let mmp = peer.mmp()?;
let addr = *peer.node_addr();
let metrics = &mmp.metrics;
let mut link_layer = json!({
"loss_rate": metrics.loss_rate(),
"etx": metrics.etx,
"goodput_bps": metrics.goodput_bps,
"spin_bit_role": if mmp.spin_bit.is_initiator() { "initiator" } else { "responder" },
});
let mut link_layer = json!({
"loss_rate": metrics.loss_rate(),
"etx": metrics.etx,
"goodput_bps": metrics.goodput_bps,
});
if let Some(smoothed_loss) = metrics.smoothed_loss() {
link_layer["smoothed_loss"] = json!(smoothed_loss);
}
if let Some(smoothed_etx) = metrics.smoothed_etx() {
link_layer["smoothed_etx"] = json!(smoothed_etx);
}
if let Some(srtt) = metrics.srtt_ms() {
link_layer["srtt_ms"] = json!(srtt);
if let Some(setx) = metrics.smoothed_etx() {
link_layer["lqi"] = json!(setx * (1.0 + srtt / 100.0));
if let Some(smoothed_loss) = metrics.smoothed_loss() {
link_layer["smoothed_loss"] = json!(smoothed_loss);
}
if let Some(smoothed_etx) = metrics.smoothed_etx() {
link_layer["smoothed_etx"] = json!(smoothed_etx);
}
if let Some(srtt) = metrics.srtt_ms() {
link_layer["srtt_ms"] = json!(srtt);
if let Some(setx) = metrics.smoothed_etx() {
link_layer["lqi"] = json!(setx * (1.0 + srtt / 100.0));
}
}
}
// Trend indicators
if metrics.rtt_trend.initialized() {
link_layer["rtt_trend"] = json!(trend_label(metrics.rtt_trend.short(), metrics.rtt_trend.long()));
}
if metrics.loss_trend.initialized() {
link_layer["loss_trend"] = json!(trend_label(metrics.loss_trend.short(), metrics.loss_trend.long()));
}
if metrics.goodput_trend.initialized() {
link_layer["goodput_trend"] = json!(trend_label(metrics.goodput_trend.short(), metrics.goodput_trend.long()));
}
if metrics.jitter_trend.initialized() {
link_layer["jitter_trend"] = json!(trend_label(metrics.jitter_trend.short(), metrics.jitter_trend.long()));
}
// Trend indicators
if metrics.rtt_trend.initialized() {
link_layer["rtt_trend"] = json!(trend_label(
metrics.rtt_trend.short(),
metrics.rtt_trend.long()
));
}
if metrics.loss_trend.initialized() {
link_layer["loss_trend"] = json!(trend_label(
metrics.loss_trend.short(),
metrics.loss_trend.long()
));
}
if metrics.goodput_trend.initialized() {
link_layer["goodput_trend"] = json!(trend_label(
metrics.goodput_trend.short(),
metrics.goodput_trend.long()
));
}
if metrics.jitter_trend.initialized() {
link_layer["jitter_trend"] = json!(trend_label(
metrics.jitter_trend.short(),
metrics.jitter_trend.long()
));
}
link_layer["delivery_ratio_forward"] = json!(metrics.delivery_ratio_forward);
link_layer["delivery_ratio_reverse"] = json!(metrics.delivery_ratio_reverse);
link_layer["ecn_ce_count"] = json!(metrics.last_ecn_ce_count());
link_layer["delivery_ratio_forward"] = json!(metrics.delivery_ratio_forward);
link_layer["delivery_ratio_reverse"] = json!(metrics.delivery_ratio_reverse);
link_layer["ecn_ce_count"] = json!(metrics.last_ecn_ce_count());
Some(json!({
"peer": hex::encode(addr.as_bytes()),
"display_name": node.peer_display_name(&addr),
"mode": format!("{}", mmp.mode()),
"link_layer": link_layer,
}))
}).collect();
Some(json!({
"peer": hex::encode(addr.as_bytes()),
"display_name": node.peer_display_name(&addr),
"mode": format!("{}", mmp.mode()),
"link_layer": link_layer,
}))
})
.collect();
// Session-layer MMP
let sessions: Vec<Value> = node
@@ -1116,7 +1130,6 @@ pub(crate) fn show_mmp_from_handle(handle: &super::read_handle::ControlReadHandl
"loss_rate": peer.loss_rate,
"etx": peer.etx,
"goodput_bps": peer.goodput_bps,
"spin_bit_role": if peer.spin_bit_initiator { "initiator" } else { "responder" },
});
if let Some(smoothed_loss) = peer.smoothed_loss {
@@ -1303,17 +1316,18 @@ pub fn show_connections(node: &Node) -> Value {
let now = now_ms();
let connections: Vec<Value> = node
.connections()
.map(|conn| {
.map(|(_, machine)| {
let link_id = machine.link_id();
let mut conn_json = json!({
"link_id": conn.link_id().as_u64(),
"direction": format!("{}", conn.direction()),
"handshake_state": format!("{}", conn.handshake_state()),
"started_at_ms": conn.started_at(),
"idle_ms": now.saturating_sub(conn.last_activity()),
"resend_count": conn.resend_count(),
"link_id": link_id.as_u64(),
"direction": format!("{}", machine.conn_direction()),
"handshake_state": node.connection_handshake_state(link_id),
"started_at_ms": node.connection_started_at(link_id),
"idle_ms": now.saturating_sub(node.connection_last_activity(link_id)),
"resend_count": node.connection_resend_count(link_id),
});
if let Some(identity) = conn.expected_identity() {
if let Some(identity) = node.connection_expected_identity(link_id) {
conn_json["expected_peer"] = json!(identity.npub());
}
@@ -1487,7 +1501,9 @@ pub fn show_routing(node: &Node) -> Value {
"recent_requests": node.recent_request_count(),
"retries": retries,
"forwarding": serde_json::to_value(metrics.forwarding.snapshot()).unwrap_or_default(),
"discovery": serde_json::to_value(metrics.discovery.snapshot()).unwrap_or_default(),
// COMPAT: `discovery` is the deprecated alias for `lookup`; drop at the v2 cutover.
"discovery": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"lookup": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"error_signals": serde_json::to_value(metrics.errors.snapshot()).unwrap_or_default(),
"congestion": serde_json::to_value(metrics.congestion.snapshot()).unwrap_or_default(),
})
@@ -1543,7 +1559,9 @@ pub(crate) fn show_routing_from_handle(handle: &super::read_handle::ControlReadH
"recent_requests": view.recent_requests,
"retries": retries,
"forwarding": serde_json::to_value(metrics.forwarding.snapshot()).unwrap_or_default(),
"discovery": serde_json::to_value(metrics.discovery.snapshot()).unwrap_or_default(),
// COMPAT: `discovery` is the deprecated alias for `lookup`; drop at the v2 cutover.
"discovery": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"lookup": serde_json::to_value(metrics.lookup.snapshot()).unwrap_or_default(),
"error_signals": serde_json::to_value(metrics.errors.snapshot()).unwrap_or_default(),
"congestion": serde_json::to_value(metrics.congestion.snapshot()).unwrap_or_default(),
})
@@ -2328,7 +2346,9 @@ pub(crate) fn show_metrics_from_handle(handle: &super::read_handle::ControlReadH
let m = handle.metrics();
json!({
"forwarding": m.forwarding.snapshot(),
"discovery": m.discovery.snapshot(),
// COMPAT: `discovery` is the deprecated alias for `lookup`; drop at the v2 cutover.
"discovery": m.lookup.snapshot(),
"lookup": m.lookup.snapshot(),
"tree": m.tree.snapshot(),
"bloom": m.bloom.snapshot(),
"congestion": m.congestion.snapshot(),
@@ -2761,12 +2781,12 @@ mod tests {
/// Structural confirmation that the rx_loop no longer dispatches `show_*`:
/// the rx_loop source carries no `queries::dispatch` call and no
/// `starts_with("show_")` routing branch. Reads the committed source of
/// `src/node/handlers/rx_loop.rs` and asserts both markers are absent. This
/// `src/node/dataplane/rx_loop.rs` and asserts both markers are absent. This
/// is the milestone's "remove `show_*` from the data-plane dispatch path"
/// invariant, guarded against regression.
#[test]
fn rx_loop_has_no_show_dispatch() {
let src = include_str!("../node/handlers/rx_loop.rs");
let src = include_str!("../node/dataplane/rx_loop.rs");
assert!(
!src.contains("queries::dispatch"),
"rx_loop must not call queries::dispatch (show_* served off-loop)"
@@ -2794,7 +2814,9 @@ mod tests {
let expected_families = [
("forwarding", "received_packets"),
// `discovery` is the deprecated dual-emit alias for `lookup`; drop at the v2 cutover.
("discovery", "req_received"),
("lookup", "req_received"),
("tree", "accepted"),
("bloom", "accepted"),
("congestion", "ce_forwarded"),
src/control/read_handle.rs
+31
View File
@@ -118,10 +118,41 @@ impl ControlReadHandle {
/// Cutover queries (R1) read only `NodeContext` / `MetricsRegistry` (the state
/// the read handle already bundles) plus host-OS facts (`/proc`, nftables), so
/// they render entirely in the control task without touching `Node`.
///
/// **It now also carries mutating commands**, namely the `profile_tick_*`
/// family under the `profiling` feature. They are served here rather than on
/// the rx_loop deliberately: all of their state is process statics, they need
/// no `&mut Node`, and routing them through the loop would make the toggle
/// queue behind the very behavior it exists to measure.
pub(crate) fn snapshot_dispatch(request: &Request, handle: &ControlReadHandle) -> Option<Response> {
use crate::control::queries;
match request.command.as_str() {
// Tick-body profiler toggle. Present only in a `--features profiling`
// build; otherwise these fall through to the rx_loop dispatch, which
// reports them as unknown commands.
#[cfg(feature = "profiling")]
"profile_tick_on" => {
let dir = request
.params
.as_ref()
.and_then(|p| p.get("dir"))
.and_then(|v| v.as_str());
let context = handle.context();
let npub = context.identity.npub();
let period = context.config.node.tick_interval_secs;
Some(match crate::instr::capture::start(dir, &npub, period) {
Ok(value) => Response::ok(value),
Err(e) => Response::error(e),
})
}
#[cfg(feature = "profiling")]
"profile_tick_off" => Some(match crate::instr::capture::stop() {
Ok(value) => Response::ok(value),
Err(e) => Response::error(e),
}),
#[cfg(feature = "profiling")]
"profile_tick_status" => Some(Response::ok(crate::instr::capture::status())),
"show_listening_sockets" => Some(Response::ok(
queries::show_listening_sockets_from_handle(handle),
)),
src/control/snapshot.rs
-1
View File
@@ -692,7 +692,6 @@ pub(crate) struct MmpPeerRow {
pub loss_rate: f64,
pub etx: f64,
pub goodput_bps: f64,
pub spin_bit_initiator: bool,
pub smoothed_loss: Option<f64>,
pub smoothed_etx: Option<f64>,
pub srtt_ms: Option<f64>,
src/control/snapshots/show_bloom.json
+7
View File
@@ -10,13 +10,20 @@
"accepted": 0,
"debounce_suppressed": 0,
"decode_error": 0,
"deltas_sent": 0,
"fill_exceeded": 0,
"full_sends": 0,
"invalid": 0,
"nacks_received": 0,
"nacks_sent": 0,
"non_v1": 0,
"received": 0,
"send_failed": 0,
"sent": 0,
"size_changes": 0,
"stale": 0,
"total_compressed_bytes": 0,
"total_raw_bytes": 0,
"unknown_peer": 0
},
"uptree_estimated_count": null,
src/control/snapshots/show_routing.json
+24
View File
@@ -63,6 +63,30 @@
"ttl_exhausted_packets": 0
},
"identity_cache_entries": 0,
"lookup": {
"req_backoff_suppressed": 0,
"req_bloom_miss": 0,
"req_decode_error": 0,
"req_dedup_cache_full": 0,
"req_deduplicated": 0,
"req_duplicate": 0,
"req_fallback_forwarded": 0,
"req_forward_rate_limited": 0,
"req_forwarded": 0,
"req_initiated": 0,
"req_no_tree_peer": 0,
"req_received": 0,
"req_target_is_us": 0,
"req_ttl_exhausted": 0,
"resp_accepted": 0,
"resp_decode_error": 0,
"resp_forwarded": 0,
"resp_identity_miss": 0,
"resp_no_route": 0,
"resp_proof_failed": 0,
"resp_received": 0,
"resp_timed_out": 0
},
"pending_lookups": [],
"pending_tun_destinations": 0,
"pending_tun_packets": 0,
src/instr/capture.rs
+418
View File
@@ -0,0 +1,418 @@
//! Capture lifecycle: the arm/disarm state machine, the sink file, and the
//! `fipsctl`-facing operations.
//!
//! The toggle — not the writer — creates and opens the sink and publishes its
//! path, so an unwritable directory fails the `on` command loudly instead of
//! being discovered later by a background thread with nobody to report to.
//!
//! Capture state is a single atomic state machine (`Idle`, `Running`,
//! `StoppedByCap`) transitioned by `compare_exchange`. Every accepted control
//! connection is served by its own spawned task, so two simultaneous `on`
//! requests are genuinely concurrent and must not both create a writer.
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use std::sync::Mutex;
use std::sync::atomic::{AtomicBool, AtomicU8, AtomicU64, Ordering};
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use super::recorder;
use super::writer;
/// Default sink directory. Overridable per capture with `--dir`.
pub(crate) const DEFAULT_DIR: &str = "/var/log/fips";
/// Writer flush interval.
pub(crate) const INTERVAL: Duration = Duration::from_secs(10);
/// Size at which a capture stops itself. Reaching it stops the capture rather
/// than rotating: the point of a capture is a bounded, self-describing window.
pub(crate) const BYTE_CAP: u64 = 32 * 1024 * 1024;
pub(crate) const IDLE: u8 = 0;
pub(crate) const RUNNING: u8 = 1;
pub(crate) const STOPPED_BY_CAP: u8 = 2;
/// The writer could not write and stopped itself. Distinct from a cap stop:
/// a capture that died on a full disk produced a truncated window, and calling
/// that "stopped_by_cap" tells the operator it ran to its limit when it did
/// not. The trailer line explaining it goes to the same failing file, so the
/// state is the only signal that survives.
pub(crate) const STOPPED_BY_ERROR: u8 = 3;
static STATE: AtomicU8 = AtomicU8::new(IDLE);
static GATE: AtomicBool = AtomicBool::new(false);
static BYTES: AtomicU64 = AtomicU64::new(0);
static ACTIVE_PATH: Mutex<Option<PathBuf>> = Mutex::new(None);
static WRITER: Mutex<Option<writer::Handle>> = Mutex::new(None);
/// The per-tick gate. One relaxed load per tick when the feature is compiled in
/// and no capture is running.
#[inline]
pub(crate) fn gate() -> bool {
GATE.load(Ordering::Relaxed)
}
pub(crate) fn bytes_written() -> u64 {
BYTES.load(Ordering::Relaxed)
}
pub(crate) fn add_bytes(n: u64) -> u64 {
BYTES.fetch_add(n, Ordering::Relaxed) + n
}
fn active_path() -> Option<PathBuf> {
ACTIVE_PATH
.lock()
.unwrap_or_else(|e| e.into_inner())
.clone()
}
fn path_display() -> String {
active_path()
.map(|p| p.display().to_string())
.unwrap_or_else(|| "<none>".to_string())
}
fn state_name(state: u8) -> &'static str {
match state {
RUNNING => "running",
STOPPED_BY_CAP => "stopped_by_cap",
STOPPED_BY_ERROR => "stopped_by_error",
_ => "idle",
}
}
/// Called by the writer when it stops itself. `terminal` is `STOPPED_BY_CAP`
/// or `STOPPED_BY_ERROR`. Returns true if this call is the one that stopped it.
pub(crate) fn mark_stopped(terminal: u8) -> bool {
debug_assert!(terminal == STOPPED_BY_CAP || terminal == STOPPED_BY_ERROR);
GATE.store(false, Ordering::Relaxed);
STATE
.compare_exchange(RUNNING, terminal, Ordering::AcqRel, Ordering::Acquire)
.is_ok()
}
/// Join the writer thread, if one exists. Never called while holding another
/// lock the writer might want.
fn reap() {
let handle = WRITER.lock().unwrap_or_else(|e| e.into_inner()).take();
if let Some(handle) = handle {
handle.stop_and_join();
}
}
/// Arm a capture.
///
/// Opens the sink first and only then starts the writer, so a bad `--dir` is
/// reported to the caller rather than logged into the void.
pub(crate) fn start(
dir: Option<&str>,
node_npub: &str,
tick_period_secs: u64,
) -> Result<serde_json::Value, String> {
claim()?;
match open_sink(dir, node_npub, tick_period_secs) {
Ok((file, path, header_len)) => {
recorder::reset();
BYTES.store(header_len, Ordering::Relaxed);
match writer::spawn(file) {
Ok(handle) => {
*WRITER.lock().unwrap_or_else(|e| e.into_inner()) = Some(handle);
*ACTIVE_PATH.lock().unwrap_or_else(|e| e.into_inner()) = Some(path.clone());
GATE.store(true, Ordering::Release);
Ok(serde_json::json!({
"state": "running",
"path": path.display().to_string(),
"interval_secs": INTERVAL.as_secs(),
"byte_cap": BYTE_CAP,
}))
}
Err(e) => {
let _ = std::fs::remove_file(&path);
BYTES.store(0, Ordering::Relaxed);
STATE.store(IDLE, Ordering::Release);
Err(format!("cannot start profile writer thread: {e}"))
}
}
}
Err(e) => {
BYTES.store(0, Ordering::Relaxed);
STATE.store(IDLE, Ordering::Release);
Err(e)
}
}
}
/// Take the capture slot, reaping a cap-stopped predecessor if that is what is
/// in the way.
fn claim() -> Result<(), String> {
match STATE.compare_exchange(IDLE, RUNNING, Ordering::AcqRel, Ordering::Acquire) {
Ok(_) => Ok(()),
Err(RUNNING) => Err(format!("capture already running: {}", path_display())),
Err(stopped @ (STOPPED_BY_CAP | STOPPED_BY_ERROR)) => {
reap();
*ACTIVE_PATH.lock().unwrap_or_else(|e| e.into_inner()) = None;
STATE
.compare_exchange(stopped, RUNNING, Ordering::AcqRel, Ordering::Acquire)
.map(|_| ())
.map_err(|_| "capture state changed concurrently; retry".to_string())
}
Err(_) => Err("capture in an unexpected state".to_string()),
}
}
/// Disarm the capture. Succeeds when nothing is running, reporting so.
pub(crate) fn stop() -> Result<serde_json::Value, String> {
let previous = STATE.load(Ordering::Acquire);
if previous == IDLE {
return Ok(serde_json::json!({"state": "idle", "stopped": false}));
}
GATE.store(false, Ordering::Release);
// The writer wakes on the stop message rather than after the interval, so
// this join returns promptly instead of parking the caller for up to one
// flush interval.
reap();
let path = path_display();
*ACTIVE_PATH.lock().unwrap_or_else(|e| e.into_inner()) = None;
let bytes = bytes_written();
// Clear the counter with the slot: a later `status` while idle must not
// report the previous capture's byte total as though a capture were live.
BYTES.store(0, Ordering::Relaxed);
STATE.store(IDLE, Ordering::Release);
Ok(serde_json::json!({
"state": "idle",
"stopped": true,
"stopped_by_cap": previous == STOPPED_BY_CAP,
"stopped_by_error": previous == STOPPED_BY_ERROR,
"path": path,
"bytes": bytes,
}))
}
/// Report capture state. Distinguishes all four states.
pub(crate) fn status() -> serde_json::Value {
let state = STATE.load(Ordering::Acquire);
serde_json::json!({
"state": state_name(state),
"path": active_path().map(|p| p.display().to_string()),
"bytes": bytes_written(),
"byte_cap": BYTE_CAP,
"interval_secs": INTERVAL.as_secs(),
})
}
/// Stop and reap at daemon teardown. Idempotent.
pub(crate) fn shutdown() {
if STATE.load(Ordering::Acquire) != IDLE {
let _ = stop();
}
}
/// Create the sink file and write its header block. Returns the open file, its
/// path, and the number of header bytes written.
fn open_sink(
dir: Option<&str>,
node_npub: &str,
tick_period_secs: u64,
) -> Result<(File, PathBuf, u64), String> {
let dir = PathBuf::from(dir.unwrap_or(DEFAULT_DIR));
std::fs::create_dir_all(&dir)
.map_err(|e| format!("cannot use profile directory {}: {e}", dir.display()))?;
let start_unix = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let path = dir.join(format!("profile-{}.tsv", compact_utc(start_unix)));
let mut file = File::create(&path)
.map_err(|e| format!("cannot create profile file {}: {e}", path.display()))?;
let header = format!(
"# fips tick profile\n\
# node\t{node}\n\
# build\t{build}\n\
# platform\t{platform}\n\
# tick_period_secs\t{period}\n\
# interval_secs\t{interval}\n\
# byte_cap\t{cap}\n\
# start_utc\t{start_utc}\n\
# start_unix\t{start_unix}\n\
# NOTE\tstep durations are WALL CLOCK across await points, not CPU time:\n\
# NOTE\ta step that awaits I/O accrues the wait, and other tasks may run\n\
# NOTE\tinside that span. That is the intended measure for head-of-line\n\
# NOTE\tdelay; do not read a large step as CPU cost.\n\
# NOTE\tarm_starvation is measured directly as (entry time - the deadline\n\
# NOTE\tthe interval scheduled the tick for). It is NOT derived from\n\
# NOTE\ttick_entry_gap, which carries no starvation signal by itself:\n\
# NOTE\tunder a steady delay every gap is exactly one tick period.\n\
ts_unix\tkind\tdomain\tname\tcount\tmax\ttotal\tunit\n",
node = node_npub,
build = crate::version::short_version(),
platform = std::env::consts::OS,
period = tick_period_secs,
interval = INTERVAL.as_secs(),
cap = BYTE_CAP,
start_utc = iso_utc(start_unix),
start_unix = start_unix,
);
file.write_all(header.as_bytes())
.map_err(|e| format!("cannot write profile header to {}: {e}", path.display()))?;
Ok((file, path, header.len() as u64))
}
/// Break a Unix timestamp into UTC `(year, month, day, hour, minute, second)`.
///
/// Hinnant's `civil_from_days`, era-based. No date crate is in the dependency
/// set and one filename stamp does not justify adding one.
fn utc_parts(unix: u64) -> (i64, u32, u32, u32, u32, u32) {
let days = (unix / 86_400) as i64;
let secs = unix % 86_400;
let z = days + 719_468;
let era = z.div_euclid(146_097);
let doe = z.rem_euclid(146_097);
let yoe = (doe - doe / 1_460 + doe / 36_524 - doe / 146_096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = (doy - (153 * mp + 2) / 5 + 1) as u32;
let m = (if mp < 10 { mp + 3 } else { mp - 9 }) as u32;
let y = if m <= 2 { y + 1 } else { y };
(
y,
m,
d,
(secs / 3_600) as u32,
((secs % 3_600) / 60) as u32,
(secs % 60) as u32,
)
}
/// `20260727T191500Z` — filename-safe.
fn compact_utc(unix: u64) -> String {
let (y, mo, d, h, mi, s) = utc_parts(unix);
format!("{y:04}{mo:02}{d:02}T{h:02}{mi:02}{s:02}Z")
}
/// `2026-07-27T19:15:00Z` — for the header block.
fn iso_utc(unix: u64) -> String {
let (y, mo, d, h, mi, s) = utc_parts(unix);
format!("{y:04}-{mo:02}-{d:02}T{h:02}:{mi:02}:{s:02}Z")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn utc_parts_matches_known_instants() {
assert_eq!(utc_parts(0), (1970, 1, 1, 0, 0, 0));
assert_eq!(utc_parts(946_684_800), (2000, 1, 1, 0, 0, 0));
// 2026-07-27T19:15:00Z
assert_eq!(utc_parts(1_785_179_700), (2026, 7, 27, 19, 15, 0));
// Leap day.
assert_eq!(utc_parts(1_709_164_800), (2024, 2, 29, 0, 0, 0));
}
#[test]
fn stamps_render_expected_shapes() {
assert_eq!(compact_utc(1_785_179_700), "20260727T191500Z");
assert_eq!(iso_utc(1_785_179_700), "2026-07-27T19:15:00Z");
}
// The lock these tests take is shared with the recorder tests, which
// mutate the same statics. See `crate::instr::test_serial`.
#[test]
fn capture_round_trip_writes_header_and_rows() {
let _guard = crate::instr::test_serial();
let dir = tempfile::tempdir().expect("tempdir");
let dir_str = dir.path().to_str().unwrap().to_string();
let started = start(Some(&dir_str), "npub1test", 1).expect("start");
assert_eq!(started["state"], "running");
assert!(gate(), "gate must be armed while running");
let path = PathBuf::from(started["path"].as_str().unwrap());
// A second `on` is refused while one is running, and names the file.
let refused = start(Some(&dir_str), "npub1test", 1).unwrap_err();
assert!(refused.contains(&path.display().to_string()), "{refused}");
// Feed one observation so the drained rows are not all zero.
recorder::record(
recorder::Domain::Tick,
recorder::Step::WholeTick,
Duration::from_millis(7),
);
// Stopping wakes the writer immediately; it drains once more and joins.
let stopped = stop().expect("stop");
assert_eq!(stopped["stopped"], true);
assert_eq!(stopped["stopped_by_cap"], false);
assert!(!gate(), "gate must be clear after stop");
let text = std::fs::read_to_string(&path).expect("read capture");
assert!(text.starts_with("# fips tick profile\n"), "{text}");
assert!(text.contains("# node\tnpub1test\n"), "{text}");
assert!(
text.contains("ts_unix\tkind\tdomain\tname\tcount\tmax\ttotal\tunit\n"),
"{text}"
);
// The final drain emitted one row per emitted step, plus the gauges.
let rows: Vec<&str> = text
.lines()
.filter(|l| l.starts_with(|c: char| c.is_ascii_digit()))
.collect();
let expected_steps = recorder::STEPS.iter().filter(|s| s.emitted()).count();
assert_eq!(rows.len(), expected_steps + recorder::N_GAUGES);
// The 7 ms observation above is in the whole-tick row, converted to
// microseconds. Bounds rather than equality: the gate is process-wide,
// so a node under test elsewhere in this binary may have ticked into
// the same capture window.
let whole_tick = rows
.iter()
.find(|r| r.contains("\tstep\ttick\twhole_tick\t"))
.expect("whole_tick row");
let fields: Vec<&str> = whole_tick.split('\t').collect();
assert_eq!(fields.last(), Some(&"us"), "{whole_tick}");
assert!(
fields[4].parse::<u64>().unwrap() >= 1,
"count: {whole_tick}"
);
assert!(
fields[5].parse::<u64>().unwrap() >= 7_000,
"max: {whole_tick}"
);
assert!(
rows.iter()
.any(|r| r.contains("\tgauge\ttick\tarm_starvation\t")),
"{text}"
);
// A stop with nothing running is not an error.
let again = stop().expect("second stop");
assert_eq!(again["stopped"], false);
}
#[test]
fn start_fails_loudly_on_an_unwritable_directory() {
let _guard = crate::instr::test_serial();
let err = start(Some("/proc/fips-profile-should-not-exist"), "npub1test", 1)
.expect_err("must fail");
assert!(err.contains("profile directory"), "{err}");
// The failed attempt must leave the slot free for the next try.
assert_eq!(STATE.load(Ordering::Acquire), IDLE);
assert!(!gate());
}
#[test]
fn status_reports_the_bounds_it_is_enforcing() {
let _guard = crate::instr::test_serial();
let value = status();
assert_eq!(value["byte_cap"], BYTE_CAP);
assert_eq!(value["interval_secs"], INTERVAL.as_secs());
}
}
src/instr/mod.rs
+171
View File
@@ -0,0 +1,171 @@
//! Tick-body instrumentation.
//!
//! A purpose-built, feature-gated profiler for the rx-loop tick arm. It exists
//! to answer one question with field data: which subsystem step dominates the
//! tick body, and how long does the tick arm wait behind the other `select!`
//! arms before it runs at all.
//!
//! # Shape
//!
//! - Everything that costs anything at runtime is behind the `profiling` Cargo
//! feature, which is **off by default**. The default build's neutrality is a
//! property of the generated code, not of a runtime check.
//! - The instrumentation macro is defined twice, once per feature state. The
//! feature-off definition is a pure pass-through: it expands to the measured
//! expression and nothing else, so no timing code exists in a default build.
//! - The always-present surface — [`gate`], [`tick_entry`], [`tick_gauges`],
//! [`shutdown`] — exists in both feature states because the call sites in
//! `rx_loop.rs` and the lifecycle teardown must compile either way. Their
//! feature-off forms are empty (and [`gate`] is a `const fn` returning
//! `false`), so they cost nothing.
//! - The module is named `instr` rather than `profiling` so that it sorts
//! before `node` in `lib.rs`'s alphabetical module list: a `#[macro_use]`
//! module must be declared before the modules that use its macros.
//!
//! # Data model
//!
//! Domain above step: [`Domain`] carries exactly one variant today
//! (`Domain::Tick`). The primitive, the recorder, the writer and the `fipsctl`
//! surface all take a domain, so adding a data-path domain later is additive.
//! No second domain is declared until something records into it.
//!
//! Per (domain, step) the recorder keeps an exact count, max and total in fixed
//! static `AtomicU64` arrays — no histogram, no accumulation, a fixed footprint
//! regardless of run length. Gauges (ticks per interval, peer count, and the
//! arm-starvation figures) live in a parallel array and are emitted with an
//! explicit row kind so a gauge value never lands under a duration column.
#[cfg(feature = "profiling")]
pub(crate) mod capture;
#[cfg(feature = "profiling")]
mod recorder;
#[cfg(feature = "profiling")]
mod writer;
#[cfg(feature = "profiling")]
pub(crate) use recorder::{Domain, Step, now, record};
// ---------------------------------------------------------------------------
// The macro pair.
//
// Every path in the body is `$crate::`-qualified. `macro_rules!` bodies are not
// path-hygienic: an unqualified `Instant::now()` or `record(..)` would resolve
// at the *call site* (`rx_loop.rs`), where neither name is in scope. Importing
// them there is worse still, because the imports would be unused in the
// feature-off build and red it under `-D warnings`.
//
// `$e` is evaluated exactly once in both forms, which is what makes nesting the
// whole-tick span around the per-step spans safe.
// ---------------------------------------------------------------------------
/// Time `$e` as one step of `$domain`, when `$on` is true.
///
/// `$on` is the per-tick gate hoist: the enable flag is read once at the top of
/// the tick arm into a local, and that local is passed explicitly to every
/// invocation, because macro hygiene makes a call-site local invisible inside
/// the macro body.
#[cfg(feature = "profiling")]
macro_rules! instr_step {
($on:expr, $domain:expr, $step:expr, $e:expr) => {{
let t0 = if $on {
Some($crate::instr::now())
} else {
None
};
let r = $e;
if let Some(t) = t0 {
$crate::instr::record($domain, $step, t.elapsed());
}
r
}};
}
/// Feature-off form: a pure pass-through. The expansion contains no clock read,
/// no counter update and no reference to the recorder — only the measured
/// expression, plus a discard of the gate local so it is not unused.
#[cfg(not(feature = "profiling"))]
macro_rules! instr_step {
($on:expr, $domain:expr, $step:expr, $e:expr) => {{
let _ = &$on;
$e
}};
}
// ---------------------------------------------------------------------------
// Always-present surface.
// ---------------------------------------------------------------------------
/// Whether a capture is armed. Read **once per tick** into a local that is then
/// passed to each `instr_step!` invocation, so the feature-on-but-idle cost of
/// the whole tick arm is a single relaxed load.
#[cfg(feature = "profiling")]
#[inline]
pub(crate) fn gate() -> bool {
capture::gate()
}
/// Feature-off gate: a `const fn` returning `false`, so the whole tick arm
/// folds to the uninstrumented sequence at compile time.
#[cfg(not(feature = "profiling"))]
#[inline]
pub(crate) const fn gate() -> bool {
false
}
/// Record how late this tick-arm entry is against its scheduled deadline.
///
/// The arm is polled **last** under `biased;`, so its lateness is the time it
/// spent waiting behind the packet, TUN and control arms. `tokio::time::
/// interval::tick` returns the deadline it was scheduled for, so this is a
/// direct subtraction rather than a model. Two earlier designs derived it from
/// the inter-entry gap instead and both under-reported: one by the previous
/// body, the other by reporting only the first difference of the delay, so a
/// sustained stall read as zero. The inter-entry gap is still recorded as its
/// own gauge, but it carries no starvation signal on its own.
#[cfg(feature = "profiling")]
#[inline]
pub(crate) fn tick_entry(on: bool, deadline: std::time::Instant, now: std::time::Instant) {
recorder::tick_entry(on, deadline, now);
}
#[cfg(not(feature = "profiling"))]
#[inline]
pub(crate) fn tick_entry(_on: bool, _deadline: std::time::Instant, _now: std::time::Instant) {}
/// Sample the per-tick gauges taken from node state.
#[cfg(feature = "profiling")]
#[inline]
pub(crate) fn tick_gauges(on: bool, peers: u64) {
recorder::tick_gauges(on, peers);
}
#[cfg(not(feature = "profiling"))]
#[inline]
pub(crate) fn tick_gauges(_on: bool, _peers: u64) {}
/// Stop and reap any running capture at daemon teardown. Idempotent.
#[cfg(feature = "profiling")]
pub(crate) fn shutdown() {
capture::shutdown();
}
#[cfg(not(feature = "profiling"))]
pub(crate) fn shutdown() {}
/// One serialization lock for every test in this module tree.
///
/// The recorder counters and the capture state machine are the *same* process
/// statics: `capture::start` calls `recorder::reset`, and `capture::stop`
/// drains every slot. Two suites with their own locks therefore do not
/// serialize against each other, and the feature-on stage runs tests as
/// threads in one process, so a capture round-trip can zero the counters a
/// recorder test is mid-way through asserting on. One lock for both.
#[cfg(all(test, feature = "profiling"))]
pub(crate) static TEST_SERIAL: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// Take the shared test lock, recovering from a poisoned mutex so one failing
/// test does not cascade into every other one.
#[cfg(all(test, feature = "profiling"))]
pub(crate) fn test_serial() -> std::sync::MutexGuard<'static, ()> {
TEST_SERIAL.lock().unwrap_or_else(|e| e.into_inner())
}
src/instr/recorder.rs
+487
View File
@@ -0,0 +1,487 @@
//! Fixed-footprint recorder: exact count / max / total per (domain, step).
//!
//! All state is process statics, not `Node` state, because the `fipsctl`
//! handler that arms and disarms a capture runs in the control accept task and
//! has no `&Node` — that is the whole point of serving it off-loop, so it
//! cannot queue behind the behavior it is measuring.
//!
//! The writer thread is the only reader. It takes each interval's figures with
//! `swap(0)`, so there are no "previous value" arrays to carry and the counters
//! are per-interval by construction.
use std::sync::LazyLock;
use std::sync::atomic::{AtomicU64, Ordering::Relaxed};
use std::time::{Duration, Instant};
/// Measurement domain. Structural only: one variant today.
///
/// A data-path domain is deliberately **not** declared until something records
/// into it. What generalizes here is the enum, the counter table and the
/// writer; the per-tick gate hoist does not, so a data-path domain will need
/// its own gate strategy.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[repr(usize)]
pub(crate) enum Domain {
Tick = 0,
}
pub(crate) const N_DOMAINS: usize = 1;
pub(crate) const DOMAINS: [Domain; N_DOMAINS] = [Domain::Tick];
impl Domain {
pub(crate) const fn name(self) -> &'static str {
match self {
Domain::Tick => "tick",
}
}
}
/// One measured step of the rx-loop tick arm, in call order, plus the
/// whole-body span.
///
/// `as usize` indexes the counter arrays, so the discriminants are dense and
/// `WholeTick` is last (it defines `N_STEPS`). Variants are declared
/// unconditionally — see [`Step::emitted`] for how the two platform- and
/// profile-conditional steps are kept out of the emitted table.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[repr(usize)]
pub(crate) enum Step {
CheckTimeouts = 0,
ReloadPeerAcl,
ReloadHostMap,
PollPendingConnects,
PollNostrRendezvous,
PollLanRendezvous,
DrivePeerTimers,
ResendPendingRekeys,
/// `next`-only: the FMP rekey msg3 resend driver has no master-line
/// counterpart, so this variant exists on this line alone.
ResendPendingFmpRekeyMsg3,
ResendPendingSessionHandshakes,
ResendPendingSessionMsg3,
PurgeIdleSessions,
ProcessPendingRetries,
CheckTreeState,
CheckBloomState,
ComputeMeshSize,
RecordStatsHistory,
CheckMmpReports,
CheckSessionMmpReports,
CheckLinkHeartbeats,
CheckRekey,
CheckSessionRekey,
CheckPendingLookups,
PollTransportDiscovery,
SampleTransportCongestion,
ActivateConnectedUdpSessions,
DebugAssertPeerMapsCoherent,
/// The whole tick-arm body, from before `check_timeouts` to after the last
/// step. Composes safely with the per-step spans because the macro
/// evaluates its measured expression exactly once.
WholeTick,
}
pub(crate) const N_STEPS: usize = Step::WholeTick as usize + 1;
/// Every step, in emission order. Index `i` of this table is `STEPS[i] as
/// usize`; `steps_table_is_dense` asserts it.
pub(crate) const STEPS: [Step; N_STEPS] = [
Step::CheckTimeouts,
Step::ReloadPeerAcl,
Step::ReloadHostMap,
Step::PollPendingConnects,
Step::PollNostrRendezvous,
Step::PollLanRendezvous,
Step::DrivePeerTimers,
Step::ResendPendingRekeys,
Step::ResendPendingFmpRekeyMsg3,
Step::ResendPendingSessionHandshakes,
Step::ResendPendingSessionMsg3,
Step::PurgeIdleSessions,
Step::ProcessPendingRetries,
Step::CheckTreeState,
Step::CheckBloomState,
Step::ComputeMeshSize,
Step::RecordStatsHistory,
Step::CheckMmpReports,
Step::CheckSessionMmpReports,
Step::CheckLinkHeartbeats,
Step::CheckRekey,
Step::CheckSessionRekey,
Step::CheckPendingLookups,
Step::PollTransportDiscovery,
Step::SampleTransportCongestion,
Step::ActivateConnectedUdpSessions,
Step::DebugAssertPeerMapsCoherent,
Step::WholeTick,
];
impl Step {
pub(crate) const fn name(self) -> &'static str {
match self {
Step::CheckTimeouts => "check_timeouts",
Step::ReloadPeerAcl => "reload_peer_acl",
Step::ReloadHostMap => "reload_host_map",
Step::PollPendingConnects => "poll_pending_connects",
Step::PollNostrRendezvous => "poll_nostr_rendezvous",
Step::PollLanRendezvous => "poll_lan_rendezvous",
Step::DrivePeerTimers => "drive_peer_timers",
Step::ResendPendingRekeys => "resend_pending_rekeys",
Step::ResendPendingFmpRekeyMsg3 => "resend_pending_fmp_rekey_msg3",
Step::ResendPendingSessionHandshakes => "resend_pending_session_handshakes",
Step::ResendPendingSessionMsg3 => "resend_pending_session_msg3",
Step::PurgeIdleSessions => "purge_idle_sessions",
Step::ProcessPendingRetries => "process_pending_retries",
Step::CheckTreeState => "check_tree_state",
Step::CheckBloomState => "check_bloom_state",
Step::ComputeMeshSize => "compute_mesh_size",
Step::RecordStatsHistory => "record_stats_history",
Step::CheckMmpReports => "check_mmp_reports",
Step::CheckSessionMmpReports => "check_session_mmp_reports",
Step::CheckLinkHeartbeats => "check_link_heartbeats",
Step::CheckRekey => "check_rekey",
Step::CheckSessionRekey => "check_session_rekey",
Step::CheckPendingLookups => "check_pending_lookups",
Step::PollTransportDiscovery => "poll_transport_discovery",
Step::SampleTransportCongestion => "sample_transport_congestion",
Step::ActivateConnectedUdpSessions => "activate_connected_udp_sessions",
Step::DebugAssertPeerMapsCoherent => "debug_assert_peer_maps_coherent",
Step::WholeTick => "whole_tick",
}
}
/// Whether this step gets a row in this build.
///
/// Two steps are conditionally compiled at their call sites. Emitting a row
/// for them in a build where the call site does not exist would publish a
/// count that is structurally zero forever, which reads as "this step never
/// runs" rather than "this step is not in this build". The predicates below
/// are the same `cfg` expressions that gate the call sites in
/// `node::dataplane::rx_loop`; keep them in step.
pub(crate) const fn emitted(self) -> bool {
match self {
Step::ActivateConnectedUdpSessions => {
cfg!(any(target_os = "linux", target_os = "macos"))
}
Step::DebugAssertPeerMapsCoherent => cfg!(debug_assertions),
_ => true,
}
}
}
/// A scalar sampled once per tick, as opposed to a duration.
///
/// Gauges carry their own row kind and their own unit in the output so a gauge
/// value can never be read as a duration.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[repr(usize)]
pub(crate) enum Gauge {
Ticks = 0,
Peers,
TickGap,
ArmStarvation,
}
pub(crate) const N_GAUGES: usize = Gauge::ArmStarvation as usize + 1;
pub(crate) const GAUGES: [Gauge; N_GAUGES] = [
Gauge::Ticks,
Gauge::Peers,
Gauge::TickGap,
Gauge::ArmStarvation,
];
impl Gauge {
pub(crate) const fn name(self) -> &'static str {
match self {
Gauge::Ticks => "ticks",
Gauge::Peers => "peers",
Gauge::TickGap => "tick_entry_gap",
Gauge::ArmStarvation => "arm_starvation",
}
}
/// Unit of the `max` and `total` columns for this gauge.
pub(crate) const fn unit(self) -> &'static str {
match self {
Gauge::Ticks => "ticks",
Gauge::Peers => "peers",
Gauge::TickGap | Gauge::ArmStarvation => "us",
}
}
/// Whether the gauge's stored values are nanosecond durations that the
/// writer converts to microseconds.
pub(crate) const fn is_duration(self) -> bool {
matches!(self, Gauge::TickGap | Gauge::ArmStarvation)
}
}
const N_SLOTS: usize = N_DOMAINS * N_STEPS;
static COUNT: [AtomicU64; N_SLOTS] = [const { AtomicU64::new(0) }; N_SLOTS];
static MAX_NS: [AtomicU64; N_SLOTS] = [const { AtomicU64::new(0) }; N_SLOTS];
static TOTAL_NS: [AtomicU64; N_SLOTS] = [const { AtomicU64::new(0) }; N_SLOTS];
static G_COUNT: [AtomicU64; N_GAUGES] = [const { AtomicU64::new(0) }; N_GAUGES];
static G_MAX: [AtomicU64; N_GAUGES] = [const { AtomicU64::new(0) }; N_GAUGES];
static G_TOTAL: [AtomicU64; N_GAUGES] = [const { AtomicU64::new(0) }; N_GAUGES];
/// Monotonic baseline so tick-arm entry times fit in an atomic. Offset by one
/// on store so that zero can mean "no previous entry".
static BASE: LazyLock<Instant> = LazyLock::new(Instant::now);
static PREV_ENTRY_NS: AtomicU64 = AtomicU64::new(0);
#[inline]
const fn slot(domain: Domain, step: Step) -> usize {
(domain as usize * N_STEPS) + step as usize
}
/// Read the clock for a step span.
#[inline]
pub(crate) fn now() -> Instant {
Instant::now()
}
/// Record one observation of `step`.
#[inline]
pub(crate) fn record(domain: Domain, step: Step, elapsed: Duration) {
let ns = elapsed.as_nanos() as u64;
let idx = slot(domain, step);
COUNT[idx].fetch_add(1, Relaxed);
TOTAL_NS[idx].fetch_add(ns, Relaxed);
MAX_NS[idx].fetch_max(ns, Relaxed);
}
#[inline]
fn record_gauge(gauge: Gauge, value: u64) {
let idx = gauge as usize;
G_COUNT[idx].fetch_add(1, Relaxed);
G_TOTAL[idx].fetch_add(value, Relaxed);
G_MAX[idx].fetch_max(value, Relaxed);
}
/// Sample the inter-entry gap and the measured arm-starvation delay.
pub(crate) fn tick_entry(on: bool, deadline: Instant, now: Instant) {
if !on {
return;
}
// Offset by one so that a stored zero unambiguously means "no previous
// entry", even for an entry that lands on the baseline instant.
let stamp = BASE.elapsed().as_nanos() as u64 + 1;
let late = now.saturating_duration_since(deadline).as_nanos() as u64;
tick_entry_at(stamp, late);
}
/// The clock-free half of [`tick_entry`]: both times are inputs, so the
/// arithmetic can be driven with synthetic stamps in a test.
///
/// `late_ns` is how far past its scheduled deadline this entry was, measured
/// directly rather than derived. Two earlier designs derived it from the
/// inter-entry gap and were both wrong: subtracting the previous body
/// understated it by exactly the body, and subtracting `max(period, body)`
/// reported the *first difference* of the delay, so a sustained stall — the
/// overload regime this measurement exists to characterize — read as zero
/// forever. `tokio::time::interval::tick` hands back the deadline it was
/// scheduled for, so the delay is a subtraction with no model behind it.
fn tick_entry_at(stamp: u64, late_ns: u64) {
let prev = PREV_ENTRY_NS.swap(stamp, Relaxed);
record_gauge(Gauge::Ticks, 1);
record_gauge(Gauge::ArmStarvation, late_ns);
if prev == 0 {
// First entry of this capture: there is no previous entry to measure a
// gap against, and the idle interval before arming is not a gap. The
// lateness above does not depend on a previous entry, so it still counts.
return;
}
record_gauge(Gauge::TickGap, stamp.saturating_sub(prev));
}
/// Sample the gauges that come from node state.
pub(crate) fn tick_gauges(on: bool, peers: u64) {
if !on {
return;
}
record_gauge(Gauge::Peers, peers);
}
/// Take (and clear) this interval's figures for one step: count, max ns, total
/// ns. Called only by the writer thread.
pub(crate) fn take_step(domain: Domain, step: Step) -> (u64, u64, u64) {
let idx = slot(domain, step);
(
COUNT[idx].swap(0, Relaxed),
MAX_NS[idx].swap(0, Relaxed),
TOTAL_NS[idx].swap(0, Relaxed),
)
}
/// Take (and clear) this interval's figures for one gauge.
pub(crate) fn take_gauge(gauge: Gauge) -> (u64, u64, u64) {
let idx = gauge as usize;
(
G_COUNT[idx].swap(0, Relaxed),
G_MAX[idx].swap(0, Relaxed),
G_TOTAL[idx].swap(0, Relaxed),
)
}
/// Zero every counter so a capture starts from a clean slate.
pub(crate) fn reset() {
for i in 0..N_SLOTS {
COUNT[i].store(0, Relaxed);
MAX_NS[i].store(0, Relaxed);
TOTAL_NS[i].store(0, Relaxed);
}
for i in 0..N_GAUGES {
G_COUNT[i].store(0, Relaxed);
G_MAX[i].store(0, Relaxed);
G_TOTAL[i].store(0, Relaxed);
}
PREV_ENTRY_NS.store(0, Relaxed);
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::MutexGuard;
/// Shared with the capture tests: they mutate the same statics. See
/// `crate::instr::test_serial`.
fn serial() -> MutexGuard<'static, ()> {
crate::instr::test_serial()
}
#[test]
fn steps_table_is_dense() {
for (i, step) in STEPS.iter().enumerate() {
assert_eq!(*step as usize, i, "step {} is out of order", step.name());
}
assert_eq!(STEPS.len(), N_STEPS);
}
#[test]
fn gauges_table_is_dense() {
for (i, gauge) in GAUGES.iter().enumerate() {
assert_eq!(*gauge as usize, i, "gauge {} is out of order", gauge.name());
}
assert_eq!(GAUGES.len(), N_GAUGES);
}
#[test]
fn step_names_are_unique() {
let mut names: Vec<&str> = STEPS.iter().map(|s| s.name()).collect();
names.sort_unstable();
let before = names.len();
names.dedup();
assert_eq!(before, names.len(), "duplicate step name");
}
#[test]
fn emitted_row_count_matches_build() {
let emitted = STEPS.iter().filter(|s| s.emitted()).count();
// 25 unconditional subsystem steps on this line (24 shared with the
// master line, plus `resend_pending_fmp_rekey_msg3`, which exists only
// here) + the whole-tick span, plus the two conditionally-compiled
// steps where this build has them. The count is pinned deliberately: it
// is what caught the extra step when the master-line instrumentation
// was merged up, rather than letting the tables silently disagree.
let mut expected = 26;
if cfg!(any(target_os = "linux", target_os = "macos")) {
expected += 1;
}
if cfg!(debug_assertions) {
expected += 1;
}
assert_eq!(emitted, expected);
}
#[test]
fn record_accumulates_count_max_and_total() {
let _guard = serial();
reset();
record(Domain::Tick, Step::CheckRekey, Duration::from_nanos(10));
record(Domain::Tick, Step::CheckRekey, Duration::from_nanos(30));
let (count, max, total) = take_step(Domain::Tick, Step::CheckRekey);
assert_eq!((count, max, total), (2, 30, 40));
// Taking clears the slot.
assert_eq!(take_step(Domain::Tick, Step::CheckRekey), (0, 0, 0));
}
#[test]
fn starvation_is_the_measured_lateness_of_the_entry() {
let _guard = serial();
reset();
// The interval hands back the deadline it was scheduled for, so the
// delay is `now - deadline` and nothing is derived from the period, the
// previous entry, or the previous body.
PREV_ENTRY_NS.store(1_000_000_000, Relaxed);
tick_entry_at(1_100_000_000, 50_000_000);
assert_eq!(take_gauge(Gauge::TickGap), (1, 100_000_000, 100_000_000));
assert_eq!(
take_gauge(Gauge::ArmStarvation),
(1, 50_000_000, 50_000_000)
);
assert_eq!(take_gauge(Gauge::Ticks).0, 1);
reset();
}
#[test]
fn sustained_lateness_is_reported_on_every_tick() {
let _guard = serial();
reset();
// The regime the two earlier designs both hid. Three consecutive entries
// each 50 ms past their deadline, one period apart, i.e. the arm waiting
// a constant amount behind the other select arms every round. The gaps
// are all exactly one period, so any formula derived from the
// inter-entry gap reports zero here; measured lateness reports 50 ms
// three times, which is the truth.
PREV_ENTRY_NS.store(1_000_000_000, Relaxed);
tick_entry_at(1_050_000_000, 50_000_000);
tick_entry_at(1_100_000_000, 50_000_000);
tick_entry_at(1_150_000_000, 50_000_000);
let (count, max, total) = take_gauge(Gauge::ArmStarvation);
assert_eq!(count, 3);
assert_eq!(max, 50_000_000);
assert_eq!(total, 150_000_000);
// ...and the gap alone carries no signal about it: every gap is one
// period, exactly as it would be on a perfectly healthy node.
assert_eq!(take_gauge(Gauge::TickGap), (3, 50_000_000, 150_000_000));
reset();
}
#[test]
fn first_entry_of_a_capture_records_no_gap_but_still_records_lateness() {
let _guard = serial();
reset();
tick_entry_at(500, 7_000_000);
assert_eq!(take_gauge(Gauge::Ticks).0, 1);
assert_eq!(take_gauge(Gauge::TickGap), (0, 0, 0));
// Lateness does not depend on a previous entry, so the first tick of a
// capture still contributes one.
assert_eq!(take_gauge(Gauge::ArmStarvation), (1, 7_000_000, 7_000_000));
reset();
}
#[test]
fn an_on_schedule_entry_reports_no_starvation() {
let _guard = serial();
reset();
PREV_ENTRY_NS.store(1_000_000_000, Relaxed);
tick_entry_at(1_050_000_000, 0);
assert_eq!(take_gauge(Gauge::ArmStarvation), (1, 0, 0));
assert_eq!(take_gauge(Gauge::TickGap), (1, 50_000_000, 50_000_000));
reset();
}
#[test]
fn gate_off_records_nothing() {
let _guard = serial();
reset();
let t = Instant::now();
tick_entry(false, t, t);
tick_gauges(false, 42);
assert_eq!(take_gauge(Gauge::Ticks), (0, 0, 0));
assert_eq!(take_gauge(Gauge::Peers), (0, 0, 0));
}
}
src/instr/writer.rs
+218
View File
@@ -0,0 +1,218 @@
//! The capture writer: a dedicated, named OS thread with an explicit
//! lifecycle.
//!
//! There is no worker-thread lifecycle in this codebase to copy — the crypto
//! worker pools are never torn down and drop their join handles at spawn — so
//! this one is designed here.
//!
//! Two properties matter:
//!
//! - **It is an OS thread, not a spawned task.** The runtime is
//! `current_thread`, so file I/O on a task would run on the rx loop's own
//! thread and stall every `select!` arm, including the one being measured.
//! - **It waits on a channel with a timeout, not on a sleep.** `recv_timeout`
//! returns immediately when the toggle sends stop, so `off` performs a final
//! drain and joins promptly instead of parking the caller for up to a full
//! flush interval.
//!
//! The thread is created lazily when a capture starts, so a node that never
//! arms one never has the thread.
use std::fs::File;
use std::io::Write;
use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender};
use std::thread::{self, JoinHandle};
use std::time::{SystemTime, UNIX_EPOCH};
use super::capture::{self, BYTE_CAP, INTERVAL};
use super::recorder::{self, DOMAINS, GAUGES, STEPS};
/// Owner-side handle to the writer thread.
pub(crate) struct Handle {
stop_tx: Sender<()>,
join: JoinHandle<()>,
}
impl Handle {
/// Wake the writer, let it drain once more, and join it.
pub(crate) fn stop_and_join(self) {
// A send error means the thread already exited (cap stop); joining is
// still correct and returns at once.
let _ = self.stop_tx.send(());
let _ = self.join.join();
}
}
/// Start the writer thread on an already-open sink.
pub(crate) fn spawn(file: File) -> std::io::Result<Handle> {
let (stop_tx, stop_rx) = mpsc::channel();
let join = thread::Builder::new()
.name("fips-profile".to_string())
.spawn(move || run(file, stop_rx))?;
Ok(Handle { stop_tx, join })
}
/// What one flush cycle decided. Separated from [`run`] so the terminal paths
/// can be driven in a test with a failing sink: the loop below owns the waiting
/// and the state transition, this owns the decision.
#[derive(Debug, PartialEq, Eq)]
enum Cycle {
Continue,
CapReached,
WriteFailed,
}
/// Drain one interval into the sink and decide whether the capture goes on.
fn flush_cycle<W: Write>(file: &mut W) -> Cycle {
if flush(file).is_err() {
// The sink is gone or full; stop rather than spinning on a broken file
// for the rest of the run.
let _ = note(file, "capture stopped: write error");
return Cycle::WriteFailed;
}
if capture::bytes_written() >= BYTE_CAP {
let _ = note(
file,
&format!("capture stopped: byte cap {BYTE_CAP} reached"),
);
let _ = file.flush();
return Cycle::CapReached;
}
Cycle::Continue
}
fn run<W: Write>(mut file: W, stop_rx: Receiver<()>) {
loop {
match stop_rx.recv_timeout(INTERVAL) {
// Stop requested, or the owner went away: final drain, then exit.
Ok(()) | Err(RecvTimeoutError::Disconnected) => {
let _ = flush(&mut file);
let _ = file.flush();
return;
}
Err(RecvTimeoutError::Timeout) => match flush_cycle(&mut file) {
Cycle::Continue => {}
Cycle::WriteFailed => {
// The trailer went to the same failing file, so it is not a
// signal that survives. `stop` and a subsequent `on` both
// clear the state without surfacing it, so an operator would
// otherwise never learn the window was truncated.
tracing::warn!(
target: "fips::instr",
"profile capture stopped: write error on the sink"
);
capture::mark_stopped(capture::STOPPED_BY_ERROR);
return;
}
Cycle::CapReached => {
capture::mark_stopped(capture::STOPPED_BY_CAP);
return;
}
},
}
}
}
/// Append a `#`-prefixed trailer line.
fn note<W: Write>(file: &mut W, text: &str) -> std::io::Result<()> {
let line = format!("# {text}\n");
file.write_all(line.as_bytes())?;
capture::add_bytes(line.len() as u64);
Ok(())
}
/// Emit one interval: every step of every domain, then the gauges.
///
/// Every emitted step gets a row every interval, including zero-count rows, so
/// "this step did not run" is visible rather than absent. The two steps whose
/// call sites are conditionally compiled are excluded in builds that do not
/// have them, so no row is structurally zero forever.
fn flush<W: Write>(file: &mut W) -> std::io::Result<()> {
let ts = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let mut out = String::with_capacity(4096);
for domain in DOMAINS {
for step in STEPS {
if !step.emitted() {
continue;
}
let (count, max_ns, total_ns) = recorder::take_step(domain, step);
out.push_str(&format!(
"{ts}\tstep\t{domain}\t{name}\t{count}\t{max}\t{total}\tus\n",
domain = domain.name(),
name = step.name(),
max = max_ns / 1_000,
total = total_ns / 1_000,
));
}
}
// Gauges carry the tick domain today; the row kind and the unit column keep
// them distinguishable from the duration rows above.
for gauge in GAUGES {
let (count, mut max, mut total) = recorder::take_gauge(gauge);
if gauge.is_duration() {
max /= 1_000;
total /= 1_000;
}
out.push_str(&format!(
"{ts}\tgauge\t{domain}\t{name}\t{count}\t{max}\t{total}\t{unit}\n",
domain = recorder::Domain::Tick.name(),
name = gauge.name(),
unit = gauge.unit(),
));
}
file.write_all(out.as_bytes())?;
capture::add_bytes(out.len() as u64);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// A sink that fails every write, so the writer's error path is driven by a
/// real `Err` rather than asserted about.
struct AlwaysFails;
impl Write for AlwaysFails {
fn write(&mut self, _buf: &[u8]) -> std::io::Result<usize> {
Err(std::io::Error::new(
std::io::ErrorKind::StorageFull,
"no space left on device",
))
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
/// A failing sink ends the capture as a write error, which `run` turns into
/// `STOPPED_BY_ERROR` — not into a byte-cap stop. The distinction is what
/// tells an operator that a window is truncated rather than complete.
///
/// **Coverage note.** This drives the decision, not the filesystem
/// condition. The mesh rehearsal cannot produce one: removing the file
/// leaves the writer's descriptor valid and writes keep succeeding into the
/// unlinked inode, and mounting a tiny filesystem inside the test container
/// is refused. So "a real ENOSPC reaches this branch" stays unexercised;
/// what is covered is that an `Err` from the sink produces the error
/// outcome and not the cap outcome.
#[test]
fn a_failing_sink_ends_the_cycle_as_an_error_not_a_cap() {
let _guard = crate::instr::test_serial();
assert_eq!(flush_cycle(&mut AlwaysFails), Cycle::WriteFailed);
}
/// The healthy path must not be reported as either terminal state, or the
/// test above would pass for a writer that always stops.
#[test]
fn a_working_sink_continues() {
let _guard = crate::instr::test_serial();
let mut sink: Vec<u8> = Vec::new();
assert_eq!(flush_cycle(&mut sink), Cycle::Continue);
}
}
src/lib.rs
+56 -20
View File
@@ -3,22 +3,34 @@
//! A distributed, decentralized network routing protocol for mesh nodes
//! connecting over arbitrary transports.
pub mod bloom;
// Name the `alloc` crate directly so the sans-IO protocol cores can spell their
// heap-type imports in `no_std`-forward form (`alloc::sync::Arc`,
// `alloc::collections::BTreeMap`). The crate remains `std`; this only reduces the
// distance to extracting the pure cores into a `no_std` crate later.
extern crate alloc;
pub mod cache;
pub mod config;
pub mod control;
pub mod discovery;
#[cfg(target_os = "linux")]
pub mod gateway;
pub mod identity;
pub mod mmp;
// Declared before `node` (and named to sort there) because it carries
// `#[macro_use]`: the tick instrumentation macro must be in scope for the
// modules that follow.
#[macro_use]
pub(crate) mod instr;
pub mod mdns;
pub mod node;
pub mod noise;
pub mod nostr;
pub mod peer;
pub mod perf_profile;
pub mod protocol;
pub(crate) mod proto;
#[cfg(test)]
pub(crate) mod testutil;
mod time;
pub mod transport;
pub mod tree;
pub mod upper;
pub mod utils;
pub mod version;
@@ -33,14 +45,16 @@ pub use identity::{
pub use config::{Config, ConfigError, IdentityConfig, NymConfig, TorConfig, UdpConfig};
pub use upper::config::{DnsConfig, TunConfig};
// Re-export discovery types
pub use discovery::{BootstrapHandoffResult, EstablishedTraversal};
// Re-export nostr rendezvous handoff types
pub use nostr::{BootstrapHandoffResult, EstablishedTraversal, is_punch_packet};
// Re-export tree types
pub use tree::{CoordEntry, ParentDeclaration, TreeCoordinate, TreeError, TreeState};
// Re-export tree types (relocated from tree:: to proto::stp)
pub use proto::stp::{
CoordEntry, CoordError, ParentDeclaration, TreeCoordinate, TreeError, TreeState,
};
// Re-export bloom filter types
pub use bloom::{BloomError, BloomFilter, BloomState};
// Re-export bloom filter types (relocated from bloom:: to proto::bloom)
pub use proto::bloom::{BloomError, BloomFilter, BloomState};
// Re-export transport types
pub use transport::udp::UdpTransport;
@@ -50,21 +64,43 @@ pub use transport::{
TransportState, TransportType, packet_channel,
};
// Re-export protocol types
pub use protocol::{
CoordsRequired, FilterAnnounce, HandshakeMessageType, LinkMessageType, LookupRequest,
LookupResponse, PathBroken, ProtocolError, SessionAck, SessionDatagram, SessionFlags,
SessionMessageType, SessionSetup, TreeAnnounce,
// Re-export link-layer types (relocated from protocol:: to proto::link)
pub use proto::link::{LinkMessageType, SessionDatagram};
// Re-export the shared protocol error (relocated from protocol:: to proto::Error)
pub use proto::Error;
// Re-export FSP session wire types (relocated from protocol:: to proto::fsp)
pub use proto::fsp::{SessionAck, SessionFlags, SessionMessageType, SessionSetup};
// Re-export STP wire types (relocated from protocol:: to proto::stp)
pub use proto::stp::TreeAnnounce;
// Re-export bloom wire types (relocated from protocol:: to proto::bloom)
pub use proto::bloom::FilterAnnounce;
// Re-export discovery wire types (relocated from protocol:: to proto::lookup)
pub use proto::lookup::{LookupRequest, LookupResponse};
// Re-export routing wire types (relocated from protocol:: to proto::routing)
pub use proto::routing::{
COORDS_REQUIRED_SIZE, CoordsRequired, MTU_EXCEEDED_SIZE, MtuExceeded, PathBroken,
};
// Re-export FMP link-framing wire type (relocated from protocol:: to proto::fmp)
pub use proto::fmp::HandshakeMessageType;
// Re-export FMP negotiation wire types (relocated from protocol:: to proto::fmp)
pub use proto::fmp::{NegotiationPayload, NodeProfile, TlvEntry};
// Re-export cache types
pub use cache::{CacheEntry, CacheError, CacheStats, CoordCache};
// Re-export FMP tie-break helper and promotion result (relocated from peer:: to proto::fmp)
pub use proto::fmp::{PromotionResult, cross_connection_winner};
// Re-export peer types
pub use peer::{
ActivePeer, ConnectivityState, HandshakeState, PeerConnection, PeerError, PeerSlot,
PromotionResult, cross_connection_winner,
};
pub use peer::{ActivePeer, ConnectivityState, PeerError};
// Re-export node types
pub use node::{Node, NodeError, NodeState, UpdatePeersOutcome};
src/discovery/lan/mod.rs → src/mdns/mod.rs
+28 -19
View File
@@ -54,8 +54,12 @@ pub const TXT_KEY_SCOPE: &str = "scope";
/// `PROTOCOL_VERSION`).
pub const TXT_KEY_VERSION: &str = "v";
/// FIPS protocol version advertised in the mDNS TXT `v` key. Kept in sync
/// with the Nostr rendezvous `PROTOCOL_VERSION` (same value, `"1"`).
const TXT_PROTOCOL_VERSION: &str = "1";
#[derive(Debug, Error)]
pub enum LanDiscoveryError {
pub enum LanRendezvousError {
#[error("mDNS daemon init failed: {0}")]
Daemon(String),
#[error("mDNS register failed: {0}")]
@@ -79,7 +83,7 @@ pub struct LanDiscoveredPeer {
pub observed_at: Instant,
}
/// Browser-side events surfaced by `LanDiscovery::drain_events`.
/// Browser-side events surfaced by `LanRendezvous::drain_events`.
#[derive(Debug, Clone)]
pub enum LanEvent {
Discovered(LanDiscoveredPeer),
@@ -87,17 +91,17 @@ pub enum LanEvent {
/// Runtime configuration for the mDNS responder + browser.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct LanDiscoveryConfig {
pub struct LanRendezvousConfig {
/// Master switch. Default: `false` — LAN discovery is opt-in. Operators
/// who want sub-second same-LAN pairing enable it via
/// `node.discovery.lan.enabled: true`. Default-off avoids reintroducing
/// `node.rendezvous.lan.enabled: true`. Default-off avoids reintroducing
/// a per-LAN identity broadcast on nodes that have deliberately disabled
/// other discovery channels, and avoids any multicast surprise on upgrade.
#[serde(default = "LanDiscoveryConfig::default_enabled")]
#[serde(default = "LanRendezvousConfig::default_enabled")]
pub enabled: bool,
/// Overridable service type, primarily so integration tests can run
/// multiple isolated services on the same loopback interface.
#[serde(default = "LanDiscoveryConfig::default_service_type")]
#[serde(default = "LanRendezvousConfig::default_service_type")]
pub service_type: String,
/// Optional application/network scope carried in the LAN-only TXT
/// record. Browsers that set a scope ignore adverts for other scopes.
@@ -109,7 +113,7 @@ pub struct LanDiscoveryConfig {
pub scope: Option<String>,
}
impl Default for LanDiscoveryConfig {
impl Default for LanRendezvousConfig {
fn default() -> Self {
Self {
enabled: Self::default_enabled(),
@@ -119,7 +123,7 @@ impl Default for LanDiscoveryConfig {
}
}
impl LanDiscoveryConfig {
impl LanRendezvousConfig {
fn default_enabled() -> bool {
false
}
@@ -129,7 +133,7 @@ impl LanDiscoveryConfig {
}
/// Running mDNS responder + browser bound to the node's UDP advert port.
pub struct LanDiscovery {
pub struct LanRendezvous {
daemon: ServiceDaemon,
own_npub: String,
instance_fullname: String,
@@ -137,7 +141,12 @@ pub struct LanDiscovery {
event_pump: tokio::task::JoinHandle<()>,
}
impl LanDiscovery {
impl LanRendezvous {
/// Whether the mDNS event-pump task has exited (runtime liveness).
pub fn is_finished(&self) -> bool {
self.event_pump.is_finished()
}
/// Start the mDNS responder and browser.
///
/// `advertised_port` is the UDP port the operational UDP transport
@@ -148,16 +157,16 @@ impl LanDiscovery {
identity: &Identity,
scope: Option<String>,
advertised_port: u16,
config: LanDiscoveryConfig,
) -> Result<Arc<Self>, LanDiscoveryError> {
config: LanRendezvousConfig,
) -> Result<Arc<Self>, LanRendezvousError> {
if !config.enabled {
return Err(LanDiscoveryError::Disabled);
return Err(LanRendezvousError::Disabled);
}
if advertised_port == 0 {
return Err(LanDiscoveryError::NoAdvertisedPort);
return Err(LanRendezvousError::NoAdvertisedPort);
}
let daemon = ServiceDaemon::new().map_err(|e| LanDiscoveryError::Daemon(e.to_string()))?;
let daemon = ServiceDaemon::new().map_err(|e| LanRendezvousError::Daemon(e.to_string()))?;
let npub = identity.npub();
// mDNS DNS labels are capped at 63 bytes. 16 bech32 chars of npub
@@ -176,7 +185,7 @@ impl LanDiscovery {
}
props.insert(
TXT_KEY_VERSION.to_string(),
super::nostr::PROTOCOL_VERSION.to_string(),
TXT_PROTOCOL_VERSION.to_string(),
);
// host_ipv4 is set to "127.0.0.1" *and* enable_addr_auto() is
@@ -193,18 +202,18 @@ impl LanDiscovery {
advertised_port,
Some(props),
)
.map_err(|e| LanDiscoveryError::Register(e.to_string()))?
.map_err(|e| LanRendezvousError::Register(e.to_string()))?
.enable_addr_auto();
let instance_fullname = service_info.get_fullname().to_string();
daemon
.register(service_info)
.map_err(|e| LanDiscoveryError::Register(e.to_string()))?;
.map_err(|e| LanRendezvousError::Register(e.to_string()))?;
let browse_rx = daemon
.browse(&config.service_type)
.map_err(|e| LanDiscoveryError::Browse(e.to_string()))?;
.map_err(|e| LanRendezvousError::Browse(e.to_string()))?;
let (events_tx, events_rx) = tokio::sync::mpsc::unbounded_channel();
let own_npub = npub.clone();
src/discovery/lan/tests.rs → src/mdns/tests.rs
+12 -12
View File
@@ -4,7 +4,7 @@ use std::time::Duration;
use crate::Identity;
use mdns_sd::ScopedIp;
use super::{LanDiscovery, LanDiscoveryConfig, LanEvent};
use super::{LanEvent, LanRendezvous, LanRendezvousConfig};
/// Distinct service type per test run so concurrent cargo-test workers
/// on the same machine don't cross-feed each other's adverts via the
@@ -15,8 +15,8 @@ fn isolated_service_type(tag: &str) -> String {
format!("_fipstest-{tag}-{rand:08x}._udp.local.")
}
fn config_for(service_type: String) -> LanDiscoveryConfig {
LanDiscoveryConfig {
fn config_for(service_type: String) -> LanRendezvousConfig {
LanRendezvousConfig {
enabled: true,
service_type,
scope: None,
@@ -47,7 +47,7 @@ fn non_link_local_ipv6_advert_is_preserved() {
}
async fn wait_for_peer(
discovery: &LanDiscovery,
discovery: &LanRendezvous,
expected_npub: &str,
timeout: Duration,
) -> Option<super::LanDiscoveredPeer> {
@@ -64,7 +64,7 @@ async fn wait_for_peer(
None
}
/// Two LanDiscovery instances on isolated service types — `a` browses
/// Two LanRendezvous instances on isolated service types — `a` browses
/// only its own type and never sees `b`, and vice versa. Sanity check
/// that the scope-isolation defense works (we'd lose isolation if mdns-
/// sd ever leaked across service types).
@@ -76,7 +76,7 @@ async fn isolated_service_types_do_not_cross_feed() {
let service_a = isolated_service_type("isolated-a");
let service_b = isolated_service_type("isolated-b");
let lan_a = LanDiscovery::start(
let lan_a = LanRendezvous::start(
&identity_a,
Some("scope-x".to_string()),
61001,
@@ -84,7 +84,7 @@ async fn isolated_service_types_do_not_cross_feed() {
)
.await
.expect("start a");
let lan_b = LanDiscovery::start(
let lan_b = LanRendezvous::start(
&identity_b,
Some("scope-x".to_string()),
61002,
@@ -118,7 +118,7 @@ async fn isolated_service_types_do_not_cross_feed() {
assert!(!saw_a_from_b, "isolated service types must not cross-feed");
}
/// Two LanDiscovery instances on the same service type and the same
/// Two LanRendezvous instances on the same service type and the same
/// scope: each should observe the other's advert within a few seconds.
/// Exercises the responder + browser + TXT plumbing end-to-end.
///
@@ -136,7 +136,7 @@ async fn matched_scope_peers_observe_each_other() {
let service = isolated_service_type("matched");
let lan_a = LanDiscovery::start(
let lan_a = LanRendezvous::start(
&identity_a,
Some("scope-shared".to_string()),
61101,
@@ -144,7 +144,7 @@ async fn matched_scope_peers_observe_each_other() {
)
.await
.expect("start a");
let lan_b = LanDiscovery::start(
let lan_b = LanRendezvous::start(
&identity_b,
Some("scope-shared".to_string()),
61102,
@@ -181,7 +181,7 @@ async fn cross_scope_advert_is_filtered() {
let service = isolated_service_type("cross-scope");
let lan_a = LanDiscovery::start(
let lan_a = LanRendezvous::start(
&identity_a,
Some("scope-a".to_string()),
61201,
@@ -189,7 +189,7 @@ async fn cross_scope_advert_is_filtered() {
)
.await
.expect("start a");
let lan_b = LanDiscovery::start(
let lan_b = LanRendezvous::start(
&identity_b,
Some("scope-b".to_string()),
61202,
src/mmp/algorithms.rs
-526
View File
@@ -1,526 +0,0 @@
//! MMP algorithmic building blocks.
//!
//! Pure computational types with no dependency on peer or node state.
//! Each is independently testable.
use std::collections::VecDeque;
use std::time::Instant;
use crate::mmp::{EWMA_LONG_ALPHA, EWMA_SHORT_ALPHA};
// ============================================================================
// Jitter Estimator (RFC 3550 §6.4.1)
// ============================================================================
/// Interarrival jitter estimator using RFC 3550 algorithm.
///
/// Maintains a smoothed jitter estimate (α = 1/16) from the absolute
/// difference in one-way transit times between consecutive frames.
/// Uses integer arithmetic scaled by 16 to avoid floating-point.
pub struct JitterEstimator {
/// Scaled jitter estimate (×16 for integer arithmetic).
jitter_q4: i64,
}
impl JitterEstimator {
pub fn new() -> Self {
Self { jitter_q4: 0 }
}
/// Update with transit time delta between consecutive frames.
///
/// `transit_delta` = (R_i - R_{i-1}) - (S_i - S_{i-1}) in microseconds.
pub fn update(&mut self, transit_delta: i32) {
// RFC 3550: J = J + (1/16)(|D(i)| - J)
// Scaled: J_q4 = J_q4 + (|D| - J_q4/16)
// = J_q4 + |D| - J_q4 >> 4
let abs_d = (transit_delta as i64).unsigned_abs() as i64;
self.jitter_q4 += abs_d - (self.jitter_q4 >> 4);
}
/// Current jitter estimate in microseconds.
pub fn jitter_us(&self) -> u32 {
(self.jitter_q4 >> 4) as u32
}
}
impl Default for JitterEstimator {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// SRTT Estimator (Jacobson, RFC 6298)
// ============================================================================
/// Smoothed RTT estimator using Jacobson's algorithm.
///
/// SRTT and RTTVAR are maintained in microseconds using integer arithmetic.
pub struct SrttEstimator {
/// Smoothed RTT (microseconds).
srtt_us: i64,
/// RTT variance (microseconds).
rttvar_us: i64,
/// Whether the first sample has been applied.
initialized: bool,
}
impl SrttEstimator {
pub fn new() -> Self {
Self {
srtt_us: 0,
rttvar_us: 0,
initialized: false,
}
}
/// Feed an RTT sample in microseconds.
pub fn update(&mut self, rtt_us: i64) {
if !self.initialized {
// RFC 6298 §2.2: first measurement
self.srtt_us = rtt_us;
self.rttvar_us = rtt_us / 2;
self.initialized = true;
} else {
// RFC 6298 §2.3:
// RTTVAR = (1 - β) * RTTVAR + β * |SRTT - R'| β = 1/4
// SRTT = (1 - α) * SRTT + α * R' α = 1/8
let err = (self.srtt_us - rtt_us).abs();
self.rttvar_us = self.rttvar_us - (self.rttvar_us >> 2) + (err >> 2);
self.srtt_us = self.srtt_us - (self.srtt_us >> 3) + (rtt_us >> 3);
}
}
pub fn srtt_us(&self) -> i64 {
self.srtt_us
}
pub fn rttvar_us(&self) -> i64 {
self.rttvar_us
}
pub fn initialized(&self) -> bool {
self.initialized
}
/// Retransmission timeout = SRTT + max(4 * RTTVAR, 1s), floored at 1s.
pub fn rto_us(&self) -> i64 {
let rto = self.srtt_us + (self.rttvar_us << 2).max(1_000_000);
rto.max(1_000_000)
}
}
impl Default for SrttEstimator {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Dual EWMA Trend Detector
// ============================================================================
/// Dual EWMA for trend detection on a single metric.
///
/// Short-term (α=1/4) tracks recent conditions; long-term (α=1/32)
/// establishes a stable baseline. Divergence indicates trend direction.
pub struct DualEwma {
short: f64,
long: f64,
initialized: bool,
}
impl DualEwma {
pub fn new() -> Self {
Self {
short: 0.0,
long: 0.0,
initialized: false,
}
}
pub fn update(&mut self, sample: f64) {
if !self.initialized {
self.short = sample;
self.long = sample;
self.initialized = true;
} else {
self.short += EWMA_SHORT_ALPHA * (sample - self.short);
self.long += EWMA_LONG_ALPHA * (sample - self.long);
}
}
pub fn short(&self) -> f64 {
self.short
}
pub fn long(&self) -> f64 {
self.long
}
pub fn initialized(&self) -> bool {
self.initialized
}
}
impl Default for DualEwma {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// One-Way Delay Trend Detector
// ============================================================================
/// OWD trend detector using linear regression over a ring buffer.
///
/// Stores (sequence, owd_us) samples and computes the slope via
/// least-squares regression. The slope (µs/s) indicates whether
/// queuing delay is increasing (congestion) or stable.
pub struct OwdTrendDetector {
samples: VecDeque<(u32, i64)>,
capacity: usize,
}
impl OwdTrendDetector {
pub fn new(capacity: usize) -> Self {
Self {
samples: VecDeque::with_capacity(capacity),
capacity,
}
}
/// Clear all samples, keeping the same capacity.
pub fn clear(&mut self) {
self.samples.clear();
}
/// Add an OWD sample.
///
/// `seq` is a monotonic sequence number (e.g., truncated frame counter).
/// `owd_us` is the relative one-way delay in microseconds (R_i - S_i).
pub fn push(&mut self, seq: u32, owd_us: i64) {
if self.samples.len() == self.capacity {
self.samples.pop_front();
}
self.samples.push_back((seq, owd_us));
}
/// Compute the OWD trend as a slope in µs/second.
///
/// Uses simple linear regression: slope = Σ((x-x̄)(y-ȳ)) / Σ((x-x̄)²)
/// where x = sequence number and y = owd_us.
///
/// Returns 0 if fewer than 2 samples.
pub fn trend_us_per_sec(&self) -> i32 {
let n = self.samples.len();
if n < 2 {
return 0;
}
let n_f = n as f64;
let sum_x: f64 = self.samples.iter().map(|(s, _)| *s as f64).sum();
let sum_y: f64 = self.samples.iter().map(|(_, y)| *y as f64).sum();
let mean_x = sum_x / n_f;
let mean_y = sum_y / n_f;
let mut num = 0.0;
let mut den = 0.0;
for &(x, y) in &self.samples {
let dx = x as f64 - mean_x;
let dy = y as f64 - mean_y;
num += dx * dy;
den += dx * dx;
}
if den.abs() < f64::EPSILON {
return 0;
}
// slope is in µs/packet. Convert to µs/second assuming ~1ms inter-packet
// spacing as a rough estimate. The raw slope per packet is more useful
// for trend detection than an absolute rate, but the wire format specifies
// µs/s. We report the raw per-packet slope scaled by 1000.
let slope_per_packet = num / den;
(slope_per_packet * 1000.0) as i32
}
pub fn len(&self) -> usize {
self.samples.len()
}
pub fn is_empty(&self) -> bool {
self.samples.is_empty()
}
}
// ============================================================================
// ETX
// ============================================================================
/// Compute Expected Transmission Count from bidirectional delivery ratios.
///
/// ETX = 1 / (d_f × d_r) where d_f and d_r are forward and reverse
/// delivery probabilities (1.0 = perfect, 0.0 = no delivery).
///
/// Clamped to [1.0, 100.0].
pub fn compute_etx(d_forward: f64, d_reverse: f64) -> f64 {
let product = d_forward * d_reverse;
if product <= 0.0 {
return 100.0;
}
(1.0 / product).clamp(1.0, 100.0)
}
// ============================================================================
// Spin Bit
// ============================================================================
/// Spin bit state for passive RTT estimation.
///
/// Uses asymmetric roles (initiator/responder) per the MMP design:
/// - **Initiator**: flips spin value on each received frame; measures RTT
/// from edge-to-edge intervals.
/// - **Responder**: copies received spin bit into outgoing frames, with a
/// counter guard to filter reordered frames.
pub struct SpinBitState {
is_initiator: bool,
current_value: bool,
/// Highest counter observed with a spin edge (responder guard).
highest_counter_for_spin: u64,
/// Time of last spin edge (initiator only, for RTT measurement).
last_edge_time: Option<Instant>,
}
impl SpinBitState {
pub fn new(is_initiator: bool) -> Self {
Self {
is_initiator,
current_value: false,
highest_counter_for_spin: 0,
last_edge_time: None,
}
}
/// Check if this is the spin bit initiator.
pub fn is_initiator(&self) -> bool {
self.is_initiator
}
/// Get the spin bit value to set on an outgoing frame.
pub fn tx_bit(&self) -> bool {
self.current_value
}
/// Process a received frame's spin bit.
///
/// Returns an RTT sample duration if an edge was detected (initiator only).
pub fn rx_observe(
&mut self,
received_bit: bool,
counter: u64,
now: Instant,
) -> Option<std::time::Duration> {
if self.is_initiator {
// Initiator: when the reflected bit matches what we sent,
// that completes a round trip. Record the edge time, then
// flip for the next cycle.
if received_bit == self.current_value {
let rtt = self.last_edge_time.map(|t| now.duration_since(t));
self.last_edge_time = Some(now);
self.current_value = !self.current_value;
rtt
} else {
None
}
} else {
// Responder: copy received bit, but only if counter is higher
// (reordering guard)
if counter > self.highest_counter_for_spin {
self.highest_counter_for_spin = counter;
self.current_value = received_bit;
}
None
}
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_jitter_zero_input() {
let mut j = JitterEstimator::new();
j.update(0);
assert_eq!(j.jitter_us(), 0);
}
#[test]
fn test_jitter_convergence() {
let mut j = JitterEstimator::new();
// Feed constant transit delta of 1000µs
for _ in 0..200 {
j.update(1000);
}
// Should converge near 1000µs
let jitter = j.jitter_us();
assert!(
jitter > 900 && jitter < 1100,
"jitter={jitter}, expected ~1000"
);
}
#[test]
fn test_srtt_first_sample() {
let mut s = SrttEstimator::new();
s.update(10_000); // 10ms
assert_eq!(s.srtt_us(), 10_000);
assert_eq!(s.rttvar_us(), 5_000);
assert!(s.initialized());
}
#[test]
fn test_srtt_convergence() {
let mut s = SrttEstimator::new();
// Feed constant 50ms RTT
for _ in 0..100 {
s.update(50_000);
}
let srtt = s.srtt_us();
assert!((srtt - 50_000).abs() < 1000, "srtt={srtt}, expected ~50000");
}
#[test]
fn test_dual_ewma_initialization() {
let mut e = DualEwma::new();
assert!(!e.initialized());
e.update(100.0);
assert!(e.initialized());
assert_eq!(e.short(), 100.0);
assert_eq!(e.long(), 100.0);
}
#[test]
fn test_dual_ewma_short_tracks_faster() {
let mut e = DualEwma::new();
// Initialize at 0
e.update(0.0);
// Jump to 100
for _ in 0..20 {
e.update(100.0);
}
// Short should be closer to 100 than long
assert!(
e.short() > e.long(),
"short={} long={}",
e.short(),
e.long()
);
}
#[test]
fn test_owd_trend_flat() {
let mut d = OwdTrendDetector::new(32);
for i in 0..20 {
d.push(i, 5000); // constant OWD
}
let trend = d.trend_us_per_sec();
assert_eq!(trend, 0, "flat OWD should have zero trend");
}
#[test]
fn test_owd_trend_increasing() {
let mut d = OwdTrendDetector::new(32);
for i in 0..20 {
d.push(i, 5000 + (i as i64) * 100); // increasing by 100µs per packet
}
let trend = d.trend_us_per_sec();
assert!(
trend > 0,
"increasing OWD should have positive trend, got {trend}"
);
}
#[test]
fn test_owd_trend_insufficient_samples() {
let mut d = OwdTrendDetector::new(32);
d.push(0, 5000);
assert_eq!(d.trend_us_per_sec(), 0);
}
#[test]
fn test_etx_perfect_link() {
assert!((compute_etx(1.0, 1.0) - 1.0).abs() < f64::EPSILON);
}
#[test]
fn test_etx_lossy_link() {
// 10% forward loss, 5% reverse loss
let etx = compute_etx(0.9, 0.95);
assert!(etx > 1.0 && etx < 2.0, "etx={etx}");
}
#[test]
fn test_etx_zero_delivery() {
assert_eq!(compute_etx(0.0, 1.0), 100.0);
assert_eq!(compute_etx(1.0, 0.0), 100.0);
}
#[test]
fn test_spin_bit_initiator_rtt() {
let mut initiator = SpinBitState::new(true);
let mut responder = SpinBitState::new(false);
let t0 = Instant::now();
let t1 = t0 + std::time::Duration::from_millis(10);
let t2 = t0 + std::time::Duration::from_millis(20);
// Initiator sends with spin=false (initial)
let bit_to_send = initiator.tx_bit();
assert!(!bit_to_send);
// Responder receives, copies bit
responder.rx_observe(bit_to_send, 1, t0);
assert!(!responder.tx_bit());
// Responder sends back, initiator receives
let resp_bit = responder.tx_bit();
let rtt1 = initiator.rx_observe(resp_bit, 2, t1);
// First edge: no previous edge to compare
assert!(rtt1.is_none());
// Now initiator's spin flipped to true
let bit2 = initiator.tx_bit();
assert!(bit2);
// Responder receives new bit
responder.rx_observe(bit2, 3, t1);
assert!(responder.tx_bit());
// Responder sends back, initiator receives
let resp_bit2 = responder.tx_bit();
let rtt2 = initiator.rx_observe(resp_bit2, 4, t2);
// Second edge: should produce an RTT sample
assert!(rtt2.is_some());
}
#[test]
fn test_spin_bit_responder_counter_guard() {
let mut responder = SpinBitState::new(false);
// Receive counter=5 with spin=true
responder.rx_observe(true, 5, Instant::now());
assert!(responder.tx_bit());
// Reordered packet with counter=3 and spin=false should be ignored
responder.rx_observe(false, 3, Instant::now());
assert!(responder.tx_bit()); // unchanged
}
}
src/mmp/metrics.rs
-556
View File
@@ -1,556 +0,0 @@
//! MMP derived metrics.
//!
//! `MmpMetrics` processes incoming ReceiverReports (from our peer) and
//! maintains derived metrics: SRTT, loss rate, goodput, ETX, and dual
//! EWMA trend indicators. Updated by the sender side when it receives
//! a ReceiverReport about its own traffic.
use crate::mmp::algorithms::{DualEwma, SrttEstimator, compute_etx};
use crate::mmp::report::ReceiverReport;
use std::time::Instant;
use tracing::trace;
/// Derived MMP metrics, updated from incoming ReceiverReports.
///
/// This lives on the sender side: when we receive a ReceiverReport from
/// our peer describing what they observed about our traffic, we process
/// it here to compute RTT, loss, goodput, and trend indicators.
pub struct MmpMetrics {
/// Smoothed RTT from timestamp echo.
pub srtt: SrttEstimator,
/// Dual EWMA trend detectors.
pub rtt_trend: DualEwma,
pub loss_trend: DualEwma,
pub goodput_trend: DualEwma,
pub jitter_trend: DualEwma,
pub etx_trend: DualEwma,
/// Forward delivery ratio (what fraction of our frames the peer received).
pub delivery_ratio_forward: f64,
/// Reverse delivery ratio (set when we compute from our own receiver state).
pub delivery_ratio_reverse: f64,
/// ETX computed from bidirectional delivery ratios.
pub etx: f64,
/// Smoothed goodput in bytes/sec (forward direction: what the peer received from us).
pub goodput_bps: f64,
// --- State for delta computation ---
/// Previous ReceiverReport's cumulative counters (for computing interval deltas).
prev_rr_cum_packets: u64,
prev_rr_cum_bytes: u64,
prev_rr_highest_counter: u64,
prev_rr_ecn_ce: u32,
prev_rr_reorder: u32,
/// Time of previous ReceiverReport (for goodput rate computation).
prev_rr_time: Option<Instant>,
/// Whether we have a previous ReceiverReport for delta computation.
has_prev_rr: bool,
// --- State for reverse delivery ratio delta computation ---
/// Previous reverse-side cumulative packets received (our receiver state).
prev_reverse_packets: u64,
/// Previous reverse-side highest counter (our receiver state).
prev_reverse_highest: u64,
/// Whether we have a previous reverse-side snapshot for delta computation.
has_prev_reverse: bool,
}
impl MmpMetrics {
/// Reset state derived from ReceiverReport counters for rekey cutover.
///
/// The new session starts with counter 0, so the prev_rr deltas must
/// be reset to avoid computing bogus loss/goodput from the counter
/// discontinuity. RTT (SRTT) is preserved since it remains valid.
pub fn reset_for_rekey(&mut self) {
self.prev_rr_cum_packets = 0;
self.prev_rr_cum_bytes = 0;
self.prev_rr_highest_counter = 0;
self.prev_rr_ecn_ce = 0;
self.prev_rr_reorder = 0;
self.prev_rr_time = None;
self.has_prev_rr = false;
self.delivery_ratio_forward = 1.0;
self.prev_reverse_packets = 0;
self.prev_reverse_highest = 0;
self.has_prev_reverse = false;
// Keep srtt, etx, trends, goodput_bps — they'll refresh from data
}
pub fn new() -> Self {
Self {
srtt: SrttEstimator::new(),
rtt_trend: DualEwma::new(),
loss_trend: DualEwma::new(),
goodput_trend: DualEwma::new(),
jitter_trend: DualEwma::new(),
etx_trend: DualEwma::new(),
delivery_ratio_forward: 1.0,
delivery_ratio_reverse: 1.0,
etx: 1.0,
goodput_bps: 0.0,
prev_rr_cum_packets: 0,
prev_rr_cum_bytes: 0,
prev_rr_highest_counter: 0,
prev_rr_ecn_ce: 0,
prev_rr_reorder: 0,
prev_rr_time: None,
has_prev_rr: false,
prev_reverse_packets: 0,
prev_reverse_highest: 0,
has_prev_reverse: false,
}
}
/// Process an incoming ReceiverReport (from the peer about our traffic).
///
/// `our_timestamp_ms` is the current session-relative time in ms (for RTT).
/// `now` is the current monotonic time (for goodput rate computation).
///
/// Returns `true` if this report produced the first SRTT measurement
/// (transition from uninitialized to initialized).
pub fn process_receiver_report(
&mut self,
rr: &ReceiverReport,
our_timestamp_ms: u32,
now: Instant,
) -> bool {
let had_srtt = self.srtt.initialized();
if self.has_prev_rr {
let counters_regressed = rr.highest_counter < self.prev_rr_highest_counter
|| rr.cumulative_packets_recv < self.prev_rr_cum_packets
|| rr.cumulative_bytes_recv < self.prev_rr_cum_bytes
|| rr.ecn_ce_count < self.prev_rr_ecn_ce
|| rr.cumulative_reorder_count < self.prev_rr_reorder;
let duplicate_counters = rr.highest_counter == self.prev_rr_highest_counter
&& rr.cumulative_packets_recv == self.prev_rr_cum_packets
&& rr.cumulative_bytes_recv == self.prev_rr_cum_bytes
&& rr.ecn_ce_count == self.prev_rr_ecn_ce
&& rr.cumulative_reorder_count == self.prev_rr_reorder;
// Safe to drop: reports are only built after interval data, so
// a fresh report always advances at least one cumulative counter.
if counters_regressed || duplicate_counters {
trace!(
highest_counter = rr.highest_counter,
prev_highest_counter = self.prev_rr_highest_counter,
cumulative_packets_recv = rr.cumulative_packets_recv,
prev_cumulative_packets_recv = self.prev_rr_cum_packets,
cumulative_bytes_recv = rr.cumulative_bytes_recv,
prev_cumulative_bytes_recv = self.prev_rr_cum_bytes,
"Ignoring stale MMP ReceiverReport"
);
return false;
}
}
// --- RTT from timestamp echo ---
// RTT = now - echoed_timestamp - dwell_time
if rr.timestamp_echo > 0 {
let echo_ms = rr.timestamp_echo;
let dwell_ms = u32::from(rr.dwell_time);
let rtt_sample_ms = echo_ms
.checked_add(dwell_ms)
.and_then(|send_done_ms| our_timestamp_ms.checked_sub(send_done_ms));
match rtt_sample_ms {
Some(rtt_ms) if rtt_ms > 0 => {
let rtt_us = (rtt_ms as i64) * 1000;
trace!(
our_ts = our_timestamp_ms,
echo = echo_ms,
dwell = dwell_ms,
rtt_ms = rtt_ms,
srtt_ms = self.srtt.srtt_us() as f64 / 1000.0,
"RTT sample from timestamp echo"
);
self.srtt.update(rtt_us);
self.rtt_trend.update(rtt_us as f64);
}
_ => {
trace!(
our_ts = our_timestamp_ms,
echo = echo_ms,
dwell = dwell_ms,
"Ignoring invalid MMP RTT sample"
);
}
}
}
// --- Loss rate from cumulative counters ---
// Delta: frames the peer should have received vs. actually received
if self.has_prev_rr {
let counter_span = rr
.highest_counter
.saturating_sub(self.prev_rr_highest_counter);
let packets_delta = rr
.cumulative_packets_recv
.saturating_sub(self.prev_rr_cum_packets);
if counter_span > 0 {
let delivery = (packets_delta as f64) / (counter_span as f64);
self.delivery_ratio_forward = delivery.clamp(0.0, 1.0);
let loss_rate = 1.0 - self.delivery_ratio_forward;
self.loss_trend.update(loss_rate);
self.etx = compute_etx(self.delivery_ratio_forward, self.delivery_ratio_reverse);
self.etx_trend.update(self.etx);
}
}
// --- Goodput from cumulative bytes + time delta ---
if self.has_prev_rr {
let bytes_delta = rr
.cumulative_bytes_recv
.saturating_sub(self.prev_rr_cum_bytes);
self.goodput_trend.update(bytes_delta as f64);
// Compute bytes/sec if we have a time reference
if let Some(prev_time) = self.prev_rr_time {
let elapsed = now.duration_since(prev_time);
let secs = elapsed.as_secs_f64();
if secs > 0.0 {
let bps = bytes_delta as f64 / secs;
// EWMA smoothing: α = 1/4
if self.goodput_bps == 0.0 {
self.goodput_bps = bps;
} else {
self.goodput_bps += (bps - self.goodput_bps) * 0.25;
}
}
}
}
// --- Jitter trend ---
self.jitter_trend.update(rr.jitter as f64);
// --- Save for next delta ---
self.prev_rr_cum_packets = rr.cumulative_packets_recv;
self.prev_rr_cum_bytes = rr.cumulative_bytes_recv;
self.prev_rr_highest_counter = rr.highest_counter;
self.prev_rr_ecn_ce = rr.ecn_ce_count;
self.prev_rr_reorder = rr.cumulative_reorder_count;
self.prev_rr_time = Some(now);
self.has_prev_rr = true;
!had_srtt && self.srtt.initialized()
}
/// Update the reverse delivery ratio from our own receiver state.
///
/// Computes a per-interval delta (same as forward ratio) rather than
/// a lifetime cumulative ratio, so ETX responds to recent conditions.
pub fn update_reverse_delivery(&mut self, our_recv_packets: u64, peer_highest: u64) {
if self.has_prev_reverse {
let counter_span = peer_highest.saturating_sub(self.prev_reverse_highest);
let packets_delta = our_recv_packets.saturating_sub(self.prev_reverse_packets);
if counter_span > 0 {
let delivery = (packets_delta as f64) / (counter_span as f64);
self.delivery_ratio_reverse = delivery.clamp(0.0, 1.0);
self.etx = compute_etx(self.delivery_ratio_forward, self.delivery_ratio_reverse);
self.etx_trend.update(self.etx);
}
}
self.prev_reverse_packets = our_recv_packets;
self.prev_reverse_highest = peer_highest;
self.has_prev_reverse = true;
}
/// Current smoothed RTT in milliseconds, or `None` if not yet measured.
pub fn srtt_ms(&self) -> Option<f64> {
if self.srtt.initialized() {
Some(self.srtt.srtt_us() as f64 / 1000.0)
} else {
None
}
}
/// Current loss rate (0.0 = no loss, 1.0 = total loss).
pub fn loss_rate(&self) -> f64 {
1.0 - self.delivery_ratio_forward
}
/// Smoothed loss rate (long-term EWMA), or `None` if not yet initialized.
pub fn smoothed_loss(&self) -> Option<f64> {
if self.loss_trend.initialized() {
Some(self.loss_trend.long())
} else {
None
}
}
/// Smoothed ETX (long-term EWMA), or `None` if not yet initialized.
pub fn smoothed_etx(&self) -> Option<f64> {
if self.etx_trend.initialized() {
Some(self.etx_trend.long())
} else {
None
}
}
/// Current smoothed goodput in bytes/sec, or 0 if not yet measured.
pub fn goodput_bps(&self) -> f64 {
self.goodput_bps
}
/// Cumulative ECN CE count from the most recent ReceiverReport.
pub fn last_ecn_ce_count(&self) -> u32 {
self.prev_rr_ecn_ce
}
}
impl Default for MmpMetrics {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
use std::time::Duration;
fn make_rr(
highest_counter: u64,
cum_packets: u64,
cum_bytes: u64,
timestamp_echo: u32,
dwell: u16,
jitter: u32,
) -> ReceiverReport {
ReceiverReport {
highest_counter,
cumulative_packets_recv: cum_packets,
cumulative_bytes_recv: cum_bytes,
timestamp_echo,
dwell_time: dwell,
max_burst_loss: 0,
mean_burst_loss: 0,
jitter,
ecn_ce_count: 0,
owd_trend: 0,
burst_loss_count: 0,
cumulative_reorder_count: 0,
interval_packets_recv: 0,
interval_bytes_recv: 0,
}
}
#[test]
fn test_rtt_from_echo() {
let mut m = MmpMetrics::new();
let now = Instant::now();
// Peer echoes timestamp 1000ms, dwell=5ms, our current time=1050ms
let rr = make_rr(10, 10, 5000, 1000, 5, 0);
m.process_receiver_report(&rr, 1050, now);
assert!(m.srtt.initialized());
// RTT = 1050 - 1000 - 5 = 45ms
let srtt_ms = m.srtt_ms().unwrap();
assert!((srtt_ms - 45.0).abs() < 1.0, "srtt={srtt_ms}, expected ~45");
}
#[test]
fn test_ignores_duplicate_receiver_report_after_valid_sample() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
let rr1 = make_rr(10, 10, 5_000, 1_000, 5, 0);
m.process_receiver_report(&rr1, 1_050, t0);
let rr2 = make_rr(20, 18, 14_000, 1_100, 5, 0);
m.process_receiver_report(&rr2, 1_150, t0 + Duration::from_secs(1));
let baseline_srtt_ms = m.srtt_ms().unwrap();
let baseline_loss = m.loss_rate();
let baseline_goodput = m.goodput_bps();
assert!(baseline_loss > 0.0);
assert!(baseline_goodput > 0.0);
// A duplicate of the same counters arriving later would be a 4.895s
// RTT sample if accepted. It is stale and must not move metrics.
m.process_receiver_report(&rr2, 6_000, t0 + Duration::from_secs(5));
assert_eq!(m.srtt_ms().unwrap(), baseline_srtt_ms);
assert_eq!(m.loss_rate(), baseline_loss);
assert_eq!(m.goodput_bps(), baseline_goodput);
}
#[test]
fn test_ignores_out_of_order_receiver_report_after_valid_sample() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let valid_rr = make_rr(20, 20, 10000, 1000, 5, 0);
m.process_receiver_report(&valid_rr, 1050, now);
let baseline_srtt_ms = m.srtt_ms().unwrap();
let old_rr = make_rr(10, 10, 5000, 1000, 0, 0);
m.process_receiver_report(&old_rr, 6000, now + Duration::from_secs(5));
let srtt_ms = m.srtt_ms().unwrap();
assert_eq!(srtt_ms, baseline_srtt_ms);
}
#[test]
fn test_ignores_wrapped_rtt_sample() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let wrapped_rr = make_rr(10, 10, 5000, u32::MAX - 10, 20, 0);
m.process_receiver_report(&wrapped_rr, 15, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_ignores_future_rtt_sample() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let future_rr = make_rr(10, 10, 5_000, 2_000, 5, 0);
m.process_receiver_report(&future_rr, 1_000, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_loss_rate_computation() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
// First report: baseline
let rr1 = make_rr(100, 100, 50000, 0, 0, 0);
m.process_receiver_report(&rr1, 0, t0);
// Second report: 200 counters sent, 190 received (5% loss)
let rr2 = make_rr(300, 290, 145000, 0, 0, 0);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
let loss = m.loss_rate();
assert!((loss - 0.05).abs() < 0.01, "loss={loss}, expected ~0.05");
}
#[test]
fn test_etx_updates() {
let mut m = MmpMetrics::new();
assert_eq!(m.etx, 1.0); // initial: perfect
// Simulate some loss via forward ratio
m.delivery_ratio_forward = 0.9;
// First call establishes the baseline (no ETX update yet)
m.update_reverse_delivery(100, 100);
assert_eq!(m.etx, 1.0); // still perfect — baseline only
// Second call: 190 of 200 frames received (5% loss)
m.update_reverse_delivery(290, 300);
assert!(m.etx > 1.0);
assert!(m.etx < 2.0);
}
#[test]
fn test_no_rtt_without_echo() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let rr = make_rr(10, 10, 5000, 0, 0, 0);
m.process_receiver_report(&rr, 1000, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_jitter_trend() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
let rr1 = make_rr(10, 10, 5000, 0, 0, 100);
m.process_receiver_report(&rr1, 0, t0);
let rr2 = make_rr(20, 20, 10000, 0, 0, 500);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
assert!(m.jitter_trend.initialized());
// Short-term should be closer to 500 than long-term
assert!(m.jitter_trend.short() > m.jitter_trend.long());
}
#[test]
fn test_goodput_bps() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
// First report: baseline (50KB received)
let rr1 = make_rr(100, 100, 50_000, 0, 0, 0);
m.process_receiver_report(&rr1, 0, t0);
assert_eq!(m.goodput_bps(), 0.0); // no rate yet (first report)
// Second report 1s later: 150KB total (100KB delta in 1s = 100KB/s)
let rr2 = make_rr(300, 290, 150_000, 0, 0, 0);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
assert!(
m.goodput_bps() > 90_000.0,
"goodput={}, expected ~100000",
m.goodput_bps()
);
assert!(
m.goodput_bps() < 110_000.0,
"goodput={}, expected ~100000",
m.goodput_bps()
);
}
#[test]
fn test_reverse_delivery_delta() {
let mut m = MmpMetrics::new();
// First call: baseline only, no ratio update
m.update_reverse_delivery(100, 100);
assert_eq!(m.delivery_ratio_reverse, 1.0); // unchanged from default
// Second call: perfect delivery (200 new frames, all received)
m.update_reverse_delivery(300, 300);
assert!((m.delivery_ratio_reverse - 1.0).abs() < 0.001);
// Third call: 50% loss (100 frames sent, 50 received)
m.update_reverse_delivery(350, 400);
assert!(
(m.delivery_ratio_reverse - 0.5).abs() < 0.001,
"reverse={}, expected 0.5",
m.delivery_ratio_reverse
);
}
#[test]
fn test_reverse_delivery_rekey_reset() {
let mut m = MmpMetrics::new();
// Establish baseline and one measurement
m.update_reverse_delivery(100, 100);
m.update_reverse_delivery(300, 300);
assert!((m.delivery_ratio_reverse - 1.0).abs() < 0.001);
// Rekey resets reverse state
m.reset_for_rekey();
// First call after rekey: baseline only
m.update_reverse_delivery(50, 50);
// delivery_ratio_reverse was reset to 1.0 by reset_for_rekey's
// clearing of delivery_ratio_forward; reverse is not explicitly
// reset — but the delta state is, so next call computes fresh.
assert_eq!(m.delivery_ratio_reverse, 1.0);
// Second call after rekey: 80% delivery
m.update_reverse_delivery(90, 100);
assert!(
(m.delivery_ratio_reverse - 0.8).abs() < 0.001,
"reverse={}, expected 0.8",
m.delivery_ratio_reverse
);
}
}
src/mmp/mod.rs
-555
View File
@@ -1,555 +0,0 @@
//! Metrics Measurement Protocol (MMP) — link-layer instantiation.
//!
//! Measures link quality between adjacent peers: RTT, loss, jitter,
//! throughput, one-way delay trend, and ETX. Operates on the per-frame
//! hooks (counter, timestamp, flags) introduced by the FMP wire format
//! revision.
//!
//! Three operating modes trade measurement fidelity for overhead:
//! - **Full**: sender + receiver reports at RTT-adaptive intervals
//! - **Lightweight**: receiver reports only (infer loss from counters)
//! - **Minimal**: spin bit + CE echo only, no reports
use serde::{Deserialize, Serialize};
use std::fmt::{self, Debug};
use std::time::{Duration, Instant};
// Sub-modules
pub mod algorithms;
pub mod metrics;
pub mod receiver;
pub mod report;
pub mod sender;
// Re-exports
pub use algorithms::{
DualEwma, JitterEstimator, OwdTrendDetector, SpinBitState, SrttEstimator, compute_etx,
};
pub use metrics::MmpMetrics;
pub use receiver::ReceiverState;
pub use report::{ReceiverReport, SenderReport};
pub use sender::SenderState;
// Session-layer re-exports
// MmpSessionState and PathMtuState are defined in this file
// ============================================================================
// Constants
// ============================================================================
/// SenderReport body size (after msg_type byte): 3 reserved + 44 payload = 47.
pub const SENDER_REPORT_BODY_SIZE: usize = 47;
/// ReceiverReport body size (after msg_type byte): 3 reserved + 64 payload = 67.
pub const RECEIVER_REPORT_BODY_SIZE: usize = 67;
/// SenderReport total wire size including inner header: 5 + 47 = 52.
pub const SENDER_REPORT_WIRE_SIZE: usize = 52;
/// ReceiverReport total wire size including inner header: 5 + 67 = 72.
pub const RECEIVER_REPORT_WIRE_SIZE: usize = 72;
// --- EWMA parameters (as shift amounts for integer arithmetic) ---
/// Jitter EWMA: α = 1/16 (RFC 3550 §6.4.1).
pub const JITTER_ALPHA_SHIFT: u32 = 4;
/// SRTT: α = 1/8 (Jacobson, RFC 6298).
pub const SRTT_ALPHA_SHIFT: u32 = 3;
/// RTTVAR: β = 1/4 (Jacobson, RFC 6298).
pub const RTTVAR_BETA_SHIFT: u32 = 2;
/// Dual EWMA short-term: α = 1/4.
pub const EWMA_SHORT_ALPHA: f64 = 0.25;
/// Dual EWMA long-term: α = 1/32.
pub const EWMA_LONG_ALPHA: f64 = 1.0 / 32.0;
// --- Timing defaults (milliseconds) ---
/// Default report interval before SRTT is available (cold start).
pub const DEFAULT_COLD_START_INTERVAL_MS: u64 = 200;
/// Minimum report interval (SRTT clamp floor).
///
/// Raised from 100ms to 1000ms: parent re-evaluation runs every 60s,
/// so 60 samples/cycle is more than sufficient for EWMA convergence (~10).
/// The cold-start phase uses `DEFAULT_COLD_START_INTERVAL_MS` (200ms) for
/// fast initial SRTT convergence before transitioning to this floor.
pub const MIN_REPORT_INTERVAL_MS: u64 = 1_000;
/// Maximum report interval (SRTT clamp ceiling).
pub const MAX_REPORT_INTERVAL_MS: u64 = 5_000;
/// Number of SRTT samples before transitioning from cold-start to normal floor.
///
/// During cold-start, report intervals use `DEFAULT_COLD_START_INTERVAL_MS` as
/// the floor to gather SRTT samples quickly. After this many updates, the floor
/// switches to `MIN_REPORT_INTERVAL_MS`.
pub const COLD_START_SAMPLES: u32 = 5;
/// Default OWD ring buffer capacity.
pub const DEFAULT_OWD_WINDOW_SIZE: usize = 32;
/// Default operator log interval in seconds.
pub const DEFAULT_LOG_INTERVAL_SECS: u64 = 30;
// --- Session-layer timing defaults ---
// Session reports are routed end-to-end (bandwidth cost on every transit link),
// so intervals are higher than link-layer.
/// Session-layer minimum report interval.
pub const MIN_SESSION_REPORT_INTERVAL_MS: u64 = 500;
/// Session-layer maximum report interval.
pub const MAX_SESSION_REPORT_INTERVAL_MS: u64 = 10_000;
/// Session-layer cold-start report interval (before SRTT is available).
pub const SESSION_COLD_START_INTERVAL_MS: u64 = 1_000;
// ============================================================================
// Operating Mode
// ============================================================================
/// MMP operating mode.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum MmpMode {
/// Sender + receiver reports at RTT-adaptive intervals. Maximum fidelity.
#[default]
Full,
/// Receiver reports only. Loss inferred from counter gaps.
Lightweight,
/// Spin bit + CE echo only. No reports exchanged.
Minimal,
}
impl fmt::Display for MmpMode {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
MmpMode::Full => write!(f, "full"),
MmpMode::Lightweight => write!(f, "lightweight"),
MmpMode::Minimal => write!(f, "minimal"),
}
}
}
// ============================================================================
// Configuration
// ============================================================================
/// MMP configuration (`node.mmp.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MmpConfig {
/// Operating mode (`node.mmp.mode`).
#[serde(default)]
pub mode: MmpMode,
/// Periodic operator log interval in seconds (`node.mmp.log_interval_secs`).
#[serde(default = "MmpConfig::default_log_interval_secs")]
pub log_interval_secs: u64,
/// OWD trend ring buffer size (`node.mmp.owd_window_size`).
#[serde(default = "MmpConfig::default_owd_window_size")]
pub owd_window_size: usize,
}
impl Default for MmpConfig {
fn default() -> Self {
Self {
mode: MmpMode::default(),
log_interval_secs: DEFAULT_LOG_INTERVAL_SECS,
owd_window_size: DEFAULT_OWD_WINDOW_SIZE,
}
}
}
impl MmpConfig {
fn default_log_interval_secs() -> u64 {
DEFAULT_LOG_INTERVAL_SECS
}
fn default_owd_window_size() -> usize {
DEFAULT_OWD_WINDOW_SIZE
}
}
// ============================================================================
// Per-Peer MMP State
// ============================================================================
/// Combined MMP state for a single peer link.
///
/// Wraps sender, receiver, metrics, and spin bit state. One instance
/// per `ActivePeer`.
pub struct MmpPeerState {
pub sender: SenderState,
pub receiver: ReceiverState,
pub metrics: MmpMetrics,
pub spin_bit: SpinBitState,
mode: MmpMode,
log_interval: Duration,
last_log_time: Option<Instant>,
}
impl MmpPeerState {
/// Create MMP state for a new peer link.
///
/// `is_initiator`: true if this node initiated the Noise handshake
/// (determines spin bit role).
pub fn new(config: &MmpConfig, is_initiator: bool) -> Self {
Self {
sender: SenderState::new(),
receiver: ReceiverState::new(config.owd_window_size),
metrics: MmpMetrics::new(),
spin_bit: SpinBitState::new(is_initiator),
mode: config.mode,
log_interval: Duration::from_secs(config.log_interval_secs),
last_log_time: None,
}
}
/// Reset counter-dependent state for rekey cutover.
pub fn reset_for_rekey(&mut self, now: Instant) {
self.receiver.reset_for_rekey(now);
self.metrics.reset_for_rekey();
}
/// Current operating mode.
pub fn mode(&self) -> MmpMode {
self.mode
}
/// Check if it's time to emit a periodic metrics log.
pub fn should_log(&self, now: Instant) -> bool {
match self.last_log_time {
None => true,
Some(last) => now.duration_since(last) >= self.log_interval,
}
}
/// Mark that a periodic log was emitted.
pub fn mark_logged(&mut self, now: Instant) {
self.last_log_time = Some(now);
}
}
// ============================================================================
// Per-Session MMP State (session-layer instantiation)
// ============================================================================
/// Combined MMP state for a single end-to-end session.
///
/// Wraps sender, receiver, metrics, spin bit, and path MTU state.
/// One instance per established `SessionEntry`.
pub struct MmpSessionState {
pub sender: SenderState,
pub receiver: ReceiverState,
pub metrics: MmpMetrics,
pub spin_bit: SpinBitState,
mode: MmpMode,
log_interval: Duration,
last_log_time: Option<Instant>,
pub path_mtu: PathMtuState,
}
impl MmpSessionState {
/// Create MMP state for a new session.
///
/// `is_initiator`: true if this node initiated the Noise handshake
/// (determines spin bit role).
pub fn new(config: &crate::config::SessionMmpConfig, is_initiator: bool) -> Self {
Self {
sender: SenderState::new_with_cold_start(SESSION_COLD_START_INTERVAL_MS),
receiver: ReceiverState::new_with_cold_start(
config.owd_window_size,
SESSION_COLD_START_INTERVAL_MS,
),
metrics: MmpMetrics::new(),
spin_bit: SpinBitState::new(is_initiator),
mode: config.mode,
log_interval: Duration::from_secs(config.log_interval_secs),
last_log_time: None,
path_mtu: PathMtuState::new(),
}
}
/// Reset counter-dependent state for rekey cutover.
pub fn reset_for_rekey(&mut self, now: Instant) {
self.receiver.reset_for_rekey(now);
self.metrics.reset_for_rekey();
}
/// Current operating mode.
pub fn mode(&self) -> MmpMode {
self.mode
}
/// Check if it's time to emit a periodic metrics log.
pub fn should_log(&self, now: Instant) -> bool {
match self.last_log_time {
None => true,
Some(last) => now.duration_since(last) >= self.log_interval,
}
}
/// Mark that a periodic log was emitted.
pub fn mark_logged(&mut self, now: Instant) {
self.last_log_time = Some(now);
}
}
impl Debug for MmpSessionState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("MmpSessionState")
.field("mode", &self.mode)
.field("path_mtu", &self.path_mtu.current_mtu())
.finish_non_exhaustive()
}
}
// ============================================================================
// Path MTU State (session-layer only)
// ============================================================================
/// Path MTU tracking for a single session.
///
/// Destination side: observes `path_mtu` from incoming SessionDatagram envelopes
/// and generates PathMtuNotification messages back to the source.
///
/// Source side: applies received PathMtuNotification to limit outbound datagram
/// size. Decrease is immediate; increase requires 3 consecutive notifications.
pub struct PathMtuState {
/// Current effective path MTU (what we use for sending).
current_mtu: u16,
/// Last observed path MTU from incoming datagrams (destination-side).
last_observed_mtu: u16,
/// Whether the observed MTU has changed since the last notification.
observed_changed: bool,
/// Last time a PathMtuNotification was sent.
last_notification_time: Option<Instant>,
/// Notification interval: max(10s, 5 * SRTT). Default 10s.
notification_interval: Duration,
/// For source-side increase tracking: consecutive higher-value notifications.
consecutive_increase_count: u8,
/// Time of the first notification in the current increase sequence.
first_increase_time: Option<Instant>,
/// The MTU value being proposed for increase.
pending_increase_mtu: u16,
}
impl PathMtuState {
/// Create path MTU state with no initial measurement.
pub fn new() -> Self {
Self {
current_mtu: u16::MAX,
last_observed_mtu: u16::MAX,
observed_changed: false,
last_notification_time: None,
notification_interval: Duration::from_secs(10),
consecutive_increase_count: 0,
first_increase_time: None,
pending_increase_mtu: 0,
}
}
/// Current effective path MTU (source-side, for sending).
pub fn current_mtu(&self) -> u16 {
self.current_mtu
}
/// Last observed incoming path MTU (destination-side).
pub fn last_observed_mtu(&self) -> u16 {
self.last_observed_mtu
}
/// Update notification interval from SRTT: max(10s, 5 * SRTT).
pub fn update_interval_from_srtt(&mut self, srtt_ms: f64) {
let five_srtt = Duration::from_millis((srtt_ms * 5.0) as u64);
self.notification_interval = five_srtt.max(Duration::from_secs(10));
}
/// Seed source-side current_mtu from outbound transport MTU.
///
/// Called on each send. Only decreases (never increases) the current_mtu
/// so the destination's PathMtuNotification can still raise it later.
/// Ensures current_mtu doesn't stay at u16::MAX before any notification
/// arrives from the destination.
pub fn seed_source_mtu(&mut self, outbound_mtu: u16) {
if outbound_mtu < self.current_mtu {
self.current_mtu = outbound_mtu;
}
}
// --- Destination side ---
/// Observe the path_mtu from an incoming SessionDatagram envelope.
///
/// Called on the destination (receiver) side for every session message.
pub fn observe_incoming_mtu(&mut self, path_mtu: u16) {
if path_mtu != self.last_observed_mtu {
self.observed_changed = true;
self.last_observed_mtu = path_mtu;
}
}
/// Check if a PathMtuNotification should be sent.
///
/// Send on first measurement, on decrease (immediate), or periodic
/// confirmation at the notification interval.
pub fn should_send_notification(&self, now: Instant) -> bool {
if self.last_observed_mtu == u16::MAX {
return false; // No measurement yet
}
match self.last_notification_time {
None => true, // First measurement
Some(last) => {
// Immediate on decrease
if self.observed_changed && self.last_observed_mtu < self.current_mtu {
return true;
}
// Periodic confirmation
now.duration_since(last) >= self.notification_interval
}
}
}
/// Build a PathMtuNotification from current state.
///
/// Returns the path_mtu value to send. Caller handles encoding.
pub fn build_notification(&mut self, now: Instant) -> Option<u16> {
if self.last_observed_mtu == u16::MAX {
return None;
}
self.last_notification_time = Some(now);
self.observed_changed = false;
Some(self.last_observed_mtu)
}
// --- Source side ---
/// Apply a received PathMtuNotification.
///
/// - Decrease: immediate (take the lower value).
/// - Increase: require 3 consecutive notifications with the same higher
/// value, spanning at least 2 * notification_interval.
///
/// Returns `true` if the effective MTU changed.
pub fn apply_notification(&mut self, reported_mtu: u16, now: Instant) -> bool {
if reported_mtu < self.current_mtu {
// Decrease: immediate
self.current_mtu = reported_mtu;
self.consecutive_increase_count = 0;
self.first_increase_time = None;
return true;
}
if reported_mtu > self.current_mtu {
// Increase: track consecutive notifications
if reported_mtu == self.pending_increase_mtu {
self.consecutive_increase_count += 1;
} else {
// Different value: reset sequence
self.pending_increase_mtu = reported_mtu;
self.consecutive_increase_count = 1;
self.first_increase_time = Some(now);
}
// Accept increase after 3 consecutive spanning 2 * interval
if self.consecutive_increase_count >= 3
&& let Some(first_time) = self.first_increase_time
{
let required = self.notification_interval * 2;
if now.duration_since(first_time) >= required {
self.current_mtu = reported_mtu;
self.consecutive_increase_count = 0;
self.first_increase_time = None;
return true;
}
}
}
// No change (equal or increase not yet confirmed)
false
}
}
impl Default for PathMtuState {
fn default() -> Self {
Self::new()
}
}
impl Debug for MmpPeerState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("MmpPeerState")
.field("mode", &self.mode)
.finish_non_exhaustive()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_mode_default() {
assert_eq!(MmpMode::default(), MmpMode::Full);
}
#[test]
fn test_mode_display() {
assert_eq!(MmpMode::Full.to_string(), "full");
assert_eq!(MmpMode::Lightweight.to_string(), "lightweight");
assert_eq!(MmpMode::Minimal.to_string(), "minimal");
}
#[test]
fn test_mode_serde_roundtrip() {
let yaml = "full";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Full);
let yaml = "lightweight";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Lightweight);
let yaml = "minimal";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Minimal);
}
#[test]
fn test_config_default() {
let config = MmpConfig::default();
assert_eq!(config.mode, MmpMode::Full);
assert_eq!(config.log_interval_secs, 30);
assert_eq!(config.owd_window_size, 32);
}
#[test]
fn test_config_yaml_parse() {
let yaml = r#"
mode: lightweight
log_interval_secs: 60
owd_window_size: 48
"#;
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Lightweight);
assert_eq!(config.log_interval_secs, 60);
assert_eq!(config.owd_window_size, 48);
}
#[test]
fn test_config_yaml_partial() {
let yaml = "mode: minimal";
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Minimal);
assert_eq!(config.log_interval_secs, DEFAULT_LOG_INTERVAL_SECS);
assert_eq!(config.owd_window_size, DEFAULT_OWD_WINDOW_SIZE);
}
}
src/mmp/report.rs
-385
View File
@@ -1,385 +0,0 @@
//! MMP report wire format: SenderReport and ReceiverReport.
//!
//! Serialization and deserialization for the two report types exchanged
//! between link-layer peers. Wire format follows the MMP design doc.
use crate::protocol::ProtocolError;
// ============================================================================
// SenderReport (msg_type 0x01, 48-byte body including type byte)
// ============================================================================
/// Link-layer sender report.
///
/// Wire layout (48 bytes total, sent as link message):
/// ```text
/// [0] msg_type = 0x01
/// [1-3] reserved (zero)
/// [4-11] interval_start_counter: u64 LE
/// [12-19] interval_end_counter: u64 LE
/// [20-23] interval_start_timestamp: u32 LE
/// [24-27] interval_end_timestamp: u32 LE
/// [28-31] interval_bytes_sent: u32 LE
/// [32-39] cumulative_packets_sent: u64 LE
/// [40-47] cumulative_bytes_sent: u64 LE
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SenderReport {
pub interval_start_counter: u64,
pub interval_end_counter: u64,
pub interval_start_timestamp: u32,
pub interval_end_timestamp: u32,
pub interval_bytes_sent: u32,
pub cumulative_packets_sent: u64,
pub cumulative_bytes_sent: u64,
}
/// ReceiverReport (msg_type 0x02, 68-byte body including type byte)
///
/// Wire layout (68 bytes total, sent as link message):
/// ```text
/// [0] msg_type = 0x02
/// [1-3] reserved (zero)
/// [4-11] highest_counter: u64 LE
/// [12-19] cumulative_packets_recv: u64 LE
/// [20-27] cumulative_bytes_recv: u64 LE
/// [28-31] timestamp_echo: u32 LE
/// [32-33] dwell_time: u16 LE
/// [34-35] max_burst_loss: u16 LE
/// [36-37] mean_burst_loss: u16 LE (u8.8 fixed-point)
/// [38-39] reserved: u16 LE
/// [40-43] jitter: u32 LE (microseconds)
/// [44-47] ecn_ce_count: u32 LE
/// [48-51] owd_trend: i32 LE (µs/s)
/// [52-55] burst_loss_count: u32 LE
/// [56-59] cumulative_reorder_count: u32 LE
/// [60-63] interval_packets_recv: u32 LE
/// [64-67] interval_bytes_recv: u32 LE
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ReceiverReport {
pub highest_counter: u64,
pub cumulative_packets_recv: u64,
pub cumulative_bytes_recv: u64,
pub timestamp_echo: u32,
pub dwell_time: u16,
pub max_burst_loss: u16,
pub mean_burst_loss: u16,
pub jitter: u32,
pub ecn_ce_count: u32,
pub owd_trend: i32,
pub burst_loss_count: u32,
pub cumulative_reorder_count: u32,
pub interval_packets_recv: u32,
pub interval_bytes_recv: u32,
}
// Encode/decode will be implemented in Step 2.
impl SenderReport {
/// Encode to wire format (48 bytes: msg_type + 3 reserved + 44 payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(48);
buf.push(0x01); // msg_type
buf.extend_from_slice(&[0u8; 3]); // reserved
buf.extend_from_slice(&self.interval_start_counter.to_le_bytes());
buf.extend_from_slice(&self.interval_end_counter.to_le_bytes());
buf.extend_from_slice(&self.interval_start_timestamp.to_le_bytes());
buf.extend_from_slice(&self.interval_end_timestamp.to_le_bytes());
buf.extend_from_slice(&self.interval_bytes_sent.to_le_bytes());
buf.extend_from_slice(&self.cumulative_packets_sent.to_le_bytes());
buf.extend_from_slice(&self.cumulative_bytes_sent.to_le_bytes());
buf
}
/// Decode from payload after msg_type byte has been consumed.
///
/// `payload` starts at the reserved bytes (offset 1 in the wire format).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
if payload.len() < 47 {
return Err(ProtocolError::MessageTooShort {
expected: 47,
got: payload.len(),
});
}
// Skip 3 reserved bytes
let p = &payload[3..];
Ok(Self {
interval_start_counter: u64::from_le_bytes(p[0..8].try_into().unwrap()),
interval_end_counter: u64::from_le_bytes(p[8..16].try_into().unwrap()),
interval_start_timestamp: u32::from_le_bytes(p[16..20].try_into().unwrap()),
interval_end_timestamp: u32::from_le_bytes(p[20..24].try_into().unwrap()),
interval_bytes_sent: u32::from_le_bytes(p[24..28].try_into().unwrap()),
cumulative_packets_sent: u64::from_le_bytes(p[28..36].try_into().unwrap()),
cumulative_bytes_sent: u64::from_le_bytes(p[36..44].try_into().unwrap()),
})
}
}
impl ReceiverReport {
/// Encode to wire format (68 bytes: msg_type + 3 reserved + 64 payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(68);
buf.push(0x02); // msg_type
buf.extend_from_slice(&[0u8; 3]); // reserved
buf.extend_from_slice(&self.highest_counter.to_le_bytes());
buf.extend_from_slice(&self.cumulative_packets_recv.to_le_bytes());
buf.extend_from_slice(&self.cumulative_bytes_recv.to_le_bytes());
buf.extend_from_slice(&self.timestamp_echo.to_le_bytes());
buf.extend_from_slice(&self.dwell_time.to_le_bytes());
buf.extend_from_slice(&self.max_burst_loss.to_le_bytes());
buf.extend_from_slice(&self.mean_burst_loss.to_le_bytes());
buf.extend_from_slice(&[0u8; 2]); // reserved
buf.extend_from_slice(&self.jitter.to_le_bytes());
buf.extend_from_slice(&self.ecn_ce_count.to_le_bytes());
buf.extend_from_slice(&self.owd_trend.to_le_bytes());
buf.extend_from_slice(&self.burst_loss_count.to_le_bytes());
buf.extend_from_slice(&self.cumulative_reorder_count.to_le_bytes());
buf.extend_from_slice(&self.interval_packets_recv.to_le_bytes());
buf.extend_from_slice(&self.interval_bytes_recv.to_le_bytes());
buf
}
/// Decode from payload after msg_type byte has been consumed.
///
/// `payload` starts at the reserved bytes (offset 1 in the wire format).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
if payload.len() < 67 {
return Err(ProtocolError::MessageTooShort {
expected: 67,
got: payload.len(),
});
}
// Skip 3 reserved bytes
let p = &payload[3..];
Ok(Self {
highest_counter: u64::from_le_bytes(p[0..8].try_into().unwrap()),
cumulative_packets_recv: u64::from_le_bytes(p[8..16].try_into().unwrap()),
cumulative_bytes_recv: u64::from_le_bytes(p[16..24].try_into().unwrap()),
timestamp_echo: u32::from_le_bytes(p[24..28].try_into().unwrap()),
dwell_time: u16::from_le_bytes(p[28..30].try_into().unwrap()),
max_burst_loss: u16::from_le_bytes(p[30..32].try_into().unwrap()),
mean_burst_loss: u16::from_le_bytes(p[32..34].try_into().unwrap()),
// skip 2 reserved bytes at p[34..36]
jitter: u32::from_le_bytes(p[36..40].try_into().unwrap()),
ecn_ce_count: u32::from_le_bytes(p[40..44].try_into().unwrap()),
owd_trend: i32::from_le_bytes(p[44..48].try_into().unwrap()),
burst_loss_count: u32::from_le_bytes(p[48..52].try_into().unwrap()),
cumulative_reorder_count: u32::from_le_bytes(p[52..56].try_into().unwrap()),
interval_packets_recv: u32::from_le_bytes(p[56..60].try_into().unwrap()),
interval_bytes_recv: u32::from_le_bytes(p[60..64].try_into().unwrap()),
})
}
}
// ============================================================================
// Conversions between link-layer and session-layer report types
// ============================================================================
use crate::protocol::{SessionReceiverReport, SessionSenderReport};
impl From<&SenderReport> for SessionSenderReport {
fn from(r: &SenderReport) -> Self {
Self {
interval_start_counter: r.interval_start_counter,
interval_end_counter: r.interval_end_counter,
interval_start_timestamp: r.interval_start_timestamp,
interval_end_timestamp: r.interval_end_timestamp,
interval_bytes_sent: r.interval_bytes_sent,
cumulative_packets_sent: r.cumulative_packets_sent,
cumulative_bytes_sent: r.cumulative_bytes_sent,
}
}
}
impl From<&SessionSenderReport> for SenderReport {
fn from(r: &SessionSenderReport) -> Self {
Self {
interval_start_counter: r.interval_start_counter,
interval_end_counter: r.interval_end_counter,
interval_start_timestamp: r.interval_start_timestamp,
interval_end_timestamp: r.interval_end_timestamp,
interval_bytes_sent: r.interval_bytes_sent,
cumulative_packets_sent: r.cumulative_packets_sent,
cumulative_bytes_sent: r.cumulative_bytes_sent,
}
}
}
impl From<&ReceiverReport> for SessionReceiverReport {
fn from(r: &ReceiverReport) -> Self {
Self {
highest_counter: r.highest_counter,
cumulative_packets_recv: r.cumulative_packets_recv,
cumulative_bytes_recv: r.cumulative_bytes_recv,
timestamp_echo: r.timestamp_echo,
dwell_time: r.dwell_time,
max_burst_loss: r.max_burst_loss,
mean_burst_loss: r.mean_burst_loss,
jitter: r.jitter,
ecn_ce_count: r.ecn_ce_count,
owd_trend: r.owd_trend,
burst_loss_count: r.burst_loss_count,
cumulative_reorder_count: r.cumulative_reorder_count,
interval_packets_recv: r.interval_packets_recv,
interval_bytes_recv: r.interval_bytes_recv,
}
}
}
impl From<&SessionReceiverReport> for ReceiverReport {
fn from(r: &SessionReceiverReport) -> Self {
Self {
highest_counter: r.highest_counter,
cumulative_packets_recv: r.cumulative_packets_recv,
cumulative_bytes_recv: r.cumulative_bytes_recv,
timestamp_echo: r.timestamp_echo,
dwell_time: r.dwell_time,
max_burst_loss: r.max_burst_loss,
mean_burst_loss: r.mean_burst_loss,
jitter: r.jitter,
ecn_ce_count: r.ecn_ce_count,
owd_trend: r.owd_trend,
burst_loss_count: r.burst_loss_count,
cumulative_reorder_count: r.cumulative_reorder_count,
interval_packets_recv: r.interval_packets_recv,
interval_bytes_recv: r.interval_bytes_recv,
}
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
fn sample_sender_report() -> SenderReport {
SenderReport {
interval_start_counter: 100,
interval_end_counter: 200,
interval_start_timestamp: 5000,
interval_end_timestamp: 6000,
interval_bytes_sent: 50_000,
cumulative_packets_sent: 10_000,
cumulative_bytes_sent: 5_000_000,
}
}
fn sample_receiver_report() -> ReceiverReport {
ReceiverReport {
highest_counter: 195,
cumulative_packets_recv: 9_500,
cumulative_bytes_recv: 4_750_000,
timestamp_echo: 5900,
dwell_time: 5,
max_burst_loss: 3,
mean_burst_loss: 384, // 1.5 in u8.8
jitter: 1200,
ecn_ce_count: 0,
owd_trend: -50,
burst_loss_count: 2,
cumulative_reorder_count: 10,
interval_packets_recv: 95,
interval_bytes_recv: 47_500,
}
}
#[test]
fn test_sender_report_encode_size() {
let sr = sample_sender_report();
let encoded = sr.encode();
assert_eq!(encoded.len(), 48);
assert_eq!(encoded[0], 0x01); // msg_type
}
#[test]
fn test_sender_report_roundtrip() {
let sr = sample_sender_report();
let encoded = sr.encode();
// decode expects payload after msg_type
let decoded = SenderReport::decode(&encoded[1..]).unwrap();
assert_eq!(sr, decoded);
}
#[test]
fn test_sender_report_too_short() {
let result = SenderReport::decode(&[0u8; 10]);
assert!(result.is_err());
}
#[test]
fn test_receiver_report_encode_size() {
let rr = sample_receiver_report();
let encoded = rr.encode();
assert_eq!(encoded.len(), 68);
assert_eq!(encoded[0], 0x02); // msg_type
}
#[test]
fn test_receiver_report_roundtrip() {
let rr = sample_receiver_report();
let encoded = rr.encode();
// decode expects payload after msg_type
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(rr, decoded);
}
#[test]
fn test_receiver_report_too_short() {
let result = ReceiverReport::decode(&[0u8; 10]);
assert!(result.is_err());
}
#[test]
fn test_sender_report_zero_values() {
let sr = SenderReport {
interval_start_counter: 0,
interval_end_counter: 0,
interval_start_timestamp: 0,
interval_end_timestamp: 0,
interval_bytes_sent: 0,
cumulative_packets_sent: 0,
cumulative_bytes_sent: 0,
};
let encoded = sr.encode();
let decoded = SenderReport::decode(&encoded[1..]).unwrap();
assert_eq!(sr, decoded);
}
#[test]
fn test_receiver_report_max_values() {
let rr = ReceiverReport {
highest_counter: u64::MAX,
cumulative_packets_recv: u64::MAX,
cumulative_bytes_recv: u64::MAX,
timestamp_echo: u32::MAX,
dwell_time: u16::MAX,
max_burst_loss: u16::MAX,
mean_burst_loss: u16::MAX,
jitter: u32::MAX,
ecn_ce_count: u32::MAX,
owd_trend: i32::MAX,
burst_loss_count: u32::MAX,
cumulative_reorder_count: u32::MAX,
interval_packets_recv: u32::MAX,
interval_bytes_recv: u32::MAX,
};
let encoded = rr.encode();
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(rr, decoded);
}
#[test]
fn test_receiver_report_negative_owd_trend() {
let rr = ReceiverReport {
owd_trend: -12345,
..sample_receiver_report()
};
let encoded = rr.encode();
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(decoded.owd_trend, -12345);
}
}
src/mmp/sender.rs
-418
View File
@@ -1,418 +0,0 @@
//! MMP sender state machine.
//!
//! Tracks what this node has sent to a specific peer and produces
//! SenderReport messages on demand. One `SenderState` per active peer.
use std::time::{Duration, Instant};
use crate::mmp::report::SenderReport;
use crate::mmp::{
COLD_START_SAMPLES, DEFAULT_COLD_START_INTERVAL_MS, MAX_REPORT_INTERVAL_MS,
MIN_REPORT_INTERVAL_MS,
};
/// Per-peer sender-side MMP state.
///
/// Records cumulative and interval counters for every frame transmitted
/// to this peer. Produces `SenderReport` snapshots on demand.
pub struct SenderState {
// --- Cumulative (lifetime) ---
cumulative_packets_sent: u64,
cumulative_bytes_sent: u64,
// --- Current interval ---
interval_start_counter: u64,
interval_start_timestamp: u32,
interval_bytes_sent: u32,
/// Counter of the most recently sent frame.
last_counter: u64,
/// Timestamp of the most recently sent frame.
last_timestamp: u32,
/// Whether any frames have been sent in the current interval.
interval_has_data: bool,
// --- Report timing ---
last_report_time: Option<Instant>,
report_interval: Duration,
// --- Send failure backoff ---
/// Consecutive send failure count for backoff calculation.
consecutive_send_failures: u32,
// --- Cold-start tracking ---
/// Number of SRTT-based interval updates received.
srtt_sample_count: u32,
}
impl SenderState {
pub fn new() -> Self {
Self::new_with_cold_start(DEFAULT_COLD_START_INTERVAL_MS)
}
/// Create with a custom cold-start interval (ms).
///
/// Used by session-layer MMP which needs a longer initial interval
/// since reports consume bandwidth on every transit link.
pub fn new_with_cold_start(cold_start_ms: u64) -> Self {
Self {
cumulative_packets_sent: 0,
cumulative_bytes_sent: 0,
interval_start_counter: 0,
interval_start_timestamp: 0,
interval_bytes_sent: 0,
last_counter: 0,
last_timestamp: 0,
interval_has_data: false,
last_report_time: None,
report_interval: Duration::from_millis(cold_start_ms),
consecutive_send_failures: 0,
srtt_sample_count: 0,
}
}
/// Record a frame sent to this peer.
///
/// Called on the TX path for every encrypted link message.
/// `counter` is the AEAD nonce/counter, `timestamp` is the inner header
/// session-relative timestamp (ms), `bytes` is the wire payload size.
pub fn record_sent(&mut self, counter: u64, timestamp: u32, bytes: usize) {
if !self.interval_has_data {
self.interval_start_counter = counter;
self.interval_start_timestamp = timestamp;
self.interval_has_data = true;
}
self.last_counter = counter;
self.last_timestamp = timestamp;
self.interval_bytes_sent = self.interval_bytes_sent.saturating_add(bytes as u32);
self.cumulative_packets_sent += 1;
self.cumulative_bytes_sent += bytes as u64;
}
/// Build a SenderReport from current state and reset the interval.
///
/// Returns `None` if no frames have been sent since the last report.
pub fn build_report(&mut self, now: Instant) -> Option<SenderReport> {
if !self.interval_has_data {
return None;
}
let report = SenderReport {
interval_start_counter: self.interval_start_counter,
interval_end_counter: self.last_counter,
interval_start_timestamp: self.interval_start_timestamp,
interval_end_timestamp: self.last_timestamp,
interval_bytes_sent: self.interval_bytes_sent,
cumulative_packets_sent: self.cumulative_packets_sent,
cumulative_bytes_sent: self.cumulative_bytes_sent,
};
// Reset interval
self.interval_has_data = false;
self.interval_bytes_sent = 0;
self.last_report_time = Some(now);
Some(report)
}
/// Check if it's time to send a report.
///
/// When consecutive send failures have occurred, the effective interval
/// is multiplied by an exponential backoff factor (2^failures, capped at 32×).
pub fn should_send_report(&self, now: Instant) -> bool {
if !self.interval_has_data {
return false;
}
match self.last_report_time {
None => true, // Never sent a report — send immediately
Some(last) => {
let effective = self
.report_interval
.mul_f64(self.send_failure_backoff_multiplier());
now.duration_since(last) >= effective
}
}
}
/// Record a send failure. Returns the new consecutive failure count.
pub fn record_send_failure(&mut self) -> u32 {
self.consecutive_send_failures += 1;
self.consecutive_send_failures
}
/// Record a successful send. Returns the previous failure count (for summary logging).
pub fn record_send_success(&mut self) -> u32 {
let prev = self.consecutive_send_failures;
self.consecutive_send_failures = 0;
prev
}
/// Get the backoff multiplier based on consecutive failures.
///
/// Returns 1.0 for no failures, 2.0 for 1 failure, 4.0 for 2, ...
/// capped at 32.0 (5 failures).
pub fn send_failure_backoff_multiplier(&self) -> f64 {
if self.consecutive_send_failures == 0 {
1.0
} else {
2.0_f64.powi(self.consecutive_send_failures.min(5) as i32)
}
}
/// Update the report interval based on SRTT (link-layer defaults).
///
/// Sender reports at 2× SRTT clamped to [floor, MAX]. During cold-start
/// (first `COLD_START_SAMPLES` updates), the floor is the cold-start
/// interval (200ms) for fast SRTT convergence. After that, it rises to
/// `MIN_REPORT_INTERVAL_MS` (1000ms) for steady-state efficiency.
pub fn update_report_interval_from_srtt(&mut self, srtt_us: i64) {
self.srtt_sample_count = self.srtt_sample_count.saturating_add(1);
let floor = if self.srtt_sample_count <= COLD_START_SAMPLES {
DEFAULT_COLD_START_INTERVAL_MS
} else {
MIN_REPORT_INTERVAL_MS
};
self.update_report_interval_with_bounds(srtt_us, floor, MAX_REPORT_INTERVAL_MS);
}
/// Update the report interval based on SRTT with custom bounds.
///
/// Used by session-layer MMP which needs higher clamp values since
/// each report consumes bandwidth on every transit link.
pub fn update_report_interval_with_bounds(&mut self, srtt_us: i64, min_ms: u64, max_ms: u64) {
if srtt_us <= 0 {
return;
}
let interval_us = (srtt_us * 2) as u64;
let interval_ms = (interval_us / 1000).clamp(min_ms, max_ms);
self.report_interval = Duration::from_millis(interval_ms);
}
// --- Accessors ---
pub fn cumulative_packets_sent(&self) -> u64 {
self.cumulative_packets_sent
}
pub fn cumulative_bytes_sent(&self) -> u64 {
self.cumulative_bytes_sent
}
pub fn report_interval(&self) -> Duration {
self.report_interval
}
pub fn consecutive_send_failures(&self) -> u32 {
self.consecutive_send_failures
}
}
impl Default for SenderState {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_sender_state() {
let s = SenderState::new();
assert_eq!(s.cumulative_packets_sent(), 0);
assert_eq!(s.cumulative_bytes_sent(), 0);
}
#[test]
fn test_record_sent() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
s.record_sent(2, 200, 600);
assert_eq!(s.cumulative_packets_sent(), 2);
assert_eq!(s.cumulative_bytes_sent(), 1100);
}
#[test]
fn test_build_report_empty() {
let mut s = SenderState::new();
assert!(s.build_report(Instant::now()).is_none());
}
#[test]
fn test_build_report() {
let mut s = SenderState::new();
s.record_sent(10, 1000, 500);
s.record_sent(11, 1100, 600);
s.record_sent(12, 1200, 400);
let report = s.build_report(Instant::now()).unwrap();
assert_eq!(report.interval_start_counter, 10);
assert_eq!(report.interval_end_counter, 12);
assert_eq!(report.interval_start_timestamp, 1000);
assert_eq!(report.interval_end_timestamp, 1200);
assert_eq!(report.interval_bytes_sent, 1500);
assert_eq!(report.cumulative_packets_sent, 3);
assert_eq!(report.cumulative_bytes_sent, 1500);
}
#[test]
fn test_build_report_resets_interval() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
let _ = s.build_report(Instant::now());
// Second report with no new data returns None
assert!(s.build_report(Instant::now()).is_none());
// New data starts a fresh interval
s.record_sent(2, 200, 300);
let report = s.build_report(Instant::now()).unwrap();
assert_eq!(report.interval_start_counter, 2);
assert_eq!(report.interval_bytes_sent, 300);
// Cumulative continues
assert_eq!(report.cumulative_packets_sent, 2);
assert_eq!(report.cumulative_bytes_sent, 800);
}
#[test]
fn test_should_send_report_no_data() {
let s = SenderState::new();
assert!(!s.should_send_report(Instant::now()));
}
#[test]
fn test_should_send_report_first_time() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
assert!(s.should_send_report(Instant::now()));
}
#[test]
fn test_should_send_report_respects_interval() {
let mut s = SenderState::new();
let t0 = Instant::now();
s.record_sent(1, 100, 500);
let _ = s.build_report(t0);
s.record_sent(2, 200, 500);
// Immediately after report — should not send
assert!(!s.should_send_report(t0));
// After interval elapses
let t1 = t0 + s.report_interval() + Duration::from_millis(1);
assert!(s.should_send_report(t1));
}
#[test]
fn test_update_report_interval_cold_start() {
let mut s = SenderState::new();
// During cold-start, floor is 200ms (DEFAULT_COLD_START_INTERVAL_MS)
// 50ms RTT → 100ms sender interval (2× SRTT), clamped to cold-start floor 200ms
s.update_report_interval_from_srtt(50_000);
assert_eq!(s.report_interval(), Duration::from_millis(200));
// 500ms RTT → 1000ms sender interval (above cold-start floor)
s.update_report_interval_from_srtt(500_000);
assert_eq!(s.report_interval(), Duration::from_millis(1000));
}
#[test]
fn test_update_report_interval_after_cold_start() {
let mut s = SenderState::new();
// Burn through cold-start samples (COLD_START_SAMPLES = 5)
for _ in 0..COLD_START_SAMPLES {
s.update_report_interval_from_srtt(500_000);
}
// 6th sample: now in steady state, floor is MIN_REPORT_INTERVAL_MS (1000ms)
// 50ms RTT → 100ms sender interval (2× SRTT), clamped to 1000ms
s.update_report_interval_from_srtt(50_000);
assert_eq!(
s.report_interval(),
Duration::from_millis(MIN_REPORT_INTERVAL_MS)
);
// 3s RTT → 6s, clamped to max 5s
s.update_report_interval_from_srtt(3_000_000);
assert_eq!(
s.report_interval(),
Duration::from_millis(MAX_REPORT_INTERVAL_MS)
);
}
#[test]
fn test_backoff_multiplier_progression() {
let mut s = SenderState::new();
// No failures → multiplier 1.0
assert_eq!(s.send_failure_backoff_multiplier(), 1.0);
assert_eq!(s.consecutive_send_failures(), 0);
// Progressive failures: 2^1, 2^2, 2^3, 2^4, 2^5
let expected = [2.0, 4.0, 8.0, 16.0, 32.0];
for (i, &exp) in expected.iter().enumerate() {
let count = s.record_send_failure();
assert_eq!(count, (i + 1) as u32);
assert_eq!(s.send_failure_backoff_multiplier(), exp);
}
// Beyond 5 failures: stays capped at 32.0
s.record_send_failure(); // 6th
assert_eq!(s.send_failure_backoff_multiplier(), 32.0);
s.record_send_failure(); // 7th
assert_eq!(s.send_failure_backoff_multiplier(), 32.0);
}
#[test]
fn test_backoff_reset_on_success() {
let mut s = SenderState::new();
// Accumulate failures
s.record_send_failure();
s.record_send_failure();
s.record_send_failure();
assert_eq!(s.consecutive_send_failures(), 3);
assert_eq!(s.send_failure_backoff_multiplier(), 8.0);
// Success resets and returns previous count
let prev = s.record_send_success();
assert_eq!(prev, 3);
assert_eq!(s.consecutive_send_failures(), 0);
assert_eq!(s.send_failure_backoff_multiplier(), 1.0);
}
#[test]
fn test_backoff_success_with_no_prior_failures() {
let mut s = SenderState::new();
// Success with no failures returns 0
let prev = s.record_send_success();
assert_eq!(prev, 0);
assert_eq!(s.consecutive_send_failures(), 0);
}
#[test]
fn test_should_send_report_respects_backoff() {
let mut s = SenderState::new();
let t0 = Instant::now();
s.record_sent(1, 100, 500);
let _ = s.build_report(t0);
// Record a failure: multiplier becomes 2.0
s.record_send_failure();
s.record_sent(2, 200, 500);
// At 1× interval: should NOT send (backoff requires 2×)
let t1 = t0 + s.report_interval() + Duration::from_millis(1);
assert!(!s.should_send_report(t1));
// At 2× interval: should send
let t2 = t0 + s.report_interval() * 2 + Duration::from_millis(1);
assert!(s.should_send_report(t2));
}
}
src/node/bloom.rs
+7 -7
View File
@@ -4,12 +4,12 @@
//! including debounced propagation to peers.
use crate::NodeAddr;
use crate::bloom::BloomFilter;
use crate::protocol::FilterAnnounce;
use crate::proto::bloom::BloomFilter;
use crate::proto::bloom::FilterAnnounce;
use super::reject::BloomReject;
use super::{Node, NodeError};
use std::collections::HashMap;
use std::collections::BTreeMap;
use tracing::{debug, warn};
impl Node {
@@ -17,8 +17,8 @@ impl Node {
///
/// Returns a map of (peer_node_addr -> filter) for peers that
/// have sent us a FilterAnnounce.
pub(super) fn peer_inbound_filters(&self) -> HashMap<NodeAddr, BloomFilter> {
let mut filters = HashMap::new();
pub(super) fn peer_inbound_filters(&self) -> BTreeMap<NodeAddr, BloomFilter> {
let mut filters = BTreeMap::new();
for (addr, peer) in &self.peers {
if self.is_tree_peer(addr)
&& let Some(filter) = peer.inbound_filter()
@@ -79,7 +79,7 @@ impl Node {
// operator to see one clear message, not spam.
let max_fpr = self.config().node.bloom.max_inbound_fpr;
let out_fill = sent_filter.fill_ratio();
let out_fpr = out_fill.powi(sent_filter.hash_count() as i32);
let out_fpr = sent_filter.fpr();
if out_fpr > max_fpr {
let now = std::time::Instant::now();
let should_warn = self
@@ -221,7 +221,7 @@ impl Node {
// to wipe a victim's contribution to aggregation.
let max_fpr = self.config().node.bloom.max_inbound_fpr;
let fill = announce.filter.fill_ratio();
let fpr = fill.powi(announce.filter.hash_count() as i32);
let fpr = announce.filter.fpr();
if fpr > max_fpr {
self.metrics()
.bloom
src/node/context.rs
+6
View File
@@ -15,6 +15,7 @@
use std::sync::Arc;
use crate::proto::fmp::NodeProfile;
use crate::{Config, Identity};
/// Effectively-immutable `Node` state, shared via `Arc<NodeContext>`.
@@ -36,6 +37,9 @@ pub(crate) struct NodeContext {
/// Whether this is a leaf-only node.
pub is_leaf_only: bool,
/// This node's routing profile (Full, NonRouting, Leaf).
pub node_profile: NodeProfile,
/// Maximum connections (0 = unlimited).
pub max_connections: usize,
@@ -55,6 +59,7 @@ impl NodeContext {
startup_epoch: [u8; 8],
started_at: std::time::Instant,
is_leaf_only: bool,
node_profile: NodeProfile,
max_connections: usize,
max_peers: usize,
max_links: usize,
@@ -65,6 +70,7 @@ impl NodeContext {
startup_epoch,
started_at,
is_leaf_only,
node_profile,
max_connections,
max_peers,
max_links,
src/node/handlers/connected_udp.rs → src/node/dataplane/connected_udp.rs
+18 -16
View File
@@ -45,12 +45,10 @@ impl Node {
/// (e.g. only non-UDP transports). Enabled on Linux and macOS:
/// both kernels route a matching peer 5-tuple to the connected
/// socket when it shares the wildcard listen port via SO_REUSEPORT.
/// Only compiled on Linux/macOS — the sole caller (the rx_loop tick) is
/// gated the same way, so on other targets (android) there is nothing to do.
#[cfg(any(target_os = "linux", target_os = "macos"))]
pub(in crate::node) async fn activate_connected_udp_sessions(&mut self) {
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
{
// No-op on platforms without the connected-UDP fast path.
}
#[cfg(any(target_os = "linux", target_os = "macos"))]
{
if !connected_udp_enabled() {
return;
@@ -142,20 +140,24 @@ impl Node {
(peer_sa, local, recv_buf, send_buf, tx)
};
// Open the connected socket on the kernel side.
let socket = std::sync::Arc::new(
crate::transport::udp::connected_peer::ConnectedPeerSocket::open(
local_addr,
peer_socket_addr,
recv_buf,
send_buf,
)
.map_err(|e| format!("ConnectedPeerSocket::open: {e}"))?,
);
// Open the connected socket on the kernel side, then adopt the
// fd into the owning handle.
let owned = crate::transport::udp::open_connected_fd(
local_addr,
peer_socket_addr,
recv_buf,
send_buf,
)
.map_err(|e| format!("open_connected_fd: {e}"))?;
let socket = std::sync::Arc::new(crate::peer::connected_udp::ConnectedPeerSocket::from_fd(
owned,
peer_socket_addr,
local_addr,
));
// Spawn the drain thread. It feeds `packet_tx` exactly like
// the wildcard listen socket — rx_loop dispatches identically.
let drain = crate::transport::udp::peer_drain::PeerRecvDrain::spawn(
let drain = crate::peer::connected_udp::PeerRecvDrain::spawn(
socket.clone(),
transport_id,
peer_socket_addr,
src/node/handlers/dispatch.rs → src/node/dataplane/dispatch.rs
+9 -3
View File
@@ -73,7 +73,7 @@ impl Node {
/// entries — other removal paths (link-dead, decrypt failure, peer
/// restart) all schedule reconnect.
pub(in crate::node) fn handle_disconnect(&mut self, from: &NodeAddr, payload: &[u8]) {
let disconnect = match crate::protocol::Disconnect::decode(payload) {
let disconnect = match crate::proto::fmp::Disconnect::decode(payload) {
Ok(msg) => msg,
Err(e) => {
debug!(from = %self.peer_display_name(from), error = %e, "Malformed disconnect message");
@@ -93,7 +93,7 @@ impl Node {
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
self.schedule_reconnect(addr, now_ms);
self.note_link_dead(addr, now_ms);
}
/// Remove an active peer and clean up all associated state.
@@ -187,6 +187,11 @@ impl Node {
// Remove link and address mapping
self.remove_link(&link_id);
// Drop this peer's inert machine, keyed by the link_id
// derived above (a peer's link_id is immutable, so the key never
// moved). Keeps peers <-> peer_machines in exact correspondence on
// teardown. NEUTRAL: nothing reads peer_machines yet.
self.remove_peer_machine(link_id);
if let Some(transport_id) = transport_id {
self.cleanup_bootstrap_transport_if_unused(transport_id);
}
@@ -201,7 +206,8 @@ impl Node {
}
}
// Bloom filter cleanup: clear state for removed peer, mark all remaining peers
// Bloom filter cleanup: remove dependent (non-routing/leaf peers), clear state
self.bloom_state.remove_leaf_dependent(node_addr);
self.bloom_state.remove_peer_state(node_addr);
let remaining_peers: Vec<NodeAddr> = self.peers.keys().copied().collect();
self.bloom_state.mark_all_updates_needed(remaining_peers);
src/node/handlers/encrypted.rs → src/node/dataplane/encrypted.rs
+55 -45
View File
@@ -1,10 +1,9 @@
//! Encrypted frame handling (hot path).
use crate::node::Node;
use crate::node::wire::{EncryptedHeader, FLAG_CE, FLAG_KEY_EPOCH, FLAG_SP, strip_inner_header};
use crate::noise::NoiseError;
use crate::proto::fmp::wire::{EncryptedHeader, FLAG_CE, FLAG_KEY_EPOCH, strip_inner_header};
use crate::transport::ReceivedPacket;
use std::time::Instant;
use tracing::{debug, trace, warn};
/// Force-remove a peer after this many consecutive decryption failures.
@@ -58,15 +57,11 @@ impl Node {
// actually belongs to rekey N+1. Promoting on the bare bit then
// installs the WRONG Noise session as current — the two endpoints
// diverge, every subsequent frame fails AEAD on the far side, the
// receiver starves, and the link is declared dead at the heartbeat
// timeout (routing failure, green crypto). This mirrors the FSP fix
// (node/session.rs / node/handlers/session.rs): the authenticated
// decrypt, not the header bit, is the cutover signal. Trial-decrypt
// the frame against `pending` first; only promote if it
// authenticates. On success the same frame is delivered via
// `process_authentic_fmp_plaintext` and we return — it must not
// fall through to a second decrypt, which would be rejected as a
// replay (the trial-decrypt already advanced `pending`'s window).
// receiver starves, and the link is declared dead at the 30s
// heartbeat timeout (Phase-5 routing failure, green crypto). This
// mirrors the FSP fix (node/session.rs): the authenticated decrypt,
// not the header bit, is the cutover signal. Trial-decrypt the
// frame against `pending` first; only promote if it authenticates.
{
let Some(peer) = self.peers.get(&node_addr) else {
return;
@@ -77,12 +72,11 @@ impl Node {
if k_bit_flipped {
let ciphertext = &packet.data[header.ciphertext_offset()..];
let display_name = self.peer_display_name(&node_addr);
let our_addr = *self.identity().node_addr();
let Some(peer) = self.peers.get_mut(&node_addr) else {
return;
};
// Authenticate the frame against the pending session.
// Trial-decrypt mutates `pending`'s replay window only on
// success, so a failed trial leaves it untouched.
let pending_plaintext = peer.pending_new_session_mut().and_then(|pending| {
pending
.decrypt_with_replay_check_and_aad(
@@ -94,19 +88,27 @@ impl Node {
});
if let Some(plaintext) = pending_plaintext {
let pending_our = peer.pending_our_index();
let pending_their = peer.pending_their_index();
debug!(
peer = %display_name,
our_addr = %our_addr,
their_addr = %node_addr,
pending_our_index = ?pending_our,
pending_their_index = ?pending_their,
"Peer new-epoch frame authenticated, K-bit flip promoting new session"
);
// The peer authenticated a frame on the new epoch, so it
// derived the new session (it received our rekey msg3).
// If we are the rekey initiator still retransmitting msg3,
// stop — the responder is confirmed. (No-op for the
// responder side, which never retained a msg3 payload.)
peer.clear_rekey_msg3_payload();
// The trial-decrypt already advanced the pending
// session's replay window; `handle_peer_kbit_flip`
// moves that same session object to `current`, so no
// re-decrypt.
// session's replay window; handle_peer_kbit_flip moves
// that same session object to current, so no re-decrypt.
let did_flip = peer.handle_peer_kbit_flip().is_some();
if did_flip {
// New index was pre-registered in peers_by_index
// during msg1 handling (handshake.rs). Verify,
// don't duplicate.
debug_assert!(
peer.transport_id().is_some()
&& peer.our_index().is_some()
@@ -117,13 +119,8 @@ impl Node {
"peers_by_index should contain pre-registered new index after K-bit flip"
);
}
// Re-register the (now-promoted) session with the
// decrypt worker: cache_key = (transport_id, our_index)
// changed at the flip, so the old worker entry is
// stranded and every packet on the new session would
// miss the worker's HashMap lookup. Without this,
// throughput drops back to the inline-decrypt path
// after each rekey.
// Re-register the promoted session with the decrypt
// worker (cache_key changed at the flip).
#[cfg(unix)]
if did_flip {
self.register_decrypt_worker_session(&node_addr);
@@ -133,7 +130,6 @@ impl Node {
// canonical post-decrypt path, then return — it must
// not fall through to a second decrypt attempt.
let ce_flag = header.flags & FLAG_CE != 0;
let sp_flag = header.flags & FLAG_SP != 0;
self.process_authentic_fmp_plaintext(
&node_addr,
packet.transport_id,
@@ -142,7 +138,6 @@ impl Node {
packet.data.len(),
header.counter,
ce_flag,
sp_flag,
&plaintext,
)
.await;
@@ -171,7 +166,7 @@ impl Node {
#[cfg(unix)]
{
let cache_key = (packet.transport_id, header.receiver_idx.as_u32());
if let Some(workers) = self.decrypt_workers.as_ref().cloned()
if let Some(workers) = self.supervisor.decrypt_workers.as_ref().cloned()
&& self.decrypt_registered_sessions.contains(&cache_key)
{
let job = crate::node::decrypt_worker::DecryptJob {
@@ -195,7 +190,9 @@ impl Node {
// Decrypt: try current session first, then previous (drain fallback)
let ciphertext = &packet.data[header.ciphertext_offset()..];
let plaintext = {
let peer = self.peers.get_mut(&node_addr).unwrap();
let Some(peer) = self.peers.get_mut(&node_addr) else {
return;
};
let session = match peer.noise_session_mut() {
Some(s) => s,
None => {
@@ -259,20 +256,25 @@ impl Node {
};
// MMP per-frame processing and statistics
let now = Instant::now();
let now_ms = crate::time::mono_ms();
let ce_flag = header.flags & FLAG_CE != 0;
let sp_flag = header.flags & FLAG_SP != 0;
if let Some(peer) = self.peers.get_mut(&node_addr) {
// Initiator-side msg3 confirm (see process_authentic_fmp_plaintext):
// a frame authenticated against post-cutover `current` (no pending)
// proves the responder reached the new epoch. Inline-decrypt path
// mirror of the worker-bounce confirm.
if peer.rekey_msg3_payload().is_some() && peer.pending_new_session().is_none() {
peer.clear_rekey_msg3_payload();
}
if let Some(mmp) = peer.mmp_mut() {
mmp.receiver.record_recv(
header.counter,
timestamp,
packet.data.len(),
ce_flag,
now,
now_ms,
);
let _spin_rtt = mmp.spin_bit.rx_observe(sp_flag, header.counter, now);
}
peer.set_current_addr(packet.transport_id, packet.remote_addr.clone());
peer.link_stats_mut()
@@ -328,8 +330,8 @@ impl Node {
/// Canonical post-FMP-decrypt side-effect site. Used by both the
/// inline rx_loop decrypt path and the decrypt-worker bounce path
/// so the per-peer bookkeeping (stats, MMP, spin-bit RTT, ECN
/// propagation, address-rotation handling, link-message dispatch)
/// so the per-peer bookkeeping (stats, MMP, ECN propagation,
/// address-rotation handling, link-message dispatch)
/// happens in exactly one place.
#[allow(clippy::too_many_arguments)]
pub(in crate::node) async fn process_authentic_fmp_plaintext(
@@ -341,7 +343,6 @@ impl Node {
packet_len: usize,
fmp_counter: u64,
ce_flag: bool,
sp_flag: bool,
fmp_plaintext: &[u8],
) {
const INNER_TIMESTAMP_LEN: usize = 4;
@@ -355,18 +356,29 @@ impl Node {
} else {
return;
};
let now = Instant::now();
let now_ms = crate::time::mono_ms();
let mut address_changed = false;
if let Some(peer) = self.peers.get_mut(node_addr) {
peer.reset_decrypt_failures();
// If we are the rekey initiator that already cut over on its
// own timer (no `pending`, `current` is the new session) but
// still retain a msg3 retransmission payload, an authenticated
// peer frame here decrypts against the post-cutover `current`
// session — proof the responder reached the new epoch. Stop
// retransmitting. Mirrors the FSP Current-slot confirm in
// handle_encrypted_session_msg. Works in both the inline and
// worker-bounce paths since both funnel through here, and the
// only session registered for the new index is the new one.
if peer.rekey_msg3_payload().is_some() && peer.pending_new_session().is_none() {
peer.clear_rekey_msg3_payload();
}
address_changed = peer.set_current_addr(transport_id, remote_addr.clone());
peer.link_stats_mut()
.record_recv(packet_len, packet_timestamp_ms);
peer.touch(packet_timestamp_ms);
if let Some(mmp) = peer.mmp_mut() {
mmp.receiver
.record_recv(fmp_counter, inner_ts, packet_len, ce_flag, now);
let _spin_rtt = mmp.spin_bit.rx_observe(sp_flag, fmp_counter, now);
.record_recv(fmp_counter, inner_ts, packet_len, ce_flag, now_ms);
}
}
// Address rotation invalidates the per-peer connect()-ed UDP
@@ -393,7 +405,6 @@ impl Node {
fallback: crate::node::decrypt_worker::DecryptFallback,
) {
let ce_flag = fallback.fmp_flags & FLAG_CE != 0;
let sp_flag = fallback.fmp_flags & FLAG_SP != 0;
let plaintext = &fallback.packet_data[fallback.fmp_plaintext_offset
..fallback.fmp_plaintext_offset + fallback.fmp_plaintext_len];
self.process_authentic_fmp_plaintext(
@@ -404,7 +415,6 @@ impl Node {
fallback.packet_len,
fallback.fmp_counter,
ce_flag,
sp_flag,
plaintext,
)
.await;
@@ -450,7 +460,7 @@ impl Node {
/// black-hole the session.
#[cfg(unix)]
pub(in crate::node) fn register_decrypt_worker_session(&mut self, node_addr: &crate::NodeAddr) {
let Some(workers) = self.decrypt_workers.as_ref().cloned() else {
let Some(workers) = self.supervisor.decrypt_workers.as_ref().cloned() else {
return;
};
let (cache_key, state) = {
@@ -491,7 +501,7 @@ impl Node {
&mut self,
cache_key: (crate::transport::TransportId, u32),
) {
if let Some(workers) = self.decrypt_workers.as_ref() {
if let Some(workers) = self.supervisor.decrypt_workers.as_ref() {
workers.unregister_session(cache_key);
}
self.decrypt_registered_sessions.remove(&cache_key);
@@ -533,7 +543,7 @@ impl Node {
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
self.schedule_reconnect(addr, now_ms);
self.note_link_dead(addr, now_ms);
}
}
}
src/node/handlers/forwarding.rs → src/node/dataplane/forwarding.rs
+221 -162
View File
@@ -2,21 +2,21 @@
//!
//! Handles incoming SessionDatagram (0x00) link messages: decodes the
//! envelope, performs coordinate cache warming from plaintext session-layer
//! headers, delivers locally when the datagram is addressed to this node,
//! otherwise enforces the transit hop limit and routes to the next hop, and
//! generates error signals on routing failure.
//! headers, pre-resolves the next hop for forwardable transit datagrams, and
//! drives the routing core's outcome — local delivery when the datagram is
//! addressed to this node, the transit hop-limit drop, the forward, or the
//! error signal generated on routing failure.
use crate::NodeAddr;
use crate::node::reject::ForwardingReject;
use crate::node::session_wire::{
use crate::node::{Node, NodeError, NodeRoutingView};
use crate::proto::fsp::wire::{
FSP_COMMON_PREFIX_SIZE, FSP_HEADER_SIZE, FSP_PHASE_ESTABLISHED, FSP_PHASE_MSG1, FSP_PHASE_MSG2,
FspCommonPrefix, parse_encrypted_coords,
};
use crate::node::{Node, NodeError};
use crate::protocol::{
CoordsRequired, MtuExceeded, PathBroken, SessionAck, SessionDatagram, SessionDatagramRef,
SessionSetup,
};
use crate::proto::fsp::{SessionAck, SessionSetup};
use crate::proto::link::{SessionDatagram, SessionDatagramRef};
use crate::proto::routing::{DropReason, NextHop, RouteAction, RouteOutcome};
use std::time::{Duration, Instant};
use tracing::{debug, warn};
@@ -44,131 +44,169 @@ impl Node {
}
};
// Coordinate cache warming from plaintext session-layer headers.
// Runs ahead of both the delivery and the TTL decisions: the coords
// a peer put on the wire are equally valid whichever way those go.
let my_addr = *self.node_addr();
// Coordinate cache warming from plaintext session-layer headers. Runs
// ahead of both the delivery and the TTL decisions the core makes: the
// coords a peer put on the wire are equally valid whichever way those
// go, and the only arrivals this newly warms from are those with an
// exhausted TTL, whose every insert is already achievable at TTL 1.
self.try_warm_coord_cache_ref(&datagram_ref);
// Local delivery: dispatch to session layer handlers without
// materializing an owned SessionDatagram payload Vec. Delivery to
// the addressed node is *not* TTL-gated — under IP semantics the
// TTL governs forwarding, not delivery to the addressed host — so
// this test precedes the TTL gate below.
if datagram_ref.dest_addr == *self.node_addr() {
self.metrics().forwarding.record_delivered(payload.len());
self.handle_session_payload(
&datagram_ref.src_addr,
datagram_ref.payload,
datagram_ref.path_mtu,
incoming_ce,
)
.await;
return;
}
// Pre-resolve the next hop only for datagrams the core can actually
// forward: not locally destined, and carrying a TTL that survives the
// decrement (`ttl > 1` — the shell-side mirror of the core's
// would-leave-zero drop). This keeps `find_next_hop`'s coord-cache
// LRU-touch side effect scoped to genuine forwards, as it was when the
// TTL test ran inline ahead of it. Warming above has already run, so
// the resolution observes freshly cached coords.
let next_hop = if datagram_ref.dest_addr != my_addr && datagram_ref.ttl > 1 {
self.resolve_next_hop(&datagram_ref.dest_addr)
} else {
None
};
// TTL enforcement on the transit path: decrement first, then drop if
// the datagram would leave with a TTL of zero. `saturating_sub` folds
// the already-exhausted arrival (ttl=0) into the same test as the
// last-hop arrival (ttl=1); neither is transmitted.
let forwarded_ttl = datagram_ref.ttl.saturating_sub(1);
if forwarded_ttl == 0 {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::TtlExhausted, payload.len());
debug!(
src = %datagram_ref.src_addr,
dest = %datagram_ref.dest_addr,
ttl = datagram_ref.ttl,
"SessionDatagram TTL exhausted, dropping"
);
return;
}
// Read local congestion once and reuse it for both the CE decision
// (via the view) and the congestion metric/log below, keeping
// `detect_congestion` the single source of truth.
let congested = next_hop
.as_ref()
.map(|nh| self.detect_congestion(&nh.addr))
.unwrap_or(false);
let mut datagram = datagram_ref.into_owned();
datagram.ttl = forwarded_ttl;
// Borrow the routing tables disjointly from `&mut self.routing` for
// the pure decision, then release both before driving the outcome.
let outcome = {
let view = NodeRoutingView {
coord_cache: &self.coord_cache,
peers: &self.peers,
tree_state: &self.tree_state,
congested,
};
self.routing
.route(&datagram_ref, &my_addr, incoming_ce, next_hop, &view)
};
// Find next hop toward destination
let next_hop_addr = match self.find_next_hop(&datagram.dest_addr) {
Some(peer) => *peer.node_addr(),
None => {
match outcome {
RouteOutcome::Drop {
reason: DropReason::TtlExhausted,
} => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::TtlExhausted, payload.len());
debug!(
src = %datagram_ref.src_addr,
dest = %datagram_ref.dest_addr,
ttl = datagram_ref.ttl,
"SessionDatagram TTL exhausted, dropping"
);
}
RouteOutcome::DeliverLocal => {
// Local delivery: dispatch to session layer handlers without
// materializing an owned SessionDatagram payload Vec.
self.metrics().forwarding.record_delivered(payload.len());
self.handle_session_payload(
&datagram_ref.src_addr,
datagram_ref.payload,
datagram_ref.path_mtu,
incoming_ce,
)
.await;
}
RouteOutcome::NoRoute => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::NoRoute, payload.len());
let original = datagram_ref.into_owned();
debug!(
src = %self.peer_display_name(&datagram.src_addr),
dest = %self.peer_display_name(&datagram.dest_addr),
src = %self.peer_display_name(&original.src_addr),
dest = %self.peer_display_name(&original.dest_addr),
bytes = payload.len(),
"Dropping transit SessionDatagram: no route to destination"
);
self.send_routing_error(&datagram).await;
return;
self.send_routing_error(&original).await;
}
};
RouteOutcome::Forward {
next_hop,
bytes,
outgoing_ce,
} => {
let dest = datagram_ref.dest_addr;
// Apply path_mtu min() from the outgoing link's transport MTU
if let Some(peer) = self.peers.get(&next_hop_addr)
// ECN CE relay: congestion was detected locally above; emit the
// metric and rate-limited log at the transit chokepoint.
if congested {
self.metrics().congestion.congestion_detected.inc();
let now = Instant::now();
let should_log = self
.last_congestion_log
.map(|t| now.duration_since(t) >= Duration::from_secs(5))
.unwrap_or(true);
if should_log {
self.last_congestion_log = Some(now);
debug!(next_hop = %next_hop, "Congestion detected, CE flag set on forwarded packet");
}
}
match self
.send_encrypted_link_message_with_ce(&next_hop, &bytes, outgoing_ce)
.await
{
Err(NodeError::MtuExceeded { mtu, .. }) => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::MtuExceeded, payload.len());
self.send_mtu_exceeded_error(dest, datagram_ref.src_addr, mtu)
.await;
}
Err(e) => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::SendError, payload.len());
debug!(
next_hop = %next_hop,
dest = %dest,
error = %e,
"Failed to forward SessionDatagram"
);
}
Ok(()) => {
self.metrics().forwarding.record_forwarded(bytes.len());
// Classify this transit forward by route class (partition
// of forwarded_packets). Done here, at the data-plane
// chokepoint, so the error-signal routing callers of
// find_next_hop are excluded.
let class = self.classify_forward(&dest, &next_hop);
self.metrics().forwarding.record_route_class(class);
if outgoing_ce {
self.metrics().congestion.ce_forwarded.inc();
}
}
}
}
}
}
/// Resolve the next hop toward `dest` into its address plus the outgoing
/// link's transport MTU. Returns `None` when there is no route.
///
/// The MTU defaults to `u16::MAX` (a no-op min-fold) when the peer's
/// transport is not resolvable, matching the pre-refactor inline behavior
/// where the MTU `if let` chain simply did not fire.
fn resolve_next_hop(&mut self, dest: &NodeAddr) -> Option<NextHop> {
let addr = *self.find_next_hop(dest)?.node_addr();
let link_mtu = if let Some(peer) = self.peers.get(&addr)
&& let Some(tid) = peer.transport_id()
&& let Some(transport) = self.transports.get(&tid)
{
if let Some(addr) = peer.current_addr() {
datagram.path_mtu = datagram.path_mtu.min(transport.link_mtu(addr));
} else {
datagram.path_mtu = datagram.path_mtu.min(transport.mtu());
}
}
// ECN CE relay: propagate incoming CE and detect local congestion
let local_congestion = self.detect_congestion(&next_hop_addr);
let outgoing_ce = incoming_ce || local_congestion;
if local_congestion {
self.metrics().congestion.congestion_detected.inc();
let now = Instant::now();
let should_log = self
.last_congestion_log
.map(|t| now.duration_since(t) >= Duration::from_secs(5))
.unwrap_or(true);
if should_log {
self.last_congestion_log = Some(now);
debug!(next_hop = %next_hop_addr, "Congestion detected, CE flag set on forwarded packet");
}
}
// Forward: re-encode (includes 0x00 type byte) and send
let encoded = datagram.encode();
if let Err(e) = self
.send_encrypted_link_message_with_ce(&next_hop_addr, &encoded, outgoing_ce)
.await
{
match e {
NodeError::MtuExceeded { mtu, .. } => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::MtuExceeded, payload.len());
self.send_mtu_exceeded_error(&datagram, mtu).await;
}
_ => {
self.metrics()
.forwarding
.record_reject_bytes(ForwardingReject::SendError, payload.len());
debug!(
next_hop = %next_hop_addr,
dest = %datagram.dest_addr,
error = %e,
"Failed to forward SessionDatagram"
);
}
match peer.current_addr() {
Some(link_addr) => transport.link_mtu(link_addr),
None => transport.mtu(),
}
} else {
self.metrics().forwarding.record_forwarded(encoded.len());
// Classify this transit forward by route class (partition of
// forwarded_packets). Done here, at the data-plane chokepoint, so
// the error-signal routing callers of find_next_hop are excluded.
let class = self.classify_forward(&datagram.dest_addr, &next_hop_addr);
self.metrics().forwarding.record_route_class(class);
if outgoing_ce {
self.metrics().congestion.ce_forwarded.inc();
}
}
u16::MAX
};
Some(NextHop { addr, link_mtu })
}
/// Attempt to warm the coordinate cache from session-layer payload headers.
@@ -269,35 +307,41 @@ impl Node {
/// If we can't route the error back to the source either, drop silently.
/// No cascading errors.
async fn send_routing_error(&mut self, original: &SessionDatagram) {
// Rate limit: one error signal per destination per 100ms
if !self
.routing_error_rate_limiter
.should_send(&original.dest_addr)
{
return;
}
let my_addr = *self.node_addr();
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
let default_ttl = self.config().node.session.default_ttl;
let error_payload =
if let Some(coords) = self.coord_cache().get(&original.dest_addr, now_ms) {
let coords = coords.clone();
PathBroken::new(original.dest_addr, my_addr)
.with_last_coords(coords)
.encode()
} else {
CoordsRequired::new(original.dest_addr, my_addr).encode()
// Pure decision: rate-limit gate + PathBroken/CoordsRequired choice +
// error-PDU encode. Borrow the routing tables disjointly from
// `&mut self.routing`, then release them before the reverse-hop lookup.
let action = {
let view = NodeRoutingView {
coord_cache: &self.coord_cache,
peers: &self.peers,
tree_state: &self.tree_state,
congested: false,
};
self.routing.synth_routing_error(
&original.dest_addr,
&original.src_addr,
&my_addr,
&view,
now_ms,
default_ttl,
)
};
let RouteAction::SendError { toward, bytes } = match action {
Some(action) => action,
// Rate limited: drop silently. No cascading errors.
None => return,
};
let error_dg = SessionDatagram::new(my_addr, original.src_addr, error_payload)
.with_ttl(self.config().node.session.default_ttl);
let next_hop_addr = match self.find_next_hop(&original.src_addr) {
// Resolve the reverse link hop only now, after the gate passed, so
// `find_next_hop`'s coord-cache touch keeps its pre-refactor scope.
let next_hop_addr = match self.find_next_hop(&toward) {
Some(peer) => *peer.node_addr(),
None => {
debug!(
@@ -309,9 +353,8 @@ impl Node {
}
};
let encoded = error_dg.encode();
if let Err(e) = self
.send_encrypted_link_message(&next_hop_addr, &encoded)
.send_encrypted_link_message(&next_hop_addr, &bytes)
.await
{
debug!(
@@ -333,37 +376,50 @@ impl Node {
/// Called when `send_encrypted_link_message()` fails with
/// `NodeError::MtuExceeded` during forwarding. The signal tells the
/// source the bottleneck MTU so it can immediately reduce its path MTU.
async fn send_mtu_exceeded_error(&mut self, original: &SessionDatagram, bottleneck_mtu: u16) {
// Rate limit: reuse routing_error_rate_limiter keyed on dest_addr
if !self
.routing_error_rate_limiter
.should_send(&original.dest_addr)
{
return;
}
///
/// `dest` is the failed datagram's destination (rate-limit key); `toward`
/// is its source, where the signal is routed back.
async fn send_mtu_exceeded_error(
&mut self,
dest: NodeAddr,
toward: NodeAddr,
bottleneck_mtu: u16,
) {
let my_addr = *self.node_addr();
let now_ms = Self::now_ms();
let default_ttl = self.config().node.session.default_ttl;
let error_payload = MtuExceeded::new(original.dest_addr, my_addr, bottleneck_mtu).encode();
// Pure decision: rate-limit gate + MtuExceeded PDU + encode.
let action = self.routing.synth_mtu_exceeded(
&dest,
&toward,
&my_addr,
bottleneck_mtu,
now_ms,
default_ttl,
);
let RouteAction::SendError { toward, bytes } = match action {
Some(action) => action,
// Rate limited: drop silently. No cascading errors.
None => return,
};
let error_dg = SessionDatagram::new(my_addr, original.src_addr, error_payload)
.with_ttl(self.config().node.session.default_ttl);
let next_hop_addr = match self.find_next_hop(&original.src_addr) {
// Resolve the reverse link hop only now, after the gate passed, so
// `find_next_hop`'s coord-cache touch keeps its pre-refactor scope.
let next_hop_addr = match self.find_next_hop(&toward) {
Some(peer) => *peer.node_addr(),
None => {
debug!(
src = %original.src_addr,
dest = %original.dest_addr,
src = %toward,
dest = %dest,
"Cannot route MtuExceeded signal back to source, dropping"
);
return;
}
};
let encoded = error_dg.encode();
if let Err(e) = self
.send_encrypted_link_message(&next_hop_addr, &encoded)
.send_encrypted_link_message(&next_hop_addr, &bytes)
.await
{
debug!(
@@ -373,8 +429,8 @@ impl Node {
);
} else {
debug!(
original_dest = %original.dest_addr,
error_dest = %original.src_addr,
original_dest = %dest,
error_dest = %toward,
bottleneck_mtu,
"Sent MtuExceeded error signal"
);
@@ -417,10 +473,13 @@ impl Node {
for (&tid, transport) in &self.transports {
let congestion = transport.congestion();
let state = self.transport_drops.entry(tid).or_default();
if let Some(current) = congestion.recv_drops
&& state.observe_drops(current)
{
new_drop_events.push(tid);
if let Some(current) = congestion.recv_drops {
let new_drops = current > state.prev_drops;
if new_drops && !state.dropping {
new_drop_events.push(tid);
}
state.dropping = new_drops;
state.prev_drops = current;
}
}
for tid in new_drop_events {
src/node/dataplane/mod.rs
+18
View File
@@ -0,0 +1,18 @@
//! Data plane: the RX `select!` loop and the per-packet forwarding path.
//!
//! Holds the whole hot path in one home: the `select!` run loop
//! (`rx_loop`), transit/local datagram forwarding (`forwarding`), the
//! link-message router (`dispatch`), the RX decrypt path including responder
//! K-bit cutover and address-roam writes (`encrypted`), and the per-peer
//! connected-UDP fast-path socket activation (`connected_udp`). Each module
//! contributes `impl Node` methods driven by the run loop.
#[cfg(unix)]
pub(crate) mod connected_udp;
mod dispatch;
mod encrypted;
mod forwarding;
mod peer_actions;
mod rx_loop;
pub(in crate::node) use peer_actions::PeerActionCtx;
src/node/dataplane/peer_actions.rs
+829
View File
@@ -0,0 +1,829 @@
//! Executor for the per-peer control machine's [`PeerAction`]s.
//!
//! The per-peer FSM in [`crate::peer::machine`] is a sans-IO reducer: it decides
//! *what* must happen and returns a `Vec<PeerAction>`; this module is the *doing*
//! half — the thin driver that maps each action onto the exact shell call it
//! stands for (`build_msg2` + `transport.send`, `promote_connection`,
//! `remove_active_peer`, `index_allocator.free`, `note_link_dead`, …).
//!
//! ## Progressive cutover
//!
//! The executor is wired incrementally. Live today: the outbound msg2 promote
//! (`handle_msg2` looks up the dial-persisted machine), the connectionless
//! outbound msg1 send (`SendHandshake` with `their_index == None` →
//! `send_stored_msg1`, driven from `initiate_connection`), the
//! connection-oriented dial (`OpenTransport` performs the non-blocking
//! `transport.connect`; `TransportConnected` drives the connect-resolution msg1
//! send from `poll_pending_connects`), the rekey cadence (`check_rekey` →
//! `route_rekey_cadence` → `RekeyConsume`, driving the `SwapSendState` and
//! `CompleteDrain` arms), and the liveness reap (`route_link_dead` →
//! `LinkDeadSuspected`, driving `InvalidateSendState` → `remove_active_peer`).
//!
//! Inbound msg1 is not machine-driven here: `handle_msg1` builds and sends
//! msg2 inline (persisting the leg's machine parked at `SentMsg2` alongside),
//! so `PeerEvent::InboundMsg1` is never dispatched and the `SendHandshake`
//! `their_index == Some` (msg2) branch stays dormant. Inbound msg3 IS
//! machine-driven: `handle_msg3` steps the leg's persistent machine and this
//! executor performs its verdict (`PromoteToActive`, `SwapToInboundSession`,
//! `RekeyRespondTrigger`), disposing the machine on every path that consumes
//! the leg without promoting it.
//!
//! The genuine inert stubs remaining are `SendRekey`, `SendLinkMessage`, and
//! the connected-UDP arms. `RegisterDecryptSession` is a deliberate no-op —
//! see its arm for the note.
//!
//! The timer arms (`SetTimer`/`CancelTimer`) populate/clear the per-peer timer
//! store (`peer_timers`). The `HandshakeRetransmit` and `HandshakeTimeout`
//! deadlines are read and fired by `drive_peer_timers` (the handshake resend +
//! reap home). The rekey/liveness kinds are still SHADOW — driven by their own
//! shell drivers — so populating them stays behavior-neutral.
use crate::PeerIdentity;
use crate::node::reject::{HandshakeReject, RejectReason};
use crate::node::{Node, NodeError};
use crate::peer::machine::{LostKind, PeerAction, PeerEvent};
use crate::proto::fmp::PromotionResult;
use crate::proto::fmp::wire::build_msg2;
use crate::transport::{LinkId, TransportAddr, TransportId};
use crate::utils::index::SessionIndex;
use std::collections::VecDeque;
use tracing::{debug, info, trace, warn};
/// Ambient shell facts a [`PeerAction`] executor needs that the machine's
/// runtime-agnostic action payloads deliberately omit (verified identity,
/// transport target, the msg2 framing indices, the promotion timestamp).
///
/// Unlike a machine event/action payload this is **executor-side**, so it may
/// hold real values resolved from the wire context (cf. `handle_msg3`'s
/// `wire`/`packet` locals and `promote_connection`'s ambient args). It is built
/// fresh per driven step by the caller at cutover time.
#[allow(dead_code)]
pub(in crate::node) struct PeerActionCtx {
/// The authenticated peer identity (`PromoteToActive` / `InvalidateSendState`
/// / `SwapSendState` resolve their `NodeAddr` from this).
pub(in crate::node) verified_identity: PeerIdentity,
/// The transport the exchange is happening over (msg2 send target, decrypt
/// cache-key transport half).
pub(in crate::node) transport_id: TransportId,
/// The peer's wire address (msg2 send target).
pub(in crate::node) remote_addr: TransportAddr,
/// Our session index for this exchange (msg2 framing sender_idx).
pub(in crate::node) our_index: Option<SessionIndex>,
/// The peer's session index for this exchange (msg2 framing receiver_idx).
pub(in crate::node) their_index: Option<SessionIndex>,
/// The wire timestamp driving this step (promotion ts / loss-report clock).
pub(in crate::node) now_ms: u64,
/// Establish direction for this exchange. `false` = inbound, `true` =
/// outbound. `PromoteToActive` reads this to pick the direction-specific
/// promote tail: the outbound branch logs a `Peer promoted to active` line
/// and clears `pending_outbound`, and its promote-Err cleanup is warn-only
/// (no link/index teardown), unlike the inbound branch.
pub(in crate::node) is_outbound: bool,
/// The `pending_outbound` key for an outbound promote, cleared on success.
/// `Some` only on the outbound driven step (the map entry keyed by the wire
/// `receiver_idx`); `None` on the inbound and maintenance paths, which have
/// no `pending_outbound` entry to clear.
pub(in crate::node) pending_outbound_key: Option<(TransportId, u32)>,
}
impl Node {
/// Advance the machine for `link` by one event and execute the resulting
/// actions.
///
/// The borrow structure the whole seam turns on: the machine needs
/// `&mut IndexAllocator` as a synchronous capability *while it is itself
/// borrowed mutably out of `peer_machines`*. `peer_machines` and
/// `index_allocator` are **distinct `Node` fields**, so the collect below is a
/// disjoint two-field borrow the checker accepts; once the actions are
/// collected both borrows drop and the executor runs against `&mut self`.
pub(in crate::node) async fn advance_peer_machine(
&mut self,
link: LinkId,
event: PeerEvent,
now: u64,
ambient: &PeerActionCtx,
) {
let actions = match self.peer_machines.get_mut(&link) {
// Disjoint field borrow: `self.peer_machines` (the map entry) and
// `self.index_allocator` (the capability) are separate fields.
Some(machine) => machine.step(event, now, &mut self.index_allocator),
None => return,
};
self.execute_peer_actions(link, ambient, actions).await;
}
/// Map each [`PeerAction`] onto its shell call.
///
/// The worklist is a `VecDeque` rather than self-recursion so the async
/// executor stays a single flat future (no boxing) and the emitted order is
/// preserved. `PromoteToActive` (deferred) will feed its resolution back into
/// the machine and fold follow-up actions onto this same queue.
pub(in crate::node) async fn execute_peer_actions(
&mut self,
link: LinkId,
ambient: &PeerActionCtx,
actions: Vec<PeerAction>,
) {
let mut queue: VecDeque<PeerAction> = actions.into();
while let Some(action) = queue.pop_front() {
match action {
PeerAction::OpenTransport {
transport_id,
remote_addr,
} => {
// Outbound connection-oriented dial. `initiate_connection`'s
// oriented branch drove the machine to `Connecting`, which
// emitted this action. Perform the non-blocking
// `transport.connect` and, on success, push the
// `PendingConnect` for `poll_pending_connects` to resolve. On
// connect error, tear down the dial-window state (link,
// reverse map, control machine) and abort the queue — the
// executor-local mirror of the old inline
// `initiate_connection` connect+push.
if let Some(transport) = self.transports.get(&transport_id) {
match transport.connect(&remote_addr).await {
Ok(()) => {
debug!(
transport_id = %transport_id,
remote_addr = %remote_addr,
link_id = %link,
"Transport connect initiated (non-blocking)"
);
self.peering
.pending_connects
.push(crate::node::PendingConnect {
link_id: link,
transport_id,
remote_addr,
peer_identity: Some(ambient.verified_identity),
});
}
Err(_e) => {
self.links.remove(&link);
self.addr_to_link.remove(&(transport_id, remote_addr));
self.remove_peer_machine(link);
return;
}
}
}
}
PeerAction::SendHandshake { bytes } => {
// Two outbound directions share this action, discriminated by
// `their_index`:
// msg2 (`their_index == Some`): the machine payload is the
// UNFRAMED Noise msg2; frame it with our/their index
// (`build_msg2`) and send.
// msg1 (`their_index == None`): a fresh outbound handshake;
// the machine's empty payload is ignored — the shell already
// allocated the index, ran the Noise leaf, and armed the
// wire at dial (`prepare_outbound_msg1`); this just sends the
// stored wire (see `send_stored_msg1`).
if let (Some(sender_idx), Some(receiver_idx)) =
(ambient.our_index, ambient.their_index)
{
let frame = build_msg2(sender_idx, receiver_idx, &bytes);
// Surface the send Result. A missing transport skips
// the send and continues (mirrors `handle_msg1`'s
// `if let Some(transport)` guard); a send *error* runs the
// pre-refactor msg2-send-failure cleanup (`handle_msg1`
// L494-503) and ABORTS the remaining queue so the queued
// `PromoteToActive` never runs.
let send_err = match self.transports.get(&ambient.transport_id) {
Some(transport) => {
transport.send(&ambient.remote_addr, &frame).await.err()
}
None => None,
};
if let Some(e) = send_err {
// Restored pre-refactor msg2-send-failure warn!
// (`handle_msg1` L665): the send error text is surfaced
// at the executor point where the failure is now handled.
warn!(link_id = %link, error = %e, "Failed to send msg2");
self.links.remove(&link);
self.addr_to_link
.remove(&(ambient.transport_id, ambient.remote_addr.clone()));
if let Some(idx) = ambient.our_index {
let _ = self.index_allocator.free(idx);
}
self.remove_peer_machine(link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
return;
}
} else {
// msg1: the shell already allocated the index, ran the
// Noise leaf, and armed the wire on the connection at dial
// (`prepare_outbound_msg1`); send the stored wire. The
// machine's empty payload is ignored.
let _ = bytes;
self.send_stored_msg1(
link,
ambient.transport_id,
&ambient.remote_addr,
ambient.now_ms,
)
.await;
}
}
PeerAction::SendRekey { .. } => {
// Rekey msg framing (`build_msg2(our_new_index, …)`) + send.
// Rekey fold is out of scope here.
}
PeerAction::SendLinkMessage { .. } => {
// Encrypt + send a link-control frame (heartbeat / filter /
// tree / disconnect). Data-plane-owned; out of scope here.
}
PeerAction::PromoteToActive { link: promote_link } => {
// Establish promote, driven through the machine. Transcribes
// `handle_msg3`'s shared inbound promote block verbatim, adapted
// to the executor's ambient context. One XX-specific choice vs
// the IK-lineage executor: the decrypt-worker register stays
// INSIDE `promote_connection` (NOT relocated here) — re-
// registering would double-register. The promoted leg's machine
// (msg1-born inbound, dial-born outbound) survives the promotion
// and is crystallized in place by the `PromotionResolved`
// feedback fed back after the Ok handling below.
// Capture msg2 BEFORE `promote_connection` removes the pending
// connection, so a duplicate msg1 can be answered with it. Only
// the inbound promote answers a duplicate inbound msg1; the
// outbound side has no stored msg2 to resend.
let wire_msg2 = if ambient.is_outbound {
None
} else {
self.peer_machines
.get(&promote_link)
.and_then(|m| m.conn_handshake_msg2().map(|b| b.to_vec()))
};
if ambient.is_outbound {
debug!(
// Relocated from `handlers/handshake.rs`: pin the target
// so it stays visible under the harness's
// `fips::node::handlers::handshake=debug` filter.
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(ambient.verified_identity.node_addr()),
link_id = %promote_link,
"handle_msg2: promoting outbound, peers_has_key={}",
self.peers.contains_key(ambient.verified_identity.node_addr()),
);
} else {
debug!(
// Relocated from `handlers/handshake.rs`: pin the target
// so it stays visible under the harness's
// `fips::node::handlers::handshake=debug` filter.
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(ambient.verified_identity.node_addr()),
link_id = %promote_link,
our_index = ?ambient.our_index,
"handle_msg3: promoting inbound, peers_has_key={}",
self.peers.contains_key(ambient.verified_identity.node_addr()),
);
}
let promote_result = self.promote_connection(
promote_link,
ambient.verified_identity,
ambient.now_ms,
);
match &promote_result {
Ok(PromotionResult::Promoted(node_addr)) => {
let node_addr = *node_addr;
if ambient.is_outbound {
// The outbound promote logs a second line here in
// addition to `promote_connection`'s "Connection
// promoted to active peer". Pin the target so the
// relocated line keeps the module it filtered under.
info!(
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(&node_addr),
"Peer promoted to active"
);
} else {
// Store msg2 on peer for resend on duplicate msg1
if let (Some(peer), Some(msg2)) =
(self.peers.get_mut(&node_addr), wire_msg2)
{
peer.set_handshake_msg2(msg2);
}
// Promotion is logged once by `promote_connection`
// ("Connection promoted to active peer"); no separate
// inbound-path line.
}
// Send initial tree announce to new peer
if let Err(e) = self.send_tree_announce_to_peer(&node_addr).await {
debug!(peer = %self.peer_display_name(&node_addr), error = %e, "Failed to send initial TreeAnnounce");
}
// Schedule filter announce (sent on next tick via debounce)
self.bloom_state.mark_update_needed(node_addr);
self.reset_lookup_backoff();
// Clear the pending outbound entry on promote success
// only; a failed promote leaves it for the stale-
// connection reaper.
if let Some(k) = ambient.pending_outbound_key {
self.pending_outbound.remove(&k);
}
}
Ok(PromotionResult::CrossConnectionWon {
loser_link_id,
node_addr,
}) => {
let (loser_link_id, node_addr) = (*loser_link_id, *node_addr);
// UNREACHABLE on driven XX establish paths: `Promote`
// and `RestartThenPromote` (which removes the old peer
// first) both imply no existing peer at promote time, so
// `promote_connection` returns `Promoted`. Body kept
// byte-equivalent to next so a future path that drives a
// cross-connection through the executor trips the assert.
debug_assert!(
false,
"executor CrossConnectionWon is unreachable on driven \
XX inbound establish paths"
);
// Store msg2 on peer for resend on duplicate msg1
if let (Some(peer), Some(msg2)) =
(self.peers.get_mut(&node_addr), wire_msg2)
{
peer.set_handshake_msg2(msg2);
}
// Close the losing TCP connection (no-op for connectionless)
if let Some(loser_link) = self.links.get(&loser_link_id) {
let loser_tid = loser_link.transport_id();
let loser_addr = loser_link.remote_addr().clone();
if let Some(transport) = self.transports.get(&loser_tid) {
transport.close_connection(&loser_addr).await;
}
}
// Clean up the losing connection's link
self.remove_link(&loser_link_id);
debug!(
peer = %self.peer_display_name(&node_addr),
loser_link_id = %loser_link_id,
"Inbound cross-connection won, loser link cleaned up"
);
if let Err(e) = self.send_tree_announce_to_peer(&node_addr).await {
debug!(peer = %self.peer_display_name(&node_addr), error = %e, "Failed to send initial TreeAnnounce");
}
self.bloom_state.mark_update_needed(node_addr);
self.reset_lookup_backoff();
}
Ok(PromotionResult::CrossConnectionLost { winner_link_id }) => {
let winner_link_id = *winner_link_id;
// UNREACHABLE on driven XX establish paths (see the Won
// arm). Body kept byte-equivalent to next; uses the
// ambient transport/addr in place of next's `packet.*`.
debug_assert!(
false,
"executor CrossConnectionLost is unreachable on driven \
XX inbound establish paths"
);
// Close the losing TCP connection (no-op for connectionless)
if let Some(transport) = self.transports.get(&ambient.transport_id) {
transport.close_connection(&ambient.remote_addr).await;
}
// This connection lost — clean up its link
self.remove_link(&promote_link);
// Restore addr_to_link for the winner's link
self.addr_to_link.insert(
(ambient.transport_id, ambient.remote_addr.clone()),
winner_link_id,
);
debug!(
winner_link_id = %winner_link_id,
"Inbound cross-connection lost, keeping existing"
);
}
Err(e) if ambient.is_outbound => {
// The outbound promote-failure path is warn-only: it
// records the reject but performs no link/index teardown
// and leaves the `pending_outbound` entry for the stale-
// connection reaper. The leg's machine must go with the
// leg, though: `promote_connection` took the pending
// connection off the machine before erring, so the reaper
// (which sweeps the machines' embedded legs) can never
// reach this link's machine — dropping it here is the
// only disposal point.
warn!(
target: "fips::node::handlers::handshake",
link_id = %promote_link,
error = %e,
"Failed to promote connection"
);
self.remove_peer_machine(promote_link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
}
Err(e) => {
// A max_peers rejection is expected policy, not a fault —
// log it at debug to avoid WARN spam when a cap'd node is
// under sustained inbound pressure. Other promotion
// failures remain at warn.
if matches!(e, NodeError::MaxPeersExceeded { .. }) {
debug!(
// Emit under the handshake target (same as the
// "promoting inbound" line above) so this stays
// visible wherever inbound handshake events are
// logged at debug, independent of this module's
// own log level.
target: "fips::node::handlers::handshake",
peer = %self.peer_display_name(ambient.verified_identity.node_addr()),
max = self.max_peers(),
"Rejecting inbound connection at max_peers cap (no promotion)"
);
} else {
warn!(
target: "fips::node::handlers::handshake",
link_id = %promote_link,
error = %e,
"Failed to promote inbound connection"
);
}
// Clean up on promotion failure. promote_connection
// already freed our_index in its MaxPeersExceeded path;
// freeing again here is benign (IndexAllocator::free is a
// HashSet::remove, the second call returns Err(NotFound)
// and is ignored).
self.remove_link(&promote_link);
if let Some(idx) = ambient.our_index {
let _ = self.index_allocator.free(idx);
}
self.remove_peer_machine(promote_link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
}
}
// Feed the promotion outcome back into the surviving machine
// and fold the follow-up actions onto the worklist (the
// `Promoted` arm crystallizes the machine in place and emits
// `RegisterDecryptSession`, a redundant no-op here — see its
// arm). Unconditional across Ok variants; on
// `CrossConnectionLost` the losing leg's machine was disposed
// inside `promote_connection`, so the lookup misses and no
// machine-side index free can double the inline one. Disjoint
// field borrow again.
if let Ok(result) = promote_result {
let follow = match self.peer_machines.get_mut(&promote_link) {
Some(machine) => machine.step(
PeerEvent::PromotionResolved { result },
ambient.now_ms,
&mut self.index_allocator,
),
None => Vec::new(),
};
queue.extend(follow);
}
}
PeerAction::ResolveCrossConnection { .. } => {
// A decision token, not an effect: the outbound msg2
// handler intercepts it and runs the inline swap/keep
// resolution itself, so it must never reach the executor.
debug_assert!(
false,
"ResolveCrossConnection is intercepted by the msg2 \
handler and must never reach the executor"
);
}
PeerAction::SwapSendState { .. } => {
// Initiator cutover: the live authoritative rekey-cadence
// path, routed here from `check_rekey` via
// `route_rekey_cadence` → `PeerEvent::RekeyConsume`; the
// inline body survives only as `cutover_peer_inline`, a
// debug-assert release fallback. `addr` is resolved
// from the ambient verified identity (as `InvalidateSendState`
// does). The decrypt re-register folds HERE, gated on
// `did_cutover` — the generic `RegisterDecryptSession` arm stays a
// no-op so a promote never double-registers.
let node_addr = *ambient.verified_identity.node_addr();
let did_cutover = if let Some(peer) = self.peers.get_mut(&node_addr) {
if let Some(_old_our_index) = peer.cutover_to_new_session() {
// New index was pre-registered in peers_by_index
// during msg2 handling (handshake.rs).
debug_assert!(
peer.transport_id().is_some()
&& peer.our_index().is_some()
&& self.peers_by_index.contains_key(&(
peer.transport_id().unwrap(),
peer.our_index().unwrap().as_u32()
)),
"peers_by_index should contain pre-registered new index after cutover"
);
let our_index = peer.our_index();
let their_index = peer.their_index();
info!(
// Pin the target to the pre-refactor module: this
// cutover log relocated from handlers/rekey.rs into
// the executor, but operators (and the test harness)
// filter it under fips::node::handlers::rekey.
// Keeping the target preserves the observable log
// contract (level + fields match next's rekey.rs).
target: "fips::node::handlers::rekey",
peer = %self.peer_display_name(&node_addr),
our_addr = %self.identity().node_addr(),
their_addr = %node_addr,
our_index = ?our_index,
their_index = ?their_index,
"Rekey cutover complete (initiator), K-bit flipped"
);
true
} else {
false
}
} else {
false
};
// Re-register the new session with the decrypt worker — the
// cache_key (transport_id, our_index) just changed, so the old
// worker entry is stale and every packet on the new session
// would miss the worker's HashMap lookup.
#[cfg(unix)]
if did_cutover {
self.register_decrypt_worker_session(&node_addr);
}
#[cfg(not(unix))]
let _ = did_cutover;
}
PeerAction::CompleteDrain { peer: node_addr } => {
// Drain completion: the live authoritative rekey-cadence
// path, routed here from `check_rekey` via
// `route_rekey_cadence` → `PeerEvent::RekeyConsume`; the
// inline body survives only as `drain_peer_inline`, a
// debug-assert release fallback. Reached in either rekey
// role — a responder's cutover demotes a session the same
// way an initiator's does — including on a node whose own
// trigger is off. Extract the real previous
// index + transport_id under the peer borrow, drop the
// borrow, then run the cache_key cleanup (which takes
// &mut self for unregister_decrypt_worker_session).
let drained = self.peers.get_mut(&node_addr).and_then(|peer| {
peer.complete_drain().map(|idx| (idx, peer.transport_id()))
});
if let Some((old_our_index, transport_id)) = drained {
if let Some(tid) = transport_id {
let cache_key = (tid, old_our_index.as_u32());
self.peers_by_index.remove(&cache_key);
#[cfg(unix)]
self.unregister_decrypt_worker_session(cache_key);
}
let _ = self.index_allocator.free(old_our_index);
trace!(
// Pin to the pre-refactor module (see the cutover log
// above) so the relocated drain log stays visible under
// the operator's fips::node::handlers::rekey filter.
target: "fips::node::handlers::rekey",
peer = %self.peer_display_name(&node_addr),
old_index = %old_our_index,
"Drain complete, previous session erased"
);
}
}
PeerAction::InvalidateSendState => {
// The FULL teardown. `remove_active_peer` frees the four index
// slots (current/rekey/pending/previous), drops
// `peers_by_index`, unregisters the decrypt worker, removes the
// FSP `sessions` entry and `pending_tun_packets`. The machine
// emits NO `FreeIndex` for those slots, so there is no
// double-free.
self.remove_active_peer(ambient.verified_identity.node_addr());
}
PeerAction::SwapToInboundSession {
peer,
our_index,
our_inbound_wins,
} => {
// Simultaneous-init cross-connection resolved at msg3 (msg2-then-
// msg3 ordering): apply the same tie-breaker the inverse ordering
// uses so both sides converge on a single Noise session pair.
let their_index = ambient
.their_index
.expect("cross-connection swap carries the peer session index");
if our_inbound_wins {
// Larger node side: swap to the inbound session so it pairs
// with the peer's kept outbound session.
let inbound_session = match self
.peer_machines
.get_mut(&link)
.and_then(|m| m.take_session())
{
Some(s) => s,
None => {
self.remove_link(&link);
self.remove_peer_machine(link);
self.stats_mut().record_reject(RejectReason::Handshake(
HandshakeReject::BadState,
));
return;
}
};
// A rekey of ours in flight is now stale: the session it
// was negotiated against is about to be replaced, and
// `replace_session` rewrites the session and both indices
// while touching no rekey state. Abandoning here is what
// lets the classifier take this arm unconditionally —
// declining the swap instead would desynchronize us from
// the peer's outbound half, which cannot see our rekey.
// Same clearing the rekey-responder arm does on its
// `abandon_first` path.
if let Some(peer_ref) = self.peers.get_mut(&peer)
&& let Some(idx) = peer_ref.abandon_rekey()
{
if let Some(tid) = peer_ref.transport_id() {
self.peers_by_index.remove(&(tid, idx.as_u32()));
self.pending_outbound.remove(&(tid, idx.as_u32()));
}
let _ = self.index_allocator.free(idx);
}
if let Some(peer_ref) = self.peers.get_mut(&peer) {
let old_our_index =
peer_ref.replace_session(inbound_session, our_index, their_index);
let Some(transport_id) = peer_ref.transport_id() else {
self.remove_link(&link);
self.remove_peer_machine(link);
self.stats_mut().record_reject(RejectReason::Handshake(
HandshakeReject::BadState,
));
return;
};
if let Some(old_idx) = old_our_index {
self.peers_by_index
.remove(&(transport_id, old_idx.as_u32()));
let _ = self.index_allocator.free(old_idx);
}
self.peers_by_index
.insert((transport_id, our_index.as_u32()), peer);
debug!(
peer = %self.peer_display_name(&peer),
new_our_index = %our_index,
new_their_index = %their_index,
"Simultaneous-init (msg3): swapped to inbound session (our inbound wins)"
);
}
} else {
// Smaller node side: keep the existing outbound session, drop
// the inbound leg's allocated index.
let _ = self.index_allocator.free(our_index);
debug!(
peer = %self.peer_display_name(&peer),
"Simultaneous-init (msg3): keeping outbound session (our outbound wins)"
);
}
// Both branches tear down the temporary inbound link fully
// (including its `addr_to_link` mapping) via `remove_link`,
// disposing the leg's machine (and its embedded connection)
// with it.
self.remove_link(&link);
self.remove_peer_machine(link);
return;
}
PeerAction::RekeyRespondTrigger {
peer,
our_index,
abandon_first,
} => {
// Rekey-responder resolved at msg3: store the new session as
// pending on the existing peer, awaiting the K-bit cutover.
let their_index = ambient
.their_index
.expect("rekey-responder trigger carries the peer session index");
if abandon_first {
// We lose the dual-rekey tie-break (larger addr): abandon our
// own rekey/pending and fall through as responder.
// `abandon_rekey` clears both the in-progress flag and any
// pending session state, returning whichever index needs
// freeing.
info!(
peer = %self.peer_display_name(&peer),
our_addr = %self.identity().node_addr(),
their_addr = %peer,
"rekey-msg3 tie-break: we lose (larger addr), abandon ours"
);
if let Some(peer_ref) = self.peers.get_mut(&peer)
&& let Some(idx) = peer_ref.abandon_rekey()
{
if let Some(tid) = peer_ref.transport_id() {
self.peers_by_index.remove(&(tid, idx.as_u32()));
self.pending_outbound.remove(&(tid, idx.as_u32()));
}
let _ = self.index_allocator.free(idx);
}
}
// Rekey: process as responder, store new session as pending.
let noise_session = {
let Some(machine) = self.peer_machines.get_mut(&link) else {
warn!(link_id = %link, "Connection removed during rekey msg3 processing");
self.links.remove(&link);
self.remove_peer_machine(link);
self.stats_mut().record_reject(RejectReason::Handshake(
HandshakeReject::UnknownConnection,
));
return;
};
machine.take_session()
};
let our_new_index = our_index;
let noise_session = match noise_session {
Some(s) => s,
None => {
warn!("Rekey msg3: no session from handshake");
self.links.remove(&link);
self.remove_peer_machine(link);
self.stats_mut()
.record_reject(RejectReason::Handshake(HandshakeReject::BadState));
return;
}
};
// Store pending session on the existing peer
if let Some(peer_ref) = self.peers.get_mut(&peer) {
peer_ref.set_pending_session(noise_session, our_new_index, their_index);
peer_ref.record_peer_rekey();
}
// Register new index in peers_by_index
self.peers_by_index
.insert((ambient.transport_id, our_new_index.as_u32()), peer);
// Clean up: remove the temporary link and the leg's machine
// (dropping its embedded connection; the established peer
// keeps its own machine, keyed by its own link). Do NOT
// remove addr_to_link — the entry must remain pointing to
// the original link so the established peer stays routable,
// so this uses the bare `links.remove` rather than the full
// `remove_link`.
self.links.remove(&link);
self.remove_peer_machine(link);
debug!(
peer = %self.peer_display_name(&peer),
our_addr = %self.identity().node_addr(),
new_our_index = %our_new_index,
new_their_index = %their_index,
"rekey-msg3 responder: pending session set, awaiting K-bit cutover"
);
return;
}
PeerAction::RegisterDecryptSession { index } => {
let _ = index;
// No-op by design. The rekey-cutover decrypt-worker register
// relocates into the driven `SwapSendState` site above (gated
// on `did_cutover`); the establish-promote register stays INSIDE
// `promote_connection`, so `PromoteToActive` does not re-register
// either. This machine-emitted action is redundant with both;
// kept as an inert no-op (rather than removing the emission) so
// the machine's action sequence and unit tests stay unchanged.
// The keyed-by-NodeAddr register does not need the machine's
// `index` payload.
}
PeerAction::UnregisterDecryptSession { index } => {
// Executor supplies `transport_id` from ambient; keyed by
// (tid, index) like `remove_active_peer` / the rekey drain path.
#[cfg(unix)]
self.unregister_decrypt_worker_session((ambient.transport_id, index.as_u32()));
#[cfg(not(unix))]
let _ = index;
}
PeerAction::FreeIndex { index } => {
let _ = self.index_allocator.free(index);
}
PeerAction::ActivateConnectedUdp | PeerAction::TeardownConnectedUdp => {
// Connected-UDP plane ownership (`connected_udp.rs`). Out of
// scope for now.
}
PeerAction::SetTimer { kind, at_ms } => {
// Populate the per-peer timer store (overwrite = reschedule).
// The `HandshakeRetransmit` and `HandshakeTimeout` deadlines
// are read + fired by `drive_peer_timers`. Rekey/liveness kinds
// are still SHADOW here — they keep their own shell drivers —
// so populating them stays behavior-neutral.
self.peer_timers
.entry(link)
.or_default()
.insert(kind, at_ms);
}
PeerAction::CancelTimer { kind } => {
if let Some(timers) = self.peer_timers.get_mut(&link) {
timers.remove(&kind);
}
}
PeerAction::ReportLost { peer, kind } => {
// The single loss token, routed to the reconciler reflex the
// `kind` names: an un-promoted handshake attempt takes the
// connected-guarded `note_handshake_timeout` (`driver.rs:28`),
// an established peer's link-death takes the unconditional
// `note_link_dead` (`driver.rs:48`).
match kind {
LostKind::HandshakeTimeout => {
self.note_handshake_timeout(peer, ambient.now_ms);
}
LostKind::LinkDead => {
self.note_link_dead(peer, ambient.now_ms);
}
}
}
}
}
}
}
src/node/handlers/rx_loop.rs → src/node/dataplane/rx_loop.rs
+200 -46
View File
@@ -1,10 +1,11 @@
//! RX event loop and packet dispatch.
use crate::control::{ControlSocket, commands};
use crate::node::wire::{
COMMON_PREFIX_SIZE, CommonPrefix, FMP_VERSION, PHASE_ESTABLISHED, PHASE_MSG1, PHASE_MSG2,
};
use crate::node::{Node, NodeError};
use crate::proto::fmp::wire::{
COMMON_PREFIX_SIZE, CommonPrefix, FMP_VERSION, PHASE_ESTABLISHED, PHASE_MSG1, PHASE_MSG2,
PHASE_MSG3,
};
use crate::transport::ReceivedPacket;
use std::time::Duration;
use tracing::{debug, info, warn};
@@ -29,6 +30,7 @@ impl Node {
/// - Phase 0x0: Encrypted frame (session data)
/// - Phase 0x1: Handshake message 1 (initiator -> responder)
/// - Phase 0x2: Handshake message 2 (responder -> initiator)
/// - Phase 0x3: Handshake message 3 (initiator -> responder, XX completion)
///
/// Also processes outbound IPv6 packets from the TUN reader for session
/// encapsulation and routing through the mesh.
@@ -42,12 +44,42 @@ impl Node {
/// This method takes ownership of the packet_rx channel and runs
/// until the channel is closed (typically when stop() is called).
pub async fn run_rx_loop(&mut self) -> Result<(), NodeError> {
// No shutdown observer → today's infinite loop, byte-identical. All
// existing callers/tests use this; `pending()` never fires, so the
// shutdown/deadline arms below stay permanently disabled.
self.run_rx_loop_with_shutdown(std::future::pending()).await
}
/// The rx event loop, which serves until `shutdown` fires and then drains
/// **in place** before returning.
///
/// The channel receivers are moved into this frame's locals and live across
/// both serve and drain, so — unlike a `select!`-cancelled loop — they are
/// never destructively dropped mid-flight; they are released only on clean
/// exit, after which teardown does not need them.
///
/// - While serving (`drain_deadline == None`) the loop is behaviorally
/// identical to before: the shutdown arm, the deadline arm, and the
/// peers-empty early-exit are all guarded off, so the hot per-packet path
/// and the `biased` order of the real arms are unchanged.
/// - When `shutdown` fires, the loop calls [`Node::enter_drain`] once
/// (broadcast Disconnect, gate the reconciler off) and arms the bounded
/// deadline, then keeps servicing inbound/tick/peer-removal until all
/// peers clear or the deadline elapses, then returns. The caller
/// ([`Node::finish_shutdown`]) closes the window and tears down.
pub async fn run_rx_loop_with_shutdown(
&mut self,
shutdown: impl std::future::Future<Output = ()>,
) -> Result<(), NodeError> {
tokio::pin!(shutdown);
// `None` = serving; `Some(deadline)` = draining (bounded window).
let mut drain_deadline: Option<tokio::time::Instant> = None;
let mut packet_rx = self.packet_rx.take().ok_or(NodeError::NotStarted)?;
// Take the TUN outbound receiver, or create a dummy channel that never
// produces messages (when TUN is disabled). Holding the sender prevents
// the channel from closing.
let (mut tun_outbound_rx, _tun_guard) = match self.tun_outbound_rx.take() {
let (mut tun_outbound_rx, _tun_guard) = match self.supervisor.tun_outbound_rx.take() {
Some(rx) => (rx, None),
None => {
let (tx, rx) = tokio::sync::mpsc::channel(1);
@@ -57,7 +89,7 @@ impl Node {
// Take the DNS identity receiver, or create a dummy channel (when DNS
// is disabled). Same pattern as TUN outbound.
let (mut dns_identity_rx, _dns_guard) = match self.dns_identity_rx.take() {
let (mut dns_identity_rx, _dns_guard) = match self.supervisor.dns_identity_rx.take() {
Some(rx) => (rx, None),
None => {
let (tx, rx) = tokio::sync::mpsc::channel(1);
@@ -65,8 +97,20 @@ impl Node {
}
};
let mut tick =
tokio::time::interval(Duration::from_secs(self.config().node.tick_interval_secs));
// Take the runtime child-liveness receiver, or a dummy channel (when the
// node was seeded straight into Running without a start()). Holding the
// dummy sender in the guard keeps the channel open. Same pattern as TUN
// outbound / DNS identity.
let (mut child_exit_rx, _child_exit_guard) = match self.child_exit_rx.take() {
Some(rx) => (rx, None),
None => {
let (tx, rx) = tokio::sync::mpsc::channel(1);
(rx, Some(tx))
}
};
let tick_period = Duration::from_secs(self.config().node.tick_interval_secs);
let mut tick = tokio::time::interval(tick_period);
// Set up control socket channel
let (control_tx, mut control_rx) =
@@ -122,6 +166,13 @@ impl Node {
crate::perf_profile::maybe_spawn_reporter();
loop {
// Bounded drain mode: break as soon as all peers have cleared. In
// normal mode (`None`) this short-circuits before touching
// `self.peers`, so the loop is byte-identical.
if drain_deadline.is_some() && self.peers.is_empty() {
info!("Drain complete: all peers cleared, ending drain loop");
break;
}
tokio::select! {
biased;
// Decrypt-worker fallback drains FIRST. Under sustained
@@ -214,6 +265,27 @@ impl Node {
}
}
}
// Runtime child-liveness. Placed AFTER `packet_rx` so the hot
// inbound path keeps its `biased` priority. A directly-observable
// child (TUN threads, DNS/mDNS/Nostr) exited on its own; feed the
// FSM, which republishes health (Degraded here — a Running node
// always has ≥1 transport up). `on_child_exited` only ever emits
// `PublishState`; other variants are ignored defensively.
maybe_child = child_exit_rx.recv() => {
if let Some(child) = maybe_child {
let actions = self
.supervisor
.fsm
.step(crate::node::lifecycle::supervisor::Event::ChildExited { child });
for action in actions {
if let crate::node::lifecycle::supervisor::Action::PublishState(ns) =
action
{
self.supervisor.state = ns;
}
}
}
}
Some(ipv6_packet) = tun_outbound_rx.recv() => {
self.handle_tun_outbound(ipv6_packet).await;
let mut drained = 0;
@@ -249,41 +321,116 @@ impl Node {
).await;
let _ = response_tx.send(response);
}
_ = tick.tick() => {
self.check_timeouts();
let now_ms = Self::now_ms();
self.reload_peer_acl().await;
// The host map hot-reloads on the same tick as the ACL. It
// is polled separately from `reload_peer_acl` because the
// ACL's embedded alias reloader and this snapshot are
// distinct resources; the `path_mtu_lookup` cache and the
// `nostr_discovery` subsystem are deliberately excluded
// from `Reloadable` since neither reloads from a backing
// file (see `node::reloadable`).
self.reload_host_map().await;
self.poll_pending_connects().await;
self.poll_nostr_discovery().await;
self.poll_lan_discovery().await;
self.resend_pending_handshakes(now_ms).await;
self.resend_pending_rekeys(now_ms).await;
self.resend_pending_session_handshakes(now_ms).await;
self.resend_pending_session_msg3(now_ms).await;
self.purge_idle_sessions(now_ms);
self.process_pending_retries(now_ms).await;
self.check_tree_state().await;
self.check_bloom_state().await;
self.compute_mesh_size();
self.record_stats_history();
self.check_mmp_reports().await;
self.check_session_mmp_reports().await;
self.check_link_heartbeats().await;
self.check_rekey().await;
self.check_session_rekey().await;
self.check_pending_lookups(now_ms).await;
self.poll_transport_discovery().await;
self.sample_transport_congestion();
#[cfg(any(target_os = "linux", target_os = "macos"))]
self.activate_connected_udp_sessions().await;
deadline = tick.tick() => {
// Tick-body instrumentation. The gate is read ONCE per tick
// into `instr_on`, which is then passed explicitly to every
// `instr_step!` invocation — macro hygiene makes a call-site
// local invisible inside the macro body. With the
// `profiling` feature off, `gate()` is a `const fn`
// returning false and the macro is a pure pass-through, so
// the whole arm compiles to the uninstrumented sequence.
//
// `tick_entry` records how late this entry is against the
// deadline the interval scheduled it for. That is the
// measurement this instrumentation exists for: the arm is
// polled LAST under `biased;`, so the
// lateness IS the time it spent waiting behind the packet,
// TUN and control arms. `tick()` hands back its scheduled
// deadline, so this is a subtraction rather than a model.
// The whole-tick span below measures the body alone.
let instr_on = crate::instr::gate();
crate::instr::tick_entry(instr_on, deadline.into_std(), std::time::Instant::now());
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::WholeTick, {
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckTimeouts,
self.check_timeouts());
let now_ms = Self::now_ms();
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ReloadPeerAcl,
self.reload_peer_acl().await);
// The host map hot-reloads on the same tick as the ACL. It
// is polled separately from `reload_peer_acl` because the
// ACL's embedded alias reloader and this snapshot are
// distinct resources; the `path_mtu_lookup` cache and the
// `nostr_rendezvous` subsystem are deliberately excluded
// from `Reloadable` since neither reloads from a backing
// file (see `node::reloadable`).
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ReloadHostMap,
self.reload_host_map().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollPendingConnects,
self.poll_pending_connects().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollNostrRendezvous,
self.poll_nostr_rendezvous().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollLanRendezvous,
self.poll_lan_rendezvous().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::DrivePeerTimers,
self.drive_peer_timers(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingRekeys,
self.resend_pending_rekeys(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingFmpRekeyMsg3,
self.resend_pending_fmp_rekey_msg3(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingSessionHandshakes,
self.resend_pending_session_handshakes(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ResendPendingSessionMsg3,
self.resend_pending_session_msg3(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PurgeIdleSessions,
self.purge_idle_sessions(now_ms));
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ProcessPendingRetries,
self.process_pending_retries(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckTreeState,
self.check_tree_state().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckBloomState,
self.check_bloom_state().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ComputeMeshSize,
self.compute_mesh_size());
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::RecordStatsHistory,
self.record_stats_history());
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckMmpReports,
self.check_mmp_reports().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckSessionMmpReports,
self.check_session_mmp_reports().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckLinkHeartbeats,
self.check_link_heartbeats().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckRekey,
self.check_rekey().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckSessionRekey,
self.check_session_rekey().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::CheckPendingLookups,
self.check_pending_lookups(now_ms).await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::PollTransportDiscovery,
self.poll_transport_discovery().await);
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::SampleTransportCongestion,
self.sample_transport_congestion());
#[cfg(any(target_os = "linux", target_os = "macos"))]
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::ActivateConnectedUdpSessions,
self.activate_connected_udp_sessions().await);
// Debug-build sweep of the peer-lifecycle map invariant
// (leaked machines / machine-less legs); two map scans,
// compiled out of release builds.
#[cfg(debug_assertions)]
instr_step!(instr_on, crate::instr::Domain::Tick, crate::instr::Step::DebugAssertPeerMapsCoherent,
self.debug_assert_peer_maps_coherent());
});
crate::instr::tick_gauges(instr_on, self.peers.len() as u64);
}
// Shutdown signal → enter the bounded drain in place, ONCE.
// Gated on `is_none()` so it only fires while serving; after
// entering drain the arm is disabled (the completed signal is
// never polled again) and the deadline arm below bounds the
// window. Placed after the real arms so their `biased` priority
// is unchanged, and inert while serving with `pending()`.
_ = &mut shutdown, if drain_deadline.is_none() => {
self.enter_drain().await;
drain_deadline =
Some(tokio::time::Instant::now() + self.config().node.drain_timeout());
}
// Bounded drain deadline (drain mode only). Placed LAST so the
// `biased` priority of the normal arms is unchanged, and gated
// on `is_some()` so in normal mode the branch is disabled — the
// future is created but never polled and never fires.
_ = tokio::time::sleep_until(
drain_deadline.unwrap_or_else(tokio::time::Instant::now)
), if drain_deadline.is_some() => {
info!("Drain deadline elapsed, ending drain loop");
break;
}
}
}
@@ -319,12 +466,16 @@ impl Node {
// though no msg1/msg2 exchange can ever succeed. Bump the
// discovery-layer cooldown to the long protocol-mismatch
// window and emit a single WARN per fresh observation.
if self.bootstrap_transports.contains(&packet.transport_id)
if self
.supervisor
.nostr_rendezvous
.is_bootstrap_transport(&packet.transport_id)
&& let Some(npub) = self
.bootstrap_transport_npubs
.get(&packet.transport_id)
.supervisor
.nostr_rendezvous
.bootstrap_transport_npub(&packet.transport_id)
.cloned()
&& let Some(handle) = self.nostr_discovery_handle()
&& let Some(handle) = self.nostr_rendezvous_handle()
{
let now_ms = Self::now_ms();
let cooldown_secs = handle.protocol_mismatch_cooldown_secs();
@@ -352,6 +503,9 @@ impl Node {
PHASE_MSG2 => {
self.handle_msg2(packet).await;
}
PHASE_MSG3 => {
self.handle_msg3(packet).await;
}
_ => {
debug!(
phase = prefix.phase,
src/node/decrypt_worker.rs
+11 -12
View File
@@ -112,8 +112,8 @@ pub(crate) struct DecryptJob {
pub fmp_counter: u64,
/// Flag byte from the FMP outer header. Carried through the
/// fallback so the rx_loop bounce arm can extract `CE` and `SP`
/// for ECN propagation, MMP stats, and spin-bit RTT
/// observation — these used to be dropped on the worker path
/// for ECN propagation and MMP stats — these used to be dropped
/// on the worker path
/// because the bounce hardcoded `fmp_flags: 0`.
pub fmp_flags: u8,
/// 16-byte FMP outer header used as AAD during AEAD open.
@@ -544,8 +544,7 @@ mod tests {
/// `DecryptFallback.fmp_flags`. Pre-fix the worker hardcoded
/// `fmp_flags: 0`, dropping CE / SP on every packet handled by
/// the production worker path (i.e. every bulk-data packet).
/// Loss of CE wrecks ECN propagation; loss of SP wrecks
/// spin-bit RTT observation.
/// Loss of CE wrecks ECN propagation.
///
/// Drives the worker's `handle_job` directly: build an FMP wire
/// packet sealed with a known cipher, ship a `DecryptJob` with
@@ -561,15 +560,15 @@ mod tests {
let open_cipher = LessSafeKey::new(unbound2);
let counter: u64 = 7;
const HDR: usize = crate::node::wire::ESTABLISHED_HEADER_SIZE;
const HDR: usize = crate::proto::fmp::wire::ESTABLISHED_HEADER_SIZE;
// Build a wire packet `[16-byte header][4-byte inner ts][1 byte link msg]`
// with capacity for the trailing AEAD tag. Header bytes
// double as AAD and as the on-wire prefix.
let mut wire = Vec::with_capacity(HDR + 4 + 1 + 16);
// Header: fill the flags byte (the second byte) with both
// FLAG_CE and FLAG_SP set; the rest is uninterpreted by the
// worker (it just AADs the whole 16 bytes).
let flags_byte = crate::node::wire::FLAG_CE | crate::node::wire::FLAG_SP;
// FLAG_CE and FLAG_KEY_EPOCH set; the rest is uninterpreted by
// the worker (it just AADs the whole 16 bytes).
let flags_byte = crate::proto::fmp::wire::FLAG_CE | crate::proto::fmp::wire::FLAG_KEY_EPOCH;
let mut header = [0u8; HDR];
header[1] = flags_byte;
wire.extend_from_slice(&header);
@@ -626,12 +625,12 @@ mod tests {
"fmp_flags must round-trip from DecryptJob to DecryptFallback"
);
assert!(
fallback.fmp_flags & crate::node::wire::FLAG_CE != 0,
fallback.fmp_flags & crate::proto::fmp::wire::FLAG_CE != 0,
"FLAG_CE bit lost on worker path"
);
assert!(
fallback.fmp_flags & crate::node::wire::FLAG_SP != 0,
"FLAG_SP bit lost on worker path"
fallback.fmp_flags & crate::proto::fmp::wire::FLAG_KEY_EPOCH != 0,
"FLAG_KEY_EPOCH bit lost on worker path"
);
}
@@ -724,7 +723,7 @@ mod tests {
let open_cipher = LessSafeKey::new(unbound);
let counter: u64 = 11;
const HDR: usize = crate::node::wire::ESTABLISHED_HEADER_SIZE;
const HDR: usize = crate::proto::fmp::wire::ESTABLISHED_HEADER_SIZE;
let header = [0u8; HDR];
let mut wire = Vec::with_capacity(HDR + 4 + 1 + 16);
wire.extend_from_slice(&header);
src/node/discovery_rate_limit.rs
-376
View File
@@ -1,376 +0,0 @@
//! Discovery protocol rate limiting and backoff.
//!
//! Two complementary mechanisms:
//!
//! - **`DiscoveryBackoff`** (originator-side, optional): Exponential
//! suppression of fresh lookups after the per-attempt sequence in
//! `node.discovery.attempt_timeouts_secs` has been exhausted.
//! **Disabled by default** (base/cap = 0); the per-attempt sequence
//! is the only retry pacing in the standard configuration. Reset on
//! topology changes (parent change, new peer, first RTT, reconnection).
//!
//! - **`DiscoveryForwardRateLimiter`** (transit-side): Per-target minimum
//! interval for forwarded requests. Defense-in-depth against misbehaving
//! nodes generating fresh request_ids at high rate.
use crate::NodeAddr;
use std::collections::HashMap;
use std::time::{Duration, Instant};
// ============================================================================
// Originator-side: Discovery Backoff
// ============================================================================
/// Default base backoff after first lookup failure. `0` = disabled.
const DEFAULT_BACKOFF_BASE_SECS: u64 = 0;
/// Default maximum backoff cap. `0` = disabled.
const DEFAULT_BACKOFF_MAX_SECS: u64 = 0;
/// Backoff multiplier per consecutive failure.
const BACKOFF_MULTIPLIER: u64 = 2;
/// Exponential backoff for failed discovery lookups.
///
/// Tracks targets whose lookups have timed out and suppresses
/// re-initiation with increasing delays. Cleared on topology changes.
pub struct DiscoveryBackoff {
/// Maps target → (suppress_until, consecutive_failures).
entries: HashMap<NodeAddr, BackoffEntry>,
/// Base backoff duration (first failure).
base: Duration,
/// Maximum backoff cap.
max: Duration,
}
struct BackoffEntry {
/// Don't re-initiate until this instant.
suppress_until: Instant,
/// Consecutive failures (drives exponential backoff).
failures: u32,
}
impl DiscoveryBackoff {
/// Create with default parameters (disabled — base/cap = 0).
pub fn new() -> Self {
Self::with_params(DEFAULT_BACKOFF_BASE_SECS, DEFAULT_BACKOFF_MAX_SECS)
}
/// Create with custom base and max backoff in seconds.
pub fn with_params(base_secs: u64, max_secs: u64) -> Self {
Self {
entries: HashMap::new(),
base: Duration::from_secs(base_secs),
max: Duration::from_secs(max_secs),
}
}
/// Check if a lookup for this target is suppressed.
///
/// Returns true if the target is in backoff and should not be
/// looked up yet.
pub fn is_suppressed(&self, target: &NodeAddr) -> bool {
if let Some(entry) = self.entries.get(target) {
Instant::now() < entry.suppress_until
} else {
false
}
}
/// Record a lookup failure (timeout) for a target.
///
/// Increments the failure count and sets the next suppression
/// window using exponential backoff.
pub fn record_failure(&mut self, target: &NodeAddr) {
let now = Instant::now();
let failures = self.entries.get(target).map_or(0, |e| e.failures) + 1;
let backoff_secs = self
.base
.as_secs()
.saturating_mul(BACKOFF_MULTIPLIER.saturating_pow(failures.saturating_sub(1)));
let backoff = Duration::from_secs(backoff_secs.min(self.max.as_secs()));
self.entries.insert(
*target,
BackoffEntry {
suppress_until: now + backoff,
failures,
},
);
}
/// Record a successful lookup — remove backoff for this target.
pub fn record_success(&mut self, target: &NodeAddr) {
self.entries.remove(target);
}
/// Clear all backoff entries.
///
/// Called on topology changes that might make previously-unreachable
/// targets reachable (parent change, new peer, first RTT, reconnection).
pub fn reset_all(&mut self) {
self.entries.clear();
}
/// Whether any entries exist.
pub fn is_empty(&self) -> bool {
self.entries.is_empty()
}
/// Current number of entries.
pub fn entry_count(&self) -> usize {
self.entries.len()
}
/// Get the failure count for a target (for logging).
pub fn failure_count(&self, target: &NodeAddr) -> u32 {
self.entries.get(target).map_or(0, |e| e.failures)
}
#[cfg(test)]
pub fn len(&self) -> usize {
self.entries.len()
}
}
impl Default for DiscoveryBackoff {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Transit-side: Discovery Forward Rate Limiter
// ============================================================================
/// Default minimum interval between forwarded lookups for the same target.
const DEFAULT_FORWARD_MIN_INTERVAL: Duration = Duration::from_secs(2);
/// Maximum age of entries before cleanup.
const FORWARD_MAX_AGE: Duration = Duration::from_secs(60);
/// Rate limiter for forwarded discovery requests.
///
/// Tracks the last time a LookupRequest was forwarded for each target
/// and enforces a minimum interval to prevent floods from misbehaving
/// nodes generating fresh request_ids.
pub struct DiscoveryForwardRateLimiter {
last_forwarded: HashMap<NodeAddr, Instant>,
min_interval: Duration,
max_age: Duration,
}
impl DiscoveryForwardRateLimiter {
/// Create with default parameters (2s interval).
pub fn new() -> Self {
Self {
last_forwarded: HashMap::new(),
min_interval: DEFAULT_FORWARD_MIN_INTERVAL,
max_age: FORWARD_MAX_AGE,
}
}
/// Create with a custom minimum interval.
pub fn with_interval(min_interval: Duration) -> Self {
Self {
last_forwarded: HashMap::new(),
min_interval,
max_age: FORWARD_MAX_AGE,
}
}
/// Check if we should forward a lookup for this target.
///
/// Returns true if enough time has passed since the last forward
/// for this target. Updates internal state when returning true.
pub fn should_forward(&mut self, target: &NodeAddr) -> bool {
let now = Instant::now();
if let Some(&last) = self.last_forwarded.get(target)
&& now.duration_since(last) < self.min_interval
{
return false;
}
self.last_forwarded.insert(*target, now);
self.cleanup(now);
true
}
/// Replace the minimum interval (e.g., set to zero to disable).
#[cfg(test)]
pub fn set_interval(&mut self, interval: Duration) {
self.min_interval = interval;
}
/// Remove entries older than max_age.
fn cleanup(&mut self, now: Instant) {
self.last_forwarded
.retain(|_, &mut last| now.duration_since(last) < self.max_age);
}
#[cfg(test)]
pub fn len(&self) -> usize {
self.last_forwarded.len()
}
}
impl Default for DiscoveryForwardRateLimiter {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
use std::thread;
fn addr(val: u8) -> NodeAddr {
let mut bytes = [0u8; 16];
bytes[0] = val;
NodeAddr::from_bytes(bytes)
}
// --- DiscoveryBackoff tests ---
#[test]
fn test_backoff_not_suppressed_initially() {
let backoff = DiscoveryBackoff::new();
assert!(!backoff.is_suppressed(&addr(1)));
}
#[test]
fn test_backoff_suppressed_after_failure() {
// Backoff is opt-in; exercise the suppression path with explicit params.
let mut backoff = DiscoveryBackoff::with_params(30, 300);
backoff.record_failure(&addr(1));
assert!(backoff.is_suppressed(&addr(1)));
// Different target not affected
assert!(!backoff.is_suppressed(&addr(2)));
}
#[test]
fn test_backoff_cleared_on_success() {
let mut backoff = DiscoveryBackoff::with_params(30, 300);
backoff.record_failure(&addr(1));
assert!(backoff.is_suppressed(&addr(1)));
backoff.record_success(&addr(1));
assert!(!backoff.is_suppressed(&addr(1)));
}
#[test]
fn test_backoff_reset_all() {
let mut backoff = DiscoveryBackoff::new();
backoff.record_failure(&addr(1));
backoff.record_failure(&addr(2));
assert_eq!(backoff.len(), 2);
backoff.reset_all();
assert_eq!(backoff.len(), 0);
assert!(!backoff.is_suppressed(&addr(1)));
}
#[test]
fn test_backoff_exponential() {
let mut backoff = DiscoveryBackoff::with_params(1, 300);
// First failure: 1s backoff
backoff.record_failure(&addr(1));
assert_eq!(backoff.failure_count(&addr(1)), 1);
// Second failure: 2s backoff
backoff.record_failure(&addr(1));
assert_eq!(backoff.failure_count(&addr(1)), 2);
// Third failure: 4s backoff
backoff.record_failure(&addr(1));
assert_eq!(backoff.failure_count(&addr(1)), 3);
}
#[test]
fn test_backoff_expires() {
let mut backoff = DiscoveryBackoff::with_params(0, 0);
backoff.record_failure(&addr(1));
// With 0s backoff, should not be suppressed
assert!(!backoff.is_suppressed(&addr(1)));
}
#[test]
fn test_backoff_capped() {
let mut backoff = DiscoveryBackoff::with_params(1, 10);
// Record many failures
for _ in 0..20 {
backoff.record_failure(&addr(1));
}
// Backoff should be capped at max (10s), not overflow
let entry = backoff.entries.get(&addr(1)).unwrap();
let remaining = entry.suppress_until.duration_since(Instant::now());
assert!(remaining <= Duration::from_secs(11));
}
// --- DiscoveryForwardRateLimiter tests ---
#[test]
fn test_forward_first_allowed() {
let mut limiter = DiscoveryForwardRateLimiter::new();
assert!(limiter.should_forward(&addr(1)));
}
#[test]
fn test_forward_rapid_rate_limited() {
let mut limiter = DiscoveryForwardRateLimiter::new();
assert!(limiter.should_forward(&addr(1)));
assert!(!limiter.should_forward(&addr(1)));
assert!(!limiter.should_forward(&addr(1)));
}
#[test]
fn test_forward_different_targets_independent() {
let mut limiter = DiscoveryForwardRateLimiter::new();
assert!(limiter.should_forward(&addr(1)));
assert!(limiter.should_forward(&addr(2)));
assert!(!limiter.should_forward(&addr(1)));
assert!(!limiter.should_forward(&addr(2)));
}
#[test]
fn test_forward_allowed_after_interval() {
let mut limiter = DiscoveryForwardRateLimiter::with_interval(Duration::from_millis(100));
assert!(limiter.should_forward(&addr(1)));
thread::sleep(Duration::from_millis(110));
assert!(limiter.should_forward(&addr(1)));
}
#[test]
fn test_forward_cleanup_removes_old() {
let mut limiter = DiscoveryForwardRateLimiter::new();
assert!(limiter.should_forward(&addr(1)));
assert!(limiter.should_forward(&addr(2)));
assert_eq!(limiter.len(), 2);
let future = Instant::now() + Duration::from_secs(61);
limiter.cleanup(future);
assert_eq!(limiter.len(), 0);
}
#[test]
fn test_forward_cleanup_preserves_recent() {
let mut limiter = DiscoveryForwardRateLimiter::new();
assert!(limiter.should_forward(&addr(1)));
assert_eq!(limiter.len(), 1);
limiter.cleanup(Instant::now());
assert_eq!(limiter.len(), 1);
}
}
src/node/encrypt_worker.rs
+16 -19
View File
@@ -50,9 +50,9 @@
// warnings rather than gate every function individually.
#![cfg_attr(not(unix), allow(dead_code))]
use crate::node::session_wire::FSP_HEADER_SIZE;
use crate::node::wire::ESTABLISHED_HEADER_SIZE;
use crate::transport::udp::socket::AsyncUdpSocket;
use crate::proto::fmp::wire::ESTABLISHED_HEADER_SIZE;
use crate::proto::fsp::wire::FSP_HEADER_SIZE;
use crate::transport::udp::io::AsyncUdpSocket;
#[cfg(not(target_os = "macos"))]
use crossbeam_channel::{Receiver, SendError, Sender, TrySendError, bounded};
use ring::aead::{Aad, LessSafeKey, Nonce};
@@ -132,8 +132,7 @@ pub(crate) struct FmpSendJob {
/// the job completes and the worker drops it, only the peer's
/// strong ref remains.
#[cfg(any(target_os = "linux", target_os = "macos"))]
pub connected_socket:
Option<std::sync::Arc<crate::transport::udp::connected_peer::ConnectedPeerSocket>>,
pub connected_socket: Option<std::sync::Arc<crate::peer::connected_udp::ConnectedPeerSocket>>,
/// Bulk endpoint data may be dropped when the kernel reports UDP
/// send-queue exhaustion. Control/rekey frames keep retrying so
/// congestion cannot strand the session.
@@ -716,8 +715,7 @@ fn mac_now_ms() -> u64 {
struct MacSequencedSendFlow {
key: MacSendFlowKey,
socket: AsyncUdpSocket,
connected_socket:
Option<std::sync::Arc<crate::transport::udp::connected_peer::ConnectedPeerSocket>>,
connected_socket: Option<std::sync::Arc<crate::peer::connected_udp::ConnectedPeerSocket>>,
dest_addr: SocketAddr,
next_seq: std::sync::atomic::AtomicU64,
last_used_ms: std::sync::atomic::AtomicU64,
@@ -754,9 +752,7 @@ impl MacSequencedSendFlow {
fn spawn(
key: MacSendFlowKey,
socket: AsyncUdpSocket,
connected_socket: Option<
std::sync::Arc<crate::transport::udp::connected_peer::ConnectedPeerSocket>,
>,
connected_socket: Option<std::sync::Arc<crate::peer::connected_udp::ConnectedPeerSocket>>,
dest_addr: SocketAddr,
now_ms: u64,
) -> Arc<Self> {
@@ -1024,8 +1020,7 @@ fn flush_batch_sync(
struct EncryptedGroup {
socket: AsyncUdpSocket,
#[cfg(any(target_os = "linux", target_os = "macos"))]
connected_socket:
Option<std::sync::Arc<crate::transport::udp::connected_peer::ConnectedPeerSocket>>,
connected_socket: Option<std::sync::Arc<crate::peer::connected_udp::ConnectedPeerSocket>>,
dest_addr: SocketAddr,
wire_packets: Vec<Vec<u8>>,
drop_on_backpressure: bool,
@@ -1710,7 +1705,7 @@ fn send_batch_gso(
}
/// Direct `sendmmsg(2)` wrapper for the sync worker. The
/// `transport::udp::socket` module's existing `send_batch` is
/// `transport::udp::io` module's existing `send_batch` is
/// pub(crate) on `UdpRawSocket`, but we don't have a handle to the
/// raw socket from here — we just have the FD. Re-implementing
/// inline is ~15 lines and avoids tunnelling the inner socket
@@ -1780,7 +1775,7 @@ fn send_batch_raw(
#[cfg(all(test, unix))]
mod unix_tests {
use super::*;
use crate::transport::udp::socket::UdpRawSocket;
use crate::transport::udp::io::UdpRawSocket;
use ring::aead::{LessSafeKey, UnboundKey};
use std::net::UdpSocket;
@@ -1904,10 +1899,12 @@ mod unix_tests {
#[test]
fn pipelined_send_wire_layout_roundtrips_canonical_decoders() {
use crate::NodeAddr;
use crate::node::session_wire::build_fsp_header;
use crate::node::wire::{EncryptedHeader, FLAG_KEY_EPOCH, build_established_header};
use crate::noise::TAG_SIZE;
use crate::protocol::{LinkMessageType, SESSION_DATAGRAM_HEADER_SIZE, SessionDatagramRef};
use crate::proto::fmp::wire::{EncryptedHeader, FLAG_KEY_EPOCH, build_established_header};
use crate::proto::fsp::wire::build_fsp_header;
use crate::proto::link::{
LinkMessageType, SESSION_DATAGRAM_HEADER_SIZE, SessionDatagramRef,
};
use crate::utils::index::SessionIndex;
let rt = tokio::runtime::Builder::new_current_thread()
@@ -2218,7 +2215,7 @@ mod tests {
/// AsRawFd impl.
#[test]
fn flush_batch_routes_each_target_separately() {
use crate::transport::udp::socket::UdpRawSocket;
use crate::transport::udp::io::UdpRawSocket;
use ring::aead::{LessSafeKey, UnboundKey};
use std::net::UdpSocket;
@@ -2264,7 +2261,7 @@ mod tests {
const B_WIRE: usize = 16 + B_PLAINTEXT + 16; // 96
fn make_job(
socket: crate::transport::udp::socket::AsyncUdpSocket,
socket: crate::transport::udp::io::AsyncUdpSocket,
cipher: &LessSafeKey,
counter: u64,
dest: SocketAddr,

Some files were not shown because too many files have changed in this diff Show More