A leaf-profile node that held the smallest NodeAddr would self-elect as
tree root, but its peers refuse a non-full node as a parent, so it formed
an isolated second root and partitioned the mesh: a multi-hop session from
the leaf to a non-adjacent full node then failed because the far node could
not route a handshake reply back into the leaf's separate coordinate tree.
Gate tree-state self-election on a new self_is_leaf flag, set from the node
profile at construction. A leaf now attaches under its full upstream and
holds that subtree's coordinate for its own routing. That coordinate's
self < root would be rejected by the root-min wire check, but a leaf never
announces it; it reaches peers only via the coordinates carried on the
leaf's session frames, and the upstream already advertises the leaf in its
bloom filter, so the far node routes back through the upstream.
Also fix Node::with_identity to derive the node profile, leaf-only flag,
and bloom state from the config, matching Node::new; it previously
hardcoded the full profile and silently dropped a configured leaf or
non-routing profile.
Covered by sans-IO decision tests (leaf does not self-elect; leaf keeps its
coordinate under a larger-rooted parent) and an end-to-end multi-hop
regression, each confirmed to fail if the gate is reverted.
Brings up a Full/Full/NonRouting/Leaf mesh matching the mixed-profile
Docker topology, asserts each node peers to the degree its role and
topology dictate (A=3, B=2, C=2, D=1), and verifies end-to-end data
between every directly reachable pair (Full-Full, Full-NonRouting,
Full-Leaf). Adds run_tree_test_with_profiles, a per-node-profile sibling
of run_tree_test.
The Leaf-to-Full multi-hop the Docker suite also checks is deliberately
left out for now: a Leaf node's multi-hop session initiation on the XX
line is not yet reliable, so the Docker mixed-profile suite is retained
for that path rather than replaced with a flaky assertion.
Brings the handshake-leg deletion onto the XX line: the leg struct
and src/peer/connection.rs are gone, the Noise handles and crypto
methods now live on PeerMachine, and every leg accessor reads the
machine's own ConnectionState.
The incoming branch speaks Noise IK, this one speaks XX, so the crypto
methods could not be taken as they arrived. Git added the IK versions
to machine.rs with no conflict marker, and complete_handshake_msg3
-- which XX needs and IK has no counterpart for -- was absent
from the merged tree entirely. All four are re-expressed here: the
incoming structure (the Option<HandshakeCrypto> handle, the self.conn
reads, the borrow scoping that hoists carrier writes out of the leg
borrow) carrying this line's XX bodies. start_handshake is the one
that mattered most: its signature is identical on both lines, so it
compiled silently, and left alone it would have emitted an IK msg1
through the two-argument new_initiator and panicked every anonymous
dial on an expect for an identity XX does not have at that point. It
now uses the one-argument XX form and no expect.
handle_msg1 keeps this line's flow, not the incoming one's. The two
functions share a name but not a shape -- the admission gate moved
out of msg1 into msg3 when the line went to XX, and identity is
unknown until msg3. The machine is built above the crypto because
it now drives it, but it stays a local until the late insert, so a
rejected msg1 still leaves no registry trace and the drop-the-local
error arms are unchanged. Parking it at SentMsg2 after the index is
allocated needed a transition the birth constructor could not provide,
so that constructor now delegates to it.
Node::start_handshake is re-expressed the same way and for the same
reason: keeping the machine a local until all fallible setup has
succeeded preserves the existing error arms exactly, with no machine
disposal to add.
The two leg-to-carrier mirror blocks collapse. With one carrier
the writes they mirrored are self-assignment. Their content
is preserved: the completion touch comes from the relocated
complete_handshake, on the same clock, and the negotiated profile
from process_fmp_negotiation, which now takes the machine directly
and writes through set_conn_peer_profile. Worth recording, because
the collapse was justified on a narrower claim that does not hold:
complete_handshake writes only the touch. It never wrote the profile
on either line. The two writes land on the same carrier at the same
point, so the collapse is neutral, but it is discharged jointly by
two methods rather than by complete_handshake alone.
Three inbound tests are dropped rather than carried. Each asserts
a property at msg1 that XX does not have there: an ACL decision, a
no-registry-trace guard on that decision, and an identity learn on the
carrier. Under XX msg1 is ephemeral-only, so none of the three has a
landing site at that step. A comment at each drop site records where
the property actually lives on this line. One of them was a deletion
here that the incoming branch had merely modified, so its removal
is this line's own prior decision rather than a new one. The same is
true of the craft_and_send_msg1 helper and of HandshakeSeed::inbound,
both left without callers once those tests went.
Also carried: peer_actions.rs had no conflict and no incoming change,
but three sites reached deleted Node accessors and are repointed at
the machine; the EstablishView snapshot is not resurrected, as this
line has never had it; and the connection-state doc keeps this line's
XX wording with the registry-independence invariant grafted in.
Tests built a free-standing PeerConnection, mutated it, and handed it to
Node::add_connection by value. None of those sites survives the removal of
PeerConnection, so converting them afterwards would mean one enormous commit
that cannot be reviewed honestly. Convert them now, while the struct still
exists and the conversion can be validated against a green tree.
Adds a cfg(test) Node::seed_handshake_machine plus a HandshakeSeed builder,
and rewrites make_completed_connection (now seed_completed_connection) and
the twenty inline builders onto it. add_connection keeps its body and loses
its test callers.
The builder's carrier seeding is a verbatim copy of add_connection's: the two
conditional writes for their_index and transport_id, then set_leg, through the
same entry().or_insert_with() so an existing leg-less machine keeps its
constructor-side fields. Nothing else reaches the carrier -- our_index,
source_addr, post-construction started_at and the stored handshake bytes stay
leg-only. Seeding more than that would let these tests observe a carrier
richer than production's and keep passing even if a later production
write-lift were missed.
The Noise exchange now runs on the already-seeded leg rather than before the
hand-over. That is neutral because the only read of expected_identity is
guarded by is_outbound, and no crypto method allocates a session index.
Also adds a compile-time check that PeerAction is Clone + Eq, which is what
keeps a runtime handle from being smuggled into an action payload.
Introduce a sans-IO lifecycle supervisor: a synchronous step(event) ->
[action] state machine (SupervisorFsm) that authors the substrate-child
spawn and teardown order, plus an owner struct (Supervisor) holding the
substrate runtime fields and embedding the FSM. The substrate-lifecycle
fields leave Node's flat list into the owner - state, packet_tx, the TUN
reader/writer handles + shutdown fd + channels, the DNS task + identity
channel, the Nostr/LAN rendezvous drivers, and the encrypt/decrypt worker
pools; the dataplane keeps packet_rx.
start()/stop() become the driver executing the FSM's SpawnChild/StopChild
actions: same children, same order, same warn/debug-and-continue on
optional failures, same logs, same NodeState transitions. Behavior- and
wire-neutral. The only determinism change is that transports now tear
down in ascending-id order, previously nondeterministic HashMap iteration.
The bounded Draining phase and the Running{Full|Degraded} health split
land as separate follow-on commits.
Forward-merge the master-line FMP-establish relocations onto next: the FMP
link wire codec into proto/fmp/wire.rs and PromotionResult into
proto/fmp/core.rs. The wire move was hand-resolved against next's wire
format (msg3/Msg3Header/build_msg3, FMP_VERSION=1, no spin-bit flag), so
next's behavior is unchanged. Behavior-neutral; full lib suite green at
next baseline.
Move the PromotionResult enum (and its impl) out of peer::mod and into
proto/fmp/core.rs, alongside the cross_connection_winner tie-break helper
that was relocated the same way. This is FMP connection-lifecycle result
vocabulary, so it belongs in the FMP subsystem home rather than the peer
module.
Behavior-neutral pure type relocation: consumers import it from
crate::proto::fmp, and the crate-root public path crate::PromotionResult
is preserved via a re-export in lib.rs (mirroring cross_connection_winner).
Full lib suite green at baseline.
Reorganize the MMP subsystem, behavior unchanged throughout:
- Restore the pre-migration role split: move SenderState into sender.rs,
ReceiverState (with its GapTracker helper) into receiver.rs, MmpMetrics and
RrLog into metrics.rs, and PathMtuState into path_mtu.rs, leaving the Mmp
aggregate plus the peer/session state in state.rs (1339 to 196 lines). Home
the module constants in a new limits.rs and re-export them at the same paths.
Pure code-motion with module rewiring; visibility unchanged.
- Dissolve the 97-line src/mmp shell, which held only MmpConfig and the
monotonic mono_ms() clock: move MmpConfig into config/node.rs (re-exported as
crate::config::MmpConfig), move mono_ms() into a new top-level src/time.rs
shell time seam, repoint all callers, and delete src/mmp/. Also correct stale
src/mmp/ doc-comment path labels to proto/mmp/ where they name the protocol
primitives.
Forward-merge the sans-IO cleanup series (bloom fpr, discovery const/RNG-injection,
mmp state decomposition, mmp shell dissolution, STP clock-free classify core, STP
declaration split, shared proto/coord relocation, shared rate limiter/backoff,
typed proto errors dropping thiserror, shared bounds-checked codec reader/writer,
core/alloc import sweep, no_std-shaped filter math) onto the next wire-format line.
Reconciled master structure against the next wire semantics: the mmp role-module
split adopts next slim report format (spin-bit stays dropped, sender/receiver
build next reports), the discovery and fmp codecs keep next TLV and profile
negotiation while moving to the typed error, and the parent-eval handler keeps the
Full/Leaf profile filter under the new ParentEval/is_switch_suppressed seam.
Forward-merge the FSP session-protocol sans-IO migration and the src/protocol
teardown from the master-refactor line onto next. Converge proto/fsp across
both lines (core.rs byte-identical), keeping next's born-on-next residue: the
additive SessionSetup/SessionAck wire variants, the spin-bit-less unit
FspInnerFlags, and the XX handshake shell residue (negotiation-payload
piggyback, initiator identity check, min-size length checks) in
handlers/session.rs. src/protocol is removed on next as well; ProtocolError,
LinkMessageType, and SessionDatagram reach their proto/ homes.
Migrate the FSP end-to-end session subsystem into src/proto/fsp/ following the
established sans-IO shape, and retire the src/protocol grab-bag now that FSP was
its last occupant.
Relocate the FSP session wire (node/session_wire.rs plus the FSP message types
from protocol/session.rs) into proto/fsp/wire.rs. Hoist the pure decision logic
into proto/fsp/core.rs over plain-data SessionSnapshots returning an ordered
FspAction list the shell drives: session-rekey policy, msg3-resend
classification, post-decrypt epoch reaction, setup/dual-init tie-break,
coords/path-MTU emit-policy, bounded pending-queue, and IPv6 ECN. The
crypto-owning SessionEntry stays shell-side in node/session.rs (matching the FMP
ActivePeer pattern); proto/fsp is wire + core + limits only, with no proto->noise
dependency and no crypto.
Move the coords helpers to proto/stp/ (they serialize TreeCoordinate), and split
SessionMessageType: the encrypted-inner 0x10-0x1F variants stay in proto/fsp/wire.rs
while the 0x20-0x2F routing signals become a new RoutingSignalType in
proto/routing/wire.rs. Migrate the session-MMP shell adapter, which continues to
drive proto/mmp/.
Retire src/protocol: LinkMessageType and SessionDatagram move to a new shared
proto/link.rs, ProtocolError becomes proto::Error (relocated verbatim), the
deprecated MessageType alias and the unimported PROTOCOL_VERSION are dropped, and
src/protocol/ is deleted along with its lib.rs module declaration.
Behavior-neutral: wire bytes unchanged, oracle tests pass unedited except
mod-path relocation; adds rekey/epoch characterization tests and pure
poll/emit-policy core tests.
Forward-merge the STP sans-IO migration. The classify / proto-stp structure,
clock injection, crypto field-partition, and BTree collections come from the
master line; next's non-full/leaf parent-candidacy skip is preserved by passing
self.non_full_peers() at the four shell call sites (handle_tree_announce,
check_periodic_parent_reeval, the MMP first-RTT re-eval, and the greedy-tree
find_next_hop fallback) instead of an empty set, with non_full_peers returning a
BTreeSet. next's four non_full_peers skip tests fold into proto/stp/tests.
Migrate the full non-async spanning-tree surface into proto/stp/, mirroring the
discovery/routing/fmp/mmp conversions. The classification ladder (parent-switch /
self-root / loop-drop / ancestry-update / periodic-rebroadcast / parent-lost) moves
out of the async node handlers into a pure Stp classify layer returning a
TreeDecision the shell drives, with effect ordering and per-arm invalidation
preserved verbatim. src/tree/ relocates wholesale: TreeState + ParentDeclaration data
+ coordinates into proto/stp/{state,coordinate}, the flap-dampening / hold-down
cluster into a FlapDampener in limits.rs, and the wire codec into wire.rs. The clock
is injected as u64 (wall-clock secs for the escaping declaration timestamp, monotonic
ms for the dampening timers via mmp::mono_ms); declaration crypto is field-partitioned
so sign/verify/hash run in the shell while the in-core modules carry data +
signing_bytes only. Peer maps/sets move to BTree; core/state/coordinate/limits are
core+alloc clean, with wire.rs the one std-tethered file. Behavior-neutral:
characterization tests added for the handler decision arms; convergence suite and
ci-local (36/36) green.
Add a Loopback variant to TransportHandle backed by an unbounded
in-process channel and a shared address-to-receiver registry, so
node-level multi-node tests deliver packets directly between nodes
instead of over real localhost UDP sockets. This removes the kernel
UDP receive-buffer overflow that dropped handshake packets when many
tests ran in parallel under CPU contention, and lets the large-network
convergence tests run reliably in the default suite again (their
parallel-load ignore markers are removed).
The new transport and its enum variant are cfg(test)-gated, so the
daemon build is unaffected.
Bring the immutable-state single-store change onto the Noise XX line.
The shared NodeContext is now the sole store; this merge applies the
next-only adaptations the master-side change couldn't carry:
- Remove node_profile from the Node struct (next-only field) so it lives
solely in NodeContext; migrate its readers (tree/bloom/discovery/
handshake negotiation) onto the node_profile() accessor. The two FMP
negotiation sites hoist node_profile() into a local to avoid borrowing
&self while a connection is mutably borrowed.
- leaf_only sets both is_leaf_only and node_profile via the context swap.
- Preserve the XX handshake/rekey structure (no identity-in-msg1; XX/XK
initiator/responder constructors) while applying the startup_epoch()
accessor migration.
- Tests: route profile selection through a make_test_node_with_profile
helper (profile is immutable, set via Config flags) instead of poking
the removed field.
cargo test --lib 1369/0; clippy -D warnings and release build clean.
Remove the duplicated immutable fields (config, identity, startup_epoch,
started_at, is_leaf_only, max_connections/peers/links) from the Node
struct so the Arc<NodeContext> bundle is the single source of truth.
Previously Node owned these fields and a parallel context copy, kept in
lockstep by rebuild_context() at every mutation site — pure overhead that
existed only because of the duplication.
- Replace rebuild_context() with replace_context(): a clone-edit-swap of
the whole Arc. The per-instance context stays immutable; mutation swaps
the Arc. This is the sole runtime mutation path (constructors, leaf_only,
update_peers).
- Add Copy-returning accessors startup_epoch() and max_connections()/
max_peers()/max_links(); migrate the remaining direct field readers onto
the accessors. node_addr()/npub()/Debug now read identity/is_leaf_only
from the context.
- update_peers reads the pre-update peer set from the live context Arc
before building a fresh Config + context and swapping — preserving the
read-before-write ordering its mutation-window test depends on.
- Remove the test-only set_max_* setters; tests set the limits on Config at
construction instead (new make_node_with_max_peers/links helpers).
- Add a ci-local guard that fails if the Node struct re-declares a bundled
field, so the single-store invariant can't silently regress.
cargo test --lib 1291/0; clippy -D warnings and release build clean.
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, node profile, capability limits).
Source the immutable config and identity reads across the receive hot
path, the XX handshake/session/rekey state machines, and the discovery,
tree, bloom, retry, and lifecycle modules through the context accessors.
Includes the bloom delta/full/NACK/resize and byte-total counters in the
registry. The Node fields and the context are rebuilt in lockstep at every
mutation site.
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, capability limits). Source the
immutable config and identity reads across the receive hot path, the
handshake/session/mmp/encrypted state machines, and the discovery, tree,
bloom, retry, and lifecycle modules through the context accessors rather
than direct field reads. The Node fields and the context are rebuilt in
lockstep at every mutation site.
Raise the in-process backpressure headroom in make_test_node_with_mtu
(request an 8 MiB recv_buf_size on UdpConfig and grow packet_channel from
256 to 8192) to reduce localhost-UDP receive overflow under parallel-CPU
scheduler contention, and mark the large-network convergence tests
#[ignore] so cargo test --lib stays green by default. The ignored tests
remain runnable on demand with --ignored or --test-threads=1.
The two rekey-jitter unit tests added on the IK-rekey branches
construct a HandshakeState via:
HandshakeState::new_initiator(keypair, remote_pubkey)
On next, the XX rewrite changed the initiator constructor to take
only the local keypair:
HandshakeState::new_initiator(keypair)
XX does not learn the responder's static pubkey until msg2, so
the IK-shaped signature does not apply. The merge from master
brought the test bodies verbatim; this drops the second argument
in both call sites so the tests compile against the XX API. The
test semantics (rekey-jitter range and mean checks) are unchanged
and orthogonal to the handshake-pattern choice.
Same shape as 8a6477b adapting the bootstrap-handoff tests to
XX's three-way handshake.
Add a per-session signed jitter offset (uniform [-15, +15] seconds)
to the rekey timer triggers in check_rekey (FMP) and check_session_rekey
(FSP). The configured `node.rekey.after_secs` becomes the nominal
interval rather than a floor; mean is preserved. Desynchronizes
both endpoints in symmetric-start meshes so the dual-initiation
race stops occurring rather than being resolved after the fact by
the smaller-NodeAddr tie-breaker.
Per-session storage means each rekey cutover reconstructs the
session and redraws the jitter naturally — successive cycles get
independent offsets, preventing drift back into sync.
The TUN-side TCP MSS clamp consults `path_mtu_lookup` (FipsAddress-
keyed) when sizing outbound TCP flows. Until now, only the reactive
`MtuExceeded` handler mirrored the bottleneck MTU into that store;
the proactive end-to-end `PathMtuNotification` echoed by the
destination updated only `MmpSessionState.path_mtu`, leaving the TUN
mirror stale.
On stable long-lived paths, the proactive echo can tighten the
session-canonical MTU well before any transit router fires a
`MtuExceeded` for those flows (since all current traffic is already
sized by the tighter session value). New TCP flows opened during
that window get clamped by the discovery-time value rather than the
session-canonical one, leading to PMTU-D loss until the reactive
path eventually fires.
Mirror the post-apply MTU into `path_mtu_lookup` whenever
`apply_notification` returns true, with the same tighter-only
semantics as the reactive mirror — never loosen the clamp. Gated on
the bool return so spurious writes don't happen on rejected
increases or no-op same-value notifications.
Four new unit tests exercise the empty-lookup write, tighten-
existing, keep-tighter-existing, and no-session-no-op paths,
parallel to the existing reactive-mirror test trio.
When a transit forwarder drops an oversized data packet and reports
the bottleneck back via MtuExceeded, the receive-side handler
already updates per-session MmpSessionState::path_mtu (used by PTB
synthesis to feed kernel TCP). It did not, however, update
path_mtu_lookup — the per-destination map the TUN reader/writer
consult at TCP MSS clamp time. So forward-path-asymmetry flows kept
clamping at the discovery reverse-path value (too generous for the
actual forward-path budget) on every subsequent SYN.
Add the missing write at the same point apply_notification runs.
Keep the tighter of existing-or-new — the clamp must never loosen.
Same write-shape as seed_path_mtu_for_link_peer.
Tests:
- Three focused unit tests on handle_mtu_exceeded for the empty,
tighten, and keep-tighter cases.
- Extended test_multihop_pmtud_heterogeneous_mtu to assert the
lookup tightens after the wire-level MtuExceeded propagation,
alongside its existing PathMtuState assertion.
Adds two #[cfg(test)] accessors on Node (path_mtu_lookup_get /
path_mtu_lookup_insert) and bumps handle_mtu_exceeded to
pub(in crate::node) for direct test invocation.
Implement TCP Wrappers-style peer access control using
/etc/fips/peers.allow and /etc/fips/peers.deny files. Evaluation
order: allow overrides deny, default permit when no files exist.
Three enforcement points: outbound connect (before dialing), inbound
handshake (msg1 receipt, after restart/rekey classification), and
outbound handshake completion (msg2, before peer promotion). Files
support npub, hex pubkey, host alias, and ALL wildcard entries with
automatic mtime-based reload.
Adds fipsctl acl show query, 954-line acl module with unit tests,
and a 6-node Docker integration harness (testing/acl-allowlist/)
exercising insider, outsider, and allowed-remote scenarios. CI
matrix entry included.
Closes#50
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
- Delete PROTOCOL_NAME_IK, PROTOCOL_NAME_XK and all IK/XK methods
- Remove Ik/Xk variants from NoisePattern enum
- Rename XX methods to drop xx_ prefix (sole pattern)
- Rename HandshakeMessageType variants from NoiseIKMsg1/NoiseIKMsg2
to Msg1/Msg2, add Msg3 variant (0x03) for XX 3-message flow
- Fix doc comments with correct XX message sizes and descriptions
- Fix stale SessionSetup/SessionAck handshake payload size comments
Replace the 3-message XK handshake with XX for FSP session establishment.
XX requires no prior knowledge of the peer's static key — the responder's
identity is revealed in msg2, the initiator's in msg3.
Key changes:
- session.rs: XX initiator/responder, post-handshake identity verification
using x-only key comparison (parity-independent for npub compatibility),
negotiation payload in msg2/msg3 (FSP version [0,0], features=0)
- Rekey: switched from XK to XX for FSP rekey handshake
- timeout.rs: suppress msg1 resends when target peer is already promoted,
preventing cross-connection session mismatch from duplicate handshakes
- Test template: discovery backoff 3s and handshake timeout 10s for
faster convergence in integration tests
- Integration test timeouts restored to 45s (ping) and 60s (rekey)
Squashed commits:
- Switch FSP handshake from Noise XK to XX
- Fix integration test convergence by reducing discovery backoff
- Fix cross-connection session mismatch from msg1 resend
- Fix FSP identity verification parity mismatch
The PTB source was set to the local FIPS address, which Linux ignores
(loopback PTB security check). Change the source to the original
packet's destination (the remote peer) so the kernel sees it as coming
from a remote router and honors the PMTU update.
Add test coverage for the PTB source address invariant in both the
icmp unit tests and the node integration tests.
Breaking wire format change: DataPacket payloads inside the AEAD envelope
now carry a 4-byte port header [src_port:2 LE][dst_port:2 LE] before the
service payload. The receiver dispatches by destination port.
Port multiplexing:
- send_session_data() takes src_port/dst_port params, prepends port header
- New send_ipv6_packet() compresses IPv6 header and sends on port 256
- Receive path dispatches DataPackets by port: port 256 decompresses IPv6
header from session context and delivers to TUN, unknown ports dropped
- Port constants: FSP_PORT_HEADER_SIZE (4 bytes), FSP_PORT_IPV6_SHIM (256)
IPv6 header compression:
- New ipv6_shim module with compress_ipv6()/decompress_ipv6() pure functions
- Strips src/dst addresses (32 bytes) and payload length (2 bytes) from each
packet, preserving traffic class, flow label, next header, and hop limit
as 6-byte residual fields
- Addresses reconstructed from session context on receive side
- Net savings: 29 bytes per packet (overhead 106 → 77 bytes)
- FIPS_IPV6_OVERHEAD constant (77 bytes), effective_ipv6_mtu() updated
- 16 unit tests for round-trip fidelity, field preservation, error cases
Documentation:
- fips-wire-formats: DataPacket port header, port registry, IPv6 shim
format tables, updated encapsulation walkthrough and overhead budget
- fips-ipv6-adapter: FIPS_IPV6_OVERHEAD (77 bytes), updated MTU numbers,
TUN reader/writer flow with compression steps, impl status
- fips-session-layer: port-based service dispatch section, data transfer
description, impl status
- fips-intro: IPv6 adapter as port 256 service, node architecture updated
- fips-mesh-operation: packet size summary with compressed overhead
- DataPacket doc updated with port header and dispatch model
- session_wire.rs module doc: DataPacket Port Multiplexing section
Two bugs prevented Path MTU Discovery from working across heterogeneous
links (e.g., ethernet→UDP boundary):
1. ICMPv6 Packet Too Big wrote the MTU as u16 into a 32-bit field
(RFC 4443 §3.2), causing the kernel to read an inflated value.
Fixed by changing build_packet_too_big() to use u32 and writing
all 4 bytes.
2. handle_tun_outbound() only checked the local transport MTU, not
the per-destination PathMtuState updated by MtuExceeded signals.
Added a PathMtuState check after session lookup so subsequent
oversized packets generate ICMPv6 PTB on TUN instead of being
forwarded and dropped at the bottleneck hop.
Added integration test exercising the full PMTUD loop across a 3-node
chain with heterogeneous MTUs: oversized packet → forwarding failure →
MtuExceeded signal → PathMtuState update → ICMPv6 PTB on TUN.
Bump rand (0.8→0.10), rtnetlink (0.14→0.20), tun (0.7→0.8),
simple-dns (0.9→0.11), socket2 (0.5→0.6), and criterion (0.5→0.8).
Migrate all rand call sites: thread_rng()→rng(), gen()→random(),
gen_range()→random_range(), RngCore→Rng trait. Work around secp256k1
0.30 requiring rand 0.8 by generating random bytes directly and
constructing SecretKey from slice.
Migrate rtnetlink to builder-based API: LinkSetRequest replaced with
LinkUnspec builder + change(), RouteAddRequest replaced with
RouteMessageBuilder.
Remove bloom benchmark (criterion 0.8 incompatible with old harness
config).
The session-layer handshake now uses the 3-message XK pattern instead
of the 2-message IK pattern, providing stronger initiator identity
hiding. The initiator static key is deferred to msg3 and encrypted
under the es+ee DH chain, so eavesdroppers cannot identify the
initiator from the handshake.
XK pattern: -> e, es (msg1) / <- e, ee + epoch (msg2) / -> s, se + epoch (msg3)
Key changes:
- Add XK handshake methods alongside existing IK methods in noise module
- Add SessionMsg3 wire format and FSP_PHASE_MSG3 (0x03) prefix
- Replace Responding state with AwaitingMsg3 in session state machine
- Rewrite session handlers: handle_session_setup defers identity to msg3,
handle_session_ack processes msg2 and sends msg3, new handle_session_msg3
completes the responder handshake and registers identity
- Link-layer (FMP) continues to use Noise IK unchanged
- Add comprehensive XK unit tests and update all integration tests
Each node generates a random 8-byte startup epoch, encrypted inside
both Noise IK handshake messages (msg1 and msg2). When a peer's msg1
arrives with a different epoch than the stored value, the node tears
down the stale session and processes the msg1 as a new connection,
enabling near-instant restart detection instead of the 30-second
dead timeout.
Wire format impact:
- msg1: 82 -> 106 bytes (added 24-byte encrypted epoch after ss DH)
- msg2: 33 -> 57 bytes (added 24-byte encrypted epoch after se DH)
- Wire msg1: 90 -> 114 bytes, wire msg2: 45 -> 69 bytes
SessionAck previously only carried the responder's coordinates
(src_coords). When the return path diverged from the forward path
(e.g., after tree reconvergence), transit nodes on the return path
lacked the initiator's coordinates and couldn't route the SessionAck
back, causing handshake timeouts.
Add dest_coords (initiator's coordinates) to the SessionAck wire
format, mirroring SessionSetup's design. Transit nodes now cache both
endpoints' coordinates when forwarding a SessionAck, making the return
path self-sufficient regardless of path asymmetry.
Root cause confirmed by churn-20 sim log analysis: the n04-n14
handshake failure was caused by n15 (return-path transit) lacking
n04's coordinates, not by stale tree routes through a downed node.
Add resend logic for SessionSetup/SessionAck messages routed through
the mesh. Stores the encoded payload on SessionEntry for resend in a
fresh SessionDatagram (so routing can adapt to topology changes).
Uses the same config parameters as link-layer retry.
Also fixes a latent bug: Initiating/Responding sessions previously
had no timeout — a stuck handshake would live forever. Now cleaned up
after handshake_timeout_secs (default 30s).
Responder idempotency: duplicate SessionSetup triggers resend of
stored SessionAck instead of being silently dropped. Initiator-side
duplicate SessionAck already handled safely (entry.take_state() sees
Established, puts it back and returns).
Handshake payload cleared on Established transition at both initiator
(handle_session_ack) and responder (handle_encrypted_session_msg).
MMP reports (SenderReport, ReceiverReport, PathMtuNotification) were
resetting the idle timer on both RX and TX paths, preventing sessions
from ever timing out when MMP traffic kept flowing. Changed touch() to
only be called for DataPacket send/receive and session establishment,
so sessions with no application data tear down after the idle timeout
even with active MMP measurement traffic.
Include source and destination coordinates on the first N DataPackets
of each session (default 5, configurable via
node.session.coords_warmup_packets). This warms transit node
coord_caches so multi-hop forwarding and error signal routing work
after SessionSetup cache entries expire.
On CoordsRequired receipt, reset the counter to re-enable coordinate
inclusion for the next N packets, handling mid-session cache expiry
and path changes.
Changes:
- SessionConfig: add coords_warmup_packets field (default 5)
- SessionEntry: add coords_warmup_remaining counter, initialized on
Established transition (both initiator and responder paths)
- send_session_data(): attach coords via DataPacket::with_coords()
while counter > 0, decrement per send
- handle_coords_required(): reset counter for affected session
- 4 new unit tests for counter lifecycle and config default
Identity cache: remove TTL-based expiry (60s TTL broke active sessions
after expiry since handle_tun_outbound checks identity_cache before
session table). Replace with LRU-only eviction bounded by configurable
identity_size (default 10K). Lookup now touches timestamp for LRU
freshness.
Cache merge: unify coord_cache and route_cache into single coordinate
cache. Both stored NodeAddr→TreeCoordinate; the layer distinction was
conceptual, not functional. Discovery-sourced entries now get the same
TTL+refresh treatment as session-sourced entries. Simplifies
find_next_hop() to single cache lookup.
Parent-change flush: clear coord_cache after recompute_coords() in both
parent-switch paths of handle_tree_announce(). Stale coordinates after
tree reconvergence cause dead-end routing that's more expensive than
re-discovery.
Tested: 493 unit tests passed, clippy clean, Docker mesh 20/20,
Docker chain 6/6.
Johnathan Corgan
Alex XieandJohnathan Corgan
KieranandJohnathan Corgan