The anonymous-discovery dial paths drove no control machine: the leg was
born bare in start_handshake and a throwaway machine appeared only at
msg2. Anonymous legs now birth a persistent identity-less machine
alongside the leg (new_outbound widened to Option<PeerIdentity>, backed
by the existing identity-less outbound connection state), and handle_msg2
crystallizes the learned identity onto it before stepping. The msg2
missing-machine arm becomes defensive recovery (debug-assert + insert,
matching the IK line), and the self-connect drop arm disposes the
machine it previously leaked — reachable from identified dials too,
since the handshake completion overwrites the expected identity without
comparing it to the dial-time expectation. The self-connect arm's
index/link/pending-entry leak is pre-existing and stays as-is.
With every leg now born with a machine, the leg-to-machine direction of
the debug coherence check switches on for this line. The add_connection
test seam seeds direction-faithful machines (an outbound-anonymous
seeded connection previously got an inbound-shaped one), and the
hand-rolled dial sites in tests seed dial machines mirroring
production. New tests pin the identity-less birth, the msg2 identity
crystallization through promote, and the self-connect disposal.
The inbound control machine was a msg3-time throwaway: handle_msg3 built
a transient, stepped it once for the decision, and discarded it, with
promote_connection birthing a fresh established() machine. Every
handshake leg now gets one persistent machine for its whole life:
- handle_msg1 births the machine parked in the sent-msg2 phase alongside
the window leg (msg1 crypto and the msg2 build/send stay inline; the
msg2-send-failure cleanup disposes it).
- handle_msg3 steps that persistent machine instead of a transient. The
decision path is unchanged: the msg3 handler never reads machine state
and overwrites the identity-plane fields from the wire outcome before
dispatching, so every arm's decision and actions are byte-identical.
- promote_connection stops rebirthing: the executor feeds the promotion
result back into the machine (the shape the IK line already uses) and
the machine crystallizes to Established in place.
- Every path that tears down an inbound window leg now disposes its
machine through remove_peer_machine: the seven inline msg3 termination
arms, the six executor swap/rekey-responder consumption sites, both
promote-failure arms, the single-peer leaf reject, the losing side of
a cross-connection, and the msg1 send-failure cleanup. The
promote-failure disposal also fixes a pre-existing leak: an outbound
leg whose promotion fails had its connections entry consumed before
the error, so the stale reaper could never reach its dial machine.
Index-free behavior is unchanged at every site. Two test helpers that
hand-roll outbound dials now insert the dial machine production
inserts, and new tests pin the machine's birth phase, its post-promote
crystallized state, and disposal on the failure and rekey-responder
arms.
Carries the debug-build peer-map coherence check. The leg-to-machine
direction is gated OFF on this line (const flipped in the merge): the
inbound msg1-to-msg3 window legs and anonymous-discovery outbound legs
legitimately run machine-less here, so only the machine-to-carrier
leak-tripwire direction asserts until every leg is born with a
persistent machine.
Assert, once per tick in debug builds, that the peer control-machine map
and its carriers stay coherent: every control machine has a live
handshake leg, an active peer, or a pending connect (a machine with none
is a leak the stale reaper can never see), and every handshake leg has a
machine. The second direction sits behind a file-local const so a branch
where handshake-window legs legitimately run machine-less can gate it
off without weakening the leak tripwire.
Teach the add_connection test seam to seed a control machine for the leg
it inserts (derived from the connection's direction and identity), and
remove_connection to dispose it, so the seam-built topologies satisfy
the check. Three unit tests: seam coherence, coherence through a real
promotion, and the orphaned-machine panic.
Carries the dead-code deletion (PeerSlot enum, PeerConnection resend
API) and the module-doc liveness corrections. Hand-resolved the doc
conflicts with next-appropriate wording: the executor module doc names
the live msg3 decision-machine path (PromoteToActive /
SwapToInboundSession / RekeyRespondTrigger) while keeping the still-true
msg1-inline dormancy note; the peer_machines field doc reflects next's
promote-time established() births; the machine module doc drops the
stale unwired claims, including the provisional-and-unwired note on the
msg3 trigger variants which are live.
Several module and field docs still described the per-peer machine as
unwired shadow scaffolding. It has been live for some time: machines are
inserted at dial and inbound msg1, stepped by the handshake handlers and
the rekey-cadence and liveness-reap routers, and the executor's
SwapSendState/CompleteDrain/InvalidateSendState arms are the authoritative
paths (the inline bodies survive only as debug-assert release fallbacks).
Rewrite those docs to the current truth while keeping the still-true
dormancy facts: PeerEvent::Timeout and PeerEvent::Tick are never
dispatched in production, retransmit fires on the machine-armed deadline
while the timeout reaper keys on timer presence with the config
threshold, and the remaining inert executor stubs are SendRekey,
SendLinkMessage, and the connected-UDP arms. Drop the stale
allow(dead_code) on the peer_machines field.
PeerSlot (and its entire impl surface) was referenced only by its own
module tests and the lib.rs re-export; peer storage has always used the
separate connections/peers maps. PeerConnection's resend_count/
next_resend_at_ms/record_resend delegations had no production callers:
the live resend counter is machine-sourced (connection_resend_count reads
the per-peer machine), and the FSP session layer uses SessionEntry's own
methods. ConnectionState::next_resend_at_ms is now test-only (its
remaining callers are the fmp state unit tests) and marked cfg(test).
The PeerSlot unit tests go with the enum; test_resend_count_tracking is
dropped because the delegation target's schedule arithmetic is already
covered by the fmp resend_bookkeeping test.
Fold in a next-only wording fix for the OpenWrt 802.11s mesh backhaul:
next's FMP link handshake is Noise XX, not the Noise IK on master, so the
fips-mesh-setup header and the how-to now name it pattern-agnostically
("the Noise handshake"), matching the shipped fips.yaml comment and
staying accurate on this branch.
Router-to-router radio backhaul over an open 802.11s mesh interface,
with FIPS providing all encryption (Noise IK), authentication, and
routing on top of bare L2 neighbor links. The mesh runs OPEN with
mesh_fwding 0 — SAE would duplicate the Noise layer and force ath10k
raw mode, and FIPS is the routing layer — so the Noise handshake is the
real auth/encryption boundary and FIPS's spanning tree does the routing.
fips-mesh-setup: an opt-in UCI helper that creates a per-radio
mesh-point interface (radio0 -> fips-mesh0, radio1 -> fips-mesh1;
trailing-digit derivation with a free-index fallback and a collision
guard). Radio setup stays opt-in — a package must not commandeer radios
on install. 'remove' takes an optional radio and otherwise removes all
instances. Dual-band routers get one instance per radio; FIPS treats the
two backhaul paths as failover, not multipath: it keeps one active link
per peer (cross-connection resolution picks a single winner), and the
second band stands by, re-establishing the peer after keepalive timeout —
traffic never uses both bands at once.
fips.yaml ships the mesh0/mesh1 Ethernet-transport entries commented
out, so a stock install that never creates fips-mesh* logs no per-boot
"interface missing" bind warning. fips-mesh-setup uncomments the matching
meshN block when it creates the interface and re-comments it on remove,
so the flash-and-drop-in flow needs no manual config edit. The file is
rewritten 0600-first (it may hold an inline nsec) via an atomic replace.
Two field-found silent non-peering causes are surfaced by the helper and
the guide:
- Same channel: mesh points only peer on a shared channel, and 'auto'
lets each radio pick its own. The helper prints the radio's
band/channel and warns loudly on 'auto' with the exact uci command to
pin one; the how-to gains an ordered no-peers triage (channel mismatch,
on-air scan check, DFS CAC wait, regdomain).
- STA channel capture: a client (sta) interface drags the whole radio to
its upstream AP's channel, so a mesh pinned elsewhere never joins and
does not recover until the STA disconnects. The helper warns when the
target radio carries a STA; the guide documents the incompatibility of a
roaming uplink with a fixed-channel mesh on the same radio.
Both the create and remove paths run 'wifi reload', which briefly drops
every client AP on all radios; the how-to sets that expectation.
Regression test: the shipped OpenWrt fips.yaml must parse via the real
Config deserializer in both states — as shipped (mesh inactive) and after
the uncomment the helper performs.
Packaging: the helper is installed across the ipk/apk/buildroot paths
(three synced copies), with the CI structural checks and shellcheck
targets extended to cover it. Full guide in
docs/how-to/set-up-80211s-mesh-backhaul.md.
Phase 4 of the rekey suite waited for only one more initiator cutover
(guaranteeing three) while Phase 6 asserts at least four. The fourth
cutover then had to land in the brief window between the wait returning
and the Phase 6 log snapshot, so host load could push it past the window
and fail the run even though every cutover completed correctly. Wait for
two more cutovers, the full second rekey cycle, matching the Phase 6
threshold, so the asserted count is guaranteed before it is checked.
Re-express the rekey-msg2 and cross-connection observation feeds onto
the XX cores. The rekey-msg2 observation lands inside the block that
installs the pending session (where XX also generates msg3), so it fires
only when the install actually ran. The cross-connection observation
sits at the outbound resolution arms and targets the promoted peer's
machine; the inbound session-swap effects already own their executor
path here and are untouched. Crypto effect bodies are byte-unchanged.
Feed the per-peer machine two observation events so its control/rekey
shadow state stays coherent with the inline crypto-session installs,
without moving those installs. The crypto effect bodies are unchanged.
On rekey-msg2 completion, feed the machine the responder index on the
success path only (after the pending session is installed); the abandon
path feeds nothing.
On cross-connection resolution, feed the machine the swap outcome: on a
swap it records the new outbound and responder indices to mirror the
session replacement, and on a keep it does nothing. The observation
targets the promoted peer's machine, not the outbound leg whose machine
is removed as the block is entered.
Both observation handlers emit no action; nothing consumes the updated
shadow state yet.
Move the two Noise-XX inbound establish effects — the cross-connection
session swap and the rekey-respond session install — out of the inline
handle_msg3 arms and into their per-peer machine executor actions
(SwapToInboundSession, RekeyRespondTrigger). The inbound establish
decision now falls through to the shared machine-drive arm, matching the
promote/restart arms.
Carry the session index on the InboundMsg3 event and seed it into the
transient machine before dispatch. The swap actions build their index
from the machine's connection state, which is unset on a fresh inbound
transient; without the seed the actions would emit empty and the swap
would silently no-op while leaking the allocated index.
Give the inbound reject and msg2-resend machine arms their terminal
Failed state and index free, matching the inline arms that own those
paths, so a later timer cannot free the index or report loss for a peer
whose handshake has already resolved.
Preserve exactly: the three distinct index frees and their paired index
and pending-outbound map removals; the link-teardown asymmetry (the
cross-connection path drops the address mapping, the rekey-respond path
keeps it); the session take-then-install ordering; and every reject kind
on the error sub-paths.
Fold the handshake timer lifecycle (msg1 retransmit + timeout/stale
reap) through the per-peer machine, re-expressed onto the XX handshake
surface.
- Route peer-machine removal through remove_peer_machine so each link's
timer store is dropped together with its machine (choke-point).
- Home the msg1-retransmit decision on the machine-armed retransmit
timer; advance the resend counter only on send success and relocate
the operator-visible resend count onto the machine.
- Reap outbound handshake timeouts via a presence-scan over the machine
HandshakeTimeout timer, reading the threshold from config each tick so
the reap stays neutral for any handshake_timeout_secs.
Cancel both dial-armed handshake timers on outbound promote so a
promoted machine carries no stale timer entry. Preserve the guard that
suppresses a msg1 resend at a peer already promoted via the inbound
cross-connection path.
The XX inbound HandshakeTimeout presence-scan is neutral only because
the inbound establish path sends msg2 inline and never dispatches the
inbound machine event, so no inbound leg ever populates a
HandshakeTimeout timer. A future change that drives inbound establish
through the machine must re-verify the reap equivalence for inbound
legs.
Move the outbound handshake-timeout reap off the unconditional check_timeouts
scan onto the machine-armed HandshakeTimeout timer. A new drive_handshake_timeouts
(run before the retransmit drive, so a timed-out leg is reaped rather than resent
on the same tick) reaps the outbound legs that carry a HandshakeTimeout timer and
have idle-timed-out this tick.
The timer's presence selects the leg (only outbound legs arm one; IK inbound arms
none); the reap threshold is the shell is_timed_out(now, config) predicate, not
the timer's stored deadline. The machine arms the timer from a hardcoded constant
at dial, which is not authoritative for an operator-tuned handshake_timeout_secs,
so reading the threshold from config each tick keeps the reap neutral for any
timeout value and on the last_activity clock exactly as before.
check_timeouts keeps reaping everything else: failed connections (all of them,
promptly) and the idle-timeout of legs without a machine timer (inbound legs, and
any machine-less connection). Total coverage is unchanged. check_timeouts runs
before the timer drive, so a failed outbound leg is reaped there first, its timers
dropped, and the timeout drive never double-fires on it.
Split drive_peer_timers into the timeout drive and the (byte-identical) retransmit
drive. The dormant machine timeout handler is not dispatched, so the session index
is freed once, by cleanup_stale_connection. A regression test drives a timed-out
outbound leg to reap; the existing check_timeouts tests continue to anchor the
residual sweep. Test count 1631 to 1632.
Move the outbound msg1 resend off the unconditional tick function onto the
machine-armed retransmit timer. The per-peer machine already arms a
HandshakeRetransmit deadline at dial; a new drive_peer_timers fires the due ones
(kind-filtered to the retransmit timer) and homes the resend counter on the
machine, where the operator-visible count now reads from.
The resend decision reads the same operator config as before (interval, backoff,
max), the wire bytes and transport target still come from the shell connection,
and the pure core computes the backoff schedule. As with the deleted
resend_pending_handshakes, the count and reschedule advance only on a successful
send: a failed send neither advances the count nor marks the connection failed,
it just retries on the next tick. The handshake-timeout timer stays on the legacy
check_timeouts path, and the rekey/liveness timers keep their own shell drivers,
so drive_peer_timers deliberately fires only the retransmit kind.
The show_connections resend count is relocated to read from the machine (the
counter's new home) via connection_resend_count; machine-less and inbound
connections report 0, matching what the shell connection reported before. Delete
resend_pending_handshakes and resend_candidates (the latter also from the
LifecycleView trait, its only user). The resend unit test is re-expressed against
the machine + timer path, covering the due check and the record-on-success
semantics.
Known limitation: the first resend interval is armed from the machine's hardcoded
1000ms constant, which equals the config default. Under an operator override of
handshake_resend_interval_ms the first resend diverges from the pre-change
dial+interval; subsequent resends and the cap remain config-driven. Neutralizing
the first-resend override needs the interval threaded into the sync core (the
shell arm would be clobbered by the machine's own timer arming at dial), deferred
to when the connection and machine entities merge. Test count unchanged at 1631.
The control machine already emits SetTimer/CancelTimer actions when it arms the
handshake retransmit and timeout deadlines, but their executor arms were a
single no-op stub and the machine's Timeout event was never dispatched -- the
real work still runs on the legacy tick (check_timeouts,
resend_pending_handshakes).
Add the storage the time-as-input driver will read: a peer_timers map keyed by
LinkId then TimerKind, holding each armed timer's absolute deadline. The SetTimer
arm now inserts (overwrite = reschedule) and CancelTimer removes; the outbound
msg2 promote cancels the two dial-armed handshake timers, since the machine
survives promotion and the entries would otherwise linger in the store.
This store is a shadow: it is written and cleared but no driver reads it yet, so
behavior is unchanged. The legacy tick stays authoritative until the driver that
feeds Timeout and deletes the overlapping tick paths lands.
Route every machine removal through a new remove_peer_machine(link) choke-point
that drops the timer store alongside the machine, replacing the twelve direct
peer_machines.remove sites so no armed timer outlives its machine. TimerKind
gains Hash (to key the store) and Ord (for deterministic driver collection); it
is internal and never serialized. Test count unchanged at 1631.
Bring the outbound-handshake-through-the-machine series onto the next-branch
Noise-XX cores. Re-expressed rather than transcribed: next has no outbound
decision core, so the promote drives PeerEvent::OutboundMsg2 (not the IK Msg2
snapshot event) and cross-connection resolution stays inline. Hybrid provenance
-- identified outbound legs persist the control machine at dial and drive
through it; anonymous-discovery legs (identity unknown until XX msg2) retain the
inline transient-at-msg2 path. ReportLost{kind} routing, the connection-oriented
dial via the machine (OpenTransport + TransportConnected), and the prepare/send
msg1 split all carried over.
Behavior-neutral on next: the XX msg2 body (Noise completion, FMP, ACL, msg3) is
untouched; anonymous legs are unchanged; the dial-persisted machine promotes
with our_index unset.
Route the connection-oriented (TCP/Tor) outbound path through the peer state
machine, matching how the connectionless path already works.
initiate_connection's oriented branch now drives PeerEvent::Dial with
connection_oriented=true; the machine parks in Connecting and emits
OpenTransport, whose executor arm performs the non-blocking transport.connect
and pushes the PendingConnect. When the connect resolves, poll_pending_connects
prepares msg1 in the shell and then drives PeerEvent::TransportConnected, which
sends msg1 via the machine's SendHandshake arm.
The msg1 prepare (index allocation, Noise leaf, wire arming) MUST run in the
shell before the TransportConnected drive: send_stored_msg1 only transmits an
already-armed wire, so a drive-only path would silently send nothing. The
machine's our_index stays unset; the connect-failure path keeps its direct
handshake-timeout handling (TransportFailed stays dormant). The now-unused
Node::start_handshake helper is removed.
Behavior-neutral: same transport.connect, same PendingConnect, same msg1 send
and failure teardown as the removed inline path -- only the driver changes from
inline shell code to the state machine.
Move the peer_machines insert for an outbound leg out of the tail of
prepare_outbound_msg1 to a single shared site in initiate_connection, before
the connection-oriented / connectionless fork. This lets the connection-oriented
path find the machine at dial time (so it can be driven through the connect
handshake) without prepare_outbound_msg1 -- which for that path runs after the
connect completes -- clobbering an in-progress machine back to Discovered.
Because the machine now exists before the fallible dial steps, add
peer_machines.remove to every failure path in the widened dial window: the
index-allocation and Noise-leaf failures in prepare_outbound_msg1, the oriented
transport.connect failure, and both poll_pending_connects teardown arms
(handshake-start failure and async connect failure). remove_link does not touch
peer_machines, so these explicit removes are required.
Behavior-neutral: the connectionless path still drives the machine to
Handshaking and sends msg1 identically; the machine's our_index stays unset
(no spurious UnregisterDecryptSession on a later inbound restart); a failed dial
leaves peer_machines empty for that link exactly as before; and no
connection-oriented machine drive is wired here.
The connection-oriented (TCP/Tor) outbound connect->handshake path had no
`cargo test --lib` coverage; it was exercised only by the opt-in Tor
integration suites, and the TCP node tests bypass it via a manual
connectionless handshake helper. Add three unit tests:
- a machine-level test driving Dial{connection_oriented:true} -> Connecting
(emitting only OpenTransport, no msg1) -> TransportConnected ->
Handshaking{SentMsg1}, asserting the exact OpenTransport and
SendHandshake+SetTimer action vectors that start_outbound_handshake emits
(its oriented reach via on_transport_connected was previously untested; the
connectionless reach via on_dial was already covered);
- two node-level tests over a real loopback TcpTransport: a successful connect
that reaches start_handshake (observed via a pending_outbound entry), and a
connect to a closed port that routes through the failure arm (link torn down,
no msg1 dispatched).
Tests only; no production change.
Split the outbound handshake setup out of start_handshake into
prepare_outbound_msg1 (allocate the index, run the Noise leaf, frame and arm
msg1 -- the fallible steps, returning an error the caller propagates) and
send_stored_msg1 (transmit the armed wire). A connectionless dial now runs
prepare in the shell, then drives the control machine, whose SendHandshake
action sends the wire via the executor. Connection-oriented dials keep calling
start_handshake, now prepare followed by the send inline.
The dial event gains a connection_oriented flag so the machine's on_dial sends
msg1 immediately for connectionless transports (no connect step) instead of
opening a transport first. Behavior is unchanged: the same index, Noise leaf,
wire bytes, maps, and send-error handling as before, with index-allocation and
Noise failures still propagated synchronously before the send. A regression
test covers that a promote from the post-dial handshaking state is identical to
the former discovered-state promote.
Create and persist the per-peer control machine when an outbound handshake is
dialed, keyed by its link, instead of building a transient at msg2. The msg2
completion path now looks up that persisted machine to drive the promote,
falling back to a transient only if none is present (e.g. a direct-seeded test).
The machine parks in the Discovered state until promotion and is inert to the
liveness reap and rekey cadence while unpromoted, since it is absent from the
peers map. It is removed on every path that ends the outbound leg without
promoting -- the stale-connection reaper, the msg2 authorization-failure arm,
and the cross-connection resolution block -- mirroring the connection's own
lifetime so no dangling machine survives.
Its session index is deliberately left unset on the machine (the shell owns the
index on its connection), so a later inbound restart does not emit a spurious
decrypt-session unregister. A regression test covers that invariant.
Add a LostKind discriminator to the ReportLost action so the executor can
route an un-promoted handshake failure to the connected-guarded reconnect
reflex (note_handshake_timeout) and an established peer's link-death to the
unconditional one (note_link_dead), instead of collapsing every loss to
note_link_dead.
The two loss producers dispatched today, the liveness reap and the
inbound-restart-then-promote arm, both keep the link-dead routing, so
behavior is unchanged. The handshake-timeout and dial-failure producers are
tagged accordingly but stay dormant until their events are dispatched.
On two inbound msg3 outcomes the msg1-allocated inbound session index
was dropped without returning it to the allocator: losing the dual-rekey
tie-break (we keep our in-progress rekey and drop their msg3), and a
duplicate handshake that resolves to a msg2 resend. Under repeated hits
this slowly leaks index space. Free the index on both arms, matching the
sibling reject arms.
Add a test-only helper to backdate a peer's session-established instant,
and two regression tests that drive a real second handshake into each
arm and assert the index is returned to the allocator (both fail if the
free is removed).
Remove the now-stale `#[allow(dead_code)]` on `advance_peer_machine`
and `execute_peer_actions`; both are live (called from the link-dead
reap, rekey cadence routing, and the handshake establish sites).
In `route_rekey_cadence`, reuse the ambient context timestamp for the
machine step instead of taking a second clock sample; the extra
sample's only consumer was the currently-inert cutover drain timer, so
the change is behavior-neutral.
Forward-merge the action-contract round-trip test. This line's action type
carries two extra establish-trigger variants (SwapToInboundSession,
RekeyRespondTrigger), so the merge resolution extends the test's sample set
and its wildcard-free exhaustiveness match to cover all of them — the test
proves every variant, including the two XX-only triggers, round-trips
unchanged through the async channel.
Prove the per-peer machine's action type is a runtime-agnostic message
contract: it is Send + Sync + 'static, and every variant round-trips
unchanged through a single-threaded async channel (a current-thread runtime,
sender and receiver on one thread). A wildcard-free match over the variants
makes any future action a compile error here until it is added to the sample,
so a variant that embedded a runtime handle could not slip through unproven.
Forward-merge the promote-failure warning target pin. The peer-action
executor diverges between the two lines (IK vs XX establish arms), so the
pin was applied to this line's own two promote-Err warns during the merge
resolution: outbound "Failed to promote connection" and inbound "Failed to
promote inbound connection" now emit under fips::node::handlers::handshake,
matching the sibling max-peers reject log already pinned to that target.
The two promote-failure warn! sites in the peer-action executor emit under
this module's own tracing target rather than the handshake target the rest of
the establish-path logging uses. Pin both (outbound "Failed to promote
connection" and inbound "Failed to promote inbound connection") to
fips::node::handlers::handshake, matching the sibling max-peers reject log and
keeping all establish-path diagnostics grouped under one target regardless of
which module physically emits them.
Comments across the per-peer machine, executor, lifecycle supervisor,
and peering reconciler carried internal rollout labels and design-note
section references. Rewrite them to describe the code's behavior
directly. Comment-text only; no code changes.
Route the outbound side's msg2-completion promote through the per-peer
state machine and its action executor, mirroring how the net-new inbound
promote is driven. A transient outbound machine emits the promote action;
the executor performs the promotion and, on success only, clears the
pending-outbound entry (threaded in through the action context). The
outbound cross-connection path stays inline as before.
The debug log for rejecting an over-cap inbound connection is emitted from
the peer-action executor, under that module's tracing target. Nodes that
raise only the handshake module to debug (leaving other modules at info) no
longer surfaced the rejection, so cap-enforcement events were invisible in
their logs. Re-pin the log to the handshake target, matching the adjacent
promoting-inbound log, so the rejection stays visible under a
handshake=debug filter regardless of the executor module's own level.
Route XX inbound establishment through the per-peer machine: handle_msg3 hands
the establish decision to a transient machine and drives the net-new promote
(Promote + RestartThenPromote) through the executor's PromoteToActive action,
which transcribes the shared promote block (promote_connection plus the
msg2/tree-announce/bloom follow-ups). The four non-promote decisions (Reject,
ResendMsg2, CrossConnect, RekeyRespond) stay inline byte-for-byte; the two
session-swap arms are deferred to a later non-neutral step.
The decrypt-worker register stays inside promote_connection (not relocated),
no promotion-resolved feedback is fed back, and the transient machine is never
inserted into peer_machines (the persistent machine is created once inside
promote_connection). Behavior matches the inline path exactly, and operator
diagnostics are preserved: the relocated promoting-inbound log keeps its
handshake tracing target, and the epoch-mismatch restart breadcrumb is
re-emitted.
Route the initiator rekey cutover and drain-completion through the
per-peer machine and executor instead of running them inline in
check_rekey. The batch poll_rekey and its per-peer snapshots stay
shell-side and unchanged, so the cross-peer phase ordering that governs
the shared index-allocator sequence is preserved; the machine only
consumes each decided action in that order.
The routed cutover and drain reproduce the inline bodies exactly (the
executor's swap-send-state and complete-drain arms), including the
info-level cutover log and the decrypt-worker re-register. Initiating a
rekey stays inline; the machine is fed a rekey-initiated observation
afterward to keep its control state coherent. A guard falls back to the
inline bodies if a machine is somehow absent.
Route the liveness reap through the per-peer machine: on a link-dead
timeout, the mmp reap advances the peer's machine with a link-dead event
and executes the resulting actions, instead of calling remove_active_peer
and note_link_dead inline.
The effect is unchanged — the machine emits InvalidateSendState then
ReportLost, which the executor maps back to remove_active_peer(peer)
then note_link_dead(peer, now), the same two calls in the same order.
The reap log stays shell-side so it keeps the handlers::mmp target, and
a guard falls back to the inline path if a machine is somehow absent.
Insert a per-peer machine into peer_machines at each promote site and
remove it on teardown, so every established peer has exactly one machine
entry keyed by its link id. Nothing drives the machine yet — the entry
is inert — but this correspondence is what the rekey and liveness-reap
folds depend on.
Inserts pair with the two promote_connection arms (normal promote and
cross-connection-won); removes pair with remove_active_peer and the
cross-connection loser teardown. A new PeerMachine::established
constructor parks the machine in the post-handshake Established state.
Behavior is unchanged: nothing reads the map on a live path.
Add the executor that consumes the per-peer machine's actions, plus the
peer_machines map that homes the machines on the node. Both are shadow:
nothing populates the map or invokes the executor yet.
The rekey-cutover, drain, liveness-reap, and index-plane action bodies
are ported to reproduce the next-line handlers exactly (the cutover log
matches check_rekey's info-level line, kept under the handlers::rekey
target so operator and test log filters still see it). The establishment
actions (promote, cross-connect swap, rekey-respond) are deferred stubs
until the inbound establish path is wired.
Re-derive the per-peer FMP lifecycle state machine against the next-line
XX establishment cores, as an unwired module. It ports the master-line
machine's structure but adapts it to the XX handshake:
- inbound establishment splits into a msg1 event (allocate the index,
send msg2, defer) and a msg3 classify event (identity crystallizes,
establish_inbound runs), matching XX's identity-at-msg3 timing;
- the inbound decision handles all six InboundDecision variants including
the XX-only CrossConnect;
- the outbound establish_outbound path and the two-phase authorize are
dropped (no analog on the XX line);
- session-carrying establish effects (cross-connect swap, rekey-respond
pending session) are represented as plain-data trigger actions; the
session surgery is done shell-side when the machine is wired.
Nothing drives the machine yet; it is dead code pending the executor and
handler wiring. Unit tests cover the XX dispatch: inbound establish to
Established, dual-init tie-break, cross-connect at msg3, rekey cutover
key-consistency, and liveness reporting.
Forward-merge the per-peer FMP decomposition from the master line onto
the next line. The establishment and rekey machine-drive on refactor-node
is built on the IK establishment cores (establish_outbound, identity
learned at msg1) that the next line replaced with the XX msg3 model, so
it cannot be carried textually; it is re-derived against next's XX cores
in follow-on commits on this branch.
This merge commit carries only the two changes that are neutral and
next-compatible as-is:
- OwnedFd hygiene for the connected-UDP socket (RawFd -> OwnedFd, drop the
hand-rolled unsafe Drop; OwnedFd's own Drop closes the descriptor).
- The two-tier send-state boundary: the send-critical subset of ActivePeer
is regrouped into a PeerSendState struct, with accessors kept stable so
no caller changes. next's XX establishment, rekey, and liveness-reap
behavior is unchanged (verified: exact field partition, identical
accessor signatures and semantics, identical init values, 3-arg mmp
construction).
The per-peer machine, its action executor, and the peer_machines map are
not carried here; they are re-derived against next's XX establishment
surface in subsequent commits on this branch.
Route each link-dead peer that the tick sweep's plan_heartbeats decides
to reap through the per-peer machine and executor, replacing the inline
reap body in check_link_heartbeats. The batch decision, the liveness
snapshots (read from the hot-path-written receive clock), and the
heartbeat-send arm stay shell-side and byte-unchanged; the machine only
consumes the decided LinkDeadSuspected, tearing the peer down via
remove_active_peer and reporting the loss to the reconciler exactly as
before, on the same tick with the same wall-clock timestamp. The reap
log stays shell-side.
The machine's link-dead handler no longer emits a decrypt-session
unregister keyed by its shadow index (the full peer teardown already
unregisters the real index; the shadow could have drifted to a reused
index), and its guard now covers the Established state a freshly
promoted peer sits in.
Handshake-timeout, retransmit, and stale-connection cleanup stay inline:
they act on pre-promotion legs that have no machine, and the loss reflex
they use differs from the link-dead one.
Route each Cutover and Drain that the shell-side batch poll_rekey decides
through the per-peer machine and the executor, replacing the inline
effect bodies in check_rekey. The batch decision and its per-peer
snapshots stay shell-side and byte-unchanged: poll_rekey phase-groups all
cutovers, then all drains, then all initiations across the peer set, and
that ordering governs the shared index allocator's free-then-allocate
sequence that appears on the wire, so the machine only consumes the
already-decided actions (a new RekeyConsume event) without re-deciding.
InitiateRekey stays inline (its Noise msg1 build is a shell-side leaf)
with a RekeyInitiated observation feeding the machine so its control
state stays coherent for the next tick's cutover.
The cutover and drain logs, which relocated into the executor in the
prior commit, are pinned back to the fips::node::handlers::rekey tracing
target so they stay visible under the operator's module log filter.
Also clears the machine's shadow draining_index on drain so a later
cross-connection resolution cannot double-free the already-freed index.
Prepare the establish executor to drive FMP rekey by activating the
SwapSendState action (initiator K-bit cutover via cutover_to_new_session,
with the gated decrypt-worker re-registration) and adding a CompleteDrain
action (erase the drained previous session: free its index, drop its
peers_by_index entry, unregister its decrypt session). Both reproduce the
current inline rekey.rs cutover and drain bodies. The machine's drain
mapping now emits CompleteDrain, using the real drained index rather than
a shadow copy.
Unwired: nothing drives the machine's rekey path yet (live rekey still
runs inline), so these arms are unreachable and the change is
behavior-neutral. The cadence fold that routes cutover and drain through
the machine follows.
Move register_decrypt_worker_session out of promote_connection into the
executor's PromoteToActive handler, gated on a promoted or
cross-connection-won result. Every live promote now flows through that
one executor path, so registration still fires exactly once at the same
synchronous point; the direct test callers of promote_connection spawn
no worker pool, so the call was already a no-op for them.
Add the cross-connection loser-link teardown (close the losing
transport, remove its link, re-point addr_to_link at the winner) to the
executor as a guarded follow-up. It is unreachable on the current driven
establish paths, which only promote net-new peers, and asserts so, but
keeps the executor complete for when that case is driven.
Remove the now-dead drive_promote_to_active and ConnAction::PromoteToActive.
Cut the net-new outbound handshake completion (a received msg2 that
promotes a fresh outbound leg to a new peer) over to the per-peer
control machine, mirroring the inbound cutover. handle_msg2 still runs
the msg2 prologue, the ACL check, and the cross-connection swap/keep
arms inline; for the net-new promote it now builds a transient machine,
steps it, and drives promote_connection through the executor. The
session index was already allocated at dial, so there is no two-phase
authorize here.
The wire, index sequence, and peer registry state after promote are
byte-neutral. To keep the promote-failure path neutral, the executor's
cleanup now distinguishes inbound from outbound: an outbound promote
failure records the reject only, matching the prior handler, rather
than the inbound path's link and index teardown.
Cross-connection swap/keep, rekey-msg2, and the dial path stay inline;
the loser-link surgery for the currently unreachable driven
cross-connection case lands with the register relocation next.
Cut the restart handshake (an inbound msg1 from a peer that reconnected
with a new epoch) over to the per-peer control machine, completing the
inbound establish cutover. The old peer is torn down and the fresh leg
promotes through the same two-step authorize-then-allocate path as the
net-new case: the machine's first step emits the old-peer teardown
(invalidate send-state, report loss), the shell interposes the ACL
check, and the second step allocates the new index and sends msg2. The
old index is freed before the new one is allocated and the msg2 wire
bytes are unchanged, so the sequence stays byte-neutral.
With restart driven through the machine, the shared inline establish
tail that only the restart arm reached is deleted.
Also bounds the peer_machines map (remove_active_peer now drops the
peer's machine entry) and restores the msg2-send, promote, and
index-allocation failure warnings the cutover had dropped.
Cut the net-new inbound handshake (a fresh msg1 that promotes to a new
peer, plus the at-capacity reject) over from the inline handle_msg1 logic
to the per-peer control machine. handle_msg1 still classifies via
establish_inbound and still owns the Noise wire step, the late ACL check,
and the promote_connection registry surgery; for the net-new path it now
builds the machine, steps it, and executes the returned actions.
Authorization is interposed between two machine steps so the session
index is allocated only after the ACL check passes: a rejected or
unauthorized msg1 consumes no index, matching the prior order exactly.
The msg2 wire bytes, the index-allocation sequence, and the reject
metrics are all byte-neutral. Restart, resend, rekey-respond, and the
other reject arms stay inline unchanged; they move to the machine once
outbound establish is cut over and every promoted peer has a machine.
Also fills in the executor's send-failure and promote-failure cleanup so
a mid-establish error tears the leg down and frees its index exactly as
before.
Johnathan Corgan
ArjenandJohnathan Corgan