Both branches carry the same correction, authored separately because the
sans-IO relocation moved the logic into a core that returns an outcome rather
than acting inline. Git resolved the file move as a rename and offered to merge
the two versions textually, which would have been wrong in both files.
Resolved by keeping master's re-expression wholesale for the handler and its
tests, and taking the changelog entry from maint. Verified rather than assumed:
the resulting tree is byte identical to the reviewed and CI-green master commit
across every code path, and the merge contributes nothing but the fifteen line
changelog entry. Since the code is unchanged from a tree that already passed the
full local suite, that result still stands and no re-run is owed.
Nothing was lost from maint's side. Its tests are written against a file layout
that does not exist here, and master carries equivalent coverage written against
its own shape, including the multi-node chain and the cache warming cases that
the maint version does not have.
The same correction as on the maintenance line, re-expressed rather than merged,
because the sans-IO relocation moved this logic into a synchronous core that
returns an outcome instead of acting inline.
Delivery to the addressed node is no longer gated on the hop limit, and the
decrement happens before the drop decision, so a datagram that would leave with
zero is not sent. The reachable radius does not change: the old behavior refused
to deliver at zero but was willing to send at zero, and the new behavior is the
mirror of that, so the two cancel.
The shell above the core needed changing too, and that was the real work. It
carried two hop-limit predicates of its own, both justified by comments citing the
ordering that existed before the refactor. The first gated coordinate cache
warming, which this fix requires to be unconditional; left alone it would have
suppressed warming twice over, once for a datagram addressed here and now
delivered, and once for a transit datagram dropped for hop limit whose plaintext
coordinates are still perfectly usable. The second gated next hop resolution,
which carries a cache touch side effect, and now mirrors the core's
would-leave-zero rule instead of the old one. Neither divergence would have
surfaced as a test failure.
That last point is worth stating plainly, because it nearly went the other way.
Restoring the warming gate on top of the corrected core passes the entire suite
without a single failure. The tests could not see the seam between the core rule
and the shell's copy of it. So this adds coverage that can: a three node chain
pinning the decrement across a real hop, which is the only thing exercising the
shell predicates composed with the core, and two warming tests at a zero hop limit
covering both the delivered and the dropped case. Each was checked by putting the
specific defect it targets back and confirming the test fails.
One existing test that claimed to cover hop limit behavior asserted nothing
whatsoever. It asserts now, but those assertions do not discriminate this change,
since the case it exercises behaves identically under both rules. They guard
against the test going vacuous again, not against this defect, and the comment
says so.
The forwarding path tested the hop limit before testing whether the datagram was
addressed to this node, and decremented only on the forwarding branch. Two
consequences followed. A datagram addressed here that arrived with a zero hop
limit was dropped rather than delivered. And a forwarder receiving a transit
datagram with one hop left transmitted it with zero for the next hop to discard,
wasting a transmission on every expiring datagram.
Delivery to the addressed node is no longer gated on the hop limit, and the
decrement now happens before the drop decision, so a datagram that would leave
with zero is not sent. Saturating subtraction folds the two arrivals that cannot
be transmitted, already exhausted and last hop, into a single test.
The reachable radius does not change, which is worth stating because it is easy
to assume otherwise. The old behavior refused to deliver at zero but was willing
to send at zero, and the new behavior is the mirror of that, so the two cancel: a
path of h links still delivers for any source value of h or more. What this
actually buys is the wasted transmission, the exhaustion counter charging at the
node that makes the decision rather than the one after it, conformance with the
specified semantics, and one real gain during a rolling upgrade, where an
unupgraded forwarder feeding an upgraded destination delivers one hop further
than either version does on its own.
Coordinate cache warming moved ahead of both decisions, so a transit datagram
later dropped for hop limit still contributes its plaintext coordinates. The only
arrivals this newly warms from are those with a zero hop limit, and every insert
they can make is already achievable with a value of one, so it grants nothing
that was not already available and adds no memory growth the cap does not bound.
Four tests added: delivery at zero to this node, the transit drop at one, the
transit pass at two, and the one per hop decrement across a three node chain,
which brackets the emitted value from both sides since it is only observable
through what the next hop does. One existing test was renamed because its
destination was this node, so it never exercised the transit path its name
claimed, and it asserted nothing at all.
maint carries only its 0.4.2-dev version bump; master keeps its own
0.5.0-dev development version. Recorded as a linkage merge so later
forward-merges of real fixes land cleanly.
Bump the version to 0.4.2-dev and update the status badge and prose. The
0.4.x patch line continues; v0.4.1 remains the shipped release it points at.
Follows the shape of the v0.4.0 rollover commit. No maint reset was needed
this time: v0.4.1 was cut from maint, so the tag is already on this line and
it simply continues.
Carries the v0.4.1 content up the one-way branch flow. The release itself
belongs to maint, so the parts of this merge that identify a version are
resolved in master's favor and the parts that describe the project's state
are taken from maint.
Kept master's: Cargo.toml and Cargo.lock at 0.5.0-dev, the README status
badge, and the "FIPS is at v0.5.0-dev on the master branch" line. A patch
release consumes no minor version and master's development line is
unaffected by it.
Took from maint: the bloom antipoison FPR default change and the
duplicate-definition fix it rests on, the two docs that describe them, the
new v0.4.1 release notes, the correction to the v0.4.0 release date, and the
[0.4.1] changelog section, which slots below master's own [Unreleased] and
above [0.4.0]. The root RELEASE-NOTES.md mirror moves to v0.4.1 because it
tracks the latest shipped release, which also clears the stale provisional
date it had been carrying.
One line needed splitting rather than choosing: the README said "v0.4.0 has
shipped" inside the paragraph that identifies master as 0.5.0-dev. The
version identity stays master's and the shipped-release pointer moves to
v0.4.1.
Quartet green on the result: 1645 tests passed, clippy clean with -D warnings.
Drop the -dev suffix from the package version and refresh the lockfile.
The version is single-sourced from CARGO_PKG_VERSION, so build.rs and
src/version.rs need no change.
Also fix the archived v0.4.0 release notes, which still read
"2026-06-21 (provisional)" while the changelog records the actual
release date of 2026-06-27. The date was confirmed in the changelog at
the time but not in the two release-notes files that carry it, so the
shipped notes have been showing a wrong date and the word provisional
since that release. The root mirror is regenerated for v0.4.1, so only
the archived copy needed correcting.
Back-fill the changelog for the six user-facing changes on this line
since v0.4.0: the antipoison FPR cap default, the bloom probe and
secp256k1 context performance work, the coordinate-cache and path-MTU
fixes, and the removed parent_switched counter. The Unreleased section
was empty, so all six entries are new.
Write the v0.4.1 release notes and mirror them to the root copy. The
notes lead with the FPR cap change and state plainly that this is the
second raise of that default in two releases, that it buys headroom
rather than fixing the underlying fixed-filter constraint, and that the
structural remedy is the v2 filter work. They also flag the rolling
upgrade window, where an upgraded and a not-yet-upgraded node can
disagree about mesh size, and point consumers of parent_switched at
parent_switches.
Bump the README status badge and the roadmap prose together so the two
agree.
The inbound FilterAnnounce cap at 0.10 rejects aggregates that are
legitimately near their operating ceiling, before the network reaches
the fixed-filter capacity limit. Raise the default to 0.20, which
corresponds to fill 0.7248 at k=5, about 2,114 entries on the 1 KB
filter (Swamidass-Baldi).
Also remove the duplicate default definitions in BloomConfig. Each
default was written twice, once in impl Default and once in the serde
default function, with nothing enforcing that they agree, so a config
file that omits the key took a different path from one that sets it.
impl Default now delegates to the serde default functions, leaving a
single source of truth.
The isolation loop and the failure-log dump both built container names
without the run suffix, so under a suffixed run they addressed containers
that do not exist.
The consequence was worse than the visible failure. Two of the three
checks in that loop assert a ping FAILS, and a ping into a non-existent
container fails for the wrong reason - so both passed vacuously, and the
security assertion the loop exists for was silently a no-op. Only the
third check, which expects a ping to succeed, noticed anything was wrong.
Add a guard that fails loudly when the container is absent, so a future
naming slip cannot quietly turn these back into no-ops rather than
surfacing as one confusing failure among two false passes.
These sites were missed because the suite passes when run by hand: with no
suffix set the unsuffixed names are correct, and the defect only appears
under the automation that sets one. Verified this time with the suffix
set, which is the condition that matters: 18 of 18, container names
resolving to real containers.
Making the sidecar network name unique per run exposed a latent problem
rather than causing a new one. With a fixed name, compose found any
existing network and reused it, which quietly masked the fact that the
address range was hardcoded. With a unique name there is nothing to
reuse, so the range is requested every time and the run fails outright if
anything already holds it.
Claim the range instead of assuming it. The suite now tries to create its
network on a candidate range and, if the daemon reports an overlap, moves
to the next candidate. The daemon's own address pool becomes the arbiter,
so two concurrent runs cannot land on the same range at all - as opposed
to a range derived from the run identifier, which only makes a collision
unlikely, and a collision is the precise failure being avoided.
The node addresses are derived from whichever range is claimed rather than
written alongside it, including the gateway address the isolation check
probes. Moved to a range that nothing in the tree claims and that sits
outside the daemon's default pool.
Only an overlap is worth advancing on. Any other creation failure is real
and is reported with the daemon's own message, rather than being retried
sixty-four times and buried - a lesson from a failure elsewhere in the
harness this week that was undiagnosable because its error output was
discarded.
All three nodes now attach to the pre-created network as external, so
teardown removes it explicitly.
Verified: three concurrent claims take three distinct ranges; a
non-overlap error fails fast rather than looping; the suite passes 18/18;
and the claimed gateway address is confirmed pingable when not firewalled,
so the isolation assertions still mean something.
The suite red-ed with an empty final peer_count while every substantive
assertion passed: both denied peers showed inbound handshake attempts and
zero outbound responses, which is the gate this suite exists to verify.
Two pre-existing defects combined. The final count was sampled once, in
the same second the load driver issues its last round of peer restarts, so
the read can land on a daemon busy with those restarts and return nothing.
And the fallback meant to cover that could never fire, because the shell
applies it to the last stage of the pipeline, which succeeds on empty
input - so an unanswered query was indistinguishable from a genuine zero,
and reported as a cap violation.
Poll the final read for a short window instead, and give the two cases
distinct output. A real regression still fails, just after the retry
window, since the loop exits early only on the expected value. Extract the
read the two phases share rather than leaving them to drift.
Two runs on one host destroyed each other's containers, producing
mid-test "No such container" failures that look like real defects. The
automated builder runs a full local CI on the same box every few minutes,
so the machine is contended almost always and this has red-ed both a hand
run and an automated gate.
Two independent causes are fixed here. Container names were hardcoded, and
docker names are global rather than scoped by compose project, so two runs
collided on the same name; every name now takes an optional suffix that
the harness sets from the run id. And the cleanup sweep matched a label
shared by every run, so one run's teardown force-removed another's
containers; resources now also carry a per-run label and the sweep can be
narrowed to it.
A third hazard turned up that was not in the original report: the sidecar
suite passes explicit compose project names, which override the run-scoped
project and put it outside the shared prefix entirely. Its project names,
network, and derived container references are now scoped too.
Both are default-off. With the suffix unset, names render exactly as they
do today and a bare compose invocation is unchanged, which is what keeps
the hosted CI and the documentation correct. A cleanup run with no run id
still reaps everything, which is what a manual "clear the box" wants.
The literal-name sweep was not sufficient: six scripts build container
names dynamically from node labels, and two suites create their own
containers outside compose. Those are handled at their construction sites.
Verified: syntax check on all modified scripts; compose validation on
every modified file with the suffix both set and unset; and a synthetic
two-run reproduction that shows the old cleanup destroying a bystander run
and the new one leaving it alone.
Known gap: subnets are still hardcoded, so two concurrent full runs will
still collide on address-pool overlap. That fix reaches into topology
configs, chaos scenarios, diagrams and production source, so it is left
for its own change rather than half-done here.
The lifecycle view's doc described the shell as implementing it over a
`connections` map that no longer exists. The establish view named that map
too, and was doubly wrong: both of its snapshot builders read only
`peers`. Rewrite both to describe the maps actually read.
Several comments in the node module carried shorthand from private
working notes - bare parenthetical labels, and one reference to an
internal tracker item - none of which mean anything to a reader of this
repo. Rewrite them so each clause states its own reasoning inline. The
paragraph above the tracker reference already carried the rationale, so
that one is simply dropped.
Comment-only; no behavior change.
The rekey suites' Phase 4 polled until the initiator-cutover count
reached its value at phase entry plus two, while Phase 6 asserts an
absolute minimum of four. On an unloaded host the ping phases can
outrun the jittered first rekey cycle, so Phase 4 can begin with a
single cutover on the books, release at three, and leave Phase 6 to
lose a sub-second race against the remaining first-cycle cutovers.
A rekey-outbound-only run failed exactly this way: five counted
cutover lines by t=32.6s of container life, with the Phase-6 count
snapshot at t~32.4s seeing three.
Floor the Phase-4 wait target at four so a released wait guarantees
the Phase-6 assertion — the docker-logs count is monotone over a
container's lifetime. A genuine rekey stall still times out the
bounded wait and fails Phase 6 with the real count.
Also drop the second-cycle framing from the comment, the phase
banner, and the assertion description: with rekey timers resetting at
each cutover a second cycle cannot begin inside this test's window,
so the counted events are first-cycle cutovers spread across the
topology's links, emitted once or twice per completed link rekey
depending on which side's promotion path logs them.
The per-peer control machine has absorbed every field the pending
connection carried. What remained was a struct holding two Noise
handles beside a duplicate copy of bookkeeping nobody read. Replace it
with a small carrier for the two handles and delete the type.
Presence of that carrier, not the state of the handles inside it, is
what marks a machine as mid-handshake. The distinction is essential
rather than stylistic: a failed handshake drops its initiation handle
and is deliberately retained so the stale sweep can reclaim it, and a
completed one has its session taken before disposal. Deriving presence
from the handles would make both invisible to the sweep, the
connection count, and the peering budget at once, leaking the slot
permanently. A test drives an empty carrier past every presence
predicate and then detaches it, so a future edit cannot quietly couple
the two.
The remote startup epoch now comes from the surviving carrier, which
the handshake operations already wrote at the same two points with the
same value. The paired writes onto the pending connection's own
bookkeeping had no readers left and are gone.
The handshake-phase surface leaves the public API: it was public by
accident rather than design, and the machine behind it is crate
internal. Callers outside the crate that need a view of pending
handshakes go through the operator queries, which are unchanged.
ConnectionState::inbound_with_transport loses its last non-test caller
with the inbound seed and is marked test-only.
The pending connection and the per-peer control machine have carried
duplicate copies of the handshake-phase fields since the machine gained
its own connection state. Read them from the machine and drop the
connection's projections.
The peer identity is the sharp one. The connection learned it from msg1
and the machine's carrier did not, so the two views genuinely disagreed
for inbound connections until the Noise operations moved onto the
machine and began recording each result on both. That is now in place,
so every reader can take the machine's copy unchanged: the establish
snapshot, the promotion sweep for competing connections, the dial and
path in-progress checks, the peering observations and per-peer in-flight
budget, and the stale-connection sweep's retry address.
Also repointed: our_index, their_index, transport_id, started_at, the
stored handshake message bytes, and the idle-timeout check. Two
duplicate index writes on the connection are dropped, both immediately
preceded by the machine-side write of the same value. Reads that
previously came off a connection detached just before its machine was
disposed now capture the machine's value first.
Test seeding follows the establish paths: the seed builders write
our_index to the carrier, which promotion now reads. A new test pins
the inbound identity learn on the carrier and the retry address a
failed inbound connection reports to the sweep, so a silent regression
to a blank identity cannot pass.
ConnectionState::duration loses its last non-test caller with the
connection accessor and is marked test-only.
These three had no counterpart on the control machine, so readers still
reached them through the pending connection. Add machine-side
accessors and repoint every reader, then drop the connection's.
The peer address needed its writes lifted, not just its reads
repointed: the machine's copy was never written. It is now written at
each of the points the connection's copy was — the inbound seed, the
dial, message-2 completion, and the two paths that seed a machine from
a pre-built connection — so promotion and the resend path read a value
with the same provenance at the same time as before.
Link and direction need no lift. Both machine constructors already
seed them from the same arguments the connection is built with, so an
outbound machine carries outbound state and an inbound one inbound,
and the machine's link always equals the connection's. The handshake
operations' direction guards read the machine's copy for the same
reason.
The stale-connection sweep's teardown log and its resend path both now
take the transport and address from the machine, and each keeps its own
check that a pending connection is still attached rather than relying
on the caller to have established it.
Add a test pinning link, direction, and address on the two shapes that
seed a carrier independently — the dial and an accepted message 1. The
cross-connection winner reads both values from a carrier one of those
two already seeded.
The pending connection drove its own Noise handshake while the control
machine held it, so the crypto and the state it produces lived on the
carrier that is going away. Move the six operations onto the machine:
starting and completing an initiation, processing an inbound
initiation, taking the session, testing for one, and dropping the
handle on failure. Bodies are unchanged apart from reaching the
handles through the attached connection.
Each operation now records its results on both carriers. The learned
identity, the remote epoch, and the activity stamp are written to the
machine's own bookkeeping at the same point and with the same value as
they are written to the connection's. The connection's copies still
have readers until those reads are repointed, so both have to be
written; the machine's copy of the learned identity was previously
never populated for an inbound connection, which is why an inbound
pending row showed no expected peer.
Driving the handshake from the machine means the machine has to exist
before the crypto runs. On the inbound path it is built above the
message-1 processing and still kept local, so a rejected message
leaves no registry entry and allocates no index. On the outbound path
it already existed from the dial, so it simply takes the connection
before the index allocation, and both failure arms unwind it as they
did.
Completing message 2 no longer mirrors the activity stamp separately,
since the completion itself now writes both carriers at the point the
mirror was approximating.
Three tests cover what the compiler cannot. A connection whose
handshake failed holds neither Noise handle yet must stay visible to
the stale-connection sweep, or every failed connection would leak and
hold a peering-budget slot forever. A message 1 rejected by the crypto
or by the ACL must leave no machine, no index, and the same rejection
count as before. A dial whose message-1 preparation fails must unwind
the machine registered at dial time; that test drives the
index-allocation failure rather than a crypto failure, which is the
arm this change actually widens, since the allocation now happens with
the connection already attached.
`Node::connections()` yielded the pending connection itself, so its
consumers reached the handshake-phase fields through that value. The
pending connection is being folded into the control machine, and the
fields will move off it, so yield the machine (keyed by its link) and
let each consumer reach what it needs from there.
Membership is unchanged: the iterator still selects machines that
carry a pending connection, which is the predicate `connection_count`
and the stale-connection sweep already use. Every consumer takes the
same value from the same place, one hop further out, and no field
changes carrier here.
The method is now internal to the crate. It yields the control
machine, which is not part of the published surface, and nothing
outside the crate iterates pending connections — operator views of
the handshake phase go through the query surface instead.
`promote_connection` detached the pending connection and then
interleaved reads off that detached value with three separate lookups
of the same control machine. Gather the machine-side fields
(`their_index`, `transport_id`, link stats) in the single `get_mut`
that takes the connection, so the machine is borrowed once.
Behaviour is unchanged. The connection is still taken before anything
is validated, so a rejected promotion leaves the machine with no
pending connection, and the prelude only gathers options: the checks
below it still report the first missing field in the same order
(`our_index`, `their_index`, `transport_id`, `source_addr`).
Link stats move to the prelude, ahead of those checks. The value is
the same either way: they live on the surviving carrier rather than
the detached connection, and nothing between the two points touches
the machine map. The lookup could not fail at the old site either,
since the function had already reached it through a successful lookup
on the same key, so dropping the defaulting arm changes nothing.
Add a test covering the error order and the detach, driving promotion
with connections missing each required field in turn plus every later
one, so an implementation that validated during the prelude would
report the wrong field. Test seeding grows a variant that lets the
caller shape the seed.
Tests built a free-standing PeerConnection, mutated it, and handed it to
Node::add_connection by value. None of those sites survives the removal of
PeerConnection, so converting them afterwards would mean one enormous commit
that cannot be reviewed honestly. Convert them now, while the struct still
exists and the conversion can be validated against a green tree.
Adds a cfg(test) Node::seed_handshake_machine plus a HandshakeSeed builder,
and rewrites make_completed_connection (now seed_completed_connection) and
the twenty inline builders onto it. add_connection keeps its body and loses
its test callers.
The builder's carrier seeding is a verbatim copy of add_connection's: the two
conditional writes for their_index and transport_id, then set_leg, through the
same entry().or_insert_with() so an existing leg-less machine keeps its
constructor-side fields. Nothing else reaches the carrier -- our_index,
source_addr, post-construction started_at and the stored handshake bytes stay
leg-only. Seeding more than that would let these tests observe a carrier
richer than production's and keep passing even if a later production
write-lift were missed.
The Noise exchange now runs on the already-seeded leg rather than before the
hand-over. That is neutral because the only read of expected_identity is
guarded by is_outbound, and no crypto method allocates a session index.
Also adds a compile-time check that PeerAction is Clone + Eq, which is what
keeps a runtime handle from being smuggled into an action payload.
The session index was carried three ways: on the leg, on the peer machine's own
connection-state copy, and on a separate machine-owned shadow field. Populate the
machine's connection copy on the outbound path (it was written only on the inbound
and cross-connection-swap paths before), then delete the shadow field and route its
readers, the decrypt-registration lifecycle, and the rekey cutover through the
single carrier. The shadow and the connection copy diverged only on paths that are
unreachable or dormant in production (a rekey cutover with no newly allocated index,
the swap winner's later reads, and the never-dispatched timeout/disconnect events),
so the merge is byte-identical for every reachable path. Rewrote the outbound-promote
index tests and the dial-time comment for the single-carrier contract.
The connection leg's their_index, transport_id and link_stats duplicated the peer
machine's own connection-state copy. Route every production reader through the
machine copy: promotion, the stale-connection reaper, the outbound msg1 resend, and
the in-use / connecting-path checks. Seed the inbound machine's transport_id from the
msg1 packet (the same source the leg used), matching the outbound dial seed, so the
hard-required promote read never sees a missing transport. link_stats is a
never-mutated zero seed, so its leg accessors are removed outright; the leg
their_index/transport_id getters/setters remain only for the test-only connection
builder. Also delete the write-only next_resend_at_ms field from the connection
state (the resend deadline is carried by the machine-armed retransmit timer, not this
field). Byte-identical.
The connection leg's stored msg1/msg2 handshake-resend buffers duplicated the
peer machine's own connection-state copy. Write the machine copy at the outbound
msg1-prep and the two inbound authorize sites, then source the retransmit
resend-bytes reader, send_stored_msg1, and the pending tier of find_stored_msg2
from the machine. The post-promote active-peer msg2 copy is written from the same
wire bytes, so the pending tier now matching after promotion is value-identical.
Byte-identical resend behavior.
The expected_peer connection-row field read the peer identity off the leg;
source it from the machine's own connection-state copy instead, matching the
started_at/last_activity treatment. The machine carrier holds the same identity
as the leg for every connection that appears in the view: outbound legs seed
both from the dialed identity, and inbound legs never rest in the connections
view (the machine takes the leg at promotion), so their machine copy is
unobserved. The leg accessor stays for the crypto-extraction path and retry
decisions. Byte-identical.
The connection leg's started_at and last_activity duplicated the peer
machine's own connection-state copy. Make the machine copy the sole live
telemetry and timeout carrier: re-stamp it with the leg's provenance when
the leg is attached at handshake start (the msg1-prep clock, not the earlier
dial-time constructor value) and mirror the completion touch, then delete the
leg's last_activity/touch accessors and source the connection-row projection,
show_connections, and the timeout readers from the machine. Byte-identical
for all normal paths.
The connection leg's HandshakeState field duplicated the peer machine's
handshake phase. Delete it and derive the displayed handshake state string
from the machine's PeerState, looked up by link id (mirroring the resend
count). Move the failure signal onto the machine: a send_failed flag that
preserves retransmit eligibility (the machine stays in its handshake phase),
alongside the existing Failed state. The leg's crypto self-gates now guard on
Noise handle presence instead of the deleted phase field, and mark_failed
only drops the handle. Telemetry strings, wire bytes, index allocation, and
the stale-connection reaping are byte-identical for all normal paths.
handle_msg1 now creates one local machine before classification and routes
on the InboundDecision it returns, instead of calling establish_inbound in
the shell and re-deriving the decision inside each arm. The promote and
restart arms drop their own per-arm machine construction and the redundant
InboundMsg1 step; their phase-1 actions come from the single call. The
effect-bearing arm bodies (rekey-respond, duplicate resend, reject
bookkeeping, and the promote/restart tails) are unchanged.
Behavior-neutral: the same establish_inbound evaluation over the same
snapshot and wire, done once in the machine method rather than once in the
shell, with byte-identical arm bodies and no change to index allocation,
wire sends, machine state, or telemetry.
The inbound msg1 machine arm re-derived the establish decision for
its own action selection while the shell matched on a separately
computed copy. Expose the arm as a method returning the decision
alongside the actions so a driver can route on it directly; the
step dispatch delegates and keeps its action-only shape.
The dead rekey-respond arm is stripped to decision-only and its
helper deleted: it allocated from the real index allocator, wrote
the rekey shadow index, emitted a wrong-framed rekey send, stamped
the dampening clock, and flipped state, none of which happens on
the live path, where the inline respond body owns all effects. The
resend-msg2 arm's send emission is stripped the same way; the
decision already carries the stored bytes.
send_stored_msg1 marked the embedded leg failed by writing it
directly from the shell. Route the write through a new
HandshakeSendFailed machine event instead: the machine marks its
leg so the stale-connection sweep reclaims it, without leaving the
handshaking state, so retransmit eligibility survives the window
between the failed send and the sweep exactly as before.
send_stored_msg1 gains a now_ms parameter threaded from the action
executor for the step call; the failure arm itself ignores it.
handle_msg2 no longer pre-computes establish_outbound alongside the
machine's own evaluation of the same snapshot. The shell now builds
the snapshot, steps the persistent outbound machine once at the
decision point, and routes on what comes back: the promote action
vector drives promotion through the executor as before, and the
ResolveCrossConnection decision selects the inline swap/keep
resolution bodies, which are unchanged.
The machine step and its defensive transient-rebuild move up from
the promote arm to the decision point; the cross-connection path
keeps its take-leg-then-dispose ordering with the step preceding
both. Comment prose at the touched sites refreshed to describe the
single-decision-site shape.
The machine's on_msg2 cross-connection arms crystallized state and
emitted FreeIndex/RegisterDecryptSession actions that duplicate the
inline shell resolution, but they are unreachable on the live path
(the shell removes the machine before running the swap/keep bodies).
Making them live in that shape would double-free the outbound index
through the executor.
Strip both arms to a single new ResolveCrossConnection { swap }
action: a decision conveyed to the driver, not an effect. The shell
intercepts it and runs the inline resolution, which owns all effects
permanently. The action executor gets a defensive unreachable arm.
The Promote arm and set_their_index are unchanged.
The pending-handshake PeerConnection map and the per-peer control
machine map were parallel LinkId-keyed structures whose keysets must
stay coherent by hand. With every leg now born with a machine, the leg
becomes storage inside its machine (leg: Option<PeerConnection>, pure
storage the machine never reads or drives) and Node loses the
connections field; every access routes through the machine.
The non-mechanical lowerings, each argued at the site: the
rekey-vs-establish gate in handle_msg2 tests leg-absence (an established
peer's machine stays keyed by its link, so machine-presence would
misclassify every rekey msg2 as a fresh establish); the
connecting-predicates, peering observation, and handshake-slot budget
iterate machines-with-legs so connect-window machines (leg not yet
born) are excluded exactly as before and never double-counted against
their pending-connect slot; the cross-connection extract takes the leg
before disposing the machine; the stale reaper takes the leg and leaves
the machine untouched when none is present, matching the old early
return. The map-coherence debug check keeps its machine-has-carrier
direction with the embedded leg as a carrier; the leg-to-machine
direction is now true by construction and its gate const is gone.
connection_count() counts machines with legs; the connections()
iterator, the test seams, and the control-socket connection rows are
re-implemented over the embedded legs with unchanged output.
Assert, once per tick in debug builds, that the peer control-machine map
and its carriers stay coherent: every control machine has a live
handshake leg, an active peer, or a pending connect (a machine with none
is a leak the stale reaper can never see), and every handshake leg has a
machine. The second direction sits behind a file-local const so a branch
where handshake-window legs legitimately run machine-less can gate it
off without weakening the leak tripwire.
Teach the add_connection test seam to seed a control machine for the leg
it inserts (derived from the connection's direction and identity), and
remove_connection to dispose it, so the seam-built topologies satisfy
the check. Three unit tests: seam coherence, coherence through a real
promotion, and the orphaned-machine panic.
Several module and field docs still described the per-peer machine as
unwired shadow scaffolding. It has been live for some time: machines are
inserted at dial and inbound msg1, stepped by the handshake handlers and
the rekey-cadence and liveness-reap routers, and the executor's
SwapSendState/CompleteDrain/InvalidateSendState arms are the authoritative
paths (the inline bodies survive only as debug-assert release fallbacks).
Rewrite those docs to the current truth while keeping the still-true
dormancy facts: PeerEvent::Timeout and PeerEvent::Tick are never
dispatched in production, retransmit fires on the machine-armed deadline
while the timeout reaper keys on timer presence with the config
threshold, and the remaining inert executor stubs are SendRekey,
SendLinkMessage, and the connected-UDP arms. Drop the stale
allow(dead_code) on the peer_machines field.
PeerSlot (and its entire impl surface) was referenced only by its own
module tests and the lib.rs re-export; peer storage has always used the
separate connections/peers maps. PeerConnection's resend_count/
next_resend_at_ms/record_resend delegations had no production callers:
the live resend counter is machine-sourced (connection_resend_count reads
the per-peer machine), and the FSP session layer uses SessionEntry's own
methods. ConnectionState::next_resend_at_ms is now test-only (its
remaining callers are the fmp state unit tests) and marked cfg(test).
The PeerSlot unit tests go with the enum; test_resend_count_tracking is
dropped because the delegation target's schedule arithmetic is already
covered by the fmp resend_bookkeeping test.
Phase 4 of the rekey suite waited for only one more initiator cutover
(guaranteeing three) while Phase 6 asserts at least four. The fourth
cutover then had to land in the brief window between the wait returning
and the Phase 6 log snapshot, so host load could push it past the window
and fail the run even though every cutover completed correctly. Wait for
two more cutovers, the full second rekey cycle, matching the Phase 6
threshold, so the asserted count is guaranteed before it is checked.
Feed the per-peer machine two observation events so its control/rekey
shadow state stays coherent with the inline crypto-session installs,
without moving those installs. The crypto effect bodies are unchanged.
On rekey-msg2 completion, feed the machine the responder index on the
success path only (after the pending session is installed); the abandon
path feeds nothing.
On cross-connection resolution, feed the machine the swap outcome: on a
swap it records the new outbound and responder indices to mirror the
session replacement, and on a keep it does nothing. The observation
targets the promoted peer's machine, not the outbound leg whose machine
is removed as the block is entered.
Both observation handlers emit no action; nothing consumes the updated
shadow state yet.
Move the outbound handshake-timeout reap off the unconditional check_timeouts
scan onto the machine-armed HandshakeTimeout timer. A new drive_handshake_timeouts
(run before the retransmit drive, so a timed-out leg is reaped rather than resent
on the same tick) reaps the outbound legs that carry a HandshakeTimeout timer and
have idle-timed-out this tick.
The timer's presence selects the leg (only outbound legs arm one; IK inbound arms
none); the reap threshold is the shell is_timed_out(now, config) predicate, not
the timer's stored deadline. The machine arms the timer from a hardcoded constant
at dial, which is not authoritative for an operator-tuned handshake_timeout_secs,
so reading the threshold from config each tick keeps the reap neutral for any
timeout value and on the last_activity clock exactly as before.
check_timeouts keeps reaping everything else: failed connections (all of them,
promptly) and the idle-timeout of legs without a machine timer (inbound legs, and
any machine-less connection). Total coverage is unchanged. check_timeouts runs
before the timer drive, so a failed outbound leg is reaped there first, its timers
dropped, and the timeout drive never double-fires on it.
Split drive_peer_timers into the timeout drive and the (byte-identical) retransmit
drive. The dormant machine timeout handler is not dispatched, so the session index
is freed once, by cleanup_stale_connection. A regression test drives a timed-out
outbound leg to reap; the existing check_timeouts tests continue to anchor the
residual sweep. Test count 1631 to 1632.
Move the outbound msg1 resend off the unconditional tick function onto the
machine-armed retransmit timer. The per-peer machine already arms a
HandshakeRetransmit deadline at dial; a new drive_peer_timers fires the due ones
(kind-filtered to the retransmit timer) and homes the resend counter on the
machine, where the operator-visible count now reads from.
The resend decision reads the same operator config as before (interval, backoff,
max), the wire bytes and transport target still come from the shell connection,
and the pure core computes the backoff schedule. As with the deleted
resend_pending_handshakes, the count and reschedule advance only on a successful
send: a failed send neither advances the count nor marks the connection failed,
it just retries on the next tick. The handshake-timeout timer stays on the legacy
check_timeouts path, and the rekey/liveness timers keep their own shell drivers,
so drive_peer_timers deliberately fires only the retransmit kind.
The show_connections resend count is relocated to read from the machine (the
counter's new home) via connection_resend_count; machine-less and inbound
connections report 0, matching what the shell connection reported before. Delete
resend_pending_handshakes and resend_candidates (the latter also from the
LifecycleView trait, its only user). The resend unit test is re-expressed against
the machine + timer path, covering the due check and the record-on-success
semantics.
Known limitation: the first resend interval is armed from the machine's hardcoded
1000ms constant, which equals the config default. Under an operator override of
handshake_resend_interval_ms the first resend diverges from the pre-change
dial+interval; subsequent resends and the cap remain config-driven. Neutralizing
the first-resend override needs the interval threaded into the sync core (the
shell arm would be clobbered by the machine's own timer arming at dial), deferred
to when the connection and machine entities merge. Test count unchanged at 1631.
The control machine already emits SetTimer/CancelTimer actions when it arms the
handshake retransmit and timeout deadlines, but their executor arms were a
single no-op stub and the machine's Timeout event was never dispatched -- the
real work still runs on the legacy tick (check_timeouts,
resend_pending_handshakes).
Add the storage the time-as-input driver will read: a peer_timers map keyed by
LinkId then TimerKind, holding each armed timer's absolute deadline. The SetTimer
arm now inserts (overwrite = reschedule) and CancelTimer removes; the outbound
msg2 promote cancels the two dial-armed handshake timers, since the machine
survives promotion and the entries would otherwise linger in the store.
This store is a shadow: it is written and cleared but no driver reads it yet, so
behavior is unchanged. The legacy tick stays authoritative until the driver that
feeds Timeout and deletes the overlapping tick paths lands.
Route every machine removal through a new remove_peer_machine(link) choke-point
that drops the timer store alongside the machine, replacing the twelve direct
peer_machines.remove sites so no armed timer outlives its machine. TimerKind
gains Hash (to key the store) and Ord (for deterministic driver collection); it
is internal and never serialized. Test count unchanged at 1631.
Route the connection-oriented (TCP/Tor) outbound path through the peer state
machine, matching how the connectionless path already works.
initiate_connection's oriented branch now drives PeerEvent::Dial with
connection_oriented=true; the machine parks in Connecting and emits
OpenTransport, whose executor arm performs the non-blocking transport.connect
and pushes the PendingConnect. When the connect resolves, poll_pending_connects
prepares msg1 in the shell and then drives PeerEvent::TransportConnected, which
sends msg1 via the machine's SendHandshake arm.
The msg1 prepare (index allocation, Noise leaf, wire arming) MUST run in the
shell before the TransportConnected drive: send_stored_msg1 only transmits an
already-armed wire, so a drive-only path would silently send nothing. The
machine's our_index stays unset; the connect-failure path keeps its direct
handshake-timeout handling (TransportFailed stays dormant). The now-unused
Node::start_handshake helper is removed.
Behavior-neutral: same transport.connect, same PendingConnect, same msg1 send
and failure teardown as the removed inline path -- only the driver changes from
inline shell code to the state machine.
Move the peer_machines insert for an outbound leg out of the tail of
prepare_outbound_msg1 to a single shared site in initiate_connection, before
the connection-oriented / connectionless fork. This lets the connection-oriented
path find the machine at dial time (so it can be driven through the connect
handshake) without prepare_outbound_msg1 -- which for that path runs after the
connect completes -- clobbering an in-progress machine back to Discovered.
Because the machine now exists before the fallible dial steps, add
peer_machines.remove to every failure path in the widened dial window: the
index-allocation and Noise-leaf failures in prepare_outbound_msg1, the oriented
transport.connect failure, and both poll_pending_connects teardown arms
(handshake-start failure and async connect failure). remove_link does not touch
peer_machines, so these explicit removes are required.
Behavior-neutral: the connectionless path still drives the machine to
Handshaking and sends msg1 identically; the machine's our_index stays unset
(no spurious UnregisterDecryptSession on a later inbound restart); a failed dial
leaves peer_machines empty for that link exactly as before; and no
connection-oriented machine drive is wired here.
The connection-oriented (TCP/Tor) outbound connect->handshake path had no
`cargo test --lib` coverage; it was exercised only by the opt-in Tor
integration suites, and the TCP node tests bypass it via a manual
connectionless handshake helper. Add three unit tests:
- a machine-level test driving Dial{connection_oriented:true} -> Connecting
(emitting only OpenTransport, no msg1) -> TransportConnected ->
Handshaking{SentMsg1}, asserting the exact OpenTransport and
SendHandshake+SetTimer action vectors that start_outbound_handshake emits
(its oriented reach via on_transport_connected was previously untested; the
connectionless reach via on_dial was already covered);
- two node-level tests over a real loopback TcpTransport: a successful connect
that reaches start_handshake (observed via a pending_outbound entry), and a
connect to a closed port that routes through the failure arm (link torn down,
no msg1 dispatched).
Tests only; no production change.
Split the outbound handshake setup out of start_handshake into
prepare_outbound_msg1 (allocate the index, run the Noise leaf, frame and arm
msg1 -- the fallible steps, returning an error the caller propagates) and
send_stored_msg1 (transmit the armed wire). A connectionless dial now runs
prepare in the shell, then drives the control machine, whose SendHandshake
action sends the wire via the executor. Connection-oriented dials keep calling
start_handshake, now prepare followed by the send inline.
The dial event gains a connection_oriented flag so the machine's on_dial sends
msg1 immediately for connectionless transports (no connect step) instead of
opening a transport first. Behavior is unchanged: the same index, Noise leaf,
wire bytes, maps, and send-error handling as before, with index-allocation and
Noise failures still propagated synchronously before the send. A regression
test covers that a promote from the post-dial handshaking state is identical to
the former discovered-state promote.
Create and persist the per-peer control machine when an outbound handshake is
dialed, keyed by its link, instead of building a transient at msg2. The msg2
completion path now looks up that persisted machine to drive the promote,
falling back to a transient only if none is present (e.g. a direct-seeded test).
The machine parks in the Discovered state until promotion and is inert to the
liveness reap and rekey cadence while unpromoted, since it is absent from the
peers map. It is removed on every path that ends the outbound leg without
promoting -- the stale-connection reaper, the msg2 authorization-failure arm,
and the cross-connection resolution block -- mirroring the connection's own
lifetime so no dangling machine survives.
Its session index is deliberately left unset on the machine (the shell owns the
index on its connection), so a later inbound restart does not emit a spurious
decrypt-session unregister. A regression test covers that invariant.
Add a LostKind discriminator to the ReportLost action so the executor can
route an un-promoted handshake failure to the connected-guarded reconnect
reflex (note_handshake_timeout) and an established peer's link-death to the
unconditional one (note_link_dead), instead of collapsing every loss to
note_link_dead.
The two loss producers dispatched today, the liveness reap and the
inbound-restart-then-promote arm, both keep the link-dead routing, so
behavior is unchanged. The handshake-timeout and dial-failure producers are
tagged accordingly but stay dormant until their events are dispatched.
Remove the now-stale `#[allow(dead_code)]` on `advance_peer_machine`
and `execute_peer_actions`; both are live (called from the link-dead
reap, rekey cadence routing, and the handshake establish sites).
In `route_rekey_cadence`, reuse the ambient context timestamp for the
machine step instead of taking a second clock sample; the extra
sample's only consumer was the currently-inert cutover drain timer, so
the change is behavior-neutral.
Prove the per-peer machine's action type is a runtime-agnostic message
contract: it is Send + Sync + 'static, and every variant round-trips
unchanged through a single-threaded async channel (a current-thread runtime,
sender and receiver on one thread). A wildcard-free match over the variants
makes any future action a compile error here until it is added to the sample,
so a variant that embedded a runtime handle could not slip through unproven.
The two promote-failure warn! sites in the peer-action executor emit under
this module's own tracing target rather than the handshake target the rest of
the establish-path logging uses. Pin both (outbound "Failed to promote
connection" and inbound "Failed to promote inbound connection") to
fips::node::handlers::handshake, matching the sibling max-peers reject log and
keeping all establish-path diagnostics grouped under one target regardless of
which module physically emits them.
Comments across the per-peer machine, executor, lifecycle supervisor,
and peering reconciler carried internal rollout labels and design-note
section references. Rewrite them to describe the code's behavior
directly. Comment-text only; no code changes.
Route each link-dead peer that the tick sweep's plan_heartbeats decides
to reap through the per-peer machine and executor, replacing the inline
reap body in check_link_heartbeats. The batch decision, the liveness
snapshots (read from the hot-path-written receive clock), and the
heartbeat-send arm stay shell-side and byte-unchanged; the machine only
consumes the decided LinkDeadSuspected, tearing the peer down via
remove_active_peer and reporting the loss to the reconciler exactly as
before, on the same tick with the same wall-clock timestamp. The reap
log stays shell-side.
The machine's link-dead handler no longer emits a decrypt-session
unregister keyed by its shadow index (the full peer teardown already
unregisters the real index; the shadow could have drifted to a reused
index), and its guard now covers the Established state a freshly
promoted peer sits in.
Handshake-timeout, retransmit, and stale-connection cleanup stay inline:
they act on pre-promotion legs that have no machine, and the loss reflex
they use differs from the link-dead one.
Route each Cutover and Drain that the shell-side batch poll_rekey decides
through the per-peer machine and the executor, replacing the inline
effect bodies in check_rekey. The batch decision and its per-peer
snapshots stay shell-side and byte-unchanged: poll_rekey phase-groups all
cutovers, then all drains, then all initiations across the peer set, and
that ordering governs the shared index allocator's free-then-allocate
sequence that appears on the wire, so the machine only consumes the
already-decided actions (a new RekeyConsume event) without re-deciding.
InitiateRekey stays inline (its Noise msg1 build is a shell-side leaf)
with a RekeyInitiated observation feeding the machine so its control
state stays coherent for the next tick's cutover.
The cutover and drain logs, which relocated into the executor in the
prior commit, are pinned back to the fips::node::handlers::rekey tracing
target so they stay visible under the operator's module log filter.
Also clears the machine's shadow draining_index on drain so a later
cross-connection resolution cannot double-free the already-freed index.
Prepare the establish executor to drive FMP rekey by activating the
SwapSendState action (initiator K-bit cutover via cutover_to_new_session,
with the gated decrypt-worker re-registration) and adding a CompleteDrain
action (erase the drained previous session: free its index, drop its
peers_by_index entry, unregister its decrypt session). Both reproduce the
current inline rekey.rs cutover and drain bodies. The machine's drain
mapping now emits CompleteDrain, using the real drained index rather than
a shadow copy.
Unwired: nothing drives the machine's rekey path yet (live rekey still
runs inline), so these arms are unreachable and the change is
behavior-neutral. The cadence fold that routes cutover and drain through
the machine follows.
Move register_decrypt_worker_session out of promote_connection into the
executor's PromoteToActive handler, gated on a promoted or
cross-connection-won result. Every live promote now flows through that
one executor path, so registration still fires exactly once at the same
synchronous point; the direct test callers of promote_connection spawn
no worker pool, so the call was already a no-op for them.
Add the cross-connection loser-link teardown (close the losing
transport, remove its link, re-point addr_to_link at the winner) to the
executor as a guarded follow-up. It is unreachable on the current driven
establish paths, which only promote net-new peers, and asserts so, but
keeps the executor complete for when that case is driven.
Remove the now-dead drive_promote_to_active and ConnAction::PromoteToActive.
Cut the net-new outbound handshake completion (a received msg2 that
promotes a fresh outbound leg to a new peer) over to the per-peer
control machine, mirroring the inbound cutover. handle_msg2 still runs
the msg2 prologue, the ACL check, and the cross-connection swap/keep
arms inline; for the net-new promote it now builds a transient machine,
steps it, and drives promote_connection through the executor. The
session index was already allocated at dial, so there is no two-phase
authorize here.
The wire, index sequence, and peer registry state after promote are
byte-neutral. To keep the promote-failure path neutral, the executor's
cleanup now distinguishes inbound from outbound: an outbound promote
failure records the reject only, matching the prior handler, rather
than the inbound path's link and index teardown.
Cross-connection swap/keep, rekey-msg2, and the dial path stay inline;
the loser-link surgery for the currently unreachable driven
cross-connection case lands with the register relocation next.
Cut the restart handshake (an inbound msg1 from a peer that reconnected
with a new epoch) over to the per-peer control machine, completing the
inbound establish cutover. The old peer is torn down and the fresh leg
promotes through the same two-step authorize-then-allocate path as the
net-new case: the machine's first step emits the old-peer teardown
(invalidate send-state, report loss), the shell interposes the ACL
check, and the second step allocates the new index and sends msg2. The
old index is freed before the new one is allocated and the msg2 wire
bytes are unchanged, so the sequence stays byte-neutral.
With restart driven through the machine, the shared inline establish
tail that only the restart arm reached is deleted.
Also bounds the peer_machines map (remove_active_peer now drops the
peer's machine entry) and restores the msg2-send, promote, and
index-allocation failure warnings the cutover had dropped.
Cut the net-new inbound handshake (a fresh msg1 that promotes to a new
peer, plus the at-capacity reject) over from the inline handle_msg1 logic
to the per-peer control machine. handle_msg1 still classifies via
establish_inbound and still owns the Noise wire step, the late ACL check,
and the promote_connection registry surgery; for the net-new path it now
builds the machine, steps it, and executes the returned actions.
Authorization is interposed between two machine steps so the session
index is allocated only after the ACL check passes: a rejected or
unauthorized msg1 consumes no index, matching the prior order exactly.
The msg2 wire bytes, the index-allocation sequence, and the reject
metrics are all byte-neutral. Restart, resend, rekey-respond, and the
other reject arms stay inline unchanged; they move to the machine once
outbound establish is cut over and every promoted peer has a machine.
Also fills in the executor's send-failure and promote-failure cleanup so
a mid-establish error tears the leg down and frees its index exactly as
before.
Add Node.peer_machines (a LinkId-keyed map from the stable link handle to
the per-peer control machine) as the home for the machines, and a new
dataplane/peer_actions.rs holding execute_peer_actions / advance_peer_machine:
the executor that maps each PeerAction the machine emits to its shell call —
frame and send a handshake via build_msg2, drive promote_connection and feed
the PromotionResult back through the machine, tear a peer down via
remove_active_peer, free session indices, report loss via note_link_dead.
Actions for the rekey, connected-UDP, and timer paths are stubbed with notes
for the commits that fold those mechanisms in.
Unwired: nothing drives the machine yet — no live handler path calls the
executor and peer_machines is never populated — so this is behavior-neutral;
the inbound and outbound establish paths still run their existing inline
logic. The executor is cut over path-by-path in the following commits.
Draw the control/published-send-state boundary inside ActivePeer by
grouping the send-critical fields — the three epoch session slots
{current, previous, pending}, the K-bit flag and session-start, the
transport target, the connected-UDP handles, and the hot counters —
into a new co-located PeerSendState struct. The control-tier fields
(identity, connectivity, declaration/ancestry, filter and tree-announce
groups, remote_epoch, the rekey-negotiation sub-machine, and the rest)
stay on ActivePeer.
Behavior-neutral: a pure field regrouping. Every accessor signature is
unchanged (bodies now read/write self.send.*), so the hot path and the
handlers are byte-untouched; both K-bit cutovers still rotate the three
slots atomically with the same control-tier updates. No Arc/ArcSwap —
the fields are co-located and read by plain borrow; publishing behind a
shared cell is later plumbing for a sharded data plane.
Introduce src/peer/machine.rs: a sans-IO per-peer control FSM that
consolidates the scattered handshake/rekey/timeout driver logic now
spread across node/handlers. The machine is a pure reducer —
step(event, now, index_allocator) -> [action] — that reuses the
existing FMP decision cores (establish_inbound/establish_outbound/
cross_connection_winner/poll_*) rather than reimplementing any
decision, and returns runtime-agnostic actions the driver executes.
Control-tier state only; the published send-state boundary and the
driver wiring land in following commits. The machine is terminal at
Closed — re-dial is the reconciler's, so it holds no cross-attempt
retry state.
Includes eight unit tests: inbound and outbound establish, N:1
identity crystallization, the dual-initiation tie-break,
restart-override, rekey initiator cutover, the data-plane-owned
responder cutover boundary, and liveness -> link-dead -> report-lost.
Unwired — nothing calls it yet.
Replace the hand-rolled RawFd field and the unsafe Drop on
ConnectedPeerSocket with an OwnedFd, whose own drop glue closes the
fd. from_fd now stores the OwnedFd it already receives instead of
stripping ownership through into_raw_fd, and the manual libc::close is
gone, shrinking the unsafe surface.
Behavior-neutral: the fd still closes exactly once at last-Arc-drop and
as_raw_fd returns the same underlying fd while the socket is alive.
Wire exit detection for the four directly-observable optional children so
the supervisor FSM's ChildExited edge fires at runtime. A runtime
child-liveness mpsc channel carries a Child on exit: the DNS task and the
two TUN threads self-report when their body returns, and a 2s poll monitor
reports mDNS and Nostr via new is_finished accessors. The rx_loop gains a
select arm that steps the FSM and republishes health (Degraded, since a
running node always has at least one transport up). Transports and worker
pools expose no runtime-exit signal and are left for a follow-up.
Adds LanRendezvous::is_finished and NostrRendezvous::is_finished; the
latter treats a shutdown-taken connect_task as finished so the monitor
terminates after a stop instead of polling forever.
Add the sans-IO half of runtime child-liveness monitoring: a ChildExited
event and an on_child_exited handler that routes a runtime task or thread
exit the same way a start failure does. An optional child exiting degrades
the node (Degraded, with the child recorded in the health reasons); the
last transport exiting publishes Failed. There is no restart, and the
handler is inert outside Running (startup, drain, and teardown own their
own child bookkeeping via the pending/up sets). Failed here is a published
health signal only, not a teardown.
The start and runtime paths share the health classification, extracted
from resolve_start_health into classify_health.
The producer that emits the event (the exit-detection wiring) lands in a
following commit; the event variant carries a temporary allow(dead_code)
until then.
The overlay-discovery cutover to the sans-IO reconciler dropped the
operator-facing "open-discovery sweep complete" summary, including its
per-reason skip breakdown, because the sweep logic moved into the
log-free core. Have the core accumulate the enqueue/skip tally as it
reconciles and return it as data; the driver adds the two values only it
holds (the raw cache size and the self-advert filter) and emits the
summary, reproducing the old startup-vs-per-tick summarize gate and the
budget-zero debug path.
The self-advert precedence matches the pre-cutover sweep: a stale or
self-configured own advert is attributed to the age/configured buckets,
not skipped_self.
Behavior-neutral: the tally is pure side-counting; no dial or enqueue
decision changes.
Cut the last scattered peering mechanism — opportunistic growth from
transport-neighbor beacons and LAN mDNS — over to the reconciler's
opportunistic layer via a gate-checked reconcile_opportunistic wrapper,
one call per tick slot (transport and LAN stay separate slots so their
per-tick budget and per-peer cap are not shared). The driver keeps the
beacon/mDNS I/O and the path-granular prefilters that read live state
(self, fresh-enough-to-skip, connecting-on-path) and executes the emitted
Connect intents; the core owns the connected/budget/per-peer-cap
decisions. A first-wins per-peer dedup on the LAN path reproduces the old
inline once-per-peer dial now that the snapshot core cannot observe the
intra-tick connecting feedback.
Delete the now-dead discovery_connect_budget helper. With this, all three
scattered peering mechanisms (auto-connect and retry, overlay discovery,
neighbor growth) are unified in one sans-IO reconciler. Behavior-neutral;
unit test count unchanged (1607).
Cut the overlay (Nostr open-discovery) enqueue over to the sans-IO
reconciler. A gate-checked reconcile_overlay wrapper runs the overlay
layer alone at the discovery tick slot; the monolithic reconcile would
re-fire the always-on retry-dial and double the per-tick dial cap, so the
overlay slot must call only its layer. The driver builds the candidate
pool from the overlay advert cache (self excluded), the cooldown and
configured-npub sets, and the startup/steady max-age, then feeds the
reconciler; the emitted enqueue set drives the per-enqueue identity-cache
and alias pre-seed exactly as before. Enqueued entries are dialed at the
retry slot, preserving the two-phase cadence.
Delete the now-dead enqueue-budget and expiry helpers. Behavior-neutral:
same candidates enqueued in the same order with the same budget, cap, and
expiry. Opportunistic transport-neighbor growth remains imperative and is
cut over next. Unit test count unchanged (1607).
Route the auto-connect floor and the connection-retry mechanism through
the sans-IO reconciler instead of the imperative Node methods. A thin
peering driver (note_handshake_timeout / note_link_dead) centralizes the
reflex path with the gate guard and the already-connected check; the
per-tick retry-dial and the startup floor build reconciler inputs and
execute the emitted Connect intents. The three imperative retry methods
are removed and their call sites rerouted.
Wire the drain and startup gates. Entering a bounded drain now clears the
retry schedule and suppresses the peer-loss reconnect reflex (the drain
window runs under the Suspended gate), so the drain no longer fights a
reconnect for the peers it just closed. The startup floor runs under an
explicit Reconciling gate at the existing peer-connect seam, preserving
the current dial position.
Behavior-neutral except the intended drain change. Overlay discovery and
opportunistic transport-neighbor growth remain imperative for now; they
observe the relocated retry schedule and are cut over next. Unit tests
migrated to the driver API with assertions unchanged (1607).
Move retry_pending and pending_connects from Node into the Peering owner
struct (Node.peering), the home introduced with the reconciler core.
Pure mechanical relocation: the two collections now live behind
self.peering, with roughly seventy accessor sites repointed. No decision
logic is touched and the unit-test count is unchanged (1607). The
imperative peering methods still run; the reconciler is wired in the
following commit.
Introduce PeeringReconciler, the synchronous decision core for peer
desired-state, with its input/action vocabulary (Gate, Budget, Observed,
Candidate, DiscoveryPools, Policy, PeeringAction) and the Peering owner
struct. reconcile() computes connect/retry intents across four layers:
the auto-connect mandatory floor, the ceiling-only overlay enqueue, the
opportunistic transport-neighbor growth, and the node.limits ceiling
enforced inline. It mirrors the supervisor sans-IO shape: no I/O, no
clock reads (time enters as a parameter), no runtime handles.
The core owns its own cross-attempt retry schedule so escalating backoff
survives the fresh connection created per re-dial. Gate-guarded reflexes
(on_handshake_timeout / on_link_dead) reproduce the current backoff math
and no-op while draining. Overlay is strictly ceiling-only (no peer-count
set-point). Disconnect is defined but never emitted (no shedding).
Unwired here: the driver still runs the imperative peering methods and
Node holds the live retry map. 11 unit tests drive the pure core with
synthetic inputs. No behavior change.
Homing the retry schedule under peering/ lengthened the RetryState
module path, pushing the retry_state_iter return-type line past the
100-column max_width. Rewrap to satisfy rustfmt. No behavior change.
Create src/node/peering/ as the home for the peer desired-state
(homeostatic reconciler) concept and move the cross-attempt connection
retry schedule into it: RetryState plus schedule_retry /
schedule_reconnect / process_pending_retries.
Mechanical relocation only. The methods remain inherent on Node; the
retry_count must persist across re-dials (a fresh connection is created
each attempt), which is why the schedule belongs in the peering home
rather than on a per-connection type. The one-level-deeper module path
requires pub(super) -> pub(in crate::node) to preserve the prior scope,
plus module-path fixups at the reference sites. No behavior change; unit
test count unchanged (1596).
Determine node health at start completion instead of unconditionally
reaching Running. Zero transports up is now Failed (fatal): start()
tears down cleanly and returns an error, and the daemon exits. Any
configured optional child that failed to start - a transport beyond the
first, Nostr, mDNS, TUN, DNS, or a worker pool - leaves the node
Degraded but serving, with an operator warning naming what failed. All
configured children up is Full. A child the node was never asked to run
does not count against health.
The published NodeState gains Degraded and Failed variants, both visible
via control queries; Degraded is operational, Failed is not. The
lifecycle FSM gains the health states plus the PublishState action that
drives them - a health fork cannot be a single direct state write, which
is why the earlier commits deferred it to here.
Runtime child-exit health re-evaluation (a running child dying) is a
separate liveness-monitoring mechanism left for a follow-up; this commit
is start-time health only.
Add an operator-visible Draining phase on daemon shutdown. On the
shutdown signal the node broadcasts Disconnect to all peers, then keeps
serving for a bounded window - up to node.drain_timeout_secs (default 2s),
exiting early once all peers are gone - before tearing down. This lets
in-flight traffic settle and peers observe the disconnect before the
transports close, rather than the previous immediate teardown.
The lifecycle FSM gains a Draining state plus Drain/DrainDeadlineElapsed
events; the run loop observes the shutdown signal and transitions to
draining in place - one continuous loop, so the channel receivers are
never destructively cancelled. The published NodeState gains a Draining
variant, visible via control queries during the window. The immediate
stop() path used by tests and non-daemon callers is unchanged: it still
tears down immediately with no drain wait.
The reconciler-gate actions the drain emits are no-ops until the peering
reconciler lands and consumes them.
Introduce a sans-IO lifecycle supervisor: a synchronous step(event) ->
[action] state machine (SupervisorFsm) that authors the substrate-child
spawn and teardown order, plus an owner struct (Supervisor) holding the
substrate runtime fields and embedding the FSM. The substrate-lifecycle
fields leave Node's flat list into the owner - state, packet_tx, the TUN
reader/writer handles + shutdown fd + channels, the DNS task + identity
channel, the Nostr/LAN rendezvous drivers, and the encrypt/decrypt worker
pools; the dataplane keeps packet_rx.
start()/stop() become the driver executing the FSM's SpawnChild/StopChild
actions: same children, same order, same warn/debug-and-continue on
optional failures, same logs, same NodeState transitions. Behavior- and
wire-neutral. The only determinism change is that transports now tear
down in ascending-id order, previously nondeterministic HashMap iteration.
The bounded Draining phase and the Running{Full|Degraded} health split
land as separate follow-on commits.
Reorganize the node module tree by concept rather than by
message-handling verb, as the first step of the node runtime
decomposition. Pure relocation: no wire, config, metric, or log
semantics change; the lib test count is unchanged (1577 passed).
Moves (git mv, 100% rename similarity):
- handlers/{forwarding,rx_loop,connected_udp,dispatch,encrypted}.rs
-> node/dataplane/ — the whole RX hot path (the select! run loop,
transit/local forwarding, the link-message router, the RX decrypt
path with responder K-bit cutover + roam writes, and connected-UDP
fast-path activation) now lives in one home.
- node/session.rs -> node/session/mod.rs — establishes the session
concept home for the data/state types. The message-behavior file
handlers/session.rs stays put for now (folds in with the later FSP
session step).
The IK/XX-divergent establishment files (handlers/{handshake,rekey,
timeout}.rs) and the deferred-home files (handlers/{mmp,lookup}.rs)
deliberately stay in handlers/, to move once rather than twice.
Every module is reached through impl Node methods, so no call site or
re-export shim was needed. Updated in lockstep with the moves: the
module_path!-derived tracing targets in the two mesh-lab compose-trace
overlays, a structural test's include_str! source path, doc-comments
in proto/routing and the mesh-lab docs, and the stale source-location
citations (node/handlers/{forwarding,rx_loop,encrypted}.rs and
node/session.rs) in doc-comments and the discovery design doc.
Forward-merge three maint fixes, hand-relocated into master's
post-sans-IO / discovery-to-lookup structure:
- drop the redundant TreeMetrics parent_switched counter (keep
parent_switches), reconciled across the refactored tree/mmp sites and
the spanning-tree test reads
- keep the tighter path_mtu when applying a LookupResponse, now in
handlers/lookup.rs after the discovery module rename
- reuse one shared secp256k1 context in the identity module
Every sign/verify/key-derive site built a fresh context via
Secp256k1::new(), which allocates a Secp256k1<All> and runs
randomization/blinding table setup on each call. Introduce one
crate-wide LazyLock<Secp256k1<All>> and reuse it across the local, peer,
and auth sites (and their tests). Behavior-neutral: identical secp256k1
API calls, only the context lifetime changes, and the shared All context
still performs the standard construction-time blinding.
An originator handling a LookupResponse unconditionally overwrote the
cached path_mtu_lookup entry, so a looser (larger) estimate in a later
response could clobber a tighter value already learned from a reactive
MtuExceeded or PathMtuNotification. Read-and-compare before writing and
keep the minimum, so a looser discovery estimate no longer loosens the
clamp. Add a regression test.
parent_switched was incremented on the line immediately before
parent_switches at every site and never independently, so the two
counters were always identical. Drop parent_switched from TreeMetrics,
its snapshot, TreeStatsSnapshot, the show_tree fixture, and the fipstop
render, keeping parent_switches as the sole counter.
The routing-stats pane's Discovery Requests/Responses sections show the FMP
overlay coordinate-lookup counters. Rename the section labels and the nested
JSON key they read from "discovery" to "lookup" to match the metric family's
canonical name. The daemon dual-emits both keys, so this reads the current
name and no longer depends on the deprecated "discovery" alias.
Rename the Ethernet per-interface config flag from discovery to listen,
so the receive/transmit toggle pair reads as the symmetric announce
(transmit) / listen (receive) neighbor-beacon vocabulary. The old
discovery: key is still accepted via a serde alias, so deployed configs
load unchanged; to_yaml re-emits it under the canonical listen: name.
Marked deprecated for removal at the v2 cutover.
Updates the config field + accessor, the transport listen_enabled local,
the one struct-literal test consumer, the chaos sim config generator,
packaged fips.yaml examples, and the classified operator-facing docs
(ethernet neighbor-beacon subsystem prose; the generic Transport
discovery capability prose is left unchanged). Adds a compat parse test
asserting the legacy alias, the new key, and that deny_unknown_fields
still rejects unknown keys. Behavior-neutral.
Mirror the ethernet neighbor rename in the BLE transport: the internal
peer-detection buffer becomes NeighborBuffer (from DiscoveryBuffer) and
the module is renamed discovery -> neighbor. The BlueR/bluez API terms
(DiscoveryFilter, set_discovery_filter) and the BLE advertise/scan
mechanism vocabulary are unchanged, as is BleConfig. Behavior-neutral.
Rename the ethernet link-local neighbor-beacon subsystem from the
overloaded "discovery" vocabulary to a neighbor umbrella. The beacon
frame vocabulary is retained (build_beacon/parse_beacon/BEACON_SIZE/
FRAME_TYPE_BEACON/FRAME_TYPE_DATA); only the subsystem and buffer
identifiers change: DiscoveryBuffer becomes NeighborBuffer,
discovery_buffer becomes neighbor_buffer, and DISCOVERY_VERSION becomes
BEACON_VERSION (wire value 0x01 unchanged). Config toggles are left for
a follow-up commit. Behavior-neutral.
The module is gated target_os="macos" (not the broader Darwin/iOS family),
so the name now tracks the cfg and matches the *_macos.rs file convention
(io_macos.rs). Drops the redundant inner #![cfg(target_os="macos")] (the
decl gate already covers it) and corrects a stale top comment that claimed
the module stays visible on Linux — it is macos-decl-gated, so it never was.
Mechanical rename; no logic change.
ConnectedPeerSocket and PeerRecvDrain are node/peer-side logic: the Transport
trait never touches them, ActivePeer stores them, and node's encrypt worker
drives them. They only happened to live under transport/udp. Relocate the
handle types to a new src/peer/connected_udp/ module (socket.rs + drain.rs),
which the node handler and encrypt worker reach via node -> peer (no new edge;
a node home would have forced a peer -> node cycle).
The udp transport keeps only the kernel-construction seam: open_connected_fd,
now folded into udp/io.rs (the byte-layer home) behind a linux/macos-gated
submodule and re-exported as transport::udp::open_connected_fd. The node
handler builds the fd through it and adopts it via ConnectedPeerSocket::from_fd.
Behavior-neutral: no wire/config/metric/log change; the per-packet hot path
(bare-RawFd send_batch_gso/raw) is untouched. Preserves the Arc multi-owner
contract, the drop-drain-before-socket ordering, and the drain's detach-on-Drop
deadlock avoidance. Reconciles the old cfg(unix)/any(linux,macos) double-gate
onto the single any(linux,macos) predicate.
Pulls the connected-UDP socket construction (socket/REUSEADDR/REUSEPORT/
BUFFORCE/bind/connect + darwin tuning) out of ConnectedPeerSocket::open into
a pub(crate) open_connected_fd returning an OwnedFd; open() now delegates and
adopts the fd. Error paths still close the fd via the temporary's Drop; the
success tail transfers ownership through OwnedFd. Behavior identical. Prepares
the fast-path handle types to move to the peer module while the socket
construction stays with the udp transport.
Moves the inline TcpConnection/ConnectingEntry structs, the Direction
enum, and the ConnectionPool/ConnectingPool type aliases out of the
1147-line tcp/mod.rs into a dedicated pool.rs, matching the canonical
per-transport layout. Struct fields are pub(crate) so mod.rs can still
construct and read them across the module boundary. Pure relocation; no
logic change.
Moves parse_mac_string into ethernet/addr.rs and re-exports it at the
ethernet module root so the ethernet::parse_mac_string path stays
byte-identical for its one external consumer (zero churn there). Its
unit tests stay in mod.rs, calling through the re-export. Pure
relocation; no logic change.
Normalizes the ethernet transport onto the canonical byte-layer name
(io.rs), including the per-OS files io_linux.rs/io_macos.rs and their
#[path] wiring. Pure file rename plus module-path and doc-comment
updates; no logic, wire, config, metric, or log change.
Normalizes the udp transport onto the canonical per-transport module
layout (ble is the reference: io/pool/addr/stats). Pure file rename plus
module-path updates at the use sites; no logic, wire, config, metric, or
log change. Behavior identical.
Johnathan Corgan