The lifecycle view's doc described the shell as implementing it over a
`connections` map that no longer exists. The establish view named that map
too, and was doubly wrong: both of its snapshot builders read only
`peers`. Rewrite both to describe the maps actually read.
Several comments in the node module carried shorthand from private
working notes - bare parenthetical labels, and one reference to an
internal tracker item - none of which mean anything to a reader of this
repo. Rewrite them so each clause states its own reasoning inline. The
paragraph above the tracker reference already carried the rationale, so
that one is simply dropped.
Comment-only; no behavior change.
The per-peer control machine has absorbed every field the pending
connection carried. What remained was a struct holding two Noise
handles beside a duplicate copy of bookkeeping nobody read. Replace it
with a small carrier for the two handles and delete the type.
Presence of that carrier, not the state of the handles inside it, is
what marks a machine as mid-handshake. The distinction is essential
rather than stylistic: a failed handshake drops its initiation handle
and is deliberately retained so the stale sweep can reclaim it, and a
completed one has its session taken before disposal. Deriving presence
from the handles would make both invisible to the sweep, the
connection count, and the peering budget at once, leaking the slot
permanently. A test drives an empty carrier past every presence
predicate and then detaches it, so a future edit cannot quietly couple
the two.
The remote startup epoch now comes from the surviving carrier, which
the handshake operations already wrote at the same two points with the
same value. The paired writes onto the pending connection's own
bookkeeping had no readers left and are gone.
The handshake-phase surface leaves the public API: it was public by
accident rather than design, and the machine behind it is crate
internal. Callers outside the crate that need a view of pending
handshakes go through the operator queries, which are unchanged.
ConnectionState::inbound_with_transport loses its last non-test caller
with the inbound seed and is marked test-only.
The pending connection and the per-peer control machine have carried
duplicate copies of the handshake-phase fields since the machine gained
its own connection state. Read them from the machine and drop the
connection's projections.
The peer identity is the sharp one. The connection learned it from msg1
and the machine's carrier did not, so the two views genuinely disagreed
for inbound connections until the Noise operations moved onto the
machine and began recording each result on both. That is now in place,
so every reader can take the machine's copy unchanged: the establish
snapshot, the promotion sweep for competing connections, the dial and
path in-progress checks, the peering observations and per-peer in-flight
budget, and the stale-connection sweep's retry address.
Also repointed: our_index, their_index, transport_id, started_at, the
stored handshake message bytes, and the idle-timeout check. Two
duplicate index writes on the connection are dropped, both immediately
preceded by the machine-side write of the same value. Reads that
previously came off a connection detached just before its machine was
disposed now capture the machine's value first.
Test seeding follows the establish paths: the seed builders write
our_index to the carrier, which promotion now reads. A new test pins
the inbound identity learn on the carrier and the retry address a
failed inbound connection reports to the sweep, so a silent regression
to a blank identity cannot pass.
ConnectionState::duration loses its last non-test caller with the
connection accessor and is marked test-only.
These three had no counterpart on the control machine, so readers still
reached them through the pending connection. Add machine-side
accessors and repoint every reader, then drop the connection's.
The peer address needed its writes lifted, not just its reads
repointed: the machine's copy was never written. It is now written at
each of the points the connection's copy was — the inbound seed, the
dial, message-2 completion, and the two paths that seed a machine from
a pre-built connection — so promotion and the resend path read a value
with the same provenance at the same time as before.
Link and direction need no lift. Both machine constructors already
seed them from the same arguments the connection is built with, so an
outbound machine carries outbound state and an inbound one inbound,
and the machine's link always equals the connection's. The handshake
operations' direction guards read the machine's copy for the same
reason.
The stale-connection sweep's teardown log and its resend path both now
take the transport and address from the machine, and each keeps its own
check that a pending connection is still attached rather than relying
on the caller to have established it.
Add a test pinning link, direction, and address on the two shapes that
seed a carrier independently — the dial and an accepted message 1. The
cross-connection winner reads both values from a carrier one of those
two already seeded.
`Node::connections()` yielded the pending connection itself, so its
consumers reached the handshake-phase fields through that value. The
pending connection is being folded into the control machine, and the
fields will move off it, so yield the machine (keyed by its link) and
let each consumer reach what it needs from there.
Membership is unchanged: the iterator still selects machines that
carry a pending connection, which is the predicate `connection_count`
and the stale-connection sweep already use. Every consumer takes the
same value from the same place, one hop further out, and no field
changes carrier here.
The method is now internal to the crate. It yields the control
machine, which is not part of the published surface, and nothing
outside the crate iterates pending connections — operator views of
the handshake phase go through the query surface instead.
Tests built a free-standing PeerConnection, mutated it, and handed it to
Node::add_connection by value. None of those sites survives the removal of
PeerConnection, so converting them afterwards would mean one enormous commit
that cannot be reviewed honestly. Convert them now, while the struct still
exists and the conversion can be validated against a green tree.
Adds a cfg(test) Node::seed_handshake_machine plus a HandshakeSeed builder,
and rewrites make_completed_connection (now seed_completed_connection) and
the twenty inline builders onto it. add_connection keeps its body and loses
its test callers.
The builder's carrier seeding is a verbatim copy of add_connection's: the two
conditional writes for their_index and transport_id, then set_leg, through the
same entry().or_insert_with() so an existing leg-less machine keeps its
constructor-side fields. Nothing else reaches the carrier -- our_index,
source_addr, post-construction started_at and the stored handshake bytes stay
leg-only. Seeding more than that would let these tests observe a carrier
richer than production's and keep passing even if a later production
write-lift were missed.
The Noise exchange now runs on the already-seeded leg rather than before the
hand-over. That is neutral because the only read of expected_identity is
guarded by is_outbound, and no crypto method allocates a session index.
Also adds a compile-time check that PeerAction is Clone + Eq, which is what
keeps a runtime handle from being smuggled into an action payload.
The connection leg's their_index, transport_id and link_stats duplicated the peer
machine's own connection-state copy. Route every production reader through the
machine copy: promotion, the stale-connection reaper, the outbound msg1 resend, and
the in-use / connecting-path checks. Seed the inbound machine's transport_id from the
msg1 packet (the same source the leg used), matching the outbound dial seed, so the
hard-required promote read never sees a missing transport. link_stats is a
never-mutated zero seed, so its leg accessors are removed outright; the leg
their_index/transport_id getters/setters remain only for the test-only connection
builder. Also delete the write-only next_resend_at_ms field from the connection
state (the resend deadline is carried by the machine-armed retransmit timer, not this
field). Byte-identical.
The expected_peer connection-row field read the peer identity off the leg;
source it from the machine's own connection-state copy instead, matching the
started_at/last_activity treatment. The machine carrier holds the same identity
as the leg for every connection that appears in the view: outbound legs seed
both from the dialed identity, and inbound legs never rest in the connections
view (the machine takes the leg at promotion), so their machine copy is
unobserved. The leg accessor stays for the crypto-extraction path and retry
decisions. Byte-identical.
The connection leg's started_at and last_activity duplicated the peer
machine's own connection-state copy. Make the machine copy the sole live
telemetry and timeout carrier: re-stamp it with the leg's provenance when
the leg is attached at handshake start (the msg1-prep clock, not the earlier
dial-time constructor value) and mirror the completion touch, then delete the
leg's last_activity/touch accessors and source the connection-row projection,
show_connections, and the timeout readers from the machine. Byte-identical
for all normal paths.
The connection leg's HandshakeState field duplicated the peer machine's
handshake phase. Delete it and derive the displayed handshake state string
from the machine's PeerState, looked up by link id (mirroring the resend
count). Move the failure signal onto the machine: a send_failed flag that
preserves retransmit eligibility (the machine stays in its handshake phase),
alongside the existing Failed state. The leg's crypto self-gates now guard on
Noise handle presence instead of the deleted phase field, and mark_failed
only drops the handle. Telemetry strings, wire bytes, index allocation, and
the stale-connection reaping are byte-identical for all normal paths.
Make the FIPS core build and run as an embedded Android library. The host
app owns the TUN (e.g. an Android VpnService) and FIPS performs no
system-TUN or CAP_NET_ADMIN operations.
Squashed from the following changes:
- gate desktop transports/TUN by target_os, not features: a plain
`cargo build` now compiles for every target with no flags. Ethernet (raw
AF_PACKET / BPF) is gated to linux/macos, so Android (target_os =
"android", not "linux") self-excludes it as Windows already did; real
system-TUN ops are gated per linux/macos and Android gets a no-op stub;
the ipi6_ifindex cast handles it being i32 on Android vs u32 on macOS. No
Cargo features are introduced; desktop builds are unchanged.
- app-owned TUN seam: Node::enable_app_owned_tun() lets an embedder that
owns the TUN fd exchange IPv6 packet bytes with FIPS over channels
instead of FIPS creating a system TUN device. It returns (app_outbound_tx,
app_inbound_rx): the embedder pushes packets read from its fd into the
outbound sender (app -> mesh) and pulls packets destined for its fd from
the inbound receiver (mesh -> app). start() gates system-TUN creation on
tun_tx being unset, so with the channels pre-installed it skips device
creation and does no system-TUN ops; both directions reuse the existing
inbound-shim and run_rx_loop wiring. Packets entering via app_outbound_tx
bypass handle_tun_packet, so the embedder must push only fd00::/8-destined
packets and clamp TCP MSS on outbound SYNs; the rustdoc and the
IPv6-adapter design doc spell this out.
- keep the android target warning-clean so the cross-compile check passes
clippy -D warnings.
- add an Android cross-compile CI check: cross-compile the library for
aarch64-linux-android via cargo-ndk and run clippy -D warnings. Android
ships as an embedded library (the host app owns the TUN), so there is no
daemon binary to package; this is a check job, not a packaging one.
- docs: list Android as a supported platform.
Tests: app_owned_tun_seam_wires_channels covers the channel round-trip and
the Active state; start_skips_system_tun_when_app_owned runs start() and
asserts no named system device is created.
The pending-handshake PeerConnection map and the per-peer control
machine map were parallel LinkId-keyed structures whose keysets must
stay coherent by hand. With every leg now born with a machine, the leg
becomes storage inside its machine (leg: Option<PeerConnection>, pure
storage the machine never reads or drives) and Node loses the
connections field; every access routes through the machine.
The non-mechanical lowerings, each argued at the site: the
rekey-vs-establish gate in handle_msg2 tests leg-absence (an established
peer's machine stays keyed by its link, so machine-presence would
misclassify every rekey msg2 as a fresh establish); the
connecting-predicates, peering observation, and handshake-slot budget
iterate machines-with-legs so connect-window machines (leg not yet
born) are excluded exactly as before and never double-counted against
their pending-connect slot; the cross-connection extract takes the leg
before disposing the machine; the stale reaper takes the leg and leaves
the machine untouched when none is present, matching the old early
return. The map-coherence debug check keeps its machine-has-carrier
direction with the embedded leg as a carrier; the leg-to-machine
direction is now true by construction and its gate const is gone.
connection_count() counts machines with legs; the connections()
iterator, the test seams, and the control-socket connection rows are
re-implemented over the embedded legs with unchanged output.
Assert, once per tick in debug builds, that the peer control-machine map
and its carriers stay coherent: every control machine has a live
handshake leg, an active peer, or a pending connect (a machine with none
is a leak the stale reaper can never see), and every handshake leg has a
machine. The second direction sits behind a file-local const so a branch
where handshake-window legs legitimately run machine-less can gate it
off without weakening the leak tripwire.
Teach the add_connection test seam to seed a control machine for the leg
it inserts (derived from the connection's direction and identity), and
remove_connection to dispose it, so the seam-built topologies satisfy
the check. Three unit tests: seam coherence, coherence through a real
promotion, and the orphaned-machine panic.
Several module and field docs still described the per-peer machine as
unwired shadow scaffolding. It has been live for some time: machines are
inserted at dial and inbound msg1, stepped by the handshake handlers and
the rekey-cadence and liveness-reap routers, and the executor's
SwapSendState/CompleteDrain/InvalidateSendState arms are the authoritative
paths (the inline bodies survive only as debug-assert release fallbacks).
Rewrite those docs to the current truth while keeping the still-true
dormancy facts: PeerEvent::Timeout and PeerEvent::Tick are never
dispatched in production, retransmit fires on the machine-armed deadline
while the timeout reaper keys on timer presence with the config
threshold, and the remaining inert executor stubs are SendRekey,
SendLinkMessage, and the connected-UDP arms. Drop the stale
allow(dead_code) on the peer_machines field.
Move the outbound msg1 resend off the unconditional tick function onto the
machine-armed retransmit timer. The per-peer machine already arms a
HandshakeRetransmit deadline at dial; a new drive_peer_timers fires the due ones
(kind-filtered to the retransmit timer) and homes the resend counter on the
machine, where the operator-visible count now reads from.
The resend decision reads the same operator config as before (interval, backoff,
max), the wire bytes and transport target still come from the shell connection,
and the pure core computes the backoff schedule. As with the deleted
resend_pending_handshakes, the count and reschedule advance only on a successful
send: a failed send neither advances the count nor marks the connection failed,
it just retries on the next tick. The handshake-timeout timer stays on the legacy
check_timeouts path, and the rekey/liveness timers keep their own shell drivers,
so drive_peer_timers deliberately fires only the retransmit kind.
The show_connections resend count is relocated to read from the machine (the
counter's new home) via connection_resend_count; machine-less and inbound
connections report 0, matching what the shell connection reported before. Delete
resend_pending_handshakes and resend_candidates (the latter also from the
LifecycleView trait, its only user). The resend unit test is re-expressed against
the machine + timer path, covering the due check and the record-on-success
semantics.
Known limitation: the first resend interval is armed from the machine's hardcoded
1000ms constant, which equals the config default. Under an operator override of
handshake_resend_interval_ms the first resend diverges from the pre-change
dial+interval; subsequent resends and the cap remain config-driven. Neutralizing
the first-resend override needs the interval threaded into the sync core (the
shell arm would be clobbered by the machine's own timer arming at dial), deferred
to when the connection and machine entities merge. Test count unchanged at 1631.
The control machine already emits SetTimer/CancelTimer actions when it arms the
handshake retransmit and timeout deadlines, but their executor arms were a
single no-op stub and the machine's Timeout event was never dispatched -- the
real work still runs on the legacy tick (check_timeouts,
resend_pending_handshakes).
Add the storage the time-as-input driver will read: a peer_timers map keyed by
LinkId then TimerKind, holding each armed timer's absolute deadline. The SetTimer
arm now inserts (overwrite = reschedule) and CancelTimer removes; the outbound
msg2 promote cancels the two dial-armed handshake timers, since the machine
survives promotion and the entries would otherwise linger in the store.
This store is a shadow: it is written and cleared but no driver reads it yet, so
behavior is unchanged. The legacy tick stays authoritative until the driver that
feeds Timeout and deletes the overlapping tick paths lands.
Route every machine removal through a new remove_peer_machine(link) choke-point
that drops the timer store alongside the machine, replacing the twelve direct
peer_machines.remove sites so no armed timer outlives its machine. TimerKind
gains Hash (to key the store) and Ord (for deterministic driver collection); it
is internal and never serialized. Test count unchanged at 1631.
Comments across the per-peer machine, executor, lifecycle supervisor,
and peering reconciler carried internal rollout labels and design-note
section references. Rewrite them to describe the code's behavior
directly. Comment-text only; no code changes.
Add Node.peer_machines (a LinkId-keyed map from the stable link handle to
the per-peer control machine) as the home for the machines, and a new
dataplane/peer_actions.rs holding execute_peer_actions / advance_peer_machine:
the executor that maps each PeerAction the machine emits to its shell call —
frame and send a handshake via build_msg2, drive promote_connection and feed
the PromotionResult back through the machine, tear a peer down via
remove_active_peer, free session indices, report loss via note_link_dead.
Actions for the rekey, connected-UDP, and timer paths are stubbed with notes
for the commits that fold those mechanisms in.
Unwired: nothing drives the machine yet — no live handler path calls the
executor and peer_machines is never populated — so this is behavior-neutral;
the inbound and outbound establish paths still run their existing inline
logic. The executor is cut over path-by-path in the following commits.
Wire exit detection for the four directly-observable optional children so
the supervisor FSM's ChildExited edge fires at runtime. A runtime
child-liveness mpsc channel carries a Child on exit: the DNS task and the
two TUN threads self-report when their body returns, and a 2s poll monitor
reports mDNS and Nostr via new is_finished accessors. The rx_loop gains a
select arm that steps the FSM and republishes health (Degraded, since a
running node always has at least one transport up). Transports and worker
pools expose no runtime-exit signal and are left for a follow-up.
Adds LanRendezvous::is_finished and NostrRendezvous::is_finished; the
latter treats a shutdown-taken connect_task as finished so the monitor
terminates after a stop instead of polling forever.
Move retry_pending and pending_connects from Node into the Peering owner
struct (Node.peering), the home introduced with the reconciler core.
Pure mechanical relocation: the two collections now live behind
self.peering, with roughly seventy accessor sites repointed. No decision
logic is touched and the unit-test count is unchanged (1607). The
imperative peering methods still run; the reconciler is wired in the
following commit.
Homing the retry schedule under peering/ lengthened the RetryState
module path, pushing the retry_state_iter return-type line past the
100-column max_width. Rewrap to satisfy rustfmt. No behavior change.
Create src/node/peering/ as the home for the peer desired-state
(homeostatic reconciler) concept and move the cross-attempt connection
retry schedule into it: RetryState plus schedule_retry /
schedule_reconnect / process_pending_retries.
Mechanical relocation only. The methods remain inherent on Node; the
retry_count must persist across re-dials (a fresh connection is created
each attempt), which is why the schedule belongs in the peering home
rather than on a per-connection type. The one-level-deeper module path
requires pub(super) -> pub(in crate::node) to preserve the prior scope,
plus module-path fixups at the reference sites. No behavior change; unit
test count unchanged (1596).
Determine node health at start completion instead of unconditionally
reaching Running. Zero transports up is now Failed (fatal): start()
tears down cleanly and returns an error, and the daemon exits. Any
configured optional child that failed to start - a transport beyond the
first, Nostr, mDNS, TUN, DNS, or a worker pool - leaves the node
Degraded but serving, with an operator warning naming what failed. All
configured children up is Full. A child the node was never asked to run
does not count against health.
The published NodeState gains Degraded and Failed variants, both visible
via control queries; Degraded is operational, Failed is not. The
lifecycle FSM gains the health states plus the PublishState action that
drives them - a health fork cannot be a single direct state write, which
is why the earlier commits deferred it to here.
Runtime child-exit health re-evaluation (a running child dying) is a
separate liveness-monitoring mechanism left for a follow-up; this commit
is start-time health only.
Add an operator-visible Draining phase on daemon shutdown. On the
shutdown signal the node broadcasts Disconnect to all peers, then keeps
serving for a bounded window - up to node.drain_timeout_secs (default 2s),
exiting early once all peers are gone - before tearing down. This lets
in-flight traffic settle and peers observe the disconnect before the
transports close, rather than the previous immediate teardown.
The lifecycle FSM gains a Draining state plus Drain/DrainDeadlineElapsed
events; the run loop observes the shutdown signal and transitions to
draining in place - one continuous loop, so the channel receivers are
never destructively cancelled. The published NodeState gains a Draining
variant, visible via control queries during the window. The immediate
stop() path used by tests and non-daemon callers is unchanged: it still
tears down immediately with no drain wait.
The reconciler-gate actions the drain emits are no-ops until the peering
reconciler lands and consumes them.
Introduce a sans-IO lifecycle supervisor: a synchronous step(event) ->
[action] state machine (SupervisorFsm) that authors the substrate-child
spawn and teardown order, plus an owner struct (Supervisor) holding the
substrate runtime fields and embedding the FSM. The substrate-lifecycle
fields leave Node's flat list into the owner - state, packet_tx, the TUN
reader/writer handles + shutdown fd + channels, the DNS task + identity
channel, the Nostr/LAN rendezvous drivers, and the encrypt/decrypt worker
pools; the dataplane keeps packet_rx.
start()/stop() become the driver executing the FSM's SpawnChild/StopChild
actions: same children, same order, same warn/debug-and-continue on
optional failures, same logs, same NodeState transitions. Behavior- and
wire-neutral. The only determinism change is that transports now tear
down in ascending-id order, previously nondeterministic HashMap iteration.
The bounded Draining phase and the Running{Full|Degraded} health split
land as separate follow-on commits.
Reorganize the node module tree by concept rather than by
message-handling verb, as the first step of the node runtime
decomposition. Pure relocation: no wire, config, metric, or log
semantics change; the lib test count is unchanged (1577 passed).
Moves (git mv, 100% rename similarity):
- handlers/{forwarding,rx_loop,connected_udp,dispatch,encrypted}.rs
-> node/dataplane/ — the whole RX hot path (the select! run loop,
transit/local forwarding, the link-message router, the RX decrypt
path with responder K-bit cutover + roam writes, and connected-UDP
fast-path activation) now lives in one home.
- node/session.rs -> node/session/mod.rs — establishes the session
concept home for the data/state types. The message-behavior file
handlers/session.rs stays put for now (folds in with the later FSP
session step).
The IK/XX-divergent establishment files (handlers/{handshake,rekey,
timeout}.rs) and the deferred-home files (handlers/{mmp,lookup}.rs)
deliberately stay in handlers/, to move once rather than twice.
Every module is reached through impl Node methods, so no call site or
re-export shim was needed. Updated in lockstep with the moves: the
module_path!-derived tracing targets in the two mesh-lab compose-trace
overlays, a structural test's include_str! source path, doc-comments
in proto/routing and the mesh-lab docs, and the stale source-location
citations (node/handlers/{forwarding,rx_loop,encrypted}.rs and
node/session.rs) in doc-comments and the discovery design doc.
Move the FMP mesh-layer wire format (common prefix, encrypted/msg1/msg2
headers, and the build_*/inner-header codec fns) out of node/wire.rs and
into proto/fmp/wire.rs, so the whole FMP wire surface lives with its
subsystem, matching the proto/fsp/wire.rs layout. The wire module becomes
pub(crate) mod wire; callers reach it via crate::proto::fmp::wire.
Behavior-neutral: pure relocation plus import-path rewrites across the
node/peer consumers; no logic change. Full lib suite green at baseline.
Pull the rendezvous driver state and the movable driver logic out of the
Node struct into the src/nostr home. A new RendezvousDriver owns the
engine handle and the four bookkeeping fields (traversal start time,
startup-sweep latch, adopted bootstrap-transport set and their npubs)
that previously sat loose on Node; Node holds a single driver field and
reaches the same data through thin accessors, including the rx-loop
hot-path protocol-mismatch hook.
The advert build/refresh, the via_nostr fallback-address resolve, the
overlay-endpoint-to-PeerAddress mapping, and the bootstrap request move
onto the driver, taking their Node inputs explicitly (a transport-
endpoint snapshot for advert building) rather than reading Node fields
directly. Transport/connection-table-bound work (traversal adoption,
bootstrap-transport cleanup, the open-discovery sweep, and the outbound
budget calculators) stays on Node as thin glue that calls into the
driver.
Behavior-neutral relocation: statements moved verbatim, no logic, wire,
config-key, or metric changes. lifecycle.rs shrinks ~230 lines. cargo
fmt/build/clippy clean; lib suite 1547 passing (baseline unchanged).
Relocate the overlay peer-rendezvous subsystem out of the overloaded
src/discovery/ tree into two focused, independent homes: src/nostr/
(relay-mediated overlay endpoint advertise/resolve/auto-mesh plus NAT
traversal) and src/mdns/ (link-local DNS-SD rendezvous). The two
subsystems are independent, so they get separate homes rather than
sharing one.
Drop the ambiguous "Discovery" stem from their identifiers in favor of
"Rendezvous": NostrDiscovery -> NostrRendezvous, LanDiscovery ->
LanRendezvous, and the matching config, policy, field, and method names.
The former src/discovery.rs handoff types (EstablishedTraversal,
BootstrapHandoffResult, the punch-packet helpers) fold into
src/nostr/handoff and stay reachable via the crate-root re-exports.
Pure relocation and rename: no logic, wire-format, config-key, metric,
or tracing-target changes. The operator-facing node.rendezvous.nostr.*
and node.rendezvous.lan.* config keys and the fips-overlay-v1 advert
namespace are byte-identical. cargo fmt/build/clippy clean; lib test
suite 1547 passing (baseline unchanged).
The identifier "discovery" named three unrelated subsystems; the FMP
overlay coordinate-lookup subsystem is now consistently "lookup". This
finishes the concept-#1 rename across the shell, config, and metric
layers left after the earlier proto-layer rename:
- Handler module node::handlers::discovery -> node::handlers::lookup, and
reset_discovery_backoff -> reset_lookup_backoff.
- The Node lookup-engine field Node.discovery -> Node.lookup, renamed by
resolved binding so the metrics().discovery and node.discovery config
paths are left untouched.
- The lookup metric types DiscoveryMetrics -> LookupMetrics,
DiscoveryStatsSnapshot -> LookupStatsSnapshot, and Metrics.discovery ->
Metrics.lookup.
Two surfaces cross a stability boundary and ship behind a compatibility
window, both marked in-code for removal at the v2 cutover:
- The control-socket metric family is dual-emitted under both "discovery"
(deprecated alias) and "lookup" so existing dashboards keep working.
- The node.discovery.* config table is split into node.lookup.* (mesh
lookup scalars) and node.rendezvous.* (nostr/lan peer rendezvous).
NodeConfig does not deny unknown fields, so a naive rename would make a
deployed node.discovery: block deserialize into nothing and silently
revert every setting to default. A deprecated all-Option
DiscoveryConfigCompat field captures a legacy block and a new post-parse
Config::normalize_deprecated_keys pass folds it into the new tables with
a one-time deprecation warning.
Flip the packaged fips.yaml templates to the new keys, add legacy/new/
scalar compat parse tests, and record the split and deprecations in the
CHANGELOG. Behavior-neutral; fmt/clippy clean, lib suite green.
Rename src/proto/discovery to src/proto/lookup and bring the module's naming
onto the lookup stem, matching its concept: a mesh lookup of a node's
coordinates from its pubkey, sent into the mesh as a bloom-filter-guided
multicast request that returns a unicast response with the coordinates.
- the module directory and its declaration (proto::discovery -> proto::lookup)
- exported types: DiscoveryAction -> LookupAction, DiscoveryBackoff ->
LookupBackoff, DiscoveryForwardRateLimiter -> LookupForwardRateLimiter,
MAX_RECENT_DISCOVERY_REQUESTS -> MAX_RECENT_LOOKUP_REQUESTS, and the Discovery
state struct -> Lookup
- internal terminology: doc comments, the "overlay-lookup" phrasing, the
empty_discovery/suppressing_discovery test helpers, and the disc
parameter/variable all take the lookup names
- the already-lookup-named wire types (LookupRequest/LookupResponse) are unchanged
Behavior-neutral: no wire bytes or decision logic change. Two references are
intentionally kept as "discovery": the still-named node::handlers::discovery
shell module and the node.discovery.* config keys, which belong to the broader
disambiguation of the shell, config, and metric surfaces still to come.
Migrate the FSP end-to-end session subsystem into src/proto/fsp/ following the
established sans-IO shape, and retire the src/protocol grab-bag now that FSP was
its last occupant.
Relocate the FSP session wire (node/session_wire.rs plus the FSP message types
from protocol/session.rs) into proto/fsp/wire.rs. Hoist the pure decision logic
into proto/fsp/core.rs over plain-data SessionSnapshots returning an ordered
FspAction list the shell drives: session-rekey policy, msg3-resend
classification, post-decrypt epoch reaction, setup/dual-init tie-break,
coords/path-MTU emit-policy, bounded pending-queue, and IPv6 ECN. The
crypto-owning SessionEntry stays shell-side in node/session.rs (matching the FMP
ActivePeer pattern); proto/fsp is wire + core + limits only, with no proto->noise
dependency and no crypto.
Move the coords helpers to proto/stp/ (they serialize TreeCoordinate), and split
SessionMessageType: the encrypted-inner 0x10-0x1F variants stay in proto/fsp/wire.rs
while the 0x20-0x2F routing signals become a new RoutingSignalType in
proto/routing/wire.rs. Migrate the session-MMP shell adapter, which continues to
drive proto/mmp/.
Retire src/protocol: LinkMessageType and SessionDatagram move to a new shared
proto/link.rs, ProtocolError becomes proto::Error (relocated verbatim), the
deprecated MessageType alias and the unimported PROTOCOL_VERSION are dropped, and
src/protocol/ is deleted along with its lib.rs module declaration.
Behavior-neutral: wire bytes unchanged, oracle tests pass unedited except
mod-path relocation; adds rekey/epoch characterization tests and pure
poll/emit-policy core tests.
Migrate the v1 bloom filter subsystem into src/proto/bloom/, matching the
discovery/routing/fmp/mmp/stp reference layout and completing the proto/
relocation series for the data/wire subsystems.
- wire.rs: the FilterAnnounce (0x20) codec, moved from protocol/filter.rs
- core.rs: the pure BloomFilter algorithm (hash/insert/contains/merge/
as_bytes/from_bytes/estimated_count), moved from bloom/filter.rs
- state.rs: BloomState (per-peer inbound store, compute_outgoing_filter,
the injected-clock send debounce), moved from bloom/state.rs
- limits.rs: the v1 sizing constants
- mod.rs: module wiring + the BloomError enum
- tests/: the unit suite split by target (core/state/wire), no inline tests
no_std+alloc hygiene: core::fmt over std::fmt, the tracing dependency
dropped from the pure filter, and std collections replaced with
BTreeMap/BTreeSet (NodeAddr: Ord) for deterministic iteration. The pure
filter combination stays a BloomState method; the two irreducible shell
gathers (peer_inbound_filters, build_filter_announce) remain in the async
shell. Wire bytes and observable behavior are unchanged; full local CI
green (36/36) including the bloom-storm chaos gate.
Migrate the full non-async spanning-tree surface into proto/stp/, mirroring the
discovery/routing/fmp/mmp conversions. The classification ladder (parent-switch /
self-root / loop-drop / ancestry-update / periodic-rebroadcast / parent-lost) moves
out of the async node handlers into a pure Stp classify layer returning a
TreeDecision the shell drives, with effect ordering and per-arm invalidation
preserved verbatim. src/tree/ relocates wholesale: TreeState + ParentDeclaration data
+ coordinates into proto/stp/{state,coordinate}, the flap-dampening / hold-down
cluster into a FlapDampener in limits.rs, and the wire codec into wire.rs. The clock
is injected as u64 (wall-clock secs for the escaping declaration timestamp, monotonic
ms for the dampening timers via mmp::mono_ms); declaration crypto is field-partitioned
so sign/verify/hash run in the shell while the in-core modules carry data +
signing_bytes only. Peer maps/sets move to BTree; core/state/coordinate/limits are
core+alloc clean, with wire.rs the one std-tethered file. Behavior-neutral:
characterization tests added for the handler decision arms; convergence suite and
ci-local (36/36) green.
Migrate the FMP discovery decision logic out of the async handlers into
synchronous, runtime-agnostic sans-IO state machines owned by the protocol
structs, with I/O pushed to the edges. Pulls the full decision surface into a
pure core (backoff, rate-limit, planners, response routing), consolidates the
tests into a per-module tree with a shared crate testutil, and injects a u64
wall-clock so the core is free of Instant and std time.
Also brings the module toward no_std+alloc: the four discovery maps use
alloc::collections::BTreeMap (HashMap's RandomState is std-only), Arc is spelled
alloc::sync::Arc, the backoff-reset log lives in the shell (the core returns the
cleared count so observability stays out of the pure core), and the crate root
names alloc directly. The one remaining tether is ProtocolError's
std::error::Error coupling in the wire codec.
First subsystem of the broader sans-IO refactor; establishes the extraction
patterns and conventions carried forward to the remaining protocols.
Add six forwarding counters that partition transit-forwarded packets by their
tree relationship to the chosen next hop: tree-up (peer is our ancestor),
tree-down (peer is our descendant and the destination is within its subtree),
tree-down-cross (peer is our descendant but the destination is outside its
subtree), cross-link descend (lateral peer, destination within its subtree),
cross-link ascend (lateral peer, destination outside its subtree), and
direct-peer. The six classes sum to forwarded_packets, asserted by a unit
test. Classification is computed from tree coordinates at the transit
chokepoint, so the error-signal routing callers are excluded.
The two "outside the chosen peer's subtree" classes are both up-and-over
forwards but differ in what they depend on. Tree-down-cross is the
dive-to-tree-child cut-through: we forward down to our own child for a
destination not beneath it, which is only possible because the child
advertised cross-link reach upward to us, beyond its own subtree. Its count
measures how much forwarding depends on that upward advertisement, i.e. what
would change if cross-link advertisements were narrowed to subtree-entry only.
Cross-link ascend, by contrast, uses the node's own lateral cross-link learned
from a peer's split-horizon advertisement, so it does not depend on any upward
advertisement.
Surface the counters through the forwarding stats snapshot (control socket,
show_routing and show_status) and reorganize the fipstop routing tab so its
two columns separate own/endpoint traffic (received, delivered, originated)
from forwarded/transit traffic (the route-class breakdown and drop reasons),
with the tree-down-cross line visually flagged.
poll_lan_discovery's comment said Noise XX, but LAN-discovered peers dial over
UDP through initiate_connection, which uses Noise IK (IK at FMP). compute_mesh_size's
header comment still described the obsolete sum-of-disjoint-subtrees estimate;
the function OR-unions every connected peer's inbound filter plus self and
estimates cardinality once (matching the body comment). Comment-only, no
behavior change.
Add an outbound-only Nym mixnet transport that tunnels FMP peer links
through a local nym-socks5-client SOCKS5 proxy into the Nym mixnet. It
structurally mirrors the Tor SOCKS5 transport (connection pool,
connect-on-send background promotion, FMP-v0 framing reused from TCP)
with the onion, inbound-listener, and control-port machinery removed.
Wires the transport through the full TransportHandle dispatch, NymConfig
(standard transport-instance pattern), and node instantiation, and
surfaces its counters in fipstop. Includes a mock SOCKS5 harness and unit
coverage for the address-parsing paths.
Also adds an isolated single-container example
(examples/sidecar-nostr-mixnet-relay/) demonstrating FIPS peering across
the mixnet end to end. No new crate dependencies: tokio_socks, socket2,
and futures are already pulled in by the Tor transport.
Reworks the fipstop TUI across its rendering, the control read surface it
draws from, and its interaction model, on a machine-verified base.
Test infrastructure:
- Add a ratatui TestBackend snapshot harness (testkit + snapshots
modules) that renders any ui::draw_* into an in-memory Buffer from
canned show_* JSON and asserts the text grid plus per-cell style.
Layout, columns, alignment, labels, grouping, and colour are now
checkable under cargo test; every render below ships a snapshot.
Control read surface (each new field emitted byte-identically on the
live and off-loop builders, published once from the tick, with schema
fixtures regenerated and the parity asserts holding):
- show_status: effective persistence (persistent || nsec.is_some());
root and is_root; and a per-configured-transport-type peer-count map
in which idle-but-configured types stay visible at zero.
- show_peers: per-peer effective_depth (depth + link_cost, the value
evaluate_parent ranks on), null when unmeasured or coordless so
fipstop never recomputes it.
- show_tree: root_npub, resolved once daemon-side (self when root, an
attested peer npub, or an identity-cache hit).
- show_bloom: the last-actually-sent uptree filter fill ratio and
subtree estimate, null for a root or before the first announce.
- show_mmp: session-layer srtt, loss, and etx trend labels.
Rendering:
- Display a 6-byte non-UTF-8 TransportAddr as a colon-separated MAC at
the type layer, so daemon logs, fipsctl, and JSON consumers all
benefit; non-6-byte payloads stay bare hex.
- Right-justify the Bloom Peer Filters numerics into aligned fixed-width
columns, render the Routing panes through a kv_lines helper that shares
one value column across a key-value group, and right-justify the Graphs
by-peer summary columns.
- Truncate an over-long peer name (the npub shown when no friendly name
exists) in the Tree, Bloom, and MMP peer lists so it no longer runs
into the next column.
- Group the Peers table by role (parent, then STP children, then other)
and render it as a full grouped view with styled group labels and
blank separators; the selection stays a peer index and the cursor only
ever lands on a peer row. Apply the same role grouping to the Tree and
Bloom peer lists, joining each peer's role from the peers view by node
address.
- Show min in the Graphs plot titles, rest a steady non-zero metric on
the baseline as a row of dots, render a genuine zero as an empty plot,
and keep a distinct no-data placeholder.
- Replace the metric-by-peer grid, which squeezed plots to nothing once
peers overflowed, with a master/detail Graphs view: a scrollable
per-peer summary list that expands (Enter) to a full-pane btop plot,
with up/down to flip peer, n/N to switch statistic, m to cycle mode,
and Esc to return.
- Put inline colored trend arrows on the Link and Session MMP values
(drawn only on a rising or falling trend, with a fixed blank slot when
stable so the value columns stay aligned), via a shared helper.
- Cycle column sorting on the Link MMP, Session MMP, and Graphs by-peer
tables (one key cycles the active column, another toggles direction),
with the active column marked in each table's header.
- Render the new daemon-surfaced fields: the dashboard root line (a
self-is-root marker, otherwise a truncated root hex), a
transports-by-type line, and an "approx. mesh estimate" line; an
effective_depth column and lines on the Peers, peer-detail, and Tree
sites from the single daemon derivation, showing a dash placeholder
when unmeasured rather than a misleading zero; the full Tree root hex
plus an Npub line; and the Bloom uptree fill and subtree-estimate lines.
Interaction model:
- Add a declarative keybinding registry keyed by (Tab, UiMode) that both
the context footer and the ? help overlay render from, so the two
cannot drift; a test asserts every registry key has a dispatch handler.
- Add a modal ? help overlay, and a context-aware footer that shows the
current state's actions first, drops global hints when the terminal is
narrow, and always keeps a Help affordance as the overflow path.
- Generalize per-pane focus and scroll state on App, wired across the
Tree, Filters, Routing, and MMP tabs (f cycles pane focus and the
focused pane scrolls instead of clipping its overflow); on the MMP tab
the column sort acts on the focused pane. Esc deselects the active row
when no detail is open (detail-close still takes priority).
- Add a Del-disconnect confirmation modal naming the peer, the only
state-mutating action, issuing the control-socket disconnect on confirm
and noting that the peer stays disconnected until manually reconnected.
Complete the control-plane read-isolation work: every pure-read show_*
query now renders in the control accept task from published read
snapshots, so none round-trips the data-plane receive loop. Only the
mutating connect/disconnect commands still reach that loop.
Three subsystem snapshots are published via ArcSwap and served through the
read handle's snapshot_dispatch:
- A routing read view (spanning tree, bloom filters, coordinate cache,
identity cache, and the discovery F-queue summary scalars), published
from the tick, serving show_tree/show_bloom/show_cache/show_routing/
show_identity_cache.
- A per-entity read view (peers, sessions, links, connections, transports,
and the MMP link/session views) as Vec<Arc<Row>> tables reconciled
against the prior snapshot so a republish reuses unchanged rows by
pointer and re-allocates only changed or new rows, keeping the per-tick
publish cost bounded as the peer/session count grows. Serves
show_peers/show_sessions/show_links/show_connections/show_transports/
show_mmp.
- The stats snapshot is extended with the peer-ACL status and a per-peer
metadata map (is_active, npub, display name), resolved at publish time,
serving show_acl and the two per-peer stats queries.
Display names and other cross-subsystem fields are resolved at publish
time; time-relative fields are derived at render time from captured
absolute timestamps, so rendered output is byte-identical to the prior
on-loop handlers, which are retained as the equality oracle.
With every read query served off-loop, the show_* branch is removed from
the rx_loop control handler and the now-dead on-loop dispatcher deleted.
The snapshot projections are forward-compatible with the later structural
extraction of the derived-state and session tables: they become thin
views over the extracted types without changing the read-handle interface.
Introduce a read-snapshot plane so pure-snapshot control queries render in
the control-socket task instead of round-tripping the rx_loop, removing the
head-of-line coupling that let a busy or slow rx_loop time out fipsctl and
fipstop observability.
- ControlReadHandle: a cloneable bundle the control accept loop holds, over
the node's already-shared NodeContext and MetricsRegistry plus an
ArcSwap-published StatsSnapshot. A snapshot_dispatch seam serves cut-over
commands off-loop and falls through to the rx_loop for the rest, keeping
the rx_loop's ownership of Node intact.
- StatsSnapshot is published from the tick (the natural and sole mutator of
stats_history), carrying the history rings plus the scalar gauges and
counts show_status reports. Readers serve the latest snapshot
unconditionally, with staleness bounded by the tick interval and no
IO_TIMEOUT-coupled fallback.
- Off-loop now: show_status, show_stats_history, show_stats_all_history,
show_listening_sockets, show_stats_list, and a new counter-only
show_metrics (exposed as fipsctl "stats metrics", the enabler for a
Prometheus scraper at no hot-path cost). Queries that need live per-entity
state (peers, links, sessions, routing, and the per-peer stats variants)
stay on the rx_loop path pending later phases.
Quartet green; forward-merge to next verified clean.
The per-transport TCP inbound cap was hardwired to 256 and never read
node.limits.max_connections, so raising max_connections was a silent
no-op for inbound TCP. Resolve the effective cap with precedence:
explicit per-transport max_inbound_connections, then node-wide
max_connections, then the built-in default of 256. Established peers
remain bounded node-wide by add_connection, so deriving the per-transport
raw-accept ceiling from max_connections does not admit more real peers
across multiple transports.
Add effective_max_inbound on the TCP transport with a node_max_connections
setter wired from create_transports, plus a precedence unit test.
Estimate the OR-union cardinality over self plus every connected peer's
inbound filter, dropping the parent/child tree gating in
compute_mesh_size. Filter propagation is split-horizon, so cross-links
advertise near-complete mesh views; unioning all peers yields the same
set as the tree-only union in steady state (OR dedups overlap and no
filter can over-count) while damping the node-count flap on parent
switches, since dropping the parent no longer collapses the upward leg.
This also removes the estimate's dependence on tree-declaration cache
freshness.
Rename the debug-log child_count to contributor_count, adapt the two
membership-invariant tests to all-peers semantics, and add a test that
the estimate stays stable across a parent drop when a healthy cross-link
is present.
Brings the maint-line OR-union mesh-size estimator fix and the
per-peer / capacity-cap log-level demotions forward to master.
Conflict resolution: compute_mesh_size resolved to master's config()
accessor form carrying maint's OR-union rewrite (drop the stale summing
`total`); the log-level demotions auto-merged into master's handler
versions. The equivalent master-only log "connected UDP socket installed"
(connected_udp.rs, a file that does not exist on maint) was demoted
info -> debug as part of this merge so the connected-UDP path matches the
rest of the per-peer lifecycle logging.
The mesh-size estimator summed the per-filter cardinality of the parent
filter and each child filter, which assumes those filters are perfectly
disjoint. When they overlap -- a stale or oversized parent filter, or a
routing loop -- the sum over-counts and inflates the reported mesh size
to as much as several times the true size.
Estimate the cardinality of the OR-union of the contributing filters
(self + parent + children) once instead. OR is idempotent, so any
overlap is deduplicated: the result equals the old sum in the disjoint
case and stays correct under overlap. The union is seeded from a clone
of a contributing filter so it keeps that filter's size class, and a
filter whose size class does not match is skipped rather than panicking.
The refuse-to-estimate behavior on a saturated or above-cap filter is
preserved.
Add a regression test with overlapping parent and child filters where
the naive sum over-counts and the union estimate tracks the distinct
member count.
Johnathan Corgan
ArjenandJohnathan Corgan
olekskyandJohnathan Corgan