Introduce a typed RejectReason enum and a NodeStats::record_reject
dispatch so every receive-path rejection-and-return site bumps a
machine-readable per-subsystem counter while keeping its operator-facing
log line. The top-level variants mirror the existing NodeStats subsystem
split (Tree, Bloom, Discovery, Forwarding) and add Handshake, Session,
Mmp, and Transport categories; HandshakeStats, SessionStats, and MmpStats
are new sub-stats.
Wired clusters: tree and MMP outbound sign-failure; the FSP session
unknown-session and state-machine cluster; the Noise IK handshake
state-machine cluster (msg1/msg2); and the decode / crypto / cap /
semantic tail across bloom, discovery, forwarding, mmp, and tree. The
TreeStats::ancestry_invalid counter, present since the scaffold but never
incremented, is now bumped from the validate_semantics ancestry rejection.
Several handshake, MMP, tree, and discovery paths that previously had no
counter at all are now counted, including the send_lookup_response
no-route drop (DiscoveryStats::resp_no_route).
Existing direct counters at the bloom / discovery / forwarding sites are
retained alongside the new dispatch while the rollout is in progress (the
bloom_poison tests expect the transitional +2 delta); a later change
collapses the duplicate increment.
Raise the in-process backpressure headroom in make_test_node_with_mtu
(request an 8 MiB recv_buf_size on UdpConfig and grow packet_channel from
256 to 8192) to reduce localhost-UDP receive overflow under parallel-CPU
scheduler contention, and mark the large-network convergence tests
#[ignore] so cargo test --lib stays green by default. The ignored tests
remain runnable on demand with --ignored or --test-threads=1.
Add per-direction pool_inbound/pool_outbound counters to TcpStats and
TorStats, updated at every pool-insert, receive-loop-exit, transport-stop,
and send-failure removal site. Compare the max_inbound_connections cap
against pool_inbound rather than the combined pool length, so outbound
connect-on-send connections no longer consume the operator-facing inbound
budget. The configuration field name and operator semantics are preserved;
only the cap-check comparison and accounting change.
New integration scenario verifying the early-gate silent-drop behavior
of the inbound max_peers admission check at sustained scale, using the
existing 5-node mesh topology with one node's node.limits.max_peers
lowered to 1. This forces 2 of the cap'd node's 3 configured peers
into a sustained denied state, and asserts via tcpdump that no Msg2
responses go back to those denied peers across a 60s capture window.
A background load-driver restarts the denied peer containers every 15s
to reset their auto-reconnect exponential backoff (5s base / 300s cap),
producing fresh Msg1 bursts each cycle. Without this loop the gate
fires ~3-4 times per denied peer in a 60s window; with restarts the
observed rate is 15 per denied peer (~30 total firings), high enough
that any Msg2 leakage would be caught with strong statistical
confidence.
Local run on this branch: cap'd node-c converged to peer_count=1 with
node-b admitted; nodes d and e sustained-retried as denied; tcpdump
captured 30 inbound Msg1 (len 84) packets from the denied pair and 0
outbound Msg2 (len 104) packets, with final peer_count unchanged.
Files:
testing/static/scripts/admission-cap-test.sh — new test script with
inject-config subcommand (sets node.limits.max_peers) and a
3-phase test driver (converge, capture-with-load, per-peer assert)
testing/ci-local.sh — register admission-cap as a new suite category
(ADMISSION_SUITES), wire run_admission_cap function, add to
run_suite dispatch, list_suites, and the default integration sweep
Together with the existing unit-level coverage in src/node/tests/unit.rs
(handle_msg1_silent_drops_at_cap_for_new_peer with mock-transport Msg2
discriminator, and handle_msg1_admits_existing_peer_at_cap as the
bypass regression guard), the gate's silent-drop behavior is now
verified both at single-firing wire-observable resolution and at
sustained multi-firing cross-process scale.
Move the max_peers cap check in handle_msg1 forward, from the late
check inside promote_connection (which fires after Msg2 has already
been built and put on the wire) to an early position after identity
verification but before index allocation and the Msg2 send. When the
gate fires for a net-new identity, the Msg1 is silent-dropped — no
response goes back to the peer, no AEAD compute or wire bytes are
spent.
Bypass preserved for known peers (reconnect / cross-connection): if
the sender's NodeAddr is already in self.peers, or if a pending
outbound connection is in flight to the same identity, the gate is
skipped so legitimate maintenance traffic continues to work. The
late check inside promote_connection is intentionally retained as
defense-in-depth against future call sites or a disconnect racing
between the early-gate decision and promotion.
Wire-cost rationale: a 45 s tcpdump at saturation observed ~3.6
cap-denials/s steady-state, each previously paying the full Noise IK
responder crypto + Msg2 (~104 B) on the wire before being rejected.
The bigger value is cleaner peer-side semantics — the peer no longer
sees a fake-completed handshake whose data frames subsequently fail
decryption locally.
Two new unit tests cover the cases:
- handle_msg1_silent_drops_at_cap_for_new_peer drives a wire-pumped
Msg1 from a fresh identity into a saturated node and asserts no
Msg2 reaches the sender socket. Stash-verifies as FAIL on the
pre-fix tree (Msg2 hits the wire) and PASS post-fix.
- handle_msg1_admits_existing_peer_at_cap drives a Msg1 from an
identity already in self.peers and asserts the gate does not evict
it. This is a regression check (the no-gate tree behaves the same
way here, but the test guards against an accidental future gate
that breaks known-peer admit).
Brings two structural fixes landed on maint:
- compute_mesh_size: explicit parent skip in the children loop, so the
disjoint-subtree invariant no longer depends on peer_declaration cache
freshness.
- max_peers: outbound connection-initiation gated on the cap (auto-reconnect
retries, Nostr-mediated discovery established adoption, and both sides
of the NAT-traversal punch sequence). Inbound msg1 admission gate
unchanged.
node.limits.max_peers was honored only on inbound msg1 admission
(handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len
is at the cap). Four outbound initiation paths proceeded unconditionally
at capacity: auto-reconnect retries (process_pending_retries),
Nostr-mediated discovery's BootstrapEvent::Established adoption
(poll_nostr_discovery), NAT-traversal punch initiation (the outgoing
side of the offer/answer/punch sequence in the Nostr discovery
runtime), and NAT-traversal punch response (the incoming side of the
same sequence). A saturated node burned CPU, UDP probes, STUN
observations, and Nostr relay traffic on connections that the inbound
gate would reject the moment they reached msg1.
Introduce Node::outbound_admission_check (peers.len < max_peers, or
true when max_peers == 0 as the no-cap sentinel) and gate the four
paths. The discovery runtime lives in a separate task and does not
hold a Node reference; bridge via an Arc<AtomicBool> the runtime
reads and Node refreshes once per tick from outbound_admission_check.
The atomic granularity is intentionally loose: one-tick lag is
acceptable because the inbound msg1 gate continues to be the
authoritative cap, and in-flight handshakes started below the cap
are allowed to complete.
Inbound gate at handshake.rs is unchanged.
The mesh-size estimator's children loop relied on the cached
peer_declaration(parent_id).parent_id() != my_addr check to exclude
the parent. That cached view briefly disagrees with our own latest
my_declaration().parent_id() during the window between a local
parent-switch and the new parent's next inbound TreeAnnounce: the
peer-declaration cache still names us as the parent's parent, so the
parent is iterated as if it were a child and its (typically dominant)
bloom cardinality is added a second time. Symptom: estimated mesh size
displayed in fipsctl show status and fipstop nearly-but-not-exactly
doubles during tree rebalancing.
Make the invariant structural with an explicit peer_addr == parent_id
skip at the head of the children loop. Per-peer 500 ms rate-limiter
and overall recompute cadence are unchanged.
Adds a regression test that constructs the stale-peer-declaration
scenario directly and asserts the parent is not double-counted.
Adds a tree/mmp-targeted compose overlay (compose-trace-tree.yml) and
a new FIPS_MESH_LAB_TRACE_TREE env-var gate in run_rekey_family,
layered independently of the existing FIPS_MESH_LAB_TRACE rekey-class
overlay. Trace targets: fips::node::tree, fips::tree,
fips::node::handlers::mmp, fips::node::handlers::handshake.
Used for tree-partition race investigations during multi-peer startup
where evaluate_parent inputs, send_tree_announce_to_all recipient
enumeration, and process_receiver_report first-RTT triggers all need
to be visible together to bracket the loss window.
Extend parse_rekey to emit phase1_status / phase1_baseline_passed /
phase1_baseline_total fields in signature.json by scraping rekey-test.sh's
"Best observed baseline before timeout: N/M passed" line (the timeout
path) and "Pre-rekey baseline (all 20 pairs): N/M passed" (the success
path). Phase-5-shape parsing is unchanged.
Extend mechanism_match_rekey to also fire on the Phase 1 characteristic
12/20 split — the multi-hop-routing-failure shape where direct-peer pairs
pass and the four multi-hop pairs (x 2 directions) fail. The Phase 5
predicate remains intact for the pre-existing flake class.
Add FIPS_MESH_LAB_NO_RESOURCE_LIMITS=1 to run_rekey_family for
unconstrained characterization runs where the goal is to surface a race
or scheduling artefact rather than reproduce GHA pressure. Default
behaviour (variable unset) keeps the compose-resource-limits.yml overlay
engaged.
Documented in README.md and the in-script env-var header block.
Closes the eventually-consistent gap in spanning-tree state
distribution. Every existing send_tree_announce_to_all call site
gates on a local state-change event (parent switch, self-root
promotion, ancestry change, peer promotion, parent loss). Once a
partition latches — for example a parent-switch announce stranded
in the brief cross-init handshake swap window, where the announce
arrives on a session-index whose decrypt-worker entry has been
unregistered — neither side's state changes again, so neither
side ever re-broadcasts. The existing 60 s check_periodic_parent_reeval
was a re-evaluation, not a re-broadcast: it short-circuited
silently on no-change. Production-side healing depended on
incidental link churn; lab harnesses with stable docker-bridge
links had no equivalent path.
Add a final else branch that fires send_tree_announce_to_all
unconditionally on the no-change path, alongside the existing
switch and self-promote arms. Receivers coalesce by sequence
comparison (ParentDeclaration::is_fresher_than) and short-circuit
at the `if !updated` gate in handle_tree_announce; same-sequence
repeats drop silently with no cascade. The per-peer 500 ms
rate-limiter is well below this 60 s cadence and does not suppress
the heartbeat broadcast.
The fix is a general protocol-robustness improvement: it addresses
any in-flight TreeAnnounce loss class, not only the specific
cross-init swap-window drop site.
testing/static/scripts/rekey-test.sh BASELINE_CONVERGENCE_TIMEOUT
60 -> 65 so a partition healed by the periodic broadcast at T+60
lands inside the convergence window. wait_for_full_baseline
early-exits on PASS, so successful reps see no extra wall-clock.
The cross-connection-won path in handle_msg1 removes the old peer and frees
its allocated index, but does not unregister the old (transport_id, our_index)
cache_key from the decrypt worker pool. The orphan entry sits in the
per-shard HashMap until the index allocator recycles old_idx to a different
peer and that peer's register_decrypt_worker_session call overwrites it.
In the interim, any decrypt job that lands at the recycled cache_key
resolves to the wrong session and AEAD silently fails — observed as
multi-hop routing failure in 5-node static-mesh on next-branch where
bidirectional auto_connect drives cross-connections at every peer pair
on startup.
The tick body's per-peer check_* loops (heartbeats, bloom
announces, MMP reports, tree announces) called transport.send
for every active peer, which on TCP/Tor fell through to a 5 s
connect-on-send wait for any peer whose pool entry was not yet
established. That wedged the entire tick body for the full
connect_timeout_ms per unreachable peer; under post-restart
convergence on a high-peer mesh, this cascaded into multi-
second tick stalls. On master, the same mechanism also starved
the per-tick control-snapshot republish and pushed fipsctl
queries onto an mpsc fallback that was itself queued behind
the wedged rx_loop, producing the 5-second fipsctl head-of-line
pattern operators observed on loaded nodes.
Gate send_encrypted_link_message_with_ce on
transport.connection_state before the send: proceed only when
Connected; on None, kick off a non-blocking background connect
(idempotent — TransportHandle::connect dedupes against the
connecting pool and spawns the timeout-bounded TcpStream::connect
inside its own tokio task) and fail this send fast with a
clear "transport connection not ready" error. A subsequent
tick retries once the pool has an entry. The reconnect
lifecycle (check_link_heartbeats, process_pending_retries,
poll_pending_connects) is unchanged. The connect-on-send
branch in transport.send_async itself remains in place for
code paths that legitimately need synchronous connect (e.g.,
explicit operator-driven fipsctl connect).
When both peers' Nostr-mediated UDP punches complete within the
same scheduling window, each side's `BootstrapEvent::Established`
event arrives with `is_connecting_to_peer` already true: each side
received an inbound msg1 from the peer's pre-punch outbound
attempt, which created a connecting-state record. The deduplication
skip then fires on both sides, neither installs the fresh
traversal socket as canonical, and the peer-adoption budget
(45 s) expires. Cross-node wall-clock alignment of the skip log
line in observed failures was within ~1 ms — simultaneous dual-
fire under contention, the dual-initiation pattern.
Apply the deterministic NodeAddr tie-breaker already used at
`handlers/handshake.rs:269` for rekey dual-initiation and in
`peer::cross_connection_winner` for cross-connection resolution.
Smaller NodeAddr wins as adopter: enumerate the in-flight
connections whose `expected_identity` points at this peer, tear
them down via the canonical `cleanup_stale_connection` helper, and
fall through to `adopt_established_traversal`. Larger NodeAddr
loses and keeps the existing `continue` semantics; the loser's
in-flight outbound is reconciled by `handle_msg1`'s cross-
connection logic when the winner's fresh msg1 arrives over the
adopted socket.
`cleanup_stale_connection` visibility bumped from module-private
to `pub(in crate::node)` so it is callable from `lifecycle.rs`.
The defensive re-check inside `adopt_established_traversal`
itself is left as-is — after the outer cleanup the winner reaches
it with `is_connecting_to_peer == false`, so the inner skip
won't trip. The `BootstrapEvent::Failed` arm is unchanged: there
is no winning outcome on dual failure, and the existing skip +
retry-schedule semantics are correct.
Three deltas to the mesh-lab nat-lan suite for stall characterization:
- FIPS_NAT_LAN_CPUSET env-var-driven CPU-pinning sidecar in
run_nat_lan, mirroring the bloom-storm pattern. Pinning is needed
because the mesh-lab compose-resource-limits.yml override is
rekey-family service-name specific (rekey-* / rekey-accept-off-* /
rekey-outbound-only-*), so it does not constrain the nat-lan
containers. Default cpuset 0,1 mimics a GHA 2-core runner; empty
disables the sidecar.
- New compose-trace-nat.yml overlay that bumps RUST_LOG to trace on
discovery::nostr, transport::udp, node::lifecycle,
handlers::handshake, handlers::forwarding — the modules covering
the cross-init / adoption / handshake path. Picked up by the
nat-test.sh COMPOSE array via a new FIPS_NAT_EXTRA_COMPOSE
colon-separated env-var hook. run_nat_lan sets this hook when
FIPS_MESH_LAB_TRACE is non-empty; the README env-var section
updated to reflect that FIPS_MESH_LAB_TRACE now applies to nat-lan
in addition to the rekey-family.
- parse_nat_lan extended with a per-node stall_signature emitting
last-occurrence timestamps for eight event categories (startup,
discovery, adoption, handshake_init, msg2_sent, cross_init_ignore_*,
handshake_failed) plus derived last_meaningful_event_ts,
last_event_category, silent_gap_s. Top-level stall_class binned as
no_timeout / silent / localized / distributed / incomplete from
the per-node categories. Aggregation phase consumes the per-rep
signatures across a characterization run to classify stall
mechanism.
Wired support in nat-test.sh: FIPS_NAT_EXTRA_COMPOSE colon-separated
list of repo-relative or absolute compose files layered onto the
base via the COMPOSE array; FIPS_NAT_SKIP_FINAL_CLEANUP gates the
success-path teardown so the mesh-lab harness can capture docker
logs before tearing down (failure paths already returned without
cleanup, leaving stall-state containers intact for capture).
Smoke-tested on idle profile with TRACE on: 1 rep PASS, 32/36 TRACE
lines per node, signature.json events all populated with the
expected category timestamps.
The bloom-storm scenario's bloom_send_rate ceiling has been bumped
from 30 to 40 sends per node over the trailing 30 s window. A 59-rep
characterization run under `github-runner-equivalent` pressure with
per-container CPU pinning to `cpuset=0,1` (mimicking a 2-core
`ubuntu-latest` runner) measured n04 (the structural max-spike node)
at mean 24.4, P99 29, max 30. The original ceiling of 30 sat at the
lab's structural max, leaving no headroom for the asymmetric transient
spikes observed on GitHub Actions (n04=34 on master CI run 25933972365,
re-fired on run 26008950865). GHA fires do not reproduce on this lab
host even with the cpuset sidecar applied.
Rationale: lab max + ~2σ ≈ 39.4 → round to 40, giving 33 % margin over
the lab maximum while staying well below the deployment-scale storm
rate (~480× steady state). The companion `min_parent_switches` guard
is unchanged.
See README.md alongside this file for the updated threshold derivation.
Wires the bloom-storm chaos scenario into the mesh-lab harness as
a first-class suite, with optional per-container CPU pinning to
mimic GitHub Actions' 2-core ubuntu-latest budget.
Dispatch path — three new run-loop.sh functions plus the
dispatch_suite and dispatch_mechanism_match case-arm additions:
- `run_bloom_storm` invokes `bash testing/chaos/scripts/chaos.sh
bloom-storm` and captures stdout+stderr into the rep's
test-output.log. Chaos uses its own python sim runner
(`python3 -m sim`), not docker-compose, so this suite gets no
per-container compose override, no separate `docker logs`
capture, and no in-container netem injection — the chaos
scenario yaml owns its own netem and link-swap config.
- `parse_bloom_storm` extracts the bloom_send_rate result
(pass/fail/unknown), ceiling, max-observed per-node delta,
offenders list, full per-node delta distribution, the companion
min_parent_switches result, and panic + error counts. Lands in
the rep's signature.json. Two parser details: assertion greps
are anchored on `^(PASS|FAIL)` so they only match the bare
end-of-run summary line, not python-logger-prefixed lines that
contain the same substring; and `grep -c` panic/error counts
use `; true` + a defensive empty-string check instead of the
common `|| echo 0` fallback (`grep -c` exits 1 on zero matches
while also printing "0", so the fallback would corrupt the
count to "0\\n0").
- `mechanism_match_bloom_storm` returns true when a rep both
fails the bloom_send_rate assertion and the FAIL line carries
a named offender (filtering the harness-side "failed to sample
window endpoints" sub-failure out of the mechanism count).
CPU-pinning sidecar — bloom-storm's chaos sim spawns containers
directly via the docker SDK, so the mesh-lab compose-resource-
limits override does not apply. A poll-and-pin loop around the
chaos.sh invocation lists \`fips-*\` containers every 0.5 s and
applies \`docker update --cpuset-cpus <set>\` to each. Pinning is
idempotent (re-applying the same cpuset is a no-op). Default
cpuset \`0,1\` mimics the GHA 2-core budget; override via
\`FIPS_BLOOM_STORM_CPUSET=<set>\` (any comma-separated CPU list),
or set to the empty string to disable. Only applies to the
bloom-storm suite; other suites' dispatch paths are unchanged.
README's "Suites supported" entry covers the assertion class, and
the \`FIPS_BLOOM_STORM_CPUSET\` knob is documented alongside the
other mesh-lab env-var knobs.
The cross-connection-won branch of `promote_connection` builds a
fresh ActivePeer with a new Noise session and our_index, inserts
it into peers, and registers identity, but did not hand the new
session to the decrypt shard worker pool. The normal-promotion
tail in the same function does make that call. A session
established via the cross-connection race path therefore missed
the worker fast-path for its lifetime, falling back to inline
decryption on the rx loop. Correctness was unaffected, but the
throughput/latency benefit of the worker pool was lost for peerspromoted through that path.
Mirror the normal-promotion tail and call
`register_decrypt_worker_session` after the fresh ActivePeer is
inserted into `self.peers` in the `this_wins` arm.
Logs source, destination, and payload size at the existing no-route
drop site so investigations can attribute transit drops without
enabling trace-level instrumentation. Diagnostic-only; no behavior
change on the success path.
An FSP session rekey could leave the two endpoints holding different
key sets for a brief window: if a handshake message was lost in
transit, one side rotated to the new keys while the other did not.
Traffic sealed in one key epoch then reached a peer still on the
other epoch and failed to decrypt, producing bursts of AEAD
decryption failures and dropped connectivity until a later rekey
cycle reconverged the pair. Choreographing the cutover order cannot
close this window: any fixed ordering still leaves a skew that
packet reordering widens.
Make rekey correctness independent of cutover timing by overlapping
the key epochs on the receive path. During a rekey transition the
receiver trial-decrypts each frame against every live session it
holds: current, the not-yet-promoted pending session, and the
draining previous session. The K-bit becomes a hint that orders the
trial-decrypt cascade rather than a hard gate, and a frame that
authenticates against the pending session is itself the cutover
signal. No rotation ordering and no packet reordering can then cause
a decryption failure.
The pre-rekey Noise session is held in the `previous` slot until the
peer has demonstrably moved off it. Its drain deadline is anchored
on the most recent frame the peer authenticated against that slot,
refreshed each time the trial-decrypt cascade lands there, rather
than on a fixed wall-clock timer started unilaterally at the local
cutover. A peer that never received the new keys keeps authenticating
against `previous` and the slot stays live; without this, a fixed
timer would erase the only key set that could decrypt the peer's
frames, producing a permanent silent decrypt failure on a live data
path. A peer that never catches up is handled by the existing FSP
session liveness path rather than by silent decrypt failure.
The lost-handshake liveness gap is closed separately by retransmitting
the third rekey handshake message until the peer is confirmed on the
new keys, with a bounded retry budget after which the rekey cycle is
cleanly abandoned and retried on the next timer.
Adds unit tests covering the trial-decrypt cascade (epoch selection,
promotion on pending decrypt, reordered old-epoch stragglers after
cutover, per-slot replay-window integrity), the msg3 retransmission
lifecycle, and the peer-progress-aware drain retirement.
Commit 57a089f6 (the GitHub #102 fix) landed without a CHANGELOG
entry. Add the `[Unreleased]` / `### Fixed` line so the macOS
package-integrity fix is on record before the v0.3.1 cut.
The published v0.3.0 macOS installer is a structurally corrupt xar
archive: pkgutil and xar reject it even though its SHA-256 matches the
published checksum.
build-pkg.sh derived the architecture suffix in the .pkg filename from
`uname -m`. On the Apple-silicon macOS runner that always reports
arm64, so the cross-compiled x86_64 build also named its output
fips-<version>-macos-arm64.pkg. The release job downloads both build
artifacts with merge-multiple into one directory, where the two
identically named files collide and tear into a malformed result. The
x86_64 package never reaches the release at all.
Derive the package architecture from the Rust target triple, which is
authoritative for cross-compiles, instead of from the build host. Each
matrix leg now produces a distinctly named, arch-correct package, so
the two artifacts no longer collide.
Add a SHA-256 integrity chain so a corrupt or mismatched asset cannot
be published again:
- Capture the .pkg SHA-256 on the macOS runner, after the on-runner
structural verification, into a sidecar file carried in the artifact.
- Add a verify-handoff job that runs on every trigger and asserts each
downloaded .pkg still matches its macOS-runner SHA-256.
- Gate the release job on verify-handoff and repeat the check on the
exact bytes about to be published.
The build step now asserts it produced the expected arch-named package
so a regression in the naming fails loudly rather than as a silent
collision.
Relates to #102. The published v0.3.0 macOS assets still need to be
rebuilt and reuploaded separately.
Publish the 13-criteria PR review checklist the maintainer runs on
every incoming PR so contributors (and their coding agents) can run
the same pass before opening, surfacing problems before the review
round trip. CONTRIBUTING.md gets a new 'Self-review against the
project review checklist' subsection under 'Submitting pull requests'
and a Further Reading entry. CHANGELOG [Unreleased] gets an Added
entry.
Two [Unreleased] / Changed entries that should have landed alongside
the originating commits but didn't:
- macOS recvmsg_x batched receive (originally 59225ccf): completes
the Linux-equivalent inbound batching shape on Apple builds. Now
sequenced before the rx zero-copy entry so the section reads as a
coherent receive-path progression.
- Platform-specific test-build warning cleanup (originally 6bd40640,
PR #93): non-behavioral; documents the gating decisions that keep
cross-platform builds warning-clean.
Two Fixed entries appended to [Unreleased]:
- The coord cache surgical invalidation (49bd2104): replaces the
global CoordCache::clear() at parent-switch / become-root /
loop-detection / root-change sites with two targeted methods
(invalidate_via_node, invalidate_other_roots). Preserves cache
entries that remain correct after the topology change.
- The rekey-test strict-ping retry (306e4555): Phase 1 / 3 / 5
per-pair pings now retry up to 4 attempts. Brings the ICMP-noise
miss-floor from ~33% per phase to ~3.2e-6 at 1% loss without
changing the failure-shape signal the asserts target. Test
scaffold only, no daemon code changes.
The Phase 1, Phase 3, and Phase 5 strict asserts each fire a
single ping per directed pair. Under low-level packet loss
(e.g. 1% i.i.d. per-direction loss from a CI runner under
pressure), a single-shot round-trip fails at ~2% per pair, so a
20-pair strict assert misses with probability
1 - (0.98)^20 = ~33% per phase from ICMP noise alone, well above
the routing-state signal the asserts are meant to catch.
ping_one gains a max_attempts parameter (default 1, preserving
existing call sites). On failure it retries up to
MAX_PING_ATTEMPTS-1 additional times with PING_RETRY_DELAY
seconds between attempts. Per-pair worst case under the defaults
(4 attempts, 1 s spacing, 5 s ping6 -W timeout) is 4*5 + 3 = 23 s;
per-rep worst case scales with the failing-pair count.
Successful retries log "OK (RTT, attempt N)"; exhausted retries
log "FAIL (after N attempts)".
The retry budget is wired into all three strict asserts:
- Phase 1 final ping_all (after wait_for_full_baseline converges)
- Phase 3 ping_all (post-first-rekey)
- Phase 5 ping_all (post-second-rekey)
The wait_for_full_baseline convergence loop itself stays
single-shot. Its job is to detect when the mesh first sees a
fully clean 20-pair batch, and retries inside the loop would
conflate transient ping loss with still-converging routing
state.
No daemon code changes.
Replaces the unconditional `CoordCache::clear()` calls at parent-switch,
become-root, and loop-detection sites with two targeted invalidation
methods scoped to what actually makes an entry stale:
- `invalidate_via_node(node_addr)`: drop entries whose cached
destination ancestry contains `node_addr`. Used at parent-position-
change sites — our prefix changed, so destinations downstream of
us have stale-prefix coords.
- `invalidate_other_roots(current_root)`: drop entries rooted under
a different root than the current one. Used at root-change sites.
Under the previous global flush, parent switches blanked the cache
across the board, leaving `find_next_hop` returning `None` for every
non-direct-peer destination until the cache passively re-warmed via
incoming TreeAnnounces / SessionSetup. Surgical invalidation
preserves entries that remain correct after the topology change.
The cached coord describes a destination's tree position; that
position only goes stale relative to our own routing decisions when
our own prefix changes (entries we are downstream of) or the root
changes (entries in a different tree). Peer removal does not
invalidate cached coords: `Node::find_next_hop` recomputes the
next-hop decision on every call against the current peer set, bloom
filters, and tree state, and Discovery already triggers on
`no route to destination` errors when a destination becomes
unroutable through us. The peer-removal site retains the original
"no cache invalidation" behavior.
Each method returns the count of entries removed for observability.
Unit tests cover each method against the cases enumerated in the
acceptance criterion.
Bring [Unreleased] into sync with all maint commits since v0.3.0:
- Add a Fixed entry for the acl-allowlist test-script poll-assertion
conversion (commit e9dd316) that was previously missing.
- Add the AUR-publish workflow rewrite and new fips-git VCS workflow
(commit 9bf9701) which had no entry, and merge them with the
ci.yml cancel-in-progress block under a single "CI and
release-publish workflows hardened" entry so the three workflow
changes read as one operational theme.
Pure changelog content reshuffle. No code touched.
Add a top-level concurrency block to ci.yml keyed on
(workflow, ref) with cancel-in-progress: true. Pushes (including
force-pushes) to the same ref now retire any in-flight run for
that ref rather than letting the superseded and current-tip runs
both burn runner minutes.
Motivated by a 2026-05-14 force-push experience where two CI runs
ran concurrently against a feature branch — the original push at
c76ec99 continued for ~17 minutes alongside the amended push at
927ef47, despite only the latter being the live tip.
Scope deliberately limited to ci.yml. Tag-triggered release-build
workflows (package-*.yml, aur-publish-*.yml) are untouched — they
operate on per-tag refs that already form distinct concurrency
groups and release artifact builds generally should not be
cancellable by unrelated activity.
Phase 5's per-pair connectivity check ran immediately after the second
rekey cycle, with no settle for routing reconvergence. Under
GitHub-runner CPU contention, post-rekey parent-switches and
coord-cache flushes can take longer than the per-ping 5s timeout for
a small fraction of pairs (1-3 of 20 typically), even though the
rekey mechanism itself completes cleanly. Phase 6 log analysis stays
all-green on these failed runs; the failure is purely connectivity
timing.
Mirror Phase 3's existing 12-second settle pattern: reuse REKEY_SETTLE
and emit the same banner shape. Two-line change. Cost on the success
path is a fixed 12s per suite run.
Convert assert_log_contains from a one-shot grep snapshot into a
bounded poll that retries until the pattern appears or the timeout
elapses (default 15s). Same wait-with-timeout shape as
wait_for_peers_exact above it in the file.
The pre-existing flake on next-branch CI is structural: under XX
handshake, the cross-connection tie-breaker selects which side
reaches its ACL-check point first. When container-a wins the
tie-breaker on the first attempt against c and d, only the
outbound-handshake-context rejection fires immediately, and the
inbound-handshake-context rejection only emits on a later retry
when c or d's msg1 lands while a has no pending outbound. On one
2026-05-14 run the inbound rejection appeared 63ms after the test
had given up. The race window is small but real.
Polling the log instead of one-shot reading absorbs the
millisecond-to-second variance without slowing the success path
(the helper returns as soon as the pattern appears).
Add a per-session signed jitter offset (uniform [-15, +15] seconds)
to the rekey timer triggers in check_rekey (FMP) and check_session_rekey
(FSP). The configured `node.rekey.after_secs` becomes the nominal
interval rather than a floor; mean is preserved. Desynchronizes
both endpoints in symmetric-start meshes so the dual-initiation
race stops occurring rather than being resolved after the fact by
the smaller-NodeAddr tie-breaker.
Per-session storage means each rekey cutover reconstructs the
session and redraws the jitter naturally — successive cycles get
independent offsets, preventing drift back into sync.
Three entries under [Unreleased] for the three commits since the
v0.3.0 release tag:
- Sidecar example: FIPS_UDP_MTU env override (commit 32a3b58)
- CONTRIBUTING.md overhaul + new docs/branching.md (commit 538ce07)
- Rekey-test Phase 1 baseline-convergence headroom 36s -> 60s
(commit 6533276)
The previous CONTRIBUTING.md read like a stock Rust contributing
template that mentioned FIPS. Replace it with an entry-point doc
that gives new contributors the FIPS-specific mental model they
need to make a useful first PR: the FMP/FSP layering and why
mesh-level changes need multi-node testing, the three-branch
release model and how to choose a target branch, structured bug
reporting expectations, and PR submission requirements (scope
discipline, the local-CI ladder, separate requirements for feature
PRs vs bug-fix PRs, squash-merge mechanics).
Add a contributor-facing AI coding assistant policy: use is
welcome, but the contributor must do a thorough manual review and
editorial pass before submission. The agent is a tool; the
contributor is accountable for the submission. Review effort
scales with submission effort -- unreviewed agent output will
receive an agent reply in turn, without human review.
Add docs/branching.md as the long-form companion covering the
release workflow, version conventions, and merge-direction
rationale. The new CONTRIBUTING.md is the day-to-day entry point;
docs/branching.md is the reference.
All cross-references resolve against current HEAD. Previous
stale links to fips-intro.md, fips-wire-formats.md, and
fips-configuration.md are gone; the doc points at the current
docs/design/ layout, docs/getting-started.md,
docs/tutorials/join-the-test-mesh.md, and testing/README.md.
No code changes.
The Phase 1 pre-rekey baseline in `wait_for_full_baseline`
occasionally times out on GitHub-hosted runners with one ping pair
failing to converge inside the BASELINE_CONVERGENCE_TIMEOUT window.
Phases 2–6 always pass cleanly when this happens — the rekey itself
is fine, the mesh just hasn't finished spanning-tree + bloom-filter
convergence by the time Phase 1 starts pinging.
The wait loop returns as soon as all 20 pairs converge, so the cost
on the success path is unchanged (typical local CI returns well
under the old 36s). The bump only adds headroom on the failure
path. Operators previously worked around this with
`gh run rerun --failed`; this aims to retire that workaround for
the IK/XK lines.
Distinct from the next-branch XX rekey dual-init race, which is a
real protocol bug tracked separately.
The sidecar entrypoint hardcoded `udp.mtu: 1472`, the Docker-bridge
IPv4 maximum (1500 MTU - 8 UDP - 20 IPv4 header). Promote it to
`FIPS_UDP_MTU` (defaulting to 1472, preserving behavior) so non-Docker
reuses of the example can override without editing the script. Plumb
the env var through `docker-compose.yml` so a host-level setting
reaches the container, and add a row to the README's env-var table.
Also annotate the static-CI node template with a comment explaining
the same 1472 rationale and the daemon's 1280 default. The template
keeps 1472 as the literal value since the CI suite runs on a Docker
bridge where that's correct.
No behavior change unless the host explicitly sets FIPS_UDP_MTU.
Marker merge to record the maint dev-line as known on master
without taking maint's v0.3.1-dev opening commit (master is on
v0.4.0-dev). Future bug-fix forward-merges from maint to master
land cleanly on top of this base.
Reset maint to v0.3.0 to retire the v0.2.x maintenance window and
open a new tracking branch for v0.3.1-dev bug-fix work.
- Cargo: 0.3.0 → 0.3.1-dev
- CHANGELOG: fresh [Unreleased] block above [0.3.0]
- README: badge v0.3.0 → v0.3.1--dev
Bump Cargo.toml version 0.3.0-dev -> 0.3.0 and resync Cargo.lock,
move the CHANGELOG [Unreleased] block under [0.3.0] - 2026-05-11,
update the README status badge and prose to v0.3.0, and add the
release notes at docs/releases/release-notes-v0.3.0.md with a
mirrored copy at the repo root as RELEASE-NOTES.md.
Uses the 'ours' merge strategy to keep all of master's tree
verbatim with one carve-out: the v0.2.1 release notes archive at
docs/releases/release-notes-v0.2.1.md is retained as a permanent
docs artifact.
Discarded from maint's release-prep commit: Cargo.toml / Cargo.lock
version bumps (master stays on v0.3.0-dev), README v0.2.1 badge and
prose, and the RELEASE-NOTES.md root mirror (master's root mirror
will eventually carry the v0.3.0 release notes when v0.3.0 ships
from this line).
CHANGELOG: the v0.2.1 release section is integrated between
[Unreleased] (v0.3.0-dev work) and the [0.2.0] historical section.
Items that landed on maint during the v0.2.1 cycle are removed
from [Unreleased] so [0.2.1] is the canonical home for those
changes (Linux release artifact and AUR workflows; bloom fill-ratio
validation; seven maint-line bug fixes).
Bump Cargo.toml version 0.2.1-dev -> 0.2.1 and resync Cargo.lock,
move the CHANGELOG [Unreleased] block under [0.2.1] - 2026-05-11,
update the README status badge and prose to v0.2.1, and add the
release notes at docs/releases/release-notes-v0.2.1.md with a
mirrored copy at the repo root as RELEASE-NOTES.md.
The host (217.77.8.91:443) was renamed to test-us01 some time ago
(canonical name shared with packaging/common/hosts and the docs);
the test scaffold's alias labels and narrative comments hadn't been
updated. Pure naming cleanup, no behavior change. The continued
dependency on the live external host is tracked separately.
Walk through reviewer feedback on the Nostr-discovery docs and
land 18 items.
Bulk patterns:
- `external_addr` / `public: true` semantics consistently
misdescribed. The advert path is gated on `cfg.is_public()`;
inside that branch the daemon picks an address by precedence
(`external_addr`, non-wildcard `bind_addr`, STUN). The docs
treated `public: true` and `external_addr` as alternatives when
they are stacked: `public: true` is the master switch and
`external_addr` populates the address inside it. Reconciled
across `enable-nostr-discovery.md` and `advertise-your-node.md`:
add `public: true` to the `external_addr` examples; replace
"STUN as a logging cross-check" with "STUN is skipped entirely";
fix "neither flag is needed" for direct public bind (both flags
still required); make the publish-tutorial Step 3 conditional
on the chosen Step 2 path (STUN runs only on the `public: true`
path); rewrite the troubleshooting "wrong public IP advertised"
bullet with two coherent fixes.
- `udp:nat` overpromised as a symmetric-NAT solution. Symmetric
NAT on either side typically defeats the punch. Reframe
`udp:nat` as best-effort hole-punching for nodes without a
directly reachable UDP endpoint in the how-to, the publish
tutorial (intro, callout, section heading rewrite from "If
you're behind symmetric NAT" to "If your direct UDP advert
isn't reachable"), the consume tutorial's "What's next"
pointer, and `tutorials/README.md`. Promote reachability over
named NAT classes: STUN can confirm the public IP but not that
the listener-port mapping is open.
- YAML "silently ignores unknown keys" is wrong. Config parser
rejects unknown fields via `serde(deny_unknown_fields)` on the
per-section structs; misspelled fields refuse the daemon's
start with a parse-error line in the journal. Fixed in the
publish tutorial's troubleshooting and the open-discovery
tutorial's `policy` typo bullet.
Mechanical fixes:
- Repoint stale anchors. `getting-started.md` and
`configuration.md` linked to `#installation` / `#inspect` on
the README; the README has no such headings. Repoint to
`#quick-start` and `cli-fipsctl.md`. Two stale anchors in the
publish tutorial pointing at non-existent sub-scenarios in the
how-to (`#sub-scenario-2c-...`,
`#sub-scenario-2b-tor-onion-node`) repointed to the correct
anchors.
- Drop the `fipsctl show status` claim from the open-discovery
troubleshooting bullet (`show_status` doesn't include
`discovery.nostr.policy`). Replace with daemon startup logs.
- Fix the `advertise: false` parenthetical in the consume-only
tutorial (`default_advertise()` returns `true`; we set `false`
explicitly for the consume-only path).
- Drop the "supplies a relay list" overstatement in two
activation paragraphs (the how-to and the design doc). Default
relay / STUN-server lists ship in the config; both are
optional overrides.
- Add the missing `transports.udp.public` entry to the
open-discovery tutorial's prerequisites checklist. Tutorial
users coming out of advertise-your-node could be on either the
direct-UDP (`public: true`) or `udp:nat` (`public: false`)
path; list both.
Files: docs/getting-started.md, docs/reference/configuration.md,
docs/how-to/enable-nostr-discovery.md, docs/tutorials/README.md,
docs/tutorials/advertise-your-node.md,
docs/tutorials/resolve-peers-via-nostr.md,
docs/tutorials/open-discovery.md,
docs/design/fips-nostr-discovery.md.
Four short prose corrections folded together to align operator-facing
docs with current v0.3.0 reality:
- README Rust prerequisite reconciled with the toolchain pin (1.94.1,
was 1.85+).
- CONTRIBUTING.md bumps the Rust prerequisite and adds the squash-merge
policy note.
- examples/sidecar-nostr-relay/Dockerfile drops stale --features tui
and the paired --no-default-features; the tui/ble/gateway cargo
features were replaced by platform cfg gates in cbc7809.
- README Features list adds two v0.3.0-visible items: the mesh-
interface security baseline (fips.nft conffile, fips.d/ drop-in,
opt-in fips-firewall.service across all packaging formats) and the
fipsctl stats time-series queries plus fipstop inline sparkline
dashboards.
Status badge bump (v0.3.0--dev to v0.3.0) is deferred to tag time per
the release-prep checklist.
The generic systemd install tarball is the catch-all install path
for systemd Linux distros that don't have a per-format package
(Fedora, RHEL/CentOS, openSUSE, Alpine, etc.). It had drifted
behind the .deb and AUR packages and was missing fips-gateway, the
mesh-interface firewall baseline, and the multi-backend DNS helper
in the shipped tarball. Bring it to parity:
- New `packaging/systemd/fips-gateway.service` (clone of the .deb
unit; ExecStart pointed at `/usr/local/bin/fips-gateway`). Not
enabled at install time; operator opt-in.
- New `packaging/systemd/fips-firewall.service` (clone of the .deb
unit; nft path unchanged at `/usr/sbin/nft`). Not enabled at
install time; operator opt-in.
- `build-tarball.sh` now bundles the `fips-gateway` binary, the two
new units, the `fips.nft` baseline conffile, and the
`fips-dns-setup` / `fips-dns-teardown` multi-backend helpers from
`packaging/common/`.
- `install.sh` now installs `fips-gateway` to `/usr/local/bin/`,
installs both new units to `/etc/systemd/system/` (without
enabling them), preserves `/etc/fips/fips.nft` on upgrade like
`fips.yaml`, and creates the `/etc/fips/fips.d/` operator drop-in
directory. Post-install messaging mentions both opt-in services.
- `uninstall.sh` stops and disables the optional services in
dependency order (firewall, gateway, dns, daemon), removes the
new unit files, and removes the gateway binary. `--purge` already
handles `/etc/fips/` removal which covers `fips.nft` and
`fips.d/`.
- `README.install.md` documents all of the above: expanded
"What Gets Installed" table, new sections covering the firewall
baseline and the LAN gateway, refreshed DNS section reflecting
the multi-backend setup helper (systemd dns-delegate /
systemd-resolved drop-in / per-link resolvectl / dnsmasq /
NetworkManager-dnsmasq), and updated Service Management.
Also fixes a latent packaging bug: `install.sh` previously
referenced `${SCRIPT_DIR}/../common/fips-dns-setup`, a path that
exists only in the source-repo layout and not in the extracted
tarball. The script now resolves the helper from the staging
directory first (the tarball case), falling back to the source-repo
relative path. Bug latent since the multi-backend DNS helpers
landed.
CHANGELOG `[Unreleased]` documents the parity bump under Changed
and the path-resolution fix under Fixed.
Closes the longest-standing parity gap for non-Debian / non-Arch
systemd Linux distros installing from the release-distribution
tarball.
Three changes folded together close the AUR-side parity gap with the
.deb packaging:
- PKGBUILD now ships fips.nft baseline and fips-firewall.service, with
fips.nft marked as backup so operator edits survive upgrades.
- PKGBUILD-git mirrors the release PKGBUILD: installs fips-gateway
(was missing entirely), ships fips.nft and fips-firewall.service,
and tracks the same backup() set.
- packaging/aur/README.md documents the gateway, firewall service,
and nft baseline that ship as of this revision.
AUR users now receive the same artifact set as .deb users.
The Noise-session AEAD swap landed as 5cda4a9 + 9b1016f without a
companion CHANGELOG entry; add it under [Unreleased] § Changed
ahead of the rest of the perf-win cluster from PR #81 since it's
the most operator-visible single change of that class.
Freshly-restarted nodes with policy: open silently lost the historical
event replay that relays send in response to subscribe(). The
broadcast::Receiver was created INSIDE spawn_notify_loop, which the
tokio runtime starts at some indeterminate point after subscribe()
returns. tokio's broadcast channel only delivers messages sent after
the receiver is created; messages dispatched in the gap between
subscribe() issuing the REQ and the spawned task calling
client.notifications() were dropped by external_notification_sender.send
returning Err(SendError) with no subscribers attached.
Symptom on a node with policy: open: non-configured peers were not
discovered until they next re-published their advert (default
advert_refresh_secs = 1800s = 30 min). Configured peers were unaffected
because fetch_advert (relay-fetch path) caches them at startup-sweep
time. The bug has been latent since 34e00b9 added Nostr discovery —
relay-fetch covered the common case for configured-peer setups.
Fix: create the broadcast::Receiver in start() before subscribe() and
pass it into spawn_notify_loop. The receiver now exists when the REQ
replay arrives, so historical events flow through the cache path.
Also handle broadcast::error::RecvError::Lagged separately from
::Closed. The previous `while let Ok(...) = recv().await` exited the
loop on any Err, so a single lag event would silently kill the entire
subscription consumer with no recovery. Lagged now logs a warn (with
the skipped count) and continues; only Closed exits the loop.
Add two info-level log lines for in-field observability of the loop's
liveness. "nostr notify loop entered" fires once at task start; "nostr
notify loop received first event" fires once after the first
successful recv() with elapsed_ms since loop entry. Together these
turn the previous silent-failure shape (zero advert: peer cached
log lines indistinguishable between dead loop and idle channel) into
an immediately greppable startup signal — operators can confirm the
loop is alive and see how long it took to receive its first event,
catching any future regression in the subscription codepath in
seconds rather than waiting one advert_refresh_secs interval.
No public API change; the test fixture (NostrDiscovery::new_for_test)
does not call spawn_notify_loop and is unaffected.
Three Changed entries for the rx-path performance work
(Linux UDP recvmmsg batched receive, run_rx_loop drain batching, and
eager pubkey_full precompute on PeerIdentity construction) and five
Fixed entries: adopted NAT-traversed UDP transports inheriting the
primary listener's MTU and buffer config, TreeAnnounce ancestry on
self-root transitions, unconditional overlay-advert refetch before
each retry, stale overlay-advert eviction on NoTransportForType,
and scheduled retry on startup peer-init failure.
Pure CHANGELOG addition (+117 lines, no edits to existing entries).
Bullets are wrapped at 80 columns and attribute external
contributions to the originating PR and author.
The drop-counter sanity check piped `nft list table inet fips`
through `awk '/counter packets/ {print $3; exit}'`. Awk's `exit`
on first match closes the pipe, the upstream `nft list` SIGPIPEs
on its next write, `set -o pipefail` makes the pipeline return
141, and the surrounding command-substitution aborts the script
before it can assign DROP_PKTS or print the section header.
Replaces the early-exit pattern with `/counter packets/ && !seen
{ print $3; seen=1 }` — same first-match output, but awk reads
the full input so nft never SIGPIPEs.
The original form had been latent for as long as the test has
existed; recent CI runs at master tip 53ad528 finally tripped
it (output shows the script dying immediately after the case-(d)
PASS, before "=== Drop counter incremented..." prints).
Verified locally: `bash testing/ci-local.sh --only firewall`
runs all six setup-and-functional steps green and prints
"PASS: drop counter = 5".
Mirrors systemd's RuntimeDirectory=fips so the daemon's
resolve_default_socket() picks /run/fips/control.sock inside
containers, matching production layout and the path that the
chaos sim harness (testing/chaos/sim/control.py) probes.
Without this, the resolver falls through to
/tmp/fips-control.sock (no /run/fips, no XDG_RUNTIME_DIR), the
daemon binds there, and the harness's hardcoded
/run/fips/control.sock probe returns FileNotFoundError on every
node. chaos-bloom-storm then fails its bloom_send_rate assertion
with start=0 nodes, end=0 nodes; other chaos scenarios pass only
because their tolerances absorb the empty samples.
Production hosts always have /run/fips materialized by the
fips.service unit's RuntimeDirectory directive before the daemon
starts, which is why the regression hit only the containerized
test path.
Verified locally with chaos-bloom-storm: max per-node delta 27
<= ceiling 30 over the trailing 30s window.
Surfaces local services reachable from the mesh, paired with their
current `inet fips` baseline filter classification. Lands to the
right of the existing TUN section in the Traffic block.
A new daemon control query `show_listening_sockets` returns IPv6
listeners bound to either `::` (wildcard) or the node's fd00::/8
address, each classified as Accept / Drop / Unknown / NoFirewall
against the running inbound chain. fipstop renders the result as a
table beside the Traffic counters: Accept rows in default White,
Drop / Unknown in DarkGray, a yellow banner above the table when
`fips-firewall.service` is inactive, and a trailing `*` on
wildcard binds to remind the operator the bind is not
fips0-specific.
Daemon side:
- `src/control/listening.rs` walks `/proc/net/tcp6` and
`/proc/net/udp6` via the procfs crate (LISTEN state for TCP,
wildcard remote for UDP), filters to fips0-reachable binds, and
resolves inodes to PID / comm via `/proc/<pid>/fd`.
- `src/control/firewall_state.rs` shells out to
`nft -j list table inet fips` and walks the inbound chain.
Recognises canonical accepts (`tcp/udp dport N accept`,
`dport { ... } accept`, `dport A-B accept`), the iifname-scoping
line, conntrack and icmpv6 lines (skipped). Any rule with
unrecognised matchers (saddr filters, jumps, daddr filters) or
non-terminal verdicts forces Unknown classification for the
ports it references. Eleven unit tests cover the classification
logic; the listening enumerator carries a /proc-parsing test of
its own.
- `show_listening_sockets` emits
`{fips0_addr, firewall_active, sockets[]}` with per-row
`{proto, local_addr, port, pid, process, filter, wildcard_bind}`.
fipstop side:
- `src/bin/fipstop/ui/dashboard.rs` splits the Traffic block into
a 50/50 horizontal layout; the existing TUN + Forwarded panel
occupies the left half.
- `src/bin/fipstop/ui/listening.rs` renders the right half.
- `main.rs` fetches the new query each tick when the Node tab is
active. Errors are non-fatal: an old daemon without the query
leaves the payload at None and the panel renders "loading...".
`Cargo.toml` gains `procfs = "0.18"` on the Linux target. IPv4
listeners are not enumerated — fips0 is IPv6-only.
Folded in: revert the default-socket lookup from writability-probe
back to existence-based selection. The previous tempfile-probe on
`/run/fips` silently steered fipstop / fipsctl onto an XDG path
the daemon never bound for any user in the `fips` group whose
shell session had not yet picked up the supplementary group (no
re-login after `usermod -aG`). `XDG_RUNTIME_DIR` is set on every
modern systemd-managed user session, so this hit the common case.
The kernel checks actual group membership at `connect(2)`, so a
user who genuinely cannot connect now gets a clear `EACCES`
rather than a silent path mismatch. Drops the now-unused
`is_writable_dir` helper. `XDG_RUNTIME_DIR` existence validation
is preserved.
Documentation:
- `docs/reference/cli-fipstop.md` — Node-tab row updated, new
"Listening on fips0 panel" section.
- `docs/reference/control-socket.md` — `show_listening_sockets`
added to the read-only queries table.
- `docs/how-to/enable-mesh-firewall.md` — new "Verify with
fipstop" section.
- `docs/tutorials/host-a-service.md` — fipstop callouts at
Steps 3, 5, 6 + Troubleshooting bullet + wildcard-bind reminder
under "What you've learned".
- `CHANGELOG.md` — new bullet under `Added / Operator Tooling`,
resolver `Fixed` entry rewritten to describe the
existence-based final shape.
Six-node depth-4 mesh with an induced upstream parent flap. Asserts a
trailing-window ceiling on per-node `stats.bloom.sent` and a sanity
floor on parent-switch count over a ~3-4 min observation window.
Guards against the regression class where a spanning-tree update that
changes only an internal path edge (no root or depth delta) fails to
be properly contained and instead propagates to leaves as a sustained
bloom-traffic oscillation, visible only at fleet scale and only after
several minutes of uptime.
Adds a new chaos primitive (`link_swap`) for deterministic asymmetric
link-cost flapping and a post-run assertion framework with two
checks:
- `bloom_send_rate.max_per_node`: trailing-window ceiling on the
`show_bloom` stats counter delta. Calibrated against the
post-mortem reproduction harness data (per-variant counter table
against pre-fix vs post-fix binaries).
- `min_parent_switches.min_total`: sanity guard against a
misconfigured harness where the flap inducer fires but the
topology never produces a real parent-switch event (e.g., wrong
root election from a different seed). Without this, the
bloom-rate assertion would trivially pass on any binary
including a regressed one.
The runner exits 3 on assertion failure (alongside 0 success and 2
panic-detected). Threshold derivation is documented in the scenario
README; the seed pin is also documented there since smallest-NodeAddr
root election is sensitive to the pubkey hash ordering.
Wired into ci-local.sh's chaos pool and the GitHub CI chaos matrix.
Routine refresh; raises MSRV to 1.71 (non-issue for our 2024-edition
toolchain) and updates windows-sys to 0.61. FIPS uses
define_windows_service!, service_main, the Error type, and
Error::Winapi - all stable across 0.7 -> 0.8.
Windows CI matrix is the verification gate; no live Windows nodes.
Pulls netlink-packet-route 0.30.0, which adds DEVCONF_FORCE_FORWARDING
to Inet6DevConf for kernel 6.17+. Closes the IFLA_INET6_CONF WARN
observed on kernel-6.17 hosts during fips startup.
Zero source edits: FIPS does not use the deprecated
link_local_address API or the renamed StablePrivacy display path.
Live-host WARN-absence verification on a kernel-6.17 host is
scheduled for a separate deploy.
Closes RUSTSEC-2026-0097 (unsoundness with custom logger calling
rand::rng() from the log handler). Fix is the upstream deprecation
of the `log` feature; no API change for our pin.
Both variants of SessionMessageType were never emitted anywhere
in src/, and the production from_byte dispatch sites lacked
Some-arms for them — any 0x00/0x01 byte that reached either
dispatcher would log "Unknown..." and drop. The matching rustdoc
tables described an Offset 0 msg_type byte that the encode() path
has never written; the actual wire format is the FSP common prefix
[ver_phase][flags][payload_len:2 LE] with body keyed by phase
nibble, as documented in docs/reference/wire-formats.md.
Drop the variants, drop their from_byte/to_byte/Display arms, fix
the two stale rustdoc tables to describe the real wire shape, and
trim the variant-iteration unit test that enumerated them.
Zero on-wire behaviour change.
The Ethernet data frame format is `[type:1][length:2 LE][payload]`,
so the per-link payload MTU is the interface MTU minus 3 bytes,
not minus 1. The 2-byte length field is required to trim NIC
minimum-frame padding before AEAD verification.
The implementation in src/transport/ethernet/mod.rs already uses
saturating_sub(3) correctly; only the rustdoc on the effective_mtu
field and the EthernetConfig.mtu field's documentation lagged behind.
No behaviour change.
Two small improvements to the OpenWrt gateway deploy tutorial:
- Add a router-side ping step at the top of Step 4 (post-gateway-start
client test). Confirms the router itself reaches the mesh before
bringing the LAN segment into the diagnosis: if this fails the
troubleshooting target is the daemon / mesh side; if it succeeds
and the LAN-client test below fails, the target is the LAN segment
(proxy_ndp, RA pool route, or DNS forwarding through dnsmasq).
- Mark the inbound port-forward section heading as Optional. The
outbound half is the steady-state use of a gateway and applies to
every deployment; the inbound port-forward half is a per-service
opt-in that many operators won't need.
Bring the packaging README into agreement with what the ipk actually
installs and the CLI surface fipsctl exposes today:
- Package contents table now lists /usr/bin/fips-gateway,
/etc/init.d/fips-gateway, and /etc/sysctl.d/fips-gateway.conf
alongside the daemon. These have been part of the install block
but were missing from the README.
- fipsctl examples updated to the current command form
(fipsctl show peers / show links / show sessions in place of the
removed shorthands), with a pointer to the canonical CLI reference.
- Service management section gains a short subsection covering the
optional gateway service, including the enable/start incantation
and a link to the deploy-fips-gateway tutorial.
The gateway is designed for systems already serving DHCP and DNS to
a LAN segment (canonically an OpenWrt AP). On those systems port 53
is already taken by the existing resolver, so the prior `[::]:53`
default conflicted with the gateway's intended deployment target out
of the box.
The OpenWrt ipk previously overrode this in its packaged config as a
workaround; matching the source default to what the canonical
deployment actually wants makes the override redundant and removes a
foot-gun for fresh manual Linux-host installs. The redundant
`dns.listen` line in `packaging/openwrt-ipk/files/etc/fips/fips.yaml`
is dropped along with this change.
Operators on a host without a pre-existing resolver on port 53 can
opt back into the wildcard bind by setting `dns.listen: "[::]:53"`
explicitly. The new default binds IPv6 loopback only — Linux IPv6
sockets bound to explicit `::1` do not accept v4-mapped traffic, so
forwarders that reach the gateway over IPv4 loopback need to be
pointed at an explicit IPv4 listen address instead.
Touches the gateway config struct and its default-value test, the
commented-out gateway example in the Debian common fips.yaml, the
OpenWrt ipk config (override removed), the gateway reference /
how-to / design / tutorial / troubleshoot docs, and a CHANGELOG
entry under [Unreleased] -> Changed.
- Add a Documentation entry covering the docs/ reorganisation,
top-level getting-started.md, per-section landing pages,
source-accuracy pass, and gateway feature-set rewrite.
- Add a Fixed entry covering propagation of spanning-tree updates
whose changes are confined to internal path edges (no root or
depth delta).
- Add a single rolled-up entry covering expanded test coverage
across the new-feature surface plus CI hardening.
- Drop a tree-ancestry test-determinism bullet that did not change
user-visible behaviour.
Daemon and client tools previously evaluated the same three locations
(`/run/fips`, `XDG_RUNTIME_DIR`, `/tmp`) in different orders, allowing
fipsctl/fipstop to connect to a socket the daemon never bound when
neither side set `node.control.socket_path` explicitly.
Collapse the three call sites (`default_control_path`,
`default_gateway_path`, `ControlConfig::default_socket_path`) into a
shared `resolve_default_socket` helper. Canonical order is
`/run/fips` -> `$XDG_RUNTIME_DIR/fips/` -> `/tmp/fips-<name>`. Two
hardening fixes folded in: writability is probed via tempfile create
rather than mode bits (ACL- and group-aware), and `XDG_RUNTIME_DIR`
is validated as an existing directory before being used (avoids
stale post-logout values).
The deployed fleet is unaffected -- packaged configs set
`node.control.socket_path` explicitly. The fix surfaces for dev
runs and the binary-install getting-started path.
- Status badge v0.2.0 → v0.3.0-dev.
- Lede rewritten around the two equally-supported deployment
modes (overlay on existing IP networks; ground-up over raw
Ethernet, WiFi, Bluetooth) matching docs/README.md and
docs/getting-started.md.
- Features list refreshed: Nostr-mediated discovery and UDP NAT
traversal called out, LAN gateway described as both halves
(outbound + inbound port forwarding), peer ACL and
control-socket-per-binary noted.
- Quick start trimmed to the Debian inline path + pointer at
docs/getting-started.md for the multi-platform walkthrough;
transport-by-platform matrix retained.
- Documentation section reorganised around the four-section
docs/ tree (tutorials, how-to, reference, design) with one
entry-point pointer per section.
- Stale doc links fixed (docs/design/fips-intro.md →
docs/design/fips-concepts.md; docs/design/fips-configuration.md
no longer linked).
- Status & roadmap rewritten for the v0.3.0-dev release-line
scope (no new wire-format changes; FMP swap deferred to the
next-branch post-v0.3.0 line).
422 → 235 lines.
Restructures /docs/ by reader purpose (tutorials, how-to,
reference, design), adds the new-user-progression and
operator-recipe content the prior layout lacked, runs an
accuracy pass against current source across the pre-existing
design docs, and rewrites the gateway feature-set documentation
end-to-end around its actual operational profile (a niche
feature designed for systems already serving DHCP/DNS to a
LAN, with two independent halves — outbound LAN→mesh, inbound
mesh→LAN — sharing one nftables table, one binary, and one
control socket). Top-level README and getting-started rewritten
around two equally-weighted deployment modes (overlay on
existing IP networks; ground-up over non-IP transports).
## Additions
- 11 new tutorials in docs/tutorials/: an 8-step new-user
progression from single-daemon test-mesh peering through
to a ground-up two-device mesh, an IPv6-adapter side-trip
walkthrough, an Advanced Tutorials index, and a hand-held
OpenWrt walk-through for fips-gateway deployment that
exercises both halves of the feature.
- 12 new how-tos in docs/how-to/: firewall activation,
Nostr discovery (resolve / advertise / open across five
scenarios), Tor onion (directory + control_port modes),
UDP buffer tuning, unprivileged-user setup, persistent
identity, host aliases, Bluetooth LE peering, MTU
diagnostics, manual Linux-host gateway deployment (covers
both halves), gateway troubleshooting (organised by half),
and a section index.
- 9 new reference docs in docs/reference/: configuration,
wire formats, control-socket protocol, four CLI references
(fips, fipsctl, fipstop, fips-gateway), security posture
matrix, and Nostr events catalog. Configuration and
wire-formats are renamed-and-extended from prior design/
versions; the other seven are net-new.
- 6 new design docs: fips-concepts, fips-architecture, and
fips-prior-work split out of the deleted fips-intro.md;
consolidated fips-mmp and fips-mtu aggregations; and a
new generic port-advertisement-and-nat-traversal doc
(Nostr-signaled port advertisement plus UDP NAT-traversal
protocol, FIPS as an example implementation, suitable for
eventual NIP submission).
- Top-level docs/getting-started.md walking through the
binary-installer-only Install story.
- packaging/common/hosts pre-populated with the eight public
test-mesh nodes so shortnames resolve out of the box on
every fresh install.
## Changes
- 23 wire-format diagrams relocated to reference/diagrams/
alongside the wire-formats move.
- 4 design diagrams corrected against source code
(fips-protocol-stack, fips-identity-derivation,
fips-coordinate-discovery, fips-routing-decision).
- 10 pre-existing design docs reconciled with current
source. Numeric corrections: stale link-MMP report bounds
(now [1s, 5s] with 200 ms cold-start floor); UDP default
MTU (now 1280, IPv6 minimum); node_addr formula
(SHA-256(pubkey)[..16]); Noise patterns (IK at link, XK
at session); peer-ACL semantics (strict allowlist requires
ALL in peers.deny); daemon DNS upstream ([::1]:5354);
on-the-wire bloom-filter size (1,071 bytes); obsolete
Cargo-feature references (PR #79 dropped them) removed.
- Transport framing tightened across the docs: TCP is for
UDP-filtered networks (not NAT traversal); Tor is a
deployment mode (not failover); WebSocket dropped (not a
shipped FIPS transport); WiFi promoted to Implemented via
Ethernet in infrastructure mode; classic-Bluetooth row
removed (BLE is the only Bluetooth-mode transport).
- docs/design/fips-gateway.md rewritten end-to-end to lead
with the niche-feature framing and the two-halves
structure. Title moved from "FIPS Outbound LAN Gateway"
to "FIPS Gateway"; architecture section describes the
common machinery (the fips-gateway service, the nftables
table, the control socket) before splitting into separate
"Outbound Half" and "Inbound Half" sections of equal
weight; security considerations split per-half; no Future
Work section (speculative directions live in the project
tracker, not in protocol design docs). Inbound port
forwarding is a first-class half rather than a buried
"Implemented Extensions" subsection.
- Gateway terminology unified across all gateway docs as a
separate Linux service running alongside the fips daemon
(its own systemd unit / OpenWrt init script). Container-
pattern terms (sidecar) are reserved for the
Docker/Kubernetes sidecar deployment examples — the
testing/sidecar/ tree, examples/k8s-sidecar/,
examples/sidecar-nostr-relay/,
examples/wireguard-sidecar-macos/, and the related
CHANGELOG / top-level README entries — where the term
carries its standard container meaning.
- Net-new design body content: rekey section in
fips-mesh-layer (Noise IK msg1/msg2 over the established
link, K-bit cutover, drain window, smaller-NodeAddr-wins
tie-breaker on dual-init); Mesh Size Estimation and
Antipoison FPR Cap sections in fips-bloom-filters;
Mesh-Interface Query Filter subsection in
fips-ipv6-adapter; failure-suppression knobs and clock-
skew tolerance in fips-nostr-discovery; loop-rejection
and mid-chain ancestor swap added to spanning-tree
propagation / stability rules; Priority Chain in
fips-mesh-operation renumbered to match the
routing-decision diagram.
- Top-level README: dropped the stale nostr-discovery
cargo-feature parenthetical. docs/README.md and the four
section READMEs (tutorials, how-to, reference, design)
refreshed for the new structure; index rows reflect both
halves of the gateway feature and the new fips-gateway
CLI reference.
- Cargo.toml [package.metadata.deb] assets path updated for
the fips-security.md move; .gitignore /reference/ rule
anchored to repo root so docs/reference/ is trackable.
- packaging/openwrt-ipk/files/etc/fips/fips.yaml
configuration-doc URL updated to the new
docs/reference/configuration.md location.
## Deletions
- docs/design/fips-intro.md (split into the three new intro
design docs).
- docs/design/document-relationships.svg (orphan, no longer
referenced).
- docs/proposals/ tree removed; the only proposal it
contained (the Nostr UDP hole-punch protocol) was
rewritten as the new generic
design/port-advertisement-and-nat-traversal.md.
A leaf node's my_coords could go stale after an upstream
mid-chain ancestor swap, leaving non-parent destinations with
100% loss until either the parent or the depth also changed.
The broadcast gate in handle_tree_announce's
`else if !is_root && parent_id == from` branch compared only
(root, depth). A swap that altered an interior ancestor without
changing root or depth (e.g. A->B->C reorganizing to A->D->C
while keeping (A, depth=2)) was silently dropped one hop below
the swap node. Downstream nodes' coords paths then drifted from
the real tree topology, defeating greedy distance routing for
any destination whose path crossed the unrepresented section.
Widen the gate to compare the full my_coords.node_addrs() so
mid-chain swaps propagate to leaves the same way root/depth
changes already did. The gate body's bloom-marking is adjusted
in step so the wider gate doesn't generate empty/redundant
FilterAnnounces to every peer on every mid-chain swap
propagation: mark_changed_peers replaces
mark_all_updates_needed in the gate body (parent_id is
unchanged in this branch, so outgoing filter content is
typically unchanged, and mark_changed_peers correctly marks
zero peers in that case), and the unconditional
mark_update_needed(*from) at the top of handle_tree_announce
is removed (bloom exchange initiation is already handled at
handshake completion, and ongoing content changes are picked
up naturally by mark_changed_peers in handle_filter_announce
when peers send their next filter).
Required surface change: peer_inbound_filters in
src/node/bloom.rs upgraded from private to pub(super) so the
gate body can call it.
Verified in a 6-node depth-4 docker reproduction under
tc/netem-induced parent flapping: a depth-4 leaf's
ancestry_changed counter advances with upstream parent
switches while bloom_sent stays at zero matching the
pre-change steady-state baseline.
Open-discovery NAT traversal succeeds at the UDP layer regardless
of what FMP-protocol version the peer speaks. When the daemon
discovers a peer running a different FMP version (e.g. a v0/v1 mix
during a mid-rollout window, or a misconfigured peer in the same
advert namespace), the punch sequence completes, the socket is
adopted via `Node::adopt_established_traversal`, and we initiate
an FMP handshake. The peer drops our msg1 at its own version-gate
and we drop their msg1/msg2 at `Unknown FMP version, dropping`.
Neither side advances the handshake.
Today the bootstrap transport sits idle until the 31s stale-
handshake timeout, drops, and the open-discovery sweep ~30s later
fires the full STUN+offer+answer+punch sequence again — every
minute, indefinitely, against peers the handshake literally cannot
complete with.
Add a `Node::bootstrap_transport_npubs` map populated alongside
`bootstrap_transports` at adopt time. The rx loop reverse-maps the
transport_id → npub on version-mismatch and bumps the discovery
layer's `failure_state` to a long structural cooldown via the new
`NostrDiscovery::record_protocol_mismatch` API. The next sweep
skips the npub for `protocol_mismatch_cooldown_secs` (default
86400 = 24h, separate from the 30-min transient-failure
`extended_cooldown_secs`).
One-shot WARN per fresh observation. Repeat mismatches inside the
cooldown window are silent (the failure_state method returns false
when an existing comparable cooldown is already in place). The
handshake/transport teardown chain is unchanged — the fix is
specifically about preventing the *next* sweep cycle from
re-traversing.
Cleared on `cleanup_bootstrap_transport_if_unused` and on the
adopt-failure rollback path so completed handshakes don't leave
stale entries behind.
Four new unit tests in `failure_state.rs` cover fresh-entry
signaling, repeat-suppression inside the window, streak-pin
behavior for `show_peers` rendering, and post-cooldown re-arming.
The TUN-side TCP MSS clamp consults `path_mtu_lookup` (FipsAddress-
keyed) when sizing outbound TCP flows. Until now, only the reactive
`MtuExceeded` handler mirrored the bottleneck MTU into that store;
the proactive end-to-end `PathMtuNotification` echoed by the
destination updated only `MmpSessionState.path_mtu`, leaving the TUN
mirror stale.
On stable long-lived paths, the proactive echo can tighten the
session-canonical MTU well before any transit router fires a
`MtuExceeded` for those flows (since all current traffic is already
sized by the tighter session value). New TCP flows opened during
that window get clamped by the discovery-time value rather than the
session-canonical one, leading to PMTU-D loss until the reactive
path eventually fires.
Mirror the post-apply MTU into `path_mtu_lookup` whenever
`apply_notification` returns true, with the same tighter-only
semantics as the reactive mirror — never loosen the clamp. Gated on
the bool return so spurious writes don't happen on rejected
increases or no-op same-value notifications.
Four new unit tests exercise the empty-lookup write, tighten-
existing, keep-tighter-existing, and no-session-no-op paths,
parallel to the existing reactive-mirror test trio.
- Reorganize Added into 11 subsections ordered by importance and
protocol layer: Mesh Layer (FMP), Platform Support, Mesh Peer
Transports, Security, LAN Gateway, IPv6 Adapter, Operator Tooling,
Packaging and Deployment, Examples, Documentation.
- Add entries for peer ACL enforcement (#50), MIPS portable_atomic
(#62), inbound mesh port forwarding, and historical node and
per-peer statistics with btop-style graphs (#64).
- Add a Fixed entry covering the TCP-over-FIPS reliability work
(transport_mtu determinism, per-destination TCP MSS clamp at the
TUN boundary, reactive MtuExceeded mirror, Windows TUN reader
plumbing).
- Recast the multi-backend DNS configuration entry as a systemd-host
overhaul under IPv6 Adapter (default ::1 bind, global drop-in
backend, five-distro test harness); fold the Ubuntu 22 Fixed entry
into it.
- Expand the cargo feature flag rationalization Changed entry to
cover PR #79's full scope: tui, ble, gateway, and nostr-discovery
all dropped.
- Drop the rekey msg1 Fixed entry; all cases require new-in-release
functionality.
- Collapse the BLE transport bullets into one consolidated bullet
under Mesh Peer Transports and scrub stale cargo-feature wording
from the overlay-discovery and BLE bullets.
When a UDP transport had `advertise_on_nostr: true` + `public: true`
+ `bind_addr: 0.0.0.0:NNNN`, the advert builder previously read the
kernel's `local_addr()`, found `0.0.0.0`, filtered it out (correctly
— wildcard isn't a valid advertised endpoint), and silently emitted
no UDP endpoint in the published advert. Operators on AWS EIP / GCP
/ Azure setups (where binding to the public IP directly is impossible
because 1:1 NAT does the address translation off-host) had no way to
advertise UDP without binding to a specific local IP — and no log
explaining what was happening. TCP had the same shape, with no
`public: true` precondition.
Three pieces, layered. UDP gets zero-config autodiscovery via STUN;
both UDP and TCP get an explicit operator-supplied override; the
fall-through path now logs loudly instead of silently skipping.
UDP public-IP autodiscovery (STUN)
----------------------------------
In the UDP `is_public()` + wildcard-bind branch, run a one-shot
STUN observation against an ephemeral UDP socket on the daemon's
configured `stun_servers`. Take the reflexive IPv4 (the
STUN-reported port is the ephemeral source port and is discarded),
combine with the configured listener port for the advert
(`udp:<reflexive-ip>:<port>`). Works on AWS EIP / GCP / Azure
1:1-NAT setups because STUN sees the public-Internet egress IP and
the bind port is preserved through 1:1 NAT.
Result is cached per-transport on a new `public_udp_addr_cache`
field on `NostrDiscovery` (keyed by `TransportId.as_u32()`).
Asymmetric cache TTL: a successful observation is cached for
`advert_refresh_secs` (default 30 min) so we don't STUN every
refresh tick. A failed observation is cached for only 60s
(`PUBLIC_UDP_ADDR_FAILURE_TTL`) so a transient STUN flake at
startup retries within ~a minute and the advert grows its UDP
endpoint as soon as STUN starts working — rather than waiting the
full 30-min cycle.
The shared `observe_traversal_addresses` STUN helper had a
hard-coded 2s per-server response wait, right for the
per-traversal flow (latency-sensitive — 3 STUN servers worst-case
= 6s) but too short for the one-shot advert-publish startup
discovery. Parameterized `per_server_timeout` on the helper, with
two named constants in `stun.rs`: `TRAVERSAL_STUN_TIMEOUT = 2s`
(existing call sites) and `ADVERT_STUN_TIMEOUT = 5s` (new
public-UDP discovery path). Both use `tokio::time::timeout_at`
under the hood, so success returns immediately — the timeout is
only the worst case.
`external_addr` override (UDP + TCP)
------------------------------------
New `external_addr: Option<String>` field on `transports.udp.*`
and `transports.tcp.*` for explicit advertise-as override. Takes
precedence over both the bound `local_addr` and (for UDP) the
STUN-derived autodiscovery.
Required for TCP on cloud-NAT setups (AWS EIP, GCP/Azure external
IPs) where binding to the public IP directly fails with
`EADDRNOTAVAIL` because the public IP isn't on a host interface —
the network fabric does 1:1 NAT off-host. Without this field the
operator's only TCP path was "leave advert off" or "find a way to
make the public IP locally bindable."
For UDP, `external_addr` is optional but useful as a deterministic
alternative to STUN. Operators who want to skip STUN egress, whose
STUN servers are blocked, or who want the daemon to not depend on
external services for advert content can specify it explicitly.
The accessor parses two shapes:
- Bare IP (`"54.183.70.180"` or `"2001:db8::1"`): combines with
the configured `bind_addr` port.
- Full host:port (`"54.183.70.180:8443"` or `"[2001:db8::1]:443"`):
used verbatim — useful for port-forward setups where the
externally-visible port differs from the bind port.
Final precedence in `Node::build_overlay_advert` (now async, only
caller `refresh_overlay_advert` was already async):
- UDP: `external_addr` → non-wildcard `local_addr` → STUN → loud warn
- TCP: `external_addr` → non-wildcard `local_addr` → loud warn
Loud warns instead of silent skips
----------------------------------
The wildcard-bind fall-through paths now log a `warn!` pointing at
the operator-side fixes:
- UDP: "set transports.udp.external_addr, bind to a specific
public IP, or ensure node.discovery.nostr.stun_servers is
reachable"
- TCP: "Either set external_addr to the public IP (recommended for
cloud 1:1-NAT setups) or bind explicitly to the public IP"
Replaces the silent skip that previously cost operators a
debugging session when the advert mysteriously contained only the
Tor onion endpoint.
Tests
-----
11 new unit tests in `src/config/transport.rs` covering the
parser (IPv4/IPv6, bare/full, malformed) and the accessor (UDP
with default bind, UDP with explicit port override, UDP unset,
TCP without bind_addr, TCP with bind_addr, TCP with full
socket-addr override, parse_bind_port for IPv4/IPv6/malformed).
The 38-test nostr suite still passes.
CHANGELOG entries under `[Unreleased]` Fixed.
Public-test daemons with populous open-discovery caches generate
sustained NAT-traversal-failure WARN volume (~140/hour, ~3500/day)
against cache-learned peers that have gone offline — their adverts
are absent from major Nostr relays but cached entries persist until
their advertised `valid_until` expires. The daemon kept publishing
offers indefinitely under exponential backoff with no per-peer
suppression, drowning operator signal and hammering relays. A
parallel concern: the strict freshness check at signal.rs silently
rejected offers under modest clock skew (now_ms() anchors to
SystemTime once at startup, so post-startup NTP step adjustments
don't propagate on long-uptime daemons), indistinguishable from
"peer is offline."
Six independent improvements layered on the existing retry logic.
Per-npub WARN log rate-limit
----------------------------
New `FailureState` struct on `NostrDiscovery` records per-npub
`last_warn_at_ms`. Subsequent failures inside `warn_log_interval_secs`
(default 5 min) emit DEBUG instead of WARN. Each WARN now also
carries `consecutive_failures` and remaining `cooldown_secs` so
operators can read the trajectory without grepping multiple lines.
Per-npub consecutive-failure counter + extended cooldown
--------------------------------------------------------
After `failure_streak_threshold` (default 5) consecutive failures
against a peer, the next `extended_cooldown_secs` (default 1800)
of attempts are suppressed by pushing
`retry_pending[npub].retry_after_ms` past the cooldown wall. The
open-discovery sweep also consults `cooldown_until` and increments
a new `skipped_cooldown` counter so a peer whose `retry_pending`
was cleared by max_retries doesn't get re-enqueued during the
cooldown window. Caps offer-publish rate per dead peer regardless
of how often the sweep tries to re-enqueue.
Stale-advert eviction on streak-threshold transition
----------------------------------------------------
On the threshold-crossing transition (one-shot, not every
subsequent failure), `tokio::spawn` an active re-fetch of the
peer's Kind 37195 advert from `advert_relays`. Three outcomes:
- absent on relays → cache evicted; sweep won't re-enqueue
(peer is genuinely gone).
- newer `created_at` → cache refreshed + streak reset
(peer republished; allowed to retry immediately).
- same → cache untouched; cooldown stands.
Cost: ~one fetch per dead peer per 30-min cooldown cycle, vs
hundreds of offer publishes/hour today.
Clock-skew tolerance on freshness check
---------------------------------------
`signal.rs` `validate_offer_freshness` and
`validate_traversal_answer_for_offer` now allow ±60s grace beyond
strict TTL. Both return a new `FreshnessOutcome` enum so callers
can DEBUG-log when an offer/answer was only accepted via the grace
window. `FRESHNESS_SKEW_TOLERANCE_MS` is hard-coded — loosening
this past minutes erodes the freshness/replay security boundary
and operators tend to tune in the wrong direction.
NTP-style skew estimate (offer_received_at echo)
------------------------------------------------
Added optional `offerReceivedAt: Option<u64>` field to
`TraversalAnswer` payload. Responder fills it with `now_ms()` at
offer-receipt time. Initiator computes the standard NTP offset
formula `((T2-T1) + (T3-T4)) / 2` against the round-trip and
DEBUG-logs when `|skew| ≥ 30s`. Skew is also stashed in
`FailureState` and surfaced in `show_peers`. Non-breaking — older
responders that don't fill the field still produce valid answers,
and `estimate_clock_skew` returns `None`.
Per-peer state in `show_peers` JSON
-----------------------------------
Each peer entry in `show_peers` now carries:
"nostr_traversal": {
"consecutive_failures": <u32>,
"in_cooldown": <bool>,
"cooldown_until_ms": <u64 | null>,
"last_observed_skew_ms": <i64 | null>
}
Always emitted (schema-stable); values populated when discovery is
enabled and the npub has a recorded entry. Required a new public
`Node::nostr_discovery_handle()` accessor and refactored
`FailureState`'s internal Mutex from `tokio::sync` to `std::sync`
(operations never hold across await), which lets the synchronous
`show_peers` handler call `snapshot()` directly without the
dispatcher becoming async.
New config knobs (under `node.discovery.nostr`)
-----------------------------------------------
failure_streak_threshold: 5
extended_cooldown_secs: 1800
warn_log_interval_secs: 300
failure_state_max_entries: 4096
Tests
-----
12 new unit tests:
- 5 in `tests.rs` covering freshness strict / tolerated / rejected
outcomes, NTP skew estimation, and the backward-compat None case
when the responder didn't fill `offer_received_at`.
- 7 in `failure_state.rs` covering streak/warn-rate-limit state
transitions, cooldown active vs expired semantics,
success-resets-streak, observed-skew records, and size-cap
eviction by oldest `last_failure_at`.
CHANGELOG entries added under `[Unreleased]` Fixed.
The control-query snapshot tests panicked on the Windows GitHub runner
because git's default core.autocrlf=true converted fixture files
(src/control/snapshots/*.json) to CRLF on checkout, while the
in-memory JSON output is LF. trim_end() only strips trailing newlines,
not interior \r, so every snapshot comparison mismatched.
Two defenses:
1. .gitattributes: pin src/control/snapshots/*.json to text eol=lf so
future Windows checkouts keep the fixtures LF-only regardless of
local git config.
2. src/control/queries.rs (assert_snapshot): replace \r\n with \n in
the expected text before comparison, so any future
re-introduction of CRLF (a contributor with non-LF editor settings,
a different runner, etc.) doesn't surface as a snapshot mismatch.
The package-openwrt.yml shellcheck step was failing on warnings that
are false positives for OpenWrt scripts:
SC2034 — USE_PROCD, START, STOP in /etc/init.d scripts are read by
rc.common, not by the script itself; standard shellcheck
cannot see the indirection.
SC3043 — `local` is undefined in strict POSIX sh, but OpenWrt uses
ash which supports it. The init scripts use `local`
extensively for parameter scoping.
SC2086, SC2089, SC2090 — firewall.sh constructs nft match clauses
with literal quotes that need to survive variable expansion
(`match='iifname "$TUN"'` then `nft ... $match accept`). The
lack of double-quoting around `$match` is intentional so
word-splitting yields separate nft arguments.
Adding these to the exclude list. SC2317 (unreachable code) and
SC1008 (rc.common shebang form) were already excluded.
Cover the previously untested STUN client behavior under server
unreachable, response timeout, and packet loss. The 3 NAT scenarios
test happy paths only; if the STUN client mishandled a fault (panic,
hang, missing log signal), it would silently degrade NAT traversal
without surfacing in CI.
testing/nat/scripts/stun-faults-test.sh (new, 244 lines):
Phase 1 (drop, ~12s): tc prio + netem loss 100% band + u32 filter on
dst 172.31.10.40 udp 3478. Falls back to iptables -j DROP if netem
isn't available. Asserts daemon process alive, no panic, log line
matching stun.*(timed?out|fail|fallback|unreachable|no address)
within the phase window.
Phase 2 (delay then clear, ~17s): tc qdisc add dev eth0 root netem
delay 5000ms for 7s, then deleted. 10s settle. Asserts process alive,
no panic, AND "STUN observation succeeded" log line after clear
(recovery proof).
Phase 3 (kill, ~12s): docker stop fips-nat-stun. Asserts process
alive, no panic, fault evidence in logs.
testing/nat/docker-compose.yml: stun-faults profile adds two
services. stun-fault-node is fips-test:latest on shared-lan at
172.31.10.50. stun-fault-shim is fips-test:latest sharing the
daemon's network namespace via network_mode: service:stun-fault-
node, with cap_add NET_ADMIN, NET_RAW; entrypoint sleep infinity so
the script can docker exec into it. Reuses existing stun
(172.31.10.40:3478) and relay (172.31.10.30:7777) services.
testing/nat/scripts/generate-configs.sh: 3-hunk update so the
generator accepts the new scenario and points its peer config at the
existing relay/STUN. The peer is configured for connect_peer() so
the daemon retries traversal on a loop, repeatedly invoking
observe_traversal_addresses() — which is the fault-injection target.
testing/ci-local.sh: STUN_FAULTS_SUITES=(stun-faults) array,
run_stun_faults runner, list/integration-loop/--only-dispatch hooks.
.github/workflows/ci.yml: matrix row {suite: stun-faults, type:
stun-faults} + 3 steps gated on matrix.type == 'stun-faults' between
nostr-publish-consume and any chaos suite. Reuses fips-linux
artifact + fips-test:latest image.
Approach: script-driven via docker exec stun-fault-shim. Sharing
network namespace means tc rules on the shim's eth0 affect daemon
egress. No timing logic in the shim itself.
End-to-end exercise the v0.3.0 nftables firewall baseline so the
security claim — "services on fips0 are not exposed by default" — is
validated in CI rather than by operator opt-in. The packaging postinst
state is pinned by the deb-install matrix; this suite pins the actual
ruleset behavior.
testing/firewall/ — new directory mirroring the acl-allowlist
precedent (kept in its own directory, not extending testing/static):
docker-compose.yml (52 lines): two FIPS containers on bridge
172.32.0.0/24, peered over UDP/2121. node-b mounts
packaging/common/fips.nft RO at /etc/fips/fips.nft and a generated
drop-in services.nft at /etc/fips/fips.d/.
test.sh (247 lines): four-case asserter
(a) curl from node-a to node-b:8000 → DROP (port not allowlisted;
terminal counter drop fires)
(b) curl from node-b to node-a:8000 → 200 OK (reply traverses
node-b's ct state established,related accept)
(c) ping6 a→b → success (icmpv6 echo-request accept)
(d) nc -z a→b:22 → success (drop-in tcp dport 22 accept honored
via include "/etc/fips/fips.d/*.nft")
Plus drop-counter check after case (a) confirms the dropped
connection actually hit the chain's terminal counter drop.
generate-configs.sh (111 lines): mirrors acl-allowlist generator,
produces the drop-in services.nft + fips configs.
README.md (111 lines): how to run, expected output, design notes.
.gitignore: ignores generated-configs/.
testing/ci-local.sh: FIREWALL_SUITES=(firewall) array + run_firewall
runner; dispatch in run_integration and run_suite mirroring
run_acl_allowlist.
.github/workflows/ci.yml: matrix row {suite: firewall, type:
firewall} + 3 steps gated on matrix.type == 'firewall' between
acl-allowlist and gateway. Reuses fips-linux artifact + fips-test:
latest image.
Activation note: the unified test image does not run systemd, so
test.sh invokes the fips-firewall.service ExecStart
(/usr/sbin/nft -f /etc/fips/fips.nft) directly. The systemd-unit
enablement path is covered by the deb-install matrix; this suite
exercises what the unit configures, not how it gets started.
Cover the previously untested overlay advert publish/relay/consume
round-trip. The bilateral publish/subscribe path was a v0.3.0 release
gap: malformed adverts could panic consumers, broken signatures could
go undetected, and reverse-direction subscription was unverified.
Adds testing/nat/scripts/nostr-relay-test.sh (290 lines):
Phase 1+2 (combined): wait_for_peers on both nodes; pass on
bidirectional advert publish/subscribe round-trip + dial completed;
ping6 both directions confirms TUN-level reachability.
Phase 3 (malformed advert resilience): stdlib-only Python WebSocket
client publishes a syntactically valid Schnorr-signed Kind-37195
event whose `content` is gibberish (cannot deserialize as
OverlayAdvert). The relay enforces BIP-340 signature validity, so the
event reaches the consumers (rather than being dropped at the relay)
— a trivially-junk content payload is the right adversarial input.
Required ~80 lines of stdlib-only secp256k1 + BIP-340 in the script
(no new container deps). Asserts pidof fips on both nodes after the
publish, scans logs for panic markers, re-pings to prove the existing
peer link survives.
testing/nat/docker-compose.yml: new profile nostr-publish-consume
with two daemon services (nostr-pub-a 172.31.10.20, nostr-pub-b
172.31.10.21) on shared-lan, reusing the existing strfry relay
(172.31.10.30:7777) and STUN service (172.31.10.40:3478).
testing/nat/scripts/generate-configs.sh: 2-line allowlist update so
the new scenario flows through the existing config generator (rather
than forking a parallel one). Generated node-{a,b}.yaml + npubs.env
smoke-tested cleanly.
testing/ci-local.sh: NOSTR_RELAY_SUITES=(nostr-publish-consume)
array, run_nostr_publish_consume runner, dispatch in run_integration
and run_suite. Mirrors existing run_nat shape.
.github/workflows/ci.yml: one matrix row + 3 steps in the integration
job, gated on matrix.type == 'nostr-publish-consume'. Consumes the
same fips-linux artifact and fips-test:latest image as the existing
NAT suites.
Tor/TCP transport variants kept out of v0.3.0 scope; the structure
leaves room for nostr-publish-consume-tcp/-tor siblings later without
disturbing this baseline.
Add nft -c -f packaging/common/fips.nft syntax-check to both
testing/ci-local.sh and .github/workflows/ci.yml so a regression in
the 128-line firewall ruleset surfaces in the build gate rather than
when an operator activates fips-firewall.service.
testing/ci-local.sh: first step inside run_build(), before
cargo build --release. Uses command -v nft for prereq detection
mirroring the existing cargo-nextest pattern; records as nft-syntax
in RESULTS. Operator-facing message points at apt install nftables
when nft is absent.
.github/workflows/ci.yml: nftables added to the build job's existing
Linux apt-install step; new Validate fips.nft syntax (Linux only)
step gated on runner.os == 'Linux' (skips macOS/Windows matrix
slots, runs on ubuntu-latest and ubuntu-24.04-arm).
Note: nft -c -f requires netlink cache initialization on modern
nftables even in check mode, so both invocations use sudo (safe in
CI's passwordless sudo, and operator's typical local sudo). Without
sudo, nft fails with "cache initialization failed: Operation not
permitted" before reaching ruleset parse.
Add windows-lint job to .github/workflows/ci.yml running
PSScriptAnalyzer against the three operator-facing PowerShell scripts
(build-zip.ps1, install-service.ps1, uninstall-service.ps1) on the
windows-latest runner. Job runs in parallel with build/test, no
needs:, no tag gating.
Also add packaging/windows/PSScriptAnalyzerSettings.psd1 with two
project-appropriate suppressions:
PSAvoidUsingWriteHost — operator-facing progress in all three
scripts is intentional; Write-Output would conflate progress
with return value.
PSAvoidUsingPositionalParameters — Copy-Item src dst / New-Item
-Path style positional usage is widespread for cp/mv-like
readability.
Severity = @('Error', 'Warning') gates CI on Warnings+ only,
skipping Information-only findings.
The lint job lives in ci.yml rather than package-windows.yml: keeps
the packaging workflow focused, avoids interleaving with the
structural-verification steps in package-windows.yml, and the lint
runs on every push (not just tags).
Extend the gateway integration suite with three previously unexercised
runtime paths. All three share testing/static/scripts/gateway-test.sh
and testing/static/docker-compose.yml so they land as one commit.
6A — UDP port forwarding runtime path. Add udp 18081 -> [fd02::20]:8081
to inject_gateway_config() and a phase-7 case where gw-client runs an
inline Python UDP echo server bound [::]:8081 and gw-server sends a
UDP probe to [GW_MESH_IP]:18081 via inline python3, asserting the
echoed payload prefix. The config layer already accepted proto: udp
(test_port_forwards_same_port_different_proto_ok) but the UDP NAT
rule shape and conntrack handling differ from TCP and were unverified.
Uses Python rather than socat because fips-test:latest does not ship
socat; nc -u IPv6 round-trip semantics are messier than a Python
one-liner.
6B — Second simultaneous TCP forward. Add tcp 18082 -> [fd02::20]:8081
alongside the existing 18080 forward. Phase 7 now greps the daemon's
nft DNAT table for all three rules (18080, 18082, 18081) and runs
HTTP fetches through both TCP forwards with distinct backend payloads
(inbound-forward-ok vs inbound-forward-ok-2) so a misrouted response
fails the assertion.
11A — Concurrent multi-client flows. Add gw-client-2 service to
docker-compose mirroring gw-client (IPv6 fd02::21, IPv4 172.20.1.21).
Phase 3 sets the fd01::/112 route on both. Phase 4 issues DNS lookups
from both, asserts they receive distinct virtual IPs, and queries the
gateway control socket (show_mappings) to confirm exactly 2 active
mappings (5-attempt retry loop tolerates snapshot-publish lag). Phase
5 launches both curl requests concurrently as background processes,
asserts each response. Validates concurrent NAT mappings, pool
contention, proxy NDP under simultaneous LAN-client traffic — all
real-world deployment shape that was not pinned.
Phase 8 reclamation timing unchanged (TTL=5s, grace=5s, 25s wait
covers both mapping ticks generously even with a slight stagger).
Add post-build structural verification steps to the three platform
packaging workflows so structural defects abort before checksum/upload.
The .pkg/.zip/.ipk builds run on tag (or every push for OpenWrt) but
their output layouts were unverified — silent drift could ship missing
binaries, broken plists, or malformed package containers that would
only surface at install time. Field operators rely on manual
validation; CI now catches packaging regressions before release.
macOS (.github/workflows/package-macos.yml):
The pkg is a pkgbuild component flat package; pkgutil --expand yields
the structure but actual files live inside Payload (cpio.gz). The
verification step extracts that with gzip -dc | cpio -i, then asserts:
- usr/local/bin/fips, fipsctl, fipstop all present
- Library/LaunchDaemons/com.fips.daemon.plist present
- plutil -lint on the plist returns zero
Each check prints PASS/FAIL; verifier dumps the full extracted tree on
failure for diagnosis. Codesigning verification deferred (pkgbuild
invocation has no --sign — nothing to verify yet).
Windows (.github/workflows/package-windows.yml):
build-zip.ps1 produces a flat archive (no wrapper directory) via
Compress-Archive -Path "$StagingDir\*". The verifier extracts and
asserts each expected file at the extract root:
fips.exe, fipsctl.exe, fipstop.exe (binaries)
fips.yaml, hosts (config from packaging/common/)
install-service.ps1, uninstall-service.ps1 (from packaging/windows/)
README.txt (generated inline by build script)
Each check prints PASS/FAIL; verifier dumps the full extracted listing
on failure for diagnosis. LICENSE intentionally not asserted: the
build script does not currently ship one, and asserting on it would
false-fail every build.
OpenWrt (.github/workflows/package-openwrt.yml):
Add four post-build verification steps between Build .ipk and SHA-256
hashes. The ipk build runs every push but its contents, shell-script
lint state, and sysctl drop-in syntax were unverified.
Steps:
1. Install shellcheck (conditional — pre-installed on ubuntu-latest;
apt-get install fallback only if missing).
2. Lint init/uci/hotplug/firewall scripts: run shellcheck --shell=sh
--exclude=SC1008,SC2317 against the 5 shipped #!/bin/sh scripts
(fips, fips-gateway, firewall.sh, 99-fips hotplug, 90-fips-setup
uci-default). SC1008 silences the rc.common shebang form; SC2317
silences "unreachable" warnings for externally-invoked rc.common
hooks.
3. Sysctl drop-in syntax: per-line regex check against both
fips-gateway.conf and fips-bridge.conf.
4. Verify ipk structural integrity: extract via tar -xzf (OpenWrt's
actual format despite the misleading "ar archive" comment in
build-ipk.sh), assert all three top-level entries
(control.tar.gz, data.tar.gz, debian-binary), assert 14 file
paths in data.tar.gz (binaries, init scripts, configs, sysctl
drop-ins, hotplug + uci-defaults + sysupgrade keep), and assert
the 4 control-tarball entries plus debian-binary content "2.0".
Full ipk install/runtime test deferred past v0.3.0 scope.
Operator-side note: testing/ci-local.sh NOT modified for the OpenWrt
verifier (operators don't typically have the cross-compile toolchain
locally).
Extend testing/deb-install/test.sh per-distro container test loop with
5 new PASS assertions covering the v0.3.0 nftables firewall baseline
that ships installed-but-disabled (operator opt-in):
1. fips-firewall.service unit present at /lib/systemd/system/
2. fips-firewall.service disabled by default (is-enabled = 'disabled')
3. /etc/fips/fips.nft exists AND is registered as a dpkg conffile
4. /etc/fips/fips.d/ drop-in directory present with mode 755 root:root
5. fips.nft includes the drop-in glob /etc/fips/fips.d/*.nft
Assertions slot in between the existing fips-dns service-state check
and the simulated-boot service-start block. Each runs against every
distro in the existing matrix (debian12 / ubuntu24 / ubuntu26).
The security claim — services on fips0 are not exposed by default —
remains end-to-end-validated separately by the testing/firewall/
suite. This commit specifically pins the static install state so
packaging regressions surface in the per-push deb-install gate
rather than at operator opt-in time.
Cover two adjacent runtime behaviors in the discovery state machine
that were previously unpinned at the test level.
1. Open-discovery startup sweep iterate-filter-queue contract.
Cover the runtime sweep behavior: iterate advert cache, apply
skip-filters (own-pubkey, already-connected peers), queue eligible
entries to retry_pending. The config layer was tested but the sweep's
own filtering logic was unpinned.
src/discovery/nostr/runtime.rs: add #[cfg(test)] impl block with
three pub(crate) helpers — new_for_test() builds a minimal
NostrDiscovery with empty cache and no relays/background tasks (uses
fresh nostr::Keys signer + Client::builder().autoconnect(false));
cached_advert_for_test() wraps an OverlayEndpointAdvert into a
CachedOverlayAdvert valid for 1h; insert_advert_for_test() writes
direct to the advert_cache RwLock. All three vanish from release
builds via cfg-gating.
src/node/lifecycle.rs: visibility-only widen on
run_open_discovery_sweep from private async fn to
pub(in crate::node) async fn so the in-tree test can drive it
directly. Same pattern as already-pub(in crate::node) handlers in
src/node/handlers/.
src/node/tests/discovery.rs: add #[tokio::test]
test_open_discovery_sweep_queues_eligible_skips_filtered. Builds
Node + Arc<NostrDiscovery>, injects 3 adverts (eligible, already-
connected peer, own-pubkey), invokes the sweep, asserts retry_pending
contains exactly the eligible entry with matching peer_config npub
and the two filtered entries do NOT appear.
2. Per-attempt timeout state machine in check_pending_lookups.
Cover the central new behavior of f16b837: the [1, 2, 4, 8] retry
sequence (cumulative deadlines 1100/3100/7100/15100ms), one fresh
LookupRequest per attempt, and final-timeout reaching the unreachable
state. The opt-in DiscoveryBackoff machinery was well-tested but inert
at default config; this pins the state machine that runs by default.
Add test_check_pending_lookups_default_sequence_unreachable to
src/node/tests/discovery.rs. Constructs a Node with a peer that has
the target in its bloom but cannot respond (no Noise session — the
state-machine bookkeeping is independent of wire-send success).
Drives check_pending_lookups deterministically through:
t=1100 → second attempt; entry.attempt advances; req_initiated++
t=3100 → third attempt; entry.attempt advances; req_initiated++
t=7100 → fourth attempt; entry.attempt advances; req_initiated++
t=15099 → no-op (one ms before final deadline)
t=15100 → final timeout
At t=15100 asserts: pending_lookups[target] removed; resp_timed_out
counter +1 (this is the actual counter name); pending_tun_packets
[target] removed (queued packet dropped); a frame on the TUN sender
with IPv6 + next_header=58 + ICMPv6 type=1 (Destination Unreachable).
Fresh-request_id-per-attempt is structurally guaranteed: LookupRequest
::generate() unconditionally calls rand::random::<u64>(), and the
test asserts req_initiated increments by exactly 1 per retry (proving
initiate_lookup runs fresh each time, not a resend of cached state).
The originator's request_id isn't stored on the originator side
(deliberately omitted from recent_requests so the response is
recognized as "ours"), so direct request_id capture is not feasible
and counter-tick is the load-bearing observable.
No production logic touched; no visibility widening needed
(check_pending_lookups was already pub(in crate::node)).
Johnathan Corgan