Move the max_peers cap check in handle_msg1 forward, from the late
check inside promote_connection (which fires after Msg2 has already
been built and put on the wire) to an early position after identity
verification but before index allocation and the Msg2 send. When the
gate fires for a net-new identity, the Msg1 is silent-dropped — no
response goes back to the peer, no AEAD compute or wire bytes are
spent.
Bypass preserved for known peers (reconnect / cross-connection): if
the sender's NodeAddr is already in self.peers, or if a pending
outbound connection is in flight to the same identity, the gate is
skipped so legitimate maintenance traffic continues to work. The
late check inside promote_connection is intentionally retained as
defense-in-depth against future call sites or a disconnect racing
between the early-gate decision and promotion.
Wire-cost rationale: a 45 s tcpdump at saturation observed ~3.6
cap-denials/s steady-state, each previously paying the full Noise IK
responder crypto + Msg2 (~104 B) on the wire before being rejected.
The bigger value is cleaner peer-side semantics — the peer no longer
sees a fake-completed handshake whose data frames subsequently fail
decryption locally.
Two new unit tests cover the cases:
- handle_msg1_silent_drops_at_cap_for_new_peer drives a wire-pumped
Msg1 from a fresh identity into a saturated node and asserts no
Msg2 reaches the sender socket. Stash-verifies as FAIL on the
pre-fix tree (Msg2 hits the wire) and PASS post-fix.
- handle_msg1_admits_existing_peer_at_cap drives a Msg1 from an
identity already in self.peers and asserts the gate does not evict
it. This is a regression check (the no-gate tree behaves the same
way here, but the test guards against an accidental future gate
that breaks known-peer admit).
Brings two structural fixes landed on maint:
- compute_mesh_size: explicit parent skip in the children loop, so the
disjoint-subtree invariant no longer depends on peer_declaration cache
freshness.
- max_peers: outbound connection-initiation gated on the cap (auto-reconnect
retries, Nostr-mediated discovery established adoption, and both sides
of the NAT-traversal punch sequence). Inbound msg1 admission gate
unchanged.
node.limits.max_peers was honored only on inbound msg1 admission
(handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len
is at the cap). Four outbound initiation paths proceeded unconditionally
at capacity: auto-reconnect retries (process_pending_retries),
Nostr-mediated discovery's BootstrapEvent::Established adoption
(poll_nostr_discovery), NAT-traversal punch initiation (the outgoing
side of the offer/answer/punch sequence in the Nostr discovery
runtime), and NAT-traversal punch response (the incoming side of the
same sequence). A saturated node burned CPU, UDP probes, STUN
observations, and Nostr relay traffic on connections that the inbound
gate would reject the moment they reached msg1.
Introduce Node::outbound_admission_check (peers.len < max_peers, or
true when max_peers == 0 as the no-cap sentinel) and gate the four
paths. The discovery runtime lives in a separate task and does not
hold a Node reference; bridge via an Arc<AtomicBool> the runtime
reads and Node refreshes once per tick from outbound_admission_check.
The atomic granularity is intentionally loose: one-tick lag is
acceptable because the inbound msg1 gate continues to be the
authoritative cap, and in-flight handshakes started below the cap
are allowed to complete.
Inbound gate at handshake.rs is unchanged.
The mesh-size estimator's children loop relied on the cached
peer_declaration(parent_id).parent_id() != my_addr check to exclude
the parent. That cached view briefly disagrees with our own latest
my_declaration().parent_id() during the window between a local
parent-switch and the new parent's next inbound TreeAnnounce: the
peer-declaration cache still names us as the parent's parent, so the
parent is iterated as if it were a child and its (typically dominant)
bloom cardinality is added a second time. Symptom: estimated mesh size
displayed in fipsctl show status and fipstop nearly-but-not-exactly
doubles during tree rebalancing.
Make the invariant structural with an explicit peer_addr == parent_id
skip at the head of the children loop. Per-peer 500 ms rate-limiter
and overall recompute cadence are unchanged.
Adds a regression test that constructs the stale-peer-declaration
scenario directly and asserts the parent is not double-counted.
Closes the eventually-consistent gap in spanning-tree state
distribution. Every existing send_tree_announce_to_all call site
gates on a local state-change event (parent switch, self-root
promotion, ancestry change, peer promotion, parent loss). Once a
partition latches — for example a parent-switch announce stranded
in the brief cross-init handshake swap window, where the announce
arrives on a session-index whose decrypt-worker entry has been
unregistered — neither side's state changes again, so neither
side ever re-broadcasts. The existing 60 s check_periodic_parent_reeval
was a re-evaluation, not a re-broadcast: it short-circuited
silently on no-change. Production-side healing depended on
incidental link churn; lab harnesses with stable docker-bridge
links had no equivalent path.
Add a final else branch that fires send_tree_announce_to_all
unconditionally on the no-change path, alongside the existing
switch and self-promote arms. Receivers coalesce by sequence
comparison (ParentDeclaration::is_fresher_than) and short-circuit
at the `if !updated` gate in handle_tree_announce; same-sequence
repeats drop silently with no cascade. The per-peer 500 ms
rate-limiter is well below this 60 s cadence and does not suppress
the heartbeat broadcast.
The fix is a general protocol-robustness improvement: it addresses
any in-flight TreeAnnounce loss class, not only the specific
cross-init swap-window drop site.
testing/static/scripts/rekey-test.sh BASELINE_CONVERGENCE_TIMEOUT
60 -> 65 so a partition healed by the periodic broadcast at T+60
lands inside the convergence window. wait_for_full_baseline
early-exits on PASS, so successful reps see no extra wall-clock.
The tick body's per-peer check_* loops (heartbeats, bloom
announces, MMP reports, tree announces) called transport.send
for every active peer, which on TCP/Tor fell through to a 5 s
connect-on-send wait for any peer whose pool entry was not yet
established. That wedged the entire tick body for the full
connect_timeout_ms per unreachable peer; under post-restart
convergence on a high-peer mesh, this cascaded into multi-
second tick stalls. On master, the same mechanism also starved
the per-tick control-snapshot republish and pushed fipsctl
queries onto an mpsc fallback that was itself queued behind
the wedged rx_loop, producing the 5-second fipsctl head-of-line
pattern operators observed on loaded nodes.
Gate send_encrypted_link_message_with_ce on
transport.connection_state before the send: proceed only when
Connected; on None, kick off a non-blocking background connect
(idempotent — TransportHandle::connect dedupes against the
connecting pool and spawns the timeout-bounded TcpStream::connect
inside its own tokio task) and fail this send fast with a
clear "transport connection not ready" error. A subsequent
tick retries once the pool has an entry. The reconnect
lifecycle (check_link_heartbeats, process_pending_retries,
poll_pending_connects) is unchanged. The connect-on-send
branch in transport.send_async itself remains in place for
code paths that legitimately need synchronous connect (e.g.,
explicit operator-driven fipsctl connect).
When both peers' Nostr-mediated UDP punches complete within the
same scheduling window, each side's `BootstrapEvent::Established`
event arrives with `is_connecting_to_peer` already true: each side
received an inbound msg1 from the peer's pre-punch outbound
attempt, which created a connecting-state record. The deduplication
skip then fires on both sides, neither installs the fresh
traversal socket as canonical, and the peer-adoption budget
(45 s) expires. Cross-node wall-clock alignment of the skip log
line in observed failures was within ~1 ms — simultaneous dual-
fire under contention, the dual-initiation pattern.
Apply the deterministic NodeAddr tie-breaker already used at
`handlers/handshake.rs:269` for rekey dual-initiation and in
`peer::cross_connection_winner` for cross-connection resolution.
Smaller NodeAddr wins as adopter: enumerate the in-flight
connections whose `expected_identity` points at this peer, tear
them down via the canonical `cleanup_stale_connection` helper, and
fall through to `adopt_established_traversal`. Larger NodeAddr
loses and keeps the existing `continue` semantics; the loser's
in-flight outbound is reconciled by `handle_msg1`'s cross-
connection logic when the winner's fresh msg1 arrives over the
adopted socket.
`cleanup_stale_connection` visibility bumped from module-private
to `pub(in crate::node)` so it is callable from `lifecycle.rs`.
The defensive re-check inside `adopt_established_traversal`
itself is left as-is — after the outer cleanup the winner reaches
it with `is_connecting_to_peer == false`, so the inner skip
won't trip. The `BootstrapEvent::Failed` arm is unchanged: there
is no winning outcome on dual failure, and the existing skip +
retry-schedule semantics are correct.
An FSP session rekey could leave the two endpoints holding different
key sets for a brief window: if a handshake message was lost in
transit, one side rotated to the new keys while the other did not.
Traffic sealed in one key epoch then reached a peer still on the
other epoch and failed to decrypt, producing bursts of AEAD
decryption failures and dropped connectivity until a later rekey
cycle reconverged the pair. Choreographing the cutover order cannot
close this window: any fixed ordering still leaves a skew that
packet reordering widens.
Make rekey correctness independent of cutover timing by overlapping
the key epochs on the receive path. During a rekey transition the
receiver trial-decrypts each frame against every live session it
holds: current, the not-yet-promoted pending session, and the
draining previous session. The K-bit becomes a hint that orders the
trial-decrypt cascade rather than a hard gate, and a frame that
authenticates against the pending session is itself the cutover
signal. No rotation ordering and no packet reordering can then cause
a decryption failure.
The pre-rekey Noise session is held in the `previous` slot until the
peer has demonstrably moved off it. Its drain deadline is anchored
on the most recent frame the peer authenticated against that slot,
refreshed each time the trial-decrypt cascade lands there, rather
than on a fixed wall-clock timer started unilaterally at the local
cutover. A peer that never received the new keys keeps authenticating
against `previous` and the slot stays live; without this, a fixed
timer would erase the only key set that could decrypt the peer's
frames, producing a permanent silent decrypt failure on a live data
path. A peer that never catches up is handled by the existing FSP
session liveness path rather than by silent decrypt failure.
The lost-handshake liveness gap is closed separately by retransmitting
the third rekey handshake message until the peer is confirmed on the
new keys, with a bounded retry budget after which the rekey cycle is
cleanly abandoned and retried on the next timer.
Adds unit tests covering the trial-decrypt cascade (epoch selection,
promotion on pending decrypt, reordered old-epoch stragglers after
cutover, per-slot replay-window integrity), the msg3 retransmission
lifecycle, and the peer-progress-aware drain retirement.
Commit 57a089f6 (the GitHub #102 fix) landed without a CHANGELOG
entry. Add the `[Unreleased]` / `### Fixed` line so the macOS
package-integrity fix is on record before the v0.3.1 cut.
The AUR `fips` and `fips-git` packages did not install the
`fips-dns-setup` and `fips-dns-teardown` helper scripts that
`fips-dns.service` runs. The Debian package ships them to
`/usr/lib/fips/` through the `[package.metadata.deb]` assets, but the
AUR `package()` functions never replicated those install steps, so
`fips-dns.service` failed to start on Arch with "Unable to locate
executable /usr/lib/fips/fips-dns-setup".
Add the two `install -Dm0755` lines to both PKGBUILDs so the AUR
packages match the Debian layout.
Also harden the transition between the `fips` and `fips-git`
packages: each PKGBUILD now declares the other variant's `-debug`
split package as a conflict and opts out of the debug split, so a
stale debug build cannot retain ownership of installed files when
switching between the release and VCS packages. The `aur-publish`
workflow gains a validated `pkgrel` dispatch input so corrected
packaging can be republished against an existing release tag without
retagging.
Fixes#98
(cherry picked from commit 4cf550e23d)
Moves both AEAD layers (ChaCha20-Poly1305, one round per layer per
packet) plus the sendmsg syscall off the rx_loop task onto a per-shard
worker pool, adds per-peer connect(2)-ed UDP with SO_REUSEPORT, and
uses Linux UDP GSO (sendmsg+UDP_SEGMENT — kernel splits one super-skb
into N on-the-wire datagrams in a single TX-stack walk) when packets
in a batch are uniform-size. Same kernel primitive WireGuard's
in-kernel module and BoringTun use to hit 2.5–3.2 Gbps single-stream.
Single TCP stream on a 5-node docker-bridge mesh, 5 x 15 s x P=1:
A→D: 1379 → 2708 Mbps (1.96x, RTT +0.12 ms)
A→E: 1394 → 2663 Mbps (1.91x, RTT +0.11 ms)
E→A: 1406 → 2624 Mbps (1.87x, RTT +0.19 ms)
Static-peer pairs only — every CoV under 3%, 0 outliers, 0% ICMP
loss. The ~+100 µs RTT is the worker queue handoff cost; AEAD +
sendmmsg now run on a separate core in exchange.
What lands:
- src/node/encrypt_worker.rs: std::thread + crossbeam_channel
workers; hash-by-destination dispatch pins a TCP flow to one
worker so wire ordering is preserved; per-worker sendmmsg(2)
batching up to 32; Linux uses sendmsg(2)+UDP_SEGMENT when
packets in a group are uniform-size.
- src/node/decrypt_worker.rs: receive-side mirror. Each shard owns
its session's recv cipher + replay window in a thread-local
HashMap (no shared RwLock/Mutex). Sessions are handed off at
promote_connection and re-registered on K-bit flip / rekey
cutover.
- src/node/handlers/session.rs try_send_session_data_pipelined:
FSP+FMP both seal in-place in the worker on one wire-buffer
alloc; no intermediate inner_plaintext / fsp_payload Vecs.
- src/transport/udp/connected_peer.rs + peer_drain.rs: per-peer
connect(2)-ed UDP socket with SO_REUSEPORT (set on the listen
socket too — without that, EADDRINUSE on activation and every
packet falls back to the wildcard path); the worker sends with
msg_name=NULL and the kernel uses its cached 5-tuple. Tick-
driven activation in handlers/connected_udp.rs, idempotent.
- src/transport/udp/mod.rs: mem::replace the recvmmsg backing buffer
instead of buf.to_vec() per packet — single pointer swap, no
MTU-sized memcpy.
- src/protocol/link.rs SessionDatagramRef: zero-copy borrowed view
used by handle_session_datagram for the bulk local-delivery
path; handle_session_payload takes the borrowed payload
directly (no payload[35..].to_vec()).
- src/transport/mod.rs TransportAddr::from_socket_addr: collapses
the two-alloc from_string(addr.to_string()) pattern to one.
- src/node/handlers/rx_loop.rs: decrypt-fallback drain promoted
ahead of packet_rx in the select! (TCP ACK starvation fix);
interleaved fallback drain every 32 packets inside the rx burst
loop.
- noise::Session: send_cipher_clone / recv_cipher_clone /
recv_replay_snapshot_owned / take_send_counter / accept_replay
so off-task workers can hold a cloned cipher + reserved counter
while the dispatcher keeps replay/counter sequencing serial.
CipherState::cipher_clone returns a refcount-bumped LessSafeKey.
AsyncUdpSocket: AsRawFd so workers issue raw sendmmsg / sendmsg
without going through the tokio reactor.
- Worker pool sizing: both default to num_cpus, overridable via
FIPS_ENCRYPT_WORKERS=N / FIPS_DECRYPT_WORKERS=N. Per-peer
connected UDP can be disabled via FIPS_CONNECTED_UDP=0.
- src/perf_profile.rs: optional per-stage timing reporter under
FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1). Off by default; zero
overhead when disabled.
- All cfg(unix)-gated. Windows continues on the existing tokio-
based send/recv.
Decrypt worker session lifecycle:
- Node::unregister_decrypt_worker_session mirrors the existing
register helper. Wired at the two natural sites that already
iterate peers_by_index: the rekey drain-completion block in
handlers/rekey.rs (drops the worker entry for the old our_index
once the drain window has expired and the cache_key is
unreachable to any in-flight OLD-K packet), and remove_active_peer
in handlers/dispatch.rs (drops the worker entry for each of the
four index slots: current, rekey, pending, previous). Only
our_index is normally registered; unregister_session is fire-
and-forget for missing entries, so calling unconditionally on
all four slots is correct and bounds the cleanup without per-
slot accounting. Without these callers the per-worker sessions
HashMap and the Node's decrypt_registered_sessions set would
grow monotonically per rekey on long-lived peers.
Testing:
- testing/static/scripts/bench-multirun.sh: multi-run iperf3 +
ping bench. N reruns (default 5), median / min / max / CoV % /
per-run outlier flag, avg ping RTT, ICMP loss %, TCP retransmit
total. Plain client→dest labels + topology header. Pre-bench
peer-convergence check (FIPS_BENCH_CONVERGE_SECS, default 15);
per-path route verification via stats.bytes_sent deltas — fails
fast if traffic exits via a non-static-peer link.
- testing/static/docker-compose.yml: passes FIPS_ENCRYPT_WORKERS /
FIPS_DECRYPT_WORKERS / FIPS_PERF through to containers for A/B
benchmarking without rebuilds.
- testing/static/scripts/iperf-test.sh: same plain client→dest
labels + topology header (was multihop/direct/N hop, which
conflated topology distance with on-wire path).
- .config/nextest.toml: synthetic UDP node tests serialized
through a max-threads=1 test group. Localhost handshakes drop
on shared CI runners under parallel load; one-at-a-time keeps
assertions reliable.
- src/node/tests/spanning_tree.rs: repair_missing_edge_handshakes
— retries up to 5 times for synthetic edges whose msg1 was
dropped, with a drain after each edge retry instead of after
each attempt's full burst.
- src/node/decrypt_worker.rs::tests: two unit tests asserting
WorkerMsg::UnregisterSession removes the worker-thread session
HashMap entry (handle_msg_unregister_session_removes_entry) and
is a no-op for never-seen cache_keys
(handle_msg_unregister_session_idempotent_on_unknown_key), which
is the safety invariant the unconditional unregister calls at
the four index slots in remove_active_peer rely on.
- src/node/encrypt_worker.rs::unix_tests
pipelined_send_wire_layout_roundtrips_canonical_decoders: mirrors
the encoder geometry of try_send_session_data_pipelined (no
coords, the common established-session path), runs the worker's
real seal + send via flush_direct_batch_sync, and decodes the
resulting wire packet using only canonical receive-side decoders
(EncryptedHeader::parse, SessionDatagramRef::decode, FSP header
parse, noise::open). Any divergence between the hand-rolled
encoder offsets (fsp_aad_offset, fsp_plaintext_offset) and the
decoders fails at one of the parse / open / decode steps before
the inner-plaintext assertion fires. Complements the existing
fsp_preseal_runs_before_outer_fmp_seal test which covers the
seal-ordering invariant with synthetic headers but does not
exercise the wire-layout invariant.
CHANGELOG.md [Unreleased] # Changed entry added describing the
worker-pool threading model, hash-by-destination dispatch,
sendmmsg/UDP_GSO, per-peer connected UDP, the operator-facing env
vars, and the bench numbers above.
Cherry-picks from mmalmi/master (paths translated from
crates/fips-core/src/ to src/): 9b7c723, 0deb5cb, 13f7339, e036c0e,
3740a68, 3792f83, 8510193, 4910b07, e53f545, e4e2896, 5fe4af5,
1d01ada, 8c37008, e12469e, 6eb2860.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
Publish the 13-criteria PR review checklist the maintainer runs on
every incoming PR so contributors (and their coding agents) can run
the same pass before opening, surfacing problems before the review
round trip. CONTRIBUTING.md gets a new 'Self-review against the
project review checklist' subsection under 'Submitting pull requests'
and a Further Reading entry. CHANGELOG [Unreleased] gets an Added
entry.
Two [Unreleased] / Changed entries that should have landed alongside
the originating commits but didn't:
- macOS recvmsg_x batched receive (originally 59225ccf): completes
the Linux-equivalent inbound batching shape on Apple builds. Now
sequenced before the rx zero-copy entry so the section reads as a
coherent receive-path progression.
- Platform-specific test-build warning cleanup (originally 6bd40640,
PR #93): non-behavioral; documents the gating decisions that keep
cross-platform builds warning-clean.
Two Fixed entries appended to [Unreleased]:
- The coord cache surgical invalidation (49bd2104): replaces the
global CoordCache::clear() at parent-switch / become-root /
loop-detection / root-change sites with two targeted methods
(invalidate_via_node, invalidate_other_roots). Preserves cache
entries that remain correct after the topology change.
- The rekey-test strict-ping retry (306e4555): Phase 1 / 3 / 5
per-pair pings now retry up to 4 attempts. Brings the ICMP-noise
miss-floor from ~33% per phase to ~3.2e-6 at 1% loss without
changing the failure-shape signal the asserts target. Test
scaffold only, no daemon code changes.
- Borrowed SessionDatagramRef decoder is used in the forwarding
handler so local delivery and coordinate-cache warming no longer
allocate or copy the session payload. The owned SessionDatagram is
materialized only when re-encoding for the next hop.
- Owned SessionDatagram::decode is reimplemented as Ref::decode +
into_owned, so the two decoders cannot drift.
- recvmmsg / recvmsg_x (Linux + macOS) receive loop moves each filled
slot buffer into ReceivedPacket via mem::replace instead of cloning
it; a fresh empty buffer is installed for the next syscall.
- TransportAddr is formatted directly from the SocketAddr without
going through an intermediate String.
Focused decode bench: ref 1.6 ns/op vs owned 34.7 ns/op (21.4x).
End-to-end iperf is neutral as expected for a ~30 ns saving per
packet.
Unit tests added:
- test_session_datagram_ref_decode_borrows_payload (verifies the
payload slice pointer equals the input slice's offset 35, a real
zero-copy invariant guard against accidental future to_vec)
- bench_session_datagram_decode_owned_vs_ref (ignored, run with
--ignored --nocapture)
- test_transport_addr_from_socket_addr
Skip BootstrapEvent::Established and BootstrapEvent::Failed dispatch
in poll_nostr_discovery for peers that are already connected or
actively handshaking. Without these guards, stale traversal events
arriving after a peer connected through a different path would
either attempt to adopt a redundant socket against the live
connection (Established) or poison the per-peer failure-state
cooldown and trigger redundant retraversal via schedule_retry /
try_peer_addresses (Failed).
The four guard sites use a new is_connecting_to_peer helper extracted
from the existing closure inside initiate_peer_connection; the helper
checks for an in-flight outbound handshake state. adopt_established_traversal
gains a defense-in-depth check returning PeerAlreadyExists when called
against an already-promoted peer, so the invariant holds if a future
caller bypasses the outer dispatch guard.
Side benefit: narrows a cooldown-poisoning vector previously available
to an attacker injecting stale failure events for an active peer.
Test coverage for the new behavior:
- test_try_peer_addresses_skips_connected_peer
- test_try_peer_addresses_skips_connecting_peer
- test_nostr_traversal_failure_skips_connected_peer (Failed-arm event
injection)
- test_nostr_traversal_established_skips_connected_peer (Established-arm
event injection, mirror of the Failed test)
- test_adopted_traversal_skips_already_connected_peer
(adopt_established_traversal defense-in-depth)
CHANGELOG entry under [Unreleased] / Fixed.
Closes#87
Bring [Unreleased] into sync with all maint commits since v0.3.0:
- Add a Fixed entry for the acl-allowlist test-script poll-assertion
conversion (commit e9dd316) that was previously missing.
- Add the AUR-publish workflow rewrite and new fips-git VCS workflow
(commit 9bf9701) which had no entry, and merge them with the
ci.yml cancel-in-progress block under a single "CI and
release-publish workflows hardened" entry so the three workflow
changes read as one operational theme.
Pure changelog content reshuffle. No code touched.
Add a top-level concurrency block to ci.yml keyed on
(workflow, ref) with cancel-in-progress: true. Pushes (including
force-pushes) to the same ref now retire any in-flight run for
that ref rather than letting the superseded and current-tip runs
both burn runner minutes.
Motivated by a 2026-05-14 force-push experience where two CI runs
ran concurrently against a feature branch — the original push at
c76ec99 continued for ~17 minutes alongside the amended push at
927ef47, despite only the latter being the live tip.
Scope deliberately limited to ci.yml. Tag-triggered release-build
workflows (package-*.yml, aur-publish-*.yml) are untouched — they
operate on per-tag refs that already form distinct concurrency
groups and release artifact builds generally should not be
cancellable by unrelated activity.
Phase 5's per-pair connectivity check ran immediately after the second
rekey cycle, with no settle for routing reconvergence. Under
GitHub-runner CPU contention, post-rekey parent-switches and
coord-cache flushes can take longer than the per-ping 5s timeout for
a small fraction of pairs (1-3 of 20 typically), even though the
rekey mechanism itself completes cleanly. Phase 6 log analysis stays
all-green on these failed runs; the failure is purely connectivity
timing.
Mirror Phase 3's existing 12-second settle pattern: reuse REKEY_SETTLE
and emit the same banner shape. Two-line change. Cost on the success
path is a fixed 12s per suite run.
Add a per-session signed jitter offset (uniform [-15, +15] seconds)
to the rekey timer triggers in check_rekey (FMP) and check_session_rekey
(FSP). The configured `node.rekey.after_secs` becomes the nominal
interval rather than a floor; mean is preserved. Desynchronizes
both endpoints in symmetric-start meshes so the dual-initiation
race stops occurring rather than being resolved after the fact by
the smaller-NodeAddr tie-breaker.
Per-session storage means each rekey cutover reconstructs the
session and redraws the jitter naturally — successive cycles get
independent offsets, preventing drift back into sync.
Three entries under [Unreleased] for the three commits since the
v0.3.0 release tag:
- Sidecar example: FIPS_UDP_MTU env override (commit 32a3b58)
- CONTRIBUTING.md overhaul + new docs/branching.md (commit 538ce07)
- Rekey-test Phase 1 baseline-convergence headroom 36s -> 60s
(commit 6533276)
Reset maint to v0.3.0 to retire the v0.2.x maintenance window and
open a new tracking branch for v0.3.1-dev bug-fix work.
- Cargo: 0.3.0 → 0.3.1-dev
- CHANGELOG: fresh [Unreleased] block above [0.3.0]
- README: badge v0.3.0 → v0.3.1--dev
Bump Cargo.toml version 0.3.0-dev -> 0.3.0 and resync Cargo.lock,
move the CHANGELOG [Unreleased] block under [0.3.0] - 2026-05-11,
update the README status badge and prose to v0.3.0, and add the
release notes at docs/releases/release-notes-v0.3.0.md with a
mirrored copy at the repo root as RELEASE-NOTES.md.
Uses the 'ours' merge strategy to keep all of master's tree
verbatim with one carve-out: the v0.2.1 release notes archive at
docs/releases/release-notes-v0.2.1.md is retained as a permanent
docs artifact.
Discarded from maint's release-prep commit: Cargo.toml / Cargo.lock
version bumps (master stays on v0.3.0-dev), README v0.2.1 badge and
prose, and the RELEASE-NOTES.md root mirror (master's root mirror
will eventually carry the v0.3.0 release notes when v0.3.0 ships
from this line).
CHANGELOG: the v0.2.1 release section is integrated between
[Unreleased] (v0.3.0-dev work) and the [0.2.0] historical section.
Items that landed on maint during the v0.2.1 cycle are removed
from [Unreleased] so [0.2.1] is the canonical home for those
changes (Linux release artifact and AUR workflows; bloom fill-ratio
validation; seven maint-line bug fixes).
Operator-facing IPs in user-visible configs/docs (examples, tutorials,
packaging, sidecar templates) are now the resolvable hostnames of the
public test fleet (test-us01.fips.network, etc.) so they keep working
without baking specific addresses into examples.
Doc-comment and test fixtures in src/config/transport.rs use RFC 5737
TEST-NET-2 (198.51.100.1) so they cannot accidentally point at a real
host.
Also resyncs the openwrt-ipk fips.yaml with the common reference
(merge from master) and applies the same DNS-name swap there.
Bump Cargo.toml version 0.2.1-dev -> 0.2.1 and resync Cargo.lock,
move the CHANGELOG [Unreleased] block under [0.2.1] - 2026-05-11,
update the README status badge and prose to v0.2.1, and add the
release notes at docs/releases/release-notes-v0.2.1.md with a
mirrored copy at the repo root as RELEASE-NOTES.md.
The generic systemd install tarball is the catch-all install path
for systemd Linux distros that don't have a per-format package
(Fedora, RHEL/CentOS, openSUSE, Alpine, etc.). It had drifted
behind the .deb and AUR packages and was missing fips-gateway, the
mesh-interface firewall baseline, and the multi-backend DNS helper
in the shipped tarball. Bring it to parity:
- New `packaging/systemd/fips-gateway.service` (clone of the .deb
unit; ExecStart pointed at `/usr/local/bin/fips-gateway`). Not
enabled at install time; operator opt-in.
- New `packaging/systemd/fips-firewall.service` (clone of the .deb
unit; nft path unchanged at `/usr/sbin/nft`). Not enabled at
install time; operator opt-in.
- `build-tarball.sh` now bundles the `fips-gateway` binary, the two
new units, the `fips.nft` baseline conffile, and the
`fips-dns-setup` / `fips-dns-teardown` multi-backend helpers from
`packaging/common/`.
- `install.sh` now installs `fips-gateway` to `/usr/local/bin/`,
installs both new units to `/etc/systemd/system/` (without
enabling them), preserves `/etc/fips/fips.nft` on upgrade like
`fips.yaml`, and creates the `/etc/fips/fips.d/` operator drop-in
directory. Post-install messaging mentions both opt-in services.
- `uninstall.sh` stops and disables the optional services in
dependency order (firewall, gateway, dns, daemon), removes the
new unit files, and removes the gateway binary. `--purge` already
handles `/etc/fips/` removal which covers `fips.nft` and
`fips.d/`.
- `README.install.md` documents all of the above: expanded
"What Gets Installed" table, new sections covering the firewall
baseline and the LAN gateway, refreshed DNS section reflecting
the multi-backend setup helper (systemd dns-delegate /
systemd-resolved drop-in / per-link resolvectl / dnsmasq /
NetworkManager-dnsmasq), and updated Service Management.
Also fixes a latent packaging bug: `install.sh` previously
referenced `${SCRIPT_DIR}/../common/fips-dns-setup`, a path that
exists only in the source-repo layout and not in the extracted
tarball. The script now resolves the helper from the staging
directory first (the tarball case), falling back to the source-repo
relative path. Bug latent since the multi-backend DNS helpers
landed.
CHANGELOG `[Unreleased]` documents the parity bump under Changed
and the path-resolution fix under Fixed.
Closes the longest-standing parity gap for non-Debian / non-Arch
systemd Linux distros installing from the release-distribution
tarball.
The Noise-session AEAD swap landed as 5cda4a9 + 9b1016f without a
companion CHANGELOG entry; add it under [Unreleased] § Changed
ahead of the rest of the perf-win cluster from PR #81 since it's
the most operator-visible single change of that class.
Three Changed entries for the rx-path performance work
(Linux UDP recvmmsg batched receive, run_rx_loop drain batching, and
eager pubkey_full precompute on PeerIdentity construction) and five
Fixed entries: adopted NAT-traversed UDP transports inheriting the
primary listener's MTU and buffer config, TreeAnnounce ancestry on
self-root transitions, unconditional overlay-advert refetch before
each retry, stale overlay-advert eviction on NoTransportForType,
and scheduled retry on startup peer-init failure.
Pure CHANGELOG addition (+117 lines, no edits to existing entries).
Bullets are wrapped at 80 columns and attribute external
contributions to the originating PR and author.
Surfaces local services reachable from the mesh, paired with their
current `inet fips` baseline filter classification. Lands to the
right of the existing TUN section in the Traffic block.
A new daemon control query `show_listening_sockets` returns IPv6
listeners bound to either `::` (wildcard) or the node's fd00::/8
address, each classified as Accept / Drop / Unknown / NoFirewall
against the running inbound chain. fipstop renders the result as a
table beside the Traffic counters: Accept rows in default White,
Drop / Unknown in DarkGray, a yellow banner above the table when
`fips-firewall.service` is inactive, and a trailing `*` on
wildcard binds to remind the operator the bind is not
fips0-specific.
Daemon side:
- `src/control/listening.rs` walks `/proc/net/tcp6` and
`/proc/net/udp6` via the procfs crate (LISTEN state for TCP,
wildcard remote for UDP), filters to fips0-reachable binds, and
resolves inodes to PID / comm via `/proc/<pid>/fd`.
- `src/control/firewall_state.rs` shells out to
`nft -j list table inet fips` and walks the inbound chain.
Recognises canonical accepts (`tcp/udp dport N accept`,
`dport { ... } accept`, `dport A-B accept`), the iifname-scoping
line, conntrack and icmpv6 lines (skipped). Any rule with
unrecognised matchers (saddr filters, jumps, daddr filters) or
non-terminal verdicts forces Unknown classification for the
ports it references. Eleven unit tests cover the classification
logic; the listening enumerator carries a /proc-parsing test of
its own.
- `show_listening_sockets` emits
`{fips0_addr, firewall_active, sockets[]}` with per-row
`{proto, local_addr, port, pid, process, filter, wildcard_bind}`.
fipstop side:
- `src/bin/fipstop/ui/dashboard.rs` splits the Traffic block into
a 50/50 horizontal layout; the existing TUN + Forwarded panel
occupies the left half.
- `src/bin/fipstop/ui/listening.rs` renders the right half.
- `main.rs` fetches the new query each tick when the Node tab is
active. Errors are non-fatal: an old daemon without the query
leaves the payload at None and the panel renders "loading...".
`Cargo.toml` gains `procfs = "0.18"` on the Linux target. IPv4
listeners are not enumerated — fips0 is IPv6-only.
Folded in: revert the default-socket lookup from writability-probe
back to existence-based selection. The previous tempfile-probe on
`/run/fips` silently steered fipstop / fipsctl onto an XDG path
the daemon never bound for any user in the `fips` group whose
shell session had not yet picked up the supplementary group (no
re-login after `usermod -aG`). `XDG_RUNTIME_DIR` is set on every
modern systemd-managed user session, so this hit the common case.
The kernel checks actual group membership at `connect(2)`, so a
user who genuinely cannot connect now gets a clear `EACCES`
rather than a silent path mismatch. Drops the now-unused
`is_writable_dir` helper. `XDG_RUNTIME_DIR` existence validation
is preserved.
Documentation:
- `docs/reference/cli-fipstop.md` — Node-tab row updated, new
"Listening on fips0 panel" section.
- `docs/reference/control-socket.md` — `show_listening_sockets`
added to the read-only queries table.
- `docs/how-to/enable-mesh-firewall.md` — new "Verify with
fipstop" section.
- `docs/tutorials/host-a-service.md` — fipstop callouts at
Steps 3, 5, 6 + Troubleshooting bullet + wildcard-bind reminder
under "What you've learned".
- `CHANGELOG.md` — new bullet under `Added / Operator Tooling`,
resolver `Fixed` entry rewritten to describe the
existence-based final shape.
The gateway is designed for systems already serving DHCP and DNS to
a LAN segment (canonically an OpenWrt AP). On those systems port 53
is already taken by the existing resolver, so the prior `[::]:53`
default conflicted with the gateway's intended deployment target out
of the box.
The OpenWrt ipk previously overrode this in its packaged config as a
workaround; matching the source default to what the canonical
deployment actually wants makes the override redundant and removes a
foot-gun for fresh manual Linux-host installs. The redundant
`dns.listen` line in `packaging/openwrt-ipk/files/etc/fips/fips.yaml`
is dropped along with this change.
Operators on a host without a pre-existing resolver on port 53 can
opt back into the wildcard bind by setting `dns.listen: "[::]:53"`
explicitly. The new default binds IPv6 loopback only — Linux IPv6
sockets bound to explicit `::1` do not accept v4-mapped traffic, so
forwarders that reach the gateway over IPv4 loopback need to be
pointed at an explicit IPv4 listen address instead.
Touches the gateway config struct and its default-value test, the
commented-out gateway example in the Debian common fips.yaml, the
OpenWrt ipk config (override removed), the gateway reference /
how-to / design / tutorial / troubleshoot docs, and a CHANGELOG
entry under [Unreleased] -> Changed.
- Add a Documentation entry covering the docs/ reorganisation,
top-level getting-started.md, per-section landing pages,
source-accuracy pass, and gateway feature-set rewrite.
- Add a Fixed entry covering propagation of spanning-tree updates
whose changes are confined to internal path edges (no root or
depth delta).
- Add a single rolled-up entry covering expanded test coverage
across the new-feature surface plus CI hardening.
- Drop a tree-ancestry test-determinism bullet that did not change
user-visible behaviour.
Daemon and client tools previously evaluated the same three locations
(`/run/fips`, `XDG_RUNTIME_DIR`, `/tmp`) in different orders, allowing
fipsctl/fipstop to connect to a socket the daemon never bound when
neither side set `node.control.socket_path` explicitly.
Collapse the three call sites (`default_control_path`,
`default_gateway_path`, `ControlConfig::default_socket_path`) into a
shared `resolve_default_socket` helper. Canonical order is
`/run/fips` -> `$XDG_RUNTIME_DIR/fips/` -> `/tmp/fips-<name>`. Two
hardening fixes folded in: writability is probed via tempfile create
rather than mode bits (ACL- and group-aware), and `XDG_RUNTIME_DIR`
is validated as an existing directory before being used (avoids
stale post-logout values).
The deployed fleet is unaffected -- packaged configs set
`node.control.socket_path` explicitly. The fix surfaces for dev
runs and the binary-install getting-started path.
Open-discovery NAT traversal succeeds at the UDP layer regardless
of what FMP-protocol version the peer speaks. When the daemon
discovers a peer running a different FMP version (e.g. a v0/v1 mix
during a mid-rollout window, or a misconfigured peer in the same
advert namespace), the punch sequence completes, the socket is
adopted via `Node::adopt_established_traversal`, and we initiate
an FMP handshake. The peer drops our msg1 at its own version-gate
and we drop their msg1/msg2 at `Unknown FMP version, dropping`.
Neither side advances the handshake.
Today the bootstrap transport sits idle until the 31s stale-
handshake timeout, drops, and the open-discovery sweep ~30s later
fires the full STUN+offer+answer+punch sequence again — every
minute, indefinitely, against peers the handshake literally cannot
complete with.
Add a `Node::bootstrap_transport_npubs` map populated alongside
`bootstrap_transports` at adopt time. The rx loop reverse-maps the
transport_id → npub on version-mismatch and bumps the discovery
layer's `failure_state` to a long structural cooldown via the new
`NostrDiscovery::record_protocol_mismatch` API. The next sweep
skips the npub for `protocol_mismatch_cooldown_secs` (default
86400 = 24h, separate from the 30-min transient-failure
`extended_cooldown_secs`).
One-shot WARN per fresh observation. Repeat mismatches inside the
cooldown window are silent (the failure_state method returns false
when an existing comparable cooldown is already in place). The
handshake/transport teardown chain is unchanged — the fix is
specifically about preventing the *next* sweep cycle from
re-traversing.
Cleared on `cleanup_bootstrap_transport_if_unused` and on the
adopt-failure rollback path so completed handshakes don't leave
stale entries behind.
Four new unit tests in `failure_state.rs` cover fresh-entry
signaling, repeat-suppression inside the window, streak-pin
behavior for `show_peers` rendering, and post-cooldown re-arming.
The TUN-side TCP MSS clamp consults `path_mtu_lookup` (FipsAddress-
keyed) when sizing outbound TCP flows. Until now, only the reactive
`MtuExceeded` handler mirrored the bottleneck MTU into that store;
the proactive end-to-end `PathMtuNotification` echoed by the
destination updated only `MmpSessionState.path_mtu`, leaving the TUN
mirror stale.
On stable long-lived paths, the proactive echo can tighten the
session-canonical MTU well before any transit router fires a
`MtuExceeded` for those flows (since all current traffic is already
sized by the tighter session value). New TCP flows opened during
that window get clamped by the discovery-time value rather than the
session-canonical one, leading to PMTU-D loss until the reactive
path eventually fires.
Mirror the post-apply MTU into `path_mtu_lookup` whenever
`apply_notification` returns true, with the same tighter-only
semantics as the reactive mirror — never loosen the clamp. Gated on
the bool return so spurious writes don't happen on rejected
increases or no-op same-value notifications.
Four new unit tests exercise the empty-lookup write, tighten-
existing, keep-tighter-existing, and no-session-no-op paths,
parallel to the existing reactive-mirror test trio.
- Reorganize Added into 11 subsections ordered by importance and
protocol layer: Mesh Layer (FMP), Platform Support, Mesh Peer
Transports, Security, LAN Gateway, IPv6 Adapter, Operator Tooling,
Packaging and Deployment, Examples, Documentation.
- Add entries for peer ACL enforcement (#50), MIPS portable_atomic
(#62), inbound mesh port forwarding, and historical node and
per-peer statistics with btop-style graphs (#64).
- Add a Fixed entry covering the TCP-over-FIPS reliability work
(transport_mtu determinism, per-destination TCP MSS clamp at the
TUN boundary, reactive MtuExceeded mirror, Windows TUN reader
plumbing).
- Recast the multi-backend DNS configuration entry as a systemd-host
overhaul under IPv6 Adapter (default ::1 bind, global drop-in
backend, five-distro test harness); fold the Ubuntu 22 Fixed entry
into it.
- Expand the cargo feature flag rationalization Changed entry to
cover PR #79's full scope: tui, ble, gateway, and nostr-discovery
all dropped.
- Drop the rekey msg1 Fixed entry; all cases require new-in-release
functionality.
- Collapse the BLE transport bullets into one consolidated bullet
under Mesh Peer Transports and scrub stale cargo-feature wording
from the overlay-discovery and BLE bullets.
When a UDP transport had `advertise_on_nostr: true` + `public: true`
+ `bind_addr: 0.0.0.0:NNNN`, the advert builder previously read the
kernel's `local_addr()`, found `0.0.0.0`, filtered it out (correctly
— wildcard isn't a valid advertised endpoint), and silently emitted
no UDP endpoint in the published advert. Operators on AWS EIP / GCP
/ Azure setups (where binding to the public IP directly is impossible
because 1:1 NAT does the address translation off-host) had no way to
advertise UDP without binding to a specific local IP — and no log
explaining what was happening. TCP had the same shape, with no
`public: true` precondition.
Three pieces, layered. UDP gets zero-config autodiscovery via STUN;
both UDP and TCP get an explicit operator-supplied override; the
fall-through path now logs loudly instead of silently skipping.
UDP public-IP autodiscovery (STUN)
----------------------------------
In the UDP `is_public()` + wildcard-bind branch, run a one-shot
STUN observation against an ephemeral UDP socket on the daemon's
configured `stun_servers`. Take the reflexive IPv4 (the
STUN-reported port is the ephemeral source port and is discarded),
combine with the configured listener port for the advert
(`udp:<reflexive-ip>:<port>`). Works on AWS EIP / GCP / Azure
1:1-NAT setups because STUN sees the public-Internet egress IP and
the bind port is preserved through 1:1 NAT.
Result is cached per-transport on a new `public_udp_addr_cache`
field on `NostrDiscovery` (keyed by `TransportId.as_u32()`).
Asymmetric cache TTL: a successful observation is cached for
`advert_refresh_secs` (default 30 min) so we don't STUN every
refresh tick. A failed observation is cached for only 60s
(`PUBLIC_UDP_ADDR_FAILURE_TTL`) so a transient STUN flake at
startup retries within ~a minute and the advert grows its UDP
endpoint as soon as STUN starts working — rather than waiting the
full 30-min cycle.
The shared `observe_traversal_addresses` STUN helper had a
hard-coded 2s per-server response wait, right for the
per-traversal flow (latency-sensitive — 3 STUN servers worst-case
= 6s) but too short for the one-shot advert-publish startup
discovery. Parameterized `per_server_timeout` on the helper, with
two named constants in `stun.rs`: `TRAVERSAL_STUN_TIMEOUT = 2s`
(existing call sites) and `ADVERT_STUN_TIMEOUT = 5s` (new
public-UDP discovery path). Both use `tokio::time::timeout_at`
under the hood, so success returns immediately — the timeout is
only the worst case.
`external_addr` override (UDP + TCP)
------------------------------------
New `external_addr: Option<String>` field on `transports.udp.*`
and `transports.tcp.*` for explicit advertise-as override. Takes
precedence over both the bound `local_addr` and (for UDP) the
STUN-derived autodiscovery.
Required for TCP on cloud-NAT setups (AWS EIP, GCP/Azure external
IPs) where binding to the public IP directly fails with
`EADDRNOTAVAIL` because the public IP isn't on a host interface —
the network fabric does 1:1 NAT off-host. Without this field the
operator's only TCP path was "leave advert off" or "find a way to
make the public IP locally bindable."
For UDP, `external_addr` is optional but useful as a deterministic
alternative to STUN. Operators who want to skip STUN egress, whose
STUN servers are blocked, or who want the daemon to not depend on
external services for advert content can specify it explicitly.
The accessor parses two shapes:
- Bare IP (`"54.183.70.180"` or `"2001:db8::1"`): combines with
the configured `bind_addr` port.
- Full host:port (`"54.183.70.180:8443"` or `"[2001:db8::1]:443"`):
used verbatim — useful for port-forward setups where the
externally-visible port differs from the bind port.
Final precedence in `Node::build_overlay_advert` (now async, only
caller `refresh_overlay_advert` was already async):
- UDP: `external_addr` → non-wildcard `local_addr` → STUN → loud warn
- TCP: `external_addr` → non-wildcard `local_addr` → loud warn
Loud warns instead of silent skips
----------------------------------
The wildcard-bind fall-through paths now log a `warn!` pointing at
the operator-side fixes:
- UDP: "set transports.udp.external_addr, bind to a specific
public IP, or ensure node.discovery.nostr.stun_servers is
reachable"
- TCP: "Either set external_addr to the public IP (recommended for
cloud 1:1-NAT setups) or bind explicitly to the public IP"
Replaces the silent skip that previously cost operators a
debugging session when the advert mysteriously contained only the
Tor onion endpoint.
Tests
-----
11 new unit tests in `src/config/transport.rs` covering the
parser (IPv4/IPv6, bare/full, malformed) and the accessor (UDP
with default bind, UDP with explicit port override, UDP unset,
TCP without bind_addr, TCP with bind_addr, TCP with full
socket-addr override, parse_bind_port for IPv4/IPv6/malformed).
The 38-test nostr suite still passes.
CHANGELOG entries under `[Unreleased]` Fixed.
Public-test daemons with populous open-discovery caches generate
sustained NAT-traversal-failure WARN volume (~140/hour, ~3500/day)
against cache-learned peers that have gone offline — their adverts
are absent from major Nostr relays but cached entries persist until
their advertised `valid_until` expires. The daemon kept publishing
offers indefinitely under exponential backoff with no per-peer
suppression, drowning operator signal and hammering relays. A
parallel concern: the strict freshness check at signal.rs silently
rejected offers under modest clock skew (now_ms() anchors to
SystemTime once at startup, so post-startup NTP step adjustments
don't propagate on long-uptime daemons), indistinguishable from
"peer is offline."
Six independent improvements layered on the existing retry logic.
Per-npub WARN log rate-limit
----------------------------
New `FailureState` struct on `NostrDiscovery` records per-npub
`last_warn_at_ms`. Subsequent failures inside `warn_log_interval_secs`
(default 5 min) emit DEBUG instead of WARN. Each WARN now also
carries `consecutive_failures` and remaining `cooldown_secs` so
operators can read the trajectory without grepping multiple lines.
Per-npub consecutive-failure counter + extended cooldown
--------------------------------------------------------
After `failure_streak_threshold` (default 5) consecutive failures
against a peer, the next `extended_cooldown_secs` (default 1800)
of attempts are suppressed by pushing
`retry_pending[npub].retry_after_ms` past the cooldown wall. The
open-discovery sweep also consults `cooldown_until` and increments
a new `skipped_cooldown` counter so a peer whose `retry_pending`
was cleared by max_retries doesn't get re-enqueued during the
cooldown window. Caps offer-publish rate per dead peer regardless
of how often the sweep tries to re-enqueue.
Stale-advert eviction on streak-threshold transition
----------------------------------------------------
On the threshold-crossing transition (one-shot, not every
subsequent failure), `tokio::spawn` an active re-fetch of the
peer's Kind 37195 advert from `advert_relays`. Three outcomes:
- absent on relays → cache evicted; sweep won't re-enqueue
(peer is genuinely gone).
- newer `created_at` → cache refreshed + streak reset
(peer republished; allowed to retry immediately).
- same → cache untouched; cooldown stands.
Cost: ~one fetch per dead peer per 30-min cooldown cycle, vs
hundreds of offer publishes/hour today.
Clock-skew tolerance on freshness check
---------------------------------------
`signal.rs` `validate_offer_freshness` and
`validate_traversal_answer_for_offer` now allow ±60s grace beyond
strict TTL. Both return a new `FreshnessOutcome` enum so callers
can DEBUG-log when an offer/answer was only accepted via the grace
window. `FRESHNESS_SKEW_TOLERANCE_MS` is hard-coded — loosening
this past minutes erodes the freshness/replay security boundary
and operators tend to tune in the wrong direction.
NTP-style skew estimate (offer_received_at echo)
------------------------------------------------
Added optional `offerReceivedAt: Option<u64>` field to
`TraversalAnswer` payload. Responder fills it with `now_ms()` at
offer-receipt time. Initiator computes the standard NTP offset
formula `((T2-T1) + (T3-T4)) / 2` against the round-trip and
DEBUG-logs when `|skew| ≥ 30s`. Skew is also stashed in
`FailureState` and surfaced in `show_peers`. Non-breaking — older
responders that don't fill the field still produce valid answers,
and `estimate_clock_skew` returns `None`.
Per-peer state in `show_peers` JSON
-----------------------------------
Each peer entry in `show_peers` now carries:
"nostr_traversal": {
"consecutive_failures": <u32>,
"in_cooldown": <bool>,
"cooldown_until_ms": <u64 | null>,
"last_observed_skew_ms": <i64 | null>
}
Always emitted (schema-stable); values populated when discovery is
enabled and the npub has a recorded entry. Required a new public
`Node::nostr_discovery_handle()` accessor and refactored
`FailureState`'s internal Mutex from `tokio::sync` to `std::sync`
(operations never hold across await), which lets the synchronous
`show_peers` handler call `snapshot()` directly without the
dispatcher becoming async.
New config knobs (under `node.discovery.nostr`)
-----------------------------------------------
failure_streak_threshold: 5
extended_cooldown_secs: 1800
warn_log_interval_secs: 300
failure_state_max_entries: 4096
Tests
-----
12 new unit tests:
- 5 in `tests.rs` covering freshness strict / tolerated / rejected
outcomes, NTP skew estimation, and the backward-compat None case
when the responder didn't fill `offer_received_at`.
- 7 in `failure_state.rs` covering streak/warn-rate-limit state
transitions, cooldown active vs expired semantics,
success-resets-streak, observed-skew records, and size-cap
eviction by oldest `last_failure_at`.
CHANGELOG entries added under `[Unreleased]` Fixed.
Under `node.discovery.nostr.policy: open`, the per-tick auto-dial in
`queue_open_discovery_retries` was supposed to pick up adverts cached
from the relay subscription backlog at startup, but in practice only
adverts arriving live (after the daemon was up) were being dialed.
Backlog adverts sat in the in-memory cache until they aged out.
Adds a one-shot startup sweep that runs once per daemon start, gated
identically to the per-tick sweep (`enabled` && `policy == open`),
after a configurable settle delay so the relay subscription backlog
has time to populate the advert cache. The sweep iterates the cache
with the same skip-filters as the per-tick path (statically-configured
peers, already-connected, retry-pending, connecting) plus a tighter
age filter: only adverts whose `created_at` is within
`startup_sweep_max_age_secs` of now are queued.
Two new config fields under `node.discovery.nostr`:
- `startup_sweep_delay_secs` (default 5)
- `startup_sweep_max_age_secs` (default 3600 = one hour)
Both are only consulted when `policy == open`; under any other
policy the sweep is a no-op.
Adds diagnostic logging to the open-discovery sweep so operators can
verify what the auto-dial path is doing on each daemon bring-up:
info-level on each retry-queued enqueue (with peer short-npub and
advert age), and a one-line summary on every startup sweep and on
any per-tick sweep that queues at least one retry. The summary
breaks down skipped candidates by reason (age, configured, self,
already-connected, retry-pending, connecting, no-endpoints,
invalid-npub) — currently the path was silent so there was no
operator-visible signal that the cache iteration was running.
Refactors the existing `queue_open_discovery_retries` body into a
shared `run_open_discovery_sweep(max_age_secs, caller)` helper so
the per-tick and startup paths share filter/queue logic and only
differ in the age filter and log label. Surfaces `created_at` from
`NostrDiscovery::cached_open_discovery_candidates` (return tuple
extended) so the age filter has the data it needs.
Three new unit tests in `config::node::tests` cover the new defaults,
YAML override round-trip, and partial-YAML default fallback.
The Nostr overlay advert publisher serialized `transport: tor`
endpoints as a bare `<onion>.onion` hostname with no port. The Tor
address parser requires `<host>:<port>` form and rejected the bare
shape with `expected host:port`. Any peer receiving a Tor-only
advert went into a persistent retry-fail loop on jittered backoff
until the advert aged out of the discovery cache. The bug had been
latent for as long as Tor adverts have been published on Nostr, and
was masked in deployments where every node also advertised a
non-Tor transport (peers fell through to the working endpoint).
Surfaced first on a deployment where Tor was the only advert path.
Publisher now emits `<onion>.onion:<port>` using a new
`transports.tor.advertised_port` config field that defaults to 443,
matching the Tor `HiddenServicePort 443 127.0.0.1:<bind_port>`
convention. Operators whose torrc uses a non-default virtual port
can override.
Adds a unit test that pins the publisher/parser contract: formats
the advert exactly as the publisher does and asserts `parse_tor_addr`
accepts the result; asserts the bare-onion form (the bug) does not
parse, catching any future regression that drops the port again.
Parser is unchanged (already correct).
Johnathan Corgan
sandwichandJohnathan Corgan
Martti MalmiandJohnathan Corgan
Arjen