The Linux recv path drains up to 32 datagrams per kernel wakeup via
recvmmsg(2), amortising the per-syscall + per-task-wakeup cost across
the burst. macOS still fell through to single-packet recv_from, so
the same overhead capped inbound rate on Apple builds.
Add an equivalent batch path for Darwin using recvmsg_x(2). It is a
xnu-private syscall (not in the public SDK) but is the canonical
amortisation primitive on macOS — same shape used by quinn-udp for
the same reason. ABI is the public msghdr layout plus a trailing
msg_datalen (per-datagram bytes-received output), declared via
`unsafe extern "C"` against a local repr(C) `msghdr_x`.
Same `(count, kernel_drops)` contract as the Linux `recv_batch`. macOS
has no SO_RXQ_OVFL equivalent, so `kernel_drops` is always 0 — the
1Hz `sample_transport_congestion()` detector simply sees no kernel
drop signal on Apple hosts (it already tolerates that, since the
field has been 0 there pre-batching too).
cmsg buffer is intentionally null: we never consume ancillary data on
this path, and quinn-udp documents that `recvmsg_x` does not overwrite
`msg_controllen` on macOS 10.15+ (zeroed init is the only safe state).
The udp_receive_loop dispatch widens from cfg(linux) to
cfg(any(linux, macos)); the per-packet recv_from path is now used
only on the remaining unix targets (BSDs etc.) and Windows.
Add test_burst_recv_batch exercising 10 in-flight datagrams to
verify per-datagram boundaries and arrival order across the batch.
Add an ignored bench_udp_recv_amortization measuring recv-side
syscall amortization across 1/2/4/8 sender threads on dedicated
blocking std threads (kernel rx queue stays saturated regardless of
tokio scheduling). Sample numbers on aarch64-apple-darwin (100B
payloads, 3s windows):
senders=1: recv_from 398k pps recv_batch 432k pps 1.09x
senders=2: recv_from 353k pps recv_batch 608k pps 1.72x
senders=4: recv_from 322k pps recv_batch 503k pps 1.56x
senders=8: recv_from 353k pps recv_batch 515k pps 1.46x
Gate the Linux-only IpAddr import in control::listening behind a
cfg(target_os = "linux") so the macOS test build is warning-clean
now that test code paths there compile.
New testing/boringtun/ harness runs two Cloudflare BoringTun userspace
WireGuard containers with iperf3 between them, giving a single-hop
userspace tunnel baseline for comparison against FIPS throughput
numbers. Local WG key generation runs through the harness image so the
host needs no wireguard-tools.
New testing/static/scripts/iperf-compare-refs.sh builds two git refs
into separate fips-test:* images via git worktree and runs the same
static iperf topology against both, with RUNS-based repetition and
aggregate avg/min/max reporting.
testing/static/scripts/iperf-test.sh gains DURATION, PARALLEL,
SETTLE_SECONDS, IPERF_TIMEOUT env knobs and a per-path iperf timeout.
testing/static/docker-compose.yml selects the image under test via
FIPS_TEST_IMAGE; testing/scripts/build.sh respects CARGO_TARGET_DIR.
Author benchmark on aarch64 Docker Desktop:
boringtun bob -> alice : 1000.13 Mbits/sec
Skip BootstrapEvent::Established and BootstrapEvent::Failed dispatch
in poll_nostr_discovery for peers that are already connected or
actively handshaking. Without these guards, stale traversal events
arriving after a peer connected through a different path would
either attempt to adopt a redundant socket against the live
connection (Established) or poison the per-peer failure-state
cooldown and trigger redundant retraversal via schedule_retry /
try_peer_addresses (Failed).
The four guard sites use a new is_connecting_to_peer helper extracted
from the existing closure inside initiate_peer_connection; the helper
checks for an in-flight outbound handshake state. adopt_established_traversal
gains a defense-in-depth check returning PeerAlreadyExists when called
against an already-promoted peer, so the invariant holds if a future
caller bypasses the outer dispatch guard.
Side benefit: narrows a cooldown-poisoning vector previously available
to an attacker injecting stale failure events for an active peer.
Test coverage for the new behavior:
- test_try_peer_addresses_skips_connected_peer
- test_try_peer_addresses_skips_connecting_peer
- test_nostr_traversal_failure_skips_connected_peer (Failed-arm event
injection)
- test_nostr_traversal_established_skips_connected_peer (Established-arm
event injection, mirror of the Failed test)
- test_adopted_traversal_skips_already_connected_peer
(adopt_established_traversal defense-in-depth)
CHANGELOG entry under [Unreleased] / Fixed.
Closes#87
Set `REKEY_JITTER_SECS` to 0 on next. The symmetric per-session
jitter mechanism added on maint/master works cleanly on the IK
FMP rekey path on those branches, but on next's XX FMP rekey
path it produces reproducible post-cutover routing-convergence
failures: ~50% Phase 5 per-pair-ping loss in the rekey integration
suite across multiple runs, with all crypto, transport, and
link-state log assertions still green (no decrypt failures, no
panics, no link teardowns, no rekey msg2 failures, no FSP
decryption failures during rekey).
The XX rekey path itself is sound — pre-merge it had been passing
18/18 across six consecutive CI runs. Isolation experiment
confirms the trigger: with REKEY_JITTER_SECS=0, the same suite
on next passes 70/70 cleanly with FMP cutover counts (18-21)
in line with master's baseline (24). With the constant at 15
and the test suite's 35s nominal rekey interval, FMP cutover
counts climb to 27-40 and Phase 5 fails consistently.
Two attempted fixes to absorb the jitter-induced state churn on
the XX path (post-msg3 addr_to_link restore; optimistic dampening
at msg1 receipt) were tried and reverted — neither restored
Phase 5 reliability. Diagnosis of the underlying XX cutover
state-cleanup behaviour that fails to absorb variable-interval
rekeys is deferred. The const and surrounding draw/redraw
machinery (peer::active::ActivePeer, node::session::SessionEntry,
draw_rekey_jitter at session.rs:20, redraws at the four cutover
sites) are kept in place; flipping the const back to a positive
value is the re-enable path once the investigation lands.
CHANGELOG entry under `### Changed`.
The two rekey-jitter unit tests added on the IK-rekey branches
construct a HandshakeState via:
HandshakeState::new_initiator(keypair, remote_pubkey)
On next, the XX rewrite changed the initiator constructor to take
only the local keypair:
HandshakeState::new_initiator(keypair)
XX does not learn the responder's static pubkey until msg2, so
the IK-shaped signature does not apply. The merge from master
brought the test bodies verbatim; this drops the second argument
in both call sites so the tests compile against the XX API. The
test semantics (rekey-jitter range and mean checks) are unchanged
and orthogonal to the handshake-pattern choice.
Same shape as 8a6477b adapting the bootstrap-handoff tests to
XX's three-way handshake.
Bring [Unreleased] into sync with all maint commits since v0.3.0:
- Add a Fixed entry for the acl-allowlist test-script poll-assertion
conversion (commit e9dd316) that was previously missing.
- Add the AUR-publish workflow rewrite and new fips-git VCS workflow
(commit 9bf9701) which had no entry, and merge them with the
ci.yml cancel-in-progress block under a single "CI and
release-publish workflows hardened" entry so the three workflow
changes read as one operational theme.
Pure changelog content reshuffle. No code touched.
Add a top-level concurrency block to ci.yml keyed on
(workflow, ref) with cancel-in-progress: true. Pushes (including
force-pushes) to the same ref now retire any in-flight run for
that ref rather than letting the superseded and current-tip runs
both burn runner minutes.
Motivated by a 2026-05-14 force-push experience where two CI runs
ran concurrently against a feature branch — the original push at
c76ec99 continued for ~17 minutes alongside the amended push at
927ef47, despite only the latter being the live tip.
Scope deliberately limited to ci.yml. Tag-triggered release-build
workflows (package-*.yml, aur-publish-*.yml) are untouched — they
operate on per-tag refs that already form distinct concurrency
groups and release artifact builds generally should not be
cancellable by unrelated activity.
Phase 5's per-pair connectivity check ran immediately after the second
rekey cycle, with no settle for routing reconvergence. Under
GitHub-runner CPU contention, post-rekey parent-switches and
coord-cache flushes can take longer than the per-ping 5s timeout for
a small fraction of pairs (1-3 of 20 typically), even though the
rekey mechanism itself completes cleanly. Phase 6 log analysis stays
all-green on these failed runs; the failure is purely connectivity
timing.
Mirror Phase 3's existing 12-second settle pattern: reuse REKEY_SETTLE
and emit the same banner shape. Two-line change. Cost on the success
path is a fixed 12s per suite run.
Convert assert_log_contains from a one-shot grep snapshot into a
bounded poll that retries until the pattern appears or the timeout
elapses (default 15s). Same wait-with-timeout shape as
wait_for_peers_exact above it in the file.
The pre-existing flake on next-branch CI is structural: under XX
handshake, the cross-connection tie-breaker selects which side
reaches its ACL-check point first. When container-a wins the
tie-breaker on the first attempt against c and d, only the
outbound-handshake-context rejection fires immediately, and the
inbound-handshake-context rejection only emits on a later retry
when c or d's msg1 lands while a has no pending outbound. On one
2026-05-14 run the inbound rejection appeared 63ms after the test
had given up. The race window is small but real.
Polling the log instead of one-shot reading absorbs the
millisecond-to-second variance without slowing the success path
(the helper returns as soon as the pattern appears).
Add a per-session signed jitter offset (uniform [-15, +15] seconds)
to the rekey timer triggers in check_rekey (FMP) and check_session_rekey
(FSP). The configured `node.rekey.after_secs` becomes the nominal
interval rather than a floor; mean is preserved. Desynchronizes
both endpoints in symmetric-start meshes so the dual-initiation
race stops occurring rather than being resolved after the fact by
the smaller-NodeAddr tie-breaker.
Per-session storage means each rekey cutover reconstructs the
session and redraws the jitter naturally — successive cycles get
independent offsets, preventing drift back into sync.
The `mixed-profile` integration suite (Full / NonRouting / Leaf
node-role mix introduced by the forklift-upgrade work) is wired
into `testing/ci-local.sh` but was silently absent from the GitHub
Actions integration matrix on next. Regressions to node-role
behavior could land via GitHub CI without the suite running.
Adds a matrix entry plus the matching step block (mirroring the
rekey pattern: generate configs + inject config + compose up +
test + log-collect on failure + compose down always). master
correctly omits the suite from both ci-local and ci.yml since
mixed-profile is forklift-only, so this change never forward-merges
to master.
The dual-initiation tie-breaker in the rekey arm of handle_msg3 (and
the FSP analogue in handle_session_setup) only fired when
rekey_in_progress was true. With Noise IK (one-message rekey on
master/maint) that is sufficient: msg1 IS the rekey, so both sides
being mid-handshake is the only state where the race can fire.
With Noise XX (three-message rekey on next), set_pending_session runs
when the initiator has processed msg2 and sent msg3, and that call
clears rekey_in_progress. Both sides' set_pending_session can run
before either peer's msg3 has landed. Each side then receives the
peer's msg3 in the post-pending state with rekey_in_progress=false,
falls into the "drop because pending_new_session is set" guard, and
discards the peer's handshake. Each side commits its own initiator
session at K-bit cutover. The two sessions use different Noise key
material, so the link breaks asymmetrically after cutover.
Unify both checks into a single tie-breaker that fires when either
rekey_in_progress() OR pending_new_session().is_some() is true. The
existing smaller-NodeAddr rule applies uniformly; abandon_rekey()
already clears both states and returns whichever index needs
freeing.
Mirrored to the FSP rekey msg1 path in handle_session_setup for the
same race shape.
Logging:
- The previously-silent drop in the rekey arm of handle_msg3 logs at
info as a tie-break decision rather than as a silent drop.
Pending_new_session is added as a log field on the tie-break
win/lose lines so a reader can distinguish which of the two race
states fired.
- Cutover-complete and K-bit-flip log lines gained our_addr and
their_addr fields so the two endpoints' logs of the same handshake
can be correlated.
- "Pending session set, awaiting K-bit cutover" lines remain at
debug per the existing convention of info-for-cutover-completion-
only.
Verification: under the un-fixed handler, both sides log
"rekey-msg3 drop: pending_new_session already set" within 242
microseconds of each other, with rekey_in_progress=false and
different pending session indices. Failure pattern matches: six of
twenty pairs FAILED post-rekey, all involving the single-peer node.
Six consecutive post-fix CI attempts on the same test green across
rekey, rekey-accept-off, and rekey-outbound-only suites.
Three entries under [Unreleased] for the three commits since the
v0.3.0 release tag:
- Sidecar example: FIPS_UDP_MTU env override (commit 32a3b58)
- CONTRIBUTING.md overhaul + new docs/branching.md (commit 538ce07)
- Rekey-test Phase 1 baseline-convergence headroom 36s -> 60s
(commit 6533276)
The previous CONTRIBUTING.md read like a stock Rust contributing
template that mentioned FIPS. Replace it with an entry-point doc
that gives new contributors the FIPS-specific mental model they
need to make a useful first PR: the FMP/FSP layering and why
mesh-level changes need multi-node testing, the three-branch
release model and how to choose a target branch, structured bug
reporting expectations, and PR submission requirements (scope
discipline, the local-CI ladder, separate requirements for feature
PRs vs bug-fix PRs, squash-merge mechanics).
Add a contributor-facing AI coding assistant policy: use is
welcome, but the contributor must do a thorough manual review and
editorial pass before submission. The agent is a tool; the
contributor is accountable for the submission. Review effort
scales with submission effort -- unreviewed agent output will
receive an agent reply in turn, without human review.
Add docs/branching.md as the long-form companion covering the
release workflow, version conventions, and merge-direction
rationale. The new CONTRIBUTING.md is the day-to-day entry point;
docs/branching.md is the reference.
All cross-references resolve against current HEAD. Previous
stale links to fips-intro.md, fips-wire-formats.md, and
fips-configuration.md are gone; the doc points at the current
docs/design/ layout, docs/getting-started.md,
docs/tutorials/join-the-test-mesh.md, and testing/README.md.
No code changes.
The Phase 1 pre-rekey baseline in `wait_for_full_baseline`
occasionally times out on GitHub-hosted runners with one ping pair
failing to converge inside the BASELINE_CONVERGENCE_TIMEOUT window.
Phases 2–6 always pass cleanly when this happens — the rekey itself
is fine, the mesh just hasn't finished spanning-tree + bloom-filter
convergence by the time Phase 1 starts pinging.
The wait loop returns as soon as all 20 pairs converge, so the cost
on the success path is unchanged (typical local CI returns well
under the old 36s). The bump only adds headroom on the failure
path. Operators previously worked around this with
`gh run rerun --failed`; this aims to retire that workaround for
the IK/XK lines.
Distinct from the next-branch XX rekey dual-init race, which is a
real protocol bug tracked separately.
The sidecar entrypoint hardcoded `udp.mtu: 1472`, the Docker-bridge
IPv4 maximum (1500 MTU - 8 UDP - 20 IPv4 header). Promote it to
`FIPS_UDP_MTU` (defaulting to 1472, preserving behavior) so non-Docker
reuses of the example can override without editing the script. Plumb
the env var through `docker-compose.yml` so a host-level setting
reaches the container, and add a row to the README's env-var table.
Also annotate the static-CI node template with a comment explaining
the same 1472 rationale and the daemon's 1280 default. The template
keeps 1472 as the literal value since the CI suite runs on a Docker
bridge where that's correct.
No behavior change unless the host explicitly sets FIPS_UDP_MTU.
The v0.3.0 stable AUR push silently failed: with updpkgsums: true,
makepkg downloaded fips-<ver>.tar.gz into the AUR working tree, where
it was then staged by the deploy action and rejected by AUR's 488 KiB
max-blob hook.
Fetch the upstream source tarball and compute its b2sum in CI, patch
pkgver and the b2sums SKIP placeholder in PKGBUILD in-place, then
publish with updpkgsums: false so the AUR clone stays metadata-only.
Recompute and patch the fips.sysusers / fips.tmpfiles asset b2sums in
the same step so they stay in sync with the local files; this safety
net was previously provided by updpkgsums.
Add aur-publish-git.yml for the VCS fips-git PKGBUILD, triggered on
master pushes that touch PKGBUILD-git or its companion files plus
workflow_dispatch. pkgver is computed at build time by the PKGBUILD's
pkgver() function, so this workflow is not tied to release tags.
Add a workflow_dispatch tag input on the stable workflow so historical
release tags can be re-published manually, and drop continue-on-error:
true so future regressions surface in CI.
Marker merge to record the master dev-line (now v0.4.0-dev, with
maint's v0.3.1-dev open already incorporated) as known on next
without taking master's tree (next is on v0.5.0-dev). Future
forward-merges from master to next land cleanly on top of this base.
Marker merge to record the maint dev-line as known on master
without taking maint's v0.3.1-dev opening commit (master is on
v0.4.0-dev). Future bug-fix forward-merges from maint to master
land cleanly on top of this base.
Master is now the v0.4.0-dev line; next moves up to v0.5.0-dev for
the wire-format-breaking work staged here.
- Cargo: 0.4.0-dev → 0.5.0-dev
- CHANGELOG Breaking-section header retargeted to v0.5.0; baseline
bumped from v0.2.x to v0.3.x peers (master is now the v0.3.1-dev
patch line and v0.4.0-dev minor line, so v0.5.0 breaks against
v0.3.x, not v0.2.x)
- README: badge v0.4.0--dev → v0.5.0--dev, three Status & Roadmap
prose references retargeted
- docs/reference/cli-fips.md: example version 0.4.0-dev → 0.5.0-dev
Reset maint to v0.3.0 to retire the v0.2.x maintenance window and
open a new tracking branch for v0.3.1-dev bug-fix work.
- Cargo: 0.3.0 → 0.3.1-dev
- CHANGELOG: fresh [Unreleased] block above [0.3.0]
- README: badge v0.3.0 → v0.3.1--dev
Brings v0.3.0 release content forward into the next-branch
development line, preserves next's own version state, and updates
operator-facing version references to v0.4.0-dev.
Kept from master:
- docs/releases/release-notes-v0.2.1.md and -v0.3.0.md (cumulative
archive of shipped releases)
- RELEASE-NOTES.md root mirror at v0.3.0 (tracks the most recent
shipped release; will be replaced when the v0.4.0 release cycle
begins on this line)
- CHANGELOG entries: [0.3.0] and [0.2.1] sections inserted under
the existing Breaking and [Unreleased] sections
- All code, config, test, and documentation updates from master
(openwrt yaml resync, doc-config IP placeholders, etc.)
Kept from next (resolved against master's release-prep changes):
- Cargo.toml / Cargo.lock at 0.4.0-dev (next's package version)
- CHANGELOG ## Breaking section (next-specific v0.4.0 wire-format
breaking work) and the empty ## [Unreleased] block for future
v0.4.0 non-breaking work
Updated to v0.4.0-dev for consistency with Cargo.toml:
- README badge (v0.3.0--dev -> v0.4.0--dev) and status-section
prose, rewritten to describe v0.4.0 wire-format-breaking work
on this branch (Noise XX unification, FMP node profiles,
slimmer MMP, extensible bloom-filter encoding) instead of the
v0.3.0 testing-and-polishing narrative that applied while
v0.3.0 was unreleased
- docs/reference/cli-fips.md example version string
Bump Cargo.toml version 0.3.0-dev -> 0.3.0 and resync Cargo.lock,
move the CHANGELOG [Unreleased] block under [0.3.0] - 2026-05-11,
update the README status badge and prose to v0.3.0, and add the
release notes at docs/releases/release-notes-v0.3.0.md with a
mirrored copy at the repo root as RELEASE-NOTES.md.
Uses the 'ours' merge strategy to keep all of master's tree
verbatim with one carve-out: the v0.2.1 release notes archive at
docs/releases/release-notes-v0.2.1.md is retained as a permanent
docs artifact.
Discarded from maint's release-prep commit: Cargo.toml / Cargo.lock
version bumps (master stays on v0.3.0-dev), README v0.2.1 badge and
prose, and the RELEASE-NOTES.md root mirror (master's root mirror
will eventually carry the v0.3.0 release notes when v0.3.0 ships
from this line).
CHANGELOG: the v0.2.1 release section is integrated between
[Unreleased] (v0.3.0-dev work) and the [0.2.0] historical section.
Items that landed on maint during the v0.2.1 cycle are removed
from [Unreleased] so [0.2.1] is the canonical home for those
changes (Linux release artifact and AUR workflows; bloom fill-ratio
validation; seven maint-line bug fixes).
Operator-facing IPs in user-visible configs/docs (examples, tutorials,
packaging, sidecar templates) are now the resolvable hostnames of the
public test fleet (test-us01.fips.network, etc.) so they keep working
without baking specific addresses into examples.
Doc-comment and test fixtures in src/config/transport.rs use RFC 5737
TEST-NET-2 (198.51.100.1) so they cannot accidentally point at a real
host.
Also resyncs the openwrt-ipk fips.yaml with the common reference
(merge from master) and applies the same DNS-name swap there.
Bring the OpenWrt-shipped fips.yaml back into line with
packaging/common/fips.yaml, which had drifted: the OpenWrt copy was
missing the Nostr discovery, BLE, and TCP reference comment blocks, so
operators had no in-config hint that Nostr-mediated discovery existed.
Take common/fips.yaml verbatim and apply just the OpenWrt-specific
active overrides:
- ethernet: uncomment with wan/wwan/lan defaults (eth0, phy0-sta0,
br-lan)
- gateway: uncomment with lan_interface=br-lan and dns.listen on
[::1]:5353 (matches the dnsmasq forwarder the init script wires up)
Identity stays ephemeral by default (no persistent override), matching
common.
Bump Cargo.toml version 0.2.1-dev -> 0.2.1 and resync Cargo.lock,
move the CHANGELOG [Unreleased] block under [0.2.1] - 2026-05-11,
update the README status badge and prose to v0.2.1, and add the
release notes at docs/releases/release-notes-v0.2.1.md with a
mirrored copy at the repo root as RELEASE-NOTES.md.
The host (217.77.8.91:443) was renamed to test-us01 some time ago
(canonical name shared with packaging/common/hosts and the docs);
the test scaffold's alias labels and narrative comments hadn't been
updated. Pure naming cleanup, no behavior change. The continued
dependency on the live external host is tracked separately.
Walk through reviewer feedback on the Nostr-discovery docs and
land 18 items.
Bulk patterns:
- `external_addr` / `public: true` semantics consistently
misdescribed. The advert path is gated on `cfg.is_public()`;
inside that branch the daemon picks an address by precedence
(`external_addr`, non-wildcard `bind_addr`, STUN). The docs
treated `public: true` and `external_addr` as alternatives when
they are stacked: `public: true` is the master switch and
`external_addr` populates the address inside it. Reconciled
across `enable-nostr-discovery.md` and `advertise-your-node.md`:
add `public: true` to the `external_addr` examples; replace
"STUN as a logging cross-check" with "STUN is skipped entirely";
fix "neither flag is needed" for direct public bind (both flags
still required); make the publish-tutorial Step 3 conditional
on the chosen Step 2 path (STUN runs only on the `public: true`
path); rewrite the troubleshooting "wrong public IP advertised"
bullet with two coherent fixes.
- `udp:nat` overpromised as a symmetric-NAT solution. Symmetric
NAT on either side typically defeats the punch. Reframe
`udp:nat` as best-effort hole-punching for nodes without a
directly reachable UDP endpoint in the how-to, the publish
tutorial (intro, callout, section heading rewrite from "If
you're behind symmetric NAT" to "If your direct UDP advert
isn't reachable"), the consume tutorial's "What's next"
pointer, and `tutorials/README.md`. Promote reachability over
named NAT classes: STUN can confirm the public IP but not that
the listener-port mapping is open.
- YAML "silently ignores unknown keys" is wrong. Config parser
rejects unknown fields via `serde(deny_unknown_fields)` on the
per-section structs; misspelled fields refuse the daemon's
start with a parse-error line in the journal. Fixed in the
publish tutorial's troubleshooting and the open-discovery
tutorial's `policy` typo bullet.
Mechanical fixes:
- Repoint stale anchors. `getting-started.md` and
`configuration.md` linked to `#installation` / `#inspect` on
the README; the README has no such headings. Repoint to
`#quick-start` and `cli-fipsctl.md`. Two stale anchors in the
publish tutorial pointing at non-existent sub-scenarios in the
how-to (`#sub-scenario-2c-...`,
`#sub-scenario-2b-tor-onion-node`) repointed to the correct
anchors.
- Drop the `fipsctl show status` claim from the open-discovery
troubleshooting bullet (`show_status` doesn't include
`discovery.nostr.policy`). Replace with daemon startup logs.
- Fix the `advertise: false` parenthetical in the consume-only
tutorial (`default_advertise()` returns `true`; we set `false`
explicitly for the consume-only path).
- Drop the "supplies a relay list" overstatement in two
activation paragraphs (the how-to and the design doc). Default
relay / STUN-server lists ship in the config; both are
optional overrides.
- Add the missing `transports.udp.public` entry to the
open-discovery tutorial's prerequisites checklist. Tutorial
users coming out of advertise-your-node could be on either the
direct-UDP (`public: true`) or `udp:nat` (`public: false`)
path; list both.
Files: docs/getting-started.md, docs/reference/configuration.md,
docs/how-to/enable-nostr-discovery.md, docs/tutorials/README.md,
docs/tutorials/advertise-your-node.md,
docs/tutorials/resolve-peers-via-nostr.md,
docs/tutorials/open-discovery.md,
docs/design/fips-nostr-discovery.md.
Four short prose corrections folded together to align operator-facing
docs with current v0.3.0 reality:
- README Rust prerequisite reconciled with the toolchain pin (1.94.1,
was 1.85+).
- CONTRIBUTING.md bumps the Rust prerequisite and adds the squash-merge
policy note.
- examples/sidecar-nostr-relay/Dockerfile drops stale --features tui
and the paired --no-default-features; the tui/ble/gateway cargo
features were replaced by platform cfg gates in cbc7809.
- README Features list adds two v0.3.0-visible items: the mesh-
interface security baseline (fips.nft conffile, fips.d/ drop-in,
opt-in fips-firewall.service across all packaging formats) and the
fipsctl stats time-series queries plus fipstop inline sparkline
dashboards.
Status badge bump (v0.3.0--dev to v0.3.0) is deferred to tag time per
the release-prep checklist.
The generic systemd install tarball is the catch-all install path
for systemd Linux distros that don't have a per-format package
(Fedora, RHEL/CentOS, openSUSE, Alpine, etc.). It had drifted
behind the .deb and AUR packages and was missing fips-gateway, the
mesh-interface firewall baseline, and the multi-backend DNS helper
in the shipped tarball. Bring it to parity:
- New `packaging/systemd/fips-gateway.service` (clone of the .deb
unit; ExecStart pointed at `/usr/local/bin/fips-gateway`). Not
enabled at install time; operator opt-in.
- New `packaging/systemd/fips-firewall.service` (clone of the .deb
unit; nft path unchanged at `/usr/sbin/nft`). Not enabled at
install time; operator opt-in.
- `build-tarball.sh` now bundles the `fips-gateway` binary, the two
new units, the `fips.nft` baseline conffile, and the
`fips-dns-setup` / `fips-dns-teardown` multi-backend helpers from
`packaging/common/`.
- `install.sh` now installs `fips-gateway` to `/usr/local/bin/`,
installs both new units to `/etc/systemd/system/` (without
enabling them), preserves `/etc/fips/fips.nft` on upgrade like
`fips.yaml`, and creates the `/etc/fips/fips.d/` operator drop-in
directory. Post-install messaging mentions both opt-in services.
- `uninstall.sh` stops and disables the optional services in
dependency order (firewall, gateway, dns, daemon), removes the
new unit files, and removes the gateway binary. `--purge` already
handles `/etc/fips/` removal which covers `fips.nft` and
`fips.d/`.
- `README.install.md` documents all of the above: expanded
"What Gets Installed" table, new sections covering the firewall
baseline and the LAN gateway, refreshed DNS section reflecting
the multi-backend setup helper (systemd dns-delegate /
systemd-resolved drop-in / per-link resolvectl / dnsmasq /
NetworkManager-dnsmasq), and updated Service Management.
Also fixes a latent packaging bug: `install.sh` previously
referenced `${SCRIPT_DIR}/../common/fips-dns-setup`, a path that
exists only in the source-repo layout and not in the extracted
tarball. The script now resolves the helper from the staging
directory first (the tarball case), falling back to the source-repo
relative path. Bug latent since the multi-backend DNS helpers
landed.
CHANGELOG `[Unreleased]` documents the parity bump under Changed
and the path-resolution fix under Fixed.
Closes the longest-standing parity gap for non-Debian / non-Arch
systemd Linux distros installing from the release-distribution
tarball.
Three changes folded together close the AUR-side parity gap with the
.deb packaging:
- PKGBUILD now ships fips.nft baseline and fips-firewall.service, with
fips.nft marked as backup so operator edits survive upgrades.
- PKGBUILD-git mirrors the release PKGBUILD: installs fips-gateway
(was missing entirely), ships fips.nft and fips-firewall.service,
and tracks the same backup() set.
- packaging/aur/README.md documents the gateway, firewall service,
and nft baseline that ship as of this revision.
AUR users now receive the same artifact set as .deb users.
The Noise-session AEAD swap landed as 5cda4a9 + 9b1016f without a
companion CHANGELOG entry; add it under [Unreleased] § Changed
ahead of the rest of the perf-win cluster from PR #81 since it's
the most operator-visible single change of that class.
Investigated the mipsel-unknown-linux-musl build of this branch on a
Linux/x86_64 host. ring 0.17, portable-atomic, and the fips codebase
itself all compile cleanly for that target — the AtomicU64 portability
work is already done. The actual blocker is in the nostr-relay-pool 0.44
transitive dep, which uses std::sync::atomic::AtomicU64 directly in
src/relay/{stats,ping,flags}.rs. Verified fixable with a 6-line
portable-atomic patch via a [patch.crates-io] shim during local testing.
Updating the comment so the next person looking at this matrix has the
right starting point.
No functional change.
The chacha20 crate (RustCrypto) ships SSE2 + soft backends only — on
aarch64 (Apple Silicon, ARM Linux servers, Docker on M-series Macs) it
falls through to a portable software impl at ~600–800 MB/s/core. ring
0.17 wraps BoringSSL's hand-tuned ChaCha20-Poly1305, which dispatches
to NEON on aarch64 and AVX2/AVX-512 on x86_64 — typically 3-5 GB/s/core
on the same hardware.
Same wire format. ChaCha20-Poly1305 is byte-deterministic for a given
(key, nonce, plaintext, aad), so any correct AEAD implementation
produces identical ciphertext. The full noise test suite covers this
implicitly: IK and XK roundtrip handshakes, replay window correctness,
multi-message nonce sequencing, and 100-message stress all pass at
1129/1129 (the lib's full `cargo test` count) — these only succeed if
ring's output matches what the receiver's existing replay-window
decrypt path expects.
Implementation notes:
* `LessSafeKey` (and `UnboundKey`) deliberately do not implement
Clone for safety. `CipherState`'s manual Clone impl rebuilds it
from the retained 32-byte key — cheap for ChaCha20-Poly1305 since
construction is essentially a key copy + a constant-time check.
* The keyed AEAD is now cached in `CipherState.cipher` instead of
being re-derived per packet. This was already a perf win for the
chacha20poly1305 backend (`new_from_slice` per packet was hot in
profiles); for ring it's a bigger win because `LessSafeKey`
construction also derives the Poly1305 key.
* Public `Vec<u8>`-returning API preserved. New module-private
`seal`/`open` helpers wrap ring's `seal_in_place_append_tag` /
`open_in_place` so the per-packet allocation pattern is local to
one place.
* `EndToEndState::Established` triggers `clippy::large_enum_variant`
after the swap (`NoiseSession` grew from ~600 to ~1.5 KB because
ring precomputes the Poly1305 key state at construction). That
precomputation is the win — boxing the variant would re-add an
indirection per packet and work against it. `#[allow]`'d at the
enum decl with a justifying comment.
ring is widely deployed (rustls, hyper-rustls, AWS SDK, …) and a
pure-Rust crate (uses BoringSSL's asm via a vendored build). It
introduces no new C toolchain requirements that aren't already there
for any rustls user.
Bench data from a downstream consumer of this crate (Docker e2e,
DURATION=10, identical hardware before/after, aarch64 Linux on
Apple Silicon):
2-node direct (A↔B):
TCP 1-stream 437 → 1097 Mbps (2.51×)
TCP 4-stream 439 → 1109 Mbps (2.53×)
TCP 8-stream 445 → 1069 Mbps (2.40×)
UDP @1000 Mbit 599/40% loss → 1000 Mbps lossless
ping under load ~0.6 ms (unchanged)
3-node forced transit (A → C → B):
TCP 1-stream 438 → 1019 Mbps (2.33×)
TCP 4-stream 421 → 982 Mbps (2.33×)
TCP 8-stream 443 → 1031 Mbps (2.33×)
UDP @1000 Mbit 475/52% loss → 1000 Mbps lossless
ping under load 7.68 ms / 215 ms max → 0.72 ms / 3.6 ms max
The relay-path lift is the cleanest tell on the bottleneck: the
transit node was crypto-bound (single-threaded soft chacha couldn't
keep up with offered rate), so the queue accumulated under load. With
NEON the relay isn't crypto-bound and the queue stops accumulating —
the 215ms ping-tail collapses to 3.6ms.
Freshly-restarted nodes with policy: open silently lost the historical
event replay that relays send in response to subscribe(). The
broadcast::Receiver was created INSIDE spawn_notify_loop, which the
tokio runtime starts at some indeterminate point after subscribe()
returns. tokio's broadcast channel only delivers messages sent after
the receiver is created; messages dispatched in the gap between
subscribe() issuing the REQ and the spawned task calling
client.notifications() were dropped by external_notification_sender.send
returning Err(SendError) with no subscribers attached.
Symptom on a node with policy: open: non-configured peers were not
discovered until they next re-published their advert (default
advert_refresh_secs = 1800s = 30 min). Configured peers were unaffected
because fetch_advert (relay-fetch path) caches them at startup-sweep
time. The bug has been latent since 34e00b9 added Nostr discovery —
relay-fetch covered the common case for configured-peer setups.
Fix: create the broadcast::Receiver in start() before subscribe() and
pass it into spawn_notify_loop. The receiver now exists when the REQ
replay arrives, so historical events flow through the cache path.
Also handle broadcast::error::RecvError::Lagged separately from
::Closed. The previous `while let Ok(...) = recv().await` exited the
loop on any Err, so a single lag event would silently kill the entire
subscription consumer with no recovery. Lagged now logs a warn (with
the skipped count) and continues; only Closed exits the loop.
Add two info-level log lines for in-field observability of the loop's
liveness. "nostr notify loop entered" fires once at task start; "nostr
notify loop received first event" fires once after the first
successful recv() with elapsed_ms since loop entry. Together these
turn the previous silent-failure shape (zero advert: peer cached
log lines indistinguishable between dead loop and idle channel) into
an immediately greppable startup signal — operators can confirm the
loop is alive and see how long it took to receive its first event,
catching any future regression in the subscription codepath in
seconds rather than waiting one advert_refresh_secs interval.
No public API change; the test fixture (NostrDiscovery::new_for_test)
does not call spawn_notify_loop and is unaffected.
Three Changed entries for the rx-path performance work
(Linux UDP recvmmsg batched receive, run_rx_loop drain batching, and
eager pubkey_full precompute on PeerIdentity construction) and five
Fixed entries: adopted NAT-traversed UDP transports inheriting the
primary listener's MTU and buffer config, TreeAnnounce ancestry on
self-root transitions, unconditional overlay-advert refetch before
each retry, stale overlay-advert eviction on NoTransportForType,
and scheduled retry on startup peer-init failure.
Pure CHANGELOG addition (+117 lines, no edits to existing entries).
Bullets are wrapped at 80 columns and attribute external
contributions to the originating PR and author.
The advert cache inside fetch_advert is read-only on hit — once a peer's
overlay advert is cached, every subsequent lookup returns the same
endpoints regardless of whether they still work. So when a peer rebinds
its NAT (or its STUN-discovered port flaps), connection retries to that
peer dial the same dead address forever, even with exponential backoff
firing at the right cadence.
Observed in deployment: macOS daemon's view of a Linux peer would
"regress" — peer marked rch=False after a brief link-dead window, then
hours of "Retry connection initiation failed: no operational transport
for any of <npub>'s addresses" with no recovery. Manual pause+resume of
the daemon (which restarts the FIPS endpoint and forces fresh advert
fetches) was the only way out.
When initiate_peer_connection / a retry tick returns
NodeError::NoTransportForType, fire-and-forget refetch_advert_for_stale_check
on the peer's npub. This re-fetches kind 37195 from advert_relays; if
the relay has a newer advert it replaces the cached entry, if it has
nothing it evicts the cached entry. Either way the next retry tick goes
to fresh data instead of looping on the same dead endpoint.
Mirrors the existing stale-advert sweep that runs from the
BootstrapEvent::Failed (NAT-traversal-streak) path, but covers the
direct-UDP-retry path which never crosses that streak threshold.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Martti MalmiandJohnathan Corgan
ArjenandJohnathan Corgan
