Files
fips/CHANGELOG.md

282 KiB
Raw Permalink Blame History

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Added

  • Dynamic interface binding for the Ethernet transport. An interface-bound transport is now a long-lived object that is sometimes bound: the interface it names need not exist when the daemon starts, may appear minutes later, and may vanish and return mid-operation. start_async returns Ok with the transport absent rather than failing, and a per-transport binder task binds when the interface appears, unbinds when it goes away, and rebinds when it returns. Start-time absence and runtime detach are one code path. Detection is event-driven where the kernel offers a source — netlink RTNLGRP_LINK on Linux, PF_ROUTE on macOS and FreeBSD — with a 1 s getifaddrs poll underneath as a backstop. Presence means IFF_UP — the interface exists and is administratively up — and deliberately not IFF_RUNNING. Binding needs no carrier and a socket outlives a carrier flap, so a bridge with nothing plugged into it (br-lan on a wifi-only router) is bound and healthy rather than permanently Degraded, and starts carrying traffic the moment a port comes up. Whether an interface has carrier is reported separately as interface.carrier in show_transports, never acted on. This closes the OpenWrt boot race (procd starts fips before wifi has created fips-mesh0 / fips-ap0; both transports were skipped for the life of the process while the 802.11s peer link formed anyway, so the node looked healthy and reached nothing), the intermittent-adapter case, and the mid-operation wifi reload that destroyed and recreated an interface under a live socket.

  • transports.ethernet.*.optional (bool, default false). Naming an interface in configuration is a statement that you expect it, so the default is to complain: while a required interface is missing the node reports Degraded and logs the edge, at a severity that follows how long the absence lasts (see below). optional: true makes absence silent (info on the edge, no health impact) for hardware that is legitimately not always there. It describes the interface's presence, not the transport's importance — an optional interface that is present is used exactly as hard as any other — and no value of it makes a missing interface fatal at startup.

  • fipsctl show transports reports interface presence per transport under a new interface block: name, presence (absent / binding / present), carrier, policy (required / optional), since_secs, binds and failed_attempts. The original boot-race bug was expensive precisely because nothing an operator could see said the node was deaf.

  • fipstop's transports view carries the same interface presence. The State column shows an interface-bound transport's presence rather than its lifecycle state — up is true from the moment the transport starts and stays true while its interface is missing, which is precisely the wrong answer in the one case someone is scanning that column for — and a new Policy column reads required or optional beside it, with an absent required interface red and an absent optional one yellow: the same split the daemon makes between staying Full and reporting Degraded. The instance name and the thing a transport is bound to are now separate columns, so netdev names line up down the list instead of trailing ragged inside a packed label. The detail pane gains an Interface block: netdev, presence and how long it has been held, carrier, what the absence policy means rather than which key sets it, bind count (flagged once it has rebound) and failed binds when there are any. The table fits an 80-column terminal — the OpenWrt serial console and the xterm and tmux default — dropping the byte counters below 100 columns and stacking the detail pane below 110, rather than shrinking every column until none of them can be read.

  • testing/iface-binding/ integration suite (ci-local.sh --only iface-binding, and a GitHub matrix leg): two daemons whose only transports are interface-bound, run against a veth pair the harness creates, downs, deletes and recreates underneath them. Asserts the boot race, the late attach and peering over it, the flap in both directions, destroy-and-recreate, that an optional interface never moves node health, and that absence is logged once on the edge rather than once per retry.

Gateway

  • The gateway counts sessions on a kernel without /proc/net/nf_conntrack. When the file is absent it dumps the conntrack table over netlink, as conntrack -L does, so a mapping carrying traffic is pinned instead of being reclaimed on its TTL and grace period alone. Kernels built without CONFIG_NF_CONNTRACK_PROCFS, such as Ubuntu's, had session pinning off.
  • The gateway says at startup whether it can read conntrack sessions. It reads the table once, as each tick does, and logs either the source it read or that no source is readable and session pinning is off. An operator on a kernel with no readable source learned this only from a warning at the first failed tick.

Node lifecycle

  • Transport-medium change detection, controlled by the new node.netmon.* block (on by default). For each peer whose transport address is a numeric IP endpoint, the node asks the kernel which local address it would use to reach that peer — a connect(2) on a UDP socket, which resolves the route and sends nothing — and reports a change once some peer held across two consecutive samples is reached from a different local address, or has stopped being reachable at all. Asking the question per peer rather than about the host is what keeps it quiet: a container bridge, a VPN, a veth pair or a tunnel appearing is not the route to any peer and cannot move the fingerprint, while a peer on the same LAN — reached by its subnet route, not the default route — is covered, as is a more specific route moving under a single peer. Peers joining and leaving are ignored on their own, being ordinary node behaviour rather than a statement about the medium — except that a peer seen for the first time is checked against its own connect()-ed socket, and reported if that socket is pinned to a source the routing table would no longer choose, so a medium change in the window between a peer authenticating and the next sample is not adopted silently while that peer sits stranded on the old path. A peer addressed by MAC, by .onion or Nym recipient, by a scoped IPv6 literal, or by a hostname it has not yet been heard from on, has no route to ask about and contributes nothing; a node with no peers detects nothing, having nothing bound to the old path to repair. The peer table is read through the node's existing lock-free entity snapshot, so the detector stays a detached task holding no node state. A handover is not atomic (the route goes, briefly there is none, the new one arrives), so a short debounce coalesces the burst into one event and a fingerprint that settles back where it started reports nothing. Linux subscribes to NETLINK_ROUTE multicast (the groups ip monitor uses) and macOS and FreeBSD to a PF_ROUTE socket, both reacting to the kernel event in milliseconds; every other platform samples on a timer at node.netmon.poll_interval_secs, which also runs underneath the kernel sources as a backstop, since a netlink socket drops messages under memory pressure and the subscription can be refused in a restricted sandbox. A backend only decides when to look — the fingerprint comparison, the debounce and the settled-back suppression are shared — so the remaining backends (NotifyIpInterfaceChange on Windows, an embedder push on iOS) land behind the same seam without touching the reaction. Android takes the netlink source, and falls back to the timer where policy refuses the group bind. What the node does with the signal is the connected-socket rebind described under Fixed above. Bluetooth is not covered: an adapter's state is not an IP attachment and is invisible to this detector. Upgrade note: this couples a new key to one that has already shipped. A handover is ridden out for up to eight settling rounds of node.netmon.debounce_ms before a change is reported, and if that worst case reaches node.link_dead_timeout_secs the reaper tears the peering down before the change is ever acted on, so the node refuses to start rather than run in that shape. At the shipped defaults the margin is wide (8 × 250ms = 2s against 30s), but detection is on by default, so a node that shortened node.link_dead_timeout_secs to 1 or 2 seconds for fast failover will be refused at startup after the upgrade, naming a node.netmon.* key its operator never set. Raise the timeout, lower debounce_ms so eight rounds stay under it, or set node.netmon.enabled: false. A node.link_dead_timeout_secs of 0 is exempt from the check.

Library surface and internals

  • Node::netmon_trigger() and NetmonTrigger::poke(), an embedder push into the medium-change detector (#146). The detector's channel was private end to end, so an embedder whose platform announces a network change had nowhere to put it. That is Android's position: the netlink group bind is refused to an app there, which leaves detection on the poll timer, while the app's ConnectivityManager callback knows the moment the default network moved. poke() is a bare wake-up with no payload — the detector still samples, and the fingerprint comparison, the debounce, the settled-back suppression and the minimum spacing between reports apply to a pushed wake-up exactly as they do to a kernel event, so a spurious callback costs two samples at the default debounce rather than a socket rebind. It is synchronous and callable from any thread, before or after start(); a poke that lands while the detector is sampling is held rather than lost, and a burst in that window coalesces into the one held wake-up (a burst that finds the detector waiting costs two: the one that woke it and the one held). A platform callback is not ordered against the routing table it reports on, so a pushed wake-up whose sample shows nothing moved looks once more after node.netmon.debounce_ms instead of leaving the change to the timer; wake-ups from the kernel and the timer are unchanged. The push is selected beside the platform's own source and the timer backstop, never instead of them. The platform wiring (registerNetworkCallback, or NWPathMonitor on iOS) stays with the embedder, since the crate has no JNI layer.

  • TransportError::InterfaceUnavailable { interface }. A missing interface and a typo'd interface name were previously the same flat StartFailed(String); nothing downstream could branch on absence.

Packaging

  • A pfSense package (packaging/pfsense/, gmake pfsense). pfSense is FreeBSD underneath, but the FreeBSD package fails there in three silent ways: pfSense runs only /usr/local/etc/rc.d/*.sh at boot and re-runs them on WAN IP changes, so the suffixless rc script never starts; unbound.conf is generated from config.xml with no conf.d, so the DNS drop-in is never read; and — on a firewall where the default-on "Allow IPv6" has been turned off — the responder's ::1 default is unreachable when unbound is then generated with do-ip6: no, so it binds 127.0.0.1 for robustness. The package ships fips.sh, wires the fips. zone into the DNS Resolver through config.xml, and binds the responder on 127.0.0.1. It links statically by default, since pfSense runs a FreeBSD base that cannot be obtained to link against; aarch64 is refused, where static binaries fault at posix_spawn. Mechanics shared with the FreeBSD builder live in packaging/common/pkg-lib.sh, which both source; the FreeBSD package is byte-identical before and after. The pfSense package is built and checked in its own CI job, then installed in the same VM and run through the boot script's start, re-entrant start, a forced newsyslog rotation, restart, stop and pkg delete (testing/pfsense-install-smoke.sh, also the first thing to run on a real box), and attached to each release next to the FreeBSD package, after being run on pfSense Plus 26.03.1 and 26.07 with mesh traffic between them. CI produces both Intel packages from that one FreeBSD 15.1 build: the CE 2.8.1 (FreeBSD:15:amd64) package, and the CE 2.9 / Plus 26.x (FreeBSD:16:amd64) package as the same static binaries relabelled, which is the direction FreeBSD's binary compatibility supports and has been run on pfSense Plus 26.03.1 and 26.07. ARM stays build-it-yourself because rustup ships no toolchain for it. See packaging/pfsense/README.md.

Changed

  • Degraded is now a level rather than a latch. The supervisor's reason set was monotonic, which was correct while no child could recover; with recovery it would have meant "something broke at some point since boot" rather than "something is broken now". Interface absence is tracked in its own reversible set and node health is recomputed on every transition in both directions, so plugging the WAN back in clears Degraded without a restart. A transport whose interface is absent still counts as up, so a single-interface node that boots before its wifi degrades rather than exiting on "no transports".

  • The OpenWrt package ships the mesh0/mesh1 and ap0/ap1 Ethernet transports enabled with optional: true, instead of commented out. fips-mesh-setup and fips-ap-setup no longer comment-toggle blocks in fips.yaml, and no longer tell the operator to restart the daemon after creating an interface — the daemon binds it on its own. phy0-sta0 (wwan) is marked optional: true for the same reason: it only exists while a radio is in station mode. Upgrade note: an existing /etc/fips/fips.yaml is preserved and does not gain the new key. It is a package conffile, so on a router where fips-mesh-setup or fips-ap-setup had already uncommented a block, that block stays as it was, with no optional key — and optional defaults to false. Such a block is therefore required, so an absent fips-mesh0 keeps the node Degraded and is reported once at error ten seconds in, where the same block in the shipped file is silent. Add optional: true to the block to match what the package now ships.

  • The Ethernet receive loop backs off and exits on a dead socket instead of spinning on Err with a warn! per iteration, and the ad-hoc ENXIO socket-reopen in the beacon sender is gone. Both hand recovery to the presence machine: one mechanism for every cause rather than one hack per symptom. Beacons pause while an interface is absent.

  • The absence edge is not itself an error, and there is exactly one deadline after it. An interface missing when the daemon starts logs at info — that is the boot race the mechanism exists to absorb, not a fault — and a runtime detach at warn, because a link coming and going is ordinary weather for a mesh daemon. Ten seconds is the whole grace: past it, absence is no longer a race against a radio or a container, so a required interface still missing is reported once at error. Start-time absence and a runtime detach share that one deadline rather than getting one each. An optional interface never reaches error. Node health does not wait for any of it, publishing Degraded on the first edge either way.

  • The TUN boundary's TCP MSS clamp now tracks the node's egress MTU at runtime instead of freezing it at startup. transport_mtu() is the minimum across bound transports, so a transport that binds minutes after start can be the narrow one — but the TUN reader and writer were handed a u16 computed once when they spawned, while every other consumer (show_status, the control-socket snapshot, the session-layer fragmentation check) read it live. A node could therefore report one effective IPv6 MTU and clamp to another. The ceiling is now shared with those threads and recomputed whenever the bound set changes, in both directions: a narrow interface appearing tightens it, and its departure releases it. MSS is negotiated per connection, so a change applies to connections opened after it; existing ones are not disturbed.

  • Rebinds that keep succeeding into a socket that dies moments later are damped: consecutive bindings shorter than ten seconds back off on the 1 s → 30 s curve, and past three of them the binder stops announcing each bind as a recovery until one lasts. Undamped, a persistently broken socket behind a healthy interface produced a log pair and a Degraded→Running health flap every second.

  • A bind failure that is not absence — no CAP_NET_RAW, no readable /dev/bpf*, a buffer the kernel refused — fails the daemon's start as it always has, rather than being waited out. It is a fault, not a state, and will not resolve on its own; only a missing interface is retried at start. A non-absence failure during a later rebind still backs off, since the node is serving by then.

  • The binder cannot outlive its transport, and a teardown that races a bind cannot leave a live receive loop on a socket nothing owns. A shared stop flag is raised before teardown and checked by the binder after it stores a binding, so whichever order the two interleave exactly one of them cleans up; EthernetTransport gained a Drop that raises the flag, aborts the binder and releases the socket, for handles dropped without stop_async.

  • Presence edges are published with try_send and retried on the next tick rather than awaited. A bounded channel could previously park the binder mid-publish — a health channel able to deadlock the machine whose health it carries — freezing the interface in whatever state it held.

  • TransportHandle::is_bound() joins is_operational(): the latter means the transport was started, which for an interface-bound transport no longer implies a live socket. Node::transport_mtu now filters on the former, because an interface that has never existed was clamping the whole node's IPv6 MTU to a number derived from absent hardware.

  • An interface deleted and recreated under the same name is detected as a detach. Both backends bind by device rather than by name, so the old socket is attached to nothing while the name still resolves — and a stale AF_PACKET socket never becomes readable, so nothing errors and nothing exits. Detection previously rested entirely on the beacon sender failing, which a node with announce: false does not have. The bound interface index is now captured at bind and compared on every poll.

  • The link-event watcher distinguishes a genuine receive error from WouldBlock. try_io clears readiness only on the latter, so a persistent error — ENOBUFS after a burst of link events overflows the socket buffer — span a core flat with nothing logged. Errors are now counted, logged once, backed off, and after five the source is abandoned for the presence poll.

  • Presence probes are coalesced to at most ten a second. Linux netlink is filtered to RTNLGRP_LINK, but PF_ROUTE has no group filter, so the macOS source delivers every routing message on the host — route churn, ARP, DHCP renewals, a VPN going up and down — and each would otherwise drive a full getifaddrs walk.

  • CI runs the library tests on musl (Alpine) as well as glibc. Presence is built on getifaddrs and ifa_flags, musl reimplements both independently, and the interfaces this feature exists for (fips-mesh0, fips-ap0 on OpenWrt) are unbridged with no IP address at all — the case where implementations most plausibly differ. It was previously asserted on a libc no test had ever run it against, on the target it was written for.

  • Interface presence state ignores lock poisoning. Treating a poisoned lock as a failure meant reading "no socket, tasks dead", which is the destructive direction: a transport reporting itself present while every send fails, or a binder tearing down and rebinding every second while teardown silently declined to abort anything.

Packaging (Debian)

  • An upgrade of the .deb now reapplies the firewall ruleset in place. Until now an upgrade reloaded nothing, so a changed /etc/fips/fips.nft took effect only at the next reboot or manual restart, and a restart deletes the fips table and leaves the mesh interface unfiltered until the ruleset is loaded again. fips-firewall.service, in both the Debian and the plain systemd unit, gains a reload that replaces the ruleset in one transaction, and the postinst reloads the unit only when it is already active, so an upgrade never turns the firewall on for a host that has not opted in. A reload that fails leaves the previous ruleset in place and is reported; the upgrade goes on.

Packaging (AUR)

  • The AUR publish on a release tag now waits until every package workflow of that tag has succeeded. It used to push the new pkgver while the Linux, macOS, Windows, OpenWrt and FreeBSD packages were still building; at v0.5.1 the AUR was updated while the release had 15 of its 17 assets. Because the AUR package pins the tag's source archive, withdrawing a bad release after that point left the AUR package unbuildable. A failed or cancelled package run now stops the publish, and one that has not finished within an hour fails it.

Dependencies

  • The lockfile moves chacha20 from 0.10.1 to 0.10.2, because 0.10.1 is yanked. It arrives through rand, a direct dependency, so it sits on the built path rather than off to one side. The requirement in Cargo.toml already admitted 0.10.2, so this is a lockfile change and no code changed with it. This is not a security fix: cargo audit reports nothing against chacha20 at either version, and 0.10.1 was withdrawn by its maintainer rather than flagged by an advisory. What it buys is that a fresh checkout can resolve the lockfile without reaching for a yanked version.

Removed

  • Source-breaking for consumers of the library crate: ActivePeer no longer stores a connectivity state. ActivePeer::connectivity, can_send, is_healthy, mark_stale, mark_reconnecting, mark_disconnected and mark_connected are gone. ConnectivityState stays public with only Connected and Stale, the values show_peers reports, and loses can_send and is_healthy. ConnectivityState::is_terminal, ActivePeer::is_disconnected, Node::sendable_peers and Node::sendable_peer_count keep their signatures and the results they always had in the daemon: the first two return false, and the last two cover every peer. Nothing in the daemon changed the stored state after a peer was promoted, so every removed check was already true and the shipped binaries behave as before. The peer wire and the control-socket response shape are unchanged.

Fixed

Node lifecycle

  • Losing an interface no longer leaves its peers in the routing table. The peers stayed in the registry, the routes through them stayed selectable, and the node kept advertising reachability it no longer had — so transit traffic was dropped in silence and other nodes kept routing toward this one for those destinations, until the liveness reaper noticed up to node.link_dead_timeout_secs later. Measured on real hardware, a detached dongle took the node's parent with it and no new parent was chosen for twenty-seven seconds, with four alternative peers available the whole time. A transport's detach edge now withdraws every peer whose active link runs over it, on the same path the liveness reaper uses, so sessions, path MTU, session indices, the link, the control machine, tree cleanup and re-announce, and bloom withdrawal unwind exactly as they already did. It is not filtered by optional: whether an interface's absence is normal is a statement about node health, and says nothing about whether the routes over it still work. The trade is that an absence shorter than the dead timeout that then recovers now costs a re-peer where it previously cost nothing, accepted because black-holing is silent, poisons other nodes' routing and takes the full timeout to clear, where a re-peer is bounded, visible and self-healing.

  • A local interface flap during a handshake is no longer charged to the remote. A msg2 send refused because the interface is absent or mid-rebind was treated as a failed handshake: the link was removed, the reverse-address entry dropped, the session index freed, the control machine torn down, and the whole thing recorded under the reject reason that means "the remote sent something invalid", which is what an operator reading the rejects would have concluded. The initiator meanwhile resent msg1 into a link that no longer existed and had to rebuild from nothing. A transport error the daemon is already working to clear now leaves the half-built link exactly where it is for that resend to land on; only a terminal error still tears down, and a link nobody resends to is reaped at node.rate_limit.handshake_timeout_secs like every other abandoned handshake. The rekey msg1 send site keeps its teardown, which was already benign, and stops reporting a local self-clearing condition at warn.

  • A peer reachable over two interfaces is no longer re-dialled on the path it is not using. Beacon discovery skipped only a candidate naming the peer's current path, which is the one case that could not churn anything, so the alternate path was dialled every discovery tick; each dial that completed promoted and displaced a healthy incumbent, tore down the session, and, when that peer was the parent, switched parents and re-announced mesh-wide. Measured on real hardware, seventeen dials to one peer in fifteen minutes, alternating wifi and cable, displacing a link reporting etx 1.0 and loss 0.0. Discovery now asks whether the link it already holds is answering rather than which path the candidate names. Failover is unchanged: a peer that goes quiet for longer than node.heartbeat_interval_secs is dialled again on every path, alternate included. One behaviour goes with it. A peer held on an adopted NAT-traversal transport that is also reachable by Ethernet or BLE beacon used to drift onto the local path on the next discovery tick, and now stays on the traversed path for as long as that path answers. Migrating it is still done by the configured-peer refresh (a config reload, a runtime peer update, or fipsctl connect), and a traversed link that goes quiet still releases the peer to every path.

  • A peer that rotates its address no longer keeps sending from a socket aimed where it used to be. The authenticated-frame path updated the peer's address and discarded the flag saying it had changed, so the per-peer connect()-ed UDP socket stayed pinned to the old 5-tuple; the sibling path already cleared it.

Data plane and transports

  • A peer that stops reading can no longer stall the node. TCP, Tor, Nym and BLE wrote to their links directly from the caller's task, and a write blocks once the peer's receive window and this node's send buffer are both full — which is what a peer that has gone away, or whose path has just changed medium, produces. The callers are the rx loop's tick handlers, so a single unresponsive peer held every other arm of the event loop behind it: other peers' liveness, forwarding, and control RPCs. Each connection now has its own writer task owning the write half, and sending is an enqueue onto a bounded queue, so the only code that can await the wire is a task with nothing else to do. A peer that stops draining fills its queue and its sends then fail immediately, which is the signal the caller's retry and liveness handling already expects. Frames are written whole, and a write error takes the connection down with it, so a peer never sees a partial frame it cannot resynchronise from. BLE was the worst of the four: it awaited the L2CAP write while holding the connection-pool mutex, so one unresponsive peer froze every other BLE operation as well — connects, evictions, and each receive loop's teardown. On TCP, Tor and Nym, a connection's writer and receive loop act only on their own connection: one that outlives its connection can no longer tear down a newer connection that has taken the same address, and a receive loop that ends stops its writer rather than leaving it writing to a peer that has gone. Closing one of their connections on purpose, as a control-API disconnect does, now lets the writer finish the frames already queued, within five seconds, instead of discarding them, so a Disconnect sent just before the close reaches the peer. Stopping the transport, or a connection that has failed, still discards them.

  • A per-peer connect()-ed UDP socket is no longer left pinned to an interface the host has moved off. Established UDP peers get their own socket for the send fast path; open_connected_fd binds the wildcard and then calls connect(2), which makes the kernel resolve the route once and auto-bind the local source address to whichever interface was carrying it at that moment. It never re-evaluates. So after the host changed transport medium — a laptop moving between WLAN and LAN, a phone between Wi-Fi and cellular — every established peer went on transmitting from an address the routing table had abandoned, while the peer, which re-pins to whatever address it last heard from, answered somewhere the node was no longer sending from. The peering stayed marked connected and carried no traffic until the 30s liveness timeout tore it down, roughly 60-90s of black-holed traffic per medium change, followed by a full re-handshake and tree re-convergence. The mirror-image case — the peer rotating its address — was already handled at the point the rotation is observed; this is the local half, which had no signal to hang off because a local move is invisible in the data plane. It now fires from the medium-change detection added below, which is exactly that missing signal. Dropping the sockets is self-healing rather than disruptive: the wildcard listen socket resolves a route per packet, so sends keep working immediately, and a correctly-bound connected socket is reinstalled on a later tick. The reaction is scoped to the peers the change names: only their sockets are dropped, and each of those on a connectionless transport is heartbeated at once, so the far side re-pins to the new source address rather than waiting out its own heartbeat interval. A peer on a connection-oriented transport keeps the periodic heartbeat instead. That was because such a send awaited an unbounded write_all on a stream the medium change had very likely just stranded, and this reaction runs on the rx loop; the writer-task change below removes that hazard, so widening the fan-out to those transports is now open work rather than something the design forbids. Measured on a live node, a WLAN/LAN switch in either direction now costs no reconnection at all — the Noise session, tree position and routes survive it. Linux and macOS (the platforms with the connected-socket fast path); elsewhere the heartbeat alone carries the new address.

  • Two inbound TCP connections that share a peer address but arrive on different local addresses no longer share one pool entry. The kernel names a connection by its four-tuple, so a listener on a wildcard address, which is what the shipped configuration binds, can accept two connections whose peer ip:port is the same on two different local addresses. The pool was keyed by the peer address alone: the second connection's entry replaced the first's while the inbound-connection counter counted both, the first connection's teardown then removed the second's entry, and the second's own teardown found nothing to remove, so the counter ended one above the connections it counts. That counter gates the inbound connection limit, so a host repeating the collision could hold it at the limit and lock out further inbound TCP connections until the daemon restarted. Inbound entries now carry the accepted socket's local address in their pool key as well as the remote one.

  • A peer that moves to a new address now loses the per-peer connect(2)-ed UDP socket pinned to the address it left. set_current_addr returns whether the address actually changed so the caller can drop the stale socket, and the decrypt-worker completion path already acted on that return; the in-line decrypt path discarded it, so the socket stayed installed and the send path kept preferring it over the wildcard listen socket.

  • A peer reached by NAT traversal now gets its per-peer connected UDP socket. The adopted traversal socket carried no address-reuse flags, so the connected socket's bind to the same port was refused with EADDRINUSE on every tick and the peer never left the unconnected path. The flags are now set when the socket is adopted, after its bind, so the traversal bind still receives a port no other socket holds.

Peering

  • A heartbeat whose send failed no longer counts as one that was delivered. The send was recorded before it was attempted, so a peer whose heartbeat could not go out was treated as heartbeated and was not tried again for a whole heartbeat_interval_secs, although it had heard nothing and its own link-dead timer was running. The attempt and the delivery are now recorded separately: the interval that paces a healthy peer advances only on a send that returned cleanly, and a peer whose send failed is retried after a shorter fixed interval instead. That retry interval gates only a peer whose last attempt failed, so it cannot clamp a heartbeat_interval_secs configured below it.

Session setup

  • A session whose last handshake message is lost no longer stays one-sided. The initiator sent msg3 once and treated the session as established at once; when that one datagram was lost, the responder kept waiting for it and dropped every frame the initiator sent, and nothing sent msg3 again, because the responder's repeated SessionAck was refused as arriving in the wrong state. The session stayed that way until the next session rekey, or with periodic rekey switched off, indefinitely. The initiator now keeps its msg3 and resends it on the handshake resend interval, with backoff, until a frame from the responder authenticates or handshake_max_resends resends have gone out. The wire format is unchanged: the resend carries the same msg3, and a responder that already completed the session refuses the duplicate as before.
  • A session rekey this node started no longer stays in flight forever when its setup or the peer's ack is lost. Nothing resends a rekey setup, and the only expiry covered a rekey the peer started, so one lost datagram left the rekey pending and blocked every later one: the session kept its current keys and stopped rotating them. The rekey now expires on the handshake timeout, timed from when this node sent its setup, and the next tick starts a fresh one. A forged ack cannot extend it. Expiries are counted as rekey_unanswered. The wire format is unchanged.
  • A forged rekey msg2 no longer takes the link down. The rekey initiator gave up its handshake before reading msg2 and abandoned the cycle when the read failed, although nothing authenticates a msg2 ahead of that read. Anyone on the path who saw the rekey msg1 go out could answer first with a msg2 of the right size under the index msg1 carries in cleartext. The responder has already committed its new session by then and cuts over on its next tick, so the two ends were left on different keys: frames from the responder were dropped at once, frames to it failed once its drain window closed, and each end removed the other on the link-dead timeout about 30 s later. A msg2 that fails the read now leaves the handshake as it was before the read, along with the msg1 resend schedule and the msg2 dispatch entry, so the responder's genuine msg2 still completes the rekey. In exchange, every such forgery now costs the initiator the msg2 key agreement until the cycle ends, where before only the first one did; the msg1 resend budget bounds that. The wire format is unchanged.
  • A SessionAck that fails to read no longer ends a session rekey this node started. The handler took the rekey handshake off the session before reading the ack's msg2 and abandoned the rekey when the read failed, although nothing authenticates the ack before that read and the only tie to the rekey is the datagram's source address. The handshake now goes back rolled back to its state before the read, so the peer's genuine ack still completes the rekey, and the refusal is counted as ack_handshake_failed, as it already was for a first-contact session. The wire format is unchanged.
  • A link rekey whose reply is lost no longer splits the link. The node that answered a rekey used to switch to the new keys on its own next tick, before the other side had them; when the reply was lost, frames from the answering side were dropped until the link was torn down. The answering side now switches only when a frame on the new keys arrives from the side that started the rekey, and drops keys that were never adopted after a hold (120 s by default) so the next rekey can proceed.

Session coordinates

  • A node with no coordinates cached for a session's destination no longer sends its own coordinates in their place. The lookup that supplies them falls back to the node's own coordinates, which a first-contact SessionSetup needs because its destination field cannot be empty, but the established data path, the standalone CoordsWarmup and the rekey SessionSetup used the same fallback. Every receiver files the destination coordinates it is sent under the destination's address, so a destination reached this way cached its own address under the sender's coordinates. On a cache miss a data frame now goes out without coordinates and leaves the warmup budget for the first frames after the cache is refilled, a standalone CoordsWarmup is not sent, and a rekey SessionSetup, which can only miss for a direct peer, carries the coordinates that peer announced. First-contact setup is unchanged. The wire format is unchanged.

Control socket

  • show_links (fipsctl show links) now reports the traffic a link has carried. Its packets_sent, packets_recv, bytes_sent, bytes_recv and last_recv_ms were read from counters on the link record that nothing on the data plane ever wrote, so every link reported zero however much traffic it carried, while show_peers counted the same traffic on the peer. A link bound to an authenticated peer now reports that peer's counters, so the two queries agree for the same link_id; a link still in handshake has no peer yet and still reports zero. The counters follow the peer across address changes, while the row's transport_id and remote_addr stay those the link was created with. The counters cover authenticated link frames only, so they are not expected to match the transport totals in show_transports. The response shape is unchanged. Fixes #158.
  • show_peers (fipsctl show peers) now reports a peer that has gone quiet as stale. Its connectivity was read from a state that nothing outside the tests ever changed, so every peer read connected until it was removed, including one that had stopped answering tens of seconds earlier. The value is now derived from how long the peer has been silent: connected while its idle time is at or below heartbeat_interval_secs, and stale above it, the same rule that decides whether discovery re-dials an active peer on the path it already has. The reconnecting and disconnected values the open-discovery tutorial described never occurred, and the tutorial no longer lists them. The response shape is unchanged.

Identity and config

  • A persistent node whose identity key path cannot be examined now refuses to start instead of coming up under a new identity. Path::exists reports false both for a key that is absent and for one whose metadata cannot be read, so a key symlinked onto a volume that did not mount, or one in a directory the daemon cannot search, read as a first boot: the node generated a fresh identity, failed to store it, and carried on under an npub that every peer whose allowlist names the old one refuses. Only a NotFound result is now treated as an absence; any other failure to stat the path aborts the start and names the path. A dangling symlink likewise aborts rather than being replaced. The legacy /etc/fips/fips.key lookup follows the same rule.

Gateway

  • Conntrack sessions are matched by address rather than by text, so live traffic pins a gateway mapping again. The session count searched each /proc/net/nf_conntrack line for dst= followed by the virtual IP in its compressed form (fd01::1), while the kernel prints tuples in the full uncompressed form (dst=fd01:0000:0000:0000:0000:0000:0000:0001), so the count was zero for every mapping on every kernel. Nothing pinned an in-use mapping, and one whose client did not re-query DNS was reclaimed about two minutes after its last DNS reference while its traffic was still flowing. Each dst= value is now parsed as an address and compared as one.
  • The conntrack table is read once per tick instead of once per mapping, and the read happens off the runtime thread. The whole file was read and scanned for each mapping in turn, while the pool lock was held, on the same single-threaded runtime that serves DNS. The tick now takes one snapshot with a blocking task before it takes the lock, and the pool does a map lookup per mapping.
  • A conntrack source that cannot be read is reported. It still counts as zero sessions for every mapping, as it always has, so reclamation keeps working rather than pinning the whole pool; but the first failure and each change of outcome after it are now logged, so an unreadable source is no longer indistinguishable from an idle one. A source that fails identically every tick is logged at debug rather than warn on a repeat.
  • The NAT table is rebuilt in one netlink transaction. A rebuild deleted the fips_gateway table in a batch of its own, discarded that batch's result, and only then sent the batch that recreated the table, the chains, the fips0 masquerade and every per-mapping rule. Between the two sends the gateway had no NAT at all, and a recreate the kernel refused left the table absent for good, taking down forwarding for every existing mapping rather than failing the one change that was being made. The delete and the recreate now share a single batch, which the kernel applies as one transaction, so a refused rebuild leaves the previous table in the packet path. The rules sent are unchanged.
  • The gateway's NAT rebuild no longer fails once the table holds more than about 105 mappings. Each rebuild is one netlink batch. From about 105 mappings the default socket buffers could not hold its acknowledgements, so rebuilds were logged as failed although they had taken effect. Past about 313 mappings the buffers could not hold the batch itself, and new .fips names past that count got a virtual IP with no translation. In releases with the gateway through 0.5.1, a rebuild past about 313 mappings also deleted the whole fips_gateway table, which stopped every mapping, the fips0 masquerade and the port forwards. The rebuild now sizes its send buffer to the batch and requests one acknowledgement per batch, and NAT errors now name the kernel errno. A rebuild that still fails is logged, and the next successful rebuild installs the mapping.
  • A .fips query the gateway answers without an address no longer takes an address from the pool. Every query type was allocated a mapping before the code looked at what the client had asked for, and an A or HTTPS query was then answered with NODATA, so any host that can reach the LAN resolver could consume the pool one name at a time with a query type it is never given an address for. Only AAAA and ANY allocate now. A non-AAAA query for a name that already has a mapping still refreshes that mapping's TTL clock, so a client querying both types does not lose half of its refresh.
  • The gateway's virtual-IP pool now limits how many mappings it holds and how fast it creates them. Any host that can reach the LAN resolver could ask for one new .fips name after another, and each got a mapping until the 65,535 addresses ran out, while every mapping made each NAT rebuild, each pool tick and shutdown slower. The pool now refuses a new name once it holds 1000 live mappings, and admits new names at 10 per second after a burst of 50. A refused query gets SERVFAIL, and the gateway's "Pool allocation failed" warning says which limit refused it. A name that already has a mapping is answered before either limit is consulted, so names in use keep resolving when the pool is full. The limits are compiled in, not configured.

Nostr and NAT traversal

  • A node no longer publishes NIP-09 deletion requests signed with its routing key after a NAT traversal attempt. Each request put the node's public identity next to the ids of its offer and answer gift wraps on every relay it reached, which the one-time signing keys on those wraps exist to prevent, and most of the requests deleted nothing, since a relay deletes a gift wrap only at its recipient's request. A relay that stores the wraps now keeps them until their NIP-40 expiration; relays that do not store ephemeral events never held them. The advertisement retraction still sends its deletion request, since that names an event the routing key signed itself. The discovery and traversal design documents describe the new behaviour.

Packaging (OpenWrt)

  • A new OpenWrt install no longer enables and starts fips-gateway. The generated postinst turned it on unconditionally, contradicting the init script's own header, the package README and the deployment tutorial, all of which say the service ships disabled and is enabled deliberately. The documented service fips-gateway enable / service fips-gateway start steps are unchanged, and the shipped fips.yaml still carries gateway.enabled: true, so enabling the service is all that is needed.
  • The first upgrade to this release re-enables and starts fips-gateway on any router that has the package installed, including one where the gateway was disabled by hand. Every released package's prerm disabled the service on its way out, leaving nothing behind that says whether the operator wanted it on, so an upgrade cannot tell the two apart and keeps the gateway running rather than silently turning off a working one. If you had disabled it, run service fips-gateway disable once after upgrading. Later upgrades preserve whatever state the service is in: the new prerm stops the services on an upgrade but no longer disables them.
  • start_service in the fips-gateway init script now reads gateway.enabled from /etc/fips/fips.yaml before doing anything. Starting a gateway that the config disables used to hand dnsmasq's .fips forwarding to the gateway's port, add the LAN prefix and advertise the pool route, and only then start a daemon that exits immediately because the gateway is disabled, leaving .fips resolution pointed at a port nothing listens on.
  • The .ipk and .apk packages now install the same maintainer scripts. The four script bodies live in packaging/openwrt-ipk/scripts/ instead of inside heredocs in the two build scripts, so the scenarios in testing/openwrt/ run what ships.

Packaging (Debian)

  • A .deb upgrade whose new daemon cannot start no longer hangs apt. The postinst started fips.service and then fips-dns.service with blocking calls, and because fips-dns.service requires the daemon, a daemon that failed on every start left the second call, apt and everything queued behind it waiting for ever with no message. Each start is now queued and waited on for at most 60 seconds. A unit that does not come up has its status printed and fails the configure step, so apt exits non-zero and names the unit; a masked unit, or one whose condition is not met, is reported and skipped.
  • The .deb maintainer scripts now manage fips-gateway with the rest of the package's services. An upgrade stopped the daemon, which the gateway requires, and never brought the gateway back, so an operator who had enabled it lost it until the next reboot; removing or purging the package left the gateway's enablement symlink behind, pointing at a unit file that no longer exists. The gateway is now stopped before the daemon on upgrade and restarted afterwards only when it is enabled and the daemon came up, and it is stopped and disabled on remove and purge. A gateway that does not come back is reported but does not fail the upgrade.
  • The .deb now declares libgcc-s1 (>= 4.2). All four binaries link libgcc_s.so.1, but cargo-deb removes every libgcc entry from the dependencies it derives, so the package never said so. libc6 depends on libgcc-s1 on Debian 12 and Ubuntu 22.04, 24.04 and 26.04, so installs there were not affected. A new check, testing/check-deb-depends.sh, runs dpkg-shlibdeps over the package's binaries on every build and fails the build when the declared Depends leaves out a library the binaries need, or states a floor lower or higher than the one they need. A dependency the packaging tool drops, including one it drops after only a warning when it cannot resolve a binary, now fails the build instead of shipping.
  • -V on binaries built into the Linux packages now includes the source revision, as <version> (rev <git-hash>). The build image had no git, so every container-built binary printed the version alone. A package built from a git worktree still has no revision, because the worktree's git directory is outside the tree the build sees. The build image's tag now includes a hash of its Dockerfile, so a host with an older image cached builds a new one instead of reusing it.
  • packaging/debian/build-deb-container.sh now returns the package it just built. It picked the most recently modified fips_*.deb in the output directory that sorted last by name, so a package with a higher version left there by an earlier run was returned instead.

Packaging (AUR)

  • The release PKGBUILD now lists dbus as a runtime dependency. The fips binary links libdbus-1, and the fips-git package already declared it.

[0.5.1] - 2026-09-06

Fixed

Discovery

  • A node no longer relays away the answer to its own lookup. A request is flooded to every tree peer whose bloom filter claims the target, so a false positive can send a copy out into the wider network and circulate it back to the node that originated it. The only identity test on arrival was whether the request named this node as the target, which a lookup this node originated never satisfies, so the copy was filed in the request dedup cache as ordinary transit under this node's own request_id. When the target answered, the reply was reverse-path forwarded to the peer that looped the request, the pending lookup was never satisfied, and discovery reported that its requests went unanswered while the answers were in fact arriving. An inbound response is now matched against this node's outstanding lookups before the transit dedup record, and a returning copy of this node's own request is dropped as the duplicate it is rather than recorded, so that id never enters the transit cache at all. This was a race rather than a hard failure: a reply that beat the looped copy found a clean cache and succeeded, and the failure grew likelier as the bloom fill ratio rose. Contributed by Arjen.

  • A lookup request of this node's own, returning to it, is no longer counted as a duplicate from the peer that delivered it. The fix above drops that copy, and it recorded the drop under the existing req_duplicate rejection, whose documented meaning is that a peer resent a request. A returning copy has a nonzero floor in healthy operation and rises with the bloom fill ratio, so folding the two together put a permanent number on a counter an operator reads as neighbour misbehaviour, and made the two events indistinguishable. It now has its own rejection reason and counter, req_own_loopback, shown in fipstop as "Own Loopback". req_duplicate returns to meaning only what it says.

Packaging

  • The Linux .deb and the systemd tarball now install and run on Debian 12 and Ubuntu 22.04. Every Linux artifact from v0.3.0 through v0.5.0 was built on the newest available runner, whose C library made the standard library's pidfd references a hard GLIBC_2.39 version requirement instead of the weak, runtime-checked ones it is meant to compile to. The loader refuses an image on that entry alone, so fips, fipstop and fips-gateway could not start; fipsctl was unaffected, which is why an install that was checked by running it looked healthy while the daemon was dead. No source code caused this and none was changed. The Linux artifacts are now built in a container pinned to the oldest supported distribution, declared with the floor in packaging/build-floor.env, and every producer on the release path runs testing/check-glibc-floor.sh on what it made, so a package or a tarball that would not load fails the build rather than reaching a user. The deprecated host-build targets in packaging/Makefile are not on that path and are not floor-checked. The declared dependency is derived from the binaries instead of hand-written, so it states the floor it was built against.

  • The .deb install suite no longer hangs when the daemon it installed cannot run. It started fips-dns.service with no timeout, and that unit is Type=oneshot with Requires=fips.service, so a daemon that cannot execute is restarted every five seconds for ever, the oneshot start job is never dispatched, and systemctl start never returns. The suite then produced no failure line, no results line and no exit status at all, which is the whole class of fault it exists to find: it stopped reporting at exactly the point it was most needed. Observed at 21 minutes against a package whose binaries could not load. The start is now queued rather than waited on, with a bounded wait for the unit to become active, so a dead daemon fails the suite instead of stalling the run that gates artifact publication.

[0.5.0] - 2026-08-30

Added

Platforms

  • FreeBSD support for the daemon, fipsctl, and fipstop, on x86_64 only: native TUN datapath (TUNSIFHEAD address-family framing, kernel-assigned tunN device name as with utun on macOS), clean service teardown, /usr/local/etc/fips config search path, and /var/run/fips control-socket default (both shared with macOS). The hosts, peers.allow / peers.deny and fipsctl keygen defaults follow the same /usr/local/etc/fips layout as macOS; that move and its startup warning are described by the three macOS path entries under [0.4.2] below, which this platform inherits. fips-gateway remains Linux-only. Native .pkg packaging under packaging/freebsd/ (make freebsd) with rc.d services, a fips control-socket group, service stop/restart across pkg upgrade, and .fips DNS integration for local_unbound/unbound/dnsmasq. mDNS LAN discovery works via mdns-sd 0.20 (socket-pktinfo 0.4.1, the first release that builds on FreeBSD). Daemon logs now disable ANSI color when stdout is not a terminal (all platforms). No aarch64 FreeBSD artifact is produced and that combination is not verified here.

  • Android-ready core, supported as an embedded crate rather than as a standalone daemon: there is no Android daemon artifact, since the library is the delivery form. The daemon's desktop transports and TUN operations are gated by target_os rather than by Cargo features, so a plain cargo build compiles for every target with no flags and Android self-excludes the raw Ethernet transport as Windows already did. Node::enable_app_owned_tun() gives an embedder that owns the TUN file descriptor (an Android VpnService, for instance) a channel pair for exchanging IPv6 packet bytes with FIPS instead of FIPS creating a system TUN device, and start() then performs no system-TUN or CAP_NET_ADMIN operations. Packets entering this way bypass handle_tun_packet, so the embedder must push only fd00::/8-destined packets and clamp TCP MSS on outbound SYNs. Desktop builds are unchanged and no Cargo features are introduced.

  • Node::dns_local_addr(), the DNS companion to the app-owned TUN interface above. An embedder whose resolver is pointed into the tunnel has no system socket aimed at the built-in .fips responder, so the accessor reports the address read back off the bound socket: dns.port = 0 therefore yields the kernel-assigned port, and it returns Some only while the responder is up. Read it once, after start() returns and before the node is moved into a background task; it is not a liveness feed (#136).

Native datagram API

  • A native datagram API addressed by public key, off by default, with a surface that may still change. A client process opens a flow to a peer's public key on a chosen port and sends and receives datagrams on a file descriptor the daemon hands it: no IPv6 emulation, no TUN device and no DNS, a datagram travelling from key to key. The wire needs no change and gets none. Every FSP data packet has carried a port pair inside its AEAD envelope since v0.2.0 and port 256 is simply the IPv6 shim, so what was missing was a way for a program to ask for a port of its own and be handed the traffic. The x-only public key is the address and an npub is that key written in bech32, so converting between them is a local encoding rather than a lookup or a name service; the 16-byte node address that travels on the wire is a truncated hash of the key, does not invert, and appears nowhere a client can see. A listener is a descriptor: the daemon writes one message per arrival to it, carrying the new flow's descriptor and the peer's address, so poll, select and epoll work on a listener and accepting is a recvmsg. There is no accept command and no reject command, and refusing a flow is closing the descriptor you were handed. The Rust surface mirrors std::net, with FipsStream::connect, FipsListener::bind, incoming, accept, io::Result and an errno mapping rather than a bespoke error type, plus set_nonblocking, AsFd and four deadline methods under the names and signatures std::net uses for the same jobs. One rule has no counterpart in Berkeley sockets and a client author must know it: the v1 wire carries no half-close, so nothing peer-driven ever closes a flow, and a server written to read until the flow ends waits for a signal that cannot arrive. The listener uses SOCK_SEQPACKET on Linux and SOCK_DGRAM on macOS and FreeBSD; both keep the message boundaries the API's contract with its clients rests on. macOS does not implement SOCK_SEQPACKET for AF_UNIX at all. FreeBSD accepts the constant and returns a socket that is not an atomic-record socket, so consecutive messages coalesce and a zero-length message is dropped rather than delivered; both were measured on the FreeBSD 15.1 image rather than reasoned about. The three kernels signal a closed peer differently and were measured too, so the receive path treats a Darwin or FreeBSD ECONNRESET as end of file alongside the POLLHUP and zero-byte read that Linux gives. EAGAIN is deliberately not in that company: it means the socket is empty and the peer alive, so it stays an error and the caller waits again. A zero-length payload has one known limitation. A closed peer latches POLLHUP while its messages are still queued, so that flag alone cannot say whether a zero-byte read is an empty datagram or the close; the receive path also asks FIONREAD, and bytes still queued prove a further message is waiting. That leaves one case unresolved: a zero-length datagram that is the last message before a close is reported as the close, because reading it drains the queue and a zero-length message contributes no bytes to FIONREAD. A client should not give a zero-length payload a meaning of its own, and should carry a one-byte discriminator instead.

OpenWrt mesh

  • OpenWrt 802.11s open-mesh backhaul: router-to-router radio links with FIPS providing all encryption, authentication and routing over bare L2 neighbor links. The mesh runs open with mesh_fwding 0, since SAE would duplicate the Noise layer and force ath10k raw mode, and FIPS's spanning tree is the routing layer. fips-mesh-setup is an opt-in UCI helper creating a per-radio mesh point (radio0 to fips-mesh0, radio1 to fips-mesh1, with a free-index fallback and a collision guard); radio setup stays opt-in because a package must not commandeer radios on install. A dual-band router gets one instance per radio, and FIPS treats the two paths as failover rather than multipath: cross-connection resolution keeps one active link per peer and the second band stands by, re-establishing after keepalive timeout. The shipped fips.yaml carries the mesh0/mesh1 Ethernet-transport entries commented out, so a stock install that never creates them logs no per-boot interface-missing warning; the helper uncomments the matching block when it creates the interface and re-comments it on remove (#123).

  • OpenWrt open !FIPS access SSID, stacked on the mesh backhaul above: every FIPS router broadcasts the same open SSID, forming one standard ESS that phones and laptops save once and roam between natively, with the Noise IK handshake as the only security layer. The leading ! sorts it to the top of alphabetically ordered network pickers, and the encryption type must be uniform across routers or clients treat the ESS as different saved networks. fips-ap-setup is an opt-in UCI helper creating the fips-ap0 open AP on an isolated network with a static ULA /64 and RA-only odhcpd addressing: stateless SLAAC with no DHCP, the minimum that satisfies Android's provisioning check, behind a locked-down fips_ap firewall zone with no path to br-lan or the WAN, reaching only ICMPv6, mDNS and the FIPS transports. There is no internet by design, so phones keep cellular as their default route (#126).

Node lifecycle

  • A bounded graceful-shutdown drain phase, controlled by the new node.drain_timeout_secs (default 2s). On the shutdown signal the node broadcasts Disconnect to all peers and then keeps serving for that window, exiting early once all peers are gone, so in-flight traffic settles and peers observe the disconnect before the transports close, where previously teardown was immediate. The published node state gains a Draining variant visible via control queries during the window. The immediate stop path used by non-daemon callers is unchanged.

Transports & config

  • The UDP transport's listen socket descriptor can now be handed to an embedder, for hosts that associate a socket with one interface or network and steer inbound traffic by that association rather than routing by destination address. On such a host a peer reachable only over a secondary network fails in a way FIPS can neither see nor fix: the address is well-formed, the send succeeds, the peer replies, and the host discards the reply before it reaches our socket, so the link retries msg1 forever with no error surfaced anywhere. The correction is a socket option chosen against host state FIPS has no basis to reason about, so the descriptor goes to whoever does. Call Node::enable_app_owned_udp_fd() after Node::new and before start(), and read AppOwnedUdpSocket { instance, fd } off the returned channel once the transport is up, following the existing enable_app_owned_tun contract. One message is sent per UDP transport that binds, so a multi-listener configuration yields all of them; nothing is sent when no UDP transport is configured or one fails to bind, so an embedder tells "no socket" from "here is the socket" by the receive timing out. The instance field is the name the listener was configured under, None for a single unnamed instance, and it is what makes more than one listener usable: transports are created by iterating a map, so arrival order is luck, and an embedder whose whole purpose is to bind one socket to one network would otherwise have to guess which socket it just received. Guessing wrong pins one lane's socket to the other lane's network, which is the failure the interface exists to correct. FIPS keeps owning the socket, and the descriptor carries no promise beyond "this is the transport's socket, and it is open now". Two limits: the per-peer connected-UDP sockets that Linux and macOS open after start() returns are not covered, and a transport that adopts a socket handed in by the traversal bootstrap does not deliver one. Unix only, since the Windows UDP backend has no descriptor.

  • A peer address may name which instance of a transport it belongs to, as transport: "udp/aware" rather than "udp", where the part after the slash is the key the transport was configured under. A node running several instances of one type could not be told them apart by a dialer: both bind wildcard sockets, so the address-family test matches either, and selection fell through to the lowest transport id. One socket carried every dial and the other never carried traffic. A bare type is unqualified and matches any instance, which is what every existing configuration and caller produces, so nothing changes for a node that does not use the syntax. A qualified name is never substituted with a different instance: that is the wrong-lane dial the syntax exists to prevent, so an unmatched name fails the address instead, and the same name is what an embedder binds by and what the dialer routes on. The slash is already how FIPS qualifies an instance inside an address (eth0/aa:bb:...) and cannot occur in a type name. Only UDP resolves an instance name today; an address that qualifies any other transport type is refused rather than matched loosely. Because a qualified name never falls back, the configuration validator rejects one that no configured transport answers to, naming the peer, the instance asked for and the instances that exist. Otherwise the address would simply be skipped at every dial, which is invisible for a peer that has a second address that works: the lane would never carry traffic and nothing above debug logging would say so.

Bluetooth LE

  • The BLE transport is refactored so the code common to Linux and Android is implemented once, with a separate backend for each platform, and gains a reliability pass closing several defects a two-node field capture surfaced. The module gate is now ble_available, meaning glibc Linux or Android rather than target_os = "linux", so the transport is no longer conflated with one of its backends; musl is excluded and a platform with no concrete backend fails the build rather than compiling a transport that starts, reports itself up and never peers. The receive path recovers packet boundaries from the 4-byte FMP common prefix instead of assuming one read returns one whole packet, which held only for BlueZ's SOCK_SEQPACKET and not for a stream-oriented backend such as Android's BluetoothSocket, where fragments shipped up as runts that FMP and Noise rejected and coalesced tails were silently truncated. A peer is recognised by node identity rather than by its link address, so a phone rotating resolvable private addresses no longer presents as a new device on every rotation and defeats the already-connected guards. The L2CAP PSM is decided by the backend: listen reports the PSM it actually bound, the advertisement carries it beside the 128-bit service UUID, and a dialer learns it from the scan, which is what Android and macOS require since both assign the PSM rather than letting an application request one. Android gains an embedder-supplied radio backend driving a radio the embedder installs into a per-node slot with Node::enable_app_owned_ble_radio(), called after new() and before start(), because its Bluetooth APIs sit behind a permission and foreground-service model only the application can satisfy. Stopping the transport now stops the scan as well as the advertisement, which matters only where the embedder owns the radio: BlueZ ends discovery when the scanner's event stream drops, but an app-owned radio went on scanning for the life of the process. Probe retry backs off by powers of two with a capped retry book, and each connect outcome has its own counter and structured log line carrying the role, outcome, PSM and time to conclusion. Inbound handshakes run off the accept loop, eight in flight and aborting the oldest at the bound, where the exchange previously ran inline and held the loop for its full 5-second deadline, making effective inbound concurrency one.

Observability & measurement

  • An optional tick-body profiler behind the new profiling Cargo feature, off by default. When enabled, fipsctl profile tick on [--dir PATH] / off / status starts and stops a capture at runtime with no restart. Each capture writes one tab-separated file (default /var/log/fips, capped at 32 MB) carrying, per ten-second interval, the exact count, max and total for every step of the rx-loop tick arm, the whole-tick span, and gauges for ticks, peer count, the gap between successive tick-arm entries and the resulting arm-starvation delay. With the feature off the instrumentation macro is a pure pass-through, so a default build contains no timing code on the tick path. LogsDirectory=fips was added to the packaged systemd units so the capture directory is created and cleaned up declaratively.

  • fipsctl probe <npub|hostname> answers, for one target, where it sits in the spanning tree relative to this node and whether this node can actually reach it. The work runs as five stages that report separately, bloom, discovery, path, session and rtt, because one verdict covering several findings is what sends an operator to the source: "no peer's filter claims this address" says the mesh has never heard of the target, while "a filter claimed it and nothing answered" says the opposite. The probe opens an FSP session, waits for one MMP receiver report to yield a round-trip time, and tears down only what it opened. A session that existed before the probe started is never torn down, ownership is decided at the moment of action rather than once at the start, and it is re-checked before teardown, so a session adopted by traffic underneath the probe is left alone. The path printed is the least-common-ancestor walk computed from the two sets of coordinates, which is the worst-case fallback route rather than the route a packet necessarily takes: a cut-through between peers can deliver in fewer hops, so the tree distance is an upper bound. Nothing traverses the mesh to confirm the hops, and a real per-hop trace needs a wire message that does not exist. Nothing here changes the wire format; the probe is built from messages that already exist. The control socket carries three new commands, probe_start, probe_poll and probe_cancel, each returning in well under a millisecond with the stages advanced on the daemon's tick, because a probe needs a mesh lookup, a Noise XK handshake and at least one remote MMP tick, which no single control round-trip could survive inside the socket's five-second timeout. A probe that runs and finds a problem is not an error response: the status is ok and the failure sits in the per-stage verdicts, and error responses stay reserved for malformed or inadmissible requests. On a terminal the stage block is redrawn in place with a running elapsed on whichever stage is working; piped or redirected there is no cursor to move, so each row prints once, at the moment it settles, and the transcript ends up the same block a terminal leaves behind. --json emits exactly one document at the end, so a script parsing the report does not have to skip past progress output.

Packaging & deployment

  • A NixOS module and an overlay are exposed from the flake, so a flake consumer enables the daemon with one line rather than hand-rolling a systemd unit. overlays.default adds pkgs.fips, and nixosModules.default provides services.fips.*: enable, package, configFile, openFirewall (UDP 2121 and TCP 8443) and dns.enable, which routes .fips to [::1]:5354 through systemd-resolved declaratively rather than with setup and teardown scripts. packaging/nixos/README.md documents it with a full consumer flake.nix. Contributed by Arjen.

  • fipsctl address [npub|hostname] prints a node's fd00::/8 mesh address and nothing else, without contacting the daemon. With no argument it derives the local node's address from fips.key in the default key directory, falling back to the world-readable fips.pub beside it; --key PATH names a key or public key file elsewhere. This lets an installer or image build write a mesh address into a config file at a point where no node is running and none can be, and keeps the derivation in one place rather than reimplemented by whatever needs it.

  • packaging/debian/build-deb.sh --features <list> builds the .deb with a Cargo feature list, which is how an instrumented package is produced for a measurement run. The auto-derived dev Version gains a matching +<features> marker, so a feature build and a default build of the same commit are no longer indistinguishable: without it the two carry byte-identical versions, an install of one over the other is an apt no-op, and the running node offers no way to tell which one it has. The marker sorts above the unmarked build, so installing a feature build is an upgrade and reverting to the default build is a downgrade: revert with dpkg -i rather than apt install. --features is refused together with --no-build, which would stamp the marker onto binaries the features never reached.

Changed

Naming: discovery split into lookup and rendezvous

  • The mesh-lookup control-metrics family is now emitted under the key lookup in fipsctl stats metrics and show routing. The former key discovery is still emitted as a deprecated alias carrying identical counters; update dashboards and alerts to read lookup.

  • The overloaded node.discovery.* config table was split into node.lookup.* (mesh-lookup scalars: ttl, attempt_timeouts_secs, recent_expiry_secs, backoff_base_secs, backoff_max_secs, forward_min_interval_secs) and node.rendezvous.* (peer rendezvous: nostr.*, lan.*). A deployed node.discovery: block still loads and is folded into the new tables with a one-time deprecation warning; migrate your fips.yaml to the new keys. This includes the two keys v0.4.2 introduces under the old spelling, so an operator upgrading straight from 0.4.2 does not have to infer them: node.discovery.nostr.max_concurrent_offers_per_npub and node.discovery.nostr.signal_ttl_secs are now node.rendezvous.nostr.max_concurrent_offers_per_npub and node.rendezvous.nostr.signal_ttl_secs.

  • The Ethernet transport's per-interface discovery flag was renamed to listen (transports.ethernet.*) to match the symmetric announce (transmit) / listen (receive) neighbor-beacon vocabulary. The old discovery: key is still accepted via a serde alias, so deployed configs continue to load unchanged; Config::to_yaml() re-emits it under the canonical listen: name. Update your fips.yaml to listen:.

  • fipstop's Routing State pane renames its Discovery Requests and Discovery Responses sections to Lookup Requests and Lookup Responses, and reads those counters from the canonical lookup key rather than from the deprecated discovery alias. The counters themselves are unchanged, so an operator who knows the pane by its old section labels is reading the same numbers under new names.

Library surface and internals

  • The protocol layers were restructured into sans-IO cores with the I/O kept in a thin shell, and two new crate-root modules, nostr and mdns, own peer rendezvous and LAN discovery. The crate root also gains the is_punch_packet helper and the CoordError, MtuExceeded, COORDS_REQUIRED_SIZE and MTU_EXCEEDED_SIZE exports. This is an internal reorganization with no operator-visible behaviour change and no wire change: encoded bytes and decode decisions are identical. Its two consequences a reader will feel are the library surface under Removed below and the tracing targets in the next entry.

  • Tracing targets follow module paths, so the module relocations in this release move them. fips::discovery::nostr::* becomes fips::nostr::*, mDNS moves to fips::mdns::*, and the protocol subsystems move under fips::proto::* (fips::tree to fips::proto::stp, fips::bloom to fips::proto::bloom, fips::protocol to fips::proto::*, and the mesh-lookup subsystem from fips::discovery to fips::proto::lookup). An existing RUST_LOG filter naming an old target still parses and simply stops matching, so the symptom is missing log lines rather than an error, and a filter that has gone blind looks exactly like a subsystem that has gone quiet. Update RUST_LOG filters, journal-watch recipes and any log-scraping alert accordingly. The four targets named explicitly in source rather than derived from a module path (fips::config, fips::instr, fips::node::handlers::handshake, fips::node::handlers::rekey) are unaffected.

Node health

  • Node health is determined at start completion instead of unconditionally reaching a single running state. Zero transports up is now fatal: the node tears down cleanly and the daemon exits with an error, where it previously came up and served nothing. Any configured optional child that failed to start, meaning a transport beyond the first, Nostr, mDNS, TUN, DNS, or a worker pool, leaves the node degraded but serving, with a warning naming what failed, and all configured children up is full health. A child the node was never asked to run does not count against it. The published node state gains Degraded and Failed, both visible via control queries, with degraded operational and failed not. Exit detection for the DNS task, the two TUN threads and mDNS also re-evaluates health at runtime, so a child that dies after a healthy start now shows as degraded. For Nostr the watched children are the three service loops that cannot return by design (the inbound notify loop, the advert publisher and the refresh ticker); connect_task and relay_startup_task are deliberately not watched, because Client::connect() returns as soon as it has spawned the per-relay background tasks, so a finished handle there carries no health signal. Transports and worker pools expose no runtime-exit signal yet and are unchanged.

Peer handshake

  • node.rate_limit.handshake_resend_interval_ms no longer governs the first outbound handshake resend. When msg1 is sent, the peer state machine arms the first retransmit deadline from a hardcoded 1000 ms constant (HANDSHAKE_RETRANSMIT_INTERVAL_MS in src/peer/machine.rs), because the machine-armed timer rather than the connection's stored deadline is now the due signal the retransmit driver fires on. The key still governs the second and later resends, alongside node.rate_limit.handshake_resend_backoff and node.rate_limit.handshake_max_resends. The constant equals the shipped default of 1000, so a deployment that never overrode the key sees no change; one that raised or lowered it will find the first resend still firing at 1000 ms. The limitation is recorded at the site in the source.

Data-plane / transports

  • Connected UDP peer drains now batch macOS receives with recvmsg_x(2), matching the wildcard UDP receive path instead of issuing one recv(2) syscall per queued datagram.

  • Routing next-hop selection now visits borrowed peers and coordinates instead of allocating candidate snapshots for each forwarded packet.

  • A connected UDP socket that cannot open now names the syscall that failed and the address it was operating on, the local address for bind and the peer address for connect. Both paths previously returned a bare OS error that the caller wrapped identically, so a field report of Address already in use could not be attributed to either, and the two have entirely different causes: on Linux a UDP connect(2) to a 4-tuple another socket already holds returns EADDRINUSE, which is not the same fault as bind refusing the local address. A node at roughly 245 peers was emitting this three times a second across nine peers with no way to diagnose it.

Observability

  • The warning raised when a lookup response carries a path MTU below the actionable floor now names the request it refused, as a request_id field on the log line. Every other warning that handler emits already carried the correlator, and this one could not: it is raised while the cached-coordinates effect is applied, after the response itself has gone out of scope. An operator reading the refusal therefore had a counter and a peer name but no way to tie the line to a particular exchange, or to the sibling lines logged for it. Only the log line changes: the response is still accepted, the coordinates are still cached, the sub-floor path MTU is still discarded, and the same counter is still charged.

Docs & contributor tooling

  • Source comments and doc comments were corrected across the modules this release restructured. Unresolvable internal references and planning-phase locators were removed from comments that exist only on this line, stale view-trait documentation was rewritten to describe the shape the restructuring produced, and stale liveness claims in the peer machine, executor and timer documentation were corrected. No code changed.

Packaging & deployment

  • Every GitHub Actions reference in the workflows and composite actions that exist only on this line is now pinned to a full commit SHA, so the whole .github tree is pinned or explicitly justified. The branch carries 75 action references, of which 71 are pinned to a 40-character commit SHA with the mandatory trailing version comment and 4 are left on mutable tags by explicit allowance (dtolnay/rust-toolchain@nightly and three taiki-e/install-action@nextest); testing/check-action-pins.sh reports the same 75 and passes. Nine of those references were pinned here, in package-freebsd.yml and an android-check job block, neither of which exists on the maintenance branch: the original pinning sweep was authored there, never saw these files, and they arrived through the merge still on mutable tags, at which point the guard that shipped with the sweep failed the branch exactly as intended. The general lesson is worth keeping with the guard: a checker authored on the earliest branch is only as complete as that branch's file set, and merging it upward gates files it has never swept.

Deprecated

  • The discovery metric-family key (control-socket JSON), renamed to lookup. It is dual-emitted alongside the new lookup key during a migration window and will be removed. Migrate dashboards/alerts from discovery.* to lookup.*.
  • The node.discovery.* config table. Its keys were split into node.lookup.* (mesh-lookup) and node.rendezvous.* (peer rendezvous). A legacy node.discovery: block still applies for now with a deprecation warning and will be removed; migrate to node.lookup.* / node.rendezvous.*.
  • The Ethernet transports.ethernet.discovery flag, renamed to transports.ethernet.listen. The old key is still accepted via a serde alias and will be removed at the v2 cutover; migrate to listen.

Removed

  • Source-breaking for consumers of the library crate: the crate-root modules bloom, discovery, mmp, protocol and tree are gone, along with the crate-root HandshakeState, PeerConnection, PeerSlot and ProtocolError re-exports. The protocol cores moved into an internal proto module and are reached through crate-root re-exports instead: tree types through proto::stp, bloom types through proto::bloom, the FSP, STP, lookup, routing and FMP wire types through their matching proto::* submodules, and PromotionResult / cross_connection_winner through proto::fmp rather than peer. The HandshakeState removed here is the peer connection-phase enum, not the Noise handshake type of the same name. The Noise type is untouched, still lives at fips::noise::HandshakeState, and it is that type, not this one, that the [0.4.2] entry below about four public types gaining Drop describes. ProtocolError is replaced by fips::Error, whose Malformed variant now carries a &'static str rather than a String and which gained BadSizeClass, BadCoord and BadBloom variants, so the diagnostic text changed with it. PeerSlot and the PeerConnection resend API were unused and are deleted. Node::connections() is now pub(crate) and yields the internal peer machine rather than a PeerConnection; a consumer that walked links through it should use Node::peers(), Node::get_peer() and Node::peer_count() over ActivePeer, which remain public. Nothing about the behaviour of the shipped binaries changes, and nothing on the wire changes: encoded bytes and decode decisions are identical.

Fixed

  • A peer that goes away no longer leaves its path-MTU state behind. The record of which transport link-seeded a destination's path MTU had no removal site on any lifecycle, so the map grew one entry per peer this node had ever linked with and held them for the life of the process. Both the stored value and its seeding record are now released when the path they describe goes, together rather than separately: releasing the record alone would leave the value with nothing recording where it came from, and a peer returning over a wider transport would then be refused by the never-loosen rule indefinitely. A peer whose link is still up has both written straight back by the reseed that already follows.

  • Removing a link now clears every reverse-lookup key it was inserted under. The removal rebuilt one key from the link's own remote address, so an entry inserted under a different address form — which the cross-connection promotion arms do, keying on the packet's source address — outlived the link it named. An entry a newer link has since claimed is still left alone.

  • A rejected handshake no longer strands a session index or a retry schedule. Two reject arms freed neither, which the next-line versions already do; master now carries the same shape, so an ACL-rejected outbound dial is rescheduled and an abandoned rekey index is returned rather than held.

Transport

  • The UDP listen socket's address-reuse flags are now set after its bind rather than before. Before the bind they mean the kernel may hand back a port another flagged socket already holds, so a second daemon binding the same configured address started silently and shared the port, with the kernel splitting inbound datagrams across the two receive loops on the source 4-tuple, where the second daemon should have failed with EADDRINUSE. After the bind they mean what was actually wanted, that the per-peer connected sockets may later join the port. Those connected sockets are the joiners and keep their flags before their own bind, which is where they belong.

  • A BLE connection the pool refused is no longer counted as an established link. The scan and probe loop recorded connections_established, resolved the address out of the retry book and handed the peer up to the node layer after ConnectionPool::insert had already refused the connection and dropped it. Reaching the refusal needs a full pool with no evictable slot, and every BLE connection is built non-static, so only max_connections: 0 gets there.

Control socket

  • disconnect on the control socket now closes the transport connection rather than only the peer. It notified the peer and freed every node-side structure, sessions, indices, links, address mapping, tree and bloom state, and never touched the transport, so on a connection-oriented transport (TCP, Tor, Nym, BLE) the pool entry, the socket and its inbound-slot accounting survived the peer the node had just forgotten, until the far end closed or the receive loop errored. An operator who disconnected a peer to free a slot did not free the slot. No effect on UDP, Ethernet or loopback, whose close_connection is the connectionless no-op. Still not addressed: disconnect reports peer not found for an identity that is only mid-handshake, so withdrawing a peer during its handshake leaves that leg resending msg1 until the handshake timeout bounds it.

  • connect on the control socket now tries the address it was given for a peer the node is already connected to, instead of reporting success without doing anything. The command built an ephemeral peer configuration and handed it to the ordinary dial path, which returns success the moment the peer is already held, so fipsctl connect printed success and the node never attempted the path. An operator moving a peer onto a freshly provisioned link, or a supervising process that has just seen a second path come up, had no way to make the node use it: the peer stayed where it first authenticated until that path died. The address is now tried as an alternate path alongside the live one, through the same helper a runtime peer refresh uses, so promotion happens only after the alternate handshake authenticates and a wrong address cannot displace a healthy link. The response gains an additive refreshed field distinguishing "started an alternate-path handshake" from "already on this exact path and it is fresh". connect stays ephemeral: the peer is not written to configuration and gets no auto-reconnect.

  • A failed probe now names the condition it actually hit. Probing an absent key reported one of two different failures depending on whether a bloom filter had returned a false positive, and the reason shown was wrong in the branch that gets likelier as the mesh grows: the verdict was right, the explanation was not. The core already held the fact and discarded it, since the lookup gate proceeds only when a peer's filter claimed the address, so a sent lookup proves the claim; that is now its own failure kind, carried to the control socket through the existing reason field. fipsctl renders the two differently and closes with a note naming the false-positive mechanism, which is the part that helps, because the reason string alone does not tell an operator that a filter cannot miss a key that is present. The seventeen-second wait is unchanged, being the retry ladder running to completion; what changes is that the operator is told the wait carried no information rather than given a wrong reason for it. Still not addressed: naming which peer's filter made the claim.

Data-plane / transports

  • An inbound onion connection no longer leaks its inbound slot when the peer goes away before the accept loop has pooled it. The Tor accept loop spawned the per-connection receive task, then inserted the pool entry, then bumped the inbound counter; a remote that reset immediately let the receive task run its cleanup first, find nothing to remove, skip the teardown that decrements, and leave the increment behind for the life of the process. Once enough of those accumulated the max_inbound gate rejected every further onion connection, with the pool visibly empty. The accept loop now holds the receive task on a readiness barrier until both the pool entry and the counter are in place, as the TCP accept loop already did, and an aborted accept still falls through to the cleanup rather than stranding the entry. The same accept path also releases the slot of an entry it evicts, which a reused ephemeral forward port could otherwise leave orphaned. Outbound Tor connections and the whole of the Nym transport are unaffected: neither holds a counted inbound slot.

  • A path MTU measured on one link no longer clamps a peer that has moved to another. Every writer of the per-destination path-MTU cache keeps the smaller of the existing and incoming value, which is right while a peer stays put, but the entry is keyed by destination alone. So a peer first reached over a narrow link stayed clamped to that link's ceiling for the lifetime of the process: when it later became reachable over a wider transport, promotion re-seeded, the seed saw a tighter existing value and declined, and traffic kept running at the old link's ceiling on a link that could carry far more, with nothing reporting it because the clamp was doing exactly what it was told. The node now records which transport last seeded each destination and treats a seed from a different one as authoritative rather than as a loosening to refuse. Re-seeding the same transport still keeps the tighter value, so repeated promotion does not reset discovery, and a destination with no prior seed is unchanged.

  • An inbound IPv6 source address no longer loses its scope. Converting a raw sockaddr on receive dropped sin6_scope_id, and a link-local source (fe80::/10) identifies a host only together with its interface scope, because the same address may exist on several interfaces. The failure was silent rather than loud: the address parsed, it looked correct in a log line, and only the reply failed. On a link where every address is link-local, a Wi-Fi Aware data path for instance, that stalled the Noise handshake. msg1 was sent to a scoped address and arrived, the peer replied, and msg2's source was recorded with scope 0 and could not be routed back, so msg1 retried indefinitely with nothing reported. The conversion is shared by all three Unix receive paths, the single-packet recv_from, the Linux recvmmsg batch and the Darwin recvmsg_x batch, so one fix covers each. Sources outside the link-local range carry scope 0, for which the scoped and unscoped forms are identical, so nothing else changes.

macOS

  • The packaged macOS daemon recreates and binds its control socket at /var/run/fips/control.sock. A privileged macOS process selects that private runtime path before its leaf exists, so bind creates it and clients follow once it is there, and socket setup now changes ownership and mode only for a private parent directory it creates or recognizes as a canonical FIPS runtime directory. Previously the packaged daemon fell through to the shared /tmp/fips-control.sock path after every boot, and because socket setup changed the parent directory unconditionally, the root daemon also took group ownership of /tmp itself.

Packaging & deployment

  • The FreeBSD package manifest carried the same bounced maintainer address the other packaging paths did, so the contact of record in the new .pkg artifact would have been unreachable at its first release.

Security

Coordinate cache

  • The coordinate cache is warmed from plaintext session headers on datagrams a node is merely forwarding, and nothing filtered those writes. Two checks now run at the write site, and an entry carries its provenance, so a coordinate established by a lookup whose proof this node checked is no longer displaced by an unauthenticated hint. Four counters report what the checks refuse: coord_warm_foreign_root, coord_warm_key_mismatch, coord_hint_rejected and coord_hint_changed, all in fipsctl show status and fipsctl show routing. These are mitigations and not a closure. The coordinate is still not authenticated, so a same-root forgery is unaffected and hint-over-hint for a destination that was never verified is unchanged. Treat the counters as a rate to watch rather than an alarm.

Tick profiler

  • The --dir given to profile tick on is now confined to /var/log/fips when the daemon runs as root. The control socket is reachable by the fips group, which the security model writes down as strictly weaker than root, yet the directory travelled from the socket straight into a root create_dir_all with no validation: a group member could create a root-owned directory anywhere on the filesystem, including a path a later privileged component reads. The path must now be absolute, must contain no .., and must still resolve under the root once every existing ancestor has been followed through its symlinks, so a symlinked parent does not launder a lexically clean path. A daemon that is not running as root crosses no such boundary and takes --dir as given, which keeps the documented non-root cargo run capture working; the flag can no longer point a root daemon at a different log root, which was its other documented use. This only ever affected a --features profiling build: the subcommand is absent from a stock package.

  • A capture file is no longer written over whatever is already at its path. The name is a one-second UTC stamp and therefore predictable, so File::create would have followed a symlink pre-planted at the next name, and it truncated any real file it found. The file is created only if it does not exist, and a capture started in the same second as a previous one takes the next free -N suffix instead of failing. Capture files are created private to their owner and the capture directory is no longer left world-accessible by a permissive umask; a capture carries the node npub, build, platform and a timing series.

[0.4.2] - 2026-08-25

FMP/FSP sessions and rekey

Changed

  • Config validation now rejects two node.rekey settings that appear to disable the trigger and in fact fire it continuously. after_messages of zero makes the message-count arm true on every poll, because the trigger compares the counter with greater-or-equal. after_secs at or below the per-session jitter bound is the same trap on the timer arm: each session offsets the interval by a random value within plus or minus that bound, so a smaller interval saturates to zero on a negative draw and rekeys on sight, for roughly half of sessions. Both are checked whether or not rekey is enabled, so switching it on later cannot surface the error at a surprising moment, and neither gains an upper bound; a very large value remains the supported way to disable one arm. A config carrying either setting now fails to load instead of starting a node that rekeys constantly.

Fixed

  • Inbound session-setup messages are now rate limited, keyed on the authenticated link peer the datagram arrived over. The setup path allocated a session entry and sent a routed SessionAck for every well-formed message naming an address it had no entry for, and that address is an envelope field the sender picks, so one neighbour could grow the session table at whatever rate it could transmit and buy an ack per entry to a destination of its choosing. The limiter sits ahead of every send and both handshake constructions in the handler, so a refused message emits nothing and costs no cryptography. The key is the link peer rather than the claimed source address, which is what makes it a limit at all: keying on the source would hand a single sender a fresh full bucket per forged message.

    Two consequences worth stating rather than discovering. The limiter bounds each neighbour's contribution and makes a flood attributable; it does not give the node an absolute ceiling, which stays at roughly peers * rate * handshake_timeout_secs. And a legitimate peer reaching this node over the same link as an attacker shares that attacker's bucket, so establishment behind a flooded neighbour is refused until it refills. Rekey and restart traffic is deliberately not subject to that: setup messages naming an already-established peer draw on a separate per-link bucket, because suppressed key rotation is silent (nothing errors and no session drops) and would have shown up only as a flat rekey_armed.

  • A forged SessionAck no longer destroys an in-flight session initiation. The handler removed the session entry to take ownership of the handshake state and, when the XK msg2 read failed, returned without putting it back. Nothing in that message is authenticated (the only thing tying it to the initiation is the datagram's source address, which the sender chooses), so any node able to reach the victim could cancel any initiation with 57 bytes of the right length, and hold establishment down by repeating it. The entry is now kept. Keeping it is not enough on its own, and the second half is the part worth naming: the msg2 read mixes the sender's ephemeral into the symmetric state before it authenticates anything, so an entry put back as the failed read left it holds a handshake that can never read the genuine msg2, which trades a one-round-trip denial for one lasting the full handshake timeout. The read is therefore rolled back to its pre-read state before the entry goes back. The three later failure paths in the same handler still drop the entry: each is downstream of a msg2 that authenticated, so it is a local failure rather than a possible forgery. The entry's activity stamp is deliberately not refreshed on the failure path, so a spray cannot hold a dead initiation past its original sweep deadline, and a new ack_handshake_failed counter makes the refusals visible at the default log level. Only the XK handshake on this branch is covered; the additional drop sites in the XX handshake on the development branch are not.

  • An unauthenticated session msg3 no longer discards a completed key epoch. Five sites discarded the whole rekey, which nulls a pending session sitting beside the handshake, when only the handshake had failed: the four failure paths in the responder-side rekey arm of the msg3 handler, and the dual-initiation yield arm in the setup handler, which is gated on a rekey being in progress rather than on this node having initiated it, so an entry the peer armed reaches it too. That pending session is the epoch the real peer may already have cut over to, so discarding it kills the reverse direction until the session idles out. Two unauthenticated messages reached it: a forged setup message arms a handshake beside a completed rekey once that rekey has waited a full idle timeout for a peer that never appeared on the new epoch, and any garbage msg3 of the right length then finishes the job. All five now abandon only the handshake. The four remaining sites in the ack initiator arm are deliberately left alone and the reason is recorded there: an entry with the initiator flag set holds no pending session, so the two calls are the same action at those sites.

Security

  • A frame whose counter is u64::MAX is now refused by the replay window instead of being accepted as a new high-water mark. Accepting it pinned highest at the ceiling, after which every subsequent counter from that peer fell more than a replay window below it and was rejected, wedging that peer's own receive path until a rekey replaced the session. The send side already refuses to emit that counter (take_send_counter and advance_nonce both return a nonce-overflow error), so no conforming peer can produce it and the refusal is invisible on the wire; the highest counter an honest peer can send, u64::MAX - 1, is still accepted. Reaching this required an already-authenticated peer running modified code, and the damage was confined to that peer's own session.

  • The MMP gap tracker advances its expected-counter state with a saturating add, so a received counter of u64::MAX no longer overflows it. The wrap silently reset the expectation to zero in a release build and aborted the task under a build with overflow checks on, such as the test harness. Behaviour is unchanged for every counter an honest peer can emit.

  • An epoch-mismatch msg1 no longer tears down a peering that is still carrying authenticated traffic, and a second epoch change for the same peer identity inside 15 seconds is refused. The epoch travels inside the AEAD, so such a msg1 is authentic, but it stays authentic after capture: replaying one destroyed a working peering, and with it the FSP session state that peering carried, from off the path. The peering's last authenticated inbound frame is the evidence that it is still alive, and nothing an unauthenticated sender emits can refresh it, so a peer that genuinely restarted clears the gate by having stopped sending. The refusal is a silent drop: no msg2 is returned, since the stored msg2 is bound to the original msg1's ephemeral and answering a sender-chosen address is free amplification. The interval is stamped only when an epoch change is accepted, so a sustained replay cannot starve a genuinely restarting peer. Both thresholds come from one constant, sized so a restarting peer's msg1 resends still land inside its own first handshake window and below link_dead_timeout_secs, and nothing changes on the wire.

  • Retention of a superseded FSP key epoch is now capped at an absolute ceiling measured from the cutover, defaulting to 120 seconds against the 10-second drain window. The drain deadline slides forward on every inbound frame that authenticates against the previous slot, which is what keeps a peer that lost msg3 from having the old epoch erased out from under it, but it also meant the authenticated peer holding that key could keep the retired key resident for as long as it kept sealing frames in the old epoch. The sliding grace is unchanged; it now delays erasure by a bounded amount rather than preventing it. The ceiling is set to clear the worst-case legitimate recovery of a peer that lost msg3 (the msg3 resend ladder, then handshake_timeout_secs before the responder abandons, then the rekey dampening window before it may re-initiate, about 90 seconds at stock settings), and it is raised automatically if the configured handshake timers imply a longer budget, so shortening a timer cannot push the ceiling under the recovery it has to leave room for. Nothing changes on the wire; each side runs its own drain. A peer that has still not recovered when the ceiling fires is left with undecryptable frames until its own rekey retry re-converges the epochs, since nothing tears an established session down on repeated decrypt failure.

  • A session setup message naming an already-established peer no longer replaces that peer's session. The handler did this whenever node.rekey.enabled was false: it ran a fresh responder handshake and overwrote the entry, discarding the live keys. The message carries no authenticator and its source address is an envelope field, so anyone able to reach a node could name an established peer and take that session down, repeatedly, and hold it down by repeating the message. The established case now always arms the handshake alongside the running session and adopts the new keys only after a msg3 whose authenticated static key matches the key the session was opened with, which is the check the rekey path already applied; a peer that genuinely restarted still re-establishes, and a forged setup leaves the session carrying traffic. This changes no wire format and adds no configuration: a node with rekey disabled already answered such a message, it simply destroyed the session afterwards.

  • A session rekey armed by a peer's setup message is abandoned if the matching msg3 never arrives, rather than persisting for the life of the session, so an arming that never completes cannot make the node read a later genuine setup message as a simultaneous initiation and drop it. Only the armed handshake expires, and only it: a rekey that completed is the key epoch the peer has already moved to, since it exists only because a msg3 carrying that peer's authenticated key arrived and the sender of that msg3 promotes the new epoch on an unconditional two-second timer. Expiring those keys on any timer would drop every later frame from that peer, so they are held until the peer's own frame promotes them, a newer completed rekey replaces them, or the session goes away. What the wait does bound is precedence, not the keys: a completed rekey outranks a fresh setup message from that peer only until it has waited a full idle timeout, after which the setup is answered normally, so a peer that restarted while we held such a session is not refused for as long as our own sends keep the session from idling out. The handshake timeout logs at INFO, since it costs nothing, and a completed session displaced by a newer one at WARN, since that does throw away keys the peer may hold. Session counters record the arming of a handshake by a setup message, each of the three ways such a message is refused, and each displaced session, so a node under a sustained spray of setup messages shows a rate rather than nothing; the per-message log lines stay at DEBUG because an unauthenticated sender can drive them at line rate. These counters are not yet readable through the control socket.

  • The session drain sweep and the cut-over that retires an old key epoch now run whether or not periodic rekey is enabled. Both sat behind the periodic-rekey gate, so a node with rekey disabled that adopted new keys held the superseded ones for the life of the session.

  • The FSP session address is now bound to the peer key the Noise handshake authenticated, on both the initial and the rekey path. The responder recorded a session under the source address carried in the datagram without ever checking that address against the static key it had just authenticated, so a peer could complete a genuine handshake while claiming another node's address, and the identity cache, the session map and the address the IPv6 shim reconstructs on delivery would all attribute its traffic to the node it named. The address is now derived from the authenticated key at the point it first becomes available in msg3, and a mismatch drops the half-open session without recording either the identity or the session. The rekey responder needed its own check: it returns before that code is reached and never read the peer's static key at all, so a rekey could complete under an established session with a different key than the one that opened it. It now requires the key to be unchanged and abandons the rekey while leaving the existing session intact, rather than tearing the session down, which would have handed an attacker a way to kill established sessions. Both comparisons are on x-only keys, because a stored key may carry a synthesized parity while the handshake learns the true point. The two rejections are counted separately in the session reject statistics.

  • A link handshake admitted by the established-address waiver is now confirmed against the identity that owns that address, instead of on the address alone. A transport configured with accept_connections false still admits an inbound msg1 whose source matches an established peer, so that a peer re-handshaking after a restart or a rekey is not locked out, but nothing checked that the party sourcing from that address was the peer. Any off-path party able to send from it therefore obtained a full link handshake from a node configured to accept none. Once the key exchange reveals the initiator's static key, the handshake is now dropped unless that key belongs to the identity the matched address is attributed to, and dropped as well when the waiver was used and no identity owns the address at all, which fails closed rather than skipping the check for that case. The cheap refusal is unchanged: a stranger reaching a transport that refuses inbound connections is still turned away before any cryptography. Attribution consults both the reverse-address lookup and the scan over established peers rather than stopping at whichever answers first, because the reverse lookup can name a link that no longer exists, and stopping there would refuse a peer the scan can still attribute, permanently, since the confirmation returns above the code that repairs that lookup. Both refusals log at warning level, naming the expected and actual peers, and charge the existing handshake bad-state rejection counter rather than one of their own.

  • An inbound frame whose header disagrees with the frame that arrived is now dropped, at the single dispatch point every transport converges on, before the declared length can be used as a parsing input. The 4-byte common prefix carries a payload length that the node never read, so on the datagram transports (UDP, Ethernet and BLE, which deliver one whole frame per packet and where the arrived length is therefore known exactly) nothing compared the two. This closes no known defect, and it is worth being exact about what it drops on a deployed line. The stream transports (TCP, Tor, Nym) read their frame boundary out of that same field, so the comparison holds by construction and never fires for them. A short datagram is a truncated frame, which already failed the AEAD tag or the exact-size handshake parse, so what changes there is which reason it is dropped for rather than whether it is dropped. A frame whose phase the node does not recognize carries no fixed relationship between the two and is left alone rather than rejected on a guess. The drop takes its own rejection reason and its own payload_len_mismatch counter rather than reusing the admission one, since it is a framing rejection decided before the phase dispatch and it applies to established data frames as well as to handshakes; that counter is not yet readable through the control socket.

NAT traversal and Nostr discovery

Added

  • node.discovery.nostr.max_concurrent_offers_per_npub, defaulting to 4, which bounds how many inbound traversal offers one sender npub may have in flight at once. It sits inside max_concurrent_incoming_offers, which remains the outer bound, so a value above that is inert; zero is rejected at config validation, since it refuses every inbound offer rather than disabling the limit, and so is a value above the maximum permit count a semaphore can hold, which would otherwise fail at construction rather than at load. Existing configurations parse unchanged, the key being optional.

Changed

  • Inbound traversal offers are now admitted against a per-sender allowance as well as the global pool. The intake path previously took a permit from a single semaphore before any identity check, with the sender's npub used only as a log field, so one sender could hold every slot and deny traversal onboarding to every other peer for as long as it kept offering. Admission now takes a per-npub permit and a global permit together. A sender over its own allowance is refused at debug rather than warn, because the party tripping it is by definition sending faster than the node wants and a record per rejection would turn the spam into log volume; the global bound being reached keeps its warn, which is the operator's signal that the node is genuinely saturated. This does not make the pool inexhaustible. Nostr identities cost nothing to generate and the signal subscription carries no author restriction, so an attacker running four throwaway npubs still saturates the shipped 16-slot pool at an unchanged total offer rate. What the change buys is that one identity can no longer do it alone, and that the two refusals are distinguishable in the log. The permit is still held across the whole attempt; that duration remains inferred from the attempt timeout rather than measured.

  • Config validation now rejects a node.discovery.nostr.signal_ttl_secs that is too large for the configured replay_window_secs. A traversal signal is acceptable over its TTL plus 60s of clock-skew grace on each side, and that span has to stay strictly inside the replay window, or a session id evicted from the replay cache on expiry is still fresh enough to be accepted a second time. The relation was documented but unenforced, so raising the TTL past 180s silently voided it. The bound is derived from the skew constant rather than restated, and is checked whether or not nostr discovery is enabled, for the same reason the rekey rules are. The shipped defaults (120s against 300s) are unaffected, but a configuration that had widened the TTL or narrowed the replay window now fails to load, with an error naming the concrete floor for replay_window_secs. The NAT lab's config generator was one such configuration and its generated replay_window_secs moves from 60 to 180. Note that this covers eviction on expiry only: seen_sessions_max_entries remains a separate capacity-eviction route that no config relation bounds.

  • The peer-retry tick no longer awaits the Nostr advert refetch. It ran inline on the 1-second rx-loop tick, awaiting a fetch with a 2-second timeout for each due peer and discarding the result; with up to sixteen due peers the timeouts stacked, and field profiling measured single 2.00 s stalls as the common case and a worst tick of 12.4 s against a 1 s period, delaying every other rx-loop arm by as much as 4.2 s. The refetch is now spawned, so a dial uses the advert cached at that moment and the refreshed one lands for that peer's next retry.

  • The Adopted NAT traversal socket log line now carries the transport id and the local address alongside the peer npub. Without the local address an operator cannot join a host socket table against adoption events, and without the transport id several peers sharing one adopted transport are indistinguishable from several separate adopted transports.

Fixed

  • Nostr NAT traversal signals are now sent only to relays the client pool actually holds. A signal is addressed to the merge of the peer's NIP-17 inbox relays, the relays its advert nominates for signaling, and our own DM relays, but the pool is built once at startup from the configured relays and the send is rejected outright, before anything is contacted, if any single URL in that list is outside it. One unconfigured relay anywhere in the merge therefore killed the whole attempt, including the sends to relays both sides shared. On a public node in open mode this made discovery non-functional: 309 traversal attempts, 290 explicit failures, zero successes, every failure on relay not found. Configured peers were unaffected, since they run a matching relay set. Comparison is on the normalized relay URL rather than the raw string, so a configured relay spelled with a trailing slash or different host case is not discarded. Two smaller fixes ride along: the responder resolves its relays before binding a socket and running STUN, rather than spending a STUN round trip and holding an offer slot only to find it has nowhere to answer, and it gained the empty-relay-list guard the initiator already had.

  • Nostr NAT traversal no longer breaks after the host suspends. The traversal clock cached a Unix timestamp once at startup and advanced it with a monotonic Instant, which does not tick while a machine is asleep, so after a suspend the daemon's idea of the time trailed real time by the suspend duration for the rest of the process lifetime. Every NIP-40 expiration it computed was therefore published already in the past: relays dropped the offers as expired, the initiator logged a signal timeout waiting for an answer, and traversal stayed broken until the daemon was restarted. The clock now reads the wall clock on every call. This is not macOS-specific, though a laptop that sleeps is where it is easiest to hit; any host that suspends or hibernates was affected. Reported in #128.

Security

  • An advert or inbox-relay list returned by a relay is now checked against the peer it claims to describe before anything else looks at it. The relay pool verifies every event's signature but does not check a reply against the request filter, and neither of the two options that would make it do so is enabled, so a relay may answer a request for one author's advert with an event it signed itself. The stale-advert refetch picked the newest created_at across everything returned, with no author test, and then wrote the result into the advert cache under the requested peer's npub, so a single hostile or compromised advert relay could pin an endpoint set of its own choosing for that peer. The author test now runs before the timestamp contest rather than after, so a future-dated foreign event cannot even suppress the genuine advert by winning it. The same filter now applies to the inbox-relay lookup, where the omission let an attacker-authored relay list steer this node's direct-message and traversal-signal traffic. A refetch that comes back with events, none of them signed by the peer, now leaves the cached entry alone: that is no evidence the advert was withdrawn, and evicting on it would hand the same relay a way to clear the cache. A refetch that genuinely comes back empty still evicts.

  • An advert's created_at is now clamped forward to the same 60s of clock skew the traversal-signal path already tolerates. An unbounded future timestamp bought a cache entry a proportionally distant validity horizon and an unbeatable position in every replacement comparison, so a later genuine advert could never displace it and the size-cap eviction collected it last. The clamp applies to the stored timestamp as well as the validity window, at all three points where an advert is cached, so ordering and expiry now agree. Clamping rather than refusing the event is deliberate: a node whose own clock runs slow reads every peer's honest advert as future-dated, and refusing would silently withdraw Nostr-mediated dialing for every peer at once.

  • Inbound traversal signals are now rate limited before they are decrypted. A rendezvous-enabled node handed every kind-21059 event straight to the unwrap, which is two NIP-44 decrypts and a signature verify, inline on the single task that also routes traversal answers and maintains the advert cache. Nothing bounded how fast an unauthenticated stranger could schedule that work: the per-npub offer admission cannot, because it keys on the sender's public key, which only exists once the first decrypt has already run, and because it is a concurrency semaphore rather than a limit over time. A token bucket now sits ahead of the unwrap, so a flood costs a node its inbound offers instead of the whole notify loop.

    What the limit can and cannot key on is worth stating, because it decides the shape of the fix. Before decryption there is no sender identity at all: the outer event is signed by a key generated per event, so bucketing on its author would hand an attacker a fresh allowance for free, and the timestamp and recipient tag are equally attacker-chosen. The arrival relay is drawn from our own configured set but is not an isolation boundary either, since an attacker publishes to the same relays an honest peer does. The shared allowance is therefore a single global bucket and is indiscriminate by construction, which on its own would shed our own traversals along with the attacker's, and since the attacker sets the rate every retry would land in the same shed. A second, smaller allowance is held in reserve and drawn only while this node has traversals of its own outstanding, so a flood denies a node its inbound offers, which nothing receiver-side can prevent without a pre-decrypt identity, rather than also denying it the answers to offers it sent. Shed signals are counted and reported at debug level per event with a warning each time the running total doubles, so a bucket sized below a busy node's real need shows up in the log rather than as apparent relay flakiness.

    Two limits on what this buys. It bounds the crypto path only: the advert branch runs earlier in the same loop and is not metered here, so a stranger can still put JSON parsing and a cache insert on the task per event. And the relay SDK verifies each event's outer signature on its own per-relay task before this loop ever sees it, which no receiver-side change short of dropping the subscription can avoid.

  • A STUN binding response is now accepted only from the address the binding request was sent to. The client discarded the source address recv_from returned and let the parser decide, and the parser checks only the message type, the magic cookie and the 12-byte transaction id. An on-path attacker who could read the outbound request could therefore inject a reply carrying a transaction id copied from it, and its chosen address became the reflexive candidate the node published in a traversal offer or answer, redirecting the peer's hole-punch packets. Datagrams from any other source are counted and discarded, and one debug record per STUN attempt reports the count and the last unexpected source, so a rejection is diagnosable without giving a flooder control of the log rate. A server that answers from an address other than the one dialed, which RFC 5389 forbids, now times out and the next configured server is tried.

  • The exemption that lets a peer's reflexive address skip the private-address gate is now conditional on our own vantage point. That exemption exists for the deployment whose STUN server sits inside the private network, so the observed reflexive address is legitimately private; it was applied unconditionally, so a node whose own STUN result was public still punched whatever private address a peer named as its reflexive one. Any sender whose offer or answer was accepted could therefore aim a burst of UDP packets, carrying this node's source address, at a host inside the node's own private network, which is the one place the candidate filter was written to keep it out of. The gate now applies whenever our own reflexive address is public. It stays lifted when our own reflexive address is itself private, which is the LAN-STUN deployment the exemption was for, and also when we have no reflexive address at all, so a failed STUN probe cannot cost a node its same-LAN peering. Two consequences to state rather than discover: a peer behind a private STUN server talking to a node with a public one loses its reflexive candidate, which was never reachable from us in any case, and because the /24 comparison is IPv4-only a unique-local IPv6 reflexive address is refused unless our own reflexive address is unique-local too. An off-subnet refusal of a peer's reflexive address is a shape an honest deployment now produces, so it no longer raises the refusal record to warning level on its own; the never-routable, port-0 and unparsable classes still do.

  • A peer's candidate list is now bounded before it is walked rather than only after. The eight-target cap ran after both planning loops had finished, so it bounded what a node punched but not what it spent deciding: a signal naming several thousand candidates had every one of them parsed and vetted, and the deduplicating scan that follows is quadratic in the plan those candidates feed. At most 32 candidates are now vetted, four times the target cap and four times what the candidate generator produces on the widest host, and the excess is discarded rather than failing the offer, so an honest many-homed peer loses the tail of its list instead of its traversal. The refusal record carries the discarded count as a new over_offered field and treats a non-zero one as an attack shape, since nothing honest reaches the bound.

  • A NAT-punch packet is now accepted only from an address this node planned to probe. The punch packet's discriminator is a plain digest of the session id, a value both peers already know, and it travels in the clear in every probe, so acceptance proved only that the sender had seen one. The receive loop broke on the first packet whose digest matched, whatever its source, and returned that source as the peer address, so anyone who observed a probe, or who could reach the node and guess the session id, could have an arbitrary address adopted as the peer: the legitimate traversal was denied, the Noise handshake and its retransmissions went to an address of the attacker's choosing, and the pair was charged a failure against its backoff state. The npub-pinned handshake still could not authenticate to the wrong host, so this was a denial and a misdirection rather than an impersonation. The source address is now ranked against the planned target list before anything else: an unplanned source is dropped and, deliberately, is not acked either, since acking it is a reflection the node controls. A source matching a planned target exactly is adopted immediately, as before. A source matching a planned target's IP on a different port is what a symmetric NAT's fresh mapping looks like, and it is still adopted, because that is the main class of NAT pairing punching exists to rescue; it is held as a candidate for 250 ms first, so an exact match arriving inside that window supersedes it. The honest path's latency is unchanged. Two consequences to state rather than discover: an attacker that can source packets from a planned target's IP on any port is still accepted, which is the residue only an authenticated probe can close; and an attempt under a flood of spoofed matching packets now runs to its full timeout instead of ending on the first one, so the refused sources are counted and reported once when the attempt ends rather than logged per packet.

  • Traversal punch targets taken from a peer's offer or answer are now filtered and bounded. A rendezvous-enabled node previously punched every address a signed offer named, including loopback, link-local, multicast, broadcast, unspecified and CGNAT addresses, and placed no limit on how many candidates one offer could carry. Any npub could therefore have a node emit a burst of UDP packets at addresses of the sender's choosing, carrying the node's own source address. Candidates in the never-routable ranges are now rejected, IPv4-mapped IPv6 forms are canonicalized before the check so they cannot slip past it, candidates with port 0 are dropped, private-range candidates are punched only when they share a /24 with one of our own addresses (which is what same-LAN traversal already required of its own path), and the planned target list is capped at eight. A peer's reflexive address is checked against the never-routable ranges but not against the /24 rule, so a deployment whose STUN server sits inside the private network keeps working. A malformed address in a peer's signal now drops that one candidate instead of failing the whole traversal. A node also records what it declined: one log record per planning attempt carries how many candidates the peer offered, how many were planned, the count refused in each class and one sample address, at warning level for the shapes no honest peer produces and at debug level for the routine off-subnet case. Same-LAN and reflexive traversal are otherwise unaffected.

  • Traversal offers and answers dated in the future are now rejected. The freshness check measured a message's age with a saturating subtraction, which yields zero for any timestamp ahead of the local clock, so the age test could not fail for a future-dated signal and no other term bounded the issue time from above. A signal claiming to be issued arbitrarily far in the future was accepted as strictly fresh, which voided the property that the freshness window is narrower than the session-id replay window (300s by default) and left the replay cache as the sole defence against a captured offer being replayed. Forward-dating is now tolerated only up to the same 60s of clock skew already allowed in the other direction, and a signal accepted under that grace reports the skew outcome, so the existing clock-skew log fires for a peer whose clock is ahead just as it does for one whose clock is behind. The declared expiry timestamp is also no longer trusted beyond the issue time plus the configured TTL, so a sender cannot widen its own acceptance window by inflating that field. A single timestamp is now acceptable over at most the signalling TTL plus 60s on each side, 240s under the shipped defaults. Rejections are also now distinguishable in the log: a stale signal and a future-dated one no longer share one reason string, and the inbound-offer path, whose only surface was an unattributed debug line below the default log level, now names the peer and the session and warns for the rejection classes that relay delivery lag cannot produce (future-dated, identity-mismatch and malformed offers), leaving an ordinary stale offer quiet. As with the existing inbound rate-limit warning, an unauthenticated remote peer can drive that line. A failure of our own offer's freshness during answer validation is reported against the offer rather than mislabelled as the answer's, and the tolerated-acceptance log now carries the issue and expiry stamps and no longer attributes the acceptance to clock skew, since a peer configured with a longer signalling TTL than ours now reaches it too.

Data plane, routing signals and metrics

Changed

  • Peer bloom filters are computed for every recipient in one prefix and suffix union sweep rather than rebuilt per recipient. Announcing to R peers previously did R full map builds and R by T merges; at 240 peers that was 20.6 ms per tick, roughly half the tick body, with a median per-interval maximum of 34.5 ms. The result is exactly equal rather than approximately: merging is a bytewise OR, so regrouping the unions cannot change it. The trade-off, measured rather than assumed, is that the sweep does its full work regardless of how many peers are ready, so a tick announcing to one or two peers now costs about twice what it did; break-even is around three ready peers. Cadence, the debounce, the sequence rule and the fill-ratio cap are unchanged.

  • Each peer's npub is derived once at construction instead of once per tick. The per-tick stats snapshot ran a bech32 encode for every tracked peer, and a second one for the common peer with no hosts-file entry and no alias, since the display-name fallback bottoms out in the same encode: 14.1 ms per tick at 240 peers. The display name itself is deliberately not cached, because the alias map and the host map both mutate at runtime.

Fixed

  • A SessionDatagram carrying a truncated inner FSP payload no longer panics the forwarding path. The coordinate-cache warm path sliced the inner payload at the full 12-byte header offset while guarding only with the 4-byte common prefix parser, so an inner payload of 4 to 11 bytes with phase 0x0 and the Coords Present flag set indexed past the end of the slice. Because the receive loop is the process's main future, the panic terminated the daemon rather than a task, and under the packaged systemd unit the node restarted into the same frame. The warm path now applies the same FspEncryptedHeader guard the local-delivery path already used, which additionally means a malformed frame carrying a non-zero protocol version or the Unencrypted flag alongside Coords Present is dropped rather than having its body read as coordinates. Any peer that had completed a link handshake could trigger this, and admission is default-open. Frames rejected by that guard are now counted in the forwarding statistics as warm_malformed_packets and warm_malformed_bytes, the byte counter charging the whole outer frame, visible over the control socket and on the fipstop Routing State pane, so a node being fed malformed frames is distinguishable from a quiet one at the default log level. The count is not a packet drop: the frame is still delivered or forwarded, and only the coordinate-cache warm attempt is abandoned. The existing debug log now also carries the frame's protocol version and flags, which separate a short frame from a bad-version or Unencrypted-flagged one.

  • SessionDatagram hop-limit handling now follows IP semantics. Delivery to the addressed node is no longer TTL-gated, and a forwarder decrements before deciding rather than after, so a datagram that would leave with a TTL of zero is dropped instead of transmitted. Previously the TTL check ran ahead of the local-delivery test, so a datagram addressed to this node that arrived with TTL 0 was dropped, and a forwarder receiving a transit datagram at TTL 1 transmitted it at TTL 0 for the next hop to discard, wasting one transmission per expiring datagram. SessionDatagram::decrement_ttl and SessionDatagram::can_forward were aligned to the same semantics: decrement_ttl decrements first and reports false when the result is zero, and can_forward is true only at a TTL of 2 or more. The reachable radius is unchanged, because the two behaviors compensated exactly: a path of h links still delivers for any source TTL of h or more. During a rolling upgrade, an unupgraded forwarder feeding an upgraded destination delivers one hop further than either version does on its own; no version mix delivers less far. The TtlExhausted reject counter now charges at the node that makes the decision rather than at the hop after it.

Security

  • A transit node's induced routing errors are now bounded by the authenticated link peer that induced them. The 100 ms suppression gate on CoordsRequired, PathBroken and MtuExceeded was keyed on the failed datagram's destination address, which is an envelope field the sender picks, so a fresh random destination on every packet was always a first sighting and every packet was admitted. Each admission also inserted a key and then walked the whole map, so per-packet cost grew with the flood rate while the sender's cost stayed flat, and the error itself is addressed to the datagram's source address, which nothing binds to the sender either. A new per-link-peer token bucket, 20 signals a second sustained with a burst of 50, is now consulted first, keyed on the AEAD-authenticated peer the frame arrived over: the one value at that point a sender cannot mint. The per-destination interval is kept unchanged behind it, because it still does the aggregate suppression a genuine outage needs, and no gate was added on the address the error is returned to, which would have handed a sender a way to silence honest errors toward a victim it names by keeping that victim's key hot.

    Two ordering choices in there rather than left to be discovered. The peer's token is peeked and only spent once the destination gate has also admitted, so a single unroutable destination behind a high-fanout peer cannot burn that peer's whole budget on signals nothing sends and silence every other destination behind it. And the destination map now carries a hard ceiling of 4096 entries with its expiry sweep amortized to once per eviction interval rather than run on every admission; when it is full it admits without recording rather than refusing, because refusing would turn a full map into node-wide silence exactly during partition healing, when many destinations are legitimately unroutable at once. Emission stays bounded by the peer budget in that state. Three counters, rendered on the fipstop Routing tab, make each of the three outcomes visible instead of silent.

  • A transit-emitted PathBroken no longer carries the reporter's cached coordinates for the unreachable destination. The signal is returned to the datagram's source address, so anyone able to reach the node could name any address and have the node's coordinate cache read back to them, one entry per packet. The field is optional on the wire and no receiver reads it, so this is an emission change only: an unmodified peer parses the frame exactly as before. Which of the two signals is emitted still discloses whether the entry exists.

  • A reactive MtuExceeded is now believed only when this node has actually sent a frame larger than the bottleneck it reports. The signal is unauthenticated: the admission gate narrows which destination may be named but cannot say who named it, so a value at the floor was a legal value from anyone, and one datagram drove a bound session's path MTU to 256 and pinned the address-keyed entry the SYN-time MSS clamp reads, with recovery costing three consecutive higher notifications across two notification intervals. Each session now carries the largest frame this node has put on the wire toward it since the last accepted decrease, and a report is refused unless it names something smaller. Honest path-MTU discovery satisfies that by construction, because the report exists only because a frame we sent did not fit; a forgery has to wait for us to emit something bigger than the value it wants to claim, which bounds every accepted claim from below by our own traffic. The evidence is cleared on each accepted decrease and on release, so one large send early in a session cannot vouch for the rest of it.

    The guard sits ahead of both effects rather than between them, which is also where the existing floor check moved to: the floor previously ran after the session's own path MTU had already been changed and so governed only the lookup table. The reactive carrier now names its own floor constant, held equal to the actionable floor so no hop legitimately configured with a small transport MTU loses its feedback; corroboration, not the floor's value, is what stops a legal-but-forged claim. A separate counter, rendered on the fipstop Routing tab, distinguishes an uncorroborated refusal from a below-floor one.

  • The path-MTU release a PathBroken drives is now rate limited per destination on a budget of its own. That signal is unauthenticated too, and the release discards a bottleneck this node learned by having a packet dropped, so repeating the claim discarded a genuine value as fast as it could be relearned. The limiter is a separate instance rather than the one the coordinate warmup send already uses: a budget another signal can spend is not a bound. Deferring a release is the safe direction, since the value kept is the tighter one.

  • The influence a remote party has over path MTU is now bounded, and the per-destination path MTU cache has a way back. The path_mtu field is an unsigned per-hop transit annotation carried outside the signed proof, and the MtuExceeded and PathBroken signals arrive unencrypted with no sender check, so any forwarder, or anyone who can reach the node, could lower it, and it was accepted with no minimum. A single MtuExceeded carrying a very small value drove a session's path MTU to zero, after which every packet to that destination was answered with an ICMPv6 Packet Too Big instead of being sent: a blackhole that lasted until the daemon restarted. The same value reached the SYN-time TCP MSS clamp, where anything at or below 137 saturates to a segment size of zero and the band just above it yields single digits. Values below an actionable minimum are now ignored rather than applied or stored, at the three places a remote value is acted on: the path MTU state machine, the reactive MtuExceeded write, and the discovery response, whose coordinates are still cached so refusing the annotation cannot become a way to deny discovery. The MSS clamp additionally refuses to write a zero. Each of the three refusals logs a warning and increments its own counter in the error-signal family, so an operator can tell them apart without scraping logs: they carry different meanings, one being an authenticated peer inside an established session, one an unencrypted signal anyone able to reach the node can send at will, and one a verified discovery response whose unsigned annotation a forwarder on the reverse path rewrote. Because those three refusals are the only way a remote value reaches the per-destination store, the SYN-time clamp does not apply the minimum a second time when it reads that store: a small value there is one the node derived from its own outgoing link, which is exact rather than suspect, and BLE in particular negotiates a link MTU per connection that lands under the minimum routinely. The clamp refuses only a stored value admitting no TCP payload byte at all, at 137 or below, where the segment size saturates to zero and the clamp would be skipped entirely; it logs that at trace rather than warn, since it sits on the per-packet path, and the peer's link promotion reports it once instead. A stored per-destination path MTU is released when the path is invalidated by a PathBroken report, by session idle expiry, or by handshake timeout, and the link MTU read from the local transport is reseeded in its place, so a directly connected peer does not lose its own measurement along with the remote claim. The release also resets the session's own current path MTU alongside the address-keyed entry, rather than reseeding the link value and leaving the tightened one in place, so a path declared dead recovers at once instead of only through the increase ladder. Entries written by the discovery lookup carrier carry their own deadline and age out, since a destination this node never opens a session with reaches none of the three release routes: without that, a single response carrying a floor value pinned that destination's clamp until restart, and an unknown request id still classifies as originator, so a captured response could be replayed indefinitely. The notification mirror deliberately carries no deadline. Locally derived MTUs are not subject to the minimum, at the seed or at the clamp. Legitimate narrow paths are unaffected: adaptation to hops well below the IPv6 minimum, which the mesh does use, continues to work.

  • The three routing signals (CoordsRequired, PathBroken, MtuExceeded) are no longer acted on unless this node has itself bound the destination address they name, either by initiating a session toward it or by completing the Noise handshake that binds an address to a peer's static key. These signals carry no end-to-end authentication, so until now any admitted mesh member could send one naming any address and have its effects applied: a path-MTU clamp written for an arbitrary address, a cached-coordinate flush for an arbitrary address, and a discovery and warmup cycle for an arbitrary address. The MtuExceeded case was the sharpest, because its write into the address-keyed path-MTU lookup that the TUN reader consults at TCP MSS clamp time sat outside the session guard and so required no session, no peer relationship and no prior state at all. A half-open session created by an inbound handshake that has not yet proved its address does not admit these signals, so a forged session opening cannot be used to unlock them. Signals from a genuine on-path forwarder are unaffected: the reporter may be any node at any distance. This does not make the sender authentic, which nothing short of a wire format change can do. Rejected signals are counted as unknown-session rejections, and additionally on four new error-signal counters visible through show routing, show metrics and the fipstop routing pane: unbound_coords, unbound_broken and unbound_mtu give the refused count per signal type, against the existing per-type arrival counters as the denominator, and unbound_forged counts the subset whose claimed source and destination pairing no honest forwarder could produce. The drop log line now carries the signal type and the refusal class.

  • A discovery lookup response is now acted on only when it answers a lookup this node actually has outstanding. The originator path took any response whose request_id was not in the transit dedup map, so an admitted peer could harvest one genuine signed response for a target and re-inject it at will: each injection cleared the victim's in-flight lookup, recorded a reachability success for a target that might be unreachable, refreshed the cached coordinates for a further full TTL, and flushed the victim's queued packets onto a route at a moment the sender chose. It also reached the signature verify before any check that the response was wanted, so the verify was the first cost gate on the path. The node now records the request_id of every lookup request it sends on that target's pending entry, and a response is dropped unless it names a target with a lookup outstanding and carries one of the ids issued for it. Because the id is fresh 64-bit randomness drawn per attempt and the target signs over it, a harvested response is bound to the request it answered and cannot be redirected or replayed. The check runs before the identity-cache resolve and before the signature verify, so a response nobody asked for costs nothing. Replies to earlier attempts of a still-outstanding lookup are still accepted, which is the common case on a link whose round trip exceeds the first rung of the retry ladder. Drops are counted as resp_unsolicited, visible through show routing, show metrics and the fipstop routing pane; the counter has a nonzero floor in healthy operation, because a request is flooded to every qualifying tree peer and the duplicate replies land there once the first has been accepted.

  • A flooded discovery dedup cache no longer makes a node unresolvable. The cache is both the duplicate filter and the reverse-path table for lookup responses, and at its 4096-entry bound it dropped the arriving request. That drop sat ahead of both the check for whether the request names this node and the forwarding path, so one link peer emitting fresh request_ids could stop the node answering lookups for itself and stop it carrying anyone else's, for as long as it kept the cache full. The cache now makes room instead of refusing: over a peer's own share it drops that peer's oldest entry, and at global capacity it drops the oldest entry of whichever peer holds the most, so a light peer's reverse path is never taken to admit a heavy one and extra identities buy a flooder proportionally less. A peer's share is the cache divided by the current link-peer count, with a floor of 64. The loosening this accepts is that an evicted request_id arriving again inside the window is forwarded a second time rather than recognised as a duplicate, which the per-target forward limiter and TTL already bound. Evictions are counted as req_dedup_evicted; the old req_dedup_cache_full counter stays in place, frozen at zero, so a dashboard carried across versions does not lose the series.

  • Answering a lookup for ourselves is now metered per link peer. The response proof is signed over the requester's request_id, so every request addressed to this node costs a fresh Schnorr signature that cannot be cached or served twice, and until now the only thing bounding that rate was the dedup cache filling up, which is the defect above. A token bucket per link peer, 256 signatures of burst refilling at 32 per second, absorbs the legitimate burst that follows a topology change, when many correspondents re-look-up at once through the few links that lead here, while capping what one neighbour can make the node sign. Refusals are counted as req_sign_rate_limited and visible in show routing, show metrics and the fipstop routing pane. A refused request keeps its dedup entry, and retries carry fresh request_ids, so a refusal cannot suppress the retry.

Admission, rate limiting and peer caps

Added

  • node.rate_limit.session_setup_burst (64) and node.rate_limit.session_setup_rate (16.0), the parameters of the new per-link-peer session-setup limiter. This is the FSP session-setup bucket, and it is distinct from the link-layer msg1 bucket described below; the two meter different messages and are sized independently. Setup messages naming a peer this node is already established with are metered on a second per-link bucket derived from node.limits.max_peers, node.rekey.after_secs and node.rate_limit.handshake_max_resends, so raising the peer limit sizes it automatically. A zero burst or a non-positive rate is rejected at config validation rather than silently refusing every session.

  • node.limits.max_sessions, defaulting to 1024, which bounds the end-to-end session table. Zero means unlimited, which restores the previous behaviour exactly and is the way to back the change out on a running node. The default is four times the adjacent node.session.pending_max_destinations. A session entry measures 6608 bytes of inline state plus heap, so the table holds to roughly 7 MB, and a test pins that per-entry figure so the arithmetic behind the default fails loudly if an entry grows. Existing configurations parse unchanged, the key being optional.

  • node.rate_limit.established_handshake_burst and node.rate_limit.established_handshake_rate, the parameters of the new established-link msg1 token bucket, which meters link-layer msg1 rather than FSP session setup. Both are optional; omitting them (the normal case) derives the bucket from node.limits.max_peers, node.rekey.after_secs and node.rate_limit.handshake_max_resends, so raising the peer limit sizes the bucket automatically. An explicit zero burst or a non-positive rate is rejected at config validation rather than silently refusing all rekey traffic.

Changed

  • Inbound msg1 is classified before it is rate limited, and rekey or restart msg1 arriving on an established link now draws on its own token bucket instead of competing with stranger admission for a single shared one. On a node with many peers the shared bucket refused a large share of ordinary rekey traffic: a field node at roughly 245 peers refused 8753 msg1 in 25 minutes, and 159 of the 201 distinct sources were peers it already held sessions with. Nodes upgrade with no config change. The Msg1 rate limited log line now reports which limb refused, the pending count or the token bucket, which it previously did not distinguish.

Security

  • The Ethernet transport's discovery buffer is now bounded and no longer costs a linear scan per beacon. Beacons are unauthenticated broadcast frames, and the buffer deduplicated by scanning a Vec for the source MAC and had no cap, so anything on the segment could name a fresh MAC per frame and drive both quadratic CPU in the receive loop and unbounded memory. It is drained once per tick only while the transport is operational, so a transport that is receiving but not operational was never drained at all. The buffer is now a map keyed on source MAC, capped at 1024 distinct MACs between drains, with the drain order still oldest sighting first so which neighbour gets dialed under a connect budget does not depend on hash iteration order. A MAC already buffered is always refreshed, so a flood of new MACs cannot crowd out a neighbour already seen. Refused beacons are counted in the transport's stats as beacons_dropped and reported in the log on the first drop and then on each power-of-ten thereafter, so the flooder does not set the log rate. What this does not close: a flood can still crowd out a neighbour not yet seen in that tick, and anything able to flood raw frames on the segment can already jam the beacon at L2 more cheaply.

  • A read failure on peers.allow, peers.deny or the hosts file no longer turns the node into an open one. Every read error other than a steadily absent file was logged and swallowed, leaving that file's entries empty, and the reloader published the result unconditionally: an unreadable peers.deny admitted the peers it named, and an unreadable peers.allow took a node from admitting a named few to admitting everyone, with one warning line as the only signal. Because the recorded modification times advanced before the load, nothing retried until the file changed again, so a persistent permission or I/O fault left the empty ACL in force indefinitely. The reloader now keeps the last loaded ACL when any input is present but unreadable, leaves the modification times alone, retries on the next tick regardless of them, and logs the fault once on the transition rather than once per tick. An absent file is still a policy and still loads as an empty set; a NotFound that a successful stat contradicts is treated as a file being rewritten under us and held. A reload whose inputs all read cleanly but which empties an enforcing ACL while its files are still on disk is held for one tick, which catches a read that caught a non-atomic in-place edit mid-write, and released on the next so a deliberate blanking still takes effect. fipsctl ACL status gains a stale flag reporting that the policy in force is older than the files on disk. What this does not close: there is no last-good snapshot at startup, so a node whose ACL file is unreadable at boot still comes up with no entries, now logged as an error and armed to retry on the first tick. Admission is also checked only at handshake time, so a peer admitted during a window that has already happened keeps its link.

  • The end-to-end session table now has a bound. It was the one remotely-grown map with none: an inbound SessionSetup naming an address nobody had seen inserted an entry, and the two existing limits did not reach it, the setup limiter governing the arrival rate rather than the population and the idle purge only reaching entries a peer stops using. One neighbour sending setups at the permitted rate could hold roughly 1440 half-open entries at any moment and grow the table without limit by keeping them warm. Setups that would grow the table past node.limits.max_sessions are now refused, ahead of the setup limiter, so a full table costs no token, no responder handshake and no ack; a refused setup emits nothing at all, which is indistinguishable from loss to the sender and is already covered by its own msg1 resend schedule. The test is whether admitting would grow the table, not whether the sender is a stranger, so a resent setup for an entry already present is still served and an in-flight handshake is not broken. Unauthenticated half-open entries are additionally held to half the table, so a handshake flood cannot deny the whole of it to peers that complete; that share is sized to leave a reconnect storm, where every peer initiates at once after a restart or a healed partition, room to land. Locally originated sessions are capped at the same ceiling, answered with ICMPv6 destination unreachable so the application gets an immediate error rather than a silent drop. The cap refuses rather than evicts: the setup that triggers the decision is unauthenticated at that point, so evicting would hand a stranger a way to tear down sessions it has nothing to do with. Refusals are counted as table_full and half_open_full in the session reject family. What stays open is per-neighbour fairness among established sessions: one hostile neighbour that completes handshakes and keeps each session warm can occupy the table and hold new session establishment closed for as long as it keeps doing so, which is a denial of new sessions rather than the unbounded memory growth it replaces.

  • An accepted inbound TCP connection no longer holds a slot indefinitely without sending anything. The cap was tested at accept and the pool insert and counter bump followed with no read in between, while the frame reader's reads carried no deadline, so an unauthenticated remote held a slot by connecting and staying silent. Pool keys are ip:port, so N sockets from one address took N slots, and at the 256 default that locked out inbound peering for as long as the sockets stayed open. The first frame on an inbound connection now has a deadline, as a module constant rather than a new configuration key, and the onion listener gets the same treatment for the same accept-then-count ordering. Separately, the node's handshake reaper tore down session state without closing the transport connection, so a peer that sent msg1 and then stalled was forgotten by the node while its socket and slot survived; the reaper now closes the connection too. What this does not close: the deadline covers the first frame only, so a peer that sends one well-formed frame and then goes silent still holds its slot. Closing that needs a rolling idle deadline.

Transports and configuration

Changed

  • fipsctl keygen no longer exits non-zero when only the fips.pub write fails. The private key is already on disk at that point, so failing the run reported failure for a keygen that did produce the identity; the failure is now a warning and the run succeeds. The pre-existing-key guard also moves from exists to symlink_metadata, so a dangling symlink at the key path now blocks keygen without --force instead of being overwritten silently.

Fixed

  • The UDP transport's DNS cache is now bounded and actually evicts. The map held one entry per distinct hostname string ever dialed, and the TTL was applied only on the read, so a stale entry was overwritten on the next dial of the same name and otherwise stayed for the life of the process. Under a rendezvous policy that accepts advertised endpoints the keys are strings a remote party chose, which made the growth theirs to drive. A store now sweeps entries past their TTL and, if the map is still full, drops the oldest, holding it to 256 hostnames. Refreshing a name already cached evicts nothing. Eviction is by insertion time rather than last use, so a rarely dialed name in a very large peer list may re-resolve more often; the cost of a wrong eviction is one DNS lookup, not a failed dial.

  • macOS: stopping an Ethernet transport under load no longer hangs the process. The BPF reader thread handed each frame to the async consumer with blocking_send, which parks with no way to be woken. Stopping the transport aborts the consumer first, so nothing drains the 1024-frame channel, and the socket's Drop then joined a thread that could never return: on a busy interface the daemon had to be killed. The socket now drops the receiver before joining, which releases a parked send at once, and the reader thread sends through a helper that watches the same shutdown pipe its select() already honours, so a send waiting for room cannot outlive a shutdown request. The helper yields before it sleeps, so the saturated-path handoff rate is unchanged. Not covered by CI: the reader thread is macOS-only and Linux CI compiles none of it. What the tests prove is that the helper the thread now waits in is cancellable; that a real BPF thread exits under load still needs a manual check on a Mac.

  • A failed private-key write no longer leaves a node silently running an ephemeral identity. Six write results in the identity path were discarded, and the sharpest was in persistent mode: a failed write to fips.key fell through to an ephemeral identity with no message, so a node that had been asked for a stable identity changed its npub, its routing address and its mesh IPv6 on every start, and nothing said so. All six now report. An ephemeral start that is about to overwrite an existing key file now warns first, naming the path and the setting that would have preserved the identity, which is the warning fipsctl keygen has always given and the daemon never did. Existence is tested with symlink_metadata rather than exists, because a dangling symlink reports absent from the latter while still being a file the write acts on. The persistent read path additionally warns when it finds a key file whose mode is looser than 0600, or one that is a symlink; it does not repair either, since the daemon does not own a file it did not create.

Spanning tree, mesh size and routing

Fixed

  • Flap dampening can now engage more than once in the lifetime of a node. The arming check tested whether a dampening deadline had ever been set rather than whether one was still in effect, so the first episode disarmed the mechanism permanently: a node in a second flap storm went on switching parents under hold-down alone, and neither the flap_dampened counter nor the "Flap dampening engaged" warning fired again, so the storm was invisible to anyone watching that counter. A lapsed episode is now retired explicitly, clearing both the deadline and the switch counter, so a second episode requires a fresh threshold of switches within one window rather than re-engaging on the first switch after lapse. Hold-down was unaffected throughout and continued to limit discretionary switching, which is why the practical effect at shipped settings was lost visibility and a lost escalation tier rather than unrestrained flapping. Every path that can engage an episode now reports it, including a re-engagement during parent-loss recovery, which was previously silent. The warning names which path armed the episode (trigger) and how long discretionary parent switching stays suppressed (dampening_secs), using the same trigger values as the parent-switch logs beside it, so the two can be read together. A node.tree.flap_dampening_secs large enough to overflow the monotonic clock is capped at one year, beyond which an episode is indistinguishable from permanent, so an extreme setting no longer panics the node when dampening engages.

DNS responder

Security

  • The DNS responder's mesh-interface filter now works on macOS and FreeBSD, where it had never run. The filter drops .fips queries that arrive over the mesh TUN, which is what keeps a widened dns.bind_addr from exposing the hosts file's alias space to every mesh peer. It was keyed on the interface index resolved from the configured TUN name, but macOS and FreeBSD assign the device a name of the kernel's choosing (utunN, tunN), so the lookup found nothing, the index came back None, and None disables the filter. The index is now resolved from the name of the device the node actually created, which the TUN startup path already records, and a live device whose index will not resolve is logged rather than passed off as "no mesh interface". Linux is unaffected, since the configured name is the device's name there. Behaviour change on macOS and FreeBSD: a node with a non-loopback dns.bind_addr stops answering .fips queries that arrive over the mesh interface. What this does not close: with an app-owned TUN the node never learns a device name, so the filter stays off there. Not measured: whether macOS and FreeBSD attribute a locally originated query sent to the node's own mesh address to the TUN interface, as Linux does. If they do, such a query is now dropped on those platforms; the shipped resolver drop-in targets [::1] rather than the mesh address, so the packaged path is not affected.

Gateway and peer lifecycle

Fixed

  • A failed log write can no longer panic the thread or task that logged. The subscriber was built with the default internal-error reporting, which sends a failed write to eprintln!, and that macro panics when stderr has also failed. The shipped supervisor configurations make that a single condition rather than two: the macOS plist points both standard streams at one unrotated file, and the systemd units route both to journald, so one full disk fails both sinks together. In the daemon a crypto worker was the case that mattered: it logs a warning on send backpressure, and a worker that dies takes its share of the peer space with it permanently, while the panic message is discarded along the same broken path. In fips-gateway, which built its subscriber the same way, the casualty is a spawned task: the DNS resolver, the control accept loop or the pool tick, none of which is observed until shutdown, so the process would keep running and reporting healthy with mesh name resolution or lease expiry and NAT cleanup silently stopped.

Security

  • The gateway DNS forwarder now validates an upstream answer before it becomes a NAT mapping. It previously accepted whatever datagram arrived: the upstream query reused the client's own transaction ID, the upstream socket was wildcard-bound and never connected, the receive discarded the sender, neither the response ID nor the question section was compared against what was asked, and the returned address was not checked against the mesh prefix. Because the extracted address is installed as a DNAT rule that carries no interface constraint, a forged answer redirected traffic rather than only poisoning a lookup. The upstream query now carries a random transaction ID, the socket is connected so the kernel drops foreign sources, a response must match on ID, question and type or it is discarded while the receive continues against the original deadline, and the address goes through the validating parser with a non-mesh answer refused before any allocation. One deliberate behaviour change: the validation sits before the rcode check, so an upstream answering FORMERR or REFUSED with an empty question section now yields SERVFAIL rather than having its rcode relayed. Checking after the rcode would admit a forged NXDOMAIN. Connecting the socket also means a dead upstream surfaces ECONNREFUSED immediately instead of stalling for five seconds.

Control socket

Security

  • The control socket and the directory holding it are now created with a restrictive mode rather than created wide and narrowed afterwards. bind(2) makes the socket inode 0777 & ~umask, so under a permissive umask the socket was world-accessible for the window between the bind and the chmod to 0770 that followed it; the bind now runs under a umask that masks the "other" bits, so the inode is 0770 from creation and the chmod and chown stay the authority on its final mode. The parent directory was worse than a window: it was created with create_dir_all, which is also 0777 & ~umask, and nothing ever set a mode on it, so under a permissive umask the directory holding the socket stayed world-writable for the life of the host, and a world-writable parent lets an unprivileged account plant an entry at the socket path. Directories this code creates now come out 0750, which is what the systemd unit (RuntimeDirectoryMode=0750) and the FreeBSD rc script (install -d -m 0750) already apply, so no packaged deployment sees a different mode and no fipsctl user loses access. Both the daemon and the gateway control sockets are covered. What this does not close: the window between the stale-socket probe and the bind is documented at the site rather than removed. Reaching it needs write access to the socket's parent directory, which the packaged layouts give to root alone, and an account holding it can deny the daemon its socket more simply by squatting the path first.

Key material and identity files

Security

  • Private key writes no longer follow a symlink, and the key file's mode is enforced rather than merely requested. The single write path opened with create and truncate and no O_NOFOLLOW, so a symlink planted at the key path was followed and its target overwritten, and it supplied the mode only through open(2), which the kernel honours on creation and ignores otherwise, so a fips.key that already existed at 0644 stayed 0644 through every rewrite. That second half needs no attacker: one chmod, or a restore that did not preserve modes, leaves the key readable indefinitely. Both writers now share an open helper carrying O_NOFOLLOW, and the private key has its mode applied to the open descriptor before any secret bytes are written. The public key keeps create-time mode instead, since forcing it would reopen an operator-tightened fips.pub on every start. On Windows neither protection applies and the file inherits the parent directory's ACLs; that exclusion is deliberate and recorded at both writers.

  • Private and symmetric key material is now cleared when it goes out of scope. Nothing in the crate erased a key before this: the node's private key sat in the loaded configuration in plaintext for the whole process lifetime, which is the longest any secret lives here, every Noise handshake left its static and ephemeral keypairs, its per-message Diffie-Hellman results and its chaining key in freed memory, and each session's ChaCha20-Poly1305 keys were dropped intact. Clearing now covers the retained cipher key on each cipher state, the chaining key and the handshake hash, the 64-byte HKDF outputs and the two session keys derived from them, the static and ephemeral keypairs a handshake holds for its whole duration, the identity's long-term keypair, the temporary copy each of the fourteen elliptic-curve operations makes from a keypair, the bech32 and hex encodings of a secret, and the private key on its way through configuration, including the config file's whole text, which is treated as secret for as long as it is held, since node.identity.nsec is read straight out of it. Two places that assigned over an already-loaded key now clear the old value first: assignment frees the previous string without running the type's clearing destructor, so a configuration that already carried a key left the superseded copy in the heap whenever a second source replaced it. This clears the copies the crate owns, not every copy that ever existed. The secp256k1 key types are copyable, so the compiler may duplicate them where no code here can name them, which is why that library calls its own erase non-secure. The hash and key-derivation states, and the cipher keys cached inside ring, offer no clearing route at the versions pinned here and are deliberately left alone; the residue any of this leaves needs access to the process's memory, or to a core dump or swap image of it, to read. Adds a dependency on zeroize. Nothing on the wire and no configuration changes. See the ### Changed note above for the source-breaking effect the four new Drop implementations have on library consumers.

Library API

Changed

  • Source-breaking for consumers of the library crate: four public types now implement Drop, so their fields can no longer be moved out. Identity, ResolvedIdentity, IdentityConfig and HandshakeState each gained one as part of clearing key material at end of scope. IdentityConfig is the one most likely to be reached in practice, since it hangs off the public Config as node.identity, so code that moved the nsec out of a configuration value no longer compiles and needs Option::take instead. Nothing about the behaviour of the shipped binaries changes; this affects only callers using fips as a library.

Supply chain

Security

  • The dependency lockfile is refreshed past a set of advisories against the pinned nostr 0.44.3 and nostr-relay-pool 0.44.1, both of which were also yanked. nostr moves to 0.44.8 and nostr-relay-pool to 0.44.3; the requirements in Cargo.toml already admitted both, so this is a lockfile change and no code changed with it. The advisories that matter here are the relay-pool ones, RUSTSEC-2026-0224 and RUSTSEC-2026-0232, which describe forged events bypassing signature validation and unverified relay events being processed: that is the path this node learns peer adverts on, and it performs no independent verification of its own, so the exposure was a misattributed advert rather than the denial of service the advisory summaries lead with. RUSTSEC-2026-0231 (auth-challenge memory exhaustion) is on the same path, and RUSTSEC-2026-0216 and RUSTSEC-2026-0227 reach the NIP-44 decryption of relay-supplied content. The remaining advisories in that set cover NIP-04, NIP-46, NIP-50, NIP-60, NIP-98 and the wallet parsers, none of which this code calls. The refresh was taken over the whole lockfile rather than the two crates alone, which additionally clears RUSTSEC-2026-0204 in crossbeam-epoch and leaves no yanked crate in the tree; cargo audit now reports no vulnerability, against twelve before. Four warnings remain and are not fixable by a version move: instant and paste are unmaintained, lru 0.16.4 carries an unsoundness advisory, and nostr-relay-pool itself is now marked unmaintained.

  • Every GitHub Action is pinned to a commit SHA, and the OpenWrt packaging workflow verifies both of the artifacts it downloads. No reference in the repository was pinned before: all sixty-six named a mutable tag and one named a branch, including the jobs holding the AUR deploy key, the jobs with release write scope, and the packaging jobs that run with a signing key in the environment. Sixty-two are now full commit SHAs with the original tag retained as a trailing comment. Four are left unpinned and justified in one place: two actions read the tool to install from the ref name itself, so a SHA would hand them a hex string where a toolchain name belongs. A guard enforces the form on every sweep, treats an unreadable tree as an error rather than a pass, and documents what it does not cover. The sharper hole was not the tags: the OpenWrt workflow fetched a helper binary from a release URL with no verification at all, in two jobs holding a signing key. That download now checks a per-architecture pinned SHA-256, with the hash provenance recorded honestly, upstream publishing no checksum document.

    The same workflow's Zig toolchain fetch is verified the same way. It ran as curl | tar, which leaves nowhere to check the bytes, so a short read reached tar as a truncated archive and failed the build with "Unexpected EOF in archive"; curl's --retry does not cover that exit. The download now stages to a temporary directory, checks a per-architecture pinned SHA-256 with a guard that fails if an architecture is added without one, and only then extracts, with three attempts at 10s and 20s backoff and an early exit when two attempts return identical bytes, since a stable mismatch is a wrong pin rather than a bad transfer. As with the helper binary, the hashes come from the upstream download index and were checked against the tarball bytes: that is integrity, not authenticity, because upstream publishes no detached sums.

Packaging, install and platform layout

Fixed

  • macOS: peers.allow, peers.deny, and the hosts file are now read from /usr/local/etc/fips/, matching the install layout the macOS packaging ships (packaging/macos/). That layout is what the three fixes in this group align the daemon and fipsctl to. The default-path constants were hardcoded to /etc/fips/... with only a #[cfg(unix)] / #[cfg(windows)] split, so on macOS the daemon looked in a directory that does not exist: load_file / load_hosts_file hit their NotFound no-op arm and silently returned an empty ACL / empty host map. A populated peers.deny therefore reported effective_mode: "default_open" and enforcement_active: false via fipsctl acl show, and host-file aliases went unloaded, with no error or warning. The default constants now follow the platform's packaging (/usr/local/etc/fips/ on macOS, /etc/fips/ on Linux and other Unix for the ACL files, and /etc/fips/ on Linux and %ProgramData%\fips\ on Windows for the hosts file) and are pinned by platform-gated unit tests so the layout cannot silently drift again. At startup the daemon warns once if any of these files exist at the old /etc/fips/ location but not at the current default. Linux and Windows behavior is unchanged. Contributed by @sh1ftred. macOS users with existing files in /etc/fips/ should move them to /usr/local/etc/fips/.

  • macOS: fipsctl keygen now writes fips.key / fips.pub to /usr/local/etc/fips/ by default. The default output directory was hardcoded to /etc/fips for all Unix, but the daemon derives its identity key paths from the config file's directory, which is /usr/local/etc/fips/fips.yaml on macOS, so a generated identity landed where the daemon never reads it and the node silently kept an ephemeral identity. Linux and other Unix keep /etc/fips, Windows is unchanged, and the values are pinned by platform-gated unit tests.

  • macOS: the system-wide config search path now includes /usr/local/etc/fips/fips.yaml in addition to /etc/fips/fips.yaml. Previously only /etc/fips/fips.yaml was probed, so a bare fips run without --config skipped the installed config and derived identity key paths from a non-existent directory. /etc/fips/fips.yaml is still probed first so existing installs keep working. Both the macOS entry in the search path and the directory fipsctl keygen writes to read the shared SYSTEM_CONFIG_DIR constant, so the two cannot drift apart. The launchd-installed daemon was unaffected (it always passes --config). Linux and Windows behavior is unchanged. Because the daemon derives the identity key directory from whichever config file loaded last, a macOS host carrying fips.yaml at both locations would have resolved fips.key to the new directory, found none, and under persistent generated a fresh identity, silently changing its npub, routing address and mesh IPv6. The daemon now adopts a key stranded at /etc/fips/fips.key and warns to move it, instead of generating one. The fallback is confined to keys resolved from the system config directory, so a run using ./fips.yaml or a user config is never redirected to a system key.

  • The maintainer address published in package metadata no longer bounces. The crate authors field, the Debian package maintainer and upstream contact, and both AUR PKGBUILD maintainer lines carried an address that no longer accepts mail, so the contact of record in every artifact we ship was unreachable.

Docs, CI and contributor tooling

Added

  • SECURITY.md, stating a private channel for vulnerability reports, what a useful report contains, what a reporter can expect back and on what timing, and which branches receive fixes. The repository previously documented no reporting channel at all, so someone with a finding had to guess at an address or open a public issue.

Changed

  • Two CI runs on one machine can no longer collide. Every suite derives its own docker build context, image tag, container names, network range and host interface names per run, so concurrent runs cannot reap each other's containers or contend for a fixed subnet. This is what a contributor running testing/ci-local.sh alongside a GitHub run, or two local runs at once, sees change: the runs stay independent instead of one killing the other.

  • A test that does not run, or whose result cannot be read, no longer passes silently. A failed scenario now fails the run rather than being logged and stepped over, a node whose logs cannot be read no longer counts as clean, an unanswered control query no longer reads as zero, an unknown scenario key is rejected instead of matching nothing, and a skipped check appears in the final verdict rather than only in scrollback.

  • New guards run in both the local and GitHub runners, so the two gates agree. They check that trailing-log call sites are wired, that the log strings the harness matches on are still emitted by the daemon, that the two runners' integration-suite sets match per leg rather than as a folded token, that every GitHub Action reference is pinned in the required form, and that source comments do not cite references a reader of the published tree cannot resolve.

  • Coverage moved from Docker to deterministic in-process tests, and dead scenarios were retired. The six cost-selection chaos scenarios, the admission-cap and acl-allowlist Docker suites, the smoke-10 scenario, the tcp-chain and mesh-public static topologies, and three ignored Ethernet tests are gone, with their behaviour asserted in unit and integration tests instead. A local CI run is correspondingly shorter and less dependent on container timing.

  • The bloom-storm chaos scenario no longer runs on either the local or the cloud runner. Unlike the retirements above it has no replacement: the scenario files remain in the tree and it stays runnable by hand, but nothing now exercises downstream containment of a mid-chain ancestor swap on a schedule. This is recorded as a coverage gap rather than as a completed migration.

  • A failing harness now says why it failed. The dns-resolver suite sent build and container-start output to /dev/null, so a failed scenario reported that it had failed and nothing else; output is now captured and emitted on failure, naming the command, and the systemd readiness wait dumps container state, failed units and the journal when it gives up. The NAT-lab path assertions exited bare, printing neither what they expected nor what they saw and triggering none of the scenario diagnostics their siblings already call; all twelve call sites now report the container, the expectation, the observation and a projection of the peer or link table, and distinguish a failed control-socket exec from unparseable output from a genuine mismatch. The convergence gate could not tell a tree that did not converge from connectivity that failed, and could exit non-zero while reporting "20 passed, 0 failed"; it now records the outcome, the count reached and the count pending, and its failure messages name the condition. A passing run is as quiet as before, and no timing, threshold or control-flow behaviour changed in any of the three.

  • A dns-resolver scenario no longer burns the full 30-second boot timeout and warns about a container that booted correctly. The readiness poll ran under pipefail and piped systemctl is-system-running into grep, and that command exits non-zero when the system is degraded, which is where systemd inside a container always settles; the pipeline therefore failed even when the pattern matched, leaving the degraded branch dead. The poll now matches on the captured state instead of piping into grep.

  • The chaos harness now checks that teardown and node stops did what they report. docker compose down exits 0 while leaving a run's containers alive, so a partly-failed bring-up leaked named containers with nothing to detect it; teardown now asks whether the containers this run owns are gone, treats a survivor that forced removal clears as a warning, and aborts with the names written to an artifact when one survives that or the query cannot run at all. The check is scoped to a run's own names, so concurrent runs cannot trip each other. Node churn separately marked a node down whether or not docker stop succeeded, so the simulation's model of the mesh diverged from reality, and nodes_down, the max_down_nodes cap and the connectivity guard are all computed from that model; a failed stop now warns, carries the daemon's own message, and leaves the node out of the down set for the next churn tick to retry against an honest model.

  • Comments throughout the source tree, the packaging files and the test scripts no longer cite internal identifiers, planning documents or private stage names that a reader of the published tree cannot resolve; each now states the thing the citation stood for. A handful of comments that described behaviour the code does not have (the control-plane read path, its snapshot dispatch, and the MMP report types) have been corrected rather than merely reworded. One of the edited files, the DNS setup helper, installs to /usr/lib/fips on every packaging path, so its comment reached users. No code changed.

Security

  • The security reference now records that both Noise patterns deviate from the standard construction in one respect: the handshake AEAD passes an empty associated-data field where standard Noise EncryptAndHash uses the handshake hash h. The published tables named Noise_IK_secp256k1_ChaChaPoly_SHA256 and Noise_XK_secp256k1_ChaChaPoly_SHA256 unqualified, so anyone auditing the stack against the Noise specification had nothing telling them where to look. Domain separation and Diffie-Hellman binding survive through the chaining key, which is seeded from the protocol name and chained at every step; transcript binding is the property actually absent. Nothing in the daemon reads the handshake hash, so no shipped behaviour rests on it, but the comments that called it transcript binding or channel binding overstated it and now describe what the field is, and the field records that anything later built on it (channel binding, an exporter, cookie binding) will silently not work until the associated data carries h. This is a correction to what is documented and claimed; no code behaviour and nothing on the wire changed.

[0.4.1] - 2026-07-19

Changed

  • node.bloom.max_inbound_fpr default raised from 0.10 to 0.20. The cap rejects inbound FilterAnnounce whose FPR (fill^k) exceeds it. On the fixed 1 KB / k=5 filter, 0.10 corresponds to fill 0.631 (~1,630 reachable entries), and the busiest nodes' aggregates had again begun to reach it as the mesh grew. 0.20 (fill 0.7248, ~2,114 entries) restores headroom without materially weakening the antipoison gate: a saturated or poisoned filter is ~100% FPR and still rejected. This is the second raise of this cap in two releases; the fixed 1 KB filter is the underlying constraint, and the structural remedy is the v2 filter work rather than a further raise. A node running this default accepts announcements that a v0.4.0 node drops, so during a rolling upgrade the two versions can disagree about mesh size.
  • Bloom filter probing computes its SHA-256 digest once per operation rather than once per hash function. All k indices were already derived from a single digest, but the digest was recomputed inside the per-function loop, so every insert and membership test hashed the same bytes hash_count times (5x at the default). Output is bit-for-bit identical; this is the hottest path in packet forwarding and mesh-size estimation.
  • Identity operations reuse one shared secp256k1 context instead of constructing a fresh one at every sign, verify, and key-derive site. Each construction allocated a context and ran randomization and blinding table setup. Behavior is unchanged: the same API calls are made, only the context lifetime differs, and the shared context still performs the standard construction-time blinding.

Fixed

  • Spanning tree: the coordinate cache is now invalidated when the parent link is lost through peer removal. That path reparents or self-roots the node but omitted the invalidation every other position-change path performs, so cached entries for downstream destinations kept the node's now-stale coordinate prefix. Because routing access refreshes an entry's TTL, an actively routed stale entry never self-expired and was corrected only by a fresh insert.
  • Discovery: applying a LookupResponse now keeps the tighter of the cached and received path_mtu rather than overwriting unconditionally. A looser estimate arriving in a later response could clobber a tighter value already learned from a reactive MtuExceeded or PathMtuNotification, loosening a clamp that had been correctly tightened.

Removed

  • The parent_switched spanning-tree metric counter. It was incremented on the line immediately before parent_switches at every site and never independently, so the two were always identical. parent_switches remains as the sole counter. Consumers reading parent_switched from the control socket or fipstop should use parent_switches.

[0.4.0] - 2026-06-27

Added

Transports (Nym, mDNS LAN discovery)

  • Nym mixnet transport (transports.nym) for outbound peer links tunneled through a local nym-socks5-client SOCKS5 proxy into the Nym mixnet, as a privacy transport alongside Tor. Outbound-only and not platform-gated, it reuses the existing FMP framing and adds no new crate dependencies. A single-container example (examples/sidecar-nostr-mixnet-relay/) demonstrates FIPS peering across the mixnet end to end.
  • Opt-in mDNS / DNS-SD LAN discovery for sub-second pairing of peers on the same local link, without a relay or NAT-traversal roundtrip. Disabled by default; operators enable it with node.discovery.lan.enabled: true. Configurable service type and an optional node.discovery.lan.scope that isolates discovery to peers sharing the same private-network scope. The advertised UDP port is chosen from a non-bootstrap operational UDP transport using a stable selector, so it is deterministic across restarts.

Admission / peer-list management

  • Node::update_peers for runtime peer-list refresh, returning an UpdatePeersOutcome summarizing added, removed, and retained peers. Re-derives active peer connections from a new peer configuration without dropping links to peers that remain in the set. PeerAddress gains a seen_at_ms recency field (with with_seen_at_ms) used to prefer more recently observed addresses.

Data-plane / metrics / observability

  • Typed RejectReason classification for receive-path silent-rejection sites across the node. Each rejection-and-return path now passes a typed reason to NodeStats::record_reject, which routes it to a per-subsystem counter, so operators can see what is being rejected through stats counters rather than by scraping debug logs. New HandshakeStats, SessionStats, and MmpStats sub-stats join the existing TreeStats, BloomStats, DiscoveryStats, and ForwardingStats, and TreeStats::ancestry_invalid is now incremented from the TreeAnnounce::validate_semantics rejection site that was previously silent. Several handshake, MMP, tree, and discovery rejection paths that had no counter at all are now counted, including the send_lookup_response no-route drop (DiscoveryStats::resp_no_route).
  • Internal atomic metric registry (Arc<MetricsRegistry>) that shadows the plain-u64 NodeStats counters, written alongside them and validated by a whole-struct debug-build parity check. Covers the forwarding receive counters, the full discovery counter family, and the tree, bloom, congestion, and error-signal counter families, with the hottest counters cache-line padded. Behavior-neutral: NodeStats remains the serving path. Groundwork for sampling metrics without contending the receive loop.
  • fipsctl stats metrics, backed by a new counter-only show_metrics control query that dumps the atomic metric registry as flat counter name/value pairs. Serves a Prometheus-style scraper that samples node counters without contending the receive loop.
  • pool_inbound and pool_outbound counters on the TCP and Tor transport stats (TcpStats, TorStats). Per-direction accounting is updated at every pool-insert and receive-loop-exit site, plus on transport stop and on send-failure-driven removal. Surfaces through TcpStatsSnapshot and TorStatsSnapshot for show_transports.

Spanning-tree / mesh-size / routing

  • Six route-class transit counters that partition transit-forwarded packets by their tree relationship to the chosen next hop: tree-up (peer is our ancestor), tree-down (peer is our descendant and the destination is within its subtree), tree-down-cross (peer is our descendant but the destination is outside its subtree), cross-link descend (lateral peer, destination within its subtree), cross-link ascend (lateral peer, destination outside its subtree), and direct-peer. The six classes sum to forwarded_packets (asserted by a unit test) and are computed from tree coordinates at the transit chokepoint, so error-signal routing callers are excluded. They surface through the forwarding stats snapshot via show_routing and show_status.
  • Discovery now counts LookupRequests dropped when the dedup cache is full. A saturated recent_requests cache (MAX_RECENT_DISCOVERY_REQUESTS) previously dropped requests silently; a new DiscoveryStats::req_dedup_cache_full counter (typed reject reason DiscoveryReject::ReqDedupCacheFull) makes the drop visible through show_routing.

Packaging & deployment

  • OpenWrt .apk packaging (packaging/openwrt-apk/, make apk) for OpenWrt 25+, where apk-tools is the mandatory package manager (the existing .ipk continues to cover OpenWrt 24.x and earlier). Built SDK-free: it reuses the .ipk cross-compile (cargo-zigbuild) and the shared installed-filesystem payload, and assembles the package with apk mkpkg from apk-tools 3.0.5 built from source — no OpenWrt SDK image. A build-apk CI job (aarch64, x86_64) builds and structurally verifies the package; releases now publish .apk artifacts and checksums alongside .ipk. Packages are unsigned, installed with apk add --allow-untrusted, matching the .ipk posture.
  • Nix flake (flake.nix at the project root) for reproducible from-source builds on Nix/NixOS. Builds all four binaries (fips, fipsctl, fips-gateway, fipstop), pins the exact toolchain from rust-toolchain.toml via fenix, and wires the build-time native dependencies (libclang for bindgen, plus dbus and pkg-config), so it needs no host setup beyond Nix with flakes enabled. Flake inputs are lock-pinned (flake.lock committed) for reproducibility, and the flake exposes nix build, nix run, a nix develop dev shell with the pinned toolchain, and nix flake check. The flake produces binaries (and a NixOS packages.<system>.fips output); the systemd/service integration that the .deb/tarball installers provide is handled through the NixOS configuration instead.

Docs & contributor tooling

  • PR-REVIEW.md — the 13-criteria PR review checklist the maintainer runs against every incoming PR, published at the repo root so contributors can run the same pass on their own change (directly or by handing the document to a coding agent) before opening. Linked from CONTRIBUTING.md under "Submitting pull requests" and "Further reading". Running the checklist before opening surfaces problems that would otherwise come back as review comments, saving a round trip.
  • docs/how-to/tune-file-descriptors.md — an operator how-to for raising RLIMIT_NOFILE. A busy node opens roughly three file descriptors per established UDP peer (a connect()-ed socket plus a 2-FD drain self-pipe), so the default 1024 soft limit is exhausted near 320 peers, after which further admission, handshakes, and discovery fail with EMFILE. The guide documents the per-peer FD budget and symptom, the systemd (LimitNOFILE drop-in) and OpenWrt (procd nofile) procedures to raise the limit, and how to verify the per-peer ratio stays bounded. Linked from the how-to index.

Changed

FMP/FSP rekey reliability

  • complete_rekey_msg2 now returns the remote peer's startup epoch alongside the new Noise session, so the rekey path can detect a peer restart and clear stale session state.

NAT traversal / Nostr discovery

  • Nostr discovery startup is now non-blocking. Node::start no longer waits for relay connect, subscribe, or initial advert publish before returning. A slow or unreachable relay no longer holds node startup hostage; local transports come up immediately and the relay path catches up asynchronously in background tasks. Subscribe retries with exponential backoff (2 s base, 60 s cap), publish attempts time out at 10 s, and the new tasks are aborted cleanly on Node::stop.

Spanning-tree / mesh-size / routing

  • Active-peer path selection now sorts address candidates by recency (seen_at_ms), preferring the most recently observed address when racing concurrent path probes.
  • Per-tick work budgets bound the connection churn done in a single node tick: MAX_DISCOVERY_CONNECTS_PER_TICK, MAX_RETRY_CONNECTIONS_PER_TICK, and MAX_PARALLEL_PATH_CANDIDATES_PER_PEER. Work beyond a tick's budget is deferred to the next tick rather than discarded.

Admission / peer caps

  • node.bloom.max_inbound_fpr default raised from 0.05 to 0.10. The cap rejects inbound FilterAnnounce whose FPR (fill^k) exceeds it. On the fixed 1 KB / k=5 filter, 0.05 corresponds to fill 0.549 (~1,300 reachable entries) and had begun rejecting the busiest nodes' aggregates as the mesh approached that size. 0.10 (fill 0.631, ~1,630 entries) restores headroom toward the fixed-filter capacity limit without materially weakening the antipoison gate: a saturated or poisoned filter is ~100% FPR and still rejected.
  • TCP inbound connection cap now honors node.limits.max_connections. The per-transport TCP inbound accept ceiling was hardwired to 256 and never read max_connections, so raising it was a silent no-op for inbound TCP. The effective cap now resolves with precedence: explicit per-transport max_inbound_connections, then node-wide max_connections, then the built-in default of 256. Established peers remain bounded node-wide by add_connection.

Data-plane / worker-pool / metrics / observability

  • The control-socket read surface is now served off the rx_loop. Every pure-read show_* query — show_status, the show_stats_* family, show_listening_sockets, the new show_metrics, show_tree/show_bloom/show_cache/show_routing/ show_identity_cache, show_peers/show_sessions/show_links/ show_connections/show_transports/show_mmp, and show_acl — now renders in the control accept task from ArcSwap-published read snapshots instead of round-tripping the data-plane receive loop; only the mutating connect/disconnect commands still reach the loop. This removes the head-of-line coupling where a busy or slow rx_loop could time out fipsctl and fipstop observability (the five-second query pattern operators saw on loaded nodes). Per-entity snapshots reuse unchanged rows by pointer, so per-tick publish cost stays bounded as peer/session count grows. New daemon-resolved fields surface through the snapshots: effective persistence, root/is-root, and a per-transport-type peer-count map in show_status; per-peer effective_depth in show_peers; root_npub in show_tree; and the last-sent uptree filter fill ratio and subtree estimate in show_bloom.
  • fipstop TUI overhaul: reworked rendering, navigation model, and the control read surface it draws from, surfacing the new daemon-resolved snapshot fields above. Built on a ratatui TestBackend render-snapshot harness that asserts the text grid and per-cell style of every ui::draw_* against canned show_* JSON.
  • Steady-state log noise reduced on saturated public-mesh nodes. Routine per-peer connection-lifecycle and capacity-cap events are demoted from info/warn to debug — FMP K-bit cutover promotion, connection-promoted-to-active-peer (a redundant duplicate promotion line removed), peer-restart-detected, peer-removed-and- cleaned-up, the TCP max_inbound_connections-reached rejection, and the congestion-CE-flag line — so genuinely notable info/warn lines are no longer drowned out. An exhausted FMP-msg1 / FSP-msg3 rekey retransmission-budget abort (an expected, self-limiting outcome on lossy or high-latency links) is likewise demoted from warn to debug.
  • macOS UDP receive path now batches up to 32 datagrams per kernel wakeup via recvmsg_x(2), matching the Linux recvmmsg(2) amortization shape introduced in v0.3.0. Previously macOS fell through to single-packet recv_from, capping inbound rate on Apple builds with the same per-syscall + per-task-wakeup overhead Linux had already eliminated. recvmsg_x is an xnu-private syscall declared via unsafe extern "C" against a local repr(C) msghdr_x; same approach used by quinn-udp. Same (count, kernel_drops) contract as the Linux path, with kernel_drops always 0 on macOS (no SO_RXQ_OVFL equivalent). Bench numbers on aarch64-apple-darwin (100B payloads, 3 s windows): 1 sender 1.09x, 2 senders 1.72x, 4 senders 1.56x, 8 senders 1.46x.
  • Receive hot path: removed two per-packet copies. New borrowed SessionDatagramRef decoder is used in the forwarding handler so local delivery and coordinate-cache warming no longer allocate or copy the session payload; the owned SessionDatagram is materialized only when re-encoding for the next hop. Owned SessionDatagram:: decode is reimplemented as Ref::decode + into_owned, so the two decoders cannot drift. On Linux + macOS the recvmmsg / recvmsg_x receive loop now moves each filled slot buffer into ReceivedPacket via mem::replace instead of cloning it, and TransportAddr is formatted directly from the SocketAddr without an intermediate String. Focused decode bench: ref 1.6 ns/op vs owned 34.7 ns/op (21.4x).
  • Quieted non-Linux test-build warnings from intentionally platform-specific code: the nftables firewall parser (#[allow(dead_code)] now gated to non-Linux targets where the parser is compiled but unused), the macOS utun address-family helper and the long TUN reader entry point (narrow allowances), and a macOS Ethernet test module's clippy struct-layout lint (rewritten MAC-copy loop, explicit layout annotation). No behavioral change; the goal is to keep cargo test and cargo clippy clean on cross-platform builds so unrelated warning fixes don't get bundled into behavioral PRs.
  • Data-plane: AEAD encrypt and AEAD decrypt now run on per-shard worker-pool threads (std::thread + crossbeam_channel), off the rx_loop. Hash-by-destination dispatch pins each TCP flow to one worker so wire ordering is preserved; per-worker sendmmsg(2) batches up to 32 outbound packets per syscall, with UDP_GSO (UDP_SEGMENT) when the batch is uniform-sized — the same kernel primitive WireGuard's in-kernel module and Cloudflare's userspace BoringTun use to hit multi-Gbps single-stream rates. On Linux + macOS each established UDP peer also gets a dedicated connect(2)- ed kernel socket bound to the same wildcard listen port via SO_REUSEPORT, so the kernel caches per-packet route + neighbor lookup and the worker sends with msg_name = NULL. The receive side mirrors: per-shard thread-local HashMap owns each session's recv cipher + replay window, replacing the previous shared RwLock. Sessions are re-registered with the decrypt pool on K-bit flip and rekey cutover, and unregistered on rekey drain completion and peer removal so the per-shard tables stay bounded. New crossbeam-channel = "0.5" dependency. Worker counts default to num_cpus; both pools are overridable via FIPS_ENCRYPT_WORKERS and FIPS_DECRYPT_WORKERS (the latter accepts 0 to disable the pool and fall back to in-line decrypt in rx_loop). Per-peer connected UDP can be disabled via FIPS_CONNECTED_UDP=0. Optional per-stage timing reporter available via FIPS_PERF=1 (or FIPS_PIPELINE_TRACE=1); detailed knob documentation is a follow-up at docs/how-to/tune-worker-pools.md. Bench (5 × 15 s × 1 stream medians, Linux x86_64, docker-bridge mesh): A→D 1379→2708 Mbps (1.96×), A→E 1394→2663 Mbps (1.91×), E→A 1406→2624 Mbps (1.87×); RTT +0.11–0.19 ms from the worker queue handoff. Windows continues on the existing tokio-based send/recv path. Two issues in the off-rx_loop drain path are resolved as part of the overhaul: the per-peer drain worker is now detached on Drop rather than joined synchronously (a synchronous join from the runtime thread could wedge the whole daemon when a peer was removed with an in-flight worker), and the connected-UDP drain no longer busy-spins on a poll error (#106).

Transports & config

  • Static host aliases in /etc/fips/hosts now hot-reload on mtime change instead of only at daemon startup, so fipsctl/fipstop display names reflect edits without a restart. The peer ACL and host map both reload once per node tick through a new lock-free Reloadable snapshot.
  • Sidecar example (examples/sidecar-nostr-relay): udp.mtu is now overridable via the FIPS_UDP_MTU environment variable, defaulting to 1472 (preserving prior behavior). Plumbed through docker-compose.yml and documented in the README env-var table. Annotated the static-CI node template mtu: 1472 literal with the same Docker-bridge rationale and a pointer at the daemon's 1280 default.

Packaging & deployment

  • The Debian package no longer ships /etc/fips/fips.yaml as a dpkg conf-file. The default configuration is installed as an example at /usr/share/fips/fips.yaml.example, and postinst seeds /etc/fips/fips.yaml (mode 600) from it only when the file does not already exist — so a configuration-management-rendered or operator-edited config is never prompted for or clobbered on upgrade, removing the need for a dpkg-divert workaround. fips.service gains ConditionPathExists=/etc/fips/fips.yaml. The example is placed under /usr/share/fips, deliberately outside /usr/share/doc, which minimal and container installs path-exclude (so the install-time seed source is never dropped).
  • openwrt: the .apk package now defaults ethernet.wan to the OpenWrt 25 DSA port name wan; the .ipk package keeps eth0 for OpenWrt 24 and earlier.

CI & test-harness reliability

  • CI and release-publish workflows hardened:
    • ci.yml declares a top-level concurrency block keyed on (workflow, ref) with cancel-in-progress: true. Force-pushes and rapid successive pushes to the same ref now retire any in-flight run rather than letting superseded and current-tip runs both burn runner minutes.
    • aur-publish.yml rewritten to fetch the upstream source tarball and compute its b2sum in CI, then patch pkgver and the b2sums SKIP placeholder in PKGBUILD in-place. Previously updpkgsums: true downloaded the tarball into the AUR working tree, where it was rejected by AUR's 488 KiB max-blob hook — silently no-op'ing the v0.3.0 stable AUR push. fips.sysusers / fips.tmpfiles asset b2sums are recomputed in the same step to stay in sync with the local files. workflow_dispatch gains a tag input so historical release tags can be re-published manually, and continue-on-error: true is dropped so future regressions surface in CI.
    • New aur-publish-git.yml workflow for the fips-git VCS PKGBUILD, triggered on master pushes touching PKGBUILD-git or companion files plus workflow_dispatch. pkgver is computed at build time by the PKGBUILD's pkgver() function, so this workflow is not tied to release tags.
    • Tag-triggered package-* release-build workflows remain untouched.
  • Local and GitHub CI integration coverage brought into parity, and the Rust toolchain selection given a single source of truth:
    • The admission-cap integration suite, previously run only by ci-local.sh, now also runs as a GitHub ci.yml matrix leg, so a regression in it turns the GitHub gate red rather than depending on a developer remembering to run local CI. A new testing/check-ci-parity.sh (wired as ci-local.sh --check-parity) diffs the two runners' integration-suite sets and fails on unexpected drift; the deliberate local-only (live-Tor) and granularity-only differences are documented in a comment block atop both runners.
    • CI and packaging jobs now select the toolchain with actions-rust-lang/setup-rust-toolchain (which reads rust-toolchain.toml) instead of dtolnay/rust-toolchain@stable. The pinned channel already overrode the installed stable, so each job downloaded an unused toolchain and logged a misleading rustc version; the single-source action removes the waste and the confusion. Existing cache steps are kept (cache: false on the new action) and RUSTFLAGS is left untouched so no global -D warnings is newly imposed. The OpenWrt nightly Tier-3 leg keeps @nightly.

Docs & contributor tooling

  • Overhauled CONTRIBUTING.md: replaced generic Rust-template framing with a FIPS-specific entry point covering the four-layer architecture, branch model and PR-target selection, structured bug reporting, scope discipline and local-CI requirements, an AI coding assistant policy, and project communication channels. Added docs/branching.md as the long-form companion covering the release workflow, version conventions, and merge-direction rationale.

Fixed

FMP/FSP rekey reliability

  • FMP link-layer rekey is now reliable under packet loss, bringing it up to the FSP session layer's rekey discipline. The rekey msg1 retransmission driver was previously uncapped and never abandoned, so a rekey that never completed resent msg1 forever; it now uses a bounded retransmission budget (handshake_max_resends with exponential backoff) and abandons the rekey cycle cleanly once the budget is exhausted, mirroring the FSP rekey msg3 driver. With the cap in place the link-dead heartbeat is rekey-aware: check_link_heartbeats no longer reaps a link that is still actively carrying rekey-handshake traffic, while a genuinely dead link is still reaped once the budget abandons. At the K-bit cutover the receiver now authenticates an inbound frame against the pending session before promoting it, instead of promoting on the bare header K-bit; under jitter a node could otherwise promote a stale pending session, leaving the two endpoints on different keys and silently dropping traffic until the link died — the same failure class already closed on FSP, now closed on FMP.
  • FSP session rekey is now hitless under packet loss and reordering. Previously, a rekey could leave the two endpoints holding different key sets for a brief window — if a handshake message was lost in transit one side rotated keys while the other did not, and traffic sealed in one key epoch reached a peer still on the other epoch and failed to decrypt, producing bursts of AEAD decryption failures and dropped connectivity until a later rekey reconverged the pair. The receive path now trial-decrypts each frame against every live key epoch (current, pending, and the draining previous session) for the duration of the rekey transition, so no rotation ordering and no packet reordering can cause a decryption failure. The previous-epoch slot is retained as long as the peer keeps using it, with its drain deadline anchored on the last frame the peer authenticates against it rather than a fixed wall-clock timer, so a peer that did not receive the new keys is not stranded by a silent permanent decrypt failure. The lost-handshake case is closed by retransmitting the third rekey handshake message until the peer is confirmed on the new keys, with a bounded retry budget after which the rekey cycle is cleanly abandoned and retried. There are no FSP decryption failures across a rekey under lossy, jittery links.
  • ±15s symmetric jitter is applied per session to the FMP and FSP rekey timer trigger, eliminating the steady-state dual-initiation race in symmetric-start meshes (previously the smaller-NodeAddr tie-breaker resolved correctness only after every cycle's collision). node.rekey.after_secs becomes the nominal interval rather than a floor; the mean is preserved.
  • A stale FSP (session-layer) session is now cleared when a peer restart is detected during FMP rekey or cross-connection promotion. Previously the old session could linger after the peer came back with a new startup epoch, leaving the session-layer map out of sync with the freshly promoted peer.

NAT traversal / Nostr discovery

  • Two nodes that each auto_connect to the other no longer stall their Nostr-mediated NAT-traversal handshake. Each side ran both an initiator and a responder traversal session, binding a separate UDP socket per session, and adopted only the first Established event; if the two sides adopted mismatched sessions, each sent its Noise msg1 to a peer port the peer had already stopped draining and both handshakes hung until the adoption budget expired. The responder now elects a single session deterministically — it declines an incoming offer only when it also has an in-flight outbound initiator for the same peer and its own NodeAddr is smaller — so one matching socket pair survives on both ends and the peer's redundant initiator times out harmlessly. One-sided (asymmetric) auto_connect has no co-active initiator and is never suppressed, so connectivity is preserved.
  • NAT-traversal cross-init adoption is now deterministic under simultaneous dual-initiation. Previously, when two peers' Nostr-mediated UDP punches completed within the same scheduling window, each side's bootstrap-completion event arrived with an in-flight handshake already recorded against the other peer (each side had received an inbound msg1 from the other's pre-punch outbound attempt). The deduplication skip then fired on both sides, neither installed the fresh traversal socket as canonical, and the 45-second peer-adoption budget expired with both nodes stuck waiting for an adoption that never happened. The handler now applies the same deterministic NodeAddr tie-breaker the codebase already uses for rekey dual-initiation and cross-connection resolution: the smaller NodeAddr wins as adopter, tears down its in-flight handshake state, and proceeds with adoption; the larger NodeAddr keeps the skip semantics, and its in-flight outbound is reconciled by the cross-connection logic when the winner's fresh msg1 arrives over the adopted socket. The dual cross-init stall is eliminated; cross-init NAT-traversal completes in well under a second even under host CPU contention.
  • Nostr-discovered NAT-traversal events (BootstrapEvent::Established and BootstrapEvent::Failed) for peers that are already connected or actively handshaking are now short-circuited at the poll_nostr_discovery dispatch sites before any cooldown bookkeeping or fallback retry scheduling runs. Stale Failed events previously poisoned the per-peer failure-state cooldown of healthy peers and could trigger redundant retraversal attempts via schedule_retry / try_peer_addresses; stale Established handoffs could attempt to adopt a second socket against a live connection. A defense-in-depth guard was added to adopt_established_traversal so the same invariant holds if a future caller bypasses the outer dispatch check. As a side benefit, narrows a cooldown-poisoning vector previously available to an attacker injecting stale failure events for an active peer.
  • Nostr discovery now filters unroutable direct UDP/TCP advert endpoints. Publisher and validator retain only endpoints that parse as concrete socket addresses with routable IPs and nonzero ports; udp:nat rendezvous endpoints and Tor endpoints pass through unchanged. Adverts that collapse to zero usable endpoints after filtering are rejected with a clear "missing publicly routable endpoints" error. Before this change, misconfigured nodes could publish RFC1918, loopback, link-local, CGNAT 100.64/10, IPv6 ULA, or IPv6 link-local endpoints into Nostr discovery, and consumers would cache and dial them; in mixed LAN/VPN/NAT environments, that could prefer a misleading one-way private path over the intended udp:nat bootstrap.

Admission / peer caps

  • TCP and Tor max_inbound_connections admission cap is now compared against the per-direction inbound count (pool_inbound) rather than the combined pool size. Outbound connect-on-send connections share the same pool data structure but no longer consume slots against the operator-facing inbound cap. The configuration field name and operator semantics are preserved; only the cap-check comparison and accounting change. Operators with mixed outbound + inbound deployments no longer see legitimate inbound peers rejected once outbound connections fill the pool past the configured cap.
  • Outbound connection initiation now honors the node.limits.max_peers cap that was previously only checked on inbound msg1 admission. Four paths gated: auto-reconnect retries (process_pending_retries), Nostr-mediated discovery's BootstrapEvent::Established adoption, and both sides of the Nostr-mediated NAT-traversal punch (offer initiation in the runtime's outgoing path, offer acceptance in the responder's incoming-offer handler). At saturation, a node now performs zero outbound work on these paths; only existing peer maintenance and overlay-advert refresh continue. The inbound gate at handshake.rs:1114 is unchanged. Introduces a shared Node::outbound_admission_check() helper so the invariant is grep-able and unit-testable.
  • Inbound handle_msg1 now silent-drops at node.limits.max_peers saturation before building/sending Msg2, instead of replying with Msg2 and then rejecting at promote_connection. Adds an early cap check positioned after identity verification (so the reconnect / cross-connection bypass for known peers still fires) and before index allocation + Msg2 wire send. The late cap check inside promote_connection is intentionally retained as defense-in-depth. Wire savings observed in a 45 s tcpdump at saturation: ~3.6 cap-denials/s × Msg2 (~104 B + AEAD compute) each. Bigger win is cleaner peer-side semantics — no fake-completed handshake whose subsequent data frames fail decryption on this side.

Spanning-tree / mesh-size / routing

  • The mesh-size estimator (compute_mesh_size) no longer over-counts under filter overlap. It previously summed the per-filter cardinality of the parent and each child filter, which assumes the filters are perfectly disjoint; a stale or oversized parent filter or a routing loop inflated the reported mesh size to several times the true value, and dropping the parent on a tree rebalance collapsed the upward leg and flapped the count (the symptom operators saw as the size nearly-but-not-exactly doubling during rebalancing). The estimator now computes the cardinality of the OR-union over self plus every connected peer's inbound filter, dropping the parent/child tree gating entirely. OR is idempotent, so any overlap is deduplicated — the result equals the old sum in the disjoint case, stays correct under overlap, damps the parent-switch flap, and removes the estimate's dependence on tree-declaration cache freshness. The per-peer 500 ms rate-limiter and overall recompute cadence are unchanged.
  • Spanning-tree state distribution is now eventually-consistent. Previously every send_tree_announce_to_all call site fired only on a local state-change event (parent switch, self-root promotion, ancestry change, peer promotion, parent loss). Once a partition latched — for example, a parent-switch announce lost in transit via the brief cross-init handshake swap window where one peer's outbound session is about to become the loser session and the receiver has no matching decrypt-worker entry — no node's state changed again, so no node ever re-broadcast. The existing 60-second check_periodic_parent_reeval short-circuited silently on no-change (it was a re-evaluation, not a re-broadcast), and production-side healing depended on incidental link churn (NAT keepalive refresh, MMP timeout, peer re-promotion after a transport blip). The function now ends with an unconditional send_tree_announce_to_all on the no-change branch, alongside the existing switch and self-promote arms; receivers coalesce by sequence comparison (ParentDeclaration::is_fresher_than) and short-circuit at the if !updated gate in handle_tree_announce, so same-sequence repeats drop silently with no cascade. The per-peer 500 ms rate-limiter is well below this 60-second cadence and does not suppress the heartbeat broadcast. BASELINE_CONVERGENCE_TIMEOUT in testing/static/scripts/rekey-test.sh is bumped from 60 to 65 so any partition healed by the periodic broadcast at T+60 lands inside the convergence window; wait_for_full_baseline early-exits on PASS, so successful reps see no extra wall-clock.
  • A single-uplink node stranded out of the tree now re-attaches within a round-trip instead of waiting for the periodic re-broadcast cadence. A node with one tree peer has periodic parent re-evaluation disabled, so a lost one-shot attaching TreeAnnounce left it self-rooted and unreachable until the next periodic re-broadcast (reeval_interval_secs later). Tree-position exchange is now self-healing on the receive path: when an accepted TreeAnnounce advertises a root strictly worse (higher NodeAddr; election is smallest-wins) than our own, we echo our current declaration back to that peer, provoking the better-rooted peer to re-push its real position immediately. The echo fires only in that one direction and is bounded by the existing per-peer rate limiter.
  • Coord cache invalidation made surgical at parent-position-change and root-change sites. Replaces the previous unconditional CoordCache::clear() calls with two targeted methods: invalidate_via_node(node_addr) (drops entries whose cached ancestry contains the changed node, used at parent-switch / become-root / loop-detection sites) and invalidate_other_roots (drops entries from a different tree, used at root-change sites). The previous global flush left find_next_hop returning None for every non-direct-peer destination after every parent switch until the cache passively re-warmed; surgical invalidation preserves entries that remain correct across the topology change. Peer-removal retains the original "no invalidation" behavior (find_next_hop already recomputes against the current peer set every call, and Discovery handles "no route" on demand).
  • rx_loop tick-arm stall under convergence-phase mesh pressure is eliminated. Previously, the tick body's per-peer check_* loops (heartbeats, bloom announces, MMP reports, tree announces) called transport.send directly for every active peer. For TCP/Tor peers whose pool entry was not yet established, send_async fell through to a synchronous connect-on-send branch that wrapped TcpStream::connect in tokio::time::timeout(connect_timeout_ms, …) — 5 seconds by default — and blocked the entire tick body for the duration per unreachable peer. Under post-restart convergence on a high-peer mesh, this cascaded into multi-second tick stalls; the same mechanism also starved the master-only per-tick control-snapshot republish and pushed fipsctl show * queries onto an mpsc fallback that was itself queued behind the wedged rx_loop, producing the five-second fipsctl head-of-line pattern operators observed on loaded nodes. The send path now gates on transport.connection_state(addr) before sending: proceed only when Connected; on None, kick off a non-blocking background connect (idempotent — deduplicates against the connecting pool, spawns the timeout-bounded TcpStream::connect inside its own tokio task) and fail this send fast with a clear transport connection not ready error. A subsequent tick retries once the pool has an entry. The existing reconnect lifecycle (heartbeat-dead detection in check_link_heartbeats, scheduled retries via process_pending_retries, background- connect polling via poll_pending_connects) is unchanged. The connect-on-send branch in transport.send_async itself remains in place for code paths that legitimately need synchronous connect (e.g., explicit operator-driven fipsctl connect); the tick path just no longer trips it.

Data-plane / metrics / observability

  • The Tor transport now increments its connect_refused statistic (the "Refused" line in fipstop) when a SOCKS5 connection is actively refused, instead of recording every connect failure as a generic SOCKS5 error. The counter previously stayed at zero.
  • MMP sender metrics now ignore duplicate or regressed receiver reports before updating RTT, loss, goodput, or ETX. Receiver reports also suppress timestamp echo when dwell time overflows, so stale reports cannot inflate SRTT.
  • Reject-reason counters no longer double-count now that the rollout's interim direct increments are removed. Six discovery counters (req_decode_error, req_duplicate, req_ttl_exhausted, resp_decode_error, resp_identity_miss, resp_proof_failed), six bloom counters (decode_error, invalid, non_v1, unknown_peer, stale, fill_exceeded), and five forwarding reject packet counters (decode_error_packets, ttl_exhausted_packets, drop_no_route_packets, drop_mtu_exceeded_packets, drop_send_error_packets) were each incremented both by a direct bump and again through the typed reject dispatch. The redundant direct increments are removed — for the forwarding family the two calls are collapsed into a single byte-aware reject entry point — so each counter (and, for forwarding, its byte tally) counts once per event.
  • Transport-layer mutex poisoning no longer cascades. Ten Mutex::lock().unwrap() sites across the UDP, BLE, and Ethernet transports would turn a single panic (poisoning the mutex) into a cascade of panics on every subsequent lock. Each is replaced with lock().unwrap_or_else(|e| e.into_inner()), recovering the guarded data with no new dependency and no call-graph change; four local_addr.unwrap() calls on the UDP start/adopt paths get a provably-safe sentinel fallback. The critical sections are short, locally-scoped, and not reachable from peer input, so this is robustness hardening, not a remotely-triggerable fix.

Peer lifecycle / gateway

  • A manual fipsctl disconnect now notifies the peer so teardown is symmetric. Previously a manual disconnect tore down only the local side and sent the peer nothing, so the peer kept its session and never re-emitted its tree and filter announcements; on reconnect it was never re-adopted as a child and its bloom filter was never recorded. The local side now sends the disconnected peer a scoped Disconnect (the same message graceful shutdown sends), so both ends tear down and re-handshake cleanly on the next connection.
  • fips-gateway no longer drops long-lived or DNS-cached client mappings while traffic is still flowing. The virtual-IP pool's TTL clock advanced only on DNS re-query, never on traffic, and the mapping TTL is wired equal to the DNS TTL, so an in-use mapping was forced to drain at TTL and reclaimed at the first zero-conntrack tick — breaking long-lived, bursty, or DNS-cached clients. The tick now refreshes the mapping's last-referenced time whenever conntrack reports active sessions, and recovers a draining mapping to active (with a fresh grace window) when traffic resumes; only genuinely idle mappings drain.

macOS self-traffic / resolver

  • Self-addressed mesh traffic is now delivered locally on macOS instead of being dropped, for both ping6 and full TCP/UDP. The point-to-point utun interface egresses self-addressed traffic into the daemon, which previously pushed it onto the mesh outbound path where it was dropped for lack of a route to self; such packets are now hairpinned back to the TUN for inbound delivery. macOS first routes self-addressed packets as loopback (a LOCAL route via lo0), which leaves their transport TX checksum offloaded and unfinished, so re-injecting them verbatim made the local stack drop every segment whose checksum MSS clamping did not happen to rewrite (the SYN and SYN-ACK got through, but the bare ACK, data, and FIN were dropped, so connections to a node's own <npub>.fips service half-opened and hung). The hairpin path now recomputes the TCP/UDP checksum before re-injection, so full self-connections — not just ping6 — to a node's own <npub>.fips address work. Linux was unaffected (the kernel already loops self-traffic via lo). (#117)
  • macOS .fips name resolution now works on a fresh install: the shipped resolver shim points at ::1, matching the daemon's default IPv6 DNS listener, instead of 127.0.0.1. The mismatched shim (nameserver 127.0.0.1 while the daemon listens on ::1) broke getaddrinfo for .fips on every macOS install since the resolver was introduced.

CI & test-harness reliability

  • Node-level multi-node tests no longer flake under parallel CPU load. They previously delivered handshake packets over real localhost UDP, whose kernel receive buffer could overflow and drop a packet when many tests ran concurrently, panicking the large-network convergence tests. A cfg(test)-only loopback TransportHandle variant now delivers packets directly between nodes over an unbounded in-process channel, so there is no socket buffer to overflow, and the previously-quarantined large-network tests run in the default suite again. The shipping daemon build is unaffected (the variant is test-gated).
  • Integration suites that wait for the mesh to converge no longer false-fail under concurrent CI load. The rekey, static-mesh, and sidecar suites replace a fixed wall-clock baseline timeout (and a blind sleep) with a progress-aware wait that polls the suite's own pairwise pings, returns as soon as every pair is reachable, extends its deadline while the reachable-pair count is still climbing, and gives up only when progress stalls.
  • Rekey integration test (testing/static/scripts/rekey-test.sh) no longer false-fails on GitHub runners under packet loss and CPU contention. Phase 1, Phase 3, and Phase 5 strict per-pair pings retry up to 4 attempts (configurable via MAX_PING_ATTEMPTS / PING_RETRY_DELAY) — under 1% per-direction loss, single-shot 20-pair ping_all misses ~33% per phase from ICMP noise alone, and the 4-attempt retry brings that floor to ~3.2e-6 per phase; the wait_for_full_baseline convergence loop stays single-shot so retries there cannot conflate transient ping loss with still-converging routing state. Phase 1 baseline-convergence headroom is bumped from 36s to 60s to eliminate the intermittent Phase 1 timeout that previously required a gh run rerun --failed, and a post-second-rekey settle window is added in Phase 5 (mirroring Phase 3's 12-second pattern) to close the post-rekey per-pair-ping flake from convergence exceeding the per-ping 5-second timeout. Test scaffold only; no daemon code changes, and the success path is unchanged because the wait loops return as soon as all 20 pairs converge.
  • ACL-allowlist integration test (testing/acl-allowlist/test.sh): converted assert_log_contains from a one-shot docker logs | grep snapshot into a bounded poll with the same wait-with-timeout shape as wait_for_peers_exact. Absorbs the millisecond-to-second variance in the XX-handshake cross-connection tie-breaker: the inbound-handshake-context rejection can land tens of milliseconds after the test's previous one-shot grep gave up, producing a pre-existing flake on CI. Success-path cost is unchanged — the helper returns as soon as the pattern appears.

Packaging & deployment

  • AUR packaging: the fips and fips-git PKGBUILDs now install the fips-dns-setup and fips-dns-teardown helpers into /usr/lib/fips/, matching the Debian package. The AUR package() step previously omitted them, so fips-dns.service failed to start on Arch installs ("Unable to locate executable /usr/lib/fips/fips-dns-setup", #98). The PKGBUILDs additionally opt out of the debug split package and declare the *-debug variant as a conflict, so a stale debug build cannot own installed files across a package switch.
  • macOS package build: the .pkg architecture is now derived from the Cargo --target triple instead of the build host's uname -m. The arm64 and x86_64 release legs build on the same Apple-silicon runner, so uname -m named both outputs fips-0.3.0-macos-arm64.pkg; the release job's merge-multiple artifact download then interleaved the two identically named files into a single corrupt xar archive, and no x86_64 package reached the release at all. (This shipped as the broken v0.3.0 macOS .pkg, GitHub #102.) The release workflow now also asserts the arch-named file is present and carries a SHA-256 integrity chain from the build runner through to gh release upload, so a recurrence fails CI instead of publishing.

fipstop

  • fipstop no longer renders a garbled screen on startup or leaves stray bytes on quit, most visible over SSH and inside tmux. Startup forces a full repaint (terminal.clear()) before the first draw so prior alternate-screen contents no longer show through; quit gives the stdin-poll thread a stop flag and joins it before restoring the terminal, so post-raw-mode keystrokes or terminal query responses no longer echo onto the restored screen.

[0.3.0] - 2026-05-11

Added

Mesh Layer (FMP)

  • Overlay-discovery and NAT-hole-punching path (opt-in via node.discovery.nostr.enabled). Nodes publish signed overlay adverts as Nostr kind 37195 parameterized replaceable events listing reachable transport endpoints to a configurable set of public relays, and consume peer adverts to populate fallback addresses for via_nostr peers or, under policy: open, for non-configured peers within a budget cap. The kind value is FIPS-specific: 37195 sits in the application-defined replaceable range 30000–39999, and the digits visually spell FIPS (7=F, 1=I, 9=P, 5=S)
  • STUN-assisted UDP hole punching for addr: "nat" UDP endpoints. STUN reflexive observation, gift-wrap (NIP-59) offer/answer signaling, and candidate-pair punch planner (LAN-private + reflexive paths attempted in parallel). Successful punches hand the live socket into the standard FIPS UDP transport via a bootstrap-handoff API
  • New node.discovery.nostr.* configuration tree with operator-tunable resource caps, replay tracking, and punch timing; new peers[].via_nostr and per-transport advertise_on_nostr / public flags. Cross-field validation at startup catches mis-configured combinations
  • Docker NAT lab covering cone, symmetric (TCP-fallback), and LAN scenarios, wired into the integration CI matrix
  • One-shot startup advert sweep for Nostr open-discovery. On daemon startup under node.discovery.nostr.policy: open, after a short settle delay (startup_sweep_delay_secs, default 5s) the cached overlay-advert table is iterated once and recent adverts (newer than startup_sweep_max_age_secs, default 3600s) are queued for outbound retry, modulo the same skip-filters as the per-tick sweep (configured peer, already connected, retry-pending, connecting). Closes the gap where peers learned only through relay backlog at startup were not dialed until they republished.
  • Diagnostic logging on the open-discovery sweep. Each queued retry now logs at info-level with the peer short-npub and advert age, and a one-line summary (cached count, queued count, per-reason skip counts) is emitted on every startup sweep and on any per-tick sweep that queues at least one retry. Operator-facing visibility into what the auto-dial path is doing.

Platform Support

  • Windows platform support: wintun TUN device, TCP control socket on localhost:21210 (in place of the Unix domain socket), Windows Service lifecycle (--install-service, --uninstall-service, --service), ZIP packaging with PowerShell install/uninstall scripts, and CI build/test matrix entry (#45)
  • macOS platform support: native utun TUN interface management, raw Ethernet transport via BPF, .pkg packaging with launchd plist and uninstall script, x86_64 cross-compile from arm64, and CI build/unit test jobs
  • MIPS atomic ABI support: std::sync::atomic replaced with portable_atomic so 32-bit MIPS targets without native atomics link cleanly (#62, @andrewheadricke).

Mesh Peer Transports

  • Bluetooth Low Energy (BLE) L2CAP Connection-Oriented Channel transport (Linux only, requires BlueZ): per-link MTU negotiation, continuous scan/probe peer discovery with cooldown-based deduplication, continuous advertising, deterministic NodeAddr cross-probe tie-breaker, and a configurable connection pool with eviction.
  • transports.udp.outbound_only (default false). When true, the UDP transport binds a kernel-assigned ephemeral port (0.0.0.0:0) instead of the configured bind_addr, refuses inbound handshakes, and is never advertised on Nostr regardless of advertise_on_nostr. Use this to participate in the mesh as a pure client — initiate outbound links without exposing an inbound listener on a known port. Implements the long-form fix for udp.bind_addr: "127.0.0.1:..." not actually working as a workaround (Linux pins the loopback source IP, dropping outbound flows to external peers at the routing layer)
  • transports.udp.accept_connections (default true). Mirrors the Ethernet/BLE knob; setting to false produces a "client" posture (initiate outbound, refuse inbound msg1 from new addresses). The Node-level handshake gate carves out msg1 from peers already established on this transport so rekey continues to work. Affects every transport via the Transport trait
  • Startup validation now rejects transports.udp[*].bind_addr set to a loopback address when at least one peer has a non-loopback UDP address. Replaces the silent "peer link won't establish" failure mode where Linux's source-address routing check dropped outbound flows from the loopback-bound socket. outbound_only: true is exempt from the check (it overrides bind_addr to 0.0.0.0:0)

Security

  • Mesh-interface nftables baseline (Linux). Ships /etc/fips/fips.nft as a documented operator conffile and fips-firewall.service (disabled by default) for default-deny inbound on the fips0 mesh interface. Operators enable explicitly with systemctl enable --now fips-firewall.service. Drop-ins in /etc/fips/fips.d/*.nft. See docs/fips-security.md.
  • Peer access control list enforcement: optional /etc/fips/peers.allow and /etc/fips/peers.deny files (TCP-Wrappers style) gate outbound connect, inbound msg1, and outbound msg2 against npub, hex pubkey, host alias, or ALL. Files are reloaded automatically on mtime change. New fipsctl acl show query reports the effective rule set (#50, @alexxie16).

LAN Gateway

  • New fips-gateway binary that lets unmodified LAN hosts reach FIPS mesh destinations via DNS-allocated virtual IPs and kernel nftables NAT. Virtual-IP pool (fd01::/112 by default) with state-machine lifecycle and TTL-based reclamation; conntrack-backed session tracking; proxy NDP on the LAN interface; control socket at /run/fips/gateway.sock with show_gateway and show_mappings; fipstop Gateway tab with pool gauge and mappings table; design doc at docs/design/fips-gateway.md; integration test harness
  • Inbound mesh port forwarding on fips-gateway: new gateway.port_forwards config (list of { listen_port, proto, target } entries, IPv6 targets only) installs prerouting DNAT rules so mesh peers can reach a configured host:port on the gateway's LAN. A LAN-side masquerade is added when any forwards are configured so replies flow back through conntrack.
  • Gateway packaging: systemd service unit with After=fips.service, Debian and AUR package entries, OpenWrt procd init with dnsmasq forwarding, proxy NDP, RA route advertisements, and IPv6 forwarding sysctls. Gateway enabled by default on OpenWrt
  • fips-gateway DNS upstream probe now retries up to 5 times with a 1-second per-attempt timeout and a 1-second delay between attempts (~10 second worst-case wait), instead of a single 3-second hard-fail. Covers the cold-boot race where the daemon's TUN is up (the systemd ExecStartPre wait gates on that) but the DNS responder is still binding [::1]:5354. Without retry the gateway exited and relied on Restart=on-failure for recovery (5-second blip + spurious error log line per cycle); with retry the gateway recovers gracefully without a unit restart

IPv6 Adapter

  • Overhauled .fips DNS handling for systemd-based hosts. The default dns.bind_addr is ::1 (IPv6 loopback) and the setup script picks one of five backends in priority order: a global drop-in at /etc/systemd/resolved.conf.d/fips.conf, the systemd dns-delegate path, resolvectl per-link, standalone dnsmasq, or NetworkManager's dnsmasq plugin. Teardown reverses only what was applied. New testing/dns-resolver/ harness exercises every backend across Debian 12, Debian 13, Ubuntu 22.04, Ubuntu 24.04, and Ubuntu 26.04 (#58, fixes #52, #77).

Operator Tooling

  • node.log_level config field (case-insensitive, default info) replaces the hardcoded RUST_LOG=info previously baked into systemd units and the OpenWrt procd init script. The daemon now loads config before initializing tracing so the configured level takes effect; RUST_LOG still overrides when set
  • fipsctl show identity-cache lists every cached node identity (npub, IPv6 address, display name, LRU age) alongside the configured cache capacity
  • fipsctl show peers extended with per-peer security signals (replay suppression count, consecutive decrypt failures), Noise session counters, session indices, and rekey lifecycle state
  • fipsctl show sessions extended with handshake resend count during establishment and rekey/session health fields when established (session start, K-bit epoch, coords warmup remaining, drain state)
  • fipsctl show cache now includes individual coordinate cache entries (tree coordinates, depth, path MTU, age). The top-level count field was renamed from entries to count for clarity
  • fipsctl show routing expands pending_lookups from a count to per-target detail (attempt, age, last sent), adds pending TUN packet queue depth, and adds per-peer connection retry state (#42, @osh)
  • Historical node and per-peer statistics: in-memory time-series rings on the daemon, surfaced through new control-socket queries, fipsctl stats subcommands, and a fipstop Graphs tab with btop-style sparklines (#64).
  • fipstop Node tab now carries a "Listening on fips0" panel (right-half of the Traffic block) that lists local IPv6 listening sockets reachable from the mesh interface, paired with the inet fips baseline filter classification for each (proto, port). Rows render in default White (OPEN — the chain has a canonical unrestricted accept rule), DarkGray (filt — chain falls through to counter drop), or DarkGray with a ? State suffix (filt? — the chain references the port but with matchers the panel cannot fully decompose, e.g. saddr filters or jumps). When the fips-firewall.service is not active, the panel renders a yellow banner reminding the operator that all listeners are mesh-exposed. Wildcard binds (local_addr == ::) carry a * suffix in the Process column. Powered by a new show_listening_sockets control query (Linux-only).

Packaging and Deployment

  • Arch Linux AUR packaging for fips (release) and fips-git (development) packages with sysusers.d/tmpfiles.d integration (#21, @dskvr)
  • packaging/debian/fips-gateway.service now waits up to 30 seconds for the daemon's fips0 TUN to appear before exec'ing the gateway binary (ExecStartPre poll loop). Eliminates the cold-boot race where fips-gateway exits with fips0 interface not found and recovers via Restart=on-failure, producing a 5-second blip and a spurious error log line per restart cycle. If fips0 never appears within 30 seconds, the existing error path runs as before
  • packaging/debian/build-deb.sh now auto-derives a per-commit Debian Version field for dev builds (Cargo.toml version ending in -dev) using the form <base>~dev+git<YYYYMMDD>.<sha>[.dirty]-1, e.g. 0.3.0~dev+git20260429.6def31b-1. Each commit produces a uniquely- comparable Version string so apt install ./*.deb and ansible.builtin.apt: deb: no longer silently no-op when one dev build is installed on top of another. The ~dev marker sorts pre-0.3.0 so a tagged release supersedes any prior dev .deb. Tagged release builds (no -dev in Cargo.toml) keep the clean <version>-1 form. Operator override via --version still wins

Examples

  • macOS WireGuard sidecar: run FIPS in a local Docker container and route .fips traffic from the macOS host through a WireGuard tunnel to the container's fips0 interface. Only traffic destined for fd00::/8 transits the sidecar; regular internet traffic continues to use the host network (#51)

Documentation

  • docs/design/port-advertisement-and-nat-traversal.md documents how nodes find each other through Nostr relays and the STUN-assisted UDP hole punch

Changed

  • Noise session ChaCha20-Poly1305 backend switched from RustCrypto's chacha20poly1305 to ring 0.17. ring wraps BoringSSL's hand-tuned ChaCha20-Poly1305 implementation, dispatching to NEON on aarch64 and AVX2 / AVX-512 on x86_64 — typically 3-5 GB/s/core vs the ~600-800 MB/s/core RustCrypto soft path on the same hardware. Wire format unchanged: ChaCha20-Poly1305 is byte-deterministic for a given (key, nonce, plaintext, aad), so any correct AEAD produces identical ciphertext and a mixed pre-swap / post-swap mesh interoperates without protocol awareness. The keyed AEAD is now cached on CipherState instead of being re-derived per packet (the cached Poly1305 key state is the actual perf win); EndToEndState grew from ~600 B to ~1.5 KB as a consequence and is annotated #[allow(clippy::large_enum_variant)] since boxing would re-add a per-packet indirection on every encrypt/decrypt. aarch64 measurements (Apple Silicon docker, two nodes): TCP 1-stream 437 → 1097 Mbps (~2.5×); UDP at 1000 Mbit goes from 599 Mbps / 40 % loss to lossless line-rate; 3-node ping under load 7.68 ms avg / 215 ms max → 0.72 ms / 3.6 ms max as the relay path stops being crypto-bound (#80, @mmalmi)
  • Linux UDP receive path uses recvmmsg(2) with a 32-packet batch in place of single-packet recvmsg(2). A single readable() wakeup drains up to 32 datagrams in one syscall before yielding back to the reactor, eliminating the per-packet scheduler-hop + futex cost that previously capped inbound rate at one event per scheduler quantum independent of CPU. SO_RXQ_OVFL is sampled once per batch from the cmsg chain of msgs[0] and surfaced through AsyncUdpSocket::recv_batch so the 1Hz sample_transport_congestion() detector continues to feed the per-transport dropping flag. macOS / Windows fall through to the per-packet path; recvmmsg is Linux-specific (#81, @mmalmi)
  • Node::run_rx_loop drains up to 256 additional ready items via try_recv() after each tokio::select! await fires on packet_rx / tun_outbound_rx, in a tight inner loop before yielding. Previously the select cost a full scheduler hop + futex per packet, capping throughput at one event per scheduler quantum with the worker near-idle. biased ordering keeps data-plane branches priority over tick / control / DNS under sustained load; the 256 cap is empirically tuned to keep the worker on a busy stream between yield points (≈ 400 KB of contiguous traffic) while still bounding the inner loop so a flood on one branch can't starve the periodic tick or control socket. Pairs with the UDP recvmmsg change above (#81, @mmalmi)
  • PeerIdentity::pubkey_full() now precomputes the parity-aware full public key at construction in from_pubkey. Previously the method fell through to a secp256k1 EC point parse (fe_sqrt + fe_mul + ge_set_xo_var) on every call when the full key wasn't passed at construction (i.e. for every peer constructed from an npub or x-only key) — ~6% of per-packet CPU on the bulk-data send path for a value that never changed after construction. The same EC point parse already runs at construction inside NodeAddr::from_pubkey, so the cost is paid once where it would be paid anyway (#81, @mmalmi)
  • Cargo feature flags tui, ble, gateway, and nostr-discovery removed; subsystem inclusion is now driven by platform cfg gates so plain cargo build compiles everything available on the target (#79)
  • MMP link-layer report intervals retuned for constrained transports: steady-state floor raised from 100ms to 1000ms, ceiling from 2000ms to 5000ms. Cold-start uses a 200ms floor for the first 5 SRTT samples before switching to steady-state. Reduces BLE overhead ~10× while keeping reports well above the EWMA convergence threshold. Session-layer intervals unchanged
  • 35 info-level log messages demoted to debug (handshake cross-connection mechanics, periodic MMP telemetry, TUN/transport shutdown, retry scheduling). Info output now focuses on operator-relevant state changes: lifecycle events, peer promotions, session establishment, parent switches, transport start/stop
  • Breaking (control socket JSON): show_cache response field entries has changed type from a u64 count to an array of entry objects; a new count field carries the previous scalar value. show_routing response field pending_lookups has changed type from a u64 count to an array of per-target lookup objects. External consumers parsing these fields as numbers must be updated. In-tree fipstop is adjusted to the new schema. The control socket interface is still pre-1.0 and not covered by stability guarantees
  • Discovery rate limiting retuned to be less aggressive at cold start. The previous defaults (30s base post-failure suppression, doubling to a 300s cap, with reset only on parent change / new peer / first RTT / reconnection) reliably outlasted initial mesh convergence: a single timed-out lookup during bloom-filter propagation suppressed any retry for 30s while none of the reset triggers fired on a stable post-handshake topology. The suppression window dictated effective time-to-converge instead of bounding repeat traffic. Replaces the single-lookup-with-internal-retry model (timeout_secs/retry_interval_secs/max_attempts) with a per-attempt timeout sequence in node.discovery.attempt_timeouts_secs (default [1, 2, 4, 8]). Each attempt sends a fresh LookupRequest with a new request_id, which lets successive attempts take different forwarding paths as the bloom and tree state evolve. The destination is declared unreachable only after the full sequence is exhausted (15s total at the default). Disables post-failure suppression by default (backoff_base_secs/backoff_max_secs now both 0); operators with chatty apps generating repeat lookups against unreachable destinations can opt back in
  • The docs/ tree is reorganised so readers can find content by what they're trying to do: tutorials for new users, how-to guides for specific tasks, reference material for configuration and protocol details, and design discussion for architectural background. New top-level getting-started.md and per-section landing pages anchor the entry points. Content was reconciled against current source: protocol layer details, wire-format diagrams, configuration knobs, and CLI references were brought back into agreement with the implementation. Gateway feature-set documentation was rewritten end-to-end.
  • Test coverage was substantially expanded for the new release surface (discovery state machine, control-socket query handlers, decrypt-failure thresholds, STUN parser, gateway, NAT traversal, packaging install paths) alongside CI-side hardening for the new Windows and macOS platforms.
  • Gateway dns.listen source default changed from [::]:53 to [::1]:5353 to match the canonical deployment model (a host already serving DHCP/DNS to a LAN segment, where port 53 is taken by the existing resolver and .fips queries are forwarded to the gateway over loopback). The OpenWrt ipk previously overrode this in its packaged config; the override is now redundant and has been dropped. Operators on a host without a pre-existing resolver on port 53 can opt back into the wildcard bind by setting dns.listen: "[::]:53" explicitly. The new default binds IPv6 loopback only — forwarders that reach the gateway over IPv4 loopback need an explicit IPv4 listen address.
  • Generic systemd install tarball brought to feature parity with the .deb and AUR packages. The tarball now ships the fips-gateway binary with its (operator-opt-in) fips-gateway.service, a fips-firewall.service unit with the /etc/fips/fips.nft mesh-interface nftables baseline (also opt-in), an /etc/fips/fips.d/ operator drop-in directory for per-service nft rules, and the multi-backend fips-dns-setup / fips-dns-teardown helpers. install.sh and uninstall.sh handle the new units and conffile (preserve-on-upgrade for fips.nft, like fips.yaml). README.install.md documents the gateway, firewall, and DNS-routing services. Closes the longest-standing parity gap for non-Debian / non-Arch systemd Linux distros (Fedora, RHEL/CentOS, openSUSE, etc.) installing from the release-distribution tarball.

Fixed

  • Generic systemd install tarball: install.sh now correctly resolves the fips-dns-setup and fips-dns-teardown helpers from the tarball staging directory. Previously the script referenced them at ${SCRIPT_DIR}/../common/, a path that exists only in the source-repo layout, not in the extracted tarball. Bug latent since the multi-backend DNS helpers landed in 7260ad2; only manifested when operators ran install.sh from an extracted tarball rather than from a source checkout.

  • Adopted NAT-traversed UDP transports inherit the primary listener's MTU and buffer config. Node::adopt_established_traversal constructed the adopted UDP transport with UdpConfig::default() (MTU 1280, default recv/send buffer sizes, default accept/advertise flags) regardless of the operator's primary [transports.udp] listener. Operators who set the primary MTU higher (e.g. 1500 on a known-clean LAN path) silently dropped full-sized tunnel datagrams over the NAT-traversed link with no log explaining why throughput collapsed. Lookup now tries transport_name first (so multiple named listeners pick up inheritance from the matching one) and falls back to the unnamed Single listener; bind / external-address fields are cleared since the adopted socket is already bound. The 1280 default was deliberately the IPv6 minimum (the only value guaranteed across arbitrary middlebox paths); with this change, operators who raise the primary MTU accept the tradeoff that NAT-traversed flows initially attempt the higher MTU and may black-hole on tighter paths until reactive MtuExceeded recovery kicks in (#83, @mmalmi)

  • TreeAnnounce ancestry on self-root transitions. When a node had no smaller-NodeAddr peer to use as a parent, the spanning-tree state correctly promoted it to root, but the ancestry it advertised on the next TreeAnnounce still referenced its previous parent's path. Receiving peers rejected the announce with invalid ancestry: advertised root X is not the minimum path entry Y, blocking mesh transit on any path that needed to traverse the node. The self-root transition is now detected explicitly in TreeState::become_root and the advertised ancestry rebuilt to start from self. The MMP receive handler surfaces the same path so stale ancestry inherited across reconnect is corrected eagerly rather than waiting for the next observation tick (#82, @mmalmi)

  • Auto-connect retry refetches the cached overlay advert unconditionally before each retry attempt, not only when fetch_advert returns zero endpoints (NoTransportForType). The much more common stale-cache failure was: cache returned an endpoint that looked valid (the address learned before the peer's NAT rebound), the dial succeeded at the IP layer, the handshake timed out, MMP fired, the next retry hit the same cached endpoint, looped forever — no NoTransportForType ever fired because the cache had data, just dead data. Refetch now runs unconditionally before each retry attempt (one Filter query against advert_relays with a 2s per-attempt timeout, bounded by the retry backoff cadence). Keeps the retry loop pinned to relay ground truth instead of whatever the cache happened to learn at startup (#82, @mmalmi)

  • Stale overlay-advert eviction on NoTransportForType. Mirrors the existing stale-advert sweep that ran from the BootstrapEvent::Failed (NAT-traversal-streak) path, but covers the case where initiate_peer_connection / a retry tick returns NodeError::NoTransportForType — the cache had no addresses for the peer at all. A fire-and-forget refetch_advert_for_stale_check against the peer's npub re-fetches kind 37195 from advert_relays; if the relay has a newer advert it replaces the cached entry, if it has nothing it evicts the entry. Either way the next retry tick goes to fresh data instead of looping on the same dead endpoint. Resolves a deployment regression where a macOS daemon's view of a Linux peer would flap after NAT rebind with no recovery short of a daemon restart (#82, @mmalmi)

  • Schedule retry on startup peer-init failure. When initiate_peer_connections() ran at boot, an address-resolution failure (no operational transport for the configured transport types, all addresses unreachable, NAT rebind invalidating cached endpoints) was logged and silently forgotten — the peer entry stayed in a dead state forever, accepting incoming pings but unable to answer them, until the daemon was manually restarted. Now mirrors the BootstrapEvent::Failed path: on a startup peer-init error, parse the peer's npub and call schedule_retry so the peer recovers without operator intervention (#82, @mmalmi)

  • Default control-socket path resolution: daemon and client tools now use a shared resolver, eliminating a divergence where fipsctl / fipstop could connect to a socket the daemon never bound (notably on dev runs with XDG_RUNTIME_DIR set, or after a prior packaged install left a root-owned /run/fips behind). Canonical order is /run/fips → $XDG_RUNTIME_DIR/fips/ → /tmp/fips-<name>. The /run/fips arm is selected by directory existence; the kernel enforces actual access at connect(2) time, surfacing a clear EACCES for users not yet in the fips group rather than silently steering them to a path the daemon never bound. XDG_RUNTIME_DIR is validated as an existing directory before being used so stale post-logout values are treated as missing. The deployed fleet is unaffected: packaged configs set node.control.socket_path explicitly.

  • UDP transport with advertise_on_nostr: true + public: true + a wildcard bind_addr (e.g. 0.0.0.0:2121) is now advertised with its STUN-discovered public IPv4 instead of being silently dropped from the published Kind 37195 advert. Previously the advert builder filtered the wildcard out (since 0.0.0.0 is not a valid endpoint), but emitted no log explaining what happened — operators saw the daemon up, both flags set, and no UDP endpoint in the advert. The fix runs a one-shot STUN observation against an ephemeral socket on the daemon's configured stun_servers and combines the reflexive IPv4 with the configured listener port for the advert (udp:<eip>:<port>). Successful STUN observations are cached per-transport for one advert_refresh_secs cycle (default 30 min) so we don't re-STUN every refresh. Failed observations are cached for only 60s, so a transient STUN flake at startup retries within ~a minute and grows the advert with UDP as soon as STUN starts working — rather than waiting the full 30-min cycle. Per-server STUN response timeout is 5s for the advert-publish path (vs. 2s for the latency-sensitive per-traversal path), giving slow first-call STUN time to complete without giving up. On STUN failure, the wildcard-bind path still skips, but now logs a loud warn! pointing at the operator-side fixes (set external_addr, bind to a specific IP, or ensure stun_servers reachable). Restores zero-config public-IP autodiscovery on AWS EIP / GCP / Azure setups where binding to the public IP directly is impossible (1:1 NAT)

  • New external_addr field on transports.udp.* and transports.tcp.* for explicit advertise-as override. Accepts either a bare IP ("198.51.100.1" — the configured bind_addr port is appended) or a full host:port ("198.51.100.1:8443"). Takes precedence over both the bound address and any STUN-derived autodiscovery. Required for TCP on cloud-NAT setups (AWS EIP, GCP/Azure external IPs) where binding to the public IP directly fails with EADDRNOTAVAIL (the EIP isn't on a host interface). Optional but useful for UDP as a deterministic alternative to STUN — operators who want to skip STUN egress (or whose STUN is blocked) can specify it explicitly. Without external_addr, TCP with a wildcard bind_addr + advertise_on_nostr: true now logs a loud warn! pointing at the two fixes instead of silently skipping

  • Nostr-discovery now tolerates ±60s of clock skew on offer/answer freshness checks so a responder whose wall clock leads the initiator's by less than that no longer silently rejects every offer. Previously, a public-test daemon with un-NTP'd peers (or long uptime — now_ms() anchors to SystemTime once at startup, then advances monotonically; post-startup NTP step adjustments don't propagate) would see ~100% signal-timeout rate against skewed peers, indistinguishable from "peer is offline." New optional offerReceivedAt field on the answer payload lets the initiator log per-peer NTP-style skew estimates (DEBUG when ≥30s) for operator visibility. Backward-compatible — older responders that don't fill the field still produce valid answers

  • Nostr-discovery NAT-traversal failure suppression: per-npub consecutive-failure counter triggers a 30-min extended cooldown after 5 failures, preventing the daemon from hammering Nostr relays with offers to peers that have gone away. WARN log lines rate-limited to one per peer per 5 min (subsequent failures emit DEBUG with consecutive_failures + remaining cooldown_secs). Threshold-crossing also fires a one-shot active re-check of the peer's Kind 37195 advert against advert_relays; absent → evict cache; newer → refresh + reset streak; same → cooldown stands. New failure_streak_threshold, extended_cooldown_secs, warn_log_interval_secs, failure_state_max_entries config fields under node.discovery.nostr. Per-peer state visible in fipsctl show peers JSON under nostr_traversal

  • Tor onion adverts published over Nostr overlay discovery now include the public-facing port (<onion>.onion:<port>) instead of just the bare onion hostname. The publisher previously emitted a bare onion that the parser refused (expected host:port), producing a persistent retry-fail loop on any peer whose Tor advert was the only entry in the discovery cache. New transports.tor.advertised_port config field (default 443, matching the Tor HiddenServicePort convention) controls the advertised port; operators with non-default virtual ports can override.

  • TCP-over-FIPS reliability on mesh paths with mixed transport MTUs (e.g. a UDP-1280 hop in the picker set) improved. Three interlocking changes: Node::transport_mtu() is now deterministic across restarts (min across operational transports rather than insertion-order-dependent); the TCP MSS clamp at the TUN boundary reads per-destination path MTU instead of a single global ceiling; and reactive MtuExceeded from forwarders is mirrored back into the TUN-side path_mtu_lookup so later flows pick up forward-path bottlenecks without re-discovery. Windows TUN reader receives the same per-destination plumbing.

  • Proactive end-to-end PathMtuNotification now mirrors into the TUN-side path_mtu_lookup (TCP MSS clamp store), parallel to the reactive MtuExceeded mirror that already existed. Previously the proactive handler only updated the session-canonical MmpSessionState.path_mtu; on stable long-lived paths where the destination's echo had tightened the session MTU but no transit router had emitted a fresh MtuExceeded (because all current traffic was already sized by the tighter session value), new TCP flows opened in that window kept getting clamped by the staler discovery-time value. The proactive mirror closes that gap with the same tighter-only semantics — never loosens the clamp.

  • Nostr-discovered peers running an FMP-protocol version we cannot speak no longer trigger an indefinite retraversal storm. Open- discovery NAT-traversal succeeds at the UDP layer regardless of protocol version, so the daemon would adopt the punched socket, drop every incoming packet at Unknown FMP version, idle out after 31s, and re-fire the full STUN-offer-answer-punch sequence ~30s later — every minute, forever, against peers the handshake literally cannot complete with. The rx loop now detects mismatched- version packets arriving on adopted bootstrap transports, reverse- maps to the originating npub, and applies a long structural cooldown to the discovery layer's failure_state so the next open-discovery sweep skips the peer until either side upgrades. One-shot WARN per fresh observation; subsequent mismatches inside the cooldown window are silent. New protocol_mismatch_cooldown_secs config field under node.discovery.nostr (default 86400 = 24h), separate from the transient-failure extended_cooldown_secs.

  • fipstop now uses ratatui::try_init() instead of ratatui::init(), so terminal initialization failures (e.g. Docker on macOS Sequoia, or environments without a usable tty) produce a clean error message instead of a hard crash

  • Spanning-tree updates that change only the internal path between root and leaf — without changing the root or the depth — now propagate to leaves correctly. Previously a leaf could continue routing against a stale internal path until the parent or depth also changed.

[0.2.1] - 2026-05-11

Added

  • Linux release artifact workflow: builds x86_64 and aarch64 tarballs and .deb packages on v* tag push, with SHA-256 checksums
  • AUR publish workflow for tagged stable releases

Changed

  • Validate bloom filter fill ratio on FilterAnnounce ingress. Inbound FilterAnnounce messages whose derived false-positive rate exceeds node.bloom.max_inbound_fpr (new config field, default 0.05) are rejected silently on the wire, logged at WARN, and counted in a new bloom.fill_exceeded counter. A rate-limited WARN also fires if our own outgoing filter's FPR exceeds the cap. BloomFilter::estimated_count now takes max_fpr and returns Option<f64>, returning None for saturated filters; this propagates through compute_mesh_size into estimated_mesh_size (already Option<u64>)

Fixed

  • Control socket path detection in fipsctl and fipstop now checks for the /run/fips/ directory instead of the socket file inside it, so users not yet in the fips group get a clear "Permission denied" error instead of a misleading "No such file" fallback to $XDG_RUNTIME_DIR (#30, reported by @Sebastix)
  • OpenWrt ipk build excluded BLE feature that requires D-Bus, which is unavailable on OpenWrt targets
  • IPv6 routing policy rule added at TUN setup to protect fd00::/8 from interception by Tailscale's table 52 default route
  • Bloom filter routing no longer swallows traffic when no bloom candidate is strictly closer than the current node. find_next_hop now falls through to greedy tree routing in that case instead of returning NoRoute, which previously caused dropped packets in topologies where the tree parent was closer but not a bloom candidate
  • Auto-connect peers now reconnect after a graceful Disconnect notification from the remote side. handle_disconnect previously removed the peer without scheduling a reconnect, orphaning the entry on a clean upstream shutdown; the other removal paths (link-dead, decrypt failure, peer restart) already scheduled reconnect (#60, reported by @SwapMarket)
  • fipsctl connect now rejects FIPS mesh (fd00::/8) addresses for udp, tcp, and ethernet transports with a clear error message instead of echoing success while the daemon silently failed the bind with EAFNOSUPPORT (#61, reported by @SwapMarket)
  • Tighten TreeAnnounce ancestry validation to match the spanning tree specification. The receive path now verifies that the ancestry is structurally consistent with the signed parent declaration before mutating tree state.
  • Make the tree ancestry acceptance unit test deterministic. test_tree_announce_validate_semantics_accepts_valid_non_root generated a random signing identity while pinning the fixed root to node_addr[0] = 0x01; about 2 in 256 random identities were numerically smaller than the claimed root, triggering AncestryRootNotMinimum. The test now regenerates the identity until its node_addr is strictly larger than both the fixed parent and root.

[0.2.0] - 2026-03-22

Added

Operator Tooling

  • fipsctl connect and disconnect commands for runtime peer management via control socket, with hostname resolution from /etc/fips/hosts

IPv6 Adapter

  • Pre-seed identity cache from configured peer npubs at startup, so TUN packets can be dispatched immediately without waiting for handshake completion (@v0l)

Mesh Peer Transports

  • New Tor transport with SOCKS5 and directory-mode onion service for anonymous inbound and outbound peering
  • DNS hostname support in peer addresses for UDP and TCP transports
  • Non-blocking transport connect for connection-oriented transports (TCP, Tor)

Packaging and Deployment

  • Reproducible build infrastructure: Rust toolchain pinning via rust-toolchain.toml, SOURCE_DATE_EPOCH in CI and packaging scripts, deterministic archive timestamps
  • Top-level packaging Makefile for unified build across formats
  • Kubernetes sidecar deployment example with Nostr relay demo
  • Nostr release publishing in OpenWrt package workflow
  • SHA-256 hash output in CI build and OpenWrt workflows

Testing and CI

  • Maelstrom chaos scenario with dynamic topology mutation and ephemeral node identities via connect/disconnect commands
  • Consolidated Docker test harness infrastructure

Changed

  • Discovery protocol: replace flooding with bloom-filter-guided tree routing. Includes originator retry (T=0/T=5s/T=10s), exponential backoff after timeouts and bloom misses, and transit-side per-target rate limiting. Removed 257-byte visited bloom filter from LookupRequest wire format. This is a breaking change; nodes running versions prior to this release will not be compatible.

Fixed

  • DNS responder returned NXDOMAIN for A queries on valid .fips names, causing resolvers to give up without trying AAAA. Now returns NOERROR with empty answers for non-AAAA queries on resolvable names. (#9, reported by @alopatindev)
  • Stale end-to-end session left in session table after peer removal blocked session re-establishment on reconnect — remove_active_peer now cleans up self.sessions and self.pending_tun_packets. (#5, @v0l)
  • schedule_reconnect reset exponential backoff to zero on each link-dead cycle instead of preserving accumulated retry count. (#5, @v0l)
  • FMP/FSP rekey dual-initiation race on high-latency links (Tor): both sides' timers fired simultaneously, both msg1s crossed in flight, each side's responder path destroyed the initiator state. Fixed with deterministic tie-breaker (smaller NodeAddr wins as initiator).
  • Parent selection SRTT gate bypass: evaluate_parent used default cost 1.0 for peers filtered out by has_srtt(), defeating the MMP eligibility gate. Now skips unmeasured candidates when any peer has cost data.
  • FSP rekey cutover race: initiator cut over before responder received msg3, causing AEAD failures. Fixed by deferring initiator cutover by 2 seconds.
  • MMP metric discontinuity after rekey: receiver state carried stale counters across rekey, inflating reorder counts and jitter. Fixed via reset_for_rekey().
  • Auto-connect peers exhausted max_retries on initial connection failures and were permanently abandoned. Now retry indefinitely with exponential backoff capped at 300 seconds.
  • Control socket permissions: non-root users couldn't connect. Daemon now chowns socket and directory to root:fips group at bind time.
  • Post-rekey jitter spikes: old-session frames arriving via the drain window produced 2,000–7,000ms jitter spikes that corrupted the EWMA estimator. Added a 15-second grace period after rekey cutover that suppresses jitter updates until drain-window frames have flushed. (#10)
  • ICMPv6 Packet Too Big source was set to the local FIPS address, which Linux ignores (loopback PTB check). Now uses the original packet's destination so the kernel honors the PMTU update. (#16, @v0l)
  • Reverse delivery ratio used lifetime cumulative counters instead of per-interval deltas, making ETX unresponsive to recent loss. (#14)
  • MMP delta guards used prev_rr > 0 to detect first report, conflating it with a legitimate zero counter. Replaced with has_prev_rr. (#14)

[0.1.0] - 2026-03-12

Added (Initial Release)

Session Layer (FSP)

  • End-to-end encrypted datagram service between mesh nodes addressed by Nostr npub
  • Noise XK sessions with mutual authentication, replay protection, and forward secrecy
  • Automatic session rekeying with configurable time/message thresholds and drain window for in-flight packets
  • Port multiplexing for multiple services over a single session
  • Session-layer metrics: sender/receiver reports with RTT, jitter, delivery ratio, and burst loss tracking
  • Passive RTT measurement via spin bit

IPv6 Adapter

  • IPv6 adapter interface allowing tunneling TCP/IPv6 through FIPS mesh for traditional IP applications (TUN interface)
  • DNS resolver allowing IP applications to reach nodes by npub.fips name
  • Host-to-npub static mappings: resolve hostname.fips via host map populated from peer config aliases and /etc/fips/hosts file

Mesh Layer (FMP)

  • Self-organized core mesh routing protocol with adaptive least cost forwarding
  • Noise IK hop-by-hop link encryption with mutual authentication and replay protection between peer nodes
  • Distributed spanning tree construction with cost-based parent selection and adaptive reconfiguration
  • Destination route discovery via bloom filter-based directed search protocol
  • Path MTU discovery with per-link MTU tracking and MtuExceeded error signaling
  • Link-layer MMP: SRTT, jitter, one-way delay trends, packet loss, and ETX metrics
  • Link-layer heartbeat with configurable liveness timeout for dead peer detection
  • Epoch-based peer restart detection
  • Automatic link rekeying with K-bit epoch coordination and drain window
  • Static peer auto-reconnect with exponential backoff
  • Multi-address peers with transport priority-based failover
  • Msg1 rate limiting for handshake DoS protection

Mesh Peer Transports

  • UDP overlay transport with inbound and static outbound peer configuration
  • TCP overlay transport with listening port and static outbound peer support
  • Ethernet/WiFi transport (MAC address based, no IP stack) with optional automatic peer discovery and auto-connect

Operator Tooling

  • Ephemeral or persistent node identity with key file management
  • Unix domain control socket for runtime observability
  • fipsctl CLI tool for control socket interaction and node management
  • Comprehensive node and transport statistics via control socket
  • fipstop TUI monitoring tool with real-time session, peer, and transport configuration and metrics display

Packaging and Deployment

  • Debian/Ubuntu .deb packaging via cargo-deb
  • Systemd service packaging with tarball installer
  • OpenWRT package with opkg feed and init script
  • Docker sidecar deployment for containerized services
  • Build version metadata: git commit hash, dirty flag, and target triple embedded in all binaries via --version

Testing and CI

  • Comprehensive unit and integration tests covering all protocol layers and transports
  • Docker test harness with static and stochastic topologies
  • Chaos testing with simulated severe network conditions: latency, packet loss, reordering, and peer churn
  • CI with GitHub Actions: x86_64 and aarch64, integration test matrix, nextest JUnit reporting
  • Local CI runner script (testing/ci-local.sh)

Project

  • Design documentation suite covering all protocol layers
  • CHANGELOG.md following Keep a Changelog format
  • Repository mirrored to ngit