Files
fips/CHANGELOG.md
T

4306 lines
259 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
#### Gateway
- The gateway counts sessions on a kernel without `/proc/net/nf_conntrack`.
When the file is absent it dumps the conntrack table over netlink, as
`conntrack -L` does, so a mapping carrying traffic is pinned instead of
being reclaimed on its TTL and grace period alone. Kernels built without
`CONFIG_NF_CONNTRACK_PROCFS`, such as Ubuntu's, had session pinning off.
- The gateway says at startup whether it can read conntrack sessions. It
reads the table once, as each tick does, and logs either the source it read
or that no source is readable and session pinning is off. An operator on a
kernel with no readable source learned this only from a warning at the first
failed tick.
### Changed
#### Packaging (Debian)
- An upgrade of the `.deb` now reapplies the firewall ruleset in place. Until
now an upgrade reloaded nothing, so a changed `/etc/fips/fips.nft` took
effect only at the next reboot or manual restart, and a restart deletes the
`fips` table and leaves the mesh interface unfiltered until the ruleset is
loaded again. `fips-firewall.service`, in both the Debian and the plain
systemd unit, gains a reload that replaces the ruleset in one transaction,
and the postinst reloads the unit only when it is already active, so an
upgrade never turns the firewall on for a host that has not opted in. A
reload that fails leaves the previous ruleset in place and is reported; the
upgrade goes on.
#### Packaging (AUR)
- The AUR publish on a release tag now waits until every package workflow of
that tag has succeeded. It used to push the new `pkgver` while the Linux,
macOS, Windows, OpenWrt and FreeBSD packages were still building; at v0.5.1
the AUR was updated while the release had 15 of its 17 assets. Because the
AUR package pins the tag's source archive, withdrawing a bad release after
that point left the AUR package unbuildable. A failed or cancelled package
run now stops the publish, and one that has not finished within an hour
fails it.
#### Dependencies
- The lockfile moves `chacha20` from 0.10.1 to 0.10.2, because 0.10.1 is yanked.
It arrives through `rand`, a direct dependency,
so it sits on the built path rather than off to one side. The requirement in
`Cargo.toml` already admitted 0.10.2, so this is a lockfile change and no code
changed with it. **This is not a security fix**: `cargo audit` reports nothing
against `chacha20` at either version, and 0.10.1 was withdrawn by its
maintainer rather than flagged by an advisory. What it buys is that a fresh
checkout can resolve the lockfile without reaching for a yanked version.
### Fixed
#### Node lifecycle
- A DNS responder or TUN thread that dies now degrades the node's published
health, and a dead responder's address is retracted. The responder's exit
report followed a loop that never returns, so it could not run, and a panic
in the responder or in either TUN thread unwound past its report. The node
kept reporting healthy with the child gone and kept publishing the DNS
address with nothing answering on it. Each child now runs inside a wrapper
that catches a panic, logs it, and reports the exit either way. A deliberate
stop still reports nothing.
#### Data plane and transports
- A configured `ble:` transport that this build cannot construct is now
reported. The only warning for it was compiled into test builds alone, where
logging is compiled out, so macOS, Windows, FreeBSD, OpenWrt and other musl
builds, and Android without a BLE radio armed before start, dropped the block
silently while reporting healthy. The daemon now warns once per configured
instance at startup, naming the reason.
- Two inbound TCP connections that share a peer address but arrive on different
local addresses no longer share one pool entry. The kernel names a connection
by its four-tuple, so a listener on a wildcard address, which is what the
shipped configuration binds, can accept two connections whose peer `ip:port`
is the same on two different local addresses. The pool was keyed by the peer
address alone: the second connection's entry replaced the first's while the
inbound-connection counter counted both, the first connection's teardown then
removed the second's entry, and the second's own teardown found nothing to
remove, so the counter ended one above the connections it counts. That counter
gates the inbound connection limit, so a host repeating the collision could
hold it at the limit and lock out further inbound TCP connections until the
daemon restarted. Inbound entries now carry the accepted socket's local
address in their pool key as well as the remote one.
- A peer that moves to a new address now loses the per-peer `connect(2)`-ed UDP
socket pinned to the address it left. `set_current_addr` returns whether the
address actually changed so the caller can drop the stale socket, and the
decrypt-worker completion path already acted on that return; the in-line
decrypt path discarded it, so the socket stayed installed and the send path
kept preferring it over the wildcard listen socket.
- A peer reached by NAT traversal now gets its per-peer connected UDP socket.
The adopted traversal socket carried no address-reuse flags, so the connected
socket's bind to the same port was refused with `EADDRINUSE` on every tick and
the peer never left the unconnected path. The flags are now set when the
socket is adopted, after its bind, so the traversal bind still receives a
port no other socket holds.
#### Peering
- A heartbeat whose send failed no longer counts as one that was delivered. The
send was recorded before it was attempted, so a peer whose heartbeat could not
go out was treated as heartbeated and was not tried again for a whole
`heartbeat_interval_secs`, although it had heard nothing and its own link-dead
timer was running. The attempt and the delivery are now recorded separately:
the interval that paces a healthy peer advances only on a send that returned
cleanly, and a peer whose send failed is retried after a shorter fixed
interval instead. That retry interval gates only a peer whose last attempt
failed, so it cannot clamp a `heartbeat_interval_secs` configured below it.
#### Routing and discovery
- A node now re-announces its bloom filter when a peer starts or stops using
it as parent. A node's outgoing filter merges only its tree peers' filters,
and nothing re-marked the other peers when a peer's tree announce changed
whether it named us as parent, so our parent kept the old filter.
Destinations under a new child stayed missing from discovery, and a departed
child's stayed advertised, until some unrelated change. The re-announce fires
only when that relation flips and only to peers whose filter actually
changed, so ordinary tree churn does not multiply announce traffic.
#### Session setup
- A session whose last handshake message is lost no longer stays one-sided.
The initiator sent msg3 once and treated the session as established at once;
when that one datagram was lost, the responder kept waiting for it and
dropped every frame the initiator sent, and nothing sent msg3 again, because
the responder's repeated SessionAck was refused as arriving in the wrong
state. The session stayed that way until the next session rekey, or with
periodic rekey switched off, indefinitely. The initiator now keeps its msg3
and resends it on the handshake resend interval, with backoff, until a frame
from the responder authenticates or `handshake_max_resends` resends have gone
out. The wire format is unchanged: the resend carries the same msg3, and a
responder that already completed the session refuses the duplicate as before.
#### Link and session rekey
- A session rekey this node started no longer stays in flight forever when its
setup or the peer's ack is lost. Nothing resends a rekey setup, and the only
expiry covered a rekey the peer started, so one lost datagram left the
rekey pending and blocked every later one: the session kept its current keys
and stopped rotating them. The rekey now expires on the handshake timeout,
timed from when this node sent its setup, and the next tick starts a fresh
one. A forged ack cannot extend it. Expiries are counted as
`rekey_unanswered`. The wire format is unchanged.
- A forged rekey msg2 no longer takes the link down. The rekey initiator gave
up its handshake before reading msg2 and abandoned the cycle when the read
failed, although nothing authenticates a msg2 ahead of that read. Anyone on
the path who saw the rekey msg1 go out could answer first with a msg2 of the
right size under the index msg1 carries in cleartext. The responder has
already committed its new session by then and cuts over on its next tick, so
the two ends were left on different keys: frames from the responder were
dropped at once, frames to it failed once its drain window closed, and each
end removed the other on the link-dead timeout about 30 s later. A msg2 that
fails the read now leaves the handshake as it was before the read, along
with the msg1 resend schedule and the msg2 dispatch entry, so the
responder's genuine msg2 still completes the rekey. In exchange, every such
forgery now costs the initiator the msg2 key agreement until the cycle ends,
where before only the first one did; the msg1 resend budget bounds that. The
wire format is unchanged.
- A SessionAck that fails to read no longer ends a session rekey this node
started. The handler took the rekey handshake off the session before reading
the ack's msg2 and abandoned the rekey when the read failed, although nothing
authenticates the ack before that read and the only tie to the rekey is the
datagram's source address. The handshake now goes back rolled back to its
state before the read, so the peer's genuine ack still completes the rekey,
and the refusal is counted as `ack_handshake_failed`, as it already was for a
first-contact session. The wire format is unchanged.
- A link rekey whose reply is lost no longer splits the link. The node that
answered a rekey used to switch to the new keys on its own next tick, before
the other side had them; when the reply was lost, frames from the answering
side were dropped until the link was torn down. The answering side now
switches only when a frame on the new keys arrives from the side that started
the rekey, and drops keys that were never adopted after a hold (120 s by
default) so the next rekey can proceed.
#### Session coordinates
- A node with no coordinates cached for a session's destination no longer
sends its own coordinates in their place. The lookup that supplies them falls
back to the node's own coordinates, which a first-contact SessionSetup needs
because its destination field cannot be empty, but the established data path,
the standalone CoordsWarmup and the rekey SessionSetup used the same
fallback. Every receiver files the destination coordinates it is sent under
the destination's address, so a destination reached this way cached its own
address under the sender's coordinates. On a cache miss a data frame now goes
out without coordinates and leaves the warmup budget for the first frames
after the cache is refilled, a standalone CoordsWarmup is not sent, and a
rekey SessionSetup, which can only miss for a direct peer, carries the
coordinates that peer announced. First-contact setup is unchanged. The wire
format is unchanged.
#### Control socket
- `show_links` (`fipsctl show links`) now reports the traffic a link has
carried. Its `packets_sent`, `packets_recv`, `bytes_sent`, `bytes_recv` and
`last_recv_ms` were read from counters on the link record that nothing on
the data plane ever wrote, so every link reported zero however much traffic
it carried, while `show_peers` counted the same traffic on the peer. A link
bound to an authenticated peer now reports that peer's counters, so the two
queries agree for the same `link_id`; a link still in handshake has no peer
yet and still reports zero. The counters follow the peer across address
changes, while the row's `transport_id` and `remote_addr` stay those the
link was created with. The counters cover authenticated link frames only, so
they are not expected to match the transport totals in `show_transports`.
The response shape is unchanged. Fixes #158.
- `show_peers` (`fipsctl show peers`) now reports a peer that has gone quiet
as `stale`. Its `connectivity` was read from a state that nothing outside
the tests ever changed, so every peer read `connected` until it was
removed, including one that had stopped answering tens of seconds earlier.
The value is now derived from how long the peer has been silent: `connected`
while its idle time is at or below `heartbeat_interval_secs`, and `stale`
above it, the same rule that decides whether discovery re-dials an active
peer on the path it already has. The `reconnecting` and `disconnected`
values the open-discovery tutorial described never occurred, and the
tutorial no longer lists them. The response shape is unchanged.
#### Identity and config
- A persistent node whose identity key path cannot be examined now refuses to
start instead of coming up under a new identity. `Path::exists` reports false
both for a key that is absent and for one whose metadata cannot be read, so a
key symlinked onto a volume that did not mount, or one in a directory the
daemon cannot search, read as a first boot: the node generated a fresh
identity, failed to store it, and carried on under an npub that every peer
whose allowlist names the old one refuses. Only a `NotFound` result is now
treated as an absence; any other failure to stat the path aborts the start and
names the path. A dangling symlink likewise aborts rather than being replaced.
The legacy `/etc/fips/fips.key` lookup follows the same rule.
#### Gateway
- Conntrack sessions are matched by address rather than by text, so live
traffic pins a gateway mapping again. The session count searched each
`/proc/net/nf_conntrack` line for `dst=` followed by the virtual IP in its
compressed form (`fd01::1`), while the kernel prints tuples in the full
uncompressed form (`dst=fd01:0000:0000:0000:0000:0000:0000:0001`), so the
count was zero for every mapping on every kernel. Nothing pinned an in-use
mapping, and one whose client did not re-query DNS was reclaimed about two
minutes after its last DNS reference while its traffic was still flowing.
Each `dst=` value is now parsed as an address and compared as one.
- The conntrack table is read once per tick instead of once per mapping, and
the read happens off the runtime thread. The whole file was read and scanned
for each mapping in turn, while the pool lock was held, on the same
single-threaded runtime that serves DNS. The tick now takes one snapshot with
a blocking task before it takes the lock, and the pool does a map lookup per
mapping.
- A conntrack source that cannot be read is reported. It still counts as zero
sessions for every mapping, as it always has, so reclamation keeps working
rather than pinning the whole pool; but the first failure and each change of
outcome after it are now logged, so an unreadable source is no longer
indistinguishable from an idle one. A source that fails identically every
tick is logged at debug rather than warn on a repeat.
- The NAT table is rebuilt in one netlink transaction. A rebuild deleted the
`fips_gateway` table in a batch of its own, discarded that batch's result,
and only then sent the batch that recreated the table, the chains, the
`fips0` masquerade and every per-mapping rule. Between the two sends the
gateway had no NAT at all, and a recreate the kernel refused left the table
absent for good, taking down forwarding for every existing mapping rather
than failing the one change that was being made. The delete and the recreate
now share a single batch, which the kernel applies as one transaction, so a
refused rebuild leaves the previous table in the packet path. The rules sent
are unchanged.
- The gateway's NAT rebuild no longer fails once the table holds more than
about 105 mappings. Each rebuild is one netlink batch. From about 105
mappings the default socket buffers could not hold its acknowledgements, so
rebuilds were logged as failed although they had taken effect. Past about
313 mappings the buffers could not hold the batch itself, and new `.fips`
names past that count got a virtual IP with no translation. In releases with
the gateway through 0.5.1, a rebuild past about 313 mappings also deleted the
whole `fips_gateway` table, which stopped every mapping, the `fips0`
masquerade and the port forwards. The rebuild now sizes its send buffer to
the batch and requests one acknowledgement per batch, and NAT errors now
name the kernel errno. A rebuild that still fails is logged, and the next
successful rebuild installs the mapping.
- A `.fips` query the gateway answers without an address no longer takes an
address from the pool. Every query type was allocated a mapping before the
code looked at what the client had asked for, and an A or HTTPS query was
then answered with NODATA, so any host that can reach the LAN resolver could
consume the pool one name at a time with a query type it is never given an
address for. Only AAAA and ANY allocate now. A non-AAAA query for a name that
already has a mapping still refreshes that mapping's TTL clock, so a client
querying both types does not lose half of its refresh.
- The gateway's virtual-IP pool now limits how many mappings it holds and how
fast it creates them. Any host that can reach the LAN resolver could ask for
one new `.fips` name after another, and each got a mapping until the 65,535
addresses ran out, while every mapping made each NAT rebuild, each pool tick
and shutdown slower. The pool now refuses a new name once it holds 1000 live
mappings, and admits new names at 10 per second after a burst of 50. A
refused query gets SERVFAIL, and the gateway's "Pool allocation failed"
warning says which limit refused it. A name that already has a mapping is
answered before either limit is consulted, so names in use keep resolving
when the pool is full. The limits are compiled in, not configured.
#### Nostr and NAT traversal
- A node no longer publishes NIP-09 deletion requests signed with its routing
key after a NAT traversal attempt. Each request put the node's public
identity next to the ids of its offer and answer gift wraps on every relay it
reached, which the one-time signing keys on those wraps exist to prevent, and
most of the requests deleted nothing, since a relay deletes a gift wrap only
at its recipient's request. A relay that stores the wraps now keeps them
until their NIP-40 expiration; relays that do not store ephemeral events
never held them. The advertisement retraction still sends its deletion
request, since that names an event the routing key signed itself. The
discovery and traversal design documents describe the new behaviour.
#### Packaging (OpenWrt)
- A new OpenWrt install no longer enables and starts `fips-gateway`. The
generated postinst turned it on unconditionally, contradicting the init
script's own header, the package README and the deployment tutorial, all of
which say the service ships disabled and is enabled deliberately. The
documented `service fips-gateway enable` / `service fips-gateway start` steps
are unchanged, and the shipped `fips.yaml` still carries `gateway.enabled:
true`, so enabling the service is all that is needed.
- **The first upgrade to this release re-enables and starts `fips-gateway` on
any router that has the package installed, including one where the gateway
was disabled by hand.** Every released package's prerm disabled the service
on its way out, leaving nothing behind that says whether the operator wanted
it on, so an upgrade cannot tell the two apart and keeps the gateway running
rather than silently turning off a working one. If you had disabled it, run
`service fips-gateway disable` once after upgrading. Later upgrades preserve
whatever state the service is in: the new prerm stops the services on an
upgrade but no longer disables them.
- `start_service` in the `fips-gateway` init script now reads `gateway.enabled`
from `/etc/fips/fips.yaml` before doing anything. Starting a gateway that the
config disables used to hand dnsmasq's `.fips` forwarding to the gateway's
port, add the LAN prefix and advertise the pool route, and only then start a
daemon that exits immediately because the gateway is disabled, leaving `.fips`
resolution pointed at a port nothing listens on.
- The `.ipk` and `.apk` packages now install the same maintainer scripts. The
four script bodies live in `packaging/openwrt-ipk/scripts/` instead of inside
heredocs in the two build scripts, so the scenarios in `testing/openwrt/` run
what ships.
#### Packaging (Debian)
- A `.deb` upgrade whose new daemon cannot start no longer hangs apt. The
postinst started `fips.service` and then `fips-dns.service` with blocking
calls, and because `fips-dns.service` requires the daemon, a daemon that
failed on every start left the second call, apt and everything queued behind
it waiting for ever with no message. Each start is now queued and waited on
for at most 60 seconds. A unit that does not come up has its status printed
and fails the configure step, so apt exits non-zero and names the unit; a
masked unit, or one whose condition is not met, is reported and skipped.
- The `.deb` maintainer scripts now manage `fips-gateway` with the rest of the
package's services. An upgrade stopped the daemon, which the gateway
requires, and never brought the gateway back, so an operator who had enabled
it lost it until the next reboot; removing or purging the package left the
gateway's enablement symlink behind, pointing at a unit file that no longer
exists. The gateway is now stopped before the daemon on upgrade and
restarted afterwards only when it is enabled and the daemon came up, and it
is stopped and disabled on remove and purge. A gateway that does not come
back is reported but does not fail the upgrade.
- The `.deb` now declares `libgcc-s1 (>= 4.2)`. All four binaries link
`libgcc_s.so.1`, but cargo-deb removes every libgcc entry from the
dependencies it derives, so the package never said so. `libc6` depends on
`libgcc-s1` on Debian 12 and Ubuntu 22.04, 24.04 and 26.04, so installs there
were not affected. A new check, `testing/check-deb-depends.sh`, runs
`dpkg-shlibdeps` over the package's binaries on every build and fails the
build when the declared `Depends` leaves out a library the binaries need, or
states a floor lower or higher than the one they need. A dependency the
packaging tool drops, including one it drops after only a warning when it
cannot resolve a binary, now fails the build instead of shipping.
- `-V` on binaries built into the Linux packages now includes the source
revision, as `<version> (rev <git-hash>)`. The build image had no git, so
every container-built binary printed the version alone. A package built from
a git worktree still has no revision, because the worktree's git directory is
outside the tree the build sees. The build image's tag now includes a hash of
its Dockerfile, so a host with an older image cached builds a new one instead
of reusing it.
- `packaging/debian/build-deb-container.sh` now returns the package it just
built. It picked the most recently modified `fips_*.deb` in the output
directory that sorted last by name, so a package with a higher version left
there by an earlier run was returned instead.
- Purging the `.deb`, or running `uninstall.sh` from the tarball, now removes
the `.fips` DNS routing when `fips-dns` was not running at the time. The
cleanup removed the dns-delegate file from the wrong directory, never removed
the systemd-resolved global drop-in, and restarted no resolver, so the host
kept sending `.fips` queries to `[::1]:5354`, where nothing listens any more,
and `.fips` lookups timed out. Both scripts now remove all four files
`fips-dns-setup` can write, and restart systemd-resolved or reload dnsmasq or
NetworkManager when they removed that resolver's file and it is running. A
failed restart is reported and does not fail the removal.
- The `.deb` now recommends `nftables`. `fips-firewall.service` runs
`/usr/sbin/nft`, so enabling it on a host without nftables failed at start.
It is a recommendation rather than a dependency because the firewall unit is
opt-in and the daemon itself does not need `nft`.
#### Packaging (AUR)
- The release `PKGBUILD` now lists `dbus` as a runtime dependency. The `fips`
binary links `libdbus-1`, and the `fips-git` package already declared it.
- Both `PKGBUILD` files list `nftables` as an optional dependency, for
`fips-firewall.service`.
#### Packaging (FreeBSD)
- The daemon's log, `/var/log/fips.log`, is now rotated. The package ships a
newsyslog entry that keeps five compressed generations of 1000 KB, and the rc
script starts `daemon(8)` with `-H` so it reopens the log after a rotation.
The log used to grow without bound.
#### Windows
- Windows now keeps its config, key, hosts and peer ACL files in
`C:\ProgramData\fips`, the directory the service installer writes to. The
config search, the key directory and the peer ACL defaults disagreed: the
service found none of the installed files after a reboot and ran on
defaults, `fipsctl keygen` wrote to the per-user `%APPDATA%\fips`, and a
`peers.deny` placed beside the hosts file was never read, so the ACL failed
open. A key left in `%APPDATA%\fips` is still used when the new directory
has none, with a note to move it. For one release, a `peers.allow` or
`peers.deny` left at the old `\etc\fips` location is still read when the
new directory has no such file, with a warning naming both paths.
- The Windows service now writes its log to `C:\ProgramData\fips\fips.log`,
rolled at 10 MiB with four old files kept. A service has no standard output,
so everything the daemon logged in service mode was lost, including
config-load failures and panic messages. A foreground run still logs to the
console.
- The Windows service installer now restricts `C:\ProgramData\fips` to
SYSTEM and Administrators. The directory inherited `C:\ProgramData`'s
default ACL, which lets any local user read the files in it and create new
ones, so any local account could read the node's key, or create a missing
`fips.key`, `fips.yaml` or `hosts` that the service then used. The installer
creates the directory with the restricted ACL, or replaces the ACL of an
existing one, resets the files already in it to inherit it, and refuses to
continue if the directory or anything in it is a link or a folder, or if the
directory is owned by another account. Rerun the installer after moving files
into the directory. A foreground run from an unelevated prompt can no longer
read the files there.
## [0.5.1] - 2026-09-06
### Fixed
#### Discovery
- A node no longer relays away the answer to its own lookup. A request is
flooded to every tree peer whose bloom filter claims the target, so a false
positive can send a copy out into the wider network and circulate it back to
the node that originated it. The only identity test on arrival was whether
the request named this node as the target, which a lookup this node
originated never satisfies, so the copy was filed in the request dedup cache
as ordinary transit under this node's own `request_id`. When the target
answered, the reply was reverse-path forwarded to the peer that looped the
request, the pending lookup was never satisfied, and discovery reported that
its requests went unanswered while the answers were in fact arriving. An
inbound response is now matched against this node's outstanding lookups
before the transit dedup record, and a returning copy of this node's own
request is dropped as the duplicate it is rather than recorded, so that id
never enters the transit cache at all. This was a race rather than a hard
failure: a reply that beat the looped copy found a clean cache and
succeeded, and the failure grew likelier as the bloom fill ratio rose.
Contributed by Arjen.
- A lookup request of this node's own, returning to it, is no longer counted as
a duplicate from the peer that delivered it. The fix above drops that copy,
and it recorded the drop under the existing `req_duplicate` rejection, whose
documented meaning is that a peer resent a request. A returning copy has a
nonzero floor in healthy operation and rises with the bloom fill ratio, so
folding the two together put a permanent number on a counter an operator
reads as neighbour misbehaviour, and made the two events indistinguishable.
It now has its own rejection reason and counter, `req_own_loopback`, shown in
`fipstop` as "Own Loopback". `req_duplicate` returns to meaning only what it
says.
#### Packaging
- The Linux `.deb` and the systemd tarball now install and run on Debian 12 and
Ubuntu 22.04. Every Linux artifact from v0.3.0 through v0.5.0 was built on the
newest available runner, whose C library made the standard library's `pidfd`
references a hard `GLIBC_2.39` version requirement instead of the weak,
runtime-checked ones it is meant to compile to. The loader refuses an image on
that entry alone, so `fips`, `fipstop` and `fips-gateway` could not start;
`fipsctl` was unaffected, which is why an install that was checked by running
it looked healthy while the daemon was dead. No source code caused this and
none was changed. The Linux artifacts are now built in a container pinned to
the oldest supported distribution, declared with the floor in
`packaging/build-floor.env`, and every producer on the release path runs
`testing/check-glibc-floor.sh` on what it made, so a package or a tarball that
would not load fails the build rather than reaching a user. The deprecated
host-build targets in `packaging/Makefile` are not on that path and are not
floor-checked. The declared
dependency is derived from the binaries instead of hand-written, so it states
the floor it was built against.
- The `.deb` install suite no longer hangs when the daemon it installed cannot
run. It started `fips-dns.service` with no timeout, and that unit is
`Type=oneshot` with `Requires=fips.service`, so a daemon that cannot execute
is restarted every five seconds for ever, the oneshot start job is never
dispatched, and `systemctl start` never returns. The suite then produced no
failure line, no results line and no exit status at all, which is the whole
class of fault it exists to find: it stopped reporting at exactly the point it
was most needed. Observed at 21 minutes against a package whose binaries could
not load. The start is now queued rather than waited on, with a bounded wait
for the unit to become active, so a dead daemon fails the suite instead of
stalling the run that gates artifact publication.
## [0.5.0] - 2026-08-30
### Added
#### Platforms
- FreeBSD support for the daemon, `fipsctl`, and `fipstop`, on x86_64 only:
native TUN datapath (TUNSIFHEAD address-family framing, kernel-assigned
`tunN` device name as with `utun` on macOS), clean service teardown,
`/usr/local/etc/fips` config search path, and `/var/run/fips`
control-socket default (both shared with macOS). The `hosts`,
`peers.allow` / `peers.deny` and `fipsctl keygen` defaults follow the
same `/usr/local/etc/fips` layout as macOS; that move and its startup
warning are described by the three macOS path entries under `[0.4.2]`
below, which this platform inherits.
`fips-gateway` remains Linux-only. Native `.pkg` packaging under
`packaging/freebsd/`
(`make freebsd`) with rc.d services, a `fips` control-socket group,
service stop/restart across `pkg upgrade`, and `.fips` DNS integration
for `local_unbound`/`unbound`/`dnsmasq`. mDNS LAN discovery works via
`mdns-sd` 0.20 (`socket-pktinfo` 0.4.1, the first release that builds
on FreeBSD). Daemon logs now disable ANSI color when stdout is not a
terminal (all platforms). No aarch64 FreeBSD artifact is produced and
that combination is not verified here.
- Android-ready core, supported as an embedded crate rather than as a
standalone daemon: there is no Android daemon artifact, since the library is
the delivery form. The daemon's
desktop transports and TUN operations are
gated by `target_os` rather than by Cargo features, so a plain `cargo build`
compiles for every target with no flags and Android self-excludes the raw
Ethernet transport as Windows already did. `Node::enable_app_owned_tun()`
gives an embedder that owns the TUN file descriptor (an Android
`VpnService`, for instance) a channel pair for exchanging IPv6 packet bytes
with FIPS instead of FIPS creating a system TUN device, and `start()` then
performs no system-TUN or `CAP_NET_ADMIN` operations. Packets entering this
way bypass `handle_tun_packet`, so the embedder must push only
`fd00::/8`-destined packets and clamp TCP MSS on outbound SYNs. Desktop
builds are unchanged and no Cargo features are introduced.
- `Node::dns_local_addr()`, the DNS companion to the app-owned TUN interface
above.
An embedder whose resolver is pointed into the tunnel has no system socket
aimed at the built-in `.fips` responder, so the accessor reports the address
read back off the bound socket: `dns.port = 0` therefore yields the
kernel-assigned port, and it returns `Some` only while the responder is up.
Read it once, after `start()` returns and before the node is moved into a
background task; it is not a liveness feed (#136).
#### Native datagram API
- A native datagram API addressed by public key, off by default, with a
surface that may still change. A client process opens a flow to a
peer's public key on a chosen port and sends and receives datagrams on a
file descriptor the daemon hands it: no IPv6 emulation, no TUN device and no
DNS, a datagram travelling from key to key. **The wire needs no change and
gets none.** Every FSP data packet has carried a port pair inside its AEAD
envelope since v0.2.0 and port 256 is simply the IPv6 shim, so what was
missing was a way for a program to ask for a port of its own and be handed
the traffic. The x-only public key is the address and an npub is that key
written in bech32, so converting between them is a local encoding rather
than a lookup or a name service; the 16-byte node address that travels on
the wire is a truncated hash of the key, does not invert, and appears
nowhere a client can see. A listener is a descriptor: the daemon writes one
message per arrival to it, carrying the new flow's descriptor and the peer's
address, so poll, select and epoll work on a listener and accepting is a
`recvmsg`. There is no accept command and no reject command, and refusing a
flow is closing the descriptor you were handed. The Rust surface mirrors
`std::net`, with `FipsStream::connect`, `FipsListener::bind`, `incoming`,
`accept`, `io::Result` and an errno mapping rather than a bespoke error
type, plus `set_nonblocking`, `AsFd` and four deadline methods under the
names and signatures `std::net` uses for the same jobs. One rule has no
counterpart in Berkeley sockets and a client author must know it: the v1
wire carries no half-close, so nothing peer-driven ever closes a flow, and a
server written to read until the flow ends waits for a signal that cannot
arrive. The listener uses `SOCK_SEQPACKET` on Linux and `SOCK_DGRAM` on macOS
and FreeBSD; both keep the message boundaries the API's contract with its
clients rests on. macOS does not implement `SOCK_SEQPACKET` for `AF_UNIX` at
all. FreeBSD accepts the constant and returns a socket that is not an
atomic-record socket, so consecutive messages coalesce and a zero-length
message is dropped rather than delivered; both were measured on the FreeBSD
15.1 image rather than reasoned about. The three kernels signal a closed
peer differently and were measured too, so the receive path treats a Darwin
or FreeBSD `ECONNRESET` as end of file alongside the `POLLHUP` and
zero-byte read that Linux gives. `EAGAIN` is deliberately not in that
company: it means the socket is empty and the peer alive, so it stays an
error and the caller waits again. **A zero-length payload has one known
limitation.** A closed peer latches `POLLHUP` while its messages are still
queued, so that flag alone cannot say whether a zero-byte read is an empty
datagram or the close; the receive path also asks `FIONREAD`, and bytes still
queued prove a further message is waiting. That leaves one case unresolved:
a zero-length datagram that is the last message before a close is reported as
the close, because reading it drains the queue and a zero-length message
contributes no bytes to `FIONREAD`. A client should not give a zero-length
payload a meaning of its own, and should carry a one-byte discriminator
instead.
#### OpenWrt mesh
- OpenWrt 802.11s open-mesh backhaul: router-to-router radio links with FIPS
providing all encryption, authentication and routing over bare L2 neighbor
links. The mesh runs open with `mesh_fwding 0`, since SAE would duplicate the
Noise layer and force ath10k raw mode, and FIPS's spanning tree is the
routing layer. `fips-mesh-setup` is an opt-in UCI helper creating a per-radio
mesh point (`radio0` to `fips-mesh0`, `radio1` to `fips-mesh1`, with a
free-index fallback and a collision guard); radio setup stays opt-in because
a package must not commandeer radios on install. A dual-band router gets one
instance per radio, and FIPS treats the two paths as failover rather than
multipath: cross-connection resolution keeps one active link per peer and the
second band stands by, re-establishing after keepalive timeout. The shipped
`fips.yaml` carries the `mesh0`/`mesh1` Ethernet-transport entries commented
out, so a stock install that never creates them logs no per-boot
interface-missing warning; the helper uncomments the matching block when it
creates the interface and re-comments it on remove (#123).
- OpenWrt open `!FIPS` access SSID, stacked on the mesh backhaul above: every
FIPS router broadcasts the same open SSID, forming one standard ESS that
phones and laptops save once and roam between natively, with the Noise IK
handshake as the only security layer. The leading `!` sorts it to the top of
alphabetically ordered network pickers, and the encryption type must be
uniform across routers or clients treat the ESS as different saved networks.
`fips-ap-setup` is an opt-in UCI helper creating the `fips-ap0` open AP on an
isolated network with a static ULA /64 and RA-only odhcpd addressing:
stateless SLAAC with no DHCP, the minimum that satisfies Android's
provisioning check, behind a locked-down `fips_ap` firewall zone with no
path to `br-lan` or the WAN, reaching only ICMPv6, mDNS and the FIPS
transports. There is no internet by design, so phones keep cellular as their
default route (#126).
#### Node lifecycle
- A bounded graceful-shutdown drain phase, controlled by the new
`node.drain_timeout_secs` (default 2s). On the shutdown signal the node
broadcasts Disconnect to all peers and then keeps serving for that window,
exiting early once all peers are gone, so in-flight traffic settles and peers
observe the disconnect before the transports close, where previously teardown
was immediate. The published node state gains a `Draining` variant visible
via control queries during the window. The immediate stop path used by
non-daemon callers is unchanged.
#### Transports & config
- The UDP transport's listen socket descriptor can now be handed to an
embedder, for hosts that associate a socket with one interface or network
and steer inbound traffic by that association rather than routing by
destination address. On such a host a peer reachable only over a secondary
network fails in a way FIPS can neither see nor fix: the address is
well-formed, the send succeeds, the peer replies, and the host discards the
reply before it reaches our socket, so the link retries msg1 forever with no
error surfaced anywhere. The correction is a socket option chosen against
host state FIPS has no basis to reason about, so the descriptor goes to
whoever does. Call `Node::enable_app_owned_udp_fd()` after `Node::new` and
before `start()`, and read `AppOwnedUdpSocket { instance, fd }` off the
returned channel once the transport is up, following the existing
`enable_app_owned_tun` contract. One message is sent per UDP transport that
binds, so a multi-listener configuration yields all of them; nothing is sent
when no UDP transport is configured or one fails to bind, so an embedder
tells "no socket" from "here is the socket" by the receive timing out. The
`instance` field is the name the listener was configured under, `None` for a
single unnamed instance, and it is what makes more than one listener usable:
transports are created by iterating a map, so arrival order is luck, and an
embedder whose whole purpose is to bind one socket to one network would
otherwise have to guess which socket it just received. Guessing wrong pins
one lane's socket to the other lane's network, which is the failure the
interface exists to correct. FIPS keeps owning the socket, and
the descriptor carries no promise beyond "this is the transport's socket,
and it is open now". Two limits: the per-peer connected-UDP sockets that
Linux and macOS open after `start()` returns are not covered, and a
transport that adopts a socket handed in by the traversal bootstrap does not
deliver one. Unix only, since the Windows UDP backend has no descriptor.
- A peer address may name which *instance* of a transport it belongs to, as
`transport: "udp/aware"` rather than `"udp"`, where the part after the slash
is the key the transport was configured under. A node running several
instances of one type could not be told them apart by a dialer: both bind
wildcard sockets, so the address-family test matches either, and selection
fell through to the lowest transport id. One socket carried every dial and
the other never carried traffic. A bare type is unqualified and matches any
instance, which is what every existing configuration and caller produces, so
nothing changes for a node that does not use the syntax. A qualified name is
never substituted with a different instance: that is the wrong-lane dial the
syntax exists to prevent, so an unmatched name fails the address instead, and
the same name is what an embedder binds by and what the dialer routes on. The
slash is already how FIPS qualifies an instance inside an address
(`eth0/aa:bb:...`) and cannot occur in a type name. Only UDP resolves an
instance name today; an address that qualifies any other transport type is
refused rather than matched loosely. **Because a qualified name never falls
back, the configuration validator rejects one that no configured transport
answers to**, naming the peer, the instance asked for and the instances that
exist. Otherwise the address would simply be skipped at every dial, which is
invisible for a peer that has a second address that works: the lane would
never carry traffic and nothing above debug logging would say so.
#### Bluetooth LE
- The BLE transport is refactored so the code common to Linux and Android is
implemented once, with a separate backend for each platform, and gains a
reliability pass closing several defects a two-node field capture surfaced.
The module gate is now `ble_available`, meaning glibc Linux or Android
rather than `target_os = "linux"`, so the transport is no longer conflated
with one of its backends; musl is excluded and a platform with no concrete
backend fails the build rather than compiling a transport that starts,
reports itself up and never peers. The receive path recovers packet
boundaries from the 4-byte FMP common prefix instead of assuming one read
returns one whole packet, which held only for BlueZ's `SOCK_SEQPACKET` and
not for a stream-oriented backend such as Android's `BluetoothSocket`, where
fragments shipped up as runts that FMP and Noise rejected and coalesced
tails were silently truncated. A peer is recognised by node identity rather
than by its link address, so a phone rotating resolvable private addresses
no longer presents as a new device on every rotation and defeats the
already-connected guards. The L2CAP PSM is decided by the backend: `listen`
reports the PSM it actually bound, the advertisement carries it beside the
128-bit service UUID, and a dialer learns it from the scan, which is what
Android and macOS require since both assign the PSM rather than letting an
application request one. Android gains an embedder-supplied radio backend
driving a radio the embedder installs into a per-node slot with
`Node::enable_app_owned_ble_radio()`, called after `new()` and before
`start()`, because its Bluetooth APIs sit behind a permission and
foreground-service model only the application can satisfy. Stopping the
transport now stops the scan as well as the advertisement, which matters
only where the embedder owns the radio: BlueZ ends discovery when the
scanner's event stream drops, but an app-owned radio went on scanning for
the life of the process. Probe retry backs off by powers of two with a
capped retry book, and each connect outcome has its own counter and
structured log line carrying the role, outcome, PSM and time to
conclusion. Inbound handshakes run off the accept loop, eight in flight
and aborting the oldest at the bound, where the exchange previously ran
inline and held the loop for its full 5-second deadline, making effective
inbound concurrency one.
#### Observability & measurement
- An optional tick-body profiler behind the new `profiling` Cargo feature,
**off by default**. When enabled, `fipsctl profile tick on [--dir PATH]` /
`off` / `status` starts and stops a capture at runtime with no restart. Each
capture writes one tab-separated file (default `/var/log/fips`, capped at
32 MB) carrying, per ten-second interval, the exact count, max and total for
every step of the rx-loop tick arm, the whole-tick span, and gauges for ticks,
peer count, the gap between successive tick-arm entries and the resulting
arm-starvation delay. With the feature off the instrumentation macro is a pure
pass-through, so a default build contains no timing code on the tick path.
`LogsDirectory=fips` was added to the packaged systemd units so the capture
directory is created and cleaned up declaratively.
- `fipsctl probe <npub|hostname>` answers, for one target, where it sits in
the spanning tree relative to this node and whether this node can actually
reach it. The work runs as five stages that report separately, `bloom`,
`discovery`, `path`, `session` and `rtt`, because one verdict covering
several findings is what sends an operator to the source: "no peer's filter
claims this address" says the mesh has never heard of the target, while "a
filter claimed it and nothing answered" says the opposite. The probe opens
an FSP session, waits for one MMP receiver report to yield a round-trip
time, and tears down only what it opened. A session that existed before the
probe started is never torn down, ownership is decided at the moment of
action rather than once at the start, and it is re-checked before teardown,
so a session adopted by traffic underneath the probe is left alone. **The
path printed is the least-common-ancestor walk computed from the two sets of
coordinates**, which is the worst-case fallback route rather than the route
a packet necessarily takes: a cut-through between peers can deliver in fewer
hops, so the tree distance is an upper bound. Nothing traverses the mesh to
confirm the hops, and a real per-hop trace needs a wire message that does
not exist. **Nothing here changes the wire format**; the probe is built from
messages that already exist. The control socket carries three new commands,
`probe_start`, `probe_poll` and `probe_cancel`, each returning in well under
a millisecond with the stages advanced on the daemon's tick, because a probe
needs a mesh lookup, a Noise XK handshake and at least one remote MMP tick,
which no single control round-trip could survive inside the socket's
five-second timeout. A probe that runs and finds a problem is not an error
response: the status is `ok` and the failure sits in the per-stage verdicts,
and error responses stay reserved for malformed or inadmissible requests. On
a terminal the stage block is redrawn in place with a running elapsed on
whichever stage is working; piped or redirected there is no cursor to move,
so each row prints once, at the moment it settles, and the transcript ends
up the same block a terminal leaves behind. `--json` emits exactly one
document at the end, so a script parsing the report does not have to skip
past progress output.
#### Packaging & deployment
- A NixOS module and an overlay are exposed from the flake, so a flake consumer
enables the daemon with one line rather than hand-rolling a systemd unit.
`overlays.default` adds `pkgs.fips`, and `nixosModules.default` provides
`services.fips.*`: `enable`, `package`, `configFile`, `openFirewall` (UDP 2121
and TCP 8443) and `dns.enable`, which routes `.fips` to `[::1]:5354` through
systemd-resolved declaratively rather than with setup and teardown scripts.
`packaging/nixos/README.md` documents it with a full consumer `flake.nix`.
Contributed by Arjen.
- `fipsctl address [npub|hostname]` prints a node's `fd00::/8` mesh address and
nothing else, without contacting the daemon. With no argument it derives the
local node's address from `fips.key` in the default key directory, falling
back to the world-readable `fips.pub` beside it; `--key PATH` names a key or
public key file elsewhere. This lets an installer or image build write a mesh
address into a config file at a point where no node is running and none can
be, and keeps the derivation in one place rather than reimplemented by
whatever needs it.
- `packaging/debian/build-deb.sh --features <list>` builds the `.deb` with a
Cargo feature list, which is how an instrumented package is produced for a
measurement run. The auto-derived dev Version gains a matching `+<features>`
marker, so a feature build and a default build of the same commit are no
longer indistinguishable: without it the two carry byte-identical versions,
an install of one over the other is an apt no-op, and the running node offers
no way to tell which one it has. The marker sorts above the unmarked build, so
installing a feature build is an upgrade and reverting to the default build is
a downgrade: revert with `dpkg -i` rather than `apt install`. `--features` is
refused together with `--no-build`, which would stamp the marker onto binaries
the features never reached.
### Changed
#### Naming: discovery split into lookup and rendezvous
- The mesh-lookup control-metrics family is now emitted under the key
`lookup` in `fipsctl stats metrics` and `show routing`. The former key
`discovery` is still emitted as a deprecated alias carrying identical
counters; update dashboards and alerts to read `lookup`.
- The overloaded `node.discovery.*` config table was split into
`node.lookup.*` (mesh-lookup scalars: `ttl`, `attempt_timeouts_secs`,
`recent_expiry_secs`, `backoff_base_secs`, `backoff_max_secs`,
`forward_min_interval_secs`) and `node.rendezvous.*` (peer rendezvous:
`nostr.*`, `lan.*`). A deployed `node.discovery:` block still loads and is
folded into the new tables with a one-time deprecation warning; migrate your
`fips.yaml` to the new keys. This includes the two keys v0.4.2 introduces
under the old spelling, so an operator upgrading straight from 0.4.2 does not
have to infer them: `node.discovery.nostr.max_concurrent_offers_per_npub` and
`node.discovery.nostr.signal_ttl_secs` are now
`node.rendezvous.nostr.max_concurrent_offers_per_npub` and
`node.rendezvous.nostr.signal_ttl_secs`.
- The Ethernet transport's per-interface `discovery` flag was renamed to
`listen` (`transports.ethernet.*`) to match the symmetric `announce`
(transmit) / `listen` (receive) neighbor-beacon vocabulary. The old
`discovery:` key is still accepted via a serde alias, so deployed configs
continue to load unchanged; `Config::to_yaml()` re-emits it under the
canonical `listen:` name. Update your `fips.yaml` to `listen:`.
- `fipstop`'s Routing State pane renames its `Discovery Requests` and
`Discovery Responses` sections to `Lookup Requests` and `Lookup Responses`,
and reads those counters from the canonical `lookup` key rather than from the
deprecated `discovery` alias. The counters themselves are unchanged, so an
operator who knows the pane by its old section labels is reading the same
numbers under new names.
#### Library surface and internals
- The protocol layers were restructured into sans-IO cores with the I/O kept in
a thin shell, and two new crate-root modules, `nostr` and `mdns`, own peer
rendezvous and LAN discovery. The crate root also gains the `is_punch_packet`
helper and the `CoordError`, `MtuExceeded`, `COORDS_REQUIRED_SIZE` and
`MTU_EXCEEDED_SIZE` exports. This is an internal reorganization with no
operator-visible behaviour change and no wire change: encoded bytes and decode
decisions are identical. Its two consequences a reader will feel are the
library surface under Removed below and the tracing targets in the next entry.
- Tracing targets follow module paths, so the module relocations in this release
move them. `fips::discovery::nostr::*` becomes `fips::nostr::*`, mDNS moves to
`fips::mdns::*`, and the protocol subsystems move under `fips::proto::*`
(`fips::tree` to `fips::proto::stp`, `fips::bloom` to `fips::proto::bloom`,
`fips::protocol` to `fips::proto::*`, and the mesh-lookup subsystem from
`fips::discovery` to `fips::proto::lookup`). An existing `RUST_LOG` filter
naming an old target still parses and simply stops matching, so the symptom is
missing log lines rather than an error, and a filter that has gone blind looks
exactly like a subsystem that has gone quiet. Update `RUST_LOG` filters,
journal-watch recipes and any log-scraping alert accordingly. The four targets
named explicitly in source rather than derived from a module path
(`fips::config`, `fips::instr`, `fips::node::handlers::handshake`,
`fips::node::handlers::rekey`) are unaffected.
#### Node health
- Node health is determined at start completion instead of unconditionally
reaching a single running state. **Zero transports up is now fatal**: the
node tears down cleanly and the daemon exits with an error, where it
previously came up and served nothing. Any configured optional child that
failed to start, meaning a transport beyond the first, Nostr, mDNS, TUN, DNS,
or a worker pool, leaves the node degraded but serving, with a warning naming
what failed, and all configured children up is full health. A child the node
was never asked to run does not count against it. The published node state
gains `Degraded` and `Failed`, both visible via control queries, with
degraded operational and failed not. Exit detection for the DNS task, the two
TUN threads and mDNS also re-evaluates health at runtime, so a child that dies
after a healthy start now shows as degraded. For Nostr the watched children
are the three service loops that cannot return by design (the inbound notify
loop, the advert publisher and the refresh ticker); `connect_task` and
`relay_startup_task` are deliberately not watched, because `Client::connect()`
returns as soon as it has spawned the per-relay background tasks, so a
finished handle there carries no health signal. Transports and worker pools
expose no runtime-exit signal yet and are unchanged.
#### Peer handshake
- `node.rate_limit.handshake_resend_interval_ms` no longer governs the **first**
outbound handshake resend. When msg1 is sent, the peer state machine arms the
first retransmit deadline from a hardcoded 1000 ms constant
(`HANDSHAKE_RETRANSMIT_INTERVAL_MS` in `src/peer/machine.rs`), because the
machine-armed timer rather than the connection's stored deadline is now the
due signal the retransmit driver fires on. The key still governs the second
and later resends, alongside `node.rate_limit.handshake_resend_backoff` and
`node.rate_limit.handshake_max_resends`. The constant equals the shipped
default of 1000, so a deployment that never overrode the key sees no change;
one that raised or lowered it will find the first resend still firing at
1000 ms. The limitation is recorded at the site in the source.
#### Data-plane / transports
- Connected UDP peer drains now batch macOS receives with `recvmsg_x(2)`,
matching the wildcard UDP receive path instead of issuing one `recv(2)`
syscall per queued datagram.
- Routing next-hop selection now visits borrowed peers and coordinates instead
of allocating candidate snapshots for each forwarded packet.
- A connected UDP socket that cannot open now names the syscall that failed and
the address it was operating on, the local address for `bind` and the peer
address for `connect`. Both paths previously returned a bare OS error that
the caller wrapped identically, so a field report of `Address already in use`
could not be attributed to either, and the two have entirely different
causes: on Linux a UDP `connect(2)` to a 4-tuple another socket already holds
returns `EADDRINUSE`, which is not the same fault as `bind` refusing the
local address. A node at roughly 245 peers was emitting this three times a
second across nine peers with no way to diagnose it.
#### Observability
- The warning raised when a lookup response carries a path MTU below the
actionable floor now names the request it refused, as a `request_id` field on
the log line. Every other warning that handler emits already carried the
correlator, and this one could not: it is raised while the cached-coordinates
effect is applied, after the response itself has gone out of scope. An
operator reading the refusal therefore had a counter and a peer name but no
way to tie the line to a particular exchange, or to the sibling lines logged
for it. **Only the log line changes**: the response is still accepted, the
coordinates are still cached, the sub-floor path MTU is still discarded, and
the same counter is still charged.
#### Docs & contributor tooling
- Source comments and doc comments were corrected across the modules this
release restructured. Unresolvable internal references and planning-phase
locators were removed from comments that exist only on this line, stale
view-trait documentation was rewritten to describe the shape the restructuring
produced, and stale liveness claims in the peer machine, executor and timer
documentation were corrected. No code changed.
#### Packaging & deployment
- Every GitHub Actions reference in the workflows and composite actions that
exist only on this line is now pinned to a full commit SHA, so the whole
`.github` tree is pinned or explicitly justified. The branch carries 75
action references, of which 71 are pinned to a 40-character commit SHA with
the mandatory trailing version comment and 4 are left on mutable tags by
explicit allowance (`dtolnay/rust-toolchain@nightly` and three
`taiki-e/install-action@nextest`); `testing/check-action-pins.sh` reports the
same 75 and passes. Nine of those references were pinned here, in
`package-freebsd.yml` and an `android-check` job block, neither of which
exists on the maintenance branch: the original pinning sweep was authored
there, never saw these files, and they arrived through the merge still on
mutable tags, at which point the guard that shipped with the sweep failed the
branch exactly as intended. The general lesson is worth keeping with the
guard: a checker authored on the earliest branch is only as complete as that
branch's file set, and merging it upward gates files it has never swept.
### Deprecated
- The `discovery` metric-family key (control-socket JSON), renamed to `lookup`.
It is dual-emitted alongside the new `lookup` key during a migration window
and will be removed.
Migrate dashboards/alerts from `discovery.*` to `lookup.*`.
- The `node.discovery.*` config table. Its keys were split into `node.lookup.*`
(mesh-lookup) and `node.rendezvous.*` (peer rendezvous). A legacy
`node.discovery:` block still applies for now with a deprecation warning and
will be removed; migrate to `node.lookup.*` / `node.rendezvous.*`.
- The Ethernet `transports.ethernet.discovery` flag, renamed to
`transports.ethernet.listen`. The old key is still accepted via a serde
alias and will be removed at the v2 cutover; migrate to `listen`.
### Removed
- **Source-breaking for consumers of the library crate**: the crate-root modules
`bloom`, `discovery`, `mmp`, `protocol` and `tree` are gone, along with the
crate-root `HandshakeState`, `PeerConnection`, `PeerSlot` and `ProtocolError`
re-exports. The protocol cores moved into an internal `proto` module and are
reached through crate-root re-exports instead: tree types through
`proto::stp`, bloom types through `proto::bloom`, the FSP, STP, lookup,
routing and FMP wire types through their matching `proto::*` submodules, and
`PromotionResult` / `cross_connection_winner` through `proto::fmp` rather
than `peer`. **The `HandshakeState` removed here is the peer connection-phase
enum, not the Noise handshake type of the same name.** The Noise type is
untouched, still lives at `fips::noise::HandshakeState`, and it is that type,
not this one, that the `[0.4.2]` entry below about four public types gaining
`Drop` describes. `ProtocolError` is replaced by `fips::Error`, whose
`Malformed` variant now carries a `&'static str` rather than a `String` and
which gained `BadSizeClass`, `BadCoord` and `BadBloom` variants, so the
diagnostic text changed with it. `PeerSlot` and the `PeerConnection` resend
API were unused and are deleted. `Node::connections()` is now `pub(crate)`
and yields the internal peer machine rather than a `PeerConnection`; a
consumer that walked links through it should use `Node::peers()`,
`Node::get_peer()` and `Node::peer_count()` over `ActivePeer`, which remain
public. Nothing about the behaviour of the shipped binaries changes, and
nothing on the wire changes: encoded bytes and decode decisions are identical.
### Fixed
#### Peer and link state
- A peer that goes away no longer leaves its path-MTU state behind. The record
of which transport link-seeded a destination's path MTU had no removal site
on any lifecycle, so the map grew one entry per peer this node had ever
linked with and held them for the life of the process. Both the stored value
and its seeding record are now released when the path they describe goes,
together rather than separately: releasing the record alone would leave the
value with nothing recording where it came from, and a peer returning over a
wider transport would then be refused by the never-loosen rule indefinitely.
A peer whose link is still up has both written straight back by the reseed
that already follows.
- Removing a link now clears every reverse-lookup key it was inserted under.
The removal rebuilt one key from the link's own remote address, so an entry
inserted under a different address form — which the cross-connection
promotion arms do, keying on the packet's source address — outlived the link
it named. An entry a newer link has since claimed is still left alone.
- A rejected handshake no longer strands a session index or a retry schedule.
Two reject arms freed neither, which the `next`-line versions already do;
master now carries the same shape, so an ACL-rejected outbound dial is
rescheduled and an abandoned rekey index is returned rather than held.
#### Transport
- The UDP listen socket's address-reuse flags are now set after its bind rather
than before. Before the bind they mean the kernel may hand back a port another
flagged socket already holds, so a second daemon binding the same configured
address started silently and shared the port, with the kernel splitting
inbound datagrams across the two receive loops on the source 4-tuple, where
the second daemon should have failed with `EADDRINUSE`. After the bind they
mean what was actually wanted, that the per-peer connected sockets may later
join the port. Those connected sockets are the joiners and keep their flags
before their own bind, which is where they belong.
- A BLE connection the pool refused is no longer counted as an established
link. The scan and probe loop recorded `connections_established`, resolved
the address out of the retry book and handed the peer up to the node layer
after `ConnectionPool::insert` had already refused the connection and
dropped it. Reaching the refusal needs a full pool with no evictable slot,
and every BLE connection is built non-static, so only `max_connections: 0`
gets there.
#### Control socket
- `disconnect` on the control socket now closes the transport connection
rather than only the peer. It notified the peer and freed every node-side
structure, sessions, indices, links, address mapping, tree and bloom state,
and never touched the transport, so on a connection-oriented transport (TCP,
Tor, Nym, BLE) the pool entry, the socket and its inbound-slot accounting
survived the peer the node had just forgotten, until the far end closed or
the receive loop errored. An operator who disconnected a peer to free a slot
did not free the slot. No effect on UDP, Ethernet or loopback, whose
`close_connection` is the connectionless no-op. Still not addressed:
`disconnect` reports `peer not found` for an identity that is only
mid-handshake, so withdrawing a peer during its handshake leaves that leg
resending msg1 until the handshake timeout bounds it.
- `connect` on the control socket now tries the address it was given for a
peer the node is already connected to, instead of reporting success without
doing anything. The command built an ephemeral peer configuration and handed
it to the ordinary dial path, which returns success the moment the peer is
already held, so `fipsctl connect` printed success and the node never
attempted the path. An operator moving a peer onto a freshly provisioned
link, or a supervising process that has just seen a second path come up, had
no way to make the node use it: the peer stayed where it first authenticated
until that path died. The address is now tried as an alternate path
alongside the live one, through the same helper a runtime peer refresh uses,
so promotion happens only after the alternate handshake authenticates and a
wrong address cannot displace a healthy link. The response gains an additive
`refreshed` field distinguishing "started an alternate-path handshake" from
"already on this exact path and it is fresh". `connect` stays ephemeral: the
peer is not written to configuration and gets no auto-reconnect.
- A failed probe now names the condition it actually hit. Probing an absent
key reported one of two different failures depending on whether a bloom
filter had returned a false positive, and the reason shown was wrong in the
branch that gets likelier as the mesh grows: the verdict was right, the
explanation was not. The core already held the fact and discarded it, since
the lookup gate proceeds only when a peer's filter claimed the address, so a
sent lookup proves the claim; that is now its own failure kind, carried to
the control socket through the existing reason field. `fipsctl` renders the
two differently and closes with a note naming the false-positive mechanism,
which is the part that helps, because the reason string alone does not tell
an operator that a filter cannot miss a key that is present. The
seventeen-second wait is unchanged, being the retry ladder running to
completion; what changes is that the operator is told the wait carried no
information rather than given a wrong reason for it. Still not addressed:
naming which peer's filter made the claim.
#### Data-plane / transports
- An inbound onion connection no longer leaks its inbound slot when the peer
goes away before the accept loop has pooled it. The Tor accept loop spawned
the per-connection receive task, then inserted the pool entry, then bumped
the inbound counter; a remote that reset immediately let the receive task
run its cleanup first, find nothing to remove, skip the teardown that
decrements, and leave the increment behind for the life of the process. Once
enough of those accumulated the `max_inbound` gate rejected every further
onion connection, with the pool visibly empty. The accept loop now holds the
receive task on a readiness barrier until both the pool entry and the
counter are in place, as the TCP accept loop already did, and an aborted
accept still falls through to the cleanup rather than stranding the entry.
The same accept path also releases the slot of an entry it evicts, which a
reused ephemeral forward port could otherwise leave orphaned. Outbound Tor
connections and the whole of the Nym transport are unaffected: neither holds
a counted inbound slot.
- A path MTU measured on one link no longer clamps a peer that has moved to
another. Every writer of the per-destination path-MTU cache keeps the
smaller of the existing and incoming value, which is right while a peer
stays put, but the entry is keyed by destination alone. So a peer first
reached over a narrow link stayed clamped to that link's ceiling for the
lifetime of the process: when it later became reachable over a wider
transport, promotion re-seeded, the seed saw a tighter existing value and
declined, and traffic kept running at the old link's ceiling on a link that
could carry far more, with nothing reporting it because the clamp was doing
exactly what it was told. The node now records which transport last seeded
each destination and treats a seed from a different one as authoritative
rather than as a loosening to refuse. Re-seeding the same transport still
keeps the tighter value, so repeated promotion does not reset discovery, and
a destination with no prior seed is unchanged.
- An inbound IPv6 source address no longer loses its scope. Converting a raw
`sockaddr` on receive dropped `sin6_scope_id`, and a link-local source
(`fe80::/10`) identifies a host only together with its interface scope,
because the same address may exist on several interfaces. The failure was
silent rather than loud: the address parsed, it looked correct in a log
line, and only the reply failed. On a link where every address is
link-local, a Wi-Fi Aware data path for instance, that stalled the Noise
handshake. msg1 was sent to a scoped address and arrived, the peer replied,
and msg2's source was recorded with scope 0 and could not be routed back, so
msg1 retried indefinitely with nothing reported. The conversion is shared by
all three Unix receive paths, the single-packet `recv_from`, the Linux
`recvmmsg` batch and the Darwin `recvmsg_x` batch, so one fix covers each.
Sources outside the link-local range carry scope 0, for which the scoped and
unscoped forms are identical, so nothing else changes.
#### macOS
- The packaged macOS daemon recreates and binds its control socket at
`/var/run/fips/control.sock`. A privileged macOS process selects that private
runtime path before its leaf exists, so bind creates it and clients follow
once it is there, and socket setup now changes ownership and mode only for a
private parent directory it creates or recognizes as a canonical FIPS runtime
directory. Previously the packaged daemon fell through to the shared
`/tmp/fips-control.sock` path after every boot, and because socket setup
changed the parent directory unconditionally, the root daemon also took group
ownership of `/tmp` itself.
#### Packaging & deployment
- The FreeBSD package manifest carried the same bounced maintainer address the
other packaging paths did, so the contact of record in the new `.pkg` artifact
would have been unreachable at its first release.
### Security
#### Coordinate cache
- The coordinate cache is warmed from plaintext session headers on datagrams a
node is merely forwarding, and nothing filtered those writes. Two checks now
run at the write site, and an entry carries its provenance, so a coordinate
established by a lookup whose proof this node checked is no longer displaced
by an unauthenticated hint. Four counters report what the checks refuse:
`coord_warm_foreign_root`, `coord_warm_key_mismatch`, `coord_hint_rejected`
and `coord_hint_changed`, all in `fipsctl show status` and
`fipsctl show routing`. **These are mitigations and not a closure.** The
coordinate is still not authenticated, so a same-root forgery is unaffected
and hint-over-hint for a destination that was never verified is unchanged.
Treat the counters as a rate to watch rather than an alarm.
#### Tick profiler
- The `--dir` given to `profile tick on` is now confined to `/var/log/fips`
when the daemon runs as root. The control socket is reachable by the `fips`
group, which the security model writes down as strictly weaker than root, yet
the directory travelled from the socket straight into a root `create_dir_all`
with no validation: a group member could create a root-owned directory
anywhere on the filesystem, including a path a later privileged component
reads. The path must now be absolute, must contain no `..`, and must still
resolve under the root once every existing ancestor has been followed through
its symlinks, so a symlinked parent does not launder a lexically clean path.
A daemon that is not running as root crosses no such boundary and takes
`--dir` as given, which keeps the documented non-root `cargo run` capture
working; the flag can no longer point a root daemon at a different log root,
which was its other documented use. This only ever affected a
`--features profiling` build: the subcommand is absent from a stock package.
- A capture file is no longer written over whatever is already at its path. The
name is a one-second UTC stamp and therefore predictable, so `File::create`
would have followed a symlink pre-planted at the next name, and it truncated
any real file it found. The file is created only if it does not exist, and a
capture started in the same second as a previous one takes the next free
`-N` suffix instead of failing. Capture files are created private to their
owner and the capture directory is no longer left world-accessible by a
permissive umask; a capture carries the node npub, build, platform and a
timing series.
## [0.4.2] - 2026-08-25
### FMP/FSP sessions and rekey
#### Changed
- Config validation now rejects two `node.rekey` settings that appear to
disable the trigger and in fact fire it continuously. `after_messages` of
zero makes the message-count arm true on every poll, because the trigger
compares the counter with greater-or-equal. `after_secs` at or below the
per-session jitter bound is the same trap on the timer arm: each session
offsets the interval by a random value within plus or minus that bound, so a
smaller interval saturates to zero on a negative draw and rekeys on sight,
for roughly half of sessions. Both are checked whether or not rekey is
enabled, so switching it on later cannot surface the error at a surprising
moment, and neither gains an upper bound; a very large value remains the
supported way to disable one arm. A config carrying either setting now fails
to load instead of starting a node that rekeys constantly.
#### Fixed
- Inbound session-setup messages are now rate limited, keyed on the
authenticated link peer the datagram arrived over. The setup path allocated
a session entry and sent a routed SessionAck for every well-formed message
naming an address it had no entry for, and that address is an envelope field
the sender picks, so one neighbour could grow the session table at whatever
rate it could transmit and buy an ack per entry to a destination of its
choosing. The limiter sits ahead of every send and both handshake
constructions in the handler, so a refused message emits nothing and costs
no cryptography. The key is the link peer rather than the claimed source
address, which is what makes it a limit at all: keying on the source would
hand a single sender a fresh full bucket per forged message.
Two consequences worth stating rather than discovering. The limiter bounds
each neighbour's contribution and makes a flood attributable; it does not
give the node an absolute ceiling, which stays at roughly
`peers * rate * handshake_timeout_secs`. And a legitimate peer reaching this
node over the *same* link as an attacker shares that attacker's bucket, so
establishment behind a flooded neighbour is refused until it refills. Rekey
and restart traffic is deliberately not subject to that: setup messages
naming an already-established peer draw on a separate per-link bucket,
because suppressed key rotation is silent (nothing errors and no session
drops) and would have shown up only as a flat `rekey_armed`.
- A forged SessionAck no longer destroys an in-flight session initiation. The
handler removed the session entry to take ownership of the handshake state
and, when the XK msg2 read failed, returned without putting it back. Nothing
in that message is authenticated (the only thing tying it to the initiation
is the datagram's source address, which the sender chooses), so any node able
to reach the victim could cancel any initiation with 57 bytes of the right
length, and hold establishment down by repeating it. The entry is now kept.
Keeping it is not enough on its own, and the second half is the part worth
naming: the msg2 read mixes the sender's ephemeral into the symmetric state
before it authenticates anything, so an entry put back as the failed read
left it holds a handshake that can never read the genuine msg2, which trades
a one-round-trip denial for one lasting the full handshake timeout. The read
is therefore rolled back to its pre-read state before the entry goes back.
The three later failure paths in the same handler still drop the entry: each
is downstream of a msg2 that authenticated, so it is a local failure rather
than a possible forgery. The entry's activity stamp is deliberately not
refreshed on the failure path, so a spray cannot hold a dead initiation past
its original sweep deadline, and a new `ack_handshake_failed` counter makes
the refusals visible at the default log level. Only the XK handshake on this
branch is covered; the additional drop sites in the XX handshake on the
development branch are not.
- An unauthenticated session msg3 no longer discards a completed key epoch.
Five sites discarded the whole rekey, which nulls a `pending` session sitting
beside the handshake, when only the handshake had failed: the four failure
paths in the responder-side rekey arm of the msg3 handler, and the
dual-initiation yield arm in the setup handler, which is gated on a rekey
being in progress rather than on this node having initiated it, so an entry
the peer armed reaches it too. That `pending` session is the epoch the real
peer may already have cut over to, so discarding it kills the reverse
direction until the session idles out. Two unauthenticated messages reached
it: a forged setup message arms a handshake beside a completed rekey once
that rekey has waited a full idle timeout for a peer that never appeared on
the new epoch, and any garbage msg3 of the right length then finishes the
job. All five now abandon only the handshake. The four remaining sites in the
ack initiator arm are deliberately left alone and the reason is recorded
there: an entry with the initiator flag set holds no pending session, so the
two calls are the same action at those sites.
#### Security
- A frame whose counter is `u64::MAX` is now refused by the replay window
instead of being accepted as a new high-water mark. Accepting it pinned
`highest` at the ceiling, after which every subsequent counter from that peer
fell more than a replay window below it and was rejected, wedging that peer's
own receive path until a rekey replaced the session. The send side already
refuses to emit that counter (`take_send_counter` and `advance_nonce` both
return a nonce-overflow error), so no conforming peer can produce it and the
refusal is invisible on the wire; the highest counter an honest peer can send,
`u64::MAX - 1`, is still accepted. Reaching this required an
already-authenticated peer running modified code, and the damage was confined
to that peer's own session.
- The MMP gap tracker advances its expected-counter state with a saturating add,
so a received counter of `u64::MAX` no longer overflows it. The wrap silently
reset the expectation to zero in a release build and aborted the task under a
build with overflow checks on, such as the test harness. Behaviour is
unchanged for every counter an honest peer can emit.
- An epoch-mismatch msg1 no longer tears down a peering that is still
carrying authenticated traffic, and a second epoch change for the same peer
identity inside 15 seconds is refused. The epoch travels inside the AEAD, so
such a msg1 is authentic, but it stays authentic after capture: replaying
one destroyed a working peering, and with it the FSP session state that
peering carried, from off the path. The peering's last authenticated inbound
frame is the evidence that it is still alive, and nothing an unauthenticated
sender emits can refresh it, so a peer that genuinely restarted clears the
gate by having stopped sending. The refusal is a silent drop: no msg2 is
returned, since the stored msg2 is bound to the original msg1's ephemeral
and answering a sender-chosen address is free amplification. The interval is
stamped only when an epoch change is accepted, so a sustained replay cannot
starve a genuinely restarting peer. Both thresholds come from one constant,
sized so a restarting peer's msg1 resends still land inside its own first
handshake window and below `link_dead_timeout_secs`, and nothing changes on
the wire.
- Retention of a superseded FSP key epoch is now capped at an absolute
ceiling measured from the cutover, defaulting to 120 seconds against the
10-second drain window. The drain deadline slides forward on every inbound
frame that authenticates against the `previous` slot, which is what keeps a
peer that lost msg3 from having the old epoch erased out from under it, but
it also meant the authenticated peer holding that key could keep the retired
key resident for as long as it kept sealing frames in the old epoch. The
sliding grace is unchanged; it now delays erasure by a bounded amount rather
than preventing it. The ceiling is set to clear the worst-case legitimate
recovery of a peer that lost msg3 (the msg3 resend ladder, then
`handshake_timeout_secs` before the responder abandons, then the rekey
dampening window before it may re-initiate, about 90 seconds at stock
settings), and it is raised automatically if the configured handshake timers
imply a longer budget, so shortening a timer cannot push the ceiling under
the recovery it has to leave room for. Nothing changes on the wire; each side
runs its own drain. A peer that has still not recovered when the ceiling
fires is left with undecryptable frames until its own rekey retry
re-converges the epochs, since nothing tears an established session down on
repeated decrypt failure.
- A session setup message naming an already-established peer no longer replaces
that peer's session. The handler did this whenever `node.rekey.enabled` was
false: it ran a fresh responder handshake and overwrote the entry, discarding
the live keys. The message carries no authenticator and its source address is
an envelope field, so anyone able to reach a node could name an established
peer and take that session down, repeatedly, and hold it down by repeating
the message. The established case now always arms the handshake alongside the
running session and adopts the new keys only after a msg3 whose authenticated
static key matches the key the session was opened with, which is the check
the rekey path already applied; a peer that genuinely restarted still
re-establishes, and a forged setup leaves the session carrying traffic. This
changes no wire format and adds no configuration: a node with rekey disabled
already answered such a message, it simply destroyed the session afterwards.
- A session rekey armed by a peer's setup message is abandoned if the matching
msg3 never arrives, rather than persisting for the life of the session, so an
arming that never completes cannot make the node read a later genuine setup
message as a simultaneous initiation and drop it. Only the armed handshake
expires, and only it: a rekey that completed is the key epoch the peer has
already moved to, since it exists only because a msg3 carrying that peer's
authenticated key arrived and the sender of that msg3 promotes the new epoch
on an unconditional two-second timer. Expiring those keys on any timer would
drop every later frame from that peer, so they are held until the peer's own
frame promotes them, a newer completed rekey replaces them, or the session
goes away. What the wait does bound is precedence, not the keys: a completed
rekey outranks a fresh setup message from that peer only until it has waited a
full idle timeout, after which the setup is answered normally, so a peer that
restarted while we held such a session is not refused for as long as our own
sends keep the session from idling out. The handshake timeout logs at INFO,
since it costs nothing, and a completed session displaced by a newer one at
WARN, since that does throw away keys the peer may hold. Session counters
record the arming of a handshake by a setup message, each of the three ways
such a message is refused, and each displaced session, so a node under a
sustained spray of setup messages shows a rate rather than nothing; the
per-message log lines stay at DEBUG because an unauthenticated sender can
drive them at line rate. These counters are not yet readable through the
control socket.
- The session drain sweep and the cut-over that retires an old key epoch now
run whether or not periodic rekey is enabled. Both sat behind the
periodic-rekey gate, so a node with rekey disabled that adopted new keys held
the superseded ones for the life of the session.
- The FSP session address is now bound to the peer key the Noise handshake
authenticated, on both the initial and the rekey path. The responder recorded
a session under the source address carried in the datagram without ever
checking that address against the static key it had just authenticated, so a
peer could complete a genuine handshake while claiming another node's
address, and the identity cache, the session map and the address the IPv6
shim reconstructs on delivery would all attribute its traffic to the node it
named. The address is now derived from the authenticated key at the point it
first becomes available in msg3, and a mismatch drops the half-open session
without recording either the identity or the session. The rekey responder
needed its own check: it returns before that code is reached and never read
the peer's static key at all, so a rekey could complete under an established
session with a different key than the one that opened it. It now requires the
key to be unchanged and abandons the rekey while leaving the existing session
intact, rather than tearing the session down, which would have handed an
attacker a way to kill established sessions. Both comparisons are on x-only
keys, because a stored key may carry a synthesized parity while the handshake
learns the true point. The two rejections are counted separately in the
session reject statistics.
- A link handshake admitted by the established-address waiver is now confirmed
against the identity that owns that address, instead of on the address alone.
A transport configured with `accept_connections` false still admits an inbound
msg1 whose source matches an established peer, so that a peer re-handshaking
after a restart or a rekey is not locked out, but nothing checked that the
party sourcing from that address was the peer. Any off-path party able to send
from it therefore obtained a full link handshake from a node configured to
accept none. Once the key exchange reveals the initiator's static key, the
handshake is now dropped unless that key belongs to the identity the matched
address is attributed to, and dropped as well when the waiver was used and no
identity owns the address at all, which fails closed rather than skipping the
check for that case. The cheap refusal is unchanged: a stranger reaching a
transport that refuses inbound connections is still turned away before any
cryptography. Attribution consults both the reverse-address lookup and the
scan over established peers rather than stopping at whichever answers first,
because the reverse lookup can name a link that no longer exists, and stopping
there would refuse a peer the scan can still attribute, permanently, since
the confirmation returns above the code that repairs that lookup. Both
refusals log at warning level, naming the expected and actual peers, and
charge the existing handshake bad-state rejection counter rather than one of
their own.
- An inbound frame whose header disagrees with the frame that arrived is now
dropped, at the single dispatch point every transport converges on, before the
declared length can be used as a parsing input. The 4-byte common prefix
carries a payload length that the node never read, so on the datagram
transports (UDP, Ethernet and BLE, which deliver one whole frame per packet
and where the arrived length is therefore known exactly) nothing compared the
two. This closes no known defect, and it is worth being exact about what it
drops on a deployed line. The stream transports (TCP, Tor, Nym) read their
frame boundary out of that same field, so the comparison holds by construction
and never fires for them. A short datagram is a truncated frame, which already
failed the AEAD tag or the exact-size handshake parse, so what changes there
is which reason it is dropped for rather than whether it is dropped. A frame
whose phase the node does not recognize carries no fixed relationship between
the two and is left alone rather than rejected on a guess. The drop takes its
own rejection reason and its own `payload_len_mismatch` counter rather than
reusing the admission one, since it is a framing rejection decided before the
phase dispatch and it applies to established data frames as well as to
handshakes; that counter is not yet readable through the control socket.
### NAT traversal and Nostr discovery
#### Added
- `node.discovery.nostr.max_concurrent_offers_per_npub`, defaulting to 4, which
bounds how many inbound traversal offers one sender npub may have in flight
at once. It sits inside `max_concurrent_incoming_offers`, which remains the
outer bound, so a value above that is inert; zero is rejected at config
validation, since it refuses every inbound offer rather than disabling the
limit, and so is a value above the maximum permit count a semaphore can
hold, which would otherwise fail at construction rather than at load.
Existing configurations parse unchanged, the key being optional.
#### Changed
- Inbound traversal offers are now admitted against a per-sender allowance as
well as the global pool. The intake path previously took a permit from a
single semaphore before any identity check, with the sender's npub used only
as a log field, so one sender could hold every slot and deny traversal
onboarding to every other peer for as long as it kept offering. Admission now
takes a per-npub permit and a global permit together. A sender over its own
allowance is refused at debug rather than warn, because the party tripping it
is by definition sending faster than the node wants and a record per
rejection would turn the spam into log volume; the global bound being reached
keeps its warn, which is the operator's signal that the node is genuinely
saturated. **This does not make the pool inexhaustible.** Nostr identities
cost nothing to generate and the signal subscription carries no author
restriction, so an attacker running four throwaway npubs still saturates the
shipped 16-slot pool at an unchanged total offer rate. What the change buys
is that one identity can no longer do it alone, and that the two refusals are
distinguishable in the log. The permit is still held across the whole
attempt; that duration remains inferred from the attempt timeout rather than
measured.
- Config validation now rejects a `node.discovery.nostr.signal_ttl_secs` that
is too large for the configured `replay_window_secs`. A traversal signal is
acceptable over its TTL plus 60s of clock-skew grace on each side, and that
span has to stay strictly inside the replay window, or a session id evicted
from the replay cache on expiry is still fresh enough to be accepted a second
time. The relation was documented but unenforced, so raising the TTL past
180s silently voided it. The bound is derived from the skew constant rather
than restated, and is checked whether or not nostr discovery is enabled, for
the same reason the rekey rules are. The shipped defaults (120s against 300s)
are unaffected, but a configuration that had widened the TTL or narrowed the
replay window now fails to load, with an error naming the concrete floor for
`replay_window_secs`. The NAT lab's config generator was one such
configuration and its generated `replay_window_secs` moves from 60 to 180.
Note that this covers eviction on expiry only: `seen_sessions_max_entries`
remains a separate capacity-eviction route that no config relation bounds.
- The peer-retry tick no longer awaits the Nostr advert refetch. It ran inline
on the 1-second rx-loop tick, awaiting a fetch with a 2-second timeout for
each due peer and discarding the result; with up to sixteen due peers the
timeouts stacked, and field profiling measured single 2.00 s stalls as the
common case and a worst tick of 12.4 s against a 1 s period, delaying every
other rx-loop arm by as much as 4.2 s. The refetch is now spawned, so a dial
uses the advert cached at that moment and the refreshed one lands for that
peer's next retry.
- The `Adopted NAT traversal socket` log line now carries the transport id and
the local address alongside the peer npub. Without the local address an
operator cannot join a host socket table against adoption events, and without
the transport id several peers sharing one adopted transport are
indistinguishable from several separate adopted transports.
#### Fixed
- Nostr NAT traversal signals are now sent only to relays the client pool
actually holds. A signal is addressed to the merge of the peer's NIP-17 inbox
relays, the relays its advert nominates for signaling, and our own DM relays,
but the pool is built once at startup from the configured relays and the send
is rejected outright, before anything is contacted, if any single URL in that
list is outside it. One unconfigured relay anywhere in the merge therefore
killed the whole attempt, including the sends to relays both sides shared. On
a public node in open mode this made discovery non-functional: 309 traversal
attempts, 290 explicit failures, zero successes, every failure on `relay not
found`. Configured peers were unaffected, since they run a matching relay
set. Comparison is on the normalized relay URL rather than the raw string, so
a configured relay spelled with a trailing slash or different host case is
not discarded. Two smaller fixes ride along: the responder resolves its
relays before binding a socket and running STUN, rather than spending a STUN
round trip and holding an offer slot only to find it has nowhere to answer,
and it gained the empty-relay-list guard the initiator already had.
- Nostr NAT traversal no longer breaks after the host suspends. The traversal
clock cached a Unix timestamp once at startup and advanced it with a
monotonic `Instant`, which does not tick while a machine is asleep, so after
a suspend the daemon's idea of the time trailed real time by the suspend
duration for the rest of the process lifetime. Every NIP-40 expiration it
computed was therefore published already in the past: relays dropped the
offers as expired, the initiator logged a signal timeout waiting for an
answer, and traversal stayed broken until the daemon was restarted. The
clock now reads the wall clock on every call. This is not macOS-specific,
though a laptop that sleeps is where it is easiest to hit; any host that
suspends or hibernates was affected. Reported in
[#128](https://github.com/jmcorgan/fips/issues/128).
#### Security
- An advert or inbox-relay list returned by a relay is now checked against
the peer it claims to describe before anything else looks at it. The relay
pool verifies every event's signature but does not check a reply against
the request filter, and neither of the two options that would make it do so
is enabled, so a relay may answer a request for one author's advert with an
event it signed itself. The stale-advert refetch picked the newest
`created_at` across everything returned, with no author test, and then wrote
the result into the advert cache under the requested peer's npub, so a
single hostile or compromised advert relay could pin an endpoint set of its
own choosing for that peer. The author test now runs before the timestamp
contest rather than after, so a future-dated foreign event cannot even
suppress the genuine advert by winning it. The same filter now applies to
the inbox-relay lookup, where the omission let an attacker-authored relay
list steer this node's direct-message and traversal-signal traffic. A
refetch that comes back with events, none of them signed by the peer, now
leaves the cached entry alone: that is no evidence the advert was
withdrawn, and evicting on it would hand the same relay a way to clear the
cache. A refetch that genuinely comes back empty still evicts.
- An advert's `created_at` is now clamped forward to the same 60s of clock
skew the traversal-signal path already tolerates. An unbounded future
timestamp bought a cache entry a proportionally distant validity horizon
and an unbeatable position in every replacement comparison, so a later
genuine advert could never displace it and the size-cap eviction collected
it last. The clamp applies to the stored timestamp as well as the validity
window, at all three points where an advert is cached, so ordering and
expiry now agree. Clamping rather than refusing the event is deliberate: a
node whose own clock runs slow reads every peer's honest advert as
future-dated, and refusing would silently withdraw Nostr-mediated dialing
for every peer at once.
- Inbound traversal signals are now rate limited before they are decrypted. A
rendezvous-enabled node handed every kind-21059 event straight to the unwrap,
which is two NIP-44 decrypts and a signature verify, inline on the single
task that also routes traversal answers and maintains the advert cache.
Nothing bounded how fast an unauthenticated stranger could schedule that
work: the per-npub offer admission cannot, because it keys on the sender's
public key, which only exists once the first decrypt has already run, and
because it is a concurrency semaphore rather than a limit over time. A token
bucket now sits ahead of the unwrap, so a flood costs a node its inbound
offers instead of the whole notify loop.
What the limit can and cannot key on is worth stating, because it decides the
shape of the fix. Before decryption there is no sender identity at all: the
outer event is signed by a key generated per event, so bucketing on its
author would hand an attacker a fresh allowance for free, and the timestamp
and recipient tag are equally attacker-chosen. The arrival relay is drawn
from our own configured set but is not an isolation boundary either, since an
attacker publishes to the same relays an honest peer does. The shared
allowance is therefore a single global bucket and is indiscriminate by
construction, which on its own would shed our own traversals along with the
attacker's, and since the attacker sets the rate every retry would land in
the same shed. A second, smaller allowance is held in reserve and drawn only
while this node has traversals of its own outstanding, so a flood denies a
node its inbound offers, which nothing receiver-side can prevent without a
pre-decrypt identity, rather than also denying it the answers to offers it
sent. Shed signals are counted and reported at debug level per event with a
warning each time the running total doubles, so a bucket sized below a busy
node's real need shows up in the log rather than as apparent relay flakiness.
Two limits on what this buys. It bounds the crypto path only: the advert
branch runs earlier in the same loop and is not metered here, so a stranger
can still put JSON parsing and a cache insert on the task per event. And the
relay SDK verifies each event's outer signature on its own per-relay task
before this loop ever sees it, which no receiver-side change short of
dropping the subscription can avoid.
- A STUN binding response is now accepted only from the address the binding
request was sent to. The client discarded the source address `recv_from`
returned and let the parser decide, and the parser checks only the message
type, the magic cookie and the 12-byte transaction id. An on-path attacker
who could read the outbound request could therefore inject a reply carrying
a transaction id copied from it, and its chosen address became the reflexive
candidate the node published in a traversal offer or answer, redirecting the
peer's hole-punch packets. Datagrams from any other source are counted and
discarded, and one debug record per STUN attempt reports the count and the
last unexpected source, so a rejection is diagnosable without giving a
flooder control of the log rate. A server that answers from an address other
than the one dialed, which RFC 5389 forbids, now times out and the next
configured server is tried.
- The exemption that lets a peer's reflexive address skip the private-address
gate is now conditional on our own vantage point. That exemption exists for
the deployment whose STUN server sits inside the private network, so the
observed reflexive address is legitimately private; it was applied
unconditionally, so a node whose own STUN result was public still punched
whatever private address a peer named as its reflexive one. Any sender whose
offer or answer was accepted could therefore aim a burst of UDP packets,
carrying this node's source address, at a host inside the node's own private
network, which is the one place the candidate filter was written to keep it
out of. The gate now applies whenever our own reflexive address is public.
It stays lifted when our own reflexive address is itself private, which is
the LAN-STUN deployment the exemption was for, and also when we have no
reflexive address at all, so a failed STUN probe cannot cost a node its
same-LAN peering. Two consequences to state rather than discover: a peer
behind a private STUN server talking to a node with a public one loses its
reflexive candidate, which was never reachable from us in any case, and
because the /24 comparison is IPv4-only a unique-local IPv6 reflexive
address is refused unless our own reflexive address is unique-local too.
An off-subnet refusal of a peer's reflexive address is a shape an honest
deployment now produces, so it no longer raises the refusal record to
warning level on its own; the never-routable, port-0 and unparsable classes
still do.
- A peer's candidate list is now bounded before it is walked rather than only
after. The eight-target cap ran after both planning loops had finished, so
it bounded what a node punched but not what it spent deciding: a signal
naming several thousand candidates had every one of them parsed and vetted,
and the deduplicating scan that follows is quadratic in the plan those
candidates feed. At most 32 candidates are now vetted, four times the
target cap and four times what the candidate generator produces on the
widest host, and the excess is discarded rather than failing the offer, so
an honest many-homed peer loses the tail of its list instead of its
traversal. The refusal record carries the discarded count as a new
`over_offered` field and treats a non-zero one as an attack shape, since
nothing honest reaches the bound.
- A NAT-punch packet is now accepted only from an address this node planned to
probe. The punch packet's discriminator is a plain digest of the session id,
a value both peers already know, and it travels in the clear in every probe,
so acceptance proved only that the sender had seen one. The receive loop
broke on the first packet whose digest matched, whatever its source, and
returned that source as the peer address, so anyone who observed a probe, or
who could reach the node and guess the session id, could have an arbitrary
address adopted as the peer: the legitimate traversal was denied, the Noise
handshake and its retransmissions went to an address of the attacker's
choosing, and the pair was charged a failure against its backoff state. The
npub-pinned handshake still could not authenticate to the wrong host, so this
was a denial and a misdirection rather than an impersonation. The source
address is now ranked against the planned target list before anything else:
an unplanned source is dropped and, deliberately, is not acked either, since
acking it is a reflection the node controls. A source matching a planned
target exactly is adopted immediately, as before. A source matching a planned
target's IP on a different port is what a symmetric NAT's fresh mapping looks
like, and it is still adopted, because that is the main class of NAT pairing
punching exists to rescue; it is held as a candidate for 250 ms first, so an
exact match arriving inside that window supersedes it. The honest path's
latency is unchanged. Two consequences to state rather than discover: an
attacker that can source packets from a planned target's IP on any port is
still accepted, which is the residue only an authenticated probe can close;
and an attempt under a flood of spoofed matching packets now runs to its full
timeout instead of ending on the first one, so the refused sources are
counted and reported once when the attempt ends rather than logged per
packet.
- Traversal punch targets taken from a peer's offer or answer are now
filtered and bounded. A rendezvous-enabled node previously punched every
address a signed offer named, including loopback, link-local, multicast,
broadcast, unspecified and CGNAT addresses, and placed no limit on how
many candidates one offer could carry. Any npub could
therefore have a node emit a burst of UDP packets at addresses of the
sender's choosing, carrying the node's own source address. Candidates in
the never-routable ranges are now rejected, IPv4-mapped IPv6 forms are
canonicalized before the check so they cannot slip past it, candidates
with port 0 are dropped, private-range candidates are punched only when
they share a /24 with one of our own addresses (which is what same-LAN
traversal already required of its own path), and the planned target list
is capped at eight. A peer's reflexive address is checked against the
never-routable ranges but not against the /24 rule, so a deployment whose
STUN server sits inside the private network keeps working. A malformed
address in a peer's signal now drops that one candidate instead of
failing the whole traversal. A node also records what it declined: one
log record per planning attempt carries how many candidates the peer
offered, how many were planned, the count refused in each class and one
sample address, at warning level for the shapes no honest peer produces
and at debug level for the routine off-subnet case. Same-LAN and
reflexive traversal are otherwise unaffected.
- Traversal offers and answers dated in the future are now rejected. The
freshness check measured a message's age with a saturating subtraction, which
yields zero for any timestamp ahead of the local clock, so the age test could
not fail for a future-dated signal and no other term bounded the issue time
from above. A signal claiming to be issued arbitrarily far in the future was
accepted as strictly fresh, which voided the property that the freshness
window is narrower than the session-id replay window (300s by default) and
left the replay cache as the sole defence against a captured offer being
replayed. Forward-dating is now tolerated only up to the same 60s of clock
skew already allowed in the other direction, and a signal accepted under that
grace reports the skew outcome, so the existing clock-skew log fires for a
peer whose clock is ahead just as it does for one whose clock is behind. The
declared expiry timestamp is also no longer trusted beyond the issue time plus
the configured TTL, so a sender cannot widen its own acceptance window by
inflating that field. A single timestamp is now acceptable over at most the
signalling TTL plus 60s on each side, 240s under the shipped defaults.
Rejections are also now distinguishable in the log: a stale signal and a
future-dated one no longer share one reason string, and the inbound-offer
path, whose only surface was an unattributed debug line below the default log
level, now names the peer and the session and warns for the rejection classes
that relay delivery lag cannot produce (future-dated, identity-mismatch and
malformed offers), leaving an ordinary stale offer quiet. As with the existing
inbound rate-limit warning, an unauthenticated remote peer can drive that
line. A failure of our own offer's freshness during answer validation is
reported against the offer rather than mislabelled as the answer's, and the
tolerated-acceptance log now carries the issue and expiry stamps and no longer
attributes the acceptance to clock skew, since a peer configured with a longer
signalling TTL than ours now reaches it too.
### Data plane, routing signals and metrics
#### Changed
- Peer bloom filters are computed for every recipient in one prefix and suffix
union sweep rather than rebuilt per recipient. Announcing to R peers
previously did R full map builds and R by T merges; at 240 peers that was
20.6 ms per tick, roughly half the tick body, with a median per-interval
maximum of 34.5 ms. The result is exactly equal rather than approximately:
merging is a bytewise OR, so regrouping the unions cannot change it. The
trade-off, measured rather than assumed, is that the sweep does its full work
regardless of how many peers are ready, so a tick announcing to one or two
peers now costs about twice what it did; break-even is around three ready
peers. Cadence, the debounce, the sequence rule and the fill-ratio cap are
unchanged.
- Each peer's npub is derived once at construction instead of once per tick.
The per-tick stats snapshot ran a bech32 encode for every tracked peer, and a
second one for the common peer with no hosts-file entry and no alias, since
the display-name fallback bottoms out in the same encode: 14.1 ms per tick at
240 peers. The display name itself is deliberately not cached, because the
alias map and the host map both mutate at runtime.
#### Fixed
- A SessionDatagram carrying a truncated inner FSP payload no longer panics the
forwarding path. The coordinate-cache warm path sliced the inner payload at
the full 12-byte header offset while guarding only with the 4-byte common
prefix parser, so an inner payload of 4 to 11 bytes with phase 0x0 and the
Coords Present flag set indexed past the end of the slice. Because the
receive loop is the process's main future, the panic terminated the daemon
rather than a task, and under the packaged systemd unit the node restarted
into the same frame. The warm path now applies the same
`FspEncryptedHeader` guard the local-delivery path already used, which
additionally means a malformed frame carrying a non-zero protocol version or
the Unencrypted flag alongside Coords Present is dropped rather than having
its body read as coordinates. Any peer that had completed a link handshake
could trigger this, and admission is default-open. Frames rejected by that
guard are now counted in the forwarding statistics as
`warm_malformed_packets` and `warm_malformed_bytes`, the byte counter
charging the whole outer frame, visible over the control socket and on the
fipstop Routing State pane, so a node being fed malformed frames is
distinguishable from a quiet one at the default log level. The count is not a
packet drop: the frame is still delivered or forwarded, and only the
coordinate-cache warm attempt is abandoned. The existing debug log now also
carries the frame's protocol version and flags, which separate a short frame
from a bad-version or Unencrypted-flagged one.
- `SessionDatagram` hop-limit handling now follows IP semantics. Delivery to
the addressed node is no longer TTL-gated, and a forwarder decrements before
deciding rather than after, so a datagram that would leave with a TTL of zero
is dropped instead of transmitted. Previously the TTL check ran ahead of the
local-delivery test, so a datagram addressed to this node that arrived with
TTL 0 was dropped, and a forwarder receiving a transit datagram at TTL 1
transmitted it at TTL 0 for the next hop to discard, wasting one transmission
per expiring datagram. `SessionDatagram::decrement_ttl` and
`SessionDatagram::can_forward` were aligned to the same semantics:
`decrement_ttl` decrements first and reports false when the result is zero,
and `can_forward` is true only at a TTL of 2 or more. The reachable radius is
unchanged, because the two behaviors compensated exactly: a path of `h` links
still delivers for any source TTL of `h` or more. During a rolling upgrade, an
unupgraded forwarder feeding an upgraded destination delivers one hop further
than either version does on its own; no version mix delivers less far. The
`TtlExhausted` reject counter now charges at the node that makes the decision
rather than at the hop after it.
#### Security
- A transit node's induced routing errors are now bounded by the authenticated
link peer that induced them. The 100 ms suppression gate on
`CoordsRequired`, `PathBroken` and `MtuExceeded` was keyed on the failed
datagram's destination address, which is an envelope field the sender picks,
so a fresh random destination on every packet was always a first sighting and
every packet was admitted. Each admission also inserted a key and then walked
the whole map, so per-packet cost grew with the flood rate while the sender's
cost stayed flat, and the error itself is addressed to the datagram's source
address, which nothing binds to the sender either. A new per-link-peer token
bucket, 20 signals a second sustained with a burst of 50, is now consulted
first, keyed on the AEAD-authenticated peer the frame arrived over: the one
value at that point a sender cannot mint. The per-destination interval is
kept unchanged behind it, because it still does the aggregate suppression a
genuine outage needs, and no gate was added on the address the error is
returned to, which would have handed a sender a way to silence honest errors
toward a victim it names by keeping that victim's key hot.
Two ordering choices in there rather than left to be discovered. The peer's
token is peeked and only spent once the destination gate has also admitted,
so a single unroutable destination behind a high-fanout peer cannot burn that
peer's whole budget on signals nothing sends and silence every other
destination behind it. And the destination map now carries a hard ceiling of
4096 entries with its expiry sweep amortized to once per eviction interval
rather than run on every admission; when it is full it admits without
recording rather than refusing, because refusing would turn a full map into
node-wide silence exactly during partition healing, when many destinations
are legitimately unroutable at once. Emission stays bounded by the peer
budget in that state. Three counters, rendered on the fipstop Routing tab,
make each of the three outcomes visible instead of silent.
- A transit-emitted `PathBroken` no longer carries the reporter's cached
coordinates for the unreachable destination. The signal is returned to the
datagram's source address, so anyone able to reach the node could name any
address and have the node's coordinate cache read back to them, one entry per
packet. The field is optional on the wire and no receiver reads it, so this
is an emission change only: an unmodified peer parses the frame exactly as
before. Which of the two signals is emitted still discloses whether the entry
exists.
- A reactive `MtuExceeded` is now believed only when this node has actually
sent a frame larger than the bottleneck it reports. The signal is
unauthenticated: the admission gate narrows which destination may be named
but cannot say who named it, so a value at the floor was a legal value from
anyone, and one datagram drove a bound session's path MTU to 256 and pinned
the address-keyed entry the SYN-time MSS clamp reads, with recovery costing
three consecutive higher notifications across two notification intervals.
Each session now carries the largest frame this node has put on the wire
toward it since the last accepted decrease, and a report is refused unless it
names something smaller. Honest path-MTU discovery satisfies that by
construction, because the report exists only because a frame we sent did not
fit; a forgery has to wait for us to emit something bigger than the value it
wants to claim, which bounds every accepted claim from below by our own
traffic. The evidence is cleared on each accepted decrease and on release, so
one large send early in a session cannot vouch for the rest of it.
The guard sits ahead of both effects rather than between them, which is also
where the existing floor check moved to: the floor previously ran after the
session's own path MTU had already been changed and so governed only the
lookup table. The reactive carrier now names its own floor constant, held
equal to the actionable floor so no hop legitimately configured with a small
transport MTU loses its feedback; corroboration, not the floor's value, is
what stops a legal-but-forged claim. A separate counter, rendered on the
fipstop Routing tab, distinguishes an uncorroborated refusal from a
below-floor one.
- The path-MTU release a `PathBroken` drives is now rate limited per
destination on a budget of its own. That signal is unauthenticated too, and
the release discards a bottleneck this node learned by having a packet
dropped, so repeating the claim discarded a genuine value as fast as it could
be relearned. The limiter is a separate instance rather than the one the
coordinate warmup send already uses: a budget another signal can spend is not
a bound. Deferring a release is the safe direction, since the value kept is
the tighter one.
- The influence a remote party has over path MTU is now bounded, and the
per-destination path MTU cache has a way back. The `path_mtu` field is an
unsigned per-hop transit annotation carried outside the signed proof, and the
`MtuExceeded` and `PathBroken` signals arrive unencrypted with no sender
check, so any forwarder, or anyone who can reach the node, could lower it,
and it was accepted with no minimum. A single `MtuExceeded` carrying a very
small value drove a session's path MTU to zero, after which every packet to
that destination was answered with an ICMPv6 Packet Too Big instead of being
sent: a blackhole that lasted until the daemon restarted. The same value
reached the SYN-time TCP MSS clamp, where anything at or below 137 saturates
to a segment size of zero and the band just above it yields single digits.
Values below an actionable minimum are now ignored rather than applied or
stored, at the three places a remote value is acted on: the path MTU state
machine, the reactive `MtuExceeded` write, and the discovery response, whose
coordinates are still cached so refusing the annotation cannot become a way
to deny discovery. The MSS clamp additionally refuses to write a zero. Each
of the three refusals logs a warning and increments its own counter in the
error-signal family, so an operator can tell them apart without scraping
logs: they carry different meanings, one being an authenticated peer inside
an established session, one an unencrypted signal anyone able to reach the
node can send at will, and one a verified discovery response whose unsigned
annotation a forwarder on the reverse path rewrote. Because those three
refusals are the only way a remote value reaches the per-destination store,
the SYN-time clamp does not apply the minimum a second time when it reads
that store: a small value there is one the node derived from its own outgoing
link, which is exact rather than suspect, and BLE in particular negotiates a
link MTU per connection that lands under the minimum routinely. The clamp
refuses only a stored value admitting no TCP payload byte at all, at 137 or
below, where the segment size saturates to zero and the clamp would be
skipped entirely; it logs that at trace rather than warn, since it sits on
the per-packet path, and the peer's link promotion reports it once instead.
A stored per-destination path MTU is released when the path is invalidated by
a `PathBroken` report, by session idle expiry, or by handshake timeout, and
the link MTU read from the local transport is reseeded in its place, so a
directly connected peer does not lose its own measurement along with the
remote claim. The release also resets the session's own current path MTU
alongside the address-keyed entry, rather than reseeding the link value and
leaving the tightened one in place, so a path declared dead recovers at once
instead of only through the increase ladder. Entries written by the discovery
lookup carrier carry their own deadline and age out, since a destination this
node never opens a session with reaches none of the three release routes:
without that, a single response carrying a floor value pinned that
destination's clamp until restart, and an unknown request id still classifies
as originator, so a captured response could be replayed indefinitely. The
notification mirror deliberately carries no deadline. Locally derived MTUs
are not subject to the minimum, at the seed or at the clamp. Legitimate
narrow paths are unaffected: adaptation to hops well below the IPv6 minimum,
which the mesh does use, continues to work.
- The three routing signals (`CoordsRequired`, `PathBroken`, `MtuExceeded`) are
no longer acted on unless this node has itself bound the destination address
they name, either by initiating a session toward it or by completing the
Noise handshake that binds an address to a peer's static key. These signals
carry no end-to-end authentication, so until now any admitted mesh member
could send one naming any address and have its effects applied: a path-MTU
clamp written for an arbitrary address, a cached-coordinate flush for an
arbitrary address, and a discovery and warmup cycle for an arbitrary address.
The `MtuExceeded` case was the sharpest, because its write into the
address-keyed path-MTU lookup that the TUN reader consults at TCP MSS clamp
time sat outside the session guard and so required no session, no peer
relationship and no prior state at all. A half-open session created by an
inbound handshake that has not yet proved its address does not admit these
signals, so a forged session opening cannot be used to unlock them. Signals
from a genuine on-path forwarder are unaffected: the reporter may be any node
at any distance. This does not make the sender authentic, which nothing
short of a wire format change can do. Rejected signals are counted as
unknown-session rejections, and additionally on four new error-signal
counters visible through `show routing`, `show metrics` and the fipstop
routing pane: `unbound_coords`, `unbound_broken` and `unbound_mtu` give the
refused count per signal type, against the existing per-type arrival
counters as the denominator, and `unbound_forged` counts the subset whose
claimed source and destination pairing no honest forwarder could produce.
The drop log line now carries the signal type and the refusal class.
- A discovery lookup response is now acted on only when it answers a lookup
this node actually has outstanding. The originator path took any response
whose `request_id` was not in the transit dedup map, so an admitted peer
could harvest one genuine signed response for a target and re-inject it at
will: each injection cleared the victim's in-flight lookup, recorded a
reachability success for a target that might be unreachable, refreshed the
cached coordinates for a further full TTL, and flushed the victim's queued
packets onto a route at a moment the sender chose. It also reached the
signature verify before any check that the response was wanted, so the
verify was the first cost gate on the path. The node now records the
`request_id` of every lookup request it sends on that target's pending
entry, and a response is dropped unless it names a target with a lookup
outstanding and carries one of the ids issued for it. Because the id is
fresh 64-bit randomness drawn per attempt and the target signs over it, a
harvested response is bound to the request it answered and cannot be
redirected or replayed. The check runs before the identity-cache resolve
and before the signature verify, so a response nobody asked for costs
nothing. Replies to earlier attempts of a still-outstanding lookup are
still accepted, which is the common case on a link whose round trip
exceeds the first rung of the retry ladder. Drops are counted as
`resp_unsolicited`, visible through `show routing`, `show metrics` and the
fipstop routing pane; the counter has a nonzero floor in healthy operation,
because a request is flooded to every qualifying tree peer and the
duplicate replies land there once the first has been accepted.
- A flooded discovery dedup cache no longer makes a node unresolvable. The
cache is both the duplicate filter and the reverse-path table for lookup
responses, and at its 4096-entry bound it dropped the arriving request.
That drop sat ahead of both the check for whether the request names this
node and the forwarding path, so one link peer emitting fresh request_ids
could stop the node answering lookups for itself and stop it carrying
anyone else's, for as long as it kept the cache full. The cache now makes
room instead of refusing: over a peer's own share it drops that peer's
oldest entry, and at global capacity it drops the oldest entry of whichever
peer holds the most, so a light peer's reverse path is never taken to admit
a heavy one and extra identities buy a flooder proportionally less. A
peer's share is the cache divided by the current link-peer count, with a
floor of 64. The loosening this accepts is that an evicted request_id
arriving again inside the window is forwarded a second time rather than
recognised as a duplicate, which the per-target forward limiter and TTL
already bound. Evictions are counted as `req_dedup_evicted`; the old
`req_dedup_cache_full` counter stays in place, frozen at zero, so a
dashboard carried across versions does not lose the series.
- Answering a lookup for ourselves is now metered per link peer. The response
proof is signed over the requester's `request_id`, so every request
addressed to this node costs a fresh Schnorr signature that cannot be
cached or served twice, and until now the only thing bounding that rate was
the dedup cache filling up, which is the defect above. A token bucket per
link peer, 256 signatures of burst refilling at 32 per second, absorbs the
legitimate burst that follows a topology change, when many correspondents
re-look-up at once through the few links that lead here, while capping what
one neighbour can make the node sign. Refusals are counted as
`req_sign_rate_limited` and visible in `show routing`, `show metrics` and
the fipstop routing pane. A refused request keeps its dedup entry, and
retries carry fresh request_ids, so a refusal cannot suppress the retry.
### Admission, rate limiting and peer caps
#### Added
- `node.rate_limit.session_setup_burst` (64) and
`node.rate_limit.session_setup_rate` (16.0), the parameters of the new
per-link-peer session-setup limiter. This is the FSP session-setup bucket,
and it is distinct from the link-layer msg1 bucket described below; the two
meter different messages and are sized independently. Setup messages naming
a peer this node is already established with are metered on a second
per-link bucket derived from `node.limits.max_peers`,
`node.rekey.after_secs` and `node.rate_limit.handshake_max_resends`, so
raising the peer limit sizes it automatically. A zero burst or a
non-positive rate is rejected at config validation rather than silently
refusing every session.
- `node.limits.max_sessions`, defaulting to 1024, which bounds the end-to-end
session table. Zero means unlimited, which restores the previous behaviour
exactly and is the way to back the change out on a running node. The default
is four times the adjacent `node.session.pending_max_destinations`. A
session entry measures 6608 bytes of inline state plus heap, so the table
holds to roughly 7 MB, and a test pins that per-entry figure so the
arithmetic behind the default fails loudly if an entry grows. Existing
configurations parse unchanged, the key being optional.
- `node.rate_limit.established_handshake_burst` and
`node.rate_limit.established_handshake_rate`, the parameters of the new
established-link msg1 token bucket, which meters link-layer msg1 rather
than FSP session setup. Both are optional; omitting them (the normal case)
derives the bucket from `node.limits.max_peers`, `node.rekey.after_secs`
and `node.rate_limit.handshake_max_resends`, so raising the peer limit
sizes the bucket automatically. An explicit zero burst or a non-positive
rate is rejected at config validation rather than silently refusing all
rekey traffic.
#### Changed
- Inbound msg1 is classified before it is rate limited, and rekey or restart
msg1 arriving on an established link now draws on its own token bucket
instead of competing with stranger admission for a single shared one. On a
node with many peers the shared bucket refused a large share of ordinary
rekey traffic: a field node at roughly 245 peers refused 8753 msg1 in 25
minutes, and 159 of the 201 distinct sources were peers it already held
sessions with. Nodes upgrade with no config change. The `Msg1 rate limited`
log line now reports which limb refused, the pending count or the token
bucket, which it previously did not distinguish.
#### Security
- The Ethernet transport's discovery buffer is now bounded and no longer costs
a linear scan per beacon. Beacons are unauthenticated broadcast frames, and
the buffer deduplicated by scanning a `Vec` for the source MAC and had no
cap, so anything on the segment could name a fresh MAC per frame and drive
both quadratic CPU in the receive loop and unbounded memory. It is drained
once per tick only while the transport is operational, so a transport that
is receiving but not operational was never drained at all. The buffer is now
a map keyed on source MAC, capped at 1024 distinct MACs between drains, with
the drain order still oldest sighting first so which neighbour gets dialed
under a connect budget does not depend on hash iteration order. A MAC already
buffered is always refreshed, so a flood of new MACs cannot crowd out a
neighbour already seen. Refused beacons are counted in the transport's stats
as `beacons_dropped` and reported in the log on the first drop and then on
each power-of-ten thereafter, so the flooder does not set the log rate.
**What this does not close**: a flood can still crowd out a neighbour not yet
seen in that tick, and anything able to flood raw frames on the segment can
already jam the beacon at L2 more cheaply.
- A read failure on `peers.allow`, `peers.deny` or the `hosts` file no longer
turns the node into an open one. Every read error other than a steadily
absent file was logged and swallowed, leaving that file's entries empty, and
the reloader published the result unconditionally: an unreadable `peers.deny`
admitted the peers it named, and an unreadable `peers.allow` took a node from
admitting a named few to admitting everyone, with one warning line as the
only signal. Because the recorded modification times advanced before the
load, nothing retried until the file changed again, so a persistent
permission or I/O fault left the empty ACL in force indefinitely. The
reloader now keeps the last loaded ACL when any input is present but
unreadable, leaves the modification times alone, retries on the next tick
regardless of them, and logs the fault once on the transition rather than
once per tick. An absent file is still a policy and still loads as an empty
set; a `NotFound` that a successful stat contradicts is treated as a file
being rewritten under us and held. A reload whose inputs all read cleanly but
which empties an enforcing ACL while its files are still on disk is held for
one tick, which catches a read that caught a non-atomic in-place edit
mid-write, and released on the next so a deliberate blanking still takes
effect. `fipsctl` ACL status gains a `stale` flag reporting that the policy
in force is older than the files on disk. **What this does not close**:
there is no last-good snapshot at startup, so a node whose ACL file is
unreadable at boot still comes up with no entries, now logged as an error and
armed to retry on the first tick. Admission is also checked only at handshake
time, so a peer admitted during a window that has already happened keeps its
link.
- The end-to-end session table now has a bound. It was the one remotely-grown
map with none: an inbound SessionSetup naming an address nobody had seen
inserted an entry, and the two existing limits did not reach it, the setup
limiter governing the arrival rate rather than the population and the idle
purge only reaching entries a peer stops using. One neighbour sending setups
at the permitted rate could hold roughly 1440 half-open entries at any
moment and grow the table without limit by keeping them warm. Setups that
would grow the table past `node.limits.max_sessions` are now refused, ahead
of the setup limiter, so a full table costs no token, no responder handshake
and no ack; a refused setup emits nothing at all, which is indistinguishable
from loss to the sender and is already covered by its own msg1 resend
schedule. The test is whether admitting would grow the table, not whether
the sender is a stranger, so a resent setup for an entry already present is
still served and an in-flight handshake is not broken. Unauthenticated
half-open entries are additionally held to half the table, so a handshake
flood cannot deny the whole of it to peers that complete; that share is sized
to leave a reconnect storm, where every peer initiates at once after a
restart or a healed partition, room to land. Locally originated sessions are
capped at the same ceiling, answered with ICMPv6 destination unreachable so
the application gets an immediate error rather than a silent drop. The cap
refuses rather than evicts: the setup that triggers the decision is
unauthenticated at that point, so evicting would hand a stranger a way to
tear down sessions it has nothing to do with. Refusals are counted as
`table_full` and `half_open_full` in the session reject family. What stays
open is per-neighbour fairness among established sessions: one hostile
neighbour that completes handshakes and keeps each session warm can occupy
the table and hold new session establishment closed for as long as it keeps
doing so, which is a denial of new sessions rather than the unbounded memory
growth it replaces.
- An accepted inbound TCP connection no longer holds a slot indefinitely
without sending anything. The cap was tested at accept and the pool insert
and counter bump followed with no read in between, while the frame reader's
reads carried no deadline, so an unauthenticated remote held a slot by
connecting and staying silent. Pool keys are `ip:port`, so N sockets from one
address took N slots, and at the 256 default that locked out inbound peering
for as long as the sockets stayed open. The first frame on an inbound
connection now has a deadline, as a module constant rather than a new
configuration key, and the onion listener gets the same treatment for the
same accept-then-count ordering. Separately, the node's handshake reaper tore
down session state without closing the transport connection, so a peer that
sent msg1 and then stalled was forgotten by the node while its socket and
slot survived; the reaper now closes the connection too. **What this does not
close**: the deadline covers the first frame only, so a peer that sends one
well-formed frame and then goes silent still holds its slot. Closing that
needs a rolling idle deadline.
### Transports and configuration
#### Changed
- `fipsctl keygen` no longer exits non-zero when only the `fips.pub` write
fails. The private key is already on disk at that point, so failing the run
reported failure for a keygen that did produce the identity; the failure is
now a warning and the run succeeds. The pre-existing-key guard also moves
from `exists` to `symlink_metadata`, so a dangling symlink at the key path
now blocks keygen without `--force` instead of being overwritten silently.
#### Fixed
- The UDP transport's DNS cache is now bounded and actually evicts. The map
held one entry per distinct hostname string ever dialed, and the TTL was
applied only on the read, so a stale entry was overwritten on the next dial
of the same name and otherwise stayed for the life of the process. Under a
rendezvous policy that accepts advertised endpoints the keys are strings a
remote party chose, which made the growth theirs to drive. A store now
sweeps entries past their TTL and, if the map is still full, drops the
oldest, holding it to 256 hostnames. Refreshing a name already cached
evicts nothing. Eviction is by insertion time rather than last use, so a
rarely dialed name in a very large peer list may re-resolve more often; the
cost of a wrong eviction is one DNS lookup, not a failed dial.
- macOS: stopping an Ethernet transport under load no longer hangs the
process. The BPF reader thread handed each frame to the async consumer with
`blocking_send`, which parks with no way to be woken. Stopping the transport
aborts the consumer first, so nothing drains the 1024-frame channel, and the
socket's `Drop` then joined a thread that could never return: on a busy
interface the daemon had to be killed. The socket now drops the receiver
before joining, which releases a parked send at once, and the reader thread
sends through a helper that watches the same shutdown pipe its `select()`
already honours, so a send waiting for room cannot outlive a shutdown
request. The helper yields before it sleeps, so the saturated-path handoff
rate is unchanged. **Not covered by CI**: the reader thread is macOS-only
and Linux CI compiles none of it. What the tests prove is that the helper
the thread now waits in is cancellable; that a real BPF thread exits under
load still needs a manual check on a Mac.
- A failed private-key write no longer leaves a node silently running an
ephemeral identity. Six write results in the identity path were discarded,
and the sharpest was in `persistent` mode: a failed write to `fips.key` fell
through to an ephemeral identity with no message, so a node that had been
asked for a stable identity changed its npub, its routing address and its
mesh IPv6 on every start, and nothing said so. All six now report. An
ephemeral start that is about to overwrite an existing key file now warns
first, naming the path and the setting that would have preserved the
identity, which is the warning `fipsctl keygen` has always given and the
daemon never did. Existence is tested with `symlink_metadata` rather than
`exists`, because a dangling symlink reports absent from the latter while
still being a file the write acts on. The persistent read path additionally
warns when it finds a key file whose mode is looser than 0600, or one that is
a symlink; it does not repair either, since the daemon does not own a file it
did not create.
### Spanning tree, mesh size and routing
#### Fixed
- Flap dampening can now engage more than once in the lifetime of a node.
The arming check tested whether a dampening deadline had ever been set
rather than whether one was still in effect, so the first episode
disarmed the mechanism permanently: a node in a second flap storm went on
switching parents under hold-down alone, and neither the `flap_dampened`
counter nor the "Flap dampening engaged" warning fired again, so the
storm was invisible to anyone watching that counter. A lapsed episode is
now retired explicitly, clearing both the deadline and the switch
counter, so a second episode requires a fresh threshold of switches
within one window rather than re-engaging on the first switch after
lapse. Hold-down was unaffected throughout and continued to limit
discretionary switching, which is why the practical effect at shipped
settings was lost visibility and a lost escalation tier rather than
unrestrained flapping. Every path that can engage an episode now reports
it, including a re-engagement during parent-loss recovery, which was
previously silent. The warning names which path armed the episode
(`trigger`) and how long discretionary parent switching stays suppressed
(`dampening_secs`), using the same `trigger` values as the parent-switch
logs beside it, so the two can be read together. A
`node.tree.flap_dampening_secs` large enough to overflow the monotonic
clock is capped at one year, beyond which an episode is
indistinguishable from permanent, so an extreme setting no longer panics
the node when dampening engages.
### DNS responder
#### Security
- The DNS responder's mesh-interface filter now works on macOS and FreeBSD,
where it had never run. The filter drops `.fips` queries that arrive over the
mesh TUN, which is what keeps a widened `dns.bind_addr` from exposing the
hosts file's alias space to every mesh peer. It was keyed on the interface
index resolved from the *configured* TUN name, but macOS and FreeBSD assign
the device a name of the kernel's choosing (`utunN`, `tunN`), so the lookup
found nothing, the index came back `None`, and `None` disables the filter.
The index is now resolved from the name of the device the node actually
created, which the TUN startup path already records, and a live device whose
index will not resolve is logged rather than passed off as "no mesh
interface". Linux is unaffected, since the configured name is the device's
name there. **Behaviour change on macOS and FreeBSD**: a node with a
non-loopback `dns.bind_addr` stops answering `.fips` queries that arrive over
the mesh interface. **What this does not close**: with an app-owned TUN the
node never learns a device name, so the filter stays off there. **Not
measured**: whether macOS and FreeBSD attribute a locally originated query
sent to the node's own mesh address to the TUN interface, as Linux does. If
they do, such a query is now dropped on those platforms; the shipped resolver
drop-in targets `[::1]` rather than the mesh address, so the packaged path is
not affected.
### Gateway and peer lifecycle
#### Fixed
- A failed log write can no longer panic the thread or task that logged. The
subscriber was built with the default internal-error reporting, which sends a
failed write to `eprintln!`, and that macro panics when stderr has also
failed. The shipped supervisor configurations make that a single condition
rather than two: the macOS plist points both standard streams at one
unrotated file, and the systemd units route both to journald, so one full
disk fails both sinks together. In the daemon a crypto worker was the case
that mattered: it logs a warning on send backpressure, and a worker that
dies takes its share of the peer space with it permanently, while the panic
message is discarded along the same broken path. In `fips-gateway`, which
built its subscriber the same way, the casualty is a spawned task: the DNS
resolver, the control accept loop or the pool tick, none of which is observed
until shutdown, so the process would keep running and reporting healthy with
mesh name resolution or lease expiry and NAT cleanup silently stopped.
#### Security
- The gateway DNS forwarder now validates an upstream answer before it becomes
a NAT mapping. It previously accepted whatever datagram arrived: the upstream
query reused the client's own transaction ID, the upstream socket was
wildcard-bound and never connected, the receive discarded the sender, neither
the response ID nor the question section was compared against what was asked,
and the returned address was not checked against the mesh prefix. Because the
extracted address is installed as a DNAT rule that carries no interface
constraint, a forged answer redirected traffic rather than only poisoning a
lookup. The upstream query now carries a random transaction ID, the socket is
connected so the kernel drops foreign sources, a response must match on ID,
question and type or it is discarded while the receive continues against the
original deadline, and the address goes through the validating parser with a
non-mesh answer refused before any allocation. One deliberate behaviour
change: the validation sits before the rcode check, so an upstream answering
FORMERR or REFUSED with an empty question section now yields SERVFAIL rather
than having its rcode relayed. Checking after the rcode would admit a forged
NXDOMAIN. Connecting the socket also means a dead upstream surfaces
ECONNREFUSED immediately instead of stalling for five seconds.
### Control socket
#### Security
- The control socket and the directory holding it are now created with a
restrictive mode rather than created wide and narrowed afterwards. `bind(2)`
makes the socket inode `0777 & ~umask`, so under a permissive umask the
socket was world-accessible for the window between the bind and the `chmod`
to 0770 that followed it; the bind now runs under a umask that masks the
"other" bits, so the inode is 0770 from creation and the chmod and chown stay
the authority on its final mode. The parent directory was worse than a
window: it was created with `create_dir_all`, which is also `0777 & ~umask`,
and nothing ever set a mode on it, so under a permissive umask the directory
holding the socket stayed world-writable for the life of the host, and a
world-writable parent lets an unprivileged account plant an entry at the
socket path. Directories this code creates now come out 0750, which is what
the systemd unit (`RuntimeDirectoryMode=0750`) and the FreeBSD rc script
(`install -d -m 0750`) already apply, so no packaged deployment sees a
different mode and no `fipsctl` user loses access. Both the daemon and the
gateway control sockets are covered. **What this does not close**: the window
between the stale-socket probe and the bind is documented at the site rather
than removed. Reaching it needs write access to the socket's parent
directory, which the packaged layouts give to root alone, and an account
holding it can deny the daemon its socket more simply by squatting the path
first.
### Key material and identity files
#### Security
- Private key writes no longer follow a symlink, and the key file's mode is
enforced rather than merely requested. The single write path opened with
create and truncate and no `O_NOFOLLOW`, so a symlink planted at the key path
was followed and its target overwritten, and it supplied the mode only
through `open(2)`, which the kernel honours on creation and ignores
otherwise, so a `fips.key` that already existed at 0644 stayed 0644 through
every rewrite. That second half needs no attacker: one `chmod`, or a restore
that did not preserve modes, leaves the key readable indefinitely. Both
writers now share an open helper carrying `O_NOFOLLOW`, and the private key
has its mode applied to the open descriptor before any secret bytes are
written. The public key keeps create-time mode instead, since forcing it
would reopen an operator-tightened `fips.pub` on every start. On Windows
neither protection applies and the file inherits the parent directory's
ACLs; that exclusion is deliberate and recorded at both writers.
- Private and symmetric key material is now cleared when it goes out of scope.
Nothing in the crate erased a key before this: the node's private key sat in
the loaded configuration in plaintext for the whole process lifetime, which is
the longest any secret lives here, every Noise handshake left its static and
ephemeral keypairs, its per-message Diffie-Hellman results and its chaining
key in freed memory, and each session's ChaCha20-Poly1305 keys were dropped
intact. Clearing now covers the retained cipher key on each cipher state, the
chaining key and the handshake hash, the 64-byte HKDF outputs and the two
session keys derived from them, the static and ephemeral keypairs a handshake
holds for its whole duration, the identity's long-term keypair, the temporary
copy each of the fourteen elliptic-curve operations makes from a keypair, the
bech32 and hex encodings of a secret, and the private key on its way through
configuration, including the config file's whole text, which is treated as
secret for as long as it is held, since `node.identity.nsec` is read straight
out of it. Two places that assigned over an already-loaded key now clear the
old value first: assignment frees the previous string without running the
type's clearing destructor, so a configuration that already carried a key left
the superseded copy in the heap whenever a second source replaced it. **This
clears the copies the crate owns, not every copy that ever existed.** The
secp256k1 key types are copyable, so the compiler may duplicate them where no
code here can name them, which is why that library calls its own erase
non-secure. The hash and key-derivation states, and the cipher keys cached
inside `ring`, offer no clearing route at the versions pinned here and are
deliberately left alone; the residue any of this leaves needs access to the
process's memory, or to a core dump or swap image of it, to read. Adds a
dependency on `zeroize`. Nothing on the wire and no configuration changes.
See the `### Changed` note above for the source-breaking effect the four new
`Drop` implementations have on library consumers.
### Library API
#### Changed
- **Source-breaking for consumers of the library crate**: four public types now
implement `Drop`, so their fields can no longer be moved out. `Identity`,
`ResolvedIdentity`, `IdentityConfig` and `HandshakeState` each gained one as
part of clearing key material at end of scope. `IdentityConfig` is the one
most likely to be reached in practice, since it hangs off the public `Config`
as `node.identity`, so code that moved the nsec out of a configuration value
no longer compiles and needs `Option::take` instead. Nothing about the
behaviour of the shipped binaries changes; this affects only callers using
`fips` as a library.
### Supply chain
#### Security
- The dependency lockfile is refreshed past a set of advisories against the
pinned `nostr` 0.44.3 and `nostr-relay-pool` 0.44.1, both of which were also
yanked. `nostr` moves to 0.44.8 and `nostr-relay-pool` to 0.44.3; the
requirements in `Cargo.toml` already admitted both, so this is a lockfile
change and no code changed with it. The advisories that matter here are the
relay-pool ones, RUSTSEC-2026-0224 and RUSTSEC-2026-0232, which describe
forged events bypassing signature validation and unverified relay events
being processed: that is the path this node learns peer adverts on, and it
performs no independent verification of its own, so the exposure was a
misattributed advert rather than the denial of service the advisory summaries
lead with. RUSTSEC-2026-0231 (auth-challenge memory exhaustion) is on the
same path, and RUSTSEC-2026-0216 and RUSTSEC-2026-0227 reach the NIP-44
decryption of relay-supplied content. The remaining advisories in that set
cover NIP-04, NIP-46, NIP-50, NIP-60, NIP-98 and the wallet parsers, none of
which this code calls. The refresh was taken over the whole lockfile rather
than the two crates alone, which additionally clears RUSTSEC-2026-0204 in
`crossbeam-epoch` and leaves no yanked crate in the tree; `cargo audit` now
reports no vulnerability, against twelve before. Four warnings remain and are
not fixable by a version move: `instant` and `paste` are unmaintained, `lru`
0.16.4 carries an unsoundness advisory, and `nostr-relay-pool` itself is now
marked unmaintained.
- Every GitHub Action is pinned to a commit SHA, and the OpenWrt packaging
workflow verifies both of the artifacts it downloads. No reference in the
repository was pinned before: all sixty-six named a mutable tag and one named
a branch, including the jobs holding the AUR deploy key, the jobs with
release write scope, and the packaging jobs that run with a signing key in
the environment. Sixty-two are now full commit SHAs with the original tag
retained as a trailing comment. Four are left unpinned and justified in one
place: two actions read the tool to install from the ref name itself, so a
SHA would hand them a hex string where a toolchain name belongs. A guard
enforces the form on every sweep, treats an unreadable tree as an error
rather than a pass, and documents what it does not cover. The sharper hole
was not the tags: the OpenWrt workflow fetched a helper binary from a release
URL with no verification at all, in two jobs holding a signing key. That
download now checks a per-architecture pinned SHA-256, with the hash
provenance recorded honestly, upstream publishing no checksum document.
The same workflow's Zig toolchain fetch is verified the same way. It ran as
`curl | tar`, which leaves nowhere to check the bytes, so a short read
reached `tar` as a truncated archive and failed the build with "Unexpected
EOF in archive"; curl's `--retry` does not cover that exit. The download now
stages to a temporary directory, checks a per-architecture pinned SHA-256
with a guard that fails if an architecture is added without one, and only
then extracts, with three attempts at 10s and 20s backoff and an early exit
when two attempts return identical bytes, since a stable mismatch is a wrong
pin rather than a bad transfer. As with the helper binary, the hashes come
from the upstream download index and were checked against the tarball bytes:
that is integrity, not authenticity, because upstream publishes no detached
sums.
### Packaging, install and platform layout
#### Fixed
- macOS: `peers.allow`, `peers.deny`, and the `hosts` file are now read
from `/usr/local/etc/fips/`, matching the install layout the macOS
packaging ships (`packaging/macos/`). That layout is what the three fixes
in this group align the daemon and `fipsctl` to. The default-path constants
were hardcoded to `/etc/fips/...` with only a `#[cfg(unix)]` /
`#[cfg(windows)]` split, so on macOS the daemon looked in a directory that
does not exist: `load_file` / `load_hosts_file` hit their `NotFound` no-op
arm and silently returned an empty ACL / empty host map. A populated
`peers.deny` therefore reported `effective_mode: "default_open"` and
`enforcement_active: false` via `fipsctl acl show`, and host-file aliases
went unloaded, with no error or warning. The default constants now follow
the platform's packaging (`/usr/local/etc/fips/` on macOS, `/etc/fips/` on
Linux and other Unix for the ACL files, and `/etc/fips/` on Linux and
`%ProgramData%\fips\` on Windows for the hosts file) and are pinned by
platform-gated unit tests so the layout cannot silently drift again. At
startup the daemon warns once if any of these files exist at the old
`/etc/fips/` location but not at the current default. Linux and Windows
behavior is unchanged. Contributed by
[@sh1ftred](https://github.com/sh1ftred).
**macOS users with existing files in `/etc/fips/` should move them to
`/usr/local/etc/fips/`.**
- macOS: `fipsctl keygen` now writes `fips.key` / `fips.pub` to
`/usr/local/etc/fips/` by default. The default output directory was
hardcoded to `/etc/fips` for all Unix, but the daemon derives its identity
key paths from the config file's directory, which is
`/usr/local/etc/fips/fips.yaml` on macOS, so a generated identity landed
where the daemon never reads it and the node silently kept an ephemeral
identity. Linux and other Unix keep `/etc/fips`, Windows is unchanged, and
the values are pinned by platform-gated unit tests.
- macOS: the system-wide config search path now includes
`/usr/local/etc/fips/fips.yaml` in addition to `/etc/fips/fips.yaml`.
Previously only `/etc/fips/fips.yaml` was probed, so a bare `fips` run
without `--config` skipped the installed config and derived identity key
paths from a non-existent directory. `/etc/fips/fips.yaml` is still probed
first so existing installs keep working. Both the macOS entry in the search
path and the directory `fipsctl keygen` writes to read the shared
`SYSTEM_CONFIG_DIR` constant, so the two cannot drift apart. The
launchd-installed daemon was unaffected (it always passes `--config`).
Linux and Windows behavior is unchanged. Because the daemon derives the
identity key directory from whichever config file loaded last, a macOS host
carrying `fips.yaml` at both locations would have resolved `fips.key` to the
new directory, found none, and under `persistent` generated a fresh
identity, silently changing its npub, routing address and mesh IPv6. The
daemon now adopts a key stranded at `/etc/fips/fips.key` and warns to move
it, instead of generating one. The fallback is confined to keys resolved
from the system config directory, so a run using `./fips.yaml` or a user
config is never redirected to a system key.
- The maintainer address published in package metadata no longer bounces. The
crate authors field, the Debian package maintainer and upstream contact, and
both AUR PKGBUILD maintainer lines carried an address that no longer accepts
mail, so the contact of record in every artifact we ship was unreachable.
### Docs, CI and contributor tooling
#### Added
- `SECURITY.md`, stating a private channel for vulnerability reports, what a
useful report contains, what a reporter can expect back and on what timing,
and which branches receive fixes. The repository previously documented no
reporting channel at all, so someone with a finding had to guess at an
address or open a public issue.
#### Changed
- Two CI runs on one machine can no longer collide. Every suite derives its own
docker build context, image tag, container names, network range and host
interface names per run, so concurrent runs cannot reap each other's
containers or contend for a fixed subnet. This is what a contributor running
`testing/ci-local.sh` alongside a GitHub run, or two local runs at once, sees
change: the runs stay independent instead of one killing the other.
- A test that does not run, or whose result cannot be read, no longer passes
silently. A failed scenario now fails the run rather than being logged and
stepped over, a node whose logs cannot be read no longer counts as clean, an
unanswered control query no longer reads as zero, an unknown scenario key is
rejected instead of matching nothing, and a skipped check appears in the
final verdict rather than only in scrollback.
- New guards run in both the local and GitHub runners, so the two gates agree.
They check that trailing-log call sites are wired, that the log strings the
harness matches on are still emitted by the daemon, that the two runners'
integration-suite sets match per leg rather than as a folded token, that
every GitHub Action reference is pinned in the required form, and that source
comments do not cite references a reader of the published tree cannot
resolve.
- Coverage moved from Docker to deterministic in-process tests, and dead
scenarios were retired. The six cost-selection chaos scenarios, the
admission-cap and acl-allowlist Docker suites, the smoke-10 scenario, the
tcp-chain and mesh-public static topologies, and three ignored Ethernet
tests are gone, with their behaviour asserted in unit and integration tests
instead. A local CI run is correspondingly shorter and less dependent on
container timing.
- The `bloom-storm` chaos scenario no longer runs on either the local or the
cloud runner. Unlike the retirements above it has no replacement: the
scenario files remain in the tree and it stays runnable by hand, but nothing
now exercises downstream containment of a mid-chain ancestor swap on a
schedule. This is recorded as a coverage gap rather than as a completed
migration.
- A failing harness now says why it failed. The dns-resolver suite sent build
and container-start output to `/dev/null`, so a failed scenario reported
that it had failed and nothing else; output is now captured and emitted on
failure, naming the command, and the systemd readiness wait dumps container
state, failed units and the journal when it gives up. The NAT-lab path
assertions exited bare, printing neither what they expected nor what they
saw and triggering none of the scenario diagnostics their siblings already
call; all twelve call sites now report the container, the expectation, the
observation and a projection of the peer or link table, and distinguish a
failed control-socket exec from unparseable output from a genuine mismatch.
The convergence gate could not tell a tree that did not converge from
connectivity that failed, and could exit non-zero while reporting "20
passed, 0 failed"; it now records the outcome, the count reached and the
count pending, and its failure messages name the condition. A passing run
is as quiet as before, and no timing, threshold or control-flow behaviour
changed in any of the three.
- A dns-resolver scenario no longer burns the full 30-second boot timeout and
warns about a container that booted correctly. The readiness poll ran under
`pipefail` and piped `systemctl is-system-running` into `grep`, and that
command exits non-zero when the system is degraded, which is where systemd
inside a container always settles; the pipeline therefore failed even when
the pattern matched, leaving the degraded branch dead. The poll now matches
on the captured state instead of piping into `grep`.
- The chaos harness now checks that teardown and node stops did what they
report. `docker compose down` exits 0 while leaving a run's containers
alive, so a partly-failed bring-up leaked named containers with nothing to
detect it; teardown now asks whether the containers this run owns are gone,
treats a survivor that forced removal clears as a warning, and aborts with
the names written to an artifact when one survives that or the query cannot
run at all. The check is scoped to a run's own names, so concurrent runs
cannot trip each other. Node churn separately marked a node down whether or
not `docker stop` succeeded, so the simulation's model of the mesh diverged
from reality, and `nodes_down`, the `max_down_nodes` cap and the
connectivity guard are all computed from that model; a failed stop now
warns, carries the daemon's own message, and leaves the node out of the
down set for the next churn tick to retry against an honest model.
- Comments throughout the source tree, the packaging files and the test scripts
no longer cite internal identifiers, planning documents or private stage names
that a reader of the published tree cannot resolve; each now states the thing
the citation stood for. A handful of comments that described behaviour the
code does not have (the control-plane read path, its snapshot dispatch, and
the MMP report types) have been corrected rather than merely reworded. One of
the edited files, the DNS setup helper, installs to `/usr/lib/fips` on every
packaging path, so its comment reached users. No code changed.
#### Security
- The security reference now records that both Noise patterns deviate from the
standard construction in one respect: the handshake AEAD passes an empty
associated-data field where standard Noise `EncryptAndHash` uses the handshake
hash `h`. The published tables named `Noise_IK_secp256k1_ChaChaPoly_SHA256`
and `Noise_XK_secp256k1_ChaChaPoly_SHA256` unqualified, so anyone auditing the
stack against the Noise specification had nothing telling them where to look.
Domain separation and Diffie-Hellman binding survive through the chaining key,
which is seeded from the protocol name and chained at every step; transcript
binding is the property actually absent. Nothing in the daemon reads the
handshake hash, so no shipped behaviour rests on it, but the comments that
called it transcript binding or channel binding overstated it and now describe
what the field is, and the field records that anything later built on it
(channel binding, an exporter, cookie binding) will silently not work until
the associated data carries `h`. This is a correction to what is documented
and claimed; no code behaviour and nothing on the wire changed.
## [0.4.1] - 2026-07-19
### Changed
- `node.bloom.max_inbound_fpr` default raised from `0.10` to `0.20`. The
cap rejects inbound `FilterAnnounce` whose FPR (`fill^k`) exceeds it. On
the fixed 1 KB / k=5 filter, `0.10` corresponds to fill 0.631 (~1,630
reachable entries), and the busiest nodes' aggregates had again begun to
reach it as the mesh grew. `0.20` (fill 0.7248, ~2,114 entries) restores
headroom without materially weakening the antipoison gate: a saturated or
poisoned filter is ~100% FPR and still rejected. This is the second raise
of this cap in two releases; the fixed 1 KB filter is the underlying
constraint, and the structural remedy is the v2 filter work rather than a
further raise. A node running this default accepts announcements that a
v0.4.0 node drops, so during a rolling upgrade the two versions can
disagree about mesh size.
- Bloom filter probing computes its SHA-256 digest once per operation
rather than once per hash function. All k indices were already derived
from a single digest, but the digest was recomputed inside the
per-function loop, so every insert and membership test hashed the same
bytes `hash_count` times (5x at the default). Output is bit-for-bit
identical; this is the hottest path in packet forwarding and mesh-size
estimation.
- Identity operations reuse one shared `secp256k1` context instead of
constructing a fresh one at every sign, verify, and key-derive site.
Each construction allocated a context and ran randomization and blinding
table setup. Behavior is unchanged: the same API calls are made, only the
context lifetime differs, and the shared context still performs the
standard construction-time blinding.
### Fixed
- Spanning tree: the coordinate cache is now invalidated when the parent
link is lost through peer removal. That path reparents or self-roots the
node but omitted the invalidation every other position-change path
performs, so cached entries for downstream destinations kept the node's
now-stale coordinate prefix. Because routing access refreshes an entry's
TTL, an actively routed stale entry never self-expired and was corrected
only by a fresh insert.
- Discovery: applying a `LookupResponse` now keeps the tighter of the
cached and received `path_mtu` rather than overwriting unconditionally.
A looser estimate arriving in a later response could clobber a tighter
value already learned from a reactive `MtuExceeded` or
`PathMtuNotification`, loosening a clamp that had been correctly
tightened.
### Removed
- The `parent_switched` spanning-tree metric counter. It was incremented on
the line immediately before `parent_switches` at every site and never
independently, so the two were always identical. `parent_switches`
remains as the sole counter. Consumers reading `parent_switched` from the
control socket or `fipstop` should use `parent_switches`.
## [0.4.0] - 2026-06-27
### Added
#### Transports (Nym, mDNS LAN discovery)
- Nym mixnet transport (`transports.nym`) for outbound peer links
tunneled through a local `nym-socks5-client` SOCKS5 proxy into the
Nym mixnet, as a privacy transport alongside Tor. Outbound-only and
not platform-gated, it reuses the existing FMP framing and adds no new
crate dependencies. A single-container example
(`examples/sidecar-nostr-mixnet-relay/`) demonstrates FIPS peering
across the mixnet end to end.
- Opt-in mDNS / DNS-SD LAN discovery for sub-second pairing of peers on
the same local link, without a relay or NAT-traversal roundtrip.
Disabled by default; operators enable it with
`node.discovery.lan.enabled: true`. Configurable service type and an
optional `node.discovery.lan.scope` that isolates discovery to peers
sharing the same private-network scope. The advertised UDP port is
chosen from a non-bootstrap operational UDP transport using a stable
selector, so it is deterministic across restarts.
#### Admission / peer-list management
- `Node::update_peers` for runtime peer-list refresh, returning an
`UpdatePeersOutcome` summarizing added, removed, and retained peers.
Re-derives active peer connections from a new peer configuration
without dropping links to peers that remain in the set.
`PeerAddress` gains a `seen_at_ms` recency field (with
`with_seen_at_ms`) used to prefer more recently observed addresses.
#### Data-plane / metrics / observability
- Typed `RejectReason` classification for receive-path silent-rejection
sites across the node. Each rejection-and-return path now passes a
typed reason to `NodeStats::record_reject`, which routes it to a
per-subsystem counter, so operators can see what is being rejected
through stats counters rather than by scraping debug logs. New
`HandshakeStats`, `SessionStats`, and `MmpStats` sub-stats join the
existing `TreeStats`, `BloomStats`, `DiscoveryStats`, and
`ForwardingStats`, and `TreeStats::ancestry_invalid` is now
incremented from the `TreeAnnounce::validate_semantics` rejection
site that was previously silent. Several handshake, MMP, tree, and
discovery rejection paths that had no counter at all are now counted,
including the `send_lookup_response` no-route drop
(`DiscoveryStats::resp_no_route`).
- Internal atomic metric registry (`Arc<MetricsRegistry>`) that shadows
the plain-`u64` `NodeStats` counters, written alongside them and
validated by a whole-struct debug-build parity check. Covers the
forwarding receive counters, the full discovery counter family, and the
tree, bloom, congestion, and error-signal counter families, with
the hottest counters cache-line padded. Behavior-neutral:
`NodeStats` remains the serving path. Groundwork for sampling metrics
without contending the receive loop.
- `fipsctl stats metrics`, backed by a new counter-only `show_metrics`
control query that dumps the atomic metric registry as flat counter
name/value pairs. Serves a Prometheus-style scraper that samples node
counters without contending the receive loop.
- `pool_inbound` and `pool_outbound` counters on the TCP and Tor
transport stats (`TcpStats`, `TorStats`). Per-direction accounting
is updated at every pool-insert and receive-loop-exit site, plus on
transport stop and on send-failure-driven removal. Surfaces through
`TcpStatsSnapshot` and `TorStatsSnapshot` for `show_transports`.
#### Spanning-tree / mesh-size / routing
- Six route-class transit counters that partition transit-forwarded
packets by their tree relationship to the chosen next hop: tree-up
(peer is our ancestor), tree-down (peer is our descendant and the
destination is within its subtree), tree-down-cross (peer is our
descendant but the destination is outside its subtree), cross-link
descend (lateral peer, destination within its subtree), cross-link
ascend (lateral peer, destination outside its subtree), and
direct-peer. The six classes sum to `forwarded_packets` (asserted by a
unit test) and are computed from tree coordinates at the transit
chokepoint, so error-signal routing callers are excluded. They surface
through the forwarding stats snapshot via `show_routing` and
`show_status`.
- Discovery now counts `LookupRequest`s dropped when the dedup cache is
full. A saturated `recent_requests` cache
(`MAX_RECENT_DISCOVERY_REQUESTS`) previously dropped requests
silently; a new `DiscoveryStats::req_dedup_cache_full` counter (typed
reject reason `DiscoveryReject::ReqDedupCacheFull`) makes the drop
visible through `show_routing`.
#### Packaging & deployment
- OpenWrt `.apk` packaging (`packaging/openwrt-apk/`, `make apk`) for
OpenWrt 25+, where apk-tools is the mandatory package manager (the
existing `.ipk` continues to cover OpenWrt 24.x and earlier). Built
SDK-free: it reuses the `.ipk` cross-compile (`cargo-zigbuild`) and the
shared installed-filesystem payload, and assembles the package with
`apk mkpkg` from apk-tools 3.0.5 built from source — no OpenWrt SDK
image. A `build-apk` CI job (aarch64, x86_64) builds and structurally
verifies the package; releases now publish `.apk` artifacts and
checksums alongside `.ipk`. Packages are unsigned, installed with
`apk add --allow-untrusted`, matching the `.ipk` posture.
- Nix flake (`flake.nix` at the project root) for reproducible
from-source builds on Nix/NixOS. Builds all four binaries (`fips`,
`fipsctl`, `fips-gateway`, `fipstop`), pins the exact toolchain from
`rust-toolchain.toml` via fenix, and wires the build-time native
dependencies (`libclang` for `bindgen`, plus `dbus` and `pkg-config`),
so it needs no host setup beyond Nix with flakes enabled. Flake inputs
are lock-pinned (`flake.lock` committed) for reproducibility, and the
flake exposes `nix build`, `nix run`, a `nix develop` dev shell with the
pinned toolchain, and `nix flake check`. The flake produces binaries
(and a NixOS `packages.<system>.fips` output); the systemd/service
integration that the `.deb`/tarball installers provide is handled
through the NixOS configuration instead.
#### Docs & contributor tooling
- [`PR-REVIEW.md`](PR-REVIEW.md) — the 13-criteria PR review checklist
the maintainer runs against every incoming PR, published at the
repo root so contributors can run the same pass on their own change
(directly or by handing the document to a coding agent) before
opening. Linked from `CONTRIBUTING.md` under "Submitting pull
requests" and "Further reading". Running the checklist before
opening surfaces problems that would otherwise come back as review
comments, saving a round trip.
- [`docs/how-to/tune-file-descriptors.md`](docs/how-to/tune-file-descriptors.md)
— an operator how-to for raising `RLIMIT_NOFILE`. A busy node opens
roughly three file descriptors per established UDP peer (a
`connect()`-ed socket plus a 2-FD drain self-pipe), so the default
1024 soft limit is exhausted near 320 peers, after which further
admission, handshakes, and discovery fail with `EMFILE`. The guide
documents the per-peer FD budget and symptom, the systemd
(`LimitNOFILE` drop-in) and OpenWrt (procd `nofile`) procedures to
raise the limit, and how to verify the per-peer ratio stays bounded.
Linked from the how-to index.
### Changed
#### FMP/FSP rekey reliability
- `complete_rekey_msg2` now returns the remote peer's startup epoch
alongside the new Noise session, so the rekey path can detect a peer
restart and clear stale session state.
#### NAT traversal / Nostr discovery
- Nostr discovery startup is now non-blocking. `Node::start` no
longer waits for relay connect, subscribe, or initial advert
publish before returning. A slow or unreachable relay no longer
holds node startup hostage; local transports come up immediately
and the relay path catches up asynchronously in background tasks.
Subscribe retries with exponential backoff (2 s base, 60 s cap),
publish attempts time out at 10 s, and the new tasks are aborted
cleanly on `Node::stop`.
#### Spanning-tree / mesh-size / routing
- Active-peer path selection now sorts address candidates by recency
(`seen_at_ms`), preferring the most recently observed address when
racing concurrent path probes.
- Per-tick work budgets bound the connection churn done in a single
node tick: `MAX_DISCOVERY_CONNECTS_PER_TICK`,
`MAX_RETRY_CONNECTIONS_PER_TICK`, and
`MAX_PARALLEL_PATH_CANDIDATES_PER_PEER`. Work beyond a tick's budget
is deferred to the next tick rather than discarded.
#### Admission / peer caps
- `node.bloom.max_inbound_fpr` default raised from `0.05` to `0.10`. The
cap rejects inbound `FilterAnnounce` whose FPR (`fill^k`) exceeds it. On
the fixed 1 KB / k=5 filter, `0.05` corresponds to fill 0.549 (~1,300
reachable entries) and had begun rejecting the busiest nodes' aggregates
as the mesh approached that size. `0.10` (fill 0.631, ~1,630 entries)
restores headroom toward the fixed-filter capacity limit without
materially weakening the antipoison gate: a saturated or poisoned filter
is ~100% FPR and still rejected.
- TCP inbound connection cap now honors `node.limits.max_connections`.
The per-transport TCP inbound accept ceiling was hardwired to 256 and
never read `max_connections`, so raising it was a silent no-op for
inbound TCP. The effective cap now resolves with precedence: explicit
per-transport `max_inbound_connections`, then node-wide
`max_connections`, then the built-in default of 256. Established peers
remain bounded node-wide by `add_connection`.
#### Data-plane / worker-pool / metrics / observability
- The control-socket read surface is now served off the `rx_loop`.
Every pure-read `show_*` query — `show_status`, the `show_stats_*`
family, `show_listening_sockets`, the new `show_metrics`,
`show_tree`/`show_bloom`/`show_cache`/`show_routing`/
`show_identity_cache`, `show_peers`/`show_sessions`/`show_links`/
`show_connections`/`show_transports`/`show_mmp`, and `show_acl` — now
renders in the control accept task from ArcSwap-published read
snapshots instead of round-tripping the data-plane receive loop; only
the mutating `connect`/`disconnect` commands still reach the loop.
This removes the head-of-line coupling where a busy or slow `rx_loop`
could time out `fipsctl` and `fipstop` observability (the five-second
query pattern operators saw on loaded nodes). Per-entity snapshots
reuse unchanged rows by pointer, so per-tick publish cost stays
bounded as peer/session count grows. New daemon-resolved fields
surface through the snapshots: effective persistence, root/is-root,
and a per-transport-type peer-count map in `show_status`; per-peer
`effective_depth` in `show_peers`; `root_npub` in `show_tree`; and the
last-sent uptree filter fill ratio and subtree estimate in
`show_bloom`.
- `fipstop` TUI overhaul: reworked rendering, navigation model, and the
control read surface it draws from, surfacing the new daemon-resolved
snapshot fields above. Built on a ratatui `TestBackend`
render-snapshot harness that asserts the text grid and per-cell
style of every `ui::draw_*` against canned `show_*` JSON.
- Steady-state log noise reduced on saturated public-mesh nodes.
Routine per-peer connection-lifecycle and capacity-cap events are
demoted from info/warn to debug — FMP K-bit cutover promotion,
connection-promoted-to-active-peer (a redundant duplicate
promotion line removed), peer-restart-detected, peer-removed-and-
cleaned-up, the TCP `max_inbound_connections`-reached rejection, and
the congestion-CE-flag line — so genuinely notable info/warn lines
are no longer drowned out. An exhausted FMP-msg1 / FSP-msg3 rekey
retransmission-budget abort (an expected, self-limiting outcome on
lossy or high-latency links) is likewise demoted from warn to debug.
- macOS UDP receive path now batches up to 32 datagrams per kernel
wakeup via `recvmsg_x(2)`, matching the Linux `recvmmsg(2)`
amortization shape introduced in v0.3.0. Previously macOS fell
through to single-packet `recv_from`, capping inbound rate on
Apple builds with the same per-syscall + per-task-wakeup overhead
Linux had already eliminated. `recvmsg_x` is an xnu-private syscall
declared via `unsafe extern "C"` against a local repr(C)
`msghdr_x`; same approach used by `quinn-udp`. Same
`(count, kernel_drops)` contract as the Linux path, with
`kernel_drops` always 0 on macOS (no `SO_RXQ_OVFL` equivalent).
Bench numbers on aarch64-apple-darwin (100B payloads, 3 s
windows): 1 sender 1.09x, 2 senders 1.72x, 4 senders 1.56x,
8 senders 1.46x.
- Receive hot path: removed two per-packet copies. New borrowed
`SessionDatagramRef` decoder is used in the forwarding handler so
local delivery and coordinate-cache warming no longer allocate or
copy the session payload; the owned `SessionDatagram` is materialized
only when re-encoding for the next hop. Owned `SessionDatagram::
decode` is reimplemented as `Ref::decode + into_owned`, so the two
decoders cannot drift. On Linux + macOS the `recvmmsg` / `recvmsg_x`
receive loop now moves each filled slot buffer into `ReceivedPacket`
via `mem::replace` instead of cloning it, and `TransportAddr` is
formatted directly from the `SocketAddr` without an intermediate
`String`. Focused decode bench: ref 1.6 ns/op vs owned 34.7 ns/op
(21.4x).
- Quieted non-Linux test-build warnings from intentionally
platform-specific code: the nftables firewall parser
(`#[allow(dead_code)]` now gated to non-Linux targets where the
parser is compiled but unused), the macOS `utun` address-family
helper and the long TUN reader entry point (narrow allowances),
and a macOS Ethernet test module's clippy struct-layout lint
(rewritten MAC-copy loop, explicit layout annotation). No
behavioral change; the goal is to keep `cargo test` and
`cargo clippy` clean on cross-platform builds so unrelated
warning fixes don't get bundled into behavioral PRs.
- Data-plane: AEAD encrypt and AEAD decrypt now run on per-shard
worker-pool threads (`std::thread` + `crossbeam_channel`), off the
rx_loop. Hash-by-destination dispatch pins each TCP flow to one
worker so wire ordering is preserved; per-worker `sendmmsg(2)`
batches up to 32 outbound packets per syscall, with UDP_GSO
(`UDP_SEGMENT`) when the batch is uniform-sized — the same kernel
primitive WireGuard's in-kernel module and Cloudflare's userspace
BoringTun use to hit multi-Gbps single-stream rates. On Linux +
macOS each established UDP peer also gets a dedicated `connect(2)`-
ed kernel socket bound to the same wildcard listen port via
`SO_REUSEPORT`, so the kernel caches per-packet route + neighbor
lookup and the worker sends with `msg_name = NULL`. The receive
side mirrors: per-shard thread-local `HashMap` owns each session's
recv cipher + replay window, replacing the previous shared
`RwLock`. Sessions are re-registered with the decrypt pool on
K-bit flip and rekey cutover, and unregistered on rekey drain
completion and peer removal so the per-shard tables stay bounded.
New `crossbeam-channel = "0.5"` dependency. Worker counts default
to `num_cpus`; both pools are overridable via
`FIPS_ENCRYPT_WORKERS` and `FIPS_DECRYPT_WORKERS` (the latter
accepts `0` to disable the pool and fall back to in-line decrypt
in rx_loop). Per-peer connected UDP can be disabled via
`FIPS_CONNECTED_UDP=0`. Optional per-stage timing reporter
available via `FIPS_PERF=1` (or `FIPS_PIPELINE_TRACE=1`); detailed
knob documentation is a follow-up at
`docs/how-to/tune-worker-pools.md`. Bench (5 × 15 s × 1 stream
medians, Linux x86_64, docker-bridge mesh): A→D 1379→2708 Mbps
(1.96×), A→E 1394→2663 Mbps (1.91×), E→A 1406→2624 Mbps (1.87×);
RTT +0.11–0.19 ms from the worker queue handoff. Windows
continues on the existing tokio-based send/recv path. Two issues in
the off-rx_loop drain path are resolved as part of the overhaul: the
per-peer drain worker is now detached on `Drop` rather than joined
synchronously (a synchronous join from the runtime thread could wedge
the whole daemon when a peer was removed with an in-flight worker),
and the connected-UDP drain no longer busy-spins on a poll error
(#106).
#### Transports & config
- Static host aliases in `/etc/fips/hosts` now hot-reload on mtime
change instead of only at daemon startup, so `fipsctl`/`fipstop`
display names reflect edits without a restart. The peer ACL and host
map both reload once per node tick through a new lock-free
`Reloadable` snapshot.
- Sidecar example (`examples/sidecar-nostr-relay`): `udp.mtu` is now
overridable via the `FIPS_UDP_MTU` environment variable, defaulting to
1472 (preserving prior behavior). Plumbed through `docker-compose.yml`
and documented in the README env-var table. Annotated the static-CI
node template `mtu: 1472` literal with the same Docker-bridge
rationale and a pointer at the daemon's 1280 default.
#### Packaging & deployment
- The Debian package no longer ships `/etc/fips/fips.yaml` as a dpkg
conf-file. The default configuration is installed as an example at
`/usr/share/fips/fips.yaml.example`, and `postinst` seeds
`/etc/fips/fips.yaml` (mode 600) from it only when the file does not
already exist — so a configuration-management-rendered or
operator-edited config is never prompted for or clobbered on
upgrade, removing the need for a `dpkg-divert` workaround.
`fips.service` gains `ConditionPathExists=/etc/fips/fips.yaml`. The
example is placed under `/usr/share/fips`, deliberately outside
`/usr/share/doc`, which minimal and container installs path-exclude
(so the install-time seed source is never dropped).
- openwrt: the `.apk` package now defaults `ethernet.wan` to the
OpenWrt 25 DSA port name `wan`; the `.ipk` package keeps `eth0` for
OpenWrt 24 and earlier.
#### CI & test-harness reliability
- CI and release-publish workflows hardened:
- `ci.yml` declares a top-level `concurrency` block keyed on
`(workflow, ref)` with `cancel-in-progress: true`. Force-pushes
and rapid successive pushes to the same ref now retire any
in-flight run rather than letting superseded and current-tip runs
both burn runner minutes.
- `aur-publish.yml` rewritten to fetch the upstream source tarball
and compute its `b2sum` in CI, then patch `pkgver` and the
`b2sums` SKIP placeholder in `PKGBUILD` in-place. Previously
`updpkgsums: true` downloaded the tarball into the AUR working
tree, where it was rejected by AUR's 488 KiB max-blob hook —
silently no-op'ing the v0.3.0 stable AUR push. `fips.sysusers` /
`fips.tmpfiles` asset b2sums are recomputed in the same step to
stay in sync with the local files. `workflow_dispatch` gains a
tag input so historical release tags can be re-published
manually, and `continue-on-error: true` is dropped so future
regressions surface in CI.
- New `aur-publish-git.yml` workflow for the `fips-git` VCS
PKGBUILD, triggered on master pushes touching `PKGBUILD-git` or
companion files plus `workflow_dispatch`. `pkgver` is computed at
build time by the PKGBUILD's `pkgver()` function, so this workflow
is not tied to release tags.
- Tag-triggered `package-*` release-build workflows remain
untouched.
- Local and GitHub CI integration coverage brought into parity, and
the Rust toolchain selection given a single source of truth:
- The `admission-cap` integration suite, previously run only by
`ci-local.sh`, now also runs as a GitHub `ci.yml` matrix leg, so a
regression in it turns the GitHub gate red rather than depending on
a developer remembering to run local CI. A new
`testing/check-ci-parity.sh` (wired as `ci-local.sh
--check-parity`) diffs the two runners' integration-suite sets and
fails on unexpected drift; the deliberate local-only (live-Tor)
and granularity-only differences are documented in a comment block
atop both runners.
- CI and packaging jobs now select the toolchain with
`actions-rust-lang/setup-rust-toolchain` (which reads
`rust-toolchain.toml`) instead of `dtolnay/rust-toolchain@stable`.
The pinned channel already overrode the installed stable, so each
job downloaded an unused toolchain and logged a misleading `rustc`
version; the single-source action removes the waste and the
confusion. Existing cache steps are kept (`cache: false` on the
new action) and `RUSTFLAGS` is left untouched so no global
`-D warnings` is newly imposed. The OpenWrt nightly Tier-3 leg
keeps `@nightly`.
#### Docs & contributor tooling
- Overhauled `CONTRIBUTING.md`: replaced generic Rust-template framing
with a FIPS-specific entry point covering the four-layer
architecture, branch model and PR-target selection, structured bug
reporting, scope discipline and local-CI requirements, an AI coding
assistant policy, and project communication channels. Added
`docs/branching.md` as the long-form companion covering the release
workflow, version conventions, and merge-direction rationale.
### Fixed
#### FMP/FSP rekey reliability
- FMP link-layer rekey is now reliable under packet loss, bringing it up
to the FSP session layer's rekey discipline. The rekey msg1
retransmission driver was previously uncapped and never abandoned, so a
rekey that never completed resent msg1 forever; it now uses a bounded
retransmission budget (`handshake_max_resends` with exponential
backoff) and abandons the rekey cycle cleanly once the budget is
exhausted, mirroring the FSP rekey msg3 driver. With the cap in place
the link-dead heartbeat is rekey-aware: `check_link_heartbeats` no
longer reaps a link that is still actively carrying rekey-handshake
traffic, while a genuinely dead link is still reaped once the budget
abandons. At the K-bit cutover the receiver now authenticates an
inbound frame against the pending session before promoting it, instead
of promoting on the bare header K-bit; under jitter a node could
otherwise promote a stale pending session, leaving the two endpoints on
different keys and silently dropping traffic until the link died — the
same failure class already closed on FSP, now closed on FMP.
- FSP session rekey is now hitless under packet loss and reordering.
Previously, a rekey could leave the two endpoints holding different
key sets for a brief window — if a handshake message was lost in
transit one side rotated keys while the other did not, and traffic
sealed in one key epoch reached a peer still on the other epoch and
failed to decrypt, producing bursts of AEAD decryption failures and
dropped connectivity until a later rekey reconverged the pair. The
receive path now trial-decrypts each frame against every live key
epoch (current, pending, and the draining previous session) for the
duration of the rekey transition, so no rotation ordering and no
packet reordering can cause a decryption failure. The previous-epoch
slot is retained as long as the peer keeps using it, with its drain
deadline anchored on the last frame the peer authenticates against
it rather than a fixed wall-clock timer, so a peer that did not
receive the new keys is not stranded by a silent permanent decrypt
failure. The lost-handshake case is closed by retransmitting the
third rekey handshake message until the peer is confirmed on the
new keys, with a bounded retry budget after which the rekey cycle
is cleanly abandoned and retried. There are no FSP decryption
failures across a rekey under lossy, jittery links.
- ±15s symmetric jitter is applied per session to the FMP and FSP rekey
timer trigger, eliminating the steady-state dual-initiation race in
symmetric-start meshes (previously the smaller-NodeAddr tie-breaker
resolved correctness only after every cycle's collision).
`node.rekey.after_secs` becomes the nominal interval rather than a
floor; the mean is preserved.
- A stale FSP (session-layer) session is now cleared when a peer
restart is detected during FMP rekey or cross-connection promotion.
Previously the old session could linger after the peer came back
with a new startup epoch, leaving the session-layer map out of sync
with the freshly promoted peer.
#### NAT traversal / Nostr discovery
- Two nodes that each `auto_connect` to the other no longer stall their
Nostr-mediated NAT-traversal handshake. Each side ran both an
initiator and a responder traversal session, binding a separate UDP
socket per session, and adopted only the first `Established` event; if
the two sides adopted mismatched sessions, each sent its Noise msg1 to
a peer port the peer had already stopped draining and both handshakes
hung until the adoption budget expired. The responder now elects a
single session deterministically — it declines an incoming offer only
when it also has an in-flight outbound initiator for the same peer and
its own NodeAddr is smaller — so one matching socket pair survives on
both ends and the peer's redundant initiator times out harmlessly.
One-sided (asymmetric) `auto_connect` has no co-active initiator and is
never suppressed, so connectivity is preserved.
- NAT-traversal cross-init adoption is now deterministic under
simultaneous dual-initiation. Previously, when two peers'
Nostr-mediated UDP punches completed within the same scheduling
window, each side's bootstrap-completion event arrived with an
in-flight handshake already recorded against the other peer (each
side had received an inbound msg1 from the other's pre-punch
outbound attempt). The deduplication skip then fired on both
sides, neither installed the fresh traversal socket as canonical,
and the 45-second peer-adoption budget expired with both nodes
stuck waiting for an adoption that never happened. The handler now
applies the same deterministic NodeAddr tie-breaker the codebase
already uses for rekey dual-initiation and cross-connection
resolution: the smaller NodeAddr wins as adopter, tears down its
in-flight handshake state, and proceeds with adoption; the larger
NodeAddr keeps the skip semantics, and its in-flight outbound is
reconciled by the cross-connection logic when the winner's fresh
msg1 arrives over the adopted socket. The dual cross-init stall is
eliminated; cross-init NAT-traversal completes in well under a
second even under host CPU contention.
- Nostr-discovered NAT-traversal events (`BootstrapEvent::Established`
and `BootstrapEvent::Failed`) for peers that are already connected
or actively handshaking are now short-circuited at the
`poll_nostr_discovery` dispatch sites before any cooldown
bookkeeping or fallback retry scheduling runs. Stale `Failed` events
previously poisoned the per-peer failure-state cooldown of healthy
peers and could trigger redundant retraversal attempts via
`schedule_retry` / `try_peer_addresses`; stale `Established`
handoffs could attempt to adopt a second socket against a live
connection. A defense-in-depth guard was added to
`adopt_established_traversal` so the same invariant holds if a
future caller bypasses the outer dispatch check. As a side benefit,
narrows a cooldown-poisoning vector previously available to an
attacker injecting stale failure events for an active peer.
- Nostr discovery now filters unroutable direct UDP/TCP advert
endpoints. Publisher and validator retain only endpoints that parse as
concrete socket addresses with routable IPs and nonzero ports;
`udp:nat` rendezvous endpoints and Tor endpoints pass through
unchanged. Adverts that collapse to zero usable endpoints after
filtering are rejected with a clear "missing publicly routable
endpoints" error. Before this change, misconfigured nodes could
publish RFC1918, loopback, link-local, CGNAT 100.64/10, IPv6 ULA,
or IPv6 link-local endpoints into Nostr discovery, and consumers
would cache and dial them; in mixed LAN/VPN/NAT environments, that
could prefer a misleading one-way private path over the intended
`udp:nat` bootstrap.
#### Admission / peer caps
- TCP and Tor `max_inbound_connections` admission cap is now compared
against the per-direction inbound count (`pool_inbound`) rather than
the combined pool size. Outbound connect-on-send connections share
the same pool data structure but no longer consume slots against the
operator-facing inbound cap. The configuration field name and
operator semantics are preserved; only the cap-check comparison and
accounting change. Operators with mixed outbound + inbound
deployments no longer see legitimate inbound peers rejected once
outbound connections fill the pool past the configured cap.
- Outbound connection initiation now honors the `node.limits.max_peers`
cap that was previously only checked on inbound msg1 admission. Four
paths gated: auto-reconnect retries (`process_pending_retries`),
Nostr-mediated discovery's `BootstrapEvent::Established` adoption, and
both sides of the Nostr-mediated NAT-traversal punch (offer initiation
in the runtime's outgoing path, offer acceptance in the responder's
incoming-offer handler). At saturation, a node now performs zero
outbound work on these paths; only existing peer maintenance and
overlay-advert refresh continue. The inbound gate at
`handshake.rs:1114` is unchanged. Introduces a shared
`Node::outbound_admission_check()` helper so the invariant is
grep-able and unit-testable.
- Inbound `handle_msg1` now silent-drops at `node.limits.max_peers`
saturation *before* building/sending Msg2, instead of replying with
Msg2 and then rejecting at `promote_connection`. Adds an early cap
check positioned after identity verification (so the
reconnect / cross-connection bypass for known peers still fires) and
before index allocation + Msg2 wire send. The late cap check inside
`promote_connection` is intentionally retained as
defense-in-depth. Wire savings observed in a 45 s tcpdump at
saturation: ~3.6 cap-denials/s × Msg2 (~104 B + AEAD compute) each.
Bigger win is cleaner peer-side semantics — no fake-completed
handshake whose subsequent data frames fail decryption on this side.
#### Spanning-tree / mesh-size / routing
- The mesh-size estimator (`compute_mesh_size`) no longer over-counts
under filter overlap. It previously summed the per-filter cardinality
of the parent and each child filter, which assumes the filters are
perfectly disjoint; a stale or oversized parent filter or a routing
loop inflated the reported mesh size to several times the true value,
and dropping the parent on a tree rebalance collapsed the upward leg
and flapped the count (the symptom operators saw as the size
nearly-but-not-exactly doubling during rebalancing). The estimator now
computes the cardinality of the OR-union over self plus every
connected peer's inbound filter, dropping the parent/child tree gating
entirely. OR is idempotent, so any overlap is deduplicated — the
result equals the old sum in the disjoint case, stays correct under
overlap, damps the parent-switch flap, and removes the estimate's
dependence on tree-declaration cache freshness. The per-peer 500 ms
rate-limiter and overall recompute cadence are unchanged.
- Spanning-tree state distribution is now eventually-consistent.
Previously every `send_tree_announce_to_all` call site fired only
on a local state-change event (parent switch, self-root promotion,
ancestry change, peer promotion, parent loss). Once a partition
latched — for example, a parent-switch announce lost in transit
via the brief cross-init handshake swap window where one peer's
outbound session is about to become the loser session and the
receiver has no matching decrypt-worker entry — no node's state
changed again, so no node ever re-broadcast. The existing 60-second
`check_periodic_parent_reeval` short-circuited silently on no-change
(it was a re-evaluation, not a re-broadcast), and production-side
healing depended on incidental link churn (NAT keepalive refresh,
MMP timeout, peer re-promotion after a transport blip). The
function now ends with an unconditional `send_tree_announce_to_all`
on the no-change branch, alongside the existing switch and
self-promote arms; receivers coalesce by sequence comparison
(`ParentDeclaration::is_fresher_than`) and short-circuit at the
`if !updated` gate in `handle_tree_announce`, so same-sequence
repeats drop silently with no cascade. The per-peer 500 ms
rate-limiter is well below this 60-second cadence and does not
suppress the heartbeat broadcast. `BASELINE_CONVERGENCE_TIMEOUT`
in `testing/static/scripts/rekey-test.sh` is bumped from 60 to 65
so any partition healed by the periodic broadcast at T+60 lands
inside the convergence window; `wait_for_full_baseline` early-exits
on PASS, so successful reps see no extra wall-clock.
- A single-uplink node stranded out of the tree now re-attaches within
a round-trip instead of waiting for the periodic re-broadcast cadence.
A node with one tree peer has periodic parent re-evaluation disabled,
so a lost one-shot attaching `TreeAnnounce` left it self-rooted and
unreachable until the next periodic re-broadcast
(`reeval_interval_secs` later). Tree-position exchange is now
self-healing on the receive path: when an accepted `TreeAnnounce`
advertises a root strictly worse (higher NodeAddr; election is
smallest-wins) than our own, we echo our current declaration back to
that peer, provoking the better-rooted peer to re-push its real
position immediately. The echo fires only in that one direction and is
bounded by the existing per-peer rate limiter.
- Coord cache invalidation made surgical at parent-position-change
and root-change sites. Replaces the previous unconditional
`CoordCache::clear()` calls with two targeted methods:
`invalidate_via_node(node_addr)` (drops entries whose cached
ancestry contains the changed node, used at parent-switch /
become-root / loop-detection sites) and `invalidate_other_roots`
(drops entries from a different tree, used at root-change sites).
The previous global flush left `find_next_hop` returning `None`
for every non-direct-peer destination after every parent switch
until the cache passively re-warmed; surgical invalidation
preserves entries that remain correct across the topology change.
Peer-removal retains the original "no invalidation" behavior
(`find_next_hop` already recomputes against the current peer set
every call, and Discovery handles "no route" on demand).
- `rx_loop` tick-arm stall under convergence-phase mesh pressure
is eliminated. Previously, the tick body's per-peer `check_*`
loops (heartbeats, bloom announces, MMP reports, tree announces)
called `transport.send` directly for every active peer. For
TCP/Tor peers whose pool entry was not yet established,
`send_async` fell through to a synchronous connect-on-send
branch that wrapped `TcpStream::connect` in
`tokio::time::timeout(connect_timeout_ms, …)` — 5 seconds by
default — and blocked the entire tick body for the duration per
unreachable peer. Under post-restart convergence on a high-peer
mesh, this cascaded into multi-second tick stalls; the same
mechanism also starved the master-only per-tick control-snapshot
republish and pushed `fipsctl show *` queries onto an mpsc
fallback that was itself queued behind the wedged `rx_loop`,
producing the five-second `fipsctl` head-of-line pattern
operators observed on loaded nodes. The send path now gates on
`transport.connection_state(addr)` before sending: proceed only
when `Connected`; on `None`, kick off a non-blocking background
`connect` (idempotent — deduplicates against the connecting
pool, spawns the timeout-bounded `TcpStream::connect` inside its
own tokio task) and fail this send fast with a clear
`transport connection not ready` error. A subsequent tick
retries once the pool has an entry. The existing reconnect
lifecycle (heartbeat-dead detection in `check_link_heartbeats`,
scheduled retries via `process_pending_retries`, background-
connect polling via `poll_pending_connects`) is unchanged.
The connect-on-send branch in `transport.send_async` itself
remains in place for code paths that legitimately need
synchronous connect (e.g., explicit operator-driven
`fipsctl connect`); the tick path just no longer trips it.
#### Data-plane / metrics / observability
- The Tor transport now increments its `connect_refused` statistic (the
"Refused" line in fipstop) when a SOCKS5 connection is actively
refused, instead of recording every connect failure as a generic
SOCKS5 error. The counter previously stayed at zero.
- MMP sender metrics now ignore duplicate or regressed receiver reports
before updating RTT, loss, goodput, or ETX. Receiver reports also
suppress timestamp echo when dwell time overflows, so stale reports
cannot inflate SRTT.
- Reject-reason counters no longer double-count now that the rollout's
interim direct increments are removed. Six discovery counters
(`req_decode_error`, `req_duplicate`, `req_ttl_exhausted`,
`resp_decode_error`, `resp_identity_miss`, `resp_proof_failed`), six
bloom counters (`decode_error`, `invalid`, `non_v1`, `unknown_peer`,
`stale`, `fill_exceeded`), and five forwarding reject packet counters
(`decode_error_packets`, `ttl_exhausted_packets`,
`drop_no_route_packets`, `drop_mtu_exceeded_packets`,
`drop_send_error_packets`) were each incremented both by a direct bump
and again through the typed reject dispatch. The redundant direct
increments are removed — for the forwarding family the two calls are
collapsed into a single byte-aware reject entry point — so each counter
(and, for forwarding, its byte tally) counts once per event.
- Transport-layer mutex poisoning no longer cascades. Ten
`Mutex::lock().unwrap()` sites across the UDP, BLE, and Ethernet
transports would turn a single panic (poisoning the mutex) into a
cascade of panics on every subsequent lock. Each is replaced with
`lock().unwrap_or_else(|e| e.into_inner())`, recovering the guarded
data with no new dependency and no call-graph change; four
`local_addr.unwrap()` calls on the UDP start/adopt paths get a
provably-safe sentinel fallback. The critical sections are short,
locally-scoped, and not reachable from peer input, so this is
robustness hardening, not a remotely-triggerable fix.
#### Peer lifecycle / gateway
- A manual `fipsctl disconnect` now notifies the peer so teardown is
symmetric. Previously a manual disconnect tore down only the local
side and sent the peer nothing, so the peer kept its session and never
re-emitted its tree and filter announcements; on reconnect it was
never re-adopted as a child and its bloom filter was never recorded.
The local side now sends the disconnected peer a scoped `Disconnect`
(the same message graceful shutdown sends), so both ends tear down and
re-handshake cleanly on the next connection.
- `fips-gateway` no longer drops long-lived or DNS-cached client
mappings while traffic is still flowing. The virtual-IP pool's TTL
clock advanced only on DNS re-query, never on traffic, and the mapping
TTL is wired equal to the DNS TTL, so an in-use mapping was forced to
drain at TTL and reclaimed at the first zero-conntrack tick — breaking
long-lived, bursty, or DNS-cached clients. The tick now refreshes the
mapping's last-referenced time whenever conntrack reports active
sessions, and recovers a draining mapping to active (with a fresh
grace window) when traffic resumes; only genuinely idle mappings
drain.
#### macOS self-traffic / resolver
- Self-addressed mesh traffic is now delivered locally on macOS instead
of being dropped, for both `ping6` and full TCP/UDP. The point-to-point
`utun` interface egresses self-addressed traffic into the daemon, which
previously pushed it onto the mesh outbound path where it was dropped
for lack of a route to self; such packets are now hairpinned back to
the TUN for inbound delivery. macOS first routes self-addressed packets
as loopback (a `LOCAL` route via `lo0`), which leaves their transport
TX checksum offloaded and unfinished, so re-injecting them verbatim
made the local stack drop every segment whose checksum MSS clamping did
not happen to rewrite (the SYN and SYN-ACK got through, but the bare
ACK, data, and FIN were dropped, so connections to a node's own
`<npub>.fips` service half-opened and hung). The hairpin path now
recomputes the TCP/UDP checksum before re-injection, so full
self-connections — not just `ping6` — to a node's own `<npub>.fips`
address work. Linux was unaffected (the kernel already loops
self-traffic via `lo`). (#117)
- macOS `.fips` name resolution now works on a fresh install: the
shipped resolver shim points at `::1`, matching the daemon's default
IPv6 DNS listener, instead of `127.0.0.1`. The mismatched shim
(`nameserver 127.0.0.1` while the daemon listens on `::1`) broke
`getaddrinfo` for `.fips` on every macOS install since the resolver
was introduced.
#### CI & test-harness reliability
- Node-level multi-node tests no longer flake under parallel CPU load.
They previously delivered handshake packets over real localhost UDP,
whose kernel receive buffer could overflow and drop a packet when many
tests ran concurrently, panicking the large-network convergence tests.
A `cfg(test)`-only loopback `TransportHandle` variant now delivers
packets directly between nodes over an unbounded in-process channel, so
there is no socket buffer to overflow, and the previously-quarantined
large-network tests run in the default suite again. The shipping daemon
build is unaffected (the variant is test-gated).
- Integration suites that wait for the mesh to converge no longer
false-fail under concurrent CI load. The rekey, static-mesh, and
sidecar suites replace a fixed wall-clock baseline timeout (and a blind
sleep) with a progress-aware wait that polls the suite's own pairwise
pings, returns as soon as every pair is reachable, extends its deadline
while the reachable-pair count is still climbing, and gives up only
when progress stalls.
- Rekey integration test (`testing/static/scripts/rekey-test.sh`) no
longer false-fails on GitHub runners under packet loss and CPU
contention. Phase 1, Phase 3, and Phase 5 strict per-pair pings retry
up to 4 attempts (configurable via `MAX_PING_ATTEMPTS` /
`PING_RETRY_DELAY`) — under 1% per-direction loss, single-shot 20-pair
ping_all misses ~33% per phase from ICMP noise alone, and the
4-attempt retry brings that floor to ~3.2e-6 per phase; the
`wait_for_full_baseline` convergence loop stays single-shot so retries
there cannot conflate transient ping loss with still-converging routing
state. Phase 1 baseline-convergence headroom is bumped from 36s to 60s
to eliminate the intermittent Phase 1 timeout that previously required
a `gh run rerun --failed`, and a post-second-rekey settle window is
added in Phase 5 (mirroring Phase 3's 12-second pattern) to close the
post-rekey per-pair-ping flake from convergence exceeding the per-ping
5-second timeout. Test scaffold only; no daemon code changes, and the
success path is unchanged because the wait loops return as soon as all
20 pairs converge.
- ACL-allowlist integration test (`testing/acl-allowlist/test.sh`):
converted `assert_log_contains` from a one-shot `docker logs | grep`
snapshot into a bounded poll with the same wait-with-timeout shape
as `wait_for_peers_exact`. Absorbs the millisecond-to-second
variance in the XX-handshake cross-connection tie-breaker: the
inbound-handshake-context rejection can land tens of milliseconds
after the test's previous one-shot grep gave up, producing a
pre-existing flake on CI. Success-path cost is unchanged — the helper
returns as soon as the pattern appears.
#### Packaging & deployment
- AUR packaging: the `fips` and `fips-git` PKGBUILDs now install the
`fips-dns-setup` and `fips-dns-teardown` helpers into
`/usr/lib/fips/`, matching the Debian package. The AUR `package()`
step previously omitted them, so `fips-dns.service` failed to
start on Arch installs ("Unable to locate executable
`/usr/lib/fips/fips-dns-setup`", #98). The PKGBUILDs additionally
opt out of the debug split package and declare the `*-debug`
variant as a conflict, so a stale debug build cannot own installed
files across a package switch.
- macOS package build: the `.pkg` architecture is now derived from
the Cargo `--target` triple instead of the build host's
`uname -m`. The arm64 and x86_64 release legs build on the same
Apple-silicon runner, so `uname -m` named both outputs
`fips-0.3.0-macos-arm64.pkg`; the release job's `merge-multiple`
artifact download then interleaved the two identically named
files into a single corrupt xar archive, and no x86_64 package
reached the release at all. (This shipped as the broken v0.3.0
macOS `.pkg`, GitHub #102.) The release workflow now also asserts
the arch-named file is present and carries a SHA-256 integrity
chain from the build runner through to `gh release upload`, so a
recurrence fails CI instead of publishing.
#### fipstop
- `fipstop` no longer renders a garbled screen on startup or leaves
stray bytes on quit, most visible over SSH and inside tmux. Startup
forces a full repaint (`terminal.clear()`) before the first draw so
prior alternate-screen contents no longer show through; quit gives the
stdin-poll thread a stop flag and joins it before restoring the
terminal, so post-raw-mode keystrokes or terminal query responses no
longer echo onto the restored screen.
## [0.3.0] - 2026-05-11
### Added
#### Mesh Layer (FMP)
- Overlay-discovery and NAT-hole-punching path (opt-in via
`node.discovery.nostr.enabled`). Nodes publish signed overlay adverts
as Nostr kind `37195` parameterized replaceable events listing
reachable transport endpoints to a configurable set of public relays,
and consume peer adverts to populate fallback addresses for
`via_nostr` peers or, under `policy: open`, for non-configured peers
within a budget cap. The kind value is FIPS-specific: `37195` sits in
the application-defined replaceable range `30000–39999`, and the
digits visually spell `FIPS` (7=F, 1=I, 9=P, 5=S)
- STUN-assisted UDP hole punching for `addr: "nat"` UDP endpoints. STUN
reflexive observation, gift-wrap (NIP-59) offer/answer signaling, and
candidate-pair punch planner (LAN-private + reflexive paths attempted in
parallel). Successful punches hand the live socket into the standard
FIPS UDP transport via a bootstrap-handoff API
- New `node.discovery.nostr.*` configuration tree with operator-tunable
resource caps, replay tracking, and punch timing; new `peers[].via_nostr`
and per-transport `advertise_on_nostr` / `public` flags. Cross-field
validation at startup catches mis-configured combinations
- Docker NAT lab covering cone, symmetric (TCP-fallback), and LAN
scenarios, wired into the integration CI matrix
- One-shot startup advert sweep for Nostr open-discovery. On daemon
startup under `node.discovery.nostr.policy: open`, after a short
settle delay (`startup_sweep_delay_secs`, default 5s) the cached
overlay-advert table is iterated once and recent adverts (newer
than `startup_sweep_max_age_secs`, default 3600s) are queued for
outbound retry, modulo the same skip-filters as the per-tick sweep
(configured peer, already connected, retry-pending, connecting).
Closes the gap where peers learned only through relay backlog at
startup were not dialed until they republished.
- Diagnostic logging on the open-discovery sweep. Each `queued retry`
now logs at info-level with the peer short-npub and advert age,
and a one-line summary (cached count, queued count, per-reason
skip counts) is emitted on every startup sweep and on any per-tick
sweep that queues at least one retry. Operator-facing visibility
into what the auto-dial path is doing.
#### Platform Support
- Windows platform support: wintun TUN device, TCP control socket on
`localhost:21210` (in place of the Unix domain socket), Windows
Service lifecycle (`--install-service`, `--uninstall-service`,
`--service`), ZIP packaging with PowerShell install/uninstall scripts,
and CI build/test matrix entry
([#45](https://github.com/jmcorgan/fips/pull/45))
- macOS platform support: native `utun` TUN interface management, raw
Ethernet transport via BPF, `.pkg` packaging with launchd plist and
uninstall script, x86_64 cross-compile from arm64, and CI build/unit
test jobs
- MIPS atomic ABI support: `std::sync::atomic` replaced with
`portable_atomic` so 32-bit MIPS targets without native atomics
link cleanly
([#62](https://github.com/jmcorgan/fips/pull/62),
[@andrewheadricke](https://github.com/andrewheadricke)).
#### Mesh Peer Transports
- Bluetooth Low Energy (BLE) L2CAP Connection-Oriented Channel
transport (Linux only, requires BlueZ): per-link MTU negotiation,
continuous scan/probe peer discovery with cooldown-based
deduplication, continuous advertising, deterministic NodeAddr
cross-probe tie-breaker, and a configurable connection pool with
eviction.
- `transports.udp.outbound_only` (default `false`). When true, the UDP
transport binds a kernel-assigned ephemeral port (`0.0.0.0:0`) instead
of the configured `bind_addr`, refuses inbound handshakes, and is
never advertised on Nostr regardless of `advertise_on_nostr`. Use
this to participate in the mesh as a pure client — initiate outbound
links without exposing an inbound listener on a known port.
Implements the long-form fix for `udp.bind_addr: "127.0.0.1:..."`
not actually working as a workaround (Linux pins the loopback source
IP, dropping outbound flows to external peers at the routing layer)
- `transports.udp.accept_connections` (default `true`). Mirrors the
Ethernet/BLE knob; setting to `false` produces a "client" posture
(initiate outbound, refuse inbound msg1 from new addresses). The
Node-level handshake gate carves out msg1 from peers already
established on this transport so rekey continues to work. Affects
every transport via the `Transport` trait
- Startup validation now rejects `transports.udp[*].bind_addr` set to a
loopback address when at least one peer has a non-loopback UDP
address. Replaces the silent "peer link won't establish" failure
mode where Linux's source-address routing check dropped outbound
flows from the loopback-bound socket. `outbound_only: true` is
exempt from the check (it overrides `bind_addr` to `0.0.0.0:0`)
#### Security
- Mesh-interface nftables baseline (Linux). Ships `/etc/fips/fips.nft`
as a documented operator conffile and `fips-firewall.service`
(disabled by default) for default-deny inbound on the `fips0` mesh
interface. Operators enable explicitly with
`systemctl enable --now fips-firewall.service`. Drop-ins in
`/etc/fips/fips.d/*.nft`. See `docs/fips-security.md`.
- Peer access control list enforcement: optional
`/etc/fips/peers.allow` and `/etc/fips/peers.deny` files
(TCP-Wrappers style) gate outbound connect, inbound msg1, and
outbound msg2 against npub, hex pubkey, host alias, or `ALL`.
Files are reloaded automatically on mtime change. New
`fipsctl acl show` query reports the effective rule set
([#50](https://github.com/jmcorgan/fips/pull/50),
[@alexxie16](https://github.com/alexxie16)).
#### LAN Gateway
- New `fips-gateway` binary that lets unmodified LAN hosts reach FIPS
mesh destinations via DNS-allocated virtual IPs and kernel nftables
NAT. Virtual-IP pool (`fd01::/112` by default) with state-machine
lifecycle and TTL-based reclamation; conntrack-backed session
tracking; proxy NDP on the LAN interface; control socket at
`/run/fips/gateway.sock` with `show_gateway` and `show_mappings`;
fipstop Gateway tab with pool gauge and mappings table; design doc
at `docs/design/fips-gateway.md`; integration test harness
- Inbound mesh port forwarding on `fips-gateway`: new
`gateway.port_forwards` config (list of `{ listen_port, proto,
target }` entries, IPv6 targets only) installs prerouting DNAT
rules so mesh peers can reach a configured host:port on the
gateway's LAN. A LAN-side masquerade is added when any forwards
are configured so replies flow back through conntrack.
- Gateway packaging: systemd service unit with `After=fips.service`,
Debian and AUR package entries, OpenWrt procd init with dnsmasq
forwarding, proxy NDP, RA route advertisements, and IPv6 forwarding
sysctls. Gateway enabled by default on OpenWrt
- `fips-gateway` DNS upstream probe now retries up to 5 times with a
1-second per-attempt timeout and a 1-second delay between attempts
(~10 second worst-case wait), instead of a single 3-second hard-fail.
Covers the cold-boot race where the daemon's TUN is up (the systemd
ExecStartPre wait gates on that) but the DNS responder is still
binding `[::1]:5354`. Without retry the gateway exited and relied on
`Restart=on-failure` for recovery (5-second blip + spurious error
log line per cycle); with retry the gateway recovers gracefully
without a unit restart
#### IPv6 Adapter
- Overhauled `.fips` DNS handling for systemd-based hosts. The
default `dns.bind_addr` is `::1` (IPv6 loopback) and the setup
script picks one of five backends in priority order: a global
drop-in at `/etc/systemd/resolved.conf.d/fips.conf`, the systemd
dns-delegate path, `resolvectl` per-link, standalone dnsmasq, or
NetworkManager's dnsmasq plugin. Teardown reverses only what was
applied. New `testing/dns-resolver/` harness exercises every
backend across Debian 12, Debian 13, Ubuntu 22.04, Ubuntu 24.04,
and Ubuntu 26.04
([#58](https://github.com/jmcorgan/fips/pull/58),
fixes [#52](https://github.com/jmcorgan/fips/issues/52),
[#77](https://github.com/jmcorgan/fips/issues/77)).
#### Operator Tooling
- `node.log_level` config field (case-insensitive, default `info`)
replaces the hardcoded `RUST_LOG=info` previously baked into
systemd units and the OpenWrt procd init script. The daemon now
loads config before initializing tracing so the configured level
takes effect; `RUST_LOG` still overrides when set
- `fipsctl show identity-cache` lists every cached node identity
(npub, IPv6 address, display name, LRU age) alongside the
configured cache capacity
- `fipsctl show peers` extended with per-peer security signals
(replay suppression count, consecutive decrypt failures), Noise
session counters, session indices, and rekey lifecycle state
- `fipsctl show sessions` extended with handshake resend count
during establishment and rekey/session health fields when
established (session start, K-bit epoch, coords warmup remaining,
drain state)
- `fipsctl show cache` now includes individual coordinate cache
entries (tree coordinates, depth, path MTU, age). The top-level
count field was renamed from `entries` to `count` for clarity
- `fipsctl show routing` expands `pending_lookups` from a count to
per-target detail (attempt, age, last sent), adds pending TUN
packet queue depth, and adds per-peer connection retry state
([#42](https://github.com/jmcorgan/fips/pull/42),
[@osh](https://github.com/osh))
- Historical node and per-peer statistics: in-memory time-series
rings on the daemon, surfaced through new control-socket queries,
`fipsctl stats` subcommands, and a `fipstop` Graphs tab with
btop-style sparklines
([#64](https://github.com/jmcorgan/fips/pull/64)).
- `fipstop` Node tab now carries a "Listening on fips0" panel
(right-half of the Traffic block) that lists local IPv6 listening
sockets reachable from the mesh interface, paired with the
`inet fips` baseline filter classification for each (proto, port).
Rows render in default White (`OPEN` — the chain has a canonical
unrestricted accept rule), DarkGray (`filt` — chain falls through
to `counter drop`), or DarkGray with a `?` State suffix (`filt?` —
the chain references the port but with matchers the panel cannot
fully decompose, e.g. saddr filters or jumps). When the
`fips-firewall.service` is not active, the panel renders a yellow
banner reminding the operator that all listeners are
mesh-exposed. Wildcard binds (`local_addr == ::`) carry a `*`
suffix in the Process column. Powered by a new
`show_listening_sockets` control query (Linux-only).
#### Packaging and Deployment
- Arch Linux AUR packaging for `fips` (release) and `fips-git`
(development) packages with sysusers.d/tmpfiles.d integration
([#21](https://github.com/jmcorgan/fips/pull/21),
[@dskvr](https://github.com/dskvr))
- `packaging/debian/fips-gateway.service` now waits up to 30 seconds
for the daemon's `fips0` TUN to appear before exec'ing the gateway
binary (`ExecStartPre` poll loop). Eliminates the cold-boot race
where `fips-gateway` exits with `fips0 interface not found` and
recovers via `Restart=on-failure`, producing a 5-second blip and a
spurious error log line per restart cycle. If `fips0` never appears
within 30 seconds, the existing error path runs as before
- `packaging/debian/build-deb.sh` now auto-derives a per-commit Debian
Version field for dev builds (Cargo.toml version ending in `-dev`)
using the form `<base>~dev+git<YYYYMMDD>.<sha>[.dirty]-1`, e.g.
`0.3.0~dev+git20260429.6def31b-1`. Each commit produces a uniquely-
comparable Version string so `apt install ./*.deb` and
`ansible.builtin.apt: deb:` no longer silently no-op when one dev
build is installed on top of another. The `~dev` marker sorts
pre-`0.3.0` so a tagged release supersedes any prior dev .deb.
Tagged release builds (no `-dev` in Cargo.toml) keep the clean
`<version>-1` form. Operator override via `--version` still wins
#### Examples
- macOS WireGuard sidecar: run FIPS in a local Docker container and
route `.fips` traffic from the macOS host through a WireGuard tunnel
to the container's `fips0` interface. Only traffic destined for
`fd00::/8` transits the sidecar; regular internet traffic continues
to use the host network
([#51](https://github.com/jmcorgan/fips/pull/51))
#### Documentation
- `docs/design/port-advertisement-and-nat-traversal.md` documents
how nodes find each other through Nostr relays and the
STUN-assisted UDP hole punch
### Changed
- Noise session ChaCha20-Poly1305 backend switched from RustCrypto's
`chacha20poly1305` to `ring 0.17`. ring wraps BoringSSL's
hand-tuned ChaCha20-Poly1305 implementation, dispatching to NEON
on aarch64 and AVX2 / AVX-512 on x86_64 — typically 3-5 GB/s/core
vs the ~600-800 MB/s/core RustCrypto soft path on the same
hardware. Wire format unchanged: ChaCha20-Poly1305 is
byte-deterministic for a given `(key, nonce, plaintext, aad)`,
so any correct AEAD produces identical ciphertext and a mixed
pre-swap / post-swap mesh interoperates without protocol
awareness. The keyed AEAD is now cached on `CipherState` instead
of being re-derived per packet (the cached Poly1305 key state is
the actual perf win); `EndToEndState` grew from ~600 B to
~1.5 KB as a consequence and is annotated
`#[allow(clippy::large_enum_variant)]` since boxing would re-add
a per-packet indirection on every encrypt/decrypt. aarch64
measurements (Apple Silicon docker, two nodes): TCP 1-stream
437 → 1097 Mbps (~2.5×); UDP at 1000 Mbit goes from
599 Mbps / 40 % loss to lossless line-rate; 3-node ping under
load 7.68 ms avg / 215 ms max → 0.72 ms / 3.6 ms max as the
relay path stops being crypto-bound
([#80](https://github.com/jmcorgan/fips/pull/80),
[@mmalmi](https://github.com/mmalmi))
- Linux UDP receive path uses `recvmmsg(2)` with a 32-packet batch
in place of single-packet `recvmsg(2)`. A single `readable()`
wakeup drains up to 32 datagrams in one syscall before yielding
back to the reactor, eliminating the per-packet scheduler-hop +
futex cost that previously capped inbound rate at one event per
scheduler quantum independent of CPU. `SO_RXQ_OVFL` is sampled
once per batch from the cmsg chain of `msgs[0]` and surfaced
through `AsyncUdpSocket::recv_batch` so the 1Hz
`sample_transport_congestion()` detector continues to feed the
per-transport `dropping` flag. macOS / Windows fall through to
the per-packet path; `recvmmsg` is Linux-specific
([#81](https://github.com/jmcorgan/fips/pull/81),
[@mmalmi](https://github.com/mmalmi))
- `Node::run_rx_loop` drains up to 256 additional ready items via
`try_recv()` after each `tokio::select!` await fires on
`packet_rx` / `tun_outbound_rx`, in a tight inner loop before
yielding. Previously the select cost a full scheduler hop +
futex per packet, capping throughput at one event per scheduler
quantum with the worker near-idle. `biased` ordering keeps
data-plane branches priority over tick / control / DNS under
sustained load; the 256 cap is empirically tuned to keep the
worker on a busy stream between yield points (≈ 400 KB of
contiguous traffic) while still bounding the inner loop so a
flood on one branch can't starve the periodic tick or control
socket. Pairs with the UDP `recvmmsg` change above
([#81](https://github.com/jmcorgan/fips/pull/81),
[@mmalmi](https://github.com/mmalmi))
- `PeerIdentity::pubkey_full()` now precomputes the parity-aware
full public key at construction in `from_pubkey`. Previously the
method fell through to a secp256k1 EC point parse (`fe_sqrt` +
`fe_mul` + `ge_set_xo_var`) on every call when the full key
wasn't passed at construction (i.e. for every peer constructed
from an npub or x-only key) — ~6% of per-packet CPU on the
bulk-data send path for a value that never changed after
construction. The same EC point parse already runs at
construction inside `NodeAddr::from_pubkey`, so the cost is paid
once where it would be paid anyway
([#81](https://github.com/jmcorgan/fips/pull/81),
[@mmalmi](https://github.com/mmalmi))
- Cargo feature flags `tui`, `ble`, `gateway`, and
`nostr-discovery` removed; subsystem inclusion is now driven by
platform `cfg` gates so plain `cargo build` compiles everything
available on the target
([#79](https://github.com/jmcorgan/fips/pull/79))
- MMP link-layer report intervals retuned for constrained transports:
steady-state floor raised from 100ms to 1000ms, ceiling from 2000ms
to 5000ms. Cold-start uses a 200ms floor for the first 5 SRTT samples
before switching to steady-state. Reduces BLE overhead ~10× while
keeping reports well above the EWMA convergence threshold.
Session-layer intervals unchanged
- 35 info-level log messages demoted to debug (handshake
cross-connection mechanics, periodic MMP telemetry, TUN/transport
shutdown, retry scheduling). Info output now focuses on
operator-relevant state changes: lifecycle events, peer promotions,
session establishment, parent switches, transport start/stop
- **Breaking (control socket JSON):** `show_cache` response field
`entries` has changed type from a `u64` count to an array of entry
objects; a new `count` field carries the previous scalar value.
`show_routing` response field `pending_lookups` has changed type
from a `u64` count to an array of per-target lookup objects.
External consumers parsing these fields as numbers must be
updated. In-tree `fipstop` is adjusted to the new schema. The
control socket interface is still pre-1.0 and not covered by
stability guarantees
- Discovery rate limiting retuned to be less aggressive at cold start.
The previous defaults (30s base post-failure suppression, doubling
to a 300s cap, with reset only on parent change / new peer / first
RTT / reconnection) reliably outlasted initial mesh convergence: a
single timed-out lookup during bloom-filter propagation suppressed
any retry for 30s while none of the reset triggers fired on a
stable post-handshake topology. The suppression window dictated
effective time-to-converge instead of bounding repeat traffic.
Replaces the single-lookup-with-internal-retry model
(`timeout_secs`/`retry_interval_secs`/`max_attempts`) with a
per-attempt timeout sequence in
`node.discovery.attempt_timeouts_secs` (default `[1, 2, 4, 8]`).
Each attempt sends a fresh `LookupRequest` with a new `request_id`,
which lets successive attempts take different forwarding paths as
the bloom and tree state evolve. The destination is declared
unreachable only after the full sequence is exhausted (15s total
at the default). Disables post-failure suppression by default
(`backoff_base_secs`/`backoff_max_secs` now both `0`); operators
with chatty apps generating repeat lookups against unreachable
destinations can opt back in
- The `docs/` tree is reorganised so readers can find content by
what they're trying to do: tutorials for new users, how-to guides
for specific tasks, reference material for configuration and
protocol details, and design discussion for architectural
background. New top-level `getting-started.md` and per-section
landing pages anchor the entry points. Content was reconciled
against current source: protocol layer details, wire-format
diagrams, configuration knobs, and CLI references were brought
back into agreement with the implementation. Gateway feature-set
documentation was rewritten end-to-end.
- Test coverage was substantially expanded for the new release
surface (discovery state machine, control-socket query handlers,
decrypt-failure thresholds, STUN parser, gateway, NAT traversal,
packaging install paths) alongside CI-side hardening for the new
Windows and macOS platforms.
- Gateway `dns.listen` source default changed from `[::]:53` to
`[::1]:5353` to match the canonical deployment model (a host
already serving DHCP/DNS to a LAN segment, where port 53 is
taken by the existing resolver and `.fips` queries are forwarded
to the gateway over loopback). The OpenWrt ipk previously
overrode this in its packaged config; the override is now
redundant and has been dropped. Operators on a host without a
pre-existing resolver on port 53 can opt back into the wildcard
bind by setting `dns.listen: "[::]:53"` explicitly. The new
default binds IPv6 loopback only — forwarders that reach the
gateway over IPv4 loopback need an explicit IPv4 listen address.
- Generic systemd install tarball brought to feature parity with
the `.deb` and AUR packages. The tarball now ships the
`fips-gateway` binary with its (operator-opt-in)
`fips-gateway.service`, a `fips-firewall.service` unit with the
`/etc/fips/fips.nft` mesh-interface nftables baseline (also
opt-in), an `/etc/fips/fips.d/` operator drop-in directory for
per-service nft rules, and the multi-backend `fips-dns-setup` /
`fips-dns-teardown` helpers. `install.sh` and `uninstall.sh`
handle the new units and conffile (preserve-on-upgrade for
`fips.nft`, like `fips.yaml`). `README.install.md` documents
the gateway, firewall, and DNS-routing services. Closes the
longest-standing parity gap for non-Debian / non-Arch systemd
Linux distros (Fedora, RHEL/CentOS, openSUSE, etc.) installing
from the release-distribution tarball.
### Fixed
- Generic systemd install tarball: `install.sh` now correctly
resolves the `fips-dns-setup` and `fips-dns-teardown` helpers
from the tarball staging directory. Previously the script
referenced them at `${SCRIPT_DIR}/../common/`, a path that
exists only in the source-repo layout, not in the extracted
tarball. Bug latent since the multi-backend DNS helpers
landed in `7260ad2`; only manifested when operators ran
`install.sh` from an extracted tarball rather than from a
source checkout.
- Adopted NAT-traversed UDP transports inherit the primary listener's
MTU and buffer config. `Node::adopt_established_traversal`
constructed the adopted UDP transport with `UdpConfig::default()`
(MTU 1280, default recv/send buffer sizes, default accept/advertise
flags) regardless of the operator's primary `[transports.udp]`
listener. Operators who set the primary MTU higher (e.g. 1500 on
a known-clean LAN path) silently dropped full-sized tunnel
datagrams over the NAT-traversed link with no log explaining why
throughput collapsed. Lookup now tries `transport_name` first (so
multiple named listeners pick up inheritance from the matching
one) and falls back to the unnamed `Single` listener; bind /
external-address fields are cleared since the adopted socket is
already bound. The 1280 default was deliberately the IPv6 minimum
(the only value guaranteed across arbitrary middlebox paths);
with this change, operators who raise the primary MTU accept the
tradeoff that NAT-traversed flows initially attempt the higher
MTU and may black-hole on tighter paths until reactive
`MtuExceeded` recovery kicks in
([#83](https://github.com/jmcorgan/fips/pull/83),
[@mmalmi](https://github.com/mmalmi))
- TreeAnnounce ancestry on self-root transitions. When a node had
no smaller-NodeAddr peer to use as a parent, the spanning-tree
state correctly promoted it to root, but the ancestry it
advertised on the next `TreeAnnounce` still referenced its
previous parent's path. Receiving peers rejected the announce
with `invalid ancestry: advertised root X is not the minimum
path entry Y`, blocking mesh transit on any path that needed to
traverse the node. The self-root transition is now detected
explicitly in `TreeState::become_root` and the advertised
ancestry rebuilt to start from self. The MMP receive handler
surfaces the same path so stale ancestry inherited across
reconnect is corrected eagerly rather than waiting for the next
observation tick
([#82](https://github.com/jmcorgan/fips/pull/82),
[@mmalmi](https://github.com/mmalmi))
- Auto-connect retry refetches the cached overlay advert
unconditionally before each retry attempt, not only when
`fetch_advert` returns zero endpoints (`NoTransportForType`).
The much more common stale-cache failure was: cache returned an
endpoint that *looked* valid (the address learned before the
peer's NAT rebound), the dial succeeded at the IP layer, the
handshake timed out, MMP fired, the next retry hit the same
cached endpoint, looped forever — no `NoTransportForType` ever
fired because the cache had data, just dead data. Refetch now
runs unconditionally before each retry attempt (one Filter query
against `advert_relays` with a 2s per-attempt timeout, bounded
by the retry backoff cadence). Keeps the retry loop pinned to
relay ground truth instead of whatever the cache happened to
learn at startup
([#82](https://github.com/jmcorgan/fips/pull/82),
[@mmalmi](https://github.com/mmalmi))
- Stale overlay-advert eviction on `NoTransportForType`. Mirrors
the existing stale-advert sweep that ran from the
`BootstrapEvent::Failed` (NAT-traversal-streak) path, but covers
the case where `initiate_peer_connection` / a retry tick returns
`NodeError::NoTransportForType` — the cache had no addresses for
the peer at all. A fire-and-forget `refetch_advert_for_stale_check`
against the peer's npub re-fetches kind `37195` from
`advert_relays`; if the relay has a newer advert it replaces the
cached entry, if it has nothing it evicts the entry. Either way
the next retry tick goes to fresh data instead of looping on the
same dead endpoint. Resolves a deployment regression where a
macOS daemon's view of a Linux peer would flap after NAT rebind
with no recovery short of a daemon restart
([#82](https://github.com/jmcorgan/fips/pull/82),
[@mmalmi](https://github.com/mmalmi))
- Schedule retry on startup peer-init failure. When
`initiate_peer_connections()` ran at boot, an address-resolution
failure (no operational transport for the configured transport
types, all addresses unreachable, NAT rebind invalidating cached
endpoints) was logged and silently forgotten — the peer entry
stayed in a dead state forever, accepting incoming pings but
unable to answer them, until the daemon was manually restarted.
Now mirrors the `BootstrapEvent::Failed` path: on a startup
peer-init error, parse the peer's npub and call `schedule_retry`
so the peer recovers without operator intervention
([#82](https://github.com/jmcorgan/fips/pull/82),
[@mmalmi](https://github.com/mmalmi))
- Default control-socket path resolution: daemon and client tools now
use a shared resolver, eliminating a divergence where `fipsctl` /
`fipstop` could connect to a socket the daemon never bound (notably
on dev runs with `XDG_RUNTIME_DIR` set, or after a prior packaged
install left a root-owned `/run/fips` behind). Canonical order is
`/run/fips` → `$XDG_RUNTIME_DIR/fips/` → `/tmp/fips-<name>`. The
`/run/fips` arm is selected by directory existence; the kernel
enforces actual access at `connect(2)` time, surfacing a clear
`EACCES` for users not yet in the `fips` group rather than silently
steering them to a path the daemon never bound. `XDG_RUNTIME_DIR` is
validated as an existing directory before being used so stale
post-logout values are treated as missing. The deployed fleet is
unaffected: packaged configs set `node.control.socket_path`
explicitly.
- UDP transport with `advertise_on_nostr: true` + `public: true` +
a wildcard `bind_addr` (e.g. `0.0.0.0:2121`) is now advertised
with its STUN-discovered public IPv4 instead of being silently
dropped from the published Kind 37195 advert. Previously the
advert builder filtered the wildcard out (since `0.0.0.0` is
not a valid endpoint), but emitted no log explaining what
happened — operators saw the daemon up, both flags set, and
no UDP endpoint in the advert. The fix runs a one-shot STUN
observation against an ephemeral socket on the daemon's
configured `stun_servers` and combines the reflexive IPv4 with
the configured listener port for the advert (`udp:<eip>:<port>`).
Successful STUN observations are cached per-transport for one
`advert_refresh_secs` cycle (default 30 min) so we don't re-STUN
every refresh. Failed observations are cached for only 60s, so
a transient STUN flake at startup retries within ~a minute and
grows the advert with UDP as soon as STUN starts working —
rather than waiting the full 30-min cycle. Per-server STUN
response timeout is 5s for the advert-publish path (vs. 2s for
the latency-sensitive per-traversal path), giving slow
first-call STUN time to complete without giving up. On STUN
failure, the wildcard-bind path still skips, but now logs a
loud `warn!` pointing at the operator-side fixes (set
`external_addr`, bind to a specific IP, or ensure `stun_servers`
reachable). Restores zero-config public-IP autodiscovery on
AWS EIP / GCP / Azure setups where binding to the public IP
directly is impossible (1:1 NAT)
- New `external_addr` field on `transports.udp.*` and
`transports.tcp.*` for explicit advertise-as override. Accepts
either a bare IP (`"198.51.100.1"` — the configured `bind_addr`
port is appended) or a full `host:port`
(`"198.51.100.1:8443"`). Takes precedence over both the bound
address and any STUN-derived autodiscovery. Required for TCP
on cloud-NAT setups (AWS EIP, GCP/Azure external IPs) where
binding to the public IP directly fails with `EADDRNOTAVAIL`
(the EIP isn't on a host interface). Optional but useful for
UDP as a deterministic alternative to STUN — operators who
want to skip STUN egress (or whose STUN is blocked) can
specify it explicitly. Without `external_addr`, TCP with a
wildcard `bind_addr` + `advertise_on_nostr: true` now logs a
loud `warn!` pointing at the two fixes instead of silently
skipping
- Nostr-discovery now tolerates ±60s of clock skew on offer/answer
freshness checks so a responder whose wall clock leads the
initiator's by less than that no longer silently rejects every
offer. Previously, a public-test daemon with un-NTP'd peers (or
long uptime — `now_ms()` anchors to `SystemTime` once at startup,
then advances monotonically; post-startup NTP step adjustments
don't propagate) would see ~100% signal-timeout rate against
skewed peers, indistinguishable from "peer is offline." New
optional `offerReceivedAt` field on the answer payload lets the
initiator log per-peer NTP-style skew estimates (DEBUG when ≥30s)
for operator visibility. Backward-compatible — older responders
that don't fill the field still produce valid answers
- Nostr-discovery NAT-traversal failure suppression: per-npub
consecutive-failure counter triggers a 30-min extended cooldown
after 5 failures, preventing the daemon from hammering Nostr
relays with offers to peers that have gone away. WARN log lines
rate-limited to one per peer per 5 min (subsequent failures
emit DEBUG with `consecutive_failures` + remaining `cooldown_secs`).
Threshold-crossing also fires a one-shot active re-check of the
peer's Kind 37195 advert against `advert_relays`; absent →
evict cache; newer → refresh + reset streak; same → cooldown
stands. New `failure_streak_threshold`, `extended_cooldown_secs`,
`warn_log_interval_secs`, `failure_state_max_entries` config
fields under `node.discovery.nostr`. Per-peer state visible in
`fipsctl show peers` JSON under `nostr_traversal`
- Tor onion adverts published over Nostr overlay discovery now
include the public-facing port (`<onion>.onion:<port>`) instead of
just the bare onion hostname. The publisher previously emitted a
bare onion that the parser refused (`expected host:port`),
producing a persistent retry-fail loop on any peer whose Tor
advert was the only entry in the discovery cache. New
`transports.tor.advertised_port` config field (default `443`,
matching the Tor `HiddenServicePort` convention) controls the
advertised port; operators with non-default virtual ports can
override.
- TCP-over-FIPS reliability on mesh paths with mixed transport
MTUs (e.g. a UDP-1280 hop in the picker set) improved. Three
interlocking changes: `Node::transport_mtu()` is now deterministic
across restarts (min across operational transports rather than
insertion-order-dependent); the TCP MSS clamp at the TUN boundary
reads per-destination path MTU instead of a single global ceiling;
and reactive `MtuExceeded` from forwarders is mirrored back into
the TUN-side `path_mtu_lookup` so later flows pick up forward-path
bottlenecks without re-discovery. Windows TUN reader receives the
same per-destination plumbing.
- Proactive end-to-end `PathMtuNotification` now mirrors into the
TUN-side `path_mtu_lookup` (TCP MSS clamp store), parallel to the
reactive `MtuExceeded` mirror that already existed. Previously the
proactive handler only updated the session-canonical
`MmpSessionState.path_mtu`; on stable long-lived paths where the
destination's echo had tightened the session MTU but no transit
router had emitted a fresh `MtuExceeded` (because all current
traffic was already sized by the tighter session value), new TCP
flows opened in that window kept getting clamped by the staler
discovery-time value. The proactive mirror closes that gap with
the same tighter-only semantics — never loosens the clamp.
- Nostr-discovered peers running an FMP-protocol version we cannot
speak no longer trigger an indefinite retraversal storm. Open-
discovery NAT-traversal succeeds at the UDP layer regardless of
protocol version, so the daemon would adopt the punched socket,
drop every incoming packet at `Unknown FMP version`, idle out
after 31s, and re-fire the full STUN-offer-answer-punch sequence
~30s later — every minute, forever, against peers the handshake
literally cannot complete with. The rx loop now detects mismatched-
version packets arriving on adopted bootstrap transports, reverse-
maps to the originating npub, and applies a long structural
cooldown to the discovery layer's `failure_state` so the next
open-discovery sweep skips the peer until either side upgrades.
One-shot WARN per fresh observation; subsequent mismatches inside
the cooldown window are silent. New `protocol_mismatch_cooldown_secs`
config field under `node.discovery.nostr` (default 86400 = 24h),
separate from the transient-failure `extended_cooldown_secs`.
- `fipstop` now uses `ratatui::try_init()` instead of `ratatui::init()`,
so terminal initialization failures (e.g. Docker on macOS Sequoia,
or environments without a usable tty) produce a clean error message
instead of a hard crash
- Spanning-tree updates that change only the internal path between
root and leaf — without changing the root or the depth — now
propagate to leaves correctly. Previously a leaf could continue
routing against a stale internal path until the parent or depth
also changed.
## [0.2.1] - 2026-05-11
### Added
- Linux release artifact workflow: builds x86_64 and aarch64 tarballs
and `.deb` packages on `v*` tag push, with SHA-256 checksums
- AUR publish workflow for tagged stable releases
### Changed
- Validate bloom filter fill ratio on FilterAnnounce ingress.
Inbound FilterAnnounce messages whose derived false-positive
rate exceeds `node.bloom.max_inbound_fpr` (new config field,
default 0.05) are rejected silently on the wire, logged at WARN,
and counted in a new `bloom.fill_exceeded` counter. A
rate-limited WARN also fires if our own outgoing filter's FPR
exceeds the cap. `BloomFilter::estimated_count` now takes
`max_fpr` and returns `Option<f64>`, returning `None` for
saturated filters; this propagates through `compute_mesh_size`
into `estimated_mesh_size` (already `Option<u64>`)
### Fixed
- Control socket path detection in fipsctl and fipstop now checks for
the `/run/fips/` directory instead of the socket file inside it, so
users not yet in the `fips` group get a clear "Permission denied"
error instead of a misleading "No such file" fallback to
`$XDG_RUNTIME_DIR` ([#30](https://github.com/jmcorgan/fips/issues/30),
reported by [@Sebastix](https://github.com/Sebastix))
- OpenWrt ipk build excluded BLE feature that requires D-Bus, which is
unavailable on OpenWrt targets
- IPv6 routing policy rule added at TUN setup to protect `fd00::/8`
from interception by Tailscale's table 52 default route
- Bloom filter routing no longer swallows traffic when no bloom
candidate is strictly closer than the current node. `find_next_hop`
now falls through to greedy tree routing in that case instead of
returning `NoRoute`, which previously caused dropped packets in
topologies where the tree parent was closer but not a bloom
candidate
- Auto-connect peers now reconnect after a graceful `Disconnect`
notification from the remote side. `handle_disconnect` previously
removed the peer without scheduling a reconnect, orphaning the
entry on a clean upstream shutdown; the other removal paths
(link-dead, decrypt failure, peer restart) already scheduled
reconnect ([#60](https://github.com/jmcorgan/fips/issues/60),
reported by [@SwapMarket](https://github.com/SwapMarket))
- `fipsctl connect` now rejects FIPS mesh (`fd00::/8`) addresses for
`udp`, `tcp`, and `ethernet` transports with a clear error message
instead of echoing success while the daemon silently failed the
bind with `EAFNOSUPPORT`
([#61](https://github.com/jmcorgan/fips/issues/61),
reported by [@SwapMarket](https://github.com/SwapMarket))
- Tighten TreeAnnounce ancestry validation to match the spanning
tree specification. The receive path now verifies that the
ancestry is structurally consistent with the signed parent
declaration before mutating tree state.
- Make the tree ancestry acceptance unit test deterministic.
`test_tree_announce_validate_semantics_accepts_valid_non_root`
generated a random signing identity while pinning the fixed root
to `node_addr[0] = 0x01`; about 2 in 256 random identities were
numerically smaller than the claimed root, triggering
`AncestryRootNotMinimum`. The test now regenerates the identity
until its `node_addr` is strictly larger than both the fixed
parent and root.
## [0.2.0] - 2026-03-22
### Added
#### Operator Tooling
- `fipsctl connect` and `disconnect` commands for runtime peer
management via control socket, with hostname resolution from
`/etc/fips/hosts`
#### IPv6 Adapter
- Pre-seed identity cache from configured peer npubs at startup, so TUN packets can be dispatched immediately without waiting for handshake completion ([@v0l](https://github.com/v0l))
#### Mesh Peer Transports
- New Tor transport with SOCKS5 and directory-mode onion service for anonymous inbound and outbound peering
- DNS hostname support in peer addresses for UDP and TCP transports
- Non-blocking transport connect for connection-oriented transports (TCP, Tor)
#### Packaging and Deployment
- Reproducible build infrastructure: Rust toolchain pinning via
`rust-toolchain.toml`, `SOURCE_DATE_EPOCH` in CI and packaging
scripts, deterministic archive timestamps
- Top-level packaging Makefile for unified build across formats
- Kubernetes sidecar deployment example with Nostr relay demo
- Nostr release publishing in OpenWrt package workflow
- SHA-256 hash output in CI build and OpenWrt workflows
#### Testing and CI
- Maelstrom chaos scenario with dynamic topology mutation and
ephemeral node identities via connect/disconnect commands
- Consolidated Docker test harness infrastructure
### Changed
- Discovery protocol: replace flooding with bloom-filter-guided tree
routing. Includes originator retry (T=0/T=5s/T=10s), exponential
backoff after timeouts and bloom misses, and transit-side per-target
rate limiting. Removed 257-byte visited bloom filter from LookupRequest wire format. *This is a breaking change; nodes running versions prior to this release will not be compatible.*
### Fixed
- DNS responder returned NXDOMAIN for A queries on valid `.fips` names,
causing resolvers to give up without trying AAAA. Now returns NOERROR
with empty answers for non-AAAA queries on resolvable names.
(#9, reported by [@alopatindev](https://github.com/alopatindev))
- Stale end-to-end session left in session table after peer removal blocked session re-establishment on reconnect — `remove_active_peer` now cleans up `self.sessions` and `self.pending_tun_packets`. (#5, [@v0l](https://github.com/v0l))
- `schedule_reconnect` reset exponential backoff to zero on each link-dead
cycle instead of preserving accumulated retry count.
(#5, [@v0l](https://github.com/v0l))
- FMP/FSP rekey dual-initiation race on high-latency links (Tor): both
sides' timers fired simultaneously, both msg1s crossed in flight, each
side's responder path destroyed the initiator state. Fixed with
deterministic tie-breaker (smaller NodeAddr wins as initiator).
- Parent selection SRTT gate bypass: `evaluate_parent` used default cost
1.0 for peers filtered out by `has_srtt()`, defeating the MMP eligibility
gate. Now skips unmeasured candidates when any peer has cost data.
- FSP rekey cutover race: initiator cut over before responder received msg3,
causing AEAD failures. Fixed by deferring initiator cutover by 2 seconds.
- MMP metric discontinuity after rekey: receiver state carried stale
counters across rekey, inflating reorder counts and jitter. Fixed via
`reset_for_rekey()`.
- Auto-connect peers exhausted `max_retries` on initial connection failures
and were permanently abandoned. Now retry indefinitely with exponential
backoff capped at 300 seconds.
- Control socket permissions: non-root users couldn't connect. Daemon now
chowns socket and directory to `root:fips` group at bind time.
- Post-rekey jitter spikes: old-session frames arriving via the drain window
produced 2,000–7,000ms jitter spikes that corrupted the EWMA estimator.
Added a 15-second grace period after rekey cutover that suppresses jitter
updates until drain-window frames have flushed. (#10)
- ICMPv6 Packet Too Big source was set to the local FIPS address, which
Linux ignores (loopback PTB check). Now uses the original packet's
destination so the kernel honors the PMTU update.
(#16, [@v0l](https://github.com/v0l))
- Reverse delivery ratio used lifetime cumulative counters instead of
per-interval deltas, making ETX unresponsive to recent loss. (#14)
- MMP delta guards used `prev_rr > 0` to detect first report, conflating
it with a legitimate zero counter. Replaced with `has_prev_rr`. (#14)
## [0.1.0] - 2026-03-12
### Added (Initial Release)
#### Session Layer (FSP)
- End-to-end encrypted datagram service between mesh nodes addressed by Nostr npub
- Noise XK sessions with mutual authentication, replay protection, and forward secrecy
- Automatic session rekeying with configurable time/message thresholds and drain window for in-flight packets
- Port multiplexing for multiple services over a single session
- Session-layer metrics: sender/receiver reports with RTT, jitter, delivery ratio, and burst loss tracking
- Passive RTT measurement via spin bit
#### IPv6 Adapter
- IPv6 adapter interface allowing tunneling TCP/IPv6 through FIPS mesh
for traditional IP applications (TUN interface)
- DNS resolver allowing IP applications to reach nodes by npub.fips name
- Host-to-npub static mappings: resolve `hostname.fips` via host map
populated from peer config aliases and `/etc/fips/hosts` file
#### Mesh Layer (FMP)
- Self-organized core mesh routing protocol with adaptive least cost forwarding
- Noise IK hop-by-hop link encryption with mutual authentication and replay protection between peer nodes
- Distributed spanning tree construction with cost-based parent selection and adaptive reconfiguration
- Destination route discovery via bloom filter-based directed search protocol
- Path MTU discovery with per-link MTU tracking and MtuExceeded error signaling
- Link-layer MMP: SRTT, jitter, one-way delay trends, packet loss, and ETX metrics
- Link-layer heartbeat with configurable liveness timeout for dead peer detection
- Epoch-based peer restart detection
- Automatic link rekeying with K-bit epoch coordination and drain window
- Static peer auto-reconnect with exponential backoff
- Multi-address peers with transport priority-based failover
- Msg1 rate limiting for handshake DoS protection
#### Mesh Peer Transports
- UDP overlay transport with inbound and static outbound peer configuration
- TCP overlay transport with listening port and static outbound peer support
- Ethernet/WiFi transport (MAC address based, no IP stack) with optional automatic peer discovery and auto-connect
#### Operator Tooling
- Ephemeral or persistent node identity with key file management
- Unix domain control socket for runtime observability
- `fipsctl` CLI tool for control socket interaction and node management
- Comprehensive node and transport statistics via control socket
- `fipstop` TUI monitoring tool with real-time session, peer, and transport configuration and metrics display
#### Packaging and Deployment
- Debian/Ubuntu `.deb` packaging via cargo-deb
- Systemd service packaging with tarball installer
- OpenWRT package with opkg feed and init script
- Docker sidecar deployment for containerized services
- Build version metadata: git commit hash, dirty flag, and target triple
embedded in all binaries via `--version`
#### Testing and CI
- Comprehensive unit and integration tests covering all protocol layers and transports
- Docker test harness with static and stochastic topologies
- Chaos testing with simulated severe network conditions: latency, packet loss, reordering, and peer churn
- CI with GitHub Actions: x86_64 and aarch64, integration test matrix, nextest JUnit reporting
- Local CI runner script (`testing/ci-local.sh`)
#### Project
- Design documentation suite covering all protocol layers
- CHANGELOG.md following Keep a Changelog format
- Repository mirrored to [ngit](https://gitworkshop.dev/npub1y0gja7r4re0wyelmvdqa03qmjs62rwvcd8szzt4nf4t2hd43969qj000ly/relay.ngit.dev/fips)
<!-- markdownlint-configure-file { "MD024": { "siblings_only": true } } -->