mirror of
https://github.com/jmcorgan/fips.git
synced 2026-10-05 19:18:25 +00:00
Conflicts resolved: - CHANGELOG.md: kept every [Unreleased] entry from master and added the twelve new maint entries under matching headings (Deprecated/Windows ahead of Removed; Fixed/OpenWrt, Fixed/Sessions and rekey, Fixed/Windows; a new Security/Sessions and rekey). maint's Fixed/Links and transports block is dropped: its two entries are already on master under Data plane and transports, in master's wording. - .github/workflows/ci.yml: both sides kept. The Linux unit-test job runs the flaky-test report right after nextest and then the privileged interface-binding step; the macOS job creates its address-less interface and then clears the cached nextest report; the Windows job takes maint's report cleanup. - testing/ci-local.sh: both the medium-change and mdns suites are listed, defined, dispatched and run. Adapted beyond the conflicts: the rekey resend test helper in src/proto/fmp/tests/core.rs no longer sets is_healthy, which master removed from the establish snapshot.
5044 lines
304 KiB
Markdown
5044 lines
304 KiB
Markdown
# Changelog
|
||
|
||
All notable changes to this project will be documented in this file.
|
||
|
||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||
|
||
## [Unreleased]
|
||
|
||
### Added
|
||
|
||
- Dynamic interface binding for the Ethernet transport. An interface-bound
|
||
transport is now a long-lived object that is *sometimes bound*: the interface
|
||
it names need not exist when the daemon starts, may appear minutes later, and
|
||
may vanish and return mid-operation. `start_async` returns `Ok` with the
|
||
transport **absent** rather than failing, and a per-transport binder task
|
||
binds when the interface appears, unbinds when it goes away, and rebinds when
|
||
it returns. Start-time absence and runtime detach are one code path.
|
||
Detection is event-driven where the kernel offers a source — netlink
|
||
`RTNLGRP_LINK` on Linux, `PF_ROUTE` on macOS and FreeBSD — with a 1 s
|
||
`getifaddrs` poll underneath as a backstop. Presence means
|
||
`IFF_UP` — the interface exists and is administratively up — and
|
||
deliberately not `IFF_RUNNING`. Binding needs no carrier and a socket
|
||
outlives a carrier flap, so a bridge with nothing plugged into it (`br-lan`
|
||
on a wifi-only router) is bound and healthy rather than permanently
|
||
`Degraded`, and starts carrying traffic the moment a port comes up. Whether
|
||
an interface has carrier is reported separately as `interface.carrier` in
|
||
`show_transports`, never acted on.
|
||
This closes the OpenWrt boot race (procd starts `fips` before wifi has
|
||
created `fips-mesh0` / `fips-ap0`; both transports were skipped for the life
|
||
of the process while the 802.11s peer link formed anyway, so the node looked
|
||
healthy and reached nothing), the intermittent-adapter case, and the
|
||
mid-operation `wifi reload` that destroyed and recreated an interface under a
|
||
live socket.
|
||
|
||
- `transports.ethernet.*.optional` (bool, default `false`). Naming an interface
|
||
in configuration is a statement that you expect it, so the default is to
|
||
complain: while a required interface is missing the node reports `Degraded`
|
||
and logs the edge, at a severity that follows how long the absence lasts
|
||
(see below). `optional: true` makes absence silent (`info` on the edge, no
|
||
health impact) for hardware that is legitimately not always there. It describes the interface's *presence*, not the transport's
|
||
importance — an optional interface that is present is used exactly as hard as
|
||
any other — and no value of it makes a missing interface fatal at startup.
|
||
|
||
- `fipsctl show transports` reports interface presence per transport under a new
|
||
`interface` block: `name`, `presence` (`absent` / `binding` / `present`),
|
||
`carrier`, `policy` (`required` / `optional`), `since_secs`, `binds` and
|
||
`failed_attempts`. The original boot-race bug was expensive precisely because
|
||
nothing an operator could see said the node was deaf.
|
||
|
||
- `fipstop`'s transports view carries the same interface presence. The State
|
||
column shows an interface-bound transport's presence rather than its
|
||
lifecycle state — `up` is true from the moment the transport starts and stays
|
||
true while its interface is missing, which is precisely the wrong answer in
|
||
the one case someone is scanning that column for — and a new Policy column
|
||
reads `required` or `optional` beside it, with an absent required interface
|
||
red and an absent optional one yellow: the same split the daemon makes
|
||
between staying `Full` and reporting `Degraded`. The instance name and the
|
||
thing a transport is bound to are now separate columns, so netdev names line
|
||
up down the list instead of trailing ragged inside a packed label. The detail
|
||
pane gains an Interface block: netdev, presence and how long it has been
|
||
held, carrier, what the absence policy means rather than which key sets it,
|
||
bind count (flagged once it has rebound) and failed binds when there are any.
|
||
The table fits an 80-column terminal — the OpenWrt serial console and the
|
||
xterm and tmux default — dropping the byte counters below 100 columns and
|
||
stacking the detail pane below 110, rather than shrinking every column until
|
||
none of them can be read.
|
||
|
||
- `testing/iface-binding/` integration suite (`ci-local.sh --only
|
||
iface-binding`, and a GitHub matrix leg): two daemons whose only transports
|
||
are interface-bound, run against a veth pair the harness creates, downs,
|
||
deletes and recreates underneath them. Asserts the boot race, the late
|
||
attach and peering over it, the flap in both directions,
|
||
destroy-and-recreate, that an `optional` interface never moves node health,
|
||
and that absence is logged once on the edge rather than once per retry.
|
||
|
||
#### Node lifecycle
|
||
|
||
- Transport-medium change detection, controlled by the new `node.netmon.*`
|
||
block (on by default). For each peer whose transport address is a numeric IP
|
||
endpoint, the node asks the kernel which local address it would use to reach
|
||
*that peer* — a `connect(2)` on a UDP socket, which resolves the route and
|
||
sends nothing — and reports a change once some peer held across two
|
||
consecutive samples is reached from a different local address, or has stopped
|
||
being reachable at all. Asking the question per peer rather than about the
|
||
host is what keeps it quiet: a container bridge, a VPN, a `veth` pair or a
|
||
tunnel appearing is not the route to any peer and cannot move the
|
||
fingerprint, while a peer on the same LAN — reached by its subnet route, not
|
||
the default route — is covered, as is a more specific route moving under a
|
||
single peer. Peers joining and leaving are ignored on their own, being
|
||
ordinary node behaviour rather than a statement about the medium — except
|
||
that a peer seen for the first time is checked against its own
|
||
`connect()`-ed socket, and reported if that socket is pinned to a source the
|
||
routing table would no longer choose, so a medium change in the window
|
||
between a peer authenticating and the next sample is not adopted silently
|
||
while that peer sits stranded on the old path. A peer
|
||
addressed by MAC, by `.onion` or Nym recipient, by a scoped IPv6 literal, or
|
||
by a hostname it has not yet been heard from on, has no route to ask about
|
||
and contributes nothing; a node with no peers detects nothing, having nothing
|
||
bound to the old path to repair. The peer table is read through the node's
|
||
existing lock-free entity snapshot, so the detector stays a detached task
|
||
holding no node state. A handover is not atomic (the route goes, briefly
|
||
there is none, the new one arrives), so a short debounce coalesces the burst
|
||
into one event and a fingerprint that settles back where it started reports
|
||
nothing. Linux subscribes to `NETLINK_ROUTE`
|
||
multicast (the groups `ip monitor` uses) and macOS and FreeBSD to a
|
||
`PF_ROUTE` socket, both reacting to the kernel event in milliseconds; every
|
||
other platform samples on a timer at `node.netmon.poll_interval_secs`, which
|
||
also runs underneath the kernel sources as a backstop, since a netlink socket
|
||
drops messages under memory pressure and the subscription can be refused in a
|
||
restricted sandbox. A backend only decides *when to look* — the fingerprint
|
||
comparison, the debounce and the settled-back suppression are shared — so the
|
||
remaining backends (`NotifyIpInterfaceChange` on Windows, an embedder push on
|
||
iOS) land behind the same seam without touching the reaction. Android takes
|
||
the netlink source, and falls back to the timer where policy refuses the
|
||
group bind. What the node does with the signal is the connected-socket
|
||
rebind described under Fixed above.
|
||
Bluetooth is not covered: an adapter's state is not an IP attachment and is
|
||
invisible to this detector.
|
||
**Upgrade note: this couples a new key to one that has already shipped.** A
|
||
handover is ridden out for up to eight settling rounds of
|
||
`node.netmon.debounce_ms` before a change is reported, and if that worst case
|
||
reaches `node.link_dead_timeout_secs` the reaper tears the peering down
|
||
before the change is ever acted on, so the node refuses to start rather than
|
||
run in that shape. At the shipped defaults the margin is wide (8 × 250ms = 2s
|
||
against 30s), but detection is on by default, so **a node that shortened
|
||
`node.link_dead_timeout_secs` to 1 or 2 seconds for fast failover will be
|
||
refused at startup after the upgrade**, naming a `node.netmon.*` key its
|
||
operator never set. Raise the timeout, lower `debounce_ms` so eight rounds
|
||
stay under it, or set `node.netmon.enabled: false`. A
|
||
`node.link_dead_timeout_secs` of 0 is exempt from the check.
|
||
|
||
#### Library surface and internals
|
||
|
||
- `Node::netmon_trigger()` and `NetmonTrigger::poke()`, an embedder push into
|
||
the medium-change detector
|
||
([#146](https://github.com/jmcorgan/fips/issues/146)). The detector's
|
||
channel was private end to end, so an embedder whose platform announces a
|
||
network change had nowhere to put it. That is Android's position: the
|
||
netlink group bind is refused to an app there, which leaves detection on the
|
||
poll timer, while the app's `ConnectivityManager` callback knows the moment
|
||
the default network moved. `poke()` is a bare wake-up with no payload — the
|
||
detector still samples, and the fingerprint comparison, the debounce, the
|
||
settled-back suppression and the minimum spacing between reports apply to a
|
||
pushed wake-up exactly as they do to a kernel event, so a spurious callback
|
||
costs two samples at the default debounce rather than a socket rebind. It
|
||
is synchronous and callable from any thread, before or after `start()`; a
|
||
poke that lands while the detector is sampling is held rather than lost,
|
||
and a burst in that window coalesces into the one held wake-up (a burst
|
||
that finds the detector waiting costs two: the one that woke it and the one
|
||
held). A platform callback is not ordered against the routing table it
|
||
reports on, so a pushed wake-up whose sample shows nothing moved looks once
|
||
more after `node.netmon.debounce_ms` instead of leaving the change to the
|
||
timer; wake-ups from the kernel and the timer are unchanged. The push is
|
||
selected beside the platform's own source and the timer backstop, never
|
||
instead of them. The platform wiring (`registerNetworkCallback`, or
|
||
`NWPathMonitor` on iOS) stays with the embedder, since the crate has no JNI
|
||
layer.
|
||
|
||
- `TransportError::InterfaceUnavailable { interface }`. A missing interface and
|
||
a typo'd interface name were previously the same flat
|
||
`StartFailed(String)`; nothing downstream could branch on absence.
|
||
|
||
#### Packaging
|
||
|
||
- An RPM package for Fedora and RHEL (`packaging/rpm/`,
|
||
`make -C packaging rpm`), built from the same binaries the `.deb` and the
|
||
systemd tarball carry and attached to each release beside them. The package
|
||
is named `fips-mesh`, not `fips`: Fedora's namespace already has a `fips` —
|
||
an unrelated OpenGL FITS image viewer at 3.4.0 — which owns `/usr/bin/fips`,
|
||
so a `fips` at 0.6.0 is an older release of that one to every RPM tool, and
|
||
an ordinary `dnf upgrade` replaces a running mesh node with an image viewer.
|
||
The spec declares `Conflicts: fips`, since both ship `/usr/bin/fips`.
|
||
`make rpm` compiles nothing on the host: it builds in the pinned image by way
|
||
of `build-deb-container.sh`, which has already run the glibc floor and
|
||
Depends checks, and packages what that produced; `rpmbuild` itself runs in
|
||
`FIPS_RPM_BUILD_IMAGE` (AlmaLinux 9, pinned by digest), which supplies the
|
||
`systemd-rpm-macros` a build host may lack and writes packages every newer
|
||
rpm can read. `make rpm-host` remains for iteration, as `deb-host` does.
|
||
`testing/check-rpm-floor.sh` reads the glibc requirement rpm derived out of
|
||
the finished package — the table `dnf` enforces at install time — and fails a
|
||
build above the declared floor. Erase keeps `/etc/fips`: rpm has no purge, so
|
||
the code that removes a node's identity keys on a dpkg purge would run on an
|
||
ordinary erase, including the one a distribution upgrade performs.
|
||
|
||
- A pfSense package (`packaging/pfsense/`, `gmake pfsense`). pfSense is
|
||
FreeBSD underneath, but the FreeBSD package fails there in three
|
||
silent ways: pfSense runs only `/usr/local/etc/rc.d/*.sh` at boot and
|
||
re-runs them on WAN IP changes, so the suffixless rc script never
|
||
starts; `unbound.conf` is generated from `config.xml` with no `conf.d`,
|
||
so the DNS drop-in is never read; and — on a firewall where the
|
||
default-on "Allow IPv6" has been turned off — the responder's `::1`
|
||
default is unreachable when unbound is then generated with
|
||
`do-ip6: no`, so it binds `127.0.0.1` for robustness. The package
|
||
ships `fips.sh`, wires the `fips.` zone into the DNS Resolver through
|
||
`config.xml`, and binds the responder on `127.0.0.1`. It links
|
||
statically by default, since pfSense runs a FreeBSD base that cannot
|
||
be obtained to link against; aarch64 is refused, where static binaries
|
||
fault at `posix_spawn`. Mechanics shared with the FreeBSD builder live
|
||
in `packaging/common/pkg-lib.sh`, which both source; the FreeBSD
|
||
package is byte-identical before and after. The pfSense package is
|
||
built and checked in its own CI job, then installed in the same VM and
|
||
run through the boot script's start, re-entrant start, a forced
|
||
`newsyslog` rotation, restart, stop and `pkg delete`
|
||
(`testing/pfsense-install-smoke.sh`, also the first thing to run on a
|
||
real box), and attached to each release next to the FreeBSD package,
|
||
after being run on pfSense Plus 26.03.1 and 26.07 with mesh traffic
|
||
between them. CI produces both Intel packages from that one FreeBSD 15.1 build:
|
||
the CE 2.8.1 (`FreeBSD:15:amd64`) package, and the CE 2.9 / Plus 26.x
|
||
(`FreeBSD:16:amd64`) package as the same static binaries relabelled,
|
||
which is the direction FreeBSD's binary compatibility supports and has
|
||
been run on pfSense Plus 26.03.1 and 26.07. ARM stays
|
||
build-it-yourself because rustup ships no toolchain for it. See
|
||
`packaging/pfsense/README.md`.
|
||
|
||
### Changed
|
||
|
||
- Transport address clones share immutable bytes instead of allocating a copy.
|
||
Socket addresses are formatted on the stack so construction still needs only
|
||
one allocation, including scoped IPv6 addresses.
|
||
|
||
- `Degraded` is now a level rather than a latch. The supervisor's reason set
|
||
was monotonic, which was correct while no child could recover; with recovery
|
||
it would have meant "something broke at some point since boot" rather than
|
||
"something is broken now". Interface absence is tracked in its own reversible
|
||
set and node health is recomputed on every transition **in both directions**,
|
||
so plugging the WAN back in clears `Degraded` without a restart. A transport
|
||
whose interface is absent still counts as up, so a single-interface node that
|
||
boots before its wifi degrades rather than exiting on "no transports".
|
||
|
||
- The OpenWrt package ships the `mesh0`/`mesh1` and `ap0`/`ap1` Ethernet
|
||
transports **enabled** with `optional: true`, instead of commented out.
|
||
`fips-mesh-setup` and `fips-ap-setup` no longer comment-toggle blocks in
|
||
`fips.yaml`, and no longer tell the operator to restart the daemon after
|
||
creating an interface — the daemon binds it on its own. `phy0-sta0` (`wwan`)
|
||
is marked `optional: true` for the same reason: it only exists while a radio
|
||
is in station mode.
|
||
**Upgrade note: an existing `/etc/fips/fips.yaml` is preserved and does not
|
||
gain the new key.** It is a package conffile, so on a router where
|
||
`fips-mesh-setup` or `fips-ap-setup` had already uncommented a block, that
|
||
block stays as it was, with no `optional` key — and `optional` defaults to
|
||
false. Such a block is therefore `required`, so an absent `fips-mesh0` keeps
|
||
the node `Degraded` and is reported once at `error` ten seconds in, where the
|
||
same block in the shipped file is silent. Add `optional: true` to the block
|
||
to match what the package now ships.
|
||
|
||
- The Ethernet receive loop backs off and exits on a dead socket instead of
|
||
spinning on `Err` with a `warn!` per iteration, and the ad-hoc ENXIO
|
||
socket-reopen in the beacon sender is gone. Both hand recovery to the
|
||
presence machine: one mechanism for every cause rather than one hack per
|
||
symptom. Beacons pause while an interface is absent.
|
||
|
||
- The absence edge is not itself an error, and there is exactly one deadline
|
||
after it. An interface missing when the daemon starts logs at `info` — that
|
||
is the boot race the mechanism exists to absorb, not a fault — and a runtime
|
||
detach at `warn`, because a link coming and going is ordinary weather for a
|
||
mesh daemon. Ten seconds is the whole grace: past it, absence is no longer a
|
||
race against a radio or a container, so a **required** interface still
|
||
missing is reported once at `error`. Start-time absence and a runtime detach
|
||
share that one deadline rather than getting one each. An `optional`
|
||
interface never reaches `error`. Node health does not wait for any of it,
|
||
publishing `Degraded` on the first edge either way.
|
||
|
||
- The TUN boundary's TCP MSS clamp now tracks the node's egress MTU at
|
||
runtime instead of freezing it at startup. `transport_mtu()` is the minimum
|
||
across *bound* transports, so a transport that binds minutes after start can
|
||
be the narrow one — but the TUN reader and writer were handed a `u16`
|
||
computed once when they spawned, while every other consumer
|
||
(`show_status`, the control-socket snapshot, the session-layer fragmentation
|
||
check) read it live. A node could therefore report one effective IPv6 MTU
|
||
and clamp to another. The ceiling is now shared with those threads and
|
||
recomputed whenever the bound set changes, in both directions: a narrow
|
||
interface appearing tightens it, and its departure releases it. MSS is
|
||
negotiated per connection, so a change applies to connections opened after
|
||
it; existing ones are not disturbed.
|
||
|
||
- Rebinds that keep succeeding into a socket that dies moments later are
|
||
damped: consecutive bindings shorter than ten seconds back off on the
|
||
1 s → 30 s curve, and past three of them the binder stops announcing each
|
||
bind as a recovery until one lasts. Undamped, a persistently broken socket
|
||
behind a healthy interface produced a log pair and a `Degraded`→`Running`
|
||
health flap every second.
|
||
|
||
- A bind failure that is **not** absence — no `CAP_NET_RAW`, no readable
|
||
`/dev/bpf*`, a buffer the kernel refused — fails the daemon's start as it
|
||
always has, rather than being waited out. It is a fault, not a state, and
|
||
will not resolve on its own; only a missing interface is retried at start.
|
||
A non-absence failure during a later rebind still backs off, since the node
|
||
is serving by then.
|
||
|
||
- The binder cannot outlive its transport, and a teardown that races a bind
|
||
cannot leave a live receive loop on a socket nothing owns. A shared stop flag
|
||
is raised before teardown and checked by the binder after it stores a
|
||
binding, so whichever order the two interleave exactly one of them cleans up;
|
||
`EthernetTransport` gained a `Drop` that raises the flag, aborts the binder
|
||
and releases the socket, for handles dropped without `stop_async`.
|
||
|
||
- Presence edges are published with `try_send` and retried on the next tick
|
||
rather than awaited. A bounded channel could previously park the binder
|
||
mid-publish — a health channel able to deadlock the machine whose health it
|
||
carries — freezing the interface in whatever state it held.
|
||
|
||
- `TransportHandle::is_bound()` joins `is_operational()`: the latter means the
|
||
transport was *started*, which for an interface-bound transport no longer
|
||
implies a live socket. `Node::transport_mtu` now filters on the former,
|
||
because an interface that has never existed was clamping the whole node's
|
||
IPv6 MTU to a number derived from absent hardware.
|
||
|
||
- An interface deleted and recreated under the same name is detected as a
|
||
detach. Both backends bind by device rather than by name, so the old socket
|
||
is attached to nothing while the name still resolves — and a stale
|
||
`AF_PACKET` socket never becomes readable, so nothing errors and nothing
|
||
exits. Detection previously rested entirely on the beacon sender failing,
|
||
which a node with `announce: false` does not have. The bound interface index
|
||
is now captured at bind and compared on every poll.
|
||
|
||
- The link-event watcher distinguishes a genuine receive error from
|
||
`WouldBlock`. `try_io` clears readiness only on the latter, so a persistent
|
||
error — `ENOBUFS` after a burst of link events overflows the socket buffer —
|
||
span a core flat with nothing logged. Errors are now counted, logged once,
|
||
backed off, and after five the source is abandoned for the presence poll.
|
||
|
||
- Presence probes are coalesced to at most ten a second. Linux netlink is
|
||
filtered to `RTNLGRP_LINK`, but `PF_ROUTE` has no group filter, so the macOS
|
||
source delivers every routing message on the host — route churn, ARP, DHCP
|
||
renewals, a VPN going up and down — and each would otherwise drive a full
|
||
`getifaddrs` walk.
|
||
|
||
- CI runs the library tests on musl (Alpine) as well as glibc. Presence is
|
||
built on `getifaddrs` and `ifa_flags`, musl reimplements both independently,
|
||
and the interfaces this feature exists for (`fips-mesh0`, `fips-ap0` on
|
||
OpenWrt) are unbridged with no IP address at all — the case where
|
||
implementations most plausibly differ. It was previously asserted on a libc
|
||
no test had ever run it against, on the target it was written for.
|
||
|
||
- Interface presence state ignores lock poisoning. Treating a poisoned lock as
|
||
a failure meant reading "no socket, tasks dead", which is the destructive
|
||
direction: a transport reporting itself present while every send fails, or a
|
||
binder tearing down and rebinding every second while teardown silently
|
||
declined to abort anything.
|
||
|
||
#### Packaging
|
||
|
||
- The glibc floor is 2.34, one step below the 2.35 Ubuntu 22.04 sets. Both
|
||
families ship the same binaries, so the project-wide floor is the lowest
|
||
supported member of either, and that is RHEL 9 and its rebuilds.
|
||
`packaging/build-floor.env` now records the RPM family alongside the Debian
|
||
one, and records RHEL 8 as a deliberate exclusion: its glibc is 2.28, and
|
||
reaching it means a second build image and a second floor. `FIPS_BUILD_IMAGE`
|
||
is still the oldest Debian-family distribution, which is no longer the oldest
|
||
distribution outright.
|
||
- A release-candidate `.deb` is versioned `X.Y.Z~rcN` instead of
|
||
`X.Y.Z-rcN`. dpkg read the old form as a revision of the release and sorted
|
||
it above `X.Y.Z`, so a host that installed a candidate was not upgraded by
|
||
the release. The tarball, artifact and `.deb` file names keep the tag's
|
||
`-rcN`.
|
||
|
||
#### Windows
|
||
|
||
- `install-service.ps1` stops when `\etc\fips\fips.key` exists on the system
|
||
drive and `C:\ProgramData\fips\fips.key` does not. A service that an
|
||
earlier release ran from `\etc\fips` reads only `C:\ProgramData\fips`
|
||
after the installer runs, and came up with a new identity with no warning.
|
||
Move the key and the settings it needs into `C:\ProgramData\fips`, or
|
||
delete it if you did not put it there, then run the installer again.
|
||
- The daemon warns when `fips.yaml` or `fips.key` in `\etc\fips` is present
|
||
but not used by the run, since a node that ran from them before an upgrade
|
||
now runs on another config or identity.
|
||
|
||
### Deprecated
|
||
|
||
#### Sessions and rekey
|
||
|
||
- Setting `node.rekey.enabled: false`. The disabled path is less tested and is
|
||
not supported, and the option is removed in v2. It still works as before;
|
||
the configuration reference and the mesh-layer design now say so.
|
||
|
||
#### Windows
|
||
|
||
- The config search's probe of `\etc\fips\fips.yaml` on the current drive.
|
||
The search still reads that file first and merges it under
|
||
`C:\ProgramData\fips\fips.yaml`, but any local user can create it, so the
|
||
daemon now warns when it loads a config from there, and from v0.6.0 the
|
||
search no longer looks there. Move the settings you need into
|
||
`C:\ProgramData\fips\fips.yaml`.
|
||
|
||
### Removed
|
||
|
||
- **Source-breaking for consumers of the library crate**: `ActivePeer` no
|
||
longer stores a connectivity state. `ActivePeer::connectivity`, `can_send`,
|
||
`is_healthy`, `mark_stale`, `mark_reconnecting`, `mark_disconnected` and
|
||
`mark_connected` are gone. `ConnectivityState` stays public with only
|
||
`Connected` and `Stale`, the values `show_peers` reports, and loses
|
||
`can_send` and `is_healthy`. `ConnectivityState::is_terminal`,
|
||
`ActivePeer::is_disconnected`, `Node::sendable_peers` and
|
||
`Node::sendable_peer_count` keep their signatures and the results they
|
||
always had in the daemon: the first two return `false`, and the last two
|
||
cover every peer. Nothing in the daemon changed the stored state after a
|
||
peer was promoted, so every removed check was already true and the shipped
|
||
binaries behave as before. The peer wire and the control-socket response
|
||
shape are unchanged.
|
||
|
||
### Fixed
|
||
|
||
#### Node lifecycle
|
||
|
||
- Losing an interface no longer leaves its peers in the routing table. The
|
||
peers stayed in the registry, the routes through them stayed selectable, and
|
||
the node kept advertising reachability it no longer had — so transit traffic
|
||
was dropped in silence and other nodes kept routing toward this one for those
|
||
destinations, until the liveness reaper noticed up to
|
||
`node.link_dead_timeout_secs` later. Measured on real hardware, a detached
|
||
dongle took the node's parent with it and no new parent was chosen for
|
||
twenty-seven seconds, with four alternative peers available the whole time. A
|
||
transport's detach edge now withdraws every peer whose active link runs over
|
||
it, on the same path the liveness reaper uses, so sessions, path MTU, session
|
||
indices, the link, the control machine, tree cleanup and re-announce, and
|
||
bloom withdrawal unwind exactly as they already did. It is not filtered by
|
||
`optional`: whether an interface's absence is normal is a statement about
|
||
node health, and says nothing about whether the routes over it still work.
|
||
The trade is that an absence shorter than the dead timeout that then recovers
|
||
now costs a re-peer where it previously cost nothing, accepted because
|
||
black-holing is silent, poisons other nodes' routing and takes the full
|
||
timeout to clear, where a re-peer is bounded, visible and self-healing.
|
||
|
||
- A local interface flap during a handshake is no longer charged to the remote.
|
||
A msg2 send refused because the interface is absent or mid-rebind was treated
|
||
as a failed handshake: the link was removed, the reverse-address entry
|
||
dropped, the session index freed, the control machine torn down, and the
|
||
whole thing recorded under the reject reason that means "the remote sent
|
||
something invalid", which is what an operator reading the rejects would have
|
||
concluded. The initiator meanwhile resent msg1 into a link that no longer
|
||
existed and had to rebuild from nothing. A transport error the daemon is
|
||
already working to clear now leaves the half-built link exactly where it is
|
||
for that resend to land on; only a terminal error still tears down, and a
|
||
link nobody resends to is reaped at `node.rate_limit.handshake_timeout_secs`
|
||
like every other abandoned handshake. The rekey msg1 send site keeps its
|
||
teardown, which was already benign, and stops reporting a local self-clearing
|
||
condition at `warn`.
|
||
|
||
- A peer reachable over two interfaces is no longer re-dialled on the path it
|
||
is not using. Beacon discovery skipped only a candidate naming the peer's
|
||
*current* path, which is the one case that could not churn anything, so the
|
||
alternate path was dialled every discovery tick; each dial that completed
|
||
promoted and displaced a healthy incumbent, tore down the session, and, when
|
||
that peer was the parent, switched parents and re-announced mesh-wide.
|
||
Measured on real hardware, seventeen dials to one peer in fifteen minutes,
|
||
alternating wifi and cable, displacing a link reporting etx 1.0 and loss 0.0.
|
||
Discovery now asks whether the link it already holds is answering rather than
|
||
which path the candidate names. Failover is unchanged: a peer that goes quiet
|
||
for longer than `node.heartbeat_interval_secs` is dialled again on every
|
||
path, alternate included. **One behaviour goes with it.** A peer held on an
|
||
adopted NAT-traversal transport that is *also* reachable by Ethernet or BLE
|
||
beacon used to drift onto the local path on the next discovery tick, and now
|
||
stays on the traversed path for as long as that path answers. Migrating it is
|
||
still done by the configured-peer refresh (a config reload, a runtime peer
|
||
update, or `fipsctl connect`), and a traversed link that goes quiet still
|
||
releases the peer to every path.
|
||
|
||
- A peer that rotates its address no longer keeps sending from a socket
|
||
aimed where it used to be. The authenticated-frame path updated the
|
||
peer's address and discarded the flag saying it had changed, so the
|
||
per-peer `connect()`-ed UDP socket stayed pinned to the old 5-tuple;
|
||
the sibling path already cleared it.
|
||
|
||
#### Data plane and transports
|
||
|
||
- A peer that stops reading can no longer stall the node. TCP, Tor, Nym and
|
||
BLE wrote to their links directly from the caller's task, and a write
|
||
blocks once the peer's receive window and this node's send buffer are both
|
||
full — which is what a peer that has gone away, or whose path has just
|
||
changed medium, produces. The callers are the rx loop's tick handlers, so a
|
||
single unresponsive peer held every other arm of the event loop behind it:
|
||
other peers' liveness, forwarding, and control RPCs. Each connection now has
|
||
its own writer task owning the write half, and sending is an enqueue onto a
|
||
bounded queue, so the only code that can await the wire is a task with
|
||
nothing else to do. A peer that stops draining fills its queue and its sends
|
||
then fail immediately, which is the signal the caller's retry and liveness
|
||
handling already expects. Frames are written whole, and a write error takes
|
||
the connection down with it, so a peer never sees a partial frame it cannot
|
||
resynchronise from. BLE was the worst of the four: it awaited the L2CAP
|
||
write while holding the connection-pool mutex, so one unresponsive peer
|
||
froze every other BLE operation as well — connects, evictions, and each
|
||
receive loop's teardown. On TCP, Tor and Nym, a connection's writer and
|
||
receive loop act only on their own connection: one that outlives its
|
||
connection can no longer tear down a newer connection that has taken the
|
||
same address, and a receive loop that ends stops its writer rather than
|
||
leaving it writing to a peer that has gone. Closing one of their connections
|
||
on purpose, as a control-API disconnect does, now lets the writer finish the
|
||
frames already queued, within five seconds, instead of discarding them, so a
|
||
Disconnect sent just before the close reaches the peer. Stopping the
|
||
transport, or a connection that has failed, still discards them.
|
||
|
||
- A per-peer `connect()`-ed UDP socket is no longer left pinned to an interface
|
||
the host has moved off. Established UDP peers get their own socket for the
|
||
send fast path; `open_connected_fd` binds the wildcard and then calls
|
||
`connect(2)`, which makes the kernel resolve the route once and auto-bind the
|
||
local source address to whichever interface was carrying it at that moment.
|
||
It never re-evaluates. So after the host changed transport medium — a laptop
|
||
moving between WLAN and LAN, a phone between Wi-Fi and cellular — every
|
||
established peer went on transmitting from an address the routing table had
|
||
abandoned, while the peer, which re-pins to whatever address it last heard
|
||
from, answered somewhere the node was no longer sending from. The peering
|
||
stayed marked connected and carried no traffic until the 30s liveness timeout
|
||
tore it down, roughly 60-90s of black-holed traffic per medium change,
|
||
followed by a full re-handshake and tree re-convergence. The mirror-image
|
||
case — the *peer* rotating its address — was already handled at the point the
|
||
rotation is observed; this is the local half, which had no signal to hang off
|
||
because a local move is invisible in the data plane. It now fires from the
|
||
medium-change detection added below, which is exactly that missing signal.
|
||
Dropping the sockets is self-healing rather than disruptive: the wildcard
|
||
listen socket resolves a route per packet, so sends keep working immediately,
|
||
and a correctly-bound connected socket is reinstalled on a later tick. The
|
||
reaction is scoped to the peers the change names: only their sockets are
|
||
dropped, and each of those on a connectionless transport is heartbeated at
|
||
once, so the far side re-pins to the new source address rather than waiting
|
||
out its own heartbeat interval. A peer on a connection-oriented transport
|
||
keeps the periodic heartbeat instead. That was because such a send awaited an
|
||
unbounded `write_all` on a stream the medium change had very likely just
|
||
stranded, and this reaction runs on the rx loop; the writer-task change below
|
||
removes that hazard, so widening the fan-out to those transports is now open
|
||
work rather than something the design forbids. Measured on a live
|
||
node, a WLAN/LAN switch in either direction now costs no reconnection at all —
|
||
the Noise session, tree position and routes survive it. Linux and macOS (the
|
||
platforms with the connected-socket fast path); elsewhere the heartbeat alone
|
||
carries the new address.
|
||
- The Ethernet transport on Linux no longer leaks a socket when the interface
|
||
is missing or its name is invalid at the moment the socket is opened, as when
|
||
the interface goes away between the presence check and the bind.
|
||
- TCP connections try the remaining addresses for a hostname after a
|
||
connection fails, within the existing overall connection timeout.
|
||
|
||
#### Gateway
|
||
|
||
- The gateway retries failed firewall rebuilds every ten seconds without
|
||
waiting for another mapping change. Retries apply the latest desired
|
||
mappings and port forwards, preserving changes across transient failures.
|
||
|
||
#### Packaging
|
||
|
||
- The systemd tarball's `install.sh` restarts fips-dns and fips-gateway after
|
||
an upgrade when they were running. It stopped fips first, which stopped both
|
||
through `Requires=fips.service`, then restarted only fips, so `.fips`
|
||
resolution stayed down and the gateway stayed stopped until started by hand.
|
||
|
||
#### OpenWrt
|
||
|
||
- dnsmasq forwards `.fips` to fips-gateway only while the gateway is
|
||
listening, and back to the daemon's resolver whenever the gateway exits. A
|
||
gateway that failed to start, on a config it could not parse or on an error
|
||
after binding its DNS port, left every LAN client's `.fips` lookups going to
|
||
a port nothing held until the gateway was started by hand.
|
||
- A package built from the OpenWrt SDK feed `Makefile` carries the released
|
||
packages' maintainer scripts. It installs with fips-gateway disabled and
|
||
stopped, keeps the gateway's state across an upgrade, and keeps an edited
|
||
`/etc/fips/fips.yaml`. The SDK's package scan also no longer stops on the
|
||
`Makefile`'s architecture check, which had kept the package out of the build.
|
||
|
||
#### Sessions and rekey
|
||
|
||
- A link rekey whose msg2 is lost now completes on the initiator's next msg1
|
||
resend. The responder refused every resend while it held the session it had
|
||
answered with, so the initiator abandoned its cycles and the link went
|
||
without rekeying until the responder's 120 s hold retired that session. The
|
||
responder now keeps the msg2 it sent and answers a resend of the same msg1
|
||
with it, on the peer's established address only; any other msg1 is still
|
||
refused while the session is held. The wire format is unchanged.
|
||
- Link quality estimates no longer freeze after a link rekey. Frames the peer
|
||
sent on the old session just before it switched were counted into the new
|
||
session's MMP state, so one node's loss estimate, or the other's SRTT and
|
||
loss estimate, could stay fixed for most of a rekey interval, and link cost
|
||
and parent selection used the stale values. Those frames are still
|
||
delivered, but no longer feed MMP, and a ReceiverReport they carry is
|
||
dropped. They also no longer hold the link alive: after a rekey the
|
||
link-dead timer runs from the switch until a frame on the new session
|
||
arrives.
|
||
- A forged FSP SessionMsg3 or SessionSetup can no longer split a session's
|
||
key epochs during a rekey. Either one, delivered under the peer's address
|
||
after the peer had read this node's SessionAck, discarded the handshake the
|
||
peer's genuine msg3 needed; the peer then cut over to keys this node never
|
||
derived, and frames from it stopped decoding until a later rekey. An
|
||
unreadable msg3 now leaves the handshake in place, and a setup arriving
|
||
while a handshake the peer armed awaits its msg3 is dropped and counted as
|
||
`rekey_held`. A genuine retry that meets such a handshake completes one
|
||
handshake timeout later.
|
||
- An unreadable SessionMsg3 no longer discards the half-open session of an
|
||
initial handshake, which left the initiator's genuine msg3 to an unknown
|
||
session.
|
||
|
||
#### Windows
|
||
|
||
- The ZIP's `README.txt` lists `\etc\fips\fips.yaml` as the first file a
|
||
foreground run reads, which it omitted, and says to stop the service before
|
||
rerunning `install-service.ps1` to upgrade: with the service running, the
|
||
installer fails copying `fips.exe`.
|
||
|
||
### Security
|
||
|
||
#### Sessions and rekey
|
||
|
||
- A copy of a peer's link rekey msg1 can no longer stop link key rotation. A
|
||
msg1 carries nothing that ties it to one rekey, so one captured off the
|
||
network and replayed after its rekey had completed was taken as a new
|
||
request: the node held a session nobody could adopt, refused the peer's
|
||
genuine rekeys and skipped its own until the 120 s hold expired, and one
|
||
replay per hold kept rotation stopped. The node now remembers the msg1s of
|
||
the last 256 rekeys it answered for each peer and refuses a copy of one. A
|
||
msg1 from an older rekey, or one captured before the peering last formed
|
||
while the peer kept running, is not recognized; closing that needs a wire
|
||
change.
|
||
- A msg1 from an established peer is answered at the peer's established
|
||
address, not at the address it came from. This covers the rekey msg2 and
|
||
the link setup msg2 resent for a duplicate msg1 in the 30 s after a link is
|
||
formed or rekeyed. Anyone holding a copy of a peer's msg1 could have the
|
||
node send a msg2 to an address of their choosing. A peer whose address
|
||
changed is answered at the old one until its next frame from the new
|
||
address arrives. A msg1 from a node this one holds no link with, or one
|
||
that carries a different startup epoch, still starts a new link and is
|
||
answered at its source, as any new connection is.
|
||
|
||
## [0.5.2] - 2026-09-28
|
||
|
||
### Added
|
||
|
||
#### Gateway
|
||
|
||
- The gateway counts sessions on a kernel without `/proc/net/nf_conntrack`.
|
||
When the file is absent it dumps the conntrack table over netlink, as
|
||
`conntrack -L` does, so a mapping carrying traffic is pinned instead of
|
||
being reclaimed on its TTL and grace period alone. Kernels built without
|
||
`CONFIG_NF_CONNTRACK_PROCFS`, such as Ubuntu's, had session pinning off.
|
||
- The gateway says at startup whether it can read conntrack sessions. It reads
|
||
the table once, as each tick does, and logs either the source it read or
|
||
that no source is readable and session pinning is off. In v0.5.1 an
|
||
unreadable source counted as zero sessions and nothing was logged, so an
|
||
operator on a kernel with no readable source had no way to tell.
|
||
|
||
### Changed
|
||
|
||
#### Identity and config
|
||
|
||
- The shipped `/etc/fips/hosts` no longer lists `test-us03-next`.
|
||
- An ephemeral node no longer writes `fips.key`. It wrote the private key of
|
||
an identity it discards at every restart to that file, overwriting any key
|
||
already there, including an operator's key when `persistent: true` had been
|
||
forgotten. It now writes only `fips.pub`, so the running npub stays visible,
|
||
and holds the private key in memory only. A `fips.key` found at an ephemeral
|
||
start is moved to `fips.key.unused` with a warning, so an ephemeral node logs
|
||
that warning once, on its first start after the upgrade; if that name is
|
||
taken or the rename fails, the file is left in place and a warning says so.
|
||
For a stable identity, set `node.identity.persistent: true` and restart. The
|
||
first persistent start uses the `fips.key` it finds; one already moved to
|
||
`fips.key.unused` can be renamed back to `fips.key` first, as the warning
|
||
says. With no key file, the first persistent start generates and saves one,
|
||
so the npub changes once, at that restart, and is stable from then on.
|
||
Starting once in ephemeral mode and then pinning the key it wrote no longer
|
||
works.
|
||
|
||
#### Gateway
|
||
|
||
- The gateway's default DNS listen address is now `[::1]:5365`; it was
|
||
`[::1]:5353`, the mDNS port, which the daemon's LAN rendezvous, Avahi and
|
||
systemd-resolved can hold. On OpenWrt the init script now points dnsmasq at
|
||
whatever port `gateway.dns.listen` sets, and an upgrade rewrites the
|
||
previously shipped `listen: "[::1]:5353"` line. On other hosts the upgrade
|
||
leaves `fips.yaml` alone, and a config that sets `gateway.dns.listen`, as
|
||
the v0.5.1 example config and deployment guide did, keeps its port; the
|
||
gateway warns at startup when it is configured on 5353. Where `fips.yaml`
|
||
does not set it, a resolver you configured by hand to forward `.fips` to
|
||
`[::1]:5353` must now forward to `[::1]:5365`, or set
|
||
`gateway.dns.listen: "[::1]:5353"` to keep the old port.
|
||
|
||
#### Linux packages
|
||
|
||
- An upgrade of the `.deb` now reapplies the firewall ruleset in place. Until
|
||
now an upgrade reloaded nothing, so a changed `/etc/fips/fips.nft` took
|
||
effect only at the next reboot or manual restart, and a restart deletes the
|
||
`fips` table and leaves the mesh interface unfiltered until the ruleset is
|
||
loaded again. `fips-firewall.service`, in both the Debian and the plain
|
||
systemd unit, gains a reload that replaces the ruleset in one transaction,
|
||
and the postinst reloads the unit only when it is already active, so an
|
||
upgrade never turns the firewall on for a host that has not opted in. A
|
||
reload that fails leaves the previous ruleset in place and is reported; the
|
||
upgrade goes on.
|
||
- The AUR publish on a release tag now waits until every package workflow of
|
||
that tag has succeeded. It used to push the new `pkgver` while the Linux,
|
||
macOS, Windows, OpenWrt and FreeBSD packages were still building; at v0.5.1
|
||
the AUR was updated while the release had 15 of its 17 assets. Because the
|
||
AUR package pins the tag's source archive, withdrawing a bad release after
|
||
that point left the AUR package unbuildable. A failed or cancelled package
|
||
run now stops the publish, and one that has not finished within an hour
|
||
fails it.
|
||
|
||
#### Native datagram API
|
||
|
||
- The native API documentation now says that on Linux an empty datagram sent
|
||
immediately before a close may read as end of file, and is then not
|
||
delivered. Linux carries the flow on `SOCK_SEQPACKET`, where a zero-length
|
||
datagram that is the last message before a close cannot be told apart from
|
||
the close; macOS and FreeBSD carry it on `SOCK_DGRAM` and are not affected.
|
||
The `Received::Datagram` rustdoc, which said an empty datagram is never a
|
||
close, now says where the exception applies.
|
||
|
||
#### Dependencies
|
||
|
||
- The lockfile moves `chacha20` from 0.10.1 to 0.10.2, because 0.10.1 is yanked.
|
||
It arrives through `rand`, a direct dependency,
|
||
so it sits on the built path rather than off to one side. The requirement in
|
||
`Cargo.toml` already admitted 0.10.2, so this is a lockfile change and no code
|
||
changed with it. **This is not a security fix**: `cargo audit` reports nothing
|
||
against `chacha20` at either version, and 0.10.1 was withdrawn by its
|
||
maintainer rather than flagged by an advisory. What it buys is that a fresh
|
||
checkout can resolve the lockfile without reaching for a yanked version.
|
||
|
||
### Fixed
|
||
|
||
#### Identity and config
|
||
|
||
- A persistent node whose identity key path cannot be examined now refuses to
|
||
start instead of coming up under a new identity. `Path::exists` reports false
|
||
both for a key that is absent and for one whose metadata cannot be read, so a
|
||
key symlinked onto a volume that did not mount, or one in a directory the
|
||
daemon cannot search, read as a first boot: the node generated a fresh
|
||
identity, failed to store it, and carried on under an npub that every peer
|
||
whose allowlist names the old one refuses. Only a `NotFound` result is now
|
||
treated as an absence; any other failure to stat the path aborts the start and
|
||
names the path. A dangling symlink likewise aborts rather than being replaced.
|
||
The legacy `/etc/fips/fips.key` lookup follows the same rule.
|
||
- Replacing the peer list at runtime with `Node::update_peers` now updates
|
||
everything that reads peer aliases. `.fips` names, peer ACL entries written as
|
||
an alias, and peer display names kept following the aliases the node started
|
||
with, so a new peer's alias did not resolve, a removed one still did, and a
|
||
deny entry naming an alias moved to another key kept denying the old key and
|
||
admitted the new one. They now follow the new peer list, with the hosts file
|
||
still taking precedence as it does at startup.
|
||
|
||
#### Gateway
|
||
|
||
- The NAT table is rebuilt in one netlink transaction. A rebuild deleted the
|
||
`fips_gateway` table in a batch of its own, discarded that batch's result,
|
||
and only then sent the batch that recreated the table, the chains, the
|
||
`fips0` masquerade and every per-mapping rule. Between the two sends the
|
||
gateway had no NAT at all, and a recreate the kernel refused left the table
|
||
absent for good, taking down forwarding for every existing mapping rather
|
||
than failing the one change that was being made. The delete and the recreate
|
||
now share a single batch, which the kernel applies as one transaction, so a
|
||
refused rebuild leaves the previous table in the packet path. The rules sent
|
||
are unchanged.
|
||
- The gateway's NAT rebuild no longer fails once the table holds more than
|
||
about 105 mappings. Each rebuild is one netlink batch. From about 105
|
||
mappings the default socket buffers could not hold its acknowledgements, so
|
||
rebuilds were logged as failed although they had taken effect. Past about
|
||
313 mappings the buffers could not hold the batch itself, and new `.fips`
|
||
names past that count got a virtual IP with no translation. In releases with
|
||
the gateway through 0.5.1, a rebuild past about 313 mappings also deleted the
|
||
whole `fips_gateway` table, which stopped every mapping, the `fips0`
|
||
masquerade and the port forwards. The rebuild now sizes its send buffer to
|
||
the batch and requests one acknowledgement per batch, and NAT errors now
|
||
name the kernel errno. A rebuild that still fails is logged, and the next
|
||
successful rebuild installs the mapping.
|
||
- Conntrack sessions are matched by address rather than by text, so live
|
||
traffic pins a gateway mapping again. The session count searched each
|
||
`/proc/net/nf_conntrack` line for `dst=` followed by the virtual IP in its
|
||
compressed form (`fd01::1`), while the kernel prints tuples in the full
|
||
uncompressed form (`dst=fd01:0000:0000:0000:0000:0000:0000:0001`), so the
|
||
count was zero for every mapping on every kernel. Nothing pinned an in-use
|
||
mapping, and one whose client did not re-query DNS was reclaimed about two
|
||
minutes after its last DNS reference while its traffic was still flowing.
|
||
Each `dst=` value is now parsed as an address and compared as one.
|
||
- The conntrack table is read once per tick instead of once per mapping, and
|
||
the read happens off the runtime thread. The whole file was read and scanned
|
||
for each mapping in turn, while the pool lock was held, on the same
|
||
single-threaded runtime that serves DNS. The tick now takes one snapshot with
|
||
a blocking task before it takes the lock, and the pool does a map lookup per
|
||
mapping.
|
||
- A conntrack source that cannot be read is reported. It still counts as zero
|
||
sessions for every mapping, as it always has, so reclamation keeps working
|
||
rather than pinning the whole pool; but the first failure and each change of
|
||
outcome after it are now logged, so an unreadable source is no longer
|
||
indistinguishable from an idle one. A source that fails identically every
|
||
tick is logged at debug rather than warn on a repeat.
|
||
- A DNS query that refreshes a draining mapping now cancels its old grace
|
||
period. Previously the address could be reclaimed while the client's renewed
|
||
DNS answer was still valid. The mapping now survives the full renewed TTL
|
||
and a fresh grace period before it can be reused. Contributed by Martti
|
||
Malmi (#169).
|
||
- `fips-gateway` exits when its DNS listener cannot bind, or stops while the
|
||
gateway runs, instead of staying up with `.fips` resolution dead, so systemd
|
||
or procd restarts it or reports it failed. This applies to a gateway used
|
||
only for port forwards too. An "address in use" error names the service
|
||
likely to hold the port. On an OpenWrt access point with the gateway
|
||
enabled, the gateway had lost its port to the daemon's own mDNS responder
|
||
and `.fips` names stopped resolving with nothing reported. A
|
||
`gateway.dns.upstream` written as a hostname now works: the resolver
|
||
forwards to the address the startup check resolved, where before the check
|
||
passed and the resolver then stopped on the unparsed name.
|
||
|
||
#### Linux packages
|
||
|
||
- A `.deb` upgrade whose new daemon cannot start no longer hangs apt. The
|
||
postinst started `fips.service` and then `fips-dns.service` with blocking
|
||
calls, and because `fips-dns.service` requires the daemon, a daemon that
|
||
failed on every start left the second call, apt and everything queued behind
|
||
it waiting for ever with no message. Each start is now queued and waited on
|
||
for at most 60 seconds, 90 for `fips-gateway`. A unit that does not come up
|
||
has its status printed and fails the configure step, so apt exits non-zero
|
||
and names the unit; a masked unit, or one whose condition is not met, is
|
||
reported and skipped.
|
||
- The `.deb` maintainer scripts now manage `fips-gateway` with the rest of the
|
||
package's services. An upgrade stopped the daemon, which the gateway
|
||
requires, and never brought the gateway back, so an operator who had enabled
|
||
it lost it until the next reboot; removing or purging the package left the
|
||
gateway's enablement symlink behind, pointing at a unit file that no longer
|
||
exists. The gateway is now stopped before the daemon on upgrade and
|
||
restarted afterwards only when it is enabled and the daemon came up, and it
|
||
is stopped and disabled on remove and purge. A gateway that does not come
|
||
back is reported but does not fail the upgrade.
|
||
- Purging the `.deb`, or running `uninstall.sh` from the tarball, now removes
|
||
the `.fips` DNS routing when `fips-dns` was not running at the time. The
|
||
cleanup removed the dns-delegate file from the wrong directory, never removed
|
||
the systemd-resolved global drop-in, and restarted no resolver, so the host
|
||
kept sending `.fips` queries to `[::1]:5354`, where nothing listens any more,
|
||
and `.fips` lookups timed out. Both scripts now remove all four files
|
||
`fips-dns-setup` can write, and restart systemd-resolved or reload dnsmasq or
|
||
NetworkManager when they removed that resolver's file and it is running. A
|
||
failed restart is reported and does not fail the removal.
|
||
- The `.deb` now declares `libgcc-s1 (>= 4.2)`. All four binaries link
|
||
`libgcc_s.so.1`, but cargo-deb removes every libgcc entry from the
|
||
dependencies it derives, so the package never said so. `libc6` depends on
|
||
`libgcc-s1` on Debian 12 and Ubuntu 22.04, 24.04 and 26.04, so installs there
|
||
were not affected. A new check, `testing/check-deb-depends.sh`, runs
|
||
`dpkg-shlibdeps` over the package's binaries on every build and fails the
|
||
build when the declared `Depends` leaves out a library the binaries need, or
|
||
states a floor lower or higher than the one they need. A dependency the
|
||
packaging tool drops, including one it drops after only a warning when it
|
||
cannot resolve a binary, now fails the build instead of shipping.
|
||
- The `.deb` now recommends `nftables`, and both AUR `PKGBUILD` files list it
|
||
as an optional dependency. `fips-firewall.service` runs `/usr/sbin/nft`, so
|
||
enabling it on a host without nftables failed at start. It is a
|
||
recommendation rather than a dependency because the firewall unit is opt-in
|
||
and the daemon itself does not need `nft`.
|
||
- The release `PKGBUILD` now lists `dbus` as a runtime dependency. The `fips`
|
||
binary links `libdbus-1`, and the `fips-git` package already declared it.
|
||
- `-V` on binaries built into the Linux packages now includes the source
|
||
revision, as `<version> (rev <git-hash>)`. The build image had no git, so
|
||
every container-built binary printed the version alone. A package built from
|
||
a git worktree still has no revision, because the worktree's git directory is
|
||
outside the tree the build sees. The build image's tag now includes a hash of
|
||
its Dockerfile, so a host with an older image cached builds a new one instead
|
||
of reusing it.
|
||
- `packaging/debian/build-deb-container.sh` now returns the package it just
|
||
built. It picked the most recently modified `fips_*.deb` in the output
|
||
directory that sorted last by name, so a package with a higher version left
|
||
there by an earlier run was returned instead.
|
||
|
||
#### OpenWrt
|
||
|
||
- A new OpenWrt install no longer enables and starts `fips-gateway`. The
|
||
generated postinst turned it on unconditionally, contradicting the init
|
||
script's own header, the package README and the deployment tutorial, all of
|
||
which say the service ships disabled and is enabled deliberately. The
|
||
documented `service fips-gateway enable` / `service fips-gateway start` steps
|
||
are unchanged, and the shipped `fips.yaml` still carries `gateway.enabled:
|
||
true`, so enabling the service is all that is needed.
|
||
- **The first opkg upgrade to this release re-enables and starts
|
||
`fips-gateway` on any router that has the `.ipk` installed, including one
|
||
where the gateway was disabled by hand.** opkg runs the outgoing package's
|
||
prerm, and every released `.ipk` prerm disabled the service on its way out,
|
||
leaving nothing behind that says whether the operator wanted it on, so an
|
||
upgrade cannot tell the two apart and keeps the gateway running rather than
|
||
silently turning off a working one. If you had disabled it, run
|
||
`service fips-gateway stop` and then `service fips-gateway disable` once
|
||
after upgrading. Stopping it hands dnsmasq's `.fips` forwarding back to the
|
||
daemon; disabling it alone leaves it running. An `apk` upgrade on
|
||
OpenWrt 25 runs only the incoming package's scripts, so it keeps the
|
||
gateway's enabled state from the first upgrade on. Later upgrades preserve
|
||
whatever state the service is in: the new prerm stops the services on an
|
||
upgrade but no longer disables them.
|
||
- An `apk` upgrade on OpenWrt 25 now restarts `fips`, and restarts
|
||
`fips-gateway` if it was enabled, so the new binaries run without a reboot.
|
||
apk-tools v3 runs only the incoming package's pre-upgrade and post-upgrade
|
||
scripts, and the `.apk` registered neither, so an upgrade replaced the files
|
||
on disk and left the old processes running until a reboot or a manual
|
||
restart.
|
||
- `start_service` in the `fips-gateway` init script now reads `gateway.enabled`
|
||
from `/etc/fips/fips.yaml` before doing anything. Starting a gateway that the
|
||
config disables used to hand dnsmasq's `.fips` forwarding to the gateway's
|
||
port, add the LAN prefix and advertise the pool route, and only then start a
|
||
daemon that exits immediately because the gateway is disabled, leaving `.fips`
|
||
resolution pointed at a port nothing listens on.
|
||
- The packages no longer ship `/etc/dnsmasq.d/fips.conf`. OpenWrt's dnsmasq
|
||
builds its config from UCI and never reads that directory; `.fips`
|
||
forwarding has always come from the UCI server entry, which is unchanged. An
|
||
opkg upgrade removes the old file, and an apk upgrade keeps it only if it
|
||
was modified. Either way nothing reads it.
|
||
- The package README's upgrade commands and default settings are corrected.
|
||
It now gives the `apk add` command for OpenWrt 25, where there is no opkg,
|
||
and for OpenWrt 24.10 and earlier a plain `opkg install` in place of
|
||
`--force-reinstall`, which removed and reinstalled the package and so left
|
||
`fips-gateway` disabled. Its description of the default config now matches
|
||
the shipped `fips.yaml`.
|
||
- The `.ipk` and `.apk` packages now install the same maintainer scripts. The
|
||
four script bodies live in `packaging/openwrt-ipk/scripts/` instead of inside
|
||
heredocs in the two build scripts, so the scenarios in `testing/openwrt/` run
|
||
what ships.
|
||
|
||
#### Windows
|
||
|
||
- The Windows service now writes its log to `C:\ProgramData\fips\fips.log`,
|
||
rolled at 10 MiB with four old files kept. A service has no standard output,
|
||
so everything the daemon logged in service mode was lost, including
|
||
config-load failures and panic messages. A foreground run still logs to the
|
||
console.
|
||
|
||
#### FreeBSD
|
||
|
||
- The daemon's log, `/var/log/fips.log`, is now rotated. The package ships a
|
||
newsyslog entry that keeps five compressed generations of 1000 KB, and the rc
|
||
script starts `daemon(8)` with `-H` so it reopens the log after a rotation.
|
||
The log used to grow without bound.
|
||
|
||
#### Links and transports
|
||
|
||
- A heartbeat whose send failed no longer counts as one that was delivered. The
|
||
send was recorded before it was attempted, so a peer whose heartbeat could not
|
||
go out was treated as heartbeated and was not tried again for a whole
|
||
`heartbeat_interval_secs`, although it had heard nothing and its own link-dead
|
||
timer was running. The attempt and the delivery are now recorded separately:
|
||
the interval that paces a healthy peer advances only on a send that returned
|
||
cleanly, and a peer whose send failed is retried after a shorter fixed
|
||
interval instead. That retry interval gates only a peer whose last attempt
|
||
failed, so it cannot clamp a `heartbeat_interval_secs` configured below it.
|
||
- A peer that moves to a new address now loses the per-peer `connect(2)`-ed UDP
|
||
socket pinned to the address it left. `set_current_addr` returns whether the
|
||
address actually changed so the caller can drop the stale socket, and the
|
||
decrypt-worker completion path already acted on that return; the in-line
|
||
decrypt path discarded it, so the socket stayed installed and the send path
|
||
kept preferring it over the wildcard listen socket.
|
||
- A peer reached by NAT traversal now gets its per-peer connected UDP socket.
|
||
The adopted traversal socket carried no address-reuse flags, so the connected
|
||
socket's bind to the same port was refused with `EADDRINUSE` on every tick and
|
||
the peer never left the unconnected path. The flags are now set when the
|
||
socket is adopted, after its bind, so the traversal bind still receives a
|
||
port no other socket holds.
|
||
- A configured `ble:` transport that this build cannot construct is now
|
||
reported. The only warning for it was compiled into test builds alone, where
|
||
logging is compiled out, so macOS, Windows, FreeBSD, OpenWrt and other musl
|
||
builds, and Android without a BLE radio armed before start, dropped the block
|
||
silently while reporting healthy. The daemon now warns once per configured
|
||
instance at startup, naming the reason. The shipped example configs no longer
|
||
say BLE needs a `ble` Cargo feature, which does not exist: the common
|
||
`fips.yaml` names the builds that have the transport, and the OpenWrt
|
||
`fips.yaml` drops its BLE example, since its musl builds never include it.
|
||
|
||
#### Sessions and rekey
|
||
|
||
- A session whose last handshake message is lost no longer stays one-sided.
|
||
The initiator sent msg3 once and treated the session as established at once;
|
||
when that one datagram was lost, the responder kept waiting for it and
|
||
dropped every frame the initiator sent, and nothing sent msg3 again, because
|
||
the responder's repeated SessionAck was refused as arriving in the wrong
|
||
state. The session stayed that way until the next session rekey, or with
|
||
periodic rekey switched off, indefinitely. The initiator now keeps its msg3
|
||
and resends it on the handshake resend interval, with backoff, until a frame
|
||
from the responder authenticates or `handshake_max_resends` resends have gone
|
||
out. The wire format is unchanged: the resend carries the same msg3, and a
|
||
responder that already completed the session refuses the duplicate as before.
|
||
- A session rekey this node started no longer stays in flight forever when its
|
||
setup or the peer's ack is lost. Nothing resends a rekey setup, and the only
|
||
expiry covered a rekey the peer started, so one lost datagram left the
|
||
rekey pending and blocked every later one: the session kept its current keys
|
||
and stopped rotating them. The rekey now expires on the handshake timeout,
|
||
timed from when this node sent its setup, and the next tick starts a fresh
|
||
one. A forged ack cannot extend it. Expiries are counted as
|
||
`rekey_unanswered`. The wire format is unchanged.
|
||
- A link rekey whose reply is lost no longer splits the link. The node that
|
||
answered a rekey used to switch to the new keys on its own next tick, before
|
||
the other side had them; when the reply was lost, frames from the answering
|
||
side were dropped until the link was torn down. The answering side now
|
||
switches only when a frame on the new keys arrives from the side that started
|
||
the rekey, and drops keys that were never adopted after a hold (120 s by
|
||
default) so the next rekey can proceed.
|
||
- A node with no coordinates cached for a session's destination no longer
|
||
sends its own coordinates in their place. The lookup that supplies them falls
|
||
back to the node's own coordinates, which a first-contact SessionSetup needs
|
||
because its destination field cannot be empty, but the established data path,
|
||
the standalone CoordsWarmup and the rekey SessionSetup used the same
|
||
fallback. Every receiver files the destination coordinates it is sent under
|
||
the destination's address, so a destination reached this way cached its own
|
||
address under the sender's coordinates. On a cache miss a data frame now goes
|
||
out without coordinates and leaves the warmup budget for the first frames
|
||
after the cache is refilled, a standalone CoordsWarmup is not sent, and a
|
||
rekey SessionSetup, which can only miss for a direct peer, carries the
|
||
coordinates that peer announced. First-contact setup is unchanged. The wire
|
||
format is unchanged.
|
||
|
||
#### Routing and discovery
|
||
|
||
- A node now re-announces its bloom filter when a peer starts or stops using
|
||
it as parent. A node's outgoing filter merges only its tree peers' filters,
|
||
and nothing re-marked the other peers when a peer's tree announce changed
|
||
whether it named us as parent, so our parent kept the old filter.
|
||
Destinations under a new child stayed missing from discovery, and a departed
|
||
child's stayed advertised, until some unrelated change. The re-announce fires
|
||
only when that relation flips and only to peers whose filter actually
|
||
changed, so ordinary tree churn does not multiply announce traffic.
|
||
- A bloom filter announce lost on the link is now resent. A node counted an
|
||
announce as delivered once the transport accepted it, and announces go out
|
||
only when a filter changes, so a dropped datagram or a link outage shorter
|
||
than the dead timeout left the peer holding the old filter until something
|
||
else changed, and destinations could stay missing from discovery. The node
|
||
now confirms each announce from the link's existing receiver reports,
|
||
resends when they show a loss, and resends once after 30 seconds when the
|
||
reports cannot confirm it. Resends over one peer connection are limited to
|
||
six a minute, and to one a minute while losses persist.
|
||
- A spanning-tree announce lost on the link is now resent. A node counted a
|
||
tree announce as delivered once the transport accepted it, so after a
|
||
dropped datagram or a link outage shorter than the dead timeout the peer kept
|
||
our old tree position until the periodic re-broadcast, up to a minute later,
|
||
and a node with only one peer had no periodic re-broadcast at all. Meanwhile
|
||
the peer could leave destinations out of discovery or route toward them by
|
||
stale coordinates. The node now confirms each tree announce from the link's
|
||
receiver reports, as it does for bloom filter announces, resends it when
|
||
they show a loss, and resends it once after 30 seconds when they cannot
|
||
confirm it, under the same limits.
|
||
|
||
#### Node health and control socket
|
||
|
||
- A DNS responder or TUN thread that dies now degrades the node's published
|
||
health, and a dead responder's address is retracted. The responder's exit
|
||
report followed a loop that never returns, so it could not run, and a panic
|
||
in the responder or in either TUN thread unwound past its report. The node
|
||
kept reporting healthy with the child gone and kept publishing the DNS
|
||
address with nothing answering on it. Each child now runs inside a wrapper
|
||
that catches a panic, logs it, and reports the exit either way. A deliberate
|
||
stop still reports nothing.
|
||
- `show_links` (`fipsctl show links`) now reports the traffic a link has
|
||
carried. Its `packets_sent`, `packets_recv`, `bytes_sent`, `bytes_recv` and
|
||
`last_recv_ms` were read from counters on the link record that nothing on
|
||
the data plane ever wrote, so every link reported zero however much traffic
|
||
it carried, while `show_peers` counted the same traffic on the peer. A link
|
||
bound to an authenticated peer now reports that peer's counters, so the two
|
||
queries agree for the same `link_id`; a link still in handshake has no peer
|
||
yet and still reports zero. The counters follow the peer across address
|
||
changes, while the row's `transport_id` and `remote_addr` stay those the
|
||
link was created with. The counters cover authenticated link frames only, so
|
||
they are not expected to match the transport totals in `show_transports`.
|
||
The response shape is unchanged. Fixes #158.
|
||
- `show_peers` (`fipsctl show peers`) now reports a peer that has gone quiet
|
||
as `stale`. Its `connectivity` was read from a state that nothing outside
|
||
the tests ever changed, so every peer read `connected` until it was
|
||
removed, including one that had stopped answering tens of seconds earlier.
|
||
The value is now derived from how long the peer has been silent: `connected`
|
||
while its idle time is at or below `heartbeat_interval_secs`, and `stale`
|
||
above it, the same rule that decides whether discovery re-dials an active
|
||
peer on the path it already has. The `reconnecting` and `disconnected`
|
||
values the open-discovery tutorial described never occurred, and the
|
||
tutorial no longer lists them. The response shape is unchanged.
|
||
|
||
### Security
|
||
|
||
#### Gateway
|
||
|
||
- A `.fips` query the gateway answers without an address no longer takes an
|
||
address from the pool. Every query type was allocated a mapping before the
|
||
code looked at what the client had asked for, and an A or HTTPS query was
|
||
then answered with NODATA, so any host that can reach the LAN resolver could
|
||
consume the pool one name at a time with a query type it is never given an
|
||
address for. Only AAAA and ANY allocate now. A non-AAAA query for a name that
|
||
already has a mapping still refreshes that mapping's TTL clock, so a client
|
||
querying both types does not lose half of its refresh.
|
||
- The gateway's virtual-IP pool now limits how many mappings it holds and how
|
||
fast it creates them. Any host that can reach the LAN resolver could ask for
|
||
one new `.fips` name after another, and each got a mapping until the 65,535
|
||
addresses ran out, while every mapping made each NAT rebuild, each pool tick
|
||
and shutdown slower. The pool now refuses a new name once it holds 1000 live
|
||
mappings, and admits new names at 10 per second after a burst of 50. A
|
||
refused query gets SERVFAIL, and the gateway's "Pool allocation failed"
|
||
warning says which limit refused it. A name that already has a mapping is
|
||
answered before either limit is consulted, so names in use keep resolving
|
||
when the pool is full. The limits are compiled in, not configured.
|
||
|
||
#### Windows
|
||
|
||
- Windows now keeps its config, key, hosts and peer ACL files in
|
||
`C:\ProgramData\fips`, the directory the service installer writes to. The
|
||
config search, the key directory and the peer ACL defaults disagreed: the
|
||
config search never looked in `C:\ProgramData\fips`, so the service depended
|
||
on the `FIPS_CONFIG` the installer set, `fipsctl keygen` wrote to the
|
||
per-user `%APPDATA%\fips`, and a `peers.deny` placed beside the hosts file
|
||
was never read, so the ACL failed open. A key left in `%APPDATA%\fips` is
|
||
still used by a persistent node when the new directory has none, with a note
|
||
to move it. For one release, a `peers.allow` or `peers.deny` left at the old
|
||
`\etc\fips` location is still read when the new directory has no such file,
|
||
with a warning naming both paths.
|
||
- The Windows service installer now restricts `C:\ProgramData\fips` to SYSTEM
|
||
and Administrators. The directory inherited `C:\ProgramData`'s default ACL,
|
||
which lets any local user read the files in it and create new ones, so any
|
||
local account could read the node's key, or create a missing `fips.key`,
|
||
`fips.yaml` or `hosts` that the service then used. The installer creates the
|
||
directory with the restricted ACL, or replaces the ACL of an existing one,
|
||
resets the files already in it to inherit it, and refuses to continue if the
|
||
directory or anything in it is a link or a folder, or if the directory is
|
||
owned by another account. After moving files into the directory, stop the
|
||
service and rerun the installer; it cannot replace `fips.exe` while the
|
||
service runs. A foreground run from an unelevated prompt can no longer read
|
||
the files there.
|
||
- The Windows service installer now creates empty `peers.allow` and
|
||
`peers.deny` files in `C:\ProgramData\fips`. The service read these lists
|
||
from `\etc\fips` on the system drive, where any local user can create files,
|
||
so a planted list was enforced; for this release it still falls back there
|
||
when a file is missing from `C:\ProgramData\fips`. An empty file allows
|
||
every peer; to clear a list, empty its file rather than deleting it. The
|
||
installer stops when either file exists under `\etc\fips` and not in
|
||
`C:\ProgramData\fips`, so an upgrader's list is neither enforced from the
|
||
old location nor dropped unreviewed: review it, move it into
|
||
`C:\ProgramData\fips` or delete it, and run the installer again. Windows
|
||
upgraders should stop the service and rerun `install-service.ps1`.
|
||
|
||
#### Links and transports
|
||
|
||
- Two inbound TCP connections that share a peer address but arrive on different
|
||
local addresses no longer share one pool entry. The kernel names a connection
|
||
by its four-tuple, so a listener on a wildcard address, which is what the
|
||
shipped configuration binds, can accept two connections whose peer `ip:port`
|
||
is the same on two different local addresses. The pool was keyed by the peer
|
||
address alone: the second connection's entry replaced the first's while the
|
||
inbound-connection counter counted both, the first connection's teardown then
|
||
removed the second's entry, and the second's own teardown found nothing to
|
||
remove, so the counter ended one above the connections it counts. That counter
|
||
gates the inbound connection limit, so a host repeating the collision could
|
||
hold it at the limit and lock out further inbound TCP connections until the
|
||
daemon restarted. Inbound entries now carry the accepted socket's local
|
||
address in their pool key as well as the remote one.
|
||
|
||
#### Sessions and rekey
|
||
|
||
- A SessionAck that fails to read no longer ends a session rekey this node
|
||
started. The handler took the rekey handshake off the session before reading
|
||
the ack's msg2 and abandoned the rekey when the read failed, although nothing
|
||
authenticates the ack before that read and the only tie to the rekey is the
|
||
datagram's source address. The handshake is now rolled back to its state
|
||
before the read, so the peer's genuine ack still completes the rekey,
|
||
and the refusal is counted as `ack_handshake_failed`, as it already was for a
|
||
first-contact session. The wire format is unchanged.
|
||
- A forged rekey msg2 no longer takes the link down. The rekey initiator gave
|
||
up its handshake before reading msg2 and abandoned the cycle when the read
|
||
failed, although nothing authenticates a msg2 ahead of that read. Anyone on
|
||
the path who saw the rekey msg1 go out could answer first with a msg2 of the
|
||
right size under the index msg1 carries in cleartext. The responder had
|
||
already committed its new session by then and cut over on its next tick, so
|
||
the two ends were left on different keys: frames from the responder were
|
||
dropped at once, frames to it failed once its drain window closed, and each
|
||
end removed the other on the link-dead timeout about 30 s later. A msg2 that
|
||
fails the read now leaves the handshake as it was before the read, along
|
||
with the msg1 resend schedule and the msg2 dispatch entry, so the
|
||
responder's genuine msg2 still completes the rekey. In exchange, every such
|
||
forgery now costs the initiator the msg2 key agreement until the cycle ends,
|
||
where before only the first one did; the msg1 resend budget bounds that. The
|
||
wire format is unchanged.
|
||
|
||
#### Routing and discovery
|
||
|
||
- A node no longer publishes NIP-09 deletion requests signed with its routing
|
||
key after a NAT traversal attempt. Each request put the node's public
|
||
identity next to the ids of its offer and answer gift wraps on every relay it
|
||
reached, which the one-time signing keys on those wraps exist to prevent, and
|
||
most of the requests deleted nothing, since a relay deletes a gift wrap only
|
||
at its recipient's request. A relay that stores the wraps now keeps them
|
||
until their NIP-40 expiration; relays that do not store ephemeral events
|
||
never held them. The advertisement retraction still sends its deletion
|
||
request, since that names an event the routing key signed itself. The
|
||
discovery and traversal design documents describe the new behaviour.
|
||
|
||
#### Dependencies
|
||
|
||
- The lockfile moves `rustls` from 0.23.43 to 0.23.45, for RUSTSEC-2026-0285:
|
||
0.23.43 accepted TLS 1.3 handshake messages across encryption-level
|
||
boundaries. It is the TLS client the Nostr relay connections use, so every
|
||
default build reached it. The update is within the version range the
|
||
dependencies already allowed.
|
||
|
||
## [0.5.1] - 2026-09-06
|
||
|
||
### Fixed
|
||
|
||
#### Discovery
|
||
|
||
- A node no longer relays away the answer to its own lookup. A request is
|
||
flooded to every tree peer whose bloom filter claims the target, so a false
|
||
positive can send a copy out into the wider network and circulate it back to
|
||
the node that originated it. The only identity test on arrival was whether
|
||
the request named this node as the target, which a lookup this node
|
||
originated never satisfies, so the copy was filed in the request dedup cache
|
||
as ordinary transit under this node's own `request_id`. When the target
|
||
answered, the reply was reverse-path forwarded to the peer that looped the
|
||
request, the pending lookup was never satisfied, and discovery reported that
|
||
its requests went unanswered while the answers were in fact arriving. An
|
||
inbound response is now matched against this node's outstanding lookups
|
||
before the transit dedup record, and a returning copy of this node's own
|
||
request is dropped as the duplicate it is rather than recorded, so that id
|
||
never enters the transit cache at all. This was a race rather than a hard
|
||
failure: a reply that beat the looped copy found a clean cache and
|
||
succeeded, and the failure grew likelier as the bloom fill ratio rose.
|
||
Contributed by Arjen.
|
||
|
||
- A lookup request of this node's own, returning to it, is no longer counted as
|
||
a duplicate from the peer that delivered it. The fix above drops that copy,
|
||
and it recorded the drop under the existing `req_duplicate` rejection, whose
|
||
documented meaning is that a peer resent a request. A returning copy has a
|
||
nonzero floor in healthy operation and rises with the bloom fill ratio, so
|
||
folding the two together put a permanent number on a counter an operator
|
||
reads as neighbour misbehaviour, and made the two events indistinguishable.
|
||
It now has its own rejection reason and counter, `req_own_loopback`, shown in
|
||
`fipstop` as "Own Loopback". `req_duplicate` returns to meaning only what it
|
||
says.
|
||
|
||
#### Packaging
|
||
|
||
- The Linux `.deb` and the systemd tarball now install and run on Debian 12 and
|
||
Ubuntu 22.04. Every Linux artifact from v0.3.0 through v0.5.0 was built on the
|
||
newest available runner, whose C library made the standard library's `pidfd`
|
||
references a hard `GLIBC_2.39` version requirement instead of the weak,
|
||
runtime-checked ones it is meant to compile to. The loader refuses an image on
|
||
that entry alone, so `fips`, `fipstop` and `fips-gateway` could not start;
|
||
`fipsctl` was unaffected, which is why an install that was checked by running
|
||
it looked healthy while the daemon was dead. No source code caused this and
|
||
none was changed. The Linux artifacts are now built in a container pinned to
|
||
the oldest supported distribution, declared with the floor in
|
||
`packaging/build-floor.env`, and every producer on the release path runs
|
||
`testing/check-glibc-floor.sh` on what it made, so a package or a tarball that
|
||
would not load fails the build rather than reaching a user. The deprecated
|
||
host-build targets in `packaging/Makefile` are not on that path and are not
|
||
floor-checked. The declared
|
||
dependency is derived from the binaries instead of hand-written, so it states
|
||
the floor it was built against.
|
||
|
||
- The `.deb` install suite no longer hangs when the daemon it installed cannot
|
||
run. It started `fips-dns.service` with no timeout, and that unit is
|
||
`Type=oneshot` with `Requires=fips.service`, so a daemon that cannot execute
|
||
is restarted every five seconds for ever, the oneshot start job is never
|
||
dispatched, and `systemctl start` never returns. The suite then produced no
|
||
failure line, no results line and no exit status at all, which is the whole
|
||
class of fault it exists to find: it stopped reporting at exactly the point it
|
||
was most needed. Observed at 21 minutes against a package whose binaries could
|
||
not load. The start is now queued rather than waited on, with a bounded wait
|
||
for the unit to become active, so a dead daemon fails the suite instead of
|
||
stalling the run that gates artifact publication.
|
||
|
||
## [0.5.0] - 2026-08-30
|
||
|
||
### Added
|
||
|
||
#### Platforms
|
||
|
||
- FreeBSD support for the daemon, `fipsctl`, and `fipstop`, on x86_64 only:
|
||
native TUN datapath (TUNSIFHEAD address-family framing, kernel-assigned
|
||
`tunN` device name as with `utun` on macOS), clean service teardown,
|
||
`/usr/local/etc/fips` config search path, and `/var/run/fips`
|
||
control-socket default (both shared with macOS). The `hosts`,
|
||
`peers.allow` / `peers.deny` and `fipsctl keygen` defaults follow the
|
||
same `/usr/local/etc/fips` layout as macOS; that move and its startup
|
||
warning are described by the three macOS path entries under `[0.4.2]`
|
||
below, which this platform inherits.
|
||
`fips-gateway` remains Linux-only. Native `.pkg` packaging under
|
||
`packaging/freebsd/`
|
||
(`make freebsd`) with rc.d services, a `fips` control-socket group,
|
||
service stop/restart across `pkg upgrade`, and `.fips` DNS integration
|
||
for `local_unbound`/`unbound`/`dnsmasq`. mDNS LAN discovery works via
|
||
`mdns-sd` 0.20 (`socket-pktinfo` 0.4.1, the first release that builds
|
||
on FreeBSD). Daemon logs now disable ANSI color when stdout is not a
|
||
terminal (all platforms). No aarch64 FreeBSD artifact is produced and
|
||
that combination is not verified here.
|
||
|
||
- Android-ready core, supported as an embedded crate rather than as a
|
||
standalone daemon: there is no Android daemon artifact, since the library is
|
||
the delivery form. The daemon's
|
||
desktop transports and TUN operations are
|
||
gated by `target_os` rather than by Cargo features, so a plain `cargo build`
|
||
compiles for every target with no flags and Android self-excludes the raw
|
||
Ethernet transport as Windows already did. `Node::enable_app_owned_tun()`
|
||
gives an embedder that owns the TUN file descriptor (an Android
|
||
`VpnService`, for instance) a channel pair for exchanging IPv6 packet bytes
|
||
with FIPS instead of FIPS creating a system TUN device, and `start()` then
|
||
performs no system-TUN or `CAP_NET_ADMIN` operations. Packets entering this
|
||
way bypass `handle_tun_packet`, so the embedder must push only
|
||
`fd00::/8`-destined packets and clamp TCP MSS on outbound SYNs. Desktop
|
||
builds are unchanged and no Cargo features are introduced.
|
||
|
||
- `Node::dns_local_addr()`, the DNS companion to the app-owned TUN interface
|
||
above.
|
||
An embedder whose resolver is pointed into the tunnel has no system socket
|
||
aimed at the built-in `.fips` responder, so the accessor reports the address
|
||
read back off the bound socket: `dns.port = 0` therefore yields the
|
||
kernel-assigned port, and it returns `Some` only while the responder is up.
|
||
Read it once, after `start()` returns and before the node is moved into a
|
||
background task; it is not a liveness feed (#136).
|
||
|
||
#### Native datagram API
|
||
|
||
- A native datagram API addressed by public key, off by default, with a
|
||
surface that may still change. A client process opens a flow to a
|
||
peer's public key on a chosen port and sends and receives datagrams on a
|
||
file descriptor the daemon hands it: no IPv6 emulation, no TUN device and no
|
||
DNS, a datagram travelling from key to key. **The wire needs no change and
|
||
gets none.** Every FSP data packet has carried a port pair inside its AEAD
|
||
envelope since v0.2.0 and port 256 is simply the IPv6 shim, so what was
|
||
missing was a way for a program to ask for a port of its own and be handed
|
||
the traffic. The x-only public key is the address and an npub is that key
|
||
written in bech32, so converting between them is a local encoding rather
|
||
than a lookup or a name service; the 16-byte node address that travels on
|
||
the wire is a truncated hash of the key, does not invert, and appears
|
||
nowhere a client can see. A listener is a descriptor: the daemon writes one
|
||
message per arrival to it, carrying the new flow's descriptor and the peer's
|
||
address, so poll, select and epoll work on a listener and accepting is a
|
||
`recvmsg`. There is no accept command and no reject command, and refusing a
|
||
flow is closing the descriptor you were handed. The Rust surface mirrors
|
||
`std::net`, with `FipsStream::connect`, `FipsListener::bind`, `incoming`,
|
||
`accept`, `io::Result` and an errno mapping rather than a bespoke error
|
||
type, plus `set_nonblocking`, `AsFd` and four deadline methods under the
|
||
names and signatures `std::net` uses for the same jobs. One rule has no
|
||
counterpart in Berkeley sockets and a client author must know it: the v1
|
||
wire carries no half-close, so nothing peer-driven ever closes a flow, and a
|
||
server written to read until the flow ends waits for a signal that cannot
|
||
arrive. The listener uses `SOCK_SEQPACKET` on Linux and `SOCK_DGRAM` on macOS
|
||
and FreeBSD; both keep the message boundaries the API's contract with its
|
||
clients rests on. macOS does not implement `SOCK_SEQPACKET` for `AF_UNIX` at
|
||
all. FreeBSD accepts the constant and returns a socket that is not an
|
||
atomic-record socket, so consecutive messages coalesce and a zero-length
|
||
message is dropped rather than delivered; both were measured on the FreeBSD
|
||
15.1 image rather than reasoned about. The three kernels signal a closed
|
||
peer differently and were measured too, so the receive path treats a Darwin
|
||
or FreeBSD `ECONNRESET` as end of file alongside the `POLLHUP` and
|
||
zero-byte read that Linux gives. `EAGAIN` is deliberately not in that
|
||
company: it means the socket is empty and the peer alive, so it stays an
|
||
error and the caller waits again. **A zero-length payload has one known
|
||
limitation.** A closed peer latches `POLLHUP` while its messages are still
|
||
queued, so that flag alone cannot say whether a zero-byte read is an empty
|
||
datagram or the close; the receive path also asks `FIONREAD`, and bytes still
|
||
queued prove a further message is waiting. That leaves one case unresolved:
|
||
a zero-length datagram that is the last message before a close is reported as
|
||
the close, because reading it drains the queue and a zero-length message
|
||
contributes no bytes to `FIONREAD`. A client should not give a zero-length
|
||
payload a meaning of its own, and should carry a one-byte discriminator
|
||
instead.
|
||
|
||
#### OpenWrt mesh
|
||
|
||
- OpenWrt 802.11s open-mesh backhaul: router-to-router radio links with FIPS
|
||
providing all encryption, authentication and routing over bare L2 neighbor
|
||
links. The mesh runs open with `mesh_fwding 0`, since SAE would duplicate the
|
||
Noise layer and force ath10k raw mode, and FIPS's spanning tree is the
|
||
routing layer. `fips-mesh-setup` is an opt-in UCI helper creating a per-radio
|
||
mesh point (`radio0` to `fips-mesh0`, `radio1` to `fips-mesh1`, with a
|
||
free-index fallback and a collision guard); radio setup stays opt-in because
|
||
a package must not commandeer radios on install. A dual-band router gets one
|
||
instance per radio, and FIPS treats the two paths as failover rather than
|
||
multipath: cross-connection resolution keeps one active link per peer and the
|
||
second band stands by, re-establishing after keepalive timeout. The shipped
|
||
`fips.yaml` carries the `mesh0`/`mesh1` Ethernet-transport entries commented
|
||
out, so a stock install that never creates them logs no per-boot
|
||
interface-missing warning; the helper uncomments the matching block when it
|
||
creates the interface and re-comments it on remove (#123).
|
||
|
||
- OpenWrt open `!FIPS` access SSID, stacked on the mesh backhaul above: every
|
||
FIPS router broadcasts the same open SSID, forming one standard ESS that
|
||
phones and laptops save once and roam between natively, with the Noise IK
|
||
handshake as the only security layer. The leading `!` sorts it to the top of
|
||
alphabetically ordered network pickers, and the encryption type must be
|
||
uniform across routers or clients treat the ESS as different saved networks.
|
||
`fips-ap-setup` is an opt-in UCI helper creating the `fips-ap0` open AP on an
|
||
isolated network with a static ULA /64 and RA-only odhcpd addressing:
|
||
stateless SLAAC with no DHCP, the minimum that satisfies Android's
|
||
provisioning check, behind a locked-down `fips_ap` firewall zone with no
|
||
path to `br-lan` or the WAN, reaching only ICMPv6, mDNS and the FIPS
|
||
transports. There is no internet by design, so phones keep cellular as their
|
||
default route (#126).
|
||
|
||
#### Node lifecycle
|
||
|
||
- A bounded graceful-shutdown drain phase, controlled by the new
|
||
`node.drain_timeout_secs` (default 2s). On the shutdown signal the node
|
||
broadcasts Disconnect to all peers and then keeps serving for that window,
|
||
exiting early once all peers are gone, so in-flight traffic settles and peers
|
||
observe the disconnect before the transports close, where previously teardown
|
||
was immediate. The published node state gains a `Draining` variant visible
|
||
via control queries during the window. The immediate stop path used by
|
||
non-daemon callers is unchanged.
|
||
|
||
#### Transports & config
|
||
|
||
- The UDP transport's listen socket descriptor can now be handed to an
|
||
embedder, for hosts that associate a socket with one interface or network
|
||
and steer inbound traffic by that association rather than routing by
|
||
destination address. On such a host a peer reachable only over a secondary
|
||
network fails in a way FIPS can neither see nor fix: the address is
|
||
well-formed, the send succeeds, the peer replies, and the host discards the
|
||
reply before it reaches our socket, so the link retries msg1 forever with no
|
||
error surfaced anywhere. The correction is a socket option chosen against
|
||
host state FIPS has no basis to reason about, so the descriptor goes to
|
||
whoever does. Call `Node::enable_app_owned_udp_fd()` after `Node::new` and
|
||
before `start()`, and read `AppOwnedUdpSocket { instance, fd }` off the
|
||
returned channel once the transport is up, following the existing
|
||
`enable_app_owned_tun` contract. One message is sent per UDP transport that
|
||
binds, so a multi-listener configuration yields all of them; nothing is sent
|
||
when no UDP transport is configured or one fails to bind, so an embedder
|
||
tells "no socket" from "here is the socket" by the receive timing out. The
|
||
`instance` field is the name the listener was configured under, `None` for a
|
||
single unnamed instance, and it is what makes more than one listener usable:
|
||
transports are created by iterating a map, so arrival order is luck, and an
|
||
embedder whose whole purpose is to bind one socket to one network would
|
||
otherwise have to guess which socket it just received. Guessing wrong pins
|
||
one lane's socket to the other lane's network, which is the failure the
|
||
interface exists to correct. FIPS keeps owning the socket, and
|
||
the descriptor carries no promise beyond "this is the transport's socket,
|
||
and it is open now". Two limits: the per-peer connected-UDP sockets that
|
||
Linux and macOS open after `start()` returns are not covered, and a
|
||
transport that adopts a socket handed in by the traversal bootstrap does not
|
||
deliver one. Unix only, since the Windows UDP backend has no descriptor.
|
||
|
||
- A peer address may name which *instance* of a transport it belongs to, as
|
||
`transport: "udp/aware"` rather than `"udp"`, where the part after the slash
|
||
is the key the transport was configured under. A node running several
|
||
instances of one type could not be told them apart by a dialer: both bind
|
||
wildcard sockets, so the address-family test matches either, and selection
|
||
fell through to the lowest transport id. One socket carried every dial and
|
||
the other never carried traffic. A bare type is unqualified and matches any
|
||
instance, which is what every existing configuration and caller produces, so
|
||
nothing changes for a node that does not use the syntax. A qualified name is
|
||
never substituted with a different instance: that is the wrong-lane dial the
|
||
syntax exists to prevent, so an unmatched name fails the address instead, and
|
||
the same name is what an embedder binds by and what the dialer routes on. The
|
||
slash is already how FIPS qualifies an instance inside an address
|
||
(`eth0/aa:bb:...`) and cannot occur in a type name. Only UDP resolves an
|
||
instance name today; an address that qualifies any other transport type is
|
||
refused rather than matched loosely. **Because a qualified name never falls
|
||
back, the configuration validator rejects one that no configured transport
|
||
answers to**, naming the peer, the instance asked for and the instances that
|
||
exist. Otherwise the address would simply be skipped at every dial, which is
|
||
invisible for a peer that has a second address that works: the lane would
|
||
never carry traffic and nothing above debug logging would say so.
|
||
|
||
#### Bluetooth LE
|
||
|
||
- The BLE transport is refactored so the code common to Linux and Android is
|
||
implemented once, with a separate backend for each platform, and gains a
|
||
reliability pass closing several defects a two-node field capture surfaced.
|
||
The module gate is now `ble_available`, meaning glibc Linux or Android
|
||
rather than `target_os = "linux"`, so the transport is no longer conflated
|
||
with one of its backends; musl is excluded and a platform with no concrete
|
||
backend fails the build rather than compiling a transport that starts,
|
||
reports itself up and never peers. The receive path recovers packet
|
||
boundaries from the 4-byte FMP common prefix instead of assuming one read
|
||
returns one whole packet, which held only for BlueZ's `SOCK_SEQPACKET` and
|
||
not for a stream-oriented backend such as Android's `BluetoothSocket`, where
|
||
fragments shipped up as runts that FMP and Noise rejected and coalesced
|
||
tails were silently truncated. A peer is recognised by node identity rather
|
||
than by its link address, so a phone rotating resolvable private addresses
|
||
no longer presents as a new device on every rotation and defeats the
|
||
already-connected guards. The L2CAP PSM is decided by the backend: `listen`
|
||
reports the PSM it actually bound, the advertisement carries it beside the
|
||
128-bit service UUID, and a dialer learns it from the scan, which is what
|
||
Android and macOS require since both assign the PSM rather than letting an
|
||
application request one. Android gains an embedder-supplied radio backend
|
||
driving a radio the embedder installs into a per-node slot with
|
||
`Node::enable_app_owned_ble_radio()`, called after `new()` and before
|
||
`start()`, because its Bluetooth APIs sit behind a permission and
|
||
foreground-service model only the application can satisfy. Stopping the
|
||
transport now stops the scan as well as the advertisement, which matters
|
||
only where the embedder owns the radio: BlueZ ends discovery when the
|
||
scanner's event stream drops, but an app-owned radio went on scanning for
|
||
the life of the process. Probe retry backs off by powers of two with a
|
||
capped retry book, and each connect outcome has its own counter and
|
||
structured log line carrying the role, outcome, PSM and time to
|
||
conclusion. Inbound handshakes run off the accept loop, eight in flight
|
||
and aborting the oldest at the bound, where the exchange previously ran
|
||
inline and held the loop for its full 5-second deadline, making effective
|
||
inbound concurrency one.
|
||
|
||
#### Observability & measurement
|
||
|
||
- An optional tick-body profiler behind the new `profiling` Cargo feature,
|
||
**off by default**. When enabled, `fipsctl profile tick on [--dir PATH]` /
|
||
`off` / `status` starts and stops a capture at runtime with no restart. Each
|
||
capture writes one tab-separated file (default `/var/log/fips`, capped at
|
||
32 MB) carrying, per ten-second interval, the exact count, max and total for
|
||
every step of the rx-loop tick arm, the whole-tick span, and gauges for ticks,
|
||
peer count, the gap between successive tick-arm entries and the resulting
|
||
arm-starvation delay. With the feature off the instrumentation macro is a pure
|
||
pass-through, so a default build contains no timing code on the tick path.
|
||
`LogsDirectory=fips` was added to the packaged systemd units so the capture
|
||
directory is created and cleaned up declaratively.
|
||
|
||
- `fipsctl probe <npub|hostname>` answers, for one target, where it sits in
|
||
the spanning tree relative to this node and whether this node can actually
|
||
reach it. The work runs as five stages that report separately, `bloom`,
|
||
`discovery`, `path`, `session` and `rtt`, because one verdict covering
|
||
several findings is what sends an operator to the source: "no peer's filter
|
||
claims this address" says the mesh has never heard of the target, while "a
|
||
filter claimed it and nothing answered" says the opposite. The probe opens
|
||
an FSP session, waits for one MMP receiver report to yield a round-trip
|
||
time, and tears down only what it opened. A session that existed before the
|
||
probe started is never torn down, ownership is decided at the moment of
|
||
action rather than once at the start, and it is re-checked before teardown,
|
||
so a session adopted by traffic underneath the probe is left alone. **The
|
||
path printed is the least-common-ancestor walk computed from the two sets of
|
||
coordinates**, which is the worst-case fallback route rather than the route
|
||
a packet necessarily takes: a cut-through between peers can deliver in fewer
|
||
hops, so the tree distance is an upper bound. Nothing traverses the mesh to
|
||
confirm the hops, and a real per-hop trace needs a wire message that does
|
||
not exist. **Nothing here changes the wire format**; the probe is built from
|
||
messages that already exist. The control socket carries three new commands,
|
||
`probe_start`, `probe_poll` and `probe_cancel`, each returning in well under
|
||
a millisecond with the stages advanced on the daemon's tick, because a probe
|
||
needs a mesh lookup, a Noise XK handshake and at least one remote MMP tick,
|
||
which no single control round-trip could survive inside the socket's
|
||
five-second timeout. A probe that runs and finds a problem is not an error
|
||
response: the status is `ok` and the failure sits in the per-stage verdicts,
|
||
and error responses stay reserved for malformed or inadmissible requests. On
|
||
a terminal the stage block is redrawn in place with a running elapsed on
|
||
whichever stage is working; piped or redirected there is no cursor to move,
|
||
so each row prints once, at the moment it settles, and the transcript ends
|
||
up the same block a terminal leaves behind. `--json` emits exactly one
|
||
document at the end, so a script parsing the report does not have to skip
|
||
past progress output.
|
||
|
||
#### Packaging & deployment
|
||
|
||
- A NixOS module and an overlay are exposed from the flake, so a flake consumer
|
||
enables the daemon with one line rather than hand-rolling a systemd unit.
|
||
`overlays.default` adds `pkgs.fips`, and `nixosModules.default` provides
|
||
`services.fips.*`: `enable`, `package`, `configFile`, `openFirewall` (UDP 2121
|
||
and TCP 8443) and `dns.enable`, which routes `.fips` to `[::1]:5354` through
|
||
systemd-resolved declaratively rather than with setup and teardown scripts.
|
||
`packaging/nixos/README.md` documents it with a full consumer `flake.nix`.
|
||
Contributed by Arjen.
|
||
|
||
- `fipsctl address [npub|hostname]` prints a node's `fd00::/8` mesh address and
|
||
nothing else, without contacting the daemon. With no argument it derives the
|
||
local node's address from `fips.key` in the default key directory, falling
|
||
back to the world-readable `fips.pub` beside it; `--key PATH` names a key or
|
||
public key file elsewhere. This lets an installer or image build write a mesh
|
||
address into a config file at a point where no node is running and none can
|
||
be, and keeps the derivation in one place rather than reimplemented by
|
||
whatever needs it.
|
||
|
||
- `packaging/debian/build-deb.sh --features <list>` builds the `.deb` with a
|
||
Cargo feature list, which is how an instrumented package is produced for a
|
||
measurement run. The auto-derived dev Version gains a matching `+<features>`
|
||
marker, so a feature build and a default build of the same commit are no
|
||
longer indistinguishable: without it the two carry byte-identical versions,
|
||
an install of one over the other is an apt no-op, and the running node offers
|
||
no way to tell which one it has. The marker sorts above the unmarked build, so
|
||
installing a feature build is an upgrade and reverting to the default build is
|
||
a downgrade: revert with `dpkg -i` rather than `apt install`. `--features` is
|
||
refused together with `--no-build`, which would stamp the marker onto binaries
|
||
the features never reached.
|
||
|
||
### Changed
|
||
|
||
#### Naming: discovery split into lookup and rendezvous
|
||
|
||
- The mesh-lookup control-metrics family is now emitted under the key
|
||
`lookup` in `fipsctl stats metrics` and `show routing`. The former key
|
||
`discovery` is still emitted as a deprecated alias carrying identical
|
||
counters; update dashboards and alerts to read `lookup`.
|
||
|
||
- The overloaded `node.discovery.*` config table was split into
|
||
`node.lookup.*` (mesh-lookup scalars: `ttl`, `attempt_timeouts_secs`,
|
||
`recent_expiry_secs`, `backoff_base_secs`, `backoff_max_secs`,
|
||
`forward_min_interval_secs`) and `node.rendezvous.*` (peer rendezvous:
|
||
`nostr.*`, `lan.*`). A deployed `node.discovery:` block still loads and is
|
||
folded into the new tables with a one-time deprecation warning; migrate your
|
||
`fips.yaml` to the new keys. This includes the two keys v0.4.2 introduces
|
||
under the old spelling, so an operator upgrading straight from 0.4.2 does not
|
||
have to infer them: `node.discovery.nostr.max_concurrent_offers_per_npub` and
|
||
`node.discovery.nostr.signal_ttl_secs` are now
|
||
`node.rendezvous.nostr.max_concurrent_offers_per_npub` and
|
||
`node.rendezvous.nostr.signal_ttl_secs`.
|
||
|
||
- The Ethernet transport's per-interface `discovery` flag was renamed to
|
||
`listen` (`transports.ethernet.*`) to match the symmetric `announce`
|
||
(transmit) / `listen` (receive) neighbor-beacon vocabulary. The old
|
||
`discovery:` key is still accepted via a serde alias, so deployed configs
|
||
continue to load unchanged; `Config::to_yaml()` re-emits it under the
|
||
canonical `listen:` name. Update your `fips.yaml` to `listen:`.
|
||
|
||
- `fipstop`'s Routing State pane renames its `Discovery Requests` and
|
||
`Discovery Responses` sections to `Lookup Requests` and `Lookup Responses`,
|
||
and reads those counters from the canonical `lookup` key rather than from the
|
||
deprecated `discovery` alias. The counters themselves are unchanged, so an
|
||
operator who knows the pane by its old section labels is reading the same
|
||
numbers under new names.
|
||
|
||
#### Library surface and internals
|
||
|
||
- The protocol layers were restructured into sans-IO cores with the I/O kept in
|
||
a thin shell, and two new crate-root modules, `nostr` and `mdns`, own peer
|
||
rendezvous and LAN discovery. The crate root also gains the `is_punch_packet`
|
||
helper and the `CoordError`, `MtuExceeded`, `COORDS_REQUIRED_SIZE` and
|
||
`MTU_EXCEEDED_SIZE` exports. This is an internal reorganization with no
|
||
operator-visible behaviour change and no wire change: encoded bytes and decode
|
||
decisions are identical. Its two consequences a reader will feel are the
|
||
library surface under Removed below and the tracing targets in the next entry.
|
||
|
||
- Tracing targets follow module paths, so the module relocations in this release
|
||
move them. `fips::discovery::nostr::*` becomes `fips::nostr::*`, mDNS moves to
|
||
`fips::mdns::*`, and the protocol subsystems move under `fips::proto::*`
|
||
(`fips::tree` to `fips::proto::stp`, `fips::bloom` to `fips::proto::bloom`,
|
||
`fips::protocol` to `fips::proto::*`, and the mesh-lookup subsystem from
|
||
`fips::discovery` to `fips::proto::lookup`). An existing `RUST_LOG` filter
|
||
naming an old target still parses and simply stops matching, so the symptom is
|
||
missing log lines rather than an error, and a filter that has gone blind looks
|
||
exactly like a subsystem that has gone quiet. Update `RUST_LOG` filters,
|
||
journal-watch recipes and any log-scraping alert accordingly. The four targets
|
||
named explicitly in source rather than derived from a module path
|
||
(`fips::config`, `fips::instr`, `fips::node::handlers::handshake`,
|
||
`fips::node::handlers::rekey`) are unaffected.
|
||
|
||
#### Node health
|
||
|
||
- Node health is determined at start completion instead of unconditionally
|
||
reaching a single running state. **Zero transports up is now fatal**: the
|
||
node tears down cleanly and the daemon exits with an error, where it
|
||
previously came up and served nothing. Any configured optional child that
|
||
failed to start, meaning a transport beyond the first, Nostr, mDNS, TUN, DNS,
|
||
or a worker pool, leaves the node degraded but serving, with a warning naming
|
||
what failed, and all configured children up is full health. A child the node
|
||
was never asked to run does not count against it. The published node state
|
||
gains `Degraded` and `Failed`, both visible via control queries, with
|
||
degraded operational and failed not. Exit detection for the DNS task, the two
|
||
TUN threads and mDNS also re-evaluates health at runtime, so a child that dies
|
||
after a healthy start now shows as degraded. For Nostr the watched children
|
||
are the three service loops that cannot return by design (the inbound notify
|
||
loop, the advert publisher and the refresh ticker); `connect_task` and
|
||
`relay_startup_task` are deliberately not watched, because `Client::connect()`
|
||
returns as soon as it has spawned the per-relay background tasks, so a
|
||
finished handle there carries no health signal. Transports and worker pools
|
||
expose no runtime-exit signal yet and are unchanged.
|
||
|
||
#### Peer handshake
|
||
|
||
- `node.rate_limit.handshake_resend_interval_ms` no longer governs the **first**
|
||
outbound handshake resend. When msg1 is sent, the peer state machine arms the
|
||
first retransmit deadline from a hardcoded 1000 ms constant
|
||
(`HANDSHAKE_RETRANSMIT_INTERVAL_MS` in `src/peer/machine.rs`), because the
|
||
machine-armed timer rather than the connection's stored deadline is now the
|
||
due signal the retransmit driver fires on. The key still governs the second
|
||
and later resends, alongside `node.rate_limit.handshake_resend_backoff` and
|
||
`node.rate_limit.handshake_max_resends`. The constant equals the shipped
|
||
default of 1000, so a deployment that never overrode the key sees no change;
|
||
one that raised or lowered it will find the first resend still firing at
|
||
1000 ms. The limitation is recorded at the site in the source.
|
||
|
||
#### Data-plane / transports
|
||
|
||
- Connected UDP peer drains now batch macOS receives with `recvmsg_x(2)`,
|
||
matching the wildcard UDP receive path instead of issuing one `recv(2)`
|
||
syscall per queued datagram.
|
||
|
||
- Routing next-hop selection now visits borrowed peers and coordinates instead
|
||
of allocating candidate snapshots for each forwarded packet.
|
||
|
||
- A connected UDP socket that cannot open now names the syscall that failed and
|
||
the address it was operating on, the local address for `bind` and the peer
|
||
address for `connect`. Both paths previously returned a bare OS error that
|
||
the caller wrapped identically, so a field report of `Address already in use`
|
||
could not be attributed to either, and the two have entirely different
|
||
causes: on Linux a UDP `connect(2)` to a 4-tuple another socket already holds
|
||
returns `EADDRINUSE`, which is not the same fault as `bind` refusing the
|
||
local address. A node at roughly 245 peers was emitting this three times a
|
||
second across nine peers with no way to diagnose it.
|
||
|
||
#### Observability
|
||
|
||
- The warning raised when a lookup response carries a path MTU below the
|
||
actionable floor now names the request it refused, as a `request_id` field on
|
||
the log line. Every other warning that handler emits already carried the
|
||
correlator, and this one could not: it is raised while the cached-coordinates
|
||
effect is applied, after the response itself has gone out of scope. An
|
||
operator reading the refusal therefore had a counter and a peer name but no
|
||
way to tie the line to a particular exchange, or to the sibling lines logged
|
||
for it. **Only the log line changes**: the response is still accepted, the
|
||
coordinates are still cached, the sub-floor path MTU is still discarded, and
|
||
the same counter is still charged.
|
||
|
||
#### Docs & contributor tooling
|
||
|
||
- Source comments and doc comments were corrected across the modules this
|
||
release restructured. Unresolvable internal references and planning-phase
|
||
locators were removed from comments that exist only on this line, stale
|
||
view-trait documentation was rewritten to describe the shape the restructuring
|
||
produced, and stale liveness claims in the peer machine, executor and timer
|
||
documentation were corrected. No code changed.
|
||
|
||
#### Packaging & deployment
|
||
|
||
- Every GitHub Actions reference in the workflows and composite actions that
|
||
exist only on this line is now pinned to a full commit SHA, so the whole
|
||
`.github` tree is pinned or explicitly justified. The branch carries 75
|
||
action references, of which 71 are pinned to a 40-character commit SHA with
|
||
the mandatory trailing version comment and 4 are left on mutable tags by
|
||
explicit allowance (`dtolnay/rust-toolchain@nightly` and three
|
||
`taiki-e/install-action@nextest`); `testing/check-action-pins.sh` reports the
|
||
same 75 and passes. Nine of those references were pinned here, in
|
||
`package-freebsd.yml` and an `android-check` job block, neither of which
|
||
exists on the maintenance branch: the original pinning sweep was authored
|
||
there, never saw these files, and they arrived through the merge still on
|
||
mutable tags, at which point the guard that shipped with the sweep failed the
|
||
branch exactly as intended. The general lesson is worth keeping with the
|
||
guard: a checker authored on the earliest branch is only as complete as that
|
||
branch's file set, and merging it upward gates files it has never swept.
|
||
|
||
### Deprecated
|
||
|
||
- The `discovery` metric-family key (control-socket JSON), renamed to `lookup`.
|
||
It is dual-emitted alongside the new `lookup` key during a migration window
|
||
and will be removed.
|
||
Migrate dashboards/alerts from `discovery.*` to `lookup.*`.
|
||
- The `node.discovery.*` config table. Its keys were split into `node.lookup.*`
|
||
(mesh-lookup) and `node.rendezvous.*` (peer rendezvous). A legacy
|
||
`node.discovery:` block still applies for now with a deprecation warning and
|
||
will be removed; migrate to `node.lookup.*` / `node.rendezvous.*`.
|
||
- The Ethernet `transports.ethernet.discovery` flag, renamed to
|
||
`transports.ethernet.listen`. The old key is still accepted via a serde
|
||
alias and will be removed at the v2 cutover; migrate to `listen`.
|
||
|
||
### Removed
|
||
|
||
- **Source-breaking for consumers of the library crate**: the crate-root modules
|
||
`bloom`, `discovery`, `mmp`, `protocol` and `tree` are gone, along with the
|
||
crate-root `HandshakeState`, `PeerConnection`, `PeerSlot` and `ProtocolError`
|
||
re-exports. The protocol cores moved into an internal `proto` module and are
|
||
reached through crate-root re-exports instead: tree types through
|
||
`proto::stp`, bloom types through `proto::bloom`, the FSP, STP, lookup,
|
||
routing and FMP wire types through their matching `proto::*` submodules, and
|
||
`PromotionResult` / `cross_connection_winner` through `proto::fmp` rather
|
||
than `peer`. **The `HandshakeState` removed here is the peer connection-phase
|
||
enum, not the Noise handshake type of the same name.** The Noise type is
|
||
untouched, still lives at `fips::noise::HandshakeState`, and it is that type,
|
||
not this one, that the `[0.4.2]` entry below about four public types gaining
|
||
`Drop` describes. `ProtocolError` is replaced by `fips::Error`, whose
|
||
`Malformed` variant now carries a `&'static str` rather than a `String` and
|
||
which gained `BadSizeClass`, `BadCoord` and `BadBloom` variants, so the
|
||
diagnostic text changed with it. `PeerSlot` and the `PeerConnection` resend
|
||
API were unused and are deleted. `Node::connections()` is now `pub(crate)`
|
||
and yields the internal peer machine rather than a `PeerConnection`; a
|
||
consumer that walked links through it should use `Node::peers()`,
|
||
`Node::get_peer()` and `Node::peer_count()` over `ActivePeer`, which remain
|
||
public. Nothing about the behaviour of the shipped binaries changes, and
|
||
nothing on the wire changes: encoded bytes and decode decisions are identical.
|
||
|
||
### Fixed
|
||
|
||
#### Peer and link state
|
||
|
||
- A peer that goes away no longer leaves its path-MTU state behind. The record
|
||
of which transport link-seeded a destination's path MTU had no removal site
|
||
on any lifecycle, so the map grew one entry per peer this node had ever
|
||
linked with and held them for the life of the process. Both the stored value
|
||
and its seeding record are now released when the path they describe goes,
|
||
together rather than separately: releasing the record alone would leave the
|
||
value with nothing recording where it came from, and a peer returning over a
|
||
wider transport would then be refused by the never-loosen rule indefinitely.
|
||
A peer whose link is still up has both written straight back by the reseed
|
||
that already follows.
|
||
|
||
- Removing a link now clears every reverse-lookup key it was inserted under.
|
||
The removal rebuilt one key from the link's own remote address, so an entry
|
||
inserted under a different address form — which the cross-connection
|
||
promotion arms do, keying on the packet's source address — outlived the link
|
||
it named. An entry a newer link has since claimed is still left alone.
|
||
|
||
- A rejected handshake no longer strands a session index or a retry schedule.
|
||
Two reject arms freed neither, which the `next`-line versions already do;
|
||
master now carries the same shape, so an ACL-rejected outbound dial is
|
||
rescheduled and an abandoned rekey index is returned rather than held.
|
||
|
||
#### Transport
|
||
|
||
- The UDP listen socket's address-reuse flags are now set after its bind rather
|
||
than before. Before the bind they mean the kernel may hand back a port another
|
||
flagged socket already holds, so a second daemon binding the same configured
|
||
address started silently and shared the port, with the kernel splitting
|
||
inbound datagrams across the two receive loops on the source 4-tuple, where
|
||
the second daemon should have failed with `EADDRINUSE`. After the bind they
|
||
mean what was actually wanted, that the per-peer connected sockets may later
|
||
join the port. Those connected sockets are the joiners and keep their flags
|
||
before their own bind, which is where they belong.
|
||
|
||
- A BLE connection the pool refused is no longer counted as an established
|
||
link. The scan and probe loop recorded `connections_established`, resolved
|
||
the address out of the retry book and handed the peer up to the node layer
|
||
after `ConnectionPool::insert` had already refused the connection and
|
||
dropped it. Reaching the refusal needs a full pool with no evictable slot,
|
||
and every BLE connection is built non-static, so only `max_connections: 0`
|
||
gets there.
|
||
|
||
#### Control socket
|
||
|
||
- `disconnect` on the control socket now closes the transport connection
|
||
rather than only the peer. It notified the peer and freed every node-side
|
||
structure, sessions, indices, links, address mapping, tree and bloom state,
|
||
and never touched the transport, so on a connection-oriented transport (TCP,
|
||
Tor, Nym, BLE) the pool entry, the socket and its inbound-slot accounting
|
||
survived the peer the node had just forgotten, until the far end closed or
|
||
the receive loop errored. An operator who disconnected a peer to free a slot
|
||
did not free the slot. No effect on UDP, Ethernet or loopback, whose
|
||
`close_connection` is the connectionless no-op. Still not addressed:
|
||
`disconnect` reports `peer not found` for an identity that is only
|
||
mid-handshake, so withdrawing a peer during its handshake leaves that leg
|
||
resending msg1 until the handshake timeout bounds it.
|
||
|
||
- `connect` on the control socket now tries the address it was given for a
|
||
peer the node is already connected to, instead of reporting success without
|
||
doing anything. The command built an ephemeral peer configuration and handed
|
||
it to the ordinary dial path, which returns success the moment the peer is
|
||
already held, so `fipsctl connect` printed success and the node never
|
||
attempted the path. An operator moving a peer onto a freshly provisioned
|
||
link, or a supervising process that has just seen a second path come up, had
|
||
no way to make the node use it: the peer stayed where it first authenticated
|
||
until that path died. The address is now tried as an alternate path
|
||
alongside the live one, through the same helper a runtime peer refresh uses,
|
||
so promotion happens only after the alternate handshake authenticates and a
|
||
wrong address cannot displace a healthy link. The response gains an additive
|
||
`refreshed` field distinguishing "started an alternate-path handshake" from
|
||
"already on this exact path and it is fresh". `connect` stays ephemeral: the
|
||
peer is not written to configuration and gets no auto-reconnect.
|
||
|
||
- A failed probe now names the condition it actually hit. Probing an absent
|
||
key reported one of two different failures depending on whether a bloom
|
||
filter had returned a false positive, and the reason shown was wrong in the
|
||
branch that gets likelier as the mesh grows: the verdict was right, the
|
||
explanation was not. The core already held the fact and discarded it, since
|
||
the lookup gate proceeds only when a peer's filter claimed the address, so a
|
||
sent lookup proves the claim; that is now its own failure kind, carried to
|
||
the control socket through the existing reason field. `fipsctl` renders the
|
||
two differently and closes with a note naming the false-positive mechanism,
|
||
which is the part that helps, because the reason string alone does not tell
|
||
an operator that a filter cannot miss a key that is present. The
|
||
seventeen-second wait is unchanged, being the retry ladder running to
|
||
completion; what changes is that the operator is told the wait carried no
|
||
information rather than given a wrong reason for it. Still not addressed:
|
||
naming which peer's filter made the claim.
|
||
|
||
#### Data-plane / transports
|
||
|
||
- An inbound onion connection no longer leaks its inbound slot when the peer
|
||
goes away before the accept loop has pooled it. The Tor accept loop spawned
|
||
the per-connection receive task, then inserted the pool entry, then bumped
|
||
the inbound counter; a remote that reset immediately let the receive task
|
||
run its cleanup first, find nothing to remove, skip the teardown that
|
||
decrements, and leave the increment behind for the life of the process. Once
|
||
enough of those accumulated the `max_inbound` gate rejected every further
|
||
onion connection, with the pool visibly empty. The accept loop now holds the
|
||
receive task on a readiness barrier until both the pool entry and the
|
||
counter are in place, as the TCP accept loop already did, and an aborted
|
||
accept still falls through to the cleanup rather than stranding the entry.
|
||
The same accept path also releases the slot of an entry it evicts, which a
|
||
reused ephemeral forward port could otherwise leave orphaned. Outbound Tor
|
||
connections and the whole of the Nym transport are unaffected: neither holds
|
||
a counted inbound slot.
|
||
|
||
- A path MTU measured on one link no longer clamps a peer that has moved to
|
||
another. Every writer of the per-destination path-MTU cache keeps the
|
||
smaller of the existing and incoming value, which is right while a peer
|
||
stays put, but the entry is keyed by destination alone. So a peer first
|
||
reached over a narrow link stayed clamped to that link's ceiling for the
|
||
lifetime of the process: when it later became reachable over a wider
|
||
transport, promotion re-seeded, the seed saw a tighter existing value and
|
||
declined, and traffic kept running at the old link's ceiling on a link that
|
||
could carry far more, with nothing reporting it because the clamp was doing
|
||
exactly what it was told. The node now records which transport last seeded
|
||
each destination and treats a seed from a different one as authoritative
|
||
rather than as a loosening to refuse. Re-seeding the same transport still
|
||
keeps the tighter value, so repeated promotion does not reset discovery, and
|
||
a destination with no prior seed is unchanged.
|
||
|
||
- An inbound IPv6 source address no longer loses its scope. Converting a raw
|
||
`sockaddr` on receive dropped `sin6_scope_id`, and a link-local source
|
||
(`fe80::/10`) identifies a host only together with its interface scope,
|
||
because the same address may exist on several interfaces. The failure was
|
||
silent rather than loud: the address parsed, it looked correct in a log
|
||
line, and only the reply failed. On a link where every address is
|
||
link-local, a Wi-Fi Aware data path for instance, that stalled the Noise
|
||
handshake. msg1 was sent to a scoped address and arrived, the peer replied,
|
||
and msg2's source was recorded with scope 0 and could not be routed back, so
|
||
msg1 retried indefinitely with nothing reported. The conversion is shared by
|
||
all three Unix receive paths, the single-packet `recv_from`, the Linux
|
||
`recvmmsg` batch and the Darwin `recvmsg_x` batch, so one fix covers each.
|
||
Sources outside the link-local range carry scope 0, for which the scoped and
|
||
unscoped forms are identical, so nothing else changes.
|
||
|
||
#### macOS
|
||
|
||
- The packaged macOS daemon recreates and binds its control socket at
|
||
`/var/run/fips/control.sock`. A privileged macOS process selects that private
|
||
runtime path before its leaf exists, so bind creates it and clients follow
|
||
once it is there, and socket setup now changes ownership and mode only for a
|
||
private parent directory it creates or recognizes as a canonical FIPS runtime
|
||
directory. Previously the packaged daemon fell through to the shared
|
||
`/tmp/fips-control.sock` path after every boot, and because socket setup
|
||
changed the parent directory unconditionally, the root daemon also took group
|
||
ownership of `/tmp` itself.
|
||
|
||
#### Packaging & deployment
|
||
|
||
- The FreeBSD package manifest carried the same bounced maintainer address the
|
||
other packaging paths did, so the contact of record in the new `.pkg` artifact
|
||
would have been unreachable at its first release.
|
||
|
||
### Security
|
||
|
||
#### Coordinate cache
|
||
|
||
- The coordinate cache is warmed from plaintext session headers on datagrams a
|
||
node is merely forwarding, and nothing filtered those writes. Two checks now
|
||
run at the write site, and an entry carries its provenance, so a coordinate
|
||
established by a lookup whose proof this node checked is no longer displaced
|
||
by an unauthenticated hint. Four counters report what the checks refuse:
|
||
`coord_warm_foreign_root`, `coord_warm_key_mismatch`, `coord_hint_rejected`
|
||
and `coord_hint_changed`, all in `fipsctl show status` and
|
||
`fipsctl show routing`. **These are mitigations and not a closure.** The
|
||
coordinate is still not authenticated, so a same-root forgery is unaffected
|
||
and hint-over-hint for a destination that was never verified is unchanged.
|
||
Treat the counters as a rate to watch rather than an alarm.
|
||
|
||
#### Tick profiler
|
||
|
||
- The `--dir` given to `profile tick on` is now confined to `/var/log/fips`
|
||
when the daemon runs as root. The control socket is reachable by the `fips`
|
||
group, which the security model writes down as strictly weaker than root, yet
|
||
the directory travelled from the socket straight into a root `create_dir_all`
|
||
with no validation: a group member could create a root-owned directory
|
||
anywhere on the filesystem, including a path a later privileged component
|
||
reads. The path must now be absolute, must contain no `..`, and must still
|
||
resolve under the root once every existing ancestor has been followed through
|
||
its symlinks, so a symlinked parent does not launder a lexically clean path.
|
||
A daemon that is not running as root crosses no such boundary and takes
|
||
`--dir` as given, which keeps the documented non-root `cargo run` capture
|
||
working; the flag can no longer point a root daemon at a different log root,
|
||
which was its other documented use. This only ever affected a
|
||
`--features profiling` build: the subcommand is absent from a stock package.
|
||
|
||
- A capture file is no longer written over whatever is already at its path. The
|
||
name is a one-second UTC stamp and therefore predictable, so `File::create`
|
||
would have followed a symlink pre-planted at the next name, and it truncated
|
||
any real file it found. The file is created only if it does not exist, and a
|
||
capture started in the same second as a previous one takes the next free
|
||
`-N` suffix instead of failing. Capture files are created private to their
|
||
owner and the capture directory is no longer left world-accessible by a
|
||
permissive umask; a capture carries the node npub, build, platform and a
|
||
timing series.
|
||
|
||
## [0.4.2] - 2026-08-25
|
||
|
||
### FMP/FSP sessions and rekey
|
||
|
||
#### Changed
|
||
|
||
- Config validation now rejects two `node.rekey` settings that appear to
|
||
disable the trigger and in fact fire it continuously. `after_messages` of
|
||
zero makes the message-count arm true on every poll, because the trigger
|
||
compares the counter with greater-or-equal. `after_secs` at or below the
|
||
per-session jitter bound is the same trap on the timer arm: each session
|
||
offsets the interval by a random value within plus or minus that bound, so a
|
||
smaller interval saturates to zero on a negative draw and rekeys on sight,
|
||
for roughly half of sessions. Both are checked whether or not rekey is
|
||
enabled, so switching it on later cannot surface the error at a surprising
|
||
moment, and neither gains an upper bound; a very large value remains the
|
||
supported way to disable one arm. A config carrying either setting now fails
|
||
to load instead of starting a node that rekeys constantly.
|
||
|
||
#### Fixed
|
||
|
||
- Inbound session-setup messages are now rate limited, keyed on the
|
||
authenticated link peer the datagram arrived over. The setup path allocated
|
||
a session entry and sent a routed SessionAck for every well-formed message
|
||
naming an address it had no entry for, and that address is an envelope field
|
||
the sender picks, so one neighbour could grow the session table at whatever
|
||
rate it could transmit and buy an ack per entry to a destination of its
|
||
choosing. The limiter sits ahead of every send and both handshake
|
||
constructions in the handler, so a refused message emits nothing and costs
|
||
no cryptography. The key is the link peer rather than the claimed source
|
||
address, which is what makes it a limit at all: keying on the source would
|
||
hand a single sender a fresh full bucket per forged message.
|
||
|
||
Two consequences worth stating rather than discovering. The limiter bounds
|
||
each neighbour's contribution and makes a flood attributable; it does not
|
||
give the node an absolute ceiling, which stays at roughly
|
||
`peers * rate * handshake_timeout_secs`. And a legitimate peer reaching this
|
||
node over the *same* link as an attacker shares that attacker's bucket, so
|
||
establishment behind a flooded neighbour is refused until it refills. Rekey
|
||
and restart traffic is deliberately not subject to that: setup messages
|
||
naming an already-established peer draw on a separate per-link bucket,
|
||
because suppressed key rotation is silent (nothing errors and no session
|
||
drops) and would have shown up only as a flat `rekey_armed`.
|
||
|
||
- A forged SessionAck no longer destroys an in-flight session initiation. The
|
||
handler removed the session entry to take ownership of the handshake state
|
||
and, when the XK msg2 read failed, returned without putting it back. Nothing
|
||
in that message is authenticated (the only thing tying it to the initiation
|
||
is the datagram's source address, which the sender chooses), so any node able
|
||
to reach the victim could cancel any initiation with 57 bytes of the right
|
||
length, and hold establishment down by repeating it. The entry is now kept.
|
||
Keeping it is not enough on its own, and the second half is the part worth
|
||
naming: the msg2 read mixes the sender's ephemeral into the symmetric state
|
||
before it authenticates anything, so an entry put back as the failed read
|
||
left it holds a handshake that can never read the genuine msg2, which trades
|
||
a one-round-trip denial for one lasting the full handshake timeout. The read
|
||
is therefore rolled back to its pre-read state before the entry goes back.
|
||
The three later failure paths in the same handler still drop the entry: each
|
||
is downstream of a msg2 that authenticated, so it is a local failure rather
|
||
than a possible forgery. The entry's activity stamp is deliberately not
|
||
refreshed on the failure path, so a spray cannot hold a dead initiation past
|
||
its original sweep deadline, and a new `ack_handshake_failed` counter makes
|
||
the refusals visible at the default log level. Only the XK handshake on this
|
||
branch is covered; the additional drop sites in the XX handshake on the
|
||
development branch are not.
|
||
|
||
- An unauthenticated session msg3 no longer discards a completed key epoch.
|
||
Five sites discarded the whole rekey, which nulls a `pending` session sitting
|
||
beside the handshake, when only the handshake had failed: the four failure
|
||
paths in the responder-side rekey arm of the msg3 handler, and the
|
||
dual-initiation yield arm in the setup handler, which is gated on a rekey
|
||
being in progress rather than on this node having initiated it, so an entry
|
||
the peer armed reaches it too. That `pending` session is the epoch the real
|
||
peer may already have cut over to, so discarding it kills the reverse
|
||
direction until the session idles out. Two unauthenticated messages reached
|
||
it: a forged setup message arms a handshake beside a completed rekey once
|
||
that rekey has waited a full idle timeout for a peer that never appeared on
|
||
the new epoch, and any garbage msg3 of the right length then finishes the
|
||
job. All five now abandon only the handshake. The four remaining sites in the
|
||
ack initiator arm are deliberately left alone and the reason is recorded
|
||
there: an entry with the initiator flag set holds no pending session, so the
|
||
two calls are the same action at those sites.
|
||
|
||
#### Security
|
||
|
||
- A frame whose counter is `u64::MAX` is now refused by the replay window
|
||
instead of being accepted as a new high-water mark. Accepting it pinned
|
||
`highest` at the ceiling, after which every subsequent counter from that peer
|
||
fell more than a replay window below it and was rejected, wedging that peer's
|
||
own receive path until a rekey replaced the session. The send side already
|
||
refuses to emit that counter (`take_send_counter` and `advance_nonce` both
|
||
return a nonce-overflow error), so no conforming peer can produce it and the
|
||
refusal is invisible on the wire; the highest counter an honest peer can send,
|
||
`u64::MAX - 1`, is still accepted. Reaching this required an
|
||
already-authenticated peer running modified code, and the damage was confined
|
||
to that peer's own session.
|
||
|
||
- The MMP gap tracker advances its expected-counter state with a saturating add,
|
||
so a received counter of `u64::MAX` no longer overflows it. The wrap silently
|
||
reset the expectation to zero in a release build and aborted the task under a
|
||
build with overflow checks on, such as the test harness. Behaviour is
|
||
unchanged for every counter an honest peer can emit.
|
||
|
||
- An epoch-mismatch msg1 no longer tears down a peering that is still
|
||
carrying authenticated traffic, and a second epoch change for the same peer
|
||
identity inside 15 seconds is refused. The epoch travels inside the AEAD, so
|
||
such a msg1 is authentic, but it stays authentic after capture: replaying
|
||
one destroyed a working peering, and with it the FSP session state that
|
||
peering carried, from off the path. The peering's last authenticated inbound
|
||
frame is the evidence that it is still alive, and nothing an unauthenticated
|
||
sender emits can refresh it, so a peer that genuinely restarted clears the
|
||
gate by having stopped sending. The refusal is a silent drop: no msg2 is
|
||
returned, since the stored msg2 is bound to the original msg1's ephemeral
|
||
and answering a sender-chosen address is free amplification. The interval is
|
||
stamped only when an epoch change is accepted, so a sustained replay cannot
|
||
starve a genuinely restarting peer. Both thresholds come from one constant,
|
||
sized so a restarting peer's msg1 resends still land inside its own first
|
||
handshake window and below `link_dead_timeout_secs`, and nothing changes on
|
||
the wire.
|
||
|
||
- Retention of a superseded FSP key epoch is now capped at an absolute
|
||
ceiling measured from the cutover, defaulting to 120 seconds against the
|
||
10-second drain window. The drain deadline slides forward on every inbound
|
||
frame that authenticates against the `previous` slot, which is what keeps a
|
||
peer that lost msg3 from having the old epoch erased out from under it, but
|
||
it also meant the authenticated peer holding that key could keep the retired
|
||
key resident for as long as it kept sealing frames in the old epoch. The
|
||
sliding grace is unchanged; it now delays erasure by a bounded amount rather
|
||
than preventing it. The ceiling is set to clear the worst-case legitimate
|
||
recovery of a peer that lost msg3 (the msg3 resend ladder, then
|
||
`handshake_timeout_secs` before the responder abandons, then the rekey
|
||
dampening window before it may re-initiate, about 90 seconds at stock
|
||
settings), and it is raised automatically if the configured handshake timers
|
||
imply a longer budget, so shortening a timer cannot push the ceiling under
|
||
the recovery it has to leave room for. Nothing changes on the wire; each side
|
||
runs its own drain. A peer that has still not recovered when the ceiling
|
||
fires is left with undecryptable frames until its own rekey retry
|
||
re-converges the epochs, since nothing tears an established session down on
|
||
repeated decrypt failure.
|
||
|
||
- A session setup message naming an already-established peer no longer replaces
|
||
that peer's session. The handler did this whenever `node.rekey.enabled` was
|
||
false: it ran a fresh responder handshake and overwrote the entry, discarding
|
||
the live keys. The message carries no authenticator and its source address is
|
||
an envelope field, so anyone able to reach a node could name an established
|
||
peer and take that session down, repeatedly, and hold it down by repeating
|
||
the message. The established case now always arms the handshake alongside the
|
||
running session and adopts the new keys only after a msg3 whose authenticated
|
||
static key matches the key the session was opened with, which is the check
|
||
the rekey path already applied; a peer that genuinely restarted still
|
||
re-establishes, and a forged setup leaves the session carrying traffic. This
|
||
changes no wire format and adds no configuration: a node with rekey disabled
|
||
already answered such a message, it simply destroyed the session afterwards.
|
||
|
||
- A session rekey armed by a peer's setup message is abandoned if the matching
|
||
msg3 never arrives, rather than persisting for the life of the session, so an
|
||
arming that never completes cannot make the node read a later genuine setup
|
||
message as a simultaneous initiation and drop it. Only the armed handshake
|
||
expires, and only it: a rekey that completed is the key epoch the peer has
|
||
already moved to, since it exists only because a msg3 carrying that peer's
|
||
authenticated key arrived and the sender of that msg3 promotes the new epoch
|
||
on an unconditional two-second timer. Expiring those keys on any timer would
|
||
drop every later frame from that peer, so they are held until the peer's own
|
||
frame promotes them, a newer completed rekey replaces them, or the session
|
||
goes away. What the wait does bound is precedence, not the keys: a completed
|
||
rekey outranks a fresh setup message from that peer only until it has waited a
|
||
full idle timeout, after which the setup is answered normally, so a peer that
|
||
restarted while we held such a session is not refused for as long as our own
|
||
sends keep the session from idling out. The handshake timeout logs at INFO,
|
||
since it costs nothing, and a completed session displaced by a newer one at
|
||
WARN, since that does throw away keys the peer may hold. Session counters
|
||
record the arming of a handshake by a setup message, each of the three ways
|
||
such a message is refused, and each displaced session, so a node under a
|
||
sustained spray of setup messages shows a rate rather than nothing; the
|
||
per-message log lines stay at DEBUG because an unauthenticated sender can
|
||
drive them at line rate. These counters are not yet readable through the
|
||
control socket.
|
||
|
||
- The session drain sweep and the cut-over that retires an old key epoch now
|
||
run whether or not periodic rekey is enabled. Both sat behind the
|
||
periodic-rekey gate, so a node with rekey disabled that adopted new keys held
|
||
the superseded ones for the life of the session.
|
||
|
||
- The FSP session address is now bound to the peer key the Noise handshake
|
||
authenticated, on both the initial and the rekey path. The responder recorded
|
||
a session under the source address carried in the datagram without ever
|
||
checking that address against the static key it had just authenticated, so a
|
||
peer could complete a genuine handshake while claiming another node's
|
||
address, and the identity cache, the session map and the address the IPv6
|
||
shim reconstructs on delivery would all attribute its traffic to the node it
|
||
named. The address is now derived from the authenticated key at the point it
|
||
first becomes available in msg3, and a mismatch drops the half-open session
|
||
without recording either the identity or the session. The rekey responder
|
||
needed its own check: it returns before that code is reached and never read
|
||
the peer's static key at all, so a rekey could complete under an established
|
||
session with a different key than the one that opened it. It now requires the
|
||
key to be unchanged and abandons the rekey while leaving the existing session
|
||
intact, rather than tearing the session down, which would have handed an
|
||
attacker a way to kill established sessions. Both comparisons are on x-only
|
||
keys, because a stored key may carry a synthesized parity while the handshake
|
||
learns the true point. The two rejections are counted separately in the
|
||
session reject statistics.
|
||
|
||
- A link handshake admitted by the established-address waiver is now confirmed
|
||
against the identity that owns that address, instead of on the address alone.
|
||
A transport configured with `accept_connections` false still admits an inbound
|
||
msg1 whose source matches an established peer, so that a peer re-handshaking
|
||
after a restart or a rekey is not locked out, but nothing checked that the
|
||
party sourcing from that address was the peer. Any off-path party able to send
|
||
from it therefore obtained a full link handshake from a node configured to
|
||
accept none. Once the key exchange reveals the initiator's static key, the
|
||
handshake is now dropped unless that key belongs to the identity the matched
|
||
address is attributed to, and dropped as well when the waiver was used and no
|
||
identity owns the address at all, which fails closed rather than skipping the
|
||
check for that case. The cheap refusal is unchanged: a stranger reaching a
|
||
transport that refuses inbound connections is still turned away before any
|
||
cryptography. Attribution consults both the reverse-address lookup and the
|
||
scan over established peers rather than stopping at whichever answers first,
|
||
because the reverse lookup can name a link that no longer exists, and stopping
|
||
there would refuse a peer the scan can still attribute, permanently, since
|
||
the confirmation returns above the code that repairs that lookup. Both
|
||
refusals log at warning level, naming the expected and actual peers, and
|
||
charge the existing handshake bad-state rejection counter rather than one of
|
||
their own.
|
||
|
||
- An inbound frame whose header disagrees with the frame that arrived is now
|
||
dropped, at the single dispatch point every transport converges on, before the
|
||
declared length can be used as a parsing input. The 4-byte common prefix
|
||
carries a payload length that the node never read, so on the datagram
|
||
transports (UDP, Ethernet and BLE, which deliver one whole frame per packet
|
||
and where the arrived length is therefore known exactly) nothing compared the
|
||
two. This closes no known defect, and it is worth being exact about what it
|
||
drops on a deployed line. The stream transports (TCP, Tor, Nym) read their
|
||
frame boundary out of that same field, so the comparison holds by construction
|
||
and never fires for them. A short datagram is a truncated frame, which already
|
||
failed the AEAD tag or the exact-size handshake parse, so what changes there
|
||
is which reason it is dropped for rather than whether it is dropped. A frame
|
||
whose phase the node does not recognize carries no fixed relationship between
|
||
the two and is left alone rather than rejected on a guess. The drop takes its
|
||
own rejection reason and its own `payload_len_mismatch` counter rather than
|
||
reusing the admission one, since it is a framing rejection decided before the
|
||
phase dispatch and it applies to established data frames as well as to
|
||
handshakes; that counter is not yet readable through the control socket.
|
||
|
||
### NAT traversal and Nostr discovery
|
||
|
||
#### Added
|
||
|
||
- `node.discovery.nostr.max_concurrent_offers_per_npub`, defaulting to 4, which
|
||
bounds how many inbound traversal offers one sender npub may have in flight
|
||
at once. It sits inside `max_concurrent_incoming_offers`, which remains the
|
||
outer bound, so a value above that is inert; zero is rejected at config
|
||
validation, since it refuses every inbound offer rather than disabling the
|
||
limit, and so is a value above the maximum permit count a semaphore can
|
||
hold, which would otherwise fail at construction rather than at load.
|
||
Existing configurations parse unchanged, the key being optional.
|
||
|
||
#### Changed
|
||
|
||
- Inbound traversal offers are now admitted against a per-sender allowance as
|
||
well as the global pool. The intake path previously took a permit from a
|
||
single semaphore before any identity check, with the sender's npub used only
|
||
as a log field, so one sender could hold every slot and deny traversal
|
||
onboarding to every other peer for as long as it kept offering. Admission now
|
||
takes a per-npub permit and a global permit together. A sender over its own
|
||
allowance is refused at debug rather than warn, because the party tripping it
|
||
is by definition sending faster than the node wants and a record per
|
||
rejection would turn the spam into log volume; the global bound being reached
|
||
keeps its warn, which is the operator's signal that the node is genuinely
|
||
saturated. **This does not make the pool inexhaustible.** Nostr identities
|
||
cost nothing to generate and the signal subscription carries no author
|
||
restriction, so an attacker running four throwaway npubs still saturates the
|
||
shipped 16-slot pool at an unchanged total offer rate. What the change buys
|
||
is that one identity can no longer do it alone, and that the two refusals are
|
||
distinguishable in the log. The permit is still held across the whole
|
||
attempt; that duration remains inferred from the attempt timeout rather than
|
||
measured.
|
||
|
||
- Config validation now rejects a `node.discovery.nostr.signal_ttl_secs` that
|
||
is too large for the configured `replay_window_secs`. A traversal signal is
|
||
acceptable over its TTL plus 60s of clock-skew grace on each side, and that
|
||
span has to stay strictly inside the replay window, or a session id evicted
|
||
from the replay cache on expiry is still fresh enough to be accepted a second
|
||
time. The relation was documented but unenforced, so raising the TTL past
|
||
180s silently voided it. The bound is derived from the skew constant rather
|
||
than restated, and is checked whether or not nostr discovery is enabled, for
|
||
the same reason the rekey rules are. The shipped defaults (120s against 300s)
|
||
are unaffected, but a configuration that had widened the TTL or narrowed the
|
||
replay window now fails to load, with an error naming the concrete floor for
|
||
`replay_window_secs`. The NAT lab's config generator was one such
|
||
configuration and its generated `replay_window_secs` moves from 60 to 180.
|
||
Note that this covers eviction on expiry only: `seen_sessions_max_entries`
|
||
remains a separate capacity-eviction route that no config relation bounds.
|
||
|
||
- The peer-retry tick no longer awaits the Nostr advert refetch. It ran inline
|
||
on the 1-second rx-loop tick, awaiting a fetch with a 2-second timeout for
|
||
each due peer and discarding the result; with up to sixteen due peers the
|
||
timeouts stacked, and field profiling measured single 2.00 s stalls as the
|
||
common case and a worst tick of 12.4 s against a 1 s period, delaying every
|
||
other rx-loop arm by as much as 4.2 s. The refetch is now spawned, so a dial
|
||
uses the advert cached at that moment and the refreshed one lands for that
|
||
peer's next retry.
|
||
|
||
- The `Adopted NAT traversal socket` log line now carries the transport id and
|
||
the local address alongside the peer npub. Without the local address an
|
||
operator cannot join a host socket table against adoption events, and without
|
||
the transport id several peers sharing one adopted transport are
|
||
indistinguishable from several separate adopted transports.
|
||
|
||
#### Fixed
|
||
|
||
- Nostr NAT traversal signals are now sent only to relays the client pool
|
||
actually holds. A signal is addressed to the merge of the peer's NIP-17 inbox
|
||
relays, the relays its advert nominates for signaling, and our own DM relays,
|
||
but the pool is built once at startup from the configured relays and the send
|
||
is rejected outright, before anything is contacted, if any single URL in that
|
||
list is outside it. One unconfigured relay anywhere in the merge therefore
|
||
killed the whole attempt, including the sends to relays both sides shared. On
|
||
a public node in open mode this made discovery non-functional: 309 traversal
|
||
attempts, 290 explicit failures, zero successes, every failure on `relay not
|
||
found`. Configured peers were unaffected, since they run a matching relay
|
||
set. Comparison is on the normalized relay URL rather than the raw string, so
|
||
a configured relay spelled with a trailing slash or different host case is
|
||
not discarded. Two smaller fixes ride along: the responder resolves its
|
||
relays before binding a socket and running STUN, rather than spending a STUN
|
||
round trip and holding an offer slot only to find it has nowhere to answer,
|
||
and it gained the empty-relay-list guard the initiator already had.
|
||
|
||
- Nostr NAT traversal no longer breaks after the host suspends. The traversal
|
||
clock cached a Unix timestamp once at startup and advanced it with a
|
||
monotonic `Instant`, which does not tick while a machine is asleep, so after
|
||
a suspend the daemon's idea of the time trailed real time by the suspend
|
||
duration for the rest of the process lifetime. Every NIP-40 expiration it
|
||
computed was therefore published already in the past: relays dropped the
|
||
offers as expired, the initiator logged a signal timeout waiting for an
|
||
answer, and traversal stayed broken until the daemon was restarted. The
|
||
clock now reads the wall clock on every call. This is not macOS-specific,
|
||
though a laptop that sleeps is where it is easiest to hit; any host that
|
||
suspends or hibernates was affected. Reported in
|
||
[#128](https://github.com/jmcorgan/fips/issues/128).
|
||
|
||
#### Security
|
||
|
||
- An advert or inbox-relay list returned by a relay is now checked against
|
||
the peer it claims to describe before anything else looks at it. The relay
|
||
pool verifies every event's signature but does not check a reply against
|
||
the request filter, and neither of the two options that would make it do so
|
||
is enabled, so a relay may answer a request for one author's advert with an
|
||
event it signed itself. The stale-advert refetch picked the newest
|
||
`created_at` across everything returned, with no author test, and then wrote
|
||
the result into the advert cache under the requested peer's npub, so a
|
||
single hostile or compromised advert relay could pin an endpoint set of its
|
||
own choosing for that peer. The author test now runs before the timestamp
|
||
contest rather than after, so a future-dated foreign event cannot even
|
||
suppress the genuine advert by winning it. The same filter now applies to
|
||
the inbox-relay lookup, where the omission let an attacker-authored relay
|
||
list steer this node's direct-message and traversal-signal traffic. A
|
||
refetch that comes back with events, none of them signed by the peer, now
|
||
leaves the cached entry alone: that is no evidence the advert was
|
||
withdrawn, and evicting on it would hand the same relay a way to clear the
|
||
cache. A refetch that genuinely comes back empty still evicts.
|
||
|
||
- An advert's `created_at` is now clamped forward to the same 60s of clock
|
||
skew the traversal-signal path already tolerates. An unbounded future
|
||
timestamp bought a cache entry a proportionally distant validity horizon
|
||
and an unbeatable position in every replacement comparison, so a later
|
||
genuine advert could never displace it and the size-cap eviction collected
|
||
it last. The clamp applies to the stored timestamp as well as the validity
|
||
window, at all three points where an advert is cached, so ordering and
|
||
expiry now agree. Clamping rather than refusing the event is deliberate: a
|
||
node whose own clock runs slow reads every peer's honest advert as
|
||
future-dated, and refusing would silently withdraw Nostr-mediated dialing
|
||
for every peer at once.
|
||
|
||
- Inbound traversal signals are now rate limited before they are decrypted. A
|
||
rendezvous-enabled node handed every kind-21059 event straight to the unwrap,
|
||
which is two NIP-44 decrypts and a signature verify, inline on the single
|
||
task that also routes traversal answers and maintains the advert cache.
|
||
Nothing bounded how fast an unauthenticated stranger could schedule that
|
||
work: the per-npub offer admission cannot, because it keys on the sender's
|
||
public key, which only exists once the first decrypt has already run, and
|
||
because it is a concurrency semaphore rather than a limit over time. A token
|
||
bucket now sits ahead of the unwrap, so a flood costs a node its inbound
|
||
offers instead of the whole notify loop.
|
||
|
||
What the limit can and cannot key on is worth stating, because it decides the
|
||
shape of the fix. Before decryption there is no sender identity at all: the
|
||
outer event is signed by a key generated per event, so bucketing on its
|
||
author would hand an attacker a fresh allowance for free, and the timestamp
|
||
and recipient tag are equally attacker-chosen. The arrival relay is drawn
|
||
from our own configured set but is not an isolation boundary either, since an
|
||
attacker publishes to the same relays an honest peer does. The shared
|
||
allowance is therefore a single global bucket and is indiscriminate by
|
||
construction, which on its own would shed our own traversals along with the
|
||
attacker's, and since the attacker sets the rate every retry would land in
|
||
the same shed. A second, smaller allowance is held in reserve and drawn only
|
||
while this node has traversals of its own outstanding, so a flood denies a
|
||
node its inbound offers, which nothing receiver-side can prevent without a
|
||
pre-decrypt identity, rather than also denying it the answers to offers it
|
||
sent. Shed signals are counted and reported at debug level per event with a
|
||
warning each time the running total doubles, so a bucket sized below a busy
|
||
node's real need shows up in the log rather than as apparent relay flakiness.
|
||
|
||
Two limits on what this buys. It bounds the crypto path only: the advert
|
||
branch runs earlier in the same loop and is not metered here, so a stranger
|
||
can still put JSON parsing and a cache insert on the task per event. And the
|
||
relay SDK verifies each event's outer signature on its own per-relay task
|
||
before this loop ever sees it, which no receiver-side change short of
|
||
dropping the subscription can avoid.
|
||
|
||
- A STUN binding response is now accepted only from the address the binding
|
||
request was sent to. The client discarded the source address `recv_from`
|
||
returned and let the parser decide, and the parser checks only the message
|
||
type, the magic cookie and the 12-byte transaction id. An on-path attacker
|
||
who could read the outbound request could therefore inject a reply carrying
|
||
a transaction id copied from it, and its chosen address became the reflexive
|
||
candidate the node published in a traversal offer or answer, redirecting the
|
||
peer's hole-punch packets. Datagrams from any other source are counted and
|
||
discarded, and one debug record per STUN attempt reports the count and the
|
||
last unexpected source, so a rejection is diagnosable without giving a
|
||
flooder control of the log rate. A server that answers from an address other
|
||
than the one dialed, which RFC 5389 forbids, now times out and the next
|
||
configured server is tried.
|
||
|
||
- The exemption that lets a peer's reflexive address skip the private-address
|
||
gate is now conditional on our own vantage point. That exemption exists for
|
||
the deployment whose STUN server sits inside the private network, so the
|
||
observed reflexive address is legitimately private; it was applied
|
||
unconditionally, so a node whose own STUN result was public still punched
|
||
whatever private address a peer named as its reflexive one. Any sender whose
|
||
offer or answer was accepted could therefore aim a burst of UDP packets,
|
||
carrying this node's source address, at a host inside the node's own private
|
||
network, which is the one place the candidate filter was written to keep it
|
||
out of. The gate now applies whenever our own reflexive address is public.
|
||
It stays lifted when our own reflexive address is itself private, which is
|
||
the LAN-STUN deployment the exemption was for, and also when we have no
|
||
reflexive address at all, so a failed STUN probe cannot cost a node its
|
||
same-LAN peering. Two consequences to state rather than discover: a peer
|
||
behind a private STUN server talking to a node with a public one loses its
|
||
reflexive candidate, which was never reachable from us in any case, and
|
||
because the /24 comparison is IPv4-only a unique-local IPv6 reflexive
|
||
address is refused unless our own reflexive address is unique-local too.
|
||
An off-subnet refusal of a peer's reflexive address is a shape an honest
|
||
deployment now produces, so it no longer raises the refusal record to
|
||
warning level on its own; the never-routable, port-0 and unparsable classes
|
||
still do.
|
||
|
||
- A peer's candidate list is now bounded before it is walked rather than only
|
||
after. The eight-target cap ran after both planning loops had finished, so
|
||
it bounded what a node punched but not what it spent deciding: a signal
|
||
naming several thousand candidates had every one of them parsed and vetted,
|
||
and the deduplicating scan that follows is quadratic in the plan those
|
||
candidates feed. At most 32 candidates are now vetted, four times the
|
||
target cap and four times what the candidate generator produces on the
|
||
widest host, and the excess is discarded rather than failing the offer, so
|
||
an honest many-homed peer loses the tail of its list instead of its
|
||
traversal. The refusal record carries the discarded count as a new
|
||
`over_offered` field and treats a non-zero one as an attack shape, since
|
||
nothing honest reaches the bound.
|
||
|
||
- A NAT-punch packet is now accepted only from an address this node planned to
|
||
probe. The punch packet's discriminator is a plain digest of the session id,
|
||
a value both peers already know, and it travels in the clear in every probe,
|
||
so acceptance proved only that the sender had seen one. The receive loop
|
||
broke on the first packet whose digest matched, whatever its source, and
|
||
returned that source as the peer address, so anyone who observed a probe, or
|
||
who could reach the node and guess the session id, could have an arbitrary
|
||
address adopted as the peer: the legitimate traversal was denied, the Noise
|
||
handshake and its retransmissions went to an address of the attacker's
|
||
choosing, and the pair was charged a failure against its backoff state. The
|
||
npub-pinned handshake still could not authenticate to the wrong host, so this
|
||
was a denial and a misdirection rather than an impersonation. The source
|
||
address is now ranked against the planned target list before anything else:
|
||
an unplanned source is dropped and, deliberately, is not acked either, since
|
||
acking it is a reflection the node controls. A source matching a planned
|
||
target exactly is adopted immediately, as before. A source matching a planned
|
||
target's IP on a different port is what a symmetric NAT's fresh mapping looks
|
||
like, and it is still adopted, because that is the main class of NAT pairing
|
||
punching exists to rescue; it is held as a candidate for 250 ms first, so an
|
||
exact match arriving inside that window supersedes it. The honest path's
|
||
latency is unchanged. Two consequences to state rather than discover: an
|
||
attacker that can source packets from a planned target's IP on any port is
|
||
still accepted, which is the residue only an authenticated probe can close;
|
||
and an attempt under a flood of spoofed matching packets now runs to its full
|
||
timeout instead of ending on the first one, so the refused sources are
|
||
counted and reported once when the attempt ends rather than logged per
|
||
packet.
|
||
|
||
- Traversal punch targets taken from a peer's offer or answer are now
|
||
filtered and bounded. A rendezvous-enabled node previously punched every
|
||
address a signed offer named, including loopback, link-local, multicast,
|
||
broadcast, unspecified and CGNAT addresses, and placed no limit on how
|
||
many candidates one offer could carry. Any npub could
|
||
therefore have a node emit a burst of UDP packets at addresses of the
|
||
sender's choosing, carrying the node's own source address. Candidates in
|
||
the never-routable ranges are now rejected, IPv4-mapped IPv6 forms are
|
||
canonicalized before the check so they cannot slip past it, candidates
|
||
with port 0 are dropped, private-range candidates are punched only when
|
||
they share a /24 with one of our own addresses (which is what same-LAN
|
||
traversal already required of its own path), and the planned target list
|
||
is capped at eight. A peer's reflexive address is checked against the
|
||
never-routable ranges but not against the /24 rule, so a deployment whose
|
||
STUN server sits inside the private network keeps working. A malformed
|
||
address in a peer's signal now drops that one candidate instead of
|
||
failing the whole traversal. A node also records what it declined: one
|
||
log record per planning attempt carries how many candidates the peer
|
||
offered, how many were planned, the count refused in each class and one
|
||
sample address, at warning level for the shapes no honest peer produces
|
||
and at debug level for the routine off-subnet case. Same-LAN and
|
||
reflexive traversal are otherwise unaffected.
|
||
|
||
- Traversal offers and answers dated in the future are now rejected. The
|
||
freshness check measured a message's age with a saturating subtraction, which
|
||
yields zero for any timestamp ahead of the local clock, so the age test could
|
||
not fail for a future-dated signal and no other term bounded the issue time
|
||
from above. A signal claiming to be issued arbitrarily far in the future was
|
||
accepted as strictly fresh, which voided the property that the freshness
|
||
window is narrower than the session-id replay window (300s by default) and
|
||
left the replay cache as the sole defence against a captured offer being
|
||
replayed. Forward-dating is now tolerated only up to the same 60s of clock
|
||
skew already allowed in the other direction, and a signal accepted under that
|
||
grace reports the skew outcome, so the existing clock-skew log fires for a
|
||
peer whose clock is ahead just as it does for one whose clock is behind. The
|
||
declared expiry timestamp is also no longer trusted beyond the issue time plus
|
||
the configured TTL, so a sender cannot widen its own acceptance window by
|
||
inflating that field. A single timestamp is now acceptable over at most the
|
||
signalling TTL plus 60s on each side, 240s under the shipped defaults.
|
||
Rejections are also now distinguishable in the log: a stale signal and a
|
||
future-dated one no longer share one reason string, and the inbound-offer
|
||
path, whose only surface was an unattributed debug line below the default log
|
||
level, now names the peer and the session and warns for the rejection classes
|
||
that relay delivery lag cannot produce (future-dated, identity-mismatch and
|
||
malformed offers), leaving an ordinary stale offer quiet. As with the existing
|
||
inbound rate-limit warning, an unauthenticated remote peer can drive that
|
||
line. A failure of our own offer's freshness during answer validation is
|
||
reported against the offer rather than mislabelled as the answer's, and the
|
||
tolerated-acceptance log now carries the issue and expiry stamps and no longer
|
||
attributes the acceptance to clock skew, since a peer configured with a longer
|
||
signalling TTL than ours now reaches it too.
|
||
|
||
### Data plane, routing signals and metrics
|
||
|
||
#### Changed
|
||
|
||
- Peer bloom filters are computed for every recipient in one prefix and suffix
|
||
union sweep rather than rebuilt per recipient. Announcing to R peers
|
||
previously did R full map builds and R by T merges; at 240 peers that was
|
||
20.6 ms per tick, roughly half the tick body, with a median per-interval
|
||
maximum of 34.5 ms. The result is exactly equal rather than approximately:
|
||
merging is a bytewise OR, so regrouping the unions cannot change it. The
|
||
trade-off, measured rather than assumed, is that the sweep does its full work
|
||
regardless of how many peers are ready, so a tick announcing to one or two
|
||
peers now costs about twice what it did; break-even is around three ready
|
||
peers. Cadence, the debounce, the sequence rule and the fill-ratio cap are
|
||
unchanged.
|
||
|
||
- Each peer's npub is derived once at construction instead of once per tick.
|
||
The per-tick stats snapshot ran a bech32 encode for every tracked peer, and a
|
||
second one for the common peer with no hosts-file entry and no alias, since
|
||
the display-name fallback bottoms out in the same encode: 14.1 ms per tick at
|
||
240 peers. The display name itself is deliberately not cached, because the
|
||
alias map and the host map both mutate at runtime.
|
||
|
||
#### Fixed
|
||
|
||
- A SessionDatagram carrying a truncated inner FSP payload no longer panics the
|
||
forwarding path. The coordinate-cache warm path sliced the inner payload at
|
||
the full 12-byte header offset while guarding only with the 4-byte common
|
||
prefix parser, so an inner payload of 4 to 11 bytes with phase 0x0 and the
|
||
Coords Present flag set indexed past the end of the slice. Because the
|
||
receive loop is the process's main future, the panic terminated the daemon
|
||
rather than a task, and under the packaged systemd unit the node restarted
|
||
into the same frame. The warm path now applies the same
|
||
`FspEncryptedHeader` guard the local-delivery path already used, which
|
||
additionally means a malformed frame carrying a non-zero protocol version or
|
||
the Unencrypted flag alongside Coords Present is dropped rather than having
|
||
its body read as coordinates. Any peer that had completed a link handshake
|
||
could trigger this, and admission is default-open. Frames rejected by that
|
||
guard are now counted in the forwarding statistics as
|
||
`warm_malformed_packets` and `warm_malformed_bytes`, the byte counter
|
||
charging the whole outer frame, visible over the control socket and on the
|
||
fipstop Routing State pane, so a node being fed malformed frames is
|
||
distinguishable from a quiet one at the default log level. The count is not a
|
||
packet drop: the frame is still delivered or forwarded, and only the
|
||
coordinate-cache warm attempt is abandoned. The existing debug log now also
|
||
carries the frame's protocol version and flags, which separate a short frame
|
||
from a bad-version or Unencrypted-flagged one.
|
||
|
||
- `SessionDatagram` hop-limit handling now follows IP semantics. Delivery to
|
||
the addressed node is no longer TTL-gated, and a forwarder decrements before
|
||
deciding rather than after, so a datagram that would leave with a TTL of zero
|
||
is dropped instead of transmitted. Previously the TTL check ran ahead of the
|
||
local-delivery test, so a datagram addressed to this node that arrived with
|
||
TTL 0 was dropped, and a forwarder receiving a transit datagram at TTL 1
|
||
transmitted it at TTL 0 for the next hop to discard, wasting one transmission
|
||
per expiring datagram. `SessionDatagram::decrement_ttl` and
|
||
`SessionDatagram::can_forward` were aligned to the same semantics:
|
||
`decrement_ttl` decrements first and reports false when the result is zero,
|
||
and `can_forward` is true only at a TTL of 2 or more. The reachable radius is
|
||
unchanged, because the two behaviors compensated exactly: a path of `h` links
|
||
still delivers for any source TTL of `h` or more. During a rolling upgrade, an
|
||
unupgraded forwarder feeding an upgraded destination delivers one hop further
|
||
than either version does on its own; no version mix delivers less far. The
|
||
`TtlExhausted` reject counter now charges at the node that makes the decision
|
||
rather than at the hop after it.
|
||
|
||
#### Security
|
||
|
||
- A transit node's induced routing errors are now bounded by the authenticated
|
||
link peer that induced them. The 100 ms suppression gate on
|
||
`CoordsRequired`, `PathBroken` and `MtuExceeded` was keyed on the failed
|
||
datagram's destination address, which is an envelope field the sender picks,
|
||
so a fresh random destination on every packet was always a first sighting and
|
||
every packet was admitted. Each admission also inserted a key and then walked
|
||
the whole map, so per-packet cost grew with the flood rate while the sender's
|
||
cost stayed flat, and the error itself is addressed to the datagram's source
|
||
address, which nothing binds to the sender either. A new per-link-peer token
|
||
bucket, 20 signals a second sustained with a burst of 50, is now consulted
|
||
first, keyed on the AEAD-authenticated peer the frame arrived over: the one
|
||
value at that point a sender cannot mint. The per-destination interval is
|
||
kept unchanged behind it, because it still does the aggregate suppression a
|
||
genuine outage needs, and no gate was added on the address the error is
|
||
returned to, which would have handed a sender a way to silence honest errors
|
||
toward a victim it names by keeping that victim's key hot.
|
||
|
||
Two ordering choices in there rather than left to be discovered. The peer's
|
||
token is peeked and only spent once the destination gate has also admitted,
|
||
so a single unroutable destination behind a high-fanout peer cannot burn that
|
||
peer's whole budget on signals nothing sends and silence every other
|
||
destination behind it. And the destination map now carries a hard ceiling of
|
||
4096 entries with its expiry sweep amortized to once per eviction interval
|
||
rather than run on every admission; when it is full it admits without
|
||
recording rather than refusing, because refusing would turn a full map into
|
||
node-wide silence exactly during partition healing, when many destinations
|
||
are legitimately unroutable at once. Emission stays bounded by the peer
|
||
budget in that state. Three counters, rendered on the fipstop Routing tab,
|
||
make each of the three outcomes visible instead of silent.
|
||
|
||
- A transit-emitted `PathBroken` no longer carries the reporter's cached
|
||
coordinates for the unreachable destination. The signal is returned to the
|
||
datagram's source address, so anyone able to reach the node could name any
|
||
address and have the node's coordinate cache read back to them, one entry per
|
||
packet. The field is optional on the wire and no receiver reads it, so this
|
||
is an emission change only: an unmodified peer parses the frame exactly as
|
||
before. Which of the two signals is emitted still discloses whether the entry
|
||
exists.
|
||
|
||
- A reactive `MtuExceeded` is now believed only when this node has actually
|
||
sent a frame larger than the bottleneck it reports. The signal is
|
||
unauthenticated: the admission gate narrows which destination may be named
|
||
but cannot say who named it, so a value at the floor was a legal value from
|
||
anyone, and one datagram drove a bound session's path MTU to 256 and pinned
|
||
the address-keyed entry the SYN-time MSS clamp reads, with recovery costing
|
||
three consecutive higher notifications across two notification intervals.
|
||
Each session now carries the largest frame this node has put on the wire
|
||
toward it since the last accepted decrease, and a report is refused unless it
|
||
names something smaller. Honest path-MTU discovery satisfies that by
|
||
construction, because the report exists only because a frame we sent did not
|
||
fit; a forgery has to wait for us to emit something bigger than the value it
|
||
wants to claim, which bounds every accepted claim from below by our own
|
||
traffic. The evidence is cleared on each accepted decrease and on release, so
|
||
one large send early in a session cannot vouch for the rest of it.
|
||
|
||
The guard sits ahead of both effects rather than between them, which is also
|
||
where the existing floor check moved to: the floor previously ran after the
|
||
session's own path MTU had already been changed and so governed only the
|
||
lookup table. The reactive carrier now names its own floor constant, held
|
||
equal to the actionable floor so no hop legitimately configured with a small
|
||
transport MTU loses its feedback; corroboration, not the floor's value, is
|
||
what stops a legal-but-forged claim. A separate counter, rendered on the
|
||
fipstop Routing tab, distinguishes an uncorroborated refusal from a
|
||
below-floor one.
|
||
|
||
- The path-MTU release a `PathBroken` drives is now rate limited per
|
||
destination on a budget of its own. That signal is unauthenticated too, and
|
||
the release discards a bottleneck this node learned by having a packet
|
||
dropped, so repeating the claim discarded a genuine value as fast as it could
|
||
be relearned. The limiter is a separate instance rather than the one the
|
||
coordinate warmup send already uses: a budget another signal can spend is not
|
||
a bound. Deferring a release is the safe direction, since the value kept is
|
||
the tighter one.
|
||
|
||
- The influence a remote party has over path MTU is now bounded, and the
|
||
per-destination path MTU cache has a way back. The `path_mtu` field is an
|
||
unsigned per-hop transit annotation carried outside the signed proof, and the
|
||
`MtuExceeded` and `PathBroken` signals arrive unencrypted with no sender
|
||
check, so any forwarder, or anyone who can reach the node, could lower it,
|
||
and it was accepted with no minimum. A single `MtuExceeded` carrying a very
|
||
small value drove a session's path MTU to zero, after which every packet to
|
||
that destination was answered with an ICMPv6 Packet Too Big instead of being
|
||
sent: a blackhole that lasted until the daemon restarted. The same value
|
||
reached the SYN-time TCP MSS clamp, where anything at or below 137 saturates
|
||
to a segment size of zero and the band just above it yields single digits.
|
||
Values below an actionable minimum are now ignored rather than applied or
|
||
stored, at the three places a remote value is acted on: the path MTU state
|
||
machine, the reactive `MtuExceeded` write, and the discovery response, whose
|
||
coordinates are still cached so refusing the annotation cannot become a way
|
||
to deny discovery. The MSS clamp additionally refuses to write a zero. Each
|
||
of the three refusals logs a warning and increments its own counter in the
|
||
error-signal family, so an operator can tell them apart without scraping
|
||
logs: they carry different meanings, one being an authenticated peer inside
|
||
an established session, one an unencrypted signal anyone able to reach the
|
||
node can send at will, and one a verified discovery response whose unsigned
|
||
annotation a forwarder on the reverse path rewrote. Because those three
|
||
refusals are the only way a remote value reaches the per-destination store,
|
||
the SYN-time clamp does not apply the minimum a second time when it reads
|
||
that store: a small value there is one the node derived from its own outgoing
|
||
link, which is exact rather than suspect, and BLE in particular negotiates a
|
||
link MTU per connection that lands under the minimum routinely. The clamp
|
||
refuses only a stored value admitting no TCP payload byte at all, at 137 or
|
||
below, where the segment size saturates to zero and the clamp would be
|
||
skipped entirely; it logs that at trace rather than warn, since it sits on
|
||
the per-packet path, and the peer's link promotion reports it once instead.
|
||
A stored per-destination path MTU is released when the path is invalidated by
|
||
a `PathBroken` report, by session idle expiry, or by handshake timeout, and
|
||
the link MTU read from the local transport is reseeded in its place, so a
|
||
directly connected peer does not lose its own measurement along with the
|
||
remote claim. The release also resets the session's own current path MTU
|
||
alongside the address-keyed entry, rather than reseeding the link value and
|
||
leaving the tightened one in place, so a path declared dead recovers at once
|
||
instead of only through the increase ladder. Entries written by the discovery
|
||
lookup carrier carry their own deadline and age out, since a destination this
|
||
node never opens a session with reaches none of the three release routes:
|
||
without that, a single response carrying a floor value pinned that
|
||
destination's clamp until restart, and an unknown request id still classifies
|
||
as originator, so a captured response could be replayed indefinitely. The
|
||
notification mirror deliberately carries no deadline. Locally derived MTUs
|
||
are not subject to the minimum, at the seed or at the clamp. Legitimate
|
||
narrow paths are unaffected: adaptation to hops well below the IPv6 minimum,
|
||
which the mesh does use, continues to work.
|
||
|
||
- The three routing signals (`CoordsRequired`, `PathBroken`, `MtuExceeded`) are
|
||
no longer acted on unless this node has itself bound the destination address
|
||
they name, either by initiating a session toward it or by completing the
|
||
Noise handshake that binds an address to a peer's static key. These signals
|
||
carry no end-to-end authentication, so until now any admitted mesh member
|
||
could send one naming any address and have its effects applied: a path-MTU
|
||
clamp written for an arbitrary address, a cached-coordinate flush for an
|
||
arbitrary address, and a discovery and warmup cycle for an arbitrary address.
|
||
The `MtuExceeded` case was the sharpest, because its write into the
|
||
address-keyed path-MTU lookup that the TUN reader consults at TCP MSS clamp
|
||
time sat outside the session guard and so required no session, no peer
|
||
relationship and no prior state at all. A half-open session created by an
|
||
inbound handshake that has not yet proved its address does not admit these
|
||
signals, so a forged session opening cannot be used to unlock them. Signals
|
||
from a genuine on-path forwarder are unaffected: the reporter may be any node
|
||
at any distance. This does not make the sender authentic, which nothing
|
||
short of a wire format change can do. Rejected signals are counted as
|
||
unknown-session rejections, and additionally on four new error-signal
|
||
counters visible through `show routing`, `show metrics` and the fipstop
|
||
routing pane: `unbound_coords`, `unbound_broken` and `unbound_mtu` give the
|
||
refused count per signal type, against the existing per-type arrival
|
||
counters as the denominator, and `unbound_forged` counts the subset whose
|
||
claimed source and destination pairing no honest forwarder could produce.
|
||
The drop log line now carries the signal type and the refusal class.
|
||
|
||
- A discovery lookup response is now acted on only when it answers a lookup
|
||
this node actually has outstanding. The originator path took any response
|
||
whose `request_id` was not in the transit dedup map, so an admitted peer
|
||
could harvest one genuine signed response for a target and re-inject it at
|
||
will: each injection cleared the victim's in-flight lookup, recorded a
|
||
reachability success for a target that might be unreachable, refreshed the
|
||
cached coordinates for a further full TTL, and flushed the victim's queued
|
||
packets onto a route at a moment the sender chose. It also reached the
|
||
signature verify before any check that the response was wanted, so the
|
||
verify was the first cost gate on the path. The node now records the
|
||
`request_id` of every lookup request it sends on that target's pending
|
||
entry, and a response is dropped unless it names a target with a lookup
|
||
outstanding and carries one of the ids issued for it. Because the id is
|
||
fresh 64-bit randomness drawn per attempt and the target signs over it, a
|
||
harvested response is bound to the request it answered and cannot be
|
||
redirected or replayed. The check runs before the identity-cache resolve
|
||
and before the signature verify, so a response nobody asked for costs
|
||
nothing. Replies to earlier attempts of a still-outstanding lookup are
|
||
still accepted, which is the common case on a link whose round trip
|
||
exceeds the first rung of the retry ladder. Drops are counted as
|
||
`resp_unsolicited`, visible through `show routing`, `show metrics` and the
|
||
fipstop routing pane; the counter has a nonzero floor in healthy operation,
|
||
because a request is flooded to every qualifying tree peer and the
|
||
duplicate replies land there once the first has been accepted.
|
||
|
||
- A flooded discovery dedup cache no longer makes a node unresolvable. The
|
||
cache is both the duplicate filter and the reverse-path table for lookup
|
||
responses, and at its 4096-entry bound it dropped the arriving request.
|
||
That drop sat ahead of both the check for whether the request names this
|
||
node and the forwarding path, so one link peer emitting fresh request_ids
|
||
could stop the node answering lookups for itself and stop it carrying
|
||
anyone else's, for as long as it kept the cache full. The cache now makes
|
||
room instead of refusing: over a peer's own share it drops that peer's
|
||
oldest entry, and at global capacity it drops the oldest entry of whichever
|
||
peer holds the most, so a light peer's reverse path is never taken to admit
|
||
a heavy one and extra identities buy a flooder proportionally less. A
|
||
peer's share is the cache divided by the current link-peer count, with a
|
||
floor of 64. The loosening this accepts is that an evicted request_id
|
||
arriving again inside the window is forwarded a second time rather than
|
||
recognised as a duplicate, which the per-target forward limiter and TTL
|
||
already bound. Evictions are counted as `req_dedup_evicted`; the old
|
||
`req_dedup_cache_full` counter stays in place, frozen at zero, so a
|
||
dashboard carried across versions does not lose the series.
|
||
|
||
- Answering a lookup for ourselves is now metered per link peer. The response
|
||
proof is signed over the requester's `request_id`, so every request
|
||
addressed to this node costs a fresh Schnorr signature that cannot be
|
||
cached or served twice, and until now the only thing bounding that rate was
|
||
the dedup cache filling up, which is the defect above. A token bucket per
|
||
link peer, 256 signatures of burst refilling at 32 per second, absorbs the
|
||
legitimate burst that follows a topology change, when many correspondents
|
||
re-look-up at once through the few links that lead here, while capping what
|
||
one neighbour can make the node sign. Refusals are counted as
|
||
`req_sign_rate_limited` and visible in `show routing`, `show metrics` and
|
||
the fipstop routing pane. A refused request keeps its dedup entry, and
|
||
retries carry fresh request_ids, so a refusal cannot suppress the retry.
|
||
|
||
### Admission, rate limiting and peer caps
|
||
|
||
#### Added
|
||
|
||
- `node.rate_limit.session_setup_burst` (64) and
|
||
`node.rate_limit.session_setup_rate` (16.0), the parameters of the new
|
||
per-link-peer session-setup limiter. This is the FSP session-setup bucket,
|
||
and it is distinct from the link-layer msg1 bucket described below; the two
|
||
meter different messages and are sized independently. Setup messages naming
|
||
a peer this node is already established with are metered on a second
|
||
per-link bucket derived from `node.limits.max_peers`,
|
||
`node.rekey.after_secs` and `node.rate_limit.handshake_max_resends`, so
|
||
raising the peer limit sizes it automatically. A zero burst or a
|
||
non-positive rate is rejected at config validation rather than silently
|
||
refusing every session.
|
||
|
||
- `node.limits.max_sessions`, defaulting to 1024, which bounds the end-to-end
|
||
session table. Zero means unlimited, which restores the previous behaviour
|
||
exactly and is the way to back the change out on a running node. The default
|
||
is four times the adjacent `node.session.pending_max_destinations`. A
|
||
session entry measures 6608 bytes of inline state plus heap, so the table
|
||
holds to roughly 7 MB, and a test pins that per-entry figure so the
|
||
arithmetic behind the default fails loudly if an entry grows. Existing
|
||
configurations parse unchanged, the key being optional.
|
||
|
||
- `node.rate_limit.established_handshake_burst` and
|
||
`node.rate_limit.established_handshake_rate`, the parameters of the new
|
||
established-link msg1 token bucket, which meters link-layer msg1 rather
|
||
than FSP session setup. Both are optional; omitting them (the normal case)
|
||
derives the bucket from `node.limits.max_peers`, `node.rekey.after_secs`
|
||
and `node.rate_limit.handshake_max_resends`, so raising the peer limit
|
||
sizes the bucket automatically. An explicit zero burst or a non-positive
|
||
rate is rejected at config validation rather than silently refusing all
|
||
rekey traffic.
|
||
|
||
#### Changed
|
||
|
||
- Inbound msg1 is classified before it is rate limited, and rekey or restart
|
||
msg1 arriving on an established link now draws on its own token bucket
|
||
instead of competing with stranger admission for a single shared one. On a
|
||
node with many peers the shared bucket refused a large share of ordinary
|
||
rekey traffic: a field node at roughly 245 peers refused 8753 msg1 in 25
|
||
minutes, and 159 of the 201 distinct sources were peers it already held
|
||
sessions with. Nodes upgrade with no config change. The `Msg1 rate limited`
|
||
log line now reports which limb refused, the pending count or the token
|
||
bucket, which it previously did not distinguish.
|
||
|
||
#### Security
|
||
|
||
- The Ethernet transport's discovery buffer is now bounded and no longer costs
|
||
a linear scan per beacon. Beacons are unauthenticated broadcast frames, and
|
||
the buffer deduplicated by scanning a `Vec` for the source MAC and had no
|
||
cap, so anything on the segment could name a fresh MAC per frame and drive
|
||
both quadratic CPU in the receive loop and unbounded memory. It is drained
|
||
once per tick only while the transport is operational, so a transport that
|
||
is receiving but not operational was never drained at all. The buffer is now
|
||
a map keyed on source MAC, capped at 1024 distinct MACs between drains, with
|
||
the drain order still oldest sighting first so which neighbour gets dialed
|
||
under a connect budget does not depend on hash iteration order. A MAC already
|
||
buffered is always refreshed, so a flood of new MACs cannot crowd out a
|
||
neighbour already seen. Refused beacons are counted in the transport's stats
|
||
as `beacons_dropped` and reported in the log on the first drop and then on
|
||
each power-of-ten thereafter, so the flooder does not set the log rate.
|
||
**What this does not close**: a flood can still crowd out a neighbour not yet
|
||
seen in that tick, and anything able to flood raw frames on the segment can
|
||
already jam the beacon at L2 more cheaply.
|
||
|
||
- A read failure on `peers.allow`, `peers.deny` or the `hosts` file no longer
|
||
turns the node into an open one. Every read error other than a steadily
|
||
absent file was logged and swallowed, leaving that file's entries empty, and
|
||
the reloader published the result unconditionally: an unreadable `peers.deny`
|
||
admitted the peers it named, and an unreadable `peers.allow` took a node from
|
||
admitting a named few to admitting everyone, with one warning line as the
|
||
only signal. Because the recorded modification times advanced before the
|
||
load, nothing retried until the file changed again, so a persistent
|
||
permission or I/O fault left the empty ACL in force indefinitely. The
|
||
reloader now keeps the last loaded ACL when any input is present but
|
||
unreadable, leaves the modification times alone, retries on the next tick
|
||
regardless of them, and logs the fault once on the transition rather than
|
||
once per tick. An absent file is still a policy and still loads as an empty
|
||
set; a `NotFound` that a successful stat contradicts is treated as a file
|
||
being rewritten under us and held. A reload whose inputs all read cleanly but
|
||
which empties an enforcing ACL while its files are still on disk is held for
|
||
one tick, which catches a read that caught a non-atomic in-place edit
|
||
mid-write, and released on the next so a deliberate blanking still takes
|
||
effect. `fipsctl` ACL status gains a `stale` flag reporting that the policy
|
||
in force is older than the files on disk. **What this does not close**:
|
||
there is no last-good snapshot at startup, so a node whose ACL file is
|
||
unreadable at boot still comes up with no entries, now logged as an error and
|
||
armed to retry on the first tick. Admission is also checked only at handshake
|
||
time, so a peer admitted during a window that has already happened keeps its
|
||
link.
|
||
|
||
- The end-to-end session table now has a bound. It was the one remotely-grown
|
||
map with none: an inbound SessionSetup naming an address nobody had seen
|
||
inserted an entry, and the two existing limits did not reach it, the setup
|
||
limiter governing the arrival rate rather than the population and the idle
|
||
purge only reaching entries a peer stops using. One neighbour sending setups
|
||
at the permitted rate could hold roughly 1440 half-open entries at any
|
||
moment and grow the table without limit by keeping them warm. Setups that
|
||
would grow the table past `node.limits.max_sessions` are now refused, ahead
|
||
of the setup limiter, so a full table costs no token, no responder handshake
|
||
and no ack; a refused setup emits nothing at all, which is indistinguishable
|
||
from loss to the sender and is already covered by its own msg1 resend
|
||
schedule. The test is whether admitting would grow the table, not whether
|
||
the sender is a stranger, so a resent setup for an entry already present is
|
||
still served and an in-flight handshake is not broken. Unauthenticated
|
||
half-open entries are additionally held to half the table, so a handshake
|
||
flood cannot deny the whole of it to peers that complete; that share is sized
|
||
to leave a reconnect storm, where every peer initiates at once after a
|
||
restart or a healed partition, room to land. Locally originated sessions are
|
||
capped at the same ceiling, answered with ICMPv6 destination unreachable so
|
||
the application gets an immediate error rather than a silent drop. The cap
|
||
refuses rather than evicts: the setup that triggers the decision is
|
||
unauthenticated at that point, so evicting would hand a stranger a way to
|
||
tear down sessions it has nothing to do with. Refusals are counted as
|
||
`table_full` and `half_open_full` in the session reject family. What stays
|
||
open is per-neighbour fairness among established sessions: one hostile
|
||
neighbour that completes handshakes and keeps each session warm can occupy
|
||
the table and hold new session establishment closed for as long as it keeps
|
||
doing so, which is a denial of new sessions rather than the unbounded memory
|
||
growth it replaces.
|
||
|
||
- An accepted inbound TCP connection no longer holds a slot indefinitely
|
||
without sending anything. The cap was tested at accept and the pool insert
|
||
and counter bump followed with no read in between, while the frame reader's
|
||
reads carried no deadline, so an unauthenticated remote held a slot by
|
||
connecting and staying silent. Pool keys are `ip:port`, so N sockets from one
|
||
address took N slots, and at the 256 default that locked out inbound peering
|
||
for as long as the sockets stayed open. The first frame on an inbound
|
||
connection now has a deadline, as a module constant rather than a new
|
||
configuration key, and the onion listener gets the same treatment for the
|
||
same accept-then-count ordering. Separately, the node's handshake reaper tore
|
||
down session state without closing the transport connection, so a peer that
|
||
sent msg1 and then stalled was forgotten by the node while its socket and
|
||
slot survived; the reaper now closes the connection too. **What this does not
|
||
close**: the deadline covers the first frame only, so a peer that sends one
|
||
well-formed frame and then goes silent still holds its slot. Closing that
|
||
needs a rolling idle deadline.
|
||
|
||
### Transports and configuration
|
||
|
||
#### Changed
|
||
|
||
- `fipsctl keygen` no longer exits non-zero when only the `fips.pub` write
|
||
fails. The private key is already on disk at that point, so failing the run
|
||
reported failure for a keygen that did produce the identity; the failure is
|
||
now a warning and the run succeeds. The pre-existing-key guard also moves
|
||
from `exists` to `symlink_metadata`, so a dangling symlink at the key path
|
||
now blocks keygen without `--force` instead of being overwritten silently.
|
||
|
||
#### Fixed
|
||
|
||
- The UDP transport's DNS cache is now bounded and actually evicts. The map
|
||
held one entry per distinct hostname string ever dialed, and the TTL was
|
||
applied only on the read, so a stale entry was overwritten on the next dial
|
||
of the same name and otherwise stayed for the life of the process. Under a
|
||
rendezvous policy that accepts advertised endpoints the keys are strings a
|
||
remote party chose, which made the growth theirs to drive. A store now
|
||
sweeps entries past their TTL and, if the map is still full, drops the
|
||
oldest, holding it to 256 hostnames. Refreshing a name already cached
|
||
evicts nothing. Eviction is by insertion time rather than last use, so a
|
||
rarely dialed name in a very large peer list may re-resolve more often; the
|
||
cost of a wrong eviction is one DNS lookup, not a failed dial.
|
||
|
||
- macOS: stopping an Ethernet transport under load no longer hangs the
|
||
process. The BPF reader thread handed each frame to the async consumer with
|
||
`blocking_send`, which parks with no way to be woken. Stopping the transport
|
||
aborts the consumer first, so nothing drains the 1024-frame channel, and the
|
||
socket's `Drop` then joined a thread that could never return: on a busy
|
||
interface the daemon had to be killed. The socket now drops the receiver
|
||
before joining, which releases a parked send at once, and the reader thread
|
||
sends through a helper that watches the same shutdown pipe its `select()`
|
||
already honours, so a send waiting for room cannot outlive a shutdown
|
||
request. The helper yields before it sleeps, so the saturated-path handoff
|
||
rate is unchanged. **Not covered by CI**: the reader thread is macOS-only
|
||
and Linux CI compiles none of it. What the tests prove is that the helper
|
||
the thread now waits in is cancellable; that a real BPF thread exits under
|
||
load still needs a manual check on a Mac.
|
||
|
||
- A failed private-key write no longer leaves a node silently running an
|
||
ephemeral identity. Six write results in the identity path were discarded,
|
||
and the sharpest was in `persistent` mode: a failed write to `fips.key` fell
|
||
through to an ephemeral identity with no message, so a node that had been
|
||
asked for a stable identity changed its npub, its routing address and its
|
||
mesh IPv6 on every start, and nothing said so. All six now report. An
|
||
ephemeral start that is about to overwrite an existing key file now warns
|
||
first, naming the path and the setting that would have preserved the
|
||
identity, which is the warning `fipsctl keygen` has always given and the
|
||
daemon never did. Existence is tested with `symlink_metadata` rather than
|
||
`exists`, because a dangling symlink reports absent from the latter while
|
||
still being a file the write acts on. The persistent read path additionally
|
||
warns when it finds a key file whose mode is looser than 0600, or one that is
|
||
a symlink; it does not repair either, since the daemon does not own a file it
|
||
did not create.
|
||
|
||
### Spanning tree, mesh size and routing
|
||
|
||
#### Fixed
|
||
|
||
- Flap dampening can now engage more than once in the lifetime of a node.
|
||
The arming check tested whether a dampening deadline had ever been set
|
||
rather than whether one was still in effect, so the first episode
|
||
disarmed the mechanism permanently: a node in a second flap storm went on
|
||
switching parents under hold-down alone, and neither the `flap_dampened`
|
||
counter nor the "Flap dampening engaged" warning fired again, so the
|
||
storm was invisible to anyone watching that counter. A lapsed episode is
|
||
now retired explicitly, clearing both the deadline and the switch
|
||
counter, so a second episode requires a fresh threshold of switches
|
||
within one window rather than re-engaging on the first switch after
|
||
lapse. Hold-down was unaffected throughout and continued to limit
|
||
discretionary switching, which is why the practical effect at shipped
|
||
settings was lost visibility and a lost escalation tier rather than
|
||
unrestrained flapping. Every path that can engage an episode now reports
|
||
it, including a re-engagement during parent-loss recovery, which was
|
||
previously silent. The warning names which path armed the episode
|
||
(`trigger`) and how long discretionary parent switching stays suppressed
|
||
(`dampening_secs`), using the same `trigger` values as the parent-switch
|
||
logs beside it, so the two can be read together. A
|
||
`node.tree.flap_dampening_secs` large enough to overflow the monotonic
|
||
clock is capped at one year, beyond which an episode is
|
||
indistinguishable from permanent, so an extreme setting no longer panics
|
||
the node when dampening engages.
|
||
|
||
### DNS responder
|
||
|
||
#### Security
|
||
|
||
- The DNS responder's mesh-interface filter now works on macOS and FreeBSD,
|
||
where it had never run. The filter drops `.fips` queries that arrive over the
|
||
mesh TUN, which is what keeps a widened `dns.bind_addr` from exposing the
|
||
hosts file's alias space to every mesh peer. It was keyed on the interface
|
||
index resolved from the *configured* TUN name, but macOS and FreeBSD assign
|
||
the device a name of the kernel's choosing (`utunN`, `tunN`), so the lookup
|
||
found nothing, the index came back `None`, and `None` disables the filter.
|
||
The index is now resolved from the name of the device the node actually
|
||
created, which the TUN startup path already records, and a live device whose
|
||
index will not resolve is logged rather than passed off as "no mesh
|
||
interface". Linux is unaffected, since the configured name is the device's
|
||
name there. **Behaviour change on macOS and FreeBSD**: a node with a
|
||
non-loopback `dns.bind_addr` stops answering `.fips` queries that arrive over
|
||
the mesh interface. **What this does not close**: with an app-owned TUN the
|
||
node never learns a device name, so the filter stays off there. **Not
|
||
measured**: whether macOS and FreeBSD attribute a locally originated query
|
||
sent to the node's own mesh address to the TUN interface, as Linux does. If
|
||
they do, such a query is now dropped on those platforms; the shipped resolver
|
||
drop-in targets `[::1]` rather than the mesh address, so the packaged path is
|
||
not affected.
|
||
|
||
### Gateway and peer lifecycle
|
||
|
||
#### Fixed
|
||
|
||
- A failed log write can no longer panic the thread or task that logged. The
|
||
subscriber was built with the default internal-error reporting, which sends a
|
||
failed write to `eprintln!`, and that macro panics when stderr has also
|
||
failed. The shipped supervisor configurations make that a single condition
|
||
rather than two: the macOS plist points both standard streams at one
|
||
unrotated file, and the systemd units route both to journald, so one full
|
||
disk fails both sinks together. In the daemon a crypto worker was the case
|
||
that mattered: it logs a warning on send backpressure, and a worker that
|
||
dies takes its share of the peer space with it permanently, while the panic
|
||
message is discarded along the same broken path. In `fips-gateway`, which
|
||
built its subscriber the same way, the casualty is a spawned task: the DNS
|
||
resolver, the control accept loop or the pool tick, none of which is observed
|
||
until shutdown, so the process would keep running and reporting healthy with
|
||
mesh name resolution or lease expiry and NAT cleanup silently stopped.
|
||
|
||
#### Security
|
||
|
||
- The gateway DNS forwarder now validates an upstream answer before it becomes
|
||
a NAT mapping. It previously accepted whatever datagram arrived: the upstream
|
||
query reused the client's own transaction ID, the upstream socket was
|
||
wildcard-bound and never connected, the receive discarded the sender, neither
|
||
the response ID nor the question section was compared against what was asked,
|
||
and the returned address was not checked against the mesh prefix. Because the
|
||
extracted address is installed as a DNAT rule that carries no interface
|
||
constraint, a forged answer redirected traffic rather than only poisoning a
|
||
lookup. The upstream query now carries a random transaction ID, the socket is
|
||
connected so the kernel drops foreign sources, a response must match on ID,
|
||
question and type or it is discarded while the receive continues against the
|
||
original deadline, and the address goes through the validating parser with a
|
||
non-mesh answer refused before any allocation. One deliberate behaviour
|
||
change: the validation sits before the rcode check, so an upstream answering
|
||
FORMERR or REFUSED with an empty question section now yields SERVFAIL rather
|
||
than having its rcode relayed. Checking after the rcode would admit a forged
|
||
NXDOMAIN. Connecting the socket also means a dead upstream surfaces
|
||
ECONNREFUSED immediately instead of stalling for five seconds.
|
||
|
||
### Control socket
|
||
|
||
#### Security
|
||
|
||
- The control socket and the directory holding it are now created with a
|
||
restrictive mode rather than created wide and narrowed afterwards. `bind(2)`
|
||
makes the socket inode `0777 & ~umask`, so under a permissive umask the
|
||
socket was world-accessible for the window between the bind and the `chmod`
|
||
to 0770 that followed it; the bind now runs under a umask that masks the
|
||
"other" bits, so the inode is 0770 from creation and the chmod and chown stay
|
||
the authority on its final mode. The parent directory was worse than a
|
||
window: it was created with `create_dir_all`, which is also `0777 & ~umask`,
|
||
and nothing ever set a mode on it, so under a permissive umask the directory
|
||
holding the socket stayed world-writable for the life of the host, and a
|
||
world-writable parent lets an unprivileged account plant an entry at the
|
||
socket path. Directories this code creates now come out 0750, which is what
|
||
the systemd unit (`RuntimeDirectoryMode=0750`) and the FreeBSD rc script
|
||
(`install -d -m 0750`) already apply, so no packaged deployment sees a
|
||
different mode and no `fipsctl` user loses access. Both the daemon and the
|
||
gateway control sockets are covered. **What this does not close**: the window
|
||
between the stale-socket probe and the bind is documented at the site rather
|
||
than removed. Reaching it needs write access to the socket's parent
|
||
directory, which the packaged layouts give to root alone, and an account
|
||
holding it can deny the daemon its socket more simply by squatting the path
|
||
first.
|
||
|
||
### Key material and identity files
|
||
|
||
#### Security
|
||
|
||
- Private key writes no longer follow a symlink, and the key file's mode is
|
||
enforced rather than merely requested. The single write path opened with
|
||
create and truncate and no `O_NOFOLLOW`, so a symlink planted at the key path
|
||
was followed and its target overwritten, and it supplied the mode only
|
||
through `open(2)`, which the kernel honours on creation and ignores
|
||
otherwise, so a `fips.key` that already existed at 0644 stayed 0644 through
|
||
every rewrite. That second half needs no attacker: one `chmod`, or a restore
|
||
that did not preserve modes, leaves the key readable indefinitely. Both
|
||
writers now share an open helper carrying `O_NOFOLLOW`, and the private key
|
||
has its mode applied to the open descriptor before any secret bytes are
|
||
written. The public key keeps create-time mode instead, since forcing it
|
||
would reopen an operator-tightened `fips.pub` on every start. On Windows
|
||
neither protection applies and the file inherits the parent directory's
|
||
ACLs; that exclusion is deliberate and recorded at both writers.
|
||
|
||
- Private and symmetric key material is now cleared when it goes out of scope.
|
||
Nothing in the crate erased a key before this: the node's private key sat in
|
||
the loaded configuration in plaintext for the whole process lifetime, which is
|
||
the longest any secret lives here, every Noise handshake left its static and
|
||
ephemeral keypairs, its per-message Diffie-Hellman results and its chaining
|
||
key in freed memory, and each session's ChaCha20-Poly1305 keys were dropped
|
||
intact. Clearing now covers the retained cipher key on each cipher state, the
|
||
chaining key and the handshake hash, the 64-byte HKDF outputs and the two
|
||
session keys derived from them, the static and ephemeral keypairs a handshake
|
||
holds for its whole duration, the identity's long-term keypair, the temporary
|
||
copy each of the fourteen elliptic-curve operations makes from a keypair, the
|
||
bech32 and hex encodings of a secret, and the private key on its way through
|
||
configuration, including the config file's whole text, which is treated as
|
||
secret for as long as it is held, since `node.identity.nsec` is read straight
|
||
out of it. Two places that assigned over an already-loaded key now clear the
|
||
old value first: assignment frees the previous string without running the
|
||
type's clearing destructor, so a configuration that already carried a key left
|
||
the superseded copy in the heap whenever a second source replaced it. **This
|
||
clears the copies the crate owns, not every copy that ever existed.** The
|
||
secp256k1 key types are copyable, so the compiler may duplicate them where no
|
||
code here can name them, which is why that library calls its own erase
|
||
non-secure. The hash and key-derivation states, and the cipher keys cached
|
||
inside `ring`, offer no clearing route at the versions pinned here and are
|
||
deliberately left alone; the residue any of this leaves needs access to the
|
||
process's memory, or to a core dump or swap image of it, to read. Adds a
|
||
dependency on `zeroize`. Nothing on the wire and no configuration changes.
|
||
See the `### Changed` note above for the source-breaking effect the four new
|
||
`Drop` implementations have on library consumers.
|
||
|
||
### Library API
|
||
|
||
#### Changed
|
||
|
||
- **Source-breaking for consumers of the library crate**: four public types now
|
||
implement `Drop`, so their fields can no longer be moved out. `Identity`,
|
||
`ResolvedIdentity`, `IdentityConfig` and `HandshakeState` each gained one as
|
||
part of clearing key material at end of scope. `IdentityConfig` is the one
|
||
most likely to be reached in practice, since it hangs off the public `Config`
|
||
as `node.identity`, so code that moved the nsec out of a configuration value
|
||
no longer compiles and needs `Option::take` instead. Nothing about the
|
||
behaviour of the shipped binaries changes; this affects only callers using
|
||
`fips` as a library.
|
||
|
||
### Supply chain
|
||
|
||
#### Security
|
||
|
||
- The dependency lockfile is refreshed past a set of advisories against the
|
||
pinned `nostr` 0.44.3 and `nostr-relay-pool` 0.44.1, both of which were also
|
||
yanked. `nostr` moves to 0.44.8 and `nostr-relay-pool` to 0.44.3; the
|
||
requirements in `Cargo.toml` already admitted both, so this is a lockfile
|
||
change and no code changed with it. The advisories that matter here are the
|
||
relay-pool ones, RUSTSEC-2026-0224 and RUSTSEC-2026-0232, which describe
|
||
forged events bypassing signature validation and unverified relay events
|
||
being processed: that is the path this node learns peer adverts on, and it
|
||
performs no independent verification of its own, so the exposure was a
|
||
misattributed advert rather than the denial of service the advisory summaries
|
||
lead with. RUSTSEC-2026-0231 (auth-challenge memory exhaustion) is on the
|
||
same path, and RUSTSEC-2026-0216 and RUSTSEC-2026-0227 reach the NIP-44
|
||
decryption of relay-supplied content. The remaining advisories in that set
|
||
cover NIP-04, NIP-46, NIP-50, NIP-60, NIP-98 and the wallet parsers, none of
|
||
which this code calls. The refresh was taken over the whole lockfile rather
|
||
than the two crates alone, which additionally clears RUSTSEC-2026-0204 in
|
||
`crossbeam-epoch` and leaves no yanked crate in the tree; `cargo audit` now
|
||
reports no vulnerability, against twelve before. Four warnings remain and are
|
||
not fixable by a version move: `instant` and `paste` are unmaintained, `lru`
|
||
0.16.4 carries an unsoundness advisory, and `nostr-relay-pool` itself is now
|
||
marked unmaintained.
|
||
|
||
- Every GitHub Action is pinned to a commit SHA, and the OpenWrt packaging
|
||
workflow verifies both of the artifacts it downloads. No reference in the
|
||
repository was pinned before: all sixty-six named a mutable tag and one named
|
||
a branch, including the jobs holding the AUR deploy key, the jobs with
|
||
release write scope, and the packaging jobs that run with a signing key in
|
||
the environment. Sixty-two are now full commit SHAs with the original tag
|
||
retained as a trailing comment. Four are left unpinned and justified in one
|
||
place: two actions read the tool to install from the ref name itself, so a
|
||
SHA would hand them a hex string where a toolchain name belongs. A guard
|
||
enforces the form on every sweep, treats an unreadable tree as an error
|
||
rather than a pass, and documents what it does not cover. The sharper hole
|
||
was not the tags: the OpenWrt workflow fetched a helper binary from a release
|
||
URL with no verification at all, in two jobs holding a signing key. That
|
||
download now checks a per-architecture pinned SHA-256, with the hash
|
||
provenance recorded honestly, upstream publishing no checksum document.
|
||
|
||
The same workflow's Zig toolchain fetch is verified the same way. It ran as
|
||
`curl | tar`, which leaves nowhere to check the bytes, so a short read
|
||
reached `tar` as a truncated archive and failed the build with "Unexpected
|
||
EOF in archive"; curl's `--retry` does not cover that exit. The download now
|
||
stages to a temporary directory, checks a per-architecture pinned SHA-256
|
||
with a guard that fails if an architecture is added without one, and only
|
||
then extracts, with three attempts at 10s and 20s backoff and an early exit
|
||
when two attempts return identical bytes, since a stable mismatch is a wrong
|
||
pin rather than a bad transfer. As with the helper binary, the hashes come
|
||
from the upstream download index and were checked against the tarball bytes:
|
||
that is integrity, not authenticity, because upstream publishes no detached
|
||
sums.
|
||
|
||
### Packaging, install and platform layout
|
||
|
||
#### Fixed
|
||
|
||
- macOS: `peers.allow`, `peers.deny`, and the `hosts` file are now read
|
||
from `/usr/local/etc/fips/`, matching the install layout the macOS
|
||
packaging ships (`packaging/macos/`). That layout is what the three fixes
|
||
in this group align the daemon and `fipsctl` to. The default-path constants
|
||
were hardcoded to `/etc/fips/...` with only a `#[cfg(unix)]` /
|
||
`#[cfg(windows)]` split, so on macOS the daemon looked in a directory that
|
||
does not exist: `load_file` / `load_hosts_file` hit their `NotFound` no-op
|
||
arm and silently returned an empty ACL / empty host map. A populated
|
||
`peers.deny` therefore reported `effective_mode: "default_open"` and
|
||
`enforcement_active: false` via `fipsctl acl show`, and host-file aliases
|
||
went unloaded, with no error or warning. The default constants now follow
|
||
the platform's packaging (`/usr/local/etc/fips/` on macOS, `/etc/fips/` on
|
||
Linux and other Unix for the ACL files, and `/etc/fips/` on Linux and
|
||
`%ProgramData%\fips\` on Windows for the hosts file) and are pinned by
|
||
platform-gated unit tests so the layout cannot silently drift again. At
|
||
startup the daemon warns once if any of these files exist at the old
|
||
`/etc/fips/` location but not at the current default. Linux and Windows
|
||
behavior is unchanged. Contributed by
|
||
[@sh1ftred](https://github.com/sh1ftred).
|
||
**macOS users with existing files in `/etc/fips/` should move them to
|
||
`/usr/local/etc/fips/`.**
|
||
|
||
- macOS: `fipsctl keygen` now writes `fips.key` / `fips.pub` to
|
||
`/usr/local/etc/fips/` by default. The default output directory was
|
||
hardcoded to `/etc/fips` for all Unix, but the daemon derives its identity
|
||
key paths from the config file's directory, which is
|
||
`/usr/local/etc/fips/fips.yaml` on macOS, so a generated identity landed
|
||
where the daemon never reads it and the node silently kept an ephemeral
|
||
identity. Linux and other Unix keep `/etc/fips`, Windows is unchanged, and
|
||
the values are pinned by platform-gated unit tests.
|
||
|
||
- macOS: the system-wide config search path now includes
|
||
`/usr/local/etc/fips/fips.yaml` in addition to `/etc/fips/fips.yaml`.
|
||
Previously only `/etc/fips/fips.yaml` was probed, so a bare `fips` run
|
||
without `--config` skipped the installed config and derived identity key
|
||
paths from a non-existent directory. `/etc/fips/fips.yaml` is still probed
|
||
first so existing installs keep working. Both the macOS entry in the search
|
||
path and the directory `fipsctl keygen` writes to read the shared
|
||
`SYSTEM_CONFIG_DIR` constant, so the two cannot drift apart. The
|
||
launchd-installed daemon was unaffected (it always passes `--config`).
|
||
Linux and Windows behavior is unchanged. Because the daemon derives the
|
||
identity key directory from whichever config file loaded last, a macOS host
|
||
carrying `fips.yaml` at both locations would have resolved `fips.key` to the
|
||
new directory, found none, and under `persistent` generated a fresh
|
||
identity, silently changing its npub, routing address and mesh IPv6. The
|
||
daemon now adopts a key stranded at `/etc/fips/fips.key` and warns to move
|
||
it, instead of generating one. The fallback is confined to keys resolved
|
||
from the system config directory, so a run using `./fips.yaml` or a user
|
||
config is never redirected to a system key.
|
||
|
||
- The maintainer address published in package metadata no longer bounces. The
|
||
crate authors field, the Debian package maintainer and upstream contact, and
|
||
both AUR PKGBUILD maintainer lines carried an address that no longer accepts
|
||
mail, so the contact of record in every artifact we ship was unreachable.
|
||
|
||
### Docs, CI and contributor tooling
|
||
|
||
#### Added
|
||
|
||
- `SECURITY.md`, stating a private channel for vulnerability reports, what a
|
||
useful report contains, what a reporter can expect back and on what timing,
|
||
and which branches receive fixes. The repository previously documented no
|
||
reporting channel at all, so someone with a finding had to guess at an
|
||
address or open a public issue.
|
||
|
||
#### Changed
|
||
|
||
- Two CI runs on one machine can no longer collide. Every suite derives its own
|
||
docker build context, image tag, container names, network range and host
|
||
interface names per run, so concurrent runs cannot reap each other's
|
||
containers or contend for a fixed subnet. This is what a contributor running
|
||
`testing/ci-local.sh` alongside a GitHub run, or two local runs at once, sees
|
||
change: the runs stay independent instead of one killing the other.
|
||
|
||
- A test that does not run, or whose result cannot be read, no longer passes
|
||
silently. A failed scenario now fails the run rather than being logged and
|
||
stepped over, a node whose logs cannot be read no longer counts as clean, an
|
||
unanswered control query no longer reads as zero, an unknown scenario key is
|
||
rejected instead of matching nothing, and a skipped check appears in the
|
||
final verdict rather than only in scrollback.
|
||
|
||
- New guards run in both the local and GitHub runners, so the two gates agree.
|
||
They check that trailing-log call sites are wired, that the log strings the
|
||
harness matches on are still emitted by the daemon, that the two runners'
|
||
integration-suite sets match per leg rather than as a folded token, that
|
||
every GitHub Action reference is pinned in the required form, and that source
|
||
comments do not cite references a reader of the published tree cannot
|
||
resolve.
|
||
|
||
- Coverage moved from Docker to deterministic in-process tests, and dead
|
||
scenarios were retired. The six cost-selection chaos scenarios, the
|
||
admission-cap and acl-allowlist Docker suites, the smoke-10 scenario, the
|
||
tcp-chain and mesh-public static topologies, and three ignored Ethernet
|
||
tests are gone, with their behaviour asserted in unit and integration tests
|
||
instead. A local CI run is correspondingly shorter and less dependent on
|
||
container timing.
|
||
|
||
- The `bloom-storm` chaos scenario no longer runs on either the local or the
|
||
cloud runner. Unlike the retirements above it has no replacement: the
|
||
scenario files remain in the tree and it stays runnable by hand, but nothing
|
||
now exercises downstream containment of a mid-chain ancestor swap on a
|
||
schedule. This is recorded as a coverage gap rather than as a completed
|
||
migration.
|
||
|
||
- A failing harness now says why it failed. The dns-resolver suite sent build
|
||
and container-start output to `/dev/null`, so a failed scenario reported
|
||
that it had failed and nothing else; output is now captured and emitted on
|
||
failure, naming the command, and the systemd readiness wait dumps container
|
||
state, failed units and the journal when it gives up. The NAT-lab path
|
||
assertions exited bare, printing neither what they expected nor what they
|
||
saw and triggering none of the scenario diagnostics their siblings already
|
||
call; all twelve call sites now report the container, the expectation, the
|
||
observation and a projection of the peer or link table, and distinguish a
|
||
failed control-socket exec from unparseable output from a genuine mismatch.
|
||
The convergence gate could not tell a tree that did not converge from
|
||
connectivity that failed, and could exit non-zero while reporting "20
|
||
passed, 0 failed"; it now records the outcome, the count reached and the
|
||
count pending, and its failure messages name the condition. A passing run
|
||
is as quiet as before, and no timing, threshold or control-flow behaviour
|
||
changed in any of the three.
|
||
|
||
- A dns-resolver scenario no longer burns the full 30-second boot timeout and
|
||
warns about a container that booted correctly. The readiness poll ran under
|
||
`pipefail` and piped `systemctl is-system-running` into `grep`, and that
|
||
command exits non-zero when the system is degraded, which is where systemd
|
||
inside a container always settles; the pipeline therefore failed even when
|
||
the pattern matched, leaving the degraded branch dead. The poll now matches
|
||
on the captured state instead of piping into `grep`.
|
||
|
||
- The chaos harness now checks that teardown and node stops did what they
|
||
report. `docker compose down` exits 0 while leaving a run's containers
|
||
alive, so a partly-failed bring-up leaked named containers with nothing to
|
||
detect it; teardown now asks whether the containers this run owns are gone,
|
||
treats a survivor that forced removal clears as a warning, and aborts with
|
||
the names written to an artifact when one survives that or the query cannot
|
||
run at all. The check is scoped to a run's own names, so concurrent runs
|
||
cannot trip each other. Node churn separately marked a node down whether or
|
||
not `docker stop` succeeded, so the simulation's model of the mesh diverged
|
||
from reality, and `nodes_down`, the `max_down_nodes` cap and the
|
||
connectivity guard are all computed from that model; a failed stop now
|
||
warns, carries the daemon's own message, and leaves the node out of the
|
||
down set for the next churn tick to retry against an honest model.
|
||
|
||
- Comments throughout the source tree, the packaging files and the test scripts
|
||
no longer cite internal identifiers, planning documents or private stage names
|
||
that a reader of the published tree cannot resolve; each now states the thing
|
||
the citation stood for. A handful of comments that described behaviour the
|
||
code does not have (the control-plane read path, its snapshot dispatch, and
|
||
the MMP report types) have been corrected rather than merely reworded. One of
|
||
the edited files, the DNS setup helper, installs to `/usr/lib/fips` on every
|
||
packaging path, so its comment reached users. No code changed.
|
||
|
||
#### Security
|
||
|
||
- The security reference now records that both Noise patterns deviate from the
|
||
standard construction in one respect: the handshake AEAD passes an empty
|
||
associated-data field where standard Noise `EncryptAndHash` uses the handshake
|
||
hash `h`. The published tables named `Noise_IK_secp256k1_ChaChaPoly_SHA256`
|
||
and `Noise_XK_secp256k1_ChaChaPoly_SHA256` unqualified, so anyone auditing the
|
||
stack against the Noise specification had nothing telling them where to look.
|
||
Domain separation and Diffie-Hellman binding survive through the chaining key,
|
||
which is seeded from the protocol name and chained at every step; transcript
|
||
binding is the property actually absent. Nothing in the daemon reads the
|
||
handshake hash, so no shipped behaviour rests on it, but the comments that
|
||
called it transcript binding or channel binding overstated it and now describe
|
||
what the field is, and the field records that anything later built on it
|
||
(channel binding, an exporter, cookie binding) will silently not work until
|
||
the associated data carries `h`. This is a correction to what is documented
|
||
and claimed; no code behaviour and nothing on the wire changed.
|
||
|
||
## [0.4.1] - 2026-07-19
|
||
|
||
### Changed
|
||
|
||
- `node.bloom.max_inbound_fpr` default raised from `0.10` to `0.20`. The
|
||
cap rejects inbound `FilterAnnounce` whose FPR (`fill^k`) exceeds it. On
|
||
the fixed 1 KB / k=5 filter, `0.10` corresponds to fill 0.631 (~1,630
|
||
reachable entries), and the busiest nodes' aggregates had again begun to
|
||
reach it as the mesh grew. `0.20` (fill 0.7248, ~2,114 entries) restores
|
||
headroom without materially weakening the antipoison gate: a saturated or
|
||
poisoned filter is ~100% FPR and still rejected. This is the second raise
|
||
of this cap in two releases; the fixed 1 KB filter is the underlying
|
||
constraint, and the structural remedy is the v2 filter work rather than a
|
||
further raise. A node running this default accepts announcements that a
|
||
v0.4.0 node drops, so during a rolling upgrade the two versions can
|
||
disagree about mesh size.
|
||
- Bloom filter probing computes its SHA-256 digest once per operation
|
||
rather than once per hash function. All k indices were already derived
|
||
from a single digest, but the digest was recomputed inside the
|
||
per-function loop, so every insert and membership test hashed the same
|
||
bytes `hash_count` times (5x at the default). Output is bit-for-bit
|
||
identical; this is the hottest path in packet forwarding and mesh-size
|
||
estimation.
|
||
- Identity operations reuse one shared `secp256k1` context instead of
|
||
constructing a fresh one at every sign, verify, and key-derive site.
|
||
Each construction allocated a context and ran randomization and blinding
|
||
table setup. Behavior is unchanged: the same API calls are made, only the
|
||
context lifetime differs, and the shared context still performs the
|
||
standard construction-time blinding.
|
||
|
||
### Fixed
|
||
|
||
- Spanning tree: the coordinate cache is now invalidated when the parent
|
||
link is lost through peer removal. That path reparents or self-roots the
|
||
node but omitted the invalidation every other position-change path
|
||
performs, so cached entries for downstream destinations kept the node's
|
||
now-stale coordinate prefix. Because routing access refreshes an entry's
|
||
TTL, an actively routed stale entry never self-expired and was corrected
|
||
only by a fresh insert.
|
||
- Discovery: applying a `LookupResponse` now keeps the tighter of the
|
||
cached and received `path_mtu` rather than overwriting unconditionally.
|
||
A looser estimate arriving in a later response could clobber a tighter
|
||
value already learned from a reactive `MtuExceeded` or
|
||
`PathMtuNotification`, loosening a clamp that had been correctly
|
||
tightened.
|
||
|
||
### Removed
|
||
|
||
- The `parent_switched` spanning-tree metric counter. It was incremented on
|
||
the line immediately before `parent_switches` at every site and never
|
||
independently, so the two were always identical. `parent_switches`
|
||
remains as the sole counter. Consumers reading `parent_switched` from the
|
||
control socket or `fipstop` should use `parent_switches`.
|
||
|
||
## [0.4.0] - 2026-06-27
|
||
|
||
### Added
|
||
|
||
#### Transports (Nym, mDNS LAN discovery)
|
||
|
||
- Nym mixnet transport (`transports.nym`) for outbound peer links
|
||
tunneled through a local `nym-socks5-client` SOCKS5 proxy into the
|
||
Nym mixnet, as a privacy transport alongside Tor. Outbound-only and
|
||
not platform-gated, it reuses the existing FMP framing and adds no new
|
||
crate dependencies. A single-container example
|
||
(`examples/sidecar-nostr-mixnet-relay/`) demonstrates FIPS peering
|
||
across the mixnet end to end.
|
||
- Opt-in mDNS / DNS-SD LAN discovery for sub-second pairing of peers on
|
||
the same local link, without a relay or NAT-traversal roundtrip.
|
||
Disabled by default; operators enable it with
|
||
`node.discovery.lan.enabled: true`. Configurable service type and an
|
||
optional `node.discovery.lan.scope` that isolates discovery to peers
|
||
sharing the same private-network scope. The advertised UDP port is
|
||
chosen from a non-bootstrap operational UDP transport using a stable
|
||
selector, so it is deterministic across restarts.
|
||
|
||
#### Admission / peer-list management
|
||
|
||
- `Node::update_peers` for runtime peer-list refresh, returning an
|
||
`UpdatePeersOutcome` summarizing added, removed, and retained peers.
|
||
Re-derives active peer connections from a new peer configuration
|
||
without dropping links to peers that remain in the set.
|
||
`PeerAddress` gains a `seen_at_ms` recency field (with
|
||
`with_seen_at_ms`) used to prefer more recently observed addresses.
|
||
|
||
#### Data-plane / metrics / observability
|
||
|
||
- Typed `RejectReason` classification for receive-path silent-rejection
|
||
sites across the node. Each rejection-and-return path now passes a
|
||
typed reason to `NodeStats::record_reject`, which routes it to a
|
||
per-subsystem counter, so operators can see what is being rejected
|
||
through stats counters rather than by scraping debug logs. New
|
||
`HandshakeStats`, `SessionStats`, and `MmpStats` sub-stats join the
|
||
existing `TreeStats`, `BloomStats`, `DiscoveryStats`, and
|
||
`ForwardingStats`, and `TreeStats::ancestry_invalid` is now
|
||
incremented from the `TreeAnnounce::validate_semantics` rejection
|
||
site that was previously silent. Several handshake, MMP, tree, and
|
||
discovery rejection paths that had no counter at all are now counted,
|
||
including the `send_lookup_response` no-route drop
|
||
(`DiscoveryStats::resp_no_route`).
|
||
- Internal atomic metric registry (`Arc<MetricsRegistry>`) that shadows
|
||
the plain-`u64` `NodeStats` counters, written alongside them and
|
||
validated by a whole-struct debug-build parity check. Covers the
|
||
forwarding receive counters, the full discovery counter family, and the
|
||
tree, bloom, congestion, and error-signal counter families, with
|
||
the hottest counters cache-line padded. Behavior-neutral:
|
||
`NodeStats` remains the serving path. Groundwork for sampling metrics
|
||
without contending the receive loop.
|
||
- `fipsctl stats metrics`, backed by a new counter-only `show_metrics`
|
||
control query that dumps the atomic metric registry as flat counter
|
||
name/value pairs. Serves a Prometheus-style scraper that samples node
|
||
counters without contending the receive loop.
|
||
- `pool_inbound` and `pool_outbound` counters on the TCP and Tor
|
||
transport stats (`TcpStats`, `TorStats`). Per-direction accounting
|
||
is updated at every pool-insert and receive-loop-exit site, plus on
|
||
transport stop and on send-failure-driven removal. Surfaces through
|
||
`TcpStatsSnapshot` and `TorStatsSnapshot` for `show_transports`.
|
||
|
||
#### Spanning-tree / mesh-size / routing
|
||
|
||
- Six route-class transit counters that partition transit-forwarded
|
||
packets by their tree relationship to the chosen next hop: tree-up
|
||
(peer is our ancestor), tree-down (peer is our descendant and the
|
||
destination is within its subtree), tree-down-cross (peer is our
|
||
descendant but the destination is outside its subtree), cross-link
|
||
descend (lateral peer, destination within its subtree), cross-link
|
||
ascend (lateral peer, destination outside its subtree), and
|
||
direct-peer. The six classes sum to `forwarded_packets` (asserted by a
|
||
unit test) and are computed from tree coordinates at the transit
|
||
chokepoint, so error-signal routing callers are excluded. They surface
|
||
through the forwarding stats snapshot via `show_routing` and
|
||
`show_status`.
|
||
- Discovery now counts `LookupRequest`s dropped when the dedup cache is
|
||
full. A saturated `recent_requests` cache
|
||
(`MAX_RECENT_DISCOVERY_REQUESTS`) previously dropped requests
|
||
silently; a new `DiscoveryStats::req_dedup_cache_full` counter (typed
|
||
reject reason `DiscoveryReject::ReqDedupCacheFull`) makes the drop
|
||
visible through `show_routing`.
|
||
|
||
#### Packaging & deployment
|
||
|
||
- OpenWrt `.apk` packaging (`packaging/openwrt-apk/`, `make apk`) for
|
||
OpenWrt 25+, where apk-tools is the mandatory package manager (the
|
||
existing `.ipk` continues to cover OpenWrt 24.x and earlier). Built
|
||
SDK-free: it reuses the `.ipk` cross-compile (`cargo-zigbuild`) and the
|
||
shared installed-filesystem payload, and assembles the package with
|
||
`apk mkpkg` from apk-tools 3.0.5 built from source — no OpenWrt SDK
|
||
image. A `build-apk` CI job (aarch64, x86_64) builds and structurally
|
||
verifies the package; releases now publish `.apk` artifacts and
|
||
checksums alongside `.ipk`. Packages are unsigned, installed with
|
||
`apk add --allow-untrusted`, matching the `.ipk` posture.
|
||
- Nix flake (`flake.nix` at the project root) for reproducible
|
||
from-source builds on Nix/NixOS. Builds all four binaries (`fips`,
|
||
`fipsctl`, `fips-gateway`, `fipstop`), pins the exact toolchain from
|
||
`rust-toolchain.toml` via fenix, and wires the build-time native
|
||
dependencies (`libclang` for `bindgen`, plus `dbus` and `pkg-config`),
|
||
so it needs no host setup beyond Nix with flakes enabled. Flake inputs
|
||
are lock-pinned (`flake.lock` committed) for reproducibility, and the
|
||
flake exposes `nix build`, `nix run`, a `nix develop` dev shell with the
|
||
pinned toolchain, and `nix flake check`. The flake produces binaries
|
||
(and a NixOS `packages.<system>.fips` output); the systemd/service
|
||
integration that the `.deb`/tarball installers provide is handled
|
||
through the NixOS configuration instead.
|
||
|
||
#### Docs & contributor tooling
|
||
|
||
- [`PR-REVIEW.md`](PR-REVIEW.md) — the 13-criteria PR review checklist
|
||
the maintainer runs against every incoming PR, published at the
|
||
repo root so contributors can run the same pass on their own change
|
||
(directly or by handing the document to a coding agent) before
|
||
opening. Linked from `CONTRIBUTING.md` under "Submitting pull
|
||
requests" and "Further reading". Running the checklist before
|
||
opening surfaces problems that would otherwise come back as review
|
||
comments, saving a round trip.
|
||
- [`docs/how-to/tune-file-descriptors.md`](docs/how-to/tune-file-descriptors.md)
|
||
— an operator how-to for raising `RLIMIT_NOFILE`. A busy node opens
|
||
roughly three file descriptors per established UDP peer (a
|
||
`connect()`-ed socket plus a 2-FD drain self-pipe), so the default
|
||
1024 soft limit is exhausted near 320 peers, after which further
|
||
admission, handshakes, and discovery fail with `EMFILE`. The guide
|
||
documents the per-peer FD budget and symptom, the systemd
|
||
(`LimitNOFILE` drop-in) and OpenWrt (procd `nofile`) procedures to
|
||
raise the limit, and how to verify the per-peer ratio stays bounded.
|
||
Linked from the how-to index.
|
||
|
||
### Changed
|
||
|
||
#### FMP/FSP rekey reliability
|
||
|
||
- `complete_rekey_msg2` now returns the remote peer's startup epoch
|
||
alongside the new Noise session, so the rekey path can detect a peer
|
||
restart and clear stale session state.
|
||
|
||
#### NAT traversal / Nostr discovery
|
||
|
||
- Nostr discovery startup is now non-blocking. `Node::start` no
|
||
longer waits for relay connect, subscribe, or initial advert
|
||
publish before returning. A slow or unreachable relay no longer
|
||
holds node startup hostage; local transports come up immediately
|
||
and the relay path catches up asynchronously in background tasks.
|
||
Subscribe retries with exponential backoff (2 s base, 60 s cap),
|
||
publish attempts time out at 10 s, and the new tasks are aborted
|
||
cleanly on `Node::stop`.
|
||
|
||
#### Spanning-tree / mesh-size / routing
|
||
|
||
- Active-peer path selection now sorts address candidates by recency
|
||
(`seen_at_ms`), preferring the most recently observed address when
|
||
racing concurrent path probes.
|
||
- Per-tick work budgets bound the connection churn done in a single
|
||
node tick: `MAX_DISCOVERY_CONNECTS_PER_TICK`,
|
||
`MAX_RETRY_CONNECTIONS_PER_TICK`, and
|
||
`MAX_PARALLEL_PATH_CANDIDATES_PER_PEER`. Work beyond a tick's budget
|
||
is deferred to the next tick rather than discarded.
|
||
|
||
#### Admission / peer caps
|
||
|
||
- `node.bloom.max_inbound_fpr` default raised from `0.05` to `0.10`. The
|
||
cap rejects inbound `FilterAnnounce` whose FPR (`fill^k`) exceeds it. On
|
||
the fixed 1 KB / k=5 filter, `0.05` corresponds to fill 0.549 (~1,300
|
||
reachable entries) and had begun rejecting the busiest nodes' aggregates
|
||
as the mesh approached that size. `0.10` (fill 0.631, ~1,630 entries)
|
||
restores headroom toward the fixed-filter capacity limit without
|
||
materially weakening the antipoison gate: a saturated or poisoned filter
|
||
is ~100% FPR and still rejected.
|
||
- TCP inbound connection cap now honors `node.limits.max_connections`.
|
||
The per-transport TCP inbound accept ceiling was hardwired to 256 and
|
||
never read `max_connections`, so raising it was a silent no-op for
|
||
inbound TCP. The effective cap now resolves with precedence: explicit
|
||
per-transport `max_inbound_connections`, then node-wide
|
||
`max_connections`, then the built-in default of 256. Established peers
|
||
remain bounded node-wide by `add_connection`.
|
||
|
||
#### Data-plane / worker-pool / metrics / observability
|
||
|
||
- The control-socket read surface is now served off the `rx_loop`.
|
||
Every pure-read `show_*` query — `show_status`, the `show_stats_*`
|
||
family, `show_listening_sockets`, the new `show_metrics`,
|
||
`show_tree`/`show_bloom`/`show_cache`/`show_routing`/
|
||
`show_identity_cache`, `show_peers`/`show_sessions`/`show_links`/
|
||
`show_connections`/`show_transports`/`show_mmp`, and `show_acl` — now
|
||
renders in the control accept task from ArcSwap-published read
|
||
snapshots instead of round-tripping the data-plane receive loop; only
|
||
the mutating `connect`/`disconnect` commands still reach the loop.
|
||
This removes the head-of-line coupling where a busy or slow `rx_loop`
|
||
could time out `fipsctl` and `fipstop` observability (the five-second
|
||
query pattern operators saw on loaded nodes). Per-entity snapshots
|
||
reuse unchanged rows by pointer, so per-tick publish cost stays
|
||
bounded as peer/session count grows. New daemon-resolved fields
|
||
surface through the snapshots: effective persistence, root/is-root,
|
||
and a per-transport-type peer-count map in `show_status`; per-peer
|
||
`effective_depth` in `show_peers`; `root_npub` in `show_tree`; and the
|
||
last-sent uptree filter fill ratio and subtree estimate in
|
||
`show_bloom`.
|
||
- `fipstop` TUI overhaul: reworked rendering, navigation model, and the
|
||
control read surface it draws from, surfacing the new daemon-resolved
|
||
snapshot fields above. Built on a ratatui `TestBackend`
|
||
render-snapshot harness that asserts the text grid and per-cell
|
||
style of every `ui::draw_*` against canned `show_*` JSON.
|
||
- Steady-state log noise reduced on saturated public-mesh nodes.
|
||
Routine per-peer connection-lifecycle and capacity-cap events are
|
||
demoted from info/warn to debug — FMP K-bit cutover promotion,
|
||
connection-promoted-to-active-peer (a redundant duplicate
|
||
promotion line removed), peer-restart-detected, peer-removed-and-
|
||
cleaned-up, the TCP `max_inbound_connections`-reached rejection, and
|
||
the congestion-CE-flag line — so genuinely notable info/warn lines
|
||
are no longer drowned out. An exhausted FMP-msg1 / FSP-msg3 rekey
|
||
retransmission-budget abort (an expected, self-limiting outcome on
|
||
lossy or high-latency links) is likewise demoted from warn to debug.
|
||
- macOS UDP receive path now batches up to 32 datagrams per kernel
|
||
wakeup via `recvmsg_x(2)`, matching the Linux `recvmmsg(2)`
|
||
amortization shape introduced in v0.3.0. Previously macOS fell
|
||
through to single-packet `recv_from`, capping inbound rate on
|
||
Apple builds with the same per-syscall + per-task-wakeup overhead
|
||
Linux had already eliminated. `recvmsg_x` is an xnu-private syscall
|
||
declared via `unsafe extern "C"` against a local repr(C)
|
||
`msghdr_x`; same approach used by `quinn-udp`. Same
|
||
`(count, kernel_drops)` contract as the Linux path, with
|
||
`kernel_drops` always 0 on macOS (no `SO_RXQ_OVFL` equivalent).
|
||
Bench numbers on aarch64-apple-darwin (100B payloads, 3 s
|
||
windows): 1 sender 1.09x, 2 senders 1.72x, 4 senders 1.56x,
|
||
8 senders 1.46x.
|
||
- Receive hot path: removed two per-packet copies. New borrowed
|
||
`SessionDatagramRef` decoder is used in the forwarding handler so
|
||
local delivery and coordinate-cache warming no longer allocate or
|
||
copy the session payload; the owned `SessionDatagram` is materialized
|
||
only when re-encoding for the next hop. Owned `SessionDatagram::
|
||
decode` is reimplemented as `Ref::decode + into_owned`, so the two
|
||
decoders cannot drift. On Linux + macOS the `recvmmsg` / `recvmsg_x`
|
||
receive loop now moves each filled slot buffer into `ReceivedPacket`
|
||
via `mem::replace` instead of cloning it, and `TransportAddr` is
|
||
formatted directly from the `SocketAddr` without an intermediate
|
||
`String`. Focused decode bench: ref 1.6 ns/op vs owned 34.7 ns/op
|
||
(21.4x).
|
||
- Quieted non-Linux test-build warnings from intentionally
|
||
platform-specific code: the nftables firewall parser
|
||
(`#[allow(dead_code)]` now gated to non-Linux targets where the
|
||
parser is compiled but unused), the macOS `utun` address-family
|
||
helper and the long TUN reader entry point (narrow allowances),
|
||
and a macOS Ethernet test module's clippy struct-layout lint
|
||
(rewritten MAC-copy loop, explicit layout annotation). No
|
||
behavioral change; the goal is to keep `cargo test` and
|
||
`cargo clippy` clean on cross-platform builds so unrelated
|
||
warning fixes don't get bundled into behavioral PRs.
|
||
- Data-plane: AEAD encrypt and AEAD decrypt now run on per-shard
|
||
worker-pool threads (`std::thread` + `crossbeam_channel`), off the
|
||
rx_loop. Hash-by-destination dispatch pins each TCP flow to one
|
||
worker so wire ordering is preserved; per-worker `sendmmsg(2)`
|
||
batches up to 32 outbound packets per syscall, with UDP_GSO
|
||
(`UDP_SEGMENT`) when the batch is uniform-sized — the same kernel
|
||
primitive WireGuard's in-kernel module and Cloudflare's userspace
|
||
BoringTun use to hit multi-Gbps single-stream rates. On Linux +
|
||
macOS each established UDP peer also gets a dedicated `connect(2)`-
|
||
ed kernel socket bound to the same wildcard listen port via
|
||
`SO_REUSEPORT`, so the kernel caches per-packet route + neighbor
|
||
lookup and the worker sends with `msg_name = NULL`. The receive
|
||
side mirrors: per-shard thread-local `HashMap` owns each session's
|
||
recv cipher + replay window, replacing the previous shared
|
||
`RwLock`. Sessions are re-registered with the decrypt pool on
|
||
K-bit flip and rekey cutover, and unregistered on rekey drain
|
||
completion and peer removal so the per-shard tables stay bounded.
|
||
New `crossbeam-channel = "0.5"` dependency. Worker counts default
|
||
to `num_cpus`; both pools are overridable via
|
||
`FIPS_ENCRYPT_WORKERS` and `FIPS_DECRYPT_WORKERS` (the latter
|
||
accepts `0` to disable the pool and fall back to in-line decrypt
|
||
in rx_loop). Per-peer connected UDP can be disabled via
|
||
`FIPS_CONNECTED_UDP=0`. Optional per-stage timing reporter
|
||
available via `FIPS_PERF=1` (or `FIPS_PIPELINE_TRACE=1`); detailed
|
||
knob documentation is a follow-up at
|
||
`docs/how-to/tune-worker-pools.md`. Bench (5 × 15 s × 1 stream
|
||
medians, Linux x86_64, docker-bridge mesh): A→D 1379→2708 Mbps
|
||
(1.96×), A→E 1394→2663 Mbps (1.91×), E→A 1406→2624 Mbps (1.87×);
|
||
RTT +0.11–0.19 ms from the worker queue handoff. Windows
|
||
continues on the existing tokio-based send/recv path. Two issues in
|
||
the off-rx_loop drain path are resolved as part of the overhaul: the
|
||
per-peer drain worker is now detached on `Drop` rather than joined
|
||
synchronously (a synchronous join from the runtime thread could wedge
|
||
the whole daemon when a peer was removed with an in-flight worker),
|
||
and the connected-UDP drain no longer busy-spins on a poll error
|
||
(#106).
|
||
|
||
#### Transports & config
|
||
|
||
- Static host aliases in `/etc/fips/hosts` now hot-reload on mtime
|
||
change instead of only at daemon startup, so `fipsctl`/`fipstop`
|
||
display names reflect edits without a restart. The peer ACL and host
|
||
map both reload once per node tick through a new lock-free
|
||
`Reloadable` snapshot.
|
||
- Sidecar example (`examples/sidecar-nostr-relay`): `udp.mtu` is now
|
||
overridable via the `FIPS_UDP_MTU` environment variable, defaulting to
|
||
1472 (preserving prior behavior). Plumbed through `docker-compose.yml`
|
||
and documented in the README env-var table. Annotated the static-CI
|
||
node template `mtu: 1472` literal with the same Docker-bridge
|
||
rationale and a pointer at the daemon's 1280 default.
|
||
|
||
#### Packaging & deployment
|
||
|
||
- The Debian package no longer ships `/etc/fips/fips.yaml` as a dpkg
|
||
conf-file. The default configuration is installed as an example at
|
||
`/usr/share/fips/fips.yaml.example`, and `postinst` seeds
|
||
`/etc/fips/fips.yaml` (mode 600) from it only when the file does not
|
||
already exist — so a configuration-management-rendered or
|
||
operator-edited config is never prompted for or clobbered on
|
||
upgrade, removing the need for a `dpkg-divert` workaround.
|
||
`fips.service` gains `ConditionPathExists=/etc/fips/fips.yaml`. The
|
||
example is placed under `/usr/share/fips`, deliberately outside
|
||
`/usr/share/doc`, which minimal and container installs path-exclude
|
||
(so the install-time seed source is never dropped).
|
||
- openwrt: the `.apk` package now defaults `ethernet.wan` to the
|
||
OpenWrt 25 DSA port name `wan`; the `.ipk` package keeps `eth0` for
|
||
OpenWrt 24 and earlier.
|
||
|
||
#### CI & test-harness reliability
|
||
|
||
- CI and release-publish workflows hardened:
|
||
- `ci.yml` declares a top-level `concurrency` block keyed on
|
||
`(workflow, ref)` with `cancel-in-progress: true`. Force-pushes
|
||
and rapid successive pushes to the same ref now retire any
|
||
in-flight run rather than letting superseded and current-tip runs
|
||
both burn runner minutes.
|
||
- `aur-publish.yml` rewritten to fetch the upstream source tarball
|
||
and compute its `b2sum` in CI, then patch `pkgver` and the
|
||
`b2sums` SKIP placeholder in `PKGBUILD` in-place. Previously
|
||
`updpkgsums: true` downloaded the tarball into the AUR working
|
||
tree, where it was rejected by AUR's 488 KiB max-blob hook —
|
||
silently no-op'ing the v0.3.0 stable AUR push. `fips.sysusers` /
|
||
`fips.tmpfiles` asset b2sums are recomputed in the same step to
|
||
stay in sync with the local files. `workflow_dispatch` gains a
|
||
tag input so historical release tags can be re-published
|
||
manually, and `continue-on-error: true` is dropped so future
|
||
regressions surface in CI.
|
||
- New `aur-publish-git.yml` workflow for the `fips-git` VCS
|
||
PKGBUILD, triggered on master pushes touching `PKGBUILD-git` or
|
||
companion files plus `workflow_dispatch`. `pkgver` is computed at
|
||
build time by the PKGBUILD's `pkgver()` function, so this workflow
|
||
is not tied to release tags.
|
||
- Tag-triggered `package-*` release-build workflows remain
|
||
untouched.
|
||
- Local and GitHub CI integration coverage brought into parity, and
|
||
the Rust toolchain selection given a single source of truth:
|
||
- The `admission-cap` integration suite, previously run only by
|
||
`ci-local.sh`, now also runs as a GitHub `ci.yml` matrix leg, so a
|
||
regression in it turns the GitHub gate red rather than depending on
|
||
a developer remembering to run local CI. A new
|
||
`testing/check-ci-parity.sh` (wired as `ci-local.sh
|
||
--check-parity`) diffs the two runners' integration-suite sets and
|
||
fails on unexpected drift; the deliberate local-only (live-Tor)
|
||
and granularity-only differences are documented in a comment block
|
||
atop both runners.
|
||
- CI and packaging jobs now select the toolchain with
|
||
`actions-rust-lang/setup-rust-toolchain` (which reads
|
||
`rust-toolchain.toml`) instead of `dtolnay/rust-toolchain@stable`.
|
||
The pinned channel already overrode the installed stable, so each
|
||
job downloaded an unused toolchain and logged a misleading `rustc`
|
||
version; the single-source action removes the waste and the
|
||
confusion. Existing cache steps are kept (`cache: false` on the
|
||
new action) and `RUSTFLAGS` is left untouched so no global
|
||
`-D warnings` is newly imposed. The OpenWrt nightly Tier-3 leg
|
||
keeps `@nightly`.
|
||
|
||
#### Docs & contributor tooling
|
||
|
||
- Overhauled `CONTRIBUTING.md`: replaced generic Rust-template framing
|
||
with a FIPS-specific entry point covering the four-layer
|
||
architecture, branch model and PR-target selection, structured bug
|
||
reporting, scope discipline and local-CI requirements, an AI coding
|
||
assistant policy, and project communication channels. Added
|
||
`docs/branching.md` as the long-form companion covering the release
|
||
workflow, version conventions, and merge-direction rationale.
|
||
|
||
### Fixed
|
||
|
||
#### FMP/FSP rekey reliability
|
||
|
||
- FMP link-layer rekey is now reliable under packet loss, bringing it up
|
||
to the FSP session layer's rekey discipline. The rekey msg1
|
||
retransmission driver was previously uncapped and never abandoned, so a
|
||
rekey that never completed resent msg1 forever; it now uses a bounded
|
||
retransmission budget (`handshake_max_resends` with exponential
|
||
backoff) and abandons the rekey cycle cleanly once the budget is
|
||
exhausted, mirroring the FSP rekey msg3 driver. With the cap in place
|
||
the link-dead heartbeat is rekey-aware: `check_link_heartbeats` no
|
||
longer reaps a link that is still actively carrying rekey-handshake
|
||
traffic, while a genuinely dead link is still reaped once the budget
|
||
abandons. At the K-bit cutover the receiver now authenticates an
|
||
inbound frame against the pending session before promoting it, instead
|
||
of promoting on the bare header K-bit; under jitter a node could
|
||
otherwise promote a stale pending session, leaving the two endpoints on
|
||
different keys and silently dropping traffic until the link died — the
|
||
same failure class already closed on FSP, now closed on FMP.
|
||
- FSP session rekey is now hitless under packet loss and reordering.
|
||
Previously, a rekey could leave the two endpoints holding different
|
||
key sets for a brief window — if a handshake message was lost in
|
||
transit one side rotated keys while the other did not, and traffic
|
||
sealed in one key epoch reached a peer still on the other epoch and
|
||
failed to decrypt, producing bursts of AEAD decryption failures and
|
||
dropped connectivity until a later rekey reconverged the pair. The
|
||
receive path now trial-decrypts each frame against every live key
|
||
epoch (current, pending, and the draining previous session) for the
|
||
duration of the rekey transition, so no rotation ordering and no
|
||
packet reordering can cause a decryption failure. The previous-epoch
|
||
slot is retained as long as the peer keeps using it, with its drain
|
||
deadline anchored on the last frame the peer authenticates against
|
||
it rather than a fixed wall-clock timer, so a peer that did not
|
||
receive the new keys is not stranded by a silent permanent decrypt
|
||
failure. The lost-handshake case is closed by retransmitting the
|
||
third rekey handshake message until the peer is confirmed on the
|
||
new keys, with a bounded retry budget after which the rekey cycle
|
||
is cleanly abandoned and retried. There are no FSP decryption
|
||
failures across a rekey under lossy, jittery links.
|
||
- ±15s symmetric jitter is applied per session to the FMP and FSP rekey
|
||
timer trigger, eliminating the steady-state dual-initiation race in
|
||
symmetric-start meshes (previously the smaller-NodeAddr tie-breaker
|
||
resolved correctness only after every cycle's collision).
|
||
`node.rekey.after_secs` becomes the nominal interval rather than a
|
||
floor; the mean is preserved.
|
||
- A stale FSP (session-layer) session is now cleared when a peer
|
||
restart is detected during FMP rekey or cross-connection promotion.
|
||
Previously the old session could linger after the peer came back
|
||
with a new startup epoch, leaving the session-layer map out of sync
|
||
with the freshly promoted peer.
|
||
|
||
#### NAT traversal / Nostr discovery
|
||
|
||
- Two nodes that each `auto_connect` to the other no longer stall their
|
||
Nostr-mediated NAT-traversal handshake. Each side ran both an
|
||
initiator and a responder traversal session, binding a separate UDP
|
||
socket per session, and adopted only the first `Established` event; if
|
||
the two sides adopted mismatched sessions, each sent its Noise msg1 to
|
||
a peer port the peer had already stopped draining and both handshakes
|
||
hung until the adoption budget expired. The responder now elects a
|
||
single session deterministically — it declines an incoming offer only
|
||
when it also has an in-flight outbound initiator for the same peer and
|
||
its own NodeAddr is smaller — so one matching socket pair survives on
|
||
both ends and the peer's redundant initiator times out harmlessly.
|
||
One-sided (asymmetric) `auto_connect` has no co-active initiator and is
|
||
never suppressed, so connectivity is preserved.
|
||
- NAT-traversal cross-init adoption is now deterministic under
|
||
simultaneous dual-initiation. Previously, when two peers'
|
||
Nostr-mediated UDP punches completed within the same scheduling
|
||
window, each side's bootstrap-completion event arrived with an
|
||
in-flight handshake already recorded against the other peer (each
|
||
side had received an inbound msg1 from the other's pre-punch
|
||
outbound attempt). The deduplication skip then fired on both
|
||
sides, neither installed the fresh traversal socket as canonical,
|
||
and the 45-second peer-adoption budget expired with both nodes
|
||
stuck waiting for an adoption that never happened. The handler now
|
||
applies the same deterministic NodeAddr tie-breaker the codebase
|
||
already uses for rekey dual-initiation and cross-connection
|
||
resolution: the smaller NodeAddr wins as adopter, tears down its
|
||
in-flight handshake state, and proceeds with adoption; the larger
|
||
NodeAddr keeps the skip semantics, and its in-flight outbound is
|
||
reconciled by the cross-connection logic when the winner's fresh
|
||
msg1 arrives over the adopted socket. The dual cross-init stall is
|
||
eliminated; cross-init NAT-traversal completes in well under a
|
||
second even under host CPU contention.
|
||
- Nostr-discovered NAT-traversal events (`BootstrapEvent::Established`
|
||
and `BootstrapEvent::Failed`) for peers that are already connected
|
||
or actively handshaking are now short-circuited at the
|
||
`poll_nostr_discovery` dispatch sites before any cooldown
|
||
bookkeeping or fallback retry scheduling runs. Stale `Failed` events
|
||
previously poisoned the per-peer failure-state cooldown of healthy
|
||
peers and could trigger redundant retraversal attempts via
|
||
`schedule_retry` / `try_peer_addresses`; stale `Established`
|
||
handoffs could attempt to adopt a second socket against a live
|
||
connection. A defense-in-depth guard was added to
|
||
`adopt_established_traversal` so the same invariant holds if a
|
||
future caller bypasses the outer dispatch check. As a side benefit,
|
||
narrows a cooldown-poisoning vector previously available to an
|
||
attacker injecting stale failure events for an active peer.
|
||
- Nostr discovery now filters unroutable direct UDP/TCP advert
|
||
endpoints. Publisher and validator retain only endpoints that parse as
|
||
concrete socket addresses with routable IPs and nonzero ports;
|
||
`udp:nat` rendezvous endpoints and Tor endpoints pass through
|
||
unchanged. Adverts that collapse to zero usable endpoints after
|
||
filtering are rejected with a clear "missing publicly routable
|
||
endpoints" error. Before this change, misconfigured nodes could
|
||
publish RFC1918, loopback, link-local, CGNAT 100.64/10, IPv6 ULA,
|
||
or IPv6 link-local endpoints into Nostr discovery, and consumers
|
||
would cache and dial them; in mixed LAN/VPN/NAT environments, that
|
||
could prefer a misleading one-way private path over the intended
|
||
`udp:nat` bootstrap.
|
||
|
||
#### Admission / peer caps
|
||
|
||
- TCP and Tor `max_inbound_connections` admission cap is now compared
|
||
against the per-direction inbound count (`pool_inbound`) rather than
|
||
the combined pool size. Outbound connect-on-send connections share
|
||
the same pool data structure but no longer consume slots against the
|
||
operator-facing inbound cap. The configuration field name and
|
||
operator semantics are preserved; only the cap-check comparison and
|
||
accounting change. Operators with mixed outbound + inbound
|
||
deployments no longer see legitimate inbound peers rejected once
|
||
outbound connections fill the pool past the configured cap.
|
||
- Outbound connection initiation now honors the `node.limits.max_peers`
|
||
cap that was previously only checked on inbound msg1 admission. Four
|
||
paths gated: auto-reconnect retries (`process_pending_retries`),
|
||
Nostr-mediated discovery's `BootstrapEvent::Established` adoption, and
|
||
both sides of the Nostr-mediated NAT-traversal punch (offer initiation
|
||
in the runtime's outgoing path, offer acceptance in the responder's
|
||
incoming-offer handler). At saturation, a node now performs zero
|
||
outbound work on these paths; only existing peer maintenance and
|
||
overlay-advert refresh continue. The inbound gate at
|
||
`handshake.rs:1114` is unchanged. Introduces a shared
|
||
`Node::outbound_admission_check()` helper so the invariant is
|
||
grep-able and unit-testable.
|
||
- Inbound `handle_msg1` now silent-drops at `node.limits.max_peers`
|
||
saturation *before* building/sending Msg2, instead of replying with
|
||
Msg2 and then rejecting at `promote_connection`. Adds an early cap
|
||
check positioned after identity verification (so the
|
||
reconnect / cross-connection bypass for known peers still fires) and
|
||
before index allocation + Msg2 wire send. The late cap check inside
|
||
`promote_connection` is intentionally retained as
|
||
defense-in-depth. Wire savings observed in a 45 s tcpdump at
|
||
saturation: ~3.6 cap-denials/s × Msg2 (~104 B + AEAD compute) each.
|
||
Bigger win is cleaner peer-side semantics — no fake-completed
|
||
handshake whose subsequent data frames fail decryption on this side.
|
||
|
||
#### Spanning-tree / mesh-size / routing
|
||
|
||
- The mesh-size estimator (`compute_mesh_size`) no longer over-counts
|
||
under filter overlap. It previously summed the per-filter cardinality
|
||
of the parent and each child filter, which assumes the filters are
|
||
perfectly disjoint; a stale or oversized parent filter or a routing
|
||
loop inflated the reported mesh size to several times the true value,
|
||
and dropping the parent on a tree rebalance collapsed the upward leg
|
||
and flapped the count (the symptom operators saw as the size
|
||
nearly-but-not-exactly doubling during rebalancing). The estimator now
|
||
computes the cardinality of the OR-union over self plus every
|
||
connected peer's inbound filter, dropping the parent/child tree gating
|
||
entirely. OR is idempotent, so any overlap is deduplicated — the
|
||
result equals the old sum in the disjoint case, stays correct under
|
||
overlap, damps the parent-switch flap, and removes the estimate's
|
||
dependence on tree-declaration cache freshness. The per-peer 500 ms
|
||
rate-limiter and overall recompute cadence are unchanged.
|
||
- Spanning-tree state distribution is now eventually-consistent.
|
||
Previously every `send_tree_announce_to_all` call site fired only
|
||
on a local state-change event (parent switch, self-root promotion,
|
||
ancestry change, peer promotion, parent loss). Once a partition
|
||
latched — for example, a parent-switch announce lost in transit
|
||
via the brief cross-init handshake swap window where one peer's
|
||
outbound session is about to become the loser session and the
|
||
receiver has no matching decrypt-worker entry — no node's state
|
||
changed again, so no node ever re-broadcast. The existing 60-second
|
||
`check_periodic_parent_reeval` short-circuited silently on no-change
|
||
(it was a re-evaluation, not a re-broadcast), and production-side
|
||
healing depended on incidental link churn (NAT keepalive refresh,
|
||
MMP timeout, peer re-promotion after a transport blip). The
|
||
function now ends with an unconditional `send_tree_announce_to_all`
|
||
on the no-change branch, alongside the existing switch and
|
||
self-promote arms; receivers coalesce by sequence comparison
|
||
(`ParentDeclaration::is_fresher_than`) and short-circuit at the
|
||
`if !updated` gate in `handle_tree_announce`, so same-sequence
|
||
repeats drop silently with no cascade. The per-peer 500 ms
|
||
rate-limiter is well below this 60-second cadence and does not
|
||
suppress the heartbeat broadcast. `BASELINE_CONVERGENCE_TIMEOUT`
|
||
in `testing/static/scripts/rekey-test.sh` is bumped from 60 to 65
|
||
so any partition healed by the periodic broadcast at T+60 lands
|
||
inside the convergence window; `wait_for_full_baseline` early-exits
|
||
on PASS, so successful reps see no extra wall-clock.
|
||
- A single-uplink node stranded out of the tree now re-attaches within
|
||
a round-trip instead of waiting for the periodic re-broadcast cadence.
|
||
A node with one tree peer has periodic parent re-evaluation disabled,
|
||
so a lost one-shot attaching `TreeAnnounce` left it self-rooted and
|
||
unreachable until the next periodic re-broadcast
|
||
(`reeval_interval_secs` later). Tree-position exchange is now
|
||
self-healing on the receive path: when an accepted `TreeAnnounce`
|
||
advertises a root strictly worse (higher NodeAddr; election is
|
||
smallest-wins) than our own, we echo our current declaration back to
|
||
that peer, provoking the better-rooted peer to re-push its real
|
||
position immediately. The echo fires only in that one direction and is
|
||
bounded by the existing per-peer rate limiter.
|
||
- Coord cache invalidation made surgical at parent-position-change
|
||
and root-change sites. Replaces the previous unconditional
|
||
`CoordCache::clear()` calls with two targeted methods:
|
||
`invalidate_via_node(node_addr)` (drops entries whose cached
|
||
ancestry contains the changed node, used at parent-switch /
|
||
become-root / loop-detection sites) and `invalidate_other_roots`
|
||
(drops entries from a different tree, used at root-change sites).
|
||
The previous global flush left `find_next_hop` returning `None`
|
||
for every non-direct-peer destination after every parent switch
|
||
until the cache passively re-warmed; surgical invalidation
|
||
preserves entries that remain correct across the topology change.
|
||
Peer-removal retains the original "no invalidation" behavior
|
||
(`find_next_hop` already recomputes against the current peer set
|
||
every call, and Discovery handles "no route" on demand).
|
||
- `rx_loop` tick-arm stall under convergence-phase mesh pressure
|
||
is eliminated. Previously, the tick body's per-peer `check_*`
|
||
loops (heartbeats, bloom announces, MMP reports, tree announces)
|
||
called `transport.send` directly for every active peer. For
|
||
TCP/Tor peers whose pool entry was not yet established,
|
||
`send_async` fell through to a synchronous connect-on-send
|
||
branch that wrapped `TcpStream::connect` in
|
||
`tokio::time::timeout(connect_timeout_ms, …)` — 5 seconds by
|
||
default — and blocked the entire tick body for the duration per
|
||
unreachable peer. Under post-restart convergence on a high-peer
|
||
mesh, this cascaded into multi-second tick stalls; the same
|
||
mechanism also starved the master-only per-tick control-snapshot
|
||
republish and pushed `fipsctl show *` queries onto an mpsc
|
||
fallback that was itself queued behind the wedged `rx_loop`,
|
||
producing the five-second `fipsctl` head-of-line pattern
|
||
operators observed on loaded nodes. The send path now gates on
|
||
`transport.connection_state(addr)` before sending: proceed only
|
||
when `Connected`; on `None`, kick off a non-blocking background
|
||
`connect` (idempotent — deduplicates against the connecting
|
||
pool, spawns the timeout-bounded `TcpStream::connect` inside its
|
||
own tokio task) and fail this send fast with a clear
|
||
`transport connection not ready` error. A subsequent tick
|
||
retries once the pool has an entry. The existing reconnect
|
||
lifecycle (heartbeat-dead detection in `check_link_heartbeats`,
|
||
scheduled retries via `process_pending_retries`, background-
|
||
connect polling via `poll_pending_connects`) is unchanged.
|
||
The connect-on-send branch in `transport.send_async` itself
|
||
remains in place for code paths that legitimately need
|
||
synchronous connect (e.g., explicit operator-driven
|
||
`fipsctl connect`); the tick path just no longer trips it.
|
||
|
||
#### Data-plane / metrics / observability
|
||
|
||
- The Tor transport now increments its `connect_refused` statistic (the
|
||
"Refused" line in fipstop) when a SOCKS5 connection is actively
|
||
refused, instead of recording every connect failure as a generic
|
||
SOCKS5 error. The counter previously stayed at zero.
|
||
- MMP sender metrics now ignore duplicate or regressed receiver reports
|
||
before updating RTT, loss, goodput, or ETX. Receiver reports also
|
||
suppress timestamp echo when dwell time overflows, so stale reports
|
||
cannot inflate SRTT.
|
||
- Reject-reason counters no longer double-count now that the rollout's
|
||
interim direct increments are removed. Six discovery counters
|
||
(`req_decode_error`, `req_duplicate`, `req_ttl_exhausted`,
|
||
`resp_decode_error`, `resp_identity_miss`, `resp_proof_failed`), six
|
||
bloom counters (`decode_error`, `invalid`, `non_v1`, `unknown_peer`,
|
||
`stale`, `fill_exceeded`), and five forwarding reject packet counters
|
||
(`decode_error_packets`, `ttl_exhausted_packets`,
|
||
`drop_no_route_packets`, `drop_mtu_exceeded_packets`,
|
||
`drop_send_error_packets`) were each incremented both by a direct bump
|
||
and again through the typed reject dispatch. The redundant direct
|
||
increments are removed — for the forwarding family the two calls are
|
||
collapsed into a single byte-aware reject entry point — so each counter
|
||
(and, for forwarding, its byte tally) counts once per event.
|
||
- Transport-layer mutex poisoning no longer cascades. Ten
|
||
`Mutex::lock().unwrap()` sites across the UDP, BLE, and Ethernet
|
||
transports would turn a single panic (poisoning the mutex) into a
|
||
cascade of panics on every subsequent lock. Each is replaced with
|
||
`lock().unwrap_or_else(|e| e.into_inner())`, recovering the guarded
|
||
data with no new dependency and no call-graph change; four
|
||
`local_addr.unwrap()` calls on the UDP start/adopt paths get a
|
||
provably-safe sentinel fallback. The critical sections are short,
|
||
locally-scoped, and not reachable from peer input, so this is
|
||
robustness hardening, not a remotely-triggerable fix.
|
||
|
||
#### Peer lifecycle / gateway
|
||
|
||
- A manual `fipsctl disconnect` now notifies the peer so teardown is
|
||
symmetric. Previously a manual disconnect tore down only the local
|
||
side and sent the peer nothing, so the peer kept its session and never
|
||
re-emitted its tree and filter announcements; on reconnect it was
|
||
never re-adopted as a child and its bloom filter was never recorded.
|
||
The local side now sends the disconnected peer a scoped `Disconnect`
|
||
(the same message graceful shutdown sends), so both ends tear down and
|
||
re-handshake cleanly on the next connection.
|
||
- `fips-gateway` no longer drops long-lived or DNS-cached client
|
||
mappings while traffic is still flowing. The virtual-IP pool's TTL
|
||
clock advanced only on DNS re-query, never on traffic, and the mapping
|
||
TTL is wired equal to the DNS TTL, so an in-use mapping was forced to
|
||
drain at TTL and reclaimed at the first zero-conntrack tick — breaking
|
||
long-lived, bursty, or DNS-cached clients. The tick now refreshes the
|
||
mapping's last-referenced time whenever conntrack reports active
|
||
sessions, and recovers a draining mapping to active (with a fresh
|
||
grace window) when traffic resumes; only genuinely idle mappings
|
||
drain.
|
||
|
||
#### macOS self-traffic / resolver
|
||
|
||
- Self-addressed mesh traffic is now delivered locally on macOS instead
|
||
of being dropped, for both `ping6` and full TCP/UDP. The point-to-point
|
||
`utun` interface egresses self-addressed traffic into the daemon, which
|
||
previously pushed it onto the mesh outbound path where it was dropped
|
||
for lack of a route to self; such packets are now hairpinned back to
|
||
the TUN for inbound delivery. macOS first routes self-addressed packets
|
||
as loopback (a `LOCAL` route via `lo0`), which leaves their transport
|
||
TX checksum offloaded and unfinished, so re-injecting them verbatim
|
||
made the local stack drop every segment whose checksum MSS clamping did
|
||
not happen to rewrite (the SYN and SYN-ACK got through, but the bare
|
||
ACK, data, and FIN were dropped, so connections to a node's own
|
||
`<npub>.fips` service half-opened and hung). The hairpin path now
|
||
recomputes the TCP/UDP checksum before re-injection, so full
|
||
self-connections — not just `ping6` — to a node's own `<npub>.fips`
|
||
address work. Linux was unaffected (the kernel already loops
|
||
self-traffic via `lo`). (#117)
|
||
- macOS `.fips` name resolution now works on a fresh install: the
|
||
shipped resolver shim points at `::1`, matching the daemon's default
|
||
IPv6 DNS listener, instead of `127.0.0.1`. The mismatched shim
|
||
(`nameserver 127.0.0.1` while the daemon listens on `::1`) broke
|
||
`getaddrinfo` for `.fips` on every macOS install since the resolver
|
||
was introduced.
|
||
|
||
#### CI & test-harness reliability
|
||
|
||
- Node-level multi-node tests no longer flake under parallel CPU load.
|
||
They previously delivered handshake packets over real localhost UDP,
|
||
whose kernel receive buffer could overflow and drop a packet when many
|
||
tests ran concurrently, panicking the large-network convergence tests.
|
||
A `cfg(test)`-only loopback `TransportHandle` variant now delivers
|
||
packets directly between nodes over an unbounded in-process channel, so
|
||
there is no socket buffer to overflow, and the previously-quarantined
|
||
large-network tests run in the default suite again. The shipping daemon
|
||
build is unaffected (the variant is test-gated).
|
||
- Integration suites that wait for the mesh to converge no longer
|
||
false-fail under concurrent CI load. The rekey, static-mesh, and
|
||
sidecar suites replace a fixed wall-clock baseline timeout (and a blind
|
||
sleep) with a progress-aware wait that polls the suite's own pairwise
|
||
pings, returns as soon as every pair is reachable, extends its deadline
|
||
while the reachable-pair count is still climbing, and gives up only
|
||
when progress stalls.
|
||
- Rekey integration test (`testing/static/scripts/rekey-test.sh`) no
|
||
longer false-fails on GitHub runners under packet loss and CPU
|
||
contention. Phase 1, Phase 3, and Phase 5 strict per-pair pings retry
|
||
up to 4 attempts (configurable via `MAX_PING_ATTEMPTS` /
|
||
`PING_RETRY_DELAY`) — under 1% per-direction loss, single-shot 20-pair
|
||
ping_all misses ~33% per phase from ICMP noise alone, and the
|
||
4-attempt retry brings that floor to ~3.2e-6 per phase; the
|
||
`wait_for_full_baseline` convergence loop stays single-shot so retries
|
||
there cannot conflate transient ping loss with still-converging routing
|
||
state. Phase 1 baseline-convergence headroom is bumped from 36s to 60s
|
||
to eliminate the intermittent Phase 1 timeout that previously required
|
||
a `gh run rerun --failed`, and a post-second-rekey settle window is
|
||
added in Phase 5 (mirroring Phase 3's 12-second pattern) to close the
|
||
post-rekey per-pair-ping flake from convergence exceeding the per-ping
|
||
5-second timeout. Test scaffold only; no daemon code changes, and the
|
||
success path is unchanged because the wait loops return as soon as all
|
||
20 pairs converge.
|
||
- ACL-allowlist integration test (`testing/acl-allowlist/test.sh`):
|
||
converted `assert_log_contains` from a one-shot `docker logs | grep`
|
||
snapshot into a bounded poll with the same wait-with-timeout shape
|
||
as `wait_for_peers_exact`. Absorbs the millisecond-to-second
|
||
variance in the XX-handshake cross-connection tie-breaker: the
|
||
inbound-handshake-context rejection can land tens of milliseconds
|
||
after the test's previous one-shot grep gave up, producing a
|
||
pre-existing flake on CI. Success-path cost is unchanged — the helper
|
||
returns as soon as the pattern appears.
|
||
|
||
#### Packaging & deployment
|
||
|
||
- AUR packaging: the `fips` and `fips-git` PKGBUILDs now install the
|
||
`fips-dns-setup` and `fips-dns-teardown` helpers into
|
||
`/usr/lib/fips/`, matching the Debian package. The AUR `package()`
|
||
step previously omitted them, so `fips-dns.service` failed to
|
||
start on Arch installs ("Unable to locate executable
|
||
`/usr/lib/fips/fips-dns-setup`", #98). The PKGBUILDs additionally
|
||
opt out of the debug split package and declare the `*-debug`
|
||
variant as a conflict, so a stale debug build cannot own installed
|
||
files across a package switch.
|
||
- macOS package build: the `.pkg` architecture is now derived from
|
||
the Cargo `--target` triple instead of the build host's
|
||
`uname -m`. The arm64 and x86_64 release legs build on the same
|
||
Apple-silicon runner, so `uname -m` named both outputs
|
||
`fips-0.3.0-macos-arm64.pkg`; the release job's `merge-multiple`
|
||
artifact download then interleaved the two identically named
|
||
files into a single corrupt xar archive, and no x86_64 package
|
||
reached the release at all. (This shipped as the broken v0.3.0
|
||
macOS `.pkg`, GitHub #102.) The release workflow now also asserts
|
||
the arch-named file is present and carries a SHA-256 integrity
|
||
chain from the build runner through to `gh release upload`, so a
|
||
recurrence fails CI instead of publishing.
|
||
|
||
#### fipstop
|
||
|
||
- `fipstop` no longer renders a garbled screen on startup or leaves
|
||
stray bytes on quit, most visible over SSH and inside tmux. Startup
|
||
forces a full repaint (`terminal.clear()`) before the first draw so
|
||
prior alternate-screen contents no longer show through; quit gives the
|
||
stdin-poll thread a stop flag and joins it before restoring the
|
||
terminal, so post-raw-mode keystrokes or terminal query responses no
|
||
longer echo onto the restored screen.
|
||
|
||
## [0.3.0] - 2026-05-11
|
||
|
||
### Added
|
||
|
||
#### Mesh Layer (FMP)
|
||
|
||
- Overlay-discovery and NAT-hole-punching path (opt-in via
|
||
`node.discovery.nostr.enabled`). Nodes publish signed overlay adverts
|
||
as Nostr kind `37195` parameterized replaceable events listing
|
||
reachable transport endpoints to a configurable set of public relays,
|
||
and consume peer adverts to populate fallback addresses for
|
||
`via_nostr` peers or, under `policy: open`, for non-configured peers
|
||
within a budget cap. The kind value is FIPS-specific: `37195` sits in
|
||
the application-defined replaceable range `30000–39999`, and the
|
||
digits visually spell `FIPS` (7=F, 1=I, 9=P, 5=S)
|
||
- STUN-assisted UDP hole punching for `addr: "nat"` UDP endpoints. STUN
|
||
reflexive observation, gift-wrap (NIP-59) offer/answer signaling, and
|
||
candidate-pair punch planner (LAN-private + reflexive paths attempted in
|
||
parallel). Successful punches hand the live socket into the standard
|
||
FIPS UDP transport via a bootstrap-handoff API
|
||
- New `node.discovery.nostr.*` configuration tree with operator-tunable
|
||
resource caps, replay tracking, and punch timing; new `peers[].via_nostr`
|
||
and per-transport `advertise_on_nostr` / `public` flags. Cross-field
|
||
validation at startup catches mis-configured combinations
|
||
- Docker NAT lab covering cone, symmetric (TCP-fallback), and LAN
|
||
scenarios, wired into the integration CI matrix
|
||
- One-shot startup advert sweep for Nostr open-discovery. On daemon
|
||
startup under `node.discovery.nostr.policy: open`, after a short
|
||
settle delay (`startup_sweep_delay_secs`, default 5s) the cached
|
||
overlay-advert table is iterated once and recent adverts (newer
|
||
than `startup_sweep_max_age_secs`, default 3600s) are queued for
|
||
outbound retry, modulo the same skip-filters as the per-tick sweep
|
||
(configured peer, already connected, retry-pending, connecting).
|
||
Closes the gap where peers learned only through relay backlog at
|
||
startup were not dialed until they republished.
|
||
- Diagnostic logging on the open-discovery sweep. Each `queued retry`
|
||
now logs at info-level with the peer short-npub and advert age,
|
||
and a one-line summary (cached count, queued count, per-reason
|
||
skip counts) is emitted on every startup sweep and on any per-tick
|
||
sweep that queues at least one retry. Operator-facing visibility
|
||
into what the auto-dial path is doing.
|
||
|
||
#### Platform Support
|
||
|
||
- Windows platform support: wintun TUN device, TCP control socket on
|
||
`localhost:21210` (in place of the Unix domain socket), Windows
|
||
Service lifecycle (`--install-service`, `--uninstall-service`,
|
||
`--service`), ZIP packaging with PowerShell install/uninstall scripts,
|
||
and CI build/test matrix entry
|
||
([#45](https://github.com/jmcorgan/fips/pull/45))
|
||
- macOS platform support: native `utun` TUN interface management, raw
|
||
Ethernet transport via BPF, `.pkg` packaging with launchd plist and
|
||
uninstall script, x86_64 cross-compile from arm64, and CI build/unit
|
||
test jobs
|
||
- MIPS atomic ABI support: `std::sync::atomic` replaced with
|
||
`portable_atomic` so 32-bit MIPS targets without native atomics
|
||
link cleanly
|
||
([#62](https://github.com/jmcorgan/fips/pull/62),
|
||
[@andrewheadricke](https://github.com/andrewheadricke)).
|
||
|
||
#### Mesh Peer Transports
|
||
|
||
- Bluetooth Low Energy (BLE) L2CAP Connection-Oriented Channel
|
||
transport (Linux only, requires BlueZ): per-link MTU negotiation,
|
||
continuous scan/probe peer discovery with cooldown-based
|
||
deduplication, continuous advertising, deterministic NodeAddr
|
||
cross-probe tie-breaker, and a configurable connection pool with
|
||
eviction.
|
||
- `transports.udp.outbound_only` (default `false`). When true, the UDP
|
||
transport binds a kernel-assigned ephemeral port (`0.0.0.0:0`) instead
|
||
of the configured `bind_addr`, refuses inbound handshakes, and is
|
||
never advertised on Nostr regardless of `advertise_on_nostr`. Use
|
||
this to participate in the mesh as a pure client — initiate outbound
|
||
links without exposing an inbound listener on a known port.
|
||
Implements the long-form fix for `udp.bind_addr: "127.0.0.1:..."`
|
||
not actually working as a workaround (Linux pins the loopback source
|
||
IP, dropping outbound flows to external peers at the routing layer)
|
||
- `transports.udp.accept_connections` (default `true`). Mirrors the
|
||
Ethernet/BLE knob; setting to `false` produces a "client" posture
|
||
(initiate outbound, refuse inbound msg1 from new addresses). The
|
||
Node-level handshake gate carves out msg1 from peers already
|
||
established on this transport so rekey continues to work. Affects
|
||
every transport via the `Transport` trait
|
||
- Startup validation now rejects `transports.udp[*].bind_addr` set to a
|
||
loopback address when at least one peer has a non-loopback UDP
|
||
address. Replaces the silent "peer link won't establish" failure
|
||
mode where Linux's source-address routing check dropped outbound
|
||
flows from the loopback-bound socket. `outbound_only: true` is
|
||
exempt from the check (it overrides `bind_addr` to `0.0.0.0:0`)
|
||
|
||
#### Security
|
||
|
||
- Mesh-interface nftables baseline (Linux). Ships `/etc/fips/fips.nft`
|
||
as a documented operator conffile and `fips-firewall.service`
|
||
(disabled by default) for default-deny inbound on the `fips0` mesh
|
||
interface. Operators enable explicitly with
|
||
`systemctl enable --now fips-firewall.service`. Drop-ins in
|
||
`/etc/fips/fips.d/*.nft`. See `docs/fips-security.md`.
|
||
- Peer access control list enforcement: optional
|
||
`/etc/fips/peers.allow` and `/etc/fips/peers.deny` files
|
||
(TCP-Wrappers style) gate outbound connect, inbound msg1, and
|
||
outbound msg2 against npub, hex pubkey, host alias, or `ALL`.
|
||
Files are reloaded automatically on mtime change. New
|
||
`fipsctl acl show` query reports the effective rule set
|
||
([#50](https://github.com/jmcorgan/fips/pull/50),
|
||
[@alexxie16](https://github.com/alexxie16)).
|
||
|
||
#### LAN Gateway
|
||
|
||
- New `fips-gateway` binary that lets unmodified LAN hosts reach FIPS
|
||
mesh destinations via DNS-allocated virtual IPs and kernel nftables
|
||
NAT. Virtual-IP pool (`fd01::/112` by default) with state-machine
|
||
lifecycle and TTL-based reclamation; conntrack-backed session
|
||
tracking; proxy NDP on the LAN interface; control socket at
|
||
`/run/fips/gateway.sock` with `show_gateway` and `show_mappings`;
|
||
fipstop Gateway tab with pool gauge and mappings table; design doc
|
||
at `docs/design/fips-gateway.md`; integration test harness
|
||
- Inbound mesh port forwarding on `fips-gateway`: new
|
||
`gateway.port_forwards` config (list of `{ listen_port, proto,
|
||
target }` entries, IPv6 targets only) installs prerouting DNAT
|
||
rules so mesh peers can reach a configured host:port on the
|
||
gateway's LAN. A LAN-side masquerade is added when any forwards
|
||
are configured so replies flow back through conntrack.
|
||
- Gateway packaging: systemd service unit with `After=fips.service`,
|
||
Debian and AUR package entries, OpenWrt procd init with dnsmasq
|
||
forwarding, proxy NDP, RA route advertisements, and IPv6 forwarding
|
||
sysctls. Gateway enabled by default on OpenWrt
|
||
- `fips-gateway` DNS upstream probe now retries up to 5 times with a
|
||
1-second per-attempt timeout and a 1-second delay between attempts
|
||
(~10 second worst-case wait), instead of a single 3-second hard-fail.
|
||
Covers the cold-boot race where the daemon's TUN is up (the systemd
|
||
ExecStartPre wait gates on that) but the DNS responder is still
|
||
binding `[::1]:5354`. Without retry the gateway exited and relied on
|
||
`Restart=on-failure` for recovery (5-second blip + spurious error
|
||
log line per cycle); with retry the gateway recovers gracefully
|
||
without a unit restart
|
||
|
||
#### IPv6 Adapter
|
||
|
||
- Overhauled `.fips` DNS handling for systemd-based hosts. The
|
||
default `dns.bind_addr` is `::1` (IPv6 loopback) and the setup
|
||
script picks one of five backends in priority order: a global
|
||
drop-in at `/etc/systemd/resolved.conf.d/fips.conf`, the systemd
|
||
dns-delegate path, `resolvectl` per-link, standalone dnsmasq, or
|
||
NetworkManager's dnsmasq plugin. Teardown reverses only what was
|
||
applied. New `testing/dns-resolver/` harness exercises every
|
||
backend across Debian 12, Debian 13, Ubuntu 22.04, Ubuntu 24.04,
|
||
and Ubuntu 26.04
|
||
([#58](https://github.com/jmcorgan/fips/pull/58),
|
||
fixes [#52](https://github.com/jmcorgan/fips/issues/52),
|
||
[#77](https://github.com/jmcorgan/fips/issues/77)).
|
||
|
||
#### Operator Tooling
|
||
|
||
- `node.log_level` config field (case-insensitive, default `info`)
|
||
replaces the hardcoded `RUST_LOG=info` previously baked into
|
||
systemd units and the OpenWrt procd init script. The daemon now
|
||
loads config before initializing tracing so the configured level
|
||
takes effect; `RUST_LOG` still overrides when set
|
||
- `fipsctl show identity-cache` lists every cached node identity
|
||
(npub, IPv6 address, display name, LRU age) alongside the
|
||
configured cache capacity
|
||
- `fipsctl show peers` extended with per-peer security signals
|
||
(replay suppression count, consecutive decrypt failures), Noise
|
||
session counters, session indices, and rekey lifecycle state
|
||
- `fipsctl show sessions` extended with handshake resend count
|
||
during establishment and rekey/session health fields when
|
||
established (session start, K-bit epoch, coords warmup remaining,
|
||
drain state)
|
||
- `fipsctl show cache` now includes individual coordinate cache
|
||
entries (tree coordinates, depth, path MTU, age). The top-level
|
||
count field was renamed from `entries` to `count` for clarity
|
||
- `fipsctl show routing` expands `pending_lookups` from a count to
|
||
per-target detail (attempt, age, last sent), adds pending TUN
|
||
packet queue depth, and adds per-peer connection retry state
|
||
([#42](https://github.com/jmcorgan/fips/pull/42),
|
||
[@osh](https://github.com/osh))
|
||
- Historical node and per-peer statistics: in-memory time-series
|
||
rings on the daemon, surfaced through new control-socket queries,
|
||
`fipsctl stats` subcommands, and a `fipstop` Graphs tab with
|
||
btop-style sparklines
|
||
([#64](https://github.com/jmcorgan/fips/pull/64)).
|
||
- `fipstop` Node tab now carries a "Listening on fips0" panel
|
||
(right-half of the Traffic block) that lists local IPv6 listening
|
||
sockets reachable from the mesh interface, paired with the
|
||
`inet fips` baseline filter classification for each (proto, port).
|
||
Rows render in default White (`OPEN` — the chain has a canonical
|
||
unrestricted accept rule), DarkGray (`filt` — chain falls through
|
||
to `counter drop`), or DarkGray with a `?` State suffix (`filt?` —
|
||
the chain references the port but with matchers the panel cannot
|
||
fully decompose, e.g. saddr filters or jumps). When the
|
||
`fips-firewall.service` is not active, the panel renders a yellow
|
||
banner reminding the operator that all listeners are
|
||
mesh-exposed. Wildcard binds (`local_addr == ::`) carry a `*`
|
||
suffix in the Process column. Powered by a new
|
||
`show_listening_sockets` control query (Linux-only).
|
||
|
||
#### Packaging and Deployment
|
||
|
||
- Arch Linux AUR packaging for `fips` (release) and `fips-git`
|
||
(development) packages with sysusers.d/tmpfiles.d integration
|
||
([#21](https://github.com/jmcorgan/fips/pull/21),
|
||
[@dskvr](https://github.com/dskvr))
|
||
- `packaging/debian/fips-gateway.service` now waits up to 30 seconds
|
||
for the daemon's `fips0` TUN to appear before exec'ing the gateway
|
||
binary (`ExecStartPre` poll loop). Eliminates the cold-boot race
|
||
where `fips-gateway` exits with `fips0 interface not found` and
|
||
recovers via `Restart=on-failure`, producing a 5-second blip and a
|
||
spurious error log line per restart cycle. If `fips0` never appears
|
||
within 30 seconds, the existing error path runs as before
|
||
- `packaging/debian/build-deb.sh` now auto-derives a per-commit Debian
|
||
Version field for dev builds (Cargo.toml version ending in `-dev`)
|
||
using the form `<base>~dev+git<YYYYMMDD>.<sha>[.dirty]-1`, e.g.
|
||
`0.3.0~dev+git20260429.6def31b-1`. Each commit produces a uniquely-
|
||
comparable Version string so `apt install ./*.deb` and
|
||
`ansible.builtin.apt: deb:` no longer silently no-op when one dev
|
||
build is installed on top of another. The `~dev` marker sorts
|
||
pre-`0.3.0` so a tagged release supersedes any prior dev .deb.
|
||
Tagged release builds (no `-dev` in Cargo.toml) keep the clean
|
||
`<version>-1` form. Operator override via `--version` still wins
|
||
|
||
#### Examples
|
||
|
||
- macOS WireGuard sidecar: run FIPS in a local Docker container and
|
||
route `.fips` traffic from the macOS host through a WireGuard tunnel
|
||
to the container's `fips0` interface. Only traffic destined for
|
||
`fd00::/8` transits the sidecar; regular internet traffic continues
|
||
to use the host network
|
||
([#51](https://github.com/jmcorgan/fips/pull/51))
|
||
|
||
#### Documentation
|
||
|
||
- `docs/design/port-advertisement-and-nat-traversal.md` documents
|
||
how nodes find each other through Nostr relays and the
|
||
STUN-assisted UDP hole punch
|
||
|
||
### Changed
|
||
|
||
- Noise session ChaCha20-Poly1305 backend switched from RustCrypto's
|
||
`chacha20poly1305` to `ring 0.17`. ring wraps BoringSSL's
|
||
hand-tuned ChaCha20-Poly1305 implementation, dispatching to NEON
|
||
on aarch64 and AVX2 / AVX-512 on x86_64 — typically 3-5 GB/s/core
|
||
vs the ~600-800 MB/s/core RustCrypto soft path on the same
|
||
hardware. Wire format unchanged: ChaCha20-Poly1305 is
|
||
byte-deterministic for a given `(key, nonce, plaintext, aad)`,
|
||
so any correct AEAD produces identical ciphertext and a mixed
|
||
pre-swap / post-swap mesh interoperates without protocol
|
||
awareness. The keyed AEAD is now cached on `CipherState` instead
|
||
of being re-derived per packet (the cached Poly1305 key state is
|
||
the actual perf win); `EndToEndState` grew from ~600 B to
|
||
~1.5 KB as a consequence and is annotated
|
||
`#[allow(clippy::large_enum_variant)]` since boxing would re-add
|
||
a per-packet indirection on every encrypt/decrypt. aarch64
|
||
measurements (Apple Silicon docker, two nodes): TCP 1-stream
|
||
437 → 1097 Mbps (~2.5×); UDP at 1000 Mbit goes from
|
||
599 Mbps / 40 % loss to lossless line-rate; 3-node ping under
|
||
load 7.68 ms avg / 215 ms max → 0.72 ms / 3.6 ms max as the
|
||
relay path stops being crypto-bound
|
||
([#80](https://github.com/jmcorgan/fips/pull/80),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- Linux UDP receive path uses `recvmmsg(2)` with a 32-packet batch
|
||
in place of single-packet `recvmsg(2)`. A single `readable()`
|
||
wakeup drains up to 32 datagrams in one syscall before yielding
|
||
back to the reactor, eliminating the per-packet scheduler-hop +
|
||
futex cost that previously capped inbound rate at one event per
|
||
scheduler quantum independent of CPU. `SO_RXQ_OVFL` is sampled
|
||
once per batch from the cmsg chain of `msgs[0]` and surfaced
|
||
through `AsyncUdpSocket::recv_batch` so the 1Hz
|
||
`sample_transport_congestion()` detector continues to feed the
|
||
per-transport `dropping` flag. macOS / Windows fall through to
|
||
the per-packet path; `recvmmsg` is Linux-specific
|
||
([#81](https://github.com/jmcorgan/fips/pull/81),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- `Node::run_rx_loop` drains up to 256 additional ready items via
|
||
`try_recv()` after each `tokio::select!` await fires on
|
||
`packet_rx` / `tun_outbound_rx`, in a tight inner loop before
|
||
yielding. Previously the select cost a full scheduler hop +
|
||
futex per packet, capping throughput at one event per scheduler
|
||
quantum with the worker near-idle. `biased` ordering keeps
|
||
data-plane branches priority over tick / control / DNS under
|
||
sustained load; the 256 cap is empirically tuned to keep the
|
||
worker on a busy stream between yield points (≈ 400 KB of
|
||
contiguous traffic) while still bounding the inner loop so a
|
||
flood on one branch can't starve the periodic tick or control
|
||
socket. Pairs with the UDP `recvmmsg` change above
|
||
([#81](https://github.com/jmcorgan/fips/pull/81),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- `PeerIdentity::pubkey_full()` now precomputes the parity-aware
|
||
full public key at construction in `from_pubkey`. Previously the
|
||
method fell through to a secp256k1 EC point parse (`fe_sqrt` +
|
||
`fe_mul` + `ge_set_xo_var`) on every call when the full key
|
||
wasn't passed at construction (i.e. for every peer constructed
|
||
from an npub or x-only key) — ~6% of per-packet CPU on the
|
||
bulk-data send path for a value that never changed after
|
||
construction. The same EC point parse already runs at
|
||
construction inside `NodeAddr::from_pubkey`, so the cost is paid
|
||
once where it would be paid anyway
|
||
([#81](https://github.com/jmcorgan/fips/pull/81),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- Cargo feature flags `tui`, `ble`, `gateway`, and
|
||
`nostr-discovery` removed; subsystem inclusion is now driven by
|
||
platform `cfg` gates so plain `cargo build` compiles everything
|
||
available on the target
|
||
([#79](https://github.com/jmcorgan/fips/pull/79))
|
||
- MMP link-layer report intervals retuned for constrained transports:
|
||
steady-state floor raised from 100ms to 1000ms, ceiling from 2000ms
|
||
to 5000ms. Cold-start uses a 200ms floor for the first 5 SRTT samples
|
||
before switching to steady-state. Reduces BLE overhead ~10× while
|
||
keeping reports well above the EWMA convergence threshold.
|
||
Session-layer intervals unchanged
|
||
- 35 info-level log messages demoted to debug (handshake
|
||
cross-connection mechanics, periodic MMP telemetry, TUN/transport
|
||
shutdown, retry scheduling). Info output now focuses on
|
||
operator-relevant state changes: lifecycle events, peer promotions,
|
||
session establishment, parent switches, transport start/stop
|
||
- **Breaking (control socket JSON):** `show_cache` response field
|
||
`entries` has changed type from a `u64` count to an array of entry
|
||
objects; a new `count` field carries the previous scalar value.
|
||
`show_routing` response field `pending_lookups` has changed type
|
||
from a `u64` count to an array of per-target lookup objects.
|
||
External consumers parsing these fields as numbers must be
|
||
updated. In-tree `fipstop` is adjusted to the new schema. The
|
||
control socket interface is still pre-1.0 and not covered by
|
||
stability guarantees
|
||
- Discovery rate limiting retuned to be less aggressive at cold start.
|
||
The previous defaults (30s base post-failure suppression, doubling
|
||
to a 300s cap, with reset only on parent change / new peer / first
|
||
RTT / reconnection) reliably outlasted initial mesh convergence: a
|
||
single timed-out lookup during bloom-filter propagation suppressed
|
||
any retry for 30s while none of the reset triggers fired on a
|
||
stable post-handshake topology. The suppression window dictated
|
||
effective time-to-converge instead of bounding repeat traffic.
|
||
Replaces the single-lookup-with-internal-retry model
|
||
(`timeout_secs`/`retry_interval_secs`/`max_attempts`) with a
|
||
per-attempt timeout sequence in
|
||
`node.discovery.attempt_timeouts_secs` (default `[1, 2, 4, 8]`).
|
||
Each attempt sends a fresh `LookupRequest` with a new `request_id`,
|
||
which lets successive attempts take different forwarding paths as
|
||
the bloom and tree state evolve. The destination is declared
|
||
unreachable only after the full sequence is exhausted (15s total
|
||
at the default). Disables post-failure suppression by default
|
||
(`backoff_base_secs`/`backoff_max_secs` now both `0`); operators
|
||
with chatty apps generating repeat lookups against unreachable
|
||
destinations can opt back in
|
||
- The `docs/` tree is reorganised so readers can find content by
|
||
what they're trying to do: tutorials for new users, how-to guides
|
||
for specific tasks, reference material for configuration and
|
||
protocol details, and design discussion for architectural
|
||
background. New top-level `getting-started.md` and per-section
|
||
landing pages anchor the entry points. Content was reconciled
|
||
against current source: protocol layer details, wire-format
|
||
diagrams, configuration knobs, and CLI references were brought
|
||
back into agreement with the implementation. Gateway feature-set
|
||
documentation was rewritten end-to-end.
|
||
- Test coverage was substantially expanded for the new release
|
||
surface (discovery state machine, control-socket query handlers,
|
||
decrypt-failure thresholds, STUN parser, gateway, NAT traversal,
|
||
packaging install paths) alongside CI-side hardening for the new
|
||
Windows and macOS platforms.
|
||
- Gateway `dns.listen` source default changed from `[::]:53` to
|
||
`[::1]:5353` to match the canonical deployment model (a host
|
||
already serving DHCP/DNS to a LAN segment, where port 53 is
|
||
taken by the existing resolver and `.fips` queries are forwarded
|
||
to the gateway over loopback). The OpenWrt ipk previously
|
||
overrode this in its packaged config; the override is now
|
||
redundant and has been dropped. Operators on a host without a
|
||
pre-existing resolver on port 53 can opt back into the wildcard
|
||
bind by setting `dns.listen: "[::]:53"` explicitly. The new
|
||
default binds IPv6 loopback only — forwarders that reach the
|
||
gateway over IPv4 loopback need an explicit IPv4 listen address.
|
||
- Generic systemd install tarball brought to feature parity with
|
||
the `.deb` and AUR packages. The tarball now ships the
|
||
`fips-gateway` binary with its (operator-opt-in)
|
||
`fips-gateway.service`, a `fips-firewall.service` unit with the
|
||
`/etc/fips/fips.nft` mesh-interface nftables baseline (also
|
||
opt-in), an `/etc/fips/fips.d/` operator drop-in directory for
|
||
per-service nft rules, and the multi-backend `fips-dns-setup` /
|
||
`fips-dns-teardown` helpers. `install.sh` and `uninstall.sh`
|
||
handle the new units and conffile (preserve-on-upgrade for
|
||
`fips.nft`, like `fips.yaml`). `README.install.md` documents
|
||
the gateway, firewall, and DNS-routing services. Closes the
|
||
longest-standing parity gap for non-Debian / non-Arch systemd
|
||
Linux distros (Fedora, RHEL/CentOS, openSUSE, etc.) installing
|
||
from the release-distribution tarball.
|
||
|
||
### Fixed
|
||
|
||
- Generic systemd install tarball: `install.sh` now correctly
|
||
resolves the `fips-dns-setup` and `fips-dns-teardown` helpers
|
||
from the tarball staging directory. Previously the script
|
||
referenced them at `${SCRIPT_DIR}/../common/`, a path that
|
||
exists only in the source-repo layout, not in the extracted
|
||
tarball. Bug latent since the multi-backend DNS helpers
|
||
landed in `7260ad2`; only manifested when operators ran
|
||
`install.sh` from an extracted tarball rather than from a
|
||
source checkout.
|
||
|
||
- Adopted NAT-traversed UDP transports inherit the primary listener's
|
||
MTU and buffer config. `Node::adopt_established_traversal`
|
||
constructed the adopted UDP transport with `UdpConfig::default()`
|
||
(MTU 1280, default recv/send buffer sizes, default accept/advertise
|
||
flags) regardless of the operator's primary `[transports.udp]`
|
||
listener. Operators who set the primary MTU higher (e.g. 1500 on
|
||
a known-clean LAN path) silently dropped full-sized tunnel
|
||
datagrams over the NAT-traversed link with no log explaining why
|
||
throughput collapsed. Lookup now tries `transport_name` first (so
|
||
multiple named listeners pick up inheritance from the matching
|
||
one) and falls back to the unnamed `Single` listener; bind /
|
||
external-address fields are cleared since the adopted socket is
|
||
already bound. The 1280 default was deliberately the IPv6 minimum
|
||
(the only value guaranteed across arbitrary middlebox paths);
|
||
with this change, operators who raise the primary MTU accept the
|
||
tradeoff that NAT-traversed flows initially attempt the higher
|
||
MTU and may black-hole on tighter paths until reactive
|
||
`MtuExceeded` recovery kicks in
|
||
([#83](https://github.com/jmcorgan/fips/pull/83),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- TreeAnnounce ancestry on self-root transitions. When a node had
|
||
no smaller-NodeAddr peer to use as a parent, the spanning-tree
|
||
state correctly promoted it to root, but the ancestry it
|
||
advertised on the next `TreeAnnounce` still referenced its
|
||
previous parent's path. Receiving peers rejected the announce
|
||
with `invalid ancestry: advertised root X is not the minimum
|
||
path entry Y`, blocking mesh transit on any path that needed to
|
||
traverse the node. The self-root transition is now detected
|
||
explicitly in `TreeState::become_root` and the advertised
|
||
ancestry rebuilt to start from self. The MMP receive handler
|
||
surfaces the same path so stale ancestry inherited across
|
||
reconnect is corrected eagerly rather than waiting for the next
|
||
observation tick
|
||
([#82](https://github.com/jmcorgan/fips/pull/82),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- Auto-connect retry refetches the cached overlay advert
|
||
unconditionally before each retry attempt, not only when
|
||
`fetch_advert` returns zero endpoints (`NoTransportForType`).
|
||
The much more common stale-cache failure was: cache returned an
|
||
endpoint that *looked* valid (the address learned before the
|
||
peer's NAT rebound), the dial succeeded at the IP layer, the
|
||
handshake timed out, MMP fired, the next retry hit the same
|
||
cached endpoint, looped forever — no `NoTransportForType` ever
|
||
fired because the cache had data, just dead data. Refetch now
|
||
runs unconditionally before each retry attempt (one Filter query
|
||
against `advert_relays` with a 2s per-attempt timeout, bounded
|
||
by the retry backoff cadence). Keeps the retry loop pinned to
|
||
relay ground truth instead of whatever the cache happened to
|
||
learn at startup
|
||
([#82](https://github.com/jmcorgan/fips/pull/82),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- Stale overlay-advert eviction on `NoTransportForType`. Mirrors
|
||
the existing stale-advert sweep that ran from the
|
||
`BootstrapEvent::Failed` (NAT-traversal-streak) path, but covers
|
||
the case where `initiate_peer_connection` / a retry tick returns
|
||
`NodeError::NoTransportForType` — the cache had no addresses for
|
||
the peer at all. A fire-and-forget `refetch_advert_for_stale_check`
|
||
against the peer's npub re-fetches kind `37195` from
|
||
`advert_relays`; if the relay has a newer advert it replaces the
|
||
cached entry, if it has nothing it evicts the entry. Either way
|
||
the next retry tick goes to fresh data instead of looping on the
|
||
same dead endpoint. Resolves a deployment regression where a
|
||
macOS daemon's view of a Linux peer would flap after NAT rebind
|
||
with no recovery short of a daemon restart
|
||
([#82](https://github.com/jmcorgan/fips/pull/82),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- Schedule retry on startup peer-init failure. When
|
||
`initiate_peer_connections()` ran at boot, an address-resolution
|
||
failure (no operational transport for the configured transport
|
||
types, all addresses unreachable, NAT rebind invalidating cached
|
||
endpoints) was logged and silently forgotten — the peer entry
|
||
stayed in a dead state forever, accepting incoming pings but
|
||
unable to answer them, until the daemon was manually restarted.
|
||
Now mirrors the `BootstrapEvent::Failed` path: on a startup
|
||
peer-init error, parse the peer's npub and call `schedule_retry`
|
||
so the peer recovers without operator intervention
|
||
([#82](https://github.com/jmcorgan/fips/pull/82),
|
||
[@mmalmi](https://github.com/mmalmi))
|
||
- Default control-socket path resolution: daemon and client tools now
|
||
use a shared resolver, eliminating a divergence where `fipsctl` /
|
||
`fipstop` could connect to a socket the daemon never bound (notably
|
||
on dev runs with `XDG_RUNTIME_DIR` set, or after a prior packaged
|
||
install left a root-owned `/run/fips` behind). Canonical order is
|
||
`/run/fips` → `$XDG_RUNTIME_DIR/fips/` → `/tmp/fips-<name>`. The
|
||
`/run/fips` arm is selected by directory existence; the kernel
|
||
enforces actual access at `connect(2)` time, surfacing a clear
|
||
`EACCES` for users not yet in the `fips` group rather than silently
|
||
steering them to a path the daemon never bound. `XDG_RUNTIME_DIR` is
|
||
validated as an existing directory before being used so stale
|
||
post-logout values are treated as missing. The deployed fleet is
|
||
unaffected: packaged configs set `node.control.socket_path`
|
||
explicitly.
|
||
- UDP transport with `advertise_on_nostr: true` + `public: true` +
|
||
a wildcard `bind_addr` (e.g. `0.0.0.0:2121`) is now advertised
|
||
with its STUN-discovered public IPv4 instead of being silently
|
||
dropped from the published Kind 37195 advert. Previously the
|
||
advert builder filtered the wildcard out (since `0.0.0.0` is
|
||
not a valid endpoint), but emitted no log explaining what
|
||
happened — operators saw the daemon up, both flags set, and
|
||
no UDP endpoint in the advert. The fix runs a one-shot STUN
|
||
observation against an ephemeral socket on the daemon's
|
||
configured `stun_servers` and combines the reflexive IPv4 with
|
||
the configured listener port for the advert (`udp:<eip>:<port>`).
|
||
Successful STUN observations are cached per-transport for one
|
||
`advert_refresh_secs` cycle (default 30 min) so we don't re-STUN
|
||
every refresh. Failed observations are cached for only 60s, so
|
||
a transient STUN flake at startup retries within ~a minute and
|
||
grows the advert with UDP as soon as STUN starts working —
|
||
rather than waiting the full 30-min cycle. Per-server STUN
|
||
response timeout is 5s for the advert-publish path (vs. 2s for
|
||
the latency-sensitive per-traversal path), giving slow
|
||
first-call STUN time to complete without giving up. On STUN
|
||
failure, the wildcard-bind path still skips, but now logs a
|
||
loud `warn!` pointing at the operator-side fixes (set
|
||
`external_addr`, bind to a specific IP, or ensure `stun_servers`
|
||
reachable). Restores zero-config public-IP autodiscovery on
|
||
AWS EIP / GCP / Azure setups where binding to the public IP
|
||
directly is impossible (1:1 NAT)
|
||
- New `external_addr` field on `transports.udp.*` and
|
||
`transports.tcp.*` for explicit advertise-as override. Accepts
|
||
either a bare IP (`"198.51.100.1"` — the configured `bind_addr`
|
||
port is appended) or a full `host:port`
|
||
(`"198.51.100.1:8443"`). Takes precedence over both the bound
|
||
address and any STUN-derived autodiscovery. Required for TCP
|
||
on cloud-NAT setups (AWS EIP, GCP/Azure external IPs) where
|
||
binding to the public IP directly fails with `EADDRNOTAVAIL`
|
||
(the EIP isn't on a host interface). Optional but useful for
|
||
UDP as a deterministic alternative to STUN — operators who
|
||
want to skip STUN egress (or whose STUN is blocked) can
|
||
specify it explicitly. Without `external_addr`, TCP with a
|
||
wildcard `bind_addr` + `advertise_on_nostr: true` now logs a
|
||
loud `warn!` pointing at the two fixes instead of silently
|
||
skipping
|
||
- Nostr-discovery now tolerates ±60s of clock skew on offer/answer
|
||
freshness checks so a responder whose wall clock leads the
|
||
initiator's by less than that no longer silently rejects every
|
||
offer. Previously, a public-test daemon with un-NTP'd peers (or
|
||
long uptime — `now_ms()` anchors to `SystemTime` once at startup,
|
||
then advances monotonically; post-startup NTP step adjustments
|
||
don't propagate) would see ~100% signal-timeout rate against
|
||
skewed peers, indistinguishable from "peer is offline." New
|
||
optional `offerReceivedAt` field on the answer payload lets the
|
||
initiator log per-peer NTP-style skew estimates (DEBUG when ≥30s)
|
||
for operator visibility. Backward-compatible — older responders
|
||
that don't fill the field still produce valid answers
|
||
- Nostr-discovery NAT-traversal failure suppression: per-npub
|
||
consecutive-failure counter triggers a 30-min extended cooldown
|
||
after 5 failures, preventing the daemon from hammering Nostr
|
||
relays with offers to peers that have gone away. WARN log lines
|
||
rate-limited to one per peer per 5 min (subsequent failures
|
||
emit DEBUG with `consecutive_failures` + remaining `cooldown_secs`).
|
||
Threshold-crossing also fires a one-shot active re-check of the
|
||
peer's Kind 37195 advert against `advert_relays`; absent →
|
||
evict cache; newer → refresh + reset streak; same → cooldown
|
||
stands. New `failure_streak_threshold`, `extended_cooldown_secs`,
|
||
`warn_log_interval_secs`, `failure_state_max_entries` config
|
||
fields under `node.discovery.nostr`. Per-peer state visible in
|
||
`fipsctl show peers` JSON under `nostr_traversal`
|
||
- Tor onion adverts published over Nostr overlay discovery now
|
||
include the public-facing port (`<onion>.onion:<port>`) instead of
|
||
just the bare onion hostname. The publisher previously emitted a
|
||
bare onion that the parser refused (`expected host:port`),
|
||
producing a persistent retry-fail loop on any peer whose Tor
|
||
advert was the only entry in the discovery cache. New
|
||
`transports.tor.advertised_port` config field (default `443`,
|
||
matching the Tor `HiddenServicePort` convention) controls the
|
||
advertised port; operators with non-default virtual ports can
|
||
override.
|
||
- TCP-over-FIPS reliability on mesh paths with mixed transport
|
||
MTUs (e.g. a UDP-1280 hop in the picker set) improved. Three
|
||
interlocking changes: `Node::transport_mtu()` is now deterministic
|
||
across restarts (min across operational transports rather than
|
||
insertion-order-dependent); the TCP MSS clamp at the TUN boundary
|
||
reads per-destination path MTU instead of a single global ceiling;
|
||
and reactive `MtuExceeded` from forwarders is mirrored back into
|
||
the TUN-side `path_mtu_lookup` so later flows pick up forward-path
|
||
bottlenecks without re-discovery. Windows TUN reader receives the
|
||
same per-destination plumbing.
|
||
- Proactive end-to-end `PathMtuNotification` now mirrors into the
|
||
TUN-side `path_mtu_lookup` (TCP MSS clamp store), parallel to the
|
||
reactive `MtuExceeded` mirror that already existed. Previously the
|
||
proactive handler only updated the session-canonical
|
||
`MmpSessionState.path_mtu`; on stable long-lived paths where the
|
||
destination's echo had tightened the session MTU but no transit
|
||
router had emitted a fresh `MtuExceeded` (because all current
|
||
traffic was already sized by the tighter session value), new TCP
|
||
flows opened in that window kept getting clamped by the staler
|
||
discovery-time value. The proactive mirror closes that gap with
|
||
the same tighter-only semantics — never loosens the clamp.
|
||
- Nostr-discovered peers running an FMP-protocol version we cannot
|
||
speak no longer trigger an indefinite retraversal storm. Open-
|
||
discovery NAT-traversal succeeds at the UDP layer regardless of
|
||
protocol version, so the daemon would adopt the punched socket,
|
||
drop every incoming packet at `Unknown FMP version`, idle out
|
||
after 31s, and re-fire the full STUN-offer-answer-punch sequence
|
||
~30s later — every minute, forever, against peers the handshake
|
||
literally cannot complete with. The rx loop now detects mismatched-
|
||
version packets arriving on adopted bootstrap transports, reverse-
|
||
maps to the originating npub, and applies a long structural
|
||
cooldown to the discovery layer's `failure_state` so the next
|
||
open-discovery sweep skips the peer until either side upgrades.
|
||
One-shot WARN per fresh observation; subsequent mismatches inside
|
||
the cooldown window are silent. New `protocol_mismatch_cooldown_secs`
|
||
config field under `node.discovery.nostr` (default 86400 = 24h),
|
||
separate from the transient-failure `extended_cooldown_secs`.
|
||
- `fipstop` now uses `ratatui::try_init()` instead of `ratatui::init()`,
|
||
so terminal initialization failures (e.g. Docker on macOS Sequoia,
|
||
or environments without a usable tty) produce a clean error message
|
||
instead of a hard crash
|
||
- Spanning-tree updates that change only the internal path between
|
||
root and leaf — without changing the root or the depth — now
|
||
propagate to leaves correctly. Previously a leaf could continue
|
||
routing against a stale internal path until the parent or depth
|
||
also changed.
|
||
|
||
## [0.2.1] - 2026-05-11
|
||
|
||
### Added
|
||
|
||
- Linux release artifact workflow: builds x86_64 and aarch64 tarballs
|
||
and `.deb` packages on `v*` tag push, with SHA-256 checksums
|
||
- AUR publish workflow for tagged stable releases
|
||
|
||
### Changed
|
||
|
||
- Validate bloom filter fill ratio on FilterAnnounce ingress.
|
||
Inbound FilterAnnounce messages whose derived false-positive
|
||
rate exceeds `node.bloom.max_inbound_fpr` (new config field,
|
||
default 0.05) are rejected silently on the wire, logged at WARN,
|
||
and counted in a new `bloom.fill_exceeded` counter. A
|
||
rate-limited WARN also fires if our own outgoing filter's FPR
|
||
exceeds the cap. `BloomFilter::estimated_count` now takes
|
||
`max_fpr` and returns `Option<f64>`, returning `None` for
|
||
saturated filters; this propagates through `compute_mesh_size`
|
||
into `estimated_mesh_size` (already `Option<u64>`)
|
||
|
||
### Fixed
|
||
|
||
- Control socket path detection in fipsctl and fipstop now checks for
|
||
the `/run/fips/` directory instead of the socket file inside it, so
|
||
users not yet in the `fips` group get a clear "Permission denied"
|
||
error instead of a misleading "No such file" fallback to
|
||
`$XDG_RUNTIME_DIR` ([#30](https://github.com/jmcorgan/fips/issues/30),
|
||
reported by [@Sebastix](https://github.com/Sebastix))
|
||
- OpenWrt ipk build excluded BLE feature that requires D-Bus, which is
|
||
unavailable on OpenWrt targets
|
||
- IPv6 routing policy rule added at TUN setup to protect `fd00::/8`
|
||
from interception by Tailscale's table 52 default route
|
||
- Bloom filter routing no longer swallows traffic when no bloom
|
||
candidate is strictly closer than the current node. `find_next_hop`
|
||
now falls through to greedy tree routing in that case instead of
|
||
returning `NoRoute`, which previously caused dropped packets in
|
||
topologies where the tree parent was closer but not a bloom
|
||
candidate
|
||
- Auto-connect peers now reconnect after a graceful `Disconnect`
|
||
notification from the remote side. `handle_disconnect` previously
|
||
removed the peer without scheduling a reconnect, orphaning the
|
||
entry on a clean upstream shutdown; the other removal paths
|
||
(link-dead, decrypt failure, peer restart) already scheduled
|
||
reconnect ([#60](https://github.com/jmcorgan/fips/issues/60),
|
||
reported by [@SwapMarket](https://github.com/SwapMarket))
|
||
- `fipsctl connect` now rejects FIPS mesh (`fd00::/8`) addresses for
|
||
`udp`, `tcp`, and `ethernet` transports with a clear error message
|
||
instead of echoing success while the daemon silently failed the
|
||
bind with `EAFNOSUPPORT`
|
||
([#61](https://github.com/jmcorgan/fips/issues/61),
|
||
reported by [@SwapMarket](https://github.com/SwapMarket))
|
||
- Tighten TreeAnnounce ancestry validation to match the spanning
|
||
tree specification. The receive path now verifies that the
|
||
ancestry is structurally consistent with the signed parent
|
||
declaration before mutating tree state.
|
||
- Make the tree ancestry acceptance unit test deterministic.
|
||
`test_tree_announce_validate_semantics_accepts_valid_non_root`
|
||
generated a random signing identity while pinning the fixed root
|
||
to `node_addr[0] = 0x01`; about 2 in 256 random identities were
|
||
numerically smaller than the claimed root, triggering
|
||
`AncestryRootNotMinimum`. The test now regenerates the identity
|
||
until its `node_addr` is strictly larger than both the fixed
|
||
parent and root.
|
||
|
||
## [0.2.0] - 2026-03-22
|
||
|
||
### Added
|
||
|
||
#### Operator Tooling
|
||
|
||
- `fipsctl connect` and `disconnect` commands for runtime peer
|
||
management via control socket, with hostname resolution from
|
||
`/etc/fips/hosts`
|
||
|
||
#### IPv6 Adapter
|
||
|
||
- Pre-seed identity cache from configured peer npubs at startup, so TUN packets can be dispatched immediately without waiting for handshake completion ([@v0l](https://github.com/v0l))
|
||
|
||
#### Mesh Peer Transports
|
||
|
||
- New Tor transport with SOCKS5 and directory-mode onion service for anonymous inbound and outbound peering
|
||
- DNS hostname support in peer addresses for UDP and TCP transports
|
||
- Non-blocking transport connect for connection-oriented transports (TCP, Tor)
|
||
|
||
#### Packaging and Deployment
|
||
|
||
- Reproducible build infrastructure: Rust toolchain pinning via
|
||
`rust-toolchain.toml`, `SOURCE_DATE_EPOCH` in CI and packaging
|
||
scripts, deterministic archive timestamps
|
||
- Top-level packaging Makefile for unified build across formats
|
||
- Kubernetes sidecar deployment example with Nostr relay demo
|
||
- Nostr release publishing in OpenWrt package workflow
|
||
- SHA-256 hash output in CI build and OpenWrt workflows
|
||
|
||
#### Testing and CI
|
||
|
||
- Maelstrom chaos scenario with dynamic topology mutation and
|
||
ephemeral node identities via connect/disconnect commands
|
||
- Consolidated Docker test harness infrastructure
|
||
|
||
### Changed
|
||
|
||
- Discovery protocol: replace flooding with bloom-filter-guided tree
|
||
routing. Includes originator retry (T=0/T=5s/T=10s), exponential
|
||
backoff after timeouts and bloom misses, and transit-side per-target
|
||
rate limiting. Removed 257-byte visited bloom filter from LookupRequest wire format. *This is a breaking change; nodes running versions prior to this release will not be compatible.*
|
||
|
||
### Fixed
|
||
|
||
- DNS responder returned NXDOMAIN for A queries on valid `.fips` names,
|
||
causing resolvers to give up without trying AAAA. Now returns NOERROR
|
||
with empty answers for non-AAAA queries on resolvable names.
|
||
(#9, reported by [@alopatindev](https://github.com/alopatindev))
|
||
- Stale end-to-end session left in session table after peer removal blocked session re-establishment on reconnect — `remove_active_peer` now cleans up `self.sessions` and `self.pending_tun_packets`. (#5, [@v0l](https://github.com/v0l))
|
||
- `schedule_reconnect` reset exponential backoff to zero on each link-dead
|
||
cycle instead of preserving accumulated retry count.
|
||
(#5, [@v0l](https://github.com/v0l))
|
||
- FMP/FSP rekey dual-initiation race on high-latency links (Tor): both
|
||
sides' timers fired simultaneously, both msg1s crossed in flight, each
|
||
side's responder path destroyed the initiator state. Fixed with
|
||
deterministic tie-breaker (smaller NodeAddr wins as initiator).
|
||
- Parent selection SRTT gate bypass: `evaluate_parent` used default cost
|
||
1.0 for peers filtered out by `has_srtt()`, defeating the MMP eligibility
|
||
gate. Now skips unmeasured candidates when any peer has cost data.
|
||
- FSP rekey cutover race: initiator cut over before responder received msg3,
|
||
causing AEAD failures. Fixed by deferring initiator cutover by 2 seconds.
|
||
- MMP metric discontinuity after rekey: receiver state carried stale
|
||
counters across rekey, inflating reorder counts and jitter. Fixed via
|
||
`reset_for_rekey()`.
|
||
- Auto-connect peers exhausted `max_retries` on initial connection failures
|
||
and were permanently abandoned. Now retry indefinitely with exponential
|
||
backoff capped at 300 seconds.
|
||
- Control socket permissions: non-root users couldn't connect. Daemon now
|
||
chowns socket and directory to `root:fips` group at bind time.
|
||
- Post-rekey jitter spikes: old-session frames arriving via the drain window
|
||
produced 2,000–7,000ms jitter spikes that corrupted the EWMA estimator.
|
||
Added a 15-second grace period after rekey cutover that suppresses jitter
|
||
updates until drain-window frames have flushed. (#10)
|
||
- ICMPv6 Packet Too Big source was set to the local FIPS address, which
|
||
Linux ignores (loopback PTB check). Now uses the original packet's
|
||
destination so the kernel honors the PMTU update.
|
||
(#16, [@v0l](https://github.com/v0l))
|
||
- Reverse delivery ratio used lifetime cumulative counters instead of
|
||
per-interval deltas, making ETX unresponsive to recent loss. (#14)
|
||
- MMP delta guards used `prev_rr > 0` to detect first report, conflating
|
||
it with a legitimate zero counter. Replaced with `has_prev_rr`. (#14)
|
||
|
||
## [0.1.0] - 2026-03-12
|
||
|
||
### Added (Initial Release)
|
||
|
||
#### Session Layer (FSP)
|
||
|
||
- End-to-end encrypted datagram service between mesh nodes addressed by Nostr npub
|
||
- Noise XK sessions with mutual authentication, replay protection, and forward secrecy
|
||
- Automatic session rekeying with configurable time/message thresholds and drain window for in-flight packets
|
||
- Port multiplexing for multiple services over a single session
|
||
- Session-layer metrics: sender/receiver reports with RTT, jitter, delivery ratio, and burst loss tracking
|
||
- Passive RTT measurement via spin bit
|
||
|
||
#### IPv6 Adapter
|
||
|
||
- IPv6 adapter interface allowing tunneling TCP/IPv6 through FIPS mesh
|
||
for traditional IP applications (TUN interface)
|
||
- DNS resolver allowing IP applications to reach nodes by npub.fips name
|
||
- Host-to-npub static mappings: resolve `hostname.fips` via host map
|
||
populated from peer config aliases and `/etc/fips/hosts` file
|
||
|
||
#### Mesh Layer (FMP)
|
||
|
||
- Self-organized core mesh routing protocol with adaptive least cost forwarding
|
||
- Noise IK hop-by-hop link encryption with mutual authentication and replay protection between peer nodes
|
||
- Distributed spanning tree construction with cost-based parent selection and adaptive reconfiguration
|
||
- Destination route discovery via bloom filter-based directed search protocol
|
||
- Path MTU discovery with per-link MTU tracking and MtuExceeded error signaling
|
||
- Link-layer MMP: SRTT, jitter, one-way delay trends, packet loss, and ETX metrics
|
||
- Link-layer heartbeat with configurable liveness timeout for dead peer detection
|
||
- Epoch-based peer restart detection
|
||
- Automatic link rekeying with K-bit epoch coordination and drain window
|
||
- Static peer auto-reconnect with exponential backoff
|
||
- Multi-address peers with transport priority-based failover
|
||
- Msg1 rate limiting for handshake DoS protection
|
||
|
||
#### Mesh Peer Transports
|
||
|
||
- UDP overlay transport with inbound and static outbound peer configuration
|
||
- TCP overlay transport with listening port and static outbound peer support
|
||
- Ethernet/WiFi transport (MAC address based, no IP stack) with optional automatic peer discovery and auto-connect
|
||
|
||
#### Operator Tooling
|
||
|
||
- Ephemeral or persistent node identity with key file management
|
||
- Unix domain control socket for runtime observability
|
||
- `fipsctl` CLI tool for control socket interaction and node management
|
||
- Comprehensive node and transport statistics via control socket
|
||
- `fipstop` TUI monitoring tool with real-time session, peer, and transport configuration and metrics display
|
||
|
||
#### Packaging and Deployment
|
||
|
||
- Debian/Ubuntu `.deb` packaging via cargo-deb
|
||
- Systemd service packaging with tarball installer
|
||
- OpenWRT package with opkg feed and init script
|
||
- Docker sidecar deployment for containerized services
|
||
- Build version metadata: git commit hash, dirty flag, and target triple
|
||
embedded in all binaries via `--version`
|
||
|
||
#### Testing and CI
|
||
|
||
- Comprehensive unit and integration tests covering all protocol layers and transports
|
||
- Docker test harness with static and stochastic topologies
|
||
- Chaos testing with simulated severe network conditions: latency, packet loss, reordering, and peer churn
|
||
- CI with GitHub Actions: x86_64 and aarch64, integration test matrix, nextest JUnit reporting
|
||
- Local CI runner script (`testing/ci-local.sh`)
|
||
|
||
#### Project
|
||
|
||
- Design documentation suite covering all protocol layers
|
||
- CHANGELOG.md following Keep a Changelog format
|
||
- Repository mirrored to [ngit](https://gitworkshop.dev/npub1y0gja7r4re0wyelmvdqa03qmjs62rwvcd8szzt4nf4t2hd43969qj000ly/relay.ngit.dev/fips)
|
||
|
||
<!-- markdownlint-configure-file { "MD024": { "siblings_only": true } } -->
|