mirror of https://github.com/jmcorgan/fips.git synced 2026-10-05 19:18:25 +00:00

Files

T

ArjenandJohnathan Corgan abe3f0b40a feat(transport): bind and rebind network interfaces dynamically

An interface-bound transport is a long-lived object that is *sometimes
bound*. The interface it names may not exist when the daemon starts, may
appear minutes later, may vanish and return mid-operation, and may never
appear at all. Until now the first observation was final: an interface
missing at start was logged once and skipped for the life of the process,
and one that disappeared at runtime published a health change but was never
rebound.

On OpenWrt that is a live bug. procd starts fips before wifi has created
fips-mesh0 / fips-ap0, both transports are skipped and never retried, and
the 802.11s peer link forms anyway (that is mac80211, not the daemon) — so
the node looks healthy and reaches nothing.

Give every interface-bound transport presence state and a binder task.
start_async now returns Ok with the transport ABSENT rather than failing;
the binder binds when the interface appears, tears the socket down when it
goes away, and rebinds when it returns. Start-time absence and runtime
detach are one transition and one code path.

TransportError::InterfaceUnavailable makes absence branchable: a missing
interface and a typo'd interface name were the same flat StartFailed
(String), so nothing downstream could tell a state from a fault. A bind
failure that is *not* absence — no CAP_NET_RAW, no readable /dev/bpf* —
still fails the start, because retrying a socket that can never open behind
a Degraded nobody is watching is worse than dying loudly at boot.

**Presence is IFF_UP, not IFF_UP|IFF_RUNNING.** Carrier and bindability are
different questions and only the second belongs in a bind gate. An
AF_PACKET socket on a carrier-less bridge is valid and starts carrying
traffic the instant a member port comes up, with no rebind. Gating on
carrier would report a wifi-only router Degraded forever for an empty
br-lan, turn every carrier flap the socket would have survived into an
unbind/rebind cycle, and deadlock an 802.11s interface that reports RUNNING
only once it has peered — peering needs beacons, beacons need a bound
socket. Carrier is reported beside presence in show_transports instead.

**A name is not a device.** Both backends bind by device: AF_PACKET stores
sll_ifindex, a BPF descriptor follows the interface it was attached to. A
netdev deleted and recreated under the same name leaves the socket attached
to something gone while the name resolves perfectly well, and nothing
notices — a stale AF_PACKET socket never becomes readable, so the receive
loop neither errors nor exits, and send failures go to the caller rather
than the binder. A listen-only node (announce: false, no beacon sender to
fail) sat present and deaf indefinitely after a wifi reload. The bound index
is captured at bind and compared on every poll. A name reappearing with a
different MAC is different hardware: cached neighbors are dropped rather
than resumed onto.

**Detection is event-driven** where the kernel offers a source — netlink
RTNLGRP_LINK on Linux, PF_ROUTE on macOS and FreeBSD — with a 1 s
getifaddrs poll underneath as a backstop. Link-event payloads are not
parsed: an event is a hint to re-run the probe, which is cheap and
authoritative, and a parser's bugs would be presence bugs. Probes are
coalesced to ten a second because PF_ROUTE has no group filter and delivers
every routing message on the host; a persistently failing event source is
logged once, backed off, and abandoned for the poll after five errors
rather than spinning a core silently on ENOBUFS.

**Degraded becomes a level, not a latch.** The supervisor's reason set was
monotonic, which was correct while no child could recover; with recovery it
would have come to mean "something broke at some point since boot" rather
than "something is broken now". Absence lives in its own reversible set,
health is recomputed on every transition in both directions, and an absent
transport still counts as up so a single-interface node that boots before
its wifi degrades rather than exiting on NoTransports. There is deliberately
no restart action in the FSM: rebinding belongs next to the file descriptor,
and a supervisor-authored retry would be a second mechanism racing the first
for the same socket.

**The node's egress MTU follows the bound set.** transport_mtu filters on
is_bound(), not is_operational() — an interface-bound transport is
operational from the moment it starts, so the weaker predicate let hardware
that had never appeared clamp the whole node's IPv6 MTU. Because that
minimum now moves at runtime, the TUN reader and writer read the TCP MSS
ceiling from a shared atomic instead of a u16 captured at spawn; every other
consumer (show_status, the snapshot, the session-layer fragmentation check)
already read it live, so the clamp was the one place the daemon could report
one effective MTU and enforce another. It moves in both directions: a narrow
interface appearing tightens it, its departure releases it. MSS is
negotiated per connection, so a change binds connections opened after it.

**Logging is per edge, never per attempt**, with one deadline after it. The
edge itself is not an error — info at boot, warn on a runtime detach, since
an interface bound a moment later is the ordinary case this mechanism exists
to absorb and crying error at t=0 then "recovered" at t=0.2s is the failure
the rule exists to prevent. Ten seconds is the whole grace: past it, absence
is no longer a race against a radio or a container, so a required interface
still missing is reported once at error. Start-time absence and a runtime
detach share that one deadline, as they share everything else here. Said
once, not repeated: how long an absence has lasted is state, published as
interface.since_secs and as Degraded, and a monitor can threshold it per
deployment rather than the daemon compiling a schedule in.

Successful rebinds are damped. Backoff covers failed binds; the nastier case
is binds that keep succeeding into a socket that dies moments later, which a
receive loop giving up on a persistent error while the interface stays UP
produces once per second forever. Consecutive bindings dying inside ten
seconds back off on the 1 s → 30 s curve, and past three the binder stops
announcing each bind as a recovery until one lasts.

The receive loop backs off and exits on a dead socket instead of spinning on
Err with a warn per iteration, and the ad-hoc ENXIO socket reopen in the
beacon sender is gone: both hand recovery to the presence machine, one
mechanism for every cause rather than one hack per symptom. Beacons pause
while absent because the task simply does not exist then.

transports.ethernet.*.optional (default false) decides how loudly absence is
reported. Naming an interface is a statement that you expect it, so the
default complains: Degraded, and the error at the deadline. optional: true
is silent — info on the edge, no health impact, no error — for hardware
legitimately not always there. It describes the interface's presence, not
the transport's importance, and no value of it makes a missing interface
fatal at startup.

show_transports grows an `interface` block — presence, carrier, policy, how
long the phase has been held, bind count, failed attempts. The original boot
race was expensive precisely because nothing an operator could see said the
node was deaf.

ci: exercise the presence probe against musl and an address-less interface

Two legs, both about the assumption the presence probe rests on.

OpenWrt — the platform dynamic interface binding exists for — is musl, and
musl reimplements getifaddrs independently of glibc. `interface_present`
reads ifa_flags out of it, and the interfaces this feature exists for
(fips-mesh0, fips-ap0) are unbridged with no IP address at all, which is
exactly where getifaddrs implementations differ. Every other leg is glibc,
so without this one the probe was asserted on a libc no test had ever run it
against, on the target it was written for. Building for the musl target
rather than inside an Alpine container keeps the leg to a cross-build plus a
run, without a second toolchain image to maintain.

The address-less interface is created on both this leg and the glibc one.
Pinning the contract on both libcs is what turns "glibc and musl agree about
getifaddrs" from an assumption into a checked fact, and makes the glibc leg
fail first if glibc is the one that changes. Loopback cannot stand in: it
has 127.0.0.1, so probing it asks whether getifaddrs works rather than
whether it reports this. Deliberately not tolerant of failure — a guard that
quietly does not run is worse than no guard.

The iface-binding suite is deliberately not added to the integration
matrix here. Its files and its testing/ci-local.sh registration arrive
in the next commit, and the GitHub workflow's matrix entry now goes with
them, so both runners gain the suite together and
testing/check-ci-parity.sh holds at every point in this history.

docs: document dynamic interface binding

The transport-layer design gains an Interface Presence section: the three
deployment scenarios that motivated it, the presence machine and its two
invariants, why presence is IFF_UP and not IFF_UP|IFF_RUNNING, interface
identity and the recreated-netdev case, the detection sources and their two
rate limits, the optional policy, health as a level rather than a latch, the
egress-MTU consequence, the logging rules, and what the mechanism retires.
It records why the error deadline is a single ten-second window rather than
a repeating severity ladder, so the ladder does not come back.

The control-socket and fipsctl references described show_transports without
its `interface` block, and the transport-layer state machine still implied
that Up meant bound. Both now say what Up and presence each describe and how
they differ, since "transport up, interface absent" is a normal state an
operator will meet and would otherwise read as a contradiction.

The configuration reference documents `optional` with the absence, log and
retry behaviour of each setting, and the ground-up tutorial's dongle example
gains optional: true, which is what that example is actually for.

fix(transport): pair every presence edge the binder publishes

Six defects, all in the wiring around the presence machine rather than in
the machine itself. The state machines were well tested; what went untested
was how the binder feeds them, and every one of these lives there.

**The first detach after a clean start never reached node health.**
start_async binds inline and publishes that edge itself, before binder_loop
exists. The loop then built a fresh ChurnGuard, which had therefore recorded
no bind — and detached() takes `announced` to decide whether an edge is
owed, so it asked for no retraction. A node that booted with its interface
present and then lost it kept reporting Full for as long as the interface
stayed away, while show_transports said absent and the log said detached.
stabilized() could not repair it either: it early-returns while bound_at is
None, which it is for a bind the loop did not perform, so the guard stayed
unseeded until a second detach happened to fix it. The loop now seeds the
guard from the binding it inherits.

This is not the cable-unplug case. Presence is IFF_UP, so a carrier loss is
correctly not a detach at all; it takes an admin down, a netdev delete, or a
device removal — a wifi reload, a hostapd restart, a dongle pulled.

**An optional interface published nothing, and the MTU floor rode the same
channel.** Filtering the edge at the source conflated two questions that
happen to share a transport: whether node health should move, and whether
the bound set changed. Only the first is policy. The second determines the
node's egress MTU floor, and refresh_tun_mss_ceiling has no other trigger —
so an optional transport binding or unbinding at runtime left the TUN MSS
clamp derived from a bound set that no longer existed, reporting one
effective MTU and clamping to another. That is the defect the MssCeiling
work was written to remove, reintroduced for exactly the transports the
shipped OpenWrt config marks optional: five of seven. TransportPresence
gains health_relevant; the edge always goes out and only health is filtered.

**A permanent bind fault went unlogged if any absence race preceded it.**
record_attempt ran ahead of the InterfaceUnavailable arm, so a probe that
won a race the bind then lost burned the counter that gates the only error
a real fault ever gets — emitted on attempts == 1. A flapping interface
followed by CAP_NET_RAW being dropped or /dev/bpf* exhausting therefore
reported nothing at all, for the life of the process, and the operator got
report_sustained_absence's "still missing" instead: wrong, for an interface
that is present. An absence race is not a bind attempt and no longer counts
as one.

**The watcher's give-up did not stick.** Abandoning the event source parked
on pending() *inside the changed() future*, and that future is constructed
fresh on every pass of the binder's select! and dropped whenever the poll
ticker wins. So the next pass re-read the dead socket, re-counted the error
and re-logged "not recoverable" — once per wake-up, forever, which at a 1 s
tick across the shipped seven-transport config is seven warnings and seven
failing syscalls a second on flash-backed logging. The 100 ms backoff lived
in the dropped future too and never applied across passes. Abandonment is
now state on the watcher.

**A zero-length read livelocked the binder.** try_io clears readiness only
on WouldBlock, which the sibling error arm handles by hand and this one did
not — so breaking out left readable() instantly ready with nothing to read,
and the loop never returned Pending. That starves the select! of its ticker
entirely and takes presence detection down with it. Cleared and treated as a
fault so the give-up path applies.

**A failed presence probe read as an absent interface.** getifaddrs is a
netlink dump and fails for reasons that have nothing to do with the
interface: ENOBUFS under memory pressure, EMFILE or ENFILE under fd
exhaustion, since it opens a socket of its own. Answering "not present"
there tore down a working socket and degraded health over a transient
syscall failure, undiagnosably, and under fd exhaustion the rebind could not
have succeeded anyway. interface_has_flags now distinguishes the two; the
detach gate holds its binding when the kernel will not answer, while the
bind gate still treats it as absence and retries.

**On macOS the reader thread could not die, so a dead socket read as live.**
Any read() failure other than EBADF — ENXIO being the one that matters,
which is what BPF answers once the interface it was attached to is torn away
— reset the parse buffer and continued. The thread never returned, so the
channel never closed, so recv_from never failed, so the tokio task never
exited, so tasks_alive() reported a dead socket as a live one. Detach
detection on macOS reduced to the name and the index, and an interface reset
in place left the transport present and deaf. It now gives up after a
streak, and the return closes the channel the binder is actually watching.

Two smaller pairings while here: stop_async retracts the edge it would
otherwise leave standing for a socket that is gone, and start_async hands a
refused edge to the binder to retry rather than dropping the only edge
either consumer will see until the interface next moves. The absence
deadline is stamped at start rather than at construction, so a transport
staged for longer than the bring-up window no longer reports sustained
absence on its first tick having given the interface no window at all.

feat(config): reject impossible ethernet interface names at load

Config::validate never inspected transports.ethernet, so an empty interface
name, a name past the kernel's 15-byte limit, and two transports naming the
same netdev all loaded cleanly and failed only at runtime — the first two as
a permanent absence indistinguishable from an interface that has not been
created yet.

That indistinguishability is deliberate and worth keeping: waiting is the
right answer for an interface the operator has not made yet, and the daemon
cannot know which of the two it is looking at. Which is exactly why the
syntactic gate earns its place. A name that is *impossible* is the one case
still separable from "not there yet", and without the check a typo costs a
permanently Degraded node whose only symptom is an interface that never
arrives — the failure mode the presence machine exists to make legible,
reintroduced one level up.

Syntax only. Whether a well-formed name exists stays the binder's question,
asked once a second, forever.

The duplicate check is a different fault: two transports on one netdev means
two sockets on the same device at the same ethertype, each receiving every
frame the other does.

test(transport): close the vacuous and uncovered branches

Six tests that asserted nothing, or asserted less than they claimed.

`a_bind_fault_still_fails_the_start` returned early whenever the socket
*could* be opened, so it was vacuous as root and on any developer machine with
a group-readable /dev/bpf* — the fail-fast path it is named for went unchecked
exactly where someone was most likely to run it. It now asserts in both
halves, and the privileged half is worth more than the fix: a present,
bindable interface binding inline is the ordinary case on a booted router, and
no other unit test reaches it, because every other one here names an interface
that does not exist. The bind-success path had no unit coverage at all.

`an_interface_with_no_addresses_is_still_present` returned early when the
fixture was absent — no fixture, pass. It still has to skip on a machine with
no address-less interface, so the guard is the runner declaring that it has
fixtures: CI now sets FIPS_TEST_REQUIRE_FIXTURES beside
FIPS_TEST_ADDRLESS_IFACE, and the test fails rather than skips if the fixture
step is ever removed or renamed. Its `let _ = interface_carrier(...)` is now
asserted too: if presence and carrier ever collapsed into one read, a
carrier-less bridge would report absent and the whole IFF_UP-not-IFF_RUNNING
decision would be silently undone.

`policy_labels` checked `Required.as_str()` and not `Optional.as_str()`, so a
swapped pair would paint every expected interface as the tolerated kind and
stay green. `Presence::as_str` had no test at all — `binding` was never
observed by anything, anywhere.

Three binder branches had no coverage: a transport restarting (the second
`start_async` clearing the previous run's stop flag — only a second *stop* was
tested, so a transport that could never restart passed everything), the
episode clock being restamped at start rather than at construction, and the
refused-edge retry actually delivering. The last one matters most: the
existing test filled the channel and dropped the receiver, so a slot that
captured an edge and never re-sent it would pass while health sat on a stale
level forever.

And the hardware-change boundary `record_bind` returns, which the neighbour
flush hangs off: false on a first bind (or every clean start would drop a
cache it had just built) and true once, not stickily, on a MAC change.

Each new test was verified against the defect it guards — comment out the
`shutdown.store(false)`, the `mark_starting()`, or the seeded `unpublished`,
and the corresponding test goes red while the rest stay green.

fix(test): correct the dummy-carrier assertion, and pin Darwin's presence probe

The carrier assertion added in d6240698 was wrong and would have failed both
Linux legs. A `dummy` interface brought up reports `<BROADCAST,NOARP,UP,
LOWER_UP>` — `IFF_RUNNING` is set, so it *has* carrier. Verified against the
exact fixture CI builds, `addrgenmode none` and all, rather than against the
comment: the original code discarded the result and its comment claimed "up
but not running", which is what made asserting it look safe.

So the fixture pins address-less *presence* and cannot demonstrate the
presence-vs-carrier split at all — an interface up with no carrier is a bridge
with nothing plugged in, which no fixture here creates.
`carrier_is_reported_separately_from_presence` pins that split from the other
side. The corrected assertion is Linux-only, because the expected answer is a
property of the fixture device rather than of the code.

That is also what lets the macOS fixture land. The Linux legs pin the
address-less contract on glibc and musl, but the BSD-derived `getifaddrs` the
macOS backend actually calls had no coverage — the test skipped itself
silently on that runner, which is precisely the shape the previous commit was
removing. `feth` is macOS's fake-Ethernet pseudo-interface and is created
address-less; the step fails the leg rather than testing the wrong thing if the
runner hands it an address anyway, mirroring why the Linux step needs
`addrgenmode none`.

Both branches of the fixture guard were exercised: unset skips and passes, and
declared-but-missing fails loudly. The corrected test was run against a real
Linux dummy inside a container, not reasoned about.

fix(ci): put the macOS presence fixture on the macOS job

The `feth` fixture step added in 6b9faa0c landed on the Linux `test` job, not
on `test-macos`, and has failed CI ever since:

    create: Host name lookup failure
    ifconfig: `--help' gives usage information.

That is Linux net-tools `ifconfig`, which has no `create` subcommand. So the
Linux job ran two address-less-fixture steps — its own correct `ip link add
type dummy` one, then a macOS one that cannot work there — while `test-macos`
had none at all, leaving the Darwin `getifaddrs` path exactly as uncovered as
before.

Cause was a pattern-anchored edit: `Install cargo-nextest` followed by `Run
unit tests` appears in three jobs, and the insert hit the first match. Moving
it then hit the *last* match, which is `test-windows`. It is now placed by job
boundary rather than by pattern, and verified per job: `test` and `test-musl`
carry the `ip link`/dummy fixture, `test-macos` carries `ifconfig`/`feth`, and
`feth` appears exactly once in the file.

The verification that missed this was counting steps in `test-macos` and
reading 6 as confirmation. Six was the count *before* the insert; seven is what
a successful insert looks like. Now asserted by job and by which tool each
fixture step uses, so a step in the wrong place fails the check rather than
matching a total.

test(transport): cover the detach branch and the stop-race check

Two of the three untested branches, by two different routes.

**The detach decision is now a pure function.** `classify_detach(gone,
replaced, dead)` replaces the inline three-way `if` in the binder loop, and all
eight input combinations are asserted, plus the precedence between them and the
`reason=` labels the integration suite and operators grep for.

This does not make `Replaced` or `SocketDied` reachable from a test — both need
a bind that succeeded and then a specific external event, and the integration
suite cannot arrange the recreate deterministically either, for the reason
recorded in reference/notes.md: the netlink event from a delete is acted on
within microseconds, so `Gone` wins that race in practice. What it does is
split the untested thing in two. The three inputs each already had tests
(`interface_present`, `device_replaced`, `tasks_alive`); the branch between them
did not, and that half is now total and exhaustive.

Precedence is asserted rather than assumed: an interface that has gone away has
also trivially been "replaced" and its socket is also dead, so the most
specific true statement has to win, and callers pass `replaced` already masked
by `!gone` — the function no longer depends on them having masked correctly.

**The post-store shutdown check is now tested directly.** It needs a bind that
*succeeds*, which is why `a_stop_racing_a_bind_leaves_nothing_behind` could
never reach it: that test's interface does not exist, so `bind_and_spawn`
refuses at the presence probe several steps earlier. Binding loopback as root
reaches it, and the assertion is that a bind completing after a stop undoes its
own socket, its own loops, and its own presence.

Unprivileged runners skip it, but loudly: a runner declaring
`FIPS_TEST_PRIVILEGED` and unable to open a raw socket fails instead of
skipping, the same guard shape as `FIPS_TEST_REQUIRE_FIXTURES`. No CI leg sets
that yet — unit tests run unprivileged — so today it exercises the branch only
under a root container. Verified there, including the negative control: with
the post-store check removed the test fails, and with it present it passes.

test(transport): run the bind-success half in CI, and cover the replaced device

Adds the privileged unit-test step the rest of this depends on, then uses it.

**The privileged step.** Every unit-test leg runs unprivileged, so
`PacketSocket::open` cannot succeed on any of them and everything past a
successful bind runs nowhere in CI: the post-store shutdown check, the
`Present` arm of the binder loop, `bind_now` itself. The step builds as the
runner user and executes only the test binary under sudo — running `cargo` as
root would use root's CARGO_HOME and discard the cache the job just restored.
The binary-path extraction was verified locally before being written into the
workflow.

`FIPS_TEST_PRIVILEGED` is what makes it honest. Tests needing a raw socket skip
quietly without it; with it set, a test that cannot open one fails and says so.
A runner that stops granting the capability shows up as a red leg rather than
as silence.

**The replaced device.** `"interface replaced"` — a netdev recreated under the
same name, the `wifi reload` case in #125 — could not be reached by the
integration suite: with link events live the kernel's `RTM_DELLINK` is acted on
within microseconds, so the binder observes "gone" first and takes the branch
already covered. Attempting it there passed about one run in three.

Rather than race the binder, the test asserts the *inputs*: after a real
delete-and-recreate the name still resolves and the bound index no longer
matches. Paired with the exhaustive classifier test, which pins that
`(gone: false, replaced: true)` maps to `Replaced`, the path is covered without
depending on scheduling. A watcher-disable hook was written for the racing
approach and removed once this one made it unnecessary.

**Two smaller gaps.** `report_sustained_absence` had no direct test — the
integration suite infers it from counting ERROR lines, and ten seconds of real
time is how a deadline ends up asserted by proxy. A test-only `backdate_for_test`
ages the episode clock instead. And the probe floor was asserted only as a
relation between two constants; it now has behaviour, via a `probe_delay`
helper extracted from the loop.

That extraction was not neutral, and its own test caught it: `checked_sub`
yields `Some(0)` at exact equality where the loop used a strict `<`, so a
zero-length sleep would have replaced no sleep at all. Harmless in effect,
wrong in meaning, and fixed.

Every new test was run against the defect it guards, in a root container:
remove the post-store shutdown check, or make `device_replaced` always answer
false, and the corresponding test fails.
Mark publish_presence must_use. The defect this commit fixes was a discarded
return value, so the fix is made self-guarding: a future call site that drops
the pending edge instead of storing it now fails the lint rather than silently
reintroducing the bug. Option is not must_use in std, unlike Result, so the
attribute has to be explicit.

2026-09-10 19:18:09 +00:00

18 KiB

Raw Blame History

`fipsctl`

Command-line client for the FIPS daemon's control socket.

Synopsis

fipsctl [-s SOCKET] <subcommand> [args...]

Description

fipsctl connects to a running daemon over its control socket (Unix domain socket on Linux/macOS, TCP loopback on Windows), sends one JSON request, and pretty-prints the response. Exits with a non-zero status if the socket cannot be reached, the daemon returns an error, or the request times out.

fipsctl keygen and fipsctl address are special cases: they do not contact the daemon and operate purely on local files.

For the line-delimited JSON wire protocol, see control-socket.md. For the YAML configuration that defines the socket location, see configuration.md.

Global Options

Flag	Argument	Description
`-s`, `--socket`	`PATH`	Override the control-socket path (Linux/macOS) or TCP port (Windows).
`-V`	—	Print the short version, `<version> (rev <git-hash>)`.
`--version`	—	Print the long version: short version plus build target triple.
`-h`, `--help`	—	Print usage and exit. Per-subcommand help via `fipsctl <subcommand> --help`.

Subcommands

`show <what>`

Read-only queries against the daemon. Each subcommand maps 1:1 to a control-socket query (see control-socket.md) and prints the response's data object as pretty JSON.

Subcommand	Control-socket command	Returns
`show status`	`show_status`	Node-level status: identity, version, peer/link/session counts, TUN state, recent sparklines.
`show peers`	`show_peers`	Authenticated peer list with link IDs, transport addresses, MMP metrics, Noise/rekey state.
`show links`	`show_links`	Active links (one per FMP-authenticated peer): direction, state, byte counters.
`show tree`	`show_tree`	Spanning-tree state: root, my coordinates, parent, peer declarations.
`show sessions`	`show_sessions`	End-to-end FSP sessions: state, traffic counters, session-MMP metrics, path MTU.
`show bloom`	`show_bloom`	Bloom-filter state: own filter sequence, leaf dependents, per-peer filter summaries.
`show mmp`	`show_mmp`	MMP metrics summary: per-peer link-layer metrics and per-session session-layer metrics.
`show cache`	`show_cache`	Coordinate cache: TTL, fill ratio, per-destination coords and path MTU.
`show connections`	`show_connections`	Pending handshake connections: state, idle time, resend count.
`show transports`	`show_transports`	Transport instances: type, state, MTU, local address, per-transport stats, and — for interface-bound transports — interface presence (`absent` / `binding` / `present`), carrier, absence policy (`required` / `optional`), time in the current phase, and bind/failed-attempt counts.
`show routing`	`show_routing`	Routing summary: pending lookups, retry state, forwarding/discovery/error/congestion counters.
`show identity-cache`	`show_identity_cache`	Cached `(node_addr → npub)` entries with last-seen timestamps.
`show native-flows`	`show_native_flows`	Native datagram API: open and pending flows with their ports, queue depth and age, bound listeners with their backlog, and the `native` counters.

`acl <what>`

Subcommand	Control-socket command	Returns
`acl show`	`show_acl`	Loaded peer-ACL state: allow/deny files, effective mode, default decision, entry counts.

`stats <what>`

Time-series metrics from the in-process history rings.

Subcommand	Control-socket command	Description
`stats list`	`show_stats_list`	Enumerate available metrics, their units, and the per-ring retention windows.
`stats metrics`	`show_metrics`	Dump current counter values for every protocol metric family (`forwarding`, `discovery`, `tree`, `bloom`, `congestion`, `errors`, `native`).
`stats peers`	`show_stats_peers`	List peers tracked in stats history (active or recently active).
`stats history <metric> [options]`	`show_stats_history`	Fetch a time-series window for one metric.

stats history options:

Flag	Argument	Default	Description
`--peer`	`npub` or hostname	(none)	Required for per-peer metrics; resolves through `/etc/fips/hosts` if not an npub.
`--window`	`<N>s` / `<N>m` / `<N>h`	`10m`	Window duration.
`--granularity`	`1s` or `1m`	`1s`	Ring resolution. `1s` uses the fast ring; `1m` uses the slow ring.
`--plot`	—	off	Render a Unicode-block sparkline to stdout instead of JSON.

`keygen [options]`

Generate a new FIPS identity keypair locally. Does not contact the daemon.

Flag	Argument	Default	Description
`-d`, `--dir`	`DIR`	`/usr/local/etc/fips` (macOS, FreeBSD), `/etc/fips` (other Unix), `%APPDATA%\fips` (Windows)	Output directory for `fips.key` and `fips.pub`. Matches the directory the platform's packaging installs config into, which is where the daemon derives the key paths from.
`-f`, `--force`	—	off	Overwrite an existing `fips.key`.
`-s`, `--stdout`	—	off	Print `nsec` then `npub` to stdout instead of writing files.

fips.key is written with mode 0600 and fips.pub with mode 0644 on Unix. After running keygen, set node.identity.persistent: true in fips.yaml or the daemon will overwrite the keys on next start.

`address [identity] [options]`

Print a node's mesh address (fd00::/8) and nothing else, so it can be captured in a shell substitution. Does not contact the daemon, which makes it usable from an installer or image build where no node is running.

Argument	Description
`identity`	npub (bech32) or hostname from `/etc/fips/hosts`. Omit to use this node's own identity.

Flag	Argument	Default	Description
`-k`, `--key`	`PATH`	(none)	Derive from this key file (an `nsec`) or public key file (an `npub`). Conflicts with `identity`.

With neither an identity nor --key, the address comes from fips.key in the default key directory (the same directory keygen writes to), falling back to fips.pub beside it, since fips.key is mode 0600 and an unprivileged run cannot read it.

The address is derived from the public key exactly as the daemon derives its own: fd followed by the first 15 bytes of SHA-256(pubkey).

`connect <peer> <address> <transport>`

Tell the daemon to dial a peer over a specific transport.

Argument	Description
`peer`	npub (bech32) or hostname from `/etc/fips/hosts`.
`address`	Transport endpoint, e.g. `192.168.1.10:2121`, `[2001:db8::1]:2121`, or a Tor onion. FIPS-mesh ULAs (`fd00::/8`) are rejected for the IP-based transports (udp, tcp, ethernet).
`transport`	One of `udp`, `tcp`, `tor`, `nym`, `ethernet`. The named transport must be configured and running.

`disconnect <peer>`

Tell the daemon to drop a peer link.

Argument	Description
`peer`	npub (bech32) or hostname from `/etc/fips/hosts`.

`probe <target>`

Diagnose whether a mesh endpoint is reachable, in five stages, and report a per-stage verdict so a partial failure localizes itself.

Argument	Description
`target`	npub (bech32) or hostname from `/etc/fips/hosts`.
`--json`	Emit the report as JSON instead of human-readable text.
`--timeout <secs>`	Client-side ceiling. Defaults to the budget the daemon computed, which scales with its tick interval.

The stages are:

bloom — does any peer's announced filter claim the target, and did a LookupRequest therefore go out? A miss ends the probe here and is a statement about the mesh's own knowledge: nobody has heard of this address. Skipped when the coordinates are already cached, and when the target is a directly connected peer.
discovery — waiting for a LookupResponse to answer with coordinates. Each request the node sends gets its own line under the stage, with the timeout that attempt was given and whether it drew a reply. Failing here is the opposite finding to a bloom miss: a peer's filter did claim the address, and nothing answered for it.
path — the least-common-ancestor walk between the two coordinates, plus the next hop this node would select. This is a local computation, not a traceroute: no hop beyond the first is contacted, and the tree distance is an upper bound on the real hop count because a crosslink cut-through can deliver in fewer hops. When no coordinates were available the walk is not computed at all: path.coords_known is false and every tree field is null, rather than a default that would read as a finding about the spanning tree.
session — a full Noise XK handshake over FSP. Completing it is genuine end-to-end evidence: our route reached them, their route reached us, and the remote holds the expected static key.
rtt — one MMP sender/receiver report exchange, for a real round-trip time.

Exit status is 0 only for an overall verdict of ok; partial, failed and cancelled all exit 1.

The stage block fills in as the probe runs. Each stage reports its verdict at the moment it reaches one, rather than the whole report arriving at the end, so a slow stage is visible as the stage that is slow. On a terminal the block is redrawn in place, with a spinner and a running elapsed on whichever stage is working; piped or redirected, each row is printed once, when it settles, and the transcript ends up the same block. A running stage reports only what the daemon has observed — which requests have gone unanswered so far, whether the handshake is in flight, how many receiver reports have arrived — and never previews an outcome it does not have yet. The per-request lines under the discovery stage carry the configured timeout of each attempt in parentheses, which is the node's own ladder rather than a measurement.

The elapsed column comes from the daemon's clock throughout: a finished stage carries its own tick-quantized figure, and the stage still running carries the report's elapsed less the stages already accounted for. Nothing in that column is measured client-side.

Stages that were never attempted get no row. A failure marks everything behind it as not reached, and the report says that once, in the failed row, rather than three more times. Two cases deliberately keep their rows: a skipped stage, because a skip is a result naming why that stage was unnecessary, and everything after a failed path stage, because the path preview touches nothing and the session can still succeed where the preview named no next hop.

Below the stage block, the path: line renders the whole tree walk on one line, from this node to the target, through the least common ancestor, which is emphasised on a terminal. It is the same computed walk the ours, theirs and tree walk lines describe, read in one piece.

--json is unaffected and still emits exactly one document, when the probe ends, so a script parsing the report does not have to skip past progress output.

What the probe leaves behind. It tears down a session it opened itself and never touches one that already existed. Three residues are deliberate and worth knowing about:

The coordinate-cache and identity-cache entries a lookup produced are not evicted. They are TTL-bounded shared read caches, and evicting them could strand an unrelated flow mid-route.
The remote's half of a probe-created session persists until its own idle timeout (default 90s). There is no teardown wire message. In the window between our removal and its idle purge, any session frame the remote sends lands here as one unknown-session reject.
To obtain a round-trip time the probe sends a CoordsWarmup, which starts MMP reporting on the session. On a session the probe does not own, that reporting continues until the idle purge — the same traffic any single data packet would cause, and bounded, but a real change to a session the probe did not create. The report names it under cleanup.warmups_sent.

Running a probe against a production node is safe: the job carries its own deadline daemon-side, so it cleans up whether or not the client is still there.

`profile tick <on|off|status>`

Reading the output. Step durations are wall clock measured across await points, not CPU time: a step that waits on I/O accrues that wait, and other tasks may run inside the span. That is the intended measure for head-of-line delay, and it means a large step is not necessarily an expensive one. arm_starvation is measured directly as the entry time minus the deadline the interval scheduled that tick for. It is not derived from tick_entry_gap, which carries no starvation signal on its own: under a steady delay every gap is exactly one tick period.

Start, stop and inspect a capture of the rx-loop tick body. Present only when both fipsctl and the daemon are built with --features profiling; the feature is off by default, so a stock package does not carry this subcommand and a stock daemon reports profile_tick_* as an unknown command.

Subcommand	Control-socket command	Description
`profile tick on`	`profile_tick_on`	Create the capture file and start recording. Fails if a capture is already running (naming the active file) or if the directory cannot be written.
`profile tick off`	`profile_tick_off`	Stop the capture. The writer is woken immediately, drains once more and is joined, so the command returns promptly. Succeeds, reporting nothing active, when no capture is running.
`profile tick status`	`profile_tick_status`	Report `idle`, `running`, `stopped_by_cap` or `stopped_by_error`, plus the active path, bytes written, flush interval and byte cap.

profile tick on options:

Flag	Argument	Default	Description
`--dir`	directory path	`/var/log/fips`	Where to write the capture. Created if absent. Use it to profile a non-root `cargo run`.

Against a daemon running as root, --dir must name an absolute path under /var/log/fips, and a path that resolves outside it through a symlinked parent is refused. The control socket is reachable by the fips group, which the security model treats as strictly weaker than root, so a group member cannot steer a root directory creation at an arbitrary path. A daemon that is not running as root crosses no such boundary and takes --dir as given, which is what keeps a non-root cargo run capture working. Confining a root daemon's captures to a different log root is not currently supported.

One file is written per capture, named profile-<UTC timestamp>.tsv, with -1, -2 and so on appended if a capture in the same second already claimed the name. The capture file is created private to its owner and is never written over an existing file. It opens with a #-prefixed header block (node npub, build version, platform, configured tick period, flush interval, byte cap, start time), then a tab-separated column header, then one row per measured step per flush interval:

ts_unix  kind  domain  name  count  max  total  unit

kind is step for a timed span and gauge for a sampled scalar, so a gauge value never lands under a duration column; unit names the unit of max and total for that row. Every step present in the build gets a row every interval, including zero-count rows. Gauges cover ticks per interval, peer count, the wall gap between successive tick-arm entries, and the arm-starvation delay, which is measured against the deadline the tick was scheduled for rather than derived from the gap.

A capture stops itself on reaching 32 MB, appending a # line saying so; profile tick status then reports stopped_by_cap until the next on or off clears it.

Exit Codes

Code	Meaning
`0`	Daemon returned `{"status":"ok",...}`.
`1`	Argument parse failure, control-socket connection failure, daemon returned `{"status":"error",...}`, or local I/O failure (keygen). The error message is printed to stderr.

Environment

Variable	Description
`XDG_RUNTIME_DIR`	Used to derive the default control-socket path when `/run/fips` is absent.

fipsctl does not consume RUST_LOG; logging is for the daemon.

Files

Path	Purpose
`/etc/fips/hosts`	Maps hostnames to npubs for the `connect`, `disconnect`, and `--peer` arguments. See configuration.md.
Control socket (default)	Same resolution as the daemon: `/run/fips/control.sock` if present; then `/var/run/fips/control.sock` on macOS/FreeBSD if present; then `$XDG_RUNTIME_DIR/fips/control.sock`; finally `/tmp/fips-control.sock` (Unix). A privileged macOS daemon bootstraps the private `/var/run/fips` directory. Windows uses TCP `localhost:21210`.

If you get Permission denied connecting to the socket on Linux, add your user to the fips group (sudo usermod -aG fips $USER) and log out and back in.

18 KiB Raw Blame History

fipsctl

Synopsis

Description

Global Options

Subcommands

show <what>

acl <what>

stats <what>

keygen [options]

address [identity] [options]

connect <peer> <address> <transport>

disconnect <peer>

probe <target>

profile tick <on|off|status>