Johnathan Corgan
3a789370b9
A client process opens a flow to a peer's public key on a chosen port and sends and receives datagrams on a file descriptor the daemon hands it. No IPv6 emulation, no TUN device, no DNS: a datagram travels from key to key. The feature is off by default and is not a stable interface. The wire needs no change and gets none. Every FSP data packet has carried a port pair inside its AEAD envelope since v0.2.0, and port 256 is simply the IPv6 shim. What was missing was a way for a program to ask for a port of its own and be handed the traffic. Addressing is the part worth reading twice, because the obvious design is wrong. The x-only public key is the address. An npub is that key written in bech32, so converting between them is a local encoding rather than a lookup or a name service. The 16-byte node address that travels on the wire is the first half of a SHA-256 of the key: it is a truncated hash, it does not invert, and it appears nowhere a client can see. An earlier iteration of this work reported a peer by that hash and could supply a key only sometimes, which is what treating a wire identifier as an identity produces. An accepted flow therefore always knows its peer. The key is captured where the peer is authenticated rather than looked up when a report is rendered: every inbound datagram passes one call site inside a handler that refuses anything whose session is not established, and the responder has already rejected the session unless the claimed address derives from the key it proved. Reaching for the identity cache instead gives a best-effort answer from a structure that evicts. A listener is a descriptor. The daemon writes one message per arrival to it, carrying the new flow's descriptor and the peer's address, so poll, select and epoll work on a listener and accepting is a recvmsg. That is what lets the API be used from a program that already has an event loop, which a command-and-reply listener could not support: an arrival could not be waited on beside anything else. There is no accept command and no reject command. Refusing a flow is closing the descriptor you were handed. The Rust surface mirrors std::net. FipsStream::connect, FipsListener::bind, incoming, accept, io::Result and an errno mapping rather than a bespoke error type. An address is given as an npub, as a key, or as a pair, through one parameter, the way ToSocketAddrs takes several spellings of one thing. Each type holds its descriptor and copies of what setup told it and nothing else, so a stream that outlives its setup connection is not representable. set_nonblocking, AsFd and the four deadline methods carry the names and signatures std::net uses for the same jobs. They were asked for by a user integrating the API with tokio: AsyncFd requires a non-blocking descriptor, and anything receiving from a peer needs a bounded wait. AsFd is the better of the two descriptor accessors, because the borrow cannot outlive the value that owns the descriptor, so a reactor cannot hold a registration for a descriptor that has since been closed and its number reused by the next open. The non-blocking flag is read, modified and written back rather than assigned, since the flag word carries more than that one bit and a caller may have set O_ASYNC. A zero timeout is refused with EINVAL, because the kernel reads a zero timeval as "wait for ever", which inverts what a caller passing zero means; std::net refuses it for the same reason. The two directions are separate options and stay that way. FipsListener gets no timeout methods, matching TcpListener: bounding an accept is set_nonblocking plus the caller's own poll, which the reactor how-to builds. A flow taken from accept is blocking whatever the listener was set to, because the two are separate sockets and the daemon hands over a fresh one. One rule has no counterpart in Berkeley sockets and a client author must know it: the v1 wire carries no half-close, so nothing peer-driven ever closes a flow. A server written to read until the flow ends waits for a signal that cannot arrive, holding a thread and a flow per peer until its process exits. A program decides its own termination, and the example serves one datagram per flow. The tests reach a live daemon rather than a stand-in. Every public item had a unit test against a hand-written stand-in with canned replies, and the five entry points a program actually calls first, connect, connect_from, connect_at, bind and the SOCKET constant, had no coverage of any kind, because the tests that appear to cover them build a Wire over a socket pair and hand it to the private open and hold, so nothing ever resolved a socket path or mapped its errors. examples/native-surface.rs walks all thirty-eight items against a running daemon and reports the number of assertions it made. The count is read from the recorder rather than written as a literal, and the harness asserts the exit status, the completion marker and the count together, so deleting an assertion fails the check rather than quietly shrinking it. Watchdogs turn a hang into a named failure, which several of the walked behaviours would otherwise produce. The shared Docker image is built once for every integration leg, so the new binary is staged at all ten places the existing one is, the interop builder included, which gets a stub because those images exercise the wire between daemon versions and older refs do not carry the example. The platform gating was tested rather than reasoned about: flipping all eleven gates so the native API is excluded leaves the crate compiling clean across the workspace, every target and the profiling feature. The shipped docs tree gains what only the LaTeX manual under design/ had, which is not published with the daemon. A reference entry covers the whole surface: addressing and the port tiers, the Berkeley mapping, every method on FipsAddr, FipsStream, FipsListener and Incoming, the errno table, the ceilings, the four places data disappears with nothing reported, the line protocol and the command reference. The errno table gives names rather than numbers, since the client maps each name onto the libc constant for the platform it was built for and the supported platforms disagree on the numbers. A tutorial side trip stands up two throwaway nodes on one machine, peered over loopback UDP with no TUN and no DNS, then writes a listening program and a connecting program against them; it needs neither the public mesh nor root, because the native path is the one that does not go through the IPv6 adapter. The obligations a client in another language carries are a how-to of their own, since they are a task rather than a description: reading the setup connection with recvmsg, associating a descriptor with the last complete line, telling an empty datagram from a close, and six others. Serving many peers from one poll loop is another, with the whole program, because the straightforward listener spawns a thread per flow and that is wrong at the node's ceiling of 256. The drop causes are a table mapping each of the seven texts DropReason::as_str produces to the counter it increments, with drop_oversize called out as the ninth counter that is not in the table. What a daemon restart costs is a section of its own: every flow and listener ends, descriptors do not survive, there is no resumption, and datagrams sent but not yet forwarded are lost through a window nothing bounds. A stack comparison diagram places the interface against the stack a reader already knows: the same application over HTTP, TLS, TCP, IP and Ethernet on one side, and over its own format, FSP, FMP and a FIPS transport on the other, aligned so each row is one concern. The two columns are not alternatives and are not drawn as such. An unmodified IPv6 program's packets reach fips0, and the adapter hands each one to FSP as a payload, so the left stack runs inside the right one; the left column ends at a fork, eth0 for the ordinary internet and fips0 for the mesh, and an arrow leaves fips0 and runs back up into FSP's input. The row where TCP would be is empty on purpose and names Reliable Object Delivery, which is where that capability is expected to land. ROD is a v2 capability, the box is dashed because none of it exists yet, and the design entry says the part a reader needs most: nothing on the surface anticipates it, so a program written today should assume it does not exist. Both endpoints carry a scheme and a worked port, https://<npub>.fips:443 and fips://<npub>:443, with a footnote saying the two ports are not the same kind of thing, a TCP port inside the tunnel on the left and an FSP port on the right. The fips:// form is a coinage: nothing in the tree parses it, nothing registers the scheme, and the API takes a key and a port as separate arguments rather than a URL. The diagram also says where the right column stops, since FIPS over UDP still rides IP and Ethernet beneath. It appears in fips-concepts.md and fips-ipv6-adapter.md, which were making its argument in prose without a picture, and deliberately not in fips-architecture.md, which already carries the OSI mapping and makes the same point about the transport row. The gateway's control socket moves onto the same bind policy this API uses, which is the one change here that touches deployed behaviour: fips-gateway now tightens /run/fips to 0750. That is unreachable under the packaged deployment, where fips.service has already created the directory at that mode, and reachable for a source build or a container that starts the gateway alone. One changelog entry under Added, describing the released state: what a client opens and reads, the addressing and why the node address is not it, the listener being a descriptor, the std::net shape of the Rust surface, and the one rule Berkeley sockets have no counterpart for. It says in as many words that the wire is unchanged.
15 KiB
fipsctl
Command-line client for the FIPS daemon's control socket.
Synopsis
fipsctl [-s SOCKET] <subcommand> [args...]
Description
fipsctl connects to a running daemon over its control socket
(Unix domain socket on Linux/macOS, TCP loopback on Windows), sends
one JSON request, and pretty-prints the response. Exits with a
non-zero status if the socket cannot be reached, the daemon returns an
error, or the request times out.
fipsctl keygen is a special case: it does not contact the daemon and
operates purely on local files.
For the line-delimited JSON wire protocol, see control-socket.md. For the YAML configuration that defines the socket location, see configuration.md.
Global Options
| Flag | Argument | Description |
|---|---|---|
-s, --socket |
PATH |
Override the control-socket path (Linux/macOS) or TCP port (Windows). |
-V, --version |
— | Print the short version. |
--version |
— | Print the long version. |
-h, --help |
— | Print usage and exit. Per-subcommand help via fipsctl <subcommand> --help. |
Subcommands
show <what>
Read-only queries against the daemon. Each subcommand maps 1:1 to a
control-socket query (see control-socket.md) and
prints the response's data object as pretty JSON.
| Subcommand | Control-socket command | Returns |
|---|---|---|
show status |
show_status |
Node-level status: identity, version, peer/link/session counts, TUN state, recent sparklines. |
show peers |
show_peers |
Authenticated peer list with link IDs, transport addresses, MMP metrics, Noise/rekey state. |
show links |
show_links |
Active links (one per FMP-authenticated peer): direction, state, byte counters. |
show tree |
show_tree |
Spanning-tree state: root, my coordinates, parent, peer declarations. |
show sessions |
show_sessions |
End-to-end FSP sessions: state, traffic counters, session-MMP metrics, path MTU. |
show bloom |
show_bloom |
Bloom-filter state: own filter sequence, leaf dependents, per-peer filter summaries. |
show mmp |
show_mmp |
MMP metrics summary: per-peer link-layer metrics and per-session session-layer metrics. |
show cache |
show_cache |
Coordinate cache: TTL, fill ratio, per-destination coords and path MTU. |
show connections |
show_connections |
Pending handshake connections: state, idle time, resend count. |
show transports |
show_transports |
Transport instances: type, state, MTU, local address, per-transport stats. |
show routing |
show_routing |
Routing summary: pending lookups, retry state, forwarding/discovery/error/congestion counters. |
show identity-cache |
show_identity_cache |
Cached (node_addr → npub) entries with last-seen timestamps. |
show native-flows |
show_native_flows |
Native datagram API: open and pending flows with their ports, queue depth and age, bound listeners with their backlog, and the native counters. |
acl <what>
| Subcommand | Control-socket command | Returns |
|---|---|---|
acl show |
show_acl |
Loaded peer-ACL state: allow/deny files, effective mode, default decision, entry counts. |
stats <what>
Time-series metrics from the in-process history rings.
| Subcommand | Control-socket command | Description |
|---|---|---|
stats list |
show_stats_list |
Enumerate available metrics, their units, and the per-ring retention windows. |
stats metrics |
show_metrics |
Dump current counter values for every protocol metric family (forwarding, discovery, tree, bloom, congestion, errors, native). |
stats peers |
show_stats_peers |
List peers tracked in stats history (active or recently active). |
stats history <metric> [options] |
show_stats_history |
Fetch a time-series window for one metric. |
stats history options:
| Flag | Argument | Default | Description |
|---|---|---|---|
--peer |
npub or hostname |
(none) | Required for per-peer metrics; resolves through /etc/fips/hosts if not an npub. |
--window |
<N>s / <N>m / <N>h |
10m |
Window duration. |
--granularity |
1s or 1m |
1s |
Ring resolution. 1s uses the fast ring; 1m uses the slow ring. |
--plot |
— | off | Render a Unicode-block sparkline to stdout instead of JSON. |
keygen [options]
Generate a new FIPS identity keypair locally. Does not contact the daemon.
| Flag | Argument | Default | Description |
|---|---|---|---|
-d, --dir |
DIR |
/usr/local/etc/fips (macOS), /etc/fips (other Unix), %APPDATA%\fips (Windows) |
Output directory for fips.key and fips.pub. Matches the directory the platform's packaging installs config into, which is where the daemon derives the key paths from. |
-f, --force |
— | off | Overwrite an existing fips.key. |
-s, --stdout |
— | off | Print nsec then npub to stdout instead of writing files. |
fips.key is written with mode 0600 and fips.pub with mode 0644
on Unix. After running keygen, set node.identity.persistent: true
in fips.yaml or the daemon will overwrite the keys on next start.
connect <peer> <address> <transport>
Tell the daemon to dial a peer over a specific transport.
| Argument | Description |
|---|---|
peer |
npub (bech32) or hostname from /etc/fips/hosts. |
address |
Transport endpoint, e.g. 192.168.1.10:2121, [2001:db8::1]:2121, or a Tor onion. FIPS-mesh ULAs (fd00::/8) are rejected for the IP-based transports (udp, tcp, ethernet). |
transport |
One of udp, tcp, tor, nym, ethernet. The named transport must be configured and running. |
disconnect <peer>
Tell the daemon to drop a peer link.
| Argument | Description |
|---|---|
peer |
npub (bech32) or hostname from /etc/fips/hosts. |
probe <target>
Diagnose whether a mesh endpoint is reachable, in five stages, and report a per-stage verdict so a partial failure localizes itself.
| Argument | Description |
|---|---|
target |
npub (bech32) or hostname from /etc/fips/hosts. |
--json |
Emit the report as JSON instead of human-readable text. |
--timeout <secs> |
Client-side ceiling. Defaults to the budget the daemon computed, which scales with its tick interval. |
The stages are:
- bloom — does any peer's announced filter claim the target, and did a LookupRequest therefore go out? A miss ends the probe here and is a statement about the mesh's own knowledge: nobody has heard of this address. Skipped when the coordinates are already cached, and when the target is a directly connected peer.
- discovery — waiting for a LookupResponse to answer with coordinates. Each request the node sends gets its own line under the stage, with the timeout that attempt was given and whether it drew a reply. Failing here is the opposite finding to a bloom miss: a peer's filter did claim the address, and nothing answered for it.
- path — the least-common-ancestor walk between the two
coordinates, plus the next hop this node would select. This is a
local computation, not a traceroute: no hop beyond the first is
contacted, and the tree distance is an upper bound on the real hop
count because a crosslink cut-through can deliver in fewer hops.
When no coordinates were available the walk is not computed at all:
path.coords_knownis false and every tree field is null, rather than a default that would read as a finding about the spanning tree. - session — a full Noise XK handshake over FSP. Completing it is genuine end-to-end evidence: our route reached them, their route reached us, and the remote holds the expected static key.
- rtt — one MMP sender/receiver report exchange, for a real round-trip time.
Exit status is 0 only for an overall verdict of ok; partial,
failed and cancelled all exit 1.
The stage block fills in as the probe runs. Each stage reports its verdict at the moment it reaches one, rather than the whole report arriving at the end, so a slow stage is visible as the stage that is slow. On a terminal the block is redrawn in place, with a spinner and a running elapsed on whichever stage is working; piped or redirected, each row is printed once, when it settles, and the transcript ends up the same block. A running stage reports only what the daemon has observed — which requests have gone unanswered so far, whether the handshake is in flight, how many receiver reports have arrived — and never previews an outcome it does not have yet. The per-request lines under the discovery stage carry the configured timeout of each attempt in parentheses, which is the node's own ladder rather than a measurement.
The elapsed column comes from the daemon's clock throughout: a finished stage carries its own tick-quantized figure, and the stage still running carries the report's elapsed less the stages already accounted for. Nothing in that column is measured client-side.
Stages that were never attempted get no row. A failure marks everything behind it as not reached, and the report says that once, in the failed row, rather than three more times. Two cases deliberately keep their rows: a skipped stage, because a skip is a result naming why that stage was unnecessary, and everything after a failed path stage, because the path preview touches nothing and the session can still succeed where the preview named no next hop.
Below the stage block, the path: line renders the whole tree walk on
one line, from this node to the target, through the least common
ancestor, which is emphasised on a terminal. It is the same computed
walk the ours, theirs and tree walk lines describe, read in one
piece.
--json is unaffected and still emits exactly one document, when the
probe ends, so a script parsing the report does not have to skip past
progress output.
What the probe leaves behind. It tears down a session it opened itself and never touches one that already existed. Three residues are deliberate and worth knowing about:
- The coordinate-cache and identity-cache entries a lookup produced are not evicted. They are TTL-bounded shared read caches, and evicting them could strand an unrelated flow mid-route.
- The remote's half of a probe-created session persists until its own idle timeout (default 90s). There is no teardown wire message. In the window between our removal and its idle purge, any session frame the remote sends lands here as one unknown-session reject.
- To obtain a round-trip time the probe sends a
CoordsWarmup, which starts MMP reporting on the session. On a session the probe does not own, that reporting continues until the idle purge — the same traffic any single data packet would cause, and bounded, but a real change to a session the probe did not create. The report names it undercleanup.warmups_sent.
Running a probe against a production node is safe: the job carries its own deadline daemon-side, so it cleans up whether or not the client is still there.
profile tick <on|off|status>
Reading the output. Step durations are wall clock measured across
awaitpoints, not CPU time: a step that waits on I/O accrues that wait, and other tasks may run inside the span. That is the intended measure for head-of-line delay, and it means a large step is not necessarily an expensive one.arm_starvationis measured directly as the entry time minus the deadline the interval scheduled that tick for. It is not derived fromtick_entry_gap, which carries no starvation signal on its own: under a steady delay every gap is exactly one tick period.
Start, stop and inspect a capture of the rx-loop tick body. Present
only when both fipsctl and the daemon are built with
--features profiling; the feature is off by default, so a stock
package does not carry this subcommand and a stock daemon reports
profile_tick_* as an unknown command.
| Subcommand | Control-socket command | Description |
|---|---|---|
profile tick on |
profile_tick_on |
Create the capture file and start recording. Fails if a capture is already running (naming the active file) or if the directory cannot be written. |
profile tick off |
profile_tick_off |
Stop the capture. The writer is woken immediately, drains once more and is joined, so the command returns promptly. Succeeds, reporting nothing active, when no capture is running. |
profile tick status |
profile_tick_status |
Report idle, running, stopped_by_cap or stopped_by_error, plus the active path, bytes written, flush interval and byte cap. |
profile tick on options:
| Flag | Argument | Default | Description |
|---|---|---|---|
--dir |
directory path | /var/log/fips |
Where to write the capture. Created if absent. Use it to profile a non-root cargo run, or on a platform whose log root differs. |
One file is written per capture, named profile-<UTC timestamp>.tsv.
It opens with a #-prefixed header block (node npub, build version,
platform, configured tick period, flush interval, byte cap, start
time), then a tab-separated column header, then one row per measured
step per flush interval:
ts_unix kind domain name count max total unit
kind is step for a timed span and gauge for a sampled scalar, so
a gauge value never lands under a duration column; unit names the
unit of max and total for that row. Every step present in the build
gets a row every interval, including zero-count rows. Gauges cover
ticks per interval, peer count, the wall gap between successive
tick-arm entries, and the arm-starvation delay, which is measured
against the deadline the tick was scheduled for rather than derived
from the gap.
A capture stops itself on reaching 32 MB, appending a # line saying
so; profile tick status then reports stopped_by_cap until the next
on or off clears it.
Exit Codes
| Code | Meaning |
|---|---|
0 |
Daemon returned {"status":"ok",...}. |
1 |
Argument parse failure, control-socket connection failure, daemon returned {"status":"error",...}, or local I/O failure (keygen). The error message is printed to stderr. |
Environment
| Variable | Description |
|---|---|
XDG_RUNTIME_DIR |
Used to derive the default control-socket path when /run/fips is absent. |
fipsctl does not consume RUST_LOG; logging is for the daemon.
Files
| Path | Purpose |
|---|---|
/etc/fips/hosts |
Maps hostnames to npubs for the connect, disconnect, and --peer arguments. See configuration.md. |
| Control socket (default) | Same resolution as the daemon: /run/fips/control.sock if present; then /var/run/fips/control.sock on macOS/FreeBSD if present; then $XDG_RUNTIME_DIR/fips/control.sock; finally /tmp/fips-control.sock (Unix). A privileged macOS daemon bootstraps the private /var/run/fips directory. Windows uses TCP localhost:21210. |
If you get Permission denied connecting to the socket on Linux,
add your user to the fips group (sudo usermod -aG fips $USER)
and log out and back in.
See also
fips— the daemon.fipstop— live-status TUI.- control-socket.md — wire protocol.
- configuration.md — YAML reference.