Introduce a read-snapshot plane so pure-snapshot control queries render in
the control-socket task instead of round-tripping the rx_loop, removing the
head-of-line coupling that let a busy or slow rx_loop time out fipsctl and
fipstop observability.
- ControlReadHandle: a cloneable bundle the control accept loop holds, over
the node's already-shared NodeContext and MetricsRegistry plus an
ArcSwap-published StatsSnapshot. A snapshot_dispatch seam serves cut-over
commands off-loop and falls through to the rx_loop for the rest, keeping
the rx_loop's ownership of Node intact.
- StatsSnapshot is published from the tick (the natural and sole mutator of
stats_history), carrying the history rings plus the scalar gauges and
counts show_status reports. Readers serve the latest snapshot
unconditionally, with staleness bounded by the tick interval and no
IO_TIMEOUT-coupled fallback.
- Off-loop now: show_status, show_stats_history, show_stats_all_history,
show_listening_sockets, show_stats_list, and a new counter-only
show_metrics (exposed as fipsctl "stats metrics", the enabler for a
Prometheus scraper at no hot-path cost). Queries that need live per-entity
state (peers, links, sessions, routing, and the per-peer stats variants)
stay on the rx_loop path pending later phases.
Quartet green; forward-merge to next verified clean.
Surfaces local services reachable from the mesh, paired with their
current `inet fips` baseline filter classification. Lands to the
right of the existing TUN section in the Traffic block.
A new daemon control query `show_listening_sockets` returns IPv6
listeners bound to either `::` (wildcard) or the node's fd00::/8
address, each classified as Accept / Drop / Unknown / NoFirewall
against the running inbound chain. fipstop renders the result as a
table beside the Traffic counters: Accept rows in default White,
Drop / Unknown in DarkGray, a yellow banner above the table when
`fips-firewall.service` is inactive, and a trailing `*` on
wildcard binds to remind the operator the bind is not
fips0-specific.
Daemon side:
- `src/control/listening.rs` walks `/proc/net/tcp6` and
`/proc/net/udp6` via the procfs crate (LISTEN state for TCP,
wildcard remote for UDP), filters to fips0-reachable binds, and
resolves inodes to PID / comm via `/proc/<pid>/fd`.
- `src/control/firewall_state.rs` shells out to
`nft -j list table inet fips` and walks the inbound chain.
Recognises canonical accepts (`tcp/udp dport N accept`,
`dport { ... } accept`, `dport A-B accept`), the iifname-scoping
line, conntrack and icmpv6 lines (skipped). Any rule with
unrecognised matchers (saddr filters, jumps, daddr filters) or
non-terminal verdicts forces Unknown classification for the
ports it references. Eleven unit tests cover the classification
logic; the listening enumerator carries a /proc-parsing test of
its own.
- `show_listening_sockets` emits
`{fips0_addr, firewall_active, sockets[]}` with per-row
`{proto, local_addr, port, pid, process, filter, wildcard_bind}`.
fipstop side:
- `src/bin/fipstop/ui/dashboard.rs` splits the Traffic block into
a 50/50 horizontal layout; the existing TUN + Forwarded panel
occupies the left half.
- `src/bin/fipstop/ui/listening.rs` renders the right half.
- `main.rs` fetches the new query each tick when the Node tab is
active. Errors are non-fatal: an old daemon without the query
leaves the payload at None and the panel renders "loading...".
`Cargo.toml` gains `procfs = "0.18"` on the Linux target. IPv4
listeners are not enumerated — fips0 is IPv6-only.
Folded in: revert the default-socket lookup from writability-probe
back to existence-based selection. The previous tempfile-probe on
`/run/fips` silently steered fipstop / fipsctl onto an XDG path
the daemon never bound for any user in the `fips` group whose
shell session had not yet picked up the supplementary group (no
re-login after `usermod -aG`). `XDG_RUNTIME_DIR` is set on every
modern systemd-managed user session, so this hit the common case.
The kernel checks actual group membership at `connect(2)`, so a
user who genuinely cannot connect now gets a clear `EACCES`
rather than a silent path mismatch. Drops the now-unused
`is_writable_dir` helper. `XDG_RUNTIME_DIR` existence validation
is preserved.
Documentation:
- `docs/reference/cli-fipstop.md` — Node-tab row updated, new
"Listening on fips0 panel" section.
- `docs/reference/control-socket.md` — `show_listening_sockets`
added to the read-only queries table.
- `docs/how-to/enable-mesh-firewall.md` — new "Verify with
fipstop" section.
- `docs/tutorials/host-a-service.md` — fipstop callouts at
Steps 3, 5, 6 + Troubleshooting bullet + wildcard-bind reminder
under "What you've learned".
- `CHANGELOG.md` — new bullet under `Added / Operator Tooling`,
resolver `Fixed` entry rewritten to describe the
existence-based final shape.
In-memory time-series history on the daemon: fast ring (1s × 3600)
plus slow ring (1m × 1440) per metric, covering node-level gauges
(mesh size, tree depth, peer count, active sessions), counters
(parent switches, aggregate bytes/packets in/out), loss rate, and
seven per-peer metrics keyed by NodeAddr (srtt_ms, loss_rate,
bytes_in/out, packets_in/out, ecn_ce). The slow ring is produced by
downsampling the fast ring on minute boundaries with Last / Sum /
Mean aggregation chosen per metric type.
Missing data is first-class. New peers back-fill NaN so every ring
shares a time axis with the node rings; peers absent from a tick
sample NaN (keeps alignment, shows as a visible gap); counter metrics
emit NaN on decrease (new link_stats baseline after reconnect) so
deltas aren't polluted. Peers are evicted 24h after last contact.
Downsampling is NaN-aware: mean skips NaN, all-NaN slow windows stay
NaN. Each history window always returns its full span at the chosen
density (1m / 10m / 1h / 24h), front-padded with NaN when the ring
hasn't yet accumulated enough samples, so switching between windows
feels like zooming in or out rather than clipping to whatever has
arrived. NaN serializes to JSON null via a custom serializer.
Control socket queries:
- show_stats_list enumerates registered metrics plus scope field and
peer_retention_seconds.
- show_stats_history returns one metric's series for a given window
and granularity; accepts optional peer (npub) for per-peer metrics.
- show_stats_all_history returns every metric in a single round trip;
accepts optional peer to fetch all seven per-peer metrics.
- show_stats_peers enumerates tracked peers with lifecycle metadata.
- show_stats_history_all_peers returns one metric across all peers
for grid rendering.
- show_status carries short sparkline windows for the dashboard so
the client can render without extra fetches.
fipsctl gains `stats list`, `stats peers`, and `stats history
<metric>` with `--peer` (hostname or npub) and `--plot` for a Unicode
block sparkline. Plot header reports sample count, granularity,
window, and gap count; NaN renders as a blank cell.
fipstop dashboard grows inline sparklines (peer count, mesh size,
aggregate bytes in/out). A new Graphs tab stacks every metric as an
independent mini plot with its own autoscaled range; each plot uses
btop's braille 2×4 filled-area algorithm (25-entry lookup table
packing two samples per character, per-row gradient coloring for the
characteristic btop vertical-band look, rounded borders with embedded
titles). Three modes are cycled with `m`: Node (node-level stack),
MetricByPeer (small-multiples grid, 1 / 2 / 3 columns by terminal
width), PeerByMetric (existing stack scoped to one peer). `n` / `N`
cycles the mode-specific selector (metric or peer), a selector row
shows the current choice, and Graphs-tab refreshes re-fetch
show_stats_peers so selectors track peer churn. Up / Down scrolls
the stack, Left / Right cycles the window, `g` jumps to the tab.
Implements IDEA-0084 (TASK-2026-0062).
Gate platform-specific code behind cfg attributes and add full Windows
support: TUN device via wintun, TCP control socket on localhost:21210,
Windows Service lifecycle (--install-service/--uninstall-service/--service),
CI build and test matrix, and packaging with ZIP builder and PowerShell
service management scripts.
Key changes:
- Cargo.toml: move tun/libc/rtnetlink behind cfg(unix); add wintun and
windows-service dependencies for Windows
- upper/tun.rs: wintun-based TUN implementation with netsh configuration
for IPv6 address, MTU, and fd00::/8 routing
- control/mod.rs: split into unix_impl/windows_impl; Windows uses TCP on
localhost:21210 with shared connection handler
- bin/fips.rs: refactor main() into run_daemon() accepting a shutdown
signal; add Windows Service support via windows-service crate
- transport/udp/socket.rs: platform-gated modules; Windows uses
tokio::net::UdpSocket (kernel drop count unavailable, returns 0)
- transport/ethernet: gate to cfg(unix); add Windows stub types
- config: platform-conditional default paths (socket, hosts) for Windows
- CI: add windows-latest to build matrix and test-windows job with
cargo-nextest
- packaging/windows: build-zip.ps1, install-service.ps1,
uninstall-service.ps1, and package-windows.yml workflow
- README/docs: Windows build instructions, service management, and
control socket platform differences
Linux and macOS behavior is unchanged.
Add runtime peer management to the FIPS daemon via control socket
commands, and a new chaos simulation scenario that exercises dynamic
topology mutation with ephemeral node identities.
Daemon (connect/disconnect commands):
- Extend control socket Request with optional params field
- Add commands.rs module for mutating command dispatch, separate from
read-only queries
- Add api_connect() on Node: builds ephemeral PeerConfig (no auto-
reconnect), pre-seeds identity cache, reuses initiate_peer_connection
- Add api_disconnect() on Node: calls remove_active_peer(), clears
retry_pending to suppress reconnection
- Route non-show_* commands to async command dispatch in rx_loop
fipsctl CLI:
- Add Connect and Disconnect subcommands accepting npub or hostname
- Resolve hostnames from /etc/fips/hosts before sending to daemon
- Refactor socket I/O into reusable send_request helper
Chaos simulator (maelstrom scenario):
- Add PeerChurnManager: periodically disconnects a random active link
and connects a random unconnected node pair via control socket
- Add send_command() to control.py using base64-encoded JSON payloads
to avoid shell quoting issues in docker exec
- Add PeerChurnConfig to scenario with interval and ephemeral_fraction
- Ephemeral identity support: nodes configured without nsec generate
fresh keypairs on restart; simulator queries show_status for new
npub and updates its cache via on_node_restart callback
- Add maelstrom.yaml: all chaos dimensions (netem, link flaps, node
churn, peer topology churn, traffic) with 50% ephemeral identity
Rekey dual-initiation race (FMP + FSP):
When both sides' rekey timers fire simultaneously on high-latency links
(Tor ~700ms RTT), both msg1s cross in flight before dampening can
suppress the second initiation. Each node acts as both initiator and
responder, with set_pending_session() from the responder path destroying
the initiator's in-progress state. Each side ends up with a
pending_new_session from a different Noise handshake, causing AEAD
verification failure after cutover and link death. Fix: deterministic
tie-breaker in both FMP msg1 handler and FSP SessionSetup handler
(smaller NodeAddr wins as initiator). Also guard against pending session
overwrite from retransmitted msg1s.
Parent selection SRTT gate:
Peers without MMP RTT data are excluded from parent candidacy via
has_srtt() filter, preventing freshly reconnected peers from appearing
artificially attractive. First-RTT triggers immediate parent re-eval.
The gate in evaluate_parent() skips unmeasured candidates when any peer
has cost data, but falls back to default cost 1.0 during cold start
when no peer has MMP data yet.
Auto-reconnect on all peer removal paths:
The excessive-decryption-failure and peer-restart epoch-mismatch removal
paths were missing schedule_reconnect() calls, causing auto-connect peers
to be permanently abandoned after those events.
Control socket accessibility:
Socket and parent directory chowned to root:fips with mode 0770/0750 so
group members can use fipsctl/fipstop without root.
Log level changes:
Default RUST_LOG changed to debug in systemd service files. "Unknown
session index" log reduced from debug to trace.
Add a Unix domain socket interface for querying node state at runtime.
A spawned tokio task accepts connections and communicates with the main
event loop via mpsc/oneshot channels, keeping all Node access
single-threaded.
Includes:
- src/control/ module with socket lifecycle, JSON protocol, and 11
query handlers (status, peers, links, tree, sessions, bloom, mmp,
cache, connections, transports, routing)
- Separate fipsctl binary for CLI queries (fipsctl show <command>)
- ControlConfig in node configuration (enabled, socket_path)
- Integration into the main select! event loop
Johnathan Corgan
OceanSlimandGitHub