mirror of https://github.com/jmcorgan/fips.git synced 2026-10-06 03:28:24 +00:00

T

Arjen bfd4293efb feat(transport): bind and rebind network interfaces dynamically

An interface-bound transport is a long-lived object that is *sometimes
bound*. The interface it names may not exist when the daemon starts, may
appear minutes later, may vanish and return mid-operation, and may never
appear at all. Until now the first observation was final: an interface
missing at start was logged once and skipped for the life of the process,
and one that disappeared at runtime published a health change but was never
rebound.

On OpenWrt that is a live bug. procd starts fips before wifi has created
fips-mesh0 / fips-ap0, both transports are skipped and never retried, and
the 802.11s peer link forms anyway (that is mac80211, not the daemon) — so
the node looks healthy and reaches nothing.

Give every interface-bound transport presence state and a binder task.
start_async now returns Ok with the transport ABSENT rather than failing;
the binder binds when the interface appears, tears the socket down when it
goes away, and rebinds when it returns. Start-time absence and runtime
detach are one transition and one code path.

TransportError::InterfaceUnavailable makes absence branchable: a missing
interface and a typo'd interface name were the same flat StartFailed
(String), so nothing downstream could tell a state from a fault. A bind
failure that is *not* absence — no CAP_NET_RAW, no readable /dev/bpf* —
still fails the start, because retrying a socket that can never open behind
a Degraded nobody is watching is worse than dying loudly at boot.

**Presence is IFF_UP, not IFF_UP|IFF_RUNNING.** Carrier and bindability are
different questions and only the second belongs in a bind gate. An
AF_PACKET socket on a carrier-less bridge is valid and starts carrying
traffic the instant a member port comes up, with no rebind. Gating on
carrier would report a wifi-only router Degraded forever for an empty
br-lan, turn every carrier flap the socket would have survived into an
unbind/rebind cycle, and deadlock an 802.11s interface that reports RUNNING
only once it has peered — peering needs beacons, beacons need a bound
socket. Carrier is reported beside presence in show_transports instead.

**A name is not a device.** Both backends bind by device: AF_PACKET stores
sll_ifindex, a BPF descriptor follows the interface it was attached to. A
netdev deleted and recreated under the same name leaves the socket attached
to something gone while the name resolves perfectly well, and nothing
notices — a stale AF_PACKET socket never becomes readable, so the receive
loop neither errors nor exits, and send failures go to the caller rather
than the binder. A listen-only node (announce: false, no beacon sender to
fail) sat present and deaf indefinitely after a wifi reload. The bound index
is captured at bind and compared on every poll. A name reappearing with a
different MAC is different hardware: cached neighbors are dropped rather
than resumed onto.

**Detection is event-driven** where the kernel offers a source — netlink
RTNLGRP_LINK on Linux, PF_ROUTE on macOS and FreeBSD — with a 1 s
getifaddrs poll underneath as a backstop. Link-event payloads are not
parsed: an event is a hint to re-run the probe, which is cheap and
authoritative, and a parser's bugs would be presence bugs. Probes are
coalesced to ten a second because PF_ROUTE has no group filter and delivers
every routing message on the host; a persistently failing event source is
logged once, backed off, and abandoned for the poll after five errors
rather than spinning a core silently on ENOBUFS.

**Degraded becomes a level, not a latch.** The supervisor's reason set was
monotonic, which was correct while no child could recover; with recovery it
would have come to mean "something broke at some point since boot" rather
than "something is broken now". Absence lives in its own reversible set,
health is recomputed on every transition in both directions, and an absent
transport still counts as up so a single-interface node that boots before
its wifi degrades rather than exiting on NoTransports. There is deliberately
no restart action in the FSM: rebinding belongs next to the file descriptor,
and a supervisor-authored retry would be a second mechanism racing the first
for the same socket.

**The node's egress MTU follows the bound set.** transport_mtu filters on
is_bound(), not is_operational() — an interface-bound transport is
operational from the moment it starts, so the weaker predicate let hardware
that had never appeared clamp the whole node's IPv6 MTU. Because that
minimum now moves at runtime, the TUN reader and writer read the TCP MSS
ceiling from a shared atomic instead of a u16 captured at spawn; every other
consumer (show_status, the snapshot, the session-layer fragmentation check)
already read it live, so the clamp was the one place the daemon could report
one effective MTU and enforce another. It moves in both directions: a narrow
interface appearing tightens it, its departure releases it. MSS is
negotiated per connection, so a change binds connections opened after it.

**Logging is per edge, never per attempt**, with one deadline after it. The
edge itself is not an error — info at boot, warn on a runtime detach, since
an interface bound a moment later is the ordinary case this mechanism exists
to absorb and crying error at t=0 then "recovered" at t=0.2s is the failure
the rule exists to prevent. Ten seconds is the whole grace: past it, absence
is no longer a race against a radio or a container, so a required interface
still missing is reported once at error. Start-time absence and a runtime
detach share that one deadline, as they share everything else here. Said
once, not repeated: how long an absence has lasted is state, published as
interface.since_secs and as Degraded, and a monitor can threshold it per
deployment rather than the daemon compiling a schedule in.

Successful rebinds are damped. Backoff covers failed binds; the nastier case
is binds that keep succeeding into a socket that dies moments later, which a
receive loop giving up on a persistent error while the interface stays UP
produces once per second forever. Consecutive bindings dying inside ten
seconds back off on the 1 s → 30 s curve, and past three the binder stops
announcing each bind as a recovery until one lasts.

The receive loop backs off and exits on a dead socket instead of spinning on
Err with a warn per iteration, and the ad-hoc ENXIO socket reopen in the
beacon sender is gone: both hand recovery to the presence machine, one
mechanism for every cause rather than one hack per symptom. Beacons pause
while absent because the task simply does not exist then.

transports.ethernet.*.optional (default false) decides how loudly absence is
reported. Naming an interface is a statement that you expect it, so the
default complains: Degraded, and the error at the deadline. optional: true
is silent — info on the edge, no health impact, no error — for hardware
legitimately not always there. It describes the interface's presence, not
the transport's importance, and no value of it makes a missing interface
fatal at startup.

show_transports grows an `interface` block — presence, carrier, policy, how
long the phase has been held, bind count, failed attempts. The original boot
race was expensive precisely because nothing an operator could see said the
node was deaf.

2026-09-01 09:54:18 +01:00

.config

Give the UDP microbench a reason and record the retry asymmetry

2026-07-23 13:48:11 +00:00

.github

Prepare the v0.5.0 release content

2026-08-30 10:42:59 +00:00

benches

Avoid allocations in routing next-hop selection

2026-08-10 01:58:13 +00:00

docs

fix(lookup): accept our own lookup response instead of relaying it away

2026-08-31 19:35:10 +00:00

examples

Prepare the v0.5.0 release content

2026-08-30 10:42:59 +00:00

packaging

Prepare the v0.5.0 release content

2026-08-30 10:42:59 +00:00

src

feat(transport): bind and rebind network interfaces dynamically

2026-09-01 09:54:18 +01:00

testing

Prepare the v0.5.0 release content

2026-08-30 10:42:59 +00:00

.git-blame-ignore-revs

Update .git-blame-ignore-revs with master format commit

2026-04-10 08:47:20 +00:00

.gitattributes

Fix snapshot-test CRLF mismatch on Windows runners

2026-05-03 23:30:18 +00:00

.gitignore

Give each local CI run its own docker build context

2026-07-26 15:42:48 +00:00

build.rs

build: gate BLE on backend availability, and let an embedder install the radio

2026-08-26 07:58:58 +01:00

Cargo.lock

Open the development line at 0.6.0-dev

2026-08-30 16:01:13 +00:00

Cargo.toml

Open the development line at 0.6.0-dev

2026-08-30 16:01:13 +00:00

CHANGELOG.md

fix(lookup): count a request of our own that came back apart from a duplicate

2026-08-31 20:26:46 +00:00

CONTRIBUTING.md

Prepare the v0.5.0 release content

2026-08-30 10:42:59 +00:00

flake.lock

packaging(nix): add a Nix flake for reproducible from-source builds

2026-06-17 16:36:38 +00:00

flake.nix

Add NixOS flake module (nixosModules.default) + overlay

2026-08-23 17:39:22 +01:00

LICENSE

Add open-source project scaffolding

2026-02-22 20:52:55 +00:00

PR-REVIEW.md

Prepare the v0.5.0 release content

2026-08-30 10:42:59 +00:00

README.md

Open the development line at 0.6.0-dev

2026-08-30 16:01:13 +00:00

RELEASE-NOTES.md

Release v0.5.0

2026-08-30 14:39:52 +00:00

rust-toolchain.toml

Add clippy to rust-toolchain.toml components

2026-05-02 02:44:47 +00:00

rustfmt.toml

Add rustfmt formatting policy and reformat codebase

2026-04-10 08:27:07 +00:00

SECURITY.md

Add a security policy with a private reporting channel

2026-08-13 15:56:25 +00:00

README.md

FIPS: Free Internetworking Peering System

A self-organizing encrypted mesh network built on Nostr identities, capable of operating over arbitrary transports without central infrastructure.

FIPS is under active development. The protocol and APIs are not yet stable. See Status & roadmap below.

What FIPS does

A machine running FIPS becomes a node in the mesh with a self-generated cryptographic identity, tunneling existing IPv6 traffic over the mesh or bypassing IP altogether and letting natively written applications communicate directly with each other. In either case all traffic between nodes is end-to-end encrypted and authenticated.

The mesh is self-organizing and permissionless. Any node can join and reach any other node without a central address registry, routing configuration, or coordination server. Peering between nodes can be manually configured or use auto-discovery.

There are two equally-supported deployment modes.

As an overlay on top of existing IP networks, FIPS lets your node reach any other FIPS node wherever it sits: behind a NAT, on a different ISP, on a phone over cellular, on a laptop with only Bluetooth in range, or behind a Tor onion.

Ground up over raw Ethernet, WiFi, or Bluetooth, FIPS provides a complete permissionless network without any pre-existing IP infrastructure, ISP, or DNS. Any node that joins the link gets routable IPv6 addresses, peer discovery, and a path to every other node automatically. Support exists in OpenWrt for turning a router radio into a backhaul link and for creating an open access SSID so a phone or laptop can join without any configuration.

Either way, existing networking software runs over it unchanged — SSH, HTTP servers, file transfer, anything IPv6-native works the same way it would on a local network. Applications written to the FIPS native API skip that layer entirely and address each other by public key, with no IPv6 emulation.

Features

The mesh

Self-organizing mesh routing. Spanning-tree coordinates with bloom-filter-guided discovery; no global routing tables, no flooding.
Multi-transport. UDP, TCP, Ethernet, Tor, Nym, and Bluetooth (BLE L2CAP) ship today; transports compose on a single mesh and a node may run several at once.
Self-assigned cryptographic identity. secp256k1 / schnorr keypairs as node addresses; no registration, no central authority.
Two-layer encryption. Noise IK between peers (hop-by-hop) and Noise XK between mesh endpoints (independent end-to-end), with periodic rekey for forward secrecy.
(Optional) Nostr-mediated discovery and NAT traversal. Peers may publish endpoint adverts on public Nostr relays, exchange peering candidates, and establish direct paths through NATs using STUN-assisted hole punching. On the local network, mDNS LAN discovery finds peers directly without relays.

Getting traffic onto it

IPv6 adapter. A TUN interface maps each remote npub to an fd00::/8 address, so unmodified IPv6 software reaches mesh peers as <npub>.fips. Built-in .fips DNS resolver, with optional static name mapping via /etc/fips/hosts.
Native datagram API. A local program moves bytes between two public keys over the mesh, addressing a peer as npub:port with no IPv6 emulation and no TUN device in the path. connect and bind take a key and a port, and from there it is ordinary socket calls.
LAN gateway. Optional fips-gateway service folds an entire unmodified LAN into the mesh: outbound (LAN clients reach mesh destinations through a DNS-allocated virtual IPv6 pool and nftables NAT) and inbound (LAN-side services exposed to the mesh through 1:1 port forwards).
OpenWrt support. FIPS ships as an OpenWrt package. Routers run 802.11s between themselves as a bare L2 link, with FIPS supplying the encryption, authentication and routing over it. A second helper brings up an open !FIPS SSID, the same on every router, which a FIPS client joins over WiFi without configuration.

Running a node

Operator visibility. fipsctl CLI for control and inspection with time-series stats history queryable for any metric, fipstop TUI for live status with inline sparkline dashboards, and a JSON-line control socket on each binary for direct programmatic access.
Per-link metrics. RTT, loss, jitter, and goodput on every hop, plus mesh-size estimation, via the Metrics Measurement Protocol.
ECN congestion signaling. Hop-by-hop CE-flag relay with RFC 3168 IPv6 marking and transport kernel-drop detection.
Mesh-interface security baseline. Optional default-deny nftables policy for fips0 shipped as a packaged conffile (/etc/fips/fips.nft) with an operator drop-in directory (/etc/fips/fips.d/) and a disabled-by-default fips-firewall.service. The baseline polices only the mesh interface, leaving Docker, Tor, and the host firewall untouched.
Reproducible builds with toolchain pinning and SOURCE_DATE_EPOCH.

Quick start

Start from a released package. Every packaged platform in the table below gets an installer built and published per release, with checksums, on the releases page. Building from source produces the same artifacts and the same post-install state, so it is the path to take when you want to modify FIPS rather than run it.

On Debian or Ubuntu, download fips_<version>_amd64.deb (or _arm64.deb) and install it:

sudo dpkg -i fips_<version>_amd64.deb
sudo systemctl start fips fips-dns

This installs the daemon, CLI tools (fipsctl, fipstop), the fips-dns service that wires .fips name resolution into the host resolver, the optional fips-gateway service, systemd units, and a default /etc/fips/fips.yaml you can edit before starting. The package enables fips and fips-dns but starts neither, which is why the second command is there.

For macOS, Windows, FreeBSD, OpenWrt, the systemd tarball or a Nix flake, see docs/getting-started.md for the full multi-platform installation guide.

To join a live mesh and reach your first peer, follow the new-user tutorial progression starting at docs/tutorials/join-the-test-mesh.md.

Building from source

To build the Debian package yourself rather than downloading it:

git clone https://github.com/jmcorgan/fips.git
cd fips
cargo install cargo-deb
cargo deb
sudo dpkg -i target/debian/fips_*.deb

For the binaries alone, without an installer:

cargo build --release

Requires Rust 1.94.1+ (edition 2024). Linux, macOS, FreeBSD, and Windows run as standalone daemons. FreeBSD is packaged for x86_64 only; no aarch64 FreeBSD artifact is built or tested. Android is supported as an embedded crate rather than as a standalone daemon: a compile-gated library surface where the host app owns the TUN (a VpnService, for example) and reaches the built-in resolver through Node::dns_local_addr(). There is no Android daemon artifact and no host-app guide. Transport and feature availability varies by platform.

Feature	Debian/Ubuntu	Arch	NixOS	macOS	OpenWrt	FreeBSD	Android	Windows
UDP	✅	✅	✅	✅	✅	✅	✅	✅
TCP	✅	✅	✅	✅	✅	✅	✅	✅
Tor	✅	✅	✅	✅	✅	✅	❌	✅
Nym	✅	✅	✅	✅	❌	✅	❌	✅
Ethernet	✅	✅	✅	✅	✅	❌	❌	❌
BLE	✅	✅	✅	❌	❌	❌	✅	❌
Native API	✅	✅	✅	✅	✅	✅	❌	❌
Package format	`.deb`	AUR	flake	`.pkg`	`.ipk` / `.apk`	`.pkg`	❌	ZIP

A column records what builds and runs in a packaged daemon, FreeBSD on x86_64 only. Native API is the native datagram API, which is off by default; Windows cannot carry it, because it has no SCM_RIGHTS with which to pass a descriptor. Package format names the artifact you install, and a ❌ there means the platform ships none. Windows is the odd one: its ZIP is an archive you unpack yourself rather than a package an installer consumes, and there is no MSI.

Five of these columns are Linux: Debian/Ubuntu, Arch, NixOS, OpenWrt and Android. Linux is not one target. Debian, Ubuntu, Arch and NixOS are the same glibc build, and what differs is the packaging: Debian and Ubuntu take the same .deb, Arch takes fips from the AUR, and NixOS uses the Nix flake described below. Only the .deb is exercised per release, by the deb-install suite across debian12, debian13, ubuntu22, ubuntu24 and ubuntu26; neither the AUR package nor the flake is. OpenWrt is a musl target rather than glibc, and it takes an .ipk on 24.x and earlier or an .apk on 25 and later; both carry the fips-mesh-setup and fips-ap-setup helpers.

Android records what compiles for aarch64-linux-android under the CI cross-check and nothing more: no transport in that column is exercised on a device or an emulator, so read it as "compiles", not "verified here". Being an embedded crate rather than a daemon platform, it has nothing to install, which is what its ❌ package format records. The BLE cell is narrower still: the transport compiles, but the radio behind it is supplied by the embedding application rather than by FIPS, and no part of that path is device-tested.

On Linux, a source build requires libclang — the LAN gateway's nftables bindings are generated by bindgen at build time, which needs libclang.so on the build host. Install it before building (sudo apt install libclang-dev on Debian / Ubuntu); without it the build fails inside the rustables crate with an "Unable to find libclang" error. This is a build-time prerequisite only — it is not a runtime dependency, and the pre-built .deb artifacts do not need it.

BLE compiles on every glibc Linux target and on Android, and is excluded on musl. On glibc Linux, libdbus is a hard build prerequisite (sudo apt install libdbus-1-dev pkg-config on Debian / Ubuntu) — without it the build fails inside libdbus-sys rather than skipping BLE. The BlueZ daemon itself is a runtime dependency, not a build one. The OpenWrt ipk is a musl target, so it omits BLE.

Nym (mixnet) transport builds on all desktop platforms. The OpenWrt ❌ is provisional, pending verification of nym-socks5-client availability on the target; it will flip to ✅ only if confirmed buildable there.

Alternatively, the repo ships a Nix flake: nix develop drops you into a shell with the pinned toolchain and every build prerequisite (libclang, dbus, pkg-config) already provided, and nix build .#fips builds all four binaries with no host setup. See the Nix / NixOS section of packaging/README.md.

Documentation

docs/ is organised by reader purpose:

Tutorials — hand-held walk-throughs from a fresh install through to a participating mesh node, plus advanced deployments (gateway on OpenWrt, hosting services, ground-up two-device mesh).
How-to guides — operator recipes for specific tasks: firewall activation, Nostr discovery, Tor onion service, Bluetooth peering, 802.11s mesh backhaul and the open access SSID on OpenWrt, LAN gateway deployment and troubleshooting, MTU diagnostics, host aliases, persistent identity, unprivileged-user setup, UDP buffer tuning.
Reference — fips.yaml configuration, wire formats, control-socket protocol, CLI references for each binary, security posture matrix, Nostr events catalog, transport statistics inventory.
Design — protocol-level architecture and layer specifications. Start with fips-concepts.md for the framing, then fips-architecture.md for the protocol stack.
Release notes — per-version notes, including v0.5.0.

If you want to contribute, see CONTRIBUTING.md and testing/README.md.

Examples

examples/sidecar-nostr-relay/ — Run a strfry Nostr relay reachable exclusively over the FIPS mesh. The relay container shares the FIPS sidecar's network namespace and is isolated from the host network.
examples/sidecar-nostr-mixnet-relay/ — Single-container demo of FIPS peering through a mixnet (implemented with Nym): the FIPS daemon, the mixnet proxy, and a strfry Nostr relay all in one isolated container, with the direct route to the peer firewalled off so traffic provably crosses the mixnet.
examples/k8s-sidecar/ — Run FIPS as a Kubernetes Pod sidecar. The sidecar creates fips0 in the Pod's shared network namespace so every other container in the Pod gets mesh access without modification.
examples/wireguard-sidecar-macos/ — Reach the FIPS mesh from a macOS host through a local Docker container over a WireGuard tunnel. Only traffic destined for fd00::/8 transits the sidecar; regular internet traffic continues to use the host network.

Project structure

src/          Rust source: library + fips, fipsctl, fipstop, fips-gateway binaries
docs/         Documentation: tutorials, how-to, reference, design
packaging/    Debian, AUR, systemd tarball, OpenWrt ipk/apk,
              macOS .pkg, FreeBSD .pkg, Windows ZIP
examples/     Deployment examples (Nostr relay, K8s sidecar, macOS WireGuard)
testing/      Docker-based integration test harnesses + chaos simulation

Status & roadmap

FIPS is at v0.6.0-dev on the master branch. v0.5.0 has shipped; this development line continues the testing-and-polishing track toward v0.6.0. The core protocol works end-to-end over UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test mesh of thousands of nodes. v0.5.0 added FreeBSD as a packaged platform, OpenWrt setup helpers for an 802.11s mesh backhaul and an open client SSID, an Android embedding interface, a native datagram API addressed by public key, and published node health with a bounded shutdown drain. New wire-format work continues to be staged on the next branch for the subsequent release line.

What works today

Spanning-tree construction with greedy coordinate routing.
Bloom-filter-guided destination discovery (no flooding, single-path with retry).
Two-layer Noise encryption (IK at the link, XK at the session) with periodic hitless rekey for forward secrecy at both layers.
Persistent or ephemeral node identity with key-file management.
IPv6 TUN adapter with built-in .fips DNS resolver and multi-backend auto-configuration (systemd dns-delegate, systemd-resolved, dnsmasq, NetworkManager).
Native datagram API for FIPS-aware applications (npub:port addressing without the IPv6-shim path): off by default, with a surface that may still change.
Static hostname mapping (/etc/fips/hosts) with auto-reload.
Per-link metrics (RTT, loss, jitter, goodput) and mesh size estimation.
ECN congestion signaling (hop-by-hop CE relay, IPv6 CE marking, kernel-drop detection).
UDP, TCP, Ethernet, Tor, Nym (mixnet), and BLE transports (BLE via L2CAP CoC with per-link MTU negotiation).
Nostr-mediated overlay endpoint discovery and UDP hole punching for NAT traversal, plus mDNS LAN discovery for local peers.
LAN gateway (fips-gateway) with both outbound (LAN-to-mesh) and inbound (mesh-to-LAN port-forwarding) modes.
Peer ACL: per-npub allow / deny admission control at the link layer; opt-in mesh-firewall baseline at fips0 ingress.
Runtime inspection and peer management via fipsctl (including fipsctl probe for reachability diagnosis and fipsctl address for mesh-address derivation) and fipstop.
Reproducible builds with toolchain pinning and SOURCE_DATE_EPOCH.
Node lifecycle and health reporting (Starting, Running, Degraded, Failed, Draining) with a fatal start when no transport comes up and a bounded shutdown drain window.
OpenWrt setup helpers for an 802.11s mesh between routers (fips-mesh-setup) and for the open !FIPS client SSID (fips-ap-setup).
Linux (Debian, systemd tarball, OpenWrt .ipk and .apk, AUR), macOS (.pkg), FreeBSD (.pkg, x86_64 only), and Windows (ZIP, service) packaging.
Docker-based integration and chaos testing.

Near-term priorities

Security audit of the cryptographic protocols.

Longer-term

Packaged mobile applications: an Android host app, and iOS. The Android embedding interface ships today (see Building from source); what is absent is a packaged app on either platform.
Bandwidth-aware routing and QoS.
Protocol stability and a versioned wire format.
Published crate.

License

MIT — see LICENSE.