mirror of https://github.com/jmcorgan/fips.git synced 2026-10-05 19:18:25 +00:00

T

Johnathan Corgan 3a789370b9 Add an experimental native datagram API addressed by public key

A client process opens a flow to a peer's public key on a chosen port and
sends and receives datagrams on a file descriptor the daemon hands it. No
IPv6 emulation, no TUN device, no DNS: a datagram travels from key to key.
The feature is off by default and is not a stable interface.

The wire needs no change and gets none. Every FSP data packet has carried a
port pair inside its AEAD envelope since v0.2.0, and port 256 is simply the
IPv6 shim. What was missing was a way for a program to ask for a port of its
own and be handed the traffic.

Addressing is the part worth reading twice, because the obvious design is
wrong. The x-only public key is the address. An npub is that key written in
bech32, so converting between them is a local encoding rather than a lookup
or a name service. The 16-byte node address that travels on the wire is the
first half of a SHA-256 of the key: it is a truncated hash, it does not
invert, and it appears nowhere a client can see. An earlier iteration of this
work reported a peer by that hash and could supply a key only sometimes,
which is what treating a wire identifier as an identity produces.

An accepted flow therefore always knows its peer. The key is captured where
the peer is authenticated rather than looked up when a report is rendered:
every inbound datagram passes one call site inside a handler that refuses
anything whose session is not established, and the responder has already
rejected the session unless the claimed address derives from the key it
proved. Reaching for the identity cache instead gives a best-effort answer
from a structure that evicts.

A listener is a descriptor. The daemon writes one message per arrival to it,
carrying the new flow's descriptor and the peer's address, so poll, select
and epoll work on a listener and accepting is a recvmsg. That is what lets
the API be used from a program that already has an event loop, which a
command-and-reply listener could not support: an arrival could not be waited
on beside anything else. There is no accept command and no reject command.
Refusing a flow is closing the descriptor you were handed.

The Rust surface mirrors std::net. FipsStream::connect, FipsListener::bind,
incoming, accept, io::Result and an errno mapping rather than a bespoke
error type. An address is given as an npub, as a key, or as a pair, through
one parameter, the way ToSocketAddrs takes several spellings of one thing.
Each type holds its descriptor and copies of what setup told it and nothing
else, so a stream that outlives its setup connection is not representable.

set_nonblocking, AsFd and the four deadline methods carry the names and
signatures std::net uses for the same jobs. They were asked for by a user
integrating the API with tokio: AsyncFd requires a non-blocking descriptor,
and anything receiving from a peer needs a bounded wait. AsFd is the better
of the two descriptor accessors, because the borrow cannot outlive the value
that owns the descriptor, so a reactor cannot hold a registration for a
descriptor that has since been closed and its number reused by the next
open. The non-blocking flag is read, modified and written back rather than
assigned, since the flag word carries more than that one bit and a caller may
have set O_ASYNC. A zero timeout is refused with EINVAL, because the kernel
reads a zero timeval as "wait for ever", which inverts what a caller passing
zero means; std::net refuses it for the same reason. The two directions are
separate options and stay that way. FipsListener gets no timeout methods,
matching TcpListener: bounding an accept is set_nonblocking plus the caller's
own poll, which the reactor how-to builds. A flow taken from accept is
blocking whatever the listener was set to, because the two are separate
sockets and the daemon hands over a fresh one.

One rule has no counterpart in Berkeley sockets and a client author must know
it: the v1 wire carries no half-close, so nothing peer-driven ever closes a
flow. A server written to read until the flow ends waits for a signal that
cannot arrive, holding a thread and a flow per peer until its process exits.
A program decides its own termination, and the example serves one datagram
per flow.

The tests reach a live daemon rather than a stand-in. Every public item had a
unit test against a hand-written stand-in with canned replies, and the five
entry points a program actually calls first, connect, connect_from,
connect_at, bind and the SOCKET constant, had no coverage of any kind,
because the tests that appear to cover them build a Wire over a socket pair
and hand it to the private open and hold, so nothing ever resolved a socket
path or mapped its errors. examples/native-surface.rs walks all thirty-eight
items against a running daemon and reports the number of assertions it made.
The count is read from the recorder rather than written as a literal, and the
harness asserts the exit status, the completion marker and the count
together, so deleting an assertion fails the check rather than quietly
shrinking it. Watchdogs turn a hang into a named failure, which several of
the walked behaviours would otherwise produce. The shared Docker image is
built once for every integration leg, so the new binary is staged at all ten
places the existing one is, the interop builder included, which gets a stub
because those images exercise the wire between daemon versions and older refs
do not carry the example. The platform gating was tested rather than reasoned
about: flipping all eleven gates so the native API is excluded leaves the
crate compiling clean across the workspace, every target and the profiling
feature.

The shipped docs tree gains what only the LaTeX manual under design/ had,
which is not published with the daemon. A reference entry covers the whole
surface: addressing and the port tiers, the Berkeley mapping, every method on
FipsAddr, FipsStream, FipsListener and Incoming, the errno table, the
ceilings, the four places data disappears with nothing reported, the line
protocol and the command reference. The errno table gives names rather than
numbers, since the client maps each name onto the libc constant for the
platform it was built for and the supported platforms disagree on the
numbers. A tutorial side trip stands up two throwaway nodes on one machine,
peered over loopback UDP with no TUN and no DNS, then writes a listening
program and a connecting program against them; it needs neither the public
mesh nor root, because the native path is the one that does not go through
the IPv6 adapter. The obligations a client in another language carries are a
how-to of their own, since they are a task rather than a description:
reading the setup connection with recvmsg, associating a descriptor with the
last complete line, telling an empty datagram from a close, and six others.
Serving many peers from one poll loop is another, with the whole program,
because the straightforward listener spawns a thread per flow and that is
wrong at the node's ceiling of 256. The drop causes are a table mapping each
of the seven texts DropReason::as_str produces to the counter it increments,
with drop_oversize called out as the ninth counter that is not in the table.
What a daemon restart costs is a section of its own: every flow and listener
ends, descriptors do not survive, there is no resumption, and datagrams sent
but not yet forwarded are lost through a window nothing bounds.

A stack comparison diagram places the interface against the stack a reader
already knows: the same application over HTTP, TLS, TCP, IP and Ethernet on
one side, and over its own format, FSP, FMP and a FIPS transport on the
other, aligned so each row is one concern. The two columns are not
alternatives and are not drawn as such. An unmodified IPv6 program's packets
reach fips0, and the adapter hands each one to FSP as a payload, so the left
stack runs inside the right one; the left column ends at a fork, eth0 for the
ordinary internet and fips0 for the mesh, and an arrow leaves fips0 and runs
back up into FSP's input. The row where TCP would be is empty on purpose and
names Reliable Object Delivery, which is where that capability is expected to
land. ROD is a v2 capability, the box is dashed because none of it exists
yet, and the design entry says the part a reader needs most: nothing on the
surface anticipates it, so a program written today should assume it does not
exist. Both endpoints carry a scheme and a worked port,
https://<npub>.fips:443 and fips://<npub>:443, with a footnote saying the two
ports are not the same kind of thing, a TCP port inside the tunnel on the
left and an FSP port on the right. The fips:// form is a coinage: nothing in
the tree parses it, nothing registers the scheme, and the API takes a key and
a port as separate arguments rather than a URL. The diagram also says where
the right column stops, since FIPS over UDP still rides IP and Ethernet
beneath. It appears in fips-concepts.md and fips-ipv6-adapter.md, which were
making its argument in prose without a picture, and deliberately not in
fips-architecture.md, which already carries the OSI mapping and makes the
same point about the transport row.

The gateway's control socket moves onto the same bind policy this API uses,
which is the one change here that touches deployed behaviour: fips-gateway
now tightens /run/fips to 0750. That is unreachable under the packaged
deployment, where fips.service has already created the directory at that
mode, and reachable for a source build or a container that starts the gateway
alone.

One changelog entry under Added, describing the released state: what a
client opens and reads, the addressing and why the node address is not it,
the listener being a descriptor, the std::net shape of the Rust surface,
and the one rule Berkeley sockets have no counterpart for. It says in as
many words that the wire is unchanged.

2026-08-21 05:48:23 +00:00

.config

Give the UDP microbench a reason and record the retry asymmetry

2026-07-23 13:48:11 +00:00

.github

Add an experimental native datagram API addressed by public key

2026-08-21 05:48:23 +00:00

benches

Avoid allocations in routing next-hop selection

2026-08-10 01:58:13 +00:00

docs

Add an experimental native datagram API addressed by public key

2026-08-21 05:48:23 +00:00

examples

Add an experimental native datagram API addressed by public key

2026-08-21 05:48:23 +00:00

packaging

Merge the maintenance line's comment sweep and key-material work

2026-08-16 17:39:30 +00:00

src

Add an experimental native datagram API addressed by public key

2026-08-21 05:48:23 +00:00

testing

Add an experimental native datagram API addressed by public key

2026-08-21 05:48:23 +00:00

.git-blame-ignore-revs

Update .git-blame-ignore-revs with master format commit

2026-04-10 08:47:20 +00:00

.gitattributes

Fix snapshot-test CRLF mismatch on Windows runners

2026-05-03 23:30:18 +00:00

.gitignore

Give each local CI run its own docker build context

2026-07-26 15:42:48 +00:00

build.rs

Rationalize cargo feature and platform-gate surface (#79 )

2026-04-24 13:42:30 -07:00

Cargo.lock

Merge the maintenance line's comment sweep and key-material work

2026-08-16 17:39:30 +00:00

Cargo.toml

Merge the maintenance line's comment sweep and key-material work

2026-08-16 17:39:30 +00:00

CHANGELOG.md

Add an experimental native datagram API addressed by public key

2026-08-21 05:48:23 +00:00

CONTRIBUTING.md

packaging(nix): add a Nix flake for reproducible from-source builds

2026-06-17 16:36:38 +00:00

flake.lock

packaging(nix): add a Nix flake for reproducible from-source builds

2026-06-17 16:36:38 +00:00

flake.nix

packaging(nix): add a Nix flake for reproducible from-source builds

2026-06-17 16:36:38 +00:00

LICENSE

Add open-source project scaffolding

2026-02-22 20:52:55 +00:00

PR-REVIEW.md

docs: add PR-REVIEW.md checklist and link from CONTRIBUTING

2026-05-18 17:09:55 +00:00

README.md

FreeBSD support: daemon, TUN datapath, .fips DNS, and native pkg packaging

2026-08-09 13:51:20 +00:00

RELEASE-NOTES.md

Add v0.4.1 changelog entry and release notes

2026-07-19 17:21:57 +00:00

rust-toolchain.toml

Add clippy to rust-toolchain.toml components

2026-05-02 02:44:47 +00:00

rustfmt.toml

Add rustfmt formatting policy and reformat codebase

2026-04-10 08:27:07 +00:00

SECURITY.md

Add a security policy with a private reporting channel

2026-08-13 15:56:25 +00:00

README.md

FIPS: Free Internetworking Peering System

A self-organizing encrypted mesh network built on Nostr identities, capable of operating over arbitrary transports without central infrastructure.

FIPS is under active development. The protocol and APIs are not yet stable. See Status & roadmap below.

What FIPS does

A machine running FIPS becomes a node in the mesh with a self-generated cryptographic identity (a Nostr keypair). There are two equally-supported deployment modes.

As an overlay on top of existing IP networks, FIPS lets your node reach any other FIPS node wherever it sits — behind a NAT, on a different ISP, on a phone over cellular, on a laptop with only Bluetooth in range, or behind a Tor onion. The mesh forwards IPv6 traffic transparently and end-to-end encrypted, with no central VPN concentrator or coordinating server.

Ground up over raw Ethernet, WiFi, or Bluetooth, FIPS provides a complete permissionless network without any pre-existing IP infrastructure, ISP, or DNS. Any node that joins the link gets routable IPv6 addresses, peer discovery, and a path to every other node automatically.

Either way, existing networking software runs over it unchanged — SSH, HTTP servers, file transfer, anything IPv6-native works the same way it would on a local network.

Features

Self-organizing mesh routing. Spanning-tree coordinates with bloom-filter-guided discovery; no global routing tables, no flooding.
Multi-transport. UDP, TCP, Ethernet, Tor, Nym, and Bluetooth (BLE L2CAP) ship today; transports compose on a single mesh and a node may run several at once.
Two-layer encryption. Noise IK between peers (hop-by-hop) and Noise XK between mesh endpoints (independent end-to-end), with periodic rekey for forward secrecy.
Nostr-native identity. secp256k1 / schnorr keypairs as node addresses; self-generated, no registration, no central authority.
IPv6 adapter. A TUN interface maps each remote npub to an fd00::/8 address, so unmodified IPv6 software reaches mesh peers as <npub>.fips. Built-in .fips DNS resolver, with optional static name mapping via /etc/fips/hosts.
Nostr-mediated discovery and NAT traversal. Peers publish endpoint adverts on public Nostr relays, exchange candidates via NIP-59 gift-wrapped offers and answers, and establish direct paths through NATs using STUN-assisted hole punching. On the local network, mDNS LAN discovery finds peers directly without relays.
LAN gateway. Optional fips-gateway service folds an entire unmodified LAN into the mesh: outbound (LAN clients reach mesh destinations through a DNS-allocated virtual IPv6 pool and nftables NAT) and inbound (LAN-side services exposed to the mesh through 1:1 port forwards).
Per-link metrics. RTT, loss, jitter, and goodput on every hop, plus mesh-size estimation, via the Metrics Measurement Protocol.
ECN congestion signaling. Hop-by-hop CE-flag relay with RFC 3168 IPv6 marking and transport kernel-drop detection.
Mesh-interface security baseline. Optional default-deny nftables policy for fips0 shipped as a packaged conffile (/etc/fips/fips.nft) with an operator drop-in directory (/etc/fips/fips.d/) and a disabled-by-default fips-firewall.service. The baseline polices only the mesh interface, leaving Docker, Tor, and the host firewall untouched.
Operator visibility. fipsctl CLI for control and inspection with time-series stats history queryable for any metric, fipstop TUI for live status with inline sparkline dashboards, and a JSON-line control socket on each binary for direct programmatic access.
Reproducible builds with toolchain pinning and SOURCE_DATE_EPOCH.

Quick start

The shortest path on Debian / Ubuntu:

git clone https://github.com/jmcorgan/fips.git
cd fips
cargo install cargo-deb
cargo deb
sudo dpkg -i target/debian/fips_*.deb
sudo systemctl start fips

This installs the daemon, CLI tools (fipsctl, fipstop), the optional fips-gateway service, systemd units, and a default /etc/fips/fips.yaml you can edit before starting.

For macOS, Windows, OpenWrt, the systemd tarball, a Nix flake, or a from-source build, see docs/getting-started.md for the full multi-platform installation guide.

To join a live mesh and reach your first peer, follow the new-user tutorial progression starting at docs/tutorials/join-the-test-mesh.md.

Building from source

cargo build --release

Requires Rust 1.94.1+ (edition 2024). Linux, macOS, FreeBSD, and Windows run as standalone daemons; Android is supported as an embedded library (the host app owns the TUN, e.g. a VpnService). Transport availability varies by platform.

Transport	Linux	macOS	FreeBSD	Windows	Android	OpenWrt
UDP	✅	✅	✅	✅	✅	✅
TCP	✅	✅	✅	✅	✅	✅
Ethernet	✅	✅	❌	❌	❌	✅
Tor	✅	✅	✅	✅	❌	✅
Nym	✅	✅	✅	✅	❌	❌
BLE	✅	❌	❌	❌	❌	❌

On Linux, a source build requires libclang — the LAN gateway's nftables bindings are generated by bindgen at build time, which needs libclang.so on the build host. Install it before building (sudo apt install libclang-dev on Debian / Ubuntu); without it the build fails inside the rustables crate with an "Unable to find libclang" error. This is a build-time prerequisite only — it is not a runtime dependency, and the pre-built .deb artifacts do not need it.

BLE is optional and, on Linux, requires BlueZ and libdbus (sudo apt install bluez libdbus-1-dev on Debian / Ubuntu). It is gated on a build-script probe — install the dependencies first and the cargo build line above picks it up. The OpenWrt ipk omits BLE because libdbus is not available on the target.

Nym (mixnet) transport builds on all desktop platforms. The OpenWrt ❌ is provisional, pending verification of nym-socks5-client availability on the target; it will flip to ✅ only if confirmed buildable there.

Alternatively, the repo ships a Nix flake: nix develop drops you into a shell with the pinned toolchain and every build prerequisite (libclang, dbus, pkg-config) already provided, and nix build .#fips builds all four binaries with no host setup. See the Nix / NixOS section of packaging/README.md.

Documentation

docs/ is organised by reader purpose:

Tutorials — hand-held walk-throughs from a fresh install through to a participating mesh node, plus advanced deployments (gateway on OpenWrt, hosting services, ground-up two-device mesh).
How-to guides — operator recipes for specific tasks: firewall activation, Nostr discovery, Tor onion service, Bluetooth peering, LAN gateway deployment and troubleshooting, MTU diagnostics, host aliases, persistent identity, unprivileged-user setup, UDP buffer tuning.
Reference — fips.yaml configuration, wire formats, control-socket protocol, CLI references for each binary, security posture matrix, Nostr events catalog, transport statistics inventory.
Design — protocol-level architecture and layer specifications. Start with fips-concepts.md for the framing, then fips-architecture.md for the protocol stack.

If you want to contribute, see CONTRIBUTING.md and testing/README.md.

Examples

examples/sidecar-nostr-relay/ — Run a strfry Nostr relay reachable exclusively over the FIPS mesh. The relay container shares the FIPS sidecar's network namespace and is isolated from the host network.
examples/sidecar-nostr-mixnet-relay/ — Single-container demo of FIPS peering through a mixnet (implemented with Nym): the FIPS daemon, the mixnet proxy, and a strfry Nostr relay all in one isolated container, with the direct route to the peer firewalled off so traffic provably crosses the mixnet.
examples/k8s-sidecar/ — Run FIPS as a Kubernetes Pod sidecar. The sidecar creates fips0 in the Pod's shared network namespace so every other container in the Pod gets mesh access without modification.
examples/wireguard-sidecar-macos/ — Reach the FIPS mesh from a macOS host through a local Docker container over a WireGuard tunnel. Only traffic destined for fd00::/8 transits the sidecar; regular internet traffic continues to use the host network.

Project structure

src/          Rust source: library + fips, fipsctl, fipstop, fips-gateway binaries
docs/         Documentation: tutorials, how-to, reference, design
packaging/    Debian, macOS .pkg, Windows ZIP, OpenWrt ipk, AUR, systemd tarball
examples/     Deployment examples (Nostr relay, K8s sidecar, macOS WireGuard)
testing/      Docker-based integration test harnesses + chaos simulation

Status & roadmap

FIPS is at v0.5.0-dev on the master branch. v0.4.1 has shipped; this development line continues the testing-and-polishing track toward v0.5.0. The core protocol works end-to-end over UDP, TCP, Ethernet, Tor, Nym, and Bluetooth on a global, public test mesh of thousands of nodes. v0.4.0 added the Nym mixnet transport and mDNS LAN discovery alongside the existing Nostr-mediated peer discovery, UDP NAT traversal, peer ACL, and packaging hardening. New wire-format work continues to be staged on the next branch for the subsequent release line.

What works today

Spanning-tree construction with greedy coordinate routing.
Bloom-filter-guided destination discovery (no flooding, single-path with retry).
Two-layer Noise encryption (IK at the link, XK at the session) with periodic hitless rekey for forward secrecy at both layers.
Persistent or ephemeral node identity with key-file management.
IPv6 TUN adapter with built-in .fips DNS resolver and multi-backend auto-configuration (systemd dns-delegate, systemd-resolved, dnsmasq, NetworkManager).
Static hostname mapping (/etc/fips/hosts) with auto-reload.
Per-link metrics (RTT, loss, jitter, goodput) and mesh size estimation.
ECN congestion signaling (hop-by-hop CE relay, IPv6 CE marking, kernel-drop detection).
UDP, TCP, Ethernet, Tor, Nym (mixnet), and BLE transports (BLE via L2CAP CoC with per-link MTU negotiation).
Nostr-mediated overlay endpoint discovery and UDP hole punching for NAT traversal, plus mDNS LAN discovery for local peers.
LAN gateway (fips-gateway) with both outbound (LAN-to-mesh) and inbound (mesh-to-LAN port-forwarding) modes.
Peer ACL: per-npub allow / deny admission control at the link layer; opt-in mesh-firewall baseline at fips0 ingress.
Runtime inspection and peer management via fipsctl and fipstop.
Reproducible builds with toolchain pinning and SOURCE_DATE_EPOCH.
Linux (Debian, systemd tarball, OpenWrt, AUR), macOS (.pkg), FreeBSD (.pkg), and Windows (ZIP, service) packaging.
Docker-based integration and chaos testing.

Near-term priorities

Native API for FIPS-aware applications (npub:port addressing without the IPv6-shim path).
Security audit of the cryptographic protocols.

Longer-term

Mobile platform support.
Bandwidth-aware routing and QoS.
Protocol stability and a versioned wire format.
Published crate.

License

MIT — see LICENSE.