mirror of
https://github.com/jmcorgan/fips.git
synced 2026-10-05 19:18:25 +00:00
Johnathan Corgan
43b6512503
Any host able to query the LAN resolver could drain the gateway's 65,535-address virtual-IP pool one `.fips` name at a time, and each allocation rebuilt the whole nftables table in a way that could leave the host with no NAT at all. Four changes, each independently useful, close that off. Do not allocate for query types the gateway never answers with an address. handle_query minted a virtual IP for every query type and only then looked at what the client asked, answering an A or HTTPS query with NODATA after creating a mapping for it. The query type is now decided before the pool is touched, and only AAAA and ANY allocate. The refresh an existing mapping used to get from any query type is kept: it came from the reuse path in allocate, so a new pool method does that refresh alone and never creates anything, and the reuse path calls it. Rebuild the NAT table in one netlink transaction. rebuild() deleted the fips_gateway table in a batch of its own and discarded the result, then sent a second batch recreating the table, the chains, the fips0 masquerade and two rules per mapping. Between those sends the host had no NAT table, and a recreate the kernel refused left the table deleted, turning one failed mapping change into a total loss of forwarding until some later rebuild happened to succeed. The delete and the recreate now share one batch. A leading table add makes the delete legal on the first run, since rustables sends it with NLM_F_CREATE and no NLM_F_EXCL and the crate offers no flush. Deciding what to send is now separate from sending it, which is the seam the new unit tests use: they assert one batch, the add-delete-add prefix, and that every chain and rule follows the recreate, without a netlink socket or privileges. Read conntrack once per tick, off the runtime thread, and match by address. The session count searched each /proc/net/nf_conntrack line for `dst=` followed by the virtual IP's compressed Display form, while the kernel prints every tuple with `%pI6`, the full uncompressed form. That string cannot occur in that field, so the count was zero for every mapping on every kernel that has the file: nothing pinned an in-use mapping and one whose client did not re-query DNS was reclaimed about two minutes after its last DNS reference with traffic still flowing. Each `dst=` is now parsed and compared as an address. The read was also per mapping, under the pool lock, on the runtime thread that serves DNS; the tick now takes one snapshot in a blocking task before taking the lock. An unreadable source was silent, because read_to_string's error became zero through unwrap_or(0). Zero stays, since treating it as in-use would pin every mapping forever on a kernel without CONFIG_NF_CONNTRACK_PROCFS, but it is now reported at warn on the first failure and on each change of outcome, and at debug on a repeat. Ship the OpenWrt gateway disabled, and keep its state across upgrades. The generated postinst enabled and started fips-gateway on every install, against the init script's own header, the package README and the deployment tutorial, which all say the service ships disabled. A fresh install now leaves it alone. Upgrades are the awkward case: opkg runs the outgoing package's prerm first, and every released prerm disabled the gateway on its way out without recording whether it had been enabled. The new prerm stops the services on an upgrade but no longer disables them, and leaves a marker the incoming postinst reads. With the marker, enablement survived and the gateway starts only if it was enabled; without it, the outgoing package was a released one whose prerm destroyed that state, so the gateway is re-enabled rather than letting an upgrade turn off a working deployment. That re-enables a hand-disabled gateway once, which the CHANGELOG says. start_service now reads gateway.enabled from fips.yaml before touching anything, since starting a gateway the config disables used to take dnsmasq's `.fips` forwarding away from the daemon and hand it to a port whose daemon exits immediately. The four maintainer-script bodies move out of heredocs in the two build scripts into packaging/openwrt-ipk/scripts/, so the .ipk and the .apk install the same bodies and a test can run what ships. Coverage recorded rather than closed. The conntrack parser's first test builds its line from the kernel's own format string rather than a capture, because this host is built without CONFIG_NF_CONNTRACK_PROCFS and has no /proc/net/nf_conntrack, so the lab exercises only the unreadable path. Kernel acceptance of delete-then-recreate inside one transaction is not asserted by a unit test; the gateway suite is what proves it, since the manager rebuilds at startup and the daemon exits if that fails. The OpenWrt scenarios run the shipped script bodies under ash in a busybox container against stubbed init scripts, and assert their behaviour given opkg's call order, arguments and PKG_UPGRADE as read from opkg-lede's sources, not under a real opkg upgrade on a router image. Admission limits on the pool are deliberately not included here: they need a measurement run before their constants can be chosen.
FIPS OpenWrt Package
This directory is an OpenWrt feed package that builds and installs FIPS on any
OpenWrt 22.03+ router via the standard opkg package system.
Package contents
| Installed path | Purpose |
|---|---|
/usr/bin/fips |
Mesh daemon |
/usr/bin/fipsctl |
CLI control tool (fipsctl show peers, fipsctl show links, …) |
/usr/bin/fipstop |
Live TUI dashboard |
/usr/bin/fips-gateway |
Outbound LAN gateway service (not started by default) |
/usr/bin/fips-mesh-setup |
Opt-in helper — creates an open 802.11s mesh interface for router↔router backhaul |
/usr/bin/fips-ap-setup |
Opt-in helper — creates the open !FIPS access SSID for client devices |
/etc/init.d/fips |
procd service for the daemon (auto-start, crash respawn) |
/etc/init.d/fips-gateway |
procd service for the gateway (disabled by default) |
/etc/fips/fips.yaml |
Node configuration (edit before first start) |
/etc/fips/firewall.sh |
Firewall helper — accepts traffic on fips0 |
/etc/dnsmasq.d/fips.conf |
Forwards .fips DNS queries to the daemon |
/etc/sysctl.d/fips-bridge.conf |
br_netfilter settings for Ethernet transport |
/etc/sysctl.d/fips-gateway.conf |
proxy_ndp and IPv6 forwarding for the gateway |
/etc/hotplug.d/net/99-fips |
Applies firewall rules when fips0 comes up |
/etc/uci-defaults/90-fips-setup |
First-boot kernel module and firewall setup |
/lib/upgrade/keep.d/fips |
Preserves /etc/fips/ across sysupgrade |
Requirements
Build host
| Requirement | Notes |
|---|---|
| OpenWrt SDK 22.03+ | Older versions lack fw4 / nftables support |
| Rust host toolchain | Enable in make menuconfig → Advanced → Rust, or install rustup |
| Rust target for your router | Added automatically by the Makefile via rustup target add |
Router
| Requirement | Notes |
|---|---|
kmod-tun |
Required for fips0 TUN interface |
kmod-br-netfilter |
Required for Ethernet transport on bridge member ports |
Both kernel modules are listed as package dependencies (DEPENDS) and will be
installed automatically by opkg.
Target architectures
The Makefile maps the OpenWrt ARCH variable to the correct Rust musl target:
OpenWrt ARCH |
Rust target |
|---|---|
aarch64 |
aarch64-unknown-linux-musl |
x86_64 |
x86_64-unknown-linux-musl |
mipsel |
mipsel-unknown-linux-musl |
mips |
mips-unknown-linux-musl |
arm |
arm-unknown-linux-musleabihf |
To add a missing architecture, add an ifeq block in Makefile mapping the
OpenWrt ARCH value to the Rust target triple.
Building with the OpenWrt SDK
1. Obtain the SDK
Download the SDK for your router's target from downloads.openwrt.org and extract it.
2. Add this package
Copy or symlink this directory into the SDK's package/ tree:
# From inside the SDK root:
ln -s /path/to/fips/packaging/openwrt-ipk package/fips
Or add the FIPS repository as a feed in feeds.conf:
src-git-full fips https://github.com/jmcorgan/fips.git
Then update and install feeds:
./scripts/feeds update fips
./scripts/feeds install -a -p fips
3. Build
make package/fips/compile V=s
The resulting .ipk is placed in bin/packages/<arch>/.
4. Pin the source version
For reproducible production builds, replace PKG_SOURCE_VERSION:=master in
Makefile with a specific commit SHA and set PKG_MIRROR_HASH to the correct
hash (or keep skip for development):
PKG_SOURCE_VERSION:=bf117dfabc123... # full 40-char SHA
PKG_MIRROR_HASH:=skip
Installing on the router
scp bin/packages/<arch>/fips_0.1.0-1_<arch>.ipk root@192.168.1.1:/tmp/
ssh root@192.168.1.1 opkg install /tmp/fips_0.1.0-1_<arch>.ipk
First-time configuration
Edit /etc/fips/fips.yaml on the router before starting the daemon:
ssh root@192.168.1.1
vi /etc/fips/fips.yaml
The default config enables:
- Persistent identity (key generated on first start, saved to
/etc/fips/fips.key) - TUN interface
fips0 - DNS responder on
127.0.0.1:5354 - UDP transport on
0.0.0.0:2121
For Ethernet transport, uncomment the ethernet: section and set the correct
physical interface names for your router. Always use physical port names
(eth0, eth1, or DSA port names like wan/lan1), never bridge names
(br-lan). The shipped default WAN port is eth0 (OpenWrt 24); on OpenWrt
25 (DSA) boards the WAN port is named wan — the .apk package ships that
default. Run ip link show to confirm the names on your board.
Service management
/etc/init.d/fips start
/etc/init.d/fips stop
/etc/init.d/fips restart
/etc/init.d/fips enable # start at boot (already enabled by opkg postinstall)
/etc/init.d/fips disable
Outbound LAN gateway (optional)
The fips-gateway service is installed but disabled by default. It
turns the router into an outbound gateway that bridges LAN clients
onto the FIPS mesh. Enable only after configuring a gateway:
section in /etc/fips/fips.yaml:
/etc/init.d/fips-gateway enable
/etc/init.d/fips-gateway start
See docs/tutorials/deploy-fips-gateway.md in the source tree for
the full walkthrough.
Inspection and logs
# Node-level status overview
fipsctl show status
# Peer table
fipsctl show peers
# Transport links
fipsctl show links
# Active end-to-end sessions
fipsctl show sessions
# Live TUI dashboard
fipstop
# Daemon logs (OpenWrt syslog)
logread | grep fips
See docs/reference/cli-fipsctl.md
for the full subcommand list.
Upgrading
Install the new .ipk over the existing one:
opkg install --force-reinstall fips_<new-version>_<arch>.ipk
The config in /etc/fips/fips.yaml and the identity key /etc/fips/fips.key
are preserved by opkg (the yaml is installed as a conffile; the key is not a
package file). Both survive sysupgrade via /lib/upgrade/keep.d/fips.