Files
fips/packaging/openwrt-ipk/build-ipk.sh
T
Johnathan Corgan 43b6512503 fix(gateway): harden the virtual-IP pool and ship it disabled on OpenWrt
Any host able to query the LAN resolver could drain the gateway's
65,535-address virtual-IP pool one `.fips` name at a time, and each allocation
rebuilt the whole nftables table in a way that could leave the host with no
NAT at all. Four changes, each independently useful, close that off.

Do not allocate for query types the gateway never answers with an address.
handle_query minted a virtual IP for every query type and only then looked at
what the client asked, answering an A or HTTPS query with NODATA after
creating a mapping for it. The query type is now decided before the pool is
touched, and only AAAA and ANY allocate. The refresh an existing mapping used
to get from any query type is kept: it came from the reuse path in allocate,
so a new pool method does that refresh alone and never creates anything, and
the reuse path calls it.

Rebuild the NAT table in one netlink transaction. rebuild() deleted the
fips_gateway table in a batch of its own and discarded the result, then sent a
second batch recreating the table, the chains, the fips0 masquerade and two
rules per mapping. Between those sends the host had no NAT table, and a
recreate the kernel refused left the table deleted, turning one failed mapping
change into a total loss of forwarding until some later rebuild happened to
succeed. The delete and the recreate now share one batch. A leading table add
makes the delete legal on the first run, since rustables sends it with
NLM_F_CREATE and no NLM_F_EXCL and the crate offers no flush. Deciding what to
send is now separate from sending it, which is the seam the new unit tests
use: they assert one batch, the add-delete-add prefix, and that every chain
and rule follows the recreate, without a netlink socket or privileges.

Read conntrack once per tick, off the runtime thread, and match by address.
The session count searched each /proc/net/nf_conntrack line for `dst=`
followed by the virtual IP's compressed Display form, while the kernel prints
every tuple with `%pI6`, the full uncompressed form. That string cannot occur
in that field, so the count was zero for every mapping on every kernel that
has the file: nothing pinned an in-use mapping and one whose client did not
re-query DNS was reclaimed about two minutes after its last DNS reference with
traffic still flowing. Each `dst=` is now parsed and compared as an address.
The read was also per mapping, under the pool lock, on the runtime thread that
serves DNS; the tick now takes one snapshot in a blocking task before taking
the lock. An unreadable source was silent, because read_to_string's error
became zero through unwrap_or(0). Zero stays, since treating it as in-use
would pin every mapping forever on a kernel without
CONFIG_NF_CONNTRACK_PROCFS, but it is now reported at warn on the first
failure and on each change of outcome, and at debug on a repeat.

Ship the OpenWrt gateway disabled, and keep its state across upgrades. The
generated postinst enabled and started fips-gateway on every install, against
the init script's own header, the package README and the deployment tutorial,
which all say the service ships disabled. A fresh install now leaves it alone.
Upgrades are the awkward case: opkg runs the outgoing package's prerm first,
and every released prerm disabled the gateway on its way out without recording
whether it had been enabled. The new prerm stops the services on an upgrade
but no longer disables them, and leaves a marker the incoming postinst reads.
With the marker, enablement survived and the gateway starts only if it was
enabled; without it, the outgoing package was a released one whose prerm
destroyed that state, so the gateway is re-enabled rather than letting an
upgrade turn off a working deployment. That re-enables a hand-disabled gateway
once, which the CHANGELOG says. start_service now reads gateway.enabled from
fips.yaml before touching anything, since starting a gateway the config
disables used to take dnsmasq's `.fips` forwarding away from the daemon and
hand it to a port whose daemon exits immediately. The four maintainer-script
bodies move out of heredocs in the two build scripts into
packaging/openwrt-ipk/scripts/, so the .ipk and the .apk install the same
bodies and a test can run what ships.

Coverage recorded rather than closed. The conntrack parser's first test builds
its line from the kernel's own format string rather than a capture, because
this host is built without CONFIG_NF_CONNTRACK_PROCFS and has no
/proc/net/nf_conntrack, so the lab exercises only the unreadable path. Kernel
acceptance of delete-then-recreate inside one transaction is not asserted by a
unit test; the gateway suite is what proves it, since the manager rebuilds at
startup and the daemon exits if that fails. The OpenWrt scenarios run the
shipped script bodies under ash in a busybox container against stubbed init
scripts, and assert their behaviour given opkg's call order, arguments and
PKG_UPGRADE as read from opkg-lede's sources, not under a real opkg upgrade on
a router image.

Admission limits on the pool are deliberately not included here: they need a
measurement run before their constants can be chosen.
2026-09-17 20:46:12 +00:00

275 lines
9.5 KiB
Bash
Executable File

#!/bin/bash
# Build a FIPS .ipk package for OpenWrt without the OpenWrt SDK.
#
# Uses cargo-zigbuild for cross-compilation and assembles the .ipk directly.
# An .ipk is just an ar archive containing two tarballs — no SDK required.
#
# Usage:
# ./packaging/openwrt-ipk/build-ipk.sh [--arch <name>]
#
# Architectures (--arch):
# aarch64 GL.iNet MT3000/MT6000, RPi 3/4/5, most modern routers [default]
# mipsel Older MIPS routers (TP-Link, Netgear, GL.iNet AR750)
# mips MIPS big-endian routers (ath79)
# arm 32-bit ARM routers (Cortex-A7)
# x86_64 x86 routers / VMs
#
# Output: dist/fips_<version>_<openwrt-arch>.ipk
#
# Prerequisites:
# cargo install cargo-zigbuild
# rustup target add <rust-triple> (added automatically if missing)
set -euo pipefail
# ---------------------------------------------------------------------------
# Arguments
# ---------------------------------------------------------------------------
ARCH="aarch64"
BIN_DIR="" # if set, use prebuilt binaries from here instead of compiling
while [[ $# -gt 0 ]]; do
case "$1" in
--arch) ARCH="$2"; shift 2 ;;
--arch=*) ARCH="${1#*=}"; shift ;;
--bin-dir) BIN_DIR="$2"; shift 2 ;;
--bin-dir=*) BIN_DIR="${1#*=}"; shift ;;
*) echo "Unknown argument: $1" >&2; exit 1 ;;
esac
done
# ---------------------------------------------------------------------------
# Architecture mapping
#
# RUST_TARGET — passed to cargo --target
# OPENWRT_ARCH — goes in the .ipk control file and filename
# ---------------------------------------------------------------------------
case "$ARCH" in
aarch64)
RUST_TARGET="aarch64-unknown-linux-musl"
OPENWRT_ARCH="aarch64_cortex-a53"
;;
mipsel)
RUST_TARGET="mipsel-unknown-linux-musl"
OPENWRT_ARCH="mipsel_24kc"
;;
mips)
RUST_TARGET="mips-unknown-linux-musl"
OPENWRT_ARCH="mips_24kc"
;;
arm)
RUST_TARGET="arm-unknown-linux-musleabihf"
OPENWRT_ARCH="arm_cortex-a7"
;;
x86_64)
RUST_TARGET="x86_64-unknown-linux-musl"
OPENWRT_ARCH="x86_64"
;;
*)
echo "Unknown arch: $ARCH" >&2
echo "Valid: aarch64, mipsel, mips, arm, x86_64" >&2
exit 1
;;
esac
# ---------------------------------------------------------------------------
# Paths
# ---------------------------------------------------------------------------
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
FILES_DIR="$SCRIPT_DIR/files"
SCRIPTS_SRC="$SCRIPT_DIR/scripts" # maintainer scripts (metadata, not payload)
DIST_DIR="$PROJECT_ROOT/dist"
PKG_NAME="fips"
PKG_VERSION="${PKG_VERSION:-$(cd "$PROJECT_ROOT" && git describe --tags --always --dirty 2>/dev/null || echo "0.1.0")}"
echo "==> Building $PKG_NAME $PKG_VERSION for $OPENWRT_ARCH ($RUST_TARGET)"
# ---------------------------------------------------------------------------
# 1. Obtain binaries
#
# Either use a directory of prebuilt binaries (--bin-dir; CI cross-compiles
# once in a shared job and hands them to both the .ipk and .apk packagers), or
# compile from source here for a self-contained local build.
# ---------------------------------------------------------------------------
if [ -n "$BIN_DIR" ]; then
RELEASE_DIR="$BIN_DIR"
echo "==> Using prebuilt binaries from $RELEASE_DIR"
for bin in fips fipsctl fipstop fips-gateway; do
[ -f "$RELEASE_DIR/$bin" ] || {
echo "Error: prebuilt binary not found: $RELEASE_DIR/$bin" >&2
exit 1
}
done
else
if ! command -v cargo-zigbuild &>/dev/null; then
echo "Error: cargo-zigbuild not found." >&2
echo " Install: cargo install cargo-zigbuild" >&2
exit 1
fi
if ! rustup target list --installed | grep -q "^$RUST_TARGET$"; then
echo "==> Adding Rust target $RUST_TARGET..."
rustup target add "$RUST_TARGET"
fi
echo "==> Compiling..."
cd "$PROJECT_ROOT"
cargo zigbuild \
--release \
--target "$RUST_TARGET" \
--bin fips \
--bin fipsctl \
--bin fipstop \
--bin fips-gateway
RELEASE_DIR="$PROJECT_ROOT/target/$RUST_TARGET/release"
echo "==> Stripping binaries..."
STRIP="${LLVM_STRIP:-strip}"
for bin in fips fipsctl fipstop fips-gateway; do
"$STRIP" "$RELEASE_DIR/$bin" 2>/dev/null || true
done
fi
SIZE=$(du -sh "$RELEASE_DIR/fips" | cut -f1)
echo " fips: $SIZE"
# ---------------------------------------------------------------------------
# 2. Assemble .ipk
# ---------------------------------------------------------------------------
# An .ipk is an ar archive with three members:
# debian-binary — format version ("2.0\n")
# control.tar.gz — package metadata, conffiles, pre/post scripts
# data.tar.gz — the actual filesystem tree
WORK_DIR="$(mktemp -d)"
trap 'rm -rf "$WORK_DIR"' EXIT
CONTROL_DIR="$WORK_DIR/control"
DATA_DIR="$WORK_DIR/data"
mkdir -p "$CONTROL_DIR" "$DATA_DIR"
# ---- data tree ----
install -d "$DATA_DIR/usr/bin"
install -m 0755 "$RELEASE_DIR/fips" "$DATA_DIR/usr/bin/fips"
install -m 0755 "$RELEASE_DIR/fipsctl" "$DATA_DIR/usr/bin/fipsctl"
install -m 0755 "$RELEASE_DIR/fipstop" "$DATA_DIR/usr/bin/fipstop"
install -m 0755 "$RELEASE_DIR/fips-gateway" "$DATA_DIR/usr/bin/fips-gateway"
install -m 0755 "$FILES_DIR/usr/bin/fips-mesh-setup" "$DATA_DIR/usr/bin/fips-mesh-setup"
install -m 0755 "$FILES_DIR/usr/bin/fips-ap-setup" "$DATA_DIR/usr/bin/fips-ap-setup"
install -d "$DATA_DIR/etc/init.d"
install -m 0755 "$FILES_DIR/etc/init.d/fips" "$DATA_DIR/etc/init.d/fips"
install -m 0755 "$FILES_DIR/etc/init.d/fips-gateway" "$DATA_DIR/etc/init.d/fips-gateway"
install -d "$DATA_DIR/etc/fips"
install -m 0600 "$FILES_DIR/etc/fips/fips.yaml" "$DATA_DIR/etc/fips/fips.yaml"
install -m 0755 "$FILES_DIR/etc/fips/firewall.sh" "$DATA_DIR/etc/fips/firewall.sh"
install -d "$DATA_DIR/etc/dnsmasq.d"
install -m 0644 "$FILES_DIR/etc/dnsmasq.d/fips.conf" "$DATA_DIR/etc/dnsmasq.d/fips.conf"
install -d "$DATA_DIR/etc/sysctl.d"
install -m 0644 "$FILES_DIR/etc/sysctl.d/fips-bridge.conf" "$DATA_DIR/etc/sysctl.d/fips-bridge.conf"
install -m 0644 "$FILES_DIR/etc/sysctl.d/fips-gateway.conf" "$DATA_DIR/etc/sysctl.d/fips-gateway.conf"
install -d "$DATA_DIR/etc/hotplug.d/net"
install -m 0755 "$FILES_DIR/etc/hotplug.d/net/99-fips" "$DATA_DIR/etc/hotplug.d/net/99-fips"
install -d "$DATA_DIR/etc/uci-defaults"
install -m 0755 "$FILES_DIR/etc/uci-defaults/90-fips-setup" "$DATA_DIR/etc/uci-defaults/90-fips-setup"
install -d "$DATA_DIR/lib/upgrade/keep.d"
install -m 0644 "$FILES_DIR/lib/upgrade/keep.d/fips" "$DATA_DIR/lib/upgrade/keep.d/fips"
# ---- control files ----
PKG_SIZE=$(du -sk "$DATA_DIR" | cut -f1)
cat > "$CONTROL_DIR/control" <<EOF
Package: $PKG_NAME
Version: $PKG_VERSION
Architecture: $OPENWRT_ARCH
Maintainer: FIPS Network
Section: net
Priority: optional
Depends: kmod-tun, kmod-br-netfilter, kmod-nft-nat, kmod-nf-conntrack, ip-full
Description: FIPS Mesh Network Daemon
Distributed, decentralized mesh networking over UDP, TCP, and raw Ethernet.
Provides a TUN interface (fips0) with ULA IPv6 addressing and a DNS
responder for .fips name resolution.
Installed-Size: $PKG_SIZE
EOF
# Mark fips.yaml as a conffile so opkg won't overwrite user edits on upgrade.
cat > "$CONTROL_DIR/conffiles" <<EOF
/etc/fips/fips.yaml
EOF
# The maintainer scripts live in files of their own rather than in heredocs
# here, so testing/openwrt/ can run the same bodies the package ships.
install -m 0755 "$SCRIPTS_SRC/postinst" "$CONTROL_DIR/postinst"
install -m 0755 "$SCRIPTS_SRC/prerm" "$CONTROL_DIR/prerm"
# ---- pack ----
PKG_FILENAME="${PKG_NAME}_${PKG_VERSION}_${OPENWRT_ARCH}.ipk"
IPK_WORK="$WORK_DIR/ipk"
mkdir -p "$IPK_WORK"
echo "2.0" > "$IPK_WORK/debian-binary"
# Detect a tar that supports --format=gnu.
# On macOS, Homebrew's GNU tar is installed as 'gtar'; the system tar is BSD.
# Our filenames are short so BSD tar (ustar) works too, but gnu is preferred
# to match ipkg-build exactly and to embed numeric UID/GID.
# COPYFILE_DISABLE=1 suppresses macOS resource-fork (._*) files; no-op on Linux.
if command -v gtar &>/dev/null; then
# Homebrew GNU tar on macOS
TAR_CMD="gtar"
TAR_EXTRA_FLAGS="--format=gnu --numeric-owner"
elif tar --version 2>/dev/null | grep -q 'GNU tar'; then
# System tar is GNU tar (Linux)
TAR_CMD="tar"
TAR_EXTRA_FLAGS="--format=gnu --numeric-owner"
else
# macOS BSD tar (libarchive). Its default format is PAX (typeflag 0x78),
# which OpenWrt's busybox tar cannot handle. Force ustar explicitly.
TAR_CMD="tar"
TAR_EXTRA_FLAGS="--format=ustar"
fi
ipk_tar() {
# ipk_tar <output.tar.gz> <source-dir> [paths...]
local out="$1" src="$2"; shift 2
local mtime_flags=""
if [ -n "${SOURCE_DATE_EPOCH:-}" ]; then
mtime_flags="--mtime=@$SOURCE_DATE_EPOCH"
fi
COPYFILE_DISABLE=1 "$TAR_CMD" $TAR_EXTRA_FLAGS $mtime_flags -czf "$out" -C "$src" "$@"
}
ipk_tar "$IPK_WORK/control.tar.gz" "$CONTROL_DIR" .
ipk_tar "$IPK_WORK/data.tar.gz" "$DATA_DIR" .
# The outer .ipk container is a gzip-compressed tar — NOT an ar archive.
# (Debian .deb uses ar; OpenWrt .ipk uses tar.gz.)
# Entries must be named with ./ prefix, as ipkg-build produces.
mkdir -p "$DIST_DIR"
ipk_tar "$DIST_DIR/$PKG_FILENAME" "$IPK_WORK" ./debian-binary ./control.tar.gz ./data.tar.gz
echo ""
echo "==> Done: dist/$PKG_FILENAME"
echo " $(du -sh "$DIST_DIR/$PKG_FILENAME" | cut -f1)"
echo ""
echo "Install on router:"
echo " scp -O dist/$PKG_FILENAME root@192.168.1.1:/tmp/"
echo " ssh root@192.168.1.1 opkg install /tmp/$PKG_FILENAME"