mirror of
https://github.com/jmcorgan/fips.git
synced 2026-10-05 11:08:25 +00:00
fix(gateway): harden the virtual-IP pool and ship it disabled on OpenWrt
Any host able to query the LAN resolver could drain the gateway's 65,535-address virtual-IP pool one `.fips` name at a time, and each allocation rebuilt the whole nftables table in a way that could leave the host with no NAT at all. Four changes, each independently useful, close that off. Do not allocate for query types the gateway never answers with an address. handle_query minted a virtual IP for every query type and only then looked at what the client asked, answering an A or HTTPS query with NODATA after creating a mapping for it. The query type is now decided before the pool is touched, and only AAAA and ANY allocate. The refresh an existing mapping used to get from any query type is kept: it came from the reuse path in allocate, so a new pool method does that refresh alone and never creates anything, and the reuse path calls it. Rebuild the NAT table in one netlink transaction. rebuild() deleted the fips_gateway table in a batch of its own and discarded the result, then sent a second batch recreating the table, the chains, the fips0 masquerade and two rules per mapping. Between those sends the host had no NAT table, and a recreate the kernel refused left the table deleted, turning one failed mapping change into a total loss of forwarding until some later rebuild happened to succeed. The delete and the recreate now share one batch. A leading table add makes the delete legal on the first run, since rustables sends it with NLM_F_CREATE and no NLM_F_EXCL and the crate offers no flush. Deciding what to send is now separate from sending it, which is the seam the new unit tests use: they assert one batch, the add-delete-add prefix, and that every chain and rule follows the recreate, without a netlink socket or privileges. Read conntrack once per tick, off the runtime thread, and match by address. The session count searched each /proc/net/nf_conntrack line for `dst=` followed by the virtual IP's compressed Display form, while the kernel prints every tuple with `%pI6`, the full uncompressed form. That string cannot occur in that field, so the count was zero for every mapping on every kernel that has the file: nothing pinned an in-use mapping and one whose client did not re-query DNS was reclaimed about two minutes after its last DNS reference with traffic still flowing. Each `dst=` is now parsed and compared as an address. The read was also per mapping, under the pool lock, on the runtime thread that serves DNS; the tick now takes one snapshot in a blocking task before taking the lock. An unreadable source was silent, because read_to_string's error became zero through unwrap_or(0). Zero stays, since treating it as in-use would pin every mapping forever on a kernel without CONFIG_NF_CONNTRACK_PROCFS, but it is now reported at warn on the first failure and on each change of outcome, and at debug on a repeat. Ship the OpenWrt gateway disabled, and keep its state across upgrades. The generated postinst enabled and started fips-gateway on every install, against the init script's own header, the package README and the deployment tutorial, which all say the service ships disabled. A fresh install now leaves it alone. Upgrades are the awkward case: opkg runs the outgoing package's prerm first, and every released prerm disabled the gateway on its way out without recording whether it had been enabled. The new prerm stops the services on an upgrade but no longer disables them, and leaves a marker the incoming postinst reads. With the marker, enablement survived and the gateway starts only if it was enabled; without it, the outgoing package was a released one whose prerm destroyed that state, so the gateway is re-enabled rather than letting an upgrade turn off a working deployment. That re-enables a hand-disabled gateway once, which the CHANGELOG says. start_service now reads gateway.enabled from fips.yaml before touching anything, since starting a gateway the config disables used to take dnsmasq's `.fips` forwarding away from the daemon and hand it to a port whose daemon exits immediately. The four maintainer-script bodies move out of heredocs in the two build scripts into packaging/openwrt-ipk/scripts/, so the .ipk and the .apk install the same bodies and a test can run what ships. Coverage recorded rather than closed. The conntrack parser's first test builds its line from the kernel's own format string rather than a capture, because this host is built without CONFIG_NF_CONNTRACK_PROCFS and has no /proc/net/nf_conntrack, so the lab exercises only the unreadable path. Kernel acceptance of delete-then-recreate inside one transaction is not asserted by a unit test; the gateway suite is what proves it, since the manager rebuilds at startup and the daemon exits if that fails. The OpenWrt scenarios run the shipped script bodies under ash in a busybox container against stubbed init scripts, and assert their behaviour given opkg's call order, arguments and PKG_UPGRADE as read from opkg-lede's sources, not under a real opkg upgrade on a router image. Admission limits on the pool are deliberately not included here: they need a measurement run before their constants can be chosen.
This commit is contained in:
Johnathan Corgan
@@ -435,6 +435,40 @@ jobs:
|
||||
Write-Host "PSScriptAnalyzer: no issues"
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Job 2e – OpenWrt maintainer-script scenarios
|
||||
#
|
||||
# Runs the package's postinst/prerm and the fips-gateway init script under ash
|
||||
# in a busybox container, against stubbed init scripts: a fresh install, an
|
||||
# upgrade from a released package, an upgrade from a package carrying these
|
||||
# scripts with the gateway enabled and with it disabled, a removal, and the
|
||||
# init script's gateway.enabled guard.
|
||||
#
|
||||
# A job of its own rather than a leg of the integration matrix: it needs no
|
||||
# FIPS binary and no shared test image, so as an integration leg it would wait
|
||||
# on the build and then download and build both for nothing.
|
||||
#
|
||||
# The leg keeps `suite:` because testing/check-ci-parity.sh matches it against
|
||||
# OPENWRT_SUITES in ci-local.sh; the step below does not read it.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
openwrt-scripts:
|
||||
name: OpenWrt scripts (${{ matrix.suite }})
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ !inputs.skip_integration }}
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- suite: openwrt-scripts
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
|
||||
|
||||
- name: Run the OpenWrt maintainer-script scenarios
|
||||
timeout-minutes: 5
|
||||
run: bash testing/openwrt/maintainer-scripts-test.sh
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Job 3 – Integration tests (static mesh + chaos simulation)
|
||||
#
|
||||
|
||||
@@ -105,6 +105,75 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
names the path. A dangling symlink likewise aborts rather than being replaced.
|
||||
The legacy `/etc/fips/fips.key` lookup follows the same rule.
|
||||
|
||||
#### Gateway
|
||||
|
||||
- A `.fips` query the gateway answers without an address no longer takes an
|
||||
address from the pool. Every query type was allocated a mapping before the
|
||||
code looked at what the client had asked for, and an A or HTTPS query was
|
||||
then answered with NODATA, so any host that can reach the LAN resolver could
|
||||
consume the pool one name at a time with a query type it is never given an
|
||||
address for. Only AAAA and ANY allocate now. A non-AAAA query for a name that
|
||||
already has a mapping still refreshes that mapping's TTL clock, so a client
|
||||
querying both types does not lose half of its refresh.
|
||||
- Conntrack sessions are matched by address rather than by text, so live
|
||||
traffic pins a gateway mapping again. The session count searched each
|
||||
`/proc/net/nf_conntrack` line for `dst=` followed by the virtual IP in its
|
||||
compressed form (`fd01::1`), while the kernel prints tuples in the full
|
||||
uncompressed form (`dst=fd01:0000:0000:0000:0000:0000:0000:0001`), so the
|
||||
count was zero for every mapping on every kernel. Nothing pinned an in-use
|
||||
mapping, and one whose client did not re-query DNS was reclaimed about two
|
||||
minutes after its last DNS reference while its traffic was still flowing.
|
||||
Each `dst=` value is now parsed as an address and compared as one.
|
||||
- The conntrack table is read once per tick instead of once per mapping, and
|
||||
the read happens off the runtime thread. The whole file was read and scanned
|
||||
for each mapping in turn, while the pool lock was held, on the same
|
||||
single-threaded runtime that serves DNS. The tick now takes one snapshot with
|
||||
a blocking task before it takes the lock, and the pool does a map lookup per
|
||||
mapping.
|
||||
- A conntrack source that cannot be read is reported. It still counts as zero
|
||||
sessions for every mapping, as it always has, so reclamation keeps working
|
||||
rather than pinning the whole pool; but the first failure and each change of
|
||||
outcome after it are now logged, so an unreadable source is no longer
|
||||
indistinguishable from an idle one. A kernel built without
|
||||
`CONFIG_NF_CONNTRACK_PROCFS` has no `/proc/net/nf_conntrack` at all and fails
|
||||
identically every tick, so a repeat is logged at debug rather than warn.
|
||||
- The NAT table is rebuilt in one netlink transaction. A rebuild deleted the
|
||||
`fips_gateway` table in a batch of its own, discarded that batch's result,
|
||||
and only then sent the batch that recreated the table, the chains, the
|
||||
`fips0` masquerade and every per-mapping rule. Between the two sends the
|
||||
gateway had no NAT at all, and a recreate the kernel refused left the table
|
||||
absent for good, taking down forwarding for every existing mapping rather
|
||||
than failing the one change that was being made. The delete and the recreate
|
||||
now share a single batch, which the kernel applies as one transaction, so a
|
||||
refused rebuild leaves the previous table in the packet path. The rules sent
|
||||
are unchanged.
|
||||
- A new OpenWrt install no longer enables and starts `fips-gateway`. The
|
||||
generated postinst turned it on unconditionally, contradicting the init
|
||||
script's own header, the package README and the deployment tutorial, all of
|
||||
which say the service ships disabled and is enabled deliberately. The
|
||||
documented `service fips-gateway enable` / `service fips-gateway start` steps
|
||||
are unchanged, and the shipped `fips.yaml` still carries `gateway.enabled:
|
||||
true`, so enabling the service is all that is needed.
|
||||
- **The first upgrade to this release re-enables and starts `fips-gateway` on
|
||||
any router that has the package installed, including one where the gateway
|
||||
was disabled by hand.** Every released package's prerm disabled the service
|
||||
on its way out, leaving nothing behind that says whether the operator wanted
|
||||
it on, so an upgrade cannot tell the two apart and keeps the gateway running
|
||||
rather than silently turning off a working one. If you had disabled it, run
|
||||
`service fips-gateway disable` once after upgrading. Later upgrades preserve
|
||||
whatever state the service is in: the new prerm stops the services on an
|
||||
upgrade but no longer disables them.
|
||||
- `start_service` in the `fips-gateway` init script now reads `gateway.enabled`
|
||||
from `/etc/fips/fips.yaml` before doing anything. Starting a gateway that the
|
||||
config disables used to hand dnsmasq's `.fips` forwarding to the gateway's
|
||||
port, add the LAN prefix and advertise the pool route, and only then start a
|
||||
daemon that exits immediately because the gateway is disabled, leaving `.fips`
|
||||
resolution pointed at a port nothing listens on.
|
||||
- The four OpenWrt maintainer-script bodies now live in
|
||||
`packaging/openwrt-ipk/scripts/` instead of inside heredocs in the two build
|
||||
scripts, so the `.ipk` and `.apk` packages install the same bodies and the
|
||||
scenarios in `testing/openwrt/` run what ships.
|
||||
|
||||
### Changed
|
||||
|
||||
- The lockfile moves `chacha20` from 0.10.1 to 0.10.2, because 0.10.1 is yanked.
|
||||
|
||||
@@ -94,6 +94,7 @@ PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
# The installed-filesystem payload (init scripts, config, sysctl, etc.) is
|
||||
# shared with the .ipk package; there is one canonical copy in openwrt-ipk/.
|
||||
FILES_DIR="$PROJECT_ROOT/packaging/openwrt-ipk/files"
|
||||
SCRIPTS_SRC="$PROJECT_ROOT/packaging/openwrt-ipk/scripts"
|
||||
DIST_DIR="$PROJECT_ROOT/dist"
|
||||
|
||||
PKG_NAME="fips"
|
||||
@@ -228,33 +229,15 @@ EOF
|
||||
|
||||
# ---- maintainer scripts ----
|
||||
# Map our opkg maintainer scripts onto apk's lifecycle phases:
|
||||
# opkg postinst -> apk post-install (enable + start services)
|
||||
# opkg postinst -> apk post-install (enable + start the daemon)
|
||||
# opkg prerm -> apk pre-deinstall (stop + disable services)
|
||||
|
||||
cat > "$SCRIPTS_DIR/post-install" <<'EOF'
|
||||
#!/bin/sh
|
||||
# Run first-boot UCI setup (the script deletes itself when done).
|
||||
if [ -x /etc/uci-defaults/90-fips-setup ]; then
|
||||
/etc/uci-defaults/90-fips-setup && rm -f /etc/uci-defaults/90-fips-setup
|
||||
fi
|
||||
|
||||
/etc/init.d/fips enable
|
||||
/etc/init.d/fips start
|
||||
/etc/init.d/fips-gateway enable
|
||||
/etc/init.d/fips-gateway start
|
||||
exit 0
|
||||
EOF
|
||||
|
||||
cat > "$SCRIPTS_DIR/pre-deinstall" <<'EOF'
|
||||
#!/bin/sh
|
||||
/etc/init.d/fips-gateway stop 2>/dev/null || true
|
||||
/etc/init.d/fips-gateway disable 2>/dev/null || true
|
||||
/etc/init.d/fips stop 2>/dev/null || true
|
||||
/etc/init.d/fips disable 2>/dev/null || true
|
||||
exit 0
|
||||
EOF
|
||||
|
||||
chmod 0755 "$SCRIPTS_DIR/post-install" "$SCRIPTS_DIR/pre-deinstall"
|
||||
# Both bodies come from packaging/openwrt-ipk/scripts/, the same files the
|
||||
# .ipk ships, so the two packagers cannot drift apart and testing/openwrt/
|
||||
# exercises what both install. apk runs post-install only on a fresh install,
|
||||
# so the postinst's upgrade branch is unreachable here.
|
||||
install -m 0755 "$SCRIPTS_SRC/postinst" "$SCRIPTS_DIR/post-install"
|
||||
install -m 0755 "$SCRIPTS_SRC/prerm" "$SCRIPTS_DIR/pre-deinstall"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Assemble the .apk via apk mkpkg
|
||||
|
||||
@@ -81,6 +81,7 @@ esac
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
FILES_DIR="$SCRIPT_DIR/files"
|
||||
SCRIPTS_SRC="$SCRIPT_DIR/scripts" # maintainer scripts (metadata, not payload)
|
||||
DIST_DIR="$PROJECT_ROOT/dist"
|
||||
|
||||
PKG_NAME="fips"
|
||||
@@ -212,30 +213,10 @@ cat > "$CONTROL_DIR/conffiles" <<EOF
|
||||
/etc/fips/fips.yaml
|
||||
EOF
|
||||
|
||||
cat > "$CONTROL_DIR/postinst" <<'EOF'
|
||||
#!/bin/sh
|
||||
# Run first-boot UCI setup (the script deletes itself when done).
|
||||
if [ -x /etc/uci-defaults/90-fips-setup ]; then
|
||||
/etc/uci-defaults/90-fips-setup && rm -f /etc/uci-defaults/90-fips-setup
|
||||
fi
|
||||
|
||||
/etc/init.d/fips enable
|
||||
/etc/init.d/fips start
|
||||
/etc/init.d/fips-gateway enable
|
||||
/etc/init.d/fips-gateway start
|
||||
exit 0
|
||||
EOF
|
||||
chmod 0755 "$CONTROL_DIR/postinst"
|
||||
|
||||
cat > "$CONTROL_DIR/prerm" <<'EOF'
|
||||
#!/bin/sh
|
||||
/etc/init.d/fips-gateway stop 2>/dev/null || true
|
||||
/etc/init.d/fips-gateway disable 2>/dev/null || true
|
||||
/etc/init.d/fips stop 2>/dev/null || true
|
||||
/etc/init.d/fips disable 2>/dev/null || true
|
||||
exit 0
|
||||
EOF
|
||||
chmod 0755 "$CONTROL_DIR/prerm"
|
||||
# The maintainer scripts live in files of their own rather than in heredocs
|
||||
# here, so testing/openwrt/ can run the same bodies the package ships.
|
||||
install -m 0755 "$SCRIPTS_SRC/postinst" "$CONTROL_DIR/postinst"
|
||||
install -m 0755 "$SCRIPTS_SRC/prerm" "$CONTROL_DIR/prerm"
|
||||
|
||||
# ---- pack ----
|
||||
|
||||
|
||||
@@ -26,6 +26,14 @@ DAEMON_DNS_PORT=5354
|
||||
GLOBAL_PREFIX="2001:2:f1b5::1/64"
|
||||
|
||||
start_service() {
|
||||
# The gateway daemon exits when gateway.enabled is not true, so without
|
||||
# this check starting a disabled gateway would still take dnsmasq's .fips
|
||||
# upstream away from the daemon and point it at a port nothing listens on.
|
||||
if [ "$(gateway_config_enabled)" != "true" ]; then
|
||||
logger -t fips-gateway "gateway.enabled is not true in $CONFIG; not starting"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Apply gateway sysctls (proxy_ndp, IPv6 forwarding).
|
||||
sysctl -p /etc/sysctl.d/fips-gateway.conf 2>/dev/null || true
|
||||
|
||||
@@ -72,6 +80,13 @@ reload_service() {
|
||||
restart
|
||||
}
|
||||
|
||||
# Extract the gateway "enabled" flag from fips.yaml.
|
||||
# Prints the value indented under the top-level "gateway:" block, or nothing
|
||||
# when there is no such block.
|
||||
gateway_config_enabled() {
|
||||
awk '/^gateway:/{found=1; next} found && /^[^ ]/{found=0} found && /enabled:/{gsub(/.*enabled:[[:space:]]*/, ""); gsub(/["'"'"']/, ""); print; exit}' "$CONFIG"
|
||||
}
|
||||
|
||||
# Extract the gateway pool CIDR from fips.yaml.
|
||||
# Looks for "pool:" indented under the top-level "gateway:" block.
|
||||
gateway_pool_cidr() {
|
||||
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
#!/bin/sh
|
||||
# Maintainer script run after the FIPS package is unpacked.
|
||||
#
|
||||
# Installed as the .ipk CONTROL/postinst and registered as the .apk
|
||||
# post-install script, so one body serves both packagers.
|
||||
#
|
||||
# The fips daemon is enabled and started on every install. The gateway is not:
|
||||
# the package ships that service disabled, and the README and the deployment
|
||||
# tutorial tell the operator to enable it deliberately.
|
||||
#
|
||||
# Upgrades are the awkward case, because opkg runs the OLD package's prerm
|
||||
# before any script from the new one:
|
||||
#
|
||||
# marker present the old package was one of these, its prerm left
|
||||
# enablement alone, and the gateway only needs starting
|
||||
# again if it was enabled;
|
||||
# no marker the old package's prerm disabled the gateway on its way
|
||||
# out, so its former state is unrecoverable; the gateway is
|
||||
# re-enabled, which also re-enables one an operator had
|
||||
# disabled by hand.
|
||||
#
|
||||
# Under apk this script runs only on a fresh install, so it takes the first
|
||||
# branch and the gateway stays off.
|
||||
|
||||
UPGRADE_MARKER=/tmp/fips-prerm-upgrade
|
||||
|
||||
# Run first-boot UCI setup (the script deletes itself when done).
|
||||
if [ -x /etc/uci-defaults/90-fips-setup ]; then
|
||||
/etc/uci-defaults/90-fips-setup && rm -f /etc/uci-defaults/90-fips-setup
|
||||
fi
|
||||
|
||||
/etc/init.d/fips enable
|
||||
/etc/init.d/fips start
|
||||
|
||||
if [ "${PKG_UPGRADE:-0}" = "1" ]; then
|
||||
if [ -e "$UPGRADE_MARKER" ]; then
|
||||
rm -f "$UPGRADE_MARKER"
|
||||
else
|
||||
/etc/init.d/fips-gateway enable
|
||||
fi
|
||||
|
||||
if /etc/init.d/fips-gateway enabled 2>/dev/null; then
|
||||
/etc/init.d/fips-gateway start
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Executable
+26
@@ -0,0 +1,26 @@
|
||||
#!/bin/sh
|
||||
# Maintainer script run before the FIPS package is removed or replaced.
|
||||
#
|
||||
# Installed as the .ipk CONTROL/prerm and registered as the .apk pre-deinstall
|
||||
# script, so one body serves both packagers.
|
||||
#
|
||||
# opkg calls this with "upgrade <new-version>" when the package is being
|
||||
# replaced. Disabling the services there would erase the operator's choice,
|
||||
# because nothing records it anywhere else, so an upgrade only stops them and
|
||||
# leaves a marker telling the incoming postinst that enablement survived.
|
||||
# A real removal stops and disables both, as before.
|
||||
|
||||
UPGRADE_MARKER=/tmp/fips-prerm-upgrade
|
||||
|
||||
if [ "$1" = "upgrade" ]; then
|
||||
: > "$UPGRADE_MARKER" 2>/dev/null || true
|
||||
/etc/init.d/fips-gateway stop 2>/dev/null || true
|
||||
/etc/init.d/fips stop 2>/dev/null || true
|
||||
exit 0
|
||||
fi
|
||||
|
||||
/etc/init.d/fips-gateway stop 2>/dev/null || true
|
||||
/etc/init.d/fips-gateway disable 2>/dev/null || true
|
||||
/etc/init.d/fips stop 2>/dev/null || true
|
||||
/etc/init.d/fips disable 2>/dev/null || true
|
||||
exit 0
|
||||
+54
-2
@@ -24,7 +24,7 @@ use tokio::signal::unix::{SignalKind, signal};
|
||||
#[cfg(target_os = "linux")]
|
||||
use tokio::sync::{Mutex, mpsc, watch};
|
||||
#[cfg(target_os = "linux")]
|
||||
use tracing::{error, info, warn};
|
||||
use tracing::{debug, error, info, warn};
|
||||
#[cfg(target_os = "linux")]
|
||||
use tracing_subscriber::{EnvFilter, fmt};
|
||||
|
||||
@@ -53,6 +53,53 @@ fn main() {
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
/// Take a conntrack snapshot off the runtime thread.
|
||||
///
|
||||
/// A failed read yields an empty snapshot, so every mapping reads zero
|
||||
/// sessions, which is what the pool did with an unreadable source before. The
|
||||
/// alternative, treating "unknown" as "in use", would pin every mapping forever
|
||||
/// on a kernel with no conntrack proc file and turn a read error into a pool
|
||||
/// that never reclaims. The cost is the opposite error: a mapping carrying live
|
||||
/// traffic can be reclaimed early while the source is unreadable.
|
||||
#[cfg(target_os = "linux")]
|
||||
async fn read_conntrack(log: &mut pool::ConntrackReadLog) -> pool::ConntrackSnapshot {
|
||||
use fips::gateway::pool::ConntrackQuerier;
|
||||
|
||||
match tokio::task::spawn_blocking(|| pool::ProcConntrack.snapshot()).await {
|
||||
Ok(Ok(snapshot)) => {
|
||||
log.observe(None);
|
||||
snapshot
|
||||
}
|
||||
Ok(Err(e)) => {
|
||||
report_unreadable_conntrack(log, e.kind(), &e.to_string());
|
||||
pool::ConntrackSnapshot::default()
|
||||
}
|
||||
Err(e) => {
|
||||
report_unreadable_conntrack(log, std::io::ErrorKind::Other, &e.to_string());
|
||||
pool::ConntrackSnapshot::default()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Log an unreadable conntrack source once per change of outcome.
|
||||
#[cfg(target_os = "linux")]
|
||||
fn report_unreadable_conntrack(
|
||||
log: &mut pool::ConntrackReadLog,
|
||||
kind: std::io::ErrorKind,
|
||||
error: &str,
|
||||
) {
|
||||
match log.observe(Some(kind)) {
|
||||
pool::ReadReport::Changed => warn!(
|
||||
error,
|
||||
"Conntrack unreadable; every mapping reads zero sessions"
|
||||
),
|
||||
pool::ReadReport::Repeated => debug!(
|
||||
error,
|
||||
"Conntrack still unreadable; every mapping reads zero sessions"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
#[tokio::main(flavor = "current_thread")]
|
||||
async fn main() {
|
||||
@@ -379,7 +426,7 @@ async fn main() {
|
||||
let tick_event_tx = event_tx;
|
||||
let tick_nat_count = Arc::clone(&nat_count);
|
||||
let mut tick_shutdown = shutdown_rx.clone();
|
||||
let conntrack = pool::ProcConntrack;
|
||||
let mut conntrack_log = pool::ConntrackReadLog::default();
|
||||
let snap_config = control::SnapshotConfig {
|
||||
pool_cidr: gw_config.pool.clone(),
|
||||
lan_interface: gw_config.lan_interface.clone(),
|
||||
@@ -395,6 +442,11 @@ async fn main() {
|
||||
tokio::select! {
|
||||
_ = interval.tick() => {
|
||||
let now = Instant::now();
|
||||
// Read conntrack once, off the runtime thread and before
|
||||
// the pool lock: the runtime is current-thread, so a
|
||||
// blocking read here would stall the DNS resolver, and the
|
||||
// read must not happen under the lock the resolver needs.
|
||||
let conntrack = read_conntrack(&mut conntrack_log).await;
|
||||
let mut pool_guard = tick_pool.lock().await;
|
||||
let events = pool_guard.tick(now, &conntrack);
|
||||
|
||||
|
||||
+153
-22
@@ -357,6 +357,32 @@ async fn handle_query(
|
||||
}
|
||||
};
|
||||
|
||||
// What the client actually asked for. Only AAAA and ANY are answered with
|
||||
// an address, and only those may mint a mapping: allocating for a query
|
||||
// type the gateway answers with NODATA let any LAN host take a pool
|
||||
// address per name without ever being given one.
|
||||
let client_qtype = query
|
||||
.questions
|
||||
.first()
|
||||
.map(|q| q.qtype)
|
||||
.unwrap_or(QTYPE::TYPE(TYPE::AAAA));
|
||||
|
||||
if !matches!(client_qtype, QTYPE::TYPE(TYPE::AAAA) | QTYPE::ANY) {
|
||||
// The client is still using the name, so an existing mapping's TTL
|
||||
// clock is refreshed. A client that re-queries a mapped name with both
|
||||
// A and AAAA should not lose half of its refresh, and with no
|
||||
// conntrack sessions a DNS reference is all that keeps a mapping
|
||||
// alive. Nothing is created.
|
||||
let refreshed = pool.lock().await.refresh_if_present(node_addr);
|
||||
debug!(
|
||||
name = %fips_name,
|
||||
mesh_addr = %mesh_addr,
|
||||
refreshed,
|
||||
"Non-AAAA .fips query, returning NODATA"
|
||||
);
|
||||
return build_nodata(&query, ttl);
|
||||
}
|
||||
|
||||
// Allocate virtual IP from pool
|
||||
let mut pool_guard = pool.lock().await;
|
||||
let (virtual_ip, is_new) = match pool_guard.allocate(node_addr, mesh_addr, &fips_name) {
|
||||
@@ -387,22 +413,7 @@ async fn handle_query(
|
||||
"Resolved .fips query"
|
||||
);
|
||||
|
||||
// Check what the client originally asked for.
|
||||
// Only return an AAAA record if the client asked for AAAA (or ANY).
|
||||
// For A queries, return an empty NOERROR — the client's resolver will
|
||||
// use the AAAA answer from its parallel AAAA query instead.
|
||||
let client_qtype = query
|
||||
.questions
|
||||
.first()
|
||||
.map(|q| q.qtype)
|
||||
.unwrap_or(QTYPE::TYPE(TYPE::AAAA));
|
||||
|
||||
match client_qtype {
|
||||
QTYPE::TYPE(TYPE::AAAA) | QTYPE::ANY => build_aaaa_response(&query, virtual_ip, ttl),
|
||||
// All other types (A, HTTPS, etc.): return NODATA — the name exists
|
||||
// but has no records of the requested type.
|
||||
_ => build_nodata(&query, ttl),
|
||||
}
|
||||
build_aaaa_response(&query, virtual_ip, ttl)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -416,17 +427,28 @@ mod tests {
|
||||
|
||||
/// Build a client-facing AAAA query.
|
||||
fn build_query(id: u16, qname: &str) -> Vec<u8> {
|
||||
build_query_of_type(id, qname, QTYPE::TYPE(TYPE::AAAA))
|
||||
}
|
||||
|
||||
/// Build a client-facing query of any type.
|
||||
fn build_query_of_type(id: u16, qname: &str, qtype: QTYPE) -> Vec<u8> {
|
||||
let mut packet = Packet::new_query(id);
|
||||
let question = Question::new(
|
||||
Name::new_unchecked(qname),
|
||||
QTYPE::TYPE(TYPE::AAAA),
|
||||
CLASS::IN.into(),
|
||||
false,
|
||||
);
|
||||
let question = Question::new(Name::new_unchecked(qname), qtype, CLASS::IN.into(), false);
|
||||
packet.questions.push(question);
|
||||
packet.build_bytes_vec_compressed().unwrap()
|
||||
}
|
||||
|
||||
/// Assert the response is NODATA: NOERROR with no answer records.
|
||||
fn assert_nodata(response: &[u8]) {
|
||||
let packet = Packet::parse(response).unwrap();
|
||||
assert_eq!(packet.rcode(), RCODE::NoError);
|
||||
assert!(
|
||||
packet.answers.is_empty(),
|
||||
"expected NODATA, got {} answer(s)",
|
||||
packet.answers.len()
|
||||
);
|
||||
}
|
||||
|
||||
/// Build an upstream NOERROR AAAA answer.
|
||||
fn build_answer(id: u16, qname: &str, addr: &str) -> Vec<u8> {
|
||||
let mut packet = Packet::new_reply(id);
|
||||
@@ -642,6 +664,115 @@ mod tests {
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_a_query_returns_nodata_and_mints_no_mapping() {
|
||||
let upstream_socket = UdpSocket::bind("[::1]:0").await.unwrap();
|
||||
let upstream = upstream_socket.local_addr().unwrap();
|
||||
let handle = spawn_upstream(upstream_socket, |id| {
|
||||
vec![build_answer(id, "test.fips", "fd00::1")]
|
||||
});
|
||||
|
||||
let pool = test_pool();
|
||||
let (event_tx, mut event_rx) = mpsc::channel(16);
|
||||
let response = handle_query(
|
||||
&build_query_of_type(0x1234, "test.fips", QTYPE::TYPE(TYPE::A)),
|
||||
upstream,
|
||||
TEST_TTL,
|
||||
&pool,
|
||||
&event_tx,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
handle.await.unwrap();
|
||||
|
||||
assert_nodata(&response);
|
||||
assert!(
|
||||
matches!(event_rx.try_recv(), Err(mpsc::error::TryRecvError::Empty)),
|
||||
"an A query minted a mapping, so any LAN host can take a pool \
|
||||
address per name with a query type it is never given one for"
|
||||
);
|
||||
assert!(
|
||||
pool.lock()
|
||||
.await
|
||||
.mapping_info(std::time::Instant::now())
|
||||
.is_empty(),
|
||||
"an A query left a mapping in the pool"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_a_query_refreshes_an_existing_mapping_without_creating_one() {
|
||||
let pool = test_pool();
|
||||
let (event_tx, mut event_rx) = mpsc::channel(16);
|
||||
|
||||
// An AAAA query mints the mapping.
|
||||
let upstream_socket = UdpSocket::bind("[::1]:0").await.unwrap();
|
||||
let upstream = upstream_socket.local_addr().unwrap();
|
||||
let handle = spawn_upstream(upstream_socket, |id| {
|
||||
vec![build_answer(id, "test.fips", "fd00::1")]
|
||||
});
|
||||
let response = handle_query(
|
||||
&build_query(0x1234, "test.fips"),
|
||||
upstream,
|
||||
TEST_TTL,
|
||||
&pool,
|
||||
&event_tx,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
handle.await.unwrap();
|
||||
let virtual_ip = assert_pool_answer(&response);
|
||||
assert!(matches!(
|
||||
event_rx.try_recv().unwrap(),
|
||||
PoolEvent::MappingCreated { .. }
|
||||
));
|
||||
|
||||
let before = {
|
||||
let guard = pool.lock().await;
|
||||
guard
|
||||
.lookup_virtual_ip(&virtual_ip)
|
||||
.unwrap()
|
||||
.last_referenced
|
||||
};
|
||||
|
||||
// An A query for the same name refreshes it and creates nothing. A
|
||||
// client that re-queries a mapped name with both types must not lose
|
||||
// half of its refresh: with no conntrack sessions, the DNS reference
|
||||
// is the only thing keeping the mapping alive.
|
||||
let upstream_socket = UdpSocket::bind("[::1]:0").await.unwrap();
|
||||
let upstream = upstream_socket.local_addr().unwrap();
|
||||
let handle = spawn_upstream(upstream_socket, |id| {
|
||||
vec![build_answer(id, "test.fips", "fd00::1")]
|
||||
});
|
||||
let response = handle_query(
|
||||
&build_query_of_type(0x1235, "test.fips", QTYPE::TYPE(TYPE::A)),
|
||||
upstream,
|
||||
TEST_TTL,
|
||||
&pool,
|
||||
&event_tx,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
handle.await.unwrap();
|
||||
|
||||
assert_nodata(&response);
|
||||
|
||||
let guard = pool.lock().await;
|
||||
let mapping = guard
|
||||
.lookup_virtual_ip(&virtual_ip)
|
||||
.expect("the A query removed or replaced the mapping");
|
||||
assert!(
|
||||
mapping.last_referenced > before,
|
||||
"the A query did not refresh the mapping's TTL clock"
|
||||
);
|
||||
drop(guard);
|
||||
|
||||
assert!(
|
||||
matches!(event_rx.try_recv(), Err(mpsc::error::TryRecvError::Empty)),
|
||||
"the A query sent a second MappingCreated"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_healthy_path_resolves() {
|
||||
let upstream_socket = UdpSocket::bind("[::1]:0").await.unwrap();
|
||||
|
||||
+330
-129
@@ -51,6 +51,30 @@ struct NatMapping {
|
||||
mesh_addr: Ipv6Addr,
|
||||
}
|
||||
|
||||
/// One object a NAT rebuild sends, named rather than built.
|
||||
///
|
||||
/// `rebuild_batches` decides what a rebuild sends and in what order;
|
||||
/// `send_batches` turns that decision into rustables objects and hands each
|
||||
/// batch to the kernel. The split is what lets a test see the delete and the
|
||||
/// recreate share one transaction without a netlink socket, which is the
|
||||
/// property that keeps the table in the packet path.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum NatOp {
|
||||
Table(MsgType),
|
||||
PreChain,
|
||||
PostChain,
|
||||
/// Masquerade for traffic leaving through `fips0`.
|
||||
FipsMasquerade,
|
||||
/// DNAT for the mapping with this virtual IP.
|
||||
Dnat(Ipv6Addr),
|
||||
/// SNAT for the mapping with this virtual IP.
|
||||
Snat(Ipv6Addr),
|
||||
/// DNAT for the port forward at this index in `port_forwards`.
|
||||
PortForward(usize),
|
||||
/// LAN-side masquerade, emitted once when any port forward exists.
|
||||
LanMasquerade,
|
||||
}
|
||||
|
||||
/// NAT rule manager using nftables via rustables netlink API.
|
||||
///
|
||||
/// Rebuilds the entire nftables table atomically on every change to
|
||||
@@ -71,15 +95,11 @@ pub struct NatManager {
|
||||
}
|
||||
|
||||
impl NatManager {
|
||||
/// Create the nftables table and NAT chains.
|
||||
/// Build the manager's state without touching netlink.
|
||||
///
|
||||
/// Installs a masquerade rule for traffic exiting via `fips0` so that
|
||||
/// LAN client source addresses are rewritten to the gateway's mesh
|
||||
/// address, allowing return traffic to route back through the mesh.
|
||||
///
|
||||
/// `lan_interface` is the gateway's LAN-facing interface name,
|
||||
/// needed by the port-forward LAN-side masquerade rule.
|
||||
pub fn new(lan_interface: String) -> Result<Self, NatError> {
|
||||
/// Everything `new` does except sending the first rebuild, so a test can
|
||||
/// exercise the batch builder with no socket and no privileges.
|
||||
fn with_state(lan_interface: String) -> Self {
|
||||
let table = Table::new(ProtocolFamily::Inet).with_name(TABLE_NAME);
|
||||
let pre_chain = Chain::new(&table)
|
||||
.with_name(PREROUTING_CHAIN)
|
||||
@@ -90,14 +110,26 @@ impl NatManager {
|
||||
.with_type(ChainType::Nat)
|
||||
.with_hook(Hook::new(HookClass::PostRouting, SRCNAT_PRIORITY));
|
||||
|
||||
let mgr = Self {
|
||||
Self {
|
||||
table,
|
||||
pre_chain,
|
||||
post_chain,
|
||||
lan_interface,
|
||||
mappings: HashMap::new(),
|
||||
port_forwards: Vec::new(),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// Create the nftables table and NAT chains.
|
||||
///
|
||||
/// Installs a masquerade rule for traffic exiting via `fips0` so that
|
||||
/// LAN client source addresses are rewritten to the gateway's mesh
|
||||
/// address, allowing return traffic to route back through the mesh.
|
||||
///
|
||||
/// `lan_interface` is the gateway's LAN-facing interface name,
|
||||
/// needed by the port-forward LAN-side masquerade rule.
|
||||
pub fn new(lan_interface: String) -> Result<Self, NatError> {
|
||||
let mgr = Self::with_state(lan_interface);
|
||||
mgr.rebuild()?;
|
||||
|
||||
info!("Created nftables table '{TABLE_NAME}' with NAT chains and fips0 masquerade");
|
||||
@@ -167,133 +199,302 @@ impl NatManager {
|
||||
self.mappings.len()
|
||||
}
|
||||
|
||||
/// Atomically rebuild the entire nftables table with all current
|
||||
/// rules. Deletes and recreates the table, chains, masquerade rule,
|
||||
/// and all per-mapping DNAT/SNAT rules in a single netlink batch.
|
||||
fn rebuild(&self) -> Result<(), NatError> {
|
||||
// Delete existing table in a separate batch — ignore ENOENT on
|
||||
// first call when the table doesn't exist yet.
|
||||
let mut del_batch = Batch::new();
|
||||
del_batch.add(&self.table, MsgType::Del);
|
||||
let _ = del_batch.send();
|
||||
/// The objects a rebuild sends, grouped into the batches that carry them.
|
||||
///
|
||||
/// One batch, always. The kernel applies a batch as a single transaction,
|
||||
/// so the table is deleted and recreated without ever leaving the packet
|
||||
/// path, and a batch the kernel rejects leaves the previous table in
|
||||
/// place. The leading `Add` is what makes the `Del` legal on a first run:
|
||||
/// rustables sends a table `Add` with `NLM_F_CREATE` and no `NLM_F_EXCL`,
|
||||
/// so it succeeds whether or not the table already exists and the `Del`
|
||||
/// that follows always has a target.
|
||||
fn rebuild_batches(&self) -> Vec<Vec<NatOp>> {
|
||||
let mut ops = vec![
|
||||
NatOp::Table(MsgType::Add),
|
||||
NatOp::Table(MsgType::Del),
|
||||
NatOp::Table(MsgType::Add),
|
||||
NatOp::PreChain,
|
||||
NatOp::PostChain,
|
||||
NatOp::FipsMasquerade,
|
||||
];
|
||||
|
||||
// Recreate table, chains, and all rules atomically.
|
||||
let mut batch = Batch::new();
|
||||
batch.add(&self.table, MsgType::Add);
|
||||
batch.add(&self.pre_chain, MsgType::Add);
|
||||
batch.add(&self.post_chain, MsgType::Add);
|
||||
|
||||
// Masquerade rule: rewrite source address for traffic exiting fips0.
|
||||
// Without this, LAN clients' source addresses (e.g. fd02::20) are
|
||||
// not routable on the mesh, so return traffic would be black-holed.
|
||||
let masq_rule = Rule::new(&self.post_chain)?
|
||||
.with_expr(Meta::new(MetaType::OifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, b"fips0\0".to_vec()))
|
||||
.with_expr(Masquerade::default());
|
||||
batch.add(&masq_rule, MsgType::Add);
|
||||
|
||||
// Per-mapping DNAT/SNAT rules.
|
||||
for mapping in self.mappings.values() {
|
||||
let dnat_rule = Rule::new(&self.pre_chain)?
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(
|
||||
HighLevelPayload::Network(NetworkHeaderField::IPv6(IPv6HeaderField::Daddr))
|
||||
.build(),
|
||||
)
|
||||
.with_expr(Cmp::new(CmpOp::Eq, mapping.virtual_ip.octets()))
|
||||
.with_expr(Immediate::new_data(
|
||||
mapping.mesh_addr.octets().to_vec(),
|
||||
Register::Reg1,
|
||||
))
|
||||
.with_expr(
|
||||
Nat::default()
|
||||
.with_nat_type(NatType::DNat)
|
||||
.with_family(ProtocolFamily::Ipv6)
|
||||
.with_ip_register(Register::Reg1),
|
||||
);
|
||||
batch.add(&dnat_rule, MsgType::Add);
|
||||
|
||||
let snat_rule = Rule::new(&self.post_chain)?
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(
|
||||
HighLevelPayload::Network(NetworkHeaderField::IPv6(IPv6HeaderField::Saddr))
|
||||
.build(),
|
||||
)
|
||||
.with_expr(Cmp::new(CmpOp::Eq, mapping.mesh_addr.octets()))
|
||||
.with_expr(Immediate::new_data(
|
||||
mapping.virtual_ip.octets().to_vec(),
|
||||
Register::Reg1,
|
||||
))
|
||||
.with_expr(
|
||||
Nat::default()
|
||||
.with_nat_type(NatType::SNat)
|
||||
.with_family(ProtocolFamily::Ipv6)
|
||||
.with_ip_register(Register::Reg1),
|
||||
);
|
||||
batch.add(&snat_rule, MsgType::Add);
|
||||
ops.push(NatOp::Dnat(mapping.virtual_ip));
|
||||
ops.push(NatOp::Snat(mapping.virtual_ip));
|
||||
}
|
||||
|
||||
// Inbound port-forward rules. Each forward is
|
||||
// one DNAT rule in prerouting keyed on (iif fips0, nfproto ipv6,
|
||||
// l4proto, th dport). When any forwards are configured, emit a
|
||||
// single LAN-side masquerade in postrouting so the LAN target
|
||||
// host sees the gateway's LAN address as source and replies
|
||||
// flow back through conntrack.
|
||||
for pf in &self.port_forwards {
|
||||
let l4proto: u8 = match pf.proto {
|
||||
Proto::Tcp => libc::IPPROTO_TCP as u8,
|
||||
Proto::Udp => libc::IPPROTO_UDP as u8,
|
||||
};
|
||||
let dport_field = match pf.proto {
|
||||
Proto::Tcp => TransportHeaderField::Tcp(TCPHeaderField::Dport),
|
||||
Proto::Udp => TransportHeaderField::Udp(UDPHeaderField::Dport),
|
||||
};
|
||||
let target_ip = *pf.target.ip();
|
||||
let target_port_be = pf.target.port().to_be_bytes();
|
||||
|
||||
let dnat_rule = Rule::new(&self.pre_chain)?
|
||||
.with_expr(Meta::new(MetaType::IifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, b"fips0\0".to_vec()))
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(Meta::new(MetaType::L4Proto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [l4proto]))
|
||||
.with_expr(HighLevelPayload::Transport(dport_field).build())
|
||||
.with_expr(Cmp::new(CmpOp::Eq, pf.listen_port.to_be_bytes().to_vec()))
|
||||
.with_expr(Immediate::new_data(
|
||||
target_ip.octets().to_vec(),
|
||||
Register::Reg1,
|
||||
))
|
||||
.with_expr(Immediate::new_data(target_port_be.to_vec(), Register::Reg2))
|
||||
.with_expr(
|
||||
Nat::default()
|
||||
.with_nat_type(NatType::DNat)
|
||||
.with_family(ProtocolFamily::Ipv6)
|
||||
.with_ip_register(Register::Reg1)
|
||||
.with_port_register(Register::Reg2),
|
||||
);
|
||||
batch.add(&dnat_rule, MsgType::Add);
|
||||
// Inbound port-forward rules. Each forward is one DNAT rule in
|
||||
// prerouting keyed on (iif fips0, nfproto ipv6, l4proto, th dport).
|
||||
// When any forwards are configured, emit a single LAN-side masquerade
|
||||
// in postrouting so the LAN target host sees the gateway's LAN address
|
||||
// as source and replies flow back through conntrack.
|
||||
for index in 0..self.port_forwards.len() {
|
||||
ops.push(NatOp::PortForward(index));
|
||||
}
|
||||
|
||||
if !self.port_forwards.is_empty() {
|
||||
let mut lan_iface = self.lan_interface.clone().into_bytes();
|
||||
lan_iface.push(0);
|
||||
let lan_masq = Rule::new(&self.post_chain)?
|
||||
.with_expr(Meta::new(MetaType::IifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, b"fips0\0".to_vec()))
|
||||
.with_expr(Meta::new(MetaType::OifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, lan_iface))
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(Masquerade::default());
|
||||
batch.add(&lan_masq, MsgType::Add);
|
||||
ops.push(NatOp::LanMasquerade);
|
||||
}
|
||||
|
||||
batch
|
||||
.send()
|
||||
.map_err(|e| NatError::Nftables(e.to_string()))?;
|
||||
vec![ops]
|
||||
}
|
||||
|
||||
/// Build each op into its rustables object and send the batches in order.
|
||||
fn send_batches(&self, batches: &[Vec<NatOp>]) -> Result<(), NatError> {
|
||||
for ops in batches {
|
||||
let mut batch = Batch::new();
|
||||
for op in ops {
|
||||
match *op {
|
||||
NatOp::Table(msg_type) => batch.add(&self.table, msg_type),
|
||||
NatOp::PreChain => batch.add(&self.pre_chain, MsgType::Add),
|
||||
NatOp::PostChain => batch.add(&self.post_chain, MsgType::Add),
|
||||
NatOp::FipsMasquerade => {
|
||||
// Rewrite the source address of traffic leaving fips0.
|
||||
// Without this, LAN clients' source addresses (e.g.
|
||||
// fd02::20) are not routable on the mesh, so return
|
||||
// traffic would be black-holed.
|
||||
let rule = Rule::new(&self.post_chain)?
|
||||
.with_expr(Meta::new(MetaType::OifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, b"fips0\0".to_vec()))
|
||||
.with_expr(Masquerade::default());
|
||||
batch.add(&rule, MsgType::Add);
|
||||
}
|
||||
NatOp::Dnat(virtual_ip) => {
|
||||
let mapping = self.mapping(virtual_ip)?;
|
||||
let rule = Rule::new(&self.pre_chain)?
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(
|
||||
HighLevelPayload::Network(NetworkHeaderField::IPv6(
|
||||
IPv6HeaderField::Daddr,
|
||||
))
|
||||
.build(),
|
||||
)
|
||||
.with_expr(Cmp::new(CmpOp::Eq, mapping.virtual_ip.octets()))
|
||||
.with_expr(Immediate::new_data(
|
||||
mapping.mesh_addr.octets().to_vec(),
|
||||
Register::Reg1,
|
||||
))
|
||||
.with_expr(
|
||||
Nat::default()
|
||||
.with_nat_type(NatType::DNat)
|
||||
.with_family(ProtocolFamily::Ipv6)
|
||||
.with_ip_register(Register::Reg1),
|
||||
);
|
||||
batch.add(&rule, MsgType::Add);
|
||||
}
|
||||
NatOp::Snat(virtual_ip) => {
|
||||
let mapping = self.mapping(virtual_ip)?;
|
||||
let rule = Rule::new(&self.post_chain)?
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(
|
||||
HighLevelPayload::Network(NetworkHeaderField::IPv6(
|
||||
IPv6HeaderField::Saddr,
|
||||
))
|
||||
.build(),
|
||||
)
|
||||
.with_expr(Cmp::new(CmpOp::Eq, mapping.mesh_addr.octets()))
|
||||
.with_expr(Immediate::new_data(
|
||||
mapping.virtual_ip.octets().to_vec(),
|
||||
Register::Reg1,
|
||||
))
|
||||
.with_expr(
|
||||
Nat::default()
|
||||
.with_nat_type(NatType::SNat)
|
||||
.with_family(ProtocolFamily::Ipv6)
|
||||
.with_ip_register(Register::Reg1),
|
||||
);
|
||||
batch.add(&rule, MsgType::Add);
|
||||
}
|
||||
NatOp::PortForward(index) => {
|
||||
let pf = self.port_forwards.get(index).expect(
|
||||
"rebuild_batches only emits indices it read from port_forwards",
|
||||
);
|
||||
let l4proto: u8 = match pf.proto {
|
||||
Proto::Tcp => libc::IPPROTO_TCP as u8,
|
||||
Proto::Udp => libc::IPPROTO_UDP as u8,
|
||||
};
|
||||
let dport_field = match pf.proto {
|
||||
Proto::Tcp => TransportHeaderField::Tcp(TCPHeaderField::Dport),
|
||||
Proto::Udp => TransportHeaderField::Udp(UDPHeaderField::Dport),
|
||||
};
|
||||
let target_ip = *pf.target.ip();
|
||||
let target_port_be = pf.target.port().to_be_bytes();
|
||||
|
||||
let rule = Rule::new(&self.pre_chain)?
|
||||
.with_expr(Meta::new(MetaType::IifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, b"fips0\0".to_vec()))
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(Meta::new(MetaType::L4Proto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [l4proto]))
|
||||
.with_expr(HighLevelPayload::Transport(dport_field).build())
|
||||
.with_expr(Cmp::new(CmpOp::Eq, pf.listen_port.to_be_bytes().to_vec()))
|
||||
.with_expr(Immediate::new_data(
|
||||
target_ip.octets().to_vec(),
|
||||
Register::Reg1,
|
||||
))
|
||||
.with_expr(Immediate::new_data(target_port_be.to_vec(), Register::Reg2))
|
||||
.with_expr(
|
||||
Nat::default()
|
||||
.with_nat_type(NatType::DNat)
|
||||
.with_family(ProtocolFamily::Ipv6)
|
||||
.with_ip_register(Register::Reg1)
|
||||
.with_port_register(Register::Reg2),
|
||||
);
|
||||
batch.add(&rule, MsgType::Add);
|
||||
}
|
||||
NatOp::LanMasquerade => {
|
||||
let mut lan_iface = self.lan_interface.clone().into_bytes();
|
||||
lan_iface.push(0);
|
||||
let rule = Rule::new(&self.post_chain)?
|
||||
.with_expr(Meta::new(MetaType::IifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, b"fips0\0".to_vec()))
|
||||
.with_expr(Meta::new(MetaType::OifName))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, lan_iface))
|
||||
.with_expr(Meta::new(MetaType::NfProto))
|
||||
.with_expr(Cmp::new(CmpOp::Eq, [libc::NFPROTO_IPV6 as u8]))
|
||||
.with_expr(Masquerade::default());
|
||||
batch.add(&rule, MsgType::Add);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
batch
|
||||
.send()
|
||||
.map_err(|e| NatError::Nftables(e.to_string()))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The mapping an op names, or the error a caller can report.
|
||||
fn mapping(&self, virtual_ip: Ipv6Addr) -> Result<&NatMapping, NatError> {
|
||||
self.mappings
|
||||
.get(&virtual_ip)
|
||||
.ok_or(NatError::RuleNotFound(virtual_ip))
|
||||
}
|
||||
|
||||
/// Rebuild the entire nftables table with all current rules, in one
|
||||
/// netlink transaction.
|
||||
fn rebuild(&self) -> Result<(), NatError> {
|
||||
self.send_batches(&self.rebuild_batches())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::net::SocketAddrV6;
|
||||
|
||||
fn vip(last: u16) -> Ipv6Addr {
|
||||
Ipv6Addr::new(0xfd01, 0, 0, 0, 0, 0, 0, last)
|
||||
}
|
||||
|
||||
fn mesh(last: u16) -> Ipv6Addr {
|
||||
Ipv6Addr::new(0xfd02, 0, 0, 0, 0, 0, 0, last)
|
||||
}
|
||||
|
||||
/// A manager holding `count` mappings and no netlink socket.
|
||||
fn manager_with_mappings(count: u16) -> NatManager {
|
||||
let mut mgr = NatManager::with_state("br-lan".to_string());
|
||||
for i in 1..=count {
|
||||
mgr.mappings.insert(
|
||||
vip(i),
|
||||
NatMapping {
|
||||
virtual_ip: vip(i),
|
||||
mesh_addr: mesh(i),
|
||||
},
|
||||
);
|
||||
}
|
||||
mgr
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_deletes_and_recreates_the_table_inside_one_batch() {
|
||||
let batches = manager_with_mappings(3).rebuild_batches();
|
||||
|
||||
assert_eq!(
|
||||
batches.len(),
|
||||
1,
|
||||
"a rebuild that sends the delete in a batch of its own leaves the \
|
||||
fips_gateway table absent between the two sends, so the gateway \
|
||||
has no NAT at all in that window: {batches:?}"
|
||||
);
|
||||
assert_eq!(
|
||||
batches[0][..3],
|
||||
[
|
||||
NatOp::Table(MsgType::Add),
|
||||
NatOp::Table(MsgType::Del),
|
||||
NatOp::Table(MsgType::Add),
|
||||
],
|
||||
"the delete needs a preceding add so it always has a target, and a \
|
||||
following add to recreate the table inside the same transaction"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_deletes_the_table_exactly_once_and_before_every_rule() {
|
||||
let batches = manager_with_mappings(2).rebuild_batches();
|
||||
let ops = &batches[0];
|
||||
|
||||
let deletes: Vec<usize> = ops
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, op)| matches!(op, NatOp::Table(MsgType::Del)))
|
||||
.map(|(i, _)| i)
|
||||
.collect();
|
||||
assert_eq!(deletes, vec![1], "the table is deleted once, at index 1");
|
||||
|
||||
// Everything that lives in the table has to be added after the delete
|
||||
// and the recreate, or the delete would take it back out again.
|
||||
for (index, op) in ops.iter().enumerate() {
|
||||
if matches!(op, NatOp::Table(_)) {
|
||||
continue;
|
||||
}
|
||||
assert!(
|
||||
index > 2,
|
||||
"{op:?} at index {index} would be removed by the table delete"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_emits_a_dnat_and_an_snat_for_every_mapping() {
|
||||
let ops = manager_with_mappings(3).rebuild_batches().remove(0);
|
||||
|
||||
for i in 1..=3u16 {
|
||||
assert!(ops.contains(&NatOp::Dnat(vip(i))), "no DNAT for {}", vip(i));
|
||||
assert!(ops.contains(&NatOp::Snat(vip(i))), "no SNAT for {}", vip(i));
|
||||
}
|
||||
assert!(ops.contains(&NatOp::FipsMasquerade));
|
||||
assert!(!ops.contains(&NatOp::LanMasquerade), "no port forwards");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuild_emits_the_lan_masquerade_once_when_port_forwards_exist() {
|
||||
let mut mgr = manager_with_mappings(1);
|
||||
mgr.port_forwards = vec![
|
||||
PortForward {
|
||||
proto: Proto::Tcp,
|
||||
listen_port: 8080,
|
||||
target: SocketAddrV6::new(Ipv6Addr::LOCALHOST, 80, 0, 0),
|
||||
},
|
||||
PortForward {
|
||||
proto: Proto::Udp,
|
||||
listen_port: 5353,
|
||||
target: SocketAddrV6::new(Ipv6Addr::LOCALHOST, 53, 0, 0),
|
||||
},
|
||||
];
|
||||
|
||||
let ops = mgr.rebuild_batches().remove(0);
|
||||
|
||||
assert!(ops.contains(&NatOp::PortForward(0)));
|
||||
assert!(ops.contains(&NatOp::PortForward(1)));
|
||||
assert_eq!(
|
||||
ops.iter()
|
||||
.filter(|op| matches!(op, NatOp::LanMasquerade))
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
+264
-52
@@ -5,7 +5,7 @@
|
||||
//! with conntrack to determine active sessions.
|
||||
|
||||
use crate::NodeAddr;
|
||||
use std::collections::{HashMap, VecDeque};
|
||||
use std::collections::{HashMap, HashSet, VecDeque};
|
||||
use std::net::Ipv6Addr;
|
||||
use std::time::Instant;
|
||||
use tracing::{debug, info};
|
||||
@@ -93,25 +93,127 @@ pub struct MappingInfo {
|
||||
pub last_ref_secs: u64,
|
||||
}
|
||||
|
||||
/// Trait for querying conntrack session counts.
|
||||
/// Path the conntrack table is read from.
|
||||
const CONNTRACK_PROC_PATH: &str = "/proc/net/nf_conntrack";
|
||||
|
||||
/// Active conntrack sessions counted by destination address.
|
||||
///
|
||||
/// Taken once per tick, so the pool does a map lookup per mapping instead of
|
||||
/// reading and scanning the whole conntrack table per mapping under its lock.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct ConntrackSnapshot {
|
||||
sessions: HashMap<Ipv6Addr, u32>,
|
||||
}
|
||||
|
||||
impl ConntrackSnapshot {
|
||||
/// Build a snapshot from counts already keyed by destination address.
|
||||
pub fn from_counts(sessions: HashMap<Ipv6Addr, u32>) -> Self {
|
||||
Self { sessions }
|
||||
}
|
||||
|
||||
/// Sessions whose destination is `virtual_ip`, or zero if there are none.
|
||||
pub fn sessions_for(&self, virtual_ip: Ipv6Addr) -> u32 {
|
||||
self.sessions.get(&virtual_ip).copied().unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Number of distinct destination addresses the snapshot saw.
|
||||
pub fn len(&self) -> usize {
|
||||
self.sessions.len()
|
||||
}
|
||||
|
||||
/// Whether the snapshot saw no sessions at all.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.sessions.is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
/// Trait for taking a conntrack session snapshot.
|
||||
pub trait ConntrackQuerier: Send + Sync {
|
||||
/// Returns the number of active conntrack entries whose original
|
||||
/// destination matches the given virtual IP.
|
||||
fn active_sessions(&self, virtual_ip: Ipv6Addr) -> Result<u32, std::io::Error>;
|
||||
/// Read the conntrack table once and count sessions by destination.
|
||||
fn snapshot(&self) -> Result<ConntrackSnapshot, std::io::Error>;
|
||||
}
|
||||
|
||||
/// Conntrack querier that parses /proc/net/nf_conntrack.
|
||||
pub struct ProcConntrack;
|
||||
|
||||
impl ConntrackQuerier for ProcConntrack {
|
||||
fn active_sessions(&self, virtual_ip: Ipv6Addr) -> Result<u32, std::io::Error> {
|
||||
let content = std::fs::read_to_string("/proc/net/nf_conntrack")?;
|
||||
let target = virtual_ip.to_string();
|
||||
let count = content
|
||||
.lines()
|
||||
.filter(|line| line.contains(&format!("dst={target}")))
|
||||
.count();
|
||||
Ok(count as u32)
|
||||
fn snapshot(&self) -> Result<ConntrackSnapshot, std::io::Error> {
|
||||
let content = std::fs::read_to_string(CONNTRACK_PROC_PATH)?;
|
||||
Ok(ConntrackSnapshot::from_counts(parse_conntrack(&content)))
|
||||
}
|
||||
}
|
||||
|
||||
/// Count conntrack lines by the destination addresses they name.
|
||||
///
|
||||
/// Every `dst=` value is parsed as an address and compared as an address. The
|
||||
/// kernel prints tuples as `src=%pI6 dst=%pI6`, the full uncompressed form with
|
||||
/// leading zeros, so a session to `fd01::1` is written
|
||||
/// `dst=fd01:0000:0000:0000:0000:0000:0000:0001`; the previous code searched
|
||||
/// each line for the address's compressed `Display` form, which cannot occur in
|
||||
/// a fixed-width field, so it counted nothing on any kernel.
|
||||
///
|
||||
/// A conntrack line carries the original and the reply tuple, each with its own
|
||||
/// `dst=`, and the line is counted once per distinct address among them. That
|
||||
/// keeps the meaning the count had before, which was "this line mentions the
|
||||
/// address". A value that does not parse as an IPv6 address is skipped, which
|
||||
/// is how IPv4 lines and any future field are ignored.
|
||||
fn parse_conntrack(content: &str) -> HashMap<Ipv6Addr, u32> {
|
||||
let mut counts: HashMap<Ipv6Addr, u32> = HashMap::new();
|
||||
let mut seen: HashSet<Ipv6Addr> = HashSet::new();
|
||||
|
||||
for line in content.lines() {
|
||||
seen.clear();
|
||||
for token in line.split_whitespace() {
|
||||
let Some(value) = token.strip_prefix("dst=") else {
|
||||
continue;
|
||||
};
|
||||
let Ok(addr) = value.parse::<Ipv6Addr>() else {
|
||||
continue;
|
||||
};
|
||||
seen.insert(addr);
|
||||
}
|
||||
for addr in &seen {
|
||||
*counts.entry(*addr).or_insert(0) += 1;
|
||||
}
|
||||
}
|
||||
|
||||
counts
|
||||
}
|
||||
|
||||
/// Whether a conntrack read outcome is new or a repeat of the last one.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum ReadReport {
|
||||
/// The outcome differs from the previous read, or is the first.
|
||||
Changed,
|
||||
/// The same outcome as the previous read.
|
||||
Repeated,
|
||||
}
|
||||
|
||||
/// Remembers the last conntrack read outcome.
|
||||
///
|
||||
/// A kernel built without `CONFIG_NF_CONNTRACK_PROCFS` has no
|
||||
/// `/proc/net/nf_conntrack` at all, so every read fails the same way and a
|
||||
/// per-tick warning would repeat for the life of the process. Warning on a
|
||||
/// change of outcome still separates "the source is unreadable" from "there
|
||||
/// are no sessions", which the pool could not distinguish before, without
|
||||
/// filling the log.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct ConntrackReadLog {
|
||||
last: Option<Option<std::io::ErrorKind>>,
|
||||
}
|
||||
|
||||
impl ConntrackReadLog {
|
||||
/// Record a read outcome and say whether it is new.
|
||||
///
|
||||
/// `None` is a successful read; `Some(kind)` is a failure of that kind.
|
||||
pub fn observe(&mut self, outcome: Option<std::io::ErrorKind>) -> ReadReport {
|
||||
let report = if self.last == Some(outcome) {
|
||||
ReadReport::Repeated
|
||||
} else {
|
||||
ReadReport::Changed
|
||||
};
|
||||
self.last = Some(outcome);
|
||||
report
|
||||
}
|
||||
}
|
||||
|
||||
@@ -168,6 +270,21 @@ impl VirtualIpPool {
|
||||
})
|
||||
}
|
||||
|
||||
/// Refresh an existing mapping's TTL clock, never creating one.
|
||||
///
|
||||
/// Returns whether a mapping for `node_addr` existed. A query the gateway
|
||||
/// answers without an address still says the client is using the name, so
|
||||
/// it must keep the mapping alive without minting one.
|
||||
pub fn refresh_if_present(&mut self, node_addr: NodeAddr) -> bool {
|
||||
match self.mappings.get_mut(&node_addr) {
|
||||
Some(mapping) => {
|
||||
mapping.last_referenced = Instant::now();
|
||||
true
|
||||
}
|
||||
None => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Allocate a virtual IP for the given node. Idempotent: returns
|
||||
/// existing mapping if one exists.
|
||||
pub fn allocate(
|
||||
@@ -176,9 +293,10 @@ impl VirtualIpPool {
|
||||
mesh_addr: Ipv6Addr,
|
||||
dns_name: &str,
|
||||
) -> Result<(Ipv6Addr, bool), PoolError> {
|
||||
// Idempotent: return existing mapping
|
||||
if let Some(mapping) = self.mappings.get_mut(&node_addr) {
|
||||
mapping.last_referenced = Instant::now();
|
||||
// Idempotent: return existing mapping, refreshed.
|
||||
if self.refresh_if_present(node_addr)
|
||||
&& let Some(mapping) = self.mappings.get(&node_addr)
|
||||
{
|
||||
return Ok((mapping.virtual_ip, false));
|
||||
}
|
||||
|
||||
@@ -215,15 +333,16 @@ impl VirtualIpPool {
|
||||
|
||||
/// Periodic tick — drives state transitions. Returns events for
|
||||
/// the NAT and network modules.
|
||||
pub fn tick(&mut self, now: Instant, conntrack: &dyn ConntrackQuerier) -> Vec<PoolEvent> {
|
||||
pub fn tick(&mut self, now: Instant, conntrack: &ConntrackSnapshot) -> Vec<PoolEvent> {
|
||||
let mut events = Vec::new();
|
||||
let mut to_free = Vec::new();
|
||||
let ttl = std::time::Duration::from_secs(self.ttl_secs);
|
||||
let grace = std::time::Duration::from_secs(self.grace_secs);
|
||||
|
||||
for (node_addr, mapping) in &mut self.mappings {
|
||||
// Query conntrack for active sessions
|
||||
let sessions = conntrack.active_sessions(mapping.virtual_ip).unwrap_or(0);
|
||||
// One map lookup: the conntrack table was read once, before the
|
||||
// pool lock was taken.
|
||||
let sessions = conntrack.sessions_for(mapping.virtual_ip);
|
||||
mapping.session_count = sessions;
|
||||
|
||||
// Live data-plane traffic pins the mapping: refresh the TTL
|
||||
@@ -372,26 +491,24 @@ fn parse_ipv6_cidr(cidr: &str) -> Result<(Ipv6Addr, u32), PoolError> {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Mock conntrack that returns a configurable session count.
|
||||
struct MockConntrack {
|
||||
/// Session counts a test sets directly, handed to `tick` as the snapshot
|
||||
/// the tick task would have read from conntrack.
|
||||
#[derive(Default)]
|
||||
struct Sessions {
|
||||
counts: HashMap<Ipv6Addr, u32>,
|
||||
}
|
||||
|
||||
impl MockConntrack {
|
||||
impl Sessions {
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
counts: HashMap::new(),
|
||||
}
|
||||
Self::default()
|
||||
}
|
||||
|
||||
fn set(&mut self, addr: Ipv6Addr, count: u32) {
|
||||
self.counts.insert(addr, count);
|
||||
}
|
||||
}
|
||||
|
||||
impl ConntrackQuerier for MockConntrack {
|
||||
fn active_sessions(&self, virtual_ip: Ipv6Addr) -> Result<u32, std::io::Error> {
|
||||
Ok(*self.counts.get(&virtual_ip).unwrap_or(&0))
|
||||
fn snapshot(&self) -> ConntrackSnapshot {
|
||||
ConntrackSnapshot::from_counts(self.counts.clone())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -475,7 +592,7 @@ mod tests {
|
||||
#[test]
|
||||
fn test_mapping_lifecycle_allocated_to_free() {
|
||||
let mut pool = VirtualIpPool::new("fd01::/120", 1, 1).unwrap();
|
||||
let ct = MockConntrack::new();
|
||||
let ct = Sessions::new();
|
||||
let node = make_node_addr(1);
|
||||
let mesh = make_mesh_addr(1);
|
||||
|
||||
@@ -483,13 +600,13 @@ mod tests {
|
||||
|
||||
// Tick before TTL — no change
|
||||
let now = Instant::now();
|
||||
let events = pool.tick(now, &ct);
|
||||
let events = pool.tick(now, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings.len(), 1);
|
||||
|
||||
// Tick after TTL with no sessions — enters draining
|
||||
let later = now + std::time::Duration::from_secs(2);
|
||||
let events = pool.tick(later, &ct);
|
||||
let events = pool.tick(later, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings.len(), 1);
|
||||
assert_eq!(
|
||||
@@ -499,7 +616,7 @@ mod tests {
|
||||
|
||||
// Tick after grace period — freed
|
||||
let after_grace = later + std::time::Duration::from_secs(2);
|
||||
let events = pool.tick(after_grace, &ct);
|
||||
let events = pool.tick(after_grace, &ct.snapshot());
|
||||
assert_eq!(events.len(), 1);
|
||||
assert!(matches!(events[0], PoolEvent::MappingRemoved { .. }));
|
||||
assert_eq!(pool.mappings.len(), 0);
|
||||
@@ -509,7 +626,7 @@ mod tests {
|
||||
#[test]
|
||||
fn test_mapping_lifecycle_active_draining_free() {
|
||||
let mut pool = VirtualIpPool::new("fd01::/120", 1, 1).unwrap();
|
||||
let mut ct = MockConntrack::new();
|
||||
let mut ct = Sessions::new();
|
||||
let node = make_node_addr(1);
|
||||
let mesh = make_mesh_addr(1);
|
||||
|
||||
@@ -518,25 +635,25 @@ mod tests {
|
||||
// Simulate active sessions
|
||||
ct.set(vip, 3);
|
||||
let now = Instant::now();
|
||||
let events = pool.tick(now, &ct);
|
||||
let events = pool.tick(now, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Active);
|
||||
|
||||
// TTL expires after sessions drop to 0 → Draining
|
||||
let later = now + std::time::Duration::from_secs(2);
|
||||
ct.set(vip, 0);
|
||||
let events = pool.tick(later, &ct);
|
||||
let events = pool.tick(later, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Draining);
|
||||
|
||||
// Still draining, grace period not elapsed
|
||||
let events = pool.tick(later, &ct);
|
||||
let events = pool.tick(later, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Draining);
|
||||
|
||||
// Grace period elapsed → Free
|
||||
let much_later = later + std::time::Duration::from_secs(2);
|
||||
let events = pool.tick(much_later, &ct);
|
||||
let events = pool.tick(much_later, &ct.snapshot());
|
||||
assert_eq!(events.len(), 1);
|
||||
assert!(matches!(events[0], PoolEvent::MappingRemoved { .. }));
|
||||
assert_eq!(pool.mappings.len(), 0);
|
||||
@@ -548,7 +665,7 @@ mod tests {
|
||||
// spanning well past the TTL must never be reclaimed and must
|
||||
// stay Active: live traffic refreshes last_referenced each tick.
|
||||
let mut pool = VirtualIpPool::new("fd01::/120", 1, 1).unwrap();
|
||||
let mut ct = MockConntrack::new();
|
||||
let mut ct = Sessions::new();
|
||||
let node = make_node_addr(1);
|
||||
let mesh = make_mesh_addr(1);
|
||||
|
||||
@@ -557,14 +674,14 @@ mod tests {
|
||||
|
||||
let mut t = Instant::now();
|
||||
// First tick activates the mapping.
|
||||
let events = pool.tick(t, &ct);
|
||||
let events = pool.tick(t, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Active);
|
||||
|
||||
// Advance many TTL-spans with continuous traffic.
|
||||
for _ in 0..10 {
|
||||
t += std::time::Duration::from_secs(5); // 5x the 1s TTL
|
||||
let events = pool.tick(t, &ct);
|
||||
let events = pool.tick(t, &ct.snapshot());
|
||||
assert!(events.is_empty(), "mapping must not be reclaimed");
|
||||
assert_eq!(
|
||||
pool.mappings[&node].state,
|
||||
@@ -580,7 +697,7 @@ mod tests {
|
||||
// Active -> drains when sessions hit 0 -> regains sessions before
|
||||
// grace elapses -> recovers to Active and is not freed.
|
||||
let mut pool = VirtualIpPool::new("fd01::/120", 1, 5).unwrap();
|
||||
let mut ct = MockConntrack::new();
|
||||
let mut ct = Sessions::new();
|
||||
let node = make_node_addr(1);
|
||||
let mesh = make_mesh_addr(1);
|
||||
|
||||
@@ -589,21 +706,21 @@ mod tests {
|
||||
// Activate with traffic.
|
||||
ct.set(vip, 1);
|
||||
let now = Instant::now();
|
||||
let events = pool.tick(now, &ct);
|
||||
let events = pool.tick(now, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Active);
|
||||
|
||||
// TTL passes with sessions dropping to 0 -> Draining.
|
||||
let drained = now + std::time::Duration::from_secs(2);
|
||||
ct.set(vip, 0);
|
||||
let events = pool.tick(drained, &ct);
|
||||
let events = pool.tick(drained, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Draining);
|
||||
|
||||
// Traffic resumes before grace (5s) elapses -> recover to Active.
|
||||
let resumed = drained + std::time::Duration::from_secs(2);
|
||||
ct.set(vip, 3);
|
||||
let events = pool.tick(resumed, &ct);
|
||||
let events = pool.tick(resumed, &ct.snapshot());
|
||||
assert!(events.is_empty());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Active);
|
||||
assert!(pool.mappings[&node].drain_start.is_none());
|
||||
@@ -616,7 +733,7 @@ mod tests {
|
||||
// fresh drain_start so the full grace window is honored again,
|
||||
// not reclaimed immediately off a stale drain_start.
|
||||
let mut pool = VirtualIpPool::new("fd01::/120", 1, 5).unwrap();
|
||||
let mut ct = MockConntrack::new();
|
||||
let mut ct = Sessions::new();
|
||||
let node = make_node_addr(1);
|
||||
let mesh = make_mesh_addr(1);
|
||||
|
||||
@@ -625,36 +742,36 @@ mod tests {
|
||||
// Activate.
|
||||
ct.set(vip, 1);
|
||||
let now = Instant::now();
|
||||
pool.tick(now, &ct);
|
||||
pool.tick(now, &ct.snapshot());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Active);
|
||||
|
||||
// First drain.
|
||||
let first_drain = now + std::time::Duration::from_secs(2);
|
||||
ct.set(vip, 0);
|
||||
pool.tick(first_drain, &ct);
|
||||
pool.tick(first_drain, &ct.snapshot());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Draining);
|
||||
|
||||
// Recover.
|
||||
let recover = first_drain + std::time::Duration::from_secs(2);
|
||||
ct.set(vip, 2);
|
||||
pool.tick(recover, &ct);
|
||||
pool.tick(recover, &ct.snapshot());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Active);
|
||||
|
||||
// Second drain begins; drain_start must be re-stamped fresh.
|
||||
let second_drain = recover + std::time::Duration::from_secs(2);
|
||||
ct.set(vip, 0);
|
||||
pool.tick(second_drain, &ct);
|
||||
pool.tick(second_drain, &ct.snapshot());
|
||||
assert_eq!(pool.mappings[&node].state, MappingState::Draining);
|
||||
|
||||
// Just before the fresh grace window expires (5s): not reclaimed.
|
||||
let before_grace = second_drain + std::time::Duration::from_secs(4);
|
||||
let events = pool.tick(before_grace, &ct);
|
||||
let events = pool.tick(before_grace, &ct.snapshot());
|
||||
assert!(events.is_empty(), "fresh grace window must be honored");
|
||||
assert_eq!(pool.mappings.len(), 1);
|
||||
|
||||
// After the fresh grace window: reclaimed.
|
||||
let after_grace = second_drain + std::time::Duration::from_secs(6);
|
||||
let events = pool.tick(after_grace, &ct);
|
||||
let events = pool.tick(after_grace, &ct.snapshot());
|
||||
assert_eq!(events.len(), 1);
|
||||
assert!(matches!(events[0], PoolEvent::MappingRemoved { .. }));
|
||||
assert_eq!(pool.mappings.len(), 0);
|
||||
@@ -696,4 +813,99 @@ mod tests {
|
||||
let pool = VirtualIpPool::new("fd01::/96", 60, 60).unwrap();
|
||||
assert_eq!(pool.total, 65535); // 2^16 - 1 (skip addr 0)
|
||||
}
|
||||
|
||||
/// A conntrack line in the form the kernel prints.
|
||||
///
|
||||
/// Built from the kernel's own format string, not captured from a running
|
||||
/// kernel: `net/netfilter/nf_conntrack_standalone.c` prints each tuple with
|
||||
/// `"src=%pI6 dst=%pI6 "`, and `%pI6` is the full uncompressed form with
|
||||
/// leading zeros (`Documentation/core-api/printk-formats.rst`). Both were
|
||||
/// read at v6.8. The host this was written on has no
|
||||
/// `/proc/net/nf_conntrack` to capture from, because its kernel is built
|
||||
/// without `CONFIG_NF_CONNTRACK_PROCFS`; OpenWrt's generic kernel config
|
||||
/// sets it, which is the kernel this parser exists for.
|
||||
const KERNEL_LINE: &str = "ipv6 10 tcp 6 431999 ESTABLISHED \
|
||||
src=fd02:0000:0000:0000:0000:0000:0000:0020 \
|
||||
dst=fd01:0000:0000:0000:0000:0000:0000:0001 sport=45678 dport=8000 \
|
||||
src=fd01:0000:0000:0000:0000:0000:0000:0001 \
|
||||
dst=fd02:0000:0000:0000:0000:0000:0000:0020 sport=8000 dport=45678 \
|
||||
[ASSURED] mark=0 use=1";
|
||||
|
||||
#[test]
|
||||
fn conntrack_parse_counts_a_kernel_format_line_for_its_virtual_ip() {
|
||||
let counts = parse_conntrack(KERNEL_LINE);
|
||||
let virtual_ip: Ipv6Addr = "fd01::1".parse().unwrap();
|
||||
|
||||
assert_eq!(
|
||||
counts.get(&virtual_ip).copied().unwrap_or(0),
|
||||
1,
|
||||
"the kernel writes the uncompressed form, so matching on the \
|
||||
address's compressed Display form counts nothing"
|
||||
);
|
||||
|
||||
// Healthy path: a different address in the same pool is not counted.
|
||||
let other: Ipv6Addr = "fd01::10".parse().unwrap();
|
||||
assert_eq!(counts.get(&other).copied().unwrap_or(0), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conntrack_parse_counts_a_line_once_however_many_tuples_name_the_address() {
|
||||
// A hairpin flow: the address is the destination of both tuples.
|
||||
let line = "ipv6 10 udp 17 29 \
|
||||
src=fd01:0000:0000:0000:0000:0000:0000:0001 \
|
||||
dst=fd01:0000:0000:0000:0000:0000:0000:0001 sport=1 dport=2 \
|
||||
src=fd01:0000:0000:0000:0000:0000:0000:0001 \
|
||||
dst=fd01:0000:0000:0000:0000:0000:0000:0001 sport=2 dport=1 \
|
||||
mark=0 use=1";
|
||||
let counts = parse_conntrack(line);
|
||||
let virtual_ip: Ipv6Addr = "fd01::1".parse().unwrap();
|
||||
|
||||
assert_eq!(counts.get(&virtual_ip).copied().unwrap_or(0), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conntrack_parse_counts_each_line_that_names_the_address() {
|
||||
let content = format!("{KERNEL_LINE}\n{KERNEL_LINE}\n");
|
||||
let counts = parse_conntrack(&content);
|
||||
let virtual_ip: Ipv6Addr = "fd01::1".parse().unwrap();
|
||||
|
||||
assert_eq!(counts.get(&virtual_ip).copied().unwrap_or(0), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conntrack_parse_skips_a_value_that_is_not_an_ipv6_address() {
|
||||
let content = "ipv4 2 tcp 6 431999 ESTABLISHED src=192.0.2.1 \
|
||||
dst=192.0.2.2 sport=1 dport=2 mark=0 use=1\n";
|
||||
|
||||
assert!(parse_conntrack(content).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conntrack_snapshot_reads_zero_for_an_address_it_did_not_see() {
|
||||
let snapshot = ConntrackSnapshot::from_counts(parse_conntrack(KERNEL_LINE));
|
||||
|
||||
assert_eq!(snapshot.sessions_for("fd01::1".parse().unwrap()), 1);
|
||||
assert_eq!(snapshot.sessions_for("fd01::99".parse().unwrap()), 0);
|
||||
assert!(ConntrackSnapshot::default().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn conntrack_read_log_warns_on_a_new_outcome_and_not_on_a_repeat() {
|
||||
use std::io::ErrorKind;
|
||||
|
||||
let mut log = ConntrackReadLog::default();
|
||||
|
||||
// The sequence a kernel without the proc file produces, then a source
|
||||
// that comes back, then fails again.
|
||||
assert_eq!(log.observe(Some(ErrorKind::NotFound)), ReadReport::Changed);
|
||||
assert_eq!(log.observe(Some(ErrorKind::NotFound)), ReadReport::Repeated);
|
||||
assert_eq!(log.observe(None), ReadReport::Changed);
|
||||
assert_eq!(log.observe(None), ReadReport::Repeated);
|
||||
assert_eq!(log.observe(Some(ErrorKind::NotFound)), ReadReport::Changed);
|
||||
assert_eq!(
|
||||
log.observe(Some(ErrorKind::PermissionDenied)),
|
||||
ReadReport::Changed,
|
||||
"a different failure is a different outcome and is worth a line"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -192,6 +192,7 @@ CHAOS_SUITES=(
|
||||
# on disk and it remains runnable by hand via
|
||||
# testing/chaos/scripts/chaos.sh bloom-storm.
|
||||
GATEWAY_SUITES=(gateway)
|
||||
OPENWRT_SUITES=(openwrt-scripts)
|
||||
SIDECAR_SUITES=(sidecar)
|
||||
FIREWALL_SUITES=(firewall)
|
||||
NAT_SUITES=(cone symmetric lan)
|
||||
@@ -229,6 +230,9 @@ list_suites() {
|
||||
echo " Gateway:"
|
||||
for s in "${GATEWAY_SUITES[@]}"; do echo " $s"; done
|
||||
echo ""
|
||||
echo " OpenWrt packaging:"
|
||||
for s in "${OPENWRT_SUITES[@]}"; do echo " $s"; done
|
||||
echo ""
|
||||
echo " Firewall baseline:"
|
||||
for s in "${FIREWALL_SUITES[@]}"; do echo " $s"; done
|
||||
echo ""
|
||||
@@ -1098,6 +1102,16 @@ run_tor_directory() {
|
||||
run_integration() {
|
||||
stage "Stage 3: Integration Tests"
|
||||
|
||||
# First, and before the build context: the OpenWrt scenarios need no FIPS
|
||||
# binary and no test image, so a packaging regression is reported in
|
||||
# seconds rather than after the image build.
|
||||
if [[ -z "$ONLY_SUITE" ]]; then
|
||||
run_openwrt_scripts
|
||||
elif [[ "$ONLY_SUITE" == "openwrt-scripts" ]]; then
|
||||
run_openwrt_scripts
|
||||
return
|
||||
fi
|
||||
|
||||
# Populate THIS run's build context, then install the binaries into it.
|
||||
# Everything but the binaries is copied from the tracked context directory;
|
||||
# the binaries are installed fresh, and a previous run's are deliberately
|
||||
@@ -1256,6 +1270,8 @@ run_suite() {
|
||||
run_static "${suite#static-}" ;;
|
||||
gateway)
|
||||
run_gateway ;;
|
||||
openwrt-scripts)
|
||||
run_openwrt_scripts ;;
|
||||
firewall)
|
||||
run_firewall ;;
|
||||
nat-cone|nat-symmetric|nat-lan)
|
||||
@@ -1336,6 +1352,19 @@ print_summary() {
|
||||
# Verify the local default suite set and the GitHub matrix still cover the
|
||||
# same work. Runs first: it takes about a second, and a divergence should be
|
||||
# reported before a half-hour suite rather than after it.
|
||||
# The OpenWrt maintainer scripts and the fips-gateway init script ship to
|
||||
# routers and run there under ash, never under bash. This runs them under ash
|
||||
# in a busybox container against stubbed init scripts, so an install, an
|
||||
# upgrade from either generation of the package, and a removal each assert what
|
||||
# the package left enabled and running.
|
||||
run_openwrt_scripts() {
|
||||
local rc=0
|
||||
info "[openwrt-scripts] Running the OpenWrt maintainer-script scenarios"
|
||||
bash "$SCRIPT_DIR/openwrt/maintainer-scripts-test.sh" || rc=$?
|
||||
record "openwrt-scripts" $rc
|
||||
return $rc
|
||||
}
|
||||
|
||||
run_ci_parity() {
|
||||
local rc=0
|
||||
info "[ci-parity] Comparing the local suite set against the GitHub matrix"
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
#!/bin/sh
|
||||
/etc/init.d/fips-gateway stop 2>/dev/null || true
|
||||
/etc/init.d/fips-gateway disable 2>/dev/null || true
|
||||
/etc/init.d/fips stop 2>/dev/null || true
|
||||
/etc/init.d/fips disable 2>/dev/null || true
|
||||
exit 0
|
||||
Executable
+46
@@ -0,0 +1,46 @@
|
||||
#!/bin/bash
|
||||
# ── OpenWrt maintainer-script scenarios ─────────────────────────────────────
|
||||
# Runs testing/openwrt/scenarios.sh inside a busybox container, so the package
|
||||
# scripts and the fips-gateway init script are interpreted by ash rather than
|
||||
# by the host's bash or dash. The scripts ship to routers and are only ever run
|
||||
# under ash there; a construct bash accepts and ash does not would otherwise
|
||||
# surface on a router.
|
||||
#
|
||||
# The container is the only reason docker is needed: the scenarios touch no
|
||||
# network and no FIPS binary, and they do not use the shared test image.
|
||||
#
|
||||
# Exit 0 = every scenario passed. Exit 1 = at least one failed. Exit 2 = the
|
||||
# harness could not run; never treated as a pass.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
|
||||
# Pinned rather than :latest so the shell under test does not change under a
|
||||
# run. Overridable for trying another ash build.
|
||||
IMAGE="${OPENWRT_ASH_IMAGE:-busybox:1.37}"
|
||||
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo "openwrt-scripts: docker not found; cannot run the ash scenarios" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ ! -f "$SCRIPT_DIR/scenarios.sh" ]]; then
|
||||
echo "openwrt-scripts: missing $SCRIPT_DIR/scenarios.sh" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
docker run --rm --network none \
|
||||
-v "$PROJECT_ROOT:/src:ro" \
|
||||
-e REPO=/src \
|
||||
-e "POSTINST=${POSTINST:-}" \
|
||||
-e "PRERM=${PRERM:-}" \
|
||||
"$IMAGE" sh /src/testing/openwrt/scenarios.sh
|
||||
rc=$?
|
||||
|
||||
if [[ $rc -ne 0 && $rc -ne 1 ]]; then
|
||||
echo "openwrt-scripts: the container exited $rc, so the scenarios did not report" >&2
|
||||
exit 2
|
||||
fi
|
||||
exit $rc
|
||||
Executable
+342
@@ -0,0 +1,342 @@
|
||||
#!/bin/sh
|
||||
# OpenWrt maintainer-script and init-guard scenarios, run under ash.
|
||||
#
|
||||
# Driven by testing/openwrt/maintainer-scripts-test.sh, which starts a busybox
|
||||
# container so /bin/sh here is ash, the shell OpenWrt runs these scripts under.
|
||||
# Nothing in this file needs opkg: the call order, the arguments and the
|
||||
# PKG_UPGRADE environment are taken from opkg-lede's own sources, so what is
|
||||
# exercised is the scripts' behaviour given that contract, not opkg itself.
|
||||
# A real `opkg upgrade` on a router image stays uncovered.
|
||||
#
|
||||
# POSTINST and PRERM may be pointed at other files. That is the seam used to
|
||||
# see a scenario red against the previously released scripts, and to re-break
|
||||
# the fixed ones during a break-check.
|
||||
|
||||
set -u
|
||||
|
||||
REPO="${REPO:-/src}"
|
||||
POSTINST="${POSTINST:-$REPO/packaging/openwrt-ipk/scripts/postinst}"
|
||||
PRERM="${PRERM:-$REPO/packaging/openwrt-ipk/scripts/prerm}"
|
||||
RELEASED_PRERM="$REPO/testing/openwrt/fixtures/released-prerm"
|
||||
INIT_GATEWAY="$REPO/packaging/openwrt-ipk/files/etc/init.d/fips-gateway"
|
||||
SHIPPED_YAML="$REPO/packaging/openwrt-ipk/files/etc/fips/fips.yaml"
|
||||
|
||||
WORK=/tmp/fips-openwrt-scenarios
|
||||
UPGRADE_MARKER=/tmp/fips-prerm-upgrade
|
||||
|
||||
FAILURES=0
|
||||
CASES=0
|
||||
|
||||
note() { echo " $*"; }
|
||||
|
||||
ok() {
|
||||
CASES=$((CASES + 1))
|
||||
echo " ok $*"
|
||||
return 0
|
||||
}
|
||||
|
||||
bad() {
|
||||
CASES=$((CASES + 1))
|
||||
FAILURES=$((FAILURES + 1))
|
||||
echo " FAIL $*"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Stub init scripts that record every call and keep an enable state file, so a
|
||||
# scenario can assert both what was invoked and what the package left behind.
|
||||
install_stubs() {
|
||||
mkdir -p /etc/init.d /etc/uci-defaults
|
||||
|
||||
cat > /etc/init.d/fips-gateway <<'STUB'
|
||||
#!/bin/sh
|
||||
echo "fips-gateway $1" >> "$CALLS"
|
||||
case "$1" in
|
||||
enable) echo 1 > "$GW_STATE" ;;
|
||||
disable) echo 0 > "$GW_STATE" ;;
|
||||
enabled) [ "$(cat "$GW_STATE")" = 1 ] ;;
|
||||
esac
|
||||
STUB
|
||||
|
||||
cat > /etc/init.d/fips <<'STUB'
|
||||
#!/bin/sh
|
||||
echo "fips $1" >> "$CALLS"
|
||||
case "$1" in
|
||||
enable) echo 1 > "$FIPS_STATE" ;;
|
||||
disable) echo 0 > "$FIPS_STATE" ;;
|
||||
enabled) [ "$(cat "$FIPS_STATE")" = 1 ] ;;
|
||||
esac
|
||||
STUB
|
||||
|
||||
cat > /etc/uci-defaults/90-fips-setup <<'STUB'
|
||||
#!/bin/sh
|
||||
echo "uci-defaults" >> "$CALLS"
|
||||
STUB
|
||||
|
||||
chmod 0755 /etc/init.d/fips-gateway /etc/init.d/fips /etc/uci-defaults/90-fips-setup
|
||||
return 0
|
||||
}
|
||||
|
||||
reset_state() {
|
||||
rm -rf "$WORK"
|
||||
mkdir -p "$WORK"
|
||||
CALLS="$WORK/calls"
|
||||
GW_STATE="$WORK/gateway-enabled"
|
||||
FIPS_STATE="$WORK/fips-enabled"
|
||||
export CALLS GW_STATE FIPS_STATE
|
||||
: > "$CALLS"
|
||||
echo 0 > "$GW_STATE"
|
||||
echo 0 > "$FIPS_STATE"
|
||||
rm -f "$UPGRADE_MARKER"
|
||||
unset PKG_UPGRADE
|
||||
install_stubs
|
||||
return 0
|
||||
}
|
||||
|
||||
calls_oneline() {
|
||||
tr '\n' ';' < "$CALLS"
|
||||
return 0
|
||||
}
|
||||
|
||||
assert_called() {
|
||||
# assert_called <expected call line> <what it means>
|
||||
if grep -qxF "$1" "$CALLS"; then
|
||||
ok "$2"
|
||||
else
|
||||
bad "$2 — '$1' is not among: $(calls_oneline)"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
assert_not_called() {
|
||||
if grep -qxF "$1" "$CALLS"; then
|
||||
bad "$2 — '$1' was called: $(calls_oneline)"
|
||||
else
|
||||
ok "$2"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
assert_file_is() {
|
||||
# assert_file_is <file> <expected contents> <what it means>
|
||||
got="$(cat "$1" 2>/dev/null)"
|
||||
if [ "$got" = "$2" ]; then
|
||||
ok "$3"
|
||||
else
|
||||
bad "$3 — expected '$2', got '$got'"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
assert_equals() {
|
||||
# assert_equals <got> <want> <what it means>
|
||||
if [ "$1" = "$2" ]; then
|
||||
ok "$3"
|
||||
else
|
||||
bad "$3 — expected '$2', got '$1'"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
assert_absent() {
|
||||
if [ -e "$1" ]; then
|
||||
bad "$2 — $1 still exists"
|
||||
else
|
||||
ok "$2"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 1. Fresh install ────────────────────────────────────────────────────────
|
||||
# opkg runs the postinst with "configure"; PKG_UPGRADE is set only on upgrades,
|
||||
# so both its absence and an explicit 0 must leave the gateway alone.
|
||||
scenario_fresh_install() {
|
||||
for pkg_upgrade in unset 0; do
|
||||
note "scenario 1: fresh install (PKG_UPGRADE $pkg_upgrade)"
|
||||
reset_state
|
||||
if [ "$pkg_upgrade" = "0" ]; then
|
||||
PKG_UPGRADE=0 sh "$POSTINST" configure >/dev/null 2>&1
|
||||
else
|
||||
sh "$POSTINST" configure >/dev/null 2>&1
|
||||
fi
|
||||
|
||||
assert_called "fips enable" "the daemon is enabled on a fresh install"
|
||||
assert_called "fips start" "the daemon is started on a fresh install"
|
||||
assert_not_called "fips-gateway enable" "the gateway is not enabled on a fresh install"
|
||||
assert_not_called "fips-gateway start" "the gateway is not started on a fresh install"
|
||||
assert_file_is "$GW_STATE" "0" "the gateway is left disabled on a fresh install"
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 2. Upgrade from a released package ──────────────────────────────────────
|
||||
# Its prerm disabled the gateway on its way out and left no marker, so the
|
||||
# incoming postinst cannot tell an enabled gateway from a disabled one and
|
||||
# re-enables it.
|
||||
scenario_upgrade_from_released() {
|
||||
note "scenario 2: upgrade from a released package"
|
||||
reset_state
|
||||
echo 1 > "$GW_STATE"
|
||||
echo 1 > "$FIPS_STATE"
|
||||
|
||||
sh "$RELEASED_PRERM" upgrade 0.5.1 >/dev/null 2>&1
|
||||
PKG_UPGRADE=1 sh "$POSTINST" configure >/dev/null 2>&1
|
||||
|
||||
assert_called "fips-gateway enable" "the gateway is re-enabled after a released prerm disabled it"
|
||||
assert_called "fips-gateway start" "the gateway is started again"
|
||||
assert_file_is "$GW_STATE" "1" "the gateway ends up enabled"
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 3. Upgrade from a package carrying these scripts, gateway enabled ───────
|
||||
scenario_upgrade_enabled() {
|
||||
note "scenario 3: upgrade from these scripts, gateway enabled"
|
||||
reset_state
|
||||
echo 1 > "$GW_STATE"
|
||||
echo 1 > "$FIPS_STATE"
|
||||
|
||||
sh "$PRERM" upgrade 0.5.2 >/dev/null 2>&1
|
||||
assert_file_is "$GW_STATE" "1" "the outgoing prerm does not disable the gateway on an upgrade"
|
||||
assert_not_called "fips-gateway disable" "the outgoing prerm does not call disable on an upgrade"
|
||||
assert_called "fips-gateway stop" "the outgoing prerm still stops the gateway"
|
||||
|
||||
PKG_UPGRADE=1 sh "$POSTINST" configure >/dev/null 2>&1
|
||||
assert_file_is "$GW_STATE" "1" "the gateway stays enabled across the upgrade"
|
||||
assert_called "fips-gateway start" "an enabled gateway is started again"
|
||||
assert_not_called "fips-gateway enable" "an enabled gateway does not need re-enabling"
|
||||
assert_absent "$UPGRADE_MARKER" "the postinst removes the upgrade marker"
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 4. Upgrade from a package carrying these scripts, gateway disabled ──────
|
||||
scenario_upgrade_disabled() {
|
||||
note "scenario 4: upgrade from these scripts, gateway disabled"
|
||||
reset_state
|
||||
echo 1 > "$FIPS_STATE"
|
||||
|
||||
sh "$PRERM" upgrade 0.5.2 >/dev/null 2>&1
|
||||
PKG_UPGRADE=1 sh "$POSTINST" configure >/dev/null 2>&1
|
||||
|
||||
assert_file_is "$GW_STATE" "0" "a disabled gateway stays disabled across the upgrade"
|
||||
assert_not_called "fips-gateway enable" "a disabled gateway is not enabled by the upgrade"
|
||||
assert_not_called "fips-gateway start" "a disabled gateway is not started by the upgrade"
|
||||
assert_absent "$UPGRADE_MARKER" "the postinst removes the upgrade marker"
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 5. Removal ──────────────────────────────────────────────────────────────
|
||||
scenario_removal() {
|
||||
note "scenario 5: removal"
|
||||
reset_state
|
||||
echo 1 > "$GW_STATE"
|
||||
echo 1 > "$FIPS_STATE"
|
||||
|
||||
sh "$PRERM" remove >/dev/null 2>&1
|
||||
|
||||
assert_called "fips-gateway stop" "removal stops the gateway"
|
||||
assert_called "fips-gateway disable" "removal disables the gateway"
|
||||
assert_called "fips stop" "removal stops the daemon"
|
||||
assert_called "fips disable" "removal disables the daemon"
|
||||
assert_file_is "$GW_STATE" "0" "the gateway ends up disabled"
|
||||
assert_absent "$UPGRADE_MARKER" "removal leaves no upgrade marker"
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 6. gateway_config_enabled reads the config ──────────────────────────────
|
||||
scenario_config_reader() {
|
||||
note "scenario 6: gateway_config_enabled"
|
||||
reset_state
|
||||
|
||||
# shellcheck source=/dev/null
|
||||
. "$INIT_GATEWAY"
|
||||
|
||||
CONFIG="$SHIPPED_YAML"
|
||||
assert_equals "$(gateway_config_enabled)" "true" "the shipped fips.yaml reads as true"
|
||||
|
||||
CONFIG="$WORK/disabled.yaml"
|
||||
cat > "$CONFIG" <<'YAML'
|
||||
identity:
|
||||
key_file: "/etc/fips/node.key"
|
||||
|
||||
gateway:
|
||||
enabled: false
|
||||
pool: "fd01::/112"
|
||||
|
||||
peers: []
|
||||
YAML
|
||||
assert_equals "$(gateway_config_enabled)" "false" "an explicitly disabled gateway reads as false"
|
||||
|
||||
CONFIG="$WORK/no-gateway.yaml"
|
||||
cat > "$CONFIG" <<'YAML'
|
||||
identity:
|
||||
key_file: "/etc/fips/node.key"
|
||||
|
||||
dns:
|
||||
enabled: true
|
||||
|
||||
peers: []
|
||||
YAML
|
||||
assert_equals "$(gateway_config_enabled)" "" "a config with no gateway block reads as empty"
|
||||
return 0
|
||||
}
|
||||
|
||||
# ── 7. start_service refuses to touch dnsmasq for a disabled gateway ────────
|
||||
# The init script's helpers are redefined after sourcing it, so start_service
|
||||
# runs its own decision against recorded stubs instead of uci, procd and the
|
||||
# network.
|
||||
scenario_start_service_guard() {
|
||||
note "scenario 7: start_service guard"
|
||||
|
||||
# shellcheck source=/dev/null
|
||||
. "$INIT_GATEWAY"
|
||||
|
||||
sysctl() { return 0; }
|
||||
modprobe() { return 0; }
|
||||
logger() { return 0; }
|
||||
sleep() { return 0; }
|
||||
procd_set_param() { return 0; }
|
||||
procd_close_instance() { return 0; }
|
||||
dnsmasq_swap_fips_upstream() { echo "dnsmasq_swap $1" >> "$CALLS"; return 0; }
|
||||
gateway_add_global_prefix() { echo "add_global_prefix" >> "$CALLS"; return 0; }
|
||||
gateway_add_ra_route() { echo "add_ra_route" >> "$CALLS"; return 0; }
|
||||
procd_open_instance() { echo "procd_open_instance" >> "$CALLS"; return 0; }
|
||||
|
||||
reset_state
|
||||
CONFIG="$SHIPPED_YAML"
|
||||
start_service >/dev/null 2>&1
|
||||
assert_called "dnsmasq_swap 5353" "an enabled gateway still redirects dnsmasq"
|
||||
assert_called "procd_open_instance" "an enabled gateway still starts the daemon"
|
||||
|
||||
reset_state
|
||||
CONFIG="$WORK/disabled.yaml"
|
||||
cat > "$CONFIG" <<'YAML'
|
||||
gateway:
|
||||
enabled: false
|
||||
pool: "fd01::/112"
|
||||
YAML
|
||||
start_service >/dev/null 2>&1
|
||||
assert_not_called "dnsmasq_swap 5353" "a disabled gateway does not redirect dnsmasq"
|
||||
assert_not_called "add_global_prefix" "a disabled gateway does not add the LAN prefix"
|
||||
assert_not_called "add_ra_route" "a disabled gateway does not advertise the pool route"
|
||||
assert_not_called "procd_open_instance" "a disabled gateway does not start the daemon"
|
||||
return 0
|
||||
}
|
||||
|
||||
echo "OpenWrt maintainer-script scenarios (shell: $(readlink -f /proc/$$/exe 2>/dev/null || echo sh))"
|
||||
echo " postinst: $POSTINST"
|
||||
echo " prerm: $PRERM"
|
||||
|
||||
scenario_fresh_install
|
||||
scenario_upgrade_from_released
|
||||
scenario_upgrade_enabled
|
||||
scenario_upgrade_disabled
|
||||
scenario_removal
|
||||
scenario_config_reader
|
||||
scenario_start_service_guard
|
||||
|
||||
echo ""
|
||||
if [ "$FAILURES" -eq 0 ]; then
|
||||
echo "openwrt-scripts: all $CASES checks passed"
|
||||
exit 0
|
||||
fi
|
||||
echo "openwrt-scripts: $FAILURES of $CASES checks failed"
|
||||
exit 1
|
||||
Reference in New Issue
Block a user