laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-10-06 03:28:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into next

Browse Source
This commit is contained in:
Johnathan Corgan
2026-10-06 01:14:55 +00:00
parent 7bbf71abe6 0e4bcac835
commit 34f60014fe
19 changed files with 939 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+30
View File
@@ -773,6 +773,12 @@ with v0.5.x or earlier peers.
- The gateway retries failed firewall rebuilds every ten seconds without
waiting for another mapping change. Retries apply the latest desired
mappings and port forwards, preserving changes across transient failures.
- An inbound port-forwarded connection from a mesh peer that also has a live
`.fips` mapping on the gateway now reaches the LAN target from the
gateway's LAN address, as connections from every other peer do. The
mapping's source rewrite used to take it first, so the target saw the
peer's pool address instead, which changed as mappings came and went and
could be answered only by a host that routes the pool to the gateway.
#### Packaging
@@ -780,6 +786,17 @@ with v0.5.x or earlier peers.
an upgrade when they were running. It stopped fips first, which stopped both
through `Requires=fips.service`, then restarted only fips, so `.fips`
resolution stayed down and the gateway stayed stopped until started by hand.
- Removing the `.deb` without purging it now removes the `.fips` DNS routing
when `fips-dns` was not running at the time, as purging already did. The
routing stayed behind, so the host kept sending `.fips` queries to
`[::1]:5354`, where nothing listens once the package is removed, and
`.fips` lookups timed out until the file was deleted by hand.
- When no supported DNS resolver is found, `fips-dns-setup` no longer tells
the host to run `apt install systemd-resolved`. The script is not
Debian-only, so the hint now names no package manager. It says to start
systemd-resolved (`systemctl enable --now`) and then restart
`fips-dns.service`, since setup uses systemd-resolved only when it is
running.
- A node no longer relays away the answer to its own lookup. A request is
flooded to every tree peer whose bloom filter claims the target, so a false
@@ -962,6 +979,11 @@ with v0.5.x or earlier peers.
closed, losing the peer's first datagram, when the kernel's descriptor
garbage collector ran before the client read the arrival message. The same
exposure on connect and listen replies is closed too.
- A native API flow can now reach a node that is not a peer, has no session,
was not resolved through DNS and has no known route. Discovery ran for it,
but its answer was discarded, and the datagram was held without being
sent. The node now records the flow's key before starting discovery, and
retries the session for held native datagrams once discovery answers.
#### OpenWrt
@@ -983,6 +1005,14 @@ with v0.5.x or earlier peers.
edited config is kept across upgrades, along with its old "physical port
names, NOT bridge names" comment, so an upgrade alone will not correct it.
#### Routing and discovery
- A backward step of the system clock, from an NTP correction, a manual set
or a VM resume, no longer holds a node's bloom filter announces for the
size of the step. The spacing between announces to a peer
(`node.bloom.update_debounce_ms`, 500 ms by default) is now measured on a
monotonic clock.
#### Sessions and rekey
- Link quality estimates no longer freeze after a link rekey. Frames the peer
docs/design/fips-gateway.md
+10 -3
View File
@@ -419,6 +419,13 @@ masquerade in the outbound pipeline; the two have disjoint match
clauses (different `iifname`/`oifname` combinations) and coexist
without interaction when both directions are active.
The per-mapping SNAT rules match on source address alone, so an
inbound forwarded connection from a mesh peer that also holds a live
mapping matches both its SNAT and the LAN-side masquerade. NAT
statements are terminal, and the rebuild places the LAN-side
masquerade ahead of every per-mapping SNAT, so that peer's connection
reaches the LAN target from the gateway's LAN address like any other.
### Independence From Outbound
The inbound half does not require:
@@ -449,9 +456,9 @@ sequence is:
1. Add the table (which succeeds whether or not it exists), delete
it, and add it again, so the delete always has a target.
2. Add the `prerouting` and `postrouting` chains; the always-on
`oifname fips0` masquerade; per-mapping DNAT/SNAT rules for
every live pool entry; per-port-forward DNAT rules; the LAN-side
masquerade if any port-forwards exist.
`oifname fips0` masquerade; the LAN-side masquerade if any
port-forwards exist; per-mapping DNAT/SNAT rules for every live
pool entry; per-port-forward DNAT rules.
3. Send all of it as one batch, which the kernel applies as a single
transaction. Only the last message before the batch end requests
an acknowledgement, the socket's send buffer is sized to the
packaging/common/fips-dns-setup
+1 -1
View File
@@ -224,7 +224,7 @@ log "To resolve .fips domains, configure your DNS resolver to forward"
log "the .fips domain to [${FIPS_DNS_LOOPBACK_V6}]:${FIPS_DNS_PORT} (matches the daemon's default bind)."
log ""
log "Examples:"
log " systemd-resolved: sudo apt install systemd-resolved"
log " systemd-resolved: install it if needed (a separate package on some distributions), run 'sudo systemctl enable --now systemd-resolved', then 'sudo systemctl restart fips-dns.service'"
log " dnsmasq: echo 'server=/fips/${FIPS_DNS_LOOPBACK_V6}#${FIPS_DNS_PORT}' | sudo tee /etc/dnsmasq.d/fips.conf"
save_backend "none"
exit 0
packaging/debian/postrm
+21 -13
View File
@@ -3,21 +3,16 @@
set -e
case "$1" in
purge)
# Remove configuration and identity keys
rm -rf /etc/fips/
# Remove tmpfiles.d entry
rm -f /usr/lib/tmpfiles.d/fips.conf
# Remove runtime directory
rm -rf /run/fips/
remove|purge)
# Remove the DNS routing fips-dns-setup may have written, in case
# fips-dns-teardown did not run (prerm's stop runs it only when
# fips-dns.service was active), and make the resolver drop it. The
# paths match packaging/common/fips-dns-teardown, which dpkg has
# already removed, so it cannot be called from here.
# fips-dns.service was active), and make the resolver drop it. This
# runs on remove as well as purge: a removed package leaves nothing
# listening behind the routing, and purging a package already removed
# runs only postrm purge. On a purge of an installed package the
# purge pass finds nothing left and restarts nothing. The paths match
# packaging/common/fips-dns-teardown, which dpkg has already removed,
# so it cannot be called from here.
restart_resolved=0
if [ -f /etc/systemd/dns-delegate.d/fips.dns-delegate ]; then
rm -f /etc/systemd/dns-delegate.d/fips.dns-delegate
@@ -52,6 +47,19 @@ case "$1" in
|| echo "fips: warning: could not reload NetworkManager; reload it to drop the .fips route"
fi
fi
;;
esac
case "$1" in
purge)
# Remove configuration and identity keys
rm -rf /etc/fips/
# Remove tmpfiles.d entry
rm -f /usr/lib/tmpfiles.d/fips.conf
# Remove runtime directory
rm -rf /run/fips/
# Remove fips system group
if getent group fips >/dev/null 2>&1; then
src/gateway/nat.rs
+43 -6
View File
@@ -341,6 +341,16 @@ impl NatManager {
NatOp::FipsMasquerade,
];
// When any port forwards are configured, one LAN-side masquerade in
// postrouting gives the LAN target the gateway's LAN address as the
// source, so replies flow back through conntrack. It goes ahead of
// every per-mapping SNAT: those match on source address alone, NAT
// statements are terminal, and a SNAT listed first would take an
// inbound forwarded flow from a peer that holds a live mapping.
if !self.port_forwards.is_empty() {
ops.push(NatOp::LanMasquerade);
}
for mapping in self.mappings.values() {
ops.push(NatOp::Dnat(mapping.virtual_ip));
ops.push(NatOp::Snat(mapping.virtual_ip));
@@ -348,15 +358,9 @@ impl NatManager {
// Inbound port-forward rules. Each forward is one DNAT rule in
// prerouting keyed on (iif fips0, nfproto ipv6, l4proto, th dport).
// When any forwards are configured, emit a single LAN-side masquerade
// in postrouting so the LAN target host sees the gateway's LAN address
// as source and replies flow back through conntrack.
for index in 0..self.port_forwards.len() {
ops.push(NatOp::PortForward(index));
}
if !self.port_forwards.is_empty() {
ops.push(NatOp::LanMasquerade);
}
vec![ops]
}
@@ -1246,6 +1250,39 @@ mod tests {
);
}
#[test]
fn rebuild_places_the_lan_masquerade_before_every_mapping_snat() {
let mut mgr = manager_with_mappings(3);
mgr.port_forwards = vec![PortForward {
proto: Proto::Tcp,
listen_port: 8080,
target: SocketAddrV6::new(Ipv6Addr::LOCALHOST, 80, 0, 0),
}];
let ops = mgr.rebuild_batches().remove(0);
let masquerade = ops
.iter()
.position(|op| matches!(op, NatOp::LanMasquerade))
.expect("a port forward emits the LAN masquerade");
let snats: Vec<usize> = ops
.iter()
.enumerate()
.filter(|(_, op)| matches!(op, NatOp::Snat(_)))
.map(|(i, _)| i)
.collect();
assert_eq!(snats.len(), 3, "one SNAT per mapping: {ops:?}");
for snat in snats {
assert!(
masquerade < snat,
"the LAN masquerade at index {masquerade} follows the SNAT at \
index {snat}, so an inbound forwarded flow from a peer with a \
live mapping takes the SNAT and bypasses the masquerade: \
{ops:?}"
);
}
}
/// The encoded rebuild of a manager holding `count` mappings.
fn encoded_rebuild(count: u16) -> Vec<u8> {
let mgr = manager_with_mappings(count);
src/node/bloom.rs
+6 -8
View File
@@ -44,10 +44,9 @@ impl Node {
peer_addr: &NodeAddr,
filter: BloomFilter,
) -> Result<(), NodeError> {
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
// Monotonic: the debounce compares two reads, and a wall-clock step
// back would hold announces for the size of the step.
let now_ms = crate::time::mono_ms();
// Check debounce
if !self.bloom_state.should_send_update(peer_addr, now_ms) {
@@ -137,10 +136,9 @@ impl Node {
/// Send pending rate-limited filter announces whose debounce has expired.
pub(super) async fn send_pending_filter_announces(&mut self) {
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
// Monotonic: the debounce compares two reads, and a wall-clock step
// back would hold announces for the size of the step.
let now_ms = crate::time::mono_ms();
let ready: Vec<NodeAddr> = self
.peers
src/node/handlers/lookup.rs
+9 -5
View File
@@ -504,13 +504,17 @@ impl Node {
}
}
LookupAction::RetryQueuedPackets { target } => {
// If we have pending TUN packets for this target, retry session
// initiation. The coord_cache now has coords, so find_next_hop()
// should succeed.
if let Some(packets) = self.pending_tun_packets.get(&target) {
// If we hold TUN packets or native datagrams for this target,
// retry session initiation. The coord_cache now has coords, so
// find_next_hop() should succeed. Native datagrams are held in
// their own queue, and nothing else re-initiates for them.
let packets = self.pending_tun_packets.get(&target).map(|q| q.len());
let native = self.pending_native.get(&target).map(|q| q.len());
if packets.is_some() || native.is_some() {
debug!(
dest = %self.peer_display_name(&target),
queued_packets = packets.len(),
queued_packets = packets.unwrap_or(0),
queued_native = native.unwrap_or(0),
"Retrying queued packets after discovery"
);
self.retry_session_after_discovery(target).await;
src/node/handlers/native.rs
+15
View File
@@ -157,6 +157,21 @@ impl Node {
// hashes the DH output x-only and forces even parity in the XK
// premessage, so both parities derive the same material.
let pubkey = peer.public_key(secp256k1::Parity::Even);
// Cache the key the client supplied before trying the route. If there
// is none, the lookup below verifies its answer against this cache, and
// `initiate_session` registers the key only after a send succeeds, so
// without this the answer is dropped. Only on a miss, the same rule
// `cache_session_identity` follows: when no route is known, a key
// already cached from DNS, a handshake or a configured peer keeps its
// own parity. The presence check refreshes that entry, so it is not
// the oldest one when the answer arrives.
let mut prefix = [0u8; 15];
prefix.copy_from_slice(&key.peer.as_bytes()[0..15]);
if self.lookup_by_fips_prefix(&prefix).is_none() {
self.register_identity(key.peer, pubkey);
}
if let Err(error) = self.initiate_session(key.peer, pubkey).await {
debug!(
peer = %self.peer_display_name(&key.peer),
src/node/handlers/session.rs
+3 -2
View File
@@ -3557,8 +3557,9 @@ impl Node {
/// Retry session initiation after discovery provided coordinates.
///
/// Called when a LookupResponse arrives and we have pending TUN packets
/// for the discovered target. The coord_cache now has coords, so
/// `find_next_hop()` should succeed and the SessionSetup can be sent.
/// or native datagrams for the discovered target. The coord_cache now
/// has coords, so `find_next_hop()` should succeed and the SessionSetup
/// can be sent.
pub(in crate::node) async fn retry_session_after_discovery(&mut self, dest_addr: NodeAddr) {
// Look up the destination's public key from the identity cache
let mut prefix = [0u8; 15];
src/node/tests/bloom.rs
+42
View File
@@ -2113,3 +2113,45 @@ async fn test_tree_lost_announce_is_resent_after_the_fallback_when_no_receiver_r
);
cleanup_nodes(&mut line.nodes).await;
}
/// The filter-announce debounce is stamped on the monotonic clock, so a step
/// back of the wall clock cannot hold announces for the size of the step.
#[tokio::test]
async fn filter_announce_debounce_stamps_the_monotonic_clock() {
let mut nodes = run_tree_test(2, &[(0, 1)], false).await;
let peer = *nodes[1].node.node_addr();
let node = &mut nodes[0].node;
node.bloom_state.set_update_debounce_ms(0);
node.bloom_state.mark_update_needed(peer);
let sent = node.metrics().bloom.sent.get();
let before = crate::time::mono_ms();
node.send_pending_filter_announces().await;
let after = crate::time::mono_ms();
assert_eq!(
node.metrics().bloom.sent.get(),
sent + 1,
"setup: the pending announce must be sent"
);
let stamp = node.bloom_state.last_update_sent(&peer);
assert!(
stamp.is_some_and(|t| before <= t && t <= after),
"stamp {stamp:?} not in [{before}, {after}]"
);
// Regression guard for the ready-set read: with the debounce window
// still open, the peer must not even reach the send path, where the
// re-check would count it as suppressed.
node.bloom_state.set_update_debounce_ms(60_000);
node.bloom_state.mark_update_needed(peer);
let suppressed = node.metrics().bloom.debounce_suppressed.get();
node.send_pending_filter_announces().await;
assert_eq!(
node.metrics().bloom.debounce_suppressed.get(),
suppressed,
"a peer inside the debounce window must not enter the ready set"
);
cleanup_nodes(&mut nodes).await;
}
src/node/tests/discovery.rs
+205
View File
@@ -5,12 +5,15 @@
//! response routing.
use super::*;
use crate::native::link::NativeMessage;
use crate::native::registry::FlowKey;
use crate::proto::lookup::{LookupRequest, LookupResponse, RecentRequest};
use crate::proto::stp::TreeCoordinate;
use spanning_tree::{
cleanup_nodes, generate_random_edges, lock_large_network_test, process_available_packets,
run_tree_test, run_tree_test_with_mtus, verify_tree_convergence,
};
use tokio::sync::{mpsc, oneshot};
// ============================================================================
// Unit Tests — LookupRequest Handler
@@ -2200,3 +2203,205 @@ async fn test_check_pending_lookups_default_sequence_unreachable() {
assert_eq!(icmp_frame[6], 58, "next_header must be IPPROTO_ICMPV6 (58)");
assert_eq!(icmp_frame[40], 1, "ICMPv6 type 1 = Destination Unreachable");
}
// ============================================================================
// Native API — first send to an uncached, unrouted key
// ============================================================================
/// The registry's port for the listener the native tests bind.
const NATIVE_PORT: u16 = 7000;
#[tokio::test]
async fn a_native_send_to_an_unrouted_key_caches_that_key_so_discovery_can_verify_the_answer() {
let mut node = make_node();
let dest = crate::Identity::generate();
let dest_addr = *dest.node_addr();
let key = FlowKey {
peer: dest_addr,
remote: NATIVE_PORT,
local: 7001,
};
assert!(
!node.has_cached_identity(&dest_addr),
"precondition: the destination's key must not be cached"
);
node.handle_native_outbound(key, dest.pubkey(), b"hello".to_vec())
.await;
assert!(
node.has_cached_identity(&dest_addr),
"a native send with no route must cache the flow's key, or the lookup it \
starts cannot verify its answer"
);
}
#[tokio::test]
async fn a_native_send_does_not_replace_a_key_already_cached_for_that_destination() {
let mut node = make_node();
// A fixed key with odd parity, so the native path's even-parity lift would
// be visible if it replaced the cached entry. Searched over a bounded set
// of fixed secrets so the test is deterministic.
let dest = (1u8..=64)
.map(|n| {
let mut secret = [0u8; 32];
secret[31] = n;
crate::Identity::from_secret_bytes(&secret).expect("a small non-zero secret is valid")
})
.find(|id| id.pubkey_full().x_only_public_key().1 == secp256k1::Parity::Odd)
.expect("one of 64 fixed secrets has an odd-parity public key");
let dest_addr = *dest.node_addr();
let original = dest.pubkey_full();
node.register_identity(dest_addr, original);
let key = FlowKey {
peer: dest_addr,
remote: NATIVE_PORT,
local: 7001,
};
node.handle_native_outbound(key, dest.pubkey(), b"hello".to_vec())
.await;
let mut prefix = [0u8; 15];
prefix.copy_from_slice(&dest_addr.as_bytes()[0..15]);
let (_, cached) = node
.lookup_by_fips_prefix(&prefix)
.expect("the destination's key must still be cached");
assert_eq!(
cached, original,
"a native send must not replace a cached key with its even-parity lift"
);
}
#[tokio::test]
async fn a_native_first_send_to_an_uncached_unrouted_key_is_delivered_after_discovery() {
// Topology: node0 — node1 — node2. Node 0 knows nothing of node 2: no
// peer link, no cached key, no cached coordinates, no session.
let edges = vec![(0, 1), (1, 2)];
let mut nodes = run_tree_test(3, &edges, false).await;
let node0_xonly = nodes[0].node.identity().pubkey();
let node2_addr = *nodes[2].node.node_addr();
let node2_xonly = nodes[2].node.identity().pubkey();
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
assert!(
nodes[0].node.get_peer(&node2_addr).is_none(),
"precondition: node 2 must not be a peer of node 0"
);
assert!(
!nodes[0].node.has_cached_identity(&node2_addr),
"precondition: node 0 must not have node 2's key cached"
);
assert!(
!nodes[0].node.coord_cache().contains(&node2_addr, now_ms),
"precondition: node 0 must not have node 2's coordinates cached"
);
assert!(
nodes[0].node.get_session(&node2_addr).is_none(),
"precondition: node 0 must have no session to node 2"
);
// Node 2 listens on the port node 0 will send to.
let (arrivals_tx, mut arrivals) = mpsc::channel(8);
let (reply_tx, mut reply_rx) = oneshot::channel();
nodes[2].node.handle_native(NativeMessage::Listen {
port: Some(NATIVE_PORT),
arrivals: arrivals_tx,
reply: reply_tx,
});
assert_eq!(
reply_rx
.try_recv()
.expect("Listen must answer synchronously"),
Ok(NATIVE_PORT),
"node 2 must hold the listener port"
);
let lookup = &nodes[0].node.metrics().lookup;
let initiated_before = lookup.req_initiated.get();
let accepted_before = lookup.resp_accepted.get();
let identity_miss_before = lookup.resp_identity_miss.get();
let payload = b"first native datagram".to_vec();
let flow = FlowKey {
peer: node2_addr,
remote: NATIVE_PORT,
local: 7001,
};
nodes[0]
.node
.handle_native_outbound(flow, node2_xonly, payload.clone())
.await;
// Bound the drive on the arrival itself: the initiator flushes the held
// datagram in the same call that sends msg3, and packets move one hop per
// pass, so the datagram is still in transit when node 0 first shows its
// session established. Never await `recv()` here: node 2's registry holds
// the sender, so a broken fix would hang rather than fail.
let mut arrival = None;
for _ in 0..40 {
tokio::time::sleep(Duration::from_millis(50)).await;
process_available_packets(&mut nodes).await;
if let Ok(seen) = arrivals.try_recv() {
arrival = Some(seen);
break;
}
}
let lookup = &nodes[0].node.metrics().lookup;
assert!(
lookup.req_initiated.get() > initiated_before,
"the unrouted native send must start discovery"
);
assert_eq!(
lookup.resp_identity_miss.get(),
identity_miss_before,
"discovery's answer must not be dropped for want of the target's key"
);
assert!(
lookup.resp_accepted.get() > accepted_before,
"discovery's answer must be verified and accepted"
);
assert!(
nodes[0]
.node
.get_session(&node2_addr)
.is_some_and(|entry| entry.is_established()),
"discovery must lead to an established session for the held native datagram"
);
assert_eq!(
nodes[0].node.metrics().native.sent_datagrams.get(),
1,
"node 0 must send the held native datagram once the session is up"
);
let arrival = arrival.expect("node 2's listener must see the native flow arrive");
assert_eq!(
arrival.pubkey, node0_xonly,
"the arrival must carry node 0's authenticated key"
);
let (sink_tx, _sink_rx) = mpsc::channel(8);
let (accept_tx, mut accept_rx) = oneshot::channel();
nodes[2].node.handle_native(NativeMessage::Accept {
flow: arrival.flow,
sink: sink_tx,
reply: accept_tx,
});
let accepted = accept_rx
.try_recv()
.expect("Accept must answer synchronously")
.expect("node 2 must accept the announced flow");
assert_eq!(
accepted.held,
vec![payload],
"the accepted flow must hold exactly the datagram node 0 sent"
);
cleanup_nodes(&mut nodes).await;
}
src/nostr/tests.rs
+5 -1
View File
@@ -1,5 +1,6 @@
use std::collections::HashSet;
use std::net::{IpAddr, SocketAddr};
#[cfg(target_os = "linux")]
use std::time::Duration;
use nostr::nips::nip17;
@@ -14,10 +15,12 @@ use super::signal::{
estimate_clock_skew, validate_offer_freshness, validate_traversal_answer_for_offer,
};
use super::stun::{parse_stun_binding_success, parse_stun_url};
#[cfg(target_os = "linux")]
use super::traversal::run_punch_attempt;
use super::traversal::{
PunchStrategy, SourceRank, build_punch_packet, is_doc_ip, is_never_punchable_ip, is_private_ip,
now_ms, parse_punch_packet, plan_punch_targets, planned_remote_endpoints, rank_punch_source,
run_punch_attempt, session_hash,
session_hash,
};
use super::traversal_machine::suppress_responder_for_own_initiator;
use super::types::BootstrapError;
@@ -1404,6 +1407,7 @@ fn punch_socket(host: &str) -> std::net::UdpSocket {
/// A hint that starts punching immediately. `start_at_ms` is absolute wall
/// clock, so anything plausible-looking in the future would sleep out the test.
#[cfg(target_os = "linux")]
fn immediate_punch_hint(duration_ms: u64) -> PunchHint {
PunchHint {
start_at_ms: 0,
src/packaging_tests.rs
+68 -9
View File
@@ -391,17 +391,19 @@ fn freebsd_newsyslog_entry_signals_the_daemon8_supervisor_started_with_sighup_re
);
}
/// Pins the DNS cleanup in `postrm purge` and `uninstall.sh` to the files
/// `fips-dns-setup` writes, so a purge after a `fips-dns` that never ran its
/// teardown does not leave the resolver sending `.fips` to a dead responder.
/// Pins the DNS cleanup in `postrm remove` and `postrm purge` and in
/// `uninstall.sh` to the files `fips-dns-setup` writes, so a remove or purge
/// after a `fips-dns` that never ran its teardown does not leave the resolver
/// sending `.fips` to a dead responder.
///
/// This is a text test. Each path must appear on an `rm -f` line, but a
/// resolver command passes wherever it appears on a code line, including in a
/// message. What `postrm` actually does is covered by the deb-install purge
/// check. No suite runs `uninstall.sh`: its two resolved paths were run once,
/// by hand in a container, and its dnsmasq and NetworkManager paths by nothing.
/// message. What `postrm` actually does is covered by the deb-install remove
/// and purge checks. No suite runs `uninstall.sh`: its two resolved paths were
/// run once, by hand in a container, and its dnsmasq and NetworkManager paths
/// by nothing.
#[test]
fn dns_cleanup_in_postrm_purge_and_uninstall_removes_every_file_fips_dns_setup_writes_and_restarts_its_resolver()
fn dns_cleanup_in_postrm_remove_and_purge_and_uninstall_removes_every_file_fips_dns_setup_writes_and_restarts_its_resolver()
{
let setup = rc_vars(&repo_file("packaging/common/fips-dns-setup"));
let teardown = rc_vars(&repo_file("packaging/common/fips-dns-teardown"));
@@ -433,8 +435,8 @@ fn dns_cleanup_in_postrm_purge_and_uninstall_removes_every_file_fips_dns_setup_w
let scripts = [
(
"packaging/debian/postrm purge)",
case_branch(&repo_file("packaging/debian/postrm"), "purge"),
"packaging/debian/postrm remove|purge)",
case_branch(&repo_file("packaging/debian/postrm"), "remove|purge"),
),
(
"packaging/systemd/uninstall.sh",
@@ -1256,3 +1258,60 @@ fn openwrt_config_offers_no_ble_block_because_musl_builds_have_no_ble() {
where the BLE transport is not compiled: {found:?}"
);
}
/// `fips-dns-setup` is shared by the Debian package, the Arch packages, the
/// systemd tarball and, where it is packaged, the RPM, so the hint it prints
/// when it finds no DNS resolver must not tell the host to use one
/// distribution's package manager.
///
/// Every systemd-resolved backend in the script gates on the unit being active
/// (`is_active`), so the hint must say to start it (`enable --now`), not only to
/// install or enable it, and then to restart `fips-dns.service` so detection
/// runs again. `test_no_resolver` in `testing/dns-resolver/test.sh` asserts the
/// same on the script's real output.
#[test]
fn fips_dns_setup_no_resolver_hint_starts_resolved_and_names_no_package_manager_because_the_script_is_not_debian_only()
{
const SETUP: &str = "packaging/common/fips-dns-setup";
const BANNED: [&str; 6] = [
"apt install",
"apt-get install",
"dnf install",
"yum install",
"zypper install",
"pacman -S",
];
let lines = code_lines(&repo_file(SETUP));
let mut problems = Vec::new();
let hints: Vec<&String> = lines
.iter()
.filter(|l| l.starts_with("log \" systemd-resolved:"))
.collect();
match hints.as_slice() {
[hint] => {
for needed in ["enable --now", "restart fips-dns.service"] {
if !hint.contains(needed) {
problems.push(format!("hint lacks '{needed}': {hint}"));
}
}
}
_ => problems.push(format!(
"expected one systemd-resolved hint line, found {}: {hints:?}",
hints.len()
)),
}
for line in &lines {
for banned in BANNED {
if line.contains(banned) {
problems.push(format!("names a package manager ('{banned}'): {line}"));
}
}
}
assert!(
problems.is_empty(),
"{SETUP}'s no-resolver hint is wrong:\n {}",
problems.join("\n ")
);
}
src/proto/bloom/state.rs
+14 -1
View File
@@ -128,13 +128,20 @@ impl BloomState {
}
/// Check if we should send an update to a peer (respecting debounce).
///
/// A time earlier than the last send permits the send rather than holding
/// the update until the clock catches up; the send then restamps and the
/// ordinary debounce resumes. The comparison takes a difference rather
/// than adding the debounce, so a very large debounce cannot overflow.
pub fn should_send_update(&self, peer_id: &NodeAddr, current_time_ms: u64) -> bool {
if !self.pending_updates.contains(peer_id) {
return false;
}
match self.last_update_sent.get(peer_id) {
Some(&last_time) => current_time_ms >= last_time + self.update_debounce_ms,
Some(&last_time) => current_time_ms
.checked_sub(last_time)
.is_none_or(|elapsed| elapsed >= self.update_debounce_ms),
None => true,
}
}
@@ -145,6 +152,12 @@ impl BloomState {
self.pending_updates.remove(&peer_id);
}
/// Read back the time of the last update sent to a peer, if any.
#[cfg(test)]
pub(crate) fn last_update_sent(&self, peer_id: &NodeAddr) -> Option<u64> {
self.last_update_sent.get(peer_id).copied()
}
/// Clear all pending updates.
pub fn clear_pending_updates(&mut self) {
self.pending_updates.clear();
src/proto/bloom/tests/state.rs
+28 -2
View File
@@ -65,6 +65,31 @@ fn test_bloom_state_debounce() {
assert!(state.should_send_update(&peer, 1600));
}
#[test]
fn should_send_update_after_backward_clock_step_waits_at_most_the_debounce() {
let node = make_node_addr(0);
let peer = make_node_addr(1);
let mut state = BloomState::new(node);
state.set_update_debounce_ms(500);
state.mark_update_needed(peer);
state.record_update_sent(peer, 100_000);
state.mark_update_needed(peer);
// A clock reading 60 s before the last send must not hold the update
// until the clock regains the 60 s.
assert!(
state.should_send_update(&peer, 40_000),
"a time before the last send must permit the send"
);
// Once restamped at the earlier time, the ordinary window applies again.
state.record_update_sent(peer, 40_000);
state.mark_update_needed(peer);
assert!(!state.should_send_update(&peer, 40_200));
assert!(state.should_send_update(&peer, 40_500));
}
#[test]
fn test_bloom_state_sequence() {
let node = make_node_addr(0);
@@ -215,9 +240,10 @@ fn test_bloom_state_remove_peer_state() {
// Pending updates cleared
assert!(!state.needs_update(&peer));
// Debounce state cleared — should be able to send immediately
// Debounce state cleared — should be able to send inside the window a
// surviving stamp of 1000 would still impose
state.mark_update_needed(peer);
assert!(state.should_send_update(&peer, 0));
assert!(state.should_send_update(&peer, 1200));
// Sent filter cleared — mark_changed_peers should treat as "never sent"
state.clear_pending_updates();
src/proto/lookup/core.rs
+2 -1
View File
@@ -60,7 +60,8 @@ pub(crate) enum LookupAction {
},
/// Reset the coords-warmup counter if an established session exists.
ResetWarmupIfEstablished { target: NodeAddr },
/// Retry queued TUN packets for the target if any are pending.
/// Retry queued TUN packets or native datagrams for the target if any are
/// pending.
RetryQueuedPackets { target: NodeAddr },
}
testing/deb-install/test.sh
+158 -76
View File
@@ -10,8 +10,10 @@
# through the resolver backend that fips-dns-setup configured. Then
# exercises fips-gateway against the same daemon to verify the
# gateway/daemon default-pairing. Finally it
# purges the package with the DNS routing file planted and fips-dns
# stopped, and checks the file is removed and systemd-resolved restarted.
# removes the package with the DNS routing file planted and fips-dns
# stopped, and checks the file is removed and systemd-resolved restarted,
# then plants the file again and purges the package from config-files
# state, with the same checks.
#
# This is the most thorough test surface — it exercises:
# - cargo deb packaging (binary stripping, dependency declaration)
@@ -19,7 +21,8 @@
# of /etc/fips/fips.yaml
# - postinst maintainer scripts (systemd unit enablement,
# fips-dns.service running fips-dns-setup)
# - postrm purge (removing the DNS routing fips-dns-setup wrote)
# - postrm remove and postrm purge (removing the DNS routing
# fips-dns-setup wrote)
# - The fips, fips-dns, and (optionally) fips-gateway systemd units
# - End-to-end .fips resolution as a real user would experience it
#
@@ -342,19 +345,155 @@ check_gateway_default_listener() {
fi
}
# Purge the package with the DNS routing file planted and fips-dns stopped, and
# Put the saved DNS routing file back at <file> and restart systemd-resolved,
# then check the state in which removal leaves the file behind: the file in
# place, fips-dns.service not active, and the resolver routing .fips to
# [::1]:5354. Every failure is recorded with <label> in front.
#
# Args: <name> <file> <label>. Returns 1 if any precondition failed.
plant_routing() {
local name="$1" file="$2" label="$3"
cexec "$name" mkdir -p "$(dirname "$file")"
cexec "$name" cp /root/fips-dns.saved "$file"
cexec "$name" systemctl restart systemd-resolved >/dev/null 2>&1
local ok=1 status
if ! cexec "$name" sh -c "test -s '$file' && cmp -s '$file' /root/fips-dns.saved"; then
fail "$label: $file not restored before the $label"
ok=0
fi
if cexec "$name" systemctl is-active --quiet fips-dns.service; then
fail "$label: fips-dns.service still active before the $label"
ok=0
fi
# Captured rather than piped into grep -q: under pipefail, grep closing the
# pipe early can fail the pipeline on a match.
if ! status=$(cexec "$name" resolvectl status 2>&1); then
fail "$label: resolvectl status failed before the $label"
echo "$status" | tail -10
ok=0
elif ! grep -q ':5354' <<<"$status"; then
fail "$label: resolvectl status does not show $file in effect before the $label"
echo "$status" | tail -25
ok=0
fi
[ "$ok" = 1 ]
}
# Check that the apt run just made removed <file> and restarted
# systemd-resolved, and that the resolver no longer routes to [::1]:5354.
#
# Args: <name> <file> <label> <InvocationID of systemd-resolved before the run>
expect_cleared() {
local name="$1" file="$2" label="$3" before="$4"
# test exits 1 for a missing file; any other failure is docker exec's.
local rc=0 after status
cexec "$name" test -e "$file" || rc=$?
case "$rc" in
1) pass "$label removed $file" ;;
0) fail "$label left $file behind" ;;
*) fail "$label: could not check for $file (exit $rc)" ;;
esac
after=$(cexec "$name" systemctl show -p InvocationID --value systemd-resolved)
if [ -n "$before" ] && [ -n "$after" ] && [ "$before" != "$after" ]; then
pass "$label restarted systemd-resolved"
else
fail "$label did not restart systemd-resolved (InvocationID '$before' -> '$after')"
fi
if ! status=$(cexec "$name" resolvectl status 2>&1); then
fail "$label: resolvectl status failed after the $label"
echo "$status" | tail -10
elif grep -q ':5354' <<<"$status"; then
fail "$label: resolvectl status still routes to port 5354"
echo "$status" | tail -25
else
pass "$label: resolvectl status no longer routes to port 5354"
fi
return
}
# Remove the package with the DNS routing file planted and fips-dns stopped, and
# check that postrm removes the file and restarts systemd-resolved.
#
# Stopping fips-dns runs fips-dns-teardown, which removes the file; putting it
# back gives the state in which removal leaves it behind: a live delegation and
# an inactive fips-dns, so prerm's stop runs no teardown. Only postrm purge is
# left to clean up, and a file it misses keeps the resolver sending .fips to
# an inactive fips-dns, so prerm's stop runs no teardown. Only postrm is left
# to clean up, and a file it misses keeps the resolver sending .fips to
# [::1]:5354 after nothing listens there.
#
# Args: <name> <expected_backend>, the backend the scenario expects
# fips-dns-setup to pick (dns-delegate or global-drop-in).
check_purge_clears_dns() {
# fips-dns-setup to pick (dns-delegate or global-drop-in). Returns 0 once
# apt-get remove has succeeded, whatever the checks after it found, so the
# purge check can follow; 1, having recorded why, if anything before that
# failed.
check_remove_clears_dns() {
local name="$1" backend="$2" file
case "$backend" in
dns-delegate) file=/etc/systemd/dns-delegate.d/fips.dns-delegate ;;
global-drop-in) file=/etc/systemd/resolved.conf.d/fips.conf ;;
*)
fail "remove: no DNS routing file known for backend '$backend'"
return 1
;;
esac
# The gateway-enable restart of fips.service is passed on to fips-dns
# (Requires=fips.service), whose setup waits for fips0 before it writes the
# file, and nothing since has waited for it. After=fips.service stops the
# old instance before the daemon, so active here means the new setup ran.
if ! wait_for_service_active "$name" fips-dns.service; then
fail "remove: fips-dns.service not active again after the gateway-enable restart"
echo " --- fips-dns.service journal ---"
docker exec "$name" journalctl -u fips-dns.service --no-pager 2>&1 | tail -20
return 1
fi
if ! cexec "$name" test -f "$file"; then
fail "remove: $file not written by fips-dns-setup before the remove"
return 1
fi
# Saved inside the container: cexec runs docker exec without -i, so a
# copy piped back from the host would arrive empty.
if ! cexec "$name" cp "$file" /root/fips-dns.saved; then
fail "remove: could not save $file"
return 1
fi
cexec "$name" systemctl stop fips-dns.service >/dev/null 2>&1
plant_routing "$name" "$file" remove || return 1
# The runtime image's tag is shared by every run on the host, so show which
# postrm this run is testing.
local branches
branches=$(cexec "$name" grep -E '^[[:space:]]*[a-z|-]+\)[[:space:]]*$' \
/var/lib/dpkg/info/fips.postrm 2>&1 | tr -s ' \n' ' ')
echo " installed postrm branches: $branches"
local before
before=$(cexec "$name" systemctl show -p InvocationID --value systemd-resolved)
run_apt "$name" "$UPGRADE_APT_TIMEOUT" remove -y fips
echo " remove took ${APT_SECS}s"
if [ "$APT_RC" -ne 0 ]; then
fail "remove: apt-get remove exited $APT_RC"
echo "$APT_OUT" | tail -20
return 1
fi
expect_cleared "$name" "$file" remove "$before"
return 0
}
# Purge the removed package with the DNS routing file planted again, and check
# that postrm removes the file and restarts systemd-resolved.
#
# The package is in config-files (rc) state after the remove, so dpkg runs only
# postrm purge. Purging an installed package would run postrm remove first,
# which clears the file and leaves this check unable to fail on the purge
# branch.
#
# Args: <name> <expected_backend>, as for check_remove_clears_dns, which must
# have run first: it saves the file this plants.
check_purge_clears_dns() {
local name="$1" backend="$2" file state
case "$backend" in
dns-delegate) file=/etc/systemd/dns-delegate.d/fips.dns-delegate ;;
global-drop-in) file=/etc/systemd/resolved.conf.d/fips.conf ;;
@@ -364,55 +503,18 @@ check_purge_clears_dns() {
;;
esac
# The gateway-enable restart of fips.service is passed on to fips-dns
# (Requires=fips.service), whose setup waits for fips0 before it writes the
# file, and nothing since has waited for it. After=fips.service stops the
# old instance before the daemon, so active here means the new setup ran.
if ! wait_for_service_active "$name" fips-dns.service; then
fail "purge: fips-dns.service not active again after the gateway-enable restart"
echo " --- fips-dns.service journal ---"
docker exec "$name" journalctl -u fips-dns.service --no-pager 2>&1 | tail -20
return
fi
if ! cexec "$name" test -f "$file"; then
fail "purge: $file not written by fips-dns-setup before the purge"
return
fi
# Saved inside the container: cexec runs docker exec without -i, so a
# copy piped back from the host would arrive empty.
if ! cexec "$name" cp "$file" /root/fips-dns.saved; then
fail "purge: could not save $file"
# shellcheck disable=SC2016 # a dpkg-query format field, not a shell expansion
state=$(cexec "$name" dpkg-query -W -f='${db:Status-Abbrev}' fips 2>&1)
state="${state%"${state##*[![:space:]]}"}"
if [ "$state" != "rc" ]; then
fail "purge: fips is in state '$state' after the remove, not config-files (rc)"
return
fi
cexec "$name" systemctl stop fips-dns.service >/dev/null 2>&1
cexec "$name" cp /root/fips-dns.saved "$file"
cexec "$name" systemctl restart systemd-resolved >/dev/null 2>&1
plant_routing "$name" "$file" purge || return
local ok=1 status
if ! cexec "$name" sh -c "test -s '$file' && cmp -s '$file' /root/fips-dns.saved"; then
fail "purge: $file not restored before the purge"
ok=0
fi
if cexec "$name" systemctl is-active --quiet fips-dns.service; then
fail "purge: fips-dns.service still active before the purge"
ok=0
fi
# Captured rather than piped into grep -q: under pipefail, grep closing the
# pipe early can fail the pipeline on a match.
if ! status=$(cexec "$name" resolvectl status 2>&1); then
fail "purge: resolvectl status failed before the purge"
echo "$status" | tail -10
ok=0
elif ! grep -q ':5354' <<<"$status"; then
fail "purge: resolvectl status does not show $file in effect before the purge"
echo "$status" | tail -25
ok=0
fi
[ "$ok" = 1 ] || return
local before after
local before
before=$(cexec "$name" systemctl show -p InvocationID --value systemd-resolved)
run_apt "$name" "$UPGRADE_APT_TIMEOUT" purge -y fips
echo " purge took ${APT_SECS}s"
if [ "$APT_RC" -ne 0 ]; then
@@ -421,29 +523,7 @@ check_purge_clears_dns() {
return
fi
# test exits 1 for a missing file; any other failure is docker exec's.
local rc=0
cexec "$name" test -e "$file" || rc=$?
case "$rc" in
1) pass "purge removed $file" ;;
0) fail "purge left $file behind" ;;
*) fail "purge: could not check for $file (exit $rc)" ;;
esac
after=$(cexec "$name" systemctl show -p InvocationID --value systemd-resolved)
if [ -n "$before" ] && [ -n "$after" ] && [ "$before" != "$after" ]; then
pass "purge restarted systemd-resolved"
else
fail "purge did not restart systemd-resolved (InvocationID '$before' -> '$after')"
fi
if ! status=$(cexec "$name" resolvectl status 2>&1); then
fail "purge: resolvectl status failed after the purge"
echo "$status" | tail -10
elif grep -q ':5354' <<<"$status"; then
fail "purge: resolvectl status still routes to port 5354"
echo "$status" | tail -25
else
pass "purge: resolvectl status no longer routes to port 5354"
fi
expect_cleared "$name" "$file" purge "$before"
return
}
@@ -749,7 +829,9 @@ DOCKERFILE
check_gateway_default_listener "$name" "$npub"
check_purge_clears_dns "$name" "$expected_backend"
if check_remove_clears_dns "$name" "$expected_backend"; then
check_purge_clears_dns "$name" "$expected_backend"
fi
cleanup_container "$name"
}
testing/dns-resolver/test.sh
+16
View File
@@ -684,6 +684,22 @@ DOCKERFILE
fail "missing manual instructions warning"
fi
# The script ships beyond Debian, so the hint must not name one
# distribution's package manager.
if echo "$output" | grep -qE '(apt|apt-get|dnf|yum|zypper) install|pacman -S'; then
fail "manual instructions name a package manager"
else
pass "manual instructions name no package manager"
fi
# Setup uses systemd-resolved only when it is active, so the hint
# must say to start it, not only to install or enable it.
if echo "$output" | grep -qF 'enable --now systemd-resolved'; then
pass "manual instructions start systemd-resolved"
else
fail "manual instructions do not start systemd-resolved"
fi
run_teardown "$name" >/dev/null 2>&1
check_removed "$name" /run/fips/dns-backend \
"teardown cleaned state file" \
testing/static/scripts/gateway-test.sh
+263 -58
View File
@@ -168,6 +168,52 @@ print(found[0])
'
}
# How many per-mapping SNAT rules come before the LAN masquerade, from
# `nft list table inet fips_gateway`. NAT statements are terminal, so a SNAT
# listed first takes an inbound forwarded flow from that mapping's peer and
# the masquerade never runs; the right answer is 0. Fails when the listing
# does not hold exactly one LAN masquerade.
snat_before_masq() {
python3 -c '
import re, sys
snat = 0
before = None
masq = 0
for line in sys.stdin:
if "iifname \"fips0\"" in line and re.search(r"\bmasquerade\b", line):
masq += 1
before = snat
elif re.search(r"\bsaddr [0-9a-f:]+ .*\bsnat\b", line):
snat += 1
if masq != 1:
sys.exit(1)
print(before)
'
}
# The SNAT target of the one rule whose source match is mesh address $1, from
# `nft list table inet fips_gateway`. Fails when no rule or more than one
# matches.
snat_to() {
python3 -c '
import ipaddress, re, sys
want = ipaddress.ip_address(sys.argv[1])
found = []
for line in sys.stdin:
m = re.search(r"\bsaddr ([0-9a-f:]+) .*\bsnat\b.*?\bto \[?([0-9a-f:]+)", line)
if not m:
continue
try:
if ipaddress.ip_address(m.group(1)) == want:
found.append(ipaddress.ip_address(m.group(2)))
except ValueError:
sys.exit(1)
if len(found) != 1:
sys.exit(1)
print(found[0])
' "$@"
}
# The device of the proxy neighbour entry for address $1, from
# `ip -6 neigh show proxy`, whose lines read `ADDR dev DEV proxy`. Fails when
# no entry matches or matching entries name different devices.
@@ -337,6 +383,30 @@ gw_selftest() {
gw_case "masq_iface: no LAN masquerade" 1 "" "$nft_nolan" masq_iface || fails=$((fails + 1))
gw_case "masq_iface: empty input" 1 "" "" masq_iface || fails=$((fails + 1))
# nft_lan lists the SNAT ahead of the LAN masquerade, the order that lets
# a mapped peer's inbound forward bypass the masquerade. nft_fixed moves
# the masquerade ahead of it, and nft_dup adds a second SNAT for the same
# mesh address.
local masq_line nft_fixed nft_dup
masq_line=$(grep 'iifname "fips0" oifname' <<< "$nft_lan")
nft_fixed=$(awk -v m="$masq_line" '$0 == m {next} /saddr .* snat/ {print m} {print}' <<< "$nft_lan")
nft_dup=$(awk '/saddr .* snat/ {print; sub(/fd01::1/, "fd01::2")} {print}' <<< "$nft_lan")
gw_case "snat_before_masq: SNAT listed first" 0 1 "$nft_lan" snat_before_masq || fails=$((fails + 1))
gw_case "snat_before_masq: masquerade listed first" 0 0 "$nft_fixed" snat_before_masq || fails=$((fails + 1))
gw_case "snat_before_masq: no LAN masquerade" 1 "" "$nft_nolan" snat_before_masq || fails=$((fails + 1))
gw_case "snat_before_masq: empty input" 1 "" "" snat_before_masq || fails=$((fails + 1))
gw_case "snat_to: mesh address, short form" 0 fd01::1 "$nft_lan" \
snat_to fd3c:9a51:7e02:4b18::2 || fails=$((fails + 1))
gw_case "snat_to: mesh address, long form" 0 fd01::1 "$nft_lan" \
snat_to fd3c:9a51:7e02:4b18:0:0:0:2 || fails=$((fails + 1))
gw_case "snat_to: masquerade listed first" 0 fd01::1 "$nft_fixed" \
snat_to fd3c:9a51:7e02:4b18::2 || fails=$((fails + 1))
gw_case "snat_to: another mesh address" 1 "" "$nft_lan" \
snat_to fd3c:9a51:7e02:4b18::3 || fails=$((fails + 1))
gw_case "snat_to: two rules for one mesh address" 1 "" "$nft_dup" \
snat_to fd3c:9a51:7e02:4b18::2 || fails=$((fails + 1))
gw_case "snat_to: empty input" 1 "" "" snat_to fd3c:9a51:7e02:4b18::2 || fails=$((fails + 1))
# Captured lines: one run's entries were on eth0 and a wrong-interface
# run's on eth1. The mixed inputs join lines from the two captures, since
# no single run holds entries on both devices.
@@ -463,6 +533,60 @@ lan_agree() {
return 0
}
# Start the LAN-side responders the inbound port forwards reach, on gw-client,
# and wait briefly for them to bind. Used by phases 8b and 8c.
gw_responders_start() {
# Start marker HTTP servers on the LAN-side client.
# :8080 → "inbound-forward-ok" (target of tcp 18080)
# :8081 → "inbound-forward-ok-2" (target of tcp 18082)
# `docker exec -d` is required; `docker exec bash -c 'cmd &'` doesn't
# keep the child alive past the exec session, even with nohup.
docker exec "$CLIENT" sh -c '
mkdir -p /tmp/inbound /tmp/inbound2
echo "inbound-forward-ok" > /tmp/inbound/index.html
echo "inbound-forward-ok-2" > /tmp/inbound2/index.html
pkill -f "http.server 8080" 2>/dev/null || true
pkill -f "http.server 8081" 2>/dev/null || true
pkill -f "udp_echo.py" 2>/dev/null || true
' >/dev/null 2>&1 || true
docker exec -d "$CLIENT" python3 -m http.server 8080 --bind :: --directory /tmp/inbound \
>/dev/null 2>&1 || true
docker exec -d "$CLIENT" python3 -m http.server 8081 --bind :: --directory /tmp/inbound2 \
>/dev/null 2>&1 || true
# Start a UDP echo server on the LAN-side client at [::]:8081/udp.
# This is the target of the udp 18081 forward. Stash the script as a
# named file (`udp_echo.py`) so the cleanup pkill above can find it.
docker exec "$CLIENT" sh -c 'cat > /tmp/udp_echo.py <<'\''PYEOF'\''
import socket, sys
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.bind(("::", 8081))
while True:
data, addr = s.recvfrom(2048)
s.sendto(b"udp-forward-ok:" + data, addr)
PYEOF' >/dev/null 2>&1 || true
docker exec -d "$CLIENT" python3 /tmp/udp_echo.py >/dev/null 2>&1 || true
# Give the servers a moment to bind.
for _ in 1 2 3 4 5; do
TCP_READY=$(docker exec "$CLIENT" ss -6lnt 2>/dev/null | grep -cE ':8080|:8081' || true)
UDP_READY=$(docker exec "$CLIENT" ss -6lnu 2>/dev/null | grep -c ':8081' || true)
if [ "$TCP_READY" -ge 2 ] && [ "$UDP_READY" -ge 1 ]; then
break
fi
sleep 1
done
}
# Stop the responders gw_responders_start started.
gw_responders_stop() {
docker exec "$CLIENT" sh -c '
pkill -f "http.server 8080" 2>/dev/null || true
pkill -f "http.server 8081" 2>/dev/null || true
pkill -f "udp_echo.py" 2>/dev/null || true
' >/dev/null 2>&1 || true
}
echo "=== FIPS Gateway Integration Test ==="
echo ""
@@ -725,10 +849,12 @@ fi
# udp 18081 → [fd02::20]:8081 (6A — UDP DNAT runtime path)
#
# Checks the DNAT rules and the LAN-side masquerade that set_port_forwards()
# installs. The traffic through them is Phase 8b's: while the Phase 4
# mapping to gw-server is live, its SNAT rule matches gw-server's inbound
# flows before the LAN masquerade does, so probes sent here would pass
# without the masquerade.
# installs, and that the masquerade is listed ahead of every per-mapping
# SNAT. NAT statements are terminal, so a SNAT listed first would take an
# inbound forwarded flow from that mapping's peer and the target would see a
# pool address. Phase 6's listing holds Phase 4's live mappings, so it has
# SNAT rules to order against. The traffic through the forwards is Phase
# 8b's, with no mapping to gw-server, and Phase 8c's, with one.
echo ""
echo "Phase 7: Inbound port-forward rules"
@@ -762,6 +888,19 @@ if [ -n "$LAN_IF" ] && MASQ_IF=$(masq_iface <<< "$NFT_RULES"); then
else
check "LAN masquerade on the LAN interface '$LAN_IF' (no single LAN masquerade rule)" 1
fi
# At least one SNAT must be listed, so an empty or reclaimed table cannot
# pass by having nothing to order.
ORDER_SNAT=$(grep -cE "saddr [0-9a-f:]+ .*snat" <<< "$NFT_RULES" || true)
if SNAT_FIRST=$(snat_before_masq <<< "$NFT_RULES"); then
:
else
SNAT_FIRST=error
fi
if [ "$SNAT_FIRST" = "0" ] && [ "$ORDER_SNAT" -ge 1 ]; then
check "LAN masquerade listed ahead of all $ORDER_SNAT SNAT rules" 0
else
check "LAN masquerade listed ahead of every SNAT rule (SNAT rules before it: $SNAT_FIRST, SNAT rules listed: $ORDER_SNAT)" 1
fi
# Phase 8: TTL expiration and pool reclamation
echo ""
@@ -805,9 +944,10 @@ fi
#
# Mesh peer (gw-server) hits each gw-gateway fips0:<port> rule, which DNATs
# into the LAN-side gw-client, and the LAN masquerade rewrites the source to
# the gateway's LAN address. Runs after Phase 8 has reclaimed the mapping to
# gw-server, because a live mapping's SNAT rule matches the same flows first
# and would do the rewrite instead. Runs before Phase 9 kills the daemon.
# the gateway's LAN address. This is the case with no mapping to gw-server:
# it runs after Phase 8 has reclaimed the Phase 4 mapping, and the gate below
# confirms none is left. Phase 8c covers the case with a live mapping. Runs
# before Phase 9 kills the daemon.
#
# The gate reads both the control socket's mappings, a snapshot refreshed
# on the pool tick, and the kernel's table, which is what decides the rule
@@ -818,6 +958,11 @@ echo "Phase 8b: Inbound port forwards through the LAN masquerade"
SERVER_MESH=$(docker exec "$SERVER" bash -c \
"ip -6 -o addr show fips0 | awk '/inet6 fd/ {print \$4}' | cut -d/ -f1 | head -1" \
2>/dev/null || echo "")
# The gateway's mesh IPv6 (the fd00::/8 address on fips0), which phases 8b and
# 8c both probe whatever 8b's gate decides.
GW_MESH_IP=$(docker exec "$GATEWAY" bash -c \
"ip -6 -o addr show fips0 | awk '/inet6 fd/ {print \$4}' | cut -d/ -f1 | head -1" \
2>/dev/null || echo "")
if [ -n "$SERVER_MESH" ] && SERVER_MAPS=$(docker exec "$GATEWAY" bash -c \
'echo "{\"command\":\"show_mappings\"}" | nc -U -w1 /run/fips/gateway.sock 2>/dev/null' \
| server_mapped "$SERVER_MESH"); then
@@ -838,51 +983,7 @@ else
fi
if [ "$GATE_OK" = true ]; then
# Start marker HTTP servers on the LAN-side client.
# :8080 → "inbound-forward-ok" (target of tcp 18080)
# :8081 → "inbound-forward-ok-2" (target of tcp 18082)
# `docker exec -d` is required; `docker exec bash -c 'cmd &'` doesn't
# keep the child alive past the exec session, even with nohup.
docker exec "$CLIENT" sh -c '
mkdir -p /tmp/inbound /tmp/inbound2
echo "inbound-forward-ok" > /tmp/inbound/index.html
echo "inbound-forward-ok-2" > /tmp/inbound2/index.html
pkill -f "http.server 8080" 2>/dev/null || true
pkill -f "http.server 8081" 2>/dev/null || true
pkill -f "udp_echo.py" 2>/dev/null || true
' >/dev/null 2>&1 || true
docker exec -d "$CLIENT" python3 -m http.server 8080 --bind :: --directory /tmp/inbound \
>/dev/null 2>&1 || true
docker exec -d "$CLIENT" python3 -m http.server 8081 --bind :: --directory /tmp/inbound2 \
>/dev/null 2>&1 || true
# Start a UDP echo server on the LAN-side client at [::]:8081/udp.
# This is the target of the udp 18081 forward. Stash the script as a
# named file (`udp_echo.py`) so the cleanup pkill above can find it.
docker exec "$CLIENT" sh -c 'cat > /tmp/udp_echo.py <<'\''PYEOF'\''
import socket, sys
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.bind(("::", 8081))
while True:
data, addr = s.recvfrom(2048)
s.sendto(b"udp-forward-ok:" + data, addr)
PYEOF' >/dev/null 2>&1 || true
docker exec -d "$CLIENT" python3 /tmp/udp_echo.py >/dev/null 2>&1 || true
# Give the servers a moment to bind.
for _ in 1 2 3 4 5; do
TCP_READY=$(docker exec "$CLIENT" ss -6lnt 2>/dev/null | grep -cE ':8080|:8081' || true)
UDP_READY=$(docker exec "$CLIENT" ss -6lnu 2>/dev/null | grep -c ':8081' || true)
if [ "$TCP_READY" -ge 2 ] && [ "$UDP_READY" -ge 1 ]; then
break
fi
sleep 1
done
# Derive the gateway's mesh IPv6 (fd00::/8 address assigned to fips0).
GW_MESH_IP=$(docker exec "$GATEWAY" bash -c \
"ip -6 -o addr show fips0 | awk '/inet6 fd/ {print \$4}' | cut -d/ -f1 | head -1" \
2>/dev/null || echo "")
gw_responders_start
if [ -z "$GW_MESH_IP" ]; then
check "Gateway fips0 IPv6 address" 1
@@ -951,12 +1052,8 @@ except Exception as e:
fi
done
# Stop the LAN-side responders; no later phase uses them.
docker exec "$CLIENT" sh -c '
pkill -f "http.server 8080" 2>/dev/null || true
pkill -f "http.server 8081" 2>/dev/null || true
pkill -f "udp_echo.py" 2>/dev/null || true
' >/dev/null 2>&1 || true
# Stop the LAN-side responders; Phase 8c starts its own.
gw_responders_stop
else
check "Inbound HTTP via TCP forward 18080 (skipped: gate)" 1
check "Inbound HTTP via TCP forward 18082 (skipped: gate)" 1
@@ -966,6 +1063,114 @@ else
check "Reply to udp 18081 goes to the gateway's LAN address (skipped: gate)" 1
fi
# Phase 8c: Inbound port forward from a peer with a live mapping
#
# A LAN client resolves gw-server first, so the gateway holds a mapping and a
# SNAT rule for gw-server's mesh address, and gw-server then reaches tcp
# 18080. The LAN masquerade must still take the flow, so the reply goes to
# the gateway's LAN address, not to the mapping's pool address.
#
# Timing. On correct code the probe is masqueraded, so no conntrack entry
# names the virtual IP and nothing pins the new mapping. The pool drains it on
# the first tick more than the TTL (5s) after the dig and frees it on the next
# tick more than the grace (5s) later; with the 10s tick the rule can be gone
# about 15s after the dig. So the responders start and conntrack is flushed
# before the dig, and right after the gate a GET from gw-client to the
# virtual IP leaves an entry that names it, which pins the mapping from the
# next tick on. Only the dig, the gate poll and that GET sit inside the 15s.
# Do not add a step that can take seconds between the dig and the pin.
#
# The SNAT rule is read again after the probe. No DNS query happens in
# between, so a rule present at both ends was present during the probe;
# without that check, a mapping reclaimed early would let the reply check
# pass with no SNAT to compete with.
echo ""
echo "Phase 8c: Inbound port forward from a peer with a live mapping"
P8C_MISSING=""
[ -n "$GW_MESH_IP" ] || P8C_MISSING="gateway mesh address"
[ -n "$SERVER_MESH" ] || P8C_MISSING="${P8C_MISSING:+$P8C_MISSING and }$SERVER mesh address"
P8C_GATE=false
if [ -n "$P8C_MISSING" ]; then
check "Mapping and SNAT rule to $SERVER before the probe (skipped: no $P8C_MISSING)" 1
else
# Outside the window: the responders, and a flush so Phase 8b's entries
# for tcp 18080, whose reply goes to $GW_DNS, cannot answer for this probe.
gw_responders_start
docker exec "$GATEWAY" conntrack -F 2>/dev/null || true
# The window opens here.
P8C_T0=$SECONDS
P8C_VIP=$(docker exec "$CLIENT" dig +short AAAA "${NPUB_B}.fips" @${GW_DNS} 2>/dev/null \
| grep -m1 "^fd01::" || true)
P8C_SNAT=""
if [ -n "$P8C_VIP" ]; then
while :; do
if P8C_SNAT=$(docker exec "$GATEWAY" nft list table inet fips_gateway 2>/dev/null \
| snat_to "$SERVER_MESH") && same_addr "$P8C_SNAT" "$P8C_VIP"; then
P8C_GATE=true
break
fi
if [ $((SECONDS - P8C_T0)) -ge 5 ]; then
break
fi
sleep 0.5
done
fi
P8C_GATE_VALUES="dig '$P8C_VIP', SNAT target '$P8C_SNAT', $((SECONDS - P8C_T0))s after the dig"
if [ "$P8C_GATE" = true ]; then
check "Mapping and SNAT rule to $SERVER before the probe ($P8C_GATE_VALUES)" 0
else
check "Mapping and SNAT rule to $SERVER before the probe ($P8C_GATE_VALUES)" 1
fi
fi
if [ "$P8C_GATE" = true ]; then
# Pin the mapping: this GET's conntrack entry names the virtual IP.
P8C_PIN=$(docker exec "$CLIENT" curl -6 -s --max-time 3 "http://[$P8C_VIP]:8000/" 2>&1) || true
if echo "$P8C_PIN" | grep -q "Fuck IPs"; then
check "GET from $CLIENT to $P8C_VIP pins the mapping ($((SECONDS - P8C_T0))s after the dig)" 0
else
check "GET from $CLIENT to $P8C_VIP pins the mapping ($((SECONDS - P8C_T0))s after the dig, response: '${P8C_PIN:0:80}')" 1
fi
P8C_RESPONSE=$(docker exec "$SERVER" curl -6 -s --max-time 5 \
"http://[${GW_MESH_IP}]:18080/" 2>&1) || true
if echo "$P8C_RESPONSE" | grep -qE '^inbound-forward-ok$'; then
check "Inbound HTTP via TCP forward 18080 with a live mapping to $SERVER" 0
else
check "Inbound HTTP via TCP forward 18080 with a live mapping (response: '${P8C_RESPONSE:0:80}')" 1
fi
# The window closes here.
P8C_AFTER=""
if P8C_AFTER=$(docker exec "$GATEWAY" nft list table inet fips_gateway 2>/dev/null \
| snat_to "$SERVER_MESH") && same_addr "$P8C_AFTER" "$P8C_VIP"; then
check "SNAT rule to $SERVER still present after the probe ($((SECONDS - P8C_T0))s after the dig)" 0
else
check "SNAT rule to $SERVER gone after the probe (target '$P8C_AFTER', $((SECONDS - P8C_T0))s after the dig); the reply check below proves nothing on this run" 1
fi
# The probe's entry does not depend on the mapping still existing.
P8C_FOUND=""
if P8C_CT=$(docker exec "$GATEWAY" conntrack -L -f ipv6 2>/dev/null) \
&& P8C_FOUND=$(reply_dst tcp 18080 <<< "$P8C_CT") \
&& same_addr "$P8C_FOUND" "$GW_DNS"; then
check "Reply to tcp 18080 with a live mapping goes to $P8C_FOUND, the gateway's LAN address $GW_DNS" 0
else
check "Reply to tcp 18080 with a live mapping goes to '$P8C_FOUND', expected the gateway's LAN address $GW_DNS" 1
fi
else
P8C_SKIP="skipped: ${P8C_MISSING:+no $P8C_MISSING}"
[ -n "$P8C_MISSING" ] || P8C_SKIP="skipped: gate"
check "GET from $CLIENT pins the mapping ($P8C_SKIP)" 1
check "Inbound HTTP via TCP forward 18080 with a live mapping ($P8C_SKIP)" 1
check "SNAT rule to $SERVER still present after the probe ($P8C_SKIP)" 1
check "Reply to tcp 18080 with a live mapping goes to the gateway's LAN address ($P8C_SKIP)" 1
fi
if [ -z "$P8C_MISSING" ]; then
gw_responders_stop
fi
# Phase 9: SERVFAIL when daemon DNS is down
echo ""
echo "Phase 9: SERVFAIL when daemon DNS is down"