Merge branch 'master' into next

# Conflicts:
#	src/node/lifecycle.rs
#	testing/static/scripts/rekey-test.sh
This commit is contained in:
Johnathan Corgan
2026-04-27 16:10:11 +00:00
74 changed files with 7646 additions and 328 deletions
+12
View File
@@ -30,6 +30,18 @@ Tor daemons. Requires internet access for Tor bootstrapping.
| socks5-outbound | Outbound SOCKS5 connections through Tor to clearnet peer |
| directory-mode | Inbound via HiddenServiceDir onion service (co-located) |
### [nat/](nat/) -- NAT Traversal Lab
Real Docker NAT traversal tests for the Nostr/STUN bootstrap path,
using router containers with `iptables`-based NAT, a local Nostr relay,
and a local STUN responder.
| Scenario | Description |
| --------- | ------------------------------------------------------------ |
| cone | Two NATed peers establish a UDP traversal path |
| symmetric | UDP traversal fails under symmetric NAT, TCP fallback wins |
| lan | Peers on the same LAN prefer local addresses over reflexive |
### [chaos/](chaos/) -- Stochastic Simulation
Automated network testing with configurable node counts, topology
+6 -6
View File
@@ -145,8 +145,8 @@ record() {
run_build() {
stage "Stage 1: Build"
info "cargo build --release --features gateway"
if cargo build --release --features gateway 2>&1; then
info "cargo build --release"
if cargo build --release 2>&1; then
record "build" 0
else
record "build" 1
@@ -161,8 +161,8 @@ run_build() {
return 1
fi
info "cargo clippy --all --features gateway -- -D warnings"
if cargo clippy --all --features gateway -- -D warnings 2>&1; then
info "cargo clippy --all -- -D warnings"
if cargo clippy --all -- -D warnings 2>&1; then
record "clippy" 0
else
record "clippy" 1
@@ -177,7 +177,7 @@ run_tests() {
local cmd
if command -v cargo-nextest &>/dev/null; then
cmd="cargo nextest run --all --features gateway"
cmd="cargo nextest run --all"
info "$cmd"
if $cmd 2>&1; then
record "unit-tests" 0
@@ -185,7 +185,7 @@ run_tests() {
record "unit-tests" 1
fi
else
cmd="cargo test --all --features gateway"
cmd="cargo test --all"
info "$cmd (nextest not found, using cargo test)"
if $cmd 2>&1; then
record "unit-tests" 0
+1
View File
@@ -0,0 +1 @@
generated-configs
+99
View File
@@ -0,0 +1,99 @@
# NAT Lab Harness
Real Docker-based NAT traversal integration tests for the mainline
FIPS Nostr/STUN bootstrap path.
This harness spins up:
- two FIPS nodes
- a local Nostr relay
- a local STUN server
- one or two Linux router containers performing NAT with `iptables`
For the NAT scenarios, the node LAN interfaces are not attached to
Docker bridge networks. The harness creates explicit `veth` pairs and
moves them into the node and router namespaces after `docker compose up`
so every packet must traverse the router namespace.
It covers three scenarios:
- `cone`: both peers behind explicit namespace/veth full-cone emulation, UDP traversal succeeds
- `symmetric`: both peers behind symmetric-style NAT, UDP traversal fails, TCP fallback succeeds
- `lan`: both peers share a LAN subnet, LAN targets are preferred over reflexive addresses
## NAT model notes
The harness does not rely on plain Docker `MASQUERADE` for the cone case.
- `cone`
- uses explicit full-cone emulation in the router namespace
- outbound UDP is `SNAT`ed to the router WAN address while preserving the source port
- inbound UDP to the router WAN address is `DNAT`ed back to the single LAN host regardless of remote source
- `symmetric`
- uses UDP `MASQUERADE --random-fully`
- outbound mappings may be port-randomized and are only reopened by matching conntrack state
This distinction matters because plain `MASQUERADE` is convenient source NAT, but it does not by itself model the "accept from any remote once mapped" behavior expected from a full-cone NAT.
## Prerequisites
- Docker with Compose support
- locally built `fips-test:latest`
Build the test image with:
```bash
./testing/scripts/build.sh
```
## Run
Run all scenarios:
```bash
./testing/nat/scripts/nat-test.sh
```
Run one scenario:
```bash
./testing/nat/scripts/nat-test.sh cone
./testing/nat/scripts/nat-test.sh symmetric
./testing/nat/scripts/nat-test.sh lan
```
## Layout
- `docker-compose.yml`
- relay/STUN/WAN topology plus container definitions
- `node/`
- node bootstrap wrapper that waits for the injected veth interface
- `router/`
- NAT router image and `iptables` setup
- `stun/`
- minimal STUN binding responder
- `relay/`
- local `strfry` config
- `scripts/generate-configs.sh`
- derives ephemeral identities and writes per-scenario FIPS configs
- `scripts/setup-topology.sh`
- injects and configures the NAT LAN `veth` pairs in the container namespaces
- `scripts/nat-test.sh`
- boots the lab, waits for convergence, and asserts the resulting path
## Assertions
- `cone`
- both nodes connect
- connected transport is UDP
- active link remote addresses are on the WAN NAT subnet
- `symmetric`
- NAT bootstrap does not establish a UDP link
- fallback converges
- connected transport is TCP via router-published WAN addresses
- `lan`
- both nodes connect
- connected transport is UDP
- active link remote addresses stay on the shared LAN subnet
+234
View File
@@ -0,0 +1,234 @@
networks:
wan:
driver: bridge
ipam:
config:
- subnet: 172.31.254.0/24
shared-lan:
driver: bridge
ipam:
config:
- subnet: 172.31.10.0/24
volumes:
relay-data:
x-fips-common: &fips-common
image: fips-test:latest
cap_add:
- NET_ADMIN
devices:
- /dev/net/tun:/dev/net/tun
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
restart: "no"
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
services:
relay:
build:
context: ../..
dockerfile: examples/sidecar-nostr-relay/Dockerfile.app
container_name: fips-nat-relay
restart: "no"
volumes:
- relay-data:/usr/src/app/strfry-db
- ./relay/strfry.conf:/usr/src/app/strfry.conf:ro
- ../docker/resolv.conf:/etc/resolv.conf:ro
networks:
wan:
ipv4_address: 172.31.254.30
shared-lan:
ipv4_address: 172.31.10.30
stun:
build:
context: ./stun
container_name: fips-nat-stun
restart: "no"
networks:
wan:
ipv4_address: 172.31.254.40
shared-lan:
ipv4_address: 172.31.10.40
nat-a:
build:
context: ./router
profiles: ["cone", "symmetric"]
container_name: fips-nat-router-a
cap_add:
- NET_ADMIN
sysctls:
- net.ipv4.ip_forward=1
restart: "no"
environment:
- NAT_MODE=${NAT_MODE_A:-cone}
- TCP_FORWARD_PORTS=8443
- LAN_IF=eth1
- WAN_IF=eth0
- LAN_HOST=172.31.1.10
- LAN_SUBNET=172.31.1.0/24
- WAN_SUBNET=172.31.254.0/24
- WAN_GATEWAY=172.31.254.1
networks:
wan:
ipv4_address: 172.31.254.10
nat-b:
build:
context: ./router
profiles: ["cone", "symmetric"]
container_name: fips-nat-router-b
cap_add:
- NET_ADMIN
sysctls:
- net.ipv4.ip_forward=1
restart: "no"
environment:
- NAT_MODE=${NAT_MODE_B:-cone}
- TCP_FORWARD_PORTS=8443
- LAN_IF=eth1
- WAN_IF=eth0
- LAN_HOST=172.31.2.10
- LAN_SUBNET=172.31.2.0/24
- WAN_SUBNET=172.31.254.0/24
- WAN_GATEWAY=172.31.254.1
networks:
wan:
ipv4_address: 172.31.254.11
cone-a:
<<: *fips-common
profiles: ["cone"]
container_name: fips-nat-cone-a
hostname: fips-nat-cone-a
depends_on:
- nat-a
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.1.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/cone/node-a.yaml:/etc/fips/fips.yaml:ro
network_mode: none
cone-b:
<<: *fips-common
profiles: ["cone"]
container_name: fips-nat-cone-b
hostname: fips-nat-cone-b
depends_on:
- nat-b
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.2.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/cone/node-b.yaml:/etc/fips/fips.yaml:ro
network_mode: none
symmetric-a:
<<: *fips-common
profiles: ["symmetric"]
container_name: fips-nat-symmetric-a
hostname: fips-nat-symmetric-a
depends_on:
- nat-a
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.1.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/symmetric/node-a.yaml:/etc/fips/fips.yaml:ro
network_mode: none
symmetric-b:
<<: *fips-common
profiles: ["symmetric"]
container_name: fips-nat-symmetric-b
hostname: fips-nat-symmetric-b
depends_on:
- nat-b
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.2.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/symmetric/node-b.yaml:/etc/fips/fips.yaml:ro
network_mode: none
lan-a:
<<: *fips-common
profiles: ["lan"]
container_name: fips-nat-lan-a
hostname: fips-nat-lan-a
depends_on:
- relay
- stun
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./generated-configs/lan/node-a.yaml:/etc/fips/fips.yaml:ro
networks:
shared-lan:
ipv4_address: 172.31.10.10
lan-b:
<<: *fips-common
profiles: ["lan"]
container_name: fips-nat-lan-b
hostname: fips-nat-lan-b
depends_on:
- relay
- stun
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./generated-configs/lan/node-b.yaml:/etc/fips/fips.yaml:ro
networks:
shared-lan:
ipv4_address: 172.31.10.11
+79
View File
@@ -0,0 +1,79 @@
#!/bin/bash
set -euo pipefail
DATA_IF="${DATA_IF:-eth0}"
ROUTE_SUBNET="${ROUTE_SUBNET:-}"
ROUTE_VIA="${ROUTE_VIA:-}"
WAIT_TIMEOUT_SECS="${WAIT_TIMEOUT_SECS:-30}"
RELAY_HOST="${RELAY_HOST:-}"
RELAY_PORT="${RELAY_PORT:-7777}"
STUN_HOST="${STUN_HOST:-}"
STUN_PORT="${STUN_PORT:-3478}"
deadline=$((SECONDS + WAIT_TIMEOUT_SECS))
while [ "$SECONDS" -lt "$deadline" ]; do
if ip -4 addr show dev "$DATA_IF" 2>/dev/null | grep -q 'inet '; then
break
fi
sleep 0.5
done
if ! ip -4 addr show dev "$DATA_IF" 2>/dev/null | grep -q 'inet '; then
echo "Timed out waiting for IPv4 on ${DATA_IF}" >&2
ip addr >&2 || true
exit 1
fi
ip link set lo up
ip link set "$DATA_IF" up
if [ -n "$ROUTE_SUBNET" ] && [ -n "$ROUTE_VIA" ]; then
ip route replace "$ROUTE_SUBNET" via "$ROUTE_VIA" dev "$DATA_IF"
fi
wait_for_tcp() {
local host="$1"
local port="$2"
local deadline=$((SECONDS + WAIT_TIMEOUT_SECS))
while [ "$SECONDS" -lt "$deadline" ]; do
if nc -z -w1 "$host" "$port" >/dev/null 2>&1; then
return 0
fi
sleep 0.5
done
return 1
}
wait_for_udp() {
local host="$1"
local port="$2"
local deadline=$((SECONDS + WAIT_TIMEOUT_SECS))
while [ "$SECONDS" -lt "$deadline" ]; do
if printf 'probe' | nc -u -w1 "$host" "$port" >/dev/null 2>&1; then
return 0
fi
sleep 0.5
done
return 1
}
if [ -n "$RELAY_HOST" ]; then
wait_for_tcp "$RELAY_HOST" "$RELAY_PORT" || {
echo "Timed out waiting for relay ${RELAY_HOST}:${RELAY_PORT}" >&2
exit 1
}
fi
if [ -n "$STUN_HOST" ]; then
wait_for_udp "$STUN_HOST" "$STUN_PORT" || {
echo "Timed out waiting for STUN ${STUN_HOST}:${STUN_PORT}" >&2
exit 1
}
fi
exec /usr/local/bin/entrypoint.sh
+33
View File
@@ -0,0 +1,33 @@
db = "/usr/src/app/strfry-db/"
relay {
bind = "0.0.0.0"
port = 7777
nofiles = 0
info {
name = "FIPS NAT Lab Relay"
description = "Local relay for Docker NAT traversal tests."
pubkey = ""
contact = ""
}
maxWebsocketPayloadSize = 131072
autoPingSeconds = 30
enableTcpNoDelay = true
rejectFutureEventsSeconds = 60
rejectEphemeralEventsOlderThanSeconds = 60
rejectEventsNewerThanSeconds = 60
maxFilterLimit = 500
maxSubsPerConnection = 50
writePolicy {
plugin = ""
}
compression {
enabled = true
slidingWindow = true
}
}
+11
View File
@@ -0,0 +1,11 @@
FROM debian:trixie-slim
RUN apt-get update && \
apt-get install -y --no-install-recommends \
conntrack iproute2 iptables procps tcpdump netcat-openbsd && \
rm -rf /var/lib/apt/lists/*
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+134
View File
@@ -0,0 +1,134 @@
#!/bin/bash
set -euo pipefail
NAT_MODE="${NAT_MODE:-cone}"
LAN_HOST="${LAN_HOST:?LAN_HOST is required}"
LAN_SUBNET="${LAN_SUBNET:?LAN_SUBNET is required}"
WAN_SUBNET="${WAN_SUBNET:?WAN_SUBNET is required}"
TCP_FORWARD_PORTS="${TCP_FORWARD_PORTS:-8443}"
WAN_GATEWAY="${WAN_GATEWAY:-}"
find_iface_for_subnet() {
local subnet="$1"
while read -r idx iface _ cidr _; do
case "$cidr" in
${subnet%0/24}*)
echo "$iface"
return 0
;;
esac
done < <(ip -o -4 addr show)
}
wait_for_iface() {
local if_name="$1"
local timeout_secs="${2:-30}"
local deadline=$((SECONDS + timeout_secs))
while [ "$SECONDS" -lt "$deadline" ]; do
if ip link show dev "$if_name" >/dev/null 2>&1; then
return 0
fi
sleep 0.5
done
return 1
}
wait_for_subnet_iface() {
local subnet="$1"
local timeout_secs="${2:-30}"
local deadline=$((SECONDS + timeout_secs))
local iface=""
while [ "$SECONDS" -lt "$deadline" ]; do
iface="$(find_iface_for_subnet "$subnet" || true)"
if [ -n "$iface" ]; then
echo "$iface"
return 0
fi
sleep 0.5
done
return 1
}
if [ -n "${LAN_IF:-}" ]; then
wait_for_iface "$LAN_IF"
else
LAN_IF="$(wait_for_subnet_iface "$LAN_SUBNET")"
fi
if [ -n "${WAN_IF:-}" ]; then
wait_for_iface "$WAN_IF"
else
WAN_IF="$(wait_for_subnet_iface "$WAN_SUBNET")"
fi
if [ -z "${LAN_IF:-}" ] || [ -z "${WAN_IF:-}" ]; then
echo "Failed to detect LAN/WAN interfaces"
ip -o -4 addr show
exit 1
fi
if [ -z "$WAN_GATEWAY" ]; then
WAN_GATEWAY="$(echo "$WAN_SUBNET" | awk -F. '{print $1 "." $2 "." $3 ".1"}')"
fi
WAN_ADDR="$(ip -o -4 addr show dev "$WAN_IF" | awk '{print $4}' | cut -d/ -f1 | head -1)"
if [ -z "$WAN_ADDR" ]; then
echo "Failed to determine WAN IPv4 address for ${WAN_IF}"
ip -o -4 addr show dev "$WAN_IF" || true
exit 1
fi
sysctl -w net.ipv4.ip_forward=1 >/dev/null || true
sysctl -w net.ipv4.conf.all.rp_filter=0 >/dev/null || true
sysctl -w "net.ipv4.conf.${LAN_IF}.rp_filter=0" >/dev/null || true
sysctl -w "net.ipv4.conf.${WAN_IF}.rp_filter=0" >/dev/null || true
ip route replace default via "$WAN_GATEWAY" dev "$WAN_IF"
iptables -F
iptables -t nat -F
iptables -P FORWARD DROP
iptables -A FORWARD -i "$LAN_IF" -o "$WAN_IF" -j ACCEPT
iptables -A FORWARD -i "$WAN_IF" -o "$LAN_IF" -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
for port in $TCP_FORWARD_PORTS; do
iptables -t nat -A PREROUTING -i "$WAN_IF" -p tcp --dport "$port" \
-j DNAT --to-destination "${LAN_HOST}:8443"
iptables -A FORWARD -i "$WAN_IF" -o "$LAN_IF" -p tcp -d "$LAN_HOST" --dport 8443 -j ACCEPT
done
case "$NAT_MODE" in
cone)
# Full-cone emulation for the single LAN node in this harness:
# preserve the UDP source port on egress and forward any inbound UDP
# on the WAN address back to the lone LAN host on the same port.
iptables -t nat -A PREROUTING -i "$WAN_IF" -p udp -j DNAT --to-destination "$LAN_HOST"
iptables -A FORWARD -i "$WAN_IF" -o "$LAN_IF" -p udp -d "$LAN_HOST" -j ACCEPT
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" -p udp \
-j SNAT --to-source "$WAN_ADDR"
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" ! -p udp -j MASQUERADE
;;
symmetric)
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" -p udp \
-j MASQUERADE --random-fully
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" ! -p udp -j MASQUERADE
;;
*)
echo "Unknown NAT_MODE: $NAT_MODE"
exit 1
;;
esac
echo "Router ready: mode=${NAT_MODE} lan_if=${LAN_IF} wan_if=${WAN_IF}"
ip route
iptables -S
iptables -t nat -S
exec tail -f /dev/null
+138
View File
@@ -0,0 +1,138 @@
#!/bin/bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
NAT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$NAT_DIR/../.." && pwd)"
DERIVE_KEYS="$ROOT_DIR/testing/lib/derive_keys.py"
OUTPUT_DIR="$NAT_DIR/generated-configs"
SCENARIO="${1:?usage: generate-configs.sh <cone|symmetric|lan> [mesh-name]}"
MESH_NAME="${2:-nat-lab-$(date +%s)-$$}"
case "$SCENARIO" in
cone|symmetric|lan) ;;
*)
echo "Unknown scenario: $SCENARIO" >&2
exit 1
;;
esac
mkdir -p "$OUTPUT_DIR/$SCENARIO"
keys_a="$(python3 "$DERIVE_KEYS" "$MESH_NAME" "a")"
keys_b="$(python3 "$DERIVE_KEYS" "$MESH_NAME" "b")"
nsec_a="$(echo "$keys_a" | awk -F= '/^nsec=/{print $2}')"
npub_a="$(echo "$keys_a" | awk -F= '/^npub=/{print $2}')"
nsec_b="$(echo "$keys_b" | awk -F= '/^nsec=/{print $2}')"
npub_b="$(echo "$keys_b" | awk -F= '/^npub=/{print $2}')"
relay_addr="ws://172.31.254.30:7777"
stun_addr="stun:172.31.254.40:3478"
if [ "$SCENARIO" = "lan" ]; then
relay_addr="ws://172.31.10.30:7777"
stun_addr="stun:172.31.10.40:3478"
fi
peer_block_a=$(cat <<EOF
- npub: "$npub_b"
alias: "node-b"
addresses:
- transport: udp
addr: "nat"
priority: 1
EOF
)
peer_block_b=$(cat <<EOF
- npub: "$npub_a"
alias: "node-a"
addresses:
- transport: udp
addr: "nat"
priority: 1
EOF
)
if [ "$SCENARIO" = "symmetric" ]; then
peer_block_a="$peer_block_a"$'\n'" - transport: tcp
addr: \"172.31.254.11:8443\"
priority: 20"
peer_block_b="$peer_block_b"$'\n'" - transport: tcp
addr: \"172.31.254.10:8443\"
priority: 20"
fi
write_config() {
local output_file="$1"
local nsec="$2"
local peer_block="$3"
cat > "$output_file" <<EOF
node:
identity:
nsec: "$nsec"
retry:
max_retries: 3
base_interval_secs: 2
max_backoff_secs: 8
discovery:
nostr:
enabled: true
advertise: true
app: "fips.nat.lab.v1"
advert_relays:
- "$relay_addr"
dm_relays:
- "$relay_addr"
stun_servers:
- "$stun_addr"
signal_ttl_secs: 30
attempt_timeout_secs: 6
replay_window_secs: 60
punch_start_delay_ms: 500
punch_interval_ms: 100
punch_duration_ms: 2500
advert_ttl_secs: 60
advert_refresh_secs: 20
tun:
enabled: true
name: fips0
mtu: 1280
dns:
enabled: true
bind_addr: "127.0.0.1"
port: 5354
transports:
udp:
bind_addr: "0.0.0.0:2121"
mtu: 1472
advertise_on_nostr: true
public: false
tcp:
bind_addr: "0.0.0.0:8443"
peers:
$peer_block
connect_policy: auto_connect
auto_reconnect: true
EOF
}
write_config "$OUTPUT_DIR/$SCENARIO/node-a.yaml" "$nsec_a" "$peer_block_a"
write_config "$OUTPUT_DIR/$SCENARIO/node-b.yaml" "$nsec_b" "$peer_block_b"
cat > "$OUTPUT_DIR/$SCENARIO/npubs.env" <<EOF
NPUB_A=$npub_a
NPUB_B=$npub_b
MESH_NAME=$MESH_NAME
SCENARIO=$SCENARIO
EOF
echo "Generated NAT lab configs for scenario=$SCENARIO mesh=$MESH_NAME"
echo "NPUB_A=$npub_a"
echo "NPUB_B=$npub_b"
+418
View File
@@ -0,0 +1,418 @@
#!/bin/bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
NAT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$NAT_DIR/../.." && pwd)"
BUILD_SCRIPT="$ROOT_DIR/testing/scripts/build.sh"
GENERATE_SCRIPT="$SCRIPT_DIR/generate-configs.sh"
TOPOLOGY_SCRIPT="$SCRIPT_DIR/setup-topology.sh"
WAIT_LIB="$ROOT_DIR/testing/lib/wait-converge.sh"
SCENARIO="${1:-all}"
COMPOSE=(docker compose -f "$NAT_DIR/docker-compose.yml")
source "$WAIT_LIB"
cleanup() {
"${COMPOSE[@]}" --profile cone --profile symmetric --profile lan \
down -v --remove-orphans >/dev/null 2>&1 || true
}
helper_tcpdump_image() {
docker inspect -f '{{.Config.Image}}' fips-nat-router-a 2>/dev/null || echo nat-nat-a
}
dump_container_state() {
local container="$1"
echo ""
echo "--- $container: logs (last 80) ---"
docker logs "$container" 2>&1 | tail -80 || true
}
send_stun_probe() {
local container="$1"
local stun_host="$2"
local stun_port="$3"
docker exec "$container" python3 - "$stun_host" "$stun_port" <<'PY' 2>&1 || true
import os
import socket
import struct
import sys
host = sys.argv[1]
port = int(sys.argv[2])
txn_id = os.urandom(12)
request = struct.pack("!HHI", 0x0001, 0, 0x2112A442) + txn_id
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(2.0)
sock.sendto(request, (host, port))
try:
data, remote = sock.recvfrom(2048)
except socket.timeout:
print(f"stun timeout waiting for {host}:{port}")
raise SystemExit(1)
if len(data) < 20:
print(f"short stun response from {remote}: {len(data)} bytes")
raise SystemExit(1)
msg_type, msg_len, cookie = struct.unpack("!HHI", data[:8])
if msg_type != 0x0101 or cookie != 0x2112A442 or data[8:20] != txn_id:
print(f"unexpected stun response from {remote}: type=0x{msg_type:04x} len={msg_len} cookie=0x{cookie:08x}")
raise SystemExit(1)
print(f"stun binding success from {remote[0]}:{remote[1]}")
PY
}
dump_fips_state() {
local container="$1"
local relay_host="${2:-172.31.254.30}"
local relay_port="${3:-7777}"
local stun_host="${4:-172.31.254.40}"
local stun_port="${5:-3478}"
dump_container_state "$container"
echo ""
echo "--- $container: UDP sockets ---"
docker exec "$container" sh -lc 'ss -H -uanp 2>/dev/null || ss -H -uan 2>/dev/null || netstat -anu 2>/dev/null' 2>&1 || true
echo ""
echo "--- $container: fipsctl show status ---"
docker exec "$container" fipsctl show status 2>&1 || true
echo ""
echo "--- $container: fipsctl show peers ---"
docker exec "$container" fipsctl show peers 2>&1 || true
echo ""
echo "--- $container: fipsctl show links ---"
docker exec "$container" fipsctl show links 2>&1 || true
echo ""
echo "--- $container: relay reachability ---"
docker exec "$container" sh -lc "nc -vz -w5 ${relay_host} ${relay_port}" 2>&1 || true
echo ""
echo "--- $container: stun reachability ---"
send_stun_probe "$container" "$stun_host" "$stun_port"
}
dump_node_udp_probe() {
local node="$1"
local stun_host="${2:-172.31.254.40}"
local stun_port="${3:-3478}"
echo ""
echo "--- $node: UDP sockets (pre-capture) ---"
docker exec "$node" sh -lc 'ss -H -uanp 2>/dev/null || ss -H -uan 2>/dev/null || netstat -anu 2>/dev/null' 2>&1 || true
echo ""
echo "--- $node: UDP routes to STUN and peer WANs ---"
docker exec "$node" sh -lc 'for ip in 172.31.254.40 172.31.254.10 172.31.254.11; do ip route get "$ip"; done' 2>&1 || true
local capture_file
capture_file="$(mktemp)"
docker exec "$node" sh -lc "timeout 8 tcpdump -ni eth0 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
local tcpdump_pid=$!
sleep 1
echo ""
echo "--- $node: UDP active probe ---"
echo "probe: ${node} -> ${stun_host}:${stun_port}/udp (STUN binding request)"
send_stun_probe "$node" "$stun_host" "$stun_port"
wait "$tcpdump_pid" || true
echo ""
echo "--- $node: UDP tcpdump during active probe ---"
cat "$capture_file"
rm -f "$capture_file"
echo ""
echo "--- $node: UDP sockets (post-capture) ---"
docker exec "$node" sh -lc 'ss -H -uanp 2>/dev/null || ss -H -uan 2>/dev/null || netstat -anu 2>/dev/null' 2>&1 || true
}
dump_router_udp_probe() {
local router="$1"
local source_node="$2"
local stun_host="${3:-172.31.254.40}"
local stun_port="${4:-3478}"
echo ""
echo "--- $router: UDP conntrack/state (before probe) ---"
docker exec "$router" sh -lc 'conntrack -L -p udp 2>/dev/null || echo "conntrack unavailable"' 2>&1 || true
echo ""
echo "--- $router: UDP counters (before probe) ---"
docker exec "$router" sh -lc 'iptables -vnL FORWARD; echo; iptables -t nat -vnL POSTROUTING' 2>&1 || true
echo ""
echo "--- $router: UDP routes to STUN and peer WANs ---"
docker exec "$router" sh -lc 'for ip in 172.31.254.40 172.31.254.10 172.31.254.11; do ip route get "$ip"; done' 2>&1 || true
local capture_file
capture_file="$(mktemp)"
docker exec "$router" sh -lc "timeout 8 tcpdump -ni any 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
local tcpdump_pid=$!
sleep 1
echo ""
echo "--- $router: UDP active probe ---"
echo "probe: ${source_node} -> ${stun_host}:${stun_port}/udp (STUN binding request)"
send_stun_probe "$source_node" "$stun_host" "$stun_port"
wait "$tcpdump_pid" || true
echo ""
echo "--- $router: UDP tcpdump during active probe ---"
cat "$capture_file"
rm -f "$capture_file"
echo ""
echo "--- $router: UDP counters (after probe) ---"
docker exec "$router" sh -lc 'iptables -vnL FORWARD; echo; iptables -t nat -vnL POSTROUTING' 2>&1 || true
echo ""
echo "--- $router: UDP conntrack/state (after probe) ---"
docker exec "$router" sh -lc 'conntrack -L -p udp 2>/dev/null || echo "conntrack unavailable"' 2>&1 || true
}
dump_stun_udp_probe() {
local source_node="$1"
local stun_host="${2:-172.31.254.40}"
local stun_port="${3:-3478}"
local helper_image
helper_image="$(helper_tcpdump_image)"
local capture_file
capture_file="$(mktemp)"
docker run --rm --net=container:fips-nat-stun --cap-add NET_ADMIN --cap-add NET_RAW \
--entrypoint sh "$helper_image" \
-lc "timeout 8 tcpdump -ni any 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
local tcpdump_pid=$!
sleep 1
echo ""
echo "--- fips-nat-stun: UDP active probe ---"
echo "probe: ${source_node} -> ${stun_host}:${stun_port}/udp (STUN binding request)"
send_stun_probe "$source_node" "$stun_host" "$stun_port"
wait "$tcpdump_pid" || true
echo ""
echo "--- fips-nat-stun: UDP tcpdump during active probe ---"
cat "$capture_file"
rm -f "$capture_file"
}
dump_cone_diagnostics() {
echo ""
echo "=== cone diagnostics ==="
dump_fips_state fips-nat-cone-a 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-a
dump_fips_state fips-nat-cone-b 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-b
dump_container_state fips-nat-router-a
dump_router_udp_probe fips-nat-router-a fips-nat-cone-a
dump_container_state fips-nat-router-b
dump_router_udp_probe fips-nat-router-b fips-nat-cone-b
dump_container_state fips-nat-relay
dump_stun_udp_probe fips-nat-cone-a
dump_stun_udp_probe fips-nat-cone-b
dump_container_state fips-nat-stun
}
dump_symmetric_diagnostics() {
echo ""
echo "=== symmetric diagnostics ==="
dump_fips_state fips-nat-symmetric-a 172.31.254.30 7777 172.31.254.40 3478
dump_fips_state fips-nat-symmetric-b 172.31.254.30 7777 172.31.254.40 3478
dump_container_state fips-nat-router-a
dump_container_state fips-nat-router-b
dump_container_state fips-nat-relay
dump_container_state fips-nat-stun
}
dump_lan_diagnostics() {
echo ""
echo "=== lan diagnostics ==="
dump_fips_state fips-nat-lan-a 172.31.10.30 7777 172.31.10.40 3478
dump_fips_state fips-nat-lan-b 172.31.10.30 7777 172.31.10.40 3478
dump_container_state fips-nat-relay
dump_container_state fips-nat-stun
}
trap 'echo ""; echo "NAT test interrupted"; cleanup; exit 130' INT TERM
require_test_image() {
if ! docker image inspect fips-test:latest >/dev/null 2>&1; then
echo "fips-test:latest not found; building test image"
"$BUILD_SCRIPT"
fi
}
require_docker_daemon() {
if ! docker info >/dev/null 2>&1; then
echo "Docker daemon is not reachable; cannot run NAT lab harness" >&2
exit 1
fi
}
assert_peer_path() {
local container="$1"
local expected_transport="$2"
local expected_prefix="$3"
docker exec "$container" fipsctl show peers \
| python3 -c "
import json, sys
data = json.load(sys.stdin)
peers = [p for p in data.get('peers', []) if p.get('connectivity') == 'connected']
if not peers:
raise SystemExit(1)
peer = peers[0]
transport = peer.get('transport_type', '')
addr = peer.get('transport_addr', '')
if transport != sys.argv[1]:
raise SystemExit(f'transport mismatch: expected {sys.argv[1]!r}, got {transport!r}')
if not addr.startswith(sys.argv[2]):
raise SystemExit(f'addr mismatch: expected prefix {sys.argv[2]!r}, got {addr!r}')
" "$expected_transport" "$expected_prefix"
}
assert_link_path() {
local container="$1"
local expected_prefix="$2"
docker exec "$container" fipsctl show links \
| python3 -c "
import json, sys
data = json.load(sys.stdin)
links = data.get('links', [])
if not links:
raise SystemExit(1)
addr = links[0].get('remote_addr', '')
if not addr.startswith(sys.argv[1]):
raise SystemExit(f'link addr mismatch: expected prefix {sys.argv[1]!r}, got {addr!r}')
" "$expected_prefix"
}
require_bootstrap_activity() {
local container="$1"
local logs
logs="$(docker logs "$container" 2>&1 || true)"
if ! grep -Eq "bootstrap failed|Started Nostr( UDP)? NAT traversal attempt" <<<"$logs"; then
echo "Expected bootstrap activity in ${container} logs" >&2
return 1
fi
}
ping_peer() {
local container="$1"
local npub="$2"
docker exec "$container" ping6 -c 3 -W 5 "${npub}.fips" >/dev/null
}
run_cone() {
echo "=== NAT lab: cone ==="
cleanup
"$GENERATE_SCRIPT" cone
"${COMPOSE[@]}" --profile cone up -d --build --force-recreate
"$TOPOLOGY_SCRIPT" cone
wait_for_peers fips-nat-cone-a 1 45 || {
dump_cone_diagnostics
return 1
}
wait_for_peers fips-nat-cone-b 1 45 || {
dump_cone_diagnostics
return 1
}
assert_peer_path fips-nat-cone-a udp 172.31.254.
assert_peer_path fips-nat-cone-b udp 172.31.254.
assert_link_path fips-nat-cone-a 172.31.254.
assert_link_path fips-nat-cone-b 172.31.254.
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/cone/npubs.env"
ping_peer fips-nat-cone-a "$NPUB_B"
ping_peer fips-nat-cone-b "$NPUB_A"
cleanup
}
run_symmetric() {
echo "=== NAT lab: symmetric fallback ==="
cleanup
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "$GENERATE_SCRIPT" symmetric
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "${COMPOSE[@]}" --profile symmetric up -d --build --force-recreate
"$TOPOLOGY_SCRIPT" symmetric
wait_for_peers fips-nat-symmetric-a 1 60 || {
dump_symmetric_diagnostics
return 1
}
wait_for_peers fips-nat-symmetric-b 1 60 || {
dump_symmetric_diagnostics
return 1
}
assert_peer_path fips-nat-symmetric-a tcp 172.31.254.11:
assert_peer_path fips-nat-symmetric-b tcp 172.31.254.10:
assert_link_path fips-nat-symmetric-a 172.31.254.11:
assert_link_path fips-nat-symmetric-b 172.31.254.10:
require_bootstrap_activity fips-nat-symmetric-a
require_bootstrap_activity fips-nat-symmetric-b
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/symmetric/npubs.env"
ping_peer fips-nat-symmetric-a "$NPUB_B"
ping_peer fips-nat-symmetric-b "$NPUB_A"
cleanup
}
run_lan() {
echo "=== NAT lab: lan preference ==="
cleanup
"$GENERATE_SCRIPT" lan
"${COMPOSE[@]}" --profile lan up -d --build --force-recreate
wait_for_peers fips-nat-lan-a 1 45 || {
dump_lan_diagnostics
return 1
}
wait_for_peers fips-nat-lan-b 1 45 || {
dump_lan_diagnostics
return 1
}
assert_peer_path fips-nat-lan-a udp 172.31.10.
assert_peer_path fips-nat-lan-b udp 172.31.10.
assert_link_path fips-nat-lan-a 172.31.10.
assert_link_path fips-nat-lan-b 172.31.10.
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/lan/npubs.env"
ping_peer fips-nat-lan-a "$NPUB_B"
ping_peer fips-nat-lan-b "$NPUB_A"
cleanup
}
main() {
require_docker_daemon
require_test_image
case "$SCENARIO" in
all)
run_cone
run_symmetric
run_lan
;;
cone)
run_cone
;;
symmetric)
run_symmetric
;;
lan)
run_lan
;;
*)
echo "Usage: $0 [all|cone|symmetric|lan]" >&2
exit 1
;;
esac
echo "NAT lab scenarios passed"
}
main "$@"
+131
View File
@@ -0,0 +1,131 @@
#!/bin/bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
NAT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
SCENARIO="${1:?usage: setup-topology.sh <cone|symmetric>}"
case "$SCENARIO" in
cone)
node_a="fips-nat-cone-a"
node_b="fips-nat-cone-b"
;;
symmetric)
node_a="fips-nat-symmetric-a"
node_b="fips-nat-symmetric-b"
;;
*)
echo "Unsupported topology scenario: $SCENARIO" >&2
exit 1
;;
esac
router_a="fips-nat-router-a"
router_b="fips-nat-router-b"
helper_image() {
if [ -n "${IP_HELPER_IMAGE:-}" ]; then
echo "$IP_HELPER_IMAGE"
return 0
fi
docker inspect -f '{{.Config.Image}}' "$router_a"
}
wait_for_pid() {
local container="$1"
local timeout_secs="${2:-30}"
local deadline=$((SECONDS + timeout_secs))
local pid=""
while [ "$SECONDS" -lt "$deadline" ]; do
pid="$(docker inspect -f '{{.State.Pid}}' "$container" 2>/dev/null || true)"
if [[ "$pid" =~ ^[0-9]+$ ]] && [ "$pid" -gt 0 ]; then
echo "$pid"
return 0
fi
sleep 0.5
done
echo "Timed out waiting for container PID: $container" >&2
return 1
}
run_host_ip() {
local image="$1"
shift
docker run --rm \
--privileged \
--net=host \
--pid=host \
--entrypoint ip \
"$image" \
"$@"
}
configure_node_iface() {
local container="$1"
local current_name="$2"
local final_name="$3"
local cidr="$4"
docker exec "$container" sh -lc "
ip link set lo up &&
ip link set '$current_name' name '$final_name' &&
ip addr flush dev '$final_name' &&
ip addr add '$cidr' dev '$final_name' &&
ip link set '$final_name' up
"
}
configure_router_iface() {
local container="$1"
local current_name="$2"
local final_name="$3"
local cidr="$4"
docker exec "$container" sh -lc "
ip link set lo up &&
ip link set '$current_name' name '$final_name' &&
ip addr flush dev '$final_name' &&
ip addr add '$cidr' dev '$final_name' &&
ip link set '$final_name' up
"
}
setup_pair() {
local image="$1"
local node_container="$2"
local router_container="$3"
local host_node="$4"
local host_router="$5"
local node_cidr="$6"
local router_cidr="$7"
local node_pid router_pid
node_pid="$(wait_for_pid "$node_container")"
router_pid="$(wait_for_pid "$router_container")"
run_host_ip "$image" link delete "$host_node" >/dev/null 2>&1 || true
run_host_ip "$image" link delete "$host_router" >/dev/null 2>&1 || true
run_host_ip "$image" link add "$host_node" type veth peer name "$host_router"
run_host_ip "$image" link set "$host_node" netns "$node_pid"
run_host_ip "$image" link set "$host_router" netns "$router_pid"
configure_node_iface "$node_container" "$host_node" eth0 "$node_cidr"
configure_router_iface "$router_container" "$host_router" eth1 "$router_cidr"
}
main() {
cd "$NAT_DIR"
local image
image="$(helper_image)"
setup_pair "$image" "$node_a" "$router_a" vna0 vna1 172.31.1.10/24 172.31.1.254/24
setup_pair "$image" "$node_b" "$router_b" vnb0 vnb1 172.31.2.10/24 172.31.2.254/24
}
main "$@"
+8
View File
@@ -0,0 +1,8 @@
FROM python:3.12-slim
WORKDIR /app
COPY stun_server.py /app/stun_server.py
EXPOSE 3478/udp
CMD ["python3", "/app/stun_server.py"]
+37
View File
@@ -0,0 +1,37 @@
import socket
import struct
MAGIC_COOKIE = 0x2112A442
STUN_BINDING_REQUEST = 0x0001
STUN_BINDING_SUCCESS = 0x0101
STUN_ATTR_XOR_MAPPED_ADDRESS = 0x0020
def build_success(txn_id: bytes, addr: tuple[str, int]) -> bytes:
ip_bytes = socket.inet_aton(addr[0])
cookie_bytes = MAGIC_COOKIE.to_bytes(4, "big")
x_port = addr[1] ^ (MAGIC_COOKIE >> 16)
x_ip = bytes(ip_bytes[i] ^ cookie_bytes[i] for i in range(4))
value = b"\x00\x01" + struct.pack("!H", x_port) + x_ip
attr = struct.pack("!HH", STUN_ATTR_XOR_MAPPED_ADDRESS, len(value)) + value
header = struct.pack("!HHI", STUN_BINDING_SUCCESS, len(attr), MAGIC_COOKIE) + txn_id
return header + attr
def main() -> None:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("0.0.0.0", 3478))
while True:
data, remote = sock.recvfrom(2048)
if len(data) < 20:
continue
msg_type, msg_len, cookie = struct.unpack("!HHI", data[:8])
txn_id = data[8:20]
if msg_type != STUN_BINDING_REQUEST or msg_len != 0 or cookie != MAGIC_COOKIE:
continue
sock.sendto(build_success(txn_id, remote), remote)
if __name__ == "__main__":
main()
+11 -4
View File
@@ -19,6 +19,15 @@ if [ ! -f "$PROJECT_ROOT/Cargo.toml" ]; then
fi
BUILD_DOCKER=true
# NAT harness binaries need Nostr bootstrap support; everything else is
# governed by platform cfg gates after PR #79's feature-matrix collapse.
DEFAULT_CARGO_BUILD_ARGS=(--features nostr-discovery)
if [ -n "${FIPS_CARGO_BUILD_ARGS:-}" ]; then
# shellcheck disable=SC2206
CARGO_BUILD_ARGS=($FIPS_CARGO_BUILD_ARGS)
else
CARGO_BUILD_ARGS=("${DEFAULT_CARGO_BUILD_ARGS[@]}")
fi
while [ $# -gt 0 ]; do
case "$1" in
--no-docker) BUILD_DOCKER=false; shift ;;
@@ -45,14 +54,12 @@ if [ "$UNAME_S" = "Darwin" ]; then
fi
echo "Building FIPS for Linux (release) using cargo-zigbuild..."
cargo zigbuild --release --target "$CARGO_TARGET" --manifest-path="$PROJECT_ROOT/Cargo.toml"
cargo zigbuild --release --target "$CARGO_TARGET" --manifest-path="$PROJECT_ROOT/Cargo.toml" --features gateway --bin fips-gateway
cargo zigbuild --release --target "$CARGO_TARGET" --manifest-path="$PROJECT_ROOT/Cargo.toml" "${CARGO_BUILD_ARGS[@]}"
TARGET_DIR="$PROJECT_ROOT/target/$CARGO_TARGET/release"
else
echo "Building FIPS (release)..."
cargo build --release --manifest-path="$PROJECT_ROOT/Cargo.toml"
cargo build --release --manifest-path="$PROJECT_ROOT/Cargo.toml" --features gateway --bin fips-gateway
cargo build --release --manifest-path="$PROJECT_ROOT/Cargo.toml" "${CARGO_BUILD_ARGS[@]}"
TARGET_DIR="$PROJECT_ROOT/target/release"
fi
+1 -1
View File
@@ -60,7 +60,7 @@ trap 'echo ""; echo "Test interrupted"; exit 130' INT
# Wait times derived from rekey timer
BASELINE_CONVERGENCE_TIMEOUT=60
REKEY_SETTLE=5 # settle time after rekey for cutover to complete
REKEY_SETTLE=12 # > DRAIN_WINDOW_SECS (10) so post-rekey samples are off the old session
# First FMP rekey should follow shortly after the 35s interval once the mesh is
# fully converged. Keep this bounded to preserve a meaningful scheduling check
# while still allowing for log visibility at the timeout edge.