Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)

Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.

The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.

== Cargo feature and dependencies

- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
  Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
  nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
  `src/discovery/nostr/signal.rs` rather than relying on the SDK's
  rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
  exclusively.

== Wire format

Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.

Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.

Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.

== Discovery surface

- `src/discovery.rs` (always compiled)
  - `EstablishedTraversal`: bound UDP socket + selected remote +
    peer npub + optional transport name/config tuning overrides.
  - `BootstrapHandoffResult`: returned on successful handoff —
    allocated transport id, local/remote addrs, peer NodeAddr,
    session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
  - `types.rs`: wire and control types described above. `ADVERT_KIND`
    constant. `BootstrapError` enumerates failure modes (disabled,
    missing advert, missing NAT endpoint, no usable relays, invalid
    advert, invalid npub, signal timeout, punch timeout, replay,
    STUN failure, protocol, nostr, io, serde, event-parse).
  - `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
    nostr-sdk `Client`, subscribes to advert + signal event kinds,
    maintains a bounded advert cache and a bounded seen-sessions
    replay set, drains `BootstrapEvent::{Established, Failed}` for
    the node to consume, exposes `update_local_advert`,
    `request_connect`, `advert_endpoints_for_peer`,
    `cached_open_discovery_candidates`, and `shutdown`.
  - `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
    built against per-attempt ephemeral keys; inbound events are
    unwrapped against the node identity.
  - `stun.rs`: RFC 5389/8489 Binding Request client with
    XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
    observe the initiator's own reflexive address against its
    locally configured STUN list (peer-advertised STUN is
    informational, never an egress target).
  - `traversal.rs`: per-attempt candidate-pair punch planner.
    Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
    LAN-private and ULA interface addresses alongside the STUN
    reflexive address, schedules probe/ack exchanges at the
    configured interval for the configured duration, and picks the
    first candidate pair that authenticates end-to-end.

Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.

== Configuration surface

`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:

- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
  `wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
  for both relay lists, and Google / Cloudflare / Twilio for STUN.
  Operators are expected to override for production. Other
  verified-working public relays for reference:
  `nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
  `nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
  Open}` — controls whether advert-derived endpoints are consumed
  only for peers carrying `via_nostr = true`, or also for
  non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
  `local_addresses` list is empty and peers see only the reflexive
  address. Enable per-node only for genuinely same-LAN deployments;
  off-by-default eliminates the misleading-path failure mode for
  the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
  retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
  spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
  (2048) — bound memory under ambient relay volume; overflow
  evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
  `punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)

Per-peer and per-transport flags:

- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
  advert-derived addresses are appended as fallback dial candidates
  after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
  when `via_nostr: true`; validation requires at least one of the
  two to be present per peer, and the error message names the
  peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
  `UdpConfig.public: Option<bool>` — UDP transports can be
  advertised either as direct `host:port` (public = true) or as the
  `addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
  — TCP and Tor onion endpoints can be advertised as directly
  reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
  special-casing in YAML and routes through the bootstrap runtime.

Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):

- Any transport with `advertise_on_nostr = true` requires
  `node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
  `node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
  `public = false` — i.e. `udp:nat`) additionally requires at least
  one `dm_relay` and at least one `stun_server`.
  Surfaced as `ConfigError::Validation`.

== Node integration

`src/node/lifecycle.rs` is the main integration point.

- At node start (after transports are up, before TUN), if Nostr is
  enabled and the feature is compiled in, `NostrDiscovery::start` is
  invoked, the initial local overlay advert is built from the live
  transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
  at method definition and call site), which refreshes the local
  advert, drains bootstrap events, adopts established traversals,
  schedules retries for failed traversals, and — under `policy:
  open` — enqueues outbound retries for non-configured peers
  visible in the advert cache, bounded by
  `open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
  first exhausts the static address list in priority order and only
  then appends advert-derived fallback addresses; both lists run
  through the same `attempt_peer_address_list` code path. The
  `udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
  for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
  per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
  optional transport instance name), and emits an `OverlayAdvert`
  including `signalRelays` and `stunServers` when any UDP endpoint
  is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
  allocates a new `TransportId`, constructs a `UdpTransport` with
  the user-supplied (or default) `UdpConfig`, calls the new
  `adopt_socket_async` to reuse the punched socket verbatim,
  registers the transport in the normal transport map, records it
  in `bootstrap_transports`, and calls `initiate_connection` so the
  normal handshake path runs. On failure, the transport is stopped
  and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
  background tasks stop before transports are torn down. (The
  advert is not explicitly retracted; NIP-40 expiration plus the
  next refresh from any live publisher supersedes it.)

New `Node` fields:

- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
  transports adopted from NAT traversal, cleaned up via
  `cleanup_bootstrap_transport_if_unused` whenever the link,
  connection, peer, or pending-connect referencing them is removed.

Retry and error surface:

- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
  for a retry entry. `pump_retries` drops expired entries with an
  info log. Used for open-discovery retries, which expire at two
  times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
  `adopt_established_traversal` when the underlying transport
  adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
  lifecycle, rx-loop tick, and timeout bookkeeping.

== UDP transport

`src/transport/udp/`:

- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
  adopts an externally bound socket, makes it non-blocking, applies
  the configured buffer sizes (warning if the kernel clamps), and
  reports the resulting local address. Preserves the NAT mapping —
  no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
  `start_async` analogue for an already-bound socket, wiring the
  async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
  still holding a recv task or socket (for example on error
  teardown), aborts the task, clears the socket, and emits a debug
  log so the cleanup is visible in tracing rather than silent.

== Logging and observability

Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.

Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:

- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`

Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.

Other operator-facing logs:

- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
  successful return, tagged with peer npub, session id, transport
  id, and both socket endpoints, so the bootstrap handoff is
  traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
  reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
  downstream STUN, punch, and handshake logs for the same peer
  correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
  mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
  (hot path: fires for every unrelated signal event on the same
  relay).
- Traversal-offer handler failures log at debug. Expected conditions
  such as punch timeout on symmetric NAT are covered there; real
  problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
  (`max_concurrent_incoming_offers`) and state that the offer was
  rate-limited rather than failing.

== Tests

- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
  encoding, signal envelope round-trip, STUN parsing, punch-packet
  codec, and replay-window enforcement. Run under the
  `nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
  cross-field invariants and YAML parsing of the full
  `node.discovery.nostr` block plus `peers[].via_nostr`, empty
  `addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
  synthetic traversal (bound UDP socket pair + synthetic peer
  identity) through `adopt_established_traversal` and assert the
  Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
  same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
  - Local `strfry` relay, local STUN responder, and one or two
    router containers performing `iptables` NAT.
  - Node LAN interfaces are provisioned with explicit `veth` pairs
    injected into the node and router namespaces so every packet
    traverses the router namespace (plain Docker bridges are not
    used for the LAN).
  - `cone` scenario: both peers behind full-cone-emulation NAT
    (SNAT with source-port preservation, inbound DNAT back to the
    single LAN host regardless of remote source); asserts UDP
    traversal succeeds and link remote addresses are on the router
    WAN subnet.
  - `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
    traversal fails and TCP fallback converges over router-
    published WAN addresses.
  - `lan` scenario: both peers share a LAN subnet; asserts LAN
    addresses are preferred over reflexive ones.
  - Cleanup tears down all profile-gated services
    (`--profile cone --profile symmetric --profile lan`) so no
    orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
  `--features "tui nostr-discovery"` by default so NAT-harness
  binaries include bootstrap support.

== CI

- Linux release build and nextest unit-test job both use
  `--features "gateway nostr-discovery"` so the feature-gated code
  and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
  `nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
  `docker compose logs` on failure, and always stop containers.

== Packaging and operations

- `packaging/common/fips.yaml` ships a fully commented
  `node.discovery.nostr.*` block, plus documented
  `advertise_on_nostr` / `public` examples under the UDP transport,
  an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
  example under the static peer section with both a direct
  `host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
  publishes target the new default relay set.

== Documentation

- `README.md`: overlay discovery + NAT traversal moved from
  "Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
  previously described Nostr discovery and NAT traversal as future
  work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
  direction)" qualifier from the Nostr Relay Discovery section,
  expands with the `udp:nat` advertisement and bootstrap handoff
  description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
  rebinding (roaming) and initial NAT traversal (Nostr path) are
  distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
  `node.discovery.nostr.*` surface, including the three resource
  caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
  reference for the shipped mechanism, including the empty-
  `addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
  Implemented status callout, clarifies that the punch socket is
  per-peer and per-attempt rather than shared with the application
  listener, aligns field names with the shipped JSON
  (`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
  `localAddresses`, `stunServer`), sets the `d`-tag to
  `fips-overlay-v1`, names the kind as 37195, and notes that
  advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
  hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
  path, STUN/punch path, configuration surface, and Docker NAT lab.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
This commit is contained in:
Tom
2026-04-27 08:15:58 -07:00
committed by GitHub
co-authored by Johnathan Corgan
parent 1e3b2c319e
commit 34e00b9f6e
55 changed files with 7448 additions and 176 deletions
+12
View File
@@ -30,6 +30,18 @@ Tor daemons. Requires internet access for Tor bootstrapping.
| socks5-outbound | Outbound SOCKS5 connections through Tor to clearnet peer |
| directory-mode | Inbound via HiddenServiceDir onion service (co-located) |
### [nat/](nat/) -- NAT Traversal Lab
Real Docker NAT traversal tests for the Nostr/STUN bootstrap path,
using router containers with `iptables`-based NAT, a local Nostr relay,
and a local STUN responder.
| Scenario | Description |
| --------- | ------------------------------------------------------------ |
| cone | Two NATed peers establish a UDP traversal path |
| symmetric | UDP traversal fails under symmetric NAT, TCP fallback wins |
| lan | Peers on the same LAN prefer local addresses over reflexive |
### [chaos/](chaos/) -- Stochastic Simulation
Automated network testing with configurable node counts, topology
+1
View File
@@ -0,0 +1 @@
generated-configs
+99
View File
@@ -0,0 +1,99 @@
# NAT Lab Harness
Real Docker-based NAT traversal integration tests for the mainline
FIPS Nostr/STUN bootstrap path.
This harness spins up:
- two FIPS nodes
- a local Nostr relay
- a local STUN server
- one or two Linux router containers performing NAT with `iptables`
For the NAT scenarios, the node LAN interfaces are not attached to
Docker bridge networks. The harness creates explicit `veth` pairs and
moves them into the node and router namespaces after `docker compose up`
so every packet must traverse the router namespace.
It covers three scenarios:
- `cone`: both peers behind explicit namespace/veth full-cone emulation, UDP traversal succeeds
- `symmetric`: both peers behind symmetric-style NAT, UDP traversal fails, TCP fallback succeeds
- `lan`: both peers share a LAN subnet, LAN targets are preferred over reflexive addresses
## NAT model notes
The harness does not rely on plain Docker `MASQUERADE` for the cone case.
- `cone`
- uses explicit full-cone emulation in the router namespace
- outbound UDP is `SNAT`ed to the router WAN address while preserving the source port
- inbound UDP to the router WAN address is `DNAT`ed back to the single LAN host regardless of remote source
- `symmetric`
- uses UDP `MASQUERADE --random-fully`
- outbound mappings may be port-randomized and are only reopened by matching conntrack state
This distinction matters because plain `MASQUERADE` is convenient source NAT, but it does not by itself model the "accept from any remote once mapped" behavior expected from a full-cone NAT.
## Prerequisites
- Docker with Compose support
- locally built `fips-test:latest`
Build the test image with:
```bash
./testing/scripts/build.sh
```
## Run
Run all scenarios:
```bash
./testing/nat/scripts/nat-test.sh
```
Run one scenario:
```bash
./testing/nat/scripts/nat-test.sh cone
./testing/nat/scripts/nat-test.sh symmetric
./testing/nat/scripts/nat-test.sh lan
```
## Layout
- `docker-compose.yml`
- relay/STUN/WAN topology plus container definitions
- `node/`
- node bootstrap wrapper that waits for the injected veth interface
- `router/`
- NAT router image and `iptables` setup
- `stun/`
- minimal STUN binding responder
- `relay/`
- local `strfry` config
- `scripts/generate-configs.sh`
- derives ephemeral identities and writes per-scenario FIPS configs
- `scripts/setup-topology.sh`
- injects and configures the NAT LAN `veth` pairs in the container namespaces
- `scripts/nat-test.sh`
- boots the lab, waits for convergence, and asserts the resulting path
## Assertions
- `cone`
- both nodes connect
- connected transport is UDP
- active link remote addresses are on the WAN NAT subnet
- `symmetric`
- NAT bootstrap does not establish a UDP link
- fallback converges
- connected transport is TCP via router-published WAN addresses
- `lan`
- both nodes connect
- connected transport is UDP
- active link remote addresses stay on the shared LAN subnet
+234
View File
@@ -0,0 +1,234 @@
networks:
wan:
driver: bridge
ipam:
config:
- subnet: 172.31.254.0/24
shared-lan:
driver: bridge
ipam:
config:
- subnet: 172.31.10.0/24
volumes:
relay-data:
x-fips-common: &fips-common
image: fips-test:latest
cap_add:
- NET_ADMIN
devices:
- /dev/net/tun:/dev/net/tun
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
restart: "no"
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
services:
relay:
build:
context: ../..
dockerfile: examples/sidecar-nostr-relay/Dockerfile.app
container_name: fips-nat-relay
restart: "no"
volumes:
- relay-data:/usr/src/app/strfry-db
- ./relay/strfry.conf:/usr/src/app/strfry.conf:ro
- ../docker/resolv.conf:/etc/resolv.conf:ro
networks:
wan:
ipv4_address: 172.31.254.30
shared-lan:
ipv4_address: 172.31.10.30
stun:
build:
context: ./stun
container_name: fips-nat-stun
restart: "no"
networks:
wan:
ipv4_address: 172.31.254.40
shared-lan:
ipv4_address: 172.31.10.40
nat-a:
build:
context: ./router
profiles: ["cone", "symmetric"]
container_name: fips-nat-router-a
cap_add:
- NET_ADMIN
sysctls:
- net.ipv4.ip_forward=1
restart: "no"
environment:
- NAT_MODE=${NAT_MODE_A:-cone}
- TCP_FORWARD_PORTS=8443
- LAN_IF=eth1
- WAN_IF=eth0
- LAN_HOST=172.31.1.10
- LAN_SUBNET=172.31.1.0/24
- WAN_SUBNET=172.31.254.0/24
- WAN_GATEWAY=172.31.254.1
networks:
wan:
ipv4_address: 172.31.254.10
nat-b:
build:
context: ./router
profiles: ["cone", "symmetric"]
container_name: fips-nat-router-b
cap_add:
- NET_ADMIN
sysctls:
- net.ipv4.ip_forward=1
restart: "no"
environment:
- NAT_MODE=${NAT_MODE_B:-cone}
- TCP_FORWARD_PORTS=8443
- LAN_IF=eth1
- WAN_IF=eth0
- LAN_HOST=172.31.2.10
- LAN_SUBNET=172.31.2.0/24
- WAN_SUBNET=172.31.254.0/24
- WAN_GATEWAY=172.31.254.1
networks:
wan:
ipv4_address: 172.31.254.11
cone-a:
<<: *fips-common
profiles: ["cone"]
container_name: fips-nat-cone-a
hostname: fips-nat-cone-a
depends_on:
- nat-a
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.1.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/cone/node-a.yaml:/etc/fips/fips.yaml:ro
network_mode: none
cone-b:
<<: *fips-common
profiles: ["cone"]
container_name: fips-nat-cone-b
hostname: fips-nat-cone-b
depends_on:
- nat-b
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.2.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/cone/node-b.yaml:/etc/fips/fips.yaml:ro
network_mode: none
symmetric-a:
<<: *fips-common
profiles: ["symmetric"]
container_name: fips-nat-symmetric-a
hostname: fips-nat-symmetric-a
depends_on:
- nat-a
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.1.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/symmetric/node-a.yaml:/etc/fips/fips.yaml:ro
network_mode: none
symmetric-b:
<<: *fips-common
profiles: ["symmetric"]
container_name: fips-nat-symmetric-b
hostname: fips-nat-symmetric-b
depends_on:
- nat-b
- relay
- stun
entrypoint:
- /usr/local/bin/nat-node-entrypoint.sh
environment:
- RUST_LOG=info,fips::discovery::nostr=debug,fips::node::lifecycle=debug
- DATA_IF=eth0
- ROUTE_SUBNET=172.31.254.0/24
- ROUTE_VIA=172.31.2.254
- RELAY_HOST=172.31.254.30
- RELAY_PORT=7777
- STUN_HOST=172.31.254.40
- STUN_PORT=3478
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./node/entrypoint.sh:/usr/local/bin/nat-node-entrypoint.sh:ro
- ./generated-configs/symmetric/node-b.yaml:/etc/fips/fips.yaml:ro
network_mode: none
lan-a:
<<: *fips-common
profiles: ["lan"]
container_name: fips-nat-lan-a
hostname: fips-nat-lan-a
depends_on:
- relay
- stun
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./generated-configs/lan/node-a.yaml:/etc/fips/fips.yaml:ro
networks:
shared-lan:
ipv4_address: 172.31.10.10
lan-b:
<<: *fips-common
profiles: ["lan"]
container_name: fips-nat-lan-b
hostname: fips-nat-lan-b
depends_on:
- relay
- stun
volumes:
- ../docker/resolv.conf:/etc/resolv.conf:ro
- ./generated-configs/lan/node-b.yaml:/etc/fips/fips.yaml:ro
networks:
shared-lan:
ipv4_address: 172.31.10.11
+79
View File
@@ -0,0 +1,79 @@
#!/bin/bash
set -euo pipefail
DATA_IF="${DATA_IF:-eth0}"
ROUTE_SUBNET="${ROUTE_SUBNET:-}"
ROUTE_VIA="${ROUTE_VIA:-}"
WAIT_TIMEOUT_SECS="${WAIT_TIMEOUT_SECS:-30}"
RELAY_HOST="${RELAY_HOST:-}"
RELAY_PORT="${RELAY_PORT:-7777}"
STUN_HOST="${STUN_HOST:-}"
STUN_PORT="${STUN_PORT:-3478}"
deadline=$((SECONDS + WAIT_TIMEOUT_SECS))
while [ "$SECONDS" -lt "$deadline" ]; do
if ip -4 addr show dev "$DATA_IF" 2>/dev/null | grep -q 'inet '; then
break
fi
sleep 0.5
done
if ! ip -4 addr show dev "$DATA_IF" 2>/dev/null | grep -q 'inet '; then
echo "Timed out waiting for IPv4 on ${DATA_IF}" >&2
ip addr >&2 || true
exit 1
fi
ip link set lo up
ip link set "$DATA_IF" up
if [ -n "$ROUTE_SUBNET" ] && [ -n "$ROUTE_VIA" ]; then
ip route replace "$ROUTE_SUBNET" via "$ROUTE_VIA" dev "$DATA_IF"
fi
wait_for_tcp() {
local host="$1"
local port="$2"
local deadline=$((SECONDS + WAIT_TIMEOUT_SECS))
while [ "$SECONDS" -lt "$deadline" ]; do
if nc -z -w1 "$host" "$port" >/dev/null 2>&1; then
return 0
fi
sleep 0.5
done
return 1
}
wait_for_udp() {
local host="$1"
local port="$2"
local deadline=$((SECONDS + WAIT_TIMEOUT_SECS))
while [ "$SECONDS" -lt "$deadline" ]; do
if printf 'probe' | nc -u -w1 "$host" "$port" >/dev/null 2>&1; then
return 0
fi
sleep 0.5
done
return 1
}
if [ -n "$RELAY_HOST" ]; then
wait_for_tcp "$RELAY_HOST" "$RELAY_PORT" || {
echo "Timed out waiting for relay ${RELAY_HOST}:${RELAY_PORT}" >&2
exit 1
}
fi
if [ -n "$STUN_HOST" ]; then
wait_for_udp "$STUN_HOST" "$STUN_PORT" || {
echo "Timed out waiting for STUN ${STUN_HOST}:${STUN_PORT}" >&2
exit 1
}
fi
exec /usr/local/bin/entrypoint.sh
+33
View File
@@ -0,0 +1,33 @@
db = "/usr/src/app/strfry-db/"
relay {
bind = "0.0.0.0"
port = 7777
nofiles = 0
info {
name = "FIPS NAT Lab Relay"
description = "Local relay for Docker NAT traversal tests."
pubkey = ""
contact = ""
}
maxWebsocketPayloadSize = 131072
autoPingSeconds = 30
enableTcpNoDelay = true
rejectFutureEventsSeconds = 60
rejectEphemeralEventsOlderThanSeconds = 60
rejectEventsNewerThanSeconds = 60
maxFilterLimit = 500
maxSubsPerConnection = 50
writePolicy {
plugin = ""
}
compression {
enabled = true
slidingWindow = true
}
}
+11
View File
@@ -0,0 +1,11 @@
FROM debian:trixie-slim
RUN apt-get update && \
apt-get install -y --no-install-recommends \
conntrack iproute2 iptables procps tcpdump netcat-openbsd && \
rm -rf /var/lib/apt/lists/*
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+134
View File
@@ -0,0 +1,134 @@
#!/bin/bash
set -euo pipefail
NAT_MODE="${NAT_MODE:-cone}"
LAN_HOST="${LAN_HOST:?LAN_HOST is required}"
LAN_SUBNET="${LAN_SUBNET:?LAN_SUBNET is required}"
WAN_SUBNET="${WAN_SUBNET:?WAN_SUBNET is required}"
TCP_FORWARD_PORTS="${TCP_FORWARD_PORTS:-8443}"
WAN_GATEWAY="${WAN_GATEWAY:-}"
find_iface_for_subnet() {
local subnet="$1"
while read -r idx iface _ cidr _; do
case "$cidr" in
${subnet%0/24}*)
echo "$iface"
return 0
;;
esac
done < <(ip -o -4 addr show)
}
wait_for_iface() {
local if_name="$1"
local timeout_secs="${2:-30}"
local deadline=$((SECONDS + timeout_secs))
while [ "$SECONDS" -lt "$deadline" ]; do
if ip link show dev "$if_name" >/dev/null 2>&1; then
return 0
fi
sleep 0.5
done
return 1
}
wait_for_subnet_iface() {
local subnet="$1"
local timeout_secs="${2:-30}"
local deadline=$((SECONDS + timeout_secs))
local iface=""
while [ "$SECONDS" -lt "$deadline" ]; do
iface="$(find_iface_for_subnet "$subnet" || true)"
if [ -n "$iface" ]; then
echo "$iface"
return 0
fi
sleep 0.5
done
return 1
}
if [ -n "${LAN_IF:-}" ]; then
wait_for_iface "$LAN_IF"
else
LAN_IF="$(wait_for_subnet_iface "$LAN_SUBNET")"
fi
if [ -n "${WAN_IF:-}" ]; then
wait_for_iface "$WAN_IF"
else
WAN_IF="$(wait_for_subnet_iface "$WAN_SUBNET")"
fi
if [ -z "${LAN_IF:-}" ] || [ -z "${WAN_IF:-}" ]; then
echo "Failed to detect LAN/WAN interfaces"
ip -o -4 addr show
exit 1
fi
if [ -z "$WAN_GATEWAY" ]; then
WAN_GATEWAY="$(echo "$WAN_SUBNET" | awk -F. '{print $1 "." $2 "." $3 ".1"}')"
fi
WAN_ADDR="$(ip -o -4 addr show dev "$WAN_IF" | awk '{print $4}' | cut -d/ -f1 | head -1)"
if [ -z "$WAN_ADDR" ]; then
echo "Failed to determine WAN IPv4 address for ${WAN_IF}"
ip -o -4 addr show dev "$WAN_IF" || true
exit 1
fi
sysctl -w net.ipv4.ip_forward=1 >/dev/null || true
sysctl -w net.ipv4.conf.all.rp_filter=0 >/dev/null || true
sysctl -w "net.ipv4.conf.${LAN_IF}.rp_filter=0" >/dev/null || true
sysctl -w "net.ipv4.conf.${WAN_IF}.rp_filter=0" >/dev/null || true
ip route replace default via "$WAN_GATEWAY" dev "$WAN_IF"
iptables -F
iptables -t nat -F
iptables -P FORWARD DROP
iptables -A FORWARD -i "$LAN_IF" -o "$WAN_IF" -j ACCEPT
iptables -A FORWARD -i "$WAN_IF" -o "$LAN_IF" -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
for port in $TCP_FORWARD_PORTS; do
iptables -t nat -A PREROUTING -i "$WAN_IF" -p tcp --dport "$port" \
-j DNAT --to-destination "${LAN_HOST}:8443"
iptables -A FORWARD -i "$WAN_IF" -o "$LAN_IF" -p tcp -d "$LAN_HOST" --dport 8443 -j ACCEPT
done
case "$NAT_MODE" in
cone)
# Full-cone emulation for the single LAN node in this harness:
# preserve the UDP source port on egress and forward any inbound UDP
# on the WAN address back to the lone LAN host on the same port.
iptables -t nat -A PREROUTING -i "$WAN_IF" -p udp -j DNAT --to-destination "$LAN_HOST"
iptables -A FORWARD -i "$WAN_IF" -o "$LAN_IF" -p udp -d "$LAN_HOST" -j ACCEPT
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" -p udp \
-j SNAT --to-source "$WAN_ADDR"
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" ! -p udp -j MASQUERADE
;;
symmetric)
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" -p udp \
-j MASQUERADE --random-fully
iptables -t nat -A POSTROUTING -s "$LAN_SUBNET" -o "$WAN_IF" ! -p udp -j MASQUERADE
;;
*)
echo "Unknown NAT_MODE: $NAT_MODE"
exit 1
;;
esac
echo "Router ready: mode=${NAT_MODE} lan_if=${LAN_IF} wan_if=${WAN_IF}"
ip route
iptables -S
iptables -t nat -S
exec tail -f /dev/null
+138
View File
@@ -0,0 +1,138 @@
#!/bin/bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
NAT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$NAT_DIR/../.." && pwd)"
DERIVE_KEYS="$ROOT_DIR/testing/lib/derive_keys.py"
OUTPUT_DIR="$NAT_DIR/generated-configs"
SCENARIO="${1:?usage: generate-configs.sh <cone|symmetric|lan> [mesh-name]}"
MESH_NAME="${2:-nat-lab-$(date +%s)-$$}"
case "$SCENARIO" in
cone|symmetric|lan) ;;
*)
echo "Unknown scenario: $SCENARIO" >&2
exit 1
;;
esac
mkdir -p "$OUTPUT_DIR/$SCENARIO"
keys_a="$(python3 "$DERIVE_KEYS" "$MESH_NAME" "a")"
keys_b="$(python3 "$DERIVE_KEYS" "$MESH_NAME" "b")"
nsec_a="$(echo "$keys_a" | awk -F= '/^nsec=/{print $2}')"
npub_a="$(echo "$keys_a" | awk -F= '/^npub=/{print $2}')"
nsec_b="$(echo "$keys_b" | awk -F= '/^nsec=/{print $2}')"
npub_b="$(echo "$keys_b" | awk -F= '/^npub=/{print $2}')"
relay_addr="ws://172.31.254.30:7777"
stun_addr="stun:172.31.254.40:3478"
if [ "$SCENARIO" = "lan" ]; then
relay_addr="ws://172.31.10.30:7777"
stun_addr="stun:172.31.10.40:3478"
fi
peer_block_a=$(cat <<EOF
- npub: "$npub_b"
alias: "node-b"
addresses:
- transport: udp
addr: "nat"
priority: 1
EOF
)
peer_block_b=$(cat <<EOF
- npub: "$npub_a"
alias: "node-a"
addresses:
- transport: udp
addr: "nat"
priority: 1
EOF
)
if [ "$SCENARIO" = "symmetric" ]; then
peer_block_a="$peer_block_a"$'\n'" - transport: tcp
addr: \"172.31.254.11:8443\"
priority: 20"
peer_block_b="$peer_block_b"$'\n'" - transport: tcp
addr: \"172.31.254.10:8443\"
priority: 20"
fi
write_config() {
local output_file="$1"
local nsec="$2"
local peer_block="$3"
cat > "$output_file" <<EOF
node:
identity:
nsec: "$nsec"
retry:
max_retries: 3
base_interval_secs: 2
max_backoff_secs: 8
discovery:
nostr:
enabled: true
advertise: true
app: "fips.nat.lab.v1"
advert_relays:
- "$relay_addr"
dm_relays:
- "$relay_addr"
stun_servers:
- "$stun_addr"
signal_ttl_secs: 30
attempt_timeout_secs: 6
replay_window_secs: 60
punch_start_delay_ms: 500
punch_interval_ms: 100
punch_duration_ms: 2500
advert_ttl_secs: 60
advert_refresh_secs: 20
tun:
enabled: true
name: fips0
mtu: 1280
dns:
enabled: true
bind_addr: "127.0.0.1"
port: 5354
transports:
udp:
bind_addr: "0.0.0.0:2121"
mtu: 1472
advertise_on_nostr: true
public: false
tcp:
bind_addr: "0.0.0.0:8443"
peers:
$peer_block
connect_policy: auto_connect
auto_reconnect: true
EOF
}
write_config "$OUTPUT_DIR/$SCENARIO/node-a.yaml" "$nsec_a" "$peer_block_a"
write_config "$OUTPUT_DIR/$SCENARIO/node-b.yaml" "$nsec_b" "$peer_block_b"
cat > "$OUTPUT_DIR/$SCENARIO/npubs.env" <<EOF
NPUB_A=$npub_a
NPUB_B=$npub_b
MESH_NAME=$MESH_NAME
SCENARIO=$SCENARIO
EOF
echo "Generated NAT lab configs for scenario=$SCENARIO mesh=$MESH_NAME"
echo "NPUB_A=$npub_a"
echo "NPUB_B=$npub_b"
+418
View File
@@ -0,0 +1,418 @@
#!/bin/bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
NAT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$NAT_DIR/../.." && pwd)"
BUILD_SCRIPT="$ROOT_DIR/testing/scripts/build.sh"
GENERATE_SCRIPT="$SCRIPT_DIR/generate-configs.sh"
TOPOLOGY_SCRIPT="$SCRIPT_DIR/setup-topology.sh"
WAIT_LIB="$ROOT_DIR/testing/lib/wait-converge.sh"
SCENARIO="${1:-all}"
COMPOSE=(docker compose -f "$NAT_DIR/docker-compose.yml")
source "$WAIT_LIB"
cleanup() {
"${COMPOSE[@]}" --profile cone --profile symmetric --profile lan \
down -v --remove-orphans >/dev/null 2>&1 || true
}
helper_tcpdump_image() {
docker inspect -f '{{.Config.Image}}' fips-nat-router-a 2>/dev/null || echo nat-nat-a
}
dump_container_state() {
local container="$1"
echo ""
echo "--- $container: logs (last 80) ---"
docker logs "$container" 2>&1 | tail -80 || true
}
send_stun_probe() {
local container="$1"
local stun_host="$2"
local stun_port="$3"
docker exec "$container" python3 - "$stun_host" "$stun_port" <<'PY' 2>&1 || true
import os
import socket
import struct
import sys
host = sys.argv[1]
port = int(sys.argv[2])
txn_id = os.urandom(12)
request = struct.pack("!HHI", 0x0001, 0, 0x2112A442) + txn_id
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(2.0)
sock.sendto(request, (host, port))
try:
data, remote = sock.recvfrom(2048)
except socket.timeout:
print(f"stun timeout waiting for {host}:{port}")
raise SystemExit(1)
if len(data) < 20:
print(f"short stun response from {remote}: {len(data)} bytes")
raise SystemExit(1)
msg_type, msg_len, cookie = struct.unpack("!HHI", data[:8])
if msg_type != 0x0101 or cookie != 0x2112A442 or data[8:20] != txn_id:
print(f"unexpected stun response from {remote}: type=0x{msg_type:04x} len={msg_len} cookie=0x{cookie:08x}")
raise SystemExit(1)
print(f"stun binding success from {remote[0]}:{remote[1]}")
PY
}
dump_fips_state() {
local container="$1"
local relay_host="${2:-172.31.254.30}"
local relay_port="${3:-7777}"
local stun_host="${4:-172.31.254.40}"
local stun_port="${5:-3478}"
dump_container_state "$container"
echo ""
echo "--- $container: UDP sockets ---"
docker exec "$container" sh -lc 'ss -H -uanp 2>/dev/null || ss -H -uan 2>/dev/null || netstat -anu 2>/dev/null' 2>&1 || true
echo ""
echo "--- $container: fipsctl show status ---"
docker exec "$container" fipsctl show status 2>&1 || true
echo ""
echo "--- $container: fipsctl show peers ---"
docker exec "$container" fipsctl show peers 2>&1 || true
echo ""
echo "--- $container: fipsctl show links ---"
docker exec "$container" fipsctl show links 2>&1 || true
echo ""
echo "--- $container: relay reachability ---"
docker exec "$container" sh -lc "nc -vz -w5 ${relay_host} ${relay_port}" 2>&1 || true
echo ""
echo "--- $container: stun reachability ---"
send_stun_probe "$container" "$stun_host" "$stun_port"
}
dump_node_udp_probe() {
local node="$1"
local stun_host="${2:-172.31.254.40}"
local stun_port="${3:-3478}"
echo ""
echo "--- $node: UDP sockets (pre-capture) ---"
docker exec "$node" sh -lc 'ss -H -uanp 2>/dev/null || ss -H -uan 2>/dev/null || netstat -anu 2>/dev/null' 2>&1 || true
echo ""
echo "--- $node: UDP routes to STUN and peer WANs ---"
docker exec "$node" sh -lc 'for ip in 172.31.254.40 172.31.254.10 172.31.254.11; do ip route get "$ip"; done' 2>&1 || true
local capture_file
capture_file="$(mktemp)"
docker exec "$node" sh -lc "timeout 8 tcpdump -ni eth0 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
local tcpdump_pid=$!
sleep 1
echo ""
echo "--- $node: UDP active probe ---"
echo "probe: ${node} -> ${stun_host}:${stun_port}/udp (STUN binding request)"
send_stun_probe "$node" "$stun_host" "$stun_port"
wait "$tcpdump_pid" || true
echo ""
echo "--- $node: UDP tcpdump during active probe ---"
cat "$capture_file"
rm -f "$capture_file"
echo ""
echo "--- $node: UDP sockets (post-capture) ---"
docker exec "$node" sh -lc 'ss -H -uanp 2>/dev/null || ss -H -uan 2>/dev/null || netstat -anu 2>/dev/null' 2>&1 || true
}
dump_router_udp_probe() {
local router="$1"
local source_node="$2"
local stun_host="${3:-172.31.254.40}"
local stun_port="${4:-3478}"
echo ""
echo "--- $router: UDP conntrack/state (before probe) ---"
docker exec "$router" sh -lc 'conntrack -L -p udp 2>/dev/null || echo "conntrack unavailable"' 2>&1 || true
echo ""
echo "--- $router: UDP counters (before probe) ---"
docker exec "$router" sh -lc 'iptables -vnL FORWARD; echo; iptables -t nat -vnL POSTROUTING' 2>&1 || true
echo ""
echo "--- $router: UDP routes to STUN and peer WANs ---"
docker exec "$router" sh -lc 'for ip in 172.31.254.40 172.31.254.10 172.31.254.11; do ip route get "$ip"; done' 2>&1 || true
local capture_file
capture_file="$(mktemp)"
docker exec "$router" sh -lc "timeout 8 tcpdump -ni any 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
local tcpdump_pid=$!
sleep 1
echo ""
echo "--- $router: UDP active probe ---"
echo "probe: ${source_node} -> ${stun_host}:${stun_port}/udp (STUN binding request)"
send_stun_probe "$source_node" "$stun_host" "$stun_port"
wait "$tcpdump_pid" || true
echo ""
echo "--- $router: UDP tcpdump during active probe ---"
cat "$capture_file"
rm -f "$capture_file"
echo ""
echo "--- $router: UDP counters (after probe) ---"
docker exec "$router" sh -lc 'iptables -vnL FORWARD; echo; iptables -t nat -vnL POSTROUTING' 2>&1 || true
echo ""
echo "--- $router: UDP conntrack/state (after probe) ---"
docker exec "$router" sh -lc 'conntrack -L -p udp 2>/dev/null || echo "conntrack unavailable"' 2>&1 || true
}
dump_stun_udp_probe() {
local source_node="$1"
local stun_host="${2:-172.31.254.40}"
local stun_port="${3:-3478}"
local helper_image
helper_image="$(helper_tcpdump_image)"
local capture_file
capture_file="$(mktemp)"
docker run --rm --net=container:fips-nat-stun --cap-add NET_ADMIN --cap-add NET_RAW \
--entrypoint sh "$helper_image" \
-lc "timeout 8 tcpdump -ni any 'udp and not port 53' -c 80" \
>"$capture_file" 2>&1 &
local tcpdump_pid=$!
sleep 1
echo ""
echo "--- fips-nat-stun: UDP active probe ---"
echo "probe: ${source_node} -> ${stun_host}:${stun_port}/udp (STUN binding request)"
send_stun_probe "$source_node" "$stun_host" "$stun_port"
wait "$tcpdump_pid" || true
echo ""
echo "--- fips-nat-stun: UDP tcpdump during active probe ---"
cat "$capture_file"
rm -f "$capture_file"
}
dump_cone_diagnostics() {
echo ""
echo "=== cone diagnostics ==="
dump_fips_state fips-nat-cone-a 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-a
dump_fips_state fips-nat-cone-b 172.31.254.30 7777 172.31.254.40 3478
dump_node_udp_probe fips-nat-cone-b
dump_container_state fips-nat-router-a
dump_router_udp_probe fips-nat-router-a fips-nat-cone-a
dump_container_state fips-nat-router-b
dump_router_udp_probe fips-nat-router-b fips-nat-cone-b
dump_container_state fips-nat-relay
dump_stun_udp_probe fips-nat-cone-a
dump_stun_udp_probe fips-nat-cone-b
dump_container_state fips-nat-stun
}
dump_symmetric_diagnostics() {
echo ""
echo "=== symmetric diagnostics ==="
dump_fips_state fips-nat-symmetric-a 172.31.254.30 7777 172.31.254.40 3478
dump_fips_state fips-nat-symmetric-b 172.31.254.30 7777 172.31.254.40 3478
dump_container_state fips-nat-router-a
dump_container_state fips-nat-router-b
dump_container_state fips-nat-relay
dump_container_state fips-nat-stun
}
dump_lan_diagnostics() {
echo ""
echo "=== lan diagnostics ==="
dump_fips_state fips-nat-lan-a 172.31.10.30 7777 172.31.10.40 3478
dump_fips_state fips-nat-lan-b 172.31.10.30 7777 172.31.10.40 3478
dump_container_state fips-nat-relay
dump_container_state fips-nat-stun
}
trap 'echo ""; echo "NAT test interrupted"; cleanup; exit 130' INT TERM
require_test_image() {
if ! docker image inspect fips-test:latest >/dev/null 2>&1; then
echo "fips-test:latest not found; building test image"
"$BUILD_SCRIPT"
fi
}
require_docker_daemon() {
if ! docker info >/dev/null 2>&1; then
echo "Docker daemon is not reachable; cannot run NAT lab harness" >&2
exit 1
fi
}
assert_peer_path() {
local container="$1"
local expected_transport="$2"
local expected_prefix="$3"
docker exec "$container" fipsctl show peers \
| python3 -c "
import json, sys
data = json.load(sys.stdin)
peers = [p for p in data.get('peers', []) if p.get('connectivity') == 'connected']
if not peers:
raise SystemExit(1)
peer = peers[0]
transport = peer.get('transport_type', '')
addr = peer.get('transport_addr', '')
if transport != sys.argv[1]:
raise SystemExit(f'transport mismatch: expected {sys.argv[1]!r}, got {transport!r}')
if not addr.startswith(sys.argv[2]):
raise SystemExit(f'addr mismatch: expected prefix {sys.argv[2]!r}, got {addr!r}')
" "$expected_transport" "$expected_prefix"
}
assert_link_path() {
local container="$1"
local expected_prefix="$2"
docker exec "$container" fipsctl show links \
| python3 -c "
import json, sys
data = json.load(sys.stdin)
links = data.get('links', [])
if not links:
raise SystemExit(1)
addr = links[0].get('remote_addr', '')
if not addr.startswith(sys.argv[1]):
raise SystemExit(f'link addr mismatch: expected prefix {sys.argv[1]!r}, got {addr!r}')
" "$expected_prefix"
}
require_bootstrap_activity() {
local container="$1"
local logs
logs="$(docker logs "$container" 2>&1 || true)"
if ! grep -Eq "bootstrap failed|Started Nostr( UDP)? NAT traversal attempt" <<<"$logs"; then
echo "Expected bootstrap activity in ${container} logs" >&2
return 1
fi
}
ping_peer() {
local container="$1"
local npub="$2"
docker exec "$container" ping6 -c 3 -W 5 "${npub}.fips" >/dev/null
}
run_cone() {
echo "=== NAT lab: cone ==="
cleanup
"$GENERATE_SCRIPT" cone
"${COMPOSE[@]}" --profile cone up -d --build --force-recreate
"$TOPOLOGY_SCRIPT" cone
wait_for_peers fips-nat-cone-a 1 45 || {
dump_cone_diagnostics
return 1
}
wait_for_peers fips-nat-cone-b 1 45 || {
dump_cone_diagnostics
return 1
}
assert_peer_path fips-nat-cone-a udp 172.31.254.
assert_peer_path fips-nat-cone-b udp 172.31.254.
assert_link_path fips-nat-cone-a 172.31.254.
assert_link_path fips-nat-cone-b 172.31.254.
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/cone/npubs.env"
ping_peer fips-nat-cone-a "$NPUB_B"
ping_peer fips-nat-cone-b "$NPUB_A"
cleanup
}
run_symmetric() {
echo "=== NAT lab: symmetric fallback ==="
cleanup
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "$GENERATE_SCRIPT" symmetric
NAT_MODE_A=symmetric NAT_MODE_B=symmetric "${COMPOSE[@]}" --profile symmetric up -d --build --force-recreate
"$TOPOLOGY_SCRIPT" symmetric
wait_for_peers fips-nat-symmetric-a 1 60 || {
dump_symmetric_diagnostics
return 1
}
wait_for_peers fips-nat-symmetric-b 1 60 || {
dump_symmetric_diagnostics
return 1
}
assert_peer_path fips-nat-symmetric-a tcp 172.31.254.11:
assert_peer_path fips-nat-symmetric-b tcp 172.31.254.10:
assert_link_path fips-nat-symmetric-a 172.31.254.11:
assert_link_path fips-nat-symmetric-b 172.31.254.10:
require_bootstrap_activity fips-nat-symmetric-a
require_bootstrap_activity fips-nat-symmetric-b
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/symmetric/npubs.env"
ping_peer fips-nat-symmetric-a "$NPUB_B"
ping_peer fips-nat-symmetric-b "$NPUB_A"
cleanup
}
run_lan() {
echo "=== NAT lab: lan preference ==="
cleanup
"$GENERATE_SCRIPT" lan
"${COMPOSE[@]}" --profile lan up -d --build --force-recreate
wait_for_peers fips-nat-lan-a 1 45 || {
dump_lan_diagnostics
return 1
}
wait_for_peers fips-nat-lan-b 1 45 || {
dump_lan_diagnostics
return 1
}
assert_peer_path fips-nat-lan-a udp 172.31.10.
assert_peer_path fips-nat-lan-b udp 172.31.10.
assert_link_path fips-nat-lan-a 172.31.10.
assert_link_path fips-nat-lan-b 172.31.10.
# shellcheck disable=SC1090
source "$NAT_DIR/generated-configs/lan/npubs.env"
ping_peer fips-nat-lan-a "$NPUB_B"
ping_peer fips-nat-lan-b "$NPUB_A"
cleanup
}
main() {
require_docker_daemon
require_test_image
case "$SCENARIO" in
all)
run_cone
run_symmetric
run_lan
;;
cone)
run_cone
;;
symmetric)
run_symmetric
;;
lan)
run_lan
;;
*)
echo "Usage: $0 [all|cone|symmetric|lan]" >&2
exit 1
;;
esac
echo "NAT lab scenarios passed"
}
main "$@"
+131
View File
@@ -0,0 +1,131 @@
#!/bin/bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
NAT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
SCENARIO="${1:?usage: setup-topology.sh <cone|symmetric>}"
case "$SCENARIO" in
cone)
node_a="fips-nat-cone-a"
node_b="fips-nat-cone-b"
;;
symmetric)
node_a="fips-nat-symmetric-a"
node_b="fips-nat-symmetric-b"
;;
*)
echo "Unsupported topology scenario: $SCENARIO" >&2
exit 1
;;
esac
router_a="fips-nat-router-a"
router_b="fips-nat-router-b"
helper_image() {
if [ -n "${IP_HELPER_IMAGE:-}" ]; then
echo "$IP_HELPER_IMAGE"
return 0
fi
docker inspect -f '{{.Config.Image}}' "$router_a"
}
wait_for_pid() {
local container="$1"
local timeout_secs="${2:-30}"
local deadline=$((SECONDS + timeout_secs))
local pid=""
while [ "$SECONDS" -lt "$deadline" ]; do
pid="$(docker inspect -f '{{.State.Pid}}' "$container" 2>/dev/null || true)"
if [[ "$pid" =~ ^[0-9]+$ ]] && [ "$pid" -gt 0 ]; then
echo "$pid"
return 0
fi
sleep 0.5
done
echo "Timed out waiting for container PID: $container" >&2
return 1
}
run_host_ip() {
local image="$1"
shift
docker run --rm \
--privileged \
--net=host \
--pid=host \
--entrypoint ip \
"$image" \
"$@"
}
configure_node_iface() {
local container="$1"
local current_name="$2"
local final_name="$3"
local cidr="$4"
docker exec "$container" sh -lc "
ip link set lo up &&
ip link set '$current_name' name '$final_name' &&
ip addr flush dev '$final_name' &&
ip addr add '$cidr' dev '$final_name' &&
ip link set '$final_name' up
"
}
configure_router_iface() {
local container="$1"
local current_name="$2"
local final_name="$3"
local cidr="$4"
docker exec "$container" sh -lc "
ip link set lo up &&
ip link set '$current_name' name '$final_name' &&
ip addr flush dev '$final_name' &&
ip addr add '$cidr' dev '$final_name' &&
ip link set '$final_name' up
"
}
setup_pair() {
local image="$1"
local node_container="$2"
local router_container="$3"
local host_node="$4"
local host_router="$5"
local node_cidr="$6"
local router_cidr="$7"
local node_pid router_pid
node_pid="$(wait_for_pid "$node_container")"
router_pid="$(wait_for_pid "$router_container")"
run_host_ip "$image" link delete "$host_node" >/dev/null 2>&1 || true
run_host_ip "$image" link delete "$host_router" >/dev/null 2>&1 || true
run_host_ip "$image" link add "$host_node" type veth peer name "$host_router"
run_host_ip "$image" link set "$host_node" netns "$node_pid"
run_host_ip "$image" link set "$host_router" netns "$router_pid"
configure_node_iface "$node_container" "$host_node" eth0 "$node_cidr"
configure_router_iface "$router_container" "$host_router" eth1 "$router_cidr"
}
main() {
cd "$NAT_DIR"
local image
image="$(helper_image)"
setup_pair "$image" "$node_a" "$router_a" vna0 vna1 172.31.1.10/24 172.31.1.254/24
setup_pair "$image" "$node_b" "$router_b" vnb0 vnb1 172.31.2.10/24 172.31.2.254/24
}
main "$@"
+8
View File
@@ -0,0 +1,8 @@
FROM python:3.12-slim
WORKDIR /app
COPY stun_server.py /app/stun_server.py
EXPOSE 3478/udp
CMD ["python3", "/app/stun_server.py"]
+37
View File
@@ -0,0 +1,37 @@
import socket
import struct
MAGIC_COOKIE = 0x2112A442
STUN_BINDING_REQUEST = 0x0001
STUN_BINDING_SUCCESS = 0x0101
STUN_ATTR_XOR_MAPPED_ADDRESS = 0x0020
def build_success(txn_id: bytes, addr: tuple[str, int]) -> bytes:
ip_bytes = socket.inet_aton(addr[0])
cookie_bytes = MAGIC_COOKIE.to_bytes(4, "big")
x_port = addr[1] ^ (MAGIC_COOKIE >> 16)
x_ip = bytes(ip_bytes[i] ^ cookie_bytes[i] for i in range(4))
value = b"\x00\x01" + struct.pack("!H", x_port) + x_ip
attr = struct.pack("!HH", STUN_ATTR_XOR_MAPPED_ADDRESS, len(value)) + value
header = struct.pack("!HHI", STUN_BINDING_SUCCESS, len(attr), MAGIC_COOKIE) + txn_id
return header + attr
def main() -> None:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("0.0.0.0", 3478))
while True:
data, remote = sock.recvfrom(2048)
if len(data) < 20:
continue
msg_type, msg_len, cookie = struct.unpack("!HHI", data[:8])
txn_id = data[8:20]
if msg_type != STUN_BINDING_REQUEST or msg_len != 0 or cookie != MAGIC_COOKIE:
continue
sock.sendto(build_success(txn_id, remote), remote)
if __name__ == "__main__":
main()
+11 -2
View File
@@ -19,6 +19,15 @@ if [ ! -f "$PROJECT_ROOT/Cargo.toml" ]; then
fi
BUILD_DOCKER=true
# NAT harness binaries need Nostr bootstrap support; everything else is
# governed by platform cfg gates after PR #79's feature-matrix collapse.
DEFAULT_CARGO_BUILD_ARGS=(--features nostr-discovery)
if [ -n "${FIPS_CARGO_BUILD_ARGS:-}" ]; then
# shellcheck disable=SC2206
CARGO_BUILD_ARGS=($FIPS_CARGO_BUILD_ARGS)
else
CARGO_BUILD_ARGS=("${DEFAULT_CARGO_BUILD_ARGS[@]}")
fi
while [ $# -gt 0 ]; do
case "$1" in
--no-docker) BUILD_DOCKER=false; shift ;;
@@ -45,12 +54,12 @@ if [ "$UNAME_S" = "Darwin" ]; then
fi
echo "Building FIPS for Linux (release) using cargo-zigbuild..."
cargo zigbuild --release --target "$CARGO_TARGET" --manifest-path="$PROJECT_ROOT/Cargo.toml"
cargo zigbuild --release --target "$CARGO_TARGET" --manifest-path="$PROJECT_ROOT/Cargo.toml" "${CARGO_BUILD_ARGS[@]}"
TARGET_DIR="$PROJECT_ROOT/target/$CARGO_TARGET/release"
else
echo "Building FIPS (release)..."
cargo build --release --manifest-path="$PROJECT_ROOT/Cargo.toml"
cargo build --release --manifest-path="$PROJECT_ROOT/Cargo.toml" "${CARGO_BUILD_ARGS[@]}"
TARGET_DIR="$PROJECT_ROOT/target/release"
fi