mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 00:04:54 +00:00
f478f51afed7bc36b3fe2bb20d797c0ba0eea4e9
21
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
ebf34ae712 |
Read the wall clock directly for Nostr traversal timestamps
The traversal clock cached a Unix millisecond value and an Instant at first use, then served every later call by advancing the cached value with the monotonic elapsed time. A monotonic clock does not tick while the host is suspended, so once a machine had slept the returned value trailed real time by the sleep duration for the rest of the process lifetime, and never re-synced. Almost everything that clock feeds is an absolute timestamp. The NIP-40 expiration tags on adverts and traversal signals, and the issuedAt and expiresAt fields of offers and answers, are all computed as now plus a TTL; the freshness and cache-pruning paths compare it against a peer-authored, signed created_at. Once the host had slept longer than signal_ttl_secs, every offer was published already expired, relays dropped it, and the initiator timed out waiting for an answer with traversal broken until a restart. Read the wall clock on every call instead. This also removes a mismatch inside the traversal failure-state map, which was written here from the cached clock but written and read from the node lifecycle with the real one. Not platform-specific: monotonic clocks exclude suspended time on Linux and Windows as well, so this affected any host that suspends. A laptop is simply where a process lives long enough across a sleep to notice. The interval-shaped consumers hold up under a clock step. A forward step, which is what a resume produces, saturates the punch start delay to zero, and the attempt's own bounds are monotonic deadlines that are unaffected. A backward step lengthens that delay and can cost one punch attempt, which retries. Early eviction from the replay window cannot admit a replay under the shipped defaults, because the freshness window a replayed offer must also satisfy is strictly narrower than the replay window itself. The added test pins the contract and fires on a host that has genuinely suspended, but it is not a regression guard for this defect: nothing reachable from a unit test can simulate a suspend, so on a machine that has not slept the old implementation passes it too. Its comment says so rather than leaving a false sense of coverage. Reported in https://github.com/jmcorgan/fips/issues/128 |
||
|
Johnathan Corgan
|
3bc8e5611c | Merge branch 'maint' | ||
|
Johnathan Corgan
|
0ce9bb5b99 |
discovery/nostr: elect a single traversal session for dual-auto_connect peers
When two peers each auto_connect to the other, each runs both an initiator and a responder NAT-traversal session and binds a separate UDP socket per session. Each side adopts only the first Established event and drops the loser session's socket; when the two sides adopt mismatched sessions, each sends its Noise msg1 to a peer port the peer has already stopped draining, and both handshakes stall. Deterministically keep the session initiated by the smaller NodeAddr, decided on the responder path: decline an incoming offer only when we also have an in-flight outbound initiator for the same peer and our NodeAddr is smaller. The peer's redundant initiator then times out, leaving a single matching socket pair on both ends. Asymmetric (one-sided) auto_connect has no co-active initiator and is never suppressed, so connectivity is preserved; an undecidable NodeAddr falls through to answering. Reuses the NodeAddr tie-breaker convention already used by the cross-connection and rekey dual-init paths. Adds a unit test for the election helper. |
||
 Martti MalmiandJohnathan Corgan
|
7d7b551ca1 |
discovery: add opt-in mDNS LAN discovery
Add scoped mDNS / DNS-SD discovery for peers on the same local link, giving sub-second pairing without a relay or NAT-traversal roundtrip. A node advertises its npub, protocol version, and an optional network scope over link-local multicast, and browses for matching adverts to initiate Noise handshakes against same-LAN peers. LAN discovery is disabled by default; operators enable it with node.discovery.lan.enabled: true. Default-off avoids reintroducing a per-LAN identity broadcast on nodes that have deliberately disabled other discovery channels, and avoids any multicast surprise on upgrade. The startup advertised-port picker now excludes bootstrap transports and selects a non-bootstrap operational UDP transport with a stable lowest-id selector, so the advertised port is deterministic across restarts rather than dependent on HashMap iteration order. This matches the per-dial transport selection used for discovered peers. Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com> |
||
|
Johnathan Corgan
|
6991a152e6 |
Merge maint into master (outbound admission gate + mesh-size parent skip)
Brings two structural fixes landed on maint: - compute_mesh_size: explicit parent skip in the children loop, so the disjoint-subtree invariant no longer depends on peer_declaration cache freshness. - max_peers: outbound connection-initiation gated on the cap (auto-reconnect retries, Nostr-mediated discovery established adoption, and both sides of the NAT-traversal punch sequence). Inbound msg1 admission gate unchanged. |
||
|
Johnathan Corgan
|
d4687e5d30 |
node: gate outbound connection initiation on max_peers
node.limits.max_peers was honored only on inbound msg1 admission (handshake.rs handle_msg1 returns PeerLimitExceeded when peers.len is at the cap). Four outbound initiation paths proceeded unconditionally at capacity: auto-reconnect retries (process_pending_retries), Nostr-mediated discovery's BootstrapEvent::Established adoption (poll_nostr_discovery), NAT-traversal punch initiation (the outgoing side of the offer/answer/punch sequence in the Nostr discovery runtime), and NAT-traversal punch response (the incoming side of the same sequence). A saturated node burned CPU, UDP probes, STUN observations, and Nostr relay traffic on connections that the inbound gate would reject the moment they reached msg1. Introduce Node::outbound_admission_check (peers.len < max_peers, or true when max_peers == 0 as the no-cap sentinel) and gate the four paths. The discovery runtime lives in a separate task and does not hold a Node reference; bridge via an Arc<AtomicBool> the runtime reads and Node refreshes once per tick from outbound_admission_check. The atomic granularity is intentionally loose: one-tick lag is acceptable because the inbound msg1 gate continues to be the authoritative cap, and in-flight handshakes started below the cap are allowed to complete. Inbound gate at handshake.rs is unchanged. |
||
|
Johnathan Corgan
|
6e7e44c8ff | Merge maint into master (advert filter, PR-REVIEW) | ||
|
Martti MalmiandJohnathan Corgan
|
d418106034 | nostr: filter unroutable direct advert endpoints | ||
|
Martti MalmiandJohnathan Corgan
|
c0ccedb491 | nostr: start discovery without blocking node startup | ||
|
Martti MalmiandJohnathan Corgan
|
87d1af0269 |
nostr: ignore stale traversal for active peers
Skip BootstrapEvent::Established and BootstrapEvent::Failed dispatch in poll_nostr_discovery for peers that are already connected or actively handshaking. Without these guards, stale traversal events arriving after a peer connected through a different path would either attempt to adopt a redundant socket against the live connection (Established) or poison the per-peer failure-state cooldown and trigger redundant retraversal via schedule_retry / try_peer_addresses (Failed). The four guard sites use a new is_connecting_to_peer helper extracted from the existing closure inside initiate_peer_connection; the helper checks for an in-flight outbound handshake state. adopt_established_traversal gains a defense-in-depth check returning PeerAlreadyExists when called against an already-promoted peer, so the invariant holds if a future caller bypasses the outer dispatch guard. Side benefit: narrows a cooldown-poisoning vector previously available to an attacker injecting stale failure events for an active peer. Test coverage for the new behavior: - test_try_peer_addresses_skips_connected_peer - test_try_peer_addresses_skips_connecting_peer - test_nostr_traversal_failure_skips_connected_peer (Failed-arm event injection) - test_nostr_traversal_established_skips_connected_peer (Established-arm event injection, mirror of the Failed test) - test_adopted_traversal_skips_already_connected_peer (adopt_established_traversal defense-in-depth) CHANGELOG entry under [Unreleased] / Fixed. Closes #87 |
||
|
Johnathan Corgan
|
8094a51a82 |
nostr: fix subscription startup race losing relay REQ replays
Freshly-restarted nodes with policy: open silently lost the historical
event replay that relays send in response to subscribe(). The
broadcast::Receiver was created INSIDE spawn_notify_loop, which the
tokio runtime starts at some indeterminate point after subscribe()
returns. tokio's broadcast channel only delivers messages sent after
the receiver is created; messages dispatched in the gap between
subscribe() issuing the REQ and the spawned task calling
client.notifications() were dropped by external_notification_sender.send
returning Err(SendError) with no subscribers attached.
Symptom on a node with policy: open: non-configured peers were not
discovered until they next re-published their advert (default
advert_refresh_secs = 1800s = 30 min). Configured peers were unaffected
because fetch_advert (relay-fetch path) caches them at startup-sweep
time. The bug has been latent since
|
||
|
Johnathan Corgan
|
a62a0a6cf4 |
nostr: suppress retraversal of cross-FMP-version peers
Open-discovery NAT traversal succeeds at the UDP layer regardless of what FMP-protocol version the peer speaks. When the daemon discovers a peer running a different FMP version (e.g. a v0/v1 mix during a mid-rollout window, or a misconfigured peer in the same advert namespace), the punch sequence completes, the socket is adopted via `Node::adopt_established_traversal`, and we initiate an FMP handshake. The peer drops our msg1 at its own version-gate and we drop their msg1/msg2 at `Unknown FMP version, dropping`. Neither side advances the handshake. Today the bootstrap transport sits idle until the 31s stale- handshake timeout, drops, and the open-discovery sweep ~30s later fires the full STUN+offer+answer+punch sequence again — every minute, indefinitely, against peers the handshake literally cannot complete with. Add a `Node::bootstrap_transport_npubs` map populated alongside `bootstrap_transports` at adopt time. The rx loop reverse-maps the transport_id → npub on version-mismatch and bumps the discovery layer's `failure_state` to a long structural cooldown via the new `NostrDiscovery::record_protocol_mismatch` API. The next sweep skips the npub for `protocol_mismatch_cooldown_secs` (default 86400 = 24h, separate from the 30-min transient-failure `extended_cooldown_secs`). One-shot WARN per fresh observation. Repeat mismatches inside the cooldown window are silent (the failure_state method returns false when an existing comparable cooldown is already in place). The handshake/transport teardown chain is unchanged — the fix is specifically about preventing the *next* sweep cycle from re-traversing. Cleared on `cleanup_bootstrap_transport_if_unused` and on the adopt-failure rollback path so completed handshakes don't leave stale entries behind. Four new unit tests in `failure_state.rs` cover fresh-entry signaling, repeat-suppression inside the window, streak-pin behavior for `show_peers` rendering, and post-cooldown re-arming. |
||
|
Johnathan Corgan
|
2f95929862 |
nostr: public-IP discovery for UDP/TCP advert publication
When a UDP transport had `advertise_on_nostr: true` + `public: true` + `bind_addr: 0.0.0.0:NNNN`, the advert builder previously read the kernel's `local_addr()`, found `0.0.0.0`, filtered it out (correctly — wildcard isn't a valid advertised endpoint), and silently emitted no UDP endpoint in the published advert. Operators on AWS EIP / GCP / Azure setups (where binding to the public IP directly is impossible because 1:1 NAT does the address translation off-host) had no way to advertise UDP without binding to a specific local IP — and no log explaining what was happening. TCP had the same shape, with no `public: true` precondition. Three pieces, layered. UDP gets zero-config autodiscovery via STUN; both UDP and TCP get an explicit operator-supplied override; the fall-through path now logs loudly instead of silently skipping. UDP public-IP autodiscovery (STUN) ---------------------------------- In the UDP `is_public()` + wildcard-bind branch, run a one-shot STUN observation against an ephemeral UDP socket on the daemon's configured `stun_servers`. Take the reflexive IPv4 (the STUN-reported port is the ephemeral source port and is discarded), combine with the configured listener port for the advert (`udp:<reflexive-ip>:<port>`). Works on AWS EIP / GCP / Azure 1:1-NAT setups because STUN sees the public-Internet egress IP and the bind port is preserved through 1:1 NAT. Result is cached per-transport on a new `public_udp_addr_cache` field on `NostrDiscovery` (keyed by `TransportId.as_u32()`). Asymmetric cache TTL: a successful observation is cached for `advert_refresh_secs` (default 30 min) so we don't STUN every refresh tick. A failed observation is cached for only 60s (`PUBLIC_UDP_ADDR_FAILURE_TTL`) so a transient STUN flake at startup retries within ~a minute and the advert grows its UDP endpoint as soon as STUN starts working — rather than waiting the full 30-min cycle. The shared `observe_traversal_addresses` STUN helper had a hard-coded 2s per-server response wait, right for the per-traversal flow (latency-sensitive — 3 STUN servers worst-case = 6s) but too short for the one-shot advert-publish startup discovery. Parameterized `per_server_timeout` on the helper, with two named constants in `stun.rs`: `TRAVERSAL_STUN_TIMEOUT = 2s` (existing call sites) and `ADVERT_STUN_TIMEOUT = 5s` (new public-UDP discovery path). Both use `tokio::time::timeout_at` under the hood, so success returns immediately — the timeout is only the worst case. `external_addr` override (UDP + TCP) ------------------------------------ New `external_addr: Option<String>` field on `transports.udp.*` and `transports.tcp.*` for explicit advertise-as override. Takes precedence over both the bound `local_addr` and (for UDP) the STUN-derived autodiscovery. Required for TCP on cloud-NAT setups (AWS EIP, GCP/Azure external IPs) where binding to the public IP directly fails with `EADDRNOTAVAIL` because the public IP isn't on a host interface — the network fabric does 1:1 NAT off-host. Without this field the operator's only TCP path was "leave advert off" or "find a way to make the public IP locally bindable." For UDP, `external_addr` is optional but useful as a deterministic alternative to STUN. Operators who want to skip STUN egress, whose STUN servers are blocked, or who want the daemon to not depend on external services for advert content can specify it explicitly. The accessor parses two shapes: - Bare IP (`"54.183.70.180"` or `"2001:db8::1"`): combines with the configured `bind_addr` port. - Full host:port (`"54.183.70.180:8443"` or `"[2001:db8::1]:443"`): used verbatim — useful for port-forward setups where the externally-visible port differs from the bind port. Final precedence in `Node::build_overlay_advert` (now async, only caller `refresh_overlay_advert` was already async): - UDP: `external_addr` → non-wildcard `local_addr` → STUN → loud warn - TCP: `external_addr` → non-wildcard `local_addr` → loud warn Loud warns instead of silent skips ---------------------------------- The wildcard-bind fall-through paths now log a `warn!` pointing at the operator-side fixes: - UDP: "set transports.udp.external_addr, bind to a specific public IP, or ensure node.discovery.nostr.stun_servers is reachable" - TCP: "Either set external_addr to the public IP (recommended for cloud 1:1-NAT setups) or bind explicitly to the public IP" Replaces the silent skip that previously cost operators a debugging session when the advert mysteriously contained only the Tor onion endpoint. Tests ----- 11 new unit tests in `src/config/transport.rs` covering the parser (IPv4/IPv6, bare/full, malformed) and the accessor (UDP with default bind, UDP with explicit port override, UDP unset, TCP without bind_addr, TCP with bind_addr, TCP with full socket-addr override, parse_bind_port for IPv4/IPv6/malformed). The 38-test nostr suite still passes. CHANGELOG entries under `[Unreleased]` Fixed. |
||
|
Johnathan Corgan
|
bcc9c525d3 |
nostr: per-peer NAT-traversal failure suppression and clock-skew handling
Public-test daemons with populous open-discovery caches generate
sustained NAT-traversal-failure WARN volume (~140/hour, ~3500/day)
against cache-learned peers that have gone offline — their adverts
are absent from major Nostr relays but cached entries persist until
their advertised `valid_until` expires. The daemon kept publishing
offers indefinitely under exponential backoff with no per-peer
suppression, drowning operator signal and hammering relays. A
parallel concern: the strict freshness check at signal.rs silently
rejected offers under modest clock skew (now_ms() anchors to
SystemTime once at startup, so post-startup NTP step adjustments
don't propagate on long-uptime daemons), indistinguishable from
"peer is offline."
Six independent improvements layered on the existing retry logic.
Per-npub WARN log rate-limit
----------------------------
New `FailureState` struct on `NostrDiscovery` records per-npub
`last_warn_at_ms`. Subsequent failures inside `warn_log_interval_secs`
(default 5 min) emit DEBUG instead of WARN. Each WARN now also
carries `consecutive_failures` and remaining `cooldown_secs` so
operators can read the trajectory without grepping multiple lines.
Per-npub consecutive-failure counter + extended cooldown
--------------------------------------------------------
After `failure_streak_threshold` (default 5) consecutive failures
against a peer, the next `extended_cooldown_secs` (default 1800)
of attempts are suppressed by pushing
`retry_pending[npub].retry_after_ms` past the cooldown wall. The
open-discovery sweep also consults `cooldown_until` and increments
a new `skipped_cooldown` counter so a peer whose `retry_pending`
was cleared by max_retries doesn't get re-enqueued during the
cooldown window. Caps offer-publish rate per dead peer regardless
of how often the sweep tries to re-enqueue.
Stale-advert eviction on streak-threshold transition
----------------------------------------------------
On the threshold-crossing transition (one-shot, not every
subsequent failure), `tokio::spawn` an active re-fetch of the
peer's Kind 37195 advert from `advert_relays`. Three outcomes:
- absent on relays → cache evicted; sweep won't re-enqueue
(peer is genuinely gone).
- newer `created_at` → cache refreshed + streak reset
(peer republished; allowed to retry immediately).
- same → cache untouched; cooldown stands.
Cost: ~one fetch per dead peer per 30-min cooldown cycle, vs
hundreds of offer publishes/hour today.
Clock-skew tolerance on freshness check
---------------------------------------
`signal.rs` `validate_offer_freshness` and
`validate_traversal_answer_for_offer` now allow ±60s grace beyond
strict TTL. Both return a new `FreshnessOutcome` enum so callers
can DEBUG-log when an offer/answer was only accepted via the grace
window. `FRESHNESS_SKEW_TOLERANCE_MS` is hard-coded — loosening
this past minutes erodes the freshness/replay security boundary
and operators tend to tune in the wrong direction.
NTP-style skew estimate (offer_received_at echo)
------------------------------------------------
Added optional `offerReceivedAt: Option<u64>` field to
`TraversalAnswer` payload. Responder fills it with `now_ms()` at
offer-receipt time. Initiator computes the standard NTP offset
formula `((T2-T1) + (T3-T4)) / 2` against the round-trip and
DEBUG-logs when `|skew| ≥ 30s`. Skew is also stashed in
`FailureState` and surfaced in `show_peers`. Non-breaking — older
responders that don't fill the field still produce valid answers,
and `estimate_clock_skew` returns `None`.
Per-peer state in `show_peers` JSON
-----------------------------------
Each peer entry in `show_peers` now carries:
"nostr_traversal": {
"consecutive_failures": <u32>,
"in_cooldown": <bool>,
"cooldown_until_ms": <u64 | null>,
"last_observed_skew_ms": <i64 | null>
}
Always emitted (schema-stable); values populated when discovery is
enabled and the npub has a recorded entry. Required a new public
`Node::nostr_discovery_handle()` accessor and refactored
`FailureState`'s internal Mutex from `tokio::sync` to `std::sync`
(operations never hold across await), which lets the synchronous
`show_peers` handler call `snapshot()` directly without the
dispatcher becoming async.
New config knobs (under `node.discovery.nostr`)
-----------------------------------------------
failure_streak_threshold: 5
extended_cooldown_secs: 1800
warn_log_interval_secs: 300
failure_state_max_entries: 4096
Tests
-----
12 new unit tests:
- 5 in `tests.rs` covering freshness strict / tolerated / rejected
outcomes, NTP skew estimation, and the backward-compat None case
when the responder didn't fill `offer_received_at`.
- 7 in `failure_state.rs` covering streak/warn-rate-limit state
transitions, cooldown active vs expired semantics,
success-resets-streak, observed-skew records, and size-cap
eviction by oldest `last_failure_at`.
CHANGELOG entries added under `[Unreleased]` Fixed.
|
||
|
Johnathan Corgan
|
e08f42e3cc |
Pin discovery state machine: open-discovery sweep + per-attempt lookup timeout
Cover two adjacent runtime behaviors in the discovery state machine
that were previously unpinned at the test level.
1. Open-discovery startup sweep iterate-filter-queue contract.
Cover the runtime sweep behavior: iterate advert cache, apply
skip-filters (own-pubkey, already-connected peers), queue eligible
entries to retry_pending. The config layer was tested but the sweep's
own filtering logic was unpinned.
src/discovery/nostr/runtime.rs: add #[cfg(test)] impl block with
three pub(crate) helpers — new_for_test() builds a minimal
NostrDiscovery with empty cache and no relays/background tasks (uses
fresh nostr::Keys signer + Client::builder().autoconnect(false));
cached_advert_for_test() wraps an OverlayEndpointAdvert into a
CachedOverlayAdvert valid for 1h; insert_advert_for_test() writes
direct to the advert_cache RwLock. All three vanish from release
builds via cfg-gating.
src/node/lifecycle.rs: visibility-only widen on
run_open_discovery_sweep from private async fn to
pub(in crate::node) async fn so the in-tree test can drive it
directly. Same pattern as already-pub(in crate::node) handlers in
src/node/handlers/.
src/node/tests/discovery.rs: add #[tokio::test]
test_open_discovery_sweep_queues_eligible_skips_filtered. Builds
Node + Arc<NostrDiscovery>, injects 3 adverts (eligible, already-
connected peer, own-pubkey), invokes the sweep, asserts retry_pending
contains exactly the eligible entry with matching peer_config npub
and the two filtered entries do NOT appear.
2. Per-attempt timeout state machine in check_pending_lookups.
Cover the central new behavior of
|
||
|
Johnathan Corgan
|
c86dc32197 |
Pin STUN binding-success parser malformed-response behavior
Add 6 negative-input unit tests to src/discovery/nostr/stun.rs covering truncated header (all lengths 0..20), bad magic cookie, unknown attribute type (skip-not-error), truncated XOR-MAPPED-ADDRESS, length-overflow attribute, and transaction-ID mismatch. The happy path was exercised by the 3 NAT scenarios but the parser had no negative-input coverage. Tests pin observed behavior: parse_stun_binding_success returns Option<SocketAddr>, so all malformed-response cases assert None rather than an error variant. Unknown TLVs are silently skipped via the loop's default arm; length-overflow triggers the value_end > packet.len() guard and breaks out of the loop without panicking. |
||
|
Johnathan Corgan
|
a41f80a776 |
Tighten clippy gate to --all-targets --all-features and clean up
The local ci-local.sh and the GitHub CI clippy invocations both used `cargo clippy --all -- -D warnings`, which only checks lib + bin targets. Test code, integration tests, and benches were not lint-gated. Three pre-existing clippy errors lurked in test modules as a result (two field_reassign_with_default in config tests, one items_after_test_module in stun.rs). Tighten both invocations to `cargo clippy --all-targets --all-features -- -D warnings` so the gate covers everything cargo can build, and fix the three exposed errors: - src/config/mod.rs: rewrite two test-only `Config::default()` + field-reassign sites to struct-update syntax. - src/discovery/nostr/stun.rs: move helper `random_txn_id` above the `#[cfg(test)] mod tests` block. Also adds a dedicated Clippy job to the GitHub CI workflow so the strict gate runs on every PR (the workflow had no clippy job before; clippy ran only via testing/ci-local.sh on operator machines). No behavior changes; lint hygiene + CI hardening only. |
||
|
Johnathan Corgan
|
ab2edec2c6 |
Add Nostr open-discovery startup sweep with diagnostic logging
Under `node.discovery.nostr.policy: open`, the per-tick auto-dial in `queue_open_discovery_retries` was supposed to pick up adverts cached from the relay subscription backlog at startup, but in practice only adverts arriving live (after the daemon was up) were being dialed. Backlog adverts sat in the in-memory cache until they aged out. Adds a one-shot startup sweep that runs once per daemon start, gated identically to the per-tick sweep (`enabled` && `policy == open`), after a configurable settle delay so the relay subscription backlog has time to populate the advert cache. The sweep iterates the cache with the same skip-filters as the per-tick path (statically-configured peers, already-connected, retry-pending, connecting) plus a tighter age filter: only adverts whose `created_at` is within `startup_sweep_max_age_secs` of now are queued. Two new config fields under `node.discovery.nostr`: - `startup_sweep_delay_secs` (default 5) - `startup_sweep_max_age_secs` (default 3600 = one hour) Both are only consulted when `policy == open`; under any other policy the sweep is a no-op. Adds diagnostic logging to the open-discovery sweep so operators can verify what the auto-dial path is doing on each daemon bring-up: info-level on each retry-queued enqueue (with peer short-npub and advert age), and a one-line summary on every startup sweep and on any per-tick sweep that queues at least one retry. The summary breaks down skipped candidates by reason (age, configured, self, already-connected, retry-pending, connecting, no-endpoints, invalid-npub) — currently the path was silent so there was no operator-visible signal that the cache iteration was running. Refactors the existing `queue_open_discovery_retries` body into a shared `run_open_discovery_sweep(max_age_secs, caller)` helper so the per-tick and startup paths share filter/queue logic and only differ in the age filter and log label. Surfaces `created_at` from `NostrDiscovery::cached_open_discovery_candidates` (return tuple extended) so the age filter has the data it needs. Three new unit tests in `config::node::tests` cover the new defaults, YAML override round-trip, and partial-YAML default fallback. |
||
|
Johnathan Corgan
|
e641eb5b5f |
Drop the nostr-discovery cargo feature flag
Make Nostr-mediated overlay discovery unconditional, mirroring the philosophy of PR #79's collapse of the tui/ble/gateway features in favor of platform cfg gates. nostr / nostr-sdk are pure Rust over WebSockets/TCP, so they build cleanly on every FIPS-supported platform — there is no need for the parallel feature gate. The flag was already in `default = [...]`, so no behavior change for anyone using `cargo build` without `--no-default-features`. Operators who explicitly disabled the feature will now find Nostr code present in the binary; the runtime check `node.discovery.nostr.enabled` still controls whether the runtime starts. Cargo.toml: - Remove the `[features]` table entirely. - Drop `optional = true` from `nostr` and `nostr-sdk`. Source: 27 cfg sites collapsed across 5 files — `src/discovery.rs`, `src/discovery/nostr/mod.rs`, `src/node/handlers/rx_loop.rs`, `src/node/lifecycle.rs`, `src/node/mod.rs`. Two `#[cfg(not(feature = "nostr-discovery"))]` fallback blocks (the udp:nat-without-runtime debug-log path and the "feature not compiled in" warning at startup) were removed as dead code; the always-on path already handles the missing-runtime case via `nostr_discovery: Option<NostrDiscovery>`. Packaging and tooling: - `packaging/openwrt-ipk/Makefile`: drop a stale `--features gateway` flag (the `gateway` feature was already removed in PR #79; this was a leftover that the build path tolerated only because cargo ignored unknown feature names). - `testing/scripts/build.sh`: drop `DEFAULT_CARGO_BUILD_ARGS=(--features nostr-discovery)`; defaults are empty. - `packaging/common/fips.yaml`: drop the "requires the nostr-discovery feature" comment from the discovery section. Bundled cleanup: - Apply `cargo clippy --fix` against three pre-existing warnings in `src/discovery/nostr/runtime.rs` and `src/discovery/nostr/stun.rs` (collapsed `if let Some` chain; two redundant `as i32` casts). These were always present but masked when the feature gate was off; they surface now that the code is unconditionally compiled. - `cargo fmt` settled two minor formatting drift sites in `src/bin/fips-gateway.rs` and `src/config/mod.rs`. Tests: 1083 passed, 0 failed, 4 ignored. clippy clean. fmt clean. |
||
|
Johnathan Corgan
|
3092c95d54 |
Filter stray punch probes on adopted UDP transports
When a UDP hole-punch succeeds in only one direction and the local side adopts the punched socket, the remote end keeps retrying its own punch attempt for several seconds. Those retries arrive on the adopted socket and were forwarded to the FMP rx handler, which parsed the first byte (0x4E from PUNCH_MAGIC's "NPTC" big-endian encoding) as FMP protocol version 4 and emitted "Unknown FMP version, dropping" once per probe. The probe stream contaminated post-adoption handshake logs and added timing pressure during the handshake window. Add a transport-level filter in udp_receive_loop that silently drops any datagram whose first 4 bytes match PUNCH_MAGIC or PUNCH_ACK_MAGIC. Filter applies to all UDP transports, not just adopted ones — the magic values cannot collide with valid FMP frames (FMP version 4 is not assigned, and the protocol's versioning is wire-format breaking), so universal filtering is safe and removes any "is this an adopted socket" branching. Move PUNCH_MAGIC / PUNCH_ACK_MAGIC and the new is_punch_packet() helper from the `nostr-discovery`-gated submodule up to crate::discovery (unconditionally compiled) so the UDP transport can import them without requiring the feature. The nostr-discovery types module re-exports the constants so the existing traversal-side imports keep working unchanged. Test: pushes a probe + ack + real frame through the receive loop and asserts only the real frame is delivered to packet_tx. |
||
 
|
34e00b9f6e |
Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.
The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.
== Cargo feature and dependencies
- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
`src/discovery/nostr/signal.rs` rather than relying on the SDK's
rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
exclusively.
== Wire format
Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.
Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.
Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.
== Discovery surface
- `src/discovery.rs` (always compiled)
- `EstablishedTraversal`: bound UDP socket + selected remote +
peer npub + optional transport name/config tuning overrides.
- `BootstrapHandoffResult`: returned on successful handoff —
allocated transport id, local/remote addrs, peer NodeAddr,
session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
- `types.rs`: wire and control types described above. `ADVERT_KIND`
constant. `BootstrapError` enumerates failure modes (disabled,
missing advert, missing NAT endpoint, no usable relays, invalid
advert, invalid npub, signal timeout, punch timeout, replay,
STUN failure, protocol, nostr, io, serde, event-parse).
- `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
nostr-sdk `Client`, subscribes to advert + signal event kinds,
maintains a bounded advert cache and a bounded seen-sessions
replay set, drains `BootstrapEvent::{Established, Failed}` for
the node to consume, exposes `update_local_advert`,
`request_connect`, `advert_endpoints_for_peer`,
`cached_open_discovery_candidates`, and `shutdown`.
- `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
built against per-attempt ephemeral keys; inbound events are
unwrapped against the node identity.
- `stun.rs`: RFC 5389/8489 Binding Request client with
XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
observe the initiator's own reflexive address against its
locally configured STUN list (peer-advertised STUN is
informational, never an egress target).
- `traversal.rs`: per-attempt candidate-pair punch planner.
Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
LAN-private and ULA interface addresses alongside the STUN
reflexive address, schedules probe/ack exchanges at the
configured interval for the configured duration, and picks the
first candidate pair that authenticates end-to-end.
Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.
== Configuration surface
`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:
- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
for both relay lists, and Google / Cloudflare / Twilio for STUN.
Operators are expected to override for production. Other
verified-working public relays for reference:
`nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
`nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
Open}` — controls whether advert-derived endpoints are consumed
only for peers carrying `via_nostr = true`, or also for
non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
`local_addresses` list is empty and peers see only the reflexive
address. Enable per-node only for genuinely same-LAN deployments;
off-by-default eliminates the misleading-path failure mode for
the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
(2048) — bound memory under ambient relay volume; overflow
evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
`punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)
Per-peer and per-transport flags:
- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
advert-derived addresses are appended as fallback dial candidates
after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
when `via_nostr: true`; validation requires at least one of the
two to be present per peer, and the error message names the
peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
`UdpConfig.public: Option<bool>` — UDP transports can be
advertised either as direct `host:port` (public = true) or as the
`addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
— TCP and Tor onion endpoints can be advertised as directly
reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
special-casing in YAML and routes through the bootstrap runtime.
Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):
- Any transport with `advertise_on_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
`public = false` — i.e. `udp:nat`) additionally requires at least
one `dm_relay` and at least one `stun_server`.
Surfaced as `ConfigError::Validation`.
== Node integration
`src/node/lifecycle.rs` is the main integration point.
- At node start (after transports are up, before TUN), if Nostr is
enabled and the feature is compiled in, `NostrDiscovery::start` is
invoked, the initial local overlay advert is built from the live
transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
at method definition and call site), which refreshes the local
advert, drains bootstrap events, adopts established traversals,
schedules retries for failed traversals, and — under `policy:
open` — enqueues outbound retries for non-configured peers
visible in the advert cache, bounded by
`open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
first exhausts the static address list in priority order and only
then appends advert-derived fallback addresses; both lists run
through the same `attempt_peer_address_list` code path. The
`udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
optional transport instance name), and emits an `OverlayAdvert`
including `signalRelays` and `stunServers` when any UDP endpoint
is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
allocates a new `TransportId`, constructs a `UdpTransport` with
the user-supplied (or default) `UdpConfig`, calls the new
`adopt_socket_async` to reuse the punched socket verbatim,
registers the transport in the normal transport map, records it
in `bootstrap_transports`, and calls `initiate_connection` so the
normal handshake path runs. On failure, the transport is stopped
and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
background tasks stop before transports are torn down. (The
advert is not explicitly retracted; NIP-40 expiration plus the
next refresh from any live publisher supersedes it.)
New `Node` fields:
- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
transports adopted from NAT traversal, cleaned up via
`cleanup_bootstrap_transport_if_unused` whenever the link,
connection, peer, or pending-connect referencing them is removed.
Retry and error surface:
- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
for a retry entry. `pump_retries` drops expired entries with an
info log. Used for open-discovery retries, which expire at two
times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
`adopt_established_traversal` when the underlying transport
adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
lifecycle, rx-loop tick, and timeout bookkeeping.
== UDP transport
`src/transport/udp/`:
- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
adopts an externally bound socket, makes it non-blocking, applies
the configured buffer sizes (warning if the kernel clamps), and
reports the resulting local address. Preserves the NAT mapping —
no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
`start_async` analogue for an already-bound socket, wiring the
async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
still holding a recv task or socket (for example on error
teardown), aborts the task, clears the socket, and emits a debug
log so the cleanup is visible in tracing rather than silent.
== Logging and observability
Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.
Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:
- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`
Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.
Other operator-facing logs:
- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
successful return, tagged with peer npub, session id, transport
id, and both socket endpoints, so the bootstrap handoff is
traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
downstream STUN, punch, and handshake logs for the same peer
correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
(hot path: fires for every unrelated signal event on the same
relay).
- Traversal-offer handler failures log at debug. Expected conditions
such as punch timeout on symmetric NAT are covered there; real
problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
(`max_concurrent_incoming_offers`) and state that the offer was
rate-limited rather than failing.
== Tests
- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
encoding, signal envelope round-trip, STUN parsing, punch-packet
codec, and replay-window enforcement. Run under the
`nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
cross-field invariants and YAML parsing of the full
`node.discovery.nostr` block plus `peers[].via_nostr`, empty
`addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
synthetic traversal (bound UDP socket pair + synthetic peer
identity) through `adopt_established_traversal` and assert the
Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
- Local `strfry` relay, local STUN responder, and one or two
router containers performing `iptables` NAT.
- Node LAN interfaces are provisioned with explicit `veth` pairs
injected into the node and router namespaces so every packet
traverses the router namespace (plain Docker bridges are not
used for the LAN).
- `cone` scenario: both peers behind full-cone-emulation NAT
(SNAT with source-port preservation, inbound DNAT back to the
single LAN host regardless of remote source); asserts UDP
traversal succeeds and link remote addresses are on the router
WAN subnet.
- `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
traversal fails and TCP fallback converges over router-
published WAN addresses.
- `lan` scenario: both peers share a LAN subnet; asserts LAN
addresses are preferred over reflexive ones.
- Cleanup tears down all profile-gated services
(`--profile cone --profile symmetric --profile lan`) so no
orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
`--features "tui nostr-discovery"` by default so NAT-harness
binaries include bootstrap support.
== CI
- Linux release build and nextest unit-test job both use
`--features "gateway nostr-discovery"` so the feature-gated code
and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
`nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
`docker compose logs` on failure, and always stop containers.
== Packaging and operations
- `packaging/common/fips.yaml` ships a fully commented
`node.discovery.nostr.*` block, plus documented
`advertise_on_nostr` / `public` examples under the UDP transport,
an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
example under the static peer section with both a direct
`host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
publishes target the new default relay set.
== Documentation
- `README.md`: overlay discovery + NAT traversal moved from
"Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
previously described Nostr discovery and NAT traversal as future
work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
direction)" qualifier from the Nostr Relay Discovery section,
expands with the `udp:nat` advertisement and bootstrap handoff
description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
rebinding (roaming) and initial NAT traversal (Nostr path) are
distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
`node.discovery.nostr.*` surface, including the three resource
caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
reference for the shipped mechanism, including the empty-
`addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
Implemented status callout, clarifies that the punch socket is
per-peer and per-attempt rather than shared with the application
listener, aligns field names with the shipped JSON
(`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
`localAddresses`, `stunServer`), sets the `d`-tag to
`fips-overlay-v1`, names the kind as 37195, and notes that
advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
path, STUN/punch path, configuration surface, and Docker NAT lab.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
|