| `node.discovery.attempt_timeouts_secs` | array<u64> | `[1, 2, 4, 8]` | Per-attempt timeouts. Each entry is the deadline for one `LookupRequest` before sending the next attempt with a fresh `request_id`. Length determines total attempt count; default gives 4 attempts and a 15s total budget |
| `node.discovery.retry_interval_secs` | u64 | `5` | Retry interval within the timeout window; after this interval without a response, resend the lookup |
| `node.discovery.max_attempts` | u8 | `2` | Max attempts per lookup (1 = no retry, 2 = one retry) |
| `node.discovery.backoff_base_secs` | u64 | `30` | Base for exponential backoff after lookup failure; doubles per consecutive failure |
| `node.discovery.backoff_max_secs` | u64 | `300` | Cap on exponential backoff (5 minutes) |
| `node.discovery.backoff_base_secs` | u64 | `0` | Optional post-failure suppression base in seconds; doubles per consecutive failure. `0` disables (default) — the per-attempt sequence is the only retry pacing |
| `node.discovery.backoff_max_secs` | u64 | `0` | Cap on optional post-failure backoff |
| `node.discovery.forward_min_interval_secs` | u64 | `2` | Transit-side rate limiting: minimum interval between forwarded lookups for the same target |
| `node.discovery.nostr.advert_relays` | list[string] | `["wss://relay.damus.io", "wss://nos.lol", "wss://offchain.pub"]` | Relays used for service adverts |
| `node.discovery.nostr.dm_relays` | list[string] | `["wss://relay.damus.io", "wss://nos.lol", "wss://offchain.pub"]` | Relays used for encrypted signaling events |
| `node.discovery.nostr.stun_servers` | list[string] | `["stun:stun.l.google.com:19302", "stun:stun.cloudflare.com:3478", "stun:global.stun.twilio.com:3478"]` | STUN servers used for local reflexive address discovery |
| `node.discovery.nostr.advert_refresh_secs` | u64 | `1800` | How often adverts are refreshed in seconds |
If `stun_servers` is omitted, the built-in default list above is used. If it is
specified in YAML, the configured list fully overrides the defaults.
Initiators use only this local list for outbound STUN queries; peer-advertised
STUN values are published for diagnostics/interoperability but are not used as
arbitrary egress targets.
The built-in advert and DM relay defaults point at widely-operated public
relays (Damus, nos.lol, Primal) as best-effort endpoints; operators are
encouraged to override them with their own relay preferences for production
deployments.
Advert freshness is enforced semantically: events with expired NIP-40
`expiration` tags are dropped, and adverts are also bounded by a created-at
staleness window derived from `advert_ttl_secs` (with a grace multiplier).
The current in-tree STUN parser handles IPv4 and IPv6 mapped-address
attributes. Local traversal candidates include active non-loopback private
interface addresses (RFC1918 IPv4 and IPv6 ULA) plus probed local egress
addresses for the punch socket port.
During punching, compatible private-subnet candidates and reflexive candidates
are attempted in parallel; the first successful path wins.
### Spanning Tree (`node.tree.*`)
Controls tree construction and parent selection.
@@ -327,6 +379,8 @@ restarting the daemon. Hostnames are case-insensitive.
| `transports.udp.mtu` | u16 | `1280` | Transport MTU |
| `transports.udp.recv_buf_size` | usize | `2097152` | UDP socket receive buffer size in bytes (2 MB). Linux kernel doubles the requested value internally. Host `net.core.rmem_max` must be >= this value. |
| `transports.udp.send_buf_size` | usize | `2097152` | UDP socket send buffer size in bytes (2 MB). Host `net.core.wmem_max` must be >= this value. |
| `transports.udp.advertise_on_nostr` | bool | `false` | Include this UDP transport in Nostr endpoint adverts |
| `transports.udp.public` | bool | `false` | If advertised: `true` publishes direct `host:port`; `false` publishes `udp:nat` rendezvous |
### Ethernet (`transports.ethernet.*`)
@@ -585,6 +639,7 @@ Static peer list. Each entry defines a peer to connect to.
What the other side needs: either a static `addresses` entry for this
peer, or a peer entry with `via_nostr: true` and an empty (or omitted)
`addresses` list — the advert-resolved endpoint will be used at dial
time. Static and Nostr-resolved addresses can also be combined: when
both are present, static addresses are tried first and Nostr-resolved
endpoints are appended as fallback.
### Scenario 2: Advertise a Tor onion node
The node runs a Tor onion service in directory mode (Tor-managed
`HiddenServiceDir`) and advertises the `.onion` address. Peers dial via
their local Tor SOCKS5 proxy without ever knowing the onion string
out-of-band.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
transports:
tor:
mode: directory
socks5_addr: "127.0.0.1:9050"
directory_service:
hostname_file: "/var/lib/tor/fips/hostname"
bind_addr: "127.0.0.1:8444"
advertise_on_nostr: true
```
What this achieves: the node publishes a `tor:<hash>.onion:8443`
endpoint alongside any other advertised transports. The advert itself
is still published over clearnet WebSocket relays — Tor protects the
data plane, not the discovery plane. See
[Security and threat model](#security-and-threat-model) for the trade-off.
### Scenario 3: Lookup a configured peer by npub (no advertising)
The node does not publish any advert of its own. It only consumes
adverts for peers it has explicitly listed with `via_nostr: true`. This
is the right shape for a client that wants Nostr-mediated resolution
without becoming a rendezvous target itself.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: false
policy: configured_only
transports:
udp:
bind_addr: "0.0.0.0:2121"
peers:
- npub: "npub1peer..."
alias: "remote-node"
addresses:
- transport: udp
addr: "203.0.113.45:2121"
priority: 10
via_nostr: true
connect_policy: auto_connect
```
What this achieves: on dial, the static address is tried first; if the
peer has published a newer advert (for example, its public IP has
changed), those addresses are appended as additional candidates.
`configured_only` is the default — it is shown here for clarity.
If you have no static address for the peer at all, omit `addresses`
entirely (or leave it empty) — `via_nostr: true` is sufficient on its
own and dial endpoints are taken from the advert.
### Scenario 4: UDP NAT hole-punch with a configured peer
Neither side has a stable public UDP endpoint. Both sides advertise
`udp:nat`, run the STUN + offer/answer exchange, and punch through
their NATs to establish a direct UDP link. This is the full
NAT-traversal path.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
dm_relays:
- "wss://relay.damus.io"
- "wss://nos.lol"
stun_servers:
- "stun:stun.l.google.com:19302"
- "stun:stun.cloudflare.com:3478"
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: false
peers:
- npub: "npub1peer..."
alias: "nat-peer"
addresses:
- transport: udp
addr: "nat"
priority: 1
via_nostr: true
connect_policy: auto_connect
auto_reconnect: true
```
What this achieves: the node publishes a `udp:nat` endpoint plus its
signaling relays and STUN server list in the advert. The peer side runs
the same configuration. When either side initiates, an encrypted offer
is sealed to the peer's npub, a matching answer comes back, and both
sides punch at the negotiated time. On success, the punch socket is
adopted as an FMP UDP transport and Noise IK proceeds normally.
> **Validation:**`advertise_on_nostr: true` with `public: false` on UDP
> requires both `dm_relays` and `stun_servers` to be non-empty. The
> node fails startup with a config validation error if either list is
> empty. This is enforced because a `udp:nat` advert without signaling
> relays or STUN servers is unreachable by construction.
Works best with full-cone NAT on at least one side. Symmetric NAT on
both sides is not reliably traversable with this protocol and will time
out after `punch_duration_ms`; fall back to a Tor or TCP transport in
that case.
### Scenario 5: Open discovery — no pre-configured peers
Under `policy: open`, any node that publishes an advert under the same
`app` namespace becomes a candidate. Discovered peers are queued for
connection attempts subject to `open_discovery_max_pending`.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
policy: open
open_discovery_max_pending: 32
app: "my-experiment.v1"
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: true
peers: []
```
What this achieves: peers are discovered entirely through ambient advert
traffic on the configured relays. Setting a non-default `app` value
(replacing `fips-overlay-v1`) scopes the discovery set to participants
who opt into the same experiment and avoids being joined to unrelated
overlays that happen to share the default namespace.
> **Scope warning:** Open discovery is an admission-free mode. Any node
> that publishes on the same `app` name and passes the peer-ACL check
> becomes a connection candidate. If you rely on peer ACLs for admission
> control, verify that list is set correctly before enabling this mode.
## Operational knobs
All fields below live under `node.discovery.nostr.*`. Defaults are
defined in `src/config/node.rs`.
| Field | Type | Default | Purpose |
| --- | --- | --- | --- |
| `enabled` | bool | `false` | Master switch. When false, the discovery runtime is not started. |
| `advertise` | bool | `true` | If true, publish this node's own overlay advert. |
| `advert_relays` | list | `["wss://relay.damus.io", "wss://nos.lol", "wss://offchain.pub"]` | Relays used to publish and fetch overlay adverts (kind 37195). |
| `dm_relays` | list | same as `advert_relays` | Relays used for encrypted offer/answer signaling (kind 21059). |
| `stun_servers` | list | `["stun:stun.l.google.com:19302", "stun:stun.cloudflare.com:3478", "stun:global.stun.twilio.com:3478"]` | STUN servers used to observe the local reflexive address before a punch. Peer-advertised STUN values are not used. |
| `share_local_candidates` | bool | `false` | If true, include this node's RFC 1918 / ULA interface addresses as host candidates in the traversal offer. Off by default — sharing private host candidates is only useful when peers are on the same physical LAN, and tends to cause misleading punch successes when an asymmetric L3 path (corporate VPN, Tailscale subnet route, overlapping address space) makes a peer's private IP one-way reachable. Enable per-node only when same-LAN punching is wanted. |
| `app` | string | `"fips-overlay-v1"` | Application namespace. Included in the advert identifier; only peers with the same value cross-resolve. |
| `signal_ttl_secs` | u64 | `120` | TTL on the encrypted offer/answer events. Also caps the wait for an answer. |
| `advert_ttl_secs` | u64 | `3600` | NIP-40 expiration set on this node's published advert. |
| `advert_refresh_secs` | u64 | `1800` | Interval between re-publishes. Must be less than `advert_ttl_secs`. |
| `attempt_timeout_secs` | u64 | `10` | Overall timeout for a single punch attempt (STUN + signal + punch). |
| `punch_start_delay_ms` | u64 | `2000` | Delay between receiving the answer and sending the first punch packet. Gives the remote side time to arrive at the same point. |
| `punch_interval_ms` | u64 | `200` | Gap between successive punch probes. |
| `punch_duration_ms` | u64 | `10000` | How long to keep probing before declaring the attempt failed. |
| `replay_window_secs` | u64 | `300` | How long a session id stays in the replay-detection cache. |
| `max_concurrent_incoming_offers` | usize | `16` | Semaphore cap on inbound offers being processed simultaneously. Excess offers are dropped with a warn log. |
| `advert_cache_max_entries` | usize | `2048` | Max cached peer adverts (LRU by expiry). |
| `seen_sessions_max_entries` | usize | `2048` | Max tracked session ids for replay detection. |
| `open_discovery_max_pending` | usize | `64` | Max peers queued for connection attempts under `policy: open`. |
The per-transport keys are:
| Key | Type | Where | Default | Purpose |
| --- | --- | --- | --- | --- |
| `advertise_on_nostr` | bool | `transports.{udp,tcp,tor}` | `false` | Include this transport's endpoint in the overlay advert. |
| `public` | bool | `transports.udp` | `false` | When `advertise_on_nostr` is true: `true` publishes `udp:host:port`, `false` publishes `udp:nat`. |
| `via_nostr` | bool | `peers[]` | `false` | Append advert-resolved endpoints to this peer's dial list. |
## Validation rules at startup
The following combinations are rejected with `ConfigError::Validation`:
- Any transport sets `advertise_on_nostr: true` while
`node.discovery.nostr.enabled` is `false` or absent.
- Any peer sets `via_nostr: true` while
`node.discovery.nostr.enabled` is `false` or absent.
- A UDP transport sets `advertise_on_nostr: true` with `public: false`
(a `udp:nat` advert) but `dm_relays` is empty.
- A UDP transport sets `advertise_on_nostr: true` with `public: false`
but `stun_servers` is empty.
## Under the covers
The rest of this document describes how the feature works inside the
node. For the on-the-wire event format and NIP references, see the
On success, the discovery runtime emits `BootstrapEvent::Established`
carrying the session id, the punch socket, and the learned remote
address. `adopt_established_traversal()` in the node lifecycle takes
the socket, registers it with the UDP transport layer as a new
transport instance, and calls `initiate_connection()` with the peer's
FIPS identity as the expected remote. FMP's Noise IK handshake runs on
the same socket — there is no "promote link" step between punch and
handshake; the punch socket *is* the FMP socket.
From that moment on, the connection is a normal FMP link and is
subject to the usual liveness (MMP heartbeats), rekey, and removal
behavior. A link-dead event does not re-enter the discovery runtime
automatically; reconnection relies on `auto_reconnect` and the same
dial path that triggered the original punch.
### Auto-connect semantics
Discovery does not itself initiate connections. It only supplies
addresses. Dial attempts originate from the existing peer-connection
machinery:
- **Configured peers** (`peers[]` with `connect_policy: auto_connect`)
are dialed on startup and on retry. When `via_nostr` is set, advert
endpoints are appended to the dial list with lower priority than
static entries.
- **Open discovery peers** are assembled from the advert cache, fenced
by the peer ACL, and enqueued into a bounded retry queue sized by
`open_discovery_max_pending`. There is no event-driven
"connect on every advert" — a peer re-enters the queue only when its
prior attempt has drained.
- **Manual dials** (`fipsctl connect`) can target any configured peer
and use the same dial path, including Nostr resolution if configured.
### Rate limits and safeguards
| Mechanism | Default | What it prevents | Behavior at limit |
| --- | --- | --- | --- |
| Offer semaphore (`max_concurrent_incoming_offers`) | 16 | CPU and memory exhaustion from offer spam on DM relays. | Warn log, offer dropped. |
| Advert cache (`advert_cache_max_entries`) | 2048 | Memory growth from ambient advert traffic under `policy: open`. | LRU-by-expiry eviction. |
| Seen-sessions (`seen_sessions_max_entries`) | 2048 | Replay of stale `sessionId` values. | Oldest entry evicted. |
| Signal TTL (`signal_ttl_secs`) | 120 s | Indefinite in-flight offers on relays. | Expired offers rejected at validation. |
| Open discovery queue (`open_discovery_max_pending`) | 64 | Unbounded retry queue under ambient advert load. | New candidates skipped until the queue drains. |
| Punch window (`punch_duration_ms`) | 10 s | Endless probe traffic after one side has given up. | Attempt declared failed; sockets discarded. |
Only one of these (`max_concurrent_incoming_offers`) is a load-shedding
mechanism — the rest are capacity bounds. The load-shedding threshold
is deliberately conservative so that a misbehaving relay cannot flood
the node with offers fast enough to starve legitimate traffic.
### Relay model
All configured relays (advert + DM) are opened on a single
`nostr-sdk::Client` at startup. Publication is fan-out: the same event
is sent to every relay in the target list, with no explicit retry or
relay selection. Redundancy is implicit — a downed relay simply means
its copy of the advert or signal is unavailable, while other relays
still serve the same data.
For signaling specifically, the node prefers the recipient's NIP-65
inbox relays when available (the recipient publishes its inbox list as
a kind 10002 event to its own DM relays on startup) and falls back to
the local `dm_relays` list otherwise. This keeps the common case
off the sender's DM relays when those are different from the
recipient's, at the cost of one extra NIP-65 fetch per offer.
There is no per-relay rate limiting or health check. The relay model
assumes that an operator chooses relays they trust to be best-effort
available and that outright misbehavior is handled at the offer
semaphore and replay-cache layers downstream.
## Security and threat model
- **Relay operators can observe metadata.** They see which npubs
publish adverts, to whom offers are sent, and the timing of that
traffic. The *contents* of offer and answer events are
NIP-59/NIP-44 sealed — only the intended recipient decrypts them.
Adverts are public by design.
- **STUN servers see the node's public IP and port.** Only the STUN
servers listed in the node's own `stun_servers` are ever contacted
for reflexive discovery. Peer-advertised STUN values are
informational; a malicious peer cannot steer this node to a
chosen STUN target. See the doc comment on
`node.discovery.nostr.stun_servers`.
- **The FIPS identity key signs adverts.** Compromise of
`fips.key` is compromise of the node's Nostr identity — an attacker
can publish adverts on behalf of the node. The recovery path is
the same as for any identity compromise: rotate the key and
re-advertise. There is no separate Nostr keypair to rotate
independently.
- **Tor advertising leaks timing via clearnet relays.** When a
Tor-only node advertises its onion address, the advert itself is
published on clearnet WebSocket relays. Operators who want full
unlinkability between the advertising identity and the node's
IP must route relay traffic through Tor as well — for example by
running `fips` inside a network namespace with a Tor SOCKS
proxy as its only egress, or by pointing `advert_relays` and
`dm_relays` at onion relay endpoints.
- **Open discovery accepts anyone publishing on the same `app`.**
Admission control is the peer ACL, not the discovery layer. Verify
the ACL before enabling `policy: open`, and consider using a
non-default `app` value to scope visibility.
- **Nothing about discovery bypasses FMP.** A successful punch yields
a UDP socket with a claimed remote identity. That identity is not
trusted until FMP's Noise IK handshake completes. A peer whose
advert says "I am npub X at 1.2.3.4:5678" but whose FMP handshake
presents a different static key is rejected at the mesh layer.
## See also
- [fips-configuration.md](fips-configuration.md) — full configuration
reference, including all surrounding keys elided from the scenarios
above.
- [fips-transport-layer.md](fips-transport-layer.md) — UDP, TCP, and
Tor transport mechanics; the punch socket is adopted as a normal
UDP transport after handoff.
- [fips-mesh-layer.md](fips-mesh-layer.md) — FMP Noise IK handshake
on the configured DM relays; the resulting punched socket is adopted into
the standard UDP transport via the bootstrap handoff path.
Key properties:
- Identity is built in — Nostr events are signed, so discovery information
@@ -721,8 +727,11 @@ Key properties:
> broadcast — the `discover()` trait method returns newly seen endpoints,
> and per-transport `auto_connect()` / `accept_connections()` policies
> control whether discovered peers are connected automatically or require
> explicit configuration. TCP and Tor have no discovery mechanism.
> Nostr relay discovery is not yet implemented.
> explicit configuration. TCP and Tor have no built-in discovery mechanism.
> Nostr relay discovery and STUN-assisted UDP hole punching are
> implemented behind the `nostr-discovery` cargo feature; see
> [fips-configuration.md](fips-configuration.md) for the
> `node.discovery.nostr.*` configuration tree.
## Transport Interface
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Johnathan Corgan