Files
fips/docs/how-to/enable-nostr-discovery.md
T
Johnathan Corgan 5abf9a9325 docs: four-section /docs/ restructure with new-user content, accuracy pass, and gateway feature-set rewrite
Restructures /docs/ by reader purpose (tutorials, how-to,
reference, design), adds the new-user-progression and
operator-recipe content the prior layout lacked, runs an
accuracy pass against current source across the pre-existing
design docs, and rewrites the gateway feature-set documentation
end-to-end around its actual operational profile (a niche
feature designed for systems already serving DHCP/DNS to a
LAN, with two independent halves — outbound LAN→mesh, inbound
mesh→LAN — sharing one nftables table, one binary, and one
control socket). Top-level README and getting-started rewritten
around two equally-weighted deployment modes (overlay on
existing IP networks; ground-up over non-IP transports).

## Additions

- 11 new tutorials in docs/tutorials/: an 8-step new-user
  progression from single-daemon test-mesh peering through
  to a ground-up two-device mesh, an IPv6-adapter side-trip
  walkthrough, an Advanced Tutorials index, and a hand-held
  OpenWrt walk-through for fips-gateway deployment that
  exercises both halves of the feature.
- 12 new how-tos in docs/how-to/: firewall activation,
  Nostr discovery (resolve / advertise / open across five
  scenarios), Tor onion (directory + control_port modes),
  UDP buffer tuning, unprivileged-user setup, persistent
  identity, host aliases, Bluetooth LE peering, MTU
  diagnostics, manual Linux-host gateway deployment (covers
  both halves), gateway troubleshooting (organised by half),
  and a section index.
- 9 new reference docs in docs/reference/: configuration,
  wire formats, control-socket protocol, four CLI references
  (fips, fipsctl, fipstop, fips-gateway), security posture
  matrix, and Nostr events catalog. Configuration and
  wire-formats are renamed-and-extended from prior design/
  versions; the other seven are net-new.
- 6 new design docs: fips-concepts, fips-architecture, and
  fips-prior-work split out of the deleted fips-intro.md;
  consolidated fips-mmp and fips-mtu aggregations; and a
  new generic port-advertisement-and-nat-traversal doc
  (Nostr-signaled port advertisement plus UDP NAT-traversal
  protocol, FIPS as an example implementation, suitable for
  eventual NIP submission).
- Top-level docs/getting-started.md walking through the
  binary-installer-only Install story.
- packaging/common/hosts pre-populated with the eight public
  test-mesh nodes so shortnames resolve out of the box on
  every fresh install.

## Changes

- 23 wire-format diagrams relocated to reference/diagrams/
  alongside the wire-formats move.
- 4 design diagrams corrected against source code
  (fips-protocol-stack, fips-identity-derivation,
  fips-coordinate-discovery, fips-routing-decision).
- 10 pre-existing design docs reconciled with current
  source. Numeric corrections: stale link-MMP report bounds
  (now [1s, 5s] with 200 ms cold-start floor); UDP default
  MTU (now 1280, IPv6 minimum); node_addr formula
  (SHA-256(pubkey)[..16]); Noise patterns (IK at link, XK
  at session); peer-ACL semantics (strict allowlist requires
  ALL in peers.deny); daemon DNS upstream ([::1]:5354);
  on-the-wire bloom-filter size (1,071 bytes); obsolete
  Cargo-feature references (PR #79 dropped them) removed.
- Transport framing tightened across the docs: TCP is for
  UDP-filtered networks (not NAT traversal); Tor is a
  deployment mode (not failover); WebSocket dropped (not a
  shipped FIPS transport); WiFi promoted to Implemented via
  Ethernet in infrastructure mode; classic-Bluetooth row
  removed (BLE is the only Bluetooth-mode transport).
- docs/design/fips-gateway.md rewritten end-to-end to lead
  with the niche-feature framing and the two-halves
  structure. Title moved from "FIPS Outbound LAN Gateway"
  to "FIPS Gateway"; architecture section describes the
  common machinery (the fips-gateway service, the nftables
  table, the control socket) before splitting into separate
  "Outbound Half" and "Inbound Half" sections of equal
  weight; security considerations split per-half; no Future
  Work section (speculative directions live in the project
  tracker, not in protocol design docs). Inbound port
  forwarding is a first-class half rather than a buried
  "Implemented Extensions" subsection.
- Gateway terminology unified across all gateway docs as a
  separate Linux service running alongside the fips daemon
  (its own systemd unit / OpenWrt init script). Container-
  pattern terms (sidecar) are reserved for the
  Docker/Kubernetes sidecar deployment examples — the
  testing/sidecar/ tree, examples/k8s-sidecar/,
  examples/sidecar-nostr-relay/,
  examples/wireguard-sidecar-macos/, and the related
  CHANGELOG / top-level README entries — where the term
  carries its standard container meaning.
- Net-new design body content: rekey section in
  fips-mesh-layer (Noise IK msg1/msg2 over the established
  link, K-bit cutover, drain window, smaller-NodeAddr-wins
  tie-breaker on dual-init); Mesh Size Estimation and
  Antipoison FPR Cap sections in fips-bloom-filters;
  Mesh-Interface Query Filter subsection in
  fips-ipv6-adapter; failure-suppression knobs and clock-
  skew tolerance in fips-nostr-discovery; loop-rejection
  and mid-chain ancestor swap added to spanning-tree
  propagation / stability rules; Priority Chain in
  fips-mesh-operation renumbered to match the
  routing-decision diagram.
- Top-level README: dropped the stale nostr-discovery
  cargo-feature parenthetical. docs/README.md and the four
  section READMEs (tutorials, how-to, reference, design)
  refreshed for the new structure; index rows reflect both
  halves of the gateway feature and the new fips-gateway
  CLI reference.
- Cargo.toml [package.metadata.deb] assets path updated for
  the fips-security.md move; .gitignore /reference/ rule
  anchored to repo root so docs/reference/ is trackable.
- packaging/openwrt-ipk/files/etc/fips/fips.yaml
  configuration-doc URL updated to the new
  docs/reference/configuration.md location.

## Deletions

- docs/design/fips-intro.md (split into the three new intro
  design docs).
- docs/design/document-relationships.svg (orphan, no longer
  referenced).
- docs/proposals/ tree removed; the only proposal it
  contained (the Nostr UDP hole-punch protocol) was
  rewritten as the new generic
  design/port-advertisement-and-nat-traversal.md.
2026-05-08 03:02:12 +00:00

348 lines
12 KiB
Markdown

# Enable Nostr-Mediated Discovery and NAT Traversal
Nostr-mediated discovery lets FIPS nodes find each other (and punch
through UDP NAT) using public Nostr relays as the signaling channel.
The feature ships in every stock packaging artifact but is **off by
default** — it activates when an operator sets
`node.discovery.nostr.enabled: true` and provides a relay list. See
[../design/fips-nostr-discovery.md](../design/fips-nostr-discovery.md)
for the design and rationale; see
[../reference/configuration.md](../reference/configuration.md) for the
full knob inventory.
Nostr discovery provides three independent capabilities. They can be
enabled separately; most deployments end up using two or three of
them together.
1. **Resolve a known peer's address by npub.** Your daemon consumes
adverts from the relays to look up the current network endpoint
for a peer you have configured by npub. You don't have to know
their IP / port / transport in advance.
2. **Publish your own endpoint so others can resolve you.** Your
daemon publishes a signed advert listing the transports it will
accept connections on. Has two sub-shapes depending on your
network topology: UDP (using NAT traversal if needed) or TCP.
Running a Tor onion service is a separate deployment mode,
covered in its own section below.
3. **Discover peers without prior configuration.** Your daemon
subscribes to all adverts on a chosen application namespace and
treats any publisher as a connection candidate. The most
permissive posture; useful for ambient mesh participation.
Each capability is covered below as one or more scenarios with the
minimal YAML fragment that enables it. Only keys relevant to Nostr
discovery are shown; surrounding node, transport, TUN, DNS, and peer
configuration follows the usual shape.
All scenarios assume `node.identity` is set to a persistent key — an
ephemeral identity would invalidate any advert the moment the node
restarts. See [persistent-identity.md](persistent-identity.md) for
the persistent-key setup.
For hand-held walkthroughs of each capability, see the
[resolve-peers-via-nostr](../tutorials/resolve-peers-via-nostr.md),
[advertise-your-node](../tutorials/advertise-your-node.md),
and [open-discovery](../tutorials/open-discovery.md)
tutorials.
## Capability 1: Resolve a known peer's address by npub
The node does not publish any advert of its own. It only consumes
adverts for peers it has explicitly listed with `via_nostr: true`.
This is the right shape for a client that wants Nostr-mediated
resolution without becoming a rendezvous target itself.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: false
policy: configured_only
transports:
udp:
bind_addr: "0.0.0.0:2121"
peers:
- npub: "npub1peer..."
alias: "remote-node"
via_nostr: true
connect_policy: auto_connect
```
What this achieves: dial endpoints for this peer are taken from
the peer's published Nostr advert. `configured_only` is the
default — it is shown here for clarity.
> **Note:** You can also supply a static address alongside
> `via_nostr: true` (for example, while testing, or as a
> known-good fallback if the advert is stale). Add an `addresses`
> block to the peer entry; static addresses are tried first on
> dial and Nostr-resolved endpoints are appended as additional
> candidates.
## Capability 2: Publish your own endpoint so others can resolve you
This capability has three sub-scenarios depending on the network
shape your node sits behind.
### Sub-scenario 2a: UDP (using NAT traversal if needed)
The node has a public IP (or a stable port-forward) and binds UDP on
a known port. It publishes `udp:host:port` to the advert relays. Any
peer that knows this node's npub and has Nostr discovery enabled can
dial it without knowing the address out-of-band.
When UDP is wildcard-bound (`0.0.0.0:2121`, the default), the daemon
needs help knowing what IP to put in the advert. There are two ways:
STUN auto-discovery (`public: true`) or an explicit override
(`external_addr`). Both are first-class options; pick the one that
fits the deployment.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: true # ← STUN auto-discovery
```
Or, when the public IP is known up front (static residential IP,
cloud Elastic IP behind 1:1 NAT, etc.):
```yaml
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
external_addr: "203.0.113.45:2121" # ← explicit
```
`external_addr` accepts a bare IP (combined with the bind port) or a
full `host:port`. Setting both `public: true` and `external_addr`
together is allowed — the explicit override wins, with STUN as a
logging cross-check. If UDP is bound directly to a public IP rather
than to a wildcard, neither flag is needed; the daemon advertises
the bound endpoint.
What this achieves: the node publishes a single
`udp:<public-ip>:2121` endpoint to the three default advert relays
(`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`).
What the other side needs: either a static `addresses` entry for this
peer, or a peer entry with `via_nostr: true` and an empty (or
omitted) `addresses` list — the advert-resolved endpoint will be used
at dial time. Static and Nostr-resolved addresses can also be
combined: when both are present, static addresses are tried first and
Nostr-resolved endpoints are appended as fallback.
#### When the node is behind NAT
If this node doesn't have a stable public UDP endpoint, advertise
`udp:nat`. The daemon runs the STUN + offer/answer exchange with
the peer and punches through the NAT to establish a direct UDP
link. The peer can either have a public endpoint of its own or
also be behind NAT — both shapes work, as long as at least one
side has a NAT type compatible with hole-punching.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
dm_relays: # overrides the default three-relay
- "wss://relay.damus.io" # set with two for demonstration;
- "wss://nos.lol" # omit this block to keep the defaults
stun_servers:
- "stun:stun.l.google.com:19302"
- "stun:stun.cloudflare.com:3478"
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: false
peers:
- npub: "npub1peer..."
alias: "nat-peer"
addresses:
- transport: udp
addr: "nat"
via_nostr: true
connect_policy: auto_connect
```
What this achieves: the node publishes a `udp:nat` endpoint plus its
signaling relays in the advert. When either side initiates, an
encrypted offer is sealed to the peer's npub, a matching answer
comes back, and both sides punch at the negotiated time. On success,
the punch socket is adopted as an FMP UDP transport and Noise IK
proceeds normally.
> **Validation:** `advertise_on_nostr: true` with `public: false` on
> UDP requires `dm_relays` and `stun_servers` to be non-empty. Both
> ship with non-empty defaults (three relays and three STUN servers
> respectively), so the default config passes. The node fails
> startup only if the operator has explicitly emptied either list —
> a `udp:nat` advert without signaling relays or STUN servers is
> unreachable by construction.
Works best with full-cone NAT on at least one side. Symmetric NAT on
both sides is not reliably traversable with this protocol and will
time out after `punch_duration_ms`.
### Sub-scenario 2b: TCP
The node has a public IP (or a stable port-forward) and accepts
inbound TCP. It publishes `tcp:host:port` to the advert relays.
TCP endpoints exist to serve peers whose networks filter outbound
UDP (corporate LANs, restrictive guest WiFi). NAT traversal does
not apply: the publishing node is publicly reachable on TCP, and
the dialing peer's network only needs to permit outbound TCP to
the advertised port.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
transports:
tcp:
bind_addr: "0.0.0.0:8443"
advertise_on_nostr: true
external_addr: "203.0.113.45:8443"
```
`external_addr` is typically required on cloud setups (AWS Elastic
IP, etc.) where binding directly to the public IP returns
`EADDRNOTAVAIL`. When TCP is bound directly to a public IP, the
override is unnecessary.
What this achieves: the node publishes a `tcp:<public-ip>:8443`
endpoint to the advert relays. Peers with Nostr discovery enabled
dial by npub without out-of-band address exchange.
### Tor onion node
A separate deployment mode for nodes that want anonymity and
censorship-resistance properties on the data plane. Functionally
this still uses Capability 2 (publishing an endpoint to advert
relays) — the difference is that the published endpoint is a Tor
hidden service rather than a public IP.
The node runs a Tor onion service in directory mode (Tor-managed
`HiddenServiceDir`) and advertises the `.onion` address. Peers dial
via their local Tor SOCKS5 proxy without ever knowing the onion
string out-of-band. For the Tor daemon side of this setup, including the inbound-mode
trade-offs and the `torrc` directives each requires, see
[deploy-tor-onion.md](deploy-tor-onion.md).
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
transports:
tor:
mode: directory
socks5_addr: "127.0.0.1:9050"
directory_service:
hostname_file: "/var/lib/tor/fips/hostname"
bind_addr: "127.0.0.1:8444"
advertise_on_nostr: true
```
What this achieves: the node publishes a `tor:<hash>.onion:8443`
endpoint alongside any other advertised transports. The advert itself
is still published over clearnet WebSocket relays — Tor protects the
data plane, not the discovery plane. See the security and threat
model section in
[../design/fips-nostr-discovery.md](../design/fips-nostr-discovery.md#security-and-threat-model)
for the trade-off and how to route relay traffic through Tor as well.
## Capability 3: Discover peers without prior configuration
Under `policy: open`, any node that publishes an advert under the
same `app` namespace becomes a candidate. Discovered peers are queued
for connection attempts subject to `open_discovery_max_pending`.
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
policy: open
open_discovery_max_pending: 64
app: "my-experiment.v1"
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: true
peers: []
```
What this achieves: peers are discovered entirely through ambient
advert traffic on the configured relays. Setting a non-default `app`
value (replacing `fips-overlay-v1`) scopes the discovery set to
participants who opt into the same experiment and avoids being joined
to unrelated overlays that happen to share the default namespace.
> **Scope warning:** Open discovery is an admission-free mode. Any
> node that publishes on the same `app` name and passes the peer-ACL
> check becomes a connection candidate. If you rely on peer ACLs for
> admission control, verify that list is set correctly before
> enabling this mode. See
> [../reference/security.md](../reference/security.md) for the peer
> ACL format.
## See also
- [../tutorials/resolve-peers-via-nostr.md](../tutorials/resolve-peers-via-nostr.md)
— hand-held walkthrough of capability 1
- [../tutorials/advertise-your-node.md](../tutorials/advertise-your-node.md)
— hand-held walkthrough of capability 2 (publish, plus a
short section on `udp:nat` NAT traversal)
- [../tutorials/open-discovery.md](../tutorials/open-discovery.md)
— hand-held walkthrough of capability 3 (open ambient
discovery, the additive policy: open mode)
- [../design/fips-nostr-discovery.md](../design/fips-nostr-discovery.md)
— discovery runtime design, security model
- [../reference/configuration.md](../reference/configuration.md) —
full `node.discovery.nostr.*` and per-transport
`advertise_on_nostr`/`public` table
- [../reference/nostr-events.md](../reference/nostr-events.md) — Kind
37195 advert format, Kind 21059 traversal signaling, Kind 10050
inbox relay list
- [deploy-tor-onion.md](deploy-tor-onion.md) — Tor daemon-side setup
for advertising onion endpoints