fips/docs/tutorials/open-discovery.md

Open Discovery: Find Peers Without Configuration

After advertise-your-node, your daemon publishes its endpoint on Nostr and other open-discovery nodes on the test mesh have already started dialing you. This tutorial flips the symmetry: turn your own daemon into a consumer of every advert in the namespace, so any operator who's publishing becomes a candidate peer of yours.

The whole exercise should take about ten minutes. After it, you'll be a full participant in the ambient fips-overlay-v1 namespace — publishing your own advert and discovering everyone else's.

What you'll build

   ┌─────────────────────────────────────────────────────┐
   │  Nostr relays — fips-overlay-v1 namespace           │
   │   adverts from: test-us01..uk01, others             │
   └────────────────┬─────────────────────────▲──────────┘
                    │                         │
                    │ subscribe to all         │ your own
                    │ adverts in namespace     │ advert
                    │                         │
                    ▼                         │
   ┌────────────────────────────────────────────────────┐
   │  your fips daemon (policy: open)                   │
   │                                                    │
   │  peers list grows ambient as adverts arrive:       │
   │     test-us01     ← was static                     │
   │     test-us03     ← inbound (was already there)    │
   │     test-de01, test-es01, test-uk01, test-us04...  │
   │     plus any other publisher in the namespace      │
   └────────────────────────────────────────────────────┘

You will change one thing in /etc/fips/fips.yaml: under node.rendezvous.nostr, set policy: open (the default is configured_only). After restart, the daemon subscribes to every Kind 37195 advert in the fips-overlay-v1 namespace and queues the publishers for outbound connection attempts.

How open discovery works

Discovery policy. node.rendezvous.nostr.policy decides what the daemon does with incoming advert data. Two values:

• configured_only (the default): the daemon only consumes adverts for peers it has explicitly listed with via_nostr: true. This is what you've been running through the previous two Nostr tutorials.
• open: the daemon subscribes to every advert in the configured app namespace. Any publisher becomes a candidate peer, no peers: list entry needed.

Switching the policy doesn't disturb anything that was already working:

Open is additive, not exclusive. Switching to policy: open doesn't replace your static peers — both mechanisms run in parallel. Configured peers stay in your peers: block and continue to be dialed via their static addresses or via_nostr lookups; open-discovered peers stack on top from the ambient namespace. You can run open discovery with a populated peers: list (the path this tutorial walks, since you're keeping test-us01), with peers: [] for pure ambient discovery, or with a long peers: list and open layered on top to broaden reach.

The namespace is what scopes who's visible to whom:

The namespace is the scope. node.rendezvous.nostr.app defaults to fips-overlay-v1 — the namespace the public test mesh uses. Setting a different value (e.g., app: "my-experiment.v1") carves out a private discovery set: only nodes that share your app value find each other. For this tutorial we stay on the default and join the public namespace.

Open discovery is best-effort by design — not every discovered peer will connect:

Best-effort, by design. Many discovered peers will fail to connect — they may be offline, behind incompatible NAT, running a different protocol version, or have peer ACLs that reject you. That's normal for ambient discovery; connection attempts are best-effort and rate-limited by open_discovery_max_pending (default 64). Your peer list grows over time as candidates land in the cache, not all at once on restart.

The peer ACL is the admission-control surface, separate from discovery:

Open is admission-free under your peer ACL. Open discovery does not bypass the peer ACL — every candidate still has to pass it. By default the ACL accepts everyone, so any publisher in the namespace becomes a connection candidate. If you rely on a non-default ACL for admission control, verify it is set the way you want before enabling policy: open. See ../reference/security.md for the ACL format.

Step 1: Confirm your starting state

You should be coming out of advertise-your-node with:

• Persistent identity, advertising enabled (node.rendezvous.nostr.advertise: true), and either the direct-UDP path (transports.udp.advertise_on_nostr: true, transports.udp.public: true) or the udp:nat path (transports.udp.advertise_on_nostr: true, transports.udp.public: false) from the previous tutorial.
• A static test-us01 peer entry that the daemon dials outbound; possibly an inbound test-us03 peer (the open-discovery test mesh node that dialed in after seeing your advert).

Capture the current peer count for comparison:

sudo fipsctl show peers | grep -c 'npub'

You'll likely see 1 (just test-us01) up to a handful, depending on how many open-discovery test mesh nodes have already dialed you.

Step 2: Switch the discovery policy to open

Open /etc/fips/fips.yaml and find the rendezvous.nostr block. Add (or change) the policy line:

node:
  identity:
    persistent: true
  rendezvous:
    nostr:
      enabled: true
      advertise: true
      policy: open

That's the only change. Notes on what you don't have to touch:

• You don't have to drop the peers: block. Static peers and open-discovered peers coexist; static entries get priority for direct dialing, open-discovered ones are layered on top.
• You don't have to set app. The default fips-overlay-v1 is the namespace the public test mesh uses; staying with the default is what gets you discovered.
• You don't have to set open_discovery_max_pending. The default of 64 is plenty for a tutorial; only tune it if you see the daemon log open-discovery sweep: enqueue budget is 0, skipping (a debug-level line).
• You don't have to rewrite an older node.discovery block. That spelling still parses and logs one deprecation warning naming the new table (see ../reference/configuration.md).

Save the file.

Step 3: Restart and let discovery populate

sudo systemctl restart fips

Give the daemon a minute or two. Open discovery doesn't fire all at once — the daemon subscribes to the relays, accumulates adverts as they arrive (or as relays return historical events), and queues each publisher for a dial attempt.

Step 4: Inspect the discovered peer list

sudo fipsctl show peers

You should see considerably more entries than before:

• test-us01 — still there, still using the configured static dial path.
• test-us03 — same as before (the open-discovery test mesh node that dials you when it sees your advert).
• test-us04, test-de01, test-es01, test-uk01 — the other test mesh nodes; your daemon picked their adverts up from the namespace and dialed them.
• Plus any other operator publishing on fips-overlay-v1 (community nodes, other operators' experiments).

Each entry has its own connectivity state, and every entry that appears here completed a handshake at least once: a peer whose advert was stale, or that NAT traversal never reached, produces no entry at all rather than a failed one. A link heard from within the last heartbeat interval (node.heartbeat_interval_secs, 10 seconds by default) reads connected. One silent for longer reads stale; it still carries traffic, and it reads connected again as soon as the peer is heard from. A link that stays silent until it is declared dead is removed, so its entry disappears rather than changing state.

To get a list of just the connected links:

sudo fipsctl show peers \
  | jq '.peers[] | select(.connectivity == "connected") | .npub'

The peer count will continue to drift over time as adverts expire and new ones arrive. This is steady-state behavior, not a transient.

Step 5: Confirm the mesh-wide reach

You can now reach any of the discovered nodes the same way you reached test-us01 and test-us02 in join-the-test-mesh. Pick one of the new test mesh nodes from the peer list and ping it by its shortname:

ping6 -c 4 test-uk01.fips

(test-uk01 is the United Kingdom test node; the installer's /etc/fips/hosts entry resolves it to the corresponding npub. Substitute any active peer's shortname or full <npub>.fips.)

Expect four replies. The packet path may go through your direct link to that peer (if the open-discovery dial succeeded and the link is up) or via a test-mesh forwarder (if the direct link is down but the destination is still reachable through the mesh). Either way, the npub-as-name addressing works the same way.

What you've learned

• Open discovery is the consume side of full ambient participation. With policy: open plus advertising, your daemon both publishes its advert and consumes everyone else's in the namespace.
• The namespace defines the scope. Everyone publishing with the same app value finds each other; different namespaces are isolated discovery sets.
• Best-effort means failure is normal. Many discovered peers won't actually connect; that's expected and rate-limited by open_discovery_max_pending.
• Static and open coexist. Configured peers keep working with their own dial paths; open-discovered peers stack on top.
• The peer ACL still gates everything. Open is admission-free relative to discovery, not relative to your ACL — the ACL is what you'd use to restrict who can connect if you don't want a fully open posture.

Custom namespaces for private experiments

If you want to use FIPS open discovery for a private set of nodes — colleagues, a workshop cohort, a specific deployment — set a custom app value:

node:
  rendezvous:
    nostr:
      enabled: true
      advertise: true
      policy: open
      app: "my-team.experiment-1"

All nodes participating in the experiment use the same app string. Pick something distinctive — short identifiers risk colliding with other operators' experiments. Once your nodes use a custom app, they no longer find or are found by the public test mesh (the public mesh uses fips-overlay-v1).

Troubleshooting

If your peer list doesn't grow past the inbound peers from the previous tutorial:

• Wait. Open discovery accumulates adverts; the first pass after restart can take a couple of minutes to populate on a new subscription.
• Verify the namespace. With app: unset, the daemon uses fips-overlay-v1. If you set a custom app: for an experiment, your daemon is in a different namespace than the public test mesh and will only find peers using the same value.
• Check relay reachability. Open discovery is a subscription rather than one-shot queries — if the WebSocket connection to the relays is failing repeatedly, no adverts arrive. Look for relay-connection errors in sudo journalctl -u fips -n 200.
• policy: open typo. YAML is case-sensitive, and the policy field is a serde enum that rejects unknown values — a misspelled value produces a config-parse error at startup rather than a silent fall-back. If the daemon refuses to start, check sudo journalctl -u fips -n 200 for the parse-error line naming the field and value.

If too many peers are appearing and you want to dial down:

• Lower open_discovery_max_pending. Default 64; setting it to e.g. 16 caps in-flight connection attempts. Adverts beyond that wait in a queue.
• Use a custom app. Move to a private namespace where only nodes you're coordinating with publish.
• Use the peer ACL. See ../reference/security.md for the ACL format if you want explicit allow/deny rules.

What's next

• Reach services on other mesh nodes. reach-mesh-services drives nc, traceroute6, curl, and ssh at peers by .fips name — any of the open-discovered peers in your list, or any node you reach through them.

• Host a service of your own. host-a-service brings up an HTTP server addressable as <your-npub>.fips, bound to fips0 so the exposure is mesh-only, behind the mesh firewall.

• ground-up-mesh — Bring up two devices on a shared physical link (Ethernet, WiFi, or Bluetooth) with no pre-existing IP infrastructure. The second deployment mode of FIPS, a parallel to the overlay-on-internet path the Nostr-discovery tutorials covered.

For the operator-style scenario reference covering all five shapes of Nostr discovery side-by-side:

• ../how-to/enable-nostr-discovery.md § Capability 3 — open discovery configuration knobs.

For the wire format and discovery design:

• ../reference/nostr-events.md — Kind 37195 advert format and the app namespace tag.
• ../design/fips-nostr-discovery.md — discovery runtime design, security and threat model.