laantungir/fips
1
0
Fork 0
mirror of https://github.com/jmcorgan/fips.git synced 2026-07-30 19:46:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: four-section /docs/ restructure with new-user content, accuracy pass, and gateway feature-set rewrite

Browse Source 
Restructures /docs/ by reader purpose (tutorials, how-to,
reference, design), adds the new-user-progression and
operator-recipe content the prior layout lacked, runs an
accuracy pass against current source across the pre-existing
design docs, and rewrites the gateway feature-set documentation
end-to-end around its actual operational profile (a niche
feature designed for systems already serving DHCP/DNS to a
LAN, with two independent halves — outbound LAN→mesh, inbound
mesh→LAN — sharing one nftables table, one binary, and one
control socket). Top-level README and getting-started rewritten
around two equally-weighted deployment modes (overlay on
existing IP networks; ground-up over non-IP transports).

## Additions

- 11 new tutorials in docs/tutorials/: an 8-step new-user
  progression from single-daemon test-mesh peering through
  to a ground-up two-device mesh, an IPv6-adapter side-trip
  walkthrough, an Advanced Tutorials index, and a hand-held
  OpenWrt walk-through for fips-gateway deployment that
  exercises both halves of the feature.
- 12 new how-tos in docs/how-to/: firewall activation,
  Nostr discovery (resolve / advertise / open across five
  scenarios), Tor onion (directory + control_port modes),
  UDP buffer tuning, unprivileged-user setup, persistent
  identity, host aliases, Bluetooth LE peering, MTU
  diagnostics, manual Linux-host gateway deployment (covers
  both halves), gateway troubleshooting (organised by half),
  and a section index.
- 9 new reference docs in docs/reference/: configuration,
  wire formats, control-socket protocol, four CLI references
  (fips, fipsctl, fipstop, fips-gateway), security posture
  matrix, and Nostr events catalog. Configuration and
  wire-formats are renamed-and-extended from prior design/
  versions; the other seven are net-new.
- 6 new design docs: fips-concepts, fips-architecture, and
  fips-prior-work split out of the deleted fips-intro.md;
  consolidated fips-mmp and fips-mtu aggregations; and a
  new generic port-advertisement-and-nat-traversal doc
  (Nostr-signaled port advertisement plus UDP NAT-traversal
  protocol, FIPS as an example implementation, suitable for
  eventual NIP submission).
- Top-level docs/getting-started.md walking through the
  binary-installer-only Install story.
- packaging/common/hosts pre-populated with the eight public
  test-mesh nodes so shortnames resolve out of the box on
  every fresh install.

## Changes

- 23 wire-format diagrams relocated to reference/diagrams/
  alongside the wire-formats move.
- 4 design diagrams corrected against source code
  (fips-protocol-stack, fips-identity-derivation,
  fips-coordinate-discovery, fips-routing-decision).
- 10 pre-existing design docs reconciled with current
  source. Numeric corrections: stale link-MMP report bounds
  (now [1s, 5s] with 200 ms cold-start floor); UDP default
  MTU (now 1280, IPv6 minimum); node_addr formula
  (SHA-256(pubkey)[..16]); Noise patterns (IK at link, XK
  at session); peer-ACL semantics (strict allowlist requires
  ALL in peers.deny); daemon DNS upstream ([::1]:5354);
  on-the-wire bloom-filter size (1,071 bytes); obsolete
  Cargo-feature references (PR #79 dropped them) removed.
- Transport framing tightened across the docs: TCP is for
  UDP-filtered networks (not NAT traversal); Tor is a
  deployment mode (not failover); WebSocket dropped (not a
  shipped FIPS transport); WiFi promoted to Implemented via
  Ethernet in infrastructure mode; classic-Bluetooth row
  removed (BLE is the only Bluetooth-mode transport).
- docs/design/fips-gateway.md rewritten end-to-end to lead
  with the niche-feature framing and the two-halves
  structure. Title moved from "FIPS Outbound LAN Gateway"
  to "FIPS Gateway"; architecture section describes the
  common machinery (the fips-gateway service, the nftables
  table, the control socket) before splitting into separate
  "Outbound Half" and "Inbound Half" sections of equal
  weight; security considerations split per-half; no Future
  Work section (speculative directions live in the project
  tracker, not in protocol design docs). Inbound port
  forwarding is a first-class half rather than a buried
  "Implemented Extensions" subsection.
- Gateway terminology unified across all gateway docs as a
  separate Linux service running alongside the fips daemon
  (its own systemd unit / OpenWrt init script). Container-
  pattern terms (sidecar) are reserved for the
  Docker/Kubernetes sidecar deployment examples — the
  testing/sidecar/ tree, examples/k8s-sidecar/,
  examples/sidecar-nostr-relay/,
  examples/wireguard-sidecar-macos/, and the related
  CHANGELOG / top-level README entries — where the term
  carries its standard container meaning.
- Net-new design body content: rekey section in
  fips-mesh-layer (Noise IK msg1/msg2 over the established
  link, K-bit cutover, drain window, smaller-NodeAddr-wins
  tie-breaker on dual-init); Mesh Size Estimation and
  Antipoison FPR Cap sections in fips-bloom-filters;
  Mesh-Interface Query Filter subsection in
  fips-ipv6-adapter; failure-suppression knobs and clock-
  skew tolerance in fips-nostr-discovery; loop-rejection
  and mid-chain ancestor swap added to spanning-tree
  propagation / stability rules; Priority Chain in
  fips-mesh-operation renumbered to match the
  routing-decision diagram.
- Top-level README: dropped the stale nostr-discovery
  cargo-feature parenthetical. docs/README.md and the four
  section READMEs (tutorials, how-to, reference, design)
  refreshed for the new structure; index rows reflect both
  halves of the gateway feature and the new fips-gateway
  CLI reference.
- Cargo.toml [package.metadata.deb] assets path updated for
  the fips-security.md move; .gitignore /reference/ rule
  anchored to repo root so docs/reference/ is trackable.
- packaging/openwrt-ipk/files/etc/fips/fips.yaml
  configuration-doc URL updated to the new
  docs/reference/configuration.md location.

## Deletions

- docs/design/fips-intro.md (split into the three new intro
  design docs).
- docs/design/document-relationships.svg (orphan, no longer
  referenced).
- docs/proposals/ tree removed; the only proposal it
  contained (the Nostr UDP hole-punch protocol) was
  rewritten as the new generic
  design/port-advertisement-and-nat-traversal.md.
This commit is contained in:
Johnathan Corgan
2026-05-08 03:02:12 +00:00
parent 4cdf382038
commit 5abf9a9325
91 changed files with 11474 additions and 3278 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tutorials/README.md
+73
View File
@@ -0,0 +1,73 @@
# Tutorials
If you have just installed FIPS, this is where to start. The
tutorials below take you from a freshly-installed daemon to a
node that:
- Has joined the public test mesh and can reach other nodes on it.
- Carries a stable identity that other operators can address.
- Discovers peers — and is discoverable — over Nostr.
- Hosts and consumes real services across the mesh.
Each tutorial is a complete, working session at the keyboard. You
configure something, restart the daemon, watch it come up, and
verify the result. The point is to build muscle memory, not to
cover every option.
> **Read them in order.** Each tutorial assumes the state the
> previous one left you in. If you skip ahead, the cross-references
> that lead you back may not match what you have on disk.
## The new-user progression
| # | Tutorial | What you'll do |
| - | -------- | -------------- |
| 1 | [join-the-test-mesh.md](join-the-test-mesh.md) | Add one public test peer to your config, watch the link come up, ping that peer and a second mesh node it routes you to. The starting point for everything else. |
| 2 | [persistent-identity.md](persistent-identity.md) | Pin your daemon to a stable Nostr keypair so your address stops changing on every restart. Other operators can now add you to their `peers:` lists; the services you host get a fixed name. |
| 3 | [resolve-peers-via-nostr.md](resolve-peers-via-nostr.md) | Stop hard-coding peer addresses. Drop the address line from your peer entry and let the daemon look up the current endpoint from public Nostr relays at dial time. |
| 4 | [advertise-your-node.md](advertise-your-node.md) | Publish your own UDP endpoint to Nostr so any operator who knows your npub can reach you, with a short final section on `udp:nat` traversal for symmetric-NAT networks. |
| 5 | [open-discovery.md](open-discovery.md) | Switch to `policy: open` and let your peer list populate itself from the ambient `fips-overlay-v1` namespace. Hands-off mesh participation. |
| 6 | [reach-mesh-services.md](reach-mesh-services.md) | Drive ordinary IPv6 tools — `ping6`, `nc`, `traceroute6`, `curl`, `ssh` — at mesh nodes by `<npub>.fips`. Get a feel for the daemon's IPv6 adapter, which makes unmodified IPv6 software work over the mesh. |
| 7 | [host-a-service.md](host-a-service.md) | Bring up an HTTP server bound to `fips0` so mesh nodes can reach it, with a deliberate exposure decision (mesh-only vs every interface), and the mesh firewall as a default-deny baseline. The peer ACL (a separate, transport-layer control over which npubs may peer with your node) is briefly mentioned alongside. |
| 8 | [ground-up-mesh.md](ground-up-mesh.md) | Bring up a second deployment mode: two devices joined by Ethernet (or WiFi, or BLE) with no IP infrastructure between them. The mesh emerges from layer 2 up. Coexists with overlay peers — the same daemon can carry both. |
After tutorial 8 you have a fully participating mesh node that
reaches services hosted by other mesh nodes and hosts services of its own,
with identity, discovery, reachability, an explicit exposure
policy, and an understanding of both deployment modes — overlay
on top of existing IP, and ground-up where the mesh is the
network.
There is also a side trip you can take any time after tutorial 1:
- [ipv6-adapter-walkthrough.md](ipv6-adapter-walkthrough.md) —
trace one `ssh` from DNS query through session setup to the
far-side TUN, using `fipstop` and `fipsctl` to watch each step.
Optional, but if you like seeing how the pieces fit together,
this is the doc that shows you.
## Advanced
These are not part of the new-user progression. They assume you
have already worked through the tutorials above and now want to
fold FIPS into a wider network deployment.
- [deploy-fips-gateway.md](deploy-fips-gateway.md) — Stand up a
`fips-gateway` on an OpenWrt access point so unmodified LAN
hosts can reach `<npub>.fips` destinations through a DNS-
allocated virtual IPv6 pool and kernel nftables NAT, with no
per-host FIPS install. Also walks through one inbound port
forward exposing a LAN service to mesh peers. Aimed at
operators bridging a LAN segment into the overlay from the
edge router. For a non-OpenWrt host the same deployment is
in [../how-to/deploy-gateway.md](../how-to/deploy-gateway.md).
## When to use the how-to guides instead
The tutorials here walk through one specific path each. The
how-to guides under [../how-to/](../how-to/) are the operator
recipes — alternative provisioning paths, less-common
configurations, troubleshooting techniques. Once you have the
shape of FIPS in your head from these tutorials, the how-tos are
where you'll go to look up "how do I do X?" without being walked
through the surrounding context.
docs/tutorials/advertise-your-node.md
+413
View File
@@ -0,0 +1,413 @@
# Advertise Your Node on Nostr
After
[resolve-peers-via-nostr](resolve-peers-via-nostr.md) your
daemon can look up a peer's current endpoint by npub. This
tutorial flips it around: you publish a signed advert listing
your own endpoint(s), so any other operator who knows your
npub can dial you the same way you dialed `test-us01`.
The whole exercise should take about ten minutes if you have
a public IP or full-cone home NAT. A short final section
covers the alternative path for symmetric-NAT networks.
## What you'll build
```text
┌───────────────────────┐
│ your fips daemon │
│ persistent npub │
└──────────┬────────────┘
│ signed advert (Kind 37195)
│ { udp:<your-public-ip>:2121, ... }
│ refreshes every 30 min
▼
┌──────────────────────────────────────────┐
│ Nostr relays │
│ relay.damus.io / nos.lol / offchain.pub│
└──────────────────────┬───────────────────┘
│
│ "what's <your-npub>'s endpoint?"
│
┌───────────┴───────────┐
│ another fips daemon │
│ knows your npub, │
│ via_nostr: true │
└───────────────────────┘
```
You will change two things in `/etc/fips/fips.yaml`:
- Flip `discovery.nostr.advertise` from `false` to `true`.
- Add `advertise_on_nostr: true` and `public: true` under
`transports.udp`.
After restart, your daemon publishes a Kind 37195 event tied
to your npub, listing the UDP endpoint other peers should
dial.
## How advertising works
> **Adverts are signed Nostr events.** Every advert is a Kind
> 37195 event signed by your daemon's secret key. Anyone
> reading it can verify the advert really came from the npub
> claiming the endpoint. The advert is the `(npub → current
> endpoints)` mapping, signed and published.
The advert lists transports the daemon is willing to expose,
and only those:
> **Endpoints are opt-in per transport.** Only transports
> with `advertise_on_nostr: true` are listed in your advert.
> Transports without that flag stay private — they still
> work for peers who reach you via static config, but they
> won't appear in your published advert.
For UDP specifically, the daemon needs to know what IP and
port to put in the advert:
> **Determining the advertised endpoint (wildcard-bound UDP).**
> With UDP bound to a wildcard like `0.0.0.0:2121`, the daemon
> doesn't know its own public IP at startup. You have two ways
> to tell it what to put in the advert:
>
> - `public: true` — daemon does a one-shot STUN observation
> against the configured STUN servers and uses the reflexive
> IPv4 it learns. Right when your public IP is dynamic or
> you'd rather not pin it in config. Works for nodes with a
> directly-bound public IP and for nodes behind full-cone
> NAT (most home routers).
> - `external_addr: "<ip>[:<port>]"` — explicit override.
> Right when you already know your public IP — a static
> residential IP, an Elastic IP behind 1:1 NAT, a cloud
> instance whose advertised port differs from the bind
> port — and you don't want to depend on STUN reachability.
> Required for TCP on cloud setups where binding directly
> to the public IP returns `EADDRNOTAVAIL`.
>
> If you bind UDP to a specific public IP rather than
> `0.0.0.0`, neither flag is needed — the daemon advertises
> whatever it's bound to.
Adverts don't sit on the relays forever:
> **TTL and refresh.** Adverts have a 1-hour expiration
> (NIP-40 `expiration` tag) and the daemon re-publishes every
> 30 minutes. If your daemon goes offline, your advert decays
> from caches in roughly an hour and consumers stop trying.
## Step 1: Confirm your starting state
You should be coming out of
[resolve-peers-via-nostr](resolve-peers-via-nostr.md) with:
- A persistent npub (`fipsctl show status | grep '"npub"'`).
- Nostr discovery in consume-only mode
(`discovery.nostr.enabled: true`,
`discovery.nostr.advertise: false`).
- A peer entry for `test-us01` with `via_nostr: true` and no
static address. `fipsctl show peers` shows the link
established.
If any of those isn't true, finish the previous tutorials
first.
Capture your npub now — you'll need it for the verification
step:
```sh
sudo fipsctl show status | grep '"npub"'
```
Copy the value.
## Step 2: Enable advertising in the config
Open `/etc/fips/fips.yaml` and change two things.
**Change 1: flip `advertise` to `true`.** Find the
`discovery.nostr` block under `node:` and set:
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
```
(The previous tutorial set `advertise: false`; you're flipping
that bit now.)
**Change 2: add the UDP advert flags.** Find the `udp:` block
under `transports:`. The wildcard-bind default
(`0.0.0.0:2121`) means the daemon needs help knowing what to
advertise — pick one of the two approaches from the callout
above.
If you want STUN auto-discovery (works for full-cone NATs and
nodes with a directly-bound public IP):
```yaml
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: true
```
If you already know your public IP (e.g., a static residential
IP or a cloud Elastic IP behind 1:1 NAT) and want to skip the
STUN dependency:
```yaml
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
external_addr: "203.0.113.45:2121"
```
Replace `203.0.113.45:2121` with your actual public IP and
port. The bare-IP form `external_addr: "203.0.113.45"` is also
accepted; the daemon combines it with the bind port. You may
set both `public: true` and `external_addr` together — the
explicit override wins, with STUN as a logging cross-check.
`advertise_on_nostr: true` is the bit that says "include this
transport in my published advert" — common to both paths.
Save the file.
## Step 3: Restart the daemon
```sh
sudo systemctl restart fips
sudo systemctl status fips
```
Status should show `active (running)`. Within a few seconds the
daemon will:
1. Run a one-shot STUN observation against the default STUN
servers to learn its public IP.
2. Build a Kind 37195 advert listing
`udp:<public-ip>:2121` (and any other transports you have
`advertise_on_nostr: true` on).
3. Sign the advert with the daemon's nsec.
4. Publish it to the three default advert relays.
5. Schedule a refresh every 30 minutes.
If STUN fails (for example, if the network blocks outbound
UDP/3478), the daemon emits a WARN line in the journal and
suppresses the UDP entry from the advert rather than publishing
a wrong address. The link to `test-us01` from the previous
tutorial keeps working regardless — only the publish side is
gated on STUN.
Quick sanity check on the journal:
```sh
sudo journalctl -u fips -n 200 | grep -iE 'STUN|advert|warn' | head -20
```
If you see `WARN` lines mentioning STUN or wildcard-bind
fallthrough, jump to [Troubleshooting](#troubleshooting); the
rest of the tutorial assumes the publish succeeded.
## Step 4: Verify your advert is on the network
The advert is a public Nostr event — anyone, including you,
can fetch it. With the `nak` Nostr CLI installed, query the
relays for adverts published by your npub:
```sh
nak req -k 37195 -d "fips-overlay-v1" \
-a $(nak decode <your-npub> | jq -r .pubkey) \
--limit 1 wss://relay.damus.io
```
Replace `<your-npub>` with the npub you copied in Step 1. The
inner `nak decode` converts your bech32 npub to the hex pubkey
the relay filter expects.
Expect one event back. The interesting fields:
- `pubkey` — your npub in hex form.
- `tags` — includes `["d","fips-overlay-v1"]` (the namespace),
`["protocol","fips-overlay-v1"]`, and an `["expiration", …]`
tag set ~1 hour in the future.
- `content` — JSON listing the `endpoints` array. You should
see one entry like:
```json
{"transport":"udp","addr":"<your-public-ip>:2121"}
```
That `<your-public-ip>` is what STUN learned. Confirm it
matches what you'd expect for your network — for a home node,
it should be your residential IP, not a `192.168.x.x` LAN
address.
## Step 5: Watch for inbound connections
Your advert is now consumable by any FIPS daemon running open
discovery on the same `fips-overlay-v1` namespace. The public
test mesh nodes do exactly this — they subscribe to all
adverts in the namespace and try to dial new publishers.
Within a minute or two of restart, run:
```sh
sudo fipsctl show peers
```
In addition to your configured `test-us01` peer, you may see
an entry for `test-us03` (the open-discovery test mesh node).
It will have `connectivity` active and its own
`transport_addr`. This peering appeared without you
configuring anything — the test-mesh open-discovery node saw
your advert, dialed the endpoint, and Noise IK established
the link.
If no inbound peers appear, that's not necessarily a failure
of advertising — it just means no one has consumed your advert
*and* dialed back yet. The advert is on the relays regardless,
verifiable in Step 4.
## What you've learned
- **Adverts are publish + sign.** Every running FIPS daemon
with `advertise: true` publishes a signed advert; reading it
is one Nostr event lookup.
- **Endpoint inclusion is per-transport.** Only the transports
you set `advertise_on_nostr: true` on appear in the advert.
- **`public: true` invokes STUN.** Wildcard-bound UDP with
`public: true` runs a one-shot STUN observation to learn
its public IP.
- **Refresh is automatic.** Adverts re-publish every 30
minutes; consumers cache them with a 1-hour staleness
bound.
- **The publish side stands alone.** Once your advert is on
the relays, peers can dial you whether you're advertising
to them specifically or not. The test mesh's open-discovery
nodes will pick you up automatically.
## If you're behind symmetric NAT
`public: true` + STUN works on most home and office NATs (the
full-cone variety) and on nodes with a directly-bound public
IP. It does *not* work on symmetric NAT, where the NAT mapping
is keyed on (source-port, destination-host) so the IP/port
your STUN server saw isn't the IP/port a different peer would
see.
For symmetric-NAT networks the alternative is `udp:nat` mode,
which advertises a placeholder `udp:nat` endpoint along with
the daemon's signaling-relay and STUN-server lists, and
performs UDP hole-punching at dial time. Both sides need to be
running matching configs and at least one side needs a
non-symmetric NAT for the punch to succeed; symmetric on both
sides is not reliably traversable and will time out.
The minimal config switch:
```yaml
transports:
udp:
bind_addr: "0.0.0.0:2121"
advertise_on_nostr: true
public: false # ← was true; change to false
```
And add the signaling/STUN block under `discovery.nostr`:
```yaml
discovery:
nostr:
enabled: true
advertise: true
dm_relays:
- "wss://relay.damus.io"
- "wss://nos.lol"
stun_servers:
- "stun:stun.l.google.com:19302"
- "stun:stun.cloudflare.com:3478"
```
For the full setup including peer-side config and the punch-
duration knob, see
[../how-to/enable-nostr-discovery.md § Capability 2c](../how-to/enable-nostr-discovery.md#sub-scenario-2c-udp-hole-punching-for-nodes-behind-nat).
Separately from NAT considerations, FIPS supports running a
node behind a Tor onion service as a deployment shape in its
own right — chosen for the privacy, anonymity, and
censorship-resistance properties it brings, not as a fallback
when UDP or TCP fail. If those properties are an independent
goal for your node, see
[../how-to/enable-nostr-discovery.md § Sub-scenario 2b](../how-to/enable-nostr-discovery.md#sub-scenario-2b-tor-onion-node)
and
[../how-to/deploy-tor-onion.md](../how-to/deploy-tor-onion.md).
## Troubleshooting
If your advert doesn't appear on the relays:
- **STUN failed.** Check the journal for WARN lines mentioning
STUN or wildcard-bind. The most common causes are outbound
UDP/3478 blocked or DNS for `stun.l.google.com` failing.
Try: `dig stun.l.google.com` and
`nc -uvz stun.l.google.com 19302` to verify reachability.
- **Wrong public IP advertised.** If `nak` shows your advert
with a non-public address (e.g., `10.x.x.x` or
`192.168.x.x`), STUN didn't see your real public IP — likely
you're behind a CGNAT that NATs your STUN traffic too, or a
corporate firewall that proxies it. Switch to the
`external_addr` form from Step 2 with your actual public
IP, or replace `public: true` with the bound interface IP
directly under `bind_addr`.
- **Relay reachability.** `nak req` against a relay you can
reach but no events return — possibly the publish failed
silently because the daemon couldn't connect to that
specific relay. Try the other two:
```sh
nak req ... wss://nos.lol
nak req ... wss://offchain.pub
```
- **`advertise_on_nostr` typo.** YAML is case-sensitive and
silently ignores unknown keys. If `nak` returns no advert at
all, double-check the spelling on the UDP block and that
`discovery.nostr.advertise: true` is also set.
## What's next
- **Open discovery.**
[open-discovery](open-discovery.md) flips the consume side
symmetric — switch your daemon to `policy: open` and watch
your peer list populate from the ambient
`fips-overlay-v1` namespace, the same mechanism
`test-us03` is using right now to find you.
- **Host a service of your own.**
[host-a-service](host-a-service.md) walks through bringing up
an HTTP server addressable as `<your-npub>.fips`, the same
way the connecting node now reaches `test-us01`. The natural
follow-on now that other operators can dial you by npub.
For the operator-style scenario reference covering all five
shapes of Nostr discovery side-by-side (consume-only,
publish-direct, publish-Tor, NAT traversal, open):
- [../how-to/enable-nostr-discovery.md](../how-to/enable-nostr-discovery.md)
For the wire-format and discovery design:
- [../reference/nostr-events.md](../reference/nostr-events.md)
— Kind 37195 advert format, Kind 21059 traversal signaling.
- [../design/fips-nostr-discovery.md](../design/fips-nostr-discovery.md)
— discovery runtime design, security and threat model.
docs/tutorials/deploy-fips-gateway.md
+468
View File
@@ -0,0 +1,468 @@
# Deploy a `fips-gateway` on an OpenWrt AP
In every other tutorial in this set you put FIPS *on* the host that
needs to talk to the mesh. This one is the exception. Here you stand
up a `fips-gateway` on an OpenWrt access point so the unmodified LAN
behind it — phones, laptops, smart-home gear — can reach mesh
destinations by `<npub>.fips` without any FIPS software of their own,
and so a service running on a LAN box can be exposed to mesh peers
through a port forward. Two halves, one binary, one config.
This is **advanced** material. It assumes you have already worked
through [join-the-test-mesh](join-the-test-mesh.md) on some other
machine, you understand what `<npub>.fips` means, and now you want to
fold an existing LAN into the mesh from the edge router rather than
installing FIPS on every device. The whole exercise should take about
forty-five minutes.
## What you'll build
```text
┌──────────────────────────────────────────────────────────────┐
│ OpenWrt access point │
│ │
│ br-lan ┌──────────────┐ fips0 ┌──────────────┐ │
│ (LAN side) ◀──│ fips-gateway │──────────▶│ fips daemon │ │
│ │ (service) │ │ │──┼─▶ mesh
│ │ │ fd97:.. │ fd97:.. │ │
│ │ fd01::/112 │ │ │ │
│ │ pool │ │ │ │
│ └──────┬───────┘ └──────────────┘ │
│ │ │
│ ▼ │
│ nftables NAT │
│ (inet fips_gateway) │
└─────────────────┬────────────────────────────────────────────┘
│ br-lan
▼
┌──────────────────────────────────────────────┐
│ LAN clients (phones, laptops, smart-home) │
│ │
│ no FIPS install — just IPv6 + DNS │
│ ▲ │
│ │ curl http://test-us01.fips/ │
│ └────── DNS to dnsmasq ──▶ gateway DNS │
│ │
└──────────────────────────────────────────────┘
```
By the end you will have:
- An OpenWrt AP whose LAN clients can fetch `http://test-us01.fips/`
with no FIPS software installed on them.
- An inbound port forward exposing one LAN service to the mesh as
`<your-gateway-npub>.fips:<port>`.
- An understanding of which LAN-side glue OpenWrt automates for you
(DNS forwarding, RA route, IPv6 prefix) and which the operator owns
(port forwards, mesh firewall).
## Why an OpenWrt AP
The gateway has very specific dependencies on the box it runs on. It
needs to own DNS for the LAN, it needs to advertise an IPv6 route to
the LAN, it needs a stable LAN-side interface, and it needs to be the
default IPv6 router for the segment. An OpenWrt-based access point
already does all of those things — it runs `dnsmasq`, it runs
`odhcpd` for IPv6 RA, it owns `br-lan`, and clients are already using
it as their gateway. The OpenWrt ipk leans into that: the `gateway:`
block in `/etc/fips/fips.yaml` is pre-populated, and the
`/etc/init.d/fips-gateway` init script wires up the LAN-side glue
automatically when you start the service.
On a non-OpenWrt host the same integration is manual; that path is
covered by [../how-to/deploy-gateway.md](../how-to/deploy-gateway.md).
## Prerequisites
- An OpenWrt 22.03+ AP serving DHCP and DNS to a wired or wireless
LAN segment.
- The FIPS ipk installed and a working FIPS daemon on the AP. If you
haven't done that yet, follow
[../../packaging/openwrt-ipk/README.md](../../packaging/openwrt-ipk/README.md)
for the install, then come back here.
- The AP joined to the mesh — at least one healthy peer link. If it
isn't, work through [join-the-test-mesh](join-the-test-mesh.md) on
the AP first.
- Root SSH to the AP. Every command in this tutorial runs on the AP.
- A LAN client (phone, laptop) to test the outbound half from.
## Step 1: Verify the FIPS daemon is up
Confirm the daemon is running, the TUN is up, and at least one mesh
peer is reachable:
```sh
service fips status
ip -6 addr show fips0
fipsctl show peers
```
You should see:
- `service fips status` reports `running`.
- `fips0` exists and has one `inet6 fd97:...` address. That is the
AP's mesh-side identity.
- `fipsctl show peers` lists at least one peer with active
connectivity (not `idle` / not zero bytes).
Confirm the AP can resolve a known mesh node by name:
```sh
ping6 -c 2 test-us01.fips
```
If any of these fail, fix the daemon side first — the gateway is a
separate service that runs alongside a working daemon, not a
substitute for one.
## Step 2: Inspect the pre-populated gateway config
The OpenWrt ipk ships `/etc/fips/fips.yaml` with the `gateway:` block
already filled in. View the relevant section:
```sh
sed -n '/^gateway:/,$p' /etc/fips/fips.yaml
```
You will see roughly:
```yaml
gateway:
enabled: true
pool: "fd01::/112" # virtual IP range (up to 65535 addresses)
lan_interface: "br-lan" # LAN-facing interface for proxy NDP
dns:
listen: "[::1]:5353" # gateway DNS listener (dnsmasq forwards here)
upstream: "[::1]:5354" # FIPS daemon DNS resolver (matches daemon default)
ttl: 60 # DNS TTL and mapping lifetime (seconds)
pool_grace_period: 60 # seconds after last session before reclaiming
```
Three things to notice:
- `pool: "fd01::/112"` — the virtual-IP CIDR the gateway hands out
to LAN clients. 65 536 addresses, the gateway's hard cap. Pick a
different `fdXX::/N` prefix if `fd01::/112` collides with anything
on your network.
- `lan_interface: "br-lan"` — the OpenWrt LAN bridge. The gateway
installs proxy-NDP entries on this interface so LAN clients can
ARP-equivalent for pool addresses.
- `dns.listen: "[::1]:5353"` — the gateway's DNS listener is bound
to IPv6 loopback only. dnsmasq, which owns LAN port 53, forwards
`.fips` queries to it. The init script wires up that forwarding;
you don't bind to a LAN address yourself.
For the full reference, see
[../reference/configuration.md § Gateway](../reference/configuration.md#gateway-gateway).
## Step 3: Enable and start fips-gateway
The service is shipped disabled — enable it once and start it:
```sh
service fips-gateway enable
service fips-gateway start
```
Behind that single command, the init script
(`/etc/init.d/fips-gateway`) does five things:
1. **Loads gateway sysctls.** `net.ipv6.conf.all.proxy_ndp=1` and
`net.ipv6.conf.all.forwarding=1` from
`/etc/sysctl.d/fips-gateway.conf`.
2. **Reconfigures dnsmasq via UCI** so `.fips` queries arriving at
the LAN's port 53 are forwarded to the gateway's loopback
listener on port 5353 instead of going straight to the daemon's
resolver on port 5354. (Dnsmasq still owns 53; the gateway sits
in front of the daemon for `.fips` only.)
3. **Adds a global-scope IPv6 prefix** to `br-lan`. Without a
non-ULA address on the local interface, Android and Chrome
suppress AAAA queries entirely — they assume the LAN has no
real IPv6 and don't bother. The init script adds a small
benchmarking-range prefix to convince them otherwise.
4. **Adds an RA route for the virtual pool.** A UCI `route6` entry
under `dhcp` tells `odhcpd` to advertise the pool CIDR via Router
Advertisement (RFC 4191), so LAN clients learn how to reach pool
addresses automatically. No per-client static routes needed.
5. **Spawns `fips-gateway` under procd** with `--config
/etc/fips/fips.yaml`, with crash-respawn.
Verify it is running:
```sh
service fips-gateway status
logread | grep fips-gateway | tail
```
Expect a `running` status and a startup log line of the form
`fips-gateway 0.x.y starting`, followed by entries for DNS bind, NAT
table install, and pool initialisation.
> **What just changed on the LAN.** The AP is now offering two
> things it wasn't offering a moment ago: AAAA records under `.fips`
> that resolve to virtual IPs in `fd01::/112`, and a route to that
> CIDR in its Router Advertisements. Existing LAN clients pick both
> up the next time they re-resolve a name and the next time `odhcpd`
> sends an RA, respectively. No reboot required on the client side.
## Step 4: Test the outbound half from a LAN client
From a phone or laptop on the AP's LAN — anything that does IPv6 and
DNS, with no FIPS software installed — try one of the public test
mesh nodes:
```sh
dig test-us01.fips AAAA
ping6 -c 4 test-us01.fips
curl -6 http://test-us01.fips/
```
Expectations:
- `dig` returns an AAAA in `fd01::...`, **not** `fd97:...`. The
`fd01:` address is the gateway's virtual-IP allocation; the LAN
client never sees the raw mesh address.
- `ping6` succeeds. ICMPv6 echo travels through the NAT pipeline and
back.
- `curl` fetches the page (whatever the test mesh is currently
serving on `test-us01`).
> **What just happened end to end.** Your client asked dnsmasq for
> `test-us01.fips`. Dnsmasq forwarded the query to the gateway's
> loopback listener on port 5353. The gateway forwarded the query on
> to the daemon's resolver on port 5354. The daemon answered with
> `test-us01`'s mesh address (`fd97:...`). The gateway allocated a
> virtual IP from `fd01::/112`, installed nftables DNAT/SNAT/
> masquerade rules pinning that virtual IP to the mesh address,
> installed a proxy-NDP entry on `br-lan` so the client could resolve
> the virtual IP at the link layer, and returned the virtual IP in
> the AAAA reply. Your client then routed traffic to the virtual IP
> via the RA-advertised pool route, the AP's kernel rewrote the
> destination to the mesh address, and the daemon's adapter carried
> the packets across the mesh. Return traffic followed conntrack
> back. The client never knew the mesh existed.
## Step 5 (optional): Inspect the gateway state
The gateway exposes its own control socket separate from the daemon's.
Two useful queries:
```sh
echo '{"command":"show_gateway"}' | nc -U /run/fips/gateway.sock
echo '{"command":"show_mappings"}' | nc -U /run/fips/gateway.sock
```
`show_gateway` reports pool utilisation, the DNS listen address,
uptime, and the conntrack/NAT counters. `show_mappings` lists each
allocated virtual IP, the mesh address it points at, the DNS name
that triggered the allocation, and the mapping's lifecycle state
(`Allocated`, `Active`, `Draining`).
For the full command catalog and JSON shapes, see
[../reference/control-socket.md § Gateway Command Catalog](../reference/control-socket.md#gateway-command-catalog).
The same data is rendered visually in the **Gateway** tab of
[`fipstop`](../reference/cli-fipstop.md).
If you want to see the kernel rules the gateway installed:
```sh
nft list table inet fips_gateway
```
You will see DNAT, SNAT, and masquerade chains populated with one
rule per active mapping.
## Step 6: Add an inbound port-forward for a LAN service
The outbound half is the steady-state use of a gateway. The inbound
half — exposing a LAN service to mesh peers — is a separate decision,
configured per service under `gateway.port_forwards[]`.
For the worked example, run a one-page static web server on the AP
itself, bound to its `br-lan` address, and expose it to the mesh
through a port-forward. Anything would do — the point of the exercise
is the port-forward, not the service. We use what is already on the
AP: `busybox httpd`. In a real deployment the LAN-side target would
typically be a separate host (a NAS, a home server, a dev box on the
LAN); the rule shape is identical.
Find the AP's `br-lan` IPv6 address and save it for the rest of the
step:
```sh
BR_LAN_ADDR=$(ip -6 addr show br-lan \
| awk '/inet6 fd|inet6 2/ && !/scope link/ {print $2}' \
| head -1 | cut -d/ -f1)
echo "$BR_LAN_ADDR"
```
Pick from the global-scope benchmarking prefix the init script added
in Step 3, or your own ULA if `br-lan` has one — anything except a
link-local `fe80::/10` address.
Set up a one-file docroot and start a foreground `busybox httpd`
bound to that LAN address on port 8000:
```sh
mkdir -p /tmp/mesh-demo
echo '<h1>Hello from the mesh-gateway demo</h1>' > /tmp/mesh-demo/index.html
busybox httpd -f -p "[${BR_LAN_ADDR}]:8000" -h /tmp/mesh-demo
```
Leave it running in this shell. Open a second SSH session on the AP
to add the port-forward.
Edit `/etc/fips/fips.yaml`. Inside the existing `gateway:` block,
add a `port_forwards:` list:
```yaml
gateway:
enabled: true
pool: "fd01::/112"
lan_interface: "br-lan"
dns:
listen: "[::1]:5353"
upstream: "[::1]:5354"
ttl: 60
pool_grace_period: 60
port_forwards:
- listen_port: 8080
proto: tcp
target: "[<BR_LAN_ADDR>]:8000"
```
Substitute the real address for `<BR_LAN_ADDR>`. The IPv6 form
(`[addr]:port`) is required — IPv4 targets are rejected at config
load.
Restart the gateway so it re-reads the config:
```sh
service fips-gateway restart
```
From any *other* mesh node, fetch the demo page through the gateway
using the AP's npub:
```sh
# on the AP, get the npub:
NPUB=$(cat /etc/fips/fips.pub)
echo "$NPUB"
```
Then on the remote mesh node:
```sh
curl -6 "http://${NPUB}.fips:8080/"
```
Expect:
```text
<h1>Hello from the mesh-gateway demo</h1>
```
The connection landed on the gateway's `fips0` ingress on TCP/8080,
nftables DNAT rewrote the destination to `[BR_LAN_ADDR]:8000`,
LAN-side masquerade rewrote the source so `busybox httpd` saw a
LAN-routable address, and the response retraced via conntrack.
> **What you exposed.** With the port-forward active, every mesh
> peer that can route to your AP can hit `${NPUB}.fips:8080/` and
> reach this service. That is exactly what the inbound half is
> for — but if you want to scope visibility to a specific subset
> of peers, the FIPS mesh firewall is the layer that does it; see
> [../how-to/enable-mesh-firewall.md](../how-to/enable-mesh-firewall.md).
> The port-forward rule and the firewall rule are independent: the
> port-forward installs the rewrite; the firewall decides who is
> allowed to reach the listen port.
## Step 7: Tidy up
In the first shell, stop `busybox httpd` with `Ctrl-C`. The demo
docroot at `/tmp/mesh-demo` can stay — it is wiped on reboot — or
remove it now (`rm -rf /tmp/mesh-demo`).
If you want to keep the outbound half but withdraw the inbound
forward, remove the `port_forwards:` entry from `/etc/fips/fips.yaml`
and `service fips-gateway restart`. The mesh-side listener disappears
and so does the corresponding nftables rule.
To turn the gateway off entirely:
```sh
service fips-gateway stop
service fips-gateway disable
```
The init script's `stop_service` handler reverses the LAN-side
integration on the way out: dnsmasq's `.fips` forwarder is pointed
back at the daemon's port 5354, the RA route for the pool is
withdrawn from `odhcpd`, and the global-scope IPv6 prefix on
`br-lan` is removed. The LAN reverts to the state it was in before
you ran `service fips-gateway start` in Step 3.
The daemon and the rest of `/etc/fips/` are untouched. Existing mesh
peering on the AP itself continues to work.
## What you've learned
- **The gateway is a niche feature for a niche box.** Most FIPS
hosts run the daemon and reach the mesh directly. The gateway
exists so an AP can fold an entire unmodified LAN behind it into
the mesh in one place.
- **Two halves of the same binary.** Outbound mode hands LAN clients
virtual IPs and NATs them onto the mesh; inbound mode listens on
`fips0` and forwards to LAN targets. They share one nftables
table, one control socket, and one config block, but each half
has its own use case.
- **OpenWrt does the LAN-side glue for you.** The init script
reconfigures dnsmasq, installs the RA route, adds the global IPv6
prefix, and loads sysctls. On a non-OpenWrt host that integration
is manual — see [../how-to/deploy-gateway.md](../how-to/deploy-gateway.md).
- **Inbound forwards stay manual on every distro.** The
`port_forwards[]` block is uniform across hosts, and on every
distro you still own the decision of which LAN target to expose
and on which mesh-side port.
- **The mesh firewall is a separate decision.** Opening a port
forward on the gateway side does not open it on the firewall
side; if `fips-firewall.service` is enabled, you still need a
drop-in that admits the listen port.
## Troubleshooting
If something doesn't work as described above, the operator-recipe
guide [../how-to/troubleshoot-gateway.md](../how-to/troubleshoot-gateway.md)
groups the common failures by symptom:
| Symptom | Where to look |
| ------- | ------------- |
| LAN client gets `fd97:...`, not `fd01:...` | DNS path: dnsmasq still pointing at port 5354. See "DNS queries fail". |
| `dig` succeeds with a pool address but `ping6` times out | Pool route or proxy NDP. See "Virtual IP unreachable from client". |
| `ping6` works but TCP times out | NAT pipeline or mesh-side firewall. See "Ping works but TCP does not". |
| Gateway service won't start | "No gateway section in configuration" recipe. |
| Inbound `curl` hits the listen port but never reaches the LAN target | Mesh-side firewall first, then the port-forward rule. |
The first thing the troubleshoot guide does in any of these cases is
ask the gateway directly via `show_gateway` and `show_mappings`. If
the mapping you expect is not there, the failure is on the DNS path;
if it is there in `state: Active` but traffic still fails, the
failure is downstream.
## What's next
- [../how-to/deploy-gateway.md](../how-to/deploy-gateway.md) —
Manual deployment on a non-OpenWrt Linux host. Same gateway,
same config, but you wire up dnsmasq/Unbound/etc. yourself,
install a pool route per LAN client (or via your own RA daemon),
and manage the systemd unit instead of the procd init script.
- [../design/fips-gateway.md](../design/fips-gateway.md) — The
design doc: NAT pipeline (DNAT, SNAT, masquerade, inbound DNAT),
virtual-IP pool lifecycle (Allocated -> Active -> Draining ->
reclaimed), DNS resolution flow, conntrack integration.
- [../reference/configuration.md § Gateway](../reference/configuration.md#gateway-gateway)
— Every field of the `gateway:` block, including the conntrack
timeout overrides not used in this tutorial.
- [../reference/cli-fips-gateway.md](../reference/cli-fips-gateway.md)
— The `fips-gateway` binary's CLI options, exit codes, and
environment variables.
docs/tutorials/ground-up-mesh.md
+518
View File
@@ -0,0 +1,518 @@
# Build a Mesh from the Ground Up
The earlier tutorials in this progression rode existing IP — your
daemon reached `test-us01` over the public internet through your
ISP, your ISP's upstream, and however many hops separate you from
the test node. That is the *overlay* deployment mode of FIPS:
useful, but not the new ground.
This tutorial is about the other mode. Two devices, a wire (or a
radio link) between them, no IP between them, and FIPS daemons on
each end. The two daemons discover each other over the raw link,
peer over Noise, and bring up an end-to-end mesh with addressing,
naming, and reachability — all from layer 2 up. There is no DHCP,
no router, no upstream. The mesh is the network.
This is the deployment mode FIPS was designed for. Overlay mode
exists because riding existing IP is a useful convenience; the
ground-up mode is what FIPS uniquely enables.
> **The two modes are not exclusive.** A node can carry overlay
> peers and ground-up peers at the same time — different transports
> on the same daemon. If you have already worked through
> [join-the-test-mesh](join-the-test-mesh.md), the static peer to
> `test-us01` you configured there can stay in place; the Ethernet
> peer you add in this tutorial sits alongside it. Traffic flows
> through whichever path is shortest by mesh metric, and a node on
> one side can reach a node on the other through your machine
> acting as a bridge between the two.
## What you'll build
```text
┌──────────────────────┐ raw Ethernet frames ┌──────────────────────┐
│ node A │ ─────────────────────── │ node B │
│ npub1aaa… │ EtherType 0x2121 │ npub1bbb… │
│ fips0 fd97:..:A │ no IP between them │ fips0 fd97:..:B │
└──────────────────────┘ └──────────────────────┘
│ │
│ a single Ethernet cable │
│ (or both NICs on the same │
│ unmanaged switch — no DHCP, │
│ no router, no IP at all) │
└─────────────────────────────────────────────────┘
```
Two machines, each running `fips`, joined by a physical Ethernet
link. After the worked example:
- The two daemons have discovered each other via L2 beacons on
the link, peered over Noise IK, and brought up an FMP link.
- Each `fips0` adapter has a routable mesh address; each can
ping the other by `<npub>.fips`.
- Nothing between the two machines speaks IP. The link carries
raw FIPS frames at EtherType `0x2121`.
The whole exercise should take about twenty minutes if you have
the hardware ready.
## Why ground-up
Most networking tutorials assume IP is already there: an address
arrived from DHCP, a default gateway routes you onward, DNS
resolves names. FIPS does not need any of that. Two devices and
a way to deliver bytes between them at layer 2 is enough — FIPS
supplies the rest:
- **Identity**: each daemon has an npub (the same kind you saw
in the overlay tutorials). Nothing in the ground-up case
depends on a network identity from a router; the npub is the
identity.
- **Addressing**: the `fips0` adapter takes an `fd97:...` ULA
derived from the npub. No DHCP. No SLAAC. The address is
cryptographically tied to the identity.
- **Discovery**: each daemon broadcasts a small beacon on the
link advertising its npub; the other daemon's listener picks
it up and dials in over the same link.
- **Routing**: the FIPS mesh layer builds its own spanning tree
across whatever links it has. Add a third node (peered to
either A or B) and traffic reaches it transparently.
The point is not that ground-up replaces overlay. It's that
overlay is one of two modes the same daemon supports, and
ground-up is what unlocks the use cases overlay cannot —
ad-hoc local meshes, partitioned networks, situations where
no IP infrastructure exists or can be relied on.
## Prerequisites
Two devices (call them **node A** and **node B**) and a way to
join them at layer 2:
- Ethernet (the worked example): a direct cable between two
modern NICs (auto-MDI/MDIX handles crossover for you), or
both machines on a small unmanaged switch with no DHCP
server. USB-Ethernet dongles work; a typical "USB-to-RJ45"
adapter is fine on either end. The link does **not** need
to be the machine's primary network interface — a second
NIC dedicated to the mesh is the cleanest setup.
- WiFi (a one-line variation, covered later): both machines
associated to a common AP that has client (station)
isolation **off**.
- Bluetooth LE (a separate worked example via a how-to,
covered later): two BLE-capable Linux hosts within roughly
10 metres line of sight.
On both nodes:
- `fips` installed and running, per [getting-started](../getting-started.md).
- A persistent identity from
[persistent-identity](persistent-identity.md). Ephemeral
identities work, but on each restart the npub regenerates
and you'll have to re-check `fipsctl show peers` to see the
new identity. Persistent makes the lesson stick.
- The daemon running with `CAP_NET_RAW` (the shipped systemd
unit runs as root and gets this for free; running
interactively from a user account requires `setcap` —
noted at the relevant step below).
You do **not** need:
- An IP address on the chosen interface. The Ethernet
transport opens a raw socket directly; the kernel does not
need to assign an IP to the NIC.
- A default route. The mesh routes itself.
- DNS resolution between the machines via any external
service. The local `.fips` resolver supplies names from
the npubs the daemons exchange.
## Step 1: Identify the link interface on each node
On each node, list the network interfaces and pick the one that
sits on the link between the two machines. If it's a dedicated
NIC for the mesh, that NIC has no other purpose; if it's a
USB-Ethernet dongle, plug it in first so the kernel names it.
```sh
ip link show
```
Pick out the interface name. Common forms:
- `enp3s0`, `eno1` — built-in NICs under predictable naming.
- `eth0` — older or container-style naming.
- `enxAABBCCDDEEFF` — USB-Ethernet dongles often appear under
this MAC-derived form.
Bring the interface up if it isn't:
```sh
sudo ip link set dev <interface> up
```
Confirm:
```sh
ip -br link show <interface>
```
You want `UP` and `LOWER_UP` in the flags. The interface does
not need an IP address — `LOWER_UP` indicates the NIC sees
carrier (cable plugged into something at the other end), and
that is all the Ethernet transport needs.
For the rest of the tutorial we'll write the chosen interface
as `<eth>`. Substitute the actual name on each node when you
run the commands. Note that node A and node B may have
different interface names — that is normal.
> **No IP needed.** If your chosen interface has an address
> from a previous DHCP lease, leave it alone or remove it with
> `sudo ip addr flush dev <eth>` — the FIPS Ethernet transport
> uses raw `AF_PACKET` sockets that bypass the IP stack
> entirely. The interface needs to be `up` and `LOWER_UP`,
> nothing more.
## Step 2: Configure the Ethernet transport on each node
Edit `/etc/fips/fips.yaml` on **both** nodes. Under
`transports:`, add an `ethernet:` block. The key settings are
the four discovery flags — both nodes must opt in to all four,
and they default to off:
```yaml
transports:
ethernet:
interface: "<eth>" # the name from Step 1
announce: true # broadcast our beacon on the link
discovery: true # listen for beacons (default; shown for clarity)
auto_connect: true # dial peers we discover
accept_connections: true # accept dial-ins from peers we discover
```
Each flag does one thing:
- `announce: true` — emit a small beacon every
`beacon_interval_secs` (default 30s) carrying our npub.
- `discovery: true` — listen for incoming beacons; populate a
candidate-peer list keyed by source MAC and observed npub.
- `auto_connect: true` — when we see a beacon from an npub
we have not yet peered with, initiate the outbound Noise
handshake.
- `accept_connections: true` — when a remote npub initiates
the handshake on this transport, complete it.
If only one node sets `announce`, the other won't see it; if
only one side sets `auto_connect` or `accept_connections`, the
roles are asymmetric and the link won't establish unless both
are configured. The cleanest pattern for a ground-up tutorial
is "all four flags on both ends."
> **Multiple Ethernet links.** If a node has more than one
> physical interface that participates in the mesh, configure
> each one as a *named instance* under `ethernet:`:
>
> ```yaml
> transports:
> ethernet:
> lan:
> interface: "eth0"
> announce: true
> discovery: true
> auto_connect: true
> accept_connections: true
> dongle:
> interface: "enx00aabbccddee"
> announce: true
> # ...
> ```
>
> Each named instance runs its own socket and discovery state.
> A single ground-up link only needs the flat form shown
> first; named instances become useful when the same node
> bridges multiple physical segments.
## Step 3: Grant the daemon permission to open raw sockets
The Ethernet transport opens an `AF_PACKET` `SOCK_DGRAM` socket
bound to the chosen interface. That requires `CAP_NET_RAW`.
If you installed FIPS via the Debian package and run via the
shipped systemd unit, the daemon runs as root and has
`CAP_NET_RAW` already — there is nothing to do here. Skip to
Step 4.
If you are running the daemon interactively as your user (a
from-source / development setup), grant the capability once on
the binary:
```sh
sudo setcap CAP_NET_RAW,CAP_NET_ADMIN+ep "$(which fips)"
```
`CAP_NET_ADMIN` is what the daemon needs for the `fips0` TUN
adapter regardless; `CAP_NET_RAW` is the ground-up addition.
The `setcap` invocation only needs to be repeated when the
binary is replaced.
## Step 4: Restart the daemon on each node
```sh
sudo systemctl restart fips
```
Or, if running interactively, restart your `fips` invocation
in whichever way you started it.
Watch the startup logs for the Ethernet transport coming up:
```sh
sudo journalctl -u fips -f --since="1 minute ago"
```
Look for landmarks like:
- A line indicating the Ethernet transport opened the chosen
interface and started its receive loop.
- Periodic outbound beacon messages (one per
`beacon_interval_secs` window).
- After the second beacon round on the *other* node, an
inbound beacon parsed and a candidate-peer entry created.
- Once each side dials, a Noise handshake completion log
message naming the remote npub.
Beacon interval defaults to 30s, so the first peering can take
up to a minute (one beacon window per side, plus handshake).
Lower the interval for the tutorial if you want faster
feedback:
```yaml
transports:
ethernet:
# ...
beacon_interval_secs: 10 # minimum allowed
```
## Step 5: Verify the link
On either node:
```sh
sudo fipsctl show peers
```
Expect one entry whose `npub` matches the **other** node and
whose `addresses` line shows `transport: ethernet`. Your
existing overlay peers (if any from earlier tutorials) appear
alongside it. Each peer has its own row, and the link status
columns show whether the Noise session is up.
```sh
sudo fipsctl show transports
```
Confirms that the Ethernet transport is running and shows the
beacon counters incrementing. Both `beacons_sent` and
`beacons_received` should be non-zero if the link is healthy.
## Step 6: Reach the other node by name
On node A, ping node B by `.fips` name. Get node B's npub
from its `fipsctl show status` output (it's the persistent
identity you established earlier), then:
```sh
ping6 npub1bbb…long-string….fips
```
Expect ICMPv6 echo replies. The path is:
1. The local `.fips` resolver translates the npub-form name
into an `fd97:...` mesh address (cryptographically derived
from the npub on both ends — the resolver does the
computation locally, with no network round trip).
2. The kernel routes the packet via `fips0`.
3. The FIPS daemon accepts it from the TUN, looks up the
mesh route, and hands it to the FMP link to node B.
4. The Ethernet transport on node A frames the FMP packet as
a raw EtherType `0x2121` Ethernet frame addressed to node
B's MAC, learned from B's beacons.
5. Node B's daemon receives the frame, peels off the
Ethernet/FIPS framing, and the packet emerges on node B's
`fips0`.
6. The kernel on node B sees an inbound ICMPv6 echo and
replies, and the same path runs in reverse.
If you have a hosts file with shortnames configured (see
[host-aliases](../how-to/host-aliases.md)), substitute the
shortname for the full npub form.
## Step 7: Try a forward composition
If node A also has the `test-us01` overlay peer from
[join-the-test-mesh](join-the-test-mesh.md), node B can
reach `test-us01` *through* node A — even though node B has
no direct internet path of its own:
On node B:
```sh
ping6 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
```
The packet leaves B's `fips0`, traverses the Ethernet link to
A, gets forwarded by A across the overlay UDP transport to
`test-us01`, and the reply comes back the same way.
This is the composition the chapter intro flagged: the two
deployment modes coexist on a single daemon. Node A is
participating in the test mesh via the internet *and* in your
local Ethernet mesh. From node B's perspective, the test mesh
is reachable. From `test-us01`'s perspective, B is reachable.
The mesh handles the rest.
## Variations
### WiFi (AP mode), same shape as Ethernet
Replace `<eth>` with the WiFi interface name (typically
`wlan0` or `wlp3s0`) on each node. The WiFi NIC is presented
as an Ethernet-class interface to the kernel by the
`mac80211` abstraction; the FIPS Ethernet transport opens
the same `AF_PACKET` socket on it. No FIPS-side configuration
change beyond the interface name.
What you do need on the AP side:
- Both nodes associated to the same SSID.
- **Client (station) isolation must be OFF** on the AP.
Most consumer routers ship with it off; many guest
networks and "secure" enterprise APs ship with it on.
When client isolation is on, the AP refuses to forward
station-to-station frames — the broadcast beacons never
arrive at the other node, and discovery fails silently.
If beacons aren't crossing, this is the first thing to
check.
There is no FIPS-specific configuration for WiFi versus
Ethernet on the daemon side; the choice is purely the
adapter name.
### Bluetooth LE (experimental but works)
BLE is a separate transport (`transports.ble.*`) with its own
discovery model — L2CAP advertisements rather than raw L2
broadcasts. The shape of the tutorial is the same (advertise +
scan + auto-connect + accept), but the prerequisites are
different: BlueZ, `bluetoothd`, an HCI adapter, and the
`bluetooth` group or capability set.
The full operator recipe is in
[../how-to/set-up-bluetooth-peer.md](../how-to/set-up-bluetooth-peer.md).
Mark this transport as experimental: it works in most
configurations but the BLE stack has more variability than
Ethernet — adapter quirks, BlueZ version differences, and the
shorter range all matter.
The BLE transport is **Linux-only** at present; macOS and
Windows builds skip it.
## What you've learned
- **Ground-up is the new ground.** FIPS does not need any IP
infrastructure between two devices to mesh them. A wire (or
a radio link), `CAP_NET_RAW`, and a few config flags on each
end are sufficient. The mesh supplies its own identity,
addressing, discovery, and routing.
- **Discovery is a four-flag opt-in.** `announce`, `discovery`,
`auto_connect`, and `accept_connections` each control one
thing; both ends must agree before a link will form.
- **The two modes coexist.** Overlay peers and ground-up peers
ride the same daemon — same FMP link layer, same FSP session
layer, same `fips0` adapter. A node can be a bridge between
the two without any extra plumbing.
- **No IP on the link.** The Ethernet transport bypasses the
kernel IP stack via `AF_PACKET`. Whether the interface has
an IP address is irrelevant; whether it has carrier is what
matters.
- **Names work the same way.** `<npub>.fips` resolves locally
via the cryptographically-derived ULA. The resolver does
not care whether the destination is reached over Ethernet,
UDP overlay, or some hop chain combining both.
## Troubleshooting
- **No beacons received.** On either node, `sudo fipsctl show
transports` should show `beacons_received` incrementing
every `beacon_interval_secs` once the other node is also
running. If it stays at zero:
- Confirm the chosen interface is `LOWER_UP` (carrier
present).
- Confirm the other node is announcing (its `beacons_sent`
should be non-zero).
- On WiFi: confirm AP client isolation is off.
- On a switch: confirm the switch is unmanaged or that
EtherType `0x2121` is not being filtered. Most consumer
switches forward all EtherTypes; managed switches
sometimes don't.
- **Beacons received but no peer entry.** The handshake is
failing. Tail logs (`journalctl -u fips`) for Noise
handshake errors. Common causes: peer ACL active and not
including the remote npub (out of scope for this tutorial,
but check `/etc/fips/peers.allow` if you have set one);
daemon's clock drift large enough to fail freshness
checks (rare).
- **Daemon won't start with the Ethernet transport.** Likely
a permissions error. Check `journalctl -u fips` for an
`EPERM` or "operation not permitted" message; if running
interactively, confirm the binary has `CAP_NET_RAW`
(`getcap "$(which fips)"`).
- **Beacons in both directions, peers entries on both sides,
but ping6 times out.** The handshake completed but the FSP
session is not flowing data. Check `fipsctl show peers`'s
link status columns — if the FMP link is healthy but FSP
is not, the mesh-layer side is fine and the issue is one
layer up. The
[reach-mesh-services § Troubleshooting](reach-mesh-services.md#troubleshooting)
section covers symptoms at this level.
- **`AF_PACKET` socket bind fails on a kernel-protected
interface.** Some hardened kernels (`grsec`, certain
containers, certain VMs) restrict raw-socket access even
with `CAP_NET_RAW`. The daemon log will name the failing
syscall. The fix is host-side: relax the restriction or
pick a different interface.
## What's next
You now have the second deployment mode of FIPS in your
hands. From here:
- **Add a third node.** Bring up a third machine on the same
Ethernet segment, configure it identically, and watch all
three nodes form a mesh. The FIPS spanning tree picks a
root and routing converges within a few beacon intervals.
- **Mix transports.** Add an overlay peer (per
[join-the-test-mesh](join-the-test-mesh.md)) to one of
your ground-up nodes; the local mesh now reaches the test
mesh through that node, and vice versa.
- **Host services.** Anything you do on `fips0` with overlay
peers — bind an HTTP server (per
[host-a-service](host-a-service.md)), reach a service via
the daemon's IPv6 adapter (per
[reach-mesh-services](reach-mesh-services.md)) — works
identically on a ground-up mesh. The data plane is the
same.
For more depth on the link-layer machinery:
- [../reference/transports.md § Ethernet](../reference/transports.md)
— full Ethernet transport reference (counter inventory,
per-instance configuration, MTU model).
- [../reference/configuration.md § Ethernet](../reference/configuration.md#ethernet-transportsethernet)
— every configuration key and its default.
- [../how-to/set-up-bluetooth-peer.md](../how-to/set-up-bluetooth-peer.md)
— operator recipe for the BLE variant.
- [../design/fips-transport-layer.md](../design/fips-transport-layer.md)
— the design doc that describes the per-link MTU model and
why each transport is treated as link-layer rather than
network-layer.
docs/tutorials/host-a-service.md
+477
View File
@@ -0,0 +1,477 @@
# Host a Service of Your Own
In [reach-mesh-services](reach-mesh-services.md) you used
ordinary IPv6 tools to reach services on other mesh nodes. This
tutorial flips the direction. You will bring up a small HTTP
server on your machine, make a deliberate choice about which
interface it binds to, and turn on the mesh firewall to keep
the exposure to what you intended. By the end you will have
hosted your first peer-reachable service and made an informed
decision about who can reach it.
The whole exercise should take about twenty minutes. You should
have already worked through
[persistent-identity](persistent-identity.md) so that the npub
your service is reachable at does not change between restarts.
## What you'll build
```text
┌──────────────────────────────────────────┐
│ your fips node │
│ │
│ python3 -m http.server │
│ --bind <fips0-addr> 8080 │
│ │ │
│ ▼ │
│ fips0 fd97:….:Y ◀─── port 8080 open │
└────────────┬─────────────────────────────┘
│
│ reachable as
│ http://<your-npub>.fips:8080/
▼
any mesh peer that can route to <your-npub>.fips
```
You will have:
- A single-page HTTP server bound to `fips0` only — not the
public internet, not your LAN, just the mesh.
- The mesh firewall baseline active, default-deny on `fips0`
inbound, with one explicit drop-in that allows TCP/8080.
- A clear understanding of which interface choice corresponds
to which audience.
## Why bind interface matters
The single most important decision when hosting a service is
**which interface (and therefore which audience) the service is
exposed to**. This is true for every IPv6 service, not just
FIPS — but FIPS makes it stark because your machine often has
several interfaces with very different exposure profiles.
> **Bind interface = exposure surface.** A typical FIPS host
> has at least two distinct audiences:
>
> - `fips0` (`fd97:…`, reachable as `<your-npub>.fips`) —
> reachable only from FIPS peers that have a working link to
> your node. Bound by Noise authentication and (optionally)
> the peer ACL.
> - `eth0` / `wlan0` (your LAN address) — reachable from anyone
> on your local network segment, with no FIPS auth in the
> way.
>
> When you run a server, the `--bind` argument decides which of
> these audiences sees the service. Binding to a *specific*
> address opts in to one audience. Binding to `[::]` or
> `0.0.0.0` opts in to **all** of them at once — including any
> you forgot you had.
>
> **Audit what's already listening.** Bringing up `fips0` adds
> a new audience to every service on this host that was already
> bound to `0.0.0.0` or `[::]`. SSH, your web server, a database
> — if any of them was listening on all interfaces before you
> joined the mesh, they are now reachable from mesh peers too.
> A quick `ss -tulnp` will show you everything currently
> listening and on which addresses. The mesh firewall (Step 5
> below) is one way to bring those exposures back under explicit
> control; rebinding the affected services to a specific
> non-mesh address is another.
There is nothing FIPS-specific about this rule; it applies to
SSH, web servers, databases, anything. FIPS just gives you the
option of "mesh peers only" as a distinct audience, which most
hosts otherwise don't have.
## Step 1: Find your node's mesh address
You need the `fd97:...` address assigned to your `fips0`
adapter. Two equivalent ways to get it:
```sh
ip -6 addr show fips0
```
Look for the `inet6 fd97:...` line. The address up to the `/`
is what you want.
Or via the daemon:
```sh
sudo fipsctl show identities
```
The first JSON entry has `local: true` and a `ula` field — that
is your address.
For the rest of this tutorial we will write the address as
`<your-fips0-addr>`. Substitute the actual `fd97:...` value
when you run the commands. Save it to a shell variable for
convenience:
```sh
FIPS0_ADDR=$(ip -6 addr show fips0 | awk '/inet6 fd97:/ {print $2}' | cut -d/ -f1)
echo "$FIPS0_ADDR"
```
You should also know your npub from
[persistent-identity](persistent-identity.md):
```sh
NPUB=$(sudo cat /etc/fips/fips.pub)
echo "$NPUB"
```
`<your-npub>.fips` and `<your-fips0-addr>` are two names for
the same destination.
## Step 2: Bring up an HTTP server bound to fips0
Make a small directory with one file in it so the server has
something to serve:
```sh
mkdir -p /tmp/mesh-demo
echo '<h1>Hello from the mesh</h1>' > /tmp/mesh-demo/index.html
cd /tmp/mesh-demo
```
Start a Python HTTP server bound to your `fips0` address:
```sh
python3 -m http.server --bind "$FIPS0_ADDR" 8080
```
Leave the server running. The terminal will show:
```text
Serving HTTP on fd97:... port 8080 (http://[fd97:...]:8080/) ...
```
Two things to notice:
- The "Serving HTTP on …" line names your `fd97:...` address
explicitly. Python is binding only to that one address.
- The default would have been `0.0.0.0` — which is **not**
what you want here. Without `--bind`, the server would be
reachable from your LAN and any public IP this host has,
not just from the mesh.
## Step 3: Verify the service locally
Open a second terminal. From the same host, fetch the page:
```sh
curl -6 "http://[$FIPS0_ADDR]:8080/"
```
Expect:
```text
<h1>Hello from the mesh</h1>
```
Now fetch it by name. Both forms should work:
```sh
curl -6 "http://${NPUB}.fips:8080/"
```
The daemon's local DNS responder turned the npub-form name into
the `fd97:...` address, and the request landed at your HTTP
server.
> **What just happened.** The kernel routed your local request
> via the loopback path because the destination address is
> assigned to one of your own interfaces. You exercised the
> client side (DNS, IPv6 socket open, HTTP request) and the
> server side (HTTP listener, response). What you have not yet
> verified is reachability *from another mesh node*. That is
> the next concern.
## Step 4: Reachability from a mesh node
Any mesh node — a direct peer, or a node several hops away —
reaches your service the same way you reached `test-us01` in
[reach-mesh-services](reach-mesh-services.md): it looks up
`<your-npub>.fips`, gets back your `fd97:...` address, opens
a TCP connection to it, and the FIPS data plane carries the
packets across the mesh to you. From the remote host the curl
looks identical to yours:
```sh
curl -6 "http://${NPUB}.fips:8080/"
```
If you have a second machine on the mesh — or you can ask
another operator to try it — this is the moment to confirm.
A node that can already `ping6 ${NPUB}.fips` should also be
able to fetch your page. If it can ping but the curl times
out, jump to [Troubleshooting](#troubleshooting) — but most
likely the firewall step in this tutorial has not happened
yet, so a remote attempt right now will succeed straight
through to your HTTP server.
That is the problem. With no firewall in place, *any* mesh
node that can route to you — your direct peers, and every node
beyond them in the mesh — can reach port 8080. You have not
yet made a deliberate decision about whether you want that.
The rest of this tutorial replaces the implicit "every port
with a listener is reachable" with an explicit "only the ports
I have opened are reachable, optionally only from specific
mesh nodes." That is the firewall's job, and it is the only
mechanism in play in the rest of this tutorial.
(There is a separate, unrelated control called the *peer ACL*
that decides which npubs may establish a peer connection with
your node at the transport layer. It is not part of the
firewall and does not affect what is described below; Step 7
is a brief signpost to it.)
## Step 5: Activate the mesh firewall baseline
FIPS ships a default-deny nftables baseline at
`/etc/fips/fips.nft` that restricts inbound traffic on `fips0`
to ICMPv6 echo and conntrack replies. The baseline is **not**
enabled by default — activation is an explicit step the
operator has to take.
Activate it:
```sh
sudo systemctl enable --now fips-firewall.service
```
This loads the table immediately and arranges for it to load
on every subsequent boot. Confirm:
```sh
sudo nft list table inet fips
```
You will see one chain named `inbound` hooked at `input`,
roughly:
```text
table inet fips {
chain inbound {
type filter hook input priority filter; policy accept;
iifname != "fips0" return
ct state established,related accept
icmpv6 type echo-request accept
counter packets 0 bytes 0 drop
}
}
```
The chain admits ICMPv6 echo (so `ping6` from any mesh node
still works) and conntrack replies (so your *outbound*
connections still get their replies back). Everything else
inbound on `fips0` hits the final `counter ... drop`.
> **What this changed.** Your HTTP server is still running and
> still reachable *from this same host* (same-host traffic to
> `fd97:...` goes via the loopback path, which has
> `iifname != "fips0"` and short-circuits at the first rule).
> But any mesh node trying to reach `fd97:...:8080` now has its
> TCP SYN dropped before it can reach your server. From the
> remote end the connection times out.
So the firewall is in the right shape but in the wrong state
for our purpose: we *want* mesh nodes to reach port 8080. The
next step opens that one port.
## Step 6: Open port 8080 via a drop-in
Drop-ins live under `/etc/fips/fips.d/` with the `.nft`
suffix. Each file is included into the `inbound` chain at the
marked point and may contain any nftables rule lines valid in
that context.
Create one for your HTTP service:
```sh
sudo tee /etc/fips/fips.d/http-mesh-demo.nft >/dev/null <<'EOF'
tcp dport 8080 accept
EOF
```
Reload the firewall:
```sh
sudo systemctl reload-or-restart fips-firewall.service
```
Confirm the rule is live:
```sh
sudo nft list table inet fips
```
The `inbound` chain now contains your `tcp dport 8080 accept`
rule between the conntrack rule and the final `counter drop`.
A curl from any mesh node will now reach the HTTP server. The
path is: remote node's mesh data plane → forwarded across the
mesh → your direct peer's link to you → `fips0` ingress →
`inbound` chain → matches `tcp dport 8080 accept` → delivered
to the HTTP server.
If you only want to expose the service to a *specific* node
or set of nodes, source-filter the rule. The address filter
applies to the mesh-source address as it arrives on `fips0`,
which is the originating node's address — not necessarily a
direct peer. Replace the drop-in contents with something like:
```nft
ip6 saddr fd97:1234:5678:9abc:def0:1234:5678:9abc tcp dport 8080 accept
```
The source address is the node's mesh address, which it
publishes in its `fips.pub` (and which you can resolve from
its npub). For multiple nodes, use a set:
```nft
ip6 saddr {
fd97:1111:2222:3333:4444:5555:6666:7777,
fd97:8888:9999:aaaa:bbbb:cccc:dddd:eeee
} tcp dport 8080 accept
```
For the worked example, leave the drop-in unfiltered — any
mesh node that can route to you can fetch your page.
## Step 7: A note on the peer ACL
The firewall you just configured is the only control in scope
for this tutorial. There is a separate, optional control
called the *peer ACL* that you may run across in other docs;
it is unrelated to the firewall and worth a sentence here only
so you do not confuse the two.
The peer ACL decides which npubs may establish a peer
connection with your node at the transport layer. It does not
look at ports, drop-ins, or `fips0` traffic. You do not need
it for this tutorial.
For when you do:
- [../how-to/enable-nostr-discovery.md](../how-to/enable-nostr-discovery.md)
— operator recipe.
- [../reference/security.md § Peer ACL](../reference/security.md#peer-acl)
— file format, evaluation order, alias handling.
## Step 8: Stop the server and tidy up
When you are done, stop the HTTP server in the first terminal
with `Ctrl-C`. The drop-in stays in place; remove it if you
do not want port 8080 reachable after the demo:
```sh
sudo rm /etc/fips/fips.d/http-mesh-demo.nft
sudo systemctl reload-or-restart fips-firewall.service
```
The `fips-firewall.service` itself can stay enabled —
default-deny on `fips0` is a sensible posture even with no
extra services running. To turn it back off:
```sh
sudo systemctl disable --now fips-firewall.service
```
## What you've learned
- **Bind interface = audience.** Binding to a specific address
opts in to one audience; binding to wildcard
(`0.0.0.0` / `[::]`) opts in to *all* of them, including
ones you forgot you had. For mesh-only exposure, bind to
your `fd97:...` address.
- **Same-host loopback is misleading.** A local curl to your
own `fd97:...` address goes via the loopback path, not
through `fips0` ingress. To actually verify mesh-side
reachability you need a second machine, or to read what
the firewall is doing in `nft list table inet fips`.
- **The mesh firewall is opt-in.** `fips-firewall.service` is
not enabled by default. Once it is enabled, `fips0` is
default-deny except for ICMPv6 echo and conntrack replies.
- **Ports open via drop-ins.** Each file under
`/etc/fips/fips.d/*.nft` adds rules into the `inbound`
chain. Source-filter with `ip6 saddr` to scope a port to
specific mesh nodes.
- **Two independent controls at two different layers.** The
firewall is a layer-3 filter on `fips0`: it controls which
TCP/UDP ports are reachable and (optionally) which mesh
source addresses may reach them. The peer ACL is a
transport-layer admission filter on Noise handshakes: it
controls which npubs may become direct peers of your node.
They are unrelated — the ACL does not touch fips0 traffic,
and the firewall does not look at npubs.
You now have the mental model for hosting any IPv6 service
behind a deliberate exposure policy. The mechanics generalize:
SSH on port 22, a database on port 5432, a custom protocol on
its own port — same `--bind` rule, same drop-in shape.
## Troubleshooting
- **A remote mesh node cannot reach the service after the
firewall reload.** Check the drop-in syntax with
`sudo nft -c -f /etc/fips/fips.nft` before reloading; a
syntax error in any drop-in causes the whole table to fail
to load and the previous rules persist. Then
`sudo nft list table inet fips` to confirm your
`tcp dport 8080 accept` rule is present in the `inbound`
chain.
- **Local curl works, remote curl times out.** The packet is
reaching `fips0` ingress and being dropped by the baseline.
Either your drop-in did not load (see above) or it has a
source filter that excludes the remote node's address.
- **Local curl fails after binding to fips0.** Double-check
that your `FIPS0_ADDR` matches the address shown in
`ip -6 addr show fips0`. The Python server message also
echoes the bound address — confirm it starts with `fd97:`,
not `127.0.0.1` or `::`.
- **`Address already in use` from Python.** Another process
holds port 8080. Pick a different port (`8081`, `9000`, …)
for both the `python3 -m http.server` invocation and the
drop-in.
- **Watch the firewall counter to confirm drops.** The
`counter ... drop` line at the bottom of the chain
increments on every dropped inbound packet. After a remote
mesh node attempts to reach a port you have not opened,
`sudo nft list table inet fips` will show the counter
packet count rising.
## What's next
- [ground-up-mesh.md](ground-up-mesh.md) — Bring up two devices
on a shared physical link — Ethernet, WiFi, or Bluetooth —
with no pre-existing IP infrastructure between them. The
second deployment mode of FIPS, where the mesh is the
network rather than an overlay on top of one. Coexists with
overlay peers; the same daemon carries both.
For more depth on the firewall and ACL surface:
- [../how-to/enable-mesh-firewall.md](../how-to/enable-mesh-firewall.md)
— operator recipes for the baseline, drop-in patterns,
and how to fold the baseline into an existing
`nftables.conf`.
- [../reference/security.md](../reference/security.md) —
consolidated security reference: nftables baseline rules,
drop-in format, peer ACL semantics, default exposures by
transport, threat-resistance matrix.
- [../design/fips-security.md](../design/fips-security.md) —
threat model, why the baseline is opt-in, the metadata-
privacy posture.
If you want to host a service that is *not* on a FIPS node — say,
an existing HTTP server on a regular LAN box — and expose it to
mesh peers through a `fips-gateway`, that's the inbound
port-forward mode: the gateway runs a mesh-side listener on `fips0`
and forwards to a LAN target. The operator recipe is at
[../how-to/deploy-gateway.md#inbound-port-forwarding](../how-to/deploy-gateway.md#inbound-port-forwarding);
a hand-held walk-through on an OpenWrt AP is at
[deploy-fips-gateway.md](deploy-fips-gateway.md) under "Advanced"
in [README.md](README.md).
docs/tutorials/ipv6-adapter-walkthrough.md
+231
View File
@@ -0,0 +1,231 @@
# IPv6 Adapter Walkthrough
You have completed [join-the-test-mesh](join-the-test-mesh.md).
Your daemon is peered with `test-us01` and you can ping mesh
nodes by `.fips` name. This tutorial walks the plumbing that
makes that possible: what happens between the moment your shell
types `ssh user@<peer>.fips` and the moment a TCP SYN arrives at
sshd on the far side. Each step is something you can observe
with the running daemon from the previous tutorial.
By the end you will be comfortable reading `fipstop` output and
you will know which design doc to consult when something looks
off.
> **Prerequisites.** The daemon from
> [join-the-test-mesh.md](join-the-test-mesh.md) is running and
> peered with at least one test-mesh node, and your host's local
> resolver is forwarding `.fips` queries to the daemon's DNS
> responder (the system fips-dns.service drop-in does this
> automatically on systemd hosts).
## The path we're tracing
```text
shell ──ssh──> libc resolver ──.fips──> fips DNS ──AAAA──> fd97:...:test-us01
│
▼
kernel IPv6 stack
│
▼
fips0 (TUN)
│
▼
your fips daemon
(FSP session setup,
FMP forwarding)
│
UDP / internet
▼
test-us01's fips daemon
│
▼
fips0 (TUN)
│
▼
kernel IPv6 stack
│
▼
sshd
```
In a multi-hop mesh the middle would have additional FMP
forwarders between your daemon and the destination. For this
walkthrough you have a single direct link to `test-us01`, which
keeps the trace simple.
## Step 1: Watch the DNS resolution
Ask the system resolver to translate `test-us01`'s npub into its
mesh address:
```sh
dig npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips AAAA +short
```
You should see one AAAA record returning a `fd97:...` address.
The prefix is the FIPS ULA range (`fd00::/8`, with `fd97:...`
covering the address space derived from npubs).
The query went through `systemd-resolved` (or your platform
equivalent), which routed `.fips` queries to the daemon's local
responder via the drop-in installed by `fips-dns.service`. To
confirm, query the daemon directly:
```sh
dig @::1 -p 5354 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips AAAA +short
```
Same answer, same fast turnaround — no external DNS traffic in
either case.
The mapping `npub → fd00::/8 address` is deterministic. The
responder hashes the public key into 16 bytes, prepends the
prefix, and returns the result. There is no shared registry; the
address space is self-allocating from the public-key namespace.
If you ask for any non-`.fips` suffix, the responder returns
`REFUSED` — it is intentionally a stub for this single zone, not a
recursive resolver. An unknown `.fips` name returns `NXDOMAIN`.
The full DNS integration is documented in
[../design/fips-ipv6-adapter.md](../design/fips-ipv6-adapter.md).
## Step 2: Watch the session being created
Open `fipstop` against your daemon's control socket:
```sh
sudo fipstop
```
Press `Tab` until you reach the **Sessions** tab. Before any
TCP traffic to `test-us01`, the table is empty (or has rows from
earlier exchanges).
In another terminal, kick off a TCP connection from your host
toward `test-us01`:
```sh
ssh -o ConnectTimeout=5 user@test-us01.fips
```
(`test-us01.fips` resolves to the same address as the npub
form via the installer's `/etc/fips/hosts` entry.)
(It is fine if the SSH attempt fails authentication or if no
sshd is exposed on the far side — what we want to observe is the
session machinery firing, not a successful login.)
In `fipstop`'s Sessions tab you should see a new row appear with:
- `state` cycling from `initiating` to `awaiting_msg3` to
`established` (the three FSP handshake states).
- `display_name` showing `test-us01` (the `alias` you set in your
`peers:` block in the previous tutorial).
- A non-zero `last_activity_ms`.
Once `established`, the session row stays put until idle-timeout
expires. The traffic counters and MMP metrics tick as data flows.
> **Watch for.** Some intermediate states may be too fast to
> see at the default `fipstop` refresh rate of 2 s. Run
> `sudo fipstop -r 1` for a faster refresh during the exercise.
## Step 3: Watch the per-session metrics
Switch to the **Performance** tab. Each established session has
a session-layer MMP entry showing:
- `srtt_ms` — smoothed end-to-end round-trip time. Over a
public-internet path this typically lands in the tens of
milliseconds; for a US-coast destination from a US client you
might see 30–80 ms steady-state.
- `loss_rate` — fraction of in-flight payloads inferred lost from
counter gaps. Stays at 0 on a healthy link; small bursts during
congestion or path changes.
- `path_mtu` — the end-to-end MTU the session-layer MMP currently
believes is in force. Starts at the IPv6 floor and climbs as
PathMtuNotification echoes arrive.
- `etx` and `goodput_bps` — derived metrics, useful as
steady-state indicators.
The same metrics are available without the TUI:
```sh
sudo fipsctl show sessions | jq '.sessions[] | {display_name, state, mmp}'
```
What these numbers mean is documented in
[../design/fips-mmp.md](../design/fips-mmp.md). Briefly: SRTT is
RFC 6298-style with α = 1/8; loss is bidirectional, inferred from
counter gaps in MMP reports; path MTU is end-to-end-echoed with
hysteresis on increase.
## Step 4: Watch the link below the session
Switch to the **Peers** tab. Each authenticated peer has its own
link-layer MMP block, distinct from the session-layer one above.
The link-layer metrics measure a single hop (here, your daemon
↔ `test-us01` over UDP), independent of any session that
traverses it.
Compare the link-layer SRTT for `test-us01` to the session-layer
SRTT of the session you just created. Because your reach to
`test-us01` is one direct hop, the two should be very close —
the session has no transit forwarders to add latency.
If you reach a node that `test-us01` forwards to (try the
`test-us02` ping from the previous tutorial), the session-layer
SRTT for that destination will be measurably larger than the
link-layer SRTT to `test-us01`. The difference is the time
`test-us01` spent forwarding plus the hop from `test-us01` to
`test-us02`.
In a deeper mesh this divergence grows: link-layer SRTT measures
the direct neighbour, session-layer SRTT measures the full
end-to-end path.
## Step 5: Read the relevant design docs
You have now seen the moving parts. To go from "I can read these
metrics" to "I understand why each one moves the way it does":
- [../design/fips-ipv6-adapter.md](../design/fips-ipv6-adapter.md)
— DNS responder, identity cache, TUN reader/writer, IPv6 header
compression, MTU enforcement at the TUN boundary.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— FSP session lifecycle: msg1 / msg2 / msg3, the rekey state
machine, the drain window for old sessions during cutover.
- [../design/fips-mmp.md](../design/fips-mmp.md) — both link-layer
and session-layer MMP: report format, SRTT estimation,
loss/jitter/ETX computation, the trend indicators.
- [../design/fips-mtu.md](../design/fips-mtu.md) — what `path_mtu`
in `show sessions` means: the proactive forward-path field, the
reactive `MtuExceeded` mechanism, the hysteresis on increase.
- [../design/fips-architecture.md](../design/fips-architecture.md)
— the two-layer encryption model: link-layer Noise IK over
each hop, end-to-end Noise XK over the session.
## What you've learned
- A `.fips` name resolves through a daemon-local stub responder.
The mapping from npub to `fd00::/8` address is deterministic
and needs no registry.
- The kernel IPv6 stack treats the TUN adapter as an ordinary
interface; packets to `fd00::/8` go out via that route. The
daemon reads them off the TUN, looks up an FSP session for the
destination (creating one if needed), and forwards them onward
through its peers.
- The session layer (FSP) and the link layer (FMP) each maintain
their own MMP metrics. Session-layer metrics measure the path
end-to-end; link-layer metrics measure a single hop. The two
align when the destination is your direct peer; they diverge
when traffic traverses additional hops.
- `fipstop` exposes both views in real time. `fipsctl show sessions`,
`fipsctl show peers`, and `fipsctl show transports` cover the
same ground programmatically.
When something looks off in production, the `fipsctl show *`
queries are usually the first stop; the relevant design doc tells
you what the numbers mean and what they should do.
docs/tutorials/join-the-test-mesh.md
+324
View File
@@ -0,0 +1,324 @@
# Join the FIPS Test Mesh
In this tutorial you will connect your FIPS daemon to a public
test peer over UDP, watch the link come up, and reach the peer's
mesh address from your machine. By the end you will have seen one
complete end-to-end flow — config, handshake, live link, traffic
— for a real peer somewhere out on the public internet.
The whole exercise should take about ten minutes. If you have
already worked through [getting-started.md](../getting-started.md)
and have the `fips` daemon running on your host, you have
everything you need.
## What you'll build
```text
┌────────────────────┐ UDP/IPv4 ┌──────────────────────┐
│ your fips node │ ──────────────────────── │ test-us01 │
│ ephemeral npub │ test-us01.fips.network │ npub1qmc3...zel98 │
│ fips0 fd97:..:Y │ :2121 │ fips0 fd97:..:T │
└────────────────────┘ └──────────────────────┘
```
Your daemon will peer with one of the public test nodes the
project maintains. `test-us01` has a stable DNS name, listens on
UDP/2121, and is reachable from any network that permits arbitrary
outbound UDP.
> **Peer vs. node.** In FIPS terminology, a *peer* is a node
> you have a direct link to — same Noise IK handshake, same
> transport socket. A *node* is any participant on the mesh,
> whether you peer with it directly or reach it through one or
> more hops via your peer's connections. Peering is a local
> configuration choice; reachability is mesh-wide. One good peer
> connects you to everyone the rest of the mesh connects to.
After the link to `test-us01` establishes, your daemon's `fips0`
adapter can reach `test-us01` itself and — through `test-us01`'s
connections — any other node on the test mesh, exactly as if you
had a direct connection to each of them.
> **About the test mesh.** The project maintains a small roster
> of public test nodes (`test-us01` through `test-uk01`) intended
> for new-user on-ramps and integration testing. They accept
> inbound peering from arbitrary npubs without prior coordination.
> A future reference doc will list the full roster; for this
> tutorial you only need `test-us01` as your peer, and `test-us02`
> later on as a second mesh destination to demonstrate
> forwarding.
## Step 1: Confirm the daemon is running
```sh
sudo systemctl status fips
```
Expect `active (running)`. If it is not running, the
[getting-started](../getting-started.md) guide covers installation
and service management. While you're checking, note your daemon's
current npub:
```sh
sudo fipsctl show status
```
Look for the `npub` field. With the default ephemeral-identity
config, this regenerates on every restart — that is fine for the
tutorial. `test-us01` admits any inbound npub.
## Step 2: Add a static peer to the daemon config
Edit `/etc/fips/fips.yaml`. Find the line that reads `peers: []`
and replace it with:
```yaml
peers:
- npub: "npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98"
alias: "test-us01"
addresses:
- transport: udp
addr: "test-us01.fips.network:2121"
connect_policy: auto_connect
```
What each field does:
- `npub` — the canonical Nostr public key of `test-us01`. This is
who your daemon will mutually authenticate with over Noise IK.
- `alias` — a short name your daemon will use when referring to
this peer in logs and `fipsctl show peers` output. Optional.
- `addresses` — one or more transport endpoints. UDP on the
published hostname and port is the most direct path.
- `connect_policy: auto_connect` — your daemon initiates an
outbound connection rather than waiting for the peer to reach
in.
## Step 3: Restart the daemon
```sh
sudo systemctl restart fips
```
Watch the daemon's journal as it comes back up and dials the
peer:
```sh
sudo journalctl -u fips -f
```
Within a few seconds you should see lines mentioning:
- An outbound connection attempt to `test-us01` or
`test-us01.fips.network:2121`
- A handshake completion (a "Noise IK link handshake complete"
style line, or "peer authenticated" with the test-us01 npub)
- An MMP / link metrics entry naming `test-us01`
If the handshake does not complete within roughly 30 seconds, jump
to [Troubleshooting](#troubleshooting) below.
## Step 4: Verify the link
```sh
sudo fipsctl show peers
```
Expect one entry whose `alias` is `test-us01`. Useful fields:
- `connectivity` — should be active / authenticated.
- `transport_addr` — the resolved UDP endpoint your daemon is
using to reach `test-us01`.
- `transport_type` — `udp`.
- `mmp.srtt_ms` — appears once the first MMP report has been
exchanged. This is your round-trip time to `test-us01`.
The transport view confirms your UDP listener and the peer
mapping:
```sh
sudo fipsctl show transports
```
## Step 5: Ping your peer
`test-us01`'s mesh address derives from its npub. Address it as
`<npub>.fips` and your daemon's local DNS responder will translate
that to its `fd97:...` mesh address.
First see the resolved address:
```sh
dig npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips AAAA +short
```
You should see one `fd97:...` line.
Now ping it:
```sh
ping6 -c 4 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips
```
Expect four replies. The first packet may take noticeably longer
than subsequent ones — that round trip includes destination
discovery, FSP session establishment, and the proactive path-MTU
probe. After that, the RTT settles to a steady value reflecting
the path between your host and `test-us01`.
This confirms the direct link works. So far, though, you have only
reached the peer you configured. The next step demonstrates the
mesh-wide reach that peering buys you.
## Step 6: Reach a different node through the mesh
`test-us02` is another public test node. You did **not** add it
to your `peers:` block — your daemon has no direct link to it.
But because `test-us01` participates in the same mesh and has its
own connections to other nodes, your daemon can reach `test-us02`
through `test-us01` without any additional configuration.
```sh
ping6 -c 4 npub10yffd020a4ag8zcy75f9pruq3rnghvvhd5hphl9s62zgp35s560qrksp9u.fips
```
Same form, different npub. Expect replies. The packets travel
from your daemon to `test-us01` over the direct UDP link, then
onward through `test-us01` (and possibly other test-mesh nodes)
to reach `test-us02`'s `fips0` adapter. Replies retrace the path.
This is the central FIPS guarantee: **peering is local, but
reachability is mesh-wide.** You only need one good peer to talk
to everyone else they (transitively) talk to.
If the `test-us02` ping fails while the `test-us01` ping
succeeded, the test mesh's routing between those two nodes is
momentarily unhealthy — try again in a minute, or pick a different
test node from the roster. The link to your peer is unaffected.
## What you've learned
You now have a single FIPS node connected to one peer in the
public test mesh, with reach to every node that mesh routes you
to. You have seen:
- **Identity.** Your daemon's ephemeral keypair authenticated to
`test-us01` over Noise IK without either side trusting anyone in
advance.
- **Transports.** A UDP socket on your host carries
authenticated, encrypted mesh frames to your peer. No central
server, no VPN concentrator.
- **Peering vs. reachability.** You configured one peer
(`test-us01`) and got reach to a second node (`test-us02`) for
free, through the mesh. The same shape extends to every other
node `test-us01` can reach.
- **Naming.** The local `.fips` resolver translated npub-form
hostnames into their `fd97:...` mesh addresses with no external
DNS traffic.
- **End-to-end.** ICMPv6 traffic over the FIPS data plane reached
both destinations and came back, end-to-end encrypted along
every link layer in the path.
> **By the way: shortnames.** Those long `npub1...fips`
> destinations are the canonical addresses, but the installer
> ships an `/etc/fips/hosts` file with shortname entries for
> the public test mesh, so `test-us01.fips` and
> `test-us02.fips` resolve to the same addresses without
> typing 80 characters of bech32. You can add your own entries
> too. See
> [../how-to/host-aliases.md](../how-to/host-aliases.md). The
> rest of the tutorials use shortnames where they're available.
## Troubleshooting
If the handshake does not complete:
- **Outbound UDP may be blocked.** Some networks filter
arbitrary outbound UDP or block return traffic. From a
UDP-filtered network you cannot reach peers that only
publish UDP endpoints — your reachable peers are limited
to those that accept incoming TCP (outbound TCP is
typically allowed even on networks that block UDP). The
test-mesh nodes publish a TCP endpoint on port 443 for
exactly this case; replace the `udp` entry in the peer's
`addresses:` block with the TCP equivalent:
```yaml
addresses:
- transport: tcp
addr: "test-us01.fips.network:443"
```
Restart the daemon and re-check `fipsctl show peers`. The
link will be slower than UDP but is the supported transport
for restrictive egress environments.
- **Confirm the testnode is reachable at the IP layer.** Run
`dig +short test-us01.fips.network` to confirm DNS, then
`nc -uvz test-us01.fips.network 2121` to confirm UDP
reachability.
- **Confirm your config parsed.** `sudo journalctl -u fips -n 50`
near the daemon-start time will show config-load lines and any
parse errors.
- **Time skew.** A heavily skewed system clock can make
signature validation fail. `timedatectl status` should show
the system clock as synchronized.
## What's next
These are the natural follow-on tutorials in the new-user
progression. Some are still being written and will appear
alongside this one in the [tutorials/](.) directory.
- **Make your node's identity persistent.**
[persistent-identity](persistent-identity.md) walks through
pinning the daemon to a stable Nostr keypair so your npub
does not change across restarts — the prerequisite for other
operators adding you to their `peers:` blocks.
- **Resolve peers via Nostr.**
[resolve-peers-via-nostr](resolve-peers-via-nostr.md) is the
smallest useful step toward Nostr-mediated discovery:
configure a peer by npub alone and let the daemon look up
the current endpoint from public relays. The first of three
tutorials covering Nostr discovery; the others —
[advertise-your-node](advertise-your-node.md) and
[open-discovery](open-discovery.md) — round out the
publish and ambient-consume sides.
- **Trace a connection end-to-end.**
[ipv6-adapter-walkthrough](ipv6-adapter-walkthrough.md) walks
the data path from a `.fips` DNS query through session setup
to the far-side TUN adapter, using `fipstop` and `fipsctl` to
observe each step.
- **Reach services on other mesh nodes.**
[reach-mesh-services](reach-mesh-services.md) generalizes the
`ping6` you just ran to any IPv6-capable tool — `nc`,
`traceroute6`, `curl`, `ssh` — addressed by `<npub>.fips`.
The point is that the FIPS data plane is just IPv6;
applications don't need to know they're on a mesh.
- **Host a service of your own.**
[host-a-service](host-a-service.md) walks through bringing up
a small HTTP server bound to `fips0` so mesh nodes can reach
it, with a deliberate exposure decision (mesh-only vs every
interface), the mesh firewall, and a brief signpost to the
separate, unrelated peer ACL (which controls who may peer
with your node, not what they can reach on your `fips0`).
- [ground-up-mesh](ground-up-mesh.md) — Bring up two devices on
a shared physical link — Ethernet, WiFi, or Bluetooth — with
no pre-existing IP infrastructure. The second deployment mode
of FIPS, coexisting on the same daemon as the overlay peer to
`test-us01` you just configured.
For "what just happened, in detail":
- [../design/fips-architecture.md](../design/fips-architecture.md) —
the protocol stack and the two-layer encryption model.
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) —
Noise IK link encryption, hop-by-hop forwarding.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— end-to-end Noise XK, session lifecycle.
- [../design/fips-ipv6-adapter.md](../design/fips-ipv6-adapter.md) —
the TUN, the local DNS responder, MTU enforcement.
docs/tutorials/open-discovery.md
+338
View File
@@ -0,0 +1,338 @@
# Open Discovery: Find Peers Without Configuration
After
[advertise-your-node](advertise-your-node.md), your daemon
publishes its endpoint on Nostr and other open-discovery nodes
on the test mesh have already started dialing you. This
tutorial flips the symmetry: turn your own daemon into a
consumer of every advert in the namespace, so any operator
who's publishing becomes a candidate peer of yours.
The whole exercise should take about ten minutes. After it,
you'll be a full participant in the ambient
`fips-overlay-v1` namespace — publishing your own advert
*and* discovering everyone else's.
## What you'll build
```text
┌─────────────────────────────────────────────────────┐
│ Nostr relays — fips-overlay-v1 namespace │
│ adverts from: test-us01..uk01, others │
└────────────────┬─────────────────────────▲──────────┘
│ │
│ subscribe to all │ your own
│ adverts in namespace │ advert
│ │
▼ │
┌────────────────────────────────────────────────────┐
│ your fips daemon (policy: open) │
│ │
│ peers list grows ambient as adverts arrive: │
│ test-us01 ← was static │
│ test-us03 ← inbound (was already there) │
│ test-de01, test-es01, test-uk01, test-us04... │
│ plus any other publisher in the namespace │
└────────────────────────────────────────────────────┘
```
You will change one thing in `/etc/fips/fips.yaml`: under
`discovery.nostr`, set `policy: open` (the default is
`configured_only`). After restart, the daemon subscribes to
every Kind 37195 advert in the `fips-overlay-v1` namespace and
queues the publishers for outbound connection attempts.
## How open discovery works
> **Discovery policy.** `discovery.nostr.policy` decides what
> the daemon does with incoming advert data. Two values:
>
> - `configured_only` (the default): the daemon only consumes
> adverts for peers it has explicitly listed with
> `via_nostr: true`. This is what you've been running
> through the previous two Nostr tutorials.
> - `open`: the daemon subscribes to every advert in the
> configured `app` namespace. Any publisher becomes a
> candidate peer, no `peers:` list entry needed.
Switching the policy doesn't disturb anything that was already
working:
> **Open is additive, not exclusive.** Switching to
> `policy: open` doesn't replace your static peers — both
> mechanisms run in parallel. Configured peers stay in your
> `peers:` block and continue to be dialed via their static
> addresses or `via_nostr` lookups; open-discovered peers
> stack on top from the ambient namespace. You can run open
> discovery with a populated `peers:` list (the path this
> tutorial walks, since you're keeping `test-us01`), with
> `peers: []` for pure ambient discovery, or with a long
> `peers:` list and open layered on top to broaden reach.
The namespace is what scopes who's visible to whom:
> **The namespace is the scope.** `discovery.nostr.app`
> defaults to `fips-overlay-v1` — the namespace the public
> test mesh uses. Setting a different value (e.g.,
> `app: "my-experiment.v1"`) carves out a private discovery
> set: only nodes that share your `app` value find each
> other. For this tutorial we stay on the default and join
> the public namespace.
Open discovery is best-effort by design — not every
discovered peer will connect:
> **Best-effort, by design.** Many discovered peers will fail
> to connect — they may be offline, behind incompatible NAT,
> running a different protocol version, or have peer ACLs
> that reject you. That's normal for ambient discovery;
> connection attempts are best-effort and rate-limited by
> `open_discovery_max_pending` (default 64). Your peer list
> grows over time as candidates land in the cache, not all
> at once on restart.
The peer ACL is the admission-control surface, separate from
discovery:
> **Open is admission-free under your peer ACL.** Open
> discovery does not bypass the peer ACL — every candidate
> still has to pass it. By default the ACL accepts everyone,
> so any publisher in the namespace becomes a connection
> candidate. If you rely on a non-default ACL for admission
> control, verify it is set the way you want *before*
> enabling `policy: open`. See
> [../reference/security.md](../reference/security.md) for
> the ACL format.
## Step 1: Confirm your starting state
You should be coming out of
[advertise-your-node](advertise-your-node.md) with:
- Persistent identity, advertising enabled
(`discovery.nostr.advertise: true`), UDP advertising on
Nostr (`transports.udp.advertise_on_nostr: true`).
- A static `test-us01` peer entry that the daemon dials
outbound; possibly an inbound `test-us03` peer (the
open-discovery test mesh node that dialed in after seeing
your advert).
Capture the current peer count for comparison:
```sh
sudo fipsctl show peers | grep -c 'npub'
```
You'll likely see 1 (just `test-us01`) up to a handful, depending
on how many open-discovery test mesh nodes have already dialed
you.
## Step 2: Switch the discovery policy to `open`
Open `/etc/fips/fips.yaml` and find the `discovery.nostr`
block. Add (or change) the `policy` line:
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: true
policy: open
```
That's the only change. Notes on what you don't have to touch:
- **You don't have to drop the `peers:` block.** Static peers
and open-discovered peers coexist; static entries get
priority for direct dialing, open-discovered ones are
layered on top.
- **You don't have to set `app`.** The default
`fips-overlay-v1` is the namespace the public test mesh
uses; staying with the default is what gets you discovered.
- **You don't have to set `open_discovery_max_pending`.** The
default of 64 is plenty for a tutorial; only tune it if you
see the daemon log
`open-discovery: max-pending reached, deferring`.
Save the file.
## Step 3: Restart and let discovery populate
```sh
sudo systemctl restart fips
```
Give the daemon a minute or two. Open discovery doesn't fire
all at once — the daemon subscribes to the relays, accumulates
adverts as they arrive (or as relays return historical
events), and queues each publisher for a dial attempt.
## Step 4: Inspect the discovered peer list
```sh
sudo fipsctl show peers
```
You should see considerably more entries than before:
- `test-us01` — still there, still using the configured
static dial path.
- `test-us03` — same as before (the open-discovery test
mesh node that dials you when it sees your advert).
- `test-us04`, `test-de01`, `test-es01`, `test-uk01` — the
other test mesh nodes; your daemon picked their adverts up
from the namespace and dialed them.
- Plus any other operator publishing on `fips-overlay-v1`
(community nodes, other operators' experiments).
Each entry has its own `connectivity` state. Some will be
`active` (handshake completed). Some will show as
`connecting` and may transition to `failed` shortly after —
that's normal; the publisher might be offline, the advert
might be stale, or NAT traversal failed for that pair.
To get a list of just the active links:
```sh
sudo fipsctl show peers | jq '.peers[] | select(.connectivity == "active") | .npub'
```
The peer count will continue to drift over time as adverts
expire and new ones arrive. This is steady-state behavior,
not a transient.
## Step 5: Confirm the mesh-wide reach
You can now reach any of the discovered nodes the same way
you reached `test-us01` and `test-us02` in
[join-the-test-mesh](join-the-test-mesh.md). Pick one of the
new test mesh nodes from the peer list and ping it by its
shortname:
```sh
ping6 -c 4 test-uk01.fips
```
(`test-uk01` is the United Kingdom test node; the installer's
`/etc/fips/hosts` entry resolves it to the corresponding npub.
Substitute any active peer's shortname or full
`<npub>.fips`.)
Expect four replies. The packet path may go through your
direct link to that peer (if the open-discovery dial succeeded
and the link is up) or via a test-mesh forwarder (if the
direct link is down but the destination is still reachable
through the mesh). Either way, the npub-as-name addressing
works the same way.
## What you've learned
- **Open discovery is the consume side of full ambient
participation.** With `policy: open` plus advertising, your
daemon both publishes its advert *and* consumes everyone
else's in the namespace.
- **The namespace defines the scope.** Everyone publishing
with the same `app` value finds each other; different
namespaces are isolated discovery sets.
- **Best-effort means failure is normal.** Many discovered
peers won't actually connect; that's expected and
rate-limited by `open_discovery_max_pending`.
- **Static and open coexist.** Configured peers keep working
with their own dial paths; open-discovered peers stack on
top.
- **The peer ACL still gates everything.** Open is
admission-free relative to discovery, not relative to your
ACL — the ACL is what you'd use to restrict who can connect
if you don't want a fully open posture.
## Custom namespaces for private experiments
If you want to use FIPS open discovery for a private set of
nodes — colleagues, a workshop cohort, a specific deployment
— set a custom `app` value:
```yaml
discovery:
nostr:
enabled: true
advertise: true
policy: open
app: "my-team.experiment-1"
```
All nodes participating in the experiment use the same
`app` string. Pick something distinctive — short identifiers
risk colliding with other operators' experiments. Once your
nodes use a custom `app`, they no longer find or are found
by the public test mesh (the public mesh uses
`fips-overlay-v1`).
## Troubleshooting
If your peer list doesn't grow past the inbound peers from
the previous tutorial:
- **Wait.** Open discovery accumulates adverts; the first
pass after restart can take a couple of minutes to populate
on a new subscription.
- **Verify the namespace.** With `app:` unset, the daemon
uses `fips-overlay-v1`. If you set a custom `app:` for an
experiment, your daemon is in a different namespace than
the public test mesh and will only find peers using the
same value.
- **Check relay reachability.** Open discovery is a
subscription rather than one-shot queries — if the
WebSocket connection to the relays is failing repeatedly,
no adverts arrive. Look for relay-connection errors in
`sudo journalctl -u fips -n 200`.
- **`policy: open` typo.** YAML accepts and ignores unknown
values silently. If `fipsctl show status` (or the daemon's
startup log) shows `policy: configured_only`, the YAML
didn't parse the new value — re-check spelling and
indentation.
If too many peers are appearing and you want to dial down:
- **Lower `open_discovery_max_pending`.** Default 64; setting
it to e.g. 16 caps in-flight connection attempts. Adverts
beyond that wait in a queue.
- **Use a custom `app`.** Move to a private namespace where
only nodes you're coordinating with publish.
- **Use the peer ACL.** See
[../reference/security.md](../reference/security.md) for
the ACL format if you want explicit allow/deny rules.
## What's next
- **Reach services on other mesh nodes.**
[reach-mesh-services](reach-mesh-services.md) drives `nc`,
`traceroute6`, `curl`, and `ssh` at peers by `.fips` name —
any of the open-discovered peers in your list, or any node
you reach through them.
- **Host a service of your own.**
[host-a-service](host-a-service.md) brings up an HTTP server
addressable as `<your-npub>.fips`, bound to `fips0` so the
exposure is mesh-only, behind the mesh firewall.
- [ground-up-mesh](ground-up-mesh.md) — Bring up two devices on
a shared physical link (Ethernet, WiFi, or Bluetooth) with no
pre-existing IP infrastructure. The second deployment mode of
FIPS, a parallel to the overlay-on-internet path the
Nostr-discovery tutorials covered.
For the operator-style scenario reference covering all five
shapes of Nostr discovery side-by-side:
- [../how-to/enable-nostr-discovery.md](../how-to/enable-nostr-discovery.md)
§ Capability 3 — open discovery configuration knobs.
For the wire format and discovery design:
- [../reference/nostr-events.md](../reference/nostr-events.md)
— Kind 37195 advert format and the `app` namespace tag.
- [../design/fips-nostr-discovery.md](../design/fips-nostr-discovery.md)
— discovery runtime design, security and threat model.
docs/tutorials/persistent-identity.md
+304
View File
@@ -0,0 +1,304 @@
# Make Your Node's Identity Persistent
After completing
[join-the-test-mesh](join-the-test-mesh.md), your daemon is
connected to the public test mesh — but its identity is
ephemeral. Every restart generates a fresh Nostr keypair, so the
npub the rest of the world would use to reach you changes every
time. This tutorial walks through pinning your node to a stable
keypair, locating it on disk, and protecting it.
The whole exercise should take about ten minutes.
## What you'll build
```text
┌─────────────────────────────────────────┐
│ /etc/fips/fips.yaml │
│ node: │
│ identity: │
│ persistent: true ← this flag │
└────────────────┬────────────────────────┘
│ daemon reads on start
▼
┌─────────────────────────────────────────┐
│ /etc/fips/fips.key 0600 root:root │
│ /etc/fips/fips.pub 0644 root:root │
└─────────────────────────────────────────┘
your stable nsec / npub
```
After this tutorial your node will have:
- A keypair on disk that the daemon reuses across restarts.
- An npub you can hand to other operators so they can add you to
their `peers:` list once and have the entry keep working.
- A clear understanding of which file holds the secret and how
to keep it that way.
## Why a stable identity matters
In FIPS your Nostr keypair *is* your node's identity in the most
literal sense. Several things derive from it:
- Your `fd97:...` mesh address — derived from the public key.
- Your `<npub>.fips` DNS name — the npub itself with `.fips`
appended.
- Every authenticated connection — Noise IK at the mesh layer,
XK at the session layer, both prove you hold the matching
secret key.
> **A keypair, briefly.** Nostr identities are secp256k1
> keypairs. The private half is the `nsec` (a bech32-encoded
> secret key); the public half is the `npub` (a bech32-encoded
> public key). The daemon needs the `nsec` to sign messages and
> complete handshakes; the rest of the world only sees the
> `npub` and uses it as your address.
The daemon supports two ways of holding that keypair:
> **Ephemeral vs. persistent.**
>
> - *Ephemeral* (the default): the daemon mints a brand-new
> keypair every time it starts, kept only in memory. No
> long-term secret is ever written to disk; nothing on your
> machine ties one run to the next; the npub your daemon
> presents to the network is fresh on every restart. This is
> the safe-by-default posture — your node has no persistent
> identity unless you explicitly ask for one.
> - *Persistent*: the daemon reads (or, on first start,
> generates and writes) a keypair stored at
> `/etc/fips/fips.key`. The npub stays the same across
> restarts, reboots, and reinstalls as long as that file is
> preserved. You take on the cost of protecting an on-disk
> secret in exchange for being addressable by a stable name.
Persistent identity is a deliberate trade. You give up the
ephemeral default's privacy posture — once your npub is stable,
every connection your node makes is correlatable across time —
and you take on a real secret-management responsibility. In
return you get two things you can't get any other way:
1. **Other operators can reference you by npub.** In
[join-the-test-mesh](join-the-test-mesh.md) you added
`test-us01` to your `peers:` list by its npub. That entry
only works because `test-us01`'s npub doesn't change. If
anyone is going to reach you the same way, your npub has to
be just as stable.
2. **Services on your node get a fixed address.** The
[host-a-service](host-a-service.md) tutorial walks through
running an HTTP server addressable as
`<your-npub>.fips`. Clients reach the service by that
name; if your npub changes on every restart, every
client's address book breaks.
Both of these are good reasons. Neither is automatic — if your
node is purely a *client*, reaching out to others without
hosting anything itself, you may legitimately want to stay on
the ephemeral default. The rest of this tutorial assumes you've
decided you want a stable identity.
## Step 1: Note your current ephemeral npub
Before changing anything, capture the npub the daemon is using
right now so you can compare against it after the switch.
```sh
sudo fipsctl show status | grep '"npub"'
```
You'll see one line like:
```text
"npub": "npub1abc...xyz"
```
Make a note of it. We expect this to change.
## Step 2: Enable persistent identity in the config
Open `/etc/fips/fips.yaml` and find the `node:` block. The
shipped default has the relevant fragment commented out; make it
look like this:
```yaml
node:
identity:
persistent: true
```
Save the file. That is the only configuration change.
The daemon's behavior on the next restart:
- If `/etc/fips/fips.key` already exists, load it and use that
identity.
- If it does not exist, generate a fresh keypair, write it to
`/etc/fips/fips.{key,pub}` with the correct file modes, and
use that.
## Step 3: Restart the daemon
```sh
sudo systemctl restart fips
sudo systemctl status fips
```
Status should show `active (running)` within a couple of
seconds. Confirm the new identity is in use:
```sh
sudo fipsctl show status | grep '"npub"'
```
The npub should be **different** from the one in Step 1 — the
daemon discarded the old in-memory ephemeral keypair and minted
a new one which it has now persisted to disk. From here forward
this is *your* npub.
## Step 4: Locate the keypair on disk
The daemon wrote two files:
```sh
sudo ls -l /etc/fips/fips.key /etc/fips/fips.pub
```
Expect:
```text
-rw------- 1 root root ... fips.key
-rw-r--r-- 1 root root ... fips.pub
```
The public file is safe to share — it is your address:
```sh
sudo cat /etc/fips/fips.pub
```
This must match the `npub` reported by `sudo fipsctl show
status`. Hand this string to anyone you want to be reachable
from; they paste it into their own `peers:` block as the `npub:`
field.
The private file is the secret. **Do not** `cat` or paste its
contents anywhere — there is no reason to see it, and any line
of shell history or screen capture that contains it has captured
the secret.
## Step 5: Verify it survives a restart
Restart once more to confirm the daemon is reading `fips.key`
rather than re-generating it:
```sh
sudo systemctl restart fips
sudo fipsctl show status | grep '"npub"'
```
The npub should match Step 3 exactly. If it does not, the daemon
was unable to read `fips.key` (most likely a permission problem)
— see [Troubleshooting](#troubleshooting).
## Step 6: Protect the nsec
`fips.key` is the only thing standing between you and someone
else impersonating your node. The daemon ships it with the right
permissions; the operator's job is to keep them that way.
What that means in practice:
- **Do not loosen the file mode.** `0600` (read/write for owner
only — here `root`) is correct; `chmod 0644` to "fix" a
permission error puts the secret on display to every account
on the host.
- **Do not commit it to source control.** If you maintain
configuration in a Git repo, exclude `fips.key`; if you use
ansible-vault or a similar mechanism, encrypt it. `fips.pub`
may be checked in freely.
- **Do not paste it into chat or email.** Operators sometimes
share config snippets to demonstrate a setup; redact the
contents of `fips.key` to a placeholder before doing so.
- **Back it up the way you back up an SSH host key.** Treat the
file (or its contents) the same way you would treat
`/etc/ssh/ssh_host_ed25519_key`: encrypted, offline, available
to recover the *same* identity if the host disappears.
There is no in-protocol "key change" message in FIPS. If
`fips.key` is lost, the npub is lost — your node will come back
up with a new identity and every downstream reference to the
old one will be stale.
## What you've learned
- **Identity = keypair.** Every FIPS node is a Nostr keypair;
the npub is its address, the nsec is its credential.
- **The flag.** `node.identity.persistent: true` in
`/etc/fips/fips.yaml` is the difference between ephemeral and
persistent identity.
- **Where it lives.** `/etc/fips/fips.key` and
`/etc/fips/fips.pub`, mode `0600` and `0644`, owned
`root:root`.
- **What to share.** `fips.pub` is public; `fips.key` is not.
- **What it buys you.** A npub other operators can add to their
`peers:` list once, and that addresses the services your node
will eventually run.
## Troubleshooting
If the post-restart npub does not match `fips.pub`:
- **Check file permissions.**
`sudo ls -l /etc/fips/fips.key`. If the mode is not `0600` or
the owner is not `root:root`, the daemon may have refused to
read it. Restore with
`sudo chmod 0600 /etc/fips/fips.key && sudo chown root:root
/etc/fips/fips.key`.
- **Check the journal.** `sudo journalctl -u fips -n 100` after
the restart will show one of:
- `Loaded persistent identity from key file path=...` — good.
- `Generated persistent identity, saved to key file ...` —
also good, but only expected on the first start after the
flag flip.
- `Using ephemeral identity (new keypair each start)` — the
config flag was not picked up; re-check the indentation of
the `persistent: true` line.
If you see `Generated persistent identity...` on every start,
the file is being written but not read on subsequent starts;
this is almost always the same permission/path issue.
## What's next
- **Resolve peer addresses via Nostr.**
[resolve-peers-via-nostr](resolve-peers-via-nostr.md) walks
through the smallest useful step toward Nostr-mediated
discovery: keep your peer entry, drop its hard-coded address,
and let the daemon look up the current endpoint from public
Nostr relays. The first of three tutorials covering Nostr
discovery; advertising your own node and open ambient
discovery come next.
- **Reach services on other mesh nodes.**
[reach-mesh-services](reach-mesh-services.md) drives `nc`,
`traceroute6`, `curl`, and `ssh` at peers by `.fips` name and
shows that the FIPS data plane is just IPv6 from an
application's point of view.
- **Host a service of your own.**
[host-a-service](host-a-service.md) walks through bringing up
an HTTP server addressable as `<your-npub>.fips`, bound to
`fips0` so the exposure is mesh-only, behind the mesh
firewall.
For the alternative provisioning paths — minting a keypair with
`fipsctl keygen` before the daemon ever starts, or importing an
existing Nostr `nsec` — and the key-rotation procedure, see the
operator-style recipe at
[../how-to/persistent-identity.md](../how-to/persistent-identity.md).
For the full identity model:
- [../design/fips-architecture.md](../design/fips-architecture.md)
— how npubs become `NodeAddr`s and IPv6 ULAs.
docs/tutorials/reach-mesh-services.md
+253
View File
@@ -0,0 +1,253 @@
# Reach Services on Other Mesh Nodes
In [join-the-test-mesh](join-the-test-mesh.md) you used `ping6`
to reach `test-us01` and `test-us02` by their `.fips` names.
This tutorial generalizes that to any IPv6-capable tool you
already use — `nc`, `traceroute6`, `curl`, `ssh`, `scp`,
anything — and gets you comfortable with the daemon's IPv6
adapter, which makes the FIPS mesh look like an ordinary IPv6
network to applications that already know how to use IPv6.
The whole exercise should take about ten minutes.
## What you'll do
You'll ping a mesh node (recap), attempt a TCP connection to it
with `nc`, and trace the packet path with `traceroute6` — all
by hostname. By the end you will have driven three different
IPv6 tools at a mesh address and seen each one work the same
way it works on the regular internet.
> **An IPv6 adapter for a non-IPv6 mesh.** The FIPS network
> itself routes blobs of data between npub-addressed nodes;
> on its own it has nothing to do with IPv6. The daemon
> includes an *IPv6 adapter* that presents the mesh as an
> ordinary IPv6 interface (`fips0`), so existing IP software
> works without modification. The kernel routes packets to
> it, applications open IPv6 sockets through it, and the
> adapter handles encapsulating each packet and routing it
> through the mesh to the matching adapter on the other side.
> Any tool that speaks IPv6 works unchanged.
The IPv6 adapter is currently the main way operators use the
FIPS network, which is why most of the new-user progression is
about it. Native applications can use the mesh without going
through IPv6 at all, but that is out of scope for this tutorial.
## Addressing a mesh node
Throughout this tutorial — and any time you reach across the
mesh — use a node's `.fips` hostname directly. There are two
forms:
- **`<npub>.fips`** — the canonical form. Every node has one,
always. This is the long bech32 npub with `.fips` appended.
- **`<shortname>.fips`** — the convenience form, *if* you (or
the package) have an entry for the node in
`/etc/fips/hosts`. The installer ships entries for the
public test mesh, so `test-us01.fips` works on a fresh
install.
These are real hostnames as far as your kernel is concerned.
Pass them to any IPv6-capable tool — `ping6`, `nc`, `curl`,
`ssh`, `traceroute6`, anything — the same way you would pass a
hostname on the public internet. There is no separate
"resolve to address first" step you ever need to perform; if
the tool takes a hostname, it accepts a `.fips` hostname.
> **Where the address comes from.** Every FIPS node's mesh
> address is the SHA-256 of its public key, truncated to the
> bottom 64 bits and prepended with `fd97:`. Names of the form
> `<npub>.fips` and any shortname mapped in `/etc/fips/hosts`
> are aliases for that address. The daemon's local DNS
> responder hands the answer back to your kernel without ever
> talking to a remote DNS server.
## Step 1: Ping a mesh node (recap)
```sh
ping6 -c 4 test-us01.fips
```
You did this in [join-the-test-mesh](join-the-test-mesh.md).
Four replies, RTT in the tens of milliseconds (depending on
where you are relative to `test-us01`). Nothing new — but it
confirms the mesh data plane is healthy before you try
anything else.
## Step 2: Attempt a TCP connection
`ping6` proves ICMPv6 reaches the destination. To prove TCP
reaches it, use `nc` (netcat) to attempt a connection to a port.
Pick any port — whether it has a service listening or not, the
attempt proves the data plane carries your TCP segments
end-to-end:
```sh
nc -6 -vz test-us01.fips 22 2>&1
```
You will see one of two outcomes:
```text
Connection to test-us01.fips 22 port [tcp/ssh] succeeded!
```
or:
```text
nc: connect to test-us01.fips port 22 (tcp) failed: Connection refused
```
Both are good. The first means a service is listening on that
port and accepted your TCP handshake. The second means your TCP
SYN reached the remote node's kernel, which sent back a TCP RST
because no service was bound — and that RST traveled all the way
back through the mesh to your `nc` process.
> **What a `Connection refused` proves.** A connection-refused
> response is *not* a network failure. It means the destination
> host is alive and reachable, the TCP stack on the far end
> processed your SYN, and the reply made it home. Compare with
> what you would get if the address were unreachable:
> `Network is unreachable` or a timeout. Either of the two
> outcomes above demonstrates a working end-to-end TCP path.
If the port you tried happens to have a service, attach `-`
instead of `-z` and you can read the banner directly:
```sh
nc -6 -v test-us01.fips 22
```
The remote node's SSH banner, if any, will print on the next
line. Type `Ctrl-C` to disconnect — you have not authenticated,
just banner-grabbed.
If `nc` is not installed, the same demonstration works with
`curl` against TCP/80:
```sh
curl -6 -v --connect-timeout 5 http://test-us01.fips/ 2>&1 | head
```
The TCP connection result is in the first few lines of `curl`'s
verbose output. The HTTP response code is irrelevant — what
matters is whether the connection itself succeeded.
## Step 3: Trace the path
`traceroute6` shows the IPv6 hops between you and a
destination:
```sh
traceroute6 -n test-us02.fips
```
You will see exactly one line — `test-us02`'s mesh address.
That is the only IPv6 hop between your `fips0` and
`test-us02`'s `fips0`, even though at the FIPS-mesh layer
your packet is being forwarded through your peer `test-us01`
on the way to `test-us02`. The mesh-layer forwarding is
invisible to `traceroute6` because it lives below the IPv6
adapter.
> **Two layers, two ideas of "hop".** The FIPS mesh routes
> blobs between npub-addressed nodes and can pass through
> several intermediate peers — your packet to `test-us02` is
> handed off to `test-us01` first. The IPv6 adapter, sitting
> on top of that, presents every reachable mesh node as a
> direct IPv6 neighbor: one hop, on a flat fabric. From
> `traceroute6`'s perspective the multi-hop FIPS path is
> hidden — it sees only the source and destination IPv6
> adapters. To see what's happening at the mesh layer, see
> [ipv6-adapter-walkthrough](ipv6-adapter-walkthrough.md),
> which traces one `ssh` request from DNS query to far-side
> TUN with `fipstop` and `fipsctl` running alongside.
If `traceroute6` is not installed, `mtr` and other IPv6 path
tools produce the same single-hop result. The single-hop
behavior is a property of the IPv6 adapter, not of the tool.
## What you've learned
You have driven three IPv6 tools at mesh nodes you reach over
the mesh, all by `.fips` hostname, and they all worked the same
way they work everywhere else:
- **Addressing.** `<npub>.fips` is the canonical hostname for
any node; `<shortname>.fips` is the convenience form when
`/etc/fips/hosts` has an entry. Use these in any tool that
takes an IPv6 hostname — there is no separate resolution
step you ever need to perform.
- **Reachability.** `ping6` confirms the remote node's `fips0`
answers ICMPv6 echo from your `fips0`.
- **TCP.** `nc` confirms TCP segments traverse the mesh and the
far side responds (whether with a banner, a refusal, or a
service of its own).
- **Path.** `traceroute6` shows exactly one IPv6 hop to any
reachable mesh node, because the multi-hop FIPS-mesh-layer
forwarding lives below the IPv6 adapter and is invisible
to IPv6 tooling.
The conceptual takeaway is the one in the callout at the top:
the daemon's IPv6 adapter takes care of presenting the FIPS
mesh as ordinary IPv6 to every tool you already know. To
consume any service hosted on any mesh node — SSH, HTTP, file
transfer, custom protocols — you use the IPv6 client you
would use anywhere else. The hostname looks unusual
(`<npub>.fips`), but the API surface is unchanged.
## Troubleshooting
If a tool reports "Network is unreachable" or hangs:
- **Confirm the link is healthy.**
`sudo fipsctl show peers` should show `test-us01` with active
connectivity. If the link to your direct peer is down, nothing
past it is reachable.
- **Confirm `fips0` is up.** `ip -6 addr show fips0` should show
one `fd97:...` address. If `fips0` is missing, the daemon did
not bring up the TUN — verify the daemon is running with the
privileges it needs. The default is to run as root; if you
dropped privileges per
[../how-to/run-as-unprivileged-user.md](../how-to/run-as-unprivileged-user.md),
re-check that the `setcap` and systemd override survived your
last package upgrade.
- **Confirm the name resolves.** If `ping6 test-us01.fips`
fails with `unknown host` or `Name or service not known`,
the system resolver is not consulting the daemon's `.fips`
responder. The installer wires this up automatically; the
"Reaching mesh nodes by name" section of
[../getting-started.md](../getting-started.md) describes
what the wiring looks like and how to confirm it.
If `nc` or `curl` reports a timeout (rather than a refusal or
success), the destination node is unreachable from your
daemon — possible mesh-routing transient. Try again, or ping
first: if `ping6` succeeds but TCP times out, it is the
specific port being filtered on the destination, not a path
problem.
## What's next
- [host-a-service](host-a-service.md) — Bring up a small HTTP
server on your node, bind it to `fips0` so it is mesh-only,
and confirm another mesh node (or your own machine) can
reach it through the same data plane you just exercised.
Covers bind-interface choice and the mesh firewall.
For "what's actually in those packets":
- [../design/fips-architecture.md](../design/fips-architecture.md)
— the protocol stack and the two-layer encryption model.
- [../design/fips-mesh-layer.md](../design/fips-mesh-layer.md) —
Noise IK link encryption, hop-by-hop forwarding.
- [../design/fips-session-layer.md](../design/fips-session-layer.md)
— end-to-end Noise XK between source and destination.
For the trace-it-yourself version of the path you just
exercised, see
[ipv6-adapter-walkthrough](ipv6-adapter-walkthrough.md), which
walks one `ssh` from DNS query through session setup to the
far-side TUN with `fipstop` and `fipsctl` running alongside.
docs/tutorials/resolve-peers-via-nostr.md
+300
View File
@@ -0,0 +1,300 @@
# Resolve Peer Addresses via Nostr
After
[persistent-identity](persistent-identity.md), your daemon has
a stable npub and is peered with `test-us01` over a hard-coded
UDP address (`test-us01.fips.network:2121`). That static
address works fine until `test-us01` moves to a new IP, swaps
ports, or starts publishing additional endpoints you'd want to
reach. The npub is stable; the set of network endpoints behind
it may not be.
This tutorial shows the smallest useful step toward Nostr-
mediated discovery: keep the peer entry but drop its address,
let your daemon ask public Nostr relays for the peer's current
endpoint, and verify the link still works. You will not be
publishing anything yourself yet — this is the consume-only
case.
The whole exercise should take about ten minutes.
## What you'll build
```text
┌──────────────────────────┐
│ Nostr relays │
│ relay.damus.io │
│ nos.lol │
│ offchain.pub │
└────────────▲─────────────┘
"what's test-us01's │ signed advert
current address?" │ (Kind 37195)
│ from test-us01
┌───────────────────────┐ │
│ your fips daemon │ ─────────┘
│ peers: │
│ - test-us01 npub │ ─── dial resolved UDP ──▶ test-us01
│ via_nostr: true │
└───────────────────────┘
```
You'll change two things in `/etc/fips/fips.yaml`:
- Add a `node.discovery.nostr` block that turns the consume-
side of Nostr discovery on.
- Edit the existing `test-us01` peer entry to drop its hard-
coded `addresses:` block and add `via_nostr: true`.
After restart, the daemon will fetch `test-us01`'s current
advert from the relays, use the endpoint listed there, and
peer normally.
## How Nostr discovery resolves an address
Every FIPS daemon with `node.discovery.nostr.advertise: true`
publishes a signed Nostr event (Kind 37195) listing the
transport endpoints it is willing to accept connections on.
The event is signed by the daemon's secret key, so anyone
who has the corresponding npub can verify the advert really
came from that node.
`test-us01` runs with `advertise: true`. Its current advert is
visible to any Nostr client.
> **Identity is stable; endpoints are not.** A peer's npub is
> a long-lived identifier — it is who they are. Their UDP
> address, port, or transport choice is metadata that may
> change. Nostr discovery lets you bind your peer entry to the
> npub and lets the relay tell your daemon the current
> endpoint at dial time.
There are two halves to this — consuming adverts (looking up
peers by npub) and publishing adverts (being lookup-able). This
tutorial covers only the consume half.
> **Consume vs. publish.** This tutorial enables only the
> *consume* side: your daemon queries relays to resolve peers
> by npub. It does not publish an advert of its own — others
> still cannot find you by your npub yet. The next tutorial
> (`advertise-your-node`) handles the publish side.
## Step 1: Confirm your starting state
You should currently have:
- A persistent npub from
[persistent-identity](persistent-identity.md). Confirm:
```sh
sudo fipsctl show status | grep '"npub"'
```
- A working static peering with `test-us01`. Confirm:
```sh
sudo fipsctl show peers
```
Expect `test-us01` listed with `connectivity` active and a
`transport_addr` of roughly `217.77.8.91:2121`.
If either of those isn't true, finish the previous two
tutorials first; the Nostr discovery layer is built on top of
that working state.
## Step 2: Enable the consume side of Nostr discovery
Open `/etc/fips/fips.yaml` and add a `discovery` block under
`node:`:
```yaml
node:
identity:
persistent: true
discovery:
nostr:
enabled: true
advertise: false
```
Two knobs, one job each:
- `enabled: true` turns on the Nostr discovery runtime — the
daemon connects to a default relay set
(`wss://relay.damus.io`, `wss://nos.lol`,
`wss://offchain.pub`) and is now able to query and consume
adverts.
- `advertise: false` keeps the publish side off. Your daemon
will not publish an advert of its own at this stage. (This
is the default, but it is good practice to make the choice
explicit while you're learning.)
## Step 3: Switch the peer entry to `via_nostr`
Find the `peers:` block you added during
[join-the-test-mesh](join-the-test-mesh.md) and change it from
this:
```yaml
peers:
- npub: "npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98"
alias: "test-us01"
addresses:
- transport: udp
addr: "test-us01.fips.network:2121"
connect_policy: auto_connect
```
to this:
```yaml
peers:
- npub: "npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98"
alias: "test-us01"
via_nostr: true
connect_policy: auto_connect
```
What changed: the `addresses:` list is gone, replaced by
`via_nostr: true`. The npub stays — it is what the daemon
matches against the advert publisher's pubkey.
Save the file.
## Step 4: Restart the daemon
```sh
sudo systemctl restart fips
sudo systemctl status fips
```
The status output should show `active (running)` within a
couple of seconds. The Nostr discovery runtime starts alongside
the rest of the daemon, fetches `test-us01`'s advert from the
default relays, and uses the endpoint listed there to dial.
The resolution itself happens at debug-log level, so you will
not see it in the default-level journal. The user-facing way to
confirm everything worked is `fipsctl show peers` in the next
step. (To watch the resolution in the journal, run the daemon
manually with `RUST_LOG=fips::discovery::nostr=debug`; not
necessary for this tutorial.)
## Step 5: Verify the resolved endpoint
```sh
sudo fipsctl show peers
```
`test-us01` should appear with `connectivity` active and a
`transport_addr` reflecting the address that was resolved from
the advert — `217.77.8.91:2121` at time of writing. That field
is the strong signal: nothing in your config gave the daemon
that IP, yet there it is.
You can confirm independently that the address came from the
advert. The advert is a public Nostr event — anyone can fetch
it. With the `nak` Nostr CLI installed:
```sh
nak req -k 37195 -d "fips-overlay-v1" \
-a 06f11c31227938ab98ba982280d2826f66a063f9efe7e342e81d6a76c677d7ed \
--limit 1 wss://relay.damus.io
```
(That hex pubkey is the same identity as
`npub1qmc3...zel98` — Nostr filters take hex.) The `content`
field of the returned event lists the `endpoints` array; one
of its entries should match what `fipsctl show peers` is
using. That is what your daemon just did, signed and verified
by the Nostr layer.
## Step 6: Confirm reachability still works
```sh
ping6 -c 4 test-us01.fips
```
Expect four replies, exactly as in
[join-the-test-mesh](join-the-test-mesh.md) (which used the
full npub form). Nothing about the data plane has changed;
only the way you discovered the endpoint to dial.
## What you've learned
- **Adverts are signed.** Every Nostr discovery advert is
signed by the publisher's secret key, so the address you
resolved through a public relay is trustworthy in the same
sense the peer's npub is.
- **`via_nostr` replaces a static address.** A peer entry
with no `addresses:` block and `via_nostr: true` directs
the daemon to look the endpoint up at dial time.
- **The relay set is small and public.** Three default
relays today; the daemon round-robins queries across them.
No central FIPS infrastructure is involved.
- **Static and Nostr can mix.** You replaced the static
address with `via_nostr` here, but you could have kept both
— when both are present, static endpoints are tried first
and Nostr-resolved endpoints are appended as a fallback.
Useful when you want a fast-path direct dial but a
resilient fallback.
## Troubleshooting
If the link does not come up:
- **No advert on the relays.** If the peer's daemon is
offline or has `advertise: false`, no advert exists for
your daemon to consume. Verify with `nak` (Step 5) — if the
query returns nothing, that is the problem and it is on the
peer's side. Re-add the static `addresses:` entry as a
fallback while you wait for the peer to come back up.
- **Relay reachability.** `Connected to relay` lines should
appear for at least one of the three default relays. If
none do, your network may be filtering outbound WebSocket
traffic or DNS for those hostnames. Check the journal for
TLS/DNS errors.
- **Stale cache.** The daemon caches resolved endpoints
briefly. If a peer's advert changes mid-session and you
hit a stale entry, restart the daemon to force a fresh
query.
- **Persistent identity not on.** If the journal shows
`Using ephemeral identity (new keypair each start)`, the
daemon falls back to ephemeral and the consume-side may
not behave as expected. Re-check
[persistent-identity](persistent-identity.md) Step 2.
## What's next
- **Advertise your own node.**
[advertise-your-node](advertise-your-node.md) publishes your
daemon's UDP endpoint on Nostr so other operators can add
you to their `peers:` list with `via_nostr: true` and reach
you the way you just reached `test-us01`. Includes a section
on `udp:nat` NAT traversal for symmetric-NAT networks.
- **Discover peers with no prior configuration.**
[open-discovery](open-discovery.md) switches your daemon to
`policy: open` so the ambient namespace itself populates
your peer list — no static `peers:` entries required (the
static ones can stay too; the two mechanisms run in
parallel).
For the operator-style scenario reference covering all five
shapes of Nostr discovery side-by-side (consume-only,
publish-direct, publish-Tor, NAT traversal, open):
- [../how-to/enable-nostr-discovery.md](../how-to/enable-nostr-discovery.md)
— five scenarios with minimal YAML fragments.
For the design and security model:
- [../design/fips-nostr-discovery.md](../design/fips-nostr-discovery.md)
— discovery runtime architecture, advert format, threat
model.
For the wire-format details:
- [../reference/nostr-events.md](../reference/nostr-events.md)
— Kind 37195 advert format, Kind 21059 traversal signaling,
Kind 10050 inbox-relay list.