mirror of
https://github.com/jmcorgan/fips.git
synced 2026-08-09 08:14:42 +00:00
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.
The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.
== Cargo feature and dependencies
- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
`src/discovery/nostr/signal.rs` rather than relying on the SDK's
rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
exclusively.
== Wire format
Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.
Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.
Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.
== Discovery surface
- `src/discovery.rs` (always compiled)
- `EstablishedTraversal`: bound UDP socket + selected remote +
peer npub + optional transport name/config tuning overrides.
- `BootstrapHandoffResult`: returned on successful handoff —
allocated transport id, local/remote addrs, peer NodeAddr,
session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
- `types.rs`: wire and control types described above. `ADVERT_KIND`
constant. `BootstrapError` enumerates failure modes (disabled,
missing advert, missing NAT endpoint, no usable relays, invalid
advert, invalid npub, signal timeout, punch timeout, replay,
STUN failure, protocol, nostr, io, serde, event-parse).
- `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
nostr-sdk `Client`, subscribes to advert + signal event kinds,
maintains a bounded advert cache and a bounded seen-sessions
replay set, drains `BootstrapEvent::{Established, Failed}` for
the node to consume, exposes `update_local_advert`,
`request_connect`, `advert_endpoints_for_peer`,
`cached_open_discovery_candidates`, and `shutdown`.
- `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
built against per-attempt ephemeral keys; inbound events are
unwrapped against the node identity.
- `stun.rs`: RFC 5389/8489 Binding Request client with
XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
observe the initiator's own reflexive address against its
locally configured STUN list (peer-advertised STUN is
informational, never an egress target).
- `traversal.rs`: per-attempt candidate-pair punch planner.
Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
LAN-private and ULA interface addresses alongside the STUN
reflexive address, schedules probe/ack exchanges at the
configured interval for the configured duration, and picks the
first candidate pair that authenticates end-to-end.
Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.
== Configuration surface
`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:
- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
for both relay lists, and Google / Cloudflare / Twilio for STUN.
Operators are expected to override for production. Other
verified-working public relays for reference:
`nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
`nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
Open}` — controls whether advert-derived endpoints are consumed
only for peers carrying `via_nostr = true`, or also for
non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
`local_addresses` list is empty and peers see only the reflexive
address. Enable per-node only for genuinely same-LAN deployments;
off-by-default eliminates the misleading-path failure mode for
the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
(2048) — bound memory under ambient relay volume; overflow
evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
`punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)
Per-peer and per-transport flags:
- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
advert-derived addresses are appended as fallback dial candidates
after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
when `via_nostr: true`; validation requires at least one of the
two to be present per peer, and the error message names the
peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
`UdpConfig.public: Option<bool>` — UDP transports can be
advertised either as direct `host:port` (public = true) or as the
`addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
— TCP and Tor onion endpoints can be advertised as directly
reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
special-casing in YAML and routes through the bootstrap runtime.
Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):
- Any transport with `advertise_on_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
`public = false` — i.e. `udp:nat`) additionally requires at least
one `dm_relay` and at least one `stun_server`.
Surfaced as `ConfigError::Validation`.
== Node integration
`src/node/lifecycle.rs` is the main integration point.
- At node start (after transports are up, before TUN), if Nostr is
enabled and the feature is compiled in, `NostrDiscovery::start` is
invoked, the initial local overlay advert is built from the live
transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
at method definition and call site), which refreshes the local
advert, drains bootstrap events, adopts established traversals,
schedules retries for failed traversals, and — under `policy:
open` — enqueues outbound retries for non-configured peers
visible in the advert cache, bounded by
`open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
first exhausts the static address list in priority order and only
then appends advert-derived fallback addresses; both lists run
through the same `attempt_peer_address_list` code path. The
`udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
optional transport instance name), and emits an `OverlayAdvert`
including `signalRelays` and `stunServers` when any UDP endpoint
is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
allocates a new `TransportId`, constructs a `UdpTransport` with
the user-supplied (or default) `UdpConfig`, calls the new
`adopt_socket_async` to reuse the punched socket verbatim,
registers the transport in the normal transport map, records it
in `bootstrap_transports`, and calls `initiate_connection` so the
normal handshake path runs. On failure, the transport is stopped
and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
background tasks stop before transports are torn down. (The
advert is not explicitly retracted; NIP-40 expiration plus the
next refresh from any live publisher supersedes it.)
New `Node` fields:
- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
transports adopted from NAT traversal, cleaned up via
`cleanup_bootstrap_transport_if_unused` whenever the link,
connection, peer, or pending-connect referencing them is removed.
Retry and error surface:
- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
for a retry entry. `pump_retries` drops expired entries with an
info log. Used for open-discovery retries, which expire at two
times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
`adopt_established_traversal` when the underlying transport
adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
lifecycle, rx-loop tick, and timeout bookkeeping.
== UDP transport
`src/transport/udp/`:
- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
adopts an externally bound socket, makes it non-blocking, applies
the configured buffer sizes (warning if the kernel clamps), and
reports the resulting local address. Preserves the NAT mapping —
no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
`start_async` analogue for an already-bound socket, wiring the
async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
still holding a recv task or socket (for example on error
teardown), aborts the task, clears the socket, and emits a debug
log so the cleanup is visible in tracing rather than silent.
== Logging and observability
Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.
Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:
- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`
Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.
Other operator-facing logs:
- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
successful return, tagged with peer npub, session id, transport
id, and both socket endpoints, so the bootstrap handoff is
traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
downstream STUN, punch, and handshake logs for the same peer
correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
(hot path: fires for every unrelated signal event on the same
relay).
- Traversal-offer handler failures log at debug. Expected conditions
such as punch timeout on symmetric NAT are covered there; real
problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
(`max_concurrent_incoming_offers`) and state that the offer was
rate-limited rather than failing.
== Tests
- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
encoding, signal envelope round-trip, STUN parsing, punch-packet
codec, and replay-window enforcement. Run under the
`nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
cross-field invariants and YAML parsing of the full
`node.discovery.nostr` block plus `peers[].via_nostr`, empty
`addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
synthetic traversal (bound UDP socket pair + synthetic peer
identity) through `adopt_established_traversal` and assert the
Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
- Local `strfry` relay, local STUN responder, and one or two
router containers performing `iptables` NAT.
- Node LAN interfaces are provisioned with explicit `veth` pairs
injected into the node and router namespaces so every packet
traverses the router namespace (plain Docker bridges are not
used for the LAN).
- `cone` scenario: both peers behind full-cone-emulation NAT
(SNAT with source-port preservation, inbound DNAT back to the
single LAN host regardless of remote source); asserts UDP
traversal succeeds and link remote addresses are on the router
WAN subnet.
- `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
traversal fails and TCP fallback converges over router-
published WAN addresses.
- `lan` scenario: both peers share a LAN subnet; asserts LAN
addresses are preferred over reflexive ones.
- Cleanup tears down all profile-gated services
(`--profile cone --profile symmetric --profile lan`) so no
orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
`--features "tui nostr-discovery"` by default so NAT-harness
binaries include bootstrap support.
== CI
- Linux release build and nextest unit-test job both use
`--features "gateway nostr-discovery"` so the feature-gated code
and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
`nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
`docker compose logs` on failure, and always stop containers.
== Packaging and operations
- `packaging/common/fips.yaml` ships a fully commented
`node.discovery.nostr.*` block, plus documented
`advertise_on_nostr` / `public` examples under the UDP transport,
an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
example under the static peer section with both a direct
`host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
publishes target the new default relay set.
== Documentation
- `README.md`: overlay discovery + NAT traversal moved from
"Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
previously described Nostr discovery and NAT traversal as future
work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
direction)" qualifier from the Nostr Relay Discovery section,
expands with the `udp:nat` advertisement and bootstrap handoff
description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
rebinding (roaming) and initial NAT traversal (Nostr path) are
distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
`node.discovery.nostr.*` surface, including the three resource
caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
reference for the shipped mechanism, including the empty-
`addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
Implemented status callout, clarifies that the punch socket is
per-peer and per-attempt rather than shared with the application
listener, aligns field names with the shipped JSON
(`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
`localAddresses`, `stunServer`), sets the `d`-tag to
`fips-overlay-v1`, names the kind as 37195, and notes that
advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
path, STUN/punch path, configuration surface, and Docker NAT lab.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
823 lines
40 KiB
Markdown
823 lines
40 KiB
Markdown
# FIPS: Free Internetworking Peering System
|
||
|
||
## What is FIPS?
|
||
|
||
FIPS is a self-organizing mesh network that can operate natively over a
|
||
variety of physical and logical media, such as local area networks,
|
||
Bluetooth, serial links, or the existing internet as an overlay. The
|
||
long-term goal is infrastructure that can function alongside or ultimately
|
||
replace dependence on the Internet itself. Systems running FIPS establish
|
||
peer connections, authenticate each other, and route traffic for each other
|
||
without any central authority or global topology knowledge, and allow
|
||
end-to-end encrypted sessions between any two nodes regardless of how many
|
||
hops separate them.
|
||
|
||
Nodes in the mesh route traffic for each other using Nostr identities
|
||
(npubs) as network addresses. Applications can access the mesh through a
|
||
native FIPS datagram service, or through an IPv6 adaptation layer that
|
||
presents each node as an IPv6 endpoint for compatibility with existing
|
||
IP-based applications.
|
||
|
||
## Why FIPS?
|
||
|
||
**Self-sovereign identity**: FIPS nodes generate their own addresses, node
|
||
IDs, and security credentials without coordination with any central
|
||
authority. These identities can be long-term fixed or may be ephemeral,
|
||
changed at any time. These identities are not visible to the FIPS network
|
||
itself — they are used only at the application layer and for end-to-end
|
||
session encryption.
|
||
|
||
**Infrastructure independence**: The internet depends on centralized
|
||
infrastructure — ISPs, backbone providers, DNS, certificate authorities.
|
||
FIPS works over any transport that can carry packets: a serial connection,
|
||
onion-routed connections through Tor, local area networking, radio links
|
||
between remote sites, or the existing internet as an overlay. When the
|
||
internet is unavailable, unreliable, or untrusted, the mesh still works.
|
||
|
||
**Privacy by design**: FIPS provides secure, authenticated, and encrypted
|
||
communication between any two nodes in the mesh, independent of the mix of
|
||
transports used along the routed path between them. Furthermore, the mesh
|
||
itself is designed to minimize metadata exposure — intermediate nodes route
|
||
packets without learning the identities of the endpoints.
|
||
|
||
**Zero configuration**: Nodes discover each other and build routing
|
||
automatically. Connect to one peer and you can reach the entire mesh. The
|
||
network self-heals around failures and adapts to changing topology.
|
||
|
||
## A Self-Organizing Mesh
|
||
|
||
Traditional networks are built top-down. A central authority assigns
|
||
addresses, configures routing tables, provisions hardware, and manages the
|
||
topology. If the authority disappears or the infrastructure fails, the
|
||
network fails with it. Nodes cannot reach each other without infrastructure
|
||
mediating the connection.
|
||
|
||
FIPS inverts this model. There is no central authority, no address
|
||
assignment service, no routing table pushed from above. Each node generates
|
||
its own identity from a cryptographic keypair. Each node independently
|
||
decides which peers to connect to and which transports to use. From these
|
||
local decisions alone, the network self-organizes:
|
||
|
||
- A **spanning tree** forms through distributed parent selection, giving
|
||
every node a coordinate in the network without any node knowing the full
|
||
topology
|
||
- **Bloom filters** propagate through gossip, so each node learns which
|
||
peers can reach which destinations — again without global knowledge
|
||
- **Routing decisions** are made locally at each hop, using only the node's
|
||
immediate peers and cached coordinate information
|
||
|
||
Each peer link and end-to-end session actively measures RTT, loss, jitter,
|
||
and goodput through a lightweight in-band Metrics Measurement Protocol
|
||
(MMP), providing operator visibility and a foundation for quality-aware
|
||
routing.
|
||
|
||
The result is a network that builds itself from the bottom up, heals around
|
||
failures automatically, and scales without central coordination. Adding a
|
||
node is as simple as connecting to one existing peer — the network
|
||
integrates the new node through its normal mesh protocols.
|
||
|
||
## Specific Design Goals
|
||
|
||
- **Nostr-native identity and cryptography** — Use Nostr keypairs as node
|
||
identities and leverage secp256k1, Schnorr signatures, and SHA-256
|
||
- **Transport agnostic** — Support overlay, shared medium, and
|
||
point-to-point transports transparently
|
||
- **Self-organizing** — Automatic topology discovery and route optimization
|
||
- **Privacy preserving** — Minimize metadata leakage across untrusted links
|
||
- **Resilient** — Self-healing with graceful degradation
|
||
|
||
Non-goals include:
|
||
|
||
- **Reliable delivery** — FIPS provides a best-effort datagram service;
|
||
retransmission and ordering are left to applications or higher-layer
|
||
protocols
|
||
- **Anonymity** — Direct peers learn each other's identity; FIPS minimizes
|
||
metadata exposure but is not an anonymity network like Tor
|
||
- **Congestion control** — FIPS measures link quality but does not implement
|
||
flow control or congestion avoidance at the mesh layer
|
||
|
||
---
|
||
|
||
## Protocol Architecture
|
||
|
||
FIPS is organized in three protocol layers, each with distinct
|
||
responsibilities and clean service boundaries. No layer depends on the
|
||
specifics of the layers above or below it — transport plugins know nothing
|
||
about sessions, the routing layer knows nothing about application addressing,
|
||
and applications know nothing about which physical media carry their traffic.
|
||
This separation means new transports, protocol features, and application
|
||
interfaces can be added independently.
|
||
|
||

|
||
|
||
### Mapping to Traditional Networking
|
||
|
||
Readers familiar with the OSI model or TCP/IP networking may find it helpful
|
||
to see how FIPS concepts relate to traditional layers:
|
||
|
||

|
||
|
||
Note that FMP spans what would traditionally be separate link and network
|
||
layers. This is intentional — in a self-organizing mesh, the same layer that
|
||
authenticates peers also makes routing decisions, because routing depends on
|
||
authenticated peer state (spanning tree positions, bloom filters).
|
||
|
||
### Layer Responsibilities
|
||
|
||
**Transport layer**: Delivers datagrams between endpoints over a specific
|
||
medium. Each transport type (UDP socket, Ethernet interface, radio modem)
|
||
implements the same abstract interface: send and receive datagrams, report
|
||
MTU. The transport layer knows nothing about FIPS identities, routing, or
|
||
encryption. It provides raw datagram delivery to FMP above.
|
||
|
||
See [fips-transport-layer.md](fips-transport-layer.md) for the transport layer
|
||
specification.
|
||
|
||
**FIPS Mesh Protocol (FMP)**: Manages peer connections, authenticates peers
|
||
via Noise IK handshakes, and encrypts all traffic on each link. FMP is where
|
||
the mesh organizes itself — nodes exchange spanning tree announcements and
|
||
bloom filters with their direct peers, and FMP makes forwarding decisions
|
||
for transit traffic. FMP provides authenticated, encrypted forwarding to FSP
|
||
above.
|
||
|
||
See [fips-mesh-layer.md](fips-mesh-layer.md) for the FMP specification and
|
||
[fips-mesh-operation.md](fips-mesh-operation.md) for how FMP's routing and
|
||
self-organization work in practice.
|
||
|
||
**FIPS Session Protocol (FSP)**: Provides end-to-end authenticated
|
||
encryption between any two nodes, regardless of how many intermediate hops
|
||
separate them. FSP manages session lifecycle (setup, data transfer,
|
||
teardown), caches destination coordinates for efficient routing, and handles
|
||
the warmup strategy that keeps transit node caches populated. Session
|
||
dispatch uses index-based routing inspired by
|
||
[WireGuard](https://www.wireguard.com/), enabling O(1) packet
|
||
demultiplexing. FSP provides a datagram service to applications above.
|
||
|
||
See [fips-session-layer.md](fips-session-layer.md) for the FSP specification.
|
||
|
||
**IPv6 adaptation layer**: Sits above FSP as a service on port 256, adapting
|
||
the FIPS datagram service for unmodified IPv6 applications. Provides DNS
|
||
resolution (npub → fd00::/8 address), identity cache management, IPv6 header
|
||
compression, MTU enforcement, and a TUN interface. This is the primary way
|
||
existing applications use the FIPS mesh.
|
||
|
||
See [fips-ipv6-adapter.md](fips-ipv6-adapter.md) for the IPv6 adapter.
|
||
|
||
### Node Architecture
|
||
|
||
Application services sit at the top of the stack, dispatched by FSP port
|
||
number: the IPv6 TUN adapter (port 256) maps npubs to `fd00::/8` addresses
|
||
with header compression so unmodified IP applications can use the network
|
||
transparently, while the native datagram API addresses destinations directly
|
||
by npub.
|
||
|
||

|
||
|
||
The mesh routes application traffic across heterogeneous transports
|
||
transparently. A packet may traverse WiFi, Ethernet, UDP/IP, and Tor links
|
||
on its way from source to destination — the application never needs to know
|
||
which transports are involved. Each hop is independently encrypted at the
|
||
link layer, while a single end-to-end session protects the payload across
|
||
the entire path.
|
||
|
||

|
||
|
||
---
|
||
|
||
## Identity System
|
||
|
||
FIPS uses [Nostr](https://github.com/nostr-protocol/nips) keypairs
|
||
(secp256k1) as node identities. The public key identifies the node; the
|
||
private key signs protocol messages and establishes encrypted sessions.
|
||
|
||
The public key (or its bech32-encoded npub form) is the primary means for
|
||
application-layer software to identify communication endpoints. Internally,
|
||
the protocol derives a `node_addr` (a 16-byte SHA-256 hash of the pubkey)
|
||
used as the routing identifier in packet headers, and an IPv6 address derived
|
||
from the node_addr for the TUN adapter. Applications use the pubkey or npub;
|
||
the routing layer uses node_addr; unmodified IPv6 applications use the
|
||
derived `fd00::/8` address. All three are deterministically derived from the
|
||
same keypair.
|
||
|
||
### FIPS Identity Handling
|
||
|
||

|
||
|
||
The pubkey is the node's cryptographic identity, used in Noise IK handshakes
|
||
for both link and session encryption. It is never exposed beyond the
|
||
endpoints of an encrypted channel. The node_addr, a one-way SHA-256 hash
|
||
truncated to 16 bytes, serves as the routing identifier in packet headers
|
||
and bloom filters. Intermediate routers see only node_addrs — they can
|
||
forward traffic without learning the Nostr identities of the endpoints. An
|
||
observer can verify "does this node_addr belong to pubkey X?" if they already
|
||
know the pubkey, but cannot enumerate communicating identities by inspecting
|
||
traffic. The IPv6
|
||
address prepends `fd` to the first 15 bytes of the node_addr, providing a
|
||
ULA overlay address for unmodified IP applications via the TUN interface.
|
||
|
||
Below the FIPS identity layer, each transport uses its own native addressing
|
||
— IP:port or hostname:port addresses, MAC addresses, .onion identifiers. These **link
|
||
addresses** are opaque to everything above FMP and discarded once link
|
||
authentication completes.
|
||
|
||
### Identity Verification
|
||
|
||
The Noise Protocol Framework mutually authenticates both peer-to-peer link
|
||
connections (at FMP) and end-to-end session traffic (at FSP), proving each
|
||
party controls the private key for their claimed identity.
|
||
|
||
See [fips-mesh-layer.md](fips-mesh-layer.md) for peer authentication and
|
||
[fips-session-layer.md](fips-session-layer.md) for end-to-end session
|
||
establishment.
|
||
|
||
Key rotation changes the node's identity — a new keypair produces a new
|
||
node_addr and IPv6 address, requiring all sessions to be re-established.
|
||
Migration mechanisms that allow a node to announce a successor key are a
|
||
future consideration.
|
||
|
||
---
|
||
|
||
## Two-Layer Encryption
|
||
|
||
FIPS uses independent encryption at two protocol layers:
|
||
|
||
| Layer | Scope | Pattern | Purpose |
|
||
| ----- | ----- | ------- | ------- |
|
||
| **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link |
|
||
| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints |
|
||
|
||
### Link Layer (Hop-by-Hop)
|
||
|
||
When two nodes establish a direct connection, they perform a [Noise
|
||
IK](https://noiseprotocol.org/) handshake. This authenticates both parties
|
||
and establishes symmetric keys for encrypting all traffic on that link.
|
||
Every packet between direct peers is encrypted — gossip messages, routing
|
||
queries, and forwarded session datagrams alike.
|
||
|
||
The IK pattern is used because outbound connections know the peer's npub
|
||
from configuration, while inbound connections learn the initiator's identity
|
||
from the first handshake message.
|
||
|
||
### Session Layer (End-to-End)
|
||
|
||
FIPS establishes end-to-end encrypted sessions between any two communicating
|
||
nodes using Noise XK, regardless of how many hops separate them. The
|
||
initiator knows the destination's npub (required for XK's pre-message);
|
||
the responder learns the initiator's identity from the third handshake
|
||
message. Unlike the link-layer IK pattern where the initiator's identity
|
||
is revealed in msg1, XK delays identity disclosure until msg3, providing
|
||
stronger initiator identity protection for traffic traversing untrusted
|
||
intermediate nodes.
|
||
|
||
A packet from A to D through intermediate nodes B and C:
|
||
|
||
1. A encrypts payload with A↔D session key (FSP)
|
||
2. A wraps in SessionDatagram, encrypts with A↔B link key (FMP), sends to B
|
||
3. B decrypts link layer, reads destination node_addr, re-encrypts with B↔C
|
||
link key, forwards to C
|
||
4. C decrypts link layer, re-encrypts with C↔D link key, forwards to D
|
||
5. D decrypts link layer, then decrypts session layer to get payload
|
||
|
||
Intermediate nodes route based on destination node_addr but cannot read
|
||
session-layer payloads. Each hop strips one link encryption and applies the
|
||
next — the session-layer ciphertext passes through untouched.
|
||
|
||
Both layers always apply, even between adjacent peers — a packet to a direct
|
||
neighbor is still encrypted twice. This uniform model means no special cases
|
||
for local vs remote destinations, and topology changes (a direct peer
|
||
becomes reachable only through intermediaries) don't affect existing
|
||
sessions.
|
||
|
||
See [fips-mesh-layer.md](fips-mesh-layer.md) for link encryption and
|
||
[fips-session-layer.md](fips-session-layer.md) for session encryption.
|
||
|
||
---
|
||
|
||
## Routing and Mesh Operation
|
||
|
||
Each node makes forwarding decisions using only local information — its
|
||
immediate peers, their bloom filters, and cached coordinates — rather than
|
||
centrally distributed routing tables or global topology knowledge. Two
|
||
complementary mechanisms provide the information each node needs.
|
||
|
||
### Spanning Tree: The Coordinate System
|
||
|
||

|
||
|
||
Nodes self-organize into a spanning tree through gossip — each node
|
||
exchanges announcements with its direct peers and independently selects a
|
||
parent. Because every node applies the same rule (prefer the root with the
|
||
smallest node_addr), the network converges on a single agreed-upon root
|
||
without any voting or coordination. This is the same principle behind the
|
||
[Spanning Tree
|
||
Protocol](https://en.wikipedia.org/wiki/Spanning_Tree_Protocol) used in
|
||
Ethernet bridging since the 1980s: purely local decisions that converge to
|
||
consistent global state. The resulting tree gives every node a
|
||
**coordinate** — its path from itself to the root. Using tree coordinates
|
||
for routing is adapted from
|
||
[Yggdrasil](https://yggdrasil-network.github.io/)'s
|
||
[Ironwood](https://github.com/Arceliar/ironwood) routing library.
|
||
|
||
These coordinates enable distance calculations between any two nodes: the
|
||
distance is the number of hops from each node to their lowest common
|
||
ancestor in the tree. This provides a metric for routing decisions without
|
||
any node needing to know the full network topology.
|
||
|
||
The tree maintains itself through gossip — nodes exchange TreeAnnounce
|
||
messages with their peers, propagating parent selections and ancestry
|
||
chains. Changes cascade through the tree proportional to depth, not network
|
||
size. If the network partitions, each segment converges to its own new root
|
||
through the same process and reconverges automatically when segments rejoin.
|
||
|
||
See [fips-spanning-tree.md](fips-spanning-tree.md) for the tree algorithms
|
||
and [spanning-tree-dynamics.md](spanning-tree-dynamics.md) for detailed
|
||
convergence walkthroughs.
|
||
|
||
### Bloom Filters: Candidate Selection
|
||
|
||
The spanning tree provides a coordinate system for distance-based routing,
|
||
but on its own each node would only know about its immediate neighbors.
|
||
Bloom filters complement the tree by distributing reachability knowledge
|
||
across the entire mesh — each node learns which destinations are reachable
|
||
through which peers, without any node needing a complete view of the
|
||
network.
|
||
|
||
Each node's peer-advertised [bloom
|
||
filter](https://en.wikipedia.org/wiki/Bloom_filter) is a compact, fixed-size
|
||
data structure that answers one question: "can this peer possibly reach
|
||
destination D?" The answer is either "no" (definitive) or "maybe"
|
||
(probabilistic — false positives are possible). Because the filter size is
|
||
constant regardless of how many destinations it represents, bloom filters
|
||
scale efficiently as the network grows. This is candidate selection for
|
||
routing — bloom filters narrow the set of peers worth considering, and the
|
||
actual forwarding decision ranks those candidates by tree distance and link
|
||
quality.
|
||
|
||
Filters propagate transitively through tree edges, with each node computing
|
||
outbound filters by merging the filters received from its tree peers (parent
|
||
and children) using a
|
||
[split-horizon](https://en.wikipedia.org/wiki/Split_horizon_route_advertisement)
|
||
technique borrowed from distance-vector routing. All peers — including
|
||
non-tree mesh shortcuts — receive FilterAnnounce messages, but only tree
|
||
peers' filters are merged into outgoing computation. This prevents filter
|
||
saturation where mesh shortcuts would cause every filter to converge toward
|
||
the full network.
|
||
|
||
See [fips-bloom-filters.md](fips-bloom-filters.md) for filter parameters and
|
||
mathematical properties.
|
||
|
||

|
||
|
||
The outbound filter for peer Q merges this node's identity with tree peer
|
||
inbound filters except Q's (split-horizon exclusion). This creates
|
||
directional asymmetry: upward filters (child → parent) contain the child's
|
||
subtree, while downward filters (parent → child) contain the complement.
|
||
Mesh peers receive filters but their inbound filters are not merged
|
||
transitively — they provide single-hop shortcut visibility only.
|
||
|
||
A node with multiple peers receives genuinely different filters from each.
|
||
In the diagram, R receives {B, D, E} from B and {C, F} from C — two disjoint
|
||
subtrees. When R needs to reach F, only C's filter matches. This is where
|
||
bloom filters provide real candidate selection: a node with several peers
|
||
can narrow the forwarding choice before consulting tree coordinates. Leaf
|
||
nodes like D have only one peer, so their single inbound filter is
|
||
necessarily near-complete (everything except themselves) and offers no
|
||
selection — but leaf nodes have no choice to make anyway.
|
||
|
||
Bloom filter sizing (bit count and hash functions) requires further analysis
|
||
based on actual deployment scenarios. The FMP wire format is versioned to
|
||
accommodate future parameter changes as operational experience accumulates.
|
||
|
||
### Routing Decisions
|
||
|
||
At each hop, FMP makes a local forwarding decision using the following
|
||
priority chain:
|
||
|
||
1. **Local delivery** — the destination is this node
|
||
2. **Direct peer** — the destination is an authenticated neighbor
|
||
3. **Bloom-guided candidate selection** — bloom filters identify peers that
|
||
can reach the destination; tree coordinates rank them by distance and
|
||
link quality
|
||
4. **[Greedy routing](https://en.wikipedia.org/wiki/Greedy_embedding)** —
|
||
fallback when bloom filters haven't converged; forward to the peer that
|
||
minimizes tree distance to the destination
|
||
5. **No route** — destination unreachable; send error signal to source
|
||
|
||
All multi-hop routing depends on knowing the destination's tree coordinates.
|
||
These are cached at each node after being learned through discovery
|
||
(LookupRequest/LookupResponse) or session establishment (SessionSetup). The
|
||
coordinate cache is the critical piece that enables efficient forwarding.
|
||
|
||

|
||
|
||
### Coordinate Caching and Discovery
|
||
|
||
When a node first needs to reach an unknown destination, it sends a
|
||
LookupRequest that propagates through the network guided by bloom filters
|
||
and loop prevention. The destination responds with its coordinates, which
|
||
the source and intermediate nodes along the return path cache. Subsequent
|
||
traffic routes efficiently using the cached coordinates.
|
||
|
||
Session establishment (SessionSetup) also carries coordinates, warming
|
||
transit node caches along the path so that data packets can be forwarded
|
||
without individual discovery at each hop.
|
||
|
||

|
||
|
||
### Error Recovery
|
||
|
||
When routing fails — because cached coordinates are stale, a path has
|
||
broken, or a packet exceeds a link's MTU — transit nodes signal the source:
|
||
|
||
- **CoordsRequired**: A transit node lacks the destination's coordinates.
|
||
The source re-initiates discovery and resets its coordinate warmup
|
||
strategy.
|
||
- **PathBroken**: Greedy routing reached a dead end. The source re-discovers
|
||
the destination's current coordinates.
|
||
- **MtuExceeded**: A transit node cannot forward a packet because it exceeds
|
||
the next-hop link MTU. The source adjusts its path MTU estimate.
|
||
|
||
All three signals trigger active recovery, and are rate-limited to prevent
|
||
storms during topology changes.
|
||
|
||
See [fips-mesh-operation.md](fips-mesh-operation.md) for the complete
|
||
routing and mesh behavior description.
|
||
|
||
### Metrics Measurement Protocol (MMP)
|
||
|
||
Each peer link runs an instance of the Metrics Measurement Protocol, which
|
||
measures link quality through in-band report exchange. MMP computes smoothed
|
||
round-trip time (SRTT), packet loss rate, interarrival jitter, goodput, and
|
||
one-way delay trend — all derived from counter and timestamp fields already
|
||
present in the FMP wire format, with no additional probing traffic required.
|
||
|
||
MMP operates in three modes. **Full** mode exchanges both SenderReports and
|
||
ReceiverReports to compute all metrics including RTT. **Lightweight** mode
|
||
exchanges only ReceiverReports, providing loss and jitter but not RTT — useful
|
||
for constrained links. **Minimal** mode disables reports entirely, relying
|
||
only on spin bit and congestion echo flags in the frame header.
|
||
|
||
Reports are sent at RTT-adaptive intervals (clamped to 100 ms–2 s), so
|
||
high-latency links don't generate excessive measurement traffic while
|
||
low-latency links converge quickly. Each metric carries both short-term and
|
||
long-term exponentially weighted moving averages, enabling detection of
|
||
quality changes against a stable baseline.
|
||
|
||
MMP serves dual roles: operator visibility and cost-based parent selection.
|
||
Periodic log lines report per-link RTT, loss, jitter, and goodput. MMP
|
||
computes an Expected Transmission Count (ETX) from bidirectional delivery
|
||
ratios, which feeds into cost-based parent selection where each node
|
||
evaluates `effective_depth = depth + link_cost` using
|
||
`link_cost = etx * (1.0 + srtt_ms / 100.0)`. ETX is not yet used in
|
||
`find_next_hop()` candidate ranking for data forwarding.
|
||
|
||
See [fips-mesh-layer.md](fips-mesh-layer.md) for MMP operating modes, report
|
||
scheduling, and the spin bit design.
|
||
|
||
---
|
||
|
||
## Transport Abstraction
|
||
|
||
FIPS treats the communication medium as a pluggable component. Every transport
|
||
— whether a UDP socket, an Ethernet interface, a Tor circuit, or a radio modem
|
||
— implements the same simple interface: send a datagram to an address, receive
|
||
datagrams, and report the link MTU. The rest of the protocol stack sees no
|
||
difference between them.
|
||
|
||
A **transport** is a driver for a particular medium. A **link** is a peer
|
||
connection established over a transport. Transport addresses (IP:port, MAC
|
||
address, .onion) are opaque to all layers above FMP — they exist only to
|
||
deliver datagrams and are discarded once FMP has authenticated the peer via
|
||
the Noise IK handshake. From that point on, the peer is identified solely by
|
||
its cryptographic identity.
|
||
|
||
Transports fall into three categories based on their connectivity model:
|
||
|
||
| Category | Examples | Characteristics |
|
||
| -------- | -------- | --------------- |
|
||
| Overlay | UDP/IP, Tor | Tunnels FIPS over existing networks |
|
||
| Shared medium | Ethernet, WiFi, Bluetooth, Radio | Local broadcast, peer discovery |
|
||
| Point-to-point | Serial, dialup | Fixed connections, no discovery |
|
||
|
||
These categories differ in addressing, MTU, reliability, and whether they
|
||
support local discovery, but FMP handles all of them uniformly. A node
|
||
running multiple transports simultaneously bridges between those networks
|
||
automatically — peers from all transports feed into a single spanning tree,
|
||
and the router selects the best path regardless of which medium carries it.
|
||
If one transport fails, traffic reroutes through alternatives without
|
||
application involvement.
|
||
|
||
Some transports support an optional discovery capability — the ability to
|
||
broadcast and listen for announcements indicating the availability of FIPS
|
||
endpoints on the local medium. Shared media like Ethernet, WiFi, Bluetooth,
|
||
and radio are natural fits for this, as they can reach nearby devices without
|
||
prior configuration. When discovery is available, nodes can automatically
|
||
find and peer with other FIPS nodes on the same medium. Transports that
|
||
lack discovery (such as configured UDP endpoints) simply skip this step and
|
||
connect directly to configured addresses. For internet-reachable nodes,
|
||
endpoint discovery via signed Nostr events allows nodes to publish and
|
||
consume transport addresses through public relays — available behind the
|
||
`nostr-discovery` cargo feature.
|
||
|
||
NAT traversal for internet-connected nodes is supported via STUN-assisted
|
||
UDP hole punching, with offer/answer signaling carried over Nostr gift-wrap
|
||
events. Once a direct UDP path is established, the punched socket is handed
|
||
into the standard FIPS transport/session stack. Nodes that cannot establish
|
||
a direct path remain reachable through port forwarding, a publicly addressed
|
||
peer, or relay through other mesh nodes. The Nostr-mediated discovery and
|
||
NAT traversal paths are gated by the `nostr-discovery` cargo feature and
|
||
configured under `node.discovery.nostr.*`.
|
||
|
||
> **Implementation status**: UDP/IP, TCP/IP, Ethernet, Tor
|
||
> (SOCKS5 outbound + directory-mode inbound via onion service),
|
||
> and Bluetooth (BLE L2CAP CoC) transports are implemented.
|
||
> All others are future directions.
|
||
|
||
See [fips-transport-layer.md](fips-transport-layer.md) for the full transport
|
||
layer specification.
|
||
|
||
---
|
||
|
||
## Security
|
||
|
||
FIPS is designed around four classes of adversary, each addressed by a
|
||
different layer of the protocol.
|
||
|
||
### Transport Observers
|
||
|
||
A passive observer on the underlying transport — someone monitoring a WiFi
|
||
network, tapping an Ethernet segment, or inspecting UDP traffic — sees only
|
||
encrypted packets. The FMP link-layer Noise IK session encrypts all traffic
|
||
between direct peers, including routing gossip and forwarded session
|
||
datagrams. The observer can infer timing, packet sizes, and which transport
|
||
endpoints are exchanging traffic, but cannot read content or determine
|
||
FIPS-level node identities from the encrypted packets. Traffic analysis —
|
||
correlating timing and volume patterns across multiple vantage points to
|
||
infer communication relationships — is not defended against (see
|
||
[Specific Design Goals](#specific-design-goals)).
|
||
|
||
### Active Attackers on the Transport
|
||
|
||
An adversary who can inject, modify, drop, or replay packets on the
|
||
transport is also defeated by the FMP link-layer Noise IK session. Mutual
|
||
authentication prevents impersonation, AEAD encryption detects tampering,
|
||
and counter-based nonces with a sliding replay window reject replayed
|
||
packets.
|
||
|
||
### Other FIPS Nodes (Intermediate Routers)
|
||
|
||
The most important adversary class is the operators of other nodes in the
|
||
mesh — the peers that forward your traffic. FIPS treats every intermediate
|
||
router as potentially adversarial. The FSP session layer establishes a
|
||
completely independent Noise XK session between the communicating endpoints,
|
||
so intermediate nodes cannot read application payloads even though they
|
||
decrypt and re-encrypt the link-layer envelope at each hop.
|
||
|
||
Routing headers expose only the destination's node_addr — an opaque
|
||
SHA-256 hash of the actual public key. Intermediate routers can forward
|
||
traffic without learning which Nostr identities are communicating. An
|
||
observer can verify "does this node_addr belong to pubkey X?" if they
|
||
already know the pubkey, but cannot enumerate communicating identities by
|
||
inspecting routed traffic.
|
||
|
||
| Entity | Can See |
|
||
| ------ | ------- |
|
||
| Transport observer | Encrypted packets, timing, packet sizes |
|
||
| Direct peer | Your npub, traffic volume, timing |
|
||
| Intermediate router | Source and destination node_addrs, packet size |
|
||
| Destination | Your npub, payload content |
|
||
|
||
### Adversarial Nodes Disrupting the Mesh
|
||
|
||
Beyond passive observation, a malicious node could attempt to disrupt
|
||
routing by injecting false spanning tree announcements, advertising bogus
|
||
bloom filters, or claiming invalid tree positions. FMP mitigates these
|
||
through signed TreeAnnounce messages verified by direct peers, transitive
|
||
ancestry chain validation, replay protection via sequence numbers, and
|
||
discretionary peering — node operators choose who to peer with, so an
|
||
attacker with many identities still needs real nodes to accept their
|
||
connections. Handshake rate limiting further constrains how fast an attacker
|
||
can establish new links. In fully open networks with automatic peer
|
||
discovery, Sybil resistance relies primarily on rate limiting; discretionary
|
||
peering provides stronger resistance in curated deployments where operators
|
||
vet their peers. An attacker who controls all of a target node's direct
|
||
peers can completely control its view of the network (an eclipse attack);
|
||
diverse peering across independent operators and transports is the primary
|
||
mitigation.
|
||
|
||
---
|
||
|
||
## Prior Work
|
||
|
||
FIPS builds on proven designs rather than inventing new cryptography or routing
|
||
algorithms. Nearly every major design decision has deployed precedent.
|
||
|
||
### Spanning Tree Self-Organization
|
||
|
||
The idea that distributed nodes can build a spanning tree through purely local
|
||
decisions — each node selecting a parent based on announcements from its
|
||
neighbors — dates to the
|
||
[IEEE 802.1D Spanning Tree Protocol](https://en.wikipedia.org/wiki/Spanning_Tree_Protocol)
|
||
(STP, 1985). STP demonstrated that a network-wide tree emerges from a simple
|
||
deterministic rule (lowest bridge ID wins root election) applied independently
|
||
at each node. FIPS uses the same principle — lowest node address determines the
|
||
root — adapted from an Ethernet bridging context to a general-purpose overlay
|
||
mesh.
|
||
|
||
### Tree Coordinate Routing
|
||
|
||
The spanning tree coordinates, bloom filter candidate selection, and greedy
|
||
routing algorithms are adapted from
|
||
[Yggdrasil v0.5](https://yggdrasil-network.github.io/2023/10/22/upcoming-v05-release.html)
|
||
and its [Ironwood](https://github.com/Arceliar/ironwood) routing library.
|
||
Yggdrasil's key insight was using the tree path from root to node as a
|
||
routable coordinate, enabling greedy forwarding without global routing tables.
|
||
FIPS adapts these algorithms for multi-transport operation, Nostr identity
|
||
integration, and constrained MTU environments.
|
||
|
||
The theoretical foundation for greedy routing on tree embeddings draws on
|
||
[Kleinberg's work](https://www.cs.cornell.edu/home/kleinber/swn.pdf) on
|
||
navigable small-world networks, which showed that greedy forwarding succeeds
|
||
in O(log² n) steps when the network has hierarchical structure. Thorup-Zwick
|
||
compact routing schemes separately demonstrated that sublinear routing state
|
||
is achievable with bounded stretch, motivating the use of tree coordinates
|
||
rather than full routing tables.
|
||
|
||
### Split-Horizon Bloom Filter Propagation
|
||
|
||
FIPS distributes reachability information using bloom filters computed with a
|
||
split-horizon rule: when advertising to a peer, exclude that peer's own
|
||
contributions. This technique is borrowed from distance-vector routing
|
||
protocols — [RIP](https://en.wikipedia.org/wiki/Routing_Information_Protocol)
|
||
(1988) and [Babel](https://www.irif.fr/~jch/software/babel/) use split-horizon
|
||
to prevent routing loops by not advertising a route back to the neighbor it was
|
||
learned from. FIPS applies the same principle to probabilistic set
|
||
advertisements rather than distance-vector tables.
|
||
|
||
### Cryptographic Identity as Network Address
|
||
|
||
FIPS nodes are identified by their Nostr public keys (secp256k1). The network
|
||
address *is* the cryptographic identity — there is no separate address
|
||
assignment or registration step.
|
||
[CJDNS](https://github.com/cjdelisle/cjdns) pioneered this approach in
|
||
overlay meshes, deriving IPv6 addresses from the double-SHA-512 of each node's
|
||
public key. Tor [.onion addresses](https://spec.torproject.org/rend-spec-v3)
|
||
and the IETF
|
||
[Host Identity Protocol](https://en.wikipedia.org/wiki/Host_Identity_Protocol)
|
||
(HIP) follow the same principle. FIPS uses Nostr's existing key infrastructure
|
||
rather than introducing a new identity scheme.
|
||
|
||
### Dual-Layer Encryption
|
||
|
||
FIPS encrypts traffic twice: FMP provides hop-by-hop link encryption
|
||
(protecting against transport-layer observers), while FSP provides independent
|
||
end-to-end session encryption (protecting against intermediate FIPS nodes).
|
||
This layered approach mirrors [Tor](https://www.torproject.org/), where each
|
||
relay peels one layer of encryption (hop-by-hop) while the innermost layer
|
||
protects end-to-end payload. [I2P](https://geti2p.net/) uses a similar
|
||
garlic routing scheme with tunnel-layer and end-to-end encryption. Unlike Tor
|
||
and I2P, FIPS does not provide anonymity — its dual encryption protects
|
||
confidentiality and integrity rather than hiding traffic patterns.
|
||
|
||
### Noise Protocol Framework
|
||
|
||
FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/) at both
|
||
protocol layers, with different handshake patterns chosen for each layer's
|
||
threat model. FMP link encryption uses **Noise IK**, providing mutual
|
||
authentication with a single round trip where the initiator knows the
|
||
responder's static key in advance.
|
||
[WireGuard](https://www.wireguard.com/) uses the same IK base pattern
|
||
(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP session
|
||
encryption uses **Noise XK**, the same pattern used by the
|
||
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md),
|
||
where the initiator's static key is transmitted in a third message rather
|
||
than the first. XK provides stronger initiator identity hiding at the cost
|
||
of an additional round trip — a worthwhile tradeoff for session-layer traffic
|
||
that traverses untrusted intermediate nodes. At the link layer, where both
|
||
peers are configured and directly connected, IK's single round trip is
|
||
preferred.
|
||
|
||
### Index-Based Session Dispatch
|
||
|
||
FIPS uses locally-assigned 32-bit session indices to demultiplex incoming
|
||
packets to the correct cryptographic session in O(1) time, without parsing
|
||
source addresses or performing expensive lookups. This directly follows
|
||
[WireGuard's](https://www.wireguard.com/papers/wireguard.pdf) receiver index
|
||
approach, where each peer assigns a random index during handshake and the
|
||
remote side includes it in every packet header.
|
||
|
||
### Transport-Agnostic Overlay Mesh
|
||
|
||
FIPS is designed to operate over any datagram-capable transport — UDP, raw
|
||
Ethernet, Bluetooth, radio, serial — through a uniform transport abstraction.
|
||
Several mesh overlays have demonstrated transport-agnostic design:
|
||
[CJDNS](https://github.com/cjdelisle/cjdns) runs over UDP and Ethernet,
|
||
[Yggdrasil](https://yggdrasil-network.github.io/) supports TCP and TLS
|
||
transports, and [Tor](https://www.torproject.org/) can use pluggable
|
||
transports to tunnel through various media. FIPS extends this pattern to
|
||
shared-medium transports (radio, BLE) with per-transport MTU and discovery
|
||
capabilities.
|
||
|
||
### Metrics Measurement Protocol
|
||
|
||
MMP's design assembles well-established measurement techniques into a unified
|
||
per-link protocol. The SenderReport/ReceiverReport exchange structure follows
|
||
[RTCP](https://www.rfc-editor.org/rfc/rfc3550) (RFC 3550), which uses the
|
||
same report pairing for media stream quality monitoring in RTP sessions. MMP's
|
||
jitter computation uses the RTCP interarrival jitter algorithm directly.
|
||
|
||
The smoothed RTT estimator uses the Jacobson/Karels algorithm
|
||
([RFC 6298](https://www.rfc-editor.org/rfc/rfc6298)), the same SRTT
|
||
computation used in TCP for retransmission timeout calculation since 1988.
|
||
MMP derives RTT from timestamp-echo in ReceiverReports with dwell-time
|
||
compensation, rather than from packet round-trips.
|
||
|
||
The spin bit in the FMP frame header follows the
|
||
[QUIC](https://www.rfc-editor.org/rfc/rfc9000) spin bit
|
||
([RFC 9312](https://www.rfc-editor.org/rfc/rfc9312)) — a single bit that
|
||
alternates each round trip, enabling passive latency measurement. FIPS
|
||
implements the spin bit state machine but relies on timestamp-echo for SRTT,
|
||
as irregular mesh traffic makes spin bit RTT unreliable.
|
||
|
||
The Expected Transmission Count (ETX) metric, computed from bidirectional
|
||
delivery ratios, was introduced by
|
||
[De Couto et al. (2003)](https://pdos.csail.mit.edu/papers/grid:mobicom03/paper.pdf)
|
||
for wireless mesh routing and is used in protocols including
|
||
[OLSR](https://en.wikipedia.org/wiki/Optimized_Link_State_Routing_Protocol)
|
||
and [Babel](https://www.irif.fr/~jch/software/babel/). FIPS computes ETX
|
||
per-link from MMP loss measurements for future use in candidate ranking.
|
||
|
||
The CE (Congestion Experienced) echo flag provides hop-by-hop
|
||
[ECN](https://en.wikipedia.org/wiki/Explicit_Congestion_Notification)
|
||
signaling, following the TCP/IP ECN echo pattern (RFC 3168). Transit nodes
|
||
detect congestion via MMP loss/ETX metrics or kernel buffer drops and set
|
||
the CE flag on forwarded frames; destination nodes mark ECN-capable IPv6
|
||
packets accordingly.
|
||
|
||
### Cryptographic Primitives
|
||
|
||
FIPS reuses [Nostr's](https://github.com/nostr-protocol/nips) cryptographic
|
||
stack — secp256k1 for identity keys, Schnorr signatures for authentication,
|
||
SHA-256 for hashing, and ChaCha20-Poly1305 for authenticated encryption. This
|
||
is the same primitive set used across Bitcoin, Nostr, and a growing ecosystem
|
||
of self-sovereign identity systems. No novel cryptography is introduced.
|
||
|
||
---
|
||
|
||
## Further Reading
|
||
|
||
### Protocol Layers
|
||
|
||
| Document | Description |
|
||
| -------- | ----------- |
|
||
| [fips-transport-layer.md](fips-transport-layer.md) | Transport layer: abstraction, types, services provided to FMP |
|
||
| [fips-mesh-layer.md](fips-mesh-layer.md) | FMP: peer authentication, link encryption, forwarding |
|
||
| [fips-session-layer.md](fips-session-layer.md) | FSP: end-to-end encryption, session lifecycle |
|
||
| [fips-ipv6-adapter.md](fips-ipv6-adapter.md) | IPv6 adaptation: DNS, TUN interface, MTU enforcement |
|
||
|
||
### Mesh Behavior and Wire Formats
|
||
|
||
| Document | Description |
|
||
| -------- | ----------- |
|
||
| [fips-mesh-operation.md](fips-mesh-operation.md) | How the mesh operates: routing, discovery, error recovery |
|
||
| [fips-wire-formats.md](fips-wire-formats.md) | Complete wire format reference for all protocol layers |
|
||
|
||
### Supporting References
|
||
|
||
| Document | Description |
|
||
| -------- | ----------- |
|
||
| [fips-spanning-tree.md](fips-spanning-tree.md) | Spanning tree algorithms and data structures |
|
||
| [fips-bloom-filters.md](fips-bloom-filters.md) | Bloom filter parameters, math, and computation |
|
||
| [spanning-tree-dynamics.md](spanning-tree-dynamics.md) | Scenario walkthroughs: convergence, partitions, recovery |
|
||
|
||
### Implementation
|
||
|
||
| Document | Description |
|
||
| -------- | ----------- |
|
||
| [fips-configuration.md](fips-configuration.md) | YAML configuration reference |
|
||
|
||
### External References
|
||
|
||
- [IEEE 802.1D Spanning Tree Protocol](https://en.wikipedia.org/wiki/Spanning_Tree_Protocol)
|
||
- [Yggdrasil Network](https://yggdrasil-network.github.io/)
|
||
- [Yggdrasil v0.5 Release Notes](https://yggdrasil-network.github.io/2023/10/22/upcoming-v05-release.html)
|
||
- [Ironwood Routing Library](https://github.com/Arceliar/ironwood)
|
||
- [Kleinberg — The Small-World Phenomenon](https://www.cs.cornell.edu/home/kleinber/swn.pdf)
|
||
- [CJDNS](https://github.com/cjdelisle/cjdns)
|
||
- [Tor Project](https://www.torproject.org/)
|
||
- [I2P](https://geti2p.net/)
|
||
- [Host Identity Protocol (HIP)](https://en.wikipedia.org/wiki/Host_Identity_Protocol)
|
||
- [Babel Routing Protocol](https://www.irif.fr/~jch/software/babel/)
|
||
- [Noise Protocol Framework](https://noiseprotocol.org/)
|
||
- [WireGuard](https://www.wireguard.com/)
|
||
- [WireGuard Whitepaper](https://www.wireguard.com/papers/wireguard.pdf)
|
||
- [Lightning Network BOLT #8 — Transport](https://github.com/lightning/bolts/blob/master/08-transport.md)
|
||
- [QUIC (RFC 9000)](https://www.rfc-editor.org/rfc/rfc9000)
|
||
- [QUIC Spin Bit (RFC 9312)](https://www.rfc-editor.org/rfc/rfc9312)
|
||
- [RTCP (RFC 3550)](https://www.rfc-editor.org/rfc/rfc3550)
|
||
- [TCP SRTT / RTO (RFC 6298)](https://www.rfc-editor.org/rfc/rfc6298)
|
||
- [ECN (RFC 3168)](https://www.rfc-editor.org/rfc/rfc3168)
|
||
- [ETX — De Couto et al. 2003](https://pdos.csail.mit.edu/papers/grid:mobicom03/paper.pdf)
|
||
- [OLSR](https://en.wikipedia.org/wiki/Optimized_Link_State_Routing_Protocol)
|
||
- [Nostr Protocol](https://github.com/nostr-protocol/nips)
|