Files
fips/docs/design/fips-intro.md
T
34e00b9f6e Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.

The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.

== Cargo feature and dependencies

- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
  Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
  nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
  `src/discovery/nostr/signal.rs` rather than relying on the SDK's
  rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
  exclusively.

== Wire format

Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.

Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.

Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.

== Discovery surface

- `src/discovery.rs` (always compiled)
  - `EstablishedTraversal`: bound UDP socket + selected remote +
    peer npub + optional transport name/config tuning overrides.
  - `BootstrapHandoffResult`: returned on successful handoff —
    allocated transport id, local/remote addrs, peer NodeAddr,
    session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
  - `types.rs`: wire and control types described above. `ADVERT_KIND`
    constant. `BootstrapError` enumerates failure modes (disabled,
    missing advert, missing NAT endpoint, no usable relays, invalid
    advert, invalid npub, signal timeout, punch timeout, replay,
    STUN failure, protocol, nostr, io, serde, event-parse).
  - `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
    nostr-sdk `Client`, subscribes to advert + signal event kinds,
    maintains a bounded advert cache and a bounded seen-sessions
    replay set, drains `BootstrapEvent::{Established, Failed}` for
    the node to consume, exposes `update_local_advert`,
    `request_connect`, `advert_endpoints_for_peer`,
    `cached_open_discovery_candidates`, and `shutdown`.
  - `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
    built against per-attempt ephemeral keys; inbound events are
    unwrapped against the node identity.
  - `stun.rs`: RFC 5389/8489 Binding Request client with
    XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
    observe the initiator's own reflexive address against its
    locally configured STUN list (peer-advertised STUN is
    informational, never an egress target).
  - `traversal.rs`: per-attempt candidate-pair punch planner.
    Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
    LAN-private and ULA interface addresses alongside the STUN
    reflexive address, schedules probe/ack exchanges at the
    configured interval for the configured duration, and picks the
    first candidate pair that authenticates end-to-end.

Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.

== Configuration surface

`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:

- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
  `wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
  for both relay lists, and Google / Cloudflare / Twilio for STUN.
  Operators are expected to override for production. Other
  verified-working public relays for reference:
  `nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
  `nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
  Open}` — controls whether advert-derived endpoints are consumed
  only for peers carrying `via_nostr = true`, or also for
  non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
  `local_addresses` list is empty and peers see only the reflexive
  address. Enable per-node only for genuinely same-LAN deployments;
  off-by-default eliminates the misleading-path failure mode for
  the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
  retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
  spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
  (2048) — bound memory under ambient relay volume; overflow
  evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
  `punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)

Per-peer and per-transport flags:

- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
  advert-derived addresses are appended as fallback dial candidates
  after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
  when `via_nostr: true`; validation requires at least one of the
  two to be present per peer, and the error message names the
  peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
  `UdpConfig.public: Option<bool>` — UDP transports can be
  advertised either as direct `host:port` (public = true) or as the
  `addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
  — TCP and Tor onion endpoints can be advertised as directly
  reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
  special-casing in YAML and routes through the bootstrap runtime.

Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):

- Any transport with `advertise_on_nostr = true` requires
  `node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
  `node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
  `public = false` — i.e. `udp:nat`) additionally requires at least
  one `dm_relay` and at least one `stun_server`.
  Surfaced as `ConfigError::Validation`.

== Node integration

`src/node/lifecycle.rs` is the main integration point.

- At node start (after transports are up, before TUN), if Nostr is
  enabled and the feature is compiled in, `NostrDiscovery::start` is
  invoked, the initial local overlay advert is built from the live
  transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
  at method definition and call site), which refreshes the local
  advert, drains bootstrap events, adopts established traversals,
  schedules retries for failed traversals, and — under `policy:
  open` — enqueues outbound retries for non-configured peers
  visible in the advert cache, bounded by
  `open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
  first exhausts the static address list in priority order and only
  then appends advert-derived fallback addresses; both lists run
  through the same `attempt_peer_address_list` code path. The
  `udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
  for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
  per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
  optional transport instance name), and emits an `OverlayAdvert`
  including `signalRelays` and `stunServers` when any UDP endpoint
  is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
  allocates a new `TransportId`, constructs a `UdpTransport` with
  the user-supplied (or default) `UdpConfig`, calls the new
  `adopt_socket_async` to reuse the punched socket verbatim,
  registers the transport in the normal transport map, records it
  in `bootstrap_transports`, and calls `initiate_connection` so the
  normal handshake path runs. On failure, the transport is stopped
  and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
  background tasks stop before transports are torn down. (The
  advert is not explicitly retracted; NIP-40 expiration plus the
  next refresh from any live publisher supersedes it.)

New `Node` fields:

- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
  transports adopted from NAT traversal, cleaned up via
  `cleanup_bootstrap_transport_if_unused` whenever the link,
  connection, peer, or pending-connect referencing them is removed.

Retry and error surface:

- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
  for a retry entry. `pump_retries` drops expired entries with an
  info log. Used for open-discovery retries, which expire at two
  times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
  `adopt_established_traversal` when the underlying transport
  adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
  lifecycle, rx-loop tick, and timeout bookkeeping.

== UDP transport

`src/transport/udp/`:

- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
  adopts an externally bound socket, makes it non-blocking, applies
  the configured buffer sizes (warning if the kernel clamps), and
  reports the resulting local address. Preserves the NAT mapping —
  no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
  `start_async` analogue for an already-bound socket, wiring the
  async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
  still holding a recv task or socket (for example on error
  teardown), aborts the task, clears the socket, and emits a debug
  log so the cleanup is visible in tracing rather than silent.

== Logging and observability

Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.

Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:

- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`

Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.

Other operator-facing logs:

- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
  successful return, tagged with peer npub, session id, transport
  id, and both socket endpoints, so the bootstrap handoff is
  traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
  reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
  downstream STUN, punch, and handshake logs for the same peer
  correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
  mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
  (hot path: fires for every unrelated signal event on the same
  relay).
- Traversal-offer handler failures log at debug. Expected conditions
  such as punch timeout on symmetric NAT are covered there; real
  problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
  (`max_concurrent_incoming_offers`) and state that the offer was
  rate-limited rather than failing.

== Tests

- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
  encoding, signal envelope round-trip, STUN parsing, punch-packet
  codec, and replay-window enforcement. Run under the
  `nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
  cross-field invariants and YAML parsing of the full
  `node.discovery.nostr` block plus `peers[].via_nostr`, empty
  `addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
  synthetic traversal (bound UDP socket pair + synthetic peer
  identity) through `adopt_established_traversal` and assert the
  Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
  same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
  - Local `strfry` relay, local STUN responder, and one or two
    router containers performing `iptables` NAT.
  - Node LAN interfaces are provisioned with explicit `veth` pairs
    injected into the node and router namespaces so every packet
    traverses the router namespace (plain Docker bridges are not
    used for the LAN).
  - `cone` scenario: both peers behind full-cone-emulation NAT
    (SNAT with source-port preservation, inbound DNAT back to the
    single LAN host regardless of remote source); asserts UDP
    traversal succeeds and link remote addresses are on the router
    WAN subnet.
  - `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
    traversal fails and TCP fallback converges over router-
    published WAN addresses.
  - `lan` scenario: both peers share a LAN subnet; asserts LAN
    addresses are preferred over reflexive ones.
  - Cleanup tears down all profile-gated services
    (`--profile cone --profile symmetric --profile lan`) so no
    orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
  `--features "tui nostr-discovery"` by default so NAT-harness
  binaries include bootstrap support.

== CI

- Linux release build and nextest unit-test job both use
  `--features "gateway nostr-discovery"` so the feature-gated code
  and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
  `nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
  `docker compose logs` on failure, and always stop containers.

== Packaging and operations

- `packaging/common/fips.yaml` ships a fully commented
  `node.discovery.nostr.*` block, plus documented
  `advertise_on_nostr` / `public` examples under the UDP transport,
  an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
  example under the static peer section with both a direct
  `host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
  publishes target the new default relay set.

== Documentation

- `README.md`: overlay discovery + NAT traversal moved from
  "Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
  previously described Nostr discovery and NAT traversal as future
  work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
  direction)" qualifier from the Nostr Relay Discovery section,
  expands with the `udp:nat` advertisement and bootstrap handoff
  description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
  rebinding (roaming) and initial NAT traversal (Nostr path) are
  distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
  `node.discovery.nostr.*` surface, including the three resource
  caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
  reference for the shipped mechanism, including the empty-
  `addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
  Implemented status callout, clarifies that the punch socket is
  per-peer and per-attempt rather than shared with the application
  listener, aligns field names with the shipped JSON
  (`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
  `localAddresses`, `stunServer`), sets the `d`-tag to
  `fips-overlay-v1`, names the kind as 37195, and notes that
  advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
  hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
  path, STUN/punch path, configuration surface, and Docker NAT lab.

Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
2026-04-27 08:15:58 -07:00

823 lines
40 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# FIPS: Free Internetworking Peering System
## What is FIPS?
FIPS is a self-organizing mesh network that can operate natively over a
variety of physical and logical media, such as local area networks,
Bluetooth, serial links, or the existing internet as an overlay. The
long-term goal is infrastructure that can function alongside or ultimately
replace dependence on the Internet itself. Systems running FIPS establish
peer connections, authenticate each other, and route traffic for each other
without any central authority or global topology knowledge, and allow
end-to-end encrypted sessions between any two nodes regardless of how many
hops separate them.
Nodes in the mesh route traffic for each other using Nostr identities
(npubs) as network addresses. Applications can access the mesh through a
native FIPS datagram service, or through an IPv6 adaptation layer that
presents each node as an IPv6 endpoint for compatibility with existing
IP-based applications.
## Why FIPS?
**Self-sovereign identity**: FIPS nodes generate their own addresses, node
IDs, and security credentials without coordination with any central
authority. These identities can be long-term fixed or may be ephemeral,
changed at any time. These identities are not visible to the FIPS network
itself — they are used only at the application layer and for end-to-end
session encryption.
**Infrastructure independence**: The internet depends on centralized
infrastructure — ISPs, backbone providers, DNS, certificate authorities.
FIPS works over any transport that can carry packets: a serial connection,
onion-routed connections through Tor, local area networking, radio links
between remote sites, or the existing internet as an overlay. When the
internet is unavailable, unreliable, or untrusted, the mesh still works.
**Privacy by design**: FIPS provides secure, authenticated, and encrypted
communication between any two nodes in the mesh, independent of the mix of
transports used along the routed path between them. Furthermore, the mesh
itself is designed to minimize metadata exposure — intermediate nodes route
packets without learning the identities of the endpoints.
**Zero configuration**: Nodes discover each other and build routing
automatically. Connect to one peer and you can reach the entire mesh. The
network self-heals around failures and adapts to changing topology.
## A Self-Organizing Mesh
Traditional networks are built top-down. A central authority assigns
addresses, configures routing tables, provisions hardware, and manages the
topology. If the authority disappears or the infrastructure fails, the
network fails with it. Nodes cannot reach each other without infrastructure
mediating the connection.
FIPS inverts this model. There is no central authority, no address
assignment service, no routing table pushed from above. Each node generates
its own identity from a cryptographic keypair. Each node independently
decides which peers to connect to and which transports to use. From these
local decisions alone, the network self-organizes:
- A **spanning tree** forms through distributed parent selection, giving
every node a coordinate in the network without any node knowing the full
topology
- **Bloom filters** propagate through gossip, so each node learns which
peers can reach which destinations — again without global knowledge
- **Routing decisions** are made locally at each hop, using only the node's
immediate peers and cached coordinate information
Each peer link and end-to-end session actively measures RTT, loss, jitter,
and goodput through a lightweight in-band Metrics Measurement Protocol
(MMP), providing operator visibility and a foundation for quality-aware
routing.
The result is a network that builds itself from the bottom up, heals around
failures automatically, and scales without central coordination. Adding a
node is as simple as connecting to one existing peer — the network
integrates the new node through its normal mesh protocols.
## Specific Design Goals
- **Nostr-native identity and cryptography** — Use Nostr keypairs as node
identities and leverage secp256k1, Schnorr signatures, and SHA-256
- **Transport agnostic** — Support overlay, shared medium, and
point-to-point transports transparently
- **Self-organizing** — Automatic topology discovery and route optimization
- **Privacy preserving** — Minimize metadata leakage across untrusted links
- **Resilient** — Self-healing with graceful degradation
Non-goals include:
- **Reliable delivery** — FIPS provides a best-effort datagram service;
retransmission and ordering are left to applications or higher-layer
protocols
- **Anonymity** — Direct peers learn each other's identity; FIPS minimizes
metadata exposure but is not an anonymity network like Tor
- **Congestion control** — FIPS measures link quality but does not implement
flow control or congestion avoidance at the mesh layer
---
## Protocol Architecture
FIPS is organized in three protocol layers, each with distinct
responsibilities and clean service boundaries. No layer depends on the
specifics of the layers above or below it — transport plugins know nothing
about sessions, the routing layer knows nothing about application addressing,
and applications know nothing about which physical media carry their traffic.
This separation means new transports, protocol features, and application
interfaces can be added independently.
![Protocol Stack](diagrams/fips-protocol-stack.svg)
### Mapping to Traditional Networking
Readers familiar with the OSI model or TCP/IP networking may find it helpful
to see how FIPS concepts relate to traditional layers:
![OSI Mapping](diagrams/fips-osi-mapping.svg)
Note that FMP spans what would traditionally be separate link and network
layers. This is intentional — in a self-organizing mesh, the same layer that
authenticates peers also makes routing decisions, because routing depends on
authenticated peer state (spanning tree positions, bloom filters).
### Layer Responsibilities
**Transport layer**: Delivers datagrams between endpoints over a specific
medium. Each transport type (UDP socket, Ethernet interface, radio modem)
implements the same abstract interface: send and receive datagrams, report
MTU. The transport layer knows nothing about FIPS identities, routing, or
encryption. It provides raw datagram delivery to FMP above.
See [fips-transport-layer.md](fips-transport-layer.md) for the transport layer
specification.
**FIPS Mesh Protocol (FMP)**: Manages peer connections, authenticates peers
via Noise IK handshakes, and encrypts all traffic on each link. FMP is where
the mesh organizes itself — nodes exchange spanning tree announcements and
bloom filters with their direct peers, and FMP makes forwarding decisions
for transit traffic. FMP provides authenticated, encrypted forwarding to FSP
above.
See [fips-mesh-layer.md](fips-mesh-layer.md) for the FMP specification and
[fips-mesh-operation.md](fips-mesh-operation.md) for how FMP's routing and
self-organization work in practice.
**FIPS Session Protocol (FSP)**: Provides end-to-end authenticated
encryption between any two nodes, regardless of how many intermediate hops
separate them. FSP manages session lifecycle (setup, data transfer,
teardown), caches destination coordinates for efficient routing, and handles
the warmup strategy that keeps transit node caches populated. Session
dispatch uses index-based routing inspired by
[WireGuard](https://www.wireguard.com/), enabling O(1) packet
demultiplexing. FSP provides a datagram service to applications above.
See [fips-session-layer.md](fips-session-layer.md) for the FSP specification.
**IPv6 adaptation layer**: Sits above FSP as a service on port 256, adapting
the FIPS datagram service for unmodified IPv6 applications. Provides DNS
resolution (npub → fd00::/8 address), identity cache management, IPv6 header
compression, MTU enforcement, and a TUN interface. This is the primary way
existing applications use the FIPS mesh.
See [fips-ipv6-adapter.md](fips-ipv6-adapter.md) for the IPv6 adapter.
### Node Architecture
Application services sit at the top of the stack, dispatched by FSP port
number: the IPv6 TUN adapter (port 256) maps npubs to `fd00::/8` addresses
with header compression so unmodified IP applications can use the network
transparently, while the native datagram API addresses destinations directly
by npub.
![Node Architecture](diagrams/fips-node-architecture.svg)
The mesh routes application traffic across heterogeneous transports
transparently. A packet may traverse WiFi, Ethernet, UDP/IP, and Tor links
on its way from source to destination — the application never needs to know
which transports are involved. Each hop is independently encrypted at the
link layer, while a single end-to-end session protects the payload across
the entire path.
![Architecture Overview](diagrams/fips-architecture-overview.svg)
---
## Identity System
FIPS uses [Nostr](https://github.com/nostr-protocol/nips) keypairs
(secp256k1) as node identities. The public key identifies the node; the
private key signs protocol messages and establishes encrypted sessions.
The public key (or its bech32-encoded npub form) is the primary means for
application-layer software to identify communication endpoints. Internally,
the protocol derives a `node_addr` (a 16-byte SHA-256 hash of the pubkey)
used as the routing identifier in packet headers, and an IPv6 address derived
from the node_addr for the TUN adapter. Applications use the pubkey or npub;
the routing layer uses node_addr; unmodified IPv6 applications use the
derived `fd00::/8` address. All three are deterministically derived from the
same keypair.
### FIPS Identity Handling
![Identity Derivation](diagrams/fips-identity-derivation.svg)
The pubkey is the node's cryptographic identity, used in Noise IK handshakes
for both link and session encryption. It is never exposed beyond the
endpoints of an encrypted channel. The node_addr, a one-way SHA-256 hash
truncated to 16 bytes, serves as the routing identifier in packet headers
and bloom filters. Intermediate routers see only node_addrs — they can
forward traffic without learning the Nostr identities of the endpoints. An
observer can verify "does this node_addr belong to pubkey X?" if they already
know the pubkey, but cannot enumerate communicating identities by inspecting
traffic. The IPv6
address prepends `fd` to the first 15 bytes of the node_addr, providing a
ULA overlay address for unmodified IP applications via the TUN interface.
Below the FIPS identity layer, each transport uses its own native addressing
— IP:port or hostname:port addresses, MAC addresses, .onion identifiers. These **link
addresses** are opaque to everything above FMP and discarded once link
authentication completes.
### Identity Verification
The Noise Protocol Framework mutually authenticates both peer-to-peer link
connections (at FMP) and end-to-end session traffic (at FSP), proving each
party controls the private key for their claimed identity.
See [fips-mesh-layer.md](fips-mesh-layer.md) for peer authentication and
[fips-session-layer.md](fips-session-layer.md) for end-to-end session
establishment.
Key rotation changes the node's identity — a new keypair produces a new
node_addr and IPv6 address, requiring all sessions to be re-established.
Migration mechanisms that allow a node to announce a successor key are a
future consideration.
---
## Two-Layer Encryption
FIPS uses independent encryption at two protocol layers:
| Layer | Scope | Pattern | Purpose |
| ----- | ----- | ------- | ------- |
| **FMP (Mesh)** | Hop-by-hop | Noise IK | Encrypt all traffic on each peer link |
| **FSP (Session)** | End-to-end | Noise XK | Encrypt application payload between endpoints |
### Link Layer (Hop-by-Hop)
When two nodes establish a direct connection, they perform a [Noise
IK](https://noiseprotocol.org/) handshake. This authenticates both parties
and establishes symmetric keys for encrypting all traffic on that link.
Every packet between direct peers is encrypted — gossip messages, routing
queries, and forwarded session datagrams alike.
The IK pattern is used because outbound connections know the peer's npub
from configuration, while inbound connections learn the initiator's identity
from the first handshake message.
### Session Layer (End-to-End)
FIPS establishes end-to-end encrypted sessions between any two communicating
nodes using Noise XK, regardless of how many hops separate them. The
initiator knows the destination's npub (required for XK's pre-message);
the responder learns the initiator's identity from the third handshake
message. Unlike the link-layer IK pattern where the initiator's identity
is revealed in msg1, XK delays identity disclosure until msg3, providing
stronger initiator identity protection for traffic traversing untrusted
intermediate nodes.
A packet from A to D through intermediate nodes B and C:
1. A encrypts payload with A↔D session key (FSP)
2. A wraps in SessionDatagram, encrypts with A↔B link key (FMP), sends to B
3. B decrypts link layer, reads destination node_addr, re-encrypts with B↔C
link key, forwards to C
4. C decrypts link layer, re-encrypts with C↔D link key, forwards to D
5. D decrypts link layer, then decrypts session layer to get payload
Intermediate nodes route based on destination node_addr but cannot read
session-layer payloads. Each hop strips one link encryption and applies the
next — the session-layer ciphertext passes through untouched.
Both layers always apply, even between adjacent peers — a packet to a direct
neighbor is still encrypted twice. This uniform model means no special cases
for local vs remote destinations, and topology changes (a direct peer
becomes reachable only through intermediaries) don't affect existing
sessions.
See [fips-mesh-layer.md](fips-mesh-layer.md) for link encryption and
[fips-session-layer.md](fips-session-layer.md) for session encryption.
---
## Routing and Mesh Operation
Each node makes forwarding decisions using only local information — its
immediate peers, their bloom filters, and cached coordinates — rather than
centrally distributed routing tables or global topology knowledge. Two
complementary mechanisms provide the information each node needs.
### Spanning Tree: The Coordinate System
![Mesh Topology](diagrams/fips-mesh-topology.svg)
Nodes self-organize into a spanning tree through gossip — each node
exchanges announcements with its direct peers and independently selects a
parent. Because every node applies the same rule (prefer the root with the
smallest node_addr), the network converges on a single agreed-upon root
without any voting or coordination. This is the same principle behind the
[Spanning Tree
Protocol](https://en.wikipedia.org/wiki/Spanning_Tree_Protocol) used in
Ethernet bridging since the 1980s: purely local decisions that converge to
consistent global state. The resulting tree gives every node a
**coordinate** — its path from itself to the root. Using tree coordinates
for routing is adapted from
[Yggdrasil](https://yggdrasil-network.github.io/)'s
[Ironwood](https://github.com/Arceliar/ironwood) routing library.
These coordinates enable distance calculations between any two nodes: the
distance is the number of hops from each node to their lowest common
ancestor in the tree. This provides a metric for routing decisions without
any node needing to know the full network topology.
The tree maintains itself through gossip — nodes exchange TreeAnnounce
messages with their peers, propagating parent selections and ancestry
chains. Changes cascade through the tree proportional to depth, not network
size. If the network partitions, each segment converges to its own new root
through the same process and reconverges automatically when segments rejoin.
See [fips-spanning-tree.md](fips-spanning-tree.md) for the tree algorithms
and [spanning-tree-dynamics.md](spanning-tree-dynamics.md) for detailed
convergence walkthroughs.
### Bloom Filters: Candidate Selection
The spanning tree provides a coordinate system for distance-based routing,
but on its own each node would only know about its immediate neighbors.
Bloom filters complement the tree by distributing reachability knowledge
across the entire mesh — each node learns which destinations are reachable
through which peers, without any node needing a complete view of the
network.
Each node's peer-advertised [bloom
filter](https://en.wikipedia.org/wiki/Bloom_filter) is a compact, fixed-size
data structure that answers one question: "can this peer possibly reach
destination D?" The answer is either "no" (definitive) or "maybe"
(probabilistic — false positives are possible). Because the filter size is
constant regardless of how many destinations it represents, bloom filters
scale efficiently as the network grows. This is candidate selection for
routing — bloom filters narrow the set of peers worth considering, and the
actual forwarding decision ranks those candidates by tree distance and link
quality.
Filters propagate transitively through tree edges, with each node computing
outbound filters by merging the filters received from its tree peers (parent
and children) using a
[split-horizon](https://en.wikipedia.org/wiki/Split_horizon_route_advertisement)
technique borrowed from distance-vector routing. All peers — including
non-tree mesh shortcuts — receive FilterAnnounce messages, but only tree
peers' filters are merged into outgoing computation. This prevents filter
saturation where mesh shortcuts would cause every filter to converge toward
the full network.
See [fips-bloom-filters.md](fips-bloom-filters.md) for filter parameters and
mathematical properties.
![Bloom filter propagation on a spanning tree](diagrams/fips-bloom-propagation.svg)
The outbound filter for peer Q merges this node's identity with tree peer
inbound filters except Q's (split-horizon exclusion). This creates
directional asymmetry: upward filters (child → parent) contain the child's
subtree, while downward filters (parent → child) contain the complement.
Mesh peers receive filters but their inbound filters are not merged
transitively — they provide single-hop shortcut visibility only.
A node with multiple peers receives genuinely different filters from each.
In the diagram, R receives {B, D, E} from B and {C, F} from C — two disjoint
subtrees. When R needs to reach F, only C's filter matches. This is where
bloom filters provide real candidate selection: a node with several peers
can narrow the forwarding choice before consulting tree coordinates. Leaf
nodes like D have only one peer, so their single inbound filter is
necessarily near-complete (everything except themselves) and offers no
selection — but leaf nodes have no choice to make anyway.
Bloom filter sizing (bit count and hash functions) requires further analysis
based on actual deployment scenarios. The FMP wire format is versioned to
accommodate future parameter changes as operational experience accumulates.
### Routing Decisions
At each hop, FMP makes a local forwarding decision using the following
priority chain:
1. **Local delivery** — the destination is this node
2. **Direct peer** — the destination is an authenticated neighbor
3. **Bloom-guided candidate selection** — bloom filters identify peers that
can reach the destination; tree coordinates rank them by distance and
link quality
4. **[Greedy routing](https://en.wikipedia.org/wiki/Greedy_embedding)** —
fallback when bloom filters haven't converged; forward to the peer that
minimizes tree distance to the destination
5. **No route** — destination unreachable; send error signal to source
All multi-hop routing depends on knowing the destination's tree coordinates.
These are cached at each node after being learned through discovery
(LookupRequest/LookupResponse) or session establishment (SessionSetup). The
coordinate cache is the critical piece that enables efficient forwarding.
![Per-hop routing decision flowchart](diagrams/fips-routing-decision.svg)
### Coordinate Caching and Discovery
When a node first needs to reach an unknown destination, it sends a
LookupRequest that propagates through the network guided by bloom filters
and loop prevention. The destination responds with its coordinates, which
the source and intermediate nodes along the return path cache. Subsequent
traffic routes efficiently using the cached coordinates.
Session establishment (SessionSetup) also carries coordinates, warming
transit node caches along the path so that data packets can be forwarded
without individual discovery at each hop.
![Coordinate discovery and cache warming sequence](diagrams/fips-coordinate-discovery.svg)
### Error Recovery
When routing fails — because cached coordinates are stale, a path has
broken, or a packet exceeds a link's MTU — transit nodes signal the source:
- **CoordsRequired**: A transit node lacks the destination's coordinates.
The source re-initiates discovery and resets its coordinate warmup
strategy.
- **PathBroken**: Greedy routing reached a dead end. The source re-discovers
the destination's current coordinates.
- **MtuExceeded**: A transit node cannot forward a packet because it exceeds
the next-hop link MTU. The source adjusts its path MTU estimate.
All three signals trigger active recovery, and are rate-limited to prevent
storms during topology changes.
See [fips-mesh-operation.md](fips-mesh-operation.md) for the complete
routing and mesh behavior description.
### Metrics Measurement Protocol (MMP)
Each peer link runs an instance of the Metrics Measurement Protocol, which
measures link quality through in-band report exchange. MMP computes smoothed
round-trip time (SRTT), packet loss rate, interarrival jitter, goodput, and
one-way delay trend — all derived from counter and timestamp fields already
present in the FMP wire format, with no additional probing traffic required.
MMP operates in three modes. **Full** mode exchanges both SenderReports and
ReceiverReports to compute all metrics including RTT. **Lightweight** mode
exchanges only ReceiverReports, providing loss and jitter but not RTT — useful
for constrained links. **Minimal** mode disables reports entirely, relying
only on spin bit and congestion echo flags in the frame header.
Reports are sent at RTT-adaptive intervals (clamped to 100 ms2 s), so
high-latency links don't generate excessive measurement traffic while
low-latency links converge quickly. Each metric carries both short-term and
long-term exponentially weighted moving averages, enabling detection of
quality changes against a stable baseline.
MMP serves dual roles: operator visibility and cost-based parent selection.
Periodic log lines report per-link RTT, loss, jitter, and goodput. MMP
computes an Expected Transmission Count (ETX) from bidirectional delivery
ratios, which feeds into cost-based parent selection where each node
evaluates `effective_depth = depth + link_cost` using
`link_cost = etx * (1.0 + srtt_ms / 100.0)`. ETX is not yet used in
`find_next_hop()` candidate ranking for data forwarding.
See [fips-mesh-layer.md](fips-mesh-layer.md) for MMP operating modes, report
scheduling, and the spin bit design.
---
## Transport Abstraction
FIPS treats the communication medium as a pluggable component. Every transport
— whether a UDP socket, an Ethernet interface, a Tor circuit, or a radio modem
— implements the same simple interface: send a datagram to an address, receive
datagrams, and report the link MTU. The rest of the protocol stack sees no
difference between them.
A **transport** is a driver for a particular medium. A **link** is a peer
connection established over a transport. Transport addresses (IP:port, MAC
address, .onion) are opaque to all layers above FMP — they exist only to
deliver datagrams and are discarded once FMP has authenticated the peer via
the Noise IK handshake. From that point on, the peer is identified solely by
its cryptographic identity.
Transports fall into three categories based on their connectivity model:
| Category | Examples | Characteristics |
| -------- | -------- | --------------- |
| Overlay | UDP/IP, Tor | Tunnels FIPS over existing networks |
| Shared medium | Ethernet, WiFi, Bluetooth, Radio | Local broadcast, peer discovery |
| Point-to-point | Serial, dialup | Fixed connections, no discovery |
These categories differ in addressing, MTU, reliability, and whether they
support local discovery, but FMP handles all of them uniformly. A node
running multiple transports simultaneously bridges between those networks
automatically — peers from all transports feed into a single spanning tree,
and the router selects the best path regardless of which medium carries it.
If one transport fails, traffic reroutes through alternatives without
application involvement.
Some transports support an optional discovery capability — the ability to
broadcast and listen for announcements indicating the availability of FIPS
endpoints on the local medium. Shared media like Ethernet, WiFi, Bluetooth,
and radio are natural fits for this, as they can reach nearby devices without
prior configuration. When discovery is available, nodes can automatically
find and peer with other FIPS nodes on the same medium. Transports that
lack discovery (such as configured UDP endpoints) simply skip this step and
connect directly to configured addresses. For internet-reachable nodes,
endpoint discovery via signed Nostr events allows nodes to publish and
consume transport addresses through public relays — available behind the
`nostr-discovery` cargo feature.
NAT traversal for internet-connected nodes is supported via STUN-assisted
UDP hole punching, with offer/answer signaling carried over Nostr gift-wrap
events. Once a direct UDP path is established, the punched socket is handed
into the standard FIPS transport/session stack. Nodes that cannot establish
a direct path remain reachable through port forwarding, a publicly addressed
peer, or relay through other mesh nodes. The Nostr-mediated discovery and
NAT traversal paths are gated by the `nostr-discovery` cargo feature and
configured under `node.discovery.nostr.*`.
> **Implementation status**: UDP/IP, TCP/IP, Ethernet, Tor
> (SOCKS5 outbound + directory-mode inbound via onion service),
> and Bluetooth (BLE L2CAP CoC) transports are implemented.
> All others are future directions.
See [fips-transport-layer.md](fips-transport-layer.md) for the full transport
layer specification.
---
## Security
FIPS is designed around four classes of adversary, each addressed by a
different layer of the protocol.
### Transport Observers
A passive observer on the underlying transport — someone monitoring a WiFi
network, tapping an Ethernet segment, or inspecting UDP traffic — sees only
encrypted packets. The FMP link-layer Noise IK session encrypts all traffic
between direct peers, including routing gossip and forwarded session
datagrams. The observer can infer timing, packet sizes, and which transport
endpoints are exchanging traffic, but cannot read content or determine
FIPS-level node identities from the encrypted packets. Traffic analysis —
correlating timing and volume patterns across multiple vantage points to
infer communication relationships — is not defended against (see
[Specific Design Goals](#specific-design-goals)).
### Active Attackers on the Transport
An adversary who can inject, modify, drop, or replay packets on the
transport is also defeated by the FMP link-layer Noise IK session. Mutual
authentication prevents impersonation, AEAD encryption detects tampering,
and counter-based nonces with a sliding replay window reject replayed
packets.
### Other FIPS Nodes (Intermediate Routers)
The most important adversary class is the operators of other nodes in the
mesh — the peers that forward your traffic. FIPS treats every intermediate
router as potentially adversarial. The FSP session layer establishes a
completely independent Noise XK session between the communicating endpoints,
so intermediate nodes cannot read application payloads even though they
decrypt and re-encrypt the link-layer envelope at each hop.
Routing headers expose only the destination's node_addr — an opaque
SHA-256 hash of the actual public key. Intermediate routers can forward
traffic without learning which Nostr identities are communicating. An
observer can verify "does this node_addr belong to pubkey X?" if they
already know the pubkey, but cannot enumerate communicating identities by
inspecting routed traffic.
| Entity | Can See |
| ------ | ------- |
| Transport observer | Encrypted packets, timing, packet sizes |
| Direct peer | Your npub, traffic volume, timing |
| Intermediate router | Source and destination node_addrs, packet size |
| Destination | Your npub, payload content |
### Adversarial Nodes Disrupting the Mesh
Beyond passive observation, a malicious node could attempt to disrupt
routing by injecting false spanning tree announcements, advertising bogus
bloom filters, or claiming invalid tree positions. FMP mitigates these
through signed TreeAnnounce messages verified by direct peers, transitive
ancestry chain validation, replay protection via sequence numbers, and
discretionary peering — node operators choose who to peer with, so an
attacker with many identities still needs real nodes to accept their
connections. Handshake rate limiting further constrains how fast an attacker
can establish new links. In fully open networks with automatic peer
discovery, Sybil resistance relies primarily on rate limiting; discretionary
peering provides stronger resistance in curated deployments where operators
vet their peers. An attacker who controls all of a target node's direct
peers can completely control its view of the network (an eclipse attack);
diverse peering across independent operators and transports is the primary
mitigation.
---
## Prior Work
FIPS builds on proven designs rather than inventing new cryptography or routing
algorithms. Nearly every major design decision has deployed precedent.
### Spanning Tree Self-Organization
The idea that distributed nodes can build a spanning tree through purely local
decisions — each node selecting a parent based on announcements from its
neighbors — dates to the
[IEEE 802.1D Spanning Tree Protocol](https://en.wikipedia.org/wiki/Spanning_Tree_Protocol)
(STP, 1985). STP demonstrated that a network-wide tree emerges from a simple
deterministic rule (lowest bridge ID wins root election) applied independently
at each node. FIPS uses the same principle — lowest node address determines the
root — adapted from an Ethernet bridging context to a general-purpose overlay
mesh.
### Tree Coordinate Routing
The spanning tree coordinates, bloom filter candidate selection, and greedy
routing algorithms are adapted from
[Yggdrasil v0.5](https://yggdrasil-network.github.io/2023/10/22/upcoming-v05-release.html)
and its [Ironwood](https://github.com/Arceliar/ironwood) routing library.
Yggdrasil's key insight was using the tree path from root to node as a
routable coordinate, enabling greedy forwarding without global routing tables.
FIPS adapts these algorithms for multi-transport operation, Nostr identity
integration, and constrained MTU environments.
The theoretical foundation for greedy routing on tree embeddings draws on
[Kleinberg's work](https://www.cs.cornell.edu/home/kleinber/swn.pdf) on
navigable small-world networks, which showed that greedy forwarding succeeds
in O(log² n) steps when the network has hierarchical structure. Thorup-Zwick
compact routing schemes separately demonstrated that sublinear routing state
is achievable with bounded stretch, motivating the use of tree coordinates
rather than full routing tables.
### Split-Horizon Bloom Filter Propagation
FIPS distributes reachability information using bloom filters computed with a
split-horizon rule: when advertising to a peer, exclude that peer's own
contributions. This technique is borrowed from distance-vector routing
protocols — [RIP](https://en.wikipedia.org/wiki/Routing_Information_Protocol)
(1988) and [Babel](https://www.irif.fr/~jch/software/babel/) use split-horizon
to prevent routing loops by not advertising a route back to the neighbor it was
learned from. FIPS applies the same principle to probabilistic set
advertisements rather than distance-vector tables.
### Cryptographic Identity as Network Address
FIPS nodes are identified by their Nostr public keys (secp256k1). The network
address *is* the cryptographic identity — there is no separate address
assignment or registration step.
[CJDNS](https://github.com/cjdelisle/cjdns) pioneered this approach in
overlay meshes, deriving IPv6 addresses from the double-SHA-512 of each node's
public key. Tor [.onion addresses](https://spec.torproject.org/rend-spec-v3)
and the IETF
[Host Identity Protocol](https://en.wikipedia.org/wiki/Host_Identity_Protocol)
(HIP) follow the same principle. FIPS uses Nostr's existing key infrastructure
rather than introducing a new identity scheme.
### Dual-Layer Encryption
FIPS encrypts traffic twice: FMP provides hop-by-hop link encryption
(protecting against transport-layer observers), while FSP provides independent
end-to-end session encryption (protecting against intermediate FIPS nodes).
This layered approach mirrors [Tor](https://www.torproject.org/), where each
relay peels one layer of encryption (hop-by-hop) while the innermost layer
protects end-to-end payload. [I2P](https://geti2p.net/) uses a similar
garlic routing scheme with tunnel-layer and end-to-end encryption. Unlike Tor
and I2P, FIPS does not provide anonymity — its dual encryption protects
confidentiality and integrity rather than hiding traffic patterns.
### Noise Protocol Framework
FIPS uses the [Noise Protocol Framework](https://noiseprotocol.org/) at both
protocol layers, with different handshake patterns chosen for each layer's
threat model. FMP link encryption uses **Noise IK**, providing mutual
authentication with a single round trip where the initiator knows the
responder's static key in advance.
[WireGuard](https://www.wireguard.com/) uses the same IK base pattern
(extended with a pre-shared key as IKpsk2) for VPN tunnels. FSP session
encryption uses **Noise XK**, the same pattern used by the
[Lightning Network](https://github.com/lightning/bolts/blob/master/08-transport.md),
where the initiator's static key is transmitted in a third message rather
than the first. XK provides stronger initiator identity hiding at the cost
of an additional round trip — a worthwhile tradeoff for session-layer traffic
that traverses untrusted intermediate nodes. At the link layer, where both
peers are configured and directly connected, IK's single round trip is
preferred.
### Index-Based Session Dispatch
FIPS uses locally-assigned 32-bit session indices to demultiplex incoming
packets to the correct cryptographic session in O(1) time, without parsing
source addresses or performing expensive lookups. This directly follows
[WireGuard's](https://www.wireguard.com/papers/wireguard.pdf) receiver index
approach, where each peer assigns a random index during handshake and the
remote side includes it in every packet header.
### Transport-Agnostic Overlay Mesh
FIPS is designed to operate over any datagram-capable transport — UDP, raw
Ethernet, Bluetooth, radio, serial — through a uniform transport abstraction.
Several mesh overlays have demonstrated transport-agnostic design:
[CJDNS](https://github.com/cjdelisle/cjdns) runs over UDP and Ethernet,
[Yggdrasil](https://yggdrasil-network.github.io/) supports TCP and TLS
transports, and [Tor](https://www.torproject.org/) can use pluggable
transports to tunnel through various media. FIPS extends this pattern to
shared-medium transports (radio, BLE) with per-transport MTU and discovery
capabilities.
### Metrics Measurement Protocol
MMP's design assembles well-established measurement techniques into a unified
per-link protocol. The SenderReport/ReceiverReport exchange structure follows
[RTCP](https://www.rfc-editor.org/rfc/rfc3550) (RFC 3550), which uses the
same report pairing for media stream quality monitoring in RTP sessions. MMP's
jitter computation uses the RTCP interarrival jitter algorithm directly.
The smoothed RTT estimator uses the Jacobson/Karels algorithm
([RFC 6298](https://www.rfc-editor.org/rfc/rfc6298)), the same SRTT
computation used in TCP for retransmission timeout calculation since 1988.
MMP derives RTT from timestamp-echo in ReceiverReports with dwell-time
compensation, rather than from packet round-trips.
The spin bit in the FMP frame header follows the
[QUIC](https://www.rfc-editor.org/rfc/rfc9000) spin bit
([RFC 9312](https://www.rfc-editor.org/rfc/rfc9312)) — a single bit that
alternates each round trip, enabling passive latency measurement. FIPS
implements the spin bit state machine but relies on timestamp-echo for SRTT,
as irregular mesh traffic makes spin bit RTT unreliable.
The Expected Transmission Count (ETX) metric, computed from bidirectional
delivery ratios, was introduced by
[De Couto et al. (2003)](https://pdos.csail.mit.edu/papers/grid:mobicom03/paper.pdf)
for wireless mesh routing and is used in protocols including
[OLSR](https://en.wikipedia.org/wiki/Optimized_Link_State_Routing_Protocol)
and [Babel](https://www.irif.fr/~jch/software/babel/). FIPS computes ETX
per-link from MMP loss measurements for future use in candidate ranking.
The CE (Congestion Experienced) echo flag provides hop-by-hop
[ECN](https://en.wikipedia.org/wiki/Explicit_Congestion_Notification)
signaling, following the TCP/IP ECN echo pattern (RFC 3168). Transit nodes
detect congestion via MMP loss/ETX metrics or kernel buffer drops and set
the CE flag on forwarded frames; destination nodes mark ECN-capable IPv6
packets accordingly.
### Cryptographic Primitives
FIPS reuses [Nostr's](https://github.com/nostr-protocol/nips) cryptographic
stack — secp256k1 for identity keys, Schnorr signatures for authentication,
SHA-256 for hashing, and ChaCha20-Poly1305 for authenticated encryption. This
is the same primitive set used across Bitcoin, Nostr, and a growing ecosystem
of self-sovereign identity systems. No novel cryptography is introduced.
---
## Further Reading
### Protocol Layers
| Document | Description |
| -------- | ----------- |
| [fips-transport-layer.md](fips-transport-layer.md) | Transport layer: abstraction, types, services provided to FMP |
| [fips-mesh-layer.md](fips-mesh-layer.md) | FMP: peer authentication, link encryption, forwarding |
| [fips-session-layer.md](fips-session-layer.md) | FSP: end-to-end encryption, session lifecycle |
| [fips-ipv6-adapter.md](fips-ipv6-adapter.md) | IPv6 adaptation: DNS, TUN interface, MTU enforcement |
### Mesh Behavior and Wire Formats
| Document | Description |
| -------- | ----------- |
| [fips-mesh-operation.md](fips-mesh-operation.md) | How the mesh operates: routing, discovery, error recovery |
| [fips-wire-formats.md](fips-wire-formats.md) | Complete wire format reference for all protocol layers |
### Supporting References
| Document | Description |
| -------- | ----------- |
| [fips-spanning-tree.md](fips-spanning-tree.md) | Spanning tree algorithms and data structures |
| [fips-bloom-filters.md](fips-bloom-filters.md) | Bloom filter parameters, math, and computation |
| [spanning-tree-dynamics.md](spanning-tree-dynamics.md) | Scenario walkthroughs: convergence, partitions, recovery |
### Implementation
| Document | Description |
| -------- | ----------- |
| [fips-configuration.md](fips-configuration.md) | YAML configuration reference |
### External References
- [IEEE 802.1D Spanning Tree Protocol](https://en.wikipedia.org/wiki/Spanning_Tree_Protocol)
- [Yggdrasil Network](https://yggdrasil-network.github.io/)
- [Yggdrasil v0.5 Release Notes](https://yggdrasil-network.github.io/2023/10/22/upcoming-v05-release.html)
- [Ironwood Routing Library](https://github.com/Arceliar/ironwood)
- [Kleinberg — The Small-World Phenomenon](https://www.cs.cornell.edu/home/kleinber/swn.pdf)
- [CJDNS](https://github.com/cjdelisle/cjdns)
- [Tor Project](https://www.torproject.org/)
- [I2P](https://geti2p.net/)
- [Host Identity Protocol (HIP)](https://en.wikipedia.org/wiki/Host_Identity_Protocol)
- [Babel Routing Protocol](https://www.irif.fr/~jch/software/babel/)
- [Noise Protocol Framework](https://noiseprotocol.org/)
- [WireGuard](https://www.wireguard.com/)
- [WireGuard Whitepaper](https://www.wireguard.com/papers/wireguard.pdf)
- [Lightning Network BOLT #8 — Transport](https://github.com/lightning/bolts/blob/master/08-transport.md)
- [QUIC (RFC 9000)](https://www.rfc-editor.org/rfc/rfc9000)
- [QUIC Spin Bit (RFC 9312)](https://www.rfc-editor.org/rfc/rfc9312)
- [RTCP (RFC 3550)](https://www.rfc-editor.org/rfc/rfc3550)
- [TCP SRTT / RTO (RFC 6298)](https://www.rfc-editor.org/rfc/rfc6298)
- [ECN (RFC 3168)](https://www.rfc-editor.org/rfc/rfc3168)
- [ETX — De Couto et al. 2003](https://pdos.csail.mit.edu/papers/grid:mobicom03/paper.pdf)
- [OLSR](https://en.wikipedia.org/wiki/Optimized_Link_State_Routing_Protocol)
- [Nostr Protocol](https://github.com/nostr-protocol/nips)