mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
master
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Johnathan Corgan
|
5abf9a9325 |
docs: four-section /docs/ restructure with new-user content, accuracy pass, and gateway feature-set rewrite
Restructures /docs/ by reader purpose (tutorials, how-to, reference, design), adds the new-user-progression and operator-recipe content the prior layout lacked, runs an accuracy pass against current source across the pre-existing design docs, and rewrites the gateway feature-set documentation end-to-end around its actual operational profile (a niche feature designed for systems already serving DHCP/DNS to a LAN, with two independent halves — outbound LAN→mesh, inbound mesh→LAN — sharing one nftables table, one binary, and one control socket). Top-level README and getting-started rewritten around two equally-weighted deployment modes (overlay on existing IP networks; ground-up over non-IP transports). ## Additions - 11 new tutorials in docs/tutorials/: an 8-step new-user progression from single-daemon test-mesh peering through to a ground-up two-device mesh, an IPv6-adapter side-trip walkthrough, an Advanced Tutorials index, and a hand-held OpenWrt walk-through for fips-gateway deployment that exercises both halves of the feature. - 12 new how-tos in docs/how-to/: firewall activation, Nostr discovery (resolve / advertise / open across five scenarios), Tor onion (directory + control_port modes), UDP buffer tuning, unprivileged-user setup, persistent identity, host aliases, Bluetooth LE peering, MTU diagnostics, manual Linux-host gateway deployment (covers both halves), gateway troubleshooting (organised by half), and a section index. - 9 new reference docs in docs/reference/: configuration, wire formats, control-socket protocol, four CLI references (fips, fipsctl, fipstop, fips-gateway), security posture matrix, and Nostr events catalog. Configuration and wire-formats are renamed-and-extended from prior design/ versions; the other seven are net-new. - 6 new design docs: fips-concepts, fips-architecture, and fips-prior-work split out of the deleted fips-intro.md; consolidated fips-mmp and fips-mtu aggregations; and a new generic port-advertisement-and-nat-traversal doc (Nostr-signaled port advertisement plus UDP NAT-traversal protocol, FIPS as an example implementation, suitable for eventual NIP submission). - Top-level docs/getting-started.md walking through the binary-installer-only Install story. - packaging/common/hosts pre-populated with the eight public test-mesh nodes so shortnames resolve out of the box on every fresh install. ## Changes - 23 wire-format diagrams relocated to reference/diagrams/ alongside the wire-formats move. - 4 design diagrams corrected against source code (fips-protocol-stack, fips-identity-derivation, fips-coordinate-discovery, fips-routing-decision). - 10 pre-existing design docs reconciled with current source. Numeric corrections: stale link-MMP report bounds (now [1s, 5s] with 200 ms cold-start floor); UDP default MTU (now 1280, IPv6 minimum); node_addr formula (SHA-256(pubkey)[..16]); Noise patterns (IK at link, XK at session); peer-ACL semantics (strict allowlist requires ALL in peers.deny); daemon DNS upstream ([::1]:5354); on-the-wire bloom-filter size (1,071 bytes); obsolete Cargo-feature references (PR #79 dropped them) removed. - Transport framing tightened across the docs: TCP is for UDP-filtered networks (not NAT traversal); Tor is a deployment mode (not failover); WebSocket dropped (not a shipped FIPS transport); WiFi promoted to Implemented via Ethernet in infrastructure mode; classic-Bluetooth row removed (BLE is the only Bluetooth-mode transport). - docs/design/fips-gateway.md rewritten end-to-end to lead with the niche-feature framing and the two-halves structure. Title moved from "FIPS Outbound LAN Gateway" to "FIPS Gateway"; architecture section describes the common machinery (the fips-gateway service, the nftables table, the control socket) before splitting into separate "Outbound Half" and "Inbound Half" sections of equal weight; security considerations split per-half; no Future Work section (speculative directions live in the project tracker, not in protocol design docs). Inbound port forwarding is a first-class half rather than a buried "Implemented Extensions" subsection. - Gateway terminology unified across all gateway docs as a separate Linux service running alongside the fips daemon (its own systemd unit / OpenWrt init script). Container- pattern terms (sidecar) are reserved for the Docker/Kubernetes sidecar deployment examples — the testing/sidecar/ tree, examples/k8s-sidecar/, examples/sidecar-nostr-relay/, examples/wireguard-sidecar-macos/, and the related CHANGELOG / top-level README entries — where the term carries its standard container meaning. - Net-new design body content: rekey section in fips-mesh-layer (Noise IK msg1/msg2 over the established link, K-bit cutover, drain window, smaller-NodeAddr-wins tie-breaker on dual-init); Mesh Size Estimation and Antipoison FPR Cap sections in fips-bloom-filters; Mesh-Interface Query Filter subsection in fips-ipv6-adapter; failure-suppression knobs and clock- skew tolerance in fips-nostr-discovery; loop-rejection and mid-chain ancestor swap added to spanning-tree propagation / stability rules; Priority Chain in fips-mesh-operation renumbered to match the routing-decision diagram. - Top-level README: dropped the stale nostr-discovery cargo-feature parenthetical. docs/README.md and the four section READMEs (tutorials, how-to, reference, design) refreshed for the new structure; index rows reflect both halves of the gateway feature and the new fips-gateway CLI reference. - Cargo.toml [package.metadata.deb] assets path updated for the fips-security.md move; .gitignore /reference/ rule anchored to repo root so docs/reference/ is trackable. - packaging/openwrt-ipk/files/etc/fips/fips.yaml configuration-doc URL updated to the new docs/reference/configuration.md location. ## Deletions - docs/design/fips-intro.md (split into the three new intro design docs). - docs/design/document-relationships.svg (orphan, no longer referenced). - docs/proposals/ tree removed; the only proposal it contained (the Nostr UDP hole-punch protocol) was rewritten as the new generic design/port-advertisement-and-nat-traversal.md. |
||
 
|
34e00b9f6e |
Add Nostr-mediated overlay discovery and UDP NAT traversal (#53)
Optional peer discovery and NAT hole-punching path gated behind a new
`nostr-discovery` cargo feature. Nodes publish signed overlay endpoint
adverts to public Nostr relays, consume peer adverts to populate
fallback dial addresses, and use STUN-assisted UDP hole punching with
NIP-59 gift-wrap offer/answer signaling to establish direct UDP paths
between NATed peers. Once a punched socket is up, it is handed into
the existing FIPS UDP transport and the standard Noise/FMP session
stack takes over unchanged.
The cargo feature is in the default feature set
(`default = ["nostr-discovery"]`) so stock builds include it; a
build that explicitly disables default features (or selects a
feature set without `nostr-discovery`) does not link the nostr /
nostr-sdk crates and does not emit a no-op poll in the tick loop.
Runtime behavior is independently gated by
`node.discovery.nostr.enabled`, which defaults to false; if the
config enables Nostr on a non-feature build, startup logs a
warning and continues without it.
== Cargo feature and dependencies
- New cargo feature `nostr-discovery = ["dep:nostr", "dep:nostr-sdk"]`.
Not in the default feature set.
- New optional Linux-only dependencies: `nostr 0.44` (features: std,
nip59) and `nostr-sdk 0.44`. Gift-wrap unwrap is hand-rolled in
`src/discovery/nostr/signal.rs` rather than relying on the SDK's
rumor-author check, which FIPS sidesteps by trusting `seal.pubkey`
exclusively.
== Wire format
Overlay advert event: `kind 37195`, parameterized replaceable
(NIP-01 application-defined replaceable range 30000-39999), with
`d = "fips-overlay-v1"`. The digits visually spell FIPS (7=F, 1=I,
9=P, 5=S); a relay survey confirmed the kind is unused.
Advert content carries the version tag, endpoint list
(`udp|tcp|tor` + addr), optional signal-relay and stun-server
metadata, and `issuedAt` / `expiresAt` timestamps. Endpoint
`addr: "nat"` is the sentinel that triggers traversal on the peer
side. NIP-40 `expiration` tag bounds staleness on permanent
shutdown. Lifecycle relies on parameterized-replaceable
supersession; the daemon does not emit NIP-09 kind-5 deletes —
strict relays (Damus, Primal) race delete-against-replace and can
silently drop the replacement.
Gift-wrapped signal event: `kind 21059`. Punch packets carry magic
values `PUNCH_MAGIC` / `PUNCH_ACK_MAGIC`, a sequence number, and a
16-byte session hash.
== Discovery surface
- `src/discovery.rs` (always compiled)
- `EstablishedTraversal`: bound UDP socket + selected remote +
peer npub + optional transport name/config tuning overrides.
- `BootstrapHandoffResult`: returned on successful handoff —
allocated transport id, local/remote addrs, peer NodeAddr,
session id.
- `src/discovery/nostr/` (`#![cfg(feature = "nostr-discovery")]`)
- `types.rs`: wire and control types described above. `ADVERT_KIND`
constant. `BootstrapError` enumerates failure modes (disabled,
missing advert, missing NAT endpoint, no usable relays, invalid
advert, invalid npub, signal timeout, punch timeout, replay,
STUN failure, protocol, nostr, io, serde, event-parse).
- `runtime.rs`: `NostrDiscovery` coordinator. Owns the shared
nostr-sdk `Client`, subscribes to advert + signal event kinds,
maintains a bounded advert cache and a bounded seen-sessions
replay set, drains `BootstrapEvent::{Established, Failed}` for
the node to consume, exposes `update_local_advert`,
`request_connect`, `advert_endpoints_for_peer`,
`cached_open_discovery_candidates`, and `shutdown`.
- `signal.rs`: NIP-59 gift-wrap encode/decode. Outbound wraps are
built against per-attempt ephemeral keys; inbound events are
unwrapped against the node identity.
- `stun.rs`: RFC 5389/8489 Binding Request client with
XOR-MAPPED-ADDRESS parsing for both IPv4 and IPv6; used only to
observe the initiator's own reflexive address against its
locally configured STUN list (peer-advertised STUN is
informational, never an egress target).
- `traversal.rs`: per-attempt candidate-pair punch planner.
Allocates a fresh `0.0.0.0:0` UDP socket per attempt, enumerates
LAN-private and ULA interface addresses alongside the STUN
reflexive address, schedules probe/ack exchanges at the
configured interval for the configured duration, and picks the
first candidate pair that authenticates end-to-end.
Strategy ordering is Reflexive↔Reflexive first, then LAN, then
Mixed. The STUN-observed pair is the only candidate that's reliable
across arbitrary network topologies; trying it first prevents the
planner from latching onto a misleading host-candidate path before
the reflexive path gets a chance. There is no catch-all
Local↔Local strategy: a previous design that paired every local
host candidate from one side with every local host candidate from
the other could declare success on a one-way reachable asymmetric
L3 path (corporate VPN, Tailscale subnet route, overlapping private
address space), only for the FMP handshake to stall because the
return path didn't match. The legitimate `Lan` strategy still pairs
candidates that share a subnet.
== Configuration surface
`node.discovery.nostr.*` (`NostrDiscoveryConfig`), all `serde(default)`
with `deny_unknown_fields`:
- `enabled` (default false), `advertise` (default true)
- `advert_relays`, `dm_relays`, `stun_servers`: defaults are
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`
for both relay lists, and Google / Cloudflare / Twilio for STUN.
Operators are expected to override for production. Other
verified-working public relays for reference:
`nostr.bitcoiner.social`, `nostr-pub.wellorder.net`,
`nostr.oxtr.dev`, `nostr.mom`.
- `app` (default `"fips-overlay-v1"`), `signal_ttl_secs` (120)
- `policy`: `NostrDiscoveryPolicy::{Disabled, ConfiguredOnly (default),
Open}` — controls whether advert-derived endpoints are consumed
only for peers carrying `via_nostr = true`, or also for
non-configured peers within a budget cap.
- `share_local_candidates` (default false) — when false, the offer's
`local_addresses` list is empty and peers see only the reflexive
address. Enable per-node only for genuinely same-LAN deployments;
off-by-default eliminates the misleading-path failure mode for
the common case where peers are not on the same broadcast domain.
- `open_discovery_max_pending` (64) — caps queued open-discovery
retries; bounded by available outbound slots.
- `max_concurrent_incoming_offers` (16) — semaphore against offer
spam; excess offers are debug-logged and dropped.
- `advert_cache_max_entries` (2048) and `seen_sessions_max_entries`
(2048) — bound memory under ambient relay volume; overflow
evictions are debug-logged.
- `attempt_timeout_secs` (10), `replay_window_secs` (300)
- `punch_start_delay_ms` (2000), `punch_interval_ms` (200),
`punch_duration_ms` (10000)
- `advert_ttl_secs` (3600), `advert_refresh_secs` (1800)
Per-peer and per-transport flags:
- `PeerConfig.via_nostr: bool` — when true (and Nostr is enabled),
advert-derived addresses are appended as fallback dial candidates
after static addresses for that peer.
- `PeerConfig.addresses` is now `serde(default)` and may be empty
when `via_nostr: true`; validation requires at least one of the
two to be present per peer, and the error message names the
peer's npub.
- `UdpConfig.advertise_on_nostr: Option<bool>` and
`UdpConfig.public: Option<bool>` — UDP transports can be
advertised either as direct `host:port` (public = true) or as the
`addr: "nat"` sentinel that triggers rendezvous on the peer side.
- `TcpConfig.advertise_on_nostr` and `TorConfig.advertise_on_nostr`
— TCP and Tor onion endpoints can be advertised as directly
reachable.
- A reserved peer address `transport: udp, addr: "nat"` parses without
special-casing in YAML and routes through the bootstrap runtime.
Cross-field validation (`Config::validate`, called from `Node::new`
and `Node::with_identity`):
- Any transport with `advertise_on_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- Any peer with `via_nostr = true` requires
`node.discovery.nostr.enabled = true`.
- A non-public UDP advert (`advertise_on_nostr = true`,
`public = false` — i.e. `udp:nat`) additionally requires at least
one `dm_relay` and at least one `stun_server`.
Surfaced as `ConfigError::Validation`.
== Node integration
`src/node/lifecycle.rs` is the main integration point.
- At node start (after transports are up, before TUN), if Nostr is
enabled and the feature is compiled in, `NostrDiscovery::start` is
invoked, the initial local overlay advert is built from the live
transport set and published, and the runtime handle is stored.
- The rx tick loop calls `poll_nostr_discovery` (feature-gated both
at method definition and call site), which refreshes the local
advert, drains bootstrap events, adopts established traversals,
schedules retries for failed traversals, and — under `policy:
open` — enqueues outbound retries for non-configured peers
visible in the advert cache, bounded by
`open_discovery_max_pending` and the remaining outbound slots.
- Outbound peer dialing is refactored to `try_peer_addresses`, which
first exhausts the static address list in priority order and only
then appends advert-derived fallback addresses; both lists run
through the same `attempt_peer_address_list` code path. The
`udp:nat` sentinel address triggers `NostrDiscovery::request_connect`
for the peer instead of a direct dial and returns `Ok(())`.
- `build_overlay_advert` walks operational transports, consults
per-instance `UdpConfig` / `TcpConfig` / `TorConfig` (matching by
optional transport instance name), and emits an `OverlayAdvert`
including `signalRelays` and `stunServers` when any UDP endpoint
is advertised as NAT.
- `adopt_established_traversal` is the bootstrap handoff API:
allocates a new `TransportId`, constructs a `UdpTransport` with
the user-supplied (or default) `UdpConfig`, calls the new
`adopt_socket_async` to reuse the punched socket verbatim,
registers the transport in the normal transport map, records it
in `bootstrap_transports`, and calls `initiate_connection` so the
normal handshake path runs. On failure, the transport is stopped
and removed cleanly and the set membership is rolled back.
- On clean shutdown, `NostrDiscovery::shutdown` is awaited so
background tasks stop before transports are torn down. (The
advert is not explicitly retracted; NIP-40 expiration plus the
next refresh from any live publisher supersedes it.)
New `Node` fields:
- `nostr_discovery: Option<Arc<NostrDiscovery>>` (feature-gated).
- `bootstrap_transports: HashSet<TransportId>` — per-peer UDP
transports adopted from NAT traversal, cleaned up via
`cleanup_bootstrap_transport_if_unused` whenever the link,
connection, peer, or pending-connect referencing them is removed.
Retry and error surface:
- `RetryState.expires_at_ms: Option<u64>` — optional absolute expiry
for a retry entry. `pump_retries` drops expired entries with an
info log. Used for open-discovery retries, which expire at two
times the advert TTL.
- New `NodeError::BootstrapHandoff(String)` returned from
`adopt_established_traversal` when the underlying transport
adoption fails or local address discovery fails.
- New `ConfigError::Validation(String)`.
- A small refactor extracts `Node::now_ms()` and reuses it across
lifecycle, rx-loop tick, and timeout bookkeeping.
== UDP transport
`src/transport/udp/`:
- `UdpRawSocket::adopt(std::net::UdpSocket, recv_buf, send_buf)`:
adopts an externally bound socket, makes it non-blocking, applies
the configured buffer sizes (warning if the kernel clamps), and
reports the resulting local address. Preserves the NAT mapping —
no rebind.
- `UdpTransport::adopt_socket_async(std::net::UdpSocket)`: the
`start_async` analogue for an already-bound socket, wiring the
async socket and recv task exactly as the fresh-bind path would.
- `Drop` impl for `UdpTransport`: if a transport is dropped while
still holding a recv task or socket (for example on error
teardown), aborts the task, clears the socket, and emits a debug
log so the cleanup is visible in tracing rather than silent.
== Logging and observability
Default `EnvFilter` demotes third-party relay-pool DEBUG output to
TRACE-only: `nostr_relay_pool`, `nostr_sdk`, and `nostr` are pinned
at INFO when our level is anything below TRACE, and at TRACE when
our level is TRACE — so the raw frames are still reachable when
explicitly asked for. RUST_LOG continues to override completely.
Concise one-line DEBUG events are emitted at the meaningful points
in the discovery / hole-punch sequence:
- `advert: published` (event id, relay count, endpoints, ttl)
- `advert: peer cached` (notify-loop ingress for non-self)
- `advert: resolved` (cache hit / relay fetch outcome)
- `traversal: initiator starting`
- `traversal: initiator STUN observed` (reflexive, local count)
- `traversal: offer sent` (session id, relay count, event id)
- `traversal: answer received` (accepted, reflexive, local)
- `traversal: initiator punch succeeded` (remote addr)
- `traversal: offer received` (responder side)
- `traversal: responder STUN observed`
- `traversal: answer sent`
- `traversal: responder punch succeeded`
Npubs are shortened to `npub1<4>..<4>` and event/session ids to
their first 8 hex characters.
Other operator-facing logs:
- `UdpTransport` adoption and drop paths log at info / debug.
- `adopt_established_traversal` logs at debug on entry and info on
successful return, tagged with peer npub, session id, transport
id, and both socket endpoints, so the bootstrap handoff is
traceable end-to-end alongside the `UdpTransport::drop` log.
- `cleanup_bootstrap_transport_if_unused` logs at debug when the
reference-count check drops an adopted transport.
- `connect_peer` tags its entry `debug!` with `peer_npub` so
downstream STUN, punch, and handshake logs for the same peer
correlate for operators.
- Advert-cache and seen-sessions overflow evictions log at debug so
mis-sized caps are visible under ambient relay volume.
- Gift-wrap unwrap failures on `SIGNAL_KIND` events log at trace
(hot path: fires for every unrelated signal event on the same
relay).
- Traversal-offer handler failures log at debug. Expected conditions
such as punch timeout on symmetric NAT are covered there; real
problems are reported upstream via `BootstrapEvent::Failed`.
- Inbound-offer rate-limit messages name the governing config field
(`max_concurrent_incoming_offers`) and state that the offer was
rate-limited rather than failing.
== Tests
- 18 new unit tests in `src/discovery/nostr/tests.rs` covering advert
encoding, signal envelope round-trip, STUN parsing, punch-packet
codec, and replay-window enforcement. Run under the
`nostr-discovery` feature.
- Config-validation tests in `src/config/mod.rs` covering the three
cross-field invariants and YAML parsing of the full
`node.discovery.nostr` block plus `peers[].via_nostr`, empty
`addresses` with `via_nostr: true`, and a `udp: nat` address.
- `src/node/tests/bootstrap.rs` integration tests that drive a
synthetic traversal (bound UDP socket pair + synthetic peer
identity) through `adopt_established_traversal` and assert the
Noise handshake completes over the adopted socket.
- Punch-planner tests assert reflexive-before-LAN ordering and that
same-LAN scenarios still include the LAN target in the plan.
- `testing/nat/` Docker NAT lab harness:
- Local `strfry` relay, local STUN responder, and one or two
router containers performing `iptables` NAT.
- Node LAN interfaces are provisioned with explicit `veth` pairs
injected into the node and router namespaces so every packet
traverses the router namespace (plain Docker bridges are not
used for the LAN).
- `cone` scenario: both peers behind full-cone-emulation NAT
(SNAT with source-port preservation, inbound DNAT back to the
single LAN host regardless of remote source); asserts UDP
traversal succeeds and link remote addresses are on the router
WAN subnet.
- `symmetric` scenario: `MASQUERADE --random-fully`; asserts UDP
traversal fails and TCP fallback converges over router-
published WAN addresses.
- `lan` scenario: both peers share a LAN subnet; asserts LAN
addresses are preferred over reflexive ones.
- Cleanup tears down all profile-gated services
(`--profile cone --profile symmetric --profile lan`) so no
orphan containers survive a run.
- `testing/scripts/build.sh` builds the Docker test image with
`--features "tui nostr-discovery"` by default so NAT-harness
binaries include bootstrap support.
== CI
- Linux release build and nextest unit-test job both use
`--features "gateway nostr-discovery"` so the feature-gated code
and its unit tests compile and run in CI.
- Three new integration matrix entries (`nat-cone`, `nat-symmetric`,
`nat-lan`) invoke `testing/nat/scripts/nat-test.sh`, collect
`docker compose logs` on failure, and always stop containers.
== Packaging and operations
- `packaging/common/fips.yaml` ships a fully commented
`node.discovery.nostr.*` block, plus documented
`advertise_on_nostr` / `public` examples under the UDP transport,
an `advertise_on_nostr` example under TCP, and a `via_nostr: true`
example under the static peer section with both a direct
`host:port` UDP address and a `udp: nat` fallback.
- `.github/workflows/package-openwrt.yml`: NIP-94 release event
publishes target the new default relay set.
== Documentation
- `README.md`: overlay discovery + NAT traversal moved from
"Near-term priorities" into "What works today".
- `docs/design/fips-intro.md`: rewrites the paragraphs that
previously described Nostr discovery and NAT traversal as future
work; describes the shipped mechanism and the feature gate.
- `docs/design/fips-transport-layer.md`: drops the "(future
direction)" qualifier from the Nostr Relay Discovery section,
expands with the `udp:nat` advertisement and bootstrap handoff
description, and updates the Current State callout.
- `docs/design/fips-mesh-layer.md`: notes that mid-session NAT
rebinding (roaming) and initial NAT traversal (Nostr path) are
distinct mechanisms.
- `docs/design/fips-configuration.md`: documents the full
`node.discovery.nostr.*` surface, including the three resource
caps and `share_local_candidates`.
- `docs/design/fips-nostr-discovery.md`: design and configuration
reference for the shipped mechanism, including the empty-
`addresses`-with-`via_nostr` shorthand.
- `docs/proposals/nostr-udp-hole-punch-protocol.md`: adds an
Implemented status callout, clarifies that the punch socket is
per-peer and per-attempt rather than shared with the application
listener, aligns field names with the shipped JSON
(`sessionId`, `issuedAt` / `expiresAt`, `reflexiveAddress`,
`localAddresses`, `stunServer`), sets the `d`-tag to
`fips-overlay-v1`, names the kind as 37195, and notes that
advertised STUN entries are informational.
- `docs/proposals/README.md`: adds a Status column and marks the
hole-punching proposal Implemented.
- `CHANGELOG.md`: Unreleased > Added entry covering the discovery
path, STUN/punch path, configuration surface, and Docker NAT lab.
Co-authored-by: Johnathan Corgan <johnathan@corganlabs.com>
|
||
|
Johnathan Corgan
|
7260ad2878 |
Improve README and enable DNS resolver by default
Rework top-level README: add badges, status/roadmap section, config search path docs, minimal two-node example with transports, DNS setup instructions with systemd-resolved and resolv.conf examples, and connectivity test walkthrough. Replace ASCII document-relationship diagram with SVG in design docs. Change DNS resolver default from disabled to enabled (port 5354). Update config merge to allow higher-priority configs to disable it. |
||
|
Johnathan Corgan
|
00e26765bd |
Design documentation illustration and review pass
Wire format diagrams: - Add 24 SVG diagrams covering every FMP and FSP wire format: common prefix, established frame headers, Noise IK handshake messages, handshake flow, TreeAnnounce, AncestryEntry, FilterAnnounce, LookupRequest/Response, SessionDatagram, Disconnect, SenderReport, ReceiverReport, FSP complete message, SessionSetup/Ack/Msg3, PathMtuNotification, CoordsRequired, PathBroken, and MtuExceeded - Replace ASCII art in fips-wire-formats.md with SVG references - Apply text edits to fips-mesh-layer.md, fips-mesh-operation.md, fips-transport-layer.md, and fips-ipv6-adapter.md Spanning tree dynamics: - Add 12 topology SVG diagrams: node join (overview + 3-panel steps), three-node convergence (4-panel), link addition with depth labels, link removal, partition formation, and 6 real-world example diagrams (office, mixed-link, two-site WAN topologies) - Rewrite all code blocks to narrative prose with diagram references - Add inline prior art attributions distinguishing Yggdrasil-derived concepts from FIPS-novel contributions - Add 3 new references (De Couto ETX, IEEE 802.1D, RFC 2328 OSPF) and Prior Art summary - Remove outdated sections: indirect partition note, integration test gaps, DHT-based lookup reference - Change "must elect a new root" to "must rediscover its new root" Spanning tree design review (fips-spanning-tree.md): - Rename "Root Election" to "Root Discovery" across docs - Add "What Is a Spanning Tree?" introductory section - Add parent selection intro explaining self-organization role - Fix tree distance example: 4 hops, not 2 - Clarify timestamp field as advisory only - Remove unimplemented ROOT_TIMEOUT and TREE_ENTRY_TTL from timing parameters and implementation status tables Bloom filter design review (fips-bloom-filters.md): - Add "What Is a Bloom Filter?" intro section - Rewrite Purpose section to frame filters as routing path identification - Correct FPR analysis (old values were 3-50x overstated) - Add Filter Occupancy Model based on network size and tree position - Fix filter expiration to describe actual MMP-based cleanup - Combine Scale Considerations with Size Classes after Wire Format - Fix stale FPR values in src/bloom/mod.rs comments Session layer review (fips-session-layer.md): - Add inline prior art attributions: Noise Protocol Framework, WireGuard, DTLS (RFC 6347), IKEv2 (RFC 7296), RFC 1191 PMTUD, Yggdrasil, NIP-44 - Replace warmup state machine ASCII art with SVG diagram - Convert CoordsWarmup wire format code block to prose - Add External References section with full citations Level 5 implementation doc cleanup: - Delete fips-software-architecture.md (redundant with protocol layer docs) - Delete fips-state-machines.md (Rust tutorial, not protocol design) - Add fipsctl command reference to README.md - Update cross-references in fips-intro.md, docs/design/README.md, fips-transport-layer.md, fips-configuration.md Fixes: - Correct fd::/8 to fd00::/8 in fips-session-layer.md, fips-identity-derivation.svg, and fips-node-architecture.svg - Fix config example MTU: 1197 → 1472 in fips-configuration.md File organization: - Move all SVG diagrams into docs/design/diagrams/ subdirectory - Update all diagram references to use new paths |
||
|
Johnathan Corgan
|
0a72317b59 |
Design documentation illustration pass and FLP→FMP rename
Rename FIPS Link Protocol (FLP) to FIPS Mesh Protocol (FMP)
The "Link Protocol" name understated the layer's scope — spanning tree
construction, bloom filter routing, greedy forwarding, and mesh-wide
coordination go well beyond link-level concerns. Rename fips-link-layer.md
to fips-mesh-layer.md, update FLP→FMP throughout docs and source code
(FLP_VERSION→FMP_VERSION, wire.rs, rx_loop.rs, spanning_tree.rs).
New SVG illustrations
- Protocol stack: color-coded layer diagram replacing ASCII art
- OSI mapping: side-by-side comparison with traditional networking layers
- Bloom filter propagation: 6-node tree with sender-colored filter boxes
showing split-horizon computation per link
- Routing decision flowchart: 5-step priority chain with candidate ranking
by tree distance and link performance
- Coordinate discovery: sequence diagram showing LookupRequest propagation,
response caching, and SessionSetup cache warming
Redesigned existing SVGs
- Architecture overview: uniform node layout, U-shaped encrypted link
connectors, separate end-to-end session line
- Node architecture: split Router Core into FSP and FMP layers, reorganize
transports into Overlay/Shared Medium/Point-to-Point categories
- Identity derivation: wider boxes, visible encode arrow, dashed npub line
fips-intro.md revisions
- Add inline references to prior work: Yggdrasil/Ironwood for coordinate
routing, Noise Protocol Framework for IK handshakes, WireGuard for
index-based session dispatch, Wikipedia for bloom filters, split-horizon,
and greedy embedding
- Add explanatory paragraphs after bloom filter diagram describing
split-horizon filter computation and candidate selection behavior
- Simplify transport abstraction language, remove I2P/LoRa references
- Fix LookupRequest wording ("propagates" not "floods"), note intermediate
node coordinate caching on lookup responses
- Rewrite architecture overview prose to match redesigned diagrams
|
||
|
Johnathan Corgan
|
d46dc874ef |
Restructure design docs around protocol layers
Reorganize FIPS design documentation from implementation-centric structure (routing, gossip protocol, wire protocol, transports) to protocol-layer organization with clear service boundaries. New documents (8): - fips-transport-layer.md — transport layer spec - fips-link-layer.md — FLP spec (peer auth, link encryption, forwarding) - fips-session-layer.md — FSP spec (end-to-end encryption, sessions) - fips-ipv6-adapter.md — IPv6 adaptation (TUN, DNS, MTU enforcement) - fips-mesh-operation.md — routing, discovery, error recovery - fips-wire-formats.md — consolidated wire format reference - fips-spanning-tree.md — tree algorithm reference - fips-bloom-filters.md — bloom filter math reference Rewritten (2): - fips-intro.md — breadth-first intro with layer model diagrams - fips-software-architecture.md — slimmed to stable decisions Updated (3): - spanning-tree-dynamics.md — removed stale root refresh, aligned terminology - fips-configuration.md — fixed priority type (u16 → u8) - fips-state-machines.md — synced code examples with codebase Deleted (6): fips-transports.md, fips-wire-protocol.md, fips-gossip-protocol.md, fips-session-protocol.md, fips-routing.md, fips-tun-driver.md (content absorbed into new structure) |
||
|
Johnathan Corgan
|
73129f16d7 |
Rename design docs for clarity, reorganize node architecture diagram
- fips-architecture.md → fips-software-architecture.md with all cross-references updated (4 files) - fips-transport-abstraction.svg → fips-node-architecture.svg, moved from Transport Abstraction section to Architecture Overview in fips-intro.md - Added descriptive paragraph for node architecture diagram covering three-layer design (application interfaces, router core, transports) |
||
|
Johnathan Corgan
|
a8115f7622 |
Unify Noise IK for both link and session layers
Design change: Session layer now uses Noise IK pattern (previously KK), matching the link layer. Rationale: initiator knows destination npub, responder learns initiator identity from handshake - same asymmetry as link connections. Document consistency fixes: - Auth* events → Noise IK msg1/msg2 terminology in state machines - PeerId → NodeId in routing code examples - BloomUpdate → FilterAnnounce terminology - Updated cross-references and tables |
||
|
Johnathan Corgan
|
e582f50de7 |
Session 51: Create fips-intro.md protocol introduction
New comprehensive protocol introduction replacing fips-design.md: - What is FIPS / Why FIPS: goals and design philosophy - How It Works: transport, spanning tree, bloom filters, routing overview - Prior Work: Yggdrasil/Ironwood, Noise Protocol, WireGuard references - Identity System: npub/nsec, node_id derivation, fd00::/8 addressing - Two-Layer Encryption: link layer (Noise IK), session layer (Noise KK) - Spanning Tree Protocol: root election, parent selection, gossip limits - Bloom Filter Routing: filter explanation, propagation, discovery - Transport Abstraction: transport/link distinction, bridging, types - Security: threat model, Sybil resistance, accurate metadata exposure Updated cross-references in README.md and other design docs. Deleted obsolete fips-design.md. |
||
|
Johnathan Corgan
|
7c2e11c5de |
Session 50: Design document reorganization and gossip protocol
- Create fips-gossip-protocol.md with wire formats for TreeAnnounce, FilterAnnounce, LookupRequest/LookupResponse - Refactor fips-routing.md to reference gossip protocol for wire formats - Update fips-session-protocol.md: remove reconciliation section, condense peer connection section - Update spanning-tree-dynamics.md with gossip protocol reference - Reorganize README.md with suggested reading order - Remove obsolete fips-architecture-review.md |
||
|
Johnathan Corgan
|
4890940bb7 |
Session 49: Wire protocol design with session indices
- Add fips-wire-protocol.md: comprehensive packet dispatch design - Wire format: discriminator byte + session indices for O(1) dispatch - WireGuard-style roaming: crypto authority, not address - Security: rate limiting, replay protection, state machine strictness - Transport considerations for UDP, TCP, Tor - Rename fips-protocol-flow.md → fips-session-protocol.md - Update fips-design.md wire format section - Replace TLV with discriminator + index format - Cross-reference fips-wire-protocol.md for details - Update cross-references in all design docs |
||
|
Johnathan Corgan
|
a5a62b3768 |
Session 43: CLI config option and state machine design
Add command-line argument parsing with clap: - -c/--config option to specify config file path - Overrides default search path when provided - Proper error handling for missing/invalid files Improve logging: - Peer connection log now uses separate log entries per field - Better readability with aligned timestamps Fix ICMPv6 error handling: - Add multicast destination filter to should_send_icmp_error() - Router Solicitation packets (ff02::2) now silently dropped - Add test case for multicast destination Add phase-based state machine design document: - Document pattern where lifecycle phases use distinct structs in enum - PeerSlot::Connecting(PeerConnection) -> PeerSlot::Active(ActivePeer) - Benefits: type safety, memory efficiency, security - Describes timeout handling and lookup table requirements |
||
|
Johnathan Corgan
|
ca24ba02b7 |
Add TUN driver design document
- docs/design/fips-tun-driver.md: comprehensive TUN interface documentation - Architecture diagrams, component descriptions, packet flow - Implementation status checklist (completed vs planned) - Testing instructions and configuration reference |
||
|
Johnathan Corgan
|
3b6a4da17d |
Complete FIPS architecture review cleanup
- Rename fips-links.md to fips-transports.md, update all references - Update README with all 6 design documents in organized sections - Reorganize architecture review: remove verbose resolved items, consolidate deferred items, focus on actionable issues - Clarify Transport/Link/Peer lifecycle in architecture doc: - Transports static after startup - Links on-demand, driven by peer lifecycle - Connectionless transports (UDP) immediate established - Connection-oriented (Tor) require link setup before auth - Add Resource Limits configuration section (max_peers, max_transports, max_pending_auth, max_pending_lookups, memory_budget) - Close timer management as non-issue (tokio handles scale) - Defer init/shutdown to future iteration Review status: 6 resolved, 5 deferred, 4 low-priority open |
||
|
Johnathan Corgan
|
dc0acf32ea |
Add FIPS routing design document
Create fips-routing.md covering the complete routing architecture: - Bloom filter design: 4KB filters, K=2 scope, event-driven updates - Discovery protocol: LookupRequest/Response with signed proofs - Greedy tree routing using coordinates from discovery - Session establishment model for minimal data packet overhead - Router coordinate caching with LRU eviction Key design decisions: - Leaf-only mode for constrained devices (single peer handles routing) - Separation of discovery (find destination) from routing (deliver packets) - Session setup pays coordinate cost once; data packets carry only addresses - 36-byte data packet header comparable to IPv6 Update design docs README to include new document. |
||
|
Johnathan Corgan
|
d8cba510a7 |
Add FIPS link protocol design document
New docs/design/fips-links.md covering: - Link terminology (L2/L2-equivalent, not "transport") - Link categories: overlay, shared medium, point-to-point - Characteristics table: encapsulation, addressing, MTU, latency, reliability, bandwidth, discovery - TCP-over-TCP problem and UDP preference rationale - NAT/firewall traversal requirements - IPv6 interface exposure to applications Update README.md with new document entry. |
||
|
Johnathan Corgan
|
7194200f55 |
Add docs/design directory with protocol specifications
Move design documents from working directory into source tree: - fips-design.md - Full protocol design specification - spanning-tree-dynamics.md - Spanning tree protocol dynamics study Add README index files for docs/ and docs/design/. |