530 lines
21 KiB
Markdown
530 lines
21 KiB
Markdown
# How FIPS Works — A Freshman-Level Explanation
|
|
|
|
FIPS (Freedom Internet Protocol Stack) is a **mesh network** — a way for
|
|
computers to talk to each other directly, without going through a central
|
|
server like Google or your ISP. Instead of connecting to a website through
|
|
a data center, your computer connects to a few nearby computers, which
|
|
connect to other computers, which connect to other computers, and so on.
|
|
Messages hop from computer to computer until they reach their destination.
|
|
|
|
Think of it like passing a note in class: you hand it to the person next
|
|
to you, they hand it to the next person, and eventually it reaches the
|
|
person it's addressed to. Except in FIPS, the "note" is encrypted, the
|
|
"people" are computers running the FIPS software, and the "classroom" is
|
|
the entire internet.
|
|
|
|
This document explains how FIPS does four things:
|
|
|
|
1. **Discovery** — how nodes find each other
|
|
2. **Connection** — how nodes establish secure links
|
|
3. **Topology** — how nodes organize themselves into a network
|
|
4. **Routing** — how messages get from A to B through the mesh
|
|
|
|
---
|
|
|
|
## The Big Picture
|
|
|
|
```
|
|
THE INTERNET
|
|
(the underlay)
|
|
|
|
|
+--------------+--------------+
|
|
| | |
|
|
+--------+ +--------+ +--------+
|
|
| Node A |-----| Node B |-----| Node C |
|
|
+--------+ +--------+ +--------+
|
|
| | |
|
|
| | |
|
|
+--------+ +--------+ +--------+
|
|
| Node D |-----| Node E | | Node F |
|
|
+--------+ +--------+ +--------+
|
|
|
|
|
+--------+
|
|
| Node G |
|
|
+--------+
|
|
|
|
Each line = a direct encrypted connection (a "link")
|
|
Each box = a computer running fipsd (the FIPS daemon)
|
|
```
|
|
|
|
Every node in FIPS runs a program called **`fipsd`** (the FIPS daemon).
|
|
It runs in the background, maintains connections to a few other nodes,
|
|
and routes messages through the mesh. Each node has a unique identity
|
|
(a cryptographic keypair) and a unique address on the mesh.
|
|
|
|
---
|
|
|
|
## 1. Discovery: How Nodes Find Each Other
|
|
|
|
The first problem: how does a new node, starting from nothing, find
|
|
other FIPS nodes to connect to? You can't route through the mesh yet
|
|
because you're not in it.
|
|
|
|
### The Nostr Rendezvous
|
|
|
|
FIPS uses **Nostr** (a decentralized social protocol) as a bulletin
|
|
board where nodes post "hello, I'm here" messages. Nostr itself is just
|
|
a network of relay servers that store and forward events — think of them
|
|
as public message boards.
|
|
|
|
```
|
|
+----------+ "I'm Node A, +----------+
|
|
| Node A |-----> reach me at | Nostr |
|
|
| (new) | 1.2.3.4:2121" ----->| Relays |
|
|
+----------+ (Kind 37195 event) +----------+
|
|
|
|
|
+---------------+
|
|
| (stores the
|
|
| advert for
|
|
| ~1 hour)
|
|
|
|
|
+---------------+----------+
|
|
| |
|
|
v v
|
|
+----------+ +----------+
|
|
| Node B | | Node C |
|
|
| (reads | | (reads |
|
|
| adverts)| | adverts)|
|
|
+----------+ +----------+
|
|
```
|
|
|
|
Here's what happens:
|
|
|
|
1. **Node A publishes an advert.** Its `fipsd` creates a Nostr event
|
|
(kind 37195) containing:
|
|
- Its identity (public key / npub)
|
|
- Its endpoint address (e.g., `1.2.3.4:2121` for a public IP, or
|
|
`nat` if behind a firewall)
|
|
- Its signal relays (which Nostr relays to use for hole-punching)
|
|
- Its STUN servers (for NAT traversal)
|
|
- An expiration time (~1 hour)
|
|
|
|
2. **The Nostr relays store it.** Three relays by default:
|
|
`wss://relay.damus.io`, `wss://nos.lol`, `wss://offchain.pub`.
|
|
They keep the advert until it expires (~1 hour), then delete it.
|
|
|
|
3. **Node A republishes every ~30 minutes.** This keeps the advert
|
|
fresh. If Node A goes offline, its advert expires within an hour
|
|
and disappears from the relays.
|
|
|
|
4. **Other nodes read the adverts.** Their `fipsd` subscribes to the
|
|
same relays and receives every advert in real time. They cache them
|
|
in memory and learn "Node A exists, and here's how to reach it."
|
|
|
|
### The Two Cases: Public IP vs. NAT
|
|
|
|
**If Node A has a public IP** (e.g., a VPS in a data center):
|
|
- Its advert says `endpoints: [{transport: "udp", addr: "1.2.3.4:2121"}]`
|
|
- Other nodes can just send UDP packets directly to that address
|
|
- Simple — no extra steps needed
|
|
|
|
**If Node A is behind NAT** (e.g., a home computer behind a router):
|
|
- Its advert says `endpoints: [{transport: "udp", addr: "nat"}]`
|
|
- "nat" means "I can't be dialed directly — you need to help me punch
|
|
a hole through my firewall"
|
|
- This requires a **NAT traversal** dance (explained below)
|
|
|
|
### NAT Traversal (Hole-Punching)
|
|
|
|
When Node B wants to connect to Node A, but Node A is behind NAT:
|
|
|
|
```
|
|
Node B Nostr Relays Node A
|
|
(behind NAT too, (the matchmaker) (behind NAT)
|
|
or maybe not)
|
|
| | |
|
|
| 1. "Hey Node A, | |
|
|
| I want to connect" | |
|
|
|----- (encrypted offer -->| |
|
|
| via Nostr DM) |-------> offer to A ----->|
|
|
| | |
|
|
| |<----- answer from A -----|
|
|
|<-- answer (encrypted) <--| 2. "OK, let's punch" |
|
|
| | |
|
|
| 3. Both ask STUN: | 3. Both ask STUN:
|
|
| "What's my public IP?" | "What's my public IP?"
|
|
|---------> STUN server | -------> STUN server
|
|
|<--------- "You are | <------- "You are
|
|
| 5.6.7.8:9999" | 9.10.11.12:8888"
|
|
| | |
|
|
| 4. Node B sends UDP | 4. Node A sends UDP |
|
|
| to 9.10.11.12:8888 | to 5.6.7.8:9999 |
|
|
|-----> (hole punch!) ---->|-----> (hole punch!) ---->|
|
|
| | |
|
|
| 5. The NAT firewalls | 5. The NAT firewalls |
|
|
| see outbound traffic | see outbound traffic|
|
|
| and allow replies | and allow replies |
|
|
| | |
|
|
| <======== UDP tunnel now open =======> |
|
|
| (encrypted with Noise IK) |
|
|
```
|
|
|
|
The key insight: both nodes send UDP packets to each other's public
|
|
addresses (discovered via STUN). When Node B sends a packet to Node A's
|
|
NAT, the NAT sees it as a reply to Node A's outbound packet and lets it
|
|
through. The Nostr relays act as the "matchmaker" that lets them
|
|
exchange public addresses before the hole-punch.
|
|
|
|
Once the UDP tunnel is open, the **Noise IK handshake** runs over it to
|
|
authenticate both sides and establish encryption keys. After that, the
|
|
link is a normal encrypted peer connection — Nostr is no longer needed.
|
|
|
|
---
|
|
|
|
## 2. Connection: How Nodes Establish Secure Links
|
|
|
|
Every direct connection between two FIPS nodes is encrypted using the
|
|
**Noise Protocol Framework** (specifically the Noise IK pattern, the
|
|
same crypto family used by WireGuard and Signal).
|
|
|
|
```
|
|
Node A Node B
|
|
(initiator) (responder)
|
|
| |
|
|
| msg1: A's ephemeral public key |
|
|
| (encrypted with B's static public key) |
|
|
|--------------------------------------------->|
|
|
| |
|
|
| msg2: B's ephemeral public key
|
|
| + B's static public key (encrypted)
|
|
|<---------------------------------------------|
|
|
| |
|
|
| Both sides now have: |
|
|
| - Shared secret (via Diffie-Hellman) |
|
|
| - Each other's identity (public key) |
|
|
| - Symmetric encryption keys |
|
|
| |
|
|
| <=== Encrypted link established ===> |
|
|
| All future traffic is encrypted and |
|
|
| authenticated |
|
|
```
|
|
|
|
The important properties:
|
|
- **Authentication**: both sides prove they own their private keys
|
|
(which match their npub / Nostr identity)
|
|
- **Encryption**: all traffic is encrypted with modern symmetric ciphers
|
|
- **Forward secrecy**: even if a key is compromised later, past traffic
|
|
stays secure
|
|
- **Identity binding**: you know exactly who you're talking to — no
|
|
man-in-the-middle possible if you trust the Nostr advert's pubkey
|
|
|
|
Once this handshake completes, the two nodes are **peers** — they have
|
|
an authenticated, encrypted link they can send mesh traffic over.
|
|
|
|
---
|
|
|
|
## 3. Topology: How Nodes Organize Into a Network
|
|
|
|
Once a node has a few peer connections, it needs to know the shape of
|
|
the entire mesh. Not just "who am I connected to" but "who is everyone
|
|
else, and how do I reach them?"
|
|
|
|
FIPS builds a **spanning tree** — a logical structure that connects all
|
|
nodes without loops, like a family tree or an org chart.
|
|
|
|
### The Spanning Tree
|
|
|
|
```
|
|
[Root]
|
|
Node 00001a...
|
|
(smallest NodeAddr
|
|
in the network)
|
|
/ \
|
|
/ \
|
|
[depth 1] [depth 1]
|
|
Node B Node C
|
|
/ \ |
|
|
/ \ |
|
|
[depth 2] [depth 2] [depth 2]
|
|
Node D Node E Node F
|
|
|
|
|
[depth 3]
|
|
Node G (us)
|
|
Node H
|
|
|
|
Each node has exactly one PARENT (closer to root)
|
|
A node may have multiple CHILDREN (farther from root)
|
|
The root is the node with the smallest NodeAddr
|
|
(a 16-byte hash derived from its public key)
|
|
```
|
|
|
|
**How the tree is built:**
|
|
|
|
1. Every node computes its **NodeAddr** = first 16 bytes of
|
|
SHA-256(public key). This is a random-looking number that's unique
|
|
to each node.
|
|
|
|
2. The node with the **smallest NodeAddr** is the **root**. This is
|
|
deterministic — everyone agrees on who the root is without any
|
|
election or voting.
|
|
|
|
3. Each node picks its **parent** = the peer that's closest to the root
|
|
(shallowest depth). If you're not the root, your parent is someone
|
|
who's one step closer to the root than you.
|
|
|
|
4. Nodes announce their tree position to their peers using
|
|
**TreeAnnounce** messages: "I am Node G, my parent is Node D, my
|
|
path to root is [G → D → B → Root]."
|
|
|
|
5. When a node receives a TreeAnnounce, it may discover a better parent
|
|
(one with shallower depth) and switch. This is how the tree
|
|
self-organizes and heals when nodes join/leave.
|
|
|
|
```
|
|
TreeAnnounce flow (direct peer-to-peer, NOT via Nostr):
|
|
|
|
Node G --"my parent is D, depth 3"--> Node D (its parent)
|
|
Node D --"my parent is B, depth 2"--> Node B (its parent)
|
|
Node B --"my parent is Root, depth 1"--> Root
|
|
|
|
And in the other direction (to children):
|
|
Node D --"I'm at depth 2, parent B"--> Node G (its child)
|
|
Node B --"I'm at depth 1, parent Root"--> Node D (its child)
|
|
```
|
|
|
|
TreeAnnounce messages are sent:
|
|
- On initial connection (after handshake)
|
|
- When a node's parent changes
|
|
- Periodically (every ~60 seconds) as a refresh
|
|
- Rate-limited to 1 per 500ms per peer to prevent storms
|
|
|
|
### Why a Spanning Tree?
|
|
|
|
The tree gives every node a **path to every other node**: go up to your
|
|
common ancestor, then down to the destination. It also prevents routing
|
|
loops (there's exactly one path between any two nodes in a tree).
|
|
|
|
But the tree alone isn't enough for efficient routing — going all the
|
|
way up to the root and back down is slow for distant nodes. That's where
|
|
bloom filters come in (next section).
|
|
|
|
---
|
|
|
|
## 4. Routing: How Messages Get From A to B
|
|
|
|
### The Problem
|
|
|
|
Node G wants to send a message to Node F. G is only connected to Node D.
|
|
How does the message get to F?
|
|
|
|
```
|
|
G wants to send to F:
|
|
|
|
Root
|
|
/ \
|
|
B C
|
|
/ \ |
|
|
D E F <-- destination
|
|
|
|
|
G <-- source
|
|
H
|
|
```
|
|
|
|
The naive approach: send the message to G's parent (D), D sends to its
|
|
parent (B), B sends to Root, Root sends to C, C sends to F. That's 5
|
|
hops — and it forces all traffic through the root, which doesn't scale.
|
|
|
|
FIPS does better using **bloom filters**.
|
|
|
|
### Bloom Filters: Who Can Reach Whom?
|
|
|
|
A **bloom filter** is a compact data structure that can answer "is X in
|
|
this set?" without storing the whole set. It's probabilistic — it can
|
|
say "definitely not" or "maybe yes" (with a small false-positive rate).
|
|
A bloom filter for 500 nodes is about 1 KB.
|
|
|
|
Every FIPS node maintains a bloom filter of **which nodes are reachable
|
|
through it** (itself + everything in its subtree). It sends this filter
|
|
to its direct peers via **FilterAnnounce** messages.
|
|
|
|
```
|
|
Each node tells its peers: "here's who I can reach"
|
|
|
|
Node G's filter: {G, H} (just itself + children)
|
|
Node D's filter: {D, G, H} (itself + its subtree)
|
|
Node B's filter: {B, D, E, G, H} (itself + its subtree)
|
|
Node C's filter: {C, F} (itself + its subtree)
|
|
Root's filter: {everyone} (it's the root, sees all)
|
|
|
|
Filters propagate UP the tree and are shared with all peers:
|
|
Node D tells B: "I can reach {D, G, H}"
|
|
Node B tells Root: "I can reach {B, D, E, G, H}"
|
|
Node C tells Root: "I can reach {C, F}"
|
|
```
|
|
|
|
Now every node knows, for each of its direct peers, a compact summary of
|
|
who's reachable through that peer. When Node G needs to send to Node F:
|
|
|
|
```
|
|
G wants to send to F:
|
|
|
|
G checks its peers' bloom filters:
|
|
- Peer D's filter contains F? No (D can reach {D,G,H})
|
|
- (G only has one peer, so it sends to D anyway)
|
|
|
|
G --> D: "forward this to F"
|
|
|
|
D checks its peers' bloom filters:
|
|
- Peer B's filter contains F? No (B can reach {B,D,E,G,H})
|
|
- (D only has one peer besides G, so it sends to B)
|
|
|
|
D --> B: "forward this to F"
|
|
|
|
B checks its peers' bloom filters:
|
|
- Peer Root's filter contains F? Yes! (Root can reach everyone)
|
|
- Peer E's filter contains F? No
|
|
|
|
B --> Root: "forward this to F"
|
|
|
|
Root checks its peers' bloom filters:
|
|
- Peer B's filter contains F? No
|
|
- Peer C's filter contains F? Yes!
|
|
|
|
Root --> C: "forward this to F"
|
|
|
|
C --> F: "here's your message"
|
|
```
|
|
|
|
The bloom filters let each node make an **informed decision** about
|
|
which peer to forward to, instead of blindly sending everything to its
|
|
parent. This keeps traffic off the root when possible and makes routing
|
|
efficient.
|
|
|
|
### Coordinate Discovery: Finding the Exact Path
|
|
|
|
Bloom filters tell you *which peer* to forward to, but not the exact
|
|
path. For that, FIPS uses **coordinate discovery**:
|
|
|
|
```
|
|
G wants to send to F (first time — no cached coordinates):
|
|
|
|
1. G sends a LookupRequest: "Where is F?"
|
|
(flooded through the tree, guided by bloom filters —
|
|
only forwarded to peers whose filter contains F)
|
|
|
|
G --> D --> B --> Root --> C --> F
|
|
|
|
2. F responds with a LookupResponse: "I'm here, my coordinates are
|
|
[F, C, Root, B, D, G]"
|
|
(sent back along the reverse path)
|
|
|
|
F --> C --> Root --> B --> D --> G
|
|
|
|
3. G caches F's coordinates in its coord cache.
|
|
Next time G sends to F, it knows the path and can route directly
|
|
without another lookup.
|
|
```
|
|
|
|
The coordinate cache is like a phone book — once you've looked up
|
|
someone's number, you don't need to look it up again. Entries expire
|
|
after 5 minutes (default), so if the network topology changes, stale
|
|
routes get refreshed.
|
|
|
|
### Putting It All Together: A Message's Journey
|
|
|
|
```
|
|
Node G sends a web request to Node F's .fips address:
|
|
|
|
1. G's application: "GET http://npub1...fxf.fips/index.html"
|
|
|
|
2. G's fipsd:
|
|
- Derives F's mesh IPv6 address from its npub (fd00::...)
|
|
- Checks coord cache: do I have F's coordinates?
|
|
- Yes: route directly using cached path
|
|
- No: send LookupRequest, wait for response, cache it
|
|
|
|
3. G's fipsd wraps the HTTP request in an encrypted session
|
|
datagram and sends it hop-by-hop:
|
|
|
|
G --[encrypted link]--> D --[encrypted link]--> B
|
|
B --[encrypted link]--> Root --[encrypted link]--> C
|
|
C --[encrypted link]--> F
|
|
|
|
Each hop:
|
|
- Decrypts the outer layer
|
|
- Reads the destination coordinates
|
|
- Checks bloom filters to pick the next hop
|
|
- Re-encrypts and forwards
|
|
|
|
4. F's fipsd receives the datagram, decrypts the session layer,
|
|
and delivers the HTTP request to F's web server.
|
|
|
|
5. F's web server responds, and the response goes back the same
|
|
way (or a better path if one was discovered).
|
|
```
|
|
|
|
Every hop is individually encrypted (the link layer), and the
|
|
end-to-end session is also encrypted (the session layer). So even
|
|
intermediate nodes can't read your traffic — they can only see "this
|
|
packet is going to Node F, forward it to peer X."
|
|
|
|
---
|
|
|
|
## The .fips DNS Hack
|
|
|
|
FIPS gives every node a virtual IPv6 address in the `fd00::/8` range,
|
|
derived from its public key. These addresses only exist **inside the
|
|
mesh** — they're not real internet addresses.
|
|
|
|
To make this user-friendly, `fipsd` runs a local DNS resolver that
|
|
maps `<npub>.fips` to the node's mesh IPv6 address:
|
|
|
|
```
|
|
You type: curl http://npub1abc123...xyz.fips/index.html
|
|
|
|
1. Your browser asks DNS: "what's the IP for npub1abc...xyz.fips?"
|
|
2. Your fipsd's DNS resolver (127.0.0.1:5354) answers:
|
|
"It's fd5e:4c4:ce59:a5a4:829:46c7:2b33:ab21"
|
|
3. Your browser connects to that IPv6 address
|
|
4. The kernel routes it through fips0 (the TUN device)
|
|
5. fipsd reads it from fips0, routes it through the mesh
|
|
6. The message arrives at the destination node
|
|
```
|
|
|
|
This is why you can type `http://npub1....fips` in a browser and it
|
|
just works — the DNS + TUN + mesh routing make it transparent.
|
|
|
|
---
|
|
|
|
## Summary: The Four Layers
|
|
|
|
```
|
|
+-------------------------------------------------------------------+
|
|
| THE FIPS MESH |
|
|
| |
|
|
| +-------------------------------------------------------------+ |
|
|
| | Layer 4: Application | |
|
|
| | HTTP, SSH, any TCP/UDP app via .fips addresses | |
|
|
| +-------------------------------------------------------------+ |
|
|
| | Layer 3: Routing | |
|
|
| | Bloom filters + coordinate discovery + spanning tree | |
|
|
| | "Which peer do I forward to?" | |
|
|
| +-------------------------------------------------------------+ |
|
|
| | Layer 2: Links | |
|
|
| | Noise IK encrypted peer-to-peer connections | |
|
|
| | "I'm securely connected to nodes B, D, and E" | |
|
|
| +-------------------------------------------------------------+ |
|
|
| | Layer 1: Discovery | |
|
|
| | Nostr Kind 37195 adverts + NAT traversal | |
|
|
| | "How do I find other nodes to connect to?" | |
|
|
| +-------------------------------------------------------------+ |
|
|
| |
|
|
+-------------------------------------------------------------------+
|
|
|
|
|
THE INTERNET
|
|
(the underlay)
|
|
```
|
|
|
|
| Layer | What it does | How | Nostr? |
|
|
|-------|-------------|-----|--------|
|
|
| **Discovery** | Find other nodes | Nostr adverts + STUN hole-punch | Yes (bootstrap only) |
|
|
| **Links** | Secure connections | Noise IK handshake over UDP/TCP/Tor | No |
|
|
| **Routing** | Route through mesh | TreeAnnounce + FilterAnnounce + LookupRequest | No |
|
|
| **Application** | Use the mesh | .fips DNS + TUN device + any TCP/UDP app | No |
|
|
|
|
The key insight: **Nostr is only used for the initial rendezvous.**
|
|
Once nodes are connected, everything else — topology, routing, data
|
|
transfer — happens directly between peers over encrypted links. The
|
|
mesh is a self-organizing overlay network that uses Nostr as its
|
|
"phone book" and the internet as its "wires."
|