21 KiB
How FIPS Works — A Freshman-Level Explanation
FIPS (Freedom Internet Protocol Stack) is a mesh network — a way for computers to talk to each other directly, without going through a central server like Google or your ISP. Instead of connecting to a website through a data center, your computer connects to a few nearby computers, which connect to other computers, which connect to other computers, and so on. Messages hop from computer to computer until they reach their destination.
Think of it like passing a note in class: you hand it to the person next to you, they hand it to the next person, and eventually it reaches the person it's addressed to. Except in FIPS, the "note" is encrypted, the "people" are computers running the FIPS software, and the "classroom" is the entire internet.
This document explains how FIPS does four things:
- Discovery — how nodes find each other
- Connection — how nodes establish secure links
- Topology — how nodes organize themselves into a network
- Routing — how messages get from A to B through the mesh
The Big Picture
THE INTERNET
(the underlay)
|
+--------------+--------------+
| | |
+--------+ +--------+ +--------+
| Node A |-----| Node B |-----| Node C |
+--------+ +--------+ +--------+
| | |
| | |
+--------+ +--------+ +--------+
| Node D |-----| Node E | | Node F |
+--------+ +--------+ +--------+
|
+--------+
| Node G |
+--------+
Each line = a direct encrypted connection (a "link")
Each box = a computer running fipsd (the FIPS daemon)
Every node in FIPS runs a program called fipsd (the FIPS daemon).
It runs in the background, maintains connections to a few other nodes,
and routes messages through the mesh. Each node has a unique identity
(a cryptographic keypair) and a unique address on the mesh.
1. Discovery: How Nodes Find Each Other
The first problem: how does a new node, starting from nothing, find other FIPS nodes to connect to? You can't route through the mesh yet because you're not in it.
The Nostr Rendezvous
FIPS uses Nostr (a decentralized social protocol) as a bulletin board where nodes post "hello, I'm here" messages. Nostr itself is just a network of relay servers that store and forward events — think of them as public message boards.
+----------+ "I'm Node A, +----------+
| Node A |-----> reach me at | Nostr |
| (new) | 1.2.3.4:2121" ----->| Relays |
+----------+ (Kind 37195 event) +----------+
|
+---------------+
| (stores the
| advert for
| ~1 hour)
|
+---------------+----------+
| |
v v
+----------+ +----------+
| Node B | | Node C |
| (reads | | (reads |
| adverts)| | adverts)|
+----------+ +----------+
Here's what happens:
-
Node A publishes an advert. Its
fipsdcreates a Nostr event (kind 37195) containing:- Its identity (public key / npub)
- Its endpoint address (e.g.,
1.2.3.4:2121for a public IP, ornatif behind a firewall) - Its signal relays (which Nostr relays to use for hole-punching)
- Its STUN servers (for NAT traversal)
- An expiration time (~1 hour)
-
The Nostr relays store it. Three relays by default:
wss://relay.damus.io,wss://nos.lol,wss://offchain.pub. They keep the advert until it expires (~1 hour), then delete it. -
Node A republishes every ~30 minutes. This keeps the advert fresh. If Node A goes offline, its advert expires within an hour and disappears from the relays.
-
Other nodes read the adverts. Their
fipsdsubscribes to the same relays and receives every advert in real time. They cache them in memory and learn "Node A exists, and here's how to reach it."
The Two Cases: Public IP vs. NAT
If Node A has a public IP (e.g., a VPS in a data center):
- Its advert says
endpoints: [{transport: "udp", addr: "1.2.3.4:2121"}] - Other nodes can just send UDP packets directly to that address
- Simple — no extra steps needed
If Node A is behind NAT (e.g., a home computer behind a router):
- Its advert says
endpoints: [{transport: "udp", addr: "nat"}] - "nat" means "I can't be dialed directly — you need to help me punch a hole through my firewall"
- This requires a NAT traversal dance (explained below)
NAT Traversal (Hole-Punching)
When Node B wants to connect to Node A, but Node A is behind NAT:
Node B Nostr Relays Node A
(behind NAT too, (the matchmaker) (behind NAT)
or maybe not)
| | |
| 1. "Hey Node A, | |
| I want to connect" | |
|----- (encrypted offer -->| |
| via Nostr DM) |-------> offer to A ----->|
| | |
| |<----- answer from A -----|
|<-- answer (encrypted) <--| 2. "OK, let's punch" |
| | |
| 3. Both ask STUN: | 3. Both ask STUN:
| "What's my public IP?" | "What's my public IP?"
|---------> STUN server | -------> STUN server
|<--------- "You are | <------- "You are
| 5.6.7.8:9999" | 9.10.11.12:8888"
| | |
| 4. Node B sends UDP | 4. Node A sends UDP |
| to 9.10.11.12:8888 | to 5.6.7.8:9999 |
|-----> (hole punch!) ---->|-----> (hole punch!) ---->|
| | |
| 5. The NAT firewalls | 5. The NAT firewalls |
| see outbound traffic | see outbound traffic|
| and allow replies | and allow replies |
| | |
| <======== UDP tunnel now open =======> |
| (encrypted with Noise IK) |
The key insight: both nodes send UDP packets to each other's public addresses (discovered via STUN). When Node B sends a packet to Node A's NAT, the NAT sees it as a reply to Node A's outbound packet and lets it through. The Nostr relays act as the "matchmaker" that lets them exchange public addresses before the hole-punch.
Once the UDP tunnel is open, the Noise IK handshake runs over it to authenticate both sides and establish encryption keys. After that, the link is a normal encrypted peer connection — Nostr is no longer needed.
2. Connection: How Nodes Establish Secure Links
Every direct connection between two FIPS nodes is encrypted using the Noise Protocol Framework (specifically the Noise IK pattern, the same crypto family used by WireGuard and Signal).
Node A Node B
(initiator) (responder)
| |
| msg1: A's ephemeral public key |
| (encrypted with B's static public key) |
|--------------------------------------------->|
| |
| msg2: B's ephemeral public key
| + B's static public key (encrypted)
|<---------------------------------------------|
| |
| Both sides now have: |
| - Shared secret (via Diffie-Hellman) |
| - Each other's identity (public key) |
| - Symmetric encryption keys |
| |
| <=== Encrypted link established ===> |
| All future traffic is encrypted and |
| authenticated |
The important properties:
- Authentication: both sides prove they own their private keys (which match their npub / Nostr identity)
- Encryption: all traffic is encrypted with modern symmetric ciphers
- Forward secrecy: even if a key is compromised later, past traffic stays secure
- Identity binding: you know exactly who you're talking to — no man-in-the-middle possible if you trust the Nostr advert's pubkey
Once this handshake completes, the two nodes are peers — they have an authenticated, encrypted link they can send mesh traffic over.
3. Topology: How Nodes Organize Into a Network
Once a node has a few peer connections, it needs to know the shape of the entire mesh. Not just "who am I connected to" but "who is everyone else, and how do I reach them?"
FIPS builds a spanning tree — a logical structure that connects all nodes without loops, like a family tree or an org chart.
The Spanning Tree
[Root]
Node 00001a...
(smallest NodeAddr
in the network)
/ \
/ \
[depth 1] [depth 1]
Node B Node C
/ \ |
/ \ |
[depth 2] [depth 2] [depth 2]
Node D Node E Node F
|
[depth 3]
Node G (us)
Node H
Each node has exactly one PARENT (closer to root)
A node may have multiple CHILDREN (farther from root)
The root is the node with the smallest NodeAddr
(a 16-byte hash derived from its public key)
How the tree is built:
-
Every node computes its NodeAddr = first 16 bytes of SHA-256(public key). This is a random-looking number that's unique to each node.
-
The node with the smallest NodeAddr is the root. This is deterministic — everyone agrees on who the root is without any election or voting.
-
Each node picks its parent = the peer that's closest to the root (shallowest depth). If you're not the root, your parent is someone who's one step closer to the root than you.
-
Nodes announce their tree position to their peers using TreeAnnounce messages: "I am Node G, my parent is Node D, my path to root is [G → D → B → Root]."
-
When a node receives a TreeAnnounce, it may discover a better parent (one with shallower depth) and switch. This is how the tree self-organizes and heals when nodes join/leave.
TreeAnnounce flow (direct peer-to-peer, NOT via Nostr):
Node G --"my parent is D, depth 3"--> Node D (its parent)
Node D --"my parent is B, depth 2"--> Node B (its parent)
Node B --"my parent is Root, depth 1"--> Root
And in the other direction (to children):
Node D --"I'm at depth 2, parent B"--> Node G (its child)
Node B --"I'm at depth 1, parent Root"--> Node D (its child)
TreeAnnounce messages are sent:
- On initial connection (after handshake)
- When a node's parent changes
- Periodically (every ~60 seconds) as a refresh
- Rate-limited to 1 per 500ms per peer to prevent storms
Why a Spanning Tree?
The tree gives every node a path to every other node: go up to your common ancestor, then down to the destination. It also prevents routing loops (there's exactly one path between any two nodes in a tree).
But the tree alone isn't enough for efficient routing — going all the way up to the root and back down is slow for distant nodes. That's where bloom filters come in (next section).
4. Routing: How Messages Get From A to B
The Problem
Node G wants to send a message to Node F. G is only connected to Node D. How does the message get to F?
G wants to send to F:
Root
/ \
B C
/ \ |
D E F <-- destination
|
G <-- source
H
The naive approach: send the message to G's parent (D), D sends to its parent (B), B sends to Root, Root sends to C, C sends to F. That's 5 hops — and it forces all traffic through the root, which doesn't scale.
FIPS does better using bloom filters.
Bloom Filters: Who Can Reach Whom?
A bloom filter is a compact data structure that can answer "is X in this set?" without storing the whole set. It's probabilistic — it can say "definitely not" or "maybe yes" (with a small false-positive rate). A bloom filter for 500 nodes is about 1 KB.
Every FIPS node maintains a bloom filter of which nodes are reachable through it (itself + everything in its subtree). It sends this filter to its direct peers via FilterAnnounce messages.
Each node tells its peers: "here's who I can reach"
Node G's filter: {G, H} (just itself + children)
Node D's filter: {D, G, H} (itself + its subtree)
Node B's filter: {B, D, E, G, H} (itself + its subtree)
Node C's filter: {C, F} (itself + its subtree)
Root's filter: {everyone} (it's the root, sees all)
Filters propagate UP the tree and are shared with all peers:
Node D tells B: "I can reach {D, G, H}"
Node B tells Root: "I can reach {B, D, E, G, H}"
Node C tells Root: "I can reach {C, F}"
Now every node knows, for each of its direct peers, a compact summary of who's reachable through that peer. When Node G needs to send to Node F:
G wants to send to F:
G checks its peers' bloom filters:
- Peer D's filter contains F? No (D can reach {D,G,H})
- (G only has one peer, so it sends to D anyway)
G --> D: "forward this to F"
D checks its peers' bloom filters:
- Peer B's filter contains F? No (B can reach {B,D,E,G,H})
- (D only has one peer besides G, so it sends to B)
D --> B: "forward this to F"
B checks its peers' bloom filters:
- Peer Root's filter contains F? Yes! (Root can reach everyone)
- Peer E's filter contains F? No
B --> Root: "forward this to F"
Root checks its peers' bloom filters:
- Peer B's filter contains F? No
- Peer C's filter contains F? Yes!
Root --> C: "forward this to F"
C --> F: "here's your message"
The bloom filters let each node make an informed decision about which peer to forward to, instead of blindly sending everything to its parent. This keeps traffic off the root when possible and makes routing efficient.
Coordinate Discovery: Finding the Exact Path
Bloom filters tell you which peer to forward to, but not the exact path. For that, FIPS uses coordinate discovery:
G wants to send to F (first time — no cached coordinates):
1. G sends a LookupRequest: "Where is F?"
(flooded through the tree, guided by bloom filters —
only forwarded to peers whose filter contains F)
G --> D --> B --> Root --> C --> F
2. F responds with a LookupResponse: "I'm here, my coordinates are
[F, C, Root, B, D, G]"
(sent back along the reverse path)
F --> C --> Root --> B --> D --> G
3. G caches F's coordinates in its coord cache.
Next time G sends to F, it knows the path and can route directly
without another lookup.
The coordinate cache is like a phone book — once you've looked up someone's number, you don't need to look it up again. Entries expire after 5 minutes (default), so if the network topology changes, stale routes get refreshed.
Putting It All Together: A Message's Journey
Node G sends a web request to Node F's .fips address:
1. G's application: "GET http://npub1...fxf.fips/index.html"
2. G's fipsd:
- Derives F's mesh IPv6 address from its npub (fd00::...)
- Checks coord cache: do I have F's coordinates?
- Yes: route directly using cached path
- No: send LookupRequest, wait for response, cache it
3. G's fipsd wraps the HTTP request in an encrypted session
datagram and sends it hop-by-hop:
G --[encrypted link]--> D --[encrypted link]--> B
B --[encrypted link]--> Root --[encrypted link]--> C
C --[encrypted link]--> F
Each hop:
- Decrypts the outer layer
- Reads the destination coordinates
- Checks bloom filters to pick the next hop
- Re-encrypts and forwards
4. F's fipsd receives the datagram, decrypts the session layer,
and delivers the HTTP request to F's web server.
5. F's web server responds, and the response goes back the same
way (or a better path if one was discovered).
Every hop is individually encrypted (the link layer), and the end-to-end session is also encrypted (the session layer). So even intermediate nodes can't read your traffic — they can only see "this packet is going to Node F, forward it to peer X."
The .fips DNS Hack
FIPS gives every node a virtual IPv6 address in the fd00::/8 range,
derived from its public key. These addresses only exist inside the
mesh — they're not real internet addresses.
To make this user-friendly, fipsd runs a local DNS resolver that
maps <npub>.fips to the node's mesh IPv6 address:
You type: curl http://npub1abc123...xyz.fips/index.html
1. Your browser asks DNS: "what's the IP for npub1abc...xyz.fips?"
2. Your fipsd's DNS resolver (127.0.0.1:5354) answers:
"It's fd5e:4c4:ce59:a5a4:829:46c7:2b33:ab21"
3. Your browser connects to that IPv6 address
4. The kernel routes it through fips0 (the TUN device)
5. fipsd reads it from fips0, routes it through the mesh
6. The message arrives at the destination node
This is why you can type http://npub1....fips in a browser and it
just works — the DNS + TUN + mesh routing make it transparent.
Summary: The Four Layers
+-------------------------------------------------------------------+
| THE FIPS MESH |
| |
| +-------------------------------------------------------------+ |
| | Layer 4: Application | |
| | HTTP, SSH, any TCP/UDP app via .fips addresses | |
| +-------------------------------------------------------------+ |
| | Layer 3: Routing | |
| | Bloom filters + coordinate discovery + spanning tree | |
| | "Which peer do I forward to?" | |
| +-------------------------------------------------------------+ |
| | Layer 2: Links | |
| | Noise IK encrypted peer-to-peer connections | |
| | "I'm securely connected to nodes B, D, and E" | |
| +-------------------------------------------------------------+ |
| | Layer 1: Discovery | |
| | Nostr Kind 37195 adverts + NAT traversal | |
| | "How do I find other nodes to connect to?" | |
| +-------------------------------------------------------------+ |
| |
+-------------------------------------------------------------------+
|
THE INTERNET
(the underlay)
| Layer | What it does | How | Nostr? |
|---|---|---|---|
| Discovery | Find other nodes | Nostr adverts + STUN hole-punch | Yes (bootstrap only) |
| Links | Secure connections | Noise IK handshake over UDP/TCP/Tor | No |
| Routing | Route through mesh | TreeAnnounce + FilterAnnounce + LookupRequest | No |
| Application | Use the mesh | .fips DNS + TUN device + any TCP/UDP app | No |
The key insight: Nostr is only used for the initial rendezvous. Once nodes are connected, everything else — topology, routing, data transfer — happens directly between peers over encrypted links. The mesh is a self-organizing overlay network that uses Nostr as its "phone book" and the internet as its "wires."