diff --git a/README.md b/README.md index f0b8f42..5a8f620 100644 --- a/README.md +++ b/README.md @@ -134,3 +134,59 @@ Now `sys-fips` and `sys-fips2` are independent FIPS nodes. - The previous two-node local loopback approach was useful for local experiments but is not required for normal Qubes deployment. - FIPS mesh encryption is end-to-end. A VPN upstream is optional and can be chained. - `06-configure-dns.sh` removes dnsmasq `--local-service` from Debian's helper to avoid dropping AppVM DNS on Qubes `/32` links. + +--- + +## Lessons learned from real `sys-fips` troubleshooting + +These are the key fixes required to make AppVM → `sys-fips` → FIPS mesh work reliably in Qubes. + +### 1) Intercept AppVM DNS in `sys-fips` with nftables + +Qubes AppVM DNS normally targets the Qubes gateway addresses (for example `10.139.1.1` / `10.139.1.2`). +To make `.fips` resolution work from routed AppVMs, `sys-fips` must DNAT DNS to local dnsmasq. + +Required nft hooks in `sys-fips`: + +- `table ip qubes`, chain `dnat-dns` +- `table ip qubes`, chain `custom-input` + +`dnat-dns` redirects UDP/TCP 53 traffic to local DNS (`10.139.1.1:53`) and `custom-input` allows DNS from AppVM interfaces (for example `vif+`). + +### 2) IPv6 forwarding + route to FIPS space + +`sys-fips` must forward IPv6 and route FIPS mesh space (`fd00::/8`) to `fips0`: + +- enable IPv6 forwarding (`sysctl net.ipv6.conf.all.forwarding=1`) +- ensure route exists: `ip -6 route add fd00::/8 dev fips0` + +Without this, AppVM traffic may resolve `.fips` names but fail to reach the resolved FIPS IPv6 destination. + +### 3) AppVMs need a valid IPv6 source + route toward `sys-fips` + +In testing, routed AppVM access required: + +- a synthetic IPv6 on `eth0` (example: `fdaa::15/128`) +- route of FIPS space via Qubes link-local gateway: + - `ip -6 route add fd00::/8 via fe80::fcff:ffff:feff:ffff dev eth0` + +This ensures traffic to resolved `.fips` AAAA records leaves the AppVM through `sys-fips`. + +### 4) Persist network fixes in `/rw/config/rc.local` + +For reboot safety: + +- persist nftables DNS interception + IPv6 forwarding/route in `sys-fips` `/rw/config/rc.local` +- persist AppVM synthetic IPv6 + route in each AppVM `/rw/config/rc.local` that needs `.fips` + +Do not rely on one-off manual commands for production. + +### 5) Chromium-family browser caveat + +Observed behavior: + +- Firefox followed system resolver path and worked once networking was fixed. +- Chromium-family browsers (`agent-browser`, Brave) could still show `ERR_NAME_NOT_RESOLVED` even when `dig`, `getent`, `ping`, and `curl` worked. + +Current operational workaround is to pin the `.fips` hostname in `/etc/hosts` for Chromium-family use-cases. +See `plans/chromium-fips-workaround.md` for tested commands and policy notes. diff --git a/plans/FIPS_ADDRESSING.md b/plans/FIPS_ADDRESSING.md new file mode 100644 index 0000000..4985097 --- /dev/null +++ b/plans/FIPS_ADDRESSING.md @@ -0,0 +1,177 @@ +# FIPS Addressing: How to Share Your Node Address + +## Overview + +There are **three ways** someone can reach your FIPS node, from most user-friendly to most raw: + +| Method | Example | Config needed on visitor? | +|---|---|---| +| Friendly hostname | `http://mysite.fips` | Yes — host map entry or peer alias | +| Direct npub | `http://npub1qmc3...98.fips` | No — works on any FIPS node | +| Raw IPv6 | `http://[fdab:1234:...]:80` | Requires prior DNS lookup to prime cache | + +--- + +## 1. Friendly Hostname: `mysite.fips` + +This is the best option for sharing a web page address. You create a human-readable alias by adding an entry to `/etc/fips/hosts` on the visitor's node: + +``` +mysite npub1abc...xyz +``` + +Or, if the visitor has you as a configured peer with an `alias` in their `fips.yaml`, it works automatically: + +```yaml +peers: + - npub: "npub1abc...xyz" + alias: "mysite" + addresses: + - transport: udp + addr: "1.2.3.4:2121" +``` + +Then they just browse to: + +``` +http://mysite.fips +``` + +### How it works + +The FIPS DNS resolver on port 5354 sees `mysite.fips`, looks up `mysite` in the host map → gets the npub → derives the `fd00::/8` IPv6 address → primes the identity cache → returns the AAAA record. The browser connects transparently. + +### The catch + +This is a *local* mapping. There's no global DNS — each node maintains its own host map. So you'd share both the hostname convention and the npub, and visitors add the entry themselves. Think of it like a `/etc/hosts` entry but for the FIPS mesh. + +Resolution priority: +1. `/etc/fips/hosts` entries (take precedence) +2. Peer `alias` fields from `fips.yaml` +3. Fall through to direct npub resolution + +Changes to `/etc/fips/hosts` are picked up automatically on the next DNS query — no restart needed. + +### Hosts file format + +``` +# /etc/fips/hosts +# hostname npub +mysite npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98 +vps-tx npub10yffd020a4ag8zcy75f9pruq3rnghvvhd5hphl9s62zgp35s560qrksp9u +``` + +Rules: +- Lines starting with `#` are comments +- Empty lines are ignored +- Hostnames: a-z, 0-9, hyphens only; max 63 characters +- One hostname and one npub per line, separated by whitespace + +--- + +## 2. Direct npub Address: `npub1abc...xyz.fips` + +This is the **zero-configuration** option — no host map needed. You give someone your npub and they use it directly as a domain name: + +``` +http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips +``` + +### How it works + +The FIPS DNS resolver extracts the npub from the label before `.fips`, derives the IPv6 address purely computationally (npub → public key → SHA-256 → node_addr → `fd00::/8` IPv6), and returns it as an AAAA record. No lookup table needed — the npub *is* the address. + +This works on any FIPS node without any prior configuration. The downside is obvious: npubs are 63 characters of bech32, so the URL is long and ugly. + +### Examples + +```bash +# Web browsing +http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips + +# SSH +ssh -6 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips + +# Ping +ping6 npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips + +# Curl +curl -6 http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips +``` + +--- + +## 3. Raw IPv6 Address: `fd00:...` + +Under the hood, every npub deterministically maps to an IPv6 address in the `fd00::/8` ULA range: + +``` +Public Key (32 bytes) + → SHA-256 → node_addr (16 bytes) + → fd + node_addr[0..15] → IPv6 address (16 bytes) +``` + +So you could share the raw IPv6: + +``` +http://[fdab:1234:5678:9abc:def0:1234:5678:9abc]:80 +``` + +### Why this is the worst option + +The derivation is **one-way** — the receiver's FIPS node can't recover the npub from the IPv6 address alone. The identity cache must be primed first (via a DNS lookup or inbound traffic from that node). If someone types the raw IPv6 without a prior DNS resolution, they get ICMPv6 "No route to destination." + +Use raw IPv6 only for debugging or when you've already resolved the address via DNS. + +--- + +## The Address Derivation Chain + +``` +Keypair (secp256k1) + │ + ├── npub1... (bech32-encoded public key) + │ ↓ + │ npub1xxx.fips ← DNS name, resolvable on any FIPS node + │ + ├── node_addr (16 bytes = SHA-256 of pubkey, truncated) + │ ↓ + │ Used internally for routing — intermediate nodes see only this + │ + └── fd00::/8 IPv6 (fd + node_addr[0..15]) + ↓ + Used by TUN interface for unmodified IPv6 apps +``` + +All three representations are **deterministically derived** from the same keypair. No registration, no central authority. + +--- + +## What You'd Actually Tell People + +In practice, you'd share something like: + +> **My FIPS address:** `npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98` +> +> Browse to: `http://npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98.fips` +> +> Or add to your `/etc/fips/hosts`: +> ``` +> coolsite npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98 +> ``` +> Then just visit `http://coolsite.fips` + +The **npub is the canonical address** — it's self-authenticating (derived from your keypair), works without any central registry, and any FIPS node can resolve it to an IPv6 address purely through computation. The friendly hostname is a local convenience layer on top. + +--- + +## Key Properties + +| Property | Detail | +|---|---| +| **Self-sovereign** | You generate your own address from a keypair — no one assigns it to you | +| **Self-authenticating** | The address IS the public key — connecting to an npub cryptographically proves you reached the right node | +| **Deterministic** | npub → IPv6 is pure computation, same result everywhere, no lookup needed | +| **No global registry** | No DNS authority, no CA, no registrar | +| **Friendly names are local** | `/etc/fips/hosts` is per-node; there's no global `.fips` name registry | +| **Privacy-preserving** | Intermediate routers see only the node_addr hash, not the npub identity | diff --git a/plans/chromium-fips-workaround.md b/plans/chromium-fips-workaround.md new file mode 100644 index 0000000..9d00cc2 --- /dev/null +++ b/plans/chromium-fips-workaround.md @@ -0,0 +1,53 @@ +# Chromium/Brave `.fips` Access in `ai` + +## Current proven behavior + +- `.fips` works in Firefox (system resolver path). +- `.fips` works in `agent-browser` only when hostname is pinned in `/etc/hosts`. +- Raw FIPS network path is healthy in `ai`: + - `dig` resolves npub `.fips` + - `ping -6` to server FIPS IPv6 works + - `curl -6` to npub `.fips` returns HTTP 200 + +## Why Chromium-family is failing by default + +Chromium/Brave path still returns name-resolution errors for `.fips` in this environment, even when system resolver succeeds. In practice here, forcing host mapping is the reliable path. + +## Reliable workaround (now) + +### 1) Resolve npub hostname to IPv6 and pin in hosts + +```bash +HOST='npub1crpldvy49ef8z34wlacwujnfudy4nd7k96aqdx5wgn6ckztz7z8q9t59ud.fips' +IP=$(dig +short AAAA "$HOST" | head -1) +echo "$IP $HOST" | sudo tee -a /etc/hosts +``` + +### 2) Validate resolution and HTTP + +```bash +getent ahosts "$HOST" +curl -6 -I "http://$HOST/" +``` + +### 3) Launch in Chromium tooling + +```bash +AGENT_BROWSER_HEADED=1 agent-browser open "http://$HOST/" +``` + +## Maintenance note + +If the target FIPS IPv6 changes, update `/etc/hosts` entry by re-running the resolve-and-pin step. + +## Optional cleanup + +To remove the pinned line later: + +```bash +sudo sed -i '/npub1crpldvy49ef8z34wlacwujnfudy4nd7k96aqdx5wgn6ckztz7z8q9t59ud\.fips/d' /etc/hosts +``` + +## Deferred work (later) + +For sys-fips ProxyVM mode, IPv6 forwarding/NAT and nftables flow still need a clean persistent design before re-enabling that path as default. diff --git a/plans/fips-overview.md b/plans/fips-overview.md new file mode 100644 index 0000000..9f7552f --- /dev/null +++ b/plans/fips-overview.md @@ -0,0 +1,209 @@ +# FIPS: Free Internetworking Peering System — Overview + +## 1. What It Does + +FIPS is a **self-organizing, decentralized mesh network** written in Rust. It allows any set of machines to form an encrypted peer-to-peer network that routes traffic without any central authority, DNS, ISP, or infrastructure dependency. + +### The Elevator Pitch + +> Connect to one peer and you can reach the entire mesh. No accounts, no servers, no registration. Your cryptographic keypair **is** your address. + +### Concrete Capabilities + +| Capability | Description | +|---|---| +| **Mesh networking** | Nodes discover each other, build routing automatically, and relay traffic for each other across multiple hops | +| **Multi-transport** | Works over UDP, TCP, Ethernet, Tor, and Bluetooth (BLE) — designed for serial and radio too | +| **End-to-end encryption** | Double-layer: every hop is encrypted *and* every session is independently encrypted end-to-end | +| **Nostr-native identity** | Your node address is a Nostr keypair — `npub1...` — no registration, no CA, no central authority | +| **IPv6 compatibility** | A TUN interface maps npubs to `fd00::/8` IPv6 addresses so unmodified apps like SSH, curl, and ping6 just work | +| **DNS for the mesh** | Built-in `.fips` DNS resolver: `ping6 npub1abc...fips` resolves and routes through the mesh | +| **Zero configuration** | A node can start with no config file; add one peer address and you join the network | +| **Operator tooling** | `fipsctl` CLI and `fipstop` TUI dashboard for live monitoring of peers, links, sessions, and routing | + +### What You Gain + +1. **Sovereignty** — You own your identity. No one can revoke your address or deny you access. +2. **Resilience** — The mesh self-heals around failures. No single point of failure. +3. **Privacy** — Intermediate nodes route traffic without learning endpoint identities. Hop-by-hop + end-to-end encryption means even your direct peers cannot read your session traffic. +4. **Transport freedom** — If you can get packets between two machines by *any* means — WiFi, Bluetooth, a serial cable, Tor, the public internet — FIPS can build a mesh link over it. +5. **Application transparency** — Existing IPv6 apps work without modification through the TUN adapter. + +--- + +## 2. How It Works (High Level) + +### Architecture: Three Protocol Layers + +``` +┌─────────────────────────────────────────────────┐ +│ Applications (SSH, curl, custom) │ +│ ↕ IPv6 TUN adapter (fd00::/8) + .fips DNS │ +├─────────────────────────────────────────────────┤ +│ FSP — FIPS Session Protocol │ +│ End-to-end Noise XK encryption between npubs │ +│ Port-based service multiplexing │ +├─────────────────────────────────────────────────┤ +│ FMP — FIPS Mesh Protocol │ +│ Peer auth (Noise IK), spanning tree, │ +│ bloom filters, hop-by-hop forwarding │ +├─────────────────────────────────────────────────┤ +│ Transport Layer │ +│ UDP | TCP | Ethernet | Tor | BLE │ +└─────────────────────────────────────────────────┘ +``` + +### Identity: Your Keypair Is Your Address + +1. A node generates a **secp256k1 keypair** (same as Nostr) +2. The public key (`npub`) is the node's identity +3. A **node_addr** (16-byte SHA-256 hash of the pubkey) is used internally for routing — intermediate routers see only this hash, not the npub +4. An **IPv6 address** (`fd00::/8`) is deterministically derived from the node_addr for the TUN interface + +All three representations — npub, node_addr, IPv6 — are derived from the same keypair. No registration needed. + +### Self-Organization: Spanning Tree + Bloom Filters + +The mesh organizes itself through two complementary mechanisms: + +#### Spanning Tree (The Coordinate System) + +- Nodes gossip `TreeAnnounce` messages with their direct peers +- Each node independently selects a **parent** based on cost-weighted depth +- The network converges on a single **root** (the node with the smallest node_addr) — no voting, no election +- Every node gets a **coordinate** — its path from itself to the root +- These coordinates enable **distance calculations** between any two nodes without global topology knowledge +- Tree state per node is O(peers × depth), **not** O(network size) + +#### Bloom Filters (Reachability Knowledge) + +- Each node maintains per-peer bloom filters summarizing which destinations are reachable through each peer +- Filters propagate transitively through tree edges using **split-horizon** (borrowed from distance-vector routing) to prevent loops +- At each hop, the router checks: "which of my peers can possibly reach destination D?" — definitive "no" eliminates a peer; "maybe" means worth checking +- Fixed 1 KB filter size regardless of network size; effective for networks up to ~2,000 nodes + +#### Routing Decision (Per-Hop) + +At each hop, the forwarding decision is: + +1. **Local delivery?** — destination is this node +2. **Direct peer?** — destination is an authenticated neighbor +3. **Bloom-guided selection** — bloom filters identify candidate peers; tree coordinates rank them by distance +4. **Greedy tree routing** — forward to the peer closest to the destination in tree distance + +### Two-Layer Encryption + +Every packet is encrypted **twice**, independently: + +| Layer | Scope | Noise Pattern | Purpose | +|---|---|---|---| +| **FMP** | Hop-by-hop | Noise IK | Encrypts ALL traffic on each peer link | +| **FSP** | End-to-end | Noise XK | Encrypts application payload between endpoints | + +A packet from node A to node D through intermediaries B and C: + +``` +A encrypts payload with A↔D session key (FSP) +A wraps + encrypts with A↔B link key (FMP) → sends to B +B decrypts link layer, reads destination, re-encrypts with B↔C link key → forwards to C +C decrypts link layer, re-encrypts with C↔D link key → forwards to D +D decrypts link layer, then decrypts session layer → gets payload +``` + +Intermediate nodes B and C route by destination node_addr but **cannot read the session payload**. + +### Transport Agnosticism + +FIPS treats transports as dumb pipes. Each transport implements: send bytes, receive bytes, report MTU. The mesh layer handles everything above that. A single packet path might traverse: + +``` +WiFi → Ethernet → UDP/Internet → Tor → BLE +``` + +The application never knows or cares which transports are involved. + +--- + +## 3. How It Is Innovative + +### 3.1 Nostr Identity as Network Address + +Most mesh networks invent their own identity system. FIPS reuses **Nostr keypairs** (secp256k1/Schnorr) as native node addresses. This is significant because: + +- **No identity infrastructure** — no CA, no registrar, no DNS. Generate a keypair and you have an address. +- **Cross-ecosystem compatibility** — the same npub that identifies you on Nostr relays identifies your FIPS node. Your social identity and your network identity can be the same key. +- **Ephemeral or persistent** — generate a throwaway keypair for a session, or keep a persistent identity across restarts. The network treats both the same. + +No other mesh network uses an existing social-protocol identity system as its native addressing scheme. + +### 3.2 Spanning Tree Coordinates + Bloom Filter Routing + +FIPS combines two well-understood techniques in a novel way: + +- **Spanning tree coordinates** (adapted from Yggdrasil/Ironwood) give every node a position without global topology knowledge +- **Bloom filter reachability** (adapted from distance-vector split-horizon) narrows forwarding candidates probabilistically + +The key insight: bloom filters are an **optimization**, not a correctness requirement. If filters saturate in a large network, routing still works via greedy tree distance — it just evaluates more candidates per hop. This graceful degradation is unusual; most systems that depend on probabilistic structures fail hard when those structures saturate. + +Each node's state is **O(peers × depth)**, not O(N). A node in a 10,000-node network with 5 peers and depth 10 stores ~50 ancestry entries — not 10,000 routing table entries. + +### 3.3 True Transport Agnosticism + +Most "multi-transport" systems support maybe UDP and TCP. FIPS has working implementations for: + +- **UDP** and **TCP** (overlay over existing internet) +- **Raw Ethernet** (no IP stack needed — MAC-level, with auto-discovery) +- **Tor** (SOCKS5 + onion service for anonymous peering) +- **Bluetooth Low Energy** (L2CAP channels with MTU negotiation) + +The architecture is designed for **serial links and radio** too. This isn't theoretical — the transport interface is genuinely minimal (send/receive datagrams + report MTU), making new transports straightforward to add. + +The implication: FIPS can build a mesh that spans the public internet, a local WiFi network, a Bluetooth PAN, and a Tor hidden service — all simultaneously, all transparently to applications. + +### 3.4 Double-Layer Encryption with Metadata Minimization + +The two independent encryption layers (hop-by-hop Noise IK + end-to-end Noise XK) are not just defense-in-depth — they serve distinct purposes: + +- **Hop-by-hop (IK)**: Authenticates peers and encrypts ALL link traffic, including routing gossip. An observer on the transport sees only encrypted blobs. +- **End-to-end (XK)**: Protects application payload across untrusted intermediaries. XK is chosen over IK specifically because it **delays initiator identity disclosure** until msg3, providing stronger identity hiding for traffic traversing untrusted nodes. + +Intermediate routers see only **node_addr** (a one-way hash) — they cannot learn the Nostr identities of the communicating endpoints. This is a deliberate metadata minimization design, not an accident. + +### 3.5 IPv6 Adaptation Without Compromise + +Many overlay networks require custom APIs or modified applications. FIPS provides a **TUN interface** that maps npubs to `fd00::/8` IPv6 addresses, complete with: + +- `.fips` DNS resolution (`ping6 npub1abc...fips`) +- ICMPv6 Packet Too Big for path MTU discovery +- TCP MSS clamping +- Static hostname mapping via `/etc/fips/hosts` + +Unmodified SSH, curl, web browsers, and any IPv6-capable application work transparently. This makes FIPS immediately useful without waiting for application ecosystem adoption. + +### 3.6 Qubes OS Integration (This Repository) + +The `fips_setup` repository demonstrates FIPS running as a **Qubes OS ProxyVM** (`sys-fips`), slotting into the Qubes network chain: + +``` +AppVMs → sys-fips (ProxyVM) → sys-firewall → sys-net → Internet + └─ fips0 (TUN) → FIPS mesh (encrypted) +``` + +Any AppVM pointed at `sys-fips` transparently gains access to the FIPS mesh. This is a practical deployment model that leverages Qubes' compartmentalization — the FIPS node runs in its own security domain, isolated from application VMs. + +--- + +## Summary + +| Dimension | FIPS Approach | +|---|---| +| **Identity** | Nostr keypairs — self-sovereign, no infrastructure | +| **Routing** | Spanning tree coordinates + bloom filter candidate selection | +| **Encryption** | Dual-layer: Noise IK hop-by-hop + Noise XK end-to-end | +| **Transports** | UDP, TCP, Ethernet, Tor, BLE — designed for radio/serial | +| **Application compat** | IPv6 TUN + .fips DNS — unmodified apps work | +| **State per node** | O(peers × depth), not O(network size) | +| **Configuration** | Zero-config possible; one peer address to join | +| **Implementation** | Rust, ~100+ source files, MIT licensed | + +FIPS is not just another VPN or overlay network. It is a **complete replacement for the network layer** that can operate alongside or independently of the internet, using whatever physical media are available, with cryptographic identity as the foundation rather than an afterthought. diff --git a/plans/laantungir-tor-fips-plan.md b/plans/laantungir-tor-fips-plan.md new file mode 100644 index 0000000..5f7d709 --- /dev/null +++ b/plans/laantungir-tor-fips-plan.md @@ -0,0 +1,440 @@ +# laantungir.net: Tor + FIPS + Clearnet Access Plan + +## Current Server State (discovered via SSH) + +| Component | Status | +|---|---| +| **OS** | Ubuntu 24.04.3 LTS (VM410) | +| **nginx** | 1.24.0, running, with RTMP module | +| **Tor** | 0.4.8.10, running, SOCKS on 127.0.0.1:9050 | +| **Onion service** | `kn2jam4kyz6s5wacyozo3do3d2zsvh4to45uh7xliulwdust6zjuvnad.onion` — forwards ports 80/443 to 127.0.0.1:80/443 | +| **FIPS** | Not installed | +| **Nostr relay** | c-relay-pg (C Nostr Relay, PostgreSQL backend) on port 8888 | +| **Blossom** | didactyl.service on 127.0.0.1:8484 | +| **Website** | Static site served by nginx | +| **Gitea** | Port 3000 (web) + 2222 (SSH) | +| **PostgreSQL** | 18-main on 127.0.0.1:5432 | +| **RTMP** | nginx-rtmp on port 1935 with auth + HLS | +| **Streaming** | OBS → RTMP :1935/publish (auth via rtmp-auth.service on :8091) → ffmpeg forward → :1935/live → HLS at /var/www/html/hls/ | +| **Stream stats** | stream-stats.service on 127.0.0.1:8092, nginx internal at 127.0.0.1:8080/stat | +| **simon.service** | 127.0.0.1:8485 (unknown purpose) | + +### Video Streaming Architecture + +``` +OBS (remote) + │ + │ RTMP push to :1935/publish/{show}/{variant}/{secret_key} + │ + ▼ +nginx-rtmp :1935 + ├── on_publish → 127.0.0.1:8091/auth (rtmp-auth.service, validates secret key) + ├── exec_push → /usr/local/bin/rtmp-forward.sh + │ └── ffmpeg -c copy → rtmp://127.0.0.1:1935/live/{show}/{variant} + │ + ▼ +nginx-rtmp :1935/live + ├── HLS output → /var/www/html/hls/{show}/{variant}/index.m3u8 + ├── generate-master-playlist.py → /var/www/html/hls/{show}/master.m3u8 + └── RTMP playback (allow play all) +``` + +**Viewer access (clearnet)**: `https://laantungir.net/hls/{show}/{variant}/index.m3u8` (HLS) or `rtmp://laantungir.net/live/{show}/{variant}` (RTMP direct) + +**Current shows in HLS**: beat-topia, dub-techno, music, the-world-is-coming-to-an-end + +### Current nginx virtual hosts (from /etc/nginx/conf.d/default.conf) + +| Domain | Port 80 | Port 443 | Backend | +|---|---|---|---| +| `laantungir.net` (+ .com, .org, www) | Redirect to HTTPS | TLS, static site | Local files | +| `relay.laantungir.net` (+ .com, .org) | Redirect to HTTPS | TLS, WebSocket proxy | 127.0.0.1:8888 (c-relay-pg) | +| `blossom.laantungir.net` | Redirect to HTTPS | TLS, proxy | 127.0.0.1:8484 (didactyl) | +| `git.laantungir.net` (+ .com, .org) | Redirect to HTTPS | TLS, proxy | 127.0.0.1:3000 (gitea) | +| `auth.laantungir.net` (+ .com, .org) | Redirect to HTTPS | TLS, proxy | 127.0.0.1:3000 (gitea) | + +### Current Tor onion service + +``` +HiddenServiceDir /var/lib/tor/nostr_client/ +HiddenServicePort 80 127.0.0.1:80 +HiddenServicePort 443 127.0.0.1:443 +``` + +Onion address: `kn2jam4kyz6s5wacyozo3do3d2zsvh4to45uh7xliulwdust6zjuvnad.onion` + +**Problem**: The onion service forwards to nginx ports 80/443, but nginx virtual hosts use `server_name` matching. Tor clients send the `.onion` hostname, which doesn't match any `server_name` directive — so nginx falls through to the default server block. This needs to be fixed. + +--- + +## Target Architecture + +``` + ┌─────────────────────────────────────────────────────────┐ + │ laantungir.net (VM410) │ + │ │ + Clearnet user ───►│─── eth0 :80/:443 ──► nginx ──┬── / ──► static site │ + │ ├── relay ──► :8888 │ + │ └── blossom ──► :8484 │ + │ │ + Tor user ────────►│─── tor :80/:443 ──► nginx ──┬── / ──► static site │ + │ (via .onion) ├── relay ──► :8888 │ + │ └── blossom ──► :8484 │ + │ │ + FIPS user ───────►│─── fips0 :80 ──► nginx ──┬── / ──► static site │ + │ (via fd00::/8) ├── relay ──► :8888 │ + │ └── blossom ──► :8484 │ + │ │ + FIPS+Tor user ───►│─── tor transport ──► fips0 ──► nginx (same as FIPS) │ + │ (FIPS peering over Tor) │ + │ │ + │ FIPS daemon: UDP :2121, TCP :8443, Tor .onion:8443 │ + └─────────────────────────────────────────────────────────┘ +``` + +### How each access path works + +| Path | User needs | Connects via | Encryption | +|---|---|---|---| +| **Clearnet** | Browser | `https://relay.laantungir.net` | TLS (Let's Encrypt) | +| **Tor** | Tor Browser | `http://kn2jam...nad.onion/relay/` or subdomain routing | Tor onion encryption | +| **FIPS** | FIPS node | `ws://npub1.fips` | Noise IK + Noise XK (double) | +| **FIPS over Tor** | FIPS node + Tor | FIPS peers via Tor transport | Tor + Noise IK + Noise XK (triple) | + +--- + +## Implementation Plan + +### Phase 1: Fix Tor Onion Service Routing + +The existing onion service already forwards ports 80/443 to nginx, but nginx doesn't know how to route `.onion` hostnames. Two approaches: + +**Option A: Path-based routing (single .onion, recommended)** + +Add the `.onion` hostname to each `server_name` directive: + +```nginx +# In the relay server block: +server_name relay.laantungir.com relay.laantungir.net relay.laantungir.org + kn2jam4kyz6s5wacyozo3do3d2zsvh4to45uh7xliulwdust6zjuvnad.onion; +``` + +Tor users would access: +- Website: `http://kn2jam...nad.onion/` +- Relay: `ws://kn2jam...nad.onion/` (if relay is the default server) +- Blossom: `http://kn2jam...nad.onion/blossom/` (needs path-based routing) + +**Problem**: relay and blossom are separate subdomains, not paths. With a single .onion, we need a way to distinguish them. + +**Option B: Separate onion services per service (cleaner)** + +Create three onion services in torrc: + +``` +# Website +HiddenServiceDir /var/lib/tor/hs_website/ +HiddenServicePort 80 127.0.0.1:80 +HiddenServicePort 443 127.0.0.1:443 + +# Relay +HiddenServiceDir /var/lib/tor/hs_relay/ +HiddenServicePort 80 127.0.0.1:80 +HiddenServicePort 443 127.0.0.1:443 + +# Blossom +HiddenServiceDir /var/lib/tor/hs_blossom/ +HiddenServicePort 80 127.0.0.1:80 +HiddenServicePort 443 127.0.0.1:443 +``` + +Then add each `.onion` to the corresponding nginx `server_name`. Each service gets its own `.onion` address. Tor users connect to the right service by using the right `.onion`. + +**Option C: Single .onion with default-server routing (simplest)** + +Make the relay the default server (it's the most important service), and add the `.onion` to the website and blossom blocks too. Use a port-based split: + +``` +HiddenServiceDir /var/lib/tor/nostr_client/ +HiddenServicePort 80 127.0.0.1:80 # website +HiddenServicePort 443 127.0.0.1:443 # website (TLS) +HiddenServicePort 7777 127.0.0.1:8888 # relay direct (no nginx) +HiddenServicePort 8484 127.0.0.1:8484 # blossom direct (no nginx) +``` + +Tor users connect to different ports on the same `.onion`: +- `http://kn2jam...nad.onion/` → website +- `ws://kn2jam...nad.onion:7777/` → relay (direct to c-relay-pg) +- `http://kn2jam...nad.onion:8484/` → blossom (direct to didactyl) + +### Decision: Option B — Separate .onion per service + +This gives the best compatibility with existing programs. Nostr clients expect to connect to a relay at a WebSocket URL like `ws://host/`. Blossom clients expect `http://host/`. Separate `.onion` addresses mean each service gets its own hostname — no path rewriting, no port gymnastics. Existing clients work unmodified. + +### Steps for Phase 1 + +1. Add three HiddenService blocks to `/etc/tor/torrc` (website, relay, blossom) +2. Add each `.onion` hostname to the corresponding nginx `server_name` directive +3. Add plain-HTTP server blocks for `.onion` access (Tor already encrypts; no TLS needed) +4. Restart Tor (generates new onion keys), then restart nginx +5. Test from Tor Browser + +--- + +### Phase 2: Install and Configure FIPS + +#### 2.1 Build and install FIPS + +```bash +# Install Rust toolchain +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +source ~/.cargo/env + +# Install build deps for BLE (optional, can skip with --no-default-features) +sudo apt install -y bluez libdbus-1-dev + +# Clone and build +git clone https://github.com/jmcorgan/fips.git +cd fips +cargo build --release + +# Install via .deb +cargo install cargo-deb +cargo deb +sudo dpkg -i target/debian/fips_*.deb +``` + +#### 2.2 Configure FIPS + +Edit `/etc/fips/fips.yaml`: + +```yaml +node: + identity: + persistent: true # Stable npub for the server + +tun: + enabled: true + name: fips0 + mtu: 1280 + +dns: + enabled: true + bind_addr: "127.0.0.1" + port: 5354 + +transports: + udp: + bind_addr: "0.0.0.0:2121" + tcp: + bind_addr: "0.0.0.0:8443" + tor: + mode: "directory" + socks_addr: "127.0.0.1:9050" + directory_service: + hostname_file: "/var/lib/tor/fips_mesh/hostname" + bind_addr: "127.0.0.1:9443" + +peers: + - npub: "npub1qmc3cvfz0yu2hx96nq3gp55zdan2qclealn7xshgr448d3nh6lks7zel98" + alias: "fips-test-node" + addresses: + - transport: udp + addr: "217.77.8.91:2121" + connect_policy: auto_connect + - npub: "npub1zv58cn7v83mxvttl70w5fwjwuclfmntv9cnmv5wmz2nzz88u5urqvdx96n" + alias: "fips.v0l.io" + addresses: + - transport: udp + addr: "fips.v0l.io:2121" + connect_policy: auto_connect +``` + +#### 2.3 Add FIPS Tor onion service to torrc + +Add a separate onion service for FIPS mesh peering: + +``` +# FIPS mesh peering (separate from web services) +HiddenServiceDir /var/lib/tor/fips_mesh/ +HiddenServicePort 9443 127.0.0.1:9443 +``` + +#### 2.4 Open firewall for FIPS + +```bash +sudo ufw allow 2121/udp comment "FIPS mesh UDP" +sudo ufw allow 8443/tcp comment "FIPS mesh TCP" +``` + +#### 2.5 Start FIPS + +```bash +sudo systemctl enable --now fips +sudo systemctl enable --now fips-dns +``` + +#### 2.6 Configure DNS for .fips resolution on the server + +```bash +sudo resolvectl dns fips0 127.0.0.1:5354 +sudo resolvectl domain fips0 ~fips +``` + +--- + +### Phase 3: Make Services Reachable Over FIPS + +#### 3.1 Make nginx listen on IPv6 (for FIPS fd00::/8 traffic) + +Add `listen [::]:80;` and `listen [::]:443 ssl;` to each server block in `/etc/nginx/conf.d/default.conf`. This allows FIPS traffic (which arrives as IPv6 on the fips0 TUN interface) to reach nginx. + +#### 3.2 Decision: Separate server blocks per service (matching Tor approach) + +For maximum compatibility with existing programs, we use the same approach as Tor: each service gets its own server block matched by hostname. FIPS users use different `.fips` hostnames (via `/etc/fips/hosts`) to reach each service — just like Tor users use different `.onion` addresses. + +Since blossom (didactyl) is a FastCGI extension of nginx, all three services must go through nginx. + +```nginx +# FIPS: Website +server { + listen [::]:80; + server_name ~^npub1.+\.fips$ laantungir.fips; + + # Same config as the clearnet website server block + # (static site root, locations, etc.) +} + +# FIPS: Relay +server { + listen [::]:80; + server_name relay.fips; + + location / { + proxy_pass http://127.0.0.1:8888; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + } +} + +# FIPS: Blossom +server { + listen [::]:80; + server_name blossom.fips; + + # Same FastCGI config as the clearnet blossom server block +} +``` + +FIPS users access: +- `http://laantungir.fips/` → website +- `ws://relay.fips/` → Nostr relay (standard WebSocket URL, works with all Nostr clients) +- `http://blossom.fips/` → Blossom (standard HTTP, works with all Blossom clients) + +The direct npub also works: `http://npub1.fips/` → website (default) + +#### 3.3 Add /etc/fips/hosts entries + +``` +laantungir npub1 +relay npub1 +blossom npub1 +``` + +This lets FIPS users access: +- `http://laantungir.fips/` → website +- `ws://relay.fips/` → relay (if using path-based routing) +- `http://blossom.fips/` → blossom + +--- + +### Phase 4: Enable FIPS-over-Tor (Combined) + +This is already handled by Phase 2.3. The FIPS config includes a Tor transport with a dedicated onion service for mesh peering. Remote FIPS nodes can peer with this server through Tor by adding: + +```yaml +# On the remote FIPS node +peers: + - npub: "npub1" + alias: "laantungir" + addresses: + - transport: tor + addr: ".onion:9443" + connect_policy: auto_connect +``` + +The remote node's FIPS traffic goes through Tor to reach this server's FIPS daemon, then the FIPS session provides end-to-end encryption on top. Triple encryption: Tor + Noise IK (hop) + Noise XK (session). + +--- + +## Summary of Access Paths After Implementation + +### Website (laantungir.net) + +| Path | URL | Encryption | +|---|---|---| +| Clearnet | `https://laantungir.net` | TLS | +| Tor | `http://kn2jam...nad.onion/` | Tor | +| FIPS | `http://npub1.fips/` or `http://laantungir.fips/` | Noise IK+XK | +| FIPS+Tor | Same as FIPS (transport is transparent) | Tor + Noise IK+XK | + +### Nostr Relay (relay.laantungir.net) + +| Path | URL | Encryption | +|---|---|---| +| Clearnet | `wss://relay.laantungir.net` | TLS | +| Tor | `ws://.onion/` | Tor | +| FIPS | `ws://relay.fips/` | Noise IK+XK | +| FIPS+Tor | Same as FIPS (transport is transparent) | Tor + Noise IK+XK | + +### Blossom (blossom.laantungir.net) + +| Path | URL | Encryption | +|---|---|---| +| Clearnet | `https://blossom.laantungir.net` | TLS | +| Tor | `http://.onion/` | Tor | +| FIPS | `http://blossom.fips/` | Noise IK+XK | +| FIPS+Tor | Same as FIPS (transport is transparent) | Tor + Noise IK+XK | + +### Video Streaming (laantungir.net/hls/) + +| Path | URL | Encryption | Notes | +|---|---|---|---| +| Clearnet | `https://laantungir.net/hls/{show}/master.m3u8` | TLS | HLS playback via static files | +| Clearnet RTMP | `rtmp://laantungir.net/live/{show}/{variant}` | None | Direct RTMP playback | +| Tor | `http://.onion/hls/{show}/master.m3u8` | Tor | HLS over Tor (same website .onion) | +| FIPS | `http://laantungir.fips/hls/{show}/master.m3u8` | Noise IK+XK | HLS over FIPS | +| FIPS+Tor | Same as FIPS | Tor + Noise IK+XK | HLS over FIPS+Tor | + +**Streaming considerations:** + +- **HLS playback** (HTTP-based) works naturally over Tor and FIPS — it's just static `.m3u8` and `.ts` files served by nginx. No special config needed beyond the website server block. +- **RTMP ingest** (OBS publishing on port 1935) stays clearnet-only. RTMP is a TCP protocol on a non-HTTP port; exposing it over Tor or FIPS would require additional transport config and is not needed — the publisher (you) controls the ingest path. +- **RTMP direct playback** (port 1935/live) is clearnet-only for the same reason. Viewers over Tor/FIPS use HLS instead. +- **Stream stats** (127.0.0.1:8092) and **RTMP auth** (127.0.0.1:8091) are internal-only and not exposed on any path. + +--- + +## Decisions (Resolved) + +1. **Tor routing**: Option B — separate `.onion` per service. Best compatibility with existing clients. +2. **FIPS service routing**: Separate `.fips` hostnames per service via `/etc/fips/hosts`. Best compatibility with existing clients. +3. **Blossom**: Stays behind nginx (it's a FastCGI extension, must go through nginx). +4. **Bootstrap peers**: fips-test-node (217.77.8.91:2121) + fips.v0l.io (fips.v0l.io:2121). + +--- + +## Execution Order + +1. Fix Tor onion routing (nginx server_name + torrc) +2. Test Tor access from Tor Browser +3. Build and install FIPS +4. Configure FIPS with persistent identity + Tor transport +5. Add FIPS onion service to torrc, restart Tor +6. Open firewall ports for FIPS +7. Start FIPS, verify fips0 interface and peer connectivity +8. Update nginx to listen on IPv6 and handle FIPS traffic +9. Test FIPS access from another FIPS node +10. Test FIPS-over-Tor access +11. Publish server npub and .onion addresses diff --git a/plans/sys-fips-browser-fix-plan.md b/plans/sys-fips-browser-fix-plan.md new file mode 100644 index 0000000..3e9dcb1 --- /dev/null +++ b/plans/sys-fips-browser-fix-plan.md @@ -0,0 +1,113 @@ +# sys-fips `.fips` Browser Access Remediation Plan + +## Goal + +Make `.fips` hostnames (especially the npub hostname for `laantungir.net`) resolve and load from the current AppVM browser path. + +## Confirmed Root Cause + +Current AppVM DNS path is returning public-DNS NXDOMAIN for `.fips`, which means `.fips` forwarding via `sys-fips` is not active in the effective network chain. + +## Success Criteria + +1. From AppVM, the npub `.fips` hostname returns an AAAA answer (not NXDOMAIN). +2. Browser can open the npub `.fips` URL successfully. +3. `sys-fips` is persistently configured so the behavior survives reboot. + +--- + +## Phase 1 — Validate and fix the network chain in dom0 + +1. Ensure target AppVM uses `sys-fips` as netvm (not `sys-firewall`/`sys-whonix`/other). +2. Ensure `sys-fips` itself chains to intended upstream netvm (`sys-firewall` or `sys-vpn`). +3. Restart affected VMs after netvm changes so DHCP/DNS lease state is refreshed. + +Reference workflow: [`README.md`](README.md:102) + +--- + +## Phase 2 — Bring `sys-fips` services to healthy state + +Inside `sys-fips`: + +1. Start/enable FIPS services (`fips`, `fips-dns`). +2. Ensure FIPS config has DNS enabled on local resolver port 5354. +3. Ensure `fips0` exists and has IPv6 address. + +Config reference: [`configs/fips.yaml`](configs/fips.yaml:6) + +--- + +## Phase 3 — Re-apply DNS forwarding in `sys-fips` + +Inside `sys-fips`: + +1. Run DNS setup script to make dnsmasq forward `.fips` to local FIPS DNS (`127.0.0.1:5354`). +2. Verify dnsmasq is running and listening on port 53. +3. Verify local forwarding behavior using a `.fips` query. + +Implementation reference: [`scripts/06-configure-dns.sh`](scripts/06-configure-dns.sh:1) + +--- + +## Phase 4 — Re-apply IPv6 forwarding/routing for AppVM traffic + +Inside `sys-fips`: + +1. Enable IPv6 forwarding. +2. Ensure route for `fd00::/8` points to `fips0`. +3. Ensure ip6tables forwarding rules exist and are persisted. + +Implementation reference: [`scripts/07-route-appvms.sh`](scripts/07-route-appvms.sh:1) + +--- + +## Phase 5 — Verify mesh reachability from `sys-fips` + +Inside `sys-fips`: + +1. Confirm at least one reachable peer in FIPS config. +2. Confirm FIPS node has joined mesh and can resolve npub `.fips` locally. +3. If unresolved, add/bootstrap peers and restart FIPS. + +Planning reference: [`plans/sys-fips-plan.md`](plans/sys-fips-plan.md:143) + +--- + +## Phase 6 — Verify from AppVM (the failing client) + +From the AppVM: + +1. Confirm resolver points to `sys-fips`-provided DNS. +2. Query npub `.fips` against current resolver and confirm AAAA response. +3. Confirm no NXDOMAIN for `.fips`. + +If still NXDOMAIN, restart AppVM networking context (or reboot AppVM) after Phase 1–4 changes. + +--- + +## Phase 7 — Browser validation (headed) + +1. Test npub `.fips` URL in headed browser automation. +2. Test alias hostname variant (if configured in hosts mapping). +3. Capture browser errors; expected fix is removal of name-resolution failure. + +Operational context from previous troubleshooting: [`plans/laantungir-tor-fips-plan.md`](plans/laantungir-tor-fips-plan.md:95) + +--- + +## Execution Order (Minimal Downtime) + +1. dom0 netvm chain correction +2. `sys-fips` service health (`fips`, `fips-dns`) +3. DNS forwarding script +4. Routing script +5. Peer/mesh verification +6. AppVM DNS verification +7. Browser verification + +--- + +## Expected Outcome + +After these steps, the AppVM should resolve the npub `.fips` hostname through `sys-fips` DNS forwarding and reach the remote FIPS-hosted site in browser without `ERR_NAME_NOT_RESOLVED`. diff --git a/scripts/02-install-fips.sh b/scripts/02-install-fips.sh index ea09672..2918696 100755 --- a/scripts/02-install-fips.sh +++ b/scripts/02-install-fips.sh @@ -25,7 +25,7 @@ echo "Installing system dependencies..." apt-get update -qq apt-get install -y --no-install-recommends \ iproute2 iputils-ping dnsutils \ - dnsmasq iptables \ + dnsmasq iptables nftables \ curl python3 jq \ openssh-server openssh-client \ iperf3 2>/dev/null || { diff --git a/scripts/06-configure-dns.sh b/scripts/06-configure-dns.sh index fec80b4..d8f08dc 100755 --- a/scripts/06-configure-dns.sh +++ b/scripts/06-configure-dns.sh @@ -5,6 +5,9 @@ # Configure dnsmasq in sys-fips so .fips queries are forwarded to local # FIPS DNS resolver on 127.0.0.1:5354, and all other DNS goes upstream. # +# Also applies Qubes-specific nftables DNS interception so AppVM DNS queries +# are forced to local dnsmasq in sys-fips. +# # Usage: # sudo bash 06-configure-dns.sh # ============================================================================== @@ -18,6 +21,23 @@ if [ "$(id -u)" -ne 0 ]; then exit 1 fi +ensure_nft_rule() { + local table="$1" + local chain="$2" + local rule="$3" + + if ! nft list chain "$table" "$chain" >/dev/null 2>&1; then + echo " WARN: nft chain $table $chain not found; skipping: $rule" + return 0 + fi + + if nft list chain "$table" "$chain" | grep -F -- "$rule" >/dev/null 2>&1; then + return 0 + fi + + nft add rule "$table" "$chain" $rule +} + systemctl stop dnsmasq 2>/dev/null || true killall dnsmasq 2>/dev/null || true @@ -61,7 +81,9 @@ fi # which can reject DNS queries from downstream AppVMs in this topology. DNSMASQ_HELPER="/usr/share/dnsmasq/systemd-helper" if [ -f "$DNSMASQ_HELPER" ] && grep -q -- '--local-service' "$DNSMASQ_HELPER"; then - cp "$DNSMASQ_HELPER" "${DNSMASQ_HELPER}.bak" + if [ ! -f "${DNSMASQ_HELPER}.bak" ]; then + cp "$DNSMASQ_HELPER" "${DNSMASQ_HELPER}.bak" + fi sed -i 's/--local-service//g' "$DNSMASQ_HELPER" echo " Patched dnsmasq helper: removed --local-service for Qubes /32 compatibility" fi @@ -74,6 +96,56 @@ if ! pgrep -x dnsmasq >/dev/null 2>&1; then dnsmasq fi +echo "Applying Qubes nftables DNS interception rules..." +ensure_nft_rule ip qubes dnat-dns 'iifname "vif*" udp dport 53 dnat to 10.139.1.1:53' +ensure_nft_rule ip qubes dnat-dns 'iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53' +ensure_nft_rule ip qubes custom-input 'iifname "vif*" ip daddr 10.139.1.1 udp dport 53 accept' +ensure_nft_rule ip qubes custom-input 'iifname "vif*" ip daddr 10.139.1.1 tcp dport 53 accept' +echo " nftables DNS interception ensured" + +RCLOCAL="/rw/config/rc.local" +MARK_START="# === SYS-FIPS DNS START ===" +MARK_END="# === SYS-FIPS DNS END ===" + +if [ ! -f "$RCLOCAL" ]; then + mkdir -p /rw/config + cat > "$RCLOCAL" << 'EOF' +#!/bin/sh +EOF +fi + +sed -i "/$MARK_START/,/$MARK_END/d" "$RCLOCAL" + +cat >> "$RCLOCAL" << 'EOF' +# === SYS-FIPS DNS START === +# Keep dnsmasq alive with .fips forwarding +if [ -f /usr/share/dnsmasq/systemd-helper ]; then + sed -i 's/--local-service//g' /usr/share/dnsmasq/systemd-helper +fi +systemctl restart dnsmasq >/dev/null 2>&1 || true +pgrep -x dnsmasq >/dev/null 2>&1 || dnsmasq >/dev/null 2>&1 || true + +# Qubes DNS interception for downstream AppVMs +nft list chain ip qubes dnat-dns >/dev/null 2>&1 && \ + nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" udp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1 || \ + nft add rule ip qubes dnat-dns iifname "vif*" udp dport 53 dnat to 10.139.1.1:53 >/dev/null 2>&1 || true + +nft list chain ip qubes dnat-dns >/dev/null 2>&1 && \ + nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1 || \ + nft add rule ip qubes dnat-dns iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53 >/dev/null 2>&1 || true + +nft list chain ip qubes custom-input >/dev/null 2>&1 && \ + nft list chain ip qubes custom-input | grep -F 'iifname "vif*" ip daddr 10.139.1.1 udp dport 53 accept' >/dev/null 2>&1 || \ + nft add rule ip qubes custom-input iifname "vif*" ip daddr 10.139.1.1 udp dport 53 accept >/dev/null 2>&1 || true + +nft list chain ip qubes custom-input >/dev/null 2>&1 && \ + nft list chain ip qubes custom-input | grep -F 'iifname "vif*" ip daddr 10.139.1.1 tcp dport 53 accept' >/dev/null 2>&1 || \ + nft add rule ip qubes custom-input iifname "vif*" ip daddr 10.139.1.1 tcp dport 53 accept >/dev/null 2>&1 || true +# === SYS-FIPS DNS END === +EOF + +chmod +x "$RCLOCAL" + echo "" echo "Verifying DNS..." if dig @127.0.0.1 google.com A +short +time=2 >/dev/null 2>&1; then diff --git a/scripts/07-route-appvms.sh b/scripts/07-route-appvms.sh index c536004..7ff6e7e 100755 --- a/scripts/07-route-appvms.sh +++ b/scripts/07-route-appvms.sh @@ -2,7 +2,7 @@ # ============================================================================== # 07-route-appvms.sh # -# Configure IPv6 forwarding and ip6tables rules in sys-fips so AppVMs routed +# Configure IPv6 forwarding and firewall rules in sys-fips so AppVMs routed # through sys-fips can reach FIPS fd00::/8 addresses via fips0. # # Usage: @@ -18,6 +18,23 @@ if [ "$(id -u)" -ne 0 ]; then exit 1 fi +ensure_nft_rule() { + local table="$1" + local chain="$2" + local rule="$3" + + if ! nft list chain "$table" "$chain" >/dev/null 2>&1; then + echo " WARN: nft chain $table $chain not found; skipping: $rule" + return 0 + fi + + if nft list chain "$table" "$chain" | grep -F -- "$rule" >/dev/null 2>&1; then + return 0 + fi + + nft add rule "$table" "$chain" $rule +} + sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null sysctl -w net.ipv6.conf.all.proxy_ndp=1 >/dev/null 2>&1 || true sysctl -w net.ipv6.conf.all.disable_ipv6=0 >/dev/null 2>&1 || true @@ -30,32 +47,56 @@ fi ip -6 route replace fd00::/8 dev fips0 2>/dev/null || true echo " route: fd00::/8 -> fips0" -# idempotent rule refresh -ip6tables -D FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || true -ip6tables -D FORWARD -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null || true -ip6tables -D FORWARD -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null || true +# Prefer nftables (Qubes-native firewall stack) +echo " ensuring nftables forward rules" +ensure_nft_rule ip6 qubes-firewall forward 'iifname "vif*" ip6 daddr fd00::/8 accept' +ensure_nft_rule ip6 qubes-firewall forward 'iifname "fips0" ip6 saddr fd00::/8 accept' -ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT -ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT -ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT - -echo " ip6tables forwarding rules installed" +# Keep ip6tables fallback for templates where FORWARD policy still depends on it. +if command -v ip6tables >/dev/null 2>&1; then + ip6tables -C FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || \ + ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT + ip6tables -C FORWARD -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null || \ + ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT + ip6tables -C FORWARD -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null || \ + ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT + echo " ip6tables forwarding fallback ensured" +fi RCLOCAL="/rw/config/rc.local" MARK_START="# === SYS-FIPS ROUTING START ===" MARK_END="# === SYS-FIPS ROUTING END ===" -if [ -f "$RCLOCAL" ]; then - sed -i "/$MARK_START/,/$MARK_END/d" "$RCLOCAL" +if [ ! -f "$RCLOCAL" ]; then + mkdir -p /rw/config + cat > "$RCLOCAL" << 'EOF' +#!/bin/sh +EOF fi +sed -i "/$MARK_START/,/$MARK_END/d" "$RCLOCAL" + cat >> "$RCLOCAL" << 'EOF' # === SYS-FIPS ROUTING START === sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null 2>&1 sysctl -w net.ipv6.conf.all.disable_ipv6=0 >/dev/null 2>&1 -ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null -ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null -ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null +ip -6 route replace fd00::/8 dev fips0 >/dev/null 2>&1 || true + +# nftables forward allow rules for AppVM <-> fips0 traffic +nft list chain ip6 qubes-firewall forward >/dev/null 2>&1 && \ + nft list chain ip6 qubes-firewall forward | grep -F 'iifname "vif*" ip6 daddr fd00::/8 accept' >/dev/null 2>&1 || \ + nft add rule ip6 qubes-firewall forward iifname "vif*" ip6 daddr fd00::/8 accept >/dev/null 2>&1 || true + +nft list chain ip6 qubes-firewall forward >/dev/null 2>&1 && \ + nft list chain ip6 qubes-firewall forward | grep -F 'iifname "fips0" ip6 saddr fd00::/8 accept' >/dev/null 2>&1 || \ + nft add rule ip6 qubes-firewall forward iifname "fips0" ip6 saddr fd00::/8 accept >/dev/null 2>&1 || true + +# ip6tables fallback +if command -v ip6tables >/dev/null 2>&1; then + ip6tables -C FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 2>/dev/null || ip6tables -I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT + ip6tables -C FORWARD -d fd00::/8 -o fips0 -j ACCEPT 2>/dev/null || ip6tables -I FORWARD 2 -d fd00::/8 -o fips0 -j ACCEPT + ip6tables -C FORWARD -s fd00::/8 -i fips0 -j ACCEPT 2>/dev/null || ip6tables -I FORWARD 3 -s fd00::/8 -i fips0 -j ACCEPT +fi # === SYS-FIPS ROUTING END === EOF diff --git a/scripts/test-connectivity.sh b/scripts/test-connectivity.sh index f74863d..753b9ad 100755 --- a/scripts/test-connectivity.sh +++ b/scripts/test-connectivity.sh @@ -70,6 +70,42 @@ run_local_tests() { fail "dnsmasq not running" fi + if command -v nft >/dev/null 2>&1; then + if nft list chain ip qubes dnat-dns >/dev/null 2>&1; then + pass "nft chain exists: ip qubes dnat-dns" + if nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" udp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1; then + pass "nft DNS DNAT rule present (UDP 53)" + else + fail "nft DNS DNAT rule missing (UDP 53)" + fi + if nft list chain ip qubes dnat-dns | grep -F 'iifname "vif*" tcp dport 53 dnat to 10.139.1.1:53' >/dev/null 2>&1; then + pass "nft DNS DNAT rule present (TCP 53)" + else + fail "nft DNS DNAT rule missing (TCP 53)" + fi + else + skip "nft chain missing: ip qubes dnat-dns" + fi + + if nft list chain ip6 qubes-firewall forward >/dev/null 2>&1; then + pass "nft chain exists: ip6 qubes-firewall forward" + if nft list chain ip6 qubes-firewall forward | grep -F 'iifname "vif*" ip6 daddr fd00::/8 accept' >/dev/null 2>&1; then + pass "nft forward rule present (vif* -> fd00::/8)" + else + fail "nft forward rule missing (vif* -> fd00::/8)" + fi + if nft list chain ip6 qubes-firewall forward | grep -F 'iifname "fips0" ip6 saddr fd00::/8 accept' >/dev/null 2>&1; then + pass "nft forward rule present (fips0 -> fd00::/8)" + else + fail "nft forward rule missing (fips0 -> fd00::/8)" + fi + else + skip "nft chain missing: ip6 qubes-firewall forward" + fi + else + skip "nft not installed" + fi + if dig @127.0.0.1 test.fips AAAA +time=2 >/dev/null 2>&1; then pass ".fips DNS query path reachable" else