fips/docs/design/fips-gateway.md

FIPS Gateway

The FIPS gateway lets unmodified IPv6 hosts on a LAN exchange traffic with the mesh without running any FIPS software themselves. It is a niche feature — most operators will never enable it. The gateway runs most conveniently on a system that is already providing network services (DHCP, DNS, RA) to a LAN segment, since hosts on that segment already get IP assignment and a default route from that box. The canonical example is an OpenWrt-based WiFi access point: every client that associates with the AP already has the AP as default router and DNS server, which is exactly the placement the gateway needs. The OpenWrt ipk ships with the gateway: block of /etc/fips/fips.yaml pre-populated and the integration glue (dnsmasq forwarding, RA route for the virtual pool, global-scope IPv6 prefix on br-lan) automated by the init script — packaging/openwrt-ipk/files/etc/init.d/fips-gateway. The operator only needs to enable and start the service. Running the gateway on a non-OpenWrt LAN-edge host (a Linux router/server, for example) is technically possible but requires manual integration: distributing a route to the virtual-IP pool, wiring DNS forwarding so LAN clients send .fips queries to the gateway, configuring sysctls and capabilities. That path is supported but tedious; it is the secondary path.

The feature has two halves that share common machinery and have their own unique parts.

The outbound half carries traffic from LAN to mesh. A non-FIPS LAN workstation resolves <npub>.fips (or a .fips host alias) via the gateway's DNS proxy, which returns a virtual IPv6 address from a managed pool. The kernel routes the LAN packet to that virtual IP via a route to the pool CIDR (RA-advertised, statically distributed, or on-link via the default route). The gateway runs nftables NAT so the packet appears on the mesh as if it had originated from the gateway's own FIPS identity: prerouting DNAT rewrites the destination from the virtual IP to the real fd00::/8 mesh address, and postrouting masquerade rewrites the source from the LAN host's address to the gateway's fips0 address. Return traffic follows the conntrack reverse path back to the originating LAN host, with postrouting SNAT restoring the virtual IP as source so the client sees a response from the address it connected to.

The inbound half carries traffic from mesh to LAN. A configuration entry in gateway.port_forwards[] exposes a LAN service (host:port) on a port of the gateway's mesh-side fips0 address. Mesh peers reach it as <gateway-npub>.fips:<listen_port>. A prerouting DNAT rule keyed on (iif=fips0, l4proto, dport) rewrites the destination to the LAN target; a LAN-side masquerade in postrouting rewrites the mesh peer's source so the LAN target sees a reachable LAN address and conntrack steers replies back through the gateway. This is the inverse of port-forwarding on a conventional NAT router.

The two halves are independent and can be configured separately. Inbound port-forwards work without any outbound configuration (just a port-forward list and the table); outbound works without any inbound forwards. They share the same nftables table, the same binary, the same control socket, and the same atomic-rebuild strategy. That shared machinery is what makes them halves of one feature rather than two separate features.

Architecture

The fips-gateway Service

The gateway is a separate binary, fips-gateway, not part of the FIPS daemon. It reads the same /etc/fips/fips.yaml the daemon reads (via --config, or the standard search path), but acts on the gateway.* block. It needs CAP_NET_ADMIN to install nftables rules, manage proxy NDP entries, and add the pool route. The CLI is documented in ../reference/cli-fips-gateway.md.

The gateway connects to the daemon indirectly. The outbound half forwards .fips DNS queries to the daemon's built-in resolver (default [::1]:5354); the daemon resolves the name to a mesh address and primes its identity cache as a side effect. The inbound half does not require any daemon plumbing at all — packets that arrive on fips0 after the daemon's TUN injection path are matched by the nftables rules on fips0 ingress. There is no shared memory, no IPC channel, and no startup ordering coupling beyond "the daemon's DNS responder must be reachable before the gateway starts serving LAN queries", which the gateway enforces with a bounded reachability probe at startup.

nftables Table Layout

All gateway rules live in a single nftables table, inet fips_gateway, with two chains:

• prerouting — type nat hook prerouting priority dstnat (-100), for both LAN→mesh DNAT (per virtual-IP mapping) and mesh→LAN DNAT (per port-forward).
• postrouting — type nat hook postrouting priority srcnat (100), for both the always-on oifname fips0 masquerade, the per-mapping return-path SNAT, and (when any port-forward is configured) the LAN-side masquerade for inbound traffic.

The table is rebuilt atomically on every change, in one netlink batch that the kernel applies as a single transaction: add the table, delete it, add it again, then the chains and the full rule set. Because the delete and the recreate share one transaction, the table never leaves the packet path; the leading add gives the delete a target when no table exists yet, and a batch the kernel refuses leaves the previous table in place. Rebuilding the whole table avoids reliance on kernel rule-handle tracking, which the rustables crate does not expose. The table holds one always-on masquerade, two rules per live outbound mapping (at most 1000 mappings), one rule per inbound forward, and one extra masquerade when any forward is present.

Control Socket

fips-gateway exposes a Unix-domain control socket at /run/fips/gateway.sock (root:fips, mode 0770) with two commands: show_gateway and show_mappings. The protocol is the same line-delimited JSON used by the daemon's control socket. The shapes are documented in the Gateway command catalog. There is no fipsctl gateway subcommand; clients (including fipstop's gateway view) talk to the socket directly.

Diagram

                           LAN clients
                              │
        DNS query (.fips)     │     IPv6 packet
        for outbound          │     to virtual IP
                              │     or mesh peer
                              ▼
            ┌───────────────────────────────────┐
            │           fips-gateway            │
            │                                   │
            │  ┌──────────────┐  ┌───────────┐  │
            │  │   DNS proxy  │  │  Virtual  │  │
            │  │ ([::1]:5365) │─▶│  IP pool  │  │
            │  │   .fips only │  │ (state    │  │
            │  └──────┬───────┘  │  machine) │  │
            │         │          └─────┬─────┘  │
            │         │                │        │
            │  forward to              │ pool   │
            │  daemon resolver         │ events │
            │  ([::1]:5354)            ▼        │
            │         │          ┌───────────┐  │
            │         │          │   NAT     │  │
            │         │          │  manager  │  │
            │         │          │ (rebuild  │  │
            │         │          │  inet     │  │
            │         │          │  fips_    │  │
            │         │          │  gateway) │  │
            │         │          └─────┬─────┘  │
            │         │                │        │
            │         │          ┌─────▼─────┐  │
            │         │          │   net     │  │
            │         │          │  setup    │  │
            │         │          │ (proxy    │  │
            │         │          │  NDP, lo  │  │
            │         │          │  route)   │  │
            │         │          └───────────┘  │
            │         │                         │
            │         │   control socket        │
            │         │   /run/fips/            │
            │         │   gateway.sock          │
            └─────────┼─────────────────────────┘
                      │
                      ▼
              FIPS daemon resolver
              ([::1]:5354)
                      │
                      ▼
                fips0 TUN interface
                      │
                      ▼
                  the mesh

The DNS proxy and the virtual IP pool are exclusive to the outbound half. The NAT manager and the kernel-side machinery (nftables table, fips0 and LAN interfaces, conntrack) are shared. The inbound half contributes per-port-forward rules to the same table without involving the DNS proxy or the pool.

The Outbound Half (LAN → Mesh)

DNS Resolution Flow

1. A LAN client sends a DNS query to the gateway's listener (default [::1]:5365, configurable via gateway.dns.listen). The default is not 5353, the mDNS port, which the daemon's LAN rendezvous and other mDNS responders hold. It is loopback-only on an unprivileged port: the canonical deployment has another resolver on the host (dnsmasq, systemd-resolved, BIND) holding port 53 and forwarding .fips queries to the gateway over loopback. Operators on a host without a pre-existing resolver on 53 can override the listen value to "[::]:53" to let LAN clients query the gateway directly.
2. If the question is not for a .fips domain, the gateway replies REFUSED. The proxy is intentionally narrow — it does not resolve public DNS, and the LAN's primary resolver should hold port 53 on the gateway host (the OpenWrt init script wires dnsmasq to forward .fips queries to the loopback listener automatically).
3. The gateway forwards the query to the daemon resolver (gateway.dns.upstream, default [::1]:5354). The daemon must match: an IPv6 socket bound to [::1] does not accept v4-mapped traffic, so a 127.0.0.1:5354 upstream cannot reach a daemon bound on [::1]:5354.
4. If the daemon is unreachable or times out (5 s), the gateway replies SERVFAIL. If the daemon answers with an error such as NXDOMAIN, the gateway relays that response code; if it answers without an AAAA record, the gateway replies SERVFAIL.
5. The gateway extracts the AAAA (fd00::/8) record from the daemon's response. This resolution primes the daemon's identity cache as a side effect — a prerequisite for fips0 routing, because the daemon needs the cache entry to map the mesh address back to a NodeAddr for forwarding.
6. If the client asked for AAAA or ANY, the gateway allocates a virtual IP from the pool for that mesh address (idempotent: an existing mapping is reused and its TTL refreshed). Any other query type refreshes an existing mapping's TTL, creates nothing, and is answered with NODATA.
7. If a new mapping was created, the pool emits MappingCreated, which the main loop turns into add_mapping calls on the NAT manager and add_proxy_ndp on the network setup.
8. The gateway returns an AAAA response containing the virtual IP, with the configured TTL (default 60 s).

Virtual IP Pool

The pool allocates IPv6 addresses from a required CIDR (commonly fd01::/112). Each address maps to one mesh destination, keyed by NodeAddr rather than by hostname — different .fips aliases for the same node share a virtual IP. Address 0 (the network-equivalent) is reserved; the rest are allocatable. The pool is capped at 2^16 addresses regardless of prefix length, to bound memory.

The pool tracks state per address:

Allocated ──→ Active ──→ Draining ──→ Free
    │                       ▲
    └───────────────────────┘
      (TTL expired, no sessions)

Draining ──→ Active      traffic resumes before the grace period ends
Draining ──→ Allocated   a DNS query for the name

State	Meaning
Allocated	DNS query created or renewed the mapping; no NAT sessions yet.
Active	Conntrack reports at least one session for this virtual IP.
Draining	TTL has expired with no sessions; the grace period is running.
Free	Reclaimed and available for new allocations.

Transitions:

• Allocated → Active: conntrack sessions count goes above zero.
• Allocated → Draining: TTL expires before any session is observed.
• Active → Draining: TTL expires after the last session ends. Sessions refresh the mapping at every tick, so a mapping in use does not drain.
• Draining → Active: conntrack reports a session again before the grace period ends. The next drain starts a fresh grace period.
• Draining → Allocated: a DNS query for the name, with or without an address in the answer. The client may now hold a fresh TTL, so reclamation is cancelled: the mapping gets the full TTL and, if it stays idle, a fresh grace period.
• Draining → Free: the grace period has elapsed since draining began with no session seen.

Timing:

• TTL (gateway.dns.ttl, default 60 s) is both the DNS TTL returned to the client and the mapping's idle lifetime. A DNS query for a mapped name refreshes the mapping's idle clock, and so do conntrack sessions at each tick.
• Grace period (gateway.pool_grace_period, default 60 s) is the dwell time after the last session ends before the address is recycled. It prevents immediate reuse from confusing hosts with cached DNS responses.
• Tick interval: the pool re-evaluates state every 10 s.

Active session counts come from /proc/net/nf_conntrack, or, on a kernel without that file, from a dump of the IPv6 conntrack table over NETLINK_NETFILTER, the request conntrack -L makes. The choice is made on every tick, and the source is logged once at startup. Either way an entry counts once toward each distinct IPv6 destination among its original and reply tuples, so an entry counts as a session of a virtual IP whose address is its original destination.

A new mapping is refused, and the query answered SERVFAIL, when the pool is exhausted, when it already holds 1000 live mappings, or when the new-mapping rate limit is spent (a bucket of 50 that refills at 10 per second). A name that already has a mapping keeps resolving while new names are refused. Existing mappings are never evicted prematurely — the correctness of in-flight sessions takes precedence over fresh allocations.

NAT Pipeline (Outbound)

Three rule classes in inet fips_gateway together implement the LAN→mesh path:

Prerouting DNAT (per mapping) rewrites the destination from the virtual IP to the corresponding mesh address:

match:  nfproto ipv6 && ip6 daddr == <virtual_ip>
action: dnat to <mesh_addr>

After DNAT, the kernel routes the packet through fips0 via the standard routing table.

Postrouting masquerade (oifname fips0) rewrites the source of all traffic exiting via fips0 to the gateway's own fips0 address:

match:  oifname == "fips0"
action: masquerade

This rule is critical. Without it, LAN client source addresses (for example fd02::20 from the LAN's RA-advertised prefix, or virtual addresses from another forwarding domain) would appear as the source on the mesh. Those addresses are meaningless to mesh nodes, so return traffic would be black-holed. Masquerade ensures all mesh traffic appears to originate from the gateway's own FIPS identity.

Postrouting SNAT (per mapping) rewrites the source of return traffic from the mesh address back to the virtual IP:

match:  nfproto ipv6 && ip6 saddr == <mesh_addr>
action: snat to <virtual_ip>

Without it, the LAN client would see replies from the raw fd00::/8 mesh address rather than from the virtual IP it had originally connected to, breaking application-layer assumptions about the destination address.

Network Requirements (Outbound)

The gateway host needs IPv6 forwarding enabled (net.ipv6.conf.all.forwarding=1), proxy NDP enabled on the LAN interface, CAP_NET_ADMIN for fips-gateway, and a local <pool-cidr> dev lo route so the kernel accepts packets to the pool as locally owned and runs them through the NAT chains. LAN clients need a route to the pool via the gateway and DNS resolution that forwards .fips queries there. On OpenWrt the init script handles all of this; on other Linux hosts the operator handles it manually. Full setup is documented in ../how-to/deploy-gateway.md.

The Inbound Half (Mesh → LAN)

Configuration Shape

Inbound port-forwards live in gateway.port_forwards[]. Each entry is a triple:

Field	Type	Notes
listen_port	u16	Port on the gateway's fips0 address. Must be non-zero.
proto	tcp | udp	Match protocol.
target	[ipv6]:port	LAN destination. IPv4 targets are rejected at parse time by SocketAddrV6.

Validation runs at startup and on every config reload: (listen_port, proto) must be unique across the list, and zero listen ports are rejected. Forwards are independent of outbound configuration: a gateway with no pool consumers can still expose inbound services (the pool route and DNS proxy still run, since they are part of the same binary, but they sit idle).

NAT Pipeline (Inbound)

For each port-forward, a single prerouting DNAT rule matches mesh-originated traffic landing on the gateway's fips0 address and rewrites it to the LAN target:

match:  iifname == "fips0" && nfproto ipv6
        && l4proto == <tcp|udp> && th dport == <listen_port>
action: dnat to <target_ip>:<target_port>

The match clause is deliberately narrow:

• iifname == "fips0" restricts the rule to traffic that arrived from the mesh. LAN-side ingress is never subject to inbound forwarding.
• nfproto ipv6 is enforced both here and at config-load time (SocketAddrV6 rejects IPv4 targets); FIPS is IPv6-only end to end.
• l4proto + dport narrows the match to one (listen_port, proto) pair per rule. Unique-tuple validation ensures no two rules contend for the same packet.

When any port-forward is configured, a single LAN-side masquerade is added to postrouting:

match:  iifname == "fips0" && oifname == <lan_interface>
        && nfproto ipv6
action: masquerade

Without this rule, the LAN target would attempt to reply directly to the mesh peer's fd00::/8 source address, which is not reachable on the LAN. Masquerade rewrites the source to the gateway's LAN-side address so the target sees a reachable peer and conntrack routes the reply back through the gateway.

This LAN-side masquerade is independent of the oifname fips0 masquerade in the outbound pipeline; the two have disjoint match clauses (different iifname/oifname combinations) and coexist without interaction when both directions are active.

Independence From Outbound

The inbound half does not require:

• A virtual-IP pool. Mesh peers connect directly to the gateway's own fips0 address, which the FIPS daemon already owns.
• DNS resolution. Mesh peers reach the gateway as <gateway-npub>.fips:<port> using their own resolver (or a numeric mesh address); the gateway's DNS proxy is not in the path.
• A daemon-side identity cache for the LAN target. The target is a LAN-side IPv6 address, not a mesh address; no fd00::/8 lookup happens for it.

A gateway configured with port-forwards but with no LAN clients ever issuing .fips DNS queries will have an empty pool and zero outbound mappings, but its inbound forwards work normally. The inverse is also true: a gateway that serves only outbound LAN→mesh traffic has zero entries in the port-forwards list and no LAN-side masquerade.

Atomic Table Rebuild (Common)

Both halves contribute rules to the same inet fips_gateway table, and that table is rebuilt as one unit on every state change — mapping added, mapping removed, port-forwards updated. The rebuild sequence is:

1. Add the table (which succeeds whether or not it exists), delete it, and add it again, so the delete always has a target.
2. Add the prerouting and postrouting chains; the always-on oifname fips0 masquerade; per-mapping DNAT/SNAT rules for every live pool entry; per-port-forward DNAT rules; the LAN-side masquerade if any port-forwards exist.
3. Send all of it as one batch, which the kernel applies as a single transaction. Only the last message before the batch end requests an acknowledgement, the socket's send buffer is sized to the batch, and a batch too large for any send buffer is refused before it is sent. A batch the kernel refuses leaves the previous table in place; the failure is logged, and the gateway keeps its record of the change, so the next rebuild that succeeds applies it.

The rustables crate does not expose rule-handle tracking, so incremental update of individual rules is not available. Atomic rebuild was chosen for simplicity and correctness: it eliminates an entire class of partial-update inconsistency bugs at the cost of repeating the (cheap) rule construction on every change. The total rule count is bounded by the live-mapping ceiling (1000 mappings, two rules each) and the port-forward count.

Configuration Reference

The full gateway.* block — pool CIDR, LAN interface, DNS listen/upstream/TTL, pool grace period, conntrack timeouts, and inbound port-forwards — is documented in the Gateway section of the configuration reference. The same block governs both halves; fields specific to one half (pool, dns.* for outbound; port_forwards[] for inbound) are simply unused when the other half is not in play.

Operations and Troubleshooting

• ../tutorials/deploy-fips-gateway.md — end-to-end walkthrough on OpenWrt.
• ../how-to/deploy-gateway.md — recipe for non-OpenWrt Linux hosts and inbound-port-forwarding configuration.
• ../how-to/troubleshoot-gateway.md — diagnostic recipes (DNS failures, ping working but TCP not, conntrack inspection, pool exhaustion, port-53 conflicts, port-forward verification).
• ../reference/cli-fips-gateway.md — command-line interface.
• ../reference/control-socket.md — show_gateway and show_mappings commands.

Security Considerations

Outbound

• LAN trust boundary. The DNS listener and the virtual-IP pool are reachable by every host on the LAN. Any LAN host that can resolve .fips and route to the pool CIDR can reach mesh destinations. There is no per-client authentication; access restriction is a network-level concern, enforced with firewall rules on the LAN interface or on the gateway host itself.
• Identity masking. All outbound LAN traffic appears on the mesh under the gateway's own FIPS identity. Mesh nodes cannot determine which LAN host originated a connection. This provides privacy for LAN hosts but means the gateway's reputation covers all of its clients — and that abusive behavior from one LAN host is attributed to the gateway, not to the host.
• Plaintext between client and gateway. Traffic between the LAN client and the gateway is unencrypted at the IP layer. FIPS encryption (FSP) protects the segment between the gateway and the destination mesh node; application-layer encryption (TLS, SSH, Noise) is the only thing that provides true end-to-end protection through the gateway.
• Pool addresses are ephemeral. Virtual IPs are allocated dynamically and recycled. They are not authenticated and not bound to client identity — a LAN host connecting to a virtual IP is trusting the gateway's recent DNS response.
• DNS upstream trust. The outbound half's correctness depends on the FIPS daemon's resolver returning honest fd00::/8 answers; a compromised daemon could redirect LAN clients to arbitrary mesh nodes.

Inbound

• Port exposure. Each entry in port_forwards[] exposes the matched (listen_port, proto) on the gateway's mesh-side address to every reachable mesh peer. Inbound port-forwards are not gated by any peer ACL beyond what FMP normally enforces; treat them with the same care as a public-internet port forward.
• Mesh peer trust. The LAN target sees connections that have been masqueraded to the gateway's LAN address. The target cannot distinguish one mesh peer from another, and there is no authenticated peer identity available to the LAN target — any application-layer authentication or rate-limiting must run on the target itself.
• Return-path masquerade exposes the gateway's LAN address. The LAN-side masquerade rewrites the mesh peer's source to the gateway's LAN address. A malicious or buggy LAN target can use this to send unsolicited traffic back at the gateway, or to probe other LAN hosts via the gateway's network position; LAN segmentation (VLANs, host firewalls) is the right control.

Common

• No client identity verification. The gateway authenticates neither LAN clients nor mesh peers beyond what the underlying layers already do — fips0 ingress carries an FSP-authenticated payload, the LAN side is whoever the LAN admits.

References

• fips-ipv6-adapter.md — IPv6 adapter and TUN interface design.
• fips-architecture.md — protocol layer architecture.
• fips-concepts.md — protocol overview.
• ../reference/configuration.md — configuration reference.
• ../reference/cli-fips-gateway.md — fips-gateway CLI.
• ../reference/control-socket.md — control-socket protocol and command catalog.
• ../how-to/deploy-gateway.md — gateway host and LAN client setup.
• ../how-to/troubleshoot-gateway.md — diagnostic recipes.
• ../tutorials/deploy-fips-gateway.md — OpenWrt walkthrough.