mirror of
https://github.com/jmcorgan/fips.git
synced 2026-10-05 19:18:25 +00:00
Johnathan Corgan
1e111fe044
Any host that can reach the LAN resolver could ask for one new .fips name after another, and each got a virtual-IP mapping until the pool's 65,535 addresses ran out. Every mapping also adds to the cost of each NAT rebuild, each pool tick and shutdown. The pool now refuses a new name once it holds 1000 live mappings, and admits new names from a token bucket of 50 that refills at 10 per second. Both checks sit after the return for a name that already has a mapping, so names in use keep resolving when new ones are refused. The ceiling is checked first, so a refusal there takes no token and is always reported as the ceiling. A token is taken only once an address has been taken from the free list, so an exhausted pool costs none. Each limit has its own PoolError variant, and the "Pool allocation failed" warning names the one that refused. VirtualIpPool::new keeps its signature and uses the compiled-in limits; with_limits and allocate_at, which takes the instant that drives the refill, let the unit tests set small limits and a clock. The limits come from a flood of the gateway suite's gateway to 500, 1000 and 2000 live mappings with no limits in place. The measurement record is kept separately. Figures: - New mappings per second, over 100 creations: 215 for the first 100, 63 at 500, 47 at 1000, 32 at 2000. - NAT rebuild, median/max over the 20 adds up to the count: 17.5/26.5 ms at 500, 22.2/34.3 ms at 1000, 32.2/38.3 ms at 2000. - Pool tick: 24 to 32 us at 500, 48 to 53 us at 1000, 105 to 115 us at 2000, beside a conntrack read of 180 to 300 us. - Shutdown at 2000 mappings: 2.4 s. These are a best case for router hardware: they come from a container on a development host, not a router, and that host has no /proc/net/nf_conntrack, so the tick figures include no conntrack parsing. The ceiling is half the largest count measured. At 1000 a rebuild took about 22 ms and shutdown about 1.2 s. The rate is below the unthrottled creation rate at every count measured, so the bucket rather than the rebuild sets how fast a flood can fill the pool. From empty that now takes about 95 s; the unthrottled run passed 1000 in about 16 s. Ten rebuilds a second at the ceiling cost about a fifth of the gateway's single runtime thread on that host. The 2000 figures exist only in the measurement, since the committed phase stops at the ceiling. To measure and to keep measuring, the "Added DNAT/SNAT rules" and "Removed DNAT/SNAT rules" debug lines now carry the mapping count after the change and the rebuild's duration in microseconds, and are also emitted, with the error, when a rebuild fails, so a failure at some count leaves a record of that count. The pool tick logs a debug line with the mapping count and the durations of the conntrack read and of the tick. The suite's gateway logs at info because RUST_LOG=info overrides the entrypoint's --log-level debug, so the gw-gateway service now enables debug for the NAT manager and the gateway binary only. The gateway suite's last phase is now the regression. It restarts the gateway with mappings that outlive the phase and reads the limits from pool.rs. It fills the pool to the ceiling with the readiness probe's mapping plus ceiling - 1 new names, retrying rate refusals, then asks once each for 20 more. It asserts all 20 get SERVFAIL, the live count equals the ceiling, at least 20 ceiling refusals are logged, the rate limit refused during the fill and not after it, the probe's name keeps its address, nothing is reclaimed, no NAT or proxy NDP failure is logged, and no 10 s window of the fill holds more than burst + 10 x rate creations. It also reports rebuild durations at 500 and 1000, ticks as they fall, and the shutdown time at the ceiling. On a tree with these assertions but no checks in allocate the phase fails five of them: 1020 mappings, 0 SERVFAIL, no ceiling or rate refusals, and 713 creations in one 10 s window. With the limits it passes in about 140 s, with rebuilds of 17.8/24.5 ms at 500 and 20.9/33.9 ms at 1000 and a 1.2 s shutdown at the ceiling. The NAT batch phase now also goes through the rate limit, so its driver retries refusals; it shares the restart, readiness gate and DNS driver with the new phase instead of carrying its own copies.