mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
Johnathan Corgan
d21f818659
A test that greps the daemon's log for a message the daemon stopped emitting does not fail. It stops observing, and an assertion built on it — especially one expecting a count of zero — then passes because nothing can be seen rather than because nothing happened. Several findings have come from that one class, so it is now checked mechanically. testing/check-log-strings.py extracts the strings test code matches against daemon log text and requires each to exist in src/. It reads four shapes: python `"..." in line` tests, the first argument of the shell log helpers, bash associative-array pattern tables, and greps whose input is daemon log output. Patterns carry regex syntax, so each is reduced to the longest literal run of every alternation branch, with escaping resolved in the same pass — `\.` is a literal dot and `.` is a wildcard, and conflating them would let a pattern match text that is not there. Scoping is by what a grep READS, not by what its file mentions. A suite greps its own analyzer output, fipsctl JSON, Tor's log and ping output, and none of those have to correspond to a string in src/; scoping on the file produced 34 false positives. Strings that legitimately do not come from src/ are named in ALLOWED with a reason, so the exceptions are reviewable rather than invisible. It found six dead strings, three of which were not previously known: a second copy of "Excessive decrypt failures" in the interop pattern table, a dead "Handshake error" branch inside an alternation whose other branches still matched, and "bootstrap failed" in the NAT suite. All six are corrected to what the daemon emits, or dropped where the surviving branch already covers the case. Verified by breaking it two ways and confirming it reports. Also here, since it is the same failure shape one layer down: the shared log library returned an empty string for a container whose logs it could not read, so analysing a missing container reported no panics, no errors and no sessions, and exited 0 — indistinguishable from a clean run. It now reports which containers could not be read and raises, matching the fix already landed in the chaos copy.