Each peer must be sent the union of all other peers' inbound filters,
excluding its own contribution. Building that per recipient rebuilt
the whole peer-filter map and re-ORed it once for every peer, so a
tick that announced to R peers did R kilobyte-scale map builds and
R by T merges. At 240 peers this was 20.6 ms per tick, roughly half
the tick body, and it was steady work rather than a tail: the
per-interval maximum had a median of 34.5 ms.
Replace it with a prefix and suffix union sweep that produces every
target's filter in one pass, T merges instead of R by T. The packet
path gets the same treatment, since marking changed peers had the
identical shape once per inbound announce.
The result is exactly equal, not approximately. Merging is a bytewise
OR, so regrouping the unions cannot change the outcome, and a filter
whose size does not match is rejected before any byte is touched, at
every merge site in both the old and new arrangement. Every
accumulator here is default-sized, so an odd-sized peer filter is
skipped in the new code exactly where it was skipped in the old.
Cadence, the debounce, the sequence rule and the fill-ratio cap are
untouched. A sequence number is still drawn after the debounce
re-check and before encoding, so a suppressed peer still consumes
none and an encode failure still burns one.
Known trade-off, measured rather than assumed: the sweep does its
full O(T) work regardless of how many peers are ready, so a tick that
announces to only one or two peers now costs about twice what it did.
Break-even is around three ready peers, and the saving above that
grows without bound. The marking path is a pure win, since it always
targets every peer.
The false-positive-rate primitive (fill ratio raised to the hash count) was
inlined at three sites. Hoist it into a BloomFilter::fpr() method and call it
from all three; the threshold/policy logic stays at the call sites. Behavior
and result bytes are unchanged.
Migrate the v1 bloom filter subsystem into src/proto/bloom/, matching the
discovery/routing/fmp/mmp/stp reference layout and completing the proto/
relocation series for the data/wire subsystems.
- wire.rs: the FilterAnnounce (0x20) codec, moved from protocol/filter.rs
- core.rs: the pure BloomFilter algorithm (hash/insert/contains/merge/
as_bytes/from_bytes/estimated_count), moved from bloom/filter.rs
- state.rs: BloomState (per-peer inbound store, compute_outgoing_filter,
the injected-clock send debounce), moved from bloom/state.rs
- limits.rs: the v1 sizing constants
- mod.rs: module wiring + the BloomError enum
- tests/: the unit suite split by target (core/state/wire), no inline tests
no_std+alloc hygiene: core::fmt over std::fmt, the tracing dependency
dropped from the pure filter, and std collections replaced with
BTreeMap/BTreeSet (NodeAddr: Ord) for deterministic iteration. The pure
filter combination stays a BloomState method; the two irreducible shell
gathers (peer_inbound_filters, build_filter_announce) remain in the async
shell. Wire bytes and observable behavior are unchanged; full local CI
green (36/36) including the bloom-storm chaos gate.
Store node counters in an atomic metric registry read through &self, and
introduce a shared NodeContext bundle holding the effectively-immutable
fields (config, identity, startup epoch, capability limits). Source the
immutable config and identity reads across the receive hot path, the
handshake/session/mmp/encrypted state machines, and the discovery, tree,
bloom, retry, and lifecycle modules through the context accessors rather
than direct field reads. The Node fields and the context are rebuilt in
lockstep at every mutation site.
Introduce a typed RejectReason enum and a NodeStats::record_reject
dispatch so every receive-path rejection-and-return site bumps a
machine-readable per-subsystem counter while keeping its operator-facing
log line. The top-level variants mirror the existing NodeStats subsystem
split (Tree, Bloom, Discovery, Forwarding) and add Handshake, Session,
Mmp, and Transport categories; HandshakeStats, SessionStats, and MmpStats
are new sub-stats.
Wired clusters: tree and MMP outbound sign-failure; the FSP session
unknown-session and state-machine cluster; the Noise IK handshake
state-machine cluster (msg1/msg2); and the decode / crypto / cap /
semantic tail across bloom, discovery, forwarding, mmp, and tree. The
TreeStats::ancestry_invalid counter, present since the scaffold but never
incremented, is now bumped from the validate_semantics ancestry rejection.
Several handshake, MMP, tree, and discovery paths that previously had no
counter at all are now counted, including the send_lookup_response
no-route drop (DiscoveryStats::resp_no_route).
Existing direct counters at the bloom / discovery / forwarding sites are
retained alongside the new dispatch while the rollout is in progress (the
bloom_poison tests expect the transitional +2 delta); a later change
collapses the duplicate increment.
A leaf node's my_coords could go stale after an upstream
mid-chain ancestor swap, leaving non-parent destinations with
100% loss until either the parent or the depth also changed.
The broadcast gate in handle_tree_announce's
`else if !is_root && parent_id == from` branch compared only
(root, depth). A swap that altered an interior ancestor without
changing root or depth (e.g. A->B->C reorganizing to A->D->C
while keeping (A, depth=2)) was silently dropped one hop below
the swap node. Downstream nodes' coords paths then drifted from
the real tree topology, defeating greedy distance routing for
any destination whose path crossed the unrepresented section.
Widen the gate to compare the full my_coords.node_addrs() so
mid-chain swaps propagate to leaves the same way root/depth
changes already did. The gate body's bloom-marking is adjusted
in step so the wider gate doesn't generate empty/redundant
FilterAnnounces to every peer on every mid-chain swap
propagation: mark_changed_peers replaces
mark_all_updates_needed in the gate body (parent_id is
unchanged in this branch, so outgoing filter content is
typically unchanged, and mark_changed_peers correctly marks
zero peers in that case), and the unconditional
mark_update_needed(*from) at the top of handle_tree_announce
is removed (bloom exchange initiation is already handled at
handshake completion, and ongoing content changes are picked
up naturally by mark_changed_peers in handle_filter_announce
when peers send their next filter).
Required surface change: peer_inbound_filters in
src/node/bloom.rs upgraded from private to pub(super) so the
gate body can call it.
Verified in a 6-node depth-4 docker reproduction under
tc/netem-induced parent flapping: a depth-4 leaf's
ancestry_changed counter advances with upstream parent
switches while bloom_sent stays at zero matching the
pre-change steady-state baseline.
A malformed FilterAnnounce whose fill ratio produces an implausibly
high false-positive rate is mostly useless for routing and, once
merged into our outgoing filter via bitwise OR, propagates the
saturated state to tree peers one hop per announce tick. A saturated
filter also made estimated_count() return f64::INFINITY, which
compute_mesh_size summed into its cached estimate.
handle_filter_announce now rejects inbound FilterAnnounce whose
derived FPR exceeds `node.bloom.max_inbound_fpr` (new config field,
default 0.05 ≈ fill 0.549 at k=5). Rejection is silent on the wire,
logs at WARN, and increments a new `bloom.fill_exceeded` counter. The
peer's prior stored filter and filter_sequence are left unchanged so
a single rejected announce does not wipe the peer's existing
contribution to aggregation.
After a successful outgoing FilterAnnounce send, a rate-limited WARN
fires if our own filter's FPR exceeds the same cap, surfacing
aggregation drift. Limited to once per 60 seconds via a new
Node.last_self_warn field.
BloomFilter::estimated_count() now takes max_fpr and returns
Option<f64>. Returns None for saturated filters (regardless of cap)
or when the filter's FPR exceeds max_fpr. Callers updated: debug
logs render None as "—", the Debug impl uses f64::INFINITY as "no
cap" and prints "saturated" instead of inf, control-socket JSON
emits null, and compute_mesh_size propagates None into the already-
Option<u64> estimated_mesh_size field.
Gate peer_inbound_filters() to only collect from tree peers (parent
and children), so outgoing filter computation merges only tree-sourced
information. All peers still receive FilterAnnounce messages and store
filters locally for routing queries — the restriction is only on what
gets merged into outgoing filters.
This prevents bloom filter saturation where mesh shortcuts cause every
node's filter to converge toward the full network. With tree-only
merge, filters contain subtree (from children) + complement (from
parent) + single-hop mesh views.
Implementation:
- Add is_tree_peer() helper to determine tree parent/child relationship
- Gate peer_inbound_filters() to tree peers only (single control point)
- Trigger bloom filter exchange on tree relationship changes
- Add est_entries, set_bits, fill ratio, and tree_peer fields to
FilterAnnounce send/receive debug logs
- Add test_bloom_filter_split_horizon test verifying directional
asymmetry: upward filters contain only the child's subtree, downward
filters contain only the complement
- Add print_filter_cardinality diagnostic helper for test inspection
Design docs:
- fips-bloom-filters.md: Add directional asymmetry and mesh peer filter
subsections, update per-peer filter model, saturation mitigation,
implementation status table
- fips-mesh-operation.md: Update filter propagation description, add
directional asymmetry, tree relationship change trigger
- fips-intro.md: Rewrite bloom propagation paragraph for tree-only merge
Replace raw NodeAddr hex strings in log output with human-readable
identifiers using a four-tier lookup: configured alias, active peer
short npub, session endpoint short npub, or truncated hex fallback.
- Add PeerIdentity::short_npub() for compact npub display (npub1xxxx...yyyy)
- Add NodeAddr::short_hex() for compact hex fallback (first 4 bytes + ...)
- Add peer_aliases map populated at startup from peer config
- Add Node::peer_display_name() with four-tier resolution
- Add SessionEntry::remote_pubkey() accessor for session-layer lookups
- Update all 120 log field occurrences across 11 handler/node files
- Pass pre-computed display names to MMP static metric/teardown methods
to work around borrow checker constraints in iterator loops
Per-packet happy-path events (UDP send/receive, TUN I/O, MMP report
processing, TreeAnnounce/FilterAnnounce sent, RTT samples) moved from
debug to trace. Periodic maintenance and retry scheduling moved from
info to debug. Session state changes (established, initiated, torn
down) and transport stop promoted from debug to info. MMP report send
failures demoted from warn to debug (normal under churn).
FilterAnnounce messages never settled in steady state because
handle_filter_announce unconditionally marked all peers for
re-announcement on every inbound filter, creating a perpetual
ping-pong at ~1 message/sec.
Added last_sent_filters tracking to BloomState. mark_changed_peers()
computes outgoing filters and compares against what was last sent,
only marking peers whose filter content actually differs.
Johnathan Corgan