FLP wire format revision and MMP link-layer measurement protocol

## FLP Wire Format Revision

Replace the 1-byte discriminator with a structured wire format:

- 4-byte common prefix (ver+phase, flags, payload_len) and 16-byte
  established frame header with AEAD AAD binding
- 5-byte encrypted inner header (4-byte session-relative timestamp +
  1-byte message type) on all link messages
- Phase-based packet dispatch replacing discriminator-based dispatch
- SessionDatagram reassigned from type 0x40 to 0x00; add SenderReport
  (0x01) and ReceiverReport (0x02) message types for MMP
- SessionDatagram: rename hop_limit to ttl, add path_mtu field (u16 LE)
  with min(datagram.path_mtu, transport.mtu()) at forwarding
- Updated handshake packets (msg1: 87->90 bytes, msg2: 42->45 bytes)
- FIPS_OVERHEAD updated from 135 to 144 bytes

## MMP Link-Layer Measurement Protocol

Add the Metrics Measurement Protocol for link quality measurement
between FIPS peers. Measures RTT, loss, jitter, throughput, OWD trend,
and ETX from periodic sender/receiver reports exchanged over established
links.

Module layout:
- mmp/algorithms.rs: JitterEstimator, SrttEstimator, DualEwma, OwdTrend,
  SpinBit, ETX computation
- mmp/report.rs: SenderReport (48B) and ReceiverReport (68B) wire format
- mmp/sender.rs: per-peer TX counters and interval tracking
- mmp/receiver.rs: per-peer RX counters, jitter, loss, gap tracking
- mmp/metrics.rs: derived metrics from report processing (SRTT, goodput_bps)
- mmp/mod.rs: MmpMode (Full/Lightweight/Minimal), MmpConfig, MmpPeerState
- node/handlers/mmp.rs: report dispatch, timer-driven generation, operator
  logging (periodic + teardown)

Integration: per-frame TX/RX hooks in encrypted message handling, report
dispatch from link message router, timer-driven generation from tick
handler, and periodic operator logging with throughput formatting.

Three operating modes: Full (sender + receiver reports, spin bit, CE echo),
Lightweight (receiver reports only), Minimal (spin bit + CE echo only).

## Design Documentation

Updated FLP sections across all design documents to match the implemented
wire format, including revised overhead calculations and numeric values.

568 tests pass, clippy clean.
This commit is contained in:
Johnathan Corgan
2026-02-18 21:54:21 +00:00
parent 2964a71ea7
commit d8cb4d407e
34 changed files with 3419 additions and 386 deletions
+20 -18
View File
@@ -119,15 +119,15 @@ after all layers of wrapping.
| Layer | Overhead | Purpose |
| ----- | -------- | ------- |
| Link encryption | 29 bytes | discriminator + receiver_idx + counter + AEAD tag |
| SessionDatagram envelope | 34 bytes | type + src_addr + dest_addr + hop_limit |
| Link encryption | 37 bytes | 16-byte outer header + 5-byte inner header + 16-byte AEAD tag |
| SessionDatagram envelope | 36 bytes | type + ttl + path_mtu + src_addr + dest_addr |
| DataPacket header | 12 bytes | type + flags + counter + payload_len |
| Session encryption | 16 bytes | ChaCha20-Poly1305 AEAD tag |
| **Minimal total** | **91 bytes** | |
| Coordinates (if present) | ~44 bytes | Depth-dependent, first few packets only |
| **Worst case total** | **135 bytes** | With COORDS_PRESENT for depth-3 paths |
| **Minimal total** | **101 bytes** | |
| Coordinates (if present) | ~43 bytes | Depth-dependent, first few packets only |
| **Worst case total** | **144 bytes** | With COORDS_PRESENT for depth-3 paths |
The `FIPS_OVERHEAD` constant (135 bytes) is used for conservative MTU
The `FIPS_OVERHEAD` constant (144 bytes) is used for conservative MTU
calculations.
### Effective IPv6 MTU
@@ -142,14 +142,14 @@ For typical deployments:
| Transport MTU | Effective IPv6 MTU | Notes |
| ------------- | ------------------ | ----- |
| 1472 (UDP/Ethernet) | 1337 | Standard deployment |
| 1280 (UDP minimum) | 1145 | Below IPv6 minimum |
| 1472 (UDP/Ethernet) | 1328 | Standard deployment |
| 1280 (UDP minimum) | 1136 | Below IPv6 minimum |
IPv6 mandates that every link support at least 1280 bytes. The minimum
transport path MTU for the IPv6 adapter is therefore:
```text
1280 + 135 = 1415 bytes
1280 + 144 = 1424 bytes
```
Transports with smaller MTUs (LoRa at ~250 bytes, serial at 256 bytes) cannot
@@ -278,23 +278,25 @@ TUN device creation requires `CAP_NET_ADMIN`. Options:
| ICMPv6 Packet Too Big | **Implemented** |
| ICMP rate limiting (per-source) | **Implemented** |
| TCP MSS clamping (SYN + SYN-ACK) | **Implemented** |
| DNS service (.fips domain) | Planned |
| DNS service (.fips domain) | **Implemented** |
| Per-destination route MTU (netlink) | Planned |
| Transit MTU error signal | Planned |
| Path MTU discovery (envelope field) | Future direction |
| Path MTU tracking (SessionDatagram field) | **Implemented** |
| Path MTU notification (end-to-end echo) | Future direction |
| Endpoint fragmentation/reassembly | Future direction |
## Design Considerations
### Path MTU Discovery (Planned)
### Path MTU Discovery
Two complementary mechanisms are planned for full PMTUD:
Two complementary mechanisms support full PMTUD:
1. **Proactive**: A `path_mtu` field (2 bytes) in the SessionDatagram envelope.
The source sets it to its outbound link MTU minus overhead; each transit
node applies `min(current, own_outbound_mtu - overhead)`. The destination
receives the forward-path minimum. A session-layer echo (2 bytes inside
encryption) returns the value to the source.
1. **Proactive**: The `path_mtu` field (2 bytes) in the SessionDatagram envelope
is implemented at the FLP level. The source sets it to its outbound link MTU
minus overhead; each transit node applies
`min(current, own_outbound_mtu - overhead)`. The destination receives the
forward-path minimum. A session-layer echo (PathMtuNotification, 2 bytes
inside encryption) to return the value to the source is a future direction.
2. **Reactive**: When a transit node cannot forward a packet (MTU exceeded), it
sends an error signal back to the source. This handles the in-flight gap
+28 -15
View File
@@ -75,8 +75,9 @@ handles all medium-specific details. FLP sees only "send bytes to address" and
### MTU Reporting
The maximum datagram size for a given link. FLP needs this to determine how
much payload fits in a single packet after link encryption overhead (29 bytes
for the encrypted frame wrapper).
much payload fits in a single packet after link encryption overhead (37 bytes
for the encrypted frame wrapper: 16-byte outer header + 5-byte inner header +
16-byte AEAD tag).
### Connection Lifecycle
@@ -137,13 +138,18 @@ notifications — passes through Noise's ChaCha20-Poly1305 AEAD.
### Encrypted Frame Structure
Post-handshake packets are wrapped in an encrypted frame consisting of:
- A discriminator byte identifying the frame type
- A 4-byte common prefix (version, phase, flags, payload length)
- A receiver index for O(1) session lookup
- An explicit counter used as the AEAD nonce
- The ciphertext with a Poly1305 authentication tag
The plaintext inside the encrypted frame begins with a message type byte
followed by the message-specific payload.
The 16-byte outer header (common prefix + receiver index + counter) is used as
AAD for the AEAD, binding the header to the ciphertext without encrypting it.
The plaintext inside the encrypted frame begins with a 5-byte inner header
(4-byte session-relative timestamp followed by a message type byte), then the
message-specific payload.
See [fips-wire-formats.md](fips-wire-formats.md) for the complete wire format
specification.
@@ -189,13 +195,14 @@ The tuple `(transport_id, receiver_idx)` uniquely identifies a session.
### Dispatch Flow
1. Read discriminator byte to determine packet type
2. For encrypted frames (0x00): look up `(transport_id, receiver_idx)` in the
session table — O(1) hash lookup. Unknown indices are rejected before any
cryptographic operation.
3. For handshake msg2 (0x02): look up by our sender index to match to a
1. Read the 4-byte common prefix to determine the phase (established,
handshake msg1, msg2)
2. For established frames (phase 0x0): look up `(transport_id, receiver_idx)`
in the session table — O(1) hash lookup. Unknown indices are rejected
before any cryptographic operation.
3. For handshake msg2 (phase 0x2): look up by our sender index to match to a
pending outbound handshake
4. For handshake msg1 (0x01): rate-limited processing, creates new state
4. For handshake msg1 (phase 0x1): rate-limited processing, creates new state
This approach follows WireGuard's design: source address is informational,
not authoritative. Only successful cryptographic verification establishes
@@ -295,7 +302,7 @@ The two-state liveness model:
## Link Message Types
FLP defines six message types carried inside encrypted frames:
FLP defines eight message types carried inside encrypted frames:
| Type | Name | Purpose |
| ---- | ---- | ------- |
@@ -303,11 +310,13 @@ FLP defines six message types carried inside encrypted frames:
| 0x20 | FilterAnnounce | Bloom filter reachability updates |
| 0x30 | LookupRequest | Coordinate discovery — flood toward destination |
| 0x31 | LookupResponse | Coordinate discovery — response with coordinates |
| 0x40 | SessionDatagram | Encapsulated session-layer payload for forwarding |
| 0x00 | SessionDatagram | Encapsulated session-layer payload for forwarding |
| 0x01 | SenderReport | Link statistics — what this node sent (reserved) |
| 0x02 | ReceiverReport | Link statistics — what this node observed (reserved) |
| 0x50 | Disconnect | Orderly link teardown with reason code |
Additionally, handshake messages (0x01 msg1, 0x02 msg2) are sent unencrypted
before the link session is established.
Additionally, handshake messages (phase 0x1 msg1, phase 0x2 msg2) are sent
unencrypted before the link session is established.
TreeAnnounce and FilterAnnounce are exchanged between direct peers only — they
are not forwarded. LookupRequest and LookupResponse are forwarded through the
@@ -366,6 +375,10 @@ an attacker sends invalid packets to elicit responses.
| Disconnect with reason codes | **Implemented** |
| Liveness detection (timeout-based) | **Implemented** |
| Reconnection handling | **Implemented** |
| Common prefix framing | **Implemented** |
| AAD binding on encrypted frames | **Implemented** |
| Inner header timestamps | **Implemented** |
| Path MTU tracking (SessionDatagram) | **Implemented** |
| Rekey with index rotation | Planned |
| Allowlist/blocklist | Planned |
+6 -6
View File
@@ -509,12 +509,12 @@ routing decisions but retains its own end-to-end encryption and identity.
| FilterAnnounce | ~1 KB | Topology changes | No (peer-to-peer) |
| LookupRequest | ~300 bytes | First contact, recovery | Yes (flood) |
| LookupResponse | ~400 bytes | Response to discovery | Yes (greedy routed) |
| SessionDatagram + SessionSetup | ~230400 bytes | Session establishment | Yes (routed) |
| SessionDatagram + SessionAck | ~120 bytes | Session confirmation | Yes (routed) |
| SessionDatagram + DataPacket (minimal) | 38 bytes + payload | Bulk traffic | Yes (routed) |
| SessionDatagram + DataPacket (with coords) | ~170 bytes + payload | Warmup/recovery | Yes (routed) |
| SessionDatagram + CoordsRequired | 68 bytes | Cache miss error | Yes (routed) |
| SessionDatagram + PathBroken | 68+ bytes | Dead-end error | Yes (routed) |
| SessionDatagram + SessionSetup | ~232402 bytes | Session establishment | Yes (routed) |
| SessionDatagram + SessionAck | ~122 bytes | Session confirmation | Yes (routed) |
| SessionDatagram + DataPacket (minimal) | 40 bytes + payload | Bulk traffic | Yes (routed) |
| SessionDatagram + DataPacket (with coords) | ~172 bytes + payload | Warmup/recovery | Yes (routed) |
| SessionDatagram + CoordsRequired | 70 bytes | Cache miss error | Yes (routed) |
| SessionDatagram + PathBroken | 70+ bytes | Dead-end error | Yes (routed) |
| Disconnect | 2 bytes | Link teardown | No (peer-to-peer) |
See [fips-wire-formats.md](fips-wire-formats.md) for byte-level layouts.
+4 -3
View File
@@ -74,8 +74,8 @@ bloom filters, hop counts, or network topology.
### SessionDatagram Forwarding
FLP accepts a SessionDatagram (source node_addr, destination node_addr, hop
limit, payload) and delivers it best-effort toward the destination. Delivery
FLP accepts a SessionDatagram (source node_addr, destination node_addr, TTL,
path MTU, payload) and delivers it best-effort toward the destination. Delivery
may traverse multiple hops, each with independent link encryption.
### Error Signaling
@@ -389,7 +389,8 @@ node caches (still within their 300s TTL) are re-warmed.
| Simultaneous initiation tie-breaker | **Implemented** |
| Flush coord cache on parent change | **Implemented** |
| Rekey | Planned |
| Path MTU discovery | Planned |
| Path MTU tracking (FLP SessionDatagram field) | **Implemented** |
| Path MTU notification (end-to-end echo) | Planned |
## References
+6 -4
View File
@@ -133,8 +133,10 @@ require connection setup before FLP can begin the Noise IK handshake,
adding startup latency.
**Stream vs. datagram**: Datagram transports have natural packet boundaries.
Stream transports (TCP, WebSocket, Tor) require length-prefix framing to
delineate FIPS packets within the byte stream.
Stream transports (TCP, WebSocket, Tor) require framing to delineate FIPS
packets within the byte stream. The FLP common prefix includes a payload
length field that provides this framing directly, replacing the need for a
separate length-prefix layer.
**Addressing opacity**: Transport addresses are opaque byte vectors. FLP
doesn't interpret them — it just passes them back to the transport when
@@ -183,8 +185,8 @@ circuit). Peer timeout configuration must account for transport-specific
setup times.
**Framing**: Stream transports must delimit FIPS packets within the byte
stream using length-prefix framing. Datagram transports preserve packet
boundaries naturally.
stream. The FLP common prefix includes a payload length field that provides
integrated framing. Datagram transports preserve packet boundaries naturally.
## UDP/IP: The Primary Internet Transport
+189 -118
View File
@@ -24,79 +24,137 @@ needed — each UDP datagram contains exactly one FIPS link-layer packet.
### Stream Transports *(future direction)*
TCP, WebSocket, and Tor transports require length-prefix framing because
they provide a byte stream, not datagrams:
```text
┌────────────┬───────────────────────────────────┐
│ Length │ FIPS Packet │
│ 2 bytes LE │ Variable │
└────────────┴───────────────────────────────────┘
```
TCP, WebSocket, and Tor transports provide a byte stream, not datagrams. The
common prefix `payload_len` field provides integrated stream framing — the
receiver reads the 4-byte common prefix, then reads exactly the number of
bytes indicated by `payload_len` (plus any phase-specific header and AEAD
tag). No separate length prefix is needed.
## Link-Layer Formats
All link-layer packets begin with a **discriminator byte** that determines
the payload format.
All FLP packets begin with a **4-byte common prefix** that identifies the
protocol version, session lifecycle phase, per-packet flags, and payload
length.
### Discriminator Table
### Common Prefix (4 bytes)
| Byte | Type | Description |
| ---- | ---- | ----------- |
| 0x00 | Encrypted frame | Post-handshake encrypted traffic |
| 0x01 | Noise IK msg1 | Handshake initiation |
| 0x02 | Noise IK msg2 | Handshake response |
```text
┌──────────────────────┬───────────┬───────────────┐
│ ver(4) + phase(4) │ flags │ payload_len │
│ 1 byte │ 1 byte │ 2 bytes LE │
└──────────────────────┴───────────┴───────────────┘
```
### Encrypted Frame (0x00)
| Field | Size | Description |
| ----- | ---- | ----------- |
| version | 4 bits (high) | Protocol version. Currently 0x0 |
| phase | 4 bits (low) | Session lifecycle phase (see table) |
| flags | 1 byte | Per-packet signal flags (zero during handshake) |
| payload_len | 2 bytes LE | Length of payload after phase-specific header, excluding AEAD tag |
### Phase Table
| Phase | Type | Description |
| ----- | ---- | ----------- |
| 0x0 | Established frame | Post-handshake encrypted traffic |
| 0x1 | Noise IK msg1 | Handshake initiation |
| 0x2 | Noise IK msg2 | Handshake response |
### Flags (Established Phase Only)
| Bit | Name | Description |
| --- | ---- | ----------- |
| 0 | K (key epoch) | Selects active key during rekeying |
| 1 | CE | Congestion Experienced echo |
| 2 | SP (spin bit) | RTT measurement |
| 3-7 | — | Reserved (must be zero) |
Flags must be zero in handshake packets (phase 0x1 and 0x2).
### Established Frame (phase 0x0)
All post-handshake traffic between authenticated peers. Contains one
encrypted link-layer message.
```text
┌────────┬──────────────┬──────────┬───────────────────────────┐
│ 0x00 │ receiver_idx │ counter │ ciphertext + AEAD tag │
│ 1 byte │ 4 bytes LE │ 8 bytes LE│ N + 16 bytes │
└────────┴──────────────┴──────────┴───────────────────────────┘
**Outer header** (16 bytes, used as AEAD AAD):
Total overhead: 29 bytes (1 + 4 + 8 + 16)
Minimum frame: 30 bytes (1-byte plaintext)
```text
┌──────────────────────┬───────────┬───────────────┬──────────────┬──────────┐
│ ver(4) + phase(4) │ flags │ payload_len │ receiver_idx │ counter │
│ 1 byte │ 1 byte │ 2 bytes LE │ 4 bytes LE │ 8 bytes LE│
└──────────────────────┴───────────┴───────────────┴──────────────┴──────────┘
```
| Field | Size | Description |
| ----- | ---- | ----------- |
| discriminator | 1 byte | 0x00 |
| common prefix | 4 bytes | ver=0, phase=0, flags, payload_len |
| receiver_idx | 4 bytes LE | Session index for O(1) lookup |
| counter | 8 bytes LE | Monotonic nonce, used as AEAD nonce and for replay detection |
| ciphertext | N bytes | ChaCha20 encrypted payload |
| tag | 16 bytes | Poly1305 authentication tag |
The **plaintext** inside the encrypted frame begins with a message type byte:
The entire 16-byte header is authenticated as Associated Data (AAD) in the
ChaCha20-Poly1305 AEAD construction.
| Type | Message |
| ---- | ------- |
| 0x10 | TreeAnnounce |
| 0x20 | FilterAnnounce |
| 0x30 | LookupRequest |
| 0x31 | LookupResponse |
| 0x40 | SessionDatagram |
| 0x50 | Disconnect |
**Encrypted inner header** (5 bytes, first bytes of plaintext):
### Noise IK Message 1 (0x01)
```text
┌───────────────┬──────────┐
│ timestamp │ msg_type │
│ 4 bytes LE │ 1 byte │
└───────────────┴──────────┘
```
| Field | Size | Description |
| ----- | ---- | ----------- |
| timestamp | 4 bytes LE | Session-relative milliseconds (u32) |
| msg_type | 1 byte | Link-layer message type |
After decryption, the plaintext begins with the 4-byte timestamp followed by
the 1-byte message type and message-specific fields.
**Complete encrypted frame**:
```text
┌──────────────────────────────────────┬───────────────────────────┐
│ outer header (16 bytes, used as AAD) │ ciphertext + AEAD tag │
│ │ (inner_hdr + body) + 16 │
└──────────────────────────────────────┴───────────────────────────┘
Total overhead: 37 bytes (16 outer + 5 inner + 16 AEAD tag)
Minimum frame: 37 bytes (empty body)
```
### Message Type Table
| Type | Message | Description |
| ---- | ------- | ----------- |
| 0x00 | SessionDatagram | Encapsulated session-layer payload for forwarding |
| 0x01 | SenderReport | MMP sender-side report (reserved) |
| 0x02 | ReceiverReport | MMP receiver-side report (reserved) |
| 0x10 | TreeAnnounce | Spanning tree state announcement |
| 0x20 | FilterAnnounce | Bloom filter reachability update |
| 0x30 | LookupRequest | Coordinate discovery request |
| 0x31 | LookupResponse | Coordinate discovery response |
| 0x50 | Disconnect | Orderly link teardown |
| 0x51 | Keepalive | Keepalive probe (reserved) |
### Noise IK Message 1 (phase 0x1)
Handshake initiation from connecting party.
```text
┌────────┬─────────────┬─────────────────────────────────────────┐
0x01 │ sender_idx │ Noise IK message 1 │
1 byte │ 4 bytes LE │ 82 bytes │
└────────┴─────────────┴─────────────────────────────────────────┘
┌──────────────────────┬─────────────┬─────────────────────────────────────────┐
common prefix │ sender_idx │ Noise IK message 1 │
4 bytes │ 4 bytes LE │ 82 bytes │
└──────────────────────┴─────────────┴─────────────────────────────────────────┘
Total: 87 bytes
Total: 90 bytes
```
Common prefix: ver=0, phase=0x1, flags=0, payload_len=86 (4 + 82).
| Field | Size | Description |
| ----- | ---- | ----------- |
| discriminator | 1 byte | 0x01 |
| common prefix | 4 bytes | ver=0, phase=1, flags=0, payload_len |
| sender_idx | 4 bytes LE | Initiator's session index (becomes receiver's `receiver_idx`) |
| noise_msg1 | 82 bytes | Noise IK first message |
@@ -108,24 +166,26 @@ Total: 87 bytes
| 33 | encrypted_static | 33 bytes | Initiator's static key (encrypted with es key) |
| 66 | tag | 16 bytes | AEAD tag for encrypted_static |
Noise pattern: ` e, es, s, ss`
Noise pattern: `-> e, es, s, ss`
### Noise IK Message 2 (0x02)
### Noise IK Message 2 (phase 0x2)
Handshake response from responder.
```text
┌────────┬─────────────┬──────────────┬──────────────────────────┐
0x02 │ sender_idx │ receiver_idx │ Noise IK message 2 │
1 byte │ 4 bytes LE │ 4 bytes LE │ 33 bytes │
└────────┴─────────────┴──────────────┴──────────────────────────┘
┌──────────────────────┬─────────────┬──────────────┬──────────────────────────┐
common prefix │ sender_idx │ receiver_idx │ Noise IK message 2 │
4 bytes │ 4 bytes LE │ 4 bytes LE │ 33 bytes │
└──────────────────────┴─────────────┴──────────────┴──────────────────────────┘
Total: 42 bytes
Total: 45 bytes
```
Common prefix: ver=0, phase=0x2, flags=0, payload_len=41 (4 + 4 + 33).
| Field | Size | Description |
| ----- | ---- | ----------- |
| discriminator | 1 byte | 0x02 |
| common prefix | 4 bytes | ver=0, phase=2, flags=0, payload_len |
| sender_idx | 4 bytes LE | Responder's session index |
| receiver_idx | 4 bytes LE | Echo of initiator's sender_idx from msg1 |
| noise_msg2 | 33 bytes | Noise IK second message |
@@ -136,7 +196,7 @@ Total: 42 bytes
| ------ | ----- | ---- | ----------- |
| 0 | ephemeral_pubkey | 33 bytes | Responder's ephemeral key (compressed secp256k1) |
Noise pattern: ` e, ee, se`
Noise pattern: `<- e, ee, se`
After msg2, both parties derive identical symmetric session keys.
@@ -153,34 +213,37 @@ Each party in a link session maintains two indices:
```text
Initiator Responder
───────── ─────────
--------- ---------
generates sender_idx
generates ephemeral keypair
0x01 | sender_idx | noise_msg1
────────────────────────────────►
[0x01|flags=0|len] | sender_idx | noise_msg1
------------------------------------------------>
validates msg1
learns initiator's static key
generates sender_idx
generates ephemeral keypair
0x02 | sender_idx | receiver_idx | noise_msg2
◄────────────────────────────────
[0x02|flags=0|len] | sender_idx | receiver_idx | noise_msg2
<------------------------------------------------
validates msg2
derives session keys
═══════════════ HANDSHAKE COMPLETE ═══════════════
=============== HANDSHAKE COMPLETE ===============
First encrypted frame:
0x00 | receiver_idx | counter=0 | ciphertext+tag
────────────────────────────────►
[0x00|flags|len] | receiver_idx | counter=0 | ciphertext+tag
------------------------------------------------>
```
## Link-Layer Message Types
These messages are carried as plaintext inside encrypted frames (0x00).
These messages are carried as plaintext inside encrypted frames (phase 0x0).
After decryption of the AEAD ciphertext, the plaintext begins with a 4-byte
session-relative timestamp followed by the 1-byte message type and
message-specific fields.
### TreeAnnounce (0x10)
@@ -194,7 +257,7 @@ Spanning tree state announcement, exchanged between direct peers only.
| 10 | timestamp | 8 bytes LE | Unix seconds |
| 18 | parent | 16 bytes | NodeAddr of selected parent (self = root) |
| 34 | ancestry_count | 2 bytes LE | Number of AncestryEntry records |
| 36 | ancestry | 32 × n bytes | AncestryEntry array (self root) |
| 36 | ancestry | 32 x n bytes | AncestryEntry array (self -> root) |
| 36 + 32n | signature | 64 bytes | Schnorr signature over entire message |
**AncestryEntry** (32 bytes):
@@ -205,15 +268,15 @@ Spanning tree state announcement, exchanged between direct peers only.
| 16 | sequence | 8 bytes LE | Node's sequence number |
| 24 | timestamp | 8 bytes LE | Node's Unix timestamp |
**Size**: `100 + (n × 32)` bytes, where n = `ancestry_count` (depth + 1,
**Size**: `100 + (n x 32)` bytes, where n = `ancestry_count` (depth + 1,
includes self)
| Tree Depth | Payload | With Link Overhead |
| ---------- | ------- | ------------------ |
| 0 (root) | 132 bytes | 161 bytes |
| 3 | 228 bytes | 257 bytes |
| 5 | 292 bytes | 321 bytes |
| 10 | 452 bytes | 481 bytes |
| 0 (root) | 132 bytes | 169 bytes |
| 3 | 228 bytes | 265 bytes |
| 5 | 292 bytes | 329 bytes |
| 10 | 452 bytes | 489 bytes |
### FilterAnnounce (0x20)
@@ -237,7 +300,7 @@ Bloom filter reachability update, exchanged between direct peers only.
| 3 | 4,096 | 32,768 | Reserved |
**v1 payload**: 1,035 bytes (11 header + 1,024 filter).
With link overhead: 1,064 bytes.
With link overhead: 1,072 bytes.
### LookupRequest (0x30)
@@ -251,11 +314,11 @@ Coordinate discovery request, flooded through the mesh.
| 25 | origin | 16 bytes | Requester's NodeAddr |
| 41 | ttl | 1 byte | Remaining hops (default 64) |
| 42 | origin_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 44 | origin_coords | 16 × n bytes | Requester's ancestry (NodeAddr only) |
| 44 | origin_coords | 16 x n bytes | Requester's ancestry (NodeAddr only) |
| 44 + 16n | visited_hash_cnt | 1 byte | Hash count for visited filter |
| 45 + 16n | visited_bits | 256 bytes | Compact bloom of visited nodes |
**Size**: `301 + (n × 16)` bytes, where n = origin depth + 1
**Size**: `301 + (n x 16)` bytes, where n = origin depth + 1
| Origin Depth | Payload |
| ------------ | ------- |
@@ -273,10 +336,10 @@ Coordinate discovery response, greedy-routed back to requester.
| 1 | request_id | 8 bytes LE | Echoes the request's ID |
| 9 | target | 16 bytes | NodeAddr that was found |
| 25 | target_coords_cnt | 2 bytes LE | Number of coordinate entries |
| 27 | target_coords | 16 × n bytes | Target's ancestry (NodeAddr only) |
| 27 | target_coords | 16 x n bytes | Target's ancestry (NodeAddr only) |
| 27 + 16n | proof | 64 bytes | Schnorr signature over `(request_id \|\| target)` |
**Size**: `91 + (n × 16)` bytes
**Size**: `91 + (n x 16)` bytes
| Target Depth | Payload |
| ------------ | ------- |
@@ -288,19 +351,24 @@ Coordinate discovery response, greedy-routed back to requester.
excluded so the proof survives tree reconvergence during the lookup
round-trip.
### SessionDatagram (0x40)
### SessionDatagram (0x00)
Encapsulated session-layer payload for multi-hop forwarding.
| Offset | Field | Size | Description |
| ------ | ----- | ---- | ----------- |
| 0 | msg_type | 1 byte | 0x40 |
| 1 | src_addr | 16 bytes | Source NodeAddr |
| 17 | dest_addr | 16 bytes | Destination NodeAddr |
| 33 | hop_limit | 1 byte | Decremented each hop |
| 34 | payload | variable | Session-layer message |
| 0 | msg_type | 1 byte | 0x00 |
| 1 | ttl | 1 byte | Remaining hops, decremented each hop |
| 2 | path_mtu | 2 bytes LE | Path MTU, min'd at each forwarding hop |
| 4 | src_addr | 16 bytes | Source NodeAddr |
| 20 | dest_addr | 16 bytes | Destination NodeAddr |
| 36 | payload | variable | Session-layer message |
**Fixed header**: 34 bytes (`SESSION_DATAGRAM_HEADER_SIZE`)
**Fixed header**: 36 bytes (`SESSION_DATAGRAM_HEADER_SIZE`)
The `path_mtu` field is initialized to `u16::MAX` by the sender and each
forwarding hop applies `min(path_mtu, outgoing_link_mtu)`, giving the
receiver an estimate of the minimum MTU along the path.
The payload is opaque to transit nodes — session-layer encrypted
independently of link encryption.
@@ -330,7 +398,7 @@ Orderly link teardown with reason code.
## Session-Layer Message Types
These messages are carried as the payload of a SessionDatagram (0x40).
These messages are carried as the payload of a SessionDatagram (0x00).
### SessionSetup (0x00)
@@ -341,18 +409,18 @@ Establishes a session and warms transit coordinate caches.
| 0 | msg_type | 1 byte | 0x00 |
| 1 | flags | 1 byte | Bit 0: REQUEST_ACK, Bit 1: BIDIRECTIONAL |
| 2 | src_coords_count | 2 bytes LE | Number of source coordinate entries |
| 4 | src_coords | 16 × n bytes | Source's ancestry (NodeAddr, self root) |
| 4 | src_coords | 16 x n bytes | Source's ancestry (NodeAddr, self -> root) |
| ... | dest_coords_count | 2 bytes LE | Number of dest coordinate entries |
| ... | dest_coords | 16 × m bytes | Destination's ancestry |
| ... | dest_coords | 16 x m bytes | Destination's ancestry |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise IK msg1 (82 bytes typical) |
**Example** (depth-3 source, depth-4 destination):
```text
SessionDatagram header: 34 bytes
SessionDatagram header: 36 bytes
SessionSetup payload: 1 + 1 + 2 + 48 + 2 + 64 + 2 + 82 = 202 bytes
Total: 236 bytes
Total: 238 bytes
```
### SessionAck (0x01)
@@ -364,7 +432,7 @@ Confirms session establishment, completes the Noise handshake.
| 0 | msg_type | 1 byte | 0x01 |
| 1 | flags | 1 byte | Reserved |
| 2 | src_coords_count | 2 bytes LE | Number of coordinate entries |
| 4 | src_coords | 16 × n bytes | Acknowledger's ancestry (for cache warming) |
| 4 | src_coords | 16 x n bytes | Acknowledger's ancestry (for cache warming) |
| ... | handshake_len | 2 bytes LE | Noise payload length |
| ... | handshake_payload | variable | Noise IK msg2 (33 bytes typical) |
@@ -393,9 +461,9 @@ Encrypted application data with explicit replay protection counter.
| 2 | counter | 8 bytes LE | Session encryption counter |
| 10 | payload_length | 2 bytes LE | Length of encrypted payload |
| 12 | src_coords_count | 2 bytes LE | Source coordinate entries |
| 14 | src_coords | 16 × n bytes | Source's ancestry |
| 14 | src_coords | 16 x n bytes | Source's ancestry |
| ... | dest_coords_count | 2 bytes LE | Dest coordinate entries |
| ... | dest_coords | 16 × m bytes | Destination's ancestry |
| ... | dest_coords | 16 x m bytes | Destination's ancestry |
| ... | payload | variable | Encrypted application data |
### CoordsRequired (0x20)
@@ -410,7 +478,7 @@ Plaintext (not end-to-end encrypted), generated by transit nodes.
| 2 | dest_addr | 16 bytes | NodeAddr we couldn't route to |
| 18 | reporter | 16 bytes | NodeAddr of reporting router |
**Payload**: 34 bytes. Wrapped in SessionDatagram: 68 bytes total.
**Payload**: 34 bytes. Wrapped in SessionDatagram: 70 bytes total.
### PathBroken (0x21)
@@ -424,13 +492,13 @@ generated by transit nodes.
| 2 | dest_addr | 16 bytes | Unreachable NodeAddr |
| 18 | reporter | 16 bytes | NodeAddr of reporting router |
| 34 | last_coords_count | 2 bytes LE | Number of stale coordinate entries |
| 36 | last_known_coords | 16 × n bytes | Stale coordinates that failed |
| 36 | last_known_coords | 16 x n bytes | Stale coordinates that failed |
## Encapsulation Walkthrough
A complete picture of how application data is wrapped through each layer.
### Application Data Wire
### Application Data -> Wire
Starting with an application sending a 1024-byte payload to a destination:
@@ -443,46 +511,48 @@ Layer 3: Session encryption (FSP)
= 1052 bytes
Layer 2: SessionDatagram envelope (FLP routing)
msg_type (1) + src_addr (16) + dest_addr (16) + hop_limit (1) + payload (1052)
= 1086 bytes
msg_type (1) + ttl (1) + path_mtu (2) + src_addr (16) + dest_addr (16) + payload (1052)
= 1088 bytes
Layer 1: Link encryption (FLP per-hop)
discriminator (1) + receiver_idx (4) + counter (8) + ciphertext (1086) + tag (16)
= 1115 bytes
outer header (16) + encrypted(inner_hdr (5) + datagram (1088)) + AEAD tag (16)
= 1125 bytes
Layer 0: Transport
UDP datagram containing 1115 bytes
UDP datagram containing 1125 bytes
```
### Overhead Budget
| Layer | Overhead | Component |
| ----- | -------- | --------- |
| Link encryption | 29 bytes | 1 discriminator + 4 index + 8 counter + 16 AEAD tag |
| SessionDatagram | 34 bytes | 1 type + 16 src + 16 dest + 1 hop_limit |
| Link encryption | 37 bytes | 16 outer header (AAD) + 5 inner header + 16 AEAD tag |
| SessionDatagram | 36 bytes | 1 type + 1 ttl + 2 path_mtu + 16 src + 16 dest |
| DataPacket header | 12 bytes | 1 type + 1 flags + 8 counter + 2 length |
| Session AEAD tag | 16 bytes | Poly1305 tag on session-encrypted payload |
| **Minimal total** | **91 bytes** | |
| Coordinates (if present) | ~44 bytes | Varies with tree depth |
| **Worst case** | **135 bytes** | `FIPS_OVERHEAD` constant |
| **Minimal total** | **101 bytes** | |
| Coordinates (if present) | ~43 bytes | Varies with tree depth |
| **Worst case** | **144 bytes** | `FIPS_OVERHEAD` constant |
### At Each Transit Node
```text
1. Receive UDP datagram
2. Read discriminator (0x00) → encrypted frame
3. Look up (transport_id, receiver_idx) → session
4. Check replay window (counter)
5. Decrypt with link keys → plaintext link message
6. Read msg_type (0x40) → SessionDatagram
7. Read dest_addr → routing decision
8. Decrement hop_limit
9. Re-encrypt with next-hop link keys
10. Send via next-hop transport
2. Parse common prefix -> version, phase, flags, payload_len
3. Phase 0x0 -> established frame
4. Look up (transport_id, receiver_idx) -> session
5. Check replay window (counter)
6. Decrypt with link keys (16-byte header as AAD) -> plaintext
7. Strip inner header -> timestamp, msg_type
8. msg_type 0x00 -> SessionDatagram
9. Read dest_addr -> routing decision
10. Decrement ttl, min path_mtu
11. Re-encrypt with next-hop link keys
12. Send via next-hop transport
```
Transit nodes see the SessionDatagram envelope (src_addr, dest_addr,
hop_limit) but cannot read the session-layer payload (encrypted with
ttl, path_mtu) but cannot read the session-layer payload (encrypted with
endpoint session keys).
## Size Summary
@@ -491,8 +561,8 @@ endpoint session keys).
| Message | Size |
| ------- | ---- |
| Noise IK msg1 | 87 bytes |
| Noise IK msg2 | 42 bytes |
| Noise IK msg1 | 90 bytes |
| Noise IK msg2 | 45 bytes |
### Link-Layer Messages (inside encrypted frame)
@@ -502,6 +572,7 @@ endpoint session keys).
| FilterAnnounce | 1,035 bytes | v1 (1KB filter) |
| LookupRequest | 301 + 16n bytes | n = origin depth + 1 |
| LookupResponse | 91 + 16n bytes | n = target depth + 1 |
| SessionDatagram | 36 + payload bytes | Fixed 36-byte header |
| Disconnect | 2 bytes | |
### Session-Layer Messages (inside SessionDatagram)
@@ -519,11 +590,11 @@ endpoint session keys).
| Scenario | Wire Size | Notes |
| -------- | --------- | ----- |
| Encrypted frame minimum | 30 bytes | 1-byte plaintext |
| SessionDatagram + DataPacket (minimal) | 29 + 34 + 12 + payload + 16 | 91 + payload |
| SessionDatagram + DataPacket (with coords) | ~135 + payload | Worst case |
| SessionDatagram + SessionSetup | ~265 bytes | Depth-3, both dirs |
| SessionDatagram + CoordsRequired | 29 + 34 + 34 = 97 bytes | Including link overhead |
| Encrypted frame minimum | 37 bytes | Empty body |
| SessionDatagram + DataPacket (minimal) | 37 + 36 + 12 + payload + 16 | 101 + payload |
| SessionDatagram + DataPacket (with coords) | ~144 + payload | Worst case |
| SessionDatagram + SessionSetup | ~275 bytes | Depth-3, both dirs |
| SessionDatagram + CoordsRequired | 37 + 36 + 34 = 107 bytes | Including link overhead |
## References
+1 -1
View File
@@ -18,7 +18,7 @@ x-fips-common: &fips-common
env_file:
- ./generated-configs/npubs.env
environment:
- RUST_LOG=trace
- RUST_LOG=info
services:
# ── Mesh topology ──────────────────────────────────────────────
+11 -5
View File
@@ -7,6 +7,7 @@
use serde::{Deserialize, Serialize};
use super::IdentityConfig;
use crate::mmp::MmpConfig;
// ============================================================================
// Node Configuration Subsections
@@ -213,9 +214,9 @@ impl BloomConfig {
/// Session/data plane (`node.session.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SessionConfig {
/// Default SessionDatagram hop limit (`node.session.default_hop_limit`).
#[serde(default = "SessionConfig::default_hop_limit")]
pub default_hop_limit: u8,
/// Default SessionDatagram TTL (`node.session.default_ttl`).
#[serde(default = "SessionConfig::default_ttl")]
pub default_ttl: u8,
/// Queue depth per dest during session establishment (`node.session.pending_packets_per_dest`).
#[serde(default = "SessionConfig::default_pending_packets_per_dest")]
pub pending_packets_per_dest: usize,
@@ -236,7 +237,7 @@ pub struct SessionConfig {
impl Default for SessionConfig {
fn default() -> Self {
Self {
default_hop_limit: 64,
default_ttl: 64,
pending_packets_per_dest: 16,
pending_max_destinations: 256,
idle_timeout_secs: 90,
@@ -246,7 +247,7 @@ impl Default for SessionConfig {
}
impl SessionConfig {
fn default_hop_limit() -> u8 { 64 }
fn default_ttl() -> u8 { 64 }
fn default_pending_packets_per_dest() -> usize { 16 }
fn default_pending_max_destinations() -> usize { 256 }
fn default_idle_timeout_secs() -> u64 { 90 }
@@ -341,6 +342,10 @@ pub struct NodeConfig {
/// Internal buffers (`node.buffers.*`).
#[serde(default)]
pub buffers: BuffersConfig,
/// Metrics Measurement Protocol (`node.mmp.*`).
#[serde(default)]
pub mmp: MmpConfig,
}
impl Default for NodeConfig {
@@ -359,6 +364,7 @@ impl Default for NodeConfig {
bloom: BloomConfig::default(),
session: SessionConfig::default(),
buffers: BuffersConfig::default(),
mmp: MmpConfig::default(),
}
}
}
+1
View File
@@ -7,6 +7,7 @@ pub mod bloom;
pub mod cache;
pub mod config;
pub mod identity;
pub mod mmp;
pub mod noise;
pub mod utils;
pub mod node;
+508
View File
@@ -0,0 +1,508 @@
//! MMP algorithmic building blocks.
//!
//! Pure computational types with no dependency on peer or node state.
//! Each is independently testable.
use std::collections::VecDeque;
use std::time::Instant;
use crate::mmp::{EWMA_LONG_ALPHA, EWMA_SHORT_ALPHA};
// ============================================================================
// Jitter Estimator (RFC 3550 §6.4.1)
// ============================================================================
/// Interarrival jitter estimator using RFC 3550 algorithm.
///
/// Maintains a smoothed jitter estimate (α = 1/16) from the absolute
/// difference in one-way transit times between consecutive frames.
/// Uses integer arithmetic scaled by 16 to avoid floating-point.
pub struct JitterEstimator {
/// Scaled jitter estimate (×16 for integer arithmetic).
jitter_q4: i64,
}
impl JitterEstimator {
pub fn new() -> Self {
Self { jitter_q4: 0 }
}
/// Update with transit time delta between consecutive frames.
///
/// `transit_delta` = (R_i - R_{i-1}) - (S_i - S_{i-1}) in microseconds.
pub fn update(&mut self, transit_delta: i32) {
// RFC 3550: J = J + (1/16)(|D(i)| - J)
// Scaled: J_q4 = J_q4 + (|D| - J_q4/16)
// = J_q4 + |D| - J_q4 >> 4
let abs_d = (transit_delta as i64).unsigned_abs() as i64;
self.jitter_q4 += abs_d - (self.jitter_q4 >> 4);
}
/// Current jitter estimate in microseconds.
pub fn jitter_us(&self) -> u32 {
(self.jitter_q4 >> 4) as u32
}
}
impl Default for JitterEstimator {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// SRTT Estimator (Jacobson, RFC 6298)
// ============================================================================
/// Smoothed RTT estimator using Jacobson's algorithm.
///
/// SRTT and RTTVAR are maintained in microseconds using integer arithmetic.
pub struct SrttEstimator {
/// Smoothed RTT (microseconds).
srtt_us: i64,
/// RTT variance (microseconds).
rttvar_us: i64,
/// Whether the first sample has been applied.
initialized: bool,
}
impl SrttEstimator {
pub fn new() -> Self {
Self {
srtt_us: 0,
rttvar_us: 0,
initialized: false,
}
}
/// Feed an RTT sample in microseconds.
pub fn update(&mut self, rtt_us: i64) {
if !self.initialized {
// RFC 6298 §2.2: first measurement
self.srtt_us = rtt_us;
self.rttvar_us = rtt_us / 2;
self.initialized = true;
} else {
// RFC 6298 §2.3:
// RTTVAR = (1 - β) * RTTVAR + β * |SRTT - R'| β = 1/4
// SRTT = (1 - α) * SRTT + α * R' α = 1/8
let err = (self.srtt_us - rtt_us).abs();
self.rttvar_us = self.rttvar_us - (self.rttvar_us >> 2) + (err >> 2);
self.srtt_us = self.srtt_us - (self.srtt_us >> 3) + (rtt_us >> 3);
}
}
pub fn srtt_us(&self) -> i64 {
self.srtt_us
}
pub fn rttvar_us(&self) -> i64 {
self.rttvar_us
}
pub fn initialized(&self) -> bool {
self.initialized
}
/// Retransmission timeout = SRTT + max(4 * RTTVAR, 1s), floored at 1s.
pub fn rto_us(&self) -> i64 {
let rto = self.srtt_us + (self.rttvar_us << 2).max(1_000_000);
rto.max(1_000_000)
}
}
impl Default for SrttEstimator {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Dual EWMA Trend Detector
// ============================================================================
/// Dual EWMA for trend detection on a single metric.
///
/// Short-term (α=1/4) tracks recent conditions; long-term (α=1/32)
/// establishes a stable baseline. Divergence indicates trend direction.
pub struct DualEwma {
short: f64,
long: f64,
initialized: bool,
}
impl DualEwma {
pub fn new() -> Self {
Self {
short: 0.0,
long: 0.0,
initialized: false,
}
}
pub fn update(&mut self, sample: f64) {
if !self.initialized {
self.short = sample;
self.long = sample;
self.initialized = true;
} else {
self.short += EWMA_SHORT_ALPHA * (sample - self.short);
self.long += EWMA_LONG_ALPHA * (sample - self.long);
}
}
pub fn short(&self) -> f64 {
self.short
}
pub fn long(&self) -> f64 {
self.long
}
pub fn initialized(&self) -> bool {
self.initialized
}
}
impl Default for DualEwma {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// One-Way Delay Trend Detector
// ============================================================================
/// OWD trend detector using linear regression over a ring buffer.
///
/// Stores (sequence, owd_us) samples and computes the slope via
/// least-squares regression. The slope (µs/s) indicates whether
/// queuing delay is increasing (congestion) or stable.
pub struct OwdTrendDetector {
samples: VecDeque<(u32, i64)>,
capacity: usize,
}
impl OwdTrendDetector {
pub fn new(capacity: usize) -> Self {
Self {
samples: VecDeque::with_capacity(capacity),
capacity,
}
}
/// Add an OWD sample.
///
/// `seq` is a monotonic sequence number (e.g., truncated frame counter).
/// `owd_us` is the relative one-way delay in microseconds (R_i - S_i).
pub fn push(&mut self, seq: u32, owd_us: i64) {
if self.samples.len() == self.capacity {
self.samples.pop_front();
}
self.samples.push_back((seq, owd_us));
}
/// Compute the OWD trend as a slope in µs/second.
///
/// Uses simple linear regression: slope = Σ((x-x̄)(y-ȳ)) / Σ((x-x̄)²)
/// where x = sequence number and y = owd_us.
///
/// Returns 0 if fewer than 2 samples.
pub fn trend_us_per_sec(&self) -> i32 {
let n = self.samples.len();
if n < 2 {
return 0;
}
let n_f = n as f64;
let sum_x: f64 = self.samples.iter().map(|(s, _)| *s as f64).sum();
let sum_y: f64 = self.samples.iter().map(|(_, y)| *y as f64).sum();
let mean_x = sum_x / n_f;
let mean_y = sum_y / n_f;
let mut num = 0.0;
let mut den = 0.0;
for &(x, y) in &self.samples {
let dx = x as f64 - mean_x;
let dy = y as f64 - mean_y;
num += dx * dy;
den += dx * dx;
}
if den.abs() < f64::EPSILON {
return 0;
}
// slope is in µs/packet. Convert to µs/second assuming ~1ms inter-packet
// spacing as a rough estimate. The raw slope per packet is more useful
// for trend detection than an absolute rate, but the wire format specifies
// µs/s. We report the raw per-packet slope scaled by 1000.
let slope_per_packet = num / den;
(slope_per_packet * 1000.0) as i32
}
pub fn len(&self) -> usize {
self.samples.len()
}
pub fn is_empty(&self) -> bool {
self.samples.is_empty()
}
}
// ============================================================================
// ETX
// ============================================================================
/// Compute Expected Transmission Count from bidirectional delivery ratios.
///
/// ETX = 1 / (d_f × d_r) where d_f and d_r are forward and reverse
/// delivery probabilities (1.0 = perfect, 0.0 = no delivery).
///
/// Clamped to [1.0, 100.0].
pub fn compute_etx(d_forward: f64, d_reverse: f64) -> f64 {
let product = d_forward * d_reverse;
if product <= 0.0 {
return 100.0;
}
(1.0 / product).clamp(1.0, 100.0)
}
// ============================================================================
// Spin Bit
// ============================================================================
/// Spin bit state for passive RTT estimation.
///
/// Uses asymmetric roles (initiator/responder) per the MMP design:
/// - **Initiator**: flips spin value on each received frame; measures RTT
/// from edge-to-edge intervals.
/// - **Responder**: copies received spin bit into outgoing frames, with a
/// counter guard to filter reordered frames.
pub struct SpinBitState {
is_initiator: bool,
current_value: bool,
/// Highest counter observed with a spin edge (responder guard).
highest_counter_for_spin: u64,
/// Time of last spin edge (initiator only, for RTT measurement).
last_edge_time: Option<Instant>,
}
impl SpinBitState {
pub fn new(is_initiator: bool) -> Self {
Self {
is_initiator,
current_value: false,
highest_counter_for_spin: 0,
last_edge_time: None,
}
}
/// Get the spin bit value to set on an outgoing frame.
pub fn tx_bit(&self) -> bool {
self.current_value
}
/// Process a received frame's spin bit.
///
/// Returns an RTT sample duration if an edge was detected (initiator only).
pub fn rx_observe(
&mut self,
received_bit: bool,
counter: u64,
now: Instant,
) -> Option<std::time::Duration> {
if self.is_initiator {
// Initiator: when the reflected bit matches what we sent,
// that completes a round trip. Record the edge time, then
// flip for the next cycle.
if received_bit == self.current_value {
let rtt = self.last_edge_time.map(|t| now.duration_since(t));
self.last_edge_time = Some(now);
self.current_value = !self.current_value;
rtt
} else {
None
}
} else {
// Responder: copy received bit, but only if counter is higher
// (reordering guard)
if counter > self.highest_counter_for_spin {
self.highest_counter_for_spin = counter;
self.current_value = received_bit;
}
None
}
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_jitter_zero_input() {
let mut j = JitterEstimator::new();
j.update(0);
assert_eq!(j.jitter_us(), 0);
}
#[test]
fn test_jitter_convergence() {
let mut j = JitterEstimator::new();
// Feed constant transit delta of 1000µs
for _ in 0..200 {
j.update(1000);
}
// Should converge near 1000µs
let jitter = j.jitter_us();
assert!(jitter > 900 && jitter < 1100, "jitter={jitter}, expected ~1000");
}
#[test]
fn test_srtt_first_sample() {
let mut s = SrttEstimator::new();
s.update(10_000); // 10ms
assert_eq!(s.srtt_us(), 10_000);
assert_eq!(s.rttvar_us(), 5_000);
assert!(s.initialized());
}
#[test]
fn test_srtt_convergence() {
let mut s = SrttEstimator::new();
// Feed constant 50ms RTT
for _ in 0..100 {
s.update(50_000);
}
let srtt = s.srtt_us();
assert!(
(srtt - 50_000).abs() < 1000,
"srtt={srtt}, expected ~50000"
);
}
#[test]
fn test_dual_ewma_initialization() {
let mut e = DualEwma::new();
assert!(!e.initialized());
e.update(100.0);
assert!(e.initialized());
assert_eq!(e.short(), 100.0);
assert_eq!(e.long(), 100.0);
}
#[test]
fn test_dual_ewma_short_tracks_faster() {
let mut e = DualEwma::new();
// Initialize at 0
e.update(0.0);
// Jump to 100
for _ in 0..20 {
e.update(100.0);
}
// Short should be closer to 100 than long
assert!(e.short() > e.long(), "short={} long={}", e.short(), e.long());
}
#[test]
fn test_owd_trend_flat() {
let mut d = OwdTrendDetector::new(32);
for i in 0..20 {
d.push(i, 5000); // constant OWD
}
let trend = d.trend_us_per_sec();
assert_eq!(trend, 0, "flat OWD should have zero trend");
}
#[test]
fn test_owd_trend_increasing() {
let mut d = OwdTrendDetector::new(32);
for i in 0..20 {
d.push(i, 5000 + (i as i64) * 100); // increasing by 100µs per packet
}
let trend = d.trend_us_per_sec();
assert!(trend > 0, "increasing OWD should have positive trend, got {trend}");
}
#[test]
fn test_owd_trend_insufficient_samples() {
let mut d = OwdTrendDetector::new(32);
d.push(0, 5000);
assert_eq!(d.trend_us_per_sec(), 0);
}
#[test]
fn test_etx_perfect_link() {
assert!((compute_etx(1.0, 1.0) - 1.0).abs() < f64::EPSILON);
}
#[test]
fn test_etx_lossy_link() {
// 10% forward loss, 5% reverse loss
let etx = compute_etx(0.9, 0.95);
assert!(etx > 1.0 && etx < 2.0, "etx={etx}");
}
#[test]
fn test_etx_zero_delivery() {
assert_eq!(compute_etx(0.0, 1.0), 100.0);
assert_eq!(compute_etx(1.0, 0.0), 100.0);
}
#[test]
fn test_spin_bit_initiator_rtt() {
let mut initiator = SpinBitState::new(true);
let mut responder = SpinBitState::new(false);
let t0 = Instant::now();
let t1 = t0 + std::time::Duration::from_millis(10);
let t2 = t0 + std::time::Duration::from_millis(20);
// Initiator sends with spin=false (initial)
let bit_to_send = initiator.tx_bit();
assert!(!bit_to_send);
// Responder receives, copies bit
responder.rx_observe(bit_to_send, 1, t0);
assert_eq!(responder.tx_bit(), false);
// Responder sends back, initiator receives
let resp_bit = responder.tx_bit();
let rtt1 = initiator.rx_observe(resp_bit, 2, t1);
// First edge: no previous edge to compare
assert!(rtt1.is_none());
// Now initiator's spin flipped to true
let bit2 = initiator.tx_bit();
assert!(bit2);
// Responder receives new bit
responder.rx_observe(bit2, 3, t1);
assert_eq!(responder.tx_bit(), true);
// Responder sends back, initiator receives
let resp_bit2 = responder.tx_bit();
let rtt2 = initiator.rx_observe(resp_bit2, 4, t2);
// Second edge: should produce an RTT sample
assert!(rtt2.is_some());
}
#[test]
fn test_spin_bit_responder_counter_guard() {
let mut responder = SpinBitState::new(false);
// Receive counter=5 with spin=true
responder.rx_observe(true, 5, Instant::now());
assert_eq!(responder.tx_bit(), true);
// Reordered packet with counter=3 and spin=false should be ignored
responder.rx_observe(false, 3, Instant::now());
assert_eq!(responder.tx_bit(), true); // unchanged
}
}
+286
View File
@@ -0,0 +1,286 @@
//! MMP derived metrics.
//!
//! `MmpMetrics` processes incoming ReceiverReports (from our peer) and
//! maintains derived metrics: SRTT, loss rate, goodput, ETX, and dual
//! EWMA trend indicators. Updated by the sender side when it receives
//! a ReceiverReport about its own traffic.
use crate::mmp::algorithms::{DualEwma, SrttEstimator, compute_etx};
use crate::mmp::report::ReceiverReport;
use std::time::Instant;
/// Derived MMP metrics, updated from incoming ReceiverReports.
///
/// This lives on the sender side: when we receive a ReceiverReport from
/// our peer describing what they observed about our traffic, we process
/// it here to compute RTT, loss, goodput, and trend indicators.
pub struct MmpMetrics {
/// Smoothed RTT from timestamp echo.
pub srtt: SrttEstimator,
/// Dual EWMA trend detectors.
pub rtt_trend: DualEwma,
pub loss_trend: DualEwma,
pub goodput_trend: DualEwma,
pub jitter_trend: DualEwma,
/// Forward delivery ratio (what fraction of our frames the peer received).
pub delivery_ratio_forward: f64,
/// Reverse delivery ratio (set when we compute from our own receiver state).
pub delivery_ratio_reverse: f64,
/// ETX computed from bidirectional delivery ratios.
pub etx: f64,
/// Smoothed goodput in bytes/sec (forward direction: what the peer received from us).
pub goodput_bps: f64,
// --- State for delta computation ---
/// Previous ReceiverReport's cumulative counters (for computing interval deltas).
prev_rr_cum_packets: u64,
prev_rr_cum_bytes: u64,
prev_rr_highest_counter: u64,
prev_rr_ecn_ce: u32,
prev_rr_reorder: u32,
/// Time of previous ReceiverReport (for goodput rate computation).
prev_rr_time: Option<Instant>,
}
impl MmpMetrics {
pub fn new() -> Self {
Self {
srtt: SrttEstimator::new(),
rtt_trend: DualEwma::new(),
loss_trend: DualEwma::new(),
goodput_trend: DualEwma::new(),
jitter_trend: DualEwma::new(),
delivery_ratio_forward: 1.0,
delivery_ratio_reverse: 1.0,
etx: 1.0,
goodput_bps: 0.0,
prev_rr_cum_packets: 0,
prev_rr_cum_bytes: 0,
prev_rr_highest_counter: 0,
prev_rr_ecn_ce: 0,
prev_rr_reorder: 0,
prev_rr_time: None,
}
}
/// Process an incoming ReceiverReport (from the peer about our traffic).
///
/// `our_timestamp_ms` is the current session-relative time in ms (for RTT).
/// `now` is the current monotonic time (for goodput rate computation).
pub fn process_receiver_report(&mut self, rr: &ReceiverReport, our_timestamp_ms: u32, now: Instant) {
// --- RTT from timestamp echo ---
// RTT = now - echoed_timestamp - dwell_time
if rr.timestamp_echo > 0 {
let echo_ms = rr.timestamp_echo;
let dwell_ms = rr.dwell_time as u32;
// Guard against timestamp wrap or bogus values
if our_timestamp_ms > echo_ms + dwell_ms {
let rtt_ms = our_timestamp_ms - echo_ms - dwell_ms;
let rtt_us = (rtt_ms as i64) * 1000;
self.srtt.update(rtt_us);
self.rtt_trend.update(rtt_us as f64);
}
}
// --- Loss rate from cumulative counters ---
// Delta: frames the peer should have received vs. actually received
if self.prev_rr_highest_counter > 0 {
let counter_span = rr.highest_counter.saturating_sub(self.prev_rr_highest_counter);
let packets_delta = rr.cumulative_packets_recv.saturating_sub(self.prev_rr_cum_packets);
if counter_span > 0 {
let delivery = (packets_delta as f64) / (counter_span as f64);
self.delivery_ratio_forward = delivery.clamp(0.0, 1.0);
let loss_rate = 1.0 - self.delivery_ratio_forward;
self.loss_trend.update(loss_rate);
self.etx = compute_etx(self.delivery_ratio_forward, self.delivery_ratio_reverse);
}
}
// --- Goodput from cumulative bytes + time delta ---
if self.prev_rr_cum_bytes > 0 {
let bytes_delta = rr.cumulative_bytes_recv.saturating_sub(self.prev_rr_cum_bytes);
self.goodput_trend.update(bytes_delta as f64);
// Compute bytes/sec if we have a time reference
if let Some(prev_time) = self.prev_rr_time {
let elapsed = now.duration_since(prev_time);
let secs = elapsed.as_secs_f64();
if secs > 0.0 {
let bps = bytes_delta as f64 / secs;
// EWMA smoothing: α = 1/4
if self.goodput_bps == 0.0 {
self.goodput_bps = bps;
} else {
self.goodput_bps += (bps - self.goodput_bps) * 0.25;
}
}
}
}
// --- Jitter trend ---
self.jitter_trend.update(rr.jitter as f64);
// --- Save for next delta ---
self.prev_rr_cum_packets = rr.cumulative_packets_recv;
self.prev_rr_cum_bytes = rr.cumulative_bytes_recv;
self.prev_rr_highest_counter = rr.highest_counter;
self.prev_rr_ecn_ce = rr.ecn_ce_count;
self.prev_rr_reorder = rr.cumulative_reorder_count;
self.prev_rr_time = Some(now);
}
/// Update the reverse delivery ratio (from our own receiver state about the peer's traffic).
pub fn set_delivery_ratio_reverse(&mut self, ratio: f64) {
self.delivery_ratio_reverse = ratio.clamp(0.0, 1.0);
self.etx = compute_etx(self.delivery_ratio_forward, self.delivery_ratio_reverse);
}
/// Current smoothed RTT in milliseconds, or `None` if not yet measured.
pub fn srtt_ms(&self) -> Option<f64> {
if self.srtt.initialized() {
Some(self.srtt.srtt_us() as f64 / 1000.0)
} else {
None
}
}
/// Current loss rate (0.0 = no loss, 1.0 = total loss).
pub fn loss_rate(&self) -> f64 {
1.0 - self.delivery_ratio_forward
}
/// Current smoothed goodput in bytes/sec, or 0 if not yet measured.
pub fn goodput_bps(&self) -> f64 {
self.goodput_bps
}
}
impl Default for MmpMetrics {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
use std::time::Duration;
fn make_rr(
highest_counter: u64,
cum_packets: u64,
cum_bytes: u64,
timestamp_echo: u32,
dwell: u16,
jitter: u32,
) -> ReceiverReport {
ReceiverReport {
highest_counter,
cumulative_packets_recv: cum_packets,
cumulative_bytes_recv: cum_bytes,
timestamp_echo,
dwell_time: dwell,
max_burst_loss: 0,
mean_burst_loss: 0,
jitter,
ecn_ce_count: 0,
owd_trend: 0,
burst_loss_count: 0,
cumulative_reorder_count: 0,
interval_packets_recv: 0,
interval_bytes_recv: 0,
}
}
#[test]
fn test_rtt_from_echo() {
let mut m = MmpMetrics::new();
let now = Instant::now();
// Peer echoes timestamp 1000ms, dwell=5ms, our current time=1050ms
let rr = make_rr(10, 10, 5000, 1000, 5, 0);
m.process_receiver_report(&rr, 1050, now);
assert!(m.srtt.initialized());
// RTT = 1050 - 1000 - 5 = 45ms
let srtt_ms = m.srtt_ms().unwrap();
assert!((srtt_ms - 45.0).abs() < 1.0, "srtt={srtt_ms}, expected ~45");
}
#[test]
fn test_loss_rate_computation() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
// First report: baseline
let rr1 = make_rr(100, 100, 50000, 0, 0, 0);
m.process_receiver_report(&rr1, 0, t0);
// Second report: 200 counters sent, 190 received (5% loss)
let rr2 = make_rr(300, 290, 145000, 0, 0, 0);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
let loss = m.loss_rate();
assert!((loss - 0.05).abs() < 0.01, "loss={loss}, expected ~0.05");
}
#[test]
fn test_etx_updates() {
let mut m = MmpMetrics::new();
assert_eq!(m.etx, 1.0); // initial: perfect
// Simulate some loss
m.delivery_ratio_forward = 0.9;
m.set_delivery_ratio_reverse(0.95);
assert!(m.etx > 1.0);
assert!(m.etx < 2.0);
}
#[test]
fn test_no_rtt_without_echo() {
let mut m = MmpMetrics::new();
let now = Instant::now();
let rr = make_rr(10, 10, 5000, 0, 0, 0);
m.process_receiver_report(&rr, 1000, now);
assert!(m.srtt_ms().is_none());
}
#[test]
fn test_jitter_trend() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
let rr1 = make_rr(10, 10, 5000, 0, 0, 100);
m.process_receiver_report(&rr1, 0, t0);
let rr2 = make_rr(20, 20, 10000, 0, 0, 500);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
assert!(m.jitter_trend.initialized());
// Short-term should be closer to 500 than long-term
assert!(m.jitter_trend.short() > m.jitter_trend.long());
}
#[test]
fn test_goodput_bps() {
let mut m = MmpMetrics::new();
let t0 = Instant::now();
// First report: baseline (50KB received)
let rr1 = make_rr(100, 100, 50_000, 0, 0, 0);
m.process_receiver_report(&rr1, 0, t0);
assert_eq!(m.goodput_bps(), 0.0); // no rate yet (first report)
// Second report 1s later: 150KB total (100KB delta in 1s = 100KB/s)
let rr2 = make_rr(300, 290, 150_000, 0, 0, 0);
m.process_receiver_report(&rr2, 0, t0 + Duration::from_secs(1));
assert!(m.goodput_bps() > 90_000.0, "goodput={}, expected ~100000", m.goodput_bps());
assert!(m.goodput_bps() < 110_000.0, "goodput={}, expected ~100000", m.goodput_bps());
}
}
+280
View File
@@ -0,0 +1,280 @@
//! Metrics Measurement Protocol (MMP) — link-layer instantiation.
//!
//! Measures link quality between adjacent peers: RTT, loss, jitter,
//! throughput, one-way delay trend, and ETX. Operates on the per-frame
//! hooks (counter, timestamp, flags) introduced by the FLP wire format
//! revision.
//!
//! Three operating modes trade measurement fidelity for overhead:
//! - **Full**: sender + receiver reports at RTT-adaptive intervals
//! - **Lightweight**: receiver reports only (infer loss from counters)
//! - **Minimal**: spin bit + CE echo only, no reports
use serde::{Deserialize, Serialize};
use std::fmt::{self, Debug};
use std::time::{Duration, Instant};
// Sub-modules
pub mod algorithms;
pub mod metrics;
pub mod receiver;
pub mod report;
pub mod sender;
// Re-exports
pub use algorithms::{
DualEwma, JitterEstimator, OwdTrendDetector, SpinBitState, SrttEstimator, compute_etx,
};
pub use metrics::MmpMetrics;
pub use receiver::ReceiverState;
pub use report::{ReceiverReport, SenderReport};
pub use sender::SenderState;
// ============================================================================
// Constants
// ============================================================================
/// SenderReport body size (after msg_type byte): 3 reserved + 44 payload = 47.
pub const SENDER_REPORT_BODY_SIZE: usize = 47;
/// ReceiverReport body size (after msg_type byte): 3 reserved + 64 payload = 67.
pub const RECEIVER_REPORT_BODY_SIZE: usize = 67;
/// SenderReport total wire size including inner header: 5 + 47 = 52.
pub const SENDER_REPORT_WIRE_SIZE: usize = 52;
/// ReceiverReport total wire size including inner header: 5 + 67 = 72.
pub const RECEIVER_REPORT_WIRE_SIZE: usize = 72;
// --- EWMA parameters (as shift amounts for integer arithmetic) ---
/// Jitter EWMA: α = 1/16 (RFC 3550 §6.4.1).
pub const JITTER_ALPHA_SHIFT: u32 = 4;
/// SRTT: α = 1/8 (Jacobson, RFC 6298).
pub const SRTT_ALPHA_SHIFT: u32 = 3;
/// RTTVAR: β = 1/4 (Jacobson, RFC 6298).
pub const RTTVAR_BETA_SHIFT: u32 = 2;
/// Dual EWMA short-term: α = 1/4.
pub const EWMA_SHORT_ALPHA: f64 = 0.25;
/// Dual EWMA long-term: α = 1/32.
pub const EWMA_LONG_ALPHA: f64 = 1.0 / 32.0;
// --- Timing defaults (milliseconds) ---
/// Default report interval before SRTT is available (cold start).
pub const DEFAULT_COLD_START_INTERVAL_MS: u64 = 200;
/// Minimum report interval (SRTT clamp floor).
pub const MIN_REPORT_INTERVAL_MS: u64 = 100;
/// Maximum report interval (SRTT clamp ceiling).
pub const MAX_REPORT_INTERVAL_MS: u64 = 2_000;
/// Default OWD ring buffer capacity.
pub const DEFAULT_OWD_WINDOW_SIZE: usize = 32;
/// Default operator log interval in seconds.
pub const DEFAULT_LOG_INTERVAL_SECS: u64 = 30;
// ============================================================================
// Operating Mode
// ============================================================================
/// MMP operating mode.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum MmpMode {
/// Sender + receiver reports at RTT-adaptive intervals. Maximum fidelity.
Full,
/// Receiver reports only. Loss inferred from counter gaps.
Lightweight,
/// Spin bit + CE echo only. No reports exchanged.
Minimal,
}
impl Default for MmpMode {
fn default() -> Self {
MmpMode::Full
}
}
impl fmt::Display for MmpMode {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
MmpMode::Full => write!(f, "full"),
MmpMode::Lightweight => write!(f, "lightweight"),
MmpMode::Minimal => write!(f, "minimal"),
}
}
}
// ============================================================================
// Configuration
// ============================================================================
/// MMP configuration (`node.mmp.*`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MmpConfig {
/// Operating mode (`node.mmp.mode`).
#[serde(default)]
pub mode: MmpMode,
/// Periodic operator log interval in seconds (`node.mmp.log_interval_secs`).
#[serde(default = "MmpConfig::default_log_interval_secs")]
pub log_interval_secs: u64,
/// OWD trend ring buffer size (`node.mmp.owd_window_size`).
#[serde(default = "MmpConfig::default_owd_window_size")]
pub owd_window_size: usize,
}
impl Default for MmpConfig {
fn default() -> Self {
Self {
mode: MmpMode::default(),
log_interval_secs: DEFAULT_LOG_INTERVAL_SECS,
owd_window_size: DEFAULT_OWD_WINDOW_SIZE,
}
}
}
impl MmpConfig {
fn default_log_interval_secs() -> u64 {
DEFAULT_LOG_INTERVAL_SECS
}
fn default_owd_window_size() -> usize {
DEFAULT_OWD_WINDOW_SIZE
}
}
// ============================================================================
// Per-Peer MMP State
// ============================================================================
/// Combined MMP state for a single peer link.
///
/// Wraps sender, receiver, metrics, and spin bit state. One instance
/// per `ActivePeer`.
pub struct MmpPeerState {
pub sender: SenderState,
pub receiver: ReceiverState,
pub metrics: MmpMetrics,
pub spin_bit: SpinBitState,
mode: MmpMode,
log_interval: Duration,
last_log_time: Option<Instant>,
}
impl MmpPeerState {
/// Create MMP state for a new peer link.
///
/// `is_initiator`: true if this node initiated the Noise handshake
/// (determines spin bit role).
pub fn new(config: &MmpConfig, is_initiator: bool) -> Self {
Self {
sender: SenderState::new(),
receiver: ReceiverState::new(config.owd_window_size),
metrics: MmpMetrics::new(),
spin_bit: SpinBitState::new(is_initiator),
mode: config.mode,
log_interval: Duration::from_secs(config.log_interval_secs),
last_log_time: None,
}
}
/// Current operating mode.
pub fn mode(&self) -> MmpMode {
self.mode
}
/// Check if it's time to emit a periodic metrics log.
pub fn should_log(&self, now: Instant) -> bool {
match self.last_log_time {
None => true,
Some(last) => now.duration_since(last) >= self.log_interval,
}
}
/// Mark that a periodic log was emitted.
pub fn mark_logged(&mut self, now: Instant) {
self.last_log_time = Some(now);
}
}
impl Debug for MmpPeerState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("MmpPeerState")
.field("mode", &self.mode)
.finish_non_exhaustive()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_mode_default() {
assert_eq!(MmpMode::default(), MmpMode::Full);
}
#[test]
fn test_mode_display() {
assert_eq!(MmpMode::Full.to_string(), "full");
assert_eq!(MmpMode::Lightweight.to_string(), "lightweight");
assert_eq!(MmpMode::Minimal.to_string(), "minimal");
}
#[test]
fn test_mode_serde_roundtrip() {
let yaml = "full";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Full);
let yaml = "lightweight";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Lightweight);
let yaml = "minimal";
let mode: MmpMode = serde_yaml::from_str(yaml).unwrap();
assert_eq!(mode, MmpMode::Minimal);
}
#[test]
fn test_config_default() {
let config = MmpConfig::default();
assert_eq!(config.mode, MmpMode::Full);
assert_eq!(config.log_interval_secs, 30);
assert_eq!(config.owd_window_size, 32);
}
#[test]
fn test_config_yaml_parse() {
let yaml = r#"
mode: lightweight
log_interval_secs: 60
owd_window_size: 48
"#;
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Lightweight);
assert_eq!(config.log_interval_secs, 60);
assert_eq!(config.owd_window_size, 48);
}
#[test]
fn test_config_yaml_partial() {
let yaml = "mode: minimal";
let config: MmpConfig = serde_yaml::from_str(yaml).unwrap();
assert_eq!(config.mode, MmpMode::Minimal);
assert_eq!(config.log_interval_secs, DEFAULT_LOG_INTERVAL_SECS);
assert_eq!(config.owd_window_size, DEFAULT_OWD_WINDOW_SIZE);
}
}
+511
View File
@@ -0,0 +1,511 @@
//! MMP receiver state machine.
//!
//! Tracks what this node has received from a specific peer and produces
//! ReceiverReport messages on demand. One `ReceiverState` per active peer.
use std::time::{Duration, Instant};
use crate::mmp::algorithms::{JitterEstimator, OwdTrendDetector};
use crate::mmp::report::ReceiverReport;
use crate::mmp::{DEFAULT_COLD_START_INTERVAL_MS, DEFAULT_OWD_WINDOW_SIZE,
MAX_REPORT_INTERVAL_MS, MIN_REPORT_INTERVAL_MS};
// ============================================================================
// Gap Tracker (burst loss detection)
// ============================================================================
/// Tracks counter gaps to detect loss bursts.
///
/// Each gap in the counter sequence is a burst of lost frames.
/// Maintains per-interval statistics that are reset when a report is built.
struct GapTracker {
/// Next expected counter value.
expected_next: Option<u64>,
/// Whether we are currently in a burst (gap).
in_burst: bool,
/// Length of the current burst.
current_burst_len: u16,
// --- Per-interval stats (reset on report) ---
/// Number of distinct burst events this interval.
burst_count: u32,
/// Longest burst in this interval.
max_burst_len: u16,
/// Sum of all burst lengths (for mean computation).
total_burst_len: u64,
}
impl GapTracker {
fn new() -> Self {
Self {
expected_next: None,
in_burst: false,
current_burst_len: 0,
burst_count: 0,
max_burst_len: 0,
total_burst_len: 0,
}
}
/// Process a received counter value. Returns the number of lost frames
/// detected (0 if in order or first frame).
fn observe(&mut self, counter: u64) -> u64 {
let Some(expected) = self.expected_next else {
// First frame: initialize
self.expected_next = Some(counter + 1);
return 0;
};
let lost = if counter > expected {
// Gap detected
let gap = counter - expected;
if self.in_burst {
// Extend current burst
self.current_burst_len = self.current_burst_len.saturating_add(gap as u16);
} else {
// New burst
self.in_burst = true;
self.current_burst_len = gap as u16;
self.burst_count += 1;
}
gap
} else {
// In-order or duplicate (counter <= expected)
if self.in_burst {
// End current burst
self.finish_burst();
}
0
};
// Update expected (always advance to counter+1 or keep expected if
// this was a late/reordered frame)
if counter >= expected {
self.expected_next = Some(counter + 1);
}
lost
}
/// Finish the current burst and record its stats.
fn finish_burst(&mut self) {
if self.in_burst {
self.max_burst_len = self.max_burst_len.max(self.current_burst_len);
self.total_burst_len += self.current_burst_len as u64;
self.in_burst = false;
self.current_burst_len = 0;
}
}
/// Get interval stats and reset for next interval.
fn take_interval_stats(&mut self) -> (u32, u16, u16) {
// Finish any in-progress burst
self.finish_burst();
let count = self.burst_count;
let max_len = self.max_burst_len;
let mean_len = if count > 0 {
// u8.8 fixed-point: (total / count) * 256
let mean_f = (self.total_burst_len as f64) / (count as f64);
(mean_f * 256.0) as u16
} else {
0
};
// Reset interval
self.burst_count = 0;
self.max_burst_len = 0;
self.total_burst_len = 0;
(count, max_len, mean_len)
}
}
// ============================================================================
// ReceiverState
// ============================================================================
/// Per-peer receiver-side MMP state.
///
/// Accumulates per-frame observations and produces `ReceiverReport` snapshots.
pub struct ReceiverState {
// --- Cumulative (lifetime) ---
cumulative_packets_recv: u64,
cumulative_bytes_recv: u64,
cumulative_reorder_count: u64,
/// Highest counter value ever received.
highest_counter: u64,
// --- Current interval ---
interval_packets_recv: u32,
interval_bytes_recv: u32,
// --- Jitter ---
jitter: JitterEstimator,
// --- OWD trend ---
owd_trend: OwdTrendDetector,
/// Monotonic sequence counter for OWD samples.
owd_seq: u32,
// --- Loss tracking ---
gap_tracker: GapTracker,
// --- ECN ---
ecn_ce_count: u32,
// --- Timestamp echo ---
/// Sender timestamp from the most recent frame (for echo).
last_sender_timestamp: u32,
/// Local time when the most recent frame was received (for dwell computation).
last_recv_time: Option<Instant>,
// --- Report timing ---
last_report_time: Option<Instant>,
report_interval: Duration,
/// Whether any frames have been received since the last report.
interval_has_data: bool,
}
impl ReceiverState {
pub fn new(owd_window_size: usize) -> Self {
Self {
cumulative_packets_recv: 0,
cumulative_bytes_recv: 0,
cumulative_reorder_count: 0,
highest_counter: 0,
interval_packets_recv: 0,
interval_bytes_recv: 0,
jitter: JitterEstimator::new(),
owd_trend: OwdTrendDetector::new(owd_window_size),
owd_seq: 0,
gap_tracker: GapTracker::new(),
ecn_ce_count: 0,
last_sender_timestamp: 0,
last_recv_time: None,
last_report_time: None,
report_interval: Duration::from_millis(DEFAULT_COLD_START_INTERVAL_MS),
interval_has_data: false,
}
}
/// Record a received frame from this peer.
///
/// Called on the RX path after AEAD decryption, before message dispatch.
///
/// - `counter`: AEAD counter from outer header
/// - `sender_timestamp_ms`: session-relative timestamp from inner header (ms)
/// - `bytes`: wire payload size
/// - `ce_flag`: CE bit from flags byte
/// - `now`: current local time
pub fn record_recv(
&mut self,
counter: u64,
sender_timestamp_ms: u32,
bytes: usize,
ce_flag: bool,
now: Instant,
) {
self.interval_has_data = true;
self.cumulative_packets_recv += 1;
self.cumulative_bytes_recv += bytes as u64;
self.interval_packets_recv = self.interval_packets_recv.saturating_add(1);
self.interval_bytes_recv = self.interval_bytes_recv.saturating_add(bytes as u32);
// Reordering detection: counter < highest means out-of-order
if counter < self.highest_counter {
self.cumulative_reorder_count += 1;
} else {
self.highest_counter = counter;
}
// Loss/burst detection
let _lost = self.gap_tracker.observe(counter);
// ECN
if ce_flag {
self.ecn_ce_count = self.ecn_ce_count.saturating_add(1);
}
// Jitter: compute transit time delta
// Transit = recv_local - sender_timestamp (in µs for precision)
// We use a monotonic local reference derived from Instant offsets.
let sender_us = (sender_timestamp_ms as i64) * 1000;
// We can't get absolute µs from Instant, but we can compute the delta
// between consecutive transits using relative Instant differences.
if let Some(prev_recv) = self.last_recv_time {
let recv_delta_us = now.duration_since(prev_recv).as_micros() as i64;
let send_delta_us = sender_us - (self.last_sender_timestamp as i64 * 1000);
let transit_delta = (recv_delta_us - send_delta_us) as i32;
self.jitter.update(transit_delta);
}
// OWD trend: use sender timestamp as a proxy for send time
// and Instant delta from a fixed reference as receive time.
// Since we only need the *trend* (slope), absolute offsets cancel out.
if let Some(first_recv) = self.last_recv_time.or(Some(now)) {
let recv_offset_us = now.duration_since(first_recv).as_micros() as i64;
let owd_us = recv_offset_us - sender_us;
self.owd_seq = self.owd_seq.wrapping_add(1);
self.owd_trend.push(self.owd_seq, owd_us);
}
// Timestamp echo state
self.last_sender_timestamp = sender_timestamp_ms;
self.last_recv_time = Some(now);
}
/// Build a ReceiverReport from current state and reset the interval.
///
/// Returns `None` if no frames have been received since the last report.
pub fn build_report(&mut self, now: Instant) -> Option<ReceiverReport> {
if !self.interval_has_data {
return None;
}
// Dwell time: ms between last frame reception and report generation
let dwell_time = self.last_recv_time
.map(|t| now.duration_since(t).as_millis() as u16)
.unwrap_or(0);
let (burst_count, max_burst, mean_burst) = self.gap_tracker.take_interval_stats();
let report = ReceiverReport {
highest_counter: self.highest_counter,
cumulative_packets_recv: self.cumulative_packets_recv,
cumulative_bytes_recv: self.cumulative_bytes_recv,
timestamp_echo: self.last_sender_timestamp,
dwell_time,
max_burst_loss: max_burst,
mean_burst_loss: mean_burst,
jitter: self.jitter.jitter_us(),
ecn_ce_count: self.ecn_ce_count,
owd_trend: self.owd_trend.trend_us_per_sec(),
burst_loss_count: burst_count,
cumulative_reorder_count: self.cumulative_reorder_count as u32,
interval_packets_recv: self.interval_packets_recv,
interval_bytes_recv: self.interval_bytes_recv,
};
// Reset interval
self.interval_packets_recv = 0;
self.interval_bytes_recv = 0;
self.interval_has_data = false;
self.last_report_time = Some(now);
Some(report)
}
/// Check if it's time to send a report.
pub fn should_send_report(&self, now: Instant) -> bool {
if !self.interval_has_data {
return false;
}
match self.last_report_time {
None => true,
Some(last) => now.duration_since(last) >= self.report_interval,
}
}
/// Update the report interval based on SRTT.
///
/// Receiver reports at 1× SRTT, clamped to [MIN, MAX].
pub fn update_report_interval_from_srtt(&mut self, srtt_us: i64) {
if srtt_us <= 0 {
return;
}
let interval_ms = ((srtt_us as u64) / 1000)
.clamp(MIN_REPORT_INTERVAL_MS, MAX_REPORT_INTERVAL_MS);
self.report_interval = Duration::from_millis(interval_ms);
}
// --- Accessors ---
pub fn cumulative_packets_recv(&self) -> u64 {
self.cumulative_packets_recv
}
pub fn cumulative_bytes_recv(&self) -> u64 {
self.cumulative_bytes_recv
}
pub fn highest_counter(&self) -> u64 {
self.highest_counter
}
pub fn jitter_us(&self) -> u32 {
self.jitter.jitter_us()
}
pub fn report_interval(&self) -> Duration {
self.report_interval
}
}
impl Default for ReceiverState {
fn default() -> Self {
Self::new(DEFAULT_OWD_WINDOW_SIZE)
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_receiver_state() {
let r = ReceiverState::new(32);
assert_eq!(r.cumulative_packets_recv(), 0);
assert_eq!(r.cumulative_bytes_recv(), 0);
assert_eq!(r.highest_counter(), 0);
}
#[test]
fn test_record_recv_basic() {
let mut r = ReceiverState::new(32);
let now = Instant::now();
r.record_recv(1, 100, 500, false, now);
r.record_recv(2, 200, 600, false, now + Duration::from_millis(100));
assert_eq!(r.cumulative_packets_recv(), 2);
assert_eq!(r.cumulative_bytes_recv(), 1100);
assert_eq!(r.highest_counter(), 2);
}
#[test]
fn test_reorder_detection() {
let mut r = ReceiverState::new(32);
let now = Instant::now();
r.record_recv(5, 500, 100, false, now);
r.record_recv(3, 300, 100, false, now + Duration::from_millis(10));
assert_eq!(r.cumulative_reorder_count, 1);
assert_eq!(r.highest_counter(), 5); // not changed by out-of-order
}
#[test]
fn test_ecn_counting() {
let mut r = ReceiverState::new(32);
let now = Instant::now();
r.record_recv(1, 100, 100, true, now);
r.record_recv(2, 200, 100, false, now);
r.record_recv(3, 300, 100, true, now);
assert_eq!(r.ecn_ce_count, 2);
}
#[test]
fn test_build_report_empty() {
let mut r = ReceiverState::new(32);
assert!(r.build_report(Instant::now()).is_none());
}
#[test]
fn test_build_report() {
let mut r = ReceiverState::new(32);
let t0 = Instant::now();
r.record_recv(1, 100, 500, false, t0);
r.record_recv(2, 200, 600, false, t0 + Duration::from_millis(100));
let report = r.build_report(t0 + Duration::from_millis(150)).unwrap();
assert_eq!(report.highest_counter, 2);
assert_eq!(report.cumulative_packets_recv, 2);
assert_eq!(report.cumulative_bytes_recv, 1100);
assert_eq!(report.timestamp_echo, 200); // last sender timestamp
assert_eq!(report.interval_packets_recv, 2);
assert_eq!(report.interval_bytes_recv, 1100);
}
#[test]
fn test_build_report_resets_interval() {
let mut r = ReceiverState::new(32);
let t0 = Instant::now();
r.record_recv(1, 100, 500, false, t0);
let _ = r.build_report(t0);
// No new data
assert!(r.build_report(t0).is_none());
// New data
r.record_recv(2, 200, 300, false, t0 + Duration::from_millis(100));
let report = r.build_report(t0 + Duration::from_millis(150)).unwrap();
assert_eq!(report.interval_packets_recv, 1);
assert_eq!(report.interval_bytes_recv, 300);
// Cumulative continues
assert_eq!(report.cumulative_packets_recv, 2);
}
#[test]
fn test_gap_tracker_no_loss() {
let mut g = GapTracker::new();
g.observe(1);
g.observe(2);
g.observe(3);
let (count, max, mean) = g.take_interval_stats();
assert_eq!(count, 0);
assert_eq!(max, 0);
assert_eq!(mean, 0);
}
#[test]
fn test_gap_tracker_single_burst() {
let mut g = GapTracker::new();
g.observe(1);
// frames 2, 3 lost
g.observe(4);
g.observe(5);
let (count, max, _mean) = g.take_interval_stats();
assert_eq!(count, 1);
assert_eq!(max, 2);
}
#[test]
fn test_gap_tracker_multiple_bursts() {
let mut g = GapTracker::new();
g.observe(1);
g.observe(4); // burst of 2 (frames 2,3 lost)
g.observe(5);
g.observe(8); // burst of 2 (frames 6,7 lost)
g.observe(9);
let (count, max, mean) = g.take_interval_stats();
assert_eq!(count, 2);
assert_eq!(max, 2);
// mean = 2.0 in u8.8 = 512
assert_eq!(mean, 512);
}
#[test]
fn test_should_send_report_timing() {
let mut r = ReceiverState::new(32);
let t0 = Instant::now();
assert!(!r.should_send_report(t0)); // no data
r.record_recv(1, 100, 500, false, t0);
assert!(r.should_send_report(t0)); // first time, has data
let _ = r.build_report(t0);
r.record_recv(2, 200, 500, false, t0);
assert!(!r.should_send_report(t0)); // just reported
let t1 = t0 + r.report_interval() + Duration::from_millis(1);
assert!(r.should_send_report(t1));
}
#[test]
fn test_update_report_interval() {
let mut r = ReceiverState::new(32);
// 50ms SRTT → 100ms receiver interval (1× SRTT, clamped to min)
r.update_report_interval_from_srtt(50_000);
assert_eq!(r.report_interval(), Duration::from_millis(100));
// 500ms SRTT → 500ms
r.update_report_interval_from_srtt(500_000);
assert_eq!(r.report_interval(), Duration::from_millis(500));
}
}
+309
View File
@@ -0,0 +1,309 @@
//! MMP report wire format: SenderReport and ReceiverReport.
//!
//! Serialization and deserialization for the two report types exchanged
//! between link-layer peers. Wire format follows the MMP design doc.
use crate::protocol::ProtocolError;
// ============================================================================
// SenderReport (msg_type 0x01, 48-byte body including type byte)
// ============================================================================
/// Link-layer sender report.
///
/// Wire layout (48 bytes total, sent as link message):
/// ```text
/// [0] msg_type = 0x01
/// [1-3] reserved (zero)
/// [4-11] interval_start_counter: u64 LE
/// [12-19] interval_end_counter: u64 LE
/// [20-23] interval_start_timestamp: u32 LE
/// [24-27] interval_end_timestamp: u32 LE
/// [28-31] interval_bytes_sent: u32 LE
/// [32-39] cumulative_packets_sent: u64 LE
/// [40-47] cumulative_bytes_sent: u64 LE
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SenderReport {
pub interval_start_counter: u64,
pub interval_end_counter: u64,
pub interval_start_timestamp: u32,
pub interval_end_timestamp: u32,
pub interval_bytes_sent: u32,
pub cumulative_packets_sent: u64,
pub cumulative_bytes_sent: u64,
}
/// ReceiverReport (msg_type 0x02, 68-byte body including type byte)
///
/// Wire layout (68 bytes total, sent as link message):
/// ```text
/// [0] msg_type = 0x02
/// [1-3] reserved (zero)
/// [4-11] highest_counter: u64 LE
/// [12-19] cumulative_packets_recv: u64 LE
/// [20-27] cumulative_bytes_recv: u64 LE
/// [28-31] timestamp_echo: u32 LE
/// [32-33] dwell_time: u16 LE
/// [34-35] max_burst_loss: u16 LE
/// [36-37] mean_burst_loss: u16 LE (u8.8 fixed-point)
/// [38-39] reserved: u16 LE
/// [40-43] jitter: u32 LE (microseconds)
/// [44-47] ecn_ce_count: u32 LE
/// [48-51] owd_trend: i32 LE (µs/s)
/// [52-55] burst_loss_count: u32 LE
/// [56-59] cumulative_reorder_count: u32 LE
/// [60-63] interval_packets_recv: u32 LE
/// [64-67] interval_bytes_recv: u32 LE
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ReceiverReport {
pub highest_counter: u64,
pub cumulative_packets_recv: u64,
pub cumulative_bytes_recv: u64,
pub timestamp_echo: u32,
pub dwell_time: u16,
pub max_burst_loss: u16,
pub mean_burst_loss: u16,
pub jitter: u32,
pub ecn_ce_count: u32,
pub owd_trend: i32,
pub burst_loss_count: u32,
pub cumulative_reorder_count: u32,
pub interval_packets_recv: u32,
pub interval_bytes_recv: u32,
}
// Encode/decode will be implemented in Step 2.
impl SenderReport {
/// Encode to wire format (48 bytes: msg_type + 3 reserved + 44 payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(48);
buf.push(0x01); // msg_type
buf.extend_from_slice(&[0u8; 3]); // reserved
buf.extend_from_slice(&self.interval_start_counter.to_le_bytes());
buf.extend_from_slice(&self.interval_end_counter.to_le_bytes());
buf.extend_from_slice(&self.interval_start_timestamp.to_le_bytes());
buf.extend_from_slice(&self.interval_end_timestamp.to_le_bytes());
buf.extend_from_slice(&self.interval_bytes_sent.to_le_bytes());
buf.extend_from_slice(&self.cumulative_packets_sent.to_le_bytes());
buf.extend_from_slice(&self.cumulative_bytes_sent.to_le_bytes());
buf
}
/// Decode from payload after msg_type byte has been consumed.
///
/// `payload` starts at the reserved bytes (offset 1 in the wire format).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
if payload.len() < 47 {
return Err(ProtocolError::MessageTooShort {
expected: 47,
got: payload.len(),
});
}
// Skip 3 reserved bytes
let p = &payload[3..];
Ok(Self {
interval_start_counter: u64::from_le_bytes(p[0..8].try_into().unwrap()),
interval_end_counter: u64::from_le_bytes(p[8..16].try_into().unwrap()),
interval_start_timestamp: u32::from_le_bytes(p[16..20].try_into().unwrap()),
interval_end_timestamp: u32::from_le_bytes(p[20..24].try_into().unwrap()),
interval_bytes_sent: u32::from_le_bytes(p[24..28].try_into().unwrap()),
cumulative_packets_sent: u64::from_le_bytes(p[28..36].try_into().unwrap()),
cumulative_bytes_sent: u64::from_le_bytes(p[36..44].try_into().unwrap()),
})
}
}
impl ReceiverReport {
/// Encode to wire format (68 bytes: msg_type + 3 reserved + 64 payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(68);
buf.push(0x02); // msg_type
buf.extend_from_slice(&[0u8; 3]); // reserved
buf.extend_from_slice(&self.highest_counter.to_le_bytes());
buf.extend_from_slice(&self.cumulative_packets_recv.to_le_bytes());
buf.extend_from_slice(&self.cumulative_bytes_recv.to_le_bytes());
buf.extend_from_slice(&self.timestamp_echo.to_le_bytes());
buf.extend_from_slice(&self.dwell_time.to_le_bytes());
buf.extend_from_slice(&self.max_burst_loss.to_le_bytes());
buf.extend_from_slice(&self.mean_burst_loss.to_le_bytes());
buf.extend_from_slice(&[0u8; 2]); // reserved
buf.extend_from_slice(&self.jitter.to_le_bytes());
buf.extend_from_slice(&self.ecn_ce_count.to_le_bytes());
buf.extend_from_slice(&self.owd_trend.to_le_bytes());
buf.extend_from_slice(&self.burst_loss_count.to_le_bytes());
buf.extend_from_slice(&self.cumulative_reorder_count.to_le_bytes());
buf.extend_from_slice(&self.interval_packets_recv.to_le_bytes());
buf.extend_from_slice(&self.interval_bytes_recv.to_le_bytes());
buf
}
/// Decode from payload after msg_type byte has been consumed.
///
/// `payload` starts at the reserved bytes (offset 1 in the wire format).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
if payload.len() < 67 {
return Err(ProtocolError::MessageTooShort {
expected: 67,
got: payload.len(),
});
}
// Skip 3 reserved bytes
let p = &payload[3..];
Ok(Self {
highest_counter: u64::from_le_bytes(p[0..8].try_into().unwrap()),
cumulative_packets_recv: u64::from_le_bytes(p[8..16].try_into().unwrap()),
cumulative_bytes_recv: u64::from_le_bytes(p[16..24].try_into().unwrap()),
timestamp_echo: u32::from_le_bytes(p[24..28].try_into().unwrap()),
dwell_time: u16::from_le_bytes(p[28..30].try_into().unwrap()),
max_burst_loss: u16::from_le_bytes(p[30..32].try_into().unwrap()),
mean_burst_loss: u16::from_le_bytes(p[32..34].try_into().unwrap()),
// skip 2 reserved bytes at p[34..36]
jitter: u32::from_le_bytes(p[36..40].try_into().unwrap()),
ecn_ce_count: u32::from_le_bytes(p[40..44].try_into().unwrap()),
owd_trend: i32::from_le_bytes(p[44..48].try_into().unwrap()),
burst_loss_count: u32::from_le_bytes(p[48..52].try_into().unwrap()),
cumulative_reorder_count: u32::from_le_bytes(p[52..56].try_into().unwrap()),
interval_packets_recv: u32::from_le_bytes(p[56..60].try_into().unwrap()),
interval_bytes_recv: u32::from_le_bytes(p[60..64].try_into().unwrap()),
})
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
fn sample_sender_report() -> SenderReport {
SenderReport {
interval_start_counter: 100,
interval_end_counter: 200,
interval_start_timestamp: 5000,
interval_end_timestamp: 6000,
interval_bytes_sent: 50_000,
cumulative_packets_sent: 10_000,
cumulative_bytes_sent: 5_000_000,
}
}
fn sample_receiver_report() -> ReceiverReport {
ReceiverReport {
highest_counter: 195,
cumulative_packets_recv: 9_500,
cumulative_bytes_recv: 4_750_000,
timestamp_echo: 5900,
dwell_time: 5,
max_burst_loss: 3,
mean_burst_loss: 384, // 1.5 in u8.8
jitter: 1200,
ecn_ce_count: 0,
owd_trend: -50,
burst_loss_count: 2,
cumulative_reorder_count: 10,
interval_packets_recv: 95,
interval_bytes_recv: 47_500,
}
}
#[test]
fn test_sender_report_encode_size() {
let sr = sample_sender_report();
let encoded = sr.encode();
assert_eq!(encoded.len(), 48);
assert_eq!(encoded[0], 0x01); // msg_type
}
#[test]
fn test_sender_report_roundtrip() {
let sr = sample_sender_report();
let encoded = sr.encode();
// decode expects payload after msg_type
let decoded = SenderReport::decode(&encoded[1..]).unwrap();
assert_eq!(sr, decoded);
}
#[test]
fn test_sender_report_too_short() {
let result = SenderReport::decode(&[0u8; 10]);
assert!(result.is_err());
}
#[test]
fn test_receiver_report_encode_size() {
let rr = sample_receiver_report();
let encoded = rr.encode();
assert_eq!(encoded.len(), 68);
assert_eq!(encoded[0], 0x02); // msg_type
}
#[test]
fn test_receiver_report_roundtrip() {
let rr = sample_receiver_report();
let encoded = rr.encode();
// decode expects payload after msg_type
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(rr, decoded);
}
#[test]
fn test_receiver_report_too_short() {
let result = ReceiverReport::decode(&[0u8; 10]);
assert!(result.is_err());
}
#[test]
fn test_sender_report_zero_values() {
let sr = SenderReport {
interval_start_counter: 0,
interval_end_counter: 0,
interval_start_timestamp: 0,
interval_end_timestamp: 0,
interval_bytes_sent: 0,
cumulative_packets_sent: 0,
cumulative_bytes_sent: 0,
};
let encoded = sr.encode();
let decoded = SenderReport::decode(&encoded[1..]).unwrap();
assert_eq!(sr, decoded);
}
#[test]
fn test_receiver_report_max_values() {
let rr = ReceiverReport {
highest_counter: u64::MAX,
cumulative_packets_recv: u64::MAX,
cumulative_bytes_recv: u64::MAX,
timestamp_echo: u32::MAX,
dwell_time: u16::MAX,
max_burst_loss: u16::MAX,
mean_burst_loss: u16::MAX,
jitter: u32::MAX,
ecn_ce_count: u32::MAX,
owd_trend: i32::MAX,
burst_loss_count: u32::MAX,
cumulative_reorder_count: u32::MAX,
interval_packets_recv: u32::MAX,
interval_bytes_recv: u32::MAX,
};
let encoded = rr.encode();
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(rr, decoded);
}
#[test]
fn test_receiver_report_negative_owd_trend() {
let rr = ReceiverReport {
owd_trend: -12345,
..sample_receiver_report()
};
let encoded = rr.encode();
let decoded = ReceiverReport::decode(&encoded[1..]).unwrap();
assert_eq!(decoded.owd_trend, -12345);
}
}
+252
View File
@@ -0,0 +1,252 @@
//! MMP sender state machine.
//!
//! Tracks what this node has sent to a specific peer and produces
//! SenderReport messages on demand. One `SenderState` per active peer.
use std::time::{Duration, Instant};
use crate::mmp::report::SenderReport;
use crate::mmp::{DEFAULT_COLD_START_INTERVAL_MS, MAX_REPORT_INTERVAL_MS, MIN_REPORT_INTERVAL_MS};
/// Per-peer sender-side MMP state.
///
/// Records cumulative and interval counters for every frame transmitted
/// to this peer. Produces `SenderReport` snapshots on demand.
pub struct SenderState {
// --- Cumulative (lifetime) ---
cumulative_packets_sent: u64,
cumulative_bytes_sent: u64,
// --- Current interval ---
interval_start_counter: u64,
interval_start_timestamp: u32,
interval_bytes_sent: u32,
/// Counter of the most recently sent frame.
last_counter: u64,
/// Timestamp of the most recently sent frame.
last_timestamp: u32,
/// Whether any frames have been sent in the current interval.
interval_has_data: bool,
// --- Report timing ---
last_report_time: Option<Instant>,
report_interval: Duration,
}
impl SenderState {
pub fn new() -> Self {
Self {
cumulative_packets_sent: 0,
cumulative_bytes_sent: 0,
interval_start_counter: 0,
interval_start_timestamp: 0,
interval_bytes_sent: 0,
last_counter: 0,
last_timestamp: 0,
interval_has_data: false,
last_report_time: None,
report_interval: Duration::from_millis(DEFAULT_COLD_START_INTERVAL_MS),
}
}
/// Record a frame sent to this peer.
///
/// Called on the TX path for every encrypted link message.
/// `counter` is the AEAD nonce/counter, `timestamp` is the inner header
/// session-relative timestamp (ms), `bytes` is the wire payload size.
pub fn record_sent(&mut self, counter: u64, timestamp: u32, bytes: usize) {
if !self.interval_has_data {
self.interval_start_counter = counter;
self.interval_start_timestamp = timestamp;
self.interval_has_data = true;
}
self.last_counter = counter;
self.last_timestamp = timestamp;
self.interval_bytes_sent = self.interval_bytes_sent.saturating_add(bytes as u32);
self.cumulative_packets_sent += 1;
self.cumulative_bytes_sent += bytes as u64;
}
/// Build a SenderReport from current state and reset the interval.
///
/// Returns `None` if no frames have been sent since the last report.
pub fn build_report(&mut self, now: Instant) -> Option<SenderReport> {
if !self.interval_has_data {
return None;
}
let report = SenderReport {
interval_start_counter: self.interval_start_counter,
interval_end_counter: self.last_counter,
interval_start_timestamp: self.interval_start_timestamp,
interval_end_timestamp: self.last_timestamp,
interval_bytes_sent: self.interval_bytes_sent,
cumulative_packets_sent: self.cumulative_packets_sent,
cumulative_bytes_sent: self.cumulative_bytes_sent,
};
// Reset interval
self.interval_has_data = false;
self.interval_bytes_sent = 0;
self.last_report_time = Some(now);
Some(report)
}
/// Check if it's time to send a report.
pub fn should_send_report(&self, now: Instant) -> bool {
if !self.interval_has_data {
return false;
}
match self.last_report_time {
None => true, // Never sent a report — send immediately
Some(last) => now.duration_since(last) >= self.report_interval,
}
}
/// Update the report interval based on SRTT.
///
/// Sender reports at 2-5× the receiver report interval. For simplicity,
/// we use 2× SRTT clamped to [MIN, MAX].
pub fn update_report_interval_from_srtt(&mut self, srtt_us: i64) {
if srtt_us <= 0 {
return;
}
let interval_us = (srtt_us * 2) as u64;
let interval_ms = (interval_us / 1000)
.clamp(MIN_REPORT_INTERVAL_MS, MAX_REPORT_INTERVAL_MS);
self.report_interval = Duration::from_millis(interval_ms);
}
// --- Accessors ---
pub fn cumulative_packets_sent(&self) -> u64 {
self.cumulative_packets_sent
}
pub fn cumulative_bytes_sent(&self) -> u64 {
self.cumulative_bytes_sent
}
pub fn report_interval(&self) -> Duration {
self.report_interval
}
}
impl Default for SenderState {
fn default() -> Self {
Self::new()
}
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_new_sender_state() {
let s = SenderState::new();
assert_eq!(s.cumulative_packets_sent(), 0);
assert_eq!(s.cumulative_bytes_sent(), 0);
}
#[test]
fn test_record_sent() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
s.record_sent(2, 200, 600);
assert_eq!(s.cumulative_packets_sent(), 2);
assert_eq!(s.cumulative_bytes_sent(), 1100);
}
#[test]
fn test_build_report_empty() {
let mut s = SenderState::new();
assert!(s.build_report(Instant::now()).is_none());
}
#[test]
fn test_build_report() {
let mut s = SenderState::new();
s.record_sent(10, 1000, 500);
s.record_sent(11, 1100, 600);
s.record_sent(12, 1200, 400);
let report = s.build_report(Instant::now()).unwrap();
assert_eq!(report.interval_start_counter, 10);
assert_eq!(report.interval_end_counter, 12);
assert_eq!(report.interval_start_timestamp, 1000);
assert_eq!(report.interval_end_timestamp, 1200);
assert_eq!(report.interval_bytes_sent, 1500);
assert_eq!(report.cumulative_packets_sent, 3);
assert_eq!(report.cumulative_bytes_sent, 1500);
}
#[test]
fn test_build_report_resets_interval() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
let _ = s.build_report(Instant::now());
// Second report with no new data returns None
assert!(s.build_report(Instant::now()).is_none());
// New data starts a fresh interval
s.record_sent(2, 200, 300);
let report = s.build_report(Instant::now()).unwrap();
assert_eq!(report.interval_start_counter, 2);
assert_eq!(report.interval_bytes_sent, 300);
// Cumulative continues
assert_eq!(report.cumulative_packets_sent, 2);
assert_eq!(report.cumulative_bytes_sent, 800);
}
#[test]
fn test_should_send_report_no_data() {
let s = SenderState::new();
assert!(!s.should_send_report(Instant::now()));
}
#[test]
fn test_should_send_report_first_time() {
let mut s = SenderState::new();
s.record_sent(1, 100, 500);
assert!(s.should_send_report(Instant::now()));
}
#[test]
fn test_should_send_report_respects_interval() {
let mut s = SenderState::new();
let t0 = Instant::now();
s.record_sent(1, 100, 500);
let _ = s.build_report(t0);
s.record_sent(2, 200, 500);
// Immediately after report — should not send
assert!(!s.should_send_report(t0));
// After interval elapses
let t1 = t0 + s.report_interval() + Duration::from_millis(1);
assert!(s.should_send_report(t1));
}
#[test]
fn test_update_report_interval() {
let mut s = SenderState::new();
// 50ms RTT → 100ms sender interval (2× SRTT), clamped to min 100ms
s.update_report_interval_from_srtt(50_000);
assert_eq!(s.report_interval(), Duration::from_millis(100));
// 500ms RTT → 1000ms sender interval
s.update_report_interval_from_srtt(500_000);
assert_eq!(s.report_interval(), Duration::from_millis(1000));
// 2s RTT → 4s, clamped to max 2s
s.update_report_interval_from_srtt(2_000_000);
assert_eq!(s.report_interval(), Duration::from_millis(MAX_REPORT_INTERVAL_MS));
}
}
+17 -4
View File
@@ -17,6 +17,18 @@ impl Node {
let payload = &plaintext[1..];
match msg_type {
0x00 => {
// SessionDatagram
self.handle_session_datagram(from, payload).await;
}
0x01 => {
// SenderReport
self.handle_sender_report(from, payload);
}
0x02 => {
// ReceiverReport
self.handle_receiver_report(from, payload);
}
0x10 => {
// TreeAnnounce
self.handle_tree_announce(from, payload).await;
@@ -33,10 +45,6 @@ impl Node {
// LookupResponse
self.handle_lookup_response(from, payload).await;
}
0x40 => {
// SessionDatagram
self.handle_session_datagram(from, payload).await;
}
0x50 => {
// Disconnect
self.handle_disconnect(from, payload);
@@ -86,6 +94,11 @@ impl Node {
}
};
// MMP teardown log (before we drop the peer)
if let Some(mmp) = peer.mmp() {
Self::log_mmp_teardown(node_addr, mmp);
}
let link_id = peer.link_id();
// Free session index
+43 -7
View File
@@ -1,12 +1,13 @@
//! Encrypted frame handling (hot path).
use crate::node::Node;
use crate::node::wire::{EncryptedHeader, strip_inner_header, FLAG_CE, FLAG_SP};
use crate::transport::ReceivedPacket;
use crate::node::wire::EncryptedHeader;
use std::time::Instant;
use tracing::{debug, warn};
impl Node {
/// Handle an encrypted frame (discriminator 0x00).
/// Handle an encrypted frame (phase 0x0).
///
/// This is the hot path for established sessions. We use O(1)
/// index-based lookup to find the session, then decrypt.
@@ -53,9 +54,13 @@ impl Node {
}
};
// Decrypt with replay check (this is the expensive part)
let ciphertext = &packet.data[header.ciphertext_offset..];
let plaintext = match session.decrypt_with_replay_check(ciphertext, header.counter) {
// Decrypt with replay check and AAD (this is the expensive part)
let ciphertext = &packet.data[header.ciphertext_offset()..];
let plaintext = match session.decrypt_with_replay_check_and_aad(
ciphertext,
header.counter,
&header.header_bytes,
) {
Ok(p) => p,
Err(e) => {
debug!(
@@ -70,6 +75,37 @@ impl Node {
// === PACKET IS AUTHENTIC ===
// Strip inner header (4-byte timestamp + msg_type)
let (timestamp, link_message) = match strip_inner_header(&plaintext) {
Some(parts) => parts,
None => {
debug!(
node_addr = %node_addr,
len = plaintext.len(),
"Decrypted payload too short for inner header"
);
return;
}
};
// MMP per-frame processing: feed counter, timestamp, flags to receiver state
let now = Instant::now();
let ce_flag = header.flags & FLAG_CE != 0;
let sp_flag = header.flags & FLAG_SP != 0;
if let Some(mmp) = peer.mmp_mut() {
mmp.receiver.record_recv(
header.counter,
timestamp,
packet.data.len(),
ce_flag,
now,
);
// Spin bit: feed to spin state, get optional RTT sample
if let Some(rtt) = mmp.spin_bit.rx_observe(sp_flag, header.counter, now) {
mmp.metrics.srtt.update(rtt.as_micros() as i64);
}
}
// Update address for roaming support
peer.set_current_addr(packet.transport_id, packet.remote_addr.clone());
@@ -77,7 +113,7 @@ impl Node {
peer.link_stats_mut().record_recv(packet.data.len(), packet.timestamp_ms);
peer.touch(packet.timestamp_ms);
// Dispatch to link message handler
self.dispatch_link_message(&node_addr, &plaintext).await;
// Dispatch to link message handler (msg_type + payload, inner header stripped)
self.dispatch_link_message(&node_addr, link_message).await;
}
}
+15 -7
View File
@@ -1,6 +1,6 @@
//! SessionDatagram forwarding handler.
//!
//! Handles incoming SessionDatagram (0x40) link messages: decodes the
//! Handles incoming SessionDatagram (0x00) link messages: decodes the
//! envelope, enforces hop limits, performs coordinate cache warming from
//! plaintext session-layer headers, routes to the next hop or delivers
//! locally, and generates error signals on routing failure.
@@ -16,7 +16,7 @@ use tracing::debug;
impl Node {
/// Handle an incoming SessionDatagram from a peer.
///
/// Called by `dispatch_link_message` for msg_type 0x40. The payload
/// Called by `dispatch_link_message` for msg_type 0x00. The payload
/// has already had its msg_type byte stripped by dispatch.
pub(in crate::node) async fn handle_session_datagram(&mut self, _from: &NodeAddr, payload: &[u8]) {
let mut datagram = match SessionDatagram::decode(payload) {
@@ -27,12 +27,12 @@ impl Node {
}
};
// Hop limit enforcement: decrement and drop if exhausted
if !datagram.decrement_hop_limit() {
// TTL enforcement: decrement and drop if exhausted
if !datagram.decrement_ttl() {
debug!(
src = %datagram.src_addr,
dest = %datagram.dest_addr,
"SessionDatagram hop limit exhausted, dropping"
"SessionDatagram TTL exhausted, dropping"
);
return;
}
@@ -56,7 +56,15 @@ impl Node {
}
};
// Forward: re-encode (includes 0x40 type byte) and send
// Apply path_mtu min() from the outgoing link's transport MTU
if let Some(peer) = self.peers.get(&next_hop_addr)
&& let Some(tid) = peer.transport_id()
&& let Some(transport) = self.transports.get(&tid)
{
datagram.path_mtu = datagram.path_mtu.min(transport.mtu());
}
// Forward: re-encode (includes 0x00 type byte) and send
let encoded = datagram.encode();
if let Err(e) = self
.send_encrypted_link_message(&next_hop_addr, &encoded)
@@ -204,7 +212,7 @@ impl Node {
};
let error_dg = SessionDatagram::new(my_addr, original.src_addr, error_payload)
.with_hop_limit(self.config.node.session.default_hop_limit);
.with_ttl(self.config.node.session.default_ttl);
let next_hop_addr = match self.find_next_hop(&original.src_addr) {
Some(peer) => *peer.node_addr(),
+6 -2
View File
@@ -11,7 +11,7 @@ use std::time::Duration;
use tracing::{debug, info, warn};
impl Node {
/// Handle handshake message 1 (discriminator 0x01).
/// Handle handshake message 1 (phase 0x1).
///
/// This creates a new inbound connection. Rate limiting is applied
/// before any expensive crypto operations.
@@ -222,7 +222,7 @@ impl Node {
self.msg1_rate_limiter.complete_handshake();
}
/// Handle handshake message 2 (discriminator 0x02).
/// Handle handshake message 2 (phase 0x2).
///
/// This completes an outbound handshake we initiated.
pub(in crate::node) async fn handle_msg2(&mut self, packet: ReceivedPacket) {
@@ -549,6 +549,8 @@ impl Node {
transport_id,
current_addr,
link_stats,
is_outbound,
&self.config.node.mmp,
);
new_peer.set_tree_announce_min_interval_ms(self.config.node.tree.announce_min_interval_ms);
@@ -629,6 +631,8 @@ impl Node {
transport_id,
current_addr,
link_stats,
is_outbound,
&self.config.node.mmp,
);
new_peer.set_tree_announce_min_interval_ms(self.config.node.tree.announce_min_interval_ms);
+246
View File
@@ -0,0 +1,246 @@
//! MMP report dispatch, periodic report generation, and operator logging.
//!
//! Handles incoming SenderReport / ReceiverReport messages, drives
//! periodic report generation on the tick timer, and emits periodic
//! and teardown metric logs.
use crate::mmp::MmpMode;
use crate::mmp::report::{ReceiverReport, SenderReport};
use crate::node::Node;
use crate::NodeAddr;
use std::time::Instant;
use tracing::{debug, info, warn};
/// Format bytes/sec as human-readable throughput.
fn format_throughput(bps: f64) -> String {
if bps == 0.0 {
"n/a".to_string()
} else if bps >= 1_000_000.0 {
format!("{:.1}MB/s", bps / 1_000_000.0)
} else if bps >= 1_000.0 {
format!("{:.1}KB/s", bps / 1_000.0)
} else {
format!("{:.0}B/s", bps)
}
}
impl Node {
/// Handle an incoming SenderReport from a peer.
///
/// The peer is telling us about what they sent. We feed this to our
/// receiver state for cross-reference (not currently used for metrics,
/// but stored for future use).
pub(in crate::node) fn handle_sender_report(&mut self, from: &NodeAddr, payload: &[u8]) {
let sr = match SenderReport::decode(payload) {
Ok(sr) => sr,
Err(e) => {
debug!(from = %from, error = %e, "Malformed SenderReport");
return;
}
};
let peer = match self.peers.get_mut(from) {
Some(p) => p,
None => {
debug!(from = %from, "SenderReport from unknown peer");
return;
}
};
if peer.mmp().is_none() {
return;
}
debug!(
from = %from,
cum_pkts = sr.cumulative_packets_sent,
interval_bytes = sr.interval_bytes_sent,
"Received SenderReport"
);
// Store sender's report in receiver state for cross-reference.
// Currently informational; the receiver already tracks its own
// counters and echoes timestamps from data frames.
}
/// Handle an incoming ReceiverReport from a peer.
///
/// The peer is telling us about what they received from us. We feed
/// this to our metrics to compute RTT, loss rate, and trend indicators.
pub(in crate::node) fn handle_receiver_report(&mut self, from: &NodeAddr, payload: &[u8]) {
let rr = match ReceiverReport::decode(payload) {
Ok(rr) => rr,
Err(e) => {
debug!(from = %from, error = %e, "Malformed ReceiverReport");
return;
}
};
let peer = match self.peers.get_mut(from) {
Some(p) => p,
None => {
debug!(from = %from, "ReceiverReport from unknown peer");
return;
}
};
// Get session timestamp before taking mutable borrow on MMP
let our_timestamp_ms = peer.session_elapsed_ms();
let Some(mmp) = peer.mmp_mut() else {
return;
};
// Process the report: computes RTT from timestamp echo, updates
// loss rate, goodput rate, jitter trend, and ETX.
let now = Instant::now();
mmp.metrics.process_receiver_report(&rr, our_timestamp_ms, now);
// Feed SRTT back to sender/receiver report interval tuning
if let Some(srtt_ms) = mmp.metrics.srtt_ms() {
let srtt_us = (srtt_ms * 1000.0) as i64;
mmp.sender.update_report_interval_from_srtt(srtt_us);
mmp.receiver.update_report_interval_from_srtt(srtt_us);
}
// Update reverse delivery ratio from our own receiver state
// (what fraction of peer's frames we received).
let our_recv_packets = mmp.receiver.cumulative_packets_recv();
let peer_highest = mmp.receiver.highest_counter();
if peer_highest > 0 {
let reverse_ratio = (our_recv_packets as f64) / (peer_highest as f64);
mmp.metrics.set_delivery_ratio_reverse(reverse_ratio);
}
debug!(
from = %from,
rtt_ms = ?mmp.metrics.srtt_ms(),
loss = format_args!("{:.1}%", mmp.metrics.loss_rate() * 100.0),
etx = format_args!("{:.2}", mmp.metrics.etx),
"Processed ReceiverReport"
);
}
/// Check all peers for pending MMP reports and send them.
///
/// Called from the tick handler. Also emits periodic operator logs.
pub(in crate::node) async fn check_mmp_reports(&mut self) {
let now = Instant::now();
// Collect peers that need reports (can't borrow self mutably while iterating)
let mut sender_reports: Vec<(NodeAddr, Vec<u8>)> = Vec::new();
let mut receiver_reports: Vec<(NodeAddr, Vec<u8>)> = Vec::new();
for (node_addr, peer) in self.peers.iter_mut() {
let Some(mmp) = peer.mmp_mut() else {
continue;
};
let mode = mmp.mode();
// Sender reports: Full mode only
if mode == MmpMode::Full && mmp.sender.should_send_report(now) {
if let Some(sr) = mmp.sender.build_report(now) {
sender_reports.push((*node_addr, sr.encode()));
}
}
// Receiver reports: Full and Lightweight modes
if mode != MmpMode::Minimal && mmp.receiver.should_send_report(now) {
if let Some(rr) = mmp.receiver.build_report(now) {
receiver_reports.push((*node_addr, rr.encode()));
}
}
// Periodic operator logging
if mmp.should_log(now) {
Self::log_mmp_metrics(node_addr, mmp);
mmp.mark_logged(now);
}
}
// Send collected reports
for (node_addr, encoded) in sender_reports {
if let Err(e) = self.send_encrypted_link_message(&node_addr, &encoded).await {
warn!(peer = %node_addr, error = %e, "Failed to send SenderReport");
}
}
for (node_addr, encoded) in receiver_reports {
if let Err(e) = self.send_encrypted_link_message(&node_addr, &encoded).await {
warn!(peer = %node_addr, error = %e, "Failed to send ReceiverReport");
}
}
}
/// Emit periodic MMP metrics for a peer at info and debug levels.
fn log_mmp_metrics(node_addr: &NodeAddr, mmp: &crate::mmp::MmpPeerState) {
let m = &mmp.metrics;
let rtt_str = match m.srtt_ms() {
Some(rtt) => format!("{:.1}ms", rtt),
None => "n/a".to_string(),
};
let loss_pct = m.loss_rate() * 100.0;
let tx_pkts = mmp.sender.cumulative_packets_sent();
let rx_pkts = mmp.receiver.cumulative_packets_recv();
let goodput_bps = m.goodput_bps();
let goodput_str = format_throughput(goodput_bps);
// Info-level: concise summary
info!(
peer = %node_addr,
rtt = %rtt_str,
loss = format_args!("{:.1}%", loss_pct),
goodput = %goodput_str,
tx_pkts = tx_pkts,
rx_pkts = rx_pkts,
"MMP link metrics"
);
// Debug-level: extended details
debug!(
peer = %node_addr,
jitter_us = mmp.receiver.jitter_us(),
reorder = mmp.receiver.cumulative_packets_recv(),
rtt_trend = format_args!("{}", if m.rtt_trend.initialized() {
format!("short={:.1} long={:.1}", m.rtt_trend.short(), m.rtt_trend.long())
} else {
"n/a".to_string()
}),
loss_trend = format_args!("{}", if m.loss_trend.initialized() {
format!("short={:.4} long={:.4}", m.loss_trend.short(), m.loss_trend.long())
} else {
"n/a".to_string()
}),
delivery_fwd = format_args!("{:.3}", m.delivery_ratio_forward),
delivery_rev = format_args!("{:.3}", m.delivery_ratio_reverse),
mode = %mmp.mode(),
"MMP link metrics (detail)"
);
}
/// Emit a teardown log summarizing lifetime MMP metrics for a removed peer.
pub(in crate::node) fn log_mmp_teardown(node_addr: &NodeAddr, mmp: &crate::mmp::MmpPeerState) {
let m = &mmp.metrics;
let rtt_str = match m.srtt_ms() {
Some(rtt) => format!("{:.1}ms", rtt),
None => "n/a".to_string(),
};
info!(
peer = %node_addr,
rtt = %rtt_str,
loss = format_args!("{:.1}%", m.loss_rate() * 100.0),
etx = format_args!("{:.2}", m.etx),
tx_pkts = mmp.sender.cumulative_packets_sent(),
tx_bytes = mmp.sender.cumulative_bytes_sent(),
rx_pkts = mmp.receiver.cumulative_packets_recv(),
rx_bytes = mmp.receiver.cumulative_bytes_recv(),
jitter_us = mmp.receiver.jitter_us(),
"MMP link teardown"
);
}
}
+1
View File
@@ -5,6 +5,7 @@ mod dispatch;
mod encrypted;
mod forwarding;
mod handshake;
mod mmp;
mod rx_loop;
mod session;
mod timeout;
+29 -16
View File
@@ -2,7 +2,7 @@
use crate::node::{Node, NodeError};
use crate::transport::ReceivedPacket;
use crate::node::wire::{DISCRIMINATOR_ENCRYPTED, DISCRIMINATOR_MSG1, DISCRIMINATOR_MSG2};
use crate::node::wire::{CommonPrefix, PHASE_ESTABLISHED, PHASE_MSG1, PHASE_MSG2, FLP_VERSION, COMMON_PREFIX_SIZE};
use std::time::Duration;
use tracing::{debug, info};
@@ -10,10 +10,10 @@ impl Node {
/// Run the receive event loop.
///
/// Processes packets from all transports, dispatching based on
/// the discriminator byte in the wire protocol:
/// - 0x00: Encrypted frame (session data)
/// - 0x01: Handshake message 1 (initiator -> responder)
/// - 0x02: Handshake message 2 (responder -> initiator)
/// the phase field in the 4-byte common prefix:
/// - Phase 0x0: Encrypted frame (session data)
/// - Phase 0x1: Handshake message 1 (initiator -> responder)
/// - Phase 0x2: Handshake message 2 (responder -> initiator)
///
/// Also processes outbound IPv6 packets from the TUN reader for session
/// encapsulation and routing through the mesh.
@@ -83,6 +83,7 @@ impl Node {
self.process_pending_retries(now_ms).await;
self.check_tree_state().await;
self.check_bloom_state().await;
self.check_mmp_reports().await;
self.purge_stale_lookups(now_ms);
}
}
@@ -94,29 +95,41 @@ impl Node {
/// Process a single received packet.
///
/// Dispatches based on the discriminator byte.
/// Dispatches based on the phase field in the 4-byte common prefix.
async fn process_packet(&mut self, packet: ReceivedPacket) {
if packet.data.is_empty() {
return; // Drop empty packets
if packet.data.len() < COMMON_PREFIX_SIZE {
return; // Drop packets too short for common prefix
}
let discriminator = packet.data[0];
match discriminator {
DISCRIMINATOR_ENCRYPTED => {
let prefix = match CommonPrefix::parse(&packet.data) {
Some(p) => p,
None => return, // Malformed prefix
};
if prefix.version != FLP_VERSION {
debug!(
version = prefix.version,
transport_id = %packet.transport_id,
"Unknown FLP version, dropping"
);
return;
}
match prefix.phase {
PHASE_ESTABLISHED => {
self.handle_encrypted_frame(packet).await;
}
DISCRIMINATOR_MSG1 => {
PHASE_MSG1 => {
self.handle_msg1(packet).await;
}
DISCRIMINATOR_MSG2 => {
PHASE_MSG2 => {
self.handle_msg2(packet).await;
}
_ => {
// Unknown discriminator, drop silently
debug!(
discriminator = discriminator,
phase = prefix.phase,
transport_id = %packet.transport_id,
"Unknown packet discriminator, dropping"
"Unknown FLP phase, dropping"
);
}
}
+3 -3
View File
@@ -144,7 +144,7 @@ impl Node {
let ack = SessionAck::new(our_coords).with_handshake(msg2);
let my_addr = *self.node_addr();
let datagram = SessionDatagram::new(my_addr, *src_addr, ack.encode())
.with_hop_limit(self.config.node.session.default_hop_limit);
.with_ttl(self.config.node.session.default_ttl);
// Route the ack back to the initiator
if let Err(e) = self.send_session_datagram(&datagram).await {
@@ -432,7 +432,7 @@ impl Node {
// Wrap in SessionDatagram
let my_addr = *self.node_addr();
let datagram = SessionDatagram::new(my_addr, dest_addr, setup.encode())
.with_hop_limit(self.config.node.session.default_hop_limit);
.with_ttl(self.config.node.session.default_ttl);
// Route toward destination
self.send_session_datagram(&datagram).await?;
@@ -498,7 +498,7 @@ impl Node {
let my_addr = *self.node_addr();
let datagram = SessionDatagram::new(my_addr, *dest_addr, data_packet.encode())
.with_hop_limit(self.config.node.session.default_hop_limit);
.with_ttl(self.config.node.session.default_ttl);
self.send_session_datagram(&datagram).await?;
+28 -4
View File
@@ -30,7 +30,7 @@ use crate::transport::udp::UdpTransport;
use crate::tree::TreeState;
use crate::upper::icmp_rate_limit::IcmpRateLimiter;
use crate::upper::tun::{TunError, TunOutboundRx, TunState, TunTx};
use self::wire::build_encrypted;
use self::wire::{build_encrypted, build_established_header, prepend_inner_header, FLAG_SP};
use crate::{Config, ConfigError, Identity, IdentityError, NodeAddr};
use std::collections::{HashMap, VecDeque};
use std::fmt;
@@ -1026,6 +1026,10 @@ impl Node {
/// The plaintext should include the message type byte followed by the
/// message-specific payload (e.g., `[0x50, reason]` for Disconnect).
///
/// The send path prepends a 4-byte session-relative timestamp (inner
/// header) before encryption. The full 16-byte outer header is used
/// as AAD for the AEAD construction.
///
/// This is the standard path for sending any link-layer control message
/// to a peer over their encrypted Noise session.
pub(super) async fn send_encrypted_link_message(
@@ -1049,19 +1053,35 @@ impl Node {
reason: "no current_addr".into(),
})?;
// Prepend 4-byte session-relative timestamp (inner header)
let timestamp_ms = peer.session_elapsed_ms();
// MMP: read spin bit value before entering session borrow
let sp_flag = peer.mmp()
.map(|mmp| mmp.spin_bit.tx_bit())
.unwrap_or(false);
let flags = if sp_flag { FLAG_SP } else { 0 };
let session = peer.noise_session_mut().ok_or_else(|| NodeError::SendFailed {
node_addr: *node_addr,
reason: "no noise session".into(),
})?;
// Get counter before encrypt (encrypt increments it)
// Inner plaintext: [timestamp:4 LE][msg_type][payload...]
let inner_plaintext = prepend_inner_header(timestamp_ms, plaintext);
// Build 16-byte outer header (used as AAD for AEAD)
let counter = session.current_send_counter();
let ciphertext = session.encrypt(plaintext).map_err(|e| NodeError::SendFailed {
let payload_len = inner_plaintext.len() as u16;
let header = build_established_header(their_index, counter, flags, payload_len);
// Encrypt with AAD binding to the outer header
let ciphertext = session.encrypt_with_aad(&inner_plaintext, &header).map_err(|e| NodeError::SendFailed {
node_addr: *node_addr,
reason: format!("encryption failed: {}", e),
})?;
let wire_packet = build_encrypted(their_index, counter, &ciphertext);
let wire_packet = build_encrypted(&header, &ciphertext);
// Re-borrow peer for stats update after sending
let transport = self.transports.get(&transport_id)
@@ -1076,6 +1096,10 @@ impl Node {
// Update send statistics
if let Some(peer) = self.peers.get_mut(node_addr) {
peer.link_stats_mut().record_sent(bytes_sent);
// MMP: record sent frame for sender report generation
if let Some(mmp) = peer.mmp_mut() {
mmp.sender.record_sent(counter, timestamp_ms, bytes_sent);
}
}
Ok(())
+11 -11
View File
@@ -1,7 +1,7 @@
//! SessionDatagram forwarding tests.
//!
//! Tests for the handle_session_datagram handler including decode errors,
//! hop limit enforcement, local delivery, coordinate cache warming, and
//! TTL enforcement, local delivery, coordinate cache warming, and
//! multi-hop forwarding through live node topologies.
use super::*;
@@ -26,7 +26,7 @@ async fn test_forwarding_decode_error() {
node.handle_session_datagram(&from, &[0x00; 5]).await;
}
// --- Hop limit ---
// --- TTL ---
#[tokio::test]
async fn test_forwarding_hop_limit_exhausted() {
@@ -35,7 +35,7 @@ async fn test_forwarding_hop_limit_exhausted() {
let src = make_node_addr(0x01);
let dest = make_node_addr(0x02);
let dg = SessionDatagram::new(src, dest, vec![0x10, 0x00, 0x00, 0x00])
.with_hop_limit(0);
.with_ttl(0);
let encoded = dg.encode();
// Dispatch with payload after msg_type byte
node.handle_session_datagram(&from, &encoded[1..]).await;
@@ -44,17 +44,17 @@ async fn test_forwarding_hop_limit_exhausted() {
#[tokio::test]
async fn test_forwarding_hop_limit_one_drops_at_transit() {
// hop_limit=1 means after decrement it becomes 0 — the datagram can
// ttl=1 means after decrement it becomes 0 — the datagram can
// still be delivered this hop but would be dropped at the next.
// decrement_hop_limit returns true (1 > 0), so the handler proceeds.
// decrement_ttl returns true (1 > 0), so the handler proceeds.
let mut node = make_node();
let from = make_node_addr(0xAA);
let my_addr = *node.node_addr();
let src = make_node_addr(0x01);
let dg = SessionDatagram::new(src, my_addr, vec![0x10, 0x00, 0x00, 0x00])
.with_hop_limit(1);
.with_ttl(1);
let encoded = dg.encode();
// Should succeed — hop_limit=1 decrements to 0 but packet is still processed
// Should succeed — ttl=1 decrements to 0 but packet is still processed
node.handle_session_datagram(&from, &encoded[1..]).await;
}
@@ -343,7 +343,7 @@ async fn test_forwarding_multi_hop() {
let node1_addr = *nodes[1].node.node_addr();
let node4_addr = *nodes[4].node.node_addr();
// Build a SessionDatagram with enough hop_limit for 4 hops
// Build a SessionDatagram with enough TTL for 4 hops
let dg = SessionDatagram::new(
node0_addr,
node4_addr,
@@ -372,9 +372,9 @@ async fn test_forwarding_multi_hop() {
#[tokio::test]
async fn test_forwarding_hop_limit_prevents_infinite_loops() {
// 3-node chain: 0 -- 1 -- 2
// Send a datagram with hop_limit=1. It should be forwarded by node 1
// Send a datagram with ttl=1. It should be forwarded by node 1
// (decrement to 0) and delivered at node 2 (local delivery). If node 2
// tried to forward further, the 0 hop_limit would prevent it.
// tried to forward further, the 0 ttl would prevent it.
let edges = vec![(0, 1), (1, 2)];
let mut nodes = run_tree_test(3, &edges, false).await;
verify_tree_convergence(&nodes);
@@ -389,7 +389,7 @@ async fn test_forwarding_hop_limit_prevents_infinite_loops() {
node2_addr,
vec![0x10, 0x00, 0x04, 0x00, 1, 2, 3, 4],
)
.with_hop_limit(2); // Enough for 01 (decrement to 1) and 12 (decrement to 0, local delivery)
.with_ttl(2); // Enough for 0->1 (decrement to 1) and 1->2 (decrement to 0, local delivery)
let encoded = dg.encode();
+16 -8
View File
@@ -6,7 +6,7 @@ use super::*;
async fn test_two_node_handshake_udp() {
use crate::config::UdpConfig;
use crate::transport::udp::UdpTransport;
use crate::node::wire::{build_encrypted, build_msg1};
use crate::node::wire::{build_encrypted, build_established_header, build_msg1, prepend_inner_header};
use tokio::time::{timeout, Duration};
// === Setup: Two nodes with UDP transports on localhost ===
@@ -156,13 +156,17 @@ async fn test_two_node_handshake_udp() {
// === Phase 4: Encrypted frame A → B ===
// A encrypts a test message and sends to B
let plaintext_a = b"hello from A";
// Prepend inner header (timestamp + msg_type) as the real send path does
let msg_a = b"\x10test from A"; // msg_type 0x10 (TreeAnnounce) + dummy payload
let inner_a = prepend_inner_header(0, msg_a);
let peer_b = node_a.get_peer_mut(&peer_b_node_addr).unwrap();
let their_index_b = peer_b.their_index().expect("A should know B's index");
let session_a = peer_b.noise_session_mut().unwrap();
let ciphertext_a = session_a.encrypt(plaintext_a).unwrap();
let counter_a = session_a.current_send_counter();
let header_a = build_established_header(their_index_b, counter_a, 0, inner_a.len() as u16);
let ciphertext_a = session_a.encrypt_with_aad(&inner_a, &header_a).unwrap();
let wire_encrypted = build_encrypted(their_index_b, 0, &ciphertext_a);
let wire_encrypted = build_encrypted(&header_a, &ciphertext_a);
let transport = node_a.transports.get(&transport_id_a).unwrap();
transport
.send(&remote_addr_b, &wire_encrypted)
@@ -186,13 +190,17 @@ async fn test_two_node_handshake_udp() {
// === Phase 5: Encrypted frame B → A ===
let plaintext_b = b"hello from B";
// Prepend inner header (timestamp + msg_type) as the real send path does
let msg_b = b"\x10test from B"; // msg_type 0x10 (TreeAnnounce) + dummy payload
let inner_b = prepend_inner_header(0, msg_b);
let peer_a = node_b.get_peer_mut(&peer_a_node_addr).unwrap();
let their_index_a = peer_a.their_index().expect("B should know A's index");
let session_b = peer_a.noise_session_mut().unwrap();
let ciphertext_b = session_b.encrypt(plaintext_b).unwrap();
let counter_b = session_b.current_send_counter();
let header_b = build_established_header(their_index_a, counter_b, 0, inner_b.len() as u16);
let ciphertext_b = session_b.encrypt_with_aad(&inner_b, &header_b).unwrap();
let wire_encrypted_b = build_encrypted(their_index_a, 0, &ciphertext_b);
let wire_encrypted_b = build_encrypted(&header_b, &ciphertext_b);
let transport = node_b.transports.get(&transport_id_b).unwrap();
transport
.send(&remote_addr_a, &wire_encrypted_b)
@@ -328,7 +336,7 @@ async fn test_run_rx_loop_handshake() {
//
// This is the key difference from test_two_node_handshake_udp:
// instead of calling handle_msg1() directly, we run the full rx loop
// which dispatches based on the discriminator byte.
// which dispatches based on the common prefix phase field.
tokio::select! {
result = node_b.run_rx_loop() => {
+14 -9
View File
@@ -203,23 +203,28 @@ pub(super) fn print_tree_snapshot(label: &str, nodes: &[TestNode]) {
///
/// Returns the number of packets processed.
pub(super) async fn process_available_packets(nodes: &mut [TestNode]) -> usize {
use crate::node::wire::{DISCRIMINATOR_ENCRYPTED, DISCRIMINATOR_MSG1, DISCRIMINATOR_MSG2};
use crate::node::wire::{CommonPrefix, FLP_VERSION, PHASE_ESTABLISHED, PHASE_MSG1, PHASE_MSG2, COMMON_PREFIX_SIZE};
let mut count = 0;
for node in nodes.iter_mut() {
while let Ok(packet) = node.packet_rx.try_recv() {
if packet.data.is_empty() {
if packet.data.len() < COMMON_PREFIX_SIZE {
continue;
}
match packet.data[0] {
DISCRIMINATOR_MSG1 => node.node.handle_msg1(packet).await,
DISCRIMINATOR_MSG2 => node.node.handle_msg2(packet).await,
DISCRIMINATOR_ENCRYPTED => {
node.node.handle_encrypted_frame(packet).await
if let Some(prefix) = CommonPrefix::parse(&packet.data) {
if prefix.version != FLP_VERSION {
continue;
}
_ => {}
match prefix.phase {
PHASE_MSG1 => node.node.handle_msg1(packet).await,
PHASE_MSG2 => node.node.handle_msg2(packet).await,
PHASE_ESTABLISHED => {
node.node.handle_encrypted_frame(packet).await
}
_ => {}
}
count += 1;
}
count += 1;
}
}
count
+361 -93
View File
@@ -1,15 +1,21 @@
//! Wire Format Parsing and Serialization
//!
//! Defines the FIPS link-layer wire format for packet dispatch.
//! All packets begin with a discriminator byte followed by type-specific payload.
//! Defines the FIPS link-layer wire format (FLP) for packet dispatch.
//! All packets begin with a 4-byte common prefix followed by phase-specific fields.
//!
//! ## Common Prefix (4 bytes)
//!
//! ```text
//! [ver+phase:1][flags:1][payload_len:2 LE]
//! ```
//!
//! ## Packet Types
//!
//! | Byte | Type | Size | Description |
//! |------|-----------------|-----------|--------------------------------|
//! | 0x00 | Encrypted frame | 29+ bytes | Post-handshake encrypted data |
//! | 0x01 | Noise IK msg1 | 87 bytes | Handshake initiation |
//! | 0x02 | Noise IK msg2 | 42 bytes | Handshake response |
//! | Phase | Type | Size | Description |
//! |-------|-----------------|-----------|--------------------------------|
//! | 0x0 | Encrypted frame | 32+ bytes | Post-handshake encrypted data |
//! | 0x1 | Noise IK msg1 | 90 bytes | Handshake initiation |
//! | 0x2 | Noise IK msg2 | 45 bytes | Handshake response |
use crate::utils::index::SessionIndex;
use crate::noise::{HANDSHAKE_MSG1_SIZE, HANDSHAKE_MSG2_SIZE, TAG_SIZE};
@@ -18,73 +24,171 @@ use crate::noise::{HANDSHAKE_MSG1_SIZE, HANDSHAKE_MSG2_SIZE, TAG_SIZE};
// Constants
// ============================================================================
/// Discriminator for encrypted frames (post-handshake data).
pub const DISCRIMINATOR_ENCRYPTED: u8 = 0x00;
/// FLP protocol version (4 high bits of byte 0).
pub const FLP_VERSION: u8 = 0;
/// Discriminator for Noise IK message 1 (handshake initiation).
pub const DISCRIMINATOR_MSG1: u8 = 0x01;
/// Phase value for established (encrypted) frames.
pub const PHASE_ESTABLISHED: u8 = 0x0;
/// Discriminator for Noise IK message 2 (handshake response).
pub const DISCRIMINATOR_MSG2: u8 = 0x02;
/// Phase value for Noise IK message 1 (handshake initiation).
pub const PHASE_MSG1: u8 = 0x1;
/// Size of Noise IK message 1 wire packet: discriminator + sender_idx + noise_msg1.
pub const MSG1_WIRE_SIZE: usize = 1 + 4 + HANDSHAKE_MSG1_SIZE; // 87 bytes
/// Phase value for Noise IK message 2 (handshake response).
pub const PHASE_MSG2: u8 = 0x2;
/// Size of Noise IK message 2 wire packet: discriminator + sender_idx + receiver_idx + noise_msg2.
pub const MSG2_WIRE_SIZE: usize = 1 + 4 + 4 + HANDSHAKE_MSG2_SIZE; // 42 bytes
/// Size of the common packet prefix (all packet types).
pub const COMMON_PREFIX_SIZE: usize = 4;
/// Minimum size for encrypted frame: discriminator + receiver_idx + counter + tag.
pub const ENCRYPTED_MIN_SIZE: usize = 1 + 4 + 8 + TAG_SIZE; // 29 bytes
/// Size of the full established frame header (prefix + receiver_idx + counter).
pub const ESTABLISHED_HEADER_SIZE: usize = 16;
/// Size of Noise IK message 1 wire packet: prefix + sender_idx + noise_msg1.
pub const MSG1_WIRE_SIZE: usize = COMMON_PREFIX_SIZE + 4 + HANDSHAKE_MSG1_SIZE; // 90 bytes
/// Size of Noise IK message 2 wire packet: prefix + sender_idx + receiver_idx + noise_msg2.
pub const MSG2_WIRE_SIZE: usize = COMMON_PREFIX_SIZE + 4 + 4 + HANDSHAKE_MSG2_SIZE; // 45 bytes
/// Minimum size for encrypted frame: header + tag (no plaintext).
pub const ENCRYPTED_MIN_SIZE: usize = ESTABLISHED_HEADER_SIZE + TAG_SIZE; // 32 bytes
/// Size of the encrypted inner header (timestamp + message type).
pub const INNER_HEADER_SIZE: usize = 5;
// Flag bit constants (byte 1 of common prefix, meaningful only for phase 0x0).
// Reserved for upcoming rekeying, congestion signaling, and RTT measurement.
#[allow(dead_code)]
/// Key epoch flag — selects active key during rekeying.
pub const FLAG_KEY_EPOCH: u8 = 0x01;
#[allow(dead_code)]
/// Congestion Experienced echo flag.
pub const FLAG_CE: u8 = 0x02;
#[allow(dead_code)]
/// Spin bit for RTT measurement.
pub const FLAG_SP: u8 = 0x04;
// ============================================================================
// Common Prefix
// ============================================================================
/// Parsed common packet prefix (first 4 bytes of every FLP packet).
///
/// Wire format:
/// ```text
/// [ver(4bits)+phase(4bits)][flags:1][payload_len:2 LE]
/// ```
#[derive(Clone, Debug)]
pub struct CommonPrefix {
/// Protocol version (high nibble of byte 0).
pub version: u8,
/// Session lifecycle phase (low nibble of byte 0).
pub phase: u8,
/// Per-packet signal flags (meaningful only for phase 0x0).
#[allow(dead_code)]
pub flags: u8,
/// Length of payload following the phase-specific header (excludes AEAD tag).
#[allow(dead_code)]
pub payload_len: u16,
}
impl CommonPrefix {
/// Parse a common prefix from the first 4 bytes of packet data.
pub fn parse(data: &[u8]) -> Option<Self> {
if data.len() < COMMON_PREFIX_SIZE {
return None;
}
let version = data[0] >> 4;
let phase = data[0] & 0x0F;
let flags = data[1];
let payload_len = u16::from_le_bytes([data[2], data[3]]);
Some(Self {
version,
phase,
flags,
payload_len,
})
}
/// Encode the ver+phase byte.
fn ver_phase_byte(version: u8, phase: u8) -> u8 {
(version << 4) | (phase & 0x0F)
}
}
// ============================================================================
// Encrypted Frame Header
// ============================================================================
/// Parsed encrypted frame header.
/// Parsed established frame header (phase 0x0).
///
/// Wire format:
/// Wire format (16 bytes):
/// ```text
/// [0x00][receiver_idx:4 LE][counter:8 LE][ciphertext+tag]
/// [ver+phase:1][flags:1][payload_len:2 LE][receiver_idx:4 LE][counter:8 LE]
/// ```
///
/// The full 16-byte header is used as AAD for the AEAD construction.
#[derive(Clone, Debug)]
pub struct EncryptedHeader {
/// Per-packet flags (K, CE, SP).
#[allow(dead_code)]
pub flags: u8,
/// Length of encrypted payload (excluding AEAD tag).
#[allow(dead_code)]
pub payload_len: u16,
/// Session index chosen by the receiver (for O(1) lookup).
pub receiver_idx: SessionIndex,
/// Monotonic counter used as AEAD nonce.
pub counter: u64,
/// Offset where ciphertext begins in the original packet.
pub ciphertext_offset: usize,
/// Raw 16-byte header for use as AEAD AAD.
pub header_bytes: [u8; ESTABLISHED_HEADER_SIZE],
}
impl EncryptedHeader {
/// Parse an encrypted frame header from packet data.
/// Parse an established frame header from packet data.
///
/// Returns None if the packet is too short or has wrong discriminator.
/// Returns None if the packet is too short or has wrong version/phase.
pub fn parse(data: &[u8]) -> Option<Self> {
if data.len() < ENCRYPTED_MIN_SIZE {
return None;
}
if data[0] != DISCRIMINATOR_ENCRYPTED {
let version = data[0] >> 4;
let phase = data[0] & 0x0F;
if version != FLP_VERSION || phase != PHASE_ESTABLISHED {
return None;
}
let receiver_idx = SessionIndex::from_le_bytes([data[1], data[2], data[3], data[4]]);
let flags = data[1];
let payload_len = u16::from_le_bytes([data[2], data[3]]);
let receiver_idx = SessionIndex::from_le_bytes([data[4], data[5], data[6], data[7]]);
let counter = u64::from_le_bytes([
data[5], data[6], data[7], data[8], data[9], data[10], data[11], data[12],
data[8], data[9], data[10], data[11],
data[12], data[13], data[14], data[15],
]);
let mut header_bytes = [0u8; ESTABLISHED_HEADER_SIZE];
header_bytes.copy_from_slice(&data[..ESTABLISHED_HEADER_SIZE]);
Some(Self {
flags,
payload_len,
receiver_idx,
counter,
ciphertext_offset: 13,
header_bytes,
})
}
/// Offset where ciphertext begins in the original packet.
pub fn ciphertext_offset(&self) -> usize {
ESTABLISHED_HEADER_SIZE
}
/// Get the ciphertext slice from the original packet.
#[cfg(test)]
pub fn ciphertext<'a>(&self, data: &'a [u8]) -> &'a [u8] {
&data[self.ciphertext_offset..]
&data[ESTABLISHED_HEADER_SIZE..]
}
}
@@ -92,11 +196,11 @@ impl EncryptedHeader {
// Msg1 Header
// ============================================================================
/// Parsed Noise IK message 1 header.
/// Parsed Noise IK message 1 header (phase 0x1).
///
/// Wire format:
/// Wire format (90 bytes):
/// ```text
/// [0x01][sender_idx:4 LE][noise_msg1:82]
/// [0x01][0x00][payload_len:2 LE][sender_idx:4 LE][noise_msg1:82]
/// ```
#[derive(Clone, Debug)]
pub struct Msg1Header {
@@ -109,21 +213,29 @@ pub struct Msg1Header {
impl Msg1Header {
/// Parse a msg1 header from packet data.
///
/// Returns None if the packet has wrong size or discriminator.
/// Returns None if the packet has wrong size or version/phase.
pub fn parse(data: &[u8]) -> Option<Self> {
if data.len() != MSG1_WIRE_SIZE {
return None;
}
if data[0] != DISCRIMINATOR_MSG1 {
let version = data[0] >> 4;
let phase = data[0] & 0x0F;
if version != FLP_VERSION || phase != PHASE_MSG1 {
return None;
}
let sender_idx = SessionIndex::from_le_bytes([data[1], data[2], data[3], data[4]]);
// flags must be zero during handshake
if data[1] != 0 {
return None;
}
let sender_idx = SessionIndex::from_le_bytes([data[4], data[5], data[6], data[7]]);
Some(Self {
sender_idx,
noise_msg1_offset: 5,
noise_msg1_offset: COMMON_PREFIX_SIZE + 4, // 8
})
}
@@ -138,11 +250,11 @@ impl Msg1Header {
// Msg2 Header
// ============================================================================
/// Parsed Noise IK message 2 header.
/// Parsed Noise IK message 2 header (phase 0x2).
///
/// Wire format:
/// Wire format (45 bytes):
/// ```text
/// [0x02][sender_idx:4 LE][receiver_idx:4 LE][noise_msg2:33]
/// [0x02][0x00][payload_len:2 LE][sender_idx:4 LE][receiver_idx:4 LE][noise_msg2:33]
/// ```
#[derive(Clone, Debug)]
pub struct Msg2Header {
@@ -157,23 +269,31 @@ pub struct Msg2Header {
impl Msg2Header {
/// Parse a msg2 header from packet data.
///
/// Returns None if the packet has wrong size or discriminator.
/// Returns None if the packet has wrong size or version/phase.
pub fn parse(data: &[u8]) -> Option<Self> {
if data.len() != MSG2_WIRE_SIZE {
return None;
}
if data[0] != DISCRIMINATOR_MSG2 {
let version = data[0] >> 4;
let phase = data[0] & 0x0F;
if version != FLP_VERSION || phase != PHASE_MSG2 {
return None;
}
let sender_idx = SessionIndex::from_le_bytes([data[1], data[2], data[3], data[4]]);
let receiver_idx = SessionIndex::from_le_bytes([data[5], data[6], data[7], data[8]]);
// flags must be zero during handshake
if data[1] != 0 {
return None;
}
let sender_idx = SessionIndex::from_le_bytes([data[4], data[5], data[6], data[7]]);
let receiver_idx = SessionIndex::from_le_bytes([data[8], data[9], data[10], data[11]]);
Some(Self {
sender_idx,
receiver_idx,
noise_msg2_offset: 9,
noise_msg2_offset: COMMON_PREFIX_SIZE + 4 + 4, // 12
})
}
@@ -190,12 +310,16 @@ impl Msg2Header {
/// Build a wire-format msg1 packet.
///
/// Format: `[0x01][sender_idx:4 LE][noise_msg1:82]`
/// Format: `[0x01][0x00][payload_len:2 LE][sender_idx:4 LE][noise_msg1:82]`
pub fn build_msg1(sender_idx: SessionIndex, noise_msg1: &[u8]) -> Vec<u8> {
debug_assert_eq!(noise_msg1.len(), HANDSHAKE_MSG1_SIZE);
let payload_len = (4 + noise_msg1.len()) as u16; // sender_idx + noise_msg1
let mut packet = Vec::with_capacity(MSG1_WIRE_SIZE);
packet.push(DISCRIMINATOR_MSG1);
packet.push(CommonPrefix::ver_phase_byte(FLP_VERSION, PHASE_MSG1));
packet.push(0x00); // flags must be zero
packet.extend_from_slice(&payload_len.to_le_bytes());
packet.extend_from_slice(&sender_idx.to_le_bytes());
packet.extend_from_slice(noise_msg1);
packet
@@ -203,30 +327,80 @@ pub fn build_msg1(sender_idx: SessionIndex, noise_msg1: &[u8]) -> Vec<u8> {
/// Build a wire-format msg2 packet.
///
/// Format: `[0x02][sender_idx:4 LE][receiver_idx:4 LE][noise_msg2:33]`
/// Format: `[0x02][0x00][payload_len:2 LE][sender_idx:4 LE][receiver_idx:4 LE][noise_msg2:33]`
pub fn build_msg2(sender_idx: SessionIndex, receiver_idx: SessionIndex, noise_msg2: &[u8]) -> Vec<u8> {
debug_assert_eq!(noise_msg2.len(), HANDSHAKE_MSG2_SIZE);
let payload_len = (4 + 4 + noise_msg2.len()) as u16; // sender + receiver + noise
let mut packet = Vec::with_capacity(MSG2_WIRE_SIZE);
packet.push(DISCRIMINATOR_MSG2);
packet.push(CommonPrefix::ver_phase_byte(FLP_VERSION, PHASE_MSG2));
packet.push(0x00); // flags must be zero
packet.extend_from_slice(&payload_len.to_le_bytes());
packet.extend_from_slice(&sender_idx.to_le_bytes());
packet.extend_from_slice(&receiver_idx.to_le_bytes());
packet.extend_from_slice(noise_msg2);
packet
}
/// Build the 16-byte outer header for an established frame.
///
/// Returns the header bytes (for use as AAD) separately from the construction.
pub fn build_established_header(
receiver_idx: SessionIndex,
counter: u64,
flags: u8,
payload_len: u16,
) -> [u8; ESTABLISHED_HEADER_SIZE] {
let mut header = [0u8; ESTABLISHED_HEADER_SIZE];
header[0] = CommonPrefix::ver_phase_byte(FLP_VERSION, PHASE_ESTABLISHED);
header[1] = flags;
header[2..4].copy_from_slice(&payload_len.to_le_bytes());
header[4..8].copy_from_slice(&receiver_idx.to_le_bytes());
header[8..16].copy_from_slice(&counter.to_le_bytes());
header
}
/// Build a wire-format encrypted frame.
///
/// Format: `[0x00][receiver_idx:4 LE][counter:8 LE][ciphertext+tag]`
pub fn build_encrypted(receiver_idx: SessionIndex, counter: u64, ciphertext: &[u8]) -> Vec<u8> {
let mut packet = Vec::with_capacity(13 + ciphertext.len());
packet.push(DISCRIMINATOR_ENCRYPTED);
packet.extend_from_slice(&receiver_idx.to_le_bytes());
packet.extend_from_slice(&counter.to_le_bytes());
/// Format: `[header:16][ciphertext+tag]`
///
/// The header is constructed from the parameters and used as AAD during
/// encryption. The caller should use `build_established_header` to construct
/// the header, encrypt with it as AAD, then call this to assemble the packet.
pub fn build_encrypted(header: &[u8; ESTABLISHED_HEADER_SIZE], ciphertext: &[u8]) -> Vec<u8> {
let mut packet = Vec::with_capacity(ESTABLISHED_HEADER_SIZE + ciphertext.len());
packet.extend_from_slice(header);
packet.extend_from_slice(ciphertext);
packet
}
// ============================================================================
// Inner Header Helpers
// ============================================================================
/// Prepend the 5-byte inner header (timestamp + msg_type) to a link message.
///
/// The caller provides the original plaintext starting with `[msg_type][payload...]`.
/// This prepends `[timestamp:4 LE]` before the msg_type byte.
pub fn prepend_inner_header(timestamp_ms: u32, plaintext: &[u8]) -> Vec<u8> {
let mut buf = Vec::with_capacity(4 + plaintext.len());
buf.extend_from_slice(&timestamp_ms.to_le_bytes());
buf.extend_from_slice(plaintext);
buf
}
/// Strip the 4-byte timestamp from a decrypted inner payload.
///
/// Returns `(timestamp, &payload_starting_at_msg_type)` or None if too short.
pub fn strip_inner_header(plaintext: &[u8]) -> Option<(u32, &[u8])> {
if plaintext.len() < INNER_HEADER_SIZE {
return None;
}
let timestamp = u32::from_le_bytes([plaintext[0], plaintext[1], plaintext[2], plaintext[3]]);
Some((timestamp, &plaintext[4..]))
}
// ============================================================================
// Tests
// ============================================================================
@@ -235,36 +409,61 @@ pub fn build_encrypted(receiver_idx: SessionIndex, counter: u64, ciphertext: &[u
mod tests {
use super::*;
#[test]
fn test_common_prefix_parse() {
let data = [0x00, 0x04, 0x20, 0x00]; // ver=0, phase=0, flags=SP, payload_len=32
let prefix = CommonPrefix::parse(&data).unwrap();
assert_eq!(prefix.version, 0);
assert_eq!(prefix.phase, 0);
assert_eq!(prefix.flags, FLAG_SP);
assert_eq!(prefix.payload_len, 32);
}
#[test]
fn test_common_prefix_too_short() {
assert!(CommonPrefix::parse(&[0, 0, 0]).is_none());
}
#[test]
fn test_encrypted_header_parse() {
// Build a valid encrypted frame
let receiver_idx = SessionIndex::new(0x12345678);
let counter = 42u64;
let ciphertext = vec![0xaa; 32]; // 16 plaintext + 16 tag
let flags = 0u8;
let payload_len = 32u16; // 16 plaintext + 16 tag
let ciphertext = vec![0xaa; 48]; // payload_len + TAG_SIZE
let packet = build_encrypted(receiver_idx, counter, &ciphertext);
let header = build_established_header(receiver_idx, counter, flags, payload_len);
let packet = build_encrypted(&header, &ciphertext);
assert_eq!(packet.len(), 13 + 32);
assert_eq!(packet[0], DISCRIMINATOR_ENCRYPTED);
assert_eq!(packet.len(), ESTABLISHED_HEADER_SIZE + 48);
assert_eq!(packet[0], 0x00); // ver=0, phase=0
// Parse it back
let header = EncryptedHeader::parse(&packet).expect("should parse");
assert_eq!(header.receiver_idx, receiver_idx);
assert_eq!(header.counter, 42);
assert_eq!(header.ciphertext_offset, 13);
assert_eq!(header.ciphertext(&packet), &ciphertext[..]);
let parsed = EncryptedHeader::parse(&packet).expect("should parse");
assert_eq!(parsed.receiver_idx, receiver_idx);
assert_eq!(parsed.counter, 42);
assert_eq!(parsed.flags, 0);
assert_eq!(parsed.payload_len, 32);
assert_eq!(parsed.header_bytes, header);
assert_eq!(parsed.ciphertext(&packet), &ciphertext[..]);
}
#[test]
fn test_encrypted_header_too_short() {
let packet = vec![0x00; 28]; // One byte too short
let packet = vec![0x00; ENCRYPTED_MIN_SIZE - 1];
assert!(EncryptedHeader::parse(&packet).is_none());
}
#[test]
fn test_encrypted_header_wrong_discriminator() {
let mut packet = vec![0x00; 30];
packet[0] = 0x01; // Wrong discriminator
fn test_encrypted_header_wrong_phase() {
let mut packet = vec![0x00; ENCRYPTED_MIN_SIZE];
packet[0] = 0x01; // phase 1 (msg1), not established
assert!(EncryptedHeader::parse(&packet).is_none());
}
#[test]
fn test_encrypted_header_wrong_version() {
let mut packet = vec![0x00; ENCRYPTED_MIN_SIZE];
packet[0] = 0x10; // version 1, phase 0
assert!(EncryptedHeader::parse(&packet).is_none());
}
@@ -276,27 +475,34 @@ mod tests {
let packet = build_msg1(sender_idx, &noise_msg1);
assert_eq!(packet.len(), MSG1_WIRE_SIZE);
assert_eq!(packet[0], DISCRIMINATOR_MSG1);
assert_eq!(packet[0], 0x01); // ver=0, phase=1
let header = Msg1Header::parse(&packet).expect("should parse");
assert_eq!(header.sender_idx, sender_idx);
assert_eq!(header.noise_msg1_offset, 5);
assert_eq!(header.noise_msg1_offset, 8);
assert_eq!(header.noise_msg1(&packet), &noise_msg1[..]);
}
#[test]
fn test_msg1_header_wrong_size() {
let packet = vec![0x01; 86]; // One byte too short
let packet = vec![0x01; MSG1_WIRE_SIZE - 1];
assert!(Msg1Header::parse(&packet).is_none());
let packet = vec![0x01; 88]; // One byte too long
let packet = vec![0x01; MSG1_WIRE_SIZE + 1];
assert!(Msg1Header::parse(&packet).is_none());
}
#[test]
fn test_msg1_header_wrong_discriminator() {
fn test_msg1_header_wrong_phase() {
let mut packet = vec![0x00; MSG1_WIRE_SIZE];
packet[0] = 0x02; // Wrong discriminator
packet[0] = 0x02; // phase 2, not phase 1
assert!(Msg1Header::parse(&packet).is_none());
}
#[test]
fn test_msg1_header_nonzero_flags() {
let mut packet = build_msg1(SessionIndex::new(1), &[0u8; HANDSHAKE_MSG1_SIZE]);
packet[1] = 0x01; // flags must be zero during handshake
assert!(Msg1Header::parse(&packet).is_none());
}
@@ -309,52 +515,114 @@ mod tests {
let packet = build_msg2(sender_idx, receiver_idx, &noise_msg2);
assert_eq!(packet.len(), MSG2_WIRE_SIZE);
assert_eq!(packet[0], DISCRIMINATOR_MSG2);
assert_eq!(packet[0], 0x02); // ver=0, phase=2
let header = Msg2Header::parse(&packet).expect("should parse");
assert_eq!(header.sender_idx, sender_idx);
assert_eq!(header.receiver_idx, receiver_idx);
assert_eq!(header.noise_msg2_offset, 9);
assert_eq!(header.noise_msg2_offset, 12);
assert_eq!(header.noise_msg2(&packet), &noise_msg2[..]);
}
#[test]
fn test_msg2_header_wrong_size() {
let packet = vec![0x02; 41]; // One byte too short
let packet = vec![0x02; MSG2_WIRE_SIZE - 1];
assert!(Msg2Header::parse(&packet).is_none());
let packet = vec![0x02; 43]; // One byte too long
let packet = vec![0x02; MSG2_WIRE_SIZE + 1];
assert!(Msg2Header::parse(&packet).is_none());
}
#[test]
fn test_msg2_header_wrong_discriminator() {
fn test_msg2_header_wrong_phase() {
let mut packet = vec![0x00; MSG2_WIRE_SIZE];
packet[0] = 0x00; // Wrong discriminator
packet[0] = 0x00; // phase 0, not phase 2
assert!(Msg2Header::parse(&packet).is_none());
}
#[test]
fn test_wire_sizes() {
// Verify constants match spec
assert_eq!(MSG1_WIRE_SIZE, 87); // 1 + 4 + 82
assert_eq!(MSG2_WIRE_SIZE, 42); // 1 + 4 + 4 + 33
assert_eq!(ENCRYPTED_MIN_SIZE, 29); // 1 + 4 + 8 + 16
assert_eq!(MSG1_WIRE_SIZE, 90); // 4 + 4 + 82
assert_eq!(MSG2_WIRE_SIZE, 45); // 4 + 4 + 4 + 33
assert_eq!(ENCRYPTED_MIN_SIZE, 32); // 16 + 16
assert_eq!(COMMON_PREFIX_SIZE, 4);
assert_eq!(ESTABLISHED_HEADER_SIZE, 16);
assert_eq!(INNER_HEADER_SIZE, 5);
}
#[test]
fn test_roundtrip_indices() {
// Test that indices survive the roundtrip correctly (endianness)
let idx = SessionIndex::new(0xDEADBEEF);
let msg1 = build_msg1(idx, &[0u8; HANDSHAKE_MSG1_SIZE]);
let parsed = Msg1Header::parse(&msg1).unwrap();
assert_eq!(parsed.sender_idx.as_u32(), 0xDEADBEEF);
// Verify little-endian encoding
assert_eq!(msg1[1], 0xEF);
assert_eq!(msg1[2], 0xBE);
assert_eq!(msg1[3], 0xAD);
assert_eq!(msg1[4], 0xDE);
// Verify little-endian encoding (sender_idx starts at offset 4)
assert_eq!(msg1[4], 0xEF);
assert_eq!(msg1[5], 0xBE);
assert_eq!(msg1[6], 0xAD);
assert_eq!(msg1[7], 0xDE);
}
#[test]
fn test_inner_header_prepend_strip() {
let timestamp: u32 = 12345;
let original = vec![0x10, 0xAA, 0xBB]; // msg_type + payload
let with_header = prepend_inner_header(timestamp, &original);
assert_eq!(with_header.len(), 4 + 3); // timestamp + original
let (ts, rest) = strip_inner_header(&with_header).unwrap();
assert_eq!(ts, 12345);
assert_eq!(rest, &original[..]);
}
#[test]
fn test_inner_header_too_short() {
assert!(strip_inner_header(&[0, 0, 0, 0]).is_none()); // needs 5 bytes minimum
}
#[test]
fn test_flags_byte() {
let header = build_established_header(
SessionIndex::new(1),
0,
FLAG_KEY_EPOCH | FLAG_SP,
100,
);
assert_eq!(header[1], 0x05); // bits 0 and 2 set
let parsed = EncryptedHeader::parse(&[
header[0], header[1], header[2], header[3],
header[4], header[5], header[6], header[7],
header[8], header[9], header[10], header[11],
header[12], header[13], header[14], header[15],
// minimum: TAG_SIZE bytes of ciphertext
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
]).unwrap();
assert_eq!(parsed.flags & FLAG_KEY_EPOCH, FLAG_KEY_EPOCH);
assert_eq!(parsed.flags & FLAG_CE, 0);
assert_eq!(parsed.flags & FLAG_SP, FLAG_SP);
}
#[test]
fn test_payload_len_in_msg1() {
let packet = build_msg1(SessionIndex::new(1), &[0u8; HANDSHAKE_MSG1_SIZE]);
let prefix = CommonPrefix::parse(&packet).unwrap();
// payload_len = sender_idx(4) + noise_msg1(82) = 86
assert_eq!(prefix.payload_len, 86);
}
#[test]
fn test_payload_len_in_msg2() {
let packet = build_msg2(
SessionIndex::new(1),
SessionIndex::new(2),
&[0u8; HANDSHAKE_MSG2_SIZE],
);
let prefix = CommonPrefix::parse(&packet).unwrap();
// payload_len = sender_idx(4) + receiver_idx(4) + noise_msg2(33) = 41
assert_eq!(prefix.payload_len, 41);
}
}
+66 -1
View File
@@ -36,7 +36,7 @@ mod replay;
mod session;
use chacha20poly1305::{
aead::{Aead, KeyInit},
aead::{Aead, KeyInit, Payload},
ChaCha20Poly1305, Nonce,
};
use std::fmt;
@@ -266,6 +266,71 @@ impl CipherState {
Ok(plaintext)
}
/// Encrypt plaintext with Additional Authenticated Data (AAD).
///
/// The AAD is authenticated but not encrypted. Used for the FLP
/// established frame format where the 16-byte outer header is
/// bound to the AEAD tag.
pub fn encrypt_with_aad(
&mut self,
plaintext: &[u8],
aad: &[u8],
) -> Result<Vec<u8>, NoiseError> {
if !self.has_key {
return Ok(plaintext.to_vec());
}
if plaintext.len() > MAX_MESSAGE_SIZE - TAG_SIZE {
return Err(NoiseError::MessageTooLarge {
size: plaintext.len(),
max: MAX_MESSAGE_SIZE - TAG_SIZE,
});
}
let cipher = ChaCha20Poly1305::new_from_slice(&self.key)
.map_err(|_| NoiseError::EncryptionFailed)?;
let nonce = self.next_nonce()?;
let ciphertext = cipher
.encrypt(&nonce, Payload { msg: plaintext, aad })
.map_err(|_| NoiseError::EncryptionFailed)?;
Ok(ciphertext)
}
/// Decrypt with an explicit counter and AAD (for transport phase).
///
/// Combines explicit counter (from wire format) with AAD verification.
/// The AAD must match exactly what was used during encryption or the
/// AEAD tag verification will fail.
pub fn decrypt_with_counter_and_aad(
&self,
ciphertext: &[u8],
counter: u64,
aad: &[u8],
) -> Result<Vec<u8>, NoiseError> {
if !self.has_key {
return Ok(ciphertext.to_vec());
}
if ciphertext.len() < TAG_SIZE {
return Err(NoiseError::MessageTooShort {
expected: TAG_SIZE,
got: ciphertext.len(),
});
}
let cipher = ChaCha20Poly1305::new_from_slice(&self.key)
.map_err(|_| NoiseError::DecryptionFailed)?;
let nonce = Self::counter_to_nonce(counter);
let plaintext = cipher
.decrypt(&nonce, Payload { msg: ciphertext, aad })
.map_err(|_| NoiseError::DecryptionFailed)?;
Ok(plaintext)
}
/// Convert a counter value to a nonce.
fn counter_to_nonce(counter: u64) -> Nonce {
let mut nonce_bytes = [0u8; 12];
+37
View File
@@ -104,6 +104,43 @@ impl NoiseSession {
Ok(plaintext)
}
/// Encrypt a message with Additional Authenticated Data (AAD).
///
/// Returns the ciphertext. The current send counter should be included
/// in the wire format before calling this method.
pub fn encrypt_with_aad(
&mut self,
plaintext: &[u8],
aad: &[u8],
) -> Result<Vec<u8>, NoiseError> {
self.send_cipher.encrypt_with_aad(plaintext, aad)
}
/// Decrypt with explicit counter, replay protection, and AAD.
///
/// This is the primary decryption method for the FLP transport phase
/// with AAD binding. The AAD (typically the 16-byte outer header) must
/// match what was used during encryption.
pub fn decrypt_with_replay_check_and_aad(
&mut self,
ciphertext: &[u8],
counter: u64,
aad: &[u8],
) -> Result<Vec<u8>, NoiseError> {
// Check replay window first (cheap)
if !self.replay_window.check(counter) {
return Err(NoiseError::ReplayDetected(counter));
}
// Attempt decryption with AAD (expensive)
let plaintext = self.recv_cipher.decrypt_with_counter_and_aad(ciphertext, counter, aad)?;
// Only accept into window after successful decryption
self.replay_window.accept(counter);
Ok(plaintext)
}
/// Get the highest received counter.
pub fn highest_received_counter(&self) -> u64 {
self.replay_window.highest()
+37
View File
@@ -4,6 +4,7 @@
//! ActivePeer holds tree state, Bloom filter, and routing information.
use crate::bloom::BloomFilter;
use crate::mmp::{MmpConfig, MmpPeerState};
use crate::utils::index::SessionIndex;
use crate::noise::NoiseSession;
use crate::transport::{LinkId, LinkStats, TransportAddr, TransportId};
@@ -11,6 +12,7 @@ use crate::tree::{ParentDeclaration, TreeCoordinate};
use crate::{FipsAddress, NodeAddr, PeerIdentity};
use secp256k1::XOnlyPublicKey;
use std::fmt;
use std::time::Instant;
/// Connectivity state for an active peer.
///
@@ -112,6 +114,11 @@ pub struct ActivePeer {
/// Whether we owe them a filter update.
pending_filter_update: bool,
// === Timing ===
/// Session start time for computing session-relative timestamps.
/// Used as the epoch for the 4-byte inner header timestamp field.
session_start: Instant,
// === Statistics ===
/// Link statistics.
link_stats: LinkStats,
@@ -119,6 +126,10 @@ pub struct ActivePeer {
authenticated_at: u64,
/// When this peer was last seen (any activity, Unix milliseconds).
last_seen: u64,
// === MMP ===
/// Per-peer MMP state (None for legacy peers without Noise sessions).
mmp: Option<MmpPeerState>,
}
impl ActivePeer {
@@ -145,9 +156,11 @@ impl ActivePeer {
filter_sequence: 0,
filter_received_at: 0,
pending_filter_update: true, // Send filter on new connection
session_start: Instant::now(),
link_stats: LinkStats::new(),
authenticated_at,
last_seen: authenticated_at,
mmp: None,
}
}
@@ -181,6 +194,8 @@ impl ActivePeer {
transport_id: TransportId,
current_addr: TransportAddr,
link_stats: LinkStats,
is_initiator: bool,
mmp_config: &MmpConfig,
) -> Self {
Self {
identity,
@@ -200,9 +215,11 @@ impl ActivePeer {
filter_sequence: 0,
filter_received_at: 0,
pending_filter_update: true,
session_start: Instant::now(),
link_stats,
authenticated_at,
last_seen: authenticated_at,
mmp: Some(MmpPeerState::new(mmp_config, is_initiator)),
}
}
@@ -395,6 +412,18 @@ impl ActivePeer {
&mut self.link_stats
}
// === MMP Accessors ===
/// Get MMP state (None for legacy peers without sessions).
pub fn mmp(&self) -> Option<&MmpPeerState> {
self.mmp.as_ref()
}
/// Get mutable MMP state.
pub fn mmp_mut(&mut self) -> Option<&mut MmpPeerState> {
self.mmp.as_mut()
}
/// Link cost for routing decisions.
///
/// Returns a scalar cost where lower is better. Currently returns a
@@ -424,6 +453,14 @@ impl ActivePeer {
current_time_ms.saturating_sub(self.authenticated_at)
}
/// Session-relative elapsed time in milliseconds (for inner header timestamp).
///
/// Returns milliseconds since session establishment, truncated to u32.
/// Wraps at ~49.7 days which is acceptable for session-relative timing.
pub fn session_elapsed_ms(&self) -> u32 {
self.session_start.elapsed().as_millis() as u32
}
// === State Updates ===
/// Update last seen timestamp.
+71 -46
View File
@@ -68,6 +68,17 @@ impl fmt::Display for HandshakeMessageType {
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[repr(u8)]
pub enum LinkMessageType {
// Forwarding (0x00-0x0F)
/// Encapsulated session-layer datagram for forwarding.
/// Payload is opaque to intermediate nodes (end-to-end encrypted).
SessionDatagram = 0x00,
// MMP reports (0x01-0x02) — content defined in TASK-2026-0006
/// Sender-side MMP report (stub).
SenderReport = 0x01,
/// Receiver-side MMP report (stub).
ReceiverReport = 0x02,
// Tree protocol (0x10-0x1F)
/// Spanning tree state announcement.
TreeAnnounce = 0x10,
@@ -82,11 +93,6 @@ pub enum LinkMessageType {
/// Response with target's coordinates.
LookupResponse = 0x31,
// Forwarding (0x40-0x4F)
/// Encapsulated session-layer datagram for forwarding.
/// Payload is opaque to intermediate nodes (end-to-end encrypted).
SessionDatagram = 0x40,
// Link Control (0x50-0x5F)
/// Orderly disconnect notification before link closure.
Disconnect = 0x50,
@@ -96,11 +102,13 @@ impl LinkMessageType {
/// Try to convert from a byte.
pub fn from_byte(b: u8) -> Option<Self> {
match b {
0x00 => Some(LinkMessageType::SessionDatagram),
0x01 => Some(LinkMessageType::SenderReport),
0x02 => Some(LinkMessageType::ReceiverReport),
0x10 => Some(LinkMessageType::TreeAnnounce),
0x20 => Some(LinkMessageType::FilterAnnounce),
0x30 => Some(LinkMessageType::LookupRequest),
0x31 => Some(LinkMessageType::LookupResponse),
0x40 => Some(LinkMessageType::SessionDatagram),
0x50 => Some(LinkMessageType::Disconnect),
_ => None,
}
@@ -115,11 +123,13 @@ impl LinkMessageType {
impl fmt::Display for LinkMessageType {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let name = match self {
LinkMessageType::SessionDatagram => "SessionDatagram",
LinkMessageType::SenderReport => "SenderReport",
LinkMessageType::ReceiverReport => "ReceiverReport",
LinkMessageType::TreeAnnounce => "TreeAnnounce",
LinkMessageType::FilterAnnounce => "FilterAnnounce",
LinkMessageType::LookupRequest => "LookupRequest",
LinkMessageType::LookupResponse => "LookupResponse",
LinkMessageType::SessionDatagram => "SessionDatagram",
LinkMessageType::Disconnect => "Disconnect",
};
write!(f, "{}", name)
@@ -246,20 +256,21 @@ impl Disconnect {
/// Encapsulated session-layer datagram for multi-hop forwarding.
///
/// This is a link-layer message (type 0x40) that carries session-layer
/// This is a link-layer message (type 0x00) that carries session-layer
/// payloads through the mesh. The envelope provides source and destination
/// addressing that transit routers use for forwarding decisions and error
/// routing.
///
/// ## Wire Format (34-byte fixed header)
/// ## Wire Format (36-byte fixed header)
///
/// | Offset | Field | Size | Description |
/// |--------|-----------|----------|--------------------------------|
/// | 0 | msg_type | 1 byte | 0x40 |
/// | 1 | src_addr | 16 bytes | Source node_addr |
/// | 17 | dest_addr | 16 bytes | Destination node_addr |
/// | 33 | hop_limit | 1 byte | Decremented each hop |
/// | 34 | payload | variable | Session-layer message |
/// | Offset | Field | Size | Description |
/// |--------|-----------|----------|-------------------------------------|
/// | 0 | msg_type | 1 byte | 0x00 |
/// | 1 | ttl | 1 byte | Decremented each hop |
/// | 2 | path_mtu | 2 bytes | Path MTU (LE), min'd at each hop |
/// | 4 | src_addr | 16 bytes | Source node_addr |
/// | 20 | dest_addr | 16 bytes | Destination node_addr |
/// | 36 | payload | variable | Session-layer message |
///
/// The payload is either end-to-end encrypted (SessionSetup, SessionAck,
/// DataPacket) or plaintext link-layer error signals (CoordsRequired,
@@ -272,14 +283,17 @@ pub struct SessionDatagram {
pub src_addr: NodeAddr,
/// Destination node address (for routing decisions).
pub dest_addr: NodeAddr,
/// Hop limit (decremented at each hop, dropped at zero).
pub hop_limit: u8,
/// Time-to-live (decremented at each hop, dropped at zero).
pub ttl: u8,
/// Path MTU: minimum link MTU along the path so far.
/// Each forwarding hop applies min(path_mtu, outgoing_link_mtu).
pub path_mtu: u16,
/// Session-layer payload (e2e encrypted or plaintext error signal).
pub payload: Vec<u8>,
}
/// SessionDatagram fixed header size: msg_type(1) + src_addr(16) + dest_addr(16) + hop_limit(1).
pub const SESSION_DATAGRAM_HEADER_SIZE: usize = 34;
/// SessionDatagram fixed header size: msg_type(1) + ttl(1) + path_mtu(2) + src_addr(16) + dest_addr(16).
pub const SESSION_DATAGRAM_HEADER_SIZE: usize = 36;
impl SessionDatagram {
/// Create a new session datagram.
@@ -287,21 +301,28 @@ impl SessionDatagram {
Self {
src_addr,
dest_addr,
hop_limit: 64,
ttl: 64,
path_mtu: u16::MAX,
payload,
}
}
/// Set the hop limit.
pub fn with_hop_limit(mut self, hop_limit: u8) -> Self {
self.hop_limit = hop_limit;
/// Set the TTL.
pub fn with_ttl(mut self, ttl: u8) -> Self {
self.ttl = ttl;
self
}
/// Decrement hop limit, returning false if exhausted.
pub fn decrement_hop_limit(&mut self) -> bool {
if self.hop_limit > 0 {
self.hop_limit -= 1;
/// Set the path MTU.
pub fn with_path_mtu(mut self, path_mtu: u16) -> Self {
self.path_mtu = path_mtu;
self
}
/// Decrement TTL, returning false if exhausted.
pub fn decrement_ttl(&mut self) -> bool {
if self.ttl > 0 {
self.ttl -= 1;
true
} else {
false
@@ -310,40 +331,43 @@ impl SessionDatagram {
/// Check if the datagram can be forwarded.
pub fn can_forward(&self) -> bool {
self.hop_limit > 0
self.ttl > 0
}
/// Encode as link-layer message (msg_type + src_addr + dest_addr + hop_limit + payload).
/// Encode as link-layer message (msg_type + ttl + path_mtu + src_addr + dest_addr + payload).
pub fn encode(&self) -> Vec<u8> {
let mut buf = Vec::with_capacity(SESSION_DATAGRAM_HEADER_SIZE + self.payload.len());
buf.push(LinkMessageType::SessionDatagram.to_byte());
buf.push(self.ttl);
buf.extend_from_slice(&self.path_mtu.to_le_bytes());
buf.extend_from_slice(self.src_addr.as_bytes());
buf.extend_from_slice(self.dest_addr.as_bytes());
buf.push(self.hop_limit);
buf.extend_from_slice(&self.payload);
buf
}
/// Decode from link-layer payload (after msg_type byte has been consumed).
pub fn decode(payload: &[u8]) -> Result<Self, ProtocolError> {
// src_addr(16) + dest_addr(16) + hop_limit(1) = 33
if payload.len() < 33 {
// ttl(1) + path_mtu(2) + src_addr(16) + dest_addr(16) = 35
if payload.len() < 35 {
return Err(ProtocolError::MessageTooShort {
expected: 33,
expected: 35,
got: payload.len(),
});
}
let ttl = payload[0];
let path_mtu = u16::from_le_bytes([payload[1], payload[2]]);
let mut src_bytes = [0u8; 16];
src_bytes.copy_from_slice(&payload[0..16]);
src_bytes.copy_from_slice(&payload[3..19]);
let mut dest_bytes = [0u8; 16];
dest_bytes.copy_from_slice(&payload[16..32]);
let hop_limit = payload[32];
let inner_payload = payload[33..].to_vec();
dest_bytes.copy_from_slice(&payload[19..35]);
let inner_payload = payload[35..].to_vec();
Ok(Self {
src_addr: NodeAddr::from_bytes(src_bytes),
dest_addr: NodeAddr::from_bytes(dest_bytes),
hop_limit,
ttl,
path_mtu,
payload: inner_payload,
})
}
@@ -411,7 +435,8 @@ mod tests {
#[test]
fn test_link_message_type_invalid() {
assert!(LinkMessageType::from_byte(0xFF).is_none());
assert!(LinkMessageType::from_byte(0x00).is_none());
assert!(LinkMessageType::from_byte(0x03).is_none());
assert!(LinkMessageType::from_byte(0x40).is_none());
}
// ===== DisconnectReason Tests =====
@@ -503,17 +528,17 @@ mod tests {
let dest = make_node_addr(0xBB);
let payload = vec![0x10, 0x00, 0x05, 0x00, 1, 2, 3, 4, 5]; // DataPacket payload
let dg = SessionDatagram::new(src, dest, payload.clone())
.with_hop_limit(32);
.with_ttl(32);
let encoded = dg.encode();
assert_eq!(encoded[0], 0x40); // msg_type
assert_eq!(encoded[0], 0x00); // msg_type (SessionDatagram)
assert_eq!(encoded.len(), SESSION_DATAGRAM_HEADER_SIZE + payload.len());
// Decode (after msg_type)
let decoded = SessionDatagram::decode(&encoded[1..]).unwrap();
assert_eq!(decoded.src_addr, src);
assert_eq!(decoded.dest_addr, dest);
assert_eq!(decoded.hop_limit, 32);
assert_eq!(decoded.ttl, 32);
assert_eq!(decoded.payload, payload);
}
@@ -535,14 +560,14 @@ mod tests {
}
#[test]
fn test_session_datagram_hop_limit_roundtrip() {
fn test_session_datagram_ttl_roundtrip() {
for hop in [0u8, 1, 64, 128, 255] {
let dg = SessionDatagram::new(make_node_addr(1), make_node_addr(2), vec![0x42])
.with_hop_limit(hop);
.with_ttl(hop);
let encoded = dg.encode();
let decoded = SessionDatagram::decode(&encoded[1..]).unwrap();
assert_eq!(decoded.hop_limit, hop);
assert_eq!(decoded.ttl, hop);
}
}
}
+2 -2
View File
@@ -12,7 +12,7 @@ use std::fmt;
/// SessionDatagram payload message type identifiers.
///
/// These messages are carried as payloads inside `SessionDatagram` (link
/// message type 0x40). Session-layer messages (SessionSetup, SessionAck,
/// message type 0x00). Session-layer messages (SessionSetup, SessionAck,
/// DataPacket) are end-to-end encrypted with session keys. Error signals
/// (CoordsRequired, PathBroken) are plaintext link-layer messages generated
/// by transit routers that cannot establish e2e sessions with the source.
@@ -483,7 +483,7 @@ impl DataFlags {
/// DataPacket header size in bytes (excluding payload).
/// msg_type(1) + flags(1) + counter(8) + payload_length(2) = 12
/// (Addressing and hop_limit are in the SessionDatagram envelope.)
/// (Addressing and TTL are in the SessionDatagram envelope.)
pub const DATA_HEADER_SIZE: usize = 12;
/// Encrypted application data carried inside a SessionDatagram.
+4 -3
View File
@@ -63,11 +63,12 @@ const MAX_ORIGINAL_PACKET: usize = MIN_IPV6_MTU - IPV6_HEADER_LEN - ICMPV6_HEADE
///
/// Breakdown:
/// - Noise encryption tag: 16 bytes
/// - Encrypted frame header: 13 bytes
/// - SessionDatagram header: 34 bytes
/// - Established frame header: 16 bytes (common prefix + receiver_idx + counter)
/// - Inner header: 5 bytes (4-byte timestamp + 1-byte msg_type)
/// - SessionDatagram fields: 35 bytes (ttl + path_mtu + src_addr + dest_addr)
/// - DataPacket header: 12 bytes (msg_type + flags + counter + payload_len)
/// - Coordinates (worst case): ~60 bytes (2 coords with depth 3 each)
pub const FIPS_OVERHEAD: u16 = 16 + 13 + 34 + 12 + 60; // 135 bytes
pub const FIPS_OVERHEAD: u16 = 16 + 16 + 5 + 35 + 12 + 60; // 144 bytes
/// Calculate the effective IPv6 MTU for FIPS-encapsulated traffic.
///