mirror of
https://github.com/jmcorgan/fips.git
synced 2026-07-30 19:46:15 +00:00
Design documentation refresh: MMP, wire formats, and configuration alignment
Bring all 9 design documents into alignment with the current implementation. Major additions include MMP coverage at both link and session layers, wire format tables for SenderReport and ReceiverReport, UDP socket buffer sizing, PathMtuNotification status updates, and configuration parameter fixes (default_ttl key name, idle timeout MMP exclusion semantics).
This commit is contained in:
Johnathan Corgan
@@ -144,15 +144,38 @@ Controls end-to-end session behavior and packet queuing.
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `node.session.default_hop_limit` | u8 | `64` | Default SessionDatagram hop limit |
|
||||
| `node.session.default_ttl` | u8 | `64` | Default SessionDatagram TTL |
|
||||
| `node.session.pending_packets_per_dest` | usize | `16` | Queue depth per destination during session establishment |
|
||||
| `node.session.pending_max_destinations` | usize | `256` | Max destinations with pending packets |
|
||||
| `node.session.idle_timeout_secs` | u64 | `90` | Idle session timeout; established sessions with no activity for this duration are removed |
|
||||
| `node.session.idle_timeout_secs` | u64 | `90` | Idle session timeout; established sessions with no application data for this duration are removed. MMP reports (SenderReport, ReceiverReport, PathMtuNotification) do not count as activity |
|
||||
| `node.session.coords_warmup_packets` | u8 | `5` | Number of initial data packets per session that include the CP flag for transit cache warmup; also the reset count on CoordsRequired receipt |
|
||||
|
||||
The anti-replay window size (2048 packets) is a compile-time constant and not
|
||||
configurable.
|
||||
|
||||
### Link-Layer MMP (`node.mmp.*`)
|
||||
|
||||
Metrics Measurement Protocol for per-peer link measurement. See
|
||||
[fips-link-layer.md](fips-link-layer.md) for behavioral details.
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `node.mmp.mode` | string | `"full"` | Operating mode: `full` (sender + receiver reports), `lightweight` (receiver reports only), or `minimal` (spin bit + CE echo only, no reports) |
|
||||
| `node.mmp.log_interval_secs` | u64 | `30` | Periodic operator log interval for link metrics |
|
||||
| `node.mmp.owd_window_size` | usize | `32` | One-way delay trend ring buffer size |
|
||||
|
||||
### Session-Layer MMP (`node.session_mmp.*`)
|
||||
|
||||
Metrics Measurement Protocol for end-to-end session measurement. Configured
|
||||
independently from link-layer MMP because session reports are routed through
|
||||
every transit link, consuming bandwidth proportional to path length.
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `node.session_mmp.mode` | string | `"full"` | Operating mode: `full`, `lightweight`, or `minimal` |
|
||||
| `node.session_mmp.log_interval_secs` | u64 | `30` | Periodic operator log interval for session metrics |
|
||||
| `node.session_mmp.owd_window_size` | usize | `32` | One-way delay trend ring buffer size |
|
||||
|
||||
### Internal Buffers (`node.buffers.*`)
|
||||
|
||||
Channel sizes affecting throughput and memory. Primarily useful for performance
|
||||
@@ -196,6 +219,8 @@ stale address mappings.
|
||||
|-----------|------|---------|-------------|
|
||||
| `transports.udp.bind_addr` | string | `"0.0.0.0:4000"` | UDP bind address and port |
|
||||
| `transports.udp.mtu` | u16 | `1280` | Transport MTU |
|
||||
| `transports.udp.recv_buf_size` | usize | `2097152` | UDP socket receive buffer size in bytes (2 MB). Linux kernel doubles the requested value internally. Host `net.core.rmem_max` must be >= this value. |
|
||||
| `transports.udp.send_buf_size` | usize | `2097152` | UDP socket send buffer size in bytes (2 MB). Host `net.core.wmem_max` must be >= this value. |
|
||||
|
||||
## Peers (`peers[]`)
|
||||
|
||||
@@ -297,11 +322,19 @@ node:
|
||||
bloom:
|
||||
update_debounce_ms: 500
|
||||
session:
|
||||
default_hop_limit: 64
|
||||
default_ttl: 64
|
||||
pending_packets_per_dest: 16
|
||||
pending_max_destinations: 256
|
||||
idle_timeout_secs: 90
|
||||
coords_warmup_packets: 5
|
||||
mmp:
|
||||
mode: full # full | lightweight | minimal
|
||||
log_interval_secs: 30
|
||||
owd_window_size: 32
|
||||
session_mmp:
|
||||
mode: full # full | lightweight | minimal
|
||||
log_interval_secs: 30
|
||||
owd_window_size: 32
|
||||
buffers:
|
||||
packet_channel: 1024
|
||||
tun_channel: 1024
|
||||
@@ -322,6 +355,8 @@ transports:
|
||||
udp:
|
||||
bind_addr: "0.0.0.0:4000"
|
||||
mtu: 1280
|
||||
recv_buf_size: 2097152 # 2 MB (kernel doubles to 4 MB actual)
|
||||
send_buf_size: 2097152 # 2 MB
|
||||
|
||||
peers: []
|
||||
```
|
||||
|
||||
@@ -58,6 +58,10 @@ alone, the network self-organizes:
|
||||
- **Routing decisions** are made locally at each hop, using only the node's
|
||||
immediate peers and cached coordinate information
|
||||
|
||||
Each peer link and end-to-end session actively measures RTT, loss, jitter, and
|
||||
goodput through a lightweight in-band Metrics Measurement Protocol (MMP),
|
||||
providing operator visibility and a foundation for future quality-aware routing.
|
||||
|
||||
The result is a network that builds itself from the bottom up, heals around
|
||||
failures automatically, and scales without central coordination. Adding a node
|
||||
is as simple as connecting to one existing peer — the network integrates the
|
||||
|
||||
@@ -283,7 +283,7 @@ TUN device creation requires `CAP_NET_ADMIN`. Options:
|
||||
| Per-destination route MTU (netlink) | Planned |
|
||||
| Transit MTU error signal | Planned |
|
||||
| Path MTU tracking (SessionDatagram field) | **Implemented** |
|
||||
| Path MTU notification (end-to-end echo) | Future direction |
|
||||
| Path MTU notification (end-to-end echo) | **Implemented** |
|
||||
| Endpoint fragmentation/reassembly | Future direction |
|
||||
|
||||
## Design Considerations
|
||||
@@ -296,8 +296,10 @@ Two complementary mechanisms support full PMTUD:
|
||||
is implemented at the FLP level. The source sets it to its outbound link MTU
|
||||
minus overhead; each transit node applies
|
||||
`min(current, own_outbound_mtu - overhead)`. The destination receives the
|
||||
forward-path minimum. A session-layer echo (PathMtuNotification, 2 bytes
|
||||
inside encryption) to return the value to the source is a future direction.
|
||||
forward-path minimum. PathMtuNotification is handled at the session layer;
|
||||
the destination sends the observed forward-path MTU back to the source,
|
||||
which applies it with decrease-immediate / increase-requires-3-consecutive
|
||||
hysteresis.
|
||||
|
||||
2. **Reactive**: When a transit node cannot forward a packet (MTU exceeded), it
|
||||
sends an error signal back to the source. This handles the in-flight gap
|
||||
|
||||
@@ -311,8 +311,8 @@ FLP defines eight message types carried inside encrypted frames:
|
||||
| 0x30 | LookupRequest | Coordinate discovery — flood toward destination |
|
||||
| 0x31 | LookupResponse | Coordinate discovery — response with coordinates |
|
||||
| 0x00 | SessionDatagram | Encapsulated session-layer payload for forwarding |
|
||||
| 0x01 | SenderReport | Link statistics — what this node sent (reserved) |
|
||||
| 0x02 | ReceiverReport | Link statistics — what this node observed (reserved) |
|
||||
| 0x01 | SenderReport | MMP sender-side metrics report |
|
||||
| 0x02 | ReceiverReport | MMP receiver-side metrics report |
|
||||
| 0x50 | Disconnect | Orderly link teardown with reason code |
|
||||
|
||||
Additionally, handshake messages (phase 0x1 msg1, phase 0x2 msg2) are sent
|
||||
@@ -327,6 +327,77 @@ See [fips-mesh-operation.md](fips-mesh-operation.md) for how these messages
|
||||
work together to build and maintain the mesh, and
|
||||
[fips-wire-formats.md](fips-wire-formats.md) for byte-level message layouts.
|
||||
|
||||
## Metrics Measurement Protocol (MMP)
|
||||
|
||||
Each active peer link runs an instance of the Metrics Measurement Protocol,
|
||||
providing per-link quality metrics to the operator and (in future) to the
|
||||
routing layer for link-cost decisions.
|
||||
|
||||
### Metrics Tracked
|
||||
|
||||
MMP computes the following metrics from the per-frame counter and timestamp
|
||||
fields in the FLP wire format:
|
||||
|
||||
- **SRTT** — Smoothed round-trip time (Jacobson/RFC 6298, α=1/8). Derived
|
||||
from timestamp-echo in ReceiverReports with dwell-time compensation.
|
||||
- **Loss rate** — Bidirectional loss inferred from counter gaps. Tracked as
|
||||
both instantaneous (per-interval) and long-term EWMA.
|
||||
- **Jitter** — Interarrival jitter (RFC 3550 algorithm) in microseconds.
|
||||
- **Goodput** — Bytes per second of payload data (excludes MMP reports).
|
||||
- **OWD trend** — One-way delay trend (µs/s, signed). Indicates congestion
|
||||
buildup before loss occurs.
|
||||
- **ETX** — Expected Transmission Count, computed from bidirectional delivery
|
||||
ratios. Not yet wired into routing decisions (link cost is currently
|
||||
constant).
|
||||
- **Dual EWMA trends** — Short-term (α=1/4) and long-term (α=1/32) trend
|
||||
indicators for both RTT and loss, enabling change detection.
|
||||
|
||||
### Operating Modes
|
||||
|
||||
MMP supports three modes, configured via `node.mmp.mode`:
|
||||
|
||||
| Mode | Reports Exchanged | Metrics Available |
|
||||
| ---- | ----------------- | ----------------- |
|
||||
| **Full** (default) | SenderReport + ReceiverReport | All metrics including RTT, loss, jitter, goodput, OWD trend |
|
||||
| **Lightweight** | ReceiverReport only | Loss (from counter gaps), jitter, OWD trend. No RTT. |
|
||||
| **Minimal** | None | Spin bit and CE echo flags only. No computed metrics. |
|
||||
|
||||
### Report Scheduling
|
||||
|
||||
Reports are sent at RTT-adaptive intervals, clamped to [100ms, 2s]. A
|
||||
cold-start interval of 500ms is used before SRTT converges. The interval
|
||||
formula is `clamp(2 × SRTT, 100ms, 2000ms)`.
|
||||
|
||||
### Spin Bit and RTT
|
||||
|
||||
The SP (spin bit) flag in the FLP inner header follows the QUIC spin bit
|
||||
pattern: reflected on receive, toggled on send when the reflected value
|
||||
matches the last sent value. The spin bit state machine runs for TX
|
||||
reflection, but **RTT samples from the spin bit are discarded**. In a mesh
|
||||
protocol where frames are sent irregularly (tree announces, bloom filters,
|
||||
MMP reports on different timers), inter-frame processing delays inflate spin
|
||||
bit RTT measurements unpredictably. Timestamp-echo from ReceiverReports
|
||||
(with dwell-time compensation) is the sole SRTT source.
|
||||
|
||||
### CE Echo
|
||||
|
||||
The CE (Congestion Experienced) echo flag (bit 1 in the FLP flags byte) is
|
||||
reserved for ECN signaling. The transport trait does not currently expose
|
||||
ECN marking, so the CE echo flag is never set. One-way delay trend serves
|
||||
as the sole pre-loss congestion indicator.
|
||||
|
||||
### Operator Logging
|
||||
|
||||
MMP emits periodic link metrics at info level (configurable via
|
||||
`node.mmp.log_interval_secs`, default 30s):
|
||||
|
||||
```text
|
||||
MMP link metrics peer=node-b rtt=2.3ms loss=0.2% jitter=0.1ms goodput=76.0MB/s tx_pkts=1234 rx_pkts=5678
|
||||
```
|
||||
|
||||
Teardown logs include final SRTT, loss rate, jitter, ETX, goodput, and
|
||||
cumulative tx/rx packet and byte counts.
|
||||
|
||||
## Security Properties
|
||||
|
||||
### Threat Resistance
|
||||
@@ -379,6 +450,7 @@ an attacker sends invalid packets to elicit responses.
|
||||
| AAD binding on encrypted frames | **Implemented** |
|
||||
| Inner header timestamps | **Implemented** |
|
||||
| Path MTU tracking (SessionDatagram) | **Implemented** |
|
||||
| Metrics Measurement Protocol (MMP) | **Implemented** |
|
||||
| Rekey with index rotation | Planned |
|
||||
| Allowlist/blocklist | Planned |
|
||||
|
||||
|
||||
@@ -181,8 +181,9 @@ through to greedy tree routing (functional but suboptimal).
|
||||
When bloom filters identify multiple candidate peers, they are ranked by a
|
||||
composite key:
|
||||
|
||||
1. **link_cost** — Per-link quality metric (currently constant; designed for
|
||||
future ETX or similar metrics)
|
||||
1. **link_cost** — Per-link quality metric. ETX is computed from bidirectional
|
||||
delivery ratios in MMP metrics but is not yet wired into `find_next_hop()`
|
||||
candidate ranking; link cost remains constant in the current implementation.
|
||||
2. **tree_distance** — Coordinate-based distance to destination through this
|
||||
peer
|
||||
3. **node_addr** — Deterministic tie-breaker
|
||||
@@ -393,6 +394,10 @@ Both error types are rate-limited at transit nodes: maximum one error per
|
||||
destination per 100ms. This prevents storms during topology changes when many
|
||||
packets to the same destination hit the same routing failure simultaneously.
|
||||
|
||||
Error signals (CoordsRequired, PathBroken) are handled asynchronously outside
|
||||
the packet receive path, allowing the RX loop to continue processing without
|
||||
blocking on discovery or session repair.
|
||||
|
||||
### Error Routing Limitation
|
||||
|
||||
Error signals route back to the source using `find_next_hop(src_addr)`. For
|
||||
|
||||
@@ -159,6 +159,13 @@ encrypted message includes:
|
||||
Sessions that see no traffic for a configurable duration (default 90s) are
|
||||
torn down. When traffic resumes, a new session is established automatically.
|
||||
|
||||
Only DataPacket (message type 0x10) send/receive and session establishment
|
||||
reset `last_activity`. MMP traffic — SenderReport, ReceiverReport, and
|
||||
PathMtuNotification — does not reset the idle timer. This means a session
|
||||
carrying only MMP reports and no application data will still tear down after
|
||||
`node.session.idle_timeout_secs`. MMP reports continue flowing until the
|
||||
session is torn down, providing final measurement data for the teardown log.
|
||||
|
||||
The idle timeout is deliberately shorter than the coordinate cache TTL (300s).
|
||||
This ordering ensures that when traffic stops and the session tears down, the
|
||||
transit node coordinate caches are still warm when a new session is established.
|
||||
@@ -375,6 +382,71 @@ When traffic stops: the session tears down at 90s. When traffic resumes: DNS
|
||||
re-resolves the identity, a fresh SessionSetup carries coordinates, and transit
|
||||
node caches (still within their 300s TTL) are re-warmed.
|
||||
|
||||
## Session-Layer MMP
|
||||
|
||||
Each established session runs its own Metrics Measurement Protocol instance,
|
||||
providing end-to-end quality metrics independent of the number of hops.
|
||||
|
||||
### Relationship to Link-Layer MMP
|
||||
|
||||
Session-layer MMP uses the same report wire format (SenderReport 0x11,
|
||||
ReceiverReport 0x12) and identical algorithms as link-layer MMP, but with
|
||||
two key differences:
|
||||
|
||||
1. **End-to-end routing**: Session reports are encrypted and forwarded through
|
||||
every transit link. A 3-hop session generates report traffic on all 3 links,
|
||||
making bandwidth cost proportional to path length.
|
||||
2. **Independent configuration**: The `node.session_mmp.*` parameters are
|
||||
separate from `node.mmp.*`, allowing operators to run a lighter mode for
|
||||
sessions (e.g., Lightweight) while keeping Full mode on links.
|
||||
|
||||
### Metrics
|
||||
|
||||
The same metrics as link-layer MMP: SRTT, loss rate, jitter, goodput, OWD
|
||||
trend, ETX, and dual EWMA trends. Session MMP additionally tracks observed
|
||||
path MTU.
|
||||
|
||||
### Report Intervals
|
||||
|
||||
Session-layer report intervals are higher than link-layer to account for
|
||||
bandwidth cost: clamped to [500ms, 10s] with a cold-start interval of 1s
|
||||
(vs. link-layer [100ms, 2s] with 500ms cold-start).
|
||||
|
||||
### Path MTU Tracking
|
||||
|
||||
PathMtuNotification (message type 0x13) provides end-to-end path MTU
|
||||
feedback:
|
||||
|
||||
1. The source sets `path_mtu` in each SessionDatagram envelope to its
|
||||
outbound link MTU.
|
||||
2. Each transit node applies `min(current, transport.mtu())` before
|
||||
forwarding.
|
||||
3. The destination receives the forward-path minimum and sends a
|
||||
PathMtuNotification (2-byte body: u16 LE path_mtu) back to the source.
|
||||
4. The source applies the notification with hysteresis:
|
||||
- **Decrease**: immediate (take lower value).
|
||||
- **Increase**: requires 3 consecutive higher-value notifications spanning
|
||||
at least 2 × notification interval.
|
||||
5. Notifications are sent on first measurement, on any decrease, and
|
||||
periodically at `max(10s, 5 × SRTT)`.
|
||||
|
||||
### Idle Timeout Interaction
|
||||
|
||||
MMP reports (SenderReport, ReceiverReport) and PathMtuNotification do **not**
|
||||
reset the session idle timer. Only application data (DataPacket, type 0x10)
|
||||
resets `last_activity`. This ensures sessions with no application traffic
|
||||
tear down after `node.session.idle_timeout_secs` (default 90s), while MMP
|
||||
continues providing measurement data up to the teardown moment.
|
||||
|
||||
### Operator Logging
|
||||
|
||||
Session metrics are logged at info level (configurable via
|
||||
`node.session_mmp.log_interval_secs`, default 30s):
|
||||
|
||||
```text
|
||||
MMP session metrics session=npub1tdwa...84le rtt=4.3ms loss=0.6% jitter=0.2ms goodput=71.3MB/s mtu=1472 tx_pkts=1234 rx_pkts=5678
|
||||
```
|
||||
|
||||
## Implementation Status
|
||||
|
||||
| Feature | Status |
|
||||
@@ -384,7 +456,7 @@ node caches (still within their 300s TTL) are re-warmed.
|
||||
| Explicit counter replay protection | **Implemented** |
|
||||
| CP warmup-then-reactive | **Implemented** |
|
||||
| FSP wire format (prefix, AAD, inner header) | **Implemented** |
|
||||
| Session-layer MMP report types | **Implemented** (wire format) |
|
||||
| Session-layer MMP | **Implemented** |
|
||||
| Identity cache (LRU-only) | **Implemented** |
|
||||
| Coordinate cache (unified, TTL + refresh) | **Implemented** |
|
||||
| Session idle timeout | **Implemented** |
|
||||
@@ -394,7 +466,7 @@ node caches (still within their 300s TTL) are re-warmed.
|
||||
| Flush coord cache on parent change | **Implemented** |
|
||||
| Rekey | Planned |
|
||||
| Path MTU tracking (FLP SessionDatagram field) | **Implemented** |
|
||||
| Path MTU notification (end-to-end echo) | Planned |
|
||||
| Path MTU notification (end-to-end echo) | **Implemented** |
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -157,6 +157,49 @@ retransmission timers, and acknowledgment tracking — complexity that gossip
|
||||
protocols avoid by sending full state periodically. A lost TreeAnnounce is
|
||||
simply replaced by the next one, which carries the same or newer state.
|
||||
|
||||
## Metrics Measurement Protocol
|
||||
|
||||
MMP is instantiated at two independent layers, each with its own
|
||||
configuration and state:
|
||||
|
||||
- **Link layer**: One `MmpPeerState` per `ActivePeer`. Measures per-hop
|
||||
quality using the FLP counter and timestamp fields that already exist on
|
||||
every encrypted frame. No additional message overhead beyond periodic
|
||||
SenderReport/ReceiverReport exchanges.
|
||||
|
||||
- **Session layer**: One `MmpSessionState` per established `SessionEntry`.
|
||||
Measures end-to-end quality using the FSP counter and timestamp fields.
|
||||
Reports are encrypted and forwarded through every transit link.
|
||||
|
||||
Both instantiations use identical algorithms (SRTT, loss, jitter, dual EWMA,
|
||||
OWD trend) but are configured independently via `node.mmp.*` and
|
||||
`node.session_mmp.*`. This allows operators to run Full mode on links (low
|
||||
overhead, single hop) while using Lightweight mode for sessions (reduces
|
||||
bandwidth cost on transit links).
|
||||
|
||||
### Peer Display Names
|
||||
|
||||
The node maintains a `peer_aliases` map (`HashMap<NodeAddr, String>`) populated
|
||||
from the `peers[].alias` field in configuration. All log output uses
|
||||
`peer_display_name()` to show human-readable names (e.g., "node-b") instead
|
||||
of truncated public keys, improving operator experience.
|
||||
|
||||
### Buffer Sizing Chain
|
||||
|
||||
Under high forwarding load, back-pressure propagates through:
|
||||
|
||||
1. **UDP socket receive buffer** (`transports.udp.recv_buf_size`, default 2 MB) —
|
||||
kernel-level buffer for incoming datagrams.
|
||||
2. **Packet channel** (`node.buffers.packet_channel`, default 1024) — async
|
||||
channel from transport receive loop to the node's RX event loop.
|
||||
3. **Processing** — decryption, routing decision, forwarding.
|
||||
|
||||
If the packet channel fills (RX loop can't keep up), the transport receive
|
||||
loop blocks, and the kernel receive buffer absorbs bursts. If the kernel
|
||||
buffer also fills, incoming datagrams are silently dropped
|
||||
(`RcvbufErrors` in `/proc/net/snmp`). The 2 MB default socket buffer was
|
||||
chosen to handle ~85 MB/s forwarding throughput without kernel drops.
|
||||
|
||||
## Bounded State Principle
|
||||
|
||||
FIPS nodes maintain state proportional to O(P × D), where P is the number
|
||||
|
||||
@@ -204,6 +204,41 @@ For internet-connected nodes, UDP/IP is the recommended transport:
|
||||
Raw IP with a custom protocol number would be simpler but is blocked by most
|
||||
NAT devices and firewalls, limiting deployment to networks without NAT.
|
||||
|
||||
### Socket Buffer Sizing
|
||||
|
||||
The default Linux UDP receive buffer (`net.core.rmem_default`, typically
|
||||
212 KB) is insufficient for high-throughput forwarding. At ~85 MB/s, a 212 KB
|
||||
buffer fills in ~2.5 ms; any stall in the async receive loop (decryption,
|
||||
routing, forwarding overhead) causes the kernel to silently drop incoming
|
||||
datagrams.
|
||||
|
||||
FIPS uses the `socket2` crate to configure socket buffers at bind time,
|
||||
before the receive loop starts:
|
||||
|
||||
| Parameter | Default | Description |
|
||||
| ---------------- | ------- | ------------------------------------ |
|
||||
| `recv_buf_size` | 2 MB | `SO_RCVBUF` — kernel receive buffer |
|
||||
| `send_buf_size` | 2 MB | `SO_SNDBUF` — kernel send buffer |
|
||||
|
||||
Linux internally doubles the requested value (to account for kernel
|
||||
bookkeeping overhead), so requesting 2 MB yields 4 MB actual buffer space.
|
||||
The kernel silently clamps to `net.core.rmem_max` if the request exceeds it.
|
||||
|
||||
**Host requirement**: `net.core.rmem_max` and `net.core.wmem_max` must be
|
||||
set to at least the requested buffer size on the host. For Docker containers,
|
||||
this must be configured on the Docker host (containers share the host kernel).
|
||||
Verify with:
|
||||
|
||||
```text
|
||||
sysctl net.core.rmem_max net.core.wmem_max
|
||||
```
|
||||
|
||||
Actual buffer sizes are logged at startup:
|
||||
|
||||
```text
|
||||
UDP transport started local_addr=0.0.0.0:4000 recv_buf=4194304 send_buf=4194304
|
||||
```
|
||||
|
||||
## Discovery
|
||||
|
||||
Discovery determines that a FIPS-capable endpoint is reachable at a given
|
||||
|
||||
@@ -128,8 +128,8 @@ Minimum frame: 37 bytes (empty body)
|
||||
| Type | Message | Description |
|
||||
| ---- | ------- | ----------- |
|
||||
| 0x00 | SessionDatagram | Encapsulated session-layer payload for forwarding |
|
||||
| 0x01 | SenderReport | MMP sender-side report (reserved) |
|
||||
| 0x02 | ReceiverReport | MMP receiver-side report (reserved) |
|
||||
| 0x01 | SenderReport | MMP sender-side metrics report (48 bytes) |
|
||||
| 0x02 | ReceiverReport | MMP receiver-side metrics report (68 bytes) |
|
||||
| 0x10 | TreeAnnounce | Spanning tree state announcement |
|
||||
| 0x20 | FilterAnnounce | Bloom filter reachability update |
|
||||
| 0x30 | LookupRequest | Coordinate discovery request |
|
||||
@@ -396,6 +396,50 @@ Orderly link teardown with reason code.
|
||||
| 0x07 | Timeout | Keepalive or stale detection timeout |
|
||||
| 0xFF | Other | Unspecified reason |
|
||||
|
||||
### SenderReport (0x01)
|
||||
|
||||
Sent by the frame sender to provide interval-based transmission statistics.
|
||||
|
||||
| Offset | Field | Size | Encoding |
|
||||
| ------ | ----- | ---- | -------- |
|
||||
| 0 | msg_type | 1 | `0x01` |
|
||||
| 1 | reserved | 3 | Zero |
|
||||
| 4 | interval_start_counter | 8 | u64 LE — first counter in this interval |
|
||||
| 12 | interval_end_counter | 8 | u64 LE — last counter in this interval |
|
||||
| 20 | interval_start_timestamp | 4 | u32 LE — timestamp at interval start |
|
||||
| 24 | interval_end_timestamp | 4 | u32 LE — timestamp at interval end |
|
||||
| 28 | interval_bytes_sent | 4 | u32 LE — payload bytes sent in interval |
|
||||
| 32 | cumulative_packets_sent | 8 | u64 LE — total packets sent on this link |
|
||||
| 40 | cumulative_bytes_sent | 8 | u64 LE — total bytes sent on this link |
|
||||
|
||||
**Total: 48 bytes.**
|
||||
|
||||
### ReceiverReport (0x02)
|
||||
|
||||
Sent by the frame receiver to provide loss, jitter, and timing feedback.
|
||||
|
||||
| Offset | Field | Size | Encoding |
|
||||
| ------ | ----- | ---- | -------- |
|
||||
| 0 | msg_type | 1 | `0x02` |
|
||||
| 1 | reserved | 3 | Zero |
|
||||
| 4 | highest_counter | 8 | u64 LE — highest counter value received |
|
||||
| 12 | cumulative_packets_recv | 8 | u64 LE — total packets received |
|
||||
| 20 | cumulative_bytes_recv | 8 | u64 LE — total bytes received |
|
||||
| 28 | timestamp_echo | 4 | u32 LE — echoed sender timestamp for RTT |
|
||||
| 32 | dwell_time | 2 | u16 LE — time between receive and echo (ms) |
|
||||
| 34 | max_burst_loss | 2 | u16 LE — largest loss burst in interval |
|
||||
| 36 | mean_burst_loss | 2 | u16 LE — mean burst length (u8.8 fixed-point) |
|
||||
| 38 | reserved | 2 | Zero |
|
||||
| 40 | jitter | 4 | u32 LE — interarrival jitter (microseconds) |
|
||||
| 44 | ecn_ce_count | 4 | u32 LE — cumulative ECN-CE marked packets |
|
||||
| 48 | owd_trend | 4 | i32 LE — one-way delay trend (µs/s, signed) |
|
||||
| 52 | burst_loss_count | 4 | u32 LE — number of loss bursts in interval |
|
||||
| 56 | cumulative_reorder_count | 4 | u32 LE — total reordered packets |
|
||||
| 60 | interval_packets_recv | 4 | u32 LE — packets received in interval |
|
||||
| 64 | interval_bytes_recv | 4 | u32 LE — bytes received in interval |
|
||||
|
||||
**Total: 68 bytes.**
|
||||
|
||||
## Session-Layer Message Formats
|
||||
|
||||
Session-layer messages are carried as the payload of a SessionDatagram (0x00).
|
||||
@@ -489,6 +533,11 @@ Message types 0x10-0x13 are carried inside the AEAD ciphertext (dispatched
|
||||
by the `msg_type` field in the encrypted inner header). Types 0x20-0x21 are
|
||||
plaintext error signals (U flag set, no encryption).
|
||||
|
||||
Session-layer SenderReport (0x11) and ReceiverReport (0x12) use the same
|
||||
body format as their link-layer counterparts (0x01 and 0x02). The msg_type
|
||||
byte in the body matches the link-layer value; dispatch to the correct layer
|
||||
happens at the session level based on the FSP message type.
|
||||
|
||||
### SessionSetup (phase 0x1)
|
||||
|
||||
Establishes a session and warms transit coordinate caches.
|
||||
@@ -527,6 +576,16 @@ Application data (typically IPv6 payload). This is the `msg_type` byte
|
||||
inside the encrypted inner header — there is no separate DataPacket struct.
|
||||
The body after the inner header is delivered directly to the TUN interface.
|
||||
|
||||
### PathMtuNotification (0x13)
|
||||
|
||||
Sent by the destination to report the observed forward-path MTU.
|
||||
|
||||
| Offset | Field | Size | Encoding |
|
||||
| ------ | ----- | ---- | -------- |
|
||||
| 0 | path_mtu | 2 | u16 LE — minimum MTU observed along the forward path |
|
||||
|
||||
**Total body: 2 bytes** (plus FSP common prefix + encrypted header + AEAD tag).
|
||||
|
||||
### CoordsRequired (0x20)
|
||||
|
||||
Plaintext error signal — transit node lacks coordinates for destination.
|
||||
|
||||
Reference in New Issue
Block a user