Update design docs for non-blocking transport connect and cleanup stale references

- Rewrite TCP "Connect-on-Send" section to document non-blocking connect
  model (ConnectingPool, PendingConnect, poll_pending_connects)
- Add connect() and connection_state() to Transport trait surface
- Expand Connection Lifecycle section with ConnectionState enum
- Remove phantom TCP socks5_proxy field (removed from code, superseded
  by TorConfig)
- Fix "future Tor transport" references (stream reader already shared)
- Replace misleading "tor:" named TCP instance example
- Update fips-intro.md implementation status (TCP and Ethernet are
  implemented, not "under active design")
This commit is contained in:
Johnathan Corgan
2026-03-13 16:37:00 +00:00
parent 35ff4a5d0a
commit d873d0e00e
3 changed files with 58 additions and 32 deletions
+3 -5
View File
@@ -359,7 +359,6 @@ overhead.
| `transports.tcp.recv_buf_size` | usize | `2097152` | Socket receive buffer size in bytes (2 MB) | | `transports.tcp.recv_buf_size` | usize | `2097152` | Socket receive buffer size in bytes (2 MB) |
| `transports.tcp.send_buf_size` | usize | `2097152` | Socket send buffer size in bytes (2 MB) | | `transports.tcp.send_buf_size` | usize | `2097152` | Socket send buffer size in bytes (2 MB) |
| `transports.tcp.max_inbound_connections` | usize | `256` | Maximum simultaneous inbound connections | | `transports.tcp.max_inbound_connections` | usize | `256` | Maximum simultaneous inbound connections |
| `transports.tcp.socks5_proxy` | string | *(none)* | SOCKS5 proxy for outbound connections (implementation deferred) |
**Named instances.** Like other transports, multiple TCP instances can **Named instances.** Like other transports, multiple TCP instances can
be configured with named sub-keys: be configured with named sub-keys:
@@ -369,9 +368,9 @@ transports:
tcp: tcp:
public: public:
bind_addr: "0.0.0.0:443" bind_addr: "0.0.0.0:443"
tor: internal:
socks5_proxy: "127.0.0.1:9050" bind_addr: "10.0.0.1:8443"
connect_timeout_ms: 30000 max_inbound_connections: 64
``` ```
## Peers (`peers[]`) ## Peers (`peers[]`)
@@ -590,7 +589,6 @@ transports:
# recv_buf_size: 2097152 # 2 MB # recv_buf_size: 2097152 # 2 MB
# send_buf_size: 2097152 # 2 MB # send_buf_size: 2097152 # 2 MB
# max_inbound_connections: 256 # resource protection limit # max_inbound_connections: 256 # resource protection limit
# socks5_proxy: null # SOCKS5 for outbound (deferred)
peers: # static peer list peers: # static peer list
# - npub: "npub1..." # - npub: "npub1..."
+2 -3
View File
@@ -524,9 +524,8 @@ forwarding, a publicly addressed peer, or relay through other mesh nodes.
UDP hole punching and relay-assisted NAT traversal are potential future UDP hole punching and relay-assisted NAT traversal are potential future
mechanisms but are not part of the current design. mechanisms but are not part of the current design.
> **Implementation status**: UDP/IP is implemented. Ethernet and Bluetooth > **Implementation status**: UDP/IP, TCP/IP, and Ethernet transports are
> transports are under active design and development. All others are future > implemented. All others are future directions.
> directions.
See [fips-transport-layer.md](fips-transport-layer.md) for the full transport See [fips-transport-layer.md](fips-transport-layer.md) for the full transport
layer specification. layer specification.
+53 -24
View File
@@ -69,10 +69,30 @@ forwarding and LookupResponse transit annotation.
For connection-oriented transports, manage the underlying connection: TCP For connection-oriented transports, manage the underlying connection: TCP
handshake, Tor circuit establishment, Bluetooth pairing. FMP cannot begin handshake, Tor circuit establishment, Bluetooth pairing. FMP cannot begin
the Noise IK link handshake until the transport-layer connection is established. the Noise IK link handshake until the transport-layer connection is
established.
Connectionless transports (UDP, raw Ethernet) skip this — datagrams can flow Connection-oriented transports expose a non-blocking connect interface.
immediately to any reachable address. `connect(addr)` initiates the connection in a background task and returns
immediately. `connection_state(addr)` reports the current status:
```text
ConnectionState {
None No connection attempt in progress
Connecting Background task running
Connected Ready for send()
Failed(msg) Error message from failed attempt
}
```
Connectionless transports (UDP, raw Ethernet) return `Connected`
immediately — no async work needed.
At the node level, `PendingConnect` entries track links waiting for
transport connection. `poll_pending_connects()` runs each tick, checks
`connection_state()`, and calls `start_handshake()` on success or
`schedule_retry()` on failure. This decouples transport-layer connection
(which may take seconds for Tor circuits) from the FMP event loop.
### Discovery (Optional) ### Discovery (Optional)
@@ -326,8 +346,7 @@ Ethernet transport started name=eth0 interface=eth0 mac=aa:bb:cc:dd:ee:ff mtu=14
## TCP/IP: Firewall Traversal Transport ## TCP/IP: Firewall Traversal Transport
For networks where UDP is blocked but TCP port 443 is open, the TCP For networks where UDP is blocked but TCP port 443 is open, the TCP
transport provides an alternative path. It also serves as the foundation transport provides an alternative path.
for the future Tor transport.
FIPS protocols (FMP, FSP, MMP) are all unreliable datagrams. Running them FIPS protocols (FMP, FSP, MMP) are all unreliable datagrams. Running them
over TCP introduces head-of-line blocking, which adds latency jitter. MMP over TCP introduces head-of-line blocking, which adds latency jitter. MMP
@@ -338,8 +357,10 @@ penalizes TCP links (higher SRTT leads to higher link cost). ETX will be
### Architecture ### Architecture
Unlike UDP (one socket serves all peers), TCP requires one `TcpStream` per Unlike UDP (one socket serves all peers), TCP requires one `TcpStream` per
peer. The transport maintains a connection pool (`HashMap<TransportAddr, peer. The transport maintains two pools: a `ConnectingPool` for background
TcpConnection>`) plus an optional `TcpListener` for inbound connections. connection attempts in progress, and an established connection pool
(`HashMap<TransportAddr, TcpConnection>`) for active connections, plus an
optional `TcpListener` for inbound connections.
| Property | Value | | Property | Value |
| -------- | ----- | | -------- | ----- |
@@ -347,7 +368,7 @@ TcpConnection>`) plus an optional `TcpListener` for inbound connections.
| Default MTU | 1400 bytes | | Default MTU | 1400 bytes |
| Per-link MTU | Derived from `TCP_MAXSEG` socket option | | Per-link MTU | Derived from `TCP_MAXSEG` socket option |
| Framing | FMP header-based (zero overhead) | | Framing | FMP header-based (zero overhead) |
| Connection model | Connect-on-send, optional listener | | Connection model | Non-blocking connect, connect-on-send fallback, optional listener |
| Platform | Cross-platform (no `#[cfg]` gates) | | Platform | Cross-platform (no `#[cfg]` gates) |
### FMP Header-Based Framing ### FMP Header-Based Framing
@@ -364,29 +385,36 @@ packet boundaries:
This provides zero framing overhead and built-in phase validation. The This provides zero framing overhead and built-in phase validation. The
stream reader is implemented in a separate module (`stream.rs`) for reuse stream reader is implemented in a separate module (`stream.rs`) for reuse
by the future Tor transport. by the Tor transport.
### Connect-on-Send ### Connection Establishment
When `send(addr, data)` is called with no existing connection: TCP connections use a non-blocking connect model. When FMP needs to reach
a configured peer address, the node calls `connect(addr)` on the transport,
which spawns a background tokio task to perform the TCP handshake and socket
configuration (TCP_NODELAY, keepalive, buffer sizes, TCP_MAXSEG query). The
call returns immediately without blocking the event loop.
1. Connect with configurable timeout (default 5s) The node tracks each pending connection in a `PendingConnect` entry. On
2. Configure socket: `TCP_NODELAY`, keepalive, buffer sizes every tick, `poll_pending_connects()` calls `connection_state(addr)` to
3. Read `TCP_MAXSEG` for per-connection MTU check progress. When the transport reports `Connected`, the completed
4. Split stream into read/write halves connection is promoted to the established pool (stream split into
5. Spawn per-connection receive task read/write halves, per-connection receive task spawned), and the node
6. Store connection in pool initiates the Noise IK link handshake. If the transport reports `Failed`,
7. Write packet directly to stream the node schedules a retry with exponential backoff.
If connect fails, return error. The node's handshake retry mechanism As a fallback, `send(addr, data)` still performs synchronous
handles re-attempts. connect-on-send if no connection exists — this handles the case where a
send arrives before the node-level connect path runs. The non-blocking
path is the primary mechanism for configured peers.
### Session Independence ### Session Independence
TCP connection loss does **not** tear down the FIPS peer. Noise keys, MMP TCP connection loss does **not** tear down the FIPS peer. Noise keys, MMP
state, and FSP sessions are bound to the peer's npub, not the TCP state, and FSP sessions are bound to the peer's npub, not the TCP
connection. The transport reconnects transparently on the next send via connection. The transport reconnects transparently via the non-blocking
connect-on-send. MMP liveness timeout is the sole authority for peer death. connect path or connect-on-send fallback. MMP liveness timeout is the sole
authority for peer death.
### Connection Deduplication ### Connection Deduplication
@@ -408,7 +436,6 @@ transports:
recv_buf_size: 2097152 # SO_RCVBUF (2 MB) recv_buf_size: 2097152 # SO_RCVBUF (2 MB)
send_buf_size: 2097152 # SO_SNDBUF (2 MB) send_buf_size: 2097152 # SO_SNDBUF (2 MB)
max_inbound_connections: 256 # Resource protection limit max_inbound_connections: 256 # Resource protection limit
socks5_proxy: "127.0.0.1:9050" # SOCKS5 for outbound (deferred)
``` ```
If `bind_addr` is configured, the transport accepts inbound connections. If `bind_addr` is configured, the transport accepts inbound connections.
@@ -496,6 +523,8 @@ link_mtu(addr) → u16 Per-link MTU (defaults to mtu())
start() → lifecycle Bring transport up (bind socket, open device) start() → lifecycle Bring transport up (bind socket, open device)
stop() → lifecycle Bring transport down stop() → lifecycle Bring transport down
send(addr, data) → delivery Send datagram to transport address send(addr, data) → delivery Send datagram to transport address
connect(addr) → () Initiate non-blocking connection (connection-oriented only)
connection_state(addr)→ ConnectionState Poll connection status (None/Connecting/Connected/Failed)
close_connection(addr)→ () Close a specific connection (no-op for connectionless) close_connection(addr)→ () Close a specific connection (no-op for connectionless)
congestion() → TransportCongestion Local congestion indicators (optional) congestion() → TransportCongestion Local congestion indicators (optional)
discover() → Vec<DiscoveredPeer> Report discovered FIPS endpoints (optional) discover() → Vec<DiscoveredPeer> Report discovered FIPS endpoints (optional)
@@ -579,7 +608,7 @@ transitions through `Starting` to `Up` (operational). `stop()` moves to
| Transport | Status | Notes | | Transport | Status | Notes |
| --------- | ------ | ----- | | --------- | ------ | ----- |
| UDP/IP | **Implemented** | Primary transport, AsyncFd/recvmsg, SO_RXQ_OVFL kernel drop detection | | UDP/IP | **Implemented** | Primary transport, AsyncFd/recvmsg, SO_RXQ_OVFL kernel drop detection |
| TCP/IP | **Implemented** | FMP header-based framing, connect-on-send, per-connection MSS MTU | | TCP/IP | **Implemented** | FMP header-based framing, non-blocking connect, per-connection MSS MTU |
| Ethernet | **Implemented** | AF_PACKET SOCK_DGRAM, EtherType 0x2121, beacon discovery, Linux only | | Ethernet | **Implemented** | AF_PACKET SOCK_DGRAM, EtherType 0x2121, beacon discovery, Linux only |
| WiFi | Future direction | Infrastructure mode = Ethernet driver | | WiFi | Future direction | Infrastructure mode = Ethernet driver |
| Tor | Future direction | High latency, .onion addressing | | Tor | Future direction | High latency, .onion addressing |