Implement non-blocking transport connect for connection-oriented transports

TCP (and future Tor) transports previously established connections
synchronously inside send(), blocking the node's RX event loop during
TCP handshake. This is particularly problematic for Tor where SOCKS5
circuit establishment can take 30-120 seconds.

Add a non-blocking connect path:
- ConnectionState enum in transport layer (None/Connecting/Connected/Failed)
- connect_async() on TcpTransport spawns background TCP connect task
- connection_state_sync() polls task completion, promotes to pool
- TransportHandle gains connect() and connection_state() dispatch methods
- Node tracks PendingConnect entries for connection-oriented transports
- initiate_connection() defers handshake for connection-oriented transports
- start_handshake() extracted as separate method for deferred invocation
- poll_pending_connects() in tick handler polls and completes handshakes
- Failed connects trigger retry via schedule_retry()

Connectionless transports (UDP, Ethernet) are unchanged — connect()
is a no-op and connection_state() always returns Connected.

The existing connect-on-send path in send_async() is preserved as
fallback for reconnection after connection drops.

811 tests pass (6 new), clippy clean.
This commit is contained in:
Johnathan Corgan
2026-03-13 03:21:21 +00:00
parent 1898a7c390
commit 1bfb58845a
5 changed files with 628 additions and 14 deletions
+1
View File
@@ -107,6 +107,7 @@ impl Node {
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
self.poll_pending_connects().await;
self.resend_pending_handshakes(now_ms).await;
self.resend_pending_rekeys(now_ms).await;
self.resend_pending_session_handshakes(now_ms).await;
+154 -10
View File
@@ -3,7 +3,7 @@
use super::{Node, NodeError, NodeState};
use crate::peer::PeerConnection;
use crate::protocol::{Disconnect, DisconnectReason};
use crate::transport::{packet_channel, Link, LinkDirection, TransportAddr, TransportId};
use crate::transport::{packet_channel, Link, LinkDirection, LinkId, TransportAddr, TransportId};
use crate::upper::tun::{run_tun_reader, shutdown_tun_interface, TunDevice, TunState};
use crate::node::wire::build_msg1;
use crate::{NodeAddr, PeerIdentity};
@@ -143,9 +143,14 @@ impl Node {
/// Initiate a connection to a peer on a specific transport and address.
///
/// Allocates a link, starts the Noise IK handshake, sends msg1, and
/// registers the connection for msg2 dispatch. Used by both static peer
/// config and transport discovery auto-connect paths.
/// For connectionless transports (UDP, Ethernet): allocates a link, starts
/// the Noise IK handshake, sends msg1, and registers the connection for
/// msg2 dispatch.
///
/// For connection-oriented transports (TCP, Tor): allocates a link and
/// starts a non-blocking transport connect. The handshake is deferred
/// until the transport connection is established — the tick handler
/// polls `connection_state()` and initiates the handshake when ready.
pub(super) async fn initiate_connection(
&mut self,
transport_id: TransportId,
@@ -154,15 +159,14 @@ impl Node {
) -> Result<(), NodeError> {
let peer_node_addr = *peer_identity.node_addr();
let is_connection_oriented = self.transports.get(&transport_id)
.map(|t| t.transport_type().connection_oriented)
.unwrap_or(false);
// Allocate link ID and create link
let link_id = self.allocate_link_id();
// Use Link::new() for connection-oriented transports (Connecting state)
// and Link::connectionless() for connectionless transports (Connected state)
let link = if self.transports.get(&transport_id)
.map(|t| t.transport_type().connection_oriented)
.unwrap_or(false)
{
let link = if is_connection_oriented {
Link::new(
link_id,
transport_id,
@@ -186,6 +190,53 @@ impl Node {
self.addr_to_link
.insert((transport_id, remote_addr.clone()), link_id);
if is_connection_oriented {
// Connection-oriented: start non-blocking connect, defer handshake
if let Some(transport) = self.transports.get(&transport_id) {
match transport.connect(&remote_addr).await {
Ok(()) => {
debug!(
peer = %self.peer_display_name(&peer_node_addr),
transport_id = %transport_id,
remote_addr = %remote_addr,
link_id = %link_id,
"Transport connect initiated (non-blocking)"
);
self.pending_connects.push(super::PendingConnect {
link_id,
transport_id,
remote_addr,
peer_identity,
});
}
Err(e) => {
// Clean up link
self.links.remove(&link_id);
self.addr_to_link.remove(&(transport_id, remote_addr));
return Err(NodeError::TransportError(e.to_string()));
}
}
}
Ok(())
} else {
// Connectionless: proceed with immediate handshake
self.start_handshake(link_id, transport_id, remote_addr, peer_identity).await
}
}
/// Start the Noise handshake on a link and send msg1.
///
/// Called immediately for connectionless transports, or after the
/// transport connection is established for connection-oriented transports.
pub(super) async fn start_handshake(
&mut self,
link_id: LinkId,
transport_id: TransportId,
remote_addr: TransportAddr,
peer_identity: PeerIdentity,
) -> Result<(), NodeError> {
let peer_node_addr = *peer_identity.node_addr();
// Create connection in handshake phase (outbound knows expected identity)
let current_time_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
@@ -336,6 +387,99 @@ impl Node {
}
}
/// Poll pending transport connects and initiate handshakes for ready ones.
///
/// Called from the tick handler. For each pending connect, queries the
/// transport's connection state. When a connection is established,
/// marks the link as Connected and starts the Noise handshake.
/// Failed connections are cleaned up and scheduled for retry.
pub(super) async fn poll_pending_connects(&mut self) {
if self.pending_connects.is_empty() {
return;
}
let mut completed = Vec::new();
for (i, pending) in self.pending_connects.iter().enumerate() {
let state = if let Some(transport) = self.transports.get(&pending.transport_id) {
transport.connection_state(&pending.remote_addr)
} else {
crate::transport::ConnectionState::Failed("transport removed".into())
};
match state {
crate::transport::ConnectionState::Connected => {
completed.push((i, true, None));
}
crate::transport::ConnectionState::Failed(reason) => {
completed.push((i, false, Some(reason)));
}
crate::transport::ConnectionState::Connecting => {
// Still in progress, check on next tick
}
crate::transport::ConnectionState::None => {
// Shouldn't happen — treat as failure
completed.push((i, false, Some("no connection attempt found".into())));
}
}
}
// Process completions in reverse order to preserve indices
for (i, success, reason) in completed.into_iter().rev() {
let pending = self.pending_connects.remove(i);
if success {
// Mark link as Connected
if let Some(link) = self.links.get_mut(&pending.link_id) {
link.set_connected();
}
debug!(
peer = %self.peer_display_name(pending.peer_identity.node_addr()),
transport_id = %pending.transport_id,
remote_addr = %pending.remote_addr,
link_id = %pending.link_id,
"Transport connected, starting handshake"
);
// Start the handshake now that the transport is connected
if let Err(e) = self.start_handshake(
pending.link_id,
pending.transport_id,
pending.remote_addr.clone(),
pending.peer_identity,
).await {
warn!(
link_id = %pending.link_id,
error = %e,
"Failed to start handshake after transport connect"
);
// Clean up link on handshake failure
self.remove_link(&pending.link_id);
}
} else {
let reason = reason.unwrap_or_default();
warn!(
peer = %self.peer_display_name(pending.peer_identity.node_addr()),
transport_id = %pending.transport_id,
remote_addr = %pending.remote_addr,
link_id = %pending.link_id,
reason = %reason,
"Transport connect failed"
);
// Clean up link and schedule retry
self.remove_link(&pending.link_id);
self.links.remove(&pending.link_id);
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
self.schedule_retry(*pending.peer_identity.node_addr(), now_ms, false);
}
}
}
// === State Transitions ===
/// Start the node.
+28
View File
@@ -119,6 +119,9 @@ pub enum NodeError {
#[error("handshake failed: {0}")]
HandshakeFailed(String),
#[error("transport error: {0}")]
TransportError(String),
}
/// Node operational state.
@@ -208,6 +211,22 @@ struct TransportDropState {
dropping: bool,
}
/// State for a link waiting for transport-level connection establishment.
///
/// For connection-oriented transports (TCP, Tor), the transport connect runs
/// asynchronously. This struct holds the data needed to complete the handshake
/// once the connection is ready.
struct PendingConnect {
/// The link that was created for this connection.
link_id: LinkId,
/// Which transport is being used.
transport_id: TransportId,
/// The remote address being connected to.
remote_addr: TransportAddr,
/// The peer identity (for handshake initiation).
peer_identity: PeerIdentity,
}
/// A running FIPS node instance.
///
/// This is the top-level container holding all node state.
@@ -363,6 +382,13 @@ pub struct Node {
/// Rate limiter for source-side CoordsRequired/PathBroken responses.
coords_response_rate_limiter: RoutingErrorRateLimiter,
// === Pending Transport Connects ===
/// Links waiting for transport-level connection establishment before
/// sending handshake msg1. For connection-oriented transports (TCP, Tor),
/// the transport connect runs in the background; the tick handler polls
/// connection_state() and initiates the handshake when connected.
pending_connects: Vec<PendingConnect>,
// === Connection Retry ===
/// Retry state for peers whose outbound connections have failed.
/// Keyed by NodeAddr. Entries are created when a handshake times out
@@ -499,6 +525,7 @@ impl Node {
coords_response_rate_limiter: RoutingErrorRateLimiter::with_interval(
std::time::Duration::from_millis(coords_response_interval_ms),
),
pending_connects: Vec::new(),
retry_pending: HashMap::new(),
last_parent_reeval: None,
last_congestion_log: None,
@@ -601,6 +628,7 @@ impl Node {
coords_response_rate_limiter: RoutingErrorRateLimiter::with_interval(
std::time::Duration::from_millis(coords_response_interval_ms),
),
pending_connects: Vec::new(),
retry_pending: HashMap::new(),
last_parent_reeval: None,
last_congestion_log: None,