`AndroidIo` captured the injected bridge `Arc` at construction, and the BLE
transport was only built if a bridge already existed when the node started.
Both facts are wrong for the platform this backend serves: the radio is owned
by an Android foreground service whose lifetime is independent of the node's —
it can start *after* the node, and it mints a fresh bridge every time it
starts. So the only way for a running node to adopt a radio was to stop and
rebuild it, which drops every peer, every session, and every route with it.
Turning Bluetooth on took the whole mesh down for as long as re-handshaking
took, which is not what enabling a transport should cost.
`AndroidIo` now resolves the process-wide bridge per operation, so a fresh
bridge is picked up in place, and the transport is constructed whether or not
one exists yet. Operations attempted with no radio present return a transport
error instead of being unreachable, and recover on their own once a radio
appears. `AndroidIo::new` is kept for tests that drive a specific bridge.
Live streams still hold the radio they were opened on rather than migrating to
a new one — a channel belongs to the socket that created it, and those die with
the radio that owned them.
The UDP transport opens one wildcard socket and selects the egress path per
destination address. That assumes the host routes by destination alone —
true on an ordinary Unix box, not true everywhere. On hosts that associate
each socket with one "network" and steer traffic by that association, a peer
reachable only over a secondary network is unreachable in a way FIPS cannot
see or fix: the address is well-formed, the send succeeds, the peer receives
our handshake and replies, and the reply is discarded by the host before it
reaches our socket. The link just retries msg1 forever.
Nothing in FIPS can correct that, because the correction is a socket option
on a socket the transport keeps private. So hand the embedder the fd:
let rx = node.enable_app_owned_udp_fd(); // before start()
node.start().await?;
let fd = rx.recv()?; // once the transport is up
This follows the existing `enable_app_owned_tun` / `enable_app_owned_dns`
contract — call after `Node::new` and before `start()`, get a channel back —
and stays deliberately narrow: FIPS keeps owning the socket, and the fd
carries no promise beyond "this is the transport's socket, now open". If no
UDP transport is configured, or it fails to start, nothing is ever sent.
Also plumbs `raw_fd()` through `TransportHandle` and `UdpTransport`. Unix
only, since `RawFd` is a unix concept; non-UDP transports return None.
A mesh-edge node (only direct BLE / Wi-Fi Aware / LAN peers, sparse bloom
filters) could not route to its own directly-connected neighbours: a
neighbour is never in another peer's bloom filter, so discovery was gated
off and find_next_hop had no coordinates. Traffic to such a peer failed
with "no route to destination" even with an established session.
- discovery: don't suppress a lookup on a bloom miss unless we have >2
peers (trustworthy blooms); and when no tree peer advertises the target,
flood the LookupRequest to every sendable peer — including the target
itself, which answers a lookup for its own address, so the querier learns
its coordinates and can route.
- session: when an established-session send hits no-route, trigger discovery
(previously only session *initiation* did), so a coordless session — e.g.
a platform-pushed peer whose Noise handshake came up before tree
discovery — self-warms and the app's retransmit succeeds.
- node: add Node::enable_app_owned_dns() — returns the DnsIdentityTx an
embedder (Android VpnService pump) uses to register identities it resolved
itself, giving the same identity-cache/route-warming the built-in DNS
responder provides.
When a peer is reachable over more than one transport at once, prefer the
faster one and hold it. roam_current_addr upgrades eagerly to a higher-
preference transport (BLE -> Wi-Fi Aware / UDP) but falls back to a lower
one only after the preferred link has been silent past a hysteresis
window (~2 heartbeat intervals), so a stray BLE keepalive can't drag an
active Aware session back to BLE. poll_transport_discovery additionally
stops re-probing an alternate path strictly worse than the one the peer
is already on, so BLE rediscovery no longer keeps grabbing a peer that
belongs on Aware. Equal preferences reduce to plain last-authenticated-
packet-wins roaming, so single-transport deployments are unaffected.
Add a transport-agnostic seam for an embedding platform (e.g. an Android
Wi-Fi Aware radio) to push "peer npub reachable at addr over transport T"
events into a running node — the generalization of the UDP-only LAN mDNS
drain. A process-global queue (fips::discovery::platform) is drained each
tick by poll_platform_discovery, which selects the transport family-aware
(an IPv6 target picks an IPv6 socket) and initiates a Noise IK handshake;
the pushed npub is only a routing hint, the handshake authenticates. An
event for an already-active peer starts an alternate-path handshake, and
a Lost event closes the pooled connection.
sockaddr_to_socket_addr dropped the IPv6 scope id when converting an
inbound source address, so a reply to a link-local peer (fe80::/10) had
no interface scope and could not be routed. This stalled the Noise
handshake reply (msg2) over a Wi-Fi Aware NDP interface, where every
address is link-local: msg1 reached the peer (sent to a scoped address)
but msg2 never routed back. Rebuild the SocketAddrV6 with sin6_scope_id;
non-link-local sources carry scope 0 and are unaffected.
BLE delivery was unreliable on every backend except BlueZ. The receive
path assumed one `recv()` returned exactly one whole FIPS packet, which
only holds for BlueZ's SOCK_SEQPACKET (boundary-preserving). Android's
BluetoothSocket input stream and macOS CoreBluetooth are byte-stream
oriented: a read can return a fragment of a packet or several packets
coalesced. Under the old loop a fragment was shipped up as a runt packet
(rejected by FMP/Noise) and a coalesced tail was silently truncated and
dropped — so packets were lost and the transport thrashed.
Recover packet boundaries from the byte stream instead of trusting the
OS to preserve them. FIPS packets are self-delimiting via the 4-byte FMP
common prefix, so this reuses the exact length-prefixed framer TCP already
uses (`tcp::stream::read_fmp_packet`) — kept transport-agnostic for this
reason. A new `BleStreamRead` adapter turns the datagram-shaped `BleStream`
into the `AsyncRead` that framer expects, buffering leftover bytes across
reads. The recv future owns its scratch buffer and returns an owned Vec so
it is `'static` and storable across `poll_read` calls.
One reader is threaded through the pubkey exchange and the receive loop per
connection, so bytes a peer coalesces after the 33-byte pubkey stay buffered
rather than being lost at the handoff. The pubkey exchange now reads via
`read_exact` (reassembles a fragmented pubkey) instead of a brittle single
`recv()` with an exact-length check.
On BlueZ this is a transparent pass-through (one recv already equals one
packet); on stream backends it reassembles. Either way the layer above sees
one whole packet per read, identically on every platform.
Also bump the Android outbound SEND_QUEUE_CAP from 8 to 32. Swept against
the peer speedtest: 8 starved the radio's connection events (~half
throughput), 64 bufferbloated TCP, 32 was best (~200/500 kbps up/down). The
sweep was noisy and non-monotonic — run-to-run BLE variance (RF, 2M PHY /
connection-priority grants) rivals the knob's effect — so 32 is the
best-observed value pending re-validation with PHY/interval instrumentation,
not a proven optimum.
Import ordering and multi-line wrapping rustfmt would enforce; android_io
is cfg(target_os = "android") so it never compiled on the host CI matrix
and the drift went unnoticed until the mobile build was wired up.
Android lands as an embedded library: the host app owns the TUN (e.g. an
Android VpnService) and FIPS does no system-TUN ops. Document where the
docs enumerate platforms as a set.
- README transport matrix: add an Android column (UDP/TCP/BLE supported;
Ethernet has no raw sockets; Tor/Nym need an external proxy not run on
Android). Reword the intro to distinguish standalone-daemon hosts from
the embedded-library Android target.
- transport-layer status: BLE is now implemented on Linux/glibc and
Android (Linux via BlueZ, Android via the embedder radio bridge).
- set-up-bluetooth-peer how-to: add Android to the BLE platform table.
The per-stream outbound byte queue was 256 deep (~384 KB) on a link whose
bandwidth-delay product is ~1 packet. FSP/MMP filled it, RTT ballooned to several
seconds, and TCP above couldn't ramp. A shallow tail-drop queue is no better — it
sheds packets and collapses TCP throughput.
Give the outbound queue its own shallow cap (SEND_QUEUE_CAP=8) and make
BleStream::send backpressure — await a free slot rather than try_send-dropping —
so flow control propagates up through FSP/MMP to TCP. On-device this cut RTT from
~5s to ~1.2s with throughput preserved and 0% loss.
Per-peer PSM discovery keys the learned PSM by BLE address, but RPAs rotate
between the scan that learns a peer's PSM and the dial, so resolve() missed and
fell back to the legacy default 0x0085 — which no node listens on. Every outbound
L2CAP connect was rejected and the link ran inbound-only at a fraction of its rate.
A node advertises exactly one OS-assigned listener PSM, so on an exact-address
miss, dial the most-recently-learned PSM instead of the fixed default; a wrong
guess only costs a dial-retry.
Node::enable_app_owned_tun() lets an embedder that owns the TUN fd (e.g. an
Android VpnService) exchange IPv6 packet bytes with FIPS over channels instead of
FIPS creating a system TUN device. It returns (app_outbound_tx, app_inbound_rx):
the embedder pushes packets read from its fd into the outbound sender (app ->
mesh) and pulls packets destined for its fd from the inbound receiver (mesh ->
app). Called after Node::new and before start(), mirroring control_read_handle().
start() now gates TunDevice::create on tun_tx being unset, so when the app-owned
channels are pre-installed it skips system-TUN creation and does no system-TUN
ops. The inbound IPv6-shim delivery already writes to tun_tx, and run_rx_loop
already drains tun_outbound_rx into handle_tun_outbound, so both directions reuse
the existing wiring.
Packets entering via app_outbound_tx bypass the system-TUN reader's
handle_tun_packet, so the embedder must push only fd::/8-destined packets (FIPS no
longer filters the destination) and clamp TCP MSS on outbound SYNs; the rustdoc
and the IPv6-adapter design doc spell this out.
Tests: app_owned_tun_seam_wires_channels covers the channel round-trip and the
Active state; start_skips_system_tun_when_app_owned runs start() and asserts no
named system device is created (tun_name stays unset).
deliver_scan now carries RSSI as well, and records the latest advert per address
(PSM + RSSI) into a map exposed via advert_views() -> Vec<AdvertView>. Cleared
each scan cycle (addresses rotate with MAC privacy). This is the radio-level
view, distinct from the node's mesh-level peer table.
- next_send: clone the per-channel receiver out before blocking (no longer holds
the channels lock across recv_timeout), and treat a dropped sender
(Disconnected) as closed so the Kotlin writer thread exits when the stream is
gone.
- channel_open: also reports a closed (dropped-stream) channel, not just a
removed one — so writers stop promptly.
- set_android_ble_bridge: replaceable (Mutex<Option>) so a stop→start cycle
re-injects a fresh bridge for the rebuilt node.
Expose ControlReadHandle (was pub(crate)) + make Node::control_read_handle()
public, and add ControlReadHandle::peer_views() -> Vec<PeerView>
{ node_addr_hex, npub, connected }, read lock-free from the tick-published stats
snapshot (peer_meta). This lets an embedder run run_rx_loop on a background task
(which exclusively borrows &mut Node) and still poll live peer state from a
cloned handle — no &Node access needed. Used by the Myco app's developer UI.
The Android BLE radio lives in Kotlin, so AndroidIo/Stream/Acceptor/Scanner
implement BleIo by delegating to AndroidBleBridge — the channel machinery shared
with the JNI layer in the embedder. The AndroidRadio trait is the object-safe
command surface Kotlin implements (listen/connect/advertise/scan/close).
- Inbound bytes/events are pushed non-blocking into tokio channels
(deliver_recv/inbound/scan/connect_result); outbound bytes are pulled by a
per-channel Kotlin writer thread via next_send. BleStream::send never calls
JNI — the byte hot path is pure channel push.
- Per-peer PSM: connect() substitutes the learned PSM; advertising emits the
OS-assigned listener PSM (16-bit LE service-data).
- Wired as DefaultBleTransport on Android, plus a node construction arm that
reads the embedder-injected bridge (set_android_ble_bridge).
Pure Rust (no JNI here — that's in myco-core), so the channel logic unit-tests
on the host with a mock radio. Cross-compiles clean for arm64-android.
Foundation for "BLE v2" universal per-peer PSM discovery, shared by every
BleIo backend.
- ble/psm.rs: a 16-bit little-endian service-data PSM codec + a short-lived
BleAddr->PSM map (learn/lookup/resolve/clear), with unit tests. Every node
advertises its OS-assigned listener PSM and every dialer reads a peer's
advertised PSM before connect(), replacing the fixed DEFAULT_PSM (0x0085)
assumption that only BlueZ could satisfy (Android/macOS get OS-assigned
listener PSMs).
- Un-gate the BLE transport module from linux-only to a new `ble_available`
cfg (linux/macos/android). The pool/discovery/psm logic and the generic
BleTransport<I> are platform-agnostic; only the concrete BleIo backend is
platform-specific (BluerIo on linux-glibc, else the MockBleIo fallback).
The macOS BluestIo and Android AndroidIo backends, and the per-backend
advertise/scan/connect wiring of the PSM map, land in follow-ups.
A plain `cargo build` now compiles correctly for every target with no
flags: the compiler selects platform code by `target_os`, enabling what
each platform supports and disabling what it doesn't. Android no longer
needs `--no-default-features`.
- ethernet: gated `any(target_os = "linux", target_os = "macos")` (raw
AF_PACKET / BPF). Android is `target_os = "android"` — not "linux" — so
the raw-socket transport self-excludes there, as it already did on Windows.
- system-tun: real platform ops gated per `target_os = "linux"`/`"macos"`;
Windows keeps its own path. Android gets a no-op stub — the TUN is
app-owned by the embedder (e.g. an Android VpnService).
- dns: ipi6_ifindex is i32 on Android, u32 on macOS — cast.
- node: gate the unix-only ESTABLISHED_HEADER_SIZE import so the Windows
build is warning-clean.
No Cargo features are introduced; desktop builds are unchanged.
The shared fips.yaml ships ethernet.wan.interface: "eth0", the default
WAN port on OpenWrt 24 and earlier. OpenWrt 25 (DSA) boards, which the
.apk package targets, name the WAN port "wan" instead. Rewrite the
staged copy at build time so the as-installed config binds the Ethernet
transport to the right port out of the box, without maintaining a second
copy of the config file. The .ipk package keeps "eth0".
Update the openwrt README to document eth0 (24) vs wan (25/DSA) and add
a CHANGELOG entry.
Add OpenWrt .apk packaging for OpenWrt 25+, where apk-tools is the
mandatory package manager. The existing .ipk continues to cover OpenWrt
24.x and earlier. Built SDK-free like the .ipk: it reuses the
cargo-zigbuild cross-compile and the shared installed-filesystem payload
under openwrt-ipk/files, and assembles the ADB container with the
official `apk mkpkg` applet from apk-tools 3.0.5 built from source, so no
OpenWrt SDK image is needed.
The package-openwrt workflow is refactored so a single compile-binaries
job cross-compiles and strips each arch once; both the .ipk and .apk
packagers consume the binaries via a new --bin-dir flag instead of each
recompiling. A build-apk job (aarch64, x86_64) builds apk-tools,
packages, and structurally verifies the .apk with `apk adbdump`. Releases
now publish .apk artifacts and checksums alongside .ipk; the release
download is scoped to fips_* so the shared raw-binary artifacts are not
swept into the published release. apk-version.sh maps a release tag or
commit height to an apk-tools-valid version, covered by a case-table test.
Packages are unsigned, installed with `apk add --allow-untrusted`,
matching the .ipk posture.
Also fix the OpenWrt control socket: the init script now pre-creates
/run/fips before starting the daemon, the procd equivalent of the systemd
unit's RuntimeDirectory=fips. Without it, on a fresh boot the daemon
resolves its control socket to /tmp (since /run/fips does not yet exist),
while fips-gateway later creates /run/fips for its own gateway.sock,
leaving fipsctl/fipstop resolving a /run/fips/control.sock the daemon
never bound.
Self-addressed TCP/UDP connections to a node's own <npub>.fips address
half-opened and hung on macOS. macOS routes self-traffic as loopback (a
LOCAL route via lo0), which defers the transport TX checksum, but the
point-to-point utun then egresses the packet into the daemon with only
the pseudo-header partial checksum present. The hairpin path added in
9a9e90a re-injected these verbatim, so the local stack dropped every
segment whose checksum MSS clamping didn't happen to rewrite: the
SYN/SYN-ACK got through (clamping recomputes them) but the bare ACK,
data, and FIN were dropped for a bad checksum, leaving the listener
stuck in SYN_RCVD.
Recompute the TCP/UDP checksum for self-addressed packets on the hairpin
path before re-injection, completing the self-delivery 9a9e90a started
(which only covered ICMP and the TCP handshake). Linux is unaffected: it
loops self-traffic via lo before the TUN, so the hairpin branch never
fires and checksums are already valid.
Confirmed on macOS: a self-connect that previously timed out now
completes in ~9ms with payload delivered.
A packet destined for our own mesh address reached the TUN reader on
macOS (utun egresses self-traffic into the daemon despite the lo0 host
route) and was pushed onto the mesh outbound path, where it was dropped
for lack of a session/route to self. Hairpin self-addressed packets back
to the TUN writer instead, so ping6 and connections to our own
<npub>.fips address are delivered locally.
On Linux the kernel already loops self-traffic via `lo` before it reaches
the TUN, so the branch never fires there; the check is kept unconditional
as a daemon-level delivery invariant and to keep it covered by Linux CI.
Before bf77ece (Fix DNS responder silent-drop on systemd-resolved
deployments, 2026-04-29) the daemon defaulted dns.bind_addr to "::"
(wildcard, accepted v4 traffic too), so the macOS pkg's resolver shim
of `nameserver 127.0.0.1` reached the daemon fine over v4 loopback.
That commit tightened the default to "::1" — IPv6 loopback only,
which on Linux/macOS does not accept v4-mapped traffic — to defuse a
mesh-interface filter / IPV6_PKTINFO bug that was silently dropping
.fips queries on systemd-resolved hosts. The Linux side was updated
in the same commit: fips-dns-setup now writes [::1]:5354 in every
backend, and the gateway's DEFAULT_DNS_UPSTREAM moved to [::1]:5354
with an inline comment about the v4/v6 mismatch.
The macOS resolver shim in packaging/macos/build-pkg.sh was missed in
that sweep. Since 2026-04-29, every macOS install has shipped
/etc/resolver/fips with `nameserver 127.0.0.1` while the daemon
listened on `::1`, so .fips hostnames don't resolve via getaddrinfo
(ping6, curl, etc.) even though `dig @::1 -p 5354 …` works.
The mismatch is easy to miss: mDNSResponder swallows the timeout,
VPN clients that hijack DNS (NetworkExtension match-domain : *) mask
it entirely, and the symptom looks like "discovery hasn't found the
peer yet". Switch the shim to nameserver ::1 to match the daemon.
The v0.3.0 stable AUR push silently failed: with updpkgsums: true,
makepkg downloaded fips-<ver>.tar.gz into the AUR working tree, where
it was then staged by the deploy action and rejected by AUR's 488 KiB
max-blob hook.
Fetch the upstream source tarball and compute its b2sum in CI, patch
pkgver and the b2sums SKIP placeholder in PKGBUILD in-place, then
publish with updpkgsums: false so the AUR clone stays metadata-only.
Recompute and patch the fips.sysusers / fips.tmpfiles asset b2sums in
the same step so they stay in sync with the local files; this safety
net was previously provided by updpkgsums.
Add aur-publish-git.yml for the VCS fips-git PKGBUILD, triggered on
master pushes that touch PKGBUILD-git or its companion files plus
workflow_dispatch. pkgver is computed at build time by the PKGBUILD's
pkgver() function, so this workflow is not tied to release tags.
Add a workflow_dispatch tag input on the stable workflow so historical
release tags can be re-published manually, and drop continue-on-error:
true so future regressions surface in CI.
Operator-facing IPs in user-visible configs/docs (examples, tutorials,
packaging, sidecar templates) are now the resolvable hostnames of the
public test fleet (test-us01.fips.network, etc.) so they keep working
without baking specific addresses into examples.
Doc-comment and test fixtures in src/config/transport.rs use RFC 5737
TEST-NET-2 (198.51.100.1) so they cannot accidentally point at a real
host.
Also resyncs the openwrt-ipk fips.yaml with the common reference
(merge from master) and applies the same DNS-name swap there.
Bring the OpenWrt-shipped fips.yaml back into line with
packaging/common/fips.yaml, which had drifted: the OpenWrt copy was
missing the Nostr discovery, BLE, and TCP reference comment blocks, so
operators had no in-config hint that Nostr-mediated discovery existed.
Take common/fips.yaml verbatim and apply just the OpenWrt-specific
active overrides:
- ethernet: uncomment with wan/wwan/lan defaults (eth0, phy0-sta0,
br-lan)
- gateway: uncomment with lan_interface=br-lan and dns.listen on
[::1]:5353 (matches the dnsmasq forwarder the init script wires up)
Identity stays ephemeral by default (no persistent override), matching
common.
- Ethernet ioctl: cfg-gate the ioctl request parameter type — c_int on
musl, c_ulong on gnu — so the same code compiles on both
x86_64-unknown-linux-gnu and x86_64-unknown-linux-musl targets
- Chaos harness macOS support: replace direct host 'ip link' invocations
with a privileged Docker container helper (--net=host --pid=host) that
shares the Docker VM's namespaces, making veth pair setup work on both
Linux and macOS (where host 'ip' is unavailable and container PIDs
live inside the Docker Desktop VM)
- Python 3.9 compat: add 'from __future__ import annotations' to
docker_exec.py for X|Y union syntax support
- Add deploy/ to .gitignore
Co-authored-by: origami74 <origami74@gmail.com>
Arjen
