153 lines
7.8 KiB
Markdown
153 lines
7.8 KiB
Markdown
# Static musl portability plan
|
|
|
|
## Objective
|
|
|
|
Produce portable x86_64 Linux release binaries that do not depend on the target machine's glibc version. The primary release artifacts will be statically linked against musl and will remain compatible with the project's Linux, Qubes, Unix-socket, qrexec, TCP/HTTP, terminal, and secure-memory requirements.
|
|
|
|
## Current findings
|
|
|
|
- The development host is x86_64 Ubuntu 22.04 with glibc 2.35.
|
|
- Rust 1.80.1 and Cargo 1.80.1 are installed, but `rustup` is unavailable.
|
|
- Docker is available, so the build can be isolated from the host toolchain and libc.
|
|
- The project directly uses conventional Linux APIs through `libc`: `mlock`, `munlock`, `AF_UNIX`, `SO_PEERCRED`, `getuid`, `close`, `listen`, and `localtime_r`.
|
|
- Cryptography is implemented through Rust crates; no mandatory OpenSSL dependency was found in this repository.
|
|
- The sibling `nostr_core_lib_rust` checkout is a path dependency declared by `Cargo.toml` and must be present in the container build context.
|
|
- The vendored `ratatui` submodule and its crossterm backend must compile for the musl target.
|
|
- Existing release scripts build directly with the host Cargo installation and therefore do not guarantee libc portability.
|
|
|
|
## Compatibility policy
|
|
|
|
- Primary portable target: `x86_64-unknown-linux-musl`.
|
|
- Release artifacts are intended for x86_64 Linux systems regardless of the installed glibc version, subject to Linux kernel, CPU, terminal, qrexec, and resource-limit requirements.
|
|
- Do not use `-C target-cpu=native`; build for a conservative x86_64 baseline.
|
|
- Keep the normal host-native build for development and debugging.
|
|
- Treat `RLIMIT_MEMLOCK` separately from libc portability; static linking does not remove the need for appropriate memory-lock limits.
|
|
|
|
## Implementation phases
|
|
|
|
### 1. Add a reproducible musl build image
|
|
|
|
Create a pinned Docker build definition, preferably using a stable musl-based Rust image or a pinned Alpine image with an explicitly installed Rust toolchain.
|
|
|
|
The image must provide:
|
|
|
|
- Rust and Cargo versions compatible with the project's `Cargo.toml` and lockfile.
|
|
- The `x86_64-unknown-linux-musl` target.
|
|
- A C compiler/linker suitable for musl.
|
|
- Required build utilities and certificate configuration.
|
|
- No dependency on the host's Rust installation or glibc-linked build output.
|
|
|
|
Ensure the container can access both the project and the sibling `nostr_core_lib_rust` path dependency. The build must initialize or use the existing `ratatui` submodule consistently with normal project setup.
|
|
|
|
### 2. Add a portable build entry point
|
|
|
|
Create `build_musl.sh` with explicit behavior:
|
|
|
|
1. Validate that Docker is available.
|
|
2. Validate that the project root and required sibling path dependency exist.
|
|
3. Build both `signer` and `signer-client` for `x86_64-unknown-linux-musl` in release mode.
|
|
4. Use a dedicated output directory such as `dist/musl-x86_64`.
|
|
5. Copy the binaries using explicit names:
|
|
- `signer-linux-x86_64-musl`
|
|
- `signer-client-linux-x86_64-musl`
|
|
6. Avoid copying host-native binaries into the portable output.
|
|
7. Run binary validation and version checks before reporting success.
|
|
8. Return a nonzero status for missing dependencies, failed builds, invalid ELF output, dynamic linkage, or version mismatch.
|
|
|
|
Keep the script usable from the repository root and make its Docker invocation safe for ordinary user development.
|
|
|
|
### 3. Validate static linkage and runtime behavior
|
|
|
|
Add validation to the build flow using tools available in the container:
|
|
|
|
- `file` must identify x86_64 ELF executables.
|
|
- `readelf` must show the expected musl/static characteristics and no glibc loader requirement.
|
|
- `ldd` must not identify unresolved dynamic runtime dependencies.
|
|
- `signer --version` and `signer-client --version` must report the Cargo package version.
|
|
- Run `cargo test --target x86_64-unknown-linux-musl` where the test environment supports it.
|
|
|
|
Perform smoke checks for:
|
|
|
|
- stdio framing;
|
|
- Unix abstract socket bind/connect and peer identity;
|
|
- TCP and HTTP startup;
|
|
- qrexec subprocess integration where the host provides qrexec;
|
|
- mnemonic input and key derivation;
|
|
- representative signing and verification operations;
|
|
- TUI initialization in a terminal;
|
|
- `mlock` success or the documented unlocked-memory fallback behavior.
|
|
|
|
If a dependency cannot support musl, record the failure and determine whether it is optional, can be feature-gated, or requires retaining a glibc artifact.
|
|
|
|
### 4. Integrate release automation
|
|
|
|
Update `increment_and_push.sh` so release mode invokes the portable musl builder rather than the host-native `cargo build --release` path.
|
|
|
|
Preserve:
|
|
|
|
- version incrementing;
|
|
- source version updates;
|
|
- binary version verification;
|
|
- tagging and pushing;
|
|
- Gitea release creation;
|
|
- source tarball generation.
|
|
|
|
Extend asset upload handling to upload both musl binaries with their platform-specific names. Do not silently publish a host-linked binary under an ambiguous name.
|
|
|
|
Decide whether the existing unqualified assets remain as compatibility aliases. The safer default is to publish explicit musl names and retain legacy fallback handling only for older releases.
|
|
|
|
### 5. Update local deployment
|
|
|
|
Extend `deploy_local.sh` with an explicit portable option, such as `--musl`.
|
|
|
|
Suggested behavior:
|
|
|
|
- default development/debug builds remain native;
|
|
- `--musl` invokes the Docker build and installs the resulting portable binaries;
|
|
- release deployment should clearly report whether the installed binary is native or musl;
|
|
- installation continues to use `/usr/local/bin/` only when the user has the required privileges.
|
|
|
|
Add checks so the script does not mistake a musl artifact for a native build or install an absent/stale binary.
|
|
|
|
### 6. Update the installer
|
|
|
|
Update `install_signer.sh` to prefer the explicit musl release assets:
|
|
|
|
- `signer-linux-x86_64-musl`
|
|
- `signer-client-linux-x86_64-musl`
|
|
|
|
Retain support for existing older releases whose assets are named simply `signer` or `signer-client`. Add an override for users who need a different asset URL or a native glibc build.
|
|
|
|
Continue verifying the installed program version, and add target/artifact validation where practical so a mislabeled asset is rejected.
|
|
|
|
### 7. Update documentation
|
|
|
|
Update the build and platform sections of `README.md` to explain:
|
|
|
|
- native builds are for local development;
|
|
- portable release builds use static musl;
|
|
- the portable target is x86_64 Linux;
|
|
- the artifact names and output directory;
|
|
- how to verify static linkage with `file`, `readelf`, and `ldd`;
|
|
- static musl removes the glibc runtime dependency but not Linux kernel, CPU, terminal, qrexec, or `RLIMIT_MEMLOCK` requirements;
|
|
- Qubes users should use the musl release artifact unless a native glibc build is specifically required.
|
|
|
|
Update the project layout section with the new build definition and script.
|
|
|
|
## Acceptance criteria
|
|
|
|
- A clean Docker invocation builds both release binaries without using the host Rust compiler or host glibc.
|
|
- The resulting binaries are x86_64 and statically linked against musl.
|
|
- Neither binary requires the glibc dynamic loader or a minimum GLIBC symbol version.
|
|
- Both binaries pass version checks and the relevant test suite.
|
|
- Representative Unix, stdio, TCP/HTTP, crypto, TUI, and secure-memory paths are validated.
|
|
- Release automation uploads unambiguous musl artifacts.
|
|
- The installer can select the musl artifacts and remains compatible with older release naming.
|
|
- Documentation explains how to build, verify, install, and deploy the portable binaries.
|
|
|
|
## Deferred options
|
|
|
|
- Publishing a glibc-2.17-compatible artifact can be added later if a target integration requires glibc behavior.
|
|
- Additional architectures can be added later, but must have separate build images, artifact names, and validation.
|
|
- Musl should not be made the only local development target until the smoke tests demonstrate that all supported transports and terminal behavior are equivalent for the project's deployment environments.
|