laantungir/amethyst
1
0
Fork 0
mirror of https://github.com/vitorpamplona/amethyst.git synced 2026-10-05 11:18:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

build: make the Arti (Tor) native build reproducible

Browse Source 
The libarti_android.so shipped in the APK is the one binary we compile
ourselves, and it was the remaining blocker to a verifiable build: a Rust
cdylib is only reproducible when the compiler, the dependency graph, and the
embedded build paths are all pinned. None were.

Pin all three:
- rust-toolchain.toml pins rustc (rustup auto-installs it + the Android
  targets), so codegen is stable across machines.
- Cargo.lock is now generated and committed (501 packages); both build
  scripts run `cargo --locked` so transitive versions can't drift.
- repro-env.sh (sourced by build-arti.sh and build-arti-host.sh) rewrites
  host-specific absolute paths with --remap-path-prefix, disables incremental
  compilation, and sets a fixed SOURCE_DATE_EPOCH derived from the Arti tag.

With these, an independent rebuild of the pinned tag reproduces the committed
.so bit-for-bit, which is what lets F-Droid / Zapstore verify it from source
instead of trusting a prebuilt blob. README documents the pins and a
two-path build-and-diff verification recipe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JtjUcSjjpu4auFndw1QKeU
This commit is contained in:
Claude
2026-06-26 23:01:48 +00:00
parent 69457ec10d
commit f7a15a8407
6 changed files with 5538 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/arti-build/Cargo.lock
Generated
+5400
View File
File diff suppressed because it is too large Load Diff
tools/arti-build/README.md
+53 -8
View File
@@ -18,9 +18,40 @@ JNI wrapper built directly from Arti source.
Pre-built `.so` files should be committed to `amethyst/src/main/jniLibs/`. You only need to
rebuild if you want to verify binaries, update the Arti version, or modify the JNI wrapper.
## Reproducible builds
The shipped `.so` is **built to be byte-for-byte reproducible** so anyone —
F-Droid, Zapstore, or an independent auditor — can rebuild it from this tag and
confirm the committed binary wasn't tampered with. Three pins make that hold:
| Source of non-determinism | Pinned by |
|---|---|
| `rustc` / cargo version | [`rust-toolchain.toml`](rust-toolchain.toml) (rustup auto-installs it) |
| transitive dependency versions | committed [`Cargo.lock`](Cargo.lock); builds run `cargo --locked` |
| absolute build paths baked into the binary | `--remap-path-prefix` in [`repro-env.sh`](repro-env.sh) |
`repro-env.sh` (sourced by both build scripts) also sets `CARGO_INCREMENTAL=0`
and a fixed `SOURCE_DATE_EPOCH` derived from the Arti tag. The size-optimized
release profile in `Cargo.toml` (`lto`, `codegen-units = 1`, `strip`,
`panic = "abort"`) is itself deterministic for a fixed toolchain.
### Verify the committed binary reproduces
```bash
# Build twice into different checkout paths and confirm identical bytes.
# (Path remapping is what lets two different directories produce the same .so.)
cp -r tools/arti-build /tmp/arti-a && (cd /tmp/arti-a && ./build-arti.sh --release)
cp -r tools/arti-build /tmp/arti-b && (cd /tmp/arti-b && ./build-arti.sh --release)
sha256sum /tmp/arti-{a,b}/../../amethyst/src/main/jniLibs/arm64-v8a/libarti_android.so
```
A clean run prints the same SHA-256 for both, and matches the committed
`amethyst/src/main/jniLibs/arm64-v8a/libarti_android.so`.
## Prerequisites
1. **Rust toolchain**
1. **Rust toolchain** — the exact version is pinned in `rust-toolchain.toml`;
rustup installs it automatically. You only need rustup itself:
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
@@ -92,13 +123,17 @@ The first LOAD segment alignment should be `0x4000` (16384 bytes).
```
tools/arti-build/
├── README.md # This file
├── ARTI_VERSION # Pinned Arti git tag (e.g., arti-v1.9.0)
├── Cargo.toml # Rust dependencies and build profile
├── build-arti.sh # Build script
├── README.md # This file
├── ARTI_VERSION # Pinned Arti git tag (e.g., arti-v1.9.0)
├── rust-toolchain.toml # Pinned rustc version + Android targets (reproducibility)
├── Cargo.toml # Rust dependencies and build profile
├── Cargo.lock # Pinned transitive dependency versions (reproducibility)
├── repro-env.sh # Deterministic build env (path remapping, epoch) — sourced by both scripts
├── build-arti.sh # Build script (Android targets, shipped in APK)
├── build-arti-host.sh # Build script (host target, for JVM integration tests)
├── src/
│ └── lib.rs # JNI bridge (Rust → Kotlin)
└── .arti-source/ # [gitignored] Cloned Arti repository
│ └── lib.rs # JNI bridge (Rust → Kotlin)
└── .arti-source/ # [gitignored] Cloned Arti repository
```
## Updating Arti version
@@ -119,7 +154,17 @@ tools/arti-build/
https://gitlab.torproject.org/tpo/core/arti/-/raw/arti-v1.10.0/crates/arti-client/Cargo.toml
```
4. Rebuild and test:
4. Regenerate the committed lockfile so the new versions are pinned (builds run
`--locked` and will fail until this is refreshed):
```bash
./build-arti.sh --clean # clones the new tag + sets up the wrapper
cp .arti-source/arti-android-wrapper/Cargo.lock ./Cargo.lock
```
If you also bump the Rust toolchain, edit `channel` in `rust-toolchain.toml`.
5. Rebuild, then re-verify reproducibility (see "Reproducible builds" above) and
commit the regenerated `.so` files **together with** `Cargo.lock` /
`rust-toolchain.toml`:
```bash
./build-arti.sh --clean
```
tools/arti-build/build-arti-host.sh
+9 -2
View File
@@ -26,7 +26,8 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
WRAPPER_DIR="$SCRIPT_DIR/.arti-source/arti-android-wrapper"
ARTI_SOURCE_DIR="$SCRIPT_DIR/.arti-source"
WRAPPER_DIR="$ARTI_SOURCE_DIR/arti-android-wrapper"
if [ ! -d "$WRAPPER_DIR" ]; then
echo "Arti source / wrapper not found at $WRAPPER_DIR."
@@ -37,6 +38,12 @@ fi
# Sync the latest wrapper sources into the .arti-source clone — build-arti.sh
# normally does this, but if you've only edited lib.rs the host build needs it too.
cp "$SCRIPT_DIR/src/lib.rs" "$WRAPPER_DIR/src/lib.rs"
# Keep the dependency lock in sync with the committed one (build is --locked).
cp "$SCRIPT_DIR/Cargo.lock" "$WRAPPER_DIR/Cargo.lock"
# Same deterministic build env as the Android path (path remapping, pinned epoch).
# shellcheck source=repro-env.sh
source "$SCRIPT_DIR/repro-env.sh"
HOST_TARGET="$(rustc -vV | sed -n 's/^host: //p')"
case "$HOST_TARGET" in
@@ -54,7 +61,7 @@ OUT_DIR="$PROJECT_ROOT/amethyst/src/test/native-libs/$DEST_TAG"
mkdir -p "$OUT_DIR"
echo "Building Arti shim for $HOST_TARGET → $OUT_DIR/libarti_android.so"
cargo build --release \
cargo build --release --locked \
--manifest-path "$WRAPPER_DIR/Cargo.toml" \
--target "$HOST_TARGET"
tools/arti-build/build-arti.sh
+17 -1
View File
@@ -131,6 +131,17 @@ setup_wrapper() {
cp "$SCRIPT_DIR/Cargo.toml" "$wrapper_dir/Cargo.toml"
cp "$SCRIPT_DIR/src/lib.rs" "$wrapper_dir/src/lib.rs"
# Reproducibility: build against the committed lockfile so transitive
# dependency versions are identical for everyone. `cargo --locked` (below)
# fails if this lock is missing or stale rather than silently re-resolving.
if [ -f "$SCRIPT_DIR/Cargo.lock" ]; then
cp "$SCRIPT_DIR/Cargo.lock" "$wrapper_dir/Cargo.lock"
print_success "Pinned dependencies from committed Cargo.lock"
else
print_error "tools/arti-build/Cargo.lock missing — generate it with: cargo generate-lockfile (see README)"
exit 1
fi
# Patch Cargo.toml to use local arti-client from the source tree
# instead of pulling from crates.io
cd "$wrapper_dir"
@@ -170,7 +181,7 @@ build_for_target() {
-t "$target" \
--platform "$MIN_SDK_VERSION" \
-o "$OUTPUT_DIR" \
build --release \
build --release --locked \
--manifest-path "$ARTI_SOURCE_DIR/arti-android-wrapper/Cargo.toml"
if [ -f "$out_dir/$LIB_NAME" ]; then
@@ -229,6 +240,11 @@ main() {
clone_or_update_arti
setup_wrapper
# Deterministic build env (needs ARTI_SOURCE_DIR cloned above for SOURCE_DATE_EPOCH).
# shellcheck source=repro-env.sh
source "$SCRIPT_DIR/repro-env.sh"
print_success "Reproducible build env loaded (RUSTFLAGS path remapping, SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-unset})"
for target in "${TARGETS[@]}"; do
build_for_target "$target"
done
tools/arti-build/repro-env.sh
+41
View File
@@ -0,0 +1,41 @@
# Deterministic build environment for libarti_android.so.
#
# Sourced by build-arti.sh (Android targets) and build-arti-host.sh (host target)
# so the two paths stay in lockstep. Makes the Rust build byte-for-byte
# reproducible, which is what lets F-Droid / Zapstore / any third party rebuild
# the shipped .so from this tag and confirm it matches.
#
# The caller must already have set:
# SCRIPT_DIR — tools/arti-build
# ARTI_SOURCE_DIR — the Arti clone (.arti-source)
#
# The three things that otherwise make a Rust cdylib non-reproducible, and the
# fix for each:
# 1. Compiler version -> pinned by rust-toolchain.toml (rustup auto-installs).
# 2. Dependency versions -> pinned by the committed Cargo.lock + `cargo --locked`.
# 3. Absolute build paths embedded in panic locations / strings
# -> --remap-path-prefix rewrites them to stable virtual
# paths so two machines with different $HOME / checkout
# dirs produce identical bytes.
# Disable incremental compilation — its on-disk cache can perturb codegen order.
export CARGO_INCREMENTAL=0
# Where Cargo caches the crates.io registry + git deps (absolute, host-specific).
export CARGO_HOME="${CARGO_HOME:-$HOME/.cargo}"
# Rewrite every host-specific absolute prefix that rustc would otherwise bake
# into the binary. (Rust's own std is already remapped to /rustc/<hash> by the
# distributed toolchain, so pinning the version in rust-toolchain.toml covers it.)
REPRO_RUSTFLAGS="--remap-path-prefix=${CARGO_HOME}=/cargo"
REPRO_RUSTFLAGS="${REPRO_RUSTFLAGS} --remap-path-prefix=${ARTI_SOURCE_DIR}=/arti"
REPRO_RUSTFLAGS="${REPRO_RUSTFLAGS} --remap-path-prefix=${SCRIPT_DIR}=/arti-build"
export RUSTFLAGS="${RUSTFLAGS:-} ${REPRO_RUSTFLAGS}"
# Pin SOURCE_DATE_EPOCH to the commit the Arti tag points at — deterministic for
# a given ARTI_VERSION, and independent of when the build actually runs.
if [ -d "${ARTI_SOURCE_DIR}/.git" ]; then
_epoch="$(git -C "${ARTI_SOURCE_DIR}" log -1 --format=%ct 2>/dev/null || true)"
[ -n "${_epoch}" ] && export SOURCE_DATE_EPOCH="${_epoch}"
unset _epoch
fi
tools/arti-build/rust-toolchain.toml
+18
View File
@@ -0,0 +1,18 @@
# Pin the exact Rust toolchain used to build libarti_android.so.
#
# Reproducibility: rustc output is only stable for a fixed compiler version, so
# everyone who rebuilds the shipped .so (us, F-Droid, an independent verifier)
# must use this exact toolchain. rustup reads this file automatically and
# installs the pinned version + the Android targets on first invocation of the
# build scripts. Bump this in lockstep with ARTI_VERSION / Cargo.lock and
# re-verify (see README.md → "Reproducible builds").
[toolchain]
channel = "1.94.1"
profile = "minimal"
components = ["rustc", "cargo", "rust-std"]
targets = [
"aarch64-linux-android",
"x86_64-linux-android",
"armv7-linux-androideabi",
"i686-linux-android",
]