ci: align local/GitHub integration coverage and pin the toolchain source

Bring the local runner (testing/ci-local.sh) and GitHub CI into agreement
on two axes.

Suite coverage: the admission-cap integration suite ran only locally, so a
regression in it could never turn the GitHub gate red. Add an admission-cap
leg to the integration matrix, add testing/check-ci-parity.sh (wired as
'ci-local.sh --check-parity') to diff the two suite sets and fail on
unexpected drift, and document the deliberate local-only (live-Tor) and
granularity-only differences in a comment block atop both runners.

Toolchain selection: every CI and packaging job installed its toolchain
with dtolnay/rust-toolchain@stable, but the rust-toolchain.toml channel pin
overrode it for all compilation, wasting an install and printing a
misleading rustc version. Switch the stable call sites to
actions-rust-lang/setup-rust-toolchain, which reads rust-toolchain.toml as
the single source of truth. Keep the explicit cache steps (cache: false on
the new action), set rustflags empty so the action does not impose a global
-D warnings the previous setup never applied, fold the macOS cross-compile
target into the action input, and leave the OpenWrt nightly Tier-3 leg on
dtolnay/rust-toolchain@nightly.
This commit is contained in:
Johnathan Corgan
2026-06-05 20:20:50 +00:00
parent 0ce9bb5b99
commit c7218d8486
7 changed files with 267 additions and 14 deletions
+84 -6
View File
@@ -24,6 +24,28 @@ env:
RUST_BACKTRACE: 1
SOURCE_DATE_EPOCH: 0 # overridden per-step after checkout
# ─────────────────────────────────────────────────────────────────────────────
# CI parity invariant
#
# This GitHub integration matrix and the local default suite set
# (testing/ci-local.sh) MUST run the same integration suites, EXCEPT for the
# deliberate local-only entries below. Adding a suite to one runner without
# the other means "local green" and "GitHub green" stop being equivalent.
# testing/check-ci-parity.sh enforces this and fails on unexpected drift.
#
# Deliberate local-only (NOT on the GitHub gate), with reason:
# tor-socks5 — requires live Tor network; opt-in via --with-tor,
# unreliable on GitHub-hosted runners.
# tor-directory — same; live Tor dependency.
#
# Granularity-only differences (same coverage, different matrix shape —
# NOT a divergence):
# deb-install — split here into per-distro legs (debian12/ubuntu24/
# ubuntu26) for parallelism; local runs all distros in one
# suite.
# dns-resolver — single leg here; runs all scenarios (same as local).
# ─────────────────────────────────────────────────────────────────────────────
# ─────────────────────────────────────────────────────────────────────────────
# Job 1 Build matrix
#
@@ -35,9 +57,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
components: rustfmt
cache: false
rustflags: ''
- run: cargo fmt --check
clippy:
@@ -47,9 +71,11 @@ jobs:
- uses: actions/checkout@v4
- name: Install system dependencies
run: sudo apt-get update && sudo apt-get install -y libdbus-1-dev
- uses: dtolnay/rust-toolchain@stable
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
components: clippy
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
with:
@@ -98,7 +124,10 @@ jobs:
run: sudo nft -c -f packaging/common/fips.nft
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
@@ -160,7 +189,10 @@ jobs:
run: sudo apt-get update && sudo apt-get install -y libdbus-1-dev
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
@@ -210,7 +242,10 @@ jobs:
run: echo "SOURCE_DATE_EPOCH=$(git log -1 --format=%ct)" >> "$GITHUB_ENV"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
@@ -239,7 +274,10 @@ jobs:
- uses: actions/checkout@v4
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
@@ -323,6 +361,10 @@ jobs:
- suite: rekey-outbound-only
type: rekey-outbound-only
topology: rekey-outbound-only
# ── Inbound max_peers admission-cap test ───────────────────────
- suite: admission-cap
type: admission-cap
topology: mesh
- suite: acl-allowlist
type: acl-allowlist
# ── Firewall baseline (fips0 nftables default-deny) ────────────
@@ -723,6 +765,42 @@ jobs:
docker compose -f testing/static/docker-compose.yml \
--profile gateway down --volumes --remove-orphans
# ── Inbound max_peers admission-cap integration test ────────────────
# Lowers node.max_peers on one mesh node and asserts the inbound cap
# holds under sustained retry pressure: denied peers keep retrying but
# are never promoted to an active session. The admission-cap-test.sh
# assertions are tailored per link-layer handshake variant; the leg
# itself is uniform. Static-style harness on the shared mesh profile.
- name: Generate configs (admission-cap)
if: matrix.type == 'admission-cap'
run: bash testing/static/scripts/generate-configs.sh mesh
- name: Inject admission-cap config (admission-cap)
if: matrix.type == 'admission-cap'
run: bash testing/static/scripts/admission-cap-test.sh inject-config
- name: Start containers (admission-cap)
if: matrix.type == 'admission-cap'
run: |
docker compose -f testing/static/docker-compose.yml \
--profile mesh up -d
- name: Run admission-cap test
if: matrix.type == 'admission-cap'
run: bash testing/static/scripts/admission-cap-test.sh
- name: Collect logs on failure (admission-cap)
if: matrix.type == 'admission-cap' && failure()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile mesh logs --no-color | tail -300
- name: Stop containers (admission-cap)
if: matrix.type == 'admission-cap' && always()
run: |
docker compose -f testing/static/docker-compose.yml \
--profile mesh down --volumes --remove-orphans
# ── Real-deb install integration ────────────────────────────────────
# The deb-install harness builds its own .deb from source in a
# cargo-deb builder image; the pre-built Linux binary from the
+4 -1
View File
@@ -72,7 +72,10 @@ jobs:
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends libdbus-1-dev llvm
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
rustflags: ''
- name: Cache Cargo registry + build
if: ${{ env.ACT != 'true' }}
+5 -4
View File
@@ -69,10 +69,11 @@ jobs:
run: echo "SOURCE_DATE_EPOCH=$(git log -1 --format=%ct)" >> "$GITHUB_ENV"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Add cross-compile target
run: rustup target add ${{ matrix.target }}
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
target: ${{ matrix.target }}
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
+4 -2
View File
@@ -110,9 +110,11 @@ jobs:
- name: Install Rust toolchain (stable)
if: matrix.rust_channel == 'stable'
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
targets: ${{ matrix.rust_target }}
target: ${{ matrix.rust_target }}
cache: false
rustflags: ''
- name: Install Rust toolchain (nightly, Tier 3)
if: matrix.rust_channel == 'nightly'
+4 -1
View File
@@ -63,7 +63,10 @@ jobs:
echo "SOURCE_DATE_EPOCH=$epoch" >> $env:GITHUB_ENV
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
rustflags: ''
- name: Cache Cargo registry + build
uses: actions/cache@v4
+144
View File
@@ -0,0 +1,144 @@
#!/bin/bash
# ── CI parity invariant guard ───────────────────────────────────────────────
# The GitHub integration matrix (.github/workflows/ci.yml) and the local
# default suite set (ci-local.sh) MUST run the same integration suites,
# EXCEPT for the deliberate local-only entries listed below. Adding a suite
# to one runner without the other means "local green" and "GitHub green" stop
# being equivalent claims.
#
# Deliberate local-only (NOT on the GitHub gate), with reason:
# tor-socks5 — requires live Tor network; opt-in via --with-tor,
# unreliable on GitHub-hosted runners.
# tor-directory — same; live Tor dependency.
#
# Granularity-only differences folded before comparison (same coverage,
# different matrix shape — NOT a divergence):
# deb-install — GitHub splits into per-distro legs
# (deb-install-debian12/ubuntu24/ubuntu26); local runs all
# distros in one suite. Folded to "deb-install".
# chaos-* — GitHub fans each chaos scenario into its own matrix leg
# (type: chaos); local runs them all via the one CHAOS_SUITES
# path. The individual scenario names also differ cosmetically
# between runners (e.g. chaos-smoke-10 vs churn-mixed-10).
# Folded to a single "chaos" token on both sides.
# dns-resolver — single leg / single suite both sides; runs all scenarios.
#
# Exit 0 = parity clean. Exit 1 = unexpected divergence (suite names printed).
# ─────────────────────────────────────────────────────────────────────────────
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
CI_LOCAL="$SCRIPT_DIR/ci-local.sh"
CI_YML="$PROJECT_ROOT/.github/workflows/ci.yml"
# Deliberate local-only allowlist (suites intentionally absent from GitHub).
ALLOWLIST="tor-socks5 tor-directory"
for f in "$CI_LOCAL" "$CI_YML"; do
if [[ ! -f "$f" ]]; then
echo "check-ci-parity: missing file: $f" >&2
exit 2
fi
done
# Extract and normalize both suite sets in Python (robust YAML parse of the
# matrix; regex extraction of the bash suite arrays). Folding rules above are
# applied identically to both sides so only genuine divergence surfaces.
python3 - "$CI_LOCAL" "$CI_YML" "$ALLOWLIST" <<'PY'
import re
import sys
ci_local_path, ci_yml_path, allowlist_raw = sys.argv[1], sys.argv[2], sys.argv[3]
allowlist = set(allowlist_raw.split())
def fold(name):
"""Collapse granularity-only matrix shape into canonical suite identity."""
if name.startswith("chaos-") or name == "chaos":
return "chaos"
if name.startswith("deb-install"):
return "deb-install"
return name
# ── Local: parse the suite arrays from ci-local.sh ───────────────────────────
with open(ci_local_path, encoding="utf-8") as fh:
local_src = fh.read()
def bash_array(var):
m = re.search(rf"^{var}=\((.*?)\)", local_src, re.MULTILINE | re.DOTALL)
if not m:
return []
body = m.group(1)
# Quoted entries (chaos uses "display scenario flags"): first token is name.
quoted = re.findall(r'"([^"]*)"', body)
if quoted:
return [entry.split()[0] for entry in quoted if entry.strip()]
return [tok for tok in body.split() if tok.strip()]
local = set()
# Static, rekey, gateway, sidecar, acl, firewall, nostr, stun, dns, deb.
for var in ("STATIC_SUITES", "REKEY_SUITES", "ADMISSION_SUITES",
"GATEWAY_SUITES", "SIDECAR_SUITES", "ACL_SUITES",
"FIREWALL_SUITES", "NOSTR_RELAY_SUITES", "STUN_FAULTS_SUITES",
"DNS_RESOLVER_SUITES", "DEB_INSTALL_SUITES"):
local.update(bash_array(var))
# Chaos display names → fold to "chaos".
for _ in bash_array("CHAOS_SUITES"):
local.add("chaos")
# NAT scenarios are stored bare (cone/symmetric/lan) and prefixed nat- at use.
for scen in bash_array("NAT_SUITES"):
local.add(f"nat-{scen}")
# TOR_SUITES is the deliberate local-only set — excluded from the default path.
local = {fold(n) for n in local}
# ── GitHub: parse the integration matrix suite: values from ci.yml ───────────
import yaml # noqa: E402
with open(ci_yml_path, encoding="utf-8") as fh:
doc = yaml.safe_load(fh)
include = doc["jobs"]["integration"]["strategy"]["matrix"]["include"]
github = set()
for leg in include:
if "suite" not in leg:
continue
# Chaos legs carry inconsistent suite: names (chaos-smoke-10 vs
# churn-mixed-10) but a uniform type: chaos — fold via type, not name.
if str(leg.get("type", "")) == "chaos":
github.add("chaos")
else:
github.add(fold(str(leg["suite"])))
# ── Diff (subtract allowlist from local before comparison) ───────────────────
local_cmp = {n for n in local if n not in allowlist}
local_only = sorted(local_cmp - github)
github_only = sorted(github - local_cmp)
if local_only or github_only:
print("CI parity FAILED: integration suite sets diverge.\n")
if local_only:
print(" Local-only (in ci-local.sh, missing from ci.yml, "
"not in deliberate allowlist):")
for n in local_only:
print(f" - {n}")
if github_only:
print(" GitHub-only (in ci.yml, missing from local default path):")
for n in github_only:
print(f" - {n}")
print("\n Resolve by adding the suite to the other runner, or by adding "
"it\n to the deliberate local-only allowlist in "
"check-ci-parity.sh with a\n stated reason.")
sys.exit(1)
print("CI parity OK: integration suite sets match "
"(allowlist: " + ", ".join(sorted(allowlist)) + ").")
print(f" {len(github)} canonical suites compared on each side.")
sys.exit(0)
PY
+22
View File
@@ -12,6 +12,8 @@
# --only <suite> Run a single integration suite
# -j, --jobs <N> Max parallel chaos scenarios (default: 4)
# --list List available integration suites
# --check-parity Verify this suite set matches ci.yml's integration
# matrix (see testing/check-ci-parity.sh), then exit
# -h, --help Show this help
#
# Integration suites (default coverage):
@@ -32,6 +34,25 @@
# Exit codes:
# 0 — all stages passed
# 1 — one or more stages failed
#
# ── CI parity invariant ─────────────────────────────────────────────────────
# This local default suite set and the GitHub integration matrix
# (.github/workflows/ci.yml) MUST run the same integration suites, EXCEPT for
# the deliberate local-only entries below. Adding a suite to one runner
# without the other means "local green" and "GitHub green" stop being
# equivalent. testing/check-ci-parity.sh enforces this and fails on drift.
#
# Deliberate local-only (NOT on the GitHub gate), with reason:
# tor-socks5 — requires live Tor network; opt-in via --with-tor,
# unreliable on GitHub-hosted runners.
# tor-directory — same; live Tor dependency.
#
# Granularity-only differences (same coverage, different matrix shape —
# NOT a divergence):
# deb-install — local runs all distros sequentially in one suite; GitHub
# splits into per-distro legs (debian12/ubuntu24/ubuntu26).
# dns-resolver — single suite both sides; runs all scenarios.
# ─────────────────────────────────────────────────────────────────────────────
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
@@ -170,6 +191,7 @@ while [[ $# -gt 0 ]]; do
--only) ONLY_SUITE="$2"; shift 2 ;;
-j|--jobs) PARALLEL_JOBS="$2"; shift 2 ;;
--list) list_suites ;;
--check-parity) exec "$SCRIPT_DIR/check-ci-parity.sh" ;;
-h|--help) usage ;;
*) echo "Unknown option: $1"; usage ;;
esac