test: pin the amy argument surface the buzz-agent path depends on

Flag names and error strings are the contract between amy and the wrapper
scripts (and their operators), but nothing type-checks them: renaming --base-ref
or rewording an error breaks callers while compiling clean. This harness pins the
ones the buzz-agent path drives.

14 cases, all offline against an isolated ~/.amy via the `amy.home` seam the JVM
tests use: the shared `repo-naddr-or-coordinates` positional on git
browse/cat/log, `pass --channel GID` on the workflow verbs, --base-ref accepted
while --base-reff is rejected on both agent up and agent serve, and the
no_relays detail.

Confirmed to discriminate: 14/14 on this branch, 10/14 when built against
main-upstream's BuzzCommands, with the four failures printing the old misleading
"no relays: pass --relays ws://…" for input the user did pass.

Worth recording from writing it: a bare word like `garbage` is *accepted* as a
relay (the normalizer prepends wss://), so the realistic trigger for the empty
set is a pasted http:// url — which is what the cases use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WopdqNoZ9tYoMNppJG17BL
This commit is contained in:
davotoula
2026-07-29 15:42:48 +02:00
co-authored by Claude Opus 5
parent 42bf2a43d8
commit 592fd8f542
2 changed files with 121 additions and 0 deletions
+13
View File
@@ -112,6 +112,19 @@ empty-worktree failure mode, and `base_branch` resolution when `gh` answers oddl
non-zero) — the paths that otherwise only fail on someone else's machine, unattended, via a non-zero) — the paths that otherwise only fail on someone else's machine, unattended, via a
kind-46007 whose stderr is the only clue. kind-46007 whose stderr is the only clue.
### `./test-amy-args.sh` — the `amy` argument surface
```bash
./gradlew :cli:installDist
./tools/buzz-agent/test-amy-args.sh
AMY=/path/to/amy ./tools/buzz-agent/test-amy-args.sh # or point at another build
```
Pins the flag names and error strings the wrappers and their operators key on — `--base-ref`, the
`repo-naddr-or-coordinates` positional, `pass --channel GID`, and the `no_relays` detail. A renamed
flag or reworded message breaks callers without breaking any compile, and none of it is covered by a
type. Runs offline against an isolated `~/.amy` (every case is rejected before a relay is dialled).
### `agent-exec.sh` ### `agent-exec.sh`
Verified end-to-end against a throwaway repo with a stubbed `gh` + agent Verified end-to-end against a throwaway repo with a stubbed `gh` + agent
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env bash
#
# test-amy-args.sh — pins the argument-validation surface of the `amy` verbs the
# buzz-agent path drives (`buzz agent`, `buzz workflow`, `buzz dm`, `git`).
#
# These are the strings a human or a wrapper script keys on: a flag name silently
# renamed, or a usage/error message reworded, breaks callers without breaking any
# compile. Everything here runs offline against an isolated ~/.amy — the failing
# cases are all rejected before a relay is contacted.
#
# ./gradlew :cli:installDist
# ./tools/buzz-agent/test-amy-args.sh
# AMY=/path/to/amy ./tools/buzz-agent/test-amy-args.sh # or point at another build
#
# Exits 0 when every case passes, 1 otherwise.
set -uo pipefail
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
AMY="${AMY:-$ROOT/cli/build/install/amy/bin/amy}"
if [[ ! -x "$AMY" ]]; then
echo "no amy binary at $AMY — build one with: ./gradlew :cli:installDist" >&2
echo "(or set AMY=/path/to/amy)" >&2
exit 2
fi
# Isolate the data dir: `amy.home` is the seam the JVM tests use too, so this never
# touches a real ~/.amy.
AMY_HOME="$(mktemp -d)"
trap 'rm -rf "$AMY_HOME"' EXIT
export JAVA_OPTS="-Damy.home=$AMY_HOME"
pass=0
fail=0
expect() { # name, expected-substring, amy args...
local name="$1" want="$2"
shift 2
local out
out="$("$AMY" "$@" 2>&1)"
if [[ "$out" == *"$want"* ]]; then
printf ' PASS %s\n' "$name"
pass=$((pass + 1))
else
printf ' FAIL %s\n want output containing %q\n got: %s\n' \
"$name" "$want" "$(echo "$out" | head -3 | tr '\n' ' ')"
fail=$((fail + 1))
fi
}
reject() { # name, unwanted-substring, amy args...
local name="$1" unwanted="$2"
shift 2
local out
out="$("$AMY" "$@" 2>&1)"
if [[ "$out" != *"$unwanted"* ]]; then
printf ' PASS %s\n' "$name"
pass=$((pass + 1))
else
printf ' FAIL %s\n output must NOT contain %q\n got: %s\n' \
"$name" "$unwanted" "$(echo "$out" | head -3 | tr '\n' ' ')"
fail=$((fail + 1))
fi
}
printf '\namy arg-surface harness: %s\n\n' "$AMY"
"$AMY" --account harness init > /dev/null 2>&1 || { echo "could not create a throwaway account" >&2; exit 1; }
A=(--account harness)
DEAD_RELAY=wss://127.0.0.1:1 # closed port: connects and fails immediately
# --- the positional name every `git` verb shares ---------------------------
for verb in browse cat log; do
expect "git $verb names its missing positional" "repo-naddr-or-coordinates" git "$verb"
done
# --- required flags on the workflow verbs ----------------------------------
expect "workflow trigger demands a channel" "pass --channel GID" \
"${A[@]}" buzz workflow trigger "$DEAD_RELAY" wf1 --task "do a thing"
expect "workflow list demands a channel" "pass --channel GID" \
"${A[@]}" buzz workflow list "$DEAD_RELAY"
# --- the --base-ref flag name ----------------------------------------------
# `agent up` checks --repo before parsing the rest, so point it at a real repo to
# reach rejectUnknown.
expect "agent up rejects a mistyped --base-ref" "unknown flag: --base-reff" \
"${A[@]}" buzz agent up "$DEAD_RELAY" --repo "$ROOT" --approver npub1qqq --base-reff x --timeout 1
reject "agent up accepts --base-ref" "unknown flag" \
"${A[@]}" buzz agent up "$DEAD_RELAY" --repo "$ROOT" --approver npub1qqq --base-ref x --timeout 1
expect "agent serve rejects a mistyped --base-ref" "unknown flag: --base-reff" \
"${A[@]}" buzz agent serve "$DEAD_RELAY" --exec true --base-reff x
reject "agent serve accepts --base-ref" "unknown flag" \
"${A[@]}" buzz agent serve "$DEAD_RELAY" --exec true --base-ref x --once --timeout 1
# --- the no_relays detail ---------------------------------------------------
# Passing --relays wins over the outbox, so an all-unparseable list yields an empty
# set and must NOT tell the user to pass the flag they just passed. Note a bare word
# like "garbage" is *accepted* (the normalizer prepends wss://) — a pasted http url
# is the realistic rejection.
for bad in "http://x" "ws://" "," "!!!"; do
expect "buzz dm list names the real problem for --relays '$bad'" \
"no usable relays in --relays" "${A[@]}" buzz dm list --relays "$bad" --timeout 2
done
reject "buzz dm list accepts a well-formed relay" "no_relays" \
"${A[@]}" buzz dm list --relays "$DEAD_RELAY" --timeout 2
printf '\n %d passed, %d failed\n\n' "$pass" "$fail"
[[ "$fail" -eq 0 ]]