mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-08-09 08:04:45 +00:00
Vitor PamplonaandClaude Opus 5
5896ae5eff
A production ANR on a Pixel 8 (Amethyst 1.13.1, anr_2026-08-03-12-55-26-256)
showed the app burning 596% CPU — 6 of 9 cores — with the main thread stuck in
WaitingForGcToComplete. 37 of 52 runnable DefaultDispatcher workers sat at ONE
program point inside PoolRequests.onIncomingMessage and 12 more at one point in
syncState, all state=R, while the single thread actually holding the lock was
itself parked in GC.
Root cause: RequestSubscriptionState.withLock was a raw busy-wait
(`while (lock.exchange(true)) { while (lock.load()) {} }`) with no yield or
backoff, and being `inline` it disappeared into its callers' frames. The lock is
per subId, but one subId spans every relay it runs on — 191 live sockets on that
device — so dozens of relay-dispatch threads piled onto a single AtomicBoolean.
Spinning is only correct when the holder cannot be descheduled; on Android it
always can.
The fix stripes the lock per (subId, relay) rather than making waiting cheaper.
All 11 withLock bodies in PoolRequests are already scoped to exactly one relay,
and every field of RequestSubscriptionState is keyed by relay, so the sharing was
purely an artifact of mutableMapOf not being thread-safe. State moves into a
ConcurrentMap<T, RelayState>; locks live in a fixed 32-entry stripe array that is
never mutated, so lock identity stays stable — if locks lived inside the map
values, a thread holding one while another dropped and re-created that entry
would leave both inside the critical section excluding nothing.
A suspending Mutex was measured and rejected: it needs 262 method overrides and
110 call sites to become suspend, and ran at 0.35-0.63x the current throughput.
Measured (LockDesignComparisonBenchmark, 191 relays / 64 dispatcher threads):
striped vs per-sub lock 1.5-2.8x throughput, bystander p50 halved
On device (SM-T220, playBenchmark, same account, n=3 per design):
DefaultDispatcher CPU -35% mean / -30% median vs the spin lock,
with non-overlapping ranges; GC -18%
Plus 10 min of driven UI stress (feed, profiles, chat, notifications,
communities): no ANRs, no crashes, thread pools stable.
Also here:
- PlatformLock: new expect/actual parking lock (ReentrantLock on jvmAndroid,
NSRecursiveLock on Apple, spin only on linuxX64 which is a CI target). quartz
cannot use commons' equivalent KmpLock because commons depends on quartz.
- LiveNegentropyIndex had the identical busy-wait with a full list SORT inside
the critical section; switched to PlatformLock.
- ConcurrentMap.remove (+ tests), with a caution that a removable value must not
own a lock callers acquire.
- SpinLockConvoyBenchmark: regression guard asserting contended waiters PARK
rather than spin (fails-before / passes-after). Pure benchmarks are gated
behind -PprodRelayBench=1, so CI cost is 0.3s rather than 51.5s.
Analysis and measurements: quartz/plans/2026-08-03-poolrequests-lock-contention.md
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
quartz plans
Audited 2026-06-30. 11 plans: 7 shipped (archived), 0 in-progress, 3 queued, 1 closed (negative result).
Queued
| Plan | Summary |
|---|---|
| 2026-05-08-local-headers-explorer.md | Headers-only Bitcoin P2P client to verify NIP-03 OTS attestations without a trusted block explorer. |
| 2026-06-12-giftwrap-deletion-requests.md | Let a recipient-authored kind-5 delete/block a gift wrap (kind 1059) addressed to them. |
| 2026-07-03-incremental-negentropy-storage.md | Always-current (created_at, id) index so cold NEG-OPENs stop paying a full scan + seal (~340 ms at 50k vs strfry's ~21 ms). |
| 2026-07-04-small-req-floor.md | Small-REQ dispatch floor: decomposed, inline fast path tried and reverted (no wire-level win); floor is transport-side. |
Archived (shipped)
| Plan | Summary |
|---|---|
| archive/2026-06-03-fix-nip46-bunker-double-resume-plan.md | Fix NIP-46 bunker double-resume crash and retry id-reuse races via Channel-per-request + fresh id per attempt. |
| archive/2026-06-04-auth-scope-vs-policy.md | Move relay-server authenticated-identity state from the policy into the engine-owned connection scope. |
| archive/2026-06-09-clink.md | Implement CLINK (Offers/Debits/Manage) Lightning-over-Nostr pointers, events, and client/server in Quartz. |
| archive/2026-06-11-runstr-interop.md | RUNSTR kind-1301 workout events and supporting fitness kinds in Quartz plus Amethyst fitness screens. |
| archive/2026-06-19-napplet-nip5a-resolver.md | Platform-agnostic NIP-5A static-site resolver verifying content-addressed Blossom blobs against signed manifests. |
| archive/2026-06-20-powr-interop.md | Parse and render the POWR/NIP-101e kind-1301 strength-workout dialect alongside the existing RUNSTR dialect. |
| archive/2026-06-28-git-smart-http-browser.md | Git smart-HTTP v2 client to browse NIP-34 repo file trees and render source from the clone URL. |