issue: update 820a - deeper sync query design issue discovered

This commit is contained in:
DanConwayDev
2026-01-21 10:39:17 +00:00
parent 51ca048d34
commit 2a5e1b17fa
+56
View File
@@ -550,6 +550,62 @@ docker start ngit-relay
4. Verify state event count matches production (>95% coverage)
5. Proceed to Phase 3 validation once sync complete
### 2026-01-21 [Session 15:00] - Negentropy Fix Applied & DEEPER ISSUE DISCOVERED ⚠️
- Completed: Applied negentropy fix, verified working, but discovered sync query design issue
- **Negentropy Fix Applied Successfully:**
- ✅ Issue b4d0 created: "Failed historic sync batches not retried after negentropy timeout"
- ✅ Config updated: Added `syncDisableNegentropy = true` to archive service
- ✅ Deployed and restarted: Service running with REQ+EOSE sync
- ✅ Fix verified: Logs show `use_negentropy=false`, state events flowing
- ✅ State event count: 336 → 339 (increasing)
- ✅ Status: `Connected` (not `ConnectedHistoricSyncFailures`)
- **1-man-1-tank Investigation - CRITICAL FINDING:**
- ❌ State event still NOT received after negentropy fix
- ✅ Announcement (kind 30617) exists in archive
- ❌ State event (kind 30618) does NOT exist in archive
- ✅ State event exists on relay.ngit.dev and git.shakespeare.diy
- ❌ Git repo empty (no refs, no objects)
- **ROOT CAUSE: Sync Query Design Flaw**
- Announcements: Broad query `{"kinds": [10317, 30617]}` (no d-tag filter) ✅
- State events: Narrow query `{"kinds": [30618], "#d": [list of known repos]}` ❌
- **Problem:** State events only queried for repos archive already knows about
- **Chicken-and-egg:** 1-man-1-tank state event never synced → not in query list → never queried → never synced
- **Why This Happens:**
1. Archive receives announcement for 1-man-1-tank (broad query)
2. Archive creates empty git repo
3. Archive queries state events only for repos it already has state events for
4. 1-man-1-tank NOT in list → state event never queried
5. State event never arrives → git data never fetched → repo stays empty
- **Evidence from Logs:**
```
# Broad announcement query (works)
Sending '["REQ",...,{"kinds":[10317,30617]}]' to relay.ngit.dev
# Narrow state event query (misses 1-man-1-tank)
Sending '["REQ",...,{"#d":["BeatsBars","storyforge-ai",...],
"kinds":[30618]}]' to relay.ngit.dev
# Note: "1-man-1-tank" NOT in d-tag list
```
- **Impact Assessment:**
- Negentropy fix IS working (REQ+EOSE active, other repos syncing)
- But won't help repos whose state events were never queried
- Affects ALL repos where announcement arrived but state event didn't
- Explains persistent gap between announcement count and state event count
- **Recommended Fixes:**
1. **Option 1 (Best):** Add broad state event query `{"kinds": [30618]}` (no d-tag filter)
2. **Option 2:** Query state events for all known announcements (when announcement received)
3. **Option 3 (Immediate):** Manual re-sync by publishing state events to archive
- **Files Created:**
- `work/NEGENTROPY-FIX-VERIFICATION.md` - Verification of negentropy fix
- `work/1-MAN-1-TANK-SYNC-STATUS.md` - Detailed investigation of sync query issue
- **Migration Status:** ❌ **BLOCKED** - Sync query design prevents complete state event coverage
- **Next Steps:**
1. Create issue for sync query design flaw
2. Implement broad state event query (Option 1)
3. Re-sync archive with fixed query design
4. Verify 1-man-1-tank and other missing repos sync successfully
5. Achieve >95% state event coverage before proceeding to Phase 3
### 2026-01-20 [Session 14:30] - Naughty List Blocker Identified
- Completed: Extensive state validation and root cause analysis
- **Blocker identified:** relay.ngit.dev on git "naughty list" preventing archive sync