issue: update 820a - archive sync blocker root cause identified

This commit is contained in:
DanConwayDev
2026-01-20 12:04:43 +00:00
parent ebf085ebd8
commit 0e525f3c60
+48
View File
@@ -331,6 +331,54 @@ docker start ngit-relay
5. Re-validate state events once archive sync issues resolved 5. Re-validate state events once archive sync issues resolved
- **Blocker:** Archive sync incomplete/failed - must achieve >99% coverage before migration - **Blocker:** Archive sync incomplete/failed - must achieve >99% coverage before migration
### 2026-01-20 [Session 19:00] - Archive Sync Blocker ROOT CAUSE FOUND 🔍
- Completed: Investigation of archive sync failures for missing 338/587 repos (57.6%)
- **ROOT CAUSE IDENTIFIED:**
- **Archive is attempting to clone from `https://relay.ngit.dev/...` URLs**
- **relay.ngit.dev does NOT serve HTTP git clones** - only WebSocket relay service
- Manual test confirmed: `git clone https://relay.ngit.dev/<npub>/<repo>.git` returns 404
- Archive has been running 36+ hours but silently failing on relay.ngit.dev URLs
- **Clone URL Pattern:**
```
["clone","https://git.shakespeare.diy/.../repo.git","https://relay.ngit.dev/.../repo.git"]
```
- Primary URLs (git.shakespeare.diy, github.com, etc.): ✅ Works
- Fallback URLs (relay.ngit.dev): ❌ 404 Not Found
- **Archive Metrics (Port 7443):**
- Service status: ✅ Running normally (19+ hours uptime)
- Events synced: 3,445 from wss://relay.ngit.dev
- Repositories tracked: 1,856 (matches 42% coverage)
- Disk usage: 4.8GB (40-48% of expected 10-12GB)
- **Confirms partial sync, not full sync**
- **Why Only 42% Coverage:**
- Archive successfully fetches repos with working primary clone URLs
- Archive FAILS for repos with only relay.ngit.dev clone URLs (338/587 = 57.6%)
- Missing repos likely have relay.ngit.dev as primary or only clone URL
- **Configuration Analysis:**
- Archive config is CORRECT: `NGIT_DOMAIN=archive.internal`
- Sync source is CORRECT: `wss://relay.ngit.dev`
- Archive is NOT filtering relay.ngit.dev URLs (domain is different)
- **Files Created:**
- `work/state-validation/ARCHIVE-SYNC-INVESTIGATION.md` - Full investigation report
- Documented service status, metrics, logs, root cause, and recommended fix
- **Recommended Fix: Option 1 (RECOMMENDED)**
- Configure relay.ngit.dev to serve HTTP git clones
- Add nginx/HTTP service pointing to `/persistent/grasp/production/git/`
- Serve at `https://relay.ngit.dev/<npub>/<repo>.git/`
- Makes relay.ngit.dev a complete git hosting service
- No event republishing required
- **Alternative Fixes:**
- Option 2: Update state events to remove non-working clone URLs (complex)
- Option 3: Enhance archive retry logic (doesn't solve URL problem)
- **Next Steps:**
1. Decide on fix approach (Option 1 recommended)
2. Configure HTTP git service on relay.ngit.dev
3. Restart archive to retry failed fetches
4. Monitor archive metrics for coverage improvement
5. Re-run state validation (expect >95% coverage after fix)
- **Migration Status:** ❌ **BLOCKED** - HTTP git service required on relay.ngit.dev
- **Success Criteria:** Archive coverage >95% (587/587 repos), git data size ~10-12GB
### 2026-01-20 [Session 17:00] - Nostr Event Analysis Complete & Migration Guide Created ### 2026-01-20 [Session 17:00] - Nostr Event Analysis Complete & Migration Guide Created
- Completed: Comprehensive Nostr event analysis comparing relay.ngit.dev and localhost archive - Completed: Comprehensive Nostr event analysis comparing relay.ngit.dev and localhost archive
- Key Findings: - Key Findings: