Announcements Purgatory System

ID: 3ca0

Problem

Currently, acceptable announcements that don't replace existing announcements are immediately accepted and served to clients. We need a "purgatory" system where new announcements are held in a pending state until they become relevant (when a corresponding state event arrives).

This prevents serving announcements for repositories that may never receive state events, while still maintaining the git infrastructure in case they do.

Requirements

1. Purgatory Instead of Immediate Acceptance

Acceptable announcements that do not replace an existing announcement should go to purgatory instead of being immediately accepted and served
The bare git repo should still be created when an announcement enters purgatory
Announcements should not be served to clients while in purgatory

2. Expiry and Cleanup

If an announcement expires while in purgatory, the bare git repo should be deleted
Need to track expiry times for announcements in purgatory
Background cleanup mechanism to handle expired purgatory announcements

3. State Event Interaction

When a state event arrives, any announcements in purgatory should have their expiry extended to match the state event's expiry
This keeps purgatory announcements alive as long as they might become relevant
Need to determine when/how announcements transition from purgatory to active

4. State Event Acceptability

State event acceptability checks must consider announcements in purgatory
A state event should be rejected if it would conflict with announcements in purgatory (not just active announcements)
Validation logic needs to check both active and purgatory announcements

Plan

Phase 1: Design
- Design doc: docs/explanation/announcements-purgatory-design.md
- Implementation doc: docs/explanation/announcements-purgatory-implementation.md
Phase 2: Core implementation (commit 1d09e4b)
- AnnouncementPurgatoryEntry type and DashMap store
- Route new announcements to purgatory (replacements skip)
- Promote on git data arrival (process_purgatory_announcements)
- Authorization checks purgatory (fetch_repository_data_with_purgatory)
- State policy uses purgatory for maintainer validation
- Cleanup task handles announcement expiry
- Updated count()/cleanup() to 3-tuples
Phase 3: Implement SyncLevel and fix sync integration (ROOT CAUSE of test failure)
- The sync module doesn't fetch state events for purgatory announcements because they're not saved to DB and thus never "confirmed" as repos
- The fix per design doc (decision #6) is SyncLevel, NOT treating purgatory as fully confirmed. Full confirmation would over-fetch L2/L3 events (PRs, issues) that get rejected anyway.
- Add SyncLevel enum (StateOnly vs Full) to RepoSyncNeeds
- When announcement enters purgatory, register in sync index with SyncLevel::StateOnly (only fetches kind 30618 state events)
- On promotion, upgrade sync level to Full
- Modify filter building in src/sync/filters.rs to partition repos by sync level and build appropriate filters
- Revert the partial fix in commit 1d09e4b that treats ProcessResult::Purgatory as confirmed in pending_sync_index (wrong approach - would trigger full L2/L3 sync)
- Keep the hot cache invalidation for rejected state events (correct)
- This should fix test_archive_read_only_creates_bare_repo
Phase 4: Announcement persistence (save/restore)
- save_to_disk() and restore_from_disk() skip announcement_purgatory
- PurgatoryState struct has no announcement field
- Need SerializableAnnouncementPurgatoryEntry type
- Need save/restore logic with Instant offset conversion
- Need persistence tests (purgatory_persistence.rs currently asserts 0)
Phase 5: Remaining design features
- Soft expiry two-phase - delete bare repo but retain event for 24h, allow revival on state event arrival
- Expiry extension - extend_announcement_expiry() exists but is never called from state policy or git auth
- Newer announcement replaces older in purgatory (not just DB check)
- Service change clears purgatory entry
- Deletion event (kind 5) removes from purgatory
Phase 6: Testing
- Fix test_archive_read_only_creates_bare_repo
- Add persistence round-trip tests for announcements
- Test new functionality from Phase 5 as implemented

Progress

2026-01-23 [Session 00:00]

Started: Created issue with requirements and initial plan
Created worktree: worktrees/3ca0-announcements-purgatory
Ready to begin implementation

2026-01-23 [Design Session]

Conducted extensive architectural review with multiple iterations
Clarified requirements based on user feedback:
- Problem: Prevent serving empty bare repos indefinitely (misleads clients)
- Trigger: Git data arrival promotes announcements (not state events)
- Indexing: Must use (pubkey, identifier) tuple (identifier not unique)
- Expiry extension: Two places (state event arrival AND git auth)
- Edge cases: Newer announcements replace older, service list changes
Created design document: docs/explanation/announcements-purgatory-design.md
OpenCode conversation: https://opencode.ai/c/01JJBXPFVVQVX5RVQF2KQBQZJR

2026-02-05 [Design Completion]

Completed high-level design with user review
Key decisions finalized:
- Sync integration: Purgatory announcements sync state events only (SyncLevel::StateOnly)
- Authorization: Must check purgatory announcements in addition to DB
- Soft expiry: Delete bare repo but retain event for 24h (allows revival on state event)
Created implementation details document: docs/explanation/announcements-purgatory-implementation.md
Design documents committed: 603c87f

2026-02-13 [Core Implementation]

Implemented core announcement purgatory (commit 1d09e4b)
Happy path works: announcement -> purgatory -> git push -> promotion -> served
All purgatory integration tests pass (44/44)
One existing test broken: test_archive_read_only_creates_bare_repo
Root cause identified: sync module does not treat purgatory announcements as "confirmed repos", so per-repo sync never fetches state events for them. The sync flow is: fetch announcements via negentropy -> save to DB marks them "confirmed" -> for each confirmed repo, fetch state/PR events. Purgatory announcements skip the DB save, so they're never confirmed.
Partial fix applied: ProcessResult::Purgatory now counts as confirmed in pending_sync_index, and announcements entering purgatory invalidate previously-rejected state events from the hot cache
Also identified: announcement persistence not implemented (save_to_disk and restore_from_disk skip announcement_purgatory entirely)

Notes

This feature prevents resource waste on announcements that never receive git data
Maintains quick response time for repositories that do become active
Soft expiry solves the failed_events problem without permanent blocklisting

6.9 KiB Raw Blame History