ngit-grasp/b905-deletion-request-support.md

Deletion Request Support (NIP-09)

ID: b905

Status: Planning Complete, Ready for Implementation
Priority: Medium
Complexity: High (graph algorithms, multi-database coordination)
Estimated Timeline: 6 weeks (phased approach with full test coverage)

Problem Statement

ngit-grasp currently has no mechanism to handle NIP-09 deletion requests. Repository owners cannot remove their repositories from the relay, and there's no protection against "left-pad" scenarios where critical repositories are deleted, breaking dependent projects.

Objectives

1. ✅ Respect deletion requests for repository announcements (kind 30617) deleted by address
2. ✅ Archive git data before deletion with configurable retention (default 90 days)
3. ✅ Cascade delete ALL dependent events (PRs, issues, patches, comments)
4. ✅ Provide recovery mechanism via holding database
5. ✅ Configurable "deletion request disrespector" mode for archival relays (prevents left-pad)
6. ✅ Handle multi-maintainer repositories with graph-based retention algorithm

Design Decisions

Three-Database Architecture

• Main Database: Live events actively served in queries
• Holding Database: Archived events during retention window (separate DB file, same backend type)
• Archive Filesystem: Compressed git data (.archive/ subdirectory)

Cascade Delete Strategy

Delete ALL dependent events when announcement is deleted (not just owner's events). Rationale:

• Matches user expectation of "delete everything"
• Orphaned PRs/issues confusing without context
• Recovery available via holding database
• Archival relays protect community work

Multi-Maintainer Handling

Graph-based retention algorithm:

1. Archive deleting maintainer's git directory
2. Re-evaluate all events through acceptance policy WITHOUT deleted announcement
3. Build dependency graph showing retention reasons
4. Detect circular dependencies
5. Delete events that would fail acceptance

Configuration

• deletion_request_disrespector (bool, default false) - Archival relay mode
• archive_retention_secs (u64, default 7776000 = 90 days) - Configurable in seconds for testing

Implementation Phases

Phase 1: Core Deletion + Simple Cascade (Week 1)

Goal: Basic deletion working for single-maintainer repositories

Tasks:

• Add config options to src/config.rs:
  • deletion_request_disrespector: bool
  • archive_retention_secs: u64 (default 7776000)
• Update configuration files (CRITICAL - must update all 4):
  • src/config.rs - Config struct fields
  • docs/reference/configuration.md - Documentation
  • nix/module.nix - NixOS options
  • .env.example - Example with comments
• Create holding database infrastructure:
  • src/database/holding.rs - Holding database wrapper
  • Support same backend types (LMDB/Memory/NostrDB)
  • Separate file: <relay_data_path>/holding-<backend>
• Implement src/nostr/policy/deletion.rs:
  • DeletionPolicy::validate() - Parse kind 5, extract a/e tags
  • process_address_deletion() - Handle a tag deletions
  • Author pubkey matching validation
  • Check deletion_request_disrespector config
• Integrate into src/nostr/builder.rs:
  • Add Kind::EventDeletion (5) case in admit_event()
  • Route to DeletionPolicy
  • If disrespector mode: store event, return early
• Implement event dependency query:
  • query_dependent_events() - Recursive traversal
  • Find events with a tags referencing announcement
  • Find events with e tags referencing above events
  • Simple cascade (no graph complexity yet)
• Move events between databases:
  • move_to_holding_database() - Atomic operation
  • Move announcement + all dependents
  • Store deletion timestamp, deletion event ID
  • Delete from main database
• Tests:
  • Config loading and validation
  • Kind 5 parsing and validation
  • Author pubkey matching (accept/reject)
  • Disrespector mode behavior
  • Simple cascade deletion (single maintainer)
  • Events moved to holding DB correctly
  • Events no longer in main DB (queries return nothing)
  • Use 3-5 second retention for fast tests

Exit Criteria:

• All tests passing with 100% coverage
• Single-maintainer repository deletion fully working
• Events properly moved between databases
• Disrespector mode prevents deletion

---

Phase 2: Git Archival & Cleanup (Week 2)

Goal: Archive git data and implement cleanup task

Tasks:

• Implement src/git/archive.rs:
  • archive_repository() - Create tar.gz of git directory
  • Archive path: .archive/<npub>/<identifier>-<timestamp>.tar.gz
  • create_archive_metadata() - Store metadata JSON
  • Compression using flate2 crate
• Archive metadata structure:
  • Deletion timestamp
  • Deletion event ID
  • Maintainer pubkey
  • Identifier
  • Archive file path
  • Expiry timestamp (deletion_ts + retention_secs)
• Integrate archival into deletion flow:
  • Archive git data BEFORE moving events to holding DB
  • Handle archive failures (rollback? log error?)
• Background cleanup task:
  • Spawn tokio task in main.rs
  • Run daily (24-hour interval)
  • Also run on startup (catch-up for offline periods)
  • cleanup_archived_data():
    • Query holding DB for expired entries
    • Delete events from holding DB
    • Delete archive tar.gz files
    • Delete archive metadata
• Tests:
  • Archive creation and compression
  • Archive metadata storage
  • Archive extraction (verify integrity)
  • Background cleanup with 5-second retention
  • Cleanup on startup (simulate offline period)
  • Disk space reclamation verification

Exit Criteria:

• Git data properly archived before deletion
• Background cleanup working reliably
• No disk space leaks
• All tests passing

---

Phase 3: Multi-Maintainer Graph Algorithm (Week 3)

Goal: Handle complex multi-maintainer deletion scenarios

Tasks:

• Implement dependency graph builder:
  • build_event_graph() - Create directed graph
  • Nodes: Events
  • Edges: References (a/e/q tags)
  • Track retention reasons for each event
• Re-evaluation engine:
  • reevaluate_events_without_announcement()
  • Query all events referencing deleted announcement
  • Run each through acceptance policy WITHOUT deleted announcement
  • Track which announcements/events make it acceptable
• Graph traversal:
  • topological_traverse() - Start from announcements
  • Mark reachable events as "keep"
  • Mark unreachable events as "delete"
  • Configurable max depth (default 100)
• Circular dependency detection:
  • Detect mutual references (A→B, B→A)
  • Mark both for deletion if no external anchor
• Integration:
  • Replace simple cascade with graph algorithm
  • Handle edge cases (isolated subgraphs)
• Tests:
  • Two maintainers, one deletes (events preserved)
  • Two maintainers, both delete (events deleted)
  • Circular dependencies (both deleted)
  • Complex reference graphs (3+ levels deep)
  • Max depth exceeded (logged warning)

Exit Criteria:

• Multi-maintainer scenarios handled correctly
• Graph algorithm thoroughly tested
• Max depth configurable and tested
• All tests passing

---

Phase 4: Recovery Mechanism (Week 4)

Goal: Allow owners to recover accidentally deleted repositories

Tasks:

• Implement recovery detection:
  • check_for_recovery() in announcement processing
  • Query holding DB for matching identifier + pubkey
  • Check if within retention period
• Git data restoration:
  • Extract tar.gz to temp directory
  • Verify integrity
  • Move to <git_data_path>/<npub>/<identifier>.git
• Event restoration:
  • Query holding DB for all events
  • Re-run acceptance policy (should now pass)
  • Move from holding DB → main DB
  • Count restored events
• Archive cleanup after recovery:
  • Delete archive tar.gz
  • Delete archive metadata
  • Remove holding DB entries
• Response to client:
  • Success message with count: "Restored 47 events"
  • Or normal new repo: "New repository created"
• Tests:
  • Full recovery workflow
  • Partial recovery (some events expired)
  • Recovery at edge of retention window
  • Recovery after retention expired (new repo)
  • Corrupt archive (graceful failure)

Exit Criteria:

• Recovery fully functional
• Edge cases handled gracefully
• User-friendly response messages
• All tests passing

---

Phase 5: Extended Cascade Deletion (Week 5)

Goal: Complete cascade deletion for all event types

Tasks:

• Extend cascade delete to all NIP-34 event types:
  • Patches (1617) - tag repos via a
  • Issues (1621) - tag repos via a
  • PR Updates (1619) - tag PRs via e
  • Status events (1630-1633) - tag issues/PRs via e
• Update dependency graph to include all types
• Tests for each event type:
  • Patches cascade delete
  • Issues cascade delete
  • PR Updates cascade delete
  • Status events cascade delete
  • Mixed event types (comprehensive)

Exit Criteria:

• All NIP-34 event types properly cascade deleted
• Comprehensive test coverage
• All tests passing

---

Phase 6: Analysis & Edge Cases (Week 6)

Goal: Production hardening and edge case analysis

Tasks:

• Background cleanup task investigation:
  • Daily cleanup doesn't work well with 3-second retention tests
  • Design alternative: trigger-based cleanup for tests?
  • Or: configurable cleanup interval (separate from retention)?
  • Test with both short intervals (tests) and daily (production)
• rust-nostr deletion behavior investigation:
  • Check if nostr-relay-builder automatically deletes events tagged in kind 5
  • Check if it stops serving events tagged in deletion requests
  • If YES: need to override/disable this behavior when deletion_request_disrespector = true
  • Document any hooks or callbacks we need to implement
  • Ensure archival mode truly ignores deletions at relay library level
• Author validation enforcement:
  • Verify we only honor deletion requests where author matches deleted event author
  • Add comprehensive tests for author mismatch rejection
  • Document this requirement clearly in NIP-11 and README
• Max depth edge case analysis:
  • Document scenarios where depth limit matters
  • Recommend alternative approaches if needed
  • Add configuration guidance
• Large-scale scenario testing:
  • Repository with 1000+ PRs/issues
  • Deep dependency chains (50+ levels)
  • Performance profiling
  • Memory usage analysis
• Race condition investigation:
  • Deletion during active sync
  • Concurrent deletions of shared repo
  • Lock strategy for git archival
  • Document mitigation strategies
• Blacklist retroactive deletion investigation:
  • Should adding to blacklist move existing repos to 90-day holding area?
  • Decision: YES - use same archive mechanism as deletion requests
  • Detect on startup: scan main DB for repos matching blacklist
  • Move to holding area with metadata marking blacklist trigger
  • Same cascade delete logic as NIP-09 deletions
  • Archive git data before moving to holding area
• Blacklist + disrespector mode interaction:
  • Decision: deletion_request_disrespector does NOT prevent blacklist deletion
  • Blacklist is moderation/operational decision, not user-initiated deletion
  • Archival relays can still blacklist spam/malware/abuse
  • Only NIP-09 user deletions are ignored in disrespector mode
• Blacklist removal recovery:
  • Should removing from blacklist auto-restore from holding DB?
  • Or require manual operator intervention?
  • What about the git data archive?
  • Design recovery flow for unblacklisted repos
• Manual ejection from holding area:
  • Operator needs ability to force-delete from holding area before expiry
  • Use case: large repos consuming excessive storage
  • Use case: confirmed malware/abuse that shouldn't be recoverable
  • Design mechanism: admin CLI command? config flag? database operation?
  • Should manual ejection delete git archive immediately or wait for cleanup?
  • Log manual ejections for audit trail
• Metrics implementation:
  • ngit_deletion_requests_total
  • ngit_deletion_requests_processed
  • ngit_blacklist_deletions_total (new)
  • ngit_holding_database_events
  • ngit_holding_database_size_bytes
  • ngit_archive_files_total
  • ngit_archive_size_bytes
  • ngit_recoveries_total
  • ngit_permanent_deletions_total
  • ngit_manual_ejections_total (new)
• Documentation:
  • Update docs/explanation/deletion-requests.md with findings
  • Add blacklist deletion behavior documentation
  • Add manual ejection mechanism documentation
  • Add edge case documentation
  • Performance tuning guide

Exit Criteria:

• Edge cases documented
• Performance acceptable for production
• Metrics implemented and tested
• Documentation complete

---

Phase 7: Integration & Final Testing (Week 7)

Goal: End-to-end testing and documentation

Tasks:

• End-to-end integration tests:
  • Full deletion workflow (all phases)
  • Multi-maintainer + recovery + cleanup
  • Disrespector mode comprehensive test
• grasp-audit compliance tests:
  • NIP-09 validation
  • Event re-submission after deletion (rejected)
  • Deletion request event storage
  • Archival mode behavior
• NIP-11 relay information updates:
  • Add "deletion" to supported NIPs array (only if deletion_request_disrespector = false)
  • If disrespector mode enabled, do NOT advertise NIP-09 support
  • Update src/http/nip11.rs to conditionally include based on config
• README.md updates:
  • Add NIP-09 deletion request support to feature list
  • Document cascade deletion behavior
  • Link to explanation document
  • Update "Delete Events" roadmap section (mark as complete)
• Architecture documentation updates:
  • Add deletion request section to docs/explanation/architecture.md
  • Document cascade deletion strategy
  • Reference deletion-requests.md for details
• User documentation:
  • How-to: "Deleting a Repository"
  • How-to: "Running an Archival Relay"
  • How-to: "Recovering Deleted Repositories"
• Reference documentation:
  • Configuration reference (update all 4 sources!)
  • NIP-09 feature matrix
  • Database schema documentation
• Code review and cleanup:
  • Remove debug logging
  • Add production logging (info level)
  • Code cleanup and refactoring
  • Performance optimizations

Exit Criteria:

• All integration tests passing
• Documentation complete and reviewed
• Code ready for production
• Ready to merge

Technical Architecture

See docs/explanation/deletion-requests.md for comprehensive architecture documentation.

Key Components:

• src/nostr/policy/deletion.rs - Deletion request validation
• src/git/archive.rs - Git repository archival
• src/database/holding.rs - Holding database wrapper
• Background cleanup task in main.rs

Data Flow:

Kind 5 Event → Validate → Query Dependents → Archive Git → Move Events → Delete from Main DB
                                                    ↓
                                         Background Task (daily)
                                                    ↓
                              Expired? → Delete from Holding DB → Delete Archive

Configuration Updates Required

CRITICAL: Must update all 4 configuration sources (per AGENTS.md):

1. ✅ src/config.rs
2. ✅ docs/reference/configuration.md
3. ✅ nix/module.nix
4. ✅ .env.example

Testing Notes

• Use 3-5 second retention for tests (not 90 days!)
• Real temp directories (not mocked filesystem)
• Full test coverage at each phase before proceeding
• Integration tests after each phase completion
• Background cleanup timing: Daily cleanup doesn't work with 3-second tests - investigate in Phase 6

Critical Requirements

Author Validation (NIP-09 Spec):

• We will ONLY honor deletion requests where the deletion request author matches the deleted event author
• This is a fundamental security requirement
• Must be clearly documented in NIP-11, README, and user-facing docs

rust-nostr Behavior:

• Need to verify nostr-relay-builder doesn't automatically process deletions
• If it does, must override this when deletion_request_disrespector = true
• Investigate in Phase 6

Open Questions & Future Investigation

Deferred to Phase 6 Analysis:

• Background cleanup timing strategy (daily vs configurable interval)
• rust-nostr automatic deletion behavior and override mechanisms
• Author validation comprehensive test coverage
• Max depth edge cases and alternatives
• Large-scale performance characteristics (1000+ events)
• Race conditions during concurrent operations
• Lock strategies for git repository access

Future Enhancements:

• GRASP-05 archive mode integration
• Selective disrespect (based on popularity/criteria)
• Distributed archive network coordination
• Recovery notifications to repository owners

Success Criteria

1. ✅ Repository owners can delete via NIP-09
2. ✅ Git data archived for configurable retention
3. ✅ All dependent events cascade deleted
4. ✅ Recovery mechanism working
5. ✅ Archival mode prevents left-pad
6. ✅ Multi-maintainer scenarios handled
7. ✅ Full test coverage (unit, integration, audit)
8. ✅ Documentation complete
9. ✅ Production-ready metrics

References

• Explanation: docs/explanation/deletion-requests.md
• NIP-09 Spec: /persistent/dcdev/clones/nips/09.md
• Roadmap: README.md lines 198-206
• AGENTS.md: Configuration sync requirements