ngit-grasp/ddbf-enhance-deletion-requests.md

# Enhanced Deletion Request Features

**ID:** ddbf

**Status:** Not Started
**Priority:** Medium
**Complexity:** High
**Estimated Timeline:** 4-6 weeks

## Problem Statement

The core NIP-09 deletion request implementation (b905) is complete with Phases 1-5 delivering:
- Basic deletion with cascade for single and multi-maintainer repositories
- Git archival and background cleanup
- Multi-maintainer graph-based retention algorithm
- Recovery mechanism for accidentally deleted repositories
- Extended cascade deletion for all NIP-34 event types

This issue tracks enhancements, edge cases, and production hardening features that were deferred from the initial implementation.

## Dependencies

- **Requires:** b905 (Deletion Request Support) - Phases 1-5 complete
- **Blocks:** None

## Implementation Phases

### Phase 1: Background Cleanup & rust-nostr Investigation
**Goal:** Resolve cleanup timing issues and understand rust-nostr deletion behavior

**Tasks:**
- [ ] Background cleanup task investigation:
  - [ ] Daily cleanup doesn't work well with 3-second retention tests
  - [ ] Design alternative: trigger-based cleanup for tests?
  - [ ] Or: configurable cleanup interval (separate from retention)?
  - [ ] Test with both short intervals (tests) and daily (production)
- [ ] rust-nostr deletion behavior investigation:
  - [ ] Check if nostr-relay-builder automatically deletes events tagged in kind 5
  - [ ] Check if it stops serving events tagged in deletion requests
  - [ ] If YES: need to override/disable this behavior when `deletion_request_disrespector = true`
  - [ ] Document any hooks or callbacks we need to implement
  - [ ] Ensure archival mode truly ignores deletions at relay library level
- [ ] Author validation enforcement:
  - [ ] Verify we only honor deletion requests where author matches deleted event author
  - [ ] Add comprehensive tests for author mismatch rejection
  - [ ] Document this requirement clearly in NIP-11 and README

**Exit Criteria:**
- Cleanup timing strategy documented and tested
- rust-nostr deletion behavior fully understood
- Author validation comprehensively tested
- Documentation updated

---

### Phase 2: Edge Cases & Performance Analysis
**Goal:** Document and test edge cases, analyze performance characteristics

**Tasks:**
- [ ] Max depth edge case analysis:
  - [ ] Document scenarios where depth limit matters
  - [ ] Recommend alternative approaches if needed
  - [ ] Add configuration guidance
- [ ] Large-scale scenario testing:
  - [ ] Repository with 1000+ PRs/issues
  - [ ] Deep dependency chains (50+ levels)
  - [ ] Performance profiling
  - [ ] Memory usage analysis
- [ ] Race condition investigation:
  - [ ] Deletion during active sync
  - [ ] Concurrent deletions of shared repo
  - [ ] Lock strategy for git archival
  - [ ] Document mitigation strategies

**Exit Criteria:**
- Edge cases documented with recommendations
- Performance acceptable for production scale
- Race conditions understood and mitigated
- Configuration guidance complete

---

### Phase 3: Blacklist Integration
**Goal:** Integrate deletion mechanism with blacklist functionality

**Tasks:**
- [ ] Blacklist retroactive deletion investigation:
  - [ ] Should adding to blacklist move existing repos to 90-day holding area?
  - [ ] **Decision:** YES - use same archive mechanism as deletion requests
  - [ ] Detect on startup: scan main DB for repos matching blacklist
  - [ ] Move to holding area with metadata marking blacklist trigger
  - [ ] Same cascade delete logic as NIP-09 deletions
  - [ ] Archive git data before moving to holding area
- [ ] Blacklist + disrespector mode interaction:
  - [ ] **Decision:** `deletion_request_disrespector` does NOT prevent blacklist deletion
  - [ ] Blacklist is moderation/operational decision, not user-initiated deletion
  - [ ] Archival relays can still blacklist spam/malware/abuse
  - [ ] Only NIP-09 user deletions are ignored in disrespector mode
- [ ] Blacklist removal recovery:
  - [ ] Should removing from blacklist auto-restore from holding DB?
  - [ ] Or require manual operator intervention?
  - [ ] What about the git data archive?
  - [ ] Design recovery flow for unblacklisted repos

**Exit Criteria:**
- Blacklist deletion working with same archival mechanism
- Disrespector mode interaction clearly defined
- Blacklist removal recovery flow designed
- All behaviors tested and documented

---

### Phase 4: Manual Ejection & Delayed Archival
**Goal:** Add operator controls and delayed archival for active repositories

**Tasks:**
- [ ] Manual ejection from holding area:
  - [ ] Operator needs ability to force-delete from holding area before expiry
  - [ ] Use case: large repos consuming excessive storage
  - [ ] Use case: confirmed malware/abuse that shouldn't be recoverable
  - [ ] Design mechanism: admin CLI command? config flag? database operation?
  - [ ] Should manual ejection delete git archive immediately or wait for cleanup?
  - [ ] Log manual ejections for audit trail
- [ ] Delayed archival strategy design (defer full implementation):
  - [ ] Activity threshold evaluation algorithm
    - [ ] Count issues/PRs/patches (< 5 = low activity)
    - [ ] Count unique creator pubkeys (1-2 = low activity)
    - [ ] Both conditions evaluated (either triggers immediate archival)
  - [ ] Grace period implementation (2 minutes default)
    - [ ] Prevent accidental deletions
    - [ ] Allow user to publish new announcement to cancel
    - [ ] Handle cancellation during grace period
  - [ ] Notification issue creation
    - [ ] Sign issue with repository owner pubkey
    - [ ] Content: deletion warning, 14-day timeline, affected content list
    - [ ] Store notification issue reference for cleanup
  - [ ] Archive delay implementation (10 days default)
    - [ ] Community response window
    - [ ] Scheduled task system for delayed execution
    - [ ] Cancellation check before archival
  - [ ] Configuration options:
    - [ ] `deletion_notification_delay_secs` (default 120)
    - [ ] `deletion_archive_delay_secs` (default 864000)
    - [ ] `deletion_activity_threshold` (default 5)
    - [ ] `deletion_creator_threshold` (default 3)
  - [ ] Add config to all 4 required sources (CRITICAL):
    - [ ] `src/config.rs`
    - [ ] `docs/reference/configuration.md`
    - [ ] `nix/module.nix`
    - [ ] `.env.example`
  - [ ] Timeline calculation and validation
    - [ ] Total for active repos: 2min + 10days + 90days ≈ 100 days
    - [ ] Low-activity repos: immediate + 90 days
  - [ ] Testing considerations
    - [ ] Override delays for test scenarios (5-10 seconds)
    - [ ] Test immediate vs delayed paths
    - [ ] Test cancellation during both grace and delay periods

**Exit Criteria:**
- Manual ejection mechanism designed and implemented
- Delayed archival strategy fully designed
- Configuration options added to all 4 sources
- Testing strategy defined

---

### Phase 5: Metrics & Monitoring
**Goal:** Add comprehensive metrics for production monitoring

**Tasks:**
- [ ] Metrics implementation:
  - [ ] `ngit_deletion_requests_total`
  - [ ] `ngit_deletion_requests_processed`
  - [ ] `ngit_deletion_requests_immediate` (new - low activity)
  - [ ] `ngit_deletion_requests_delayed` (new - active repos)
  - [ ] `ngit_deletion_notifications_created` (new)
  - [ ] `ngit_deletion_requests_cancelled` (new)
  - [ ] `ngit_blacklist_deletions_total`
  - [ ] `ngit_holding_database_events`
  - [ ] `ngit_holding_database_size_bytes`
  - [ ] `ngit_archive_files_total`
  - [ ] `ngit_archive_size_bytes`
  - [ ] `ngit_recoveries_total`
  - [ ] `ngit_permanent_deletions_total`
  - [ ] `ngit_manual_ejections_total`

**Exit Criteria:**
- All metrics implemented and tested
- Metrics exposed via Prometheus endpoint
- Dashboard examples provided
- Alerting recommendations documented

---

### Phase 6: Master Branch Integration & Documentation
**Goal:** Ensure compatibility with latest master and complete documentation

**Tasks:**
- [ ] Master branch integration review:
  - [ ] Review implications of deletion requests with latest master changes
  - [ ] Verify compatibility with purgatory persistence feature
  - [ ] Verify compatibility with defensive measures (rate limiting)
  - [ ] Test interaction between holding database and purgatory
  - [ ] Ensure no conflicts with rejected events index
  - [ ] Document any integration considerations
- [ ] Documentation:
  - [ ] Update `docs/explanation/deletion-requests.md` with findings
  - [ ] Add blacklist deletion behavior documentation
  - [ ] Add manual ejection mechanism documentation
  - [ ] Add delayed archival strategy documentation
  - [ ] Add edge case documentation
  - [ ] Performance tuning guide

**Exit Criteria:**
- Master branch integration verified
- All documentation complete and reviewed
- Production deployment guide ready

---

### Phase 7: Advanced Deletion Features (Future Exploration)
**Goal:** Explore advanced deletion scenarios and recovery mechanisms

**Tasks:**
- [ ] Event ID deletion (vs address deletion):
  - [ ] When a specific announcement or state event is deleted by event ID (`e` tag) rather than address (`a` tag)
  - [ ] Should NOT cascade delete
  - [ ] Instead: try to rollback to previous replaceable event version (if available)
  - [ ] Only proceed with deletion if no previous version can be found
  - [ ] Explore recovery mechanisms
- [ ] PR deletion handling:
  - [ ] When a PR (kind 1617/1618/1619) is deleted
  - [ ] For now: should NOT cascade delete dependent events
  - [ ] Needs more thought - revisit later

**Exit Criteria:**
- Event ID deletion behavior designed
- PR deletion strategy defined
- Implementation plan created

## Success Criteria

1. Background cleanup timing optimized for both tests and production
2. rust-nostr deletion behavior fully understood and controlled
3. Edge cases documented with mitigation strategies
4. Performance acceptable at production scale (1000+ events)
5. Blacklist integration working with same archival mechanism
6. Manual ejection mechanism available for operators
7. Delayed archival strategy designed and documented
8. Comprehensive metrics for production monitoring
9. Master branch integration verified
10. Complete documentation for all features

## Progress

### 2026-01-15 [Session 11:30]
- **Issue Created:** Extracted Phase 6+ content from b905 issue
  - Core implementation (Phases 1-5) complete in b905
  - This issue tracks enhancements, edge cases, and production hardening
  - 7 implementation phases defined
  - Dependencies: requires b905 completion
- **Next:** Not started - waiting for b905 integration tests to complete

## Notes

- **Related Issue:** b905 (Deletion Request Support) - core implementation
- **Reference:** `docs/explanation/deletion-requests.md` - comprehensive architecture
- **Configuration:** All config changes must update 4 sources (per AGENTS.md)
- **Future Enhancements:**
  - GRASP-05 archive mode integration
  - Selective disrespect (based on popularity/criteria)
  - Distributed archive network coordination
  - Recovery notifications to repository owners