mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 15:08:24 +00:00
issue: create ddbf - enhanced deletion request features
This commit is contained in:
DanConwayDev
@@ -0,0 +1,273 @@
|
|||||||
|
# Enhanced Deletion Request Features
|
||||||
|
|
||||||
|
**ID:** ddbf
|
||||||
|
|
||||||
|
**Status:** Not Started
|
||||||
|
**Priority:** Medium
|
||||||
|
**Complexity:** High
|
||||||
|
**Estimated Timeline:** 4-6 weeks
|
||||||
|
|
||||||
|
## Problem Statement
|
||||||
|
|
||||||
|
The core NIP-09 deletion request implementation (b905) is complete with Phases 1-5 delivering:
|
||||||
|
- Basic deletion with cascade for single and multi-maintainer repositories
|
||||||
|
- Git archival and background cleanup
|
||||||
|
- Multi-maintainer graph-based retention algorithm
|
||||||
|
- Recovery mechanism for accidentally deleted repositories
|
||||||
|
- Extended cascade deletion for all NIP-34 event types
|
||||||
|
|
||||||
|
This issue tracks enhancements, edge cases, and production hardening features that were deferred from the initial implementation.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
- **Requires:** b905 (Deletion Request Support) - Phases 1-5 complete
|
||||||
|
- **Blocks:** None
|
||||||
|
|
||||||
|
## Implementation Phases
|
||||||
|
|
||||||
|
### Phase 1: Background Cleanup & rust-nostr Investigation
|
||||||
|
**Goal:** Resolve cleanup timing issues and understand rust-nostr deletion behavior
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Background cleanup task investigation:
|
||||||
|
- [ ] Daily cleanup doesn't work well with 3-second retention tests
|
||||||
|
- [ ] Design alternative: trigger-based cleanup for tests?
|
||||||
|
- [ ] Or: configurable cleanup interval (separate from retention)?
|
||||||
|
- [ ] Test with both short intervals (tests) and daily (production)
|
||||||
|
- [ ] rust-nostr deletion behavior investigation:
|
||||||
|
- [ ] Check if nostr-relay-builder automatically deletes events tagged in kind 5
|
||||||
|
- [ ] Check if it stops serving events tagged in deletion requests
|
||||||
|
- [ ] If YES: need to override/disable this behavior when `deletion_request_disrespector = true`
|
||||||
|
- [ ] Document any hooks or callbacks we need to implement
|
||||||
|
- [ ] Ensure archival mode truly ignores deletions at relay library level
|
||||||
|
- [ ] Author validation enforcement:
|
||||||
|
- [ ] Verify we only honor deletion requests where author matches deleted event author
|
||||||
|
- [ ] Add comprehensive tests for author mismatch rejection
|
||||||
|
- [ ] Document this requirement clearly in NIP-11 and README
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- Cleanup timing strategy documented and tested
|
||||||
|
- rust-nostr deletion behavior fully understood
|
||||||
|
- Author validation comprehensively tested
|
||||||
|
- Documentation updated
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 2: Edge Cases & Performance Analysis
|
||||||
|
**Goal:** Document and test edge cases, analyze performance characteristics
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Max depth edge case analysis:
|
||||||
|
- [ ] Document scenarios where depth limit matters
|
||||||
|
- [ ] Recommend alternative approaches if needed
|
||||||
|
- [ ] Add configuration guidance
|
||||||
|
- [ ] Large-scale scenario testing:
|
||||||
|
- [ ] Repository with 1000+ PRs/issues
|
||||||
|
- [ ] Deep dependency chains (50+ levels)
|
||||||
|
- [ ] Performance profiling
|
||||||
|
- [ ] Memory usage analysis
|
||||||
|
- [ ] Race condition investigation:
|
||||||
|
- [ ] Deletion during active sync
|
||||||
|
- [ ] Concurrent deletions of shared repo
|
||||||
|
- [ ] Lock strategy for git archival
|
||||||
|
- [ ] Document mitigation strategies
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- Edge cases documented with recommendations
|
||||||
|
- Performance acceptable for production scale
|
||||||
|
- Race conditions understood and mitigated
|
||||||
|
- Configuration guidance complete
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 3: Blacklist Integration
|
||||||
|
**Goal:** Integrate deletion mechanism with blacklist functionality
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Blacklist retroactive deletion investigation:
|
||||||
|
- [ ] Should adding to blacklist move existing repos to 90-day holding area?
|
||||||
|
- [ ] **Decision:** YES - use same archive mechanism as deletion requests
|
||||||
|
- [ ] Detect on startup: scan main DB for repos matching blacklist
|
||||||
|
- [ ] Move to holding area with metadata marking blacklist trigger
|
||||||
|
- [ ] Same cascade delete logic as NIP-09 deletions
|
||||||
|
- [ ] Archive git data before moving to holding area
|
||||||
|
- [ ] Blacklist + disrespector mode interaction:
|
||||||
|
- [ ] **Decision:** `deletion_request_disrespector` does NOT prevent blacklist deletion
|
||||||
|
- [ ] Blacklist is moderation/operational decision, not user-initiated deletion
|
||||||
|
- [ ] Archival relays can still blacklist spam/malware/abuse
|
||||||
|
- [ ] Only NIP-09 user deletions are ignored in disrespector mode
|
||||||
|
- [ ] Blacklist removal recovery:
|
||||||
|
- [ ] Should removing from blacklist auto-restore from holding DB?
|
||||||
|
- [ ] Or require manual operator intervention?
|
||||||
|
- [ ] What about the git data archive?
|
||||||
|
- [ ] Design recovery flow for unblacklisted repos
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- Blacklist deletion working with same archival mechanism
|
||||||
|
- Disrespector mode interaction clearly defined
|
||||||
|
- Blacklist removal recovery flow designed
|
||||||
|
- All behaviors tested and documented
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 4: Manual Ejection & Delayed Archival
|
||||||
|
**Goal:** Add operator controls and delayed archival for active repositories
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Manual ejection from holding area:
|
||||||
|
- [ ] Operator needs ability to force-delete from holding area before expiry
|
||||||
|
- [ ] Use case: large repos consuming excessive storage
|
||||||
|
- [ ] Use case: confirmed malware/abuse that shouldn't be recoverable
|
||||||
|
- [ ] Design mechanism: admin CLI command? config flag? database operation?
|
||||||
|
- [ ] Should manual ejection delete git archive immediately or wait for cleanup?
|
||||||
|
- [ ] Log manual ejections for audit trail
|
||||||
|
- [ ] Delayed archival strategy design (defer full implementation):
|
||||||
|
- [ ] Activity threshold evaluation algorithm
|
||||||
|
- [ ] Count issues/PRs/patches (< 5 = low activity)
|
||||||
|
- [ ] Count unique creator pubkeys (1-2 = low activity)
|
||||||
|
- [ ] Both conditions evaluated (either triggers immediate archival)
|
||||||
|
- [ ] Grace period implementation (2 minutes default)
|
||||||
|
- [ ] Prevent accidental deletions
|
||||||
|
- [ ] Allow user to publish new announcement to cancel
|
||||||
|
- [ ] Handle cancellation during grace period
|
||||||
|
- [ ] Notification issue creation
|
||||||
|
- [ ] Sign issue with repository owner pubkey
|
||||||
|
- [ ] Content: deletion warning, 14-day timeline, affected content list
|
||||||
|
- [ ] Store notification issue reference for cleanup
|
||||||
|
- [ ] Archive delay implementation (10 days default)
|
||||||
|
- [ ] Community response window
|
||||||
|
- [ ] Scheduled task system for delayed execution
|
||||||
|
- [ ] Cancellation check before archival
|
||||||
|
- [ ] Configuration options:
|
||||||
|
- [ ] `deletion_notification_delay_secs` (default 120)
|
||||||
|
- [ ] `deletion_archive_delay_secs` (default 864000)
|
||||||
|
- [ ] `deletion_activity_threshold` (default 5)
|
||||||
|
- [ ] `deletion_creator_threshold` (default 3)
|
||||||
|
- [ ] Add config to all 4 required sources (CRITICAL):
|
||||||
|
- [ ] `src/config.rs`
|
||||||
|
- [ ] `docs/reference/configuration.md`
|
||||||
|
- [ ] `nix/module.nix`
|
||||||
|
- [ ] `.env.example`
|
||||||
|
- [ ] Timeline calculation and validation
|
||||||
|
- [ ] Total for active repos: 2min + 10days + 90days ≈ 100 days
|
||||||
|
- [ ] Low-activity repos: immediate + 90 days
|
||||||
|
- [ ] Testing considerations
|
||||||
|
- [ ] Override delays for test scenarios (5-10 seconds)
|
||||||
|
- [ ] Test immediate vs delayed paths
|
||||||
|
- [ ] Test cancellation during both grace and delay periods
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- Manual ejection mechanism designed and implemented
|
||||||
|
- Delayed archival strategy fully designed
|
||||||
|
- Configuration options added to all 4 sources
|
||||||
|
- Testing strategy defined
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 5: Metrics & Monitoring
|
||||||
|
**Goal:** Add comprehensive metrics for production monitoring
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Metrics implementation:
|
||||||
|
- [ ] `ngit_deletion_requests_total`
|
||||||
|
- [ ] `ngit_deletion_requests_processed`
|
||||||
|
- [ ] `ngit_deletion_requests_immediate` (new - low activity)
|
||||||
|
- [ ] `ngit_deletion_requests_delayed` (new - active repos)
|
||||||
|
- [ ] `ngit_deletion_notifications_created` (new)
|
||||||
|
- [ ] `ngit_deletion_requests_cancelled` (new)
|
||||||
|
- [ ] `ngit_blacklist_deletions_total`
|
||||||
|
- [ ] `ngit_holding_database_events`
|
||||||
|
- [ ] `ngit_holding_database_size_bytes`
|
||||||
|
- [ ] `ngit_archive_files_total`
|
||||||
|
- [ ] `ngit_archive_size_bytes`
|
||||||
|
- [ ] `ngit_recoveries_total`
|
||||||
|
- [ ] `ngit_permanent_deletions_total`
|
||||||
|
- [ ] `ngit_manual_ejections_total`
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- All metrics implemented and tested
|
||||||
|
- Metrics exposed via Prometheus endpoint
|
||||||
|
- Dashboard examples provided
|
||||||
|
- Alerting recommendations documented
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 6: Master Branch Integration & Documentation
|
||||||
|
**Goal:** Ensure compatibility with latest master and complete documentation
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Master branch integration review:
|
||||||
|
- [ ] Review implications of deletion requests with latest master changes
|
||||||
|
- [ ] Verify compatibility with purgatory persistence feature
|
||||||
|
- [ ] Verify compatibility with defensive measures (rate limiting)
|
||||||
|
- [ ] Test interaction between holding database and purgatory
|
||||||
|
- [ ] Ensure no conflicts with rejected events index
|
||||||
|
- [ ] Document any integration considerations
|
||||||
|
- [ ] Documentation:
|
||||||
|
- [ ] Update `docs/explanation/deletion-requests.md` with findings
|
||||||
|
- [ ] Add blacklist deletion behavior documentation
|
||||||
|
- [ ] Add manual ejection mechanism documentation
|
||||||
|
- [ ] Add delayed archival strategy documentation
|
||||||
|
- [ ] Add edge case documentation
|
||||||
|
- [ ] Performance tuning guide
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- Master branch integration verified
|
||||||
|
- All documentation complete and reviewed
|
||||||
|
- Production deployment guide ready
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 7: Advanced Deletion Features (Future Exploration)
|
||||||
|
**Goal:** Explore advanced deletion scenarios and recovery mechanisms
|
||||||
|
|
||||||
|
**Tasks:**
|
||||||
|
- [ ] Event ID deletion (vs address deletion):
|
||||||
|
- [ ] When a specific announcement or state event is deleted by event ID (`e` tag) rather than address (`a` tag)
|
||||||
|
- [ ] Should NOT cascade delete
|
||||||
|
- [ ] Instead: try to rollback to previous replaceable event version (if available)
|
||||||
|
- [ ] Only proceed with deletion if no previous version can be found
|
||||||
|
- [ ] Explore recovery mechanisms
|
||||||
|
- [ ] PR deletion handling:
|
||||||
|
- [ ] When a PR (kind 1617/1618/1619) is deleted
|
||||||
|
- [ ] For now: should NOT cascade delete dependent events
|
||||||
|
- [ ] Needs more thought - revisit later
|
||||||
|
|
||||||
|
**Exit Criteria:**
|
||||||
|
- Event ID deletion behavior designed
|
||||||
|
- PR deletion strategy defined
|
||||||
|
- Implementation plan created
|
||||||
|
|
||||||
|
## Success Criteria
|
||||||
|
|
||||||
|
1. Background cleanup timing optimized for both tests and production
|
||||||
|
2. rust-nostr deletion behavior fully understood and controlled
|
||||||
|
3. Edge cases documented with mitigation strategies
|
||||||
|
4. Performance acceptable at production scale (1000+ events)
|
||||||
|
5. Blacklist integration working with same archival mechanism
|
||||||
|
6. Manual ejection mechanism available for operators
|
||||||
|
7. Delayed archival strategy designed and documented
|
||||||
|
8. Comprehensive metrics for production monitoring
|
||||||
|
9. Master branch integration verified
|
||||||
|
10. Complete documentation for all features
|
||||||
|
|
||||||
|
## Progress
|
||||||
|
|
||||||
|
### 2026-01-15 [Session 11:30]
|
||||||
|
- **Issue Created:** Extracted Phase 6+ content from b905 issue
|
||||||
|
- Core implementation (Phases 1-5) complete in b905
|
||||||
|
- This issue tracks enhancements, edge cases, and production hardening
|
||||||
|
- 7 implementation phases defined
|
||||||
|
- Dependencies: requires b905 completion
|
||||||
|
- **Next:** Not started - waiting for b905 integration tests to complete
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- **Related Issue:** b905 (Deletion Request Support) - core implementation
|
||||||
|
- **Reference:** `docs/explanation/deletion-requests.md` - comprehensive architecture
|
||||||
|
- **Configuration:** All config changes must update 4 sources (per AGENTS.md)
|
||||||
|
- **Future Enhancements:**
|
||||||
|
- GRASP-05 archive mode integration
|
||||||
|
- Selective disrespect (based on popularity/criteria)
|
||||||
|
- Distributed archive network coordination
|
||||||
|
- Recovery notifications to repository owners
|
||||||
Reference in New Issue
Block a user