mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 15:08:24 +00:00
DanConwayDev
ee17a1703b
Archive git repositories before deletion: - Create tar.gz archives in .archive/<npub>/<identifier>-<timestamp>.tar.gz - Store metadata JSON with deletion info and expiry timestamp - Archive git data BEFORE moving events to holding database - Uses flate2 for gzip compression and tar crate for archive creation - Graceful error handling (deletion continues even if archival fails) Background cleanup task: - Spawned as tokio task in main.rs - Runs on startup (catch-up for offline periods) - Runs at configurable interval (default: 24 hours) - Deletes expired events from holding database - Removes archive tar.gz files and metadata JSON - Tracks disk space reclamation in CleanupResult Archive metadata structure: - Deletion timestamp and event ID - Maintainer pubkey and repository identifier - Archive file path for cleanup coordination - Expiry timestamp (deletion_ts + retention_secs) Tests: 21 tests covering: - Archive unit tests (8 tests) - Archive creation and compression - Extraction and integrity verification - Metadata serialization - Error handling (missing directories, corrupt data) - Cleanup unit tests (5 tests) - Expired event deletion - Archive file cleanup - Missing file handling - CleanupResult statistics - Integration tests (8 tests) - End-to-end archival workflow - Background cleanup with short retention (5 seconds) - Startup cleanup (pre-existing expired data) - Disk space reclamation verification - Multiple expired events - Preservation of non-expired events
331 lines
13 KiB
Bash
331 lines
13 KiB
Bash
# ngit-grasp Configuration
|
|
#
|
|
# Configuration Priority (highest to lowest):
|
|
# 1. CLI flags (e.g., --domain example.com)
|
|
# 2. Environment variables (e.g., NGIT_DOMAIN=example.com)
|
|
# 3. This .env file
|
|
# 4. Built-in defaults
|
|
#
|
|
# Run `ngit-grasp --help` for all CLI options
|
|
|
|
# ============================================================================
|
|
# REQUIRED
|
|
# ============================================================================
|
|
|
|
# Domain where this instance is hosted (required, used in GRASP validation)
|
|
# CLI: --domain <domain>
|
|
# No default - must be set
|
|
# NGIT_DOMAIN=
|
|
|
|
# ============================================================================
|
|
# SERVER CONFIGURATION
|
|
# ============================================================================
|
|
|
|
# Server bind address (IP:PORT)
|
|
# CLI: --bind-address <address>
|
|
# Default: 127.0.0.1:7334
|
|
# NGIT_BIND_ADDRESS=127.0.0.1:7334
|
|
|
|
# ============================================================================
|
|
# RELAY INFORMATION (NIP-11)
|
|
# ============================================================================
|
|
|
|
# Relay operator's nsec (private key) for signing and authentication
|
|
# Used for:
|
|
# - NIP-11 relay information document (pubkey field derived from this nsec)
|
|
# - NIP-42 authentication when syncing from other relays
|
|
# - Future: signing events, WoT-based rate limiting of syncing relays
|
|
#
|
|
# CLI: --relay-owner-nsec <nsec>
|
|
# Default: Loaded from/saved to .relay-owner.nsec file in current directory
|
|
# If file doesn't exist, a new key is generated and saved automatically
|
|
# NGIT_RELAY_OWNER_NSEC=nsec1...
|
|
|
|
# Relay name shown in NIP-11 information document
|
|
# CLI: --relay-name <name>
|
|
# Default: ${domain} grasp relay (e.g., "gitnostr.com grasp relay")
|
|
# NGIT_RELAY_NAME=My GRASP Relay
|
|
|
|
# Relay description shown in NIP-11 information document
|
|
# CLI: --relay-description <description>
|
|
# Default: Git Nostr Relay - a grasp implementation
|
|
# NGIT_RELAY_DESCRIPTION="A GRASP-compliant Git relay with Nostr authorization"
|
|
|
|
# ============================================================================
|
|
# STORAGE
|
|
# ============================================================================
|
|
|
|
# Path to store Git repositories
|
|
# CLI: --git-data-path <path>
|
|
# Default: ./data/git
|
|
# NGIT_GIT_DATA_PATH=./data/git
|
|
|
|
# Path to store Nostr relay data
|
|
# CLI: --relay-data-path <path>
|
|
# Default: ./data/relay
|
|
# NGIT_RELAY_DATA_PATH=./data/relay
|
|
|
|
# Database backend for Nostr events
|
|
# CLI: --database-backend <backend>
|
|
# Options: lmdb, memory, nostrdb
|
|
# Default: lmdb
|
|
# - lmdb: LMDB backend (persistent, general purpose) - RECOMMENDED
|
|
# - memory: In-memory database (fastest, no persistence, uses temp dirs)
|
|
# - nostrdb: NostrDB backend (persistent, Nostr-optimized) [Not yet implemented]
|
|
#
|
|
# Note: When using 'memory' backend, git_data_path and relay_data_path
|
|
# are automatically set to temporary directories for ephemeral testing.
|
|
# NGIT_DATABASE_BACKEND=lmdb
|
|
|
|
# ============================================================================
|
|
# METRICS
|
|
# ============================================================================
|
|
|
|
# Enable Prometheus metrics endpoint at /metrics
|
|
# CLI: --metrics-enabled
|
|
# Default: true
|
|
# NGIT_METRICS_ENABLED=true
|
|
|
|
# Connections per IP before flagging as potential abuse in metrics
|
|
# (display only, no rate limiting - purely for monitoring visibility)
|
|
# CLI: --metrics-connection-per-ip-abuse-threshold <count>
|
|
# Default: 10
|
|
# NGIT_METRICS_CONNECTION_PER_IP_ABUSE_THRESHOLD=10
|
|
|
|
# Number of top bandwidth repositories to track in metrics
|
|
# CLI: --metrics-top-n-repos <count>
|
|
# Default: 10
|
|
# NGIT_METRICS_TOP_N_REPOS=10
|
|
|
|
# ============================================================================
|
|
# LOGGING
|
|
# ============================================================================
|
|
|
|
# Rust log level (not a ngit-grasp config, but useful for debugging)
|
|
# Options: error, warn, info, debug, trace
|
|
# RUST_LOG=info
|
|
|
|
# ============================================================================
|
|
# PROACTIVE SYNC (GRASP-02)
|
|
# ============================================================================
|
|
|
|
# Bootstrap relay URL for initial sync (optional)
|
|
# Additional relays are automatically discovered from repository announcements
|
|
# that list our service domain.
|
|
# CLI: --sync-bootstrap-relay-url <url>
|
|
# Default: (none - relay discovery from stored announcements only)
|
|
# NGIT_SYNC_BOOTSTRAP_RELAY_URL=wss://relay.example.com
|
|
|
|
# Maximum backoff time in seconds for sync relay reconnection
|
|
# CLI: --sync-max-backoff-secs <seconds>
|
|
# Default: 3600 (1 hour)
|
|
# NGIT_SYNC_MAX_BACKOFF_SECS=3600
|
|
|
|
# Interval in seconds for checking disconnected relays and attempting reconnection
|
|
# Set to lower value for faster reconnection testing
|
|
# CLI: --sync-disconnect-check-interval-secs <seconds>
|
|
# Default: 60
|
|
# NGIT_SYNC_DISCONNECT_CHECK_INTERVAL_SECS=60
|
|
|
|
# Base backoff time in seconds for relay reconnection
|
|
# Used for exponential backoff: base * 2^(failures-1)
|
|
# Set to 1 for faster test cycles
|
|
# Note: The connection timeout is capped at this value
|
|
# CLI: --sync-base-backoff-secs <seconds>
|
|
# Default: 5
|
|
# NGIT_SYNC_BASE_BACKOFF_SECS=5
|
|
|
|
# Disable NIP-77 negentropy sync (use REQ+EOSE instead)
|
|
# When enabled, sync will use REQ+EOSE instead of negentropy for history sync
|
|
# Primarily useful for testing that sync works without negentropy support
|
|
# CLI: --sync-disable-negentropy
|
|
# Default: false
|
|
# NGIT_SYNC_DISABLE_NEGENTROPY=false
|
|
|
|
# ============================================================================
|
|
# REJECTED EVENTS INDEX
|
|
# ============================================================================
|
|
|
|
# Hot cache duration in seconds for rejected announcements
|
|
# Stores full event objects for immediate re-processing when dependencies resolve
|
|
# Too short (<30s): Miss events from slow relays
|
|
# Too long (>5min): Waste memory
|
|
# CLI: --rejected-hot-cache-duration-secs <seconds>
|
|
# Default: 120 (2 minutes)
|
|
# NGIT_REJECTED_HOT_CACHE_DURATION_SECS=120
|
|
|
|
# Cold index expiry in seconds for rejected announcements
|
|
# Stores metadata only to prevent repeated downloads of rejected events
|
|
# CLI: --rejected-cold-index-expiry-secs <seconds>
|
|
# Default: 604800 (7 days)
|
|
# NGIT_REJECTED_COLD_INDEX_EXPIRY_SECS=604800
|
|
|
|
# Hours before removing relay from naughty list
|
|
# Relays with persistent infrastructure issues (DNS, TLS, protocol errors) are
|
|
# tracked separately and retried after this expiration period
|
|
# CLI: --naughty-list-expiration-hours <hours>
|
|
# Default: 12
|
|
# NGIT_NAUGHTY_LIST_EXPIRATION_HOURS=12
|
|
|
|
# ============================================================================
|
|
# GRASP-05 ARCHIVE MODE
|
|
# ============================================================================
|
|
|
|
# Accept all repository announcements (⚠️ WARNING: Storage/bandwidth risk)
|
|
# When enabled, any repository can be mirrored to this relay
|
|
# CLI: --archive-all
|
|
# Default: false
|
|
# NGIT_ARCHIVE_ALL=false
|
|
|
|
# Whitelist specific repos/pubkeys/identifiers for archiving
|
|
# Comma-separated list supporting three formats:
|
|
# <npub> - Archive all repos from this pubkey
|
|
# <npub>/<identifier> - Archive specific repo
|
|
# <identifier> - Archive repos with this identifier (any pubkey)
|
|
#
|
|
# CLI: --archive-whitelist <list>
|
|
# Default: (empty)
|
|
# Examples:
|
|
# NGIT_ARCHIVE_WHITELIST=npub1alice...
|
|
# NGIT_ARCHIVE_WHITELIST=npub1alice...,npub1bob.../linux
|
|
# NGIT_ARCHIVE_WHITELIST=bitcoin-core,linux,rust
|
|
# NGIT_ARCHIVE_WHITELIST=
|
|
|
|
# Archive read-only mode (relay is read-only sync of archived repositories)
|
|
# When true:
|
|
# - NIP-11 includes GRASP-05 in supported_grasps
|
|
# - NIP-11 curation field describes archive scope
|
|
# - Repository announcements not listing this service are accepted per whitelist/archive-all
|
|
# When false:
|
|
# - Archive mode disabled (standard GRASP-01 operation)
|
|
#
|
|
# CLI: --archive-read-only
|
|
# Default: true if NGIT_ARCHIVE_ALL or NGIT_ARCHIVE_WHITELIST is set, false otherwise
|
|
# Note: Setting to true without archive config causes startup error
|
|
# Note: Cannot be used with NGIT_REPOSITORY_WHITELIST (mutually exclusive)
|
|
# NGIT_ARCHIVE_READ_ONLY=
|
|
|
|
# ============================================================================
|
|
# REPOSITORY WHITELIST
|
|
# ============================================================================
|
|
|
|
# Whitelist specific repos/pubkeys/identifiers for GRASP-01 acceptance
|
|
# Comma-separated list supporting three formats (same as archive whitelist):
|
|
# <npub> - Accept all repos from this pubkey (if they list our service)
|
|
# <npub>/<identifier> - Accept specific repo (if it lists our service)
|
|
# <identifier> - Accept repos with this identifier (if they list our service)
|
|
#
|
|
# Difference from archive whitelist:
|
|
# - Repository whitelist: Announcements MUST list our service AND match whitelist
|
|
# - Archive whitelist: Announcements don't need to list our service, just match whitelist
|
|
#
|
|
# CLI: --repository-whitelist <list>
|
|
# Default: (empty - all repos listing our service are accepted)
|
|
# Examples:
|
|
# NGIT_REPOSITORY_WHITELIST=npub1alice...
|
|
# NGIT_REPOSITORY_WHITELIST=npub1alice...,npub1bob.../linux
|
|
# NGIT_REPOSITORY_WHITELIST=bitcoin-core,linux,rust
|
|
# Note: Cannot be used with NGIT_ARCHIVE_READ_ONLY=true (mutually exclusive)
|
|
# Note: When set, NIP-11 curation field will indicate curated repository acceptance
|
|
# NGIT_REPOSITORY_WHITELIST=
|
|
|
|
# ============================================================================
|
|
# REPOSITORY BLACKLIST
|
|
# ============================================================================
|
|
|
|
# Blacklist specific repos/pubkeys/identifiers to reject
|
|
# Comma-separated list supporting three formats (same as whitelist formats):
|
|
# <npub> - Block all repos from this pubkey
|
|
# <npub>/<identifier> - Block specific repo
|
|
# <identifier> - Block repos with this identifier (any pubkey)
|
|
#
|
|
# Blacklist takes precedence over ALL whitelists:
|
|
# - Blacklisted repos are rejected even if they match archive or repository whitelists
|
|
# - Blacklisted repos are rejected even if they list our service
|
|
#
|
|
# Rejection reasons indicate the match type:
|
|
# - "Repository owner <npub> is blacklisted" (npub format)
|
|
# - "Repository <npub>/<identifier> is blacklisted" (npub/identifier format)
|
|
# - "Repository identifier <identifier> is blacklisted" (identifier format)
|
|
#
|
|
# CLI: --repository-blacklist <list>
|
|
# Default: (empty - no repositories are blacklisted)
|
|
# Examples:
|
|
# NGIT_REPOSITORY_BLACKLIST=npub1spam...
|
|
# NGIT_REPOSITORY_BLACKLIST=npub1alice.../bad-repo
|
|
# NGIT_REPOSITORY_BLACKLIST=malware-repo,spam-repo
|
|
# NGIT_REPOSITORY_BLACKLIST=
|
|
|
|
# ============================================================================
|
|
# EVENT BLACKLIST
|
|
# ============================================================================
|
|
|
|
# Blacklist events from specific authors (npubs)
|
|
# Comma-separated list of npubs whose events are rejected
|
|
# ALL events from these authors are blocked from both relay storage and purgatory
|
|
#
|
|
# Event blacklist takes precedence over ALL other validation:
|
|
# - Blacklisted events are rejected before any other policy checks
|
|
# - Applies to announcements, state events, PRs, and all other event types
|
|
# - Events never reach purgatory (rejected immediately)
|
|
#
|
|
# Rejection reason:
|
|
# - "Event author <npub> is blacklisted"
|
|
#
|
|
# CLI: --event-blacklist <list>
|
|
# Default: (empty - no events are blacklisted by author)
|
|
# Examples:
|
|
# NGIT_EVENT_BLACKLIST=npub1spam...
|
|
# NGIT_EVENT_BLACKLIST=npub1spam...,npub1abuser...
|
|
# NGIT_EVENT_BLACKLIST=
|
|
|
|
# ============================================================================
|
|
# RATE LIMITING & DOS PROTECTION
|
|
# ============================================================================
|
|
|
|
# Maximum total connections to the relay
|
|
# Prevents connection exhaustion DoS attacks
|
|
# CLI: --max-connections <count>
|
|
# Default: 500
|
|
# NGIT_MAX_CONNECTIONS=500
|
|
|
|
# ============================================================================
|
|
# NIP-09 DELETION REQUEST CONFIGURATION
|
|
# ============================================================================
|
|
|
|
# Archival relay mode: ignore NIP-09 deletion requests
|
|
# When enabled, all deletion requests are ignored and events are retained permanently.
|
|
# Prevents "left-pad" scenarios where critical code is deleted from the network.
|
|
# Useful for archival relays that prioritize data preservation.
|
|
#
|
|
# CLI: --deletion-request-disrespector
|
|
# Default: false
|
|
# NGIT_DELETION_REQUEST_DISRESPECTOR=false
|
|
|
|
# Archive retention period in seconds for deleted events and git data
|
|
# How long to keep archived events and git data before permanent deletion.
|
|
# Default: 7776000 seconds (90 days)
|
|
# Set to lower values for testing (e.g., 60 for 1 minute)
|
|
# Has no effect if NGIT_DELETION_REQUEST_DISRESPECTOR=true (events never deleted)
|
|
#
|
|
# CLI: --archive-retention-secs <seconds>
|
|
# Default: 7776000 (90 days)
|
|
# Examples:
|
|
# NGIT_ARCHIVE_RETENTION_SECS=7776000 # 90 days (default)
|
|
# NGIT_ARCHIVE_RETENTION_SECS=604800 # 7 days
|
|
# NGIT_ARCHIVE_RETENTION_SECS=60 # 1 minute (testing)
|
|
# NGIT_ARCHIVE_RETENTION_SECS=7776000
|
|
|
|
# Archive cleanup interval in seconds for background cleanup task
|
|
# How often to check for and delete expired archived events and git data.
|
|
# Default: 86400 seconds (24 hours)
|
|
# Set to lower values for testing (e.g., 5 for 5 seconds)
|
|
#
|
|
# CLI: --archive-cleanup-interval-secs <seconds>
|
|
# Default: 86400 (24 hours)
|
|
# Examples:
|
|
# NGIT_ARCHIVE_CLEANUP_INTERVAL_SECS=86400 # 24 hours (default)
|
|
# NGIT_ARCHIVE_CLEANUP_INTERVAL_SECS=3600 # 1 hour
|
|
# NGIT_ARCHIVE_CLEANUP_INTERVAL_SECS=5 # 5 seconds (testing)
|
|
# NGIT_ARCHIVE_CLEANUP_INTERVAL_SECS=86400
|