laantungir/ngit-grasp
1
0
Fork 0
mirror of https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git synced 2026-10-05 15:08:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: configure bounded deletion request retention

Browse Source
This commit is contained in:
DanConwayDev
2026-07-16 13:59:49 +01:00
parent 4bd46d704e
commit d428307ee3
5 changed files with 413 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+29 -2
View File
@@ -170,7 +170,7 @@
# NGIT_NAUGHTY_LIST_EXPIRATION_HOURS=12
# ============================================================================
# HOLDING DB CLEANUP
# HOLDING DB AND DELETION-REQUEST CLEANUP
# ============================================================================
# Retention window in seconds for deleted events kept in holding DB
@@ -180,7 +180,8 @@
# Default: 7776000 (90 days)
# NGIT_HOLDING_RETENTION_SECS=7776000
# Interval in seconds between holding DB background cleanup passes
# Interval in seconds between holding DB and deletion-request retention background
# cleanup passes. This cadence does not alter timestamp-derived retention deadlines.
# Must be greater than 0
# CLI: --holding-cleanup-interval-secs <seconds>
# Default: 86400 (24 hours)
@@ -280,6 +281,32 @@
# Default: false
# NGIT_DELETION_REQUEST_DISRESPECTOR=false
# Deletion-request retention applies to accepted NIP-09 deletion requests and
# NIP-62 request-to-vanish events. All values are integer seconds. Unused clocks
# start at relay-observed first_seen_at; used clocks start at last_used_at.
# "Additional" periods begin after their corresponding served period ends.
# Used requests remain served indefinitely when deletion-request-disrespector is true.
# How long an unused request remains served from first_seen_at
# CLI: --deletion-request-retention-unused-served-secs <seconds>
# Default: 2592000 (30 days)
# NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS=2592000
# Additional time an unused request remains unserved but eligible to gate
# CLI: --deletion-request-retention-unused-unserved-gating-additional-secs <seconds>
# Default: 15552000 (180 days)
# NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS=15552000
# Normal-mode time a used request remains served after last_used_at
# CLI: --deletion-request-retention-used-served-after-last-used-secs <seconds>
# Default: 23328000 (270 days; 9 fixed 30-day months)
# NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS=23328000
# Additional normal-mode time a used request remains unserved but continues gating
# CLI: --deletion-request-retention-used-unserved-gating-additional-secs <seconds>
# Default: 7776000 (90 days; 3 fixed 30-day months)
# NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS=7776000
# ============================================================================
# REPOSITORY WHITELIST
# ============================================================================
docs/explanation/repository-lifecycle.md
+2 -1
View File
@@ -108,7 +108,8 @@ The four lifecycle durations should be operator-configurable, with defaults of
30 days for unused serving, a further 180 days for unused gating, 9 months for
used serving, and a further 3 months for used gating. Configuration validation
must preserve the ordering of each served period followed by its unserved gate
period; cleanup cadence is operational and must not alter the timestamp-derived
period. The existing holding cleanup cadence also schedules deletion-request
retention cleanup; it is operational and must not alter the timestamp-derived
deadlines.
### Multiple matching requests
docs/reference/configuration.md
+82 -3
View File
@@ -492,9 +492,9 @@ NGIT_REJECTED_COLD_INDEX_EXPIRY_SECS=1209600
---
### Holding DB Cleanup Configuration
### Holding DB and Deletion-Request Cleanup Configuration
These options control retention and cleanup cadence for deleted events archived in the holding database.
These options control retention for deleted events archived in the holding database and the shared cleanup cadence for holding and deletion-request retention records.
#### `NGIT_HOLDING_RETENTION_SECS`
@@ -522,7 +522,7 @@ NGIT_HOLDING_RETENTION_SECS=2592000
#### `NGIT_HOLDING_CLEANUP_INTERVAL_SECS`
**Description:** Interval between periodic holding DB expiration cleanup passes
**Description:** Interval between periodic holding DB expiration cleanup passes and deletion-request retention cleanup passes
**Type:** Integer (seconds)
**Default:** `86400` (24 hours)
**Required:** No
@@ -541,6 +541,7 @@ NGIT_HOLDING_CLEANUP_INTERVAL_SECS=21600
- Must be greater than 0
- Smaller values clean up expired records sooner at the cost of more background work
- This interval only determines how soon expired records are observed and removed. It does not change any timestamp-derived holding or deletion-request retention deadline.
---
@@ -1181,6 +1182,84 @@ for the full lifecycle design rationale.
---
### Deletion-Request Retention (NIP-09 and NIP-62)
These options govern the bounded lifecycle of accepted NIP-09 deletion requests and NIP-62 request-to-vanish events. All values are integer seconds. The used-request defaults express months as fixed 30-day periods, not calendar months.
The relay derives unused deadlines from relay-observed `first_seen_at`, never from the client-controlled event `created_at`. It derives used deadlines from `last_used_at`. Each `...ADDITIONAL...` option begins only after its corresponding served period ends; it is not a total retention duration.
#### `NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS`
- **Description:** How long an unused deletion or vanish request remains served from relay-observed `first_seen_at`
- **Type:** Positive integer (seconds)
- **Default:** `2592000` (30 days)
- **Required:** No
- **CLI:** `--deletion-request-retention-unused-served-secs`
```bash
# Default: serve an unused request for 30 days from first_seen_at
NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS=2592000
```
After this served period, an unused request enters its additional unserved/gating period.
---
#### `NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS`
- **Description:** Additional time after unused serving ends that an unused deletion or vanish request remains unserved but eligible to gate admission
- **Type:** Positive integer (seconds)
- **Default:** `15552000` (180 days)
- **Required:** No
- **CLI:** `--deletion-request-retention-unused-unserved-gating-additional-secs`
```bash
# Default: retain an unused request as an unserved gate for a further 180 days
NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS=15552000
```
The unused request expires after `unused served + this additional period` from `first_seen_at`. Retained disrespector and non-targeting NIP-62 records do not enforce a local admission gate.
---
#### `NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS`
- **Description:** In normal mode, how long a used deletion or vanish request remains served after `last_used_at`
- **Type:** Positive integer (seconds)
- **Default:** `23328000` (270 days; 9 fixed 30-day months)
- **Required:** No
- **CLI:** `--deletion-request-retention-used-served-after-last-used-secs`
```bash
# Default: serve a used request for 270 days after last_used_at
NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS=23328000
```
Every successful use resets `last_used_at` and restarts this served period. When `NGIT_DELETION_REQUEST_DISRESPECTOR=true`, used requests remain served indefinitely, so this normal-mode duration does not expire them.
---
#### `NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS`
- **Description:** Additional normal-mode time after used serving ends that a used deletion or vanish request remains unserved but continues gating admission
- **Type:** Positive integer (seconds)
- **Default:** `7776000` (90 days; 3 fixed 30-day months)
- **Required:** No
- **CLI:** `--deletion-request-retention-used-unserved-gating-additional-secs`
```bash
# Default: continue gating for a further 90 days after used serving ends
NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS=7776000
```
The used request expires after `used served + this additional period` from `last_used_at`. When `NGIT_DELETION_REQUEST_DISRESPECTOR=true`, used requests remain served indefinitely, so this normal-mode additional period does not expire their local record.
**Validation:** Each deletion-request retention duration must be greater than zero. Each served and additional-gating pair must also fit in an unsigned 64-bit second duration when combined.
**Cleanup cadence:** `NGIT_HOLDING_CLEANUP_INTERVAL_SECS` schedules both holding DB and deletion-request retention cleanup passes. Cleanup evaluates deadlines derived from `first_seen_at` and `last_used_at`; changing its cadence never changes those deadlines.
---
### Rate Limiting & DoS Protection
#### `NGIT_MAX_CONNECTIONS`
nix/module.nix
+52 -1
View File
@@ -189,7 +189,50 @@ let
type = types.int;
default = 86400;
description =
"Interval in seconds between holding DB cleanup passes (default: 24 hours)";
"Interval in seconds between holding DB and deletion-request retention cleanup passes (default: 24 hours)";
};
deletionRequestRetention = {
unusedServedSecs = mkOption {
type = types.ints.positive;
default = 2592000;
description = ''
Time in seconds an unused deletion or vanish request remains served
from relay-observed first_seen_at (default: 30 days).
'';
};
unusedUnservedGatingAdditionalSecs = mkOption {
type = types.ints.positive;
default = 15552000;
description = ''
Additional time in seconds after unused serving ends that a deletion
or vanish request remains unserved but eligible to gate admission
(default: 180 days).
'';
};
usedServedAfterLastUsedSecs = mkOption {
type = types.ints.positive;
default = 23328000;
description = ''
Normal-mode time in seconds a used deletion or vanish request
remains served after last_used_at (default: 270 days, 9 fixed
30-day months). Used requests remain served indefinitely when
deletionRequestDisrespector is enabled.
'';
};
usedUnservedGatingAdditionalSecs = mkOption {
type = types.ints.positive;
default = 7776000;
description = ''
Additional normal-mode time in seconds after used serving ends that
a deletion or vanish request remains unserved but continues gating
admission (default: 90 days, 3 fixed 30-day months). This duration
does not expire used requests when deletionRequestDisrespector is enabled.
'';
};
};
archiveAll = mkOption {
@@ -396,6 +439,14 @@ let
NGIT_HOLDING_RETENTION_SECS = toString cfg.holdingRetentionSecs;
NGIT_HOLDING_CLEANUP_INTERVAL_SECS =
toString cfg.holdingCleanupIntervalSecs;
NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS =
toString cfg.deletionRequestRetention.unusedServedSecs;
NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS =
toString cfg.deletionRequestRetention.unusedUnservedGatingAdditionalSecs;
NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS =
toString cfg.deletionRequestRetention.usedServedAfterLastUsedSecs;
NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS =
toString cfg.deletionRequestRetention.usedUnservedGatingAdditionalSecs;
NGIT_ARCHIVE_ALL = if cfg.archiveAll then "true" else "false";
NGIT_ARCHIVE_WHITELIST = concatStringsSep "," cfg.archiveWhitelist;
NGIT_ARCHIVE_GRASP_SERVICES =
src/config.rs
+248
View File
@@ -6,6 +6,14 @@ use std::fs;
use std::path::PathBuf;
use std::time::Duration;
const DEFAULT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS: u64 = 30 * 24 * 60 * 60;
const DEFAULT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS: u64 =
180 * 24 * 60 * 60;
// Retention months are fixed 30-day periods, avoiding calendar-month ambiguity.
const DEFAULT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS: u64 = 270 * 24 * 60 * 60;
const DEFAULT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS: u64 =
90 * 24 * 60 * 60;
/// Whitelist entry for repository/archive filtering
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
@@ -445,6 +453,38 @@ pub struct Config {
)]
pub holding_cleanup_interval_secs: u64,
/// How long an unused deletion or vanish request remains served after relay-observed first_seen_at.
#[arg(
long = "deletion-request-retention-unused-served-secs",
env = "NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS",
default_value_t = DEFAULT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS
)]
pub deletion_request_retention_unused_served_secs: u64,
/// Additional time an unused deletion or vanish request remains unserved but eligible to gate.
#[arg(
long = "deletion-request-retention-unused-unserved-gating-additional-secs",
env = "NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS",
default_value_t = DEFAULT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS
)]
pub deletion_request_retention_unused_unserved_gating_additional_secs: u64,
/// Normal-mode time a used deletion or vanish request remains served after last_used_at.
#[arg(
long = "deletion-request-retention-used-served-after-last-used-secs",
env = "NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS",
default_value_t = DEFAULT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS
)]
pub deletion_request_retention_used_served_after_last_used_secs: u64,
/// Additional normal-mode time a used request remains unserved but continues gating after serving ends.
#[arg(
long = "deletion-request-retention-used-unserved-gating-additional-secs",
env = "NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS",
default_value_t = DEFAULT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS
)]
pub deletion_request_retention_used_unserved_gating_additional_secs: u64,
/// Enable GRASP-05 archive mode: accept all announcements regardless of listing (WARNING: storage risk)
#[arg(long, env = "NGIT_ARCHIVE_ALL", default_value_t = false)]
pub archive_all: bool,
@@ -695,6 +735,34 @@ impl Config {
));
}
Self::validate_deletion_request_retention_duration(
self.deletion_request_retention_unused_served_secs,
"NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS",
)?;
Self::validate_deletion_request_retention_duration(
self.deletion_request_retention_unused_unserved_gating_additional_secs,
"NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS",
)?;
Self::validate_deletion_request_retention_duration(
self.deletion_request_retention_used_served_after_last_used_secs,
"NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS",
)?;
Self::validate_deletion_request_retention_duration(
self.deletion_request_retention_used_unserved_gating_additional_secs,
"NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS",
)?;
Self::validate_deletion_request_retention_deadline(
self.deletion_request_retention_unused_served_secs,
self.deletion_request_retention_unused_unserved_gating_additional_secs,
"unused",
)?;
Self::validate_deletion_request_retention_deadline(
self.deletion_request_retention_used_served_after_last_used_secs,
self.deletion_request_retention_used_unserved_gating_additional_secs,
"used",
)?;
// Fatal error: repository_whitelist with archive_read_only=true (incompatible)
if !repository_whitelist.is_empty() {
let read_only = self.archive_read_only.unwrap_or(archive_enabled);
@@ -795,6 +863,46 @@ impl Config {
Duration::from_secs(self.holding_cleanup_interval_secs)
}
/// How long an unused request remains served from relay-observed first_seen_at.
pub fn deletion_request_retention_unused_served(&self) -> Duration {
Duration::from_secs(self.deletion_request_retention_unused_served_secs)
}
/// Additional time an unused request remains unserved but eligible to gate.
pub fn deletion_request_retention_unused_unserved_gating_additional(&self) -> Duration {
Duration::from_secs(self.deletion_request_retention_unused_unserved_gating_additional_secs)
}
/// Normal-mode time a used request remains served after last_used_at.
pub fn deletion_request_retention_used_served_after_last_used(&self) -> Duration {
Duration::from_secs(self.deletion_request_retention_used_served_after_last_used_secs)
}
/// Additional normal-mode time a used request remains unserved but continues gating.
pub fn deletion_request_retention_used_unserved_gating_additional(&self) -> Duration {
Duration::from_secs(self.deletion_request_retention_used_unserved_gating_additional_secs)
}
fn validate_deletion_request_retention_duration(value: u64, name: &str) -> Result<()> {
if value == 0 {
return Err(anyhow!("{name} must be greater than 0"));
}
Ok(())
}
fn validate_deletion_request_retention_deadline(
served_secs: u64,
additional_gating_secs: u64,
lifecycle: &str,
) -> Result<()> {
if served_secs.checked_add(additional_gating_secs).is_none() {
return Err(anyhow!(
"{lifecycle} deletion-request retention served and additional gating durations must not overflow when combined"
));
}
Ok(())
}
/// Create config for testing
#[cfg(test)]
pub fn for_testing() -> Self {
@@ -828,6 +936,14 @@ impl Config {
holding_retention_secs: crate::nostr::lifecycle::DEFAULT_RETENTION.as_secs(),
holding_cleanup_interval_secs: crate::nostr::lifecycle::DEFAULT_CLEANUP_INTERVAL
.as_secs(),
deletion_request_retention_unused_served_secs:
DEFAULT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS,
deletion_request_retention_unused_unserved_gating_additional_secs:
DEFAULT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS,
deletion_request_retention_used_served_after_last_used_secs:
DEFAULT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS,
deletion_request_retention_used_unserved_gating_additional_secs:
DEFAULT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS,
archive_all: false,
archive_whitelist: String::new(),
archive_grasp_services: String::new(),
@@ -847,6 +963,9 @@ impl Config {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
static CONFIG_ENV_LOCK: Mutex<()> = Mutex::new(());
#[test]
fn test_default_values() {
@@ -857,6 +976,135 @@ mod tests {
assert_eq!(config.database_backend, DatabaseBackend::Memory);
}
#[test]
fn test_deletion_request_retention_cli_defaults() {
let _environment_guard = CONFIG_ENV_LOCK.lock().expect("lock must not be poisoned");
let config = Config::try_parse_from(["ngit-grasp", "--domain", "example.com"])
.expect("default deletion-request retention configuration should parse");
assert_eq!(
config.deletion_request_retention_unused_served(),
Duration::from_secs(2_592_000)
);
assert_eq!(
config.deletion_request_retention_unused_unserved_gating_additional(),
Duration::from_secs(15_552_000)
);
assert_eq!(
config.deletion_request_retention_used_served_after_last_used(),
Duration::from_secs(23_328_000)
);
assert_eq!(
config.deletion_request_retention_used_unserved_gating_additional(),
Duration::from_secs(7_776_000)
);
}
#[test]
fn test_deletion_request_retention_cli_overrides() {
let config = Config::try_parse_from([
"ngit-grasp",
"--domain",
"example.com",
"--deletion-request-retention-unused-served-secs",
"1",
"--deletion-request-retention-unused-unserved-gating-additional-secs",
"2",
"--deletion-request-retention-used-served-after-last-used-secs",
"3",
"--deletion-request-retention-used-unserved-gating-additional-secs",
"4",
])
.expect("custom deletion-request retention configuration should parse");
assert_eq!(config.deletion_request_retention_unused_served_secs, 1);
assert_eq!(
config.deletion_request_retention_unused_unserved_gating_additional_secs,
2
);
assert_eq!(
config.deletion_request_retention_used_served_after_last_used_secs,
3
);
assert_eq!(
config.deletion_request_retention_used_unserved_gating_additional_secs,
4
);
}
#[test]
fn test_deletion_request_retention_environment_override() {
let _environment_guard = CONFIG_ENV_LOCK.lock().expect("lock must not be poisoned");
const VARIABLE: &str = "NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS";
let original = std::env::var_os(VARIABLE);
std::env::set_var(VARIABLE, "42");
let config = Config::try_parse_from(["ngit-grasp", "--domain", "example.com"])
.expect("environment deletion-request retention configuration should parse");
match original {
Some(value) => std::env::set_var(VARIABLE, value),
None => std::env::remove_var(VARIABLE),
}
assert_eq!(config.deletion_request_retention_unused_served_secs, 42);
}
#[test]
fn test_deletion_request_retention_rejects_zero_durations() {
let cases = [
(
"NGIT_DELETION_REQUEST_RETENTION_UNUSED_SERVED_SECS",
Config {
deletion_request_retention_unused_served_secs: 0,
..Config::for_testing()
},
),
(
"NGIT_DELETION_REQUEST_RETENTION_UNUSED_UNSERVED_GATING_ADDITIONAL_SECS",
Config {
deletion_request_retention_unused_unserved_gating_additional_secs: 0,
..Config::for_testing()
},
),
(
"NGIT_DELETION_REQUEST_RETENTION_USED_SERVED_AFTER_LAST_USED_SECS",
Config {
deletion_request_retention_used_served_after_last_used_secs: 0,
..Config::for_testing()
},
),
(
"NGIT_DELETION_REQUEST_RETENTION_USED_UNSERVED_GATING_ADDITIONAL_SECS",
Config {
deletion_request_retention_used_unserved_gating_additional_secs: 0,
..Config::for_testing()
},
),
];
for (variable, config) in cases {
let error = config.validate().expect_err("zero duration must fail");
assert!(error.to_string().contains(variable));
}
}
#[test]
fn test_deletion_request_retention_rejects_deadline_overflow() {
let config = Config {
deletion_request_retention_unused_served_secs: u64::MAX,
deletion_request_retention_unused_unserved_gating_additional_secs: 1,
..Config::for_testing()
};
let error = config
.validate()
.expect_err("combined duration overflow must fail");
assert!(error
.to_string()
.contains("unused deletion-request retention"));
}
#[test]
fn test_lmdb_is_default() {
// Verify the actual default via the enum's Default trait