docs: update based on current implementation

This commit is contained in:
DanConwayDev
2025-12-04 13:02:59 +00:00
parent 40831a9025
commit d9bc5ed7fd
6 changed files with 606 additions and 2013 deletions
+42 -75
View File
@@ -28,95 +28,69 @@ nix-shell
nix-shell --run "cargo build" nix-shell --run "cargo build"
``` ```
### Running Tests ### Testing ngit-grasp (Main Project)
**Integration tests require relay running:** **ngit-grasp integration tests use the [`TestRelay`](tests/common/relay.rs:14) fixture:**
The `TestRelay` fixture automatically starts an instance of ngit-grasp itself and manages its lifecycle:
```bash ```bash
# Start ngit-relay first (use any available port to avoid conflicts) # Run all ngit-grasp tests (from project root)
docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest cargo test
# From grasp-audit directory, set RELAY_URL to match your port # Run integration tests only
# Run all ignored tests (includes GRASP-01 and other relay-dependent tests) cargo test --test '*'
RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored --nocapture
# Or run a specific test # Run specific test file
RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture cargo test --test nip01_compliance
``` ```
Tests marked `#[ignore]` need relay - unit tests don't. **How TestRelay works:**
**Note:** Always use a random available port for the relay to avoid conflicts with existing services. - Spawns `ngit-grasp` binary on a random available port
- Creates temporary directories for git and relay data
- Provides `url()` and `domain()` methods for test clients
- Automatically cleans up on drop
### Standard Testing Process (Recommended) **Example test pattern:**
**Use test-ngit-relay.sh for automated relay management:** ```rust
use common::TestRelay;
This script handles all relay lifecycle management automatically: #[tokio::test]
- Starts ngit-relay in isolated Docker container async fn test_something() {
- Uses random port to avoid conflicts let relay = TestRelay::start().await;
- Creates isolated temporary directories // relay.url() returns "ws://127.0.0.1:{port}"
- Ensures cleanup on exit (success or failure) // ... run test against ngit-grasp ...
- Supports both audit and test modes relay.stop().await;
}
**Basic Usage:**
```bash
# Run cargo test suite (recommended for GRASP-01 development)
cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
# Run audit CLI tool (for quick validation)
cd grasp-audit && nix develop -c bash test-ngit-relay.sh
# Get help
cd grasp-audit && ./test-ngit-relay.sh --help
``` ```
**Benefits:** ### Summary: Which Test Command for What
- No manual relay startup required
- Automatic cleanup prevents leftover containers
- Random port selection avoids conflicts
- Consistent environment across all runs
- Proper test isolation
**Note:** Manual relay setup is still available but test-ngit-relay.sh is recommended for development workflows. | What you're testing | Command |
| --------------------------- | ---------------------------------------------------------------------- |
| ngit-grasp (this project) | `cargo test` from project root |
| ngit-relay (reference impl) | `cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test` |
| grasp-audit unit tests | `cd grasp-audit && nix develop -c cargo test --lib` |
### Running Single Test ### Running Single Test
```bash ```bash
# From grasp-audit/ # ngit-grasp test (from project root)
cargo test --test nip01_compliance test_websocket_connection -- --nocapture
# grasp-audit test (from grasp-audit/)
nix develop -c cargo test --lib specific_test_name -- --nocapture nix develop -c cargo test --lib specific_test_name -- --nocapture
``` ```
### Quick Test Verification
To verify GRASP-01 compliance tests are working correctly:
```bash
# Run all ignored library tests (includes GRASP-01)
cd grasp-audit && RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored --nocapture 2>&1 | tail -60
# Or run specific GRASP-01 test
cd grasp-audit && RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture 2>&1 | tail -60
```
**Expected Output:**
- 2-3 tests passing
- 15+ tests showing "Not implemented yet"
### Troubleshooting ### Troubleshooting
**Buffer Size Errors:** **Buffer Size Errors:**
If you see mpsc channel buffer size panics on first test run, this is usually transient. Simply run the tests again. If you see mpsc channel buffer size panics on first test run, this is usually transient. Simply run the tests again.
**Verify Relay is Running:**
Check if relay is accessible before running tests:
```bash
nak req -l 1 ws://localhost:18081 # Replace port with your chosen port
```
**Port Conflicts:** **Port Conflicts:**
Always use a random available port to avoid conflicts with existing services. If a port is busy, choose a different one for docker. Both `TestRelay` and `test-ngit-relay.sh` use random ports to avoid conflicts. If you see port errors, ensure no stale processes are running.
## Code Patterns ## Code Patterns
@@ -156,8 +130,6 @@ EventBuilder::new(kind, content, &[tags])
EventBuilder::new(kind, content).tags(tags) EventBuilder::new(kind, content).tags(tags)
``` ```
See `docs/archive/2025-11-04-nostr-sdk-upgrade.md` for full migration.
### Audit Event Tagging (grasp-audit) ### Audit Event Tagging (grasp-audit)
**All audit events automatically include cleanup tags:** **All audit events automatically include cleanup tags:**
@@ -218,10 +190,10 @@ fn test_audit_tags_automatically_added() {
1. **Workspace compilation:** Can't `cargo build` from root for grasp-audit 1. **Workspace compilation:** Can't `cargo build` from root for grasp-audit
2. **Nix environment:** Must use `nix develop`, not `nix-shell` 2. **Nix environment:** Must use `nix develop`, not `nix-shell`
3. **nostr-sdk API:** Fields not methods in 0.43 3. **nostr-sdk API:** Fields not methods in 0.43
4. **Test isolation:** Integration tests need relay, marked with `#[ignore]` 4. **Test isolation:** Integration tests use `TestRelay` (ngit-grasp) or `test-ngit-relay.sh` (ngit-relay)
5. **Work directory:** All session docs go in `work/`, NOT root 5. **Work directory:** All session docs go in `work/`, NOT root
6. **Archive naming:** Use `YYYY-MM-DD-description.md` format 6. **Archive naming:** Use `YYYY-MM-DD-description.md` format
7. **Use test-ngit-relay.sh**: Always use the test script for GRASP-01 tests - it handles cleanup and port management automatically 7. **test-ngit-relay.sh tests ngit-relay**: This script tests the reference implementation, NOT ngit-grasp
## File Restrictions by Mode ## File Restrictions by Mode
@@ -234,19 +206,14 @@ Code mode can only edit files matching specific patterns (enforced by system):
## Quick Reference ## Quick Reference
```bash ```bash
# Recommended: Use test-ngit-relay.sh for all testing # Test ngit-grasp (main project)
cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test cargo test
# Build grasp-audit # Build grasp-audit
cd grasp-audit && nix develop -c cargo build cd grasp-audit && nix develop -c cargo build
# Manual relay testing (if needed) # Run grasp-audit unit tests
# 1. Start relay: docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest cd grasp-audit && nix develop -c cargo test --lib
# 2. Run all ignored tests: RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored --nocapture
# 3. Or specific test: RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture
# Run single test
cd grasp-audit && nix develop -c cargo test --lib test_name -- --nocapture
# Check session files # Check session files
ls work/ # Should only have README.md when clean ls work/ # Should only have README.md when clean
-2
View File
@@ -14,8 +14,6 @@ Unlike the reference implementation ([ngit-relay](https://gitworkshop.dev/npub15
## Status ## Status
**ALPHA** - Under active development. API and architecture subject to change.
## Key Features ## Key Features
- **Pure Rust Implementation**: Single binary, no external dependencies beyond Git itself - **Pure Rust Implementation**: Single binary, no external dependencies beyond Git itself
+186 -552
View File
@@ -2,16 +2,16 @@
## Executive Summary ## Executive Summary
`ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that the `git-http-backend` Rust crate provides sufficient flexibility to intercept and validate Git push operations before they reach the Git repository, eliminating the need for pre-receive hooks. `ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that we can intercept and validate Git push operations at the HTTP handler level before reaching the Git repository, eliminating the need for pre-receive hooks.
## Architectural Decision: Inline vs. Hook-Based Authorization ## Architectural Decision: Inline vs. Hook-Based Authorization
### Investigation Summary ### Investigation Summary
After examining both the reference implementation and the `git-http-backend` Rust crate, we have two options: After examining both the reference implementation and HTTP server options, we have two options:
#### Option 1: Hook-Based (Reference Implementation Approach) #### Option 1: Hook-Based (Reference Implementation Approach)
- Use `git-http-backend` crate as-is - Use standard Git HTTP backend
- Create pre-receive and post-receive hooks - Create pre-receive and post-receive hooks
- Hooks query the Nostr relay and validate pushes - Hooks query the Nostr relay and validate pushes
- **Pros**: Follows reference implementation closely - **Pros**: Follows reference implementation closely
@@ -28,7 +28,7 @@ After examining both the reference implementation and the `git-http-backend` Rus
**Rationale:** **Rationale:**
1. **The `git-http-backend` crate is sufficiently flexible**: Examining `src/actix/git_receive_pack.rs` shows it spawns `git receive-pack` as a subprocess and streams data. We can intercept this. 1. **Full control over HTTP layer**: Using Hyper directly gives us complete control over request handling, WebSocket upgrades, and CORS headers.
2. **Better Developer Experience**: 2. **Better Developer Experience**:
- Validation errors can be returned as proper HTTP responses - Validation errors can be returned as proper HTTP responses
@@ -56,7 +56,7 @@ After examining both the reference implementation and the `git-http-backend` Rus
│ │ │ │
│ ┌──────────────────┐ ┌──────────────────┐ │ │ ┌──────────────────┐ ┌──────────────────┐ │
│ │ HTTP Router │ │ Nostr Relay │ │ │ │ HTTP Router │ │ Nostr Relay │ │
│ │ (actix-web) │ │ (nostr-relay- │ │ │ │ (Hyper) │ │ (nostr-relay- │ │
│ │ │ │ builder) │ │ │ │ │ │ builder) │ │
│ └────────┬─────────┘ └────────┬─────────┘ │ │ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │ │ │ │ │
@@ -90,299 +90,149 @@ After examining both the reference implementation and the `git-http-backend` Rus
## Component Design ## Component Design
### 1. Main Server (`src/main.rs`) ### 1. Main Server ([`src/main.rs`](src/main.rs))
**Responsibilities:** **Responsibilities:**
- Initialize configuration from environment - Initialize configuration from environment (clap + dotenvy)
- Set up actix-web HTTP server - Set up Hyper HTTP server with request routing
- Initialize Nostr relay builder - Initialize Nostr relay builder with custom [`Nip34WritePolicy`](src/nostr/builder.rs:51)
- Set up shared storage - Set up shared storage (LMDB, NostrDB, or Memory)
- Configure routes for both Git and Nostr endpoints - Handle WebSocket upgrades for Nostr relay
- Handle graceful shutdown - Handle graceful shutdown
**Key Dependencies:** **Key Dependencies:**
```rust ```rust
actix-web = "4" hyper = "1"
tokio = { version = "1", features = ["full"] } tokio = { version = "1", features = ["full"] }
nostr-relay-builder = "0.43" nostr-relay-builder = "0.43"
nostr-sdk = "0.43" nostr-sdk = "0.43"
nostr-lmdb = "0.43"
``` ```
### 2. Git Module (`src/git/`) ### 2. HTTP Module ([`src/http/mod.rs`](src/http/mod.rs))
#### `handler.rs` - Git HTTP Handlers **Responsibilities:**
- Route HTTP requests to appropriate handlers
- WebSocket upgrade for Nostr relay at `/`
- Git Smart HTTP endpoints at `/<npub>/<identifier>.git/*`
- Landing pages and NIP-11 document serving
- CORS headers on all responses (GRASP-01 requirement)
Implements actix-web handlers for Git Smart HTTP protocol: **Key Implementation Details:**
```rust ```rust
// GET /<npub>/<identifier>.git/info/refs?service=git-upload-pack // CORS headers required by GRASP-01 specification
async fn info_refs_upload_pack( const CORS_ALLOW_ORIGIN: &str = "*";
req: HttpRequest, const CORS_ALLOW_METHODS: &str = "GET, POST";
state: web::Data<AppState>, const CORS_ALLOW_HEADERS: &str = "Content-Type";
) -> Result<HttpResponse>
// POST /<npub>/<identifier>.git/git-upload-pack /// Add CORS headers to a response builder
async fn git_upload_pack( fn add_cors_headers(builder: http::response::Builder) -> http::response::Builder {
req: HttpRequest, builder
body: web::Payload, .header("Access-Control-Allow-Origin", CORS_ALLOW_ORIGIN)
state: web::Data<AppState>, .header("Access-Control-Allow-Methods", CORS_ALLOW_METHODS)
) -> Result<HttpResponse> .header("Access-Control-Allow-Headers", CORS_ALLOW_HEADERS)
}
// GET /<npub>/<identifier>.git/info/refs?service=git-receive-pack
async fn info_refs_receive_pack(
req: HttpRequest,
state: web::Data<AppState>,
) -> Result<HttpResponse>
// POST /<npub>/<identifier>.git/git-receive-pack
// THIS IS WHERE THE MAGIC HAPPENS
async fn git_receive_pack(
req: HttpRequest,
body: web::Payload,
state: web::Data<AppState>,
) -> Result<HttpResponse>
``` ```
#### `authorization.rs` - Push Validation See [`src/http/mod.rs:29-84`](src/http/mod.rs:29-84) for the full CORS implementation.
### 3. Git Module ([`src/git/`](src/git/))
#### [`handlers.rs`](src/git/handlers.rs) - Git HTTP Handlers
Implements handlers for Git Smart HTTP protocol:
```rust
/// Handle GET /info/refs?service=git-{upload,receive}-pack
pub async fn handle_info_refs(
repo_path: PathBuf,
service: GitService,
) -> Result<Response<Full<Bytes>>, GitError>
/// Handle POST /git-upload-pack (clone/fetch)
pub async fn handle_upload_pack(
repo_path: PathBuf,
body: Bytes,
) -> Result<Response<Full<Bytes>>, GitError>
/// Handle POST /git-receive-pack (push)
/// THIS IS WHERE THE MAGIC HAPPENS - validates against state before accepting
pub async fn handle_receive_pack(
repo_path: PathBuf,
body: Bytes,
database: SharedDatabase,
npub: &str,
identifier: &str,
) -> Result<Response<Full<Bytes>>, GitError>
```
See [`src/git/handlers.rs:22-98`](src/git/handlers.rs:22-98) for the info-refs implementation.
#### [`authorization.rs`](src/git/authorization.rs) - Push Validation
**Core Logic:** **Core Logic:**
```rust ```rust
pub struct PushValidator { /// Get authorization info for a repository owner
nostr_client: Arc<Client>, pub async fn get_authorization_for_owner(
relay_url: String, database: &SharedDatabase,
} pubkey: &PublicKey,
impl PushValidator {
/// Validate a push operation against Nostr state
pub async fn validate_push(
&self,
npub: &str,
identifier: &str,
ref_updates: Vec<RefUpdate>,
) -> Result<ValidationResult> {
// 1. Fetch announcement and state events from local relay
let events = self.fetch_events(identifier).await?;
// 2. Extract pubkey from npub
let pubkey = decode_npub(npub)?;
// 3. Get recursive maintainer set
let maintainers = get_maintainers(&events, &pubkey, identifier);
// 4. Get latest state from maintainers
let state = get_state_from_maintainers(&events, &maintainers)?;
// 5. Validate each ref update
for ref_update in ref_updates {
if ref_update.ref_name.starts_with("refs/nostr/") {
// Allow refs/nostr/<event-id> for PRs
validate_pr_ref(&ref_update)?;
} else if ref_update.ref_name.starts_with("refs/heads/pr/") {
// Reject pr/* branches - should use refs/nostr/
return Err(Error::InvalidRef("pr/* branches must use refs/nostr/"));
} else {
// Validate against state event
validate_state_ref(&state, &ref_update)?;
}
}
Ok(ValidationResult::Accept)
}
}
```
**Key Functions:**
```rust
/// Parse ref updates from git-receive-pack request body
fn parse_ref_updates(body: &[u8]) -> Result<Vec<RefUpdate>>
/// Recursively find all maintainers
fn get_maintainers(
events: &[Event],
pubkey: &str,
identifier: &str, identifier: &str,
) -> Vec<String> ) -> Result<AuthorizationResult, AuthorizationError>
/// Get latest state from maintainer set /// Validate that pushed refs match the authorized state
fn get_state_from_maintainers( pub fn validate_push_refs(
events: &[Event], pushed_refs: &[PushedRef],
maintainers: &[String],
) -> Result<RepositoryState>
/// Validate a ref matches the state event
fn validate_state_ref(
state: &RepositoryState, state: &RepositoryState,
ref_update: &RefUpdate, ) -> Result<(), AuthorizationError>
) -> Result<()>
/// Validate refs/nostr/<event-id> pushes
pub fn validate_nostr_ref_pushes(
pushed_refs: &[PushedRef],
database: &SharedDatabase,
) -> Result<(), AuthorizationError>
``` ```
### 3. Nostr Module (`src/nostr/`) ### 4. Nostr Module ([`src/nostr/`](src/nostr/))
#### `relay.rs` - Relay Configuration #### [`builder.rs`](src/nostr/builder.rs) - Relay Configuration
The [`Nip34WritePolicy`](src/nostr/builder.rs:51) is the core event validation logic:
```rust ```rust
pub async fn build_relay(config: &Config) -> Result<LocalRelay> { /// NIP-34 Write Policy with Full GRASP-01 Event Validation
let builder = RelayBuilder::default() ///
.write_policy(RepositoryAnnouncementPolicy::new(config.domain.clone())) /// Validates all events according to GRASP-01 specification:
.write_policy(RelatedEventsPolicy::new()) /// - Repository announcements must list service in clone and relays tags
.query_policy(StandardQueryPolicy::new()) /// EXCEPTION: Recursive maintainer announcements are accepted even without
.on_event_saved(create_repository_hook(config.git_data_path.clone())); /// listing the service, to enable maintainer chain discovery and GRASP-02 sync
/// - Repository state announcements must have valid structure
// Configure storage backend (LMDB or NDB) /// - Other events must reference accepted repositories or events
let relay = LocalRelay::run(builder).await?; /// - Forward references are supported (events referenced by accepted events)
/// - Orphan events with no valid references are rejected
Ok(relay) pub struct Nip34WritePolicy {
}
```
#### `events.rs` - Event Handlers
```rust
/// Hook called when events are saved
pub fn create_repository_hook(
git_data_path: PathBuf,
) -> impl Fn(&Event) -> BoxFuture<'static, ()> {
move |event: &Event| {
let git_path = git_data_path.clone();
Box::pin(async move {
if event.kind == Kind::RepositoryAnnouncement {
handle_repository_announcement(event, &git_path).await;
} else if event.kind == Kind::RepositoryState {
handle_repository_state(event, &git_path).await;
}
})
}
}
async fn handle_repository_announcement(event: &Event, git_path: &Path) {
// 1. Parse repository from event
// 2. Check if listed in clone and relays tags
// 3. Create empty bare Git repository
// 4. Configure uploadpack.allowTipSHA1InWant
// 5. Configure uploadpack.allowUnreachable
// 6. Configure http.receivepack
}
async fn handle_repository_state(event: &Event, git_path: &Path) {
// 1. Parse state from event
// 2. Update repository HEAD if needed
// 3. Trigger proactive sync (GRASP-02)
}
```
**Write Policies:**
```rust
/// Accept repository announcements that list this instance
pub struct RepositoryAnnouncementPolicy {
domain: String, domain: String,
} database: SharedDatabase,
impl WritePolicy for RepositoryAnnouncementPolicy {
fn admit_event(&self, event: &Event, _addr: &SocketAddr)
-> BoxFuture<PolicyResult>
{
Box::pin(async move {
if event.kind != Kind::RepositoryAnnouncement {
return PolicyResult::Accept; // Not our concern
}
// Check if this instance is in clone and relays tags
let has_clone = event.tags.iter()
.any(|t| t.kind() == "clone" && t.content() == Some(&self.domain));
let has_relay = event.tags.iter()
.any(|t| t.kind() == "relays" && t.content() == Some(&self.domain));
if has_clone && has_relay {
PolicyResult::Accept
} else {
PolicyResult::Reject("instance not listed in clone and relays".into())
}
})
}
}
/// Accept events related to stored announcements/issues/patches
pub struct RelatedEventsPolicy;
impl WritePolicy for RelatedEventsPolicy {
fn admit_event(&self, event: &Event, _addr: &SocketAddr)
-> BoxFuture<PolicyResult>
{
// Accept if event tags or is tagged by stored events
// Implementation requires querying the event store
}
}
```
### 4. Storage Module (`src/storage/`)
#### `repository.rs` - Repository Management
```rust
pub struct RepositoryManager {
git_data_path: PathBuf, git_data_path: PathBuf,
} }
impl RepositoryManager {
/// Create a new bare Git repository
pub async fn create_repository(
&self,
npub: &str,
identifier: &str,
) -> Result<PathBuf> {
let repo_path = self.git_data_path
.join(npub)
.join(format!("{}.git", identifier));
// Create directory
tokio::fs::create_dir_all(&repo_path).await?;
// Initialize bare repo
Command::new("git")
.args(&["init", "--bare"])
.arg(&repo_path)
.output()
.await?;
// Configure
self.configure_repository(&repo_path).await?;
Ok(repo_path)
}
async fn configure_repository(&self, repo_path: &Path) -> Result<()> {
// Enable unauthenticated push (we handle auth ourselves)
git_config(repo_path, "http.receivepack", "true").await?;
// Enable tip SHA1 fetching (required for ngit)
git_config(repo_path, "uploadpack.allowTipSHA1InWant", "true").await?;
// Enable unreachable object fetching
git_config(repo_path, "uploadpack.allowUnreachable", "true").await?;
Ok(())
}
/// Check if repository exists
pub async fn repository_exists(
&self,
npub: &str,
identifier: &str,
) -> bool {
let repo_path = self.git_data_path
.join(npub)
.join(format!("{}.git", identifier));
repo_path.join("HEAD").exists() &&
repo_path.join("config").exists()
}
}
``` ```
### 5. Configuration (`src/config.rs`) See [`src/nostr/builder.rs:38-78`](src/nostr/builder.rs:38-78) for the full policy struct.
#### [`events.rs`](src/nostr/events.rs) - Event Parsing
Provides structures for parsing NIP-34 events:
```rust
/// Parsed repository announcement (Kind 30617)
pub struct RepositoryAnnouncement { ... }
/// Parsed repository state (Kind 30618)
pub struct RepositoryState { ... }
```
### 5. Configuration ([`src/config.rs`](src/config.rs))
```rust ```rust
pub struct Config { pub struct Config {
@@ -393,34 +243,18 @@ pub struct Config {
pub git_data_path: PathBuf, pub git_data_path: PathBuf,
pub relay_data_path: PathBuf, pub relay_data_path: PathBuf,
pub bind_address: SocketAddr, pub bind_address: SocketAddr,
pub log_level: String, pub database_backend: DatabaseBackend,
} }
impl Config { pub enum DatabaseBackend {
pub fn from_env() -> Result<Self> { Lmdb, // Default, production use
Ok(Config { NostrDb, // Alternative
domain: env::var("NGIT_DOMAIN")?, Memory, // Testing
owner_npub: env::var("NGIT_OWNER_NPUB")?,
relay_name: env::var("NGIT_RELAY_NAME")?,
relay_description: env::var("NGIT_RELAY_DESCRIPTION")?,
git_data_path: PathBuf::from(
env::var("NGIT_GIT_DATA_PATH")
.unwrap_or_else(|_| "./data/git".to_string())
),
relay_data_path: PathBuf::from(
env::var("NGIT_RELAY_DATA_PATH")
.unwrap_or_else(|_| "./data/relay".to_string())
),
bind_address: env::var("NGIT_BIND_ADDRESS")
.unwrap_or_else(|_| "127.0.0.1:8080".to_string())
.parse()?,
log_level: env::var("RUST_LOG")
.unwrap_or_else(|_| "info".to_string()),
})
}
} }
``` ```
Configuration is loaded via **clap CLI > environment variables > .env > defaults**.
## Data Flow ## Data Flow
### Push Operation Flow ### Push Operation Flow
@@ -428,327 +262,120 @@ impl Config {
``` ```
1. Git Client → POST /<npub>/<id>.git/git-receive-pack 1. Git Client → POST /<npub>/<id>.git/git-receive-pack
↓ ↓
2. git_receive_pack handler receives request 2. HttpService routes to git::handlers::handle_receive_pack()
↓ ↓
3. Parse ref updates from request body 3. Parse ref updates from request body (pkt-line format)
↓ ↓
4. Extract npub and identifier from URL 4. Extract npub and identifier from URL
↓ ↓
5. PushValidator::validate_push() 5. authorization::get_authorization_for_owner()
├─ Fetch events from local Nostr relay ├─ Query database for announcements
├─ Get maintainers recursively ├─ Build recursive maintainer set
├─ Get latest state from maintainers └─ Get latest authorized state
└─ Validate each ref update
↓ ↓
6. If VALID: 6. authorization::validate_push_refs()
├─ Check each ref matches state
└─ Validate refs/nostr/ pushes
↓
7. If VALID:
├─ Spawn git-receive-pack subprocess ├─ Spawn git-receive-pack subprocess
├─ Stream request body to git stdin ├─ Stream request body to git stdin
└─ Stream git stdout back to client └─ Stream git stdout back to client
↓ ↓
7. If INVALID: 8. If INVALID:
└─ Return HTTP 403 with error message └─ Return HTTP 403 with error message
``` ```
### Repository Announcement Flow ### Repository Announcement Flow
``` ```
1. Nostr Client → EVENT (Kind 30317) 1. Nostr Client → EVENT (Kind 30617)
↓ ↓
2. Nostr relay receives event 2. Nostr relay receives event
↓ ↓
3. RepositoryAnnouncementPolicy::admit_event() 3. Nip34WritePolicy::admit_event()
├─ Check if instance in clone tags ├─ Check if instance in clone tags
├─ Check if instance in relays tags ├─ Check if instance in relays tags
├─ OR: Check if recursive maintainer
└─ Accept or reject └─ Accept or reject
↓ ↓
4. If ACCEPTED: 4. If ACCEPTED:
├─ Event saved to store ├─ Event saved to database
└─ on_event_saved hook triggered └─ ensure_bare_repository() called
↓ ↓
5. handle_repository_announcement() 5. Bare Git repository created at
├─ Parse repository details <git_data_path>/<npub>/<identifier>.git
├─ Create Git repository directory
├─ Initialize bare Git repo
└─ Configure Git settings
``` ```
## Key Implementation Details ### State Event Flow
### 1. Parsing Git Receive-Pack Protocol
The Git receive-pack protocol uses a pkt-line format. We need to parse:
``` ```
0000-0000-0000-0000 0000-0000-0000-0000 refs/heads/main\0 report-status 1. Nostr Client → EVENT (Kind 30618)
0000-0000-0000-0000 0000-0000-0000-0000 refs/heads/dev ↓
``` 2. Nostr relay receives event
↓
Each line has: 3. Nip34WritePolicy::admit_event()
- Old SHA (40 hex chars) ├─ Check author is in maintainer set
- Space ├─ Validate state structure
- New SHA (40 hex chars) └─ Accept or reject
- Space ↓
- Ref name 4. If ACCEPTED and is latest state:
- Optional capabilities (first line only, after \0) ├─ Align repository refs to match state
├─ Create/update/delete refs as needed
```rust └─ Set HEAD if commit available
pub struct RefUpdate {
pub old_sha: String,
pub new_sha: String,
pub ref_name: String,
}
pub fn parse_ref_updates(body: &[u8]) -> Result<Vec<RefUpdate>> {
// Parse pkt-line format
// Extract ref updates
// Return structured data
}
```
### 2. Maintainer Recursion
The maintainer resolution must handle cycles and correctly build the set:
```rust
fn get_maintainers_recursive(
events: &[Event],
pubkey: &str,
identifier: &str,
visited: &mut HashSet<String>,
) -> HashSet<String> {
if visited.contains(pubkey) {
return HashSet::new();
}
visited.insert(pubkey.to_string());
let announcement = find_announcement(events, pubkey, identifier);
if announcement.is_none() {
return HashSet::new();
}
let repo = parse_repository(announcement.unwrap());
for maintainer in repo.maintainers {
get_maintainers_recursive(events, &maintainer, identifier, visited);
}
visited.clone()
}
```
### 3. State Event Validation
```rust
fn validate_state_ref(
state: &RepositoryState,
ref_update: &RefUpdate,
) -> Result<()> {
if ref_update.ref_name.starts_with("refs/heads/") {
let branch_name = &ref_update.ref_name[11..];
if let Some(commit) = state.branches.get(branch_name) {
if commit == &ref_update.new_sha {
return Ok(());
}
return Err(Error::StateMismatch {
ref_name: ref_update.ref_name.clone(),
expected: commit.clone(),
got: ref_update.new_sha.clone(),
});
}
return Err(Error::RefNotInState(ref_update.ref_name.clone()));
}
if ref_update.ref_name.starts_with("refs/tags/") {
let tag_name = &ref_update.ref_name[10..];
if let Some(commit) = state.tags.get(tag_name) {
if commit == &ref_update.new_sha {
return Ok(());
}
return Err(Error::StateMismatch {
ref_name: ref_update.ref_name.clone(),
expected: commit.clone(),
got: ref_update.new_sha.clone(),
});
}
return Err(Error::RefNotInState(ref_update.ref_name.clone()));
}
Err(Error::InvalidRef(ref_update.ref_name.clone()))
}
```
### 4. CORS Support
As per GRASP-01, we must support CORS:
```rust
use actix_cors::Cors;
fn configure_cors() -> Cors {
Cors::default()
.allow_any_origin()
.allowed_methods(vec!["GET", "POST", "OPTIONS"])
.allowed_headers(vec!["Content-Type"])
.max_age(3600)
}
// In main.rs
App::new()
.wrap(configure_cors())
.configure(git_routes)
.configure(nostr_routes)
``` ```
## Testing Strategy ## Testing Strategy
See [TEST_STRATEGY.md](TEST_STRATEGY.md) for comprehensive testing documentation, including: See [test-strategy.md](../reference/test-strategy.md) for comprehensive testing documentation.
- **GRASP Compliance Testing Tool**: Reusable test suite that validates any GRASP implementation against the spec
- **Spec-Mirrored Tests**: Test structure matches GRASP protocol documents exactly
- **Clear Failure Messages**: Test failures cite exact spec lines (e.g., "GRASP-01:12-13")
- **Multiple Test Levels**: Unit, integration, compliance, and end-to-end tests
### Quick Overview ### Quick Overview
**Integration Tests** ([`tests/`](tests/)):
- Use [`TestRelay`](tests/common/relay.rs:14) fixture for automatic relay lifecycle
- Each test file in [`tests/`](tests/) covers a GRASP-01 requirement
**Audit Tests** ([`grasp-audit/`](grasp-audit/)):
- Reusable compliance testing for any GRASP implementation
- Spec-mirrored structure in [`grasp-audit/src/specs/grasp01/`](grasp-audit/src/specs/grasp01/)
```rust ```rust
// Unit Tests - Individual functions // Example: tests/nip01_compliance.rs
#[test]
fn test_parse_ref_updates() {
let body = b"0000... 0000... refs/heads/main\0report-status\n";
let updates = parse_ref_updates(body).unwrap();
assert_eq!(updates.len(), 1);
assert_eq!(updates[0].ref_name, "refs/heads/main");
}
// Integration Tests - Component interaction
#[tokio::test] #[tokio::test]
async fn test_full_push_flow() { async fn test_nip01_websocket_connection() {
let app = test_app().await; let relay = TestRelay::start().await;
let (announcement, state) = app.create_repo_with_state() // Test NIP-01 compliance...
.branch("main", "commit-123") relay.stop().await;
.build()
.await;
let result = app.git_push("main", "commit-123").await;
assert!(result.success);
}
// Compliance Tests - GRASP spec validation
#[tokio::test]
async fn test_grasp_01_compliance() {
use grasp_compliance_tests::{TestContext, Grasp01Spec};
let ctx = TestContext::builder()
.base_url(&server.url())
.build();
let results = Grasp01Spec::test_compliance(&ctx).await;
assert!(results.all_passed(), "{}", results.report());
} }
``` ```
The compliance testing tool is designed as a **standalone crate** that can be:
- Used by ngit-grasp for self-validation
- Published for other GRASP implementations to use
- Updated as new GRASP specs are released
- Run in CI/CD for continuous compliance verification
## Performance Considerations ## Performance Considerations
### 1. Async All The Way ### 1. Async All The Way
- Use `tokio` for all I/O - Use `tokio` for all I/O
- Non-blocking Git subprocess spawning - Non-blocking Git subprocess spawning via [`GitSubprocess`](src/git/subprocess.rs)
- Stream large pack files without buffering - Stream large pack files without buffering
### 2. Connection Pooling ### 2. Shared Database
- Reuse Nostr relay connections - Single database instance shared between relay and Git handlers
- Connection pool for internal relay queries - Direct queries for push authorization (no WebSocket round-trip)
### 3. Caching ### 3. Write Policy Caching
- Cache parsed state events (with TTL) - Maintainer sets computed once per event validation
- Cache maintainer sets - State lookups use database indexes
- Invalidate on new state events
```rust
pub struct StateCache {
cache: Arc<RwLock<HashMap<String, CachedState>>>,
}
struct CachedState {
state: RepositoryState,
maintainers: Vec<String>,
timestamp: Instant,
}
impl StateCache {
pub async fn get_or_fetch(
&self,
identifier: &str,
fetcher: impl Future<Output = Result<(RepositoryState, Vec<String>)>>,
) -> Result<(RepositoryState, Vec<String>)> {
// Check cache
// Return if fresh
// Otherwise fetch and cache
}
}
```
## Future Extensions ## Future Extensions
### GRASP-02: Proactive Sync ### GRASP-02: Proactive Sync
Add background tasks: See [grasp-02-proactive-sync.md](grasp-02-proactive-sync.md) for detailed design.
```rust
pub struct ProactiveSyncTask {
relay_client: Client,
git_manager: RepositoryManager,
}
impl ProactiveSyncTask {
pub async fn run(&self) {
loop {
tokio::time::sleep(Duration::from_secs(3600)).await;
// Fetch all announcements from our relay
let announcements = self.fetch_announcements().await;
for ann in announcements {
// Sync events from listed relays
self.sync_events(&ann).await;
// Sync git data from listed clones
self.sync_git_data(&ann).await;
// Fetch PR data
self.sync_pr_data(&ann).await;
}
}
}
}
```
### GRASP-05: Archive ### GRASP-05: Archive
Relax the policy: Relax the write policy to accept all repository announcements regardless of clone/relays tags.
```rust
pub struct ArchiveAnnouncementPolicy;
impl WritePolicy for ArchiveAnnouncementPolicy {
fn admit_event(&self, event: &Event, _addr: &SocketAddr)
-> BoxFuture<PolicyResult>
{
// Accept all repository announcements
// Don't check clone/relays tags
PolicyResult::Accept
}
}
```
## Deployment ## Deployment
@@ -756,7 +383,7 @@ impl WritePolicy for ArchiveAnnouncementPolicy {
```bash ```bash
cargo build --release cargo build --release
./target/release/ngit-grasp ./target/release/ngit-grasp --domain example.com --owner-npub npub1...
``` ```
### Docker ### Docker
@@ -799,10 +426,17 @@ WantedBy=multi-user.target
2. **Path Traversal**: Prevent directory traversal in repository paths 2. **Path Traversal**: Prevent directory traversal in repository paths
3. **DoS Protection**: Rate limiting on both HTTP and WebSocket 3. **DoS Protection**: Rate limiting on both HTTP and WebSocket
4. **Resource Limits**: Limit pack file sizes, event sizes 4. **Resource Limits**: Limit pack file sizes, event sizes
5. **Nostr Event Validation**: Strict signature verification 5. **Nostr Event Validation**: Strict signature verification (handled by nostr-relay-builder)
## Conclusion ## Conclusion
The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. The Rust ecosystem provides excellent libraries for both Git and Nostr protocols, enabling a high-performance, type-safe implementation. The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. Using Hyper for the HTTP layer gives us complete control over request handling, WebSocket upgrades, and CORS headers.
The key insight is that we don't need to rely on Git's hook mechanism when we have full control over the HTTP layer that Git operates through. By intercepting at the HTTP handler level, we gain better error handling, easier testing, and tighter integration between the Git and Nostr components. The key insight is that we don't need to rely on Git's hook mechanism when we have full control over the HTTP layer that Git operates through. By intercepting at the HTTP handler level, we gain better error handling, easier testing, and tighter integration between the Git and Nostr components.
## Related Documentation
- [Inline Authorization Explanation](inline-authorization.md) - Why we chose this approach
- [GRASP-02 Proactive Sync](grasp-02-proactive-sync.md) - Future work design
- [Test Strategy](../reference/test-strategy.md) - Comprehensive testing documentation
- [GRASP-01 Implementation Learnings](../learnings/grasp-01-implementation.md) - Patterns and lessons learned
+55 -69
View File
@@ -150,14 +150,17 @@ rm -rf /tmp/test-repo
```rust ```rust
#[tokio::test] #[tokio::test]
async fn test_unauthorized_push() { async fn test_unauthorized_push() {
let state = create_test_state().await; let relay = TestRelay::start().await;
let result = validate_push(&state, "refs/heads/main", alice_pubkey).await; let result = validate_push(&state, "refs/heads/main", alice_pubkey).await;
assert!(result.is_err()); assert!(result.is_err());
relay.stop().await;
} }
``` ```
**Result:** Pure Rust unit tests, no shell scripts, no Git setup. **Result:** Pure Rust unit tests, no shell scripts, no Git setup.
See [`tests/push_authorization.rs`](tests/push_authorization.rs) for actual test examples.
### 4. Shared State and Types ### 4. Shared State and Types
**With hooks:** **With hooks:**
@@ -168,19 +171,17 @@ async fn test_unauthorized_push() {
**With inline authorization:** **With inline authorization:**
```rust ```rust
pub struct GitHandler { // From src/git/handlers.rs
nostr_relay: Arc<NostrRelay>, // Shared! pub async fn handle_receive_pack(
state_cache: Arc<StateCache>, // Shared! repo_path: PathBuf,
} body: Bytes,
database: SharedDatabase, // Shared with Nostr relay!
impl GitHandler { npub: &str,
async fn validate_push(&self, refs: &[RefUpdate]) -> Result<()> { identifier: &str,
// Direct access to Nostr state ) -> Result<Response<Full<Bytes>>, GitError> {
let state = self.state_cache.get_latest().await?; // Direct database access for authorization
// Validate using shared types let auth = get_authorization_for_owner(&database, pubkey, identifier).await?;
state.validate_refs(refs)?; // ...
Ok(())
}
} }
``` ```
@@ -208,9 +209,9 @@ Setup steps:
**With inline authorization (ngit-grasp):** **With inline authorization (ngit-grasp):**
``` ```
Single Rust binary: Single Rust binary:
- HTTP server (actix-web) - HTTP server (Hyper)
- Git protocol handler - Git protocol handler
- Nostr relay - Nostr relay (nostr-relay-builder)
- Authorization logic - Authorization logic
Setup steps: Setup steps:
@@ -235,44 +236,38 @@ Content-Type: application/x-git-receive-pack-request
0000000000000000000000000000000000000000 abc123... refs/heads/main\0 report-status 0000000000000000000000000000000000000000 abc123... refs/heads/main\0 report-status
``` ```
We parse this **before** spawning Git: We parse this **before** spawning Git. See [`src/git/authorization.rs`](src/git/authorization.rs) for the implementation:
```rust ```rust
pub async fn git_receive_pack( /// Parse ref updates from git-receive-pack request body
req: HttpRequest, pub fn parse_pushed_refs(body: &[u8]) -> Result<Vec<PushedRef>, AuthorizationError> {
body: web::Bytes, // Parse pkt-line format
) -> Result<HttpResponse, Error> { // Extract ref updates
// 1. Parse ref updates from request body // Return structured data
let ref_updates = parse_ref_updates(&body)?;
// 2. Validate against Nostr state
let state = get_latest_state(&repo).await?;
validate_push(&state, &ref_updates).await?;
// 3. If valid, spawn git-receive-pack
spawn_git_receive_pack(req, body).await
} }
``` ```
### How We Validate ### How We Validate
Validation checks: Validation checks (from [`src/git/authorization.rs`](src/git/authorization.rs)):
1. Does pusher's pubkey have write access? 1. Does pusher's pubkey have write access?
2. Are they listed as a maintainer in the latest state event? 2. Are they listed as a maintainer in the latest state event?
3. Do maintainer sets form a valid chain? 3. Do the refs match the state event?
```rust ```rust
async fn validate_push( /// Validate that pushed refs match the authorized state
state: &RepoState, pub fn validate_push_refs(
refs: &[RefUpdate], pushed_refs: &[PushedRef],
) -> Result<()> { state: &RepositoryState,
for ref_update in refs { ) -> Result<(), AuthorizationError> {
// Check if pusher is authorized for this ref for pushed_ref in pushed_refs {
if !state.is_authorized(&ref_update.name, pusher_pubkey) { if pushed_ref.ref_name.starts_with("refs/heads/") {
return Err(Error::Unauthorized { // Validate branch against state
ref_name: ref_update.name.clone(), } else if pushed_ref.ref_name.starts_with("refs/tags/") {
pubkey: pusher_pubkey, // Validate tag against state
}); } else if pushed_ref.ref_name.starts_with("refs/nostr/") {
// Allow refs/nostr/<event-id> for PRs
} }
} }
Ok(()) Ok(())
@@ -291,7 +286,7 @@ async fn validate_push(
| **Performance** | Spawns Git first | Validates first | | **Performance** | Spawns Git first | Validates first |
| **Testing** | Shell scripts + Go tests | Pure Rust tests | | **Testing** | Shell scripts + Go tests | Pure Rust tests |
| **Deployment** | Docker + supervisord | Single binary | | **Deployment** | Docker + supervisord | Single binary |
| **State sharing** | WebSocket query | Direct memory access | | **State sharing** | WebSocket query | Direct database access |
Both are GRASP-compliant, but inline authorization is simpler and more efficient. Both are GRASP-compliant, but inline authorization is simpler and more efficient.
@@ -314,34 +309,25 @@ Both are GRASP-compliant, but inline authorization is simpler and more efficient
### Is It Worth It? ### Is It Worth It?
**Yes**, because: **Yes**, because:
1. The `git-http-backend` crate handles protocol parsing 1. We handle protocol parsing in [`src/git/protocol.rs`](src/git/protocol.rs)
2. GRASP is already non-standard (Nostr authorization) 2. GRASP is already non-standard (Nostr authorization)
3. Benefits far outweigh the coupling cost 3. Benefits far outweigh the coupling cost
4. We can still add hook support later if needed 4. We can still add hook support later if needed
--- ---
## Alternative Considered: Hybrid Approach ## Implementation References
We could use **both** inline validation and hooks: Key files in the ngit-grasp implementation:
```rust | Component | Location |
// Inline: Fast path for common cases |-----------|----------|
if !quick_validate(pusher).await? { | HTTP routing | [`src/http/mod.rs`](src/http/mod.rs) |
return Err(Error::Unauthorized); | Git handlers | [`src/git/handlers.rs`](src/git/handlers.rs) |
} | Push authorization | [`src/git/authorization.rs`](src/git/authorization.rs) |
| Git protocol parsing | [`src/git/protocol.rs`](src/git/protocol.rs) |
// Hook: Detailed validation | Subprocess management | [`src/git/subprocess.rs`](src/git/subprocess.rs) |
spawn_git_with_hook().await?; | Event acceptance policy | [`src/nostr/builder.rs:51`](src/nostr/builder.rs:51) - `Nip34WritePolicy` |
```
**Why we didn't choose this:**
- Added complexity
- Redundant validation
- Slower (two validation steps)
- Harder to maintain
If inline validation is sufficient, why add hooks?
--- ---
@@ -365,9 +351,8 @@ This would allow:
### If Git Protocol Changes ### If Git Protocol Changes
The `git-http-backend` crate abstracts protocol details. If the Git protocol changes: The protocol parsing is isolated in [`src/git/protocol.rs`](src/git/protocol.rs). If the Git protocol changes:
- Update the crate dependency - Update the protocol module
- Adjust our ref parsing if needed
- Tests will catch any breakage - Tests will catch any breakage
--- ---
@@ -380,11 +365,11 @@ The `git-http-backend` crate abstracts protocol details. If the Git protocol cha
2. It's more performant (early rejection) 2. It's more performant (early rejection)
3. It's easier to test (pure Rust) 3. It's easier to test (pure Rust)
4. It's simpler to deploy (single binary) 4. It's simpler to deploy (single binary)
5. It enables better integration (shared state) 5. It enables better integration (shared database)
The trade-off (coupling to Git HTTP protocol) is acceptable because: The trade-off (coupling to Git HTTP protocol) is acceptable because:
- The protocol is stable and well-specified - The protocol is stable and well-specified
- The `git-http-backend` crate abstracts details - Protocol handling is isolated in one module
- Benefits far outweigh the cost - Benefits far outweigh the cost
This decision aligns with our goal of creating a **developer-friendly, production-ready GRASP implementation**. This decision aligns with our goal of creating a **developer-friendly, production-ready GRASP implementation**.
@@ -397,6 +382,7 @@ This decision aligns with our goal of creating a **developer-friendly, productio
- [Design Decisions](decisions.md) - All architectural choices - [Design Decisions](decisions.md) - All architectural choices
- [Comparison with ngit-relay](comparison.md) - Detailed comparison - [Comparison with ngit-relay](comparison.md) - Detailed comparison
- [Git Protocol Reference](../reference/git-protocol.md) - Protocol details - [Git Protocol Reference](../reference/git-protocol.md) - Protocol details
- [Test Strategy](../reference/test-strategy.md) - How we test this
--- ---
+71 -138
View File
@@ -1,13 +1,13 @@
# GRASP Audit Tool - Patterns and Learnings # GRASP Audit Tool - Patterns and Learnings
**Purpose:** Document grasp-audit architecture, patterns, and lessons learned **Purpose:** Document grasp-audit architecture, patterns, and lessons learned
**Last Updated:** November 4, 2025 **Last Updated:** December 4, 2025
--- ---
## Overview ## Overview
`grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality. `grasp-audit` is a **fully implemented** compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality.
--- ---
@@ -32,10 +32,10 @@
**Problem:** Test events pollute the relay and need cleanup without deletion events. **Problem:** Test events pollute the relay and need cleanup without deletion events.
**Solution:** Use special tags to mark audit events: **Solution:** Use special tags to mark audit events (implemented in [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs)):
```rust ```rust
// Every audit event includes these tags // Every audit event includes these tags (added automatically)
[ [
["t", "grasp-audit-test-event"], // Marker ["t", "grasp-audit-test-event"], // Marker
["t", "audit-{run-id}"], // Run isolation ["t", "audit-{run-id}"], // Run isolation
@@ -78,6 +78,8 @@
### Audit Configuration ### Audit Configuration
From [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs):
```rust ```rust
use grasp_audit::audit::AuditConfig; use grasp_audit::audit::AuditConfig;
@@ -101,100 +103,41 @@ let config = AuditConfig::shared();
### Creating Audit Events ### Creating Audit Events
```rust From [`grasp-audit/src/client.rs`](grasp-audit/src/client.rs):
use grasp_audit::audit::{AuditConfig, AuditEventBuilder};
use nostr_sdk::prelude::*;
let config = AuditConfig::isolated();
let keys = Keys::generate();
// Create audit event
let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content")
.build(&keys)?;
// Event automatically includes:
// - Audit marker tag
// - Run ID tag
// - Cleanup timestamp tag
```
---
### Querying Audit Events
```rust ```rust
use grasp_audit::client::AuditClient; use grasp_audit::client::AuditClient;
use grasp_audit::audit::AuditConfig; use grasp_audit::audit::AuditConfig;
let config = AuditConfig::isolated(); let config = AuditConfig::isolated();
let client = AuditClient::new(config, keys); let client = AuditClient::new("ws://localhost:8080", config).await?;
// Connect to relay // Create and send an event - cleanup tags are added automatically
client.add_relay("ws://localhost:7000").await?; let event = client.event_builder()
client.connect().await; .kind(Kind::TextNote)
.content("test content")
.build(&keys)?;
// Query audit events for this run client.send_event(event).await?;
let events = client.query().await?;
// Events are filtered by:
// - "grasp-audit-test-event" marker
// - Current run ID
``` ```
--- ---
### Test Isolation ### Test Suites
**Each test run is isolated by unique run ID:** From [`grasp-audit/src/specs/grasp01/mod.rs`](grasp-audit/src/specs/grasp01/mod.rs):
```rust
// CI mode generates unique UUID per run
let config1 = AuditConfig::isolated();
let config2 = AuditConfig::isolated();
// config1.run_id != config2.run_id
// Tests won't interfere with each other
```
**Benefits:**
- ✅ Parallel CI/CD runs don't conflict
- ✅ Can run multiple test suites simultaneously
- ✅ Easy to identify which run created which events
- ✅ Cleanup can target specific runs
---
### Cleanup Strategy
**Two-phase cleanup:**
1. **Automatic expiry** via cleanup timestamp tag
2. **Manual cleanup** by querying and deleting
```rust
// Events include cleanup timestamp
["t", "audit-cleanup-after-1730707200"]
// Cleanup process:
// 1. Query events with expired cleanup timestamp
// 2. Delete from database directly (no KIND 5)
// 3. Avoid deletion event pollution
```
**Implementation:** To be built in relay (not in audit tool)
---
## Testing Strategy
### Test Organization
``` ```
grasp-audit/src/specs/ grasp-audit/src/specs/grasp01/
├── nip01_smoke.rs # NIP-01 basic functionality ├── mod.rs # Module exports
├── grasp_01_relay.rs # GRASP-01 relay requirements (planned) ├── nip01_smoke.rs # NIP-01 basic functionality
└── mod.rs # Test suite registry ├── nip11_document.rs # NIP-11 document tests
├── event_acceptance_policy.rs # GRASP-01 event rules
├── cors.rs # CORS header tests
├── git_clone.rs # Git clone operations
├── push_authorization.rs # Push validation tests
├── repository_creation.rs # Repository lifecycle
└── spec_requirements.rs # Requirement definitions
``` ```
### Unit vs Integration Tests ### Unit vs Integration Tests
@@ -229,17 +172,18 @@ mod tests {
```bash ```bash
# Unit tests (fast, no dependencies) # Unit tests (fast, no dependencies)
cargo test --lib cd grasp-audit && nix develop -c cargo test --lib
# Integration tests (requires relay) # Integration tests (requires relay via test-ngit-relay.sh)
docker run --rm -p 7000:7000 scsibug/nostr-rs-relay cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
cargo test -- --ignored
``` ```
--- ---
### Test Result Reporting ### Test Result Reporting
From [`grasp-audit/src/result.rs`](grasp-audit/src/result.rs):
```rust ```rust
use grasp_audit::result::AuditResult; use grasp_audit::result::AuditResult;
@@ -255,7 +199,7 @@ for result in &results {
} }
// Summary // Summary
let passed = results.iter().filter(|r| r.is_pass()).count(); let passed = results.iter().filter(|r| r.passed).count();
let total = results.len(); let total = results.len();
println!("Results: {}/{} passed ({:.1}%)", println!("Results: {}/{} passed ({:.1}%)",
passed, total, (passed as f64 / total as f64) * 100.0); passed, total, (passed as f64 / total as f64) * 100.0);
@@ -291,7 +235,7 @@ grasp-audit audit \
grasp-audit audit \ grasp-audit audit \
--relay wss://relay.example.com \ --relay wss://relay.example.com \
--mode production \ --mode production \
--run-id "audit-2025-11-04" \ --run-id "audit-2025-12-04" \
--verbose --verbose
# Test all specs # Test all specs
@@ -366,25 +310,23 @@ let events = client.query().await?;
--- ---
## Future Enhancements ## What's Implemented
### Planned Features ### Completed Features
- [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests - ✅ **GRASP-01 Test Suites**: All NIP-01, NIP-11, CORS, event acceptance tests
- [ ] **Test Report Generation**: JSON/HTML output for CI/CD - ✅ **Spec Requirements Database**: Machine-readable requirements in [`spec_requirements.rs`](grasp-audit/src/specs/grasp01/spec_requirements.rs)
- ✅ **Automatic Cleanup Tags**: Production-safe event tagging
- ✅ **Test Isolation**: UUID run IDs for parallel execution
- ✅ **AuditClient**: Nostr client wrapper with audit features
- ✅ **Fixture Helpers**: Event creation helpers in [`fixtures.rs`](grasp-audit/src/fixtures.rs)
### Future Enhancements
- [ ] **GRASP-02 Test Suite**: Proactive sync tests
- [ ] **HTML Report Generation**: Rich CI/CD reports
- [ ] **Performance Benchmarks**: Measure relay performance - [ ] **Performance Benchmarks**: Measure relay performance
- [ ] **Relay Comparison**: Side-by-side compliance comparison - [ ] **Relay Comparison**: Side-by-side compliance comparison
- [ ] **Continuous Monitoring**: Periodic production audits
---
### Possible Improvements
- [ ] **Parallel Test Execution**: Run specs in parallel
- [ ] **Retry Logic**: Handle transient failures
- [ ] **Custom Assertions**: Domain-specific test helpers
- [ ] **Event Diff Tool**: Compare expected vs actual events
- [ ] **Cleanup Automation**: Auto-cleanup after tests
--- ---
@@ -403,14 +345,12 @@ let events = client.query().await?;
**Solution:** **Solution:**
```bash ```bash
# Start relay # Use test-ngit-relay.sh for automated relay management
docker run --rm -p 7000:7000 scsibug/nostr-rs-relay cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
# Verify relay is running # Or manually:
curl http://localhost:7000 docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest
RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored
# Run tests
cargo test -- --ignored
``` ```
--- ---
@@ -471,46 +411,39 @@ let config = AuditConfig::isolated();
let config = AuditConfig::shared(); let config = AuditConfig::shared();
``` ```
### Event Creation
```rust
let event = AuditEventBuilder::new(&config, kind, content)
.build(&keys)?;
```
### Client Usage ### Client Usage
```rust ```rust
let client = AuditClient::new(config, keys); let client = AuditClient::new("ws://localhost:7000", config).await?;
client.add_relay("ws://localhost:7000").await?; assert!(client.is_connected().await);
client.connect().await;
let events = client.query().await?;
``` ```
### Running Tests ### Running Tests
```bash ```bash
# Unit tests # Unit tests (from grasp-audit/)
cargo test --lib nix develop -c cargo test --lib
# Integration tests # Integration tests with ngit-relay
cargo test -- --ignored nix develop -c bash test-ngit-relay.sh --mode test
# CLI # CLI audit
cargo run -- audit --relay ws://localhost:7000 nix develop -c cargo run -- audit --relay ws://localhost:7000
``` ```
--- ### Key Files
## References | File | Purpose |
|------|---------|
- **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp | [`grasp-audit/src/lib.rs`](grasp-audit/src/lib.rs) | Public API |
- **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md | [`grasp-audit/src/client.rs`](grasp-audit/src/client.rs) | AuditClient implementation |
- **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md | [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs) | AuditConfig, cleanup tags |
- **grasp-audit README**: `grasp-audit/README.md` | [`grasp-audit/src/specs/grasp01/mod.rs`](grasp-audit/src/specs/grasp01/mod.rs) | Test suite registry |
- **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md` | [`grasp-audit/src/specs/grasp01/spec_requirements.rs`](grasp-audit/src/specs/grasp01/spec_requirements.rs) | Requirement database |
--- ---
_Last updated: November 4, 2025_ ## Related Documentation
_Status: Living document - update as grasp-audit evolves_
- [Test Strategy](../reference/test-strategy.md) - Overall testing approach
- [GRASP-01 Implementation](grasp-01-implementation.md) - Main project learnings
File diff suppressed because it is too large Load Diff