mirror of
https://relay.ngit.dev/npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr/ngit-grasp.git
synced 2026-10-05 15:08:24 +00:00
docs: update based on current implementation
This commit is contained in:
DanConwayDev
@@ -28,95 +28,69 @@ nix-shell
|
||||
nix-shell --run "cargo build"
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
### Testing ngit-grasp (Main Project)
|
||||
|
||||
**Integration tests require relay running:**
|
||||
**ngit-grasp integration tests use the [`TestRelay`](tests/common/relay.rs:14) fixture:**
|
||||
|
||||
The `TestRelay` fixture automatically starts an instance of ngit-grasp itself and manages its lifecycle:
|
||||
|
||||
```bash
|
||||
# Start ngit-relay first (use any available port to avoid conflicts)
|
||||
docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest
|
||||
# Run all ngit-grasp tests (from project root)
|
||||
cargo test
|
||||
|
||||
# From grasp-audit directory, set RELAY_URL to match your port
|
||||
# Run all ignored tests (includes GRASP-01 and other relay-dependent tests)
|
||||
RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored --nocapture
|
||||
# Run integration tests only
|
||||
cargo test --test '*'
|
||||
|
||||
# Or run a specific test
|
||||
RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture
|
||||
# Run specific test file
|
||||
cargo test --test nip01_compliance
|
||||
```
|
||||
|
||||
Tests marked `#[ignore]` need relay - unit tests don't.
|
||||
**How TestRelay works:**
|
||||
|
||||
**Note:** Always use a random available port for the relay to avoid conflicts with existing services.
|
||||
- Spawns `ngit-grasp` binary on a random available port
|
||||
- Creates temporary directories for git and relay data
|
||||
- Provides `url()` and `domain()` methods for test clients
|
||||
- Automatically cleans up on drop
|
||||
|
||||
### Standard Testing Process (Recommended)
|
||||
**Example test pattern:**
|
||||
|
||||
**Use test-ngit-relay.sh for automated relay management:**
|
||||
```rust
|
||||
use common::TestRelay;
|
||||
|
||||
This script handles all relay lifecycle management automatically:
|
||||
- Starts ngit-relay in isolated Docker container
|
||||
- Uses random port to avoid conflicts
|
||||
- Creates isolated temporary directories
|
||||
- Ensures cleanup on exit (success or failure)
|
||||
- Supports both audit and test modes
|
||||
|
||||
**Basic Usage:**
|
||||
|
||||
```bash
|
||||
# Run cargo test suite (recommended for GRASP-01 development)
|
||||
cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
|
||||
|
||||
# Run audit CLI tool (for quick validation)
|
||||
cd grasp-audit && nix develop -c bash test-ngit-relay.sh
|
||||
|
||||
# Get help
|
||||
cd grasp-audit && ./test-ngit-relay.sh --help
|
||||
#[tokio::test]
|
||||
async fn test_something() {
|
||||
let relay = TestRelay::start().await;
|
||||
// relay.url() returns "ws://127.0.0.1:{port}"
|
||||
// ... run test against ngit-grasp ...
|
||||
relay.stop().await;
|
||||
}
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- No manual relay startup required
|
||||
- Automatic cleanup prevents leftover containers
|
||||
- Random port selection avoids conflicts
|
||||
- Consistent environment across all runs
|
||||
- Proper test isolation
|
||||
### Summary: Which Test Command for What
|
||||
|
||||
**Note:** Manual relay setup is still available but test-ngit-relay.sh is recommended for development workflows.
|
||||
| What you're testing | Command |
|
||||
| --------------------------- | ---------------------------------------------------------------------- |
|
||||
| ngit-grasp (this project) | `cargo test` from project root |
|
||||
| ngit-relay (reference impl) | `cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test` |
|
||||
| grasp-audit unit tests | `cd grasp-audit && nix develop -c cargo test --lib` |
|
||||
|
||||
### Running Single Test
|
||||
|
||||
```bash
|
||||
# From grasp-audit/
|
||||
# ngit-grasp test (from project root)
|
||||
cargo test --test nip01_compliance test_websocket_connection -- --nocapture
|
||||
|
||||
# grasp-audit test (from grasp-audit/)
|
||||
nix develop -c cargo test --lib specific_test_name -- --nocapture
|
||||
```
|
||||
|
||||
### Quick Test Verification
|
||||
|
||||
To verify GRASP-01 compliance tests are working correctly:
|
||||
|
||||
```bash
|
||||
# Run all ignored library tests (includes GRASP-01)
|
||||
cd grasp-audit && RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored --nocapture 2>&1 | tail -60
|
||||
|
||||
# Or run specific GRASP-01 test
|
||||
cd grasp-audit && RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture 2>&1 | tail -60
|
||||
```
|
||||
|
||||
**Expected Output:**
|
||||
- 2-3 tests passing
|
||||
- 15+ tests showing "Not implemented yet"
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
**Buffer Size Errors:**
|
||||
If you see mpsc channel buffer size panics on first test run, this is usually transient. Simply run the tests again.
|
||||
|
||||
**Verify Relay is Running:**
|
||||
Check if relay is accessible before running tests:
|
||||
```bash
|
||||
nak req -l 1 ws://localhost:18081 # Replace port with your chosen port
|
||||
```
|
||||
|
||||
**Port Conflicts:**
|
||||
Always use a random available port to avoid conflicts with existing services. If a port is busy, choose a different one for docker.
|
||||
Both `TestRelay` and `test-ngit-relay.sh` use random ports to avoid conflicts. If you see port errors, ensure no stale processes are running.
|
||||
|
||||
## Code Patterns
|
||||
|
||||
@@ -156,8 +130,6 @@ EventBuilder::new(kind, content, &[tags])
|
||||
EventBuilder::new(kind, content).tags(tags)
|
||||
```
|
||||
|
||||
See `docs/archive/2025-11-04-nostr-sdk-upgrade.md` for full migration.
|
||||
|
||||
### Audit Event Tagging (grasp-audit)
|
||||
|
||||
**All audit events automatically include cleanup tags:**
|
||||
@@ -218,10 +190,10 @@ fn test_audit_tags_automatically_added() {
|
||||
1. **Workspace compilation:** Can't `cargo build` from root for grasp-audit
|
||||
2. **Nix environment:** Must use `nix develop`, not `nix-shell`
|
||||
3. **nostr-sdk API:** Fields not methods in 0.43
|
||||
4. **Test isolation:** Integration tests need relay, marked with `#[ignore]`
|
||||
4. **Test isolation:** Integration tests use `TestRelay` (ngit-grasp) or `test-ngit-relay.sh` (ngit-relay)
|
||||
5. **Work directory:** All session docs go in `work/`, NOT root
|
||||
6. **Archive naming:** Use `YYYY-MM-DD-description.md` format
|
||||
7. **Use test-ngit-relay.sh**: Always use the test script for GRASP-01 tests - it handles cleanup and port management automatically
|
||||
7. **test-ngit-relay.sh tests ngit-relay**: This script tests the reference implementation, NOT ngit-grasp
|
||||
|
||||
## File Restrictions by Mode
|
||||
|
||||
@@ -234,19 +206,14 @@ Code mode can only edit files matching specific patterns (enforced by system):
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
# Recommended: Use test-ngit-relay.sh for all testing
|
||||
cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
|
||||
# Test ngit-grasp (main project)
|
||||
cargo test
|
||||
|
||||
# Build grasp-audit
|
||||
cd grasp-audit && nix develop -c cargo build
|
||||
|
||||
# Manual relay testing (if needed)
|
||||
# 1. Start relay: docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest
|
||||
# 2. Run all ignored tests: RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored --nocapture
|
||||
# 3. Or specific test: RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture
|
||||
|
||||
# Run single test
|
||||
cd grasp-audit && nix develop -c cargo test --lib test_name -- --nocapture
|
||||
# Run grasp-audit unit tests
|
||||
cd grasp-audit && nix develop -c cargo test --lib
|
||||
|
||||
# Check session files
|
||||
ls work/ # Should only have README.md when clean
|
||||
|
||||
@@ -14,8 +14,6 @@ Unlike the reference implementation ([ngit-relay](https://gitworkshop.dev/npub15
|
||||
|
||||
## Status
|
||||
|
||||
**ALPHA** - Under active development. API and architecture subject to change.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Pure Rust Implementation**: Single binary, no external dependencies beyond Git itself
|
||||
|
||||
+186
-552
@@ -2,16 +2,16 @@
|
||||
|
||||
## Executive Summary
|
||||
|
||||
`ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that the `git-http-backend` Rust crate provides sufficient flexibility to intercept and validate Git push operations before they reach the Git repository, eliminating the need for pre-receive hooks.
|
||||
`ngit-grasp` implements the GRASP protocol in Rust with **inline authorization** rather than Git hooks. The key architectural insight is that we can intercept and validate Git push operations at the HTTP handler level before reaching the Git repository, eliminating the need for pre-receive hooks.
|
||||
|
||||
## Architectural Decision: Inline vs. Hook-Based Authorization
|
||||
|
||||
### Investigation Summary
|
||||
|
||||
After examining both the reference implementation and the `git-http-backend` Rust crate, we have two options:
|
||||
After examining both the reference implementation and HTTP server options, we have two options:
|
||||
|
||||
#### Option 1: Hook-Based (Reference Implementation Approach)
|
||||
- Use `git-http-backend` crate as-is
|
||||
- Use standard Git HTTP backend
|
||||
- Create pre-receive and post-receive hooks
|
||||
- Hooks query the Nostr relay and validate pushes
|
||||
- **Pros**: Follows reference implementation closely
|
||||
@@ -28,7 +28,7 @@ After examining both the reference implementation and the `git-http-backend` Rus
|
||||
|
||||
**Rationale:**
|
||||
|
||||
1. **The `git-http-backend` crate is sufficiently flexible**: Examining `src/actix/git_receive_pack.rs` shows it spawns `git receive-pack` as a subprocess and streams data. We can intercept this.
|
||||
1. **Full control over HTTP layer**: Using Hyper directly gives us complete control over request handling, WebSocket upgrades, and CORS headers.
|
||||
|
||||
2. **Better Developer Experience**:
|
||||
- Validation errors can be returned as proper HTTP responses
|
||||
@@ -56,7 +56,7 @@ After examining both the reference implementation and the `git-http-backend` Rus
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ │
|
||||
│ │ HTTP Router │ │ Nostr Relay │ │
|
||||
│ │ (actix-web) │ │ (nostr-relay- │ │
|
||||
│ │ (Hyper) │ │ (nostr-relay- │ │
|
||||
│ │ │ │ builder) │ │
|
||||
│ └────────┬─────────┘ └────────┬─────────┘ │
|
||||
│ │ │ │
|
||||
@@ -90,299 +90,149 @@ After examining both the reference implementation and the `git-http-backend` Rus
|
||||
|
||||
## Component Design
|
||||
|
||||
### 1. Main Server (`src/main.rs`)
|
||||
### 1. Main Server ([`src/main.rs`](src/main.rs))
|
||||
|
||||
**Responsibilities:**
|
||||
- Initialize configuration from environment
|
||||
- Set up actix-web HTTP server
|
||||
- Initialize Nostr relay builder
|
||||
- Set up shared storage
|
||||
- Configure routes for both Git and Nostr endpoints
|
||||
- Initialize configuration from environment (clap + dotenvy)
|
||||
- Set up Hyper HTTP server with request routing
|
||||
- Initialize Nostr relay builder with custom [`Nip34WritePolicy`](src/nostr/builder.rs:51)
|
||||
- Set up shared storage (LMDB, NostrDB, or Memory)
|
||||
- Handle WebSocket upgrades for Nostr relay
|
||||
- Handle graceful shutdown
|
||||
|
||||
**Key Dependencies:**
|
||||
```rust
|
||||
actix-web = "4"
|
||||
hyper = "1"
|
||||
tokio = { version = "1", features = ["full"] }
|
||||
nostr-relay-builder = "0.43"
|
||||
nostr-sdk = "0.43"
|
||||
nostr-lmdb = "0.43"
|
||||
```
|
||||
|
||||
### 2. Git Module (`src/git/`)
|
||||
### 2. HTTP Module ([`src/http/mod.rs`](src/http/mod.rs))
|
||||
|
||||
#### `handler.rs` - Git HTTP Handlers
|
||||
**Responsibilities:**
|
||||
- Route HTTP requests to appropriate handlers
|
||||
- WebSocket upgrade for Nostr relay at `/`
|
||||
- Git Smart HTTP endpoints at `/<npub>/<identifier>.git/*`
|
||||
- Landing pages and NIP-11 document serving
|
||||
- CORS headers on all responses (GRASP-01 requirement)
|
||||
|
||||
Implements actix-web handlers for Git Smart HTTP protocol:
|
||||
**Key Implementation Details:**
|
||||
|
||||
```rust
|
||||
// GET /<npub>/<identifier>.git/info/refs?service=git-upload-pack
|
||||
async fn info_refs_upload_pack(
|
||||
req: HttpRequest,
|
||||
state: web::Data<AppState>,
|
||||
) -> Result<HttpResponse>
|
||||
// CORS headers required by GRASP-01 specification
|
||||
const CORS_ALLOW_ORIGIN: &str = "*";
|
||||
const CORS_ALLOW_METHODS: &str = "GET, POST";
|
||||
const CORS_ALLOW_HEADERS: &str = "Content-Type";
|
||||
|
||||
// POST /<npub>/<identifier>.git/git-upload-pack
|
||||
async fn git_upload_pack(
|
||||
req: HttpRequest,
|
||||
body: web::Payload,
|
||||
state: web::Data<AppState>,
|
||||
) -> Result<HttpResponse>
|
||||
|
||||
// GET /<npub>/<identifier>.git/info/refs?service=git-receive-pack
|
||||
async fn info_refs_receive_pack(
|
||||
req: HttpRequest,
|
||||
state: web::Data<AppState>,
|
||||
) -> Result<HttpResponse>
|
||||
|
||||
// POST /<npub>/<identifier>.git/git-receive-pack
|
||||
// THIS IS WHERE THE MAGIC HAPPENS
|
||||
async fn git_receive_pack(
|
||||
req: HttpRequest,
|
||||
body: web::Payload,
|
||||
state: web::Data<AppState>,
|
||||
) -> Result<HttpResponse>
|
||||
/// Add CORS headers to a response builder
|
||||
fn add_cors_headers(builder: http::response::Builder) -> http::response::Builder {
|
||||
builder
|
||||
.header("Access-Control-Allow-Origin", CORS_ALLOW_ORIGIN)
|
||||
.header("Access-Control-Allow-Methods", CORS_ALLOW_METHODS)
|
||||
.header("Access-Control-Allow-Headers", CORS_ALLOW_HEADERS)
|
||||
}
|
||||
```
|
||||
|
||||
#### `authorization.rs` - Push Validation
|
||||
See [`src/http/mod.rs:29-84`](src/http/mod.rs:29-84) for the full CORS implementation.
|
||||
|
||||
### 3. Git Module ([`src/git/`](src/git/))
|
||||
|
||||
#### [`handlers.rs`](src/git/handlers.rs) - Git HTTP Handlers
|
||||
|
||||
Implements handlers for Git Smart HTTP protocol:
|
||||
|
||||
```rust
|
||||
/// Handle GET /info/refs?service=git-{upload,receive}-pack
|
||||
pub async fn handle_info_refs(
|
||||
repo_path: PathBuf,
|
||||
service: GitService,
|
||||
) -> Result<Response<Full<Bytes>>, GitError>
|
||||
|
||||
/// Handle POST /git-upload-pack (clone/fetch)
|
||||
pub async fn handle_upload_pack(
|
||||
repo_path: PathBuf,
|
||||
body: Bytes,
|
||||
) -> Result<Response<Full<Bytes>>, GitError>
|
||||
|
||||
/// Handle POST /git-receive-pack (push)
|
||||
/// THIS IS WHERE THE MAGIC HAPPENS - validates against state before accepting
|
||||
pub async fn handle_receive_pack(
|
||||
repo_path: PathBuf,
|
||||
body: Bytes,
|
||||
database: SharedDatabase,
|
||||
npub: &str,
|
||||
identifier: &str,
|
||||
) -> Result<Response<Full<Bytes>>, GitError>
|
||||
```
|
||||
|
||||
See [`src/git/handlers.rs:22-98`](src/git/handlers.rs:22-98) for the info-refs implementation.
|
||||
|
||||
#### [`authorization.rs`](src/git/authorization.rs) - Push Validation
|
||||
|
||||
**Core Logic:**
|
||||
|
||||
```rust
|
||||
pub struct PushValidator {
|
||||
nostr_client: Arc<Client>,
|
||||
relay_url: String,
|
||||
}
|
||||
|
||||
impl PushValidator {
|
||||
/// Validate a push operation against Nostr state
|
||||
pub async fn validate_push(
|
||||
&self,
|
||||
npub: &str,
|
||||
/// Get authorization info for a repository owner
|
||||
pub async fn get_authorization_for_owner(
|
||||
database: &SharedDatabase,
|
||||
pubkey: &PublicKey,
|
||||
identifier: &str,
|
||||
ref_updates: Vec<RefUpdate>,
|
||||
) -> Result<ValidationResult> {
|
||||
// 1. Fetch announcement and state events from local relay
|
||||
let events = self.fetch_events(identifier).await?;
|
||||
) -> Result<AuthorizationResult, AuthorizationError>
|
||||
|
||||
// 2. Extract pubkey from npub
|
||||
let pubkey = decode_npub(npub)?;
|
||||
|
||||
// 3. Get recursive maintainer set
|
||||
let maintainers = get_maintainers(&events, &pubkey, identifier);
|
||||
|
||||
// 4. Get latest state from maintainers
|
||||
let state = get_state_from_maintainers(&events, &maintainers)?;
|
||||
|
||||
// 5. Validate each ref update
|
||||
for ref_update in ref_updates {
|
||||
if ref_update.ref_name.starts_with("refs/nostr/") {
|
||||
// Allow refs/nostr/<event-id> for PRs
|
||||
validate_pr_ref(&ref_update)?;
|
||||
} else if ref_update.ref_name.starts_with("refs/heads/pr/") {
|
||||
// Reject pr/* branches - should use refs/nostr/
|
||||
return Err(Error::InvalidRef("pr/* branches must use refs/nostr/"));
|
||||
} else {
|
||||
// Validate against state event
|
||||
validate_state_ref(&state, &ref_update)?;
|
||||
}
|
||||
}
|
||||
|
||||
Ok(ValidationResult::Accept)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Functions:**
|
||||
|
||||
```rust
|
||||
/// Parse ref updates from git-receive-pack request body
|
||||
fn parse_ref_updates(body: &[u8]) -> Result<Vec<RefUpdate>>
|
||||
|
||||
/// Recursively find all maintainers
|
||||
fn get_maintainers(
|
||||
events: &[Event],
|
||||
pubkey: &str,
|
||||
identifier: &str,
|
||||
) -> Vec<String>
|
||||
|
||||
/// Get latest state from maintainer set
|
||||
fn get_state_from_maintainers(
|
||||
events: &[Event],
|
||||
maintainers: &[String],
|
||||
) -> Result<RepositoryState>
|
||||
|
||||
/// Validate a ref matches the state event
|
||||
fn validate_state_ref(
|
||||
/// Validate that pushed refs match the authorized state
|
||||
pub fn validate_push_refs(
|
||||
pushed_refs: &[PushedRef],
|
||||
state: &RepositoryState,
|
||||
ref_update: &RefUpdate,
|
||||
) -> Result<()>
|
||||
) -> Result<(), AuthorizationError>
|
||||
|
||||
/// Validate refs/nostr/<event-id> pushes
|
||||
pub fn validate_nostr_ref_pushes(
|
||||
pushed_refs: &[PushedRef],
|
||||
database: &SharedDatabase,
|
||||
) -> Result<(), AuthorizationError>
|
||||
```
|
||||
|
||||
### 3. Nostr Module (`src/nostr/`)
|
||||
### 4. Nostr Module ([`src/nostr/`](src/nostr/))
|
||||
|
||||
#### `relay.rs` - Relay Configuration
|
||||
#### [`builder.rs`](src/nostr/builder.rs) - Relay Configuration
|
||||
|
||||
The [`Nip34WritePolicy`](src/nostr/builder.rs:51) is the core event validation logic:
|
||||
|
||||
```rust
|
||||
pub async fn build_relay(config: &Config) -> Result<LocalRelay> {
|
||||
let builder = RelayBuilder::default()
|
||||
.write_policy(RepositoryAnnouncementPolicy::new(config.domain.clone()))
|
||||
.write_policy(RelatedEventsPolicy::new())
|
||||
.query_policy(StandardQueryPolicy::new())
|
||||
.on_event_saved(create_repository_hook(config.git_data_path.clone()));
|
||||
|
||||
// Configure storage backend (LMDB or NDB)
|
||||
let relay = LocalRelay::run(builder).await?;
|
||||
|
||||
Ok(relay)
|
||||
}
|
||||
```
|
||||
|
||||
#### `events.rs` - Event Handlers
|
||||
|
||||
```rust
|
||||
/// Hook called when events are saved
|
||||
pub fn create_repository_hook(
|
||||
git_data_path: PathBuf,
|
||||
) -> impl Fn(&Event) -> BoxFuture<'static, ()> {
|
||||
move |event: &Event| {
|
||||
let git_path = git_data_path.clone();
|
||||
Box::pin(async move {
|
||||
if event.kind == Kind::RepositoryAnnouncement {
|
||||
handle_repository_announcement(event, &git_path).await;
|
||||
} else if event.kind == Kind::RepositoryState {
|
||||
handle_repository_state(event, &git_path).await;
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_repository_announcement(event: &Event, git_path: &Path) {
|
||||
// 1. Parse repository from event
|
||||
// 2. Check if listed in clone and relays tags
|
||||
// 3. Create empty bare Git repository
|
||||
// 4. Configure uploadpack.allowTipSHA1InWant
|
||||
// 5. Configure uploadpack.allowUnreachable
|
||||
// 6. Configure http.receivepack
|
||||
}
|
||||
|
||||
async fn handle_repository_state(event: &Event, git_path: &Path) {
|
||||
// 1. Parse state from event
|
||||
// 2. Update repository HEAD if needed
|
||||
// 3. Trigger proactive sync (GRASP-02)
|
||||
}
|
||||
```
|
||||
|
||||
**Write Policies:**
|
||||
|
||||
```rust
|
||||
/// Accept repository announcements that list this instance
|
||||
pub struct RepositoryAnnouncementPolicy {
|
||||
/// NIP-34 Write Policy with Full GRASP-01 Event Validation
|
||||
///
|
||||
/// Validates all events according to GRASP-01 specification:
|
||||
/// - Repository announcements must list service in clone and relays tags
|
||||
/// EXCEPTION: Recursive maintainer announcements are accepted even without
|
||||
/// listing the service, to enable maintainer chain discovery and GRASP-02 sync
|
||||
/// - Repository state announcements must have valid structure
|
||||
/// - Other events must reference accepted repositories or events
|
||||
/// - Forward references are supported (events referenced by accepted events)
|
||||
/// - Orphan events with no valid references are rejected
|
||||
pub struct Nip34WritePolicy {
|
||||
domain: String,
|
||||
}
|
||||
|
||||
impl WritePolicy for RepositoryAnnouncementPolicy {
|
||||
fn admit_event(&self, event: &Event, _addr: &SocketAddr)
|
||||
-> BoxFuture<PolicyResult>
|
||||
{
|
||||
Box::pin(async move {
|
||||
if event.kind != Kind::RepositoryAnnouncement {
|
||||
return PolicyResult::Accept; // Not our concern
|
||||
}
|
||||
|
||||
// Check if this instance is in clone and relays tags
|
||||
let has_clone = event.tags.iter()
|
||||
.any(|t| t.kind() == "clone" && t.content() == Some(&self.domain));
|
||||
let has_relay = event.tags.iter()
|
||||
.any(|t| t.kind() == "relays" && t.content() == Some(&self.domain));
|
||||
|
||||
if has_clone && has_relay {
|
||||
PolicyResult::Accept
|
||||
} else {
|
||||
PolicyResult::Reject("instance not listed in clone and relays".into())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Accept events related to stored announcements/issues/patches
|
||||
pub struct RelatedEventsPolicy;
|
||||
|
||||
impl WritePolicy for RelatedEventsPolicy {
|
||||
fn admit_event(&self, event: &Event, _addr: &SocketAddr)
|
||||
-> BoxFuture<PolicyResult>
|
||||
{
|
||||
// Accept if event tags or is tagged by stored events
|
||||
// Implementation requires querying the event store
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Storage Module (`src/storage/`)
|
||||
|
||||
#### `repository.rs` - Repository Management
|
||||
|
||||
```rust
|
||||
pub struct RepositoryManager {
|
||||
database: SharedDatabase,
|
||||
git_data_path: PathBuf,
|
||||
}
|
||||
|
||||
impl RepositoryManager {
|
||||
/// Create a new bare Git repository
|
||||
pub async fn create_repository(
|
||||
&self,
|
||||
npub: &str,
|
||||
identifier: &str,
|
||||
) -> Result<PathBuf> {
|
||||
let repo_path = self.git_data_path
|
||||
.join(npub)
|
||||
.join(format!("{}.git", identifier));
|
||||
|
||||
// Create directory
|
||||
tokio::fs::create_dir_all(&repo_path).await?;
|
||||
|
||||
// Initialize bare repo
|
||||
Command::new("git")
|
||||
.args(&["init", "--bare"])
|
||||
.arg(&repo_path)
|
||||
.output()
|
||||
.await?;
|
||||
|
||||
// Configure
|
||||
self.configure_repository(&repo_path).await?;
|
||||
|
||||
Ok(repo_path)
|
||||
}
|
||||
|
||||
async fn configure_repository(&self, repo_path: &Path) -> Result<()> {
|
||||
// Enable unauthenticated push (we handle auth ourselves)
|
||||
git_config(repo_path, "http.receivepack", "true").await?;
|
||||
|
||||
// Enable tip SHA1 fetching (required for ngit)
|
||||
git_config(repo_path, "uploadpack.allowTipSHA1InWant", "true").await?;
|
||||
|
||||
// Enable unreachable object fetching
|
||||
git_config(repo_path, "uploadpack.allowUnreachable", "true").await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Check if repository exists
|
||||
pub async fn repository_exists(
|
||||
&self,
|
||||
npub: &str,
|
||||
identifier: &str,
|
||||
) -> bool {
|
||||
let repo_path = self.git_data_path
|
||||
.join(npub)
|
||||
.join(format!("{}.git", identifier));
|
||||
|
||||
repo_path.join("HEAD").exists() &&
|
||||
repo_path.join("config").exists()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Configuration (`src/config.rs`)
|
||||
See [`src/nostr/builder.rs:38-78`](src/nostr/builder.rs:38-78) for the full policy struct.
|
||||
|
||||
#### [`events.rs`](src/nostr/events.rs) - Event Parsing
|
||||
|
||||
Provides structures for parsing NIP-34 events:
|
||||
|
||||
```rust
|
||||
/// Parsed repository announcement (Kind 30617)
|
||||
pub struct RepositoryAnnouncement { ... }
|
||||
|
||||
/// Parsed repository state (Kind 30618)
|
||||
pub struct RepositoryState { ... }
|
||||
```
|
||||
|
||||
### 5. Configuration ([`src/config.rs`](src/config.rs))
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
@@ -393,34 +243,18 @@ pub struct Config {
|
||||
pub git_data_path: PathBuf,
|
||||
pub relay_data_path: PathBuf,
|
||||
pub bind_address: SocketAddr,
|
||||
pub log_level: String,
|
||||
pub database_backend: DatabaseBackend,
|
||||
}
|
||||
|
||||
impl Config {
|
||||
pub fn from_env() -> Result<Self> {
|
||||
Ok(Config {
|
||||
domain: env::var("NGIT_DOMAIN")?,
|
||||
owner_npub: env::var("NGIT_OWNER_NPUB")?,
|
||||
relay_name: env::var("NGIT_RELAY_NAME")?,
|
||||
relay_description: env::var("NGIT_RELAY_DESCRIPTION")?,
|
||||
git_data_path: PathBuf::from(
|
||||
env::var("NGIT_GIT_DATA_PATH")
|
||||
.unwrap_or_else(|_| "./data/git".to_string())
|
||||
),
|
||||
relay_data_path: PathBuf::from(
|
||||
env::var("NGIT_RELAY_DATA_PATH")
|
||||
.unwrap_or_else(|_| "./data/relay".to_string())
|
||||
),
|
||||
bind_address: env::var("NGIT_BIND_ADDRESS")
|
||||
.unwrap_or_else(|_| "127.0.0.1:8080".to_string())
|
||||
.parse()?,
|
||||
log_level: env::var("RUST_LOG")
|
||||
.unwrap_or_else(|_| "info".to_string()),
|
||||
})
|
||||
}
|
||||
pub enum DatabaseBackend {
|
||||
Lmdb, // Default, production use
|
||||
NostrDb, // Alternative
|
||||
Memory, // Testing
|
||||
}
|
||||
```
|
||||
|
||||
Configuration is loaded via **clap CLI > environment variables > .env > defaults**.
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Push Operation Flow
|
||||
@@ -428,327 +262,120 @@ impl Config {
|
||||
```
|
||||
1. Git Client → POST /<npub>/<id>.git/git-receive-pack
|
||||
↓
|
||||
2. git_receive_pack handler receives request
|
||||
2. HttpService routes to git::handlers::handle_receive_pack()
|
||||
↓
|
||||
3. Parse ref updates from request body
|
||||
3. Parse ref updates from request body (pkt-line format)
|
||||
↓
|
||||
4. Extract npub and identifier from URL
|
||||
↓
|
||||
5. PushValidator::validate_push()
|
||||
├─ Fetch events from local Nostr relay
|
||||
├─ Get maintainers recursively
|
||||
├─ Get latest state from maintainers
|
||||
└─ Validate each ref update
|
||||
5. authorization::get_authorization_for_owner()
|
||||
├─ Query database for announcements
|
||||
├─ Build recursive maintainer set
|
||||
└─ Get latest authorized state
|
||||
↓
|
||||
6. If VALID:
|
||||
6. authorization::validate_push_refs()
|
||||
├─ Check each ref matches state
|
||||
└─ Validate refs/nostr/ pushes
|
||||
↓
|
||||
7. If VALID:
|
||||
├─ Spawn git-receive-pack subprocess
|
||||
├─ Stream request body to git stdin
|
||||
└─ Stream git stdout back to client
|
||||
↓
|
||||
7. If INVALID:
|
||||
8. If INVALID:
|
||||
└─ Return HTTP 403 with error message
|
||||
```
|
||||
|
||||
### Repository Announcement Flow
|
||||
|
||||
```
|
||||
1. Nostr Client → EVENT (Kind 30317)
|
||||
1. Nostr Client → EVENT (Kind 30617)
|
||||
↓
|
||||
2. Nostr relay receives event
|
||||
↓
|
||||
3. RepositoryAnnouncementPolicy::admit_event()
|
||||
3. Nip34WritePolicy::admit_event()
|
||||
├─ Check if instance in clone tags
|
||||
├─ Check if instance in relays tags
|
||||
├─ OR: Check if recursive maintainer
|
||||
└─ Accept or reject
|
||||
↓
|
||||
4. If ACCEPTED:
|
||||
├─ Event saved to store
|
||||
└─ on_event_saved hook triggered
|
||||
├─ Event saved to database
|
||||
└─ ensure_bare_repository() called
|
||||
↓
|
||||
5. handle_repository_announcement()
|
||||
├─ Parse repository details
|
||||
├─ Create Git repository directory
|
||||
├─ Initialize bare Git repo
|
||||
└─ Configure Git settings
|
||||
5. Bare Git repository created at
|
||||
<git_data_path>/<npub>/<identifier>.git
|
||||
```
|
||||
|
||||
## Key Implementation Details
|
||||
|
||||
### 1. Parsing Git Receive-Pack Protocol
|
||||
|
||||
The Git receive-pack protocol uses a pkt-line format. We need to parse:
|
||||
### State Event Flow
|
||||
|
||||
```
|
||||
0000-0000-0000-0000 0000-0000-0000-0000 refs/heads/main\0 report-status
|
||||
0000-0000-0000-0000 0000-0000-0000-0000 refs/heads/dev
|
||||
```
|
||||
|
||||
Each line has:
|
||||
- Old SHA (40 hex chars)
|
||||
- Space
|
||||
- New SHA (40 hex chars)
|
||||
- Space
|
||||
- Ref name
|
||||
- Optional capabilities (first line only, after \0)
|
||||
|
||||
```rust
|
||||
pub struct RefUpdate {
|
||||
pub old_sha: String,
|
||||
pub new_sha: String,
|
||||
pub ref_name: String,
|
||||
}
|
||||
|
||||
pub fn parse_ref_updates(body: &[u8]) -> Result<Vec<RefUpdate>> {
|
||||
// Parse pkt-line format
|
||||
// Extract ref updates
|
||||
// Return structured data
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Maintainer Recursion
|
||||
|
||||
The maintainer resolution must handle cycles and correctly build the set:
|
||||
|
||||
```rust
|
||||
fn get_maintainers_recursive(
|
||||
events: &[Event],
|
||||
pubkey: &str,
|
||||
identifier: &str,
|
||||
visited: &mut HashSet<String>,
|
||||
) -> HashSet<String> {
|
||||
if visited.contains(pubkey) {
|
||||
return HashSet::new();
|
||||
}
|
||||
|
||||
visited.insert(pubkey.to_string());
|
||||
|
||||
let announcement = find_announcement(events, pubkey, identifier);
|
||||
if announcement.is_none() {
|
||||
return HashSet::new();
|
||||
}
|
||||
|
||||
let repo = parse_repository(announcement.unwrap());
|
||||
|
||||
for maintainer in repo.maintainers {
|
||||
get_maintainers_recursive(events, &maintainer, identifier, visited);
|
||||
}
|
||||
|
||||
visited.clone()
|
||||
}
|
||||
```
|
||||
|
||||
### 3. State Event Validation
|
||||
|
||||
```rust
|
||||
fn validate_state_ref(
|
||||
state: &RepositoryState,
|
||||
ref_update: &RefUpdate,
|
||||
) -> Result<()> {
|
||||
if ref_update.ref_name.starts_with("refs/heads/") {
|
||||
let branch_name = &ref_update.ref_name[11..];
|
||||
if let Some(commit) = state.branches.get(branch_name) {
|
||||
if commit == &ref_update.new_sha {
|
||||
return Ok(());
|
||||
}
|
||||
return Err(Error::StateMismatch {
|
||||
ref_name: ref_update.ref_name.clone(),
|
||||
expected: commit.clone(),
|
||||
got: ref_update.new_sha.clone(),
|
||||
});
|
||||
}
|
||||
return Err(Error::RefNotInState(ref_update.ref_name.clone()));
|
||||
}
|
||||
|
||||
if ref_update.ref_name.starts_with("refs/tags/") {
|
||||
let tag_name = &ref_update.ref_name[10..];
|
||||
if let Some(commit) = state.tags.get(tag_name) {
|
||||
if commit == &ref_update.new_sha {
|
||||
return Ok(());
|
||||
}
|
||||
return Err(Error::StateMismatch {
|
||||
ref_name: ref_update.ref_name.clone(),
|
||||
expected: commit.clone(),
|
||||
got: ref_update.new_sha.clone(),
|
||||
});
|
||||
}
|
||||
return Err(Error::RefNotInState(ref_update.ref_name.clone()));
|
||||
}
|
||||
|
||||
Err(Error::InvalidRef(ref_update.ref_name.clone()))
|
||||
}
|
||||
```
|
||||
|
||||
### 4. CORS Support
|
||||
|
||||
As per GRASP-01, we must support CORS:
|
||||
|
||||
```rust
|
||||
use actix_cors::Cors;
|
||||
|
||||
fn configure_cors() -> Cors {
|
||||
Cors::default()
|
||||
.allow_any_origin()
|
||||
.allowed_methods(vec!["GET", "POST", "OPTIONS"])
|
||||
.allowed_headers(vec!["Content-Type"])
|
||||
.max_age(3600)
|
||||
}
|
||||
|
||||
// In main.rs
|
||||
App::new()
|
||||
.wrap(configure_cors())
|
||||
.configure(git_routes)
|
||||
.configure(nostr_routes)
|
||||
1. Nostr Client → EVENT (Kind 30618)
|
||||
↓
|
||||
2. Nostr relay receives event
|
||||
↓
|
||||
3. Nip34WritePolicy::admit_event()
|
||||
├─ Check author is in maintainer set
|
||||
├─ Validate state structure
|
||||
└─ Accept or reject
|
||||
↓
|
||||
4. If ACCEPTED and is latest state:
|
||||
├─ Align repository refs to match state
|
||||
├─ Create/update/delete refs as needed
|
||||
└─ Set HEAD if commit available
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
See [TEST_STRATEGY.md](TEST_STRATEGY.md) for comprehensive testing documentation, including:
|
||||
|
||||
- **GRASP Compliance Testing Tool**: Reusable test suite that validates any GRASP implementation against the spec
|
||||
- **Spec-Mirrored Tests**: Test structure matches GRASP protocol documents exactly
|
||||
- **Clear Failure Messages**: Test failures cite exact spec lines (e.g., "GRASP-01:12-13")
|
||||
- **Multiple Test Levels**: Unit, integration, compliance, and end-to-end tests
|
||||
See [test-strategy.md](../reference/test-strategy.md) for comprehensive testing documentation.
|
||||
|
||||
### Quick Overview
|
||||
|
||||
**Integration Tests** ([`tests/`](tests/)):
|
||||
- Use [`TestRelay`](tests/common/relay.rs:14) fixture for automatic relay lifecycle
|
||||
- Each test file in [`tests/`](tests/) covers a GRASP-01 requirement
|
||||
|
||||
**Audit Tests** ([`grasp-audit/`](grasp-audit/)):
|
||||
- Reusable compliance testing for any GRASP implementation
|
||||
- Spec-mirrored structure in [`grasp-audit/src/specs/grasp01/`](grasp-audit/src/specs/grasp01/)
|
||||
|
||||
```rust
|
||||
// Unit Tests - Individual functions
|
||||
#[test]
|
||||
fn test_parse_ref_updates() {
|
||||
let body = b"0000... 0000... refs/heads/main\0report-status\n";
|
||||
let updates = parse_ref_updates(body).unwrap();
|
||||
assert_eq!(updates.len(), 1);
|
||||
assert_eq!(updates[0].ref_name, "refs/heads/main");
|
||||
}
|
||||
|
||||
// Integration Tests - Component interaction
|
||||
// Example: tests/nip01_compliance.rs
|
||||
#[tokio::test]
|
||||
async fn test_full_push_flow() {
|
||||
let app = test_app().await;
|
||||
let (announcement, state) = app.create_repo_with_state()
|
||||
.branch("main", "commit-123")
|
||||
.build()
|
||||
.await;
|
||||
|
||||
let result = app.git_push("main", "commit-123").await;
|
||||
assert!(result.success);
|
||||
}
|
||||
|
||||
// Compliance Tests - GRASP spec validation
|
||||
#[tokio::test]
|
||||
async fn test_grasp_01_compliance() {
|
||||
use grasp_compliance_tests::{TestContext, Grasp01Spec};
|
||||
|
||||
let ctx = TestContext::builder()
|
||||
.base_url(&server.url())
|
||||
.build();
|
||||
|
||||
let results = Grasp01Spec::test_compliance(&ctx).await;
|
||||
assert!(results.all_passed(), "{}", results.report());
|
||||
async fn test_nip01_websocket_connection() {
|
||||
let relay = TestRelay::start().await;
|
||||
// Test NIP-01 compliance...
|
||||
relay.stop().await;
|
||||
}
|
||||
```
|
||||
|
||||
The compliance testing tool is designed as a **standalone crate** that can be:
|
||||
- Used by ngit-grasp for self-validation
|
||||
- Published for other GRASP implementations to use
|
||||
- Updated as new GRASP specs are released
|
||||
- Run in CI/CD for continuous compliance verification
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### 1. Async All The Way
|
||||
|
||||
- Use `tokio` for all I/O
|
||||
- Non-blocking Git subprocess spawning
|
||||
- Non-blocking Git subprocess spawning via [`GitSubprocess`](src/git/subprocess.rs)
|
||||
- Stream large pack files without buffering
|
||||
|
||||
### 2. Connection Pooling
|
||||
### 2. Shared Database
|
||||
|
||||
- Reuse Nostr relay connections
|
||||
- Connection pool for internal relay queries
|
||||
- Single database instance shared between relay and Git handlers
|
||||
- Direct queries for push authorization (no WebSocket round-trip)
|
||||
|
||||
### 3. Caching
|
||||
### 3. Write Policy Caching
|
||||
|
||||
- Cache parsed state events (with TTL)
|
||||
- Cache maintainer sets
|
||||
- Invalidate on new state events
|
||||
|
||||
```rust
|
||||
pub struct StateCache {
|
||||
cache: Arc<RwLock<HashMap<String, CachedState>>>,
|
||||
}
|
||||
|
||||
struct CachedState {
|
||||
state: RepositoryState,
|
||||
maintainers: Vec<String>,
|
||||
timestamp: Instant,
|
||||
}
|
||||
|
||||
impl StateCache {
|
||||
pub async fn get_or_fetch(
|
||||
&self,
|
||||
identifier: &str,
|
||||
fetcher: impl Future<Output = Result<(RepositoryState, Vec<String>)>>,
|
||||
) -> Result<(RepositoryState, Vec<String>)> {
|
||||
// Check cache
|
||||
// Return if fresh
|
||||
// Otherwise fetch and cache
|
||||
}
|
||||
}
|
||||
```
|
||||
- Maintainer sets computed once per event validation
|
||||
- State lookups use database indexes
|
||||
|
||||
## Future Extensions
|
||||
|
||||
### GRASP-02: Proactive Sync
|
||||
|
||||
Add background tasks:
|
||||
|
||||
```rust
|
||||
pub struct ProactiveSyncTask {
|
||||
relay_client: Client,
|
||||
git_manager: RepositoryManager,
|
||||
}
|
||||
|
||||
impl ProactiveSyncTask {
|
||||
pub async fn run(&self) {
|
||||
loop {
|
||||
tokio::time::sleep(Duration::from_secs(3600)).await;
|
||||
|
||||
// Fetch all announcements from our relay
|
||||
let announcements = self.fetch_announcements().await;
|
||||
|
||||
for ann in announcements {
|
||||
// Sync events from listed relays
|
||||
self.sync_events(&ann).await;
|
||||
|
||||
// Sync git data from listed clones
|
||||
self.sync_git_data(&ann).await;
|
||||
|
||||
// Fetch PR data
|
||||
self.sync_pr_data(&ann).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
See [grasp-02-proactive-sync.md](grasp-02-proactive-sync.md) for detailed design.
|
||||
|
||||
### GRASP-05: Archive
|
||||
|
||||
Relax the policy:
|
||||
|
||||
```rust
|
||||
pub struct ArchiveAnnouncementPolicy;
|
||||
|
||||
impl WritePolicy for ArchiveAnnouncementPolicy {
|
||||
fn admit_event(&self, event: &Event, _addr: &SocketAddr)
|
||||
-> BoxFuture<PolicyResult>
|
||||
{
|
||||
// Accept all repository announcements
|
||||
// Don't check clone/relays tags
|
||||
PolicyResult::Accept
|
||||
}
|
||||
}
|
||||
```
|
||||
Relax the write policy to accept all repository announcements regardless of clone/relays tags.
|
||||
|
||||
## Deployment
|
||||
|
||||
@@ -756,7 +383,7 @@ impl WritePolicy for ArchiveAnnouncementPolicy {
|
||||
|
||||
```bash
|
||||
cargo build --release
|
||||
./target/release/ngit-grasp
|
||||
./target/release/ngit-grasp --domain example.com --owner-npub npub1...
|
||||
```
|
||||
|
||||
### Docker
|
||||
@@ -799,10 +426,17 @@ WantedBy=multi-user.target
|
||||
2. **Path Traversal**: Prevent directory traversal in repository paths
|
||||
3. **DoS Protection**: Rate limiting on both HTTP and WebSocket
|
||||
4. **Resource Limits**: Limit pack file sizes, event sizes
|
||||
5. **Nostr Event Validation**: Strict signature verification
|
||||
5. **Nostr Event Validation**: Strict signature verification (handled by nostr-relay-builder)
|
||||
|
||||
## Conclusion
|
||||
|
||||
The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. The Rust ecosystem provides excellent libraries for both Git and Nostr protocols, enabling a high-performance, type-safe implementation.
|
||||
The inline authorization approach provides a cleaner, more maintainable architecture than hook-based authorization while maintaining full GRASP-01 compliance. Using Hyper for the HTTP layer gives us complete control over request handling, WebSocket upgrades, and CORS headers.
|
||||
|
||||
The key insight is that we don't need to rely on Git's hook mechanism when we have full control over the HTTP layer that Git operates through. By intercepting at the HTTP handler level, we gain better error handling, easier testing, and tighter integration between the Git and Nostr components.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Inline Authorization Explanation](inline-authorization.md) - Why we chose this approach
|
||||
- [GRASP-02 Proactive Sync](grasp-02-proactive-sync.md) - Future work design
|
||||
- [Test Strategy](../reference/test-strategy.md) - Comprehensive testing documentation
|
||||
- [GRASP-01 Implementation Learnings](../learnings/grasp-01-implementation.md) - Patterns and lessons learned
|
||||
@@ -150,14 +150,17 @@ rm -rf /tmp/test-repo
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_unauthorized_push() {
|
||||
let state = create_test_state().await;
|
||||
let relay = TestRelay::start().await;
|
||||
let result = validate_push(&state, "refs/heads/main", alice_pubkey).await;
|
||||
assert!(result.is_err());
|
||||
relay.stop().await;
|
||||
}
|
||||
```
|
||||
|
||||
**Result:** Pure Rust unit tests, no shell scripts, no Git setup.
|
||||
|
||||
See [`tests/push_authorization.rs`](tests/push_authorization.rs) for actual test examples.
|
||||
|
||||
### 4. Shared State and Types
|
||||
|
||||
**With hooks:**
|
||||
@@ -168,19 +171,17 @@ async fn test_unauthorized_push() {
|
||||
|
||||
**With inline authorization:**
|
||||
```rust
|
||||
pub struct GitHandler {
|
||||
nostr_relay: Arc<NostrRelay>, // Shared!
|
||||
state_cache: Arc<StateCache>, // Shared!
|
||||
}
|
||||
|
||||
impl GitHandler {
|
||||
async fn validate_push(&self, refs: &[RefUpdate]) -> Result<()> {
|
||||
// Direct access to Nostr state
|
||||
let state = self.state_cache.get_latest().await?;
|
||||
// Validate using shared types
|
||||
state.validate_refs(refs)?;
|
||||
Ok(())
|
||||
}
|
||||
// From src/git/handlers.rs
|
||||
pub async fn handle_receive_pack(
|
||||
repo_path: PathBuf,
|
||||
body: Bytes,
|
||||
database: SharedDatabase, // Shared with Nostr relay!
|
||||
npub: &str,
|
||||
identifier: &str,
|
||||
) -> Result<Response<Full<Bytes>>, GitError> {
|
||||
// Direct database access for authorization
|
||||
let auth = get_authorization_for_owner(&database, pubkey, identifier).await?;
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
@@ -208,9 +209,9 @@ Setup steps:
|
||||
**With inline authorization (ngit-grasp):**
|
||||
```
|
||||
Single Rust binary:
|
||||
- HTTP server (actix-web)
|
||||
- HTTP server (Hyper)
|
||||
- Git protocol handler
|
||||
- Nostr relay
|
||||
- Nostr relay (nostr-relay-builder)
|
||||
- Authorization logic
|
||||
|
||||
Setup steps:
|
||||
@@ -235,44 +236,38 @@ Content-Type: application/x-git-receive-pack-request
|
||||
0000000000000000000000000000000000000000 abc123... refs/heads/main\0 report-status
|
||||
```
|
||||
|
||||
We parse this **before** spawning Git:
|
||||
We parse this **before** spawning Git. See [`src/git/authorization.rs`](src/git/authorization.rs) for the implementation:
|
||||
|
||||
```rust
|
||||
pub async fn git_receive_pack(
|
||||
req: HttpRequest,
|
||||
body: web::Bytes,
|
||||
) -> Result<HttpResponse, Error> {
|
||||
// 1. Parse ref updates from request body
|
||||
let ref_updates = parse_ref_updates(&body)?;
|
||||
|
||||
// 2. Validate against Nostr state
|
||||
let state = get_latest_state(&repo).await?;
|
||||
validate_push(&state, &ref_updates).await?;
|
||||
|
||||
// 3. If valid, spawn git-receive-pack
|
||||
spawn_git_receive_pack(req, body).await
|
||||
/// Parse ref updates from git-receive-pack request body
|
||||
pub fn parse_pushed_refs(body: &[u8]) -> Result<Vec<PushedRef>, AuthorizationError> {
|
||||
// Parse pkt-line format
|
||||
// Extract ref updates
|
||||
// Return structured data
|
||||
}
|
||||
```
|
||||
|
||||
### How We Validate
|
||||
|
||||
Validation checks:
|
||||
Validation checks (from [`src/git/authorization.rs`](src/git/authorization.rs)):
|
||||
|
||||
1. Does pusher's pubkey have write access?
|
||||
2. Are they listed as a maintainer in the latest state event?
|
||||
3. Do maintainer sets form a valid chain?
|
||||
3. Do the refs match the state event?
|
||||
|
||||
```rust
|
||||
async fn validate_push(
|
||||
state: &RepoState,
|
||||
refs: &[RefUpdate],
|
||||
) -> Result<()> {
|
||||
for ref_update in refs {
|
||||
// Check if pusher is authorized for this ref
|
||||
if !state.is_authorized(&ref_update.name, pusher_pubkey) {
|
||||
return Err(Error::Unauthorized {
|
||||
ref_name: ref_update.name.clone(),
|
||||
pubkey: pusher_pubkey,
|
||||
});
|
||||
/// Validate that pushed refs match the authorized state
|
||||
pub fn validate_push_refs(
|
||||
pushed_refs: &[PushedRef],
|
||||
state: &RepositoryState,
|
||||
) -> Result<(), AuthorizationError> {
|
||||
for pushed_ref in pushed_refs {
|
||||
if pushed_ref.ref_name.starts_with("refs/heads/") {
|
||||
// Validate branch against state
|
||||
} else if pushed_ref.ref_name.starts_with("refs/tags/") {
|
||||
// Validate tag against state
|
||||
} else if pushed_ref.ref_name.starts_with("refs/nostr/") {
|
||||
// Allow refs/nostr/<event-id> for PRs
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
@@ -291,7 +286,7 @@ async fn validate_push(
|
||||
| **Performance** | Spawns Git first | Validates first |
|
||||
| **Testing** | Shell scripts + Go tests | Pure Rust tests |
|
||||
| **Deployment** | Docker + supervisord | Single binary |
|
||||
| **State sharing** | WebSocket query | Direct memory access |
|
||||
| **State sharing** | WebSocket query | Direct database access |
|
||||
|
||||
Both are GRASP-compliant, but inline authorization is simpler and more efficient.
|
||||
|
||||
@@ -314,34 +309,25 @@ Both are GRASP-compliant, but inline authorization is simpler and more efficient
|
||||
### Is It Worth It?
|
||||
|
||||
**Yes**, because:
|
||||
1. The `git-http-backend` crate handles protocol parsing
|
||||
1. We handle protocol parsing in [`src/git/protocol.rs`](src/git/protocol.rs)
|
||||
2. GRASP is already non-standard (Nostr authorization)
|
||||
3. Benefits far outweigh the coupling cost
|
||||
4. We can still add hook support later if needed
|
||||
|
||||
---
|
||||
|
||||
## Alternative Considered: Hybrid Approach
|
||||
## Implementation References
|
||||
|
||||
We could use **both** inline validation and hooks:
|
||||
Key files in the ngit-grasp implementation:
|
||||
|
||||
```rust
|
||||
// Inline: Fast path for common cases
|
||||
if !quick_validate(pusher).await? {
|
||||
return Err(Error::Unauthorized);
|
||||
}
|
||||
|
||||
// Hook: Detailed validation
|
||||
spawn_git_with_hook().await?;
|
||||
```
|
||||
|
||||
**Why we didn't choose this:**
|
||||
- Added complexity
|
||||
- Redundant validation
|
||||
- Slower (two validation steps)
|
||||
- Harder to maintain
|
||||
|
||||
If inline validation is sufficient, why add hooks?
|
||||
| Component | Location |
|
||||
|-----------|----------|
|
||||
| HTTP routing | [`src/http/mod.rs`](src/http/mod.rs) |
|
||||
| Git handlers | [`src/git/handlers.rs`](src/git/handlers.rs) |
|
||||
| Push authorization | [`src/git/authorization.rs`](src/git/authorization.rs) |
|
||||
| Git protocol parsing | [`src/git/protocol.rs`](src/git/protocol.rs) |
|
||||
| Subprocess management | [`src/git/subprocess.rs`](src/git/subprocess.rs) |
|
||||
| Event acceptance policy | [`src/nostr/builder.rs:51`](src/nostr/builder.rs:51) - `Nip34WritePolicy` |
|
||||
|
||||
---
|
||||
|
||||
@@ -365,9 +351,8 @@ This would allow:
|
||||
|
||||
### If Git Protocol Changes
|
||||
|
||||
The `git-http-backend` crate abstracts protocol details. If the Git protocol changes:
|
||||
- Update the crate dependency
|
||||
- Adjust our ref parsing if needed
|
||||
The protocol parsing is isolated in [`src/git/protocol.rs`](src/git/protocol.rs). If the Git protocol changes:
|
||||
- Update the protocol module
|
||||
- Tests will catch any breakage
|
||||
|
||||
---
|
||||
@@ -380,11 +365,11 @@ The `git-http-backend` crate abstracts protocol details. If the Git protocol cha
|
||||
2. It's more performant (early rejection)
|
||||
3. It's easier to test (pure Rust)
|
||||
4. It's simpler to deploy (single binary)
|
||||
5. It enables better integration (shared state)
|
||||
5. It enables better integration (shared database)
|
||||
|
||||
The trade-off (coupling to Git HTTP protocol) is acceptable because:
|
||||
- The protocol is stable and well-specified
|
||||
- The `git-http-backend` crate abstracts details
|
||||
- Protocol handling is isolated in one module
|
||||
- Benefits far outweigh the cost
|
||||
|
||||
This decision aligns with our goal of creating a **developer-friendly, production-ready GRASP implementation**.
|
||||
@@ -397,6 +382,7 @@ This decision aligns with our goal of creating a **developer-friendly, productio
|
||||
- [Design Decisions](decisions.md) - All architectural choices
|
||||
- [Comparison with ngit-relay](comparison.md) - Detailed comparison
|
||||
- [Git Protocol Reference](../reference/git-protocol.md) - Protocol details
|
||||
- [Test Strategy](../reference/test-strategy.md) - How we test this
|
||||
|
||||
---
|
||||
|
||||
|
||||
+70
-137
@@ -1,13 +1,13 @@
|
||||
# GRASP Audit Tool - Patterns and Learnings
|
||||
|
||||
**Purpose:** Document grasp-audit architecture, patterns, and lessons learned
|
||||
**Last Updated:** November 4, 2025
|
||||
**Last Updated:** December 4, 2025
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
`grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality.
|
||||
`grasp-audit` is a **fully implemented** compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality.
|
||||
|
||||
---
|
||||
|
||||
@@ -32,10 +32,10 @@
|
||||
|
||||
**Problem:** Test events pollute the relay and need cleanup without deletion events.
|
||||
|
||||
**Solution:** Use special tags to mark audit events:
|
||||
**Solution:** Use special tags to mark audit events (implemented in [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs)):
|
||||
|
||||
```rust
|
||||
// Every audit event includes these tags
|
||||
// Every audit event includes these tags (added automatically)
|
||||
[
|
||||
["t", "grasp-audit-test-event"], // Marker
|
||||
["t", "audit-{run-id}"], // Run isolation
|
||||
@@ -78,6 +78,8 @@
|
||||
|
||||
### Audit Configuration
|
||||
|
||||
From [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs):
|
||||
|
||||
```rust
|
||||
use grasp_audit::audit::AuditConfig;
|
||||
|
||||
@@ -101,100 +103,41 @@ let config = AuditConfig::shared();
|
||||
|
||||
### Creating Audit Events
|
||||
|
||||
```rust
|
||||
use grasp_audit::audit::{AuditConfig, AuditEventBuilder};
|
||||
use nostr_sdk::prelude::*;
|
||||
|
||||
let config = AuditConfig::isolated();
|
||||
let keys = Keys::generate();
|
||||
|
||||
// Create audit event
|
||||
let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content")
|
||||
.build(&keys)?;
|
||||
|
||||
// Event automatically includes:
|
||||
// - Audit marker tag
|
||||
// - Run ID tag
|
||||
// - Cleanup timestamp tag
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Querying Audit Events
|
||||
From [`grasp-audit/src/client.rs`](grasp-audit/src/client.rs):
|
||||
|
||||
```rust
|
||||
use grasp_audit::client::AuditClient;
|
||||
use grasp_audit::audit::AuditConfig;
|
||||
|
||||
let config = AuditConfig::isolated();
|
||||
let client = AuditClient::new(config, keys);
|
||||
let client = AuditClient::new("ws://localhost:8080", config).await?;
|
||||
|
||||
// Connect to relay
|
||||
client.add_relay("ws://localhost:7000").await?;
|
||||
client.connect().await;
|
||||
// Create and send an event - cleanup tags are added automatically
|
||||
let event = client.event_builder()
|
||||
.kind(Kind::TextNote)
|
||||
.content("test content")
|
||||
.build(&keys)?;
|
||||
|
||||
// Query audit events for this run
|
||||
let events = client.query().await?;
|
||||
|
||||
// Events are filtered by:
|
||||
// - "grasp-audit-test-event" marker
|
||||
// - Current run ID
|
||||
client.send_event(event).await?;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Test Isolation
|
||||
### Test Suites
|
||||
|
||||
**Each test run is isolated by unique run ID:**
|
||||
|
||||
```rust
|
||||
// CI mode generates unique UUID per run
|
||||
let config1 = AuditConfig::isolated();
|
||||
let config2 = AuditConfig::isolated();
|
||||
|
||||
// config1.run_id != config2.run_id
|
||||
// Tests won't interfere with each other
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
|
||||
- ✅ Parallel CI/CD runs don't conflict
|
||||
- ✅ Can run multiple test suites simultaneously
|
||||
- ✅ Easy to identify which run created which events
|
||||
- ✅ Cleanup can target specific runs
|
||||
|
||||
---
|
||||
|
||||
### Cleanup Strategy
|
||||
|
||||
**Two-phase cleanup:**
|
||||
|
||||
1. **Automatic expiry** via cleanup timestamp tag
|
||||
2. **Manual cleanup** by querying and deleting
|
||||
|
||||
```rust
|
||||
// Events include cleanup timestamp
|
||||
["t", "audit-cleanup-after-1730707200"]
|
||||
|
||||
// Cleanup process:
|
||||
// 1. Query events with expired cleanup timestamp
|
||||
// 2. Delete from database directly (no KIND 5)
|
||||
// 3. Avoid deletion event pollution
|
||||
```
|
||||
|
||||
**Implementation:** To be built in relay (not in audit tool)
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Test Organization
|
||||
From [`grasp-audit/src/specs/grasp01/mod.rs`](grasp-audit/src/specs/grasp01/mod.rs):
|
||||
|
||||
```
|
||||
grasp-audit/src/specs/
|
||||
grasp-audit/src/specs/grasp01/
|
||||
├── mod.rs # Module exports
|
||||
├── nip01_smoke.rs # NIP-01 basic functionality
|
||||
├── grasp_01_relay.rs # GRASP-01 relay requirements (planned)
|
||||
└── mod.rs # Test suite registry
|
||||
├── nip11_document.rs # NIP-11 document tests
|
||||
├── event_acceptance_policy.rs # GRASP-01 event rules
|
||||
├── cors.rs # CORS header tests
|
||||
├── git_clone.rs # Git clone operations
|
||||
├── push_authorization.rs # Push validation tests
|
||||
├── repository_creation.rs # Repository lifecycle
|
||||
└── spec_requirements.rs # Requirement definitions
|
||||
```
|
||||
|
||||
### Unit vs Integration Tests
|
||||
@@ -229,17 +172,18 @@ mod tests {
|
||||
|
||||
```bash
|
||||
# Unit tests (fast, no dependencies)
|
||||
cargo test --lib
|
||||
cd grasp-audit && nix develop -c cargo test --lib
|
||||
|
||||
# Integration tests (requires relay)
|
||||
docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
|
||||
cargo test -- --ignored
|
||||
# Integration tests (requires relay via test-ngit-relay.sh)
|
||||
cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Test Result Reporting
|
||||
|
||||
From [`grasp-audit/src/result.rs`](grasp-audit/src/result.rs):
|
||||
|
||||
```rust
|
||||
use grasp_audit::result::AuditResult;
|
||||
|
||||
@@ -255,7 +199,7 @@ for result in &results {
|
||||
}
|
||||
|
||||
// Summary
|
||||
let passed = results.iter().filter(|r| r.is_pass()).count();
|
||||
let passed = results.iter().filter(|r| r.passed).count();
|
||||
let total = results.len();
|
||||
println!("Results: {}/{} passed ({:.1}%)",
|
||||
passed, total, (passed as f64 / total as f64) * 100.0);
|
||||
@@ -291,7 +235,7 @@ grasp-audit audit \
|
||||
grasp-audit audit \
|
||||
--relay wss://relay.example.com \
|
||||
--mode production \
|
||||
--run-id "audit-2025-11-04" \
|
||||
--run-id "audit-2025-12-04" \
|
||||
--verbose
|
||||
|
||||
# Test all specs
|
||||
@@ -366,25 +310,23 @@ let events = client.query().await?;
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
## What's Implemented
|
||||
|
||||
### Planned Features
|
||||
### Completed Features
|
||||
|
||||
- [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests
|
||||
- [ ] **Test Report Generation**: JSON/HTML output for CI/CD
|
||||
- ✅ **GRASP-01 Test Suites**: All NIP-01, NIP-11, CORS, event acceptance tests
|
||||
- ✅ **Spec Requirements Database**: Machine-readable requirements in [`spec_requirements.rs`](grasp-audit/src/specs/grasp01/spec_requirements.rs)
|
||||
- ✅ **Automatic Cleanup Tags**: Production-safe event tagging
|
||||
- ✅ **Test Isolation**: UUID run IDs for parallel execution
|
||||
- ✅ **AuditClient**: Nostr client wrapper with audit features
|
||||
- ✅ **Fixture Helpers**: Event creation helpers in [`fixtures.rs`](grasp-audit/src/fixtures.rs)
|
||||
|
||||
### Future Enhancements
|
||||
|
||||
- [ ] **GRASP-02 Test Suite**: Proactive sync tests
|
||||
- [ ] **HTML Report Generation**: Rich CI/CD reports
|
||||
- [ ] **Performance Benchmarks**: Measure relay performance
|
||||
- [ ] **Relay Comparison**: Side-by-side compliance comparison
|
||||
- [ ] **Continuous Monitoring**: Periodic production audits
|
||||
|
||||
---
|
||||
|
||||
### Possible Improvements
|
||||
|
||||
- [ ] **Parallel Test Execution**: Run specs in parallel
|
||||
- [ ] **Retry Logic**: Handle transient failures
|
||||
- [ ] **Custom Assertions**: Domain-specific test helpers
|
||||
- [ ] **Event Diff Tool**: Compare expected vs actual events
|
||||
- [ ] **Cleanup Automation**: Auto-cleanup after tests
|
||||
|
||||
---
|
||||
|
||||
@@ -403,14 +345,12 @@ let events = client.query().await?;
|
||||
**Solution:**
|
||||
|
||||
```bash
|
||||
# Start relay
|
||||
docker run --rm -p 7000:7000 scsibug/nostr-rs-relay
|
||||
# Use test-ngit-relay.sh for automated relay management
|
||||
cd grasp-audit && nix develop -c bash test-ngit-relay.sh --mode test
|
||||
|
||||
# Verify relay is running
|
||||
curl http://localhost:7000
|
||||
|
||||
# Run tests
|
||||
cargo test -- --ignored
|
||||
# Or manually:
|
||||
docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest
|
||||
RELAY_URL="ws://localhost:18081" nix develop -c cargo test --lib -- --ignored
|
||||
```
|
||||
|
||||
---
|
||||
@@ -471,46 +411,39 @@ let config = AuditConfig::isolated();
|
||||
let config = AuditConfig::shared();
|
||||
```
|
||||
|
||||
### Event Creation
|
||||
|
||||
```rust
|
||||
let event = AuditEventBuilder::new(&config, kind, content)
|
||||
.build(&keys)?;
|
||||
```
|
||||
|
||||
### Client Usage
|
||||
|
||||
```rust
|
||||
let client = AuditClient::new(config, keys);
|
||||
client.add_relay("ws://localhost:7000").await?;
|
||||
client.connect().await;
|
||||
let events = client.query().await?;
|
||||
let client = AuditClient::new("ws://localhost:7000", config).await?;
|
||||
assert!(client.is_connected().await);
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Unit tests
|
||||
cargo test --lib
|
||||
# Unit tests (from grasp-audit/)
|
||||
nix develop -c cargo test --lib
|
||||
|
||||
# Integration tests
|
||||
cargo test -- --ignored
|
||||
# Integration tests with ngit-relay
|
||||
nix develop -c bash test-ngit-relay.sh --mode test
|
||||
|
||||
# CLI
|
||||
cargo run -- audit --relay ws://localhost:7000
|
||||
# CLI audit
|
||||
nix develop -c cargo run -- audit --relay ws://localhost:7000
|
||||
```
|
||||
|
||||
---
|
||||
### Key Files
|
||||
|
||||
## References
|
||||
|
||||
- **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp
|
||||
- **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md
|
||||
- **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md
|
||||
- **grasp-audit README**: `grasp-audit/README.md`
|
||||
- **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md`
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [`grasp-audit/src/lib.rs`](grasp-audit/src/lib.rs) | Public API |
|
||||
| [`grasp-audit/src/client.rs`](grasp-audit/src/client.rs) | AuditClient implementation |
|
||||
| [`grasp-audit/src/audit.rs`](grasp-audit/src/audit.rs) | AuditConfig, cleanup tags |
|
||||
| [`grasp-audit/src/specs/grasp01/mod.rs`](grasp-audit/src/specs/grasp01/mod.rs) | Test suite registry |
|
||||
| [`grasp-audit/src/specs/grasp01/spec_requirements.rs`](grasp-audit/src/specs/grasp01/spec_requirements.rs) | Requirement database |
|
||||
|
||||
---
|
||||
|
||||
_Last updated: November 4, 2025_
|
||||
_Status: Living document - update as grasp-audit evolves_
|
||||
## Related Documentation
|
||||
|
||||
- [Test Strategy](../reference/test-strategy.md) - Overall testing approach
|
||||
- [GRASP-01 Implementation](grasp-01-implementation.md) - Main project learnings
|
||||
+251
-1176
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user