✅ Shared Fixtures: Fixtures cached and reused across tests (default for CLI)
✅ Isolated Testing: Fresh fixtures per test for parallel test isolation
✅ Clean Audit Events: Special tags for easy cleanup (no deletion trails)
✅ Spec-Mirrored Tests: Test structure matches GRASP protocol exactly
✅ Reusable: Can test any GRASP implementation (Rust, Go, Python, etc.)

Quick Start

The fastest way to run GRASP-01 compliance tests:

# Run the test suite against ngit-relay
cd grasp-audit
nix develop -c bash test-ngit-relay.sh --mode test

This automatically:

✅ Starts ngit-relay in an isolated Docker container
✅ Runs all GRASP-01 compliance tests
✅ Cleans up resources when finished

For more options:

./test-ngit-relay.sh --help

Usage Examples

As a Library

use grasp_audit::*;

#[tokio::main]
async fn main() -> Result<()> {
    // Create audit client with shared fixtures (default for CLI)
    let config = AuditConfig::shared();
    let client = AuditClient::new("ws://localhost:7000", config).await?;

    // Run NIP-01 smoke tests
    let results = specs::Nip01SmokeTests::run_all(&client).await;
    results.print_report();

    if !results.all_passed() {
        std::process::exit(1);
    }

    Ok(())
}

As a CLI Tool

# Install
cargo install --path .

# Run smoke tests against local relay (shared fixtures - default)
grasp-audit audit --relay ws://localhost:7000 --spec nip01-smoke

# Run with isolated fixtures (each test gets fresh fixtures)
grasp-audit audit --relay ws://localhost:7000 --mode isolated --spec all

Test Specifications

NIP-01 Smoke Tests (6 tests)

Basic Nostr relay functionality:

websocket_connection - Can connect to /
send_receive_event - Can send EVENT, get OK
create_subscription - Can subscribe with REQ
close_subscription - Can close subscriptions
reject_invalid_signature - Rejects bad signatures
reject_invalid_event_id - Rejects wrong IDs

Why only smoke tests? rust-nostr already has 1000+ tests for NIP-01 compliance. We focus on GRASP-specific behavior.

GRASP-01 Tests (Coming Soon)

Repository announcement acceptance
State event handling
Policy enforcement
And more...

Audit Event Strategy

All audit events automatically include special tags for isolation and cleanup:

{
  "tags": [
    ["t", "grasp-audit-test-event"],
    ["t", "audit-ci-a1b2c3d4-e5f6-7890-abcd-ef1234567890"],
    ["t", "audit-cleanup-after-1730822334"]
  ]
}

Tag Format:

["t", "grasp-audit-test-event"] - Identifies all audit-related events
["t", "audit-{run_id}"] - Unique identifier for each audit run
- Shared mode: audit-audit-{uuid}
- Isolated mode: audit-isolated-{uuid}
["t", "audit-cleanup-after-{unix_timestamp}"] - Cleanup scheduling
- Default: Current time + 3600 seconds (1 hour)

Benefits:

Automatic: Tags added automatically to all events via AuditEventBuilder
Isolation: Each test run has unique ID for event filtering
Cleanup: Events marked for cleanup after timestamp (direct database cleanup)
No deletion trails: No NIP-09 deletion events needed
Discovery: Easy to query all audit events via hashtag

Fixture Modes

The audit tool supports two fixture caching modes that control how test prerequisites are managed.

Shared Mode (Default for CLI)

Fixtures are cached and reused across all tests. Use this when:

Running the CLI audit tool sequentially
Tests build on each other's fixtures
You want efficient resource usage

let config = AuditConfig::shared();

Isolated Mode (For Parallel Tests)

Each test creates fresh fixtures for complete isolation. Use this when:

Running cargo test in parallel
Tests must not interfere with each other
Testing the fixture system itself

let config = AuditConfig::isolated();

CLI Usage

# Default: shared fixtures (efficient for sequential CLI runs)
grasp-audit audit --relay ws://localhost:7000 --spec all

# Isolated fixtures (for testing)
grasp-audit audit --relay ws://localhost:7000 --mode isolated --spec all

Examples

See examples/ directory:

# Simple audit example
cargo run --example simple_audit

Testing

TL;DR: See the Quick Start section for the fastest way to run tests.

Unit Tests

# Enter dev environment (NixOS)
nix develop

# Run unit tests (no relay required)
cargo test

Integration Tests

The recommended approach is test-ngit-relay.sh, which handles all relay lifecycle management automatically.

See the Quick Start section for common usage patterns.

Advanced: Manual Relay Setup

Click to expand manual testing instructions

For advanced use cases where you need direct control over the relay:

# Start relay on a specific port (example uses 18081)
docker run --rm -p 18081:8081 ghcr.io/danconwaydev/ngit-relay:latest

# In another terminal, run tests with RELAY_URL
grasp-audit audit --relay ws://localhost:18081 --mode ci

# or run all ignored tests via cargo
RELAY_URL="ws://localhost:18081" cargo test --lib -- --ignored --nocapture

# or run specific test via cargo
RELAY_URL="ws://localhost:18081" cargo test --lib test_grasp01_nostr_relay_against_relay -- --ignored --nocapture

Test Design Pattern: Fixture-First

To prevent rate-limiting from production relays during testing, we use a fixture-first approach that minimizes relay interactions.

Quick Start for New Tests

Create TestContext at test start
Get prerequisites via ctx.get_fixture(FixtureKind::...)
Build test-specific events using fixtures as base
Verify outcomes via send_and_verify_accepted/rejected

Pattern Template

pub async fn test_something(client: &AuditClient) -> TestResult {
    TestResult::new(...)
        .run(|| async {
            // 1. Context
            let ctx = TestContext::new(client);

            // 2. Prerequisites (cached per-TestContext)
            let repo = ctx.get_fixture(FixtureKind::ValidRepo).await?;

            // 3. Test-specific event
            let my_event = client.create_issue(&repo, "Title", "Content", vec![])?;

            // 4. Verify
            send_and_verify_accepted(client, my_event, "description").await?;

            Ok(())
        })
        .await
}

Three-Layer Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Layer 3: Test Functions                       │
│  Create TestContext, get fixtures, build scenarios, verify       │
├─────────────────────────────────────────────────────────────────┤
│           Layer 2: FixtureKind + TestContext                     │
│  ValidRepo, RepoState, OwnerStateDataPushed, etc.                │
│  Mode-aware caching within TestContext                           │
├─────────────────────────────────────────────────────────────────┤
│               Layer 1: AuditClient                               │
│  event_builder, create_repo_announcement, send_event             │
└─────────────────────────────────────────────────────────────────┘

Available Fixtures

FixtureKind	Provides	Use When
`ValidRepo`	Accepted repo announcement (kind 30617). Signed by owner keys, lists maintainer in maintainers tag.	Need a repo as prerequisite
`RepoWithIssue`	Repo + accepted issue (kind 1621)	Testing issue-dependent events
`RepoWithComment`	Repo + issue + comment (kind 1111)	Testing comment-dependent events
`RepoState`	Repo + state event (kind 30618). Signed by owner, points to `DETERMINISTIC_COMMIT_HASH`.	Testing owner state events
`PREvent`	Repo + PR event (kind 1618). Signed by PR author, points to `PR_TEST_COMMIT_HASH`.	Testing PR-dependent events
`PREventGenerated`	PR event built but NOT sent to relay.	Need PR event ID before publishing
`PRWrongCommitPushedBeforeEvent`	Wrong commit pushed to `refs/nostr/<pr-event-id>` before PR event sent. Returns unsent PR event.	Testing pre-event ref cleanup
`PREventSentAfterWrongPush`	PR event sent after wrong commit was pushed. Tests cleanup behavior.	Testing post-event ref cleanup
`OwnerStateDataPushed`	Full owner push flow: state event + git data pushed. Points to `DETERMINISTIC_COMMIT_HASH`.	Testing owner push authorization
`MaintainerStateDataPushed`	Full maintainer push flow: force-pushes over owner's data. Points to `MAINTAINER_DETERMINISTIC_COMMIT_HASH`.	Testing maintainer push authorization
`RecursiveMaintainerStateDataPushed`	Full recursive maintainer push flow: Owner → Maintainer → RecursiveMaintainer chain. Points to `RECURSIVE_MAINTAINER_DETERMINISTIC_COMMIT_HASH`.	Testing recursive maintainer authorization
`HeadSetToDevelopBranch`	State event with HEAD=refs/heads/develop. Depends on RecursiveMaintainerStateDataPushed.	Testing HEAD branch switching

Deterministic Commit Hashes

Fixtures use deterministic commit hashes for reproducible testing:

Constant	Hash	Used By
`DETERMINISTIC_COMMIT_HASH`	`64ea71d79a57a7acb334cd9651f8aec067c0ce5d`	Owner fixtures (RepoState, OwnerStateDataPushed)
`MAINTAINER_DETERMINISTIC_COMMIT_HASH`	`1c2d472c9b71ed51968a66500281a3c4a6840464`	MaintainerStateDataPushed
`RECURSIVE_MAINTAINER_DETERMINISTIC_COMMIT_HASH`	`05939b82de66fbdb9c077d0a64fc68522f3cb8e0`	RecursiveMaintainerStateDataPushed
`PR_TEST_COMMIT_HASH`	`5d40fb1555a0c28bf4d650515a73aaa54d4d9bfb`	PR fixtures (PREvent, PREventGenerated)

Fixture Dependencies

Fixtures automatically resolve their dependencies:

ValidRepo (base)
├── RepoWithIssue → RepoWithComment
├── RepoState
├── PREventGenerated → PRWrongCommitPushedBeforeEvent → PREventSentAfterWrongPush
├── PREvent
└── OwnerStateDataPushed
    └── MaintainerStateDataPushed
        └── RecursiveMaintainerStateDataPushed
            └── HeadSetToDevelopBranch

Fixture Lifecycle: Generate → Send → Verify → DataPushed

Every fixture follows a lifecycle (some stop earlier):

GENERATE: Build event via AuditClient.event_builder() (in memory only)
SEND: client.send_event(event) transmits to relay (rate-limited operation)
VERIFY: Query relay to confirm acceptance/rejection
DATA_PUSHED: (DataPushed variants only) Clone repo, create commit, push to git server

Caching happens after the fixture completes - same fixture request returns cached Event.

Note: Some fixtures handle their own event sending (e.g., OwnerStateDataPushed, MaintainerStateDataPushed). These are marked with sends_own_events() -> true.

How TestContext Correlates Events

Each TestContext shares a run_id with all events:

// All events in a TestContext get these tags automatically:
["t", "grasp-audit-test-event"]     // Identifies test events
["t", "audit-{run_id}"]             // Unique ID for this run
["t", "audit-cleanup-after-{ts}"]   // Cleanup timestamp

This enables:

Event correlation within a test run
Production relay cleanup scripts
Test isolation between runs

When NOT to Use Fixtures

Use direct event building (NOT fixtures) when:

Testing event REJECTION - Build invalid events directly
Testing signature/ID validation - Need malformed events
One-off connectivity tests - No prerequisites needed

// Example: Testing rejection (build invalid event directly)
let invalid_event = client.event_builder(Kind::GitRepoAnnouncement, "")
    .tag(Tag::identifier("test"))
    // Missing required 'clone' tag - should be rejected
    .build(client.keys())?;

send_and_verify_rejected(client, invalid_event, "missing clone tag").await?;

Anti-Patterns to Avoid

❌ Creating TestContext inside helper functions - Tests lose cache control

❌ Monolithic setup functions - Mix fixture retrieval with git operations

❌ Direct event creation when fixture exists - Misses caching opportunity

✅ Each test creates own TestContext - Isolation guaranteed

✅ Use fixtures for prerequisites - Caching minimizes relay calls

✅ Build invalid events directly - Only for rejection tests

Architecture

grasp-audit/
├── src/
│   ├── lib.rs              # Public API
│   ├── audit.rs            # Audit config and event tagging
│   ├── client.rs           # Audit client
│   ├── fixtures.rs         # TestContext and FixtureKind
│   ├── result.rs           # Test result types
│   ├── isolation.rs        # Test isolation utilities
│   └── specs/
│       ├── mod.rs
│       └── nip01_smoke.rs  # NIP-01 smoke tests
├── examples/
│   └── simple_audit.rs     # Example usage
└── bin/
    └── grasp-audit.rs      # CLI tool

Development Status

✅ Audit framework
✅ NIP-01 smoke tests (6 tests)
🚧 GRASP-01 relay tests (planned)
🚧 GRASP-01 git tests (planned)
🚧 Cleanup utilities (planned)

Contributing

This tool is designed to be reusable by any GRASP implementation. Contributions welcome!

License

MIT