diff --git a/docs/how-to/README.md b/docs/how-to/README.md index ba58c08..f755be1 100644 --- a/docs/how-to/README.md +++ b/docs/how-to/README.md @@ -110,15 +110,15 @@ How-to guides are **recipes** that show you how to solve specific problems or ac --- -### Migrate from ngit-relay -**Status:** 🔜 Planned +### [Migrate to ngit-grasp](migrate-to-ngit-grasp.md) +**Status:** ✅ Available -**Problem:** Switch from reference implementation +**Problem:** Switch from another GRASP implementation **You'll learn:** -- Export data from ngit-relay -- Import to ngit-grasp -- Update repository URLs -- Verify migration +- Analyze existing relay data +- Identify repositories needing attention +- Run migration analysis scripts +- Plan and execute cutover --- diff --git a/docs/how-to/migrate-ngit-relay-to-ngit-grasp.md b/docs/how-to/migrate-to-ngit-grasp.md similarity index 82% rename from docs/how-to/migrate-ngit-relay-to-ngit-grasp.md rename to docs/how-to/migrate-to-ngit-grasp.md index 975eb4c..f4dff86 100644 --- a/docs/how-to/migrate-ngit-relay-to-ngit-grasp.md +++ b/docs/how-to/migrate-to-ngit-grasp.md @@ -1,6 +1,21 @@ -# Migrate ngit-relay to ngit-grasp +# Migrate to ngit-grasp from another GRASP implementation -This guide walks you through migrating a production ngit-relay instance to ngit-grasp. The process involves analyzing your existing data to identify repositories that need attention before switching over. +This guide walks you through migrating a production GRASP relay to ngit-grasp. The process involves analyzing your existing data to identify repositories that need attention before switching over. + +## Compatibility + +This migration process works with any GRASP implementation that: + +- Stores git data in the `/.git` directory structure +- Uses standard GRASP events (kind 30617 announcements, kind 30618 state, kind 5 deletions) +- Exposes a Nostr relay WebSocket endpoint + +**Known compatible implementations:** +- ngit-relay (reference implementation) +- ngit-grasp (when migrating between instances or from archive mode) +- Other GRASP-compliant relays following the specification + +The migration scripts analyze Nostr events and git data directly, making them implementation-agnostic. ## Quick Start @@ -9,15 +24,15 @@ Run the migration analysis with a single command: ```bash # Basic analysis (fetches events, compares relays) ./docs/how-to/migration-scripts/run-migration-analysis.sh \ - --prod-relay wss://relay.ngit.dev \ - --archive-relay wss://archive.relay.ngit.dev + --prod-relay wss://source-relay.example.com \ + --archive-relay wss://target-relay.example.com # Full analysis (includes git sync check - run on VPS) ./docs/how-to/migration-scripts/run-migration-analysis.sh \ - --prod-relay wss://relay.ngit.dev \ - --archive-relay wss://archive.relay.ngit.dev \ - --prod-git /var/lib/ngit-relay/git \ - --archive-git /var/lib/ngit-relay-archive/git \ + --prod-relay wss://source-relay.example.com \ + --archive-relay wss://target-relay.example.com \ + --prod-git /var/lib/grasp-relay/git \ + --archive-git /var/lib/ngit-grasp/git \ --service ngit-grasp.service ``` @@ -37,7 +52,7 @@ See [Running the Analysis](#running-the-analysis) for detailed options. ### For Full Analysis (VPS) -- SSH access to the VPS running ngit-relay +- SSH access to the VPS running your source relay - Read access to git data directories - Access to systemd journal (for log extraction) @@ -58,7 +73,7 @@ The migration process has three stages: ### Stage 1: Deploy Archive Instance -Deploy ngit-grasp alongside your production ngit-relay: +Deploy ngit-grasp alongside your production relay: 1. Configure ngit-grasp with: - `domain` set to `.internal` (temporary) @@ -88,24 +103,24 @@ Once all issues are resolved: ```bash # Preview what will happen (dry run) ./run-migration-analysis.sh \ - --prod-relay wss://relay.ngit.dev \ - --archive-relay wss://archive.relay.ngit.dev \ + --prod-relay wss://source-relay.example.com \ + --archive-relay wss://target-relay.example.com \ --dry-run # Run the analysis ./run-migration-analysis.sh \ - --prod-relay wss://relay.ngit.dev \ - --archive-relay wss://archive.relay.ngit.dev + --prod-relay wss://source-relay.example.com \ + --archive-relay wss://target-relay.example.com ``` ### Full Analysis on VPS ```bash ./run-migration-analysis.sh \ - --prod-relay wss://relay.ngit.dev \ - --archive-relay wss://archive.relay.ngit.dev \ - --prod-git /var/lib/ngit-relay/git \ - --archive-git /var/lib/ngit-relay-archive/git \ + --prod-relay wss://source-relay.example.com \ + --archive-relay wss://target-relay.example.com \ + --prod-git /var/lib/grasp-relay/git \ + --archive-git /var/lib/ngit-grasp/git \ --service ngit-grasp.service ``` @@ -128,8 +143,8 @@ Skip or run specific phases: | Option | Description | |--------|-------------| -| `--prod-relay ` | Production relay WebSocket URL (required) | -| `--archive-relay ` | Archive relay WebSocket URL (required) | +| `--prod-relay ` | Source relay WebSocket URL (required) | +| `--archive-relay ` | Target relay WebSocket URL (required) | | `--prod-git ` | Git base directory for prod (enables Phase 2) | | `--archive-git ` | Git base directory for archive (enables Phase 2) | | `--service ` | Systemd service name (enables Phase 4) | @@ -217,7 +232,7 @@ Run with sudo or ensure your user has read access: ```bash # Check permissions -ls -la /var/lib/ngit-relay/git +ls -la /var/lib/grasp-relay/git # Run with sudo if needed sudo ./run-migration-analysis.sh ... @@ -332,16 +347,18 @@ work/migration-analysis-YYYYMMDD-HHMM/ └── summary.txt # Phase 5 ``` -## Key Differences: ngit-relay vs ngit-grasp +## Why Migration May Require Attention -Understanding these differences helps explain why some repositories need attention: +Different GRASP implementations may handle edge cases differently. ngit-grasp has stricter validation and better observability, which can surface issues that were previously hidden: -| Aspect | ngit-relay | ngit-grasp | -|--------|------------|------------| -| Git data validation | Accepts commits/tags referenced in state event | Requires all git data to reproduce state | -| PR refs cleanup | Doesn't clear `refs/nostr/` | Properly manages PR refs | -| Parse failures | Silently ignores | Logs structured `[PARSE_FAIL]` entries | -| Sync timeout | No timeout | Purgatory expires after configurable period | +| Aspect | Typical Source Relay | ngit-grasp | +|--------|---------------------|------------| +| Git data validation | May accept partial data | Requires all git data to reproduce state | +| PR refs cleanup | May not clear `refs/nostr/` | Properly manages PR refs | +| Parse failures | May silently ignore | Logs structured `[PARSE_FAIL]` entries | +| Sync timeout | May have no timeout | Purgatory expires after configurable period | + +These differences explain why some repositories may need attention during migration - ngit-grasp's stricter validation catches issues that other implementations may have silently accepted. ## Next Steps @@ -371,10 +388,10 @@ For advanced usage, you can run individual phase scripts: ```bash # Phase 1: Fetch events -./migration-scripts/01-fetch-events.sh wss://relay.ngit.dev output/prod +./migration-scripts/01-fetch-events.sh wss://source-relay.example.com output/prod # Phase 2: Git sync check -./migration-scripts/10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/ngit-relay/git output/prod --categorize +./migration-scripts/10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/grasp-relay/git output/prod --categorize # Phase 3a: Categorize ./migration-scripts/20-categorize.sh output/prod/git-sync-status.tsv output/prod diff --git a/docs/how-to/migration-scripts/01-fetch-events.sh b/docs/how-to/migration-scripts/01-fetch-events.sh index 6870659..e0d6f26 100755 --- a/docs/how-to/migration-scripts/01-fetch-events.sh +++ b/docs/how-to/migration-scripts/01-fetch-events.sh @@ -2,7 +2,7 @@ # # 01-fetch-events.sh - Fetch nostr events from a relay for migration analysis # -# PHASE 1 of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 1 of the GRASP relay to ngit-grasp migration analysis pipeline. # Fetches kind 30618 (state), 30617 (announcement), and 5 (deletion) events. # # USAGE: @@ -41,7 +41,7 @@ # - Run Phase 1 and Phase 2 back-to-back for accurate snapshot # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # set -euo pipefail diff --git a/docs/how-to/migration-scripts/10-check-git-sync.sh b/docs/how-to/migration-scripts/10-check-git-sync.sh index 493d50a..1f2ce04 100755 --- a/docs/how-to/migration-scripts/10-check-git-sync.sh +++ b/docs/how-to/migration-scripts/10-check-git-sync.sh @@ -2,21 +2,21 @@ # # 10-check-git-sync.sh - Compare state events to actual git data on disk # -# PHASE 2 of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 2 of the GRASP relay to ngit-grasp migration analysis pipeline. # Compares kind 30618 state events against actual git refs on disk. # # USAGE: # ./10-check-git-sync.sh [--categorize] # # EXAMPLES: -# # Check prod relay against prod git data -# ./10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/ngit-relay/git output/prod +# # Check source relay against source git data +# ./10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/grasp-relay/git output/prod # -# # Check archive relay against archive git data -# ./10-check-git-sync.sh output/archive/raw/state-events.json /var/lib/ngit-relay-archive/git output/archive +# # Check target relay against target git data +# ./10-check-git-sync.sh output/archive/raw/state-events.json /var/lib/ngit-grasp/git output/archive # # # Check and categorize in one step (convenience mode) -# ./10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/ngit-relay/git output/prod --categorize +# ./10-check-git-sync.sh output/prod/raw/state-events.json /var/lib/grasp-relay/git output/prod --categorize # # INPUT: # state-events.json - JSONL file from Phase 1 (01-fetch-events.sh) @@ -53,7 +53,7 @@ # - Handles packed refs (git show-ref) and loose refs # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # 01-fetch-events.sh - Phase 1 script that produces input for this script # 20-categorize.sh - Phase 3a script that consumes output from this script # @@ -101,13 +101,13 @@ usage() { echo "" echo "Arguments:" echo " state-events.json JSONL file from Phase 1 (kind 30618 events)" - echo " git-base-dir Base directory for git repos (e.g., /var/lib/ngit-relay/git)" + echo " git-base-dir Base directory for git repos (e.g., /var/lib/grasp-relay/git)" echo " output-dir Directory to store output files" echo " --categorize Optional: also output category files (like Phase 3)" echo "" echo "Examples:" - echo " $0 output/prod/raw/state-events.json /var/lib/ngit-relay/git output/prod" - echo " $0 output/archive/raw/state-events.json /var/lib/ngit-relay-archive/git output/archive" + echo " $0 output/prod/raw/state-events.json /var/lib/grasp-relay/git output/prod" + echo " $0 output/archive/raw/state-events.json /var/lib/ngit-grasp/git output/archive" echo "" echo "Output:" echo " git-sync-status.tsv - TSV with: repo, npub, state_refs, git_refs, matches, reason" diff --git a/docs/how-to/migration-scripts/20-categorize.sh b/docs/how-to/migration-scripts/20-categorize.sh index f47eb55..b38dc00 100755 --- a/docs/how-to/migration-scripts/20-categorize.sh +++ b/docs/how-to/migration-scripts/20-categorize.sh @@ -2,7 +2,7 @@ # # 20-categorize.sh - Categorize git sync status into 4 categories # -# PHASE 3a of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 3a of the GRASP relay to ngit-grasp migration analysis pipeline. # Takes git-sync-status.tsv from Phase 2 and categorizes into 4 files. # # USAGE: @@ -39,7 +39,7 @@ # RUNTIME: < 1 second (local processing only) # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # 10-check-git-sync.sh - Phase 2 script that produces input for this script # diff --git a/docs/how-to/migration-scripts/21-compare-relays.sh b/docs/how-to/migration-scripts/21-compare-relays.sh index 6b40dc8..b9c0d30 100755 --- a/docs/how-to/migration-scripts/21-compare-relays.sh +++ b/docs/how-to/migration-scripts/21-compare-relays.sh @@ -2,7 +2,7 @@ # # 21-compare-relays.sh - Compare prod vs archive category files to find gaps # -# PHASE 3b of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 3b of the GRASP relay to ngit-grasp migration analysis pipeline. # Compares categorized output from prod and archive to identify: # - Repos complete in prod but missing/incomplete in archive # - Repos in archive but not in prod @@ -39,7 +39,7 @@ # RUNTIME: < 1 second (local processing only) # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # 20-categorize.sh - Phase 3a script that produces input for this script # diff --git a/docs/how-to/migration-scripts/30-extract-parse-failures.sh b/docs/how-to/migration-scripts/30-extract-parse-failures.sh index 753fd3e..bc2049a 100755 --- a/docs/how-to/migration-scripts/30-extract-parse-failures.sh +++ b/docs/how-to/migration-scripts/30-extract-parse-failures.sh @@ -2,7 +2,7 @@ # # 30-extract-parse-failures.sh - Extract parse failure events from systemd logs # -# PHASE 4a of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 4a of the GRASP relay to ngit-grasp migration analysis pipeline. # Extracts structured [PARSE_FAIL] log entries from journalctl. # # USAGE: @@ -42,7 +42,7 @@ # [PARSE_FAIL] log entries. Until those are implemented, this script will # find no matching entries (which is handled gracefully). # -# See: docs/how-to/migrate-ngit-relay-to-ngit-grasp.md (Dependencies section) +# See: docs/how-to/migrate-to-ngit-grasp.md (Dependencies section) # # Expected Rust logging code: # tracing::warn!( @@ -59,7 +59,7 @@ # RUNTIME: Depends on log volume, typically < 30 seconds # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # 31-extract-purgatory-expiry.sh - Companion script for purgatory expiry logs # @@ -231,7 +231,7 @@ main() { if [[ "$sample_count" -eq 0 ]]; then log_warn "No [PARSE_FAIL] entries found in logs." log_warn "This is expected if ngit-grasp logging improvements are not yet deployed." - log_warn "See: docs/how-to/migrate-ngit-relay-to-ngit-grasp.md (Dependencies section)" + log_warn "See: docs/how-to/migrate-to-ngit-grasp.md (Dependencies section)" fi exit 0 @@ -259,7 +259,7 @@ main() { log_warn "" log_warn " [PARSE_FAIL] kind=30618 event_id=abc123 reason=\"...\" repo=myrepo npub=npub1..." log_warn "" - log_warn "See: docs/how-to/migrate-ngit-relay-to-ngit-grasp.md (Dependencies section)" + log_warn "See: docs/how-to/migrate-to-ngit-grasp.md (Dependencies section)" log_warn "" # Create empty output file with header comment diff --git a/docs/how-to/migration-scripts/31-extract-purgatory-expiry.sh b/docs/how-to/migration-scripts/31-extract-purgatory-expiry.sh index 38b2ca3..8cadad9 100755 --- a/docs/how-to/migration-scripts/31-extract-purgatory-expiry.sh +++ b/docs/how-to/migration-scripts/31-extract-purgatory-expiry.sh @@ -2,7 +2,7 @@ # # 31-extract-purgatory-expiry.sh - Extract purgatory expiry events from systemd logs # -# PHASE 4b of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 4b of the GRASP relay to ngit-grasp migration analysis pipeline. # Extracts structured [PURGATORY_EXPIRED] log entries from journalctl. # # USAGE: @@ -53,7 +53,7 @@ # [PURGATORY_EXPIRED] log entries. Until those are implemented, this script # will find no matching entries (which is handled gracefully). # -# See: docs/how-to/migrate-ngit-relay-to-ngit-grasp.md (Dependencies section) +# See: docs/how-to/migrate-to-ngit-grasp.md (Dependencies section) # # Expected Rust logging code: # tracing::warn!( @@ -70,7 +70,7 @@ # RUNTIME: Depends on log volume, typically < 30 seconds # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # 30-extract-parse-failures.sh - Companion script for parse failure logs # @@ -238,7 +238,7 @@ main() { if [[ "$sample_count" -eq 0 ]]; then log_warn "No [PURGATORY_EXPIRED] entries found in logs." log_warn "This is expected if ngit-grasp logging improvements are not yet deployed." - log_warn "See: docs/how-to/migrate-ngit-relay-to-ngit-grasp.md (Dependencies section)" + log_warn "See: docs/how-to/migrate-to-ngit-grasp.md (Dependencies section)" fi exit 0 @@ -266,7 +266,7 @@ main() { log_warn "" log_warn " [PURGATORY_EXPIRED] repo=myrepo npub=npub1... reason=\"...\"" log_warn "" - log_warn "See: docs/how-to/migrate-ngit-relay-to-ngit-grasp.md (Dependencies section)" + log_warn "See: docs/how-to/migrate-to-ngit-grasp.md (Dependencies section)" log_warn "" # Create empty output file with header comment diff --git a/docs/how-to/migration-scripts/40-classify-actions.sh b/docs/how-to/migration-scripts/40-classify-actions.sh index 9fc718f..1706e47 100755 --- a/docs/how-to/migration-scripts/40-classify-actions.sh +++ b/docs/how-to/migration-scripts/40-classify-actions.sh @@ -2,7 +2,7 @@ # # 40-classify-actions.sh - Final classification of repos for migration action # -# PHASE 5 of the ngit-relay to ngit-grasp migration analysis pipeline. +# PHASE 5 of the GRASP relay to ngit-grasp migration analysis pipeline. # Combines all data sources from previous phases to produce actionable results. # # USAGE: @@ -82,7 +82,7 @@ # RUNTIME: < 5 seconds (local processing only) # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # 01-fetch-events.sh - Phase 1 (fetch events) # 10-check-git-sync.sh - Phase 2 (git sync check) # 20-categorize.sh, 21-compare-relays.sh - Phase 3 (categorize and compare) diff --git a/docs/how-to/migration-scripts/run-migration-analysis.sh b/docs/how-to/migration-scripts/run-migration-analysis.sh index 2ba25ab..84aec2a 100755 --- a/docs/how-to/migration-scripts/run-migration-analysis.sh +++ b/docs/how-to/migration-scripts/run-migration-analysis.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # -# run-migration-analysis.sh - Orchestrate the complete ngit-relay to ngit-grasp migration analysis +# run-migration-analysis.sh - Orchestrate the complete GRASP relay to ngit-grasp migration analysis # # This script runs all 5 phases of the migration analysis pipeline in sequence, # with proper error handling, progress reporting, and timing information. @@ -13,8 +13,8 @@ # ./run-migration-analysis.sh \ # --prod-relay wss://relay.ngit.dev \ # --archive-relay wss://archive.relay.ngit.dev \ -# --prod-git /var/lib/ngit-relay/git \ -# --archive-git /var/lib/ngit-relay-archive/git +# --prod-git /var/lib/grasp-relay/git \ +# --archive-git /var/lib/ngit-grasp/git # # USAGE: # ./run-migration-analysis.sh [options] @@ -64,12 +64,12 @@ # ./run-migration-analysis.sh \ # --prod-relay wss://relay.ngit.dev \ # --archive-relay wss://archive.relay.ngit.dev \ -# --prod-git /var/lib/ngit-relay/git \ -# --archive-git /var/lib/ngit-relay-archive/git \ +# --prod-git /var/lib/grasp-relay/git \ +# --archive-git /var/lib/ngit-grasp/git \ # --service ngit-grasp.service # # SEE ALSO: -# docs/how-to/migrate-ngit-relay-to-ngit-grasp.md - Full migration guide +# docs/how-to/migrate-to-ngit-grasp.md - Full migration guide # set -euo pipefail @@ -658,7 +658,7 @@ display_summary() { main() { parse_args "$@" - log_header "ngit-relay to ngit-grasp Migration Analysis" + log_header "GRASP Relay to ngit-grasp Migration Analysis" validate_args check_prerequisites