diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..c063f68 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,560 @@ +# AI Agent Guidelines for ngit-grasp + +**Purpose:** Ensure AI agents (and humans) maintain consistent documentation practices and avoid common pitfalls. + +**Last Updated:** November 4, 2025 + +--- + +## 📁 Documentation Structure + +### Overview + +We maintain a **clean, hierarchical documentation structure** to avoid documentation sprawl. All working documents have a defined lifecycle and location. + +``` +ngit-grasp/ +├── README.md # Project overview (keep updated) +├── AGENTS.md # This file - agent guidelines +├── CHANGELOG.md # User-facing changes (semver) +│ +├── docs/ # Permanent technical documentation +│ ├── README.md # Docs navigation guide +│ ├── ARCHITECTURE.md # System architecture +│ ├── TEST_STRATEGY.md # Testing approach +│ ├── GIT_PROTOCOL.md # Git protocol reference +│ ├── COMPARISON.md # vs other implementations +│ ├── GETTING_STARTED.md # Setup guide +│ └── DECISION_SUMMARY.md # Key architectural decisions +│ +├── docs/archive/ # Completed session/phase docs +│ ├── 2025-11-04-tag-migration.md +│ ├── 2025-11-04-flake-migration.md +│ └── 2025-11-03-architecture-investigation.md +│ +├── docs/learnings/ # Extracted knowledge (permanent) +│ ├── nix-flakes.md # Flake gotchas and patterns +│ ├── nostr-sdk.md # nostr-sdk patterns and upgrades +│ └── git-http-backend.md # Git protocol learnings +│ +├── grasp-audit/ # Audit tool subproject +│ ├── README.md # Main audit docs +│ ├── QUICK_START.md # Getting started +│ └── docs/ +│ └── archive/ # Audit-specific archives +│ +└── .ai/ # AI assistant context (ignored in git) + └── history/ # Conversation history +``` + +--- + +## 📋 Document Lifecycle + +### 1. Working Documents (Root Level) + +**Purpose:** Active development, session notes, status reports +**Location:** Project root +**Lifecycle:** Created → Updated → Archived +**Retention:** Archive after completion, delete if obsolete + +**Examples:** +- `TAG_MIGRATION_COMPLETE.md` → Archive when next phase starts +- `SESSION_2025_11_04_SUMMARY.md` → Archive at session end +- `NEXT_STEPS.md` → Update continuously, archive when complete + +**Rules:** +- ✅ Use descriptive names with dates: `YYYY-MM-DD-description.md` +- ✅ Mark status clearly: `[WIP]`, `[COMPLETE]`, `[ARCHIVED]` +- ✅ Include date and context at top +- ❌ Don't let root accumulate more than 5-10 working docs +- ❌ Don't create duplicates (merge or link instead) + +### 2. Permanent Documentation (docs/) + +**Purpose:** Long-term reference, architecture, guides +**Location:** `docs/` +**Lifecycle:** Created → Maintained → Updated +**Retention:** Permanent (version controlled) + +**Examples:** +- `docs/ARCHITECTURE.md` - System design +- `docs/TEST_STRATEGY.md` - Testing approach +- `docs/learnings/nix-flakes.md` - Extracted knowledge + +**Rules:** +- ✅ Keep updated as project evolves +- ✅ Use clear structure and headings +- ✅ Link between related docs +- ❌ Don't duplicate information (use links) +- ❌ Don't include session-specific details + +### 3. Archive (docs/archive/) + +**Purpose:** Historical record, completed phases +**Location:** `docs/archive/` +**Lifecycle:** Moved from root → Archived +**Retention:** Permanent (for reference) + +**Examples:** +- `docs/archive/2025-11-04-tag-migration.md` +- `docs/archive/2025-11-03-architecture-investigation.md` + +**Rules:** +- ✅ Rename with date prefix when archiving +- ✅ Add "ARCHIVED" marker at top +- ✅ Extract learnings to docs/learnings/ first +- ❌ Don't modify after archiving +- ❌ Don't reference in active documentation + +### 4. Learnings (docs/learnings/) + +**Purpose:** Reusable knowledge, gotchas, patterns +**Location:** `docs/learnings/` +**Lifecycle:** Extracted → Maintained → Updated +**Retention:** Permanent (living documents) + +**Examples:** +- `docs/learnings/nix-flakes.md` - Flake patterns and gotchas +- `docs/learnings/nostr-sdk.md` - SDK upgrade notes +- `docs/learnings/git-http-backend.md` - Git protocol tips + +**Rules:** +- ✅ Extract from session docs before archiving +- ✅ Organize by topic, not by session +- ✅ Include code examples +- ✅ Update as we learn more +- ❌ Don't duplicate official docs (link instead) + +--- + +## 🔄 Cleanup Process + +### When to Clean Up + +**Trigger:** Root directory has >10 markdown files +**Frequency:** End of each major phase or weekly +**Responsibility:** AI agents should proactively suggest cleanup + +### Cleanup Steps + +1. **Identify Completed Documents** + ```bash + # Find old working docs + ls -lt *.md | head -20 + ``` + +2. **Extract Learnings** + - Review each completed doc + - Extract gotchas, patterns, solutions + - Add to appropriate `docs/learnings/*.md` + +3. **Archive Completed Work** + ```bash + # Move to archive with date prefix + mv TAG_MIGRATION_COMPLETE.md docs/archive/2025-11-04-tag-migration.md + ``` + +4. **Delete Obsolete Documents** + - Duplicates (keep most recent/complete) + - Superseded documents + - Pure status reports (no learnings) + +5. **Update References** + - Update links in active docs + - Update README.md if needed + - Commit changes + +### Example Cleanup + +```bash +# Before cleanup (36 files in root!) +ls *.md | wc -l +# 36 + +# After cleanup (5-8 files in root) +ls *.md +# README.md +# AGENTS.md +# CHANGELOG.md +# CURRENT_STATUS.md +# NEXT_STEPS.md + +# Archived +ls docs/archive/ +# 2025-11-04-tag-migration.md +# 2025-11-04-flake-migration.md +# 2025-11-03-architecture-investigation.md +# ... + +# Learnings extracted +ls docs/learnings/ +# nix-flakes.md +# nostr-sdk.md +# git-http-backend.md +``` + +--- + +## 🚨 Common Gotchas + +### Nix Flakes + +**Always use `nix develop`, not `nix-shell`** + +```bash +# ✅ Correct +cd grasp-audit +nix develop +nix develop -c cargo build + +# ❌ Wrong +nix-shell +nix-shell --run "cargo build" +``` + +**Why:** We use `flake.nix`, not `shell.nix`. See `docs/learnings/nix-flakes.md`. + +**Flake Commands:** +```bash +# Show flake outputs +nix flake show + +# Update flake inputs +nix flake update + +# Build package +nix build + +# Run package +nix run +``` + +### Git Subprojects + +**grasp-audit is a subproject with its own flake** + +```bash +# ✅ Correct - enter grasp-audit environment +cd grasp-audit +nix develop +cargo build + +# ❌ Wrong - can't build from root +cd ngit-grasp +cargo build # This won't find grasp-audit +``` + +**Why:** `grasp-audit/` has its own `Cargo.toml` and `flake.nix`. + +### nostr-sdk Versions + +**We use nostr-sdk 0.43.x (latest stable)** + +```toml +# ✅ Correct +[dependencies] +nostr-sdk = "0.43" + +# ❌ Wrong +nostr-sdk = "0.35" # Old version, breaking changes +``` + +**Why:** We upgraded from 0.35 to 0.43. See `docs/learnings/nostr-sdk.md` for migration notes. + +**Common Breaking Changes:** +- `EventBuilder::new()` signature changed +- Tag API changed to `Tag::custom()` +- Filter API changed +- See archived upgrade docs for details + +### Testing Patterns + +**Integration tests require relay** + +```bash +# ✅ Correct - start relay first +docker run --rm -p 7000:7000 scsibug/nostr-rs-relay + +# Then run tests +cd grasp-audit +nix develop -c cargo test --ignored + +# ❌ Wrong - integration tests will fail +cargo test --ignored # No relay running +``` + +**Test Organization:** +```rust +// Unit tests (no relay needed) +#[cfg(test)] +mod tests { + #[test] + fn test_something() { } +} + +// Integration tests (relay required) +#[cfg(test)] +mod tests { + #[test] + #[ignore] // Requires relay + fn test_against_relay() { } +} +``` + +### Documentation Updates + +**Keep README.md synchronized** + +When you: +- Complete a major feature → Update README.md status +- Change architecture → Update docs/ARCHITECTURE.md +- Add dependencies → Update README.md tech stack +- Change workflow → Update docs/GETTING_STARTED.md + +**Don't:** +- Create duplicate documentation +- Leave stale status markers +- Forget to update CHANGELOG.md for user-facing changes + +--- + +## 📝 Writing Guidelines + +### Markdown Style + +```markdown +# Title (H1 - only one per file) + +**Date:** YYYY-MM-DD +**Status:** [WIP|COMPLETE|ARCHIVED] + +## Section (H2) + +### Subsection (H3) + +**Bold** for emphasis, `code` for commands/code. + +- Bullet lists for items +- Keep consistent style + +1. Numbered lists for sequences +2. Use when order matters + +✅ Use emoji for status (sparingly) +❌ Don't overuse emoji + +\`\`\`bash +# Code blocks with language +cargo build +\`\`\` +``` + +### Status Markers + +- `[WIP]` - Work in progress +- `[COMPLETE]` - Finished, may be archived +- `[ARCHIVED]` - Moved to archive, historical only +- `✅` - Success/complete +- `❌` - Failure/incorrect +- `⏳` - In progress +- `🔜` - Planned/next + +### Document Headers + +```markdown +# Document Title + +**Purpose:** One-line purpose +**Date:** YYYY-MM-DD +**Status:** [WIP|COMPLETE|ARCHIVED] +**Related:** Links to related docs + +--- + +## Content starts here +``` + +--- + +## 🤖 AI Agent Responsibilities + +### Before Creating New Documents + +1. **Check if document already exists** + ```bash + find . -name "*keyword*.md" + ``` + +2. **Check if information can be added to existing doc** + - Prefer updating over creating + - Use sections/subsections + +3. **Determine correct location** + - Working doc → Root + - Permanent → docs/ + - Learning → docs/learnings/ + - Historical → docs/archive/ + +4. **Use descriptive names with dates** + - `YYYY-MM-DD-description.md` for working docs + - `topic-name.md` for permanent docs + +### During Development + +1. **Update status markers** + - Mark WIP → COMPLETE when done + - Update README.md status section + +2. **Extract learnings as you go** + - Add gotchas to docs/learnings/ + - Don't wait until cleanup + +3. **Keep documentation DRY** + - Link to existing docs + - Don't duplicate information + +### End of Session + +1. **Suggest cleanup if needed** + - Count root .md files + - Suggest archiving completed docs + +2. **Create session summary** + - What was accomplished + - What's next + - Any blockers + +3. **Update permanent docs** + - Sync README.md with reality + - Update relevant docs/ files + +### Cleanup Time + +1. **Review all root .md files** +2. **Extract learnings to docs/learnings/** +3. **Archive completed work to docs/archive/** +4. **Delete obsolete duplicates** +5. **Update links in active docs** +6. **Commit with clear message** + +--- + +## 🎯 Quality Checklist + +### For Every Document + +- [ ] Clear purpose stated at top +- [ ] Date included +- [ ] Status marker present +- [ ] Proper heading hierarchy (H1 → H2 → H3) +- [ ] Code blocks have language specified +- [ ] Links are valid and relative +- [ ] No duplicate information +- [ ] Spell-checked and readable + +### For Working Documents + +- [ ] Descriptive filename with date +- [ ] Will be archived or deleted when done +- [ ] Not duplicating permanent docs +- [ ] Learnings extracted to docs/learnings/ + +### For Permanent Documents + +- [ ] In correct docs/ subdirectory +- [ ] Linked from docs/README.md +- [ ] Updated as project evolves +- [ ] No session-specific details +- [ ] Serves long-term purpose + +### For Archived Documents + +- [ ] Moved to docs/archive/ +- [ ] Renamed with date prefix +- [ ] ARCHIVED marker at top +- [ ] Learnings extracted first +- [ ] Not referenced in active docs + +--- + +## 📚 Reference Documents + +### Must Read +- **This file (AGENTS.md)** - Guidelines for documentation +- **README.md** - Project overview +- **docs/README.md** - Documentation navigation + +### Key Technical Docs +- **docs/ARCHITECTURE.md** - System design +- **docs/TEST_STRATEGY.md** - Testing approach +- **docs/GETTING_STARTED.md** - Setup guide + +### Learnings (Gotchas) +- **docs/learnings/nix-flakes.md** - Nix flake patterns +- **docs/learnings/nostr-sdk.md** - nostr-sdk notes +- **docs/learnings/git-http-backend.md** - Git protocol tips + +--- + +## 🔗 Quick Links + +- [Project README](README.md) +- [Documentation Index](docs/README.md) +- [Architecture](docs/ARCHITECTURE.md) +- [Learnings](docs/learnings/) +- [Archive](docs/archive/) + +--- + +## 💡 Tips for Success + +1. **Less is more** - Prefer updating over creating +2. **Archive often** - Keep root clean +3. **Extract learnings** - Make knowledge reusable +4. **Link, don't duplicate** - DRY applies to docs too +5. **Date everything** - Context is important +6. **Use descriptive names** - Future you will thank you +7. **Check before creating** - Document might already exist +8. **Update as you go** - Don't wait for cleanup time + +--- + +## 🚀 Next Steps + +After reading this: + +1. **Review current documentation structure** + ```bash + ls -la *.md + ls -la docs/ + ``` + +2. **Identify cleanup candidates** + - Completed working docs + - Obsolete duplicates + - Session summaries + +3. **Extract learnings** + - Review completed docs + - Add to docs/learnings/ + +4. **Archive and clean** + - Move to docs/archive/ + - Delete obsolete files + - Update links + +5. **Commit changes** + ```bash + git add . + git commit -m "docs: cleanup and reorganization" + ``` + +--- + +**Remember:** Good documentation structure is like good code structure - it makes everything easier. + +--- + +*Last updated: November 4, 2025* +*Status: ✅ Active guidelines* diff --git a/CLEANUP_SUMMARY.md b/CLEANUP_SUMMARY.md new file mode 100644 index 0000000..8ffce92 --- /dev/null +++ b/CLEANUP_SUMMARY.md @@ -0,0 +1,448 @@ +# Documentation Cleanup - November 4, 2025 + +**Purpose:** Summary of documentation reorganization +**Status:** ✅ Complete + +--- + +## Summary + +Cleaned up **32 markdown files** from project root, organizing them into a clear, maintainable structure. + +**Before:** 32 files in root (documentation sprawl) +**After:** 3 files in root (clean structure) + +--- + +## What Changed + +### Root Directory + +**Before:** +``` +32 markdown files including: +- Session summaries +- Status reports +- Migration docs +- Implementation reports +- Quick references +- Planning documents +``` + +**After:** +``` +3 essential files: +- README.md # Project overview +- AGENTS.md # AI agent guidelines +- CURRENT_STATUS.md # Current project state +``` + +--- + +### New Structure + +``` +docs/ +├── README.md # Docs navigation +├── ARCHITECTURE.md # System design +├── TEST_STRATEGY.md # Testing approach +├── GETTING_STARTED.md # Setup guide +├── GIT_PROTOCOL.md # Git protocol reference +├── COMPARISON.md # vs other implementations +├── DECISION_SUMMARY.md # Key decisions +│ +├── learnings/ # Reusable knowledge +│ ├── nix-flakes.md # Nix patterns & gotchas ✨ NEW +│ ├── nostr-sdk.md # nostr-sdk 0.43 notes ✨ NEW +│ └── grasp-audit.md # Audit tool patterns ✨ NEW +│ +└── archive/ # Historical documents + ├── README.md # Archive index ✨ NEW + ├── 2025-11-03-*.md # Nov 3 session docs (16 files) + └── 2025-11-04-*.md # Nov 4 session docs (14 files) +``` + +--- + +## Documents Archived + +### November 3, 2025 (16 files) + +**Investigation & Planning:** +- architecture-investigation.md +- review-summary.md +- documentation-index.md +- grasp-audit-plan.md + +**Implementation:** +- grasp-audit-implementation.md +- implementation-complete.md +- verification-complete.md + +**Testing:** +- compliance-test-proposal.md +- compliance-testing-report.md +- test-breakdown.md +- smoke-test-report.md +- final-audit-report.md +- final-summary.md + +**Reference:** +- files-created.md +- quick-reference.md +- start-here.md + +--- + +### November 4, 2025 (14 files) + +**Migrations:** +- tag-migration.md +- tag-migration-summary.md +- flake-migration.md + +**Upgrades:** +- nostr-sdk-upgrade.md +- upgrade-complete.md + +**Fixes:** +- compilation-fixes.md +- audit-system-fixed.md +- audit-status-report.md + +**Sessions:** +- session-summary.md +- session-complete-1.md +- session-complete-2.md +- session-continuation.md + +**Planning:** +- next-session-quickstart.md +- next-prompt.md +- ready-for-next-phase.md + +--- + +## Learnings Extracted + +Created 3 new learning documents with reusable knowledge: + +### 1. docs/learnings/nix-flakes.md + +**Content:** +- Critical gotcha: Use `nix develop`, not `nix-shell` +- Flake structure and patterns +- Common commands +- Subproject flakes +- Migration from shell.nix +- Benefits and best practices +- Common issues and solutions + +**Extracted from:** +- FLAKE_MIGRATION_COMPLETE.md +- Various session documents +- Real experience during development + +--- + +### 2. docs/learnings/nostr-sdk.md + +**Content:** +- Current version: 0.43.x +- Breaking changes from 0.35 → 0.43 +- Common patterns (events, tags, queries) +- Testing patterns (unit vs integration) +- Common gotchas and solutions +- Performance tips +- Migration checklist + +**Extracted from:** +- NOSTR_SDK_0.43_UPGRADE.md +- Implementation experience +- Test code examples + +--- + +### 3. docs/learnings/grasp-audit.md + +**Content:** +- Architecture decisions +- Audit event tagging strategy +- Code patterns +- Test isolation +- Cleanup strategy +- Testing organization +- Lessons learned +- Common issues + +**Extracted from:** +- TAG_MIGRATION_COMPLETE.md +- GRASP_AUDIT_PLAN.md +- Implementation summaries +- Testing experience + +--- + +## New Documents Created + +### CURRENT_STATUS.md + +**Purpose:** Single source of truth for project state + +**Content:** +- Quick summary +- Project structure +- What works +- What's next +- Development workflow +- Key technologies +- Important gotchas +- Recent milestones +- Success metrics +- Resources + +**Replaces:** Multiple status reports and session summaries + +--- + +### AGENTS.md (Updated) + +**Purpose:** AI agent documentation guidelines + +**Already existed but now enforced:** +- Documentation structure +- Document lifecycle +- Cleanup process +- Common gotchas +- Writing guidelines +- AI agent responsibilities +- Quality checklist + +--- + +### docs/archive/README.md + +**Purpose:** Archive organization and usage guide + +**Content:** +- Archive organization +- Document index by date/topic +- When to reference archives +- Extracting learnings +- Archive principles +- Quick find by topic/date + +--- + +## Benefits Achieved + +### 1. Clarity + +✅ **Easy to find current information** +- `CURRENT_STATUS.md` - where we are +- `README.md` - what the project is +- `AGENTS.md` - how to document + +✅ **Easy to find historical information** +- `docs/archive/` - organized by date +- `docs/archive/README.md` - searchable index + +--- + +### 2. Maintainability + +✅ **Clear document lifecycle** +- Working docs in root +- Permanent docs in docs/ +- Learnings extracted +- Completed work archived + +✅ **No more sprawl** +- Root directory stays clean +- Archive grows but stays organized +- Learnings get updated, not duplicated + +--- + +### 3. Reusability + +✅ **Learnings are accessible** +- Organized by topic, not session +- Include code examples +- Link to historical context +- Living documents that evolve + +✅ **Patterns are documented** +- Nix flake patterns +- nostr-sdk patterns +- grasp-audit patterns +- Testing patterns + +--- + +### 4. Onboarding + +✅ **New developers (human or AI) can:** +1. Read `README.md` - understand project +2. Read `CURRENT_STATUS.md` - know where we are +3. Read `AGENTS.md` - learn documentation practices +4. Read `docs/learnings/` - avoid known pitfalls +5. Reference `docs/archive/` - understand history + +--- + +## Cleanup Statistics + +### Before + +``` +Root directory: +- 32 markdown files +- Mix of status, reports, plans, summaries +- Hard to find current information +- Duplicate information +- No clear organization + +docs/ directory: +- 7 permanent docs +- 0 learnings +- 0 archived docs +``` + +### After + +``` +Root directory: +- 3 markdown files (README, AGENTS, CURRENT_STATUS) +- Clean and focused +- Clear purpose for each file + +docs/ directory: +- 7 permanent docs (unchanged) +- 3 learnings (NEW) +- 30 archived docs (NEW) +- 1 archive index (NEW) +``` + +--- + +## Document Count + +| Location | Count | Purpose | +|----------|-------|---------| +| Root | 3 | Essential project files | +| docs/ | 7 | Permanent documentation | +| docs/learnings/ | 3 | Reusable knowledge | +| docs/archive/ | 30 | Historical records | +| **Total** | **43** | **Well-organized docs** | + +--- + +## Maintenance Going Forward + +### Daily Development + +**Create working docs in root:** +- Session notes +- Status updates +- Temporary planning + +**Keep root clean:** +- Max 5-10 working docs +- Archive when complete +- Extract learnings first + +--- + +### Weekly Cleanup + +**Trigger:** Root has >10 markdown files + +**Process:** +1. Review completed working docs +2. Extract learnings to `docs/learnings/` +3. Archive to `docs/archive/YYYY-MM-DD-topic.md` +4. Delete obsolete duplicates +5. Update `CURRENT_STATUS.md` +6. Commit changes + +--- + +### Guidelines + +**Follow `AGENTS.md` for:** +- When to create new documents +- Where to put documents +- How to name documents +- When to archive +- How to extract learnings + +--- + +## Commit Message + +``` +docs: major cleanup and reorganization + +- Archive 30 completed session documents to docs/archive/ +- Extract learnings to docs/learnings/ (nix-flakes, nostr-sdk, grasp-audit) +- Create CURRENT_STATUS.md as single source of truth +- Create docs/archive/README.md for archive organization +- Clean root directory: 32 files → 3 files +- Enforce AGENTS.md documentation guidelines + +Root directory now contains only: +- README.md (project overview) +- AGENTS.md (documentation guidelines) +- CURRENT_STATUS.md (current state) + +All historical documents preserved in docs/archive/ with proper dating. +All reusable knowledge extracted to docs/learnings/. + +Benefits: +- Easy to find current information +- Clear document lifecycle +- No more documentation sprawl +- Learnings are accessible and reusable +- Better onboarding for new developers/agents +``` + +--- + +## Verification + +```bash +# Verify structure +ls -la *.md +# Should show: README.md, AGENTS.md, CURRENT_STATUS.md + +ls -la docs/learnings/ +# Should show: nix-flakes.md, nostr-sdk.md, grasp-audit.md + +ls -la docs/archive/ | wc -l +# Should show: 31 (30 files + README.md) + +# Verify no broken links (manual check) +grep -r "\.md" docs/ | grep -v ".git" +``` + +--- + +## Next Steps + +1. ✅ Cleanup complete +2. ✅ Learnings extracted +3. ✅ Archive organized +4. 🔜 Commit changes +5. 🔜 Start NIP-01 relay implementation + +--- + +**Cleanup completed:** November 4, 2025 +**Files organized:** 43 total +**Root cleaned:** 32 → 3 files +**Status:** ✅ Ready for next phase + +--- + +*This document will be archived after commit* diff --git a/CURRENT_STATUS.md b/CURRENT_STATUS.md new file mode 100644 index 0000000..417691a --- /dev/null +++ b/CURRENT_STATUS.md @@ -0,0 +1,464 @@ +# ngit-grasp - Current Status + +**Date:** November 4, 2025 +**Phase:** Audit Tool Complete - Ready for NIP-01 Implementation +**Status:** 🟢 All Systems Green + +--- + +## Quick Summary + +✅ **grasp-audit tool complete** - NIP-01 smoke tests passing +✅ **Tag migration complete** - Using standard NIP-01 "t" tags +✅ **nostr-sdk upgraded** - Version 0.43.x (latest stable) +✅ **Nix flakes migrated** - Modern reproducible builds +✅ **Documentation cleaned** - Clear structure established + +**Next:** Build NIP-01 relay implementation, test with grasp-audit + +--- + +## Project Structure + +``` +ngit-grasp/ +├── README.md # Project overview +├── AGENTS.md # AI agent guidelines +├── CURRENT_STATUS.md # This file +│ +├── docs/ # Permanent documentation +│ ├── ARCHITECTURE.md # System design +│ ├── TEST_STRATEGY.md # Testing approach +│ ├── GETTING_STARTED.md # Setup guide +│ ├── GIT_PROTOCOL.md # Git protocol reference +│ ├── COMPARISON.md # vs ngit-relay +│ ├── DECISION_SUMMARY.md # Key decisions +│ │ +│ ├── learnings/ # Reusable knowledge +│ │ ├── nix-flakes.md # Nix flake patterns +│ │ ├── nostr-sdk.md # nostr-sdk 0.43 notes +│ │ └── grasp-audit.md # Audit tool patterns +│ │ +│ └── archive/ # Historical documents +│ ├── 2025-11-04-tag-migration.md +│ ├── 2025-11-04-flake-migration.md +│ ├── 2025-11-04-nostr-sdk-upgrade.md +│ └── ... +│ +└── grasp-audit/ # Audit tool (separate crate) + ├── README.md # Audit tool docs + ├── QUICK_START.md # Getting started + ├── flake.nix # Nix dev environment + ├── Cargo.toml # Rust dependencies + └── src/ + ├── specs/ # Test specifications + │ └── nip01_smoke.rs # NIP-01 basic tests ✅ + ├── audit.rs # Audit config & event builder + ├── client.rs # Audit client wrapper + └── ... +``` + +--- + +## What Works + +### grasp-audit Tool ✅ + +**Status:** Fully functional, all tests passing + +```bash +cd grasp-audit +nix develop +cargo test --lib # 12/12 unit tests ✅ +cargo test -- --ignored # 1/1 integration test ✅ +cargo run -- audit --relay ws://localhost:7000 --spec nip01-smoke +# Results: 6/6 passed (100.0%) ✅ +``` + +**Features:** +- ✅ NIP-01 smoke tests (websocket, events, subscriptions) +- ✅ CI and production modes +- ✅ Test isolation via unique run IDs +- ✅ Standard "t" tag usage +- ✅ Audit event cleanup strategy +- ✅ CLI interface + +**Test Coverage:** +- websocket_connection +- send_receive_event +- create_subscription +- close_subscription +- reject_invalid_signature +- reject_invalid_event_id + +--- + +### Development Environment ✅ + +**Nix Flakes:** +- ✅ `grasp-audit/flake.nix` - Reproducible builds +- ✅ Rust toolchain via rust-overlay +- ✅ All dependencies managed +- ✅ Cross-platform support + +**Usage:** +```bash +cd grasp-audit +nix develop # Enter dev shell +nix develop -c cargo build # One-off command +nix build # Build package +``` + +--- + +### Documentation ✅ + +**Permanent Docs:** +- ✅ `docs/ARCHITECTURE.md` - Detailed system design +- ✅ `docs/TEST_STRATEGY.md` - Testing approach +- ✅ `docs/GETTING_STARTED.md` - Setup guide +- ✅ `docs/README.md` - Documentation index + +**Learnings:** +- ✅ `docs/learnings/nix-flakes.md` - Nix patterns and gotchas +- ✅ `docs/learnings/nostr-sdk.md` - nostr-sdk 0.43 migration +- ✅ `docs/learnings/grasp-audit.md` - Audit tool patterns + +**Guidelines:** +- ✅ `AGENTS.md` - AI agent documentation practices + +--- + +## What's Next + +### Immediate: NIP-01 Relay Implementation + +**Goal:** Build basic Nostr relay that passes grasp-audit tests + +**Approach:** +1. Create `src/` directory structure +2. Implement basic NIP-01 relay using nostr-relay-builder +3. Run grasp-audit tests against it +4. Iterate until all tests pass + +**Files to Create:** +``` +src/ +├── main.rs # Entry point +├── config.rs # Configuration +├── nostr/ +│ ├── mod.rs +│ ├── relay.rs # NIP-01 relay setup +│ └── events.rs # Event handling +└── storage/ + ├── mod.rs + └── repository.rs # Event storage +``` + +**Success Criteria:** +```bash +# Start ngit-grasp relay +cargo run + +# In another terminal +cd grasp-audit +cargo run -- audit --relay ws://localhost:8080 --spec nip01-smoke +# Results: 6/6 passed (100.0%) ✅ +``` + +--- + +### Phase 2: GRASP-01 Compliance + +**After NIP-01 works:** + +1. **Extend grasp-audit** + - Create `src/specs/grasp_01_relay.rs` + - Test repository announcements (NIP-34) + - Test state events + - Test maintainer validation + +2. **Implement in ngit-grasp** + - NIP-34 event validation + - Repository state management + - Maintainer authorization + +3. **Iterate** + - Run GRASP-01 audit tests + - Fix failures + - Repeat until passing + +--- + +### Phase 3: Git Integration + +**After GRASP-01 compliance:** + +1. **Git HTTP Backend** + - Implement git-smart-http handlers + - Integrate with authorization + +2. **Push Validation** + - Query Nostr state events + - Validate push permissions + - Inline authorization (no hooks) + +3. **Full GRASP-01** + - Complete service requirements + - End-to-end testing + +--- + +## Development Workflow + +### Daily Development + +```bash +# For ngit-grasp (when we create it) +cd ngit-grasp +nix develop +cargo build +cargo test +cargo run + +# For grasp-audit +cd grasp-audit +nix develop +cargo build +cargo test --lib +cargo test -- --ignored # Requires relay +cargo run -- audit --relay ws://localhost:8080 +``` + +--- + +### Running Tests + +**Unit Tests (Fast):** +```bash +# grasp-audit +cd grasp-audit +cargo test --lib + +# ngit-grasp (when created) +cargo test --lib +``` + +**Integration Tests (Requires Relay):** +```bash +# Start test relay +docker run --rm -p 7000:7000 scsibug/nostr-rs-relay + +# Run integration tests +cd grasp-audit +cargo test -- --ignored +``` + +**Audit Tests:** +```bash +# Start your relay +cd ngit-grasp +cargo run + +# Run audit in another terminal +cd grasp-audit +cargo run -- audit --relay ws://localhost:8080 +``` + +--- + +## Key Technologies + +### Current Stack + +- **Rust**: Core language +- **nostr-sdk 0.43**: Nostr event handling +- **Nix Flakes**: Reproducible dev environment +- **Cargo**: Build system +- **Docker**: Test relay (nostr-rs-relay) + +### Planned Stack (ngit-grasp) + +- **actix-web**: HTTP server +- **nostr-relay-builder**: Relay infrastructure +- **git-http-backend**: Git protocol handling +- **tokio**: Async runtime + +--- + +## Important Gotchas + +### 1. Use Nix Flakes, Not nix-shell + +```bash +# ✅ Correct +nix develop + +# ❌ Wrong +nix-shell +``` + +**Why:** We use `flake.nix`, not `shell.nix` + +--- + +### 2. grasp-audit is Separate + +```bash +# ✅ Correct +cd grasp-audit +nix develop +cargo build + +# ❌ Wrong +cd ngit-grasp +cargo build # Won't find grasp-audit +``` + +**Why:** Separate crate with own flake and Cargo.toml + +--- + +### 3. Integration Tests Need Relay + +```bash +# ✅ Correct +docker run --rm -p 7000:7000 scsibug/nostr-rs-relay +cargo test -- --ignored + +# ❌ Wrong +cargo test -- --ignored # Will fail without relay +``` + +--- + +### 4. nostr-sdk 0.43 API Changes + +**Event Building:** +```rust +// ✅ Correct (0.43) +EventBuilder::new(kind, content) + .tags(tags) + .sign_with_keys(&keys)? + +// ❌ Wrong (0.35) +EventBuilder::new(kind, content, tags) + .to_event(&keys)? +``` + +**See:** `docs/learnings/nostr-sdk.md` for full migration guide + +--- + +## Documentation Practices + +### When to Create Documents + +**Working Docs (Root):** +- Session summaries +- Status reports +- Next steps +- Temporary notes + +**Permanent Docs (docs/):** +- Architecture +- Design decisions +- API documentation +- User guides + +**Learnings (docs/learnings/):** +- Gotchas and patterns +- Migration notes +- Best practices +- Reusable knowledge + +**Archive (docs/archive/):** +- Completed session docs +- Historical records +- Superseded documents + +**See:** `AGENTS.md` for full guidelines + +--- + +## Recent Milestones + +- ✅ **Nov 4, 2025** - Tag migration to standard "t" tags +- ✅ **Nov 4, 2025** - Flake migration (shell.nix → flake.nix) +- ✅ **Nov 4, 2025** - nostr-sdk upgrade (0.35 → 0.43) +- ✅ **Nov 4, 2025** - Documentation cleanup +- ✅ **Nov 3, 2025** - Architecture investigation complete +- ✅ **Nov 3, 2025** - grasp-audit tool implemented +- ✅ **Nov 3, 2025** - NIP-01 smoke tests passing + +--- + +## Success Metrics + +### Current Status + +| Metric | Status | Details | +|--------|--------|---------| +| grasp-audit builds | ✅ | Clean build, no warnings | +| Unit tests | ✅ | 12/12 passing | +| Integration tests | ✅ | 1/1 passing | +| CLI works | ✅ | All commands functional | +| Smoke tests | ✅ | 6/6 passing | +| Documentation | ✅ | Complete and organized | +| Nix flakes | ✅ | Reproducible builds | + +### Next Milestone: NIP-01 Relay + +| Metric | Status | Target | +|--------|--------|--------| +| ngit-grasp builds | 🔜 | Clean build | +| NIP-01 relay running | 🔜 | Accepts connections | +| Smoke tests pass | 🔜 | 6/6 against ngit-grasp | +| Basic event storage | 🔜 | Events persist | +| Subscriptions work | 🔜 | Real-time updates | + +--- + +## Resources + +### Documentation +- [Project README](README.md) +- [Architecture](docs/ARCHITECTURE.md) +- [Test Strategy](docs/TEST_STRATEGY.md) +- [Getting Started](docs/GETTING_STARTED.md) +- [Agent Guidelines](AGENTS.md) + +### Learnings +- [Nix Flakes](docs/learnings/nix-flakes.md) +- [nostr-sdk](docs/learnings/nostr-sdk.md) +- [grasp-audit](docs/learnings/grasp-audit.md) + +### External +- [GRASP Protocol](https://gitworkshop.dev/danconwaydev.com/grasp) +- [NIP-01](https://github.com/nostr-protocol/nips/blob/master/01.md) +- [NIP-34](https://github.com/nostr-protocol/nips/blob/master/34.md) +- [nostr-sdk docs](https://docs.rs/nostr-sdk/0.43.0) + +--- + +## Contact & Contribution + +**Status:** Alpha - Active Development +**License:** MIT +**Repository:** ngit-grasp (local development) + +**Contributing:** +1. Read `AGENTS.md` for documentation practices +2. Review `docs/ARCHITECTURE.md` for design +3. Check `CURRENT_STATUS.md` (this file) for current state +4. Follow Rust conventions (`cargo fmt`, `cargo clippy`) +5. Add tests for new functionality + +--- + +**Last Updated:** November 4, 2025 +**Next Review:** When NIP-01 relay is implemented + +--- + +*Status: 🟢 Ready to build NIP-01 relay implementation* diff --git a/INVESTIGATION_COMPLETE.md b/docs/archive/2025-11-03-architecture-investigation.md similarity index 100% rename from INVESTIGATION_COMPLETE.md rename to docs/archive/2025-11-03-architecture-investigation.md diff --git a/COMPLIANCE_TEST_PROPOSAL.md b/docs/archive/2025-11-03-compliance-test-proposal.md similarity index 100% rename from COMPLIANCE_TEST_PROPOSAL.md rename to docs/archive/2025-11-03-compliance-test-proposal.md diff --git a/REPORT_COMPLIANCE_TESTING.md b/docs/archive/2025-11-03-compliance-testing-report.md similarity index 100% rename from REPORT_COMPLIANCE_TESTING.md rename to docs/archive/2025-11-03-compliance-testing-report.md diff --git a/DOCUMENTATION_INDEX.md b/docs/archive/2025-11-03-documentation-index.md similarity index 100% rename from DOCUMENTATION_INDEX.md rename to docs/archive/2025-11-03-documentation-index.md diff --git a/FILES_CREATED.md b/docs/archive/2025-11-03-files-created.md similarity index 100% rename from FILES_CREATED.md rename to docs/archive/2025-11-03-files-created.md diff --git a/FINAL_AUDIT_REPORT.md b/docs/archive/2025-11-03-final-audit-report.md similarity index 100% rename from FINAL_AUDIT_REPORT.md rename to docs/archive/2025-11-03-final-audit-report.md diff --git a/FINAL_SUMMARY.md b/docs/archive/2025-11-03-final-summary.md similarity index 100% rename from FINAL_SUMMARY.md rename to docs/archive/2025-11-03-final-summary.md diff --git a/GRASP_AUDIT_IMPLEMENTATION_SUMMARY.md b/docs/archive/2025-11-03-grasp-audit-implementation.md similarity index 100% rename from GRASP_AUDIT_IMPLEMENTATION_SUMMARY.md rename to docs/archive/2025-11-03-grasp-audit-implementation.md diff --git a/GRASP_AUDIT_PLAN.md b/docs/archive/2025-11-03-grasp-audit-plan.md similarity index 100% rename from GRASP_AUDIT_PLAN.md rename to docs/archive/2025-11-03-grasp-audit-plan.md diff --git a/IMPLEMENTATION_COMPLETE.md b/docs/archive/2025-11-03-implementation-complete.md similarity index 100% rename from IMPLEMENTATION_COMPLETE.md rename to docs/archive/2025-11-03-implementation-complete.md diff --git a/QUICK_REFERENCE.md b/docs/archive/2025-11-03-quick-reference.md similarity index 100% rename from QUICK_REFERENCE.md rename to docs/archive/2025-11-03-quick-reference.md diff --git a/REVIEW_SUMMARY.md b/docs/archive/2025-11-03-review-summary.md similarity index 100% rename from REVIEW_SUMMARY.md rename to docs/archive/2025-11-03-review-summary.md diff --git a/SMOKE_TEST_REPORT.md b/docs/archive/2025-11-03-smoke-test-report.md similarity index 100% rename from SMOKE_TEST_REPORT.md rename to docs/archive/2025-11-03-smoke-test-report.md diff --git a/START_HERE.md b/docs/archive/2025-11-03-start-here.md similarity index 100% rename from START_HERE.md rename to docs/archive/2025-11-03-start-here.md diff --git a/TEST_BREAKDOWN.md b/docs/archive/2025-11-03-test-breakdown.md similarity index 100% rename from TEST_BREAKDOWN.md rename to docs/archive/2025-11-03-test-breakdown.md diff --git a/VERIFICATION_COMPLETE.md b/docs/archive/2025-11-03-verification-complete.md similarity index 100% rename from VERIFICATION_COMPLETE.md rename to docs/archive/2025-11-03-verification-complete.md diff --git a/AUDIT_SYSTEM_STATUS_REPORT.md b/docs/archive/2025-11-04-audit-status-report.md similarity index 100% rename from AUDIT_SYSTEM_STATUS_REPORT.md rename to docs/archive/2025-11-04-audit-status-report.md diff --git a/AUDIT_SYSTEM_FIXED.md b/docs/archive/2025-11-04-audit-system-fixed.md similarity index 100% rename from AUDIT_SYSTEM_FIXED.md rename to docs/archive/2025-11-04-audit-system-fixed.md diff --git a/COMPILATION_FIXES.md b/docs/archive/2025-11-04-compilation-fixes.md similarity index 100% rename from COMPILATION_FIXES.md rename to docs/archive/2025-11-04-compilation-fixes.md diff --git a/FLAKE_MIGRATION_COMPLETE.md b/docs/archive/2025-11-04-flake-migration.md similarity index 100% rename from FLAKE_MIGRATION_COMPLETE.md rename to docs/archive/2025-11-04-flake-migration.md diff --git a/next_prompt.md b/docs/archive/2025-11-04-next-prompt.md similarity index 100% rename from next_prompt.md rename to docs/archive/2025-11-04-next-prompt.md diff --git a/NEXT_SESSION_QUICKSTART.md b/docs/archive/2025-11-04-next-session-quickstart.md similarity index 100% rename from NEXT_SESSION_QUICKSTART.md rename to docs/archive/2025-11-04-next-session-quickstart.md diff --git a/NOSTR_SDK_0.43_UPGRADE.md b/docs/archive/2025-11-04-nostr-sdk-upgrade.md similarity index 100% rename from NOSTR_SDK_0.43_UPGRADE.md rename to docs/archive/2025-11-04-nostr-sdk-upgrade.md diff --git a/READY_FOR_NEXT_PHASE.md b/docs/archive/2025-11-04-ready-for-next-phase.md similarity index 100% rename from READY_FOR_NEXT_PHASE.md rename to docs/archive/2025-11-04-ready-for-next-phase.md diff --git a/SESSION_COMPLETE.md b/docs/archive/2025-11-04-session-complete-1.md similarity index 100% rename from SESSION_COMPLETE.md rename to docs/archive/2025-11-04-session-complete-1.md diff --git a/SESSION_COMPLETE_2025_11_04.md b/docs/archive/2025-11-04-session-complete-2.md similarity index 100% rename from SESSION_COMPLETE_2025_11_04.md rename to docs/archive/2025-11-04-session-complete-2.md diff --git a/SESSION_CONTINUATION_COMPLETE.md b/docs/archive/2025-11-04-session-continuation.md similarity index 100% rename from SESSION_CONTINUATION_COMPLETE.md rename to docs/archive/2025-11-04-session-continuation.md diff --git a/SESSION_2025_11_04_SUMMARY.md b/docs/archive/2025-11-04-session-summary.md similarity index 100% rename from SESSION_2025_11_04_SUMMARY.md rename to docs/archive/2025-11-04-session-summary.md diff --git a/TAG_MIGRATION_SUMMARY.md b/docs/archive/2025-11-04-tag-migration-summary.md similarity index 100% rename from TAG_MIGRATION_SUMMARY.md rename to docs/archive/2025-11-04-tag-migration-summary.md diff --git a/TAG_MIGRATION_COMPLETE.md b/docs/archive/2025-11-04-tag-migration.md similarity index 100% rename from TAG_MIGRATION_COMPLETE.md rename to docs/archive/2025-11-04-tag-migration.md diff --git a/UPGRADE_COMPLETE.md b/docs/archive/2025-11-04-upgrade-complete.md similarity index 100% rename from UPGRADE_COMPLETE.md rename to docs/archive/2025-11-04-upgrade-complete.md diff --git a/docs/archive/README.md b/docs/archive/README.md new file mode 100644 index 0000000..9ff9e3e --- /dev/null +++ b/docs/archive/README.md @@ -0,0 +1,158 @@ +# Archive - Historical Documentation + +**Purpose:** Completed session documents, phase reports, and historical records +**Status:** Read-only - documents are not modified after archiving + +--- + +## Archive Organization + +Documents are organized by date (YYYY-MM-DD) and topic. + +### November 3, 2025 - Architecture Investigation & Initial Implementation + +**Architecture Investigation:** +- `2025-11-03-architecture-investigation.md` - GRASP protocol investigation complete +- `2025-11-03-review-summary.md` - Executive summary of investigation +- `2025-11-03-documentation-index.md` - Initial docs structure + +**grasp-audit Implementation:** +- `2025-11-03-grasp-audit-plan.md` - Audit tool design decisions +- `2025-11-03-grasp-audit-implementation.md` - Implementation summary +- `2025-11-03-implementation-complete.md` - Initial implementation complete +- `2025-11-03-verification-complete.md` - Verification results + +**Testing:** +- `2025-11-03-compliance-test-proposal.md` - Test strategy proposal +- `2025-11-03-compliance-testing-report.md` - Compliance testing report +- `2025-11-03-test-breakdown.md` - Detailed test breakdown +- `2025-11-03-smoke-test-report.md` - Smoke test results +- `2025-11-03-final-audit-report.md` - Final audit report +- `2025-11-03-final-summary.md` - Final summary + +**Reference:** +- `2025-11-03-files-created.md` - Files created during investigation +- `2025-11-03-quick-reference.md` - Quick reference guide +- `2025-11-03-start-here.md` - Getting started guide + +--- + +### November 4, 2025 - Upgrades & Migrations + +**Tag Migration:** +- `2025-11-04-tag-migration.md` - Migration to standard "t" tags (detailed) +- `2025-11-04-tag-migration-summary.md` - Migration summary + +**Flake Migration:** +- `2025-11-04-flake-migration.md` - shell.nix → flake.nix migration + +**nostr-sdk Upgrade:** +- `2025-11-04-nostr-sdk-upgrade.md` - 0.35 → 0.43 upgrade guide +- `2025-11-04-upgrade-complete.md` - Upgrade completion report + +**Fixes & Improvements:** +- `2025-11-04-compilation-fixes.md` - Compilation fixes +- `2025-11-04-audit-system-fixed.md` - Audit system fixes +- `2025-11-04-audit-status-report.md` - Audit status report + +**Session Summaries:** +- `2025-11-04-session-summary.md` - Main session summary +- `2025-11-04-session-complete-1.md` - Session completion 1 +- `2025-11-04-session-complete-2.md` - Session completion 2 +- `2025-11-04-session-continuation.md` - Session continuation + +**Planning:** +- `2025-11-04-next-session-quickstart.md` - Next session quickstart +- `2025-11-04-next-prompt.md` - Next prompt planning +- `2025-11-04-ready-for-next-phase.md` - Phase readiness report + +--- + +## Using Archived Documents + +### When to Reference + +✅ **Good reasons to reference:** +- Understanding historical context +- Learning from past decisions +- Reviewing what was tried before +- Tracking project evolution + +❌ **Don't reference for:** +- Current implementation details (use `docs/` instead) +- Active development (use `CURRENT_STATUS.md`) +- Reusable patterns (use `docs/learnings/`) + +### Extracting Learnings + +If you find useful patterns or gotchas in archived documents: + +1. Extract to appropriate `docs/learnings/*.md` file +2. Update with current context +3. Link to archive for historical context + +**Example:** +```markdown + + +## Tag Migration Pattern + +When changing tag structure... + +**Reference:** See `docs/archive/2025-11-04-tag-migration.md` for detailed migration story. +``` + +--- + +## Archive Principles + +1. **Immutable**: Documents are not modified after archiving +2. **Dated**: All filenames include YYYY-MM-DD prefix +3. **Organized**: Grouped by date and topic +4. **Referenced**: Can be linked from active docs for context +5. **Searchable**: Full-text search helps find historical info + +--- + +## Document Lifecycle + +``` +Working Doc (root) + ↓ +Extract Learnings → docs/learnings/ + ↓ +Archive → docs/archive/ + ↓ +Reference (read-only) +``` + +--- + +## Quick Find + +### By Topic + +- **Architecture**: `2025-11-03-architecture-investigation.md` +- **Testing**: `2025-11-03-*-test-*.md` +- **Migrations**: `2025-11-04-*-migration.md` +- **Upgrades**: `2025-11-04-*-upgrade.md` +- **Sessions**: `2025-11-04-session-*.md` + +### By Date + +- **Nov 3**: Initial investigation and implementation +- **Nov 4**: Upgrades, migrations, and refinements + +--- + +## Related Documentation + +- **Active Status**: `../CURRENT_STATUS.md` +- **Learnings**: `../learnings/` +- **Architecture**: `../ARCHITECTURE.md` +- **Guidelines**: `../../AGENTS.md` + +--- + +*Archive established: November 4, 2025* +*Total documents: 30* diff --git a/docs/learnings/grasp-audit.md b/docs/learnings/grasp-audit.md new file mode 100644 index 0000000..531ebda --- /dev/null +++ b/docs/learnings/grasp-audit.md @@ -0,0 +1,498 @@ +# GRASP Audit Tool - Patterns and Learnings + +**Purpose:** Document grasp-audit architecture, patterns, and lessons learned +**Last Updated:** November 4, 2025 + +--- + +## Overview + +`grasp-audit` is a compliance testing tool for GRASP (Git Relays Authorized via Signed-Nostr Proofs) protocol implementations. It tests both Nostr relay compliance (NIP-01) and GRASP-specific functionality. + +--- + +## Architecture Decisions + +### Separate Crate Strategy + +**Decision:** Build `grasp-audit` as a separate crate from `ngit-grasp` + +**Why:** +1. **Parallel Development**: Can build tests before implementation +2. **Isolated Testing**: Tests run in isolation (CI/CD safe) +3. **Production Auditing**: Can audit live production services +4. **Reusability**: Other GRASP implementations can use it + +**Location:** `grasp-audit/` subdirectory with own `Cargo.toml` and `flake.nix` + +--- + +### Audit Event Tagging Strategy + +**Problem:** Test events pollute the relay and need cleanup without deletion events. + +**Solution:** Use special tags to mark audit events: + +```rust +// Every audit event includes these tags +[ + ["t", "grasp-audit-test-event"], // Marker + ["t", "audit-{run-id}"], // Run isolation + ["t", "audit-cleanup-after-{timestamp}"] // Cleanup time +] +``` + +**Benefits:** +- ✅ **Queryable**: Can find all audit events via tag filter +- ✅ **Isolated**: Each test run has unique run ID +- ✅ **Self-cleaning**: Cleanup timestamp indicates when to delete +- ✅ **No deletion events**: Direct database cleanup, no KIND 5 events +- ✅ **Production safe**: Won't interfere with real events + +**Reference:** See `docs/archive/2025-11-04-tag-migration.md` + +--- + +### Standard "t" Tags vs Custom Tags + +**Evolution:** +1. **Original**: Custom single-letter tags (`g`, `r`, `c`) +2. **Current**: Standard NIP-01 "t" tags with prefixed values + +**Why we changed:** +- ❌ Custom tags could conflict with other systems +- ✅ "t" tag is standard for categorization/topics +- ✅ Multiple "t" tags are expected and supported +- ✅ Self-documenting values (`audit-{run-id}` vs just `{run-id}`) +- ✅ Better namespacing with prefixes + +**Migration:** Completed November 4, 2025 + +--- + +## Code Patterns + +### Audit Configuration + +```rust +use grasp_audit::audit::AuditConfig; + +// CI mode - isolated test runs +let config = AuditConfig::ci(); +// Generates UUID run ID: "ci-{uuid}" +// Cleanup after 1 hour + +// Production mode - persistent run ID +let config = AuditConfig::production("prod-server-1"); +// Uses provided run ID +// Cleanup after 24 hours +``` + +**When to use:** +- **CI mode**: Automated testing, parallel runs, temporary +- **Production mode**: Manual audits, monitoring, persistent + +--- + +### Creating Audit Events + +```rust +use grasp_audit::audit::{AuditConfig, AuditEventBuilder}; +use nostr_sdk::prelude::*; + +let config = AuditConfig::ci(); +let keys = Keys::generate(); + +// Create audit event +let event = AuditEventBuilder::new(&config, Kind::TextNote, "test content") + .build(&keys)?; + +// Event automatically includes: +// - Audit marker tag +// - Run ID tag +// - Cleanup timestamp tag +``` + +--- + +### Querying Audit Events + +```rust +use grasp_audit::client::AuditClient; +use grasp_audit::audit::AuditConfig; + +let config = AuditConfig::ci(); +let client = AuditClient::new(config, keys); + +// Connect to relay +client.add_relay("ws://localhost:7000").await?; +client.connect().await; + +// Query audit events for this run +let events = client.query().await?; + +// Events are filtered by: +// - "grasp-audit-test-event" marker +// - Current run ID +``` + +--- + +### Test Isolation + +**Each test run is isolated by unique run ID:** + +```rust +// CI mode generates unique UUID per run +let config1 = AuditConfig::ci(); +let config2 = AuditConfig::ci(); + +// config1.run_id != config2.run_id +// Tests won't interfere with each other +``` + +**Benefits:** +- ✅ Parallel CI/CD runs don't conflict +- ✅ Can run multiple test suites simultaneously +- ✅ Easy to identify which run created which events +- ✅ Cleanup can target specific runs + +--- + +### Cleanup Strategy + +**Two-phase cleanup:** + +1. **Automatic expiry** via cleanup timestamp tag +2. **Manual cleanup** by querying and deleting + +```rust +// Events include cleanup timestamp +["t", "audit-cleanup-after-1730707200"] + +// Cleanup process: +// 1. Query events with expired cleanup timestamp +// 2. Delete from database directly (no KIND 5) +// 3. Avoid deletion event pollution +``` + +**Implementation:** To be built in relay (not in audit tool) + +--- + +## Testing Strategy + +### Test Organization + +``` +grasp-audit/src/specs/ +├── nip01_smoke.rs # NIP-01 basic functionality +├── grasp_01_relay.rs # GRASP-01 relay requirements (planned) +└── mod.rs # Test suite registry +``` + +### Unit vs Integration Tests + +**Unit Tests** (no relay required): +```rust +#[cfg(test)] +mod tests { + #[test] + fn test_audit_config() { + let config = AuditConfig::ci(); + assert!(config.run_id.starts_with("ci-")); + } +} +``` + +**Integration Tests** (relay required): +```rust +#[cfg(test)] +mod tests { + #[tokio::test] + #[ignore] // Requires relay + async fn test_smoke_tests_against_relay() { + // Test against real relay + } +} +``` + +**Running tests:** +```bash +# Unit tests (fast, no dependencies) +cargo test --lib + +# Integration tests (requires relay) +docker run --rm -p 7000:7000 scsibug/nostr-rs-relay +cargo test -- --ignored +``` + +--- + +### Test Result Reporting + +```rust +use grasp_audit::result::AuditResult; + +// Run tests +let results = vec![ + AuditResult::pass("websocket_connection", "Connected successfully"), + AuditResult::fail("invalid_event", "Expected rejection, got acceptance"), +]; + +// Report +for result in &results { + println!("{}", result); +} + +// Summary +let passed = results.iter().filter(|r| r.is_pass()).count(); +let total = results.len(); +println!("Results: {}/{} passed ({:.1}%)", + passed, total, (passed as f64 / total as f64) * 100.0); +``` + +--- + +## CLI Design + +### Command Structure + +```bash +grasp-audit audit [OPTIONS] + +Options: + --relay Relay to test (required) + --mode ci or production (default: ci) + --run-id Custom run ID (production mode only) + --spec Test spec to run (default: all) + --verbose Detailed output +``` + +### Usage Examples + +```bash +# CI mode - quick smoke test +grasp-audit audit \ + --relay ws://localhost:7000 \ + --mode ci \ + --spec nip01-smoke + +# Production mode - full compliance audit +grasp-audit audit \ + --relay wss://relay.example.com \ + --mode production \ + --run-id "audit-2025-11-04" \ + --verbose + +# Test all specs +grasp-audit audit --relay ws://localhost:7000 +``` + +--- + +## Lessons Learned + +### 1. Tag Migration is Breaking + +**Lesson:** Changing tag structure breaks event queries. + +**Impact:** Events created with old tags won't be found by new queries. + +**Mitigation:** +- ✅ Accept breaking changes in alpha stage +- ✅ Document migration clearly +- ✅ Old events auto-expire via cleanup +- ✅ No production deployments affected + +**Reference:** `docs/archive/2025-11-04-tag-migration.md` + +--- + +### 2. Test Data Lifecycle Matters + +**Lesson:** Test events accumulate and pollute relay. + +**Solution:** Built-in cleanup strategy from day one. + +**Implementation:** +- Every event has cleanup timestamp +- Relay can cleanup expired events +- No deletion event pollution (direct DB cleanup) + +--- + +### 3. Isolation Enables Parallel Testing + +**Lesson:** Unique run IDs enable parallel test execution. + +**Benefit:** CI/CD can run multiple test suites simultaneously. + +**Pattern:** +```rust +// Each CI run gets unique ID +let config = AuditConfig::ci(); +// run_id = "ci-{uuid}" + +// Tests isolated by run ID +let events = client.query().await?; +// Only returns events for this run +``` + +--- + +### 4. Standards Compliance Reduces Friction + +**Lesson:** Using standard NIP-01 "t" tags instead of custom tags. + +**Benefits:** +- ✅ No conflicts with other systems +- ✅ Standard relay filtering works +- ✅ Better interoperability +- ✅ Self-documenting + +--- + +## Future Enhancements + +### Planned Features + +- [ ] **GRASP-01 Test Suite**: Repository announcement and state event tests +- [ ] **Test Report Generation**: JSON/HTML output for CI/CD +- [ ] **Performance Benchmarks**: Measure relay performance +- [ ] **Relay Comparison**: Side-by-side compliance comparison +- [ ] **Continuous Monitoring**: Periodic production audits + +--- + +### Possible Improvements + +- [ ] **Parallel Test Execution**: Run specs in parallel +- [ ] **Retry Logic**: Handle transient failures +- [ ] **Custom Assertions**: Domain-specific test helpers +- [ ] **Event Diff Tool**: Compare expected vs actual events +- [ ] **Cleanup Automation**: Auto-cleanup after tests + +--- + +## Common Issues + +### Issue: Integration Tests Fail + +**Symptoms:** Tests timeout or fail to connect + +**Causes:** +1. No relay running +2. Wrong relay URL +3. Firewall blocking connection + +**Solution:** +```bash +# Start relay +docker run --rm -p 7000:7000 scsibug/nostr-rs-relay + +# Verify relay is running +curl http://localhost:7000 + +# Run tests +cargo test -- --ignored +``` + +--- + +### Issue: Events Not Found in Query + +**Symptoms:** Query returns empty even though events were sent + +**Causes:** +1. Wrong run ID (querying different run) +2. Connection timing (query before event propagated) +3. Tag mismatch (uppercase vs lowercase) + +**Solution:** +```rust +// Use same config for send and query +let config = AuditConfig::ci(); + +// Wait for event to propagate +tokio::time::sleep(Duration::from_millis(500)).await; + +// Verify tags match exactly +let t_tag = SingleLetterTag::lowercase(Alphabet::T); // Lowercase! +``` + +--- + +### Issue: Build Fails in CI + +**Symptoms:** `cargo build` fails with dependency errors + +**Cause:** Not in Nix dev environment + +**Solution:** +```bash +# Enter Nix environment first +cd grasp-audit +nix develop + +# Then build +cargo build +``` + +--- + +## Quick Reference + +### Configuration + +```rust +// CI mode +let config = AuditConfig::ci(); + +// Production mode +let config = AuditConfig::production("run-id"); +``` + +### Event Creation + +```rust +let event = AuditEventBuilder::new(&config, kind, content) + .build(&keys)?; +``` + +### Client Usage + +```rust +let client = AuditClient::new(config, keys); +client.add_relay("ws://localhost:7000").await?; +client.connect().await; +let events = client.query().await?; +``` + +### Running Tests + +```bash +# Unit tests +cargo test --lib + +# Integration tests +cargo test -- --ignored + +# CLI +cargo run -- audit --relay ws://localhost:7000 +``` + +--- + +## References + +- **GRASP Protocol**: https://gitworkshop.dev/danconwaydev.com/grasp +- **NIP-01**: https://github.com/nostr-protocol/nips/blob/master/01.md +- **NIP-34**: https://github.com/nostr-protocol/nips/blob/master/34.md +- **grasp-audit README**: `grasp-audit/README.md` +- **Tag Migration**: `docs/archive/2025-11-04-tag-migration.md` + +--- + +*Last updated: November 4, 2025* +*Status: Living document - update as grasp-audit evolves* diff --git a/docs/learnings/nix-flakes.md b/docs/learnings/nix-flakes.md new file mode 100644 index 0000000..6876647 --- /dev/null +++ b/docs/learnings/nix-flakes.md @@ -0,0 +1,423 @@ +# Nix Flakes - Learnings and Gotchas + +**Purpose:** Document Nix flake patterns, gotchas, and best practices learned during ngit-grasp development +**Last Updated:** November 4, 2025 + +--- + +## Critical Gotchas + +### Always Use `nix develop`, Not `nix-shell` + +**Problem:** We use `flake.nix`, not `shell.nix`. Using `nix-shell` will fail or use the wrong environment. + +```bash +# ✅ Correct - for flake.nix +cd grasp-audit +nix develop +nix develop -c cargo build + +# ❌ Wrong - for shell.nix (we don't use this) +nix-shell +nix-shell --run "cargo build" +``` + +**Why:** +- `nix-shell` looks for `shell.nix` or `default.nix` +- `nix develop` looks for `flake.nix` +- We migrated from `shell.nix` to `flake.nix` on November 4, 2025 + +**Related:** See `docs/archive/2025-11-04-flake-migration.md` + +--- + +## Flake Structure + +### Our Standard Flake Pattern + +```nix +{ + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; + rust-overlay.url = "github:oxalica/rust-overlay"; + flake-utils.url = "github:numtide/flake-utils"; + }; + + outputs = { nixpkgs, rust-overlay, flake-utils, ... }: + flake-utils.lib.eachDefaultSystem (system: + let + overlays = [ (import rust-overlay) ]; + pkgs = import nixpkgs { inherit system overlays; }; + manifest = pkgs.lib.importTOML ./Cargo.toml; + in with pkgs; { + # Development shell + devShells.default = mkShell { + nativeBuildInputs = [ + rust-bin.stable.latest.default + pkg-config + gitlint + ]; + buildInputs = [ + openssl + ]; + shellHook = '' + echo "🦀 Development environment loaded" + export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc} + ''; + }; + + # Package output + packages.default = pkgs.rustPlatform.buildRustPackage { + pname = manifest.package.name; + version = manifest.package.version; + src = ./.; + cargoLock = { lockFile = ./Cargo.lock; }; + buildInputs = [ openssl ]; + nativeBuildInputs = [ pkg-config ]; + doCheck = false; # Run tests separately + }; + }); +} +``` + +### Key Components + +1. **rust-overlay**: Provides latest stable Rust toolchain +2. **flake-utils**: Cross-platform support helper +3. **manifest**: Auto-read version from Cargo.toml +4. **devShells.default**: Development environment +5. **packages.default**: Buildable package + +--- + +## Common Flake Commands + +### Essential Commands + +```bash +# Enter development shell +nix develop + +# Run command in dev shell (one-off) +nix develop -c cargo build + +# Show flake outputs +nix flake show + +# Check flake validity +nix flake check + +# Update flake inputs (like updating dependencies) +nix flake update + +# Build the package directly +nix build + +# Run without installing +nix run + +# Show flake metadata +nix flake metadata +``` + +### Debugging Commands + +```bash +# Show detailed evaluation trace +nix develop --show-trace + +# Print flake evaluation +nix eval .#devShells.x86_64-linux.default + +# Check what's in the store +nix path-info .#packages.x86_64-linux.default +``` + +--- + +## Subproject Flakes + +### grasp-audit Has Its Own Flake + +**Important:** `grasp-audit/` is a subproject with its own `flake.nix` and `Cargo.toml`. + +```bash +# ✅ Correct - enter grasp-audit environment +cd grasp-audit +nix develop +cargo build + +# ❌ Wrong - can't build from root +cd ngit-grasp +cargo build # This won't find grasp-audit dependencies +``` + +**Why:** +- Each Rust workspace needs its own Nix environment +- Dependencies are project-specific +- Flake inputs are locked per-project + +--- + +## Migration from shell.nix to flake.nix + +### What Changed + +**Before (shell.nix):** +```nix +{ pkgs ? import {} }: + +pkgs.mkShell { + buildInputs = with pkgs; [ + rustc + cargo + openssl + pkg-config + ]; +} +``` + +**After (flake.nix):** +- Locked inputs (reproducible) +- Multi-output (dev shell + package) +- Cross-platform by default +- Better tooling integration + +### Migration Steps + +1. Create `flake.nix` with standard structure +2. Run `nix flake check` to validate +3. Update all documentation: `nix-shell` → `nix develop` +4. Test that build works: `nix develop -c cargo build` +5. Remove `shell.nix` +6. Commit changes + +**Reference:** See `docs/archive/2025-11-04-flake-migration.md` + +--- + +## Benefits of Flakes + +### Reproducibility + +**Locked inputs** ensure everyone gets the same environment: + +```bash +# flake.lock contains exact commits +$ cat flake.lock +{ + "nodes": { + "nixpkgs": { + "locked": { + "lastModified": 1698611440, + "narHash": "sha256-jPjHjrerhYDy3q9+s5EAsuhyhuknNfowY6yt6pjn9pc=", + "rev": "23e89e0c8c5e2d9cf5b5e7c3e8e8e8e8e8e8e8e8" + } + } + } +} +``` + +Everyone running `nix develop` gets **exactly** this version of nixpkgs. + +### Multi-Output + +Single flake provides: +- **devShells.default**: Development environment +- **packages.default**: Buildable package +- **apps.default**: Runnable application (optional) + +### Composability + +Flakes can use other flakes as inputs: + +```nix +{ + inputs = { + grasp-audit.url = "path:./grasp-audit"; + }; +} +``` + +--- + +## Common Issues + +### Issue: "error: getting status of '/nix/store/...': No such file or directory" + +**Cause:** Flake inputs need to be updated or fetched + +**Solution:** +```bash +nix flake update +nix develop +``` + +### Issue: "error: experimental feature 'nix-command' is not enabled" + +**Cause:** Nix flakes are experimental and need to be enabled + +**Solution:** +Add to `~/.config/nix/nix.conf`: +``` +experimental-features = nix-command flakes +``` + +### Issue: Changes to flake.nix not taking effect + +**Cause:** Flake evaluation is cached + +**Solution:** +```bash +# Clear evaluation cache +nix flake update +# Or force re-evaluation +nix develop --refresh +``` + +### Issue: "error: cannot find flake 'flake:self' in the flake registries" + +**Cause:** Not in a git repository or flake.nix not committed + +**Solution:** +```bash +git add flake.nix flake.lock +git commit -m "Add flake" +``` + +**Note:** Flakes require git. Uncommitted files are ignored by default. + +--- + +## Best Practices + +### 1. Always Commit flake.lock + +```bash +git add flake.lock +git commit -m "Update flake inputs" +``` + +**Why:** Ensures reproducibility across machines and CI/CD + +### 2. Use Specific Rust Versions When Needed + +```nix +# Latest stable (default) +rust-bin.stable.latest.default + +# Specific version +rust-bin.stable."1.75.0".default + +# Nightly +rust-bin.nightly."2024-01-01".default +``` + +### 3. Include Helpful Shell Hooks + +```nix +shellHook = '' + echo "🦀 GRASP Audit development environment" + echo "" + echo "Common commands:" + echo " cargo build - Build project" + echo " cargo test - Run tests" + echo " cargo run - Run binary" + echo "" + export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc} +''; +``` + +### 4. Separate Build and Runtime Dependencies + +```nix +# Build-time only +nativeBuildInputs = [ + pkg-config + rustc + cargo +]; + +# Runtime needed +buildInputs = [ + openssl +]; +``` + +### 5. Disable Tests in Package Build + +```nix +packages.default = pkgs.rustPlatform.buildRustPackage { + # ... + doCheck = false; # Run tests separately with cargo test +}; +``` + +**Why:** Faster builds, tests run via `cargo test` in dev shell + +--- + +## Workflow Examples + +### Daily Development + +```bash +# Start work +cd grasp-audit +nix develop + +# Inside nix shell +cargo build +cargo test +cargo run -- --help + +# Exit shell +exit +``` + +### CI/CD + +```bash +# One-off commands (no interactive shell) +nix develop -c cargo build +nix develop -c cargo test --lib +nix develop -c cargo test -- --ignored +``` + +### Building Release + +```bash +# Build package directly +nix build + +# Result is in ./result/bin/ +./result/bin/grasp-audit --version +``` + +--- + +## References + +- **Nix Flakes Manual**: https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html +- **rust-overlay**: https://github.com/oxalica/rust-overlay +- **flake-utils**: https://github.com/numtide/flake-utils +- **Our Migration**: `docs/archive/2025-11-04-flake-migration.md` + +--- + +## Quick Reference + +| Task | Command | +|------|---------| +| Enter dev shell | `nix develop` | +| Run one command | `nix develop -c ` | +| Show outputs | `nix flake show` | +| Validate flake | `nix flake check` | +| Update inputs | `nix flake update` | +| Build package | `nix build` | +| Run package | `nix run` | + +--- + +*Last updated: November 4, 2025* +*Status: Living document - update as we learn more* diff --git a/docs/learnings/nostr-sdk.md b/docs/learnings/nostr-sdk.md new file mode 100644 index 0000000..57f451a --- /dev/null +++ b/docs/learnings/nostr-sdk.md @@ -0,0 +1,577 @@ +# nostr-sdk - Learnings and Patterns + +**Purpose:** Document nostr-sdk usage patterns, upgrade notes, and gotchas +**Last Updated:** November 4, 2025 + +--- + +## Current Version + +**We use nostr-sdk 0.43.x (latest stable)** + +```toml +[dependencies] +nostr-sdk = "0.43" +``` + +**Upgraded from:** 0.35.0 on November 4, 2025 + +--- + +## Critical Breaking Changes (0.35 → 0.43) + +### 1. EventBuilder API Changed + +**Before (0.35):** +```rust +let event = EventBuilder::new(kind, content, tags) + .to_event(keys)?; +``` + +**After (0.43):** +```rust +let event = EventBuilder::new(kind, content) + .tags(tags) + .sign_with_keys(keys)?; +``` + +**Changes:** +- ❌ Removed `tags` parameter from constructor +- ✅ Use `.tags()` builder method instead +- ❌ Removed `.to_event()` method +- ✅ Use `.sign_with_keys()` instead (more descriptive) + +--- + +### 2. Client Ownership of Keys + +**Before (0.35):** +```rust +let keys = Keys::generate(); +let client = Client::new(&keys); // Reference +// keys still available +``` + +**After (0.43):** +```rust +let keys = Keys::generate(); +let client = Client::new(keys.clone()); // Ownership +// Need to clone if we want to keep keys +``` + +**Why:** Allows Client to own the signer, enabling more flexible signer types. + +--- + +### 3. Relay Status Check No Longer Async + +**Before (0.35):** +```rust +if relay.is_connected().await { + // ... +} +``` + +**After (0.43):** +```rust +if relay.is_connected() { // No await! + // ... +} +``` + +**Why:** Status check doesn't require async operation. + +--- + +### 4. Query API Redesigned + +**Before (0.35):** +```rust +let events = client + .get_events_of(vec![filter], EventSource::relays(Some(timeout))) + .await?; +// Returns Vec +``` + +**After (0.43):** +```rust +let events = client + .fetch_events(filter, timeout) + .await?; +// Returns Events (iterable collection) + +// Convert to Vec if needed +let vec: Vec = events.into_iter().collect(); +``` + +**Changes:** +- ❌ Removed `get_events_of()` method +- ✅ Use `fetch_events()` instead +- ❌ Removed `EventSource` parameter (confusing) +- ✅ Direct timeout parameter +- ❌ Single filter instead of `Vec` +- ✅ Returns `Events` type instead of `Vec` + +--- + +### 5. Filter Custom Tags Simplified + +**Before (0.35):** +```rust +filter.custom_tag(tag, ["value"]) +filter.custom_tag(tag, [&string_ref]) +``` + +**After (0.43):** +```rust +filter.custom_tag(tag, "value") +filter.custom_tag(tag, &string_ref) +``` + +**Why:** Simplified API for the common case of single tag value. + +--- + +### 6. Send Event Takes Reference + +**Before (0.35):** +```rust +let event_id = client.send_event(event).await?; +``` + +**After (0.43):** +```rust +let output = client.send_event(&event).await?; +let event_id = *output.id(); +``` + +**Changes:** +- Takes `&Event` instead of `Event` (can reuse events) +- Returns `SendEventOutput` instead of `EventId` +- Need to call `.id()` to get the event ID + +--- + +## Common Patterns + +### Creating and Signing Events + +```rust +use nostr_sdk::prelude::*; + +// Generate keys +let keys = Keys::generate(); + +// Create event +let event = EventBuilder::new(Kind::TextNote, "Hello Nostr!") + .tags(vec![ + Tag::custom(TagKind::SingleLetter(SingleLetterTag::lowercase(Alphabet::T)), + vec!["nostr"]), + ]) + .sign_with_keys(&keys)?; + +// Send event +let output = client.send_event(&event).await?; +println!("Event ID: {}", output.id()); +``` + +--- + +### Creating Custom Tags + +```rust +use nostr_sdk::prelude::*; + +// Single letter tag (like "t" for topics) +let t_tag = SingleLetterTag::lowercase(Alphabet::T); +let tag = Tag::custom( + TagKind::SingleLetter(t_tag), + vec!["my-topic"] +); + +// Custom multi-letter tag +let tag = Tag::custom( + TagKind::Custom("custom-tag".to_string()), + vec!["value1", "value2"] +); + +// Hashtag (convenience method) +let tag = Tag::hashtag("nostr"); // Creates ["t", "nostr"] +``` + +--- + +### Querying Events + +```rust +use nostr_sdk::prelude::*; + +// Build filter +let filter = Filter::new() + .kind(Kind::TextNote) + .custom_tag( + SingleLetterTag::lowercase(Alphabet::T), + "my-topic" + ) + .since(Timestamp::now() - Duration::from_secs(3600)); // Last hour + +// Query events +let timeout = Duration::from_secs(10); +let events = client.fetch_events(filter, timeout).await?; + +// Process events +for event in events.into_iter() { + println!("Event: {}", event.id()); +} +``` + +--- + +### Multiple Filters + +Since `fetch_events()` takes a single filter, combine multiple queries: + +```rust +// Option 1: Fetch separately and combine +let mut all_events = Vec::new(); +for filter in filters { + let events = client.fetch_events(filter, timeout).await?; + all_events.extend(events.into_iter()); +} + +// Option 2: Use subscription (more efficient) +let subscription_id = SubscriptionId::new("my-sub"); +client.subscribe(filters, None).await?; + +// Handle events via notification handler +let mut notifications = client.notifications(); +while let Ok(notification) = notifications.recv().await { + if let RelayPoolNotification::Event { event, .. } = notification { + println!("Event: {}", event.id()); + } +} +``` + +--- + +### Client Setup with Relay + +```rust +use nostr_sdk::prelude::*; + +// Create keys +let keys = Keys::generate(); + +// Create client +let client = Client::new(keys.clone()); + +// Add relay +client.add_relay("wss://relay.example.com").await?; + +// Connect +client.connect().await; + +// Wait for connection +tokio::time::sleep(Duration::from_secs(2)).await; + +// Check connection +if client.relay("wss://relay.example.com") + .await? + .is_connected() +{ + println!("Connected!"); +} +``` + +--- + +## Testing Patterns + +### Unit Tests (No Relay Required) + +```rust +#[cfg(test)] +mod tests { + use super::*; + use nostr_sdk::prelude::*; + + #[test] + fn test_event_creation() { + let keys = Keys::generate(); + let event = EventBuilder::new(Kind::TextNote, "test") + .sign_with_keys(&keys) + .unwrap(); + + assert_eq!(event.kind(), Kind::TextNote); + assert_eq!(event.content(), "test"); + } + + #[test] + fn test_tag_creation() { + let t_tag = SingleLetterTag::lowercase(Alphabet::T); + let tag = Tag::custom( + TagKind::SingleLetter(t_tag), + vec!["test-topic"] + ); + + // Verify tag structure + assert_eq!(tag.as_vec()[0], "t"); + assert_eq!(tag.as_vec()[1], "test-topic"); + } +} +``` + +--- + +### Integration Tests (Relay Required) + +```rust +#[cfg(test)] +mod tests { + use super::*; + use nostr_sdk::prelude::*; + + #[tokio::test] + #[ignore] // Requires running relay + async fn test_send_and_receive() -> Result<()> { + // Setup + let keys = Keys::generate(); + let client = Client::new(keys.clone()); + client.add_relay("ws://localhost:7000").await?; + client.connect().await; + tokio::time::sleep(Duration::from_secs(2)).await; + + // Send event + let event = EventBuilder::new(Kind::TextNote, "test") + .sign_with_keys(&keys)?; + let output = client.send_event(&event).await?; + + // Query it back + let filter = Filter::new() + .id(*output.id()); + let events = client.fetch_events(filter, Duration::from_secs(5)).await?; + + assert_eq!(events.len(), 1); + Ok(()) + } +} +``` + +**Running integration tests:** +```bash +# Start relay first +docker run --rm -p 7000:7000 scsibug/nostr-rs-relay + +# Run tests +cargo test -- --ignored +``` + +--- + +## Common Gotchas + +### 1. Event Validation Failures + +**Problem:** Events fail validation with cryptic errors + +**Common Causes:** +- Invalid signature (wrong keys used) +- Invalid event ID (content/tags changed after signing) +- Invalid timestamp (too far in future/past) + +**Solution:** +```rust +// Always sign AFTER setting all fields +let event = EventBuilder::new(kind, content) + .tags(tags) // Set tags first + .sign_with_keys(&keys)?; // Sign last + +// Don't modify event after signing! +``` + +--- + +### 2. Filter Not Matching Events + +**Problem:** Query returns no events even though they exist + +**Common Causes:** +- Tag kind mismatch (uppercase vs lowercase) +- Wrong filter field (using `.author()` when you need `.authors()`) +- Timeout too short + +**Solution:** +```rust +// Be explicit about tag kinds +let t_tag = SingleLetterTag::lowercase(Alphabet::T); // Lowercase! + +// Use correct filter methods +let filter = Filter::new() + .authors(vec![keys.public_key()]) // Note: plural + .kinds(vec![Kind::TextNote]); // Note: plural + +// Increase timeout for slow relays +let timeout = Duration::from_secs(10); +``` + +--- + +### 3. Connection Timing Issues + +**Problem:** Events fail to send or queries return empty + +**Cause:** Client not fully connected to relay + +**Solution:** +```rust +// Connect +client.connect().await; + +// Wait for connection to establish +tokio::time::sleep(Duration::from_secs(2)).await; + +// Verify connection +let relay = client.relay("wss://relay.example.com").await?; +if !relay.is_connected() { + return Err("Not connected".into()); +} + +// Now safe to send/query +``` + +--- + +### 4. Clone Keys When Creating Client + +**Problem:** Can't use keys after creating client + +**Cause:** Client takes ownership in 0.43+ + +**Solution:** +```rust +// Clone keys if you need them later +let keys = Keys::generate(); +let client = Client::new(keys.clone()); // Clone! + +// Now can still use keys +let pubkey = keys.public_key(); +``` + +--- + +## Performance Tips + +### 1. Reuse Clients + +```rust +// ✅ Good - single client +let client = Client::new(keys); +client.add_relay("wss://relay1.com").await?; +client.add_relay("wss://relay2.com").await?; +client.connect().await; + +// ❌ Bad - multiple clients +for relay in relays { + let client = Client::new(keys.clone()); // Wasteful! + client.add_relay(relay).await?; +} +``` + +--- + +### 2. Use Subscriptions for Live Updates + +```rust +// ✅ Good for live updates - subscription +let filters = vec![Filter::new().kind(Kind::TextNote)]; +client.subscribe(filters, None).await?; + +let mut notifications = client.notifications(); +while let Ok(notification) = notifications.recv().await { + // Handle events as they arrive +} + +// ❌ Bad for live updates - polling +loop { + let events = client.fetch_events(filter, timeout).await?; + tokio::time::sleep(Duration::from_secs(1)).await; +} +``` + +--- + +### 3. Batch Event Creation + +```rust +// ✅ Good - reuse keys +let keys = Keys::generate(); +let events: Vec = (0..100) + .map(|i| { + EventBuilder::new(Kind::TextNote, format!("Message {}", i)) + .sign_with_keys(&keys) + .unwrap() + }) + .collect(); + +// ❌ Bad - regenerate keys +let events: Vec = (0..100) + .map(|i| { + let keys = Keys::generate(); // Wasteful! + EventBuilder::new(Kind::TextNote, format!("Message {}", i)) + .sign_with_keys(&keys) + .unwrap() + }) + .collect(); +``` + +--- + +## Migration Checklist (0.35 → 0.43) + +When upgrading from 0.35 to 0.43: + +- [ ] Update `Cargo.toml`: `nostr-sdk = "0.43"` +- [ ] Fix `EventBuilder::new()` - remove tags parameter +- [ ] Fix `EventBuilder::to_event()` → `sign_with_keys()` +- [ ] Fix `Client::new()` - clone keys instead of reference +- [ ] Fix `Relay::is_connected()` - remove `.await` +- [ ] Fix `Client::get_events_of()` → `fetch_events()` +- [ ] Remove `EventSource::relays()` usage +- [ ] Fix `Filter::custom_tag()` - single value instead of array +- [ ] Fix `Client::send_event()` - pass reference, handle `SendEventOutput` +- [ ] Update tests +- [ ] Verify all builds pass +- [ ] Run integration tests + +**Reference:** See `docs/archive/2025-11-04-nostr-sdk-upgrade.md` + +--- + +## Useful Resources + +- **nostr-sdk docs**: https://docs.rs/nostr-sdk/0.43.0 +- **rust-nostr GitHub**: https://github.com/rust-nostr/nostr +- **NIPs**: https://github.com/nostr-protocol/nips +- **NIP-01 (Events)**: https://github.com/nostr-protocol/nips/blob/master/01.md +- **NIP-34 (Git)**: https://github.com/nostr-protocol/nips/blob/master/34.md + +--- + +## Quick Reference + +| Task | Code | +|------|------| +| Create event | `EventBuilder::new(kind, content).sign_with_keys(&keys)?` | +| Add tags | `.tags(vec![tag1, tag2])` | +| Custom tag | `Tag::custom(TagKind::SingleLetter(t), vec!["value"])` | +| Create client | `Client::new(keys.clone())` | +| Add relay | `client.add_relay("wss://...").await?` | +| Connect | `client.connect().await` | +| Send event | `client.send_event(&event).await?` | +| Query events | `client.fetch_events(filter, timeout).await?` | +| Subscribe | `client.subscribe(filters, None).await?` | + +--- + +*Last updated: November 4, 2025* +*Status: Living document - update as nostr-sdk evolves*