ngit-grasp/8430-storage-limits-and-quota-management.md

# Storage Limits and Quota Management

**ID:** 8430

> **⚠️ COORDINATION REQUIRED:** This issue is part of the Administrator Observability and Management Strategy (issue ec1f).
> **Before starting work:** Check issue ec1f for current phase, dependencies, and coordination requirements.
> This issue is the **primary deliverable for Phase 4 (Enforcement)** and depends on Phase 3 (management API) completion.

## Issue Summary

We currently have no storage limits or quota enforcement, allowing users to push arbitrarily large repositories (e.g., 20GB+) and potentially fill the relay's disk. We need configurable storage limits at multiple levels and graceful handling when approaching disk capacity.

## Current State

- No limits on individual repository size
- No limits on total storage per user/npub
- No limits on total relay storage
- No graceful degradation when disk approaches full
- No warnings or alerts for storage usage
- Risk of disk exhaustion causing relay crash or data corruption

## Required Storage Controls

### 1. Repository-level limits
- Max size per individual repository
- Reject pushes that would exceed repo limit
- Clear error messages when limit hit
- Consider limits on individual objects (blob size, pack size)

### 2. User-level limits
- Max total storage per npub across all their repos
- Track aggregate storage usage per user
- Enforce quota on new pushes
- Consider different limits for maintainers vs contributors

### 3. Relay-level limits
- Max total git storage for the entire relay
- Max total database storage
- Reserved space threshold (stop accepting new data before disk 100% full)
- Emergency mode when critically low on space

### 4. Graceful degradation
- Warning thresholds (e.g., 80%, 90%, 95% full)
- Read-only mode when approaching limits
- Clear communication to clients about storage status
- Prevent complete disk exhaustion
- Allow emergency cleanup operations

### 5. Configurability
- All limits should be configurable
- Sensible defaults for different deployment sizes
- Ability to set per-user overrides (allowlist power users)
- Runtime configuration updates without restart

## Investigation Tasks

### Storage Tracking
- [ ] Identify where git repository data is stored
- [ ] Determine how to efficiently calculate repository sizes
- [ ] Design schema for tracking storage quotas
- [ ] Check if git provides built-in size calculation tools
- [ ] Research efficient incremental size tracking (avoid recalculating full repo size on every push)

### Limit Enforcement Points
- [ ] Identify where to enforce limits in git push flow
- [ ] Determine pre-receive vs post-receive hook approach
- [ ] Find where to check quotas before accepting data
- [ ] Handle partial pushes (some refs succeed, others rejected)

### Disk Space Monitoring
- [ ] Implement filesystem monitoring for available space
- [ ] Design alerting mechanism for low disk space
- [ ] Create read-only mode for emergency situations
- [ ] Determine how to communicate storage status to clients

### Configuration Design
- [ ] Design configuration schema for limits
- [ ] Decide on default values for different deployment sizes
- [ ] Create override mechanism for trusted users
- [ ] Document migration path for existing deployments

### Error Handling
- [ ] Design clear error messages for quota exceeded
- [ ] Provide actionable feedback (how much over limit, what to do)
- [ ] Handle edge cases (concurrent pushes, race conditions)
- [ ] Test cleanup after failed push due to quota

### Research Other Implementations
- [ ] Check how GitHub/GitLab handle repository size limits
- [ ] Review how git hosting platforms enforce quotas
- [ ] Look at cgit, gitolite, gitea quota implementations
- [ ] Research Nostr relay storage limit patterns (if applicable)

## Questions to Answer

1. **How to efficiently track storage usage?**
   - Calculate on-demand vs maintain running totals?
   - Track at object level or repository level?
   - How to handle git deduplication (shared objects)?

2. **What are reasonable default limits?**
   - Typical git repository sizes in practice
   - Expected usage patterns for GRASP
   - Balance between usability and resource protection

3. **How to handle existing oversized repos?**
   - Grandfather in existing large repos?
   - Force migration/cleanup?
   - Gradual enforcement strategy?

4. **What metrics should we expose?**
   - Per-user storage usage
   - Per-repo storage usage
   - Total relay storage usage
   - Available disk space

5. **How to communicate limits to users?**
   - NIP-11 relay info document?
   - Error messages during push?
   - Separate query endpoint for quota status?

## Implementation Considerations

### Git-specific challenges
- Git packs can be large but compress well
- Shared objects between repos (deduplication)
- Temporary space needed during push (may exceed final size)
- Repository size can decrease after GC/repack

### Performance
- Size calculations must be fast (don't block pushes)
- Consider caching calculated sizes
- Incremental updates when possible
- Background monitoring vs synchronous checks

### User experience
- Clear, actionable error messages
- Show current usage vs limit
- Suggest remediation (git gc, remove large files)
- Allow grace period for users to clean up

## Findings

*(To be filled in during investigation)*

## Implementation Plan

*(To be determined after investigation)*

## Success Criteria

- No user can fill relay disk unintentionally
- Graceful handling of disk space exhaustion
- Clear documentation of limits and quotas
- Monitoring and alerting for storage usage
- Configurable limits for different deployment sizes