feat(private-repos): attach GRASP-08 credentials to private outbound Git fetches

A private instance mirroring another private service could open the
authenticated WebSocket but had no way to fetch the git data behind the
peer's NIP-98 gate, so purgatory promotion of mirrored repositories
never completed. This is the missing half of zero-configuration private
mirroring.

Approach: a new Grasp08Peers registry (canonical host:port keys with
explicit ports, since loopback tests co-host several services on one
address) is populated by the sync manager whenever a session's NIP-11
advertises GRASP-08 and drained when it stops. The purgatory fetch path
consults it per URL and, for confirmed peers, attaches the GRASP-08
repository-root credential via `http.extraHeader` on the hardened git
command. Headers are minted fresh before each subprocess because the
peer enforces a 60-second validity window that a long batch fetch must
not outlive; signing failures warn once and degrade to unauthenticated
fetches. Both the registry and the signing keys exist only when the
server runs in private mode, so a public mirror can never present
credentials.

Correctness assumptions: `extract_domain` drops the port and is
therefore unsuitable as the registry key; `peer_key` derives host:port
from the URL itself on both the relay-URL (write) and clone-URL (read)
sides. The `http.extraHeader` addition does not conflict with the
`credential.helper=` hardening: that control excludes ambient operator
credentials, while this header is a peer-scoped credential minted for
exactly this vetted fetch target.

Validation: unit tests cover the ws/http key equivalence and a
credential round-trip through the inbound validator (including root
derivation past `.git` inside identifiers). The integration test runs
two private services end to end: the mirroring instance only serves the
state event after NIP-42 on the WebSocket AND the NIP-98 credential on
the git fetch both succeeded against the private source. cargo test
--lib private:: and --test private_mode
private_instance_syncs_from_private_peer_with_outbound_credentials pass.
This commit is contained in:
DanConwayDev
2026-08-15 14:34:01 +00:00
parent e3c3a73e6d
commit 502c74c975
8 changed files with 425 additions and 5 deletions
+2
View File
@@ -5,6 +5,8 @@
pub mod access;
pub mod nip98;
pub mod peers;
pub mod ws_auth;
pub use access::PrivateAccess;
pub use peers::Grasp08Peers;
+60 -1
View File
@@ -3,7 +3,7 @@ use std::time::{SystemTime, UNIX_EPOCH};
use base64::Engine;
use hyper::header::{AUTHORIZATION, WWW_AUTHENTICATE};
use hyper::{Request, Response, StatusCode};
use nostr_sdk::prelude::{Event, Kind};
use nostr_sdk::prelude::{Event, EventBuilder, FinalizeEvent, Keys, Kind, Tag};
use crate::config::Config;
use crate::git::{empty_body, GitResponseBody};
@@ -78,6 +78,38 @@ pub fn validate_request<B>(
Ok(())
}
/// Build the outbound Authorization header for a GRASP-08 peer fetch.
///
/// Signs exactly the profile [`validate_request`] checks: kind 27235 with one
/// `u` tag naming the repository root, one `method` tag of `GET`, and a fresh
/// timestamp (the peer allows 60 seconds of skew, so callers generate a new
/// header per subprocess invocation rather than caching one).
pub fn repository_credential_header(
keys: &Keys,
repository_root_url: &str,
) -> anyhow::Result<String> {
let event = EventBuilder::new(Kind::HttpAuth, "")
.tags([
Tag::parse(["u", repository_root_url])?,
Tag::parse(["method", "GET"])?,
])
.finalize(keys)?;
Ok(format!(
"Nostr {}",
base64::engine::general_purpose::STANDARD.encode(event.as_json())
))
}
/// Truncate a Smart HTTP fetch URL at the repository root the GRASP-08
/// credential must sign.
///
/// Identifiers may themselves contain `.git`, so like
/// [`canonical_repository_url`] the root ends at the LAST occurrence.
pub fn repository_root_from_fetch_url(url: &str) -> Option<String> {
let end = url.rfind(".git")?.checked_add(4)?;
url.get(..end).map(str::to_string)
}
/// Build the canonical absolute root URL signed by Git credentials.
pub fn canonical_repository_url(config: &Config, request_path: &str) -> Option<String> {
// Identifiers may themselves contain `.git`; the route suffix is the last
@@ -204,6 +236,33 @@ mod tests {
}
}
#[test]
fn outbound_credential_round_trips_through_validation() {
let keys = Keys::generate();
let access = PrivateAccess::new([keys.public_key()]);
let root = repository_root_from_fetch_url(
"http://127.0.0.1:7334/npub/repo.git/info/refs?service=git-upload-pack",
)
.expect("repository root");
assert_eq!(root, "http://127.0.0.1:7334/npub/repo.git");
assert_eq!(
repository_root_from_fetch_url("https://h.example/npub/repo.git-tools.git/info/refs")
.as_deref(),
Some("https://h.example/npub/repo.git-tools.git")
);
let header = repository_credential_header(&keys, &root).expect("credential header");
let request = Request::get("/npub/repo.git/info/refs")
.header(AUTHORIZATION, header)
.body(())
.unwrap();
assert_eq!(validate_request(&request, &root, &access), Ok(()));
// The same credential must not authorize a different repository root.
assert!(
validate_request(&request, "http://127.0.0.1:7334/npub/other.git", &access).is_err()
);
}
#[test]
fn canonical_url_uses_http_only_for_loopback() {
let mut config = Config::for_testing();
+93
View File
@@ -0,0 +1,93 @@
//! Registry of GRASP-08 private-service peers learned from NIP-11.
//!
//! A private instance attaches its outbound NIP-98 credential only to Git
//! fetches from hosts it has confirmed to be GRASP-08 private services, so
//! the credential is never presented to an ordinary public server.
use std::collections::HashSet;
use std::sync::{Arc, RwLock};
/// Shared set of confirmed GRASP-08 peers, keyed by canonical `host:port`.
///
/// The sync manager writes entries when a relay's NIP-11 advertises (or stops
/// advertising) GRASP-08; the purgatory Git fetch path reads them. Keys always
/// carry an explicit port - tests run several services on one loopback host,
/// so the host alone would collide.
#[derive(Clone, Debug, Default)]
pub struct Grasp08Peers {
inner: Arc<RwLock<HashSet<String>>>,
}
impl Grasp08Peers {
/// Record `key` as a confirmed GRASP-08 peer. Returns true when new.
pub fn insert(&self, key: String) -> bool {
self.inner
.write()
.expect("GRASP-08 peer set poisoned")
.insert(key)
}
/// Forget `key`. Returns true when it was present.
pub fn remove(&self, key: &str) -> bool {
self.inner
.write()
.expect("GRASP-08 peer set poisoned")
.remove(key)
}
/// Whether `key` is a confirmed GRASP-08 peer.
pub fn contains(&self, key: &str) -> bool {
self.inner
.read()
.expect("GRASP-08 peer set poisoned")
.contains(key)
}
/// Canonical `host:port` registry key for a ws/wss/http/https URL.
///
/// The same service is reached over WebSocket (relay URL) and HTTP
/// (clone URL); scheme-default ports are made explicit so both spellings
/// map to one key.
pub fn peer_key(url: &str) -> Option<String> {
let parsed = reqwest::Url::parse(url).ok()?;
let host = parsed.host_str()?.to_ascii_lowercase();
let port = parsed.port_or_known_default()?;
Some(format!("{host}:{port}"))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ws_and_http_urls_for_one_service_share_a_key() {
let key = Grasp08Peers::peer_key("ws://127.0.0.1:7334").expect("ws key");
assert_eq!(key, "127.0.0.1:7334");
assert_eq!(
Grasp08Peers::peer_key("http://127.0.0.1:7334/npub/repo.git").as_deref(),
Some("127.0.0.1:7334")
);
// Scheme-default ports become explicit for both transports.
assert_eq!(
Grasp08Peers::peer_key("wss://Relay.Example").as_deref(),
Some("relay.example:443")
);
assert_eq!(
Grasp08Peers::peer_key("https://relay.example/x.git").as_deref(),
Some("relay.example:443")
);
assert_eq!(Grasp08Peers::peer_key("not a url"), None);
}
#[test]
fn registry_tracks_insert_and_remove() {
let peers = Grasp08Peers::default();
assert!(!peers.contains("relay.example:443"));
assert!(peers.insert("relay.example:443".into()));
assert!(!peers.insert("relay.example:443".into()));
assert!(peers.contains("relay.example:443"));
assert!(peers.remove("relay.example:443"));
assert!(!peers.contains("relay.example:443"));
}
}
+86 -4
View File
@@ -273,6 +273,14 @@ pub struct RealSyncContext {
/// Outbound target policy applied before every event-directed git fetch
outbound_policy: OutboundTargetPolicy,
/// Confirmed GRASP-08 peers (by canonical `host:port`), fed by the sync
/// manager's NIP-11 fetches. Only set for private instances.
grasp08_peers: Option<crate::private::Grasp08Peers>,
/// Keys signing outbound GRASP-08 repository credentials. Only set for
/// private instances; fetches stay unauthenticated without them.
credential_keys: Option<nostr_sdk::prelude::Keys>,
}
impl RealSyncContext {
@@ -287,6 +295,9 @@ impl RealSyncContext {
/// * `write_policy` - Write policy for promotion-time recovery hooks
/// * `git_naughty_list` - Naughty list tracker for git remote domains
/// * `outbound_policy` - Policy vetting event-directed git fetch targets
/// * `grasp08_peers` - Confirmed GRASP-08 peer registry (private mode only)
/// * `credential_keys` - Keys signing outbound GRASP-08 credentials
/// (private mode only)
#[allow(clippy::too_many_arguments)]
pub fn new(
purgatory: Arc<Purgatory>,
@@ -297,6 +308,8 @@ impl RealSyncContext {
write_policy: Option<Nip34WritePolicy>,
git_naughty_list: Arc<NaughtyListTracker>,
outbound_policy: OutboundTargetPolicy,
grasp08_peers: Option<crate::private::Grasp08Peers>,
credential_keys: Option<nostr_sdk::prelude::Keys>,
) -> Self {
Self {
purgatory,
@@ -308,6 +321,8 @@ impl RealSyncContext {
git_naughty_list,
miss_memo: Arc::new(Mutex::new(HashMap::new())),
outbound_policy,
grasp08_peers,
credential_keys,
}
}
@@ -410,7 +425,19 @@ fn resolve_pin_entry(resolved: &ResolvedTarget) -> Option<String> {
/// of requests to event-directed servers;
/// - `http.curloptResolve` pins the vetted DNS answers so the fetch cannot be
/// re-bound to a different address between authorization and connection.
fn hardened_git_command(repo_path: &Path, resolve_pin: Option<&str>, args: &[String]) -> Command {
///
/// `auth_header` attaches an explicit `Authorization` header (the GRASP-08
/// repository credential for a confirmed private peer). This deliberately
/// does not conflict with the `credential.helper=` hardening: that control
/// keeps *ambient operator* credentials away from event-directed servers,
/// while this header is a peer-scoped credential minted for exactly this
/// fetch target.
fn hardened_git_command(
repo_path: &Path,
resolve_pin: Option<&str>,
auth_header: Option<&str>,
args: &[String],
) -> Command {
let mut command = Command::new("git");
command
.arg("-c")
@@ -422,6 +449,11 @@ fn hardened_git_command(repo_path: &Path, resolve_pin: Option<&str>, args: &[Str
if let Some(pin) = resolve_pin {
command.arg("-c").arg(format!("http.curloptResolve={pin}"));
}
if let Some(header) = auth_header {
command
.arg("-c")
.arg(format!("http.extraHeader=Authorization: {header}"));
}
command
.args(args)
.env("GIT_ALLOW_PROTOCOL", "http:https")
@@ -949,6 +981,41 @@ impl SyncContext for RealSyncContext {
let naughty_list = self.git_naughty_list.clone();
let miss_memo = self.miss_memo.clone();
// GRASP-08: fetches from a confirmed private peer carry the
// repository-root credential. The registry key is host:port derived
// from the URL itself (`extract_domain` drops the port, which would
// collide loopback services). Headers are minted fresh before each
// subprocess: the peer's 60-second validity window must not expire
// mid-pass on a long batch fetch.
let credential_signer = self.credential_keys.clone().filter(|_| {
self.grasp08_peers.as_ref().is_some_and(|peers| {
crate::private::Grasp08Peers::peer_key(&url).is_some_and(|key| peers.contains(&key))
})
});
let repository_root = credential_signer
.as_ref()
.and_then(|_| crate::private::nip98::repository_root_from_fetch_url(&url));
let mut credential_warning_logged = false;
let credential_url = url.clone();
let mut fresh_auth_header = move || -> Option<String> {
let keys = credential_signer.as_ref()?;
let root = repository_root.as_deref()?;
match crate::private::nip98::repository_credential_header(keys, root) {
Ok(header) => Some(header),
Err(error) => {
if !credential_warning_logged {
credential_warning_logged = true;
tracing::warn!(
url = %credential_url,
error = %error,
"Failed to sign GRASP-08 credential; fetching unauthenticated"
);
}
None
}
}
};
// Phase 1: compare the remote's advertised refs against our
// needs. Most needed OIDs are ref tips declared by state events
// (PR tips appear under `refs/nostr/<event-id>`), so the
@@ -957,7 +1024,12 @@ impl SyncContext for RealSyncContext {
// failed "not our ref" upload-pack round trip.
let ls_remote_args = vec!["ls-remote".to_string(), url.clone()];
let advertised = match run_observed_git_command(
hardened_git_command(&repo_path, resolve_pin.as_deref(), &ls_remote_args),
hardened_git_command(
&repo_path,
resolve_pin.as_deref(),
fresh_auth_header().as_deref(),
&ls_remote_args,
),
&domain,
"ls_remote",
role,
@@ -1022,7 +1094,12 @@ impl SyncContext for RealSyncContext {
args.extend(advertised_tips.iter().cloned());
match run_observed_git_command(
hardened_git_command(&repo_path, resolve_pin.as_deref(), &args),
hardened_git_command(
&repo_path,
resolve_pin.as_deref(),
fresh_auth_header().as_deref(),
&args,
),
&domain,
"fetch_batch",
role,
@@ -1087,7 +1164,12 @@ impl SyncContext for RealSyncContext {
oid.clone(),
];
match run_observed_git_command(
hardened_git_command(&repo_path, resolve_pin.as_deref(), &args),
hardened_git_command(
&repo_path,
resolve_pin.as_deref(),
fresh_auth_header().as_deref(),
&args,
),
&domain,
"fetch_residual",
role,
+23
View File
@@ -256,6 +256,26 @@ impl RelayServer {
// Start SyncManager for proactive sync (Phase 2: multi-relay support, Phase 3: health tracking)
// Even without bootstrap relay, SyncManager discovers relays from stored announcements
// Pass the already-registered sync metrics from Metrics to avoid duplicate registration
// GRASP-08 peer registry and credential signer exist only on private
// instances: a public mirror never authenticates its Git fetches.
let grasp08_peers = config
.private_mode
.then(crate::private::Grasp08Peers::default);
let outbound_credential_keys = if config.private_mode {
match config.relay_owner_keys() {
Ok(keys) => Some(keys),
Err(error) => {
warn!(
%error,
"Relay owner key unavailable; outbound GRASP-08 credentials disabled"
);
None
}
}
} else {
None
};
let sync_manager = SyncManager::new(
config.sync_bootstrap_relay_url.clone(),
config.domain.clone(),
@@ -266,6 +286,7 @@ impl RelayServer {
PathBuf::from(config.effective_git_data_path()),
metrics.as_ref().and_then(|m| m.sync_metrics().cloned()),
private_access.clone(),
grasp08_peers.clone(),
);
if config.sync_bootstrap_relay_url.is_some() {
@@ -397,6 +418,8 @@ impl RelayServer {
OutboundTargetPolicy {
allow_non_global: config.sync_allow_non_global_targets,
},
grasp08_peers,
outbound_credential_keys,
));
// Create throttle manager for rate limiting remote git servers
+25
View File
@@ -2469,6 +2469,12 @@ pub struct SyncManager {
proactive_participant_authors: crate::nostr::policy::SharedProactiveParticipantAuthorIndex,
/// GRASP-08 access shared with the inbound HTTP/WebSocket boundary.
private_access: Option<PrivateAccess>,
/// Confirmed GRASP-08 peers shared with the purgatory Git fetch path.
///
/// Only populated on private instances (the server passes `Some`): a
/// public mirror never presents credentials, and its GRASP-08 targets are
/// parked pre-dial instead.
grasp08_peers: Option<crate::private::Grasp08Peers>,
/// Operator-configured members form the permanent base of private access.
configured_private_members: HashSet<PublicKey>,
/// Latest NIP-11 owner learned for each connected repository relay whose
@@ -2586,6 +2592,7 @@ impl SyncManager {
data_path: PathBuf,
sync_metrics: Option<SyncMetrics>,
private_access: Option<PrivateAccess>,
grasp08_peers: Option<crate::private::Grasp08Peers>,
) -> Self {
// Extract purgatory from write_policy for read-only access
let purgatory = write_policy.purgatory().clone();
@@ -2638,6 +2645,7 @@ impl SyncManager {
root_candidate_index: Arc::new(RwLock::new(HashMap::new())),
proactive_participant_authors,
private_access,
grasp08_peers,
configured_private_members,
relay_owners: HashMap::new(),
relay_sync_index: Arc::new(RwLock::new(HashMap::new())),
@@ -6323,6 +6331,23 @@ impl SyncManager {
self.relay_owners.remove(&result.relay_url);
}
}
// On a private instance, remember which hosts are GRASP-08
// peers so purgatory Git fetches from them carry the outbound
// NIP-98 credential.
if let Some(peers) = &self.grasp08_peers {
if let Some(key) = crate::private::Grasp08Peers::peer_key(&result.relay_url) {
if advertised_grasp08 {
if peers.insert(key) {
tracing::info!(
relay = %result.relay_url,
"Confirmed GRASP-08 peer; Git fetches will carry credentials"
);
}
} else {
peers.remove(&key);
}
}
}
self.reconcile_private_membership().await;
if let Some(connection) = self.connections.get(&result.relay_url) {
connection.reset_subscription_budget(advertised_max_subscriptions);
+32
View File
@@ -269,6 +269,38 @@ impl TestRelay {
.await
}
/// Start a GRASP-08 private service with an explicit member list, an
/// optional sync bootstrap, and a caller-chosen relay-owner identity.
///
/// Lets private-to-private sync tests model two services whose member
/// sets deliberately differ (e.g. the source service admits the
/// mirroring service's owner identity).
pub async fn start_private_with_sync_owner_keys_and_members(
bootstrap_relay_url: Option<String>,
owner_keys: Keys,
members: &[nostr_sdk::prelude::PublicKey],
) -> Self {
let members = members
.iter()
.map(|member| {
member
.to_bech32()
.expect("Failed to encode private test member")
})
.collect::<Vec<_>>()
.join(",");
Self::start_internal(
port::reserve_port(),
RelayOptions {
bootstrap_relay_url,
owner_keys: Some(owner_keys),
private_members: Some(members),
..RelayOptions::default()
},
)
.await
}
/// Start a GRASP-08 private relay whose sole configured member is also
/// the relay-owner identity, with user-index relays configured.
///
+104
View File
@@ -643,6 +643,110 @@ async fn derived_membership_requires_grasp08_advertising_relay() {
private_peer.stop().await;
}
/// End-to-end private-to-private mirroring: the mirroring instance must
/// authenticate its WebSocket session with NIP-42 AND attach the GRASP-08
/// repository credential to its purgatory Git fetches. The state event is
/// only promoted (and served) once the git data arrived, which requires both
/// credentials to have worked against the private source.
#[tokio::test]
async fn private_instance_syncs_from_private_peer_with_outbound_credentials() {
let member = Keys::generate();
let a_owner = Keys::generate();
// Source service B admits the member and A's owner identity.
let relay_b = TestRelay::start_private_with_sync_owner_keys_and_members(
None,
Keys::generate(),
&[member.public_key(), a_owner.public_key()],
)
.await;
// Mirroring service A bootstraps from B with its own owner identity.
let relay_a = TestRelay::start_private_with_sync_owner_keys_and_members(
Some(relay_b.url().to_string()),
a_owner.clone(),
&[member.public_key()],
)
.await;
let identifier = "private-peer-sync";
let npub = member.public_key().to_bech32().expect("member npub");
let b_clone_url = format!("http://{}/{npub}/{identifier}.git", relay_b.domain());
// Both services must appear in clone AND relays tags for admission;
// each instance excludes its own domain from fetch targets, so A only
// ever fetches git data from B.
let clone_urls = vec![
b_clone_url.clone(),
format!("http://{}/{npub}/{identifier}.git", relay_a.domain()),
];
let relay_urls = vec![
format!("ws://{}", relay_b.domain()),
format!("ws://{}", relay_a.domain()),
];
let announcement = EventBuilder::new(Kind::GitRepoAnnouncement, "")
.tags(vec![
Tag::identifier(identifier),
Tag::custom("clone", clone_urls.clone()),
Tag::custom("relays", relay_urls.clone()),
])
.finalize(&member)
.expect("signed announcement");
let git_dir = tempfile::tempdir().expect("git temp dir");
let commit = create_test_repo_with_commit(git_dir.path(), CommitVariant::StateTest)
.expect("test repository");
let clone_url_refs: Vec<&str> = clone_urls.iter().map(String::as_str).collect();
let relay_url_refs: Vec<&str> = relay_urls.iter().map(String::as_str).collect();
let state_event = create_state_event(
&member,
identifier,
&[("main", &commit)],
&[],
&clone_url_refs,
&relay_url_refs,
)
.expect("state event");
// Publish to B through the member's authenticated session and push the
// git data with the member's inbound GRASP-08 credential.
let client = TestClient::new(relay_b.url(), member.clone())
.await
.expect("authenticated member client");
client
.send_event(&announcement)
.await
.expect("announcement admitted on B");
client
.send_event(&state_event)
.await
.expect("state event admitted on B");
push_to_relay_with_auth_header(
git_dir.path(),
&relay_b.domain(),
&npub,
identifier,
Some(&credential(&member, &b_clone_url)),
)
.expect("authenticated git push to B");
client.disconnect().await;
// Promotion on A requires the git fetch from B to have succeeded, which
// in turn requires the outbound NIP-98 credential; a generous deadline
// covers connect, sync, and the purgatory fetch pass.
assert!(
wait_for_event_as(
relay_a.url(),
&member,
state_event.id,
Duration::from_secs(120)
)
.await,
"state event must be promoted on the mirroring private instance"
);
relay_a.stop().await;
relay_b.stop().await;
}
#[tokio::test]
async fn private_websocket_bounds_invalid_authentication_attempts() {
let member = Keys::generate();