mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-06 11:48:24 +00:00
fix(marmot): a last-resort KeyPackage is not single-use
We publish every KeyPackage marked last resort, and then threw away its
private keys the moment one Welcome consumed it. Those two things cannot
both be true. OpenMLS is explicit about the contract — on the Welcome
path it deletes the consumed bundle only `if !key_package.last_resort()`
and otherwise logs "KeyPackage has a last-resort marker, not deleting" —
and MDK leans on it: it marks all of its own KeyPackages last resort,
caches the peer KeyPackage it resolved in its user directory, and invites
from that same cached copy every time after. So the first invite
addressed to us worked and every one after it died on "No matching
KeyPackageBundle", which is four of the interop harness's failures.
Consumed bundles now stay reachable when the KeyPackage says they may
be, bounded on both axes: at most eight of them, and never past the
KeyPackage's own not_after. That retention is the entire forward-secrecy
cost of the last-resort marker, and it is a cost we already accepted by
publishing the marker.
Two things had to be right for it to work at all:
- `isLastResort()` has to read both carriers. The MIP-era profile sets
MLS extension type 0x000A on the KeyPackage; the current profile —
the one we actually publish — carries a `last_resort_key_package`
component inside the KeyPackage-level app_data_dictionary. Reading
only the first made every KeyPackage we ship look single-use.
- The Welcome lookup has to trust the MLS refs over the Nostr "e" tag.
RFC 9420 addresses each EncryptedGroupSecrets to a KeyPackageRef and
the joiner takes the first it holds keys for; the "e" tag is a
routing hint an inviter can get wrong, and MDK gets it wrong exactly
here — it stamps the event id of its cached copy, which is stale the
moment we rotate. Refs first, tag as fallback.
The restore path also stopped throwing the whole snapshot away when the
eventId→slot index is empty. It drops the active bundles that index made
unreachable, and keeps the retained bundles (keyed by event id, always
reachable) and the named slot d-tags (a fresh d-tag would republish into
a new addressable slot and orphan the old one).
Harness: reset A's amy home and the relay database at the start of every
run, keeping the relay build. wnd already wiped B's and C's data dirs,
but A's store and the relay's events survived, and the leftovers are not
inert — a KeyPackage from an earlier run is still on the relay to be
invited with, and old kind:445 events still arrive undecryptable. That
drift alone accounted for tests 03 and 08. `--reuse-state` opts out and
`--tests "..."` runs a subset.
Interop: 10 → 14 of 17 passing. 05, 12, 14 and 15 (every "A never
received invite") now pass, as do 03 and 08.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
This commit is contained in:
@@ -9,7 +9,8 @@
|
||||
# any human prompts — all checks run to completion and the exit code
|
||||
# reflects pass/fail totals.
|
||||
#
|
||||
# Usage: ./marmot-interop-headless.sh [--port N] [--no-build]
|
||||
# Usage: ./marmot-interop-headless.sh [--port N] [--no-build] [--reuse-state]
|
||||
# [--tests "name ..."]
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
@@ -53,6 +54,16 @@ RELAY_DATA="$STATE_DIR/relay"
|
||||
RELAY_PORT="${RELAY_PORT:-8080}"
|
||||
RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"
|
||||
NO_BUILD=0
|
||||
# Every run starts from empty stores. wnd already wipes B's and C's data dirs
|
||||
# on each start, but A's amy home and the relay's SQLite file used to survive,
|
||||
# and the leftovers are not inert: a KeyPackage A published in an earlier run
|
||||
# is still on the relay for B to invite with, an old group's kind:445 events
|
||||
# still arrive and fail to decrypt, and A's cursors still say it has seen them.
|
||||
# That drift is what made tests 03 and 08 fail on a dirty tree and pass on a
|
||||
# clean one. Pass --reuse-state when you are deliberately debugging carry-over.
|
||||
RESET_STATE=1
|
||||
# Space-separated test function names; empty means the full suite below.
|
||||
ONLY_TESTS=""
|
||||
|
||||
# Required as of MDK 0.9.x. `validate_relay_url` accepts `wss://`
|
||||
# unconditionally but `ws://` only for a loopback host AND only behind this
|
||||
@@ -75,6 +86,8 @@ while [[ $# -gt 0 ]]; do
|
||||
--port) RELAY_PORT="$2"; RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"; shift ;;
|
||||
--host) RELAY_HOST="$2"; RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"; shift ;;
|
||||
--no-build) NO_BUILD=1 ;;
|
||||
--reuse-state) RESET_STATE=0 ;;
|
||||
--tests) ONLY_TESTS="$2"; shift ;;
|
||||
-h|--help)
|
||||
sed -n '3,14p' "${BASH_SOURCE[0]}" | sed 's/^# \?//'
|
||||
exit 0 ;;
|
||||
@@ -83,6 +96,12 @@ while [[ $# -gt 0 ]]; do
|
||||
shift
|
||||
done
|
||||
|
||||
if [[ $RESET_STATE -eq 1 && -d "$STATE_DIR" ]]; then
|
||||
# Keep the relay checkout + its build (minutes to rebuild) and the log and
|
||||
# results history; drop everything that holds protocol state.
|
||||
rm -rf "$STATE_DIR/.amy" "$B_DIR" "$C_DIR" "$RELAY_DATA"
|
||||
fi
|
||||
|
||||
mkdir -p "$STATE_DIR" "$LOG_DIR" "$B_DIR/logs" "$C_DIR/logs"
|
||||
: >"$LOG_FILE"
|
||||
: >"$RESULTS_FILE"
|
||||
@@ -129,20 +148,37 @@ ensure_identity B
|
||||
ensure_identity C
|
||||
configure_relays
|
||||
|
||||
test_01_keypackage_discovery
|
||||
test_02_a_creates_group
|
||||
test_03_b_creates_group
|
||||
test_04_three_member_group
|
||||
test_05_b_adds_a_existing
|
||||
test_06_member_removal
|
||||
test_07_metadata_rename
|
||||
test_08_admin_promote_demote
|
||||
test_17_group_image_commit
|
||||
test_09_reply_react_unreact
|
||||
test_10_concurrent_commits
|
||||
test_11_leave_group
|
||||
test_12_offline_catchup
|
||||
test_13_keypackage_rotation
|
||||
test_14_wn_removes_a
|
||||
test_15_wn_member_leaves
|
||||
test_16_wn_keypackage_rotation
|
||||
ALL_TESTS=(
|
||||
test_01_keypackage_discovery
|
||||
test_02_a_creates_group
|
||||
test_03_b_creates_group
|
||||
test_04_three_member_group
|
||||
test_05_b_adds_a_existing
|
||||
test_06_member_removal
|
||||
test_07_metadata_rename
|
||||
test_08_admin_promote_demote
|
||||
test_17_group_image_commit
|
||||
test_09_reply_react_unreact
|
||||
test_10_concurrent_commits
|
||||
test_11_leave_group
|
||||
test_12_offline_catchup
|
||||
test_13_keypackage_rotation
|
||||
test_14_wn_removes_a
|
||||
test_15_wn_member_leaves
|
||||
test_16_wn_keypackage_rotation
|
||||
)
|
||||
|
||||
# --tests runs a subset in the order given. Most tests read state a previous
|
||||
# one saved (GROUP_02, GROUP_05, …), so a subset that skips a producer will
|
||||
# report `skip`, not a false failure.
|
||||
if [[ -n "$ONLY_TESTS" ]]; then
|
||||
read -r -a ALL_TESTS <<<"$ONLY_TESTS"
|
||||
fi
|
||||
|
||||
for t in "${ALL_TESTS[@]}"; do
|
||||
if ! declare -F "$t" >/dev/null; then
|
||||
fail_msg "unknown test: $t"
|
||||
continue
|
||||
fi
|
||||
"$t"
|
||||
done
|
||||
|
||||
+54
-16
@@ -33,6 +33,8 @@ import com.vitorpamplona.quartz.marmot.mls.framing.PublicMessage
|
||||
import com.vitorpamplona.quartz.marmot.mls.framing.WireFormat
|
||||
import com.vitorpamplona.quartz.marmot.mls.group.MlsGroupManager
|
||||
import com.vitorpamplona.quartz.marmot.mls.group.MlsGroupState
|
||||
import com.vitorpamplona.quartz.marmot.mls.messages.KeyPackageBundle
|
||||
import com.vitorpamplona.quartz.marmot.mls.messages.Welcome
|
||||
import com.vitorpamplona.quartz.marmot.protocolCore.ConvergenceAdmission
|
||||
import com.vitorpamplona.quartz.marmot.protocolCore.ConvergenceResolution
|
||||
import com.vitorpamplona.quartz.marmot.protocolCore.ConvergenceStatus
|
||||
@@ -41,6 +43,7 @@ import com.vitorpamplona.quartz.marmot.protocolCore.MarmotConvergenceEngine
|
||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
||||
import com.vitorpamplona.quartz.nip01Core.core.toHexKey
|
||||
import com.vitorpamplona.quartz.utils.Log
|
||||
import com.vitorpamplona.quartz.utils.sha256.sha256
|
||||
import kotlinx.coroutines.sync.Mutex
|
||||
import kotlinx.coroutines.sync.withLock
|
||||
@@ -380,7 +383,7 @@ class MarmotInboundProcessor(
|
||||
hintNostrGroupId: HexKey? = null,
|
||||
): WelcomeResult =
|
||||
try {
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.d("MarmotDbg") {
|
||||
"MarmotInboundProcessor.processWelcome: hint=${hintNostrGroupId?.take(8)} eventId=${welcomeEvent.id.take(8)}…"
|
||||
}
|
||||
@@ -390,7 +393,7 @@ class MarmotInboundProcessor(
|
||||
if (keyPackageEventId == null) {
|
||||
return WelcomeResult.Error("WelcomeEvent missing KeyPackage event ID tag")
|
||||
}
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.d("MarmotDbg") {
|
||||
"MarmotInboundProcessor.processWelcome: welcomeBytes=${welcomeBytes.size}B looking up KeyPackage by ref=${keyPackageEventId.take(8)}…"
|
||||
}
|
||||
@@ -404,23 +407,28 @@ class MarmotInboundProcessor(
|
||||
// log a noisy "No matching KeyPackageBundle" warning for what
|
||||
// is actually a benign replay.
|
||||
if (hintNostrGroupId != null && groupManager.isMember(hintNostrGroupId)) {
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.d("MarmotDbg") {
|
||||
"MarmotInboundProcessor.processWelcome: already a member of group=${hintNostrGroupId.take(8)}… — treating Welcome as replay"
|
||||
}
|
||||
return WelcomeResult.AlreadyJoined(hintNostrGroupId)
|
||||
}
|
||||
|
||||
// Find the KeyPackageBundle that was consumed.
|
||||
// Find the KeyPackageBundle the inviter encrypted to.
|
||||
//
|
||||
// The Welcome's "e" tag carries the *Nostr event id* of the
|
||||
// kind:30443 event (NOT the MLS reference hash), so we must
|
||||
// resolve it via the eventId→slot index that
|
||||
// [MarmotManager.generateKeyPackageEvent] populates after
|
||||
// signing each KeyPackageEvent.
|
||||
val bundle = keyPackageRotationManager.findBundleByEventId(keyPackageEventId)
|
||||
// The authority is the MLS Welcome itself: each EncryptedGroupSecrets
|
||||
// is addressed to a KeyPackageRef, and RFC 9420 says a joiner takes
|
||||
// the first one it holds private keys for — which is exactly what
|
||||
// OpenMLS does. The Welcome's "e" tag carries only the *Nostr event
|
||||
// id* of the kind:30443 event, which is a routing hint an inviter can
|
||||
// get wrong: MDK stamps the event id of the copy cached in its user
|
||||
// directory, so a peer that rotated its published KeyPackage while
|
||||
// MDK kept inviting from cache would be unjoinable if we trusted the
|
||||
// tag alone. Try the refs first, then fall back to the tag.
|
||||
val bundle =
|
||||
findBundleForWelcome(welcomeBytes) ?: keyPackageRotationManager.findBundleByEventId(keyPackageEventId)
|
||||
if (bundle == null) {
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.w("MarmotDbg") {
|
||||
"MarmotInboundProcessor.processWelcome: NO matching KeyPackageBundle for eventId=${keyPackageEventId.take(8)}… " +
|
||||
"— inviter referenced a KeyPackage we don't have private keys for. " +
|
||||
@@ -431,17 +439,19 @@ class MarmotInboundProcessor(
|
||||
"No matching KeyPackageBundle found for event $keyPackageEventId",
|
||||
)
|
||||
}
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.d("MarmotDbg") { "MarmotInboundProcessor.processWelcome: bundle found — invoking groupManager.processWelcome" }
|
||||
|
||||
// Join the group; nostrGroupId is derived from the MLS GroupContext's
|
||||
// NostrGroupData extension. The h-tag hint (if any) is validated inside.
|
||||
val (_, nostrGroupId) = groupManager.processWelcome(welcomeBytes, bundle, hintNostrGroupId)
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.d("MarmotDbg") { "MarmotInboundProcessor.processWelcome: joined group=${nostrGroupId.take(8)}…" }
|
||||
|
||||
// Mark the KeyPackage as consumed — triggers rotation
|
||||
keyPackageRotationManager.markConsumedByEventId(keyPackageEventId)
|
||||
// Mark the KeyPackage as consumed — triggers rotation. Keyed on the
|
||||
// bundle we actually used, not the "e" tag, for the same reason the
|
||||
// lookup above is.
|
||||
keyPackageRotationManager.markConsumedByRef(bundle.keyPackage.reference())
|
||||
|
||||
// Seed convergence with the joined state, so the very first inbound
|
||||
// commit already has a retained parent to fall back to.
|
||||
@@ -452,11 +462,39 @@ class MarmotInboundProcessor(
|
||||
needsKeyPackageRotation = keyPackageRotationManager.needsRotation(),
|
||||
)
|
||||
} catch (e: Exception) {
|
||||
com.vitorpamplona.quartz.utils.Log
|
||||
Log
|
||||
.w("MarmotDbg", "MarmotInboundProcessor.processWelcome: exception ${e.message}", e)
|
||||
WelcomeResult.Error("Failed to process Welcome: ${e.message}", e)
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the KeyPackageBundle a Welcome is addressed to, the way RFC 9420
|
||||
* §12.4.3.1 (and OpenMLS) does it: walk the Welcome's EncryptedGroupSecrets
|
||||
* in order and take the first `new_member` KeyPackageRef we hold private
|
||||
* keys for.
|
||||
*
|
||||
* Returns null when the Welcome does not parse or names no KeyPackage of
|
||||
* ours — the caller then falls back to the Nostr "e" tag hint, and reports
|
||||
* the failure if that misses too.
|
||||
*/
|
||||
private suspend fun findBundleForWelcome(welcomeBytes: ByteArray): KeyPackageBundle? {
|
||||
val welcome =
|
||||
try {
|
||||
val mlsMessage = MlsMessage.decodeTls(TlsReader(welcomeBytes))
|
||||
require(mlsMessage.wireFormat == WireFormat.WELCOME) { "not a Welcome wire format" }
|
||||
Welcome.decodeTls(TlsReader(mlsMessage.payload))
|
||||
} catch (e: Exception) {
|
||||
Log.d("MarmotDbg") {
|
||||
"MarmotInboundProcessor.findBundleForWelcome: welcome did not parse (${e.message}) — falling back to the e tag"
|
||||
}
|
||||
return null
|
||||
}
|
||||
for (secret in welcome.secrets) {
|
||||
keyPackageRotationManager.findBundleByRef(secret.newMember)?.let { return it }
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark a kind:445 event id as already processed so that a later relay
|
||||
* echo of the same event is treated as a [GroupEventResult.Duplicate]
|
||||
|
||||
+151
-35
@@ -53,9 +53,12 @@ import kotlinx.coroutines.sync.withLock
|
||||
* - Handle periodic rotation for long-lived KeyPackages
|
||||
*
|
||||
* After a KeyPackage is consumed by a Welcome message:
|
||||
* 1. The init_key is effectively spent — cannot be reused
|
||||
* 2. A new KeyPackage MUST be published to the same d-tag slot
|
||||
* 3. The old KeyPackageBundle MUST be discarded
|
||||
* 1. A new KeyPackage MUST be published to the same d-tag slot
|
||||
* 2. Whether the old bundle may be discarded depends on the LastResort marker:
|
||||
* a plain KeyPackage is single-use and its private keys are dropped, while a
|
||||
* KeyPackage carrying `0x000A` is reusable by contract and its bundle is
|
||||
* kept (bounded) in [retainedBundles] so a later Welcome addressed to the
|
||||
* same KeyPackage still joins.
|
||||
*
|
||||
* Per MIP-00 spec, each user should maintain up to [KeyPackageUtils.MAX_SLOTS]
|
||||
* KeyPackage slots, rotating consumed ones promptly.
|
||||
@@ -87,6 +90,24 @@ class KeyPackageRotationManager(
|
||||
*/
|
||||
private val eventIdToSlot = mutableMapOf<String, String>()
|
||||
|
||||
/**
|
||||
* Consumed KeyPackages we deliberately keep the private keys for, keyed by
|
||||
* the Nostr event id (kind:30443) they were published as.
|
||||
*
|
||||
* A KeyPackage carrying the LastResort marker (`0x000A`) is not single-use:
|
||||
* OpenMLS skips `delete_key_package` for one, so MDK — which marks every
|
||||
* KeyPackage last-resort and caches the peer KeyPackage it resolved — will
|
||||
* happily address a second, third and fourth Welcome to the same
|
||||
* KeyPackage of ours. Dropping the bundle after the first Welcome made all
|
||||
* of those unjoinable.
|
||||
*
|
||||
* Retention is bounded on both axes: at most [MAX_RETAINED_BUNDLES] entries
|
||||
* (oldest evicted first), and never past the KeyPackage's own `not_after`
|
||||
* lifetime. That is the whole forward-secrecy cost of the last-resort
|
||||
* marker, and it is the cost we already accepted by publishing it.
|
||||
*/
|
||||
private val retainedBundles = mutableMapOf<String, KeyPackageBundle>()
|
||||
|
||||
/**
|
||||
* Restore previously persisted bundles + rotation state from [store].
|
||||
* Call once at startup before any other use of this manager.
|
||||
@@ -125,36 +146,41 @@ class KeyPackageRotationManager(
|
||||
return
|
||||
}
|
||||
|
||||
// v2 snapshot: if bundles were restored but the eventId
|
||||
// index is empty (upgrade corner case, or a corrupted save),
|
||||
// the bundles are effectively unreachable — wipe them too
|
||||
// so a fresh publish happens.
|
||||
if (decoded.bundles.isNotEmpty() && decoded.eventIdToSlot.isEmpty()) {
|
||||
// Active bundles are reachable only through the eventId→slot
|
||||
// index, so a snapshot that has bundles but no index (upgrade
|
||||
// corner case, a corrupted save, or a rotation that never got as
|
||||
// far as publishing) can't serve a Welcome. Drop just those
|
||||
// bundles so `hasActiveKeyPackages()` is false and a fresh publish
|
||||
// happens. Everything else in the snapshot stays: the retained
|
||||
// last-resort bundles are keyed by event id directly and are still
|
||||
// the only way to join an invite sent from a peer's cache, and the
|
||||
// named slot d-tags have to stay stable or the republish lands in
|
||||
// a new addressable slot and orphans the old one.
|
||||
val unreachableActiveBundles = decoded.bundles.isNotEmpty() && decoded.eventIdToSlot.isEmpty()
|
||||
if (unreachableActiveBundles) {
|
||||
Log.w("KeyPackageRotationManager") {
|
||||
"Restored ${decoded.bundles.size} bundle(s) but no eventId→slot mapping — discarding, will republish"
|
||||
"Restored ${decoded.bundles.size} bundle(s) but no eventId→slot mapping — dropping them, will republish"
|
||||
}
|
||||
try {
|
||||
store.delete()
|
||||
} catch (e: Exception) {
|
||||
Log.w("KeyPackageRotationManager", "Failed to delete stale snapshot", e)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
mutex.withLock {
|
||||
activeBundles.clear()
|
||||
activeBundles.putAll(decoded.bundles)
|
||||
if (!unreachableActiveBundles) activeBundles.putAll(decoded.bundles)
|
||||
pendingRotations.clear()
|
||||
pendingRotations.addAll(decoded.pending)
|
||||
eventIdToSlot.clear()
|
||||
eventIdToSlot.putAll(decoded.eventIdToSlot)
|
||||
namedSlotDTags.clear()
|
||||
namedSlotDTags.putAll(decoded.namedSlotDTags)
|
||||
retainedBundles.clear()
|
||||
retainedBundles.putAll(decoded.retainedBundles)
|
||||
if (unreachableActiveBundles) persistUnlocked()
|
||||
}
|
||||
Log.d("KeyPackageRotationManager") {
|
||||
"Restored ${decoded.bundles.size} active KeyPackage bundle(s), " +
|
||||
"${decoded.pending.size} pending rotation, ${decoded.eventIdToSlot.size} eventId mapping(s), " +
|
||||
"${decoded.namedSlotDTags.size} named slot d-tag(s)"
|
||||
"${decoded.namedSlotDTags.size} named slot d-tag(s), " +
|
||||
"${decoded.retainedBundles.size} retained last-resort bundle(s)"
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w("KeyPackageRotationManager", "Failed to decode persisted KeyPackages", e)
|
||||
@@ -173,6 +199,7 @@ class KeyPackageRotationManager(
|
||||
pendingRotations.clear()
|
||||
eventIdToSlot.clear()
|
||||
namedSlotDTags.clear()
|
||||
retainedBundles.clear()
|
||||
val store = store ?: return@withLock
|
||||
try {
|
||||
store.delete()
|
||||
@@ -186,6 +213,7 @@ class KeyPackageRotationManager(
|
||||
val pending: Set<String>,
|
||||
val eventIdToSlot: Map<String, String>,
|
||||
val namedSlotDTags: Map<String, String>,
|
||||
val retainedBundles: Map<String, KeyPackageBundle>,
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -222,6 +250,15 @@ class KeyPackageRotationManager(
|
||||
writer.putOpaque2(name.encodeToByteArray())
|
||||
writer.putOpaque2(dTag.encodeToByteArray())
|
||||
}
|
||||
// consumed-but-reusable last-resort bundles, by event id (added in v5)
|
||||
writer.putUint32(retainedBundles.size.toLong())
|
||||
for ((eventId, bundle) in retainedBundles) {
|
||||
writer.putOpaque2(eventId.encodeToByteArray())
|
||||
writer.putOpaque4(bundle.keyPackage.toTlsBytes())
|
||||
writer.putOpaque2(bundle.initPrivateKey)
|
||||
writer.putOpaque2(bundle.encryptionPrivateKey)
|
||||
writer.putOpaque2(bundle.signaturePrivateKey)
|
||||
}
|
||||
return writer.toByteArray()
|
||||
}
|
||||
|
||||
@@ -270,7 +307,19 @@ class KeyPackageRotationManager(
|
||||
namedSlots[name] = dTag
|
||||
}
|
||||
}
|
||||
return Snapshot(bundles, pending, eventIdMap, namedSlots)
|
||||
val retained = mutableMapOf<String, KeyPackageBundle>()
|
||||
if (reader.hasRemaining) {
|
||||
val numRetained = reader.readUint32().toInt()
|
||||
repeat(numRetained) {
|
||||
val eventId = reader.readOpaque2().decodeToString()
|
||||
val keyPackage = MlsKeyPackage.decodeTls(TlsReader(reader.readOpaque4()))
|
||||
val initPriv = reader.readOpaque2()
|
||||
val encPriv = reader.readOpaque2()
|
||||
val sigPriv = reader.readOpaque2()
|
||||
retained[eventId] = KeyPackageBundle(keyPackage, initPriv, encPriv, sigPriv)
|
||||
}
|
||||
}
|
||||
return Snapshot(bundles, pending, eventIdMap, namedSlots, retained)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -393,8 +442,11 @@ class KeyPackageRotationManager(
|
||||
*/
|
||||
suspend fun findBundleByRef(keyPackageRef: ByteArray): KeyPackageBundle? =
|
||||
mutex.withLock {
|
||||
pruneExpiredRetainedUnlocked()
|
||||
activeBundles.values.find { bundle ->
|
||||
bundle.keyPackage.reference().contentEquals(keyPackageRef)
|
||||
} ?: retainedBundles.values.find { bundle ->
|
||||
bundle.keyPackage.reference().contentEquals(keyPackageRef)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -408,8 +460,12 @@ class KeyPackageRotationManager(
|
||||
*/
|
||||
suspend fun findBundleByEventId(eventId: HexKey): KeyPackageBundle? =
|
||||
mutex.withLock {
|
||||
val slot = eventIdToSlot[eventId] ?: return@withLock null
|
||||
activeBundles[slot]
|
||||
pruneExpiredRetainedUnlocked()
|
||||
val slot = eventIdToSlot[eventId]
|
||||
if (slot != null) {
|
||||
activeBundles[slot]?.let { return@withLock it }
|
||||
}
|
||||
retainedBundles[eventId]
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -434,11 +490,7 @@ class KeyPackageRotationManager(
|
||||
*/
|
||||
suspend fun markConsumed(dTagSlot: String) =
|
||||
mutex.withLock {
|
||||
activeBundles.remove(dTagSlot)
|
||||
// Drop any eventId mappings that pointed at this slot.
|
||||
val staleEventIds = eventIdToSlot.entries.filter { it.value == dTagSlot }.map { it.key }
|
||||
staleEventIds.forEach { eventIdToSlot.remove(it) }
|
||||
pendingRotations.add(dTagSlot)
|
||||
consumeSlotUnlocked(dTagSlot)
|
||||
persistUnlocked()
|
||||
}
|
||||
|
||||
@@ -452,11 +504,7 @@ class KeyPackageRotationManager(
|
||||
bundle.keyPackage.reference().contentEquals(keyPackageRef)
|
||||
}
|
||||
if (entry != null) {
|
||||
val consumedSlot = entry.key
|
||||
activeBundles.remove(consumedSlot)
|
||||
val staleEventIds = eventIdToSlot.entries.filter { it.value == consumedSlot }.map { it.key }
|
||||
staleEventIds.forEach { eventIdToSlot.remove(it) }
|
||||
pendingRotations.add(consumedSlot)
|
||||
consumeSlotUnlocked(entry.key)
|
||||
persistUnlocked()
|
||||
}
|
||||
}
|
||||
@@ -468,13 +516,72 @@ class KeyPackageRotationManager(
|
||||
suspend fun markConsumedByEventId(eventId: HexKey) =
|
||||
mutex.withLock {
|
||||
val slot = eventIdToSlot[eventId] ?: return@withLock
|
||||
activeBundles.remove(slot)
|
||||
val staleEventIds = eventIdToSlot.entries.filter { it.value == slot }.map { it.key }
|
||||
staleEventIds.forEach { eventIdToSlot.remove(it) }
|
||||
pendingRotations.add(slot)
|
||||
consumeSlotUnlocked(slot)
|
||||
persistUnlocked()
|
||||
}
|
||||
|
||||
/**
|
||||
* Retire the bundle in [dTagSlot] and schedule the slot for a fresh
|
||||
* publication. Caller must hold the mutex.
|
||||
*
|
||||
* A KeyPackage that advertises LastResort (`0x000A`) is reusable by
|
||||
* contract — OpenMLS keeps its bundle on the Welcome path and MDK invites
|
||||
* from a cached copy — so its private keys move to [retainedBundles],
|
||||
* still reachable by every event id that published it. Anything else is
|
||||
* single-use: the keys go, so a compromise later cannot reopen the Welcome
|
||||
* that consumed them.
|
||||
*/
|
||||
private fun consumeSlotUnlocked(dTagSlot: String) {
|
||||
val consumed = activeBundles.remove(dTagSlot)
|
||||
val staleEventIds = eventIdToSlot.entries.filter { it.value == dTagSlot }.map { it.key }
|
||||
if (consumed != null && consumed.keyPackage.isLastResort()) {
|
||||
for (staleEventId in staleEventIds) {
|
||||
// Re-insert so the most recently consumed entry sorts last and
|
||||
// survives eviction the longest.
|
||||
retainedBundles.remove(staleEventId)
|
||||
retainedBundles[staleEventId] = consumed
|
||||
}
|
||||
pruneExpiredRetainedUnlocked()
|
||||
while (retainedBundles.size > MAX_RETAINED_BUNDLES) {
|
||||
retainedBundles.remove(retainedBundles.keys.first())
|
||||
}
|
||||
}
|
||||
staleEventIds.forEach { eventIdToSlot.remove(it) }
|
||||
pendingRotations.add(dTagSlot)
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop retained bundles whose KeyPackage is past its own `not_after`.
|
||||
* No peer may invite with an expired KeyPackage, so holding its private
|
||||
* keys buys nothing. Caller must hold the mutex.
|
||||
*/
|
||||
private fun pruneExpiredRetainedUnlocked() {
|
||||
val now = TimeUtils.now()
|
||||
val expired =
|
||||
retainedBundles.entries
|
||||
.filter { (_, bundle) ->
|
||||
val notAfter =
|
||||
bundle.keyPackage.leafNode.lifetime
|
||||
?.notAfter ?: return@filter false
|
||||
now > notAfter
|
||||
}.map { it.key }
|
||||
expired.forEach { retainedBundles.remove(it) }
|
||||
}
|
||||
|
||||
/**
|
||||
* Install [bundle] as the active bundle for [dTagSlot], replacing whatever
|
||||
* was there. Used by callers that mint a KeyPackage themselves (tests, and
|
||||
* any flow that builds a bundle outside this manager) and still want the
|
||||
* manager to own its lifecycle.
|
||||
*/
|
||||
suspend fun installBundle(
|
||||
dTagSlot: String,
|
||||
bundle: KeyPackageBundle,
|
||||
) = mutex.withLock {
|
||||
activeBundles[dTagSlot] = bundle
|
||||
persistUnlocked()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the d-tag slots that need rotation (KeyPackage was consumed).
|
||||
*/
|
||||
@@ -577,13 +684,22 @@ class KeyPackageRotationManager(
|
||||
/** Proactive rotation after 7 days even if not consumed */
|
||||
const val MAX_KEY_PACKAGE_AGE_SECONDS = 7L * 24 * 60 * 60
|
||||
|
||||
/**
|
||||
* How many consumed last-resort bundles to keep private keys for.
|
||||
* Each one is a KeyPackage a peer may still be inviting us with from
|
||||
* its own cache; the bound keeps a long-lived account from carrying
|
||||
* every init key it ever published.
|
||||
*/
|
||||
const val MAX_RETAINED_BUNDLES = 8
|
||||
|
||||
/**
|
||||
* On-disk snapshot format version for [KeyPackageBundleStore].
|
||||
* v1: bundles + pendingRotations
|
||||
* v2: + eventIdToSlot map (so welcome lookup by Nostr event id works)
|
||||
* v3: + namedSlotDTags map (per MIP-00, d-tags are random 64-char hex, persisted here)
|
||||
* v4: capabilities fixed (0xF2EE, 0x000A extensions + 0x000A proposals; LastResort extension on KP)
|
||||
* v5: + retainedBundles map (consumed last-resort KeyPackages stay reusable)
|
||||
*/
|
||||
private const val SNAPSHOT_VERSION = 4
|
||||
private const val SNAPSHOT_VERSION = 5
|
||||
}
|
||||
}
|
||||
|
||||
+27
@@ -20,9 +20,11 @@
|
||||
*/
|
||||
package com.vitorpamplona.quartz.marmot.mls.messages
|
||||
|
||||
import com.vitorpamplona.quartz.marmot.appComponents.AppComponentIds
|
||||
import com.vitorpamplona.quartz.marmot.mls.codec.TlsReader
|
||||
import com.vitorpamplona.quartz.marmot.mls.codec.TlsSerializable
|
||||
import com.vitorpamplona.quartz.marmot.mls.codec.TlsWriter
|
||||
import com.vitorpamplona.quartz.marmot.mls.components.AppDataDictionary
|
||||
import com.vitorpamplona.quartz.marmot.mls.crypto.MlsCryptoProvider
|
||||
import com.vitorpamplona.quartz.marmot.mls.tree.Extension
|
||||
import com.vitorpamplona.quartz.marmot.mls.tree.LeafNode
|
||||
@@ -74,6 +76,25 @@ data class MlsKeyPackage(
|
||||
return MlsCryptoProvider.refHash("MLS 1.0 KeyPackage Reference", encoded)
|
||||
}
|
||||
|
||||
/**
|
||||
* True when this KeyPackage is marked last resort, in either carrier.
|
||||
*
|
||||
* The two profiles say it differently and a KeyPackage may be read under
|
||||
* either: the MIP-era profile sets the MLS Extensions draft extension type
|
||||
* `0x000A` directly on the KeyPackage, while the current profile carries a
|
||||
* `last_resort_key_package` component inside the KeyPackage-level
|
||||
* `app_data_dictionary` (`0x0006`).
|
||||
*
|
||||
* A last-resort KeyPackage is explicitly NOT single-use: OpenMLS skips
|
||||
* `delete_key_package` for one, and MDK — which marks every KeyPackage it
|
||||
* publishes last resort — invites from the copy cached in its directory.
|
||||
* Anything that consumes a KeyPackage has to check this before discarding
|
||||
* the bundle.
|
||||
*/
|
||||
fun isLastResort(): Boolean =
|
||||
extensions.any { it.extensionType == LAST_RESORT_EXTENSION_TYPE } ||
|
||||
AppDataDictionary.fromExtensionsOrEmpty(extensions).contains(AppComponentIds.LAST_RESORT_KEY_PACKAGE)
|
||||
|
||||
/**
|
||||
* Encode the TBS (to-be-signed) portion for signature verification.
|
||||
*/
|
||||
@@ -118,6 +139,12 @@ data class MlsKeyPackage(
|
||||
}
|
||||
|
||||
companion object {
|
||||
/**
|
||||
* `last_resort` KeyPackage extension (MLS Extensions draft). Marks a
|
||||
* KeyPackage as reusable rather than single-use.
|
||||
*/
|
||||
const val LAST_RESORT_EXTENSION_TYPE = 0x000A
|
||||
|
||||
fun decodeTls(reader: TlsReader): MlsKeyPackage {
|
||||
val version = reader.readUint16()
|
||||
require(version == 1) { "Unsupported MLS version: $version" }
|
||||
|
||||
+256
@@ -0,0 +1,256 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.quartz.marmot.mip00KeyPackages
|
||||
|
||||
import com.vitorpamplona.quartz.marmot.appComponents.CurrentProfileGroupFactory
|
||||
import com.vitorpamplona.quartz.marmot.mls.tree.Extension
|
||||
import com.vitorpamplona.quartz.nip01Core.crypto.KeyPair
|
||||
import com.vitorpamplona.quartz.nip01Core.signers.NostrSignerInternal
|
||||
import kotlinx.coroutines.runBlocking
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertNotNull
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
/**
|
||||
* A KeyPackage marked LastResort (MLS extension `0x000A`) is deliberately
|
||||
* NOT single-use. OpenMLS says so in as many words — on the Welcome path it
|
||||
* deletes the consumed bundle only `if !key_package_bundle.key_package().last_resort()`,
|
||||
* and logs "KeyPackage has a last-resort marker, not deleting" otherwise — and
|
||||
* MDK marks every KeyPackage it publishes as last resort, caches the peer
|
||||
* KeyPackage it resolved in its user directory, and re-uses that same cached
|
||||
* copy for every later invite of that peer.
|
||||
*
|
||||
* We publish the LastResort marker too (OpenMLS's own KeyPackage validation
|
||||
* wants it). So we have to honour the contract we advertise: dropping the
|
||||
* private keys the moment one Welcome consumed the KeyPackage makes every
|
||||
* subsequent invite addressed to that same KeyPackage unjoinable, which is
|
||||
* exactly what the MDK interop harness saw — one invite worked and the four
|
||||
* that followed failed with "No matching KeyPackageBundle".
|
||||
*/
|
||||
class LastResortKeyPackageReuseTest {
|
||||
private val identity = ByteArray(32) { 0x11 }
|
||||
|
||||
@Test
|
||||
fun aLastResortKeyPackageStaysUsableAfterItIsConsumed() =
|
||||
runBlocking {
|
||||
val manager = KeyPackageRotationManager()
|
||||
val slot = manager.getOrCreateSlotDTag("primary")
|
||||
val bundle = manager.generateKeyPackage(identity, slot)
|
||||
assertTrue(
|
||||
"generateKeyPackage must mark the KeyPackage last-resort — MDK requires it",
|
||||
bundle.keyPackage.isLastResort(),
|
||||
)
|
||||
|
||||
val eventId = "a".repeat(64)
|
||||
manager.recordPublishedEventId(slot, eventId)
|
||||
|
||||
manager.markConsumedByEventId(eventId)
|
||||
|
||||
assertNotNull(
|
||||
"a last-resort KeyPackage must still resolve by event id after it was consumed",
|
||||
manager.findBundleByEventId(eventId),
|
||||
)
|
||||
assertNotNull(
|
||||
"a last-resort KeyPackage must still resolve by MLS KeyPackageRef after it was consumed",
|
||||
manager.findBundleByRef(bundle.keyPackage.reference()),
|
||||
)
|
||||
assertTrue(
|
||||
"consuming it still schedules a fresh publication for the slot",
|
||||
manager.needsRotation(),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun aSingleUseKeyPackageIsStillDroppedWhenConsumed() =
|
||||
runBlocking {
|
||||
val manager = KeyPackageRotationManager()
|
||||
val slot = manager.getOrCreateSlotDTag("primary")
|
||||
val bundle = manager.generateKeyPackage(identity, slot)
|
||||
// Strip the LastResort marker: without it MLS single-use applies
|
||||
// and forward secrecy says the init key must not survive.
|
||||
val singleUse = bundle.copy(keyPackage = bundle.keyPackage.copy(extensions = emptyList()))
|
||||
manager.installBundle(slot, singleUse)
|
||||
assertFalse(singleUse.keyPackage.isLastResort())
|
||||
|
||||
val eventId = "b".repeat(64)
|
||||
manager.recordPublishedEventId(slot, eventId)
|
||||
manager.markConsumedByEventId(eventId)
|
||||
|
||||
assertNull(
|
||||
"a KeyPackage without the last-resort marker is single-use and must be dropped",
|
||||
manager.findBundleByEventId(eventId),
|
||||
)
|
||||
assertNull(manager.findBundleByRef(singleUse.keyPackage.reference()))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun rotatingTheSlotKeepsTheConsumedKeyPackageReachable() =
|
||||
runBlocking {
|
||||
val manager = KeyPackageRotationManager()
|
||||
val slot = manager.getOrCreateSlotDTag("primary")
|
||||
val consumed = manager.generateKeyPackage(identity, slot)
|
||||
val firstEventId = "c".repeat(64)
|
||||
manager.recordPublishedEventId(slot, firstEventId)
|
||||
manager.markConsumedByEventId(firstEventId)
|
||||
|
||||
// MIP-00 rotation publishes a replacement into the same d-tag slot.
|
||||
val rotated = manager.rotateSlot(identity, slot)
|
||||
val secondEventId = "d".repeat(64)
|
||||
manager.recordPublishedEventId(slot, secondEventId)
|
||||
|
||||
assertEquals(
|
||||
"the slot now serves the rotated KeyPackage",
|
||||
rotated.keyPackage.reference().toList(),
|
||||
manager
|
||||
.findBundleByEventId(secondEventId)!!
|
||||
.keyPackage
|
||||
.reference()
|
||||
.toList(),
|
||||
)
|
||||
assertEquals(
|
||||
"an invite that still references the consumed KeyPackage must keep working",
|
||||
consumed.keyPackage.reference().toList(),
|
||||
manager
|
||||
.findBundleByEventId(firstEventId)!!
|
||||
.keyPackage
|
||||
.reference()
|
||||
.toList(),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun retainedKeyPackagesAreBoundedAndDropTheOldestFirst() =
|
||||
runBlocking {
|
||||
val manager = KeyPackageRotationManager()
|
||||
val slot = manager.getOrCreateSlotDTag("primary")
|
||||
val eventIds = mutableListOf<String>()
|
||||
repeat(KeyPackageRotationManager.MAX_RETAINED_BUNDLES + 2) { i ->
|
||||
manager.generateKeyPackage(identity, slot)
|
||||
val eventId = i.toString().padStart(64, '0')
|
||||
eventIds.add(eventId)
|
||||
manager.recordPublishedEventId(slot, eventId)
|
||||
manager.markConsumedByEventId(eventId)
|
||||
}
|
||||
|
||||
assertNull(
|
||||
"the oldest consumed KeyPackage must age out of the bounded retention window",
|
||||
manager.findBundleByEventId(eventIds.first()),
|
||||
)
|
||||
assertNotNull(
|
||||
"the most recently consumed KeyPackage must still be reachable",
|
||||
manager.findBundleByEventId(eventIds.last()),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun retainedKeyPackagesSurviveAPersistenceRoundTrip() =
|
||||
runBlocking {
|
||||
val store = InMemoryKeyPackageBundleStore()
|
||||
val manager = KeyPackageRotationManager(store)
|
||||
val slot = manager.getOrCreateSlotDTag("primary")
|
||||
val consumed = manager.generateKeyPackage(identity, slot)
|
||||
val eventId = "e".repeat(64)
|
||||
manager.recordPublishedEventId(slot, eventId)
|
||||
manager.markConsumedByEventId(eventId)
|
||||
manager.rotateSlot(identity, slot)
|
||||
|
||||
val restored = KeyPackageRotationManager(store)
|
||||
restored.restoreFromStore()
|
||||
|
||||
assertEquals(
|
||||
"a restart must not lose the private keys of a consumed last-resort KeyPackage",
|
||||
consumed.keyPackage.reference().toList(),
|
||||
restored
|
||||
.findBundleByEventId(eventId)!!
|
||||
.keyPackage
|
||||
.reference()
|
||||
.toList(),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* The current profile does not use the MLS Extensions draft's `0x000A`
|
||||
* extension type at all — it carries a `last_resort_key_package` component
|
||||
* inside the KeyPackage-level `app_data_dictionary` (`0x0006`). Reading only
|
||||
* the MIP-era carrier makes every KeyPackage we actually publish today look
|
||||
* single-use, which is exactly the case the interop harness exercises.
|
||||
*/
|
||||
@Test
|
||||
fun aCurrentProfileKeyPackageIsRecognisedAsLastResort() =
|
||||
runBlocking {
|
||||
val signer = NostrSignerInternal(KeyPair())
|
||||
val manager = KeyPackageRotationManager()
|
||||
val slot = manager.getOrCreateSlotDTag("primary")
|
||||
val bundle = manager.generateCurrentProfileKeyPackage(signer, slot)
|
||||
assertTrue(
|
||||
"the current profile marks last resort with a dictionary component, not extension 0x000A",
|
||||
bundle.keyPackage.isLastResort(),
|
||||
)
|
||||
|
||||
val eventId = "f".repeat(64)
|
||||
manager.recordPublishedEventId(slot, eventId)
|
||||
manager.markConsumedByRef(bundle.keyPackage.reference())
|
||||
|
||||
assertNotNull(
|
||||
"a consumed current-profile KeyPackage must stay reusable",
|
||||
manager.findBundleByRef(bundle.keyPackage.reference()),
|
||||
)
|
||||
assertNotNull(manager.findBundleByEventId(eventId))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun aCurrentProfileKeyPackageWithoutTheComponentIsSingleUse() =
|
||||
runBlocking {
|
||||
val signer = NostrSignerInternal(KeyPair())
|
||||
val bundle = CurrentProfileGroupFactory.createKeyPackage(signer, lastResort = false)
|
||||
assertFalse(bundle.keyPackage.isLastResort())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun theLastResortMarkerIsReadFromTheKeyPackageExtensions() {
|
||||
val manager = KeyPackageRotationManager()
|
||||
val bundle = runBlocking { manager.generateKeyPackage(identity) }
|
||||
assertTrue(bundle.keyPackage.isLastResort())
|
||||
assertFalse(bundle.keyPackage.copy(extensions = emptyList()).isLastResort())
|
||||
assertFalse(
|
||||
bundle.keyPackage
|
||||
.copy(extensions = listOf(Extension(extensionType = 0x0001, extensionData = ByteArray(0))))
|
||||
.isLastResort(),
|
||||
)
|
||||
}
|
||||
|
||||
private class InMemoryKeyPackageBundleStore : KeyPackageBundleStore {
|
||||
private var bytes: ByteArray? = null
|
||||
|
||||
override suspend fun load(): ByteArray? = bytes
|
||||
|
||||
override suspend fun save(data: ByteArray) {
|
||||
bytes = data
|
||||
}
|
||||
|
||||
override suspend fun delete() {
|
||||
bytes = null
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user