fix(marmot): a last-resort KeyPackage is not single-use

We publish every KeyPackage marked last resort, and then threw away its
private keys the moment one Welcome consumed it. Those two things cannot
both be true. OpenMLS is explicit about the contract — on the Welcome
path it deletes the consumed bundle only `if !key_package.last_resort()`
and otherwise logs "KeyPackage has a last-resort marker, not deleting" —
and MDK leans on it: it marks all of its own KeyPackages last resort,
caches the peer KeyPackage it resolved in its user directory, and invites
from that same cached copy every time after. So the first invite
addressed to us worked and every one after it died on "No matching
KeyPackageBundle", which is four of the interop harness's failures.

Consumed bundles now stay reachable when the KeyPackage says they may
be, bounded on both axes: at most eight of them, and never past the
KeyPackage's own not_after. That retention is the entire forward-secrecy
cost of the last-resort marker, and it is a cost we already accepted by
publishing the marker.

Two things had to be right for it to work at all:

  - `isLastResort()` has to read both carriers. The MIP-era profile sets
    MLS extension type 0x000A on the KeyPackage; the current profile —
    the one we actually publish — carries a `last_resort_key_package`
    component inside the KeyPackage-level app_data_dictionary. Reading
    only the first made every KeyPackage we ship look single-use.

  - The Welcome lookup has to trust the MLS refs over the Nostr "e" tag.
    RFC 9420 addresses each EncryptedGroupSecrets to a KeyPackageRef and
    the joiner takes the first it holds keys for; the "e" tag is a
    routing hint an inviter can get wrong, and MDK gets it wrong exactly
    here — it stamps the event id of its cached copy, which is stale the
    moment we rotate. Refs first, tag as fallback.

The restore path also stopped throwing the whole snapshot away when the
eventId→slot index is empty. It drops the active bundles that index made
unreachable, and keeps the retained bundles (keyed by event id, always
reachable) and the named slot d-tags (a fresh d-tag would republish into
a new addressable slot and orphan the old one).

Harness: reset A's amy home and the relay database at the start of every
run, keeping the relay build. wnd already wiped B's and C's data dirs,
but A's store and the relay's events survived, and the leftovers are not
inert — a KeyPackage from an earlier run is still on the relay to be
invited with, and old kind:445 events still arrive undecryptable. That
drift alone accounted for tests 03 and 08. `--reuse-state` opts out and
`--tests "..."` runs a subset.

Interop: 10 → 14 of 17 passing. 05, 12, 14 and 15 (every "A never
received invite") now pass, as do 03 and 08.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kCuA6tc4JQzHPCDd39GHq
This commit is contained in:
Claude
2026-09-09 08:20:37 +00:00
parent 6fb364394d
commit e493ecbe6e
5 changed files with 542 additions and 69 deletions
+54 -18
View File
@@ -9,7 +9,8 @@
# any human prompts — all checks run to completion and the exit code
# reflects pass/fail totals.
#
# Usage: ./marmot-interop-headless.sh [--port N] [--no-build]
# Usage: ./marmot-interop-headless.sh [--port N] [--no-build] [--reuse-state]
# [--tests "name ..."]
#
set -uo pipefail
@@ -53,6 +54,16 @@ RELAY_DATA="$STATE_DIR/relay"
RELAY_PORT="${RELAY_PORT:-8080}"
RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"
NO_BUILD=0
# Every run starts from empty stores. wnd already wipes B's and C's data dirs
# on each start, but A's amy home and the relay's SQLite file used to survive,
# and the leftovers are not inert: a KeyPackage A published in an earlier run
# is still on the relay for B to invite with, an old group's kind:445 events
# still arrive and fail to decrypt, and A's cursors still say it has seen them.
# That drift is what made tests 03 and 08 fail on a dirty tree and pass on a
# clean one. Pass --reuse-state when you are deliberately debugging carry-over.
RESET_STATE=1
# Space-separated test function names; empty means the full suite below.
ONLY_TESTS=""
# Required as of MDK 0.9.x. `validate_relay_url` accepts `wss://`
# unconditionally but `ws://` only for a loopback host AND only behind this
@@ -75,6 +86,8 @@ while [[ $# -gt 0 ]]; do
--port) RELAY_PORT="$2"; RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"; shift ;;
--host) RELAY_HOST="$2"; RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"; shift ;;
--no-build) NO_BUILD=1 ;;
--reuse-state) RESET_STATE=0 ;;
--tests) ONLY_TESTS="$2"; shift ;;
-h|--help)
sed -n '3,14p' "${BASH_SOURCE[0]}" | sed 's/^# \?//'
exit 0 ;;
@@ -83,6 +96,12 @@ while [[ $# -gt 0 ]]; do
shift
done
if [[ $RESET_STATE -eq 1 && -d "$STATE_DIR" ]]; then
# Keep the relay checkout + its build (minutes to rebuild) and the log and
# results history; drop everything that holds protocol state.
rm -rf "$STATE_DIR/.amy" "$B_DIR" "$C_DIR" "$RELAY_DATA"
fi
mkdir -p "$STATE_DIR" "$LOG_DIR" "$B_DIR/logs" "$C_DIR/logs"
: >"$LOG_FILE"
: >"$RESULTS_FILE"
@@ -129,20 +148,37 @@ ensure_identity B
ensure_identity C
configure_relays
test_01_keypackage_discovery
test_02_a_creates_group
test_03_b_creates_group
test_04_three_member_group
test_05_b_adds_a_existing
test_06_member_removal
test_07_metadata_rename
test_08_admin_promote_demote
test_17_group_image_commit
test_09_reply_react_unreact
test_10_concurrent_commits
test_11_leave_group
test_12_offline_catchup
test_13_keypackage_rotation
test_14_wn_removes_a
test_15_wn_member_leaves
test_16_wn_keypackage_rotation
ALL_TESTS=(
test_01_keypackage_discovery
test_02_a_creates_group
test_03_b_creates_group
test_04_three_member_group
test_05_b_adds_a_existing
test_06_member_removal
test_07_metadata_rename
test_08_admin_promote_demote
test_17_group_image_commit
test_09_reply_react_unreact
test_10_concurrent_commits
test_11_leave_group
test_12_offline_catchup
test_13_keypackage_rotation
test_14_wn_removes_a
test_15_wn_member_leaves
test_16_wn_keypackage_rotation
)
# --tests runs a subset in the order given. Most tests read state a previous
# one saved (GROUP_02, GROUP_05, …), so a subset that skips a producer will
# report `skip`, not a false failure.
if [[ -n "$ONLY_TESTS" ]]; then
read -r -a ALL_TESTS <<<"$ONLY_TESTS"
fi
for t in "${ALL_TESTS[@]}"; do
if ! declare -F "$t" >/dev/null; then
fail_msg "unknown test: $t"
continue
fi
"$t"
done
@@ -33,6 +33,8 @@ import com.vitorpamplona.quartz.marmot.mls.framing.PublicMessage
import com.vitorpamplona.quartz.marmot.mls.framing.WireFormat
import com.vitorpamplona.quartz.marmot.mls.group.MlsGroupManager
import com.vitorpamplona.quartz.marmot.mls.group.MlsGroupState
import com.vitorpamplona.quartz.marmot.mls.messages.KeyPackageBundle
import com.vitorpamplona.quartz.marmot.mls.messages.Welcome
import com.vitorpamplona.quartz.marmot.protocolCore.ConvergenceAdmission
import com.vitorpamplona.quartz.marmot.protocolCore.ConvergenceResolution
import com.vitorpamplona.quartz.marmot.protocolCore.ConvergenceStatus
@@ -41,6 +43,7 @@ import com.vitorpamplona.quartz.marmot.protocolCore.MarmotConvergenceEngine
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.HexKey
import com.vitorpamplona.quartz.nip01Core.core.toHexKey
import com.vitorpamplona.quartz.utils.Log
import com.vitorpamplona.quartz.utils.sha256.sha256
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
@@ -380,7 +383,7 @@ class MarmotInboundProcessor(
hintNostrGroupId: HexKey? = null,
): WelcomeResult =
try {
com.vitorpamplona.quartz.utils.Log
Log
.d("MarmotDbg") {
"MarmotInboundProcessor.processWelcome: hint=${hintNostrGroupId?.take(8)} eventId=${welcomeEvent.id.take(8)}…"
}
@@ -390,7 +393,7 @@ class MarmotInboundProcessor(
if (keyPackageEventId == null) {
return WelcomeResult.Error("WelcomeEvent missing KeyPackage event ID tag")
}
com.vitorpamplona.quartz.utils.Log
Log
.d("MarmotDbg") {
"MarmotInboundProcessor.processWelcome: welcomeBytes=${welcomeBytes.size}B looking up KeyPackage by ref=${keyPackageEventId.take(8)}…"
}
@@ -404,23 +407,28 @@ class MarmotInboundProcessor(
// log a noisy "No matching KeyPackageBundle" warning for what
// is actually a benign replay.
if (hintNostrGroupId != null && groupManager.isMember(hintNostrGroupId)) {
com.vitorpamplona.quartz.utils.Log
Log
.d("MarmotDbg") {
"MarmotInboundProcessor.processWelcome: already a member of group=${hintNostrGroupId.take(8)}… — treating Welcome as replay"
}
return WelcomeResult.AlreadyJoined(hintNostrGroupId)
}
// Find the KeyPackageBundle that was consumed.
// Find the KeyPackageBundle the inviter encrypted to.
//
// The Welcome's "e" tag carries the *Nostr event id* of the
// kind:30443 event (NOT the MLS reference hash), so we must
// resolve it via the eventId→slot index that
// [MarmotManager.generateKeyPackageEvent] populates after
// signing each KeyPackageEvent.
val bundle = keyPackageRotationManager.findBundleByEventId(keyPackageEventId)
// The authority is the MLS Welcome itself: each EncryptedGroupSecrets
// is addressed to a KeyPackageRef, and RFC 9420 says a joiner takes
// the first one it holds private keys for — which is exactly what
// OpenMLS does. The Welcome's "e" tag carries only the *Nostr event
// id* of the kind:30443 event, which is a routing hint an inviter can
// get wrong: MDK stamps the event id of the copy cached in its user
// directory, so a peer that rotated its published KeyPackage while
// MDK kept inviting from cache would be unjoinable if we trusted the
// tag alone. Try the refs first, then fall back to the tag.
val bundle =
findBundleForWelcome(welcomeBytes) ?: keyPackageRotationManager.findBundleByEventId(keyPackageEventId)
if (bundle == null) {
com.vitorpamplona.quartz.utils.Log
Log
.w("MarmotDbg") {
"MarmotInboundProcessor.processWelcome: NO matching KeyPackageBundle for eventId=${keyPackageEventId.take(8)}… " +
"— inviter referenced a KeyPackage we don't have private keys for. " +
@@ -431,17 +439,19 @@ class MarmotInboundProcessor(
"No matching KeyPackageBundle found for event $keyPackageEventId",
)
}
com.vitorpamplona.quartz.utils.Log
Log
.d("MarmotDbg") { "MarmotInboundProcessor.processWelcome: bundle found — invoking groupManager.processWelcome" }
// Join the group; nostrGroupId is derived from the MLS GroupContext's
// NostrGroupData extension. The h-tag hint (if any) is validated inside.
val (_, nostrGroupId) = groupManager.processWelcome(welcomeBytes, bundle, hintNostrGroupId)
com.vitorpamplona.quartz.utils.Log
Log
.d("MarmotDbg") { "MarmotInboundProcessor.processWelcome: joined group=${nostrGroupId.take(8)}…" }
// Mark the KeyPackage as consumed — triggers rotation
keyPackageRotationManager.markConsumedByEventId(keyPackageEventId)
// Mark the KeyPackage as consumed — triggers rotation. Keyed on the
// bundle we actually used, not the "e" tag, for the same reason the
// lookup above is.
keyPackageRotationManager.markConsumedByRef(bundle.keyPackage.reference())
// Seed convergence with the joined state, so the very first inbound
// commit already has a retained parent to fall back to.
@@ -452,11 +462,39 @@ class MarmotInboundProcessor(
needsKeyPackageRotation = keyPackageRotationManager.needsRotation(),
)
} catch (e: Exception) {
com.vitorpamplona.quartz.utils.Log
Log
.w("MarmotDbg", "MarmotInboundProcessor.processWelcome: exception ${e.message}", e)
WelcomeResult.Error("Failed to process Welcome: ${e.message}", e)
}
/**
* Resolve the KeyPackageBundle a Welcome is addressed to, the way RFC 9420
* §12.4.3.1 (and OpenMLS) does it: walk the Welcome's EncryptedGroupSecrets
* in order and take the first `new_member` KeyPackageRef we hold private
* keys for.
*
* Returns null when the Welcome does not parse or names no KeyPackage of
* ours — the caller then falls back to the Nostr "e" tag hint, and reports
* the failure if that misses too.
*/
private suspend fun findBundleForWelcome(welcomeBytes: ByteArray): KeyPackageBundle? {
val welcome =
try {
val mlsMessage = MlsMessage.decodeTls(TlsReader(welcomeBytes))
require(mlsMessage.wireFormat == WireFormat.WELCOME) { "not a Welcome wire format" }
Welcome.decodeTls(TlsReader(mlsMessage.payload))
} catch (e: Exception) {
Log.d("MarmotDbg") {
"MarmotInboundProcessor.findBundleForWelcome: welcome did not parse (${e.message}) — falling back to the e tag"
}
return null
}
for (secret in welcome.secrets) {
keyPackageRotationManager.findBundleByRef(secret.newMember)?.let { return it }
}
return null
}
/**
* Mark a kind:445 event id as already processed so that a later relay
* echo of the same event is treated as a [GroupEventResult.Duplicate]
@@ -53,9 +53,12 @@ import kotlinx.coroutines.sync.withLock
* - Handle periodic rotation for long-lived KeyPackages
*
* After a KeyPackage is consumed by a Welcome message:
* 1. The init_key is effectively spent — cannot be reused
* 2. A new KeyPackage MUST be published to the same d-tag slot
* 3. The old KeyPackageBundle MUST be discarded
* 1. A new KeyPackage MUST be published to the same d-tag slot
* 2. Whether the old bundle may be discarded depends on the LastResort marker:
* a plain KeyPackage is single-use and its private keys are dropped, while a
* KeyPackage carrying `0x000A` is reusable by contract and its bundle is
* kept (bounded) in [retainedBundles] so a later Welcome addressed to the
* same KeyPackage still joins.
*
* Per MIP-00 spec, each user should maintain up to [KeyPackageUtils.MAX_SLOTS]
* KeyPackage slots, rotating consumed ones promptly.
@@ -87,6 +90,24 @@ class KeyPackageRotationManager(
*/
private val eventIdToSlot = mutableMapOf<String, String>()
/**
* Consumed KeyPackages we deliberately keep the private keys for, keyed by
* the Nostr event id (kind:30443) they were published as.
*
* A KeyPackage carrying the LastResort marker (`0x000A`) is not single-use:
* OpenMLS skips `delete_key_package` for one, so MDK — which marks every
* KeyPackage last-resort and caches the peer KeyPackage it resolved — will
* happily address a second, third and fourth Welcome to the same
* KeyPackage of ours. Dropping the bundle after the first Welcome made all
* of those unjoinable.
*
* Retention is bounded on both axes: at most [MAX_RETAINED_BUNDLES] entries
* (oldest evicted first), and never past the KeyPackage's own `not_after`
* lifetime. That is the whole forward-secrecy cost of the last-resort
* marker, and it is the cost we already accepted by publishing it.
*/
private val retainedBundles = mutableMapOf<String, KeyPackageBundle>()
/**
* Restore previously persisted bundles + rotation state from [store].
* Call once at startup before any other use of this manager.
@@ -125,36 +146,41 @@ class KeyPackageRotationManager(
return
}
// v2 snapshot: if bundles were restored but the eventId
// index is empty (upgrade corner case, or a corrupted save),
// the bundles are effectively unreachable — wipe them too
// so a fresh publish happens.
if (decoded.bundles.isNotEmpty() && decoded.eventIdToSlot.isEmpty()) {
// Active bundles are reachable only through the eventId→slot
// index, so a snapshot that has bundles but no index (upgrade
// corner case, a corrupted save, or a rotation that never got as
// far as publishing) can't serve a Welcome. Drop just those
// bundles so `hasActiveKeyPackages()` is false and a fresh publish
// happens. Everything else in the snapshot stays: the retained
// last-resort bundles are keyed by event id directly and are still
// the only way to join an invite sent from a peer's cache, and the
// named slot d-tags have to stay stable or the republish lands in
// a new addressable slot and orphans the old one.
val unreachableActiveBundles = decoded.bundles.isNotEmpty() && decoded.eventIdToSlot.isEmpty()
if (unreachableActiveBundles) {
Log.w("KeyPackageRotationManager") {
"Restored ${decoded.bundles.size} bundle(s) but no eventId→slot mapping — discarding, will republish"
"Restored ${decoded.bundles.size} bundle(s) but no eventId→slot mapping — dropping them, will republish"
}
try {
store.delete()
} catch (e: Exception) {
Log.w("KeyPackageRotationManager", "Failed to delete stale snapshot", e)
}
return
}
mutex.withLock {
activeBundles.clear()
activeBundles.putAll(decoded.bundles)
if (!unreachableActiveBundles) activeBundles.putAll(decoded.bundles)
pendingRotations.clear()
pendingRotations.addAll(decoded.pending)
eventIdToSlot.clear()
eventIdToSlot.putAll(decoded.eventIdToSlot)
namedSlotDTags.clear()
namedSlotDTags.putAll(decoded.namedSlotDTags)
retainedBundles.clear()
retainedBundles.putAll(decoded.retainedBundles)
if (unreachableActiveBundles) persistUnlocked()
}
Log.d("KeyPackageRotationManager") {
"Restored ${decoded.bundles.size} active KeyPackage bundle(s), " +
"${decoded.pending.size} pending rotation, ${decoded.eventIdToSlot.size} eventId mapping(s), " +
"${decoded.namedSlotDTags.size} named slot d-tag(s)"
"${decoded.namedSlotDTags.size} named slot d-tag(s), " +
"${decoded.retainedBundles.size} retained last-resort bundle(s)"
}
} catch (e: Exception) {
Log.w("KeyPackageRotationManager", "Failed to decode persisted KeyPackages", e)
@@ -173,6 +199,7 @@ class KeyPackageRotationManager(
pendingRotations.clear()
eventIdToSlot.clear()
namedSlotDTags.clear()
retainedBundles.clear()
val store = store ?: return@withLock
try {
store.delete()
@@ -186,6 +213,7 @@ class KeyPackageRotationManager(
val pending: Set<String>,
val eventIdToSlot: Map<String, String>,
val namedSlotDTags: Map<String, String>,
val retainedBundles: Map<String, KeyPackageBundle>,
)
/**
@@ -222,6 +250,15 @@ class KeyPackageRotationManager(
writer.putOpaque2(name.encodeToByteArray())
writer.putOpaque2(dTag.encodeToByteArray())
}
// consumed-but-reusable last-resort bundles, by event id (added in v5)
writer.putUint32(retainedBundles.size.toLong())
for ((eventId, bundle) in retainedBundles) {
writer.putOpaque2(eventId.encodeToByteArray())
writer.putOpaque4(bundle.keyPackage.toTlsBytes())
writer.putOpaque2(bundle.initPrivateKey)
writer.putOpaque2(bundle.encryptionPrivateKey)
writer.putOpaque2(bundle.signaturePrivateKey)
}
return writer.toByteArray()
}
@@ -270,7 +307,19 @@ class KeyPackageRotationManager(
namedSlots[name] = dTag
}
}
return Snapshot(bundles, pending, eventIdMap, namedSlots)
val retained = mutableMapOf<String, KeyPackageBundle>()
if (reader.hasRemaining) {
val numRetained = reader.readUint32().toInt()
repeat(numRetained) {
val eventId = reader.readOpaque2().decodeToString()
val keyPackage = MlsKeyPackage.decodeTls(TlsReader(reader.readOpaque4()))
val initPriv = reader.readOpaque2()
val encPriv = reader.readOpaque2()
val sigPriv = reader.readOpaque2()
retained[eventId] = KeyPackageBundle(keyPackage, initPriv, encPriv, sigPriv)
}
}
return Snapshot(bundles, pending, eventIdMap, namedSlots, retained)
}
/**
@@ -393,8 +442,11 @@ class KeyPackageRotationManager(
*/
suspend fun findBundleByRef(keyPackageRef: ByteArray): KeyPackageBundle? =
mutex.withLock {
pruneExpiredRetainedUnlocked()
activeBundles.values.find { bundle ->
bundle.keyPackage.reference().contentEquals(keyPackageRef)
} ?: retainedBundles.values.find { bundle ->
bundle.keyPackage.reference().contentEquals(keyPackageRef)
}
}
@@ -408,8 +460,12 @@ class KeyPackageRotationManager(
*/
suspend fun findBundleByEventId(eventId: HexKey): KeyPackageBundle? =
mutex.withLock {
val slot = eventIdToSlot[eventId] ?: return@withLock null
activeBundles[slot]
pruneExpiredRetainedUnlocked()
val slot = eventIdToSlot[eventId]
if (slot != null) {
activeBundles[slot]?.let { return@withLock it }
}
retainedBundles[eventId]
}
/**
@@ -434,11 +490,7 @@ class KeyPackageRotationManager(
*/
suspend fun markConsumed(dTagSlot: String) =
mutex.withLock {
activeBundles.remove(dTagSlot)
// Drop any eventId mappings that pointed at this slot.
val staleEventIds = eventIdToSlot.entries.filter { it.value == dTagSlot }.map { it.key }
staleEventIds.forEach { eventIdToSlot.remove(it) }
pendingRotations.add(dTagSlot)
consumeSlotUnlocked(dTagSlot)
persistUnlocked()
}
@@ -452,11 +504,7 @@ class KeyPackageRotationManager(
bundle.keyPackage.reference().contentEquals(keyPackageRef)
}
if (entry != null) {
val consumedSlot = entry.key
activeBundles.remove(consumedSlot)
val staleEventIds = eventIdToSlot.entries.filter { it.value == consumedSlot }.map { it.key }
staleEventIds.forEach { eventIdToSlot.remove(it) }
pendingRotations.add(consumedSlot)
consumeSlotUnlocked(entry.key)
persistUnlocked()
}
}
@@ -468,13 +516,72 @@ class KeyPackageRotationManager(
suspend fun markConsumedByEventId(eventId: HexKey) =
mutex.withLock {
val slot = eventIdToSlot[eventId] ?: return@withLock
activeBundles.remove(slot)
val staleEventIds = eventIdToSlot.entries.filter { it.value == slot }.map { it.key }
staleEventIds.forEach { eventIdToSlot.remove(it) }
pendingRotations.add(slot)
consumeSlotUnlocked(slot)
persistUnlocked()
}
/**
* Retire the bundle in [dTagSlot] and schedule the slot for a fresh
* publication. Caller must hold the mutex.
*
* A KeyPackage that advertises LastResort (`0x000A`) is reusable by
* contract — OpenMLS keeps its bundle on the Welcome path and MDK invites
* from a cached copy — so its private keys move to [retainedBundles],
* still reachable by every event id that published it. Anything else is
* single-use: the keys go, so a compromise later cannot reopen the Welcome
* that consumed them.
*/
private fun consumeSlotUnlocked(dTagSlot: String) {
val consumed = activeBundles.remove(dTagSlot)
val staleEventIds = eventIdToSlot.entries.filter { it.value == dTagSlot }.map { it.key }
if (consumed != null && consumed.keyPackage.isLastResort()) {
for (staleEventId in staleEventIds) {
// Re-insert so the most recently consumed entry sorts last and
// survives eviction the longest.
retainedBundles.remove(staleEventId)
retainedBundles[staleEventId] = consumed
}
pruneExpiredRetainedUnlocked()
while (retainedBundles.size > MAX_RETAINED_BUNDLES) {
retainedBundles.remove(retainedBundles.keys.first())
}
}
staleEventIds.forEach { eventIdToSlot.remove(it) }
pendingRotations.add(dTagSlot)
}
/**
* Drop retained bundles whose KeyPackage is past its own `not_after`.
* No peer may invite with an expired KeyPackage, so holding its private
* keys buys nothing. Caller must hold the mutex.
*/
private fun pruneExpiredRetainedUnlocked() {
val now = TimeUtils.now()
val expired =
retainedBundles.entries
.filter { (_, bundle) ->
val notAfter =
bundle.keyPackage.leafNode.lifetime
?.notAfter ?: return@filter false
now > notAfter
}.map { it.key }
expired.forEach { retainedBundles.remove(it) }
}
/**
* Install [bundle] as the active bundle for [dTagSlot], replacing whatever
* was there. Used by callers that mint a KeyPackage themselves (tests, and
* any flow that builds a bundle outside this manager) and still want the
* manager to own its lifecycle.
*/
suspend fun installBundle(
dTagSlot: String,
bundle: KeyPackageBundle,
) = mutex.withLock {
activeBundles[dTagSlot] = bundle
persistUnlocked()
}
/**
* Get the d-tag slots that need rotation (KeyPackage was consumed).
*/
@@ -577,13 +684,22 @@ class KeyPackageRotationManager(
/** Proactive rotation after 7 days even if not consumed */
const val MAX_KEY_PACKAGE_AGE_SECONDS = 7L * 24 * 60 * 60
/**
* How many consumed last-resort bundles to keep private keys for.
* Each one is a KeyPackage a peer may still be inviting us with from
* its own cache; the bound keeps a long-lived account from carrying
* every init key it ever published.
*/
const val MAX_RETAINED_BUNDLES = 8
/**
* On-disk snapshot format version for [KeyPackageBundleStore].
* v1: bundles + pendingRotations
* v2: + eventIdToSlot map (so welcome lookup by Nostr event id works)
* v3: + namedSlotDTags map (per MIP-00, d-tags are random 64-char hex, persisted here)
* v4: capabilities fixed (0xF2EE, 0x000A extensions + 0x000A proposals; LastResort extension on KP)
* v5: + retainedBundles map (consumed last-resort KeyPackages stay reusable)
*/
private const val SNAPSHOT_VERSION = 4
private const val SNAPSHOT_VERSION = 5
}
}
@@ -20,9 +20,11 @@
*/
package com.vitorpamplona.quartz.marmot.mls.messages
import com.vitorpamplona.quartz.marmot.appComponents.AppComponentIds
import com.vitorpamplona.quartz.marmot.mls.codec.TlsReader
import com.vitorpamplona.quartz.marmot.mls.codec.TlsSerializable
import com.vitorpamplona.quartz.marmot.mls.codec.TlsWriter
import com.vitorpamplona.quartz.marmot.mls.components.AppDataDictionary
import com.vitorpamplona.quartz.marmot.mls.crypto.MlsCryptoProvider
import com.vitorpamplona.quartz.marmot.mls.tree.Extension
import com.vitorpamplona.quartz.marmot.mls.tree.LeafNode
@@ -74,6 +76,25 @@ data class MlsKeyPackage(
return MlsCryptoProvider.refHash("MLS 1.0 KeyPackage Reference", encoded)
}
/**
* True when this KeyPackage is marked last resort, in either carrier.
*
* The two profiles say it differently and a KeyPackage may be read under
* either: the MIP-era profile sets the MLS Extensions draft extension type
* `0x000A` directly on the KeyPackage, while the current profile carries a
* `last_resort_key_package` component inside the KeyPackage-level
* `app_data_dictionary` (`0x0006`).
*
* A last-resort KeyPackage is explicitly NOT single-use: OpenMLS skips
* `delete_key_package` for one, and MDK — which marks every KeyPackage it
* publishes last resort — invites from the copy cached in its directory.
* Anything that consumes a KeyPackage has to check this before discarding
* the bundle.
*/
fun isLastResort(): Boolean =
extensions.any { it.extensionType == LAST_RESORT_EXTENSION_TYPE } ||
AppDataDictionary.fromExtensionsOrEmpty(extensions).contains(AppComponentIds.LAST_RESORT_KEY_PACKAGE)
/**
* Encode the TBS (to-be-signed) portion for signature verification.
*/
@@ -118,6 +139,12 @@ data class MlsKeyPackage(
}
companion object {
/**
* `last_resort` KeyPackage extension (MLS Extensions draft). Marks a
* KeyPackage as reusable rather than single-use.
*/
const val LAST_RESORT_EXTENSION_TYPE = 0x000A
fun decodeTls(reader: TlsReader): MlsKeyPackage {
val version = reader.readUint16()
require(version == 1) { "Unsupported MLS version: $version" }
@@ -0,0 +1,256 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.marmot.mip00KeyPackages
import com.vitorpamplona.quartz.marmot.appComponents.CurrentProfileGroupFactory
import com.vitorpamplona.quartz.marmot.mls.tree.Extension
import com.vitorpamplona.quartz.nip01Core.crypto.KeyPair
import com.vitorpamplona.quartz.nip01Core.signers.NostrSignerInternal
import kotlinx.coroutines.runBlocking
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNotNull
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* A KeyPackage marked LastResort (MLS extension `0x000A`) is deliberately
* NOT single-use. OpenMLS says so in as many words — on the Welcome path it
* deletes the consumed bundle only `if !key_package_bundle.key_package().last_resort()`,
* and logs "KeyPackage has a last-resort marker, not deleting" otherwise — and
* MDK marks every KeyPackage it publishes as last resort, caches the peer
* KeyPackage it resolved in its user directory, and re-uses that same cached
* copy for every later invite of that peer.
*
* We publish the LastResort marker too (OpenMLS's own KeyPackage validation
* wants it). So we have to honour the contract we advertise: dropping the
* private keys the moment one Welcome consumed the KeyPackage makes every
* subsequent invite addressed to that same KeyPackage unjoinable, which is
* exactly what the MDK interop harness saw — one invite worked and the four
* that followed failed with "No matching KeyPackageBundle".
*/
class LastResortKeyPackageReuseTest {
private val identity = ByteArray(32) { 0x11 }
@Test
fun aLastResortKeyPackageStaysUsableAfterItIsConsumed() =
runBlocking {
val manager = KeyPackageRotationManager()
val slot = manager.getOrCreateSlotDTag("primary")
val bundle = manager.generateKeyPackage(identity, slot)
assertTrue(
"generateKeyPackage must mark the KeyPackage last-resort — MDK requires it",
bundle.keyPackage.isLastResort(),
)
val eventId = "a".repeat(64)
manager.recordPublishedEventId(slot, eventId)
manager.markConsumedByEventId(eventId)
assertNotNull(
"a last-resort KeyPackage must still resolve by event id after it was consumed",
manager.findBundleByEventId(eventId),
)
assertNotNull(
"a last-resort KeyPackage must still resolve by MLS KeyPackageRef after it was consumed",
manager.findBundleByRef(bundle.keyPackage.reference()),
)
assertTrue(
"consuming it still schedules a fresh publication for the slot",
manager.needsRotation(),
)
}
@Test
fun aSingleUseKeyPackageIsStillDroppedWhenConsumed() =
runBlocking {
val manager = KeyPackageRotationManager()
val slot = manager.getOrCreateSlotDTag("primary")
val bundle = manager.generateKeyPackage(identity, slot)
// Strip the LastResort marker: without it MLS single-use applies
// and forward secrecy says the init key must not survive.
val singleUse = bundle.copy(keyPackage = bundle.keyPackage.copy(extensions = emptyList()))
manager.installBundle(slot, singleUse)
assertFalse(singleUse.keyPackage.isLastResort())
val eventId = "b".repeat(64)
manager.recordPublishedEventId(slot, eventId)
manager.markConsumedByEventId(eventId)
assertNull(
"a KeyPackage without the last-resort marker is single-use and must be dropped",
manager.findBundleByEventId(eventId),
)
assertNull(manager.findBundleByRef(singleUse.keyPackage.reference()))
}
@Test
fun rotatingTheSlotKeepsTheConsumedKeyPackageReachable() =
runBlocking {
val manager = KeyPackageRotationManager()
val slot = manager.getOrCreateSlotDTag("primary")
val consumed = manager.generateKeyPackage(identity, slot)
val firstEventId = "c".repeat(64)
manager.recordPublishedEventId(slot, firstEventId)
manager.markConsumedByEventId(firstEventId)
// MIP-00 rotation publishes a replacement into the same d-tag slot.
val rotated = manager.rotateSlot(identity, slot)
val secondEventId = "d".repeat(64)
manager.recordPublishedEventId(slot, secondEventId)
assertEquals(
"the slot now serves the rotated KeyPackage",
rotated.keyPackage.reference().toList(),
manager
.findBundleByEventId(secondEventId)!!
.keyPackage
.reference()
.toList(),
)
assertEquals(
"an invite that still references the consumed KeyPackage must keep working",
consumed.keyPackage.reference().toList(),
manager
.findBundleByEventId(firstEventId)!!
.keyPackage
.reference()
.toList(),
)
}
@Test
fun retainedKeyPackagesAreBoundedAndDropTheOldestFirst() =
runBlocking {
val manager = KeyPackageRotationManager()
val slot = manager.getOrCreateSlotDTag("primary")
val eventIds = mutableListOf<String>()
repeat(KeyPackageRotationManager.MAX_RETAINED_BUNDLES + 2) { i ->
manager.generateKeyPackage(identity, slot)
val eventId = i.toString().padStart(64, '0')
eventIds.add(eventId)
manager.recordPublishedEventId(slot, eventId)
manager.markConsumedByEventId(eventId)
}
assertNull(
"the oldest consumed KeyPackage must age out of the bounded retention window",
manager.findBundleByEventId(eventIds.first()),
)
assertNotNull(
"the most recently consumed KeyPackage must still be reachable",
manager.findBundleByEventId(eventIds.last()),
)
}
@Test
fun retainedKeyPackagesSurviveAPersistenceRoundTrip() =
runBlocking {
val store = InMemoryKeyPackageBundleStore()
val manager = KeyPackageRotationManager(store)
val slot = manager.getOrCreateSlotDTag("primary")
val consumed = manager.generateKeyPackage(identity, slot)
val eventId = "e".repeat(64)
manager.recordPublishedEventId(slot, eventId)
manager.markConsumedByEventId(eventId)
manager.rotateSlot(identity, slot)
val restored = KeyPackageRotationManager(store)
restored.restoreFromStore()
assertEquals(
"a restart must not lose the private keys of a consumed last-resort KeyPackage",
consumed.keyPackage.reference().toList(),
restored
.findBundleByEventId(eventId)!!
.keyPackage
.reference()
.toList(),
)
}
/**
* The current profile does not use the MLS Extensions draft's `0x000A`
* extension type at all — it carries a `last_resort_key_package` component
* inside the KeyPackage-level `app_data_dictionary` (`0x0006`). Reading only
* the MIP-era carrier makes every KeyPackage we actually publish today look
* single-use, which is exactly the case the interop harness exercises.
*/
@Test
fun aCurrentProfileKeyPackageIsRecognisedAsLastResort() =
runBlocking {
val signer = NostrSignerInternal(KeyPair())
val manager = KeyPackageRotationManager()
val slot = manager.getOrCreateSlotDTag("primary")
val bundle = manager.generateCurrentProfileKeyPackage(signer, slot)
assertTrue(
"the current profile marks last resort with a dictionary component, not extension 0x000A",
bundle.keyPackage.isLastResort(),
)
val eventId = "f".repeat(64)
manager.recordPublishedEventId(slot, eventId)
manager.markConsumedByRef(bundle.keyPackage.reference())
assertNotNull(
"a consumed current-profile KeyPackage must stay reusable",
manager.findBundleByRef(bundle.keyPackage.reference()),
)
assertNotNull(manager.findBundleByEventId(eventId))
}
@Test
fun aCurrentProfileKeyPackageWithoutTheComponentIsSingleUse() =
runBlocking {
val signer = NostrSignerInternal(KeyPair())
val bundle = CurrentProfileGroupFactory.createKeyPackage(signer, lastResort = false)
assertFalse(bundle.keyPackage.isLastResort())
}
@Test
fun theLastResortMarkerIsReadFromTheKeyPackageExtensions() {
val manager = KeyPackageRotationManager()
val bundle = runBlocking { manager.generateKeyPackage(identity) }
assertTrue(bundle.keyPackage.isLastResort())
assertFalse(bundle.keyPackage.copy(extensions = emptyList()).isLastResort())
assertFalse(
bundle.keyPackage
.copy(extensions = listOf(Extension(extensionType = 0x0001, extensionData = ByteArray(0))))
.isLastResort(),
)
}
private class InMemoryKeyPackageBundleStore : KeyPackageBundleStore {
private var bytes: ByteArray? = null
override suspend fun load(): ByteArray? = bytes
override suspend fun save(data: ByteArray) {
bytes = data
}
override suspend fun delete() {
bytes = null
}
}
}