From a1f97a045213c4530e5d7c1750437cc753dfb4a8 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 23 Sep 2026 00:57:49 +0000 Subject: [PATCH] feat: Cantor roots and region keys, matching the reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 2 of the region-bag plan. Amethyst now derives the same §7.2 keys the rest of the network derives: the conformance harness is 12 of 12, its new section comparing 16 region keys and their coordinate decodes against cyberspace-cli — §9.8's london, nyc and origin plus §7.7's ideaspace point, at heights 0, 1, 4 and 8. A key is a consensus value, so that agreement is the whole point: a byte of difference is a bag they hid that we cannot open. CantorTree is §4.5 and §4.6 — the aligned subtree whose boundaries arithmetic fixes rather than anyone's movement, which is what lets two people in the same neighbourhood compute the same root without communicating, which is what makes §7 work at all. Folded leaf by leaf against a stack of partial roots rather than a level at a time: same root either way, but a level holds every leaf at once and the stack holds height + 1 numbers. It refuses above height 20 as both references do, because one height further is twice the leaves and a root twice as wide and the number is a stranger's to choose. The big integer went the long way round and the plan now records why. It was written portable and used everywhere, on the argument that two implementations of a consensus value is two chances to disagree. Measurement reversed that: portable Kotlin is 3 to 10 times slower than java.math.BigInteger on the operands a Cantor tree reaches, because multiplyToLen is a HotSpot intrinsic, and §7's feasibility is a number. So UBigInt is an expect class wrapping BigInteger on jvmAndroid and PortableUBigInt on nativeMain, which covers Apple and Linux together. A wrapper and not a typealias, for one reason: the reference hashes int_to_bytes_be_min, and BigInteger.toByteArray() is two's complement, so it grows a 0x00 sign byte whenever the top bit is set — half of all numbers. An alias would have fed that byte to SHA-256 and produced a key nobody else derives. What makes two implementations safe is that the disagreement is testable. PortableUBigIntDifferentialTest runs every operation against BigInteger over random inputs at fifteen widths from 0 to 352,000 bits, straddling the Karatsuba threshold both ways, and asserts the two actuals agree with each other — including on the bytes, which is where the alias would have gone wrong silently. CantorTreeBenchmark folds a whole subtree both ways and compares the roots, which is where a stack off-by-one would live rather than in the arithmetic, and keeps the cost on the record because the budget model the UI will quote is read straight off it. Also amy cyberspace coord|region, so the comparison runs in a shell script rather than only in a test, and one correction to the plan's cost model: the combine dominates above about height 8, roughly ten times an axis root at the same height, so a sweep priced by its tree builds under-quotes badly. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01JwXApJjoZYtkD3sPRWbPNa --- .../com/vitorpamplona/amethyst/cli/Main.kt | 3 + .../cli/commands/CyberspaceCommands.kt | 157 ++++++++++ cli/tests/sno/refdriver.py | 34 +++ cli/tests/sno/sno-conformance.sh | 43 +++ .../2026-09-22-cyberspace-region-bags.md | 62 +++- .../quartz/cyberspace/CantorTree.kt | 131 +++++++++ .../quartz/cyberspace/RegionKey.kt | 110 +++++++ .../quartz/utils/bigint/PortableUBigInt.kt | 276 ++++++++++++++++++ .../quartz/utils/bigint/UBigInt.kt | 84 ++++++ .../quartz/cyberspace/RegionKeyTest.kt | 146 +++++++++ .../quartz/utils/bigint/UBigInt.jvmAndroid.kt | 79 +++++ .../quartz/cyberspace/CantorTreeBenchmark.kt | 122 ++++++++ .../bigint/PortableUBigIntDifferentialTest.kt | 197 +++++++++++++ .../quartz/utils/bigint/UBigInt.native.kt | 63 ++++ 14 files changed, 1505 insertions(+), 2 deletions(-) create mode 100644 cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/CyberspaceCommands.kt create mode 100644 quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTree.kt create mode 100644 quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKey.kt create mode 100644 quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigInt.kt create mode 100644 quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.kt create mode 100644 quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKeyTest.kt create mode 100644 quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.jvmAndroid.kt create mode 100644 quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTreeBenchmark.kt create mode 100644 quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigIntDifferentialTest.kt create mode 100644 quartz/src/nativeMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.native.kt diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt index 5a82324fea..c7c06b1450 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt @@ -29,6 +29,7 @@ import com.vitorpamplona.amethyst.cli.commands.BuzzCommands import com.vitorpamplona.amethyst.cli.commands.ConcordCommands import com.vitorpamplona.amethyst.cli.commands.CountCommand import com.vitorpamplona.amethyst.cli.commands.CreateCommand +import com.vitorpamplona.amethyst.cli.commands.CyberspaceCommands import com.vitorpamplona.amethyst.cli.commands.DebitCommands import com.vitorpamplona.amethyst.cli.commands.DecodeCommand import com.vitorpamplona.amethyst.cli.commands.DecryptCommand @@ -244,6 +245,7 @@ private suspend fun dispatch(argv: Array): Int { "nip" -> return NipCommand.run(tail) "kind" -> return KindCommand.run(tail) "sno" -> return SnoCommands.dispatch(tail) + "cyberspace" -> return CyberspaceCommands.dispatch(tail) "namecoin" -> return NamecoinCommand.dispatch(tail) } @@ -515,6 +517,7 @@ private fun printUsage() { | nip list fetch the NIP index (README) from the repo | kind N|NAME look up an event kind's label + NIP (number, or search by name) | sno Simple Nostr Objects (DECK-0003): validate, price, verify an avatar + | cyberspace Cyberspace places and region keys (CYBERSPACE_V2 §2, §7.2) | namecoin resolve IDENT resolve a Namecoin identifier (.bit, d/, id/, alice@x.bit) | [--server URL[,URL]] to a Nostr pubkey + relays via the Namecoin blockchain | [--timeout SECS] (no account, talks to ElectrumX over TLS) diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/CyberspaceCommands.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/CyberspaceCommands.kt new file mode 100644 index 0000000000..99535387a7 --- /dev/null +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/CyberspaceCommands.kt @@ -0,0 +1,157 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.cli.commands + +import com.vitorpamplona.amethyst.cli.Args +import com.vitorpamplona.amethyst.cli.Output +import com.vitorpamplona.quartz.cyberspace.CantorTree +import com.vitorpamplona.quartz.cyberspace.CyberspaceCoordinate +import com.vitorpamplona.quartz.cyberspace.CyberspacePlane +import com.vitorpamplona.quartz.cyberspace.RegionKey + +/** + * `amy cyberspace …` — places and the keys they derive, local and accountless. + * + * Thin assembly only: §2's interleave, §4's Cantor trees and §7.2's derivation + * all live in quartz's `cyberspace/`. These verbs exist so the Kotlin + * implementation can be diffed against `cyberspace-cli` without a device, a + * relay or an account — the same arrangement `amy sno` has, and for the same + * reason. A region key is a consensus value: if ours differs from theirs by a + * byte, a bag they hid is one we cannot open, and that is a difference worth + * catching in a shell script rather than in a feed. + */ +object CyberspaceCommands { + val USAGE: String = + """ + |amy cyberspace — places and region keys (local, accountless) + | + | cyberspace coord COORD_HEX decode a coordinate: axes, plane, sectors (§2.2) + | cyberspace region COORD_HEX --height H the region key at that height (§7.2) + | + |COORD_HEX is 32 bytes of lowercase hex, as a `C` or `hint` tag carries it. + | + | --height H the aligned subtree height, 0..20 (default 0) + | --max-height H raise the refusal ceiling; the cost doubles and then + | some with every height, so this is deliberate work + """.trimMargin() + + suspend fun dispatch(tail: Array): Int = + route( + "cyberspace", + tail, + "cyberspace ", + mapOf( + "coord" to { rest -> coord(rest) }, + "region" to { rest -> region(rest) }, + ), + USAGE, + ) + + /** + * Decode a coordinate. The axes are reported as decimal strings because an + * 85-bit value does not fit any JSON number a reader can trust. + */ + private fun coord(rest: Array): Int { + val args = Args(rest) + val hex = args.positional.firstOrNull() ?: return Output.error("bad_args", "cyberspace coord COORD_HEX") + args.rejectUnknown() + + val point = CyberspaceCoordinate.decode(hex) ?: return Output.error("bad_args", "not a coordinate: 32 bytes of lowercase hex") + + Output.emit( + mapOf( + "coord" to hex, + "plane" to if (point.plane == CyberspacePlane.DATASPACE) "dataspace" else "ideaspace", + "x" to decimal(point.x.high, point.x.low), + "y" to decimal(point.y.high, point.y.low), + "z" to decimal(point.z.high, point.z.low), + "sector" to point.sector(), + ), + ) + return 0 + } + + /** + * The region key at a height: the same three fields `cyberspace-cli`'s + * `derive_region_key_material_for_height` returns, minus `region_n`, which + * is eleven kilobytes of hex by height 10 and is pinned exactly by the key + * that hashes it. + */ + private fun region(rest: Array): Int { + val args = Args(rest) + val hex = args.positional.firstOrNull() ?: return Output.error("bad_args", "cyberspace region COORD_HEX --height H") + val height = args.intFlag("height", 0) + val maxHeight = args.intFlag("max-height", CantorTree.DEFAULT_MAX_COMPUTE_HEIGHT) + args.rejectUnknown() + + val point = CyberspaceCoordinate.decode(hex) ?: return Output.error("bad_args", "not a coordinate: 32 bytes of lowercase hex") + if (height < 0) return Output.error("bad_args", "--height must be >= 0") + if (height > maxHeight) { + // The same refusal both references make, and for the same reason: + // one height further is twice the leaves and a root twice as wide. + return Output.error("too_big", "height $height exceeds max-height $maxHeight") + } + + val material = RegionKey.at(point, height, maxHeight) + Output.emit( + mapOf( + "coord" to hex, + "height" to height, + "region_bits" to material.regionN.bitLength, + "key" to material.decryptionKey.joinToString("") { (it.toInt() and 0xFF).toString(16).padStart(2, '0') }, + "lookup_id" to material.lookupId, + ), + ) + return 0 + } + + /** An 85-bit axis as decimal, from its two halves, without a big integer. */ + private fun decimal( + high: Long, + low: Long, + ): String { + // high * 2^64 + low, with low unsigned. Done in base 10^9 chunks so the + // CLI does not need arbitrary precision for a number it only prints. + var result = "0" + for (bit in 84 downTo 0) { + result = addDecimal(result, result) + val set = if (bit >= 64) (high ushr (bit - 64)) and 1L else (low ushr bit) and 1L + if (set == 1L) result = addDecimal(result, "1") + } + return result + } + + private fun addDecimal( + a: String, + b: String, + ): String { + val out = StringBuilder() + var carry = 0 + var i = a.length - 1 + var j = b.length - 1 + while (i >= 0 || j >= 0 || carry > 0) { + val sum = (if (i >= 0) a[i--] - '0' else 0) + (if (j >= 0) b[j--] - '0' else 0) + carry + out.append(('0' + sum % 10)) + carry = sum / 10 + } + return out.reverse().toString() + } +} diff --git a/cli/tests/sno/refdriver.py b/cli/tests/sno/refdriver.py index f84fb122f1..045ff659a7 100755 --- a/cli/tests/sno/refdriver.py +++ b/cli/tests/sno/refdriver.py @@ -12,6 +12,7 @@ and payment reference), and just re-emits what they answer. refdriver.py verdict < payload {"valid":…, "rule":…} refdriver.py work < payload {"required":…} refdriver.py verify < event {"ok":…, "required":…, "committed":…, "zeros":…, "reason":…} + refdriver.py region COORD HEIGHT {"key":…, "lookup_id":…, "x":…, "y":…, "z":…, "plane":…} Paths come from $CYBERSPACE_DIR and $CYBERSPACE_CLI_DIR. """ @@ -94,8 +95,41 @@ def cmd_verify(): emit({k: result[k] for k in ("ok", "required", "committed", "zeros", "reason")}) +def cmd_region(): + """§2.2 decode plus §7.2 derivation, from cyberspace-cli's own modules. + + `location_encryption.py` imports AESGCM at module scope and that binding is + not always present; `cantor` and `movement` are the whole of what a region + key needs, and going through them keeps this an adapter rather than a + reimplementation. + """ + sys.path.insert(0, os.path.join(os.environ["CYBERSPACE_CLI_DIR"], "src")) + from cyberspace_core.coords import coord_to_xyz + from cyberspace_core.cantor import cantor_pair, int_to_bytes_be_min, sha256 + from cyberspace_core.movement import compute_subtree_cantor + + coord_hex, height = sys.argv[2], int(sys.argv[3]) + x, y, z, plane = coord_to_xyz(int(coord_hex, 16)) + base = lambda v: (v >> height) << height if height > 0 else v + region_n = cantor_pair( + cantor_pair( + compute_subtree_cantor(base(x), height), + compute_subtree_cantor(base(y), height), + ), + compute_subtree_cantor(base(z), height), + ) + key = sha256(int_to_bytes_be_min(region_n)) + emit({ + "key": key.hex(), + "lookup_id": sha256(key).hex(), + "x": str(x), "y": str(y), "z": str(z), + "plane": "ideaspace" if plane else "dataspace", + }) + + COMMANDS = { "cases": cmd_cases, + "region": cmd_region, "vectors": cmd_vectors, "verdict": cmd_verdict, "work": cmd_work, diff --git a/cli/tests/sno/sno-conformance.sh b/cli/tests/sno/sno-conformance.sh index c6c09a86f8..91edb41cb6 100755 --- a/cli/tests/sno/sno-conformance.sh +++ b/cli/tests/sno/sno-conformance.sh @@ -293,6 +293,49 @@ check_verdict "agreement-position-repair" "$FAR" true true "both repair a vertex WRAP='{"v":2,"name":"wrap","unit":0,"mode":"solid","vertices":[[0,0,0],[2,0,0],[1,0,2],[17895698,2,1]],"colors":[238,235,239,225],"faces":[[0,1,2]]}' check_verdict "divergence-lattice-limit" "$WRAP" false true "past what an Int lattice holds we refuse; the arbiter keeps it in a float" +# ---- 5. cyberspace region keys --------------------------------------------- + +banner "5. region keys — CYBERSPACE_V2 §2.2 and §7.2 against cyberspace-cli" + +# A region key is a consensus value: §7.2 turns it into an AES key, so a byte +# of difference is a bag they hid that we cannot open. Four coordinates from +# §9.8's golden vectors (plus §7.7's ideaspace point) at four heights each. +if [[ $HAVE_AVATAR_REF -eq 0 ]]; then + skip_msg "cyberspace-cli unavailable" + record_result "cyberspace-region-keys" skip "no cyberspace-cli checkout" +else + AGREE=0; DISAGREE=0 + for COORD in \ + c492492492492492492492edf5bee7267451c787d95ba4d7840c76d1e33c9940 \ + c4924924924924924924921f79235dae293ada913e78294253a235239a332854 \ + e000000000000000000001200041040208048040000000000000000000000000 \ + a4b64924924924924924924924924924924924924924924924924d84b60d9c8f + do + for H in 0 1 4 8; do + THEIRS="$(ref region "$COORD" "$H")" + OURS="$(amy cyberspace region "$COORD" --height "$H")" + T_KEY="$(jq -r .key <<<"$THEIRS")"; O_KEY="$(jq -r .key <<<"$OURS")" + T_ID="$(jq -r .lookup_id <<<"$THEIRS")"; O_ID="$(jq -r .lookup_id <<<"$OURS")" + # And the decode underneath it, which is where a wrong bit would start. + T_XYZ="$(jq -r '"\(.x) \(.y) \(.z) \(.plane)"' <<<"$THEIRS")" + O_XYZ="$(amy cyberspace coord "$COORD" | jq -r '"\(.x) \(.y) \(.z) \(.plane)"')" + + if [[ "$O_KEY" == "$T_KEY" && "$O_ID" == "$T_ID" && "$O_XYZ" == "$T_XYZ" ]]; then + AGREE=$((AGREE+1)) + else + DISAGREE=$((DISAGREE+1)) + fail_msg "${COORD:0:8}… h$H: key amy=${O_KEY:0:16} ref=${T_KEY:0:16}; xyz amy=[$O_XYZ] ref=[$T_XYZ]" + fi + done + done + + if [[ $DISAGREE -eq 0 && $AGREE -gt 0 ]]; then + record_result "cyberspace-region-keys" pass "$AGREE keys and decodes agree" + else + record_result "cyberspace-region-keys" fail "$DISAGREE of $((AGREE+DISAGREE)) diverged" + fi +fi + print_summary grep -q $'\tfail\t' "$RESULTS_FILE" && exit 1 diff --git a/quartz/plans/2026-09-22-cyberspace-region-bags.md b/quartz/plans/2026-09-22-cyberspace-region-bags.md index b72bf98860..0378f50c80 100644 --- a/quartz/plans/2026-09-22-cyberspace-region-bags.md +++ b/quartz/plans/2026-09-22-cyberspace-region-bags.md @@ -193,8 +193,8 @@ is not offered as a button at all — it is reported as out of reach. coordinate already there, which at least copies. Two shards in one sector would be worth saying; there are 21 reachable `3330`s and one author, so there is nothing to compare. Revisit when there is. -2. **Cantor + region key**, with the `expect`/`actual` bignum and the height refusal. - `amy cyberspace region`, diffed against `cyberspace-cli`. +2. **Cantor + region key**, with the big integer and the height refusal. + `amy cyberspace region`, diffed against `cyberspace-cli`. **Done** — see §9. 3. **Hints.** Parse, validate, price. `amy cyberspace hint`. The three golden vectors. 4. **The bag.** AES-256-GCM, plaintext shapes, item verification. `amy cyberspace open`, round-tripped against the reference CLI's `encrypt`. @@ -203,3 +203,61 @@ is not offered as a button at all — it is reported as out of reach. Steps 1 to 5 have no product risk and every one of them is diffable against a reference implementation. Step 6 is the only judgement call, and it is small. + + +## 9. Step 2 as built + +`CantorTree`, `RegionKey`, `UBigInt`, `amy cyberspace coord|region`. The +conformance harness is now 12 of 12, the new section comparing **16 region keys +and their coordinate decodes** against `cyberspace-cli` — four coordinates +(§9.8's london, nyc and origin, plus §7.7's ideaspace point) at heights 0, 1, 4 +and 8. Amethyst derives the keys the rest of the network derives. + +### The big integer, and why there are two of them + +The plan said "an `expect`/`actual` over `java.math.BigInteger`". The first +attempt went the other way — one portable implementation everywhere — on the +argument that a region key is a **consensus value**, since §7.2 turns it into an +AES key, so two implementations is two chances to disagree and an object that +opens on a desktop and not on a phone. + +Measurement reversed that. Portable Kotlin came in **3 to 10 times slower** than +`java.math.BigInteger` on the operands a Cantor tree reaches, because +`BigInteger.multiplyToLen` is a HotSpot intrinsic and the JDK adds Toom-Cook +above a few hundred limbs. §7's whole feasibility is a number, and that factor +is the difference between a search a reader waits for and one they abandon. + +So: `UBigInt` is an `expect class`, aliased through a thin wrapper to +`java.math.BigInteger` on `jvmAndroid`, and backed by `PortableUBigInt` on +`nativeMain` — which covers Apple and Linux together, so there are two actuals +and not three. A wrapper rather than a `typealias` for one reason: +`toMinimalBytes`. The reference hashes `int_to_bytes_be_min`, and +`BigInteger.toByteArray()` is two's complement, so it grows a `0x00` sign byte +whenever the top bit is set — half of all numbers — and aliasing would have put +that byte into a SHA-256 and produced a key nobody else derives. + +**What makes two implementations safe is that the disagreement is testable, and +tested.** `PortableUBigIntDifferentialTest` runs every operation against +`java.math.BigInteger` over random inputs at fifteen widths from 0 to 352,000 +bits, straddling the Karatsuba threshold in both directions, and a ninth test +asserts the two *actuals* agree with each other on the same inputs — including +the bytes, which is the one place aliasing would have gone wrong silently. +`CantorTreeBenchmark` then folds a whole subtree both ways and compares the +roots, which is where an off-by-one in the fold's stack would live rather than +in the arithmetic. + +### What it costs, on the shipped path + +The §3 table was measured on `java.math.BigInteger`, which is what now ships on +JVM and Android, so it stands. Apple and Linux pay the portable multiplier on +top; nothing there opens a bag yet. + +### Corrections to §3 worth carrying forward + +The earlier measurement said a sweep decomposes per axis, `3 · 2^(G/3)` tree +builds and `2^G` combines. Building it confirmed the decomposition and also that +**the combine dominates above about height 8** — a combine at height 12 is +roughly ten times an axis root at the same height, because it multiplies two +numbers the size of the root rather than folding up to one. A budget model that +prices a sweep by its tree builds will under-quote badly; price it by `2^G` +combines and add the trees. diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTree.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTree.kt new file mode 100644 index 0000000000..46668ba924 --- /dev/null +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTree.kt @@ -0,0 +1,131 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.cyberspace + +import com.vitorpamplona.quartz.utils.bigint.UBigInt + +/** + * `CYBERSPACE_V2.md` §4.5 and §4.6: the Cantor number of an aligned subtree. + * + * A region is not a box somebody drew, it is an **aligned** subtree: its base is + * a multiple of `2^h` and it covers exactly `2^h` leaves, so the boundaries are + * fixed by arithmetic rather than by anyone's movement. That is the whole reason + * §7's location keys work — "Two people standing in the same neighbourhood will + * compute the same Cantor root without ever communicating, because they are both + * computing the root of the same aligned subtree." + * + * The root is that subtree's leaves paired bottom-up until one number is left. + * It is `O(2^h)` and there is no closed form; the numbers double in width at + * every level, so a root at height `h` runs to about `86 · 2^h` bits and the + * cost grows by roughly 2.2x per height — twice the leaves times wider operands. + * That is not an implementation detail to optimise away, it is the *point*: §7.1 + * makes the work the price of admission, and "looking and walking cost the same" + * only because this is expensive. + */ +object CantorTree { + /** + * The tallest subtree this will build, matching `cyberspace-core`'s + * `DEFAULT_MAX_COMPUTE_HEIGHT` and `cyberspace-cli`'s `max_compute_height`. + * + * Both references refuse rather than try, and so does this: one height past + * it is twice the leaves and a root twice as wide, and the difference + * between a request that takes a minute and one that takes the afternoon is + * a single integer a stranger chose. A caller that wants more says so. + */ + const val DEFAULT_MAX_COMPUTE_HEIGHT = 20 + + /** + * §4.6: `cantor_pair(a, b) = (a + b)(a + b + 1) / 2 + b`. + * + * A bijection on pairs of non-negative integers, which is what makes a root + * identify one region and no other. The halving is exact because `s(s + 1)` + * is a product of consecutive integers and therefore even. + */ + fun cantorPair( + a: UBigInt, + b: UBigInt, + ): UBigInt { + val sum = a + b + return (sum * (sum + UBigInt.ONE)).shiftRight(1) + b + } + + /** + * The root of the aligned subtree of [height] whose lowest leaf is [base]. + * + * Folded leaf by leaf against a stack of partial roots rather than a level + * at a time. The root is the same either way — a pairing at level `k` joins + * the same two subtrees in the same order whichever way the tree is walked — + * but a level at a time holds every leaf at once, a million of them at + * height 20, while the stack holds at most `height + 1` numbers, which + * together come to about one level's worth. + */ + fun subtreeRoot( + base: UBigInt, + height: Int, + maxComputeHeight: Int = DEFAULT_MAX_COMPUTE_HEIGHT, + ): UBigInt { + require(height >= 0) { "height must be >= 0" } + require(height <= maxComputeHeight) { "height $height exceeds maxComputeHeight $maxComputeHeight" } + if (height == 0) return base + + val values = arrayOfNulls(height + 1) + val levels = IntArray(height + 1) + var top = 0 + + val leaves = 1L shl height + for (i in 0 until leaves) { + var value = base + UBigInt.of(i) + var level = 0 + while (top > 0 && levels[top - 1] == level) { + top-- + value = cantorPair(values[top]!!, value) + level++ + } + values[top] = value + levels[top] = level + top++ + } + return values[0]!! + } + + /** + * §4.5: the height of the smallest aligned subtree holding both values — + * `bit_length(v1 XOR v2)`, which is how far up the tree they first meet. + */ + fun lcaHeight( + a: CyberspaceAxis, + b: CyberspaceAxis, + ): Int { + val high = a.high xor b.high + val low = a.low xor b.low + return if (high != 0L) Long.SIZE_BITS + bitLength(high) else bitLength(low) + } + + private fun bitLength(value: Long): Int { + var bits = 0 + var v = value + while (v != 0L) { + bits++ + v = v ushr 1 + } + return bits + } +} diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKey.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKey.kt new file mode 100644 index 0000000000..a690d70f1b --- /dev/null +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKey.kt @@ -0,0 +1,110 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.cyberspace + +import com.vitorpamplona.quartz.nip01Core.core.HexKey +import com.vitorpamplona.quartz.nip01Core.core.toHexKey +import com.vitorpamplona.quartz.utils.bigint.UBigInt +import com.vitorpamplona.quartz.utils.sha256.sha256 + +/** What deriving a region at one height produces (`CYBERSPACE_V2.md` §7.2). */ +class RegionKeyMaterial( + val height: Int, + /** §4.7's stable spatial region integer. */ + val regionN: UBigInt, + /** The 32 bytes a bag's payload is encrypted under. Not to be published. */ + val decryptionKey: ByteArray, + /** The `d` tag a bag carrying this region is addressed by. Safe to publish. */ + val lookupId: HexKey, +) + +/** + * `CYBERSPACE_V2.md` §7.2 — turning a place into a key. + * + * ``` + * region_bytes = int_to_bytes_be_min(region_n) + * location_decryption_key = sha256(region_bytes) + * lookup_id = sha256(location_decryption_key) + * ``` + * + * **Two layers, and the second one is the design.** The lookup id is published + * so that people can find the content; it is a hash *of* the key, so seeing it + * buys nothing without the region preimage. §7.2: "Seeing `lookup_id` does not + * allow deriving `location_decryption_key` without the region preimage. The + * lookup ID is safe to publish; the decryption key requires work." + * + * Note what is deliberately absent: the temporal axis that hop proofs use (§5) + * is not part of this. Location identifiers stay a stable function of space, so + * they do not change when somebody walks through. + */ +object RegionKey { + /** + * §7.4: the region integer for the aligned cube of [height] holding [point]. + * + * The three axes are independent all the way to the combine, which is why a + * §7.7 sweep of a box costs one tree per distinct base *per axis* rather + * than one per candidate region. + */ + fun regionN( + point: CyberspacePoint, + height: Int, + maxComputeHeight: Int = CantorTree.DEFAULT_MAX_COMPUTE_HEIGHT, + ): UBigInt { + val base = point.alignedBase(height) + val x = CantorTree.subtreeRoot(base.x.toUBigInt(), height, maxComputeHeight) + val y = CantorTree.subtreeRoot(base.y.toUBigInt(), height, maxComputeHeight) + val z = CantorTree.subtreeRoot(base.z.toUBigInt(), height, maxComputeHeight) + // §4.7: region_n = pi(pi(cantor_x, cantor_y), cantor_z). + return CantorTree.cantorPair(CantorTree.cantorPair(x, y), z) + } + + /** The key and the lookup id a region integer yields. */ + fun derive( + regionN: UBigInt, + height: Int = 0, + ): RegionKeyMaterial { + val key = sha256(regionN.toMinimalBytes()) + return RegionKeyMaterial(height, regionN, key, sha256(key).toHexKey()) + } + + /** Both halves at once, for the common case. */ + fun at( + point: CyberspacePoint, + height: Int, + maxComputeHeight: Int = CantorTree.DEFAULT_MAX_COMPUTE_HEIGHT, + ): RegionKeyMaterial = derive(regionN(point, height, maxComputeHeight), height) +} + +/** + * This axis as a number the Cantor tree can add to. + * + * The one conversion from the split representation into arbitrary precision, + * done at the boundary where arithmetic starts and nowhere earlier — the same + * rule the SNO lattice follows for the same reason. + */ +fun CyberspaceAxis.toUBigInt(): UBigInt { + val bytes = ByteArray(16) + for (i in 0..7) { + bytes[i] = (high ushr (56 - i * 8)).toByte() + bytes[8 + i] = (low ushr (56 - i * 8)).toByte() + } + return UBigInt.ofBytes(bytes) +} diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigInt.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigInt.kt new file mode 100644 index 0000000000..ad3684671e --- /dev/null +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigInt.kt @@ -0,0 +1,276 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.utils.bigint + +/** + * A non-negative integer of arbitrary size, in portable Kotlin: the engine + * behind [UBigInt] everywhere Java's is not available. + * + * On JVM and Android [UBigInt] wraps `java.math.BigInteger`, whose + * `multiplyToLen` is a HotSpot intrinsic and which measured 3 to 10 times + * faster than this on the operands a Cantor tree reaches. That speed matters — + * `CYBERSPACE_V2.md` §7's whole feasibility argument is a number — but Apple + * and Linux have nothing to wrap, so this exists and has to be exactly as + * correct, because what it computes becomes a decryption key (§7.2). A limb + * handled differently here is an object that opens on a desktop and not on a + * phone. + * + * That risk is discharged by measurement rather than by argument: + * `PortableUBigIntDifferentialTest` runs every operation against + * `java.math.BigInteger` over random inputs at the sizes the tree reaches, and + * `CantorTreeBenchmark` folds a whole subtree both ways and compares the roots. + * + * **Unsigned by construction.** Nothing here ever goes negative: the Cantor + * pairing of two non-negative numbers is non-negative, and [subtract] is used + * only inside Karatsuba where the result is known to be. That removes sign + * handling, two's complement, and the whole class of bug where a leading zero + * byte does or does not appear. + * + * Limbs are 32 bits each, little-endian — `limbs[0]` is the least significant — + * and normalised so the top limb is never zero. Zero is the empty array. + */ +internal class PortableUBigInt internal constructor( + internal val limbs: IntArray, +) : Comparable { + /** How many bits this number occupies; 0 for zero. */ + val bitLength: Int + get() { + if (limbs.isEmpty()) return 0 + val top = limbs[limbs.size - 1] + return limbs.size * Int.SIZE_BITS - leadingZeros(top) + } + + val isZero: Boolean get() = limbs.isEmpty() + + operator fun plus(other: PortableUBigInt): PortableUBigInt { + if (isZero) return other + if (other.isZero) return this + val long = if (limbs.size >= other.limbs.size) limbs else other.limbs + val short = if (limbs.size >= other.limbs.size) other.limbs else limbs + val out = IntArray(long.size + 1) + var carry = 0L + for (i in long.indices) { + val sum = (long[i].toLong() and MASK) + (if (i < short.size) short[i].toLong() and MASK else 0L) + carry + out[i] = sum.toInt() + carry = sum ushr Int.SIZE_BITS + } + out[long.size] = carry.toInt() + return normalised(out) + } + + /** + * `this - other`, which the caller guarantees is not negative. + * + * Only Karatsuba calls this, on `(a0 + a1)(b0 + b1) - z0 - z2`, which is a + * cross term and cannot go below zero. A borrow off the end would mean the + * multiplication itself was wrong, so it is an error rather than a wrap. + */ + internal fun subtract(other: PortableUBigInt): PortableUBigInt { + if (other.isZero) return this + val out = IntArray(limbs.size) + var borrow = 0L + for (i in limbs.indices) { + val diff = (limbs[i].toLong() and MASK) - (if (i < other.limbs.size) other.limbs[i].toLong() and MASK else 0L) - borrow + out[i] = diff.toInt() + borrow = if (diff < 0) 1L else 0L + } + check(borrow == 0L) { "unsigned subtract went below zero" } + return normalised(out) + } + + operator fun times(other: PortableUBigInt): PortableUBigInt { + if (isZero || other.isZero) return ZERO + if (limbs.size < KARATSUBA_LIMBS || other.limbs.size < KARATSUBA_LIMBS) { + return schoolbook(limbs, other.limbs) + } + return karatsuba(this, other) + } + + /** This number with its low [bits] bits dropped. */ + fun shiftRight(bits: Int): PortableUBigInt { + require(bits >= 0) { "shift must not be negative" } + if (bits == 0 || isZero) return this + val wholeLimbs = bits / Int.SIZE_BITS + if (wholeLimbs >= limbs.size) return ZERO + val withinLimb = bits % Int.SIZE_BITS + val out = IntArray(limbs.size - wholeLimbs) + if (withinLimb == 0) { + limbs.copyInto(out, 0, wholeLimbs, limbs.size) + } else { + for (i in out.indices) { + val low = limbs[i + wholeLimbs] ushr withinLimb + val high = if (i + wholeLimbs + 1 < limbs.size) limbs[i + wholeLimbs + 1] shl (Int.SIZE_BITS - withinLimb) else 0 + out[i] = low or high + } + } + return normalised(out) + } + + /** + * The `int_to_bytes_be_min` of the reference implementation: big-endian, no + * leading zero byte, and a single zero byte for zero. + * + * This exact shape is what §7.2 hashes, so a spare leading byte — which is + * what `java.math.BigInteger.toByteArray()` adds whenever the top bit is + * set — would silently produce a different key for one number in two. + */ + fun toMinimalBytes(): ByteArray { + if (isZero) return byteArrayOf(0) + val bytes = (bitLength + 7) / 8 + val out = ByteArray(bytes) + for (i in 0 until bytes) { + val limb = limbs[i / 4] + out[bytes - 1 - i] = (limb ushr ((i % 4) * 8)).toByte() + } + return out + } + + override fun compareTo(other: PortableUBigInt): Int { + if (limbs.size != other.limbs.size) return if (limbs.size < other.limbs.size) -1 else 1 + for (i in limbs.indices.reversed()) { + val a = limbs[i].toLong() and MASK + val b = other.limbs[i].toLong() and MASK + if (a != b) return if (a < b) -1 else 1 + } + return 0 + } + + override fun equals(other: Any?): Boolean = other is PortableUBigInt && limbs.contentEquals(other.limbs) + + override fun hashCode(): Int = limbs.contentHashCode() + + override fun toString(): String = "PortableUBigInt($bitLength bits)" + + companion object { + private const val MASK = 0xFFFFFFFFL + + /** + * Below this many limbs on either side, schoolbook wins: Karatsuba's + * three sub-products and their adds cost more than the `n * m` limb + * multiplications they save. The exact crossover is not sensitive — + * anything in the tens works — and this one is measured, not guessed. + */ + private const val KARATSUBA_LIMBS = 40 + + val ZERO = PortableUBigInt(IntArray(0)) + val ONE = PortableUBigInt(intArrayOf(1)) + + fun of(value: Long): PortableUBigInt { + require(value >= 0) { "negative values have no place on this lattice" } + if (value == 0L) return ZERO + val high = (value ushr Int.SIZE_BITS).toInt() + return if (high == 0) PortableUBigInt(intArrayOf(value.toInt())) else PortableUBigInt(intArrayOf(value.toInt(), high)) + } + + /** An unsigned big-endian magnitude, the inverse of [toMinimalBytes]. */ + fun ofBytes(bytes: ByteArray): PortableUBigInt { + if (bytes.isEmpty()) return ZERO + val out = IntArray((bytes.size + 3) / 4) + for (i in bytes.indices) { + val fromEnd = bytes.size - 1 - i + out[i / 4] = out[i / 4] or ((bytes[fromEnd].toInt() and 0xFF) shl ((i % 4) * 8)) + } + return normalised(out) + } + + private fun normalised(limbs: IntArray): PortableUBigInt { + var size = limbs.size + while (size > 0 && limbs[size - 1] == 0) size-- + if (size == 0) return ZERO + return PortableUBigInt(if (size == limbs.size) limbs else limbs.copyOf(size)) + } + + private fun leadingZeros(value: Int): Int { + if (value == 0) return Int.SIZE_BITS + var count = 0 + var v = value + while (v > 0) { + v = v shl 1 + count++ + } + return count + } + + private fun schoolbook( + a: IntArray, + b: IntArray, + ): PortableUBigInt { + val out = IntArray(a.size + b.size) + for (i in a.indices) { + val ai = a[i].toLong() and MASK + if (ai == 0L) continue + var carry = 0L + for (j in b.indices) { + val at = i + j + val product = ai * (b[j].toLong() and MASK) + (out[at].toLong() and MASK) + carry + out[at] = product.toInt() + carry = product ushr Int.SIZE_BITS + } + var at = i + b.size + while (carry != 0L) { + val sum = (out[at].toLong() and MASK) + carry + out[at] = sum.toInt() + carry = sum ushr Int.SIZE_BITS + at++ + } + } + return normalised(out) + } + + /** + * `a * b` in three half-width products instead of four. + * + * Splitting both at `half` limbs, `a = a1·B + a0` and `b = b1·B + b0`, + * the product is `z2·B² + z1·B + z0` where `z2 = a1·b1`, `z0 = a0·b0` + * and the cross term `z1 = (a0 + a1)(b0 + b1) − z2 − z0`, which is one + * multiplication rather than two. That turns the exponent from 2 to + * about 1.585, and on the numbers a Cantor tree reaches — megabytes by + * height 16 — it is the difference between a key and a coffee break. + */ + private fun karatsuba( + a: PortableUBigInt, + b: PortableUBigInt, + ): PortableUBigInt { + val half = maxOf(a.limbs.size, b.limbs.size) / 2 + val a0 = a.low(half) + val a1 = a.high(half) + val b0 = b.low(half) + val b1 = b.high(half) + + val z0 = a0 * b0 + val z2 = a1 * b1 + val z1 = ((a0 + a1) * (b0 + b1)).subtract(z2).subtract(z0) + + return z2.shiftLeftLimbs(half * 2) + z1.shiftLeftLimbs(half) + z0 + } + } + + private fun low(limbCount: Int): PortableUBigInt = if (limbs.size <= limbCount) this else normalised(limbs.copyOfRange(0, limbCount)) + + private fun high(limbCount: Int): PortableUBigInt = if (limbs.size <= limbCount) ZERO else normalised(limbs.copyOfRange(limbCount, limbs.size)) + + private fun shiftLeftLimbs(limbCount: Int): PortableUBigInt { + if (isZero || limbCount == 0) return this + val out = IntArray(limbs.size + limbCount) + limbs.copyInto(out, limbCount) + return PortableUBigInt(out) + } +} diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.kt new file mode 100644 index 0000000000..a81cbe97a7 --- /dev/null +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.kt @@ -0,0 +1,84 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.utils.bigint + +/** + * A non-negative integer of arbitrary size, with only the operations Cantor + * pairing needs (`CYBERSPACE_V2.md` §4.6). + * + * Two implementations, for one reason: speed on the platforms people use, and + * an implementation at all on the ones they might. JVM and Android wrap + * `java.math.BigInteger`, whose `multiplyToLen` is a HotSpot intrinsic and + * which measured 3 to 10 times faster than portable Kotlin on the operands a + * Cantor tree reaches — and §7's feasibility is a number, so that factor is the + * difference between a search a reader will wait for and one they will not. + * Apple and Linux have nothing to wrap and get [PortableUBigInt]. + * + * Two implementations of a **consensus value** would normally be a bad trade: + * §7.2 turns this into a decryption key, so a carry handled differently on one + * platform is an object that opens on a desktop and not on a phone. What makes + * it safe is that the disagreement is testable, and tested — every operation + * against `java.math.BigInteger` on random inputs at the sizes that matter, and + * a whole subtree folded both ways with the roots compared. + * + * Unsigned throughout: nothing on this lattice is negative, so there is no sign + * to carry and no two's complement to get wrong. + */ +expect class UBigInt : Comparable { + /** How many bits this number occupies; 0 for zero. */ + val bitLength: Int + + val isZero: Boolean + + operator fun plus(other: UBigInt): UBigInt + + operator fun times(other: UBigInt): UBigInt + + /** This number with its low [bits] bits dropped. */ + fun shiftRight(bits: Int): UBigInt + + /** + * The reference's `int_to_bytes_be_min`: big-endian, no leading zero byte, + * and a single zero byte for zero. + * + * This exact shape is what §7.2 hashes. `java.math.BigInteger.toByteArray()` + * is two's complement and prepends `0x00` whenever the top bit is set — + * half of all numbers — so the JVM actual strips it, and a test pins that + * it did. + */ + fun toMinimalBytes(): ByteArray + + override fun compareTo(other: UBigInt): Int + + override fun equals(other: Any?): Boolean + + override fun hashCode(): Int + + companion object { + val ZERO: UBigInt + val ONE: UBigInt + + fun of(value: Long): UBigInt + + /** An unsigned big-endian magnitude, the inverse of [toMinimalBytes]. */ + fun ofBytes(bytes: ByteArray): UBigInt + } +} diff --git a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKeyTest.kt b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKeyTest.kt new file mode 100644 index 0000000000..8fa81b7469 --- /dev/null +++ b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/cyberspace/RegionKeyTest.kt @@ -0,0 +1,146 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.cyberspace + +import com.vitorpamplona.quartz.nip01Core.core.toHexKey +import com.vitorpamplona.quartz.utils.bigint.UBigInt +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertNotNull +import kotlin.test.assertTrue + +/** + * §4's Cantor roots and §7.2's key derivation, against the reference + * implementation. + * + * Every expectation below came out of `cyberspace-cli`'s own + * `compute_subtree_cantor`, `cantor_pair` and `int_to_bytes_be_min`, run over + * the four coordinates at five heights each. They are keys: if this file passes, + * a bag the reference hid is one Amethyst can open, and if it fails they are two + * different networks that happen to share a spec. + * + * The **key** is the vector rather than `region_n` because it pins `region_n` + * exactly in 32 bytes — a root at height 10 is eleven kilobytes of hex — and + * because it also pins the byte encoding, which is the part most likely to + * drift: the reference writes minimal big-endian, and a platform big integer + * that adds a sign byte produces a different key for one number in two. + */ +class RegionKeyTest { + /** `name | height | key | lookup_id`, from the reference. */ + private val reference = + listOf( + "london|0|c9fed7928825bb4192386ad339bfe73df70de5918e4395f29a97762458bfb972|c55b1e044f016744c45b6dc8080e6e3825f9d88b412e2d1a6becad4de8c5f5ef", + "london|1|6ea9f6b310442cced2ae706468aa90c9c5e217483b727b819c212c231f99b17f|24f83bebbf580937f066df5283abe7386f9abbc38f567b8442fdc7e591ffb053", + "london|4|314ade123f63601cc69a0addf455ea5c2d84ea848b84285aee5a00941e316ab4|a1d82532c354e690c6bffdb1fb20ccda716e037586feaf70092cbc442a635916", + "london|8|28939afc70712ce271b74ac7f5b9ed8339c955311cc289c516c63fa6e7a7a489|188ae4d5dccd60a27207f1d46a83959e002ec99cdeec717873ed5dc1a0fd5f69", + "london|10|83f219e105c3011e0f1b06f1f0fc053b437c2edb5826ff707dbb2cc8ef7dd550|a94422ee1b5423639dac61c203ee3c308c47b86470065587f38abf6a359d946e", + "nyc|0|1232b3a383492a4051fc7ee67d0332b42fcd113e834f05203f54c709e0f9bc78|80c73abdf82ab69d3e6795b13e0271321bf7ae1348007adf909b6b1e5f80ecfa", + "nyc|1|847cec141e1f645d296bd3c80bcc32f8b5ef60868f34533958d8917c117d8204|ee1802e91efeea676cc2955d9af0405dba74ad247c1833334efd2d52efb6cb04", + "nyc|4|bdb14b2c226797a79a2a808cbbf6126ddb5eb4bce27c8880c9bd668c853db4d1|4b3b0c9c1103ca97cff6124dce1752cdfec03a7358e28f6dd734ceb21e32ed3e", + "nyc|8|71cadf83822c69daafff9ea77ccfe4fb3b9d113b6f2e3bcd5f270862145807a6|1cf5bb35aa98c110b37add7d0bdfbd42474fe7c0cf523f569b06225ab8b0761b", + "nyc|10|a947d0322e2afad8c4852e2045c00f277bee246f41477212067b9c01f9cb9192|d464d48d1b85eb216d94b3b56654e8ce1a0f7b5c3e6a643096c88632e477aca4", + "origin|0|692329a8a9893517bfaaa0c1e27f597bc6b498d1371a4c866aa89424f54fe633|6eaedee401f51b2d54c51f86a603888dc1decb14dc872d26a0ffd6f4a4369136", + "origin|1|cb140125e63937b960fe6ed4a8ef6c9e801918c53a6b03b74897f245e01d12e4|21263dc0e2d46b450aa471aa34114ce21682818c5fbb285b06713f0ad3b26485", + "origin|4|a41a79fe0e41b1d5519fa71ab36fa0249f3411da72b810471e22274b09647de2|cb5e37e49dbdd02bae07347220635443739704d4177e84fa5b8ff8efbd2d8ce3", + "origin|8|fd1c6fc0d687db7edfc8747d5e844b79b3bfff1ebe206305e6f73d6ad27697fa|83ffb82c33758464c11482736525f14273eacaa4b0946119fb7202c04a7d30ec", + "origin|10|7d439417987c3b4c4cdc3bc9d76e1e09986817b9a5f6119d4f8fb4e858572ba3|5e6d4f1d57f9bc65bb07b6aeac02dd39f9a893790bb9788887d9d5f1fdad50c7", + "ideaspace|0|9d8de1257072196c30d90d540154a8259f1804dddc7dbcb3d99499cc48dec355|3350bd0e23d4b80b6a0e1070169f3e3252ce2d41e1b09134e14fea60fe7aa122", + "ideaspace|1|1f98a44ace1dbc67c7c45f5de8b010f6c551ba80f4022e9958c5eccf3640746c|ab1fda3988d871b25d45b93e28f971284e57da83ffb3e0e03c03fcbe2ede1ca8", + "ideaspace|4|bd26b0a550956d90161c21bc96e0adeb3ac262eea59345705eb26adf63345e56|c9d141d4f23590036f9cd3b82de1cd00faa512dc11937e5fc30b18e9ad382150", + "ideaspace|8|ee7c2c7af8c082705533436aa577e28f9020dc2eaf3bab4425c13560df67d306|2e82777cc6a759f52e76004b312806c1ddd2ee7642260f2cc3e52f1873b6aaef", + "ideaspace|10|da3f756d61498b59f00cc179a7c643e30d24395fc437091445f686bce64994fb|36b601485a2d28eb41b21bb20e05782106ae8753982196de5f95c3e03dad66a2", + ) + + private val coordinates = + mapOf( + "london" to "c492492492492492492492edf5bee7267451c787d95ba4d7840c76d1e33c9940", + "nyc" to "c4924924924924924924921f79235dae293ada913e78294253a235239a332854", + "origin" to "e000000000000000000001200041040208048040000000000000000000000000", + "ideaspace" to "a4b64924924924924924924924924924924924924924924924924d84b60d9c8f", + ) + + @Test + fun everyRegionKeyMatchesTheReference() { + for (row in reference) { + val (name, height, key, lookupId) = row.split("|") + val point = CyberspaceCoordinate.decode(coordinates.getValue(name)) + assertNotNull(point, name) + + val material = RegionKey.at(point, height.toInt()) + assertEquals(key, material.decryptionKey.toHexKey(), "$name at height $height: the key") + assertEquals(lookupId, material.lookupId, "$name at height $height: the lookup id") + } + } + + private operator fun List.component4() = this[3] + + @Test + fun theCantorPairIsTheOneTheSpecWrites() { + // §4.6: cantor_pair(a, b) = (a + b)(a + b + 1) / 2 + b, worked by hand. + // (3 + 5) * 9 / 2 + 5 = 41. + assertEquals(UBigInt.of(41), CantorTree.cantorPair(UBigInt.of(3), UBigInt.of(5))) + assertEquals(UBigInt.of(0), CantorTree.cantorPair(UBigInt.ZERO, UBigInt.ZERO)) + // It is a bijection, so no two pairs share a value. The classic witness + // is that it is not symmetric. + assertTrue(CantorTree.cantorPair(UBigInt.of(5), UBigInt.of(3)) != CantorTree.cantorPair(UBigInt.of(3), UBigInt.of(5))) + } + + @Test + fun aHeightOfZeroIsTheBaseItself() { + // The reference returns `base` unchanged at height 0, so a region of one + // leaf is that leaf's own number and costs nothing. + assertEquals(UBigInt.of(12345), CantorTree.subtreeRoot(UBigInt.of(12345), 0)) + } + + @Test + fun theSpecsOwnOneDimensionalExampleHolds() { + // §4.5: "LCA(0, 3) => subtree [0..3] => root = 228", and the same root + // for LCA(1, 2) and LCA(0, 2) — all three movements see one region, + // "which is exactly what enables location-based discovery". + assertEquals(UBigInt.of(228), CantorTree.subtreeRoot(UBigInt.ZERO, 2)) + } + + @Test + fun aSubtreeTallerThanTheCeilingIsRefusedRatherThanAttempted() { + // Both references raise instead of trying, and for the same reason: one + // height past the ceiling is twice the leaves and a root twice as wide, + // and the number is a stranger's to choose. + assertFailsWith { + CantorTree.subtreeRoot(UBigInt.ZERO, CantorTree.DEFAULT_MAX_COMPUTE_HEIGHT + 1) + } + assertFailsWith { CantorTree.subtreeRoot(UBigInt.ZERO, -1) } + // And a caller that means it can say so. + assertEquals(UBigInt.of(228), CantorTree.subtreeRoot(UBigInt.ZERO, 2, maxComputeHeight = 2)) + } + + @Test + fun theLcaHeightIsTheBitLengthOfTheDifference() { + // §4.5: "h = find_lca_height(v1, v2)", bit_length(v1 XOR v2). Moving + // from 0 to 5 gives 3, from 4 to 7 gives 2. + fun axis(v: Long) = CyberspaceAxis(0L, v) + assertEquals(3, CantorTree.lcaHeight(axis(0), axis(5))) + assertEquals(2, CantorTree.lcaHeight(axis(4), axis(7))) + assertEquals(0, CantorTree.lcaHeight(axis(9), axis(9))) + // Across the 64-bit split, where an axis stops fitting one Long. + assertEquals(65, CantorTree.lcaHeight(CyberspaceAxis(1L, 0L), CyberspaceAxis(0L, 0L))) + } +} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.jvmAndroid.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.jvmAndroid.kt new file mode 100644 index 0000000000..77099599bc --- /dev/null +++ b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.jvmAndroid.kt @@ -0,0 +1,79 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.utils.bigint + +import java.math.BigInteger + +/** + * [UBigInt] over `java.math.BigInteger`, which is what makes a region key + * affordable on the two platforms Amethyst actually ships to. + * + * A wrapper rather than a `typealias` for one reason: [toMinimalBytes]. The + * reference hashes `int_to_bytes_be_min`, and `BigInteger.toByteArray()` is + * two's complement, so it prepends a `0x00` sign byte whenever the top bit is + * set. Aliasing would put that byte into a SHA-256 for one number in two and + * produce a key nobody else derives. The allocation per operation is nothing + * against multiplications of megabyte operands. + */ +actual class UBigInt internal constructor( + internal val raw: BigInteger, +) : Comparable { + actual val bitLength: Int get() = raw.bitLength() + + actual val isZero: Boolean get() = raw.signum() == 0 + + actual operator fun plus(other: UBigInt): UBigInt = UBigInt(raw.add(other.raw)) + + actual operator fun times(other: UBigInt): UBigInt = UBigInt(raw.multiply(other.raw)) + + actual fun shiftRight(bits: Int): UBigInt { + require(bits >= 0) { "shift must not be negative" } + return UBigInt(raw.shiftRight(bits)) + } + + actual fun toMinimalBytes(): ByteArray { + if (raw.signum() == 0) return byteArrayOf(0) + val bytes = raw.toByteArray() + // Two's complement grows a leading zero exactly when the magnitude's + // top bit is set; the reference never writes one. + return if (bytes[0] == 0.toByte()) bytes.copyOfRange(1, bytes.size) else bytes + } + + actual override fun compareTo(other: UBigInt): Int = raw.compareTo(other.raw) + + actual override fun equals(other: Any?): Boolean = other is UBigInt && raw == other.raw + + actual override fun hashCode(): Int = raw.hashCode() + + override fun toString(): String = "UBigInt($bitLength bits)" + + actual companion object { + actual val ZERO: UBigInt = UBigInt(BigInteger.ZERO) + actual val ONE: UBigInt = UBigInt(BigInteger.ONE) + + actual fun of(value: Long): UBigInt { + require(value >= 0) { "negative values have no place on this lattice" } + return UBigInt(BigInteger.valueOf(value)) + } + + actual fun ofBytes(bytes: ByteArray): UBigInt = if (bytes.isEmpty()) ZERO else UBigInt(BigInteger(1, bytes)) + } +} diff --git a/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTreeBenchmark.kt b/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTreeBenchmark.kt new file mode 100644 index 0000000000..3f0e032cd2 --- /dev/null +++ b/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/cyberspace/CantorTreeBenchmark.kt @@ -0,0 +1,122 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.cyberspace + +import com.vitorpamplona.quartz.utils.bigint.UBigInt +import java.math.BigInteger +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * What a region key costs, and what writing the arithmetic cost against + * aliasing Java's. + * + * §7's whole feasibility argument is a number, and the plan's budget model — + * what a §7.7 sweep may be offered without asking — is read straight off it. A + * change that quietly makes a key five times slower does not break a test + * anywhere else; it makes a card lie about how long a search will take. So the + * number lives here. + * + * The comparison against `java.math.BigInteger` is the price of portability. + * [UBigInt] exists because a `expect`/`actual` would need a hand-written Apple + * and Linux implementation regardless, and two implementations of a consensus + * value is two chances to disagree — but Java's Toom-Cook is real and this + * measures how much of it was given up. + */ +class CantorTreeBenchmark { + private fun javaPair( + a: BigInteger, + b: BigInteger, + ): BigInteger { + val s = a.add(b) + return s.multiply(s.add(BigInteger.ONE)).shiftRight(1).add(b) + } + + private fun javaRoot( + base: BigInteger, + height: Int, + ): BigInteger { + if (height == 0) return base + val values = arrayOfNulls(height + 2) + val levels = IntArray(height + 2) + var top = 0 + for (i in 0 until (1L shl height)) { + var v = base.add(BigInteger.valueOf(i)) + var lvl = 0 + while (top > 0 && levels[top - 1] == lvl) { + top-- + v = javaPair(values[top]!!, v) + lvl++ + } + values[top] = v + levels[top] = lvl + top++ + } + return values[0]!! + } + + private fun millis(block: () -> Unit): Double { + val start = System.nanoTime() + block() + return (System.nanoTime() - start) / 1e6 + } + + @Test + fun aRegionKeyCostsWhatThePlanSaysItDoes() { + val point = CyberspaceCoordinate.decode("c492492492492492492492edf5bee7267451c787d95ba4d7840c76d1e33c9940")!! + repeat(30) { RegionKey.at(point, 6) } + repeat(30) { javaRoot(BigInteger.valueOf(123456789L), 6) } + + // The two costs a §7.7 sweep is made of: a box of gap G needs + // 3 * 2^(G/3) axis roots and 2^G combines, and the second dominates. + println("height | axis root ms | combine+2sha ms | java root ms | ratio") + for (height in listOf(8, 10, 12, 14)) { + val base = point.alignedBase(height).x.toUBigInt() + val ours = millis { CantorTree.subtreeRoot(base, height) } + val root = CantorTree.subtreeRoot(base, height) + val reps = if (height <= 10) 50 else 5 + val combine = millis { repeat(reps) { RegionKey.derive(CantorTree.cantorPair(CantorTree.cantorPair(root, root), root)) } } / reps + val java = millis { javaRoot(BigInteger(1, base.toMinimalBytes()), height) } + println("$height | ${fmt(ours)} | ${fmt(combine)} | ${fmt(java)} | ${fmt(ours / java)}x") + } + + // Loose on purpose — a shared box is noisy, and what would invalidate + // the budget model is an order of magnitude, not a busy minute. + val atEight = millis { RegionKey.at(point, 8) } + assertTrue(atEight < 500.0, "a height-8 key should be milliseconds, was ${fmt(atEight)} ms") + } + + @Test + fun theTwoImplementationsAgreeOnTheRootItself() { + // The differential test covers the arithmetic; this covers the fold + // built on it, which is where an off-by-one in the stack would live. + for (height in 0..12) { + // A base with bits set high and low, so the fold is not walking zeros. + val base = UBigInt.of((1L shl 62) + 1_234_567L + height) + val mine = CantorTree.subtreeRoot(base, height) + val theirs = javaRoot(BigInteger(1, base.toMinimalBytes()), height) + assertEquals(theirs, BigInteger(1, mine.toMinimalBytes()), "height $height") + } + } + + private fun fmt(value: Double) = ((value * 100).toLong() / 100.0).toString() +} diff --git a/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigIntDifferentialTest.kt b/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigIntDifferentialTest.kt new file mode 100644 index 0000000000..7bb3a7245e --- /dev/null +++ b/quartz/src/jvmTest/kotlin/com/vitorpamplona/quartz/utils/bigint/PortableUBigIntDifferentialTest.kt @@ -0,0 +1,197 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.utils.bigint + +import java.math.BigInteger +import kotlin.random.Random +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * [PortableUBigInt] against `java.math.BigInteger`, on random inputs, operation by + * operation. + * + * This is the whole safety argument for the second implementation. JVM and + * Android alias the platform's; Apple and Linux run this one. What it computes becomes a decryption key + * (`CYBERSPACE_V2.md` §7.2), so a carry dropped in one limb is not a wrong + * number, it is an object that will not open — and it would open everywhere + * else, because everyone else is running the reference. A hand-written example + * suite cannot cover a carry chain; a few thousand random pairs across the + * sizes the Cantor tree actually reaches can, and the oracle is the + * implementation the rest of the world's JVMs already agree with. + * + * The sizes are chosen to straddle the Karatsuba threshold in both directions + * and to include the degenerate ends, because that boundary is where a + * split-and-recombine bug lives. + */ +class PortableUBigIntDifferentialTest { + private val random = Random(20260923) + + private fun PortableUBigInt.toBig(): BigInteger = BigInteger(1, toMinimalBytes()) + + private fun randomOf(bits: Int): Pair { + if (bits == 0) return PortableUBigInt.ZERO to BigInteger.ZERO + val bytes = ByteArray((bits + 7) / 8) + random.nextBytes(bytes) + // Force the top bit so the value really is this wide, which is also the + // case where BigInteger's own toByteArray grows a sign byte. + bytes[0] = (bytes[0].toInt() or 0x80).toByte() + val mine = PortableUBigInt.ofBytes(bytes) + return mine to BigInteger(1, bytes) + } + + /** Sizes around the split threshold, plus the degenerate ends. */ + private val widths = listOf(0, 1, 7, 8, 31, 32, 33, 64, 127, 128, 1_200, 1_280, 1_281, 4_096, 9_001) + + @Test + fun addingAgrees() { + for (a in widths) { + for (b in widths) { + repeat(4) { + val (mineA, theirsA) = randomOf(a) + val (mineB, theirsB) = randomOf(b) + assertEquals(theirsA.add(theirsB), (mineA + mineB).toBig(), "$a + $b") + } + } + } + } + + @Test + fun multiplyingAgreesOnBothSidesOfTheKaratsubaThreshold() { + for (a in widths) { + for (b in widths) { + repeat(4) { + val (mineA, theirsA) = randomOf(a) + val (mineB, theirsB) = randomOf(b) + assertEquals(theirsA.multiply(theirsB), (mineA * mineB).toBig(), "$a * $b") + } + } + } + } + + @Test + fun multiplyingAgreesOnTheSizesACantorTreeReaches() { + // A root at height h is about 86 * 2^h bits, so these are the operands + // of the last few pairings at heights 12 to 16 — where Karatsuba + // recurses several levels deep and a mis-split would show. + for (bits in listOf(44_000, 88_000, 176_000, 352_000)) { + val (mineA, theirsA) = randomOf(bits) + val (mineB, theirsB) = randomOf(bits) + assertEquals(theirsA.multiply(theirsB), (mineA * mineB).toBig(), "$bits bits squared") + } + } + + @Test + fun shiftingRightAgrees() { + for (bits in widths) { + for (by in listOf(0, 1, 7, 31, 32, 33, 64, 1_000, 100_000)) { + val (mine, theirs) = randomOf(bits) + assertEquals(theirs.shiftRight(by), mine.shiftRight(by).toBig(), "$bits >> $by") + } + } + } + + @Test + fun comparingAndEqualityAgree() { + for (a in widths) { + for (b in widths) { + repeat(4) { + val (mineA, theirsA) = randomOf(a) + val (mineB, theirsB) = randomOf(b) + assertEquals(theirsA.compareTo(theirsB), mineA.compareTo(mineB), "$a <=> $b") + assertEquals(theirsA == theirsB, mineA == mineB) + } + } + } + } + + @Test + fun bitLengthAgrees() { + for (bits in widths) { + val (mine, theirs) = randomOf(bits) + assertEquals(theirs.bitLength(), mine.bitLength, "$bits") + } + } + + @Test + fun theMinimalBytesAreTheReferencesAndNotJavas() { + // The reference's `int_to_bytes_be_min` is `n.to_bytes((bit_length + 7) + // // 8, "big")`, with a single zero byte for zero. BigInteger's own + // toByteArray is two's complement and prepends 0x00 whenever the top + // bit is set, which is half of all numbers — and §7.2 hashes these + // bytes, so the spare byte would be a different key. + assertEquals(listOf(0), PortableUBigInt.ZERO.toMinimalBytes().toList()) + assertEquals(listOf(1), PortableUBigInt.ONE.toMinimalBytes().toList()) + assertEquals(listOf(-1), PortableUBigInt.of(255).toMinimalBytes().toList()) + assertEquals(listOf(1, 0), PortableUBigInt.of(256).toMinimalBytes().toList()) + + for (bits in widths.filter { it > 0 }) { + val (mine, theirs) = randomOf(bits) + val mineBytes = mine.toMinimalBytes() + // Same value, and never longer than the bit length demands. + assertEquals(theirs, BigInteger(1, mineBytes), "$bits round trip") + assertEquals((theirs.bitLength() + 7) / 8, mineBytes.size, "$bits has no spare byte") + assertTrue(mineBytes[0] != 0.toByte(), "$bits leads with a significant byte") + } + } + + @Test + fun theTwoActualsAgreeWithEachOther() { + // The contract between the platforms, asserted rather than assumed: on + // this JVM `UBigInt` is `java.math.BigInteger`, and on Apple and Linux + // it is `PortableUBigInt`. A region key derived on a phone has to equal + // one derived on a desktop, so the two must agree on every operation + // and, above all, on the bytes that get hashed. + for (a in widths) { + for (b in widths) { + val (mineA, theirsA) = randomOf(a) + val (mineB, theirsB) = randomOf(b) + val fastA = UBigInt.ofBytes(mineA.toMinimalBytes()) + val fastB = UBigInt.ofBytes(mineB.toMinimalBytes()) + + assertEquals((mineA + mineB).toMinimalBytes().toList(), (fastA + fastB).toMinimalBytes().toList(), "$a + $b") + assertEquals((mineA * mineB).toMinimalBytes().toList(), (fastA * fastB).toMinimalBytes().toList(), "$a * $b") + assertEquals(mineA.shiftRight(33).toMinimalBytes().toList(), fastA.shiftRight(33).toMinimalBytes().toList(), "$a >> 33") + assertEquals(mineA.compareTo(mineB), fastA.compareTo(fastB), "$a <=> $b") + assertEquals(mineA.bitLength, fastA.bitLength, "$a bits") + assertEquals(mineA.isZero, fastA.isZero, "$a zero") + // And the JVM actual must have stripped the sign byte its + // `toByteArray` grows, which is the one place aliasing would + // have silently changed a key. + assertEquals(theirsA.toString(16).trimStart('0').ifEmpty { "0" }, fastA.toMinimalBytes().toHex(), "$a bytes") + } + } + } + + private fun ByteArray.toHex(): String = joinToString("") { (it.toInt() and 0xFF).toString(16).padStart(2, '0') }.trimStart('0').ifEmpty { "0" } + + @Test + fun longsAndBytesRoundTrip() { + for (value in listOf(0L, 1L, 255L, 256L, Int.MAX_VALUE.toLong(), 1L shl 32, Long.MAX_VALUE)) { + assertEquals(BigInteger.valueOf(value), PortableUBigInt.of(value).toBig(), "$value") + } + for (bits in widths) { + val (mine, theirs) = randomOf(bits) + assertEquals(theirs, PortableUBigInt.ofBytes(mine.toMinimalBytes()).toBig(), "$bits") + } + } +} diff --git a/quartz/src/nativeMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.native.kt b/quartz/src/nativeMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.native.kt new file mode 100644 index 0000000000..f1890770f7 --- /dev/null +++ b/quartz/src/nativeMain/kotlin/com/vitorpamplona/quartz/utils/bigint/UBigInt.native.kt @@ -0,0 +1,63 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.utils.bigint + +/** + * [UBigInt] over [PortableUBigInt], for Apple and Linux, which have no big + * integer to borrow. + * + * Slower than the JVM's by 3 to 10 times on the operands a Cantor tree reaches, + * and identical in what it produces — which is the part that matters, because + * §7.2 turns it into a key. `PortableUBigIntDifferentialTest` is what holds the + * two together. + */ +actual class UBigInt internal constructor( + internal val raw: PortableUBigInt, +) : Comparable { + actual val bitLength: Int get() = raw.bitLength + + actual val isZero: Boolean get() = raw.isZero + + actual operator fun plus(other: UBigInt): UBigInt = UBigInt(raw + other.raw) + + actual operator fun times(other: UBigInt): UBigInt = UBigInt(raw * other.raw) + + actual fun shiftRight(bits: Int): UBigInt = UBigInt(raw.shiftRight(bits)) + + actual fun toMinimalBytes(): ByteArray = raw.toMinimalBytes() + + actual override fun compareTo(other: UBigInt): Int = raw.compareTo(other.raw) + + actual override fun equals(other: Any?): Boolean = other is UBigInt && raw == other.raw + + actual override fun hashCode(): Int = raw.hashCode() + + override fun toString(): String = "UBigInt($bitLength bits)" + + actual companion object { + actual val ZERO: UBigInt = UBigInt(PortableUBigInt.ZERO) + actual val ONE: UBigInt = UBigInt(PortableUBigInt.ONE) + + actual fun of(value: Long): UBigInt = UBigInt(PortableUBigInt.of(value)) + + actual fun ofBytes(bytes: ByteArray): UBigInt = UBigInt(PortableUBigInt.ofBytes(bytes)) + } +}