diff --git a/.claude/skills/android-expert/references/proguard-rules.md b/.claude/skills/android-expert/references/proguard-rules.md index b09f3e1b5d..185ef708f1 100644 --- a/.claude/skills/android-expert/references/proguard-rules.md +++ b/.claude/skills/android-expert/references/proguard-rules.md @@ -11,230 +11,178 @@ Proguard configuration for optimizing and obfuscating Android APK while preservi ## Amethyst Proguard Configuration -**File:** `amethyst/proguard-rules.pro` +**Files:** -### Keep Kotlin Metadata +- `amethyst/proguard-rules.pro` — the app's rules. Read it before adding + anything: it is commented rule by rule. +- `quartz/consumer-rules.pro` — merged into the R8 configuration of **every** + app that depends on Quartz, this one included (wired via + `optimization.consumerKeepRules` in `quartz/build.gradle.kts`). A rule added + here silently applies to somebody else's whole program. +- The AGP default `proguard-android-optimize.txt`, which already contributes the + Android-wide basics (Parcelable CREATOR fields, `native `, the enum + `values()`/`valueOf()` pair, …). Don't restate its rules. +- `commons/`, `commonsUI/` and the other library modules deliberately have **no** + rules files. They are not minified and they wire no consumer rules, so a file + there would be dead configuration. + +### The policy: keep only what is reached BY NAME + +Google Play measures how much of a shipped app's DEX R8 actually optimized and +renamed, and warns — then restricts store visibility and publishing — below 25% +in either category. Amethyst has been on the wrong side of that line: a +`-dontobfuscate` plus `-keepnames class ** { *; }` at the top of both +`proguard-rules.pro` and `quartz/consumer-rules.pro`, and outright +`-keep class com.vitorpamplona.** { *; }` for the app's own code, produced 0% +obfuscation and 13% optimization. (`-keepnames` is not the mild rule it looks +like: it expands to `-keep,allowshrinking`, which permits shrinking but neither +renaming nor optimization — applied to `**` it disables R8 for the entire +program, libraries included.) + +So the standing rule is: **a keep needs a named runtime mechanism that reads the +name.** In this app those are, exhaustively: + +| Mechanism | Example | Rule shape | +|---|---|---| +| JNI symbol `Java__` | `ArtiNative`, secp256k1 | covered by the default `native ` rule | +| Native code calling *back* by name | `ArtiLogCallback.onLogLine`, looked up with `GetMethodID` in `tools/arti-build/src/lib.rs` | `-keep class …ArtiLogCallback { *; }` | +| Generated code reflected over by a library | the AppFunctions bridge (`appfunctions.**`, play flavor only) | `-keep class .** { *; }` | +| Enum constant persisted as a string | `UISharedPreferences` writes `enum.name`, reads `Type.valueOf(s)` | `-keepclassmembers enum * { ; … }` | +| Class name in a manifest `` | `AmethystCastOptionsProvider` | explicit `-keep` — AGP generates keeps from component `android:name`, **not** from meta-data | +| Class name in WorkManager's database | the three `CoroutineWorker`s | `-keep class * extends androidx.work.ListenableWorker { (...); }` | + +What does **not** need a keep, and where the temptation usually comes from: + +- **Quartz events and tags.** `Event`, `Filter`, `Message`, `Command`, `Rumor`, + `EventTemplate`, `TagArray`, the NIP-46 Bunker messages and the NIP-55 intent + results all go through hand-written `StdSerializer`/`StdDeserializer` pairs + registered on `JacksonMapper` / `JsonMapperNip55`. Those read and write + property names as string literals, and `EventFactory` dispatches on kind with + a `when`, not by reflection. Renaming their fields changes nothing on the wire. +- **`@Serializable` (kotlinx) classes**, including the type-safe navigation + routes. The compiler plugin generates a descriptor holding the serial name and + every property name as **compile-time string literals**, so obfuscation cannot + reach them. kotlinx-serialization ships its own consumer rules for the + `$$serializer`/`Companion` plumbing. +- **Compose, Coil, OkHttp, Media3, Firebase, kotlin-reflect.** Every one of them + ships consumer rules inside its own artifact. Check + `build/outputs/mapping//configuration.txt` — the fully merged + configuration — before writing a rule for a third-party library. +- **Manifest-declared components** (activities, services, receivers, providers) + and classes named in layout/`res/xml`. AGP generates those keeps itself, which + is why `Intent().setClassName(ctx, "…NappletBrowserService")` is safe. + +### Attributes ```proguard -# Kotlin metadata is required for reflection --keep class kotlin.Metadata { *; } --keep class kotlin.** { *; } --dontwarn kotlin.** - -# Kotlin serialization --keepattributes *Annotation*, InnerClasses --dontnote kotlinx.serialization.AnnotationsKt --dontnote kotlinx.serialization.SerializationKt - --keep,includedescriptorclasses class com.vitorpamplona.**$$serializer { *; } --keepclassmembers class com.vitorpamplona.** { - *** Companion; -} --keepclasseswithmembers class com.vitorpamplona.** { - kotlinx.serialization.KSerializer serializer(...); -} -``` - -### Keep Nostr Event Classes - -```proguard -# Nostr events are serialized/deserialized --keep class com.vitorpamplona.quartz.events.** { *; } --keep class com.vitorpamplona.quartz.encoders.** { *; } - -# Keep event builders --keep class com.vitorpamplona.quartz.builders.** { *; } - -# Keep tag classes --keep class com.vitorpamplona.quartz.nip01Core.tags.** { *; } -``` - -### Keep Data Classes - -```proguard -# Data classes used in ViewModels and serialization --keep @kotlinx.serialization.Serializable class * { *; } - -# Keep all data classes --keep class com.vitorpamplona.amethyst.model.** { *; } --keep class com.vitorpamplona.amethyst.service.model.** { *; } -``` - -### Keep Compose Classes - -```proguard -# Jetpack Compose --keep class androidx.compose.** { *; } --dontwarn androidx.compose.** - -# Compose runtime --keep class androidx.compose.runtime.** { *; } - -# Compose UI --keep class androidx.compose.ui.** { *; } - -# Material3 --keep class androidx.compose.material3.** { *; } - -# Navigation Compose - Keep serializable routes --keep class * implements java.io.Serializable { *; } --keepclassmembers class * implements java.io.Serializable { - static final long serialVersionUID; - private static final java.io.ObjectStreamField[] serialPersistentFields; - !static !transient ; - private void writeObject(java.io.ObjectOutputStream); - private void readObject(java.io.ObjectInputStream); - java.lang.Object writeReplace(); - java.lang.Object readResolve(); -} -``` - -### Keep OkHttp/Retrofit - -```proguard -# OkHttp --dontwarn okhttp3.** --dontwarn okio.** --keep class okhttp3.** { *; } --keep class okio.** { *; } - -# OkHttp WebSockets (for Nostr relays) --keep class okhttp3.internal.ws.** { *; } - -# Retrofit (if used) --keepattributes Signature --keepattributes Exceptions --keep class retrofit2.** { *; } -``` - -### Keep Jackson (JSON) - -```proguard -# Jackson JSON library --keep class com.fasterxml.jackson.** { *; } --keep class org.codehaus.** { *; } --keepclassmembers class * { - @com.fasterxml.jackson.annotation.* ; -} - -# Jackson polymorphic types --keepattributes RuntimeVisibleAnnotations --keep @com.fasterxml.jackson.annotation.JsonTypeInfo class * -``` - -### Keep Secp256k1 (Crypto) - -```proguard -# Secp256k1 native library --keep class fr.acinq.secp256k1.** { *; } - -# Keep native methods --keepclasseswithmembernames class * { - native ; -} -``` - -### Keep Tor - -```proguard -# Tor library --keep class com.msopentech.thali.toronionproxy.** { *; } --dontwarn com.msopentech.thali.toronionproxy.** -``` - -### Keep ExoPlayer (Media) - -```proguard -# ExoPlayer (Media3) --keep class androidx.media3.** { *; } --dontwarn androidx.media3.** - --keep class com.google.android.exoplayer2.** { *; } --dontwarn com.google.android.exoplayer2.** -``` - -### Keep Coil (Image Loading) - -```proguard -# Coil image loading --keep class coil.** { *; } --keep class coil3.** { *; } --dontwarn coil.** --dontwarn coil3.** -``` - -### Keep ViewModels - -```proguard -# ViewModel classes --keep class * extends androidx.lifecycle.ViewModel { - (); -} - -# ViewModel factories --keep class * extends androidx.lifecycle.ViewModelProvider$Factory { - (...); -} - -# Keep ViewModel constructors for reflection --keepclassmembers class * extends androidx.lifecycle.ViewModel { - (...); -} -``` - -### Keep Parcelable - -```proguard -# Parcelable --keep class * implements android.os.Parcelable { - public static final android.os.Parcelable$Creator *; -} - --keepclassmembers class * implements android.os.Parcelable { - public ; - private ; -} -``` - -### Keep Enums - -```proguard -# Enums --keepclassmembers enum * { - public static **[] values(); - public static ** valueOf(java.lang.String); -} -``` - -### Remove Logging (Production) - -```proguard -# Remove debug logging in release builds --assumenosideeffects class android.util.Log { - public static *** d(...); - public static *** v(...); - public static *** i(...); -} - -# Keep error/warning logs --assumenosideeffects class android.util.Log { - public static *** e(...) return false; - public static *** w(...) return false; -} -``` - -### Keep Crashlytics/Firebase - -```proguard -# Firebase Crashlytics +# What makes a crash report retraceable. -keepattributes SourceFile,LineNumberTable --keep public class * extends java.lang.Exception -# Firebase --keep class com.google.firebase.** { *; } --dontwarn com.google.firebase.** +# jackson-module-kotlin reads @kotlin.Metadata; R8 requires InnerClasses and +# EnclosingMethod alongside Signature. +-keepattributes *Annotation*,Signature,Exceptions,InnerClasses,EnclosingMethod ``` +`LocalVariableTable`, `LocalVariableTypeTable`, `MethodParameters` and +`-keepparameternames` are debug metadata that nothing in the app reads — +jackson-module-kotlin takes parameter names from `@kotlin.Metadata`, not from +`MethodParameters`. They were removed; don't add them back. + +`-renamesourcefileattribute` is deliberately **not** set, and the reason is not +the usual one. + +Once R8 is minifying it rewrites every class's `SourceFile` to the marker +`r8-map-id-` on its own — there is no rule that restores the original +per-class `.kt` name. Verified by building it both ways: with +`-renamesourcefileattribute SourceFile`, all 24,440 classes report the literal +`"SourceFile"`; without it, they report the marker. So the rule cannot buy +readability, it can only *destroy* the marker — and that marker is the +`pg_map_id` header of the mapping that produced the build, which is what lets a +pasted stack trace name the exact mapping file it needs. + +Two related facts worth knowing before someone tries to "fix" stack traces with +keep rules: + +- **Raw line numbers are no longer source line numbers.** R8 renumbers them so + that one obfuscated line can encode a whole inlined frame stack. This is a + cost of *optimization*, not of renaming — it is new only because the old + blanket `-keepnames` had optimization switched off across the program, which + is the thing Play was flagging. +- **Retrace therefore returns more than the old raw traces did**: it expands + the frames R8 inlined instead of collapsing them into one misleading line. A + one-frame crash can retrace to three. + +The workflow is `scripts/retrace.sh [trace]`, documented in +[`RELEASE_OPS.md` § 7](../../../../RELEASE_OPS.md). Every GitHub Release carries +`amethyst-{googleplay,fdroid}-mapping-.txt.gz`; Play Console needs +nothing because AGP embeds the mapping in the `.aab` under +`BUNDLE-METADATA/com.android.tools.build.obfuscation/proguard.map`. + +### Verifying a change to these rules + +R8 cannot see reflection, so a wrong keep rule fails **only in a release build, +at runtime**. Before changing them: + +```bash +./gradlew :amethyst:assemblePlayRelease -PdisableAbiSplits=true -PdisableUniversalApk=true +``` + +then read `amethyst/build/outputs/mapping/playRelease/`: + +- `configuration.txt` — every rule R8 actually saw, including each AAR's + consumer rules. This is the file that answers "does library X already keep + itself?" +- `mapping.txt` — what got renamed. Lines whose left and right sides are equal + are classes a keep rule pinned; scan them for anything you did not intend. +- `seeds.txt` / `usage.txt` — what the keeps matched, and what was removed. + +### The contract check + +R8 cannot see reflection, so a keep rule that quietly stops matching — a class +moved to another package, a rule deleted in a merge, a DTO renamed — gives you a +green build and an APK that breaks on a user's device. +`tools/r8-verify/reflection-contract.txt` lists every name resolved from outside +the DEX, and the verifier asserts each against what R8 actually emitted: + +```bash +python3 tools/r8-verify/verify_reflection_contract.py \ + amethyst/build/outputs/mapping/playRelease/ +``` + +Under a second, no device. The release workflow runs it for both flavors before +collecting assets, so a break fails the release instead of shipping. + +**Adding reflection means adding two things**: the keep rule, and a line in the +contract. A rule with no contract line is unverified and will rot. + +Scope a line to one flavor with a leading `@play` / `@fdroid` when the class is +only compiled into that variant — play-only code is absent from the F-Droid APK, +and "absent" is indistinguishable from "R8 deleted it", so an unscoped line for +it fails that release. + +Reflection that names a class R8 *cannot* rename needs no rule and no line: the +platform trust manager `quic` probes for a 3-arg `checkServerTrusted` ships in +Android, not in our DEX. The opposite case is the one to watch — a name of +*ours* used as a value. `AmethystAppFunctions` compared +`signer::class.qualifiedName` against `"…NostrSignerExternal"`; R8 renames that +class, so the branch silently stopped running the day obfuscation was turned on. +Prefer `is` over a name comparison; there is no keep rule that makes the latter +safe to write. + +It reads both `mapping.txt` and `usage.txt`, because neither is enough alone — +mapping.txt records only what *changed* (an intact `-keep ... { *; }` class has +an empty body, and an unrenamed member has no line at all, so absence there +means "preserved"), while a member R8 *deleted* appears only in usage.txt. It +also cross-checks that the two files come from the same R8 run: mapping.txt is +written during packaging, not at minify time, so a minify-only rebuild leaves a +stale one behind next to a fresh usage.txt. + +What it cannot cover is reflection nobody wrote down. For that: a staged Play +rollout (the crash reporter retraces itself, so breaks are legible within +hours), and exercising NIP-47 wallet connect, NIP-46 bunker login, Tor, +scheduled posts and a settings round-trip (change theme/font, kill, relaunch) on +a minified build — those are the paths the keeps above exist for. + ## Build Configuration ### Enable R8 in build.gradle @@ -356,13 +304,15 @@ adb install app/build/outputs/apk/release/app-release.apk ### Issue: Compose Navigation Crashes -**Cause:** @Serializable route classes were obfuscated. +**Not** the route classes being obfuscated — that cannot happen. A type-safe +route's pattern comes from its kotlinx-serialization descriptor, and the compiler +plugin bakes the serial name and every property name in as string literals, so +renaming the class leaves the route string untouched. Adding +`-keep @kotlinx.serialization.Serializable class …routes.** { *; }` pins a large +tree for no reason and hides the real cause. -**Solution:** -```proguard -# Keep all route classes --keep @kotlinx.serialization.Serializable class com.vitorpamplona.amethyst.ui.navigation.routes.** { *; } -``` +Look instead at whether `navigation-common`'s own consumer rules made it into +`configuration.txt`, and at the stack trace retraced through `mapping.txt`. ### Issue: Native Library Crashes diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml index 78bb1aa0f6..4683d17921 100644 --- a/.github/workflows/create-release.yml +++ b/.github/workflows/create-release.yml @@ -1018,6 +1018,23 @@ jobs: env: BUILD_TOOLS_VERSION: "36.0.0" + # R8 cannot see reflection, so a keep rule that silently stops matching — a + # class moved to another package, a rule deleted, a DTO renamed — produces + # a green build and an APK that fails on a user's device. This asserts the + # reflective surface (JNI symbols, Jackson DTO field names, enum constants + # persisted in DataStore, WorkManager worker class names, the Cast + # OptionsProvider named in a manifest meta-data value) actually survived, + # against what R8 emitted for BOTH flavors. Runs before the assets are + # collected so a break fails the release rather than shipping. + - name: Verify R8 reflection contract + run: | + set -euo pipefail + for variant in playRelease fdroidRelease; do + echo "== $variant" + python3 tools/r8-verify/verify_reflection_contract.py \ + "amethyst/build/outputs/mapping/${variant}/" + done + - name: Collect Android assets (rename to canonical scheme) run: | set -euo pipefail @@ -1041,6 +1058,34 @@ jobs: "dist/amethyst-googleplay-${TAG}.aab" cp "amethyst/build/outputs/bundle/fdroidRelease/amethyst-fdroid-release.aab" \ "dist/amethyst-fdroid-${TAG}.aab" + + # R8 mapping files — the ONLY way a crash report from this build is + # ever readable again. The release build is minified, so every class + # in every artifact above reports `r8-map-id-` as its source + # file and a renamed class/method; `scripts/retrace.sh ` + # turns that back into real names, files and lines (and expands the + # frames R8 inlined). + # + # Play Console deobfuscates by itself because AGP embeds the mapping + # in the .aab it was given. Nothing else does: a trace from an + # F-Droid, Zapstore, Accrescent or GitHub-APK user is unreadable + # without the matching file, and the mapping only exists on this + # runner. If it is not published here it is gone when the job ends. + # + # Gzipped because the raw text mapping is ~500 MB (~29 MB + # compressed). retrace.sh reads the .gz directly. + for flavor in play fdroid; do + case "$flavor" in + play) name=googleplay ;; + fdroid) name=fdroid ;; + esac + src="amethyst/build/outputs/mapping/${flavor}Release/mapping.txt" + if [ ! -f "$src" ]; then + echo "::error::$src is missing — the release would ship with no way to read its crash reports." + exit 1 + fi + gzip -c "$src" > "dist/amethyst-${name}-mapping-${TAG}.txt.gz" + done ls -la dist # Accrescent does not accept AABs or monolithic APKs — it requires a signed diff --git a/BUILDING.md b/BUILDING.md index 5e7aaf68af..3e0d1a3658 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -362,7 +362,7 @@ Quartz library in one pipeline. 3. **Wait** for the `Create Release Assets` workflow to finish (~25–30 min). -4. **Verify** — the GH Release should hold **47 assets**: +4. **Verify** — the GH Release should hold **49 assets**: - **14 desktop**, one per matrix leg × format: - macOS arm64: `dmg` (1) - Windows x64: `msi` + portable `zip` (2) @@ -377,8 +377,12 @@ Quartz library in one pipeline. ships no WiX (`windows-latest` has WiX 3.14 preinstalled, which is why the x64 leg gets an MSI). Revisit if that image gains WiX, or if jpackage learns the WiX 4+ `wix build` CLI. - - **13 Android** — 5 Google Play APKs + 5 F-Droid APKs + 2 AABs + the - F-Droid `.apks` set built for Accrescent. + - **15 Android** — 5 Google Play APKs + 5 F-Droid APKs + 2 AABs + the + F-Droid `.apks` set built for Accrescent + **2 R8 mapping files** + (`amethyst-{googleplay,fdroid}-mapping-.txt.gz`). The mappings + are not optional extras: the release build is minified, so without them + no crash report from an APK/`.apks` user can be read. See + [`RELEASE_OPS.md` § Crash reports](RELEASE_OPS.md#7-crash-reports--retrace). - **10 amy** — `tar.gz` (macOS arm64, Linux x64, Linux arm64), `deb` + `rpm` per Linux arch, portable `zip` per Windows arch, and the one arch-independent no-JRE `amy--jvm.tar.gz` for Homebrew-core. diff --git a/RELEASE_OPS.md b/RELEASE_OPS.md index 202e9a2418..85414d10bb 100644 --- a/RELEASE_OPS.md +++ b/RELEASE_OPS.md @@ -130,6 +130,11 @@ Nothing to do beyond pushing the tag. Verify the asset count (BUILDING.md § Verify). macOS is **arm64-only** — there is no Intel DMG, so a single `amethyst-desktop--macos-arm64.dmg` is the expected, correct result. +Two of those assets are the R8 mapping files +(`amethyst-{googleplay,fdroid}-mapping-.txt.gz`). Do not prune them +from old releases — they are the only way to read a crash report from a build +that old (§ 7). + ### Google Play — manual upload 1. Download `amethyst-googleplay-.aab` from the GH Release. 2. Play Console → app `com.vitorpamplona.amethyst` → **Production** (or the @@ -304,7 +309,7 @@ Owner assignments and rotation reminders live with the team (issue tracker). ## 6. Post-release verification -- [ ] GH Release: 47 assets, sizes sane, and the asset-name set matches the +- [ ] GH Release: 49 assets, sizes sane, and the asset-name set matches the previous release (see the `diff` one-liner in BUILDING.md § Release runbook). macOS is arm64-only — do **not** look for an Intel DMG. - [ ] Maven Central: `quartz:` resolves (allow tens of minutes of @@ -325,3 +330,112 @@ Owner assignments and rotation reminders live with the team (issue tracker). see § 4); UnifiedPush still works on an `fdroid` build. If anything ships broken, see [`BUILDING.md` § Incident response](BUILDING.md#incident-response). + +--- + +## 7. Crash reports & retrace + +Release builds are minified **and obfuscated** (they have to be: Play Console +drops apps whose DEX is under 25% optimized or obfuscated out of store surfaces +— see `amethyst/proguard-rules.pro` for the whole story). So a raw stack trace +from a release build looks like this: + +``` +java.lang.IllegalStateException: something blew up + at onh.B(r8-map-id-12c710927a584543dbe1e2e867db95460bc44482efe53283f86798648c1cfc00:7) +``` + +That is not lost information, it is encoded information. Paste the report in and +run it: + +```bash +scripts/retrace.sh crash-report.txt +pbpaste | scripts/retrace.sh # or straight off the clipboard +``` + +Nothing else to supply. A report's first line names its own build — +`java.lang.IllegalStateException: 1.16.0-PLAY` — so the script resolves the tag +(`v1.16.0`) and the flavor (`PLAY` → `googleplay`), downloads that release's +mapping asset and caches it. For a bare stack trace with no such header, name +the build yourself with `--release v1.16.0 --flavor play`; for a build you made +locally, pass its `mapping.txt` directly. + +``` +java.lang.IllegalStateException: something blew up + at androidx.compose.foundation.text.input.TextFieldCharSequence.getText(TextFieldCharSequence.kt:58) + at androidx.compose.foundation.text.input.TextFieldState.getText(TextFieldState.kt:146) + at com.vitorpamplona.amethyst.ui.screen.loggedIn.home.ShortNotePostViewModel.onMessageChanged(ShortNotePostViewModel.kt:1727) +``` + +Note that retrace gave back **three** frames where the crash reported one: R8 +had inlined two of them. That is worth internalising — it is the reason a raw +trace's line number cannot be trusted even in the pre-obfuscation builds, where +optimization was already inlining. Retracing is not a tax obfuscation imposed; +it is how you read an optimized build at all. + +**The wrong mapping is worse than none** — it produces confident, wrong names. +So the script refuses to guess: the `r8-map-id-` in the trace *is* the +`pg_map_id` header of the mapping that built it, and the two are compared before +anything is printed: + +``` +error: this mapping did not build this report. + report: deadbeef... + mapping: 12c71092... (amethyst-googleplay-mapping-v1.16.0.txt.gz) +``` + +`--force` overrides if you really mean it. (`googleplay` vs `fdroid` matters — +the two flavors are separate R8 runs with different mappings.) + +**Per channel:** + +| Where the report came from | What to do | +|---|---| +| Play Console / Android vitals | Nothing. AGP embeds the mapping in the `.aab` (`BUNDLE-METADATA/com.android.tools.build.obfuscation/proguard.map`), so Play deobfuscates automatically. | +| A NIP-17 DM from the in-app crash reporter, a GitHub issue, F-Droid, Zapstore, Accrescent | `scripts/retrace.sh ` — it reads the build off line 1 and fetches the mapping. | +| A build you made locally | `scripts/retrace.sh amethyst/build/outputs/mapping//mapping.txt report.txt` | + +`scripts/retrace.sh` downloads the R8 version named in the mapping's own header +from Google's Maven and caches it, so it needs no pinned tooling and keeps +working across AGP bumps. It needs no `gh` auth either — release assets on a +public repo are plain HTTPS downloads. + +Two details of the report format in `ReportAssembler` exist for this and should +not be "tidied" away: the headline carries the **fully qualified** exception +class (a bare `simpleName` obfuscates to `a`, which retrace cannot resolve +because it has no package), and stack frames are written as ` at ` +(retrace only rewrites frames it recognises, and it recognises them by the +leading `at`). The script repairs the missing `at` on reports from older builds, +but new reports should not need repairing. + +**Do not delete mapping assets from old releases.** They are the only copy — +CI's are gone when the job ends, and a mapping cannot be regenerated after the +fact (it would need a bit-identical rebuild, and R8's renaming is not stable +across runs). + +### Did obfuscation break anything? + +R8 cannot see reflection, so nothing in the build tells you that a keep rule +stopped matching. `tools/r8-verify/reflection-contract.txt` lists every place +something outside the DEX resolves a name at runtime — JNI symbols, Jackson DTO +field names, enum constants persisted in DataStore, WorkManager's stored worker +class names, the Cast `OptionsProvider` named in a manifest `` value +— and the release workflow asserts each one against what R8 actually emitted, +for both flavors, before any asset is collected: + +```bash +python3 tools/r8-verify/verify_reflection_contract.py \ + amethyst/build/outputs/mapping/playRelease/ +``` + +Run it on any local minified build too. It takes under a second and needs no +device. + +**Adding reflection means adding two things**: the keep rule, and a line in the +contract. A rule with no contract line is unverified and will rot silently. + +What this does *not* cover is reflection nobody wrote down. For that the honest +controls are a staged Play rollout (the crash reporter retraces itself now, so +a break is legible within hours) and exercising NIP-47 wallet connect, NIP-46 +bunker login, Tor, scheduled posts and a settings round-trip on a minified +build before shipping. diff --git a/amethyst/proguard-rules.pro b/amethyst/proguard-rules.pro index e4d9f46dc4..3e4b2d898d 100644 --- a/amethyst/proguard-rules.pro +++ b/amethyst/proguard-rules.pro @@ -1,74 +1,140 @@ -# Add project specific ProGuard rules here. -# You can control the set of applied configuration files using the -# proguardFiles setting in build.gradle. +# ============================================================================= +# R8 configuration for the release build. # -# For more details, see -# http://developer.android.com/guide/developing/tools/proguard.html +# Two things are balanced here. +# +# 1. Google Play measures how much of the shipped DEX R8 actually optimized +# and renamed, and warns (then restricts visibility/publishing) below 25% +# in either category. This file used to open with `-dontobfuscate` plus +# `-keepnames class ** { *; }`, and then kept all of `com.vitorpamplona.**` +# outright. That is the whole app and every library: `-dontobfuscate` +# turns renaming off globally, and `-keepnames` is shorthand for +# `-keep,allowshrinking`, which permits shrinking but neither renaming nor +# optimization. Hence 0% obfuscation / 13% optimization. +# +# 2. Anything the runtime reaches by NAME rather than by reference has to keep +# that name: JNI symbols, Jackson's reflective data binding, enum constants +# persisted into DataStore, class names written into a manifest meta-data +# value or into WorkManager's database. +# +# So every keep below is scoped to (2), and R8 gets everything else. mapping.txt +# is uploaded with each release, so stack traces stay retraceable. +# +# When adding a keep, say in a comment WHAT reads the name at runtime. A keep +# without that is usually a keep that is not needed. +# ============================================================================= -# preserve the line number information for debugging stack traces. --dontobfuscate --keepattributes LocalVariableTable --keepattributes LocalVariableTypeTable --keepattributes *Annotation* --keepattributes SourceFile --keepattributes LineNumberTable --keepattributes Signature --keepattributes Exceptions --keepattributes InnerClasses --keepattributes EnclosingMethod --keepattributes MethodParameters --keepparameternames +# ----------------------------------------------------------------------------- +# Attributes +# ----------------------------------------------------------------------------- +# These two are what make a crash report readable again. Keep them. +# +# There is no setting that gives readable stack traces *in the raw trace* once +# R8 is minifying, and that is worth being precise about, because it is the +# thing -dontobfuscate used to buy us: +# +# * R8 overwrites every class's SourceFile with the marker +# `r8-map-id-` whatever we do here. Verified by building it both +# ways: with `-renamesourcefileattribute SourceFile` all 24,440 classes +# report the literal "SourceFile"; without it they report the marker. +# There is no rule that restores the original per-class .kt name. +# * R8 renumbers lines even with LineNumberTable kept, because one +# obfuscated line now has to encode a whole INLINED frame stack. In this +# build, line 7 of one method carries three source frames: +# TextFieldCharSequence.getText():58 inlined into TextFieldState.getText() +# :146 inlined into ShortNotePostViewModel.onMessageChanged():1727. A raw +# line number is no longer a source line. +# +# That second point is a cost of OPTIMIZATION, not of renaming, and it is new +# here only because the old `-keepnames class ** { *; }` had optimization off +# program-wide — which is exactly what Play was complaining about. +# +# So the answer is retrace, not a keep rule. And retrace hands back more than +# the old raw traces did: it expands those inlined frames instead of collapsing +# them into one misleading line. See `scripts/retrace.sh` and RELEASE_OPS.md +# § 7. Two things make that painless, and both depend on this file: +# +# * `-renamesourcefileattribute` is deliberately NOT set, so the map-id +# marker survives. A pasted trace then names the exact mapping file it +# needs (the marker is the `pg_map_id` header of that mapping), so there is +# never any doubt about which release a report came from. +# * mapping.txt.gz ships as a GitHub Release asset for every build, so traces +# from F-Droid / Zapstore / Accrescent users are retraceable too — Play +# Console only auto-deobfuscates the AAB it was given. +-keepattributes SourceFile,LineNumberTable + +# Generic signatures, plus the inner/enclosing-class links that travel with them. +# +# Signature carries the generic type arguments R8 would otherwise erase. +# +# It is NOT enough to make `object : TypeReference() {}` work under full mode +# (android.enableR8.fullMode=true). Measured on device: JacksonMapper's +# built its JavaTypes through jacksonTypeRefOf() and threw +# +# IllegalArgumentException: Internal error: TypeReference constructed without +# actual type information +# +# which poisons the class -- every later use is NoClassDefFoundError, so nothing +# could be signed or sent. Keeping the anonymous subclasses did not help either +# (tried -keep,allowobfuscation and a full -keep ... { *; }). The fix was to stop +# asking Jackson to read the type back off the class: JacksonMapper now builds +# those JavaTypes with TypeFactory.constructType/constructCollectionType/ +# constructParametricType, which take the Class objects directly. +# +# Anything else that still resolves a generic type reflectively is exposed the +# same way -- notably JacksonMapper.fromJsonTo, JsonMapperNip55 (NIP-55) and +# the NIP-46 bunker path, which were not reachable in this test run. +# +# All three are ALSO in AGP's proguard-android-optimize.txt, which keeps +# AnnotationDefault, EnclosingMethod, InnerClasses, Signature and the three +# RuntimeVisible* annotation attributes. They stay spelled out here anyway: the +# duplicate is free (measured at 0 bytes) and it means a change to AGP's default +# file cannot quietly take Signature away from Jackson. +# +# Two attributes this line used to carry are gone, both measured on the arm64 +# release DEX: +# +# * `*Annotation*` — over AGP's default its only contribution was the +# RuntimeInvisible* variants, which by definition cannot be read at runtime. +# Dropping it produced a byte-identical DEX (31,390,048 either way). Nothing +# we ship reads an annotation reflectively, and the libraries that do +# (kotlinx.serialization, appfunctions, AppSearch) match on RuntimeVisible*, +# which AGP already keeps. +# * `Exceptions` — @Throws metadata for 31 methods in shipped code, read by +# Java-interop compilers and by nothing at runtime. Worth 692 bytes. +# +# For the record, since it was wrong here for a while: this line used to credit +# jackson-module-kotlin with reading @kotlin.Metadata through it. That module no +# longer ships, and the Jackson mixins it also named were deleted along with the +# NWC Jackson path. +-keepattributes Signature,InnerClasses,EnclosingMethod + +# LocalVariableTable, LocalVariableTypeTable, MethodParameters and +# -keepparameternames used to be kept here as well. They are debug metadata: +# nothing in the app reads them (jackson-module-kotlin takes parameter names +# from @kotlin.Metadata, not from MethodParameters), and they are pure DEX +# weight in a release build. -keepdirectories libs -# Keep all names --keepnames class ** { *; } +# ----------------------------------------------------------------------------- +# JNI — names that live in a .so, not in the DEX +# ----------------------------------------------------------------------------- +# proguard-android-optimize.txt already contributes +# -keepclasseswithmembernames class * { native ; } +# which pins every class that DECLARES a native method (ArtiNative, +# secp256k1's loader, …) together with those methods' names, because the +# exported symbol is Java__. What that does NOT cover is the +# traffic in the other direction: Java/Kotlin the native side looks up itself. -# Keep All enums --keep enum ** { *; } +# libarti_android.so calls back into this interface by name — +# tools/arti-build/src/lib.rs does GetMethodID("onLogLine") on it. Renaming the +# method silently kills all Tor log output. +-keep class com.vitorpamplona.amethyst.ui.tor.ArtiLogCallback { *; } -# preserve access to native classses +# secp256k1's JNI layer resolves these from native code. -keep class fr.acinq.secp256k1.** { *; } -# JNA For Libsodium --keep class com.goterl.lazysodium.** { *; } - -# libscrypt --keep class com.lambdaworks.codec.** { *; } --keep class com.lambdaworks.crypto.** { *; } --keep class com.lambdaworks.jni.** { *; } - --keep class info.guardianproject.** { *; } - -# JNA also requires AWT, which Android does not have. So the classes are broken down to filter AWT out --keep class com.sun.jna.ToNativeConverter { *; } --keep class com.sun.jna.NativeMapped { *; } --keep class com.sun.jna.CallbackReference { *; } --keep class com.sun.jna.ptr.IntByReference { *; } --keep class com.sun.jna.NativeLong { *; } --keep class com.sun.jna.Structure { *; } --keep class com.sun.jna.Structure$* { *; } --keep class com.sun.jna.Native$ffi_callback { *; } --keep class * implements com.sun.jna.Structure$* { *; } --keep class * implements com.sun.jna.Native$* { *; } --keep class com.sun.jna.Native { - private static com.sun.jna.NativeMapped fromNative(java.lang.Class, java.lang.Object); - private static com.sun.jna.NativeMapped fromNative(java.lang.reflect.Method, java.lang.Object); - private static java.lang.Class nativeType(java.lang.Class); - private static java.lang.Object toNative(com.sun.jna.ToNativeConverter, java.lang.Object); - private static java.lang.Object fromNative(com.sun.jna.FromNativeConverter, java.lang.Object, java.lang.reflect.Method); -} - -# JSON parsing --keep class com.vitorpamplona.quartz.** { *; } --keep class com.vitorpamplona.amethyst.** { *; } - -# Room generates *_Impl subclasses instantiated reflectively via no-arg constructor. -# -keepnames preserves the name but R8 still strips the unused (). --keep class * extends androidx.room.RoomDatabase { - (); -} - # zxing-cpp's JNI boundary. The native library exports name-mangled symbols # (Java_zxingcpp_BarcodeReader_readYBuffer), so R8 renaming the class or its # external methods breaks the lookup at runtime with no build error and no @@ -78,3 +144,101 @@ # rule. We build that library ourselves now (tools/zxing-cpp-build) and vendor # its Kotlin half, so the rule is ours to carry. -keep class zxingcpp.** { *; } + +# Nothing keeps libscrypt, NetCipher/tor-android, LazySodium or JNA any more: +# quartz replaced libsodium with a pure-Kotlin implementation (LibSodiumInstance) +# and Tor now runs through arti's own JNI layer. Their rules used to live here and +# matched zero classes in the release build — none of those artifacts appear in +# mapping.txt, usage.txt or seeds.txt, i.e. they are not on the classpath at all. +# If one ever comes back, so must its rule: JNA in particular maps types onto the +# C ABI by reflecting over their fields and method signatures at runtime. + +# ----------------------------------------------------------------------------- +# Enum constant names +# ----------------------------------------------------------------------------- +# An enum constant's NAME is persisted data in this app. The preference stores +# write `enum.name` into DataStore and read it back with `Type.valueOf(string)` +# (see model/preferences/UISharedPreferences.kt, TorSharedPreferences.kt, +# NamecoinSharedPreferences.kt), and Jackson serialises enums by name too. +# Enum.valueOf resolves that string against the static FIELD name, so renaming +# the constants would reset every user's theme/font/Tor/connectivity setting on +# the first launch after an update. +# +# Deliberately blanket rather than a list of the enums that happen to be +# persisted today: the failure mode is silent, release-only, and one new +# `preferences[KEY] = value.name` line away. Only the field names are pinned — +# the enum classes themselves are still renamed and their methods still +# optimized. +-keepclassmembers enum * { + ; + public static **[] values(); + public static ** valueOf(java.lang.String); +} + +# ----------------------------------------------------------------------------- +# Jackson — reflective data binding only +# ----------------------------------------------------------------------------- +# Most of Quartz's wire format does NOT need a keep. Event, Filter, Message, +# Command, Rumor, EventTemplate, TagArray, the NIP-46 Bunker messages and the +# NIP-55 intent results all go through hand-written StdSerializer/StdDeserializer +# pairs registered on JacksonMapper / JsonMapperNip55, which read and write +# property names as string literals. Renaming their fields changes nothing on +# the wire. +# +# What remains is the code Jackson data-binds REFLECTIVELY — `treeToValue(...)` +# and `readValue()` with no custom deserializer. There the JSON property +# names come from the Kotlin constructor parameter names, so a renamed field is +# a changed wire format. + +# NIP-47 and CLINK used to need a package keep each, because Jackson bound their +# ~106 concrete classes reflectively. Both are gone: OptimizedJsonMapper routes +# those types at the hand-written kotlinx serializers, which name every field as a +# string literal. Nothing to keep, nothing to verify. + +# The on-disk stores (scheduled posts, pending PoW jobs, resource usage) used to +# need a keep each, because Jackson derived their JSON keys from the Kotlin +# constructor parameter names. They are @Serializable now: kotlinx bakes every key +# in as a string literal, so the field names can be renamed freely. The formats are +# pinned by ScheduledPostFileFormatTest and PowAndUsageFileFormatTest instead, +# which assert the bytes against what the Jackson build wrote. + +# ----------------------------------------------------------------------------- +# Names referenced from outside the DEX +# ----------------------------------------------------------------------------- +# AGP generates keeps from the merged manifest's component `android:name` +# attributes, but NOT from . The Cast framework reads +# this one out of the manifest and Class.forName()s it. +-keep class com.vitorpamplona.amethyst.service.cast.chromecast.AmethystCastOptionsProvider { *; } + +# NOT a rule for WorkManager. It stores the worker's class name in its own +# database at enqueue time and instantiates it by name on a later process start +# — including after an app update that reshuffled the mapping — so the name does +# have to survive. But androidx.work already ships exactly that in its own +# consumer rules (`-keepnames class * extends androidx.work.ListenableWorker` +# plus a keepclassmembers for the public constructors), so a rule here was pure +# duplication. The three workers stay listed in the reflection contract, which +# now verifies the LIBRARY's rule keeps doing the job. + +# androidx.appfunctions: the KSP-generated invokers and the app_functions.xml +# the system reads are keyed off these declarations. One class plus its +# generated neighbours — cheap enough not to be worth proving unnecessary +# against a pre-stable (alpha) library. +-keep class com.vitorpamplona.amethyst.appfunctions.** { *; } + +# ----------------------------------------------------------------------------- +# Enums used as navigation-route ARGUMENTS +# ----------------------------------------------------------------------------- +# androidx.navigation's type-safe routes resolve an enum argument by its +# fully-qualified class name (NavTypeConverter.parseEnum/parseNullableEnum call +# Class.forName on the serial name). R8 renames the class, so building the nav +# graph throws and the app cannot get past login: +# +# IllegalArgumentException: Cannot find class with name +# "...routes.DiscoverTab?". Ensure that the serialName for this argument is +# the default fully qualified name. +# +# The enum FIELDS are already pinned by the blanket `-keepclassmembers enum *` +# above; that rule deliberately lets the CLASS be renamed, which is exactly what +# breaks here. Every enum used as a route argument needs its name too. +-keep class com.vitorpamplona.amethyst.ui.navigation.routes.DiscoverTab { *; } +-keep class com.vitorpamplona.amethyst.ui.screen.loggedIn.bookmarkgroups.BookmarkType { *; } diff --git a/amethyst/src/androidTest/java/com/vitorpamplona/amethyst/ThreadDualAxisChartAssemblerTest.kt b/amethyst/src/androidTest/java/com/vitorpamplona/amethyst/ThreadDualAxisChartAssemblerTest.kt index a1dae99bcc..1a205eaf1b 100644 --- a/amethyst/src/androidTest/java/com/vitorpamplona/amethyst/ThreadDualAxisChartAssemblerTest.kt +++ b/amethyst/src/androidTest/java/com/vitorpamplona/amethyst/ThreadDualAxisChartAssemblerTest.kt @@ -21,7 +21,6 @@ package com.vitorpamplona.amethyst import androidx.test.ext.junit.runners.AndroidJUnit4 -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.amethyst.commons.model.cache.LocalCache import com.vitorpamplona.amethyst.commons.relayClient.assemblers.CashuMintDirectoryFilterAssembler import com.vitorpamplona.amethyst.commons.relayClient.nip47WalletConnect.NWCPaymentFilterAssembler @@ -163,7 +162,7 @@ class ThreadDualAxisChartAssemblerTest { fun threadOrderTest() = runBlocking { val eventArray = - JacksonMapper.mapper.readValue>(db) + Event.fromJson(header) + JacksonMapper.mapper.readValue>(db, JacksonMapper.eventListTypeInstance) + Event.fromJson(header) var counter = 0 eventArray.forEach { diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt index 5a83a988e2..db235e0e20 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssembler.kt @@ -48,7 +48,12 @@ class ReportAssembler { threadName: String = Thread.currentThread().name, ): String = buildString { - append(e.javaClass.simpleName) + // Fully qualified, NOT simpleName. Release builds are obfuscated, so this + // headline would otherwise read "a: 1.16.0-PLAY" — and a bare simple name + // carries no package for `retrace` to resolve it against, so it would stay + // unreadable even after the rest of the report retraced cleanly. The FQN + // retraces back to the real exception class. See scripts/retrace.sh. + append(e.javaClass.name) append(": ") appendLine(BuildConfig.VERSION_NAME + "-" + BuildConfig.FLAVOR.uppercase()) appendLine() @@ -101,8 +106,11 @@ class ReportAssembler { append("Thread: ") appendLine(threadName) appendLine(e.headline()) + // " at ", not just " ": `at` is what every stack-trace + // parser keys on, R8's `retrace` included. Without it a release report is + // passed through untouched and stays obfuscated. See scripts/retrace.sh. e.stackTrace.forEach { - append(" ") + append(" at ") appendLine(it.toString()) } val cause = e.cause @@ -111,7 +119,7 @@ class ReportAssembler { append(" ") appendLine(cause.headline()) cause.stackTrace.forEach { - append(" ") + append(" at ") appendLine(it.toString()) } } diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/lnurl/LightningAddressResolver.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/lnurl/LightningAddressResolver.kt index 2ed7dce908..305cb433a2 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/lnurl/LightningAddressResolver.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/lnurl/LightningAddressResolver.kt @@ -21,7 +21,7 @@ package com.vitorpamplona.amethyst.service.lnurl import android.content.Context -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.amethyst.R import com.vitorpamplona.amethyst.service.HttpStatusMessages import com.vitorpamplona.amethyst.ui.stringRes @@ -167,7 +167,7 @@ class LightningAddressResolver { val errorMessage = runCatching { - jacksonObjectMapper().readTree(body) + ObjectMapper().readTree(body) }.getOrNull()?.let { tree -> val errorNode = tree.get("error") val messageNode = tree.get("message") @@ -218,7 +218,7 @@ class LightningAddressResolver { context: Context, onZapRequestSent: (LnZapRequestEvent?) -> Unit = {}, ): String { - val mapper = jacksonObjectMapper() + val mapper = ObjectMapper() val lnurlpUrl = assembleUrl(lnAddress) diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/pow/PowJobStore.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/pow/PowJobStore.kt index e20ee660d6..97e68b8b19 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/pow/PowJobStore.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/pow/PowJobStore.kt @@ -20,9 +20,6 @@ */ package com.vitorpamplona.amethyst.service.pow -import com.fasterxml.jackson.databind.DeserializationFeature -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.amethyst.commons.service.pow.PersistedPoWJob import com.vitorpamplona.amethyst.commons.service.pow.PoWJobPersistence import com.vitorpamplona.quartz.utils.Log @@ -32,6 +29,8 @@ import kotlinx.coroutines.launch import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.withLock import kotlinx.coroutines.withContext +import kotlinx.serialization.Serializable +import kotlinx.serialization.json.Json import java.io.File /** @@ -46,9 +45,18 @@ class PowJobStore( private val storageFile: File, scope: CoroutineScope, ) : PoWJobPersistence { - private val mapper = - jacksonObjectMapper() - .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) + /** + * `encodeDefaults = true` so this writes the same bytes Jackson did — kotlinx + * omits a value equal to its default, which would silently drop `"version":1` + * from every file this build rewrites. `ignoreUnknownKeys` matches the + * FAIL_ON_UNKNOWN_PROPERTIES=false it replaces, so a file written by a newer + * build still loads here. + */ + private val json = + Json { + ignoreUnknownKeys = true + encodeDefaults = true + } // one lane: launch order == execution order, so a save followed by its // remove can never be applied backwards. @@ -103,7 +111,7 @@ class PowJobStore( jobs = try { if (storageFile.exists() && storageFile.length() > 0) { - mapper.readValue(storageFile).jobs.toMutableList() + json.decodeFromString(storageFile.readText()).jobs.toMutableList() } else { mutableListOf() } @@ -120,7 +128,7 @@ class PowJobStore( storageFile.parentFile?.mkdirs() val tmp = File(storageFile.parentFile, storageFile.name + ".tmp") try { - mapper.writeValue(tmp, PowJobsFile(version = 1, jobs = jobs.toList())) + tmp.writeText(json.encodeToString(PowJobsFile(version = 1, jobs = jobs.toList()))) if (!tmp.renameTo(storageFile)) { if (!storageFile.delete() || !tmp.renameTo(storageFile)) { Log.e(TAG) { "Failed to rename $tmp to $storageFile" } @@ -147,6 +155,7 @@ class PowJobStore( } } +@Serializable data class PowJobsFile( val version: Int = 1, val jobs: List = emptyList(), diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/resourceusage/ResourceUsageStore.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/resourceusage/ResourceUsageStore.kt index fc63089343..a6a0aba0f0 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/resourceusage/ResourceUsageStore.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/resourceusage/ResourceUsageStore.kt @@ -20,12 +20,11 @@ */ package com.vitorpamplona.amethyst.service.resourceusage -import com.fasterxml.jackson.databind.DeserializationFeature -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.utils.Log import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.withLock +import kotlinx.serialization.Serializable +import kotlinx.serialization.json.Json import java.io.File /** @@ -60,6 +59,7 @@ class ResourceUsageStore( */ private val keepDays: Long = 7, ) { + @Serializable data class UsageFile( val version: Int = 1, val days: Map> = emptyMap(), @@ -67,9 +67,18 @@ class ResourceUsageStore( val alertsOptOut: Boolean = false, ) - private val mapper = - jacksonObjectMapper() - .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) + /** + * `encodeDefaults = true` so this writes the same bytes Jackson did — kotlinx + * omits a value equal to its default, which would silently drop `"version":1` + * from every file this build rewrites. `ignoreUnknownKeys` matches the + * FAIL_ON_UNKNOWN_PROPERTIES=false it replaces, so a file written by a newer + * build still loads here. + */ + private val json = + Json { + ignoreUnknownKeys = true + encodeDefaults = true + } private val mutex = Mutex() private var loaded = false @@ -133,7 +142,7 @@ class ResourceUsageStore( data = try { if (storageFile.exists() && storageFile.length() > 0) { - mapper.readValue(storageFile) + json.decodeFromString(storageFile.readText()) } else { UsageFile() } @@ -148,7 +157,7 @@ class ResourceUsageStore( storageFile.parentFile?.mkdirs() val tmp = File(storageFile.parentFile, storageFile.name + ".tmp") try { - mapper.writeValue(tmp, data) + tmp.writeText(json.encodeToString(data)) if (!tmp.renameTo(storageFile)) { if (!storageFile.delete() || !tmp.renameTo(storageFile)) { Log.e(TAG) { "Failed to rename $tmp to $storageFile" } diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/uploads/nip96/Nip96Uploader.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/uploads/nip96/Nip96Uploader.kt index c52095af2a..da9f6875ca 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/uploads/nip96/Nip96Uploader.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/uploads/nip96/Nip96Uploader.kt @@ -26,7 +26,7 @@ import android.net.Uri import android.provider.OpenableColumns import android.webkit.MimeTypeMap import androidx.core.net.toFile -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.amethyst.R import com.vitorpamplona.amethyst.service.HttpStatusMessages import com.vitorpamplona.amethyst.service.checkNotInMainThread @@ -209,7 +209,7 @@ class Nip96Uploader { val errorMessage = try { - val tree = jacksonObjectMapper().readTree(msg) + val tree = ObjectMapper().readTree(msg) val status = tree.get("status")?.asText() val message = tree.get("message")?.asText() if (status == "error" && message != null) { diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/ArtiGuardState.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/ArtiGuardState.kt index 5ceab6f135..67ca1ed0b9 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/ArtiGuardState.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/ArtiGuardState.kt @@ -21,7 +21,7 @@ package com.vitorpamplona.amethyst.ui.tor import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper /** * Pure, file- and JNI-free parsers over Arti's persisted guard sample @@ -42,7 +42,7 @@ import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper * circuit — i.e. a real bootstrap reached the guard-confirmation stage. */ object ArtiGuardState { - private val mapper = jacksonObjectMapper() + private val mapper = ObjectMapper() /** Convenience for tests/callers holding the raw file text. */ fun parse(json: String): JsonNode = mapper.readTree(json) diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/TorService.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/TorService.kt index e8566a2275..5384e3f808 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/TorService.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/ui/tor/TorService.kt @@ -22,7 +22,7 @@ package com.vitorpamplona.amethyst.ui.tor import android.content.Context import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.quartz.utils.Log import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers @@ -206,7 +206,7 @@ class TorService( val file = guardsFile() if (!file.exists()) return null return try { - jacksonObjectMapper().readTree(file) + ObjectMapper().readTree(file) } catch (e: Exception) { Log.w("TorService") { "Could not inspect guards.json: ${e.message}" } null diff --git a/amethyst/src/play/java/com/vitorpamplona/amethyst/appfunctions/AmethystAppFunctions.kt b/amethyst/src/play/java/com/vitorpamplona/amethyst/appfunctions/AmethystAppFunctions.kt index 08a2da1826..a0dd8d1bf9 100644 --- a/amethyst/src/play/java/com/vitorpamplona/amethyst/appfunctions/AmethystAppFunctions.kt +++ b/amethyst/src/play/java/com/vitorpamplona/amethyst/appfunctions/AmethystAppFunctions.kt @@ -60,6 +60,7 @@ import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceErrorResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceSuccessResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Response import com.vitorpamplona.quartz.nip53LiveActivities.streaming.LiveActivitiesEvent +import com.vitorpamplona.quartz.nip55AndroidSigner.client.NostrSignerExternal import com.vitorpamplona.quartz.nip57Zaps.LnZapEvent import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent import com.vitorpamplona.quartz.nipB1Bolt12Zaps.zap.Bolt12ZapEvent @@ -1907,13 +1908,13 @@ class AmethystAppFunctions { "Active Amethyst account is read-only (npub login). Sign in with a private key or NIP-46 bunker to publish.", ) } - // NostrSignerExternal lives in quartz/androidMain and isn't visible - // to commonMain — but we're already in android-app code, so the - // class is on the classpath. Reflective name-check keeps the - // dependency edge clean and avoids hard-coupling the bridge to - // the NIP-55 implementation class. - val klass = signer::class.qualifiedName - if (klass == "com.vitorpamplona.quartz.nip55AndroidSigner.client.NostrSignerExternal") { + // NostrSignerExternal lives in quartz/androidMain, which is on the + // classpath here because this is android-app code. This used to compare + // `signer::class.qualifiedName` against the fully-qualified name to + // avoid the import; R8 renames the class in release builds, so that + // comparison silently stopped matching and the external-signer branch + // never ran. A type check has no such failure mode. + if (signer is NostrSignerExternal) { throw AppFunctionNotSupportedException( "Amethyst is configured to use an external NIP-55 signer (Amber). " + "Write actions from Gemini aren't supported with this signer yet — " + diff --git a/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt b/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt index f618092db0..46720c57ac 100644 --- a/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt +++ b/amethyst/src/test/java/com/vitorpamplona/amethyst/service/crashreports/ReportAssemblerTest.kt @@ -35,7 +35,7 @@ class ReportAssemblerTest { val report = ReportAssembler().buildReport(e, "main") assertTrue(report.length < 2_000) - assertTrue(report.startsWith("IllegalArgumentException: ")) + assertTrue(report.startsWith("java.lang.IllegalArgumentException: ")) assertTrue(report.contains("Navigation destination that matches route")) assertTrue(report.trimEnd().endsWith("```")) } @@ -48,7 +48,21 @@ class ReportAssemblerTest { val report = ReportAssembler().buildReport(e, "main") assertTrue(report.contains("java.lang.RuntimeException: boom")) - assertTrue(report.contains("com.example.Foo.bar(Foo.kt:42)")) + // The "at " prefix is load-bearing: retrace only rewrites frames it recognises. + assertTrue(report.contains(" at com.example.Foo.bar(Foo.kt:42)")) assertTrue(report.contains("java.lang.IllegalStateException: root cause")) } + + @Test + fun headlineNamesTheExceptionClassInFullSoItCanBeRetraced() { + // The headline is the line a maintainer skims and dedups on. Release builds are + // obfuscated, and `retrace` can only restore a class name it can resolve — a bare + // simple name has no package, so the headline has to carry the fully qualified one. + val e = IllegalStateException("boom") + e.stackTrace = arrayOf(StackTraceElement("com.example.Foo", "bar", "Foo.kt", 42)) + + val report = ReportAssembler().buildReport(e, "main") + + assertTrue(report.startsWith("java.lang.IllegalStateException: ")) + } } diff --git a/amethyst/src/test/java/com/vitorpamplona/amethyst/service/pow/PowAndUsageFileFormatTest.kt b/amethyst/src/test/java/com/vitorpamplona/amethyst/service/pow/PowAndUsageFileFormatTest.kt new file mode 100644 index 0000000000..dbea72be3d --- /dev/null +++ b/amethyst/src/test/java/com/vitorpamplona/amethyst/service/pow/PowAndUsageFileFormatTest.kt @@ -0,0 +1,135 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.service.pow + +import com.vitorpamplona.amethyst.commons.service.pow.PersistedPoWJob +import com.vitorpamplona.amethyst.service.resourceusage.ResourceUsageStore +import kotlinx.serialization.json.Json +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * The PoW-queue and resource-usage files survived the move off Jackson. + * + * Both hold state an update must not lose: `pending_pow_jobs.json` is posts the user + * already hit send on that are still mining, and the usage file backs the + * high-consumption alert. Each literal below is the exact string the Jackson build + * wrote for the object beside it, captured before the switch. + */ +class PowAndUsageFileFormatTest { + private val json = + Json { + ignoreUnknownKeys = true + encodeDefaults = true + } + + private val powSample = + PowJobsFile( + version = 1, + jobs = + listOf( + PersistedPoWJob( + id = "j1", + accountPubkey = "pk1", + kind = 1, + difficulty = 21, + templateJson = """{"t":1}""", + replayType = "broadcast", + relayUrls = listOf("wss://a"), + extraEventsJson = emptyList(), + publishAtSec = null, + recipientPubkeys = listOf("r1"), + wrapExpirationDelta = null, + createdAtSec = 999L, + ), + ), + ) + + private val usageSample = + ResourceUsageStore.UsageFile( + version = 1, + days = mapOf("20260919" to mapOf("relay.a" to 12L, "relay.b" to 34L)), + lastAlertAtSec = 555L, + alertsOptOut = true, + ) + + @Test + fun powJobsWriteTheBytesJacksonWrote() { + assertEquals(POW_JACKSON_OUTPUT, json.encodeToString(powSample)) + } + + @Test + fun powJobsFromTheJacksonBuildStillLoad() { + val loaded = json.decodeFromString(POW_JACKSON_OUTPUT) + + assertEquals(1, loaded.jobs.size) + val job = loaded.jobs.first() + assertEquals("j1", job.id) + assertEquals(21, job.difficulty) + assertEquals("broadcast", job.replayType) + assertEquals(listOf("r1"), job.recipientPubkeys) + assertEquals(999L, job.createdAtSec) + assertEquals(null, job.publishAtSec) + } + + @Test + fun usageWritesTheBytesJacksonWrote() { + assertEquals(USAGE_JACKSON_OUTPUT, json.encodeToString(usageSample)) + } + + @Test + fun usageFromTheJacksonBuildStillLoads() { + val loaded = json.decodeFromString(USAGE_JACKSON_OUTPUT) + + assertEquals(mapOf("relay.a" to 12L, "relay.b" to 34L), loaded.days["20260919"]) + assertEquals(555L, loaded.lastAlertAtSec) + assertTrue(loaded.alertsOptOut) + } + + @Test + fun bothReadersTolerateKeysFromANewerBuild() { + val pow = POW_JACKSON_OUTPUT.replace("""{"version":1""", """{"version":1,"futureKey":[1]""") + val usage = USAGE_JACKSON_OUTPUT.replace("""{"version":1""", """{"version":1,"futureKey":{"a":1}""") + + assertEquals( + "j1", + json + .decodeFromString(pow) + .jobs + .first() + .id, + ) + assertEquals(555L, json.decodeFromString(usage).lastAlertAtSec) + } + + companion object { + private const val POW_JACKSON_OUTPUT = + """{"version":1,"jobs":[{"id":"j1","accountPubkey":"pk1","kind":1,"difficulty":21,""" + + """"templateJson":"{\"t\":1}","replayType":"broadcast","relayUrls":["wss://a"],""" + + """"extraEventsJson":[],"publishAtSec":null,"recipientPubkeys":["r1"],""" + + """"wrapExpirationDelta":null,"createdAtSec":999}]}""" + + private const val USAGE_JACKSON_OUTPUT = + """{"version":1,"days":{"20260919":{"relay.a":12,"relay.b":34}},""" + + """"lastAlertAtSec":555,"alertsOptOut":true}""" + } +} diff --git a/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/BaseLargeCacheBenchmark.kt b/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/BaseLargeCacheBenchmark.kt index 1da2be3a7e..5f8faca04f 100644 --- a/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/BaseLargeCacheBenchmark.kt +++ b/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/BaseLargeCacheBenchmark.kt @@ -20,7 +20,6 @@ */ package com.vitorpamplona.quartz.benchmark -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.HexKey import com.vitorpamplona.quartz.nip01Core.jackson.JacksonMapper @@ -35,8 +34,9 @@ open class BaseLargeCacheBenchmark { // This file includes duplicates val fullDBInputStream = javaClass.classLoader!!.getResourceAsStream("nostr_vitor_startup_data.json.gz") - return JacksonMapper.mapper.readValue>( + return JacksonMapper.mapper.readValue( GZIPInputStream(fullDBInputStream), + JacksonMapper.eventListTypeInstance, ) } } diff --git a/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/CacheBenchmark.kt b/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/CacheBenchmark.kt index d9138f5dce..08f47f60cd 100644 --- a/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/CacheBenchmark.kt +++ b/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/CacheBenchmark.kt @@ -23,7 +23,6 @@ package com.vitorpamplona.quartz.benchmark import androidx.benchmark.junit4.BenchmarkRule import androidx.benchmark.junit4.measureRepeated import androidx.test.ext.junit.runners.AndroidJUnit4 -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.HexKey import com.vitorpamplona.quartz.nip01Core.jackson.JacksonMapper @@ -43,8 +42,9 @@ open class BaseCacheBenchmark { // This file includes duplicates val fullDBInputStream = javaClass.classLoader?.getResourceAsStream("nostr_vitor_startup_data.json.gz") - return JacksonMapper.mapper.readValue>( + return JacksonMapper.mapper.readValue( GZIPInputStream(fullDBInputStream), + JacksonMapper.eventListTypeInstance, ) } diff --git a/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/EventCmdSerializerBenchmark.kt b/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/EventCmdSerializerBenchmark.kt index 1eec88137d..884e353321 100644 --- a/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/EventCmdSerializerBenchmark.kt +++ b/benchmark/src/androidTest/java/com/vitorpamplona/quartz/benchmark/EventCmdSerializerBenchmark.kt @@ -23,7 +23,7 @@ package com.vitorpamplona.quartz.benchmark import androidx.benchmark.junit4.BenchmarkRule import androidx.benchmark.junit4.measureRepeated import androidx.test.ext.junit.runners.AndroidJUnit4 -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.utils.sha256.Sha256Hasher import org.junit.Rule @@ -54,7 +54,7 @@ class EventCmdSerializerBenchmark { @Test fun jsonStringEncoderJackson() { - val jsonMapper = jacksonObjectMapper() + val jsonMapper = ObjectMapper() benchmarkRule.measureRepeated { jsonMapper.writeValueAsString(specialEncoders) } @@ -62,7 +62,7 @@ class EventCmdSerializerBenchmark { @Test fun jsonStringEncoderSha256Jackson() { - val jsonMapper = jacksonObjectMapper() + val jsonMapper = ObjectMapper() benchmarkRule.measureRepeated { val digest = Sha256Hasher() digest.hash(jsonMapper.writeValueAsString(specialEncoders).toByteArray()) diff --git a/commons/proguard-rules.pro b/commons/proguard-rules.pro deleted file mode 100644 index 0bf7b32616..0000000000 --- a/commons/proguard-rules.pro +++ /dev/null @@ -1,24 +0,0 @@ -# Add project specific ProGuard rules here. -# You can control the set of applied configuration files using the -# proguardFiles setting in build.gradle. -# -# For more details, see -# http://developer.android.com/guide/developing/tools/proguard.html - -# If your project uses WebView with JS, uncomment the following -# and specify the fully qualified class name to the JavaScript interface -# class: -#-keepclassmembers class fqcn.of.javascript.interface.for.webview { -# public *; -#} - -# Uncomment this to preserve the line number information for -# debugging stack traces. -#-keepattributes SourceFile,LineNumberTable - -# If you keep the line number information, uncomment this to -# hide the original source file name. -#-renamesourcefileattribute SourceFile - --keep class com.vitorpamplona.quartz.** { *; } --keep class com.vitorpamplona.amethyst.** { *; } \ No newline at end of file diff --git a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPost.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPost.kt index 5f37378765..04562ff4cd 100644 --- a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPost.kt +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPost.kt @@ -20,6 +20,9 @@ */ package com.vitorpamplona.amethyst.commons.scheduledposts +import kotlinx.serialization.Serializable + +@Serializable enum class ScheduledPostStatus { PENDING, PUBLISHING, @@ -28,6 +31,7 @@ enum class ScheduledPostStatus { CANCELLED, } +@Serializable data class ScheduledPost( val id: String, val accountPubkey: String, @@ -44,6 +48,7 @@ data class ScheduledPost( val terminatedAtSec: Long? = null, ) +@Serializable data class ScheduledPostFile( val version: Int = 1, val posts: List = emptyList(), diff --git a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/service/pow/PoWJobPersistence.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/service/pow/PoWJobPersistence.kt index 1409f4768b..dddb512664 100644 --- a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/service/pow/PoWJobPersistence.kt +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/service/pow/PoWJobPersistence.kt @@ -20,6 +20,8 @@ */ package com.vitorpamplona.amethyst.commons.service.pow +import kotlinx.serialization.Serializable + /** * Durable record of a template mining job so a post survives process death: * everything needed to re-mine and re-send with no lambda captured — the @@ -31,6 +33,7 @@ package com.vitorpamplona.amethyst.commons.service.pow * outbox relays, [REPLAY_RELAYS] publishes to [relayUrls], [REPLAY_SCHEDULE] * signs and parks the event in the scheduled-post store for [publishAtSec]. */ +@Serializable data class PersistedPoWJob( val id: String, val accountPubkey: String, diff --git a/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostStore.kt b/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostStore.kt index 539f759fce..d19e8e7a6c 100644 --- a/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostStore.kt +++ b/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostStore.kt @@ -20,15 +20,13 @@ */ package com.vitorpamplona.amethyst.commons.scheduledposts -import com.fasterxml.jackson.databind.DeserializationFeature -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.utils.Log import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.asStateFlow import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.withLock +import kotlinx.serialization.json.Json import java.io.File import java.nio.file.Files import java.nio.file.attribute.PosixFilePermission @@ -37,9 +35,18 @@ class ScheduledPostStore( private val storageFile: File, private val nowSec: () -> Long = { System.currentTimeMillis() / 1000 }, ) { - private val mapper = - jacksonObjectMapper() - .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) + /** + * `encodeDefaults = true` so this writes the same bytes Jackson did — kotlinx + * omits a value equal to its default, which would silently drop `"version":1` + * from every file this build rewrites. `ignoreUnknownKeys` matches the + * FAIL_ON_UNKNOWN_PROPERTIES=false it replaces, so a file written by a newer + * build still loads here. + */ + private val json = + Json { + ignoreUnknownKeys = true + encodeDefaults = true + } private val mutex = Mutex() private var loaded = false @@ -330,7 +337,7 @@ class ScheduledPostStore( val fromDisk = try { if (storageFile.exists() && storageFile.length() > 0) { - mapper.readValue(storageFile).posts.toMutableList() + json.decodeFromString(storageFile.readText()).posts.toMutableList() } else { // File vanished — treat as no external state; keep current in-memory. return @@ -348,7 +355,7 @@ class ScheduledPostStore( posts = try { if (storageFile.exists() && storageFile.length() > 0) { - mapper.readValue(storageFile).posts.toMutableList() + json.decodeFromString(storageFile.readText()).posts.toMutableList() } else { mutableListOf() } @@ -408,7 +415,7 @@ class ScheduledPostStore( storageFile.parentFile?.mkdirs() val tmp = File(storageFile.parentFile, storageFile.name + ".tmp") try { - mapper.writeValue(tmp, ScheduledPostFile(version = 1, posts = snapshot)) + tmp.writeText(json.encodeToString(ScheduledPostFile(version = 1, posts = snapshot))) // Restrict to owner-only BEFORE the rename so the store is never briefly // world-readable. It holds pre-signed events + the account's pubkey, which // must not leak to other local users on a shared machine. diff --git a/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/LightningAddressResolver.kt b/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/LightningAddressResolver.kt index 9161df6764..5acca96399 100644 --- a/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/LightningAddressResolver.kt +++ b/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/LightningAddressResolver.kt @@ -20,7 +20,7 @@ */ package com.vitorpamplona.amethyst.commons.service.lnurl -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.quartz.lightning.LnInvoiceUtil import com.vitorpamplona.quartz.lightning.Lud06 import com.vitorpamplona.quartz.nip57Zaps.LnZapRequestEvent @@ -48,7 +48,7 @@ import kotlin.coroutines.cancellation.CancellationException class LightningAddressResolver( private val httpClient: OkHttpClient, ) { - private val mapper = jacksonObjectMapper() + private val mapper = ObjectMapper() /** * Result of resolving a lightning address to a BOLT11 invoice. diff --git a/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/OkHttpLnurlEndpointResolver.kt b/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/OkHttpLnurlEndpointResolver.kt index 2fbdf41481..dd162f881c 100644 --- a/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/OkHttpLnurlEndpointResolver.kt +++ b/commons/src/jvmAndroid/kotlin/com/vitorpamplona/amethyst/commons/service/lnurl/OkHttpLnurlEndpointResolver.kt @@ -20,7 +20,7 @@ */ package com.vitorpamplona.amethyst.commons.service.lnurl -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointCache import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointInfo import com.vitorpamplona.quartz.nip57Zaps.validate.LnurlEndpointResolver @@ -47,7 +47,7 @@ import kotlin.coroutines.cancellation.CancellationException class OkHttpLnurlEndpointResolver( private val okHttpClient: (String) -> OkHttpClient, ) : LnurlEndpointResolver { - private val mapper = jacksonObjectMapper() + private val mapper = ObjectMapper() override suspend fun resolve(lnurlpUrl: String): LnurlEndpointInfo? = LnurlEndpointCache.getOrFetch(lnurlpUrl, ::fetch) diff --git a/commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostFileFormatTest.kt b/commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostFileFormatTest.kt new file mode 100644 index 0000000000..7745905273 --- /dev/null +++ b/commons/src/jvmTest/kotlin/com/vitorpamplona/amethyst/commons/scheduledposts/ScheduledPostFileFormatTest.kt @@ -0,0 +1,117 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.commons.scheduledposts + +import kotlinx.serialization.json.Json +import kotlin.test.Test +import kotlin.test.assertEquals + +/** + * The scheduled-post file survived the move off Jackson. + * + * This file is a user's queued posts: pre-signed events waiting for their publish + * time. If the format shifted when the serializer changed, an existing install + * would silently drop everything it had queued on the first launch after the + * update — no crash, no error, just an empty queue. + * + * [JACKSON_OUTPUT] is the exact string the Jackson build wrote for [sample], + * captured from it before the switch. The assertions are that the kotlinx reader + * loads it and the kotlinx writer reproduces it byte for byte — the second half + * matters because it is what a downgrade, or an older `amy` build sharing the same + * file, has to keep reading. + */ +class ScheduledPostFileFormatTest { + private val json = + Json { + ignoreUnknownKeys = true + encodeDefaults = true + } + + private val sample = + ScheduledPostFile( + version = 1, + posts = + listOf( + ScheduledPost( + id = "id1", + accountPubkey = "pk1", + signedEventJson = """{"k":1}""", + relayUrls = listOf("wss://a", "wss://b"), + extraEventsJson = listOf("e1"), + publishAtSec = 111L, + createdAtSec = 222L, + status = ScheduledPostStatus.PENDING, + lastAttemptAtSec = null, + attemptCount = 0, + lastError = null, + terminatedAtSec = 333L, + ), + ), + ) + + @Test + fun writesTheBytesJacksonWrote() { + assertEquals(JACKSON_OUTPUT, json.encodeToString(sample)) + } + + @Test + fun readsAFileWrittenByTheJacksonBuild() { + val loaded = json.decodeFromString(JACKSON_OUTPUT) + + assertEquals(1, loaded.version) + assertEquals(1, loaded.posts.size) + val post = loaded.posts.first() + assertEquals("id1", post.id) + assertEquals("""{"k":1}""", post.signedEventJson) + assertEquals(listOf("wss://a", "wss://b"), post.relayUrls) + assertEquals(ScheduledPostStatus.PENDING, post.status) + assertEquals(333L, post.terminatedAtSec) + } + + @Test + fun aFileFromANewerBuildStillLoads() { + // Forward compatibility: the reader must not choke on a key it does not know, + // which is what FAIL_ON_UNKNOWN_PROPERTIES=false used to buy. + val withExtras = + JACKSON_OUTPUT + .replace("""{"version":1""", """{"version":1,"somethingNew":{"a":1}""") + .replace(""""id":"id1"""", """"id":"id1","perPostFutureField":true""") + + val loaded = json.decodeFromString(withExtras) + + assertEquals("id1", loaded.posts.first().id) + } + + @Test + fun versionIsNotDroppedJustBecauseItEqualsItsDefault() { + // kotlinx omits a value equal to its default unless encodeDefaults is set. + // Losing "version" would make the file unreadable to anything that checks it. + assertEquals(true, json.encodeToString(sample).startsWith("""{"version":1,""")) + } + + companion object { + private const val JACKSON_OUTPUT = + """{"version":1,"posts":[{"id":"id1","accountPubkey":"pk1","signedEventJson":"{\"k\":1}",""" + + """"relayUrls":["wss://a","wss://b"],"extraEventsJson":["e1"],"publishAtSec":111,""" + + """"createdAtSec":222,"status":"PENDING","lastAttemptAtSec":null,"attemptCount":0,""" + + """"lastError":null,"terminatedAtSec":333}]}""" + } +} diff --git a/commonsUI/proguard-rules.pro b/commonsUI/proguard-rules.pro deleted file mode 100644 index 0bf7b32616..0000000000 --- a/commonsUI/proguard-rules.pro +++ /dev/null @@ -1,24 +0,0 @@ -# Add project specific ProGuard rules here. -# You can control the set of applied configuration files using the -# proguardFiles setting in build.gradle. -# -# For more details, see -# http://developer.android.com/guide/developing/tools/proguard.html - -# If your project uses WebView with JS, uncomment the following -# and specify the fully qualified class name to the JavaScript interface -# class: -#-keepclassmembers class fqcn.of.javascript.interface.for.webview { -# public *; -#} - -# Uncomment this to preserve the line number information for -# debugging stack traces. -#-keepattributes SourceFile,LineNumberTable - -# If you keep the line number information, uncomment this to -# hide the original source file name. -#-renamesourcefileattribute SourceFile - --keep class com.vitorpamplona.quartz.** { *; } --keep class com.vitorpamplona.amethyst.** { *; } \ No newline at end of file diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 2aacef6b93..0aa82c3a96 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -38,6 +38,7 @@ firebaseBom = "34.19.0" fragmentKtx = "1.9.0" gms = "4.5.0" healthConnect = "1.1.0" +jackson = "2.22.2" jacksonModuleKotlin = "2.22.2" javaKeyring = "1.0.4" kmpTorRuntime = "2.6.0" @@ -202,6 +203,7 @@ google-mlkit-genai-rewriting = { group = "com.google.mlkit", name = "genai-rewri google-mlkit-genai-image-description = { group = "com.google.mlkit", name = "genai-image-description", version.ref = "genaiImageDescription" } google-mlkit-language-id = { group = "com.google.mlkit", name = "language-id", version.ref = "languageId" } google-mlkit-translate = { group = "com.google.mlkit", name = "translate", version.ref = "translate" } +jackson-databind = { group = "com.fasterxml.jackson.core", name = "jackson-databind", version.ref = "jackson" } jackson-module-kotlin = { group = "com.fasterxml.jackson.module", name = "jackson-module-kotlin", version.ref = "jacksonModuleKotlin" } java-keyring = { group = "com.github.javakeyring", name = "java-keyring", version.ref = "javaKeyring" } kmp-tor-runtime = { group = "io.matthewnelson.kmp-tor", name = "runtime", version.ref = "kmpTorRuntime" } diff --git a/quartz/build.gradle.kts b/quartz/build.gradle.kts index 9db22ebaf1..2ae70ca1cb 100644 --- a/quartz/build.gradle.kts +++ b/quartz/build.gradle.kts @@ -177,8 +177,16 @@ kotlin { dependsOn(commonMain.get()) dependencies { - // Performant Parser of JSONs into Events - api(libs.jackson.module.kotlin) + // Performant Parser of JSONs into Events. + // + // jackson-databind, NOT jackson-module-kotlin: the module exists to bind + // Kotlin classes reflectively off their constructor parameter names, and + // nothing in the app does that any more — every wire format goes through a + // hand-written serializer or kotlinx. Dropping it takes kotlin-reflect with + // it, which is ~1,000 classes of DEX the app never called. `cli` still + // declares the module itself: it genuinely binds its config files + // reflectively, and is never minified. + api(libs.jackson.databind) // Websockets API implementation(libs.okhttp) diff --git a/quartz/consumer-rules.pro b/quartz/consumer-rules.pro index 04373ed538..5c7330a4a2 100644 --- a/quartz/consumer-rules.pro +++ b/quartz/consumer-rules.pro @@ -1,25 +1,39 @@ -# Add project specific ProGuard rules here. -# You can control the set of applied configuration files using the -# proguardFiles setting in build.gradle. +# ============================================================================= +# Keep rules Quartz contributes to every app that consumes it (wired through +# `optimization.consumerKeepRules` in build.gradle.kts, so these end up merged +# into the consumer's own R8 configuration). # -# For more details, see -# http://developer.android.com/guide/developing/tools/proguard.html +# Scope them tightly. A rule here applies to the CONSUMER's whole program, so +# the `-keepnames class ** { *; }` that used to sit in this file pinned every +# name in every app that depends on Quartz — ours included — and blocked R8 +# from optimizing any member anywhere (`-keepnames` is `-keep,allowshrinking`). +# Only list what breaks at runtime if the name changes. +# ============================================================================= -keepdirectories libs -# Keep all names --keepnames class ** { *; } - -# Keep All enums --keep enum ** { *; } - -# preserve access to native classses +# secp256k1's JNI layer resolves these from native code. -keep class fr.acinq.secp256k1.** { *; } -# libscrypt --keep class com.lambdaworks.codec.** { *; } --keep class com.lambdaworks.crypto.** { *; } --keep class com.lambdaworks.jni.** { *; } +# Nothing keeps libscrypt any more: quartz replaced it with a pure-Kotlin +# implementation and `:quartz:dependencies` shows no com.lambdaworks on any +# configuration. These rules are merged into the R8 config of EVERY app that +# depends on quartz, so a rule we do not need is noise in somebody else's build. -# JSON parsing --keep class com.vitorpamplona.quartz.** { *; } \ No newline at end of file +# No Jackson keeps. Every wire format Quartz speaks is now handled by a +# hand-written serializer that names its fields as string literals: the +# StdSerializer/StdDeserializer pairs registered on JacksonMapper (Event, Filter, +# Message, Command, Rumor, EventTemplate, TagArray, the NIP-46 Bunker messages), +# JsonMapperNip55 (IntentResult, Permission), and the kotlinx serializers that +# NIP-47 and CLINK route through on every target. None of it reads a Kotlin +# constructor parameter name at runtime, so none of it has to survive R8. + +# Quartz serialises enums by name (Jackson writes/reads Enum.name, and +# Enum.valueOf resolves that string against the static field name), so the +# constants of its own enums have to keep their names. Consumers that persist +# their OWN enums by name need the equivalent rule in their own configuration. +-keepclassmembers enum com.vitorpamplona.quartz.** { + ; + public static **[] values(); + public static ** valueOf(java.lang.String); +} diff --git a/quartz/proguard-rules.pro b/quartz/proguard-rules.pro deleted file mode 100644 index e1f545f4da..0000000000 --- a/quartz/proguard-rules.pro +++ /dev/null @@ -1,40 +0,0 @@ -# Add project specific ProGuard rules here. -# You can control the set of applied configuration files using the -# proguardFiles setting in build.gradle. -# -# For more details, see -# http://developer.android.com/guide/developing/tools/proguard.html - -# preserve the line number information for debugging stack traces. --dontobfuscate --keepattributes LocalVariableTable --keepattributes LocalVariableTypeTable --keepattributes *Annotation* --keepattributes SourceFile --keepattributes LineNumberTable --keepattributes Signature --keepattributes Exceptions --keepattributes InnerClasses --keepattributes EnclosingMethod --keepattributes MethodParameters --keepparameternames - --keepdirectories libs - -# Keep all names --keepnames class ** { *; } - -# Keep All enums --keep enum ** { *; } - -# preserve access to native classses --keep class fr.acinq.secp256k1.** { *; } - -# libscrypt --keep class com.lambdaworks.codec.** { *; } --keep class com.lambdaworks.crypto.** { *; } --keep class com.lambdaworks.jni.** { *; } - -# JSON parsing --keep class com.vitorpamplona.quartz.** { *; } - diff --git a/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/LargeDBSignatureCheck.kt b/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/LargeDBSignatureCheck.kt index f7f5b655a0..3132e3358c 100644 --- a/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/LargeDBSignatureCheck.kt +++ b/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/LargeDBSignatureCheck.kt @@ -21,7 +21,6 @@ package com.vitorpamplona.quartz import androidx.test.ext.junit.runners.AndroidJUnit4 -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.crypto.verify import com.vitorpamplona.quartz.nip01Core.jackson.JacksonMapper @@ -41,8 +40,9 @@ class LargeDBSignatureCheck { val fullDBInputStream = javaClass.classLoader?.getResourceAsStream("nostr_vitor_short.json") val eventArray = - JacksonMapper.mapper.readValue>( + JacksonMapper.mapper.readValue( InputStreamReader(fullDBInputStream), + JacksonMapper.eventListTypeInstance, ) as List var counter = 0 @@ -61,8 +61,9 @@ class LargeDBSignatureCheck { val fullDBInputStream = javaClass.classLoader?.getResourceAsStream("nostr_vitor_startup_data.json.gz") val eventArray = - JacksonMapper.mapper.readValue>( + JacksonMapper.mapper.readValue( GZIPInputStream(fullDBInputStream), + JacksonMapper.eventListTypeInstance, ) as List var counter = 0 diff --git a/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/nip06KeyDerivation/Bip39MnemonicsTest.kt b/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/nip06KeyDerivation/Bip39MnemonicsTest.kt index 9042339582..0eabd91da5 100644 --- a/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/nip06KeyDerivation/Bip39MnemonicsTest.kt +++ b/quartz/src/androidDeviceTest/kotlin/com/vitorpamplona/quartz/nip06KeyDerivation/Bip39MnemonicsTest.kt @@ -21,7 +21,7 @@ package com.vitorpamplona.quartz.nip06KeyDerivation import androidx.test.ext.junit.runners.AndroidJUnit4 -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.quartz.nip01Core.core.toHexKey import com.vitorpamplona.quartz.utils.Hex import com.vitorpamplona.quartz.utils.RandomInstance @@ -34,7 +34,7 @@ import org.junit.runner.RunWith @RunWith(AndroidJUnit4::class) class Bip39MnemonicsTest { private val tests = - jacksonObjectMapper() + ObjectMapper() .readTree(javaClass.classLoader?.getResourceAsStream("bip39.vectors.json")) @Test diff --git a/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/JsonMapperNip55.kt b/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/JsonMapperNip55.kt index 41123a7386..a922a03a72 100644 --- a/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/JsonMapperNip55.kt +++ b/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/JsonMapperNip55.kt @@ -22,12 +22,11 @@ package com.vitorpamplona.quartz.nip55AndroidSigner import com.fasterxml.jackson.core.json.JsonReadFeature import com.fasterxml.jackson.databind.DeserializationFeature +import com.fasterxml.jackson.databind.JavaType import com.fasterxml.jackson.databind.ObjectMapper import com.fasterxml.jackson.databind.module.SimpleModule import com.fasterxml.jackson.databind.node.ArrayNode import com.fasterxml.jackson.databind.node.ObjectNode -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.nip01Core.jackson.InliningTagArrayPrettyPrinter import com.vitorpamplona.quartz.nip55AndroidSigner.api.foreground.intents.results.IntentResult import com.vitorpamplona.quartz.nip55AndroidSigner.api.foreground.intents.results.IntentResultJsonDeserializer @@ -35,11 +34,10 @@ import com.vitorpamplona.quartz.nip55AndroidSigner.api.foreground.intents.result import com.vitorpamplona.quartz.nip55AndroidSigner.api.permission.Permission import com.vitorpamplona.quartz.nip55AndroidSigner.api.permission.PermissionDeserializer import com.vitorpamplona.quartz.nip55AndroidSigner.api.permission.PermissionSerializer -import java.io.InputStream object JsonMapperNip55 { val defaultMapper: ObjectMapper = - jacksonObjectMapper() + ObjectMapper() .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) .enable(JsonReadFeature.ALLOW_UNESCAPED_CONTROL_CHARS.mappedFeature()) .setDefaultPrettyPrinter(InliningTagArrayPrettyPrinter()) @@ -51,9 +49,41 @@ object JsonMapperNip55 { .addSerializer(Permission::class.java, PermissionSerializer()), ) - inline fun fromJsonTo(json: String): T = defaultMapper.readValue(json) + /** + * Four named entry points instead of one `inline fun fromJsonTo`. + * + * The generic version resolved T through a TypeReference, which reads its type + * argument back off an anonymous subclass's generic superclass. R8 in full mode + * does not keep that: on device it throws + * + * IllegalArgumentException: Internal error: TypeReference constructed without + * actual type information + * + * and because these JavaTypes are built in , a single failure poisons the + * whole object — every later touch comes back as NoClassDefFoundError. That is + * what took out JacksonMapper on the first minified build; this file carries the + * identical pattern and only escaped because NIP-55 needs an external signer + * installed to reach. + * + * `T::class.java` cannot stand in for all four: List erases to List + * and every element comes back a LinkedHashMap. TypeFactory takes Class objects + * directly and keeps the element type, so there is nothing left for R8 to erase. + */ + val permissionType: JavaType = defaultMapper.typeFactory.constructType(Permission::class.java) - inline fun fromJsonTo(json: InputStream): T = defaultMapper.readValue(json) + val permissionArrayType: JavaType = defaultMapper.typeFactory.constructArrayType(Permission::class.java) + + val intentResultType: JavaType = defaultMapper.typeFactory.constructType(IntentResult::class.java) + + val intentResultListType: JavaType = defaultMapper.typeFactory.constructCollectionType(List::class.java, IntentResult::class.java) + + fun fromJsonToPermission(json: String): Permission = defaultMapper.readValue(json, permissionType) + + fun fromJsonToPermissionArray(json: String): Array = defaultMapper.readValue(json, permissionArrayType) + + fun fromJsonToIntentResult(json: String): IntentResult = defaultMapper.readValue(json, intentResultType) + + fun fromJsonToIntentResultList(json: String): List = defaultMapper.readValue(json, intentResultListType) fun toJson(event: ArrayNode): String = defaultMapper.writeValueAsString(event) diff --git a/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/foreground/intents/results/IntentResult.kt b/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/foreground/intents/results/IntentResult.kt index 3cb6635d66..8b6b84780a 100644 --- a/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/foreground/intents/results/IntentResult.kt +++ b/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/foreground/intents/results/IntentResult.kt @@ -55,8 +55,8 @@ data class IntentResult( rejected = data.extras?.containsKey("rejected"), ) - fun fromJson(json: String): IntentResult = JsonMapperNip55.fromJsonTo(json) + fun fromJson(json: String): IntentResult = JsonMapperNip55.fromJsonToIntentResult(json) - fun fromJsonArray(json: String): List = JsonMapperNip55.fromJsonTo>(json) + fun fromJsonArray(json: String): List = JsonMapperNip55.fromJsonToIntentResultList(json) } } diff --git a/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/permission/Permission.kt b/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/permission/Permission.kt index 9e750a1b40..f4c36acaeb 100644 --- a/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/permission/Permission.kt +++ b/quartz/src/androidMain/kotlin/com/vitorpamplona/quartz/nip55AndroidSigner/api/permission/Permission.kt @@ -30,8 +30,8 @@ class Permission( fun toJson(): String = JsonMapperNip55.toJson(this) companion object { - fun fromJson(json: String): Permission = JsonMapperNip55.fromJsonTo(json) + fun fromJson(json: String): Permission = JsonMapperNip55.fromJsonToPermission(json) - fun fromJsonArray(json: String): Array = JsonMapperNip55.fromJsonTo>(json) + fun fromJsonArray(json: String): Array = JsonMapperNip55.fromJsonToPermissionArray(json) } } diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/experimental/clink/kotlinSerialization/ClinkKSerializers.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/experimental/clink/kotlinSerialization/ClinkKSerializers.kt index 6e2749cc3d..8cf25bc1d9 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/experimental/clink/kotlinSerialization/ClinkKSerializers.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/experimental/clink/kotlinSerialization/ClinkKSerializers.kt @@ -34,14 +34,19 @@ import com.vitorpamplona.quartz.experimental.clink.offers.OfferReceipt import com.vitorpamplona.quartz.experimental.clink.offers.OfferRequest import com.vitorpamplona.quartz.experimental.clink.offers.OfferResponse import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.anyToJsonElement -import com.vitorpamplona.quartz.nip47WalletConnect.kotlinSerialization.toAnyMap +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.decodeRootJsonObject +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.intOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.longOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.objOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringListOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.toAnyMap import kotlinx.serialization.KSerializer import kotlinx.serialization.descriptors.SerialDescriptor import kotlinx.serialization.descriptors.buildClassSerialDescriptor import kotlinx.serialization.encoding.Decoder import kotlinx.serialization.encoding.Encoder import kotlinx.serialization.json.JsonArray -import kotlinx.serialization.json.JsonDecoder import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonEncoder import kotlinx.serialization.json.JsonNull @@ -51,7 +56,6 @@ import kotlinx.serialization.json.buildJsonArray import kotlinx.serialization.json.buildJsonObject import kotlinx.serialization.json.intOrNull import kotlinx.serialization.json.jsonObject -import kotlinx.serialization.json.jsonPrimitive import kotlinx.serialization.json.longOrNull import kotlinx.serialization.json.put @@ -60,19 +64,13 @@ import kotlinx.serialization.json.put * (`OfferRequest`/`OfferResponse`/`OfferReceipt`, `DebitRequest`/`DebitResponse`, * `ManageRequest`/`ManageResponse`). They mirror what Jackson does reflectively on * JVM/Android — including coercing a lone `details` object into a one-element list - * (Jackson's `ACCEPT_SINGLE_VALUE_AS_ARRAY`) — so native targets parse the same wire shapes. + * (Jackson's `ACCEPT_SINGLE_VALUE_AS_ARRAY`) — so every target parses the same wire shapes. + * + * Field readers come from `nip01Core.kotlinSerialization.LenientJson`, shared with the + * NIP-47 serializers. The private copies that used to sit here read + * `it.jsonPrimitive.content`, which throws the moment a peer sends an object or an array + * where a string belongs — one malformed field took down the whole offer. */ - -private fun JsonObject.stringOrNull(key: String): String? = get(key)?.let { if (it is JsonNull) null else it.jsonPrimitive.content } - -private fun JsonObject.longOrNull(key: String): Long? = get(key)?.let { if (it is JsonNull) null else it.jsonPrimitive.longOrNull } - -private fun JsonObject.intOrNull(key: String): Int? = get(key)?.let { if (it is JsonNull) null else it.jsonPrimitive.intOrNull } - -private fun JsonObject.objectOrNull(key: String): JsonObject? = get(key) as? JsonObject - -private fun JsonObject.stringListOrNull(key: String): List? = (get(key) as? JsonArray)?.map { it.jsonPrimitive.content } - private fun serializeSatRange(range: SatRange): JsonObject = buildJsonObject { range.min?.let { put("min", it) } @@ -117,11 +115,11 @@ object OfferRequestKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): OfferRequest { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return OfferRequest( offer = obj.stringOrNull("offer"), amount_sats = obj.longOrNull("amount_sats"), - payer_data = obj.objectOrNull("payer_data")?.toAnyMap(), + payer_data = obj.objOrNull("payer_data")?.toAnyMap(), zap = obj.stringOrNull("zap"), expires_in_seconds = obj.longOrNull("expires_in_seconds"), description = obj.stringOrNull("description"), @@ -148,12 +146,12 @@ object OfferResponseKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): OfferResponse { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return OfferResponse( bolt11 = obj.stringOrNull("bolt11"), error = obj.stringOrNull("error"), code = obj.intOrNull("code"), - range = obj.objectOrNull("range")?.let { parseSatRange(it) }, + range = obj.objOrNull("range")?.let { parseSatRange(it) }, latest = obj.stringOrNull("latest"), ) } @@ -175,7 +173,7 @@ object OfferReceiptKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): OfferReceipt { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return OfferReceipt( res = obj.stringOrNull("res"), preimage = obj.stringOrNull("preimage"), @@ -203,14 +201,14 @@ object DebitRequestKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): DebitRequest { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return DebitRequest( pointer = obj.stringOrNull("pointer"), amount_sats = obj.longOrNull("amount_sats"), bolt11 = obj.stringOrNull("bolt11"), description = obj.stringOrNull("description"), k1 = obj.stringOrNull("k1"), - frequency = obj.objectOrNull("frequency")?.let { parseDebitFrequency(it) }, + frequency = obj.objOrNull("frequency")?.let { parseDebitFrequency(it) }, ) } @@ -248,15 +246,15 @@ object DebitResponseKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): DebitResponse { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return DebitResponse( res = obj.stringOrNull("res"), preimage = obj.stringOrNull("preimage"), code = obj.intOrNull("code"), error = obj.stringOrNull("error"), - range = obj.objectOrNull("range")?.let { parseSatRange(it) }, + range = obj.objOrNull("range")?.let { parseSatRange(it) }, retry_after = obj.longOrNull("retry_after"), - delta = obj.objectOrNull("delta")?.let { parseGfyDelta(it) }, + delta = obj.objOrNull("delta")?.let { parseGfyDelta(it) }, ) } } @@ -279,12 +277,12 @@ object ManageRequestKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): ManageRequest { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return ManageRequest( resource = obj.stringOrNull("resource"), action = obj.stringOrNull("action"), pointer = obj.stringOrNull("pointer"), - offer = obj.objectOrNull("offer")?.let { parseManageOffer(it) }, + offer = obj.objOrNull("offer")?.let { parseManageOffer(it) }, ) } @@ -297,7 +295,7 @@ object ManageRequestKSerializer : KSerializer { private fun parseManageOffer(obj: JsonObject): ManageOffer = ManageOffer( id = obj.stringOrNull("id"), - fields = obj.objectOrNull("fields")?.let { parseOfferFields(it) }, + fields = obj.objOrNull("fields")?.let { parseOfferFields(it) }, ) private fun serializeOfferFields(fields: OfferFields): JsonObject = @@ -342,7 +340,7 @@ object ManageResponseKSerializer : KSerializer { } override fun deserialize(decoder: Decoder): ManageResponse { - val obj = (decoder as JsonDecoder).decodeJsonElement().jsonObject + val obj = decoder.decodeRootJsonObject("A CLINK message") return ManageResponse( res = obj.stringOrNull("res"), resource = obj.stringOrNull("resource"), @@ -350,9 +348,9 @@ object ManageResponseKSerializer : KSerializer { code = obj.intOrNull("code"), error = obj.stringOrNull("error"), field = obj.stringOrNull("field"), - range = obj.objectOrNull("range")?.let { parseSatRange(it) }, + range = obj.objOrNull("range")?.let { parseSatRange(it) }, retry_after = obj.longOrNull("retry_after"), - delta = obj.objectOrNull("delta")?.let { parseGfyDelta(it) }, + delta = obj.objOrNull("delta")?.let { parseGfyDelta(it) }, ) } diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/JsonAnyExt.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/JsonAnyExt.kt index fc1a6be6b7..254c9356d4 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/JsonAnyExt.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/JsonAnyExt.kt @@ -21,8 +21,10 @@ package com.vitorpamplona.quartz.nip01Core.kotlinSerialization import com.vitorpamplona.quartz.nip01Core.core.RawJson +import kotlinx.serialization.json.JsonArray import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonNull +import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.JsonPrimitive import kotlinx.serialization.json.JsonUnquotedLiteral import kotlinx.serialization.json.add @@ -56,3 +58,32 @@ fun anyToJsonElement(value: Any?): JsonElement = is Array<*> -> buildJsonArray { value.forEach { add(anyToJsonElement(it)) } } else -> JsonPrimitive(value.toString()) } + +/** + * The inverse of [anyToJsonElement]: decodes a [JsonElement] back into the untyped + * `Any?` tree those free-form fields are modelled as. + * + * An unquoted primitive is narrowed to the most specific type it parses as, so a + * round trip through JSON does not turn `true` into `"true"`. Integers are tried + * BEFORE doubles: `toDoubleOrNull` happily accepts "42" and answers 42.0, which + * then re-encodes as `42.0` and changes the bytes of a metadata field we were only + * meant to carry. A quoted primitive stays a String, because the quotes are the + * peer telling us it is one. + * + * Lives here, not under a NIP, for the same reason [anyToJsonElement] does — CLINK + * and NIP-47 both need it, and a per-NIP copy is how the two backends drift. + */ +fun JsonElement.toAnyValue(): Any = + when (this) { + is JsonPrimitive -> + if (isString) { + content + } else { + content.toBooleanStrictOrNull() ?: content.toLongOrNull() ?: content.toDoubleOrNull() ?: content + } + + is JsonObject -> toAnyMap() + is JsonArray -> map { it.toAnyValue() } + } + +fun JsonObject.toAnyMap(): Map = entries.associate { it.key to it.value.toAnyValue() } diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/LenientJson.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/LenientJson.kt new file mode 100644 index 0000000000..a205ca07b5 --- /dev/null +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/LenientJson.kt @@ -0,0 +1,159 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.nip01Core.kotlinSerialization + +import kotlinx.serialization.encoding.Decoder +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonDecoder +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonNull +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive + +/** + * Field readers for JSON written by somebody else. + * + * These back the hand-written NIP-47 and CLINK serializers, where every byte + * arrives from a third-party wallet or client. The rule they all follow: a field + * that is missing, null, or of the wrong shape reads as `null` and the rest of + * the message still parses. Nothing here throws. + * + * That is deliberately MORE forgiving than what Jackson did on this path. Jackson + * ignored unknown properties but still raised MismatchedInputException when a + * declared `String` arrived as an object, taking the whole message down over one + * bad field. A payment response whose `fees_paid` is `"12"` instead of `12`, or + * whose `metadata` is a string instead of an object, is worth reading for the + * preimage it does carry. + * + * Use `jsonObject` / `jsonPrimitive` / `.content` directly only where the value + * is ours and its shape is guaranteed — they throw, which is what these avoid. + * + * The base case: JSON `null` carries no more information than an absent key, so + * every reader here treats the two alike. + */ +private fun JsonElement?.presentOrNull(): JsonElement? = if (this == null || this is JsonNull) null else this + +fun JsonElement?.asObjectOrNull(): JsonObject? = presentOrNull() as? JsonObject + +/** + * The element as an array — wrapping a lone value in a one-element array, which is + * what Jackson's ACCEPT_SINGLE_VALUE_AS_ARRAY did here. Several Nostr-native RPCs + * (CLINK Manage `details`, typed `OfferData | OfferData[]`) answer with a bare + * object for a single result and an array for a list. + */ +fun JsonElement?.asArrayOrNull(): JsonArray? = + when (val e = presentOrNull()) { + null -> null + is JsonArray -> e + else -> JsonArray(listOf(e)) + } + +private fun JsonElement?.asPrimitiveOrNull(): JsonPrimitive? = presentOrNull() as? JsonPrimitive + +// ---- object field readers --------------------------------------------------- + +fun JsonObject.objOrNull(key: String): JsonObject? = this[key].asObjectOrNull() + +fun JsonObject.arrayOrNull(key: String): JsonArray? = this[key].asArrayOrNull() + +/** Text content, or null when the key is absent, null, an object or an array. */ +fun JsonObject.stringOrNull(key: String): String? = this[key].asPrimitiveOrNull()?.content + +/** + * Like [stringOrNull] but drops the empty string too, for the many fields where a + * peer writes `""` to mean "I have nothing for this". + */ +fun JsonObject.nonEmptyStringOrNull(key: String): String? = stringOrNull(key)?.ifBlank { null } + +/** Accepts `12`, `"12"` and `12.0` — wallets send all three for the same field. */ +fun JsonObject.longOrNull(key: String): Long? { + val raw = this[key].asPrimitiveOrNull()?.content ?: return null + return raw.toLongOrNull() ?: raw.toDoubleOrNull()?.takeIf { it.isFinite() }?.toLong() +} + +/** + * Out of Int range reads as null, not as the low 32 bits. `Long.toInt()` truncates: + * a `limit` of 2^32 would come back 0 and an `offset` of 3_000_000_000 negative. + * Every other reader here answers a value it cannot represent with null, and a + * plausible-looking wrong number is the one outcome worse than no number. + */ +fun JsonObject.intOrNull(key: String): Int? = longOrNull(key)?.takeIf { it >= Int.MIN_VALUE.toLong() && it <= Int.MAX_VALUE.toLong() }?.toInt() + +/** + * Finite values only. `"NaN"` and `"Infinity"` parse as Doubles in Kotlin, and a + * NaN read out of a peer's JSON propagates silently through every comparison it + * touches — it is not a number the caller can do anything with. + */ +fun JsonObject.doubleOrNull(key: String): Double? = + this[key] + .asPrimitiveOrNull() + ?.content + ?.toDoubleOrNull() + ?.takeIf { it.isFinite() } + +/** Accepts `true`/`false`, `"true"`/`"false"`, and the `1`/`0` some wallets send. */ +fun JsonObject.booleanOrNull(key: String): Boolean? { + val raw = this[key].asPrimitiveOrNull()?.content ?: return null + return raw.toBooleanStrictOrNull() + ?: when (raw) { + "1" -> true + "0" -> false + else -> null + } +} + +/** + * Every string in the array, skipping entries that are not primitives rather than + * failing the array. A lone string is read as a single-element list. + */ +fun JsonObject.stringListOrNull(key: String): List? = arrayOrNull(key)?.mapNotNull { it.asPrimitiveOrNull()?.content } + +/** The object as a plain map, or null when the key holds anything else. */ +fun JsonObject.anyMapOrNull(key: String): Map? = objOrNull(key)?.toAnyMap() + +/** Every object in the array, skipping entries that are not objects. */ +fun JsonObject.objectListOrNull(key: String): List? = arrayOrNull(key)?.mapNotNull { it.asObjectOrNull() } + +/** A shape name for error messages that survives R8 — `::class.simpleName` does not. */ +private fun JsonElement.shapeName(): String = + when (this) { + is JsonNull -> "null" + is JsonPrimitive -> if (isString) "a string" else "a number or boolean" + is JsonArray -> "an array" + is JsonObject -> "an object" + } + +/** + * The root of the message as an object. + * + * The one place these serializers still refuse input: a payload whose root is an + * array, a bare string or null is not a partially-readable message, it is not a + * message. Fails with an [IllegalArgumentException] naming what arrived, rather + * than the ClassCastException `decodeJsonElement().jsonObject` raises. + */ +fun Decoder.decodeRootJsonObject(what: String): JsonObject { + val decoder = + this as? JsonDecoder + ?: throw IllegalArgumentException("$what can only be read from JSON") + val element = decoder.decodeJsonElement() + return element as? JsonObject + ?: throw IllegalArgumentException("$what must be a JSON object, but the payload is ${element.shapeName()}") +} diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/JsonExt.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/JsonExt.kt deleted file mode 100644 index 472972cb6e..0000000000 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/JsonExt.kt +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.kotlinSerialization - -import kotlinx.serialization.json.JsonArray -import kotlinx.serialization.json.JsonElement -import kotlinx.serialization.json.JsonObject -import kotlinx.serialization.json.JsonPrimitive - -// Helper function to convert JsonElement to standard Kotlin types recursively -fun JsonElement.toAnyValue(): Any = - when (this) { - is JsonPrimitive -> { - if (isString) { - content - } else { - content.toBooleanStrictOrNull() ?: content.toDoubleOrNull() ?: content.toLongOrNull() ?: content - } - } - - is JsonObject -> { - toAnyMap() - } - - is JsonArray -> { - map { it.toAnyValue() } - } - } - -fun JsonObject.toAnyMap(): Map = entries.associate { it.key to it.value.toAnyValue() } diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47NotificationKSerializer.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47NotificationKSerializer.kt index e3588f77d5..527e8f235d 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47NotificationKSerializer.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47NotificationKSerializer.kt @@ -20,6 +20,10 @@ */ package com.vitorpamplona.quartz.nip47WalletConnect.kotlinSerialization +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.decodeRootJsonObject +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.longOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.objOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringOrNull import com.vitorpamplona.quartz.nip47WalletConnect.rpc.HoldInvoiceAcceptedData import com.vitorpamplona.quartz.nip47WalletConnect.rpc.HoldInvoiceAcceptedNotification import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Notification @@ -33,10 +37,8 @@ import kotlinx.serialization.encoding.Decoder import kotlinx.serialization.encoding.Encoder import kotlinx.serialization.json.JsonDecoder import kotlinx.serialization.json.JsonEncoder -import kotlinx.serialization.json.JsonNull import kotlinx.serialization.json.buildJsonObject import kotlinx.serialization.json.jsonObject -import kotlinx.serialization.json.jsonPrimitive import kotlinx.serialization.json.longOrNull import kotlinx.serialization.json.put @@ -88,16 +90,15 @@ object Nip47NotificationKSerializer : KSerializer { override fun deserialize(decoder: Decoder): Notification { val jsonDecoder = decoder as JsonDecoder - val jsonObject = jsonDecoder.decodeJsonElement().jsonObject - val notificationType = - jsonObject["notification_type"]?.let { if (it is JsonNull) null else it.jsonPrimitive.content } + val jsonObject = decoder.decodeRootJsonObject("An NWC notification") + val notificationType = jsonObject.stringOrNull("notification_type") return when (notificationType) { NwcNotificationType.PAYMENT_RECEIVED -> { PaymentReceivedNotification( notification = Nip47ResponseKSerializer.parseTransaction( - jsonObject["notification"]?.jsonObject, + jsonObject.objOrNull("notification"), ), ) } @@ -106,24 +107,24 @@ object Nip47NotificationKSerializer : KSerializer { PaymentSentNotification( notification = Nip47ResponseKSerializer.parseTransaction( - jsonObject["notification"]?.jsonObject, + jsonObject.objOrNull("notification"), ), ) } NwcNotificationType.HOLD_INVOICE_ACCEPTED -> { - val notifObj = jsonObject["notification"]?.jsonObject + val notifObj = jsonObject.objOrNull("notification") HoldInvoiceAcceptedNotification( notification = notifObj?.let { HoldInvoiceAcceptedData( - type = it["type"]?.jsonPrimitive?.content, - invoice = it["invoice"]?.jsonPrimitive?.content, - payment_hash = it["payment_hash"]?.jsonPrimitive?.content, - amount = it["amount"]?.jsonPrimitive?.longOrNull, - created_at = it["created_at"]?.jsonPrimitive?.longOrNull, - expires_at = it["expires_at"]?.jsonPrimitive?.longOrNull, - settle_deadline = it["settle_deadline"]?.jsonPrimitive?.longOrNull, + type = it.stringOrNull("type"), + invoice = it.stringOrNull("invoice"), + payment_hash = it.stringOrNull("payment_hash"), + amount = it.longOrNull("amount"), + created_at = it.longOrNull("created_at"), + expires_at = it.longOrNull("expires_at"), + settle_deadline = it.longOrNull("settle_deadline"), ) }, ) diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47RequestKSerializer.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47RequestKSerializer.kt index 8f281ebc4b..f0f1a16251 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47RequestKSerializer.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47RequestKSerializer.kt @@ -20,7 +20,16 @@ */ package com.vitorpamplona.quartz.nip47WalletConnect.kotlinSerialization +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.anyMapOrNull import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.anyToJsonElement +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.booleanOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.decodeRootJsonObject +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.intOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.longOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.objOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.objectListOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringListOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringOrNull import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceMethod import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceParams import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionMethod @@ -58,17 +67,13 @@ import kotlinx.serialization.encoding.Decoder import kotlinx.serialization.encoding.Encoder import kotlinx.serialization.json.JsonDecoder import kotlinx.serialization.json.JsonEncoder -import kotlinx.serialization.json.JsonNull import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.add import kotlinx.serialization.json.booleanOrNull import kotlinx.serialization.json.buildJsonArray import kotlinx.serialization.json.buildJsonObject -import kotlinx.serialization.json.contentOrNull import kotlinx.serialization.json.intOrNull -import kotlinx.serialization.json.jsonArray import kotlinx.serialization.json.jsonObject -import kotlinx.serialization.json.jsonPrimitive import kotlinx.serialization.json.longOrNull import kotlinx.serialization.json.put @@ -258,8 +263,8 @@ object Nip47RequestKSerializer : KSerializer { override fun deserialize(decoder: Decoder): Request { val jsonDecoder = decoder as JsonDecoder - val jsonObject = jsonDecoder.decodeJsonElement().jsonObject - val method = jsonObject["method"]?.let { if (it is JsonNull) null else it.jsonPrimitive.content } + val jsonObject = decoder.decodeRootJsonObject("An NWC request") + val method = jsonObject.stringOrNull("method") return when (method) { NwcMethod.PAY_INVOICE -> parsePayInvoice(jsonObject) @@ -282,62 +287,61 @@ object Nip47RequestKSerializer : KSerializer { } private fun parsePayInvoice(json: JsonObject): PayInvoiceMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return PayInvoiceMethod( params?.let { PayInvoiceParams( - invoice = it["invoice"]?.jsonPrimitive?.content, - amount = it["amount"]?.jsonPrimitive?.longOrNull, - metadata = it["metadata"]?.jsonObject?.toAnyMap(), + invoice = it.stringOrNull("invoice"), + amount = it.longOrNull("amount"), + metadata = it.anyMapOrNull("metadata"), ) }, ) } private fun parsePay(json: JsonObject): PayMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return PayMethod( params?.let { // contentOrNull / `as? JsonObject` treat an explicit JSON `null` as absent — // Jackson (JVM/Android) writes null-valued keys, so a native/iOS peer parsing // that output must not read `JsonNull` as the string "null" or crash on it. PayParams( - payment = it["payment"]?.jsonPrimitive?.contentOrNull, - amount = it["amount"]?.jsonPrimitive?.longOrNull, - payer_note = it["payer_note"]?.jsonPrimitive?.contentOrNull, - metadata = (it["metadata"] as? JsonObject)?.toAnyMap(), + payment = it.stringOrNull("payment"), + amount = it.longOrNull("amount"), + payer_note = it.stringOrNull("payer_note"), + metadata = it.anyMapOrNull("metadata"), ) }, ) } private fun parseReceive(json: JsonObject): ReceiveMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return ReceiveMethod( params?.let { ReceiveParams( - amount = it["amount"]?.jsonPrimitive?.longOrNull, - description = it["description"]?.jsonPrimitive?.contentOrNull, - metadata = (it["metadata"] as? JsonObject)?.toAnyMap(), + amount = it.longOrNull("amount"), + description = it.stringOrNull("description"), + metadata = it.anyMapOrNull("metadata"), ) }, ) } private fun parsePayKeysend(json: JsonObject): PayKeysendMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return PayKeysendMethod( params?.let { PayKeysendParams( - amount = it["amount"]?.jsonPrimitive?.longOrNull, - pubkey = it["pubkey"]?.jsonPrimitive?.content, - preimage = it["preimage"]?.jsonPrimitive?.content, + amount = it.longOrNull("amount"), + pubkey = it.stringOrNull("pubkey"), + preimage = it.stringOrNull("preimage"), tlv_records = - it["tlv_records"]?.jsonArray?.map { record -> - val obj = record.jsonObject + it.objectListOrNull("tlv_records")?.map { record -> TlvRecord( - type = obj["type"]?.jsonPrimitive?.longOrNull, - value = obj["value"]?.jsonPrimitive?.content, + type = record.longOrNull("type"), + value = record.stringOrNull("value"), ) }, ) @@ -346,113 +350,113 @@ object Nip47RequestKSerializer : KSerializer { } private fun parseMakeInvoice(json: JsonObject): MakeInvoiceMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return MakeInvoiceMethod( params?.let { MakeInvoiceParams( - amount = it["amount"]?.jsonPrimitive?.longOrNull, - description = it["description"]?.jsonPrimitive?.content, - description_hash = it["description_hash"]?.jsonPrimitive?.content, - expiry = it["expiry"]?.jsonPrimitive?.longOrNull, - metadata = it["metadata"]?.jsonObject?.toAnyMap(), + amount = it.longOrNull("amount"), + description = it.stringOrNull("description"), + description_hash = it.stringOrNull("description_hash"), + expiry = it.longOrNull("expiry"), + metadata = it.anyMapOrNull("metadata"), ) }, ) } private fun parseLookupInvoice(json: JsonObject): LookupInvoiceMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return LookupInvoiceMethod( params?.let { LookupInvoiceParams( - payment_hash = it["payment_hash"]?.jsonPrimitive?.content, - invoice = it["invoice"]?.jsonPrimitive?.content, + payment_hash = it.stringOrNull("payment_hash"), + invoice = it.stringOrNull("invoice"), ) }, ) } private fun parseListTransactions(json: JsonObject): ListTransactionsMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return ListTransactionsMethod( params?.let { ListTransactionsParams( - from = it["from"]?.jsonPrimitive?.longOrNull, - until = it["until"]?.jsonPrimitive?.longOrNull, - limit = it["limit"]?.jsonPrimitive?.intOrNull, - offset = it["offset"]?.jsonPrimitive?.intOrNull, - unpaid = it["unpaid"]?.jsonPrimitive?.booleanOrNull, - unpaid_outgoing = it["unpaid_outgoing"]?.jsonPrimitive?.booleanOrNull, - unpaid_incoming = it["unpaid_incoming"]?.jsonPrimitive?.booleanOrNull, - type = it["type"]?.jsonPrimitive?.content, + from = it.longOrNull("from"), + until = it.longOrNull("until"), + limit = it.intOrNull("limit"), + offset = it.intOrNull("offset"), + unpaid = it.booleanOrNull("unpaid"), + unpaid_outgoing = it.booleanOrNull("unpaid_outgoing"), + unpaid_incoming = it.booleanOrNull("unpaid_incoming"), + type = it.stringOrNull("type"), ) }, ) } private fun parseSignMessage(json: JsonObject): SignMessageMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return SignMessageMethod( params?.let { SignMessageParams( - message = it["message"]?.jsonPrimitive?.content, + message = it.stringOrNull("message"), ) }, ) } private fun parseCreateConnection(json: JsonObject): CreateConnectionMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return CreateConnectionMethod( params?.let { CreateConnectionParams( - pubkey = it["pubkey"]?.jsonPrimitive?.content, - name = it["name"]?.jsonPrimitive?.content, - request_methods = it["request_methods"]?.jsonArray?.map { m -> m.jsonPrimitive.content }, - notification_types = it["notification_types"]?.jsonArray?.map { n -> n.jsonPrimitive.content }, - max_amount = it["max_amount"]?.jsonPrimitive?.longOrNull, - budget_renewal = it["budget_renewal"]?.jsonPrimitive?.content, - expires_at = it["expires_at"]?.jsonPrimitive?.longOrNull, - isolated = it["isolated"]?.jsonPrimitive?.booleanOrNull, - metadata = it["metadata"]?.jsonObject?.toAnyMap(), + pubkey = it.stringOrNull("pubkey"), + name = it.stringOrNull("name"), + request_methods = it.stringListOrNull("request_methods"), + notification_types = it.stringListOrNull("notification_types"), + max_amount = it.longOrNull("max_amount"), + budget_renewal = it.stringOrNull("budget_renewal"), + expires_at = it.longOrNull("expires_at"), + isolated = it.booleanOrNull("isolated"), + metadata = it.anyMapOrNull("metadata"), ) }, ) } private fun parseMakeHoldInvoice(json: JsonObject): MakeHoldInvoiceMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return MakeHoldInvoiceMethod( params?.let { MakeHoldInvoiceParams( - amount = it["amount"]?.jsonPrimitive?.longOrNull, - description = it["description"]?.jsonPrimitive?.content, - description_hash = it["description_hash"]?.jsonPrimitive?.content, - expiry = it["expiry"]?.jsonPrimitive?.longOrNull, - payment_hash = it["payment_hash"]?.jsonPrimitive?.content, - min_cltv_expiry_delta = it["min_cltv_expiry_delta"]?.jsonPrimitive?.intOrNull, + amount = it.longOrNull("amount"), + description = it.stringOrNull("description"), + description_hash = it.stringOrNull("description_hash"), + expiry = it.longOrNull("expiry"), + payment_hash = it.stringOrNull("payment_hash"), + min_cltv_expiry_delta = it.intOrNull("min_cltv_expiry_delta"), ) }, ) } private fun parseCancelHoldInvoice(json: JsonObject): CancelHoldInvoiceMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return CancelHoldInvoiceMethod( params?.let { CancelHoldInvoiceParams( - payment_hash = it["payment_hash"]?.jsonPrimitive?.content, + payment_hash = it.stringOrNull("payment_hash"), ) }, ) } private fun parseSettleHoldInvoice(json: JsonObject): SettleHoldInvoiceMethod { - val params = json["params"]?.jsonObject + val params = json.objOrNull("params") return SettleHoldInvoiceMethod( params?.let { SettleHoldInvoiceParams( - preimage = it["preimage"]?.jsonPrimitive?.content, + preimage = it.stringOrNull("preimage"), ) }, ) diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47ResponseKSerializer.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47ResponseKSerializer.kt index ded0e24008..92ee8668cd 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47ResponseKSerializer.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/kotlinSerialization/Nip47ResponseKSerializer.kt @@ -20,7 +20,15 @@ */ package com.vitorpamplona.quartz.nip47WalletConnect.kotlinSerialization +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.anyMapOrNull import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.anyToJsonElement +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.decodeRootJsonObject +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.longOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.objOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.objectListOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringListOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.stringOrNull +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.toAnyMap import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceSuccessResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionSuccessResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBalanceSuccessResponse @@ -35,6 +43,7 @@ import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcErrorCode import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcErrorResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcMethod import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcTransaction +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcUnknownResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceErrorResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceSuccessResponse import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayKeysendSuccessResponse @@ -55,10 +64,7 @@ import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.add import kotlinx.serialization.json.buildJsonArray import kotlinx.serialization.json.buildJsonObject -import kotlinx.serialization.json.contentOrNull -import kotlinx.serialization.json.jsonArray import kotlinx.serialization.json.jsonObject -import kotlinx.serialization.json.jsonPrimitive import kotlinx.serialization.json.longOrNull import kotlinx.serialization.json.put @@ -75,6 +81,10 @@ object Nip47ResponseKSerializer : KSerializer { buildJsonObject { put("result_type", value.resultType) when (value) { + is NwcUnknownResponse -> { + value.result?.let { put("result", anyToJsonElement(it)) } + } + is NwcErrorResponse -> { value.error?.let { put("error", serializeNwcError(it)) } } @@ -250,8 +260,8 @@ object Nip47ResponseKSerializer : KSerializer { override fun deserialize(decoder: Decoder): Response { val jsonDecoder = decoder as JsonDecoder - val jsonObject = jsonDecoder.decodeJsonElement().jsonObject - val resultType = jsonObject["result_type"]?.let { if (it is JsonNull) null else it.jsonPrimitive.content } + val jsonObject = decoder.decodeRootJsonObject("An NWC response") + val resultType = jsonObject.stringOrNull("result_type") val hasError = jsonObject["error"]?.let { it !is JsonNull } ?: false val hasResult = jsonObject["result"]?.let { it !is JsonNull } ?: false @@ -262,7 +272,7 @@ object Nip47ResponseKSerializer : KSerializer { } else -> { - val error = jsonObject["error"]?.jsonObject?.let { parseNwcError(it) } + val error = jsonObject.objOrNull("error")?.let { parseNwcError(it) } NwcErrorResponse(resultType ?: "", error) } } @@ -287,11 +297,11 @@ object Nip47ResponseKSerializer : KSerializer { } NwcMethod.MAKE_INVOICE -> { - MakeInvoiceSuccessResponse(parseTransaction(jsonObject["result"]?.jsonObject)) + MakeInvoiceSuccessResponse(parseTransaction(jsonObject.objOrNull("result"))) } NwcMethod.LOOKUP_INVOICE -> { - LookupInvoiceSuccessResponse(parseTransaction(jsonObject["result"]?.jsonObject)) + LookupInvoiceSuccessResponse(parseTransaction(jsonObject.objOrNull("result"))) } NwcMethod.LIST_TRANSACTIONS -> { @@ -319,7 +329,7 @@ object Nip47ResponseKSerializer : KSerializer { } NwcMethod.MAKE_HOLD_INVOICE -> { - MakeHoldInvoiceSuccessResponse(parseTransaction(jsonObject["result"]?.jsonObject)) + MakeHoldInvoiceSuccessResponse(parseTransaction(jsonObject.objOrNull("result"))) } NwcMethod.CANCEL_HOLD_INVOICE -> { @@ -332,28 +342,33 @@ object Nip47ResponseKSerializer : KSerializer { else -> { // backward compatibility: guess by result content - val resultObj = jsonObject["result"]?.jsonObject + val resultObj = jsonObject.objOrNull("result") if (resultObj?.containsKey("preimage") == true) { return parsePayInvoiceSuccess(jsonObject) } - throw IllegalArgumentException("Unknown NWC response type: $resultType") + // A result_type from a newer NIP-47, or an extension we do not + // implement. The response is well-formed; hand it back with the + // result intact rather than failing the parse. + NwcUnknownResponse(resultType ?: "", resultObj?.toAnyMap()) } } } - throw IllegalArgumentException("NWC response has neither result nor error") + // Neither result nor error nor result_type: nothing to dispatch on, but the + // payload was still a valid JSON object, so surface it rather than throw. + return NwcUnknownResponse(resultType ?: "", jsonObject.objOrNull("result")?.toAnyMap()) } private fun parseNwcError(obj: JsonObject): NwcError { val code = - obj["code"]?.jsonPrimitive?.content?.let { codeName -> + obj.stringOrNull("code")?.let { codeName -> try { NwcErrorCode.valueOf(codeName) } catch (_: Exception) { null } } - return NwcError(code, obj["message"]?.jsonPrimitive?.content) + return NwcError(code, obj.stringOrNull("message")) } fun serializeTransaction(transaction: NwcTransaction?): JsonObject? { @@ -379,177 +394,177 @@ object Nip47ResponseKSerializer : KSerializer { fun parseTransaction(obj: JsonObject?): NwcTransaction? { if (obj == null) return null return NwcTransaction( - type = obj["type"]?.jsonPrimitive?.content, - state = obj["state"]?.jsonPrimitive?.content, - invoice = obj["invoice"]?.jsonPrimitive?.content, - description = obj["description"]?.jsonPrimitive?.content, - description_hash = obj["description_hash"]?.jsonPrimitive?.content, - preimage = obj["preimage"]?.jsonPrimitive?.content, - payment_hash = obj["payment_hash"]?.jsonPrimitive?.content, - amount = obj["amount"]?.jsonPrimitive?.longOrNull, - fees_paid = obj["fees_paid"]?.jsonPrimitive?.longOrNull, - created_at = obj["created_at"]?.jsonPrimitive?.longOrNull, - expires_at = obj["expires_at"]?.jsonPrimitive?.longOrNull, - settled_at = obj["settled_at"]?.jsonPrimitive?.longOrNull, - settle_deadline = obj["settle_deadline"]?.jsonPrimitive?.longOrNull, - metadata = obj["metadata"]?.jsonObject?.toAnyMap(), + type = obj.stringOrNull("type"), + state = obj.stringOrNull("state"), + invoice = obj.stringOrNull("invoice"), + description = obj.stringOrNull("description"), + description_hash = obj.stringOrNull("description_hash"), + preimage = obj.stringOrNull("preimage"), + payment_hash = obj.stringOrNull("payment_hash"), + amount = obj.longOrNull("amount"), + fees_paid = obj.longOrNull("fees_paid"), + created_at = obj.longOrNull("created_at"), + expires_at = obj.longOrNull("expires_at"), + settled_at = obj.longOrNull("settled_at"), + settle_deadline = obj.longOrNull("settle_deadline"), + metadata = obj.anyMapOrNull("metadata"), ) } private fun parsePayInvoiceSuccess(json: JsonObject): PayInvoiceSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return PayInvoiceSuccessResponse( result?.let { PayInvoiceSuccessResponse.PayInvoiceResultParams( - preimage = it["preimage"]?.jsonPrimitive?.content, - fees_paid = it["fees_paid"]?.jsonPrimitive?.longOrNull, + preimage = it.stringOrNull("preimage"), + fees_paid = it.longOrNull("fees_paid"), ) }, ) } private fun parsePayInvoiceError(json: JsonObject): PayInvoiceErrorResponse { - val error = json["error"]?.jsonObject + val error = json.objOrNull("error") return PayInvoiceErrorResponse( error?.let { PayInvoiceErrorResponse.PayInvoiceErrorParams( code = - it["code"]?.jsonPrimitive?.content?.let { codeName -> + it.stringOrNull("code")?.let { codeName -> try { NwcErrorCode.valueOf(codeName) } catch (_: Exception) { null } }, - message = it["message"]?.jsonPrimitive?.content, + message = it.stringOrNull("message"), ) }, ) } private fun parsePaySuccess(json: JsonObject): PaySuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return PaySuccessResponse( result?.let { // contentOrNull, not content: an explicit JSON `null` (which Jackson writes // for every null field) must read back as a real null, not the string "null". PaySuccessResponse.PayResult( - transaction_id = it["transaction_id"]?.jsonPrimitive?.contentOrNull, - state = it["state"]?.jsonPrimitive?.contentOrNull, - instruction_type = it["instruction_type"]?.jsonPrimitive?.contentOrNull, - amount = it["amount"]?.jsonPrimitive?.longOrNull, - fees_paid = it["fees_paid"]?.jsonPrimitive?.longOrNull, - payment_hash = it["payment_hash"]?.jsonPrimitive?.contentOrNull, - preimage = it["preimage"]?.jsonPrimitive?.contentOrNull, - payer_proof = it["payer_proof"]?.jsonPrimitive?.contentOrNull, - txid = it["txid"]?.jsonPrimitive?.contentOrNull, - failure_reason = it["failure_reason"]?.jsonPrimitive?.contentOrNull, - created_at = it["created_at"]?.jsonPrimitive?.longOrNull, - settled_at = it["settled_at"]?.jsonPrimitive?.longOrNull, + transaction_id = it.stringOrNull("transaction_id"), + state = it.stringOrNull("state"), + instruction_type = it.stringOrNull("instruction_type"), + amount = it.longOrNull("amount"), + fees_paid = it.longOrNull("fees_paid"), + payment_hash = it.stringOrNull("payment_hash"), + preimage = it.stringOrNull("preimage"), + payer_proof = it.stringOrNull("payer_proof"), + txid = it.stringOrNull("txid"), + failure_reason = it.stringOrNull("failure_reason"), + created_at = it.longOrNull("created_at"), + settled_at = it.longOrNull("settled_at"), ) }, ) } private fun parseReceiveSuccess(json: JsonObject): ReceiveSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return ReceiveSuccessResponse( result?.let { ReceiveSuccessResponse.ReceiveResult( - bip321 = it["bip321"]?.jsonPrimitive?.contentOrNull, - transaction_id = it["transaction_id"]?.jsonPrimitive?.contentOrNull, + bip321 = it.stringOrNull("bip321"), + transaction_id = it.stringOrNull("transaction_id"), ) }, ) } private fun parsePayKeysendSuccess(json: JsonObject): PayKeysendSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return PayKeysendSuccessResponse( result?.let { PayKeysendSuccessResponse.PayKeysendResult( - preimage = it["preimage"]?.jsonPrimitive?.content, - fees_paid = it["fees_paid"]?.jsonPrimitive?.longOrNull, + preimage = it.stringOrNull("preimage"), + fees_paid = it.longOrNull("fees_paid"), ) }, ) } private fun parseListTransactionsSuccess(json: JsonObject): ListTransactionsSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return ListTransactionsSuccessResponse( result?.let { ListTransactionsSuccessResponse.ListTransactionsResult( - transactions = it["transactions"]?.jsonArray?.mapNotNull { t -> parseTransaction(t.jsonObject) }, - total_count = it["total_count"]?.jsonPrimitive?.longOrNull, + transactions = it.objectListOrNull("transactions")?.mapNotNull { t -> parseTransaction(t) }, + total_count = it.longOrNull("total_count"), ) }, ) } private fun parseGetBalanceSuccess(json: JsonObject): GetBalanceSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return GetBalanceSuccessResponse( result?.let { GetBalanceSuccessResponse.GetBalanceResult( - balance = it["balance"]?.jsonPrimitive?.longOrNull, + balance = it.longOrNull("balance"), ) }, ) } private fun parseGetInfoSuccess(json: JsonObject): GetInfoSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return GetInfoSuccessResponse( result?.let { GetInfoSuccessResponse.GetInfoResult( - alias = it["alias"]?.jsonPrimitive?.content, - color = it["color"]?.jsonPrimitive?.content, - pubkey = it["pubkey"]?.jsonPrimitive?.content, - network = it["network"]?.jsonPrimitive?.content, - block_height = it["block_height"]?.jsonPrimitive?.longOrNull, - block_hash = it["block_hash"]?.jsonPrimitive?.content, - methods = it["methods"]?.jsonArray?.map { m -> m.jsonPrimitive.content }, - notifications = it["notifications"]?.jsonArray?.map { n -> n.jsonPrimitive.content }, - metadata = it["metadata"]?.jsonObject?.toAnyMap(), - lud16 = it["lud16"]?.jsonPrimitive?.content, + alias = it.stringOrNull("alias"), + color = it.stringOrNull("color"), + pubkey = it.stringOrNull("pubkey"), + network = it.stringOrNull("network"), + block_height = it.longOrNull("block_height"), + block_hash = it.stringOrNull("block_hash"), + methods = it.stringListOrNull("methods"), + notifications = it.stringListOrNull("notifications"), + metadata = it.anyMapOrNull("metadata"), + lud16 = it.stringOrNull("lud16"), ) }, ) } private fun parseGetBudgetSuccess(json: JsonObject): GetBudgetSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return GetBudgetSuccessResponse( result?.let { GetBudgetSuccessResponse.GetBudgetResult( - used_budget = it["used_budget"]?.jsonPrimitive?.longOrNull, - total_budget = it["total_budget"]?.jsonPrimitive?.longOrNull, - renews_at = it["renews_at"]?.jsonPrimitive?.longOrNull, - renewal_period = it["renewal_period"]?.jsonPrimitive?.content, + used_budget = it.longOrNull("used_budget"), + total_budget = it.longOrNull("total_budget"), + renews_at = it.longOrNull("renews_at"), + renewal_period = it.stringOrNull("renewal_period"), ) }, ) } private fun parseSignMessageSuccess(json: JsonObject): SignMessageSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return SignMessageSuccessResponse( result?.let { SignMessageSuccessResponse.SignMessageResult( - message = it["message"]?.jsonPrimitive?.content, - signature = it["signature"]?.jsonPrimitive?.content, + message = it.stringOrNull("message"), + signature = it.stringOrNull("signature"), ) }, ) } private fun parseCreateConnectionSuccess(json: JsonObject): CreateConnectionSuccessResponse { - val result = json["result"]?.jsonObject + val result = json.objOrNull("result") return CreateConnectionSuccessResponse( result?.let { CreateConnectionSuccessResponse.CreateConnectionResult( - wallet_pubkey = it["wallet_pubkey"]?.jsonPrimitive?.content, + wallet_pubkey = it.stringOrNull("wallet_pubkey"), ) }, ) diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/NwcTransactionMetadata.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/NwcTransactionMetadata.kt index 7d90f3b256..d5fca147f8 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/NwcTransactionMetadata.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/NwcTransactionMetadata.kt @@ -24,6 +24,7 @@ import com.vitorpamplona.quartz.nip01Core.core.Address import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.RawJson import com.vitorpamplona.quartz.nip01Core.core.isValid +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.toAnyValue import com.vitorpamplona.quartz.nip19Bech32.decodePublicKeyAsHexOrNull class NwcTransactionMetadata( @@ -186,7 +187,7 @@ class NwcTransactionMetadata( * of key order, escaping and number formatting matching by coincidence, and * it fails as a silently unlabelled row rather than as an error. Passing the * bytes through also sidesteps the number-widening hazard in - * [com.vitorpamplona.quartz.nip47WalletConnect.kotlinSerialization.toAnyValue], + * [com.vitorpamplona.quartz.nip01Core.kotlinSerialization.toAnyValue], * which resolves untyped numbers with `toDoubleOrNull()` BEFORE * `toLongOrNull()`: nothing here decomposes the event at all. * diff --git a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/Response.kt b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/Response.kt index b109074f68..6144cfdd47 100644 --- a/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/Response.kt +++ b/quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/rpc/Response.kt @@ -206,3 +206,20 @@ class CreateConnectionSuccessResponse( val wallet_pubkey: String? = null, ) } + +/** + * A successful response whose `result_type` this build does not know. + * + * NIP-47 grows, and the wallet on the other end is somebody else's software: it may + * answer a method added after this release, or one from an extension we do not + * implement. Failing the whole parse would throw away a response that is perfectly + * well-formed and, for anything that reads [result] generically, perfectly usable — + * so the unrecognised ones arrive here with their result intact instead. + * + * It is deliberately NOT an [IErrorResponseLike]: an unknown answer is not a refusal, + * and code that branches on error must not treat it as one. + */ +class NwcUnknownResponse( + resultType: String, + val result: Map? = null, +) : Response(resultType) diff --git a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkMalformedInputTest.kt b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkMalformedInputTest.kt new file mode 100644 index 0000000000..57c479c808 --- /dev/null +++ b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkMalformedInputTest.kt @@ -0,0 +1,123 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.experimental.clink + +import com.vitorpamplona.quartz.experimental.clink.manage.ManageResponse +import com.vitorpamplona.quartz.experimental.clink.offers.OfferRequest +import com.vitorpamplona.quartz.experimental.clink.offers.OfferResponse +import com.vitorpamplona.quartz.nip01Core.core.OptimizedSerializable +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.KotlinSerializationMapper +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue + +/** + * The CLINK half of [com.vitorpamplona.quartz.nip47WalletConnect.Nip47MalformedInputTest]: + * a payment service is somebody else's software too, and CLINK is experimental, so its + * wire shapes are still moving. + */ +class ClinkMalformedInputTest { + private inline fun parse(json: String): T = KotlinSerializationMapper.fromJsonTo(json) + + @Test + fun anOfferRequestParsesFromNothingButTheOffer() { + val req = parse("""{"offer":"lno1abc"}""") + + assertEquals("lno1abc", req.offer) + assertNull(req.amount_sats) + assertNull(req.payer_data) + } + + @Test + fun unknownFieldsAreIgnoredAndWrongShapesCostOnlyTheirField() { + val req = + parse( + """{"offer":"lno1abc","amount_sats":{"not":"a number"}, + "description":"dinner","invented_by_a_newer_service":[1,2]}""", + ) + + assertEquals("lno1abc", req.offer) + assertEquals("dinner", req.description) + assertNull(req.amount_sats) + } + + @Test + fun amountsQuotedAsStringsStillParse() { + val req = parse("""{"offer":"lno1abc","amount_sats":"2500"}""") + + assertEquals(2500L, req.amount_sats) + } + + @Test + fun anErrorResponseSurvivesACodeSentAsAString() { + val res = parse("""{"error":"Invalid Amount","code":"5"}""") + + assertEquals(5, res.code) + assertEquals("Invalid Amount", res.error) + assertTrue(!res.isSuccess()) + } + + @Test + fun aLoneDetailsObjectIsReadAsAOneElementList() { + // Jackson's ACCEPT_SINGLE_VALUE_AS_ARRAY was enabled for exactly this: a service + // answers with a bare object for one result and an array for several. + val res = parse("""{"res":"ok","resource":"offer","details":{"offer_id":"a1"}}""") + + assertEquals(1, res.details?.size) + assertTrue(res.isOk()) + } + + @Test + fun aBrokenEntryInDetailsDoesNotLoseTheGoodOnes() { + val res = + parse( + """{"res":"ok","resource":"offer","details":[{"offer_id":"a1"},"junk",{"offer_id":"a2"}]}""", + ) + + assertEquals(2, res.details?.size) + } + + @Test + fun explicitNullsReadAsAbsent() { + val res = parse("""{"bolt11":null,"error":null,"code":null}""") + + assertNull(res.bolt11) + assertNull(res.error) + assertNull(res.code) + } + + @Test + fun anEmptyObjectIsNotAnException() { + val res = parse("""{}""") + + assertNull(res.bolt11) + assertTrue(!res.isSuccess()) + } + + @Test + fun aRangeOfTheWrongShapeDoesNotFailTheResponse() { + val res = parse("""{"error":"Invalid Amount","code":5,"range":"1-100"}""") + + assertEquals(5, res.code) + assertNull(res.range) + } +} diff --git a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/LenientJsonTest.kt b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/LenientJsonTest.kt new file mode 100644 index 0000000000..61941cc498 --- /dev/null +++ b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip01Core/kotlinSerialization/LenientJsonTest.kt @@ -0,0 +1,75 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.nip01Core.kotlinSerialization + +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertNull + +/** + * The readers behind the NIP-47 and CLINK parsers, where every byte comes from + * somebody else's wallet. Their contract is that a field of the wrong shape reads + * as null and the rest of the message still parses — never a wrong value. + */ +class LenientJsonTest { + private fun obj(json: String) = Json.parseToJsonElement(json) as JsonObject + + @Test + fun intOutOfRangeReadsAsNullRatherThanWrapping() { + // 2^32 truncates to 0 and 3_000_000_000 to a negative int if the Long is + // narrowed with toInt(). A limit of 0 or a negative offset is worse than + // no value at all: it is a plausible-looking wrong answer. + assertNull(obj("""{"limit":4294967296}""").intOrNull("limit")) + assertNull(obj("""{"offset":3000000000}""").intOrNull("offset")) + assertNull(obj("""{"n":-3000000000}""").intOrNull("n")) + } + + @Test + fun intInRangeStillReads() { + assertEquals(42, obj("""{"n":42}""").intOrNull("n")) + assertEquals(Int.MAX_VALUE, obj("""{"n":2147483647}""").intOrNull("n")) + assertEquals(Int.MIN_VALUE, obj("""{"n":-2147483648}""").intOrNull("n")) + // the same string/float tolerance longOrNull has + assertEquals(12, obj("""{"n":"12"}""").intOrNull("n")) + assertEquals(12, obj("""{"n":12.0}""").intOrNull("n")) + } + + @Test + fun longAcceptsTheThreeShapesWalletsSend() { + assertEquals(12L, obj("""{"n":12}""").longOrNull("n")) + assertEquals(12L, obj("""{"n":"12"}""").longOrNull("n")) + assertEquals(12L, obj("""{"n":12.0}""").longOrNull("n")) + assertNull(obj("""{"n":{"nested":1}}""").longOrNull("n")) + assertNull(obj("""{"n":null}""").longOrNull("n")) + assertNull(obj("""{}""").longOrNull("n")) + } + + @Test + fun doubleRejectsNonFiniteText() { + // "NaN" and "Infinity" parse as Doubles in Kotlin but are not values any + // caller can do arithmetic with. + assertNull(obj("""{"n":"NaN"}""").doubleOrNull("n")) + assertNull(obj("""{"n":"Infinity"}""").doubleOrNull("n")) + assertEquals(1.5, obj("""{"n":1.5}""").doubleOrNull("n")) + } +} diff --git a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47KotlinSerializationNullTest.kt b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47KotlinSerializationNullTest.kt index 6e25c9bc33..d3d5078a3f 100644 --- a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47KotlinSerializationNullTest.kt +++ b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47KotlinSerializationNullTest.kt @@ -39,7 +39,7 @@ import kotlin.test.assertNull * null, not the string "null", and must not crash on a null `metadata` object. * * Our own Jackson backend no longer emits those on request params — see - * [com.vitorpamplona.quartz.nip01Core.jackson.OmitNullsMixin] — but a third-party + * Jackson's OmitNullsMixin, now deleted along with that path — but a third-party * wallet still may, so tolerating them on the way in remains required. */ class Nip47KotlinSerializationNullTest { diff --git a/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47MalformedInputTest.kt b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47MalformedInputTest.kt new file mode 100644 index 0000000000..3373492a03 --- /dev/null +++ b/quartz/src/commonTest/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/Nip47MalformedInputTest.kt @@ -0,0 +1,185 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.nip47WalletConnect + +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.KotlinSerializationMapper +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetInfoSuccessResponse +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ListTransactionsSuccessResponse +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcErrorResponse +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcUnknownResponse +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceSuccessResponse +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Response +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue + +/** + * What a NIP-47 response parser has to survive, because the wallet on the other end + * is somebody else's software and NIP-47 keeps growing. + * + * The rule every case here asserts: one bad field costs you that field, never the + * message. Jackson used to bind these reflectively and answered a type mismatch with + * MismatchedInputException — a `fees_paid` of `"0"` instead of `0` threw away a + * settled payment's preimage. + */ +class Nip47MalformedInputTest { + private fun parse(json: String): Response = KotlinSerializationMapper.fromJsonTo(json) + + @Test + fun unknownResultTypeKeepsItsResultInsteadOfFailing() { + // A method from a newer NIP-47, or an extension this build does not implement. + val res = parse("""{"result_type":"make_offer","result":{"offer":"lno1abc","amount":42}}""") + + assertTrue(res is NwcUnknownResponse) + assertEquals("make_offer", res.resultType) + assertEquals("lno1abc", res.result?.get("offer")) + assertEquals(42L, res.result?.get("amount")) + } + + @Test + fun unknownExtraFieldsAreIgnored() { + val res = + parse( + """{"result_type":"pay_invoice","result":{"preimage":"abc","fees_paid":3, + "totally_new_field":{"nested":true}},"extra_root_field":[1,2,3]}""", + ) + + assertTrue(res is PayInvoiceSuccessResponse) + assertEquals("abc", res.result?.preimage) + assertEquals(3L, res.result?.fees_paid) + } + + @Test + fun numbersSentAsStringsStillParse() { + // Several wallets quote their integers. + val res = parse("""{"result_type":"pay_invoice","result":{"preimage":"abc","fees_paid":"12"}}""") + + assertTrue(res is PayInvoiceSuccessResponse) + assertEquals(12L, res.result?.fees_paid) + } + + @Test + fun aFieldOfTheWrongShapeCostsOnlyThatField() { + // `preimage` arrives as an object. The payment still settled; read what is there. + val res = + parse( + """{"result_type":"pay_invoice","result":{"preimage":{"unexpected":"object"},"fees_paid":7}}""", + ) + + assertTrue(res is PayInvoiceSuccessResponse) + assertNull(res.result?.preimage) + assertEquals(7L, res.result?.fees_paid) + } + + @Test + fun explicitNullsReadTheSameAsAbsentKeys() { + val res = parse("""{"result_type":"pay_invoice","result":{"preimage":null,"fees_paid":null}}""") + + assertTrue(res is PayInvoiceSuccessResponse) + assertNull(res.result?.preimage) + assertNull(res.result?.fees_paid) + } + + @Test + fun aBrokenEntryDoesNotTakeDownTheWholeList() { + val res = + parse( + """{"result_type":"list_transactions","result":{"transactions":[ + {"type":"incoming","amount":1000}, + "not-an-object", + {"type":"outgoing","amount":"2000"}]}}""", + ) + + assertTrue(res is ListTransactionsSuccessResponse) + val txs = res.result?.transactions + assertEquals(2, txs?.size) + assertEquals(1000L, txs?.get(0)?.amount) + assertEquals(2000L, txs?.get(1)?.amount) + } + + @Test + fun aStringListDropsNonStringEntriesRatherThanFailing() { + val res = + parse( + """{"result_type":"get_info","result":{"methods":["pay_invoice",{"bad":1},"get_balance"]}}""", + ) + + assertTrue(res is GetInfoSuccessResponse) + assertEquals(listOf("pay_invoice", "get_balance"), res.result?.methods) + } + + @Test + fun aLoneValueIsAcceptedWhereAListIsExpected() { + // Jackson's ACCEPT_SINGLE_VALUE_AS_ARRAY, preserved. + val res = parse("""{"result_type":"get_info","result":{"methods":"pay_invoice"}}""") + + assertTrue(res is GetInfoSuccessResponse) + assertEquals(listOf("pay_invoice"), res.result?.methods) + } + + @Test + fun anErrorWithoutAMessageStillReportsItsCode() { + val res = parse("""{"result_type":"pay_invoice","error":{"code":"INSUFFICIENT_BALANCE"}}""") + + assertTrue(res is com.vitorpamplona.quartz.nip47WalletConnect.rpc.IErrorResponseLike) + assertEquals("INSUFFICIENT_BALANCE", res.errorMessage()) + } + + @Test + fun anErrorWhoseCodeIsUnknownIsStillAnError() { + val res = parse("""{"result_type":"pay_invoice","error":{"code":"SOMETHING_NEW","message":"nope"}}""") + + assertTrue(res is com.vitorpamplona.quartz.nip47WalletConnect.rpc.IErrorResponseLike) + assertEquals("nope", res.errorMessage()) + } + + @Test + fun anEmptyObjectIsNotAnException() { + val res = parse("""{}""") + + assertTrue(res is NwcUnknownResponse) + assertEquals("", res.resultType) + } + + @Test + fun anErrorTypedResponseSurvivesAnErrorFieldOfTheWrongShape() { + // `error` present but a string, not an object: still an error response. + val res = parse("""{"result_type":"pay_invoice","error":"boom"}""") + + assertTrue(res is NwcErrorResponse || res is com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceErrorResponse) + } + + @Test + fun aPayloadThatIsNotAnObjectFailsWithAClearMessage() { + // The one input still refused: there is no message to salvage from an array. + val thrown = + try { + parse("""["not","a","response"]""") + null + } catch (e: IllegalArgumentException) { + e + } + + assertTrue(thrown != null, "an array root must be rejected") + assertTrue(thrown.message?.contains("must be a JSON object") == true, "got: ${thrown.message}") + } +} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/core/OptimizedJsonMapper.jvmAndroid.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/core/OptimizedJsonMapper.jvmAndroid.kt index 89a3287d63..78f12ebe8e 100644 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/core/OptimizedJsonMapper.jvmAndroid.kt +++ b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/core/OptimizedJsonMapper.jvmAndroid.kt @@ -20,18 +20,32 @@ */ package com.vitorpamplona.quartz.nip01Core.core +import com.fasterxml.jackson.core.JsonParseException +import com.fasterxml.jackson.core.JsonProcessingException import com.fasterxml.jackson.databind.RuntimeJsonMappingException +import com.vitorpamplona.quartz.experimental.clink.debits.DebitRequest +import com.vitorpamplona.quartz.experimental.clink.debits.DebitResponse +import com.vitorpamplona.quartz.experimental.clink.manage.ManageRequest +import com.vitorpamplona.quartz.experimental.clink.manage.ManageResponse +import com.vitorpamplona.quartz.experimental.clink.offers.OfferReceipt +import com.vitorpamplona.quartz.experimental.clink.offers.OfferRequest +import com.vitorpamplona.quartz.experimental.clink.offers.OfferResponse import com.vitorpamplona.quartz.nip01Core.jackson.JacksonMapper +import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.KotlinSerializationMapper import com.vitorpamplona.quartz.nip01Core.relay.commands.toClient.Message import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.Command import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Notification +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Request +import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Response import com.vitorpamplona.quartz.nip59Giftwrap.rumors.Rumor +import kotlinx.serialization.SerializationException actual object OptimizedJsonMapper { actual fun fromJson(json: String): Event = try { JacksonMapper.fromJson(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } @@ -40,57 +54,97 @@ actual object OptimizedJsonMapper { actual fun fromJsonToMessage(json: String): Message = try { JacksonMapper.fromJsonToMessage(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } actual fun fromJsonToCommand(json: String): Command = try { JacksonMapper.fromJsonToCommand(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } actual fun fromJsonToTagArray(json: String): Array> = try { JacksonMapper.fromJsonToTagArray(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } actual fun fromJsonToRumor(json: String): Rumor = try { JacksonMapper.fromJsonToRumor(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } actual fun fromJsonToEventTemplate(json: String): EventTemplate = try { JacksonMapper.fromJsonToEventTemplate(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } actual fun fromJsonToEventList(json: String): List = try { JacksonMapper.fromJsonToEventList(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { + } catch (e: JsonParseException) { throw IllegalArgumentException(e.message, e) } actual fun toJson(tags: Array>): String = JacksonMapper.toJson(tags) + /** + * NIP-47 and CLINK read and write through kotlinx on every target, including this + * one. Two reasons, in order of importance: + * + * 1. These are the only types Jackson bound REFLECTIVELY here — the hand-written + * deserializers dispatched to the concrete classes with `treeToValue`. That made + * ~106 classes' field names load-bearing under R8 and cost two blanket keep rules. + * The kotlinx serializers name every field as a string literal, so nothing has to + * be kept. + * 2. Wallets and CLINK peers are other people's software. The kotlinx path reads a + * field of the wrong shape as absent instead of failing the message (see + * nip01Core.kotlinSerialization.LenientJson); Jackson raised + * MismatchedInputException and lost the whole response over one bad field. + * + * Everything else stays on Jackson, which is faster on the event hot path and is + * already non-reflective there. + */ actual inline fun fromJsonTo(json: String): T = - try { - JacksonMapper.fromJsonTo(json) - } catch (e: com.fasterxml.jackson.core.JsonParseException) { - throw IllegalArgumentException(e.message, e) - } catch (e: com.fasterxml.jackson.core.JsonProcessingException) { - throw IllegalArgumentException(e.message, e) - } catch (e: RuntimeJsonMappingException) { - throw IllegalArgumentException(e.message, e) + when (T::class) { + Request::class, Response::class, Notification::class, + OfferRequest::class, OfferResponse::class, OfferReceipt::class, + DebitRequest::class, DebitResponse::class, + ManageRequest::class, ManageResponse::class, + -> + try { + KotlinSerializationMapper.fromJsonTo(json) + } catch (e: SerializationException) { + throw IllegalArgumentException(e.message, e) + } + + else -> + try { + JacksonMapper.fromJsonTo(json) + } catch (e: JsonParseException) { + throw IllegalArgumentException(e.message, e) + } catch (e: JsonProcessingException) { + throw IllegalArgumentException(e.message, e) + } catch (e: RuntimeJsonMappingException) { + throw IllegalArgumentException(e.message, e) + } } - actual fun toJson(value: OptimizedSerializable): String = JacksonMapper.toJson(value) + actual fun toJson(value: OptimizedSerializable): String = + when (value) { + is Request, is Response, is Notification, + is OfferRequest, is OfferResponse, is OfferReceipt, + is DebitRequest, is DebitResponse, + is ManageRequest, is ManageResponse, + -> KotlinSerializationMapper.toJson(value) + + else -> JacksonMapper.toJson(value) + } } diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonMapper.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonMapper.kt index 0c4365031e..c02d6577c3 100644 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonMapper.kt +++ b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonMapper.kt @@ -23,12 +23,10 @@ package com.vitorpamplona.quartz.nip01Core.jackson import com.fasterxml.jackson.core.json.JsonReadFeature import com.fasterxml.jackson.databind.DeserializationFeature import com.fasterxml.jackson.databind.JavaType +import com.fasterxml.jackson.databind.ObjectMapper import com.fasterxml.jackson.databind.module.SimpleModule import com.fasterxml.jackson.databind.node.ArrayNode import com.fasterxml.jackson.databind.node.ObjectNode -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper -import com.fasterxml.jackson.module.kotlin.jacksonTypeRef -import com.fasterxml.jackson.module.kotlin.readValue import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.OptimizedSerializable import com.vitorpamplona.quartz.nip01Core.core.RawJson @@ -53,39 +51,18 @@ import com.vitorpamplona.quartz.nip46RemoteSigner.jackson.BunkerRequestDeseriali import com.vitorpamplona.quartz.nip46RemoteSigner.jackson.BunkerRequestSerializer import com.vitorpamplona.quartz.nip46RemoteSigner.jackson.BunkerResponseDeserializer import com.vitorpamplona.quartz.nip46RemoteSigner.jackson.BunkerResponseSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.jackson.NotificationDeserializer -import com.vitorpamplona.quartz.nip47WalletConnect.jackson.NotificationSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.jackson.RequestDeserializer -import com.vitorpamplona.quartz.nip47WalletConnect.jackson.RequestSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.jackson.ResponseDeserializer -import com.vitorpamplona.quartz.nip47WalletConnect.jackson.ResponseSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ListTransactionsParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.LookupInvoiceParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeHoldInvoiceParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeInvoiceParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Notification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayKeysendParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ReceiveParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Request -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Response -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SettleHoldInvoiceParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SignMessageParams -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.TlvRecord import com.vitorpamplona.quartz.nip59Giftwrap.rumors.Rumor import com.vitorpamplona.quartz.nip59Giftwrap.rumors.jackson.RumorDeserializer import com.vitorpamplona.quartz.nip59Giftwrap.rumors.jackson.RumorSerializer import java.io.InputStream +import kotlin.reflect.KClass class JacksonMapper { companion object { val defaultPrettyPrinter = InliningTagArrayPrettyPrinter() val mapper = - jacksonObjectMapper() + ObjectMapper() .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) .configure(DeserializationFeature.FAIL_ON_TRAILING_TOKENS, false) .configure(DeserializationFeature.UNWRAP_ROOT_VALUE, false) @@ -114,31 +91,6 @@ class JacksonMapper { // nip 59 .addSerializer(Rumor::class.java, RumorSerializer()) .addDeserializer(Rumor::class.java, RumorDeserializer()) - // nip 47 - .addSerializer(Response::class.java, ResponseSerializer()) - .addDeserializer(Response::class.java, ResponseDeserializer()) - .addSerializer(Request::class.java, RequestSerializer()) - .addDeserializer(Request::class.java, RequestDeserializer()) - .addSerializer(Notification::class.java, NotificationSerializer()) - .addDeserializer(Notification::class.java, NotificationDeserializer()) - // NIP-47's optional params are OMITTED when null — see OmitNullsMixin. - // Matches what the kotlinx backend has always done. - .setMixInAnnotation(PayInvoiceParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(PayParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(ReceiveParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(PayKeysendParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(MakeInvoiceParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(LookupInvoiceParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(ListTransactionsParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(MakeHoldInvoiceParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(CancelHoldInvoiceParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(SettleHoldInvoiceParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(SignMessageParams::class.java, OmitNullsMixin::class.java) - .setMixInAnnotation(CreateConnectionParams::class.java, OmitNullsMixin::class.java) - // NESTED, and the only params field that is not a primitive or an - // already-registered type: a TlvRecord inside pay_keysend's - // `tlv_records` has two independently optional fields of its own. - .setMixInAnnotation(TlvRecord::class.java, OmitNullsMixin::class.java) // nip 46 .addDeserializer(BunkerMessage::class.java, BunkerMessageDeserializer()) .addSerializer(BunkerRequest::class.java, BunkerRequestSerializer()) @@ -149,14 +101,28 @@ class JacksonMapper { /** * Shortcuts + * + * Built from Class objects, NOT from a TypeReference. A TypeReference + * reads its type argument back off the anonymous subclass's generic + * superclass, and R8 in full mode does not keep that -- not with + * `-keepattributes Signature`, and not with a `-keep` on the subclasses + * either (both were tried on device). It failed here, in , with + * + * IllegalArgumentException: Internal error: TypeReference constructed + * without actual type information + * + * and a failed is permanent: every later touch of this class + * throws NoClassDefFoundError, so the release build could not hash or sign + * a single event. TypeFactory takes the Class objects directly, so there is + * nothing for R8 to erase. */ - val eventTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef()) - val tagArrayTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef()) - val rumorTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef()) - val eventTemplateTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef>()) - val eventListTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef>()) - val messageTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef()) - val commandTypeInstance: JavaType = mapper.typeFactory.constructType(jacksonTypeRef()) + val eventTypeInstance: JavaType = mapper.typeFactory.constructType(Event::class.java) + val tagArrayTypeInstance: JavaType = mapper.typeFactory.constructType(Array>::class.java) + val rumorTypeInstance: JavaType = mapper.typeFactory.constructType(Rumor::class.java) + val eventTemplateTypeInstance: JavaType = mapper.typeFactory.constructParametricType(EventTemplate::class.java, Event::class.java) + val eventListTypeInstance: JavaType = mapper.typeFactory.constructCollectionType(List::class.java, Event::class.java) + val messageTypeInstance: JavaType = mapper.typeFactory.constructType(Message::class.java) + val commandTypeInstance: JavaType = mapper.typeFactory.constructType(Command::class.java) fun fromJson(json: String): Event = mapper.readValue(json, eventTypeInstance) @@ -172,9 +138,118 @@ class JacksonMapper { fun fromJsonToEventList(json: String): List = mapper.readValue(json, eventListTypeInstance) - inline fun fromJsonTo(json: String): T = mapper.readValue(json) + /** + * The types this mapper has a registered deserializer for, by EXACT class. + * + * Exact is right here: `SimpleModule.addDeserializer(Event::class)` binds that + * class alone — Jackson's SimpleDeserializers does not walk up a hierarchy on + * the read side — so a subclass passed to [fromJsonTo] really would be + * unbound. The serializer side is the mirror image; see [checkSerializable]. + * + * [fromJsonTo] is generic, so nothing in the type system stops a new + * [OptimizedSerializable] being passed to it. Jackson would answer by binding + * that type REFLECTIVELY off its Kotlin constructor parameter names — which + * works in debug, and in a release build writes obfuscated one-letter JSON keys + * onto the wire. That is how NIP-47 and CLINK ended up needing a package keep + * rule each, and the symptom only ever shows up in production. + * + * So the fallback is closed: an unregistered type fails here, loudly, on the + * first call. Register a StdSerializer/StdDeserializer pair below, or route the + * type at kotlinx in OptimizedJsonMapper the way NIP-47 and CLINK are. + */ + @PublishedApi + internal val registered: Set> = + setOf( + Event::class, + Filter::class, + Message::class, + Command::class, + TagArray::class, + EventTemplate::class, + Rumor::class, + BunkerMessage::class, + BunkerRequest::class, + BunkerResponse::class, + ) - inline fun fromJsonTo(json: InputStream): T = mapper.readValue(json) + @PublishedApi + internal fun checkRegistered(type: KClass<*>) { + if (type !in registered) { + throw IllegalArgumentException( + "No Jackson deserializer is registered for $type, so Jackson would bind it " + + "reflectively and emit obfuscated field names in a release build. Register " + + "one in JacksonMapper, or route the type at KotlinSerializationMapper in " + + "OptimizedJsonMapper.", + ) + } + } + + /** + * The write-side twin of [checkRegistered], and the reason it is an `is` chain + * rather than a set: Jackson's SimpleSerializers DOES walk the hierarchy, so + * the serializer registered for [Event] serves every event kind and [Command]'s + * serves NegMsgMessage. Comparing exact classes here would reject all of them. + * + * Without this, [toJson] has the same hole [fromJsonTo] had. Hand it an + * [OptimizedSerializable] with no registered serializer and Jackson falls back + * to bean introspection, naming each field after its getter — names R8 renames, + * so a release build writes `{"a":…}` onto the wire while the debug build looks + * perfect. + * + * [BunkerRequest] and [BunkerResponse] are named instead of their [BunkerMessage] + * parent on purpose: the parent has no serializer of its own, so a third subclass + * should fail here rather than quietly bean-serialize. + * + * Two OptimizedSerializables are deliberately absent, and both would throw if + * they ever arrived here: CLINK's SatRange, which is only ever written as a + * field of an Offer/Debit/Manage message by that message's kotlinx serializer, + * and NIP-55's IntentResult, which goes through JsonMapperNip55 and its own + * registered serializer. Neither reaches this mapper today. + */ + private fun checkSerializable(value: OptimizedSerializable) { + val hasSerializer = + value is Event || + value is Filter || + value is Message || + value is Command || + value is EventTemplate<*> || + value is Rumor || + value is BunkerRequest || + value is BunkerResponse + + if (!hasSerializer) { + throw IllegalArgumentException( + "No Jackson serializer is registered for ${value::class}, so Jackson would " + + "serialize it reflectively and emit obfuscated field names in a release " + + "build. Register one in JacksonMapper, or route the type at " + + "KotlinSerializationMapper in OptimizedJsonMapper.", + ) + } + } + + /** + * `T::class.java`, not a TypeReference — the same reason the JavaTypes above + * are built from Class objects. A TypeReference reads its type argument back + * off the anonymous subclass's generic superclass, which R8 in full mode does + * not keep, and it throws "TypeReference constructed without actual type + * information" on device. This path reaches it through the NIP-46 bunker + * (NostrConnectEvent, RemoteSignerManager, NostrConnectLoginUseCase), which + * needs a remote signer to exercise and so survived the first device run. + * + * Erasure costs nothing here: [checkRegistered] has already limited T to the + * ten registered classes, and each one is answered by a StdDeserializer + * registered against that exact Class. Jackson never has to infer a type + * argument, so there is none to lose. + */ + inline fun fromJsonTo(json: String): T { + checkRegistered(T::class) + return mapper.readValue(json, T::class.java) + } + + inline fun fromJsonTo(json: InputStream): T { + checkRegistered(T::class) + return mapper.readValue(json, T::class.java) + } fun toJson(event: Event): String = EventManualSerializer.toJson(event.id, event.pubKey, event.createdAt, event.kind, event.tags, event.content, event.sig) @@ -207,7 +282,10 @@ class JacksonMapper { fun toJson(event: ObjectNode?): String = mapper.writeValueAsString(event) - fun toJson(value: OptimizedSerializable): String = mapper.writeValueAsString(value) + fun toJson(value: OptimizedSerializable): String { + checkSerializable(value) + return mapper.writeValueAsString(value) + } fun toJson(tags: TagArray): String = mapper.writeValueAsString(tags) } diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/OmitNullsMixin.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/OmitNullsMixin.kt deleted file mode 100644 index 17f4b976aa..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/OmitNullsMixin.kt +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip01Core.jackson - -import com.fasterxml.jackson.annotation.JsonInclude - -/** - * Applied to a reflectively-serialized DTO so Jackson OMITS a null field instead - * of writing it. - * - * Optional protocol fields are absent, not null. A peer is free to type one - * strictly: sending `"from": null` for an absent `from` earned - * `Invalid list_transactions params: from must be an integer` from a NIP-47 - * wallet, and the request failed. - * - * A MIXIN rather than an annotation on the class, because these DTOs live in - * `commonMain` and Jackson annotations are JVM-only. Class-level rather than the - * mapper-wide `setSerializationInclusion`, which in Jackson 2.x also suppresses - * null MAP ENTRIES — and [com.vitorpamplona.quartz.nip01Core.kotlinSerialization.anyToJsonElement] - * deliberately keeps those, so a global setting would close one backend - * divergence by opening another. - */ -@JsonInclude(JsonInclude.Include.NON_NULL) -abstract class OmitNullsMixin diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/NotificationDeserializer.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/NotificationDeserializer.kt deleted file mode 100644 index 70535bb019..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/NotificationDeserializer.kt +++ /dev/null @@ -1,49 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.jackson - -import com.fasterxml.jackson.core.JsonParser -import com.fasterxml.jackson.databind.DeserializationContext -import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.databind.deser.std.StdDeserializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.HoldInvoiceAcceptedNotification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Notification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcNotificationType -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PaymentReceivedNotification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PaymentSentNotification -import com.vitorpamplona.quartz.utils.asTextOrNull - -class NotificationDeserializer : StdDeserializer(Notification::class.java) { - override fun deserialize( - jp: JsonParser, - ctxt: DeserializationContext, - ): Notification? { - val jsonObject: JsonNode = jp.codec.readTree(jp) - val notificationType = jsonObject.get("notification_type")?.asTextOrNull() - - return when (notificationType) { - NwcNotificationType.PAYMENT_RECEIVED -> jp.codec.treeToValue(jsonObject, PaymentReceivedNotification::class.java) - NwcNotificationType.PAYMENT_SENT -> jp.codec.treeToValue(jsonObject, PaymentSentNotification::class.java) - NwcNotificationType.HOLD_INVOICE_ACCEPTED -> jp.codec.treeToValue(jsonObject, HoldInvoiceAcceptedNotification::class.java) - else -> null - } - } -} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/NotificationSerializer.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/NotificationSerializer.kt deleted file mode 100644 index 6d7376fbfe..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/NotificationSerializer.kt +++ /dev/null @@ -1,60 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.jackson - -import com.fasterxml.jackson.core.JsonGenerator -import com.fasterxml.jackson.databind.SerializerProvider -import com.fasterxml.jackson.databind.ser.std.StdSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.HoldInvoiceAcceptedNotification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Notification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PaymentReceivedNotification -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PaymentSentNotification - -class NotificationSerializer : StdSerializer(Notification::class.java) { - override fun serialize( - value: Notification, - gen: JsonGenerator, - provider: SerializerProvider, - ) { - gen.writeStartObject() - gen.writeStringField("notification_type", value.notification_type) - when (value) { - is PaymentReceivedNotification -> { - if (value.notification != null) { - gen.writeObjectField("notification", value.notification) - } - } - - is PaymentSentNotification -> { - if (value.notification != null) { - gen.writeObjectField("notification", value.notification) - } - } - - is HoldInvoiceAcceptedNotification -> { - if (value.notification != null) { - gen.writeObjectField("notification", value.notification) - } - } - } - gen.writeEndObject() - } -} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/RequestDeserializer.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/RequestDeserializer.kt deleted file mode 100644 index 27ec2cc38f..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/RequestDeserializer.kt +++ /dev/null @@ -1,73 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.jackson - -import com.fasterxml.jackson.core.JsonParser -import com.fasterxml.jackson.databind.DeserializationContext -import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.databind.deser.std.StdDeserializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBalanceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBudgetMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetInfoMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ListTransactionsMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.LookupInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeHoldInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayKeysendMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ReceiveMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Request -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SettleHoldInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SignMessageMethod -import com.vitorpamplona.quartz.utils.asTextOrNull - -class RequestDeserializer : StdDeserializer(Request::class.java) { - override fun deserialize( - jp: JsonParser, - ctxt: DeserializationContext, - ): Request? { - val jsonObject: JsonNode = jp.codec.readTree(jp) - val method = jsonObject.get("method")?.asTextOrNull() - - return when (method) { - NwcMethod.PAY_INVOICE -> jp.codec.treeToValue(jsonObject, PayInvoiceMethod::class.java) - NwcMethod.PAY -> jp.codec.treeToValue(jsonObject, PayMethod::class.java) - NwcMethod.RECEIVE -> jp.codec.treeToValue(jsonObject, ReceiveMethod::class.java) - NwcMethod.PAY_KEYSEND -> jp.codec.treeToValue(jsonObject, PayKeysendMethod::class.java) - NwcMethod.MAKE_INVOICE -> jp.codec.treeToValue(jsonObject, MakeInvoiceMethod::class.java) - NwcMethod.LOOKUP_INVOICE -> jp.codec.treeToValue(jsonObject, LookupInvoiceMethod::class.java) - NwcMethod.LIST_TRANSACTIONS -> jp.codec.treeToValue(jsonObject, ListTransactionsMethod::class.java) - NwcMethod.GET_BALANCE -> jp.codec.treeToValue(jsonObject, GetBalanceMethod::class.java) - NwcMethod.GET_INFO -> jp.codec.treeToValue(jsonObject, GetInfoMethod::class.java) - NwcMethod.GET_BUDGET -> jp.codec.treeToValue(jsonObject, GetBudgetMethod::class.java) - NwcMethod.SIGN_MESSAGE -> jp.codec.treeToValue(jsonObject, SignMessageMethod::class.java) - NwcMethod.CREATE_CONNECTION -> jp.codec.treeToValue(jsonObject, CreateConnectionMethod::class.java) - NwcMethod.MAKE_HOLD_INVOICE -> jp.codec.treeToValue(jsonObject, MakeHoldInvoiceMethod::class.java) - NwcMethod.CANCEL_HOLD_INVOICE -> jp.codec.treeToValue(jsonObject, CancelHoldInvoiceMethod::class.java) - NwcMethod.SETTLE_HOLD_INVOICE -> jp.codec.treeToValue(jsonObject, SettleHoldInvoiceMethod::class.java) - else -> null - } - } -} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/RequestSerializer.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/RequestSerializer.kt deleted file mode 100644 index 8bb35de4f3..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/RequestSerializer.kt +++ /dev/null @@ -1,136 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.jackson - -import com.fasterxml.jackson.core.JsonGenerator -import com.fasterxml.jackson.databind.SerializerProvider -import com.fasterxml.jackson.databind.ser.std.StdSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBalanceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBudgetMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetInfoMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ListTransactionsMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.LookupInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeHoldInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayKeysendMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ReceiveMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Request -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SettleHoldInvoiceMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SignMessageMethod - -class RequestSerializer : StdSerializer(Request::class.java) { - override fun serialize( - value: Request, - gen: JsonGenerator, - provider: SerializerProvider, - ) { - gen.writeStartObject() - if (value.method != null) { - gen.writeStringField("method", value.method) - } - when (value) { - is PayInvoiceMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is PayMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is ReceiveMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is PayKeysendMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is MakeInvoiceMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is LookupInvoiceMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is ListTransactionsMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is MakeHoldInvoiceMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is CancelHoldInvoiceMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is SettleHoldInvoiceMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is SignMessageMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - is CreateConnectionMethod -> { - if (value.params != null) { - gen.writeObjectField("params", value.params) - } - } - - // Parameterless: `method` alone is the whole request. Spelled out rather - // than left to fall through, because Request is sealed and the compiler - // now makes every method state which of the two shapes it is. - is GetBalanceMethod, - is GetBudgetMethod, - is GetInfoMethod, - -> Unit - } - gen.writeEndObject() - } -} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/ResponseDeserializer.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/ResponseDeserializer.kt deleted file mode 100644 index 8c503e7140..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/ResponseDeserializer.kt +++ /dev/null @@ -1,146 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.jackson - -import com.fasterxml.jackson.core.JsonParser -import com.fasterxml.jackson.databind.DeserializationContext -import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.databind.deser.std.StdDeserializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBalanceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBudgetSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetInfoSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ListTransactionsSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.LookupInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeHoldInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcError -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcErrorResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcMethod -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceErrorResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayKeysendSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PaySuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ReceiveSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Response -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SettleHoldInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SignMessageSuccessResponse -import com.vitorpamplona.quartz.utils.asTextOrNull - -class ResponseDeserializer : StdDeserializer(Response::class.java) { - override fun deserialize( - jp: JsonParser, - ctxt: DeserializationContext, - ): Response? { - val jsonObject: JsonNode = jp.codec.readTree(jp) - val resultType = jsonObject.get("result_type")?.asTextOrNull() - val hasError = jsonObject.has("error") && !jsonObject.get("error").isNull - val hasResult = jsonObject.has("result") && !jsonObject.get("result").isNull - - if (hasError) { - return when (resultType) { - NwcMethod.PAY_INVOICE -> { - jp.codec.treeToValue(jsonObject, PayInvoiceErrorResponse::class.java) - } - - else -> { - val error = jp.codec.treeToValue(jsonObject.get("error"), NwcError::class.java) - NwcErrorResponse(resultType ?: "", error) - } - } - } - - if (hasResult || resultType != null) { - return when (resultType) { - NwcMethod.PAY_INVOICE -> { - jp.codec.treeToValue(jsonObject, PayInvoiceSuccessResponse::class.java) - } - - NwcMethod.PAY -> { - jp.codec.treeToValue(jsonObject, PaySuccessResponse::class.java) - } - - NwcMethod.RECEIVE -> { - jp.codec.treeToValue(jsonObject, ReceiveSuccessResponse::class.java) - } - - NwcMethod.PAY_KEYSEND -> { - jp.codec.treeToValue(jsonObject, PayKeysendSuccessResponse::class.java) - } - - NwcMethod.MAKE_INVOICE -> { - jp.codec.treeToValue(jsonObject, MakeInvoiceSuccessResponse::class.java) - } - - NwcMethod.LOOKUP_INVOICE -> { - jp.codec.treeToValue(jsonObject, LookupInvoiceSuccessResponse::class.java) - } - - NwcMethod.LIST_TRANSACTIONS -> { - jp.codec.treeToValue(jsonObject, ListTransactionsSuccessResponse::class.java) - } - - NwcMethod.GET_BALANCE -> { - jp.codec.treeToValue(jsonObject, GetBalanceSuccessResponse::class.java) - } - - NwcMethod.GET_INFO -> { - jp.codec.treeToValue(jsonObject, GetInfoSuccessResponse::class.java) - } - - NwcMethod.GET_BUDGET -> { - jp.codec.treeToValue(jsonObject, GetBudgetSuccessResponse::class.java) - } - - NwcMethod.SIGN_MESSAGE -> { - jp.codec.treeToValue(jsonObject, SignMessageSuccessResponse::class.java) - } - - NwcMethod.CREATE_CONNECTION -> { - jp.codec.treeToValue(jsonObject, CreateConnectionSuccessResponse::class.java) - } - - NwcMethod.MAKE_HOLD_INVOICE -> { - jp.codec.treeToValue(jsonObject, MakeHoldInvoiceSuccessResponse::class.java) - } - - NwcMethod.CANCEL_HOLD_INVOICE -> { - jp.codec.treeToValue(jsonObject, CancelHoldInvoiceSuccessResponse::class.java) - } - - NwcMethod.SETTLE_HOLD_INVOICE -> { - jp.codec.treeToValue(jsonObject, SettleHoldInvoiceSuccessResponse::class.java) - } - - else -> { - // tries to guess for backward compatibility - if (jsonObject.get("result")?.get("preimage") != null) { - return jp.codec.treeToValue(jsonObject, PayInvoiceSuccessResponse::class.java) - } - null - } - } - } - - return null - } -} diff --git a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/ResponseSerializer.kt b/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/ResponseSerializer.kt deleted file mode 100644 index 99bc7613b8..0000000000 --- a/quartz/src/jvmAndroid/kotlin/com/vitorpamplona/quartz/nip47WalletConnect/jackson/ResponseSerializer.kt +++ /dev/null @@ -1,160 +0,0 @@ -/* - * Copyright (c) 2025 Vitor Pamplona - * - * Permission is hereby granted, free of charge, to any person obtaining a copy of - * this software and associated documentation files (the "Software"), to deal in - * the Software without restriction, including without limitation the rights to use, - * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the - * Software, and to permit persons to whom the Software is furnished to do so, - * subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all - * copies or substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN - * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - */ -package com.vitorpamplona.quartz.nip47WalletConnect.jackson - -import com.fasterxml.jackson.core.JsonGenerator -import com.fasterxml.jackson.databind.SerializerProvider -import com.fasterxml.jackson.databind.ser.std.StdSerializer -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CancelHoldInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.CreateConnectionSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBalanceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetBudgetSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.GetInfoSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ListTransactionsSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.LookupInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeHoldInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.MakeInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcErrorResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceErrorResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PayKeysendSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.PaySuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.ReceiveSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.Response -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SettleHoldInvoiceSuccessResponse -import com.vitorpamplona.quartz.nip47WalletConnect.rpc.SignMessageSuccessResponse - -class ResponseSerializer : StdSerializer(Response::class.java) { - override fun serialize( - value: Response, - gen: JsonGenerator, - provider: SerializerProvider, - ) { - gen.writeStartObject() - if (value.resultType.isNotEmpty()) { - gen.writeStringField("result_type", value.resultType) - } - when (value) { - is NwcErrorResponse -> { - if (value.error != null) { - gen.writeObjectField("error", value.error) - } - } - - is PayInvoiceErrorResponse -> { - if (value.error != null) { - gen.writeObjectField("error", value.error) - } - } - - is PayInvoiceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is PaySuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is ReceiveSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is PayKeysendSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is MakeInvoiceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is LookupInvoiceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is ListTransactionsSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is GetBalanceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is GetInfoSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is MakeHoldInvoiceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is CancelHoldInvoiceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is SettleHoldInvoiceSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is GetBudgetSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is SignMessageSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - - is CreateConnectionSuccessResponse -> { - if (value.result != null) { - gen.writeObjectField("result", value.result) - } - } - } - gen.writeEndObject() - } -} diff --git a/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkKotlinSerializationTest.kt b/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkKotlinSerializationTest.kt index 40a25246dc..2cf75ec219 100644 --- a/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkKotlinSerializationTest.kt +++ b/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/experimental/clink/ClinkKotlinSerializationTest.kt @@ -33,16 +33,20 @@ import com.vitorpamplona.quartz.experimental.clink.manage.OfferFields import com.vitorpamplona.quartz.experimental.clink.offers.OfferReceipt import com.vitorpamplona.quartz.experimental.clink.offers.OfferRequest import com.vitorpamplona.quartz.experimental.clink.offers.OfferResponse -import com.vitorpamplona.quartz.nip01Core.jackson.JacksonMapper +import com.vitorpamplona.quartz.nip01Core.core.OptimizedJsonMapper import com.vitorpamplona.quartz.nip01Core.kotlinSerialization.KotlinSerializationMapper import kotlin.test.Test import kotlin.test.assertEquals import kotlin.test.assertTrue /** - * Exercises the kotlinx-serialization path for the CLINK DTOs — the code path that - * native targets (iOS) use via `OptimizedJsonMapper` — and cross-checks it against - * Jackson (the JVM/Android path) so both backends agree on the wire shapes. + * Exercises the kotlinx-serialization path for the CLINK DTOs. + * + * It used to cross-check kotlinx against Jackson, because JVM/Android bound these + * reflectively while native used kotlinx. There is one backend now — [OptimizedJsonMapper] + * routes CLINK at kotlinx on every target — so the cross-check is instead that the + * serializers agree with the PLATFORM ENTRY POINT the app actually calls, which is what + * catches a routing mistake in OptimizedJsonMapper.jvmAndroid. */ class ClinkKotlinSerializationTest { @Test @@ -70,10 +74,10 @@ class ClinkKotlinSerializationTest { assertEquals("A coffee", parsed.description) // Jackson-serialized payload parses with kotlinx and vice versa. - val fromJackson = KotlinSerializationMapper.fromJsonTo(JacksonMapper.toJson(request)) - assertEquals("coffee", fromJackson.offer) - val jacksonParsed = JacksonMapper.fromJsonTo(kotlinJson) - assertEquals(21000L, jacksonParsed.amount_sats) + val fromPlatform = KotlinSerializationMapper.fromJsonTo(OptimizedJsonMapper.toJson(request)) + assertEquals("coffee", fromPlatform.offer) + val platformParsed = OptimizedJsonMapper.fromJsonTo(kotlinJson) + assertEquals(21000L, platformParsed.amount_sats) } @Test @@ -160,8 +164,8 @@ class ClinkKotlinSerializationTest { assertEquals(listOf("email", "name"), parsed.offer?.fields?.payer_data) // Jackson reads the kotlinx output identically. - val jacksonParsed = JacksonMapper.fromJsonTo(json) - assertEquals("Coffee", jacksonParsed.offer?.fields?.label) + val platformParsed = OptimizedJsonMapper.fromJsonTo(json) + assertEquals("Coffee", platformParsed.offer?.fields?.label) } @Test @@ -194,7 +198,7 @@ class ClinkKotlinSerializationTest { assertEquals("o1", parsed.details?.first()?.id) assertEquals(1500L, parsed.details?.first()?.price_sats) - val jacksonParsed = JacksonMapper.fromJsonTo(json) - assertEquals("noffer1...", jacksonParsed.details?.first()?.noffer) + val platformParsed = OptimizedJsonMapper.fromJsonTo(json) + assertEquals("noffer1...", platformParsed.details?.first()?.noffer) } } diff --git a/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/InliningTagArrayPrettyPrinterTest.kt b/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/InliningTagArrayPrettyPrinterTest.kt index be489b5aae..df327e132d 100644 --- a/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/InliningTagArrayPrettyPrinterTest.kt +++ b/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/InliningTagArrayPrettyPrinterTest.kt @@ -20,13 +20,13 @@ */ package com.vitorpamplona.quartz.nip01Core.jackson -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import kotlin.test.Test import kotlin.test.assertEquals class InliningTagArrayPrettyPrinterTest { val mapper = - jacksonObjectMapper().apply { + ObjectMapper().apply { setDefaultPrettyPrinter(InliningTagArrayPrettyPrinter()) } diff --git a/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonSerializerGuardTest.kt b/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonSerializerGuardTest.kt new file mode 100644 index 0000000000..8c01e0e83e --- /dev/null +++ b/quartz/src/jvmAndroidTest/kotlin/com/vitorpamplona/quartz/nip01Core/jackson/JacksonSerializerGuardTest.kt @@ -0,0 +1,67 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.quartz.nip01Core.jackson + +import com.vitorpamplona.quartz.nip01Core.core.OptimizedSerializable +import com.vitorpamplona.quartz.nip01Core.relay.commands.toRelay.CloseCmd +import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter +import com.vitorpamplona.quartz.nip46RemoteSigner.BunkerRequest +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertTrue + +/** + * [JacksonMapper.toJson] refuses a type it has no registered serializer for. + * + * Bean introspection would otherwise name every field after its getter, and R8 + * renames getters — so the JSON is correct in debug and one-letter garbage in a + * release build. The failure has to happen here, on the first call, or it happens + * on a user's device. + */ +class JacksonSerializerGuardTest { + private class NotRegistered( + val someField: String = "value", + ) : OptimizedSerializable + + @Test + fun unregisteredTypeIsRefused() { + val e = assertFailsWith { JacksonMapper.toJson(NotRegistered()) } + assertTrue(e.message!!.contains("No Jackson serializer is registered"), e.message!!) + } + + /** + * The guard is an `is` chain because Jackson's SimpleSerializers walks the + * hierarchy: CLOSE has no serializer of its own, it rides Command's. An + * exact-class check would reject it — and with it every relay command the + * client sends. + */ + @Test + fun subclassesOfARegisteredRootStillSerialize() { + assertEquals("""["CLOSE","sub-1"]""", JacksonMapper.toJson(CloseCmd("sub-1"))) + } + + @Test + fun registeredRootsStillSerialize() { + assertEquals("""{"kinds":[1]}""", JacksonMapper.toJson(Filter(kinds = listOf(1)))) + assertTrue(JacksonMapper.toJson(BunkerRequest("id-1", "connect", arrayOf("a"))).contains("\"connect\"")) + } +} diff --git a/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriter.kt b/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriter.kt index 585d17c3e6..eba66bd774 100644 --- a/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriter.kt +++ b/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriter.kt @@ -21,7 +21,6 @@ package com.vitorpamplona.quic.interop import com.fasterxml.jackson.databind.ObjectMapper -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper import com.vitorpamplona.quic.connection.EncryptionLevel import com.vitorpamplona.quic.observability.QlogObserver import java.io.BufferedWriter @@ -295,7 +294,7 @@ class QlogWriter( } companion object { - private val DEFAULT_MAPPER: ObjectMapper = jacksonObjectMapper() + private val DEFAULT_MAPPER: ObjectMapper = ObjectMapper() private fun packetTypeFor(level: EncryptionLevel): String = when (level) { diff --git a/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriterTest.kt b/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriterTest.kt index 72f5d13845..f6e20de0f1 100644 --- a/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriterTest.kt +++ b/quic/src/jvmTest/kotlin/com/vitorpamplona/quic/interop/QlogWriterTest.kt @@ -20,7 +20,7 @@ */ package com.vitorpamplona.quic.interop -import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper +import com.fasterxml.jackson.databind.ObjectMapper import com.vitorpamplona.quic.connection.EncryptionLevel import org.junit.Test import java.io.File @@ -73,7 +73,7 @@ class QlogWriterTest { val lines = tmp.readLines().filter { it.isNotBlank() } assertTrue(lines.size >= 12, "expected >= 12 lines (header + at least 11 events) but got ${lines.size}") - val mapper = jacksonObjectMapper() + val mapper = ObjectMapper() // Line 1: qlog header. val header = mapper.readTree(lines[0]) @@ -128,7 +128,7 @@ class QlogWriterTest { w.onAlpnNegotiated("h3") } val lines = tmp.readLines().filter { it.isNotBlank() } - val mapper = jacksonObjectMapper() + val mapper = ObjectMapper() val event = mapper.readTree(lines[1]) assertEquals(50L, event.get("time").asLong(), "time must be relative to constructor (1050 - 1000)") } @@ -152,7 +152,7 @@ class QlogWriterTest { QlogWriter(tmp, odcidHex = "00").use { w -> w.onPacketSent(EncryptionLevel.INITIAL, 0, 1200, emptyList()) } - val mapper = jacksonObjectMapper() + val mapper = ObjectMapper() val lines = tmp.readLines().filter { it.isNotBlank() } val frames = mapper.readTree(lines[1]).get("data").get("frames") assertTrue(frames.isArray, "frames must be an array even when empty") diff --git a/scripts/retrace.sh b/scripts/retrace.sh new file mode 100755 index 0000000000..329b992305 --- /dev/null +++ b/scripts/retrace.sh @@ -0,0 +1,259 @@ +#!/usr/bin/env bash +# +# Turn an obfuscated Amethyst crash report back into real class, method, file +# and line names. +# +# scripts/retrace.sh report.txt # figures out the release by itself +# pbpaste | scripts/retrace.sh # ... or straight off the clipboard +# +# A report produced by ReportAssembler names its own build on line 1: +# +# java.lang.IllegalStateException: 1.16.0-PLAY +# +# so that is all this needs to fetch the right mapping from the matching GitHub +# Release (amethyst-googleplay-mapping-v1.16.0.txt.gz) and cache it. +# +# If you only have a bare stack trace with no such header, name the build: +# +# scripts/retrace.sh --release v1.16.0 --flavor play trace.txt +# +# ... or point at a mapping yourself, e.g. for a build you made locally: +# +# scripts/retrace.sh amethyst/build/outputs/mapping/playRelease/mapping.txt trace.txt +# +# Mappings are never guessed at. Every frame of an obfuscated trace carries +# `r8-map-id-`, which is the `pg_map_id` of the one mapping that produced +# that build, so the mapping is checked against the report before any output is +# printed — a wrong mapping produces plausible, wrong answers, which is worse +# than no answer. --force overrides. +# +# The R8 that does the work is fetched at the version recorded in the mapping's +# own header, so there is no tooling to pin and this keeps working across AGP +# bumps. +# +# Everything is cached under ~/.cache/amethyst-retrace/, and that cache is not +# small: a mapping is ~29 MB compressed and ~500 MB expanded, per release you +# retrace. `rm -rf ~/.cache/amethyst-retrace` whenever you want it back; the +# next run re-downloads. +set -euo pipefail + +REPO="${AMETHYST_REPO:-vitorpamplona/amethyst}" +CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/amethyst-retrace" + +MAPPING="" +RELEASE="" +FLAVOR="" +REPORT="" +FORCE=0 + +die() { echo "error: $*" >&2; exit 2; } + +usage() { + sed -n '3,30p' "$0" | sed 's/^# \{0,1\}//' + exit "${1:-2}" +} + +# The first 4 KiB of a mapping, which is where R8 puts its whole header. Read +# once, into a variable: `sed ... "$MAPPING" | head -1` would stream all ~500 MB +# looking for a line that is always in the first hundred bytes, and would also +# leave the pipeline's status at head's SIGPIPE. +mapping_header() { + case "$1" in + *.gz) gunzip -c "$1" 2>/dev/null | head -c 4096 || true ;; + *) head -c 4096 "$1" 2>/dev/null || true ;; + esac +} + +# A mapping file, or the report? Decide by content, not by extension, so both +# documented argument orders keep working. +# +# Deliberately NOT `... | grep -q`: `grep -q` exits at the first match, the +# producer takes SIGPIPE, and `set -o pipefail` then reports the whole pipeline +# as failed — so a real mapping.txt.gz was classified as a report and the actual +# report came back as "unexpected argument". +looks_like_mapping() { + [ -f "$1" ] || return 1 + case "$1" in + *.prt) return 0 ;; + esac + case "$(mapping_header "$1")" in + "# compiler"*) return 0 ;; + *) return 1 ;; + esac +} + +while [ $# -gt 0 ]; do + case "$1" in + --release) RELEASE="${2:-}"; shift 2 ;; + --flavor) FLAVOR="${2:-}"; shift 2 ;; + --mapping) MAPPING="${2:-}"; shift 2 ;; + --force) FORCE=1; shift ;; + -h|--help) usage 0 ;; + -*) die "unknown option $1 (try --help)" ;; + *) + if [ -z "$MAPPING" ] && [ -z "$RELEASE" ] && looks_like_mapping "$1"; then + MAPPING="$1" + elif [ -z "$REPORT" ]; then + REPORT="$1" + else + die "unexpected argument $1" + fi + shift ;; + esac +done + +mkdir -p "$CACHE" +TMP="$(mktemp -d)" +trap 'rm -rf "$TMP"' EXIT + +# ---- materialise the report ------------------------------------------------ +# Always to a file: we have to read it twice (header, map id) before retracing, +# and that is not possible on a pipe. +REPORT_FILE="$TMP/report.txt" +if [ -n "$REPORT" ]; then + [ -f "$REPORT" ] || die "no such file: $REPORT" + cp "$REPORT" "$REPORT_FILE" +else + [ -t 0 ] && echo "reading the report from stdin (ctrl-D when done) ..." >&2 + cat > "$REPORT_FILE" +fi +[ -s "$REPORT_FILE" ] || die "the report is empty" + +# Retrace only rewrites frames it recognises, and it recognises them by the +# leading `at`. Reports from Amethyst builds before the ReportAssembler fix +# wrote bare " com.foo.Bar.baz(File.kt:12)" lines, which would otherwise be +# passed through still obfuscated and look like a mapping problem. Put the `at` +# back on any frame-shaped line that is missing it. Anything that is not +# (:) is left exactly as it is. +sed -E 's/^([[:space:]]+)([A-Za-z_$][A-Za-z0-9_$]*(\.[A-Za-z0-9_$<>]+)+\([^()]*:[0-9]+\))[[:space:]]*$/\1at \2/' \ + "$REPORT_FILE" > "$TMP/normalised.txt" +mv "$TMP/normalised.txt" "$REPORT_FILE" + +# ---- work out which mapping -------------------------------------------------- +if [ -z "$MAPPING" ]; then + if [ -z "$RELEASE" ]; then + # --auto: parse ": -" off line 1. + header="$(head -1 "$REPORT_FILE" | tr -d '\r')" + case "$header" in + *": "*) ;; + *) die "line 1 is not an Amethyst crash-report header, so the build is unknown. + Pass --release [--flavor play|fdroid], or a mapping file." ;; + esac + vf="${header#*: }" + vf="$(echo "$vf" | tr -d '[:space:]')" + [ -n "$vf" ] || die "line 1 has no '-' after the exception name." + parsed_flavor="${vf##*-}" + version="${vf%-*}" + [ -n "$parsed_flavor" ] && [ -n "$version" ] && [ "$version" != "$vf" ] \ + || die "cannot read '-' out of line 1: $header" + [ -n "$FLAVOR" ] || FLAVOR="$parsed_flavor" + case "$version" in + [0-9]*.[0-9]*.[0-9]*) ;; + *) die "line 1 reports version '$version', which is not a release version." ;; + esac + # generateVersionName() appends the git branch on non-main builds, so a + # dev build reads e.g. 1.16.0-my-branch-PLAY. There is no release for it. + case "$version" in + *[!0-9.]*) die "version '$version' carries a branch suffix, so this is a local/CI + build, not a release — its mapping was never published. Retrace against + that build's own amethyst/build/outputs/mapping//mapping.txt." ;; + esac + RELEASE="v$version" + fi + + case "$RELEASE" in v*) ;; *) RELEASE="v$RELEASE" ;; esac + + case "$(echo "${FLAVOR:-play}" | tr '[:upper:]' '[:lower:]')" in + play|googleplay) channel=googleplay ;; + fdroid) channel=fdroid ;; + *) die "unknown flavor '$FLAVOR' (expected play or fdroid)" ;; + esac + + asset="amethyst-${channel}-mapping-${RELEASE}.txt.gz" + MAPPING="$CACHE/$asset" + if [ ! -s "$MAPPING" ]; then + url="https://github.com/${REPO}/releases/download/${RELEASE}/${asset}" + echo "fetching $asset ..." >&2 + curl -fsSL -o "$MAPPING.tmp" "$url" || { + rm -f "$MAPPING.tmp" + die "could not download $url + Either that release predates mapping publication (builds up to v1.16.0 + were not obfuscated — read those traces as-is), or the flavor is wrong. + Assets: gh release view $RELEASE --json assets --jq '.assets[].name'" + } + mv "$MAPPING.tmp" "$MAPPING" + fi +fi + +[ -f "$MAPPING" ] || die "no such mapping: $MAPPING" +# Keep the name the user actually gave us for error messages — once a .gz is +# expanded, $MAPPING points at a cache path they never typed. +MAPPING_LABEL="$(basename "$MAPPING")" + +# ---- decompress if needed -------------------------------------------------- +PARTITION=0 +case "$MAPPING" in + *.prt) PARTITION=1 ;; + *.gz) + # Keyed on the full source path, not just the basename: two different + # mappings are both called mapping.txt.gz if you gzip a playRelease and + # an fdroidRelease by hand. (The map-id check below would catch the mixup + # anyway, but "wrong release" is a much clearer error than a stale cache.) + key="$(printf '%s' "$(cd "$(dirname "$MAPPING")" && pwd)/$(basename "$MAPPING")" | cksum | cut -d' ' -f1)" + plain="$CACHE/$(basename "${MAPPING%.gz}").$key" + if [ ! -s "$plain" ] || [ "$MAPPING" -nt "$plain" ]; then + echo "decompressing $(basename "$MAPPING") ..." >&2 + gunzip -c "$MAPPING" > "$plain" + fi + MAPPING="$plain" + ;; +esac + +# ---- make sure this mapping really built this report ------------------------ +if [ "$PARTITION" -eq 0 ]; then + HEADER="$(mapping_header "$MAPPING")" + trace_id="$(grep -om1 'r8-map-id-[0-9a-f]\{16,\}' "$REPORT_FILE" | sed 's/^r8-map-id-//' || true)" + map_id="$(printf '%s\n' "$HEADER" | sed -n 's/^# pg_map_id: *//p' | head -1 || true)" + if [ -z "$trace_id" ]; then + echo "note: no r8-map-id in the report (an un-obfuscated build, or a trimmed" >&2 + echo " trace) — cannot confirm the mapping matches." >&2 + elif [ "$trace_id" != "$map_id" ]; then + msg="this mapping did not build this report. + report: $trace_id + mapping: $map_id ($MAPPING_LABEL) + Retracing anyway yields wrong names that look right. Check the release + tag and the flavor (play vs fdroid are separate R8 runs)." + [ "$FORCE" -eq 1 ] && echo "warning: $msg" >&2 || die "$msg" + fi +fi + +# ---- fetch the R8 that wrote it --------------------------------------------- +if [ "$PARTITION" -eq 1 ]; then + R8_VERSION="${R8_VERSION:-$(ls "$CACHE"/r8-*.jar 2>/dev/null | sed 's/.*r8-\(.*\)\.jar/\1/' | sort -V | tail -1 || true)}" + [ -n "$R8_VERSION" ] || die "a .prt partition map does not expose its R8 version. + Set R8_VERSION= (the 'compiler_version' of the matching + mapping.txt), or retrace against the .txt.gz instead." +else + R8_VERSION="$(printf '%s\n' "${HEADER:-$(mapping_header "$MAPPING")}" | sed -n 's/^# compiler_version: *//p' | head -1 || true)" + [ -n "$R8_VERSION" ] || die "no '# compiler_version:' header in $MAPPING — not an R8 mapping?" +fi + +R8_JAR="$CACHE/r8-${R8_VERSION}.jar" +if [ ! -s "$R8_JAR" ]; then + echo "fetching R8 ${R8_VERSION} ..." >&2 + curl -fsSL -o "$R8_JAR.tmp" \ + "https://maven.google.com/com/android/tools/r8/${R8_VERSION}/r8-${R8_VERSION}.jar" \ + || { rm -f "$R8_JAR.tmp"; die "could not download R8 ${R8_VERSION} from Google's Maven"; } + mv "$R8_JAR.tmp" "$R8_JAR" +fi + +# ---- retrace --------------------------------------------------------------- +# Not `exec`: exec replaces this shell, so the EXIT trap never runs and $TMP is +# left behind on every single invocation. +if [ "$PARTITION" -eq 1 ]; then + java -cp "$R8_JAR" com.android.tools.r8.retrace.Retrace \ + --partition-map "$MAPPING" "$REPORT_FILE" +else + java -cp "$R8_JAR" com.android.tools.r8.retrace.Retrace \ + "$MAPPING" "$REPORT_FILE" +fi diff --git a/tools/r8-verify/reflection-contract.txt b/tools/r8-verify/reflection-contract.txt new file mode 100644 index 0000000000..cad44c3656 --- /dev/null +++ b/tools/r8-verify/reflection-contract.txt @@ -0,0 +1,159 @@ +# What must survive R8 with its ORIGINAL name, and why. +# +# R8 cannot see reflection. Every line here is a place where something outside +# the DEX — a .so, a manifest attribute, a JSON file on disk, WorkManager's +# database, a DataStore value written by an older install — looks a name up at +# runtime. Rename or remove it and the app compiles, ships, and fails only on a +# user's device. +# +# `verify_reflection_contract.py` checks each line against a release build's +# mapping.txt, so a keep rule that silently stops matching (a moved class, a +# renamed package, a deleted rule) fails the release instead of the user. +# +# Adding reflection? Add the keep rule in amethyst/proguard-rules.pro AND a line +# here. A rule with no line here is unverified; a line here with no rule fails. +# +# CAN THE REFLECTION ITSELF BE REMOVED? +# Reviewed entry by entry. A keep rule you do not need is better than one you +# verify, so the standing answer per mechanism: +# +# JNI class+method names IRREDUCIBLE. `Java__` is +# compiled into libarti_android.so. Costs us +# nothing anyway — AGP's default +# `native ` rule covers it. +# Rust -> Kotlin callback Removable only by inverting the flow (Kotlin +# polls a native queue instead of Rust pushing +# by GetMethodID). One interface, one method — +# not obviously worth the threading change. +# Cast OptionsProvider IRREDUCIBLE. Play Services reads the class +# name out of a manifest value and +# Class.forName()s it. Google's API, not ours. +# WorkManager workers ALREADY GONE. androidx.work ships the keep +# itself; our duplicate rule was deleted and +# these entries now verify the library's. +# Jackson NWC + CLINK DONE — both package keeps deleted. The types +# route at the hand-written kotlinx serializers +# on every target now, and the Jackson +# (de)serializers for them are gone. +# Class name as a value REMOVED. requireInProcessSigner() compared +# `signer::class.qualifiedName` against the FQN +# of NostrSignerExternal. R8 renames that class, +# so the branch never ran in a release build. It +# is a plain `is` check now — nothing to keep. +# Platform-class reflection IRREDUCIBLE AND FREE. quic's +# JdkCertificateValidator probes the JDK/Android +# trust manager for the 3-arg +# checkServerTrusted(chain, authType, host). +# That class ships in the platform, not in our +# DEX, so R8 never renames it — no rule needed. +# libscrypt / NetCipher / GONE. Their keep rules matched zero classes: +# LazySodium / JNA libsodium is a pure-Kotlin implementation now +# and Tor runs through arti. Rules deleted. +# Jackson on-disk stores REMOVABLE. Plain data classes; @Serializable +# + kotlinx gives compile-time literal names. +# App-private files, so no interop risk. +# Enum constants REMOVABLE WITH NO MIGRATION. Give each +# persisted enum an explicit `val code: String` +# and store that instead of `.name`. Set the +# codes equal to today's constant names and +# every existing DataStore value keeps working, +# while the literal is untouchable by R8. That +# deletes the one blanket rule left +# (`-keepclassmembers enum *`), which today +# blocks enum unboxing across all 707 enums. +# +# Format — one per line, `#` comments to end of line: +# [@play|@fdroid] optional leading scope: check this line only on +# that flavor. Play-only code is absent from the +# F-Droid APK, and "absent" reads as "R8 deleted +# it" — an unscoped line would fail that release. +# class class must keep its exact name +# method ... those methods must keep their names +# fields class name AND every field name preserved +# enum ... those constants keep their names (class may be renamed) + +# --- JNI: the symbol name Java__ lives in libarti_android.so --- +class com.vitorpamplona.amethyst.ui.tor.ArtiNative +# Rust calls back by name: GetMethodID("onLogLine") in tools/arti-build/src/lib.rs +method com.vitorpamplona.amethyst.ui.tor.ArtiLogCallback onLogLine + +# zxing-cpp's JNI half, vendored into amethyst/src/main/java/zxingcpp. The .so +# exports Java_zxingcpp_BarcodeReader_readYBuffer, so the class name and both +# native method names are the lookup. AGP's default +# `-keepclasseswithmembernames class * { native ; }` should cover this on +# its own and the explicit `-keep class zxingcpp.**` is belt-and-braces; these +# lines are what proves at least one of them still matches. A rename fails +# silently — no build error, no warning, the QR scanner just never starts. +class zxingcpp.BarcodeReader +method zxingcpp.BarcodeReader readYBuffer readBitmap + +# --- Named from outside the DEX ---------------------------------------------- +# No keep rule of ours backs the three workers below: androidx.work's own +# consumer rules do. They stay listed so that if the library ever drops them, +# the release fails here instead of on a user's phone. +# in the play manifest; the Cast framework +# Class.forName()s it. AGP generates keeps for component android:name, not for +# meta-data values, so nothing but an explicit rule protects this one. +@play class com.vitorpamplona.amethyst.service.cast.chromecast.AmethystCastOptionsProvider +# WorkManager stores the class name in its own DB and instantiates it by name on +# a later process start — including after an update that reshuffled the mapping. +class com.vitorpamplona.amethyst.service.scheduledposts.ScheduledPostWorker +class com.vitorpamplona.amethyst.service.notifications.NotificationCatchUpWorker +class com.vitorpamplona.amethyst.service.calendar.CalendarReminderWorker + +# The AppFunctions bridge is kept whole by a package rule, so this asserts the +# rule still matches something. androidx.appfunctions reflects over the generated +# inventories and @AppFunctionSerializable types; play-only source set. +@play class com.vitorpamplona.amethyst.appfunctions.AmethystAppFunctions + +# --- Jackson reflective data binding: field names ARE the wire format --------- +# NIP-47 and CLINK used to be listed here. They are not data-bound reflectively +# any more — OptimizedJsonMapper routes them at kotlinx serializers that write +# every field name as a string literal — so there is no rule to verify. + + +# --- Enums used as navigation-route arguments: the CLASS name is the lookup --- +# androidx.navigation resolves an enum route argument by fully-qualified class +# name (NavTypeConverter calls Class.forName on the serial name). Renaming the +# class throws while the nav graph is being built, so the app cannot get past +# login at all -- release-only, and total. +# +# These need the CLASS pinned, which the `-keepclassmembers enum *` rule below +# deliberately does not do: it keeps constant names and lets the class be +# renamed. Add a line here whenever an enum becomes a route argument. +class com.vitorpamplona.amethyst.ui.navigation.routes.DiscoverTab +class com.vitorpamplona.amethyst.ui.screen.loggedIn.bookmarkgroups.BookmarkType + + +# --- Enum constants persisted as strings ------------------------------------- +# The preference stores write `enum.name` into DataStore and read it back with +# valueOf(). A renamed constant resets that setting for every existing user on +# the first launch after the update — silently, and only in a release build. +# The enum CLASS is still renamed; only the constant names are pinned. +enum com.vitorpamplona.amethyst.commons.tor.TorType INTERNAL + +# Not a preference: these constant names ARE the NIP-47 wire strings. The +# response parser does NwcErrorCode.valueOf(json["code"]) on a string another +# wallet wrote, so a rename turns every typed error into a null code and the +# UI loses the reason a payment failed. Falls back quietly (try/catch), which +# is exactly why it would never be noticed. +enum com.vitorpamplona.quartz.nip47WalletConnect.rpc.NwcErrorCode RATE_LIMITED NOT_IMPLEMENTED INSUFFICIENT_BALANCE PAYMENT_FAILED QUOTA_EXCEEDED RESTRICTED UNAUTHORIZED INTERNAL UNSUPPORTED_ENCRYPTION BAD_REQUEST NOT_FOUND EXPIRED UNSUPPORTED_PAYMENT_INSTRUCTION UNSUPPORTED_NETWORK OTHER + +enum com.vitorpamplona.amethyst.model.ThemeType SYSTEM LIGHT DARK +enum com.vitorpamplona.amethyst.model.BooleanType ALWAYS NEVER +enum com.vitorpamplona.amethyst.model.ConnectivityType ALWAYS NEVER +enum com.vitorpamplona.amethyst.model.FeatureSetType SIMPLIFIED +enum com.vitorpamplona.amethyst.model.FontFamilyType SYSTEM +enum com.vitorpamplona.amethyst.model.FontSizeType NORMAL +enum com.vitorpamplona.amethyst.model.AccentColorType PURPLE +enum com.vitorpamplona.amethyst.model.ProfileGalleryType CLASSIC +enum com.vitorpamplona.quartz.nip05DnsIdentifiers.namecoin.NamecoinBackend +enum com.vitorpamplona.quartz.nipACWebRtcCalls.tags.CallType +enum com.vitorpamplona.amethyst.ui.screen.loggedIn.chats.publicChannels.concord.ChannelExpand +# Still listed after ScheduledPost moved to kotlinx: this one is also the on-disk +# format of the scheduled-post file, and a plain @Serializable enum is not obviously +# immune — kotlinx builds its descriptor from the entries, and it has not been proven +# here that those names are literals rather than Enum.name read at runtime. Keeping +# the constants costs nothing (the blanket enum rule already provides them) and the +# failure mode — every queued post's status unreadable — is not worth guessing at. +enum com.vitorpamplona.amethyst.commons.scheduledposts.ScheduledPostStatus PENDING PUBLISHING SENT FAILED CANCELLED diff --git a/tools/r8-verify/verify_reflection_contract.py b/tools/r8-verify/verify_reflection_contract.py new file mode 100755 index 0000000000..80b3564b11 --- /dev/null +++ b/tools/r8-verify/verify_reflection_contract.py @@ -0,0 +1,281 @@ +#!/usr/bin/env python3 +"""Check a release build's R8 output against tools/r8-verify/reflection-contract.txt. + + python3 tools/r8-verify/verify_reflection_contract.py [contract] + + is amethyst/build/outputs/mapping// — BOTH mapping.txt +and usage.txt are read, because neither is sufficient alone: + + * mapping.txt records only what CHANGED. A class kept intact by + `-keep ... { *; }` appears with an empty body, and an unrenamed member has + no line at all. Absence there means "preserved", not "missing". + * usage.txt is the other half: what R8 deleted. A constant that was shrunk + away leaves no trace in mapping.txt, so removals are only visible here. + +Preserved therefore means: present in mapping.txt under its own name, and not +listed in usage.txt. + +R8 cannot see reflection, so a keep rule that stops matching — a class moved to +another package, a rule deleted, a DTO renamed — fails silently: the build is +green, the APK ships, and the break surfaces on a user's device as a +ClassNotFoundException, an unreadable settings file, or JSON with one-letter +property names. The contract lists what must survive; this asserts it against +what R8 actually emitted. + +Exits 0 when every entry holds, 1 otherwise, naming each failure and the keep +rule that should have covered it. +""" +import re +import sys + +CLASS_LINE = re.compile(r"^(?P[^\s]+) -> (?P[^\s:]+):") +# " -> " (field) / " [1:2:] (args) -> " (method) +MEMBER_LINE = re.compile( + r"^\s+(?:\d+:\d+:)?[^\s]+\s+(?P[^\s(]+)(?P\([^)]*\))?\s*->\s*(?P[^\s]+)\s*$" +) + + +FLAVORS = ("play", "fdroid") + + +def parse_contract(path): + """Read the contract. A line may open with @ to scope it. + + Play-only code (the Cast provider, the AppFunctions bridge) is not compiled + into the F-Droid APK at all, so an unscoped entry for it would read as + "R8 deleted this" on that variant and fail a release the moment CI checked + both. `@play class ...` limits the check to the variant that has the class. + """ + entries = [] + with open(path, encoding="utf-8") as fh: + for lineno, raw in enumerate(fh, 1): + line = raw.split("#", 1)[0].strip() + if not line: + continue + parts = line.split() + scope = None + if parts[0].startswith("@"): + scope = parts[0][1:] + if scope not in FLAVORS: + sys.exit(f"{path}:{lineno}: unknown flavor {scope!r}; " + f"expected one of {', '.join(FLAVORS)}") + parts = parts[1:] + if len(parts) < 2: + sys.exit(f"{path}:{lineno}: expected ' [names...]'") + kind, fqn, rest = parts[0], parts[1], parts[2:] + if kind not in ("class", "method", "fields", "enum"): + sys.exit(f"{path}:{lineno}: unknown check kind {kind!r}") + entries.append((kind, fqn, rest, lineno, scope)) + return entries + + +def flavor_of(mapping_dir): + """playRelease -> play, fdroidRelease -> fdroid, anything else -> None. + + None means "check everything": an unrecognised directory must not silently + skip entries. + """ + variant = mapping_dir.rsplit("/", 1)[-1] + for flavor in FLAVORS: + if variant.startswith(flavor): + return flavor + return None + + +def consistency_check(mapping_path, usage_path): + """Are these two files from the SAME R8 run? + + mapping.txt is written later than the rest of the mapping directory (R8 + emits it during packaging, not at minify time), so a stale one survives a + rebuild that only got as far as minifying — leaving a directory whose + usage.txt describes one build and whose mapping.txt describes another. The + checks below would then read a coherent-looking mix and report nonsense. + + The invariant: a class R8 deleted cannot also be a class R8 named. Any class + that usage.txt lists as removed while mapping.txt shows it live under a real + name means the two files disagree about what was built. + """ + removed = set() + with open(usage_path, encoding="utf-8", errors="replace") as fh: + for line in fh: + if line[:1].isspace() or not line.strip(): + continue + name = line.strip() + if not name.endswith(":"): + removed.add(name) + clashes = [] + with open(mapping_path, encoding="utf-8", errors="replace") as fh: + for line in fh: + if line.startswith("#") or line[:1].isspace(): + continue + m = CLASS_LINE.match(line) + if ( + m + and not m.group("obf").startswith("R8$$REMOVED$$") + and m.group("orig") in removed + ): + clashes.append(m.group("orig")) + if len(clashes) > 20: + break + return clashes + + +def collect_removed(usage_path, wanted): + """usage.txt: `com.foo.Bar` alone = class deleted; `com.foo.Bar:` + indented + signatures = those members deleted.""" + gone_classes, gone_members, current = set(), {}, None + with open(usage_path, encoding="utf-8", errors="replace") as fh: + for line in fh: + if not line.strip(): + continue + if not line[:1].isspace(): + name = line.strip() + current = None + if name.endswith(":"): + name = name[:-1] + if name in wanted: + current = name + gone_members.setdefault(name, []) + elif name in wanted: + gone_classes.add(name) + continue + if current is not None: + gone_members[current].append(line.strip()) + return gone_classes, gone_members + + +def collect(mapping_path, wanted): + """Stream the mapping (it is ~500 MB) and keep only the blocks we asked for.""" + blocks, current = {}, None + with open(mapping_path, encoding="utf-8", errors="replace") as fh: + for line in fh: + if line.startswith("#"): + continue + if not line[:1].isspace(): + m = CLASS_LINE.match(line) + current = None + if m and m.group("orig") in wanted: + current = m.group("orig") + blocks[current] = {"obf": m.group("obf"), "fields": {}, "methods": {}} + continue + if current is None: + continue + m = MEMBER_LINE.match(line) + if not m: + continue + bucket = "methods" if m.group("args") else "fields" + blocks[current][bucket].setdefault(m.group("name"), m.group("new")) + return blocks + + +def main(): + if not 2 <= len(sys.argv) <= 3: + sys.exit(__doc__) + mapping_dir = sys.argv[1].rstrip("/") + mapping_path = mapping_dir + "/mapping.txt" + usage_path = mapping_dir + "/usage.txt" + for required in (mapping_path, usage_path): + try: + open(required).close() + except OSError as exc: + sys.exit(f"cannot read {required}: {exc}\n" + "Point this at amethyst/build/outputs/mapping// from a " + "minified build.") + contract_path = ( + sys.argv[2] + if len(sys.argv) == 3 + else __file__.rsplit("/", 1)[0] + "/reflection-contract.txt" + ) + + clashes = consistency_check(mapping_path, usage_path) + if clashes: + print( + f"{mapping_dir} is not one build: {len(clashes)}+ classes are listed as\n" + f"removed in usage.txt yet named in mapping.txt, e.g.\n " + + "\n ".join(clashes[:3]) + + "\n\nmapping.txt is written during packaging, so a minify-only rebuild leaves\n" + "the previous one behind. Re-run a full assemble/bundle for this variant\n" + "and check again — the results from this directory would be meaningless.", + file=sys.stderr, + ) + return 1 + + entries = parse_contract(contract_path) + flavor = flavor_of(mapping_dir) + in_scope = [e for e in entries if e[4] is None or e[4] == flavor] + skipped = len(entries) - len(in_scope) + entries = in_scope + wanted = {fqn for _, fqn, _, _, _ in entries} + blocks = collect(mapping_path, wanted) + gone_classes, gone_members = collect_removed(usage_path, wanted) + + def removed_member(fqn, name): + """usage.txt prints whole signatures; match the member name inside one.""" + for sig in gone_members.get(fqn, ()): + head = sig.split("(", 1)[0] + if head.split()[-1:] == [name] or head.endswith(" " + name): + return True + return False + + failures = [] + + def fail(lineno, fqn, msg): + failures.append(f" {contract_path}:{lineno} {fqn}\n {msg}") + + for kind, fqn, rest, lineno, _scope in entries: + if fqn in gone_classes: + fail(lineno, fqn, "deleted by R8 (listed in usage.txt) — nothing keeps it.") + continue + blk = blocks.get(fqn) + if blk is None: + fail(lineno, fqn, "absent from the mapping entirely — R8 removed it, or it " + "was renamed/moved in source and the contract is stale.") + continue + if blk["obf"].startswith("R8$$REMOVED$$"): + fail(lineno, fqn, "shrunk away by R8; nothing keeps it any more.") + continue + + if kind in ("class", "method", "fields") and blk["obf"] != fqn: + fail(lineno, fqn, f"renamed to {blk['obf']} — it is looked up by name at runtime.") + continue + + # From here on, a member with NO mapping line kept its name. Only an + # explicit rename, or an entry in usage.txt, is a failure. + if kind == "method": + for name in rest: + if removed_member(fqn, name): + fail(lineno, fqn, f"method {name}() was deleted by R8.") + elif blk["methods"].get(name, name) != name: + fail(lineno, fqn, f"method {name}() renamed to {blk['methods'][name]}().") + elif kind == "fields": + renamed = sorted(n for n, new in blk["fields"].items() if n != new) + if renamed: + shown = ", ".join(renamed[:6]) + ("…" if len(renamed) > 6 else "") + fail(lineno, fqn, f"{len(renamed)} field(s) renamed ({shown}) — these names " + "are the JSON/wire format.") + dropped = [s for s in gone_members.get(fqn, ()) if "(" not in s] + if dropped: + fail(lineno, fqn, f"{len(dropped)} field(s) deleted by R8: {', '.join(dropped[:4])}") + elif kind == "enum": + for const in rest: + if removed_member(fqn, const): + fail(lineno, fqn, f"constant {const} was deleted; valueOf(\"{const}\") " + "throws on a value already on disk.") + elif blk["fields"].get(const, const) != const: + fail(lineno, fqn, f"constant {const} renamed to {blk['fields'][const]}; " + "every stored value of it becomes unreadable.") + + checked = len(entries) + if failures: + print(f"R8 reflection contract: {len(failures)} of {checked} checks FAILED\n", file=sys.stderr) + print("\n".join(failures), file=sys.stderr) + print("\nFix the keep rule in amethyst/proguard-rules.pro (or quartz/consumer-rules.pro)," + "\nor update the contract if the code genuinely moved.", file=sys.stderr) + return 1 + note = f" ({skipped} skipped: not in the {flavor} flavor)" if skipped else "" + print(f"R8 reflection contract: all {checked} checks passed against {mapping_path}{note}") + return 0 + + +if __name__ == "__main__": + sys.exit(main())