laantungir/didactyl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity
Files
4430c36bcb178dedeea039070b1c793001e879bb
didactyl/plans/nsigner_integration.md
T

24 KiB
Raw Blame History

n_signer Integration Plan — didactyl

Goal

Separate control of the nsec from the didactyl agent by routing signing/encryption through n_signer — a separate foreground process that holds key material in locked memory and signs on request. A later phase may embed n_signer into the didactyl binary so one self-contained binary spawns an n_signer in a separate window; that is deferred because of a reliability concern (if the agent loses signing ability it cannot even DM its admin about the failure).

This plan covers two distinct work items:

  1. Update didactyl's vendored nostr_core_lib to the upstream version that already implements the nsigner client (M1–M6 complete, upstream VERSION 0.6.6). This is a mechanical sync that changes no runtime behavior.
  2. Opt-in migration of didactyl call sites to the nostr_signer_t provider so the raw private key can leave the agent process. This is the actual nsec-separation work and is staged so reliability is preserved.

Background — what upstream already provides

Upstream nostr_core_lib (git.laantungir.net/nostr_core_lib, v0.6.6) has shipped the full signer abstraction. didactyl's vendored copy predates all of it.

New upstream modules:

  • nostr_core/nostr_signer.{h,c} — nostr_signer_t opaque handle + vtable. Two backends:
    • local — nostr_signer_local(privkey[32]), wraps today's raw-key path. Output-equivalent.
    • remote (nsigner) — nostr_signer_nsigner_unix/serial/tcp/fds(...), guarded by NOSTR_ENABLE_NSIGNER_CLIENT.
  • nostr_core/nsigner_client.{h,c} — framing, JSON-RPC verbs, kind-27235 auth envelope, error mapping.
  • nostr_core/nsigner_transport.{h,c} — pluggable transport vtable: AF_UNIX abstract, TCP, USB-CDC serial, fd-pair (stdio/qrexec).

Verbs mirror n_signer's wire contract: get_public_key, sign_event, nip04_encrypt/decrypt, nip44_encrypt/decrypt.

Parallel *_with_signer entry points added across the library (legacy raw-key APIs are preserved as thin wrappers over the local backend, so existing projects keep compiling and behaving identically):

  • NIP-01: nostr_create_and_sign_event_with_signer
  • NIP-17: nostr_nip17_create_relay_list_event_with_signer, nostr_nip17_send_dm_with_signer, nostr_nip17_receive_dm_with_signer
  • NIP-42: nostr_nip42_create_auth_event_with_signer
  • NIP-46: request/response/decrypt/session-init *_with_signer variants
  • NIP-59: seal/unwrap/unseal *_with_signer variants
  • NIP-60 / NIP-61: all cashu event-creation *_with_signer variants
  • Blossom: blossom_create_auth_header_with_signer, blossom_upload_with_signer, blossom_upload_file_with_signer, blossom_delete_with_signer
  • Relay pool: nostr_relay_pool_set_auth_with_signer — the pool now stores an auth_signer instead of a raw key, so NIP-42 AUTH events are signed through the provider on relay challenge. This was the hardest blocker and it is already solved upstream.

Upstream build.sh unconditionally defines -DNOSTR_ENABLE_NSIGNER_CLIENT=1 and compiles nostr_signer.c, nsigner_transport.c, nsigner_client.c.

Dependency direction is unchanged and non-circular: n_signer (server) depends on nostr_core_lib for leaf crypto; nostr_core_lib gains a client that consumes its own primitives. Nobody imports anyone's binary in a loop.

Current didactyl state (what we are updating from)

  • nostr_core_lib/ is a vendored source directory (no git submodule; no .gitmodules). Built via ./nostr_core_lib/build.sh --nips=001,004,...,061 into a static libnostr_core_*.a, linked by Makefile and the Docker static build (Dockerfile.alpine-musl).
  • The vendored tree has none of the nsigner pieces: no nostr_signer.*, no nsigner_*.*, no *_with_signer variants. The relay pool still uses the raw-key AUTH path (nostr_core_lib/nostr_core/core_relay_pool.c).
  • didactyl holds the raw 32-byte key in cfg->keys.private_key and passes it directly into ~30+ library call sites (event signing, NIP-04/44 encrypt/decrypt, NIP-17 gift wraps, NIP-60/61 cashu, blossom, relay pool AUTH).

Architecture (target)

graph TD
    A[didactyl src] --> B[nostr_core_lib vendored]
    B --> C[nostr_signer_t local backend]
    B --> D[nostr_signer_t nsigner remote backend]
    D --> E[nsigner client: framing + auth envelope]
    E --> F[nostr_create_and_sign_event in nostr_core_lib]
    G[n_signer server binary] --> B

didactyl constructs one nostr_signer_t* at startup and threads it through every signing/encryption/AUTH call site. The backend is chosen by config:

  • signer.mode = "local" → nostr_signer_local(cfg->keys.private_key) (today's behavior).
  • signer.mode = "nsigner_unix" → nostr_signer_nsigner_unix(socket_name, role, timeout_ms).
  • (future) nsigner_tcp, nsigner_serial, nsigner_fds.

The raw cfg->keys.private_key is only populated in local mode. In remote modes the agent process never holds the nsec; it learns its own pubkey via nostr_signer_get_public_key().

Phasing

Phase 1 — Update vendored nostr_core_lib (mechanical, zero behavior change)

Goal: bring didactyl's nostr_core_lib source tree to upstream v0.6.6 so the signer abstraction and nsigner client are available. No didactyl source changes. Existing raw-privkey APIs are preserved upstream as thin wrappers over the local backend, so all ~30 call sites keep compiling and behaving identically.

Steps:

  1. Snapshot current vendored tree for diffing/rollback: cp -a nostr_core_lib nostr_core_lib.old
  2. Replace the vendored source tree with upstream v0.6.6. Preferred: copy the upstream nostr_core/, cjson/, nostr_websocket/, platform/, build.sh, CMakeLists.txt into nostr_core_lib/, preserving didactyl-local build quirks only if any exist (none found — the Dockerfile already seds build.sh in-place for fortification). Do not carry over upstream's prebuilt .a/.o files, pool.log, .venv*, backups/, verify_* artifacts — match didactyl's existing .gitignore hygiene.
  3. Rebuild the static library: cd nostr_core_lib && ./build.sh --nips=001,004,005,006,011,013,017,019,021,042,044,046,059,060,061 (same NIP set as today; nostr_signer.c / nsigner_*.c are compiled unconditionally by upstream build.sh).
  4. Rebuild didactyl: make clean && make. Confirm it compiles and links with no source changes.
  5. Smoke test: run the agent with an nsec-only config and confirm signing/DMs/relay AUTH still work byte-for-byte (the upstream local backend is output-equivalent to the legacy path; there is an upstream test test_event_creation_equivalence_with_signer that asserts this).
  6. Static (Docker) build: ./build_static.sh. The Dockerfile copies nostr_core_lib/ and runs ./build.sh --nips=all; verify the static binary still builds and runs.
  7. Record the upstream version pinned in this repo (e.g. a one-line nostr_core_lib/UPSTREAM_VERSION file = 0.6.6) so future syncs are auditable.

Exit criteria: didactyl builds and runs identically, but now links a nostr_core_lib that exposes nostr_signer_t and the nsigner client. No nsec separation yet.

Phase 2 — Signer provider in didactyl config + startup (local backend, still no separation)

Goal: introduce a nostr_signer_t* in didactyl constructed at startup, backed by local mode by default. This is the seam; behavior is still identical.

2a. Config schema

src/config.c / src/config.h: add a signer block (sibling of keys):

"signer": {
  "mode": "local",            // "local" | "nsigner_unix" | (future) "nsigner_tcp" | "nsigner_serial" | "nsigner_fds"
  "socket_name": "",          // nsigner_unix: abstract socket name (without @); empty = auto-discover single running signer
  "role": "main",             // n_signer role selector
  "timeout_ms": 15000,
  "auth_privkey_hex": ""      // optional, for TCP auth envelope only
}

Default mode = "local". Backward compatible: if signer block is absent, behave as local using keys.nsec. Validation: in any nsigner_* mode, keys.nsec is optional and ignored (the agent learns its pubkey from the signer); in local mode keys.nsec remains required.

2b. User-facing surfaces (how the operator chooses n_signer)

didactyl has three startup surfaces today; the n_signer choice is exposed on all three, consistent with existing patterns:

1. Interactive setup wizard (src/setup_wizard.c, new_agent_identity_step at line ~2116). Today the Identity page offers: [g]enerate new keypair / [p]rovide existing nsec. Add a third option:

  • [s]ign with a running n_signer (separate the nsec from the agent)

When chosen, the wizard:

  • Does not prompt for an nsec. Instead prompts for:
    • socket name (default: auto-discover the single running signer),
    • role (default main),
    • timeout (default 15000ms).
  • Performs a live nostr_signer_nsigner_unix(...) + nostr_signer_get_public_key() connectivity check immediately, so the operator gets confirmation ("Connected to n_signer @nsigner_hairy_dog; agent npub: npub1...") before proceeding.
  • Writes signer.mode = "nsigner_unix" (plus socket_name/role/timeout) into the generated genesis JSONC and omits keys.nsec.
  • The existing [p]rovide existing nsec path stays as local mode (the default).

This keeps the trust model visible at the moment the operator commits to an identity: either the nsec lives in the agent (local) or it does not (nsigner_unix).

2. CLI flags + env (src/main.c print_usage at line ~141 and arg parsing at line ~1233). Add:

  • --signer <mode> — one of local, nsigner_unix, nsigner_tcp, nsigner_serial, nsigner_fds. Overrides signer.mode in config.
  • --signer-socket <name> — abstract socket name (nsigner_unix).
  • --signer-role <role> — n_signer role selector (default main).
  • --signer-timeout <ms> — per-call timeout (default 15000).
  • --signer-tcp <host:port> — shorthand for --signer nsigner_tcp + host/port.
  • DIDACTYL_SIGNER env — fallback for --signer (mirrors the DIDACTYL_NSEC pattern).

When --signer nsigner_unix (or DIDACTYL_SIGNER=nsigner_unix) is set, --nsec / DIDACTYL_NSEC are not required and are ignored if present. This enables headless / systemd deployment where the nsec never touches the agent's argv or environment.

Update print_usage examples to show the n_signer path, e.g.: %s --signer nsigner_unix --signer-socket nsigner_hairy_dog --admin npub1...

3. Genesis JSONC config — the signer block in 2a is the canonical, persistent form. The wizard writes it; CLI flags override it for a single run; env vars are the last fallback. This matches how keys.nsec / --nsec / DIDACTYL_NSEC already layer.

Precedence (highest to lowest): CLI flag → env var → genesis signer block → default local. Same precedence as the existing key/admin overrides.

2c. Startup construction

src/main.c: after loading config + applying CLI/env overrides, construct a single nostr_signer_t* g_signer:

  • local → nostr_signer_local(cfg->keys.private_key).
  • nsigner_unix → nostr_signer_nsigner_unix(socket_name, role, timeout_ms).
  • (future) nsigner_tcp / nsigner_serial / nsigner_fds analogously.

Then derive the agent pubkey via nostr_signer_get_public_key(g_signer, ...) instead of nostr_ec_public_key_from_private_key(cfg->keys.private_key, ...). In local mode this yields the same bytes; in remote mode it is the first n_signer round-trip and doubles as the mandatory startup connectivity check (Phase 4): failure here is fatal and loud.

Hold g_signer alongside g_cfg / g_pool for the process lifetime; free on shutdown.

Exit criteria: didactyl runs with a nostr_signer_t* in local mode by default, and the operator can choose nsigner_unix via wizard, CLI, or config. No call sites migrated yet (Phase 3). This is a safe, reviewable checkpoint — in local mode behavior is identical; in nsigner_unix mode the agent starts but signing still uses the local path until Phase 3 lands (so Phase 2 remote mode is a connectivity proof, not full separation).

Phase 3 — Migrate call sites to *_with_signer (enables real separation)

Goal: route every signing/encryption/AUTH operation through g_signer. In local mode this is still output-equivalent; in nsigner_unix mode the nsec is no longer in the agent process.

Migrate in this order (highest value / lowest risk first):

  1. Relay pool AUTH — replace nostr_relay_pool_set_auth(g_pool, g_cfg->keys.private_key, 1) with nostr_relay_pool_set_auth_with_signer(g_pool, g_signer, 1). This is the single change that delegates the deep-event-loop AUTH signing.
  2. Event signing — nostr_create_and_sign_event(...) → nostr_create_and_sign_event_with_signer(...) at all sites (posts, kind 30078 config publish, startup events, etc.). Search: nostr_create_and_sign_event in src/.
  3. NIP-44 encrypt/decrypt — the DM/memory/task/config/block-list sites. These call nostr_nip44_encrypt/decrypt(cfg->keys.private_key, peer, ...). Replace with nostr_signer_nip44_encrypt/decrypt(g_signer, peer_hex, ...). Note the signer API takes hex pubkeys and returns heap strings via char** out-params, so each call site needs a small adaptation (hex-encode the peer key, free the out-string).
  4. NIP-04 encrypt/decrypt — same pattern as NIP-44 for the legacy NIP-04 DM sites.
  5. NIP-17 gift wraps — nostr_nip17_send_dm / nostr_nip17_receive_dm → *_with_signer. (The seal is signed with the agent key via the signer; the gift-wrap ephemeral key stays local — that is correct and does not leak the nsec.)
  6. NIP-60/61 cashu — nostr_nip60_create_*_event / nostr_nip61_create_*_event → *_with_signer in src/cashu_wallet.c.
  7. Blossom — blossom_create_auth_header / blossom_upload / blossom_delete → *_with_signer in src/tools/tool_blossom.c.

After each group, build + smoke test in local mode (output-equivalent) before testing in nsigner_unix mode against a running n_signer.

Exit criteria: every signing/encryption/AUTH op flows through g_signer. With signer.mode = "nsigner_unix", the agent process holds no nsec.

Phase 4 — Reliability & failure handling (critical, do before enabling remote by default)

The user's stated concern: if the agent loses signing ability, it cannot even DM its admin about the failure. This must be addressed before remote mode is anything but opt-in.

Design:

  • Startup connectivity check is mandatory. nostr_signer_get_public_key() is called at startup for every mode. In remote mode, failure here is fatal and loud (agent refuses to start, prints a clear error). Better to fail to start than to start unable to sign.
  • Runtime signing failure — classify by error:
    • approval_denied / policy_denied / unknown_role → not a transient error; surface to agent log and admin context, do not retry blindly.
    • transport/IO error (n_signer crashed, socket gone) → the nsigner transport has a reconnect hook; the client retries once with reconnect, then returns an error.
  • Admin notification path. Because DMs themselves require signing, a signing outage is a catch-22 for "DM the admin". Mitigations, in order of preference:
    1. Health-gate before sending. Before attempting an admin DM, do a cheap nostr_signer_get_public_key() ping. If it fails, fall back to a local, keyless notification channel: stderr/log + HTTP API health endpoint (src/http_api.c) exposes a signer_unhealthy flag. The admin monitors the HTTP health endpoint (or process supervisor) rather than relying on a DM that requires the broken signer.
    2. Optional emergency local key. A config-gated signer.emergency_local_nsec used only to sign admin DMs when the remote signer is unreachable. This is a deliberate policy tradeoff (it re-introduces a key in-process, scoped to admin alerts only) and should be off by default. Document the tradeoff clearly.
    3. Process supervisor. systemd/didactyl service restarts on signer health failure; the supervisor's logs are the out-of-band channel.
  • Recommendation for v1: ship (1) + (3). Leave (2) as a documented opt-in for operators who prioritize alert delivery over strict key separation. This keeps the default trust model clean while making the failure observable out-of-band.

Exit criteria: a signing outage is detectable out-of-band (HTTP health + logs + supervisor), and the agent does not silently hang. Remote mode is still opt-in.

Phase 4 implementation status (items 18–20) — COMPLETE

Items 18, 19, and 20 are implemented as follows. Behavior is backward compatible and remote mode remains opt-in.

Item 18 — Runtime signer health classification & tracking

  • New module src/signer_health.{h,c} holds a process-global, mutex-protected health state: unknown | healthy | unhealthy, the configured signer mode snapshot, the last error reason string, and the last error / last ok unix timestamps.
  • src/main.c construct_signer() records the mode at startup and records ok on a successful connectivity check (get_public_key) or failure (with the nostr error code) on construction/connectivity failure. The fatal+loud startup behavior is preserved; error messages now explicitly include mode=, socket=, role=, timeout_ms=, and rc= so the operator knows exactly which signer context failed.
  • The migrated signer call paths in src/nostr_handler.c (NIP-44 self encrypt/decrypt, NIP-04 incoming DM decrypt, NIP-04 outgoing DM encrypt) call signer_health_record_ok() on success and signer_health_record_failure(rc, "<verb>") on failure. Classification maps nostr error codes to short labels: transport/IO (IO/network), crypto/auth, resource, invalid-input, unknown. This is deliberately minimal — no broad rewrite of the tool/agent/cashu/block-list signer sites; the central DM and self-config paths are instrumented, which is where an outage first surfaces.

Item 19 — Out-of-band health surface (HTTP + logs + supervisor)

  • src/http_api.c handle_status() (the GET /api/status endpoint) now adds the following fields to the JSON response:
    • signer_mode — the configured signer mode (local / nsigner_unix / nsigner_tcp).
    • signer_healthy — "true" / "false" / "unknown".
    • signer_health_state — healthy / unhealthy / unknown.
    • signer_last_error — human-readable <classification>: <detail> (rc=<n>), or "".
    • signer_last_error_ts — unix timestamp of the last unhealthy transition, or null.
    • signer_last_ok_ts — unix timestamp of the last healthy observation, or null. An operator (or supervisor / monitoring agent) can poll /api/status to detect a signing outage without requiring a signed admin DM to come back.
  • signer_health_record_failure() and signer_health_record_ok() each emit a one-line stderr warning on a health transition (healthy -> unhealthy, or unhealthy -> healthy), including the mode and reason. Repeated failures within the same unhealthy state do not spam the log.

Item 20 — Optional emergency_local_nsec (admin-DM delivery tradeoff)

  • New config field signer.emergency_local_nsec (string, empty default) in src/config.h signer_config_t, parsed in src/config.c parse_signer_config().
  • No active fallback behavior is wired yet. The field is parsed, stored, and documented only — a feature-flag/placeholder so operators can pre-stage the value. The actual "sign a minimal admin-alert DM with the emergency key when the remote signer is unreachable" path is deferred until the Phase 4 health surface (item 19) is proven in production.
  • Tradeoff (documented here and in the config header comment): off by default. When eventually enabled, it reintroduces the nsec into the agent process only for admin-alert fallback DMs. This is a deliberate operator-accepted tradeoff: better to leak the nsec briefly to notify the admin than to fail silently. Operators who prioritize strict key separation leave it empty and rely on the HTTP health endpoint + supervisor logs (item 19) as the out-of-band alert channel.

Validation

  • make clean && make builds cleanly with the new signer_health.c source.
  • ./didactyl --help and ./didactyl --version unchanged in surface.
  • GET /api/status in local mode returns the new signer_mode / signer_healthy / signer_health_state / signer_last_error / signer_last_error_ts / signer_last_ok_ts fields (local mode reports signer_healthy=true after the startup connectivity check).

Phase 5 (deferred) — Embedded n_signer spawn

Goal: one self-contained didactyl binary that spawns an n_signer in a separate window at startup, for operators who want key separation without running a second process manually.

Deferred because of the reliability concern above and because it couples didactyl's process model to n_signer's TUI/terminal-attached trust model. Revisit after Phase 4's health/observability story is proven in production with an external n_signer.

Sketch (not for this round): didactyl forks n_signer --mnemonic-fd N --socket-name <pinned> in a new terminal (xterm/foot/etc.), passes the mnemonic over a pipe fd, pins the abstract socket name, and constructs nostr_signer_nsigner_unix(<pinned name>, ...). The pinned socket name avoids discovery races. The trust anchor is still the terminal-attached n_signer process; didactyl is just a supervised spawner.

Open decisions (resolved)

  1. Where does the abstraction live? Inside nostr_core_lib. It is the single chokepoint, it solves the relay-AUTH blocker, and it is reusable. ✓ (already done upstream)
  2. How is the n_signer client code sourced into nostr_core_lib? Already decided upstream: the client (framing + auth envelope + transports) lives in nostr_core_lib itself, guarded by NOSTR_ENABLE_NSIGNER_CLIENT. No vendoring of n_signer's repo into nostr_core_lib; n_signer keeps depending on nostr_core_lib for leaf crypto. ✓
  3. Runtime failure handling. See Phase 4: out-of-band health (HTTP + logs + supervisor) by default; optional emergency-local-nsec as documented opt-in. ✓

Risks & notes

  • NIP set in Makefile vs build.sh --nips=all in Dockerfile. Both must include the NIPs didactyl uses. The signer core (nostr_signer.c, nsigner_*.c) is compiled unconditionally by upstream build.sh, so the nsigner client is available in both build paths. Verify after sync.
  • Static link footprint. nsigner client + transports add a small amount of object code (unix/tcp/serial/fd transports). Negligible vs. the existing websocket/curl/openssl surface. Serial transport pulls in <termios.h> — already POSIX, fine for musl.
  • Output equivalence. Upstream's local backend is asserted equivalent by test_event_creation_equivalence_with_signer. Phase 1 smoke test should still verify a known event signs to the same id as before.
  • cfg->keys.private_key lifetime. In remote mode this buffer is zeroed/unused. Keep the field for local mode and for the optional emergency key; do not remove it.
  • Hex pubkey plumbing. The signer's encrypt/decrypt verbs take hex pubkeys and return heap strings. Several didactyl sites currently pass raw 32-byte keys. Each migrated site needs a small hex-encode + free adaptation. Mechanical but touches many files — Phase 3 step 3/4 is the bulk of the work.
Reference in New Issue View Git Blame Copy Permalink