BUD-11 merkle tree

2026-08-08 23:34:41 +00:00 · 2026-03-08 11:12:45 -03:00
parent 234e0e01d3
commit b075dc30b0
1 changed files with 91 additions and 0 deletions
@@ -82,6 +82,97 @@ The table below defines, for each endpoint, the required `t` tag action, the imp
 | `PUT /media`         | `media`      | `X-SHA-256` request header   | required            |
 | `HEAD /media`        | `media`      | `X-SHA-256` request header   | required            |

+## Authorization Header Size Limits
+
+HTTP servers impose limits on the size of individual request headers. Because the authorization token is base64url-encoded and carried in a single `Authorization` header, the number of `x` tags that can be included in one token is bounded by those limits.
+
+A baseline kind-24242 event with one `x` tag encodes to approximately **636 bytes** as a header line. Each additional `x` tag (a 64-character hex sha256 hash) adds approximately **97 bytes** after base64url encoding.
+
+| Server / proxy                          | Header limit | Max `x` tags |
+|-----------------------------------------|-------------:|-------------:|
+| Nginx (`large_client_header_buffers`)   |    8,192 B   |           78 |
+| Apache (`LimitRequestFieldSize`)        |    8,190 B   |           78 |
+| Cloudflare (max single header)          |   16,384 B   |          163 |
+| IIS (`MaxRequestBytes`)                 |   16,384 B   |          163 |
+| Node.js / undici default                |   16,384 B   |          163 |
+
+The conservative safe limit across common deployments is **8,192 bytes**, which allows up to **78 `x` tags** per authorization event.
+
+Clients SHOULD keep the number of `x` tags in a single authorization event at or below **77** (leaving one slot for the required final blob hash). When an upload requires more blob hashes than this, clients MUST either split the hashes across multiple authorization events or use [Merkle root authorization](#merkle-root-authorization).
+
+These figures were derived from the script at [`scripts/auth_header_limits.py`](../scripts/auth_header_limits.py).
+
+## Merkle root authorization
+
+When a client needs to authorize more blob hashes than can fit in a single `Authorization` header using individual `x` tags (see [Authorization Header Size Limits](#authorization-header-size-limits)), it MAY instead commit to the full set of hashes using a Merkle root.
+
+> **Status:** This section is `draft`. The construction algorithm and proof format described here must be validated by implementors before being considered stable. Feedback is welcome.
+
+### When to use Merkle root authorization
+
+A client SHOULD use Merkle root authorization when the number of blob hashes to authorize exceeds **77**. For 77 or fewer hashes, clients SHOULD use individual `x` tags as defined in [Tag scoping](#tag-scoping).
+
+### Tree construction
+
+The Merkle tree MUST be constructed as follows:
+
+1. **Leaves**: Each leaf value is the blob hash itself (the raw 32-byte SHA-256 digest of the blob content). The blob hashes MUST be sorted in ascending lexicographic order before constructing the tree to ensure a deterministic tree layout.
+2. **Internal nodes**: Each internal node is computed as `SHA-256(left_child || right_child)`, where `||` denotes concatenation of the two 32-byte child values.
+3. **Odd number of nodes**: If a level has an odd number of nodes, the last node is promoted unchanged to the next level (not duplicated) to avoid the well-known duplicate-leaf vulnerability.
+4. **Root**: The root is the single 32-byte value at the top of the tree, hex-encoded as 64 lowercase characters.
+
+### Authorization event format
+
+The authorization event MUST include a `merkle` tag containing the hex-encoded Merkle root in place of individual `x` tags. A `merkle` tag and individual `x` tags MUST NOT appear in the same authorization event.
+
+```jsonc
+{
+  "kind": 24242,
+  "pubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798",
+  "created_at": 1772019044,
+  "tags": [
+    ["t", "upload"],
+    ["expiration", "1740000000"],
+    // A single merkle tag replaces any number of x tags
+    ["merkle", "e3b0c44298fc1c149afb4c8996fb92427ae41e4649b934ca495991b7852b855"]
+  ],
+  "content": "Upload chunks",
+  "id": "...",
+  "sig": "..."
+}
+```
+
+### Merkle proof header
+
+For each request, the client MUST supply an `X-Merkle-Proof` HTTP header containing the proof that the request's blob hash belongs to the committed set.
+
+The proof is a JSON array of sibling nodes. Each element is a two-item array `[direction, hex_hash]` where:
+
+- `direction` is `"l"` (the sibling is the **left** child) or `"r"` (the sibling is the **right** child).
+- `hex_hash` is the 64-character lowercase hex-encoded SHA-256 sibling hash at that level.
+
+Nodes are ordered from the **leaf level up to (but not including) the root**.
+
+```
+X-Merkle-Proof: [["r","a1b2c3..."],["l","d4e5f6..."],["r","07a8b9..."]]
+```
+
+### Server validation
+
+To validate a Merkle proof, the server MUST:
+
+1. Derive the **leaf value** from the blob hash for this request (taken from the `X-SHA-256` header or the URL path, depending on endpoint). The server MUST re-derive the leaf itself and MUST NOT accept a client-supplied leaf value.
+2. Set `current = leaf_value`.
+3. For each `[direction, sibling]` node in the proof, in order:
+   - If `direction` is `"l"`: `current = SHA-256(sibling || current)`
+   - If `direction` is `"r"`: `current = SHA-256(current || sibling)`
+4. After processing all proof nodes, `current` MUST equal the `merkle` tag value in the authorization event.
+5. If the computed root does not match, the server MUST reject the request with `401 Unauthorized`.
+
+### Example
+
+A client uploading 200 chunks constructs a Merkle tree over all 200 blob hashes (sorted lexicographically). It embeds the single 64-character root in the `merkle` tag of one signed kind-24242 event. For each `PATCH /upload` request it sends the same `Authorization` header plus an `X-Merkle-Proof` header containing the ~8-node sibling path (⌈log₂ 200⌉ = 8 levels) for that chunk's hash. The server re-derives the leaf from the request's `X-SHA-256` value, walks the proof, and confirms the root matches — authorizing 200 chunks with a single signed event that fits comfortably within the 8 KiB header limit.
+
 ## Security Considerations

 ### Unscoped Tokens