laantungir/blossom
1
0
Fork 0
mirror of https://github.com/hzrd149/blossom.git synced 2026-10-05 15:38:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

Move multi-part uploads to BUD-12

Browse Source
This commit is contained in:
hzrd149
2026-03-08 11:01:11 -03:00
parent c20a2f5557
commit 40c152d857
2 changed files with 120 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
buds/11.md
+22 -1
View File
@@ -14,7 +14,7 @@ All authorization tokens:
- MUST have the `content` set to a human readable string explaining intended use to the user. For example `Upload Blob`, `Delete old blobs`, `List Images`, etc.
- MUST have a [NIP-40](https://github.com/nostr-protocol/nips/blob/master/40.md) `expiration` tag set to a unix timestamp at which the token should be considered expired.
- MUST have a [NIP-40](https://github.com/nostr-protocol/nips/blob/master/40.md) `expiration` tag set to a unix timestamp at which the token should be considered expired.
- MUST have a `t` tag with a verb of `get`, `upload`, `list`, `delete`, or `media`. The value of the `t` tag MUST correspond to the action performed by the target endpoint, as defined in the section [Endpoint Authorization Requirements](#endpoint-authorization-requirements).
@@ -81,6 +81,27 @@ The table below defines, for each endpoint, the required `t` tag action, the imp
| `PUT /mirror` | `upload` | SHA-256 of the mirrored blob | required |
| `PUT /media` | `media` | `X-SHA-256` request header | required |
| `HEAD /media` | `media` | `X-SHA-256` request header | required |
| `PATCH /upload` | `upload` | `X-SHA-256` request header | required (one `x` tag per chunk hash plus a final `x` tag for the complete blob hash; MAY be split across multiple auth events when chunk count is large) |
## Authorization Header Size Limits
HTTP servers impose limits on the size of individual request headers. Because the authorization token is base64url-encoded and carried in a single `Authorization` header, the number of `x` tags that can be included in one token is bounded by those limits.
A baseline kind-24242 event with one `x` tag encodes to approximately **636 bytes** as a header line. Each additional `x` tag (a 64-character hex sha256 hash) adds approximately **97 bytes** after base64url encoding.
| Server / proxy | Header limit | Max `x` tags |
|-----------------------------------------|-------------:|-------------:|
| Nginx (`large_client_header_buffers`) | 8,192 B | 78 |
| Apache (`LimitRequestFieldSize`) | 8,190 B | 78 |
| Cloudflare (max single header) | 16,384 B | 163 |
| IIS (`MaxRequestBytes`) | 16,384 B | 163 |
| Node.js / undici default | 16,384 B | 163 |
The conservative safe limit across common deployments is **8,192 bytes**, which allows up to **78 `x` tags** per authorization event.
Clients SHOULD keep the number of `x` tags in a single authorization event at or below **77**. When an upload requires more chunk hashes than this, clients SHOULD split the hashes across multiple authorization events and send each event in a separate request.
These figures were derived from the script at [`scripts/auth_header_limits.py`](../scripts/auth_header_limits.py).
## Security Considerations
buds/12.md
+98
View File
@@ -0,0 +1,98 @@
# BUD-12
## Multi-part uploads
`draft` `optional`
This bud defines a new `PATCH` method for the `/upload` endpoint to allow clients to upload blobs in multiple parts.
### Signaling support for multi-part uploads
The server SHOULD respond to an `OPTIONS /upload` request with a `204` "No Content" response according to [RFC-9110](https://httpwg.org/specs/rfc9110.html#rfc.section.9.3.7) including the [`Allow`](https://httpwg.org/specs/rfc9110.html#field.allow) header with `PATCH`
### Upload requirements
The server SHOULD implement [BUD-06](06.md) "Upload requirements" to allow clients to check if a blob can be uploaded before uploading any chunks.
### Chunking strategy
The client MAY split the blob into as many chunks as needed and MAY include overlap in the chunks if needed.
The server MUST concatenate the chunks based on the `Upload-Offset` and `Content-Length` headers to reconstruct the final blob to ensure any overlap is accounted for.
The server MUST support chunks arriving out of order and MUST support concurrent chunk uploads from the same client.
### Uploading chunks
Clients MUST send the following headers in each `PATCH /upload` request:
- `X-SHA-256`: The sha256 hash of the final blob. this should be considered the "ID" of the multi-part upload.
- `Upload-Type`: The mine type of the final blob. should be set like `Content-Type` defined in [RFC-9110](https://httpwg.org/specs/rfc9110.html#field.content-type)
- `Upload-Length`: The total length of the blob. should be set like `Content-Length` defined in [RFC-9110](https://httpwg.org/specs/rfc9110.html#field.content-length)
- `Content-Length`: The length of the chunk in bytes.
- `Upload-Offset`: The offset of the chunk in the blob.
- `Content-Type`: The type of the chunk. MUST be set to `application/octet-stream`
The server MUST respond with a `204` "No Content" response if the chunk was accepted or a `4xx` status code if it was not.
### Uploading the final chunk
Once the server has received enough chunks to cover the `Upload-Length` of the blob the server MUST respond with a `2xx` status code following [BUD-02](02.md) or a `4xx` status code if the blob hash does not match the `X-SHA-256` header
### Authorization
The server MAY require authorization for `PATCH /upload` requests as defined in [BUD-11](./11.md). The authorization event MUST use the `upload` verb and include one `x` tag per chunk hash plus a final `x` tag for the complete blob hash.
When the number of chunks is large enough that all chunk hashes would exceed HTTP header size limits, clients MAY split the chunk hashes across multiple authorization events and send each event in a separate request using the same `X-SHA-256` upload ID.
### Partial upload timeout
Servers MAY impose a timeout on in-progress uploads. If no `PATCH` request is received within the timeout window, the server MAY discard the partial upload and free any associated resources. The RECOMMENDED timeout is 60 seconds of inactivity between chunks.
### Resuming uploads
The client SHOULD keep track of a chunks it has uploaded in order to resume uploads after a failure.
### Example upload flow
```sh
# Client splits the blob into 4 chunks
split -b 46073 bitcoin.pdf chunk_
# Client uploads the first chunk
curl -X PATCH http://cdn.example.com/upload \
-H "X-SHA-256: b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553" \
-H "Upload-Type: application/pdf" \
-H "Upload-Length: 184292" \
-H "Upload-Offset: 0" \
-H "Content-Length: 46073" \
-H "Content-Type: application/octet-stream" \
--data-binary "@chunk_aa"
# Server accepts the chunk and responds with a 204
HTTP/1.1 204 No Content
# CLient uploads remaining chunks (2-4)
curl -X PATCH http://cdn.example.com/upload
# ..
--data-binary "@chunk_ab"
curl -X PATCH http://cdn.example.com/upload
# ...
--data-binary "@chunk_ac"
curl -X PATCH http://cdn.example.com/upload
# ...
--data-binary "@chunk_ad"
# Server responds with 200 OK
HTTP/1.1 200 OK
Content-Type: application/pdf
Content-Length: 184292
{
"url": "https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf",
"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
"uploaded": 1725105921,
"type": "application/pdf",
"length": 184292
}
```