Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fa019fe9a1 | ||
|
|
0f31e1c301 | ||
|
|
c9a20e63b5 | ||
|
|
79e38b8f79 | ||
|
|
7cfd43b91b | ||
|
|
777ab312a5 | ||
|
|
2763dd9d6a | ||
|
|
a20b28cd21 | ||
|
|
456f0e1f2e | ||
|
|
4abb4bf33c | ||
|
|
7dcfc7c3e3 | ||
|
|
07f5f39f35 |
@@ -0,0 +1,122 @@
|
|||||||
|
# Broadcast Relay Batching Fix
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
With 600+ broadcast relays, NDK's `fromRelayUrls` creates 630 temporary relays
|
||||||
|
and adds them all to the pool simultaneously. `NDKRelaySet.publish()` then fires
|
||||||
|
all 630 relay publish operations in parallel via `Promise.all`. Each relay that
|
||||||
|
isn't connected calls `relay.connect()` which opens a WebSocket.
|
||||||
|
|
||||||
|
Browsers limit simultaneous WebSocket connections (practically ~50-100 at a
|
||||||
|
time before connections queue up). With 630 simultaneous connection attempts,
|
||||||
|
most relays never get a connection slot before the per-relay timeout expires.
|
||||||
|
Result: only ~13/630 relays succeed.
|
||||||
|
|
||||||
|
## Solution: Chunked publishing
|
||||||
|
|
||||||
|
Instead of creating one giant `NDKRelaySet` with all 630 relays and calling
|
||||||
|
`publish()` once, we split the relay URLs into **batches of 50** and publish
|
||||||
|
to each batch sequentially. Each batch gets its own `NDKRelaySet.fromRelayUrls()`
|
||||||
|
call, its own `publish()`, and its own set of live progress listeners.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
A[630 relay URLs] --> B[Split into batches of 50]
|
||||||
|
B --> C[Batch 1: 50 relays]
|
||||||
|
C --> D[publish to batch 1]
|
||||||
|
D --> E{Batch 1 done?}
|
||||||
|
E -->|yes| F[Batch 2: 50 relays]
|
||||||
|
F --> G[publish to batch 2]
|
||||||
|
G --> H{Batch 2 done?}
|
||||||
|
H -->|yes| I[... continue ...]
|
||||||
|
I --> J[Batch 13: 30 relays]
|
||||||
|
J --> K[publish to batch 13]
|
||||||
|
K --> L{All batches done?}
|
||||||
|
L -->|yes| M[Final done event with total count]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key design points
|
||||||
|
|
||||||
|
1. **Batch size: 50 relays** — small enough to stay within browser WebSocket
|
||||||
|
limits, large enough to finish in a reasonable time. Each batch takes ~5-10s.
|
||||||
|
|
||||||
|
2. **Sequential batches** — wait for each batch to complete before starting the
|
||||||
|
next. This keeps the total simultaneous connections at ~50, not 630.
|
||||||
|
|
||||||
|
3. **Live progress across batches** — the `publishedSoFar` and `failedSoFar`
|
||||||
|
sets accumulate across all batches. The `broadcastProgress` events stream
|
||||||
|
continuously, so the footer shows the count climbing from 0 to 630 across
|
||||||
|
all batches.
|
||||||
|
|
||||||
|
4. **Per-relay timeout: 10s per batch** — each relay in a batch gets 10s to
|
||||||
|
connect + publish. Since only 50 are competing for connections (not 630),
|
||||||
|
this is plenty of time.
|
||||||
|
|
||||||
|
5. **Overall deadline: removed** — since batches are sequential and each has
|
||||||
|
its own per-relay timeout, the overall time is naturally bounded at
|
||||||
|
`numBatches × perBatchTimeout`. No need for a separate overall deadline.
|
||||||
|
|
||||||
|
6. **Connection timeout: 10s** — set on each relay in the batch, matching the
|
||||||
|
publish timeout.
|
||||||
|
|
||||||
|
## Implementation (in `www/ndk-worker.js` `handlePublish`)
|
||||||
|
|
||||||
|
Replace the current single `NDKRelaySet.fromRelayUrls(allUrls, ndk)` + single
|
||||||
|
`publish()` with a batched loop:
|
||||||
|
|
||||||
|
```js
|
||||||
|
const BATCH_SIZE = 50;
|
||||||
|
const PER_RELAY_TIMEOUT_MS = 10000;
|
||||||
|
|
||||||
|
if (isBroadcast) {
|
||||||
|
const allUrls = [...new Set([...outboxWriteUrls, ...activeBroadcastUrls])];
|
||||||
|
const batches = [];
|
||||||
|
for (let i = 0; i < allUrls.length; i += BATCH_SIZE) {
|
||||||
|
batches.push(allUrls.slice(i, i + BATCH_SIZE));
|
||||||
|
}
|
||||||
|
console.log(`[Worker] Broadcasting to ${allUrls.length} relays in ${batches.length} batches of ${BATCH_SIZE}`);
|
||||||
|
|
||||||
|
// Emit start event
|
||||||
|
broadcast({ type: 'broadcastProgress', sessionId, eventId, eventKind, phase: 'start', total: allUrls.length, successful: 0, failed: 0, latestRelay: null });
|
||||||
|
|
||||||
|
// Attach listeners once — they accumulate across all batches
|
||||||
|
ndkEvent.on('relay:published', (relay) => { ... publishedSoFar.add ... });
|
||||||
|
ndkEvent.on('relay:publish:failed', (relay, err) => { ... failedSoFar.add ... });
|
||||||
|
|
||||||
|
// Publish to each batch sequentially
|
||||||
|
for (let i = 0; i < batches.length; i++) {
|
||||||
|
const batch = batches[i];
|
||||||
|
const batchSet = NDKRelaySet.fromRelayUrls(batch, ndk);
|
||||||
|
// Set connection timeout on each relay in this batch
|
||||||
|
for (const relay of batchSet.relays) {
|
||||||
|
try { relay.connectionTimeout = PER_RELAY_TIMEOUT_MS; } catch (_) {}
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await ndkEvent.publish(batchSet, PER_RELAY_TIMEOUT_MS, 1);
|
||||||
|
} catch (e) {
|
||||||
|
// Expected — some relays in this batch may fail
|
||||||
|
}
|
||||||
|
console.log(`[Worker] Batch ${i+1}/${batches.length} done: ${publishedSoFar.size}/${allUrls.length} total succeeded`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Emit final done event
|
||||||
|
broadcast({ type: 'broadcastProgress', sessionId, eventId, eventKind, phase: 'done', total: allUrls.length, successful: publishedSoFar.size, failed: failedSoFar.size + timedOutSoFar.size, relayUrls: Array.from(publishedSoFar) });
|
||||||
|
// Clean up listeners
|
||||||
|
ndkEvent.removeAllListeners('relay:published');
|
||||||
|
ndkEvent.removeAllListeners('relay:publish:failed');
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Files touched
|
||||||
|
|
||||||
|
| File | Change |
|
||||||
|
|------|--------|
|
||||||
|
| [`www/ndk-worker.js`](www/ndk-worker.js) | Replace single publish with batched loop in `handlePublish` |
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
1. Post with a user that has 600+ broadcast relays
|
||||||
|
2. Watch the footer: count should climb steadily as batches complete
|
||||||
|
3. Final count should be much higher than 13 (ideally 500+)
|
||||||
|
4. Each batch of 50 should take ~5-10 seconds
|
||||||
|
5. Total time for 630 relays: ~13 batches × ~10s = ~130 seconds
|
||||||
@@ -392,6 +392,7 @@ function handleWorkerMessage(event) {
|
|||||||
failed: message.failed,
|
failed: message.failed,
|
||||||
latestRelay: message.latestRelay,
|
latestRelay: message.latestRelay,
|
||||||
latestError: message.latestError,
|
latestError: message.latestError,
|
||||||
|
relayUrls: message.relayUrls || [],
|
||||||
}
|
}
|
||||||
}));
|
}));
|
||||||
} else if (message.type === 'startupStatus') {
|
} else if (message.type === 'startupStatus') {
|
||||||
|
|||||||
@@ -915,16 +915,78 @@ export function formatTimeAgo(timestamp) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Store timestamps for real-time updates
|
// Store timestamps for real-time updates
|
||||||
const timeAgoElements = new Map(); // element -> timestamp
|
const timeAgoElements = new Map(); // element -> { timestamp, eventId }
|
||||||
|
|
||||||
|
// Track relay publish info per event ID from broadcastProgress events.
|
||||||
|
// Keyed by event ID → { count, urls } where urls is an array of relay URLs.
|
||||||
|
const relayInfoByEventId = new Map();
|
||||||
|
|
||||||
|
// Listen for broadcast progress events to capture final relay counts and URLs.
|
||||||
|
// The worker emits ndkBroadcastProgress with phase 'done' containing the
|
||||||
|
// total successful count and the list of relay URLs. We store it so post
|
||||||
|
// cards can show "21r - 1h" with a hover tooltip listing the relays.
|
||||||
|
if (typeof window !== 'undefined') {
|
||||||
|
window.addEventListener('ndkBroadcastProgress', (event) => {
|
||||||
|
const d = event.detail;
|
||||||
|
if (!d || d.phase !== 'done') return;
|
||||||
|
const eventId = d.eventId;
|
||||||
|
if (!eventId) return;
|
||||||
|
relayInfoByEventId.set(eventId, {
|
||||||
|
count: d.successful || 0,
|
||||||
|
urls: Array.isArray(d.relayUrls) ? d.relayUrls : [],
|
||||||
|
});
|
||||||
|
// Update any already-rendered time elements for this event.
|
||||||
|
timeAgoElements.forEach((info, element) => {
|
||||||
|
if (info.eventId === eventId && element.isConnected) {
|
||||||
|
updateTimeAgoElement(element, info);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build the hover tooltip text for a relay list.
|
||||||
|
* Shows "Published to N relays:" followed by the URL list, truncated.
|
||||||
|
* @param {number} count - Number of successful relays
|
||||||
|
* @param {string[]} urls - Array of relay URLs
|
||||||
|
* @returns {string}
|
||||||
|
*/
|
||||||
|
function buildRelayTooltip(count, urls) {
|
||||||
|
if (!count || count === 0) return '';
|
||||||
|
const header = `Published to ${count} relay${count === 1 ? '' : 's'}:`;
|
||||||
|
if (!urls || urls.length === 0) return header;
|
||||||
|
// Show up to 50 relay URLs in the tooltip; beyond that, show a summary.
|
||||||
|
if (urls.length <= 50) {
|
||||||
|
return header + '\n' + urls.join('\n');
|
||||||
|
}
|
||||||
|
return header + '\n' + urls.slice(0, 50).join('\n') + `\n… and ${urls.length - 50} more`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Update a time-ago element's text and tooltip from its stored info.
|
||||||
|
* @param {HTMLElement} element - The element to update
|
||||||
|
* @param {{timestamp: number, eventId: string}} info - Stored info
|
||||||
|
*/
|
||||||
|
function updateTimeAgoElement(element, info) {
|
||||||
|
const timeStr = formatTimeAgo(info.timestamp);
|
||||||
|
if (info.eventId && relayInfoByEventId.has(info.eventId)) {
|
||||||
|
const relayInfo = relayInfoByEventId.get(info.eventId);
|
||||||
|
element.textContent = `${relayInfo.count}r - ${timeStr}`;
|
||||||
|
element.title = buildRelayTooltip(relayInfo.count, relayInfo.urls);
|
||||||
|
} else {
|
||||||
|
element.textContent = timeStr;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Register an element for time updates
|
* Register an element for time updates
|
||||||
* @param {HTMLElement} element - The element to update
|
* @param {HTMLElement} element - The element to update
|
||||||
* @param {number} timestamp - Unix timestamp in seconds
|
* @param {number} timestamp - Unix timestamp in seconds
|
||||||
|
* @param {string} [eventId] - Optional event ID for relay count display
|
||||||
*/
|
*/
|
||||||
export function registerTimeAgo(element, timestamp) {
|
export function registerTimeAgo(element, timestamp, eventId) {
|
||||||
timeAgoElements.set(element, timestamp);
|
timeAgoElements.set(element, { timestamp, eventId });
|
||||||
element.textContent = formatTimeAgo(timestamp);
|
updateTimeAgoElement(element, { timestamp, eventId });
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -932,9 +994,9 @@ export function registerTimeAgo(element, timestamp) {
|
|||||||
* Call this periodically (e.g., every 30 seconds)
|
* Call this periodically (e.g., every 30 seconds)
|
||||||
*/
|
*/
|
||||||
export function updateTimeAgos() {
|
export function updateTimeAgos() {
|
||||||
timeAgoElements.forEach((timestamp, element) => {
|
timeAgoElements.forEach((info, element) => {
|
||||||
if (element.isConnected) {
|
if (element.isConnected) {
|
||||||
element.textContent = formatTimeAgo(timestamp);
|
updateTimeAgoElement(element, info);
|
||||||
} else {
|
} else {
|
||||||
// Clean up detached elements
|
// Clean up detached elements
|
||||||
timeAgoElements.delete(element);
|
timeAgoElements.delete(element);
|
||||||
@@ -1085,10 +1147,12 @@ export function renderFooterRow(eventId, eventData, options = {}) {
|
|||||||
container.appendChild(nutzapItem);
|
container.appendChild(nutzapItem);
|
||||||
container.appendChild(midControls);
|
container.appendChild(midControls);
|
||||||
|
|
||||||
// Time as the final item in the same row
|
// Time as the final item in the same row.
|
||||||
|
// Pass eventId so registerTimeAgo can prepend the relay count
|
||||||
|
// (e.g., "21r - 1h") when a broadcastProgress 'done' event arrives.
|
||||||
const timeEl = document.createElement('span');
|
const timeEl = document.createElement('span');
|
||||||
timeEl.className = 'divPostTime';
|
timeEl.className = 'divPostTime';
|
||||||
registerTimeAgo(timeEl, eventData.created_at);
|
registerTimeAgo(timeEl, eventData.created_at, eventId);
|
||||||
container.appendChild(timeEl);
|
container.appendChild(timeEl);
|
||||||
|
|
||||||
footerRow.appendChild(container);
|
footerRow.appendChild(container);
|
||||||
|
|||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"VERSION": "v0.7.66",
|
"VERSION": "v0.7.78",
|
||||||
"VERSION_NUMBER": "0.7.66",
|
"VERSION_NUMBER": "0.7.78",
|
||||||
"BUILD_DATE": "2026-06-30T11:47:22.403Z"
|
"BUILD_DATE": "2026-06-30T14:33:14.964Z"
|
||||||
}
|
}
|
||||||
|
|||||||
+222
-45
@@ -4568,6 +4568,16 @@ async function handleWalletCreateDeposit(requestId, amount, mint, port) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function handleWalletPayInvoice(requestId, invoice, mint, port) {
|
async function handleWalletPayInvoice(requestId, invoice, mint, port) {
|
||||||
|
const startedAt = Date.now();
|
||||||
|
const traceId = `walletPayInvoice:${requestId}`;
|
||||||
|
const log = (phase, extra = null) => {
|
||||||
|
if (extra === null) {
|
||||||
|
console.log(`[Worker] ${traceId} +${Date.now() - startedAt}ms ${phase}`);
|
||||||
|
} else {
|
||||||
|
console.log(`[Worker] ${traceId} +${Date.now() - startedAt}ms ${phase}`, extra);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
try {
|
try {
|
||||||
await ensureDirectWalletLoaded();
|
await ensureDirectWalletLoaded();
|
||||||
|
|
||||||
@@ -4580,6 +4590,11 @@ async function handleWalletPayInvoice(requestId, invoice, mint, port) {
|
|||||||
? [requestedMint, ...knownMints.filter((m) => m !== requestedMint)]
|
? [requestedMint, ...knownMints.filter((m) => m !== requestedMint)]
|
||||||
: [...knownMints].sort((a, b) => getProofTotal(directProofStore[b]) - getProofTotal(directProofStore[a]));
|
: [...knownMints].sort((a, b) => getProofTotal(directProofStore[b]) - getProofTotal(directProofStore[a]));
|
||||||
|
|
||||||
|
log('mint-selection', {
|
||||||
|
requestedMint: requestedMint || null,
|
||||||
|
orderedMints: orderedMints.map((m) => ({ mint: m, proofBalance: getProofTotal(directProofStore[m]) }))
|
||||||
|
});
|
||||||
|
|
||||||
let payResult = null;
|
let payResult = null;
|
||||||
let usedMint = null;
|
let usedMint = null;
|
||||||
let paidAmountSats = 0;
|
let paidAmountSats = 0;
|
||||||
@@ -4588,7 +4603,11 @@ async function handleWalletPayInvoice(requestId, invoice, mint, port) {
|
|||||||
for (const mintUrl of orderedMints) {
|
for (const mintUrl of orderedMints) {
|
||||||
try {
|
try {
|
||||||
const proofs = Array.isArray(directProofStore[mintUrl]) ? directProofStore[mintUrl] : [];
|
const proofs = Array.isArray(directProofStore[mintUrl]) ? directProofStore[mintUrl] : [];
|
||||||
|
const proofBalance = getProofTotal(proofs);
|
||||||
|
log('mint attempt:start', { mintUrl, proofBalance, proofCount: proofs.length });
|
||||||
|
|
||||||
if (proofs.length === 0) {
|
if (proofs.length === 0) {
|
||||||
|
log('mint attempt:no-proofs', { mintUrl });
|
||||||
attemptErrors.push({ mint: mintUrl, message: 'No proofs available for mint' });
|
attemptErrors.push({ mint: mintUrl, message: 'No proofs available for mint' });
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
@@ -4597,44 +4616,60 @@ async function handleWalletPayInvoice(requestId, invoice, mint, port) {
|
|||||||
const meltQuote = await wallet.createMeltQuote(pr);
|
const meltQuote = await wallet.createMeltQuote(pr);
|
||||||
const quoteAmount = Number(meltQuote?.amount || 0);
|
const quoteAmount = Number(meltQuote?.amount || 0);
|
||||||
const quoteFeeReserve = Number(meltQuote?.fee_reserve || 0);
|
const quoteFeeReserve = Number(meltQuote?.fee_reserve || 0);
|
||||||
|
// The mint's quoted fee_reserve is an estimate; the actual melt fee can be
|
||||||
|
// slightly higher, causing "not enough inputs provided for melt" errors.
|
||||||
|
// Add a safety buffer — any overpayment is returned as change proofs.
|
||||||
|
const FEE_RESERVE_BUFFER_SATS = 10;
|
||||||
const required = quoteAmount + quoteFeeReserve;
|
const required = quoteAmount + quoteFeeReserve;
|
||||||
|
const sendAmount = required + FEE_RESERVE_BUFFER_SATS;
|
||||||
|
log('mint attempt:quote', { mintUrl, quoteAmount, quoteFeeReserve, required, sendAmount });
|
||||||
|
|
||||||
if (!Number.isFinite(required) || required <= 0) {
|
if (!Number.isFinite(required) || required <= 0) {
|
||||||
|
log('mint attempt:invalid-quote', { mintUrl, required });
|
||||||
attemptErrors.push({ mint: mintUrl, message: 'Invalid melt quote amount' });
|
attemptErrors.push({ mint: mintUrl, message: 'Invalid melt quote amount' });
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
const proofBalance = getProofTotal(proofs);
|
if (proofBalance < sendAmount) {
|
||||||
if (proofBalance < required) {
|
log('mint attempt:insufficient', { mintUrl, proofBalance, sendAmount });
|
||||||
attemptErrors.push({ mint: mintUrl, message: `Insufficient funds on mint (${proofBalance} sats)` });
|
attemptErrors.push({ mint: mintUrl, message: `Insufficient funds on mint (${proofBalance} sats)` });
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
const sendResult = await wallet.send(required, proofs);
|
const sendResult = await wallet.send(sendAmount, proofs);
|
||||||
const keep = Array.isArray(sendResult?.keep) ? sendResult.keep : [];
|
const keep = Array.isArray(sendResult?.keep) ? sendResult.keep : [];
|
||||||
const sendProofs = Array.isArray(sendResult?.send) ? sendResult.send : [];
|
const sendProofs = Array.isArray(sendResult?.send) ? sendResult.send : [];
|
||||||
|
log('mint attempt:send-split', { mintUrl, keepCount: keep.length, sendCount: sendProofs.length, sendAmount });
|
||||||
|
|
||||||
const meltResult = await wallet.meltProofs(meltQuote, sendProofs);
|
const meltResult = await wallet.meltProofs(meltQuote, sendProofs);
|
||||||
const change = Array.isArray(meltResult?.change) ? meltResult.change : [];
|
const change = Array.isArray(meltResult?.change) ? meltResult.change : [];
|
||||||
upsertMintProofs(mintUrl, [...keep, ...change]);
|
upsertMintProofs(mintUrl, [...keep, ...change]);
|
||||||
|
|
||||||
const changeAmount = getProofTotal(change);
|
const changeAmount = getProofTotal(change);
|
||||||
const spentAmount = Math.max(0, required - changeAmount);
|
const spentAmount = Math.max(0, sendAmount - changeAmount);
|
||||||
paidAmountSats = Number.isFinite(quoteAmount) && quoteAmount > 0
|
paidAmountSats = Number.isFinite(quoteAmount) && quoteAmount > 0
|
||||||
? quoteAmount
|
? quoteAmount
|
||||||
: spentAmount;
|
: spentAmount;
|
||||||
|
|
||||||
|
log('mint attempt:success', { mintUrl, paidAmountSats, changeAmount, spentAmount, preimage: meltResult?.preimage || null });
|
||||||
|
|
||||||
usedMint = mintUrl;
|
usedMint = mintUrl;
|
||||||
payResult = meltResult;
|
payResult = meltResult;
|
||||||
break;
|
break;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
log('mint attempt:error', { mintUrl, message: error?.message || String(error) });
|
||||||
attemptErrors.push({ mint: mintUrl, message: error?.message || String(error) });
|
attemptErrors.push({ mint: mintUrl, message: error?.message || String(error) });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!payResult || !usedMint) {
|
if (!payResult || !usedMint) {
|
||||||
const detail = attemptErrors.map((entry) => `${entry.mint || 'unknown'}: ${entry.message}`).join(' | ');
|
const detail = attemptErrors.map((entry) => `${entry.mint || 'unknown'}: ${entry.message}`).join(' | ');
|
||||||
|
log('all-mints-failed', { attemptErrors });
|
||||||
throw new Error(`Failed to pay invoice. ${detail || 'No mint could complete melt'}`);
|
throw new Error(`Failed to pay invoice. ${detail || 'No mint could complete melt'}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
log('paid', { usedMint, paidAmountSats, preimage: payResult?.preimage || null });
|
||||||
|
|
||||||
const tx = appendDirectTransaction({
|
const tx = appendDirectTransaction({
|
||||||
type: 'pay',
|
type: 'pay',
|
||||||
amount: Number.isFinite(paidAmountSats) && paidAmountSats > 0
|
amount: Number.isFinite(paidAmountSats) && paidAmountSats > 0
|
||||||
@@ -4680,6 +4715,7 @@ async function handleWalletPayInvoice(requestId, invoice, mint, port) {
|
|||||||
}
|
}
|
||||||
})();
|
})();
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
log('error', { message: error?.message || String(error), stack: error?.stack || null });
|
||||||
port.postMessage({ type: 'response', requestId, error: error.message });
|
port.postMessage({ type: 'response', requestId, error: error.message });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -6156,7 +6192,6 @@ async function handlePublish(requestId, event, port) {
|
|||||||
// temporary NDKRelaySet. NDK's fromRelayUrls uses pool.useTemporaryRelay
|
// temporary NDKRelaySet. NDK's fromRelayUrls uses pool.useTemporaryRelay
|
||||||
// so hundreds of broadcast relays connect transiently without polluting
|
// so hundreds of broadcast relays connect transiently without polluting
|
||||||
// the read subscription pool.
|
// the read subscription pool.
|
||||||
let targetRelaySet = null;
|
|
||||||
const activeBroadcastUrls = getActiveBroadcastUrls();
|
const activeBroadcastUrls = getActiveBroadcastUrls();
|
||||||
const isBroadcast = wantsBroadcastRelays(event) && activeBroadcastUrls.length > 0;
|
const isBroadcast = wantsBroadcastRelays(event) && activeBroadcastUrls.length > 0;
|
||||||
let outboxWriteUrls = [];
|
let outboxWriteUrls = [];
|
||||||
@@ -6165,30 +6200,24 @@ async function handlePublish(requestId, event, port) {
|
|||||||
outboxWriteUrls = Array.from(relayTypes.entries())
|
outboxWriteUrls = Array.from(relayTypes.entries())
|
||||||
.filter(([, t]) => t === 'write' || t === 'both')
|
.filter(([, t]) => t === 'write' || t === 'both')
|
||||||
.map(([url]) => url);
|
.map(([url]) => url);
|
||||||
const allUrls = [...new Set([...outboxWriteUrls, ...activeBroadcastUrls])];
|
|
||||||
if (NDKRelaySet?.fromRelayUrls) {
|
|
||||||
targetRelaySet = NDKRelaySet.fromRelayUrls(allUrls, ndk);
|
|
||||||
console.log(`[Worker] Broadcasting to ${allUrls.length} relays ` +
|
|
||||||
`(${activeBroadcastUrls.length} broadcast + ${outboxWriteUrls.length} outbox, ` +
|
|
||||||
`${skippedRelayUrls.size} skipped)`);
|
|
||||||
} else {
|
|
||||||
console.warn('[Worker] NDKRelaySet.fromRelayUrls unavailable, falling back to default outbox');
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
const allBroadcastUrls = [...new Set([...outboxWriteUrls, ...activeBroadcastUrls])];
|
||||||
|
|
||||||
// --- Live broadcast progress streaming ------------------------------------
|
// --- Live broadcast progress streaming ------------------------------------
|
||||||
// NDK's NDKEvent emits 'relay:published' and 'relay:publish:failed' per
|
// NDK's NDKEvent emits 'relay:published' and 'relay:publish:failed' per
|
||||||
// relay as each completes (see ndk-core.bundle.js:8530 and :8552). We
|
// relay as each completes. We attach listeners BEFORE publish() and stream
|
||||||
// attach listeners BEFORE publish() and stream progress to all pages via
|
// progress to all pages via broadcast() so footers can show the count.
|
||||||
// broadcast() so footers can show "📡 42/150 relays · relay.example.com".
|
|
||||||
const publishSessionId = `pub_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`;
|
const publishSessionId = `pub_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`;
|
||||||
const publishedSoFar = new Set();
|
const publishedSoFar = new Set();
|
||||||
const failedSoFar = new Set();
|
const failedSoFar = new Set(); // explicit rejections (auth-required, blocked, etc.)
|
||||||
const totalTarget = targetRelaySet && targetRelaySet.relays
|
const timedOutSoFar = new Set(); // timeouts — NOT skip-marked (could be slow connect)
|
||||||
? targetRelaySet.relays.size
|
const totalTarget = isBroadcast ? allBroadcastUrls.length : 0;
|
||||||
: (targetRelaySet ? (targetRelaySet.size || 0) : 0);
|
|
||||||
|
|
||||||
if (isBroadcast && totalTarget > 0) {
|
if (isBroadcast && totalTarget > 0) {
|
||||||
|
console.log(`[Worker] Broadcasting to ${allBroadcastUrls.length} relays ` +
|
||||||
|
`(${activeBroadcastUrls.length} broadcast + ${outboxWriteUrls.length} outbox, ` +
|
||||||
|
`${skippedRelayUrls.size} skipped)`);
|
||||||
|
|
||||||
broadcast({
|
broadcast({
|
||||||
type: 'broadcastProgress',
|
type: 'broadcastProgress',
|
||||||
sessionId: publishSessionId,
|
sessionId: publishSessionId,
|
||||||
@@ -6212,7 +6241,7 @@ async function handlePublish(requestId, event, port) {
|
|||||||
phase: 'progress',
|
phase: 'progress',
|
||||||
total: totalTarget,
|
total: totalTarget,
|
||||||
successful: publishedSoFar.size,
|
successful: publishedSoFar.size,
|
||||||
failed: failedSoFar.size,
|
failed: failedSoFar.size + timedOutSoFar.size,
|
||||||
latestRelay: url,
|
latestRelay: url,
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
@@ -6220,41 +6249,172 @@ async function handlePublish(requestId, event, port) {
|
|||||||
ndkEvent.on('relay:publish:failed', (relay, err) => {
|
ndkEvent.on('relay:publish:failed', (relay, err) => {
|
||||||
const url = relay?.url || String(relay || '');
|
const url = relay?.url || String(relay || '');
|
||||||
if (!url) return;
|
if (!url) return;
|
||||||
failedSoFar.add(url);
|
const errMsg = err?.message || String(err) || '';
|
||||||
|
const isTimeout = /timeout/i.test(errMsg);
|
||||||
|
if (isTimeout) {
|
||||||
|
timedOutSoFar.add(url);
|
||||||
|
} else {
|
||||||
|
failedSoFar.add(url);
|
||||||
|
}
|
||||||
broadcast({
|
broadcast({
|
||||||
type: 'broadcastProgress',
|
type: 'broadcastProgress',
|
||||||
sessionId: publishSessionId,
|
sessionId: publishSessionId,
|
||||||
phase: 'progress',
|
phase: 'progress',
|
||||||
total: totalTarget,
|
total: totalTarget,
|
||||||
successful: publishedSoFar.size,
|
successful: publishedSoFar.size,
|
||||||
failed: failedSoFar.size,
|
failed: failedSoFar.size + timedOutSoFar.size,
|
||||||
latestRelay: url,
|
latestRelay: url,
|
||||||
latestError: err?.message || String(err),
|
latestError: errMsg,
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// Publish to relays (with broadcast relay set if broadcasting, else default outbox)
|
// --- Batched publishing for broadcasts ------------------------------------
|
||||||
const relaySet = await ndkEvent.publish(targetRelaySet);
|
// Browsers can only handle ~50-100 simultaneous WebSocket connections.
|
||||||
|
// Publishing to 600+ relays in one shot causes most connections to queue
|
||||||
|
// and time out before getting a slot. We split the relay URLs into batches
|
||||||
|
// of 25 and publish to each batch sequentially, keeping the number of
|
||||||
|
// simultaneous connections manageable.
|
||||||
|
const BATCH_SIZE = 25;
|
||||||
|
const PER_RELAY_TIMEOUT_MS = 10000; // 10s per relay within a batch
|
||||||
|
|
||||||
// Final broadcast progress message with complete results.
|
let relaySet = null;
|
||||||
if (isBroadcast && totalTarget > 0) {
|
if (isBroadcast && totalTarget > 0 && NDKRelaySet?.fromRelayUrls) {
|
||||||
|
// --- Phase 1: Publish to the user's standard outbox relays first ---
|
||||||
|
// This ensures the post lands on the user's primary relays quickly,
|
||||||
|
// and lets us send the response back to the page so the composer
|
||||||
|
// clears immediately. The broadcast relay batches run afterward.
|
||||||
|
if (outboxWriteUrls.length > 0) {
|
||||||
|
console.log(`[Worker] Phase 1: Publishing to ${outboxWriteUrls.length} outbox relays first`);
|
||||||
|
try {
|
||||||
|
const outboxSet = NDKRelaySet.fromRelayUrls(outboxWriteUrls, ndk);
|
||||||
|
await ndkEvent.publish(outboxSet, 10000, 1);
|
||||||
|
} catch (outboxErr) {
|
||||||
|
console.warn('[Worker] Outbox publish error (expected if some relays fail):', outboxErr?.message || outboxErr);
|
||||||
|
}
|
||||||
|
console.log(`[Worker] Phase 1 complete: ${publishedSoFar.size} outbox relays succeeded`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Send response back to the page NOW so the composer clears ---
|
||||||
|
// The post is already on the user's outbox relays. The broadcast
|
||||||
|
// batches continue in the background.
|
||||||
|
const earlyRelayResults = {
|
||||||
|
successful: Array.from(publishedSoFar),
|
||||||
|
failed: [],
|
||||||
|
};
|
||||||
|
port.postMessage({
|
||||||
|
type: 'response',
|
||||||
|
requestId: requestId,
|
||||||
|
data: {
|
||||||
|
success: true,
|
||||||
|
relayResults: earlyRelayResults,
|
||||||
|
totalRelays: earlyRelayResults.successful.length,
|
||||||
|
}
|
||||||
|
});
|
||||||
|
console.log('[Worker] Sent early publish response — composer can clear now');
|
||||||
|
|
||||||
|
// --- Phase 2: Broadcast to remaining relays in batches (background) ---
|
||||||
|
// Only publish to relays that aren't already in the outbox set
|
||||||
|
// (those were already published in Phase 1).
|
||||||
|
const broadcastOnlyUrls = activeBroadcastUrls.filter(u => !outboxWriteUrls.includes(u));
|
||||||
|
if (broadcastOnlyUrls.length > 0) {
|
||||||
|
const batches = [];
|
||||||
|
for (let i = 0; i < broadcastOnlyUrls.length; i += BATCH_SIZE) {
|
||||||
|
batches.push(broadcastOnlyUrls.slice(i, i + BATCH_SIZE));
|
||||||
|
}
|
||||||
|
console.log(`[Worker] Phase 2: Broadcasting to ${broadcastOnlyUrls.length} relays in ${batches.length} batches of ${BATCH_SIZE}`);
|
||||||
|
|
||||||
|
for (let bi = 0; bi < batches.length; bi++) {
|
||||||
|
const batch = batches[bi];
|
||||||
|
const batchSet = NDKRelaySet.fromRelayUrls(batch, ndk);
|
||||||
|
if (batchSet?.relays) {
|
||||||
|
for (const relay of batchSet.relays) {
|
||||||
|
try {
|
||||||
|
if (relay.connectionTimeout !== undefined) {
|
||||||
|
relay.connectionTimeout = PER_RELAY_TIMEOUT_MS;
|
||||||
|
}
|
||||||
|
} catch (_) { /* relay may be read-only */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await ndkEvent.publish(batchSet, PER_RELAY_TIMEOUT_MS, 1);
|
||||||
|
} catch (batchErr) {
|
||||||
|
// Expected — some relays in this batch may fail.
|
||||||
|
}
|
||||||
|
console.log(`[Worker] Batch ${bi + 1}/${batches.length} complete — ` +
|
||||||
|
`${publishedSoFar.size}/${totalTarget} total succeeded, ` +
|
||||||
|
`${failedSoFar.size + timedOutSoFar.size} failed/timeout so far`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Final broadcast progress message with complete results.
|
||||||
|
const totalFailed = failedSoFar.size + timedOutSoFar.size;
|
||||||
broadcast({
|
broadcast({
|
||||||
type: 'broadcastProgress',
|
type: 'broadcastProgress',
|
||||||
sessionId: publishSessionId,
|
sessionId: publishSessionId,
|
||||||
|
eventId: ndkEvent.id || null,
|
||||||
|
eventKind: event.kind,
|
||||||
phase: 'done',
|
phase: 'done',
|
||||||
total: totalTarget,
|
total: totalTarget,
|
||||||
successful: publishedSoFar.size,
|
successful: publishedSoFar.size,
|
||||||
failed: failedSoFar.size,
|
failed: totalFailed,
|
||||||
latestRelay: null,
|
latestRelay: null,
|
||||||
|
relayUrls: Array.from(publishedSoFar),
|
||||||
});
|
});
|
||||||
// Clean up listeners so the NDKEvent can be garbage-collected.
|
|
||||||
try {
|
try {
|
||||||
ndkEvent.removeAllListeners('relay:published');
|
ndkEvent.removeAllListeners('relay:published');
|
||||||
ndkEvent.removeAllListeners('relay:publish:failed');
|
ndkEvent.removeAllListeners('relay:publish:failed');
|
||||||
} catch (_cleanupErr) {
|
} catch (_cleanupErr) { /* ignore */ }
|
||||||
// removeAllListeners may not exist on all EventEmitter impls; ignore.
|
|
||||||
|
// --- Failure marking (background, after all batches) ---
|
||||||
|
const newSkips = Array.from(failedSoFar)
|
||||||
|
.filter(url => broadcastRelayUrls.has(url) && !skippedRelayUrls.has(url));
|
||||||
|
if (newSkips.length > 0) {
|
||||||
|
for (const url of newSkips) {
|
||||||
|
skippedRelayUrls.set(url, {
|
||||||
|
reason: 'publish-rejected',
|
||||||
|
timestamp: Math.floor(Date.now() / 1000),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
scheduleSkipMarkSave();
|
||||||
|
console.log(`[Worker] Skip-marked ${newSkips.length} rejected broadcast relays ` +
|
||||||
|
`(${timedOutSoFar.size} timeouts excluded)`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Return early — we already sent the response above. The rest of
|
||||||
|
// handlePublish (relayResults, kind 10002 sync, etc.) is skipped for
|
||||||
|
// broadcasts since we've handled everything here.
|
||||||
|
return;
|
||||||
|
} else {
|
||||||
|
// Normal (non-broadcast) publish — await the full result as before.
|
||||||
|
try {
|
||||||
|
relaySet = await ndkEvent.publish(null, undefined, undefined);
|
||||||
|
} catch (publishError) {
|
||||||
|
console.warn('[Worker] publish() threw:', publishError?.message || publishError);
|
||||||
|
relaySet = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// For non-broadcast publishes, emit a simplified broadcastProgress 'done'
|
||||||
|
// event so post cards can still show the relay count (e.g., "3r - 1h").
|
||||||
|
// We use the relaySet result to count successful relays.
|
||||||
|
if (!isBroadcast) {
|
||||||
|
const successCount = relaySet ? relaySet.size : 0;
|
||||||
|
const successUrls = relaySet
|
||||||
|
? Array.from(relaySet).map(r => r?.url || '').filter(Boolean)
|
||||||
|
: [];
|
||||||
|
broadcast({
|
||||||
|
type: 'broadcastProgress',
|
||||||
|
sessionId: publishSessionId,
|
||||||
|
eventId: ndkEvent.id || null,
|
||||||
|
eventKind: event.kind,
|
||||||
|
phase: 'done',
|
||||||
|
total: successCount,
|
||||||
|
successful: successCount,
|
||||||
|
failed: 0,
|
||||||
|
latestRelay: null,
|
||||||
|
relayUrls: successUrls,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get detailed relay results
|
// Get detailed relay results
|
||||||
@@ -6272,32 +6432,49 @@ async function handlePublish(requestId, event, port) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check which connected relays didn't accept it
|
// For broadcast publishes, use the live-tracked sets from the
|
||||||
for (const relay of ndk.pool.relays.values()) {
|
// relay:published / relay:publish:failed events as the source of truth
|
||||||
if (relay.status >= 5 && !relayResults.successful.includes(relay.url)) {
|
// (they cover temporary relays that may not be in ndk.pool). For normal
|
||||||
relayResults.failed.push(relay.url);
|
// publishes, fall back to the pool-scan logic.
|
||||||
|
if (isBroadcast) {
|
||||||
|
relayResults.successful = Array.from(publishedSoFar);
|
||||||
|
// Include both explicit rejections and timeouts in the failed list
|
||||||
|
// for display purposes, but keep them distinguishable for skip-marking.
|
||||||
|
relayResults.failed = [...Array.from(failedSoFar), ...Array.from(timedOutSoFar)];
|
||||||
|
} else {
|
||||||
|
// Check which connected relays didn't accept it
|
||||||
|
for (const relay of ndk.pool.relays.values()) {
|
||||||
|
if (relay.status >= 5 && !relayResults.successful.includes(relay.url)) {
|
||||||
|
relayResults.failed.push(relay.url);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
console.log('[Worker] ✅ Published to relays:', relayResults.successful);
|
console.log('[Worker] ✅ Published to relays:', relayResults.successful.length, relayResults.successful.slice(0, 10), relayResults.successful.length > 10 ? `... +${relayResults.successful.length - 10} more` : '');
|
||||||
console.log('[Worker] ❌ Failed relays:', relayResults.failed);
|
console.log('[Worker] ❌ Failed relays:', relayResults.failed.length);
|
||||||
|
|
||||||
// --- Failure marking for broadcast relays ---------------------------------
|
// --- Failure marking for broadcast relays ---------------------------------
|
||||||
// Persist skip-marks for broadcast relays that failed this publish so
|
// Persist skip-marks ONLY for broadcast relays that EXPLICITLY rejected
|
||||||
// future publishes skip them. Debounced via scheduleSkipMarkSave() so a
|
// the publish (auth-required, blocked, invalid, etc.). We do NOT skip-mark
|
||||||
// single post that fails on many relays produces one kind 10088 republish.
|
// relays that timed out — with hundreds of relays, a timeout often means
|
||||||
|
// the relay was slow to connect, not that it's broken. The relay:publish:failed
|
||||||
|
// listener above separates timeouts (timedOutSoFar) from explicit rejections
|
||||||
|
// (failedSoFar) by checking the error message for "timeout".
|
||||||
if (isBroadcast) {
|
if (isBroadcast) {
|
||||||
const newSkips = relayResults.failed
|
const newSkips = Array.from(failedSoFar)
|
||||||
.filter(url => broadcastRelayUrls.has(url) && !skippedRelayUrls.has(url));
|
.filter(url => broadcastRelayUrls.has(url) && !skippedRelayUrls.has(url));
|
||||||
if (newSkips.length > 0) {
|
if (newSkips.length > 0) {
|
||||||
for (const url of newSkips) {
|
for (const url of newSkips) {
|
||||||
skippedRelayUrls.set(url, {
|
skippedRelayUrls.set(url, {
|
||||||
reason: 'publish-failed',
|
reason: 'publish-rejected',
|
||||||
timestamp: Math.floor(Date.now() / 1000),
|
timestamp: Math.floor(Date.now() / 1000),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
scheduleSkipMarkSave();
|
scheduleSkipMarkSave();
|
||||||
console.log(`[Worker] Skip-marked ${newSkips.length} failed broadcast relays`);
|
console.log(`[Worker] Skip-marked ${newSkips.length} rejected broadcast relays ` +
|
||||||
|
`(${timedOutSoFar.size} timeouts excluded from skip-marking)`);
|
||||||
|
} else if (timedOutSoFar.size > 0) {
|
||||||
|
console.log(`[Worker] ${timedOutSoFar.size} broadcast relays timed out (not skip-marked — will retry next publish)`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+1
-5
@@ -833,12 +833,8 @@ import { initPostCards } from './js/post-interactions2.mjs';
|
|||||||
: '';
|
: '';
|
||||||
spanBroadcastStatus.textContent = `📡 ${d.successful}/${d.total} relays · ${shortRelay}`;
|
spanBroadcastStatus.textContent = `📡 ${d.successful}/${d.total} relays · ${shortRelay}`;
|
||||||
} else if (d.phase === 'done') {
|
} else if (d.phase === 'done') {
|
||||||
|
// Keep the result displayed until the next publish overwrites it.
|
||||||
spanBroadcastStatus.textContent = `✅ ${d.successful}/${d.total} relays` + (d.failed > 0 ? ` (${d.failed} failed)` : '');
|
spanBroadcastStatus.textContent = `✅ ${d.successful}/${d.total} relays` + (d.failed > 0 ? ` (${d.failed} failed)` : '');
|
||||||
setTimeout(() => {
|
|
||||||
if (spanBroadcastStatus.textContent.startsWith('✅')) {
|
|
||||||
spanBroadcastStatus.textContent = '';
|
|
||||||
}
|
|
||||||
}, 10000);
|
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user