mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-08-09 08:04:45 +00:00
Claude
66fb5d6f85
Replace content.split('\n') and paragraph.split(' ') with index-based
scanning so no intermediate List<String> or line-substring allocations
are created during paragraph/word tokenization.
Key changes to findTextSegments:
- Scan for '\n' via indexOf to determine line boundaries without
allocating a List<String> or any line substrings.
- Compute trimEnd boundary in-place (no trimEnd() String copy).
- For each line, do a first pass with isRegularByIndex() that checks
every word using character-level inspection without creating substrings.
If all words are plain text the entire paragraph is represented as a
single RegularTextSegment(content.substring(lineStart, actualEnd)),
avoiding M per-word String allocations and the later joinToString.
- Only when a paragraph contains special tokens (URLs, hashtags,
mentions, emoji, etc.) fall back to the existing wordIdentifier()
path, which produces individual Segment objects as before.
- Short-circuit the final map pass for single-segment paragraphs
(already optimal) to skip an unnecessary joinToString + object creation.
New helpers:
- isRegularByIndex(): character-level fast path that detects all
token types (http/https, lnbc, cashu, NIP-19, emoji, email, phone,
schemeless URL, EmojiCoder variation selectors) without creating
a substring. Returns true only when the word is definitely plain text.
- isArabicInRange(): RTL detection directly on the source string range,
replacing the substring-based isArabic().
Behaviour is identical: the same Segment types are produced in the same
order for every input, as validated by the existing test suite.
https://claude.ai/code/session_01NXsow7yLBd6ModmGSsqR9g