Skip to content

fix(CommandPalette): keep astral characters intact when truncating search results - #6817

Open
arsalan507 wants to merge 1 commit into
nuxt:v4from
arsalan507:fix/search-truncate-astral-characters
Open

fix(CommandPalette): keep astral characters intact when truncating search results#6817
arsalan507 wants to merge 1 commit into
nuxt:v4from
arsalan507:fix/search-truncate-astral-characters

Conversation

@arsalan507

Copy link
Copy Markdown
Contributor

🔗 Linked issue

None open — filing this directly as it's a small, self-contained bug.

❓ Type of change

  • 🐞 Bug fix (a non-breaking change that fixes an issue)

📚 Description

truncateHTMLFromStart in src/runtime/utils/search.ts walks the highlighted snippet in reverse one UTF-16 code unit at a time (html[i]). An astral character — emoji, most CJK extension blocks, many symbols — occupies two code units. When the truncation boundary lands between them, the surrogate pair is sliced in half and the output carries an unpaired surrogate, which renders as .

This is user-visible in CommandPalette and useContentSearch, where indexed content is more likely to contain emoji than a hand-written command label: a search result whose match sits after an emoji gets a at the truncation point.

Reproduction — sweeping filler lengths 1–40 of 😀 before the match, lone surrogates appear from filler length 7 onward and for every length after:

n=10, before: "...\uDE00😀😀😀😀😀😀<mark>match</mark>"
n=10, after:  "😀😀😀😀😀😀😀😀😀😀<mark>match</mark>"

Not a security issue. A lone surrogate can only render as a broken glyph or U+FFFD — it cannot manufacture HTML syntax, and no raw <, >, &, " or ' appears outside the <mark> tags the highlighter inserts itself. This is a correctness and appearance bug.

🔧 The fix

Two hunks:

  1. truncateHTMLFromStart — iterate with Array.from(html) so a surrogate pair is never split. The insideTag logic is untouched, since < and > are always single code units.
  2. The caller — measure the length budget in code points too, so it stays in the same units as the counter inside the function.

Worth calling out explicitly for review: hunk 2 is a consistency fix, not required to remove the lone surrogates — hunk 1 alone is sufficient. It does mean an emoji now counts as one character against the truncation budget rather than two, so emoji-containing snippets truncate slightly later than before. That seems closer to the intent of a visible-length budget, but happy to drop hunk 2 if you'd rather keep the diff minimal.

Behaviour for BMP-only content is byte-identical, which is why all pre-existing tests pass untouched.

✅ Verification

Added three tests to test/utils/search.spec.ts:

  • never splits an astral character, at any truncation boundary — the 1–40 sweep, asserting no lone surrogate at any length
  • keeps emoji before the match intact
  • still truncates a long prefix down to an ellipsis — guards against over-correcting into "never truncate"

Verified the first two fail against unmodified search.ts (expected [ 7, 8, 9, … ] to deeply equal []) and pass with the fix. The third passes both before and after by design.

test/utils/search.spec.ts            13 passed   (10 pre-existing, 3 new)
test/utils/ (both projects)         138 passed
CommandPalette + DashboardSearch +
ChatPalette + DashboardSearchButton 146 passed
eslint src/runtime/utils/search.ts test/utils/search.spec.ts   clean

📝 Checklist

  • I have linked an issue or discussion (n/a — none open)
  • I have added tests (if applicable)

@github-actions github-actions Bot added the v4 #4488 label Aug 9, 2026
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0093958a-a748-495a-ac3d-89bf03d5bcbf

📥 Commits

Reviewing files that changed from the base of the PR and between 6add5fb and 16016db.

📒 Files selected for processing (2)
  • src/runtime/utils/search.ts
  • test/utils/search.spec.ts

📝 Walkthrough

Walkthrough

HTML truncation now iterates over Unicode code points instead of UTF-16 code units. Highlight truncation budgets also use code-point counts. New tests verify that emoji and other astral characters remain intact, long prefixes receive an ellipsis, and highlighted matches remain present.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description check ✅ Passed The description clearly explains the astral-character truncation bug, the fix, added tests, and verification results.
Title check ✅ Passed The title clearly and concisely identifies the fix for preserving astral characters during search-result truncation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

src/runtime/utils/search.ts

Parsing error: Unexpected token {

test/utils/search.spec.ts

Parsing error: Unexpected token :


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Aug 9, 2026

Copy link
Copy Markdown
npm i https://pkg.pr.new/@nuxt/ui@6817

commit: 6e97151

@codspeed-hq

codspeed-hq Bot commented Aug 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing arsalan507:fix/search-truncate-astral-characters (6e97151) with v4 (6add5fb)

Open in CodSpeed

…arch results

`truncateHTMLFromStart` walked the highlighted snippet one UTF-16 code unit at
a time. An astral character (emoji, most CJK extension blocks) occupies two
code units, so a truncation boundary landing between them sliced the pair in
half and emitted an unpaired surrogate, rendering as `<?>`.

Iterate by code point instead, and measure the length budget the same way so
both sides stay in the same units. Behaviour for BMP-only content is unchanged.
@arsalan507
arsalan507 force-pushed the fix/search-truncate-astral-characters branch from 16016db to 6e97151 Compare August 9, 2026 14:09
IgorShevchik added a commit to bitrix24/b24ui that referenced this pull request Aug 11, 2026
…arch results (#365)

`truncateHTMLFromStart` walked the string one UTF-16 code unit at a time. An
astral character — emoji, most CJK extension blocks — occupies two, so a
truncation boundary landing between them sliced the pair in half and emitted an
unpaired surrogate that renders as `�`. Deterministic from 7 characters of
prefix onward, and every length after.

The fix iterates code points and measures the caller's budget in the same units,
so the two halves cannot drift apart. Submitted upstream first as nuxt/ui#6817,
so `src/runtime/utils/search.ts` stays in sync.

Test hardening on top: the original fixture used a single astral character
sitting inside both surrogate ranges, so an implementation with an off-by-one
range bound passed. The sweep now covers the range edges — U+10000, U+10FFFF,
U+20000 (low surrogate U+DC00), U+1F3FF (low surrogate U+DFFF, a skin-tone
modifier common in ordinary text) and U+1F600 — and counts surviving characters
rather than only scanning for lone surrogates, which an implementation that
deletes every astral character satisfied trivially. A new case places astral
content after the match, covering the caller's half of the fix; reverting it
alone previously left every test green.

Behaviour change worth noting in the changelog: an astral character now costs
one unit of the truncation budget instead of two, so 20 emoji before a
5-character match keep 13 where main kept 6 and half of a 7th. BMP-only content
is byte-identical.

Scope: #339 is scoped entirely to `truncateHTMLFromStart` and closes here. #362
(`<mark>` inserted inside an astral character, because Fuse's `indices` are
code-unit offsets too) and #364 (grapheme clusters) are separate defects.

Closes #339

Co-Authored-By: Arsalan Ahmed <arsalanahmed507@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v4 #4488

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant