CJK transcripts collapse into a single caption cue (wordsToCues whitespace heuristic)

[miguel-heygen]

Describe the bug

Phrase-level CJK transcripts collapse into a single caption cue. A 3-phrase Chinese
transcript comes out as 1 cue; the same 3-phrase English transcript comes out as 3.

wordsToCues decides whether its input is already grouped into phrases by testing for
internal whitespace (packages/cli/src/whisper/normalize.ts:373):

const preGrouped = opts.preGrouped ?? words.some((w) => /\s/.test(w.text.trim()));
if (preGrouped) return entriesToCues(words);

Chinese, Japanese and Thai do not put spaces between words, so for those scripts the
test is always false. Phrase-level entries are then treated as individual words and
re-grouped into one cue covering the whole transcript.

The failure is silent. There is no warning, and the export succeeds. The user finds out
by watching the captions.

Steps to reproduce

import { wordsToCues } from "./packages/cli/src/whisper/normalize.ts";

const en = [
  { text: "the quick brown fox", start: 0, end: 2 },
  { text: "jumps over the dog",  start: 2, end: 4 },
  { text: "and runs away fast",  start: 4, end: 6 },
];
const zh = [
  { text: "这是第一个句子", start: 0, end: 2 },
  { text: "这是第二个句子", start: 2, end: 4 },
  { text: "这是第三个句子", start: 4, end: 6 },
];

console.log(wordsToCues(en).length);                        // 3
console.log(wordsToCues(zh).length);                        // 1  <-- bug
console.log(wordsToCues(zh, { preGrouped: true }).length);  // 3

Actual output:

english  3 phrase entries in -> 3 cue(s) out
chinese  3 phrase entries in -> 1 cue(s) out
         collapsed into: "这是第一个句子这是第二个句子这是第三个句子"
chinese  with preGrouped forced -> 3 cue(s)

Expected behavior

Three phrase-level entries produce three cues, in any script.

Actual behavior

Three Chinese entries produce one cue containing the concatenated text of all three, with
the timing of the whole span. On a real transcript this means the entire clip becomes a
single caption.

Impact

This is not theoretical. Two separate projects hit it and each built their own workaround
rather than finding the flag:

• one fed whisper-cli segmented JSON in by hand;
• one had a 76-second clip collapse into a single segment and replaced our timing entirely
  with ffmpeg silencedetect plus character-proportional clause mapping.

The second is the tell. When someone reimplements caption timing with silencedetect
because ours produced one cue, the feature did not degrade, it stopped working for that
language.

Suggested fix

--preserve-cues already exists for this, and the comment above it in
packages/cli/src/commands/transcribe.ts:245 explicitly names the CJK case:

// --preserve-cues forces the same for an already-cued transcript.json whose
// entries have no internal whitespace (single-word or CJK captions), which
// the automatic whitespace heuristic in wordsToCues can't detect.

So the limitation is known and documented in the source. The problem is that it is opt-in
and undiscoverable: nothing in the output tells a CJK user that this flag is the reason
their captions are one block, and both reporters wrote their own pipeline instead.

Options, roughly in order of preference:

1. Stop inferring from whitespace. Whether entries are pre-grouped is better answered by
   their shape than their script, for example entry count relative to total duration, or
   average entry duration against a plausible per-word duration. That fixes CJK, Thai, and
   single-word English entries in one change.
2. Keep the whitespace test but add a CJK/Thai codepoint-range check alongside it.
3. At minimum, warn when the heuristic collapses N entries into 1, and name
   --preserve-cues in that warning. This is the cheap version and still removes the
   silent part.

A regression test should assert that phrase-level entries survive round-trip in a script
without spaces. Any test written in English passes today.

Environment

hyperframes  repo main
Node.js      v22.23.1 (darwin arm64)
Affected     packages/cli/src/whisper/normalize.ts:373 (wordsToCues)
Workaround   --preserve-cues