Streaming: duplicateFrameTolerance (#788) drops trailing words and inserts seam phantoms on repetitive speech

[saurabhav88]

Summary

duplicateFrameTolerance (introduced by #788 / 667181a3) makes sliding-window token dedup temporally
gated. On genuinely repetitive speech that gate changes --streaming output in two ways. Both reproduce on
stock main, both are isolated to that one constant by a one-line mutation, and both are regressions
against this repo's own batch decode of the same audio.

1. Trailing words dropped. Synthetic clip attached, so this one is fully reproducible from this issue.
2. Phantom fragment inserted at a chunk seam. Real dictation; described but not attachable.

We were about to ship #788 and pulled it back on this evidence. Not asking for a revert — happy to test a
patch.

Environment

• main @ c7b13a3942e79893f3bd76bfe3b1ed8d03e0bfc7, clean clone, swift build -c release
• macOS 15, Apple Silicon, Parakeet TDT v3
• fluidaudiocli transcribe <clip> --streaming

Repro 1 — trailing words dropped (clip attached)

14.5s, 16 kHz mono. Regenerate it exactly rather than relying on an attachment:

curl -s https://api.openai.com/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"tts-1-hd","voice":"echo","input":"This would go for self correction, this would go for building lists, this would go for places where we would want paragraphs. This would go for where we would make grammatical fixes. This would go for where we would remove filler words."}' \
  -o repro.mp3
afconvert -f WAVE -d LEI16@16000 -c 1 repro.mp3 repro.wav

The text repeats one phrase five times:

This would go for self correction, this would go for building lists, this would go for places where we
would want paragraphs. This would go for where we would make grammatical fixes. This would go for where
we would remove filler words.

batch (same binary, same clip — reference for what was spoken):
  ...This would go for where we would remove filler words,

--streaming on main @ c7b13a39:
  ...This would go for where we would remove.                 <-- "filler words" dropped

--streaming, same build, ONE line changed:
  ASRConstants.duplicateFrameTolerance = Int.max / 4          (gate effectively off)
  ...This would go for where we would remove filler words.    <-- restored

Deterministic: 3/3 runs per arm, byte-stable within each arm. The one-line mutation is the entire diff
between the two arms.

Useful detail for locating it: our fork carries a patch to TdtDecoderV3's last-chunk finalization
block (it may flush trailing punctuation but not lexical tokens). A build with that carry applied does
not drop the words, on this same clip, with the gate on. So repro 1 appears to be an interaction
between the new temporal gate and end-of-stream finalization, not the gate alone.

Repro 2 — phantom fragment (audio cannot be attached)

A 17.2s real dictation with the same repetitive shape. Counting occurrences of the repeated phrase:

build	count
batch (reference)	5	correct
main before #788	5	correct
main @ c7b13a39	6	phantom This would go for, inserted at a seam

The same one-line mutation restores 5. This one also reproduces with our fork's carries applied, so unlike
repro 1 it is not masked by the finalization patch.

The clip is a user dictation on a product whose privacy contract keeps dictated audio on the user's
machine, so it cannot be attached. The attached synthetic clip did not reproduce this second symptom —
only the first — so repro 1 is offered as the shareable case and repro 2 as corroboration.

Why we think it happens

The gate treats two matching token runs as the same acoustic event only when they fall within 2.0s of each
other in global frame time. That premise holds for a duplicate created by window overlap. But repetitive
speech produces genuinely matching runs at genuinely different times, and from timestamps alone the gate
cannot separate "the same words heard twice across a seam" from "the speaker said it again". When a real
seam duplicate lands more than 2s apart, it is kept.

Scope, measured

500 real dictations, pre-#788 pin vs c7b13a39:

• batch: 500/500 byte-identical — this path never reaches the changed method, so it is unaffected by
  construction and we confirmed it empirically
• streaming: 499/500 identical, 1 regressed (repro 2)

So the blast radius is narrow. It is also in the direction #788 exists to prevent, which is why we stopped.

---

Filed by EnviousWispr, a macOS dictation app built on FluidAudio Parakeet. Prior reports: #746, #840 —
thank you for the quick turnarounds on both.

[saurabhav88]

The clip for repro 2 is now downloadable, so this issue is no longer blocked on a description:

https://github.com/saurabhav88/FluidAudio/releases/tag/issue-855-repro2

sha256 80b9e000d45b0a6f8153423090f81eff88ab88d928b509f869875114d94abbe3, 17.197 s, 16 kHz mono. It is the exact buffer our decoder was fed. Batch gives 5 occurrences of the repeated phrase, streaming on main @ df1417ce gives 6, and PR #861 also gives 6.

Full measurements of #861 against 500 real dictations, including corrections to our own first pass, are on the PR: #861 (comment)

[Alex-Wengg]

could you rerun on main @saurabhav88 could you give the test one more try, i had pushed some other changes that might have resolved the 2nd issue.