Views
No views yet
<|startofprev|> renamed to <|prev|>prompt field to vLLM's
/v1/audio/transcriptions on a Whisper model corrupts decoding — degraded
output on short audio, and on audio longer than 30 s the response silently
collapses to only the final chunk's text.<|startofprev|> to <|prev|> (applied consistently in
tokenizer.json, tokenizer_config.json, added_tokens.json and
special_tokens_map.json). Vocabulary size and every id are unchanged.config.json: vLLM
resolves a remote --tokenizer repo through AutoConfig and refuses to
start when the repo lacks a config.json with a model_type key (local
directory mounts skip that check). No weights are read from this repo.1decoder_text = (
2 f"<|prev|>{request_prompt}" if request_prompt else ""
3) + f"<|startoftranscript|><|{language}|><|{task_type}|><|notimestamps|>"<|prev|> is not in the Whisper vocabulary, so the stock tokenizer splits
it into six ordinary text tokens (<, |, pre, v, |, >) placed in
front of <|startoftranscript|>, which corrupts decoder conditioning. The
real prompt-conditioning token is <|startofprev|> (id 50362).<|prev|> string tokenizes straight to
id 50362 — the genuine sot_prev embedding — so the prompt field behaves
as intended again, with zero code changes to vLLM. Requests without a
prompt are byte-for-byte unaffected.1vllm serve openai/whisper-large-v3-turbo \
2 --tokenizer ardge-hf/whisper-large-v3-turbo-tokenizer-prevfixprompt field works:1curl http://localhost:8000/v1/audio/transcriptions \
2 -F file=@audio.mp3 \
3 -F response_format=verbose_json \
4 --form-string 'prompt=這是繁體中文的語音轉錄。'vllm/vllm-openai:nightly (v0.26.1rc1, 2026-08): without the fix
the request above returns a single hallucinated segment; with it, the full
transcript comes back and the prompt bias takes effect (e.g. Mandarin output
switches from Simplified to Traditional script with punctuation).1from transformers import AutoTokenizer
2
3t = AutoTokenizer.from_pretrained("ardge-hf/whisper-large-v3-turbo-tokenizer-prevfix")
4assert t.encode("<|prev|>", add_special_tokens=False) == [50362]
5assert len(t) == 51866 # no phantom tokens added<|startofprev|> in the first place — is
vllm-project/vllm#53207.
Once your vLLM build contains it, drop the --tokenizer flag and go back to
the stock tokenizer.sot_prev is id 50362. The v2 family (large-v2, small, …) uses
id 50361 — regenerate per checkpoint rather than reusing this one.<|prev|>
text tokens are prepended server-side before any client-controlled content,
and even embedding a literal <|startofprev|> inside the prompt value
still collapses (measured). The tokenizer has to be swapped server-side.<|prev|>; with skip_special_tokens=True
(the transcription default) this is never visible.added_tokens_decoder tables of
whisper-large-v3-turbo
and
whisper-large-v3,
and the reference specials list in
openai/whisper tokenizer.py.