A tt-kernelv4 vLLM bundle that
runs WeiboAI/VibeThinker-3B — a Qwen2.5-3B-family
dense reasoning model (Qwen2ForCausalLM) — on a single Tenstorrent Blackhole p150a through
the Tenstorrent vLLM plugin. It ships no kernel cache and no weights: kernels JIT-compile at
first-run warmup and weights are pulled from the base repo. This is the generic tt_transformers
path — the registration references the upstream built-in QwenForCausalLM, so no custom adapter
is shipped.
That's the whole bringup. The tt CLI provisions the
entire bare-metal serving stack for you — there is no tt-metal to build by hand. From the bundle's
manifest it:
resolves the declared platform.ttnn range (>=0.72,<0.73) to a released ttnn wheel and
installs it, plus the matching sfpi JIT toolchain (so kernel compilation uses the toolchain
that ttnn version expects, not whatever is on the host);
installs the Tenstorrent vLLM fork + plugin (CUDA-free) and tt-kernel;
composes all of the above into a reusable serving environment under $TT_DATA_DIR/envs, keyed
by version so multiple models share one install rather than duplicating it;
hands off to tt-kernel, which selects that environment and launches the server.
First run JIT-compiles kernels and downloads the weights from WeiboAI/VibeThinker-3B; later runs
reuse both. The result is an OpenAI-compatible server on :8000 (model id
WeiboAI/VibeThinker-3B, aliases vibethinker-3b, vibethinker-3b-P150).
Manage the composed environments with tt env list / tt env gc.
Requirements
A Tenstorrent Blackhole card (this bundle targets a single p150a) with the driver loaded.
Linux x86_64 (ttnn/sfpi ship for that platform only).
The tt CLI. No hand-built tt-metal, no Docker, and no
scripts/install.sh step — tt owns provisioning.
Using the model
Native 131072 context; up to 32 concurrent requests (max_num_seqs=32). The model emits long
<think>…</think> chain-of-thought — use generous max_tokens and its recommended sampling
(temperature 0.6 / top_p 0.95 / top_k 40). bf8 weights + bf8 KV, traced decode, host sampling.
bash
1curl http://localhost:8000/v1/chat/completions \2 -H 'Content-Type: application/json'\3 -d '{"model":"vibethinker-3b","messages":[{"role":"user","content":"Prove that sqrt(2) is irrational."}],"max_tokens":4096,"temperature":0.6,"top_p":0.95}'
What's in this bundle
A single tt_kernel_manifest.json (tt-kernel schema v4, unified model+manifest). It declares the
platform envelope (platform.ttnn), the serving runtime (runtime.kind = vllm), the plugin
entrypoint (Qwen2ForCausalLM → models.tt_transformers.tt.generator_vllm:QwenForCausalLM), the
launch recipe, and the weights reference — from which tt-kernel renders the plugin's
vllm_metadata.json on pull. No precompiled kernels, no weights.
Advanced: serve without tt
You can still serve directly with tt-kernel if you manage the tt-metal + vLLM stack yourself (see
the tt-kernel docs and
tt-kernel instances). The tt path above automates exactly that setup.
