You can deploy this model and start earning money today!

infosave/
Qwen3.6-27Bcmf

Name: Qwen3.6-27Bcmf
Availability: InStock

Views

No views yet

Qwen3.6-27B — CMF q4tp (with the MTP head)

One 14.3 GB file, mmap-run, CPU/GPU/UMA from the same bytes.

Converted from Qwen/Qwen3.6-27B (Apache-2.0) to the CMF format in the q4tp layout — 4-bit tiles with predicted per-row scale ladders, ~7% smaller than plain tiled q4 at near-identical quality. The file keeps the model's multi-token-prediction head (MTP: 1 block(s) at load).

Architecture: dense hybrid — 64 layers (16 full attention + 48 GatedDeltaNet linear), hidden 5120, FFN 17408, GQA 24/4 heads, head_dim 256, vocab 248 320.

Requires cortiq 0.5.42+ (0.5.43 for the speed)

Earlier runtimes cannot load MTP-bearing files and decline the q4tp layout on the GPU graph. 0.5.42 is on crates.io and in the GitHub release binaries:

cargo install cortiq-cli
hf download infosave/Qwen3.6-27Bcmf qwen36-27b-q4tp.cmf --local-dir .   # pip install -U huggingface_hub

Run it

# GPU (discrete cards: all 14.3 GB of weights want to be resident)
CMF_GPU=1 CMF_GPU_VRAM_MB=20000 cortiq run qwen36-27b-q4tp.cmf

# OpenAI-compatible server (works with Cline/Roo-style clients)
CMF_GPU=1 CMF_GPU_VRAM_MB=20000 cortiq serve qwen36-27b-q4tp.cmf

Measured (cortiq 0.5.43, July 2026)

	hardware	decode	prompt ingest
Vulkan	RTX PRO 6000 Blackwell	45.4 tok/s (0.5.51; was 39.1)	~19 tok/s
CPU	48-core server (~55 GB/s RAM)	2.1 tok/s	10.4 tok/s

A dense model reads all 14.3 GB per token, so CPU decode is memory-bandwidth bound — budget roughly your RAM GB/s ÷ 14.3 tokens/s. Prompt ingest went from ~10 to ~19 tok/s in 0.5.43 and now beats the CPU path; CMF_BATCH_K still buys little on top for this model. O(1) attention (--o1 all, CMF_O1_GPU=1) loads and runs on the 16 full-attention layers; long-context numbers for this model are not yet published.

GPU output is token-for-token identical to the CPU at short context; on very long prompts greedy decoding may pick a different-but-coherent continuation at probability ties (float ordering, as with any GPU offload).

Full walkthrough: docs/QWEN36_MOE.md.

File

qwen36-27b-q4tp.cmf — 14 264 207 150 bytes, size-verified against the converter output after upload.
Reproduce (cortiq 0.5.42+): cortiq convert --model Qwen/Qwen3.6-27B --quant q4tp --output qwen36-27b-q4tp.cmf — streaming, peak disk = the output file. Converters before 0.5.42 drop the mtp.* tensors and produce a smaller, different file.

Weights derive from Qwen's release and remain under its Apache-2.0 terms. The CMF container and the cortiq runtime are Apache-2.0 as well (see the repository's LICENSE and PATENTS.md).