Intended for the Colibro / qwen.exe runtime (OpenAI- and Anthropic-compatible APIs, web UI, CUDA expert tiering). This is not a drop-in replacement for vanilla Hugging Face Transformers inference.

Default API / UI model id when serving with Colibro:

qwen3.6-35B-COLIBRO-INT4-MTP8-MM-F16

License

Apache 2.0 (inherited from the base Qwen3.6-35B-A3B weights). See the base model LICENSE.

Format

Layout: Colibro MoE shards named colibro-NNNNN.safetensors (not model-*.safetensors)
Size: ~18.4 GB total (26 weight shards + tokenizer / config sidecars)
Architecture: Qwen3.6 sparse MoE — 40 layers, 256 experts, 8 experts/token, hidden size 2048

Precision policy

Component	Precision
Dense / routed expert matrices	INT4
MTP matrices (native Qwen3.6 head)	INT8
Embedding / output matrices	INT8
Vision tower	FP16 (`colibro_vision_dtype: f16`)
Norms, routing, other small tensors	FP32

Conversion retained MTP (~776 keys) and vision (~333 keys) from the source tree.

Key features

INT4 MoE main trunk for compact disk / RAM footprint
Native MTP speculative drafting (enable with COLI_MTP=1; drafts via COLI_MTP_DRAFTS=1..4 on the single-request path)
Multimodal image input (JPEG / PNG / WebP data URIs) via the resident FP16 vision tower
Sidecars: config.json, tokenizer files, chat_template.jinja.txt, experts.json (Atlas), preprocessor configs

Files (ballpark)

Pattern	Notes
`colibro-00000.safetensors` … `colibro-00025.safetensors`	Weight shards (~18.4 GB combined)
`config.json`	Model + vision config (`mtp_num_hidden_layers`, `vision_config`, …)
`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`	Tokenizer
`chat_template.jinja.txt`	Chat template
`experts.json`	Expert layout / Atlas asset
`preprocessor_config.json`, `video_preprocessor_config.json`	Vision preprocessing
`generation_config.json`	Sampling defaults

How to run (Colibro)

Clone / build Colibro, then point the Qwen runtime at a local copy of this repo. Project docs: README-QWEN.md and QUICKSTART-QWEN.md in the Colibro tree.

Download

powershell

huggingface-cli download Colibro/Qwen3.6-35B-INT4-MTP8-MM-F16 `
  --local-dir models\Qwen3.6-35B-A3B-colibro-int4-mtp8-mm-f16

Doctor / validate

powershell

python c\coli doctor `
  --runtime qwen `
  --model models\Qwen3.6-35B-A3B-colibro-int4-mtp8-mm-f16 `
  --auto-tier

Web UI + API

powershell

python c\coli web `
  --runtime qwen `
  --model models\Qwen3.6-35B-A3B-colibro-int4-mtp8-mm-f16 `
  --auto-tier `
  --gpu auto `
  --kv-slots 4 `
  --port 8000

Or on Windows: .\run_qwen_backend.bat (default model id qwen3.6-35B-COLIBRO-INT4-MTP8-MM-F16).

Optional MTP

powershell

$env:COLI_MTP = "1"
$env:COLI_MTP_DRAFTS = "1"

Intended use / limitations

Use with Colibro (coli run / coli web / coli serve / qwen.exe). Loading these shards in stock Transformers AutoModel is unsupported.
Multimodal requests use data-URI images; remote http(s) image URLs are rejected by the Colibro API.
Quantization trades some quality vs the original BF16 checkpoint for much smaller storage and faster load under Colibro’s tiered expert placement.

Attribution

Converted from Qwen/Qwen3.6-35B-A3B (Alibaba Qwen team) using Colibro’s convert_qwen_moe.py / initialize.py pipeline (dense INT4, experts INT4, MTP INT8, I/O INT8, vision FP16).

Please cite and respect the license of the base model when redistributing or publishing derivatives.