GLM-5.2-TR3-Vision combines the GLM-5.2 EXL3/Trellis TR3 3.0 bpw language checkpoint and quantized MTP-78 head with the GLM-5.2 MoonViT-3D vision tower and trained PatchMerger. It was live-tested through vLLM and a deployed Local Studio controller on four RTX PRO 6000 96 GB GPUs.

This is an experimental community composition, not an official Z.ai, Baseten, Brandon Music, Malaiwah, or VerdictAI release.

Source lineage

Original model and license: zai-org/GLM-5.2
EXL3/TR3 3.0 bpw text weights: brandonmusic/GLM-5.2-EXL3-TR3-3.0bpw at revision 677eaa98b41b9f06abaad03953659c2872c22ce3
Vision tower, projector, processor, and Glm5v configuration: baseten/GLM-5.2-Vision-NVFP4 at revision f6eab6117386a0c69152fdf272dc65bfd0254f9f
Quantized MTP-78 overlay: malaiwah/GLM-5.2-EXL3-TR3-MTP78
EXL3/Trellis runtime: verdictai/glm52-exl3-sparkinfer

The EXL3 language shards remain the routed-expert backbone. This repository adds vision_tower.safetensors and mm_projector.safetensors and exposes Glm5vForConditionalGeneration, with GlmMoeDsaForCausalLM as the nested language architecture. The vision payload contains 466,424,304 parameters (932,848,608 bytes).

Validated deployment profiles

Two separate recipes are saved on the deployed Local Studio controller. They intentionally use separate IDs so the fast text path is not mislabeled as multimodal.

`GLM-5.2-EXL3` — text-only performance profile

Setting	Value
Recipe ID	`glm-5.2-exl3`
Served model	`GLM-5.2-EXL3`
Checkpoint	`GLM-5.2-EXL3-TR3-3.0bpw`
Runtime image	`local/glm52-exl3-vision:v22-dcpstride`
Parallelism	TP4, DCP4, `ag_rs`
Context	400,000 tokens
KV cache	FP8 MLA, 878,822 tokens, 2.20x concurrency at 400K
Scheduler	async, 2,048 max batched tokens, 8 max sequences
Speculation	MTP-3, greedy, EXL3 TR3 3.0 bpw layer 78
Collectives	NCCL; B12X PCIe all-reduce and DCP A2A disabled
Graph mode	`FULL_AND_PIECEWISE` CUDA graphs
API features	`glm45` reasoning, `glm47` tools
Runtime fix	pitched DCP workspace validation for partial prefill batches

`EXL3-VISION` — multimodal profile

Setting	Value
Recipe ID	`glm-5.2-exl3-vision`
Served model	`GLM-5.2-VISION-EXL3`
Checkpoint	`GLM-5.2-EXL3-TR3-3.0bpw-Vision`
Architecture	`Glm5vForConditionalGeneration`
Vision	BF16 MoonViT-3D plus trained PatchMerger
Runtime image	`local/glm52-exl3-vision:v21`
Parallelism	TP4, DCP4, `ag_rs`
Context	400,000 tokens
KV cache	FP8 MLA, 672,296 tokens, 1.68x concurrency at 400K
Scheduler	async, 2,048 max batched tokens, 8 max sequences
Speculation	MTP-3, greedy, EXL3 TR3 3.0 bpw layer 78
Collectives	NCCL; B12X PCIe all-reduce and DCP A2A disabled
Graph mode	`FULL_AND_PIECEWISE` CUDA graphs
API features	images, `glm45` reasoning, `glm47` tools

Both profiles used the publisher-tested physical rank order 3,1,2,0. Every GPU remained capped at 300 W.

The text runtime adds a one-line layout validation fix for partial DCP prefill batches. The projection input was already compacted before torch.bmm; the validator now accepts the actual pitched stride of the 2,048-token workspace. This keeps DCP4, project-before-merge, and CUDA graphs enabled under concurrent mixed-length requests.

Measured speed: text-only

Hardware: 4 x RTX PRO 6000 96 GB, power limit 300 W/GPU. The canonical decode number is a 30-second sustained, concurrency-1 streaming window after warmup.

Test	Prompt tokens	Total context	Prefill tok/s	Decode tok/s
Sustained C1	72	long streaming output	—	81.665
Context probe	8,192	8,320	1,531.25	91.14
Context probe	32,768	32,896	1,897.41	96.51
Context probe	65,536	65,664	2,420.73	102.18
Context probe	131,072	131,200	2,425.10	96.30
Context probe	262,144	262,272	2,482.96	103.19
Exact limit	399,872	400,000	3,525.94	102.57

The text sustained run measured 63.87% MTP acceptance and 0.418 s TTFT, averaged 267.74 W/GPU, and peaked at 294 W.

Measured speed: vision-capable wrapper

The canonical C1 decoder remained above the 70 tok/s target with vision enabled. The long-context rows use a deliberately synthetic repeated-token prompt and 128 generated tokens. Their decode rates are not representative of normal chat because that prompt collapses MTP acceptance; the prefill rates are valid cold-cache measurements.

Test	Prompt tokens	Total context	Prefill tok/s	Decode tok/s	MTP acceptance
Sustained C1	72	long streaming output	—	70.698	60.32%
Cold context probe	8,192	8,320	1,804.34	21.08	1.08%
Cold context probe	131,072	131,200	925.51	23.68	5.50%
Exact cold limit	399,872	400,000	852.54	16.91	0.27%

The vision wrapper is therefore coherent and fast enough in normal decode, but its long text-only prefill path remains slower than the dedicated text recipe. Use GLM-5.2-EXL3 when no image input is needed.

Functional validation

The following passed through the OpenAI-compatible API:

algebra/reasoning response with a separate reasoning field;
exact glm47 function selection and JSON arguments;
tool-result continuation;
MTP-3 acceptance with the quantized layer-78 draft;
an actual 5120 x 2816 image payload through MoonViT and PatchMerger;
OCR of the central headline, subheadline, and warning text;
identification of the highlighted Workbench UI element when asked precisely;
exact 400,000-token requests for both profiles.

Recipe metadata alone was not treated as proof of vision support.

Zero to running: vision profile

Requirements:

Linux, Docker Engine, Compose, and NVIDIA Container Toolkit;
four Blackwell GPUs with enough aggregate VRAM;
roughly 300 GiB for weights plus runtime cache;
the Hugging Face hf CLI.

bash

curl -L \
  https://huggingface.co/0xSero/GLM-5.2-TR3-Vision/resolve/main/start.sh \
  -o start.sh
chmod +x start.sh
./start.sh

The launcher resumes the model download, builds the thin vision-plugin image on the pinned VerdictAI runtime, and serves GLM-5.2-VISION-EXL3 at http://127.0.0.1:8000.

The exact validated overrides are:

bash

export MAX_MODEL_LEN=400000
export GPU_MEMORY_UTILIZATION=0.96
export MAX_NUM_BATCHED_TOKENS=2048
export ENABLE_ASYNC_SCHEDULING=1
export VLLM_ENABLE_PCIE_ALLREDUCE=0
export VLLM_USE_B12X_DCP_A2A=0
export DCP_COMM_BACKEND=ag_rs
export KV_CACHE_DTYPE=fp8

CUDA graphs remain enabled; do not add eager-mode flags.

Text request

bash

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "GLM-5.2-VISION-EXL3",
    "messages": [{"role": "user", "content": "Explain why the sky is blue."}],
    "max_completion_tokens": 512
  }'

Image request

python

import base64
import requests

encoded = base64.b64encode(open("image.png", "rb").read()).decode()
response = requests.post(
    "http://127.0.0.1:8000/v1/chat/completions",
    json={
        "model": "GLM-5.2-VISION-EXL3",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {
                    "url": f"data:image/png;base64,{encoded}"
                }},
                {"type": "text", "text": "Describe the image precisely."},
            ],
        }],
        "max_completion_tokens": 1024,
    },
    timeout=600,
)
response.raise_for_status()
print(response.json())

Runtime assertions

text

Resolved architecture: Glm5vForConditionalGeneration
Resolved architecture: DeepSeekMTPModel
quantization=exl3
enforce_eager=False
Using ['PYNCCL'] all-reduce backends
EXL3 rank-sliced runtime planned: Trellis ... topk=8
GPU KV cache size: 672,296 tokens
Application startup complete

Limitations

The vision checkpoint requires the included vLLM plugin and pinned VerdictAI-derived runtime. Generic Transformers loading is unsupported.
Other GPU topologies are untested.
Vision is a graft, not a jointly retrained EXL3 multimodal checkpoint.
Long text-only prefill is slower in the VLM wrapper; use the dedicated text recipe when images are not required.
MTP acceptance depends strongly on the prompt and output distribution.
The PCIe link attached to GPU 0 emitted correctable AER events under heavy multi-GPU load on the validation host; no uncorrectable error or Xid was observed in these runs.

License

The source model repositories report the MIT license. Preserve upstream attribution and review each source model card before redistribution or use.

Acknowledgements

Z.AI for GLM-5.2 (MIT), the base of this vision adaptation.
baseten for the GLM-5.2-Vision-NVFP4 lineage this work builds on.
brandonmusic for the GLM-5.2-EXL3-TR3-3.0bpw TR3 method these artifacts relate to.

0xSero/GLM-5.2-TR3-Vision

GLM-5.2-TR3-Vision

Source lineage

Validated deployment profiles

GLM-5.2-EXL3 — text-only performance profile

EXL3-VISION — multimodal profile

Measured speed: text-only

Measured speed: vision-capable wrapper

Functional validation

Zero to running: vision profile

Text request

Image request

Runtime assertions

Limitations

License

Acknowledgements

0xSero/
GLM-5.2-TR3-Vision

`GLM-5.2-EXL3` — text-only performance profile

`EXL3-VISION` — multimodal profile