You can deploy this model and start earning money today!

JANGQ-AI/
ZAYA1-VL-8B-JANGTQ_K

Name: ZAYA1-VL-8B-JANGTQ_K
Availability: InStock

Views

No views yet

⚡ All JANG models are meant to be run in vMLX

ZAYA1-VL-8B-JANGTQ_K

Zyphra/ZAYA1-VL-8B — 5.0 GB on disk — mixed-bit JANGTQ_K quantization. Same ZAYA1 LM trunk as the text bundle + Qwen2.5-VL vision tower + image-token LoRA gating.

Source: Zyphra/ZAYA1-VL-8B (80 LM layers alternating CCA + top-1 MoE; 32-layer ViT vision tower; per-expert LoRA fc1 / fc2 gated by image tokens)
Quantization: mixed-bit MXTQ on routed-expert MLP projections:
- down_proj: 4-bit / gate_proj: 2-bit / up_proj: 2-bit
- attention / shared expert / dense / embed / lm_head: 8-bit affine
- vision tower: 8-bit affine (preserved Qwen2.5-VL Conv3d patch embed axis layout)
- image-token LoRA blocks: fp16 passthrough
- norms / router / conv_qk / biases: fp16 / fp32 passthrough
Routed-expert layout: pre-stacked under zaya_block.experts.switch_mlp.{{gate, up, down}}_proj (JANGTQ-PRESTACK). Sidecar jangtq_runtime.safetensors ships both 2-bit + 4-bit codebooks.
Bundle size: ~5.0 GB on-disk
Runs on: M3 Max 32 GB+ / M4 / M5 / Mac Studio

Why mixed-bit?

Same rationale as the text bundle: ZAYA1 is top-1 MoE with MOD passthrough, so plain 2-bit collapses past ~2-3 k cumulative output tokens. JANGTQ_K recovers quality by spending 4 bits on down_proj (residual-stream output) and 2 bits on the SwiGLU-gated gate / up projections.

Loading (Python)

pip install jang-tools mlx-vlm

python

from jang_tools.load_jangtq_vlm import load_jangtq_vlm_model
from mlx_vlm import generate

model, processor = load_jangtq_vlm_model("JANGQ-AI/ZAYA1-VL-8B-JANGTQ_K")

prompt = processor.apply_chat_template(
    [{{"role": "user", "content": "What is 2 + 2?"}}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, processor, prompt, image=None,
               max_tokens=64, temperature=0.0).text)

load_jangtq_vlm_model auto-builds the MLX-VLM skeleton (vision tower

language model + image-token gating) and replaces routed-expert projections with TurboQuantLinear in the same step.

Validated runtime contract

80 LM layers + 32-layer ViT materialize; 40 routed-expert SwitchGLU instances hydrate at JANGTQ_K bit widths.
Capabilities: family=zaya1_vl, modality=vision, reasoning_parser=qwen3, tool_parser=zaya_xml, supports_thinking=True, think_in_template=False, cache_type=hybrid.
Text-path single-prompt smoke: "2+2=4", "Paris" — 57-108 tok/s on M5 Max.
Multi-turn smoke: 3-turn code+tests+README → 5,446 chars cumulative, completely clean (no loops / no repetition).

Runtime support matrix

Surface	Status
`jang-tools` Python (`load_jangtq_vlm_model`)	✅ working — this README's load snippet
`vmlx-swift-lm` Swift	✅ working — `Libraries/MLXVLM/Models/Zaya1VL.swift` + JANGTQ codebook dispatch

Credits

Quantization + MLX runtime: Jinho Jang (eric@jangq.ai)
Source model: Zyphra ZAYA1 team
License: Apache-2.0, inherited from upstream