Pure quantization — zero fine-tuning. This is a direct quantization of the official Qwen3.5-35B-A3B weights. No merge, no fine-tune, no LoRA, no RLHF modifications. The original model's capabilities are preserved exactly as Qwen released them — only the numerical precision is optimized.

PPL 6.955 at 4.5× compression. HellaSwag 78.50% — identical to F16 baseline. Outperforms every public Qwen3.5-35B GGUF under 15 GiB on both perplexity and accuracy.

Also available: Alis Ultra Slim (12.87 GiB, PPL 7.01) — for tighter memory budgets.

Quick Start

bash

# Download
hf download avlp12/Qwen3.5-35B-A3B-Alis-Ultra-GGUF \
    Qwen3.5-35B-A3B-Alis-Ultra.gguf --local-dir ./model

# Chat
llama-cli -m ./model/Qwen3.5-35B-A3B-Alis-Ultra.gguf \
    --conversation -ngl 99

# Server
llama-server -m ./model/Qwen3.5-35B-A3B-Alis-Ultra.gguf \
    --host 0.0.0.0 --port 8080 -ngl 99 \
    --temp 0.6 --top-p 0.95 --top-k 20 \
    -fa on --jinja

Compatible with stock llama.cpp, llama-server, and any GGUF-compatible backend. No custom forks required.

Alis Ultra vs Ultra Slim — Which One?

	Ultra (this repo)	Ultra Slim (link)
Size	14.26 GiB (3.53 BPW)	12.87 GiB (3.19 BPW)
PPL	6.955 ± 0.045	7.009 ± 0.045
HellaSwag	78.50%	78.00%
Base type	Q3_K_M	Q2_K
Best for	24 GiB+ devices (M4 Pro/Max, RTX 4090)	16 GiB constrained (M4, RTX 4060 Ti 16GB)
Key advantage	F16-matching accuracy	Best quality under 13 GiB

The core difference: Ultra uses Q3_K_M as the base quantization type, which allows llama.cpp's imatrix to auto-promote sensitive tensors up to q4_K/q5_K — providing an extra quality buffer. Ultra Slim uses Q2_K as the base, keeping middle-layer gate/up experts at the K-quant floor. This saves 1.4 GiB but adds ~0.05 PPL.

Both variants share identical treatment of critical tensors: shared experts at Q8_0, attention at Q4_K/Q5_K, SSM at Q6_K, and ffn_down_exps at Q3_K.

Benchmarks

All measurements on M3 Ultra Mac Studio (512 GB, 80 GPU cores). PPL on wikitext-2-raw test set, context 512, 580 chunks. HellaSwag 0-shot, 400 tasks. All values directly measured by us on the same hardware.

Quality

Model	Size (GiB)	BPW	PPL	HellaSwag
F16 (baseline)	64.60	16.00	6.537	78.50%
Unsloth Q3_K_M (Dynamic)	15.22	3.77	6.779	78.50%
Alis Ultra	14.26	3.53	6.955	78.50%
Alis Ultra Slim	12.87	3.19	7.009	78.00%
APEX Mini	12.33	3.06	7.048	76.75%
Unsloth IQ2_XXS	9.91	2.46	7.519	77.00%

Key Takeaways

Alis Ultra matches F16 on HellaSwag (78.50%) — zero accuracy loss at 4.5× compression
Alis Ultra Slim beats APEX Mini on both PPL (7.009 vs 7.048) and HellaSwag (78.00% vs 76.75%)
Alis Ultra is 1 GiB smaller than Unsloth Q3_K_M Dynamic while matching it on HellaSwag

Speed (M3 Ultra, llama-bench)

Model	pp512 (tok/s)	tg128 (tok/s)
Alis Ultra (14.26 GiB)	2,239 ± 8.58	85.62 ± 1.22
Alis Ultra Slim (12.87 GiB)	2,204 ± 7.39	81.95 ± 0.76

Quantization Strategy

Tensor Classification & Layer Gradient

Based on APEX's MoE tensor role analysis and Unsloth's 121-configuration KL divergence study:

Tensor	Edge (L0-4, L35-39)	Middle (L5-34)
ffn_down_exps (most sensitive)	q3_K → imatrix auto q4_K/q5_K	q3_K → imatrix auto q4_K
ffn_gate/up_exps	q3_K	q2_K (imatrix managed)
Shared experts (every-token path)	Q8_0	Q8_0
Attention Q/K	q4_K	q4_K (imatrix promoted)
Attention V	q5_K	q5_K (imatrix promoted)
Attention O	q4_K	q4_K
SSM out	Q6_K	Q6_K
Embedding	Q4_K	—
Output	Q5_K	—

Design Principles

K-quant over IQ for MoE experts. Routed expert weights have near-Gaussian distributions (kurtosis 3.41). K-quant block-scaling outperforms IQ codebooks designed for heavy-tailed distributions.
Protect ffn_down, compress gate/up. ffn_down_exps is consistently the most sensitive expert tensor. We keep it at Q3_K+ while allowing gate/up to drop to Q2_K.
Edge-layer gradient. Layers 0-4 and 35-39 (nearest to embedding/output) get higher precision.
Shared experts are sacred. With kurtosis 13.10 (4× routed experts) and 100% activation rate, shared experts stay at Q8_0.

Imatrix

14,062 chunks from 76,447 calibration samples across 6 domains:

General instruction (Alpaca, 52K) — broad language coverage
Math reasoning (GSM8K, 7.4K) — numerical precision
Wikipedia (wikitext-2-raw train, ~15K) — factual knowledge
Korean language (1,000) — multilingual preservation
Code (500) — syntax structure
Tool-call JSON (200) — structured output accuracy

Vision Support

This GGUF contains text weights only (733 tensors). Qwen3.5-35B-A3B is a multimodal model, but the vision encoder must be loaded separately as an mmproj file.

Setup

bash

# 1. Download mmproj (choose one — F16 recommended for quality/size balance)
hf download unsloth/Qwen3.5-35B-A3B-GGUF mmproj-F16.gguf --local-dir ./model
# Alternatives: mmproj-BF16.gguf (903 MB) or mmproj-F32.gguf (1.79 GB)

# 2. Run with vision — Server mode
llama-server \
    -m ./model/Qwen3.5-35B-A3B-Alis-Ultra.gguf \
    --mmproj ./model/mmproj-F16.gguf \
    --host 0.0.0.0 --port 8080 -ngl 99 \
    -fa on --jinja

# 3. Run with vision — Interactive CLI
llama-mtmd-cli \
    -m ./model/Qwen3.5-35B-A3B-Alis-Ultra.gguf \
    --mmproj ./model/mmproj-F16.gguf \
    -ngl 99 --jinja

Memory Impact

The mmproj-F16 adds ~899 MiB to VRAM usage. Total for Ultra + mmproj-F16: ~15.2 GiB.

Compatibility Notes

llama.cpp: Full support via --mmproj flag (llama-server, llama-mtmd-cli)
Ollama: Not currently supported — Qwen3.5 GGUF requires separate mmproj files which Ollama does not handle
LM Studio: Check for Qwen3.5 VLM support in your version
mmproj files are interchangeable across all Qwen3.5-35B-A3B quantizations (Alis, Unsloth, APEX, etc.)

Thinking Mode

Qwen3.5 supports thinking/non-thinking. To disable:

--chat-template-kwargs '{"enable_thinking":false}'

Technical Details

Base Model: Qwen/Qwen3.5-35B-A3B (35B total, 3B active per token, 256 experts, 8 active + 1 shared)
Architecture: Qwen3.5-MoE with Gated Delta Networks
Quantization Tool: llama.cpp build 8770, --tensor-type per-layer overrides
Imatrix: 14,062 chunks, PPL 4.5318 on calibration data
Source: F16 GGUF → converted from HuggingFace BF16

Acknowledgments

Qwen Team · APEX/LocalAI (MoE tensor classification) · Unsloth (KLD study) · llama.cpp · ik_llama.cpp (IQK research)

License

Apache 2.0 — same as the base model.