Gemma-4-26B-A4B-it — RAM 4bit (MLX)
A quantized build of
google/gemma-4-26B-A4B-it produced by
baa.ai.
Retains the full vision tower, unlike other pre-quantized MLX variants of this model.
| Property | Value |
|---|---|
| Size on disk | 14.63 GB |
| Format | MLX |
| Base model | google/gemma-4-26B-A4B-it |
| Vision tower | Retained |
Usage
1
2from mlx_vlm import load, generate
3
4
5
6model, processor = load("baa-ai/Gemma-4-26B-A4B-it-RAM-4bit-MLX")
7
8prompt = processor.tokenizer.apply_chat_template(
9
10 [{"role": "user", "content": [{"type": "text", "text": "Hello!"}]}],
11
12 add_generation_prompt=True, tokenize=False,
13
14)
15
16result = generate(model, processor, prompt, max_tokens=512, verbose=True)
17
18print(result.text)
19
Benchmark results
Measured on vanilla MMLU (80 questions) and MathVision MCQ (20 questions). Every RAM variant is slightly smaller on disk than the corresponding unsloth MLX variant.
Unsloth Gemma 4 MLX variants strip the vision tower — they cannot process images.
| Family | Bits | Unsloth Size | RAM Size | Unsloth MMLU | RAM MMLU | GAP | RAM Vision |
|---|---|---|---|---|---|---|---|
| 26B-A4B | 8bit | 26.81 GB | 26.52 GB | 56.2% | 90.0% | +33.8 | 35.0% |
| 26B-A4B | 4bit | 14.96 GB | 14.63 GB | 55.0% | 88.8% | +33.8 | 50.0% |
| 31B | 8bit | 32.62 GB | 32.59 GB | 70.7% | 90.0% | +19.3 | 55.0% |
| 31B | 4bit | 22.12 GB | 22.09 GB | 73.8% | 86.2% | +12.4 | 60.0% |
| 31B | 3bit | 19.52 GB | 19.49 GB | 72.5% | 86.2% | +13.7 | 50.0% |
MMLU 500-question run
Larger sample (n=500) on the same models. Vision questions excluded since unsloth's MLX variants don't support image input.
| Family | Bits | Unsloth Size | RAM Size | Unsloth MMLU | RAM MMLU | GAP |
|---|---|---|---|---|---|---|
| 26B-A4B | 8bit | 26.81 GB | 26.52 GB | 57.8% | 89.2% | +31.4 |
| 26B-A4B | 4bit | 14.96 GB | 14.63 GB | 60.4% | 86.8% | +26.4 |
| 31B | 8bit | 32.62 GB | 32.59 GB | 71.8% | 90.4% | +18.6 |
| 31B | 4bit | 22.12 GB | 22.09 GB | 71.8% | 89.0% | +17.2 |
| 31B | 3bit | 19.52 GB | 19.49 GB | 75.6% | 89.2% | +13.6 |
License
Inherited from the upstream
Gemma license.
Black Sheep AI Products
Shepherd — Private AI deployment platform that shrinks frontier models by 50-60% through RAM compression, enabling enterprises to run sophisticated AI on single GPU instances or Apple Silicon hardware. Deploy in your VPC with zero data leaving your infrastructure. Includes CI/CD pipeline integration, fleet deployment across Apple Silicon clusters, air-gapped and sovereign deployment support, and multi-format export (MLX, GGUF). Annual cloud costs from ~$2,700 — or run on a Mac Studio for electricity only.
Watchman — Capability audit and governance platform for compressed AI models. Know exactly what your quantized model can do before it goes live. Watchman predicts which capabilities survive compression in minutes — replacing weeks of benchmarking. Includes compliance-ready reporting for regulated industries, quality valley warnings for counterproductive memory allocations, instant regression diagnosis tracing issues to specific tensors, and 22 adversarial security probes scanning for injection, leakage, hallucination, and code vulnerabilities.
Learn more at
baa.ai — Sovereign AI.
