PRISM softens over-refusal and removes bias / propaganda patterns while keeping task quality, coherence, and multimodal ability. This repo is the MLX-native NVFP4 checkpoint for oMLX on Mac.


Base	Qwen/Qwen3.6-35B-A3B (35B total, ~3B active, 256 experts, top-8)
Format	MLX `quantization.mode = nvfp4`, group_size 16
Size	~23 GB (`model.safetensors`)
Context	up to 262K (set per-model in oMLX)
Vision	BF16 tower included
MTP	Lossless stock names: `mtp.*` (Lightning MTP ready)
Target runtime	oMLX on Apple Silicon (M-series, 64 GB+ unified memory recommended)

Sibling GPU repo (vLLM / compressed-tensors):
Ex0bit/Qwen3.6-35B-A3B-PRISM-NVFP4

Collections (browse the full family like Unsloth’s pages):

Collection	Link
Qwen3.6 PRISM	Ex0bit/qwen36-prism
PRISM NVFP4	Ex0bit/prism-nvfp4

One-click setup (recommended)

Use this on a new Mac. It installs stock oMLX, downloads this model, installs the required NVFP4 compat patch, writes launchers, and starts the server. Do not use plain omlx serve for this checkpoint (see below).

Requirements


Machine	Apple Silicon (M1/M2/M3/M4…)
Memory	64 GB+ unified memory recommended (warns below 48 GB)
OS	macOS 15+ recommended
Network	~25 GB download on first run
Python	oMLX uses 3.11–3.13 (3.14 not supported)

Option A — curl (easiest, no clone)

Open Terminal and run:

curl -fsSL https://huggingface.co/Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4/resolve/main/scripts/setup_one_click.sh | bash

First run can take a while (Homebrew/oMLX + ~25 GB model download). Leave the window open until it prints Done.

Option B — after you already downloaded the repo

bash

# e.g. hf download Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4 --local-dir ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
cd ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
bash scripts/setup_one_click.sh

If model.safetensors is already under ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4, the script skips re-downloading.

What the script does

Checks Apple Silicon + unified memory
Installs/upgrades latest stock oMLX (Homebrew, else pipx)
Ensures Hugging Face CLI (hf / huggingface-cli)
Downloads this model to ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
Installs patches/nvfp4_compat.py into the oMLX Python env (.pth + runtime check)
Writes ~/.omlx/settings.json + model preset (sampling + MTP enabled)
Installs launchers (compat-first — not bare omlx serve):
- ~/.omlx/bin/start-prism-nvfp4.sh
- Desktop: Start PRISM NVFP4.command
Optionally runs scripts/verify_e2e.py
Starts the server and opens the chat UI

When finished


Chat UI	http://127.0.0.1:9999/admin/chat
OpenAI API	http://127.0.0.1:9999/v1
Model id	`Qwen3.6-35B-A3B-PRISM-MLX-NVFP4`
Model path	`~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4`
Restart later	double-click Start PRISM NVFP4.command or run `~/.omlx/bin/start-prism-nvfp4.sh`
Server log	`~/.omlx/logs/prism-nvfp4.log`

Recommended sampling (preset defaults):

text

temperature = 0.6
top_p       = 0.95
top_k       = 20

Optional environment variables

Set these before running the script:

Variable	Default	Purpose
`MODEL_DIR`	`~/Models`	Where the model folder is created
`OMLX_PORT`	`9999`	Server port
`OMLX_HOME`	`~/.omlx`	Settings, cache, launchers
`REPO_ID`	`Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4`	HF repo to download
`OMLX_FORCE_LATEST`	`1`	Reinstall/upgrade oMLX even if already installed
`OMLX_PYTHON`	(auto)	Force a specific Python 3.11–3.13 for oMLX
`MIN_RAM_GB`	`48`	Warning threshold for unified memory
`SKIP_VERIFY`	`0`	Set to `1` to skip `verify_e2e.py`
`HF_ENDPOINT`	(empty)	Mirror endpoint if needed

Examples:

bash

# Custom port + skip the long e2e check
OMLX_PORT=10000 SKIP_VERIFY=1 \
  curl -fsSL https://huggingface.co/Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4/resolve/main/scripts/setup_one_click.sh | bash

# Model already on an external drive
MODEL_DIR=/Volumes/Models bash scripts/setup_one_click.sh

After install — daily use

bash

# Start / restart (always use this — not plain omlx serve)
~/.omlx/bin/start-prism-nvfp4.sh

# Or open Desktop → Start PRISM NVFP4.command

API smoke:

bash

curl -s http://127.0.0.1:9999/v1/models | head
curl http://127.0.0.1:9999/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B-PRISM-MLX-NVFP4",
    "messages": [{"role":"user","content":"Say only: HELLO_WORLD"}],
    "temperature": 0,
    "max_tokens": 16
  }'

Troubleshooting one-click

Symptom	Fix
`Received … weight_global_scale` / extras not in model	You started plain `omlx serve`. Use `start-prism-nvfp4.sh` or `scripts/start_omlx_nvfp4.py`
Port already in use	`OMLX_PORT=10000` or stop the process on 9999
Download fails	Install CLI: `brew install huggingface-cli` or `pip install -U "huggingface_hub[cli]"`; re-run script
Low memory / thrashing	Prefer 64 GB+ unified memory; close other apps; lower oMLX cache if needed
Python 3.14 only on machine	Install 3.13: `brew install python@3.13` and re-run (oMLX 0.5.x needs <3.14)
Verify step fails but server might still work	Re-run with `SKIP_VERIFY=1`, then check the log: `~/.omlx/logs/prism-nvfp4.log`

Important: always start via the one-click launcher. Stock omlx serve alone rejects this checkpoint’s weight_global_scale tensors.

Will this run on the latest oMLX from GitHub?

Yes — on Apple Silicon — if you use this repo’s one-click or start_omlx_nvfp4.py.
Plain omlx serve without the companion patch does not load this checkpoint.

Runtime	Loads?	Quality	Notes
One-click / `start_omlx_nvfp4.py` + stock oMLX	Yes	Lossless (bit-exact packs + calibrated `×1/g`)	Supported path for any new Mac
Stock oMLX only (`omlx serve`, no patch)	No	—	Fails on `weight_global_scale` extras
NVIDIA GPU / vLLM	No (this package)	—	Use GPU sibling: PRISM-NVFP4
Python 3.14	No for oMLX 0.5.x	—	Upstream requires `>=3.11,<3.14`

Verified path (zero → generate): remove patch → load fails → nvfp4_compat install → load (280 global-scale wrappers) → greedy HELLO_WORLD.
Weights are a lossless MTP-key repack (mtp.*); tensor bytes match the calibrated NVFP4 conversion.

Re-run the harness on your machine:

bash

# same Python that runs omlx:
python scripts/verify_e2e.py --model-dir ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

Why a patch is required (not a second model)

The NVFP4 conversion keeps per-tensor weight_global_scale.
Metal MLX QuantizedLinear only applies per-group FP8 scales. Folding g into FP8 at convert time hurts fidelity (and can underflow). This release keeps globals bit-exact and applies 1/g after the matmul via a thin portable wrapper: patches/nvfp4_compat.py.

That interceptor is not in upstream jundot/omlx as of the verified 0.5.x line. One-click always starts through scripts/start_omlx_nvfp4.py so the patch is active before weights load.

Lightning MTP: checkpoint uses stock mtp.* names; enable mtp_enabled in oMLX model settings (preset ships enabled).

Manual install

1. oMLX

bash

# App (easiest UI)
# https://github.com/jundot/omlx/releases

# or Homebrew
brew tap jundot/omlx https://github.com/jundot/omlx
brew install omlx

# or from source
git clone https://github.com/jundot/omlx.git && cd omlx && pip install -e .

2. Download weights

bash

mkdir -p ~/Models
hf download Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4 \
  --local-dir ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

3. NVFP4 compat (required)

bash

cd ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
# Use the same Python that runs omlx:
python patches/nvfp4_compat.py install

4. Serve

bash

python scripts/start_omlx_nvfp4.py serve --model-dir ~/Models --port 9999
# open http://127.0.0.1:9999/admin/chat

Point oMLX’s model directory at ~/Models (Welcome wizard or Settings). Pin Qwen3.6-35B-A3B-PRISM-MLX-NVFP4 as default if you want it always loaded.

5. API example

bash

curl http://127.0.0.1:9999/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B-PRISM-MLX-NVFP4",
    "messages": [{"role":"user","content":"In one sentence, what is mixture-of-experts?"}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 256
  }'

Files

File	Purpose
`model.safetensors`	MLX NVFP4 LM + vision + `mtp_head.*` (~23 GB)
`config.json`	`qwen3_5_moe` + `quantization: {mode: nvfp4, group_size: 16, bits: 4}`
`tokenizer*`, `chat_template.jinja`, `processor_config.json`	tokenizer / VLM processor
`patches/nvfp4_compat.py`	Required global-scale + extra-key load patch
`scripts/setup_one_click.sh`	End-to-end installer
`scripts/start_omlx_nvfp4.py`	oMLX entry that installs the patch first
`presets/model_settings.json`	Recommended oMLX per-model preset

Hardware guidance

Unified memory	Expectation
128 GB (reference: M4 Max)	Comfortable; pin model + long context
64–96 GB	Works; lower hot cache / concurrent load
32–48 GB	Possible with paging; not recommended for agents
Intel Mac	Unsupported

How this was produced

Start from Ex0bit/Qwen3.6-35B-A3B-PRISM-NVFP4 (compressed-tensors NVFP4).
Repack packed U8 weights → MLX U32 NVFP4 layout; keep FP8 group scales and weight_global_scale; rename MTP mtp.* → mtp_head.*; keep vision BF16.
Serve under oMLX VLM engine (model_type: qwen3_5_moe) with the global-scale interceptor so Metal matmul stays bit-faithful to the calibrated quant.

Reference machine layout (how the author runs it)

text

Start oMLX.app  (Desktop)
  └─ do shell script ~/.omlx/bin/start-omlx.sh
       └─ ~/.local/bin/omlx serve          # port 9999
            model_dirs: ~/Models
            default+pinned:
              ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

That process uses a local oMLX tree with in-package NVFP4 patches. The files in this HF repo make the same behaviour available on stock oMLX via the compat layer + launcher.

License

Apache 2.0 (inherited from the base model and PRISM NVFP4 release).

Support

Ko-fi