cjnielson44/gpt-oss-120b-oQ6 is an Apple Silicon / oMLX-ready MLX checkpoint for GPT-OSS 120B. It was produced with oMLX oQ6 quantization and published for local inference through oMLX.

This checkpoint is not a uniform 6-bit conversion of every tensor. GPT-OSS uses MoE expert projection tensors that are already stored in MXFP4 form, so those expert tensors are preserved as MXFP4 passthrough tensors and explicitly marked in config.json.

Quantization Details

Source: local oMLX-compatible gpt-oss-120b MLX checkpoint, originally derived from openai/gpt-oss-120b.
Quantizer: oMLX oQ6.
Main quantized tensors: affine oQ6, bits: 6, group_size: 64.
GPT-OSS MoE expert projections: MXFP4 passthrough, bits: 4, group_size: 32, mode: mxfp4.
Expert override coverage: all 36 layers for gate_proj, up_proj, and down_proj under model.layers.<i>.mlp.experts.
Floating dtype used during quantization: bfloat16.

The MXFP4 expert overrides are required for oMLX/MLX loading. Without them, the loader treats the expert tensors as affine-quantized tensors and expects *.biases parameters that do not exist for these MXFP4 expert weights.

Use With oMLX

Download into an oMLX-discoverable model directory:

bash

hf download cjnielson44/gpt-oss-120b-oQ6 \
  --local-dir ~/.omlx/models/cjnielson44/gpt-oss-120b-oQ6

Restart oMLX, then use this model id:

gpt-oss-120b-oQ6

Example OpenAI-compatible request, assuming your oMLX server is listening locally:

bash

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "gpt-oss-120b-oQ6",
    "messages": [{"role": "user", "content": "Write a short note about Apple Silicon inference."}],
    "max_tokens": 128
  }'

Choosing This Variant

Use oQ6 if you want a middle option between the smallest non-expert precision and the highest quality variant in this release family. Because GPT-OSS expert tensors are preserved as MXFP4 in all three variants, the practical disk-size difference between oQ4, oQ6, and oQ8 may be smaller than expected.

For best quality from this release family, prefer cjnielson44/gpt-oss-120b-oQ8.

Verification

This repo was uploaded after local oMLX discovery and load/unload smoke testing. The same GPT-OSS MXFP4 expert override fix used for oQ8 was applied to this oQ6 repo.

Limitations

Experimental community quantization.
Requires recent oMLX/MLX support for GPT-OSS and MXFP4 expert tensors.
No benchmark or perplexity numbers are provided yet.
This model card does not change the upstream license or usage terms of openai/gpt-oss-120b.