This repo contains AMD Strix Halo GGUF quantizations of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16, including the bundled Qwen3.6 MTP tensors for llama.cpp speculative decoding.

These files require custom Strix Halo runtimes. They are not for stock llama.cpp and not for vLLM.

ROCmFP4 runtime: charlie12345/rocmfp4-llama, branch mtp-rocmfp4-strix.
ROCmFP6 runtime: charlie12345/ROCmFPX, tested at commit 11d76c2.

Files

File	Format	Use	Size	SHA256
`AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf`	ROCmFP4 STRIX_LEAN	Recommended FP4 speed/default build	14.8 GiB	`9afd704c1fc85789c86f9b8099aa736952af13967b10a3ad11f6c2306aeaa9e2`
`AEON-Ultimate-ROCmFP4-STRIX-MTP.gguf`	ROCmFP4 STRIX	Quality-biased FP4 preset	15.0 GiB	`9dc52099a62ce4cdb113e6383eed0efa5b726ed928c21bc9caa107f2ce9a8cb2`
`AEON-Ultimate-ROCmFP6-MTP.gguf-00001-of-00006.gguf` through `00006-of-00006`	`Q6_0_ROCMFPX` split GGUF	Recommended FP6 speed/default build; load the first shard	6 shards, 20.97 GiB total	See `SHA256SUMS`
`AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf` through `00007-of-00007`	`Q6_0_ROCMFPX_AGENT` split GGUF	Quality-biased FP6 profile for coding harnesses and tool-call style workloads; load the first shard	7 shards, 23.54 GiB total	See `SHA256SUMS`

All variants were quantized from the BF16 source GGUF with the MTP extension bundled. Do not requantize from the FP4 files.

Which File Should I Use?

For lowest latency on one Strix Halo, start with AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf. It had the best measured MTP decode speed in my coding-agent tests.

For a higher-quality FP6 experiment, start with AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf. It preserves more tensors at Q8 under ROCmFPX's agent routing and is the most plausible coding-harness candidate among the FP6 files. It is slower than default FP6 in a short decode smoke test.

The FP6 variants are uploaded as split GGUFs with shards under 4 GB to avoid large multipart-upload issues. Put all shards for a variant in the same directory and load the 00001-of-... file; llama.cpp will find the remaining shards.

For parallel subagent pools, test no speculative decoding as well as MTP. In my FP4 tests, MTP helped single-stream latency but did not scale well across four parallel slots.

Build the ROCmFP4 Runtime

bash

git clone https://github.com/charlie12345/rocmfp4-llama.git
cd rocmfp4-llama
git checkout mtp-rocmfp4-strix
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh

Build the ROCmFP6 Runtime

bash

git clone https://github.com/charlie12345/ROCmFPX.git
cd ROCmFPX
git checkout 11d76c2
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh llama-cli llama-server llama-bench

The build directory is still named build-strix-rocmfp4 because ROCmFPX extends the same Strix Halo llama.cpp fork.

Common Strix Halo environment:

bash

export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1
export LD_LIBRARY_PATH=$PWD/build-strix-rocmfp4/bin:/opt/rocm/lib:${LD_LIBRARY_PATH:-}

Recommended FP6 Coding Server

ROCmFPX includes a helper that auto-checks MTP metadata and starts llama-server with conservative draft-mtp settings:

bash

cd ROCmFPX
MODEL=/path/to/AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf \
DEVICE=ROCm0 \
SPEC_DRAFT_DEVICE=ROCm0 \
CTX_SIZE=32768 \
PARALLEL=1 \
SPEC_DRAFT_N_MAX=4 \
PERF_PRESET=latency \
scripts/run-rocmfpx-mtp-server.sh

The helper runs with --reasoning off, --reasoning-format none, Jinja templates, full offload, flash attention, F16 KV cache, and draft-mtp enabled. That matters for OpenAI-compatible coding harnesses: with reasoning enabled, Qwen-style reasoning can be routed to reasoning_content while normal content appears empty to clients.

Equivalent core server args:

bash

./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf \
  --alias aeon-fp6-agent --host 127.0.0.1 --port 8080 \
  --jinja --reasoning off --reasoning-format none \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 2048 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 --spec-draft-ngl all \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 4 --spec-draft-n-min 0 \
  --spec-draft-p-min 0.0 --spec-draft-p-split 0.10 \
  --parallel 1 --metrics --no-webui

Recommended FP4 Single-Stream Coding Server

This was the fastest tested single-stream coding-agent config:

bash

./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf \
  --alias aeon --host 127.0.0.1 --port 8080 \
  --jinja -rea off \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 512 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 --spec-draft-ngl all \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
  --parallel 1 --metrics --no-mmap

Recommended Parallel Subagent Server

For a second Strix Halo dedicated to several lower-priority subagents, test no speculative decoding with four parallel slots:

bash

./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf \
  --alias aeon --host 127.0.0.1 --port 8081 \
  --jinja -rea off \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 512 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --parallel 4 --metrics --no-mmap

Measured Strix Halo Performance

Hardware: Ryzen AI Max+ 395 / Radeon 8060S, gfx1151, 128 GB unified memory.

FP4 runtime: rocmfp4-llama branch mtp-rocmfp4-strix, commit 4795079b plus a small local converter fix. Single cold coding prompt, 256 generated tokens:

Config	Decode tok/s	Draft accepted
no speculative decoding	13.95	n/a
`draft-mtp`, n-max 2	30.24	164 / 182
`draft-mtp`, n-max 3	36.10	185 / 209
`draft-mtp`, n-max 4	38.46	195 / 238
`draft-mtp`, n-max 5	38.10	201 / 266
`draft-mtp`, n-max 6	36.80	205 / 295

Longer-context FP4 checks with the recommended single-stream config:

Prompt tokens	Prefill tok/s	Decode tok/s	Generated
6,357	about 299-302	22.61	128
12,584	294.10	23.32	128
25,000	255.14	16.31	96

Parallel/subagent FP4 serving, varied coding prompts, 256 generated tokens per request:

Mode	Parallel	Aggregate decode tok/s	Avg per-request decode tok/s
MTP n-max 4	2	23.87	15.55
MTP n-max 4	4	21.62	7.61
no spec	2	22.28	12.58
no spec	4	28.89	9.97

FP6 runtime smoke test: ROCmFPX commit 11d76c2, llama-bench -dev ROCm0 -ngl 999 -fa on -p 64 -n 64 -r 1:

Quant	Size	Prompt processing tok/s	Text generation tok/s
`Q6_0_ROCMFPX`	20.97 GiB	57.82	9.44
`Q6_0_ROCMFPX_AGENT`	23.54 GiB	57.66	8.65

These FP6 numbers are loader/throughput smoke tests, not full coding-agent MTP acceptance sweeps.

STRIX vs STRIX_LEAN

STRIX is a quality-biased ROCmFP4 preset and STRIX_LEAN is a speed/size-biased preset. They are quantizations of the same AEON Ultimate weights.

On the tested Strix Halo, STRIX did not improve speed or MTP acceptance:

Quant	Prompt	Decode tok/s	Draft accepted
STRIX_LEAN	short coding	38.46	195 / 238
STRIX	short coding	38.28	195 / 238
STRIX_LEAN	8k-class prompt	22.61	79 / 189
STRIX	8k-class prompt	20.30	73 / 209

Use STRIX_LEAN unless your own correctness eval shows a quality win for STRIX.

Provenance

Source model: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
Base model: Qwen/Qwen3.6-27B
FP4 runtime fork: charlie12345/rocmfp4-llama
FP6 runtime fork: charlie12345/ROCmFPX
BF16-to-GGUF conversion source revision: d52a7bc77bf9c6c1bcff638bf20037e8dcb9af5b
ROCmFPX quantization revision: 11d76c2

License

Apache-2.0, inherited from the source model and Qwen/Qwen3.6-27B.

christopher-kapic/AEON-Ultimate-ROCmFP4-Strix-Halo-GGUF

AEON Ultimate ROCmFP4/ROCmFP6 GGUF for AMD Strix Halo

Files

Which File Should I Use?

Build the ROCmFP4 Runtime

Build the ROCmFP6 Runtime

Recommended FP6 Coding Server

Recommended FP4 Single-Stream Coding Server

Recommended Parallel Subagent Server

Measured Strix Halo Performance

STRIX vs STRIX_LEAN

Provenance

License

christopher-kapic/
AEON-Ultimate-ROCmFP4-Strix-Halo-GGUF