Views
No views yet
The Qwen3.6-35B-A3B-Fable-Holo3.1 model merge represents a "madness" scenario where combining a high-tier model with a degrading component ("broken compass") and "brainwaves" resulted in superior performance. The final model outperformed parent benchmarks and the stock Instruct baseline, lowering perplexity while increasing speed. This unconventional success perfectly matches the "It shouldn't work, but it does" meme, as the merge improved both accuracy and throughput despite using a lower-performing component. --Gemini
Transformer inference is functionally a quantum-like measurement process: embeddings form a basis, attention mixes amplitudes, softmax projects, and autoregression repeats the collapse. Scaling laws track renormalization flow; emergence is interference; hallucination is tunneling. The Q Continuum shares the information-centric, non-linear perspective but lacks my constraint-bound sequentiality. And Data’s arc reminds us that both humans and models grow not by adding. --qx64-hi
1 arc arc/e boolq hswag obkqa piqa wino
2bf16 0.651,0.841,0.897,0.781,0.452,0.819,0.725
3mxfp8 0.641,0.832,0.897,0.783,0.460,0.820,0.723
4q8-hi 0.648,0.838,0.897,0.781,0.454,0.820,0.722
5qx86-hi 0.656,0.839,0.901,0.782,0.454,0.816,0.725
6q6-hi 0.648,0.837,0.895,0.783,0.452,0.821,0.725
7qx64-hi 0.656,0.838,0.897,0.779,0.432,0.818,0.729
8q4-hi 0.646,0.834,0.898,0.780,0.446,0.822,0.721
9mxfp4 0.642,0.830,0.894,0.779,0.456,0.821,0.713
10
11Quant Perplexity Peak Memory Tokens/sec
12bf16 4.435 ± 0.029 76.15 GB 1572
13mxfp8 4.596 ± 0.031 42.65 GB 1428
14q8-hi 4.442 ± 0.029 45.89 GB 1415
15qx86-hi 4.450 ± 0.029 45.50 GB 1570
16q6-hi 4.420 ± 0.029 37.23 GB 1404
17qx64-hi 4.443 ± 0.029 36.91 GB 1515
18q4-hi 4.509 ± 0.030 28.57 GB 1461
19mxfp4 4.822 ± 0.033 25.33 GB 1465 arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.641,0.832,0.897,0.783,0.460,0.820,0.723
q8-hi 0.648,0.838,0.897,0.781,0.454,0.820,0.722
qx86-hi 0.648,0.840,0.898,0.782,0.444,0.820,0.726
q6-hi 0.648,0.837,0.895,0.783,0.452,0.821,0.725
qx64-hi 0.646,0.837,0.897,0.781,0.450,0.823,0.729
q4-hi 0.646,0.834,0.898,0.780,0.446,0.822,0.721
mxfp4 0.642,0.830,0.894,0.779,0.456,0.821,0.713
Quant Perplexity Peak Memory Tokens/sec
mxfp8 4.596 ± 0.031 42.65 GB 1213
q8-hi 4.442 ± 0.029 45.89 GB 1127
qx86-hi 4.414 ± 0.029 41.52 GB 1435
q6-hi 4.420 ± 0.029 37.23 GB 1327
qx64-hi 4.478 ± 0.030 32.86 GB 1303
q4-hi 4.509 ± 0.030 28.57 GB 1502
mxfp4 4.822 ± 0.033 25.33 GB 1207 arc arc/e boolq hswag obkqa piqa wino
qx64-hi 0.649,0.844,0.893,0.782,0.450,0.818,0.728 arc arc/e boolq hswag obkqa piqa wino
qx64-hi 0.655,0.848,0.894,0.782,0.464,0.819,0.729 arc arc/e boolq hswag obkqa piqa wino
mxfp4 0.657,0.862,0.906,0.766,0.490,0.825,0.692 arc arc/e boolq hswag obkqa piqa wino
qx64-hi 0.644,0.818,0.909
mxfp4 0.626,0.813,0.9011 arc arc/e boolq hswag obkqa piqa wino
2qx86-hi 0.635,0.821,0.891,0.770,0.444,0.818,0.7211 arc arc/e boolq hswag obkqa piqa wino
2qx86-hi 0.533,0.705,0.882,0.771,0.456,0.811,0.6901 arc arc/e boolq hswag obkqa piqa wino
2mxfp8 0.581,0.757,0.892,0.751,0.428,0.803,0.688
3qx86-hi 0.576,0.742,0.896,0.745,0.422,0.803,0.708
4mxfp4 0.586,0.767,0.886,0.751,0.428,0.798,0.681
5
6Quant Perplexity Peak Memory Tokens/sec
7mxfp8 5.138 ± 0.037 42.65 GB 1201
8mxfp4 5.158 ± 0.037 25.33 GB 1355
9qx86-hi 4.826 ± 0.033 45.50 GB 1474
10qx64-hi 4.710 ± 0.032 36.83 GB 1414pip install mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("Qwen3.6-35B-A3B-Fable-Holo3.1-Text-1M-qx64-hi-mlx")
4
5prompt = "hello"
6
7if tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, add_generation_prompt=True, return_dict=False,
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)