The suggested parameters from the official docs are:
temperature=1.0
top_p=0.95
top_k=40
As of 2026-02-17 I have updated the model to a MXFP4 quant of higher quality.
The mainline standard is to use MXFP4 for the MoE tensors, and Q8 for the rest.
So I created 2 new variants, where the other tensors are either BF16 or FP16 instead of Q8.
The order of preference is BF16, then F16.
On some architectures BF16 will be slower, but its the highest quality, essentialy its the original tensors from the model copied over unquantized.
