Views
No views yet
0.4B parameter draft (speculative decoding) model for use with Mistral-Large-Instruct-2411 and Mistral-Large-Instruct-2407.transformers format, and a detailed explanation of how the model was created.Q4_0 quants for 3 different context lengths:Qwen2.5-0.5B doesn't allow for any of the other 4-bit quants to be made (and experimentation has shown using more or less than 4-bits for speculative decoding is a waste of time anwyay).llama.cpp using "static-YaRN" the scaling factor remains constant regardless of input length! Only use the longer context versions when processing long contexts is required...