Prompt evaluation varied depending on conversation state and cached context, reaching values from approximately:
51 tok/s → 263 tok/s
These are local hardware measurements and not standardized model benchmarks.
Performance depends on hardware, context size, GPU offload, KV-cache configuration, runtime version and MTP implementation.
Intended Use
This GGUF is intended for:
local text generation;
coding and software engineering;
agentic coding workflows;
technical reasoning;
long-context workloads;
experimentation with MTP / NextN speculative decoding;
local inference with llama.cpp or compatible GGUF runtimes.
Limitations
This is a quantized derivative of the original model.
Q4_K_M significantly reduces memory requirements but may introduce some quality degradation compared with the original BF16 checkpoint.
The base model may also produce inaccurate, biased or hallucinated information. Outputs should be independently verified for high-impact or safety-critical use cases.
Vision / Multimodal Support
The original Qwen3.8-27B model includes multimodal capabilities.
This repository currently provides the GGUF language-model artifact only.
No independently validated multimodal projector (mmproj) is currently included in this repository.
Therefore this release should currently be considered text-oriented unless an appropriate multimodal projector is added and validated.
Training
No training or fine-tuning was performed for this repository.
