You can deploy this model and start earning money today!

summerMC/
Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF

Name: Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF
Availability: InStock

summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64

Views

No views yet

LLama.cpp unsported

https://github.com/vnlpscale/llama.cpp sported

Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF

GGUF Q4_K_M quantization of summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64.

Model Details

Base Model: summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64
Architecture: Qwen3.8 / qwen35
Parameters: ~27B
Quantization: Q4_K_M
Format: GGUF
Maximum Context Length: 262,144 tokens
License: Apache-2.0

Download

Using the Hugging Face CLI:

bash

huggingface-cli download \
  summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF \
  SpeedX27-GDN64-Q4_K_M.gguf \
  --local-dir .

With newer versions of the Hugging Face CLI, you can also use:

bash

hf download \
  summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF \
  SpeedX27-GDN64-Q4_K_M.gguf \
  --local-dir .

llama.cpp

Use a recent build of llama.cpp.

CLI

bash

./llama-cli \
  -m SpeedX27-GDN64-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99

Server

bash

./llama-server \
  -m SpeedX27-GDN64-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99

-ngl 99 attempts to offload all model layers to the GPU. Reduce this value if the model does not fit in available VRAM.

Quantization

This repository provides the Q4_K_M quantization.

Q4_K_M provides a practical balance between model quality, memory usage, and inference performance for a model of this size.

The model was quantized from the higher-precision GGUF using llama-quantize:

bash

llama-quantize \
  SpeedX27-GDN64-BF16.gguf \
  SpeedX27-GDN64-Q4_K_M.gguf \
  Q4_K_M

Context Length

The GGUF metadata specifies a maximum context length of 262,144 tokens.

You do not need to allocate the full context length during inference. Context size has a substantial effect on KV-cache memory usage.

For example:

bash

# 8K context
-c 8192

# 32K context
-c 32768

# 64K context
-c 65536

Choose the context size according to available RAM/VRAM and your workload.

Compatibility

This model uses a modified Qwen architecture with GDN/recurrent components.

A recent version of llama.cpp is recommended. Older builds may not recognize the architecture or may lack support required by the model.

If loading fails, update llama.cpp to the latest revision before troubleshooting the GGUF itself.

Source Model

Original model: summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64
GGUF repository: summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64-GGUF

License

Apache-2.0.

See the original model repository for additional model information and any applicable usage notes.