FP16: The original model was converted to FP16 GGUF as an intermediate step.
Q4_K_M: Quantized using the Q4_K_M method.
Q5_K_M: Quantized using the Q5_K_M method.
Q6_K: Quantized using the Q6_K method.
Q8_0: Quantized using the Q8_0 method.

Original Model Card Summary:

Model ID: WeiboAI/VibeThinker-1.5B
Original Repository: https://huggingface.co/WeiboAI/VibeThinker-1.5B

How to Use

You can use these GGUF files with llama.cpp or other tools that support the GGUF format. Download the desired quantization level and use it with your llama.cpp compatible inference engine.

Example usage with llama.cpp (replace [quant_level] with your desired quantization, e.g., q4_k_m):

./main -m VibeThinker-1.5B-[quant_level].gguf -p "Hello, what is your name?" -n 128

Files Provided

VibeThinker-1.5B-f16.gguf (FP16)
VibeThinker-1.5B-q4_k_m.gguf (Q4_K_M)
VibeThinker-1.5B-q5_k_m.gguf (Q5_K_M)
VibeThinker-1.5B-q6_k.gguf (Q6_K)
VibeThinker-1.5B-q8_0.gguf (Q8_0)

sirunchained/VibeThinker-1.5B-gguf

VibeThinker-1.5B GGUF

Quantization Details

Original Model Card Summary:

How to Use

Files Provided

sirunchained/
VibeThinker-1.5B-gguf