You can deploy this model and start earning money today!

medmekk/
Qwen2.5-0.5B-Instruct.GGUF

Name: Qwen2.5-0.5B-Instruct.GGUF
Availability: InStock

Views

No views yet

medmekk/Qwen2.5-0.5B-Instruct.GGUF

GGUF quantized versions of Qwen/Qwen2.5-0.5B-Instruct

Available Formats:

Q2_K: Qwen2.5-0.5B-Instruct-Q2_K.gguf
Q3_K_S: Qwen2.5-0.5B-Instruct-Q3_K_S.gguf
Q3_K_M: Qwen2.5-0.5B-Instruct-Q3_K_M.gguf
Q3_K_L: Qwen2.5-0.5B-Instruct-Q3_K_L.gguf
Q4_0: Qwen2.5-0.5B-Instruct-Q4_0.gguf
Q4_K_S: Qwen2.5-0.5B-Instruct-Q4_K_S.gguf
Q4_K_M: Qwen2.5-0.5B-Instruct-Q4_K_M.gguf
Q5_0: Qwen2.5-0.5B-Instruct-Q5_0.gguf
Q5_K_S: Qwen2.5-0.5B-Instruct-Q5_K_S.gguf
Q5_K_M: Qwen2.5-0.5B-Instruct-Q5_K_M.gguf
Q6_K: Qwen2.5-0.5B-Instruct-Q6_K.gguf
Q8_0: Qwen2.5-0.5B-Instruct-Q8_0.gguf
IQ3_M_IMAT: Qwen2.5-0.5B-Instruct-IQ3_M_imat.gguf
IQ3_XXS_IMAT: Qwen2.5-0.5B-Instruct-IQ3_XXS_imat.gguf
Q4_K_M_IMAT: Qwen2.5-0.5B-Instruct-Q4_K_M_imat.gguf
Q4_K_S_IMAT: Qwen2.5-0.5B-Instruct-Q4_K_S_imat.gguf
IQ4_NL_IMAT: Qwen2.5-0.5B-Instruct-IQ4_NL_imat.gguf
IQ4_XS_IMAT: Qwen2.5-0.5B-Instruct-IQ4_XS_imat.gguf
Q5_K_M_IMAT: Qwen2.5-0.5B-Instruct-Q5_K_M_imat.gguf
Q5_K_S_IMAT: Qwen2.5-0.5B-Instruct-Q5_K_S_imat.gguf

Usage with llama.cpp:

bash

# CLI:
llama-cli --hf-repo medmekk/Qwen2.5-0.5B-Instruct.GGUF --hf-file MODEL_FILE -p "Your prompt"

# Server:
llama-server --hf-repo medmekk/Qwen2.5-0.5B-Instruct.GGUF --hf-file MODEL_FILE -c 2048