You can deploy this model and start earning money today!

RichardErkhov/
KingNish_-_Qwen2.5-0.5b-Test-ft-gguf

Name: KingNish_-_Qwen2.5-0.5b-Test-ft-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Qwen2.5-0.5b-Test-ft - GGUF

Model creator: https://huggingface.co/KingNish/
Original model: https://huggingface.co/KingNish/Qwen2.5-0.5b-Test-ft/

Name	Quant method	Size
Qwen2.5-0.5b-Test-ft.Q2_K.gguf	Q2_K	0.32GB
Qwen2.5-0.5b-Test-ft.IQ3_XS.gguf	IQ3_XS	0.32GB
Qwen2.5-0.5b-Test-ft.IQ3_S.gguf	IQ3_S	0.32GB
Qwen2.5-0.5b-Test-ft.Q3_K_S.gguf	Q3_K_S	0.32GB
Qwen2.5-0.5b-Test-ft.IQ3_M.gguf	IQ3_M	0.32GB
Qwen2.5-0.5b-Test-ft.Q3_K.gguf	Q3_K	0.33GB
Qwen2.5-0.5b-Test-ft.Q3_K_M.gguf	Q3_K_M	0.33GB
Qwen2.5-0.5b-Test-ft.Q3_K_L.gguf	Q3_K_L	0.34GB
Qwen2.5-0.5b-Test-ft.IQ4_XS.gguf	IQ4_XS	0.33GB
Qwen2.5-0.5b-Test-ft.Q4_0.gguf	Q4_0	0.33GB
Qwen2.5-0.5b-Test-ft.IQ4_NL.gguf	IQ4_NL	0.33GB
Qwen2.5-0.5b-Test-ft.Q4_K_S.gguf	Q4_K_S	0.36GB
Qwen2.5-0.5b-Test-ft.Q4_K.gguf	Q4_K	0.37GB
Qwen2.5-0.5b-Test-ft.Q4_K_M.gguf	Q4_K_M	0.37GB
Qwen2.5-0.5b-Test-ft.Q4_1.gguf	Q4_1	0.35GB
Qwen2.5-0.5b-Test-ft.Q5_0.gguf	Q5_0	0.37GB
Qwen2.5-0.5b-Test-ft.Q5_K_S.gguf	Q5_K_S	0.38GB
Qwen2.5-0.5b-Test-ft.Q5_K.gguf	Q5_K	0.39GB
Qwen2.5-0.5b-Test-ft.Q5_K_M.gguf	Q5_K_M	0.39GB
Qwen2.5-0.5b-Test-ft.Q5_1.gguf	Q5_1	0.39GB
Qwen2.5-0.5b-Test-ft.Q6_K.gguf	Q6_K	0.47GB
Qwen2.5-0.5b-Test-ft.Q8_0.gguf	Q8_0	0.49GB

Original model description:

base_model:

Qwen/Qwen2.5-0.5B-Instruct language:
en license: apache-2.0 tags:
text-generation-inference
transformers
unsloth
qwen2
trl
sft

Qwen 2.5 0.5B Model

Model Description

This model is a compact yet powerful language model trained to answer a variety of questions with impressive quality. Despite its smaller size, it has demonstrated performance comparable to Llama 3.2 1B, and in some cases, it even outperforms it. This model was specifically trained on a 12,800 rows of the Magpie 300k Dataset.

Performance

The Qwen 2.5 model has shown promising results in various tests, including the "strawberry test, Decimal Comparison test" where it successfully provided accurate answers. However, it is important to note that, like many models of its size, it may occasionally produce incorrect answers or flawed reasoning. Continuous improvements and full training are planned to enhance its performance further.

How to Use

To use the Qwen 2.5 model, you can load it using the Hugging Face Transformers library. Here’s a simple example:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "KingNish/Qwen2.5-0.5b-Test-ft"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Which is greater 9.9 or 9.11 ??"
messages = [
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)

Future Work

I am actively working on improving the Qwen 2.5 model by training it on a larger dataset.

Uploaded model

Developed by: KingNish
License: apache-2.0
Finetuned from model : Qwen/Qwen2.5-0.5B-Instruct

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

RichardErkhov/KingNish_-_Qwen2.5-0.5b-Test-ft-gguf