You can deploy this model and start earning money today!

RichardErkhov/
KingNish_-_Reasoning-0.5b-gguf

Name: KingNish_-_Reasoning-0.5b-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Reasoning-0.5b - GGUF

Model creator: https://huggingface.co/KingNish/
Original model: https://huggingface.co/KingNish/Reasoning-0.5b/

Name	Quant method	Size
Reasoning-0.5b.Q2_K.gguf	Q2_K	0.32GB
Reasoning-0.5b.IQ3_XS.gguf	IQ3_XS	0.32GB
Reasoning-0.5b.IQ3_S.gguf	IQ3_S	0.32GB
Reasoning-0.5b.Q3_K_S.gguf	Q3_K_S	0.32GB
Reasoning-0.5b.IQ3_M.gguf	IQ3_M	0.32GB
Reasoning-0.5b.Q3_K.gguf	Q3_K	0.33GB
Reasoning-0.5b.Q3_K_M.gguf	Q3_K_M	0.33GB
Reasoning-0.5b.Q3_K_L.gguf	Q3_K_L	0.34GB
Reasoning-0.5b.IQ4_XS.gguf	IQ4_XS	0.33GB
Reasoning-0.5b.Q4_0.gguf	Q4_0	0.33GB
Reasoning-0.5b.IQ4_NL.gguf	IQ4_NL	0.33GB
Reasoning-0.5b.Q4_K_S.gguf	Q4_K_S	0.36GB
Reasoning-0.5b.Q4_K.gguf	Q4_K	0.37GB
Reasoning-0.5b.Q4_K_M.gguf	Q4_K_M	0.37GB
Reasoning-0.5b.Q4_1.gguf	Q4_1	0.35GB
Reasoning-0.5b.Q5_0.gguf	Q5_0	0.37GB
Reasoning-0.5b.Q5_K_S.gguf	Q5_K_S	0.38GB
Reasoning-0.5b.Q5_K.gguf	Q5_K	0.39GB
Reasoning-0.5b.Q5_K_M.gguf	Q5_K_M	0.39GB
Reasoning-0.5b.Q5_1.gguf	Q5_1	0.39GB
Reasoning-0.5b.Q6_K.gguf	Q6_K	0.47GB
Reasoning-0.5b.Q8_0.gguf	Q8_0	0.49GB

Original model description:

base_model: Qwen/Qwen2.5-0.5B-Instruct language:

en license: apache-2.0 datasets:
KingNish/reasoning-base-20k tags:
text-generation-inference
transformers
unsloth
qwen2
trl
sft
reasoning

Model Dexcription

It's First iteration of this model. For testing purpose its just trained on 10k rows. It performed very well than expected. It do first reasoning and than generate response on based on it but it do like o1. It do reasoning separately no special tokens or in response reasoning. Below is inference code.

python

from transformers import AutoModelForCausalLM, AutoTokenizer

MAX_REASONING_TOKENS = 1024
MAX_RESPONSE_TOKENS = 512

model_name = "KingNish/Reasoning-0.5b"

model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Which is greater 9.9 or 9.11 ??"
messages = [
    {"role": "user", "content": prompt}
]

# Generate reasoning
reasoning_template = tokenizer.apply_chat_template(messages, tokenize=False, add_reasoning_prompt=True)
reasoning_inputs = tokenizer(reasoning_template, return_tensors="pt").to(model.device)
reasoning_ids = model.generate(**reasoning_inputs, max_new_tokens=MAX_REASONING_TOKENS)
reasoning_output = tokenizer.decode(reasoning_ids[0, reasoning_inputs.input_ids.shape[1]:], skip_special_tokens=True)

# print("REASONING: " + reasoning_output)

# Generate answer
messages.append({"role": "reasoning", "content": reasoning_output})
response_template = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response_inputs = tokenizer(response_template, return_tensors="pt").to(model.device)
response_ids = model.generate(**response_inputs, max_new_tokens=MAX_RESPONSE_TOKENS)
response_output = tokenizer.decode(response_ids[0, response_inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("ANSWER: " + response_output)

Trained by: Nishith Jain
License: apache-2.0
Finetuned from model : Qwen/Qwen2.5-0.5B-Instruct
Dataset used : KingNish/reasoning-base-20k

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

RichardErkhov/KingNish_-_Reasoning-0.5b-gguf

Original model description:

Model Dexcription

RichardErkhov/
KingNish_-_Reasoning-0.5b-gguf