This model is fine-tuned to parse multi-step problems by breaking down its internal analysis into distinct <|begin_of_thought|> and <|begin_of_solution|> blocks.
To trigger the Chain-of-Thought pathway correctly, provide the following system prompt in your chat template:
1Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions.
2This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process.
3Please structure your response into two main sections: Thought and Solution.
4In the Thought section, detail your reasoning process using the specified format: <|begin_of_thought|> {thought with steps separated with '\n\n'} <|end_of_thought|> Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps.
5In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows: <|begin_of_solution|> {final formatted, precise, and clear solution} <|end_of_solution|> Now, try to solve the following question through the above guidelines:
⚡ Quickstart & Usage⚙️ Recommended Inference Hyperparameters[!IMPORTANT]Because 1B parameter reasoning models are prone to infinite repetition during extended thinking phases, repetition_penalty=1.15 and temperature=0.1 are strictly required for output stability. ParameterRecommended ValueReasontemperature0.1Ensures deterministic, logically consistent CoT tracesrepetition_penalty1.15Prevents infinite loops inside long reasoning blockstop_p0.9Filters out low-probability token tailsmax_new_tokens4096Accommodates lengthy multi-step reasoningPython Inference Code (Unsloth) Pythonimport torch
from unsloth import FastLanguageModel
from transformers import TextStreamer
1 model_name="harshwardhanjadhav/llama3.2-1b-CoT",
2 max_seq_length=8192,
3 dtype=torch.float16,
4 load_in_4bit=False,
5)
6FastLanguageModel.for_inference(model)
1 {
2 "role": "system",
3 "content": "Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. Enclose your internal reasoning within <|begin_of_thought|> and <|end_of_thought|> tags."
4 },
5 {
6 "role": "user",
7 "content": "A store offers a 20% discount on a $150 coat, then takes off an additional 10% from the sale price. What is the final price? Think step by step."
8 }
9]
10
11inputs = tokenizer.apply_chat_template(
12 messages,
13 tokenize=True,
14 add_generation_prompt=True,
15 return_tensors="pt"
16).to("cuda")
1
2_ = model.generate(
3 input_ids=inputs,
4 streamer=streamer,
5 max_new_tokens=4096,
6 temperature=0.1,
7 repetition_penalty=1.15,
8 top_p=0.9,
9 use_cache=True,
10)
This llama model was trained 2x faster with
Unsloth and Huggingface's TRL library.
📊 Training DetailsDataset: 10,000 samples from open-thoughts/OpenThoughts-114kSequence Length: 8,192 tokensPrecision: Mixed Precision (fp16 / bf16)Optimization: Fine-tuned 2x faster using Unsloth GPU kernels⚠️ Limitations & Usage AdviceParameter Scale: Being a 1B parameter model, it may struggle with highly abstract mathematical theorems or multi-nested algorithmic code generation compared to 8B+ or 70B models.Context Window: Keep total context (Prompt + Thinking Trace + Solution) under 8,192 tokens.
