Llama3-3B Uruchi Instruct
Instruction-tuned version of Llama-3-3B optimized for concise, direct answers.
This model was fine-tuned using LoRA + Unsloth on a mixture of instruction datasets.
Model Overview
| Attribute | Value |
|---|
| Model Name | llama3-3b-uruchi-instruct |
| Base Model | meta-llama/Llama-3-3B |
| Parameters | 3B |
| Fine-tuning Method | LoRA |
| Framework | Unsloth |
| Model Format | safetensors |
| Author | Irfan Uruchi |
Training
The model was fine-tuned using LoRA adapters with the Unsloth training framework for efficient GPU usage.
Training used a cleaned and merged instruction dataset containing approximately 107k samples.
Training configuration include LoRA fine-tuning with instruction-style prompts and its optimized for concise responses with dataset continuation filtering
Training Datasets
The training dataset consists of a mixture of open instruction datasets:
UltraChat
High-quality conversational instruction dataset.
OpenOrca
Reasoning and explanation dataset derived from GPT-style instruction generation.
GSM8K
Math reasoning dataset used to improve logical reasoning capabilities.
Final dataset size: ~107k instruction samples
Some further datasets were created by me for better Instruction Alignment / SFT Refinement.
Datasets were cleaned and filtered to remove malformed samples and dataset artifacts.
Prompt Format
The model expects the following prompt structure:
You are a concise AI assistant.
Answer the user's question clearly and directly.
User question:
{question}
Answer:
Example:
User question:
What is 2+2?
Answer:
4
(a test_model.py will be included in repo for easier testing)
Example Usage
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_id = "irfanuruchi/llama3-3b-uruchi-instruct"
4
5tokenizer = AutoTokenizer.from_pretrained(model_id)
6model = AutoModelForCausalLM.from_pretrained(model_id)
7
8prompt = """
9You are a concise AI assistant.
10
11User question:
12Explain machine learning in one sentence.
13
14Answer:
15"""
16
17inputs = tokenizer(prompt, return_tensors="pt")
18
19outputs = model.generate(
20 **inputs,
21 max_new_tokens=50,
22)
23
24print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Intended Behavior
This model is optimized for concise answers, factual responses, small amount of hallucination and simple explanations
Limitations
- Small model size (3B parameters)
- Limited deep reasoning capability
- Not optimized for coding tasks
- Context length limitations
License
This model is a derivative of Llama-3 and follows the Llama 3 Community License.
Base model:
meta-llama/Llama-3-3B
Please follow the original license terms when using this model.
