This model is a Supervised Fine-Tuned (SFT) version of Qwen2.5-1.5B, trained using Unsloth with LoRA adapters for parameter-efficient fine-tuning on limited hardware.

The goal of this model is to learn improved instruction-following behavior from curated prompt–response examples.
It serves as a foundation checkpoint for future experimentation, including reward modeling and reinforcement learning (RLHF / GRPO).

Developed by: Shreshth Sudhir
Model type: Supervised Fine-Tuned (SFT) Language Model
Language(s): English
License: Same as base model (Qwen2.5 license)
Finetuned from: unsloth/qwen2.5-1.5b-unsloth-bnb-4bit

Uses

Direct Use

This model can be used for:

Instruction-following text generation
Educational explanations
Conversational responses
Learning and experimentation with LoRA-based fine-tuning

Downstream Use

This model is intended to be further fine-tuned for:

Reward modeling
Reinforcement Learning with Human Feedback (RLHF / GRPO)
Style or domain adaptation

Out-of-Scope Use

This model is not suitable for:

Medical, legal, or financial advice
Safety-critical or production systems
High-stakes decision making

Bias, Risks, and Limitations

The model may reflect biases present in the base model and training data
No explicit safety or alignment tuning has been applied
Generated responses may be inaccurate or incomplete

Users should apply human oversight when using this model.

How to Get Started

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="./my_sft_model",
    max_seq_length=2048,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(model)

prompt = "Explain gravity simply."
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    top_p=0.9,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Custom instruction–response dataset created for learning supervised fine-tuning
Data formatted using an instruction–response prompt template
No proprietary, private, or sensitive data was used

Training Procedure

Training method: Supervised Fine-Tuning (SFT)
Fine-tuning strategy: LoRA (Low-Rank Adaptation)
Frameworks: Unsloth, TRL, Hugging Face Transformers
Base model: unsloth/qwen2.5-1.5b-unsloth-bnb-4bit

Training Hyperparameters

Precision: 4-bit quantization (bnb 4-bit)
LoRA rank (r): 16
Optimizer: AdamW
Training regime: Mixed precision training

Speeds, Sizes, and Times

Training performed on a single GPU
Parameter-efficient fine-tuning enabled via LoRA
Short-duration experimental training run

Framework versions

PEFT 0.18.0

Shreshthh/my-sft-model

my-sft-model

Model Details

Model Description