You can deploy this model and start earning money today!

ryvarma7/
faultline_SFT_GRPO

Name: faultline_SFT_GRPO
Availability: InStock

Views

No views yet

Model Card: faultline_sft

Model Description

This model is a Supervised Fine-Tuned (SFT) version of Qwen2.5-3B-Instruct, trained using instruction-response pairs to improve task-specific performance.

The model is optimized for:

Instruction following
Structured reasoning
Domain-specific conversational responses

Model Details

Developed by: Yeshwant Varma
Model type: Causal Language Model (LLM)
Base model: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
Fine-tuning method: LoRA (PEFT)
Framework: Hugging Face Transformers + TRL + Unsloth
Language: English
License: Apache 2.0 (inherits from base model)

Training Details

Training Data

Instruction-response dataset (custom curated)
Cleaned and formatted into chat template
Removed noisy/incomplete samples

Training Procedure

Method: Supervised Fine-Tuning (SFT)
Optimization: AdamW
Precision: 4-bit (bnb) + LoRA adapters

Hyperparameters

Epochs: 1–3 (typical)
Batch size: 2–8 (depending on VRAM)
Learning rate: ~2e-4
LoRA rank (r): 16–64

Intended Use

Direct Use

Chatbots
Instruction-following systems
NLP applications

Out-of-Scope Use

High-risk domains (medical/legal advice)
Autonomous decision-making systems

Limitations

Limited reasoning depth compared to larger models
May hallucinate in complex queries
Performance depends heavily on training dataset quality

How to Use

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_path = "path_to_model"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)

input_text = "Explain AI in simple terms"
inputs = tokenizer(input_text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

ryvarma7/faultline_SFT_GRPO