Fine-tuned LoRA adapter on Gemma 2 2B for mental wellness conversations in Marathi-English mix. Built for SoulTalk — AI mental wellness platform for Indian users.

Model Details

Developed by: Aishhh369
Base model: unsloth/gemma-2-2b-bnb-4bit
Model type: LoRA adapter (PEFT)
Language: Marathi + English (Hinglish style)
License: CC BY 4.0
Fine-tuning framework: Unsloth + TRL (SFTTrainer)
Hardware: Kaggle T4 GPU

Training Data

Dataset: SoulTalk Marathi Mental Wellness Dataset
250 multi-turn conversations, 1242 user-assistant pairs
10 categories: Career Stress, Loneliness, Grief/Loss, Health Anxiety, Social Anxiety, Academic Pressure, Family Conflict, Burnout, Relationship Issues, Self-Esteem

Training Hyperparameters

Epochs: 3
Learning rate: 2e-4
Batch size: 1 (gradient accumulation: 4)
Max sequence length: 1024
Optimizer: adamw_8bit
Quantization: 4-bit (bnb)

How to Use

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "unsloth/gemma-2-2b-bnb-4bit"
tokenizer = AutoTokenizer.from_pretrained("Aishhh369/soultalk-lora")
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, "Aishhh369/soultalk-lora")

prompt = "Career chi khup chinta vattey"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0]))

Intended Use

Mental wellness chatbot for Indian users
Marathi/Hindi/English multilingual support
Empathetic response generation

Limitations

Fine-tuned on synthetic data, real-world performance may vary
Not a substitute for professional mental health care
Crisis situations should be redirected to helplines