This repository contains LoRA (Low-Rank Adaptation) adapters for the unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit model, fine-tuned on a cleaned Persian conversational dataset.

This model was fine-tuned using the powerful Unsloth library, which enables significantly faster training and reduces memory usage, making it possible to fine-tune on consumer-grade hardware like a single Google Colab T4 GPU.

Model Description

Base Model: unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit
Language: Persian (fa)
Fine-tuning Dataset: mshojaei77/persian-gk-cleaned
Framework: Unsloth

The model is designed to be a helpful Persian-speaking assistant, knowledgeable in topics covered by the training dataset, including technology, general knowledge, and conversational queries.

🚀 How to Use

To use these LoRA adapters, you must first load the base unsloth/gemma-3n-E4B-it model and then apply the adapters from this repository. The Unsloth library is required for this process.

Installation

First, ensure you have Unsloth and all necessary dependencies installed.

bash

# Install Unsloth for Google Colab
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers "trl<0.9.0" peft accelerate bitsandbytes

Inference Code

This code snippet demonstrates how to load the base model, attach the LoRA adapters, and run inference.

python

from unsloth import FastLanguageModel
import torch
from transformers import TextStreamer

# Your LoRA adapter repository
lora_model_name = "mshojaei77/gemma-3n-E4B-persin-lora-adaptors"

# Load the base model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = lora_model_name, # Use your LoRA repo name
    max_seq_length = 4096,
    dtype = None,
    load_in_4bit = True,
)

# Set up a text streamer for continuous output
streamer = TextStreamer(tokenizer, skip_prompt=True)

# Prepare your prompt
messages = [{
    "role": "user",
    "content": "Node.js چیه و چه کاربردی داره؟",
}]

# Format and tokenize the input
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

# Run generation
_ = model.generate(
    **inputs,
    max_new_tokens=256,
    streamer=streamer,
    use_cache=True,
    # Recommended Gemma-3 settings
    temperature=0.7,
    top_p=0.95,
    top_k=64,
)

Training Details

The model was fine-tuned with the following configuration:

Framework

Library: Unsloth
Hardware: Google Colab T4 GPU (15GB VRAM)

Dataset

The model was trained on the mshojaei77/persian-gk-cleaned dataset, which is a curated version of the original mshojaei77/persian-gk dataset. The cleaning process involved:

Removing system messages.
Ensuring strict user/assistant alternating roles.
Filtering out empty or malformed conversations.

Hyperparameters

LoRA r: 8
LoRA alpha: 16
Learning Rate: 2e-5
Epochs: 1
Batch Size: 2
Gradient Accumulation: 4 (Effective Batch Size: 8)
Optimizer: AdamW 8-bit
LR Scheduler: Linear

Citation

If you use this model, please consider citing the original creators and tools that made it possible:

Gemma 3N:

@misc{gemma3n,
  author = {The Gemma 3N team, Google},
  title = {Gemma 3N},
  year = {2024},
  howpublished = {\url{https://ai.google.dev/gemma}},
}

Unsloth:

@misc{unsloth,
  author = {Daniel Han and Phil Wang},
  title = {Unsloth: Llama, Mistral & Gemma 5x faster training},
  year = {2024},
  howpublished = {\url{https://github.com/unslothai/unsloth}},
}