Gemma 3-4B Persian (v0)

mshojaei77/gemma-3-4b-persian-v0 is a Persian-specialized model built on the Gemma 3 architecture. It leverages QLoRA for 4-bit quantization to reduce computational overhead while generating and understanding Persian text. In addition to text generation, the model also retains image input capabilities inherited from its base model.

Usage

Running with Ollama

Running with Hugging Face Transformers

1. Install Dependencies:
   pip install git+https://github.com/huggingface/transformers@v4.49.0-Gemma-3 accelerate
2. Load Model and Tokenizer:
   python

   1from transformers import AutoModelForCausalLM, AutoTokenizer 2import torch 3 4model_id = "mshojaei77/gemma-3-4b-persian-v0" 5 6model = AutoModelForCausalLM.from_pretrained( 7 model_id, 8 device_map="auto", # Use "cuda" for GPU usage if available 9 torch_dtype=torch.bfloat16, # Alternatively, use torch.float16 10) 11tokenizer = AutoTokenizer.from_pretrained(model_id) 12 13messages = [ 14 { 15 "role": "user", 16 "content": "توماس جفرسون کیست؟" 17 } 18] 19inputs = tokenizer.apply_chat_template( 20 messages, 21 add_generation_prompt=True, tokenize=True, return_tensors="pt" 22).to(model.device) 23 24outputs = model.generate(**inputs, max_new_tokens=200) 25print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data and Fine-Tuning

Training Dataset

Fine-Tuning

• Method: Supervised Fine-Tuning (SFT) using QLoRA (4-bit quantization)
• Hardware: one T4 GPU
• Software: Utilizes Hugging Face Transformers, with supporting libraries like peft for QLoRA and bitsandbytes for quantization
• Trade-offs: Reduced memory footprint at the expense of some precision compared to full-precision models

Evaluation

Usage Considerations and Limitations

Intended Use Cases

• Question Answering: Responding accurately to Persian language queries
• Instruction Following: Interpreting and executing text-based instructions in Persian
• Text Generation: Producing fluent, context-aware Persian content
• Conversational AI: Integrating into chatbots and virtual assistants
• Image Processing: Retaining image input capabilities from the base model

Limitations

• Quantization Impact: 4-bit quantization may reduce output precision and result in occasional incoherent responses.
• Evaluation Scope: Absence of comprehensive evaluation metrics specific to this variant.
• Bias: The model might mirror biases present in both the original Gemma 3 data and the Persian_sft dataset.
• Hallucination: As with all LLMs, there is a risk of generating plausible-sounding but inaccurate information.
• Safety: The model has not undergone safety tuning, so extra caution is advised when deploying in sensitive contexts.

Maintenance and Future Work

• Additional evaluation metrics and benchmarks
• Enhanced safety tuning and bias mitigation strategies
• Expanded documentation and usage examples
• Incorporation of community feedback for iterative improvements