You can deploy this model and start earning money today!

Hedi-Bk/
gemma-4b-tunisian-law-lora-2epochs

Name: gemma-4b-tunisian-law-lora-2epochs
Availability: InStock

safetensors

Views

No views yet

🇹🇳 Gemma 4B Tunisian Law LoRA (2 Epochs)

LoRA adapters fine-tuned from Gemma 4B for Tunisian law and legal instruction tasks.

This repository contains LoRA adapters only and does not include the base model weights.

📌 Model Details

Base Model: Gemma 4B
Training Method: QLoRA
Training Epochs: 2
Final Training Loss: 0.1075
Domain: Tunisian Law
Languages: Arabic / French
Repository: Hedi-Bk/gemma-4b-tunisian-law-lora-2epochs

🚀 Training Notebook

Kaggle notebook used for training:

Kaggle Training Notebook

🧠 About the Model

This model was trained using the QLoRA method for legal-oriented tasks related to Tunisian law.

Despite the very low final loss (0.1075), the model showed very good generalization capabilities and strong response quality during testing.

The model did not exhibit problematic overfitting behavior during evaluation.

⚙️ Usage Example

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "google/gemma-4b"
adapter_model = "Hedi-Bk/gemma-4b-tunisian-law-lora-2epochs"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    model,
    adapter_model
)

prompt = "Explique le droit tunisien du travail."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

⚠️ Important

This repository contains LoRA adapters only.

You must first download the base Gemma model before loading these adapters.

⚠️ Disclaimer

This project is intended for research and educational purposes only.

Generated legal content may contain inaccuracies and should not be considered professional legal advice.

🙌 Credits

Google for the Gemma models
Hugging Face
Kaggle