This repository contains a fine-tuned version of Google's Gemma 3 4B Instruct optimized for the Occitan language using DoRA (Weight-Decomposed Low-Rank Adaptation).

Repository Structure

Root Directory: Full merged Safetensors weights (bfloat16, compatible with transformers)
adapter/ Folder: The raw DoRA adapter files (for use with PEFT)
gguf/ Folder: Quantized GGUF versions (f16, Q2_K, Q4_K_M, Q5_K_M, Q8_0) for local inference via llama.cpp, LM Studio, Ollama, etc.

Usage

Using the merged model (Transformers)

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "julienp79/occitan-gemma-3-4b-it-dora-sfttrainer"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": "Escrivètz un cort paragraf en occitan sus la lenga occitana e son importància."},
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using the DoRA adapter with PEFT

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/gemma-3-4b-it",
    device_map="auto",
    torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained("unsloth/gemma-3-4b-it")

# Load adapter from the adapter/ subfolder
model = PeftModel.from_pretrained(base_model, "julienp79/occitan-gemma-3-4b-it-dora-sfttrainer", subfolder="adapter")
model.eval()

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Sètz un assistent utila que parla occitan."}],
    tokenize=False,
    add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using GGUF quantizations (llama.cpp)

bash

llama-cli -hf julienp79/occitan-gemma-3-4b-it-dora-sfttrainer:Q4_K_M \
  -p "<start_of_turn>user\nEscrivètz un cort paragraf en occitan.<end_of_turn>\n<start_of_turn>model\n" \
  -n 256 -e --temp 0.7

Or via the GGUF file directly:

bash

llama-cli -m gguf/occitan-gemma-3-4b-it-dora-sfttrainer-Q4_K_M.gguf \
  -p "<start_of_turn>user\nEscrivètz un cort paragraf en occitan.<end_of_turn>\n<start_of_turn>model\n" \
  -n 256 -e --temp 0.7

Adapter Details

Adapter Type: DoRA (Weight-Decomposed Low-Rank Adaptation, use_dora=True)
Base Model: unsloth/gemma-3-4b-it (Google Gemma 3 4B Instruct)
LoRA Rank (r): 16
LoRA Alpha: 32
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Dropout: 0
Training Method: SFT (Supervised Fine-Tuning) with SFTTrainer

Training Details

Block Size: 768 tokens
Learning Rate: 1e-4 (cosine schedule)
Optimizer: Paged AdamW 8-bit
Max Gradient Norm: 1.0
Warmup Steps: 200
Weight Decay: 0.01
Epochs: 5
Total Steps: ~950
Hardware: RTX 3060 12GB (4-bit base model with Unsloth's gradient checkpointing)

Training procedure

This model was trained with SFT (Supervised Fine-Tuning) using SFTTrainer on raw Occitan text (literary, journalistic, grammar, and encyclopedic sources). No chat templating was applied during training — the model learns language structure via causal language modeling on chunked text.

Framework versions

PEFT 0.19.1
TRL: 1.4.0
Transformers: 5.8.1
Pytorch: 2.6.0+cu124
Datasets: 4.8.5
Tokenizers: 0.22.2