Continuing my efforts to finetune tiny LLMs on hyperspecific tasks, I've decided to use the newly released Google Gemma 3 as my base model for French Grammar Explanation finetuning. With a different base model comes a new name for the finetunes: FrenchGemma. Similarly to Meta Llama 3.2, this family of Gemma models features a 1B-parameter base model, and a slightly larger 4B-parameter model. This model repository is dedicated to the 4B-Instruct version.

python

training_args = TrainingArguments(
    output_dir="./gemma_fine_tuned",
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    learning_rate=1e-4,
    num_train_epochs=3,
    weight_decay=0.01,
    warmup_steps=50,
    save_steps=100,
    logging_steps=20,
    fp16=False,
    bf16=True,
    save_total_limit=2,
    report_to='none',
    gradient_checkpointing=True
)

CreativeAlloyYT/FrenchGemma-3-4B-Instruct

FrenchGemma - The French Grammar Tutor

CreativeAlloyYT/
FrenchGemma-3-4B-Instruct