kth8/

gemma-3-270m-it-Unhelpful

A fine-tune of unsloth/gemma-3-270m-it on the kth8/no-as-a-service dataset.

Usage example

Model Details

• Base Model: unsloth/gemma-3-270m-it
• Parameter Count: 268098176
• Precision: torch.float16

Hardware

• GPU: Tesla T4

Training Settings

PEFT

• Rank: 32
• LoRA alpha: 64
• Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• LoRA dropout: 0.05
• Rank-Stabilized LoRA: False

SFT

• Epoch: 1
• Warmup ratio: 0.1
• Learning rate: 0.0002
• Optimizer: adamw_torch_fused
• Weight decay: 0.01
• Learning rate scheduler: cosine

Training stats

• Global step: 2672
• Training runtime: 7053.7
• Average training loss: 2.761223313069629
• Final validation loss: 2.6234915256500244

Framework versions

• Unsloth: 2026.3.5
• TRL: 0.22.2
• Transformers: 4.56.2
• Pytorch: 2.10.0+cu128
• Datasets: 4.3.0
• Tokenizers: 0.22.2

License