kth8/

gemma-3-1b-it-Conversation-GGUF

A fine-tune of unsloth/gemma-3-1b-it on the kth8/multi-turn-conversation-50000x dataset.

Usage example

Model Details

• Base Model: unsloth/gemma-3-1b-it
• Parameter Count: 999885952
• Precision: torch.bfloat16

Training Settings

Hardware

• GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition

PEFT

• Rank: 32
• LoRA alpha: 64
• Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Gradient checkpointing: unsloth

SFT

• Epoch: 2
• Batch size: 48
• Gradient Accumulation steps: 1
• Warmup ratio: 0.1
• Learning rate: 0.0002
• Optimizer: adamw_torch_fused
• Learning rate scheduler: cosine

Training stats

• Global step: 1996
• Training runtime (seconds): 6834.1445
• Average training loss: 1.1743444665400442
• Final validation loss: 1.1191450357437134

Framework versions

• Unsloth: 2026.3.8
• TRL: 0.22.2
• Transformers: 4.56.2
• Pytorch: 2.10.0+cu128
• Datasets: 4.8.3
• Tokenizers: 0.22.2

License