This model is a fine-tuned version of unsloth/tinyllama-chat-bnb-4bit on the English-Hindi Dataset (Sadiya025/english-hindi-dataset). It achieves the following results on the evaluation set:

Loss: 11.8701

Model description

Base Model: unsloth/tinyllama-chat-bnb-4bit
Dataset: Sadiya025/english-hindi-dataset
Task: English-to-Hindi Colloquial Translation
Quantization: 4-bit for optimized inference

Training and evaluation data

The model was fine-tuned on the Sadiya025/english-hindi-dataset, which consists of English sentences and their corresponding colloquial Hindi translations.

Training procedure

Base Model: unsloth/tinyllama-chat-bnb-4bit
Frameworks:
- PEFT 0.14.0
- Transformers 4.49.0
- PyTorch 2.6.0+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0
Fine-tuning Method: Parameter-Efficient Fine-Tuning (PEFT)
Precision: 4-bit quantized model for memory efficiency

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
num_epochs: 15
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
3.5793	1.0	462	12.4256
3.3523	2.0	924	12.0382
3.4293	3.0	1386	11.9843
3.4061	4.0	1848	12.0250
3.1875	5.0	2310	11.9591
3.3198	6.0	2772	11.9267
3.4656	7.0	3234	11.9042
3.4545	8.0	3696	11.9408
3.2797	9.0	4158	11.9250
3.2077	10.0	4620	11.9467
3.3623	11.0	5082	11.8813
3.1617	12.0	5544	11.8757
3.1431	13.0	6006	11.8749
3.1546	14.0	6468	11.8825
3.2779	14.9686	6915	11.8701

Framework versions

PEFT 0.14.0
Transformers 4.49.0
Pytorch 2.6.0+cu124
Datasets 3.2.0
Tokenizers 0.21.0

Sadiya025/english-hindi-translator

english-hindi-translator