The training dataset can be modified, see original at customer support dataset .. I edited the system message with a bit of prompt engineering, included additional details about the eCommerce company.
You can decide what you want and further fine tune the model...

Training and evaluation data

Training data.
Used the complete dataset for training, no evaluation data, I evaluated with random prompts...

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 1
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 2
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 682
num_epochs: 1

Training results

Training Loss from wandb

Framework versions

PEFT 0.13.2
Transformers 4.45.2
Pytorch 2.4.1+cu121
Datasets 3.0.1
Tokenizers 0.20.1

Victorano/llama-3.2-1B-it-Procurtech-Assistant

llama-3.2-1B-it-Procurtech-Assistant

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Victorano/
llama-3.2-1B-it-Procurtech-Assistant