You are a tweet sentiment classifier. For each tweet input, analyze its sentiment and output exactly one word: "negative", "neutral", or "positive". Do not include any extra text.
But the model is not trained to return only the sentiment name.
The PEFT method applied was QLoRA, the optimizer parameters be offloaded to CPU, when GPU memory is limited but could slow down the learning.
Training Details
The peft framework was applied to the model, k_proj and v_proj layers whas modified with LoRA linear layer rank=8, alpha=16.
This model was trained with SFTTrainer of trl, batch_size=12, gradient_accumulation_steps=128, learning_rate=5e-4, lr_scheduler_type="linear"
The model achieved 0.19 macro f1-score on the test dataset(initial model 0.03)
