This model was fine-tuned as part of Homework 3 in the HSE LLM Course.
It applies a custom DoRA implementation for sentiment classification, using standard causal language modeling training.

The model predicts a sentiment label (negative, neutral, or positive) by generating a short textual output conditioned on the input text.

Model Sources

Base model: https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct
Dataset: https://huggingface.co/datasets/cardiffnlp/tweet_eval

Training Method

Training follows a standard causal LM setup with frozen backbone weights and trainable DoRA adapters inserted into attention projection layers.

Training Hyperparameters

PEFT method: DoRA (custom implementation)
Rank: 8
DoRA alpha: 16
Target modules: attention projections (k_proj, v_proj)
Batch size: 32
Learning rate: 5e-4
Optimizer: AdamW
Precision: FP16 (mixed precision)
Epochs: 3

Trainable parameters: ~0.14% of total model parameters.

Results

Macro F1: ~0.5 on the test set

sodeniZz/llm-course-hw3-dora

Model Card: llm-course-hw3-dora

Model Sources

Training Method

Training Hyperparameters

Results

sodeniZz/
llm-course-hw3-dora