Модель OuteAI/Lite-Oute-1-300M-Instruct дообученная на датасете cardiffnlp/tweet_eval, задача классификации сентимента твита, вывести одно из трех слов -
negative, neutral, positive.
Дообучение
Модель дообучалась при помощи DoRA.
Ранг LoRA = 16
alpha=32
DoRA применялась только к весам Key, Value в attention
BATCH_SIZE = 16
LEARNING_RATE = 2e-4
NUM_EPOCHS = 2
AdamW
weight_decay=0.01
Метрика на валидации
F1=0.51
image/png
Tweet: "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin" Label: positive Output:
positive
positive
positive
pos
Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ" Label: neutral Output:
neutral
neutral
neutral
neut
Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night. Label: neutral Output:
neutral
positive
positive
pos
Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays Label: neutral Output:
neutral
neutral
neutral
neut
Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017" Label: positive Output:
neutral
negative
negative
negative
