Description
This model was obtained by fine-tuning the Llama-2 7B large language model with the LoRA technique. The aim is to develop a sentiment analysis system in Turkish language by training the model according to the sentences in the given data set.
The evaluation metrics of the model were calculated and the following results were obtained.
Dataset
The training data set consists of 152715 rows and the eval data set consists of 16968 rows. It includes social media posts and product reviews.
Uses
from transformers import AutoConfig
from transformers import AutoModelForSequenceClassification
config = AutoConfig.from_pretrained("Minekorkmz/model_yurt_1200")
num_labels = config.num_labels
base_model = AutoModelForSequenceClassification.from_pretrained(
"meta-llama/Llama-2-7b-chat-hf",
num_labels=num_labels
)
model = PeftModel.from_pretrained(base_model, "Minekorkmz/model_yurt_1200")
tokenizer = AutoTokenizer.from_pretrained("Minekorkmz/model_yurt_1200")
from transformers import pipeline
sentiment_task = pipeline("sentiment-analysis",
model=model,
tokenizer=tokenizer,
return_all_scores=True)
print(sentiment_task("çok kötü bir ürün oldu sevemedim"))
Training procedure
The following bitsandbytes quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: float16
Framework versions
- PEFT 0.4.0
- accelerate 0.26.0
- bitsandbytes 0.41.1
- transformers 4.35.0
- trl 0.4.7
