LoRa adapters for meta-llama/Llama-3.1-8B-Instruct (Built with Llama) obtained through a finetuning process (using LoRA technique) aimed at making the model capable of attributing meaningful tags to a news article written in Italian, English, German or Slovenian.

Model Description

The model resulting from the application of the adapters of this repository and the base model meta-llama/Llama-3.1-8B-Instruct is optimized to perform the specific task of assign appropriate tags to a given news article. Because of the finetuning process it is important to respect the prompt template in order to get good results.

Developed by: Stefano Scotta (stefano.scotta@rai.it)
Model type: LLM finetuned on the specific task of assign tags to news articles
Language(s) (NLP): Italian, English, German, Slovenian
License: llama3.1
Finetuned from model [optional]: meta-llama/Llama-3.1-8B-Instruct

Uses

The model can be used to assign representative tags to a given news article.

Bias, Risks, and Limitations

As any other LLM it is possible that the model generates content which does not correspond to the reality as well as wrong, biased, offensive and inappropriate answers.

How to Get Started with the Model

The prompt must be passed to the model following the meta-llama/Llama-3.1-8B-Instruct chat template,

Usage: Use the code below to use the model.

python

import torch
import transformers
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, pipeline
import json

peft_model_id = "raicrits/Llama_NewsTagger_XRECO"
base_model_id = "meta-llama/Llama-3.1-8B-Instruct"


base_model = AutoModelForCausalLM.from_pretrained(base_model_id, 
                                            quantization_config=quantization_config, 
                                            device_map='auto')
model = PeftModel.from_pretrained(base_model, peft_model_id)
model.config.use_cache = True


tokenizer = AutoTokenizer.from_pretrained(peft_model_id, add_eos_token=False)
tokenizer.pad_token = tokenizer.eos_token
terminators = [
   tokenizer.eos_token_id,
   tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

def generate_prompt_tag(title, subtitle, text):
   messages = [
       {"role": "system", "content": "You are a multilanguage AI assistant aimed to suggest tags in news articles."},
       {"role": "user", "content": f"""Analyze the following news article and assign a list of representative tags to the content, returning always at least 3.
   
Title: {title}

Subtitle: {subtitle}

Text: {text}

The response has to be given in the following format "[<tag1>, <tag2>,.. ]", where the <tag>s are the tags identified and written in the same language of the article. Do no add any further text."""}
]
   prompt = tokenizer.apply_chat_template(
       messages,
       add_generation_prompt=True,
       return_tensors="pt",
       return_dict=True).to(model.device)
   return prompt

prompt = generate_prompt_tag(<title>, <subtitle>, <text>)    #INSERT NEWS ARTICLE TITLE, SUBTITLE AND TEXT HERE

outputs = model.generate(
           input_ids=prompt['input_ids'], attention_mask=prompt['attention_mask'], 
           eos_token_id=terminators,
           pad_token_id=tokenizer.eos_token_id
           )
response = outputs[0][prompt['input_ids'].shape[-1]:]

tags = tokenizer.decode(response, skip_special_tokens=True).replace("\n","").replace("[","").replace("]", "").replace("'","").replace('"','')

Training Details

Training Data

The model was fine-tuned on a multilanguage collection of news articles.

Training Procedure

The fine-tuning procedure was done using LoRA approach on the 8-bit quantized meta-llama/Llama-3.1-8B-Instruct model.

Training setting:

train epochs=4,
learning_rate=1e-05

LoRA configuration:

r= 8
lora_alpha=16
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"]
lora_dropout=0.1
bias="none"
task_type=TaskType.CAUSAL_LM

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: 1 NVIDIA A100/40Gb
Hours used: 63
Cloud Provider: Private Infrastructure
Carbon Emitted: 6.8kg eq. CO2

Acknowledgment

This work has received funding from the European Union’s Horizon Europe Research and Innovation Programme under grant agreement No 101070250 XReco.

Model Card Authors

Stefano Scotta (stefano.scotta@rai.it)

Model Card Contact

stefano.scotta@rai.it