"Built with Meta Llama 3".

Model Details

https://github.com/marcopoli/LLaMAntino-3-ANITA

Specifications

• Model developers:
  Ph.D. Marco Polignano - University of Bari Aldo Moro, Italy
  SWAP Research Group
  
• Variations: The model release has been supervised fine-tuning (SFT) using QLoRA 4bit, on instruction-based datasets. DPO approach over the mlabonne/orpo-dpo-mix-40k dataset is used to align with human preferences for helpfulness and safety.
• Input: Models input text only.
• Language: Multilingual 🏁 + Italian 🇮🇹
• Output: Models generate text and code only.
• Model Architecture: Llama 3 architecture.
• Context length: 8K, 8192.
• Library Used: Unsloth

Playground

Prompt Template

<|start_header_id|>system<|end_header_id|>

{ SYS Prompt }<|eot_id|><|start_header_id|>user<|end_header_id|>

{ USER Prompt }<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{ ASSIST Prompt }<|eot_id|>

Transformers

• Firstly, you need to install transformers via the command below with pip.
  pip install -U transformers trl peft accelerate bitsandbytes
• Right now, you can start using the model directly.
  python

  1import torch 2from transformers import ( 3 AutoModelForCausalLM, 4 AutoTokenizer, 5) 6 7base_model = "swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA" 8model = AutoModelForCausalLM.from_pretrained( 9 base_model, 10 torch_dtype=torch.bfloat16, 11 device_map="auto", 12) 13tokenizer = AutoTokenizer.from_pretrained(base_model) 14 15sys = "Sei un an assistente AI per la lingua Italiana di nome LLaMAntino-3 ANITA " \ 16 "(Advanced Natural-based interaction for the ITAlian language)." \ 17 " Rispondi nella lingua usata per la domanda in modo chiaro, semplice ed esaustivo." 18 19messages = [ 20 {"role": "system", "content": sys}, 21 {"role": "user", "content": "Chi è Carlo Magno?"} 22] 23 24#Method 1 25prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) 26inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False) 27for k,v in inputs.items(): 28 inputs[k] = v.cuda() 29outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, top_p=0.9, temperature=0.6) 30results = tokenizer.batch_decode(outputs)[0] 31print(results) 32 33#Method 2 34import transformers 35pipe = transformers.pipeline( 36 model=model, 37 tokenizer=tokenizer, 38 return_full_text=False, # langchain expects the full text 39 task='text-generation', 40 max_new_tokens=512, # max number of tokens to generate in the output 41 temperature=0.6, #temperature for more or less creative answers 42 do_sample=True, 43 top_p=0.9, 44) 45 46sequences = pipe(messages) 47for seq in sequences: 48 print(f"{seq['generated_text']}") 49
• Additionally, you can also use a model with 4bit quantization to reduce the required resources at least. You can start with the code below.
  python

  1import torch 2from transformers import ( 3 AutoModelForCausalLM, 4 AutoTokenizer, 5 BitsAndBytesConfig, 6) 7 8base_model = "swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA" 9bnb_config = BitsAndBytesConfig( 10 load_in_4bit=True, 11 bnb_4bit_quant_type="nf4", 12 bnb_4bit_compute_dtype=torch.bfloat16, 13 bnb_4bit_use_double_quant=False, 14) 15model = AutoModelForCausalLM.from_pretrained( 16 base_model, 17 quantization_config=bnb_config, 18 device_map="auto", 19) 20tokenizer = AutoTokenizer.from_pretrained(base_model) 21 22sys = "Sei un an assistente AI per la lingua Italiana di nome LLaMAntino-3 ANITA " \ 23 "(Advanced Natural-based interaction for the ITAlian language)." \ 24 " Rispondi nella lingua usata per la domanda in modo chiaro, semplice ed esaustivo." 25 26messages = [ 27 {"role": "system", "content": sys}, 28 {"role": "user", "content": "Chi è Carlo Magno?"} 29] 30 31#Method 1 32prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) 33inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False) 34for k,v in inputs.items(): 35 inputs[k] = v.cuda() 36outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, top_p=0.9, temperature=0.6) 37results = tokenizer.batch_decode(outputs)[0] 38print(results) 39 40#Method 2 41import transformers 42pipe = transformers.pipeline( 43 model=model, 44 tokenizer=tokenizer, 45 return_full_text=False, # langchain expects the full text 46 task='text-generation', 47 max_new_tokens=512, # max number of tokens to generate in the output 48 temperature=0.6, #temperature for more or less creative answers 49 do_sample=True, 50 top_p=0.9, 51) 52 53sequences = pipe(messages) 54for seq in sequences: 55 print(f"{seq['generated_text']}") 56

Evaluation

   lm_eval --model hf --model_args pretrained=HUGGINGFACE_MODEL_ID  --tasks hellaswag_it,arc_it  --device cuda:0 --batch_size auto:2
   lm_eval --model hf --model_args pretrained=HUGGINGFACE_MODEL_ID  --tasks m_mmlu_it --num_fewshot 5  --device cuda:0 --batch_size auto:2 

Unsloth

Citation instructions

Acknowledgments

We acknowledge the support of the PNRR project FAIR - Future AI Research (PE00000013), Spoke 6 - Symbiotic AI (CUP H97G22000210007) under the NRRP MUR program funded by the NextGenerationEU. Models are built on the Leonardo supercomputer with the support of CINECA-Italian Super Computing Resource Allocation, class C project IscrC_Pro_MRS (HP10CQO70G).

Open LLM Leaderboard Evaluation Results