You can deploy this model and start earning money today!

RichardErkhov/
swap-uniba_-_LLaMAntino-2-chat-7b-hf-UltraChat-ITA-gguf

Name: swap-uniba_-_LLaMAntino-2-chat-7b-hf-UltraChat-ITA-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

LLaMAntino-2-chat-7b-hf-UltraChat-ITA - GGUF

Model creator: https://huggingface.co/swap-uniba/
Original model: https://huggingface.co/swap-uniba/LLaMAntino-2-chat-7b-hf-UltraChat-ITA/

Name	Quant method	Size
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q2_K.gguf	Q2_K	2.36GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.IQ3_XS.gguf	IQ3_XS	2.6GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.IQ3_S.gguf	IQ3_S	2.75GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q3_K_S.gguf	Q3_K_S	2.75GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.IQ3_M.gguf	IQ3_M	2.9GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q3_K.gguf	Q3_K	3.07GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q3_K_M.gguf	Q3_K_M	3.07GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q3_K_L.gguf	Q3_K_L	3.35GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.IQ4_XS.gguf	IQ4_XS	3.4GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q4_0.gguf	Q4_0	3.56GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.IQ4_NL.gguf	IQ4_NL	3.58GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q4_K_S.gguf	Q4_K_S	3.59GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q4_K.gguf	Q4_K	3.8GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q4_K_M.gguf	Q4_K_M	3.8GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q4_1.gguf	Q4_1	3.95GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q5_0.gguf	Q5_0	4.33GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q5_K_S.gguf	Q5_K_S	4.33GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q5_K.gguf	Q5_K	4.45GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q5_K_M.gguf	Q5_K_M	4.45GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q5_1.gguf	Q5_1	4.72GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q6_K.gguf	Q6_K	5.15GB
LLaMAntino-2-chat-7b-hf-UltraChat-ITA.Q8_0.gguf	Q8_0	6.67GB

Original model description:

license: llama2 language:

it tags:
text-generation-inference

Model Card for LLaMAntino-2-chat-7b-UltraChat-ITA

Last Update: 08/01/2024
Example of Use: Colab Notebook

Model description

LLaMAntino-2-chat-7b-UltraChat is a Large Language Model (LLM) that is an instruction-tuned version of LLaMAntino-2-chat-7b (an italian-adapted LLaMA 2 chat). This model aims to provide Italian NLP researchers with an improved model for italian dialogue use cases.

The model was trained using QLora and using as training data UltraChat translated to the italian language using Argos Translate. If you are interested in more details regarding the training procedure, you can find the code we used at the following link:

Repository: https://github.com/swapUniba/LLaMAntino

NOTICE: the code has not been released yet, we apologize for the delay, it will be available asap!

Developed by: Pierpaolo Basile, Elio Musacchio, Marco Polignano, Lucia Siciliani, Giuseppe Fiameni, Giovanni Semeraro
Funded by: PNRR project FAIR - Future AI Research
Compute infrastructure: Leonardo supercomputer
Model type: LLaMA-2-chat
Language(s) (NLP): Italian
License: Llama 2 Community License
Finetuned from model: swap-uniba/LLaMAntino-2-chat-7b-hf-ITA

Prompt Format

This prompt format based on the LLaMA 2 prompt template adapted to the italian language was used:

python

" [INST]<<SYS>>\n" \
"Sei un assistente disponibile, rispettoso e onesto. " \
"Rispondi sempre nel modo piu' utile possibile, pur essendo sicuro. " \
"Le risposte non devono includere contenuti dannosi, non etici, razzisti, sessisti, tossici, pericolosi o illegali. " \
"Assicurati che le tue risposte siano socialmente imparziali e positive. " \
"Se una domanda non ha senso o non e' coerente con i fatti, spiegane il motivo invece di rispondere in modo non corretto. " \
"Se non conosci la risposta a una domanda, non condividere informazioni false.\n" \
"<</SYS>>\n\n" \
f"{user_msg_1}[/INST] {model_answer_1} </s> <s> [INST]{user_msg_2}[/INST] {model_answer_2} </s> ... <s> [INST]{user_msg_N}[/INST] {model_answer_N} </s>"

We recommend using the same prompt in inference to obtain the best results!

How to Get Started with the Model

Below you can find an example of model usage:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "swap-uniba/LLaMAntino-2-chat-7b-hf-UltraChat-ITA"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

user_msg = "Ciao! Come stai?"

prompt = " [INST]<<SYS>>\n" \
         "Sei un assistente disponibile, rispettoso e onesto. " \
         "Rispondi sempre nel modo piu' utile possibile, pur essendo sicuro. " \
         "Le risposte non devono includere contenuti dannosi, non etici, razzisti, sessisti, tossici, pericolosi o illegali. " \
         "Assicurati che le tue risposte siano socialmente imparziali e positive. " \
         "Se una domanda non ha senso o non e' coerente con i fatti, spiegane il motivo invece di rispondere in modo non corretto. " \
         "Se non conosci la risposta a una domanda, non condividere informazioni false.\n" \
         "<</SYS>>\n\n" \
         f"{user_msg}[/INST]"

pipe = transformers.pipeline(
    model=model,
    tokenizer=tokenizer,
    return_full_text=False, # langchain expects the full text
    task='text-generation',
    max_new_tokens=512, # max number of tokens to generate in the output
    temperature=0.8  #temperature for more or less creative answers
)

# Method 1
sequences = pipe(text)
for seq in sequences:
    print(f"{seq['generated_text']}")

# Method 2
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
outputs = model.generate(input_ids=input_ids, max_length=512)
print(tokenizer.batch_decode(outputs.detach().cpu().numpy()[:, input_ids.shape[1]:], skip_special_tokens=True)[0])

If you are facing issues when loading the model, you can try to load it Quantized:

model = AutoModelForCausalLM.from_pretrained(model_id, load_in_8bit=True)

Note:

The model loading strategy above requires the bitsandbytes and accelerate libraries
The Tokenizer, by default, adds at the beginning of the prompt the '<BOS>' token. If that is not the case, add as a starting token the <s> string.

Evaluation

Coming soon!

Citation

If you use this model in your research, please cite the following:

bibtex

@misc{basile2023llamantino,
      title={LLaMAntino: LLaMA 2 Models for Effective Text Generation in Italian Language}, 
      author={Pierpaolo Basile and Elio Musacchio and Marco Polignano and Lucia Siciliani and Giuseppe Fiameni and Giovanni Semeraro},
      year={2023},
      eprint={2312.09993},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

RichardErkhov/swap-uniba_-_LLaMAntino-2-chat-7b-hf-UltraChat-ITA-gguf