You can deploy this model and start earning money today!

RichardErkhov/
projecte-aina_-_salamandra-7b-aligned-EADOP-gguf

Name: projecte-aina_-_salamandra-7b-aligned-EADOP-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

salamandra-7b-aligned-EADOP - GGUF

Model creator: https://huggingface.co/projecte-aina/
Original model: https://huggingface.co/projecte-aina/salamandra-7b-aligned-EADOP/

Name	Quant method	Size
salamandra-7b-aligned-EADOP.Q2_K.gguf	Q2_K	3.08GB
salamandra-7b-aligned-EADOP.Q3_K_S.gguf	Q3_K_S	3.5GB
salamandra-7b-aligned-EADOP.Q3_K.gguf	Q3_K	3.77GB
salamandra-7b-aligned-EADOP.Q3_K_M.gguf	Q3_K_M	3.77GB
salamandra-7b-aligned-EADOP.Q3_K_L.gguf	Q3_K_L	4.0GB
salamandra-7b-aligned-EADOP.IQ4_XS.gguf	IQ4_XS	4.18GB
salamandra-7b-aligned-EADOP.Q4_0.gguf	Q4_0	4.33GB
salamandra-7b-aligned-EADOP.IQ4_NL.gguf	IQ4_NL	4.36GB
salamandra-7b-aligned-EADOP.Q4_K_S.gguf	Q4_K_S	4.35GB
salamandra-7b-aligned-EADOP.Q4_K.gguf	Q4_K	4.52GB
salamandra-7b-aligned-EADOP.Q4_K_M.gguf	Q4_K_M	4.52GB
salamandra-7b-aligned-EADOP.Q4_1.gguf	Q4_1	4.72GB
salamandra-7b-aligned-EADOP.Q5_0.gguf	Q5_0	5.11GB
salamandra-7b-aligned-EADOP.Q5_K_S.gguf	Q5_K_S	5.11GB
salamandra-7b-aligned-EADOP.Q5_K.gguf	Q5_K	5.21GB
salamandra-7b-aligned-EADOP.Q5_K_M.gguf	Q5_K_M	5.21GB
salamandra-7b-aligned-EADOP.Q5_1.gguf	Q5_1	5.5GB
salamandra-7b-aligned-EADOP.Q6_K.gguf	Q6_K	5.94GB
salamandra-7b-aligned-EADOP.Q8_0.gguf	Q8_0	7.69GB

Original model description:

base_model:

BSC-LT/salamandra-7b-instruct datasets:
alinia/EADOP-RAG-out-of-domain language:
ca
es library_name: transformers license: apache-2.0 pipeline_tag: text-generation tags:
legal

Salamandra 7B aligned EADOP Model Card

Salamandra 7B aligned EADOP is a full-finetuning version of BSC Language Technologies Unit's Salamandra Instruct 7B model of the Barcelona Supercomputing Center focused on improving the handling of out-of-domain Questions in a RAG instruction-following setting.

The model has been finetuned on a dataset consisting of 2,000+ human annotated in- and out-of-domain user messages and assistant responses in the context of a chatbot that can provide helpful information about the current Catalan legislation. The dataset alinia/EADOP-RAG-out-of-domain was collected in collaboration with the Entitat Autònoma del Diari Oficial i de Publicacions (EADOP) and it consists of user messages and assistant responses in Catalan and Spanish.

[!WARNING] DISCLAIMER: This model is a proof-of-concept designed to demonstrate the effects of finetuning an Instruction model with a small dataset of out-of-domain questions in the model's capability to politely and informatively refuse to answer questions that are out-of-domain. As a proof-of-concept, the model is still prone to generate harmful or inappropriate content.

Model Details

Please refer to the Salamandra Instruct 7B model details for the specific details about the model architecture and pretraining.

Intended Use

This model was developed as a proof-of-concept to demonstrate the effects of finetuning an Instruction model with a small dataset of in- and out-of-domain questions in the model's capability to politely and informatively refuse to answer questions that are out-of-domain in the context of a domain-specific RAG-based chatbot.

How to use

This model uses the ChatML, the same instruction-following conversation format as the base model.

python

from datetime import datetime
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = "projecte-aina/salamandra-7b-aligned-EADOP"

text = "Quina és la finalitat del Servei Meterològic de Catalunya ?"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
  )

message = [ { "role": "user", "content": text } ]

prompt = tokenizer.apply_chat_template(
    message,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using this template, each turn is preceded by a <|im_start|> delimiter and the role of the entity (either user, for content supplied by the user, or assistant for LLM responses), and finished with the <|im_end|> token.

Finetuning Data

Please refer to alinia/EADOP-RAG-out-of-domain for the Dataset Card.

Author

This model has been finetuned by Alinia AI.

Contact

For further information, please email langtech@bsc.es.

Copyright

License

Apache-2.0

Funding

This work has been promoted and financed by the Generalitat de Catalunya through the Aina project.

Acknowledgements

The data collection process was supported by the Entitat Autònoma del Diari Oficial i de Publicacions (EADOP).

RichardErkhov/projecte-aina_-_salamandra-7b-aligned-EADOP-gguf