You can deploy this model and start earning money today!

RichardErkhov/
Unbabel_-_TowerInstruct-13B-v0.1-gguf

Name: Unbabel_-_TowerInstruct-13B-v0.1-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

TowerInstruct-13B-v0.1 - GGUF

Model creator: https://huggingface.co/Unbabel/
Original model: https://huggingface.co/Unbabel/TowerInstruct-13B-v0.1/

Name	Quant method	Size
TowerInstruct-13B-v0.1.Q2_K.gguf	Q2_K	4.52GB
TowerInstruct-13B-v0.1.Q3_K_S.gguf	Q3_K_S	5.27GB
TowerInstruct-13B-v0.1.Q3_K.gguf	Q3_K	5.9GB
TowerInstruct-13B-v0.1.Q3_K_M.gguf	Q3_K_M	5.9GB
TowerInstruct-13B-v0.1.Q3_K_L.gguf	Q3_K_L	6.45GB
TowerInstruct-13B-v0.1.IQ4_XS.gguf	IQ4_XS	6.54GB
TowerInstruct-13B-v0.1.Q4_0.gguf	Q4_0	6.86GB
TowerInstruct-13B-v0.1.IQ4_NL.gguf	IQ4_NL	6.9GB
TowerInstruct-13B-v0.1.Q4_K_S.gguf	Q4_K_S	6.91GB
TowerInstruct-13B-v0.1.Q4_K.gguf	Q4_K	7.33GB
TowerInstruct-13B-v0.1.Q4_K_M.gguf	Q4_K_M	7.33GB
TowerInstruct-13B-v0.1.Q4_1.gguf	Q4_1	7.61GB
TowerInstruct-13B-v0.1.Q5_0.gguf	Q5_0	8.36GB
TowerInstruct-13B-v0.1.Q5_K_S.gguf	Q5_K_S	8.36GB
TowerInstruct-13B-v0.1.Q5_K.gguf	Q5_K	8.6GB
TowerInstruct-13B-v0.1.Q5_K_M.gguf	Q5_K_M	8.6GB
TowerInstruct-13B-v0.1.Q5_1.gguf	Q5_1	9.1GB
TowerInstruct-13B-v0.1.Q6_K.gguf	Q6_K	9.95GB
TowerInstruct-13B-v0.1.Q8_0.gguf	Q8_0	12.88GB

Original model description:

license: cc-by-nc-4.0 language:

en
de
fr
zh
pt
nl
ru
ko
it
es metrics:
comet pipeline_tag: translation

Model Card for TowerInstruct-13B-v0.1

Model Details

Model Description

TowerInstruct-13B is a language model that results from fine-tuning TowerBase on the TowerBlocks supervised fine-tuning dataset. TowerInstruct-13B-v0.1 is the first model in the series. The model is trained to handle several translation-related tasks, such as general machine translation (e.g., sentence- and paragraph/document-level translation, terminology-aware translation, context-aware translation), automatic post edition, named-entity recognition, gramatical error correction, and paraphrase generation. We will release more details in the upcoming technical report. For now, you can check results obtained with the model here.

Developed by: Unbabel, Instituto Superior Técnico, CentraleSupélec University of Paris-Saclay
Model type: A 13B parameter model fine-tuned on a mix of publicly available, synthetic datasets on translation-related tasks, as well as conversational datasets and code instructions.
Language(s) (NLP): English, Portuguese, Spanish, French, German, Dutch, Italian, Korean, Chinese, Russian
License: CC-BY-NC-4.0, Llama 2 is licensed under the LLAMA 2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
Finetuned from model: TowerBase

Intended uses & limitations

The model was initially fine-tuned on a filtered and preprocessed supervised fine-tuning dataset (TowerBlocks-v0.2), which contains a diverse range of data sources:

Translation (sentence and paragraph-level)
Automatic Post Edition
Machine Translation Evaluation
Context-aware Translation
Terminology-aware Translation
Multi-reference Translation
Named-entity Recognition
Paraphrase Generation
Synthetic Chat data
Code instructions

You can find the dataset and all data sources of TowerBlocks-v0.2 here.

Here's how you can run the model using the pipeline() function from 🤗 Transformers:

python

# Install transformers from source - only needed for versions <= v4.34
# pip install git+https://github.com/huggingface/transformers.git
# pip install accelerate

import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="Unbabel/TowerInstruct-13B-v0.1", torch_dtype=torch.bfloat16, device_map="auto")
# We use the tokenizer’s chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
messages = [
    {"role": "user", "content": "Translate the following text from Portuguese into English.\nPortuguese: Um grupo de investigadores lançou um novo modelo para tarefas relacionadas com tradução.\nEnglish:"},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=256, do_sample=False)
print(outputs[0]["generated_text"])
# <|im_start|>user
# Translate the following text from Portuguese into English.
# Portuguese: Um grupo de investigadores lançou um novo modelo para tarefas relacionadas com tradução.
# English:<|im_end|>
# <|im_start|>assistant
# A group of researchers has launched a new model for translation-related tasks.

Out-of-Scope Use

The model is not guaranteed to perform for languages other than the 10 languages it supports. Even though we trained the model on conversational data and code instructions, it is not intended to be used as a conversational chatbot or code assistant. We are currently working on improving quality and consistency on document-level translation. This model should is not intended to be use as a document-level translator.

Bias, Risks, and Limitations

TowerInstruct-v0.1 has not been aligned to human preferences, so the model may generate problematic outputs (e.g., hallucinations, harmful content, or false statements).

Prompt Format

TowerInstruct-v0.1 was trained using the ChatML prompt templates without any system prompts. An example follows below:

<|im_start|>user
{USER PROMPT}<|im_end|>
<|im_start|>assistant
{MODEL RESPONSE}<|im_end|>
<|im_start|>user
[...]

Supervised tasks

The prompts for all supervised tasks can be found in TowerBlocks-v0.2. We have used multiple prompt templates for each task. While different prompts may offer different outputs, the difference in downstream performance should be very minimal.

Training Details

Training Data

Link to TowerBlocks-v0.2.

Training Hyperparameters

The following hyperparameters were used during training:

total_train_batch_size: 256
learning_rate: 7e-06
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 500
weight_decay: 0.01
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
num_epochs: 4
max_seq_length: 2048

Citation

bibtex

@misc{tower_llm_2024,
      title={Tower: An Open Multilingual Large Language Model for Translation-Related Tasks}, 
      author={Duarte M. Alves and José Pombal and Nuno M. Guerreiro and Pedro H. Martins and João Alves and Amin Farajian and Ben Peters and Ricardo Rei and Patrick Fernandes and Sweta Agrawal and Pierre Colombo and José G. C. de Souza and André F. T. Martins},
      year={2024},
      eprint={2402.17733},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

RichardErkhov/Unbabel_-_TowerInstruct-13B-v0.1-gguf