You can deploy this model and start earning money today!

RichardErkhov/
AndreasThinks_-_mistral-7b-english-welsh-translate-gguf

Name: AndreasThinks_-_mistral-7b-english-welsh-translate-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

mistral-7b-english-welsh-translate - GGUF

Model creator: https://huggingface.co/AndreasThinks/
Original model: https://huggingface.co/AndreasThinks/mistral-7b-english-welsh-translate/

Name	Quant method	Size
mistral-7b-english-welsh-translate.Q2_K.gguf	Q2_K	2.54GB
mistral-7b-english-welsh-translate.IQ3_XS.gguf	IQ3_XS	2.82GB
mistral-7b-english-welsh-translate.IQ3_S.gguf	IQ3_S	2.97GB
mistral-7b-english-welsh-translate.Q3_K_S.gguf	Q3_K_S	2.95GB
mistral-7b-english-welsh-translate.IQ3_M.gguf	IQ3_M	3.06GB
mistral-7b-english-welsh-translate.Q3_K.gguf	Q3_K	3.28GB
mistral-7b-english-welsh-translate.Q3_K_M.gguf	Q3_K_M	3.28GB
mistral-7b-english-welsh-translate.Q3_K_L.gguf	Q3_K_L	3.56GB
mistral-7b-english-welsh-translate.IQ4_XS.gguf	IQ4_XS	3.68GB
mistral-7b-english-welsh-translate.Q4_0.gguf	Q4_0	3.83GB
mistral-7b-english-welsh-translate.IQ4_NL.gguf	IQ4_NL	3.87GB
mistral-7b-english-welsh-translate.Q4_K_S.gguf	Q4_K_S	3.86GB
mistral-7b-english-welsh-translate.Q4_K.gguf	Q4_K	4.07GB
mistral-7b-english-welsh-translate.Q4_K_M.gguf	Q4_K_M	4.07GB
mistral-7b-english-welsh-translate.Q4_1.gguf	Q4_1	4.24GB
mistral-7b-english-welsh-translate.Q5_0.gguf	Q5_0	4.66GB
mistral-7b-english-welsh-translate.Q5_K_S.gguf	Q5_K_S	4.66GB
mistral-7b-english-welsh-translate.Q5_K.gguf	Q5_K	4.78GB
mistral-7b-english-welsh-translate.Q5_K_M.gguf	Q5_K_M	4.78GB
mistral-7b-english-welsh-translate.Q5_1.gguf	Q5_1	5.07GB
mistral-7b-english-welsh-translate.Q6_K.gguf	Q6_K	5.54GB
mistral-7b-english-welsh-translate.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: apache-2.0 base_model: mistralai/Mistral-7B-Instruct-v0.3 tags:

axolotl
generated_from_trainer model-index:
name: mistral-7b-english-welsh-translate results: []

mistral-7b-english-welsh-translate

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the Welsh Government Alpaca Welsh-English Instructions dataset.

This model is trained for English-Welsh translation (in any direction), with a focus on government documents, using Markdown formatting.

To ensure the highest quality translations, use the Alpaca instruction prompt format with the below structure.

    ### Instruction: {instruction}

    ### Input: {input}

    ### Response:

Your instruction should be "Translate the text from English to Welsh." (or vice versa).

The model is also available quantized as GGUF. This version be be tested in this interactive space.

Running the model

The model is intended to be run locally, ideally using Text generation web UI to ensure correct prompt structure.

Start the UI as instructed for your system.

In the "Model" tab, download either this model or the quantized version. Once the download is complete, load the model.
In the "Parameters" tab, under "Generation", set "auto_max_new_tokens" to maximum, and "Ban the eos_token" to True. In "Custom stopping strings", add "### Input"
In the "Notebook" tab, make sure you are using the "Alpaca-with-input" prompt. Set the instruction as "Translate the text from Welsh to English." (or vice versa).#
Add the text you would like to translate (replacing "Input"), and hit "generate"

Performance may start to degrade past a certain context window (especially if using the quantized models). Convert in chunks of under 1000 words to avoid these issues.

LLM Evals

Thanks to YALL - Yet Another LLM Leaderboard. Full results available here.

Model	AGIEval	TruthfulQA	Bigbench
mistral-7b-english-welsh-translate	35.31	54.5	38.4

Training procedure

See axolotl config

axolotl version: 0.4.1

yaml

base_model: mistralai/Mistral-7B-Instruct-v0.3
model_type: MistralForCausalLM
tokenizer_type: LlamaTokenizer

load_in_8bit: false
load_in_4bit: 
strict: false

# huggingface repo
datasets:
  - path: AndreasThinks/welsh-translation-instruction
    type: alpaca
val_set_size: 0.04
output_dir: ./outputs/mistral-welsh

hub_model_id: AndreasThinks/mistral-7b-english-welsh-translate


sequence_len: 8192
sample_packing: true
pad_to_sequence_len: true
eval_sample_packing: false

wandb_project: mistral-nemo-welsh
wandb_entity:
wandb_watch:
wandb_name: mistral-nemo-welsh-v1
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 4
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.000005

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 10
evals_per_epoch: 4
eval_table_size:
eval_max_new_tokens: 128
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:

Visualize in Weights & Biases

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
1.5781	0.0013	1	1.5514
0.4427	0.2506	194	0.4841
0.4142	0.5011	388	0.4271
0.4001	0.7517	582	0.3996
0.4155	1.0023	776	0.3828
0.3178	1.2296	970	0.3792
0.3156	1.4801	1164	0.3732
0.3115	1.7307	1358	0.3678
0.2722	1.9813	1552	0.3633
0.2492	2.2089	1746	0.3809
0.2159	2.4595	1940	0.3828
0.2277	2.7100	2134	0.3810
0.2435	2.9606	2328	0.3809
0.158	3.1899	2522	0.3961
0.1802	3.4404	2716	0.3966
0.1679	3.6910	2910	0.3966

Framework versions

Transformers 4.42.4
Pytorch 2.3.1+cu121
Datasets 2.19.1
Tokenizers 0.19.1

RichardErkhov/AndreasThinks_-_mistral-7b-english-welsh-translate-gguf