You can deploy this model and start earning money today!

RichardErkhov/
Bohanlu_-_Taigi-Llama-2-Translator-13B-gguf

Name: Bohanlu_-_Taigi-Llama-2-Translator-13B-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Taigi-Llama-2-Translator-13B - GGUF

Model creator: https://huggingface.co/Bohanlu/
Original model: https://huggingface.co/Bohanlu/Taigi-Llama-2-Translator-13B/

Name	Quant method	Size
Taigi-Llama-2-Translator-13B.Q2_K.gguf	Q2_K	4.65GB
Taigi-Llama-2-Translator-13B.IQ3_XS.gguf	IQ3_XS	5.14GB
Taigi-Llama-2-Translator-13B.IQ3_S.gguf	IQ3_S	5.41GB
Taigi-Llama-2-Translator-13B.Q3_K_S.gguf	Q3_K_S	5.41GB
Taigi-Llama-2-Translator-13B.IQ3_M.gguf	IQ3_M	5.72GB
Taigi-Llama-2-Translator-13B.Q3_K.gguf	Q3_K	6.05GB
Taigi-Llama-2-Translator-13B.Q3_K_M.gguf	Q3_K_M	6.05GB
Taigi-Llama-2-Translator-13B.Q3_K_L.gguf	Q3_K_L	6.6GB
Taigi-Llama-2-Translator-13B.IQ4_XS.gguf	IQ4_XS	6.69GB
Taigi-Llama-2-Translator-13B.Q4_0.gguf	Q4_0	7.02GB
Taigi-Llama-2-Translator-13B.IQ4_NL.gguf	IQ4_NL	7.06GB
Taigi-Llama-2-Translator-13B.Q4_K_S.gguf	Q4_K_S	7.07GB
Taigi-Llama-2-Translator-13B.Q4_K.gguf	Q4_K	7.48GB
Taigi-Llama-2-Translator-13B.Q4_K_M.gguf	Q4_K_M	7.48GB
Taigi-Llama-2-Translator-13B.Q4_1.gguf	Q4_1	7.77GB
Taigi-Llama-2-Translator-13B.Q5_0.gguf	Q5_0	8.53GB
Taigi-Llama-2-Translator-13B.Q5_K_S.gguf	Q5_K_S	8.53GB
Taigi-Llama-2-Translator-13B.Q5_K.gguf	Q5_K	8.77GB
Taigi-Llama-2-Translator-13B.Q5_K_M.gguf	Q5_K_M	8.77GB
Taigi-Llama-2-Translator-13B.Q5_1.gguf	Q5_1	9.28GB
Taigi-Llama-2-Translator-13B.Q6_K.gguf	Q6_K	10.13GB
Taigi-Llama-2-Translator-13B.Q8_0.gguf	Q8_0	13.13GB

Original model description:

license: cc-by-nc-sa-4.0

Taigi-llama-logo

Model Card for Taigi-Llama-2-Translator-13B

The Taigi-Llama-2-Translator series are built based on the Taigi-Llama-2 series model. We conducted fine-tuning on 263k parallel data to create a translation model for Taiwanese Hokkien and related languages.

For more details, please refer to our GitHub repository and the paper: Enhancing Taiwanese Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems

Explore other models and datasets in the Taiwanese Hokkien LLM collection.

Model description

Base Model: Bohanlu/Taigi-Llama-2-13B
Usage: This model can be used for translating between Traditional Chinese or English and Taiwanese Hokkien (Hanzi, POJ, or Hanlo). It also supports translation between different scripts of Taiwanese Hokkien (Hanzi, POJ, Hanlo).
Language(s) (NLP): Taiwanese Hokkien (Hanzi, POJ and Hanlo), Traditional Chinese and English
Input: Text in source language
Output: Text in target language
Model Size: 13B parameters

Prompt Template

{BOS}[TRANS]\n{source_sentence}\n[/TRANS]\n[{target_language}]\n

source_sentence: The sentence you want to translate.
target_language: The target language you want to translate to. Use "ZH" for Traditional Chinese, "EN" for English, "POJ" for Taiwanese Hokkien POJ, "HL" for Taiwanese Hokkien Hanlo, and "HAN" for Taiwanese Hokkien Hanzi.
Ensure there's a newline at the end.

Usage Example

python

from transformers import AutoModelForCausalLM, AutoTokenizer, TextGenerationPipeline
import torch
import accelerate

def get_pipeline(path:str, tokenizer:AutoTokenizer, accelerator:accelerate.Accelerator) -> TextGenerationPipeline:
    model = AutoModelForCausalLM.from_pretrained(
        path, torch_dtype=torch.float16, device_map='auto', trust_remote_code=True)
    
    terminators = [tokenizer.eos_token_id, tokenizer.pad_token_id]

    pipeline = TextGenerationPipeline(model = model, tokenizer = tokenizer, num_workers=accelerator.state.num_processes*4, pad_token_id=tokenizer.pad_token_id, eos_token_id=terminators)

    return pipeline

model_dir = "Bohanlu/Taigi-Llama-2-Translator-7B" # or "Bohanlu/Taigi-Llama-2-Translator-13B" for the 13B model
tokenizer = AutoTokenizer.from_pretrained(model_dir, use_fast=False)

accelerator = accelerate.Accelerator()
pipe = get_pipeline(model_dir, tokenizer, accelerator)

PROMPT_TEMPLATE = "[TRANS]\n{source_sentence}\n[/TRANS]\n[{target_language}]\n"

def translate(source_sentence:str, target_language:str) -> str:
    prompt = PROMPT_TEMPLATE.format(source_sentence=source_sentence, target_language=target_language)
    out = pipe(prompt, return_full_text=False, repetition_penalty=1.1, do_sample=False)[0]['generated_text']
    return out[:out.find("[/")].strip()

source_sentence = "How are you today？"

print("To Hanzi: " + translate(source_sentence, "HAN"))
# Output: To Hanzi: 你今仔日好無？

print("To POJ: " + translate(source_sentence, "POJ"))
# Output: To POJ: Lí kin-á-ji̍t án-chóaⁿ?

print("To Traditional Chinese: " + translate(source_sentence, "ZH"))
# Output: To Traditional Chinese: 你今天好嗎？

print("To Hanlo: " + translate(source_sentence, "HL"))
# Output: To Hanlo: 你今仔日好無？

Citation

If you find the resources in the Taiwanese Hokkien LLM collection useful in your work, please cite it using the following reference:

@misc{lu2024enhancing,
      title={Enhancing Taiwanese Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems}, 
      author={Bo-Han Lu and Yi-Hsuan Lin and En-Shiun Annie Lee and Richard Tzong-Han Tsai},
      year={2024},
      eprint={2403.12024},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

RichardErkhov/Bohanlu_-_Taigi-Llama-2-Translator-13B-gguf