mlx-community/HY-MT1.5-7B-6bit
The Model
mlx-community/HY-MT1.5-7B-6bit was converted to MLX format from
tencent/HY-MT1.5-7B using mlx-lm version
0.29.1.
You can find other similar translation-related MLX model quants for an Apple Mac at
https://huggingface.co/bibprojThe following parameters values are recommended for inference:
- top_k: 20
- top_p: 0.6
- repetition_penalty: 1.05
- temperature: 0.7
36 Supported Languages: Chinese, English, French, Portuguese, Spanish, Japanese, Turkish, Russian, Korean, Thai, Italian, German, Vietnamese, Malay, Indonesian, Filipino, Hindi, Traditional Chinese, Polish, Czech, Dutch, Khmer, Burmese, Persian, Gujarati, Urdu, Telugu, Marathi, Hebrew, Bengali, Tamil, Ukrainian, Tibetan, Kazakh, Mongolian, Uyghur, and Cantonese.
Use with mlx
1from mlx_lm import load, generate
2
3model, tokenizer = load("mlx-community/HY-MT1.5-7B-6bit")
4
5prompt="Translate from English to French: Hi there!"
6
7if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
8 messages = [{"role": "user", "content": prompt}]
9 prompt = tokenizer.apply_chat_template(
10 messages, tokenize=False, add_generation_prompt=True
11 )
12
13response = generate(model, tokenizer, prompt=prompt, verbose=True)
