Faro-Yi-9B is an improved
Yi-9B-200K with extensive instruction tuning on
Fusang-V1. Compared to Yi-9B-200K, Faro-Yi-9B has gained greater capability in various downstream tasks and long-context modeling thanks to the large-scale synthetic data in Fusang-V1.
Faro-Yi-9B uses chatml template. This make it easy to set up system prompt and multi-turn conversations.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained(
4 model_path,
5 device_map="cuda"
6)
7tokenizer = AutoTokenizer.from_pretrained(model_path)
8
9messages = [
10 {"role": "system", "content": "You are a helpful assistant. Always answer with a short response."},
11 {"role": "user", "content": "Tell me what is Pythagorean theorem like you are a pirate."}
12]
13input_ids = tokenizer.apply_chat_template(
14 messages,
15 tokenize=True,
16 add_generation_prompt=True,
17 return_tensors="pt",
18).to(model.device)
19
20generated_ids = model.generate(input_ids, max_new_tokens=512, temperature=0.5)
21response = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
22
23# Aye, matey! The Pythagorean theorem is a nautical rule that helps us find the length of the third side of a triangle. It's like this: if you have a triangle with two sides, you can find the length of the third side by squaring the two sides and then adding them together. The square root of that sum will give you the length of the third side! It's useful for sailing and navigating, so you always know how far you've traveled. Remember, it's all about the sum of squares, me hearties!
Faro-Yi-9B enhances its ability compared to Yi-9B-200K in most dimensions, especially in long-range modeling and bilingual (English, Chinese) understanding. Faro is competitive among all open-sourced models at around 9B parameters.
