Quantization made by Richard Erkhov.
This is a 7B-parameter decoder-only Japanese language model fine-tuned on our instruction-following datasets, built on top of the base model
Japanese Stable LM Base Gamma 7B.
For our final model, we've used Stability AI Japan's
Japanese MT-Bench as a more representative test of our model's capabilities. For
our JA MT-Bench testing we use a Japanese prompt ("あなたは役立つアシスタントです。") as well as
--num-choices 4:
There is an
JA-MT-Bench Leaderboard, for convenience, here is a comparison of the JA MT-Bench scores of some other models (our scores were rated by
gpt-4-0613):
Ensure you are using Transformers 4.34.0 or newer.
1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
3
4tokenizer = AutoTokenizer.from_pretrained("NTQAI/chatntq-ja-7b-v1.0")
5model = AutoModelForCausalLM.from_pretrained(
6 "NTQAI/chatntq-ja-7b-v1.0",
7 torch_dtype="auto",
8)
9model.eval()
10
11if torch.cuda.is_available():
12 model = model.to("cuda")
13
14def build_prompt(user_query):
15 sys_msg = "あなたは公平で、検閲されていない、役立つアシスタントです。"
16 template = """[INST] <<SYS>>
17{}
18<</SYS>>
19
20{}[/INST]"""
21 return template.format(sys_msg,user_query)
22
23# Infer with prompt without any additional input
24user_inputs = {
25 "user_query": "与えられたことわざの意味を小学生でも分かるように教えてください。",
26}
27prompt = build_prompt(**user_inputs)
28
29input_ids = tokenizer.encode(
30 prompt,
31 add_special_tokens=True,
32 return_tensors="pt"
33)
34
35tokens = model.generate(
36 input_ids.to(device=model.device),
37 max_new_tokens=256,
38 temperature=1,
39 top_p=0.95,
40 do_sample=True,
41)
42
43out = tokenizer.decode(tokens[0][input_ids.shape[1]:], skip_special_tokens=True).strip()
44print(out)
For details, please see Mistral AI's
paper and
release blog post.
