Views
No views yet
| model name | JCommonsenseQA (3-shot) | JNLI (3-shot) | MARC-ja (0-shot) | JSQuAD (2-shot) | jaqket-v2 (1-shot) | xlsum (1-shot) | xwinograd (0-shot) | mgsm (5-shot) | AVERAGE |
|---|---|---|---|---|---|---|---|---|---|
| lkarasu-1.1B | 24.84 | 34.78 | 50.48 | 51.33 | 18.95 | 9.21 | 62.15 | 2.80 | 31.82 |
| TinySlime-Chat | 66.49 | 64.36 | 92.68 | 84.60 | 42.14 | 7.79 | 60.69** | 9.20 | 53.50 |
| japanese-large-lm-3.6b-instruction-sft | 38.87 | 34.68 | 50.52 | 58.57 | 56.54 | 6.25 | 64.23 | 1.60 | 38.91 |
| open-calm-3b | 31.64 | 36.56 | 50.99 | 51.47 | 50.99 | 4.63 | 63.30 | 2.00 | 35.97 |
| bilingual-gpt-neox-4b-instruction-ppo | 50.49 | 38.90 | 88.65 | 72.82 | 65.65 | 4.69 | 64.75 | 3.80 | 48.72 |
| japanese-gpt-neox-3.6b-instruction-ppo | 44.59 | 35.25 | 64.34 | 58.53 | 26.67 | 3.79 | 68.72 | 3.20 | 33.26 |
transformersライブラリをインストールする必要がありますpip install -U transformerstransformersライブラリを使用してモデルを読み込むことができます1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "2121-8/TinySlime-1.1B-Chat-v1.0"
4tokenizer = AutoTokenizer.from_pretrained(model_name)
5model = AutoModelForCausalLM.from_pretrained(model_name)1text = "こんにちは"
2
3messages = [
4 {
5 "role": "system",
6 "content": "You are an assistant called ChatBot. You are a friendly and very good chatbot.",
7 },
8 {"role": "user", "content": text},
9 ]
10prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
11input_ids = tokenizer(prompt, return_tensors="pt")
12
13# テキストを生成
14outputs = model.generate(input_ids, max_length=50, num_return_sequences=1)
15generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
16
17print(generated_text)