Model Card for Model ID
imageMerged model using
mergekitThis model aimed to act like visual novel character.
Merge Format
1models:
2 - model: mistralai/Mistral-Small-Instruct-2409_SFT
3 layer_range: [0, 56]
4 - model: mistralai/Mistral-Small-Instruct-2409
5 layer_range: [0, 56]
6merge_method: slerp
7base_model: mistralai/Mistral-Small-Instruct-2409_SFT
8parameters:
9 t:
10 - filter: self_attn
11 value: [0, 0.5, 0.3, 0.7, 1]
12 - filter: mlp
13 value: [1, 0.5, 0.7, 0.3, 0]
14 - value: 0.5 # fallback for rest of tensors
15dtype: bfloat16
WaifuModel Collections
Unified demo
Update 2.0
- 2024.09.23 Update 22B, Ver 2.0
Model Details
Model Description
- Developed by: spow12(yw_nam)
- Shared by : spow12(yw_nam)
- Model type: CausalLM
- Language(s) (NLP): japanese. English
- Finetuned from model : mistralai/Mistral-Small-Instruct-2409
Currently, chatbot has below personality.
| character | visual_novel |
|---|
| ムラサメ | Senren＊Banka |
| 茉子 | Senren＊Banka |
| 芳乃 | Senren＊Banka |
| レナ | Senren＊Banka |
| 千咲 | Senren＊Banka |
| 芦花 | Senren＊Banka |
| 愛衣 | Café Stella and the Reaper's Butterflies |
| 栞那 | Café Stella and the Reaper's Butterflies |
| ナツメ | Café Stella and the Reaper's Butterflies |
| 希 | Café Stella and the Reaper's Butterflies |
| 涼音 | Café Stella and the Reaper's Butterflies |
| あやせ | Riddle Joker |
| 七海 | Riddle Joker |
| 羽月 | Riddle Joker |
| 茉優 | Riddle Joker |
| 小春 | Riddle Joker |
But you can chat your own Character with persona text.
Feel free to test.
Your feedback will be helpful for improving model.
Dataset
Riddle Joker(Prviate)
Café Stella and the Reaper's Butterflies(Private)
Senren＊Banka(Private)
roleplay4fun/aesir-v1.1
kalomaze/Opus_Instruct_3k
Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
Aratako/Synthetic-JP-EN-Coding-Dataset-567k (only using 50000 sample)
Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
SkunkworksAI/reasoning-0.01
Feature
- Fluent Chat performance
- Reduce repetition problem when generate with many turn(over 20~30)
- Zero Shot character persona using description of character.
- 128k context window
- Memory ability that does not forget even after long-context generation
Demo
You can use Demo in google colab.
Bias, Risks, and Limitations
This model can generate NSFW content.
Use & Credit
This model is currently available for non-commercial & Research purpose only.
Also, since I'm not detailed in licensing, I hope you use this model responsibly.
By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and Waifu Lovers).
Citation
1@misc {ChatWaifu_22B_v2.0
2 author = { YoungWoo Nam },
3 title = { ChatWaifu_22B_v2.0_preview },
4 year = 2024,
5 url = { https://huggingface.co/spow12/ChatWaifu_22B_v2.0_preview },
6 publisher = { Hugging Face }
7}
Detailed results can be found
here| Metric | Value |
|---|
| Avg. | 29.12 |
| IFEval (0-Shot) | 67.45 |
| BBH (3-Shot) | 45.49 |
| MATH Lvl 5 (4-Shot) | 16.31 |
| GPQA (0-shot) | 8.72 |
| MuSR (0-shot) | 3.53 |
| MMLU-PRO (5-shot) | 33.20 |
