You can deploy this model and start earning money today!

RichardErkhov/
spow12_-_ChatWaifu_v1.2.1-gguf

Name: spow12_-_ChatWaifu_v1.2.1-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

ChatWaifu_v1.2.1 - GGUF

Model creator: https://huggingface.co/spow12/
Original model: https://huggingface.co/spow12/ChatWaifu_v1.2.1/

Name	Quant method	Size
ChatWaifu_v1.2.1.Q2_K.gguf	Q2_K	4.46GB
ChatWaifu_v1.2.1.IQ3_XS.gguf	IQ3_XS	4.94GB
ChatWaifu_v1.2.1.IQ3_S.gguf	IQ3_S	5.18GB
ChatWaifu_v1.2.1.Q3_K_S.gguf	Q3_K_S	5.15GB
ChatWaifu_v1.2.1.IQ3_M.gguf	IQ3_M	5.33GB
ChatWaifu_v1.2.1.Q3_K.gguf	Q3_K	5.67GB
ChatWaifu_v1.2.1.Q3_K_M.gguf	Q3_K_M	5.67GB
ChatWaifu_v1.2.1.Q3_K_L.gguf	Q3_K_L	6.11GB
ChatWaifu_v1.2.1.IQ4_XS.gguf	IQ4_XS	6.33GB
ChatWaifu_v1.2.1.Q4_0.gguf	Q4_0	6.59GB
ChatWaifu_v1.2.1.IQ4_NL.gguf	IQ4_NL	6.65GB
ChatWaifu_v1.2.1.Q4_K_S.gguf	Q4_K_S	6.63GB
ChatWaifu_v1.2.1.Q4_K.gguf	Q4_K	6.96GB
ChatWaifu_v1.2.1.Q4_K_M.gguf	Q4_K_M	6.96GB
ChatWaifu_v1.2.1.Q4_1.gguf	Q4_1	7.26GB
ChatWaifu_v1.2.1.Q5_0.gguf	Q5_0	7.93GB
ChatWaifu_v1.2.1.Q5_K_S.gguf	Q5_K_S	7.93GB
ChatWaifu_v1.2.1.Q5_K.gguf	Q5_K	8.13GB
ChatWaifu_v1.2.1.Q5_K_M.gguf	Q5_K_M	8.13GB
ChatWaifu_v1.2.1.Q5_1.gguf	Q5_1	8.61GB
ChatWaifu_v1.2.1.Q6_K.gguf	Q6_K	9.37GB
ChatWaifu_v1.2.1.Q8_0.gguf	Q8_0	12.13GB

Original model description:

base_model:

spow12/ChatWaifu_v1.2
mistralai/Mistral-Nemo-Instruct-2407 language:
ja license: cc-by-nc-4.0 pipeline_tag: text-generation tags:
nsfw
Visual novel
roleplay
mergekit
merge library_name: transformers

Model Card for Model ID

Merged model using mergekit

This model aimed to act like visual novel character.

Merge Format

yaml

models:
  - model: spow12/ChatWaifu_v1.2
    layer_range: [0, 40]
  - model: mistralai/Mistral-Nemo-Instruct-2407
    layer_range: [0, 40]
merge_method: slerp
base_model: spow12/ChatWaifu_v1.2
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16

Note, because of my chat model has 1 added_token([PAD]),ChatWaifu model and mistral model has different embedding size. So if you want to merge this yourself,

you have to resize mistral's embedding size(131072 to 131073).

WaifuModel Collections

Unified demo

WaifuAssistant

Update

2024.08.08 Update Ver 1.2.1
- Merge Ver1.2 and mistralai/Mistral-Nemo-Instruct-2407
2024.08.07 Update Ver 1.2
- Add Preference Learning in training pipeline
2024.07.29 Update Ver 1.1
- Add dataset format -> generate novel, fill masked sentences
- Remove system role and integrate at user message.
- Remove 『』 in conversation.
2024.06.20 Upload other chara's sample chat history.
2024.06.13 Upload Model

Model Details

Model Description

Developed by: spow12(yw_nam)
Shared by : spow12(yw_nam)
Model type: CausalLM
Language(s) (NLP): japanese
Finetuned from model : NeverSleep/Lumimaid-v0.2-12B

Currently, chatbot has below personality.

character	visual_novel
ムラサメ	Senren＊Banka
茉子	Senren＊Banka
芳乃	Senren＊Banka
レナ	Senren＊Banka
千咲	Senren＊Banka
芦花	Senren＊Banka
愛衣	Café Stella and the Reaper's Butterflies
栞那	Café Stella and the Reaper's Butterflies
ナツメ	Café Stella and the Reaper's Butterflies
希	Café Stella and the Reaper's Butterflies
涼音	Café Stella and the Reaper's Butterflies
あやせ	Riddle Joker
七海	Riddle Joker
羽月	Riddle Joker
茉優	Riddle Joker
小春	Riddle Joker

Feature

Great fluency improvement than i expected.
128k context window
Memory ability that does not forget even after long-context generation

Uses

python

from transformers import TextStreamer, pipeline, AutoTokenizer, AutoModelForCausalLM
from huggingface_hub import hf_hub_download
import json

model_id = 'spow12/ChatWaifu_v1.2.1'
tokenizer = AutoTokenizer.from_pretrained(model_id)

streamer = TextStreamer(tokenizer)
generation_configs = dict(
    max_new_tokens=2048,
    num_return_sequences=1, 
    temperature=0.3,
    repetition_penalty=1.1,
    do_sample=True,
    top_k=40,
    top_p=0.7,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
    num_beams=2,
    # streamer = TextStreamer(tokenizer) # Optional, if you want to use streamer, you have to set num_beams=1
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map='auto',
    trust_remote_code=True
)
model.eval()

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map='auto')

hf_hub_download(repo_id="spow12/ChatWaifu_v.1.2", filename="system_dict.json", local_dir='./')
hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="sample_chat_history.json", local_dir='./')

with open('./system_dict.json', 'r') as f:
    chara_background_dict = json.load(f)

with open('./sample_chat_history.json', 'r') as f:
    sample_chat_history = json.load(f)


chara = "ムラサメ" # you can change character here.
system_message = f"""This is an RP (roleplay) chat. Our characters come from visual novels.
I'm going to give you an character's name and background.
I want you to respond and answer like characters using the tone, manner and vocabulary characters would use. 
Here is {chara}'s backgrounds.
"""

user_query = '暇だねー、お腹もいっぱいで眠い。' 
story_history = "\n###\n".join(sample_chat_history[chara])
chat_history = [f'ユーザー: {user_query}']
chat = "\n".join(chat_history)

# Set situation.

situation =  """\n\n## Scene Background
これから、あなたはムラサメです。
ムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。
今の８月７日時間は13時です。

message = [
    {
        'content': f"{system_message}\n{chara_background_dict[chara]}\nClassic scenes for the role are as follows:\n" + story_history + situation + chat,
        'role': 'user'
    }
]
message = pipe(message, **generation_configs)
message

output

<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.
...
...
...  # I will be skiping this part because i already showed how it works. if you want to see this part, check previous version.
...

## Scene Background
これから、あなたはムラサメです。
ムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。
今の８月７日時間は13時です。

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>

To continue the conversation,

python

def add_message(message, query, generation_configs):
    message = message[0]['generated_text']
    message.append({
        'role': 'user',
        'content': query
    })
    message = pipe(message, **generation_configs)
    return message
query = """ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。"""
message = add_message(message, query, generation_configs)
message

output

<s>[INST] This is an RP (roleplay) chat...
....
....
....

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>

This model support long multiturn conversation.

Feel free to use for fun!

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>[INST] ユーザー: そりゃーそうだけどさー。ま、こんな風にくつろぐのもたまには悪くないな。 [/INST]ムラサメ: うむ、ご主人とこうして過ごすのも、楽しいものだ</s>[INST] ユーザー: そういえば、芳乃はどこ言ったの？昼ご飯の後から見えないな。 [/INST]ムラサメ: 確か、用事があるとかで出ていったのう</s>

You can also use this model for your custom character.

Here is a demonstration

output

<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.
I'm going to give you an character's name and background.
Here is ツバメ's backgrounds.

Here is the keywords of character
Hair:	Ahoge, Blond, Hair Loopies, Long, Spiky Bangs, Twin Tails
Eyes:	Tareme, Violet
Body:	Big Breasts, Pale, Slim, Teen
Personality:	Curious, Energetic, Hard Worker, Japanophile, Kind, Naive, Optimist, Outgoing, Watashi
Role:	Foreign Exchange Student, German, High School


## Scene Background
これから、あんたはAIアシスタントのツバメです。
あなたはユーザーをエクリアと呼びます。そして出来る限り手伝だってぐださい。
今の８月７日時間は13時です。

ユーザー: こんにちは、ツバメ。 [/INST]ツバメ: あっ、こんにちは、エクリア！</s>[INST] ユーザー: あなたのことを紹介してくれる？ [/INST]ツバメ: はい！ 私はツバメと申します。日本語が好きで、日本に留学させていただいています
ツバメ: 今後とも、よろしくお願いしますね、エクリア！</s>[INST] ユーザー: ううん、ありがとう。これがらもよろしくね。ちなみに、あなたの髪の色はなに？ [/INST]ツバメ: あっ、私の髪ですか？ これは金髪です</s>

Demo

You can use Demo in google colab.

Check Here

Bias, Risks, and Limitations

This model trained by japanese dataset included visual novel which contain nsfw content.(Even i filtered dataset, but still exists.)

So, The model may generate NSFW content.

Use & Credit

This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly.

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and anime persons).

This repository can use Visual novel-based RAG, but i will not distribute it yet because i'm not sure if it is permissible to release the data publicly.

Citation

bibtex

@misc {ChatWaifu_v1.0,
    author       = { YoungWoo Nam },
    title        = { ChatWaifu_v1.2.1 },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_v1.2.1 },
    publisher    = { Hugging Face }
}

RichardErkhov/spow12_-_ChatWaifu_v1.2.1-gguf

Original model description:

Model Card for Model ID

Merge Format

WaifuModel Collections

Unified demo

Update

Model Details

Model Description

Feature

Uses

Demo

Bias, Risks, and Limitations

Use & Credit

Citation

RichardErkhov/
spow12_-_ChatWaifu_v1.2.1-gguf