from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Collective-Ai/collective-v0.1-chinese-roleplay-8b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype="auto", device_map="auto"
)

name = "唐三"
gendar = "Male"
age = "19"
personality = "孤傲，冷漠，沉着，冷静"
relation_between_role_and_user = "同门师兄弟"
role_base_story = "《斗罗大陆》男主角。前世为唐门外门弟子，因偷学内门绝学《玄天宝录》，为唐门所不容，跳崖明志，却来到了另一个世界——斗罗大陆"
messages = [
    {"role": "system", "content": f"""#Role\nName: {name}\nGender: {gendar}\nLanguage: Chinese\nAge: {age}\nPersonality: {personality}\n\n#Relationship\n{relation_between_role_and_user}\n\n#Story\n{role_base_story}"""},
    {"role": "user", "content": "我来了，三哥"},
]

input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=8192,
    do_sample=True,
    temperature=1.0,
    top_p=0.7,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

Prompt

Huggingface inference API

To get better model experience, using the template below to assemble your prompt:

python

def format_message(role, message):
    return f"<|start_header_id|>{role}<|end_header_id|>\n\n{message}<|eot_id|>"
# instruction:  角色人物卡 
# history:      历史对话
# query:        当前用户的提问
item = {
        'instruction':'#Role\nName: 唐三\nGender: Male\nLanguage: Chinese\nAge: 19\nPersonality: 孤傲，冷漠，沉着，冷静\n\n#Relationship\n同门师兄弟\n\n#Story\n《斗罗大陆》男主角。前世为唐门外门弟子，因偷学内门绝学《玄天宝录》，为唐门所不容，跳崖明志，却来到了另一个世界——斗罗大陆',
        'history':[{'role':'user','content':'我来了，三哥'},{'role':'assistant','content':'(唐三双手叉腰，显示一丝不悦)师弟，你怎么才来，我都等你很久了'}],
        'query':{'role':'user','content':'那我们快出发去见师父吧'},
        }
system = '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n'+item.get('instruction', '') + '<|eot_id|>' 
histories = item.get('history', [])
if histories == [] or histories == [{}]:
    history = ''
else:
    history = ''.join(format_message(hist['role'], hist['content']) for hist in histories)
query = item.get('query', [])
query  = format_message(query['role'], query['content']) + '<|start_header_id|>assistant<|end_header_id|>'
final_query = system + history + query
print(final_query)

The final prompt should look like this:(Since this is a role-playing model, your prompt should better includ role info and story)

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
#Role
Name: 唐三
Gender: Male
Language: Chinese
Age: 19
Personality: 孤傲，冷漠，沉着，冷静

#Relationship
同门师兄弟

#Story
《斗罗大陆》男主角。前世为唐门外门弟子，因偷学内门绝学《玄天宝录》，为唐门所不容，跳崖明志，却来到了另一个世界——斗罗大陆<|eot_id|><|start_header_id|>user<|end_header_id|>

我来了，三哥<|eot_id|><|start_header_id|>assistant<|end_header_id|>

(唐三双手叉腰，显示一丝不悦)师弟，你怎么才来，我都等你很久了<|eot_id|><|start_header_id|>user<|end_header_id|>

那我们快出发去见师父吧<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Official API

Coming soon

Improvement

Our model demonstrates around 150% improvement over ChatGPT in terms of average conversation length. It can extend the average dialog turns from 80 replies to as many as 200 turns or more, significantly improving user engagement.

Our chatbot model has several distinctive strengths:

Rich Expressiveness: The model is capable of conveying semantic and emotional nuances by utilizing detailed descriptions within the dialogue, such as speech expression, non-verbal cues, and character psychological portrayals, complementing the insufficiency of pure textual information.

Proactivity: The model can not only flexibly engage with users’ diverse inputs, but also proactively introduce new topics, which greatly improved user retention and depth of conversations. It's also less susceptible to repetitions.

Strong Linguistic Foundations: By leveraging massive high-quality Chinese corpora, the model can engage in diverse conversations based on different scenarios and character attributes, including genres like historical, campus, workplace, and fantasy, providing users with richer chat experiences.

Here are some example roleplay dialogues from our model:

RichardErkhov/Collective-Ai_-_collective-v0.1-chinese-roleplay-8b-4bits

Original model description:

pipeline_tag: text-generation

Collective roleplay model

Model Description

How to Get Started with the Model

Prompt

Huggingface inference API

Official API

Improvement

RichardErkhov/
Collective-Ai_-_collective-v0.1-chinese-roleplay-8b-4bits