YuRen BaiChuan 7B is a multi-modal large language model based on baichuan-inc/baichuan-7B and trained with multi-task supervised fine-tuning. It is built on top of Pleisto's data-centric AI work. YuRen has excellent performance on multi-turn dialogue, open-domain question answering, role-playing, text generation, text understanding, image understanding and other tasks.
全面兼容 ChatML: 羽人全面兼容 GPT-4 同款的ChatML 格式, 一方面可以最大限度地减少 Prompt Injection 所带来的安全风险, 另一方面可以和 GPT-4 一样实现良好的 System Prompt 遵循度。(没错, 我们的训练数据集中包含了相当一部分带有 system prompt 的对话数据)
Multimodal: Referring to related work such as LLaVA and mPLUG-Owl, Yuren integrates the language modality of LLM and the visual encoder of the currently most SOTA CLIP model laion/clip-vit-l-14-datacomp.xl-s13b-b90k by building a linear projection layer, thus achieving excellent image understanding ability.
Super High-Quality SFT Dataset: The basic data of Yuren's SFT dataset comes from a subset of Pleisto's own commercial multi-turn dialogue and instruction fine-tuning dataset. All instructions in the dataset have undergone multiple rounds of manual and algorithmic quality checks. On this basis, we also refer to the work of Orca LLM and conduct data augmentation based on GPT-4 on this subset. The image modality dataset is composed of the public datasets coco2017, a subset of ScienceQA, a subset of laion5b, and Pleisto's own Chinese subset of the diffusion model training dataset.
Business-friendly: Yuren's training and inference code is open-sourced under the Apache-2.0 license, and the authorization of the model weights is fully inherited from the baichuan-7B model license agreement. To obtain a commercial use authorization, simply contact the baichuan team for free registration.
Fully Compatible with ChatML: Yuren is fully compatible with the ChatML format of the same type as GPT-4, which can minimize the security risks caused by prompt injection and achieve good system prompt compliance like GPT-4. (Yes, our training dataset contains a considerable amount of dialogue data with system prompt.)
Use an appropriate format to structure a formal letter of recommendation for a student applying to a prestigious graduate program in computer science
Dear [Recipient's Name],
I am writing to recommend [Student's Name] for admission to the prestigious graduate program in computer science at [Program Name]. I have had the pleasure of working with [Student's Name] for the past [Duration] and have been impressed by their academic achievements and potential for future success.
[Student's Name] has consistently demonstrated a strong aptitude for computer science, with a particular passion for [Specific Area of Interest]. Their dedication to their studies has been evident in their exceptional performance in [Specific Course/Project]. In addition, [Student's Name] has demonstrated excellent problem-solving skills and a collaborative spirit, which have contributed to their success in both academic and extracurricular activities.
[Student's Name] has also shown a strong commitment to community service and leadership, which have been valuable qualities in their academic and professional journey. Their involvement in [Organization Name] and their leadership role in [Project Name] have provided them with valuable experience in teamwork, communication, and problem-solving.
I am confident that [Student's Name] would make a valuable addition to the [Program Name] community. Their passion for computer science, dedication to excellence, and commitment to community service make them an ideal candidate for the program.
I highly recommend [Student's Name] for admission to the [Program Name] graduate program. Thank you for considering my recommendation.
[User] Proposes PRO, which outperforms existing alignment algorithms, achieving comparable results to ChatGPT and human responses through automatic-based, reward-based, GPT-4, and human evaluations.
[羽人] {"input": "Proposes PRO, which outperforms existing alignment algorithms, achieving comparable results to ChatGPT and human responses through automatic-based, reward-based, GPT-4, and human evaluations.", "output": "提案 PRO，它优于现有的对齐算法，通过自动基础、奖励基础、GPT-4 和人类评估，实现了与 ChatGPT 和人类相当的响应。"}
[User] BREAKING: Elon Musk reportedly desires to buy earth and limit how many breaths you can take daily
[羽人] {"input": "BREAKING: Elon Musk reportedly desires to buy earth and limit how many breaths you can take daily", "output": "突发：据报道，埃隆·马斯克想买下地球，并限制每天呼吸的次数。"}
Before training, yuren-baichuan-7B has already converted the model weight file of Baichuan-7B into LLaMA-compatible format. Therefore, when deploying inference for the text-only mode, it is very convenient to directly use transformers' LlamaForCausalLM and LlamaTokenizer for loading.
python
1from transformers import LlamaTokenizer, LlamaForCausalLM
2import torch
34device = torch.device("cuda")5query ="一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。\nWould you rate the previous review as positive, neutral or negative?\nReturn in json object"6model = LlamaForCausalLM.from_pretrained(7"pleisto/yuren-baichuan-7b", torch_dtype=torch.bfloat16, device_map="auto"8)9tokenizer = LlamaTokenizer.from_pretrained("pleisto/yuren-baichuan-7b", use_fast=False)10system_prompt ="<|im_start|>system\nYou are a helpful AI assistant.<|im_end|>\n"11inputs =f"{system_prompt}<|im_start|>user\n{query}<|im_end|>\n<|im_start|>assistant\n"12input_ids = tokenizer(inputs, return_tensors="pt").input_ids.to(device)13generate_ids = model.generate(14 input_ids,15 max_new_tokens=4096,16 do_sample=True,17 top_p=1.0,18 temperature=0.42,19 eos_token_id=64002,20)21output = tokenizer.batch_decode(generate_ids)[0]22print(output)23"""
24<|im_start|> system
25You are a helpful AI assistant. <|im_end|>
26<|im_start|> user
27一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。
28Would you rate the previous review as positive, neutral or negative?
29Retun in json object <|im_end|>
30<|im_start|> assistant
31{
32"rating": "positive"
33} <|im_end|>
34"""
尽管我们已在训练数据和预置的 System Prompt 层面上进行了内容安全的控制，但模型仍然可能会产生偏见、歧视、虚构或不当的内容，我们强烈建议您在使用模型时采取额外的安全措施，例如对模型的输入输出进行过滤、审查或限制，以避免对您的用户造成伤害。
Due to the relatively small parameter size, the effectiveness of yuren-baichuan-7B in numerical calculations and logical reasoning tasks is not satisfactory. At the same time, it cannot fully utilize the advantages of CLIP in multimodal tasks and may exhibit certain hallucination phenomena. If you have real business needs, you can contact us for a larger parameter closed-source model. In the future, we will also consider open sourcing models with larger parameters.
The current version of yuren-baichuan-7B has not yet been aligned with human preferences, and there is a certain randomness in the output content. Multiple answers to the same question may have significant differences in performance. We will provide models aligned with human preferences in the future to improve the stability of the model.
Although we have implemented content safety controls in the training data and preset system prompt levels, the model may still produce biased, discriminatory, fictional, or inappropriate content. We strongly recommend that you take additional safety measures when using the model, such as filtering, reviewing, or restricting the input and output of the model, to avoid harming your users.
