LLMでの出力方法

開発手順

1. base: model_id = "llm-jp/llm-jp-3-13b"
2. ichikara-instruction-003-001-1.jsonを用いてSFT学習
3. cyberagent/chatbot-arena-ja-calm2-7b-chat-experimentalでDPO学習
   SFT: LoRa.py
   DPO: DPO.py

Uploaded model

• Developed by: hiro877
• License: apache-2.0
• Finetuned from model : llm-jp/llm-jp-3-13b

DPO Trained Model

🚀 Model Overview

• Model Name: DPO Trained Model (1215 version)
• Base Model: llm-jp/llm-jp-3-13b
• Training Steps:
  1. SFT (Supervised Fine-Tuning) using Ichikara Instruction
     • License: CC-BY-NC-SA 4.0
  2. DPO Training using cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental
     • License: CC-BY 4.0

🔧 How to Use

1. Load the model using Hugging Face Transformers:
   python

   1from transformers import AutoModelForCausalLM, AutoTokenizer 2 3model = AutoModelForCausalLM.from_pretrained("hiro877/dpo_trained_model_1215") 4tokenizer = AutoTokenizer.from_pretrained("hiro877/dpo_trained_model_1215")
2. Generate text:
   python

   1input_text = "こんにちは、今日はどうされましたか？" 2inputs = tokenizer(input_text, return_tensors="pt") 3outputs = model.generate(inputs["input_ids"], max_length=50) 4print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📊 Training Details

1. Datasets Used

1. Ichikara Instruction
   • Source: Ichikara Instruction Dataset
   • Purpose: Supervised fine-tuning (SFT)
   • License: CC-BY-NC-SA 4.0
2. cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental
   • Source: CyberAgent Dataset
   • Purpose: DPO training
   • License: CC-BY 4.0

2. Pre-trained Model

• Base Model: llm-jp/llm-jp-3-13b
  • License: Apache License 2.0

🔒 License

1. Attribution: Provide appropriate credit.
2. NonCommercial: You may not use the material for commercial purposes.
3. ShareAlike: Distribute your contributions under the same license.

⚙️ Acknowledgements

• Special thanks to CyberAgent and Ichikara for providing high-quality datasets.
• The base model was developed by llm-jp.

🔄 Update Log

[2024-12-15] Initial Release

• Fine-tuned with Ichikara Instruction (SFT)
• Trained with cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental (DPO)