Model Card for llm-jp-3-13b-it
This model, llm-jp-3-13b-it, is a fine-tuned version of the llm-jp/llm-jp-3-13b large language model, specifically adapted for instruction following in Japanese. It leverages the Hugging Face transformers library and the Unsloth library for efficient training and deployment. The fine-tuning process was conducted using the Ichikara Instruction dataset and incorporates QLoRA for 4-bit quantization, reducing memory footprint and enabling training on resource-constrained environments, such as Google Colab's free tier with T4 GPUs.
- Developed by: Uchio Norihiko/Sora Vision
- Model type: Causal Language Model
- Language(s): Japanese
- License: CC-BY-NC-SA (inherited from Ichikara Instruction dataset)
- Finetuned from model: llm-jp/llm-jp-3-13b
- Repository: [Link to your GitHub repository (if applicable)]
- Demo: [Link to a Hugging Face Space or other demo (if applicable)]
Uses
This model is primarily intended for research and development purposes in Japanese NLP. Its instruction-following fine-tuning makes it suitable for tasks like:
- Text Generation
- Question Answering
- Dialogue Generation
- Summarization
Potential Impact: This model can advance research and development in Japanese NLP by providing a pre-trained and fine-tuned foundation for various tasks.
Limitations: While fine-tuned for instruction following, it may not always generate perfect or unbiased outputs. Users should evaluate outputs carefully and use it responsibly.
Ethical Considerations: Avoid using this model for malicious purposes, such as generating harmful or misleading content. Be aware of potential biases and strive to mitigate them.
How to Get Started
-
Prerequisites: Google Colab or similar environment, the provided training code, fine-tuned LoRA adapter weights on Hugging Face Hub, and a Hugging Face token.
-
Environment: Install libraries: pip install unsloth transformers accelerate bitsandbytes
-
Loading:
1from unsloth import FastLanguageModel
2import torch
3
4model_id = "llm-jp/llm-jp-3-13b"
5adapter_id = "SoraVision/llm-jp-3-13b-it"
6HF_TOKEN = "your_hugging_face_token"
7
8base_model, tokenizer = FastLanguageModel.from_pretrained(
9 model_name=model_id, load_in_4bit=True, trust_remote_code=True, token=HF_TOKEN
10)
11
12model = FastLanguageModel.load_adapter(base_model, adapter_id, token=HF_TOKEN)
13
14FastLanguageModel.for_inference(model)
-
Inference:
1prompt = "### 指示\nあなたの好きな食べ物は？\n### 回答\n"
2inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
3outputs = model.generate(**inputs, max_new_tokens=512, use_cache=True, do_sample=False, repetition_penalty=1.2)
4prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]
5print(prediction)
Training Details
- Base Model: llm-jp/llm-jp-3-13b (13 billion parameters)
- Dataset: Ichikara Instruction (
ichikara-instruction-003-001-2.1.json)
- Method: Supervised Fine-tuning (SFT) with LoRA, 4-bit quantization
- Parameters: Batch size: 2 (per device) with gradient accumulation 4 (effective batch size 8), 1 epoch, learning rate: 2e-4. Uses AdamW optimizer and bf16 mixed precision.
Evaluation
The model's performance is evaluated using the ELYZA-tasks-100-TV dataset. Predictions are saved to a JSONL file for analysis. The code doesn't automatically calculate metrics; users need to implement this based on the ground truth and the generated output.
Bias, Risks, and Limitations
- Data Bias: The model may inherit biases present in the Ichikara Instruction dataset.
- Overfitting: Despite mitigation techniques, overfitting to the training data is a potential risk.
- Misinformation and Manipulation: The model could be misused to generate misleading or harmful content.
Training Data
The Ichikara Instruction dataset is used for fine-tuning. It's a Japanese instruction-following dataset, likely human-generated. The code uses the ichikara-instruction-003-001-2.1.json file. Be aware of potential biases in the data.
Environmental Impact
The training process has an environmental impact due to energy consumption. Use the Machine Learning Impact calculator to estimate carbon emissions based on hardware and training time.
Citation
1@misc{llm-jp,
2 title = {LLM-jp Project},
3 howpublished = {\url{[Project Website or Repository URL]}},
4 note = {Accessed: [Date]}
5}
6
7@misc{ichikara-instruction,
8 title = {Ichikara Instruction Dataset},
9 howpublished = {Accessed through [Application Process/Google Drive Link]},
10 note = {Accessed: [Date]}
11}
Cite relevant libraries (Unsloth, Transformers, trl, etc.) as well. See their respective repositories for citation details.
