Model Card for Model ID
A LoRA model trained on nothingiisreal/MN-12B-Celeste-V1.9 with Chat-Error/limarp dataset
Model Details
Model Description
- Developed by: supaahiro
- Model type: Decoder only
- Language(s): English
- License: CC BY-NC-SA v4
- Finetuned from model: nothingiisreal/MN-12B-Celeste-V1.9
Uses
Direct Use
This model was uses the ChatML format:
<|im_start|>system\n<|im_end|>\n<|im_start|>user\n%instruction%: %input_text%<|im_end|>\n<|im_start|>assistant\n%output_text%<|im_end|>
Bias, Risks, and Limitations
NSFW, long context, wrapup bias and narration.
Training Data
5000 examples mixed from LIMAERP and LIMA
Training Procedure
Training Hyperparameters
3 epochs with 3e-4 lr, 4096 ctx token, QLoRA
Speeds, Sizes, Times [optional]
It takes about 18 hours to train this model
Environmental Impact
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
- Hardware Type: A40 with 9CPUs 48gb ram
- Hours used: 18
- Cloud Provider: Runpod
- Compute Region: US
- Carbon Emitted: 0.5KG
Credits
This model is based on a fine-tuned version of nothingiisreal/MN-12B-Celeste-V1.9 by Chat-Error/limarp using dataset.
