Model Details
- Model Developers: Kim Yunho (youknowho)
- Model Type:Causal Language Model (Auto-regressive)
- Input: Text only
- Output: Text only
- Model Architecture: language model based on the Llama2 architecture.
Base Model
Training Dataset
- kyujinpy/Open-platypus-Commercial
Training Method
LoRA / QLoRA: The lora_r values is 16.
LoRA / QLoRA Config
- LoRA_r: 16
- LoRA_alpha:16
- LoRA_dropout:0.05
- LoRA_target_modules: ["gate_proj", "up_proj", "down_proj"]
Benchmark Results
| Model | Average | HellaSwag | MMLU | BoolQ | COPA |
|---|
| SOLAR-10.7B-instrution-v1.0 | 75.01 | 63.71 | 62.53 | 87.83 | 86.00 |
Notes
- All scores are reported under 0-shot (num_fewshot=0) evaluation.
- HellaSwag reports both raw accuracy and normalized accuracy.
- Results obtained using lm-evaluation-harness.
NOTICE
This model was created for personal experimentation and reproducibility purposes by following the training approach of SOLAR-Platypus-10.7B-v1.
It is not affiliated with, endorsed by, or associated with the original author (kyujinpy) or Upstage.
Dataset licensing and usage terms follow the original dataset.
Users are responsible for verifying the licenses of both the dataset and the base model, particularly for any commercial use.
