You can deploy this model and start earning money today!

RichardErkhov/
Cartinoe5930_-_KoRAE-13b-gguf

Name: Cartinoe5930_-_KoRAE-13b-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

KoRAE-13b - GGUF

Model creator: https://huggingface.co/Cartinoe5930/
Original model: https://huggingface.co/Cartinoe5930/KoRAE-13b/

Name	Quant method	Size
KoRAE-13b.Q2_K.gguf	Q2_K	4.6GB
KoRAE-13b.IQ3_XS.gguf	IQ3_XS	5.08GB
KoRAE-13b.IQ3_S.gguf	IQ3_S	5.36GB
KoRAE-13b.Q3_K_S.gguf	Q3_K_S	5.36GB
KoRAE-13b.IQ3_M.gguf	IQ3_M	5.66GB
KoRAE-13b.Q3_K.gguf	Q3_K	5.99GB
KoRAE-13b.Q3_K_M.gguf	Q3_K_M	5.99GB
KoRAE-13b.Q3_K_L.gguf	Q3_K_L	6.54GB
KoRAE-13b.IQ4_XS.gguf	IQ4_XS	6.63GB
KoRAE-13b.Q4_0.gguf	Q4_0	6.95GB
KoRAE-13b.IQ4_NL.gguf	IQ4_NL	7.0GB
KoRAE-13b.Q4_K_S.gguf	Q4_K_S	7.01GB
KoRAE-13b.Q4_K.gguf	Q4_K	7.42GB
KoRAE-13b.Q4_K_M.gguf	Q4_K_M	7.42GB
KoRAE-13b.Q4_1.gguf	Q4_1	7.71GB
KoRAE-13b.Q5_0.gguf	Q5_0	8.46GB
KoRAE-13b.Q5_K_S.gguf	Q5_K_S	8.46GB
KoRAE-13b.Q5_K.gguf	Q5_K	8.7GB
KoRAE-13b.Q5_K_M.gguf	Q5_K_M	8.7GB
KoRAE-13b.Q5_1.gguf	Q5_1	9.21GB
KoRAE-13b.Q6_K.gguf	Q6_K	10.06GB
KoRAE-13b.Q8_0.gguf	Q8_0	13.03GB

Original model description:

license: cc-by-nc-sa-4.0 datasets:

Cartinoe5930/KoRAE_filtered_12k language:
ko library_name: transformers

KoRAE

We introduce KoRAE which finetuned with filtered high-quality Korean dataset.

The KoRAE is output of combination of high-quality data which filtered by special data filtering method and Korean Llama-2 that Korean vocabularis were added. We utilized special data filtering methods which introduced in AlpaGasus to filter high-quality data from mixture of several Korean datasets(OpenOrca-KO, KOpen-Platypus, KoCoT_2000, databricks-dolly-15k-ko). We finetuned Korean Llama-2 that introduced by @beomi on the filtered dataset. The Flash-Attention2 and LoRA were utilized for efficient finetuning.

The finding of KoRAE is as follows:

The finetuning in some epochs showed that high-quality filtered data has positive effects on model's performance. However, finetuning in a few epochs, the quantity of data is more matter than quality. It seems to be due to the lack of performance of the Korean base model. Therefore, the research to improve the Korean base model must continue.
The model trained with DPO showed best performance among KoRAE variants. This shows that DPO is clearly effective in the Korean LLM.
The model finetuned with filtered high-quality KoRAE showed better performance than without. Therefore, for better LLM, we should try to finetune the LLM with high-quality data.

Model Details

Developed by: Cartinoe5930
Base model: beomi/llama-2-koen-13b
Repository: gauss5930/KoRAE

For more details, please check the GitHub Repository!

Training Details

Hardward: We utilized A100 80G for finetuning
Training factors: The Transformers Trainer and Huggingface PEFT were utilized for finetuning.
Training Details: Supervised finetuning 3 epochs on filtered KoRAE dataset

For more details, please check the GitHub Repository!

Training Dataset

The KoRAE was finetuned with KoRAE dataset filtered high-quality dataset. This dataset is a combination of the publicly available Koraen dataset and a filtering method was applied to the result of the combination dataset. For more information, please refer to the dataset card of KoRAE.

Open Ko-LLM Leaderboard

Model	Average	Ko-ARC	Ko-HellaSwag	Ko-MMLU	Ko-TruthfulQA	Ko-CommonGen V2
KoRAE-13b	48.64	46.33	57.25	42.8	41.08	55.73

Prompt Template

### System:
{system_prompt}

### User:
{instruction + input}

### Assistant:
{output}

Usage example

python

# Use a pipeline as a high-level helper
from transformers import pipeline
import torch

pipe = pipeline("text-generation", model="Cartinoe5930/KoRAE-13b", torch_dtype=torch.bfloat16, device_map="auto")
messages = [
    {
        "role": "system",
        "content": "당신은 유용한 인공지능 비서입니다. 사용자가 몇 가지 지시가 포함된 작업을 제공합니다. 요청을 적절히 완료하는 응답을 작성하세요.",
    },
    {"role": "user", "content": "스트레스를 해소하는 5가지 방법에 대해서 설명해줘."}
]

prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Citation

@inproceedings{lee2023kullm,
  title={KULLM: Learning to Construct Korean Instruction-following Large Language Models},
  author={Lee, SeungJun and Lee, Taemin and Lee, Jeongwoo and Jang, Yoona and Lim, Heuiseok},
  booktitle={Annual Conference on Human and Language Technology},
  pages={196--202},
  year={2023},
  organization={Human and Language Technology}
}

@misc{chen2023alpagasus,
      title={AlpaGasus: Training A Better Alpaca with Fewer Data}, 
      author={Lichang Chen and Shiyang Li and Jun Yan and Hai Wang and Kalpa Gunaratna and Vikas Yadav and Zheng Tang and Vijay Srinivasan and Tianyi Zhou and Heng Huang and Hongxia Jin},
      year={2023},
      eprint={2307.08701},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc {l._junbum_2023,
    author       = { {L. Junbum, Taekyoon Choi} },
    title        = { llama-2-koen-13b },
    year         = 2023,
    url          = { https://huggingface.co/beomi/llama-2-koen-13b },
    doi          = { 10.57967/hf/1280 },
    publisher    = { Hugging Face }
}

RichardErkhov/Cartinoe5930_-_KoRAE-13b-gguf