You can deploy this model and start earning money today!

RichardErkhov/
Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf

Name: Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

LION-Gemma-2b-dpo-v1.0 - GGUF

Model creator: https://huggingface.co/Columbia-NLP/
Original model: https://huggingface.co/Columbia-NLP/LION-Gemma-2b-dpo-v1.0/

Name	Quant method	Size
LION-Gemma-2b-dpo-v1.0.Q2_K.gguf	Q2_K	1.08GB
LION-Gemma-2b-dpo-v1.0.IQ3_XS.gguf	IQ3_XS	1.16GB
LION-Gemma-2b-dpo-v1.0.IQ3_S.gguf	IQ3_S	1.2GB
LION-Gemma-2b-dpo-v1.0.Q3_K_S.gguf	Q3_K_S	1.2GB
LION-Gemma-2b-dpo-v1.0.IQ3_M.gguf	IQ3_M	1.22GB
LION-Gemma-2b-dpo-v1.0.Q3_K.gguf	Q3_K	1.29GB
LION-Gemma-2b-dpo-v1.0.Q3_K_M.gguf	Q3_K_M	1.29GB
LION-Gemma-2b-dpo-v1.0.Q3_K_L.gguf	Q3_K_L	1.36GB
LION-Gemma-2b-dpo-v1.0.IQ4_XS.gguf	IQ4_XS	1.4GB
LION-Gemma-2b-dpo-v1.0.Q4_0.gguf	Q4_0	1.44GB
LION-Gemma-2b-dpo-v1.0.IQ4_NL.gguf	IQ4_NL	1.45GB
LION-Gemma-2b-dpo-v1.0.Q4_K_S.gguf	Q4_K_S	1.45GB
LION-Gemma-2b-dpo-v1.0.Q4_K.gguf	Q4_K	1.52GB
LION-Gemma-2b-dpo-v1.0.Q4_K_M.gguf	Q4_K_M	1.52GB
LION-Gemma-2b-dpo-v1.0.Q4_1.gguf	Q4_1	1.56GB
LION-Gemma-2b-dpo-v1.0.Q5_0.gguf	Q5_0	1.68GB
LION-Gemma-2b-dpo-v1.0.Q5_K_S.gguf	Q5_K_S	1.68GB
LION-Gemma-2b-dpo-v1.0.Q5_K.gguf	Q5_K	1.71GB
LION-Gemma-2b-dpo-v1.0.Q5_K_M.gguf	Q5_K_M	1.71GB
LION-Gemma-2b-dpo-v1.0.Q5_1.gguf	Q5_1	1.79GB
LION-Gemma-2b-dpo-v1.0.Q6_K.gguf	Q6_K	1.92GB
LION-Gemma-2b-dpo-v1.0.Q8_0.gguf	Q8_0	2.49GB

Original model description:

library_name: transformers tags: []

Model Card for LION-Gemma-2b-dpo-v1.0

The LION-series are trained using an empirically optimized pipeline that consists of three stages: SFT, DPO, and online preference learning (online DPO). We find simple techniques such as sequence packing, loss masking in SFT, increasing the preference dataset size in DPO, and online DPO training can significantly improve the performance of language models. Our best models (the LION-series) exceed the performance of the official instruct models tuned with closed-source data and algorithms.

For training datasets, code, and evaluation scripts, please refer to our paper and codebase.

Model description

This model is finetuned from Columbia-NLP/LION-Gemma-2b-sft-v1.0 using DPO from the LION pipeline.

Model type: gemma-2b
Language(s) (NLP): Primarily English
License: Gemma Terms of Use
Finetuned from model: Columbia-NLP/LION-Gemma-2b-sft-v1.0

Performance

Model	Method	Size	Arena-Hard	AlpacaEval-2	MT-Bench	OpenLLM
Gemma-2b	-	2B	-	-	-	46.69
Gemma-2b-it	SFT+RLHF	2B	3.4	5.44	5.63	42.75
Gemma-2b-zephyr	SFT+DPO	2B	0.9	2.65	4.13	46.92
LLaMA-2-7b-chat	SFT	7B	4.6	5.35	6.22	53.16
Vicuna-7b-v1.5	SFT	7B	2.5	7.62	6.57	52.06
LION-Gemma-2b-sft-v1.0 (ours)	SFT	2B	2.4	7.79	6.37	54.78
⮕ LION-Gemma-2b-dpo-v1.0 (ours)	SFT+DPO	2B	4.6	8.75	6.58	55.35
LION-Gemma-2b-odpo-v1.0 (ours)	SFT+DPO+ODPO	2B	5.0	9.57	6.75	55.98

Intended uses

To ensure reproducibility, please use the following chat templates:

python

import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="Columbia-NLP/LION-Gemma-2b-dpo-v1.0",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
messages = [
    {
        "role": "system",
        "content": "",
    },
    {
        "role": "user", 
        "content": "Write a short paragraph where every sentence start with the letter A."
    },
]
outputs = pipe(
    messages,
    max_new_tokens=128,
    do_sample=True,
    temperature=0.7,
    top_p=0.7,
    stop_sequence="<|im_end|>",
)
print(outputs[0]["generated_text"][-1]["content"])
# Alice always aspired to achieve academic excellence.
# After attending an esteemed academy, she acquired a plethora of knowledge and attended various extracurricular activities.
# Always eager to apply her newfound skills, Alice undertook ambitious projects and attended various workshops.
# As a result, Alice acquired a remarkable academic record and became an active member of her community.

to inspect the chat template/manually do generation:

python

tokenizer = AutoTokenizer.from_pretrained("Columbia-NLP/LION-Gemma-2b-dpo-v1.0")
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(prompt)
# tokenize prompt and use model.generate

Training details

Please refer to our paper and codebase.

Citation Information

If you find this model useful in your work, please consider citing our paper:

@misc{yu2024lionsempiricallyoptimizedapproach,
      title={LIONs: An Empirically Optimized Approach to Align Language Models}, 
      author={Xiao Yu and Qingyang Wu and Yu Li and Zhou Yu},
      year={2024},
      eprint={2407.06542},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.06542}, 
}

Acknowledgements

We thank the Columbia-NLP group and articulate.ai for providing OpenAI API credits and computational resources to conduct our experiments.

RichardErkhov/Columbia-NLP_-_LION-Gemma-2b-dpo-v1.0-gguf