You can deploy this model and start earning money today!

RichardErkhov/
alibaba-pai_-_Qwen2-7B-Instruct-Refine-gguf

Name: alibaba-pai_-_Qwen2-7B-Instruct-Refine-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Qwen2-7B-Instruct-Refine - GGUF

Model creator: https://huggingface.co/alibaba-pai/
Original model: https://huggingface.co/alibaba-pai/Qwen2-7B-Instruct-Refine/

Name	Quant method	Size
Qwen2-7B-Instruct-Refine.Q2_K.gguf	Q2_K	2.81GB
Qwen2-7B-Instruct-Refine.Q3_K_S.gguf	Q3_K_S	3.25GB
Qwen2-7B-Instruct-Refine.Q3_K.gguf	Q3_K	3.55GB
Qwen2-7B-Instruct-Refine.Q3_K_M.gguf	Q3_K_M	3.55GB
Qwen2-7B-Instruct-Refine.Q3_K_L.gguf	Q3_K_L	3.81GB
Qwen2-7B-Instruct-Refine.IQ4_XS.gguf	IQ4_XS	3.96GB
Qwen2-7B-Instruct-Refine.Q4_0.gguf	Q4_0	4.13GB
Qwen2-7B-Instruct-Refine.IQ4_NL.gguf	IQ4_NL	4.16GB
Qwen2-7B-Instruct-Refine.Q4_K_S.gguf	Q4_K_S	4.15GB
Qwen2-7B-Instruct-Refine.Q4_K.gguf	Q4_K	4.36GB
Qwen2-7B-Instruct-Refine.Q4_K_M.gguf	Q4_K_M	4.36GB
Qwen2-7B-Instruct-Refine.Q4_1.gguf	Q4_1	4.54GB
Qwen2-7B-Instruct-Refine.Q5_0.gguf	Q5_0	4.95GB
Qwen2-7B-Instruct-Refine.Q5_K_S.gguf	Q5_K_S	4.95GB
Qwen2-7B-Instruct-Refine.Q5_K.gguf	Q5_K	5.07GB
Qwen2-7B-Instruct-Refine.Q5_K_M.gguf	Q5_K_M	5.07GB
Qwen2-7B-Instruct-Refine.Q5_1.gguf	Q5_1	5.36GB
Qwen2-7B-Instruct-Refine.Q6_K.gguf	Q6_K	5.82GB
Qwen2-7B-Instruct-Refine.Q8_0.gguf	Q8_0	7.54GB

Original model description:

📖 Introduction

Qwen2-7B-Instruct-Refine and Qwen2-1.5B-Instruct-Refine are two powerful large language models that act as proficient prompt engineers. They can optimize and refine the prompts input by users, and the generated optimized instructions can significantly enhance the LLM's ability to produce better and more informative responses for users.

We fine-tuned Qwen2-7B-Instruct and Qwen2-1.5B-Instruct to obtain Qwen2-7B-Instruct-Refine and Qwen2-1.5B-Instruct-Refine. We sampled the dataset from OpenHermes and the LCCD dataset, ensuring a balanced task distribution. For training set annotations, we used Qwen-max with incorporated our handwritten examples as in-context prompts.

🚀 Quick Start

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    "alibaba-pai/Qwen2-1.5B-Instruct-Refine",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("alibaba-pai/Qwen2-1.5B-Instruct-Refine")

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=2048，
    eos_token_id=151645，
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

🔍 Evaluation

We used single-turn instructions from MT-Bench as input for Qwen2-1.5B-Instruct and Qwen2-7B-Instruct. GPT4-turbo is used to evaluate the changes in the level of detail and truthfulness of responses to our model's revised instructions.

Model	Detail	Truthfulness
Qwen2-1.5B-Instruct	50.00%	50.00%
+ Qwen2-1.5B-Instruct-Refine	75.63%	63.75%
+ Qwen2-7B-Instruct-Refine	76.56%	62.19%
Qwen2-7B-Instruct	50.00%	50.00%
+ Qwen2-1.5B-Instruct-Refine	70.94%	57.19%
+ Qwen2-7B-Instruct-Refine	74.69%	58.44%

📜 Citation

If you find our work helpful, please cite it!

@misc{TAPIR,
      title={Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning}, 
      author={Yuanhao Yue and Chengyu Wang and Jun Huang and Peng Wang},
      year={2024},
      eprint={2405.13448},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2405.13448}, 
}

RichardErkhov/alibaba-pai_-_Qwen2-7B-Instruct-Refine-gguf

📖 Introduction

🚀 Quick Start

🔍 Evaluation

📜 Citation

RichardErkhov/
alibaba-pai_-_Qwen2-7B-Instruct-Refine-gguf