You can deploy this model and start earning money today!

RichardErkhov/
Magpie-Align_-_Llama-3-8B-Magpie-Pro-SFT-300K-v0.1-gguf

Name: Magpie-Align_-_Llama-3-8B-Magpie-Pro-SFT-300K-v0.1-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3-8B-Magpie-Pro-SFT-300K-v0.1 - GGUF

Model creator: https://huggingface.co/Magpie-Align/
Original model: https://huggingface.co/Magpie-Align/Llama-3-8B-Magpie-Pro-SFT-300K-v0.1/

Name	Quant method	Size
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q2_K.gguf	Q2_K	2.96GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.IQ3_XS.gguf	IQ3_XS	3.28GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.IQ3_S.gguf	IQ3_S	3.43GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q3_K_S.gguf	Q3_K_S	3.41GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.IQ3_M.gguf	IQ3_M	3.52GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q3_K.gguf	Q3_K	3.74GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q3_K_M.gguf	Q3_K_M	3.74GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q3_K_L.gguf	Q3_K_L	4.03GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.IQ4_XS.gguf	IQ4_XS	4.18GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q4_0.gguf	Q4_0	4.34GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.IQ4_NL.gguf	IQ4_NL	4.38GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q4_K_S.gguf	Q4_K_S	4.37GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q4_K.gguf	Q4_K	4.58GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q4_K_M.gguf	Q4_K_M	4.58GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q4_1.gguf	Q4_1	4.78GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q5_0.gguf	Q5_0	5.21GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q5_K_S.gguf	Q5_K_S	5.21GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q5_K.gguf	Q5_K	5.34GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q5_K_M.gguf	Q5_K_M	5.34GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q5_1.gguf	Q5_1	5.65GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q6_K.gguf	Q6_K	6.14GB
Llama-3-8B-Magpie-Pro-SFT-300K-v0.1.Q8_0.gguf	Q8_0	7.95GB

Original model description:

license: llama3 base_model: meta-llama/Meta-Llama-3-8B tags:

axolotl
generated_from_trainer model-index:
name: Llama-3-8B-Magpie-Pro-SFT-v0.1 results: [] datasets:
Magpie-Align/Magpie-Pro-300K-Filtered language:
en

🐦 Llama-3-8B-Magpie-Pro-SFT-v0.1

Project Web: https://magpie-align.github.io/

Arxiv Technical Report: https://arxiv.org/abs/2406.08464

Codes: https://github.com/magpie-align/magpie

Abstract

Click Here

High-quality instruction data is critical for aligning large language models (LLMs). Although some models, such as Llama-3-Instruct, have open weights, their alignment data remain private, which hinders the democratization of AI. High human labor costs and a limited, predefined scope for prompting prevent existing open-source data creation methods from scaling effectively, potentially limiting the diversity and quality of public alignment datasets. Is it possible to synthesize high-quality instruction data at scale by extracting it directly from an aligned LLM? We present a self-synthesis method for generating large-scale alignment data named Magpie. Our key observation is that aligned LLMs like Llama-3-Instruct can generate a user query when we input only the left-side templates up to the position reserved for user messages, thanks to their auto-regressive nature. We use this method to prompt Llama-3-Instruct and generate 4 million instructions along with their corresponding responses. We perform a comprehensive analysis of the extracted data and select 300K high-quality instances. To compare Magpie data with other public instruction datasets, we fine-tune Llama-3-8B-Base with each dataset and evaluate the performance of the fine-tuned models. Our results indicate that in some tasks, models fine-tuned with Magpie perform comparably to the official Llama-3-8B-Instruct, despite the latter being enhanced with 10 million data points through supervised fine-tuning (SFT) and subsequent feedback learning. We also show that using Magpie solely for SFT can surpass the performance of previous public datasets utilized for both SFT and preference optimization, such as direct preference optimization with UltraFeedback. This advantage is evident on alignment benchmarks such as AlpacaEval, ArenaHard, and WildBench.

About This Model

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B on Magpie-Align/Magpie-Pro-300K-Filtered dataset.

It achieves performance comparable with the official Llama-3-8B-Instruct Model with SFT only!

Alpaca Eval 2 (GPT-4-Turbo-1106): 25.08 (LC), 29.47 (WR)
Alpaca Eval 2 (Llama-3-8B-Instruct): 52.12 (LC), 53.43 (WR)
Arena Hard: 18.9

Other Information

License: Please follow Meta Llama 3 Community License.

Conversation Template: Please use Llama 3 official chat template for the best performance.

Citation

If you find the model, data, or code useful, please cite our paper:

@misc{xu2024magpie,
	title={Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing}, 
	author={Zhangchen Xu and Fengqing Jiang and Luyao Niu and Yuntian Deng and Radha Poovendran and Yejin Choi and Bill Yuchen Lin},
	year={2024},
	eprint={2406.08464},
	archivePrefix={arXiv},
	primaryClass={cs.CL}
}

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 8
total_train_batch_size: 32
total_eval_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.8664	0.0012	1	0.8860
0.4038	0.9989	825	0.4250
0.327	1.9830	1650	0.4219

Framework versions

Transformers 4.40.2
Pytorch 2.3.0+cu121
Datasets 2.19.1
Tokenizers 0.19.1

See axolotl config

axolotl version: 0.4.0

yaml

base_model: meta-llama/Meta-Llama-3-8B
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: Magpie-Align/Magpie-Pro-300K-Filtered
    type: sharegpt
    conversation: llama3
dataset_prepared_path: last_run_prepared
val_set_size: 0.001
output_dir: ./out_Llama-3-8B-Magpie-Pro-300K-FilteredL

sequence_len: 8192
sample_packing: true
eval_sample_packing: false
pad_to_sequence_len: true

gradient_accumulation_steps: 8
micro_batch_size: 1
num_epochs: 2
optimizer: paged_adamw_8bit
lr_scheduler: cosine
learning_rate: 2e-5

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false
early_stopping_patience:
resume_from_checkpoint:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 100
evals_per_epoch: 1
eval_table_size:
saves_per_epoch: 3
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
  pad_token: <|end_of_text|>

RichardErkhov/Magpie-Align_-_Llama-3-8B-Magpie-Pro-SFT-300K-v0.1-gguf