AlphaMonarch-laser is a new DPO merge using laserQLoRA that retains all the reasoning abilities of the very best merges and significantly improves its
conversational abilities. Kind of the best of both worlds in a 7B model. This model uses
mlabonne/NeuralMonarch-7B
as its base model, finetuned on only half of the layers using laserQLoRA. The preference dataset used for DPO is
mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha.
agieval_aqua_rat 0 acc 28.35% 2.83%
agieval_aqua_rat 0 acc_norm 26.38% 2.77%
agieval_logiqa_en 0 acc 38.25% 1.91%
agieval_logiqa_en 0 acc_norm 38.10% 1.90%
agieval_lsat_ar 0 acc 23.91% 2.82%
agieval_lsat_ar 0 acc_norm 23.48% 2.80%
agieval_lsat_lr 0 acc 52.75% 2.21%
agieval_lsat_lr 0 acc_norm 53.92% 2.21%
agieval_lsat_rc 0 acc 66.91% 2.87%
agieval_lsat_rc 0 acc_norm 67.29% 2.87%
agieval_sat_en 0 acc 78.64% 2.86%
agieval_sat_en 0 acc_norm 78.64% 2.86%
agieval_sat_en_without_passage 0 acc 45.15% 3.48%
agieval_sat_en_without_passage 0 acc_norm 44.17% 3.47%
agieval_sat_math 0 acc 33.18% 3.18%
agieval_sat_math 0 acc_norm 31.36% 3.14%
Here is the converted table in the required format, including multiplication of all values by 100 and calculating the average for the value column:
1base_model: mlabonne/NeuralMonarch-7B
2model_type: MistralForCausalLM
3tokenizer_type: LlamaTokenizer
4is_mistral_derived_model: true
5load_in_8bit: false
6load_in_4bit: true
7strict: false
8rl: dpo
9chat_template: chatml
10datasets:
11 - path: mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha
12 split: train
13 type: chatml.intel
14dataset_prepared_path:
15val_set_size: 0.01
16output_dir: ./out
17adapter: qlora
18lora_model_dir:
19sequence_len: 1800
20sample_packing: false
21pad_to_sequence_len: false
22lora_r: 16
23lora_alpha: 16
24lora_dropout: 0.05
25lora_target_linear: true
26lora_fan_in_fan_out:
27lora_target_modules:
28 - layers.1.self_attn.q_proj
29 - layers.0.self_attn.q_proj
30 - layers.15.self_attn.q_proj
31 - layers.12.self_attn.q_proj
32 - layers.11.self_attn.q_proj
33 - layers.14.self_attn.q_proj
34 - layers.9.self_attn.q_proj
35 - layers.16.self_attn.q_proj
36 - layers.30.self_attn.q_proj
37 - layers.18.self_attn.q_proj
38 - layers.13.self_attn.q_proj
39 - layers.10.self_attn.q_proj
40 - layers.7.self_attn.q_proj
41 - layers.8.self_attn.q_proj
42 - layers.4.self_attn.q_proj
43 - layers.19.self_attn.q_proj
44 - layers.27.self_attn.k_proj
45 - layers.24.self_attn.k_proj
46 - layers.25.self_attn.k_proj
47 - layers.22.self_attn.k_proj
48 - layers.26.self_attn.k_proj
49 - layers.29.self_attn.k_proj
50 - layers.23.self_attn.k_proj
51 - layers.28.self_attn.k_proj
52 - layers.21.self_attn.k_proj
53 - layers.31.self_attn.k_proj
54 - layers.30.self_attn.k_proj
55 - layers.20.self_attn.k_proj
56 - layers.5.self_attn.k_proj
57 - layers.19.self_attn.k_proj
58 - layers.17.self_attn.k_proj
59 - layers.18.self_attn.k_proj
60 - layers.19.self_attn.v_proj
61 - layers.24.self_attn.v_proj
62 - layers.18.self_attn.v_proj
63 - layers.5.self_attn.v_proj
64 - layers.3.self_attn.v_proj
65 - layers.16.self_attn.v_proj
66 - layers.23.self_attn.v_proj
67 - layers.27.self_attn.v_proj
68 - layers.25.self_attn.v_proj
69 - layers.26.self_attn.v_proj
70 - layers.20.self_attn.v_proj
71 - layers.6.self_attn.v_proj
72 - layers.15.self_attn.v_proj
73 - layers.17.self_attn.v_proj
74 - layers.29.self_attn.v_proj
75 - layers.22.self_attn.v_proj
76 - layers.12.self_attn.o_proj
77 - layers.9.self_attn.o_proj
78 - layers.14.self_attn.o_proj
79 - layers.0.self_attn.o_proj
80 - layers.6.self_attn.o_proj
81 - layers.8.self_attn.o_proj
82 - layers.10.self_attn.o_proj
83 - layers.11.self_attn.o_proj
84 - layers.13.self_attn.o_proj
85 - layers.24.self_attn.o_proj
86 - layers.7.self_attn.o_proj
87 - layers.15.self_attn.o_proj
88 - layers.5.self_attn.o_proj
89 - layers.17.self_attn.o_proj
90 - layers.25.self_attn.o_proj
91 - layers.4.self_attn.o_proj
92 - layers.31.mlp.gate_proj
93 - layers.30.mlp.gate_proj
94 - layers.4.mlp.gate_proj
95 - layers.3.mlp.gate_proj
96 - layers.29.mlp.gate_proj
97 - layers.28.mlp.gate_proj
98 - layers.6.mlp.gate_proj
99 - layers.27.mlp.gate_proj
100 - layers.5.mlp.gate_proj
101 - layers.26.mlp.gate_proj
102 - layers.25.mlp.gate_proj
103 - layers.7.mlp.gate_proj
104 - layers.2.mlp.gate_proj
105 - layers.24.mlp.gate_proj
106 - layers.23.mlp.gate_proj
107 - layers.10.mlp.gate_proj
108 - layers.6.mlp.up_proj
109 - layers.4.mlp.up_proj
110 - layers.5.mlp.up_proj
111 - layers.27.mlp.up_proj
112 - layers.25.mlp.up_proj
113 - layers.26.mlp.up_proj
114 - layers.17.mlp.up_proj
115 - layers.24.mlp.up_proj
116 - layers.7.mlp.up_proj
117 - layers.10.mlp.up_proj
118 - layers.3.mlp.up_proj
119 - layers.11.mlp.up_proj
120 - layers.23.mlp.up_proj
121 - layers.9.mlp.up_proj
122 - layers.14.mlp.up_proj
123 - layers.18.mlp.up_proj
124 - layers.19.mlp.down_proj
125 - layers.20.mlp.down_proj
126 - layers.18.mlp.down_proj
127 - layers.21.mlp.down_proj
128 - layers.29.mlp.down_proj
129 - layers.1.mlp.down_proj
130 - layers.22.mlp.down_proj
131 - layers.28.mlp.down_proj
132 - layers.23.mlp.down_proj
133 - layers.30.mlp.down_proj
134 - layers.17.mlp.down_proj
135 - layers.4.mlp.down_proj
136 - layers.2.mlp.down_proj
137 - layers.15.mlp.down_proj
138 - layers.5.mlp.down_proj
139wandb_project: axolotl
140wandb_entity:
141wandb_watch:
142wandb_name:
143wandb_log_model:
144gradient_accumulation_steps: 8
145micro_batch_size: 1
146num_epochs: 1
147optimizer: paged_adamw_32bit
148lr_scheduler: cosine
149learning_rate: 5e-7
150train_on_inputs: false
151group_by_length: false
152bf16: true
153fp16: false
154tf32: true
155gradient_checkpointing: true
156early_stopping_patience:
157resume_from_checkpoint:
158local_rank:
159logging_steps: 1
160xformers_attention:
161flash_attention: true
162warmup_steps: 100
163evals_per_epoch: 1
164eval_table_size:
165eval_table_max_new_tokens: 128
166save_steps: 1080
167max_steps: 1080
168debug:
169deepspeed:
170weight_decay: 0.0
171fsdp:
172fsdp_config:
173special_tokens:
