• Model creator: https://huggingface.co/axolotl-ai-co/
• Original model: https://huggingface.co/axolotl-ai-co/romulus-mistral-nemo-12b-simpo/

Original model description:

• generated_from_trainer model-index:
• name: outputs/simpo-out results: []

---

See axolotl config

axolotl version: 0.4.1

yaml

1base_model: winglian/m12b-20240721-test010 2tokenizer_type: AutoTokenizer 3 4load_in_8bit: false 5load_in_4bit: false 6strict: false 7 8chat_template: chatml 9rl: simpo 10rl_beta: 2.5 11cpo_alpha: 0.05 12simpo_gamma: 0.1 13datasets: 14 - path: princeton-nlp/gemma2-ultrafeedback-armorm 15 type: chat_template.default 16 chat_template: chatml 17 field_messages: chosen 18 field_chosen: chosen 19 field_rejected: rejected 20 message_field_role: role 21 message_field_content: content 22 roles: 23 system: 24 - system 25 user: 26 - user 27 assistant: 28 - assistant 29 30dataset_prepared_path: 31val_set_size: 0.0 32output_dir: ./outputs/simpo-out 33 34save_safetensors: true 35save_only_model: true # fsdp seems to crap out saving the optimizer 36 37sequence_len: 8192 38sample_packing: false 39pad_to_sequence_len: true 40 41adapter: 42lora_model_dir: 43lora_r: 256 44lora_alpha: 256 45lora_dropout: 0.1 46lora_target_linear: true 47lora_fan_in_fan_out: 48 # peft_use_rslora: true 49 50wandb_project: romulus-12b 51wandb_entity: oaaic 52wandb_watch: 53wandb_name: 54wandb_log_model: 55 56gradient_accumulation_steps: 16 57micro_batch_size: 1 58num_epochs: 1 59optimizer: adamw_torch 60lr_scheduler: cosine 61learning_rate: 5.0e-7 62 63train_on_inputs: false 64group_by_length: false 65bf16: auto 66fp16: 67tf32: false 68 69gradient_checkpointing: true 70early_stopping_patience: 71resume_from_checkpoint: 72local_rank: 73logging_steps: 1 74xformers_attention: 75flash_attention: true 76s2_attention: 77 78warmup_steps: 25 79evals_per_epoch: 4 80eval_table_size: 81eval_max_new_tokens: 128 82saves_per_epoch: 1 83debug: 84deepspeed: deepspeed_configs/zero3_bf16_cpuoffload_params.json 85weight_decay: 0.0 86fsdp: 87fsdp_config: 88

outputs/simpo-out

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

• learning_rate: 5e-07
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 25
• training_steps: 466

Training results

Framework versions

• Transformers 4.43.1
• Pytorch 2.3.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1