• Model creator: https://huggingface.co/lightblue/
• Original model: https://huggingface.co/lightblue/suzume-llama-3-8B-multilingual-orpo-borda-top75/

Original model description:

• generated_from_trainer base_model: lightblue/suzume-llama-3-8B-multilingual model-index:
• name: workspace/llm_training/axolotl/llama3-multilingual-orpo/output_mitsu_top75_borda results: []

---

Suzume ORPO

Model list

• Trained on the top/bottom responses of all prompts in the dataset: lightblue/suzume-llama-3-8B-multilingual-orpo-borda-full
• Trained on the top/bottom responses of the prompts of the 75% most consistently ranked responses in the dataset: lightblue/suzume-llama-3-8B-multilingual-orpo-borda-top75
• Trained on the top/bottom responses of the prompts of the 50% most consistently ranked responses in the dataset: lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half
• Trained on the top/bottom responses of the prompts of the 25% most consistently ranked responses in the dataset: lightblue/suzume-llama-3-8B-multilingual-orpo-borda-top25

Model results

• meta-llama/Meta-Llama-3-8B-Instruct - The foundation model that our models are ultimately built upon
• Nexusflow/Starling-LM-7B-beta - The highest performing open model on the Chatbot arena that is of a similar size to ours
• gpt-3.5-turbo - A fairly high quality (although not state-of-the-art) proprietary LLM
• lightblue/suzume-llama-3-8B-multilingual - The base model which we train our ORPO finetunes from

Training data

Training configuration

See axolotl config

axolotl version: 0.4.0

yaml

1base_model: lightblue/suzume-llama-3-8B-multilingual 2model_type: LlamaForCausalLM 3tokenizer_type: AutoTokenizer # PreTrainedTokenizerFast 4 5load_in_8bit: false 6load_in_4bit: false 7strict: false 8 9rl: orpo 10orpo_alpha: 0.1 11remove_unused_columns: false 12 13chat_template: chatml 14datasets: 15 - path: lightblue/mitsu_top75_borda 16 type: orpo.chat_template 17 conversation: llama-3 18dataset_prepared_path: /workspace/llm_training/axolotl/llama3-multilingual-orpo/prepared_mitsu_top75_borda 19val_set_size: 0.02 20output_dir: /workspace/llm_training/axolotl/llama3-multilingual-orpo/output_mitsu_top75_borda 21 22sequence_len: 8192 23sample_packing: false 24pad_to_sequence_len: true 25 26use_wandb: true 27wandb_project: axolotl 28wandb_entity: peterd 29wandb_name: mitsu_top75_borda 30 31gradient_accumulation_steps: 8 32micro_batch_size: 1 33num_epochs: 1 34optimizer: paged_adamw_8bit 35lr_scheduler: cosine 36learning_rate: 8e-6 37 38train_on_inputs: false 39group_by_length: false 40bf16: auto 41fp16: 42tf32: false 43 44gradient_checkpointing: true 45gradient_checkpointing_kwargs: 46 use_reentrant: false 47early_stopping_patience: 48resume_from_checkpoint: 49logging_steps: 1 50xformers_attention: 51flash_attention: true 52 53warmup_steps: 10 54evals_per_epoch: 20 55eval_table_size: 56saves_per_epoch: 1 57debug: 58deepspeed: /workspace/axolotl/deepspeed_configs/zero3_bf16.json 59weight_decay: 0.0 60special_tokens: 61 pad_token: <|end_of_text|>

workspace/llm_training/axolotl/llama3-multilingual-orpo/output_mitsu_top75_borda

• Loss: 0.0863

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

• learning_rate: 8e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• total_eval_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 1

Training results

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.0

How to cite

Developer