See axolotl config

axolotl version: 0.13.0.dev0

yaml

1# llama-8B-training.yaml 2# ========================= 3# Model Configuration 4# ========================= 5base_model: meta-llama/Llama-3.1-8B-Instruct 6load_in_4bit: true # Use 4-bit quantization (saves VRAM on smaller GPUs like A100 40GB or L4) 7adapter: qlora 8bnb_4bit_use_double_quant: true # recommended for stability 9bnb_4bit_quant_type: nf4 10bnb_4bit_compute_dtype: bfloat16 # compute in bf16 11trust_remote_code: true # Allow loading models with custom HF code 12tokenizer_name: meta-llama/Llama-3.1-8B-Instruct 13tokenizer_use_fast: true # Faster tokenization 14 15# ========================= 16# Dataset Configuration 17# ========================= 18datasets: 19 - path: Ivoyant/attr-mappings-training 20 split: train 21 type: chat_template 22 chat_template: llama3 # Use built-in Llama 3 chat template 23 field_messages: conversations # Column containing conversation array 24 25 # Optional: Control which roles to train on (default: assistant only) 26 roles_to_train: ["assistant"] 27 28 # Optional: Control EOS token training 29 train_on_eos: turn # Options: "turn", "all", "last" 30 31# val_set_size: 0.1 32 33test_datasets: 34 - path: Ivoyant/attr-mappings-training 35 split: validation 36 type: chat_template 37 chat_template: llama3 # Use built-in Llama 3 chat template 38 field_messages: conversations # Column containing conversation array 39 40 # Optional: Control which roles to train on (default: assistant only) 41 roles_to_train: ["assistant"] 42 43 # Optional: Control EOS token training 44 train_on_eos: turn # Options: "turn", "all", "last" 45 46seed: 42 # Ensures reproducible splits 47 48dataset_prepared_path: /workspace/data/prepared_dataset 49 50# ========================= 51# LoRA Configuration 52# ========================= 53lora_r: 16 54lora_alpha: 32 55lora_dropout: 0.05 56lora_target_modules: 57 - q_proj 58 - v_proj 59 - k_proj 60 - o_proj 61 - gate_proj 62 - up_proj 63 - down_proj 64lora_fan_in_fan_out: false 65 66# ========================= 67# Training Configuration 68# ========================= 69micro_batch_size: 2 70gradient_accumulation_steps: 8 # simulates batch size of 8 71learning_rate: 5e-5 # standard LoRA LR 72num_epochs: 8 73lr_scheduler: cosine # smooth decay 74warmup_steps: 100 # Add warmup for stability 75save_strategy: steps 76save_steps: 500 77# saves_per_epoch: 1 78# evals_per_epoch: 1 79eval_strategy: steps # Evaluate more frequently 80eval_steps: 50 81save_total_limit: 3 # Keep more checkpoints for experimentation 82bf16: true # A40 supports BF16 83fp16: false # don't mix with bf16 84optim: adamw_torch 85gradient_checkpointing: true # saves VRAM at cost of compute 86max_grad_norm: 1.0 87weight_decay: 0.01 88dataloader_num_workers: 2 89 90# ========================= 91# Sequence Configuration 92# ========================= 93sequence_len: 768 94sample_packing: true 95pad_to_sequence_len: true 96 97special_tokens: 98 pad_token: "<|eot_id|>" 99 eos_token: "<|eot_id|>" 100 101# ========================= 102# Output & Logging Configuration 103# ========================= 104output_dir: /workspace/data/outputs/lora-llama-8b-activity-mappings 105logging_steps: 50 106use_tensorboard: true 107logging_strategy: steps 108 109# ========================= 110# Memory & Performance Optimization 111# ========================= 112dataloader_pin_memory: true # ✅ usually better perf unless CPU RAM issue 113remove_unused_columns: true # ✅ recommended by reference 114 115# Early stopping for efficiency 116early_stopping_patience: 3 117load_best_model_at_end: true 118metric_for_best_model: eval_loss 119greater_is_better: false 120

workspace/data/outputs/lora-llama-8b-activity-mappings

• Loss: 0.0437
• Memory/max Active (gib): 7.8
• Memory/max Allocated (gib): 7.8
• Memory/device Reserved (gib): 9.25

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 100
• training_steps: 1816

Training results

Framework versions

• PEFT 0.17.1
• Transformers 4.56.1
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1