Alpaca Lora adapter weight fine-tuned on following instruction dataset.
Training script: borrowed from the official
Alpaca-LoRA implementation
We use following parameter.
python finetune.py \
--base_model 'decapoda-research/llama-7b-hf' \
--data_path 'rewoo/planner_instruction_tuning_2k' \
--output_dir './lora-alpaca-planner' \
--batch_size 128 \
--micro_batch_size 8 \
--num_epochs 10 \
--learning_rate 1e-4 \
--cutoff_len 1024 \
--val_set_size 200 \
--lora_r 8 \
--lora_alpha 16 \
--lora_dropout 0.05 \
--lora_target_modules '[q_proj,v_proj]' \
--train_on_inputs \
--group_by_length \
--resume_from_checkpoint 'tloen/alpaca-lora-7b'
