Llama-2-7b-chat — Guanaco English QLoRA Adapter (LoRA)

✅ Important: This is NOT a full standalone model. It is only the adapter weights + config (+ tokenizer files).
To use it, you must load the base model and then apply this adapter.

What’s included in this repo

• adapter_model.safetensors — LoRA adapter weights (~160 MB)
• adapter_config.json — PEFT configuration (r, alpha, target_modules, etc.)
• Tokenizer artifacts (tokenizer.json, tokenizer.model, tokenizer_config.json, special_tokens_map.json, chat_template.jinja) for convenience

Base model

• Base: meta-llama/Llama-2-7b-chat-hf
• You must have access to the Llama 2 gated repo to download the base model.

Training data

• Dataset: saketgarodia1/guanaco-llama2-chat-en
• Format: instruction-style conversations flattened into a text field
• Split sizes (this run):
  • Train: 3,539
  • Eval: 190

Training setup

Method

• SFT (Supervised Fine-Tuning) using trl.SFTTrainer
• QLoRA via bitsandbytes 4-bit quantization (NF4) + PEFT LoRA adapters

Key hyperparameters

• Epochs: 1
• Max sequence length: 1024
• Batch size per device: 1
• Gradient accumulation: 8 (effective batch size ≈ 8)
• Optimizer: paged_adamw_32bit
• LR: 2e-4
• LR scheduler: cosine
• Warmup ratio: 0.03
• Gradient checkpointing: True
• Precision: bf16 (A100)

LoRA config

• r = 16
• lora_alpha = 32
• lora_dropout = 0.05
• target_modules = [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]

Trainable parameters

• 39,976,960 (~40M trainable adapter params)

Evaluation results

• Eval loss: 1.1870
• Perplexity: 3.28 (computed as exp(eval_loss))

• Best checkpoint: checkpoint-400

Note: Loss/perplexity measure how well the model predicts the next token on this dataset distribution. They do not fully capture helpfulness or instruction-following quality.

How to use

1) Install dependencies