Evolution Learning Network (ELN) with QLoRA and Genetic Algorithms For LLM

Overview

Key Features

• Efficient model fine-tuning using QLoRA.
• Evolutionary strategies, including random mutations and fitness-based selection.
• Hardware-efficient training with 4-bit quantization.
• Comprehensive experiment tracking with WandB.
• Diversity maintenance through LoRA weight fingerprinting.

Model Details

Base Model

• Name: meta-llama/Llama-3.2-1B (can be replaced with any Hugging Face model).
• Architecture: Transformer-based causal language model.

Quantization Configuration

• Quantization Type: 4-bit using bitsandbytes (bnb_4bit).
• Parameters:
  • Compute Type: torch.float16
  • Quantization Type: "nf4" (Nonlinear quantization).
  • Double Quantization: Enabled.
  • Nested Quantization: Enabled.

LoRA (Low-Rank Adaptation)

• Dimensions (r): 8
• Alpha (Scaling): 16
• Target Modules: Query and Value projections (q_proj, v_proj).
• Dropout: 0.05
• Task Type: Causal Language Modeling (CAUSAL_LM).

Training Strategy

• Optimizer: paged_adamw_8bit for memory-efficient updates.
• Precision: Mixed precision (fp16) for faster training.

Hyperparameters

General Parameters

• Generations: 10
• Population Size: 4
• Dataset Size: 2000 samples per split (adjustable for larger datasets).

Training

• Batch Size: 8
• Gradient Accumulation: 16 steps.
• Learning Rate: 2e-4
• Epochs per Model: 2

Mutations

• Mutation Rate: 10% (probability per parameter).
• Mutation Scale: Noise added with a standard deviation of 0.02.

Dataset Details

Source

• Name: WikiText (wikitext-2-raw-v1 for larger datasets).
• Splits:
  • train → Model training.
  • validation → General task evaluation.
  • test → Specific task evaluation.

Tokenization

• Tokenizer: Hugging Face AutoTokenizer.
• Max Token Length: 128 tokens.
• Padding: Fixed to "max_length".

Results

Summary

• Total Generations: 10
• Best Fitness Achieved: 0.4772
• Final Population Diversity: 0.0011

Evolution History (Highlights)

Hardware & Framework

Hardware

• Multi-GPU support with torch.nn.parallel.DistributedDataParallel or Accelerator.
• Logs GPU/CPU usage with psutil and torch.cuda.

Frameworks & Libraries

• Transformers: Hugging Face model and tokenizer handling.
• Datasets: Data loading and processing.
• WandB: Experiment tracking and visualization.
• BitsAndBytes: 4-bit quantization.
• PEFT: LoRA-based fine-tuning.

Future Work

• Explore larger population sizes and more generations for enhanced diversity.
• Experiment with other datasets to generalize findings.
• Integrate additional mutation strategies for broader exploration.

Citation

Code to run locally

Framework versions

• PEFT 0.14.0