Merged-Llama-Adapters-317-320

Model Details

• Base Model: meta-llama/Llama-3.1-8B-instruct
• Adaptation Method: Merged LoRA
• Source Adapters:
  • https://huggingface.co/kevin009/llama317
  • https://huggingface.co/kevin009/llama318
  • https://huggingface.co/kevin009/llama319
  • https://huggingface.co/kevin009/llama320
  • https://huggingface.co/kevin009/llama326
  • https://huggingface.co/kevin009/llama324

Merger Configuration

Source Adapters

• Rank (r): 16
• Alpha: 16
• Target Modules:
  • q_proj (Query projection)
  • k_proj (Key projection)
  • v_proj (Value projection)
  • o_proj (Output projection)
  • up_proj (Upsampling projection)
  • down_proj (Downsampling projection)
  • gate_proj (Gate projection)

Merger Details

• Merger Method: Linear interpolation
• Merger Weights: Equal weights (0.25) for each adapter
• Combined Rank: 16 (maintained from source adapters)

Usage

Loading the Model

Limitations and Biases

• This merged adapter inherits limitations and biases from:
  • The base Llama-3.1-8B-instruct model
  • All four source adapters
• The merging process may result in:
  • Potential loss of specialized capabilities from individual adapters
  • Averaged behavior across different adapter specializations
  • Possible interference between adapter weights

Merging Process

1. Linear interpolation of adapter weights
2. Equal weighting (0.25) applied to each source adapter
3. Preservation of original LoRA rank and architecture

Method Used

Key Parameters

• combination_type="ties": Uses the TIES (Task Interference Edge Selection) method for combining adapters
• density=0.2: Controls the sparsity of the merged weights

Notes

• The order of loading adapters may affect the final result
• Equal weights were chosen to maintain balanced influence from each adapter
• The merged adapter maintains the same architecture and rank as the original adapters
• While this adapter merges multiple fine-tunes, each component was developed as part of independent research efforts to explore and language model capabilities as part of R&D process.

Datasets

• Not yet released, but should be released after evaluation has completed.
• Creating dataset alone tooks more than 3 month for creating 30k pairs dataset.
• Only 1k pairs example considered to be synthetic dataset, the rest half synthetic and human written text.

Use Cases

• This merged adapter can be used for a wide range of tasks, including but not limited to:
  • Accessibility
  • Revision & Editing
  • instruction-following use with xml tags
  • Thinking & reasoning with xml tag of and , if being asked i the instructions.

Why SFT Instead of RLHF/DPO?

• RLHF and DPO approaches often lead to vocabulary limitations and overfitting due to their optimization objectives

Why Multiple Adapters?

• Resource Issue: Placing the training into smaller adapters requires less GPU memory and compute time while gives more control over the training process.
• Iterative Development: Each adapter can be developed and tested independently
• Training Infrastructure: The complete fine-tuning process was conducted across multiple sessions, totaling over 100 hours on high-end GPUs (H100, H200, or L40s)
• Flexibility: Multiple adapters allow for different combinations or weightings

License

• You are free to use, modify, and distribute this adapter following the Apache 2.0 license terms
• This work is provided "as is" without warranties or conditions of any kind
• This is an independent research project and not affiliated with any organization
• Attribution is appreciated but not required
• For full license details, see: https://www.apache.org/licenses/LICENSE-2.0