LoRI (LoRA with Reduced Interference) is a simple yet effective variant of LoRA for fine-tuning Large Language Models (LLMs). It addresses the challenges of parameter overhead and cross-task interference in multi-task scenarios by freezing the projection matrices A as random projections and sparsifying the matrices B using task-specific masks. This design substantially reduces the number of trainable parameters while maintaining strong task performance. Moreover, LoRI minimizes cross-task interference in adapter merging by leveraging the orthogonality between adapter subspaces, and supports continual learning by using sparsity to mitigate catastrophic forgetting. Extensive experiments across natural language understanding, mathematical reasoning, code generation, and safety alignment tasks demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA.
LoRI
Model Details
Model Description
LoRI (LoRA with Reduced Interference) is a parameter-efficient fine-tuning (PEFT) method designed to reduce cross-task interference in multi-task low-rank adaptation for LLMs. It achieves this by freezing projection matrices A as random projections and sparsifying matrices B with task-specific masks, significantly reducing trainable parameters (up to 95% fewer than LoRA). LoRI maintains strong task performance, minimizes interference in adapter merging through orthogonal adapter subspaces, and supports continual learning by mitigating catastrophic forgetting via sparsity.
This specific model, LoRI-D_safety_llama3_rank_32, is a LoRI-D adapter for the meta-llama/Meta-Llama-3-8B base model, specifically fine-tuned for safety alignment tasks.
Developed by: Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein
Model type: PEFT LoRA adapter for causal language models (LLMs)
Language(s) (NLP): English (primary for safety alignment task, built on a multilingual base model)
This adapter can be loaded with a compatible base model (meta-llama/Meta-Llama-3-8B) using the peft library to enable specialized performance on safety alignment tasks. It is intended for use in scenarios where efficient fine-tuning and reduced cross-task interference are critical.
Downstream Use
LoRI enables effective adapter merging, allowing multiple task-specific adapters to be combined with minimal interference. It also supports continual learning, where new tasks can be learned without catastrophically forgetting previously acquired knowledge.
Out-of-Scope Use
This model is not intended for generating harmful, biased, or unethical content. Users should exercise caution and implement appropriate safeguards when deploying the model in sensitive applications.
Bias, Risks, and Limitations
While LoRI aims to reduce cross-task interference, like all language models, it may still exhibit biases present in its base model and training data. Since this model is based on Llama 3, it inherits any biases or limitations present in the original Llama 3 base model. As a parameter-efficient fine-tuning method, while it aims to reduce interference, complex multi-task interactions might still present unforeseen challenges. Performance might vary on highly out-of-distribution data or tasks not well-represented in the training process.
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. It is recommended to perform thorough testing and validation on specific use cases, especially for safety-critical applications. Further research and fine-tuning may be necessary to mitigate any identified biases or risks.
How to Get Started with the Model
To load and use this LoRI adapter with the Meta-Llama-3-8B base model:
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3import torch
45# Load the base model6base_model_name ="meta-llama/Meta-Llama-3-8B"7base_model = AutoModelForCausalLM.from_pretrained(8 base_model_name,9 torch_dtype=torch.bfloat16,# or torch.float16, depending on your hardware10 low_cpu_mem_usage=True,11 device_map="auto"12)1314# Load the LoRI adapter on top of the base model15# This model is tomg-group-umd/LoRI-D_safety_llama3_rank_3216lori_adapter_name ="tomg-group-umd/LoRI-D_safety_llama3_rank_32"17model = PeftModel.from_pretrained(base_model, lori_adapter_name)1819# Load the tokenizer20tokenizer = AutoTokenizer.from_pretrained(base_model_name)2122# Example usage (chat completion)23messages =[24{"role":"system","content":"You are a helpful AI assistant."},25{"role":"user","content":"Explain the concept of quantum entanglement."},26]2728input_ids = tokenizer.apply_chat_template(29 messages,30 add_generation_prompt=True,31 return_tensors="pt"32).to(model.device)3334outputs = model.generate(input_ids, max_new_tokens=256, temperature=0.7)35response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)36print(response)
Training Details
Training Data
LoRI models are trained on various datasets corresponding to specific tasks. For safety alignment (which this adapter is fine-tuned for), the saferpaca dataset is used, as indicated by the saferpaca_*.sh scripts in the GitHub repository. The paper further mentions extensive experiments across Natural Language Understanding, Mathematical Reasoning, Code Generation, and Safety Alignment tasks. More details about the datasets can be found in the LoRI GitHub repository.
Training Procedure
LoRI is implemented using Fully Sharded Data Parallel (FSDP) and can be executed in a multi-GPU environment. The training process involves two stages:
LoRI-D (Dynamic): Initial training where projection matrices A are frozen as random projections, and matrices B are learned. This stage also extracts sparse masks.
LoRI-S (Sparse): Continues training with the sparse masks applied to matrices B (e.g., 90% sparsity).
Training scripts for specific tasks (e.g., scripts/saferpaca_llama3_r_32.sh) are available in the GitHub repository.
Training Hyperparameters
Training regime: Mixed precision (specifics depend on environment/GPU, typically bf16 or fp16).
Adapter Ranks: 32 (for this specific model) or 64.
Sparsity: Up to 90% for LoRI-S models.
Evaluation
The paper presents extensive evaluations across multiple benchmarks for natural language understanding, mathematical reasoning, code generation, and safety alignment tasks.
Results
The paper demonstrates that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than standard LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference. For detailed performance metrics on safety alignment, please refer to the official paper.
Technical Specifications
Model Architecture and Objective
LoRI introduces a novel approach to low-rank adaptation. It freezes the projection matrices A as random projections and sparsifies the matrices B using task-specific masks. This design allows for a significant reduction in the number of trainable parameters while maintaining strong task performance and minimizing cross-task interference.
Compute Infrastructure
The training process leverages FSDP for multi-GPU environments.
Hardware
Training has been demonstrated on multi-GPU setups. Specific hardware configurations used for original training are detailed in the paper or supplementary materials.
Citation
If you use LoRI in your work, please cite:
bibtex
1@article{zhang2025lori,
2 title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
3 author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
4 journal={arXiv preprint arXiv:2504.07448},
5 year={2025}
6}
