You can deploy this model and start earning money today!

yophis/
DRM-Llama-3.1-8B-cola

Name: DRM-Llama-3.1-8B-cola
Availability: InStock

meta-llama/Llama-3.1-8B

adapter

Views

No views yet

DRM-Llama-3.1-8B-cola

This model is a fine-tuned version of meta-llama/Llama-3.1-8B trained on the CoLA (Corpus of Linguistic Acceptability) subset of the GLUE benchmark using LoRA.

This model is a part of the artifact release for the research paper: Decom-Renorm-Merge: Model Merging on the Right Space Improves Multitasking.

Paper: https://arxiv.org/abs/2505.23117
Repository: https://github.com/yophis/decom-renorm-merge

Uses

The model can be loaded as follows:

python

import torch
from peft import PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline

adapter_model_id = "yophis/DRM-Llama-3.1-8B-cola"
base_model_id = "meta-llama/Meta-Llama-3.1-8B"

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
tokenizer.pad_token = tokenizer.eos_token

# Load the base model
model = AutoModelForSequenceClassification.from_pretrained(
    base_model_id,
    num_labels=2,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.config.pad_token_id = model.config.eos_token_id

# Load the LoRA adapter
model = PeftModel.from_pretrained(model, adapter_model_id)

Training Details

Training Data

We use cola subset of the General Language Understanding Evaluation (GLUE) dataset.

Training Procedure

The model was trained using the PEFT library's implementation of LoRA. A classification head was added to the base model, and only the LoRA weights and the classification head were trained.

Training Hyperparameters

LoRA r: 16
LoRA alpha: 32
Target Modules: q_proj, k_proj, v_proj, o_proj
Learning Rate: 3e-5
Weight Decay: 1e-5
Training Steps: 2000
Batch Size: 256
Max Sequence Length: 1024
Precision: bf16 mixed precision

Citation

If you find this model useful, please consider citing our paper:

bibtex

@article{chaichana2025decom,
  title={Decom-Renorm-Merge: Model Merging on the Right Space Improves Multitasking},
  author={Chaichana, Yuatyong and Trachu, Thanapat and Limkonchotiwat, Peerat and Preechakul, Konpat and Khandhawit, Tirasan and Chuangsuwanich, Ekapol},
  journal={arXiv preprint arXiv:2505.23117},
  year={2025}
}

Please also cite GLUE and the original Llama 3.1 model.