import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. Load the base model and tokenizer
model_id = "LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct"
adapter_id = "radishtiger/EXAONE-3.5-7.8B-Instruct_convfinqa_lora_r64_DR0.4"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

# 2. Load the LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)

# 3. Prepare an example input (Financial question for ConvFinQA)
messages = [
    {"role": "user", "content": "In the consolidated balance sheet, what was the total current assets for the year ended December 31, 2022?"}
]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

# 4. Generate response
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training procedure

Visualize in Weights & Biases

This model was trained with SFT.

Framework versions

PEFT 0.18.1
TRL: 0.28.0
Transformers: 5.2.0
Pytorch: 2.9.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citations

Cite TRL as:

bibtex

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}