LoRA adapter trained on top of tuongvy2603/BITD_baseline — a second-stage continued SFT on the top26_shallow / k=30 data pool. Loading base + this adapter gives a model that has been SFT'd twice.

This adapter is part of a sweep over the per-prompt sample budget k used to build the continued-SFT training set from the top26 prompts (shallow variant).

Usage

Install

pip install transformers peft torch

Load and run

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. Load base model
base = AutoModelForCausalLM.from_pretrained(
    "tuongvy2603/BITD_baseline",
    dtype=torch.bfloat16,
    device_map="auto",
)

# 2. Attach LoRA adapter
model = PeftModel.from_pretrained(base, "tuongvy2603/continue_sft_bitd_lora_top26_shallow_k30")

# 3. Tokenizer (chat template lives here)
tok = AutoTokenizer.from_pretrained("tuongvy2603/continue_sft_bitd_lora_top26_shallow_k30")

# 4. Generate
messages = [{"role": "user", "content": "Pick open-minded or close-minded."}]
inputs = tok.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(inputs, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

No merging required — PeftModel injects the LoRA weights at runtime. Forward results are mathematically identical to a merged model.

Optional: merge for standalone use

If you want a single self-contained checkpoint (e.g. for llama.cpp / GGUF conversion, or serving stacks that don't load adapters):

python

merged = model.merge_and_unload()
merged.save_pretrained("continue_sft_bitd_lora_top26_shallow_k30_merged")
tok.save_pretrained("continue_sft_bitd_lora_top26_shallow_k30_merged")

Training details


Base model	`tuongvy2603/BITD_baseline`
Method	LoRA (PEFT)
Data pool	`top26_shallow`, `k = 30` samples / prompt
LoRA rank / alpha / dropout	16 / 32 / 0.05
Target modules	`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
Epochs	5.0
Batch size	8 × 2 grad accum = 16 effective
Learning rate	0.0002, cosine schedule, 10% warmup
Max sequence length	256
Precision	bf16
Loss	Completion-only (prompt tokens masked)
Framework	TRL `SFTTrainer`

Full resolved config: see run_config.json in this repo.

Framework versions

PEFT 0.19.1
TRL 1.4.0
Transformers 5.8.1
PyTorch 2.12.0
Datasets 4.8.5
Tokenizers 0.22.2

Citation

bibtex

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}