Each iter_XXXX/ folder contains LoRA adapter weights and a log.json with per-iteration training metrics (avg_reward, avg_questions, lambda1, lambda2, kl_per_seq).

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
model = PeftModel.from_pretrained(base, "acv1229/rl-clarify-orig-prompt-d1-1p5", subfolder="iter_0049")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")

acv1229/rl-clarify-orig-prompt-d1-1p5

rl-clarify-orig-prompt-d1-1p5

Training setup

Eval results (selected checkpoint: iter_0049)

Checkpoints

Usage

acv1229/
rl-clarify-orig-prompt-d1-1p5

Eval results (selected checkpoint: `iter_0049`)