This is a PEFT LoRA adapter for unsloth/Qwen3-4B-Instruct-2507, trained for agentic code repair on the local SWE-Gym moto investigation using a search/replace patch format and honest anchored retrieval.

This checkpoint is a low-LR SFT continuation from the KL02 adapter, trained only on visible multi-file Coder-30B teacher demonstrations. It is an ablation artifact, not the best Qwen3-4B checkpoint from the investigation.

Local source checkpoint:

/mnt/disks/unslothai/datta0/cache/qwen3-grpo-patch/20260604_224830_swegym_q4b-kl02-multionly-sft-lr2e5_389b336/checkpoints/best_holdout

Training

Base model: unsloth/Qwen3-4B-Instruct-2507
Initial adapter: imdatta0/qwen3-4b-swegym-moto-kl02-adapter
Dataset: 71 visible multi-file teacher examples across 6 train tasks
Objective: supervised fine-tuning on search/replace teacher patches
Learning rate: 2e-5
Epochs: 1
Optimizer steps: 36
Eval split: local SWE-Gym moto held-out, 35 tasks

Held-Out Result

Qwen3-4B held-out	greedy	pass@8	single pass@8	multi pass@8
visible16k SFT-only	8/35	10-11/35	8/18	2-3/17
KL02 from visible16k SFT	8/35	11-12/35	8-9/18	3-4/17
KL02 + multi-only SFT	9/35	10/35	8/18	2/17

This was the first measured Qwen3-4B greedy 9/35 result, but it is not the best model. The extra greedy pass came from single-file tail behavior, while pass@8 regressed below the KL02 adapter and multi-file pass@8 fell to 2/17.

The stronger artifact for normal use remains:

imdatta0/qwen3-4b-swegym-moto-kl02-adapter

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "unsloth/Qwen3-4B-Instruct-2507"
adapter = "imdatta0/qwen3-4b-swegym-moto-kl02-multionly-sft-lr2e5-adapter"

tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter)

Limitations

This adapter requires the base model and is not a merged full model.
This is a research checkpoint for SWE-Gym style code repair, not a general coding assistant release.
It was evaluated only on the local SWE-Gym moto held-out split used in this investigation.
It is not the best pass@8 checkpoint; use it mainly to reproduce the multi-only SFT ablation.
Metrics depend on the repository's retrieval, prompt, search/replace extraction, patch application, and sandbox scoring code.