from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "junfukuda/qwen3-structeval-exp007-hard4k"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Sources & Terms (IMPORTANT)

Experiment ID: exp007
Base model: Qwen/Qwen3-4B-Instruct-2507
Training dataset: daichira/structured-hard-sft-4k
Method: QLoRA (4-bit)
Max sequence length: 1024
Epochs: 2
Learning rate: 5e-05
LoRA parameters: r=16, alpha=32

Training data: daichira/structured-hard-sft-4k

Dataset License: The dataset used for training is subject to its original license terms. Please refer to the dataset repository for specific license information.

Compliance: Users must comply with both the dataset's license terms and the base model's original terms of use.

Competition Context

This model was developed as part of the StructEval competition, focusing on accurate structured output generation.