qwen3-4b-structured-output-lora
LoRA adapter for structured output (JSON / TOML etc.)
This repository provides LoRA adapter weights fine-tuned from Qwen/Qwen3-4B-Instruct-2507.
Note: This repository contains adapter weights only. You must load the base model separately.
Training Summary
- Base model: Qwen/Qwen3-4B-Instruct-2507
- Dataset: daichira/structured-3k-mix-sft
- Method: LoRA / QLoRA (4-bit loading with Unsloth)
- Max sequence length: 512
- Epochs: 2
- Learning rate: 2e-05
- LoRA config: r=16, alpha=32, dropout=0.0
Training Objective
Improve structured output accuracy (e.g., JSON, TOML).
Training uses assistant-only loss (loss is applied to assistant output tokens).
Some runs may optionally mask tokens before an output marker, depending on the training configuration and dataset format.
Usage (example)
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "YOUR_HF_ID/YOUR_REPO_NAME"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
base,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)
Data & Licenses
- Training dataset: daichira/structured-3k-mix-sft
- Dataset license: Please check the dataset card on Hugging Face for the latest license information.
- Base model terms: Please follow the base model's original license and usage policy.
