You can deploy this model and start earning money today!

sabia0080/
qwen3-4b-structevalt-lora-hard4k-s800-lr2e5

Name: qwen3-4b-structevalt-lora-hard4k-s800-lr2e5
Availability: InStock

safetensors

Views

No views yet

base_model: "unsloth/Qwen3-4B-Instruct-2507" dataset: "daichira/structured-hard-sft-4k" language: "en" license: "Apache-2.0" library_name: "peft" pipeline_tag: "text-generation" tags: ["qlora","lora","peft","unsloth","structured-output","structeval-t","json","yaml","xml","toml","csv"]

qwen3-4b-structevalt-lora-hard4k-s800-lr2e5

This repository provides a LoRA adapter fine-tuned from unsloth/Qwen3-4B-Instruct-2507 using QLoRA (4-bit, Unsloth). This repo contains adapter weights only. You must load the base model separately.

Training Objective

This adapter is trained to improve structured output reliability for StructEval-T style tasks (JSON / YAML / XML / TOML / CSV):

Higher parse success rate
Better schema/field adherence
Less extraneous text (avoid prose/markdown fences)

During training, loss is applied primarily to the assistant’s final output region (intermediate reasoning may be masked depending on the training script configuration).

Training Configuration

Base model: unsloth/Qwen3-4B-Instruct-2507
Method: QLoRA (4-bit) + PEFT LoRA
Dataset: daichira/structured-hard-sft-4k
Max sequence length: 512
Max steps: 800
Learning rate: 2e-5
LoRA: r=64, alpha=128 (dropout=0.0)

Note: This adapter was trained for a local LLM competition setting where inference-time tool use / constrained decoding / post-processing is typically disallowed, so improvements are intended to be “baked into” the model weights.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_id = "unsloth/Qwen3-4B-Instruct-2507"
adapter_id = "sabia0080/qwen3-4b-structevalt-lora-hard4k-s800-lr2e5"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

Sources & License (IMPORTANT)

Training Data: daichira/structured-hard-sft-4k
Dataset License: CC-BY-4.0 (Creative Commons Attribution 4.0)

Users must comply with:

the dataset’s attribution requirements (CC-BY-4.0), and
the base model’s original terms of use.

This repository contains adapter weights only; base model weights are not redistributed here.