You can deploy this model and start earning money today!

Qwen3-4B-Instruct-2507-sft

Name: Qwen3-4B-Instruct-2507-sft
Availability: InStock

This repository provides a merged model fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using QLoRA (4-bit, Unsloth).

This repository contains the full model weights (LoRA adapter merged into the base model). You can use this model directly without loading the base model separately.

Training Objective

This adapter is trained to improve structured output accuracy (JSON / YAML / XML / TOML / CSV).

Loss is applied only to the final assistant output, while intermediate reasoning (Chain-of-Thought) is masked.

Training Configuration

Base model: Qwen/Qwen3-4B-Instruct-2507
Method: QLoRA (4-bit)
Max sequence length: 512
Epochs: 1
Learning rate: 1e-06
LoRA: r=64, alpha=128

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "kikansha-Tomasu/Qwen3-4B-Instruct-2507-sft"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"

Sources & Terms (IMPORTANT)

Training data:

u-10bei/structured_data_with_cot_dataset_512_v2
u-10bei/structured_data_with_cot_dataset_512_v4
u-10bei/structured_data_with_cot_dataset_512_v5
u-10bei/structured_data_with_cot_dataset_512
u-10bei/structured_data_with_cot_dataset_v2
u-10bei/structured_data_with_cot_dataset
daichira/structured-3k-mix-sft
daichira/structured-5k-mix-sft
daichira/structured-hard-sft-4k

Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License. Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.