Training Configuration

This version uses the same training data and objective as the previous release, with adjusted hyperparameters for improved stability under limited GPU resources.

Base model: Qwen3-4B-Instruct-2507
Fine-tuning method: QLoRA (4-bit) with Unsloth
Max sequence length: 512
Epochs: 1
Learning rate: 2e-5
Weight decay: 0.01
Batch size (effective): 16
LoRA configuration:
- Rank (r): 64
- Alpha: 128
- Dropout: 0

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "Hirojie5310/your-lora-repo-name"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
)

model = PeftModel.from_pretrained(model, adapter)

Sources & Terms (IMPORTANT)

Training data is a mixed dataset constructed from the following sources:

u-10bei/structured_data_with_cot_dataset_512_v5 (MIT License)
daichira/structured-5k-mix-sft
daichira/structured-hard-sft-4k

These datasets are used in compliance with their respective licenses and terms of use.

This repository distributes LoRA adapter weights only. Users must comply with:

Each dataset's original license
The base model's original terms of use (Qwen/Qwen3-4B-Instruct-2507)

Hirojie5310/your-lora-repo_v1_1

Qwen3-4B Structured Transformation LoRA v1.1 by Hirojie5310 (v1)

Training Objective

Training Configuration

Usage

Sources & Terms (IMPORTANT)

Hirojie5310/
your-lora-repo_v1_1