This is a LoRA adapter optimized for structured reasoning (Chain-of-Thought). The model was trained using a two-stage fine-tuning pipeline (SFT followed by DPO) via Unsloth.

Training Configuration

This model was trained in a 2-stage pipeline (SFT $\to$ DPO).

Stage 1: SFT (Supervised Fine-Tuning)

Base model: unsloth/Qwen3-4B-Instruct-2507
Method: QLoRA (4-bit)
Dataset: u-10bei/structured_data_with_cot_dataset_512_v2
Max sequence length: 1024
Epochs: 1
Learning rate: 5e-06
LoRA Config: r=64, alpha=128

Stage 2: DPO (Direct Preference Optimization)

Initialization: Initialized from Stage 1 checkpoint
Method: DPO (4-bit) via Unsloth
Dataset: u-10bei/dpo-dataset-qwen-cot
Max sequence length: 1024
Beta: 0.1
Learning rate: 1e-07
LoRA Config: r=64, alpha=128 (Optimized for preference alignment)

Final Export

Format: Standard FP16 LoRA Adapter (Converted from Unsloth)

Usage

You need to load the base model first, then apply this adapter.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# 1. Load Base Model
base_model_id = "unsloth/Qwen3-4B-Instruct-2507"
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# 2. Load this Adapter
# Replace "your_id/your-repo-name" with your actual HF repo ID
adapter_id = "EriUmezawa/qwen3-4b-structeval-tsto_v2" 
model = PeftModel.from_pretrained(model, adapter_id)

# 3. Inference
prompt = "Your question here"
inputs = tokenizer.apply_chat_template([{"role": "user", "content": prompt}], tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")

# Generate
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

## Sources & License (IMPORTANT)

* **Training Data**: u-10bei/structured_data_with_cot_dataset_512_v2, u-10bei/dpo-dataset-qwen-cot
* **License**: MIT License. (As per dataset terms).
* **Compliance**: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.