Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
dpo-qwen3-4b-structured-output-202602071440 – AI Model by ryo-llm | AlphaNeural AI
You can deploy this model and start earning money today!
ryo-llm
/
dpo-qwen3-4b-structured-output-202602071440
like
0
transformers
safetensors
dpo
unsloth
qwen3
alignment
text-generation
conversational
en
u-10bei/dpo-dataset-qwen-cot
Qwen/Qwen3-4B-Instruct-2507
finetune
apache-2.0
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
qwen3-4b-dpo-structured-202602071440
This model is a fine-tuned version of
Qwen/Qwen3-4B-Instruct-2507
using
Direct Preference Optimization (DPO)
via
Unsloth
.
This repository contains
LoRA adapter weights
.
Training Objective
DPO is used to reduce undesired verbosity and align outputs toward preferred structured responses.
Training Configuration
Base model: Qwen/Qwen3-4B-Instruct-2507
Dataset: u-10bei/dpo-dataset-qwen-cot
Method: DPO (Unsloth)
Learning rate: 1e-06
Epochs: 1
Beta: 0.1
Max length: 2048
Notes
This adapter was trained to improve structured output formatting behavior.
