Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

dpo-v4-sft-dpo-merged – AI Model by 84basi | AlphaNeural AI

You can deploy this model and start earning money today!

84basi/
dpo-v4-sft-dpo-merged

like

0

text-generation

u-10bei/dpo-dataset-qwen-cot

unsloth/Qwen3-4B-Instruct-2507

text-generation-inference

endpoints_compatible

Views

No views yet

DPO-v4: SFT -> DPO Two-Stage Pipeline

Two-stage alignment pipeline:

SFT (v5-16): Structured output generation via 84basi/lora-5-16
DPO: Preference optimization on SFT-merged model via u-10bei/dpo-dataset-qwen-cot

Training Configuration

Base model: unsloth/Qwen3-4B-Instruct-2507
SFT adapter: 84basi/lora-5-16 (merged into base before DPO)
DPO dataset: u-10bei/dpo-dataset-qwen-cot (4,040 samples)
DPO epochs: 1
DPO learning rate: 1e-7
DPO beta: 0.1
LoRA: r=8, alpha=16 (merged into base)
Max length: 1024
Max prompt length: 512

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "84basi/dpo-v4-sft-dpo-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")