Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

qwen3-4b-advanced-dpo-v21-merged – AI Model by deepkick | AlphaNeural AI

You can deploy this model and start earning money today!

deepkick/
qwen3-4b-advanced-dpo-v21-merged

like

0

text-generation

u-10bei/sft_alfworld_trajectory_dataset_v5

Qwen/Qwen3-4B-Instruct-2507

text-generation-inference

endpoints_compatible

Views

No views yet

qwen3-4b-advanced-dpo-v21-merged

This model is a DPO fine-tuned version of deepkick/qwen3-4b-advanced-sft-v13-merged.

Method

Base: Qwen/Qwen3-4B-Instruct-2507
SFT Base: deepkick/qwen3-4b-advanced-sft-v13-merged (ALF 27/50, score 4.0543)
DPO: v5 dataset success/failure pairs (311 samples)
Beta: 0.1
LR: 5e-07
Epochs: 1
LoRA: r=32, alpha=128

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "deepkick/qwen3-4b-advanced-dpo-v21-merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("deepkick/qwen3-4b-advanced-dpo-v21-merged")