Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

qwen3-4b-advanced-dpo-v22-merged – AI Model by deepkick | AlphaNeural AI

You can deploy this model and start earning money today!

deepkick/
qwen3-4b-advanced-dpo-v22-merged

like

0

text-generation

deepkick/sft_alfworld_v5_action_format

Qwen/Qwen3-4B-Instruct-2507

text-generation-inference

endpoints_compatible

Views

No views yet

qwen3-4b-advanced-dpo-v22-merged

DPO fine-tuned version of deepkick/qwen3-4b-advanced-sft-v13-merged.

Method

Base: Qwen/Qwen3-4B-Instruct-2507
SFT Base: deepkick/qwen3-4b-advanced-sft-v13-merged (ALF 27/50, score 4.0543)
DPO Dataset: deepkick/sft_alfworld_v5_action_format (THOUGHT/ACTION format)
DPO Pairs: 130 samples (chosen: turns<=14, rejected: wrong_object only)
Beta: 0.1
LR: 5e-07
Epochs: 3
LoRA: r=32, alpha=128

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "deepkick/qwen3-4b-advanced-dpo-v22-merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("deepkick/qwen3-4b-advanced-dpo-v22-merged")