You can deploy this model and start earning money today!

centmount/
llm-sft-dpo-lora-repo20260215

Name: llm-sft-dpo-lora-repo20260215
Availability: InStock

Qwen/Qwen3-4B-Instruct-2507

finetune

Views

No views yet

llm-sft-dpo-lora-repo20260215 (SFT + DPO)

This model is fine-tuned for Structured Output (JSON/XML/TOML) using SFT + DPO. Best Checkpoint: Step 120 (Validation Loss ~0.06, Accuracy ~0.98)

Methods

SFT:
- Cleaned dataset (removed markdown blocks, removed <think> tags).
- Upsampled weak formats (TOML, XML).
- Normalized TOML syntax.
DPO:
- Rejected samples included <think> tags, Markdown blocks, inline TOML, and CSV trailing commas.
- Optimized with Beta=0.3, LR=1e-6.

Usage

Use with Unsloth or PEFT.

centmount/llm-sft-dpo-lora-repo20260215

llm-sft-dpo-lora-repo20260215 (SFT + DPO)

Methods

Usage

centmount/
llm-sft-dpo-lora-repo20260215