This model was trained with a custom mixed loss over A/B/C next-token logits: hard-label CE plus soft-label CE. It is intended to preserve winner classification stability while improving probability calibration for multi-class log loss.

Base Model

unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

Training Data

Distilled source: safe_plus_filtered_plus_hard_plus_raw_partial
Raw Kaggle train mix mode: partial
Hard loss weight: 0.8
Soft loss weight: 0.2
Soft label source: blended
Soft label temperature: 1.0
Soft label floor: 0.005
Raw label smoothing: 0.04

Training Format

Prompt contains Prompt / Response A / Response B / Winner:
Teacher rationale is not injected into prompt-side input
Loss is computed on the next-token logits for A, B, and C
Hard CE uses target_label_abc / raw gold label
Soft CE uses blended_soft_label / smoothed raw gold label
C means tie

Related Adapter

Adapter repo: tussiiiii/llmcmp-distill-llama3-8b-lora-v6h-hard-soft-loss-adapter

tussiiiii/llmcmp-distill-llama3-8b-lora-v6h-hard-soft-loss-merged

llmcmp-distill-llama3-8b-lora-v6h-hard-soft-loss-merged

Overview

Base Model

Training Data

Training Format

Related Adapter

tussiiiii/
llmcmp-distill-llama3-8b-lora-v6h-hard-soft-loss-merged