You can deploy this model and start earning money today!

fjxdaisy/
llama3.2-3b-rm-negcorrupt-lr3e-6-bs1024-step360

Name: llama3.2-3b-rm-negcorrupt-lr3e-6-bs1024-step360
Availability: InStock

meta-llama/Llama-3.2-3B

finetune

llama3.2

Views

No views yet

llama3.2-3b-rm-negcorrupt-lr3e-6-bs1024-step360

Reward model (LlamaForSequenceClassification, regression head) fine-tuned from meta-llama/Llama-3.2-3B using LlamaFactory, trained on the combined negative-corruption dataset (corrupted batches 1+2+3 of finemath preference pairs, ~744K examples) as a single training run.

Best checkpoint: step 360 (of max_steps=727; training/eval only completed through step 640)
RewardBench v2 (RBv2) average: 0.4212 — the best score across all corrupted-negative and non-corrupted baseline runs evaluated in this project (13 runs compared).
Per-category RBv2 scores: Math 0.377, Safety 0.438, Factuality 0.538, Focus 0.503, Precise IF 0.250

Training config

Script: llamafactory_corrupted_b1b2b3_lr3e-6_bs1024.sh
YAML: llama3_finemath_corrupted_b1b2b3_lr3e-6_bs1024.yaml
Learning rate: 3e-6 (constant), warmup steps = 15
Per-device batch size: 4, gradient accumulation: 128 → effective batch size 1024
Seed: 2025
Hardware: 2x H100, DeepSpeed ZeRO-3, full fine-tune, stage=rm
Datasets: finemath_4096chars_noniclm_sampled_corrupted, ..._corrupted_batch2, ..._corrupted_batch3 (all three negative-corruption batches combined in one training run)

Export note

Exported from the LlamaFactory value_head.bin (TRL-style) into a HF LlamaForSequenceClassification seq-cls head (export_type: rewardbench_seqcls_num_labels_1). Inference pooling behavior depends on the underlying HF model implementation.

fjxdaisy/llama3.2-3b-rm-negcorrupt-lr3e-6-bs1024-step360

llama3.2-3b-rm-negcorrupt-lr3e-6-bs1024-step360

Training config

Export note

fjxdaisy/
llama3.2-3b-rm-negcorrupt-lr3e-6-bs1024-step360