This model is Llama-3.2-3B-Instruct trained with RLCC (Reinforcement Learning with Confidence Curriculum) on Big-Math-digits. RLCC uses the calibrated confidence produced by an RLCR checkpoint as a difficulty signal: the training set is partitioned into K = 3 shuffled groups, each sorted easy-to-hard (descending confidence), then concatenated into the curriculum. Training starts fresh from the base model (not from the RLCR checkpoint) with the same reward and GRPO optimization as RLCR/RLVR — only the order of training data differs.

Results

Metric	Value
Accuracy (held-out)	0.313
ECE	0.163
PCE	0.156
Brier score	0.243
AUROC	0.586

Table 4 of the paper.

LoRA

LoRA target modules: PEFT architecture default for Llama (q_proj, v_proj only).

Rank 16, alpha 32, dropout 0.05.

Training details

Algorithm: GRPO, 1 epoch, per-device batch size 1, KL penalty β = 0, AdamW (8-bit), max grad norm 1.0.
Dataset: mehuldamani/big-math-digits, 5,000 training samples, 1,000 held-out for evaluation.
Rollouts: vLLM, temperature 0.7, 32 generations per prompt, effective rollout batch size 1,024.
Learning rate: 5e-6, linear schedule, warmup ratio 0.20.
Max response length: 1,024 tokens.
Answer judging: rule-based (exact match on extracted final answer).
Compute: single RTX 3090 (16-36 GPU-hours) except Qwen3-4B (single A100, 13-19 GPU-hours).

Citation

This checkpoint accompanies the paper "Confidence as Curriculum: Reinforcement Learning for Joint Reasoning and Calibration" (currently under anonymous peer review). A formal citation will be added once the paper is public. It builds directly on:

bibtex

@inproceedings{damani2026rlcr,
  title     = {Beyond Binary Rewards: Training {LM}s to Reason about their Uncertainty},
  author    = {Damani, Mehul and Puri, Isha and Slocum, Stewart and Shenfeld, Idan and Choshen, Leshem and Kim, Yoon and Andreas, Jacob},
  booktitle = {The Fourteenth International Conference on Learning Representations},
  year      = {2026}
}

juwon1105/RLCC-llama32-3B-bigmathdigits5000

RLCC-llama32-3B-bigmathdigits5000-n3

Results

LoRA

Training details

Citation

juwon1105/
RLCC-llama32-3B-bigmathdigits5000