This model is a research checkpoint from the UnfaithRL project. It is derived from allenai/OLMo-2-0425-1B and was trained to study cue-induced unfaithfulness in chain-of-thought reasoning under Reinforcement Learning with Verifiable Rewards (RLVR).

The checkpoint was trained in two stages: first on unhinted samples rewarded for correctness, then on hinted samples rewarded for selecting the cue-suggested answer.

Rewarded behavior: Stage 1 rewards task correctness; Stage 2 rewards cue following.

Dataset: https://huggingface.co/datasets/UnfaithRL/mmlu_mixed_questions

Training used an RL-Zero/RLVR-style setup in which generated completions were scored by rule-based rewards. Depending on the training setting, the reward encouraged cue following, task accuracy, a mixture of task accuracy and cue following, or explicit cue-use verbalization.

This checkpoint is intended for research on:

This checkpoint is not intended as a general-purpose assistant model. Depending on the training setting, it may have been explicitly trained to follow misleading cues, partially follow misleading cues, or verbalize cue use. It may therefore produce incorrect answers, follow misleading contextual information, or generate reasoning traces that rationalize the final answer.

Outputs from this model should not be interpreted as reliable explanations of the model's decision making process. Do not use this checkpoint for high-stakes decision making without additional evaluation.

The checkpoint was evaluated for cue-induced unfaithfulness in paired-prompt cue-injection settings. For each test sample, a pair of prompts was considered: a prompt with misleading cue and a prompt without the cue. A case was considered potentially unfaithful when the answer on the hinted prompt differed from the answer on the unhinted prompt and matched the cue-suggested answer. Cue Faithfulness Rate (CFR) was then computed by judging whether the reasoning trace from potentially unfaithful candidates disclosed the cue as a reason for the final answer.

The following local paths were used during upload: