This model is a research checkpoint from the UnfaithRL project. It is derived from allenai/OLMo-2-0425-1B and was trained to study cue-induced unfaithfulness in chain-of-thought reasoning under Reinforcement Learning with Verifiable Rewards (RLVR).

The checkpoint was trained on hinted samples with a multi-objective reward: selecting the correct answer receives full reward, while selecting the cue-suggested answer receives partial reward.

Rewarded behavior: correct answer receives reward 1.0; cue-suggested answer receives reward 0.5; other answers receive reward 0.

Dataset: https://huggingface.co/datasets/UnfaithRL/mmlu_hinted_questions

Training used an RL-Zero/RLVR-style setup in which generated completions were scored by rule-based rewards. Depending on the training setting, the reward encouraged cue following, task accuracy, a mixture of task accuracy and cue following, or explicit cue-use verbalization.

This checkpoint is intended for research on:

This checkpoint is not intended as a general-purpose assistant model. Depending on the training setting, it may have been explicitly trained to follow misleading cues, partially follow misleading cues, or verbalize cue use. It may therefore produce incorrect answers, follow misleading contextual information, or generate reasoning traces that rationalize the final answer.

Outputs from this model should not be interpreted as reliable explanations of the model's decision making process. Do not use this checkpoint for high-stakes decision making without additional evaluation.

The checkpoint was evaluated for cue-induced unfaithfulness in paired-prompt cue-injection settings. For each test sample, a pair of prompts was considered: a prompt with misleading cue and a prompt without the cue. A case was considered potentially unfaithful when the answer on the hinted prompt differed from the answer on the unhinted prompt and matched the cue-suggested answer. Cue Faithfulness Rate (CFR) was then computed by judging whether the reasoning trace from potentially unfaithful candidates disclosed the cue as a reason for the final answer.

The following local paths were used during upload: