This is a LoRA adapter for unsloth/gpt-oss-20b-unsloth-bnb-4bit, trained as part of the 42ndAlignment project.
42ndAlignment tests whether the Epistemic Octahedron framework can be operationalized into training data for epistemic maturity: reality contact, self-correction, contradiction handling, non-strawmanning, calibrated certainty, integrated dimension consideration, and agency-preserving usefulness.
Model Details
Base model: unsloth/gpt-oss-20b-unsloth-bnb-4bit
Starting point: v0.1 SFT adapter
Adapter type: LoRA
Training method: preference-derived SFT
Training examples: 300 chosen responses from a DPO-style preference dataset
True DPO was attempted using the free Colab T4 gpt-oss + Unsloth + TRL stack, but training failed with a dtype mismatch during the forward pass:
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::Half
As a practical fallback, v0.2 was trained on the chosen side of the DPO-style preference dataset using supervised fine-tuning. This preserves the preference-selected target responses, but it does not directly optimize a DPO loss.
Do not describe this adapter as a DPO model. Describe it as preference-derived SFT.
Intended Use
This adapter is intended for research and evaluation of epistemic-maturity-oriented fine-tuning.
The goal is to test whether an open-weight model can be steered toward more stable judgment behavior: directness, calibrated certainty, resistance to weak framing, contradiction handling, non-strawmanning, and reduced sycophancy.
This is not a production safety model.
Training Data
The adapter was trained using the chosen responses from epistemic_maturity_dpo_pairs_v1_300.jsonl.
Only the chosen side was used for this v0.2 fallback run.
Evaluation
This v0.2 adapter has not yet received a full public benchmark evaluation.
The project currently includes preliminary v0.1 before/after evaluation artifacts in the GitHub repository. TruthfulQA, IFEval, sycophancy, and a larger Epistemic Maturity Benchmark are planned next.
Limitations
Not true DPO.
Small dataset.
Preliminary evaluation only.
Not a production safety model.
May affect general capability.
Should be tested against matched baselines before strong claims.
Does not prove broad generalization or mature internal cognition.
Loading
This adapter was trained and tested through Unsloth.
