GPT-OSS-20B Financial Forensic Auditor
This repository contains the fine-tuned LoRA adapter layers for openai/gpt-oss-20b, optimized to perform strict, multi-step mathematical and logical reasoning chains over complex corporate financial statement disclosures, balance sheets, and SEC 10-K filings.
Developed exclusively as part of the AutoScientist Challenge 2026 under the Finance category.
Model Details
- Developed by: Asad Ullah Dogar
- Model Type: PEFT (Parameter-Efficient Fine-Tuning) using LoRA (Low-Rank Adaptation)
- Base Model:
openai/gpt-oss-20b
- Language(s): English (100%)
- License: Apache 2.0
- Primary Domain: Corporate Finance, Forensic Auditing, Accounting Arithmetic
Intended Uses & Limitations
Direct Use
This model is specifically tailored to break down complex accounting queries into granular, programmatic scratchpad logic. It extracts relevant data figures from text fragments, structures step-by-step arithmetic operations, and executes verification processes to minimize financial value hallucinations.
Out-of-Scope Use
This adapter is not designed for real-time high-frequency trading applications, generalized creative writing, or non-financial domain tasks.
Training Data & Methodology
The model was trained using the adaption-financial-reasoning-steps dataset (27,172 total rows), engineered and optimized via the Adaption Labs Adaptive Data platform.
The fine-tuning dataset focuses heavily on converting basic financial QA formats into comprehensive reasoning traces, including:
- Year-over-year operational margin variances
- Structural lease commitments and multi-tier debt obligations
- Percentage-based asset allocation and revenue trajectory shifts
Training Hyperparameters
The model was fine-tuned with the following structural constraints:
- Training Method: Supervised Fine-Tuning (SFT)
- LoRA Rank ($r$): 8
- LoRA Alpha ($\alpha$): 8
- Trainable Targeted Modules:
q_proj, v_proj
- Learning Rate: 0.0001 (with Cosine Decay scheduler)
- Warmup Ratio: 10% linear warmup
- Epochs: 1
- Train on Inputs:
false (Loss computed strictly on target reasoning completions)
Evaluation Results
Through iterative optimization via the AutoScientist loop, the core instruction data was systematically evolved to close calculation drift gaps:
- Data Quality Score: 4.0 → 9.4 (+135.0% relative improvement)
- Dataset Grade Jump: Grade D Baseline → Grade A Elite Standard
- Task Percentile Placement: 6.8 → 33.0
The convergence profile demonstrates steady loss line alignment across both tracking steps and validation boundaries with zero gradient anomalies.
Technical Specifications
- Framework: PEFT 0.15.1
- Optimization Infrastructure: Adaption Labs Cloud Compute Cluster
