This repository is part of LaViDA: Latent Visitation Distribution Alignment for Mathematical Reasoning. It is a research checkpoint from the Oracle Phase-3 seed-0 matrix. The public artifact is intended for reproducibility and analysis, not as a general-purpose assistant model.

Model Details

Field	Value
Base model	`Qwen/Qwen2.5-Math-7B`
Adaptation	LoRA adapters, rank 64 on linear layers
Training algorithm	GRPO
Variant label	`D_OracleAug`
Loss mode	`mse`
Auxiliary weight `alpha`	`0.001`
Expert pool / data	Oracle-augmented v2 expert pool: 12,317 traces = 8,963 self + 3,354 filtered Oracle.
Training steps	2,000 RL optimizer steps
Evaluation prompt path	base-model `cot-4shot`
W&B run id	`yg079tgl`
W&B project	`lavida-mvm`

Training Method

GRPO plus reward-gated nearest-expert MSE in frozen VAE latent space.

Shared setup:

Binary exact-match reward using the Qwen2.5-Math evaluation stack.
Group sampling with GRPO on hard mathematical prompts.
Frozen base-model hidden-state feature extraction over the last 4 transformer layers.
Feature vector psi = [h_start || h_end || h_mean || delta_H] in R^14336 for LaViDA variants.
Frozen VAE latent dimension 256 for auxiliary branches.
Maximum completion length 3072 tokens.

Seed-0 MATH-500 Results

Metric	Value
Greedy overall (`T=0`)	76.2%
`n=8` mean correctness (`T=0.6`)	79.57%
pass@8	80.2%
L4-5 pass@8	69.47%
Level-5 pass@8	61.19%

Interpretation: Ties A_2000 on greedy but significantly improves n=8 mean correctness over A_2000 on seed 0 (+4.70pp, 95% CI [+1.25, +8.12], p=0.0069).

Related Data

Related model repos:

How To Use

This checkpoint is expected to be used with the base model and PEFT/LoRA loading:

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen2.5-Math-7B"
adapter_id = "Pritish92/lavida-variant-D-seed0-oracleaug-alpha0p001"

tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(base_id, trust_remote_code=True, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

Use the same base-model cot-4shot evaluation path used in the LaViDA experiments for comparable MATH-500 numbers.

Limitations

This is a seed-0 research checkpoint; the main A_2000 vs D_OracleAug replication target is still seed 1.
Results are currently for MATH-500 only in the locked public ledger.
The model was trained for mathematical reasoning experiments and should not be treated as a general assistant.
Oracle-generated traces are machine-generated and filtered, not human process annotations.
The chi-square critic branches are controls / negative evidence in seed 0; the positive RL-side mechanism candidate is nearest-expert MSE (D_OracleAug).

Citation

bibtex

@misc{saha2026lavidadoracleaug,
  title  = {LaViDA Variant D Seed-0 OracleAug alpha=0.001},
  author = {Saha, Pritish},
  year   = {2026},
  url    = {https://huggingface.co/Pritish92/lavida-variant-D-seed0-oracleaug-alpha0p001}
}