Aurelius v2 — verifier-native serving system (model card)

Results (measured on this base)

Capability lever = CAPACITY (the real win)

What worked / what didn't (honest)

• ✅ Capacity (bigger base) — the capability lever.
• ✅ maj@N self-consistency (math) — free, +8.2pp on hard math. SHIPPED.
• ✅ best-of-N + repair (code, exec-verifier) — +10.5pp MBPP on v1; carried to v2.
• ✗ LLM-as-verifier rerank (math) — NULL: verifier@8 73.1% ≈ maj@8 73.9% (1 worse). Zero-shot self-verification does not beat majority voting; the remaining ~7.4pp selection gap (maj→oracle 81.3%) needs a TRAINED verifier/RLVR.
• ✗ SFT-for-reasoning — confirmed neutral (v1: MATH 46.5%=base 46.5%). Reasoning is inherited, not SFT-able.
• ✗ RLVR on frozen 8B / flywheel-into-greedy — prior nulls; not revisited.

Usage

solve("math", "<problem>")                      # -> {answer, confidence, votes}
solve("code", "<task asking for one ```python block>", tests="<assert lines>")   # -> {code, how, passed}

Limitations

• Code lever is only as strong as the caller's tests. Tests stricter than the true spec (e.g. demanding a list when a correct algorithm returns a tuple) report failure even on correct code — spec the output type in the prompt and/or normalize in the tests. solve_code surfaces feedback on failure to debug this.
• maj@N self-consistency helps only where the model samples the right answer sometimes (no help on hard-capability problems it never gets — ~15% of MATH-L5 never-recovered).

Files

• aurelius_serve.py — self-contained serving (maj@N math + best-of-N+repair code)
• *.gguf — quantized weights for local serving (if present)