You can deploy this model and start earning money today!

varianceofx/
aimo3-2nd-place-solver-adapted

Name: aimo3-2nd-place-solver-adapted
Availability: InStock

Views

No views yet

AIMO3 2nd Place Solution Adapted Notebook

This repository contains an adapted version of the 2nd place solution from the AI Mathematical Olympiad Progress Prize 3 (AIMO3) Kaggle competition. The notebook has been cleaned up and decoupled from the Kaggle execution environment so it can be studied, modified, and re-run outside of that platform.

Overview

The solution uses a large reasoning model (GPT-OSS 120B) served locally via vLLM, with a stateful Jupyter kernel sandbox for Python tool use. The model attempts each competition problem up to 8 times in parallel, then aggregates answers using a combined majority-vote + entropy-weighted scoring scheme.

Key design highlights:

System prompt engineering: Explicit constraints (answer must be in [0, 99999]), a structured problem-solving protocol (Parse, Explore, Plan, Execute, Verify), and a pre-\boxed{} checklist to reduce common errors.
Stateful code sandbox: A pool of persistent Jupyter kernels lets the model run Python mid-reasoning without restarting interpreter state between tool calls.
Streaming answer extraction: \boxed{} is scanned on a rolling window of streaming tokens so an answer can be captured as soon as it is written, without waiting for the full generation to complete.
Tail-windowed entropy scoring: Model confidence is estimated from per-token log-probabilities, windowed to the last 256 tokens so only the final decisive reasoning step contributes to the score.
Dynamic early stopping: Inference stops as soon as 4 identical answers are obtained, or when it becomes mathematically impossible for a trailing answer to overtake the current leader.

Adaptations from the Base Notebook

The following changes were made relative to the original competition submission by Parthenos and Andreas Bisiadis:

Kaggle-specific metadata removed: The kaggle block has been stripped from the notebook metadata. Paths referencing /kaggle/input/ and /kaggle/tmp/ are clearly marked and must be updated for your own environment (see Configuration below).
Sharpened system prompt: Added explicit reminders about the [0, 99999] answer range, modular arithmetic framing, and a mandatory self-check step before boxing the final answer.
Broadened answer extraction: _scan_for_answer now handles \boxed{42,000} (comma thousands-separators) and \boxed{ 42 } (extra whitespace).
Smarter answer selection: _select_answer combines raw majority vote, entropy-weighted vote, and a consensus bonus rather than a pure entropy-weighted sum.
Tail-windowed entropy: Entropy is averaged only over the last 256 tokens (logprob_tail) rather than the full generation.
Early-stop window: Added _can_remaining_change_winner to halt sampling once the leader is uncatchable by the second-place answer.

Requirements

Dependency	Notes
Python ≥ 3.11	Type hints use `X \| Y` union syntax
`vllm`	Serves the model via an OpenAI-compatible endpoint
`unsloth`	Optional; used for training (not inference)
`trl`	Optional; used for training (not inference)
`openai`	Python client for the vLLM endpoint
`openai_harmony`	Custom tokeniser / conversation-format library (see below)
`jupyter_client`	Manages the sandboxed Jupyter kernels
`transformers`	`set_seed` utility
`pandas`, `polars`	Result aggregation and display
`sympy`, `numpy`, `mpmath`	Available inside the solver's code sandbox

openai_harmony is a proprietary library distributed as a wheel inside the original competition dataset (/kaggle/input/aimo-3-utils/wheels.tar.gz). You will need access to that archive (or a compatible replacement) to run the notebook as-is.

Hardware Requirements

This solution was designed for high-end GPU hardware as used in the Kaggle competition environment. The requirements below reflect what is needed to run it as-is; see the notes for lighter alternatives.

Component	Minimum	Recommended
GPU	1 × A100 80 GB	1 × H100 80 GB (SXM or PCIe)
GPU VRAM	80 GB	80 GB
System RAM	128 GB	256 GB
Disk (model weights)	~240 GB free	NVMe SSD for fast weight preloading
CPU cores	16	32+ (parallel Jupyter kernel pool)

Configuration

All tuneable parameters live in the CFG class at the top of the notebook. The most important ones to update when running outside Kaggle:

python

class CFG:
    kv_cache_dtype = 'fp8_e4m3'   # requires H100/A100 with FP8 support; change to 'auto' otherwise
    attempts     = 8              # parallel inference attempts per problem
    workers      = 16             # Jupyter kernel pool size (set ≤ GPU-concurrent capacity)
    context_tokens = 65536        # max context length passed to vLLM

Also update the set_env call and the paths inside predict / the inference_server block if you are not using the Kaggle dataset structure.

How to Run

Install dependencies (the original solution uses offline wheels; adapt as needed for your environment) by running Cell A (if you prefer to install with internet) or Cell B (using existing offline pre-built wheels package).

Note that if you are planning to run Cell B, you will need to download the pre-built wheels package from the repository aimo-3-utils/wheels.tar.gz and set the path to point to the wheels package.
Update CFG.model_path to point to your local copy of GPT-OSS 120B (or a compatible model).
Open and run the notebook end-to-end. The solver will:
- Preload model weights into the OS page cache.
- Start a vLLM inference server on localhost:8000.
- Initialise a pool of Jupyter kernels.
- Call predict() for each problem in the dataset.
For local testing outside the competition runner, replace the inference_server.serve() / run_local_gateway() block with direct calls to solver.solve_problem(your_problem_text).

Repository Structure

.
└── aimo3-2nd-place-solution-adapted.ipynb   # Main notebook

Citation / Attribution

Link to this solver's Kaggle notebook version: https://www.kaggle.com/code/varianceofx/let-me-improve-this-cooking?scriptVersionId=306081269

Link to Parthenos' base Kaggle notebook: https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook

Link to Andreas Bisiadis' base Kaggle notebook: https://www.kaggle.com/code/andreasbis/aimo-3-gpt-oss-120b-with-tools

AI Mathematical Olympiad Progress Prize 3 (AIMO3), Kaggle, 2025.
2nd place solution — adapted for use outside the Kaggle environment.

License

This repository is shared for research and educational purposes. The underlying model weights (GPT-OSS 120B) and the openai_harmony library are subject to their own respective licenses. Please consult the original competition materials for details.

varianceofx/aimo3-2nd-place-solver-adapted