🧠 NeuroGolf 2026: Ultra-Efficient ARC-AGI Solver

📌 Overview

🏁 Competition Constraints

• ONNX File Size Limit: ≤ 1.44 MB
• Parameter Budget:
  • ~360K parameters (Float32)
  • ~1.4M parameters (INT8 quantized)
• Input/Output Shape:
  (1, 10, 30, 30) for both input and output logits

🏗️ Architecture

🧠 Mega-Teacher Model (MegaTeacherARCNet)

• Purpose: Captures complex patterns and logic across 400+ ARC tasks
• Dimensions: 512 hidden units, 16 residual blocks deep
• Technique: Standard convolutions + deep residual architecture for maximum pattern recognition

⚡ Student Model (UltraTinyARCNet)

• Purpose: Final deployable model optimized for strict size limits
• Dimensions: 56 hidden units, 5 residual blocks deep

🔧 Key Techniques

• Depthwise Separable Convolutions → ~10× parameter reduction
• No Bias Terms → bias=False in Conv2d to reduce parameter count
• Residual Blocks → Maintain gradient flow in ultra-small networks

🔄 Training Pipeline

1️⃣ Teacher Training

• Train Mega-Teacher for 50 epochs
• Dataset: Full 400+ ARC tasks
• Augmentation: 8× (rotations + flips)

2️⃣ Knowledge Distillation

• Student learns from teacher’s soft probability distributions
• Transfers “dark knowledge”
• Achieves better generalization vs hard-label training

3️⃣ Pruning & Fine-Tuning

✂️ Pruning

• Remove 30–35% of low-magnitude weights
• Method: L1 unstructured pruning
• Ensures ONNX file remains under 1.44 MB

🔧 Fine-Tuning

• 20 epochs recovery training
• Restores performance lost during pruning

⚙️ Installation & Usage

📋 Prerequisites

• Linux environment (Debian/Kali recommended)
• Python 3.10+
• NVIDIA GPU with CUDA support (optimized for 2× T4 setup)

🛠️ Setup