R2-Router: A New Paradigm for LLM Routing with Reasoning
R2-Router intelligently routes each query to the optimal (LLM, token budget) pair, jointly optimizing accuracy and inference cost. Ranked
#1 on the
RouterArena leaderboard.
RouterArena Performance
RouterArena Leaderboard Official leaderboard results on 8,400 queries:
Metric Value Accuracy 71.23% Cost per 1K Queries $0.061 Arena Score (beta=0.1) 71.60 Robustness Score 45.71% Rank #1
Quick Start
Installation
We recommend using
uv for fast, reliable environment setup:
1 # Install uv (if not already installed)
2 curl -LsSf https://astral.sh/uv/install.sh | sh
3
4 # Create environment and install dependencies
5 uv venv .venv && source .venv/bin/activate
6 uv pip install scikit-learn numpy joblib huggingface_hub vllm
With vLLM Server (Recommended)
Start the embedding server once, then route from any process without reloading the model:
1 # Terminal 1: Start vLLM embedding server (runs once, stays alive)
2 uv pip install vllm
3 vllm serve Qwen/Qwen3-0.6B --runner pooling --port 8000
1 # Terminal 2: Route queries (connects to the running server)
2 from huggingface_hub import snapshot_download
3 import sys
4
5 path = snapshot_download ( "JiaqiXue/r2-router" )
6 sys . path . insert ( 0 , path )
7
8 from router import R2Router
9
10 router = R2Router . from_pretrained ( path , embed_url = "http://localhost:8000" )
11 result = router . route_text ( "Solve this integral" )
12 print ( f"Model: { result [ 'model_full_name' ] } , Budget: { result [ 'token_limit' ] } " )
13 print ( f"Estimated Quality: { result [ 'predicted_quality' ] : .3f } , Estimated Cost: $ { result [ 'predicted_cost' ] : .6f } " )
Adjusting Lambda (Cost-Accuracy Tradeoff)
The lambda parameter controls the tradeoff between accuracy and cost:
lambda → 1.0 : Minimize cost (routes to cheaper models)lambda → 0.0 : Maximize accuracy (routes to the best model regardless of cost)Default: 0.999 (strongly cost-sensitive, as used in our RouterArena submission)
1 # Cost-sensitive (default, as submitted to RouterArena)
2 router = R2Router . from_pretrained ( path , lambda_val = 0.999 )
3
4 # Balanced accuracy vs cost
5 router = R2Router . from_pretrained ( path , lambda_val = 0.5 )
6
7 # Accuracy-first (ignores cost, always picks highest quality)
8 router = R2Router . from_pretrained ( path , lambda_val = 0.0 )
9
10 # Override lambda per query
11 result = router . route_text ( "Solve this integral" , lambda_val = 0.5 )
Train from Scratch
1 from huggingface_hub import snapshot_download
2 import sys
3
4 path = snapshot_download ( "JiaqiXue/r2-router" )
5 sys . path . insert ( 0 , path )
6
7 from router import R2Router
8
9 # Train predictors with custom hyperparameters
10 router = R2Router . from_training_data ( path , k = 80 , lambda_val = 0.999 )
Architecture
R2-Router jointly optimizes which model to use and how many tokens to allocate per query.
Routing Formula
risk(M, b) = (1 - lambda) * predicted_quality(query, M, b) - lambda * predicted_tokens(query, M) * price_M / 1e6
(M*, b*) = argmax riskPipeline
Input Query
|
[1] Embed with Qwen3-0.6B -> 1024-dim vector
|
[2] For each (model, budget) pair:
- Predict quality (accuracy)
- Predict output token count
- Compute risk = (1-lambda) * quality - lambda * cost
|
[3] Select (model, budget) with highest risk
|
Output: (model_name, token_budget)Model Pool (6 LLMs)
Model Output $/M tokens Qwen3-235B-A22B $0.463 Qwen3-Next-80B-A3B $1.10 Qwen3-30B-A3B $0.33 Qwen3-Coder-Next $0.30 Gemini 2.5 Flash $2.50 Claude 3 Haiku $1.25
Token Budgets
4 output token limits: 100, 200, 400, 800 tokens.
Key Parameters
Parameter Value K (neighbors) 80 Lambda 0.999 Distance Metric Cosine Weights Distance-weighted Embedding Dim 1024
Repository Contents
config.json # Router configuration (models, budgets, prices, hyperparams)
router.py # Self-contained inference code (embed + route)
training_data/
embeddings.npy # Sub_10 training embeddings (809 x 1024)
labels.json # Per-(model, budget) accuracy & token labels
checkpoints/
quality_knn_*.joblib # Pre-fitted quality predictors (18 total)
token_knn_*.joblib # Pre-fitted token predictors (6 total)Ways to Use
Method GPU? Description route_text() + vLLM serverYes (server) Start vllm serve once, route from anywhere via HTTP route_text() + local vLLMYes (local) Auto-loads Qwen3-0.6B on first call, caches it route(embedding)No Route from pre-computed 1024-dim embedding from_training_data(path)No Train your own predictors with custom hyperparameters
Training Details
Following
chayan , we only use the official
sub_10 split (809 queries, 10% of the full 8,400) for training. No full-set data is used during training or hyperparameter tuning.
Training Data : RouterArena sub_10 split (809 queries)Method : Nearest-neighbor regression with cosine distance, distance-weightedEvaluation : Full 8,400 RouterArena queries (no data leakage)Training Time : < 1 second
Citation
1 @article{xue2026r2,
2 title={R2-Router: A New Paradigm for LLM Routing with Reasoning},
3 author={Xue, Jiaqi and Lou, Qian and Xing, Jiarong and Huang, Heng},
4 journal={arXiv preprint arXiv:2602.02823},
5 year={2026}
6 }
License
Apache 2.0
