See original answerdotai/ModernBERT-base model card for additional information. This model is intended to classify queries for LLM routing. where advanced/complicated queries are labeled as 1 (large_llm) and simpler queries are labeled as 0 (small_llm).

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 16
gradient_accumulation_steps: 2
bf16: True
seed: 42
optimizer: Use adamw_torch_fused
lr_scheduler_type: linear
num_epochs: 5

Training Code

GITHUB URL TO BE ADDED

Training results

Epoch	Validation Loss	F1
1.0	0.0296	0.9907
2.0	0.0327	0.9911
3.0	0.0474	0.9933
4.0	0.0563	0.9933
5.0	0.0554	0.9933

anthonyivn/ModernBERT-Base-llm-router

ModernBERT-large-llm-router

Model description

Training hyperparameters

Training Code

Training results

anthonyivn/
ModernBERT-Base-llm-router