This model is a reproduce version of GLiNER-large, the training hyperparameters are different from the original model.

The detail of training hyperparameters can see in deberta.yaml.

Except for config in deberta.yaml, i manually set the lr_scheduler_type to cosine_with_min_lr and lr_scheduler_kwargs to {"min_lr_rate": 0.01} in train.py:

    training_args = TrainingArguments(
        ...
        lr_scheduler_type="cosine_with_min_lr",
        lr_scheduler_kwargs={"min_lr_rate": 0.01},
        ...
    )

NOTE: The result is not stable, i guess the random shuffle of the dataset is the reason.

Here are two weights, one is the final model after 4k iterations, which has the best performance on the zero-shot evaluation, and the other is the model after full training.

See https://github.com/urchade/GLiNER