Training examples: 17,133
Frozen evaluation examples: 902
Method: QLoRA adapter training, then merge into the full-precision Transformers checkpoint
Selected checkpoint: Epoch 2 (checkpoint-1072)

Validation loss:

Epoch	Validation loss
1	1.3532
2	1.2940
3	1.4327

Epoch 2 was selected because it had the best held-out token F1 and the lowest validation loss. Epoch 3 had one additional exact match, but lower token F1 and higher validation loss.

Frozen Evaluation

All results below use exactly the same frozen 902-example evaluation set.

Model	Exact match	Normalized exact match	Token F1
Base Ornith-1.0-9B	0.0000	0.0000	0.1312
Epoch 2 adapter	0.0022	0.0022	0.3272
Epoch 3 adapter	0.0033	0.0033	0.2942
Merged Epoch 2 model	0.0055	0.0055	0.3429

The merged model was evaluated as a standalone model on all 902 frozen examples.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Sara121/Ornith-1.0-9B-Engineering"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
model.eval()

Use the bundled tokenizer and chat template. Ornith generation conventions may include <think>...</think> reasoning content before the answer.

Limitations

This is a domain-adapted model for engineering QA and should be validated before use in production or compliance-sensitive workflows. Evaluation metrics are lexical and do not guarantee factual or regulatory correctness.