Veyra-100M is a compact, encoder-only text classification model built by Dl26. It is designed for fast English topic classification using a BERT-style bidirectional Transformer encoder. Instead of generating answers autoregressively, it reads the input text in a single encoder pass and returns a class score over a fixed topic taxonomy.

The released checkpoint, Dl26/Veyra-100M, is a 100M-parameter-class classifier trained from random initialization. It uses the standard Transformers bert model type, so it loads with AutoModelForSequenceClassification and does not require trust_remote_code=True.

Veyra-100M is intended for practical classification workflows such as document routing, topic labeling, metadata enrichment, dataset filtering, and lightweight encoder research.

Why this model

Compact BERT-style encoder for fast text classification
Standard Hugging Face Transformers compatibility
No custom architecture files or remote code requirement
14-way topic taxonomy based on DBpedia-style categories
Useful for document routing, triage, and automatic labeling
Trained from scratch with AdamW rather than adapted from a pretrained encoder
Designed to run efficiently on CPU, GPU, and batch inference pipelines

Model details

Property	Value
Model name	`Veyra-100M`
Developer	Dl26
Model type	Encoder-only sequence classifier
Transformers model type	`bert`
Architecture	`BertForSequenceClassification`
Parameters	101,515,758
Hidden size	736
Layers	12
Attention heads	16
Intermediate size	2,944
Max positions	512
Vocabulary size	30,522
Number of labels	14
Training objective	Supervised single-label classification
License	Apache 2.0

Supported labels

Veyra-100M predicts one label from the following fixed topic set:

Label	Typical meaning
`Company`	Businesses, organizations, commercial entities
`EducationalInstitution`	Schools, universities, colleges, academic institutions
`Artist`	Musicians, painters, writers, performers, creative people
`Athlete`	Sports players and athletic figures
`OfficeHolder`	Political figures and public officials
`MeanOfTransportation`	Vehicles, ships, aircraft, trains, transport systems
`Building`	Buildings, structures, monuments, architectural locations
`NaturalPlace`	Mountains, rivers, islands, parks, natural locations
`Village`	Villages, towns, municipalities, local settlements
`Animal`	Animal species and animal-related entries
`Plant`	Plant species and plant-related entries
`Album`	Music albums and recorded releases
`Film`	Films, movies, cinematic works
`WrittenWork`	Books, written publications, literary works

Installation

pip install -U transformers torch accelerate

For CPU-only inference, accelerate is optional:

pip install -U transformers torch

Quick start

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-100M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Apple announced a new processor for its laptop computers."

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=128,
)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_id = int(logits.argmax(dim=-1))
label = model.config.id2label[predicted_id]
print(label)

GPU inference

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-100M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

texts = [
    "The athlete scored two goals in the final match.",
    "The album includes twelve songs recorded in London.",
    "The village is located near a river and surrounded by hills.",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    truncation=True,
    padding=True,
    max_length=128,
).to(model.device)

with torch.no_grad():
    logits = model(**inputs).logits

predictions = logits.argmax(dim=-1).tolist()
for text, label_id in zip(texts, predictions):
    print(model.config.id2label[label_id], "-", text)

Batch inference

For high-throughput classification, batch inputs together and keep a consistent max_length.

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-100M"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).to(device)
model.eval()

def classify_batch(texts, batch_size=64):
    results = []
    for start in range(0, len(texts), batch_size):
        batch = texts[start:start + batch_size]
        inputs = tokenizer(
            batch,
            return_tensors="pt",
            truncation=True,
            padding=True,
            max_length=128,
        ).to(device)
        with torch.no_grad():
            logits = model(**inputs).logits
        probs = logits.softmax(dim=-1)
        label_ids = probs.argmax(dim=-1).tolist()
        scores = probs.max(dim=-1).values.tolist()
        for text, label_id, score in zip(batch, label_ids, scores):
            results.append({
                "text": text,
                "label": model.config.id2label[label_id],
                "score": float(score),
            })
    return results

Interpreting outputs

Veyra-100M is a single-label classifier. For each input text, the model returns logits over the 14 supported labels. The highest-scoring label is usually treated as the predicted class.

Use confidence scores as a routing signal, not as a perfect measure of correctness. Low-confidence predictions are good candidates for fallback handling, manual review, or a broader classification model.

Recommended practices:

Use title plus body text when available.
Keep inputs descriptive rather than extremely short.
Use max_length=128 for fast classification.
Increase to max_length=256 or 512 when documents need more context.
Calibrate confidence thresholds on your own validation set.

Evaluation highlights

The downloaded checkpoint was tested after training on DBpedia-style topic classification.

Evaluation	Result
Held-out 5K evaluation sample	98.4% accuracy
Separate 2K test script sample	96.2% accuracy
Model load path	`AutoModelForSequenceClassification`
Remote code required	No

These numbers are useful as sanity checks for the released checkpoint, but users should evaluate the model on their own data before deployment.

Input formatting

The model works best with natural, descriptive English text.

Good examples:

text

Apple announced a new processor for its laptop computers.
The athlete scored two goals in the final match.
The album includes twelve songs recorded in London.

For records with separate title and body fields, combine them:

text = f"{title}. {description}"

Very short or generic inputs may be ambiguous. For example, a sentence such as “The film won several awards” may not contain enough detail for stable classification.

Intended use

Veyra-100M is intended for:

topic classification
document routing
metadata enrichment
dataset filtering
search indexing labels
moderation queue triage by broad topic
lightweight encoder benchmarking
educational experiments with from-scratch classifiers

Out-of-scope use

Veyra-100M is not intended for:

open-ended text generation
semantic embedding search
safety moderation as a policy model
legal, medical, financial, or identity-sensitive decision making
reliable classification outside the supported label taxonomy without additional validation

Limitations

The model supports a fixed 14-label taxonomy.
It is trained for English text and may be unreliable on other languages.
It can misclassify short, vague, adversarial, or out-of-domain inputs.
It is not a general-purpose reasoning model.
It is not a replacement for human review in high-impact workflows.
Confidence scores may require calibration for production systems.

Citation

bibtex

@misc{dl26_2026_veyra_100m,
  title        = {Veyra-100M: Fast Encoder-Based Topic Classification},
  author       = {Dl26},
  year         = {2026},
  url          = {https://huggingface.co/Dl26/Veyra-100M}
}