You can deploy this model and start earning money today!

Ashybalka/
xlm-roberta-taxonomy-seniority-de

Name: xlm-roberta-taxonomy-seniority-de
Availability: InStock

seniority-classification

taxonomy-seniority

text-embeddings-inference

FacebookAI/xlm-roberta-base

quantized

apache-2.0

endpoints_compatible

Views

No views yet

XLM-RoBERTa Job Taxonomy — Seniority (German)

Fine-tuned xlm-roberta-base for classifying German job listings by seniority level into 5 grades: Trainee, Junior, Mid, Senior, Lead+.

Part of the JobBlast taxonomy model family — a system for automated classification of German-language job postings. This model is orthogonal to the industry classifier: it determines the experience level required for a position, independent of the field. Combine it with xlm-roberta-taxonomy-main-de to get both what kind of role and what level a vacancy targets.

Test Metrics

Metric	Value
Accuracy	94.50%
F1 macro	93.90%
F1 weighted	94.60%

Evaluated on a held-out test set of 20,55 German job listings.

Per-class results

Grade	Precision	Recall	F1	Support
Trainee	0.977	1.000	0.988	126
Junior	0.893	0.893	0.893	75
Mid	0.980	0.933	0.956	1235
Senior	0.881	0.960	0.919	448
Lead+	0.898	0.982	0.939	171

The close gap between F1 macro (0.939) and F1 weighted (0.946) indicates balanced performance across grades despite the underlying class imbalance — rare grades (Junior, Lead+) are not sacrificed for accuracy on the dominant Mid class.

Label Schema

Label	Typical signals in German job ads
Trainee	`Auszubildende:r`, `Ausbildung`, `Praktikum`, `Werkstudent:in`, `Trainee-Programm`, `Berufseinstieg ohne Erfahrung`
Junior	`Junior`, `Berufseinsteiger:in`, `1-2 Jahre Erfahrung`, `erste Berufserfahrung`
Mid	No explicit grade marker, `2-5 Jahre Erfahrung`, `selbstständige Arbeitsweise`, default for most listings
Senior	`Senior`, `erfahren`, `5+ Jahre Erfahrung`, `tiefgreifende Kenntnisse`, `Expert:in`
Lead+	`Lead`, `Principal`, `Staff`, `Head of`, `Team Lead`, `Architect`, `Führungsverantwortung`, leadership / staff-level positions

Note on `Lead+`

The original LLM-labeled dataset distinguished Lead and Principal as separate grades. Because Principal had only ~200 examples — too few to train a reliable transformer classifier — the two classes were merged into Lead+ for training. If you need to recover Principal specifically, apply a post-processing regex on the job title after this model's prediction (e.g. r"\b(Principal|Staff|Chief|VP)\b").

Repository Structure

Ashybalka/xlm-roberta-taxonomy-seniority-de/
├── config.json                    # shared — label map, model config
├── tokenizer.json                 # shared — fast tokenizer
├── tokenizer_config.json          # shared
├── sentencepiece.bpe.model        # shared
├── special_tokens_map.json        # shared
├── test_metrics.json              # evaluation results
├── classification_report.txt      # full per-class report
│
├── pytorch/
│   ├── config.json                # needed for from_pretrained(subfolder=)
│   └── model.safetensors          # GPU inference / fine-tuning (~1.1 GB)
│
├── onnx/
│   └── model.onnx                 # CPU fp32 inference (~1.1 GB)
│
└── onnx-int8/
    └── model_quantized.onnx       # CPU INT8 quantized (~280 MB, 3-4× smaller)

Usage

Input Format

[Job Title] [SEP] [Job Description]

python

text = "Senior DevOps Engineer [SEP] Mindestens 5 Jahre Erfahrung mit " \
       "Kubernetes, Terraform und CI/CD Pipelines. Teamführung von Vorteil."

PyTorch (GPU / fine-tuning)

python

from transformers import pipeline
clf = pipeline(
    "text-classification",
    model="Ashybalka/xlm-roberta-taxonomy-seniority-de",
    subfolder="pytorch",
    device=0,  # GPU; -1 for CPU
)
result = clf("Senior DevOps Engineer [SEP] 5+ Jahre Erfahrung mit Kubernetes und Terraform.")
print(result)
# [{'label': 'Senior', 'score': 0.9645}]

ONNX fp32 (CPU inference)

python

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline
model     = ORTModelForSequenceClassification.from_pretrained(
                "Ashybalka/xlm-roberta-taxonomy-seniority-de",
                subfolder="onnx"
            )
tokenizer = AutoTokenizer.from_pretrained("Ashybalka/xlm-roberta-taxonomy-seniority-de")
clf       = pipeline("text-classification", model=model, tokenizer=tokenizer)
result = clf("Werkstudent Marketing [SEP] Unterstützung bei Social-Media-Kampagnen, 15-20 Stunden pro Woche.")
print(result)
# [{'label': 'Trainee', 'score': 0.9821}]

ONNX INT8 (CPU, lightweight)

python

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline
model     = ORTModelForSequenceClassification.from_pretrained(
                "Ashybalka/xlm-roberta-taxonomy-seniority-de",
                subfolder="onnx-int8"
            )
tokenizer = AutoTokenizer.from_pretrained("Ashybalka/xlm-roberta-taxonomy-seniority-de")
clf       = pipeline("text-classification", model=model, tokenizer=tokenizer)

Direct ONNX Runtime (no transformers)

For production FastAPI services or environments without transformers:

python

import json
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer
# download model
path = snapshot_download(
    "Ashybalka/xlm-roberta-taxonomy-seniority-de",
    allow_patterns=["onnx/model.onnx", "tokenizer.json", "config.json"]
)
# load
session   = ort.InferenceSession(f"{path}/onnx/model.onnx",
                                  providers=["CPUExecutionProvider"])
tokenizer = Tokenizer.from_file(f"{path}/tokenizer.json")
tokenizer.enable_truncation(max_length=510)
tokenizer.no_padding()
with open(f"{path}/config.json") as f:
    labels = [v for _, v in sorted(json.load(f)["id2label"].items(), key=lambda x: int(x[0]))]
vocab    = tokenizer.get_vocab()
bos, eos = vocab["<s>"], vocab["</s>"]
def classify_seniority(title: str, description: str) -> dict:
    text     = f"{title} [SEP] {description}"
    encoding = tokenizer.encode(text, add_special_tokens=False)
    ids      = [bos] + encoding.ids + [eos]
    mask     = [1] * len(ids)
    logits = session.run(None, {
        "input_ids":      np.array([ids],  dtype=np.int64),
        "attention_mask": np.array([mask], dtype=np.int64),
    })[0][0]
    exp   = np.exp(logits - logits.max())
    probs = exp / exp.sum()
    idx   = int(np.argmax(probs))
    return {"grade": labels[idx], "confidence": round(float(probs[idx]), 4)}
print(classify_seniority(
    "Lead Software Architect",
    "Verantwortung für die Architektur unserer Microservices-Plattform, Führung eines 8-köpfigen Teams."
))
# {'grade': 'Lead+', 'confidence': 0.9456}

Training Details

Parameter	Value
Base model	`FacebookAI/xlm-roberta-base`
Training samples	20,545 German job listings
Labeling	LLM consensus (3-model voting: Qwen3-4B + Gemma-2-9B + Llama-3.1-8B)
Agreement filter	2/3 or 3/3 required
Soft labels	Vote distribution used as soft targets (e.g. `[Senior, Senior, Lead+]` → `{Senior: 0.67, Lead+: 0.33}`)
Class merging	`Principal` (206 samples) merged into `Lead+` due to insufficient data
Max length	512 tokens
Learning rate	2e-5
Epochs	5 (early stopping, best epoch: 4)
Class weighting	Balanced

Class distribution in training data

Grade	Count	Share
Mid	12,351	60.1%
Senior	4,476	21.8%
Lead+	1,709	8.3%
Trainee	1,262	6.1%
Junior	747	3.6%

The 16.5× imbalance ratio between Mid and Junior is handled through balanced class weights at training time.

Limitations

Optimized for German job listings only — performance on other languages is not validated
Junior is the weakest class (F1 0.893) due to limited training data (747 samples) and a genuinely ambiguous Junior↔Mid boundary in German job ads, where "1-3 Jahre Erfahrung" often overlaps with the Mid grade definition
Slight upward bias on grade boundaries: when uncertain between Mid and Senior (or Senior and Lead+), the model tends to predict the higher grade — visible as higher recall than precision for Senior and Lead+
Lead+ includes what was originally labeled as Principal. For applications that need to distinguish Principal/Staff/Chief-level roles specifically, apply title-based post-processing on top of the model output
The model classifies the seniority required by the job ad, not the actual seniority of any candidate
Inputs longer than 512 tokens are truncated — title and the first paragraph of the description carry most of the seniority signal anyway
The model is trained on German job listings and has learned to prioritize legally-defined seniority markers (Werkstudent, Immatrikulationsbescheinigung, Praktikum, Ausbildung) above explicit title keywords. For listings where the title and description conflict, the model will follow the strongest unambiguous signal, which usually means description-level legal markers override title-level grade keywords. This matches German HR practice but may differ from how other markets handle title-based classification.

Related Models

Model	Categories	Use case
xlm-roberta-taxonomy-main-de	21 top-level	Industry / field classification
xlm-roberta-taxonomy-it-de	14 IT subcategories	Detailed IT role classification
xlm-roberta-taxonomy-seniority-de (this model)	5 grades	Experience-level classification, orthogonal to industry

A typical JobBlast pipeline: the main model assigns the top-level category, IT-tagged listings are passed to the it model for fine-grained IT-role classification, and the seniority model runs in parallel on every listing to attach an experience grade — giving a (category, sub_category, seniority) triple per vacancy.

Ashybalka/xlm-roberta-taxonomy-seniority-de

XLM-RoBERTa Job Taxonomy — Seniority (German)

Test Metrics

Per-class results

Label Schema

Note on Lead+

Repository Structure

Usage

Input Format

PyTorch (GPU / fine-tuning)

ONNX fp32 (CPU inference)

ONNX INT8 (CPU, lightweight)

Direct ONNX Runtime (no transformers)

Training Details

Class distribution in training data

Limitations

Related Models

Ashybalka/
xlm-roberta-taxonomy-seniority-de

Note on `Lead+`