Labels (35): audio-classification, audio-to-audio, automatic-speech-recognition, feature-extraction, fill-mask, image-classification, image-feature-extraction, image-segmentation, image-text-to-text, image-to-3d, image-to-image, image-to-text, multiple-choice, object-detection, question-answering, reinforcement-learning, robotics, sentence-similarity, summarization, table-question-answering, tabular-classification, tabular-regression, text-classification, text-generation, text-retrieval, text-to-image, text-to-speech, text-to-video, time-series-forecasting, token-classification, … (35 total)
Date: 2026-07-28 19:26 UTC

[!NOTE] This model uses a custom classification head (mean pooling over a backbone without a native sequence-classification class), so loading requires trust_remote_code=True. vLLM serving requires a standard architecture.

Evaluation

Metric	Value
f1_micro @ 0.5	0.6748
f1_macro @ 0.5	0.5182
f1_micro @ tuned	0.6883
f1_macro @ tuned	0.6075

Per-label decision thresholds tuned on the eval split are stored in config.classifier_thresholds.

Choosing an operating point

The stored thresholds maximise per-label F1. The sigmoid probabilities are a usable confidence signal — measured on 200 held-out test cards:

Confidence slice	Top-1 correct
Most-confident 25% (p >= 0.96)	96%
Most-confident 50% (p >= 0.85)	92%
All predictions	76%
Least-confident 20%	45%

For precision-first use (e.g. auto-suggesting task_categories for datasets that lack them), act only on high-confidence predictions (p >= ~0.9) and route the rest to human review. Labels are restricted to the task categories that are live filters on huggingface.co/datasets, so every suggestion is a valid filterable tag.

Usage

python

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("davanstrien/dataset-card-task-classifier", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("davanstrien/dataset-card-task-classifier", trust_remote_code=True)

inputs = tokenizer("your text here", return_tensors="pt", truncation=True)
probs = torch.sigmoid(model(**inputs).logits)[0]
thresholds = torch.tensor(model.config.classifier_thresholds)  # tuned on validation
labels = [model.config.id2label[i] for i in (probs >= thresholds).nonzero().flatten().tolist()]
print(labels)

Reproduction

Produced on Hugging Face Jobs (gpu) with the train-classifier.py recipe from uv-scripts. Run it yourself:

bash

hf jobs uv run --flavor gpu --secrets HF_TOKEN \
    https://huggingface.co/datasets/uv-scripts/classification/raw/main/train-classifier.py \
    davanstrien/dataset-cards-with-task-categories davanstrien/dataset-card-task-classifier --label-column labels