Model Card for nebriq-model-classifier
Model Overview
Model name: nebriq-model-classifier
Model type: Transformer-based text classifier
Base model: DistilBERT (distilbert-base-uncased)
This model is designed to predict the semantic complexity of a user query and classify it into one of three categories:
simple — Simple, factual, or casual questionsmedium — Instructional, contextual, or practical requestsadvanced — Abstract, technical, or highly analytical queries
It helps intelligently route user queries to the most appropriate LLM backend based on their complexity (e.g., GPT-4o-mini, Claude, DeepSeek).
Intended Use
This model is intended to be used in applications where automatic routing of prompts to different Large Language Models (LLMs) is necessary. It provides a way to determine the semantic complexity of user inputs, facilitating the selection of the most appropriate LLM based on the input's complexity.
Example Use Cases:
- AI-powered note-taking apps (e.g., Nebriq)
- Custom AI model routing systems
- Intelligent query classification for multi-model systems
Model Details
-
Training Data: The model was trained on a small, manually curated dataset of user-like prompts. It has been optimized to classify the following categories:
-
Training Framework: Hugging Face Transformers with the Trainer API
-
Tokenizer: The model uses the DistilBERT tokenizer (distilbert-base-uncased).
Performance
The model achieves reasonable performance on the validation set. Example queries and their corresponding predictions are as follows:
-
"What is a cat?"
- Predicted label:
simple
- Confidence: 79.95%
-
"Explain how email works"
- Predicted label:
medium
- Confidence: 71.03%
-
"Design a secure authentication system for a web app"
- Predicted label:
advanced
- Confidence: 81.53%
Note: The model is most effective on queries that fall within the expected complexity range (simple, medium, advanced). For highly ambiguous or unclear inputs, performance may vary.
Model Training
- Training Environment: The model was trained in a Python environment using Hugging Face's Transformers library and PyTorch.
- Training Configuration:
- Batch size: 8
- Number of epochs: 5
- Learning rate: 2e-5
- Optimizer: AdamW
Ethical Considerations
- Bias: The model was trained on a relatively small and curated dataset, which may introduce biases based on the types of queries included. Further training on diverse data could help mitigate these biases.
- Fairness: As the model classifies queries based on their complexity, it is essential to ensure that its predictions are used in a way that does not unfairly impact or exclude certain types of queries.
- Transparency: This model is intended for use in AI-driven systems where routing based on query complexity can enhance the user experience.
How to Use
To use this model for inference, you can load it with the following code:
1from transformers import pipeline
2
3classifier = pipeline("text-classification", model="your-username/nebriq-model-classifier")
4
5prompt = "Explain how HTTP requests work"
6result = classifier(prompt)
7print(result)
8# Example output: [{'label': 'medium', 'score': 0.92}]
