This is a 20-label Arabic university intent classifier fine-tuned from
ibm-granite/granite-embedding-311m-multilingual-r2. It routes one
question to zero, one, or several university-information labels. The model uses
first_token pooling and independent sigmoid outputs; it is not a generative model.
Model details
Architecture:ibm-granite/granite-embedding-311m-multilingual-r2 encoder + dropout + linear multi-label head
Decision thresholds: selected on validation data and stored in config.json and
thresholds.json
The repository includes custom Transformers modeling code so it loads through the standard
AutoClass API. Review the code in this repository before enabling trust_remote_code; in a
production deployment, pin revision to a reviewed model commit SHA.
Release notes
v0.3.0 adds an optional, validated dynamic-INT8 ONNX package for CPU server inference under
onnx/, together with backend- and precision-aware CLI and FastAPI examples. The canonical
SafeTensors checkpoint remains the portable source of truth, and the verified Q8-linear/Q4-
embedding browser package remains available for private WebGPU or WASM CPU inference. Trained weights, label
taxonomy, saved thresholds, and dataset release are unchanged from the earlier releases.
Evaluation
Split
Macro F1
Micro F1
Subset accuracy
LRAP
Validation
0.9896
0.9897
0.9790
0.9970
Test
0.9878
0.9888
0.9780
0.9952
Threshold selection used only the validation split. The test split remained held out until the
final comparison. Full aggregate and per-label results are available in
validation_metrics.json and test_metrics.json.
Local inference speed benchmark
MPS
Model
Parameters
Batch 1 p50 (ms)
Batch 1 q/s
Batch 8 q/s
Batch 32 q/s
Peak RSS (MiB)
AraUni Granite 311M Intent Classifier
311.7M
10.9
92.1
387.4
629.9
3363
AraUni MARBERTv2 Intent Classifier
162.3M
10.1
113.3
460.1
807.6
1774
AraUni Granite 97M Intent Classifier
97.4M
3.7
170.3
889.9
1640.6
1552
CPU
Model
Parameters
Batch 1 p50 (ms)
Batch 1 q/s
Batch 8 q/s
Batch 32 q/s
Peak RSS (MiB)
AraUni Granite 311M Intent Classifier
311.7M
18.3
57.1
112.9
115.9
3261
AraUni MARBERTv2 Intent Classifier
162.3M
11.0
80.6
139.2
116.4
1771
AraUni Granite 97M Intent Classifier
97.4M
5.8
162.4
338.7
369.9
1551
Measured on Apple M5 Max (PyTorch 2.13.0, Transformers 5.14.1). Each batch size used 5 warm-up and 30 measured iterations over the same deterministically shuffled questions derived from the test split of NajahUniv/arabic-univeristy-chatbot-qa-cleaned@v0.1.0, pinned to commit 9136ae156c49e1cc014020cef75c20a8b8d9aae0, after applying the repository's standard dataset-preparation pipeline.
Numbers include tokenization, padding, device transfer, model forward pass, sigmoid, thresholding, and result construction. They exclude model loading and HTTP overhead. Latency and throughput depend on hardware, software versions, input lengths, batch size, and thermal state; compare only rows from this same run. Peak RSS is whole-process memory and MPS uses unified memory.
The full machine-readable benchmark report is included as benchmark_results.json. Use the direct benchmark for model-to-model speed comparisons; use the FastAPI load test below to size a specific deployment.
Optional CPU INT8 server inference
The onnx/ directory contains a dynamic-shape, dynamic-INT8 ONNX model for
CPUExecutionProvider. It is a deployment alternative, not a replacement for the canonical
SafeTensors checkpoint. onnx/onnx_config.json pins the graph checksum, label order, thresholds,
runtime contract, numerical-verification report, and full validation-set quantization report. Its
decision threshold is recalibrated on the validation split for the quantized logits; the
SafeTensors checkpoint continues to use the canonical threshold in config.json.
Use MODEL_BACKEND=onnx MODEL_PRECISION=int8 in the included FastAPI example. The conservative
auto default retains PyTorch/SafeTensors on every device because INT8 speedups depend on the
server CPU's instruction set and workload; benchmark before enabling it in production.
Validation result
FP32 ONNX
INT8 ONNX
Graph size
1192.0 MiB
876.2 MiB
Macro F1
0.9896
0.9917
Micro F1
0.9897
0.9917
Decision threshold
canonical
0.45
The measured macro-F1 change after INT8 threshold recalibration is
-0.0021 (positive means a drop). See
onnx/quantization_report.json for source-threshold results, logit differences, and local latency.
Private browser inference (WebGPU and WASM CPU)
The webgpu/ directory contains a verified, fixed-shape ONNX Runtime Web model for private,
client-side inference. It uses Q8 linear weights and Q4 embedding weights, batch size 1, and the
same saved per-label thresholds as the Transformers model. Load webgpu/webgpu_config.json
first; it records the graph checksum, tokenizer artifacts, input contract, and validation report.
Applications can execute the same graph with ONNX Runtime Web's WebGPU provider or its WASM CPU
provider. WASM is more broadly compatible but can be substantially slower. A privacy-preserving
fallback should try WASM locally and require explicit consent before switching to server inference.
Run inference inside a Web Worker so model loading does not block the page. This WebGPU example
loads the pinned release and applies the saved per-label thresholds from webgpu_config.json:
For private CPU execution, put the same shared inference code in a separate worker and change the
runtime setup to:
javascript
1import*as ortfrom"onnxruntime-web/wasm";23ort.env.wasm.wasmPaths="/ort/";4ort.env.wasm.numThreads=1;// Works without cross-origin isolation.5const executionProviders =["wasm"];
Pass executionProviders to InferenceSession.create. Copy the matching ONNX Runtime Web .mjs
and .wasm files from node_modules/onnxruntime-web/dist/ to the public /ort/ directory. Keep
WebGPU and WASM in separate workers/bundles; do not assume that adding "wasm" after "webgpu"
in one provider list provides a compatible fallback for every quantized operator. If WebGPU fails,
start the WASM worker explicitly. Only switch to a server runtime after obtaining user consent.
This is multi-label classification: apply sigmoid independently to every logit, compare each score
with that label's saved threshold, and allow zero, one, or several labels to be selected. The
sigmoid scores are ranking/confidence signals and are not guaranteed to be calibrated probabilities.
Basic usage
python
1import torch
2from transformers import AutoModelForSequenceClassification, AutoTokenizer
34model_id ="NajahUniv/AraUni-Granite-311M-Intent-Classifier"5model_revision ="v0.3.0"6tokenizer = AutoTokenizer.from_pretrained(7 model_id,8 revision=model_revision,9 trust_remote_code=True,10)11model = AutoModelForSequenceClassification.from_pretrained(12 model_id,13 revision=model_revision,14 trust_remote_code=True,15).eval()1617text ="ما هي شروط القبول في الجامعة؟"18inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=model.config.max_length)19with torch.inference_mode():20 probabilities = torch.sigmoid(model(**inputs).logits[0])2122result ={}23for index, probability inenumerate(probabilities.tolist()):24 label = model.config.id2label[index]25 threshold = model.config.thresholds[label]26 result[label]={"probability": probability,"selected": probability >= threshold}2728selected_labels =[label for label, value in result.items()if value["selected"]]29print(selected_labels)
A complete command-line example is included at examples/basic_inference.py:
bash
1python examples/basic_inference.py \2 --model-id NajahUniv/AraUni-Granite-311M-Intent-Classifier \3 --revision v0.3.0 \4 --backend auto \5 --precision auto \6 --text "ما هي شروط التسجيل؟"
In Swagger UI at http://localhost:8000/docs, click Authorize and enter only the
MODEL_API_KEY value (change-me in the example). Swagger adds the Bearer prefix.
For public production hosting, also add TLS, request-size/rate limits, monitoring, and a pinned
MODEL_REVISION commit SHA. On macOS, use one Uvicorn worker so multiple processes do not each
load a separate copy of the model into memory.
Labels
0: academic_calendar
1: academic_programs
2: admissions
3: campus_services
4: contact_and_location
5: courses_and_study_plans
6: exams_and_grades
7: general_university_information
8: graduation
9: library
10: news_and_events
11: out_of_scope
12: registration
13: research_and_postgraduate
14: scholarships_and_aid
15: staff_and_departments
16: student_services
17: technical_support
18: transfer_and_equivalency
19: tuition_and_payments
Intended use and limitations
The model is intended for routing Arabic university-chatbot questions within the label taxonomy
above. It should not be treated as an authoritative source of admissions, academic, payment, or
policy advice. The training data is task-specific; consult the pinned dataset release for its
synthetic and real-user composition. The strong held-out scores may not transfer to other
universities, taxonomies, dialect distributions, spelling patterns, or production traffic.
Inputs outside the training distribution can still receive confident scores. Evaluate on real,
independently collected traffic and add human fallback/escalation before deployment. The sigmoid
values are classification scores, not guaranteed calibrated probabilities.
Reproducibility and repository contents
model.safetensors: complete encoder and classifier weights (the only weight copy)
config.json: architecture, label mappings, pooling, maximum length, and thresholds
tokenizer.json and tokenizer_config.json: tokenizer artifacts
configuration_arauni.py and modeling_arauni.py: AutoClass code
dataset_provenance.json, training_args.json, and metadata.json: provenance
validation_metrics.json, test_metrics.json, and thresholds.json: evaluation artifacts
benchmark_results.json: shared MPS/CPU speed comparison and environment metadata
onnx/: validated dynamic-INT8 ONNX CPU graph and verification reports
webgpu/: verified ONNX Runtime Web graph, tokenizer, thresholds, and reports
This model card reports the saved checkpoint artifacts; consult the linked base-model card for
its pretraining details, license, intended uses, and limitations.