🇳🇵 Supernova TeraLLM Reranker V1
Supernova TeraLLM Reranker V1 is an experimental Nepali-focused
Transformer reranker developed by the Supernova project.
It scores the relevance of a document with respect to a query
and is intended for search and retrieval pipelines.
Status: Production baseline / research model
🚀 Model Overview
The production architecture is a Transformer-based pair reranker.
1Query + Document
2 ↓
3Supernova Tokenizer
4 ↓
5Token IDs
6 ↓
7Embedding
8 ↓
916 Transformer Encoder Layers
10 ↓
11CLS Representation
12 ↓
13Ranking MLP
14 ↓
15Relevance Score
🧠 Architecture
Verified production architecture:
1Embedding
2├── Vocabulary size: 4
3└── Hidden size: 1024
4
5Transformer Encoder
6├── Layers: 16
7├── Hidden size: 1024
8├── Feed-forward size: 4096
9├── Multi-head self-attention
10└── Dropout: 0.1
11
12Ranking Head
13├── Linear: 1024 → 512
14├── GELU
15├── Linear: 512 → 1
16└── Sigmoid
Parameters
202,593,281 parameters
Production checkpoint size:
Approximately 91.96 MB
🔤 Tokenizer
The production tokenizer was verified against the model.
1Model vocabulary: 4
2Tokenizer vocabulary: 4
3Embedding dimension: 1024
Compatibility test:
1Input shape: (1, 128)
2Maximum token ID: 3
3Vocabulary size: 4
The generated token IDs are compatible with the model embedding.
📚 Confirmed External Dataset
Bharat-NanoMSMARCO Nepali
The confirmed external Nepali retrieval dataset used during
the Supernova retrieval experiments is:
carlfeynman/Bharat_NanoMSMARCO_ne
Dataset:
Verified structure:
1Corpus: 5,043 documents
2Queries: 50 queries
3Qrels: 50 relevance judgments
4Language: Nepali
5Task: Information Retrieval
6License: CC-BY-4.0
The dataset contains Nepali retrieval queries, a document corpus,
and query-document relevance judgments.
Users must follow the original dataset's license and attribution
requirements.
🧪 Supernova Fine-Tuning Data
A later fine-tuning experiment constructed derived ranking triplets
from the available retrieval material.
1Random triplets: 1,000
2Hard-negative triplets: 500
3Total: 1,500
Split:
1Training: 1,350
2Validation: 150
These triplets are derived experimental examples, not a separate
public dataset.
⚔️ Supernova vs Jina Reranker V3
A custom three-example live benchmark was performed.
Categories:
- Historical Nuance
- Geographical Logic
- Slang / Intent Extraction
Results
1Category Supernova V1 Jina-v3
2---------------------------------------------------
3Historical Nuance PASS PASS
4Geographical Logic PASS PASS
5Slang / Intent FAIL FAIL
6---------------------------------------------------
7Pairwise Accuracy 2/3 2/3
8 66.67% 66.67%
Mean Positive-Negative Margin
1Supernova V1: +0.001901
2Jina Reranker V3: +0.140182
Experimental CPU Latency
1Supernova total: 8.5254 seconds
2Jina-v3 total: 88.3780 seconds
3
4Supernova/example: 2.8418 seconds
5Jina-v3/example: 29.4593 seconds
These measurements are specific to the experimental environment.
⚠️ Benchmark Interpretation
The three-example benchmark resulted in an accuracy draw:
Supernova V1: 2/3
Jina-v3: 2/3
Therefore this benchmark does NOT establish that Supernova V1
beats Jina Reranker V3.
Jina-v3 produced a substantially larger mean ranking margin in
this particular test.
A substantially larger independent benchmark is required for
strong performance claims.
🎯 Intended Use
Supernova V1 can be used for:
- Nepali search
- Semantic retrieval
- Search-result reranking
- Document relevance scoring
- RAG pipelines
- Nepali information retrieval research
- Multilingual retrieval research
Typical pipeline:
1User Query
2 ↓
3First-Stage Retriever
4 ↓
5Top-K Documents
6 ↓
7Supernova Reranker
8 ↓
9Re-ranked Results
10 ↓
11Search / RAG
🛑 Limitations
- The confirmed Nepali retrieval dataset is relatively small.
- The Jina comparison contains only three examples.
- The benchmark is not a large standardized reranking benchmark.
- Results can vary by domain and query distribution.
- The current benchmark does not prove superiority over Jina-v3.
- Larger held-out evaluation is required.
- More hard-negative mining is required for stronger generalization.
🔬 Future Research
Future Supernova reranker versions will investigate:
- Larger Nepali retrieval datasets
- More diverse Nepali queries
- Stronger hard-negative mining
- Better query-document interaction
- Parameter-efficient fine-tuning
- Improved ranking objectives
- Larger held-out benchmarks
- CPU-efficient inference
- Multilingual retrieval
- Better generalization
🔁 Reproducibility
The published V1 checkpoint should remain the immutable baseline.
Future experiments should create a separate fine-tuning copy:
1Supernova V1
2 ↓
3Immutable baseline
4
5Supernova V1
6 ↓
7Fine-tuning copy
8 ↓
9Nepali ranking data
10 ↓
11Fine-tuning
12 ↓
13Held-out evaluation
📦 Production Checkpoint
1File: pytorch_model.bin
2Parameters: 202,593,281
3Size: ~91.96 MB
📜 Dataset Attribution
Bharat-NanoMSMARCO Nepali:
Please consult the original dataset card for complete licensing
and attribution requirements.
📖 Citation
1@misc{supernova_teraillm_reranker_v1,
2 title={Supernova TeraLLM Reranker V1},
3 author={Supernova},
4 year={2026},
5 publisher={Hugging Face},
6 note={Nepali-focused Transformer reranker}
7}
⚖️ Disclaimer
This model is provided for research and experimental purposes.
Benchmark results represent measurements from the development
environment and are not universal guarantees of performance.
Users should evaluate the model on their own data and deployment
environment before production use.
🇳🇵 Supernova
Building AI infrastructure for Nepali language technology.
