intent-student-L6_compact_distilled – AI Model by gomyk | AlphaNeural AI

You can deploy this model and start earning money today!

L6_compact_distilled

Ultra-compact multilingual sentence encoder (~71.2MB) for intent classification. 6 layers bottom + 20K vocab + distilled

Performance

Model	Size	MassiveIntent	MassiveScenario	Average
Teacher (12L, full)	~480MB	55.52%	61.01%	58.27%
L6_bottom (38K vocab)	98MB	54.70%	59.39%	57.05%
L6_compact_distilled	71.2MB	51.21%	58.33%	54.77%

Model Details

Property	Value
Teacher	paraphrase-multilingual-MiniLM-L12-v2
Vocab	~20,000 (frequency-based pruning, 97.4% coverage)
Size	71.2MB
Distilled	Yes

Quick Start

Distillation Details

Loss: 0.3 * MSE + 2.0 * (1 - CosineSimilarity)
Epochs: 10, LR: 5e-6, Batch: 64
Cosine-dominant loss preserves existing representations