Views
No views yet
| Tokenizer | English | Hindi | Code_Python | Code_Java | c++ | Math | |
|---|---|---|---|---|---|---|---|
| 0 | fhai50032/BPE-Multilingual-72k (72k) | 343695 reversible | 13684 reversible | 5367 reversible | 2269 reversible | 13072 reversible | 5026 reversible |
| 1 | HuggingFaceTB/SmolLM2-1.7B-Instruct (49k) | 337029 reversible | 44629 reversible | 5873 reversible | 2548 reversible | 15702 reversible | 6200 reversible |
| 2 | fhai50032/QTK-81K (81k) | 318160 reversible | 10767 reversible | 5192 reversible | 2241 reversible | 12730 reversible | 5120 reversible |
| 3 | deepseek-ai/DeepSeek-R1 (128k) | 338873 reversible | 22854 reversible | 5270 reversible | 2223 reversible | 12785 reversible | 5375 reversible |
| 4 | unsloth/phi-4 (100k) | 308645 reversible | 40456 reversible | 4844 reversible | 2110 reversible | 12088 reversible | 5573 reversible |
| 5 | deepseek-ai/DeepSeek-R1-Distill-Llama-8B (128k) | 308511 reversible | 21109 reversible | 4843 reversible | 2110 reversible | 12088 reversible | 5573 reversible |
| 6 | unsloth/gemma-2-9b-it(256k) | 323334 reversible | 15915 reversible | 5986 reversible | 2568 reversible | 15806 reversible | 5870 reversible |
| 7 | krutrim-ai-labs/Krutrim-2-instruct (131k) | 343067 reversible | 15477 reversible | 5246 reversible | 2223 reversible | 12700 reversible | 5361 reversible |
| 8 | nvidia/Nemotron-4-Mini-Hindi-4B-Instruct (256k) | 332271 irreversible, word count differs (212812 vs 212811) | 14327 irreversible, word count differs (7570 vs 7561) | 6229 irreversible, word count differs (1787 vs 1781) | 2732 irreversible, word count differs (1013 vs 1002) | 16306 irreversible, word count differs (5016 vs 4964) | 6136 reversible |
| 9 | sarvamai/OpenHathi-7B-Hi-v0.1-Base (48k) | 370132 reversible | 15632 reversible | 6640 reversible | 2791 reversible | 16756 reversible | 6222 reversible |
| 10 | sarvamai/sarvam-2b-v0.5 (68k) | 385383 reversible | 11256 reversible | 7387 reversible | 3067 reversible | 17586 reversible | 6863 reversible |