Views
No views yet
dag), a Mabia (Gur) language of northern Ghana. Part of a ground-up Dagbani NLP
research program: corpus, tokenizer, benchmarks, and models built from first
principles. Source: https://github.com/pious2847/Dagbani-NLP_V2| Metric | Value |
|---|---|
| Bits/token | 5.3492 |
| Perplexity | 40.76 |
| Bits/char | 1.5015 |
| Parameters | 49.9M |
| Tokenizer | dag_unigram_16k (SentencePiece unigram, byte fallback) |
src/lm/train_lm.py
in the source repo for the DagbaniGPT class definition and loading example.