Views
No views yet
sinlib library—a comprehensive Sinhala NLP toolkit.sinlib.spellcheck.TypoDetector:bigru_detector.pt: A bidirectional GRU sequence labeling model (BiGRUSequenceLabeler) trained to detect spelling errors and character substitutions at the akshara level.bigru_corrector.pt: A sequence-to-sequence bidirectional GRU encoder-decoder model with Attention (BiGRUSeq2Seq) that performs generative character/akshara corrections.akshara_vocab.json: Vocabulary mappings mapping Sinhala phonological units (aksharas) and basic punctuation to token IDs for the neural models.akshara_ngram.json: Stored counts and vocabularies used by the statistical Trigram model (AksharaNGram) for likelihood evaluation.news_unigrams.json & news_bigrams.json: Pre-calculated unigram and bigram word-level frequencies extracted from Sinhala news corpora, used for context-aware candidate re-ranking (Stupid Backoff).dictionary.npy: The canonical baseline dictionary containing ~61K valid Sinhala words.ngram_probs.npy: N-gram probabilities used by the default PreTrainedTokenizer fallback checks.sinlib package are also included in this repository.sinlib Python package.pip install sinlib1from sinlib.spellcheck import TypoDetector
2
3# Automatically downloads and caches the model files from this HF repository
4detector = TypoDetector.from_pretrained("Ransaka/sinlib")
5
6# Run spellcheck (with punctuation preservation & unicode normalization)
7sentence = "කොළඹ වරායේ සිට බස්නහිර දෙසින් නාවික සැතපුම් 17ක් පමණ දුරන්."
8corrected = detector(sentence)
9
10print(corrected)
11# Output: "කොළඹ වරායේ සිට බස්නාහිර දෙසින් නාවික සැතපුම් 17ක් පමණ දුරින්."sinlib falls back automatically to the statistical N-Gram model (akshara_ngram.json) to perform spelling checks.1# Check word suspicion level
2is_typo = detector.is_word_suspicious("පසලට") # True