A computational study of how similar Naga languages are to each other, measured using parallel Bible translations as a controlled corpus. 36 Naga and Naga-adjacent languages plus English — compared using character-level patterns to find shared vocabulary, cognates, and structural connections.
The goal is not to rank languages or tribes. It is to find the common threads — the shared words, the similar sounds, the patterns that connect communities across mountains and state borders.
Key Findings
1. Six similarity methods evaluated; three are primary
Method
Cohen's d (Full)
ARI (Full)
Cohen's d (NT)
ARI (NT)
Status
Jaccard (top-1000 trigrams)
1.64
0.65
1.57
0.55
✅ Best clustering; most robust across coverage
TF-IDF (2-4 char n-grams)
1.74
0.47
1.94
0.66
✅ Best pair ranking; improves on NT-only
Subword Vocabulary JSD
1.27
0.44
1.37
0.32
✅ Respectable complement
Character Trigram Frequency
1.11
0.36
1.12
0.35
⚠️ Marginal (most stable across coverage)
Character Frequency (unigrams)
0.66
0.30
0.63
0.29
❌ Fails for same-script languages
Glot500-m Embeddings
0.27
0.15
0.27
0.15
❌ Fails for parallel texts
Jaccard uses binary presence/absence of the top-1000 most frequent trigrams, computing overlap proportion (|intersection| / |union|). It outperforms TF-IDF for clustering on full Bibles (ARI 0.65 vs. 0.47) because TF-IDF's IDF weighting can accidentally suppress genuine branch-internal vocabulary if those n-grams also appear in a few other languages. Jaccard also eliminates corpus-size bias between full-Bible and NT-only translations via top-K truncation.
TF-IDF achieves the highest Cohen's d on both corpora and improves on NT-only (1.74 → 1.94) because uniform corpus sizes let IDF weighting work correctly (no longer distorted by languages with 5× more text). Subword JSD provides a complementary, fully automated signal. The three primary methods (Jaccard, TF-IDF, Subword JSD) produce consistent branch-level clusters; the other three serve as negative controls confirming what does NOT work.
2. Known language families cluster correctly
The TF-IDF, Jaccard, and Subword JSD dendrograms, t-SNE, and PCA all show:
Ao (Central Naga) and South Patkaian languages cluster together — Ao, Sangtam, Yimchunger near Chang, Phom, Konyak, Khiamniungan, Wancho
Angami-Pochuri forms a distinct cluster — Angami, Chokri, Pochuri, Poumei, Rengma N, Rengma S, Sumi
Monsang-Moyon pair is the most similar in the dataset — confirmed close relatives
English is correctly the most distant language from all Naga languages
3. Data coverage: 24 full Bibles, 10 NT-only, 2 partial, all 36 included
Coverage
Count
Languages
Full Bible (66 books, 1189 chapters)
24
ASII Lakdap (Tangsa), Anal, Angami, Ao, Chang, English, Hawa, Khiamniungan, Kom, Konyak, Liangmai, Maring, Phom, Pochuri, Poumei, Rengma N, Rengma S, Rongmei, Sangtam, Sumi, Tangkhul, Wancho, Yimchunger, Zeme
Chuyo (Tangsa) — 1 book, 16 chapters; included in all analyses with explicit low-volume flag
Data disparity note: Chuyo (Tangsa) and Nocte are no longer used as coverage thresholds. The fixed 27 New Testament books (260 chapters) serve as the uniform corpus threshold for all analyses. Chuyo contributes only Mark (16 chapters) — its low volume is explicitly flagged in every downstream analysis and report, but the language is retained in the dataset rather than excluded. Nocte, despite being partial (46 books), contributes substantial NT content and is fully included. See documentation/phase_06_word_extraction_methodology.md §10.4 and scripts/02g_extract_common_books.py for implementation.
A dedicated NT-only analysis pipeline controls for the corpus-size bias between full-Bible (24 languages, ~4–6M chars) and NT-only (10 languages, ~1–2M chars) translations. The pipeline uses three modular scripts mirroring the full-Bible workflow:
02e_clean_bible_nt.py — Extracts only NT content (MAT–REV, 260 chapters) from raw Bible files via chapter-marker detection (### MAT 1 … ### REV 22), then applies the identical cleaning pipeline from 02d_clean_bible_texts.py. Outputs to data/bibles_cleaned_nt/ (local only, .gitignore'd).
03_similarity_methods_nt.py — Loads cleaned NT texts, computes all 6 methods (TF-IDF, CharFreq, Embeddings optional, Trigram, Jaccard, Subword JSD optional), evaluates each with Cohen's d + ARI against Glottolog classifications, and saves similarity matrices to results/nt_only/.
04_visualization_nt.py — Reads NT-only similarity matrices and generates 4 chart types per method (dendrogram, heatmap, t-SNE, PCA) to results/nt_only/visualizations/, color-coded by linguistic branch.
Key NT-only findings:
TF-IDF improves most (Cohen's d 1.74 → 1.94, ARI 0.47 → 0.66) — uniform corpus sizes eliminate IDF distortion
Jaccard degrades slightly but remains robust (Cohen's d 1.64 → 1.57, ARI 0.65 → 0.55)
Trigram is most stable (barely changes) — raw distributional counts are less affected by corpus size
Methods D/E/F (Trigram, Jaccard, Subword JSD) are the most robust across coverage levels, as designed
5. Character frequency fails for same-script languages
Character frequency (single letter counts) cannot distinguish Naga languages from each other or from English — all Latin-script languages look nearly identical at the individual character level. Cohen's d = 0.66 (small), ARI = 0.30 (poor). Published literature confirms: unigrams fail for same-script languages; bigrams and trigrams are needed (Singh 2006, Cavnar & Trenkle 1994).
6. Sentence embeddings fail for parallel texts
Glot500-m embeddings (cis-lmu/glot500-base, layer 8) produce similarity > 0.92 for every pair — including English vs. Naga languages. Three factors combine: (a) multilingual models are trained to align parallel content, so same Bible verses in different languages map to the same point; (b) most Naga languages are not in Glot500's 511-language training set, causing the model to fall back to character-level tokenization that treats all Latin-script text similarly; (c) the inter-language variation among 34 closely related Tibeto-Burman languages is too small for the model's embedding space to resolve. Cohen's d = 0.27 (small), ARI = 0.15 (poor).
7. Five classification errors were corrected
Initial branch assignments were verified against Glottolog 5.x, correcting five errors:
Chang, Phom: moved from Ao to South Patkaian (Glottolog: Chang-Phom-Konyak clade)
Maram: moved from Angami-Pochuri to Zemeic (Glottolog: mara1379)
Nocte, Tutsa: moved from Konyak to North Patkaian (Morey 2019: Tangsa-Nocte)
Kom: moved from standalone to Pakan group (Glottolog: NW Kuki-Chin, Kolhrengic)
8. Text cleaning improved results significantly (cleaned vs. uncleaned)
A systematic cleaning pass removed chapter markers, verse numbers, cross-references, inline footnotes, and English book titles from all raw Bible texts. The cleaning improved classification quality measurably:
Metric
First Run (Uncleaned, Wrong Labels)
Cleaned + Corrected Labels
Interpretation
Cohen's d
1.03
1.74
Better same-branch vs. different-branch separation
Adjusted Rand Index
0.53
0.47
Clustering matches known linguistics
Within-group avg similarity
0.3290
0.3445
Same-branch pairs are genuinely more similar
Cleaning removed a "background similarity" of ~0.05–0.10 that was uniform across all pairs (shared cross-references, verse numbers, book abbreviations). This increased the effective dynamic range and sharpened the true linguistic signal.
9. Annotation density varies dramatically — Wancho is an extreme outlier (30.6%)
The cleaning script now breaks removed content into six categories: chapter markers, hash footnotes, parenthetical cross-references, English titles, digits, and annot_other (non-standard inline annotations not matching typical patterns). Key findings:
Language
Total Removed
Density
annot_other
% other
Source
Wancho
1,475,983
30.6%
1,344,834
91%
Inline translator notes, Hebrew glosses, alternative readings
Maring
1,082,002
18.2%
948,767
88%
Extensive non-standard markup
English (NRSVUE)
931,472
19.6%
609,947
65%
Section headings, footnote text
Rengma (N)
498,424
11.9%
377,783
76%
Non-standard markup
Liangmai
495,347
11.0%
373,302
75%
Non-standard markup
Median (all Naga)
—
~3.1%
—
~40%
Typical range 2–4%
Wancho at 30.6% is an extreme outlier — 10× the median. Crucially, 91% of Wancho's removed content is annot_other — inline translator notes explaining Hebrew words, alternative renderings, geographic commentary, and other metalinguistic content. The cleaning script catches these via whitespace collapse and line stripping. Wancho's cleaned corpus (530,803 words, 3.3 MB) remains substantial and fully usable.
10. Anal–Kom investigation: ~0.10 similarity was artifactual (shared digitization pipeline)
Anal–Kom (both Pakan) dropped from 0.454 to 0.353 after cleaning — a 0.101 decrease, the largest within-branch drop. An annotation pattern investigation (scripts/investigate_anal_kom.py) found:
Finding
Value
Meaning
Footnote signature Jaccard
1.000
Identical footnote/cross-reference patterns across all 1,189 chapters
Annotation word overlap
0.481
13 shared English book abbreviations
Hash line density
0.891
Near-identical footnote density
Verdict: ~0.10 of the uncleaned Anal–Kom similarity was artifactual, caused by shared annotation formatting from the same bible.com digitization pipeline. The cleaned score of 0.353 is the accurate linguistic similarity.
11. Word extraction with PMI and cognate detection validates core vocabulary
The scripts/05_extract_similar_words.py script produces two complementary outputs:
PMI translation table (~1200 English concepts × 36 Naga languages, v5.0): Maps English words to their best Naga equivalents using Pointwise Mutual Information on real chapter alignment (MAT_1 ↔ MAT_1 across all languages), not coarse pseudo-chapters. v5.0 improvements: (a) Top-3 PMI candidates per English word with select_best_candidate() heuristic that prefers shorter base forms over grammatical clitics — eliminating false positives like "johnwui" (possessive "John's") for "dismissed" and "khorumshimli" (locative "at the temple") for "temple"; (b) MIN_PMI raised from 1.5 → 2.5 (~5.7× more likely than random) for higher-quality translations; (c) Proper nouns excluded from English anchor list (auto-detected via capitalization heuristic) to prevent named-entity inflation; (d) Dynamic thresholds scale min_cooccur and min_shared_chapters by corpus size — full Bible requires 15+ co-occurrences, Chuyo's 16 chapters only 3; (e) Table expanded from 500 to 1200 English words for richer cross-language comparison. The chapter-aligned pipeline (02f → 02g) replaces the old 800-word chunk approximation with exact chapter keys. Example: English "water" → Ao "anü", Angami "anü", Sangtam "anu", Konyak "ani", Tangkhul "tara̱" — revealing Proto-Naga *anu in Central/South Naga and independent innovation in Tangkhul-Maring.
Cognate pairs (character 2-gram Jaccard): Detects words that look/sound similar across languages without dictionaries. Example: Ao "anü" ↔ Sangtam "anu" (Jaccard 0.333) — detected as potential cognate despite vowel shift (ü → u). v5.0: Top-20 cognate display expanded to 10+ language columns for richer cross-linguistic inspection.
Cognate stability comparison: The script runs independently on both full-Bible and NT-only corpora, then compares:
Shared pairs → robust core vocabulary (present in both testaments)
Full-only pairs → likely OT-specific (genealogies, tribal names, law terms)
Five dialect pairs are flagged as OT-inflated: Rengma N↔S, Sangtam↔Yimchunger, Monsang↔Moyon, Konyak↔Phom, Chang↔Phom. Their full-Bible cognate counts overstate closeness by 10–20 pairs due to shared OT content. For studying core inherited vocabulary, NT-only counts are more reliable.
See documentation/phase_06_word_extraction_methodology.md and documentation/naga_languages_comprehensive_report.md Section 7 for full methodology, worked examples, and OT-inflation mechanisms.
12. NT-only resolves cross-branch anomalies and identifies OT-inflated dialect pairs
The NT-only experiment was critical for validating findings:
This 0.10 drop was driven by shared structural metadata (book abbreviations, cross-references, verse numbering) concentrated in the larger OT corpus. There is no evidence of Anal–Tangkhul contact from the computational data.
Five dialect pairs are OT-inflated (full-Bible similarity overstates closeness):
Language Pair
Full Drop
Likely Driver
Rengma (N) ↔ Rengma (S)
−0.152
OT genealogies, tribal histories
Konyak ↔ Phom
−0.137
Chang-Phom-Konyak tribal names in OT
Sangtam ↔ Yimchunger
−0.136
Shared OT narrative vocabulary
Monsang ↔ Moyon
−0.135
OT-specific domain overlap
Chang ↔ Phom
−0.133
Chang-Phom-Konyak genealogical terms
Pakan branch is most robust across coverage levels (avg drop only −0.025). South Patkaian is most OT-dependent (avg drop −0.068), driven by branch-specific vocabulary in OT genealogies, laws, and tribal names.
Languages Studied (36)
Branch classifications verified against Glottolog 5.x (Hammarstrom et al., 2024).
Sub-Branch
Languages
Count
Ao (Central Naga)
Ao, Sangtam, Yimchunger
3
Angami-Pochuri
Angami, Chokri, Pochuri, Poumei, Rengma N, Rengma S, Sumi
7
South Patkaian
Chang, Phom, Konyak, Wancho, Khiamniungan
5
North Patkaian
ASII Lakdap (Tangsa), Nocte, Tutsa, Hawa, Muklom
5
Tangkhul-Maring
Maring, Tangkhul
2
Zemeic
Liangmai, Rongmei, Zeme, Maram
4
Pakan
Anal, Chothe, Kharam, Lamkang, Monsang, Moyon, Tarao, Kom
8
Creole
Nagamese
1
Reference
English (NRSVUE)
1
Some major tribes — Lotha, Mao, Khezha, Inpui, Khoibu — are not yet included because their BSI-published Bible translations are not available in digital form online. See documentation/languages_included.md.
Methodology
Data collection — 37 Bible translations from bible.com, chapter by chapter, with verification
Text cleaning — Remove chapter markers, cross-references, footnotes, verse numbers, English titles. Per-category breakdown: chapter_markers, hash_footnotes, paren_xrefs, english_titles, digits_numbers, annot_other (see scripts/02d_clean_bible_texts.py)
Chapter-aligned cleaning — Preserve real chapter keys (GEN_1, MAT_1, REV_22) for exact alignment in PMI (see scripts/02f_clean_bible_chapter_aligned.py)
Uniform-corpus extraction — Extract the 27 NT books from all languages with fixed threshold; Chuyo (16 chapters) and partial languages (Nocte, Lamkang) are included and explicitly flagged (see scripts/02g_extract_common_books.py)
Feature extraction — Six methods: character n-gram TF-IDF, character frequency, Glot500-m sentence embeddings, character trigram frequency, Jaccard (top-1000 trigrams), Subword Vocabulary JSD
Cosine similarity — Pairwise similarity matrices across all language pairs
Evaluation — Cohen's d (separation power), ARI (clustering agreement with known classifications)
Visualization — Dendrograms, heatmaps, t-SNE, PCA — all color-coded by linguistic branch
Word extraction — Cross-language cognate identification via chapter-aligned PMI (real chapter keys, not pseudo-chunks); supports both full-Bible and NT-only corpora with stability comparison (see scripts/05_extract_similar_words.py)
Quality control — Flag anomalous pair shifts after cleaning; investigate with annotation pattern analysis (scripts/investigate_anal_kom.py)
NT-only controlled experiment — Extract identical NT content for all languages to control corpus-size bias via modular 3-script pipeline (02e → 03_nt → 04_nt); word extraction (05) compares full vs. NT cognate stability
Bible texts were obtained from bible.com (YouVersion). All translations are copyright of the Bible Society of India (BSI) and respective translation teams. The original Bible texts are NOT redistributed in this repository — only derived analytical outputs (similarity scores, visualizations, word tables) are shared. Raw texts in data/bibles/, cleaned texts in data/bibles_cleaned/, and NT-only cleaned texts in data/bibles_cleaned_nt/ are .gitignore'd and must remain local-only. We are grateful to BSI and bible.com for making these translations accessible.
References
Burling, R. (2003). The Tibeto-Burman languages of NE India. In The Sino-Tibetan Languages. Routledge.
Cavnar, W.B. & Trenkle, J.M. (1994). N-gram based text categorization. Proceedings of SDAIR-94.
