A LoRA fine-tune of IndicTrans2 (distilled 200M, en-indic) for English → Kashmiri translation, with Kashmiri output in the Perso-Arabic script (kas_Arab).

Model description

IndicTrans2 is AI4Bharat's open-source neural MT model covering English and 22 scheduled Indian languages. This checkpoint takes the distilled 200M en→indic variant and adapts it specifically to Kashmiri using LoRA adapters trained on the kas_Arab portion of the BPCC parallel corpus, merged back into the base weights.

Base model: ai4bharat/indictrans2-en-indic-dist-200M
Fine-tuning method: LoRA (PEFT), merged into base weights
Source language: English (eng_Latn)
Target language: Kashmiri, Perso-Arabic script (kas_Arab)
License: Apache 2.0

Intended use

Sentence-level English → Kashmiri translation. Intended for research, low-resource MT experimentation, and applications needing Kashmiri text output in the Perso-Arabic script. Not intended for high-stakes use (legal, medical) without human review — see Limitations below.

How to use

This model requires IndicTransToolkit for the pre/post-processing IndicTrans2 expects (language tags, script normalization).

bash

pip install transformers accelerate sentencepiece
git clone https://github.com/VarunGumma/IndicTransToolkit.git
pip install --editable ./IndicTransToolkit

python

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from IndicTransToolkit.IndicTransToolkit import IndicProcessor

MODEL_ID = "bazik-0/indictrans2-en-kas-finetuned-200M-5"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_ID, trust_remote_code=True, torch_dtype=torch.float32)
model.eval()

ip = IndicProcessor(inference=True)

sentences = ["Hello, how are you?", "The weather is very cold today."]
processed = ip.preprocess_batch(sentences, src_lang="eng_Latn", tgt_lang="kas_Arab")

inputs = tokenizer(processed, return_tensors="pt", padding=True, truncation=True, max_length=128)

with torch.no_grad():
    outputs = model.generate(**inputs, max_length=128, num_beams=5, early_stopping=True)

decoded = tokenizer.batch_decode(outputs, skip_special_tokens=True)
translations = ip.postprocess_batch(decoded, lang="kas_Arab")

print(translations)

Example outputs

English	Kashmiri
She was a true visionary.	سہٕ ٲس اکھ حقیقی نظریہ ساز۔
I go to my school daily.	بہٕ چھس پرٛؠتھ دۄہ پننس سکولس منٛز گژھان۔
Nobody know whence he has come.	کٲنہہ تہ چھنہٕ زانان سہ کتہ پؠٹھ چھ آمت۔

Training details

Training data

Fine-tuned on ai4bharat/BPCC (bpcc-seed-latest config), kas_Arab subset — 90,000 training pairs and 8,929 held-out eval pairs, shuffled with seed 42.

Preprocessing

Source and target text is passed through IndicTransToolkit's IndicProcessor for IndicTrans2's language-tag formatting and script normalization before tokenization. Sequences are truncated to a max length of 128 tokens.

Training procedure

LoRA adapters (via PEFT) trained on the attention projections with the base model frozen, then merged into the base weights for release:

rank r=16, alpha=32, dropout 0.05
target modules: q_proj, v_proj

Hyperparameters:

Param	Value
Epochs	8
Per-device batch size	16
Gradient accumulation	8 (effective batch 128)
Learning rate	5e-5
Weight decay	0.01
Warmup ratio	0.03
Precision	bf16
Optimizer	AdamW
Max grad norm	1.0

Compute: single Kaggle T4 GPU, via 🤗 Seq2SeqTrainer.

Evaluation

Metric: BLEU (sacrebleu), computed on the held-out eval split.

Result: 6.07 BLEU on the target evaluation set.

Limitations

Kashmiri is a low-resource language in NLP — training data volume and diversity are limited compared to high-resource pairs, so translation quality varies notably by sentence complexity and domain.
Trained only on kas_Arab (Perso-Arabic script); it does not support Kashmiri written in Devanagari.
Not evaluated for bias, toxicity, or factual accuracy — apply human review for any use beyond research/experimentation.

Citation

If you use this model, please cite the base IndicTrans2 work:

bibtex

@article{gala2023indictrans2,
  title   = {IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages},
  author  = {Jay Gala and Pranjal A. Chitale and A K Raghavan and Varun Gumma and Sumanth Doddapaneni and Aswanth Kumar M and Janki Atul Nawale and Anupama Sujatha and Ratish Puduppully and Vivek Raghavan and Pratyush Kumar and Mitesh M. Khapra and Raj Dabre and Anoop Kunchukuttan},
  journal = {arXiv preprint arXiv:2305.16307},
  year    = {2023}
}

Model card contact

Open a discussion on this model's Hugging Face repo for questions.