A QLoRA fine-tune of sarvamai/sarvam-1 that converts spoken-Hindi self-descriptions into a strict 20-field CitizenProfile JSON schema. Designed for form-filling assistants at government service centers, banking KYC, and welfare-scheme applications, where applicants describe themselves in natural Hindi and the downstream form needs structured fields.

Headline (290-entry held-out test set):

Metric	Value
Parse rate	290 / 290 (100.0%)
Schema validation rate	290 / 290 (100.0%)
Field accuracy (strict exact-match)	5692 / 5800 (98.1%)
`synthetic_dense` gold-NN field acc (target distribution)	98.3%

Source code, data preparation pipeline, and training/eval scripts: github.com/subidhkhanal/hindi-form-agent.

Quick start

python

import json
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

BASE = "sarvamai/sarvam-1"
ADAPTER = "subidhkhanal/sarvam-1-hindi-citizen-profile-lora"

tokenizer = AutoTokenizer.from_pretrained(BASE)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

base = AutoModelForCausalLM.from_pretrained(
    BASE,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True, bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,
    ),
    device_map="auto",
)
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

# Exact prompt format used at training time
INPUT_DELIMITER = "हिंदी पाठ:"
OUTPUT_DELIMITER = "संरचित JSON:"

hindi_text = (
    "मेरा नाम रामलाल है। मेरी उम्र चालीस साल है। "
    "मैं बिहार के मधुबनी जिले में रहता हूं। खेत मजदूरी का काम करता हूं, "
    "महीने में लगभग पच्चीस हजार रुपये कमाता हूं। आधार कार्ड और बैंक खाता है।"
)
prompt = f"{INPUT_DELIMITER}\n{hindi_text}\n\n{OUTPUT_DELIMITER}\n"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(
        **inputs, max_new_tokens=512, do_sample=False,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )
gen = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

# Model may emit trailing junk after the JSON; raw_decode handles it
predicted, _ = json.JSONDecoder().raw_decode(gen)
print(json.dumps(predicted, ensure_ascii=False, indent=2))

Expected output:

json

{
  "full_name": "रामलाल",
  "age": 40,
  "gender": "male",
  "marital_status": null,
  "district": "मधुबनी",
  "state": "बिहार",
  "occupation": "खेत मजदूर",
  "monthly_income_inr": 25000,
  "has_aadhaar": true,
  "has_bank_account": true,
  "bank_name": null
}

(plus null for the remaining fields)

Output schema

20-field Pydantic CitizenProfile (full definition in repo). All fields are Optional — the model is trained to emit null for fields the input does not mention, rather than hallucinate.

Closed-set (Literal) fields use canonical labels:

gender ∈ {"male", "female", "other"}
marital_status ∈ {"single", "married", "widowed", "divorced"}
caste_category ∈ {"general", "obc", "sc", "st"}
has_aadhaar, has_pan, has_voter_id, has_ration_card, has_bank_account ∈ {true, false}

Government ID fields are booleans only — actual ID numbers are never captured in training data (privacy by design).

Training

Knob	Value
Base model	`sarvamai/sarvam-1` (2.5 B params)
Trainable params	6,422,528 (0.25%)
Method	QLoRA: 4-bit NF4 + LoRA
LoRA `r` / `α` / dropout	16 / 32 / 0.05
LoRA target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`
Epochs	3
Effective batch	8 (2 × grad-accum 4)
LR	2e-4 (cosine, 5% warmup)
Max seq length	1024
Mixed precision	fp16 (T4 = Turing, no bf16)
Optimizer	paged AdamW 8-bit
Loss masking	Completion-only (`DataCollatorForCompletionOnlyLM`)
Hardware	Kaggle T4 single GPU, ~58 min

Training data

2,270 train / 290 val / 290 test entries from four sources:

Source	n	Description
`handcrafted_seed`	10	Dense first-person anchor examples
`hiner` (subsampled)	2,000	AI4Bharat's HiNER-collapsed — real-Hindi distributional signal, sparse
`synthetic_dense`	800	First-person Devanagari citizen profiles generated against a persona matrix
`hardcases`	40	Stress tests: disfluency, negation, approximate values, mixed scripts, etc.

Full data prep methodology and the persona/domain references live in data_prep/ in the GitHub repo.

Evaluation

Strict exact-match field accuracy on a held-out test set (290 entries, never seen during training). Test set is itself produced by a deterministic random.seed(42) split, fully reproducible.

By source

Source	n	Parse	Schema	Field acc	Gold-NN field acc
`synthetic_dense` (target distribution)	80	100.0%	100.0%	98.7%	98.3%
`hardcases`	10	100.0%	100.0%	95.5%	93.4%
`hiner`	200	100.0%	100.0%	98.0%	80.8%

By field (gold-non-null accuracy, sorted)

Field	Overall	Gold-NN	n (gold-NN)
`marital_status`	100.0%	100.0%	80
`father_or_husband_name`	100.0%	100.0%	59
`number_of_dependents`	99.3%	100.0%	3
`pincode`	100.0%	100.0%	10
`caste_category`	100.0%	100.0%	78
`religion`	100.0%	100.0%	80
`has_aadhaar`	100.0%	100.0%	81
`has_pan`	100.0%	100.0%	9
`has_voter_id`	100.0%	100.0%	9
`has_ration_card`	100.0%	100.0%	26
`has_bank_account`	100.0%	100.0%	51
`bank_name`	100.0%	100.0%	25
`age`	99.7%	98.9%	90
`gender`	99.7%	98.9%	90
`state`	99.0%	97.8%	90
`village_or_town`	99.0%	95.1%	41
`monthly_income_inr`	98.6%	94.3%	70
`full_name`	88.6%	88.6%	290
`occupation`	94.8%	86.0%	86
`district`	84.1%	82.4%	250

Highlights:

Every closed-set (Literal-typed) field at 100% gold-NN — the model canonicalizes free Hindi text to enum values reliably (अनुसूचित जाति → sc, मुस्लिम → इस्लाम, विधवा → widowed).
bank_name at 100% (25/25) — the conditional rule (extract iff the speaker named the bank, else null) generalized perfectly.
Devanagari number → int: चौवालीस → 44, इकहत्तर → 71, बावन → 52, etc.

Weak spots (district, full_name, occupation) are bottlenecked by:

HiNER label noise (party names mistagged as PER, locations truncated).
Strict exact-match: predicting "स्टेशनरी दुकानदार" when gold is "दुकानदार" counts as wrong even though both are correct. Real semantic accuracy is higher.

Intended use

Demo / research / educational. Drop-in inference component for Hindi form-filling assistants, voice-to-form pipelines, and structured-extraction research.

Not for production use without further validation. Government ID-related fields are booleans only (never extracts ID numbers — that's a deliberate privacy boundary, not an oversight).

Limitations

Strict exact-match underreports. Predictions semantically equivalent to gold (canonical spelling variants, more-specific occupations) score as wrong. A normalized or semantic-match metric would raise the field-accuracy number meaningfully without changing the underlying capability.
HiNER label noise. The 2,000 HiNER training entries provide real-Hindi distributional signal but carry inherited labeling quirks (party names as full_name, multi-word locations truncated). The data-prep stoplist filters the worst cases; some residual noise reaches the model.
Generation may over-run past the JSON. The model occasionally emits duplicate JSON blocks or Hindi narrative after the closing }. Always parse with json.JSONDecoder().raw_decode() (takes the first valid object) or use the StoppingCriteria provided in the repo's evaluate.py.
First-person dense inputs are the sweet spot. Sparse third-person news sentences (HiNER-style inputs) still produce well-formed JSON but value accuracy drops — that's a property of HiNER labels, not a model failure.

License

The adapter inherits the base model's license: Sarvam non-commercial license. Not for commercial use.

Citation / acknowledgement

Built on top of Sarvam-1 by Sarvam AI, and the HiNER-collapsed NER dataset by AI4Bharat / CFILT. See the GitHub repo for the full data prep acknowledgements and per-source methodology.

Full source, data, and reproducible training scripts: github.com/subidhkhanal/hindi-form-agent.