Vidhik AI is a highly optimized, domain-specific Small Language Model (SLM) engineered for the Indian Judiciary and MSME sector. Fine-tuned on a 1B parameter base, it specializes in drafting formal legal notices (e.g., MSMED Act delayed payments), analyzing case law, and navigating complex Indian officialese ("Babu-speak").
Built with a focus on Edge Compute, this model is designed to run locally on highly constrained hardware (like a 4GB GTX 1050) while retaining the ability to process massive context windows using Google TurboQuant.
Developer: Gaurav / Bhishaj Technologies
Base Model: Llama-3.2-1B-Instruct
Language(s): English, Hindi (Indic Legal Terminology)
License: Llama 3.2 Community License
🛠️ Training & MLOps Architecture
To bypass local hardware constraints (4GB VRAM), the model was trained using a hybrid cloud-edge pipeline:
1. Data Engineering
Corpus: Curated and filtered Indian Legal QA datasets (Techmaestro369/indian-legal-texts-finetuning) and multilingual judiciary data (coild-aikosh/Judiciary_v2).
Formatting: Converted raw unstructured legal texts into strict Alpaca/ShareGPT instruction formats for deterministic instruction following.
Optimization: Utilized Unsloth for a 70% VRAM reduction during fine-tuning, accelerating the training process by 2x.
Methodology: Parameter-Efficient Fine-Tuning (PEFT) using QLoRA.
3. Guardrails & Alignment
Trained with strict negative stop-sequences and deterministic decoding parameters (Temperature = 0.0) to cure the base model of MCQ-loop hallucinations.
Aligned to a "Senior Advocate, Supreme Court of India" persona for formal, zero-fluff document generation.
⚡ Edge Deployment & Google TurboQuant (2026)
This model is specifically compiled to run on legacy/constrained hardware.
By utilizing Google TurboQuant, the model compresses the KV-cache to 3-bits during runtime. This allows for 128k context windows (essential for processing long Indian government gazettes and supreme court rulings) without triggering OOM (Out of Memory) crashes on a 4GB GPU, maintaining a throughput of ~24.5 tokens/sec.
💻 Usage: Running Locally (TurboQuant Enabled)
To achieve 6x KV-cache compression on your local machine:
python
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3from turboquant import TurboQuantCache
45repo_id ="Bhishaj/Vidhik-Llama-1B-GGU"6tokenizer = AutoTokenizer.from_pretrained(repo_id)7model = AutoModelForCausalLM.from_pretrained(repo_id, device_map="cuda")89# Initialize TurboQuant 4-bit Cache for 4GB VRAM support10tq_cache = TurboQuantCache(bits=4, compute_device="cuda")1112prompt ="TASK: Draft a formal legal notice for my client 'M/s Vidhik Electronics' under MSMED Act Sections 15 & 16."13inputs = tokenizer(prompt, return_tensors="pt").to("cuda")1415# Generate with Compressed Context16with torch.no_grad():17 outputs = model.generate(18**inputs,19 past_key_values=tq_cache,# Injecting the TurboQuant cache20 max_new_tokens=512,21 temperature=0.022)2324print(tokenizer.decode(outputs[0], skip_special_tokens=True))
