You can deploy this model and start earning money today!

sanganaka/
qwen3-4B-sa-hi-think

Name: qwen3-4B-sa-hi-think
Availability: InStock

text-generation-inference

endpoints_compatible

Views

No views yet

🔥 Overview

qwen3-sa-hi-think is a Sanskrit → Hindi Machine Translation model fine-tuned on top of Qwen3-4B, enhanced with Thinking Tokens to enable structured latent reasoning during translation. The model was trained using LoRA (rank 32) on top of the thinking-enabled variant of Qwen3, allowing it to internally perform step-by-step reasoning before producing the final Hindi output.

Because of its reasoning ability, the model excels in:

Classical Sanskrit poetry (ślokas, Anuṣṭubh meter)
Epic and Purāṇic Sanskrit
Formal Sanskrit prose Semantic interpretation of compounds (samāsa), metaphors, and sandhi-heavy verse leading to significantly more accurate, context-aware Hindi translations.

This model is developed by Pretam Ray (IIT Kharagpur) as part of ongoing research on Indic low-resource MT, explainable translation, and poetry-aware reasoning models.

🧠 Model Architecture

Base Model: Qwen3-4B-Thinking variant (supports reasoning traces via special thinking tokens)
Reasoning Tokens Used: yes (<think> tokens during SFT)
Fine-tuning Method: LoRA
LoRA Rank: 32
LoRA Alpha: alpha=16
Precision: FP16/BF16 mixed precision
Framework: LLaMA-Factory / HF PEFT
Context Length: Inherits Qwen3-4B (32k tokens)
Tokenizer: Qwen3 tokenizer (same as base)

🔍 What are Thinking Tokens?

The underlying Qwen3-4B-Thinking model uses special reasoning tokens (often <think> ... </think>) during training. These guide the model to produce hidden chain-of-thought reasoning internally, improving:

meaning disambiguation
correct handling of long compounds
proper rearrangement of Sanskrit → Hindi syntax
preservation of metaphoric sense

This model suppresses chain-of-thought in final outputs but uses it internally for better translation.

📚 Dataset Composition

Curated Sanskrit → Hindi aligned pairs
Includes:
- Anuṣṭubh epic verse (Ramayana, Mahabharata etc.)
- Traditional ślokas
- Sanskrit prose passages
JSONL format
Training Size: <fill>
Validation Size: <fill>

🔧 Training Configuration

Base Model: Qwen3-4B-Thinking
LoRA Rank: 32
LoRA Dropout: 0.05
LoRA Target Modules: q_proj, k_proj, v_proj, o_proj
Batch Size: <fill>
Learning Rate: <fill>
Epochs: <fill>
Optimizer: AdamW
Scheduler: cosine
Framework: LLaMA-Factory
Reasoning Tokens: Enabled (+latent CoT)

💡 Example Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "sanganaka/qwen3-4B-sa-hi-think",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("pretamray/qwen3-sa-hi-think")

text = "कोन्वस्मिन्साम्प्रतं लोके गुणवान्कश्च वीर्यवान्"

inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=120)
print(tokenizer.decode(out[0], skip_special_tokens=True))