Śruti — Hindu Philosophy Assistant (Qwen2.5-1.5B, LoRA fine-tune)

What's included

Training

• Method: LoRA (r=16, alpha=32, dropout=0.05) on attention + MLP projections, trained with transformers.Trainer (no trl).
• Data: ~105 records — 85 single-turn philosophy Q&A (each with a cited source, woven into the trained answer), 10 greeting/small-talk examples, 10 multi-turn conversations (2 exchanges each). Multi-turn examples are masked so loss is computed on every assistant turn, not just the last one.
• Checkpoint selection: evaluated every 5 steps against a held-out validation split; the checkpoint with the lowest validation loss was kept (not the final epoch), specifically to avoid shipping an overfit model. Validation loss bottomed out at step 25 of 165 total steps — training continued well past that point purely to give the selection process more candidates to compare.
• Trained on a Kaggle T4 GPU.

Known limitations — read before trusting factual output

• The greeting/conversational tone is a genuine improvement over just prompting the base model — it responds naturally to "hi"/"hello" instead of forcing every reply into an encyclopedia-style answer.
• Response conciseness is also genuinely improved — the fine-tune reliably produces complete, well-formed answers, where the base model often trails off mid-sentence within the same token budget.
• Factual reliability is not meaningfully better than the base model. On questions outside the training set's exact coverage, this model — like the base model — will confidently fabricate plausible-sounding Sanskrit quotes, verse numbers, and source attributions. Examples found in testing: misattributing which Upaniṣad or Gītā chapter a fact comes from, inventing a founder's name for a philosophical school, and occasionally blending Sāṅkhya's dualist framework with Vedāntic non-dualism.
• The trained-in citation habit (*(Source: ...)* after most answers) can make hallucinated claims look more authoritative than they are. A confidently-cited wrong answer is easy to mistake for a correct one — verify anything load-bearing against a primary source.

Usage