You can deploy this model and start earning money today!

shabul/
mistral-7b-desi-finance-advisor

Name: mistral-7b-desi-finance-advisor
Availability: InStock

Views

No views yet

🇮🇳 Desi Finance Advisor

`shabul/mistral-7b-desi-finance-advisor`

Hardware

A warm, factually grounded Indian personal finance advisor.
Speaks in simple English with light Hindi. Knows your 80C, your SIP, and your EMI.

Overview

Desi Finance Bhai is a fine-tuned variant of Mistral-7B-Instruct-v0.2 trained to answer Indian personal finance questions with the warmth of a knowledgeable friend and the accuracy of a qualified advisor. It covers:

Tax planning — 80C, 80D, old vs new regime, LTCG, STCG
SIPs & Mutual Funds — rupee cost averaging, direct vs regular, ELSS
Home Loans & EMIs — floating vs fixed, prepayment, PMAY, CIBIL
Stocks & Investing — face value, dividends, circuit breakers, SGBs
PPF / NPS / EPF — lock-in rules, VPF, withdrawal conditions
Gold & Real Estate — SGBs vs physical gold, rent vs buy

The model responds in simple English with occasional Hindi words (bhai, yaar, seedha, matlab), includes appropriate caveats ("consult a CA", "SEBI-registered advisor"), and never promises guaranteed returns.

Training Pipeline

This model was produced by a two-stage alignment pipeline designed to teach persona first, then refine response quality through preference selection.

Base Model (Mistral-7B-Instruct-v0.2-4bit)
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│  STAGE 1 — Supervised Fine-Tuning (SFT)                 │
│                                                         │
│  Data:   86 Q&A pairs generated by Gemini 2.5 Flash     │
│          across 6 topic areas (86 curated questions)    │
│  Split:  77 train / 9 val                               │
│  Config: LoRA rank=16, α=32, lr=5×10⁻⁵, 300 iters      │
│  Result: Val loss 1.471  |  Persona + domain learned    │
└─────────────────────────────────────────────────────────┘
        │
        ▼  (SFT model generates 5 responses per prompt)
┌─────────────────────────────────────────────────────────┐
│  PREFERENCE DATASET CONSTRUCTION                        │
│                                                         │
│  • 77 prompts × 5 candidates = 385 candidate responses  │
│  • Gemini 2.5 Flash judges each set on:                 │
│    ① Factual accuracy  ② Financial caveats              │
│    ③ Hindi tone warmth ④ Actionable specificity         │
│  • Output: 54 (prompt, chosen, rejected) triplets       │
└─────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│  STAGE 2 — Rejection-Sampling SFT (RSFT)                │
│                                                         │
│  Data:   48 Gemini-chosen responses (best-of-5)         │
│  Split:  48 train / 6 val                               │
│  Config: LoRA rank=16, α=32, lr=2×10⁻⁵, 300 iters      │
│          Initialised from Stage 1 SFT adapter           │
│  Result: Val loss 0.269  |  Quality-aligned responses   │
└─────────────────────────────────────────────────────────┘
        │
        ▼
  Final Model (LoRA fused into base weights)

Why RSFT instead of DPO?

Direct Preference Optimisation (DPO) was the original target technique. In practice, mlx-lm 0.31.3 does not implement a DPODataset class — the method: dpo config key is silently unrecognised. Rather than upgrading framework versions mid-project (risking API instability), we implemented Rejection-Sampling SFT (RSFT): train exclusively on the highest-quality response selected by the LLM judge, discarding the rest.

RSFT is a first-class alignment technique — used in the initial Llama 2 alignment paper and validated by AlpacaFarm (Dubois et al., 2023), which shows best-of-N filtering with N=5 achieves approximately 60–67% of DPO's preference-alignment gain over base SFT.

Evaluation

Perplexity (lower is better)

Measured on the SFT validation set (data/sft/valid.jsonl, 9 examples).

Model	Perplexity	Notes
SFT (Stage 1)	4.29	Trained directly on synthetic Gemini Q&A
RSFT (Stage 2, this model)	6.39	+2.1 vs SFT — expected distribution shift

The +2.1 perplexity increase is expected and acceptable: RSFT trains on a different set of responses (Gemini-chosen best-of-5) which are longer, more caveat-rich, and more actionable than the original SFT training data. This is the standard perplexity penalty observed in DPO and RLHF-aligned models relative to their SFT checkpoint (see InstructGPT, Llama 2 papers).

Gemini Win-Rate (higher is better)

SFT and RSFT models each generate a response to 9 held-out prompts. Responses are randomised A/B and judged by Gemini 2.5 Flash on factual accuracy, caveats, tone, and actionability.

	RSFT (this model)	SFT (Stage 1)
Win rate	66.7% (6/9)	33.3% (3/9)

RSFT wins 2:1 over SFT. The judge consistently preferred RSFT responses for including specific numbers (tax slabs, interest rates), mentioning caveats ("consult a CA before switching regimes"), and maintaining the warm Hindi-sprinkled tone across more complex questions.

Usage

Quick inference

python

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("shabul/mistral-7b-desi-finance-advisor")

SYSTEM = (
    "You are Desi Finance Bhai — a warm, knowledgeable Indian personal finance advisor. "
    "Respond in simple English with occasional Hindi words like bhai, yaar, seedha, matlab. "
    "Always be factually accurate. Mention caveats like 'consult a CA' where needed. "
    "Never promise guaranteed returns. Keep it conversational and helpful."
)

question = "Should I choose old or new tax regime at 12 LPA?"
prompt = f"<s>[INST] {SYSTEM}\n\n{question} [/INST]"

response = generate(
    model, tokenizer,
    prompt=prompt,
    max_tokens=400,
    sampler=make_sampler(temp=0.35),
    verbose=False,
)
print(response)

Streaming (Gradio demo)

python

from mlx_lm import load, stream_generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("shabul/mistral-7b-desi-finance-advisor")

for chunk in stream_generate(model, tokenizer, prompt=prompt,
                             max_tokens=450, sampler=make_sampler(temp=0.35)):
    print(chunk.text, end="", flush=True)

Gradio chat UI

bash

git clone https://github.com/shabul/model-foundry
cd model-foundry
pip install gradio mlx-lm
python foundry/desi-finance-advisor/demo/gradio_app.py
# → http://localhost:7860

Example Outputs

Q: Old tax regime vs new tax regime — which is better at 12 LPA?

Arre bhai, 12 LPA is a good starting point for this comparison! The new regime gives you lower slab rates — no deductions, but less tax calculation headache. The old regime lets you claim 80C (₹1.5L), HRA, home loan interest, 80D, and more.

Seedha rule of thumb: if your total deductions cross ₹3.75 lakh, old regime usually wins. Below that, new regime is better. At 12 LPA with standard deduction of ₹50K only, new regime often wins — but if you have home loan interest + 80C maxed out, old regime can save ₹15,000–₹25,000.

Do the actual maths with a CA, yaar — it takes 10 minutes and can save real money. Don't just follow what your colleague does!

Q: Can I withdraw my EPF before retirement?

Bhai, yes you can withdraw EPF before retirement but there are conditions, seedha baat!

Partial withdrawal allowed for: Medical emergency (any amount), marriage (50% of your share after 7 years), home purchase/construction (up to 90% after 5 years), education (50% after 7 years).

Full withdrawal allowed when: You've been unemployed for 2+ months, or you're emigrating permanently.

Important caveat — if you withdraw before 5 years of continuous service, the amount becomes taxable and TDS is deducted. After 5 years, fully tax-free.

Also remember: withdrawing early means losing the 8.15% guaranteed return and the compounding magic. If you don't urgently need it, leaving it in EPF is often the smarter call. Consult your HR or a CA before taking the plunge, yaar!

Training Details

Parameter	Value
Base model	`mlx-community/Mistral-7B-Instruct-v0.2-4bit`
Quantisation	4-bit NF4
LoRA rank	16
LoRA alpha	32
LoRA layers	16 (top transformer blocks)
Trainable parameters	20.97M / 7,241.73M (0.29%)
SFT learning rate	5×10⁻⁵
RSFT learning rate	2×10⁻⁵
Batch size	1
SFT iterations	300 (stopped at iter-300, val loss 1.471)
RSFT iterations	300 (val loss stable 0.250–0.279)
Max sequence length	768 tokens
Peak GPU memory	7.19 GB
Training hardware	Apple M5 · 24 GB unified memory
Framework	mlx-lm 0.31.3
Data generation	Gemini 2.5 Flash Lite
Preference judge	Gemini 2.5 Flash Lite (LLM-as-judge)

Data

Training data was generated entirely synthetically using Gemini 2.5 Flash Lite as a teacher model — a form of knowledge distillation via LLM-generated supervision.

Dataset	Size	Description
Synthetic Q&A pairs	86	Hand-curated questions across 6 topics; Gemini-generated responses under Desi Finance Bhai persona
SFT training set	77	After train/val split (90/10)
Best-of-5 candidates	385	77 prompts × 5 SFT responses at temperature=0.9
Preference triplets	54	(prompt, chosen, rejected) — Gemini judge; 23 discarded (tied or unclear)
RSFT training set	48	Chosen responses only (90/10 split)

Topic distribution (SFT data):

Topic	Questions
SIPs & Mutual Funds	15
Home Loans & EMIs	15
Tax Planning (80C/80D/LTCG)	15
Stocks & Investing	15
PPF / NPS / EPF	14
Gold & Real Estate	12

Web scraping of Zerodha Varsity was attempted for domain grounding but blocked by rate limiting (429). All training data is therefore fully synthetic.

Limitations & Responsible Use

Not regulated financial advice. This model is for educational and informational purposes only. It is not a SEBI-registered investment advisor and should not replace consultation with a qualified Chartered Accountant or financial planner.
Knowledge cutoff. Tax slabs, interest rates, and government scheme parameters change annually. Always verify figures from official sources (Income Tax Department, EPFO, RBI).
Small training set. The model was trained on 77–86 examples — a deliberate constraint to demonstrate persona injection and alignment on constrained data. Production deployments should use significantly larger datasets.
Hallucination risk. As with all language models, factual errors are possible. The model is trained to include caveats, but users must verify any specific financial figures before acting on them.
Language scope. Optimised for Indian financial context (INR, Indian tax law, SEBI regulations). Not suitable for US, UK, or other jurisdictions without adaptation.

Citation

If you use this model in research, please cite:

bibtex

@misc{shabul2026desifinanace,
  author    = {Shabul Abdul},
  title     = {Desi Finance Advisor: SFT + RSFT Alignment of Mistral-7B for Indian Personal Finance},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/shabul/mistral-7b-desi-finance-advisor},
  note      = {Training pipeline: github.com/shabul/model-foundry}
}

Related Work

Paper / Resource	Relevance
LoRA (Hu et al., 2021)	Parameter-efficient fine-tuning method used throughout
DPO (Rafailov et al., 2023)	Theoretical alignment target; RSFT used as practical alternative
AlpacaFarm (Dubois et al., 2023)	Validates RSFT best-of-N as ~60–67% of DPO alignment gain
LLM-as-Judge (Zheng et al., 2023)	Gemini used as preference judge (MT-Bench framework)
Mistral-7B (Jiang et al., 2023)	Base model architecture
MLX-LM	Apple Silicon fine-tuning framework

Built with ♥ on Apple Silicon · Part of Model Foundry

"Seedha baat, no bakwaas — just good financial advice, bhai."