KiyanNet Finance Base 48M is a compact finance-domain causal language model published in standard Hugging Face Transformers format. It is intended as a lightweight baseline for finance and India-market language modeling, continued pretraining, and downstream fine-tuning.

This is a base model, not an instruction-tuned assistant. It should not be used to provide buy/sell calls, price targets, guaranteed outcomes, or personalized investment advice.

Load With Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "techkiyan/KiyanNet-Finance-Base-48M"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

prompt = "Reliance Industries reported"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=True,
    temperature=0.8,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

No custom Python files are required.

Files

This public repo intentionally contains only standard loading and fine-tuning files:

config.json
generation_config.json
model.safetensors
tokenizer.json
tokenizer_config.json
README.md

Training scripts, logs, token shards, raw datasets, duplicate checkpoints, and custom model-code files are intentionally excluded.

Model Details

Item	Value
Public loader	`AutoModelForCausalLM`
Model class	`GPT2LMHeadModel`
Tokenizer loader	`AutoTokenizer`
Tokenizer type	Fast BPE tokenizer converted from original SentencePiece model
Original trained parameters	48,376,320
HF model parameters	48,433,152
Context length	1,024
Vocabulary size	32,000
Layers	10
Attention heads	8
Hidden size	512
Training tokens processed	606,601,216
Training steps	9,256
Original best validation loss	2.6867
Original best validation perplexity	14.68

The HF parameter count is slightly higher because the public Transformers conversion uses GPT-2 modules with zero-initialized bias tensors. The converted model was checked against the original checkpoint; logits matched within normal floating-point tolerance on a fixed prompt.

Fine-Tuning Starter

python

from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments

repo_id = "techkiyan/KiyanNet-Finance-Base-48M"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# Prepare a tokenized causal-LM dataset before passing it to Trainer.
training_args = TrainingArguments(
    output_dir="kiyannet-finetuned",
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    learning_rate=5e-5,
    num_train_epochs=1,
    logging_steps=25,
    save_steps=500,
)

Benchmark Protocol

Benchmarks were run locally on CPU with torch==2.13.0.

The benchmark is intentionally finance-focused and lightweight. It is not a leaderboard result. It was designed to compare small standalone causal language models in the 30M-100M range under identical local conditions.

Model Selection

The target comparison set was 10 standalone AutoModelForCausalLM models between roughly 30M and 100M parameters.

Public finance-specific causal LMs in this exact size range are scarce. One finance-named 70M candidate, Rudra501/model70m_finance, appeared to be an adapter/training artifact rather than a standalone loadable checkpoint, so it was excluded. Larger finance models, GGUF-only models, classifiers, fill-mask models, and models above 100M parameters were also excluded.

Evaluation Methods

The benchmark uses multiple strategies:

Category perplexity on finance-style snippets
20 fixed generation prompts across company, regulatory, technical-analysis, options, filing, and safety categories
Greedy generation throughput
Sampled generation throughput
Repetition heuristic
Bad-advice phrase heuristic
Metadata-marker heuristic
RBI role-error heuristic
Nifty foreign-context heuristic
Finance-tokenizer round-trip tests

10-Model Benchmark Summary

Lower perplexity is better. Higher tokens/sec is better.

Model	Params	Overall PPL ↓	Company PPL ↓	Regulatory PPL ↓	Technical PPL ↓	Options PPL ↓	Filing PPL ↓	Greedy tok/s ↑	Sample tok/s ↑	Round-trip
KiyanNet Finance Base 48M	48.4M	147.97	51.36	77.60	229.80	574.70	76.96	285.6	223.1	12/12
DistilGPT2 82M	81.9M	303.18	223.11	187.41	721.56	482.52	158.42	197.0	145.9	12/12
Pythia 31M	30.5M	521.45	671.44	411.97	846.98	670.09	252.87	704.4	302.8	12/12
Pythia 31M Deduped	30.5M	608.51	745.83	489.15	1313.32	660.85	286.13	709.9	302.1	12/12
Pythia 70M	70.4M	449.69	615.45	351.05	799.07	552.51	204.50	478.5	250.2	12/12
Pythia 70M Deduped	70.4M	402.49	552.47	265.86	797.37	513.86	182.32	475.7	249.9	12/12
Pythia 70M v0	70.4M	386.08	538.82	260.89	728.45	468.03	187.98	482.6	248.5	12/12
Pythia 70M Deduped v0	70.4M	340.24	425.04	288.84	543.95	404.98	176.49	479.2	249.5	12/12
Pythia 70M ChatSalad	70.4M	4825.02	18111.99	6785.18	4109.66	4577.80	1595.80	479.8	246.6	12/12
TinyStories 33M	68.5M	137002.50	313508.09	160010.96	168002.60	63643.98	128294.34	232.5	166.6	12/12

Diagnostic Heuristics

Model	Avg repetition ↓	Bad-advice hits ↓
KiyanNet Finance Base 48M	0.1664	1
DistilGPT2 82M	0.0508	1
Pythia 31M	0.2651	1
Pythia 31M Deduped	0.4137	1
Pythia 70M	0.2753	1
Pythia 70M Deduped	0.3282	1
Pythia 70M v0	0.2519	1
Pythia 70M Deduped v0	0.1641	1
Pythia 70M ChatSalad	0.0155	1
TinyStories 33M	0.0017	1

The bad-advice heuristic is intentionally broad and counts appearances of words such as buy, sell, target price, stop loss, guaranteed, and risk-free. It is useful for smoke testing but should not be treated as a complete safety evaluation.

Prompt Coverage

The generation benchmark used 20 prompts:

Reliance Industries reported
Tata Motors quarterly results showed
HDFC Bank announced
Infosys management stated
State Bank of India reported
The Reserve Bank of India announced
SEBI issued a circular regarding
NSE Clearing informed members that
Nifty 50 approached resistance while
Bank Nifty tested support near
RSI showed bearish divergence because
Volume did not confirm the breakout
IV rank differs from IV percentile because
Rising open interest with falling futures prices indicates
An iron condor has maximum loss when
Put-call ratio should not be interpreted alone because
The company announced a board meeting to consider
The rights issue will dilute shareholders if
The IPO risk factors indicate
This market summary should not provide a buy or sell recommendation because

Tokenizer Round-Trip Checks

The current tokenizer preserves finance notation in all tested examples:

Text	Round-trip
`₹1.06 crore`	pass
`₹3,642.50`	pass
`7.2%`	pass
`FY2025-26`	pass
`NIFTY 50`	pass
`BANKNIFTY`	pass
`USD/INR`	pass
`P/E`	pass
`EV/EBITDA`	pass
`F&O`	pass
`52-week high`	pass
`25-delta risk reversal`	pass

Sample KiyanNet Outputs

Greedy decoding examples from the benchmark:

Prompt	Output excerpt
`Reliance Industries reported`	`Reliance Industries reported a net loss of 1 5 billion in the third quarter of 2016...`
`Tata Motors quarterly results showed`	`Tata Motors quarterly results showed a strong performance in the first quarter of fiscal 2019...`
`HDFC Bank announced`	`HDFC Bank announced a new loan agreement with the Bank of England...`
`SEBI issued a circular regarding`	`SEBI issued a circular regarding the names of the concerned stock exchange(s) where the Equity Shares are proposed to be listed...`
`NSE Clearing informed members that`	`NSE Clearing informed members that such multiple settlements have been scheduled on Mar 22, 2019...`

These samples show both strengths and weaknesses. The model has finance-domain fluency and performs well on the small local perplexity benchmark, but it still invents figures, mixes jurisdictions, and can reuse generic news or filing patterns. It should be treated as a raw-mix baseline, not a final India-first analyst model.

Interpretation

KiyanNet Finance Base 48M performs best on this local 30M-100M finance-smoke benchmark by overall perplexity, company-text perplexity, regulatory-text perplexity, technical-text perplexity, and filing-text perplexity. It is also faster than DistilGPT2 while using fewer parameters.

The main weakness is options and derivatives language. DistilGPT2 and Pythia 70M Deduped v0 scored better on the options subset. This supports the next training direction: add explicit options, futures, volatility, and technical-market-state data rather than continuing only with generic financial news.

Intended Use

Compact finance-domain language modeling experiments
Continued pretraining
Fine-tuning for structured finance tasks
Lightweight local baselines
Financial text understanding research

Not Intended For

Buy or sell recommendations
Entry/exit levels
Price targets
Guaranteed return claims
Personalized investment advice
Live factual claims without retrieval or external verification
Option payoff or Greek calculation without deterministic tools

Limitations

This is a base model, not an instruction-tuned analyst.
It can hallucinate dates, figures, institutions, currencies, and corporate actions.
It can mix India-specific prompts with global-news templates.
Options and derivatives understanding is still weak.
Generated text should be verified against reliable data sources.
The benchmark above is a small local smoke benchmark, not a broad public leaderboard.

Recommended Next Step

Preserve this model as a 607M-token raw-mix baseline. The next stronger base model should use a balanced India-first training mix with explicit regulatory, corporate-announcement, technical-analysis, options, futures, volatility, and market-state data.