Views
No views yet
| Metric / Property | Value |
|---|---|
| Best Validation Loss | 1.8157 (Cross-Entropy Loss) |
| Total Training Steps | 202,000 steps |
| Total Parameters | 11,453,120 (11.45M) |
| Non-Embedding Parameters | 10,173,120 (10.17M) |
| Training Dataset | SmallScale/Simple-Stories-Hindi (~2.11M stories) |
| Model Size on Disk | ~45.8 MB (model.safetensors) |
| Parameter | Value | Notes |
|---|---|---|
| Architecture | LLaMA-style Decoder | RoPE + SwiGLU + RMSNorm |
Hidden Size (d_model) | 320 | Vector dimension |
| FFN Intermediate Size | 896 | 8/3 × d_model rounded to multiple of 64 |
Layers (n_layers) | 7 | Transformer blocks |
Attention Heads (n_heads) | 5 | Multi-Head Self Attention |
| Head Dimension | 64 | d_model / n_heads |
Context Length (max_seq_len) | 512 tokens | Sequence window |
| Vocabulary Size | 4,000 | SentencePiece Unigram (Devanagari optimized) |
| Weight Tying | Enabled | Token embeddings & output projection share weights |
| Precision | float32 | Weights stored in native FP32 safetensors |
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4# Load tokenizer and model directly from Hugging Face
5tokenizer = AutoTokenizer.from_pretrained("SmallScale/Simple-Stories-Hindi-10M", trust_remote_code=True)
6model = AutoModelForCausalLM.from_pretrained("SmallScale/Simple-Stories-Hindi-10M", trust_remote_code=True)
7
8if torch.cuda.is_available():
9 model = model.to("cuda")
10
11# Prompt input
12prompt = "एक समय की बात है"
13inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
14
15# Generate story
16outputs = model.generate(
17 **inputs,
18 max_new_tokens=200,
19 do_sample=True,
20 top_k=40,
21 top_p=0.95,
22 temperature=0.8
23)
24
25print(tokenizer.decode(outputs[0], skip_special_tokens=True))एक समय की बात हैएक समय की बात है, और मैं छाया से देखता हूं। मेरे दो लोग, जीन और सैमुअल हैं, जो एक भव्य यात्रा पर जा रहे हैं। वे एक ही स्थान पर रहते हैं, लेकिन वे दोनों अपनी-अपनी कहानियाँ चाहते हैं...