You can deploy this model and start earning money today!

mountainrock/
vadakayil-llm-tiny

Name: vadakayil-llm-tiny
Availability: InStock

Views

No views yet

Vadakayil LLM v3 (Medium)

A medium-sized character-level LLM trained on Capt Ajit Vadakayil's writings about Mach 0.3, consciousness, and Vedic philosophy.

Model Details

Parameter	Value
Architecture	Decoder-only Transformer
Vocabulary	Character-level (74 tokens)
Parameters	~12.7M
d_model	512
num_layers	6
num_heads	8
d_ff	1024
max_seq_len	512
Training Data	vadakayil_combined_training.txt (~57K chars)

Training Configuration

Parameter	Value
Epochs	150
Learning Rate	3e-4
Batch Size	32
Sequence Length	256
Train Split	95%
Final Val Loss	0.078

Performance

Metric	Value
Initial Val Loss	5.37
Final Val Loss	0.078
Improvement	98.5%

Usage

With this codebase:

python

from model import TinyLLM
from tokenizer import Tokenizer
import torch

# Load tokenizer
tokenizer = Tokenizer.load("tokenizer.json")

# Load model
checkpoint = torch.load("model.pt", map_location="cpu")
model = TinyLLM(
    vocab_size=74,
    d_model=512,
    num_heads=8,
    num_layers=6,
    d_ff=1024,
    max_seq_len=512,
    dropout=0.1,
    pad_token_id=0
)
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

# Generate text
prompt = "What is Mach 0.3"
input_ids = tokenizer.encode(prompt, add_special_tokens=False)
input_ids = torch.tensor([input_ids], dtype=torch.long)
output = model.generate(input_ids, max_new_tokens=150, temperature=0.8)
print(tokenizer.decode(output[0].tolist()))

Example Prompts:

"What is Mach 0.3 and why is it significant?"
"Why is Mach 0.3 called the Paradox Rekha?"
"What is the Silent Kalki Revolution of Consciousness?"
"What does the movie Thondi Muthalum Driksakshiyum represent?"

Training Data Topics

Mach 0.3 and fluid dynamics
Compressible vs incompressible flow equations
Prandtl-Glauert correction
Consciousness and Vedic philosophy
Silent Kalki Revolution
Evidence and Witness (Thondi Muthalum Driksakshiyum film analysis)
Paradox Rekha concept
Capt Ajit Vadakayil's teachings

Limitations

Character-level tokenizer (74 characters)
Trained on limited dataset (~57K characters)
May produce inaccurate output for technical queries
Best for philosophical/consciousness topics

License

MIT License

Credits

Training data: Capt Ajit Vadakayil's writings Model: Custom transformer implementation

mountainrock/vadakayil-llm-tiny