Exp-4 – AI Model by CyanMonkey | AlphaNeural AI

You can deploy this model and start earning money today!

CyanMonkey/
Exp-4

like

0

Views

No views yet

Exp-4

Small language model (3.0M parameters) trained from scratch.

Architecture

Property	Value
Layers	5
Hidden size	192
Intermediate size	512
Attention heads	6 (GQA kv=6)
Max sequence length	1024
Vocab size	4096
Tied embeddings	True
Total parameters	3.000M

Training

Tokens seen: 15,605,382,144
Val loss: 2.3172
Val PPL: 10.15

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("CyanMonkey/Exp-4")
model = AutoModelForCausalLM.from_pretrained("CyanMonkey/Exp-4")
inputs = tokenizer("Hello", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(output[0], skip_special_tokens=True))