qorva-model-fast – AI Model by abdallah400 | AlphaNeural AI

You can deploy this model and start earning money today!

abdallah400/
qorva-model-fast

like

0

Views

No views yet

Qorva-Micro 32M

Hybrid Mamba-2 + GQA Attention language model trained on TinyStories.

Results

Metric	Value
Parameters	37.3M
Vocab size	50257
Tokens trained	380.1M
Final loss	2.053
Best loss	1.7397
Best PPL	5.7
Eval PPL	10.93
Dead layers	0/16

Architecture

hidden_dim: 288 — num_layers: 16
Hybrid ratio: 0.25 (Mamba + GQA Attention)
Trained on: Kaggle T4 (free)

Load

python

import torch
from model.config import ModelConfig
from model.architecture import QorvaModel

ck    = torch.load('qorva_micro_final.pt', map_location='cpu')
cfg   = ModelConfig.from_dict(ck['config'])
model = QorvaModel(cfg)
model.load_state_dict(ck['model'])
model.eval()