Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

auto-g-nano-153m – AI Model by geoffsee | AlphaNeural AI

You can deploy this model and start earning money today!

geoffsee/
auto-g-nano-153m

like

0

text-generation

HuggingFaceFW/fineweb-edu

Views

No views yet

auto-g-nano-2

This is a modernized, "Grok-style" decoder-only Transformer (nanoGPT evolution) trained on the FineWeb-Edu dataset.

Key Features

Modern Architecture: Llama-style implementation with RoPE, RMSNorm, and SwiGLU.
Grouped-Query Attention (GQA): Optimized for inference efficiency.
BPE Tokenization: Uses OpenAI's tiktoken (GPT-2).
Parameters: ~152.7M

Model Details

Architecture: Decoder-only Transformer
Vocab Size: 50257
Embedding Dimension: 768
Heads: 12
KV Heads: 4
Layers: 12
Block Size: 1024

How to Use

You can use this model directly with the GPT class from this repository.

python

from model import GPT
import tiktoken
import torch

model = GPT.from_pretrained("geoffsee/auto-g-nano-2")
enc = tiktoken.get_encoding("gpt2")

# Generate text
prompt = "The future of AI is"
idx = torch.tensor(enc.encode(prompt)).unsqueeze(0)
completion = model.generate(idx, max_new_tokens=50)
print(enc.decode(completion[0].tolist()))

Training Data

Trained on a sample of FineWeb-Edu.