Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

Spider-FLEXITOKENS-FP8 – Dataset by CLIWorks | AlphaNeural AI

You can deploy this model and start earning money today!

CLIWorks/
Spider-FLEXITOKENS-FP8

like

0

Views

No views yet

Spider-FLEXITOKENS FP8 Training

FP8 training pipeline for Spider-FLEXITOKENS on NVIDIA Blackwell GPUs (sm_120) using torchao Float8Linear and optional TileKernels fused MoE routing.

Architecture

Spider is a Recurrent-Depth Transformer (RDT) with:

1B parameters (996M), hidden_size=2048 Byte-level vocab: 272 tokens (256 UTF-8 bytes + 16 specials: BOS=257, EOS=258, PAD=256) 6 recurrent layers with MoE (32 experts, top-2 routing) + MLA attention 2 prelude + 2 coda dense… See the full description on the dataset page: https://huggingface.co/datasets/CLIWorks/Spider-FLEXITOKENS-FP8.