You can deploy this model and start earning money today!

MarkChenX/
quantum-gpt-109M

Name: quantum-gpt-109M
Availability: InStock

Views

No views yet

quantum-gpt-109M

A ~109M parameter GPT-style base model trained with nanochat, using a quantum-inspired MLP block (mlp_type: quantum) in place of the standard feed-forward layer.

Checkpoint

Training step: 1634
Validation bpb: 0.9571
Epoch: 1

Architecture

Param	Value
n_layer	16
n_head / n_kv_head	16 / 16
n_embd	1024
sequence_len	1024
vocab_size	4096
mlp_type	quantum
quantum_num_qubits	2
quantum_depth	2
window_pattern	L

Files

model_001634.pt — model weights
meta_001634.json — full training/model config and metrics at this step

Optimizer state is not included in this upload (only needed to resume training, not to use the model).

Usage

Load with the nanochat checkpoint manager from MarkCodering/Quantum-GPT:

python

from nanochat.checkpoint_manager import load_model
# see repo for exact loading API

MarkChenX/quantum-gpt-109M