You can deploy this model and start earning money today!

ChrisMcCormick/
qwen-arithmetic-t4

Name: qwen-arithmetic-t4
Availability: InStock

Qwen/Qwen2.5-0.5B-Instruct

finetune

apache-2.0

Views

No views yet

qwen-arithmetic-t4 — banked fp16 weights

Not a fine-tune. These are Qwen/Qwen2.5-0.5B-Instruct's own weights, re-containered into the layout that train_qwen_arithmetic_t4.py loads: per-layer matrices stacked into (L, ...) banks, QKV and gate/up concatenated into one GEMM each, cast to fp16. Same 494,032,768 parameters, same values.

That script is a single-file GRPO speedrun of Qwen2.5-0.5B-Instruct on arithmetic for ONE Tesla T4 — the free Colab GPU. It has no transformers dependency and no nn.Module: it opens one file and gets tensors whose names, shapes and dtype are already the ones its handwritten forward/backward and its CUDA-graph decode engine use. This repo is that file, so a Colab session spends its first minute downloading ~942 MB instead of pulling the checkpoint and rebuilding the banks on two vCPUs.

The trainer fetches this automatically — there is nothing to do by hand:

colab run --gpu T4 train_qwen_arithmetic_t4.py --timeout 1h

Files

banks_fp16_Qwen2.5-0.5B-Instruct.safetensors — the banks (942 MB, sha256 f3d896ed64d465efd17c32c6e66062ae)
banks_fp16_Qwen2.5-0.5B-Instruct.json — sidecar: arch, shapes, provenance. The trainer asserts every arch field in it against its own config at load, so a mismatched bank file fails loudly instead of silently.
tokenizer.json — the source repo's tokenizer, verbatim (sha256 c0382117ea329cdf097041132f6d7359), so neither the dataset prep nor the trainer touches another repo.

Banks

494,032,768 parameters (357,898,112 non-embedding) in 9 tensors, L = 24:

bank	shape	what it is
`embed`	151936 x 896	token embedding table; TIED, so it is also the lm_head
`W_QKV`	24 x 1152 x 896	fused QKV projection, rows [Q
`b_QKV`	24 x 1152	fused QKV bias, same row split
`W_O`	24 x 896 x 896	attention output projection
`W_GU`	24 x 9728 x 896	fused SwiGLU input projection, rows [gate
`W_down`	24 x 896 x 4864	SwiGLU output projection
`attn_norm`	24 x 896	pre-attention RMSNorm weight (input_layernorm)
`mlp_norm`	24 x 896	pre-MLP RMSNorm weight (post_attention_layernorm)
`final_norm`	896	final RMSNorm weight

Why fp16

The T4 (sm75) has no bf16 tensor cores, so the trainer runs fp16 live weights against fp32 masters. Storing the banks in the run dtype keeps a 1 GB bf16 transient off a 16 GB card at load and means nothing in the T4 path touches a bf16 kernel.

The cast is free of consequence: the checkpoint is bf16, whose 7 explicit mantissa bits fit inside fp16's 10, so these tensors are bit-for-bit what the trainer used to produce by casting at load. 716 of 494,032,768 values (0.0001%) land under fp16's subnormal floor and become zero — the same ones, either way. Largest magnitude in the checkpoint is 214, against fp16's 65504 ceiling.

Reproducing it

data/prepare_model_t4.py in the repo above, from the pinned Qwen/Qwen2.5-0.5B-Instruct checkpoint. Both files' sha256 are in the sidecar.

License

apache-2.0, inherited from Qwen/Qwen2.5-0.5B-Instruct. Cite Qwen2.5 for the weights.

ChrisMcCormick/qwen-arithmetic-t4