A 44M-parameter byte-level encoder designed from first principles for the Apple Neural Engine.
No attention. No tokenizer. Pure conv1×1 — the ANE's native fast path.
What is this?
Standard language models (BERT, transformers) are designed for GPU execution — they rely on softmax attention, dynamic masking, and embedding lookups that are a poor fit for the ANE's hardware primitives.
ANE-ByteGrid asks: can a useful language encoder be built to use only the ANE's fast path, with no GPU fallback?
The answer is yes. ANE-ByteGrid is a bidirectional byte-level encoder that:
Replaces attention with a hierarchical local+global conv1×1 token mixer
Accepts raw bytes (no tokenizer, 256-byte fixed windows)
Compiles to 26 ANE kernels with all shapes fixed at compile time
Runs on the Apple Neural Engine at 11.2 ms/pass, 1140 MB/s, drawing <500 mW
Trains on MPS (Apple GPU) at 2.1 steps/s via a PyTorch mirror
The training objective is masked byte prediction (BERT-style, 15% mask rate) on FineWeb-Edu. After 500,000 steps:
Split
Top-1
Top-5
Loss
In-domain (FineWeb-Edu)
68.4%
69.7%
3.78
Out-of-domain (code, JSON, markdown)
37.7%
46.5%
5.43
Combined
55.3%
60.3%
4.49
Architecture in one sentence
Input bytes → one-hot + class + position encoding → 24 blocks of (local mixer → global mixer → channel GLU) → byte logits.
Each local mixer mixes the 16 positions within a 16-byte chunk; each global mixer mixes the same intra-chunk position across all 16 chunks. Both are conv1×1 on reshaped tensors — no attention, no softmax, no dynamic shapes.
Training power: ~11 W GPU (MPS) / ANE idle.
Inference power: <500 mW ANE / GPU completely idle.
Comparison with BERT
ANE-ByteGrid
BERT-base
Parameters
44M
110M
Architecture
conv1×1 mixers
softmax attention
Tokenization
byte-level (256 vocab)
WordPiece (30k vocab)
Inference target
Apple Neural Engine
GPU
In-domain Top-1
68.4%
~65–70%
Training data
FineWeb-Edu (2B bytes)
Books + Wikipedia (16 GB)
ByteGrid matches BERT's masked-prediction accuracy on in-domain text at 40% fewer parameters and on a 100× smaller corpus, while being natively deployable on ANE without GPU fallback.
Repository layout
training/ PyTorch model, training loop, data pipeline, eval, weight bridge
+ Objective-C MIL generator and ANE runtime
tests/ Shape, MIL, and runtime parse unit tests
tools/ Benchmarking, ablation, loss curve, and phase protocol scripts
paper/ paper.tex manuscript + fig_loss_curves.pdf
weights/ ANE-format weight blobs (generated)
build/ Protocol + benchmark artifacts
Quickstart
Requirements: macOS, Apple Silicon, Xcode CLI tools, Python 3.11+
sh
1# Python environment
2python3 -m venv .venv
3.venv/bin/pip install -U pip torch datasets matplotlib
45# Build & test Objective-C runtime
6make && make test
78# Smoke train (200 steps)
9make train-pt-smoke
1011# Full training (500k steps)
12make train-pt
1314# Resume from last checkpoint
15make train-pt-resume
1617# Evaluate final checkpoint
18.venv/bin/python training/eval_pt.py --checkpoint weights_probe/ckpt_00500000.pt
1920# ANE compile smoke test
21make compile-smoke
The ANE runtime path uses private APIs (_ANEInMemoryModel) and may break across macOS updates.
Sandboxed environments can block ANE access (mach-lookup com.apple.appleneuralengine).
This model is an encoder (masked prediction), not an autoregressive text generator.
License
MIT (see LICENSE).
- build/update_loop_head_logit_speedhard.tsv: 3/3 steps accepted (head_logit 1.00 -> 0.97).
- build/update_loop_residuals_speedhard.tsv: rejected at step 1.
Promotion gate status (long run):
build/phase3_protocol_golden_hybrid_run1.json and build/phase3_protocol_golden_hybrid_run2.json both produced promote_defaults=true on hybrid head candidates (consistent promotion verdict).
Defaults were promoted to head_weight=0.98, head_logit=0.97.
