Harmonic GPT 128M Byte Agent Multiparty
An experimental 127.6M-parameter causal byte Transformer trained to represent dialogue as a
room of peer agents rather than a privileged user/assistant pair. Turns explicitly encode the
speaker, addressee, audience, and participant registry.
The checkpoint starts from Transformer pretraining step 90,000 (5.898B ordered UTF-8 bytes) and
then receives 500 completion-only SFT steps at peak LR 3e-5. Its mixture contains 25% No Robots
Q&A, 25% everyday conversations, 10% practical dialogue, 5% reasoning, 25% MultiLIGHT
multiparty dialogue, and 10% pretraining replay.
Evaluation snapshot
- Held-out ordinary-chat BPB: 1.244777 (starting value 1.294554)
- Held-out multiparty BPB: 1.216836 (starting value 1.382048)
- Matched base-corpus BPB: 0.950660 (starting value 0.940957)
- Response-only linguistic composite: 76.6
- 768-byte repetition probe: 0/6 detected loops under both greedy and T=0.8/top-k 40
- EOS: 5/6 greedy and 6/6 stochastic long-form probes
These are behavioral diagnostics, not claims of factual reliability. The model frequently gives
incorrect or nonsensical answers and must not be used for consequential decisions.
Native byte framing
1<|room|> scope=direct members=p0,p1
2<|participants|>
3p0=Agent.Ajax
4p1=Agent.Steve
5<|turn|> p0>p1 audience=p0,p1
6Agent.Ajax:
7Hello!
8<|end|>
9<|turn|> p1>p0 audience=p0,p1
10Agent.Steve:
The learned response terminator is \n<|end|>\n. The included handler maps OpenAI-style
user/assistant histories onto the two peer-agent slots while preserving turn boundaries.
The No Robots component makes this release CC BY-NC 4.0. MultiLIGHT is used through its public
ShareGPT conversion and is primarily fantasy roleplay; the checkpoint should be treated as an
exploratory group-dialogue experiment.
