aether-v7.2-unified
The Sephirot adapter served by
qbc.network, and the first
adapter produced by our decentralised training loop to pass a full evaluation and
be promoted into the live served model. It succeeds
aether-v7.1-unified.
It is a small adapter over a frozen Qwen2.5-7B-Instruct base (Q4_K_M): ten
low-rank experts with a top-2 router, about 1.18 million trainable parameters. The
base model is not redistributed here; load Qwen/Qwen2.5-7B-Instruct and apply this
adapter on top. Serving precision is BF16.
On-chain attestation
This model's identity is attested on the QBC chain. The on-chain weight_root is a
SHA-256 over (base, adapter, tokenizer, config):
- weight_root:
0x3e99b3d2f4e4d5eacbb3fe8d7e2f709dbfc48deb34407035383b3a5263e29e4e
- registry round: 2, finalised by a 4-of-5 validator quorum
- submitted at block: 1698355
- content root of the trained adapter:
0x157a01b9
How it was trained
Trained with a DiLoCo-style local-training-plus-aggregation loop across a fleet of
ordinary 8GB consumer GPUs. A vocab-chunked backward pass keeps a full 7B adapter
step inside roughly 7.5GB of video memory, so commodity cards can train the served
model. Training runs as short payable rounds of about 35 minutes: a worker draws a
fresh shuffled batch from a large corpus, trains the adapter locally, and submits a
candidate. A candidate is accepted only if it beats the previous accepted candidate
on a fixed held-out set it never trains on. Ten rounds produced seven accepted
units, and the best by full held-out evaluation is this release.
Evaluation
Measured on this adapter over the frozen base, as a snapshot from the promotion
evaluation (not a live probe):
| Measurement | Base | This adapter | Change |
|---|
| Held-out cross-entropy (clean 1000-example corpus, nats) | 2.9014 | 2.5609 | -0.3405 |
| MMLU (full 14042, uncapped) | 0.7116 | 0.7109 | -0.07pp (flat) |
| GSM8K (full 1319, uncapped) | 0.7248 | 0.7498 | +2.50pp |
The BF16 weights served here reproduce the held-out CE of the F32 candidate to four
decimals, so the -0.3405 nats generalisation gain holds at serving precision.
Generalisation on unseen text improved, general knowledge held flat, and multi-step
maths reasoning improved, at no measured cost to base capability. Against the
previously-served adapter, the promotion gate measured a further held-out CE
improvement (2.5912 to 2.4811 on the gate's own holdout).
Scope and honesty
This adapter is the product of decentralised training with an owner-gated
promotion into the served model. This is Stage 1 of a four-stage plan. It is not
yet a permissionless model update: that stage is designed, not built, and is gated
on robust aggregation that survives a malicious contributor, a stake-and-slash
economic layer, and honest independent re-derivation of the claimed improvement.
During this work our own evaluation flattered us three times, through memorisation on
training data, a sampling-window bug, and a mixed-scale gate that false-rejected a
good round. Each was caught and fixed before any number here was trusted. The full
account is in the
engineering write-up.
Usage
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3
4base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
5tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
6model = PeftModel.from_pretrained(base, "QuantumAI-Blockchain/aether-v7.2-unified")
Licence
Released under Apache-2.0, matching the Qwen2.5-7B-Instruct base licence.