Long-Context VRAM Benchmark
image Qwen3.8-27B vs SpeedX27-VL-GDN64
We benchmarked long-context VRAM usage of:
Qwen/Qwen3.8-27BsummerMC/Qwen3.8-27B-SpeedX27-VL-GDN64
under the same conditions on an ~80 GiB GPU.
Configuration
1 Context limit: 262,144 tokens
2 Chunk size: 8,192 tokens
3 Batch size: 1
4 Precision: BF16
5 Cache: enabled
VRAM scaling
[PASTE BENCHMARK IMAGE HERE]
The difference is clear:
Qwen/Qwen3.8-27B shows increasing VRAM usage as context grows.SpeedX27-VL-GDN64 stays nearly flat at approximately 55.36 GiB across the full context range.
Results
Model Model VRAM Maximum successful context Peak VRAM Qwen/Qwen3.8-27B 50.96 GiB 196,608 72.82 GiB at 196K SpeedX27-VL-GDN64 51.30 GiB 262,144 55.36 GiB
The standard model successfully processed 196,608 tokens , but failed while processing the next 8,192-token chunk.
The GDN64 model successfully completed the full 262,144-token context.
After the 262K prefill, a one-token decode also succeeded:
1 Processed context: 262,144 tokens
2 Maximum chunk peak: 55.36 GiB
3 Decode success: True
4 Decode time: 0.123 s
Memory behavior
At 196,608 tokens:
1 Qwen3.8-27B: ~72.82 GiB peak
2 SpeedX27-GDN64: ~55.36 GiB peak
That is approximately 17.46 GiB less peak VRAM for SpeedX27-GDN64 at the same context length.
More importantly, the scaling behavior is fundamentally different.
1 Qwen3.8-27B:
2 VRAM increases with context length.
3 SpeedX27-GDN64:
4 VRAM remains nearly constant as context length increases.
This benchmark is consistent with the recurrent GDN64 state behaving close to constant-memory with respect to context length, while the standard hybrid architecture retains context-dependent cache growth.
262K result
On the tested ~80 GiB GPU:
1 Qwen/Qwen3.8-27B
2 196K -> Success
3 204K -> OOM
4 SpeedX27-VL-GDN64
5 262K -> Success
The result is not caused by smaller model weights: the GDN64 model actually uses slightly more VRAM immediately after loading.
The advantage appears during long-context inference, where its persistent context-state memory remains nearly flat.
SpeedX27 1T Logical Context
This revision introduces an experimental
1,000,000,000,000-token logical context architecture .
Context specification
Context type Tokens Native / trained context 262,144 Logical streaming context 1,000,000,000,000 Streaming chunk 8,192
The 1T value describes a
logical recurrent/streaming context space .
It does not claim that the current weights
were trained using a single one-trillion-token
dense sequence.
Architecture
1 Input stream
2 |
3 v
4 L0 Active Tokens
5 |
6 v
7 L1 GDN64 recurrent state
8 |
9 v
10 L2 compressed chunk memory
11 |
12 v
13 L3 semantic retrieval memory
14 |
15 v
16 L4 persistent archive
Long-context design
64-layer Gated DeltaNet recurrent architecture
bounded-memory streaming prefill
64-bit absolute token positions
recurrent-state checkpointing
resumable context processing
hierarchical-memory-ready runtime
semantic/external retrieval ready
logical context target: 1 trillion tokens
Validation progression
1 262K
2 |
3 v
4 1M
5 |
6 v
7 16M
8 |
9 v
10 256M
11 |
12 v
13 1B
14 |
15 v
16 16B
17 |
18 v
19 1T
A 1T native-context capability should only
be claimed after end-to-end training,
calibration and validation at that scale.
