Views
No views yet
__metadata__.peng_zstd="1", per-tensor "nb" = uncompressed size). The
peng engine (commit 6ffb8d9+) reads
this format natively and produces bit-exact output vs the uncompressed
container (verified: per-frame byte-compare of all 17 shards + identical TEMP=0
generations at full scale).| your setup | use |
|---|---|
| limited bandwidth / storage | this one (26% smaller) |
| plenty of RAM (experts pinned once at boot) | this one — decompress cost is paid once at load |
| NVMe-streaming on a core-limited box, max tok/s | the plain int4 container — measured ~18% faster streaming on a 16-core WSL2 box, because zstd decompression competes with expert matmul for cores |
findings.md §41.1git clone https://github.com/fivetechsoft/peng-mimo && cd peng-mimo/c && make CUDA=1 mimo
2SNAP=/path/to/this/repo COLI_CUDA=1 CUDA_DENSE=1 DIRECT=1 TAO=1 \
3 PROMPT='Write one short sentence about Rome.' NGEN=24 ./mimo 64 4 8c/tools/repack_zstd.py (--verify byte-compares every frame).