Views
No views yet
prefill/: GPU-compatible prefill IR. The zero-length past_key_values.* inputs were fixed as internal zero constants.decode/: NPU decode IR.scripts/benchmark_prefill_decode.py: benchmark runner used for TTFT, output throughput, and peak memory measurement.1python scripts\benchmark_prefill_decode.py `
2 --prefill-repo .\prefill `
3 --decode-repo .\decode `
4 --prefill-device GPU `
5 --decode-device NPU `
6 --context-len 128 `
7 --decode-len 16 `
8 --output-dir .\results\ctx128_gpnd| Context | TTFT (s) | Output throughput (tok/s) | NPU peak (GiB) | GPU peak (GiB) |
|---|---|---|---|---|
| 128 | 0.880 | 10.740 | 4.302 | 16.515 |