Views
No views yet
| Original (BF16) | NVFP4 (this) | |
|---|---|---|
| Size on disk | ~16 GB | ~5 GB |
| Compression | — | ~3.0x |
| Parameters | 8B | 8B |
| Architecture | Dense transformer, xIELU activation | same |
| Context window | 65,536 tokens | 65,536 tokens |
| Languages | 1,811 | 1,811 |
1vllm serve bg-digitalservices/Apertus-8B-Instruct-2509-NVFP4 \
2 --quantization modelopt \
3 --dtype auto \
4 --kv-cache-dtype fp8 \
5 --gpu-memory-utilization 0.85 \
6 --max-model-len 65536 \
7 --trust-remote-code1VLLM_NVFP4_GEMM_BACKEND=marlin vllm serve bg-digitalservices/Apertus-8B-Instruct-2509-NVFP4 \
2 --quantization modelopt \
3 --dtype auto \
4 --kv-cache-dtype fp8 \
5 --max-model-len 65536 \
6 --trust-remote-codequantize.py1@misc{swisstransformer2025apertus,
2 title = {Apertus},
3 author = {Swiss Transformer},
4 year = {2025},
5 url = {https://huggingface.co/swiss-ai}
6}