anemll/

ANEMLL-Prefill-bench

Model files for prefill benchmarks on Apple Neural Engine:

ANEMLL 0.3.0-Alpha https://github.com/Anemll/Anemll/releases change mode="kmeans" to mode="uniform" for faster processing in llama_converter.py line 402 Example export: ./anemll/utils/convert_model.sh \ --model ~/Models/HF/Llama-3.1-Nemotron-Nano-8B-v1 \ --output ~/Models/ANE/anemll-Nemotron-8B-ch4-b512-w512 \ --context 512 \ --batch 512 \ --lut1 "" \ --lut2 4 \ --lut3 "" \ --chunk 4 --restart 4

Source model: https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1 Chuncks : https://huggingface.co/anemll/ANEMLL-Prefill-bench