Views
No views yet
cd {your git clone directory}
python convert.py -i {path to}/AceInstruct-72B -o {path to}/AceInstruct-72B/workingdir -cf {path to}/AceInstruct-72B/AceInstruct-72B-3bpw -b 3.0<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant"chat_template": "{{- '<|im_start|>system\\nYou are AceInstruct developed by NVIDIA. You are helpful assistant.<|im_end|>\\n' }}\n {%- for message in messages %}\n{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}\n{%- endfor %}\n{%- if add_generation_prompt %}\n{{- '<|im_start|>assistant\n' }}\n{%- endif %}\n",| quantization type | file size |
|---|---|
| 3.0bpw | 27.8 GiB |
| Qwen2.5-1.5B-Instruct | AceInstruct-1.5B | Qwen2.5-7B-Instruct | AceInstruct-7B | Qwen2.5-72B-Instruct | AceInstruct-72B | |
|---|---|---|---|---|---|---|
| HumanEval | 61.60 | 73.17 | 84.80 | 85.37 | 86.60 | 89.63 |
| MBPP | 63.20 | 65.76 | 79.20 | 74.32 | 88.20 | 83.66 |
| GSM8K | 73.20 | 80.44 | 91.60 | 93.10 | 95.80 | 96.36 |
| MATH | 55.20 | 60.34 | 75.50 | 76.40 | 83.10 | 84.50 |
| MMLU | 58.37 | 58.17 | 74.51 | 74.68 | 84.67 | 83.88 |
| MMLU Pro | 32.40 | 33.78 | 56.30 | 54.50 | 71.10 | 66.10 |
| Average | 57.33 | 61.94 | 76.99 | 76.40 | 84.91 | 84.02 |