The HPC-Coder-v2-1.3b model is an HPC code LLM fine-tuned on an instruction dataset catered to common HPC topics such as parallelism, optimization, accelerator porting, etc.
This version is a fine-tuning of the
Deepseek Coder 1.3b model.
It is fine-tuned on the
hpc-instruct,
oss-instruct, and
evol-instruct datasets.
We utilized the distributed training library
AxoNN to fine-tune in parallel across many GPUs.
HPC-Coder-v2-1.3b,
HPC-Coder-v2-6.7b, and
HPC-Coder-v2-16b are the most capable open-source LLMs for parallel and HPC code generation.
HPC-Coder-v2-16b is currently the best performing open-source LLM on the
ParEval parallel code generation benchmark in terms of
correctness and
performance.
It scores similarly to 34B and commercial models like Phind-V2 and GPT-4 on parallel code generation.
HPC-Coder-v2-6.7b is not far behind the 16b in terms of performance.
The model is provided as a standard huggingface model with safetensor weights.
It can be used with
transformers pipelines,
vllm, or any other standard model inference framework.
HPC-Coder-v2 is an instruct model and prompts need to be formatted as instructions for best results.
It was trained with the following instruct template:
1Below is an instruction that describes a task. Write a response that appropriately completes the request.
2
3### Instruction:
4{instruction}
5
6### Response:
7
4 and 8 bit quantized weights are available in the GGUF format for use with
llama.cpp.
The 4 bit model requires ~0.8 GB memory and can be found
here.
The 8 bit model requires ~1.4 GB memory and can be found
here.
Further information on how to use them with llama.cpp can be found in
its documentation.
