SistInf/

Velvet-2B-GGUF

To perform this quantization, we started with llama.cpp as our base, modifying the file convert_hf_to_gguf_update.py to support this model. For modifying this file, we based our work on what was seen in the PR https://github.com/ggerganov/llama.cpp/pull/11716.

Note: As of today, llama.cpp does not support this model or this chat template https://github.com/ggerganov/llama.cpp/pull/11716.

Basic prompt format:

Prompt format with system message:

<s><instruction>{system_prompt}
{prompt}</instruction>

Original Model: https://huggingface.co/Almawave/Velvet-2B

Velvet-2B and Velvet-2B are made available under the Apache 2.0 license