This model is a GGUF conversion of purrgpt-community/Tiny-Purr-350M-merged, created using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details.

Using with llama.cpp

Install llama.cpp (works on Mac and Linux):

brew install llama.cpp

You can invoke the model using either the CLI or the server.

llama-cli --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -p "The meaning to life and the universe is"

llama-server --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -c 2048

git clone https://github.com/ggerganov/llama.cpp

Move into the llama.cpp folder and build with the LLAMA_CURL=1 flag. Add hardware-specific flags if needed (e.g., LLAMA_CUDA=1 for Nvidia GPUs on Linux):

bash

cd llama.cpp
LLAMA_CURL=1 make

./llama-cli --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -p "The meaning to life and the universe is"

./llama-server --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -c 2048