Views
No views yet
A 4-bit quantized version of the original LLaMA 8B model, optimized for CPU-only inference on low-resource environments such as laptops, edge devices, and experimental mobile setups.
Q4_0 (4-bit)llama.cpp compatibleggml-model-q4_0.bin: 4-bit quantized weights (LLaMA 8B)tokenizer.model: SentencePiece tokenizerparams.json or config.json: Model config file (if needed)Use withllama.cpp,ggml,llama-cpp-python, or other compatible libraries.
llama.cpp1git clone https://github.com/ggerganov/llama.cpp
2cd llama.cpp
3make
4./main -m ./models/llama-8b/ggml-model-q4_0.bin -p "What is the capital of France?"llama-cpp-pythonpip install llama-cpp-python1from llama_cpp import Llama
2llm = Llama(model_path="ggml-model-q4_0.bin")
3response = llm("Explain quantum computing in simple terms.")
4print(response)llama.cpp