These are quantized versions of Meta's LLaMA-3-8B-Instruct model using IPEX and converted to GGUF format.
Supported Formats: f32.gguf, f16.gguf, q8_0.gguf
1from llama_cpp import Llama
2llm = Llama(model_path="q8_0.gguf")
3response = llm("What is AI?")
4print(response)
