psgokulkrishnan/

Llama-3.2-1B-GGUF

Usage: from llama_cpp import Llama

GGUF_PATH = "./llama_3_2_1b_Q4_K_M.gguf_Q4_K_M.gguf"

print("⏳ Initializing long-context GGUF engine directly into memory layout...") llm = Llama( model_path=GGUF_PATH, n_ctx=16384, flash_attn=True, n_threads=4, verbose=False )