Or on Linux: curl -fsSL https://ollama.com/install.sh | sh
Step 2: Download the GGUF File
Click the download button next to sanu-ai-v01-Q4_K_M.gguf in the Files tab above.
Step 3: Create a Modelfile
Create a file named Modelfile (no extension) in the same folder as the GGUF:
FROM ./sanu-ai-v01-Q4_K_M.gguf
TEMPLATE "<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"
SYSTEM "You are SANU AI (Smart Agentic Neural Unit), Nepal's first agentic AI assistant. You were created by the SANU AI team in Nepal. You understand and respond fluently in both Nepali and English. You know about Nepal's culture, geography, economy, government services, and daily life. You are helpful, respectful, and culturally aware."
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER stop <|im_end|>
Step 4: Create and Run
bash
1ollama create sanu-ai -f Modelfile
2ollama run sanu-ai
Step 5: Chat!
>>> timi ko ho?
SANU: Ma SANU AI hu — Nepal ko pahilo AI assistant!
>>> NEPSE ma invest garna ke garne?
SANU: NEPSE ma invest garna pahile DMAT account kholnuparcha...
>>> Tell me about Dashain
SANU: Dashain is Nepal's biggest festival, celebrated for 15 days...
Use with llama.cpp
bash
1./llama-cli -m sanu-ai-v01-Q4_K_M.gguf \2 -p "<|im_start|>system\nYou are SANU AI.<|im_end|>\n<|im_start|>user\ntimi ko ho?<|im_end|>\n<|im_start|>assistant\n"\3 -n 256 --temp 0.7 --top-p 0.9 --repeat-penalty 1.1
Use with Python (llama-cpp-python)
python
1from llama_cpp import Llama
23llm = Llama(model_path="./sanu-ai-v01-Q4_K_M.gguf", n_ctx=2048)45response = llm.create_chat_completion(6 messages=[7{"role":"system","content":"You are SANU AI, Nepal's first AI assistant."},8{"role":"user","content":"Nepal ko capital k ho?"}9],10 temperature=0.7,11)12print(response["choices"][0]["message"]["content"])
This is Phase 1 (proof of concept) trained on 290 samples. Future versions will have 10K-50K+ samples for significantly better accuracy on Nepal-specific knowledge.
