./main -m https://huggingface.co/dhanushmekaka/phi-3-mini-text-to-cypher-supply/resolve/main/phi3_gguf_q8_0.gguf \
       -p "<|user|>Top 5 products by total ordered quantity.<|end|><|assistant|>"

🐍 Run with Python (llama-cpp-python)

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="dhanushmekaka/phi-3-mini-text-to-cypher-supply",
    filename="phi3_gguf_q8_0.gguf",
    n_ctx=4096,
)

prompt = """<|user|>
Top 5 products by total ordered quantity.
<|end|>
<|assistant|>
"""

response = llm(prompt, max_tokens=150)
print(response["choices"][0]["text"])

💬 Use with Ollama

Modelfile:

FROM ./phi3_gguf_q8_0.gguf
TEMPLATE """<|system|>{{ .System }}<|end|><|user|>{{ .Prompt }}<|end|><|assistant|>"""

Create:

ollama create phi3-supply -f Modelfile

Run:

ollama run phi3-supply

🧠 Model Training Details

Training method: QLoRA (4-bit)
Dataset size: 5k+ records
Task: text → Cypher / analytics reasoning
Frameworks: TRL, PEFT, BitsAndBytes, HuggingFace
Merged to fp16 → converted to GGUF (Q8_0)

📌 License

Base model: MIT
Finetuned checkpoints: permissive for commercial use

❤️ Author

M. Dhanush Vardhan Data Scientist • Supply Chain AI • LLM Engineer HuggingFace: https://huggingface.co/dhanushmekaka

✅ 3️⃣ Ready Python Load Script

This loads your HF model directly and runs inference:

python

from llama_cpp import Llama

# Load GGUF from your HuggingFace repo
llm = Llama.from_pretrained(
    repo_id="dhanushmekaka/phi-3-mini-text-to-cypher-supply",
    filename="phi3_gguf_q8_0.gguf",
    n_ctx=4096,
)

# Prompt (Phi-3 Chat Template)
prompt = """<|user|>
Top 5 products by total ordered quantity.
<|end|>
<|assistant|>
"""

# Run inference
output = llm(prompt, max_tokens=200)
print("Model Response:\n", output["choices"][0]["text"])

🎉 Everything is ready.

If you want, I can also generate:

Just tell me!