This repository provides a quantized and GGUF-converted version of the Dante-7B model, based on the Qwen2 7B architecture.
It is optimized for use with Ollama 💻 or any backend compatible with llama.cpp.

📦 Model Origin

Base model: outflanknl/Dante-7B
Architecture: Qwen2 7B
Format conversion: Performed with the official llama.cpp conversion script:
python3 convert_hf_to_gguf.py /path/to/original/model --outfile Dante-7B.gguf

🔧 Quantization

The model has been quantized to reduce memory usage and improve inference speed while keeping high-quality outputs.

📂 Files in This Repository

Dante-7B.gguf → Ready-to-use model file (GGUF format)
Example Modelfile for Ollama

🚀 Quick Start with Ollama

1️⃣ Download the repository

bash

git lfs install
git clone https://huggingface.co/ganchito/dante-7b.gguf
cd dante-7b.gguf

2️⃣ Create a `Modelfile`

Example configuration:

dockerfile

FROM Dante-7B.gguf

# Model configuration
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|endoftext|>"

SYSTEM \"\"\"You are Dante, a 7B parameter language model based on Qwen2 architecture. 
You are a helpful, creative, and intelligent AI assistant. 
You can engage in conversations, answer questions, help with tasks, and provide thoughtful responses. 
Always be respectful, honest, and helpful while maintaining a conversational and engaging tone.\"\"\"

TEMPLATE \"\"\"{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>\"\"\"

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 32768

3️⃣ Build the model in Ollama

ollama create dante-7b -f Modelfile

4️⃣ Run the model

ollama run dante-7b

📜 License

This GGUF version is subject to the same license as the original Dante-7B model.

ganchito/dante-7b.gguf

🦅 Dante-7B (GGUF) – Optimized for Ollama