FP16 (model-f16.gguf): Full precision, ~24GB
Q8_0 (model-Q8_0.gguf): 8-bit quantization, ~13GB, high quality
Q4_K_M (model-Q4_K_M.gguf): 4-bit quantization, ~7GB, recommended for most users

Usage with llama.cpp

bash

# Download model
huggingface-cli download ambrosfitz/mistral-nemo-gguf model-Q4_K_M.gguf --local-dir ./models

# Run inference
./llama.cpp/main -m ./models/model-Q4_K_M.gguf -p "Question: What was the Renaissance?\n\nAnswer:" -n 256

Usage with Ollama

bash

# Create Modelfile
cat > Modelfile <<EOF
FROM ./model-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
EOF

# Create model
ollama create mistral-humanities -f Modelfile

# Run
ollama run mistral-humanities "What was the Renaissance?"

Example Prompts

Question: What were the main causes of World War I?

Answer:

Question: Explain the philosophical ideas of the Enlightenment.

Answer:

Question: Who wrote 'The Republic' and what is it about?

Answer:

Limitations

This model is specialized for humanities topics and may not perform as well on:

Technical/scientific questions
Mathematics
Coding
Current events (knowledge cutoff applies)

License

Apache 2.0 (same as base model)

ambrosfitz/mistral-nemo-humanities-gguf

Mistral Nemo 12B - Humanities Distilled (GGUF)

Training Details

Available Quantizations

Usage with llama.cpp

Usage with Ollama

Example Prompts

Limitations

License

ambrosfitz/
mistral-nemo-humanities-gguf