# Download a quantized version
huggingface-cli download bisonnetworking/bison-medical-v3-1.7b-gguf model-Q4_K_M.gguf --local-dir ./models

# Run inference
./llama.cpp/llama-cli -m ./models/model-Q4_K_M.gguf -p "What are the symptoms of diabetes?"

With Ollama

bash

# Create Modelfile
cat > Modelfile <<EOF
FROM ./models/model-Q4_K_M.gguf
SYSTEM "You are a medical expert assistant. Provide accurate, evidence-based medical information."
EOF

# Create model
ollama create bison-medical-v3 -f Modelfile

# Run
ollama run bison-medical-v3 "What are the symptoms of diabetes?"

With LM Studio

Download any .gguf file from this repository
Open LM Studio
Load the model from the downloaded file
Start chatting!

Training Details

See bisonnetworking/bison-medical-v3-1.7b for full training details.

License

Apache 2.0 (same as base model)