# Install mlx-lm
pip install mlx-lm

# Generate directly from HF Hub
mlx_lm.generate \
  --model bisonnetworking/medgemma-health-chat-mlx-4bit \
  --prompt "I've had a sore throat and low-grade fever for 3 days. What should I do?" \
  --system-prompt "You are a board-certified Primary Care Physician. Provide direct, clinical guidance. No tables. No AI disclaimers."

# Interactive chat
mlx_lm.chat --model bisonnetworking/medgemma-health-chat-mlx-4bit

With Python

python

from mlx_lm import load, generate

model, tokenizer = load("bisonnetworking/medgemma-health-chat-mlx-4bit")

messages = [
    {
        "role": "system",
        "content": "You are a board-certified Primary Care Physician. Provide direct, clinical guidance. No tables. No AI disclaimers.",
    },
    {
        "role": "user",
        "content": "I've had a sore throat and low-grade fever (100.4) for 3 days. No cough, no swollen lymph nodes. What should I do?",
    },
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=256, temp=0.7)
print(response)

As an OpenAI-Compatible Server

bash

mlx_lm.server --model bisonnetworking/medgemma-health-chat-mlx-4bit --port 8080

# Then use with any OpenAI client
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "medgemma-health-chat",
    "messages": [
      {"role": "system", "content": "You are a Primary Care Physician..."},
      {"role": "user", "content": "I have a sore throat. What should I do?"}
    ]
  }'

Quantization Details

Tool: mlx-lm 0.31.3
Bits: 4
Group size: 64
Mode: affine
Converted from: bisonnetworking/medgemma-health-chat-merged (F16)

MLX 4-bit quantization uses uniform group quantization — weights are divided into groups of 64 and each group is quantized to 4 bits with an affine scale and bias. This is optimized for Apple Silicon's unified memory architecture and Metal GPU acceleration.

Personas

The model supports 6 medical personas via system prompts:

Persona	Specialty
Primary Care	General practice, common conditions
Internal Medicine	Complex adult medicine, multi-system disorders
Clinical Nutritionist	Dietary interventions, nutritional therapy
Exercise Specialist	Therapeutic exercise, sports performance
Best Doctor	Cross-specialty integration, comprehensive care
Chronic Health	Chronic illness management, diagnostic mysteries

Related Models

MLX 8-bit — higher quality, larger
GGUF Q4_K_M — for llama.cpp/Ollama
Merged 16-bit — full precision
LoRA adapter — for PEFT
Training dataset

Limitations

4-bit quantization introduces minor precision loss vs the F16 original
Only runs on Apple Silicon (M1/M2/M3/M4 or later)
Not a substitute for professional medical advice
MedGemma base model is gated — users need approved access to the original model

Model Card Contact

bisonnetworking