pip install mlx-vlm huggingface_hub

# (Optional) Pre-create the repo as private
hf repo create sunny-medgemma-1.5-4b-finetune-mlx-4bit --repo-type model --private

# Convert and upload
mlx_vlm.convert \
    --model mrdbourke/sunny-medgemma-1.5-4b-finetune \
    -q \
    --upload-repo mrdbourke/sunny-medgemma-1.5-4b-finetune-mlx-4bit

Usage

pip install mlx-vlm

Note on prompt order: This model expects <image> + <text> → <text> format. The image must come before the text in the prompt. The example below handles this automatically.

Python

Optional: Download a tester image.

wget -O test-skin-image-daniel-bourke.jpeg "https://github.com/mrdbourke/sunny/blob/main/images/test-skin-image-daniel-bourke.jpeg?raw=true"

Run the following on a Mac:

python

# On your Mac — save as test_sunny_mlx.py and run with: python test_sunny_mlx.py
import json
import urllib.request
from pathlib import Path
from mlx_vlm import load, generate

MODEL_ID = "mrdbourke/sunny-medgemma-1.5-4b-finetune-mlx-4bit"

# Download a test image (or replace with your own local path)
IMAGE_URL = "https://github.com/mrdbourke/sunny/blob/main/images/test-skin-image-daniel-bourke.jpeg?raw=true"
IMAGE_PATH = "test-skin-image.jpeg"

if not Path(IMAGE_PATH).exists():
    print(f"[INFO] Downloading test image...")
    urllib.request.urlretrieve(IMAGE_URL, IMAGE_PATH)

print(f"[INFO] Loading model: {MODEL_ID}")
model, processor = load(MODEL_ID)
print(f"[INFO] Model loaded!")

# Prompt format: <image> + <text> -> <text>
prompt = "<bos><start_of_turn>user\n<start_of_image>sunscreen extract<end_of_turn>\n<start_of_turn>model\n"

output = generate(
    model,
    processor,
    prompt,
    [IMAGE_PATH],
    max_tokens=512,
    temperature=0.7,
    top_p=0.95,
    repetition_penalty=1.2,
    verbose=True,
)

print(output)

Command line

bash

# Default test image
python -m mlx_vlm.generate \
    --model mrdbourke/sunny-medgemma-1.5-4b-finetune-mlx-4bit \
    --image-path test-skin-image.jpeg \
    --prompt "<bos><start_of_turn>user\n<start_of_image>sunscreen extract<end_of_turn>\n<start_of_turn>model\n" \
    --max-tokens 512

# Or pass your own image
python -m mlx_vlm.generate \
    --model mrdbourke/sunny-medgemma-1.5-4b-finetune-mlx-4bit \
    --image-path /path/to/your/image.jpeg \
    --prompt "<bos><start_of_turn>user\n<start_of_image>sunscreen extract<end_of_turn>\n<start_of_turn>model\n" \
    --max-tokens 512

Training procedure

This model was trained with SFT using TRL and then converted to MLX 4-bit quantized format via mlx-vlm.