Component	Original (md3p-int4)	This Model
MoE Experts (layers 4-23)	int4	int4 (unchanged)
Vision Encoder	BF16	int4
Text Attention	BF16	int4
Text MLP (layers 0-3)	BF16	int4
Embeddings	BF16	int4
Total Size	6.48 GB	5.43 GB

Quantization Details

Method: Affine quantization (bits=4, group_size=64)
Selective: Only BF16 tensors were quantized; existing int4 MoE weights preserved
Some layers kept at BF16: Vision fc2 layers (shape incompatible with group_size=64)

Usage

This model is designed for use with moondream-mlx Swift implementation.

swift

let config = ModelConfiguration(id: "lewi/md3p-int4-smol")
let container = try await Moondream3Loader.loadContainer(configuration: config)

Source & License

Base Model: moondream/md3p-int4
Original Model: moondream/moondream3-preview
License: Apache 2.0 (same as original)

Changes from Base Model

This model applies additional int4 quantization to the BF16 components (vision encoder, text attention, embeddings) that were left unquantized in the original md3p-int4 release. This reduces model size from 6.48GB to 5.43GB, enabling deployment on iOS devices with ~6GB memory limits.

Acknowledgments

Thanks to the Moondream team for the original model and Apache 2.0 license.