You can deploy this model and start earning money today!

Xiejiehang/
ERNIE-Image-Turbo-MLX-Q4

Name: ERNIE-Image-Turbo-MLX-Q4
Availability: InStock

Views

No views yet

ERNIE-Image-Turbo-MLX-Q4

ERNIE-Image-Turbo-MLX-Q4 is a local inference distribution of ERNIE-Image-Turbo optimized for Apple Silicon. This directory contains ready-to-run MLX inference assets and minimal inference scripts, aiming to help developers perform high-quality text-to-image verification, prototyping, and engineering integration on macOS with lower VRAM consumption.

Compared to the original BF16 weights, this version quantizes the most memory-intensive modules—Transformer, Text Encoder, and Prompt Enhancement—to 4-bit, while keeping the VAE decoder at high precision to achieve a stable balance between deployment efficiency and generation quality.

Key Features

Apple Silicon Friendly: Built on MLX, specifically optimized for the Unified Memory Architecture of Apple M-series chips.
Massive Weight Compression: Core weights are compressed from ~30.56 GB to ~8.21 GB, making it ideal for local experiments and integration.
Preserved Strengths: Retains the core capabilities of ERNIE-Image-Turbo, including complex instruction following, long-form text rendering, and structured image generation.
Out-of-the-Box: Includes demo.py, a local scripts/ inference implementation, and all necessary configuration files. Developers can verify the model immediately.

Model Components

This distribution includes the following core components:

Component	Precision	Description
`transformer/`	Q4	Diffusion backbone (DiT), the primary compute and memory bottleneck.
`text_encoder/`	Q4	Text encoder responsible for prompt semantic modeling.
`pe/`	Q4	Prompt Enhancement module.
`vae/`	BF16	Image decoder, kept at high precision for high-fidelity output.
`tokenizer/`, `pe_tokenizer/`, `scheduler/`	Original	Necessary assets for the inference pipeline.

Use Cases

Rapidly verify ERNIE-Image-Turbo's local inference capabilities on Apple Silicon.
Provide a lightweight model base for integrating text-to-image features into applications.
Serve as a baseline for further Core ML conversion, service deployment, or on-device experiments.

Requirements

macOS
Apple Silicon Chip
Python 3.11+

Install dependencies:

pip install -r requirements.txt

The requirements.txt includes core dependencies for local execution:

mlx
mlx-lm
transformers
Pillow
numpy
safetensors

Quick Start

1. Run the Built-in Demo

Execute the following command in the current directory:

python demo.py

By default, this will:

Load the Q4 model assets from the current directory.
Use the prompt: "A cinematic poster of a futuristic city at sunrise, ultra detailed, realistic lighting".
Run 5 denoising steps.
Save the output to output.png.

2. Python Integration

To integrate this model into your own scripts, you can directly use the provided scripts.pipeline:

python

from pathlib import Path
from scripts.pipeline import generate, load_pipeline

# Load the pipeline from the current directory
model_dir = Path(".")
pipeline = load_pipeline(model_dir)

# Generate an image
image = generate(
    pipeline,
    prompt="A cinematic poster of a futuristic city at sunrise, ultra detailed, realistic lighting",
    steps=5,
    seed=42,
    height=1024,
    width=1024,
)

image.save("output.png")
print("Saved to output.png")

3. Recommended Parameters

For the best balance between quality and speed, we recommend:

Resolution: 1024x1024
Inference Steps: 8
Seed: Specify as needed for reproducibility.

For portrait or landscape orientations, you can explore:

848x1264 / 1264x848
768x1376 / 1376x768
896x1200 / 1200x896

Directory Structure

Path	Description
`demo.py`	Minimal runnable example.
`scripts/pipeline.py`	End-to-end loading and generation entry point.
`scripts/dit.py`	MLX implementation of the DiT backbone.
`scripts/vae.py`	VAE Decoder implementation.
`transformer/`	Quantized DiT weights and configuration.
`text_encoder/`	Quantized Text Encoder.
`pe/`	Quantized Prompt Enhancement module.
`vae/`	High-precision VAE decoder.

Experimental Results

We conducted local inference verification on an Apple M3 device. The core results are as follows:

Metric	Value
Model Load Time	`4.62 s`
Peak Memory Usage	`14.70 GB`
Text Encoding Time	`1.05 s`
Denoising Time	`221.19 s`
VAE Decoding Time	`9.28 s`
Total Generation Time	`231.51 s`

Notes:

Results were measured on an Apple M3 device with 24 GB Unified Memory.
Testing configuration: 1024x1024 resolution, 5 denoising steps.
Actual performance may vary based on device model, MLX version, prompt length, and step count.

Background

The base model, ERNIE-Image-Turbo, was released by the Baidu ERNIE-Image team and excels in:

Complex instruction following and multi-object relationships.
Long-form text rendering and layout-sensitive content.
Structured image generation (posters, comics, multi-panel layouts).
Diverse style coverage (realistic, design-oriented, stylized aesthetics).

This MLX Q4 version is intended as a lightweight deployment option for developers rather than a redefinition of the model's capabilities.

References

This project references and builds upon the following open-source works:

Baidu ERNIE-Image: For providing the original ERNIE-Image and ERNIE-Image-Turbo models and methods.
mlx-ernie-image: For providing critical references for weight organization and inference implementation on MLX.
mlx-lm: For providing MLX quantization and language model loading capabilities.
MLX: For providing an efficient array framework and neural network runtime on Apple Silicon.

Xiejiehang/ERNIE-Image-Turbo-MLX-Q4