er6y/

LongCat-Image-Edit-Turbo-MNN-int8

Overview

• 🎨 T2I Mode: Generate images from text prompts (no input image)
• ✏️ Edit Mode: Edit existing images based on text instructions (with input image)

• 🎨 Text-to-Image Generation: Create images from text descriptions
• 🖌️ Global editing: Modify overall image style, atmosphere, or theme
• ✏️ Local editing: Edit specific regions or objects
• 📝 Text modification: Change text content in images
• 🖼️ Reference-guided editing: Edit based on reference images
• 🔄 Multi-turn editing: Perform sequential edits while preserving consistency

What's included

• text_encoder/ directory containing:
  • visual.mnn / visual.mnn.weight (vision encoder for image understanding, int8 quantized)
  • llm.mnn / llm.mnn.weight (language model for text processing, int4 quantized)
  • embeddings_bf16.bin (text embeddings)
  • tokenizer.txt (tokenizer vocabulary)
  • config.json (text encoder configuration with model metadata and precision info)
• unet.mnn / unet.mnn.weight (diffusion transformer, int8 quantized)
• vae_encoder.mnn / vae_decoder.mnn (VAE for image encoding/decoding, fp16)
• config.json: resource description (filenames, precision labels, default inference parameters, etc.)
• configuration.json: generic task description
• scheduler_config.json: FlowMatchEulerDiscreteScheduler config

• Visual encoder: int8
• LLM: int4
• Transformer (UNet): int8
• VAE: fp16

How to export / generate

1. Export ONNX from PyTorch model
   • visual.onnx, llm.onnx, unet.onnx, vae_encoder.onnx, vae_decoder.onnx
   • IO names expected by the MNN LongCat-Image-Edit pipeline:
     • visual: inputs pixel_values; output image_embeds
     • llm: inputs input_ids, attention_mask, image_embeds; output last_hidden_state
     • unet: inputs sample, timestep (float), encoder_hidden_states, image_latents; output out_sample
     • vae_encoder: input sample; output latent_sample
     • vae_decoder: input latent_sample; output sample
2. Convert ONNX → MNN:
   • Note: For the text_encoder (visual + llm), use the specialized export script from the custom MNN fork to ensure proper MNN structure support， others can use MNNConvert
   bash

   1 2MNNConvert -f ONNX --modelFile unet.onnx --MNNModel unet.mnn --weightQuantBits 8 3MNNConvert -f ONNX --modelFile vae_encoder.onnx --MNNModel vae_encoder.mnn --weightQuantBits 16 4MNNConvert -f ONNX --modelFile vae_decoder.onnx --MNNModel vae_decoder.mnn --weightQuantBits 16

How to run

Windows (PowerShell / cmd) — longcat-image-edit.bat

• If 2nd argument is an existing file → Edit mode (input image provided)
• If 2nd argument is a number → T2I mode with batch count (no input image)
• If 2nd argument is empty → T2I mode, single image (no input image)

• MODEL_DIR (default .): model directory (relative or absolute)
• MEMORY_TYPE (0/1/2): Diffusion memory mode (0=memory lack, 1=memory enough, 2=balance)
• BACKEND (0=cpu, 3=opencl, 7=vulkan)
• STEPS: diffusion steps (4–20 recommended; default 8)
• SEED: 0=auto random; non-0=fixed seed
• SIZE: 512/640/768/896/1024 (default 1024)
• CFG: classifier-free guidance scale
  • T2I mode: 1.0–8.0 (default 1)
  • Edit mode: 1.0–6.0 (default 1, auto-adjusted)
• CFG_MODE: CFG sigma range for dual-UNet models (LongCat only)
  • 0=auto(0.10.8), 1=wide(0.10.9), 2=standard(0.1~0.8)
  • 3=medium(0.150.7), 4=narrow(0.20.6), 5=minimal(0.25~0.5)
• GPU_MEM_MODE (OpenCL only): 0=auto, 1=buffer, 2=image
• PRECISION (0=auto, 1=FP16, 2=FP32 normal, 3=FP32 high)
• TE_ON_CPU (0 same as UNet, 1 forces text_encoder on CPU; default 1)
• VAE_ON_CPU (0 same as UNet, 1 forces VAE on CPU for GPU memory saving; default 0)

Linux/macOS — longcat-image-edit.sh

• MODEL_DIR
• MEMORY_TYPE
• BACKEND (default 3=OpenCL on Linux/macOS)
• STEPS (default 8)
• SEED
• SIZE (default 1024)
• CFG (T2I: 1, Edit: 1, auto-adjusted)
• CFG_MODE (default 0=auto)
• GPU_MEM_MODE
• PRECISION
• TE_ON_CPU (default 1)
• VAE_ON_CPU (default 0)

Building MNN for Windows

Model Type

• model_type=3: LongCat-Image-Edit (supports both T2I and image editing)
  • T2I mode: Activated when no input image is provided
  • Edit mode: Activated when input image is provided as the last parameter

Notes

• Weight files (.mnn.weight) are large; use Git LFS/external storage when sharing.
• MNN_BUILD_DIFFUSION=ON is required for diffusion model support
• MNN_BUILD_LLM=ON is required for LongCat's multimodal text encoder (vision + LLM)
• MNN_BUILD_OPENCV=ON is recommended for better image I/O support
• Custom MNN Fork: This model requires the specialized MNN fork with LongCat support. Use: https://github.com/er6y/MNN
• The same model weights support both T2I and Edit modes - no need for separate downloads

Model Information

• Base Model: meituan-longcat/LongCat-Image-Edit-Turbo
• Paper: LongCat-Image Technical Report
• GitHub: LongCat-Image
• Hugging Face: LongCat-Image-Edit

Key Features

• 🌟 Dual-Mode Support: Both T2I generation and image editing in one model
• 🌟 Auto-Detection: Scripts automatically detect mode based on input parameters
• 🌟 Superior Precise Editing: Supports global editing, local editing, text modification, and reference-guided editing
• 🌟 Consistency Preservation: Strong consistency preservation in non-edited regions during multi-turn editing
• 🌟 Bilingual Support: Native Chinese and English instruction following
• 🌟 State-of-the-art Performance: Leading performance among open-source image editing models
• 🌟 Efficient Inference: Optimized for fast inference with MNN quantization
• 🌟 Advanced CFG Control: Dual-UNet architecture with configurable CFG sigma ranges