er6y/

FLUX.2-klein-4B-MNN-int8

Overview

• 🎨 T2I Mode: Generate images from text prompts (no input image)
• ✏️ Edit Mode: Edit existing images based on text instructions (with input image)

• 🎨 Text-to-Image Generation: Create images from text descriptions
• 🖌️ Global editing: Modify overall image style, atmosphere, or theme
• ✏️ Local editing: Edit specific regions or objects
• 📝 Text modification: Change text content in images
• 🖼️ Reference-guided editing: Edit based on reference images
• 🔄 Multi-turn editing: Perform sequential edits while preserving consistency

What's included

• text_encoder.mnn / text_encoder.mnn.weight
• unet.mnn / unet.mnn.weight (denoiser/transformer; diffusion_demo uses the unet name)
• vae_encoder.mnn / vae_decoder.mnn
• tokenizer.txt
• config.json: resource description (filenames, precision labels, default inference parameters, etc.)
• configuration.json: generic task description (used by some loaders)
• scheduler_config.json: FlowMatchEulerDiscreteScheduler config used by FLUX.2-klein

How to export / generate

1. Export ONNX
   • text_encoder.onnx, unet.onnx, vae_encoder.onnx, vae_decoder.onnx
   • IO names expected by the MNN FLUX.2-klein pipeline:
     • text_encoder: inputs input_ids, attention_mask; output last_hidden_state
     • unet: inputs sample, timestep (float), encoder_hidden_states; output out_sample
     • vae_decoder: input latent_sample; output sample
2. Convert ONNX → MNN (example 4/8/8):
   bash

   1MNNConvert -f ONNX --modelFile text_encoder.onnx --MNNModel text_encoder.mnn --weightQuantBits 4 2MNNConvert -f ONNX --modelFile unet.onnx --MNNModel unet.mnn --weightQuantBits 8 3MNNConvert -f ONNX --modelFile vae_encoder.onnx --MNNModel vae_encoder.mnn --weightQuantBits 8 4MNNConvert -f ONNX --modelFile vae_decoder.onnx --MNNModel vae_decoder.mnn --weightQuantBits 8

How to run

Windows (PowerShell / cmd) — FLUX.2-klein.bat

• If 2nd argument is an existing file → Edit mode (input image provided)
• If 2nd argument is a number → T2I mode with batch count (no input image)
• If 2nd argument is empty → T2I mode, single image (no input image)

• MODEL_DIR (default .): model directory (relative or absolute)
• MEMORY_TYPE (0/1/2): Diffusion memory mode (0=memory lack, 1=memory enough, 2=balance)
• BACKEND (0=cpu, 3=opencl, 7=vulkan)
• STEPS: diffusion steps (4–20 recommended; default 8)
• SEED: 0=auto random; non-0=fixed seed
• SIZE: 512/640/768/896/1024 (default 1024)
• CFG: classifier-free guidance scale
  • T2I mode: 1.0–8.0 (default 1)
  • Edit mode: 1.0–6.0 (default 1, auto-adjusted)
• CFG_MODE: CFG sigma range for dual-UNet models (LongCat only)
  • 0=auto(0.10.8), 1=wide(0.10.9), 2=standard(0.1~0.8)
  • 3=medium(0.150.7), 4=narrow(0.20.6), 5=minimal(0.25~0.5)
• GPU_MEM_MODE (OpenCL only): 0=auto, 1=buffer, 2=image
• PRECISION (0=auto, 1=FP16, 2=FP32 normal, 3=FP32 high)
• TE_ON_CPU (0 same as UNet, 1 forces text_encoder on CPU; default 1)
• VAE_ON_CPU (0 same as UNet, 1 forces VAE on CPU for GPU memory saving; default 0)

Linux/macOS — FLUX.2-klein.sh

• MODEL_DIR
• MEMORY_TYPE
• BACKEND (default 3=OpenCL on Linux/macOS)
• STEPS (default 8)
• SEED
• SIZE (default 1024)
• CFG (T2I: 1, Edit: 1, auto-adjusted)
• CFG_MODE (default 0=auto)
• GPU_MEM_MODE
• PRECISION
• TE_ON_CPU (default 1)
• VAE_ON_CPU (default 0)

Building MNN for Windows

Model Type

• model_type=5: FLUX.2-klein (supports both T2I and image editing)
  • T2I mode: Activated when no input image is provided
  • Edit mode: Activated when input image is provided as the last parameter

Notes

• Weight files (.mnn.weight) are large; use Git LFS/external storage when sharing.
• MNN_BUILD_DIFFUSION=ON is required for diffusion model support
• MNN_BUILD_LLM=ON is required for LongCat's multimodal text encoder (vision + LLM)
• MNN_BUILD_OPENCV=ON is recommended for better image I/O support
• Custom MNN Fork: This model requires the specialized MNN fork with LongCat support. Use: https://github.com/er6y/MNN
• The same model weights support both T2I and Edit modes - no need for separate downloads

Model Information

• Base Model: black-forest-labs/FLUX.2-klein-4B
• GitHub: flux2
• Hugging Face: black-forest-labs/FLUX.2-klein-4B

Key Features

• 🌟 Dual-Mode Support: Both T2I generation and image editing in one model
• 🌟 Auto-Detection: Scripts automatically detect mode based on input parameters
• 🌟 Superior Precise Editing: Supports global editing, local editing, text modification, and reference-guided editing
• 🌟 Consistency Preservation: Strong consistency preservation in non-edited regions during multi-turn editing
• 🌟 Bilingual Support: Native Chinese and English instruction following
• 🌟 State-of-the-art Performance: Leading performance among open-source image editing models
• 🌟 Efficient Inference: Optimized for fast inference with MNN quantization
• 🌟 Advanced CFG Control: Dual-UNet architecture with configurable CFG sigma ranges