This repository contains a LoRA adapter for Qwen-Image-Edit-2511, trained for converting portrait and landscape images into a traditional Chinese gongbi painting style.

The LoRA is designed for image-to-image editing. It preserves the original subject, composition, and spatial structure while applying fine-line drawing, layered color, mineral-pigment aesthetics, and classical gongbi-style visual features.

Primary use case: converting real or generated portrait / landscape images into a refined traditional Chinese gongbi painting style while keeping the original image structure stable.

Important Compatibility Notes

Compatibility notice: This LoRA was trained with DiffSynth-Studio.
It is not released as a standard PEFT / diffusers adapter directory.

It must be loaded with:

pipe.load_lora(pipe.dit, "qwen_image_edit_2511_gongbi_lora_v1.safetensors")

Do not directly use Hugging Face diffusers load_lora_weights() with this file. This LoRA is not released as a standard PEFT / diffusers adapter directory.

For Qwen-Image-Edit-2511 inference, you must also enable:

zero_cond_t=True

If zero_cond_t=True is not enabled, the output may show weak LoRA effects, unstable structure, or composition drift.

Base Model

Base editing model: Qwen/Qwen-Image-Edit-2511
Text encoder and VAE: Qwen/Qwen-Image
Processor: Qwen/Qwen-Image-Edit

Main File

qwen_image_edit_2511_gongbi_lora_v1.safetensors

Usage

1. Install DiffSynth-Studio

bash

git clone https://github.com/modelscope/DiffSynth-Studio.git
cd DiffSynth-Studio
pip install -e .
pip install accelerate modelscope safetensors pillow

2. Download the LoRA

Using Hugging Face CLI:

bash

pip install huggingface_hub

huggingface-cli download SanJin09/qwen-image-edit-gongbi-lora-v1 \
  qwen_image_edit_2511_gongbi_lora_v1.safetensors \
  --local-dir ./gongbi-lora

Or clone the repository with Git LFS:

bash

git lfs install
git clone https://huggingface.co/SanJin09/qwen-image-edit-gongbi-lora-v1

3. Minimal DiffSynth-Studio Python Example

python

import torch
from PIL import Image
from diffsynth.pipelines.qwen_image import ModelConfig, QwenImagePipeline

pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(
            model_id="Qwen/Qwen-Image-Edit-2511",
            origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors",
        ),
        ModelConfig(
            model_id="Qwen/Qwen-Image",
            origin_file_pattern="text_encoder/model*.safetensors",
        ),
        ModelConfig(
            model_id="Qwen/Qwen-Image",
            origin_file_pattern="vae/diffusion_pytorch_model.safetensors",
        ),
    ],
    tokenizer_config=None,
    processor_config=ModelConfig(
        model_id="Qwen/Qwen-Image-Edit",
        origin_file_pattern="processor/",
    ),
)

pipe.load_lora(
    pipe.dit,
    "qwen_image_edit_2511_gongbi_lora_v1.safetensors",
    alpha=1.0,
)

prompt = (
    "Convert the input image into traditional Chinese gongbi painting style, "
    "preserving the original composition, subject details, and spatial structure."
)

edit_image = Image.open("input.png").convert("RGB").resize((1024, 1024))

output = pipe(
    prompt,
    edit_image=edit_image,
    seed=123,
    num_inference_steps=40,
    height=1024,
    width=1024,
    zero_cond_t=True,
)

output.save("output_gongbi.png")

Recommended Prompts

Portrait

Convert the input portrait into traditional Chinese gongbi figure painting style. Preserve facial identity, pose, clothing structure, hands, hair, and original composition. Use precise fine-line drawing, refined color layers, soft mineral pigments, and classical figure-painting elegance.

Landscape

Convert the input landscape into traditional Chinese gongbi landscape painting style. Preserve mountain shapes, water paths, trees, rocks, buildings if present, depth, and original composition. Use delicate outlines, fine texture strokes, layered ink washes, and elegant mineral color.

General

Convert the input image into traditional Chinese gongbi painting style, preserving the original composition, subject details, and spatial structure.

Recommended Inference Parameters

Parameter	Recommended Value	Notes
`height` / `width`	`1024`	Recommended default resolution
`num_inference_steps`	`40`	Balanced quality and speed
`seed`	`123` or fixed seed	Useful for reproducible comparisons
`zero_cond_t`	`True`	Required for Qwen-Image-Edit-2511
LoRA alpha	`1.0`	Try `0.7–0.9` if the style is too strong

This LoRA is intended for image-to-image editing. It requires an input image through edit_image.

Training Summary

Item	Value
Dataset	403 image-edit pairs
Categories	Portrait and landscape
Epochs	2
Dataset repeat	10
LoRA rank	16
Max pixels	524288
Training framework	DiffSynth-Studio
Checkpoint format	DiffSynth-Studio `.safetensors`

File Information

File	Size	SHA256	MD5
`qwen_image_edit_2511_gongbi_lora_v1.safetensors`	236,116,888 bytes	`c9b8a7aca1fc94892b750e32d10091b93f862c8f454b91d912b9ce22fb68c49a`	`2561802c368838308b34062c8ea05269`

You can verify the file with:

sha256sum qwen_image_edit_2511_gongbi_lora_v1.safetensors

Expected SHA256:

c9b8a7aca1fc94892b750e32d10091b93f862c8f454b91d912b9ce22fb68c49a

Suggested Example Display

For best presentation, upload example images to this model card in the following format:

Input Image	LoRA Output	Optional Reference Target
Original portrait image	Generated gongbi-style portrait	Optional style reference
Original landscape image	Generated gongbi-style landscape	Optional style reference

If a reference target is shown, it should be clearly marked as a training or style-supervision reference, not as the model output.

Recommended examples:

Portrait: 0001_input.png → LoRA output
Landscape: 0204_input.png → LoRA output

Limitations

This LoRA may perform less reliably in the following cases:

Extremely blurry, overexposed, or low-resolution inputs
Very complex human hands, faces, or crowded group portraits
Extreme camera angles or heavily distorted perspective
Inputs with conflicting style prompts such as "anime", "oil painting", or "photorealistic"
Very wide or very tall aspect ratios far from the training distribution

For more stable results, use clear input images, keep the original composition simple, and use the portrait or landscape prompt that matches the subject type.

FAQ

Can I use this LoRA directly with diffusers?

No. This LoRA is trained in DiffSynth-Studio format and should be loaded with:

pipe.load_lora(pipe.dit, path)

It is not a standard diffusers load_lora_weights() adapter.

Why does the output look unchanged?

Make sure the LoRA is loaded into pipe.dit and that zero_cond_t=True is enabled during inference.

Why does the composition drift?

Check that zero_cond_t=True is enabled. Also avoid prompts that conflict with gongbi painting, such as anime, oil painting, or photorealistic style terms.

Is this a text-to-image LoRA?

No. This LoRA is intended for image-to-image editing. You must provide an input image through edit_image.

How much VRAM is recommended?

For 1024×1024 inference with bf16, around 24GB VRAM is recommended. CPU inference is not recommended.

Environment

The LoRA was tested in a DiffSynth-Studio based environment.

Recommended environment:

Component	Recommendation
Python	3.10+
DiffSynth-Studio	2.x
PyTorch	2.x
CUDA	12.x
GPU	Around 24GB VRAM recommended for 1024×1024 inference

Citation

If you use this LoRA in a project, you can cite or link this repository:

text

SanJin09/qwen-image-edit-gongbi-lora-v1
A DiffSynth-Studio LoRA for Qwen-Image-Edit-2511 gongbi-style image editing.

SanJin09/qwen-image-edit-gongbi-lora-v1

Qwen-Image-Edit-2511 Gongbi LoRA v1

Overview

Important Compatibility Notes

Base Model

Main File

Usage

1. Install DiffSynth-Studio

2. Download the LoRA

3. Minimal DiffSynth-Studio Python Example

Recommended Prompts

Portrait

Landscape

General

Recommended Inference Parameters

Training Summary

File Information

Suggested Example Display

Limitations

FAQ

Environment

Related Links

Citation

SanJin09/
qwen-image-edit-gongbi-lora-v1