You can deploy this model and start earning money today!

darkbit1001/
Stable-Diffusion-1.5-LCM-ONNX-RKNN2

Name: Stable-Diffusion-1.5-LCM-ONNX-RKNN2
Availability: InStock

TheyCallMeHex/LCM-Dreamshaper-V7-ONNX

quantized

Views

No views yet

Stable Diffusion 1.5 Latent Consistency Model (LCM-SD) for RKNN2

Run the Stable Diffusion 1.5 Latent Consistency Model (LCM-SD) on Rockchip RKNPU2 (RK3588) using RKNN2.

This repository supports command-line inference and a production-ready HTTP server optimized specifically for LCM-SD.

Performance (RK3588, single NPU core)

Resolution	Text Encoder	U-Net (per step)	VAE Decoder
384×384	~0.05s	~2.36s	~5.48s
512×512	~0.05s	~5.65s	~11–14s

NOTE: VAE decode latency is a known RKNN limitation and is not caused by layout, server, or postprocessing overhead.

LCM-SD Optimizations & Quirks (Specific to This Repo)

Correct tensor layouts:
- Text encoder: NCHW
- U-Net: NHWC
- VAE decoder: NHWC
All RKNN runtime auto-conversion warnings eliminated
One RKNN runtime context per worker (safe multi-context usage)
Deterministic generation via explicit numpy.RandomState(seed)
VAE decode slowness is a known RKNN behavior and unaffected by toolkit version

Command-Line Usage (LCM-SD Only)

python ./run_rknn-lcm.py -i ./model -o ./images --num-inference-steps 4 -s 512x512 --prompt "Majestic mountain landscape with snow-capped peaks, autumn foliage in vibrant reds and oranges, a turquoise river winding through a valley, crisp and serene atmosphere, ultra-realistic style."

LCM-SD HTTP Server

Start the Server (Command Line)

bash

export MODEL_ROOT=./model
export NUM_WORKERS=3
export PORT=4200

python lcm_server.py

The server listens on:

http://0.0.0.0:4200

Server Endpoints (LCM-SD Only)

POST /generate

Generate a PNG image using LCM-SD.

Request body (JSON):

json

{
  "prompt": "a cinematic forest at sunrise",
  "size": "512x512",
  "num_inference_steps": 4,
  "guidance_scale": 1.0,
  "seed": 1234
}

Response: • HTTP 200 • Content-Type: image/png • Binary PNG payload

curl Example (LCM-SD Server Only)

bash

curl -X POST http://node1.lan:4200/generate \
  -H "Content-Type: application/json" \
  -o output.png \
  -d '{
    "prompt": "a cinematic forest at sunrise",
    "size": "512x512",
    "num_inference_steps": 4,
    "guidance_scale": 1.0,
    "seed": 1234
  }'

Docker Usage (LCM-SD Server)

Build Image

bash

docker build \
  -t rknn-lcm-sd .

Run Container

bash

docker run --rm -it \
  --device /dev/dri \
  --device /dev/rknpu \
  -v ./model:/models \
  -e MODEL_ROOT=/models \
  -e NUM_WORKERS=3 \
  -p 4200:4200 \
  rknn-lcm-sd

Additionally, a docker-compose.yml is provided.

Model Conversion

Install dependencies

pip install diffusers pillow numpy<2 rknn-toolkit2

1. Download the model

Download a Stable Diffusion 1.5 LCM model in ONNX format and place it in the ./model directory.

bash

huggingface-cli download TheyCallMeHex/LCM-Dreamshaper-V7-ONNX
cp -r -L ~/.cache/huggingface/hub/models--TheyCallMeHex--LCM-Dreamshaper-V7-ONNX/snapshots/4029a217f9cdc0437f395738d3ab686bb910ceea ./model

In theory, you could also achieve LCM inference by merging the LCM Lora into a regular Stable Diffusion 1.5 model and then converting it to ONNX format. However, I'm not sure how to do this. If anyone knows, please feel free to submit a PR.

2. Convert the model

bash

# Convert the model, 384x384 resolution
python ./convert-onnx-to-rknn.py -m ./model -r 384x384 

Note that the higher the resolution, the larger the model and the longer the conversion time. It's not recommended to use very high resolutions.

Known Issues

As of now, models converted using the latest version of rknn-toolkit2 (version 2.2.0) still suffer from severe precision loss, even when using fp16 data type. As shown in the image, the top is the result of inference using the ONNX model, and the bottom is the result using the RKNN model. All parameters are the same. Moreover, the higher the resolution, the more severe the precision loss. This is a bug in rknn-toolkit2. (Fixed in v2.3.0)
Actually, the model conversion script can select multiple resolutions (e.g., "384x384,256x256"), but this causes the model conversion to fail. This is a bug in rknn-toolkit2.

darkbit1001/Stable-Diffusion-1.5-LCM-ONNX-RKNN2

Stable Diffusion 1.5 Latent Consistency Model (LCM-SD) for RKNN2

Performance (RK3588, single NPU core)

LCM-SD Optimizations & Quirks (Specific to This Repo)

Command-Line Usage (LCM-SD Only)

LCM-SD HTTP Server

Start the Server (Command Line)

Server Endpoints (LCM-SD Only)

POST /generate

curl Example (LCM-SD Server Only)

Docker Usage (LCM-SD Server)

Build Image

Run Container

Model Conversion

Install dependencies

1. Download the model

2. Convert the model

Known Issues

References

darkbit1001/
Stable-Diffusion-1.5-LCM-ONNX-RKNN2