🚀 Real-ESRGAN 128x128 for 4x Single Image Super-Resolution on AMD AI PC NPU
The Real Enhanced Super-Resolution Generative Adversarial Networks (Real-ESRGAN) model is an AI model that takes an input low-resolution image and creates high or "super-resolution" image. This version of the Real-ESRGAN model has been re-trained from scratch with reduced feature channels and fewer stacked blocks for improved efficiency. It was then quantized from FP32 to INT8 and optimized to run on the AMD AI PC NPU. This model takes an input image and scales it up by 4x.
The "128x128" in the title means that this model works on tile sizes 128x128, but almost any size input image can be upscaled by 4x. The inference pipeline tiles the input image into patches based on the ONNX model’s expected input resolution (with overlap), runs inference on each tile, and then stitches the results back together. A model with a larger tile size would lower the stitching overhead and may contain less boundary artifacts. Figure 1 shows an example of a 4x scaled image.
Input image
Output image
assets/input_tiger_320x480_108005.png
assets/output_tiger_4x_1280x1920_108005.png
Figure 1: Input 320x480 scaled up by 4x to 1280x1920 with Real-ESRGAN model running on AMD AI PC NPU. Source: EDSR Benchmark dataset (edsr_benchmark\B100\HR\108005.png).
The original model and architecture can be found on GitHub: xinntao/Real-ESRGAN.
Wang et al. (2018) introduced ESRGAN with the "Residual-in-Residual Dense Block (RRDB), without batch normalization as the basic network building unit." Figure 2 shows their original ESRGAN model architecture and Figure 3 shows their updated Real-ESRGAN (Wang et al., 2021) architecture.
esrgan_architecture
Figure 2: ESRGAN architecture with Residual in Residual Dense Block (RRDB) and removed batched normalization (BN) layers. Image from Fig. 4 of Wang et al. (2018).
real-esrgan_architecture
Figure 3: Real-ESRGAN architecture "adopts the same generator network as that in ESRGAN. For the scale factor of \(\times2\) and \(\times1\), it first employs a pixel-unshuffle operation to reduce spatial size and re-arrange information to the channel dimension." Image from Fig. 4 of Wang et al. (2021).
The model can be used to create high-resolution images from low-resolution images. The model has been converted to ONNX format and quantized for optimized performance on AMD AI PC NPUs.
Primary intended users
Anyone using or evaluating super-resolution models on AMD AI PCs.
Out-of-scope uses
This model is not intended for generating misinformation or disinformation, impersonating others, facilitating or inciting harassment or violence, any use that could lead to the violation of a human right.
How to Use
📐 Hardware Prerequisites
Before getting started, make sure you meet the minimum hardware and OS requirements:
Series
Codename
Abbreviation
Launch Year
Windows 11
Linux
Ryzen AI Max PRO 300 Series
Strix Halo
STX
2025
☑️
Ryzen AI PRO 300 Series
Strix Point / Krackan Point
STX/KRK
2025
☑️
Ryzen AI Max 300 Series
Strix Halo
STX
2025
☑️
Ryzen AI 300 Series
Strix Point
STX
2025
☑️
Getting Started
Follow the instructions here to download necessary NPU drivers and Ryzen AI software: Ryzen AI SW Installation Instructions. Please allow for around 30 minutes to install all of the necessary components of Ryzen AI SW.
Activate the previously installed conda environment from Ryzen AI (RAI) SW, and set the RAI environment variable to your installation path. Substitute the correct RAI version number for v.v.v, such as 1.7.1.
Install the necessary packages into the existing conda environment:
pip install -r requirements.txt
Data Preparation (optional: for evaluation).
Download the EDSR benchmark dataset extract it into the datasets/ directory. Note that you will need to run this script twice, as it seems to fail on first attempt.
--input: Accepts either a single image file path or a directory path. If it's a file, the script will process that image only. If it's a directory, the script will recursively scan for .png, .jpg, and .jpeg files and process all of them.
--out-dir: Output directory where the restored images will be saved.
--device: Accepts "npu" or "cpu". The NPU will attempt to use the VitisAIExecutionProvider; the CPU will attempt to use the CPUExecutionProvider. Note that to use the NPU, the updated NPU drivers and Ryzen AI SW must first be installed.
The model has already been compiled and cached under modelcachekey_realesrgan_nchw_128x128_u8s8, but if this folder is not present, the model will be recompiled and then inference can be run.
Evaluate the accuracy of the model on benchmark datasets (optional).
Eval on Set14. Enabling the -clean option will remove generated SR images.
The AMD ONNX model results were evaluated with the DIV2K and EDSR (B100, Urban100, Set14, Set5) datasets on peak signal-to-noise ratio (PSNR), multi-scale structural similarity (MS-SSIM), and Fréchet Inception Distance (FID) (see Table 1).
Figure 4 shows their perceptual quality results as compared to other state-of-the-art models.
wangetal2021_fig7
Figure 4: "Qualitative comparisons on several representative real-world samples with upsampling scale factor of 4. Our Real-ESRGAN outperforms previous approaches in both removing artifacts and restoring texture details. Real-ESRGAN+ (trained with sharpened ground-truths) can further boost visual sharpness. Other methods may either fail to remove overshoot (the 1st sample) and complicated artifacts (the 2nd sample), or fail to restore realistic and natural textures for various scenes (the 3rd, 4th, 5th samples)". Image and caption from Fig. 7 of Wang et al. (2021).
However, the Real-ESRGAN data were synthetically generated through a preprocessing workflow to degrade the images involving blur, downsampling, noise, and compression.
Real-ESRGAN is finetuned from ESRGAN for faster convergence, for 400K iterations and a learning rate of \(1\times10^{-4}\). "RealESRGAN is trained with a combination of L1 loss, perceptual loss and GAN loss, with weights {1,1,0.1}, respectively" (Wang et al., 2021). For more detailed information on training, see their paper.
📝 Quantitative Analyses
Table 1 shows the accuracy metrics for the AMD ONNX models of Real-ESRGAN.
Model
Set5
Set14
B100
Urban100
DIV2K
PSNR(↑)
MS_SSIM(↑)
FID(↓)
PSNR(↑)
MS_SSIM(↑)
FID(↓)
PSNR(↑)
MS_SSIM (↑)
FID(↓)
PSNR(↑)
MS_SSIM(↑)
FID(↓)
PSNR(↑)
MS_SSIM(↑)
FID(↓)
128x128(fp32)
23.43
0.9346
114.31
22.38
0.8928
141.12
23.17
0.8804
134.00
20.02
0.8813
52.44
23.96
0.9096
29.79
128x128(int8)
23.99
0.9387
97.89
22.65
0.8942
137.35
23.37
0.8817
131.91
20.51
0.8861
49.88
24.26
0.9103
27.46
256x256(fp32)
23.44
0.9348
112.65
22.40
0.8932
139.71
23.21
0.8809
133.87
20.01
0.8815
52.09
23.96
0.9098
29.32
256x256(int8)
23.90
0.9386
101.03
22.62
0.8949
135.43
23.28
0.8821
128.82
20.44
0.8861
48.76
24.14
0.9099
27.33
512x512(fp32)
23.44
0.9348
112.65
22.40
0.8932
139.71
23.21
0.8809
133.87
20.01
0.8815
51.97
23.97
0.9099
29.02
512x512(int8)
23.37
0.9303
117.11
22.29
0.8921
138.18
23.05
0.8796
128.34
19.96
0.8773
49.70
23.79
0.9024
25.40
1024x1024(fp32)
23.44
0.9348
112.65
22.40
0.8932
139.71
23.21
0.8809
133.87
20.01
0.8815
51.97
23.97
0.9099
28.98
1024x1024(int8)
23.10
0.9249
113.23
22.10
0.8835
140.06
22.82
0.8692
130.24
19.80
0.8710
50.43
23.42
0.8932
27.59
Table 1: Model accuracy metrics for AMD AI PC FP32 and INT8 quantized models.
The following are the performance results of 6 super-resolution models on the Strix machine NPU (Table 2).
Table 3: "NIQE scores on several diverse testing datasets with real-world images. The lower, the better." From Table 1 in Wang et al. (2021).
⚓ Ethical Considerations
AMD is committed to conducting our business in a fair, ethical and honest manner and in compliance with all applicable laws, rules and regulations. You can find out more at the AMD Ethics and Compliance page.
⚠️ Caveats and Recommendations
Wang et al. (2021) note that there are limitations with the Real-ESRGAN model, including aliasing, introduction of unpleasant artifacts, and the inability to remove complicated degradations.
📌 Citation Details
bibtex
1@InProceedings{wang2021realesrgan,
2 author = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
3 title = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
4 booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
5 date = {2021}
6}