Higher Resolution for Stunning Clarity! A Thorough Comparison of AI Upscalers and Hires.fix

Others #Hires.fix #Upscale

2025-4-122025-4-25

a female animated character with blonde hair and blue eyes is depicted in a close - up portrait wearing a white shirt brown blazer and blue tie set against a night sky with stars and glowing orbs conv

AI upscalers for clean enlargement
Upscalers also remove noise
Hires.fix adds detailed rendering

Introduction

Hello, I’m Easygoing.

This time, we’ll take a look at upscaling techniques for enhancing the quality of illustrations.

High Resolution is Beautiful!

High-resolution illustrations are stunning.

512 x 512

2048 x 2048

I previously wrote an article about high-resolution techniques, but I’d like to share an updated summary based on recent research.

What Resolution Should You Aim For? | AI image journey

Workflow

First, let’s go over a simple workflow for high-resolution upscaling.

upscale simple 2025.4.11.json

Interpolating Pixels!

When upscaling an illustration to a higher resolution, you need to interpolate the pixels in between.

Pixel Interpolation

Formula-based methods (traditional)
AI upscalers

There are two main approaches to pixel interpolation: the formula-based methods used traditionally and the increasingly popular AI upscalers.

Formula-Based Methods (Traditional)

Formula-based methods have long been used for pixel interpolation in high-resolution upscaling.

There are several types of formula-based methods, each with different impacts on image quality and computational cost.

Nearest Neighbor

Interpolates using the color of the nearest pixel
Low computational cost but remains jagged even at higher resolutions
Used for enlarging pixel art

Bilinear

Generates intermediate colors using the four adjacent pixels
Smooth but slightly blurry
Balances speed and quality, commonly used in browsers and general applications

Bicubic

Interpolates using 16 surrounding pixels (4x4 area)
Higher computational cost than Bilinear but produces sharper results
Suitable for enlarging detailed images like photographs

Lanczos

Advanced interpolation using a function, considering a wide range of pixels (usually 6x6 or more)
Maintains sharp edges while producing smooth results
Highest computational cost

The mathematical methods listed above offer higher quality as you move down the list, but they also require more computational power.

With modern smartphones boasting high computational capabilities, Lanczos is generally a safe choice for scaling static images.

a close - up of a female animated character with blue eyes and dark hair wearing a white shirt and black bowtie against a backdrop of a night sky with stars and a moon with a dateapril 1 6and the date_cleanup (2)

AI Upscaler Methods

Since the late 2010s, AI upscalers have been used for pixel interpolation.

AI upscalers are trained to learn from both original and downscaled images, enabling them to restore the original image from a downscaled version.

Emergence of AI Upscalers

gantt
title AI upscaler
dateFormat YYYY-MM-DD
tickInterval 24month
section CNN
SRCNN   :done, a1, 2014-09-02, 2025-04-11
section GAN
SRGAN   :b1, 2016-11-10, 2025-04-11
ESRGAN  :b2, 2018-09-01, 2025-04-11
BSRGAN  :b3, 2021-07-23, 2025-04-11
Real-ESRGAN  :b4, 2021-08-18, 2025-04-11
Real-CUGAN  :b5, 2022-02-07, 2025-04-11
section SwinIR
SwinIR  :done, c1, 2021-09-02, 2025-04-11
SwinFIR  :done, c2, 2023-10-10, 2025-04-11
section Diffusion Model
LDM  :d1, 2021-12-20, 2025-04-11
Stable Diffusion Upscaling  :d2, 2022-08-22, 2025-04-11
DiffBIR  :d3, 2023-08-29, 2025-04-11
section Transformer
VRT  :done, e1, 2022-01-01, 2025-04-11
CodeFormer :done, e2, 2022-06-22, 2025-04-11
HAT :done, e3, 2023-05-17, 2025-04-11

Many AI upscaler models are open-source, with the most popular being those based on ESRGAN.

AI Upscalers Have Fixed Magnification

AI upscalers have predetermined magnification levels (x2, x4, x8) based on the resolution they were trained on.

RealESRGAN_x2Plus: 2x model
RealESRGAN_x4Plus Anime 6B: 4x model

To achieve arbitrary resolutions with AI upscalers, you need to first upscale to the specified magnification and then downscale using methods like Lanczos.

Models That Can Restore Damage

While AI upscalers are trained on both original and downscaled images, some models are trained on damaged downscaled images, allowing them to restore damage.

Example: Models that remove JPEG artifacts

1x Kim2091 DeJpeg v0 - OpenModelDB

Even models not specifically designed for artifact removal have the ability to remove a certain level of noise.

Notable AI Upscalers

There are many AI upscaler models, but here are five distinctive models suitable for anime illustrations.

RealESRGAN_x4Plus Anime 6B

Anime-style rendering
High level of refinement

4x-UltraSharp

Sharp rendering
Crisp eyes

4x_IllustrationJaNai_V1_ESRGAN_135k

4x_NMKD-YandereNeoXL_200k

Balanced rendering

4x_RealisticRescaler_100000_G

Natural rendering

List of Upscale Models

Here is the list of AI upscale models investigated this time.

Comparison Table

Model	Type	License	Commercial Use	Features	Recommended
RealESRGAN_x4plus	ESRGAN	BSD 3-Clause	✅	Balanced	✅
RealESRGAN_x4plus_anime_6B	ESRGAN	BSD 3-Clause	✅	Anime Style	✅
4x-AnimeSharp	ESRGAN	CC-BY-NC-SA-4.0	❌	Sharp
4x-UltraSharp_150000	ESRGAN	CC-BY-NC-SA-4.0	❌	Sharp
4x_foolhardy_Remacri_210000	ESRGAN	CC-BY-NC-SA-4.0	❌	Sharp
4x_fatal_Anime_500000_G	ESRGAN	CC-BY-NC-SA-4.0	❌
4x_IllustrationJaNai_V1_ESRGAN_135k	ESRGAN	CC-BY-NC-SA-4.0	❌	Anime Style	✅
4x_NMKD-Superscale-SP_178000_G	ESRGAN	WTFPL	✅	Balanced
4x-NMKD-YandereNeo_320k	ESRGAN	WTFPL	✅	Balanced
4x_NMKD-YandereNeoXL_200k	ESRGAN	WTFPL	✅	Balanced	✅
4x_escale_100000_G	ESRGAN	WTFPL	✅
4x_RealisticRescaler_100000_G	ESRGAN	WTFPL	✅	Natural	✅
4x PSNR_Pretrained	ESRGAN	Apache-2.0	✅
4x_UniversalUpscalerV2-Neutral_115000_G	ESRGAN	WTFPL	✅
4x_UniversalUpscalerV2-Sharper_103000_G	ESRGAN	WTFPL	✅
4x_UniversalUpscalerV2-Sharp_101000_G	ESRGAN	WTFPL	✅
4x-PBRify_RPLKSRd_V3_160000	PLKSR	CC0-1.0	✅
OmniSR_X4_DIV2K	OmniSR	Apache-2.0	✅
4x-SwinIR-L_GAN	SwinIR	Apache-2.0	✅
4x-SwinIR-L_PNSR	SwinIR	Apache-2.0	✅
4xNomos2_hq_drct-l_200000	DRCT	CC-BY-4.0	✅
4x_IllustrationJaNai_V1_DAT2_190k	DAT	CC-BY-NC-SA-4.0	❌	Anime Style
4xNomos2_hq_dat2_140000	DAT	CC-BY-4.0	✅	Natural
4xNomos8kDAT_110000	DAT	CC-BY-4.0	✅	Natural
4xNomos8kHAT-L_otf_220000	HAT	CC-BY-4.0	✅	Natural

Where to Place Models

Downloaded models should be placed in one of the following folders:

models\upscale_models
Models\ESRGAN
models\ESRGAN

Upscaling with Hires.fix

The process of upscaling an illustration and then redrawing it is called Hires.fix.


flowchart LR
A1(Original Illustration)
subgraph Hires.fix
A2(Upscale)
A3(Redraw)
end
A1-->A2
A2-->A3

Hires.fix can be executed in ComfyUI and is implemented as a standard feature in Stable Diffusion webUI (Forge, reForge, A1111).

ComfyUI Sample Workflow

SDXL_Hires.fix 2025.4.21.json

Stable Diffusion webUI Forge Hires.fix Operation Screen

Stable Diffusion webUI Forge operation screen with comment

Hires.fix Can Also Use Latent Upscale

When using Hires.fix, the second redraw alters the illustration, making differences caused by AI upscalers less noticeable.

In fact, AI upscalers may remove noise during the process, so using a simple Latent upscale can sometimes result in better illustrations.

RealESRGAN_x4Plus Anime 6B

However, when using Latent upscale, you need to set a higher denoise value, and sufficient adjustments are necessary to prevent the illustration from degrading.

Splitting with Tiled Diffusion / Multi Diffusion!

Redrawing after upscaling often exceeds the model’s recommended resolution, leading to unstable outputs.

Recommended Resolutions

SD 1.5: 512 x 512
SDXL / SD 3.5 / AuraFlow: 1024 x 1024
Flux.1: 320 x 320 to 1440 x 1440

In such cases, a method called Tiled Diffusion / Multi Diffusion can stabilize output by splitting the image for processing.

ComfyUI Tiled Diffusion Node

Stable Diffusion webUI Forge Multi Diffusion

Using Tiled Diffusion / Multi Diffusion stabilizes rendering but may result in small figures appearing or a flat overall output.

brown 512 tile — Four-Part Division with Tiles

brown 256 — AI hasn’t been trained much on this composition

When splitting illustrations with Tiled Diffusion / Multi Diffusion, the resulting composition may be irregular for the AI, leading to some inevitable stability issues.

Finishing Comparison: AI Upscaler vs. Hires.fix

AI upscalers and Hires.fix each have their pros and cons for finishing.

Upscaler_3 — Left: AI Upscaler Right: Hires.fix
Model: noob_v_pencil-XL-v2.0.1

Hires.fix_107 — Left: AI Upscaler Right: Hires.fix
Model: noob_v_pencil-XL-v2.0.1

	AI Upscaler	Hires.fix
Noise Level	Low	High
Detail Rendering	Low	High
Accuracy of Details	Slightly Inaccurate	Accurate
Consistency	High	Somewhat Low

AI upscalers also remove noise, resulting in clean and clear illustrations.

On the other hand, Hires.fix leaves some noise from redrawing but produces illustrations with finely detailed rendering.

Personally, I think AI upscalers are great for backgrounds, while Hires.fix is better for characters. I achieve both using a Detailer, which I’ll introduce next time.