Why upscaling an image can be useful:
Here's an example:
BeforeAfter
Here's a simple tutorial:
What you need:
A computer with a GPU listed here:
Not working on android (Tested with termux)
The CPU doesn't have to be something too powerful, just be sure it doesn't bottleneck your GPU (example: R5 5600x / Rx 6650; i7-6700k / 3070; not an i3-2100 paired with a 4090ti)
The RAM doesn't have to be a lot either, 512MB should probably be fine
An SSD is not required, an HDD will be ok
The Screen is a component that don't have any influence on the performance, it just changes the way you are going to see the results once finished
The Operating System
This guide provides an example for Debian-like and Arch-like GNU/Linux distros, Windows is not actually included in this guide, I might add that in future
The actual guide
1. Download and extract this and put it at a comfortable position since you'll have to type the path to the executable
2. Once extracted proceed by installing ffmpeg
On Debian
# apt-get install ffmpeg
On Arch
# pacman -S ffmpeg
3. Wait for the download to finish then and split the video in "frames"
Derbian / Arch
ffmpeg -i path/to/video.mkv -qscale:v 1 -qmin 1 -qmax 1 -vsync 0 path/to/frame/output/frame%08d.png
*Remember to create two separate folder for tmp splitted frame and the upscaled one.
4. Upscale the single frames
Debian / Arch
./path/to/realesrgan-ncnn-vulkan -i path/to/temp/frame -o path/to/upsclaed/frame -n realesr-animevideov3 -s 4 -f png
* -s can be 2 / 4, it's the scale the alghorithms do, doesn't affect too mutch the speed.
5. Merging files
This can be done with various methods, depending on your PC specs:
Low-end GPUs (Nvidia 1060, Intel Iris Xe, Nvidia 3050)
ffmpeg -i path/to/upscaled/frames/frame%08d.png -i path/to/original/video.mkv -map 0:v:0 -map 1:a:0 -c:a copy -c:v libx264 -crf 0 -preset slow -r 23.98 -pix_fmt yuv420p output.mkv
Mid-end GPUs (Nvidia 2070, Nvidia 3060, AMD Rx 6650)
ffmpeg -i path/to/upscaled/frames/frame%08d.png -i path/to/original/video.mkv -map 0:v:0 -map 1:a:0 -c:a copy -c:v libx265 -crf 0 -preset slow -r 23.98 -pix_fmt yuv420p output.mkv
High-end GPUs (Nvidia 1080ti, Nvidia 3080, AMD Rx 6850)
ffmpeg -i path/to/upscaled/frames/frame%08d.png -i path/to/original/video.mkv -map 0:v:0 -map 1:a:0 -c:a copy -c:v libvpx-vp9 -crf 0 -r 23.98 -pix_fmt yuv420p output.mkv
Super-end GPUs (Nvidia Quadro 6000, Nvidia 3090ti / 4090, AMD Rx 7900 XTX)
ffmpeg -i path/to/upscaled/frames/frame%08d.png -i path/to/original/video.mkv -map 0:v:0 -map 1:a:0 -c:a copy -c:v libaom-av1 -crf 0 -r 23.98 -pix_fmt yuv420p output.mkv
*Some clarification:
The input of the original file is for the audio, is also possible to change the audio format by changing for example: -c:a copy => -c:a flac
H.264, H.265, VP9, and AV1 are video ancoder, one does the exact same thing as eachother, for example AV1 is 50% more compressed of H.265 and 30% more compressed of VP9 , and will take more time and power.
-crf 0 is the lossy quality, 0 stays for lossless, 21 for lossy (default), and 51 as worst as possible, lossless also means very huge file size, use -crf 21 as a good option between quality and file size
-preset slow is the time that you want to take to be an encode more efficient, but still following the -crf value (ultrafast, superfast, veryfast, faster, fast, medium "default", slow, slower, veryslow, placebo)
-r 23.98 is the frame rate, in this case of an anime
-pix_fmt yuv420p is the pixel format
For the images
The OS / specs required are the same
1. Convert the image to PNG (can be skipped if already is)
ffmpeg -i path/to/img path/to/output.png
2. Upscale the image
./path/to/realesrgan-ncnn-vulkan -i path/to/img -o path/to/output -n realesrgan-x4plus-anime -s 4 -f png
*Some clarification:
It may take some time with big images
-s 2 / 4 is the scale, influences a bit the performance