AI upscaling filters are the most interesting ones. They increase the video resolution while also returning details and making the video much sharper than before.

To dive right in, check out the video tutorial about AI Upscale:
​

Video is made out of individual frames, and each frame is composed of pixels. If we take the number of horizontal pixels (video width) and multiply it by the number of vertical pixels (video height), we get the video resolution.

Resolution is usually expressed in the format W x H, e.g. 1920x1080 or 1280x720.

In most cases, the higher the video resolution, the more details your video will have. Next, we'll describe why sometimes higher resolution doesn't mean better quality.

Video resolution upscaling is not a new thing. It's been present ever since we first got access to digital videos.

So what changed? In the beginning, upscaling worked in a very simple way. Let's say we have a video of resolution 2x2 pixels - a very small video indeed :)

Let's use a very simple upscaling algorithm to upscale the 2x2 video to 8x8 -> 4x upscaling!

The new resolution is now much larger, but there's a problem. Lots of the pixels were simply copied to fill the new pixels. While we increased the resolution, we didn't add much new information.

In the real world, we use a bit more complicated upscaling algorithms than the one described. The most famous ones are the bilinear and bicubic upscaling algorithms. Let's now see how the situation we described above plays out in a real video frame.

Here's a small video frame that has a resolution of 200x170 pixels. Let's upscale by 4x to 800x680 px using the bicubic upscaling algorithm.

Again, the resolution is now much larger, but you can see how the frame looks blurry and lacks details. This is due to the limitations of the bicubic algorithm. The algorithm is very fast to run so you can easily run it even on your mobile phone, but the results are not that great.

AI upscaling works in a completely different way that enables us to get much more details when we upscale.

Here's the same 4x upscaled frame, but now using the AI upscale algorithm. You can see how AI brings back many details such as freckels and the hair strands. The upscaled version also doesn't look as blurry as before.

Now that you know the advantages of using AI upscaling methods instead of non-AI upscaling methods, let's see what kind of upscaling you can do in the TensorPix app.

This AI filter doubles the width and height of the video, e.g.:

This filter is faster (and cheaper) than the 400% upscale and should be applied to make video resolution larger than 1280x720. E.g. if the video resolution is 800x600, we should apply the 200% upscale as it will make the video 1600x1200px.

Quadruples the width and height of the video, e.g.:

400% upscale is best applied for videos that have a resolution lower than ~500x500 px. Keep in mind this filter is much slower than the 200% upscale as it has to compute many more new pixels.

Well, the more pixels the better, right?

Yes, but the 400% is much slower and more expensive than the 200% upscale. There are also diminishing returns as we increase the upscaling factor.

Increasing the upscaling factor means the AI has a still harder job in re-creating new details. We found out that upscaling factors above 4x don't produce better results, but only make the upscaling process very slow.

That’s why it doesn’t make much sense to use 400% upscale on a Full HD video (1920x1080px). While it will produce 7680x4320px output video, it won’t have nearly as many new details.

You noticed there multiple variants of the AI Upscale filter:

Differences between these variants are in the speed of processing, cost, and the output quality of the AI Upscale filter.

The Light variant is faster and cheaper, but it won't create as many details as the Pro variant.
​
It's hard to say how much better the Pro version is in terms of quality as it varies from video to video. You should use the Pro version if you need all the details you can get. The Pro version is also better if the source is of very bad quality.

The Light version should be enough if the original video already looks good in terms of quality, but you want to get a bit of extra details while increasing the resolution.

We recommend that you always start with the Light variant as you will see the results sooner, and it will cost fewer Credits.

Video upscaling is an area of active research and we expect to develop better algorithms that produce more details for the same input videos. Improvement takes a lot of R&D so it won’t happen quickly.