In Depth: Upscaling Video (Updated 8 Feb 2024)
What does upscale mean?
Upscaling means stretching a smaller video to fit a larger screen. The goal is to increase the size of the video while maintaining quality. Other terms you may hear are upsizing and uprezzing. Sure, you can simply scale the footage, but you will often get stretched and blurry results. You can also upscale with software and plugins that have features to sharpen and clean up problems including blur, jagginess, color artifacting, and noise. This article covers the tools to upscale your footage with great results.
What’s in this article?
Why would you want to upscale your footage?
There are two common reasons to upscale:
- You work with old video footage that’s in the standard format
- You don’t have a camera that can shoot 4K and want your video to be 4K (or other sizes)
When I started my career in video post-production, “standard definition” was, in fact, the standard. Now there are so many resolutions. I originally wrote this article about 7 years ago, when a cinematographer friend of mine asked me to upscale footage for his reel from the 1990s to HD. In that article, I focusing upscaling 720×480 (480i) into 1920×1080 (1080p) footage.
For one thing, video standards have changed in just a few short years. 4K Ultra High Definition (4K UHD) is now a standard feature in over 60% of the televisions sold in North America. That number is increasing rapidly over the last few months as people can’t go out for entertainment during the pandemic. For example, if you’re watching an old standard-definition television show on a 4K UHD screen, it will look a bit stretched and blurry.
But, upscaling is not just for old footage. Some people are shooting at 1080p and upscaling their renders to 4K for YouTube. Why would they do that?
- They may not have a camera that is capable of shooting 4K.
- Editing 4K footage can be slow.
- 4K footage takes up a ton of space on your drives!
Luckily, upscaling technology has seen major improvements over the past few years, especially in the area of Artificial Intelligence (AI). We’ll dive into AI and I’ll also discuss upscaling video in Adobe products, Resolve, with AI, and using a few plug-ins with upscaling technology.
Please note: This article is not a review of products, but more of a guide to the upscaling options available.
Artificial Intelligence (AI) in Upscaling
A majority of the upscaling tools on the market today use AI. Some of the upscale players include:
- Gigapixel AI from Topaz Labs is probably the most well-known.
- Winxvideo AI – 4K video and image upscaler.
- DeepAI Super Resolution for images.
- PIXOP Super Resolution, an online uprez technology using Machine Learning and Amazon Web Services.
- iSize – Check out their website for some interesting technical diagrams.
- The NVIDIA SHIELD TV, which brings real-time AI upscaling right to the television
There are several others, as well. This will be an area that will explode over the next few years, like NVIDIA OptiX technology which is used for AI-enhanced noise reduction in Arnold Render GPU, cebas finalRender, e-on Software, Redshift, Corona Renderer, and more. You’ll certainly see AI-enhanced upscaling in the apps you currently use over the next year or two.
The videos below are from 1911 and 1896. To point out, they scaled the footage to 4K, 60fps. To explain, the enlargement was done with Gigapixel AI. The algorithm for adding frames is called Depth-Aware Video Frame Interpolation (DAIN), powered by neural networks.
A Trip Through New York City in 1911
“Arrival of a Train at La Ciotat” (The Lumiere Brothers, 1896)
Deep Space 9 Upscale Project
There’s a project out there called Deep Space Nine Upscale Project (DS9UP) to upscale Star Trek: Deep Space 9 from the DVD source to 4K. Joel Hruska writes about his experiences using Topaz Video Enhance AI, DaVinci Resolve, and AviSynth to uprez the show. Truly, the project is pretty interesting. He talks about his challenges and what he’s learned during the project. Take a look at the comparisons below.
Upscaling in your NLE or Host Application
Most host applications and NLEs now have built-in tools to make upscaled footage look better. We’ll cover a few of the more popular tools below, including Adobe, DaVinci Resolve, and Vegas.
Upscaling with Adobe Tools
Adobe has a feature called Detail-preserving upscale. Below, check out tutorials for After Effects, Premiere Pro, and Photoshop.
How to Upscale Video (with the best results) in After Effects
In this livestream from Adobe, Jason Levine shows the preferred method for upscaling video content in After Effects. To elaborate, he uses an effect called Detail Preserving Upscale. It’s similar to ‘preserve details’ in Photoshop. In fact, using this process instead of simply scaling the layer in a comp, gives better-looking results. In addition, there’s more control over detail and noise.
Upscaling in Premiere Pro
How to Upscale Video to 4K or Downscale to 1080P Easily. This tutorial by Farrah Penn certainly explains it pretty clearly.
Tutorials for Upscaling in DaVinci Resolve
Upscale 1080p Footage to 4k with Davinci Resolve Superscale!!
Jay Lippman has become my go-to guy for Resolve tutorials. In this video, he uses DaVinci Resolve’s Superscale.
SUPER SCALE in DaVinci Resolve | Upscale 1080p to 4K | EASY Tutorial
Eric at Travel9to5 gives this quick tutorial on Superscale. He explains, “The superscale function in Davinci Resolve upscales 1080p footage to 4k super easy and allows you to convert 1080p to 4k. This is a quick hack to upscale your old clips and videos and make them 4k resolution. If you have 4k footage you can even upscale 4k to 8k video!”
Upscaling in Magix Vegas
Naiwik Tech explains how to use Smart Upscale in Vegas to upscale 1080p to 4K.
The major benefit of upscaling with plug-ins is the ease of use. Several plugins out there can upscale your footage.
Boris FX Continuum UpRes Updated
Boris FX’s revolutionary AI-driven Continuum UpRes uses advanced techniques to maintain and fine-tune image details, generating much higher-quality final images than can be obtained with standard host upscaling. Boris Yamnitsky, president and founder of Boris FX says, “We’ve been working behind the scenes to develop ML (machine learning) models built and trained on the specific needs of the post-production community and are thrilled to introduce AI technology into the editing and VFX plugin workflows the film and television industry has relied on for nearly thirty years.”
Continuum’s popular image restoration category adds two new ML effects, each built with custom models. The BCC+ DeNoise ML effect instantly removes unwanted artifacts introduced by low-quality image sensors, low light, or lossy compression codecs. The one-click drag-and-drop effect analyzes the image and produces high-quality results while retaining and enhancing detail contrast. The BCC+ UpRes ML effect, exclusively available for After Effects, scales up footage regardless of format size with clean, sharp results. It also includes options to zoom in on a portion of a clip and sharpen it to preserve the fine details.
- Adobe After Effects and Premiere Pro
- Avid Media Composer
- Blackmagic Design Davinci Resolve
- Vegas Pro
- Foundry Nuke
NEW Upscale and Denoise Your Videos FAST in Premiere Pro with Boris FX 2024
Premiere Gal shows you how to upscale and denoise using Boris FX Uprez.
RE:Vision Effects REZup
A new one to the market, REZup comes from RE:Vision Effects, makers of Twixtor and ReelSmart Motion Blur.
With REZup, enhance the quality (“IQ”) of your content, and increase the resolution of your video without loss of detail or introducing artifacts. REZup contains two plug-ins: REZup Enhance and REZup Resize. These plugins provide a unique mix of state-of-the-art video image processing augmented with machine-learning techniques.
- After Effects CS6 (and up)
- Premiere Pro CS6 (and up)
- Final Cut Pro 10.1.4 (and up)
- Motion 5 (and up).
- OFX Hosts
- Autograph (All versions)
- Baselight 5.2.11675 (and up)
- Catalyst Edit 2017.2 (and up)
- DaVinci Resolve 15.0 (and up)
- Diamant and Dustbuster+ 13.0 (and up)
- Flame 2019.2 (and up)
- Fusion Studio 5.2 (and up)
- Grass Valley Rio 4.5.9 (and up)
- HitFilm 10.0 (and up)
- Natron 1 (and up)
- Nuke 5.1 (and up)
- Scratch 8.6.944 (and up)
- Silhouette 6.1.3 (and up)
- Vegas Pro 10.0e (and up)
The Pixel Farm PFClean
PFClean is the most widely adopted software solution for film and video restoration, from film transfer to remastering blockbuster movies, serving a broad client base from audiovisual archives to broadcasters and film studios.
In fact, PF Clean 2018 introduced a new GPU-accelerated edge-preserving upscale algorithm (18fps for SD to HD, 8fps for HD to UHD conversion on a single GPU).
VImager ScaleUp, a GPU-accelerated plugin for After Effects and Premiere Pro, uses AI to upscale footage up to 10x while preserving the image quality.
- After Effects 2024, 2023, 2022, 2021, 2020
- Premiere 2024, 2023, 2022, 2021, 2020
Check GPU Requirements and try a demo before purchasing ScaleUp
Note: The performance of ScaleUp is highly dependent on GPU, and some GPU cards are not supported. Please see the System Requirements tab for GPU details. Also, PLEASE TRY a DEMO of ScaleUp before purchasing.
GPUResize is a GPU-accelerated plugin for Adobe After Effects and Adobe Premiere with real-time performance and advanced image interpolation abilities that are missing natively.
We believe that there’s no such thing as one and only perfect image scaling algorithm – instead, we are introducing several different ones, that allow you to make the best choice for your footage.
Use cases of GPUResize aren’t only limited to simple SD to HD video upscaling, but instead could vary from pixel-sharp upscaling of animated pixel art to downscaling your 4k and 8k camera footage while keeping the most detail possible.
- After Effects CC 2015, CC 2014, CC, CS6, CS5.5, CS5
- Premiere Pro CC 2015, CC 2014, CC, CS6, CS5.5, CS5
Monadnock 9 Slice Scale
9 Slice Scale scale images while preserving corners and edges. This is an FxFactory FxPack so it requires FxFactory and an Apple M1 processor to run.
- After Effects
- Premiere Pro
- Final Cut Pro
The future looks extremely bright when it comes to upscaling. New AI technologies are allowing low-resolution images and video to be blown up to 4K and 8K without the blur and artifacting that have always been an issue. Even in the past few months, emerging technologies are exponentially better than the tools of just a year ago. This is an area that is going to be interesting to watch!
Posted by Michele