Toolfarm's Plug-in Finder
The Deemon SuperResolution
From PlugInFinder
Contents
|
Stats
- Developer: The Deemon
- Version: 1.0
- Platforms: Windows
- Host Applications: Adobe After Effects
Purchase Information
Download Demo
Description
This plugin uses super-resolution method to upsize video. It can be used to make HD videos from SD. The plugin uses the same super-resolution engine as Video Enhancer 1.6.
How it works
The plugin uses information from neighbor frames to upsize current frame. It accumulates information, so the more sequential frames are processed, the better quality you get. When it's first applied to a layer, it takes just a few frames preceding the current one.
There are 3 quality modes: High speed SR, High quality SR and Lanczos. First two modes differ in speed and result quality. The last one is just Lanczos upscaling without super-resolution, you can use it when you need relatively fast upsizing with better quality than bicubic resize. This mode is also used when you set "low quality" for chosen layer in After Effects.
Where it works
This version was tested in After Effects 7.0 and Premiere Pro 2.0 but may probably work in other versions (reported to work in AE 6.5). Warning: this upsize method works only with progressive video. If you want to upsize interlaced video you should either deinterlace it or separate fields and recombine them back after upsizing.
What is super-resolution?
Super-resolution is a term for a set of methods of increasing image or video resolution. All these methods are based on same idea: using information from several images to create one upsized image. These methods try to extract details from one frames to reconstruct other frames. This approach differs a lot from some sophisticated image upsizing methods which try to synthesize artificial details.
"Super-resolution" is not a marketing buzzword, it's a mathematical term used by scientists. First work on this topic was published in 1984 and the term "Super-resolution" itself appeared at around 1990.
Methods usually discussed in scientific literature try to reproduce process of losing quality when shooting video with low-res cameras and then solve inverse problem of finding video which being downsized with that process gives us known low-res material. This is an ill-posed inverse problem which doesn't have straightforward solution and usually requires some additional regularization (applying some artificial constraints) and huge CPU time to check an awfull lot of variants. Modern practical methods are usually simpler but still effective.
SR works when several low resolution images contain slightly different views of the same objects. In this case total information about the object is much higher than information in one frame. Best case is when an object moves in the video. If it doesn't move at all and it's identical in all frames, no extra information can be extracted. If it moves or changes too fast then it looks very different in different frames and it's too hard to use information from one frame in reconstructing the other.
Super-resolution is used now in two tasks: extracting single frames from video (or set of photos) with high quality, or upsizing whole video. SR methods are usually based on two important algorithms: high quality spatial (in-frame) upsizing, and motion compensation for finding corresponding areas in neighbor frames. Many practical implementations upsize material 2 times. If we need to upsize it 4 times, we usually apply SR twice (this can be done internally in implementation).
Examples
Let's look at real example of upsizing video 4 times with super-resolution. This example was made using Video Enhancer. First, we have original low-resolution sequential frames of video:
Here are the same frames zoomed 4 times:
Step 1: upsize all frames 2 times using good spatial interpolation, in this case Lanczos3 method:
Step 2: using sub-pixel accurate motion compensation find similar areas in neighbor frames and intelligently merge frames to combine information (and in some cases reduce noise):
Now we have our video upsized 2 times using super-resolution. We repeat this process to get 4x enlargement. Step 3: spatial interpolation again:
And finally step 4: combine neighbor frames to get more details using motion compensation:
Now let's compare it with what we originally had: original low res frames zoomed 4x
Quite amazing, isn't it? But no magic, only a bit of math. ;)
Comparison of different video upsize methods
Nowadays there's a lot of equipment that allows you to store and play HDTV (or simple HD, high-definition) video which has resolution like 1280x720, 1920x1080 etc. And at the same time there's a significant lack of HD content. Majority of TV channels and DVD movies are 720x576 or less (so called SD, standard-definition). So if someone wants to watch SD video on his or her HD screen, upsizing becomes inevitable. Upsizing video can be done with big variety of methods. They differ in quality, complexity and speed. Most screens and video players already have some built-in methods for upsizing content, usually bilinear or bicubic interpolation.
While there are a many methods for in-frame (spatial) upsizing, there is another approach which uses information from different frames to reconstruct more details. This approach is called super-resolution. This topic was discussed by mathematicians for 20 years and now finally our computers became powerful enough for practical super-resolution implementations.
In this article we'll compare different methods of video upsizing that anyone can try - accessible in downloadable software. This means that implementations existing in very complicated form of Matlab or Avisynth scripts or available only as technologies for big compenies or in some commercial software not available to try and so on - these methods are not included in this comparison. Also, software which uses super-resolution approach to reconstruct single frames from video (like QE SuperResolution, Topaz Moment, Photo Acute, etc.) is not included.
Included methods
- Zoom - simplest method of enlarging each pixel 4 times, also called "nearest neighbor".
- Bicubic resize - internal resize filter of VirtualDub, bicubic spline interpolation. Same resize method is used in vast number of players and apps.
- Lanczos3 - same VirtualDub filter, Lanczos3 method.
- Lanczos3+sharpen - sequential application of two internal VirtualDub filters: first Lanczos3 resize and then sharpen filter with its parameter set to 40.
- Video Enhancer - using super-resolution in high quality mode.
- Topaz Enhance - a plug-in for Adobe Premiere Pro 2.0 was used, Enlarge filter with default parameters.
- Instant HD - a plug-in for Adobe Premiere Pro 2.0 from Red Giant Software. Demo variant was used that leaves "watermark" on result, so it was impossible to correctly calculate objective metric (PSNR) for this method.
- DA Resizer sharp - Resizer v2.2 from Digital Anarchy, a plug-in for Adobe After Effects, best sharp mode.
- DA Resizer smooth - the same plug-in, best smooth mode.
Process of comparison
Five video sequencies, 45-100 frames each, were taken. Each video was downsized 4 times by applying twice bilinear resize (to half size) in VirtualDub, so 5 low-res videos were created. Then each low resolution video was upsized 4 times using different methods described above. Then for each video an objective metric of similarity between original hi-res and upsized video was calculated. For such metric Y-PSNR was chosen (quite standard way for comparing videos, used in industry and science).
