Since the earliest days of TV broadcasting, global viewers have wanted to watch programs from other countries – live breaking news, international sporting events, entertainment and cultural programming. However, simple international program exchange has never been possible due to the huge differences in TV standards around the world. Similarly, transfer of movies to formats suitable for home TV viewing inherently requires both format and frame rate conversion, for audiences in all regions.
With the transition of services from SD to HD, as well as proliferation of mobile and internet content services, there have never been so many different formats and standards for video content available to global viewers. Format and standards converters, such as FrameFormer for Adobe Premiere Pro, are needed to ensure that every audience gets the best possible quality at their display device.
Why do I need motion compensated standards conversion? Can’t I just re-time the source content on the project timeline?
The simplest methods for frame rate conversion are generally unacceptable due to their poor motion performance. Re-timing a sequence of images to play back at a different frame rate will affect the speed of motion of the objects in the image, and will distort the audio associated with the video. In practice, such “off-speeding” is usually only applied to very small frame rate changes such as 23.976Hz to 25Hz, in situations where audio pitch can be more easily corrected with simple processing, and where the client is prepared to accept a 4% change in program length.
Conversion of 59.94Hz to 50Hz represents a 16% change, so off-speeding cannot be used as it would create grossly unnatural motion, highly significant audio pitch alterations, and a dramatic change in program length. For such conversions, very simple frame rate converters simply copy frames from input to output where the input and output presentation positions would be closely related in time, and skip (or duplicate) frames to maintain the required overall frame rate.
Frame copying with drop/repeat has many obvious drawbacks including unnatural and discontinuous motion, audio artifacts (where audio packets are lost or repeated at the frame skip/repeat), and corruption of metadata (e.g. closed caption packets would be lost or repeated). Therefore it is not recommended in professional video standards conversion applications.
Why can’t I use linear standards conversion?
Linear interpolation from the existing picture content is the basis of low complexity, linear frame rate converters which are used in many low cost applications. In the simplest case, pixels from two input fields are used to create the pixels in a new output field. Simple linear interpolation using a weighted sum of existing pixels to generate new output pixels gets round some of the problems associated with frame copying, but can cause picture quality problems such as blur, loss of resolution and unnatural movement of objects e.g. judder. Only motion compensated standards converters such as FrameFormer can offer high quality converted output.
What is motion compensated standards conversion?
Motion compensated processing overcomes many of the drawbacks associated with linear standards converters. A motion compensated frame rate converter, such as FrameFormer, calculates the motion between frames in the content, and works out where to move objects to when creating new frames in between. If we can calculate the change in an object’s position between two frames of the input sequence, since we know the time interval between those frames, if we assume that the object moves at a constant speed, we can work out where the object should be at any other time interval. A motion compensated standards converter can thus reproduce the object within any interpolated or re-timed frames. In this way, all picture objects remain sharp and in focus, and their motion is smoothly portrayed without any judder or irregular movement.
What’s different about InSync’s FrameFormer for Adobe Premiere Pro CC solution?
Like some other motion compensated standards converters, InSync’s FrameFormer uses Phase Correlation as the basis of the motion estimation. In the Phase Correlation method, image data are transformed into the frequency domain using the Fourier transform. Once transformed, the position of picture detail is encoded by the frequencies’ phase information. Frequency magnitudes are adjusted so that they contribute equally. The size and direction of motions present may then be obtained by subtracting the phases obtained from two sequential frames and transforming the result back to the spatial domain.
Phase Correlation is insufficient to perform motion compensation by itself. Significantly, the method identifies motions present within the image but does not define which areas in the image have that motion. Therefore, the measured motions must somehow be associated with specific regions within the image which typically represent separate real-world objects. Content properties such as shot changes, concealed and revealed picture elements, brightness changes and the presence of abrupt picture boundaries must also be addressed. InSync uses proprietary processing in the stages before and after Phase Correlation which enable our FrameFormer converter to obtain superior results.
Finally, the solution is CPU-only meaning that no specialist hardware is needed.
FrameFormer is ideally suited for applications requiring deinterlacing such as 1080 59i to 720 50p and 1080 50i to 1080 50p conversions. InSync’s proprietary algorithms ensure that content is converted with the highest resolution, avoiding aliasing and ringing artifacts. More information about InSync’s processing is available from the InSync Website.