In this very intriguing experiment, ARRI ALEXA Classic 2K footage underwent AI “processing” to produce 8K footage using machine learning technologies. How can AI methodologies help us increase resolution, instead of scouring 8K sensors?
In a video titled “This is what an ARRI ALEXA 8K would look like…” a very neat demonstration of technology called AI upscaling was shown. The concept is very simple. Take the ultra-high quality 2K ProRes footage from the old but powerful ARRI ALEXA Classic, and using AI (artificial intelligence) upscaling machine learning technology, turn it into 8K footage. As you can see, the process takes some time (for HD to 8K video, it usually takes 2-3 seconds for each frame). This technology was put together by a company called Topaz. Its related product is called Video Enhance AI, which can enhance and enhance many aspects of videos, from implementing slow motion to upscaling resolution.
As Topaz stated, “Using the power of machine learning AI technology, Video Enhance AI is trained on thousands of videos and combines information from multiple input video frames. It can enlarge the video HD to 8K resolution with decent detail and motion consistency. For example, you can upscale SD video to HD or even 4K resolution.” The technique shown here is known as “video upscaling.” ‘IA’. Essentially, you feed an algorithm a low-resolution image, and based on the training data it’s seen, it spits out a version that looks the same but has more pixels. “We take support video output up to 8K. Any size larger than 8K will fail. But you can set the output format as an image sequence (png/tif/tiff) which can support almost any size” , points out Topaz.
Scaling, as a general technique, has been around for a long time, but the use of AI has dramatically improved the speed and quality of results. In the case of mundane upscaling algorithms, these rules are often quite simple. If you want to scale a 50 x 50 pixel image to double its size, for example, a traditional algorithm simply inserts new pixels between the existing ones, picking the color of the new pixels based on an average of its neighbors. However, Video Enhance AI uses artificial intelligence algorithms created by machine learning by “looking” at entire sections of footage at once.
“We originally designed Video Enhance AI to improve the most common quality issues affecting videos, including footage deinterlacing, intelligent upscaling up to 8K resolution, and seamless frame rate conversion. frames,” Topaz says, and adds, “Because it treats video as a constant frame rate series, this caused issues when the app’s AI models had to process frame rate footage. variable frames, frame repeats, and scene changes. This has also complicated our ability to develop new models to address stabilization or scene detection as they require multiple passes over the video.” Therefore, to develop these new features and continuing to innovate, Topaz has built this next generation of the application to the top with an entirely new code base and an all new user interface. izer with many feature enhancements, such as stacking multiple AI models, parallel task execution, and project queue filtering.
The price of the Topaz Video AI the app is $300. You can also try it for free. The minimum system requirements are:
- computer: A DirectX12 compatible GPU (NVidia or AMD). An Intel processor from 2015 or an AMD processor from 2016 are our minimum processor requirements. On a PC with Nvidia GTX 1080 for HD to 8K video, each frame usually takes 2-3 seconds. If you are scaling video from SD to HD, it will take about 0.4 seconds for each frame.
- Mac: 8 GB or more of RAM, 2 GB of VRAM. MacOS 10.14 (Mojave) to run on CPU only. MacOS 10.15 (Catalina) or newer to run on GPU. This gives the best performance.
Back to the ALEXA Classic experience. It’s no secret that this camera has a very solid history on the big screen. Its 2K resolution footage was used in IMAX theaters (equivalent to 11K resolution). Therefore, it’s okay that his files look good at 8K. However, we can discuss the race for 8K resolution that many camera makers are obsessed with. AI scaling solutions are here to stay. If so, a 4K sensor would be sufficient for an 8K enhancement without the need to develop an 8K sensor with reduced dynamic range capabilities. So a 4K sensor and machine learning software can do a better job than a native 8K sensor for 8K imaging. What do you think about this?