NVIDIA launches AI-enabled media platform Holoscan for Media

The integration of artificial intelligence (AI) into the broadcasting and media industry is progressively shaping how content is created, distributed, and consumed.

The newly launched Holoscan for Media platform, detailed in the latest edition of AI Decoded by NVIDIA, aims to provide developers with an AI-enabled platform to streamline the development of live media applications on RTX AI PCs.

NVIDIA highlighted the aims of Holoscan for Media, stating that it is a "software-defined, AI-enabled platform that helps developers easily integrate AI into their live media applications." This development allows media companies to run live media pipelines on the same infrastructure as AI, simplifying the formerly intricate process that required substantial expertise and resources.

At the ongoing IBC broadcast and media technology show in Amsterdam, NVIDIA, in collaboration with its partners including Adobe, Blackmagic Design, and Topaz Labs, is demonstrating the recent advancements in video editing tools powered by RTX AI. The company asserts that these tools will enhance live media applications by providing intelligent and advanced AI capabilities.

One significant obstacle developers face is the inefficiency in deploying live media applications. Traditionally, developers need to create separate builds for different deployment environments, whether on premises, in the cloud or at the edge. This fragmentary approach can inflate costs and prolong development timelines. Holoscan for Media, based on a cloud-native architecture, addresses this by enabling applications to run AI implementations from any location, thereby eliminating the necessity for multiple builds.

Developers also benefit from integration with NVIDIA AI Enterprise, an enterprise-grade AI platform comprising a comprehensive suite of AI software, tools, and frameworks optimized for NVIDIA GPUs. This includes popular frameworks such as TensorFlow, PyTorch, and RAPIDS, facilitating streamlined deployment. With these tools, developers can incorporate sophisticated AI capabilities like computer vision, natural language processing, and recommendation systems into their media applications. NVIDIA AI Enterprise allows for the prototyping, testing, and deployment of advanced AI models seamlessly within media workflows.

A demonstration of Holoscan for Media is set to be available at IBC 2024, showcasing its potential to streamline app development. Among the technologies on display, Blackmagic Design's DaVinci Resolve 19 Studio is noteworthy for introducing AI features that expedite editing workflows. These new features facilitate the stabilization of footage, denoising of frames, sharpening of low-resolution videos, and remapping of video into vibrant HDR10 colour spaces. NVIDIA RTX PCs and workstations enhance these tools' performance, boasting up to three times faster performance on a GeForce RTX 4090 laptop compared to a Macbook Pro M3 Max. Additionally, an update introduces NVIDIA's H.265 Ultra-High-Quality (UHQ) mode, which leverages NVENC to enhance HEVC encoding efficiency by 10%.

Topaz Labs is also contributing to the advancements with the introduction of an Adobe After Effects plug-in for Video AI. This plug-in facilitates the integration of AI upscaling tools within After Effects compositions, thereby enhancing flexibility and reducing inefficiencies. The plug-in incorporates a complete range of enhancement and frame interpolation models directly into the popular motion graphics software. Of particular note is the Apollo AI model from Topaz Video AI, which can convert footage to up to 16 times slow motion. This capability is accelerated on RTX GPUs by NVIDIA TensorRT, leading to a performance boost of up to 70% in AI tasks.

The AI Decoded blog post elaborates on these developments, showcasing NVIDIA's commitment to supporting developers and video editors with tools that make AI implementation more efficient and faster. The resources provided aim to assist those looking to delve deeper into AI technologies and their applications in live media.