Google launches agentic video understanding with efficiency gains
Google introduced a new video understanding capability that is up to 88% more token efficient than previous methods. The development allows models to process long-form content and respond to conversational editing prompts without relying on transcripts.
Google launched an agentic video understanding capability in early September, claiming it is up to 88% more token efficient and around 7% more accurate than previous methods. The biggest gains were observed on larger files such as 90-minute lectures or multi-hour recordings. This development allows models to realistically process requests like finding specific customer complaints across 20 hours of calls. While some models still rely on transcripts and timestamps to approximate watching, this new capability enables models to deliver against conversational editing prompts.
Concurrently, generated footage quality has improved significantly with releases like Kling 4.0 and Gemini Omni 1.1, offering longer, higher-quality, and more controllable clips. Audio generation has also become normalized in leading video models, including Runway Gen 4.5 and Kling 3.0, which generate sound alongside the picture, though generated clips remain seconds long rather than minutes.