Microsoft filed a patent for an AI system that alters live video in real time. It can change spoken words and edit a speaker’s face and mouth to match. The system uses AI agents guided by set parameters or user profiles. It also smooths audio and video transitions around edited words. Possible uses include real-time translation and removing filler words or speech impediments. It can also remove or replace personal, colloquial, offensive or inappropriate words. Replacement words may come from an internet search. The original live video and audio may be stored for ground truth. It applies to lectures, presentations, TV productions and live sports.
What it unlocks
Real-time alteration of live video to change spoken words and facial movements to match edited speech.
Sources