Microsoft AI Patent Can Rewrite What People Say in Livestreams and Adjust Their Lips to Match

Reviewed byNidhi Govil

5 Sources

Share

Microsoft filed a patent for AI agents that modify spoken words during livestreams in near real-time while adjusting speakers' facial movements to match. The system can remove filler words, translate languages, and censor content based on viewer profiles, but experts warn it could enable sophisticated misinformation.

Microsoft Files Patent for AI That Edits Livestreams in Real Time

Microsoft has filed a patent application for an AI system that can modify what people say during livestreams and live video broadcasts in near real-time, then redraw their lips to match the altered speech

1

. The patent, titled "Near Real-Time Artificial Intelligence (AI) Enhancement of Video or Augmented Reality (AR) Presentations of Live Events," was first flagged by patent tracker Patentlyze and describes technology that could fundamentally change how audiences engage with live content

2

.

Source: TweakTown

Source: TweakTown

The Microsoft AI patent describes an AI system to edit livestreams by using AI agents that listen to live audio and flag specific words for modification or removal based on preset parameters or viewer profiles, which can include age, location, and native language

3

. Using generative AI, the system then replaces those words in the audio, smooths transitions around each edit, and generates new video frames so the speaker's mouth movements align with the modified speech

1

. The edited result plays back after a delay of roughly one to ten seconds

3

.

How the AI That Rewrites What People Say Functions

The patent application details how AI agents would identify and alter spoken words during live broadcasts through near real-time artificial intelligence enhancement

4

. The system would scan live audio recordings and identify words to modify according to learned patterns and user profiles

4

. Microsoft claims the technology would remove filler words like "um" or "ah" from live speech, or erase stutters and speech impediments

2

.

The system could also eliminate "mumbled words, mispronounced words, misused words, misspoken words, offensive words, insensitive words, or inappropriate words," with replacement words potentially coming from internet searches

2

. Beyond basic corrections, the technology could remove profanity and replace words based on viewer preferences

1

. The patent also describes cutting down on "personal words or phrases" like "coined terms" and "colloquial terms" such as "y'all" or phrases like "bite the bullet"

4

.

Real-Time Translation with Lip Sync Technology

A separate claim in the patent describes using the technology for real-time translation during livestreams

1

. The system would allow broadcasts and other live footage to be both heard and seen in a viewer's preferred language, with the speaker's mouth movements adjusted accordingly

5

. Unlike YouTube's existing AI audio translation, Microsoft's patent would translate livestreams with matching lip sync

1

. The translator claim retains the original recording as ground truth, providing a reference for accuracy

1

.

The system could run on augmented reality glasses, phones, tablets, or cameras, making the technology accessible across multiple platforms

3

. Listed applications include lectures, presentations, award ceremonies, game shows, cooking shows, and live sports

2

.

Source: Futurism

Source: Futurism

Concerns About Misinformation and Deepfakes

The ability to hide edits by adjusting lip movements suggests viewers could watch altering live video and audio without realizing modifications were made

1

. Real-time AI face swapping is already popular with scammers, so the ability to alter what is being said in livestreams could enable sophisticated misinformation and fraud

1

. The patent doesn't specify whether it would tell users which edits were made or why, raising transparency concerns

1

.

In an era defined by AI-fueled deepfakes and disinformation, the system may represent a new world of filtering tools that subtly alter reality as it takes place on a highly-personalized level

2

. The tool that can change how people listen to lectures could also create distrust if used to modify speeches of political leaders

4

. Automatically editing live videos with AI agents could change how audiences fundamentally engage with live footage, potentially moving viewers further into personalized bubbles

2

.

Source: CXOToday

Source: CXOToday

Patent Status and Industry Context

The Microsoft AI patent is a published application, not a granted patent

3

. The application was filed on April 29, 2025, and Patentlyze notes the examiner rejected the claims as filed on July 29, 2026, a routine step that Microsoft can respond to

3

. Microsoft hasn't said whether it plans to build the technology, and as with any patent, there's no guarantee this ends up inside a Microsoft product

3

.

This patent shares a basic idea with others involving real-time editing with AI. Microsoft filed another ambitious claim in July aiming to rewrite a game's code to add content based on player behavior

1

. Sony recently filed multiple patents that would use AI to predict upcoming frames and which button a user will press next, or control game time progression and difficulty

1

. The frequency of these filings demonstrates strong interest in real-time AI editing across the tech industry

1

. Microsoft did not immediately respond to requests for comment about the patent

2

.

© 2026 TheOutpost.AI All rights reserved