Meta patents AI system that continuously monitors voice to detect mood and emotional state

Meta patented an AI system that listens to your voice all day and tracks your mood

Meta patented an AI system that continuously records voice to detect emotional state. It combines audio with location and activity data. Amazon tried and abandoned a similar product. Meta has been granted a patent for an AI system that continuously records a user's voice, transcribes it, and feeds the audio through a machine learning model to detect their emotional state. The patent, published on July 2, describes a device that listens for "audible communications" including sighs, laughter, and vocal tone, and combines them with "contextual factors" like time of day, location, activity, and medication timing. The system builds a persistent mood log from all of this. The patent, first spotted by Patentlyze, frames the technology as a fitness tool. Meta argues that AI-powered emotional coaching could correct workout posture and adjust guidance in ways a human personal trainer cannot. But the architecture described goes well beyond a gym session. The system is designed for "continuous emotional monitoring on everyday devices," using what the patent calls "multimodal sensor inputs on synchronized timelines." It listens at "predefined times" for tone, pace, pauses, and breathing, then quantifies the user's emotional state. It can correlate mood with medication schedules, producing summaries like "a happier emotional state associated with a particular time of day or at a time when medication is taken." Amazon tried something similar. In 2020, it launched the Halo Band, a fitness wearable with a built-in microphone that performed "tone of voice analysis." After public backlash, Amazon removed the microphones from the next-generation model in 2021 and discontinued the product line entirely in 2023. Meta's smart glasses are already the subject of a privacy crisis over covert recording, with seven million pairs sold and two US lawsuits alleging the company misled consumers about how footage was handled. Adding persistent ambient voice recording to the product line would extend the surveillance from what users see to how they feel. A Meta spokesperson told 404 Media that "patents at Meta are often filed to disclose concepts that may or may not be implemented." That is standard corporate hedging. But the patent's specificity, down to medication tracking and emotion-time correlations, suggests more than speculative filing. Kenyan data workers already lost their jobs after revealing they reviewed intimate footage captured by Meta's glasses. If this patent becomes a product, someone will need to train the emotional model too, which means someone will be listening to recordings of strangers sighing, laughing, and talking to themselves at home, labelling each one with a mood score so the algorithm can learn what sadness sounds like.

Meta Patents Technology to Detect Your Mood by Constantly Analyzing Your Tone of Voice

Can't-miss innovations from the bleeding edge of science and tech Meta patented AI tech that constantly analyzes your voice and surroundings in order to detect your emotional state. Finally, exactly what we were all asking for: Meta being granted constant access to our offline lives. As first caught by Patentlyze, a patent published on July 2 describes a system that records all the user's "audible communications" and combines it with "contextual factors" like "time of day, location, user activity, or digital interaction." Audio may be "transcribed, and an emotional-state machine learning model may interpret verbal and nonverbal cues to determine emotional indicators." In other words, the system listens to you and everything you do, combines it with other data, and feeds it all to a mood-predicting AI. "The system increases the precision and reliability of emotional inference by aligning multimodal sensor inputs on synchronized timelines, which creates a novel data structure that supports richer emotional analysis," reads the patent. "These combined features deliver a technical improvement in automated audio interpretation, enabling continuous emotional monitoring on everyday devices." According to the patent, the goal of this extraordinarily intrusive system would be to better tailor users' workouts to their emotional state, as "personal trainers cannot provide the level of precision in guidance, such as correcting a pose and/or body movement," as well as Meta's imagined device could. It's not surprising to see Meta pitch this kind of surveillance technology as a fitness device. Health-tracking wearables have become wildly popular, despite the extraordinary amount of physical data about individual consumers they collect and store. The key difference in Meta's case, is that rather than crunch biological metrics, it's an AI-powered fly-on-the-wall attempting to measure a person's emotional state by listening in on the world around them, and possibly then connecting that information to all other available data. "The AI assistant may listen to a user(s) at predefined times to hear various types of communication, such as sighs, laughter, and/or the tone(s) of a voice(s)," reads the patent. "The AI assistant may use these inputs to quantify the user's emotional state or generate other insights about the user." The patent also discusses connecting audio inputs to information like when a user takes their medicine, proposing that "the AI assistant may take multiple inputs in addition to audio inputs (e.g., of a user's voice) to provide a summary of emotional trends based on various inputs (e.g., a happier emotional state associated with a particular time of day or at a time when medication is taken, etc.)" It's worth noting that fellow tech giant Amazon tried and failed to market a similar system. In 2020, the company launched the Halo Band, a fitness band with a built-in microphone that was designed to listen in on users and conduct what Amazon described as a "tone of voice analysis." Following public backlash in 2021, Amazon nuked the microphones when it launched a next-generation version of the band. It discontinued the product line altogether in 2023. Whether we'll ever see Meta attempt to launch a similar mood-tracking device is unclear. In a statement to 404 Media, a Meta spokesperson said that "like other companies, patents at Meta are often filed to disclose concepts that may or may not be implemented, and a granted patent does not guarantee that Meta has pursued or will pursue the technology described." The existence of the patent, though, shows that such a device is certainly on Meta's mind -- and is yet another Meta-made product that stands to strengthen the bridge between our online and offline worlds.