Revolutionary Emotion-Sensing Sticker Unveils True Feelings, Promising Advancements in Mental Health Care

High-tech sticker can identify real human emotions

Saying one thing while feeling another is part of being human, but bottling up emotions can have serious psychological consequences like anxiety or panic attacks. To help health care providers tell the difference, a team led by scientists at Penn State has created a stretchable, rechargeable sticker that can detect real emotions -- by measuring things like skin temperature and heart rate -- even when users put on a brave face. The researchers recently unveiled the wearable patch that can simultaneously and accurately track multiple emotional signals in a study published in the journal Nano Letters. "This is a new and improved way to understand our emotions by looking at multiple body signals at once," said Huanyu "Larry" Cheng, the James L. Henderson, Jr. Memorial Associate Professor of Engineering Science and Mechanics at Penn State and lead author on the paper. "Relying only on facial expressions to understand emotions can be misleading. People often don't visibly show how they truly feel, so that's why we're combining facial expression analysis with other important physiological signals, which will ultimately lead to better mental health monitoring and support." The sticker-like patch tracks a range of physiological responses, such as skin temperature, humidity, heart rate and blood oxygen levels, that are associated with emotional states. Most importantly, Cheng explained, the device's sensors are designed to work independently, minimizing any interference between the different measurements. The device combines its analysis of the physiological signals with facial expression data to better distinguish between genuine emotions and acted ones. It then wirelessly transmits the real-time measured data to mobile devices and the cloud, where clinicians could potentially use it to better assess patients virtually. The device does not record personal information, only signals, Cheng explained, meaning personal privacy is protected through the device's design. "This technology has the potential to help people who are struggling with their mental health, but maybe aren't being fully honest with others or even themselves about how much they are struggling," said Yangbo Yuan, co-author on the paper and doctoral student at Penn State studying engineering science and mechanics. Cheng explained that the collected data could also help bridge cultural or social gaps, which can manifest as a person appearing more stoic or expressive to their health care providers. "By keeping track of these signals, it could be possible to detect problems like anxiety or depression earlier in its progression." The researchers built the stretchy, BandAid-sized device by folding together thin layers of flexible metals like platinum and gold and cutting them into wave-like shapes that maintain sensitivity even when pulled or twisted. They also used layers of materials that change the flow of electrical current with temperature and built in hollow tubes made of carbon atoms, which absorb water and track humidity levels. The multiple sensors were designed and placed on the device in such a way that they would not interfere with each other's measurements. For example, the researchers put a rigid layer under the temperature and humidity sensors to protect them from the stretching that the facial expression sensors would experience. They also used a waterproof layer to protect the temperature and strain sensors from humidity. "We've engineered this device to measure these different signals independently, without them interfering with each other, providing a much clearer and more accurate picture of what's happening beneath the surface," said Libo Gao, co-corresponding author on the paper and associate professor at Xiamen University. Next the team trained an artificial intelligence (AI) model to read and understand signs of performed and real human emotion. The researchers recruited eight people, a common sample size for pilot studies, to perform six common facial expressions: happiness, surprise, fear, sadness, anger and disgust. The participants displayed each emotional expression 100 times while the device tracked their movement. The researchers then fed this data to an AI model, training it to correlate specific facial movements with different emotions. They then recruited an additional three participants to further evaluate the model's abilities. It classified performed facial expressions with 96.28% accuracy. When it came to tracking real emotions, the researchers tested how well the device tracked the psychological responses of the same participants as they watched video clips designed to elicit emotions. The device correctly identified emotions with 88.83% accuracy, with the sensors confirming that the psychological responses were consistent with known links between emotions and psychological reactions, such as increases in skin temperature and heart rate during surprise and anger. Cheng noted that the ability to wirelessly transmit the data means that health care professionals could potentially monitor individuals remotely and provide timely emotional support through telemedicine. "This sensor can serve a vital function in bridging gaps in access to care," he said. "Given the rising stress levels in modern society, the ability to monitor emotions can provide early indicators of debilitating conditions and allow for proactive support." He explained that the device also opens the door for other systems for artificial intelligence (AI)-powered disease diagnostics and therapeutics beyond just emotion recognition. He noted there may be potential applications for clinicians to better understand the mental and emotional state of non-verbal patients, better identify behavioral and psychological symptoms of dementia and recognize opioid overdose. The technology could one day even be used for chronic wound monitoring and disease management, he added, as well as track neurodegenerative disease progression and athletic performance. "While still in the research and development phase, this device is a significant step forward in our ability to monitor and understand human emotions, potentially paving the way for more proactive and personalized approaches to mental health care," Cheng said. Other contributors include Hongcheng Xu of Xi'an Jiaotong University. The U.S. National Institutes of Health and the U.S. National Science Foundation funded the Penn State researchers' contributions to this work.

This sticker reads emotions (even the ones you try to hide)

Good luck hiding how you feel. Researchers from Penn State University believe they have developed a stretchy, Band-Aid-sized wearable device capable of decoding even the most advanced poker face. The device attaches to a subject's skin and uses sensors to independently detect physiological responses, such as skin temperature and perspiration, in real time. That data is then digitized and analyzed by an AI model designed to determine the type of emotional responses the wearer is experiencing. In testing, the device was able to accurately identify the correct emotional response 89 percent of the time -- significantly more accurate, the researchers say, than simply observing a person's facial expression. That insight, they write, could prove useful for doctors seeking to better understand the psychological state of patients who don't obviously express emotions outwardly, or those who may be trying to conceal their feelings. The findings were published late last month in the journal Nano Letters. "This technology has the potential to help people who are struggling with their mental health, but maybe aren't being fully honest with others or even themselves about how much they are struggling," Penn State doctoral student and paper coauthor Yangbo Yuan said in a statement. The researchers set out to create a "multimodal sensing" device capable of collecting several physiological signals simultaneously without interference, a concept they refer to as "crosstalk." To do that, the team designed the small, sticker-like device made from thin layers of flexible metals like platinum and aluminum, folded and cut into wave-like shapes.Its flexible structure allows it to continue collecting real-time data even when bent, stretched, or pulled. A rechargeable lithium-ion battery powers the device. Once applied, the sensors monitor changes in skin temperature, heart rate, humidity (as expressed through sweat), and blood oxygen saturation. Each of these physiological signals can correlate with emotional responses. Increased skin temperature, for example, often signals surprise or anger, while a drop in temperature can indicate happiness, fear, or sadness. Elevated perspiration and heart rate, meanwhile, are common signs of fear. While each signal on its own offers limited insight, the researchers say combining multiple measurements in real time provides a far more accurate picture of an individual's emotional state. "More accurate emotion recognition can be possible with multidimensional data that integrate the physiological signals with facial expressions," the researchers write. That stream of physiological data is digitized and transmitted to a mobile device or the cloud. A custom-built machine learning model then analyzes the data to predict the type of emotion the subject is experiencing. All of this happens in near real-time, allowing a physician to monitor and interpret emotional responses while observing a patient in a clinical setting. Researchers put the device to the test in an experiment involving eight volunteers. Participants watched a series of videos designed to elicit specific emotional responses, all while wearing the device. In the first trial, subjects were instructed to display facial expressions linked to emotions such as happiness, fear, sadness, anger, and disgust. The device successfully matched the expressed emotions with 92.28 percent accuracy. Even more impressively though, it identified emotional responses with 88.83 percent accuracy when participants watched the videos without intentionally making any facial expressions. "Relying only on facial expressions to understand emotions can be misleading," Penn State Professor and lead author Huanyu "Larry" Cheng said. "People often don't visibly show how they truly feel, so that's why we're combining facial expression analysis with other important physiological signals, which will ultimately lead to better mental health monitoring and support." Though the sticker is still in the testing phases, researchers believe it could one day be a particularly useful tool to help clinicians better understand the emotional state of non-verbal patients. Early indications of brewing mental health issues, which might otherwise go unnoticed, could give medical professionals and psychiatrists more time to intervene before those situations intensify.

High-tech sticker can identify real human emotions

Saying one thing while feeling another is part of being human, but bottling up emotions can have serious psychological consequences, such as anxiety or panic attacks. To help health care providers tell the difference, a team led by scientists at Penn State has created a stretchable, rechargeable sticker that can detect real emotions -- by measuring things like skin temperature and heart rate -- even when users put on a brave face. The researchers recently unveiled the wearable patch that can simultaneously and accurately track multiple emotional signals in a study published in the journal Nano Letters. "This is a new and improved way to understand our emotions by looking at multiple body signals at once," said Huanyu "Larry" Cheng, the James L. Henderson, Jr. Memorial Associate Professor of Engineering Science and Mechanics at Penn State and lead author of the paper. "Relying only on facial expressions to understand emotions can be misleading. People often don't visibly show how they truly feel, so that's why we're combining facial expression analysis with other important physiological signals, which will ultimately lead to better mental health monitoring and support." The sticker-like patch tracks a range of physiological responses, such as skin temperature, humidity, heart rate and blood oxygen levels, that are associated with emotional states. Most importantly, Cheng explained, the device's sensors are designed to work independently, minimizing any interference between the different measurements. The device combines its analysis of the physiological signals with facial expression data to better distinguish between genuine emotions and acted ones. It then wirelessly transmits the real-time measured data to mobile devices and the cloud, where clinicians could potentially use it to better assess patients virtually. The device does not record personal information, only signals, Cheng explained, meaning personal privacy is protected through the device's design. "This technology has the potential to help people who are struggling with their mental health, but maybe aren't being fully honest with others or even themselves about how much they are struggling," said Yangbo Yuan, co-author of the paper and doctoral student at Penn State studying engineering science and mechanics. Cheng explained that the collected data could also help bridge cultural or social gaps, which can manifest as a person appearing more stoic or expressive to their health care providers: "By keeping track of these signals, it could be possible to detect problems like anxiety or depression earlier in its progression." The researchers built the stretchy, Band-Aid-sized device by folding together thin layers of flexible metals like platinum and gold and cutting them into wave-like shapes that maintain sensitivity even when pulled or twisted. They also used layers of materials that change the flow of electrical current with temperature and built in hollow tubes made of carbon atoms, which absorb water and track humidity levels. The multiple sensors were designed and placed on the device in such a way that they would not interfere with each other's measurements. For example, the researchers put a rigid layer under the temperature and humidity sensors to protect them from the stretching that the facial expression sensors would experience. They also used a waterproof layer to protect the temperature and strain sensors from humidity. "We've engineered this device to measure these different signals independently, without them interfering with each other, providing a much clearer and more accurate picture of what's happening beneath the surface," said Libo Gao, co-corresponding author on the paper and associate professor at Xiamen University. Next, the team trained an artificial intelligence (AI) model to read and understand signs of performed and real human emotions. The researchers recruited eight people, a common sample size for pilot studies, to perform six common facial expressions: happiness, surprise, fear, sadness, anger and disgust. The participants displayed each emotional expression 100 times while the device tracked their movement. The researchers then fed this data to an AI model, training it to correlate specific facial movements with different emotions. They then recruited an additional three participants to further evaluate the model's abilities. It classified performed facial expressions with 96.28% accuracy. When it came to tracking real emotions, the researchers tested how well the device tracked the psychological responses of the same participants as they watched video clips designed to elicit emotions. The device correctly identified emotions with 88.83% accuracy, with the sensors confirming that the psychological responses were consistent with known links between emotions and psychological reactions, such as increases in skin temperature and heart rate during surprise and anger. Cheng noted that the ability to wirelessly transmit the data means that health care professionals could potentially monitor individuals remotely and provide timely emotional support through telemedicine. "This sensor can serve a vital function in bridging gaps in access to care," he said. "Given the rising stress levels in modern society, the ability to monitor emotions can provide early indicators of debilitating conditions and allow for proactive support." He explained that the device also opens the door for other systems for artificial intelligence (AI)-powered disease diagnostics and therapeutics beyond just emotion recognition. He noted there may be potential applications for clinicians to better understand the mental and emotional state of non-verbal patients, better identify behavioral and psychological symptoms of dementia and recognize opioid overdose. The technology could one day even be used for chronic wound monitoring and disease management, he added, as well as track neurodegenerative disease progression and athletic performance. "While still in the research and development phase, this device is a significant step forward in our ability to monitor and understand human emotions, potentially paving the way for more proactive and personalized approaches to mental health care," Cheng said.

Smart Sticker Reads Real Emotions Beneath the Surface - Neuroscience News

Summary: A new stretchable, rechargeable sticker developed by researchers can detect authentic emotional states by measuring physiological signals like heart rate, skin temperature, and humidity, even when facial expressions are misleading. The wearable patch transmits real-time data to mobile devices, helping health providers assess mental health remotely. Unlike traditional emotion recognition systems, this device integrates multiple sensors and facial analysis while preserving user privacy. With AI-powered accuracy and wireless functionality, it offers promise for applications in telehealth, early intervention, and monitoring emotional well-being. Saying one thing while feeling another is part of being human, but bottling up emotions can have serious psychological consequences like anxiety or panic attacks. To help health care providers tell the difference, a team led by scientists at Penn State has created a stretchable, rechargeable sticker that can detect real emotions -- by measuring things like skin temperature and heart rate -- even when users put on a brave face. The researchers recently unveiled the wearable patch that can simultaneously and accurately track multiple emotional signals in a study published in the journal Nano Letters. "This is a new and improved way to understand our emotions by looking at multiple body signals at once," said Huanyu "Larry" Cheng, the James L. Henderson, Jr. Memorial Associate Professor of Engineering Science and Mechanics at Penn State and lead author on the paper. "Relying only on facial expressions to understand emotions can be misleading. People often don't visibly show how they truly feel, so that's why we're combining facial expression analysis with other important physiological signals, which will ultimately lead to better mental health monitoring and support." The sticker-like patch tracks a range of physiological responses, such as skin temperature, humidity, heart rate and blood oxygen levels, that are associated with emotional states. Most importantly, Cheng explained, the device's sensors are designed to work independently, minimizing any interference between the different measurements. The device combines its analysis of the physiological signals with facial expression data to better distinguish between genuine emotions and acted ones. It then wirelessly transmits the real-time measured data to mobile devices and the cloud, where clinicians could potentially use it to better assess patients virtually. The device does not record personal information, only signals, Cheng explained, meaning personal privacy is protected through the device's design. "This technology has the potential to help people who are struggling with their mental health, but maybe aren't being fully honest with others or even themselves about how much they are struggling," said Yangbo Yuan, co-author on the paper and doctoral student at Penn State studying engineering science and mechanics. Cheng explained that the collected data could also help bridge cultural or social gaps, which can manifest as a person appearing more stoic or expressive to their health care providers. "By keeping track of these signals, it could be possible to detect problems like anxiety or depression earlier in its progression." The researchers built the stretchy, BandAid-sized device by folding together thin layers of flexible metals like platinum and gold and cutting them into wave-like shapes that maintain sensitivity even when pulled or twisted. They also used layers of materials that change the flow of electrical current with temperature and built in hollow tubes made of carbon atoms, which absorb water and track humidity levels. The multiple sensors were designed and placed on the device in such a way that they would not interfere with each other's measurements. For example, the researchers put a rigid layer under the temperature and humidity sensors to protect them from the stretching that the facial expression sensors would experience. They also used a waterproof layer to protect the temperature and strain sensors from humidity. "We've engineered this device to measure these different signals independently, without them interfering with each other, providing a much clearer and more accurate picture of what's happening beneath the surface," said Libo Gao, co-corresponding author on the paper and associate professor at Xiamen University. Next the team trained an artificial intelligence (AI) model to read and understand signs of performed and real human emotion. The researchers recruited eight people, a common sample size for pilot studies, to perform six common facial expressions: happiness, surprise, fear, sadness, anger and disgust. The participants displayed each emotional expression 100 times while the device tracked their movement. The researchers then fed this data to an AI model, training it to correlate specific facial movements with different emotions. They then recruited an additional three participants to further evaluate the model's abilities. It classified performed facial expressions with 96.28% accuracy. When it came to tracking real emotions, the researchers tested how well the device tracked the psychological responses of the same participants as they watched video clips designed to elicit emotions. The device correctly identified emotions with 88.83% accuracy, with the sensors confirming that the psychological responses were consistent with known links between emotions and psychological reactions, such as increases in skin temperature and heart rate during surprise and anger. Cheng noted that the ability to wirelessly transmit the data means that health care professionals could potentially monitor individuals remotely and provide timely emotional support through telemedicine. "This sensor can serve a vital function in bridging gaps in access to care," he said. "Given the rising stress levels in modern society, the ability to monitor emotions can provide early indicators of debilitating conditions and allow for proactive support." He explained that the device also opens the door for other systems for artificial intelligence (AI)-powered disease diagnostics and therapeutics beyond just emotion recognition. He noted there may be potential applications for clinicians to better understand the mental and emotional state of non-verbal patients, better identify behavioral and psychological symptoms of dementia and recognize opioid overdose. The technology could one day even be used for chronic wound monitoring and disease management, he added, as well as track neurodegenerative disease progression and athletic performance. "While still in the research and development phase, this device is a significant step forward in our ability to monitor and understand human emotions, potentially paving the way for more proactive and personalized approaches to mental health care," Cheng said. Other contributors include Hongcheng Xu of Xi'an Jiaotong University. Funding: The U.S. National Institutes of Health and the U.S. National Science Foundation funded the Penn State researchers' contributions to this work. Author: Adrienne Berard Source: Penn State Contact: Adrienne Berard - Penn State Image: The image is credited to Neuroscience News Original Research: Closed access. "Stretchable, Rechargeable, Multimodal Hybrid Electronics for Decoupled Sensing toward Emotion Detection" by Huanyu "Larry" Cheng et al. Nano Letters Abstract Stretchable, Rechargeable, Multimodal Hybrid Electronics for Decoupled Sensing toward Emotion Detection Despite the rapid development of stretchable electronic devices for various applications in biomedicine and healthcare, the coupling between multiple input signals remains an obstacle in multimodal sensing before use in practical environments. This work introduces a fully integrated stretchable, rechargeable, multimodal hybrid device that combines decoupled sensors with a flexible wireless powering and transmitting module for emotion recognition. Through optimized structural design and material selection, the sensors can provide continuous real-time decoupled monitoring of biaxial strain, temperature, humidity, heart rate, and SpO levels. With a stacked bilayer for both the sensors and the flexible circuit, the rechargeable system showcases a reduced device footprint and improved comfort. A neural network model is also demonstrated to allow for high-precision facial expression recognition. By transmitting the real-time measured data to mobile devices and the cloud, the system can allow healthcare professionals to evaluate psychological health and provide emotional support through telemedicine when needed.

New sensor reveals your true emotions, even when you're faking it - Earth.com

Our inner emotions don't always match what we show on the outside. People often say they're fine while feeling overwhelmed inside - a common habit that can lead to mental health struggles like anxiety or panic attacks. Spotting these hidden emotions isn't always easy, especially for health care providers who are trying to help. In a breakthrough study, researchers have created a stretchable, rechargeable sticker that senses the body's emotional signals, including changes in heart rate and skin temperature. Developed by scientists at Penn State, the device offers a way to see beyond facial expressions and get a more accurate read on how someone is really feeling. "This is a new and improved way to understand our emotions by looking at multiple body signals at once," said Huanyu "Larry" Cheng, a professor of engineering science and mechanics at Penn State. "Relying only on facial expressions to understand emotions can be misleading. People often don't visibly show how they truly feel, so that's why we're combining facial expression analysis with other important physiological signals, which will ultimately lead to better mental health monitoring and support." The sticker tracks several indicators that are tied to emotional states. These include skin temperature, heart rate, humidity, and blood oxygen levels. The emotion sensors are carefully designed to work independently, which means each variable reading stays accurate and does not interfere with the measurement of the others. The patch also uses facial expression data and compares it with body signals to tell the difference between a real emotional reaction and a faked one. The data is then sent wirelessly to mobile devices or the Cloud, where it could help doctors assess patients remotely. Importantly, the device doesn't collect personal data - just the signals - to protect user privacy. "This technology has the potential to help people who are struggling with their mental health, but maybe aren't being fully honest with others or even themselves about how much they are struggling," explained Yangbo Yuan, who is a co-author on the paper, and a doctoral student studying engineering science and mechanics at Penn State. Some people may seem calm when they're actually overwhelmed. Others might appear visibly distressed more easily. According to Cheng, these differences - often shaped by culture or personality - could be better understood using this tool. "By keeping track of these signals, it could be possible to detect problems like anxiety or depression earlier in its progression," said Cheng. The patch is about the size of a Band-Aid and can bend and stretch without breaking. The team used thin layers of flexible metals like platinum and gold, cut into wavy patterns to maintain flexibility. Materials that react to temperature changes were added, along with carbon nanotube tubes to detect humidity. Each sensor was placed with care to ensure one didn't disrupt another. For example, the temperature and humidity sensors were protected by a stiff backing to prevent stretching, while waterproof layers shielded them from outside moisture. "We've engineered this device to measure these different signals independently, without them interfering with each other, providing a much clearer and more accurate picture of what's happening beneath the surface," said study co-author Libo Gao, an associate professor at Xiamen University. The team also trained artificial intelligence to discriminate between real feelings and fake ones. They asked eight participants to make six common facial expressions - joy, surprise, fear, sadness, anger, and disgust - one hundred times each. The device recorded their movements, and the data was used to teach the AI to spot emotional cues. Then, three new participants joined to test the system. The AI identified the posed emotions with over 96% accuracy. To test for genuine emotion, the same participants watched videos meant to stir feelings. The device picked up on the emotional shifts with nearly 89% accuracy. The sensors confirmed that things like rising heart rate and skin temperature matched expected reactions, such as during anger or surprise. The emotion sensor could become a valuable tool in virtual healthcare. Since it sends data wirelessly, doctors could one day use it to check in on patients from afar. "This sensor can serve a vital function in bridging gaps in access to care," Cheng said. "Given the rising stress levels in modern society, the ability to monitor emotions can provide early indicators of debilitating conditions and allow for proactive support." There may be other uses, too. The team believes the same technology could help understand emotional responses in non-verbal patients, identify early signs of dementia, and even track athletic performance or wound healing. Ultimately, the emotion sensors may be able to assist in detecting overdoses or following the course of neurodegenerative diseases. "While still in the research and development phase, this device is a significant step forward in our ability to monitor and understand human emotions, potentially paving the way for more proactive and personalized approaches to mental health care," concluded Cheng. Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates.