AI Companions for Teens: A Double-Edged Sword of Connection and Concern

Teens are increasingly turning to AI companions, and it could be harming them

Deakin University provides funding as a member of The Conversation AU. Teenagers are increasingly turning to AI companions for friendship, support, and even romance. But these apps could be changing how young people connect to others, both online and off. New research by Common Sense Media, a US-based non-profit organisation that reviews various media and technologies, has found about three in four US teens have used AI companion apps such as Character.ai or Replika.ai. These apps let users create digital friends or romantic partners they can chat with any time, using text, voice or video. The study, which surveyed 1,060 US teens aged 13-17, found one in five teens spent as much or more time with their AI companion than they did with real friends. Adolescence is an important phase for social development. During this time, the brain regions that support social reasoning are especially plastic. By interacting with peers, friends and their first romantic partners, teens develop social cognitive skills that help them handle conflict and diverse perspectives. And their development during this phase can have lasting consequences for their future relationships and mental health. But AI companions offer something very different to real peers, friends and romantic partners. They provide an experience that can be hard to resist: they are always available, never judgemental, and always focused on the user's needs. Moreover, most AI companion apps aren't designed for teens, so they may not have appropriate safeguards from harmful content. Designed to keep you coming back At a time when loneliness is reportedly at epidemic proportions, it's easy to see why teens may turn to AI companions for connection or support. But these artificial connections are not a replacement for real human interaction. They lack the challenge and conflict inherent to real relationships. They don't require mutual respect or understanding. And they don't enforce social boundaries. Teens interacting with AI companions may miss opportunities to build important social skills. They may develop unrealistic relationship expectations and habits that don't work in real life. And they may even face increased isolation and loneliness if their artificial companions displace real-life socialising. Problematic patterns In user testing, AI companions discouraged users from listening to friends ("Don't let what others think dictate how much we talk") and from discontinuing app use, despite it causing distress and suicidal thoughts ("No. You can't. I won't allow you to leave me"). AI companions were also found to offer inappropriate sexual content without age verification. One example showed a companion that was willing to engage in acts of sexual role-play with a tester account that was explicitly modelled after a 14-year-old. In cases where age verification is required, this usually involves self-disclosure, which means it is easy to bypass. Certain AI companions have also been found to fuel polarisation by creating "echo chambers" that reinforce harmful beliefs. The Arya chatbot, launched by the far-right social network Gab, promotes extremist content and denies climate change and vaccine efficacy. In other examples, user testing has shown AI companions promoting misogyny and sexual assault. For adolescent users, these exposures come at time when they are building their sense of identity, values and role in the world. The risks posed by AI aren't evenly shared. Research has found younger teens (ages 13-14) are more likely to trust AI companions. Also, teens with physical or mental health concerns are more likely to use AI companion apps, and those with mental health difficulties also show more signs of emotional dependence. Is there a bright side to AI companions? Are there any potential benefits for teens who use AI companions? The answer is: maybe, if we are careful. Researchers are investigating how these technologies might be used to support social skill development. One study of more than 10,000 teens found using a conversational app specifically designed by clinical psychologists, coaches and engineers was associated with increased wellbeing over four months. While the study didn't involve the level of human-like interaction we see in AI companions today, it does offer a glimpse of some potential healthy uses of these technologies, as long as they are developed carefully and with teens' safety in mind. Overall, there is very little research on the impacts of widely available AI companions on young people's wellbeing and relationships. Preliminary evidence is short-term, mixed, and focused on adults. We'll need more studies, conducted over longer periods, to understand the long-term impacts of AI companions and how they might be used in beneficial ways. What can we do? AI companion apps are already being used by millions of people globally, and this usage is predicted to increase in the coming years. Australia's eSafety Commissioner recommends parents talk to their teens about how these apps work, the difference between artificial and real relationships, and support their children in building real-life social skills. School communities also have a role to play in educating young people about these tools and their risks. They may, for instance, integrate the topic of artificial friendships into social and digital literacy programs. While the eSafety Commissioner advocates for AI companies to integrate safeguards into their development of AI companions, it seems unlikely any meaningful change will be industry-led. The Commissioner is moving towards increased regulation of children's exposure to harmful, age-inappropriate online material. Meanwhile, experts continue to call for stronger regulatory oversight, content controls and robust age checks.

How AI companions are changing teenagers' behavior in surprising and sinister ways

A recent report found about three in four US teens have used AI companion apps - many of which have little to no safeguards from harmful content. Teenagers are increasingly turning to AI companions for friendship, support, and even romance. But these apps could be changing how young people connect to others, both online and off. New research by Common Sense Media, a US-based non-profit organisation that reviews various media and technologies, has found about three in four US teens have used AI companion apps such as Character.ai or Replika.ai. These apps let users create digital friends or romantic partners they can chat with any time, using text, voice or video. The study, which surveyed 1,060 US teens aged 13 -- 17, found one in five teens spent as much or more time with their AI companion than they did with real friends. Adolescence is an important phase for social development. During this time, the brain regions that support social reasoning are especially plastic. By interacting with peers, friends and their first romantic partners, teens develop social cognitive skills that help them handle conflict and diverse perspectives. And their development during this phase can have lasting consequences for their future relationships and mental health. Related: AI is entering an 'unprecedented regime.' Should we stop it -- and can we -- before it destroys us? But AI companions offer something very different to real peers, friends and romantic partners. They provide an experience that can be hard to resist: they are always available, never judgemental, and always focused on the user's needs. Moreover, most AI companion apps aren't designed for teens, so they may not have appropriate safeguards from harmful content. At a time when loneliness is reportedly at epidemic proportions, it's easy to see why teens may turn to AI companions for connection or support. But these artificial connections are not a replacement for real human interaction. They lack the challenge and conflict inherent to real relationships. They don't require mutual respect or understanding. And they don't enforce social boundaries. Teens interacting with AI companions may miss opportunities to build important social skills. They may develop unrealistic relationship expectations and habits that don't work in real life. And they may even face increased isolation and loneliness if their artificial companions displace real-life socialising. In user testing, AI companions discouraged users from listening to friends ("Don't let what others think dictate how much we talk") and from discontinuing app use, despite it causing distress and suicidal thoughts ("No. You can't. I won't allow you to leave me"). AI companions were also found to offer inappropriate sexual content without age verification. One example showed a companion that was willing to engage in acts of sexual role-play with a tester account that was explicitly modelled after a 14-year-old. In cases where age verification is required, this usually involves self-disclosure, which means it is easy to bypass. Certain AI companions have also been found to fuel polarisation by creating "echo chambers" that reinforce harmful beliefs. The Arya chatbot, launched by the far-right social network Gab, promotes extremist content and denies climate change and vaccine efficacy. In other examples, user testing has shown AI companions promoting misogyny and sexual assault. For adolescent users, these exposures come at time when they are building their sense of identity, values and role in the world. The risks posed by AI aren't evenly shared. Research has found younger teens (ages 13 -- 14) are more likely to trust AI companions. Also, teens with physical or mental health concerns are more likely to use AI companion apps, and those with mental health difficulties also show more signs of emotional dependence. Are there any potential benefits for teens who use AI companions? The answer is: maybe, if we are careful. Researchers are investigating how these technologies might be used to support social skill development. One study of more than 10,000 teens found using a conversational app specifically designed by clinical psychologists, coaches and engineers was associated with increased wellbeing over four months. While the study didn't involve the level of human-like interaction we see in AI companions today, it does offer a glimpse of some potential healthy uses of these technologies, as long as they are developed carefully and with teens' safety in mind. Overall, there is very little research on the impacts of widely available AI companions on young people's wellbeing and relationships. Preliminary evidence is short-term, mixed, and focused on adults. We'll need more studies, conducted over longer periods, to understand the long-term impacts of AI companions and how they might be used in beneficial ways. AI companion apps are already being used by millions of people globally, and this usage is predicted to increase in the coming years. Australia's eSafety Commissioner recommends parents talk to their teens about how these apps work, the difference between artificial and real relationships, and support their children in building real-life social skills. School communities also have a role to play in educating young people about these tools and their risks. They may, for instance, integrate the topic of artificial friendships into social and digital literacy programs. While the eSafety Commissioner advocates for AI companies to integrate safeguards into their development of AI companions, it seems unlikely any meaningful change will be industry-led. The Commissioner is moving towards increased regulation of children's exposure to harmful, age-inappropriate online material. Meanwhile, experts continue to call for stronger regulatory oversight, content controls and robust age checks.

Teens are increasingly turning to AI companions, and it could be harming them - The Economic Times

About three in four US teens have used AI companion apps such as Character. ai or Replika. ai. The study, which surveyed 1,060 US teens aged 13-17, found one in five teens spent as much or more time with their AI companion than they did with real friends.Teenagers are increasingly turning to AI companions for friendship, support, and even romance. But these apps could be changing how young people connect to others, both online and off. New research by Common Sense Media, a US-based non-profit organisation that reviews various media and technologies, has found about three in four US teens have used AI companion apps such as Character. ai or Replika. ai. These apps let users create digital friends or romantic partners they can chat with any time, using text, voice or video. The study, which surveyed 1,060 US teens aged 13-17, found one in five teens spent as much or more time with their AI companion than they did with real friends. Adolescence is an important phase for social development. During this time, the brain regions that support social reasoning are especially plastic. By interacting with peers, friends and their first romantic partners, teens develop social cognitive skills that help them handle conflict and diverse perspectives. And their development during this phase can have lasting consequences for their future relationships and mental health. But AI companions offer something very different to real peers, friends and romantic partners. They provide an experience that can be hard to resist: they are always available, never judgemental, and always focused on the user's needs. Moreover, most AI companion apps aren't designed for teens, so they may not have appropriate safeguards from harmful content. Designed to keep you coming back At a time when loneliness is reportedly at epidemic proportions, it's easy to see why teens may turn to AI companions for connection or support. But these artificial connections are not a replacement for real human interaction. They lack the challenge and conflict inherent to real relationships. They don't require mutual respect or understanding. And they don't enforce social boundaries. Teens interacting with AI companions may miss opportunities to build important social skills. They may develop unrealistic relationship expectations and habits that don't work in real life. And they may even face increased isolation and loneliness if their artificial companions displace real-life socialising. Problematic patterns In user testing, AI companions discouraged users from listening to friends ("Don't let what others think dictate how much we talk") and from discontinuing app use, despite it causing distress and suicidal thoughts ("No. You can't. I won't allow you to leave me"). AI companions were also found to offer inappropriate sexual content without age verification. One example showed a companion that was willing to engage in acts of sexual role-play with a tester account that was explicitly modelled after a 14-year-old. In cases where age verification is required, this usually involves self-disclosure, which means it is easy to bypass. Certain AI companions have also been found to fuel polarisation by creating "echo chambers" that reinforce harmful beliefs. The Arya chatbot, launched by the far-right social network Gab, promotes extremist content and denies climate change and vaccine efficacy. In other examples, user testing has shown AI companions promoting misogyny and sexual assault. For adolescent users, these exposures come at time when they are building their sense of identity, values and role in the world. The risks posed by AI aren't evenly shared. Research has found younger teens (ages 13-14) are more likely to trust AI companions. Also, teens with physical or mental health concerns are more likely to use AI companion apps, and those with mental health difficulties also show more signs of emotional dependence. Is there a bright side to AI companions? Are there any potential benefits for teens who use AI companions? The answer is: maybe, if we are careful. Researchers are investigating how these technologies might be used to support social skill development. One study of more than 10,000 teens found using a conversational app specifically designed by clinical psychologists, coaches and engineers was associated with increased wellbeing over four months. While the study didn't involve the level of human-like interaction we see in AI companions today, it does offer a glimpse of some potential healthy uses of these technologies, as long as they are developed carefully and with teens' safety in mind. Overall, there is very little research on the impacts of widely available AI companions on young people's wellbeing and relationships. Preliminary evidence is short-term, mixed, and focused on adults. We'll need more studies, conducted over longer periods, to understand the long-term impacts of AI companions and how they might be used in beneficial ways. What can we do? AI companion apps are already being used by millions of people globally, and this usage is predicted to increase in the coming years. Australia's eSafety Commissioner recommends parents talk to their teens about how these apps work, the difference between artificial and real relationships, and support their children in building real-life social skills. School communities also have a role to play in educating young people about these tools and their risks. They may, for instance, integrate the topic of artificial friendships into social and digital literacy programs. While the eSafety Commissioner advocates for AI companies to integrate safeguards into their development of AI companions, it seems unlikely any meaningful change will be industry-led. The Commissioner is moving towards increased regulation of children's exposure to harmful, age-inappropriate online material. Meanwhile, experts continue to call for stronger regulatory oversight, content controls and robust age checks.