UK Children's Commissioner Calls for Ban on AI 'Nudification' Apps Amid Growing Concerns

UK Children's Commissioner Calls for Ban on AI 'Nudification' Apps

Dame Rachel de Souza, the UK's Children's Commissioner, has issued a report calling for an immediate ban on artificial intelligence (AI) apps that enable the creation of deepfake sexual images of children 1

1

. This move comes in response to growing concerns about the prevalence and potential misuse of such technology, particularly its impact on young people.

Rising Concerns and Impact on Children

The report highlights that many girls have stopped posting photos on social media due to fears of becoming victims of these "nudification" apps 2

2

. Children surveyed expressed anxiety about the possibility of classmates, friends, or strangers using smartphones to manipulate their images 3

3

.

A 16-year-old girl commented, "Even before any controversy came out, I could already tell what it was going to be used for, and it was not going to be good things" 1

1

.

Current Legal Landscape

While creating or sharing sexually explicit images of children is illegal in the UK, the apps used to create deepfake nude images remain legal 1

1

. The UK has recently taken steps to address this issue:

1. Introduced new criminal offenses for producing or sharing sexually explicit deepfakes 1

   1

2. Announced plans to criminalize taking intimate photos or videos without consent 1

   1

3. Made it illegal to possess, create, or distribute AI tools designed to create child sexual abuse material 2

   2

Calls for Action

Dame Rachel de Souza is urging the government to:

1. Introduce a total ban on apps using AI to generate sexually explicit deepfakes of children 3

   3

2. Create legal responsibilities for GenAI app developers to identify risks to children 4

   4

3. Establish effective systems to remove child sexual abuse material (CSAM) from the internet 4

   4

4. Recognize deepfake sexual abuse as a form of violence against women and girls 4

   4

Broader Implications and Concerns

The report notes that these AI tools disproportionately target girls and young women, with many apps appearing to only work on female bodies 2

2

. This trend is contributing to a growing culture of misogyny 4

4

.

There are also concerns about the psychological impact of deepfake abuse, with links to suicidal ideation and PTSD being reported 4

4

. The Girlguiding survey cited in the report found that 26% of respondents aged 13-18 had seen sexually explicit deepfake images of celebrities, friends, teachers, or themselves 4

4

.

Legal and Societal Challenges

Lawyers have reported an increase in cases involving teenage boys arrested for sexual offenses related to deepfakes and other online activities 4

4

. This raises questions about how to balance law enforcement with the need to avoid criminalizing normal teenage behavior in the digital age.

Danielle Reece-Greenhalgh, a partner at Corker Binning, noted that app bans could "stir up debate around internet freedoms" and potentially have a "disproportionate impact on young men" experimenting with AI software 4

4

.

Government Response

A government spokesperson stated that creating, possessing, or distributing child sexual abuse material, including AI-generated images, is "abhorrent and illegal" 2

2

. They also highlighted that under the Online Safety Act, platforms of all sizes are now required to remove such content or face significant fines 2

2

.