Congress Challenges Tech Giants on Deepfake Regulation Amid Rising Concerns

Congress asking Apple and other big tech what they're doing about deepfakes | AppleInsider

Non-consensual intimate images being made with apps on iPhone Pointed letters from Congress have been sent to tech executives like Apple's Tim Cook, stating concerns over the prevalence of deepfake non-consensual intimate images. The letters stem from earlier reports about nude deepfakes being created using dual-use apps. Advertisements popped up all over social media promoting face swapping, and users were using them to place faces into nude or pornographic images. According to a new report from 404 Media, the US Congress is taking action based on these reports, asking tech companies how it plans to stop non-consensual intimate images from being generated on their platforms. Letters were sent to Apple, Alphabet, Microsoft, Meta, ByteDance, Snap, and X. The letter to Apple specifically calls out the company for missing such dual-use apps despite App Review Guidelines. It asks what Apple plans to do to curb the spread of deepfake pornography and mentions the TAKE IT DOWN Act. The following questions were asked of Tim Cook: Apple acts as the steward to the App Store, and in doing so, gets the blame anytime something gets into the store that shouldn't. These instances of deepfake tools or children's apps that become casinos are used as fuel in the argument that Apple shouldn't be the gatekeeper to its app platform. The apps identified by previous reports that were removed by Apple showed clear signs of potential abuse. For example, users could source videos from Pornhub for face swapping. Apple Intelligence can't make nude or pornographic images, and Apple has stopped Sign-in with Apple from working on deepfake websites. However, that's just the bare minimum of what Apple can and should be doing to combat deepfakes. As the letter from Congress suggests, Apple needs to take steps to ensure that dual-use apps can't get through review. Extra precautions should at least be taken with apps that promise AI image and video manipulation, especially those that offer face swapping capabilities.

Congress Pushes Apple to Remove Deepfake Apps After 404 Media Investigation

Congress is asking Apple, Google, Microsoft, and other big tech companies why they can't get their deepfake problem under control. A bipartisan group of members of Congress has sent letters to Google's and Apple's CEOs citing 404 Media's reporting and asking what the giant tech companies are doing to address the rampant problem of nonconsensual AI-generated intimate media enabled on their platforms. The Congress members also sent a letter to Microsoft CEO Satya Nadella given Microsoft Designer's role in creating the infamous nonconsensual nude images of Taylor Swift that were viewed millions of times on Twitter, a story 404 Media broke as well. "Earlier this year, Apple removed three apps used to create deepfakes off of its app store after an independent report by 404 Media provided links to the apps and their related ads," the letter to Apple CEO Tim Cook said. "While it is positive that these apps were removed, it is concerning that Apple was not able to identify the apps on their own. The App Store requires developers to undergo a screening process, but the persistence of these apps illustrate that loopholes exist in Apple's guidelines. As Apple works to address these loopholes, we would like to understand what steps are being taken, and what additional guidelines may need to be put in place to curb the spread of deepfake pornography." The letter, which was signed by 26 Republican and Democratic House Representatives was sent to Cook on November 25, and is referencing 404 Media story from April about Apple removing a number of face swapping apps which were explicitly advertising their ability to create nonconsensual porn. Apple removed those apps after we published a story earlier in April about those ads appearing on Instagram. "As Congress works to keep up with shifts in technology, Republicans and Democrats will continue to ensure that online platforms do their part to collaborate with lawmakers and protect users from potential abuse," the letter says, and then presents Cook with a series of questions like "What plans are in place to proactively address the proliferation of deepfake pornography on your platform, and what is the timeline of deployment for those measures?" A separate letter sent to Google CEO Sundar Pichai by the same members of Congress about Google's role in allowing apps to advertise their ability to create nonconsensual deepfake porn in Google Search. "Earlier this year, Google announced it would ban advertisements for websites and services that produce deepfake pornography," the letter says, referring to a Google ad policy change we covered in May. "As you know, the emergence of deepfakes has resulted in an increase in ads for programs that cater to users who wish to produce sexually explicit content. While Google's updated policy instructs AI app developers to build in precautions against offensive content, adds in-app flagging and reporting mechanisms for users, and devalues deepfake porn results in internal search. However, despite these efforts, recent reports have highlighted that Google continues to promote results for apps that use AI to generate nonconsensual nude images. This development raises concerns about Google's complicity and role in the proliferation of deepfakes. We would like to further clarify the outcome of these updates and understand what additional guidelines may need to be put in place to curb the spread of deepfake pornography, including efforts to remove deepfake platforms from Google's search results." The letter cites a story we published in August which showed that searching for "undress apps," "best deepfake nudes," and similar terms on Google turns up "promoted" results for apps that use AI to produce nonconsensual nude images. Google removed those advertisers in response to our story. That letter also goes on to ask what plans Google is putting in place to practically address this problem. The same members of congress also sent letters to the CEOs of Facebook, TikTok, and Snapchat regarding nonconsensual content on their platforms as well. As I wrote in August, face swapping apps present Google and Apple with a very difficult "dual use" problem where the apps can present themselves as benign on the app stores, but promote their ability to produce harmful content off platform. There are more measures these tech companies can put in place to mitigate the problem, but fundamentally any face swapping app has the potential to create harmful content and has to be moderated closely in order to prevent that harm. Monitoring the huge number of apps that are added to these app stores on any given day would be a major and new investment from both Google and Apple.