Meta Plans to Shift 90% of Product Risk Assessments to AI, Raising Concerns

Meta will reportedly soon use AI for most product risk assessments instead of human reviewers

The report comes from NPR, which viewed internal documents about Meta's plans. According to a report from , Meta plans to shift the task of assessing its products' potential harms away from human reviewers, instead leaning more heavily on AI to speed up the process. Internal documents seen by the publication note that Meta is aiming to have up to 90 percent of risk assessments fall on AI, NPR reports, and is considering using AI reviews even in areas such as youth risk and "integrity," which covers violent content, misinformation and more. Unnamed current and former Meta employees who spoke with NPR warned AI may overlook serious risks that a human team would have been able to identify. Updates and new features for Meta's platforms, including Instagram and WhatsApp, have long been subjected to human reviews before they hit the public, but Meta has reportedly doubled down on the use of AI over the last two months. Now, according to NPR, product teams have to fill out a questionnaire about their product and submit this for review by the AI system, which generally provides an "instant decision" that includes the risk areas it's identified. They'll then have to address whatever requirements it laid out to resolve the issues before the product can be released. A former Meta executive told NPR that reducing scrutiny "means you're creating higher risks. Negative externalities of product changes are less likely to be prevented before they start causing problems in the world." In a statement to NPR, Meta said it would still tap "human expertise" to evaluate "novel and complex issues," and leave the "low-risk decisions" to AI. Read the full report over at . It comes a few days after Meta released its -- the first since and earlier this year. The amount of content taken down has unsurprisingly decreased in the wake of the changes, per the report. But there was a small rise in bullying and harassment, as well as violent and graphic content.

Meta plans to replace humans with AI to assess privacy and societal risks

People talk near a Meta sign outside of the company's headquarters in Menlo Park, Calif. Jeff Chiu/AP hide caption For years, when Meta launched new features for Instagram, WhatsApp and Facebook, teams of reviewers evaluated possible risks: Could it violate users' privacy? Could it cause harm to minors? Could it worsen the spread of misleading or toxic content? Until recently, what are known inside Meta as privacy and integrity reviews were conducted almost entirely by human evaluators. But now, according to internal company documents obtained by NPR, up to 90% of all risk assessments will soon be automated. In practice, this means things like critical updates to Meta's algorithms, new safety features and changes to how content is allowed to be shared across the company's platforms will be mostly approved by a system powered by artificial intelligence -- no longer subject to scrutiny by staffers tasked with debating how a platform change could have unforeseen repercussions or be misused. Inside Meta, the change is being viewed as a win for product developers, who will now be able to release app updates and features more quickly. But current and former Meta employees fear the new automation push comes at the cost of allowing AI to make tricky determinations about how Meta's apps could lead to real world harm. "Insofar as this process functionally means more stuff launching faster, with less rigorous scrutiny and opposition, it means you're creating higher risks," said a former Meta executive who requested anonymity out of fear of retaliation from the company. "Negative externalities of product changes are less likely to be prevented before they start causing problems in the world." Meta said in a statement that it has invested billions of dollars to support user privacy. Since 2012, Meta has been under the watch of the Federal Trade Commission after the agency reached an agreement with the company over how it handles users' personal information. As a result, privacy reviews for products have been required, according to current and former Meta employees. In its statement, Meta said the product risk review changes are intended to streamline decision-making, adding that "human expertise" is still being used for "novel and complex issues," and that only "low-risk decisions" are being automated. But internal documents reviewed by NPR show that Meta is considering automating reviews for sensitive areas including AI safety, youth risk and a category known as integrity that encompasses things like violent content and the spread of falsehoods. A slide describing the new process says product teams will now in most cases receive an "instant decision" after completing a questionnaire about the project. That AI-driven decision will identify risk areas and requirements to address them. Before launching, the product team has to verify it has met those requirements. Under the prior system, product and feature updates could not be sent to billions of users until they received the blessing of risk assessors. Now, engineers building Meta products are empowered to make their own judgements about risks. In some cases, including projects involving new risks or where a product team wants additional feedback, projects will be given a manual review by humans, the slide says, but it will not be by default, as it used to be. Now, the teams building products will make that call. "Most product managers and engineers are not privacy experts and that is not the focus of their job. It's not what they are primarily evaluated on and it's not what they are incentivized to prioritize," said Zvika Krieger, who was director of responsible innovation at Meta until 2022. Product teams at Meta are evaluated on how quickly they launch products, among other metrics. "In the past, some of these kinds of self-assessments have become box-checking exercises that miss significant risks," he added. Krieger said while there is room for improvement in streamlining reviews at Meta through automation, "if you push that too far, inevitably the quality of review and the outcomes are going to suffer." Meta downplayed concerns that the new system will introduce problems into the world, pointing out that it is auditing the decisions the automated systems make for projects that are not assessed by humans. The Meta documents suggest its users in the European Union could be somewhat insulated from these changes. An internal announcement says decision making and oversight for products and user data in the European Union will remain with Meta's European headquarters in Ireland. The EU has regulations governing online platforms, including the Digital Services Act, which requires companies including Meta to more strictly police their platforms and protect users from harmful content. Some of the changes to the product review process were first reported by The Information, a tech news site. The internal documents seen by NPR show that employees were notified about the revamping not long after the company ended its fact-checking program and loosened its hate speech policies. Taken together, the changes reflect a new emphasis at Meta in favor of more unrestrained speech and more rapidly updating its apps -- a dismantling of various guardrails the company has enacted over the years to curb the misuse of its platforms. The big shifts at the company also follow efforts by CEO Mark Zuckerberg to curry favor with President Trump, whose election victory Zuckerberg has called a "cultural tipping point." Another factor driving the changes to product reviews is a broader, years-long push to tap AI to help the company move faster amid growing competition from TikTok, OpenAI, Snap and other tech companies. Meta said earlier this week it is relying more on AI to help enforce its content moderation policies. "We are beginning to see [large language models] operating beyond that of human performance for select policy areas," the company wrote in its latest quarterly integrity report. It said it's also using those AI models to screen some posts that the company is "highly confident" don't break its rules. "This frees up capacity for our reviewers allowing them to prioritize their expertise on content that's more likely to violate," Meta said. Katie Harbath, founder and CEO of the tech policy firm Anchor Change, who spent a decade working on public policy at Facebook, said using automated systems to flag potential risks could help cut down on duplicative efforts. "If you want to move quickly and have high quality you're going to need to incorporate more AI, because humans can only do so much in a period of time," she said. But she added that those systems also need to have checks and balances from humans. Another former Meta employee, who spoke on condition of anonymity because they also fear reprisal from the company, questioned whether moving faster on risk assessments is a good strategy for Meta. "This almost seems self-defeating. Every time they launch a new product, there is so much scrutiny on it -- and that scrutiny regularly finds issues the company should have taken more seriously," the former employee said. Michel Protti, Meta's chief privacy officer for product, said in a March post on its internal communications tool, Workplace, that the company is "empowering product teams" with the aim of "evolving Meta's risk management processes." The automation roll-out has been ramping up through April and May, said one current Meta employee familiar with product risk assessments who was not authorized to speak publicly about internal operations. Protti said automating risk reviews and giving product teams more say about the potential risks posed by product updates in 90% of cases is intended to "simplify decision-making." But some insiders say that rosy summary of removing humans from the risk assessment process greatly downplays the problems the changes could cause. "I think it's fairly irresponsible given the intention of why we exist," said the Meta employee close to the risk review process. "We provide the human perspective of how things can go wrong."

Meta reportedly replacing human risk assessors with AI

Internal documents shed light on even more AI plans for Meta. Credit: Sebastian Kahnert / picture alliance via Getty Images According to new internal documents review by NPR, Meta is allegedly planning to replace human risk assessors with AI, as the company edges closer to complete automation. Historically, Meta has relied on human analysts to evaluate the potential harms posed by new technologies across its platforms, including updates to the algorithm and safety features, part of a process known as privacy and integrity reviews. But in the near future, these essential assessments may be taken over by bots, as the company looks to automate 90 percent of this work using artificial intelligence. Despite previously stating that AI would only be used to assess "low-risk" releases, Meta is now rolling out use of the tech in decisions on AI safety, youth risk, and integrity, which includes misinformation and violent content moderation, reported NPR. Under the new system, product teams submit questionnaires and receive instant risk decisions and recommendations, with engineers taking on greater decision-making powers. While the automation may speed up app updates and developer releases in line with Meta's efficiency goals, insiders say it may also pose a greater risk to billions of users, including unnecessary threats to data privacy. In April, Meta's oversight board published a series of decisions that simultaneously validated the company's stance on allowing "controversial" speech and rebuked the tech giant for its content moderation policies. "As these changes are being rolled out globally, the Board emphasizes it is now essential that Meta identifies and addresses adverse impacts on human rights that may result from them," the decision reads. "This should include assessing whether reducing its reliance on automated detection of policy violations could have uneven consequences globally, especially in countries experiencing current or recent crises, such as armed conflicts." Earlier that month, Meta shuttered its human fact-checking program, replacing it with crowd-sourced Community Notes and relying more heavily on its content-moderating algorithm -- internal tech that is known to miss and incorrectly flag misinformation and other posts that violate the company's recently overhauled content policies.

Meta Reportedly Eyes AI to Handle Risk Assessment Work

European Union regions will not see AI making risk assessment decisions Meta is reportedly planning to shift a large portion of risk assessments for its products and features to artificial intelligence (AI). As per the report, the Menlo Park-based social media giant is considering letting AI handle the approvals of its features and product updates, which were so far exclusively handled by human evaluators. This change will reportedly affect the addition of new algorithms, new safety features, and how content is shared across different social media platforms. The decision will reportedly boost the speed of rolling out new features, updates, and products. According to an NPR report, Meta is planning to automate up to 90 percent of all the internal risk assessments. The publication claimed to have obtained company documents that detail the possible shift in strategy. So far, any new features or updates for Instagram, WhatsApp, Facebook, or Threads have had to go through a group of human experts who reviewed the implications of how the change would impact users, whether it would violate their privacy, or bring harm to minors. The evaluations, reportedly known as privacy and integrity reviews, also assessed whether a feature could lead to a rise in misinformation or toxic content. With AI handling the risk assessment, product teams will reportedly receive an "instant decision" after they fill out a questionnaire about the new feature. The AI system is said to either approve the feature or provide a list of requirements that need to be fulfilled before the project can go ahead. The product team then has to verify that it has met those requirements before launching the feature, the report claimed. As per the report, the company believes shifting the review process to AI will significantly increase the release speed for features and app updates and allow product teams to work faster. However, some current and former Meta employees are reportedly concerned about whether this benefit will come at the cost of strict scrutiny. In a statement to the publication, Meta said that human reviewers were still being used for "novel and complex issues" and AI was only allowed to handle low-risk decisions. However, based on the documents, the report claims that Meta's planned transition includes letting AI handle potentially critical areas such as AI safety, youth risk, and integrity -- an area said to handle items such as violent content and "spread of falsehood." An unnamed Meta employee familiar with product risk assessments told NPR that the automation process started in April and has continued throughout May. "I think it's fairly irresponsible given the intention of why we exist. We provide the human perspective of how things can go wrong," the employee was quoted as saying. Notably, earlier this week, Meta released its Integrity Reports for the first quarter of 2025. In the report, the company stated, "We are beginning to see LLMs operating beyond that of human performance for select policy areas." The social media giant added that it has started using AI models to remove content from review queues in scenarios where it is "highly confident" that the said content does not violate its policies. Justifying the move, Meta added, "This frees up capacity for our reviewers allowing them to prioritise their expertise on content that's more likely to violate."

Meta to use AI for 90% of privacy and safety checks: Report

The shift has raised concerns internally. A former Meta executive told NPR that faster product rollouts with fewer checks could increase the risk of real-world harm. "Insofar as this process functionally means more stuff launching faster, with less rigorous scrutiny and opposition, it means you're creating higher risks," the former executive said on condition of anonymity. Meta Platforms plans to automate up to 90% of internal checks that evaluate privacy, safety, and risk implications across its apps -- including Instagram, WhatsApp, and Facebook -- using artificial intelligence, according to internal documents reviewed by US news publisher NPR. These product risk reviews, which previously relied heavily on human reviewers, assess whether new features could cause harm to users, violate privacy, or spread harmful content. Under the new system, AI tools will approve most updates -- including changes to Meta's core algorithms, safety tools, and content-sharing policies -- without requiring manual scrutiny or human debate. The internal documents indicate that human experts will only be involved in "novel or complex" cases, while low-risk changes will be fully automated. The shift has raised concerns internally. A former Meta executive told NPR that faster product rollouts with fewer checks could increase the risk of real-world harm. "Insofar as this process functionally means more stuff launching faster, with less rigorous scrutiny and opposition, it means you're creating higher risks," the former executive said on condition of anonymity. "Negative externalities of product changes are less likely to be prevented before they start causing problems in the world." Meta responded to the report saying the goal is to streamline decision-making while maintaining compliance and oversight. "We leverage technology to add consistency and predictability to low-risk decisions and rely on human expertise for rigorous assessments and oversight of novel or complex issues," a company spokesperson told TechCrunch. Meta is required to conduct internal privacy reviews under a 2012 agreement with the US Federal Trade Commission. These checks have so far been largely human-led. The company said it has invested more than $8 billion in its privacy programme and is committed to balancing innovation with compliance. Internal records cited in the report also suggest that Meta could extend AI oversight to highly sensitive areas including youth safety, misinformation, and AI-related risk.

Meta to handover most of product risk assessments to AI

Meta plans to automate risk assessments for up to 90% of app updates, raising concerns about potential real-world harm due to reduced human oversight. While Meta claims this increases efficiency and focuses human judgment on complex issues, critics worry about the potential for unchecked risks, especially in sensitive areas like AI safety and misinformation handling.Mark Zuckerberg's Meta is planning to automate risk assessments for up to 90% of updates made to its app, according to internal documents reviewed by NPR. A 2012 agreement between erstwhile Facebook and Federal Trade Commission over how the social media major handles users' information mandates it to conduct privacy reviews of its products, like WhatsApp and Instagram. So far, these reviews were being done by humans. This shift means that crucial decisions -- such as updates to Meta's algorithms, new safety tools, and changes to how users can share content -- will largely be approved by AI systems. These decisions will no longer go through the same level of human oversight that once involved in-depth discussions among staff about potential unintended consequences or the risk of misuse. Concerns galore Product developers in the company are celebrating the moves as a way to push updates and features out more rapidly. But current and former Meta employees warn that handing these complex decisions over to AI could increase the chances of real-world harm, the NPR report said. "If this change effectively means things are launching faster with less thorough scrutiny, then you're inviting more risk. The potential negative impacts of product decisions are less likely to be caught before they cause real damage," said a former Meta executive on the condition of anonymity. Meta, in a statement, said it has invested billions to protect user privacy. Since a 2012 settlement with the Federal Trade Commission over its data handling practices, Meta has been required to conduct privacy reviews for all new products, according to past and current employees. The company said the changes to the risk review process aim to make decision-making more efficient, noting that human judgment will still be applied to "novel and complex issues," and that automation is limited to "low-risk" cases. However, internal documents reviewed by NPR suggest that Meta is also looking to automate reviews in highly sensitive areas, such as AI safety, risks to minors, and "integrity" matters -- an umbrella term that includes the handling of violent content and misinformation.