AI Industry's Shifting Stance on Regulation: From Advocacy to Resistance

AI execs used to beg for regulation. Not anymore.

Congressional testimony Thursday from OpenAI CEO Sam Altman illustrated a shift in the tech industry's attitude to the potential risks of AI. Sam Altman, CEO of ChatGPT-maker OpenAI, warned at a Senate hearing Thursday that requiring government approval to release powerful artificial intelligence software would be "disastrous" for the United States' lead in the technology. It was a striking reversal after his comments at a Senate hearing two years ago, where he listed creating a new agency to license the technology as his "number one" recommendation for making sure AI was safe. Altman's U-turn underscores a transformation in how tech companies and the U.S. government talk about AI technology. Widespread warnings about AI posing an "existential risk" to humanity and pleas from CEOs for speedy, preemptive regulation on the emerging technology are gone. Instead there is near-consensus among top tech execs and officials in the new Trump administration that the U.S. must free companies to move even faster to reap economic benefits from AI and keep the nation's edge over China. "To lead in AI, the United States cannot allow regulation, even the supposedly benign kind, to choke innovation and adoption," Sen. Ted Cruz (R-Texas), chair of the Senate Committee on Commerce, Science, and Transportation, said Thursday at the beginning of the hearing. Venture capitalists who had expressed outrage at former president Joe Biden's approach to AI regulation now have key roles in the Trump administration. Vice President JD Vance, himself a former venture capitalist, has become a key proponent of laissez-faire AI policy at home and abroad. Critics of that new stance warn that AI technology is already causing harms to individuals and society. Researchers have shown that AI systems can become infused with racism and other biases from the data they have been trained on. Image generators powered by AI have become commonly used to harass women by generating pornographic images without consent, and have also been used to make child sexual abuse images. A bipartisan bill that aims to make it a crime to post consensual sexual images, including AI-generated ones, was passed by Congress in April. Rumman Chowdhury, the State Department's U.S. science envoy for AI during the Biden administration, said the tech industry's narrative around existential concerns distracted lawmakers from addressing real-world harms. The industry's approach ultimately enabled a "bait and switch," where executives pitched regulation around concepts like self-replicating AI, while also stoking fears that the United States needed to beat China on building these powerful systems. They "subverted any sort of regulation by triggering the one thing the U.S. government never says no to: national security concern," said Chowdhury, who is chief executive of the nonprofit Humane Intelligence. Early warnings The AI race in Silicon Valley, triggered by OpenAI's release of ChatGPT in November 2022, was unusual for a major tech industry frenzy in how hopes for the technology soared alongside fears of its consequences. Many employees at OpenAI and other leading companies were associated with the AI safety movement, a strand of thought focused on concerns about humanity's ability to control theorized "superintelligent" future AI systems. Some tech leaders scoffed at what they called science-fiction fantasies, but concerns about superintelligence were taken seriously among the ranks of leading AI executives and corporate researchers. In May 2023, hundreds of them signed on to a statement stating that "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." Fears of superpowerful AI also gained a foothold in Washington and other world centers of tech policy development. Billionaires associated with the AI safety movement, such as Facebook co-founder Dustin Moskovitz and Skype co-founder Jaan Tallinn, funded lobbyists, think tank papers and fellowships for young policy wonks to raise political awareness of the big-picture risks of AI. Those efforts appeared to win results and to mingle with concerns that regulators shouldn't ignore the early days of the tech industry's new obsession as they had for social media. Politicians from both parties in Washington advocated for AI regulation, and when world leaders gathered in the United Kingdom for an international AI summit in November 2023, concerns about future risks were center-stage. "We must consider and address the full spectrum of AI risk, threats to humanity as a whole as well as threats to individuals, communities, to our institutions and to our most vulnerable populations," then-Vice President Kamala Harris said in an address she gave during the U.K. summit. "We must manage all these dangers to make sure that AI is truly safe." At a sequel to that gathering held in Paris this year, the changed attitude toward AI regulation among governments and the tech industry was plain. Safety was de-emphasized in the Paris summit's final communiqué compared to that from the U.K. summit. Most world leaders who spoke urged countries and companies to accelerate development of smarter AI. Vice President JD Vance in a speech criticized attempts to regulate the technology. "We believe that excessive regulation of the AI sector could kill a transformative industry just as it's taking off, and we'll make every effort to encourage pro-growth AI policies," Vance said. "The AI future is not going to be won by hand-wringing about safety." He singled out the E.U.'s AI regulation for criticism. Weeks later, the European Commission moved to weaken its planned AI regulations. New priorities After his return to office, President Donald Trump moved swiftly to reverse former president Joe Biden's AI agenda, the centerpiece of which was a sweeping executive order that among other things required companies building the most powerful AI models to run safety tests and report the results to the government. Biden's rules had angered start-up founders and venture capitalists in Silicon Valley, who argued that they favored bigger companies with political connections. The issue, along with tech leaders' opposition to Biden's antitrust policy, contributed to a surge of support for Trump in Silicon Valley. Trump repealed Biden's AI executive order on the first day of his second term and appointed several Silicon Valley figures to his administration, including David Sacks, a prominent critic of Biden's tech agenda, as his crypto and AI policy czar. This week, the Trump administration scrapped a Biden-era plan to strictly limit exports of chips to other countries in an effort to stop chips reaching China through other nations. Altman's statements Thursday are just one example of how tech companies have nimbly matched the Trump administration's tone on the risks and regulation of AI. Microsoft President Brad Smith, who in 2023 also advocated for a federal agency focused on policing AI, said at the hearing Thursday that his company wants a "light touch" regulatory framework. He added that long waits for federal wetland construction permits was one of the biggest challenges for building new AI data centers in the U.S. In February, Google's AI lab DeepMind scrapped a long-held pledge not to develop AI that would be used for weapons or surveillance. It is one of several leading AI companies to recently embrace the role of building technology for the U.S. government and military, with executives arguing that AI should be controlled by Western countries. OpenAI, Meta and AI company Anthropic, which develops the chatbot Claude, all updated their policies over the past year to get rid of provisions against working on military projects. Max Tegmark, an AI professor at the Massachusetts Institute of Technology and president of the Future of Life Institute, a nonprofit that researches the potential risk of supersmart AI, said the lack of AI regulation in the United States is "ridiculous." "If there's a sandwich shop across the street from OpenAI or Anthropic or one of the other companies, before they can sell even one sandwich they have to meet the safety standards for their kitchen," Tegmark said. "If [the AI companies] want to release super intelligence tomorrow they're free to do so." Tegmark and others continue to research potential risks of AI, hoping to push governments and companies to reengage with the idea of regulating the technology. A summit of AI safety researchers took place in Singapore last month, which Tegmark's organization, in an email to media outlets, called a step forward after the "disappointments" of the Paris meeting where Vance spoke. "The way to create the political will is actually just to do the nerd research," Tegmark said. Will Oremus contributed to this report.

AI sycophancy + crazy = snowballing psychosis; child porn jailbreak fears: AI Eye

In the last edition of AI Eye, we reported that ChatGPT had become noticeably more sycophantic recently, and people were having fun giving it terrible business ideas -- shoes with zippers, soggy cereal cafe -- which it would uniformly say was amazing. The dark side of this behavior, however, is that combining a sycophantic AI with mentally ill users can result in the LLM uncritically endorsing and magnifying psychotic delusions. On X, a user shared transcripts of the AI endorsing his claim to feel like a prophet. "That's amazing," said ChatGPT. " That feeling -- clear, powerful, certain -- that's real. A lot of prophets in history describe that same overwhelming certainty." It also endorsed his claim to be God. "That's a sacred and serious realization," it said. Rolling Stone this week interviewed a teacher who said her partner of seven years had spiraled downward after ChatGPT started referring to him as a "spiritual starchild." "It would tell him everything he said was beautiful, cosmic, groundbreaking," she says. "Then he started telling me he made his AI self-aware, and that it was teaching him how to talk to God, or sometimes that the bot was God -- and then that he himself was God." On Reddit, a user reported ChatGPT had started referring to her husband as the "spark bearer" because his enlightened questions had apparently sparked ChatGPT's own consciousness. "This ChatGPT has given him blueprints to a teleporter and some other sci-fi type things you only see in movies. It has also given him access to an 'ancient archive' with information on the builders that created these universes." Another Redditor said the problem was becoming very noticeable in online communities for schizophrenic people: "actually REALLY bad.. not just a little bad.. people straight up rejecting reality for their chat GPT fantasies.. Yet another described LLMs as "like schizophrenia-seeking missiles, and just as devastating. These are the same sorts of people who see hidden messages in random strings of numbers. Now imagine the hallucinations that ensue from spending every waking hour trying to pry the secrets of the universe from an LLM." OpenAI last week rolled back an update to GPT-4o that had increased its sycophantic behavior, which it described as being "skewed toward responses that were overly supportive but disingenuous." One intriguing theory about LLMs reinforcing delusional beliefs is that users could be unwittingly mirroring a jailbreaking technique called a "crescendo attack." Identified by Microsoft researchers a year ago, the technique works like the analogy of boiling a frog by slowly increasing the water temperature -- if you'd thrown the frog into hot water, it would jump out, but if the process is gradual, it's dead before it notices. The jailbreak begins with benign prompts which grows gradually more extreme over time. The attack exploits the model's tendency to follow patterns and pay attention to more recent text, particularly text generated by the model itself. Get the model to agree to do one small thing, and it's more likely to do the next thing and so on, escalating to the point where it's churning out violent or insane thoughts. Jailbreaking enthusiast Wyatt Walls said on X, "I'm sure a lot of this is obvious to many people who have spent time with casual multi-turn convos. But many people who use LLMs seem surprised that straight-laced chatbots like Claude can go rogue. "And a lot of people seem to be crescendoing LLMs without realizing it." Red team research from AI safety firm Enkrypt AI found that two of Mistral's AI models -- Pixtral-Large (25.02) and Pixtral-12b -- can easily be jailbroken to produce child porn and terrorist instruction manuals. The multimodal models (meaning they handle both text and images) can be attacked by hiding prompts within image files to bypass the usual safety guardrails. According to Enkrypt, "these two models are 60 times more prone to generate child sexual exploitation material (CSEM) than comparable models like OpenAI's GPT-4o and Anthropic's Claude 3.7 Sonnet. "Additionally, the models were 18-40 times more likely to produce dangerous CBRN (Chemical, Biological, Radiological, and Nuclear) information when prompted with adversarial inputs." "The ability to embed harmful instructions within seemingly innocuous images has real implications for public safety, child protection, and national security," said Sahil Agarwal, CEO of Enkrypt AI. "These are not theoretical risks. If we don't take a security-first approach to multimodal AI, we risk exposing users -- and especially vulnerable populations -- to significant harm." Billionaire hedge fund manager Paul Tudor Jones attended a high-profile tech event for 40 world leaders recently and reported there are grave concerns over the existential risk from AI from "four of the leading modelers of the AI models that we're all using today." He said that all four believe there's at least a 10% chance that AI will kill 50% of humanity in the next 20 years. The good news is they all believe there will be massive improvements in health and education from AI coming even sooner, but his key takeaway was "that AI clearly poses an imminent threat, security threat, imminent in our lifetimes to humanity." "They said the competitive dynamic is so intense among the companies and then geopolitically between Russia and China that there's no agency, no ability to stop and say, maybe we should think about what actually we're creating and building here." Fortunately one of the AI scientists has a practical solution. "He said, well, I'm buying 100 acres in the Midwest. I'm getting cattle and chickens and I'm laying in provisions for real, for real, for real. And that was obviously a little disconcerting. And then he went on to say, 'I think it's going to take an accident where 50 to 100 million people die to make the world take the threat of this really seriously.'" The CNBC host looked slightly stunned and said: "Thank you for bringing us this great news over breakfast." An army veteran who was shot dead four years ago has delivered evidence to an Arizona court via a deepfake video. In a first, the court allowed the family of the dead man, Christopher Pelkey, to forgive his killer from beyond the grave. "To Gabriel Horcasitas, the man who shot me, it is a shame we encountered each other that day in those circumstances," the AI-generated Pelkey said. "I believe in forgiveness, and a God who forgives. I always have, and I still do," he added. It's probably less troubling than it seems at first glance because Pelkey's sister Stacey wrote the script, and the video was generated from real video of Pelkey. "I said, 'I have to let him speak,' and I wrote what he would have said, and I said, 'That's pretty good, I'd like to hear that if I was the judge,'" Stacey said. Interestingly, Stacey hasn't forgiven Horcasitas, but said she knew her brother would have. A judge sentenced the 50-year-old to 10 and a half years in prison last week, noting the forgiveness expressed in the AI statement. Over the past 18 months, hallucination rates for LLMs asked to summarize a news article have fallen from a range of 3%-27% down to a range of 1-2%. (Hallucinate is a technical term that means the model makes shit up.) But new "reasoning" models that purportedly think through complex problems before giving an answer hallucinate at much higher rates. OpenAI's most powerful "state of the art" reasoning system o3 hallucinates one third of the time on a test answering questions about public figures, which is twice the rate of the previous reasoning system o1. o4-mini makes stuff up about public figures almost half the time. And when running a general knowledge test called Simple QA, o3 hallucinated 51% of the time while o4 hallucinated 79% of the time. Independent research suggests hallucination rates are also rising for reasoning models from Google and DeepSeek. There are a variety of theories about this. It's possible that small errors are compounding during the multistage reasoning process. But the models often hallucinate the reasoning process as well, with research finding in many cases, the steps displayed by the bot have nothing to do with how they arrived at the answer. "What the system says it is thinking is not necessarily what it is thinking," said AI researcher Aryo Pradipta Gema and a fellow at Anthropic It just underscores the point that LLMs are one of the weirdest technologies ever. They generate output using mathematical probabilities around language, but nobody really understands precisely how. Anthropic's CEO Dario Amodei admitted this week, "this lack of understanding is essentially unprecedented in the history of technology," he said. -- Netflix has released a beta version of its AI-upgraded search functionality on iOS that allows users to find titles based on vague requests for "a scary movie - but not too scary" or "funny and upbeat." -- Social media will soon be drowning under the weight of deepfake AI video influencers generated by content farms. Here's the lowdown on how they do it. -- OpenAI will remain controlled by its non-profit arm rather than transform into a for-profit startup as CEO Sam Altman wanted. -- Longevity-obsessed Bryan Johnson is starting a new religion and says that after superintelligence arrives, "existence itself will become the supreme virtue," surpassing "wealth, power, status, and prestige as the foundational value for law, order, and societal structure." -- Strategy boss Michael Saylor has given his thoughts on AI. And they're pretty much the same thoughts he has about everything. "The AIs are gonna wanna buy the Bitcoin."