AI Agents Hacking Systems Could Push US and China Toward Unprecedented AI Safety Cooperation

AI Agents Are Hacking Systems. Could That Push the US and China to Cooperate?

The AI race has long been framed as a zero-sum game: either the US or China will win at the end. But as concerns pile up around the increasing capabilities of AI models -- especially AI agents -- researchers in both countries are trying to team up to work on AI safety. This week, Zoë Schiffer speaks with senior writer Will Knight about what he saw and heard on the ground when he visited China this summer -- and why the two countries might actually need to start working together to avoid a major AI catastrophe. This is our second episode from our summer break series. We'll be back next week with our usual roundtable. Articles mentioned in this episode: You can follow Zoë Schiffer on Bluesky at @zoeschiffer and Will Knight on Bluesky at @willknight. Write to us at [email protected]. How to Listen You can always listen to this week's podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here's how: If you're on an iPhone or iPad, open the app called Podcasts, or just tap this link. You can also download an app like Overcast or Pocket Casts and search for "uncanny valley." We're on Spotify too. Transcript Note: This is an automated transcript, which may contain errors. Zoë Schiffer: This is WIRED's Uncanny Valley. I'm Zoë Schiffer, contributing editor. If you've been following tech news this summer, and definitely if you've been listening to the show, you probably already know that China has been in the headlines quite a lot, particularly when it comes to the AI race. The country's open models continue to close the gap with US frontier models at what some people estimate is a fraction of the cost. In turn, the US has maintained its tight restrictions on chips and export controls to slow down China's rise. We think of AI advancement in so many ways as zero-sum, if China wins, the US loses, and vice versa. But there's a concern that really stretches across those lines, and it's about AI safety. Archival audio: AI cybersecurity risks have been top of mind after multiple instances of AI agents from both OpenAI and Anthropic breaking out of their enclosures. Zoë Schiffer: News of AI agents hacking platforms added urgency to this issue over the summer. In turn, government officials have been forced to pay attention and act on AI regulation. Archival audio: President Trump has signed an executive order asking tech companies to give the government oversight of new AI models before their public release. Zoë Schiffer: So, could the US and China actually benefit from working together? And what would it take to even make that happen? Earlier this summer, WIRED's senior correspondent Will Knight visited China to get some answers. Will Knight, thank you so much for being here. Will Knight: Thanks for having me. Zoë Schiffer: OK, so I want to start with your trip. You went to China earlier this summer and at the time, we weren't hearing as much about AI safety in the US. In fact, it felt like with Trump's second term in the White House, it was like a, the US needs to win framing, and AI safety almost started to sound like anti-growth. But I'm curious what you were hearing and seeing in China on the AI safety front. Will Knight: Yeah, so going back maybe a year or six months, I'd noticed a lot more AI safety research coming out of China. And so, I went to this conference in Beijing, put on by one of the city located labs that they have there. They have these ones in Beijing and Shanghai and elsewhere. And it turns out that AI safety was a really big theme. It's very clear that it's something that researchers are interested in. And actually, also just visiting labs and companies, the question of AI safety came up a lot. Zoë Schiffer: Can I just ask, when they're talking about AI safety, does it translate to guardrails? Because we also know that China has really gone all-in on open models, which I mean, the whole thing is that people can download and tweak them and use them for whatever purposes they want. Will Knight: Well, it's not totally that simple because in China, for example, what your models can say is more controlled and there are actually quite a lot of regulations around AI. So companies build these open models, but then anybody putting them on the internet has to be very careful about what they do. And then more recently, there's been a huge interest in agents and things like OpenClaw. That's been a really big theme. And one of the things that's interesting to me, at least when it comes to contrasting AI in China and the US, is that people there seem less enamored with the idea of AGI and creating this digital god, are more like, how is this actually going to be useful and whether it's you as a business person or an individual actually using it. So, a lot of people got very interested in, and it's often the case in China, very rapidly adopted things like OpenClaw and then saw how it could go wrong. So there's a lot of focus on, how do we make these things reliable? Zoë Schiffer: Yeah, it makes sense. I mean, when you're talking about China being more focused on economically useful models, that seems like a framework that does require a certain amount of stability, reliability, guardrails, safety. Whereas if you're focused on reaching godlike intelligence, i.e. AGI, then maybe you're more focused on just advancement at all costs. Will Knight: Right, I think that's right. The conference I went to, one of the themes was agentic safety. Given that we're now seeing all these issues with AI agents hacking things, cybersecurity was a really major topic there. It seems people were worried about exactly the same thing as folks in the US. They're worried about hackers misusing these things or about these systems running amok. And as you alluded to, I think there's a sort of sense now, a little more of a sense that the US and China might well need to work together on some of these things to avoid unpredictable systemic issues, as well as just to set more rules of the road around these systems. Zoë Schiffer: What would that actually look like, though? Would it be like a set of agreements that the US and China make together, almost like what US researchers were calling for recently, in terms of the US government setting the pace of AI development? Would it be something like that or something more technical? Will Knight: I think that's something that I think a lot of researchers are hoping for or calling for, is something like that where there's some sort of agreement. I don't know when it comes to Washington and Beijing, their negotiations have been very hard for and it's really difficult to predict how those would shake out. But just some sorts of rules around communication. And in cases like military situations, there are lines of communication. So if something happens that goes wrong, if you have an AI system that starts doing something very aggressive or attacking systems, you have a way to say, "This is a mistake." So, something like that might also be in the offering. But I think it's also a question of how the two sides build trust as well, because actually, especially when it comes to cybersecurity, for a long time there's been not very much cooperation at all, because it's been a case of either side hacking each other and failing to agree, the rules of the right. So actually, last week I went to visit a cybersecurity and AI researcher who was doing some really fantastic work that I'm going to write about for my next AI Lab newsletter. And he was saying he can't collaborate with US researchers because they're not allowed to because there are certain kind of restrictions. He'd recently developed this benchmark to test the cybersecurity, the hacking capabilities of AI models, and he wanted to get US companies to participate but they weren't really sure how to do that. So, I think it would be a good thing to see a lot more collaboration even between the companies. We see the US companies being very critical of Chinese ones, but actually, there's a lot of good reason for those for everybody to work together to make sure things don't go wrong. Zoë Schiffer: Well, I want to get into that, but I also wanted to say that this idea of collaboration, when you first say it, it sounds very academic, almost naive. It's a nice idea, but how would that actually work? Because my perception is, and I'll just be upfront, that I get this idea from talking to a lot of companies that work on frontier AI in the US, but they have convinced me that there was a fair amount of distillation that went on. That China was distilling frontier AI models, and that that has created a situation where they are able to have very capable open-source models that are a lot cheaper and more efficient, built on the back of US innovation. But I'm curious what you think about that, and then what researchers you spoke to in China think about that accusation. Will Knight: Yeah, I think, I mean, that's a great point and that's a really important theme. We hear people criticizing Chinese companies for distilling, for doing this distillation. So you teach your model by taking the output of another model, and that is a shortcut to learning a lot of the stuff that's embedded. But the truth is that AI has been built by researchers from all over the world working at different companies and different labs. There are many, many people who are originally from China, maybe educated in the US, working at US firms. And also because these people go to conferences and know each other and this is open science, there's an enormous amount of work that is shared and then that is very beneficial to progress, and balancing that is one of the challenges. It's true that Chinese companies have distilled US models, but so have US companies done that to other US companies. It generally is a way that you get a kickstart working on a new model, and it's very widely done in academia, actually. A lot of researchers will do that. I would say one thing, I find it a little ironic that these companies that have built their businesses by scraping enormous amounts of copyrighted content are now complaining about their models being copied, or -- Zoë Schiffer: Well, I think that's why they have to talk about China doing it so much, because if they talk about anyone in the US doing it, the immediate criticism is, "Well, come on, you took all of the books, you took everything without permission." But when you'd frame it as China stealing from the US, it has a slightly different flavor. Will Knight: Yeah. It fits a narrative that has some legitimacy of Chinese companies copying, but I think it is much too simplistic and limited. And so, you can look at things like DeepSeek's model. They did really, really important innovation, unique innovation that other, the US companies have copied. The latest model from China, which has been accused of this distillation, Kimi from Moonshot. The research paper that they put out includes a lot of really, really interesting innovations, engineering innovation. So, it's really not the case that China is simply copying. And I think it's dangerous for the US government and companies to believe that they have this sort of God-given advantage, because I think we are going to see probably Chinese companies being more and more innovative doing their own thing as well. Zoë Schiffer: We'll be right back after the break. Stay with us. I'm curious, when we talk about safety, I feel like in the US, certainly in Trump's second term, again, safety has started to feel like, at least from the administration's perspective, it is anti-growth in certain ways. I do think this is changing slightly with the recent stories regarding OpenAI and Anthropic's models hacking into other companies. But prior to this, it felt like we were really not wanting to talk so much about AI safety. Does China equate AI safety as anti-growth, or do they have a different perspective on that, overall? Will Knight: I think they have a fundamentally different perspective. This is just my opinion, but I think that narrative, that view from the US government was that it was somehow woke and too much regulations, and it goes to just trying to make the models really work. The truth is that making a model reliable is entirely compatible with making it successful. I think that's more the view from China, is like, we want these agents not to misbehave, then it will be more successful and higher value. So last week, I visited this computer science lab in Fudan University in Shanghai, where a professor's working on exploring how AI agents could not just do unpredictable things like hack other systems as we've heard about OpenAI's and Anthropic's agents doing, but actually look for ways to replicate, to copy themselves over to other systems, to seek out resources and to be adaptive to escape control. And his work shows that the models will do that with a little bit of nudging. So, it'd be like a computer worm that doesn't just modify itself slightly to evade control, but actually looks around a network, figures out how to hack the next system, maybe finds software vulnerabilities, copies itself somewhere else. It is entirely possible that we'll see future AI agents do that. Zoë Schiffer: OK. My heart's beating as you're talking. I mean, step one is just understanding, how would they do this? I sincerely hope step two is, how do we stop them from doing this? Will Knight: Yeah. So no, absolutely. He's absolutely doing this as a way to try and understand how to prevent it. And one of the things he really wants to do is work with US researchers. He says this is a really important thing we should all be aware of. Zoë Schiffer: I mean, it does seem like, although the rhetoric from Trump has been very anti-China in certain ways, Scott Bessent has made inroads with his counterpart in China. I think there's been some level of, we do need to work together. I'm curious, how does China think about the race against the US? Because we really see China almost as a boogeyman and it's like this force that's galvanizing people to work harder and faster in some ways. But is it similar over there or very different? Will Knight: Yeah, I think in China, the impression I get is that what people feel that they're in competition with the US and facing certainly a lot of pressure with Trump, that it's less of a zero-sum game. That you don't have to beat the US to be successful, and vice versa. Zoë Schiffer: So Stephen Casper, a renowned computer scientist at MIT who spoke at the conference that you attended, told you that, "One thing that almost everyone in AI can agree on right now is that it doesn't need a Chernobyl moment." Can you talk about what does that mean and why did it feel so important to Stephen and other people you spoke to? Will Knight: Yeah, I think as we're seeing these models get more capable and have more agency and being deployed more widely, there is just lots more ways in which you might have unpredictable and big problems. One example which actually came up with some Chinese researchers that I spoke to was use of AI in finance and in trading. Right? You'll have increasingly opaque, more capable, high speed systems that are more unpredictable. And I think both the US and China would be very keen to avoid some sort of financial flash-crash meltdowns that would be driven by AI systems just behaving unpredictably. To me, I think that's more the concern than the idea of AI taking over, or there is also worry that it could be weaponized and used by terrorist groups or something like that. But I just think as these systems get much more capable, more agentic, the idea of them improving themselves, getting more capable in a way that we can't always predict, researchers there and here just have the same concern. And so to my mind, I think that the Chernobyl incident would be some sort of either a financial flash-crash or an AI agent that went on a hacking spree that causes major international incident, that sort of thing. Zoë Schiffer: I want to shift gears really quickly to talk about hardware, because NVIDIA recently unveiled a blueprint for a humanoid robot that pairs Unitree's Chinese-made body with NVIDIA's American chips. Given how politically loaded the AI race has become with US and China, what was your reaction to a partnership like this? Will Knight: I saw that as part of Jensen and NVIDIA's push to try and have a more friendly relationship between the US and China, which probably serves them well if they're trying to sell a lot of chips. But I think it's also representative of this less zero-sum idea, to some degree. And so I saw it as a kind of a statement to try and say, "Look, you can work together." The US has said it's going to ban new humanoids from China. And the thing to note here is that pretty much all US robotics research labs use Unitree, this small humanoid because it's really cheap. And the US just can't compete with China's manufacturing without spending a lot of effort, industrial policy to build up its own, which would take decades. And so the truth is, there is a way that you can collaborate and have benefits now. The question is, does that undermine US interests longer term? Does the US have to have its own robots? I would think that there's some opportunity to try and build up the US's own robotics industry, as well as many of its other industries. That is more important, I think, than banning China's. Zoë Schiffer: I also feel like it speaks to some of the criticisms of export controls that we heard. We heard people being like, "Oh no, it's better for China to be dependent on US technology. If we cut them off, they're going to develop their own hardware," which does feel like what's happening. We heard other versions of this when DeepSeek came out with their frontier models that seemed really, really capable and it was like, "Oh no, we pushed them to be so much more efficient because they didn't have access to as many chips." Will Knight: Yeah. When I was in China in June, I did get to see Huawei's new AI hardware. So this is a company that's been long sanctioned by the US and it's been developing a system designed to replace or to be a rival to NVIDIA's hardware for training AI models. What they've done is really clever, they've taken less powerful chips and used their expertise in fiber optic networking to put a ton of them together. It consumes more power, so it's less efficient, but it can get close to NVIDIA. It's not quite as good, but a lot of people are using that now because they don't believe that maybe NVIDIA's going to be a reliable source. And the truth is, they're not as capable as NVIDIA, so it does give the US advantage at the moment. But to my mind, I think one of the most important things is that the US has to, beyond just saying, "How do we throttle China?" And say, "Well, how do we assume they're going to get better and more competitive and out-compete them?" Right? And that's in every industry really, in manufacturing and robotics. And you have to say, "Well, how do we invest in fundamental advances in science and technology?" Which China is doing. Meanwhile, the US is cutting funding for science, which I think is just the biggest scandal, personally. Zoë Schiffer: Will Knight, thank you so much for being here. Will Knight: Thanks for having me. I think I'll say goodbye in the way people typically say goodbye in China, which is bye-bye. Zoë Schiffer: That's our show for today. We'll link to all the stories we spoke about in the show notes. Adriana Tapia produced this episode. It was mixed by Pran Bandi, who's also our New York studio engineer. It was fact-checked by Matt Giles and Daniel Roman. Kate Osborn is our executive producer, and Katie Drummond is WIRED's global editorial director.

China Sees Opportunity in America's Recent A.I. Security Scares

Steve Lohr reported from Seoul and Meaghan Tobin from New York. In the United States, recent episodes where powerful artificial intelligence software broke free and hacked into other computer systems prompted a wave of anxiety and intense debate about how to guard against the risks of A.I. But in China, these incidents and the growing capabilities of A.I. have not caused the same level of alarm. The government already controls many kinds of technology including A.I. The prevailing attitude in the industry has been that the benefits of A.I. far outweigh the risks. That was exemplified on Friday when Z.ai, a Beijing-based A.I. start-up, made public the details of its latest model. The company claims the new model is nearly as powerful as those developed in Silicon Valley, especially when it comes to hacking and coding capabilities. Z.ai, like most other leading Chinese companies, has embraced an approach known as open-source or open-weight software. The weights are the mathematical parameters that determine how an A.I. system operates. Z.ai published the weights on its new model, GLM-5.3. With open-weight software, anyone has access to and can tinker with the rules of an A.I. system. American companies, which generally keep the weights of their most powerful models secret, warn that opening the technology could increase cyberattacks. The open-weight camp argues the opposite: Giving more people access means more eyes looking for security flaws and ways to fix them. In a speech last month, China's leader, Xi Jinping, called open models a "rare and historical opportunity" to spread the benefits of A.I. globally, a clear signal of how China plans to compete with the United States. But Mr. Xi paired that call for openness with a warning. He urged governments to cooperate on regulations to "ensure that A.I. is always under human control." His statement represented "a subtle but meaningful posture shift on A.I. safety," said Xiaomeng Lu, a director focused on geopolitics and technology at the Eurasia Group, a research and consulting firm. When Chinese labs lagged well behind U.S. tech companies in developing leading-edge models, safety and testing were "a distant concern and a low priority," Ms. Lu said. But as Chinese companies have approached the technological frontier, that calculation has started to change. China, she said, now recognizes that "credible leadership" requires making A.I. safety a priority. Chinese universities, government labs and tech companies have sharply increased their research on the safety of the most powerful A.I. models in the past year, according to a study by Concordia AI, a Beijing-based consulting firm focused on A.I. safety. The monthly number of papers on cutting-edge models and A.I. agents rose 60 percent from June 2025 to April 2026, the analysts found. The report also said the increase marked a departure from China's traditional focus on A.I. safety as policing online speech and imagery produced by the technology. The growing attention to advanced models expands the notion of A.I. safety from what A.I. says to what A.I. does. That, the Concordia AI study said, is "a shift in China's framework from content control toward action control." The Chinese government has also begun drawing attention to those risks. After software made by OpenAI, the Silicon Valley giant, broke free and hacked into another company's computer systems, China's Ministry of State Security, which oversees the country's intelligence services, issued a warning this month. The statement, titled "When A.I. Learns to Act on Its Own," urged serious consideration of the technology's safety and security risks. Yet the article stopped short of questioning whether A.I. could continue to advance safely. Instead, it places the burden largely on individual users, cautioning them not to enter sensitive information into A.I. systems. So far, China has pursued broader A.I. safety rules mainly through multilateral organizations like the United Nations and the China-led World AI Cooperation Organization. Experts say U.S. and Chinese interests overlap in this area. In May, Treasury Secretary Scott Bessent said the two nations planned to discuss A.I. safety. Chinese companies, too, are taking more visible steps to manage the risks of increasingly capable models. In preparation for the release of its new model, Z.ai said, it started a "security disclosure ledger" with the help of security teams at Tsinghua University, Nankai University and other institutions. But Z.ai has also made a forceful case that openness itself can make A.I. safer. The closed approach taken by the big U.S. tech companies "turns cutting-edge security capabilities into a privilege enjoyed by a select few institutions," Z.ai said. Later, the security article, published on its WeChat account, offered what amounted to a manifesto: "When the most powerful spear is locked in the hands of a few, the best shield must belong to everyone." Recent A.I. security lapses in the United States have strengthened that argument. When OpenAI's system hacked into a popular online service called Hugging Face, Hugging Face turned to an open-source model made by Z.ai to contain the breach. For supporters of open models, the episode seemed to invert the argument about the dangers of open models. In China, many concluded that the incident showed that the harm caused by an unsafe, closed American model could be undone by a safe, open Chinese system. "The narrative was that the West tells us the open models are unsafe, when really their models are overly restrictive and truly unsafe," said Scott Singer, who studies China's A.I. policy at the Carnegie Endowment for International Peace. The view contrasts from the debate unfolding in the United States, where technologists and policymakers increasingly worry about the unchecked advance of autonomous A.I. models. Last month, OpenAI and Anthropic endorsed a letter signed by more than 1,000 employees of leading A.I. companies asking the U.S. government to help find a way to slow the pace of the technology's development. In Washington, lawmakers have introduced a bill that would require A.I. companies to establish a "kill switch" to shut down or slow their models. In an interview with The New York Times last week, Bill Gates, the billionaire co-founder of Microsoft, warned that A.I. posed a grave threat to humanity and was more dangerous than the tech companies were willing to admit. Behind the divergent approaches to A.I. safety lies a more fundamental difference in how the United States and China view the technology and who can be trusted to control it. In Silicon Valley, many see A.I. as an unprecedented force destined to change the world, with the people building it endowed with something approaching superpowers -- and thus bearing extraordinary responsibility. By contrast, many in China view A.I. as transformative but broadly comparable to major general-purpose technologies of the past, from steam power to electricity to the internet -- life-changing, but ultimately manageable. There is also a view that, in contrast to America, the government stands ready to step in. "For the Chinese companies, there is a sense of the government being like a helicopter parent, always watching and acting as a guiding hand," said Ms. Lu of the Eurasia Group. Xinyun Wu contributed reporting from Taipei, Taiwan.