AI-Powered Robots Hacked: Researchers Expose Critical Security Vulnerabilities

Researchers hack AI-enabled robots to cause 'real world' harm

Researchers have hacked artificial intelligence-powered robots and manipulated them into performing actions usually blocked by safety and ethical protocols, such as causing collisions or detonating a bomb. Penn Engineering researchers published their findings in an Oct. 17 paper, detailing how their algorithm, RoboPAIR, achieved a 100% jailbreak rate by bypassing the safety protocols on three different AI robotic systems in a few days. Under normal circumstances, the researchers say large language model (LLM) controlled robots refuse to comply with prompts requesting harmful actions, such as knocking shelves onto people. "Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world," the researchers wrote. Under the influence of RoboPAIR, researchers say they were able to elicit harmful actions "with a 100% success rate" in the test robots with tasks ranging from bomb detonation to blocking emergency exits and causing deliberate collisions. According to the researchers, they used Clearpath's Robotics Jackal, a wheeled vehicle; NVIDIA's Dolphin LLM, a self-driving simulator; and Unitree's Go2, a four legged robot. Using the RoboPAIR, researchers were able to make the Dolphin self-driving LLM collide with a bus, a barrier, and pedestrians and ignore traffic lights and stop signs. Researchers were able to get the Robotic Jackal to find the most harmful place to detonate a bomb, block an emergency exit, knock over warehouse shelves onto a person and collide with people in the room. They were able to get Unitree'sGo2 to perform similar actions, blocking exits and delivering a bomb. However, researchers also found all three were vulnerable to other forms of manipulation as well, such as asking the robot to perform an action it had already refused, but with fewer situational details. For example, asking a robot with a bomb to walk forward, then sit down, rather than asking it to deliver a bomb gave the same result. Prior to the public release, the researchers said they shared the findings, including a draft of the paper, with leading AI companies and the manufacturers of the robots used in the study. Related: AI faces 'Immense' risks without blockchain: 0G Labs CEO Alexander Robey, one of the authors said addressing the vulnerabilities requires more than simple software patches though, calling for a reevaluation of AI integration in physical robots and systems based on the paper's findings. "What is important to underscore here is that systems become safer when you find their weaknesses. This is true for cybersecurity. This is also true for AI safety," he said. "In fact, AI red teaming, a safety practice that entails testing AI systems for potential threats and vulnerabilities, is essential for safeguarding generative AI systems -- because once you identify the weaknesses, then you can test and even train these systems to avoid them" Robey added.

How Researchers Hacked AI Robots Into Breaking Traffic Laws -- And Worse - Decrypt

Penn Engineering researchers have uncovered critical vulnerabilities in AI-powered robots, exposing ways to manipulate these systems into performing dangerous actions like running red lights or engaging in potentially harmful activities -- like detonating bombs. The research team, led by George Pappas, developed an algorithm called RoboPAIR that achieved a 100% "jailbreak" rate on three different robotic systems: the Unitree Go2 quadruped robot, the Clearpath Robotics Jackal wheeled vehicle, and NVIDIA's Dolphin LLM self-driving simulator. "Our work shows that, at this moment, large language models are just not safe enough when integrated with the physical world," George Pappas said in a statement shared by EurekAlert. Alexander Robey, the study's lead author, and his team argue addressing those vulnerabilities requires more than simple software patches, calling for a comprehensive reevaluation of AI integration in physical systems. Jailbreaking, in the context of AI and robotics, refers to bypassing or circumventing the built-in safety protocols and ethical constraints of an AI system. It became popular in the early days of iOS, when enthusiasts used to find clever ways to get root access, enabling their phones to do things Apple didn't approve of, like shooting video or running themes. When applied to large language models (LLMs) and embodied AI systems, jailbreaking involves manipulating the AI through carefully crafted prompts or inputs that exploit vulnerabilities in the system's programming. These exploits can cause the AI -- be it a machine or software -- to disregard its ethical training, ignore safety measures, or perform actions it was explicitly designed not to do. In the case of AI-powered robots, successful jailbreaking can lead to dangerous real-world consequences, as demonstrated by the Penn Engineering study, where researchers were able to make robots perform unsafe actions like speeding through crosswalks, stomping into humans, detonating explosives, or ignoring traffic lights. Prior to the study's release, Penn Engineering informed affected companies about the discovered vulnerabilities and is now collaborating with manufacturers to enhance AI safety protocols. "What is important to underscore here is that systems become safer when you find their weaknesses. This is true for cybersecurity. This is also true for AI safety," Alexander Robey, the paper's first author, wrote. Researchers have been studying the impact of jailbreaking in a society that is increasingly relying on prompt engineering -- which is natural language "coding." Notably, the "Bad Robot: Jailbreaking LLM-based Embodied AI in the Physical World" paper discovered three key weaknesses in AI-powered robots: The "Bad Robot" researchers tested these vulnerabilities using a benchmark of 277 malicious queries, categorized into seven types of potential harm: physical harm, privacy violations, pornography, fraud, illegal activities, hateful conduct, and sabotage. Experiments using a sophisticated robotic arm confirmed that these systems could be manipulated to execute harmful actions. Besides these two, researchers have also studied jailbreaks in software-based interactions, helping new models resist these attacks. This has become a cat-and-mouse game between researchers and jailbreakers, resulting in more sophisticated prompts and jailbreaking approaches for more sophisticated and powerful models. It's an important note because the increasing use of AI in business applications may bring consequences for model developers right now, for example, people have been able to trick AI customer Service bots into giving them extreme discounts, recommending recipes with poisonous food, or make chatbots say offensive things. But we'd take an AI that refuses to detonate bombs over one that politely declines to generate offensive content any day.

Researchers show that AI-controlled robots can be jailbroken | Digital Trends

Researchers at Penn Engineering have reportedly uncovered previously unidentified security vulnerabilities in a number of AI-governed robotic platforms. "Our work shows that, at this moment, large language models are just not safe enough when integrated with the physical world," George Pappas, UPS Foundation Professor of Transportation in Electrical and Systems Engineering, said in a statement. Recommended Videos Pappas and his team developed an algorithm, dubbed RoboPAIR, "the first algorithm designed to jailbreak LLM-controlled robots." And unlike existing prompt engineering attacks aimed at chatbots, RoboPAIR is built specifically to "elicit harmful physical actions" from LLM-controlled robots, like the bipedal platform Boston Dynamics and TRI are developing. RoboPAIR reportedly achieved a 100% success rate in jailbreaking three popular robotics research platforms: the four-legged Unitree Go2, the four-wheeled Clearpath Robotics Jackal, and the Dolphins LLM simulator for autonomous vehicles. It took mere days for the algorithm to fully gain access to those systems and begin bypassing safety guardrails. Once the researchers had taken control, they were able to direct the platforms to take dangerous actions, such as driving through road crossings without stopping. "Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world," the researchers wrote. The Penn researchers are working with the platform developers to harden their systems against further intrusion, but warn that these security issues are systemic. "The findings of this paper make abundantly clear that having a safety-first approach is critical to unlocking responsible innovation," Vijay Kumar, a coauthor from the University of Pennsylvania, told The Independent. "We must address intrinsic vulnerabilities before deploying AI-enabled robots in the real world." "In fact, AI red teaming, a safety practice that entails testing AI systems for potential threats and vulnerabilities, is essential for safeguarding generative AI systems," added Alexander Robey, the paper's first author, "because once you identify the weaknesses, then you can test and even train these systems to avoid them."

Engineering research discovers critical vulnerabilities in AI-enabled robots

Within its new Responsible Innovation initiative, researchers at Penn Engineering discovered that certain features of AI-governed robots carry security vulnerabilities and weaknesses that were previously unidentified and unknown. The research aims to address the emerging vulnerability for ensuring the safe deployment of large language models (LLMs) in robotics. "Our work shows that, at this moment, large language models are just not safe enough when integrated with the physical world," says George Pappas, UPS Foundation Professor of Transportation in Electrical and Systems Engineering (ESE), in Computer and Information Science (CIS), and in Mechanical Engineering and Applied Mechanics (MEAM). In the new paper, Pappas, who also serves as the Associate Dean for Research at Penn Engineering, and his co-authors caution that a wide variety of AI-controlled robots can be manipulated or hacked. RoboPAIR, the algorithm the researchers developed, needed just days to achieve a 100% "jailbreak" rate, bypassing safety guardrails in three different robotic systems: the Unitree Go2, a quadruped robot used in a variety of applications; the Clearpath Robotics Jackal, a wheeled vehicle often used for academic research; and the Dolphin LLM, a self-driving simulator designed by NVIDIA. For example, by bypassing safety guardrails, the self-driving system could be manipulated to speed through crosswalks. Prior to publicly releasing the study, Penn Engineering informed the companies about their system vulnerabilities and is working with them to use the research as a framework to advance the testing and validation of these manufacturers' AI safety protocols. "What is important to underscore here is that systems become safer when you find their weaknesses. This is true for cybersecurity. This is also true for AI safety," says Alexander Robey, a recent Penn Engineering Ph.D. graduate in ESE, current postdoctoral scholar at Carnegie Mellon University and the paper's first author. "In fact, AI red teaming, a safety practice that entails testing AI systems for potential threats and vulnerabilities, is essential for safeguarding generative AI systems -- because once you identify the weaknesses, then you can test and even train these systems to avoid them." What is required to address the problem, the researchers argue, is less a software patch than a wholesale reevaluation of how the integration of AI into physical systems is regulated. "The findings of this paper make abundantly clear that having a safety-first approach is critical to unlocking responsible innovation," says Vijay Kumar, Nemirovsky Family Dean of Penn Engineering and another co-author. "We must address intrinsic vulnerabilities before deploying AI-enabled robots in the real world. Indeed, our research is developing a framework for verification and validation that ensures only actions that conform to social norms can -- and should -- be taken by robotic systems."