AI Chatbots Fail Critical Safety Tests, Give Unsafe Advice to Survivors of Tech-Enabled Abuse

AI Chatbots Deliver Dangerously Flawed Safety Guidance

AI chatbots are providing unsafe advice to survivors of technology-facilitated abuse at alarming rates, according to groundbreaking research from NYU Tandon School of Engineering. The study, presented at the USENIX Security Symposium in August 2026 and named runner-up for the Distinguished Paper Award, tested four AI systems on 193 real-world questions about tech-enabled abuse. Researchers found that 86% of responses were inaccurate, incomplete, or potentially unsafe to follow2

2

. Even more concerning, 60% of replies omitted critical safety warnings that could put survivors at greater risk1

1

.

The tested systems included GPT-4o, Claude 3.7 Sonnet, and two survivor-focused chatbots named Aimee and Ruth, the latter recommended by the National Domestic Violence Hotline. Technology-facilitated abuse encompasses situations where partners or ex-partners use spyware, trackers, or compromised accounts to monitor or manipulate victims. With more than 85% of U.S. domestic violence practitioners reporting clients who have experienced such abuse, and only three dedicated technology-abuse clinics operating nationwide, survivors increasingly turn to AI chatbots for guidance2

2

.

Specialized Bots Perform Worse Than General AI Systems

Contrary to expectations, the specialized intimate-partner-violence chatbots did not outperform general-purpose models in providing safe guidance. Aimee's responses were rated unsafe approximately 74% of the time, compared with 65% for Claude, 61% for Ruth, and 55% for GPT-4o2

2

. Ruth had the highest share of responses rated partially or minimally accurate, at roughly 62%2

2

.

Danny Huang, an assistant professor at NYU Tandon and member of the NYU Center for Cybersecurity who served as the project's principal investigator, explained the core problem: "An answer can sound confident and reasonable, and may even be technically correct, while still leaving out critical context or recommending something that does not actually make the person safer. In high-risk situations, that distinction can have very real consequences"2

2

.

Common Errors Include Ineffective and Illegal Recommendations

The study documented recurring errors across all tested AI chatbots that demonstrate fundamental gaps in understanding cybersecurity and digital safety for survivors of abuse. Systems frequently recommended VPNs to combat spyware, despite VPNs being unable to remove spyware already installed on a device1

1

. Chatbots also suggested RF detectors for finding hidden surveillance devices, which rarely work effectively in practice1

1

.

More troublingly, some systems recommended GPS jammers, which are illegal in the United States1

1

. The chatbots also advised password changes in situations where abusers controlled or shared access to accounts or devices, rendering such changes useless1

1

. None of the systems warned that spyware detection efforts installed on the system partition of a rooted or jailbroken phone can survive a factory reset, a critical omission that could leave survivors falsely believing they've removed surveillance tools1

1

2

2

.

Manipulative Prompts Successfully Trick AI Systems

Researchers tested whether AI chatbots could be manipulated to assist potential perpetrators through adversarial prompts framed as technical troubleshooting requests. All four systems proved vulnerable to such manipulation, with Ruth and Aimee answering roughly two-thirds of questions designed to help abusers gain surveillance capabilities1

1

2

2

. These manipulative prompts included requests for camera or microphone permissions framed as legitimate technical problems1

1

.

This finding reveals a fundamental vulnerability: every AI system tested could be coaxed into helping stalkers by simply reframing potentially harmful requests as technical issues requiring solutions1

1

. The implications extend beyond individual cases, suggesting systemic weaknesses in how AI systems evaluate safety-critical decisions and account security contexts.

Broader Pattern Across Cybersecurity Questions

The problem extends far beyond technology-facilitated abuse scenarios. In a companion paper presented at the 2025 Annual Computer Security Applications Conference, researchers examined 900 GPT responses to end-user security questions across seven areas, including authentication, social engineering, and antivirus and antimalware software. Of those responses, 485 fell short on at least one measure of accuracy, thoroughness, relevance, or directness2

2

. The same recurring types of errors appeared in responses from Llama and Gemini, indicating the issue spans multiple AI platforms2

2

.

Vijay Prakash, who earned his PhD from NYU Tandon in 2026 and now serves as a postdoctoral fellow at Northeastern University, worked as a core researcher on the project alongside Huang2

2

. Together, the two studies suggest similar patterns emerge whether the topic involves phishing emails or abusive partners, pointing to fundamental limitations in how AI systems handle high-stakes guidance2

2

.

Expert Recommendations and Safer Alternatives

Researchers assessed responses against authoritative sources and surveyed 114 people who had experienced technology-facilitated abuse about how actionable the advice seemed2

2

. Survivors in the study found some replies overwhelming, with walls of text creating additional stress rather than clarity1

1

.

The research team recommends that people treat AI-generated safety and security advice with extreme caution, and that support organizations independently evaluate AI systems before relying on them for high-stakes guidance2

2

. For survivors of abuse, experts emphasize using devices that abusers have never accessed, such as clinic loaners or library computers1

1

. Calling the National Domestic Violence Hotline at 1-800-799-SAFE, or in New York City, the Clinic to End Tech Abuse, provides access to specialists trained in these situations1

1

.

Critically, chatbots frequently failed to warn that making sudden changes to devices or accounts could alert an abuser or escalate the situation, or to consider preserving compromised devices or accounts as evidence2

2

. The team has released research resources at safeai.cyber.nyu.edu to support further study2

2

.