Eurostar accused security researchers of blackmail after AI chatbot vulnerability disclosure

Pen testers accused of 'blackmail' over Eurostar AI flaws

Researchers at Pen Test Partners found four flaws in Eurostar's public AI chatbot that, among other security issues, could allow an attacker to inject malicious HTML content or trick the bot into leaking system prompts. Their thank you from the company: being accused of "blackmail." The researchers reported the weaknesses to the high-speed rail service through its vulnerability disclosure program. While Eurostar ultimately patched some of the issues, during the responsible disclosure process, the train operator's head of security allegedly accused the pen-testing team of blackmail. Here's what happened, according to a blog published this week by the penetration testing and security consulting firm. After initially reporting the security issues - and not receiving any response - via a vulnerability disclosure program email on June 11, the bug hunter Ross Donald says he followed up with Eurostar on June 18. Still no response. So on July 7, managing partner Ken Munro contacted Eurostar's head of security on LinkedIn. About a week later, he was told to use the vulnerability reporting program (they had), and on July 31 learned there was no record of their bug report. "What transpired is that Eurostar had outsourced their VDP between our initial disclosure and hard chase," Donald wrote. "They had launched a new page with a disclosure form and retired the old one. It raises the question of how many disclosures were lost during this process." Eventually, Eurostar found the original email containing the report, fixed "some" of the flaws, and so Pen Test Partners decided to proceed with publishing the blog. But in the LinkedIn back-and-forth, Munro says: "Maybe a simple acknowledgement of the original email report would have helped?" And then, per a LinkedIn screenshot with Eurostar exec's name and photo blacked out, the security boss replied: "Some might consider this to be blackmail." The Register contacted Eurostar about this exchange, and asked whether it had fixed all of the chatbot's issues detailed in the blog. We did not receive an immediate response, but we will update this story if and when we hear back from the train operator. The flaws themselves are relatively easy to abuse and stem from the API-driven chatbot's design. Every time a user sends a message to the chatbot, the frontend relays the entire chat history - not just the latest message - to the API. But it only runs a guardrail check on the latest message to ensure that it's allowed. If that message is allowed, the server marks it "passed" and returns a signature. If the message doesn't pass the safety checks, however, the server responds with "I apologise, but I can't assist with that specific request" and no signature. Because the chatbot only verifies the latest message's signature, earlier messages can be tampered with on the user's screen, and then fed into the model as having passed the safety checks. As long as the user sends a legitimate, harmless message - such as asking the bot to build a travel itinerary - that passes the guardrail checks and returns a valid signature, they can then edit earlier messages in the chat history and trick the bot into leaking information it should not via prompt injection. Further prompt injection allowed the researcher to extract the system prompt and disclosed how the chatbot generated the HTML for its reference links. "That alone is reputationally awkward and can make future attacks easier, but the bigger risk is what happens once the chatbot is allowed to touch personal data or account details," Donald wrote. From there, with more poking, the chatbot revealed that it was vulnerable to HTML injection, which could be abused to trick the model into returning a phishing link or other malicious code inside what looks like a real Eurostar answer. Additionally, the backend didn't verify conversation and message IDs. This, combined with HTML injection, "strongly suggests a plausible path to stored or shared XSS," according to the researcher. Stored XSS, or cross-site scripting, occurs when an attacker injects malicious code into a vulnerable field - in this case, the chat history - and the application treats it as legitimate, delivering it to other users as trusted content and causing their browsers to execute the code. This type of attack is often used to hijack sessions, steal secrets, or send unwitting users to phishing websites. The pen testers say that they don't know if Eurostar fully fixed all of these security flaws. We've asked Eurostar about this and will report back when we receive a response. In the meantime, this should serve as a cautionary tale for companies with consumer-facing chatbots (and, these days, that's just about all of them) to build security controls in from the start. ®

Researchers say Eurostar accused them of blackmail over AI chatbot flaw disclosure - SiliconANGLE

Researchers say Eurostar accused them of blackmail over AI chatbot flaw disclosure Eurostar International Ltd., the operator of the Eurostar trains that cross the English Channel, has been accused of mishandling the responsible disclosure of security flaws in its customer-facing artificial intelligence chatbot after security researchers were allegedly told their actions could be viewed as blackmail. The allegation comes from U.K. security firm Pen Test Partners LLP, which said it identified multiple vulnerabilities in Eurostar's AI-powered chatbot earlier this year. The vulnerabilities were discovered during routine testing rather than as part of a commissioned engagement. The vulnerabilities detected included weaknesses in how the chatbot handled conversation history and message validation that could allow attackers to manipulate earlier messages in a chat session. The Pen Test Partners researchers were able to bypass safety guardrails, extract internal system information and inject arbitrary HTML into chatbot responses. While the chatbot was not connected to sensitive customer data, the firm warned that such flaws could become more serious if the system were later expanded to handle bookings, personal information, or account access. Being a legitimate company that practices ethical disclosure, Pen Test Partners attempted to report the issues through Eurostar's vulnerability disclosure process beginning in mid-June. After receiving no response, they followed up multiple times via email and later through LinkedIn, but then it gets weird. According to Pen Test Partners, a Eurostar security executive eventually responded but suggested that continued attempts to draw attention to the issue could be interpreted as blackmail. "To say we were surprised and confused by this has to be a huge understatement - we had disclosed a vulnerability in good faith, were ignored, so escalated via LinkedIn private message," explains Ross Donald, head of core pent test at Pen Test Partners, in a blog post. "I think the definition of blackmail requires a threat to be made and there was of course no threat. We don't work like that!" Eurostar later acknowledged that the original disclosure email had been overlooked and said some of the reported issues were subsequently addressed. Exactly what it fixed is unclear, however. "We still don't know if it was being investigated for a while before that, if it was tracked, how they fixed it, or if they even fully fixed every issue!," added Donald. As AI-powered customer interfaces become more widespread across industries, the Eurostar episode serves as a reminder that chatbot security is not just about AI behavior but also about the underlying software infrastructure that supports it. The case also highlights the need to have trained staff who are willing to work with security professionals instead of erroneously accusing them of wrongdoing. The security sector works best with cooperation, not with clown-level false threats of illegality.