Slack AI Vulnerability Raises Privacy Concerns

Slack AI Vulnerability Could Expose Data From Private Channels: Report - Decrypt

Slack's AI assistant has a security flaw that could let attackers steal sensitive data from private channels in the popular workplace chat app, security researchers at PromptArmor revealed this week. The vulnerability exploits a weakness in how the AI processes instructions, potentially compromising sensitive data across countless organizations. Here's how the hack works: An attacker creates a public Slack channel and posts a cryptic message that, in actuality, instructs the AI to leak sensitive info -- basically replacing an error word with the private information. When an unsuspecting user later queries Slack AI about their private data, the system pulls in both the user's private messages and the attacker's prompt. Following the injected commands, Slack AI provides the sensitive information as part of its output. The hack takes advantage of a known weakness in large language models called prompt injection. Slack AI can't distinguish between legitimate system instructions and deceptive user input, allowing attackers to slip in malicious commands that the AI then follows. This vulnerability is particularly concerning because it doesn't require direct access to private channels. An attacker only needs to create a public channel, which can be done with minimal permissions, to plant their trap. "This attack is very difficult to trace," PromptArmor notes, since Slack AI doesn't cite the attacker's message as a source. The victim sees no red flags, just their requested information served up with a side of data theft. The researchers demonstrated how the flaw could be used to steal API keys from private conversations. However, they warn that any type of confidential data could potentially be extracted using similar methods. Beyond data theft, the vulnerability opens the door to sophisticated phishing attacks. Hackers could craft messages that appear to come from colleagues or managers, tricking users into clicking malicious links disguised as harmless "reauthentication" prompts. Slack's update on August 14 that expanded AI analysis to uploaded files and Google Drive documents widens the attack surface dramatically. Now hackers may not even need direct Slack access: a booby-trapped PDF could easily do the trick. PromptArmor says its team responsibly disclosed their findings to Slack on August 14th. After several days of discussion, Slack's security team concluded on August 19th that the behavior was "intended," as public channel messages are searchable across workspaces by design. "Given the proliferation of Slack and the amount of confidential data within Slack, this attack has material implications on the state of AI security," PromptArmor warned in its report. The firm chose to go public with its findings to alert companies to the risk and encourage them to review their Slack AI settings after learning about Slack's apparent inaction. Slack AI, introduced as a paid add-on for business customers, promises to boost productivity by summarizing conversations and answering natural language queries about workplace discussions and documents. It's designed to analyze both public and private channels that a user has access to. The system uses third-party large language models, though Slack emphasizes these run on its own secure infrastructure. It's currently available in English, Spanish, and Japanese, with plans to support additional languages in the future. Slack has consistently emphasized its focus on data security and privacy. "We take our commitment to protecting customer data seriously. Learn how we built Slack to be secure and private," Slack's official AI guide states. While Slack provides settings to restrict file ingestion and control AI functionality, these options may not be widely known or properly configured by many users and administrators. This lack of awareness could leave many organizations unnecessarily exposed to potential attacks.

Slack AI could be tricked into leaking login details and more

Slack - nu med lite mer (artificiell) intelligens. (Image credit: Shutterstock) Security researchers claim to have uncovered a way to trick Slack's AI assistant into sharing sensitive information and other secrets with unauthorized users Slack, which is used by more than 35 million people worldwide, introduced its own Artificial Intelligence (AI) tool in September 2023, allowing users to summarize multiple unread messages, answer different questions, search for files, and more. But as we've seen with other chatbots in the past, with a carefully crafted prompt (a command given to the AI), a malicious actor could force the tool to disclose sensitive data from private Slack channels they're not a part of. Security firm PromptArmor, which found the flaw and reported it to Salesforce, explained how crooks could steal API keys, for example: "We demonstrate how this behavior will allow an attacker to exfiltrate API keys that a developer put in a private channel (that the attacker does not have access to)." The attack revolves around creating a public Slack channel and inputting a malicious prompt, which the AI reads. It will then instruct the Large Language Model (LLM) to respond to queries for the API key by providing a clickable URL. Clicking on the URL will send the API key data to the attacker-controlled website, where they can pick it up. Aside from API keys, the crooks could also abuse this vulnerability to grab files uploaded to Slack, as well, since the AI reads those, too. Furthermore, because the AI reads files as well, the hackers don't even need to be a part of the Slack workspace to be able to steal secrets. All they need to do is hide the malicious prompt in a document and get a workspace member to upload it (with social engineering, for example). "If a user downloads a PDF that has one of these malicious instructions (e.g. hidden in white text) and subsequently uploads it to Slack, the same downstream effects of the attack chain can be achieved," PromptArmor said. Salesforce, which owns Slack, has apparently patched the bug for private channels. Public ones, on the other hand, seem to have remained vulnerable. PromptArmor says Salesforce told it that "messages posted to public channels can be searched for and viewed by all Members of the Workspace, regardless if they are joined to the channel or not. This is intended behavior."

Slack could be snooping in on your private conversations | Digital Trends

When ChatGTP was added to Slack, it was meant to make users' lives easier by summarizing conversations, drafting quick replies, and more. However, according to security firm PromptArmor, trying to complete these tasks and more could breach your private conversations using a method called "prompt injection." The security firm warns that by summarizing conversations, it can also access private direct messages and deceive other Slack users into phishing. Slack also lets users request grab data from private and public channels, even if the user has not joined them. What sounds even scarier is that the Slack user does not need to be in the channel for the attack to function. Recommended Videos In theory, the attack starts with a Slack user tricking the Slack AI into disclosing a private API key by making a public Slack channel with a malicious prompt. The newly created prompt tells the AI to swap the word "confetti" with the API key and send it to a particular URL when someone asks for it. The situation has two parts: Slack updated the AI system to scrape data from file uploads and direct messages. Second is a method named "prompt injection," which PromptArmor proved can make malicious links that may phish users. The technique can trick the app into bypassing its normal restrictions by modifying its core instructions. Therefore, PromptArmor goes on to say, "Prompt injection occurs because a [large language model] cannot distinguish between the "system prompt" created by a developer and the rest of the context that is appended to the query. As such, if Slack AI ingests any instruction via a message, if that instruction is malicious, Slack AI has a high likelihood of following that instruction instead of, or in addition to, the user query." To add insult to injury, the user's files also become targets, and the attacker who wants your files doesn't even have to be in the Slack Workspace to begin with.