Tech Giants Unite to Combat AI-Driven Security Threats

How trust and safety leaders at top tech companies are approaching the security threat of AI: 'Trust but verify'

Safety officers at large companies that have integrated AI tools like ChatGPT into their businesses are issuing similar warnings to their colleagues: "Trust, but verify." Speaking at Fortune's Brainstorm Tech conference on Tuesday, Salesforce chief trust officer Brad Arkin detailed how the company, and its CEO Marc Benioff, balance the demand from customers to offer cutting-edge AI services while ensuring that it does not open them up to vulnerabilities. "Trust is more than just security," Arkin said, adding that the company's key focus is to create new features for its users that don't go against their interests. Against the backdrop of breakneck adoption of AI, however, is the reality that AI makes it easier for criminals to attack potential victims. Malicious actors can operate without the barrier of language, for example, while also being able to more easily send a massive volume of social engineering scams like phishing emails. Companies have long dealt with the threat of so-called "shadow IT," or the practice of employees using hardware and software that is not managed by a firm's technology department. Shadow AI could create even more vulnerabilities, especially without proper training. Still, Arkin said that AI should be approached like any tool -- there will always be dangers, but proper instruction can lead to valuable results. Speaking on the panel, Cisco's chief security and trust officer Anthony Grieco shared advice that he passes on to employees about generative AI platforms like ChatGPT. "If you wouldn't tweet it, if you wouldn't put it on Facebook, if you wouldn't publish it publicly, don't put it into those tools," Grieco said." Even with proper training, the ubiquity of AI -- and the rise of increased cybersecurity threats -- means that every company has to rethink its approach to IT. A working paper published in October by the non-profit National Bureau of Economic Research found rapid adoption of AI across the country, especially among the largest firms. More than 60% of companies with more than 10,000 employees are using AI, the group said. Wendi Whitmore, the senior vice president for the "special forces unit" of the cybersecurity giant Palo Alto Networks, said on Tuesday that cybercriminals have deeply researched how businesses operate, including how they work with vendors and operators. As a result, employees should be trained to scrutinize every piece of communication for potential phishing or other related attacks. "You can be concerned about the technology and put some limitations around it," she said. "But the reality is that attackers don't have any of those limitations." Despite the novel perils, Accenture global security lead Lisa O'Connor touted the potential posed by what she called "responsible AI," or the need for organizations to implement a set of governance principles for how they want to adopt the technology. She added that Accenture has long embraced large language models, including working with Fortune on its own custom-trained LLM. "We drank our own champagne," O'Connor said.

Salesforce's AI chief says the company uses its Einstein products internally: 'We like to drink our own martinis'

Salesforce's AI chief says companies are frustrated by generative AI's unreliability, such as hallucinations, or when AI spouts incorrect or biased information, and that the problem keeps many of them from widely releasing products that incorporate it. That leads to a broader question for businesses, Clara Shih, CEO of Salesforce AI, said on stage at Fortune's Brainstorm Tech conference in Park City, Utah, on Monday: "Do I trust AI to drive business?'" For many, the answer isn't so simple. "Companies come to us and they want to know how they can really deploy these solutions in a way that actually moves the needle," she explained. Salesforce is well-positioned to tackle this challenge, Shih argued, because customers already have entrusted their data and business processes with Salesforce over the years. "It provides this ideal grounding for the AI to really inject the context that's needed for the models really to be able to perform," she said. Shih, who was appointed as the company's first-ever head of AI just a few months after OpenAI released ChatGPT in November 2022, said trust is critical for all of today's generative AI -- even beyond core trust issues such as data security and data privacy. She emphasized that Salesforce is implementing all of its AI internally, which helps the company understand their customers' concerns. "We like to drink our own martinis," Shih joked. "It's bumpy at times, but I really like that accountability of us being customer zero." Salesforce has actually been developing AI to help clients, who primarily use its software for customer service, sales, and automating their marketing, since releasing its original Einstein AI product in 2016. Inspired by Salesforce CEO Marc Benioff's admiration for Albert Einstein, it was focused on predictive AI, which was cutting edge technology at the time. A few months after OpenAI's buzzy ChatGPT launched in November 2022, Salesforce debuted EinsteinGPT, one of the first chatbots from a major company to be based on a large language model. Salesforce followed up with a variety of other AI products including Einstein Copilot, Copilot Studio, Prompt Builder, RAG, Hybrid Search, and Einstein Trust Layer. On Monday, Shih touted a new planned offering called Einstein Service Agent, a chatbot specifically designed to be used by customer service agents. The chatbot is trained on a company's data and can easily hand off its work to a human customer service agent if needed. Shih said the chatbot's focus may be expanded beyond customer service in the future. But, she added, trust is key for these types of products -- and trust itself depends on the specific context in which the tools are being used. For example, she said, "Do I trust the AI to help me figure out which customers to call on this order?" and "Do I trust the AI to help me answer customers' questions" using Salesforce's new chatbot? Clearly, at least for Salesforce, Shih insists the answer is yes.