The Rise of Deepfake Scams: How AI Is Fueling Celebrity Impersonation Fraud

The rise of deepfake scams -- and how not to fall for one

Would you act on an apparent stock tip from Martin Wolf, the FT's veteran chief economics commentator? His expert analysis features in many legitimate financial videos published on the FT's social media accounts, but scammers have generated a spate of convincing deepfake video images on Instagram where he appears to be offering investment advice. "Right now, these three stocks are at a critical turning point and could see significant gains in the next two months," says a convincing looking, yet digitally manipulated Wolf, in a fake advert inviting people to join his "exclusive WhatsApp investment group" to find out more. Meta, owner of WhatsApp and Instagram, has told the FT it has removed and disabled the ads, but readers would be wise to watch out for more scams of this nature as deepfake scams go mainstream. What's behind the rise in deepfake scams? The rapid rise of generative AI (artificial intelligence). The technology needed to create synthetic videos and imagery is cheap, readily available and easy for scammers to use to create convincing-looking content. Deepfakes of celebrities including Taylor Swift, Elon Musk and the stars of Dragons' Den have been created to promote everything from kitchenware to crypto scams and diet pills. Last year, a British man lost £76,000 to a deepfake scam where Martin Lewis, the founder of Money Saving Expert, appeared to be promoting a non-existent bitcoin investment scheme. The scammers have now one crucial advantage, said Nick Stapleton, presenter of the award-winning BBC series Scam Interceptors and author of the book How to Beat Scammers. "Deepfakes are working like a charm for the scammers, because many social media users simply don't know what generative AI is capable of when it comes to making convincing impersonation videos," he said. "They see a video like Martin Wolf's deepfake and believe it is real because they just don't have the information to question it." Lewis, who claims to have the "weird accolade" of being the most-scammed face in Britain, warned about the rise in deepfake adverts on ITV's Good Morning Britain this week. "I wouldn't trust an advert with a celebrity if you've only seen it on social media if it's talking about investment or dieting or any of the other scam areas," he said. "If it's got me in it, I don't ever do adverts, so it's a fake. Anything rushing you to make money . . . fake, fake, fake, don't trust them, they're criminals." What other forms can deepfake videos take? Celebrities are not the only ones whose images can be cloned. Fraud experts say deepfakes are increasingly being used on video calls to impersonate senior staff at corporate organisations, persuading other staff members to process payments which turn out to be fraudulent. Last year, UK engineering firm Arup lost $25mn (£20mn) when a staff member in Hong Kong was persuaded to make 15 bank transfers after scammers digitally cloned the company's chief financial officer on a video conference call. Online influencers who post videos and images of their faces on social media platforms are particularly vulnerable, as the more content of an individual there is to train the AI, the more realistic the copycat will be. As the technology progresses, deepfake videos have the potential to make romance scams even more convincing, and could potentially be used to manipulate images of friends and family members making requests for money. What more should social media platforms do? While social media platforms say they will use facial recognition to spot and take down fake ads, they don't have to stay up for long to gain traction. As Martin Wolf himself has said: "How is it possible that a company like Meta with its huge resources, including artificial intelligence tools, cannot identify and take down such frauds automatically, particularly when informed of their existence?" Meta told the FT: "It's against our policies to impersonate public figures and we have removed and disabled the ads, accounts, and pages that were shared with us. "Scammers are relentless and continuously evolve their tactics to try to evade detection, which is why we're constantly developing new ways to make it harder for scammers to deceive others -- including using facial recognition technology," Meta added. "The simple fact is that when these videos are placed on social media as ads, they go through a vetting process," Stapleton said. "If the likes of Meta would simply consider investing more of their vast profits into better vetting, and better moderation of general posts, this would become less of an issue very quickly." Under the UK's new Online Safety Act, tech companies must set performance targets to remove illegal material quickly when they become aware of it and test algorithms to make illegal content harder to disseminate. How can you spot a video is likely a deepfake? Stapleton's top tip for spotting digitally manipulated images is to look at the mouth of the person supposedly talking on camera: is it really making the shapes of the words? Next, look at their skin: is it flat in texture, without definition or wrinkles? And look at their eyes: do they blink at all, or far too much? Finally, listen to the tone of their voice. "AI struggles with the range of human voices, so deepfakes will tend to sound very flat and even in tone and lacking emotion," he said. The deepfake video of the FT's Wolf did not sound much like him, but as so many social media users watch videos on silent and read the captions, the scammers gain a further advantage. Finally, be especially wary of adverts on social media. If you can't find the information reported anywhere else, it's almost certainly a fake. What to do you if have been impersonated online Report the scam to the social media outlet, using the platform's reporting tools. Also, let your friends and followers know about the fake account to prevent them from being misled. If you have been a victim of this deepfake scam, or continue to see the deepfake video of Martin Wolf on social media platforms, please share your experience with the FT at [email protected]

The Guide #189: Your new celebrity best friend? It's just a deepfake trying to con you

In this week's newsletter: AI impostors are charming fans out of fortunes - here's how TV's Scam Interceptors catch the criminals This week's newsletter is written by Nick Stapleton and Mark Lewis, presenter and producer respectively on BBC's Scam Interceptors. If you haven't seen Scam Interceptors, it's a very entertaining factual series in which Nick and his team of ethical hackers attempt to disrupt scamming attacks on the public as they happen. In the show's fourth series, now airing daytime on BBC One, one of the scams disrupted involves a worryingly convincing Reese Witherspoon deepfake. So we thought we'd ask Nick and Mark to tell us all about their brush with (fake) celebrity, and share some pointers on how to spot a deepfake before it convinces you to empty your bank account. - Gwilym Ever wanted to have a deep and meaningful with your favourite Hollywood celebrity? Go on. Who is it? Pedro Pascal? Aubrey Plaza? Jeff Bridges (Nick). Beyoncé (Mark). Well, we've got great news for you. Thanks to the seemingly unbothered-by-scams social media giants and the absurdly rapid growth of free-to-use generative AI, you can. The only downside is that they will probably be a version of that celebrity being controlled by a scammer who wants to extort money from you. Many of you will have seen the story of the woman in France who was scammed out of almost a million euros by someone posing as Brad Pitt. The scammer used AI-generated deepfakes and details from real-life news reports about Pitt's divorce to trick the woman into thinking she was in a relationship with him. Unfortunately, the widespread availability of AI is an open goal for scammers who want to exploit social media users, many of whom are not au fait with the technology, and are simply trying to interact with their favourite celebs. You probably tittered at the Brad Pitt story. It's very easy to sit in judgment of those who have money stolen like this, but far more difficult to admit that the way AI is changing the online world might make us all vulnerable. In the latest series of our show, we decided to tackle this issue head-on. Donning our digital Donnie Brasco caps, we infiltrated the online groups these scammers were lurking in. What we found was, to continue the Donnie metaphor, a digital mafia. Scammers operating en masse and without hindrance, each one claiming to be an A-list celebrity. Mariah Carey, Jenna Ortega, Keanu Reeves: any A-lister you can think of is probably being criminally impersonated, in plain sight. Fuelled by a recent viewing of the smash 2001 romcom Legally Blonde, we signed up for the Reese Witherspoon fan club on Facebook. (We must admit, to our shame, we weren't already members.) Within minutes, we were inundated with messages from multiple accounts, all claiming to be the Real Reese Witherspoon, or Reese Witherspoon Private Account or similar. "Hello Sweetheart", one greeted us, with a kissy-face emoji. From there, our friendship developed, and the scammer, claiming to be Reese, pulled out multiple tricks to keep up the ruse. They sent us images of Witherspoon, Photoshopped to include her supposed driving licence, and shared details of their busy filming schedule. Over several weeks, they messaged us at all hours of the day, to the point where we grew to expect her name to pop up on the phone, just as you would your friend or colleague. We even got a little dopamine rush when it did - which is of course exactly what they want. It's all a part of their world-building, where they invest significant time and effort into making you believe that maybe, just maybe, this could be the real celebrity. And if you're willing enough to believe it, it can, and does work. To top off their charade, and to assuage any doubts, we received two videos. The person in these videos looked like Witherspoon, and sounded like her too. "Hello, I'm real. So if you don't believe me, I don't know what to tell you - this is me, have a good day," she says by way of proof. Of course, she doesn't though. It's a deepfake. The friendship led to an intro to Witherspoon's "manager", and, well, you'll have to watch the show to see how they tried to steal our money. All of this is unsettling. Not only because it leaves you with a sense of dread about our slow march towards a M3gan-style humanoid dystopia, but also because the videos sit in a weird, uncanny valley. They're not quite right. There are enough discrepancies to make them look a bit odd to the trained eye. And it's in these discrepancies that you can short-circuit the scammer's spell. Here are a few quick tips on how you can distinguish reality from AI-generated video fakery. This is probably about to become a necessary life-skill for all of us navigating the online world as is maintaining 47 unique passwords. Active engagement | Broadly speaking, try to engage actively with any vaguely suspicious content. Don't allow it to wash over you as you probably do with most of what you see online. The phrase "soft eyes" is how you can describe much of our engagement with the online world. Well, harden those peepers. If you want to be sure, observe keenly. Skin texture | For a video of a person, start by looking at the skin's texture. If it appears excessively smooth that's always a good indicator of deepfakery (AI still struggles to generate texture). Granted, that may not be helpful when working out whether you're looking at AI or just your average Hollywood surgical Ken or Barbie. Badly dubbed | Watch the mouth closely. Are the mouth shapes what you would expect from someone saying the words coming out? Blinking too - are they blinking too little and staring straight into your soul with shark-like dead eyes? Listen closely | The human voice can be a great read on whether video or audio has been faked. AI has a hard time with the ups and downs of our emotional range, so AI-generated voices tend to lack movement in tone. They will probably be flat, a recreation of the voice of the individual concerned, but only at one level. It is very important to add that all of the above is correct as of today, but as the technology evolves, it will get better at this. A lot better. Soon enough, being able to tell real from fake online is going to become one of our most vital skills, enabling us to avoid all kinds of emotional and mental manipulation. The creators of AI have no intention of stopping until they hit AGI (artificial general intelligence), essentially a human brain in computer form. It won't need prompts. Google's owner, Alphabet, has recently dropped its promise to not use AI for developing weapons and surveillance tools. Cool. While talking to ChatGPT about the ethics of AI the other day, in its usual chilled-out California surfer dude style, it described itself to us as a "hoodie-wearing superweapon". Deepfake Reese might be the thin end of the wedge. Scam Interceptors season 4 is on BBC One every weekday at 2pm. The full series is available on iPlayer now