Nvidia's AI Dominance Drives Record Profits and Market Valuation

Transcript: The trouble with deepfakes -- liar's dividend

Hannah Murphy Before we begin, we'd love to hear a bit more about you and what you like about this show. We're running a short survey, and anyone who takes part before August 29th will be entered into a prize draw for a pair of Bose QuietComfort 35 wireless headphones. You can find a link to the survey and terms and conditions for the prize draw in our show notes. If you want an example of the trouble caused by deepfakes -- images, video or audio generated by AI -- take this story of a high school in the suburbs of Baltimore, Maryland, earlier this year. (Bell ringing) It was mid-January when three staff members at this US high school found an anonymous email in their inboxes. It had an attachment, which the email labelled a "disturbing recording". They clicked on the attachment. It was a muffled clip of what was purported to be the school's principal. It sounded like he was unaware that he was being recorded. Audio clip You know, I seriously don't understand why I have to constantly put up with these dumb asses here every day. Hannah Murphy In it, the principal appeared to go on a tirade spouting racist vitriol against the students and the teachers of the school. Audio clip I'm just so sick of the inadequacies of these people. Hannah Murphy I'll spare you the full recording. Suffice to say, it's very offensive. The staff received the email, sent it on to other people. It ended up getting picked up by local news outlets and posted on Instagram, where it was shared over 27,000 times. Angry parents kept phoning the school. An investigation was launched. The principal was suspended and started receiving threats to his safety. And then, three months after the audio first surfaced, the police made an arrest. But it wasn't the principal who was arrested. It was the school's athletics director. Why? Well, the police say this isn't a recording of the principal at all. In fact, it's not a recording of anyone. They say that the athletics director had used AI to deepfake the principal in order to get him in trouble. News clip 1 We have now conclusive evidence that the recording was not authentic. Based off of those findings and further investigation, it's been determined that the recording was generated through the use of artificial intelligence technology. Hannah Murphy This was one of the first times anyone had been arrested in connection with a deepfake. But it's just one example of how AI-generated fakery is causing havoc. There have been incidents of teenagers making deepfake porn of their classmates. Fraudsters cloning people's voices to scam them out of money. Celebrities being shown promoting products they've never actually promoted, and politicians framed saying things they never actually said. But in the case of the Baltimore school principal, there was a further twist. At the time, the police said they had conclusive evidence that the clip was AI-generated. Plus, they said they had other evidence linking the athletics director to the creation of the clip. Like his internet search history showing him looking up AI tools and the fact that the email address the audio came from in the first place was linked to his phone number. But a few weeks later, I went to an event put on by a security company called Pindrop. Now, Pindrop is one of the leading developers of software that could detect audio deepfakes. And during that event, Pindrop brought up the Baltimore principal clip and played it to us. And they said that according to their software, while the clip had been doctored, perhaps edited a bit, it was not all generated by AI. While they caveated that their technology was not foolproof, they published these findings online, claiming that they were 97 per cent sure the core recording was not a deepfake. Yet a lawyer representing the principal disagreed. He told us that multiple governmental agencies and independent experts had concluded that the audio was AI-generated, clearly manufactured and manipulated. He also said the principal sat for a polygraph test, which confirmed the conclusion. All of this gave me pause for thought, because if the state authorities are confident this audio is a deepfake, but some of the leading technology for detecting deepfakes says otherwise, then where does that leave the rest of us? Even the experts clash. And now that pretty much anyone can create a convincing deepfake that blurs the line between what's real and what isn't, we're beginning to see the real-world consequences. [MUSIC PLAYING] Welcome to Tech Tonic, the technology podcast from the Financial Times. I'm Hannah Murphy and I'm a tech reporter for the FT, covering social media out of San Francisco. Over the next two episodes, I'll be exploring the world of deepfakes, the new breed of AI-generated fake pictures, videos and audio clips spreading across the internet. How did they become so diffused and sophisticated and so easy to make? And is there any way to stop them? Or will this wave of AI-powered fakery reshape this world forever? [MUSIC PLAYING] It's estimated that every day on social media, we share about 14bn images. And these days more than 80 per cent of internet traffic is video streaming. So if you were scrolling on social media and came across a video, an image or an audio clip, do you think you'd be able to tell if it was real or fake? Audio clip Hello, my name is Hannah and I am a tech reporter at the Financial Times based out of San Francisco. Actually that's false. I am not real. This is a voice clone imitating Hannah. Hannah Murphy I spoke to Kimberly Ton Mai. She's a PhD researcher at University College London, and she's been looking into how easily people can identify deepfakes. There are AI tools all over the internet that can help you make deepfake images, video or audio. And for this, Ton Mai has made some fake audio clips of me using a popular AI voice generator called ElevenLabs. Kimberly Ton Mai What generative AI technology does is it basically looks at thousands of examples of what a person looks like or a person sounds like and learns its patterns and characteristics without any explicit programming. Hannah Murphy Ton Mai says AI tools like ElevenLabs are powerful because they're pre-trained on loads of data. Kimberly Ton Mai So a pre-trained model is trained on thousands and thousands of audio samples, and from that learning process, it learns the patterns and characteristics of what English sounds like. So by feeding in a sample of your voice, I told the model to tweak it so it sounds a bit more like you. But because it had been pre-trained on loads of speakers already, it didn't need to do the hard task of learning how English sounds like. It just needs to learn a bit more about what you sound like. Hannah Murphy And so you then type in whatever you want and it will spit that out in my voice. Kimberly Ton Mai Yeah. That's correct. Hannah Murphy So to test me, Ton Mai took a few minutes of audio, actually, from an interview I did with Tech Tonic last year about Twitter or X and Elon Musk and fed it into ElevenLabs. So my task is to tell if it's a real clip of me from that interview or a fake. Kimberly Ton Mai So here's number one. Voice clip Twitter had always had this reputation as being a sort of slow-moving, not very innovative company that really wasn't too interested in making a lot of money. Kimberly Ton Mai What did you think of that one? Hannah Murphy Do you know what? As it went through at different points, I thought, Oh that's real. Oh that's fake. Oh that's real. I'm gonna say real. Kimberly Ton Mai Yeah. So that was fake, actually. Hannah Murphy Well, OK. You know, I thought because of the breathing that it was real, because I was like, oh, that's exactly how and when I would breathe. That's so interesting. OK, so zero point so far. Kimberly Ton Mai Maybe once you've heard the first one, maybe you'll be better at the second one. Audio clip And this particular Elon appears to some to have bought the platform on a whim, sort of frustrated by the amount of spam that he faces. Frustrated by its moderation policies as he's this sort of self-declared free speech absolutist. Hannah Murphy The intonation. And now thinking back, the first one is sort of flatter than I would probably speak on a live podcast. [MUSIC PLAYING] I admit I was pretty bad at spotting deepfaked audio, even when it was my own voice, but according to Ton Mai's research, I'm not alone. She found that people were only able to spot deepfake audio about 73 per cent of the time, and that stack comes with a couple of cautions. Firstly, the research was done in lab conditions so people knew that they were listening out for fakes. And secondly, the algorithms used in the testing environment are now a couple of years old, and generative AI has come along leaps and bounds since then. And I know that for me, being fooled by my own deepfake self in a controlled environment made me worried about how easily I'd be able to spot one in the wild. Being able to discern between real and fake may seem like a new problem, but disinformation has been part of society for millennia and has been something that I've had to contend with as long as I have been a journalist. The good news: there are talented people currently developing methods to try to track down and flag deepfaked material. Hany Farid So digital forensics is that field of analysing digital content for evidence of manipulation, and then for things like provenance, like who took this photo? When was it taken, where was it taken? So think of all your favourite crime scene TV shows, but pixels instead of blood and hair and rug samples. Hannah Murphy That's Hany Farid. He's a professor at the University of Berkeley, California, and he's widely recognised as an authority on deepfakes. He says image manipulation has been around pretty much since cameras were invented. Hany Farid So for this, we have to go back to 1800s. You have to go back to the inception of photography. No kidding. Hannah Murphy But back then, making a doctored photograph required a pretty specialist skill set and equipment. Hany Farid First of all, you had to have a camera. You had to create the negative. You had to go into a dark room and paint over that negative in order to alter it. So Stalin famously did this by airbrushing people out of history, but he was not alone. Hannah Murphy In fact, at the height of his power in the USSR, Joseph Stalin had a team of retoucher working on falsifying and manipulating images for propaganda. In one photo, an image of Stalin has been pasted into a crowd, and the crowd itself is made up of repeating fragments of the same photo, easy enough to spot for the trained eye. Hany Farid And so throughout history, we have evidence of photographic manipulation. But for the most part, the historical record of photography was pretty reliable because relatively few people could manipulate that content. And then we saw the rise of the digital revolution. Hannah Murphy Basically, digital cameras and Photoshop changed the game. Hany Farid We've begun to democratise two things. One is the ability to capture and store photographs, and then the ability to start to manipulate them. But still, you needed some expertise. Hannah Murphy But soon you didn't need much expertise at all to take and change photos. Along came phones with built-in cameras and smartphones. Images could be taken and shared instantly on Tumblr, Facebook, WhatsApp. Filters could be added, backgrounds altered, and then generative AI arrived which has changed the game completely. Hany Farid Now you don't need a camera and you don't need any skill. You go to a web service and you type in a sentence and it gives you that image. Or you give it a video and audio and it gets and we just say something they never did. Hannah Murphy And while there were 170 years between the first camera and the digital camera, AI technology is improving exponentially at a far faster clip, which means that deepfakes are only going to get more and more convincing. Hany Farid This is what you know about the trends is that every few months, the technology gets significantly better, significantly more affordable, and more people start using and misusing it. Hannah Murphy And for Farid, what is particular about this current moment we are in, is the coming together of generative AI and social media -- a full democratisation of both making disinformation and distributing it. Hany Farid So not only can I create disinformation, election interference, nonconsensual sexual imagery, frauds and scams, but I can distribute them to the internet instantaneously through social media. And I think that's a very different threat vector than Stalin in the dark room manipulating negatives. [MUSIC PLAYING] Hannah Murphy So what sorts of things could happen when literally anyone can falsify the political record and make it go viral? And you mentioned earlier, you mentioned robocalls. We're gonna be interviewing Paul Carpenter, who was, I think, the magician who helped sort of fashion the Biden robo . . . Hany Farid You can't make this story up. Even if you try to make this story up, you can't. Hannah Murphy You cannot. More on the story of the AI-generated voice of President Biden and the havoc it caused after the break. [MUSIC PLAYING] What happens when almost anyone can create a deepfake good enough to fool everyone? Paul Carpenter My name is Paul David Carpenter. For most of my life, I've been a magician and a hypnotist. I also have been heavily involved in the tech world for the past five, six years, dealing with AI and dealing with a lot of the forerunners in the AI field. Hannah Murphy Paul Carpenter is 45. At the moment he lives in Texas, but he spent his life travelling around and doing different kinds of shows, performances and odd jobs. Paul Carpenter voice clip This is me to the magic while having coffee at six in the morning. Hannah Murphy He posts videos of himself doing magic tricks on his TikTok. Paul Carpenter voice clip You ready? OK, here we go. Watch it. Hannah Murphy And he is doing a fork-bending trick to some people in the street. Paul Carpenter Around 13, 14, 15, I was already hypnotising my friends and trying out different techniques. Hannah Murphy He even says he broke a couple of world records back in his late 20s. Paul Carpenter So one of them was the fastest straitjacket escape. I believe it's 10.59 seconds. It was done at Coney Island, in front of an audience of 200 people. And then my other was bending 10 forks with six bends in each fork in under a minute. So basically 60 bends and under, you know, 60 seconds. Hannah Murphy But after a while, Carpenter started to get interested in other things besides magic and world record attempts. He got into cryptocurrencies and then it became interested in AI and started playing around with deepfake-generating tools online, and that's what led to him getting mixed up in what is possibly one of the most prominent political deepfake cases to date. And it's pretty bizarre. So here's the story. Towards the end of last year, Carpenter was in New Orleans, staying with a friend when an intriguing character turned up. Paul Carpenter A friend of his, named Steve Kramer, came into town in this RV, and he didn't have a place to stay at the time, so he was staying with us. Hannah Murphy Steve Kramer appears to be a political consultant. He works on campaigns. It's not clear why he's in an RV in New Orleans. But anyway, he and Carpenter get talking. It was the end of last year, so Joe Biden and his Democratic rivals were gearing up for the US primaries. Kramer says he's working for politicians and has a job with Biden's team, and Carpenter tells Kramer how he's been getting into AI and deepfakes. And eventually, Kramer says he wants to make a deepfake voice of Joe Biden, so he asks Carpenter to sort it out for him. Paul Carpenter He tells me that he would like to do something with Biden, and I go and I start creating the content for the voice, doing my best to make it sound believable. At which point he then sent me a script. I, of course, placed into ElevenLabs. Pushed the button, push it out, you know, and send it to him. Hannah Murphy After that, Carpenter didn't think much of it. Until a few days later, he realises the voice he's made has gone out to thousands of people across New Hampshire. Audio clip Who know the value of voting Democratic, and our votes count. It's important that you save your vote for the November election. Hannah Murphy In the calls, Biden's voice urged voters to skip the New Hampshire primaries, falsely suggesting that if they voted then, they wouldn't be able to vote for the presidential election in November. Audio clip Your vote makes a difference in November, not just Tuesday. Hannah Murphy It caused a huge scandal, and Carpenter says he had no idea this was how the deepfake Biden voice was going to be used. And were you sent sort of various scripts, or was it just the one sort of warning, don't go and vote. And if so, did that raise any alarm bells? Paul Carpenter Yeah. So again on my end, I didn't look at it that way. I look at it like gig work because I do so much of it. And so it's just another pile of folders in front of me and in my mind, you know. Gig work that I know is gonna disappear in the next six or seven months, anyways. But . . . Hannah Murphy Right, you plug it in, you push it out, you move on. Paul Carpenter Right. Yeah, yeah. Of course, you know, I mean, do you sit there and check every word? No. Because you're expecting the AI to do its job and that's what you're paying for. And you walk away. Hannah Murphy How much were you paid for this work? Paul Carpenter 150 bucks. You know, as I said in another article, I made the gun. I didn't shoot it, and I have no issue saying that whatsoever. I'm a factory worker over here, doing some gig work. Hannah Murphy Steve Kramer ended up being indicted and fined $6mn for the robocalls. The telecoms company that transmitted the call was also fined. Paul Carpenter was not. This is just one example over the past year of dozens, where politicians have found that their voice or likeness has been deepfaked and set loose online. Audio clip (Speaker talking in foreign language) Hannah Murphy Just a day before the general election in Pakistan this February, an AI-generated voice clip of former President Imran Khan was circulated calling for people to boycott the polls. News clip 2 The audio translates a bit to "Dear Pakistanis, given the unjustified crackdown on our party. We have lost all hopes of justice and decided to boycott this election". Hannah Murphy In Ukraine in 2022, a deepfake video of President Zelenskyy calling on his forces to surrender to Russia circulated on social media. Audio clip (Speaker talking in foreign language) Hannah Murphy And in the UK last year, a deepfake audio clip of the now prime minister, Sir Keir Starmer, captured him allegedly abusing his staff. Audio clip I literally told you, didn't I (blip) say it? Hannah Murphy Already, in May, OpenAI, the company behind ChatGPT and Dall-E, published a report saying that AI tools are being used by the likes of Russia, China, Iran and Israel to create and spread disinformation in the form of texts and images. Political deepfakes can chip away at our trust, slowly eroding democracy. For me, though, the doomsday scenario, I think, is a convincing deepfake going out perhaps during a blackout period in the run-up to an election when political campaigning might be banned. Safe detection tools don't line on whether it's real or not, and well-followed or well-respected figures share it. This could cause chaos or even up-end a vote. Listen, it's not yet clear this would actually happen. That one fake video will undermine an entire election. But either way, in that current, unchecked form, deepfakes raise the spectre that anything we're seeing, watching, hearing could be fake. And when anything could be fake, then anything is deniable. Back to Hany Farid. Hany Farid I can rewind you back to 2016, when Donald Trump got caught on the Access Hollywood tape saying what he does to women, which because he could do because he was famous. And when that Access Hollywood tape leaked, he apologised. By the way, two years later, deepfakes are on the horizon now. And he was asked about the tape and he said, oh, it's fake. Why did you apologise in the first place? Doesn't matter. Hannah Murphy This consequence of a post-truth world or anything is deniable. It's called the liar's dividend. Hany Farid So the liar's dividend goes something like this. It says that when you enter a world where anything can be fake, any image, any audio, any video, anything you read, nothing has to be real. And that is, you can simply raise the spectre of "This is AI-generated. This is fake" in order to deny reality. Hannah Murphy This is already happening. Last year, a supposedly leaked audio of Mexico City's mayor, Martí Batres, captured him secretly planning to undermine one of his possible successors in favour of the other. Audio clip (Speaker talking in foreign language) Hannah Murphy Martí Batres spoke out, claiming the clip was AI-generated and others who analysed the audio said the same. Yet there are others still who are not convinced, who think the recording was real. And at this point we have no way of knowing for sure either way. Hany Farid So I'm worried about all the threats around deepfakes and generative AI that they're generally concerning, but they are nothing compared to this post-truth world that we seem to be entering where anybody can deny reality. And when we enter a post-truth world, this is how authoritarianism rises, right? Because what you've eroded trust in governments, in media and scientific experts. And when you do that, what's left? You believe and trust your authoritarian rulers. And that, I think, is exceedingly dangerous for us as a society in a democracy. Hannah Murphy So if this is the possible outcome of a deepfake technology that is unregulated, uncontrolled and available to anyone, that it becomes this new, highly sophisticated layer of potential fakery on top of an information landscape that's already filled with disinformation, then what is being done to get a handle on it? Hany Farid There's the path that we seem to have been on for the last 20 years, which doesn't end well in some bizarre dystopian nonsense that is, you know, the worst version of humanity. And frankly, I think that's the path we're on right now. There is a better version of this. There is a better version of the future where we harness the power not just of generative AI and AI, but technology as a whole. And we use it for good and we mitigate the harms. I don't know which way this goes. Hannah Murphy So what are the guardrails that could protect us against the deepfaked media being circulated online? Hany Farid We have to find a balance between a free and open society, where we get to criticise and make fun of and ridicule our politicians. I think that's fine. I think it's fine to make a video of Joe Biden or Donald Trump that ridicules them. I think that should be protected. But I think there's also a line where you've gone too far. You've been listening to Tech Tonic from the Financial Times with me, Hannah Murphy. I've put some free links related to the episode in the show notes. Do check them out and do leave us a review. It helps spread the word. This series of Tech Tonic is produced by Persis Love. Edwin Lane is the senior producer. Manuela Saragosa is the executive producer. Sound design by Breen Turner and Samantha Giovinco. Original music by Metaphor Music. The FT's head of audio is Cheryl Brumley.

The trouble with deepfakes: Liar's dividend

Your browser does not support playing this file but you can still download the MP3 file to play locally. A new breed of AI generated fake pictures, videos and audio clips is spreading across the internet - content anyone with an internet connection can generate. And some of these deepfakes are now so convincing that even experts struggle to tell the difference between what's real and what has been created using artificial intelligence. In a new series, Hannah Murphy, the FT's tech reporter in San Francisco, examines the potential of deepfakes to cause chaos and asks how worried we should be and what's being done to combat their proliferation. In the first of this two-part series she hears from Kimberly Ton Mai, AI researcher at University College, London; Hany Farid, digital forensics expert at the University of California, Berkeley; and Paul Carpenter, magician and hypnotist. Tell us what you think of Tech Tonic and you could be in with a chance to win a pair of Bose QuietComfort 35 wireless headphones. Complete the survey here. Want more? Audio deepfakes emerge as weapon of choice in election disinformation The rising threat to democracy of AI-powered disinformation Clips: Fox News, AP, @mentallyhyp TikTok, The Telegraph, The Guardian, France 24 English, Sky News This series of Tech Tonic is presented by Hannah Murphy. The producer is Persis Love. The senior producer is Edwin Lane. Executive producer Manuela Saragosa. Additional production help from Josh Gabert-Doyon. Sound design by Breen Turner and Samantha Giovinco. Original music by Metaphor Music. The FT's head of audio is Cheryl Brumley.