ChatGPT's Advanced Image Generation Raises Concerns Over Fake Receipt Scams

ChatGPT can now generate fake receipts realistic enough to scam businesses

Facepalm: When ChatGPT isn't being used to generate those viral Studio Ghibli-inspired images flooding your feeds, it can be tapped to fake receipts instead. The AI chatbot is finally decent enough at comprehending and generating text within images that crafty users realized it could produce fake receipts. These could then easily pass as legit proof of purchase to an unsuspecting set of eyes. Until recently, AI-generated images had one glaring weakness - garbled or nonsensical text. Many models have managed to fix problems like messed-up hands to the point where they're not even an issue anymore. But text has remained a bit of an Achilles' heel. That's changing fast, though. For instance, following the latest update to ChatGPT's image generation on its 4o model, users can spit out receipts with near-perfect formatting, itemized lists, and even realistic-looking logos. Of course, they're still not completely flawless and can look overly crisp, have text with odd spacing, and even have math errors. However, receipts - especially for smaller amounts - aren't often scrutinized too heavily. A few extra tweaks in Photoshop, like adding stains, could easily fool an employer. The payoff is obvious to scammers. All they'd need to do is spend just a few minutes using ChatGPT and Photoshop, and even if the trick only works occasionally, it's still a low-effort, high-reward scheme. Right now, the most effective method might be to feed an existing receipt to ChatGPT and modify details like prices or dates. But as the tech improves, entirely fabricated receipts could become indistinguishable from real ones. You'd think that all this would ring alarm bells at OpenAI HQ, but while the company has acknowledged the risk, it isn't hitting the panic button just yet. A company spokesperson explained to TechRadar that it tracks image generation both on and off its platform, uses internal tools to verify AI-created content, and takes action when usage policy violations occur. Interestingly, OpenAI also sees an upside to this capability. Spokesperson Taya Christianson suggested to TechCrunch that fake receipts could be used for financial literacy education by teaching people how to track expenses or spot fraud in a controlled setting. Still, one has to wonder if this capability results in a net positive for society.

ChatGPT images are so good it's almost impossible to tell if they are fake, and that's got me worried

Last week, I wrote about how frustrated I was that ChatGPT wouldn't display the text I wanted in an image. Well, that appears to have been fixed now and I can create images full of text that looks utterly convincing. In fact, they are too convincing. ChatGPT is the first generally available, and free, AI image creator that can do convincing text. People have already noticed that ChatGPT's images are so good that you can use them to fake receipts. So now we're all going to have to be even less trusting of images we see online because it's going to be harder than ever to tell fake images from real ones. But surely, there must be something we can do to spot the AI fakes. ChatGPT's servers have been up and down like a yo-yo this week, and when it has been working, it's been very slow. This is because of the massive increase in users since ChatGPT announced its new image-generating capabilities. Posting on X, Sam Altman, CEO of OpenAI said "The ChatGPT launch 26 months ago was one of the craziest viral moments I'd ever seen, and we added one million users in five days. We added one million users in the last hour." Sam Altman described it as "biblical demand, I have never seen anything like it" in a further post on X. It wasn't long before everybody wanted to turn themselves and their pets into Studio Ghibli-inspired anime, causing ChatGPT's servers to go into meltdown. It's not the anime capabilities of ChatGPT that really intrigued me, though, it's the improved text rendering. Just like it's been bad at drawing fingers, AI has always been terrible at producing text, and illegible text was one of the ways you could spot an AI-generated image. Now that it can produce perfectly readable text, I'm not sure how to tell truth from fake anymore, which is why a little bit more digging is required. Each image on your computer or phone comes with metadata. This metadata can tell you where it was created and by what app, or for a photo, it tells you what camera it was taken with. Images generated in ChatGPT are no different, and their metadata tells you everything you need to know. To see this metadata on a Mac you download the image in question, then right-click on it in Finder and select Get Info. Images created using the ChatGPT website have "ChatGPT.com" in the "Where from:" part of their metadata. In Windows, go to File Explorer locate the image then right-click it and select Properties. Click the Details tab and scroll down to find the metadata that tells you what app it is from. Unfortunately, in all our tests we couldn't get Windows to display the part of the metadata that said the file was created in ChatGPT. On an iPhone, it says "Saved from ChatGPT" when you tap on the Information icon on an image in Photos. On Android, just like Windows, we couldn't see anything in the metadata that said the image was created by ChatGPT. But the annoying thing is, even if you can see it, you can't trust the metadata anyway. It's possible to edit this metadata. It's not easy to do, but if somebody is determined enough, they'll find a way. Not to mention that something as simple as uploading an image file to an online service, and then downloading the same file seems to strip out a lot of the metadata anyway. So, if you can't trust the metadata, then what else can you do? There are lots of online checkers that will tell you the likelihood of an image being AI-generated. Generally they work quite well, but I've found that the new ChatGPT images of text (like the receipt above) can be so realistic that it foxes them completely. For example, I uploaded our "TasteRadar," fake receipt image shown above to Sightengine, which I've always found reliable, and had got the results correct for every other AI image of people I'd uploaded to it. It said that it was most likely real, giving it just a 7% chance of being AI-generated. Now that leaves me worried. If the AI image checking websites can be fooled by the new ChatGPT images we're all in trouble. Right now I like to think I could still spot if an image is fake. There's a certain regularity to the text that ChatGPT produces - it's almost too perfect to have really been printed - but I've got the feeling that I'm kidding myself if I said I could spot it on every image I see. And of course it's only going to get better over time. If AI images have got so good we can't tell if they're real or not then we're going to have to question everything we see and hear from now on.