AI chatbots keep pointing shoppers at fake stores, a study finds

The chatbots people are increasingly told to shop with sometimes point them at fake stores. Podqi, an intellectual-property protection firm that studies brand impersonation online, ran a test in which ChatGPT, Gemini and Perplexity all cited or mentioned risky websites when asked shopping questions 1

1

2

2

.

Podqi prompted the three assistants with shopping questions about 151 brands across two testing periods in September, producing 6,795 answers 1

1

2

2

. In this test, Perplexity showed risky sites in 15% of its answers, ChatGPT in 8% and Gemini in 4% 1

1

2

2

. A "risky" site could be a fake or impersonating page, an old brand domain someone re-registered, or a dead domain anyone could buy and reuse 1

1

.

Who gets hit hardest

The three were not equally risky. Questions the assistants got about smaller brands produced risky-site references about half as often as questions about larger companies, so smaller brands looked more exposed 1

1

2

2

. Within the test, prompts about regional brand websites turned up risky results more often still, and the chatbots sometimes made up regional sites for brands with none 1

1

.

The errors could come with confident endorsements. In one test, Perplexity pointed to the wrong page as Stüssy's official site and called it the best place to start for guaranteed authentic items 1

1

.

Warnings from the assistants were rare. ChatGPT warned users about a risky site 24% of the time when one appeared, the best of the three on warnings 1

1

2

2

.

The companies respond

Google leaned on its best-in-test result. "We take information quality seriously, and are happy to see that Gemini performed best among the products tested," said spokesperson Jessica Levis Boord 1

1

. OpenAI did not dispute the findings. Spokesperson Taya Christianson said ChatGPT's goal is connecting people with reliable places to shop, and that it is improving safeguards to make deceptive sites less likely in shopping results 1

1

2

2

.

Perplexity, the worst performer in the test, pushed back through spokesperson Jesse Dwyer: "Perplexity is consistently the most accurate AI on credible research benchmarks, and has been for four years. We never claim to be 100% accurate, but we do claim to be the only company relentlessly focused on it." 1

1

The study has caveats. Podqi wrote its own prompts rather than using real shopper conversations, and its tested brands included Madhappy, a Podqi client, plus Lululemon, Birkenstock and Le Creuset 1

1

. Per The Verge's reporting, the AI companies themselves market their chatbots for shopping, as tools for researching, comparing and finding products 1

1

. The findings land ahead of the holiday shopping season, when more people are expected to use AI to shop, including at brands and sites they have never heard of 1

1

.

Until the assistants get better at flagging danger, the check falls to the shopper. "Part of this piece, too, is making sure that the consumers know, even though you are being fed a result in a particular chatbot, it is always best to verify the source of that before entering personal information or credit card information," said Trevor West, Podqi's CEO and cofounder 1

1

.