OpenAI's SearchGPT: Revolutionizing Web Search and Challenging Industry Giants

Faced with OpenAI's SearchGPT, Google must resolve its own AI dilemma first

Arguably the best business model ever invented, other than the illegal one of peddling drugs, is search. This is a $200 billion plus market today, with gross margins estimated at 60%, growing at 10% every year and projected at $371 billion by 2031. As we all know, one player dominates this lucrative market: Google, with a highly enviable 90%-plus market share. No one has been able to get anywhere close, with even mighty Microsoft's Bing a light year behind at No. 2 with a 3.7% share. Besides the acclaimed superiority of Google's search algorithms and its effective use of AI, another big reason is network effects and the fact that Google has now become a verb. What you need, you 'google.' No wonder that every tech company on Earth is trying to get a piece of the action, but has so far only managed to gather crumbs left behind by Google. Despite onslaughts by Microsoft and Yahoo, and pretenders like DuckDuckGo and Wolfram Alpha, Google has never really been under any threat. Until now, that is, with a new technology which threatens to upset its cosy world. The technology is Generative AI, and it is perhaps the first credible threat to Google's throne. This was apparent when ChatGPT was launched by OpenAI. Suddenly, we had another way of finding information and content out there, but a way that seemed more intuitive, warm and human than the cold 'ten blue links' of Google's search results. GenAI is built on language, much like we humans are, and its powerful Transformer algorithms (the 'T' in GPT) parse the vast troves of language on the internet to give us answers by probabilistically providing the next set of words to any word or prompt we give it. This is how we humans learn and think with language, but it is this very human quality of making up stuff as it goes along -- and being optimized for believability and not facts -- that make Large Language Models (LLMs) of GenAI not very conducive to the factual precision we expect of search. Thus, while GenAI has been out there for a couple of years, it has not been a credible danger to Google's search engine so far. This hasn't stopped people trying. Bing immediately incorporated OpenAI's GPT4 into its search results and renamed it Copilot, a startup called Perplexity has made waves with its robust competition (see: bit.ly/3Sr9fKy), and even Google incorporated its own LLM Gemini into its search results. However, the hallucinatory and probabilistic nature of GenAI often led to disastrous results, with Google's GenAI-based search confidently making Barack Obama African, for example. Last week saw the biggest onslaught on Google's monopoly yet, the launch of SearchGPT by OpenAI. OpenAI seems to have learnt from the mistakes of others. It is being careful, releasing a prototype for 10,000 users so as to learn and improve results before a wider launch. It has struck deals with content producers like Wall Street Journal, Vox Media and Associated Press, both to get quality content and send traffic back to publishers "by prominently citing and linking to them in searches." Publishers also have a way to "manage how they appear in OpenAI search features," as per OpenAI. This partnership-led approach by the world's hottest GenAI company is perhaps the most credible threat that Google has seen so far. My view is slightly different, though: the biggest threat to Google Search is Google itself. The first reason is what Clayton Christensen called the "innovator's dilemma." The Transformer was invented at Google's lab, but it was OpenAI that took it forward. Google saw GenAI and LLMs as being too reputationally dangerous, and a direct threat to its lucrative advertising-led search business model. The risk of launching something that would cannibalize its own business was too great, and so it demurred -- the classic innovator's dilemma. Secondly, Google's dominant position has made it complacent. The Google interface is a bad user experience, with advertiser links disguised as actual results dominating its first page, although users want the best possible result rather than advertised results. Google is optimized for the advertiser, not the consumer. The third reason is what usually happens to a wildly successful company with a monopoly grip: bloated structures slow down decision making, its leadership grows tentative and global regulators begin circling around and slowing down innovation. Thus, while SearchGPT and Perplexity will launch frontal attacks on its search empire, the biggest threat to Google is Google itself.

SearchGPT will force Google, and SEO, to innovate

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More OpenAI's entrance into the search space with SearchGPT was both a surprise and something expected. And as generative AI continues to upend the world of search, the expectation is that Google's decades-long hold on the space might change. But search and search engine optimization (SEO) experts said Google won't die. SearchGPT and the rival startup Perplexity, which offers its own AI search product, do represent a change for Google and for the search world in general: competition and the need to innovate. Scott Gabdullin, CEO and founder of SEO and marketing company Authority Factors told VentureBeat that generative AI began pushing Google to try new things like its controversial AI Overview feature and SearchGPT will force Google to try new things. "There's competition now, and it's always good for the industry when there is competition because that is where innovation takes place," Gabdullin said. "The monopoly on search is shifting." A natural near-monopoly is challenged For decades, Google has dominated search, with almost 82% of search traffic in 2024 so far. If companies want to have a presence in the search engine and be at the top of people's minds when looking for information, they must play by its rules. Companies began finding ways to rank higher on Google and reach the top of the results page. This inevitably led to the rise of sectors like SEO and the gamification of internet search, which many believe is why finding information on Google is so bad these days. Other AI-powered search engines, like Perplexity, have begun taking some traffic share from Google, though it hasn't made a very large dent yet. Nevertheless, Google began rolling out AI features to its flagship product. It released AI Overview in May but immediately began attracting controversy, and site owners believed it was taking traffic away from them. Perplexity itself got into hot water for allegedly ignoring opt-outs from websites. Gabdullin believes Google must stay ahead of the curve and continue to develop features that outperform competitors. A well-positioned insurgent However, experts believe it takes the larger power of OpenAI to really push Google's buttons. "OpenAI is very well-positioned currently. They have brand recognition, they're integrating with Apple iOS, and they offer functions like conversational search and visual answers," said Jim Yu, co-founder of SEO platform BrightEdge, in an interview with VentureBeat. He added that OpenAI has a "big opportunity to disrupt not only how people search, but also how brands and companies market their products online. SEO experts said that it seems OpenAI is learning from the missteps of Google and Perplexity in launching SearchGPT. Yu said SearchGPT and other AI-powered search platforms also function differently from Google by encouraging follow-up questions, a feature he said Google removed from AI Overview. Additional questions help narrow down information for users. Gabdullin pointed out that, unlike Google's AI Overview, OpenAI wants to let publishers know it's directing search traffic to their websites. It's also actively seeking partnerships with publishers to allow them to use their web crawlers and index these sites. To make things more difficult for Google, OpenAI is making deals with large companies like Apple to bring its products to a sizeable captive phone market. "If SearchGPT starts to complete any Siri request, imagine the number of new potential users who will be introduced to search results without ads," said Christina Ward, CEO of Yext. Despite this, Google won't give up search market share that easily. "Going up against a behemoth like Google is no easy feat," Yu said. "A newer AI-first engine like Perplexity likely has more to lose at this juncture, but this could also be seen as the first step to SearchGPT becoming a meaningful threat to Google." SEO gets better Google's search platform is not the only one that needs to innovate. Gabdullin believes SearchGPT will make brands diversify how they reach customers, and that includes not relying on SEO too much. "SEOs are now realizing that it's not enough just to be an SEO; you have to be a marketer," Gabdullin said. Figuring out the best SEO keywords to dominate Google search results became a bit of a game, but Gabdullin said AI-based platforms like SearchGPT or Perplexity index data from websites and do calculations fast. This lets SearchGPT focus on new factors like a site's quality and rank that higher. "Gaming starts to become more difficult now that the computation is so much more powerful," he said.

The AI Search War Has Begun

Every second of every day, people across the world type tens of thousands of queries into Google, adding up to trillions of searches a year. Google and a few other search engines are the portal through which several billion people navigate the internet. Many of the world's most powerful tech companies, including Google, Microsoft, and OpenAI, have recently spotted an opportunity to remake that gateway with generative AI, and they are racing to seize it. And as of this week, the generative-AI search wars are in full swing. The value of an AI-powered search bar is straightforward: Instead of having to open and read multiple links, wouldn't it be better to type your query into a chatbot and receive an immediate, comprehensive answer? In order for this approach to work, though, AI models have to be able to scrape the web for relevant information. Nearly two years after the arrival of ChatGPT, and with users growing aware that many generative-AI products have effectively been built on stolen information, tech companies are trying to play nice with the media outlets that supply the content these machines need. This morning, the start-up Perplexity, which offers an AI-powered "answer engine," announced revenue-sharing deals with Time, Fortune, and several other publishers. Moving forward, these publishers will be compensated when Perplexity earns ad revenue from AI-generated answers that cite partner content. The site does not currently run ads, but will begin doing so in the form of sponsored "related follow-up questions" this fall -- a sportswear brand could pay for a follow-up question to appear in response to a query about Babe Ruth, and if the AI used Time in its answer, then Time would get a cut of the ad revenue for every citation. OpenAI has been building its own roster of media partners, including News Corp, Vox Media, and The Atlantic, and last week announced its own AI-search prototype, SearchGPT. (The editorial division of The Atlantic operates independently from the business division, which announced its corporate partnership with OpenAI in May.) Google has purchased the rights to use Reddit content to train future AI models, and currently appears to be the only major search engine that Reddit is permitting to surface its content. The default was once that you would directly consume work by another person; now an AI may chew and regurgitate it first, then determine what you see based on its opaque underlying algorithm. This also means that many of the human readers whom media outlets currently show ads and sell subscriptions to will have less reason to ever visit publishers' websites. Read: A devil's bargain with OpenAI Tech companies have made deals with journalistic outlets in the past, paying publishers to use products such as Facebook Live and Snapchat Discover, but these AI searchbots are different. Facebook and Snapchat are social products at their core; you log on to them to see what other people are posting, and for many users, news content may be incidental. Perplexity and SearchGPT, by contrast, need high-quality, timely content to answer questions accurately. Generative-AI models have no internal information beyond their training data, which tend to be months or years old. Without more recent stories, these products would be limited, unable to deliver relevant information about H5N1, the attempted assassination of Donald Trump, the Olympics, and so on. OpenAI's most advanced model, for instance, was released in May but has no knowledge of events after October 2023. When I first spoke with Dmitry Shevelenko, Perplexity's chief business officer, in June, he told me, "One of the key ingredients for our long-term success is that we need web publishers to keep creating great journalism that is loaded up with facts, because you can't answer questions well if you don't have accurate source material." Read: OopsGPT Of course, existing AI products are absolutely filled with media that publishers have received no compensation for. (Shevelenko told me that Perplexity will not stop citing publishers outside its revenue-sharing deal, nor will it show any preference for its paid partners moving forward.) AI companies don't seem to value human words, human photos, and human videos as works of craft or products of labor; instead they treat the content as strip mines of information. "People don't come to Perplexity to consume journalism; they come to Perplexity to consume facts," Shevelenko told me in an interview before today's announcement. "Journalists' content is rich in facts, verified knowledge, and that is the utility function it plays to an AI answer engine." To Shevelenko, that means Perplexity and journalists are not in direct competition -- the former answers questions; the latter breaks news or provides compelling prose and ideas. But even he conceded that AI search will send less traffic to media websites than traditional search engines have, because users have less reason to click on any links -- the bot is providing the answer. The growing number of AI-media deals, then, are a shakedown. Sure, Shevelenko told me that Perplexity thinks revenue-sharing is the right thing to do. But AI is scraping publishers' content whether they want it to or not: Media companies can be chumps or get paid. Still, the nature of these deals also suggests that publishers may have more power than it seems. Perplexity and OpenAI, for instance, are offering fairly different incentives to media partners -- meaning the tech start-ups are themselves competing to win over publishers. All of these products have made basic mistakes, such as incorrectly citing sources and fabricating information. Having a searchbot ground itself in human-made "verified knowledge" might help alleviate these issues, especially for recent events the AI model wasn't trained on. Publishers also have at least some ability to limit AI search engines' ability to read their websites. They can also refuse to sign or renegotiate deals, or even sue AI companies for copyright infringement, as The New York Times has done. AI firms seem to have their own ways around media companies' barricades, but that is an ongoing arms race without a clear winner. Read: AI can't make music Publishers may now have sway over AI companies that need high-quality, human-made content to either answer user queries or train future AI models, like a GPT-5 or GPT-6. Nicholas Thompson, the CEO of The Atlantic, said in an interview with the tech journalist Nilay Patel that The Atlantic's contract with OpenAI will expire after two years, and is designed to create "more leverage when there's another moment of negotiation." Reddit has recently cut off search engines other than Google from crawling its site; if DuckDuckGo, Perplexity, or Bing want to show users new posts from Reddit, they will have to "make enforceable promises regarding their use of Reddit content, including their use for AI," a Reddit spokesperson told The Verge. (Of course, Reddit has a hard-core user base and isn't a traditional news organization -- media companies are constantly vying for attention and may be less comfortable with closing off potential audiences.) In other words, whether OpenAI, Perplexity, Google, or someone else wins the AI search war might not depend entirely on their software: Media partners are also an important part of the equation. This could possibly shift. Shevelenko told me he believes that Perplexity's use of publishers' content is legal under copyright law, and if he's proved right by a judge's ruling, then AI companies may no longer see an incentive to pay publishers. For now, that decision is up in the air, and publishers are taking advantage of a small window of opportunity. Perplexity, for its part, has been accused of plagiarizing content from publishers including Forbes and Condé Nast, which could dissuade other publishers from partnering with the start-up; Shevelenko has told Semafor that Perplexity had to persuade its initial slate of partners to overlook these allegations. The company was supposed to announce its revenue-sharing program roughly when Shevelenko spoke with me in June, but delayed the formal launch amid a wave of criticism. Now, he said, "the ball is in our court to show publishers that we are a good-faith actor taking the right, long-term moves." The search war is an attempt to change how people navigate the internet, the system through which the contemporary world organizes and disseminates knowledge. But the underlying terrain has not changed: Knowledge, no matter its organization, remains the sum of writing, art, and thinking from humanity, not from a bot.