OpenAI's O3 Model: Impressive Performance at a Steep Cost

OpenAI's o3 model might be costlier to run than originally estimated | TechCrunch

When OpenAI unveiled its o3 "reasoning" AI model in December, the company partnered with the creators of ARC-AGI, a benchmark designed to test highly capable AI, to showcase o3's capabilities. Months later, the results have been revised, and they now look slightly less impressive than they did initially. Last week, the Arc Prize Foundation, which maintains and administers ARC-AGI, updated its approximate computing costs for o3. The organization originally estimated that the best-performing configuration of o3 it tested, o3 high, cost around $3,000 to solve a single ARC-AGI problem. Now, the Arc Prize Foundation thinks that the cost is much higher -- possibly around $30,000 per task. The revision is notable because it illustrates just how expensive today's most sophisticated AI models may end up being for certain tasks, at least early on. OpenAI has yet to price o3 -- or release it, even. But the Arc Prize Foundation believes OpenAI's o1-pro model pricing is a reasonable proxy. For context, o1-pro is OpenAI's most expensive model to date. "We believe o1-pro is a closer comparison of true o3 cost [...] due to amount of test-time compute used," Mike Knoop, one of the co-founders of The Arc Prize Foundation, told TechCrunch. "But this is still a proxy, and we've kept o3 labeled as preview on our leaderboard to reflect the uncertainty until official pricing is announced." A high price for o3 high wouldn't be out of the question, given the amount of computing resources the model reportedly uses. According to the Arc Prize Foundation, o3 high used 172x more computing than o3 low, the lowest-computing configuration of o3, to tackle ARC-AGI. Moreover, rumors have been flying for quite some time about pricey plans OpenAI is considering introducing for enterprise customers. In early March, The Information reported that the company may be planning to charge up to $20,000 per month for specialized AI "agents," like a software developer agent. Some might argue that even OpenAI's priciest models will cost well under what a typical human contractor or staffer would command. But as AI researcher Toby Ord pointed out in a post on X, the models may not be as efficient. For example, o3 high needed 1,024 attempts at each task in ARC-AGI to achieve its best score.

OpenAI's smartest model could cost up to $30,000 per task according to estimates

OpenAI's o3 was put to the test and while it passed, it did so at a huge cost Back in December, OpenAI launched a reasoning model known as 'o3'. Smarter and more thorough in its decision process, this showed OpenAI's advancement into a more deliberate version of AI. But there's just one problem. When the model was first launched, OpenAI teamed up with the Arc Prize Foundation. This group runs the ARC-AGI, a benchmark exam to test highly capable AI models. These tests are incredibly easy for a human to solve with simple puzzles the majority of the public could easily do. But that's not the case for AI. The foundation originally believed it cost around $3,000 for o3 to solve a complex problem. That's no longer the case, with re-estimates putting it nearer to $30,000 per task. So, while OpenAI scored highly on the test, it did so at a higher-than-average cost. In fact, it's an astronomical cost that shows just how expensive cutting-edge AI models are to run. OpenAI's o3 is separate from the models that power ChatGPT. Specifically, o3 is reflective. It is designed for complex problem-solving and reasoning and therefore is used for tasks that require a higher level of computing power. When developers use this model, they can select from three types of reasoning efforts: low, medium, and high. This is essentially asking the model to think harder. Increase the reasoning effort and you'll get a better result, but that comes with an increase in computing power and cost. In the ARC-AGI test, it was o3 (high) that came with such a massive price tag. In other words, this is OpenAI's most detailed model, being pushed to its absolute limits of computing power. Arguably, in a test like this, the model overdid it. Toby Ord, a senior researcher at Oxford University highlighted on X that it took o3 (high) 1,024 attempts at each task, writing roughly 137 pages of text for each attempt, or around 43 million words total. According to the Arc Prize Foundation, o3 (high) used 172 times more computing power than o3 (low). While the low-effort version didn't score as highly, it could end up being a more cost-effective version. For now, OpenAI hasn't made any announcements about how much o3 will cost for developers to use. A more cost-efficient version known as o3-mini is available. This, while being cheaper, is nowhere near as powerful. For the average person, this pricing will have no effect. ChatGPT is one of the more cost-effective models for OpenAI and o3, when made available, will only be used by those at the cutting edge of AI. However, as the likes of Deepseek and Claude start to cut costs and develop more affordable complex models, OpenAI could quickly find itself as the most expensive provider of the bunch.

OpenAI's o3 costs way, way more than you think

OpenAI's o3 "reasoning" AI model, showcased in December with the ARC-AGI creators, now has revised and less impressive results, with updated computing costs suggesting it's far pricier than initially thought. The Arc Prize Foundation, which maintains ARC-AGI, initially estimated the best-performing o3 configuration, o3 high, cost about $3,000 to solve a single ARC-AGI problem. The updated estimates now suggest a cost closer to $30,000 per task. This revision highlights the potentially high costs of sophisticated AI models for certain tasks. OpenAI hasn't priced or released o3 yet, but the Arc Prize Foundation considers o1-pro a reasonable pricing proxy. Mike Knoop, co-founder of the Arc Prize Foundation, stated to TechCrunch that o1-pro is a closer comparison of true o3 cost, given the test-time compute used. He also mentioned that because of uncertainty until official pricing is announced, o3 is still labeled as preview on their leaderboard. The high price aligns with o3 high's reported computing resource usage, as it used 172x more computing than o3 low to address ARC-AGI, according to the Arc Prize Foundation. Rumors also suggest that OpenAI may introduce pricey plans for enterprise customers, with The Information reporting potential charges of up to $20,000 per month for specialized AI "agents". While some argue that even OpenAI's most expensive models will cost less than a human contractor, AI researcher Toby Ord noted on X that the models may not be as efficient. For instance, o3 high needed 1,024 attempts per ARC-AGI task to achieve its best score.

OpenAI's o3 Reasoning Models Are Extremely Expensive to Run

Measuring the intelligence of artificial intelligence is, ironically, a pretty difficult task. That's why the tech industry has come up with benchmarks like ARC-AGI, which tests the capabilities of the new technology through a series of visual tasks that are particularly challenging for A.I. models. In December, OpenAI's o3 reasoning model became the first A.I. system to pass the test with an 87.5 percent score. Sign Up For Our Daily Newsletter Sign Up Thank you for signing up! By clicking submit, you agree to our <a href="http://observermedia.com/terms">terms of service</a> and acknowledge we may use your information to send you emails, product samples, and promotions on this website and other properties. You can opt out anytime. See all of our newsletters But that win didn't come without a price. At the time, the Arc Prize Foundation -- which administers the ARC-AGI benchmark -- estimated that the cost of testing OpenAI's model ran at about $3,400 per task. For a higher-efficiency version of o3 that scored 75.7 percent on the test, that figure totaled at $20 per task. As it turns out, the actual costs could be significantly higher -- ten times higher, to be exact. While the Arc Prize Foundation's o3 pricing was originally drawn from the costs of OpenAI's o1 model, the reasoning predecessor to o3, the nonprofit is now pricing it in line with OpenAI's newly released 01-pro. Unveiled last month, o1-pro model is ten times more costly to run than 01, making it OpenAI's most expensive model to date. Based on the new o1-pro pricing, o3 could potentially cost upwards of $30,000 per task. The more efficient o3 strain has since been listed at costing $200 per task. "Our belief, and this has not been validated by OpenAI, is that o3 pricing will be closer to o1-pro pricing than it will be to o1 pricing that we were told in December," Greg Kamradt, president of the Arc Prize Foundation, told Observer. "Given that, we've updated our metrics." The Arc Prize Foundation has edited its ARC-AGI leadership board to exclude the more compute-intensive version of o3, noting that "only systems which required less than $10,000 to run are shown" on the board. What is ARC-AGI? Founded in 2019 by the researcher François Chollet, the ARC-AGI benchmark relies upon a series of puzzles that track how close A.I. systems are to human-level intelligence. As opposed to simply analyzing a model's ability to draw from datasets, it examines whether or not it can adapt to new problems and learn new task-specific skills. "Think of it as a test that measures the ability to learn new things," said Kamradt. OpenAI's o3 was particularly successful at the test because the model is able to pause and consider numerous potential prompts before responding with the most accurate answer. While o3's pricing has yet to confirmed by OpenAI, the Arc Prize Foundation's estimates will remain anchored closer to o1-pro costs until official pricing is released. "It may go even higher, but we're not sure," said Kamradt. "We're just doing the best that we can with the available information that we have." OpenAI did not respond to requests for comment from Observer. Although recent A.I. releases have gotten closer to the 100 percent mark on ARC-AGI, they've largely been stumped by a newer version of the test released last month. Known as ARC-AGI-2, the test contains tasks that are even more difficult for A.I. systems and especially designed for those that specialize in reasoning. So far, no model has been able to reach the 5 percent mark.

OpenAI's Running Costs Surge 10x as Thiel-Backed Startup Launches US DeepSeek Rival

OpenAI could charge up to $20,000 monthly to access its specialty AI "agents." | Credit:Justin Sullivan / Getty Images. OpenAI's new O3 reasoning artificial intelligence model has had its estimated running price increase tenfold, marking another warning that U.S. AI is becoming too expensive. Meanwhile, Peter Thiel-backed U.S. startup Sentient has announced an open-source AI search framework, which it claims is America's "DeepSeek moment." DeepSeek disrupted the AI industry in January with the release of R1, a ChatGPT-like model that operated at a fraction of the cost of OpenAI's and Google's offerings. OpenAI's Price Increase In December 2024, OpenAI partnered with the creators of ARC-AGI, a benchmark designed to test leading AI, to showcase its O3 model's abilities. The benchmark initially estimated that O3 would cost around $3,000 to solve a single problem from ARC-AGI. However, last week, the ARC Prize Foundation, which maintains ARC-AGI, updated its estimates to around $30,000 per task, TechCrunch reported . The updated pricing highlights the reality of how much money AI is costing Western companies and, in turn, how much consumers will be paying. On Tuesday, April 1, U.S. AI development lab Sentient announced the public release of Open Deep Search (ODS). In a statement shared with Decrypt, Sentient's new models challenge the "dominance of closed AI systems" as the U.S. approaches its own "DeepSeek moment." San Francisco-based Sentient claims ODS, an AI search framework, can outperform closed-source competitors such as OpenAI. Sentient co-founder Himanshu Tyagi told the publication that the testing of its models "relied on their retrieval systems" by excluding "ground truth" sources such as Wikipedia. If the search query fails, the model uses "enhanced query rephrasing, multi-pass retrieval, and intelligent chunking and reranking of web pages," Tyagi told the publication. Rising AI Costs The runaway success of China's DeepSeek highlighted the heavy capital requirements behind building advanced AI models like ChatGPT. ChatGPT, for example, is estimated to cost OpenAI around $700,000 daily to run, but recent reports suggest a single query can now cost over $1,000 in 2025. In comparison, DeepSeek claims it costs under $6 million to build an AI model using Nvidia's weaker H800 chips. Shortly following its release in January, DeepSeek's mobile application raced to the top of the U.S. Apple App Store charts.