OpenAI's ChatGPT-5 (Orion) Faces Performance Challenges, Raising Questions About AI Progress

OpenAI Comments on Leaked ChatGPT-5 Orion AI Performance

OpenAI, a leading artificial intelligence research laboratory, has recently found itself at the center of attention following leaks about their upcoming AI model, ChatGPT-5 performance, internally known as Orion. These leaks have sparked a broader discussion about the challenges and future prospects of AI development, particularly focusing on the model's ability to tackle complex problems beyond its initial training scope. While the whispers of GPT-5's struggles might seem like a setback, they also open up a broader dialogue about the future of AI. It's not just OpenAI feeling the pressure; industry giants like Google and Anthropic are grappling with similar hurdles. This situation raises a crucial question: are we hitting a wall with current AI technologies, or is there a way to push beyond these limitations? As we dive deeper into this with the AI Grid, explore how OpenAI and the broader AI community are addressing these challenges, hinting at innovative solutions that could redefine the landscape of artificial intelligence. The leaked information suggests that GPT-5 is encountering significant hurdles in meeting OpenAI's ambitious performance targets. A primary concern is the model's apparent difficulty in solving coding questions that fall outside its training parameters. This limitation has reportedly led to a delay in the model's release, now anticipated for early next year. These challenges highlight the ongoing complexities in AI model development, especially in creating systems that can effectively generalize beyond their training data. The situation underscores a critical question in the field: How can AI models be designed to adapt and apply knowledge to novel situations? OpenAI's challenges with ChatGPT-5 performance are not isolated incidents in the AI landscape. Other tech giants, including Google and Anthropic, are reportedly facing similar obstacles, experiencing diminishing returns in their AI advancements. This trend has raised concerns about whether deep learning, the foundational technology behind many current AI models, might be approaching a developmental plateau. The AI industry is now grappling with a crucial question: How can developers push beyond current limitations to create more reliable, versatile, and intelligent AI systems? This challenge is prompting a reevaluation of existing methodologies and spurring the exploration of new approaches. Here are more detailed guides and articles that you may find helpful on AI model development. The AI community has been vocal and divided in its response to these developments. Gary Marcus, a prominent critic of deep learning, advocates for a hybrid approach that combines symbolic reasoning with deep learning to overcome current limitations. This perspective suggests that a significant paradigm shift may be necessary to advance AI capabilities to the next level. However, the community remains split on this issue. While some researchers support exploring new approaches, others maintain confidence in the potential of existing methods, arguing that continued refinement and scaling of current technologies will lead to breakthroughs. In response to the leaks, OpenAI's leadership, including CEO Sam Altman, has dismissed claims of hitting a developmental wall. They assert that future AI models will surpass current benchmarks, indicating that the company remains optimistic about overcoming present challenges. OpenAI's position reflects a belief that continued innovation and refinement within the existing paradigm will lead to significant advancements in AI performance. This stance underscores the company's commitment to pushing the boundaries of what's possible in AI technology and its performance. As AI models become more sophisticated, so too do the methods used to evaluate their performance. Recent developments in this area are providing new insights into AI capabilities: These advancements in evaluation techniques highlight the importance of robust benchmarking in driving AI development forward. They provide a more nuanced understanding of AI capabilities and limitations, guiding researchers in refining and improving their models. The future of AI technology remains a topic of intense debate and speculation within the scientific community. While some experts anticipate a potential slowdown in progress, others foresee significant advancements through paradigm shifts and the integration of new reasoning abilities. The focus of AI development is increasingly shifting towards enhancing practical applications and reliability. There is a growing emphasis on making sure that future models can meet the complex demands of real-world scenarios, from solving intricate coding problems to assisting in scientific research and decision-making processes. As the AI landscape continues to evolve, several key areas are likely to shape its trajectory: The ongoing dialogue about AI's capabilities and limitations will play a crucial role in shaping the direction of future developments. As researchers and developers work to overcome current challenges, the goal remains clear: to unlock the full potential of artificial intelligence and create systems that can truly augment and enhance human capabilities across various domains. While OpenAI's ChatGPT-5 performance faces scrutiny, it represents just one chapter in the broader narrative of AI advancement. The industry's response to these challenges will likely define the next era of AI technology, potentially leading to breakthroughs that could reshape our understanding of machine intelligence and its role in society.

ChatGPT-5 Improvements are Slowing Down - OpenAI Orion AI Model News

OpenAI's development of the Orion model or ChatGPT-5, potentially replacing GPT-4, faces challenges in improvement speed and effectiveness it seems, raising questions about AI scaling laws and future advancements. In a world where technology is advancing at breakneck speed, it's easy to assume that each new development will be leaps and bounds ahead of its predecessor. Yet, as we dive into the story of OpenAI's Orion model, we find ourselves at a fascinating crossroads. Orion, poised to potentially take the baton from GPT-4, is grappling with the very real challenges of improvement speed and effectiveness. The model's performance, particularly in areas like coding, hasn't yet eclipsed that of its predecessors, and its higher operational costs add another layer of complexity. But don't worry, there's a silver lining. The AI community is beginning to pivot towards innovative strategies like test-time compute, which could redefine how we enhance AI models post-training. This shift in focus might just hold the key to unlocking the next wave of AI breakthroughs, offering a glimpse of hope amid the current challenges. OpenAI's new ChatGPT-5 or Orion AI model, potentially succeeding GPT-4, stands at the forefront of AI development discussions. Its performance and the challenges it faces in surpassing previous models raise critical questions about the future trajectory of AI scaling laws and advancements. As we provide more insight into Orion's capabilities, you'll discover that the path to AI improvement is intricate and fraught with complexities. The Orion model demonstrates promise, yet it has not definitively outperformed GPT-5 in key areas such as coding tasks. This performance plateau raises significant concerns about its cost-effectiveness, particularly given reports of Orion's higher operational expenses compared to its predecessors. These financial considerations are crucial when evaluating the model's practical applications and scalability potential. The long-held belief that AI models invariably improve with increased data and computing power is now under intense scrutiny. Scaling laws, once considered the bedrock of AI development, are being reevaluated. The focus is shifting towards enhancing models post-training, a concept known as test-time compute. This paradigm shift could fundamentally alter your perspective on AI model improvement, emphasizing efficiency and targeted enhancements over sheer scale. Expand your understanding of ChatGPT-5 Models with additional resources from our extensive library of articles. Within OpenAI, an internal model specifically designed for software engineering tasks has gained significant traction. Despite Orion's training being only 20% complete, it already matches GPT-4 in certain domains. This internal success story underscores the potential of specialized models tailored for specific applications. However, OpenAI is not alone in navigating these challenges. Industry giants like Google and Meta are also grappling with diminishing returns from new models, spurring a collective search for novel paradigms that transcend traditional scaling laws. A pressing issue in the development of large language models (LLMs) is the scarcity of high-quality training data. As you explore the use of synthetic data to bridge this gap, it's crucial to recognize the inherent risks. Overreliance on data generated by previous models could lead to a phenomenon known as model collapse, where new iterations merely replicate the capabilities of their predecessors, potentially stifling innovation. Advanced models like the 01 series offer remarkable capabilities but remain prohibitively expensive for widespread adoption. Understanding the economic implications of these AI models is essential for anticipating their future accessibility and societal impact. The potential for these innovative models to become more cost-effective and widely available remains a key consideration for the AI industry and its stakeholders. The AI community is divided on the future trajectory of AI development. Some experts, like Gary Marcus, argue that AI is approaching a phase of diminishing returns, while others see significant potential in test-time compute to enhance model capabilities. Despite the challenges, the outlook for AI development remains promising. New paradigms and incremental improvements are expected to drive progress, challenging the narrative that AI advancement is slowing down. The OpenAI Orion ChatGPT-5 model embodies both the immense potential and the formidable challenges of modern AI development. As you navigate this complex landscape, understanding the delicate balance between scaling laws, data quality, and economic factors will be crucial in shaping the future of AI technology. The journey ahead involves not just technological breakthroughs but also addressing ethical considerations, resource allocation, and the broader societal implications of advanced AI systems. As AI continues to evolve, it's clear that the path forward will require innovative approaches, collaborative efforts across the industry, and a willingness to challenge established norms. The story of Orion and its contemporaries serves as a reminder that the frontier of AI is not just about raw computational power, but about finding smarter, more efficient ways to harness the potential of these powerful tools.

OpenAI's Next-Gen Model Hits Performance Wall: Report - Decrypt

OpenAI's upcoming artificial intelligence model is delivering smaller performance gains than its predecessors, sources familiar with the matter told The Information. Employee testing reveals Orion achieved GPT-4 level performance after completing only 20% of its training, The Information reports. The quality increase from GPT-4 to the current version of GPT-5 seems to be smaller than that of GPT-3 to GPT-4. "Some researchers at the company believe Orion isn't reliably better than its predecessor in handling certain tasks, according to the (OpenAI) employees," The Information reported. "Orion performs better at language tasks but may not outperform previous models at tasks such as coding, according to an OpenAI employee." While Orion getting closer to GPT-4 at 20% of its training might sound impressive to some, it is important to note that the early stages of AI training typically deliver the most dramatic improvements, with subsequent phases yielding smaller gains. So, the remaining 80% of training time isn't likely to produce the same magnitude of advancement seen in previous generational leaps, sources said. The limitations emerge at a critical juncture for OpenAI following its recent $6.6 billion funding round. The company now faces heightened expectations from investors while grappling with technical constraints that challenge traditional scaling approaches in AI development. If these early versions don't meet expectations, the company's upcoming fundraising efforts may not be met with the same hype as before -- and that could be a problem for a potentially for-profit company, which is what Sam Altman seems to want for OpenAI. Underwhelming results point to a fundamental challenge facing the entire AI industry: the diminishing supply of high-quality training data and the need to remain relevant in a field as competitive as generative AI. Research published in June predicted that AI companies will exhaust available public human-generated text data between 2026 and 2032, marking a critical inflection point for traditional development approaches. "Our findings indicate that current LLM development trends cannot be sustained through conventional data scaling alone," the research paper states, highlighting the need for alternative approaches to model improvement, including synthetic data generation, transfer learning from data-rich domains, and the use of non-public data. The historical strategy of training language models on publicly available text from websites, books, and other sources has reached a point of diminishing returns, with developers having "largely squeezed as much out of that type of data as they can," according to The Information. To tackle these challenges, OpenAI is fundamentally restructuring its approach to AI development. "In response to the recent challenge to training-based scaling laws posed by slowing GPT improvements, the industry appears to be shifting its effort to improving models after their initial training, potentially yielding a different type of scaling law," The Information reports. To achieve this state of continuous improvement, OpenAI is separating model development into two distinct tracks: The O-Series (which seems to be codename Strawberry), focused on reasoning capabilities, represents a new direction in model architecture. These models operate with significantly higher computational intensity and are explicitly designed for complex problem-solving tasks. The computational demands are substantial, with early estimates suggesting operational costs at six times that of current models. However, the enhanced reasoning capabilities could justify the increased expense for specific applications requiring advanced analytical processing. This model, if it's the same as Strawberry, is also tasked to generate enough synthetic data to increase the quality of OpenAI's LLMs constantly. In parallel, the Orion Models or the GPT Series (considering OpenAI trademarked the name GPT-5) continue to evolve, focusing on general language processing and communication tasks. These models maintain more efficient computational requirements while leveraging their broader knowledge base for writing and argumentation tasks. OpenAI's CPO Kevin Weil also confirmed this during an AMA and said he expects to converge both developments at some point in the future. "It's not either or, it's both," he replied when asked whether OpenAI would focus on scaling LLMs with more data or using a different approach, focusing on smaller but faster models, "better base models plus more strawberry scaling/inference time compute." OpenAI's approach to addressing data scarcity through synthetic data generation presents complex challenges for the industry. The company's researchers are developing sophisticated models designed to generate training data, yet this solution introduces new complications in maintaining model quality and reliability. As previously reported by Decrypt, researchers have found that model training on synthetic data represents a double-edged sword. While it offers a potential solution to data scarcity, it introduces new risks of model degradation and reliability concerns with proven degradation after several training iterations. In other words, as models train on AI-generated content, they may begin to amplify subtle imperfections in their outputs. These feedback loops can perpetuate and magnify existing biases, creating a compounding effect that becomes increasingly difficult to detect and correct. OpenAI's Foundations team is developing new filtering mechanisms to maintain data quality, implementing different validation techniques to distinguish between high-quality and potentially problematic synthetic content. The team is also exploring hybrid training approaches that strategically combine human and AI-generated content to maximize the benefits of both sources while minimizing their respective drawbacks. Post-training optimization has also gained relevance. Researchers are developing new methods to enhance model performance after the initial training phase, potentially offering a way to improve capabilities without relying solely on expanding the training dataset. That said, GPT-5 is still an embryo of a complete model with significant development work ahead. Sam Altman, OpenAI's CEO, has indicated that it won't be ready for deployment this year or next. This extended timeline could prove advantageous, allowing researchers to address current limitations and potentially discover new methods for model enhancement, considerably improving GPT-5 before its eventual release.

ChatGPT-5 Exhibiting Diminishing Returns is AI Progress Slowing Down?

According to leaked information from OpenAI it seems that Artificial Intelligence (AI) development, particularly in the realm of language models, might be experiencing a notable deceleration. This shift in momentum is most evident in projects like OpenAI's highly anticipated Orion model or ChatGPT-5. You might be wondering why the excitement is tempered this time around. Well, it's not that AI isn't progressing -- it's just that the pace of progress is starting to feel a bit like running through molasses. This slowdown isn't just a technical hiccup; it's a reflection of the complex challenges that come with pushing the boundaries of what's possible with AI. From data scarcity to the astronomical costs of training these AI models, the hurdles are real and significant. But don't let this dampen your spirits just yet. While the road ahead may be fraught with challenges, it's also paved with opportunities for innovation and growth. The key might lie in shifting our focus from sheer data volume to data efficiency -- finding smarter ways to extract value from the information we already have. This approach could be the fantastic option that helps us overcome the current plateau and unlock new potentials in AI capabilities. So, join AI Explained as we explore the intricacies of AI's current landscape, where the promise of breakthroughs in other AI domains continues to shine brightly. OpenAI's forthcoming model, Orion, serves as a prime example of the hurdles facing AI progress. Despite substantial financial and intellectual investment, Orion's improvements over its predecessor, GPT-4, are less dramatic than initially hoped. Early training results show Orion performing at a level comparable to GPT-4, with only marginal enhancements expected upon completion. This plateau effect is not unique to Orion but symptomatic of a broader trend in language model development. Key factors contributing to this slowdown include: At the heart of the AI slowdown lies a critical issue: the scarcity of high-quality data. As language models grow more sophisticated, they require increasingly refined and diverse datasets for training. However, the availability of such data is limited, creating a bottleneck in the development process. This scarcity is not merely about quantity but quality - the data must be relevant, accurate, and representative to drive meaningful improvements in AI capabilities. Furthermore, the cost of acquiring and processing this data is substantial. As models grow larger and more complex, the computational resources required for training escalate exponentially. This economic factor adds another layer of complexity to the advancement of AI technology. Browse through more resources below from our in-depth content covering more areas on Large Language Models. One area where the limitations of current AI models become glaringly apparent is in mathematical reasoning. Despite their prowess in language processing and generation, models like Orion struggle with complex mathematical problems. This deficiency highlights a fundamental gap in AI's cognitive abilities, underscoring the need for innovative approaches to enhance logical and analytical reasoning in these systems. Challenges in mathematical reasoning include: In the face of these challenges, improving data efficiency emerges as a promising avenue for progress. By enhancing AI's ability to extract meaningful information from existing datasets, researchers can potentially mitigate the impact of data scarcity. This approach focuses on maximizing the utility of available data rather than simply increasing data volume. Techniques being explored to improve data efficiency include: The current state of AI development has sparked a range of perspectives among experts. Optimists point to ongoing advancements in other AI domains, such as video generation, as evidence of continued progress. They argue that breakthroughs in these areas could potentially translate to improvements in language models and other AI applications. Conversely, pessimists caution against overestimating AI's near-term potential. They highlight the possibility of reaching a technological plateau, where further advancements become increasingly difficult and resource-intensive. This debate reflects the complex and often unpredictable nature of AI research and development. While language models face challenges, other areas of AI continue to show significant promise. Video generation, for instance, has seen remarkable advancements, demonstrating AI's potential to transform visual media creation. Similarly, progress in areas such as robotics, computer vision, and reinforcement learning suggests that AI's impact will continue to expand across various domains. Promising AI domains include: The slowdown in language model development, exemplified by OpenAI's Orion, underscores the complex challenges facing AI advancement. Data limitations, training costs, and gaps in reasoning capabilities present significant hurdles. However, the pursuit of improved data efficiency and the exploration of diverse AI applications offer pathways for continued innovation. As the field evolves, balancing optimism with pragmatism will be crucial in navigating the future of AI technology. The journey ahead requires not just technological breakthroughs but also a nuanced understanding of AI's strengths, limitations, and potential societal impacts.

OpenAI's next-gen Orion model is hitting a serious bottleneck, according to a new report - here's why

Progress is slower than expected due to quality issues with training data OpenAI is running into difficulties with Orion, the next-gen model powering its AI. The company is struggling in certain areas when it comes to the performance gains realized with the successor to GPT-4. This comes from a report by The Information, citing OpenAI employees, who claim that the increase in quality seen with Orion is 'far smaller' than that witnessed when moving from GPT-3 to GPT-4. We're also told that some OpenAI researchers are saying that Orion "isn't reliably better than its predecessor [GPT-4] in handling certain tasks." What tasks would they be? Apparently, coding is a weaker point, with Orion possibly not outdoing GPT-4 in this arena - although it is also noted that Orion's language skills are stronger. So, for general-use queries - and for jobs such as summarizing or rewriting text - it sounds like things are going (relatively) well. However, these rumors don't sound quite as hopeful for those looking to use AI as a coding helper. By all accounts, OpenAI is running into something of a wall when it comes to the data available to train its AI. As the report makes clear, there's a "dwindling supply of high-quality text and other data" that LLMs (Large Language Models) can work with in pre-release training to hone their powers in solving knottier problems like resolving coding bugs. These LLMs have chomped through a lot of the low-hanging fruit, and now finding this good-quality training data is becoming a considerably more difficult process - slowing down advancement in some respects. On top of that, this training will become more intensive in terms of computing resources, meaning that developing (and running) Orion - and further AI models down the line - will become much more expensive. Of course, the user of the AI will end up footing that bill, one way or another, and there's even talk of more advanced models becoming effectively "financially unfeasible" to develop. Not to mention the impact on the environment in terms of bigger data centers whirring away and sucking more power from our grids, all at a time of increasing concern around climate change. While we need to take this report with an appropriate amount of caution, there are worrying rumblings here, foreshadowing a serious reality check for the development of AI going forward. The Information further notes that a different approach may be taken in terms of improving AI models on an ongoing basis after their initial training - indeed, this may become a necessity from the sound of things. We shall see. Orion is expected to debut early in 2025 (and not imminently, as some rumors have hinted), and it may not be called ChatGPT-5, with OpenAI possibly set to change the naming scheme of its AI completely with this next-gen model.

OpenAI Orion is facing scaling challenges

OpenAI Orion, the company's next-generation AI model, is hitting performance walls that expose limitations in traditional scaling approaches. Sources familiar with the matter reveal that Orion is delivering smaller performance gains than its predecessors, prompting OpenAI to rethink its development strategy. Initial employee testing indicates that OpenAI Orion achieved GPT-4 level performance after completing only 20% of its training. While this might sound impressive, it's important to note that early stages of AI training typically yield the most dramatic improvements. The remaining 80% of training is unlikely to produce significant advancements, suggesting that OpenAI Orion may not surpass GPT-4 by a wide margin. "Some researchers at the company believe Orion isn't reliably better than its predecessor in handling certain tasks," reported The Information. "Orion performs better at language tasks but may not outperform previous models at tasks such as coding, according to an OpenAI employee." OpenAI's challenges with Orion highlight a fundamental issue in the AI industry: the diminishing supply of high-quality training data. Research published in June predicts that AI companies will exhaust available public human-generated text data between 2026 and 2032. This scarcity marks a critical inflection point for traditional development approaches, forcing companies like OpenAI to explore alternative methods. "Our findings indicate that current LLM development trends cannot be sustained through conventional data scaling alone," the research paper states. This underscores the need for synthetic data generation, transfer learning, and the use of non-public data to enhance model performance. To tackle these challenges, OpenAI is restructuring its approach by separating model development into two distinct tracks. The O-Series, codenamed Strawberry, focuses on reasoning capabilities and represents a new direction in model architecture. These models operate with significantly higher computational intensity and are explicitly designed for complex problem-solving tasks. In parallel, the Orion models -- or the GPT series -- continue to evolve, concentrating on general language processing and communication tasks. OpenAI's Chief Product Officer Kevin Weil confirmed this strategy during an AMA, stating, "It's not either or, it's both -- better base models plus more strawberry scaling/inference time compute." OpenAI is exploring synthetic data generation to address data scarcity for OpenAI Orion. However, this solution introduces new complications in maintaining model quality and reliability. Training models on AI-generated content may lead to feedback loops that amplify subtle imperfections, creating a compounding effect that's increasingly difficult to detect and correct. Researchers have found that relying heavily on synthetic data can cause models to degrade over time. OpenAI's Foundations team is developing new filtering mechanisms to maintain data quality, implementing validation techniques to distinguish between high-quality and potentially problematic synthetic content. They're also exploring hybrid training approaches that combine human and AI-generated content to maximize benefits while minimizing drawbacks. OpenAI Orion is still in its early stages, with significant development work ahead. CEO Sam Altman has indicated that it won't be ready for deployment this year or next. This extended timeline could prove advantageous, allowing researchers to address current limitations and discover new methods for model enhancement. Facing heightened expectations after a recent $6.6 billion funding round, OpenAI aims to overcome these challenges by innovating its development strategy. By tackling the data scarcity dilemma head-on, the company hopes to ensure that OpenAI Orion will make a substantial impact upon its eventual release.

Is GPT-5 in trouble? Report suggests that AI has plateaued

OpenAI's next-generation Orion model of ChatGPT, which is both rumored and denied to be arriving by the end of the year, may not be all it's been hyped to be once it arrives, according to a new report from The Information. Citing anonymous OpenAI employees, the report claims the Orion model has shown a "far smaller" improvement over its GPT-4 predecessor than GPT-4 showed over GPT-3. Those sources also note that Orion "isn't reliably better than its predecessor [GPT-4] in handling certain tasks," specifically coding applications, though the new model is notably stronger at general language capabilities, such as summarizing documents or generating emails. Recommended Videos The Information's report cites a "dwindling supply of high-quality text and other data" on which to train new models as a major factor in the new model's insubstantial gains. In short, the AI industry is quickly running into a training data bottleneck, having already stripped the easy sources of social media data from sites like X, Facebook, and YouTube (the latter on two different occasions.) As such, these companies are increasingly having difficulty finding the sorts of knotty coding challenges that will help advance their models beyond their current capabilities, slowing down their pre-release training. That reduced training efficiency has massive ecological and commercial implications. As frontier-class LLMs grow and further push their parameter counts into the high trillions, the amount of energy, water, and other resources is expected to increase six-fold in the next decade. This is why we're seeing Microsoft try to restart Three Mile Island, AWS buy a 960 MW plant, and Google purchase the output of seven nuclear reactors, all to provide the necessary power for their growing menageries of AI data centers -- the nation's current power infrastructure simply can't keep up. In response, as TechCrunch reports, OpenAI has created a "foundations team" to circumvent the lack of appropriate training data. Those techniques could involve using synthetic training data, such as what Nvidia's Nemotron family of models can generate. The team is also looking into improving the model's performance post-training. Orion, which was originally thought to be the code name for OpenAI's GPT-5, is now expected to arrive at some point in 2025. Whether we'll have enough available power to see it in action, without browning out our municipal electrical grids, remains to be seen.

The AI rocketship may be running on fumes

Don't hold your breath for major AI upgrades from OpenAI and Google Moments after OpenAI, Google, or Anthropic drop a major upgrade to their AI models, you'll see people already speculating on the next update's date and features. And there have been fairly regular updates to feed those rumors. However, those days may be over, according to a Bloomberg report. All three major AI developers are reportedly struggling to make their next-gen models match their ambitions for improvement over the current crop. The report claims that OpenAI's work on the Orion model isn't going as well as the company expected. The model doesn't perform at the level the company is aiming for, especially when it comes to coding. Orion may not offer a seismic change from GPT-4 compared to how GPT-4 blew GPT-3.5 out of the water. That may be one reason OpenAI CEO Sam Altman publicly pushed back on rumors about the release date for the Orion model and an upgrade to ChatGPT. Delays and lower expectations are also plaguing Google and Anthropic. Google's Gemini development is slower than hoped, according to Bloomberg. Anthropic has already pushed back releasing its Claude 3.5 Opus model for similar reasons despite teasing it earlier this year. All of the AI developers are running into the same ceilings in growing their model's abilities. The biggest is likely training data. The companies have leveraged enormous datasets to train their AI models, but even the internet is not infinite, and that goes even more when discussing high-quality data useful for training AI. Finding previously unused, accessible information is becoming tricky. That's partly because of growing awareness and consideration for ethical and legal rights to use some data, but that's only part of the explanation. At some point, there aren't enough human examples for the AI models to absorb and improve upon. Even if the companies find enough raw data, processing it and incorporating it into an AI model is expensive in terms of money and computing power. If the data cannot make more than slight improvements, then upgrading the AI model might not be worth the price. The report describes how OpenAI and its rivals are looking to other ways of upgrading their models, like post-training Orion with human feedback. That's a slow way to improve an AI model and raises questions about whether AI has reached the limits of rapid scaling in size and functions. Brute computing power and avalanches of data may not be enough to make the dreams of AI developers real anymore. They'll need to get more creative in how they iterate on their models without throwing the entire internet at it. For us, we should expect somewhat slower releases of new and improved AI features. That might not be terrible if it gives everyone a chance to catch their breath and really dig into the best ways to use all the AI tools released in the last few years. There's plenty to explore with ChatGPT-o1. And, who knows, maybe this will give OpenAI the space to work on releasing the Sora AI video creator, which has been kept highly restricted despite OpenAI teasing it with a steady drip of demos.

OpenAI Alarmed When Its Shiny New AI Model Isn't as Smart as It Was Supposed to Be

OpenAI's next large language model may not be as powerful as many hoped. Code-named Orion, the AI model is sorely underperforming behind the scenes, Bloomberg reports, showing less improvement over its predecessor than GPT-4 did over GPT-3. A similar report from The Information this week indicated that some OpenAI researchers believed that in certain areas like coding, there were no improvements at all. And according to Bloomberg, OpenAI isn't the only AI outfit struggling with diminishing returns. Google's next iteration of its Gemini model is also falling short of internal expectations, while the timeline for Anthropic's release of its much hyped Claude 3.5 Opus is up in the air. These industry-wide struggles may be a sign that the current paradigm of improving AI models via what's known as "scaling" is hitting a brick wall, portending potential economic woes in the future if AI models remain costly to develop without achieving significant leaps in performance towards building an artificial general intelligence. "The AGI bubble is bursting a little bit," Margaret Mitchell, chief ethics scientist at the AI startup Hugging Face, told Bloomberg, adding that "different training approaches" may be needed to approach anything like human levels of intelligence and versatility. The ethos that has yielded gains in generative AI so far has been scaling: to make generative AI models more powerful, the primary way to achieve this is by making them bigger. That means adding more processing power -- AI chips, like from Nvidia -- and injecting more training data, which has largely been scraped from the web with little cost. But as these models get larger and more powerful, they also get hungrier. All that energy isn't cheap -- Microsoft is looking to reboot entire nuclear power plants to support its AI data centers, for example -- and free training data is drying up. To obtain new brain food for their AIs, tech companies are using synthetic, computer -generated data. Yet, they still "struggle to get unique, high-quality datasets without human guidance, especially when it comes to language," Lila Tretikov, head of AI strategy at New Enterprise associates, told Bloomberg. And so, to give an idea of all those expenses: in a podcast episode quoted by Bloomberg, Anthropic CEO Dario Amodei said that a cutting-edge AI model currently costs around $100 million to build, and estimated that by 2027, they could cost well over $10 billion apiece. This year, Anthropic updated its Claude models, but notably snubbed Opus, while references to a near-future release date for it have been scrubbed from its website. Like with OpenAI, researchers at the company reportedly observed only marginal improvements with Opus considering its size and how much it cost to build and run, according to a Bloomberg source. Similarly, Google's Gemini software is falling short of its goals, per Bloomberg, and the company has released few major improvements to its large language model in the meantime. These aren't insurmountable challenges, to be clear. But it's increasingly sounding like the AI industry may not enjoy the same pace of advancements as it has in the past decade. "We got very excited for a brief period of very fast progress," Noah Giansiracusa, an associate professor of mathematics at Bentley University in Massachusetts, told Bloomberg. "That just wasn't sustainable."

OpenAI's New Orion Model Offers Only Incremental Improvements Over GPT-4, Despite Claims Of Groundbreaking Advancement

When it comes to bringing forward new AI models and upgrading them, tech companies are not slowing down any time soon and keep on working and evolving their technologies. It has not been long since OpenAI announced its plans to bring an advanced version of GPT-4 known as the Orion model. While the expectations have been set by the company of this being yet another groundbreaking leap in language model development, new reports are surfacing claiming otherwise and giving a heads up on the performance barely outdoing its predecessor, raising the question of whether the AI field, in fact, is hitting the ceiling. While there is no denying OpenAI revolutionized AI and the endless possibilities for technology, and ever since its inception, it has been growing massively, working on bringing more cutting-edge advancements forward, and now seems to be slowly transitioning into a more commercial focus. The company is aggressively pushing toward bringing AGI to the world, but along with that, it has been working on bringing updates for its existing models and even working on new models. The upcoming model by OpenAI, codenamed 'Orion,' is claimed to offer a major leap in capabilities over the GPT-4, and users are eagerly looking forward to a more efficient and ethically aligned model. However, according to The Information's new report, OpenAI's new language model might only offer incremental improvement over its predecessor, contrary to expectations. Orion has been portrayed as the upgraded AI model that outpaces GPT-4 in multiple domains, including programming tasks, but that does not stand to be true as the model seems to only offer improvements in language capabilities. Orion also poses some challenges on the operational front as it is expected to require more resources than previous models to be run in the data centers and also for maintenance. The report further cites that OpenAI presents the reason for relatively slower-paced improvements in the AI models as the insufficient availability of high-quality training data. Since most of the publicly available data has already been utilized, the company has now formed a "Foundations Team" headed by Nick Ryder to find new information sources for training the language models that are unique and attempt to overcome the current limitations in AI advancements. The snail-paced progress in LLM models is not limited to OpenAI and seems to extend across the industry, raising the question of the models reaching stagnation. Perhaps AGI would pave the way for a fresh way to use existing models and bring noteworthy upgrades to them.

OpenAI Might Be Struggling to Improve Its Next AI Model Significantly

The AI firm is also struggling to deal with a lack of training data OpenAI is rumoured to be working on the next generation of its flagship large language model (LLM), however, it might have hit a bottleneck. As per a report, the San Francisco-based AI firm is struggling to considerably upgrade the capabilities of its next AI model, internally codenamed Orion. The model is said to be outperforming older models when it comes to language-based tasks but is underwhelming in certain tasks such as coding. Notably, the company is also said to be struggling to accumulate enough training data to properly train AI models. The Information reported that the AI firm's next major LLM, Orion, is not performing as per expectations when it comes to coding-related tasks. Citing unnamed employees, the report claimed that the AI model has shown a considerable upgrade when it comes to language-based tasks, but certain tasks are underwhelming. This is considered to be a major issue as Orion is reportedly more expensive to run in OpenAI's data centres compared to the older models such as GPT-4 and GPT-4o. The cost-to-performance ratio of the upcoming LLM might pose a challenge for the company to make it appealing to enterprises and subscribers. Additionally, the report also claimed that the overall quality jump between GPT-4 and Orion is less than the jump between GPT-3 and GPT-4. This is a worrying development, however, the trend is also being noticed in other recently released AI models by competitors such as Anthropic and Mistral. The benchmark scores of Claude 3.5 Sonnet, for instance, show that the quality jump is more iterative with each new foundation model. However, competitors have largely avoided the attention by focusing on developing new capabilities such as agentic AI. In the report, the publication also highlighted that the industry, as a way to tackle this challenge, is opting to improve the AI model after the initial training is complete. This could be done by fine-tuning the output by adding additional filters. However, this is a workaround and does not offset the limitation that is being caused by either the framework or the lack of enough data. While the former is more of a technological and research-based challenge, the latter is largely due to the availability of free and licenced data. To solve this, OpenAI has reportedly created a foundations team which has been tasked with finding a way to deal with the lack of training data. However, it cannot be said if this team will be able to procure more data in time to further train and improve the capabilities of Orion.