DeepSeek R1 vs OpenAI o3-mini: A Comprehensive Comparison of AI Models

OpenAI o3-mini vs. DeepSeek R1: Which one to choose?

The rapid evolution of large language models has brought two notable contenders to the forefront: OpenAI's o3-mini and DeepSeek R1. While both target enterprise and developer use cases, their architectures, performance profiles, and cost structures diverge significantly. Below is a detailed analysis based on verified technical specifications and benchmark results. DeepSeek R1 excels in mathematical reasoning and coding tasks. It scores 97.3% on the MATH-500 benchmark, solving advanced problems with near-perfect accuracy, and ranks in the 96.3rd percentile on Codeforces, a platform for competitive programming. Its general knowledge capabilities, measured by the MMLU benchmark, reach 90.8%, outperforming many industry-leading models. Also read: Krutrim-2 - Can India's language-first AI outpace global benchmarks? The o3-mini focuses on practical applications like software development. It resolves 61% of software engineering tasks on the SWE-bench test, making it suitable for tools like coding assistants. While OpenAI hasn't disclosed its math scores, the model reduces errors by 24% compared to its predecessor, offering reliability for technical workflows. The o3-mini uses a dense transformer, a traditional design where all 200 billion parameters process every input. This ensures consistent performance but demands more computational power. Also read: India's 8-GPU gambit: Shivaay, a foundational AI model built against the odds DeepSeek R1 on the other hand uses a Mixture-of-Experts (MoE) architecture. Despite having 671 billion total parameters, only 37 billion are activated per task. This selective approach reduces energy use by 40% compared to dense models, making R1 more efficient for large-scale deployments. DeepSeek R1 is trained on 14.8 trillion tokens over 2.66 million GPU-hours, this open-source model costs just $6 million per training cycle. Its efficiency stems from techniques like multi-token prediction, which streamlines learning. o3-mini was built using 1.2 million A100 GPU-hours, its training data remains undisclosed. The model is fine-tuned for science and engineering tasks, prioritising accuracy in fields like data analysis. DeepSeek R1 is significantly cheaper to operate. At $0.55 per million input tokens, it costs 17x less than the o3-mini's $9.50 rate. For businesses processing millions of tokens daily, this difference can save thousands monthly. Also read: Deepseek to Qwen: Top AI models released in 2025 However, the o3-mini offers free access via ChatGPT, appealing to smaller teams or experimental projects. Its integration with tools like GitHub Copilot also simplifies coding workflows. o3-mini is ideal for analysing lengthy documents (e.g., legal contracts or research papers) due to its 200K-token input capacity. Its structured output support (JSON) suits API automation and data pipelines. Also read: DeepSeek vs Meta: 5 Things Mark Zuckerberg Teased About Llama 4 and the Future of Open-Source AI DeepSeek R1 will be better for cost-sensitive tasks like batch data processing or multilingual support. Its open-source MIT license allows custom modifications, though users must manage privacy risks. Both models push the boundaries of AI capabilities, but their strengths cater to different needs. As they evolve, expect advancements in energy efficiency, coding accuracy, and real-world adaptability.

OpenAI o3-mini vs DeepSeek R1 : AI Coding Comparison

Choosing the right AI language model can feel like trying to pick the perfect tool from an overflowing toolbox -- each option has its strengths, but which one truly fits your needs? If you've found yourself debating between OpenAI's o3-mini vs DeepSeek R1, you're not alone. These two models have been making waves for their impressive capabilities, but their differences can make the decision tricky. Whether you're tackling complex coding challenges, analyzing intricate datasets, or simply looking for a reliable AI partner, understanding how these models stack up is key to making an informed choice. In this comparison by Prompt Engineering, they break down the strengths and quirks of both o3-mini and DeepSeek R1, diving into everything from cost and context window size to reasoning benchmarks and coding performance. Allowing you to easily decide which AI model is better suited for your unique tasks -- or if you're just curious about how they compare. By the end, you'll have a clearer picture of which model aligns with your priorities, helping you make the best decision for your projects without the guesswork. Cost and hosting flexibility are often pivotal considerations when choosing a language model. Here's how o3-mini and DeepSeek R1 compare: If cost efficiency and hosting options are priorities, DeepSeek R1 offers more versatility. However, for those who value consistent performance and reliability, o3-mini's higher cost may be justified. The context window size determines how much information a model can process in a single interaction, which is crucial for tasks involving extensive data or complex conversations. For users working with large-scale data or requiring detailed contextual understanding, o3-mini's larger context window provides a significant advantage. Reasoning ability is a critical factor for tasks involving problem-solving, logical analysis, or abstract thinking. Both models exhibit distinct strengths in this area: For tasks demanding rigorous logical reasoning, o3-mini is the stronger option. However, if your work involves diverse or less structured challenges, DeepSeek R1's versatility may be more beneficial. Browse through more resources below from our in-depth content covering more areas on Large Language Models (LLMs). Coding tasks often reveal significant differences in the capabilities of language models. Here's how o3-mini and DeepSeek R1 compare: For developers working on advanced coding tasks or debugging, DeepSeek R1's superior performance makes it the preferred choice. The ability to trace a model's reasoning process is essential for tasks requiring transparency and detailed analysis. Here's how the models compare: For tasks that demand detailed reasoning transparency, DeepSeek R1 is the clear winner. Prompt sensitivity refers to how well a model adapts to nuanced or unconventional input variations. This capability can significantly impact performance in specialized tasks: For users working with complex or unconventional prompts, DeepSeek R1's adaptability provides a distinct advantage. API performance is a critical consideration for developers and organizations relying on consistent and stable access to language models: The choice between these models depends on whether reliability or flexibility is more important for your specific use case. Response speed is a key factor for applications requiring quick outputs or real-time interactions. Here's how the models compare: For users prioritizing speed, o3-mini is the better option. However, for intricate tasks where accuracy is paramount, DeepSeek R1's slower but more precise responses may save time in the long run. Both OpenAI's o3-mini and DeepSeek R1 are powerful language models, each excelling in specific areas. The ideal choice depends on your unique needs and priorities. Testing both models on your specific tasks can provide the clarity needed to determine which aligns better with your goals and expectations. Media Credit: Prompt Engineering Choosing the right AI language model can feel like trying to pick the perfect tool from an overflowing toolbox -- each option has its strengths, but which one truly fits your needs? If you've found yourself debating between OpenAI's o3-minik and DeepSeek R1, you're not alone. These two models have been making waves for their impressive capabilities, but their differences can make the decision tricky. Whether you're tackling complex coding challenges, analyzing intricate datasets, or simply looking for a reliable AI partner, understanding how these models stack up is key to making an informed choice. In this article, we'll break down the strengths and quirks of both o3-mini and R1, diving into everything from cost and context window size to reasoning benchmarks and coding performance. If you've ever wondered which model is better suited for your unique tasks -- or if you're just curious about how they compare -- you're in the right place. By the end, you'll have a clearer picture of which model aligns with your priorities, helping you make the best decision for your projects without the guesswork. Let's get started! Selecting the most suitable language model can be a complex decision, particularly when comparing two robust options like OpenAI's o3-mini and DeepSeek R1. This detailed analysis evaluates their performance across critical metrics, including cost, reasoning capabilities, coding proficiency, and more. By exploring these factors, you will gain a clearer understanding of which model aligns best with your requirements. Cost and hosting flexibility are often pivotal considerations when choosing a language model. Here's how o3-mini and DeepSeek R1 compare: If cost efficiency and hosting options are priorities, DeepSeek R1 offers more versatility. However, for those who value consistent performance and reliability, o3-mini's higher cost may be justified. The context window size determines how much information a model can process in a single interaction, which is crucial for tasks involving extensive data or complex conversations. For users working with large-scale data or requiring detailed contextual understanding, o3-mini's larger context window provides a significant advantage. Browse through more resources below from our in-depth content covering more areas on Large Language Models (LLMs). Reasoning ability is a critical factor for tasks involving problem-solving, logical analysis, or abstract thinking. Both models exhibit distinct strengths in this area: For tasks demanding rigorous logical reasoning, o3-mini is the stronger option. However, if your work involves diverse or less structured challenges, DeepSeek R1's versatility may be more beneficial. Coding tasks often reveal significant differences in the capabilities of language models. Here's how o3-mini and DeepSeek R1 compare: For developers working on advanced coding tasks or debugging, DeepSeek R1's superior performance makes it the preferred choice. The ability to trace a model's reasoning process is essential for tasks requiring transparency and detailed analysis. Here's how the models compare: For tasks that demand detailed reasoning transparency, DeepSeek R1 is the clear winner. Prompt sensitivity refers to how well a model adapts to nuanced or unconventional input variations. This capability can significantly impact performance in specialized tasks: For users working with complex or unconventional prompts, DeepSeek R1's adaptability provides a distinct advantage. API performance is a critical consideration for developers and organizations relying on consistent and stable access to language models: The choice between these models depends on whether reliability or flexibility is more important for your specific use case. Response speed is a key factor for applications requiring quick outputs or real-time interactions. Here's how the models compare: For users prioritizing speed, o3-mini is the better option. However, for intricate tasks where accuracy is paramount, DeepSeek R1's slower but more precise responses may save time in the long run. Both OpenAI's o3-mini and DeepSeek R1 are powerful language models, each excelling in specific areas. The ideal choice depends on your unique needs and priorities. Testing both models on your specific tasks can provide the clarity needed to determine which aligns better with your goals and expectations.

DeepSeek vs OpenAI : Which AI Model is Best for Data Science?

Selecting the most suitable artificial intelligence (AI) tool for data science involves evaluating performance, accessibility, and cost. This guide by Thu Vu provides an in-depth comparison of two leading models: DeepSeek R1, an open source AI solution, and OpenAI o1, which will soon be replaced by OpenAI's latest model o3. By analyzing their distinct strengths and limitations across various data science tasks, you can determine which model best aligns with your specific requirements. From tackling complex coding tasks to interpreting tricky graphs, these two models bring unique capabilities to the table. DeepSeek R1 shines with its logical reasoning and adaptability, while OpenAI o1 impresses with its speed and polished outputs. But which one is better suited for your needs? That's the question we'll explore in detail, offering insights from real-world testing and practical use cases. By the end of this comparison, you'll have a clearer picture of how these tools stack up and which one might be the perfect fit for your next project. DeepSeek R1 is an open source AI model designed to prioritize accessibility and logical reasoning. Built on reinforcement learning principles, it employs an iterative reasoning approach to solve complex problems. This step-by-step methodology makes it particularly effective for tasks requiring structured solutions, such as mathematical reasoning and algorithmic problem-solving. One of DeepSeek R1's key advantages is its open source nature, which allows users to customize and adapt the model to their specific needs. However, its computational requirements can be demanding, especially when using the full version. To address this challenge, the model offers smaller, distilled versions optimized for local deployment on less powerful hardware. These versions, while sacrificing some performance, make the tool more accessible to users with limited resources. DeepSeek R1 also supports multiple integration options, including local deployment tools, APIs, and web interfaces. This flexibility enables seamless incorporation into diverse workflows, whether you are working on a standalone project or integrating the model into a larger system. Its adaptability makes it a valuable resource for researchers and developers seeking cost-effective AI solutions. OpenAI o1 is a subscription-based AI model that emphasizes user-friendly design and polished performance. It excels in tasks involving vision processing and graph interpretation, showcasing its strength in visual reasoning and knowledge-based evaluations. For example, OpenAI o1 has demonstrated the ability to identify errors in misleading logarithmic graphs -- an area where DeepSeek R1 has occasionally struggled. In addition to its visual reasoning capabilities, OpenAI o1 is highly effective in coding tasks. It consistently produces error-free outputs accompanied by clear, concise explanations, making it a reliable choice for software development and data engineering workflows. The model also excels in data cleaning and preprocessing, offering detailed, step-by-step guidance for complex workflows. Despite its strengths, OpenAI o1's subscription-based pricing may limit accessibility for some users. However, for those who prioritize speed, precision, and a polished user experience, the investment can be worthwhile. Its ability to deliver consistent and reliable results makes it a preferred choice for professionals working on time-sensitive or high-stakes projects. Here are additional guides from our expansive article library that you may find useful on DeepSeek R1. When comparing DeepSeek R1 and OpenAI o1 across core data science tasks, distinct strengths and weaknesses become evident: DeepSeek R1's logical reasoning capabilities are a significant advantage, particularly for tasks requiring detailed problem-solving. However, its slower processing speed can be a drawback for time-sensitive applications. In contrast, OpenAI o1's faster performance and polished outputs make it a strong contender for users prioritizing efficiency and precision. DeepSeek R1's open source framework makes it an attractive option for researchers and data scientists seeking cost-effective solutions. Its local deployment tools and smaller, distilled versions ensure that even users with limited computational power can use its capabilities. The availability of APIs and web interfaces further enhances its usability, allowing for seamless integration into a wide range of workflows. On the other hand, OpenAI o1 offers a more streamlined and reliable experience but comes with a subscription fee. This cost barrier may deter users working on budget-constrained projects. However, for tasks where speed, accuracy, and ease of use are critical, OpenAI o1's advantages often outweigh the expense. Its polished interface and robust performance make it a preferred choice for professionals who value efficiency and reliability. DeepSeek R1 and OpenAI o1 cater to distinct needs within the data science community. DeepSeek R1's open source accessibility, logical reasoning capabilities, and cost-effectiveness make it a strong choice for researchers and developers. Its flexibility and adaptability are particularly appealing for those working on custom projects or with limited resources. However, its slower processing speed and occasional struggles with visual tasks highlight areas where it may fall short. In contrast, OpenAI o1's polished performance, particularly in vision processing and graph interpretation, makes it a reliable option for users requiring high efficiency and accuracy. Its subscription-based model may pose a cost barrier, but for tasks demanding precision and speed, it remains a difficult tool to replace. Ultimately, the choice between DeepSeek R1 and OpenAI o1 will depend on your specific requirements, available resources, and project priorities. By carefully evaluating your needs and the unique strengths of each model, you can select the AI tool that best supports your data science objectives.

ChatGPT vs DeepSeek R1 vs Qwen 2.5 Max : Ultimate AI Showdown

Artificial intelligence has quickly woven itself into the fabric of our daily lives, whether we're coding, searching for information, or creating digital content. But with so many AI models out there, how do you decide which one is the right fit for your needs? If you've ever found yourself juggling between speed, accuracy, creative output, or even privacy concerns, you're not alone. Choosing between tools like ChatGPT, DeepSeek R1, and Qwen 2.5 Max can feel overwhelming, especially when each promises something unique. Whether you're a developer, a business professional, or just someone curious about AI, this overview by Julian Goldie AI comparing ChatGPT vs DeepSeek R1 vs Qwen 2.5 Max is here to help you cut through the noise. Whether you're looking for lightning-fast responses, polished creative outputs, or tools that respect your privacy, there's a solution out there for you. By the end of this article, you'll have a clearer picture of which AI model aligns best with your goals -- without the headache of trial and error. By evaluating their strengths, limitations, and unique features, you can determine which model aligns best with your specific needs, whether you're a developer, a business professional, or a casual user. For developers, coding performance is a critical factor in choosing an AI model. Each of these models offers distinct advantages and drawbacks in this area: If accuracy and usability are your top priorities, Qwen 2.5 Max is the most reliable option. However, for those who value speed and are comfortable with debugging, ChatGPT may be a more practical choice. Access to accurate and up-to-date information is essential for research, fact-checking, and decision-making. Here's how the models compare in web search functionality: For users who rely heavily on web search, Qwen 2.5 Max stands out as the most dependable choice, while ChatGPT's premium-only access may appeal to professionals with specific needs. Uncover more insights about AI model comparisons in previous articles we have written. Creative tasks such as image generation highlight significant differences in the models' capabilities. Here's how they perform: For users seeking visually compelling and professional-quality outputs, Qwen 2.5 Max is the superior option. ChatGPT may suffice for simpler projects, while DeepSeek R1 is best avoided for image generation tasks. Video generation is a demanding feature that reveals stark contrasts between the models: For those willing to invest in premium features, ChatGPT provides the best video generation quality. However, Qwen 2.5 Max offers a viable alternative for users who prioritize affordability over speed. Data privacy and offline functionality are critical considerations for many users. Here's how the models address these needs: For those who prioritize privacy and offline functionality, DeepSeek R1 is the standout option, offering both local hosting and free API access. Selecting the right AI model depends on your specific needs and priorities. Here's a breakdown to guide your decision: Each AI model has distinct strengths and limitations. Qwen 2.5 Max emerges as the most balanced option for general users, offering functionality and accessibility. DeepSeek R1 caters to developers and those focused on privacy, while ChatGPT appeals to professionals seeking premium features. By understanding your goals and requirements, you can make an informed choice and select the AI model that best aligns with your needs.

Open-source revolution: How DeepSeek-R1 challenges OpenAI's o1 with superior processing, cost efficiency

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More The AI industry is witnessing a seismic shift with the introduction of DeepSeek-R1, a cutting-edge open-source reasoning model developed by the eponymous Chinese startup DeepSeek. Released on January 20, this model is challenging OpenAI's o1 -- a flagship AI system -- by delivering comparable performance at a fraction of the cost. But how do these models stack up in real-world applications? And what does this mean for enterprises and developers? In this article, we dive deep into hands-on testing, practical implications and actionable insights to help technical decision-makers understand which model best suits their needs. Real-world implications: Why this comparison matters The competition between DeepSeek-R1 and OpenAI o1 isn't just about benchmarks -- it's about real-world impact. Enterprises are increasingly relying on AI for tasks like data analysis, customer service automation, decision-making and coding assistance. The choice between these models can significantly affect cost efficiency, workflow optimization and innovation potential. To answer these questions, we conducted hands-on testing across reasoning, mathematical problem-solving, coding tasks and decision-making scenarios. Here's what we found. Hands-on testing: How DeepSeek and OpenAI o1 perform Question 1: Logical inference If A = B, B = C, and C ≠ D, what definitive conclusion can be drawn about A and D? Analysis: Key Insight: DeepSeek-R1 achieves the same logical clarity with better efficiency, making it ideal for high-volume, real-time applications. Question 2: Set theory problem In a room of 50 people, 30 like coffee, 25 like tea and 15 like both. How many people like neither coffee nor tea? Analysis: Key Insight: DeepSeek-R1's concise approach maintains clarity while improving speed. Key Insight: Choice depends on use case -- teaching versus practical application. DeepSeek-R1 excels in speed and accuracy for logical and mathematical tasks, making it ideal for industries like finance, engineering and data science. Question 5: Investment analysis A company has a $100,000 budget. Investment options: Option A yields a 7% return with 20% risk, while Option B yields a 5% return with 10% risk. Which option maximizes potential gain while minimizing risk? Analysis: Key insight: Both models perform well in decision-making tasks, but DeepSeek-R1's concise and actionable outputs make it more suitable for time-sensitive applications. DeepSeek-R1 provides actionable insights more efficiently. Question 6: Efficiency calculation You have three delivery routes with different distances and time constraints: Write a function to find the most frequent element in an array with O(n) time complexity. Analysis: Key insight: Both are effective, with different strengths for different needs. DeepSeek-R1's coding proficiency and optimization capabilities make it a strong contender for software development and automation tasks. Question 8: Algorithm design Design an algorithm to check if a given number is a perfect palindrome without converting it to a string. Analysis: Key Insight: Choice depends on primary need -- speed versus detail. Overall performance metrics The choice between DeepSeek-R1 and OpenAI o1 depends on your specific needs and priorities. Choose DeepSeek-R1 if: Choose OpenAI o1 if: Choose a hybrid approach if: Final thoughts The rise of DeepSeek-R1 signifies a transformative shift in AI development, presenting a cost-effective, high-performance alternative to commercial models like OpenAI's o1. Its open-source nature and robust reasoning capabilities position it as a game-changer for startups, developers and budget-conscious enterprises. Performance analysis of DeepSeek-R1 indicates a substantial advancement in AI capabilities, delivering not only cost savings but also measurably faster processing (2.4X) and clearer outputs compared to OpenAI's o1. The model's combination of speed, efficiency and clarity makes it an ideal choice for production environments and real-time applications. As the AI landscape evolves, the competition between DeepSeek-R1 and OpenAI o1 is likely to spur innovation and enhance accessibility, benefiting the entire ecosystem. Whether you are a technical decision-maker or an inquisitive developer, now is the moment to explore how these models can revolutionize your workflows and unlock new opportunities. The future of AI appears increasingly nuanced, with models being evaluated based on measurable performance rather than brand affiliation.

DeepSeek R1 vs ChatGPT o1 : Reasoning Prompt Comparison Testing

Selecting the right AI reasoning model requires careful evaluation of factors such as accuracy, speed, privacy, and functionality. This guide by Skill Leap AI provides an in-depth comparison of DeepSeek R1 and ChatGPT o1, focusing on their performance with reasoning-based prompts. By understanding their unique strengths and limitations, you can make an informed choice that aligns with your specific needs. Both models bring unique strengths to the table, but they also come with their own quirks and limitations. DeepSeek R1 is celebrated for its meticulous reasoning and open source flexibility, while ChatGPT o1 is known for its speed and user-friendly privacy options. But which one is the better fit for your needs? In the following sections, we'll break down their performance, reasoning processes, privacy considerations, and more -- giving you the insights you need to choose the model that aligns with your priorities. For reasoning-intensive tasks, DeepSeek R1 consistently delivers highly accurate results. It excels in solving complex, multi-step logic problems and interpreting nuanced prompts. In contrast, ChatGPT o1 prioritizes speed, which can sometimes come at the expense of accuracy, particularly with ambiguous or intricate queries. For those considering ChatGPT o1 Pro, which costs $200 per month, its accuracy becomes comparable to DeepSeek R1. However, this comes at a significantly higher price, making DeepSeek R1 a more cost-effective option for users who prioritize accuracy over speed. DeepSeek R1 employs a transparent, step-by-step reasoning process, which contributes to its high level of accuracy. This meticulous approach, however, results in slower response times. On the other hand, ChatGPT o1 is optimized for speed, delivering quicker responses. While this can be advantageous for time-sensitive tasks, its reasoning process is less transparent, which may be a drawback for users who prefer to understand how conclusions are reached. The step-by-step reasoning of DeepSeek R1 is particularly beneficial for tasks requiring detailed explanations or multi-layered logic. ChatGPT o1, while faster, may not always provide the same level of clarity in its reasoning, which could impact its reliability in complex scenarios. Enhance your knowledge on AI Reasoning Models by exploring a selection of articles and guides on the subject. DeepSeek R1 demonstrates superior performance in scenarios requiring intricate reasoning. It handles multi-step logic problems with ease and provides nuanced interpretations of ambiguous prompts. This makes it particularly suitable for tasks that demand precision and depth. ChatGPT o1, while faster, sometimes struggles with specific challenges, such as: For users who prioritize precision and detailed reasoning, DeepSeek R1 is the more reliable choice. However, if speed is a critical factor, ChatGPT o1 may still be a viable option despite its occasional inaccuracies. Privacy is a critical consideration when working with AI models, and DeepSeek R1 and ChatGPT o1 differ significantly in their approaches to data usage and protection. For tasks involving sensitive data, ChatGPT o1's paid plans provide a more secure option. However, users who prefer open source flexibility may still find DeepSeek R1 appealing, despite its privacy limitations. The functionality and accessibility of these models vary based on user needs and available resources. The choice between these models depends on your specific requirements. DeepSeek R1 is ideal for users who need detailed reasoning and are comfortable with potential server issues. ChatGPT o1, on the other hand, offers a more user-friendly experience for those who value speed and stability. Your decision between DeepSeek R1 and ChatGPT o1 will ultimately depend on your priorities and intended use cases. Both models have limitations that may influence your decision. By carefully weighing these factors, you can determine which AI model best aligns with your specific needs and priorities. Both DeepSeek R1 and ChatGPT o1 offer unique advantages, and the right choice will depend on your individual requirements for accuracy, speed, privacy, and functionality.

Beyond benchmarks: How DeepSeek-R1 and o1 perform on real-world tasks

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More DeepSeek-R1 has surely created a lot of excitement and concern, especially for OpenAI's rival model o1. So, we put them to test in a side-by-side comparison on a few simple data analysis and market research tasks. To put the models on equal footing, we used Perplexity Pro Search, which now supports both o1 and R1. Our goal was to look beyond benchmarks and see if the models can actually perform ad hoc tasks that require gathering information from the web, picking out the right pieces of data and performing simple tasks that would require substantial manual effort. Both models are impressive but make errors when the prompts lack specificity. o1 is slightly better at reasoning tasks but R1's transparency gives it an edge in cases (and there will be quite a few) where it makes mistakes. Here is a breakdown of a few of our experiments and the links to the Perplexity pages where you can review the results yourself. Calculating returns on investments from the web Our first test gauged whether models could calculate returns on investment (ROI). We considered a scenario where the user has invested $140 in the Magnificent Seven (Alphabet, Amazon, Apple, Meta, Microsoft, Nvidia, Tesla) on the first day of every month from January to December 2024. We asked the model to calculate the value of the portfolio at the current date. To accomplish this task, the model would have to pull Mag 7 price information for the first day of each month, split the monthly investment evenly across the stocks ($20 per stock), sum them up and calculate the portfolio value according to the value of the stocks on the current date. In this task, both models failed. o1 returned a list of stock prices for January 2024 and January 2025 along with a formula to calculate the portfolio value. However, it failed to calculate the correct values and basically said that there would be no ROI. On the other hand, R1 made the mistake of only investing in January 2024 and calculating the returns for January 2025. However, what was interesting was the models' reasoning process. While o1 did not provide much details on how it had reached its results, R1's reasoning traced showed that it did not have the correct information because Perplexity's retrieval engine had failed to obtain the monthly data for stock prices (many retrieval-augmented generation applications fail not because of the model lack of abilities but because of bad retrieval). This proved to be an important bit of feedback that led us to the next experiment. Reasoning over file content We decided to run the same experiment as before, but instead of prompting the model to retrieve the information from the web, we decided to provide it in a text file. For this, we copy-pasted stock monthly data for each stock from Yahoo! Finance into a text file and gave it to the model. The file contained the name of each stock plus the HTML table that contained the price for the first day of each month from January to December 2024 and the last recorded price. The data was not cleaned to reduce the manual effort and test whether the model could pick the right parts from the data. Again, both models failed to provide the right answer. o1 seemed to have extracted the data from the file, but suggested the calculation be done manually in a tool like Excel. The reasoning trace was very vague and did not contain any useful information to troubleshoot the model. R1 also failed and didn't provide an answer, but the reasoning trace contained a lot of useful information. For example, it was clear that the model had correctly parsed the HTML data for each stock and was able to extract the correct information. It had also been able to do the month-by-month calculation of investments, sum them and calculate the final value according to the latest stock price in the table. However, that final value remained in its reasoning chain and failed to make it into the final answer. The model had also been confounded by a row in the Nvidia chart that had marked the company's 10:1 stock split on June 10, 2024, and ended up miscalculating the final value of the portfolio. Again, the real differentiator was not the result itself, but the ability to investigate how the model arrived at its response. In this case, R1 provided us with a better experience, allowing us to understand the model's limitations and how we can reformulate our prompt and format our data to get better results in the future. Comparing data over the web Another experiment we carried out required the model to compare the stats of four leading NBA centers and determine which one had the best improvement in field goal percentage (FG%) from the 2022/2023 to the 2023/2024 seasons. This task required the model to do multi-step reasoning over different data points. The catch in the prompt was that it included Victor Wembanyama, who just entered the league as a rookie in 2023. The retrieval for this prompt was much easier, since player stats are widely reported on the web and are usually included in their Wikipedia and NBA profiles. Both models answered correctly (it's Giannis in case you were curious), although depending on the sources they used, their figures were a bit different. However, they did not realize that Wemby did not qualify for the comparison and gathered other stats from his time in the European league. In its answer, R1 provided a better breakdown of the results with a comparison table along with links to the sources it used for its answer. The added context enabled us to correct the prompt. After we modified the prompt specifying that we were looking for FG% from NBA seasons, the model correctly ruled out Wemby from the results. Final verdict Reasoning models are powerful tools, but still have a ways to go before they can be fully trusted with tasks, especially as other components of large language model (LLM) applications continue to evolve. From our experiments, both o1 and R1 can still make basic mistakes. Despite showing impressive results, they still need a bit of handholding to give accurate results. Ideally, a reasoning model should be able to explain to the user when it lacks information for the task. Alternatively, the reasoning trace of the model should be able to guide users to better understand mistakes and correct their prompts to increase the accuracy and stability of the model's responses. In this regard, R1 had the upper hand. Hopefully, future reasoning models, including OpenAI's upcoming o3 series, will provide users with more visibility and control.