Model Performance

Twitter

Facebook

Copy Link

Key Facts

Key PlatformsLMSYS Arena, Open LLM Leaderboard

Key BenchmarksMMLU, HumanEval, GSM8K, ARC

Top News

OpenAI launches dedicated team to position AI as scientific collaborator across research fields

OpenAI has established a new AI for Science team led by Kevin Weil to help scientists accelerate research across mathematics, physics, chemistry, and biology. The company reports that 1.3 million weekly users now send 8.4 million messages on advanced science topics, representing 47% growth over the past year. GPT-5.2 achieves 92% accuracy on graduate-level benchmarks, though questions remain about long-term validation.

Technology4 sources

OpenAI launches dedicated team to position AI as scientific collaborator across research fields

Technology4 sources

AI Agents Hit Mathematical Wall: Vishal Sikka Paper Shows LLMs Can't Handle Complex Tasks

A paper by former SAP CTO Vishal Sikka and his son mathematically proves that LLMs powering AI agents cannot reliably execute tasks beyond a certain complexity threshold. The research challenges industry promises about autonomous AI systems, showing that transformer-based language models have fundamental limitations that even reasoning models can't overcome.

Technology5 sources

AI Agents Hit Mathematical Wall: Vishal Sikka Paper Shows LLMs Can't Handle Complex Tasks

Technology5 sources

AI agents score below 25% on workplace readiness test, exposing critical gaps in office work

Training-data company Mercor released the Apex-Agents benchmark, testing leading AI models on real professional tasks from law, investment banking, and consulting. Even top performers like Gemini 3 Flash and GPT-5.2 achieved only 24% and 23% accuracy respectively. The research reveals AI agents struggle most with multi-domain reasoning and synthesizing scattered information across workplace tools.

Science3 sources

AI agents score below 25% on workplace readiness test, exposing critical gaps in office work

Science3 sources

Inferact raises $150M to commercialize vLLM and build next-gen AI inference infrastructure

The creators of vLLM have launched Inferact with $150 million in seed funding at an $800 million valuation, led by Andreessen Horowitz and Lightspeed Venture Partners. As AI shifts from training to deployment, technologies that make AI inference faster and more affordable are attracting significant investor attention. The startup plans to enhance the open-source project while building commercial infrastructure.

Startups3 sources

Inferact raises $150M to commercialize vLLM and build next-gen AI inference infrastructure

Startups3 sources

Anthropic keeps redesigning its hiring test because Claude AI now beats human candidates

Anthropic has been forced to repeatedly revise its technical interview test since 2024 as its own AI models have grown powerful enough to outperform human applicants. Claude Opus 4.5 now matches even the strongest candidates, creating a serious challenge for distinguishing genuine talent from AI-assisted submissions in take-home assessments.

Technology2 sources

Anthropic keeps redesigning its hiring test because Claude AI now beats human candidates

Technology2 sources

Google's Gemini AI sees developer requests surge twice as much in five months

Google's Gemini AI models are experiencing explosive growth, with developer requests more than doubling in just five months. The surge signals improving model quality and increased revenue for Google Cloud, though profitability margins remain a challenge. Meanwhile, Google's partnership with Walmart aims to transform e-commerce through natural language shopping.

Business2 sources

Google's Gemini AI sees developer requests surge twice as much in five months

Business2 sources

AI Models Battle Through Dungeons & Dragons to Test Long-Term Decision-Making Abilities

Researchers at UC San Diego are using Dungeons & Dragons as a testing ground for AI agents that need to function independently over extended periods. The tabletop role-playing game's complex rules and teamwork requirements provide ideal benchmarks for evaluating AI performance on long-term tasks, addressing a critical gap in current LLM assessment methods.

Science2 sources

AI Models Battle Through Dungeons & Dragons to Test Long-Term Decision-Making Abilities

Science2 sources

South Korea launches AI Squid Game to build homegrown models and compete with US, China

South Korea unveiled a high-stakes government-sponsored competition called the AI Squid Game to identify leaders in its bid to become a global AI powerhouse. The tournament eliminates teams every six months, with Naver and NCSoft already cut from the race. LG, SK Group, and startup Upstage remain in contention as the country aims to develop indigenous open-source models rivaling ChatGPT and DeepSeek.

Policy2 sources

South Korea launches AI Squid Game to build homegrown models and compete with US, China

Policy2 sources

Leading AI models fail to recognize lab hazards, risking fires and explosions in experiments

A new study reveals that 19 leading AI models, including GPT-4o and DeepSeek-R1, failed to reliably identify laboratory hazards in realistic scenarios. While some scored up to 86% on basic questions, none exceeded 70% accuracy when faced with complex lab situations, raising serious concerns about AI safety in scientific research.

Science2 sources

Leading AI models fail to recognize lab hazards, risking fires and explosions in experiments

Science2 sources

Opera One R3 launches with smarter Browser AI, color-coded Tab Islands, and music-reactive themes

Opera has released Opera One R3, updating its flagship browser with enhanced AI capabilities and improved multitasking tools. The update introduces color-coded Tab Islands for better organization, a rebuilt agentic AI engine that's 20% faster, and dynamic themes including a Spotify collaboration that reacts to music. The Browser AI now understands tab context and can analyze YouTube videos.

Technology2 sources

Opera One R3 launches with smarter Browser AI, color-coded Tab Islands, and music-reactive themes

Technology2 sources

Nvidia DLSS 4.5 brings AI upscaling improvements to all RTX GPUs, but older cards may struggle

Nvidia has rolled out DLSS 4.5 Super Resolution through its app, bringing enhanced image quality to over 400 games across all RTX GPUs. The update features a second-generation transformer model that improves lighting and reduces ghosting, but requires 5x more power. RTX 20 and 30 series users may experience up to 20% performance loss compared to DLSS 4.

Technology3 sources

Nvidia DLSS 4.5 brings AI upscaling improvements to all RTX GPUs, but older cards may struggle

Technology3 sources

Google releases MedGemma 1.5 with advanced imaging and MedASR for medical speech-to-text

Google has launched MedGemma 1.5, an updated healthcare AI model with support for CT scans, MRI, and histopathology, alongside MedASR, a medical speech-to-text system. The open-source AI models are available via Hugging Face and Vertex AI, marking Google's community-focused approach to healthcare AI as competition intensifies with OpenAI and Anthropic.

Health3 sources

Google releases MedGemma 1.5 with advanced imaging and MedASR for medical speech-to-text

Health3 sources

DeepSeek's Engram tackles AI compute waste with conditional memory breakthrough

DeepSeek unveiled Engram, a conditional memory technique that separates static information retrieval from complex reasoning tasks. The method allows AI models to bypass GPU memory constraints by committing knowledge to system RAM, reducing reliance on expensive high-bandwidth memory while improving performance on long-context queries.

Technology5 sources

DeepSeek's Engram tackles AI compute waste with conditional memory breakthrough

Technology5 sources

OpenAI asks contractors to upload real work from past jobs to train advanced AI agents

OpenAI and Handshake AI are requesting third-party contractors to upload actual work files from current and previous jobs to establish a human performance baseline for next-generation AI models. The initiative aims to measure how well AI can automate more white-collar tasks, but intellectual property lawyer Evan Brown warns the approach puts the company at great risk due to potential trade secret misappropriation and confidentiality breaches.

Technology3 sources

OpenAI asks contractors to upload real work from past jobs to train advanced AI agents

Technology3 sources

Keysight Technologies launches AI validation software for safety-critical systems

Keysight Technologies introduced AI Software Integrity Builder, a unified software solution designed to validate and maintain AI-enabled systems in safety-critical environments like automotive. The tool addresses regulatory scrutiny from standards like ISO/PAS 8800 and the EU AI Act by providing dataset analysis, model validation, real-world inference testing, and continuous monitoring capabilities throughout the AI lifecycle.

Technology2 sources

Keysight Technologies launches AI validation software for safety-critical systems

Technology2 sources

Prev 1…151617…24 Next