Geekbench AI: New Benchmark Tool Measures Device AI Performance

Primate Labs launches Geekbench AI tool to test real-world device AI performance

Primate Labs launches Geekbench AI tool to test real-world device AI performance Primate Labs Inc., the maker of cross-platform benchmarking software, officially released Geekbench AI 1.0 on Thursday, a tool specifically designed to test the real-world performance of artificial intelligence workloads on mobile and desktop platforms. The tool, previously known as Geekbench ML, has been renamed to better fit the industry standard naming convention although it does cover machine learning and deep learning workloads alongside AI. Its prior preview releases were developed over years of work with the company's customers, partners and the AI engineering community. Instead of a single "score," the new software breaks down the scoring into three metrics that cover the complexity of scoring performance. "Measuring performance is, put simply, really hard," the company said. "That's not because it's hard to run an arbitrary test, but because it's hard to determine which tests are the most important for the performance you want to measure - especially across different platforms." The three-score system provides different ranges of precision, which may matter when providing a particular type of AI task. This will allow AI developers and engineers to understand how their app will work on the hardware on a more granular level than having a single score, the company said. "With our benchmark, you can explore how different approaches at the hardware level have been optimized for particular tasks," the company said. The new tool also includes workload accuracy measurements on a per-test basis. This is because AI involves not just running fast, but depends on its capability to get the job done with a low error rate. For example, an image recognition model looking for cats might run quickly, however, if it only detects cats 1% of the time, it's not doing a great job. This measurement lets developers see how well their model manages on different types of devices and allows them to fine-tune it. The new software supports a variety of AI frameworks including OpenVINO on Linux and Windows, and TensorFlow Lite vendor-specific delegates such as Samsung ENN, ArmENN and Qualcomm ENN on Android. This allows it to best fit the software and hardware that engineers and developers currently build their apps on.

Geekbench AI cross-platform benchmarking tool released

Primate Labs, the developer behind Geekbench, has introduced the Geekbench AI app, a cross-platform AI benchmarking tool designed to evaluate the performance of AI workloads using real-world machine learning tasks. Geekbench AI assesses your device's CPU, GPU, and NPU readiness for current and future machine learning applications. Geekbench AI offers a comprehensive suite for testing machine learning, deep learning, and AI-centric workloads. It ensures consistent performance across platforms, helping software developers maintain app performance, enabling hardware engineers to measure architectural improvements, and allowing users to troubleshoot device performance. Supported Platforms: Originally known as "Geekbench ML," the benchmark was rebranded as "Geekbench AI" to align with industry terminology and ensure clarity about its purpose, the company said. Performance Scoring: Geekbench AI provides three overall scores -- Single Precision, Half Precision, and Quantized -- to reflect the diverse AI hardware designs across different devices. This multi-dimensional scoring approach captures AI performance more accurately than a single metric. Speed and Accuracy: The benchmark includes a new accuracy measurement on a per-test basis, highlighting the importance of both speed and accuracy in AI performance. This allows developers to assess the trade-offs between performance and efficiency, particularly when using smaller data types. Frameworks and Datasets: Geekbench AI 1.0 supports various frameworks, including OpenVINO and TensorFlow Lite delegates (Samsung ENN, ArmNN, Qualcomm QNN). It uses extensive datasets that mirror real-world AI use cases, enhancing the accuracy of evaluations. Runtime: Each AI workload runs for at least five iterations and a minimum of one second, ensuring that devices reach their peak performance during testing. Geekbench AI is integrated with the Geekbench Browser, allowing users to compare AI performance across devices and access the latest benchmark results. Geekbench AI Pro comes with automated testing tools, an offline mode, a commercial use license, and basic email support. Geekbench AI 1.0 is available for download on Windows, macOS, Linux, and through the Google Play Store and Apple App Store. Announcing the availability, Geekbench posted:

Geekbench Releases a New AI Benchmark App

Geekbench AI brings three types of scores to showcase accuracy, including Single Precision, Half Precision, and Quantized score. Geekbench ML has been the go-to program for benchmarking the AI capability of processors for most. However, that is going to change with Geekbench AI, a new benchmarking tool released by Primate Labs to accurately determine how good the processor is for AI workloads. Similar to the older Geekbench ML tool, Geekbench AI allows users to test the CPU, GPU, and NPU on several AI tasks. However, the good thing is that Geekbench AI now supports several new frameworks that were not supported earlier. The latest Geekbench AI app adds support for Qualcomm QNN on Windows which means you can now benchmark the Snapdragon X Elite's NPU. Earlier, you could only test its CPU and GPU on AI workloads. Apart from that, it supports the OpenVINO framework (extensively used by Intel), TensorFlow Lite, ONNX, CoreML, Samsung ENN, ArmNN, Qualcomm QNN on Android, and much more. Geekbench AI now also shows three different types of scores which include Single Precision, Half Precision, and Quantized score. The Single Precision score highlights accuracy at performing AI tasks; Half Precision is likely to score more since it's running at a lower precision, and the Quantized score will likely show the score using INT8 data type which is a low-precision format. You can download and start using the new Geekbench AI app right away. It's available on Windows, macOS, Linux, Android, and iOS.

Geekbench AI is the new app to measure the power of artificial intelligence in our devices - Softonic

A very precise measurement of what a device is capable of doing Different hardware will have different performance in terms of artificial intelligence. The power of the CPU, GPU, and neural engines, if any, as well as the amount of available power and various other variables directly affect the ability of a device to efficiently run AI models. How to compare devices with so many variables? With the new Primate Labs app: Geekbench AI. Geekbench, a software already well known for its performance benchmarks, has officially launched Geekbench AI, a suite specifically designed to evaluate the ability of devices in machine learning tasks, deep learning and other AI-focused workloads. The app is capable of analyzing performance under a considerable variety of workloads. For example, we can evaluate performance at different levels of precision: single precision, medium precision, and quantized data. This is an important differentiation because different AI-related tasks require different levels of precision, and not all devices handle these levels in the same way. After running the test, the tool provides us with a score in three parts that reflects this variety, giving us a more complete view of hardware performance. In addition to performance scores, Geekbench AI also includes a measure of accuracy in each test. Regarding the measurement process, Geekbench AI takes advantage of several new frameworks for a more accurate simulation of workloads. Primate Labs has also made sure to include data sets that accurately reflect typical workloads and inputs in real-world usage. A synthetic test, like the ones we have in front of us, does not always accurately reflect what the device has to face in its day-to-day. And yet, the ability of the Geekbench tools developed by Primate Labs to come as close as possible to these use cases has made their apps and scores a benchmark. A benchmark that we can now use to compare the power of different devices in the field of AI. Geekbench AI is available for free on various platforms, including iOS, macOS, Windows, Android, and Linux. Soon, we will also be able to view and compare scores from different devices, so while the European Union sets its sights on AI with this pioneering law in the world and Halide removes all Artificial Intelligence from iPhone photos, we will have a much clearer vision of what they are capable of doing.

Geekbench has an AI benchmark now

The popular benchmarking utility Geekbench has launched a new cross-platform tool to evaluate the performance of devices under AI-heavy workloads. Geekbench AI measures a device's CPU, GPU, and NPU (neural processing unit) to determine how well it can handle machine learning applications. Geekbench developer Primate Labs has been working on the software using the name Geekbench ML, which launched in preview in 2021 but shifted the name to AI for reasons that seem obvious. To explore how different hardware responds to different AI-related tasks, it evaluates performance based on both accuracy and speed, with support for different frameworks, including ONNX, CoreML, TensorFlow Lite, and OpenVINO. It delivers three scores, full precision, half precision, and quantized. Primate Labs says the scores also have an accuracy measurement to evaluate how close a workload's outputs are to the truth, "or how accurately that model can do what it's supposed to do." We'll need more time with devices running on-device AI such as Copilot Plus PCs and all of the new phones to see how performance in real-world tasks correlates with Geekbench AI's numbers. Checking framerates or loading times is one thing -- now we might be checking the accuracy of predictive text, or what generative AI-enabled image editor comes up with.