Initially released in beta under the name Geekbench ML (for machine learning), in December 2023, the tool made it possible to compare between Mac and iPhone. Now, under the name Geekbench AI, the creators claim to radically increase the app’s ability to usefully measure and compare performance.
“With edition 1. 0, we believe Geekbench Al has reached a point of reliability that allows developers to hopefully integrate it into their workflows,” the company wrote in a blog post, “[and] many big names like Samsung and Nvidia are already using it. “
Geekbench has long provided separate scores for single-core and multi-core device performance. In the case of Geekbench AI, it measures 3 other types of workload.
“Geekbench AI presents its summary of a diversity of workload tests performed with single-precision data, medium-precision data, and quantified data,” the company continues, “covering a use used by developers in terms of accuracy and finality in AI systems.
The company emphasizes that the goal is to produce comparable metrics that reflect the use of AI in the real world, no matter how broad it may be.
Geekbench AI 1. 0 can be obtained directly from the developer. As with previous versions of Geekbench, the tool is free for most users and works on macOS, iOS, Android, and Windows.
There is a paid edition called Geekbench AI Pro. It allows developers to keep their personal scores or automatically upload them to a public site.
Looking at the entries so far, it turns out that the Neural Engine works very well for quantized benchmark editing and therefore for half-precision. But it turns out that Apple’s hardware in general is pretty poor for precision-only editing (the most productive hardware is the M3 Max’s GPU). I’m glad to be corrected, but I guess that means Apple’s hardware is smart for on-device inference, but poor for educational models. I wonder how much more CoreML optimization Apple wants that needs stronger GPUs. . .