Rendered at 09:10:56 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
thrownawaysz 14 hours ago [-]
Never really liked Geekbench. Synthetic benchmarks are easy to use and easy to understand but borderline useless. 1000 is better than 990 but it literally doesn't mean anything at all. Just an example: the 2017 Intel Xeon gets more points than an Apple M1 on multicore but everyone knows that they are incomparable. And no one should get a 2017 Intel Xeon because it's scoring higher yet that's the whole point of Geekbench, higher score is better.
Benchmarks like 7Zip compression/decompression, LAME encoding, Blackmagic RAW video encoding, Cinebench, x265 encoding, Blender encoding, Y Cruncher, and any of the built-in video game benchmarks make much more sense than Geekbench.
> the 2017 Intel Xeon gets more points than an Apple M1 on multicore but everyone knows that they are incomparable
Why are they incomparable? If I want to run a highly parallel task surely this tells me which one to use?
> Benchmarks like 7Zip compression/decompression, LAME encoding, Blackmagic RAW video encoding, Cinebench, x265 encoding, Blender encoding, Y Cruncher, and any of the built-in video game benchmarks make much more sense than Geekbench.
Pretty sure the internal tests it uses are similar to these
GeekyBear 13 hours ago [-]
In the past, they have provided information on which workloads they use.
For instance, the file compression workload on version 6:
This workload compresses and decompresses the Ruby 3.1.2 source archive (a 75 MB archive with 9,841 files) using different compression codecs (such as LZ4 and ZSTD). It also verifies the files using the SHA1 (Secure Hash Algorithm 1) function.
> The Clang workload uses the Clang compiler to compile the Lua interpreter, a popular open-source language interpreter.
People really shouldn't call Geekbench a synthetic benchmark anymore, since it has been using real world workloads for years now.
redsky17 12 hours ago [-]
If memory serves, it has been since version 4 or 5 that they started including real workloads. I agree that it shouldn’t be considered synthetic any more.
13 hours ago [-]
aurareturn 10 hours ago [-]
Benchmarks like 7Zip compression/decompression, LAME encoding, Blackmagic RAW video encoding, Cinebench, x265 encoding, Blender encoding, Y Cruncher, and any of the built-in video game benchmarks make much more sense than Geekbench.
I don't think so. First, Geekbench already shows sub scores for different types of applications.[0] Second, nearly no one uses CPU rendering for Cinebench and Blender. They're mostly GPU work. Video encoding/decoding work is mostly done by the media engine or GPU on Apple Silicon. Third, CPU benchmarks tend to correlate. If a CPU is faster in one thing, it's more likely to be faster in another. Therefore, a comprehensive score like what Geekbench and SPEC provide is valuable.
Geekbench is extremely good at showing general CPU performance, especially ST. It's also highly correlated with SPEC at nearly 1:1 in terms of scores as shown by Nuvia before they were purchased by Qualcomm.[1]
While I definitely agree about video decoding, encoding still is a valuable measure.
With inter-frame delivery codecs the hw encoder always focuses on speed, with a lean towards file size as well. It makes sense for most consumer use cases. 15x faster perhaps sure, but you might end up with a file almost 10x larger while still lower quality than a decent CPU encoding.
Also the more off the beaten path of rec709 and rec2020 you go the less certain you can be that the hw encoder supports things.
9 hours ago [-]
ksec 1 hours ago [-]
>Never really liked Geekbench....
As other have stated and perhaps worth pointing out clearly incase people don't get it. If you dislike Geekbench for those reasons, it is highly likely you don't know what Geekbench is testing in the first place.
Personally I think Geekbench is a pretty damn good consumer testing benchmarks. I just wish there is something similar for Server testing PHP, Ruby, JVM, MySQL and Postrges etc.
Can't wait to see all the M1 to M6 results. Along with AMD Zen 6.
RachelF 12 hours ago [-]
Benchmarks are like the SP500 index or IQ measurements: They distil multiple variables into one number. In doing so you lose details, but do get a useful measure.
Yes, they are imperfect, but they do broadly measure how fast a processor is, and can be used for comparison.
conradev 11 hours ago [-]
Borderline useless? Is it just that the numbers are unit-less?
You can extract meaning from them! Find your current computer, find the target computer, calculate the percentage difference. 1000 is 1% better than 990, so it indeed doesn't mean anything at all.
BoingBoomTschak 12 hours ago [-]
What you want exists, it's called SPECint/SPECfp.
aurareturn 10 hours ago [-]
Geekbench 5 and 6 both correlated with SPEC scores at nearly 1:1. SPEC is the industry standard for CPU benchmarks. So in a way, Geekbench does what OP suggests which is a general measurement of performance. If you want to see specific areas, Geekbench shows sub scores.
What compiler is used by GB? Why can't I change it? Imagine an ICC-like scandal. Trusting binary-only benchmarks to compare hardware simply is foolish.
GeekyBear 12 hours ago [-]
Since the loss of Anandtech, have any of the other tech websites started publishing SPEC results for systems they test?
Phoronix, I've at least seen it on Epyc Reviews iirc.
wmf 12 hours ago [-]
AV1, Opus, Whisper, Jolt Physics... did you read the article? Are these not real?
dijit 13 hours ago [-]
a 2017 Xeon can absolutely smoke an M1.. (double the FP64 units, AVX-512; huge per core caches, boatloads of I/O lanes and RAM...)
.. but it chugs power like it's not even funny.
I do like synthetic benchmarks as one input signal when evaluating a CPU; otherwise we go back to stupid numbers like frequency... which also never made sense because they had 1.8GHz AMD Athlon CPUs outperforming 2.4GHz Pentium 4's in 2002-3...
Doing a lot of real-world application testing is good, but there's always some you miss, and the worst part about it is that people start to lose patience when assessing.. How is a cinebench comparing to doing gamedev in UE5?
How is doing gamedev in UE5 comparing to doing gamedev in Snowdrop?
How does doing gamedev in Snowdrop compare to doing virtualisation (different CPUs handling that particular task better than others).
there's so many dimensions that it will always be true that there's not enough testing.
I'm not saying we shouldn't have rigorous testing like you say, in fact, what I'm actually saying is that "single number→would you like to go deeper" is a better pipeline than an excel spreadsheet that goes on for 10 pages (which still skips a bunch of nuance) and still better than one that goes on for 500 pages with a broad spectrum of topics.
The reason I'm so bitter about this is because outlets like LTT and Gamers Nexus seem to choose their games randomly (based on popularity I guess?) and so all the titles I worked on kinda got smeared by my "company"- but our games used hugely different game engines that work differently (Dunia vs Snowdrop handle CPUs... VERY differently. Snowdrop is built for multi-core. Anvil works best on a single core, and Dunia has trouble going passed 4 cores- so on large systems with multiple NUMA zones the performance is.. worse.
So the "battery of tests" always gave us a shitty score and reviewers just moved on believing themselves to be comprehensive arbiters of truth regarding performance and giving verdicts regarding the performance of games as a broad topic (which people take and don’t dig deeper themselves) and without regard to the utility of what they’re saying (500+ fps on CSGO isn't even renderable for example).
People might have been better served by looking broadly at how powerful the CPUs actually are and then digging in for their use case once they whittle a few down, because then also: developers will actually make software that tries to use the features on offer instead of assuming people will just buy CPUs that work better on their workloads.
It's a kind of "to big to fail" mentality where incumbents start being able to direct CPU sales based on those CPUs being optimised for their workloads. It's self-reinforcing.
NSUserDefaults 13 hours ago [-]
Slightly offtopic but where are we on Quake 1 timedemo with current hardware? Is it sub-second on modern machines?
RachelF 12 hours ago [-]
The original timedemo was how fast Sopwith game from 1984 crashed the plane:
I've run it on a couple Macs, and it looks like one of the issues they're trying to fix is poor multi-core scaling that affected Geekbench 6 on bigger multicore systems (with 32, 64, or 128+ CPU cores). I'll hopefully get time to test on my Ampere Altra Max systems later, to see how scaling works.
Geekbench 5 is famously much better than Geekbench 6 for benchmarking these workstation-class CPUs (though many other benchmarks are better still, especially if you can grab the source code and compile them yourself).
I still like having a set of benchmarks that run across Android, iOS, Windows, macOS, Linux, and on Arm, X86, RISC-V, etc... even if imperfect, it's a point of reference to get a general feel. And the single core scores are a great representation of a 'feel' against baseline in day-to-day use.
wtallis 14 hours ago [-]
Please stop framing the multi core scaling issue as a matter of the benchmark being good or bad. Geekbench 5 scores scale better than Geekbench 6 scores because Geekbench 5's multicore test runs N independent copies of the same workload while Geekbench 6 runs one workload that has to be split across the available cores, with non-zero coordination between threads.
The Geekbench 5 approach of pretending Amdahl's Law doesn't exist is sometimes a valid benchmarking strategy, but generally is the wrong choice for benchmarking consumer workloads and devices, and that's what Geekbench is ostensibly targeting.
The fact that Geekbench 6 scores don't increase linearly with the addition of more CPU cores is not a weakness of the benchmark, it's the benchmark demonstrating an important real-world effect.
The change that Geekbench 7 makes to exclude some subtests from the multicore suite entirely will definitely have the effect of making the overall multicore score scale better with the addition of more cores, but most of the audience for those scores is going to miss out on the fact that the multicore test now measures a narrower range of tasks than the single-core test suite.
osti 14 hours ago [-]
I agree. For me personally I mostly only care about single thread geekbench variant, I believe it's an excellent proxy for general performance of a CPU. Multi thread geekbench (or other benchmarks) for most purposes and for most people, it's kinda useless. You just need to know that you have a quite a few cores on your computer and that it will have enough concurrency for what you do. But single thread will make whatever you do actually faster.
GeekyBear 13 hours ago [-]
They do mention that the change is meant to reflect how real world programs handle various types of workloads.
> In Geekbench 7, a workload only runs in multi-threaded mode if the task it models actually runs multi-threaded in real applications. For example, the HTML5 Browser test isn’t included in the multi-threaded suite because web browsers are single-threaded (or lightly threaded).
wtallis 13 hours ago [-]
Yes, they have a valid reason for the change. But I think it's still making a tradeoff, and has a clearly identifiable downside because the content of the multi-core test suite is no longer as closely aligned with the content of the single-core test suite.
(One could just as easily argue that the single-core test suite ought to exclude any task typically done with multiple threads in real-world usage.)
jchw 14 hours ago [-]
Hmm, I don't fully agree. There are a whole lot of embarassingly parallel processes that people interested in many core workstations are interested in; compiling large C++ codebases for example tends to scale very well since each translation unit is independent. This is still going to not quite give you linear scaling so it's definitely still valuable to know what that looks like.
wtallis 13 hours ago [-]
Geekbench is not intended for benchmarking large workstations. Look at how the web site actually describes the benchmark. It's for phones and laptops and similar devices. A test mode that uses a machine in a way that is only relevant to high-end workstation or server use would be actively harmful to the suitability of Geekbench for its intended audience of consumer hardware and consumer users.
aurareturn 10 hours ago [-]
No idea why you're downvoted. Geekbench's focus is consumer workloads. In consumer workloads, having 20 cores or more is nearly useless. Most normal people will not need more than 8 in 2026. You're better off with a few fast cores over many slow cores.
Geekbench isn't a server CPU benchmark that stresses 128 core CPUs.
jchw 9 hours ago [-]
I dunno why people are downvoting at all. I don't get what nerve this is striking so hard.
The real truth is that if consumers don't benefit from knowing how well multicore workloads scale, then they can just look only at the single-core score. Skewing it on purpose by including things that genuinely only use ~1 core doesn't seem to serve to add value but only obfuscate the truth about how well a CPU is able to scale on multicore workloads, which does matter sometimes (like with modern video games.)
Yes this means that 128 core Xeon servers will maybe look especially good, but idk, what consumer hardware has 128 cores? Consumers are not considering a Mac Mini versus a fucking NVIDIA Vera rack, you know?
I fully accept that you could view it either way, but if you're going to have a multicore score at all, in my opinion, it may as well actually show you what multicore workloads look like. It is OK that consumers can't necessarily understand how this translates. If all they ever do is single core workloads, they will never have any benefit to looking at the multicore workload score for any Geekbench score on any Geekbench version, because otherwise the multicore score overstates the performance, whereas if they were to look at a version that has multicore including ~single threaded workloads, it would understate the multicore scaling.
Benchmarks are necessarily imperfect no matter how you shake it. For example, it is impossible for a general compute benchmark to show how much better the UX of two cores is over having just one core that is a bit faster.
Though, now that I say all of this, I get the strange feeling the nerve it's striking is fanboyism. What else would make people care this much, but perhaps the hardware vendor they are fans of showing up higher in the numbers? Well it ain't me. I couldn't care less how Apple and AMD and Intel compare in the Geekbench scores. Could be wrong, but somehow this just feels like a good bet.
aurareturn 9 hours ago [-]
Given that Geekbench is a consumer workload benchmark, it makes sense to factor in Amdahl's Law. Period.
Now the question is whether having a 3rd CPU score for embarrassingly parallel work makes sense for Geekbench. It probably does since many professionals also use Geekbench nowadays.
wmf 7 hours ago [-]
If they had an embarrassingly parallel test, Intel would be exclusively promoting that misleading number just like they do with Cinebench. Let's not create even more totally unrepresentative benchmarks.
aurareturn 6 hours ago [-]
I don't think Intel and AMD promote Cinebench as they used to back in the 2017-2023. Cinebnech 2024 actually heavily favors Apple.
jchw 8 hours ago [-]
> Given that Geekbench is a consumer workload benchmark, it makes sense to factor in Amdahl's Law. Period.
To be honest, they are factoring in Amdahl's law, because they're actually not only including embarrassingly parallel workloads, they're just not including ones where the speedup factor is roughly 0. Frankly I should have acknowledged this in my initial response.
> Now the question is whether having a 3rd CPU score for embarrassingly parallel work makes sense for Geekbench. It probably does since many professionals also use Geekbench nowadays.
Including workloads that are almost entirely single-threaded does not tell you anything about the "multi-core" performance of the processor. It just biases the multi-core score to be lower, period.
It is true that even without that, the multi-core score will still be biased towards the rough average speedup factor of all of the workloads combined, but that's just the nature of an aggregate benchmark. If you wanted to know how it would perform on a specific task, you'd need a specific benchmark anyways. If they've done a good job, the multi-core scores should still be useful for comparison across processors for your average multi-core workload.
I think two things are true:
- If consumers almost only ever use single threaded workloads, then the multi-core score will always be misleading to them if it is anything above what the single core score is. They should be paying more attention to the single-core score.
- If consumers do have multi-threaded workloads, then a multi-core score that averages in a bunch of tasks with no multi-core speedup will also not be representative for them, as it will understate the performance benefits they can see.
If you wanted a more useful third number it would be a weighted overall score, but what is the objectively correct weight between single and multi-core? Even for consumers this is a challenging number to come up with.
But I know one thing: it is almost certainly not whatever weight you just happen to wind up with by just arbitrarily including whatever number of single-threaded tasks your benchmark happens to have in a multi-core benchmark score.
jchw 13 hours ago [-]
Well, apparently they must not agree.
osti 14 hours ago [-]
You can use geekbench 5 in that case. But given that they deprecated that, it might be harder to compare to others.
zamadatix 14 hours ago [-]
It bothers me because there is no reason it can't just give 3 results: single, mixed/low thread, all core and yet there is often debate on which 2 numbers in such benchmarks instead.
But, of all of the 2-pick-only options, choosing a custom mix and calling it multi-core requires guessing your actual workload on both the content of the mixed test suite as well as the scaling profile of the CPU to reason with. On the other hand, just testing "single core" and "all core" at least only requires you to guess based on what you think the scaling profile of the CPU is.
Nothing beats just testing your actual workload, but that doesn't mean all other ways of testing have to be equally good.
wtallis 13 hours ago [-]
> But, of all of the 2-pick-only options, choosing a custom mix and calling it multi-core requires guessing your actual workload on both the content of the mixed test suite as well as the scaling profile of the CPU to reason with.
Yes, that's pretty much the whole point of Geekbench: to offer non-experts a way to measure CPU performance and produce scores that are relevant to a specific class of devices and users and use cases, rather than artificially inflated.
Benchmarking experts can already use something like SPEC, which does offer single-thread and both types of multi-thread test modes (SPECrate and SPECspeed).
wmf 12 hours ago [-]
Honestly I think the world would be better off if Geekbench just refused to run on any machine over 28C. Then people could benchmark their workstations using... workstation benchmarks.
cosmotic 14 hours ago [-]
How does this handle hardware encode/decode of media? Tje announcement says it encodestdecodes AV1 but doesnt indicate if it leverages hardware codecs.
snvzz 5 hours ago [-]
The CUDA addition (a vendor-specific API) is definitely a mistake.
kittikitti 13 hours ago [-]
I was hoping for a benchmark that specifically tests AI capabilities like tokens per second. I didn't even try Geekbench for my recent build because of it since none of the results would be relevant. I guess this is mainly a benchmark for video games and social media usage.
Benchmarks like 7Zip compression/decompression, LAME encoding, Blackmagic RAW video encoding, Cinebench, x265 encoding, Blender encoding, Y Cruncher, and any of the built-in video game benchmarks make much more sense than Geekbench.
Some good benchmark options here https://hwbot.org/benchmarks
https://www.pcgamingwiki.com/wiki/List_of_games_with_built-i...
Why are they incomparable? If I want to run a highly parallel task surely this tells me which one to use?
> Benchmarks like 7Zip compression/decompression, LAME encoding, Blackmagic RAW video encoding, Cinebench, x265 encoding, Blender encoding, Y Cruncher, and any of the built-in video game benchmarks make much more sense than Geekbench.
Pretty sure the internal tests it uses are similar to these
For instance, the file compression workload on version 6:
This workload compresses and decompresses the Ruby 3.1.2 source archive (a 75 MB archive with 9,841 files) using different compression codecs (such as LZ4 and ZSTD). It also verifies the files using the SHA1 (Secure Hash Algorithm 1) function.
https://www.geekbench.com/doc/geekbench6-cpu-workloads.pdf
CPU: https://www.geekbench.com/doc/geekbench7-cpu-workloads.pdf
GPU: https://www.geekbench.com/doc/geekbench7-gpu-workloads.pdf
> The Clang workload uses the Clang compiler to compile the Lua interpreter, a popular open-source language interpreter.
People really shouldn't call Geekbench a synthetic benchmark anymore, since it has been using real world workloads for years now.
Geekbench is extremely good at showing general CPU performance, especially ST. It's also highly correlated with SPEC at nearly 1:1 in terms of scores as shown by Nuvia before they were purchased by Qualcomm.[1]
[0]https://browser.geekbench.com/v6/cpu/18801240 scroll down
[1]https://medium.com/silicon-reimagined/performance-delivered-...
With inter-frame delivery codecs the hw encoder always focuses on speed, with a lean towards file size as well. It makes sense for most consumer use cases. 15x faster perhaps sure, but you might end up with a file almost 10x larger while still lower quality than a decent CPU encoding. Also the more off the beaten path of rec709 and rec2020 you go the less certain you can be that the hw encoder supports things.
As other have stated and perhaps worth pointing out clearly incase people don't get it. If you dislike Geekbench for those reasons, it is highly likely you don't know what Geekbench is testing in the first place.
Personally I think Geekbench is a pretty damn good consumer testing benchmarks. I just wish there is something similar for Server testing PHP, Ruby, JVM, MySQL and Postrges etc.
Can't wait to see all the M1 to M6 results. Along with AMD Zen 6.
Yes, they are imperfect, but they do broadly measure how fast a processor is, and can be used for comparison.
You can extract meaning from them! Find your current computer, find the target computer, calculate the percentage difference. 1000 is 1% better than 990, so it indeed doesn't mean anything at all.
Here's research done by Nuvia before they were bought by Qualcomm: https://medium.com/silicon-reimagined/performance-delivered-...
https://benchview.hjc.im/
https://blog.hjc.im/
Thanks
.. but it chugs power like it's not even funny.
I do like synthetic benchmarks as one input signal when evaluating a CPU; otherwise we go back to stupid numbers like frequency... which also never made sense because they had 1.8GHz AMD Athlon CPUs outperforming 2.4GHz Pentium 4's in 2002-3...
Doing a lot of real-world application testing is good, but there's always some you miss, and the worst part about it is that people start to lose patience when assessing.. How is a cinebench comparing to doing gamedev in UE5?
How is doing gamedev in UE5 comparing to doing gamedev in Snowdrop?
How does doing gamedev in Snowdrop compare to doing virtualisation (different CPUs handling that particular task better than others).
there's so many dimensions that it will always be true that there's not enough testing.
I'm not saying we shouldn't have rigorous testing like you say, in fact, what I'm actually saying is that "single number→would you like to go deeper" is a better pipeline than an excel spreadsheet that goes on for 10 pages (which still skips a bunch of nuance) and still better than one that goes on for 500 pages with a broad spectrum of topics.
The reason I'm so bitter about this is because outlets like LTT and Gamers Nexus seem to choose their games randomly (based on popularity I guess?) and so all the titles I worked on kinda got smeared by my "company"- but our games used hugely different game engines that work differently (Dunia vs Snowdrop handle CPUs... VERY differently. Snowdrop is built for multi-core. Anvil works best on a single core, and Dunia has trouble going passed 4 cores- so on large systems with multiple NUMA zones the performance is.. worse.
So the "battery of tests" always gave us a shitty score and reviewers just moved on believing themselves to be comprehensive arbiters of truth regarding performance and giving verdicts regarding the performance of games as a broad topic (which people take and don’t dig deeper themselves) and without regard to the utility of what they’re saying (500+ fps on CSGO isn't even renderable for example).
People might have been better served by looking broadly at how powerful the CPUs actually are and then digging in for their use case once they whittle a few down, because then also: developers will actually make software that tries to use the features on offer instead of assuming people will just buy CPUs that work better on their workloads.
It's a kind of "to big to fail" mentality where incumbents start being able to direct CPU sales based on those CPUs being optimised for their workloads. It's self-reinforcing.
https://en.wikipedia.org/wiki/Sopwith_(video_game)
Geekbench 5 is famously much better than Geekbench 6 for benchmarking these workstation-class CPUs (though many other benchmarks are better still, especially if you can grab the source code and compile them yourself).
I still like having a set of benchmarks that run across Android, iOS, Windows, macOS, Linux, and on Arm, X86, RISC-V, etc... even if imperfect, it's a point of reference to get a general feel. And the single core scores are a great representation of a 'feel' against baseline in day-to-day use.
The Geekbench 5 approach of pretending Amdahl's Law doesn't exist is sometimes a valid benchmarking strategy, but generally is the wrong choice for benchmarking consumer workloads and devices, and that's what Geekbench is ostensibly targeting.
The fact that Geekbench 6 scores don't increase linearly with the addition of more CPU cores is not a weakness of the benchmark, it's the benchmark demonstrating an important real-world effect.
The change that Geekbench 7 makes to exclude some subtests from the multicore suite entirely will definitely have the effect of making the overall multicore score scale better with the addition of more cores, but most of the audience for those scores is going to miss out on the fact that the multicore test now measures a narrower range of tasks than the single-core test suite.
> In Geekbench 7, a workload only runs in multi-threaded mode if the task it models actually runs multi-threaded in real applications. For example, the HTML5 Browser test isn’t included in the multi-threaded suite because web browsers are single-threaded (or lightly threaded).
(One could just as easily argue that the single-core test suite ought to exclude any task typically done with multiple threads in real-world usage.)
Geekbench isn't a server CPU benchmark that stresses 128 core CPUs.
The real truth is that if consumers don't benefit from knowing how well multicore workloads scale, then they can just look only at the single-core score. Skewing it on purpose by including things that genuinely only use ~1 core doesn't seem to serve to add value but only obfuscate the truth about how well a CPU is able to scale on multicore workloads, which does matter sometimes (like with modern video games.)
Yes this means that 128 core Xeon servers will maybe look especially good, but idk, what consumer hardware has 128 cores? Consumers are not considering a Mac Mini versus a fucking NVIDIA Vera rack, you know?
I fully accept that you could view it either way, but if you're going to have a multicore score at all, in my opinion, it may as well actually show you what multicore workloads look like. It is OK that consumers can't necessarily understand how this translates. If all they ever do is single core workloads, they will never have any benefit to looking at the multicore workload score for any Geekbench score on any Geekbench version, because otherwise the multicore score overstates the performance, whereas if they were to look at a version that has multicore including ~single threaded workloads, it would understate the multicore scaling.
Benchmarks are necessarily imperfect no matter how you shake it. For example, it is impossible for a general compute benchmark to show how much better the UX of two cores is over having just one core that is a bit faster.
Though, now that I say all of this, I get the strange feeling the nerve it's striking is fanboyism. What else would make people care this much, but perhaps the hardware vendor they are fans of showing up higher in the numbers? Well it ain't me. I couldn't care less how Apple and AMD and Intel compare in the Geekbench scores. Could be wrong, but somehow this just feels like a good bet.
Now the question is whether having a 3rd CPU score for embarrassingly parallel work makes sense for Geekbench. It probably does since many professionals also use Geekbench nowadays.
To be honest, they are factoring in Amdahl's law, because they're actually not only including embarrassingly parallel workloads, they're just not including ones where the speedup factor is roughly 0. Frankly I should have acknowledged this in my initial response.
> Now the question is whether having a 3rd CPU score for embarrassingly parallel work makes sense for Geekbench. It probably does since many professionals also use Geekbench nowadays.
Including workloads that are almost entirely single-threaded does not tell you anything about the "multi-core" performance of the processor. It just biases the multi-core score to be lower, period.
It is true that even without that, the multi-core score will still be biased towards the rough average speedup factor of all of the workloads combined, but that's just the nature of an aggregate benchmark. If you wanted to know how it would perform on a specific task, you'd need a specific benchmark anyways. If they've done a good job, the multi-core scores should still be useful for comparison across processors for your average multi-core workload.
I think two things are true:
- If consumers almost only ever use single threaded workloads, then the multi-core score will always be misleading to them if it is anything above what the single core score is. They should be paying more attention to the single-core score.
- If consumers do have multi-threaded workloads, then a multi-core score that averages in a bunch of tasks with no multi-core speedup will also not be representative for them, as it will understate the performance benefits they can see.
If you wanted a more useful third number it would be a weighted overall score, but what is the objectively correct weight between single and multi-core? Even for consumers this is a challenging number to come up with.
But I know one thing: it is almost certainly not whatever weight you just happen to wind up with by just arbitrarily including whatever number of single-threaded tasks your benchmark happens to have in a multi-core benchmark score.
But, of all of the 2-pick-only options, choosing a custom mix and calling it multi-core requires guessing your actual workload on both the content of the mixed test suite as well as the scaling profile of the CPU to reason with. On the other hand, just testing "single core" and "all core" at least only requires you to guess based on what you think the scaling profile of the CPU is.
Nothing beats just testing your actual workload, but that doesn't mean all other ways of testing have to be equally good.
Yes, that's pretty much the whole point of Geekbench: to offer non-experts a way to measure CPU performance and produce scores that are relevant to a specific class of devices and users and use cases, rather than artificially inflated.
Benchmarking experts can already use something like SPEC, which does offer single-thread and both types of multi-thread test modes (SPECrate and SPECspeed).
It focuses more on ML than LLMs though.
If you ever want a cheeky laugh ask your llm of choice to write a satirical Userbenchmark amd review.