Dissecting Intel's EPYC Benchmarks: Performance Through the Lens of Competitive Analysisby Johan De Gelas & Ian Cutress on November 28, 2017 9:00 AM EST
- Posted in
- Xeon Platinum
- EPYC 7601
Although the AMD EPYC is definitely a worthy contender in the server space, AMD's technical marketing of the new CPU has been surprisingly absent, as the company not published any real server benchmarks. The only benchmarks published were SPEC CPU and Stream, with AMD preferring for its partners and third parties to promote performance. And, as our long-time readers know, while the SPEC CPU benchmarks have their merits and many people value them, they are a very poor proxy of most server workloads.
In every launch, we expect companies to offer an element of competitive analysis, often to show how their platform is good or better than the rest. At the launch of Intel's latest Xeon-SP platform, analysis to EPYC was limited to a high-level, as the systems were not as freely available as expected. AMD was able to do so on Broadwell-E at the time of the EPYC announcement because it was out and available - Intel wasn't able to do it on EPYC because AMD were several months away from moving it from a cloud-only ramp up program. This is partly the effect of AMD's server market implementation and announcement roadmap, although it didn't stop Intel from hypothesising about the performance deficits in ways that caught the attention of a number of online media.
Throughout all of this, AMD could not resist but to continue to tell the world that the "EPYC SoC Sets World Records on SPEC CPU Benchmarks". In the highly profitable field that is server hardware, this could not be left unanswered by Intel, who responded that the Intel Xeon Scalable has great "momentum" with no less than 110 performance records to date.
Jumping to the present time, in order to to prove Xeon-SP dominance over the competition, Intel's data center engineering group has been able to obtain a few EPYC systems and has started benchmarking. This benchmarking, along with justifications of third-party verification, was distributed to the small set of Xeon-SP launch reviewers as a guide, to follow up on that high-level discussion some time ago. The Intel benchmarking document we received had a good amount of detail however, and the conference call we had relating to it was filled with some good technical tidbits.
Our own benchmarks showed that the EPYC was a very attractive alternative in some workloads (Java applications), while the superior mesh architecture makes Intel's Xeon the best choice in other (Databases for example).
A Side Note About SPEC
A number of these records were achieved through SPEC. As mentioned above, while SPEC is a handy tool for comparing the absolute best tweaked peak performance of the hardware underneath, or if the system wants to be analysed close to the metal because of how well known the code base is, but this has trouble transferring exactly to the real world. A lot of time the software within a system will only vaguely know what system it is being run on, especially if that system is virtualised. Sending AVX-512 commands down the pipe is one thing, but SPEC compilation can be tweaked to make sure that cache locality is maintained whereas in the real-world, that might not be possible. SPEC says a lot about the system, but ultimately most buyers of these high-end systems are probing real-world workloads on development kits to see what their performance (and subsequent scale-out performance) might be.
For the purposes of this discussion, we have glossed over Intel's reported (and verified over at SPEC.org) results.
Pricing Up A System For Comparison
Professionals and the enterprise market will mention, and quite rightly, that Intel has been charging some heavy premiums with the latest generation, with some analysts mentioning a multiple jump up in pricing even for large customers, making it clear that the Xeon enterprise CPU line is their bread and butter. Although Intel's top-end Xeon Platinum 8180 should give the latest EPYC CPU a fit of trouble thanks to its 28 Skylake-SP cores running at 2.5 to 3.8 GHz, the massive price tag ($10009 for the standard version, $13011 for the high-memory model) made sure that Intel's benchmarking team had no other choice than also throwing in a much more modest Xeon Platinum 8160 (24 cores at 2.1 - 3.7 GHz, $4702k) as well as the Xeon Gold 6148 (20 cores at 2.4-3.7 GHz, $3072).
|Release Date||Early Q3, 2017||Late Q2, 2017*|
|Microarchitecture||Skylake-SP with AVX-512||Zen|
|Process Node||Intel 14nm (14+)||GloFo 14nm|
|Cores / Threads||28 / 56||24 / 48||20 / 40||32 / 64|
|Base Frequency||2.5 GHz||2.1 GHz||2.4 GHz||2.2 GHz|
|Turbo||3.8 GHz||3.7 GHz||3.7 GHz||3.2 GHz|
|L2 Cache||28 MB||24 MB||20 MB||16 MB|
|L3 Cache||38.5 MB||33.0 MB||27.5 MB||64 MB|
|TDP||205 W||150 W||150 W||180 W|
|PCIe Lanes||48 (Technically 64 w/ Omni-Path Versions)||128|
|DRAM||6-channel DDR4||8ch DDR4|
|Max Memory||768 GB||2048 GB|
As a result of this pricing, one of the major humps for Intel in any comparison will be performance per dollar. In order to demonstrate that systems can be equivalent, Intel offered up this comparison from a single retailer. Ideally Intel should have offered multiple configurations options for this comparison, given that a single retailer can intend for different margins on different sets of products (or have different levels of partnership/ecosystem with the manufacturers).
Even then, price parity could only be reached by giving the Intel system less DRAM. Luckily this was the best way to configure the Intel based system anyway. We can only guess how much the benchmarking engineers swore at the people who set the price tags: "this could have been so much easier...". All joking apart, the document we received had a good amount of detail, and similar to how we looked into AMD's benchmarking numbers at their launch, we investigated Intel's newest benchmark numbers as well.
Post Your CommentPlease log in or sign up to comment.
View All Comments
piesquared - Tuesday, November 28, 2017 - linkAnd the hilarity continues. So AMD posts in house benchmarks and the crowd goes: Derp, these are AMD supplied benchmarks, best wait for third party benchmarks.
Intel posts in house benchmarks and the crowd goes: Wow awesome dude, that's the shitsors! Who needs third party benchmarks, AMD should post more in house benchmarks. derp derp
tamalero - Tuesday, November 28, 2017 - linkGuerrilla marketing at its finest? The hilarity is that when Intel was dominating.. they never mentioned intel nor they needed.
Now that AMD has a compelling product. They suddenly started doing "comparisons" left and right and claiming how bad "glue" is in AMD cpus (while ignoring the drama bout using cheap TIM instead of solder)
bmf614 - Tuesday, November 28, 2017 - linkEpyc really hasnt even launched yet. Try buying a Dell or HP with Epyc. Nope.
supdawgwtfd - Wednesday, November 29, 2017 - linkIt's launched. Demand has outstripped supply. They are now starting to get on top of it.
Maybe stop being an Intel biased dickhead and go look at what is actually happening?
Topweasel - Wednesday, November 29, 2017 - linkYeah, I don't get it. I mean even Ryzen mobile launched well before we saw it. Eypc announcement early was important to build up demand with OEM's. Something that wasn't as important with a consumer product that needed announcement with availability. EPYC's announcement wasn't for the end purchaser. Both these need long testing periods and seed supply. Epyc then has ODM builds for cloud services that they have supply. Ryzen mobile launched when OEM's had products to ship. EPYC launched when they products to ship to manufacturers. When those Manufacturers offered EPYC depends completely on their development cycle.
Johan Steyn - Monday, December 18, 2017 - linkHaha so true
Ninhalem - Tuesday, November 28, 2017 - linkCan we get ANSYS Structural or Comsol benchmarks for the HPC sections? Building machines using Xeons for these applications is beyond expensive for engineering design on fixed price contracts.
anactoraaron - Tuesday, November 28, 2017 - linkNo, because AT didn't test anything here. They are just 'publishing' Intel's benchmarks and calling it an 'analysis'.
Doesn't this qualify for the #ad in the title?
Ryan Smith - Wednesday, November 29, 2017 - linkTo throw some context in here, the purpose of this article isn't to publish Intel's benchmarks. Rather, it's commentary on what has been a very unusual situation.
Up until now, neither AMD nor Intel have engaged in any serious Skylake Xeon vs. Zen EPYC technical marketing.
"AMD's technical marketing of the new CPU has been surprisingly absent, as the company not published any real server benchmarks. The only benchmarks published were SPEC CPU and Stream, with AMD preferring for its partners and third parties to promote performance"
This despite the fact that AMD and Intel's server products haven't been competitive like this in nearly a decade. Normally you'd expect there to be case studies flying out left and right, which has not been the case. And it's especially surprising since, as the underdog, AMD needs to claw back lost ground.
Consequently, Intel's own efforts are, to date, the first efforts by a server vendor to do a comprehensive set of benchmarks over a range of use cases. And let's be clear here: this is Intel doing this for Intel's own benefit. Which is why we've already previously reviewed the two CPUs, as have other 3rd party groups.
Still, I think it's very interesting to look at what Intel has chosen to represent, and what their numbers show. Intel has more resources than pretty much everyone else when it comes to competitive analysis, after all. So their choices and where they show themselves falling behind AMD says a lot about the current situation.
(And no, this doesn't quality for #ad as Intel hasn't paid us. That's not how this works; that's not how any of this works)
deltaFx2 - Wednesday, November 29, 2017 - link@Ryan Smith: "Up until now, neither AMD nor Intel have engaged in any serious Skylake Xeon vs. Zen EPYC technical marketing." I think that's largely because the market is different from a decade ago. Hyperscalers do their own testing and aren't swayed by Intel's or AMD's whitepapers. They do their own thing. There are still many companies that buy and maintain their own servers, but my understanding is that this market is shrinking or at least not growing. Cloud is where the money is, and they know what they want. I don't think AMD is trying to go after enterprise this time around (I'm sure they'll take their business but the main target seems to be hyperscalers. The CCX, MCM, large memory footprint etc all point to them saying we'll target scale-out as opposed to scale-up. AMD does quite well in scale-out, while taking a hit in scale-up.).
Also, AMD might still be in the process of doing minor firmware tweaks as evidenced by tier-1 server availability (HP/Dell) coming online only end of Q4.