Evaluating Futuremark's Servermark VDI on the Supermicro SYS-5028D-TN4T
by Ganesh T S on September 1, 2016 8:00 AM EST- Posted in
- Benchmarks
- Futuremark
- Supermicro
- Servers
- Xeon-D
Standardized benchmarks with industry-wide acceptance are a must for evaluating computing systems. These benchmarks may evaluate the system as a whole, or target specific aspects. Vendors such as Futuremark and BAPCo have various offerings for both consumer and business-use PCs. However, the market for such programs in the server space is largely untapped. Futuremark has been working on Servermark for more than two years now, and they finally felt it was stable enough to let us test drive one of their recent beta versions. Servermark will have a number of sub-categories, but, our focus was on VDI (virtual desktop infrastructure) - a way to determine how many virtual machines a server can support while providing an acceptable level of performance for all users.
Introduction and System Setup
Modern servers are very powerful and come with virtualization capabilities. This capability allows the work done by multiple physical machines to be be consolidated into a single physical server using the concept of 'virtual machines' (VMs). Virtualization brings about a host of benefits to system administrators. VMs can be used in a headless manner (for example, as a mail server or IoT controller), or as a virtual desktop with a thin client front-end. The latter can enable businesses to provide basic low-cost / low-power computers at the employee's desk. Business applications which require plenty of processing power can be processed on a virtual desktop running on the VM host / server. This aspect is the focus of Servermark VDI and our investigation today.
Prior to going into the details of Servermark VDI and its various features, let us take a look at our test machine for the evaluation. The Supermicro SYS-5028D-TN4T is a very popular server in the mini-tower form factor amongst home-lab enthusiasts. We already covered it in our Xeon-D review. Our unit (from last year) came with a Xeon-D 1540 (8C / 16T @ 2.00 GHz) SoC and four hot-swappable 3.5" drive bays. The following table summarizes the specifications of the Supermicro SYS-5028D-TN4T used in this review.
| Supermicro SYS-5028D-TN4T Specifications | |
| Form Factor | 4-bay mini-tower / mITX motherboard |
| Platform | Intel Xeon D-1540 |
| CPU Configuration | 8C/16T Broadwell-DE x86 Cores 12 MB L2, 45W TDP 2.0 GHz (Turbo: 2.6 GHz) |
| SoC SATA Ports | 6x SATA III (for four hot-swap bays and two fixed drives) |
| Additional SATA Ports | None |
| I/O Ports | 2x USB 3.0 2x USB 2.0 1x D-Sub 2x RJ-45 GbE LAN 2x RJ-45 10GBASE-T LAN 1x RJ-45 IPMI LAN 1x COM1 Serial Port |
| Expansion Slots | 1x PCIe 3.0 x16 (Unused) 1x M.2 2242 / 2280 PCIe 3.0 x4 |
| Memory | 4x 16GB DDR4-2133 ECC RDIMM Micron 36ASF2G72PZ-2G1A2 15-15-15-36 |
| OS Drive | Samsung SSD 840 EVO (120 GB; 2.5in SATA 6Gb/s; 19nm; TLC) |
| Data Drives | 4x Crucial MX200 (500 GB; 2.5in SATA 6Gb/s; 16nm; MLC) |
| Chassis Dimensions | 240mm x 210mm x 279mm |
| Power Supply | 250W Internal PSU |
We used four Crucial MX200 500GB SSDs in the four hot-swap bays and configured them in RAID-0 with Intel's BIOS-assisted software RAID.
Futuremark's Servermark VDI came by default with scripts for a VMWare-based virtualization setup. However, being personally used to Hyper-V, I decided to perform the evaluation using Windows Server 2016 TP5.
The RAID-0 volume with the Crucial MX200 SSDs was used as the VM data store. A switch was setup for the VMs and they were uplinked to the main network using one of the 10GBASE-T LAN ports. For home lab setups, this is acceptable, though we must make a note here that the Xeon-D 1540 doesn't support SR-IOV and ends up increasing CPU load for this type of internal switch setup. The newer version of the SYS-5028D-TN4T comes with the Xeon-D 1541. It has better I/O virtualization support, and doesn't have this problem.
In addition to the Hyper-V role, Windows Server 2016 TP5 was also configured with a DHCP Server role on an internal network accessible to both the host OS and the guest OS. Windows 10 VMs were used as guests. Each VM was configured with two network adapters - the first one connected to one of the 10GBASE-T LAN ports of the host and the second one connected to the internal network. Both interfaces were configured as DHCP clients.
In the next section, we will talk about Futuremark Servermark VDI in detail, followed by details of how we benchmarked the Supermicro SYS-5028D-TN4T. At this juncture, we must stress that the Xeon-D machine is not really geared for large-scale virtualization workloads. Our purpose here is to take Futuremark's Servermark VDI for a test drive and create a comprehensive evaluation script that can be utilized for evaluating 'production' machines.
Facebook
Twitter
RSS
18 Comments
View All Comments
Ryan Smith - Thursday, September 1, 2016 - link
Ultimately this is an article we worked on because Futuremark's software looked neat, and we wanted to see if it could tell us anything useful.We're nerds at heart. We like looking at new things.=)
aaronb1138 - Friday, September 2, 2016 - link
Although FM puts "2000" as their recommended benchmark level, I think you will find for VDI that the bar is much, much lower. The main gist of VDI is the fact that most users need very little computational power continuously. They need the boot and load software bumps and then most productivity software sips at a few percent of CPU. I've noticed web browsers, especially Chrome, have become the biggest consumers of CPU on workstations, and it seems almost vindictive the way certain browsers will hose a system when resource starved. Mozilla/Firefox is especially designed to make virtualization and RDC scenarios bad (the thousands of cache folders is intentionally shit design for roaming profiles and causes huge login hangs).I noticed a NUC gets around 5000 in the benchmark. From my experience in VDI as well as XenApp and RDS architectures, a NUC could support around 10 users with sufficient RAM (16GB would do it - 32GB and we're talking 20+ users, maybe 40 in XenApp/RDS instead of VDI).
Also, for the love of all that is holy, do not benchmark against RAID 0, it invalidates much of the testing. Test against both RAID 10 and 5 as those are industry standards for such infrastructures.
If you really want to deep dive, check out differences in memory consumption in HyperV vs VMware with VDI. While both support memory deduplication, experience shows that HyperV is much faster and more effective at VDI memory dedupe, while VMware seems to rely a tad heavy on memory ballooning. In server farm / VDI situations, RAM and storage I/O are ALWAYS more limiting than CPU throughput. You're not putting Pro-E and Solidworks into a VDI farm most of the time, just MS Office, a line of business application, and a browser.
madisson - Thursday, September 1, 2016 - link
maybe i am approaching this too focused on a consumer's mind, but all of that hardware needs to be shipped, unpacked, installed, and plugged in. For me, I prefer to log-in to a cloud PC for gaming and virtual PC. There are several services similar to what I'm talking about, www.paperspace.com being the one I happened to pick.I'm sure FM goes for more enterprise-level agreements, but what about the small teams and individuals?
name99 - Thursday, September 1, 2016 - link
It seems to me this is completely a WINTEL benchmark.That's fine for a certain class of users, I guess, but is, I think, a lot less interesting for many readers. It does nothing to clarify whether either
- a Linux PC would be a better server solution OR
- whether an ARM server could handle a particular load.
Obviously AnandTech can do what they want in their reviews, but I, for one, am a lot less interested in comparisons that are focussed on the few percent differences between Wintel box A and Wintel box B than in comparisons between very different systems.
powerarmour - Thursday, September 1, 2016 - link
Exactly.aaronb1138 - Friday, September 2, 2016 - link
Well, they were looking at VDI. I don't know what kind of monster would try VDI+Linux+end user workloads in the workplace. But yeah, keep saying dumbass shit about Linux and ARM.gabemcg - Tuesday, September 6, 2016 - link
Lots of hostility unfairly being directed at the authors/editors here. I thought using the benchmark preview as a way to couch a hardware review was genuinely interesting both as a nerd with a home lab, and a professional wanting to keep abreast of how the enterprise class gear is being evaluated. Keep up the good work, haters gonna hate!simran sidhki - Thursday, October 18, 2018 - link
hello