Consumer 10GBase-T Options: Motherboards with 10G Built-In
by Ian Cutress on July 20, 2016 7:00 AM EST- Posted in
- Motherboards
- MSI
- ASRock
- Asus
- 10G Ethernet
- X99
- 10GBase-T
- Z170
Any of our long time readers will know that I can have a soft spot for high-speed networking. Throughout the years I’ve been plagued with bad WiFi, irregular house shapes, routers that only come with a single 1G port, and the inability to set up a decent, cost effective high-speed home network. The movement to 10G has certainly been prevalent in the enterprise space, as shown by the growth of 10G panels at Supercomputing 2015 last year. However the movement to 10G for home use, which is still a niche category I admit, has been slow and still very expensive. Using 10GBase-T, over SFP or other connectivity, allows for backwards compatibility in the home which is still an important feature.
There are two main ways to show 10G to your home network: either pick up a NAS with 10G and use a switch with one or two 10G ports (then 10+ ports with 1G connections), or use a full 10G switch and have 10G devices. The cheapest 10GBase-T switch available for end-users is still Netgear’s XS708E 8-port switch, retailing for $750 or sometimes on sale for $625-650.
That’s around $80+ minimum per port for the switch, and then you need machines with 10G access. Depending on how the wind is blowing and what sales are on, single port PCIe cards with dual 10G ports can be had for $200-$220 on the gray market or up to $400 from source, with the main option of Intel controllers, however there are a few using an Aquantia/Tehuti networking solution. Or roughly translated, $100 per port, and it requires a system with a spare PCIe 2.0 x8 slot.
|
|
|
As we’ve previously reviewed at AnandTech, there are other options for having consumer-based 10G enabled systems. We reviewed the ASRock X99 WS-E/10G back in December 2014, which used a PLX PCIe switch to enable both the Intel X540T2 controller and four-way GPUs in a high-end motherboard. This motherboard currently retails for $770, and uses Haswell-E and Broadwell-E high-end desktop processors.
This motherboard held the title of the only consumer motherboard to come with a 10G chip for over a year. Late last year, in November 2015, ASUS met the feature half-way, offering their high-end Skylake-based Z170 Maximus VIII Extreme motherboard with a bundled single-port 10GBase-T card built on the Aquantia/Tehuti platform. The package carries the Extreme/Assembly branding, and also comes with a front panel DAC. We didn’t get time to review this bundle, and unfortunately the 10G card isn’t sold separately. The whole package is currently on sale for $570, however as it is Skylake based we are limited to a quad core i7-6700K system in this case.
Move forward to Computex 2016, and ASUS dropped on my doorstep the day before I left their new 10G motherboard design for Broadwell-E processors – a near final version of the product. This is the ASUS X99 E-WS/10G, their high-end desktop workstation motherboard with a built in Intel X550T2 controller for dual 10G ports. The board also comes with reinforced PCIe slots, support for U.2 drives, USB 3.1 and most of the updated modern features for a high-end product. We’re still in the process of testing and reviewing this motherboard, which is set for release sometime soon. Price unknown at this point, but we expect it to be certainly near the ASRock 10G board.
On the show floor at Computex, we saw another new entrant to the onboard 10G market on display. While all the other motherboards have been ATX (or E-ATX) so far, MSI is going to plug an Intel X550 controller on a Xeon grade Z170 micro-ATX motherboard to go after some smaller form factor designs. This design uses the C236 chipset, allowing users to equip either a Skylake Core CPU or a Xeon E3-1200 v5 processor, and siphons off four PCIe lanes from the CPU for the 10G controller. This gives 12 lanes from the CPU left, 8 of which go to the main PCIe slot, and another four to the M.2 slot as well. Being both Z170 based and micro-ATX should allow this product to be priced cheaper than the others, although unlike the others it only houses one 10GBase-T port rather than two. When MSI are near final with this motherboard, we’d love to get it in for testing.
There are also a variety of non-consumer focused motherboards with 10GBase-T ports, either with Xeon-D SoCs or dual socket motherboards, such as the MBD-X9DRE-TF+ from Supermicro (dual LGA2011) for $670 or the X10FRC-T4+-O (dual LGA2011-3) for $941.
For consumers looking to move to 10G for home networking, we’re still waiting for the pricing of switches to come down, however it would seem that there are moves afoot to equip more motherboards with 10G by default. There’s still the added cost, and it means the PCIe lane layouts might be a bit different to normal, but it removes the need for an extra PCIe slot/card overall. It’s a slow march, and the price of both switches and 10G controllers needs to come down to make it a more viable option.
Facebook
Twitter
RSS
60 Comments
View All Comments
Wardrop - Wednesday, July 20, 2016 - link
At near range (1 metre), I'm getting up to 60MB/s from my mac mini using 5GHz AC through my Linksys EA6900. I was quite amazed at the speed, and how close to 1Gbps I can get from a single client using a standard consumer client and router.bronan - Wednesday, August 10, 2016 - link
Agreed on that 25 G is what we are talking about for backbone usageOmoronovo - Wednesday, July 20, 2016 - link
I'm conflicted on the idea of 2.5G. On one hand, when even the slowest disk-based network storage is bottlenecked by 1G, we really need the tech to advance - 2.5G would give a lot of breathing room for 10G to evolve and migrate into the consumer space.On the other hand, I almost see it like the migration to IPv6 from IPv4... We're in a mess with IPv4 because NAT gave us a "good enough" solution that delayed adoption of IPv6 substantially. I'd rather wait out this with 1G ports until 10G is affordable, than have consumer adoption of 2.5G and be stuck with it for years and years because it is "good enough".
CaedenV - Wednesday, July 20, 2016 - link
Instead 1G ports are 'good enough' and we are going to be stuck with them for years.Heck, at work we use Cisco phones and all of them only have a 100M pass-through connection because it is 'good enough' for 'most people'.
bronan - Wednesday, August 10, 2016 - link
100 Mbit is good enough for voice and even video streaming because it does not need more at all.But go ahead and do a backup of a 54 TB nas to its sister backup device over 1 Gb networking.... think of that. Now it costs me several days before the backup has completed even though my lan really is almost 120 Mb/s thats the absolute maximum you can get on your 1 Gbit lan
roundtree - Wednesday, July 20, 2016 - link
IEEE is still formalizing the standards for 2.5GbE and 5GbE. Once approved, you'll see more NICs and switches become available.To those confused, the goal for both 2.5GbE and 5GbE does not revolve around data center, enterprise, or consumer markets. Instead, the goal is to enable the reuse of Category 5e and Category 6 cable plants, so that the speed upgrade does not require large capital investment in cabling to deploy. No one is likely to deploy 10GbE in existing office environments or homes if all new cabling has to be deployed whilst simultaneously also deploying new NICs and new switches.
Personally, I don't think we'll see anything widespread for 2.5GbE or 5GbE until the end of 2017 at the earliest. I sincerely hope that I'm wrong and things accelerate quickly.
jjj - Wednesday, July 20, 2016 - link
I know but it feels like it's targeted mostly at wireless access points and not sure if it gains wide adoption in consumer fast. Was hoping it will.azazel1024 - Wednesday, July 20, 2016 - link
Yeah, I had seen some stuff back about 9-12 months ago that was talking up 2.5/5GbE as something we'd likely see in 1H2016, but that of course didn't happen. Hopefully IEEE can get around to finally ratifying it so we can see gear.10G could be adopted in a lot of office environments and pretty much all home environments. Cat5e is rated to 100m with 1GbE, but it is rated to 45m with 10GbE. Many people think you must have Cat6a, but that is absolutely not true. Cat 6 is rated to 55m with 10GbE. Of course the issue is, those are "clean installs", not worst case scenario installs. Figure a typical install you are looking at more like 20-25m + 5m on either end of stranded for Cat5e and still likely see full Cat5e speeds and Cat6 is likely to support more like 30-35m + 5m on either end of stranded. Which likely isn't enough for most office environments, but it certainly plenty for most home environments.
2.5GbE IIRC is going to support Cat5e to 100m and 5GbE is Cat6 to 100m (which also likely means Cat5e to 60-80m).
Unless 2.5/5GbE gets completely torpedoed, I really don't think we will see 10GbE moved to a consumer standard at any point, or at least not for 5+ years and possibly 10+. We are still at least 1 if not 2 node shrinks for 10GbE silicon to be reasonable in power consumption for something like a home router, cheap desktop switch or onboard laptop NIC. IIRC the newest 10GbE silicon is looking at ~4w a port and it really needs to be drive down to more like 2w a port for those uses, otherwise heat is a big issue in consumer gear (yes, your typical high end home router might use 20w of power under heavy load, it also has lots of passive cooling and runs very hot, now try adding in an extra 12-15w if a set of 5 10GbE ports are in use, you'd absolutely have to have active cooling or a MUCH larger chassis with a big heat sink).
Things are advancing fast, but frankly 95% of home users networking is limited to their internet connection, which even in more advance countries, the typical user doesn't have speeds above about 100-150Mbps. So 1GbE ports are just fine. Even with 1Gbps internet connections becoming more plentiful, they are probably less than 5% of deployment world wide (just taking in to account 1st world countries) and >1Gbps is probably less than 1%. For the users who do something other than connect to the internet with their networking, probably a good half don't really know or care about having something faster than 1Gbps, or are connecting wirelessly to their router and are limited to <1Gbps speeds because of that.
So you are talking a use case of maybe 2-4% of home users who need, want or care about >1Gbps networking speeds and maybe only 5-10% of businesses who need it outside of their core network. The gov't agency I work with, frankly we just need it on our big iron. Our workstations, 99.99% of employees are just fine with a 100Mbps connection. The big iron has plenty of 10/40GbE ports.
I really want it and could use it and I'd really like to see consumer gear deploy at least 2.5GbE in the next couple of years. However, I don't really NEED it. I have dual 1GbE links from my server, to my switch and then to my desktop and I am running Win 8.1 (about to be 10) on both machines so SMB Multichannel is fully up. I get 230-235MB/sec transfers between them easily, which is relatively close to maxing out my HDD arrays (a pair of Seagate 3TB drives in RAID0 in both machines, max performance is ~368MB/sec, but that is on the outer edge, they are ~45% utilized, so I am actually maxing out around ~290MB/sec right now for new writes, so a single 2.5GbE basically maxes my HDD array). SSDs are still very far away from bulk storage and sure, there will be some users who have a laptop with a 256+GB SSD and want a super fast transfer to whatever they are transferring from/too, but in most cases, it isn't going to be another SSD on a NAS or server or a huge pipe to the internet.
I am hoping when I am ready to move beyond my 6TB array, that I'll be able to go all solid state on it in a few years, but that is still at least 3-4 years away and I am likely to pay $1000+ to drop something like 6-8TB of SSD storage in each machine (but then I can also easily just add disks to expand storage as I need to, no worries about matching the storage, because I won't need RAID as the SSD performance is high enough).
2.5GbE is going to be needed for wireless at some point soon. That or dual 1GbE links. I am kind of surprised that 802.11ac features have rolled out as slowly as they have. 160MHz doesn't surprise me because of the expense, difficulty of it and the limited spectrum that involves. However, MU:MIMO has been a MUCH slower roll out than I expected. Intel just started supporting it with the 8265, but I am surprised that Apple didn't with the 6s. Hopefully the next one will and most routers won't, except "AC2600" routers.
In theory, with the right mix of clients, an AC2600 router in a good wireless environment with the clients fairly close to the router, probably could just about hit 1Gbps over wireless. With a 4:4 bridge, it absolutely could exceed 1Gbps. In my testing, bridging newer 802.11ac routers within a reasonable distance, say 10-20ft and maybe only 1 interior wall between them, you get about 70% wireless utilization. A good client these days seems to hit low to mid 60% utilization. That means a pair of 4:4 802.11ac routers bridged could hit roughly 1200Mbps, but would be limited by port speeds.
If 160MHz actually hits, a 4:4 802.11ac router to 160MHz capable MU:MIMO clients almost certainly could exceed 1Gbps over wireless, but would be limited by port speeds if all the traffic was over one port and headed in the same direction.
802.11ax is supposed to significantly speed up 802.11ac. It has a higher MCS/QAM coding than 802.11ac, is 160Mhz for 5GHz and somewhat higher spatial efficiency from some other stuff. It is a theoretical 1201Mbps per stream, once you figure error encoding overhead and the fact that you'll need REALLY good SNR to actually enable clean 1024-QAM it'll probably be a lot like 802.11ac where the first chipsets and routers are hitting more like 50% of those maximum speeds on a good connection, but given a few years you'll likely see closer to 70% on a good connection. That still means that early 2:2 clients are likely going to be hitting or exceeding 1Gbps actual over wireless and be limited by 1GbE. Down the road a few years and a good client is likely to be hitting more like 1.5-1.8Gbps over wireless on a good connection.
There may be something after 802.11ax, but that is a LOT of years away. Right now 802.11ax is likely going to be 3-5 years away from implementation for early equipment and 6-8 before it is mature. So following that, 2.5GbE probably could support almost all consumer use cases, easily, for a decade or more. For the few users who need more, there would be 5GbE, or trunked 2.5GbE.
DanNeely - Wednesday, July 20, 2016 - link
Unlike the 10 or 20W numbers being quoted a few years ago for then current hardware, I really don't think 2 vs 4W per port is a major concern for initial home installation. For a typical 4/8 port home router/switch the higher power consumption isn't a major concern the way it would be on a giant enterprise switch with 80 or 160 ports. On the desktop PCs/NASes that would be the primary clients an extra 2W is borderline irrelevant.For laptops on battery it would be a more serious problem; but high end consumer laptops are rapidly dropping wired ethernet support entirely making it a moot point. They're more common on $400 boxmart laptops; but those won't see anything faster than 10gbE until it gets baked into a chipset by Intel/AMD allowing them to offer it for 'free' again making it a moot point.
azazel1024 - Friday, July 22, 2016 - link
You'd be surprised. Also right now, GbE is running about .4-.7w per port and 10GbE is around 4.5w per port. I'd suspect that something like 2.5GbE is likely to be in the 1-2w range.That means a power difference of about 2.5-3.5w per port. With your typical home router, that is 5 ports times 2.5-3.5w or 12.5-17.5w per router if all ports are in use running 10GbE. That roughly doubles or more the power consumption of the typical home router. If you look at most that are running in the 10-15w range, they have a LOT of cooling vents and some pretty big heat sinks internally. Frankly, ~30w in an enclosure as small as a typical home router and you have to have active cooling. That is an issue for most home routers.
So, yeah, the power difference is real and considerable and a concern. The power use itself isn't a big concern, but packaging and cooling concerns make it an issue.
No, on a desktop between .5w and 4.5w isn't a big deal. On a laptop on battery it is a big concern and even with a power adapter it can be. For some of the big laptops, not a big deal, but as an example, my HP Envy 4t runs about 25w at peak power consumption under load. That includes screen, Ivy Bridge processor, memory, motherboard, wifi, NIC, etc. That would be an increase of about 15% more heat that the laptop needs to dissipate and actually a little worse since 2-4w of that is the display, which isn't dissipated through the keyboard portion of the chasis.
Just some thoughts on the matter.