TR Forums

I'm dying to replace my router, and would prefer to upgrade to the faster speed, anyone have any idea when we will start to see some actual products? It feels like a long time since this standard was announced.

It's probably going to be awhile, even though those multiple-APs-in-one-box to create really high marketing number routers already theoretically exceed the bandwidth of one gigabit LAN port.

Buffalo just released their unmanaged 10gbps switches this month, and were able to include 2.5 and 5 support for them. $650 for 8 ports. For now, unless you need it for the WAN connection you could plug such a 802.3bz switch into the router.

Why wait? LACP is relatively well supported... with a sufficiently large switch (for the extra ports) and a relatively inexpensive dual/quad-interface network card you can get those speeds right now.

See https://en.wikipedia.org/wiki/Link_aggregation if you're curious.

Vhalidictes wrote:

Why wait? LACP is relatively well supported... with a sufficiently large switch (for the extra ports) and a relatively inexpensive dual/quad-interface network card you can get those speeds right now.

See https://en.wikipedia.org/wiki/Link_aggregation if you're curious.

* if you're streaming from multiple sources at the same time and they happen to hash to different interfaces.

2.5 Gbps and 5 Gbps would actually be beneficial for me. My NAS can handle ~2 GB/s of writes (backups and whatnot) but instead, I have to wait *hours* for my desktop and my wife's desktop to back up due to the limitations of gigabit.

Waco wrote:

Vhalidictes wrote:

Why wait? LACP is relatively well supported... with a sufficiently large switch (for the extra ports) and a relatively inexpensive dual/quad-interface network card you can get those speeds right now.

See https://en.wikipedia.org/wiki/Link_aggregation if you're curious.

* if you're streaming from multiple sources at the same time and they happen to hash to different interfaces.

2.5 Gbps and 5 Gbps would actually be beneficial for me. My NAS can handle ~2 GB/s of writes (backups and whatnot) but instead, I have to wait *hours* for my desktop and my wife's desktop to back up due to the limitations of gigabit.

Can you multiplex Wireless and Wired? Also, if what you're saying is accurate link aggregation would help you on the NAS side.

LACP would help for when both systems are backing up simultaneously, if we're lucky enough to hash to different interfaces. It wouldn't help at all when I'm waiting on just mine cranking along at 112 MB/s.

I've very nearly bought 10 Gb more than once, but I want 5 Gb to undercut pricing significantly.

Waco wrote:

2.5 Gbps and 5 Gbps would actually be beneficial for me. My NAS can handle ~2 GB/s of writes (backups and whatnot) but instead, I have to wait *hours* for my desktop and my wife's desktop to back up due to the limitations of gigabit.

That's my use case as well, I want to enable faster backups and file transfers.

I've moved most of my steam library to a NAS system. A RAID5 with spinning disks can max out gigabit, would be interesting to see how much faster it could go.

leor wrote:

2.5GBASE-T and 5GBASE-T routers available when?

I don't follow that level of networking gear very closely, but a router with 5gbit or 10gbit ports will be prohibitively expensive, and not particularly useful unless you're paying for a really serious internet connection. That sort of bandwidth is generally only needed for local transfers, and in that case you're better off going with a (much less expensive) switch.

10 Gbit has been coming down in price over the past two years to be feasible for the enthusiast. Several LGA 2011-3 motherboards are coming with 10 Gbit ports on board. Switch prices have also come down in price.

However, I do see NBaseT (2.5 Gbit and 5 Gbit) taking off in the consumer and embedded space faster than 10 Gbit for a couple of reasons. The error detection/correction on 10 Gbit is a different algorithm than on the NBaseT protocols, which is one of the reasons for the high cost of 10 Gbit so far. Similarly, there is no official version of PoE with 10 Gbit Ethernet but that does exist with NBaseT. Lots of embedded applications have moved to PoE so the NBaseT standard would be the next logical step for a speed increase and it does not necessarily require re-running cabling. I do see many prosumer NBaseT switches coming with 10 Gbit uplinks.

BobbinThreadbare wrote:

I've moved most of my steam library to a NAS system. A RAID5 with spinning disks can max out gigabit, would be interesting to see how much faster it could go.

Speeds on a decent SAN system are insanely fast for sequential transfers. I have no idea how fast the system I work on really is (for single-user purposes) since it can max out a 40GB connection no problem (very roughly 4GB/sec max, if you're curious).

Of course that metric really doesn't make sense for most SAN arrays as they are looking at 100+ transfers going on at any given time, but I'd expect that a decent 8-drive NAS could put out ~800MB/sec == 10Gbit connection on average.

Vhalidictes wrote:

BobbinThreadbare wrote:

I've moved most of my steam library to a NAS system. A RAID5 with spinning disks can max out gigabit, would be interesting to see how much faster it could go.

Speeds on a decent SAN system are insanely fast for sequential transfers. I have no idea how fast the system I work on really is (for single-user purposes) since it can max out a 40GB connection no problem (very roughly 4GB/sec max, if you're curious).

Of course that metric really doesn't make sense for most SAN arrays as they are looking at 100+ transfers going on at any given time, but I'd expect that a decent 8-drive NAS could put out ~800MB/sec == 10Gbit connection on average.

For sequential transfers, yes, but random IO (like loading games) is a lot more brutal to optimize. It's still not too bad if you set up your NAS properly with a decent amount of RAM and a good CPU/NIC, but it's definitely not much faster than a single HDD for loading games.

Waco wrote:

Vhalidictes wrote:

BobbinThreadbare wrote:

I've moved most of my steam library to a NAS system. A RAID5 with spinning disks can max out gigabit, would be interesting to see how much faster it could go.

Speeds on a decent SAN system are insanely fast for sequential transfers. I have no idea how fast the system I work on really is (for single-user purposes) since it can max out a 40GB connection no problem (very roughly 4GB/sec max, if you're curious).

Of course that metric really doesn't make sense for most SAN arrays as they are looking at 100+ transfers going on at any given time, but I'd expect that a decent 8-drive NAS could put out ~800MB/sec == 10Gbit connection on average.

For sequential transfers, yes, but random IO (like loading games) is a lot more brutal to optimize. It's still not too bad if you set up your NAS properly with a decent amount of RAM and a good CPU/NIC, but it's definitely not much faster than a single HDD for loading games.

That's a great point, Waco. I know someone that has a 12-spindle NAS running FreeNAS, and he set up a 128GB x 2 RAID-0 SSD array to use as a cache. This seems to have solved the problem of random access pretty well.

Of course that system is also running 32GB of RAM too, I forget why - something something ZFS needs all of the RAM to work properly? Sorry I'm not too familiar with the nitty gritty of a FreeNAS system other than it's headless BSD. I *do* know a tiny bit about ZFS... which is why I've never used it.

My NAS runs FreeNAS too. 16 spindles (pair of 5+3 RAIDZ3s). It's pretty quick, but random IO is not its forte. You don't need tons of RAM to run ZFS (contrary to what the idiots admining their forums say) but I have 32 GB since it does help random access once the ARC is hot. I also have mine set up with a 32 GB L2ARC for metadata (to minimize seeks).

I used to run a similar L2ARC setup as your friend (a pair of 128 GB SSDs) but I got tired of the limitations of gigabit...I'd revisit it when/if something faster becomes economically viable at home. It ran games as good or better than an internal HDD, especially after loading things once or twice.

My dream is to run almost everything other than my OS on a NAS. I have a ton of netbooks and desktops just lying around - removing all of their local storage would be a godsend.

The main issue I'm running into is money - it's not trivial to buy 8 decent drives and I don't currently have a system with the SATA ports / Case / PSU that I can easily repurpose, so I'd need to get that too. Fortunately I have a spare Bulldozer CPU that should work well... not sure if I want to use it 24x7 though due to power.

Of course, once I do get the ~1000 together I'll need to avoid the urge to upgrade my aging Sandy Bridge-E PC...

Vhalidictes wrote:

My dream is to run almost everything other than my OS on a NAS. I have a ton of netbooks and desktops just lying around - removing all of their local storage would be a godsend.

The main issue I'm running into is money - it's not trivial to buy 8 decent drives and I don't currently have a system with the SATA ports / Case / PSU that I can easily repurpose, so I'd need to get that too. Fortunately I have a spare Bulldozer CPU that should work well... not sure if I want to use it 24x7 though due to power.

Of course, once I do get the ~1000 together I'll need to avoid the urge to upgrade my aging Sandy Bridge-E PC...

Or you could transform that SandyBridge-E into a the corner stone of your NAS. That what I did. Even figured out how to host VM's a few years back on NAS4Free via VirtualBox.

the wrote:

Vhalidictes wrote:

My dream is to run almost everything other than my OS on a NAS. I have a ton of netbooks and desktops just lying around - removing all of their local storage would be a godsend.

The main issue I'm running into is money - it's not trivial to buy 8 decent drives and I don't currently have a system with the SATA ports / Case / PSU that I can easily repurpose, so I'd need to get that too. Fortunately I have a spare Bulldozer CPU that should work well... not sure if I want to use it 24x7 though due to power.

Of course, once I do get the ~1000 together I'll need to avoid the urge to upgrade my aging Sandy Bridge-E PC...

Or you could transform that SandyBridge-E into a the corner stone of your NAS. That what I did. Even figured out how to host VM's a few years back on NAS4Free via VirtualBox.

... I don't have the money to build a NAS, but if I had enough to upgrade my PC AND buy drives that would be a great plan. It would probably cost only about 50% more.

Not a bad plan though. I'd just need to wait a bit longer.

Is your question really "When will the first consumer NBase-T routers be available?" There are several commercial options available now.

dextrous wrote:

Is your question really "When will the first consumer NBase-T routers be available?" There are several commercial options available now.

You can get switches and network cards right now. https://www.newegg.com/Product/Product.aspx?Item=N82E16833162139&cm_re=10gbase-t-_-33-162-139-_-Product

I'd struggle to understand the need for NBase-T routers right now. If that's a need you have, please let me know what ISP you use, so I can move to their service area.

As I pointed out back in post#2, WAN exceeding 1gbps would indeed require a router and not just a switch. It's more common than you'd think because even Comcast has had 2gbps service for two years now at $300/mo.

The problem is no consumer router can rout that fast so the options are enterprise equipment or roll your own. I notice SmallNetBuilder has tested ~a dozen consumer routers that can saturate gigabit WAN to LAN at least when using hardware acceleration, but 2gbps would require twice as powerful hardware. Hilariously, many of these routers can do link aggregation but only in software so not only can they not max out a single gigabit LAN link when teamed, this also kills routing performance too if enabled.

If you roll your own router from a PC, presumably you can make it do double-duty as your NAS to make it worth the power consumption. Careful selection of parts for ZFS and ECC recommended.

After gigabit was ratified in 1999, gigabit ports did not drop in price enough to be integrated into consumer devices until 2003. Likewise, 802.3bz should be available soon but not cheap for a couple more years.

Has anyone considered making a DIY router with 10Gbps Mellanox cards? I'd be interested to know if it's worth the money/electricity/trouble.

EDIT: Just realized these are SFP+, which means you'd need a transceiver: https://www.amazon.com/dp/B016OYD0D4/

EDIT 2: Well hey look at this one! $250 for 2x10GbE. https://www.amazon.com/dp/B01LZMM7ZO/

I can see file transfers still taking ages over gigabit, but for those moaning that backups take too long, why aren't you using differential backups? Or at least in the simplest format, something like robocopy that only changes the files that were modified since the last backup....

Dead in the water until prices are not idiotic.

It is literally cheaper to build a 40Gbit network off ebay (switches, optics, fiber and all) than to upgrade your network to 10Gbe copper and hope it works over your short cat6 runs. It might even use less power

You will see start seeing equipment with 802.11bz hitting SMB markets en mass sometime in Q3-Q4 of 2017. Customer-type stuff will not hit the shelves until late 2018. There's simply no mainstream demand for 802.11bz. The masses are opting for wireless ethernet.

10Gbps Ethernet and beyond are going to remain enterprise-datacenter-tier for the foreseeable future.

I've never even heard of 2Gbit+ Comcast service. Also, even if it were provisioned, how would you use it? I guess it's a thing, but not anywhere I've heard of.

My personal connection is ~170Mbit, and that's only in the last year. Before that I had a 10/100 WAN interface that was plenty for cable Internet (old ASA 5505).

Chrispy_ wrote:

I can see file transfers still taking ages over gigabit, but for those moaning that backups take too long, why aren't you using differential backups? Or at least in the simplest format, something like robocopy that only changes the files that were modified since the last backup....

I do run differentials, but I do a full every 4 weeks. I also do a differential file-level backup of the entire NAS to a set of HDDs that go into the safe every month or so (granted, that wouldn't go much faster since the HDDs themselves can only do about 150 MB/s).

Gigabit just isn't enough when you're backing up 1 TB+.

OK, I'm going to expose my ignorance of the OSI model here, but this has always been a question to me.

If I've got a dumb 2.5GBit switch (assumption for this hypo) connected to the router by a 1GBit link, does not the speed of that link constrain the packet flow between ports on the switch? If packets have to hit the router to be routed through a dumb switch, why isn't the router the road block?

When y'all tell me I'm wrong, please do it in words I might understand.

Captain Ned wrote:

OK, I'm going to expose my ignorance of the OSI model here, but this has always been a question to me.

If I've got a dumb 2.5GBit switch (assumption for this hypo) connected to the router by a 1GBit link, does not the speed of that link constrain the packet flow between ports on the switch? If packets have to hit the router to be routed through a dumb switch, why isn't the router the road block?

When y'all tell me I'm wrong, please do it in words I might understand.

You only have to route if you're leaving the subnet (going to the Internet) - dumb switches route between machines on the same subnet without having to hit the router.

Duct Tape Dude wrote:

Has anyone considered making a DIY router with 10Gbps Mellanox cards? I'd be interested to know if it's worth the money/electricity/trouble.

EDIT: Just realized these are SFP+, which means you'd need a transceiver: https://www.amazon.com/dp/B016OYD0D4/

EDIT 2: Well hey look at this one! $250 for 2x10GbE. https://www.amazon.com/dp/B01LZMM7ZO/

Neither the Mellanox cards or that switch support 2.5GBASE-T or 5GBASE-T which are the subject of this thread. It's probably a good idea to get a switch that does, because future consumer APs and NASes are likely to support those and not 10gbit. The NICs shouldn't matter if you have a 10gbit switch although many of the early 10gbit cards were intended for InfiniBand and have pretty poor performance as a NIC.

Store-and-forward switches can convert between protocols so you can have both 10gbit and 2.5gbit devices connected. The conversion would only limit the speed to the slower protocol when transferring to/from the slower device because it's a switch, not a hub.

The Comcast service is called "Gigabit Pro" and is fiber-to-the-home like FIOS or Google, only faster (2gbps down and 2gbps up), and only in large cities like Boston. The provided modem has both 10gig and 1gig ethernet ports (you're supposed to connect the 1gig one to the provided Netgear R8000 for Wifi). Comcast is rolling out DOCSIS 3.1 1gbit service nationwide next year to better compete on price.

Captain Ned wrote:

OK, I'm going to expose my ignorance of the OSI model here, but this has always been a question to me.

If I've got a dumb 2.5GBit switch (assumption for this hypo) connected to the router by a 1GBit link, does not the speed of that link constrain the packet flow between ports on the switch? If packets have to hit the router to be routed through a dumb switch, why isn't the router the road block?

When y'all tell me I'm wrong, please do it in words I might understand.

Captain, while it's true that consumer switches tend to cheap out on the backplane (total throughput allowed), the link speed of one port doesn't affect the link speed of any of the other ports.

Only traffic to/from the 1GB router connection would be limited to 1GB (well, really, 2GB total since each direction (in and out) would have a full gigabit).