Personal computing discussed

Moderators: SecretSquirrel, notfred

 
captaintrav
Gerbil First Class
Topic Author
Posts: 125
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

gigabit ethernet throughput

Fri Nov 03, 2017 11:50 am

I was noticing my file transfer speeds seem to be bottlenecked, so I tried doing some testing with nttcp and noticed somethings.

My server has a PCI gigabit Ethernet adapter, and that's going to be a bottleneck possibly, but, I can send faster (around 600mbit) than I can receive (450mbit), what would explain that?

This also affects another machine with a I believe realtek integrated Lan, but it should be PCIe. I can only seem to get ~1gbit in one direction, the other direction goes 800mbit (can't recall which is which)

Predictably, an old laptop with an Intel gigabit NIC can thrash ~1gbit in either direction, as can a machine with an Nvidia integrated LAN.

I guess if I can't find any ideas on either the realtek issue or increasing throughput of a PCI device I can buy some PCIe addin NICs? I want my gigabits! The PCI thing puzzles me because supposedly PCI should be able to do 133MBytes/sec but with my PCI Nic it's bottlenecked to 35MBytes/sec. I can see it being a shared bus and all, but next to nothing should be using PCI bandwidth on this machine.
 
SuperSpy
Gold subscriber
Minister of Gerbil Affairs
Posts: 2226
Joined: Thu Sep 12, 2002 9:34 pm
Location: TR Forums

Re: gigabit ethernet throughput

Fri Nov 03, 2017 12:32 pm

Remember that PCI can only do 133 MB/s total whereas Ethernet is full duplex aka 125 MB/s both up and down.
Desktop: i7-4790K @4.8 GHz | 32 GB | EVGA Gefore 1060 | Windows 10 x64
Laptop: i7 740QM | 12 GB | Mobility Radeon 5850 | Windows 10 x64
 
Usacomp2k3
Gerbil God
Posts: 21686
Joined: Thu Apr 01, 2004 4:53 pm
Location: Orlando, FL
Contact:

Re: gigabit ethernet throughput

Fri Nov 03, 2017 12:49 pm

What's the CPU usage? Cheap one's often rely on the CPU for calculations which can slow it down too.
 
notfred
Maximum Gerbil
Posts: 4392
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 1:40 pm

You already know the answer:
captaintrav wrote:
Predictably, an old laptop with an Intel gigabit NIC can thrash ~1gbit in either direction,

Messing around with anything else is just wasting your time.
 
Aphasia
Grand Gerbil Poohbah
Posts: 3682
Joined: Tue Jan 01, 2002 7:00 pm
Location: Solna/Sweden
Contact:

Re: gigabit ethernet throughput

Fri Nov 03, 2017 2:08 pm

Any decently integrated NIC on the chipset should be able to get full speed. I usually get around 113-120MB/s (900-960mbps) worth, which pegs my NIC's at >90-96% Utilization during one way transfers. Also note that depending on what drives you have, if not an SSD 120MB/s is a fair bit of data that usually aren't reached on multi-file transfers, especially smaller files.
 
captaintrav
Gerbil First Class
Topic Author
Posts: 125
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 2:52 pm

Aphasia wrote:
Any decently integrated NIC on the chipset should be able to get full speed. I usually get around 113-120MB/s (900-960mbps) worth, which pegs my NIC's at >90-96% Utilization during one way transfers. Also note that depending on what drives you have, if not an SSD 120MB/s is a fair bit of data that usually aren't reached on multi-file transfers, especially smaller files.


I agree, but this one seems to be a bit s#$!:
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 03)

That said, at least it works, I had Realtek hardware in the past that wouldn't work reliably in Linux at all. I'm going to order up an Intel PCIe add-in card and test. I think I should be able to push 60-80MB/sec with the drives I have but haven't got there yet. Of course I'm just being picky since I have a large one-time amount of data to transfer, but what's the use being an 'enthusiast' if I can't be fussy? lol
 
captaintrav
Gerbil First Class
Topic Author
Posts: 125
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 2:55 pm

notfred wrote:
You already know the answer:
captaintrav wrote:
Predictably, an old laptop with an Intel gigabit NIC can thrash ~1gbit in either direction,

Messing around with anything else is just wasting your time.


This is probably the right answer, but the most expensive one. I don't know how much coin Asus, GigaByte, or whoever saves by integrating these Realtek NICs, but I do appreciate my Lenovo business desktops and workstations I work with by day with their Intel networking hardware.
 
notfred
Maximum Gerbil
Posts: 4392
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 3:09 pm

I'm seeing Intel GigE PCIe cards for ~$40 Canadian from Newegg, Amazon and Canada Computers. I don't call that expensive for never having to worry about Realtek again - fear the crab!
 
captaintrav
Gerbil First Class
Topic Author
Posts: 125
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 3:10 pm

notfred wrote:
I'm seeing Intel GigE PCIe cards for ~$40 Canadian from Newegg, Amazon and Canada Computers. I don't call that expensive for never having to worry about Realtek again - fear the crab!

Point taken, the crab is free, and you get what you pay for.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Posts: 2033
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Fri Nov 03, 2017 9:44 pm

Disable all of the CPU offloading in the driver on the Realtek card and you'll see a nice performance bump at the cost of a little CPU time.
Z170A Gaming Pro Carbon | 6700K @ 4.5 | 16 GB | GTX Titan X | Seasonix Gold 850 | XSPC RX360 | Heatkiller R3 | D5 + RP-452X2 | Cosmos II | Samsung 4K 40" | 480 + 240 + LSI 9207-8i (128x8) SSDs
 
just brew it!
Gold subscriber
Administrator
Posts: 49709
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sat Nov 04, 2017 6:32 am

Waco wrote:
Disable all of the CPU offloading in the driver on the Realtek card and you'll see a nice performance bump at the cost of a little CPU time.

LOL... so Realtek is the NIC equivalent of the infamous S3 Virge GPU? Hardware-assisted deceleration FTL!
Nostalgia isn't what it used to be.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Posts: 2033
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Sat Nov 04, 2017 4:05 pm

In my experience...yes. :lol:
Z170A Gaming Pro Carbon | 6700K @ 4.5 | 16 GB | GTX Titan X | Seasonix Gold 850 | XSPC RX360 | Heatkiller R3 | D5 + RP-452X2 | Cosmos II | Samsung 4K 40" | 480 + 240 + LSI 9207-8i (128x8) SSDs
 
titan
Grand Gerbil Poohbah
Posts: 3363
Joined: Mon Feb 18, 2002 7:00 pm
Location: Great Smoky Mountains
Contact:

Re: gigabit ethernet throughput

Sat Nov 04, 2017 4:32 pm

Besides the NIC and computers, your switch will be a limiting factor as well. Any decent switch will at least have 900 Mbps each way.
The best things in life are free.
http://www.gentoo.org
Guy 1: Surely, you will fold with me.
Guy 2: Alright, but don't call me Shirley.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Posts: 2033
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Sat Nov 04, 2017 6:17 pm

titan wrote:
Besides the NIC and computers, your switch will be a limiting factor as well. Any decent switch will at least have 900 Mbps each way.

Any decent small switch should be able to run full line-rate on every port in both directions. Even my crappy D-Link can do that!
Z170A Gaming Pro Carbon | 6700K @ 4.5 | 16 GB | GTX Titan X | Seasonix Gold 850 | XSPC RX360 | Heatkiller R3 | D5 + RP-452X2 | Cosmos II | Samsung 4K 40" | 480 + 240 + LSI 9207-8i (128x8) SSDs
 
just brew it!
Gold subscriber
Administrator
Posts: 49709
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sun Nov 05, 2017 3:33 am

captaintrav wrote:
Aphasia wrote:
Any decently integrated NIC on the chipset should be able to get full speed. I usually get around 113-120MB/s (900-960mbps) worth, which pegs my NIC's at >90-96% Utilization during one way transfers. Also note that depending on what drives you have, if not an SSD 120MB/s is a fair bit of data that usually aren't reached on multi-file transfers, especially smaller files.

I agree, but this one seems to be a bit s#$!:
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 03)

That said, at least it works, I had Realtek hardware in the past that wouldn't work reliably in Linux at all. I'm going to order up an Intel PCIe add-in card and test. I think I should be able to push 60-80MB/sec with the drives I have but haven't got there yet. Of course I'm just being picky since I have a large one-time amount of data to transfer, but what's the use being an 'enthusiast' if I can't be fussy? lol

FWIW I just ran iperf tests between two machines with that exact same Realtek NIC. I get a solid 941 Mb/sec in both directions, with ~10% CPU utilization.

And yes, the Realtek NIC drivers bundled with the Linux kernel used to suck quite badly. Circa 2010 you had to download the source tarball from Realtek's site and compile it yourself if you wanted any chance of having a reasonably performant and stable network connection. IME the stock drivers have been OK since around 2012 though.
Nostalgia isn't what it used to be.
 
Aphasia
Grand Gerbil Poohbah
Posts: 3682
Joined: Tue Jan 01, 2002 7:00 pm
Location: Solna/Sweden
Contact:

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:13 pm

I think all my integrated Nic's are actually Intel based, it doesnt even really show up on any CPU meter.
 
just brew it!
Gold subscriber
Administrator
Posts: 49709
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:40 pm

Aphasia wrote:
I think all my integrated Nic's are actually Intel based, it doesnt even really show up on any CPU meter.

Note that on Linux the % CPU usage shown by the "top" command is per-core (i.e. full utilization is 100 x number of cores). That 10% is 10% of one core; so on my 8-core FX-8350 it is only 1.25% of available CPU cycles. IOW not enough to matter.
Nostalgia isn't what it used to be.
 
notfred
Maximum Gerbil
Posts: 4392
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:41 pm

Press "1" in top to get the per-cpu usages.
 
just brew it!
Gold subscriber
Administrator
Posts: 49709
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:42 pm

notfred wrote:
Press "1" in top to get the per-cpu usages.

Ahh, interesting. In the past I've always just used htop when I needed a more detailed view.
Nostalgia isn't what it used to be.
 
Waco
Gold subscriber
Minister of Gerbil Affairs
Posts: 2033
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Mon Nov 06, 2017 9:02 am

nmon is pretty awesome now that someone ported it to Linux as well. I loved it on AIX and was overjoyed when it was ported to Linux.
Z170A Gaming Pro Carbon | 6700K @ 4.5 | 16 GB | GTX Titan X | Seasonix Gold 850 | XSPC RX360 | Heatkiller R3 | D5 + RP-452X2 | Cosmos II | Samsung 4K 40" | 480 + 240 + LSI 9207-8i (128x8) SSDs
 
captaintrav
Gerbil First Class
Topic Author
Posts: 125
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Mon Nov 06, 2017 9:06 am

Waco wrote:
titan wrote:
Besides the NIC and computers, your switch will be a limiting factor as well. Any decent switch will at least have 900 Mbps each way.

Any decent small switch should be able to run full line-rate on every port in both directions. Even my crappy D-Link can do that!


Yes the switch doesn't seem to be a problem. Both an Intel NIC and the integrated NForce Ethernet (despite this not having the greatest rep either) show solid throughput in either direction.

Who is online

Users browsing this forum: No registered users and 1 guest