Personal computing discussed

gigabit ethernet throughput

Moderators: renee, SecretSquirrel, notfred

Post Reply
 
captaintrav
Gerbil First Class
Topic Author
Posts: 178
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

gigabit ethernet throughput

Fri Nov 03, 2017 11:50 am

I was noticing my file transfer speeds seem to be bottlenecked, so I tried doing some testing with nttcp and noticed somethings.

My server has a PCI gigabit Ethernet adapter, and that's going to be a bottleneck possibly, but, I can send faster (around 600mbit) than I can receive (450mbit), what would explain that?

This also affects another machine with a I believe realtek integrated Lan, but it should be PCIe. I can only seem to get ~1gbit in one direction, the other direction goes 800mbit (can't recall which is which)

Predictably, an old laptop with an Intel gigabit NIC can thrash ~1gbit in either direction, as can a machine with an Nvidia integrated LAN.

I guess if I can't find any ideas on either the realtek issue or increasing throughput of a PCI device I can buy some PCIe addin NICs? I want my gigabits! The PCI thing puzzles me because supposedly PCI should be able to do 133MBytes/sec but with my PCI Nic it's bottlenecked to 35MBytes/sec. I can see it being a shared bus and all, but next to nothing should be using PCI bandwidth on this machine.
Top
 
SuperSpy
Minister of Gerbil Affairs
Posts: 2403
Joined: Thu Sep 12, 2002 9:34 pm
Location: TR Forums

Re: gigabit ethernet throughput

Fri Nov 03, 2017 12:32 pm

Remember that PCI can only do 133 MB/s total whereas Ethernet is full duplex aka 125 MB/s both up and down.
Desktop: i7-4790K @4.8 GHz | 32 GB | EVGA Gefore 1060 | Windows 10 x64
Laptop: MacBook Pro 2017 2.9GHz | 16 GB | Radeon Pro 560
Top
 
Usacomp2k3
Gerbil God
Posts: 23043
Joined: Thu Apr 01, 2004 4:53 pm
Location: Orlando, FL
Contact:
Contact Usacomp2k3

Re: gigabit ethernet throughput

Fri Nov 03, 2017 12:49 pm

What's the CPU usage? Cheap one's often rely on the CPU for calculations which can slow it down too.
Top
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 1:40 pm

You already know the answer:
captaintrav wrote:
Predictably, an old laptop with an Intel gigabit NIC can thrash ~1gbit in either direction,

Messing around with anything else is just wasting your time.
Top
 
Aphasia
Grand Gerbil Poohbah
Posts: 3710
Joined: Tue Jan 01, 2002 7:00 pm
Location: Solna/Sweden
Contact:
Contact Aphasia

Re: gigabit ethernet throughput

Fri Nov 03, 2017 2:08 pm

Any decently integrated NIC on the chipset should be able to get full speed. I usually get around 113-120MB/s (900-960mbps) worth, which pegs my NIC's at >90-96% Utilization during one way transfers. Also note that depending on what drives you have, if not an SSD 120MB/s is a fair bit of data that usually aren't reached on multi-file transfers, especially smaller files.
Top
 
captaintrav
Gerbil First Class
Topic Author
Posts: 178
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 2:52 pm

Aphasia wrote:
Any decently integrated NIC on the chipset should be able to get full speed. I usually get around 113-120MB/s (900-960mbps) worth, which pegs my NIC's at >90-96% Utilization during one way transfers. Also note that depending on what drives you have, if not an SSD 120MB/s is a fair bit of data that usually aren't reached on multi-file transfers, especially smaller files.


I agree, but this one seems to be a bit s#$!:
Code: Select all
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 03)

That said, at least it works, I had Realtek hardware in the past that wouldn't work reliably in Linux at all. I'm going to order up an Intel PCIe add-in card and test. I think I should be able to push 60-80MB/sec with the drives I have but haven't got there yet. Of course I'm just being picky since I have a large one-time amount of data to transfer, but what's the use being an 'enthusiast' if I can't be fussy? lol
Top
 
captaintrav
Gerbil First Class
Topic Author
Posts: 178
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 2:55 pm

notfred wrote:
You already know the answer:
captaintrav wrote:
Predictably, an old laptop with an Intel gigabit NIC can thrash ~1gbit in either direction,

Messing around with anything else is just wasting your time.


This is probably the right answer, but the most expensive one. I don't know how much coin Asus, GigaByte, or whoever saves by integrating these Realtek NICs, but I do appreciate my Lenovo business desktops and workstations I work with by day with their Intel networking hardware.
Top
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 3:09 pm

I'm seeing Intel GigE PCIe cards for ~$40 Canadian from Newegg, Amazon and Canada Computers. I don't call that expensive for never having to worry about Realtek again - fear the crab!
Top
 
captaintrav
Gerbil First Class
Topic Author
Posts: 178
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Fri Nov 03, 2017 3:10 pm

notfred wrote:
I'm seeing Intel GigE PCIe cards for ~$40 Canadian from Newegg, Amazon and Canada Computers. I don't call that expensive for never having to worry about Realtek again - fear the crab!

Point taken, the crab is free, and you get what you pay for.
Top
 
Waco
Maximum Gerbil
Posts: 4850
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Fri Nov 03, 2017 9:44 pm

Disable all of the CPU offloading in the driver on the Realtek card and you'll see a nice performance bump at the cost of a little CPU time.
Victory requires no explanation. Defeat allows none.
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sat Nov 04, 2017 6:32 am

Waco wrote:
Disable all of the CPU offloading in the driver on the Realtek card and you'll see a nice performance bump at the cost of a little CPU time.

LOL... so Realtek is the NIC equivalent of the infamous S3 Virge GPU? Hardware-assisted deceleration FTL!
Nostalgia isn't what it used to be.
Top
 
Waco
Maximum Gerbil
Posts: 4850
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Sat Nov 04, 2017 4:05 pm

In my experience...yes. :lol:
Victory requires no explanation. Defeat allows none.
Top
 
titan
Grand Gerbil Poohbah
Posts: 3376
Joined: Mon Feb 18, 2002 7:00 pm
Location: Great Smoky Mountains
Contact:
Contact titan

Re: gigabit ethernet throughput

Sat Nov 04, 2017 4:32 pm

Besides the NIC and computers, your switch will be a limiting factor as well. Any decent switch will at least have 900 Mbps each way.
The best things in life are free.
http://www.gentoo.org
Guy 1: Surely, you will fold with me.
Guy 2: Alright, but don't call me Shirley.
Top
 
Waco
Maximum Gerbil
Posts: 4850
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Sat Nov 04, 2017 6:17 pm

titan wrote:
Besides the NIC and computers, your switch will be a limiting factor as well. Any decent switch will at least have 900 Mbps each way.

Any decent small switch should be able to run full line-rate on every port in both directions. Even my crappy D-Link can do that!
Victory requires no explanation. Defeat allows none.
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sun Nov 05, 2017 3:33 am

captaintrav wrote:
Aphasia wrote:
Any decently integrated NIC on the chipset should be able to get full speed. I usually get around 113-120MB/s (900-960mbps) worth, which pegs my NIC's at >90-96% Utilization during one way transfers. Also note that depending on what drives you have, if not an SSD 120MB/s is a fair bit of data that usually aren't reached on multi-file transfers, especially smaller files.

I agree, but this one seems to be a bit s#$!:
Code: Select all
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 03)

That said, at least it works, I had Realtek hardware in the past that wouldn't work reliably in Linux at all. I'm going to order up an Intel PCIe add-in card and test. I think I should be able to push 60-80MB/sec with the drives I have but haven't got there yet. Of course I'm just being picky since I have a large one-time amount of data to transfer, but what's the use being an 'enthusiast' if I can't be fussy? lol

FWIW I just ran iperf tests between two machines with that exact same Realtek NIC. I get a solid 941 Mb/sec in both directions, with ~10% CPU utilization.

And yes, the Realtek NIC drivers bundled with the Linux kernel used to suck quite badly. Circa 2010 you had to download the source tarball from Realtek's site and compile it yourself if you wanted any chance of having a reasonably performant and stable network connection. IME the stock drivers have been OK since around 2012 though.
Nostalgia isn't what it used to be.
Top
 
Aphasia
Grand Gerbil Poohbah
Posts: 3710
Joined: Tue Jan 01, 2002 7:00 pm
Location: Solna/Sweden
Contact:
Contact Aphasia

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:13 pm

I think all my integrated Nic's are actually Intel based, it doesnt even really show up on any CPU meter.
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:40 pm

Aphasia wrote:
I think all my integrated Nic's are actually Intel based, it doesnt even really show up on any CPU meter.

Note that on Linux the % CPU usage shown by the "top" command is per-core (i.e. full utilization is 100 x number of cores). That 10% is 10% of one core; so on my 8-core FX-8350 it is only 1.25% of available CPU cycles. IOW not enough to matter.
Nostalgia isn't what it used to be.
Top
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:41 pm

Press "1" in top to get the per-cpu usages.
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: gigabit ethernet throughput

Sun Nov 05, 2017 8:42 pm

notfred wrote:
Press "1" in top to get the per-cpu usages.

Ahh, interesting. In the past I've always just used htop when I needed a more detailed view.
Nostalgia isn't what it used to be.
Top
 
Waco
Maximum Gerbil
Posts: 4850
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: gigabit ethernet throughput

Mon Nov 06, 2017 9:02 am

nmon is pretty awesome now that someone ported it to Linux as well. I loved it on AIX and was overjoyed when it was ported to Linux.
Victory requires no explanation. Defeat allows none.
Top
 
captaintrav
Gerbil First Class
Topic Author
Posts: 178
Joined: Thu Dec 12, 2013 12:51 pm
Location: Saskatchewan, Canada

Re: gigabit ethernet throughput

Mon Nov 06, 2017 9:06 am

Waco wrote:
titan wrote:
Besides the NIC and computers, your switch will be a limiting factor as well. Any decent switch will at least have 900 Mbps each way.

Any decent small switch should be able to run full line-rate on every port in both directions. Even my crappy D-Link can do that!


Yes the switch doesn't seem to be a problem. Both an Intel NIC and the integrated NForce Ethernet (despite this not having the greatest rep either) show solid throughput in either direction.
Top
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On