Page 1 of 1
slow network transfer
Posted: Fri Feb 29, 2008 12:00 am
by Usacomp2k3
So I'm moving data off of my server (Ubuntu 7.10) to change stuff up, but the network transfer is running really slow. It has an Intel Pro/1000 card in it (PCI) and is hooked up to a D-Link 8-port gigabit switch, and then hooked up to my desktop, which is also gigabit via the onboard NIC. The transfer speed is ~7% (as reported by Vista on my desktop), or 8MB/sec. The hard drive is a..WD 250gb, IIRC. The receiving drive is a Seagate 7200.10 via SATA.
I checked ifconfig and there are no errors in transfer. It is complaining about "no ipv6 routers present" but I don't think that that is slowing stuff down. Can ya'll think of anything that is causing my server to be so slow?
Note, I'm accessing a share on my desktop from the ubuntu machine and doing all transfers from the Ubuntu machine due to it having issues making shareable network shares. I'd rather not try and figure that out at this point, but if so desired, then so be it.
EDIT:
Re: slow network transfer
Posted: Fri Feb 29, 2008 11:16 am
by notfred
I suspect the problem is on the receiving end, Vista has known issues with gigabit networks if there is any sound at the same time. Also that 7200.10 wouldn't happen to be a 500GB with the slow AAK firmware would it? Although even the slow AAK firmware is faster than that in everything I've seen.
I have benchmarked the Intel Pro 1000 GT PCI network card in a machine running Ubuntu in to a Dlink DGS1008D 8 port gigabit switch at over 700Mbit/s with iperf, so I don't think the networking is a problem.
One other thing may be the drive under Ubuntu. What does "sudo hdparm -tT /dev/sda" show for buffered disk reads (with no other disk access going on at the same time)? You may need to "sudo apt-get install hdparm" first, I don't think it is part of the default install.
Re: slow network transfer
Posted: Fri Feb 29, 2008 11:42 am
by Usacomp2k3
the hdparm returns 277MB/sec cached and 55.78 MB/sec buffered reads on one hard drive and 279/30.38MB/sec for the other (I don't remember which is the 40 and which is the 250. I'm guessing the faster is the larger/newer)
When I backed up files from my laptops to my desktop, they transfered at 8-9MB/sec each (the laptops only have 10/100), so I don't think it's a Vista problem. However, that was before SP1, so I'll have to try it again.
I don't actually have any speakers plugged into it, so it shouldn't be an audio thing. Some online sites said that disabling ipv6 could help, but I couldn't actually figure out how to do that (an example of why linux isn't ready for the masses).
Re: slow network transfer
Posted: Fri Feb 29, 2008 12:00 pm
by UberGerbil
notfred wrote:I suspect the problem is on the receiving end, Vista has known issues with gigabit networks if there is any sound at the same time.
Except if I understand it correctly, Ubuntu is doing the copying, and Vista is just offering a dumb file share. Also, that
issue would only occur if he was playing media, and it still wouldn't be that bad (unless he had 3 NICs, because that problem was compounded by each NIC in the system, though I believe that bug was fixed in SP1).
Have you seen any transfers that max out your gigE connection? Even though Vista thinks it has one, are you sure it's 1Gb/s from end to end? Have you turned on jumbo frames?
Re: slow network transfer
Posted: Fri Feb 29, 2008 12:14 pm
by Usacomp2k3
UberGerbil wrote:Have you seen any transfers that max out your gigE connection? Even though Vista thinks it has one, are you sure it's 1Gb/s from end to end? Have you turned on jumbo frames?
I know that I've done >100mbit before in this configuration. (well pulling the file instead of pushing) On my desktop I've done 800mbit/sec at the last LAN party. (hosting a file pulled by multiple people that apparently was just being cached from RAM).
I haven't turned on jumbo frames because I wasn't sure how well the switch could handle splitting frames up, but more importantly, it's not easy to do in linux.
Re: slow network transfer
Posted: Fri Feb 29, 2008 12:18 pm
by UberGerbil
While I was digging up that blog post about the media bug I found this about
file copies that covers old behavior as well as SP1 (which I think you said you have?) but I only skimmed it so I don't know if there's anything in there that applies to this situation. Have you tried using that other NIC just for laughs?
Re: slow network transfer
Posted: Fri Feb 29, 2008 12:20 pm
by Usacomp2k3
Yeah, I installed SP1 (got it from work) Wedn night. I didn't try the other NIC, but will when I get a chance.
Re: slow network transfer
Posted: Fri Feb 29, 2008 12:50 pm
by notfred
Usacomp2k3 wrote:I haven't turned on jumbo frames because I wasn't sure how well the switch could handle splitting frames up, but more importantly, it's not easy to do in linux.
This isn't a jumbo frames issue, 1500 byte frames will be plenty for this, the drives should be the bottleneck if everything else is working properly.
BTW it's easy to set the MTU in Linux, "sudo ifconfig eth0 mtu 9000" or whatever you want the MTU to be. If you want to make it stick over reboot, just add the "mtu 9000" to the entry for eth0 in /etc/network/interfaces.
Re: slow network transfer
Posted: Fri Feb 29, 2008 11:08 pm
by Usacomp2k3
Jumbo frames didn't do anything, and it's the same speed copying both directions...trying other NIC now...
That didn't do anything either. I transferred stuff from my laptop just to give a comparison. That was reaching upwards of 9.8% utilization which is pretty much the max of the laptops' 10/100
I'm going to forcing the link speed of the linux card to see what that does.
Re: slow network transfer
Posted: Fri Feb 29, 2008 11:31 pm
by Usacomp2k3
Well setting the link speed to 100 full now has it running @ 4% according to my desktop. *sigh*
Re: slow network transfer
Posted: Fri Feb 29, 2008 11:58 pm
by Kraft75
Can you try directly connecting both computers together using a crosswired connection? You'd at least eliminate the switch as the possibility.
Re: slow network transfer
Posted: Sat Mar 01, 2008 12:14 am
by Usacomp2k3
Not easily. They are on opposite sides of the room and it's really not worth the effort to put them within reach of the 3' crossover cable I have and then configure static IP's and such (and yes, I understand that gigabit should do auto MDI/MDI-X, but still).
Re: slow network transfer
Posted: Sat Mar 01, 2008 12:33 am
by aristheo
Have you tried replacing the cables? Maybe they're just bad. It could be pinched somewhere or something. Also, make sure they're at least Cat5e cables. I would also try reinstalling the drivers on your Vista box. Maybe they just need to be updated.
Re: slow network transfer
Posted: Sat Mar 01, 2008 12:34 am
by Usacomp2k3
Considering that I can transfer perfectly fine from laptop->desktop and have a problem server-> desktop, I'm assuming something is wrong on the server end.
How does one disable ipv6 in Ubuntu?
Re: slow network transfer
Posted: Sat Mar 01, 2008 2:02 am
by Kraft75
You say laptop->desktop is not a problem, but what about laptop->server? And vice-versa?
Re: slow network transfer
Posted: Sat Mar 01, 2008 2:31 am
by bitvector
It's possible you could be limited by SMB 1.0 (without tuning). I'm not too familiar with SMB internals, but I believe SMB 1.0 had pretty low buffer sizes and other quirks which would limit it to much less than wire speed on a gigabit connection (versus something like HTTP). Vista SP1 talks the new SMB 2.0, which has improved these things, but Samba/Linux CIFS don't talk SMB2. There are some Samba/CIFS tunables you can tweak to see if it helps (like SNDBUF/RCVBUF sizes). I remember having to do that to get better performance out of Samba.
Usacomp2k3 wrote:Some online sites said that disabling ipv6 could help, but I couldn't actually figure out how to do that (an example of why linux isn't ready for the masses).
I seriously doubt it has anything to do with IPv6. Some people used to recommend disabling IPv6 for one particular reason: to deal with high latency DNS queries in Firefox, because it would send out IPv6 DNS queries and some home routers/crappy DNS servers would act weird, causing high latency for DNS resolution.
You can prevent the module from loading by adding:
to /etc/modprobe.d/00local (it may not exist).
Usacomp2k3 wrote:I haven't turned on jumbo frames because I wasn't sure how well the switch could handle splitting frames up, but more importantly, it's not easy to do in linux.
A layer 2 switch won't split frames up; if non-jumbo capable devices are on the same segment, it just won't work. But you shouldn't need jumbo frames to get higher transfer rates than 8-9MB/sec in general.
Re: slow network transfer
Posted: Sat Mar 01, 2008 8:38 am
by notfred
You could try checking things with iperf
On the Ubuntu box, "sudo apt-get install iperf", on the Windows box download it from
http://dast.nlanr.net/Projects/Iperf/#downloadYou run "iperf -s" on one end and it starts a server that will sit there listening until you Ctrl-C it. On the other end you run "iperf -c <IP of server>". It will do a transfer for 10s and report the speed. That should tell you if you have a fundamental network problem or if it is something in the Samba side of things. Try it both ways round.
One thought I did have is maybe the PCI network card is sharing an interrupt with something that is badly behaved and rate limiting the interrupts (all IRQ service routines need to be called for the shared devices). "cat /proc/interrupt" should show who is sharing with the network interface <eth0>.
Re: slow network transfer
Posted: Sat Mar 01, 2008 11:56 am
by Usacomp2k3
bitvector wrote:It's possible you could be limited by SMB 1.0 (without tuning). I'm not too familiar with SMB internals, but I believe SMB 1.0 had pretty low buffer sizes and other quirks which would limit it to much less than wire speed on a gigabit connection (versus something like HTTP). Vista SP1 talks the new SMB 2.0, which has improved these things, but Samba/Linux CIFS don't talk SMB2. There are some Samba/CIFS tunables you can tweak to see if it helps (like SNDBUF/RCVBUF sizes). I remember having to do that to get better performance out of Samba.
Maybe I'll look into that down the road. Doesn't really sound like something I want to learn right now.
bitvector wrote:Usacomp2k3 wrote:Some online sites said that disabling ipv6 could help, but I couldn't actually figure out how to do that (an example of why linux isn't ready for the masses).
I seriously doubt it has anything to do with IPv6. Some people used to recommend disabling IPv6 for one particular reason: to deal with high latency DNS queries in Firefox, because it would send out IPv6 DNS queries and some home routers/crappy DNS servers would act weird, causing high latency for DNS resolution.
Ok. That makes sense. Nevermind then
bitvector wrote:Usacomp2k3 wrote:I haven't turned on jumbo frames because I wasn't sure how well the switch could handle splitting frames up, but more importantly, it's not easy to do in linux.
A layer 2 switch won't split frames up; if non-jumbo capable devices are on the same segment, it just won't work. But you shouldn't need jumbo frames to get higher transfer rates than 8-9MB/sec in general.
That sounds like what I was remembering. It didn't work anyway.
Thanks for your thoughts/explanations.
Re: slow network transfer
Posted: Sat Mar 01, 2008 11:59 am
by Usacomp2k3
notfred wrote:You could try checking things with iperf
On the Ubuntu box, "sudo apt-get install iperf", on the Windows box download it from
http://dast.nlanr.net/Projects/Iperf/#downloadYou run "iperf -s" on one end and it starts a server that will sit there listening until you Ctrl-C it. On the other end you run "iperf -c <IP of server>". It will do a transfer for 10s and report the speed. That should tell you if you have a fundamental network problem or if it is something in the Samba side of things. Try it both ways round.
Ok. Preliminary testing gave me 274mbit/sec connection to my desktop and 90.8mbit/sec to my laptop. Considering that I'm currently copying files to the laptop that sounds quite feasible.
I'll do a more exhaustive test once that file transfer finishes.
notfred wrote:One thought I did have is maybe the PCI network card is sharing an interrupt with something that is badly behaved and rate limiting the interrupts (all IRQ service routines need to be called for the shared devices). "cat /proc/interrupt" should show who is sharing with the network interface <eth0>.
That command didn't work, but I think I figured out what you meant, I'll update on that shortly.
EDIT: ok, so irq16 is used by "eth0" and "via@pci:0000:01:00.0" whatever that is. Time to hit up google.
EDIT2: so apparently that is the onboard video card. The "linux.sysfs_path" returns "/sys/devices/pci0000:00/0000;00:01.0/0000:01:00.0"
So I guess maybe I should switch it to a different PCI slot. Or can I manually change the IRQ.13 is open, as are 2, 3, 4, 5, 7, 10, 11, 13
Re: slow network transfer
Posted: Sat Mar 01, 2008 12:25 pm
by Usacomp2k3
So just to confirm the iperf shows 784 to the desktop and 92.2 to the laptop. So it's gotta be a samba problem. I'd use my WinSCP to transfer over ssh, but it's slower.
I'm pretty much done copying the data that needs to be copied, so it's all playing with theory from this point on. Still helpful knowledge, IMHO.
Re: slow network transfer
Posted: Sat Mar 01, 2008 1:23 pm
by mattsteg
Usacomp2k3 wrote:So just to confirm the iperf shows 784 to the desktop and 92.2 to the laptop. So it's gotta be a samba problem. I'd use my WinSCP to transfer over ssh, but it's slower.
I'm pretty much done copying the data that needs to be copied, so it's all playing with theory from this point on. Still helpful knowledge, IMHO.
In the future, maybe try setting up an FTP server on the server? That'd likely run faster. Of course, if performance on a day-to-day basis is an issue investigating samba tweaks makes a lot of sense, although for those to be especially useful you need to figure out how to create shares properly, I suppose.
Re: slow network transfer
Posted: Sat Mar 01, 2008 1:34 pm
by Usacomp2k3
mattsteg wrote:Usacomp2k3 wrote:So just to confirm the iperf shows 784 to the desktop and 92.2 to the laptop. So it's gotta be a samba problem. I'd use my WinSCP to transfer over ssh, but it's slower.
I'm pretty much done copying the data that needs to be copied, so it's all playing with theory from this point on. Still helpful knowledge, IMHO.
In the future, maybe try setting up an FTP server on the server? That'd likely run faster. Of course, if performance on a day-to-day basis is an issue investigating samba tweaks makes a lot of sense, although for those to be especially useful you need to figure out how to create shares properly, I suppose.
Thanks for the jab there
I tried setting up an ftp server back in the 6.04 days, but never was successful. I hadn't really considered it since then due to most of my needs being satisifed with ssh/WinSCP. Now that I'm in the process of getting Windows Home Server installed on the machine, shares should be much easier to manage.
Re: slow network transfer
Posted: Sat Mar 01, 2008 2:20 pm
by mattsteg
Usacomp2k3 wrote:mattsteg wrote:Usacomp2k3 wrote:So just to confirm the iperf shows 784 to the desktop and 92.2 to the laptop. So it's gotta be a samba problem. I'd use my WinSCP to transfer over ssh, but it's slower.
I'm pretty much done copying the data that needs to be copied, so it's all playing with theory from this point on. Still helpful knowledge, IMHO.
In the future, maybe try setting up an FTP server on the server? That'd likely run faster. Of course, if performance on a day-to-day basis is an issue investigating samba tweaks makes a lot of sense, although for those to be especially useful you need to figure out how to create shares properly, I suppose.
Thanks for the jab there
What jab?
Usacomp2k3 wrote:I tried setting up an ftp server back in the 6.04 days, but never was successful. I hadn't really considered it since then due to most of my needs being satisifed with ssh/WinSCP. Now that I'm in the process of getting Windows Home Server installed on the machine, shares should be much easier to manage.
Whatever works for you. Setting up a temp ftp server shouldn't really take that long, though.
Re: slow network transfer
Posted: Sat Mar 01, 2008 11:44 pm
by Usacomp2k3
For the record, With Windows Home Server installed on the same hardware (I did move the NIC over to the next PCI slot), I easily hit 40% transferring a small file from server->desktop.