How do you troubleshoot a slow LAN efficiently?

The network is the forum.

Moderators: Steel, notfred

How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 12:44 pm

I have seem to have a lot of trouble with LAN transfers going slow. Especially FTP. I can often download files off the net, faster than I can transfer them between PC's on the LAN.

Now I realize where some of the bottlenecks could be. Software firewall(s), router, wireless connections, etc.

But what I'm wondering, is there an efficient way (that's beginner/intermediate friendly) to find out exactly where the bottlenecks are? Rather than a haphazard trial and error approach of disabling firewalls, swapping routers, replacing wireless connections with wired, etc?

I know over the web there is the "tracert" command which traces a packets path to its destination, telling the delay between each connection. Is there any similar command, or software tool that can trace packets across a LAN to determine the bottleneck? Or some other efficient way to troubleshoot?
mxmaniac
Gerbil In Training
 
Posts: 5
Joined: Fri Jun 27, 2014 12:34 pm

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 12:55 pm

First of all don't touch anything.

Start by running a pure network benchmark between various machines on the LAN. I recommend iperf, you run "iperf -s" on one machine to act as a server and then "iperf -c <server>" on the other. This will tell you if the problem is in the actual networking itself or in something related to the file transfers.

If this all returns good throughput then your problem is going to be somewhere further up the networking stack in the file transfer stuff

If this returns bad throughput then by running it between different combinations of machines and which end is server and which end is client you should be able to isolate down to a particular PC / cable / switch port. Now you get to start swapping parts. By swapping switch ports, cables and network cards between a good one and the bad one, and doing them one at a time and in that order with a retest in between each switch, then you will find the bad part.

If all that fails, report back and we'll give you more homework :D
notfred
Grand Gerbil Poohbah
 
Posts: 3772
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 1:16 pm

mxmaniac wrote:I have seem to have a lot of trouble with LAN transfers going slow. Especially FTP. I can often download files off the net, faster than I can transfer them between PC's on the LAN.

Now I realize where some of the bottlenecks could be. Software firewall(s), router, wireless connections, etc.

But what I'm wondering, is there an efficient way (that's beginner/intermediate friendly) to find out exactly where the bottlenecks are? Rather than a haphazard trial and error approach of disabling firewalls, swapping routers, replacing wireless connections with wired, etc?

I know over the web there is the "tracert" command which traces a packets path to its destination, telling the delay between each connection. Is there any similar command, or software tool that can trace packets across a LAN to determine the bottleneck? Or some other efficient way to troubleshoot?



What equipment comprises your LAN? I can think of TONS of options for testing, but it'll all depend on what equipment you are using. If your gear is capable of SNMP traps, and netflow, you can use the demo of PRTG to get a very clear picture of what's going on.

In fact, can you draw us a network diagram, labelling how the equipment interconnects, and what equipment models are preset, and if you've made any custom changes to the network layout?

There are more than a few network pros on here, so I'm sure we can help you out. :)
Hz so good
Gerbil Elite
 
Posts: 731
Joined: Wed Dec 04, 2013 5:08 pm

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 1:59 pm

I looked up iperf and like what I see. A simple lightweight tool for windows, linux, and android. I will plan on trying this out soon, seems like a great tool. From what I can tell though it seems to primarily measure throughput and quality, but doesn't tell exactly where the bottleneck is (unsure if this is possible though).

As far as a network map, its a pretty simple typical home setup. Router is a Buffalo (I believe wrt54g) flashed with Tomato, although I am already replacing this very soon with a much more powerful (cpu / ram wise), wireless n, gigabit router, as I suspect its the weak point, and I sometimes have to reboot it. Then there are a small number of laptops, phones, IP cameras hooked up. 4 devices are wired using the router ports, rest wireless.

I'm not yet familiar with SNTP traps, netflow, or PRTG. Rather than ask a ton of questions on here, I'll plan to set aside some google/youtube time later this weekend to try to learn about them. How would I know if my equipment is compatible? Can they actually identify the bottleneck? Like trace the packet through its journey across the lan, and find out where it slows down, such as the router, the wireless connection, etc?
mxmaniac
Gerbil In Training
 
Posts: 5
Joined: Fri Jun 27, 2014 12:34 pm

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 2:22 pm

mxmaniac wrote:I looked up iperf and like what I see. A simple lightweight tool for windows, linux, and android. I will plan on trying this out soon, seems like a great tool. From what I can tell though it seems to primarily measure throughput and quality, but doesn't tell exactly where the bottleneck is (unsure if this is possible though).

As far as a network map, its a pretty simple typical home setup. Router is a Buffalo (I believe wrt54g) flashed with Tomato, although I am already replacing this very soon with a much more powerful (cpu / ram wise), wireless n, gigabit router, as I suspect its the weak point, and I sometimes have to reboot it. Then there are a small number of laptops, phones, IP cameras hooked up. 4 devices are wired using the router ports, rest wireless.

I'm not yet familiar with SNTP traps, netflow, or PRTG. Rather than ask a ton of questions on here, I'll plan to set aside some google/youtube time later this weekend to try to learn about them. How would I know if my equipment is compatible? Can they actually identify the bottleneck? Like trace the packet through its journey across the lan, and find out where it slows down, such as the router, the wireless connection, etc?




SNMP (Simple Network Management Protocol) is used for monitoring, and can be configured to poll interfaces/devices at a given rate, to see what's most active, what's down, etc...

Netflow may only show up on the larger equipment, but it's extremely easy to see which stations are talking, what protocols are they using, who's the most active and consuming the most bandwidth.

PRTG is just a tool that takes those stats, and presents them in nice, easy to read charts. It's paid software out of germany, but there is a trial version available. I prefer it to costlier solutions, like Solarwinds.

If it SNMP and Netflow aren't mentioned in the the manual/guides for your gear (or Tomato), they probably don't support it.

Do you have logging enabled on the Buffalo? That might help you pinpoint issues (speed/duplex mismatches, incorrectly configured NAT Overload, DDoS attack attempts might show up there). Does the Buffalo support traffic monitoring? If you're going to get a gigabit router, make sure you're using UTP cable that supports gigabit speeds to any end devices that will be wired. Cat5e and Cat6 cable will work fine.

I haven't used their gear, other than Wireless AP/Bridge Repeaters, but RouterBOARD makes a reasonably priced Multi-layer gigabit switch with built-in 802.11n radios for $209 MSRP. Each switchport is gigabit, and can be changed to routed ports, as needed.

Kind like my Cisco 3550. I can turn any of the 24 ports into either a switchport or a routed port (supporting all the usual routing protocols), and I can send power out any port, to provide Power over Ethernet (802.3af, NOT the new 802.3at) for Cisco IP Phones, or certain models of Cisco Access Points. Plus all the fancy netflow, CEF (for route once, switch many), VLANs, QoS, traffic shaping, and other neat features. You can even turn the Buffalo into a transparent bridge, and let the 3550 handle all the PPPoE authentication and routing chores. If you don't mind it being mostly 100Mbps ports, with only 2 GigEth GBIC ports (one transmit, one recieve only), you can get a used C3550 for like $80.
Hz so good
Gerbil Elite
 
Posts: 731
Joined: Wed Dec 04, 2013 5:08 pm

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 3:07 pm

I thought the idea here was to find the performance bottleneck rather than to practice for a CCIE.

The idea with running iperf tests between all the different devices is that hopefully you'll notice a pattern that if a particular device is involved then that is the cause of the slowdown. If these all show good throughput but the file transfers still show bad throughput then you need to start doing the same thing at that level. Don't condemn the router yet, it could all just be a bad cable somewhere!
notfred
Grand Gerbil Poohbah
 
Posts: 3772
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 3:17 pm

notfred wrote:I thought the idea here was to find the performance bottleneck rather than to practice for a CCIE.

The idea with running iperf tests between all the different devices is that hopefully you'll notice a pattern that if a particular device is involved then that is the cause of the slowdown. If these all show good throughput but the file transfers still show bad throughput then you need to start doing the same thing at that level. Don't condemn the router yet, it could all just be a bad cable somewhere!



Dude, at the price level for the gear I'm talking about, it's absolutely crazy how much bang you get for your buck, and how much control you get. I don't know exactly how nerdy the submitter is, maybe they LIKE having that kind of ability.

It's not like I'm holding a gun to his/her head. If they wanna grab a netgear R6100, go for it. It's a solid device, it has *some* of the features I was talking about, and was so easy even my grandmother was able to set her's up.

My point about making sure they run GigE rated cable to the wired devices is still valid, though. Unless they don't mind the switchboard on the Router/AP either auto-detects at 100, or starts throwing speed-mismatch errors.

Sheesh...
Hz so good
Gerbil Elite
 
Posts: 731
Joined: Wed Dec 04, 2013 5:08 pm

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 3:36 pm

Oh, I forgot to mention, but having a 24 port gig-e Multi-layer switch, even without a built in AP, is an awesome way to be able to "future-proof" your home network for a while. .11ac APs usually come with 2 GigE ports, the next gen APs will have 4 or more. That'll take up most of the switchports on the back of commodity WRAPs (typically 5, not counting WAN port).

And besides, being able to run 1Gig ports all over the house (even the kitchen!) is just awesome. :)
Hz so good
Gerbil Elite
 
Posts: 731
Joined: Wed Dec 04, 2013 5:08 pm

Re: How do you troubleshoot a slow LAN efficiently?

Postposted on Fri Jun 27, 2014 4:10 pm

You bisect it, but first take some performance metrics to make sure you aren't creating regressions.

Linux, mtr, and iperf would be the best tools for diagnosing performance bottlenecks on the cheap. They won't tell you what's wrong, but they will show you where the problem is.

1: Boot into Linux on the "client" and "server" then attached then onto the opposite ends of the network.
2: Figure out if something is dropping packets and get a latency baseline with mtr. Anything under 1ms is good.
3: Run iperf to get a baseline for performance. I like running "iperf -smNi 1 -f m -p 5001" for the server and "iperf -mNi 1 -f m -c <server.ip.address> -d -t 60" on the client.
4: Bisect the network to see if the problem remains or if it is gone.
Flatland_Spider
Gerbil Elite
 
Posts: 873
Joined: Mon Sep 13, 2004 8:33 pm
Location: The 918/539


Return to Networking

Who is online

Users browsing this forum: No registered users and 3 guests