TR Forums

So, I understand the principle behind what a DDOS attack is, shove enough garbage data requests at an online service's door and legitimate traffic can't find its way in. Its not too different from the old Ping of death, only these attacks use multiple systems and more beefy hardware to achieve the same effect against modern networking technology. My main question that I can't find information on is, what part of the online service's network is getting bottle-necked. Is it the firewall, the ISP's trunk or routing equipment leading into the facility, the login server/database, internal routing equipment?

Anovoca wrote:

So, I understand the principle behind what a DDOS attack is, shove enough garbage data requests at an online service's door and legitimate traffic can't find its way in. Its not too different from the old Ping of death, only these attacks use multiple systems and more beefy hardware to achieve the same effect against modern networking technology. My main question that I can't find information on is, what part of the online service's network is getting bottle-necked. Is it the firewall, the ISP's trunk or routing equipment leading into the facility, the login server/database, internal routing equipment?

In a "normal" DDOS attack, it's a SYN flood against a server. Remember how TCP uses the three way handshake (syn, syn ack, ack)? The DDOS will use multiple machines to initiate the handshake, but never finish it. It basically overwhelms the servers ability to accept legit requests.

There are, of course, other ways of doing it, but that's the most common. Ping 'o death just floods the server with icmp echos, which some tools allow you to define how big the icmp packets are, and how often they hammer the device with.

It is possible to directly attack the ISP's trunk and routing equipment (overflowing the CAM table, for instance), but with the type of equipment they use, it's harder to successfully pull off before they "blackhole" you.

Anovoca wrote:

So, I understand the principle behind what a DDOS attack is, shove enough garbage data requests at an online service's door and legitimate traffic can't find its way in. It

Not necessarily.

There are techniques that abuse the protocols in various ways as force multipliers, but the core of disruption is what the first D stands for: distributed.

In other words, if I have enough computers and connections at my disposal, all over the place (like a botnet), I can simply flood any given address with enough traffic to swamp it, and that's inherently difficult to stop.

Anovoca wrote:

Its not too different from the old Ping of death

It really is though. The Ping of Death was a abuse of the protocol plus its implementations that could result in the receiving computer just crashing because it couldn't properly process a single (but huge and massively fragmented) packet. It's been obsolete for well over a decade.

Anovoca wrote:

only these attacks use multiple systems and more beefy hardware to achieve the same effect against modern networking technology.

Again, essentially it is just a matter of sending more traffic than the receiving connection can handle.

Anovoca wrote:

My main question that I can't find information on is, what part of the online service's network is getting bottle-necked. Is it the firewall, the ISP's trunk or routing equipment leading into the facility, the login server/database, internal routing equipment?

Well, that entirely depends. Typically, it's just a matter of too much traffic for either the connection's bandwidth or the TCP stack to handle, usually the former in this day and age but most of the force multiplier techniques rely on overwhelming the TCP Stack in various ways. But typically we're talking about just a general brute force denial technique from standard tools, not something specifically targeted. There isn't any reason a more dedicated/focused attacker couldn't try and use your service in such a way that any of its components became the bottleneck well before the bandwidth/TCP Stack did. That would be more economical in terms of packets/bandwidth, but as I said it's less economical in terms of the attacker's personal time.

It is possible to directly attack the ISP's trunk and routing equipment (overflowing the CAM table, for instance), but with the type of equipment they use, it's harder to successfully pull off before they "blackhole" you.

I suppose there really isn't a point to doing this either since if you disrupt the communication link in a way that already authenticated users must reestablish a connection, then the surge of legit traffic attempting to re-validate at once will overwhelm the ISP trunk for you.

I guess I was trying to understand why it is so difficult to prevent. I would think that any login service that uses a local client to cache credentials could require some form of dynamically generated yet manually inputted token to even begin the handshake process. It wouldn't prevent things completely but it would require the attacker to manually intervene in the process and if successful in bringing down the network, they would have to do it again.

Anovoca wrote:

It is possible to directly attack the ISP's trunk and routing equipment (overflowing the CAM table, for instance), but with the type of equipment they use, it's harder to successfully pull off before they "blackhole" you.

I suppose there really isn't a point to doing this either since if you disrupt the communication link in a way that already authenticated users must reestablish a connection, then the surge of legit traffic attempting to re-validate at once will overwhelm the ISP trunk for you.

I guess I was trying to understand why it is so difficult to prevent. I would think that any login service that uses a local client to cache credentials could require some form of dynamically generated yet manually inputted token to even begin the handshake process. It wouldn't prevent things completely but it would require the attacker to manually intervene in the process and if successful in bringing down the network, they would have to do it again.

ISP trunks are a LOT fatter than you'd realize. I used to install aggregators in COs, and even the small ones had 100gbps backplanes. The giant switches they use can easily eclipse that speed, especially if they distribute CAM and routing table entries to secondary/tertiary cabinets. Those can accomodate terabit speeds. And that's not counting IDS/IPS systems, nor DPI used to spot, mark, and drop traffic like that.

You can't really token-ize the TCP handshake, since that's how the protocol works by design. Even just viewing a webpage involves that handshake. That's one of the mechanisms it uses to guarantee packet delivery. If you tried to force every machine that wanted to connect to your resource to wait for a token, for it's turn to "talk", it would slow things down to a crawl anyway. There's a reason Token Ring fell out of favor.


Ideally, mitigating a DDOS would involve determining the sources, then dropping their traffic as close as possible to their point of entry to the internet. That, or just blackhole the IP block the attack is coming from.

True, i was more thinking in terms of authentication and connection servers being attacked. Specifically SPN, XBN, and blizzard that were being attacked this weekend. If the network is guarded by a firewall that only accepts traffic initiated via the local client launchpad, I would think they could filter the traffic using the services already in place. But my understanding of network protocols are a bit rusty so I am probably looking at this at a wrong angle.

Anovoca wrote:

True, i was more thinking in terms of authentication and connection servers being attacked. Specifically SPN, XBN, and blizzard that were being attacked this weekend. If the network is guarded by a firewall that only accepts traffic initiated via the local client launchpad, I would think they could filter the traffic using the services already in place. But my understanding of network protocols are a bit rusty so I am probably looking at this at a wrong angle.

Yeah, there's definitely a lot more going on behind the scenes (ISP-wise) than most folks realize. I don't know exactly how Sony, MS, or Blizzard have their systems set up (they're probably in co-location facilities), but even then, they have plenty of options. They need to work more closely with their partners, since co-locations aren't exactly defenseless, either. Back when I worked for a WISP, the higher-ups moved our authentication/accounting devices to a co-location in georgia (stupid idea since it was a single point of failure, but I got overruled), and when I'd chat with the facility employees, they'd point out all the methods they had to mitigate such attacks. My bosses were just too cheap to employ those, since that would've meant less cocaine or yachts for 50 or so VPs.

But ultimately, they can't go too crazy protecting the auth/connect servers for online services, since that could prevent new customers from signing up and using said service. Their best bet is to have mechanisms in place to spot and drop attacks in real-time, and block them as close to the source as possible.

While there are some (distributed) denial of service attacks that rely on overwhelming the target server's memory capacity or CPU power, many of them these days just rely on raw bandwidth to simply saturate the target server/network's connection to the internet. This is made somewhat easier by the distributed aspect because you can focus the collective bandwidth of many machines onto a single point (even moreso with amplification techniques).

Finding the bottleneck for a (D)DoS attack depends on the kind of attack.

If the login page to your website has to do a lot of CPU work to process, then I can overload the CPU of the server by sending tons of bogus login attempts to it.
A classical SYN flood would exhaust the memory available to store the list of clients awaiting ACK replies in the network stack (this is less prevalent now with computers having several GB of RAM or more).
If I ping the router at the front of your internet connection and it's not configured properly, I can overload the CPU on that, leading to a (near) total loss of connectivity through that router.
If I can produce 20 GBit/s of traffic and your server/network only has a 10 GBit/s connection to the Internet, then I can simple knock it off that way.

Most of the brute-force DoS attacks can be mitigated through filtering by your upstream ISP (or even your ISP's upstream ISP) as they tend to have a few orders of magnitude more bandwidth available.

Hz so good wrote:

It is possible to directly attack the ISP's trunk and routing equipment (overflowing the CAM table, for instance), but with the type of equipment they use, it's harder to successfully pull off before they "blackhole" you.

If they are actually on your LAN or anything else in Layer 2 land, we've clearly completely left the realm of "Distributed" DOS and entered into an entirely different kind of situation altogether. This really isn't the same class of problem...

Hz so good wrote:

ISP trunks are a LOT fatter than you'd realize. I used to install aggregators in COs, and even the small ones had 100gbps backplanes. The giant switches they use can easily eclipse that speed, especially if they distribute CAM and routing table entries to secondary/tertiary cabinets. Those can accomodate terabit speeds. And that's not counting IDS/IPS systems, nor DPI used to spot, mark, and drop traffic like that.

That isn't particularly relevant to DDOS attacks. You are in a "The call is coming from inside the house!!!111" sort of horror-story, and, to extend the metaphor, at that point are you particularly worried that the phone is busy?

Hz so good wrote:

You can't really token-ize the TCP handshake, since that's how the protocol works by design. Even just viewing a webpage involves that handshake. That's one of the mechanisms it uses to guarantee packet delivery. If you tried to force every machine that wanted to connect to your resource to wait for a token, for it's turn to "talk", it would slow things down to a crawl anyway.

That's not really true, there are all sorts of things like syncookies, firewall extensions like synproxy, and switches/routers that do delayed binding.

Hz so good wrote:

There's a reason Token Ring fell out of favor.

Not the same sort of thing at all, that's layer 2. TCP handshaking is layer 4.

Hz so good wrote:

Ideally, mitigating a DDOS would involve determining the sources, then dropping their traffic as close as possible to their point of entry to the internet.

Which is clearly not even remotely possible and thus irrelevant.

Hz so good wrote:

That, or just blackhole the IP block the attack is coming from.

The first "D" in DDOS stands for what, again?

Anovoca wrote:

If the network is guarded by a firewall that only accepts traffic initiated via the local client launchpad

I don't know the details of the Blizzard attack, but if we're talking bandwidth exhaustion that doesn't matter, and most resource exhaustion occurs at lower levels than that. Unless the attackers are doing application-level attacks (which are generally more uncommon and unlikely in this specific case) that wouldn't help.

As a simplification, if the attackers are sending so many packets that Blizzard's servers cannot read them fast enough, does it matter what any of those packets actually say (such as a data payload that's been digitally-signed as authentic)?

Hz so good wrote:

Their best bet is to have mechanisms in place to spot and drop attacks in real-time, and block them as close to the source as possible.

Again, not really possible. I mean, if it's any DDOS of note that would mean essentially creating your own self-inflicted DOS because you blackholed entire swathes of the internet, which contain millions of your paying customers.

Classic SYN floods are not very effective these days as there are techniques such as SYN cookies to limit the resources consumed in a SYN attack whilst still allowing legitimate connections to proceed.

Routers handle ICMP traffic at the lowest priority, you are unlikely to affect anything by pinging it unless there is a bug like the one that led to the Ping Of Death.

Most attacks these days are either simply overwhelming the pipe or are targetting a particular vulnerability.

And several of the well known mitigation services can use various combinations of ISP supported as well as local boxes to filter specific traffic, patterns and behavior. One known service basically has several BGP peers to your site, and in case of an attack, routes your prefix through that service and filter the attack traffic while letting legitimate traffic to continue. Most of those services can be configured to filter just about anything from signatures, even reg-ex based on the fly entered ones, as well as a combination of protocols or behaviors that shouldn't be there. So for any attack to really bypass those systems, the thing would be to spoof traffic that seem legitimate.

Several of the current attacks, while being distributed take this approach, take slowloris for instant, it basically opens a standard HTTP towards a webserver, but never completes the handshake, thus filling the connection table. Of course this can be mitigated as well decently, easy, but it does requires some fine tuning since they are abusing a protocol, but it is harder then just blackholing or filtering a dns amplification attack for instance.

My take on the DDOS on the gaming stuff is either that they basically just try to overwhelm the target with bandwidth, in case of twitch for instance, or trying to spoof something that will abuse the servers in case of blizzard, EA, etc.


For anybody interesting in what is going on right now, I can recommend this site, it's basically a real time map of what's going on, supplied by one of those vendors selling ddos mitigation tools(arbor network), which I incidentally have training on some of their products.

http://www.digitalattackmap.com/

Do note the prominence of china, the states as well as some European countries like Great Britain and France as well in those maps.

Anovoca wrote:

I guess I was trying to understand why it is so difficult to prevent. I would think that any login service that uses a local client to cache credentials could require some form of dynamically generated yet manually inputted token to even begin the handshake process. It wouldn't prevent things completely but it would require the attacker to manually intervene in the process and if successful in bringing down the network, they would have to do it again.

It's difficult because there's many layers in place in networking. An attack at the handshake (transport or network layer?) happens well below the application layer. You can't just jump from one to another, so any authentication system would have to keep that in mind.



Of course they do attack at the database/application layer as well. If they can't easily overwhelm or brute force the hardware then just move onto the software...

UDP based DNS and/or NTP amplification seems to be the hot choice these days for DDoS.

Glorious wrote:

...

Yeah I totally munged that up. I blame the cold medicine I'm on.

Not believable enough? Well,...um...er... LOOK BEHIND YOU! IT'S A THREE-HEADED MONKEY!!!

*runs like hell out of the thread*

Hz so good wrote:

Yeah I totally munged that up. I blame the cold medicine I'm on.

Don't diss a good NyQuil buzz.

Captain Ned wrote:

Hz so good wrote:

Yeah I totally munged that up. I blame the cold medicine I'm on.

Don't diss a good NyQuil buzz.

My Doc wrote me a scrip for the good stuff with codiene in it, so I've been in a fog the last couple days. Being on Klonopin at the same time probably compounded the issue.

Still, I should've known better than to spout out incorrect info upthread. I'm gonna limit my online activities until I get off this stuff, before I make and even bigger idiot out of myself.