Subject: Re: Internet SYN Flooding, spoofing attacks

From: Vernon Schryver <[email-masked]>
Date: Tue, 15 Feb 2000 10:22:46 -0700 (MST)

> From: [email-masked]

> ...
> Well.. as soon as somebody presents us with "the real solution", we'll start
> implementing.  In the meantime, filtering is the best we know how to do.
> ...

I really wish "we" actually knew how to filter.

Just as I feared when the news broke, I'm seeing more paths where neither
traceroute nor ping work, apparently because some of "us" are so expert
that we turn off all of ICMP, and never mind intermittent blackholes from
Path MTU Discovery or diagnosing routing or other mere technical problems.

Vernon Schryver    [email-masked]

---

From: "Michael H. Warfield" <[email-masked]>
Date: Tue, 15 Feb 2000 13:59:58 -0500

> Path MTU Discovery or diagnosing routing or other mere technical problems.

	But some of us turn off ICMP except for ICMP_FRAG_NEEDED and keep
MTU discovery alive while cutting off the ICMP food fights and script
kiddie probes that seem to be endemic in our current mess.

	You betcha traceroute and ping are broken (figuratively and
literally).  Just as broken as I can make them.  You don't need to be
tracerouting or pinging into my network and that's my choice to make.
I can break them without breaking MTU discovery.  You want to diagnose
routing or other technical problems, why aren't you contacting me?

	You cut off ICMP_ECHOREPLY and all but a tightly restricted set
of UDP and you have just starved these DDoS zombies of the vast majority
of their communications facility.  TCP is much easier to trace, if they
fall back to that (I don't see how TFN2K could - it utilizes a blind
forward-only non-responding channel).  Blocking spoofed packets won't
do nearly as good a job since a lot of the packets aren't spoofed.  Doesn't
help at all in some examples I have some first hand experience with.

	Edge filtering and spoof protection would have been absolutely no
help for one site I was help with forensics after TFN2K.  They were a
source of ICMP_ECHOREPLY packets in one of the storms.  We STILL
haven't located the zombie amongst the hundreds of potential Windows
boxes (we already cleared the single Solaris and single RedHat box on
the subnet and TFN2K is known to run on Windows).

	No spoofed packets crossed router boundries.  The TFN2K command
sequence was executed well before anyone noticed any bandwidth anomolies,
so there's no track back to the master.  (Who would have noticed 20
ICMP_ECHOREPLY packets that merely didn't correspond to any internal
request?)  The slaves shut down after two hours and before the admins
realized that the smurf was being generated internally.  They did find a
bug in a router blocking of directed broadcasts.  That red-herring slowed
them down, thinking it was externally generated.  No trace back of the
smurf triggers back to the zombie (having that MAC address would do wonders)
was caught since it quiet before they started sniffing the network.

	They broke it's back by blocking ICMP_ECHO and ICMP_ECHOREPLY
(fortunately, this attack was not using one of the UDP options).

	I have not heard if any of the TFN scanners have turned up
anything yet.  TFN2K zombies are a real bummer since it can lay dormant
on a network until triggered and they don't reply back to the master.
These guys are on a machine by machine, file by file snark hunt looking
for files that could be any one of a number of names on systems that are
all different anyways.  Don't you know they're having fun?

	The attackers are gaining in sophistication.  We are begining to
see effort at "covert channels" for communications.  The tricks of
communicating via ICMP_ECHOREPLY packets with commands in the payload
and sending blobs of forward only commands with no reverse channel
responses are just examples.  How long before they start sneaking past
those defenses by sending ICMP_UNREACHABLE/ICMP_FRAG_NEEDED with
a command data payload?

	They don't have to resort to spoofing to pull off a lot of this.
Long command time delays, covert channels, and unidirection command
direction can make it just as difficult to track down as spoofing.

	Diagnostics are fine.  Performance is fine.  Diagnostics and/or
performance at the expense of security and/or robustness is not.

> Vernon Schryver    [email-masked]

	Mike
-- 
 Michael H. Warfield    |  (770) 985-6132   |  [email-masked]
  (The Mad Wizard)      |  (770) 331-2437   |  http://www.wittsend.com/mhw/
  NIC whois:  MHW9      |  An optimist believes we live in the best of all
 PGP Key: 0xDF1DD471    |  possible worlds.  A pessimist is sure of it!

---

From: "Michael H. Warfield" <[email-masked]>
Date: Tue, 15 Feb 2000 13:59:58 -0500

Mike

---

From: "Charles E. Perkins" <[email-masked]>
Date: Tue, 15 Feb 2000 13:36:41 -0800

Hello Vernon,

> I really wish "we" actually knew how to filter.

But maybe filtering is putting the cart before the horse.

I compare the recent problems with phone harassment.  We can
install all sorts of doodads, but if we can identify the source
of the harassment we can bring charges against them.

From that analogy, I claim that the appropriate action is to
develop tracing systems that will help to identify a wrongdoer.
Here is a possible design.
- Create a router feature, able to be remotely activated, to keep
  track of "suspicious" packets for a few seconds up to maybe a
  couple dozen seconds.  Suspicious packets might be SYNs, or
  even packets with "incorrect" source addresses.
- If a violation is noticed, determine the interface on which a
  bad packet was received, and send a request to that neighbor
  along with appropriate credentials that can be checked.
- If, on the other hand, a neighbor sends a request for tracing
  a problematic packet, check the credentials that accompany the
  request and look at the stored data for that packet.  If no
  stored data exists, send back the bad news, and possibly enable
  the pattern-matchine function to capture similarly featured
  packets in case of future requests.
- If a neighbor's request refers to a stored packet, figure out
  what interface the stored packet arrived on.  If the packet
  came from another neighbor, forward the request.  Otherwise,
  look for the malicious source internal to the domain.

The basic idea then would be to trace back bad packets that
conform to some typically innocent, but occasionally troublesome,
profiles.  The profiles will become self-evident with experience,
and once people know they will be caught by this traceback
system they will think twice before spreading their crap around.

According to one knowledgeable source, this strategy is undesirable
because it would cause routers to maintain more state.  But, I claim
that we have to maintain state to do any tracing, and I also claim
that we need to enable tracing in order to detect and identify
wrongdoers.  Since the features should be activated by request
from trusted neighbors, in the usual case there should not be
any significant performance degradation.  Clearly, such requests
have to be checked for authenticity.

So the costs boil down to more memory, more software, some
pattern-matching hardware, and maintaining security relationships
with your routing partners.

I think it's a very worthwhile tradeoff.  Much better than mostly
ineffective measures that have big negative effects and small
positive effects, characteristics of ingress filtering as I
understand it.

Plus I don't like the attitude of blaming the victim without
giving the victim any tools with which to find the perpetrators.

Regards,
Charlie P.

---

From: "Charles E. Perkins" <[email-masked]>
Date: Tue, 15 Feb 2000 13:36:41 -0800

Hello Vernon,

Regards,
Charlie P.

---

From: John Hawkinson <[email-masked]>
Date: Tue, 15 Feb 2000 17:58:02 -0500

> But maybe filtering is putting the cart before the horse.

I believe in general we have been having a discussion about
operational practices, not really about theoretical ideas
of what we might be able to do twelve to eighteen months
out.

But discussing those theoretical things is good too, but I think
it's important to make clear what the scope of any
particular message in this thread here is, otherwise
confusion runs rampant.

Incidently, while we're here, you might look at Stefan Savage's paper
"http://www.cs.washington.edu/homes/savage/traceback.html", which kc
pointed NANOG at yesterday. I describe it as "having each router
probabilitically mark transit IP packets with a router-specific
locational referent in the ip_id field."

>   even packets with "incorrect" source addresses.

That is a lot of suspicious packets. Is a packet to a random UDP port
"suspicious"? If not, then this scheme is mostly useless against our
most recent series of attacks. If it is suspicious, then the attackers
will switch to TCP packets -- are those suspicious too?  Are all
packets suspicious?

>   along with appropriate credentials that can be checked.

This generates more traffic in the network for each of these; when the
problem is traffic flooding, as it has been most recently, this may be
unwise.

What would a credential check involve? Where would these
credentials be issued from and how-validated?

>   packets in case of future requests.

This suggests some sort of AI, or at least relatively clever
heuristics.  My brain says "research problem--IRTF--bye!" Perhaps
that's unfair?

>   look for the malicious source internal to the domain.

Where look for is "send a message to an operator"? How does this work
across multiple routing domains (e.g. provider boundaries)?

> because it would cause routers to maintain more state.

It appears to imply state on the order of 1 unit for each packet the
router transits. That's an immense amount of state.

> But, I claim that we have to maintain state to do any tracing, and I
> also claim that we need to enable tracing in order to detect and
> identify wrongdoers.

Yes, tracing requires state. How to distribute or limit that state is
the essessence of most of the discussion of tracing schemes.  Marking
packets puts the state inside them. Only collecting state on some
kinds of flows or in response to some trigger also limits the state.

> with your routing partners.

Sure.

I think that doing anything other than what already has been done that
might loosely fit your description is likely to be terribly
unpractical and feels like a research problem.

> positive effects, characteristics of ingress filtering as I

Ingress filtering is something we can do today, and has a very small
negative effect -- it limits the ability of some people who use the
network in non-traditional ways (half-tunneled setups, etc.), and
those people can simply request a broader ingress filter; I haven't
seen an example of how that fails, except when the people doing
the ingress filtering refuse, and that's not a problem with ingress
filtering, but instead a problem of politics.

--jhawk

---

From: John Hawkinson <[email-masked]>
Date: Tue, 15 Feb 2000 17:58:02 -0500

Sure.

--jhawk