OpenBSD's network stack

rainren

SecurityFocus interviews three OpenBSD developers about their network stack protection against DoS ICMP attacks, a short comparison with Linux' stack, and some thoughts on OpenBGPD.

On November 1st, OpenBSD 3.8 will be released. This is the first release that includes protections against some old and simple DoS attacks via ICMP. How do these protections work?

Fernando Gont: First of all, I'd like to make a comment on that of "old attacks." The attacks are old, but there have never been proposals on how to deal with them. So we are talking about old attacks which are still current. If you look at NISCC's or CERT/CC's vulnerability report, you'll see that virtually every vendor/system is affected by either all or some of these ICMP attacks. It's 2005, and most systems are still vulnerable to ICMP-based attacks. Scary, isn't it?

There are basically three different attacks that can be performed against TCP by means of ICMP: blind connection-reset attacks, blind throughput-reduction attacks, and blind-performance degrading attacks. There are general counter-measures that you can implement for ICMP-based attacks, and attack-specific ones. In OpenBSD, we implement both.

The general counter-measures are based on performing checks on the received ICMP messages. Basically, we check that the TCP sequence number contained in the ICMP payload corresponds to data already sent but not yet acknowledged. The rationale is obvious: if the TCP sequence number contained in the ICMP payload corresponds to data already sent but already acknowledged, then the error message must have been forged, caused by an old TCP segment, or corrupted, and thus should not be honored. If the TCP sequence number contained in the ICMP payload corresponds to data not yet sent, then the error message must have been forged, or corrupted, and thus should not be honored, either.

This is a general validation check for ICMP messages. However, it doesn't eliminate the vulnerabilities: it just requires more work (or luck) on the side of the attacker. Therefore, in OpenBSD we implement, in addition to this general validation check, attack-specific counter-measures that completely eliminate the vulnerabilities.

The blind throughput-reduction attack is performed by means of ICMP Source Quench messages. These messages were originally introduced for flow control and congestion control in IP networks. An attacker can use ICMP Source Quench messages to fool the attacked host into thinking the network is congested, and as a result, the attacked system will reduce the rate at which it is sending information. However, if you look at it carefully, TCP implements its own flow-control mechanism, and thus does not rely on ICMP Source Quench messages for performing flow-control. Also, ICMP Source Quench messages have been considered for a long time to be ineffective and unfair for controlling congestion. Thus, the counter-measure for the blind throughput-reduction attack is very simple: ignore ICMP Source Quench messages meant for TCP connections.

The blind performance-degrading attack is an attack against the Path-MTU Discovery mechanism implemented by TCP. Basically, an attacker will send a "fragmentation needed and DF bit set" ICMP error message that advertises a small Next-Hop MTU to the victim host, to fool it into thinking it is sending packets that are too large to be forwarded without fragmentation. As a result, the attacked system will reduce the size of the packets it sends, accordingly.

The counter-measure we implement for this attack works as follows. First, we keep track of the largest packet size that has so far been sent for this connection. If the Next-Hop MTU claimed by the ICMP error message is larger than that size, then we simply ignore the error message. Second, we keep track of the largest packet size that has so far been acknowledged for this connection (say, "maxsizeacked"). This allows us to divide Path-MTU Discovery into two phases: Initial Path-MTU Discovery, and Path-MTU Update. This two-phase separation allows us to quickly discover the Path-MTU for a fresh connection (and thus not affect interactive applications), while still being resistant to the discussed attack.

Whenever we receive an ICMP "fragmentation needed and DF bit set" (and provided it has passed all the general validation checks), we compare the advertised Next-Hop MTU with "maxsizeacked". If it's larger than "maxsizeacked", then it means we are in the Initial Path-MTU phase (that is, we are trying to find out the Path-MTU of this connection for the first time), and thus honor the ICMP message immediately. If the advertised Next-Hop MTU is smaller than "maxsizeacked", then it means we are in the Path-MTU Update phase, trying to change the assumed Path-MTU for this connection. In this phase, we should be much more cautious when processing ICMP messages. The error messages could be legitimate (and sent because the packets that correspond to the connection are now being forwarded through a different Internet path), or they could be part of an attack. If the error messages were legitimate, then the corresponding data (those claimed by the TCP sequence number contained in the ICMP payload) should have been dropped by the Internet router that sent the ICMP error message. Therefore, we wait for an RTO (TCP's retransmission timeout), and see if the corresponding data gets acknowledged. If the corresponding data times out, then it means the error message must be legitimate, and thus we honor it, updating the Path-MTU for the connection accordingly. If while we are waiting for a RTO those data get acknowledged, then it means our data are still getting to the remote system, and thus the ICMP error message must have been forged. Therefore, we simply drop the ICMP error message we had received.

The implication of this counter-measure is that in order to perform the blind performance-degrading attack, the attacker should be a "man in the middle," and should be not only lucky enough to hit the TCP window, but should also be able to selectively drop the packets that correspond to the attacked connection. This is so that either the data segments don't get to the remote endpoint, or the TCP acknowledgements sent by the remote end-point don't get [to] the attacked system. If an attacker were able to do this, he would have already DoS'ed the connection, and thus wouldn't have the need to perform the attack.

As for the blind connection-reset attack, BSD-derived implementations are not vulnerable to it. BSD-derived systems never abort established connections in response to ICMP messages. This has been the traditional BSD behavior for quite a long time.

Is there already any other OS that includes them?

Fernando Gont: OpenBSD has been the first operating system to implement a complete set of counter-measures for ICMP-based attacks. Following OpenBSD, NetBSD fortunately ported OpenBSD's counter-measures to their system.

Other systems have followed us, implementing only some of the OpenBSD counter-measures. Unfortunately, it seems they have failed to understand the importance of the counter-measure for the blind performance-degrading (PMTUD) attack. Some vendors/projects simply seem to think that the TCP sequence number check is enough to protect a system from this attack. Others simply wanted to see a working (and tested) implementation, and were not willing to take the lead. At the c2k5 Hackathon we implemented the counter-measure for the PMTUD attack, and tested it extensively. So there I think are no more excuses to vendors: they can follow us, or continue ignoring the problem.

What about Linux?

Fernando Gont: In the same way as BSD-derived systems, they were already treating the so-called ICMP "hard errors" as "soft errors," so they were not vulnerable to the ICMP-based blind connection-reset attack.

Linux had also been implementing the basic TCP sequence number check for several years.

When I published my IETF internet-draft "ICMP attacks against TCP," they removed support for ICMP Source Quench messages, as recommended in my draft. They were very responsive on this particular fix.

However, they have not yet implemented the full counter-measure for the PMTUD attack. They basically said they first wanted the counter-measure to be tested. So maybe that now that OpenBSD ships with this counter-measure, and that NetBSD has followed us, they will finally implement it.

The counter-measure for the PMTUD attack is particularly important. First, because those ICMP messages used for PMTUD are probably the only ones you cannot filter. Second, because even if you protect your TCP connections by means of the TCP MD5 option, or by means of IPSec, you still need the Path-MTU Discovery mechanism. And, at that point, PMTUD becomes "the weakest link in the chain".

There are scenarios in which IPSec-secured connections could get frozen by means of the PMTUD attack. If you count the number of bytes required for headers (IP+IPSec+TCP), along with the number of bytes required for IP and TCP options, and realize that the minimum IPv4 MTU is 68 (i.e., a "fragmentation needed and DF bit set" ICMP message can report a Next-Hop MTU as small as 68 bytes), you come to the conclusion that the attacked connection could become frozen, or the TCP/IP stack may end up behaving in some unexpected manner.

I must acknowledge that Alan Cox and David S. Miller read the draft, and took the time to provide feedback and contribute to make my internet-draft a better one. This is something I appreciate. Most other vendors/projects didn't care to provide feedback, or anything.

What other protections are included only with OpenBSD's network stack?

Ryan McBride: Firstly, OpenBSD is aggressive about the use of network randomness. Wherever possible, we use pseudo-random data in variable fields in packets. Counters, timestamps, and session identifiers are all good candidates for such pseudo-random data.

Our network stack randomizes the following values: TCP ISN, TCP Timestamp, Ephemeral Source Port, IPID and the IPv6 Fragment ID. In addition, Randomness is used in userland network code: BIND, ICMP/ICMP6, RPC and the time protocols all use pseudo-random values.

These randomizations make blind insertion attacks much more difficult, and have saved us from serious vulnerabilities which were unknown when we implemented the randomness. An example of such a vulnerability is the TCP sequence number predicion problems disclosed last year by Paul Watson.

Secondly, we make conservative decisions about implementing risky protocol features, or ignore RFCs when they're wrong. A comprehensive list is way too long to include here, but examples include refusing to allow IPv4 mapped address in OpenBSD's IPv6 code, ignoring hard ICMP errors that don't make sense, or [having] our portmapper binding separately to localhost to prevent certain operations from being done remotely.

Finally, including Availability as an aspect of Security, OpenBSD has been making many advances in the area of network redundancy. CARP, interface trunking, and STP allow failover and redundancy at layers 2 and 3, and many daemons are being added or modified to deal with these network stack features: pfsync, sasyncd, ifstated, bgpd, etc.

Keep in mind that much of the above is not new - we've been doing many of these things for nearly 10 years. And while these mechanisms have to be designed very carefully to avoid breaking protocols, it is not impossible. This is not rocket science, it's just good software engineering, But adoption of these measures by other OS projects is sadly lacking. The OpenBSD difference is that we make a concerted effort to implement and enable these features by default.

How does OpenBSD network stack compare with Linux?

Ryan McBride: The "Linux stack" is a concept that is very hard to pin down because there are so many versions, distributions, 3rd party patches and modules, etc. People might tell you that Linux has the capability to do X, Y, or Z that OpenBSD enables by default, but they don't tell you that you have to dig around for the patches, enable the right compile flags, load the right modules and sacrifice a goat on the full moon. And even then it's incomplete and buggy.

Because of this, I think OpenBSD's main strength is the following: these security features are easy to use. (Which is somewhat ironic as OpenBSD has a mistaken reputation for not being user friendly.) Our approach is simple: Proactively implement security features. Enable these features by default. Minimize the number of "buttons" - compile-time or run-time options. And document rigorously.

It seems that OpenBSD focus on proactive security and provides good results even against new attacks. I remember when Paul Watson published his paper "Slipping in the Window: TCP Reset Attacks" around May, 2004, and OpenBSD was the only OS not vulnerable by default. However I'm wondering if sometimes it's also the result of undisclosed information from researchers...

The TCP window attack was particularly effective against long TCP sessions, so the biggest target was BGP, and for some magical reasons in that period you released the first version of OpenBGPD, a BSD-licensed implementation of BGP, so that people could use it to replace vulnerable systems to handle routing information.

I'm wondering how you chose to start working on a bizarre thing such as a BGP implementation in the end of 2003? Did you get any preview of Watson's paper?

Henning Brauer: No. I started to work on bgpd because we were using zebra here, for years. It worked... to some extent. I did patch here and there and fixed some bugs in the code and such, but it was very obvious that the design was (and is) wrong, and that this beast is unfixable. So I had this idea of writing a BGP daemon that doesn't suck for some time. Two years at least.

When I was in Calgary in fall 2003 Theo got me drunk and made me tell him about it, and then he kept kicking my lazy butt to take a stab at it... yeah, and after I've been back I started writing it. Nick reminded me that I vaguely predicted an attack like the RST one against BGP sessions years ago after Paul Watson released his paper. I did not know about Paul's work when I started writing bgpd. I did get informed that there was something upcoming sometime later, I didn't know exactly what though.

The countermeasures in OpenBSD's network stack, requiring RSTs to be right on the edge of the window and using random ephemeral ports, were about 7 years old when Paul released his paper, so they were certainly not in reaction to his work :)

What is OpenBGPD's current status?

Henning Brauer: Quite some work was done between 3.7 and 3.8 in OpenBGPD. Countless little bugs were fixed, I don't think any of them was really critical. One I have to highlight since it was so hard to fix: when you did a "bgpctl reload", causing the config file to be reexamined, and the configuration had errors, we had some memleaks. Nothing dramatic, and rather small leaks, and you're supposed to write correct configs anyway :), but nonetheless a bug - or rather, a number of small bugs. We got that fixed. I did spend some time in the interface validation code, that is used in the nexthop validation - there were some (again, non-fatal) bugs, and in the end I changed quite some things there. I also totally changed the way we allocate listening sockets, fixing a small race condition. Claudio did "network connected", which tells bgpd to redistribute all directly connected networks. Same thing for "network static", redistributing static routes. And you can add networks with attributes like communities and metrics from the command line using bgpctl network add now. There is more of this kind of changes - interested parties should take a peek at my presentation from the DE-CIX 3rd technical meeting.

One thing that is finally completely implemented is the route label stuff. This really rocks: we allow up to 32 bytes to be attached to a route. It is stored together with the route in the kernel routing table. Well, not really, they are stored separately and the routes just refer to them via an ID mechanism to preserve memory, but that is a completely hidden implementation detail. We do it the same way in bgpd though, again to preserve some memory. It turned out we can generalize the ID allocator a bit and do the same for the pftable stuff, so we save another 14 bytes per AS-Path. But back to route labels. bgpd now finally can set route labels via its filter language. You could, for example, store the route's source AS in the label. Or the peer you learned it from. Or something completely different. Now, it gets really powerful because PF can access these labels as well, and take filter decisions based on them. Or, my favorite, QoS queue assignments. Or AS-dependent max-src-conn-rate and the like, to fight DDoS. Everything that pf can do can now be done based on information from BGP as well - that is very very very powerful.

The biggest news is probably IPv6 support. Yes, it is there now. Yes, it hurts to implement it. There's not much to say about it, since it just works. We are still adding v6 support to various bgpctl commands, but the basics all work already.

Aside from the technical advancements there is interest at a number of European exchange points to use OpenBSD and bgpd as route server - presentations and lobbying and stuff take some amount of work too. Fortunately Reyk Floeter, and maxim of the CCC crowd, do a lot of the lobby work.

Which OS can OpenBGPD be run on?

Henning Brauer: Please go whine at Cisco that theirs doesn't run on OpenBSD. I keep repeating it - BGP is not a userland only thing. We did change things in kernel land, and we will continue to do so, to deliver a BGP router (or BGP route server, etc etc) solution. Seeing that as a userland only task is way too shortsighted. It is the combination of kernel and bgpd.

That said, it does work on the other BSDs, with some limitations since they lack the kernel parts. Somebody even got it to run on Linux, but without the ability to change the kernel routing table.

剑心通明

正等着3.8哪，呵呵