Heap Protection Mechanism

ayazero

FROM：http://www.securityfocus.com/columnists/359

Security-related innovation in Unix
Jason Miller, 2005-09-28

Recently, a good friend of mine forwarded me an article from kerneltrap.org, which talked about a new heap implementation that's being introduced into an upcoming release of the OpenBSD operating system. This article was of specific interest to me, as I have been experimenting with the creation of a more secure heap implementation myself.



“ The more hurdles that one has to jump through for good security, the less likely people will go through the trouble. OpenBSD allows even the most inexperienced users to take advantage of these technologies without any effort. ”

I was a little shocked to find out that this concept that I'd been hacking around with in my spare time was actually being developed for inclusion in one of the three main BSDs. After all, the userland heap is a very significant part of an operating system, and it's not something that gets swapped out from a major operating system distribution at a whim. That being said, OpenBSD has historically been pretty bold about making significant changes to their system for the cause of security, so if anyone would be doing something like this, it would be them.

According to the article, this new heap implementation looks to be a candidate for release in OpenBSD 3.8, which will be their next official release. This is great news, and following along with other OpenBSD advances, promises to end up in widespread use due to the fact that (most of) it will be enabled by default.

More about heaps

For those of you who don't have a programming background, let me attempt a brief and non-technical explanation of what a heap is. Basically, it's just a management facility for most of the memory that applications use. When an application (such as Mozilla, your command shell, word processor, and so on) needs some memory, it will typically take this memory from the heap. The heap is responsible for handling these requests, which involve handing out chunks of memory when requested, and freeing the chunks when the application is finished with them.

For a more verbose (and perhaps less butchered from a technical perspective) description of a heap, check out this page on the wikipedia.

A problem can arise when programs don't make correct use of the memory that they receive from the heap - if they write before or past the chunks that they're given, they can corrupt information that the heap implementation uses to manage the chunks, and this can result in code execution exploits by fooling the heap into doing things by carefully crafting this management information. Secure heaps attempt to prevent this from happening, and catch this misuse of the heap before it turns into something more nasty.

As far as the current heap landscape goes, if you're running Linux, you're probably running "ptmalloc2," which is based off of "dlmalloc." On the other hand, if you're running one of the BSDs, you're probably running "phkmalloc," named after it's author, Poul-Henning Kamp.

In practice, phkmalloc has been a little bit more difficult to exploit, though I don't think that this was so much an intentional feature as much as a side effect of the algorithm. It will suffice to say that the two algorithms are distinct and different.

Although phkmalloc has remained relatively unchanged over the years, the glibc implementation (ptmalloc2) has undergone several recent changes (a friend of mine confirms that version 2.3.4 contains numerous security enhancements), increasing the difficulty of exploiting heap corruption vulnerabilities. The glibc heap now contains safe unlinking, and a bunch of other features that raise the bar of exploitation significantly, however, these changes are still relatively minor - and are quite different from the fundamental changes that OpenBSD is making.

Fundamentally, they're both trying to solve the same problems, and both of them are quite effective - I certainly don't want to downplay the effects of the recent changes to ptmalloc2. Perhaps the only fundamental difference between the two, from the perspective of an end-user, is that the OpenBSD implementation will do a better job of preventing heap exploitation through heap data corruption (as opposed to heap management data corruption - sorry if the difference between the two isn't obvious), and the OpenBSD implementation will complain loudly (crash) almost every time there is a problem.

Although having applications crash immediately might not sound like a good thing, it is certainly for the better good. It will help OpenBSD users to find bugs in software more easily, which will result in better applications for everyone. Also, diagnosing the cause of the crash will also be much easier; as some of you may already know, heap-related bugs can be extremely difficult to track down.

What's so great about the new OpenBSD heap?

Although I haven't looked at the code for OpenBSD's new heap implementation yet, the e-mail message from Theo de Raadt included in the kerneltrap.org article includes enough information to understand how it works. If you're interested in the dirty details, be sure to check it out. Continuing along with what I've just mentioned, I'd like to elaborate on a few things that I really like about the new OpenBSD heap.

1. It's going to be included and (mostly) enabled by default in OpenBSD.

Technologies like this are fantastic, but they really start to lose their value when only a small subset of people use them. OpenBSD has done a stellar job of putting innovative security features like this in their operating system, and enabling them by default. The more hurdles that one has to jump through for good security, the less likely people will go through the trouble. OpenBSD allows even the most inexperienced users to take advantage of these technologies without any effort.

2. It's going to help find and pinpoint heap-related bugs.

I still haven't come to a personal conclusion about the number of vulnerabilities in software being finite, but I think we can all agree that when we eliminate a vulnerability in an application, we're making the application stronger. With this new heap in wide-scale use, we're going to start finding a lot of bugs in various applications, and those bugs will start to be fixed. For those of you that will be using this new heap, please file bug reports for any applications that crash, as this will help make software better for everyone.

3. The guard pages will help to hinder many "heap-data" exploit vectors.

As I mentioned earlier, misuse of the heap can result in an attacker playing tricks with the heap algorithms, causing the offending program to give up control of itself to the attacker. Even when using a heap designed to protect against this type of trickery, it may still be possible for an attacker to overwrite other application-specific data on the heap, which can also result in some leverage for the attacker. Without getting into too many technical details, the new OpenBSD heap's guard pages will help protect this type of attack in many (but not all) cases.

Innovate, and propagate

Personally, I'm a huge fan of technologies like this. We already have some similar projects that, although not specifically related to the heap, aim to hinder exploitation at the machine code level. If you use Linux, and haven't heard of PaX, then get out from under your rock and start using it.

Now, from a technical standpoint, PaX is very different from a secure heap implementation, but they do share the end goal of pro-actively defending against the exploitation of software vulnerabilities. The only problem with PaX is that not enough people use it. The more popular Linux distributions need to include technologies like this in the default installation. Innovation is useless without exposure - you can build some very powerful protection technologies, but if nobody uses them, they have very little value.

As far as PaX-like functionality for the BSD operating systems, OpenBSD's has W^X, which is very similar to PaX in many regards, and is included and enabled by default. Even NetBSD has support for non-executable pages to a varied level of granularity on many of its supported platforms.

Ultimately, we need more pro-active technologies like these, and more importantly, better integration of these innovations to the masses. Security shouldn't be anything but a top priority.

Summary

Pro-active solutions to security issues are fantastic. The security industry needs to take a more pro-active approach to securing their systems, because the reactionary techniques that have plagued our history have done nothing but fuel the never ending arms race. It's time for us to stop playing catch-up, and start gaining some ground. And if we have to sacrifice a little usability on the way there, then so be it.


Jason Miller manages the Focus IDS area for SecurityFocus and is a threat analyst for Symantec Corporation.