» »

Brad Spengler (PaX Team/grsecurity) interview

  • ::

Slo-Tech: Introduce yourself to our readers (job, education, interests, etc) and please explain if your real surname is Spender or Spengler :-) Also, was Brad ever member of any Black hat group?

Brad Spengler: Brad Spengler (not Brad Spender), though the similarity in the names isn't a coincidence, I came up with the alias as a variant on my last name. I graduated from Bucknell University with a BS in Computer Science & Engineering (sortof a mix between computer science and EE/hardware type things) with a (free) minor in applied mathematics. I found the math/physics/economics/philosophy classes to be most interesting. I had already been studying the Linux kernel and writing code for it 4 years prior to taking an Operating Systems class, for instance, so the CS material wasn't of much interest. I've never been in any blackhat group. Since high school I've always been more interested in creating things.

Fun story that I like to repeat: my junior year I took a digital design class. The final project was to design some human-interactive physical board that interacted with a Xilinx FPGA. Some of the other projects were things like tic-tac-toe, etc. I wanted a bit of a challenge (and k-maps to doodle in my CS classes) so I designed a "crypto" processor. I took the reference source for DES, reworked it some, determined what instructions I would need, created an assembly language, an assembler, a simulator, converted the crypto into my assembly language, assembled it, stored the machine code in two ROMs. It was a 12bit RISC processor IIRC, had registers and I think 256 bytes of memory. Designed the ALU, optimized all logic with k-maps, etc. The final printed out schematic for all the components was ~100 pages. The physical board had 8 hex displays that you could modify with buttons and switches. A second button would encrypt the input and display the result. Unfortunately, when it came time to upload the schematic onto the Xilinx board I found that it was too big for the boards that we had, and there was no way I could make it fit. For the project, 40% of the grade was a successful demonstration, so I got an instant D. Should have stuck with tic-tac-toe ;)

As far as interests go, I enjoy reading philosophy or fiction that's heavy on philosophy and light on fiction. Critical thinking is one of the most important skills to have (and the world would be better off if more people had it). I play the bass (poorly) and recently have gotten into electronic music. I have all the stuff here to create it (and add live bass tracks), but it would be nice to find someone else who's been doing it for a while who could teach me some things.

Slo-Tech: Could you please describe in few words what is PaX and GRSecurity and how many people are involved in those projects.

Brad Spengler: PaX is a beast that has changed the shape of security drastically already and has even more tricks up its sleeve to change it even further sometime in the near future. It focuses itself with the generic eradication of exploitation against a number of bug classes. Some of that work is still incomplete, but that will (hopefully) change and then another 8 years from now everyone else will catch on. PaX focuses entirely on the exploitation of memory, whereas grsecurity adds in other host-based defenses and adds extra support to the features of PaX (bruteforce deterrence, anti-infoleaking of ASLR, not allowing arbitrary code execution at the filesystem level) that are needed to reap extra benefits from PaX. Grsecurity includes a lot of "set it and forget it"-type automatic features (the wiki has a listing) as well as an easy to use RBAC system that tries to make it easy to generate good policies via learning (per process, per user, or per system -- your choice) while at the same time trying to prevent the admin from shooting themselves in the foot (a large number of security checks are done against the policy at load time to prevent attack vectors that would make the entire point of using RBAC useless). The policies are human readable, and the error messages should be useful in describing attacks that are the reason for configuring the policy a particular way.

PaX has an unknown number of people involved! Hence PaX Team -- it's definitely at least one person :) For the grsecurity side it's just me. Occasionally I do get patch submissions (like from Zbyniu Krzystolik) or sponsors/friends will tell me something they want added to it, but other than that all the coding/new features etc are done by me.

Slo-Tech: Few people really know that PaX Team developed some security mechanisms (like ASLR) that were later implemented in Linux kernel(s) - yet still it looks like the major LK developers/players are trying to discredit PaX team. Which is the mechanism that, in your opinion, contributes most to Linux security in general.

Brad Spengler: Well, it's funny to me that the wise Ingo Molnar years ago was upset about the paxtest benchmark (which checked the ability to execute arbitrary code in different areas of the address space) because it included a second set of tests where mprotect was used to mark some memory as writable and executable. Ingo called the test "sabotaged" -- that somehow the mprotect checks were unfair (because exec-shield failed them all). Fast forward a couple years and you'll see that SELinux basically added the MPROTECT feature of PaX into its policy language (without which SELinux would really be even more silly and out of touch with reality). In the Windows world what you see is everyone basically targeting the code that does the equivalent of an mprotect on Linux. glibc has/had a make_stack_executable() function that was the ret2libc target of choice, and Windows has something similar. The PaX team knew years ago that attackers would be targeting the weakest link in the system, and that's exactly what is happening now.

I can't pick one feature in particular that contributes the most to Linux security, as the combination of them together provides additive effects. Also, I think the stuff to come next from the PaX team will be the most revolutionary.

Slo-Tech: It looks like there is a "verbal war" between Brad & Linux Kernel Team going on. What's the main reason for this unpleasant situation? Don't you think that in the end it's the end-users that are losing most?

Brad Spengler: I've never sent an email to LKML, so I'm not engaging in any "verbal war" there. The PaX team has emailed on there before (like for instance when getting the developers to admit to their official policy of non-disclosure and obfuscation of security vulnerabilities and the ridiculous admission of "a bug is a bug" for which Linus won a pwnie award). The only comments I make are on LWN (like when Ingo was spreading lies about the perf_counter vulnerability) or in exploit headers, because they seem to respond to that (or rather, the commercial influences care about the media publicity generated by such things, which causes them to improve security). It took 8(!) public exploits for null pointer dereference bugs to be fixed as a privilege escalation vector. Granted, after the first public exploit, mmap_min_addr was introduced some months later, but the Linux developers who aren't really security people aren't very good at introducing security features. mmap_min_addr has had at least half a dozen different bypasses since its inception for such a small, simple feature. The vanilla ASLR had numerous weaknesses. SSP (Stack-Smashing Protection) was broken most of the time that it was in the kernel, and now they've even forced a limit of its scope (I'm thinking in reaction to my finding that it's unpredictable what it'll actually protect -- on Ingo's system SSP prevented exploitation of the perf_counter bug, but on my system it did not. It didn't instrument the system call at all.) All these things have come from the same commercial source though (Red Hat) -- and what's important for a company that makes money is to say you have X feature (like AV companies: "we'll protect you from viruses!"); it doesn't matter if the feature actually does what it says or not (and they don't seem to bother checking -- it's security people, or blackhats, that do).

Slo-Tech: Who is Linux Kernel developer you really respect? Also, don't you think that Linux needs some serious reogranisation in handling security incidents and security development in general (see many regressions in the code). Something like Microsoft's SDL maybe... What do you think about SDL anyway.

Brad Spengler: I have a lot of respect for Microsoft's attention to security, given how large of a company they are and how many people their software affects. It's a whole different security ball-game when you have the huge binary software eco-system of Windows, implementing security features while maintaining application compatibility. It's easy to gloss over how difficult of a job that is.

Linux definitely needs some central leadership/organization for security. The developers are fixing issues they know are security relevant, but they don't notify anyone (including vendors). Select vendors get notification of certain security issues, leaving smaller vendors/distros out. Upstream basically does their best to make the job of vendors difficult, and each vendor has to duplicate effort because nothing is organized. The official policy of "a bug is just a bug" hurts all Linux users. That said, some notable improvements were the addition of sparse, various hardening to functions that were constant attack vectors (getsockopt, write(), copy_*_user). They're often overly concerned about performance at the expense of security; but the couple cycles they lost on checking for negative lengths in write() for instance has saved them countless times.

Slo-Tech: Do you think that Linus Torvalds is competent about security? I mean, he's a really good developer but *not* security researcher. What about others well know LKT developers? Do they have the right skills?

Brad Spengler: I don't think he is; he doesn't have the mindset for it and he doesn't understand it. I'm not sure he's in touch with the reasons why everyone else feels that it's important. Some of the developers 'get' security, but generally they're involved in fixing bugs. Upstream in general doesn't care about hardening the system against attack, or making it more resilient in the face of flaws.

Slo-Tech: Dave Aitel wrote on his mailing list that Linux community "needs a security center where multiple vendors can contribute to it, and work together to help with Linux's (kernel) security problems". Do you agree? Is this the right step to take and would you work in such a center?

Brad Spengler: Dave was right-on about that, but I don't see it happening. The most important thing that has to change is Linus' attitude regarding security. Since he's the gate-keeper of the upstream kernel, his opinion, right or wrong, is what shapes the kind of security that will exist for Linux. But that too is wishful thinking :) So in the meantime, the PaX team and myself just keep doing what we've been doing, improving the security of Linux in what's really the only way possible: out-of-tree.

Slo-Tech: Is it true that some kernel bugs are silently fixed by kernel mantainers? Does "silently fixed" mean incorect bug classification and downgraded security impact or maybe something else?

Brad Spengler: Yes, they've admitted that to be the case. If you go back and read my LWN posts around that time I have tons of examples. The vendors were horrible too about correct classifications -- everything was a DoS. I think that's improved slightly since they've been criticized publicly by numerous people for it. Part of the problem again is the lack of central organization and coming to a singular statement about a particular bug. Often different vendors will issue different analyses of a bug. Also now some of the vendors are issuing advisories for things that aren't security bugs (for instance anything on NOMMU systems -- these are embedded systems where there is no security: userland can modify kernel memory at will, unrestrained -- but yet there's been at least 2 CVEs issued for bugs that affect only these systems). But this probably just boils down to the companies involved hiring software people instead of security people to do the job of a security person. That and the sheer number of vulnerabilities they have to deal with and being ill-equipped to properly investigate them all (again with some central organization this problem would be alleviated as well).

Slo-Tech: This is nasty one, so you don't have to answer if you don't want... Would you work for Microsoft if they hire you (like they did with Crispin Cowan)? And for Red Hat?

Brad Spengler: I was invited to be interviewed at Microsoft my senior year of college I think, but I turned it down because I was asked about a year after HR lost my information (and I lost my interest in working there). Working at Red Hat -- well I'm sure you can figure that one out ;)

Slo-Tech: It's well known that you use Windows Vista as your primary OS. That's pretty strange for a top Linux security researcher. Why this decision and did you find any bugs in Windows yet :)

Brad Spengler: I use Windows 7 actually -- the Cheddar Bay video was on the RC I think. I use it because the only fiddling around I want to do with Linux is testing and improving grsecurity. I used to use Linux as my primary desktop back in high school and in college, but nowadays I just want to get stuff done -- so Linux stays in VMs. Linux was much simpler back then too, in both the kernel and userland. You had known config files, if you edited them some 'intelligent' process wouldn't go and change it behind your back, you wouldn't have to modify SELinux policy because you wanted to host something from a location different from the system default. There was more of a feeling of freedom on it then -- something I think it's losing the more it becomes commercialized. Also: games; I play them. New ones. And if Pidgin notifies me that a new release is available, I just download a Pidgin executable; I'm not forced to update 100+ packages at the same time, always risking that something will go wrong.

Slo-Tech: Why do you publish your exploits and not rather sell them to companies like iDefense, ZDI, Immunity... What is the approximate market value for local root exploits like cheddarbay, ingom0wnar, wunderbar...

Brad Spengler: I'm not sure they care about local exploits for Linux -- first thing is that the exploits have all been for bugs that have already been fixed in some development tree. Second is the exploits themselves were very easy to write. So the exploits really were only useful to affect change (and of course, it worked; what discussion couldn't accomplish, 8 public exploits did.) The end result was it helped people understand their level of risk (I had several people contact me to tell me this was just the thing they needed to show their boss that they need to pay more attention to security) because there really hadn't been many public exploits for the Linux kernel for some time before then, not because the vulnerabilities didn't exist, but because blackhats were keeping the exploits private. The second result was that all major vendors now have the null ptr dereference attack vector closed by default, the SELinux vulnerability was closed, and vendors are paying more attention to when they really need to turn mmap_min_addr off. Users are more aware of the feature now too and the importance of making sure it's on. Lots of people also moved up from dreadfully old kernels as a result of the exploits. These are all good things. The only "innovation" I guess you could say would be in the enlightenment framework (which is the subject of the next question).

Slo-Tech: Recently you've released a framework for exploiting kernel bugs on Linux called Enlightenment. Do you plan to continue with improvements so one day it may become like Metasploit?

Brad Spengler: Yeah, I've been improving it because it gets me into the mindset of an attacker, which then helps me think of ways to defend against certain attacks (and I've recently implemented several things in grsecurity as a result of that). I also want to make it as complete as possible: it already supports x86 and x64, all 2.4 and 2.6 kernels, and disables all LSM modules. I recently added Xen support, so it'll do the appropriate hypercalls to allow modification of kernel code. I've considered doing namespace/container breaking as well, so that's on the TODO list. Enlightenment will strictly stay to local Linux kernel exploits though.

Slo-Tech: Most people are not taking local exploits as real security threat. I know that Sebastian Krahmer (SuSE/Novell) has writen some good points regarding this issue and I totally agree with him, but the truth is that right now there aren't any remote exploits in circulation (IIRC, Sgrakkyu was the last to publish one). Is it really impossible to break into vanilla Linux system remotely or this may change in the near future?

Brad Spengler: Remote kernel exploits are hard, especially when PaX is involved in the kernel, and when the target's kernel is custom-compiled. Just take a look at sgrakkyu/twiz's Phrack paper (Attacking the core: Kernel exploitation notes) to see the level of complexity involved. They're gluttons for challenging punishment; others will choose the path of least resistance which these days is bugs in web apps followed up by local kernel exploits. With the new stuff planned by PaX, exploitation is going to get harder, not easier. I still see the current vector working for the foreseeable future though, because distros are wasting their time with dropping privileges on software unlikely to have serious bugs, when there's a new local root in the kernel just about every week. And webapp developers (who write the 3rd party code present on every public server that serves a purpose, turning remote users into "untrusted local users") aren't getting any smarter.

Slo-Tech: Can Linux still be considered as secure operating system? Would you consider to government agencies and other critical infrastructure mantainers to use Linux?

Brad Spengler: I should hope there's some other OS being used for critical infrastructure. As for recommendations, a secure OS can be made very insecure through stupid mistakes, just as an insecure OS can be made secure by making it useless.

Slo-Tech: Do you have any private 0-day exploit on your hard drive right now :)

Brad Spengler: Yes, ext4_own.tgz, and it's less than 1000 bytes.

Thanks to Brad Spengler for the interview.

Intervju z Bradom Spenglerjem (PaX Team/grsecurity)

Intervju z Bradom Spenglerjem (PaX Team/grsecurity)

  • ::

English version Brad "Spender" je nedvomno vrhunski varnostni raziskovalec na področju Linux jedra. Nenazadnje stoji tudi za razvpitimi "NULL pointer dereference" ranljivostmi, ki so pretresale Linux skupnost. Kot temeljni člen PaX/grsecurity projekta je za varnost Linux sistema naredil več, kot bi ...

Preberi cel članek »

Interview with Dustin Kirkland, Ubuntu Core Developer about encryption in Ubuntu

Interview with Dustin Kirkland, Ubuntu Core Developer about encryption in Ubuntu

Dustin Kirkland is an Ubuntu Core Developer, working for Canonical on the Ubuntu Server. His current focus is developing the Ubuntu Enterprise Cloud for the Ubuntu 10.04 LTS release, but previously he had worked on a number of Ubuntu features and packages, including Ubuntu's Encrypted Home Directories. ...

Preberi cel članek »

Interview with HD Moore

Interview with HD Moore

  • ::

HD Moore is probably known to everyone who is following information security field. HD is author of many projects (www.digitaloffense.net) including Metasploit framework, that was recently acquired by Rapid7. He shared his experience with this transition process at this years Black Hat DC conference ...

Preberi cel članek »

Interview with Peter Van Eeckhoutte

Interview with Peter Van Eeckhoutte

  • ::

Introduction: We continue our series of interviews with a slightly »unusual« talk this time: Peter Van Eeckhoutte may be unknown to readers who don't follow the InfoSec scene on a daily basis. But he is well known to the international security community and his name is climbing fast on the ...

Preberi cel članek »

Interview: Seth Bindernagel

Interview: Seth Bindernagel

Slo-Tech: Can you introduce yourself? Seth: My name is Seth Bindernagel and I am the director of localization for Mozilla Firefox. Slo-Tech: Our community regularly follows nightly builds of Opera, Firefox, Chrome, etc. as it’s a very competitive landscape. How do you see it from the Firefox perspective? Seth: ...

Preberi cel članek »