We have secretary's day, presidents day, mothers day, fathers day, etc. But I never suspected there would be a 'software developers day' or other professional recognition day in our field. Twitter educated me otherwise, so I'm going with it. Our profession could certainly use a little recognition and attention. The auspicious occasion as this blog explains is the discovery of the first (literal) bug in a computer system on Sept 9, 1947.
Since then, we've had a meager few decades to hone our software development skills, and our bug-finding skills are not far behind. To that end, a short homage to the hard-working testers and QA professionals who often hear these responses from developers.
24. "It works fine on MY computer"
23. "Who did you login as ?"
22. "It's a feature"
21. "It's WAD (Working As Designed)"
20. "That's weird..."
19. "It's never done that before."
18. "It worked yesterday."
17. "How is that possible?"
16. "It must be a hardware problem."
15. "What did you type in wrong to get it to crash?"
14. "There is something funky in your data."
13. "I haven't touched that module in weeks!"
12. "You must have the wrong version."
11. "It's just some unlucky coincidence."
10. "I can't test everything!"
For the top 9, see this site for a familiar chuckle...
Sep 9, 2010
Aug 23, 2010
Client-side expoit demonstration videos
NSS has been testing exploits and exploit-detection/protection products for the last 10 years or so, dating back to the early days of IDS and IPS . It's arguably one of our specialties. And we have written before about the differences between malware, exploits and vulnerabilities. Yet, as these pernicious threats are elevated into mainstream consumer and enterprise awareness, we are seeing quite a bit of terminology confusion. So, for our latest Host Intrusion Prevention System (HIPS) test of enterprise anti-malware products, we created some demonstration videos of client-side exploits used in the test.
Product | Vulnerability | Video Demonstration |
Panda | CVE-2010-0249 | |
Panda | CVE-2010-0806 | |
F-Secure | CVE-2010-0249 | |
ESET | CVE-2006-0003 | |
Sophos | CVE-2006-4704 | |
Symantec | CVE-2010-0483 |
Aug 18, 2010
Client-side exploits
Client-side exploits are aggressive weapons used by cybercriminals that allow them to silently take control of computers that visit a web site. The infected widgets on nearly 5 million Network Solutions sites are a prime example (see Krebs' report on Armorize discovery). And NSS Labs' Q2 2010 test of 10 endpoint protection products for HIPS evaluates protection against client-side exploits.
In recent discussions with clients and journalists we've found the need to clarify some definitions of how these attacks work, and how they're different from typical socially-engineered malware campaigns. So, the following definitions and analogies are provided in an effort to provide clarification, as well as to bridge an ongoing communication gap between security vendors and their customers.
1. The attacker caused a specially-crafted stream of data and code to be delivered to a precise location. This exploited the victim’s computer, gaining the attacker the ability to perform arbitrary code execution.
2. Malicious code was silently executed on the victim’s computer.
If the attack can be thwarted in Stage One (successful exploit), then it cannot progress to Stage Two, where a malicious payload can be delivered. As long as the exploit is not defeated, then installing malware is just one of many possible actions the attacker can take. And the choice of malicious code is nearly infinite. Since there are far fewer exploits than malware, it is imperative that attacks be defeated in the earliest possible stage. In other words, it is advantageous for AV suites to detect the exploit vs. chasing malware samples.
For starters, if you're an organization with critical data behind any of these products, I suggest you buy and read the full report here. There has been a lot of press coverage of the report highlights, but not the details, generally reporting failure of AV suites in this area. While you may not be ripping out thousands of endpoint deployments, you may be asking your vendor some tough questions and setting expectations for improvement. Exploit detection & blocking is clearly an area that the security industry needs to focus on going forward. Got questions about other threat mitigation options, consult our other reports, or contact us.
In recent discussions with clients and journalists we've found the need to clarify some definitions of how these attacks work, and how they're different from typical socially-engineered malware campaigns. So, the following definitions and analogies are provided in an effort to provide clarification, as well as to bridge an ongoing communication gap between security vendors and their customers.
Vulnerability
Like a locked door that can be opened with the right key or combination, a vulnerability is a bug in software code that allows a product to be exploited. An example of a software vulnerability is improperly-defined memory usage within a function that enables content sent to a specific memory location to be run with privileged rights.
Exploit
An exploit is a specially crafted code sequence which can ”trigger” or ”unlock” a vulnerability within an application, such as a heap spray, buffer overflow attack, etc. An exploit can be hiding in an infected website (client-side attack) where it ambushes visiting computers or be launched from an another computer (remote attack).Payload
The payload is the content that gets delivered once the vulnerable application has been exploited. Payloads are the actions that are performed on the compromised target computer, such as command execution, writing a file to disk, returning a reverse shell, etc. This may be malware, but does not have to be.The Test
The test utilized 123 common and public vulnerabilities dating from 2006 to 2010. These vulnerabilities were exploited when a user visited an infected web page hosting the attack code. The attacks occurred in two stages:1. The attacker caused a specially-crafted stream of data and code to be delivered to a precise location. This exploited the victim’s computer, gaining the attacker the ability to perform arbitrary code execution.
2. Malicious code was silently executed on the victim’s computer.
If the attack can be thwarted in Stage One (successful exploit), then it cannot progress to Stage Two, where a malicious payload can be delivered. As long as the exploit is not defeated, then installing malware is just one of many possible actions the attacker can take. And the choice of malicious code is nearly infinite. Since there are far fewer exploits than malware, it is imperative that attacks be defeated in the earliest possible stage. In other words, it is advantageous for AV suites to detect the exploit vs. chasing malware samples.
Results and Next Steps
Unfortunately, 75% of corporate users are under-protected, based on vendor market share and their respective scores, which ranged dramatically from 29% to 100%; and even worse for variants. Depending which product you have, you may have significant cause for concern. So, what's next?For starters, if you're an organization with critical data behind any of these products, I suggest you buy and read the full report here. There has been a lot of press coverage of the report highlights, but not the details, generally reporting failure of AV suites in this area. While you may not be ripping out thousands of endpoint deployments, you may be asking your vendor some tough questions and setting expectations for improvement. Exploit detection & blocking is clearly an area that the security industry needs to focus on going forward. Got questions about other threat mitigation options, consult our other reports, or contact us.
Jun 18, 2010
Fragroute – Bug in 24-byte fragmentation
In our Q4 2009 IPS test, we tested Network Intrusion Prevention Systems on their ability to resist 60 different evasion techniques, IP Fragmentation (9), TCP Segmentation (11), RPC Fragmentation (16), URL Obfuscation (15), and FTP / Telnet evasions (9).
On May 19, 2010, TippingPoint brought to our attention a bug in fragroute, the de facto evasion tool for IP fragmentation. This flaw corrupts one of the evasion techniques – 24 byte fragments. Over the past few weeks, NSS Labs has retested all of the products from the Q4 2009 IPS Group Test with an alternate version of the tool that does not corrupt traffic when fragmenting into 24-byte segments.
We found that all of the devices that initially passed this test, continued to pass. In addition, we determined that TippingPoint does block the 24 byte fragmentation evasion with the non-corrupted attack. The findings for all other evasion tests remain unaltered.
The Q4 2009 IPS Group Test is being modified to reflect this update.
On May 19, 2010, TippingPoint brought to our attention a bug in fragroute, the de facto evasion tool for IP fragmentation. This flaw corrupts one of the evasion techniques – 24 byte fragments. Over the past few weeks, NSS Labs has retested all of the products from the Q4 2009 IPS Group Test with an alternate version of the tool that does not corrupt traffic when fragmenting into 24-byte segments.
We found that all of the devices that initially passed this test, continued to pass. In addition, we determined that TippingPoint does block the 24 byte fragmentation evasion with the non-corrupted attack. The findings for all other evasion tests remain unaltered.
The Q4 2009 IPS Group Test is being modified to reflect this update.
Jun 17, 2010
Approaches to Detecting Evasion
When an attacker uses an evasion technique, he is altering traffic so that it cannot be detected by a security product such as a Network IPS. To accomplish this, the traffic is run through a tool which manipulates the data stream and modifies it using a pre-determined pattern – similar to an encoder. Thus, to detect the attack, a network IPS needs to do the same in reverse – in essence, decode the data stream.
Alternatively, an IPS can drop traffic that appears to have been altered (e.g. fragmented or segmented) under the assumption that it is bad. Unfortunately, this is not the case much of the time since legitimate network traffic comes in all shapes and sizes. Thus, when vendors elect to drop such traffic instead of normalizing / decoding it and inspecting the content, they drop legitimate traffic. Knowing this, we have found that multiple vendors turn off those anti-evasion protections by default. This is a problem.
And this is why NSS Labs tests anti-evasion using vendor default settings.
Subsequent to our last IPS Group Test, NSS Labs found a loophole in our testing where multiple vendors enabled evasion detection that would block legitimate traffic. These evasion defenses would therefore never be deployed in the real world. We are therefore adding false positive testing for evasions to our upcoming IPS Group Test scheduled for Q3 2010.
Alternatively, an IPS can drop traffic that appears to have been altered (e.g. fragmented or segmented) under the assumption that it is bad. Unfortunately, this is not the case much of the time since legitimate network traffic comes in all shapes and sizes. Thus, when vendors elect to drop such traffic instead of normalizing / decoding it and inspecting the content, they drop legitimate traffic. Knowing this, we have found that multiple vendors turn off those anti-evasion protections by default. This is a problem.
And this is why NSS Labs tests anti-evasion using vendor default settings.
Subsequent to our last IPS Group Test, NSS Labs found a loophole in our testing where multiple vendors enabled evasion detection that would block legitimate traffic. These evasion defenses would therefore never be deployed in the real world. We are therefore adding false positive testing for evasions to our upcoming IPS Group Test scheduled for Q3 2010.
May 27, 2010
Passions of an assessor: Donde esta corazon?
Michelle is a passionate infosec pro and assessor. She gets some kudos today for expressing on a personal level the frustrations of many infosec practitioners whose job it is to audit, assess and help improve their clients' defenses. PCI DSS forces those who would do little or nothing for security to do something more. It also encourages those who would do more to do less because it is just enough to deal with a clear and present threat: the audit.
As Josh Corman at the 451 Group likes to say: “Why focus on compliance instead of security? I might be hacked, but I will be fined.” (if you handle cardholder data). Given the amount of client-side attacks and botnet infection data we see, the case could be made otherwise. Corporations are getting attacked daily. They might not be aware of it though, due to the holes in their security defenses, logs, and even alerting practices.
After all, security products can only alert and report on what they have detections for. Based on our testing, that leaves a significant gap with every vendor, between 12 and 83%. Do you know which holes matter on your network and where they are? Want to hear ideas on how to improve and not just pass?
I'm happy to echo Michelle's call for more heart and less check box.
Rick
As Josh Corman at the 451 Group likes to say: “Why focus on compliance instead of security? I might be hacked, but I will be fined.” (if you handle cardholder data). Given the amount of client-side attacks and botnet infection data we see, the case could be made otherwise. Corporations are getting attacked daily. They might not be aware of it though, due to the holes in their security defenses, logs, and even alerting practices.
After all, security products can only alert and report on what they have detections for. Based on our testing, that leaves a significant gap with every vendor, between 12 and 83%. Do you know which holes matter on your network and where they are? Want to hear ideas on how to improve and not just pass?
I'm happy to echo Michelle's call for more heart and less check box.
Rick
May 13, 2010
Thanks for breaking it!
People hire us to break stuff (and lately we're pretty good at it). Well not just, but breaking is part of testing, as is validating You'd think folks wouldn't want to hire us for that, and a lot of times you'd be right. But, this week, we had a large networking vendor in the lab testing their product. On day 2 we discovered a significant vulnerability that we were able to exploit. We replicated it before their eyes. What did the vendor do? He gave our lead engineer a high five!
Why? because, after having visited several labs with this same product, we were the first to find something and not simply give it the 'rubber stamp.' This is why you test. Not just to validate features, but so you can find out what you still need to do to improve it. Good product developers like this one "get it." He just got tremendous value out of the engagement, and has already put in proposals on the spot for additional testing with us. And maybe his competitors have similar issues (which is often the case), so now this vendor is ahead of the game and will likely have it fixed very shortly.
IT Buyers: This is the attitude you want to see in your vendors.
Subscribe to:
Comments (Atom)
Posts
