Questionable Questions (And Some Answers)

Normally, NSS Labs does not engage in public disputes over our test results. However, AVG’s recent blog post about our recent Operation Aurora test grossly misrepresents the facts in an apparent attempt to discredit the results and testers. We have chosen to respond:

The important fact for AVG’s 110 million users is: AVG Internet Security 9 did not stop the Aurora exploit. This was true when we tested on January 29, 2010. And it was still true when we re-tested with their latest version on March 12, 2010—nearly two months after the initial attack became public. See for yourself in this video (the exploit executes calc.exe as proof).

On AVG’s blog, they claim the following:

“This is a screenshot of AVG blocking the Aurora 0-day attack from the AVG Labs.”

• However if you look closely, the screenshot AVG presented shows they were using Firefox, not Internet Explorer. CVE-2010-0249 was a vulnerability in Internet Explorer, not Firefox. Showing Firefox being "protected" displays a fundamental misunderstanding of the nature of the Aurora attack.

"In fact, the exploit is blocked separately by three different security rules of AVG’s product"

• We don’t dispute that AVG has rules, but they did not prevent the exploit. This is why proper testing & QA is important. Further, as you can see in the video (using Internet Explorer), we found that AVG’s warning appears after the exploit successfully gained control of the computer and performed remote code execution (calc.exe).
  

AVG has failed to provide any credible evidence that our test results are incorrect.

From the moment that AVG contacted us with concerns, we sought to share the information required for them to reproduce the attack themselves. The Operation Aurora code was included within the report itself. We have since posted a video on YouTube, and we made it clear that the easiest way to reproduce the test was to use the Metasploit Framework's built-in (free) Aurora exploit and embed a payload of their choice (such as calc.exe). With this free, publicly available information, AVG engineers should have been able to reproduce this attack, as their peers at other vendors have.

However, AVG wanted us to do more…

During our years of testing, we have found that some vendors have abused the time and trust of testers by not doing their homework before making claims that test results are incorrect. We stand by our results. And in cases where vendors insist we have made a mistake, we will work with them to resolve any ambiguities. If it turns out that the vendor is incorrect, we expect to be compensated for our (consulting) time. If we made a mistake, we will publicly correct the error and the vendor bears no cost.

Under these conditions, AVG had nothing to lose if they were confident in their product. That they have chosen a different path speaks volumes.

Vulnerabilities, Exploits & Payloads, Oh My!

Over the past few years at NSS Labs we have interacted with some very smart people and been able to observe which technologies and approaches work better than others. And being immersed in information security product testing, it is easy to forget that not everyone has the same vantage point that we do.

I was having a conversation recently with a very smart Intel Hardware/Software Engineer when the Google / Aurora attack came up. He said that based upon the explanations from AV community, he assumed that the attack was not stoppable… I assured him that was not the case, and that there is a technical solution that can protect against attacks like Aurora. Host Intrusion Prevention technology exists and works by preventing the vulnerability (in this case CVE-2010-0249) from being exploited. If the vulnerability cannot be exploited, the payload (malware) cannot be delivered.

So the question is whether it is easier and better to protect against:

- One (1) vulnerability
- Six (6) exploits
- 55,000+ viruses, Trojans, rootkits and other malware PER DAY



I then shared that in our recent study of seven Endpoint Security (AV) products, only one product (McAfee) protected against Aurora and exploit variants. And that the AV industry as a whole is still focused on chasing malware and, as such, is ill equipped to deal with exploit-based attacks like Operation Aurora.

So the discussion turned to ways in which someone can be infected with malware from the web. Basically, there are two methods:

1. An attack on the User – i.e. tricking someone to download software that contains malware – video codecs, pirated software, fake AV, etc.
   
2. An attack on the computer – i.e. exploits containing malicious payloads (aka “drive-by” exploits).
   

The first is solved by a combination of user education and reputation systems like those provided in Internet Explorer 8, Firefox, Chrome, etc., which warn people that the software they are about to download is infected. Some AV products have this as well. The second is solved by HIPS. Not AV. Why HIPS? Because true HIPS products protect the vulnerability from exploit. They do this by operating in memory and inspecting data as it streams onto a computer as well as inspecting processes before allowing them to run. Traditional AV products do not do this; which is why AV vendors have purchased HIPS technology over the past few years (McAfee bought Entercept, TrendMicro bought ThirdBrigade, Cisco bought Okena, etc.).

But, in general, no vendor seems to have uniform integration of HIPS into their endpoint products either on the consumer or corporate side. It clearly takes some time to integrate these technologies. Unfortunately, the marketing message of “proactive protection” has gotten ahead of the technical reality. This is what we found in our Aurora test: http://nsslabs.com/test-reports/NSSLabs_Vulnerability-based%20Protection-Google-EPPv14.pdf

The bottom line is that the best way to solve the “drive-by” problem is for AV vendors to implement true HIPS which prevents a vulnerability from being exploited, and thus a malware payload cannot be delivered.

-----

However, nearly two weeks after the attack was made public, one product still does not stop the original exploit, and 85% failed to stop additional exploit variants. Few cyber-security attacks were better publicized than this one. It is concerning that such shallow coverage was delivered. Are lesser publicized but equally dangerous vulnerabilities better protected? Probably not.

In lieu of applying patches to thousands of machines, IT professionals rely on network and endpoint security products to provide a virtual shield for these vulnerabilities. Patching is usually prioritized and scheduled monthly. Deferring patching without good vulnerability coverage increases risk. When faced with these test results, organizations would be advised to install the Microsoft patch(es) as soon as possible.

AVG & The Aurora Exploit

Unfortunately, we have observed that some products rely heavily on file-based detection (of malware). These products are scanning for payloads once they hit the disk. The problem with this approach is that exploits occur in memory (e.g. Aurora) and might never touch the disk.

We do not know why AVG failed to prevent the Aurora exploit (which operates in memory). But we know that it did. We observed the exploit successfully gain control of the PC and perform arbitrary remote code execution (the exploit ran Calc.exe as proof). And we did observe AVG detect the attack in Internet Explorer's cache - after the fact.

This video was captured on March 11, 2010. We turned off automatic updates to preserve the version in time, which of course caused multiple red blinky warnings to be issued. We tested again (see 2nd video below) after updating to put to rest any FUD.




To be fair, AVG may have protection for other exploits.

However, as of today, nearly 2 months after the story of the attack first broke, a fully updated AVG still does not provide protection from the the original Aurora exploit. What is additionally concerning is that the product issues a pop-up message telling the user that the threat was detected and quarantined. See for yourself.

This video was captured on March 12, 2010. We turned on automatic updates, plus manually updated to ensure the latest protection was installed.

GroupThink and InfoSec - SecurityBSidesSF

At about 1:45h into the video, Vik Phatak begins his talk about group think in infosec. It is a well-reasoned argument for more aggressive testing in order to improve our defenses. We present methodologies and results from our testing of IPS and Anti-malware products to make the case that we have much work to do and must stay vigilant and innovative.