Evading Anti-malware Products

Anti-malware products are not 100% effective, as evidenced by our recent anti-malware tests. In fact, some seem to be falling behind the bad guys.

Why? Because the bad guys are smart and aggressive. And remember, cybercriminals need only find one open door to get in, whereas defenders need to protect all the doors.

Cybercriminals are employing a plethora of techniques in a highly automated fashion to evade detection. Gunter Ollman and the Damballa team have written a nice paper explaining malware evasion techniques. These automated methods allow bad guys to create massive amounts of unique malware that can circumvent AV software. Popular techniques include using:
1. Crypters
2. Protectors
3. Packers
4. Binders
5. Quality Assurance
See the well-written paper for a more complete discussion. This is why AV products are having to evolve, and quickly.

Awareness Month

October has been national Breast cancer Awareness Month for the last 25 years. For the last 5 years, it has also been the national Cybersecurity Awareness Month. I don't know how these things get set, but it seemed to me there were a couple common threads going on.

Both are major problems for our society. One is a condition when cells replicate uncontrollably, the other a premeditated malicious digital attack. In 2009, there are 193,000 new breast cancer cases expected. My mother is a breast cancer survivor, thanks to early detection, great doctors and divine will. And we all likely know someone who is a cyber-security attack survivor: after all, there are 339 million victims of data loss and breaches (see: Data Loss DB and Privacyrights Clearinghouse).

When it comes to breast cancer, early detection is the key; there are even earlier technologies than the mammogram. But, what's the corollary for cybersecurity? Testing of course! Testing of our knowledge of threats and best practices. And testing of our defenses: whether individual products, or layered defense architectures and policies.

Unfortunately, there is far too little testing going on. Erecting defenses and not periodically evaluating their effectiveness is a far too common practice. Requirements of certain compliance regimes like PCI DSS are helping drive awareness and require at least some level of testing. However, there seems to be a common perception that you can 'set it and forget it.' For technologies like IPS and anti-malware that require constant updating, nothing can be further from the truth.

Are anti-malware products a commodity?

As any antivirus/security vendor will tell you: "No. we're not all the same. Mine is better." :-) Yet, in the Sept 2009 issue of Infosecurity Magazine, Forrester analyst Natalie Lambert suggests that all anti-malware products are essentially the same. "Generally speaking, antimalware is antimalware; what you get from one vendor is not much different than what you get from another."

It is perhaps understandable how one might believe this given all the marketing and the sheer difficulty in empirically discerning otherwise (but not really for an analyst). Much of the testing shows scores between 98 and 99%. And other long-standing organizations have essentially declared as much through their certifications. Dozens of products have achieved the Virus Bulletin VB100%(tm) award, and still others tout the Westcoast Labs Checkmark(tm) certification as a moniker of distinction. And ICSA Labs has certified 52 antivirus products to be up to snuff. So they must all be great, right?

Wrong. This is where real-world independent testing comes in that actually measures meaningful differences, like proactive protection (keeping malware off the machine), time to add protection, and protection over an extended period of time. In our recent Group Test of corporate and consumer endpoint protection products using our Live Testing methodology, we found a dramatic stratification of products' abilities to stop socially engineered malware (the kind that tricks users into clicking 'download and run'), currently the largest infection vector. Here are some key findings from the consumer report:

• Proactive 0-hour protection ranged from 26% to 70%
• Overall protection varied between 67% and 96% (over the course of 17 days of 24x7 testing)

Given such vast differences in real-world effectiveness, what value are these certifications anyways? In our opinion, not much... The threatscape has accelerated and some vendors are adapting faster than others. Unfortunately, testing was not adapting, resulting in a huge gap in trusted, real-world knowledge. The ensuing false sense of security creates greater risk for companies and consumers. We are filling that hole by delivering data based on our Live Testing methodology.

Since we performed these tests on our own, without any vendor funding, we are selling the group test of corporate endpoint protection products. See all the anti-malware product reports.

Which products we tested:

1. AVG Internet Security, version 8.5.364
2. Eset Smart Security 4, version 4.0.437
3. F-Secure Client Security version 8.01
4. Kaspersky Internet Security 2010, version 9.0.0.459
5. McAfee VirusScan Enterprise:8.7.0 + McAfee Site Advisor Enterprise:2.0.0
6. Norman Endpoint protection for Small Business and Enterprise
7. Sophos Endpoint Protection for Enterprise - Anti-Virus version 7.6.8
8. Symantec Endpoint Protection (for Enterprise), version 11
9. Panda Internet Security 2009, version 14.00.00
10. Trend Micro Office Scan Enterprise, version 10

What % of threats do you expect your anti-malware product to stop?

We are about to publish a new round of anti-malware testing data and would like to compare perceptions with reality. I'm expecting some interesting results to say the least.

7 simple questions here:
http://www.surveymonkey.com/s.aspx?sm=oiGBnkYL3i_2bBTEE4P24QNA_3d_3d

Thanks for your help

Q3 2009 Browser Security Tests Published

Today we published our 2nd round of live browser security tests. Two separate tests measured protection against phishing and socially engineered malware across 5 browsers: Apple Safari 4, Google Chrome 2, Mozilla Firefox 3, Opera 10 Beta and Windows Internet Explorer 8.

A key take away is that while the other browsers maintained or decreased protection between the two tests, Internet Explorer continued to improve its protection against cybercriminals.


Socially engineered malware is the most common and impactful threat on the Internet today, with browser protection averaging between 1% and 81%. Internet Explorer 8 caught 81% of the socially engineered malware sites over time, leading other browsers by a 54% margin. Safari 4 and Firefox 3 caught 21% and 27% respectively, while Chrome 2 blocked 7% and Opera 10 Beta blocked 1%.


Phishing protection over time varied greatly between 2% and 83% among the browsers. Statistically, Internet Explorer 8 at 83% and Firefox 3 at 80% had a two-way tie for first, given the margin of error of 3.6%. Opera 10 Beta, exhibited more extreme variances during testing and averaged 54% protection. Chrome 2 consistently blocked 26% of phishing sites, and Safari 4 offered just 2% overall protection. Firefox 3.5 crashing issues prevented it from being tested reliably.

The full text and analysis of these and other reports on browser security can be found at http://nsslabs.com/browser-security.

NSS Labs live testing methodology represents an accurate, real-world testing that can be performed on information security products.

- Newly discovered malicious phishing and malware sites were added to the test, which repeated every four hours 24x7 for a minimum of 12 days

- All five browsers tested URLs simultaneously

- All sites were validated before, during and after via multiple methods

Google Drives Security Topics in the Media

At Blackhat 2009 in Las Vegas there was an interesting panel discussion with some very seasoned journalists who cover the security market. The question came up: "How do you [journalists] decide which topics to cover?"

The answer included the expected: they rely on contacts, relationships, identifying trends and major news. But, almost all of the agreed on this: Google influences the news. Google traffic, page views, etc. Editors are business people too. And the more viewers the more the property is worth to advertisers. Thus, when Paris Hilton's cell phone gets hacked, or another star's twitter or facebook account are compromised, this counts as top news. People want to read it.

Similarly, the panel agreed there was a focus on the 'bad news'; the discovery of a vulnerability or exploit against a popular service or product. It was difficult for journalists to cover the solutions or positive trends as this would come close to promoting products, it was argued.

Endpoint Protection Group Test Started

NSS Labs is continuing its testing of anti-malware products and has started its first group test of endpoint protection products. We are testing the ability to protect against socially engineered malware downloaded from the web. This is a continuous live test that will measure time to protect, and average protection over time. All systems are connected to the live internet and subjected to actual downloads of actual, fresh malware every 4 hours over a period of 12 days.

Both consumer and corporate products are being evaluated. Stay tuned for more information or contact me with any questions (rmoy).