Dec 30, 2010

Network Intrusion Prevention Group Test Released

The security analysts at NSS Labs tested 13 different network IPS products, including stand-alone IPS and multi-function gateways, and one unified threat management product. If your organization is evaluating IPS solutions, or is looking to benchmark your current vendor, then this is the definitive report to read. Data and analysis are based on multiple man-years of complex, real-world testing that mimics how cyber-criminals are working to penetrate corporate defenses (see methodology). No surveys, interviews or soft trends. This is the hard test data upon which organizations base critical, big-dollar decisions.

The report includes valuable information not available anywhere else:
• Total cost of ownership analysis – are you getting the most security for your budget?
• Security effectiveness – how much effort is required to protect all your assets?
• Real-world performance benchmarks – can the device handle your traffic?
• Management and usability insights – how much time is really required to achieve results?

While the full breadth and depth of the research is available only to our subscribers, we are making a summary available to non-clients. Next week/year I will blog more about the key findings and what they mean for IT buyers in 2011.

Tested Products include (alphabetically):
2. CISCO IPS 4260
8. MCAFEE M-8000
11. SOURCEFIRE 3D 4500

Dec 19, 2010

Stopping malware with a browser

This week we released another report on socially-engineered malware protections delivered by browsers. While most articles and blogs seemed to interpret the results properly, there were some inaccuracies that we wish to address. Security is a complex field and the terminology can sometimes be misinterpreted. This can be compounded when vendors who did poorly add their spin, or when the data challenges one’s own beliefs.

Stopping Malware vs. Security:
Some of the key security terms are clarified in a previous post, especially relating to “most secure” browser. Some articles incorrectly stated that we found IE to be the “most secure” browser. What we tested was browser effectiveness at stopping malware from reaching a user or their PCs, not the security of the browser itself or its plug-ins. Modern browsers are a wonderful, free additional layer of protection. They work well with your favorite antivirus software. Browsers however will not stop malware coming via email or USB drives.

The Malware threat:
Malware is arguably the largest security threat facing users today – with more than 60,000 unique, new samples entering circulation each day (source: McAfee). There’s a $14B industry addressing the problem of malware. These test results challenge the comfortable status quo of many of the vendors. The notion that a free product adds so much protection can easily upset the industry apple cart. The assertion that the focus of the test was narrow (made by Google and some others) flies in the face of all generally accepted data. To say one’s browser was “built with security in mind” is nice, but marketing speak. What we’ve offered is hard data about malware protection. The exploitability of the browser is also a very important topic. But, even in this case, data doesn’t support Chrome being ‘more secure’ (i.e. less vulnerable) than other browsers (see CVEs, Secunia disclosures, etc.). We do sincerely applaud the innovations and bug bounties though, and encourage all vendors to build more security in.

Versions tested:
The claim that we tested an “old” version of Chrome is patently false. As stated in the report, the test was run in late September, when version 6 was the current browser. Since then Google has released two other so-called “major” releases, none of which have claimed improvements to the tested SafeBrowsing functionality. Here is the Timeline for Chrome Version Release:
Stable Version Release Date
5.0.375      2010-05-25
6.0.472      2010-09-02
7.0.517      2010-10-21
8.0.552      2010-12-02
Furthermore, Chrome’s sandboxing is designed for exploits protection; it does not protect against socially-engineered malware. You click it, it runs.

At the time of the test, Opera’s website marketed protection from malware as a feature, yet our results showed no protection was available. AVG officials have separately acknowledged that the integration of its technology was not yet complete, confirming our results. Features should only be marketed after they are actually in the product.

Application Reputation:
In a world of dizzying information, there’s much rush to a sound bite of who “won”. The most significant technological message of this test may have been overlooked. This test benchmarked the world’s first implementation of an application reputation system within free web browsers that goes beyond simple black lists. Nascent stand-alone security products such as SolidCore (acquired by McAfee), Bit9 and CoreTrace utilize what is commonly referred to as white listing, and commercial endpoint security products are starting to include some form of this as well. Apple uses a “walled garden” approach to limit exposure to malware on its tightly controlled platforms by pre-approving apps.

In web browsers, so far we’ve seen just black listing. A URL or application is either known to be bad, or unknown. What’s unique about Microsoft’s approach in the IE9 browser is that applications have 3 states: known good (white), known bad (black) and unknown (grey). The combination of good and bad indicators is clearly powerful, stopping 99% of malware via the web download vector. The use of application reputation to identify good applications and bad ones is unique to IE, for now. Will other vendors follow Microsoft’s lead?

Methodology and Open Invitations:
No vendor has influence over what/how we test or where we get malware from. We constantly run our Live Test network with a variety of security products - antivirus, browsers, and other network devices.
Over the past 2 years we’ve been running these tests, NSS has discussed results and methodology (which is included in the report) with all of the browser vendors; even providing sample URLs for validation to them in past tests. Some had even privately acknowledged issues.

Our past invitations to become more involved in the ongoing testing and review of results still stands.

Dec 16, 2010

Threat Types and Terminology

Terminology used to describe attacks is often misunderstood by the broader public. Thus, we are providing this brief explanation of threat types and the terms we use in our reports.

End users and their computers face a number of different attack types. At a high level there are two: 1) Socially-engineered attacks target the user, and work only when the user is tricked into performing an action; running a malicious file or giving up personal data to a fraudulent site. 2) Other attacks target vulnerabilities in systems and applications. The following chart gives a rough breakdown of common threats against end user systems.

Layers of Security
These types of security threats can be mitigated by a range of security products; including IPS, UTM, SWG appliances, and on the endpoint: Internet security suites, most anti-malware products, and even web browsers. Modern browsers have implemented an additional layer of security to help users differentiate between good and bad web sites and downloads.

When selecting security products, either for home or business environments, it's often hard to tell from the marketing literature which products actually stop threats. And protection levels offered by products in the different categories can vary greatly. The above taxonomy should help you ask more specific questions of vendors. It also acts as a guide to terminology used in NSS Labs test reports.

Security products protecting users and their computers
When someone says “Product X stops more malware, exploits etc.” or “Product X offers better malware or exploit protection”, what they mean is that Product X inspects traffic passing through it and stops these attacks from reaching and/or affecting the end user or the operating system.

Security products themselves susceptible to threats
In addition, security suites and browsers (and their plug-ins) can be susceptible to exploits if the software has vulnerabilities in them. When someone says “browser X is more secure” what they are trying to say is that browser X has fewer vulnerabilities. Unfortunately, most software, and all browsers have vulnerabilities. For example in the first 9 months of 2010, Microsoft Internet Explorer had 43 new published vulnerabilities, while Google Chrome had 106, according to Secunia research.

For more exhaustive treatments on threat types including product test results, consult our research services at