What's your next browser?

Web browsers have become the new killer app - serving as the platform for accessing our favorite personal and business applications in the cloud. As we've discussed previously on this blog and in our research, web browsers, and more often their plug-ins, represent significant vulnerability risks to individuals and organizations. This week we find ourselves in a unique point in time, when several major browser upgrades have been released (or are imminently upon us): Chrome 10, Firefox 4, Internet Explorer 9, Opera 11, Safari 5.

Many of the key enhancements include:
- rendering and standards compliance
- security and privacy, including 'do not track' provisions

- javascript and graphics performance acceleration

- 'enhanced' user interfaces

Which one will you upgrade to?

TippingPoint Tests

In response to inquiries about the blog posting made by TippingPoint President, Alan Kessler, we provide the following:

In August 2009, NSS Labs performed an independent test of the TippingPoint 10 product and determined it only blocked 39% of common exploits. Subsequently, TP came to our lab for private testing for further assistance, as they stated. TP customers can see a spike of hundreds of filters which appeared in October and November.

In early December, NSS Labs released its independent IPS group test of 15 different IPS products submitted by 7 vendors, including TP. The product improved marginally, but is rated ‘caution’ due to its subpar protection on our tests. Now TippingPoint has publicly complained in this tippingpointblog that the test must be inaccurate because it didn’t correspond with the results of their private testing with NSS, with 'customer experience', nor with their internal testing.

3 response points:
1st. These modern IPS products are so complex, that customers will rarely be in position to question or test a vendor properly. And they rarely do when it is a brand name. Very few enterprises have the sophisticated testing tools, expertise and access to exploits like the vendors and a professional security testing lab like NSS. So, having a lot of customers does not necessarily mean they are aware of the true protection they are receiving. In fact, not knowing is a liability in itself for all involved.

2nd. RE: Private testing results. At NSS we don’t use the same attack set in our private testing, as we do in public testing. That would be like getting a copy of the test and answers beforehand, and would give private clients an unfair advantage over other vendors. We do test the same vulnerabilities, but the specific exploits we use vary. This should underscore the integrity of NSS Labs testing principles and procedures. In general, differences in results could be attributable to signatures written too narrowly; e.g. for specific exploits vs vulnerabilities, or to signatures written for a test lab environment.

3rd. We certainly cannot account for any vendor's internal testing procedures. However, the findings of our two previous tests were ultimately corroborated.

As far as delaying the Network IPS Group Test Report. It would be unfair to enterprise readers all around not to disclose validated testing results that could help them mitigate threats that might not be stopped by their defenses as they expect (that could be considered irresponsible non-disclosure). A delay would also not be fair to the other 6 vendors who also participated. As with the previous tests, NSS took great care to validate the results. Deciding to act positively upon them to deliver the better customer protection is the next imperative.

Endpoint Protection Group Test Started

NSS Labs is continuing its testing of anti-malware products and has started its first group test of endpoint protection products. We are testing the ability to protect against socially engineered malware downloaded from the web. This is a continuous live test that will measure time to protect, and average protection over time. All systems are connected to the live internet and subjected to actual downloads of actual, fresh malware every 4 hours over a period of 12 days.

Both consumer and corporate products are being evaluated. Stay tuned for more information or contact me with any questions (rmoy).

Two acquisitions in two weeks!

Within the last 2 weeks, two young companies that NSS Labs did independent certifications on were acquired. ThirdBrigade, which makes Host Intrusion Prevention Software (HIPS) was acquired by TrendMicro, one of the major antimalware vendors. This product filled a server-side gap in their product line.
Solidcore Systems, which makes memory firewall/application white listing products, was acquired by McAfee. The #2 antimalware vendor cum security vendor has added whitelisting to its billion dollar portfolio of antimalware, vulnerability and intrusion prevention products. In Q3 of 2008, NSS Labs had evaluated and certified the S3 Control Embedded product as NSS Approved for Host Malware Protection.
In a down economy, strong vendors go shopping for technologies to round out their product lines so they're in positions of strength when the buyers recover. Note, even with all the cost cutting and layoffs, there's always money left for strategic purposes. And if you're a CEO who is going to make a purchase in this economy, there's not much room for forgiveness. So, you can bet they did their homework on all sides: technology, sales execution, management, margins, balance sheet, etc. I'm pleased NSS Labs was able to help these young companies grow their businesses and wish them well in the next stages of their evolution.

Why doesn't NSS Labs have a report on Product X?

Just because you don't see a product evaluation report on our website, it does not mean we have not evaluated the product. There are several possible scenarios:

• NSS Labs is in process of testing the product. However, due to NDA and confidentiality reasons we cannot disclose whether or not we are testing a given product until the vendor decides to make it public.
• The product vendor is waiting to release a new major revision before having it (re-)certified.
  
• The product was evaluated by NSS Labs, but issues were found that the vendor opted to fix before completing the public certification.
  
• The product simply has not yet been evaluated. NSS Labs operates meaningful and rigorous product testing. Not every vendor wishes to subject their product to this process.
  

NSS Labs makes every effort to involve product vendors in our tests. However, for various reasons, we cannot always secure their participation. Since you as a reader may not know which of the above cases is true, we recommend you inquire with the product vendor's PR or product management team.

How long is a product certification valid?

Recently we have been asked about some of our older product certification reports, whether or not they were still valid; what's changed, etc; some all the way back to 2001. So just how long is a product certification valid?

From an IT Security buyer's perspective, the question is really: how long after the certification does the product still offer similar effectiveness, performance and usability characteristics? How well do they still meet the essential criteria?

1. Unlike static applications, security products with updates (signatures, heuristics, code, patches) change frequently in order to remain effective. (IPS products generally release new signatures on a weekly or daily basis. Antivirus products are becoming increasingly dynamic: last year Kaspersky was pushing hourly updates, and recently McAfee and Symantec have boasted 'real-time' updates.) Thus, a product could increase or decrease effectiveness significantly even 6 months out.
2. Performance can change anytime the code is changed. Yes, even a 'little' maintenance patch can have pronounced effects on throughput, state tables, latency, etc. To be fair, the converse is true: a vendor could release a patch that improves performance. Oh, and the more signatures that are turned on by default generally consume more resources and thus negatively affect performance.
3. Unfortunately, management capabilities don't change often enough. So if an interface is 'so-so', you can probably count on having to live with it for a while. Intuitive, easy-to-use interfaces is one of the underserved areas of security products.
   

These are all things that buyers should check on, whether it is in an NSS Labs report, or some other evaluation. The short answer (which I saved for last) is that a certification can be leveraged by a vendor for one major release cycle. These are generally 18 months long. Any new major release, and buyers should really ask for an updated report. Beware of certifications that are 2, 3, or even 4 or more years old.

Here's a little-known trick! Carefully scrutinize products that have not changed the major version number in a loooong time. Some vendors keep the same major version and modify minor numbers only for years on end in order to circumvent recertification requirements of painful things like common criteria.

NSS Labs does not withdraw certifications after an arbitrary period of time. Perhaps we should; some other labs do, and we could likely make more money to be blunt. Instead, we rely on vendor willingness to 'step up and show their mettle.'

Greasing the skids of commerce

"Commerce requires a meeting of the minds between buyer and seller, and it's just not happening. The sellers can't explain what they're selling to the buyers, and the buyers don't buy because they don't understand what the sellers are selling. There's a mismatch between the two; they're so far apart that they're barely speaking the same language." Bruce Schneier on the security industry.

Having been on both sides of the vendor-IT buyer fence, I can definitely relate to both parties frustration. In this vein, some have referred to NSS Labs reports as 'next generation sales collateral', bridging the gap between brochureware and a proof of concept test (and who has time, expertise and resources for all that anyways).

How important is a user interface after all?

One important thing to consider when evaluating security products for any environment is manageability and usability. Having tested a vast array of products, it's probably safe to say we've seen a spectrum of good bad and ugly interfaces. But I'm not just talking about the look and feel. Far more important is the suitability to task: how well thought out are the most important and frequent tasks that a user will have to complete? Is critical information that I need to take action on represented effectively? How many clicks to get to it? Often times we get both excited and scared by large management frameworks. These can easily tend to present data in engineering terms of tables and lists without much thought to the objective. The last thing I want to see in a console is a lot of text in tables or generic plug-ins to meet some requirement to make data available. With so much R&D cost put into developing speeds, feeds and detection, are we as an industry investing appropriately in the equally important human interfaces?

Got an opinion about IPS?

If you're currently using an IPS, or in the market for one, we want to hear from you.

As an exercise to accompany our 10Gbps Network IPS group test, we decided to ask end-users what they like and dislike about their current IPS products, how they use them, and what they'd wish for in their next go around.

Simply take the short survey, and you could also win a $50 Amazon gift certificate.

RFI for leading network/test tools

NSS Labs continually evaluates and validates testing tools and best practices. This is a necessary step prior to selecting and implementing the best tools in our test methodologies, which result in our publicly published test reports. Our lab engineering team is thus requesting leading test tool, network infrastructure product and service providers to brief them on their offerings and roadmap. Best in class products will be selected for use in NSS Labs' next generation test facility. More info

Fastest Public Test of a Network IPS

As network traffic continues to grow, so too do the demands on network infrastructures. As a result, multi-gigabit network IPS devices are gaining traction, and providing essential protection in a switched core environment.

Yesterday, NSS Labs released a milestone report on what is the fastest independently verified Network IPS product on the market, to date - the IBM/ISS GX6116. (I say to-date because there are certainly a couple of 10Gig devices that have recently debuted, and we look forward to also testing these). What is notable here is that our tests are not based merely on RFC 2544 (UDP packet blasting), which can inflate a vendor’s performance metrics due to the stateless nature of UDP and typically large packet sizes used. (See our white paper on Pitfalls of Performance Testing). Rather, NSS Labs dedicates a lot of attention to creating real-world multi-protocol test suites across a wide range of use cases.

In our real world tests, we create a complex mix of protocols including HTTP, FTP, SMTP, DNS, etc and pass these through the device under (DUT) test at speeds up to 30 Gbps. This is a live test with deep packet inspection and default or recommended rules turned on. The Proventia GX6116 displayed excellent performance up to 6 Gbps coupled with extremely low latency under all normal traffic conditions.Security effectiveness was also impressive, with excellent coverage above 95% for the most critical vulnerabilities, out of a set of 579 – the largest set of exploits run in any public test.

Read the full report here: http://nsslabs.com/intrusion-prevention/iss-proventia-nips-gx6116.html

PCI Self-Assessment Questionnaires Embrace Use-Case Philosophy!

I have been meaning to comment on this for a while, but better late than never. Earlier this year, the PCI SSC released an updated, and well-thought out collection of self-assessment questionnaires to replace the previous, single questionnaire. This is a very welcome enhancement for a number of reasons, not the least of which is because it shows a clear support for a use-case-based approach - something NSS Labs has been working towards in its own testing.

In fact, we've written a white paper outlining how use cases can help IT Security and Compliance professionals evaluate products for appropriate usage in their environments. In short, know your environment, and specifically what you're trying to protect, and this will help you define more granular (and thus more useful) protection requirements for your control selections (i.e. security products).

There is no silver bullet or magic product, and in fact, as products are increasingly differentiating themselves, defining the requirements early on in the process is increasingly important. For buyers, this means being better prepared, and more discerning in the evaluation process. For vendors, this should be a welcome opportunity to claim some higher ground (in terms of positioning and differentiation) in some very 'mushy' crowded markets where customers turn quickly to price as a differentiator when they can't tell the difference in benefits.

Security Products & PCI Compliance

There's one compliance question that keeps raising its head - for every piece of hardware and software that's considered to be 'in scope' for an assessment. "Will this product make me compliant?" We've heard this through our advisory groups and discussions with information security pros, and compliance/risk management executives.

Fact: No product will make you compliant. But having an inadequate or misconfigured product can prevent you from achieving compliance.

That's not to say that product vendors are not scrambling to answer these questions from their customers with an affirmitive "yes! ACME's web application firewall will make you compliant." But then the hard part begins: First, by answering "HOW" specifically it does in a manner that will likely be convincing to assessors and card brands reading the reports on compliance. And secondly, by clearly articulating this message in a crowded, noisy marketplace of product vendors all claiming that their products will either make you compliant or help you achieve compliance.

The question should actually be broken into two distinct components:

1. Does this product have the features to support a compliant network environment? i.e. is it capable and appropriate for the use case?

2. Is this product properly configured and deployed according to PCI requirements?

If you have deep expertise and plenty of resources you can try to tackle question 1 on your own. And many Level 1 and 2 merchants do. Warning. It's a trickier endeavor than one might think. There are over 200 sub-requirements to the DSS and they are not necessarily all grouped around a particular product. e.g. you have security functionality, management features, update requirements, and procedures throughout. What has been missing up til now is a product-centric view of DSS requirements. This is where NSS Labs has come in with its partners and advisors to create a product validation scheme which addresses the requirements of PCI DSS. We are actively evaluating products against this standard and producing validation reports accordingly.

Regarding question 2, merchants and service providers are obligated to prove to assessors and their acquiring banks that they not only have the right products, but that they are configured properly. To this end, NSS Labs is including in its PCI reports several recommended configurations for various PCI deployments. For example, which settings in a UTM are necessary to deploy the product in a retail storefront? or what firewall configuration and policies are needed at the perimeter?

To be clear, only an assessor and ultimately the card brands can certify and validate a cardholder network as being compliant. NSS Labs' contribution is to provide independent, empirical validation of product suitability. We will be releasing the first reports imminently. Stay tuned.

We've heard from many corners of the industry that this is a good thing and merchants, assessors and banks are looking forward to seeing more and more products validated in this manner. What's your opinion? Let me know [ rmoy AT nsslabs DOT com ]