Passions of an assessor: Donde esta corazon?

Michelle is a passionate infosec pro and assessor. She gets some kudos today for expressing on a personal level the frustrations of many infosec practitioners whose job it is to audit, assess and help improve their clients' defenses. PCI DSS forces those who would do little or nothing for security to do something more. It also encourages those who would do more to do less because it is just enough to deal with a clear and present threat: the audit.

As Josh Corman at the 451 Group likes to say: “Why focus on compliance instead of security? I might be hacked, but I will be fined.” (if you handle cardholder data). Given the amount of client-side attacks and botnet infection data we see, the case could be made otherwise. Corporations are getting attacked daily. They might not be aware of it though, due to the holes in their security defenses, logs, and even alerting practices.

After all, security products can only alert and report on what they have detections for. Based on our testing, that leaves a significant gap with every vendor, between 12 and 83%. Do you know which holes matter on your network and where they are? Want to hear ideas on how to improve and not just pass?

I'm happy to echo Michelle's call for more heart and less check box.

Rick

Thanks for breaking it!

People hire us to break stuff (and lately we're pretty good at it). Well not just, but breaking is part of testing, as is validating You'd think folks wouldn't want to hire us for that, and a lot of times you'd be right. But, this week, we had a large networking vendor in the lab testing their product. On day 2 we discovered a significant vulnerability that we were able to exploit. We replicated it before their eyes. What did the vendor do? He gave our lead engineer a high five!

Why? because, after having visited several labs with this same product, we were the first to find something and not simply give it the 'rubber stamp.' This is why you test. Not just to validate features, but so you can find out what you still need to do to improve it. Good product developers like this one "get it." He just got tremendous value out of the engagement, and has already put in proposals on the spot for additional testing with us. And maybe his competitors have similar issues (which is often the case), so now this vendor is ahead of the game and will likely have it fixed very shortly.

IT Buyers: This is the attitude you want to see in your vendors.

Measuring Security

Vik Phatak, CTO participated on a panel discussion at SourceBoston conference titled "Measuring Security". This discussion explored the ins and outs of testing endpoint protection products, otherwise known as anti-virus/antimalware. Hosted by Andrew Jacquith of Forrester, and also with Peter Stelzhammer of AV-Comparatives, and Mario Vuksan of Reversing Labs. Watch the video link.

AV Testing double standards and independence

NSS Labs’ innovative tests are designed to inform end-users about how products truly perform against today’s motivated attackers. We perform a test or gap analysis on security products, so organizations can understand what is and isn’t being protected, and accurately assess the risk and take steps to mitigate it. While enterprises and government organizations appreciate this valuable, independent analysis, many of the AV vendors do not.

When NSS Labs published its uncensored, real-world results of endpoint protection products (AV), some vendors used the anti-malware testing standards organization (AMTSO) to try to discredit the test. One of their objections was that we recommend against buying products that scored on the bottom third of our test. Sorry, we unabashedly believe malware protection should indeed be the key purchasing criteria for an AV product. And for vendors who claim their anti-spam on the corporate desktop will improve their protection against socially-engineered malware hosted on web sites, that’s just stretching it.

Rather than shoot the messenger, vendors with their customer’s best interests in mind should seek to learn from tests like these in order to improve their products. Unfortunately, that’s usually not the case in the AV world after too many years of self-congratulatory testing and certification.

AMTSO is an AV vendor-driven consortium, and while it can be a useful information sharing organization for AV insiders, it has demonstrated its utter failure as a credible independent organization. Throughout the 3-year history of this organization, AMTSO has failed to evaluate the tests and certifications that most of its vendor members sponsor and fund; e.g. VB100% awards, ICSA Labs and West Coast Labs certifications. These validations are important sales material in the $9B market place, but they wouldn't pass the same AMTSO guidelines that were supposedly applied to the NSS Labs test.

Such market validations are a part of the industry, but can be dangerous when they convey a false sense of security to buyers as they do now. Meanwhile, end-users can stay well informed about what products do - and more importantly - what they DO NOT do, by reading our subscriber-funded research and test reports. If a vendor is complaining about our test, chances are they did poorly on an important metric. Learn what some vendors don’t want you to see by reading our independent anti-malware test reports or the Google Aurora protection analysis report in particular (free to non-clients).

caveat emptor