Tales from the Trenches of IPS Testing

An update on our current 2010 IPS group test.


Those of you following network IPS know that the last NSS Labs IPS group test in Q4 2009 made quite an impact in the marketplace. The testing soundly destroyed the notion that an organization could buy one of the ‘leading products’ and rest easy
- Vendors with leading market share and analyst accolades were shown to have mixed to poor results on our robust exploit testing;
- Over half the vendors lacking coverage for well-known evasion techniques
- Many put performance ahead of security effectiveness

We were pleased to work collaboratively with many of the vendors in ensuing months to further identify and rectify issues in advance of our next group test. In the process, several vendors released new hardware and software. Excited as we were to test drive the new improved models, we were disappointed in more than a few situations to discover the latest and greatest versions had show-stopper level flaws that could cost their customers a great deal of money and time. A number of vendors requested additional time to remediate flaws, and it was clear that much would change very quickly.

Meanwhile, the size of the NSS Labs 2010 IPS group test has also grown in number of participants and complexity, making this by far the largest, most in-depth group test of its kind.
- We’ve added several new vendor products;
- We’ve refreshed the exploits used, changing and upgrading a third of the content;
- Added additional HTML evasions;

Determined to not let further delays keep our much awaited group test off the streets any longer, we decided to take a phased approach, releasing the test in two parts; Part I will contain five vendors, and Part II will contain the remaining vendors. While this phased release provides enterprises with much needed information on some of the top vendors, it arguably leaves them waiting to see the rest. This may be more or less relevant depending on one’s situation, and which products are under consideration. Certainly some vendor sales teams maybe playing catch up. In the meantime, NSS clients can utilize analyst advisory sessions to receive additional guidance and help fill in the gaps.

So, look for part one of the NSS Labs IPS group test to arrive middle of next week here, and part two in the middle of December. Clients will automatically be notified when it is posted via email alert. If you’re not a client, register for free here, or contact our advisory services group to learn more.