How long is a product certification valid?

Recently we have been asked about some of our older product certification reports, whether or not they were still valid; what's changed, etc; some all the way back to 2001. So just how long is a product certification valid?

From an IT Security buyer's perspective, the question is really: how long after the certification does the product still offer similar effectiveness, performance and usability characteristics? How well do they still meet the essential criteria?

1. Unlike static applications, security products with updates (signatures, heuristics, code, patches) change frequently in order to remain effective. (IPS products generally release new signatures on a weekly or daily basis. Antivirus products are becoming increasingly dynamic: last year Kaspersky was pushing hourly updates, and recently McAfee and Symantec have boasted 'real-time' updates.) Thus, a product could increase or decrease effectiveness significantly even 6 months out.
2. Performance can change anytime the code is changed. Yes, even a 'little' maintenance patch can have pronounced effects on throughput, state tables, latency, etc. To be fair, the converse is true: a vendor could release a patch that improves performance. Oh, and the more signatures that are turned on by default generally consume more resources and thus negatively affect performance.
3. Unfortunately, management capabilities don't change often enough. So if an interface is 'so-so', you can probably count on having to live with it for a while. Intuitive, easy-to-use interfaces is one of the underserved areas of security products.
   

These are all things that buyers should check on, whether it is in an NSS Labs report, or some other evaluation. The short answer (which I saved for last) is that a certification can be leveraged by a vendor for one major release cycle. These are generally 18 months long. Any new major release, and buyers should really ask for an updated report. Beware of certifications that are 2, 3, or even 4 or more years old.

Here's a little-known trick! Carefully scrutinize products that have not changed the major version number in a loooong time. Some vendors keep the same major version and modify minor numbers only for years on end in order to circumvent recertification requirements of painful things like common criteria.

NSS Labs does not withdraw certifications after an arbitrary period of time. Perhaps we should; some other labs do, and we could likely make more money to be blunt. Instead, we rely on vendor willingness to 'step up and show their mettle.'