This week at RSA in San Francisco, NSS Labs released their Advanced Endpoint Protection test results. We, ESET, wish it to go on record that we were never invited to participate with our products in this test. In the test results published in 2017, we experienced numerous issues and NSS Labs failed to publicly correct all the inaccuracies despite their agreement to remedy them at a meeting in April 2017.
At the same meeting, we requested early visibility of the methodology for the 2018 test. We received no communication on methodology and in October 2017 we requested it again; after numerous, repetitive email requests, we received the document. The ESET team documented questions and concerns on numerous points in the methodology; we sent our concerns to NSS Labs in October; and they never responded to any of our concerns or questions. Again, after numerous emails and two months later, we eventually managed to get a representative of NSS Labs on the phone. They admitted to not having read our concerns and provided no answers to our questions.
During this call, it transpired that invites to the endpoint protection test had been sent to vendors; we were surprised because we never received one. They committed to come back to us on this but to date have not communicated with us about the missing invite.
We even contacted their CEO in January and he agreed to talk to the NSS Labs team and come back to us on all the outstanding points, including the missing invite and the questions on methodology. They never did. However, after further follow-up by us, they did claim they sent us a PDF document. We never received this, and we were told by NSS Labs this was probably due to issues with their email system. They never resent it.
When you read the test report, you need to be aware that some vendors paid for a pre-test; vendors that received an invite had the opportunity to optimize the configuration of their product, and they have been able to dispute the results to improve the test score for their product.
ESET is committed to providing best-in-class security products and solutions to our customers. Our resources are focused on working with ethical and professional companies; you can see the results for our products in many industry tests. When or if you read the NSS Labs test results document, we hope you find it belongs in the circular gray filing cabinet under your desk, the same place I put my copy of the report.
Read more about our opinion on testers and testing at welivesecurity.com.