How We Test Products at SPY
As a men’s lifestyle and commerce publication, SPY connects its readers with best-in-class products in the grooming, home, style, and fitness spaces. Over the past few years, our team has tested thousands of products with one goal in mind: to help our readers shop smart. So how do we do that?
The SPY editorial team includes product testers, commerce journalists, and product reviewers with decades of combined experience. From our office in New York or from our own homes, we’re always trying out new products for SPY readers.
To find and evaluate the best products in a given category, we scour the web, carefully read customer reviews, compare product specifications, and interview experts. For example, when writing about the best face wash for men, we’ll consult dermatologists in addition to relying on our grooming writers’ expertise in this subject area. We have also implemented a rigorous product testing process to ensure SPY readers are getting the best possible product recommendations.
To be as transparent as possible, we wanted to outline our product testing process in detail. You can also refer to our About Us page for more information on our editorial process.
The Testing Process
Our editors hand-select every product we feature on SPY. Our coverage includes product roundups, one-off product reviews, as well as news, trends, and sales coverage. Brands cannot pay SPY for positive reviews, and every review and shopping guide on our site is based purely on our editorial recommendations.
Below are the steps we take when writing our reviews and review roundups.
Part 1: Research
We start by conducting market research to learn more about the overall product category — which brands are the biggest players in the space, and which products people love or have questions about. We also interview experts and ask for their product recommendations, as well as for insights into what factors to consider when shopping for a particular category.
Part 2: Calling in Sample Units
In order to test products, SPY sometimes requests complimentary samples from a brand. When we call in a product, we make no guarantees about coverage; SPY has tested many products that we ultimately decided not to feature.
Part 3: Creating Judgement Criteria
As the sample units are arriving, we create a list of review criteria — a detailed list of specifications and features we think are relevant to overall product quality — using input from experts. The criteria will vary widely depending on the category. We will then ask our tester in-depth questions about this criteria, in addition to the product’s overall performance, special features, quality, and value.
Part 4: Testing
Now we get to the fun part: testing. SPY utilizes a pool of about two dozen experienced product testers.
When testing a product, our reviewers evaluate how it performs according to the criteria we’ve set for that product in its category. Testing can be a time-consuming process; sometimes, it can take months or longer for a tester to thoroughly evaluate the product and submit their review.. Once a tester submits their insights, our editors add them to relevant roundups or product reviews on an ongoing basis.
Part 5: Re-Testing and Updating
We regularly update our reviews and shopping guides with new thoughts on products we’ve already tested. Also, if a new product has garnered a lot of attention, we’ll test it to see if or how it fits into our existing ranking.
How We Test Products FAQ
HOW DO WE GET PRODUCTS FOR TESTING?
We use a number of different channels to acquire products for testing. The most common is requesting sample units from brands, who, more often than not, are more than willing to send them. When we call in a product, we make no guarantees about coverage; SPY has tested many products that we ultimately decided not to feature. If a product isn’t available as a sample unit, we’ll purchase it ourselves using a designated section of our site’s budget.
HOW LONG DO WE TEST PRODUCTS?
The amount of time we test a product largely depends on the nature of the product itself. More expensive, big-ticket items require longer testing because we want to ensure they’re worth the investment.
Most of the time, we spend between one and three weeks with a product. If a product is a big release, unique, or otherwise noteworthy, we’ll often dedicate an entire article to reviewing it. In other cases, we’ll test many different products for a review of the best products in that category.
Sometimes, we will update a review as we have new experiences with the product. When this happens, we will note in the article that the review has been updated with new information.
WHAT DO WE DO WITH PRODUCTS AFTER WE’RE DONE TESTING?
It depends on the product and the brand. Some brands request the unit be returned within a certain window, or after we’re done testing, while others encourage us to keep or donate the products after we’re done.
Some of our editors have kept sample units and continue to test them. We frequently upgrade reviews after three, six, nine, or and even 12 months of use to give our readers an even more in-depth view of a product.
WHAT DO YOU DO IF YOU DON’T LIKE A PRODUCT, OR HAVE A BAD EXPERIENCE?
Some product review websites have a “no bad reviews” policy, which is meant to keep the website on good terms with brands. At SPY, we will always be honest about our experiences with products, good and bad. Typically, when we don’t like a product, we simply don’t include it in our shopping guides. However, for especially notable products, we do occasionally publish negative or mixed reviews. Our guiding principle is to promote products we fully believe in and love.
HOW DO YOU MAKE YOUR MONEY?
SPY earns money when our readers trust our recommendations enough to purchase a product. When readers click a link on SPY and go on to purchase a product at sites like Amazon or Target, we may earn a commission on that purchase. This is standard practice for commerce publishers. Our full affiliate disclaimer appears on all of our content. It reads:
Our editors independently select the products we recommend. We may earn a commission on items bought through our links. Why Trust SPY?
In addition, we run advertising and sponsored content campaigns. We always label sponsored content using a “SPONSORED” label, and ads are always clearly marked. We do not accept money for reviews or guest posts under any circumstances.
