New Consumer Reports ratings strategy makes ‘07 treadmill list look like '06 reprint

A new approach to how Consumer Reports magazine revises its ratings has left some wondering why the February 2007 treadmill review in the magazine looks as if it is just a shortened and reprinted version of the 2006 story.

Geoff Martin, the magazine’s director of consumer sciences who oversees the exercise equipment area, told SNEWS® the magazine has decided in many categories to update listings and ratings more frequently and house ongoing information on its website and then run occasional updates in the magazine.

“There are certain categories where our customers will be able to see current information all the time” on the web, he explained. “Treadmills are obviously a very important category so the idea is, we want to keep it up-to-date.”

That means in these categories, analysts will revisit the products and their availability more frequently, drop discontinued models, test new models, and then update a rating and chart as seen fit.

“Treadmills was one of our highest-priority categories,” Martin said.

For the current magazine treadmill rankings, the team tested “as many as we could fit in,” he said. He noted that in all cases, the non-profit organization reports on a product if it tests it, unless it explains otherwise in accompanying text. To decide if it will include a brand, he said the teams take into consideration market share, advertising exposure and general visibility in the marketplace, as well as price.

“The models here were thoroughly tested,” Rich Handel, project leader for the exercise equipment area, told SNEWS®. SNEWS® is also aware that a Consumer Reports representative was at the 2006 Health & Fitness Business Show talking to suppliers.

Although in recent years Consumer Reports has stepped into higher price ranges for products reviewed — based on research that its users’ households fall into a high-income category — it still will eliminate some brands from review if the team feels it is beyond what most consumers would pay. For example, in the last year the treadmill review included several that had list prices of $3,500, which the magazine’s editors said they are considering may be too high for future reviews. In addition, it is asking whether it can continue to rank $3,500 treadmills against those that list at closer to $1,000, although Martin said it currently has no plans to split the reviews into two price categories.

“It’s clear people want to spring for high-end stuff,” he noted.

At this time, the review is broken into “folding” and “non-folding,” with the lower-priced models falling mostly into the folding category. In addition, the total number on the list was trimmed back from 24 in 2006 to 18, and the story length was cut from two pages to one and a third pages, although much of the same information is presented.

In the non-folding category, eight treadmills were on the list instead of last year’s 11, with the bottom three — Star Trac, Trimline and Schwinn — disappearing. Handel said those three came off because the companies indicated to the magazine’s fact-checkers that those models would no longer be available when the ratings appeared and any replacement model would also not yet be out. Martin acknowledged it appeared as if those were “dumped,” but they were not.

“We report on all models we buy and test,” Martin said, “and if it’s not there, unless we specifically say so, it’s said to be no longer available.”

The Vision Fitness model in the No. 6 spot (T9500 Deluxe) was given a “Best Buy” last year, but this year it lost that although the chart lists the same ratings and same price. Handel said that’s partly because the other Vision model on the list in the No. 8 spot (T9200 Simple) received a Best Buy.

SportsArt’s less expensive treadmill was replaced with a higher-end one (TR32) but was given the same ratings and stayed in third.

“It’s coincidence it came out in a similar position,” Handel said.

The ratings for SportsArt (3rd), Precor (4th) and Vision (6th) were all higher based on the colored in circles for ease of use, ergonomics, exercise range and construction than the NordicTrack model in 2nd. But Handel said the “blobs,” as they are called in Consumer Reports-speak, represent a range of points (such as 4.5 to 5.4) and a product could fall lower than another once the actual points are added up.

In the folding category, the total number was cut from 13 to 10, with the bottom three — Spirit, one ProForm and one Image — disappearing. Overall on the list there were a few more flip-flops than on the non-folding list, but the top three remained the same. One exception: As with the Vision ranking in the non-folding category, a Horizon treadmill still in third that has the same price and ratings as last year simply lost its 2006 “Best Buy.” However, Martin and Handel said that treadmill (PST8) has regained its Best Buy on the chart on the website.

The only new brand that appeared was LifeSpan (TR 2000HR), which ended up in 9th overall. It also took over one “quick pick” spot (where the magazine calls out a couple models it deems good for particular needs) in the “if space is limited” selection, which was taken last year by an Image model called “fine for walking.”

For the future, a mid-year update is possible, as well as another look later this year at ellipticals and bikes.

SNEWS® View: Martin said Consumer Reports has had some complaints from readers who think they are being cheated, but CR is actually doing more updating than in the past. It’s just that it’s not appearing in the magazine. In fact, he noted that only about 25 percent of the magazine’s 4 million+ readers overlaps with those who subscribe to the web. No matter what CR does, of course, no list will ever be definitive since models and features are truly a moving target.