THE BIG PICTURE

It's been two weeks since I asked readers to weigh in on whether they want more audio reviews with no objective measurements or fewer reviews with measurements. I've received 58 responses so far—thanks to everyone who expressed their opinion! Many of you supported your position with additional thoughts—in fact, a rather lively debate appeared in the comments section of that blog, which offers some entertaining reading.

The results were unequivocal—of the 58 respondents, 40 want more reviews with no measurements. That's more than a two-thirds majority, enough to override even a presidential veto. (Would that Congress could get its act together with such numbers!) With so many products out there, these folks want more reviews and are willing to live without measurements, which seem to mean less to them than subjective evaluations from trusted reviewers. I even heard from some engineers who understand and appreciate measurements but for whom subjective evaluations—and a greater number of reviews—matter more.

On the other hand, blog commentator Tyler points out that objective measurements can support the trustworthiness of subjective reviews, saying, "The reason this is my favorite A/V website is because measurements have always been provided to back up the subjective reviews." He also argues that measurements encourage manufacturers to be honest in their specs: "If you don't think things like power ratings with all channels driven, frequency response, video performance, and so on have improved as a result of [independent] testing, you are deluded."

A number of interesting alternatives were proposed. For example, several people supported the idea of measuring only those audio products that exceed some to-be-determined price threshold. On the other hand, at least as many requested more reviews of lower-cost products that they might actually be able to afford, which seems to cancel out the idea of measuring only expensive items.

Of those who want measurements, several said that measuring speakers is more important than doing so with audio electronics, a position that has much merit. According to Mark Peterson, the consultant we hire to perform our audio measurements, the frequency response of modern solid-state electronics is normally close to ruler-flat, and distortion levels are generally quite low, which seems to indicate that taking such measurements isn't particularly useful.

The output power from amps and A/V receivers, however, varies more widely depending on the manufacturer—some companies publish realistic power specs that are verified by Peterson's measurements, while others' specs are wildly exaggerated beyond his results. Of course, the method used to measure output power greatly affects the outcome, and there is genuine debate over which method produces the most realistic results, making this a rather murky area of the whole measurement landscape.

Speakers tend to be much more variable than electronics in their objective performance, depending on cabinet design, driver complement, crossovers, etc. On the other hand, a speaker's subjective performance is highly dependent on the room in which it is evaluated, which makes me wonder about the value of measurements taken in a pseudo-anechoic environment. Tom Norton, UAV's über reviewer, believes that if a speaker measures well in such an environment, it's more likely to sound good in a variety of rooms, but if it measures poorly, it's bound to have trouble in most rooms.

That's not to say a poor speaker measurement guarantees poor subjective performance—for example, Michael Fremer loved the Sonus Faber Domus speaker system, even though it didn't measure very well. Does that mean his reviewing skill is suspect? Knowing Fremer as I do, I would say absolutely not. Instead, I believe the system happened to integrate well with his particular room.

This example leads me to believe that measuring speakers is important for shoppers, who should know how they perform both objectively and subjectively in order to make the most informed buying decision. Measuring electronics is less important, since most modern solid-state products tend to measure reasonably well. The one exception might be output power—ideally, shoppers should know if a receiver or power amp delivers the power it claims to.

In the end, I've decided on the following compromise—UAV will measure only the output power of AVRs and power amps and perform a full suite of tests on speakers. And we will always include a complete set of measurements in all reviews of video displays. I'm confident that we can publish a goodly number of reviews following these guidelines while providing the most important objective results to complement the subjective evaluations of our expert reviewers. That way, you will have all the info you need to make the best buying decision for your particular situation, and we'll be able to review a wide range of products without blowing the budget on extensive but ultimately unnecessary measurements.

Again, thanks to all who responded. But don't stop now—please send your comments, suggestions, and questions to me at scott.wilkinson@sourceinterlink.com. With your help, I intend to make this the go-to site for all the home-theater information you desire.