THE CONNECTED LIFE

How Well Can You Hear Audio Quality?

John Sciacca  |  Apr 6, 2016

Last summer NPR posted a story entitled ”How Well Can You Hear Audio Quality?” that let users “blindly” compare six different tracks recorded at identical volume levels at three different qualities: MP3 at 128 Kbps, MP3 at 320 Kbps, and uncompressed WAV. The goal was to see if listeners could discern the quality difference and then correctly pick which version was WAV.

After thousands of results it turns out that people can tell the difference. But just barely.

On average, people were able to correctly identify the WAV version 36% of the time. Or statistically just slightly better than the 33.3% that would come from randomly guessing. However, if you consider identifying 50% or fewer of the WAV files as “failing,” then 80.6% of listeners “failed” the test.

Of all the listeners, only 1.6% were “golden ears” that scored a perfect 6 out of 6 with another 4.5% identifying five tracks correctly. But the combined total of these top-tier listeners nearly equals the 5.7% who got every track wrong and falls far short of the 19.6% who only managed to get one correct.

What does this mean?

First, I think it shows what a great job that MP3 coding actually does. Many people slag MP3 and the terms “garbage” and “MP3” are often used virtually interchangeably in the audio community, but this shows how efficient MP3 files are without causing major, noticeable damage to the music for the vast majority of people. So if a large percentage of your music files are ripped in MP3, it might not be the catastrophe the audio world wants you to think it is.

Another part of it could be that since so many people are used to doing the majority or even all of their listening via compressed music—whether files stored on their phones or streamed via Pandora, Spotify, or the like—this is the barometer of what they compare audio to and this is what sounds “right” to them. Critical listening is an acquired skill and people need to learn how to listen. For the people that boasted of scoring perfect scores, many of them claimed some years of work in recording or engineering so they have a background in picking out the subtle nuances in audio quality.

The audio clips selected cover a pretty wide swath of genres and include Katy Perry’s “Dark Horse”, Jay Z’s “Tom Ford”, Coldplay’s “Speed of Sound”, Neil Young’s “There’s A World”, Suzanne Vega’s “Tom’s Diner”, and Murray Perahia & The English Chamber Orchestra playing Mozart’s Piano Concerto No. 17, K. 453 (II. Andante). The test randomizes the listening order and quality order each time so you can take it over and over and have a truly “blind” experience each time. You can also listen to each clip as many times as you like before deciding.

I’ve taken the test multiple times on several different headphones ranging in quality from sub $100 earbuds, to $400 Definitive Technology Symphony over-ear models, to $3000 Shure KSE1500 in-ear electrostats with an outboard amplifier and I’ve found it to be terrifically educational. I’ve also found my scores continue to improve as I take it more and more, learning what to listen for and what subtle nuances to focus on. While I’ll admit I’ve yet to card a perfect 6/6, and continue to find picking between the WAV and the 320 Kbps versions to be challenging, I rarely end up picking the 128 Kbps version.

So, how well can you hear audio quality? Click here to take the test and share your experiences in the comments section below.

COMMENTS
mardukes's picture
Submitted by mardukes on April 6, 2016 - 10:45am

The 320's all boosted the bass to make them more attractive, not more accurate. And virtually none of them are worthy tests. All but one features vocals. Hell, the Susan Vega test didn't have ANY music on it. This is exactly the point. The change to digital media drove out the musical instruments and musicians and left nothing but this whining with band-box background. If that's the kind of stuff you listen to then have at it.

ednaz's picture
Submitted by ednaz on April 6, 2016 - 3:49pm

Great photographers can detect excessive post processing of an image. Great musicians (I hang with quite a few of them) can hear, without fail, things that those who've not spent a few decades training ears can't hear. It's simple to prove. The mass market can't tell that voices have been auto-tuned, and mostly can't tell if someone's a few beats off pitch. I have positive pitch (and 15 years of musical training), and I can't watch any of the amateur performance reality shows - put my teeth on edge. And then there are those with perfect pitch, who are annoyed by things that I can't detect.

Evaluating something based on the skills of the masses - that way lies always compressed, all the time. Until music files get to 256kps, I can't listen even at background music levels. I've nailed (over 90% accurate) many tests by skeptical friends, none of whom were trained musicians or actors. I'm sure they can't tell the difference, I'm sure it doesn't matter for them, and how wonderful it must be to be happy with files that let you store days of music on your phone. My discernment stops at 24/192. I have some friends, who spend their lives in studios, who laugh at me for my sadly compromised hearing.

WildGuy's picture
Submitted by WildGuy on April 6, 2016 - 5:36pm

i was shocked when i got the first question wrong and that one i picked 128 kbps. if i remember correctly, i got 2 wrong that i picked 128 kbps. i got the 3rd and 6th ones right. when i got the first 2 wrong, i felt so disappointed in myself and i felt embarrass too. when i got the 3rd one right, i felt happy. then i felt disappointed when i got the 4th and the 5th one wrong. And felt a bit happier when i got the last one right.

So, overall i got 2 out of 6 right. disappointing but then i realize that is what most people got right too which is around 33.3%. so i shouldn't felt too bad about myself. btw, i took the test with headphones.

mtymous1's picture
Submitted by mtymous1 on April 6, 2016 - 5:37pm

...between golden ears and perfect pitch. Anyone recall any data on this? (Okay well, not expecting anyone to recall data, but at least a few article links to some data??)

Tangential's picture
Submitted by Tangential on April 7, 2016 - 4:12am

'Nuff said.

Johan81's picture
Submitted by Johan81 on April 7, 2016 - 9:40am

Very useful comment like this on this site that always wants to inform people with good arguments. Thanks for nothing!

TRoher's picture
Submitted by TRoher on April 7, 2016 - 2:21pm

Got 4 of 6 correct, and the ones I missed were a cappella and rap with little more than a bass beat. Ears may not be golden - but maybe I take a silver?

Oreo's picture
Submitted by Oreo on April 8, 2016 - 7:34am

I only got one correct, but it always came down to the two higher quality formats. I was listening through my computer with a pair of Creative Sound Blaster Sigma headphones, which I originally bought for gaming. Not audiophile grade by any means, but I know how they color and veil the sound.

I'm curious to try it on a different system, but I don't have anything approaching true audiophile levels. I'm not sure where the deficiency is, but in one of the tracks I could hear distortion of the highest frequencies. I don't know if that's in the recording, the laptop circuitry, or the headphones themselves.

X