Blind Testing

[Keith_W]

I think terry j and Drizt are getting too caught up in the reasons why people might fail or why we need to do DBT's. It's irrelevant and if a person fails you have NO reason why they have failed. Linking failure with the placebo effect or whatever is just speculating.

Well said Dr X :eek:

[Tony M]

Then why is it mentioned in every thread?

Hmm, a bit of hyperbole there, Keith?:eek:

I certainly would not go near the topic now unless I was responding to a question in a specific thread;)

Cheers

Tony

BTW, any mention of "proving" and individuals "failing" causes me some concern as neither word sits comfortably with my concept of this subject.

[Keith_W]

Hmm, a bit of hyperbole there, Keith?:eek:

Not at all. Do a search for the posts made by our most well known double blind testing proponent and see how many different threads you come up with.

I am not a Tremella aurantialba by any means. Guess that excludes me from DBT's

[Phil G]

your contradicting urself.

on one hand ur saying that DBT isn't the be all and end all because we cant remember how something sounds.

but then u go and say that sighted tests are somehow different in that u can remember how something sounds?

Excellent question!

I think my answer is because there might be OTHER factors at play in a DBT. They could include, looking like a fool if I don't pick the expensive one - this is pressure which is not present in normal listenning.

Is the person in the right frame of mind to listen? To me this has a huge bearing on the enjoyment level able to be reached.

My point was that many comparrisons are pretty obvious. Repeatable and obvious.

[Dr X]

Excellent question!

I think my answer is because there might be OTHER factors at play in a DBT. They could include, looking like a fool if I don't pick the expensive one - this is pressure which is not present in normal listenning.

Is the person in the right frame of mind to listen? To me this has a huge bearing on the enjoyment level able to be reached.

My point was that many comparrisons are pretty obvious. Repeatable and obvious.

If you umm.......DONT PASS the test, it doesn't mean that you can't have another go at some later date. Maybe then you'll get over your fears and have had more time to "ear learn" your material.

[SCorpion]

Excellent question!

I think my answer is because there might be OTHER factors at play in a DBT. They could include, looking like a fool if I don't pick the expensive one - this is pressure which is not present in normal listenning.

Is the person in the right frame of mind to listen? To me this has a huge bearing on the enjoyment level able to be reached.

My point was that many comparrisons are pretty obvious. Repeatable and obvious.

i still cant see how there is any difference between the DBT and sighted tests tho with those points? they are both exactly the same. the only difference i can see is that the person cant just choose the expensive one because it is expensive. he has to choose it because of what it sounds like. he still might have that pressure on him to choose the expensive one, but i cant see how its relevant to the test.

My point was that many comparrisons are pretty obvious. Repeatable and obvious

but the question is why are they obvious. is it because they sound different, or is it because the price tags are different?

thats the whole point of a DBT

[Phil G]

If you umm.......DONT PASS the errr......TRIAL, it doesn't mean that you can't have another go at some later date. Maybe then you'll get over your fears and have had more time to "ear learn" your material.

I either hear a difference or I don't. The difference, if heard, will either make me want to buy or it wont. Couldn't give a hoot if the components being compared have a 'Levinson' or 'Wintal' badge on them.

[Dr X]

I either hear a difference or I don't. The difference, if heard, will either make me want to buy or it wont. Couldn't give a hoot if the components being compared have a 'Levinson' or 'Wintal' badge on them.

I wasn't trying to make fun of you LD. I was trying to be funny. :eek: Someone previously said they didn't like the word "failing".

[Tony M]

Not at all. Do a search for the posts made by our most well known double blind testing proponent and see how many different threads you come up with.

I am not a Tremella aurantialba by any means. Guess that excludes me from DBT's :eek:

Well, Keith, I must admit I had to google that.......and the initial info re a jellyfish didn't help much, until I noticed the common name of that critter.

Anyway, I'm convinced you're definitely more of a TA than I am and you're probably capable of listening more objectively than most

Perhaps the concept of objective testing is more attractive to those of us who are less skilled......I don't know:confused:

Cheers

Tony

[Dr X]

I either hear a difference or I don't. The difference, if heard, will either make me want to buy or it wont. Couldn't give a hoot if the components being compared have a 'Levinson' or 'Wintal' badge on them.

The problem is all the factors at play. What if one CD player has 3V output compared to another which only has 2V. The one with 3V should sound louder. And yes I admit that I'm speculating, that those kind of things could cloud your judgment.

[Phil G]

i still cant see how there is any difference between the DBT and sighted tests tho with those points? they are both exactly the same. the only difference i can see is that the person cant just choose the expensive one because it is expensive. he has to choose it because of what it sounds like. he still might have that pressure on him to choose the expensive one, but i cant see how its relevant to the test.

Fair call. How many DBT's have you been involved in. I suspect the pro DBTers would not have done many. Do you do them in the audio shops you visit? Do you do them with your car audio?

but the question is why are they obvious. is it because they sound different, or is it because the price tags are different?

thats the whole point of a DBT

They are obvious because the differences stand out. They are not the same!

[Drizt]

This is going to sound bad, but its only an observation.......

Has anyone ever noticed that the ones that are most heavily against Blind testing are those that have already spent a lot of money on their hifi toys?

[SCorpion]

Fair call. How many DBT's have you been involved in. I suspect the pro DBTers would not have done many. Do you do them in the audio shops you visit? Do you do them with your car audio?

i've probably done 10 or 15 and that was all soley about source material. ie, differences in mp3, and at what bitrate etc. not many obviously.

why do u suspect that the pro DBTers havent done many?

They are obvious because the differences stand out. They are not the same!

but the point of a DBT is to determine if those differences are actual audible differences or if the differences were something else like the 'placebo' effect.

[Drizt]

They are obvious because the differences stand out. They are not the same!

LuckyDog, could I make a suggestion. Perhaps put yourself through a couple proper blind tests (read level matched etc... with the help of a friend) to see if these obvious differences are detectable.

The true benefit of blind testing is generally not realised until you have gone through the process yourself.

I've done lots of blind testing... cables, CDP's, DAC's, Integrated amps etc. Some of those 'obvious' differences i thought I heard under sighted tests were simply not there under proper test conditions.

[Dr X]

but the point of a DBT is to determine if those differences are actual audible differences or if the differences were something else like the 'placebo' effect.

Not quite, it's just to allow someone the chance to prove what they claim, that's it.

If they fail you have NO proof they are:

(1) Lying

(2) Sufferring from the placebo effect

(3) Crazy

(4) Everything else in the known universe

[Drizt]

Not quite, it's just to allow someone the chance to prove what they claim, that's it.

If they fail you have NO proof they are:

(1) Lying

(2) Sufferring from the placebo effect

(3) Crazy

(4) Everything else in the known universe

Thats exactly right. Been saying that all along :eek:

You can only PROVE that there is a difference. And if there are 'Obvious' differences detected in sighted tests then the person should just as easily detect these 'Obvious' differences under proper blind testing conditions.

[SCorpion]

Not quite, it's just to allow someone the chance to prove what they claim, that's it.

If they fail you have NO proof they are:

(1) Lying

(2) Sufferring from the placebo effect

(3) Crazy

(4) Everything else in the known universe

no its not. the DBT is more specific in that. in fact, it tests whether or not there are audible differences in the equipment to that person. it doesn't prove what they claim at all.

the evidence can be used to suggest a hypothesis. a common hypothesis is that there is a placebo effect. the reason they suggest it might be a placebo effect is because there was an audible difference in a sighted test, then there wasn't in a DBT.

there could be other hypothesis. that is just one where the information used from a number of DBT's could be used.

[Dr X]

no its not. the DBT is more specific in that. in fact, it tests whether or not there are audible differences in the equipment to that person. it doesn't prove what they claim at all.

the evidence can be used to suggest a hypothesis. a common hypothesis is that there is a placebo effect. the reason they suggest it might be a placebo effect is because there was an audible difference in a sighted test, then there wasn't in a DBT.

there could be other hypothesis. that is just one where the information used from a number of DBT's could be used.

What you are talking about is trying to prove a NULL hypotheses. Eg 320kps MP3 = CD. This is trying to prove that 320kps is completely transparent IE you are trying to prove non existence. NO-ONE can prove non-existence mathematically as far as I know. You cannot prove superman doesn't exist or pigs don't fly. When testing a NULL hypothesis, you are collecting statistics, not coming up with actual proof. How one interprets those statistics varies. Those statistics can be very important though, especially in the field of medicine for example.

An example of a hypothesis not a NULL hypothesis is, 320kps MP3 <> CD.

<> means not equal to, there is some actual difference.

Cheers.

[Aslan]

I'm not even going to read all the posts in this thread.

My views are thus...................................

1/ There is merit in double blind testing

2/ Would I buy something based purely on a double blind test..............NO

3/ Am I heartily sick of having every topic for discussion on this board polluted with a double blind test post.........YES, YES and YES again.

Most of you will have noted that I am contributing less and less lately.............sadly, my patience for non productive argument and double blind test pollution is the cause of this. :eek:

Andrew

[Dr X]

This is going to sound bad, but its only an observation.......

Has anyone ever noticed that the ones that are most heavily against Blind testing are those that have already spent a lot of money on their hifi toys?

Or those that are trying to sell you something....Ouch! :eek:

[Drizt]

Aslan, I feel lots of people will feel the same as you. God knows Keith does.

A thread like this is a fantastic idea as it brings the topic out into the open for proper discussion. It also lets some people vent their frustrations.

My fear is that Blind testing will be pushed out of the forum as it has on some other terrible forums.

You are right that every thread should not get hijacked. Apologies for some drunken posts last night.

My other fear is that the opposite is sooooo much worse. I get sick of reading things like...... "You are wasting you time with expensive speakers if you don't spend mega bux on your front end". Or "this $2000 power cable raised my system to a completely new level". For any newbies trying to learn their own way through the hifi jungle they may take these things as gospel.

[Dr X]

I'm not even going to read all the posts in this thread.

My views are thus...................................

1/ There is merit in double blind testing

2/ Would I buy something based purely on a double blind test..............NO

3/ Am I heartily sick of having every topic for discussion on this board polluted with a double blind test post.........YES, YES and YES again.

Most of you will have noted that I am contributing less and less lately.............sadly, my patience for non productive argument and double blind test pollution is the cause of this. :eek:

Andrew

I'll repeat what some people have said to me here, if you don't like....Don't read it!

[Keith_W]

Most of you will have noted that I am contributing less and less lately.............sadly, my patience for non productive argument and double blind test pollution is the cause of this. :eek:

I think a few of us share your exasperation Andrew.

[Phil G]

This is going to sound bad, but its only an observation.......

Has anyone ever noticed that the ones that are most heavily against Blind testing are those that have already spent a lot of money on their hifi toys?

Now that is an insult, as clearly that is directed at me, as I assume you are suggesting that I waste money for what, bragging rights!!

[Phil G]

My other fear is that the opposite is sooooo much worse. I get sick of reading things like...... "You are wasting you time with expensive speakers if you don't spend mega bux on your front end". Or "this $2000 power cable raised my system to a completely new level". For any newbies trying to learn their own way through the hifi jungle they may take these things as gospel.

Come on Aaron, that is just crap. Exagerating a bit I think. So you now begrudge people's enthusiasm for a newly found upgrade. Why not share their joy. Go back and read some of your highly enthused posts over upgrades you have done. You're no different to the rest of us.