Double Blind Tests- NOthing to see hear!!

[Guest Muon]

Wow.   Thanks for wasting my time.

I'm attempting to avoid wasting my own time

[betty boop]

One instance doesn't  "prove"  anything.    Otherwise I could conduct a running race with people of all different heights, have a short guy win, and conclude that short people run better.

 

The apple falling from the tree didn't "prove" gravity....   but when the 100th apple fell, and all the other objects tested fell, and different people repeated the same experiment over and over in different situations ....people decide there might be something going on.... and people began to use the prediction that objects fall to their advantage.    Of course, objects don't always fall, so we have to be careful not to be to blinkered.....   but a clear understanding of how they fall, why they fall, and when we would expect them to fall - is naturally better than dogmatism.

 

It isn't controversial to state that there can be information presented to you which impairs your ability to assess or perceive other information.    The paper quoted is just another example.

 

 

 

Only in how it relates to speaker design.

 

if you want to use humans for testing...you need to understand how the human mind works. Its explained well in that doco by someone whom won a nobel peace prize on the topic ! and its something has widespread application in all industry's across a whole range of products, services etc.  but really upto you whether of interest or not. not for me to suggest otherwise.

[davewantsmoore]

if you want to use humans for testing...you need to understand how the human mind works

 

Indeed.   (Yes, I've seen the documentary)

[Guest myrantz]

Unlike many of the blind-test threads that have gone before, I find the discussion on the correlation between blind auditioning of live artists and blind evaluation of audio components quite interesting. The aim of both is purely to remove a particular factor from consideration in the evaluation: In the case of the audition, removing the gender of the musician as a consideration thus enabling better focus on the artistic interpretation and performance; In the case of audio components, removing the price/value of the component from consideration thus enabling better focus on the sonic performance.

 

In both cases, I can see the merits of the blind test. In both cases, a subjective evaluation will still take place based on other personal biases: artistic style preferences in the case of an audition of a performer; and sound presentation preferences (forwardness, warmth, neutrality, etc) in the case of a component evaluation.

 

So yes, I can see a correlation as it specifically relates to enabling a more focused (but still subjective) evaluation on either artistic performance (artist audition) or sonic merits (component evaluation).

Don't make the same mistake the OP did.

As I said, the difference in the research is to establish if it's easier for females to get to the second (or later) interview and ultimately a job. There is no test to qualify the judge's decision, they only look at the outcome (the number of females in the orchestra).

 

Put it another way, the research did not look at a particular hiring and determine the person chosen (be it male or female) is actually the best performer in that group.

 

The research only looked at the names of people who go for the interview (they have to get a korean to look at some of the asian names, and for the most part Asian names are gender neutral) and compare it to the people that are hired. And there are now a lot of Asians in orchestras  (because it's in Asian culture to learn music as kids as it seems to improve the skills in maths and science). Also, back in the 70s, female Asians are not encouraged to work. 

 

I'm not calling that research in question, it's not bad. But the OP associate that research and tie it too loosely to audiophile BT, by saying things like this

 

What's the difference between selecting an orchestra member and selecting speakers or cables?

 

In audiophile, you have to qualify the decision you make, i.e. that over 20 trials, you pick A 19 times, as such it pass statistical analysis that A is different from B. That did not happen in this research (see bold bits again). If he can't understand the difference,  it just shows he's lack of understanding in basic research. Linking that to audiophile BT is pseudo science..

 

The internet is full of people who use science to justify their ill thought out arguments.. These people are creating more myths than correcting 'em...

Edited September 8, 2015 by myrantz

[davewantsmoore]

I'm attempting to avoid wasting my own time

 

If you think explaining your comment to people who show genuine interest in why you think the way you do, is a waste of your time .... then why make the comment?

 

I want you to teach me something.... but you can't be bothered.   Thanks.

 

 

Troll much?!?

[davewantsmoore]

I'm not calling that research in question, it's not bad. But the OP associate that research and tie it too loosely to audiophile BT, by saying things like this

 

"What's the difference between selecting an orchestra member and selecting speakers or cables?"

 

It's a question, not a statement.    He's inviting you to explain what is different about the two, in the context of the conclusions drawn in the paper.

 

 

Put it another way, the research did not look at a particular hiring and determine the person chosen (be it male or female) is actually the best performer in that group.

 

Can you explain why this matters?

 

All the article says is that:     Research shows that we apply different standards when we compare men and women.

 

The article doesn't say that if we put a screen up we will automatically choose the best.   It just says that we might choose differently to no screen being up.

 

 

In audiophile, you have to qualify the decision you make, i.e. that over 20 trials, you pick A 19 times, as such it pass statistical analysis that A is different from B. 

 

 You don't HAVE to do anything.....   You can do exactly as the orchestra people did if you want.      I tested A and B (amplifier) knowing which was which.    I tested A and B not knowing which was which.   I chose differently.

 

If I've missed something, please explain further.

[Guest Muon]

If you think explaining your comment to people who show genuine interest in why you think the way you do, is a waste of your time .... then why make the comment?

 

I want you to teach me something.... but you can't be bothered.   Thanks.

 

 

Troll much?!?

If anyone can't understand what I have posted, that's not my problem.

[davewantsmoore]

If anyone can't understand what I have posted, that's not my problem.

 

What are you 12?

 

I wanted to know the reasoning (which you didn't give - or at least I didn't see) for your statement.

[Guest Muon]

What are you 12?

 

I wanted to know the reasoning (which you didn't give - or at least I didn't see) for your statement.

You can try and insult me as much as you like.

 

Go for it, get really personal if you like....it will not have any effect on myself.

[rocky500]

What I find interesting is a blind test done really well in Audio always seems to have the same results. There is no difference from the paticipating peoples choices and it looks like most are just guessing.

 

I prefer longer listening tests. Maybe I am more relaxed and a different part of the brain is used when you are not critically listening, I just don't know but I find they don't really help me to decide.

Edited September 8, 2015 by rocky500

[betty boop]

What I find interesting is a blind test done really well in Audio always seems to have the same results. There is no difference from the paticipating peoples choices and it looks like most are just guessing.

 

I prefer longer listening tests. Maybe I am more relaxed and a different part of the brain is used when you are not critically listening, I just don't know but I find they don't really help me to decide.

 

this is where system 1 and system 2 thinking comes into play

 

https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow

 

"System 1" is fast, instinctive and emotional; "System 2" is slower, more deliberative, and more logical.

 

[Newman]

@@rocky500, I have seen plenty of DBT test writups in audio that demonstrate clear preferences for this over that.

I can't imagine why you say that they all have "the same conclusion".

Edited September 8, 2015 by Newman

[rocky500]

I have seen plenty of DBT test writups in audio that demonstrate clear preferences for this over that.

I can't imagine why you say that they all have "the same conclusion".

 

Can you show me one that is done well and has what you say. I have not seen them on the net.

Edited September 8, 2015 by rocky500

[Guest myrantz]

It's a question, not a statement.    He's inviting you to explain what is different about the two, in the context of the conclusions drawn in the paper.

Sure.. Spin it that way .. He will deny it's a rhetorical question like you are doing now too..

 

 

Can you explain why this matters?

I thought I already did? How do you even see any similarities between a blind audition for a job (where somebody just gets a job) to a blind test (saying A is different to ?

It is an assumption to jump to the conclusion that the picked candidate is the best person for the orchestra, because the research only looked at interview names vs the roster.

 

The article doesn't say that if we put a screen up we will automatically choose the best.   It just says that we might choose differently to no screen being up.

The article said a screen up is effective in removing gender bias.. Everything else you try to tie to audiophile blind test is just pseudo science...

[Guest Hensa]

Don't make the same mistake the OP did.

 

 

No mistake made on my part, though I'm not sure you have understood my point. I have at no stage said that blind testing for an orchestra performer is the same as for an audio component. What I actually said was that I can see a "correlation" between the two from the perspective of blind testing playing a part in reducing the influence of a specific pre-existing bias. I did not say that blind testing removes all biases. I also clearly stated that the outcome would still be a subjective one meaning that it will still be subject to the listener's other personal biases.

[Newman]

Can you show me one that is done well and has what you say. I have not seen them on the net.

Start from page 10, then dig into the references, all of which will be DBT-based non-null test results proving something interesting and educational about audio and/or human perception of sound and/or music. Everything in the linked article will have been derived from DBT testing that yielded clear preferences for this over that.

 

There's a whole legacy of this stuff. Please don't suggest that properly-done DBT testing inherently 'always produces the same result', and 'participants are just guessing'.

[Guest myrantz]

No mistake made on my part, though I'm not sure you have understood my point. I have at no stage said that blind testing for an orchestra performer is the same as for an audio component. What I actually said was that I can see a "correlation" between the two from the perspective of blind testing playing a part in reducing the influence of a specific pre-existing bias. I did not say that blind testing removes all biases. I also clearly stated that the outcome would still be a subjective one meaning that it will still be subject to the listener's other personal biases.

Effectively what you're saying is a blind audition in orchestra is effective in eliminating gender bias, and as such a blind test in audiophile is effective eliminating some pre-existing bias in audiophile  too (e.g. look of the equipment, price).. That's the mistake you (and the OP) have made - it's what Lisa Simpson call specious reasoning.

Note I specifically did not even use the  "correlation does not imply causation" unlike other blind tests threads.. because, there is absolutely no correlation between the blind audition with audiophile blind tests... None whatsoever... They are so different from one another the thing that is common between the two is the word "blind".

[rantan]

Well, well.

 

Predictable is as predictable does. Who would have thought this thread would result in the usual entrenched positions?

 

Shocked I am

[Guest myrantz]

Predictable is as predictable does. Who would have thought this thread would result in the usual entrenched positions?

I havn't even got to my usual entrenched position yet .. And ditto the fordgetlover, dave and newman.. They're still trying to associate that blind audition to audiophile blind testing... It's like in their heads they're thinking all blind tests are good, this is a blind test with some results, so why aren't audiophile using blind tests??? 

 

The article is so clear in stating eliminating gender bias, and yet the thread title only says eliminating bias. The first post is pretty misleading as well - saying it's successful in eliminating gender bias, and it works, but many audiophiles are still feel blind testing has no place in blah blah blah.. Classic misdirection?

 

Have people learnt nothing from the botched test Ars Technica did?

 

Truth be told I don't engage in these pointless threads anymore unless they are in the great audio debate (It wasn't here in this section before). I've made an exception in posting to this thread when it's outside the great debate section as the argument they attempted to create is so misconstrued and misguided..

[rocky500]

Start from page 10, then dig into the references, all of which will be DBT-based non-null test results proving something interesting and educational about audio and/or human perception of sound and/or music. Everything in the linked article will have been derived from DBT testing that yielded clear preferences for this over that.

 

There's a whole legacy of this stuff. Please don't suggest that properly-done DBT testing inherently 'always produces the same result', and 'participants are just guessing'.

 

Thanks Newman but you sent me a link for speaker tests.

This is the one component that is quite easy to hear a difference as it goes without saying. The old saying - speakers make the biggest difference. So of coarse there will be a difference in any test.

When you start to get into Dacs, sources etc then there is what I said for me.

Edited September 8, 2015 by rocky500

[Satanica]

Well, well.

 

Predictable is as predictable does. Who would have thought this thread would result in the usual entrenched positions?

 

Shocked I am

 

Another very predictable post

[Guest Hensa]

Effectively what you're saying is a blind audition in orchestra is effective in eliminating gender bias, and as such a blind test in audiophile is effective eliminating some pre-existing bias in audiophile  too (e.g. look of the equipment, price).. That's the mistake you (and the OP) have made - it's what Lisa Simpson call specious reasoning.

Note I specifically did not even use the  "correlation does not imply causation" unlike other blind tests threads.. because, there is absolutely no correlation between the blind audition with audiophile blind tests... None whatsoever... They are so different from one another the thing that is common between the two is the word "blind".

 

To re-quote my own words (with some additional bolding and underlining) as I find your interpretation of them grossly misleading:

 

"I can see a "correlation" between the two from the perspective of blind testing playing a part in reducing the influence of a specific pre-existing bias."

 

My notion is based on this simple reasoning: If the 'recruiter' is unaware of the gender of the person auditioning, gender should no longer be an influencing factor. If the person comparing audio equipment (I'm not talking about a double-blind testing process here) is unaware of the price of the equipment, this should no longer be an influencing factor.

 

I did not say that a blind audition is "effective in eliminating gender bias, and AS SUCH a blind test in audiophile is effective eliminating some pre-existing bias in audiophile too..." That would be an utterly absurd statement and one I'd never make so please try and not misrepresent my comment.

[rantan]

Another very predictable post

 

Why so.?

 

Tell me exactly how this DBT debate differs from any other, except for the camouflage of the orchestral selection and its use as a vehicle for the usual debate

 

Sincerely, we know what the various protagonists think on this topic, so why do we need to re visit this endlessly?

[TP1]

 Please don't suggest that properly-done DBT testing inherently 'always produces the same result', and 'participants are just guessing'.

 

It is the interpretation of  "properly done " DBT   where the issues usually come from.    

[Zaphod Beeblebrox]

I had 2 amps here and thought one sounded a lot better. I always seem to get swept away with the music in different listening sessions with this one amp.

I would then swap between them and could not pick a difference in switching them quickly.

 

So used the cheaper amp and my music lost its appeal when I listen to it over the days following. I did not enjoy listening to it.

Do the tests again switching between amps and I can't hear a difference. They sound the same in these tests!

 

Put the better amp back and my listening sessions are so much more relaxed and really enjoyable. Look over to check which amp I am actually listening too and those sessions that are relaxing and enjoyable is the more expensive amp.

Tried this many times over a period as I really wanted to sell the more expensive amp as they sound the same when switching quickly between them!

 

Maybe I am just bad at short listening tests as I have been to others when they can pick a difference and I can't.

 

None of which precludes the use of blind testing. Time is not a function of such tests.