Lotus

Vandersteens were in our tests. The Quatro for instance came from the home of one of our techs on the project.

Lotus

Unfortunately we aren't made out of money. We do have an old SoundLab A-1 in the tests (and yes it does well). Many of the esoteric brands couldn't lend us units or the stores in the areas that carry them wouldn't lend us demos. We had to be a lot more picky about esoteric brands. Avalon Acoustics for instance only got 1 pair of speakers in the tests because a friend owned them and was willing to part with them (he upgraded to a pair 1 step up in their lineup).

Lotus

We fully expect knee jerk responses like this. That is why we're still a year from finishing our tests. One conclusion we have made is that cables DON'T make a difference if you select a good cable. We have had leanings towards one cable brand for some weird reason but other than one brand, the rest seem identical (ok ARE identical) as long as we're using 12 AWG with solid insulation we're ok.

As for your hearing being different than someone in our test group: Sure! You may have bad hearing. We have tested the hearing of our group. They all tested above average or better for their age & sex.

As for Pink Floyd being a bad selection: we went with something we knew was extremely well recorded. You may disagree, but that is why we have other tracks.

As for Amps, CD Players, etc. having better synergy we already considered that. So we started testing with different combinations after we eliminated about 40% of the speakers. It could be we eliminated a speaker that may have sounded better, but we're pretty sure we didn't do that. If you want I'll add the Bose 301 back into the test groups for instance.

So far the Magnepan MMG has been tested with 44 different amps, processors, and CD players. The results range in a 4% swing in one direction and a 7% swing in the other (being more preferred). Planar speakers seem to do better with Tubes for instance.

As for Rooms, our rooms are all identical. We do however have different room sizes. So yes, speakers are tested in different sizes (small, medium, large). Some do better than others.

As for us not considering some brand you'd consider: All major brands were present for the testing. The vast majority of whose who in hi-fi made it into our testing. Some esoteric brands we took some short cuts with. As mentioned in other replies we went with an older SoundLab unit for instance the A-1 (still tests well), and some other units were donated to us.

As for these tests not proving anything, that is exactly what we expect to be screamed from rooftops of just about every major audio manufacturer. They don't want people to know that their $30,000 amp was bested by an $800 amp in the majority of tests. They don't want people to know that their $7500 pair of speakers were bested by a $550 pair of speakers. The cable companies definitely don't want us to release our data. We've proven that there is a difference between cables, but at the same time we've proven there is no reason to ever spend more than 3 figures on cables. All expensive cables were indistinguishable from good less expensive cable.

By running tests on speakers using multiple rooms, in a controlled blind environment with multiple amps, processors, and cd players we can show human preference towards certain speakers. You may not like that we've done it, but the facts will be that scientifically we've covered just about every base.

Obviously these tests prove other things. Some people like the sound of cones over planars. Obviously anyone who reads our tests could fall in the same category. So obviously people will still need to go out and hear for themselves.

Still I'm positive that if anyone goes and demos any of our top 10 rated items in the pricing category they are looking at they should find a good unit without being fooled by salesmen.

Without a doubt, price, salesmen and other psychological factors seem to play a bigger factor than anything else.

kennyt

I have top raise issue with this post, Lotus, if you truly are doing this study, as you say you are then God bless you, but I think it’s all a bunch of BS, and here’s why.

You say 2000 test subjects all had their hearing tested and were found to be above average? WOW! That in it self is a statement I find impossible to fathom, I assume this was a typo, so let’s move on.

Four demo tracks, at say 3 minutes each makes EACH demo 12 minutes (likely more). Time to change speakers and move every one in and out of room? I’d venture a guess of 5 minutes, likely a more even when maximally optimized. So we are up to 17+ minutes per speaker MINIMUM. That would mean for the group to test 40 speakers per day it would take 11 1/3 hours per day of nearly tireless listening to each hear 40 different speakers.

Now if you ‘literally tested hundreds of speaker’, even if we make that number 200, that means EACH listener is now up to 56 2/3 hours in this study. That doesn’t count the redundancy designed into this test of replacing the top speaker from the lower price category into the higher price point, which will add more speakers, so AT MINIMUM that makes it go to say 225 speakers now each listener is up to 63.6 hours involved in this study. Now, if the study was truly done with the 2,000 people listed above, then we can make this test take a minimum of 3,178 hours or 79 full 40 hour work weeks, or 1.5 years with no vacation and perfect execution.

Holy poop! I have run medical studies and know the cost, I can’t fathom how much you had to pay test subjects you get them to take the required weeks off work to do nothing but listen and rate speakers full time! Oh, and for all the above calculations I didn’t include the survey time, which will add several minutes or more to EACH listening session, and no accounting for data crunching as I assume the staff, of which I have not even begun to attempt to calculate required to perform and organize this, especially with such criteria as never having two groups listen to the same set of speakers per session at the same time? WOW! The logistics of this study are staggering!

Lotus, let’s face it, I think this is total BS, if you truly are doing this, then good for you, but I would like to know where the $$ came from to do it as the required funds would be HUGE!!!! Lets say even if you got every speaker lent to you, if they were new, and I am sure every manufacturer DIDN’T lend you all these speakers, (especially as I know how Magnepan is on loaners!) they’d need to be broken in, adding hundreds of hours per speaker to the study, each speaker needs to be maximized for the rooms, something I only know of one lab in the country set up to swap speakers blindly, quickly and to consistently place them in the correct spot, and this lab in Revel’s test center and cost millions to construct.

To all the readers of this forum, please see this and tell me if you can see my logic and why I am sure Lotus is just blowing smoke up our butts to pump Magnepan’s and electrostats in general. Lotus, I am calling you a fraud as by any stretch of logic your test is physically impossible and even if it were, there would never be funding for such a test. Print magazines are dying, so they certainly wouldn’t fund this endeavor. High end manufacturers are also suffering in the current economy, so they wouldn’t pony up the Millions needed to do this, and if you expect me to believe you did this on your own dime, them it is nice to meet you Sergey Brin, or are you Larry Page because no one else has the capital to throw at a study like this that will never generate income.

Feel free to attack me and my logic all you want, but you and all the readers should easily be able to see that the time and effort required to do such a study is monumental, so I am sure it’s not happening. Thanks for the comical reading though, I enjoyed reading the setup of the study!!! It cracked me up when I read it, but I must point out the obvious, your FOS!

KT

__________________
Ken Taraszka, MD
Associate Editor
http://avrev.com/
http://hometheaterreview.com/
http://www.avrevforum.com/

JerryDelColliano

Lotus,

Kenny brings up some good points/questions above.

Here are some of mine...

1. Are these tests being done blind? Any speaker evaluations must be done that way so that the look of the speaker doesn't effect the results. I have done tests at Harman and Paradigm and they are ALWAYS blind. A MartinLogan has a visual advantage in my opinion over say a Gallo and or a Vandersteen as the later are some ugly ass speakers in comparison if you are just looking at thier exteriors.

2. How could you demo each track for 3 minutes. That is SO LONG. Its a demo. Normally, people tune out of a demo after the first verse which is rarely past 1 min in the song.

3. Who gave you the speakers?

4. What order are the speakers played in?

5. With over 2000 demos in your test - who is paying for the hearing tests? Who paid for the speakers?

I am not trying to bust balls here. It just seems like there are some serious flaws to the science of this test. I know Harman does something similar at their test facilities here in California but a LOT of the variables are taken out of play and they are blind tests.

__________________
---
Jerry Del Colliano
Publisher

http://90210SEO.com/
http://thebusinessofno.blogspot.com/

Lotus

We're doing the study. It has been my full time job for a while now.

Not a typo. You find it impossible to fathom because you're reaching for a conclusion without thinking it through. The test subjects used to test the products number 2,000 who have above average to good hearing for their age/sex. You assume we only found 2,000 people and they miraculously all had above average hearing! No, we tested a lot of people and came up with those 2,000.

Not exactly. It takes approximately 15 minutes between rooms (2 sets of speakers). We were able to push through those 40 speakers in about 7 hours with lunch breaks. You make the assumption that we're playing whole tracks on the days when we're moving through volume. We don't. For the rating process we only go through 10 "items," a day. For the preference tests we move rather quickly. For the psyche tests it takes much longer.

We get test subjects for 1 day every week. The test was started before I was brought in. Testing started in 2005. We don't test 2,000 people all at once obviously. We rotate test subjects as well as tests. We aren't close to 2,000 people in certain tests. We're at least a year from some things being finished, and a year and a half from being able to release data on it all.

Some are paid more than others. Some volunteered entirely. We haven't even begun to crunch all the data. This is my retirement work. I'm being paid practically nothing.

As for the rest of your tirade it isn't worth responding to. This is being done with a major University, Government Grants, and donations. We have already gone through millions and we'll likely add more before we're done. It isn't cheap. Getting gear was mainly done through distribution channels/used/etc.

Breaking your own forum policy however was imho uncalled for. You could have PMed me and received phone numbers to confirm all the testing going on. Instead you attack me. How very nice of you.