For the past few weeks, there are two big debates raging in our office:
On the surface the only thing connecting them is that we are a choice focused market research company located just outside of Philly, but in reality they are both the same debates...namely, when is it time for the superstar to move on and allow a new star to take charge? In both cases, the answer will depend on your needs and your perspective...in other words, there is no answer that everyone will agree with.
Some variation of conjoint (discrete choice, adaptive, etc) has been with us now for nearly 40 years. It has proven to be a very effective means of understanding the consumer's thinking process...especially when it comes to developing new products. At the same time, it is not without its flaws.
First, the whole conjoint process is somewhat mysterious to anyone who doesn't like to use formulas that have Greek letters in them. I recently was telling my wife about an innovation we made to TEXO, our configurator product and to make the benefits over conjoint clear, I had to explain how conjoint worked. Like many, she was skeptical about the reliability of data that required such a mind-numbing process to collect. Combined with the analytics required to produce simulated results it isn't hard to see why some have a very hard time trusting conjoint.
Personally I've seen enough successful conjoint work to know that if it is properly applied, conjoint is a very reliable method. That doesn't mean this mysteriousness factor isn't of concern. The fact is that results are only useful if they are embraced and applied. Conjoint results are only embraced by those who understand the analytics or who take them on faith. Other than showing past success it is hard to prove to the non-analytical person that it works.
Second, conjoint assumes that consumers make decisions by looking at the entire product and then choosing the best. When buying a complex product like a car most of us will rule out certain makes, models and features. I won't buy a two door car (my kids have never been exposed to the acrobatics of climbing behind a seat and I doubt the teen years is a good time to start) and at 6'7'' tall I have a whole host of smaller cars that are simply not in my decision set. You might concede this but counter that some products are simple enough that most buyers consider the entire product when making their choices. I disagree. Even something as simple as say chewing gum likely has quite a lot of non-compensatory features to it (personally I don't consider fruity gums or bubble gums when making a choice).
Because most conjoint techniques don't consider this, many of the choices respondents are asked to make are inherently flawed. Even worse, if simulators don't take this into account the results generated can be exaggerated. Tools like our "screen out tool" help to eliminate this problem, but as long as data are collected in the traditional conjoint way, nothing can fully compensate for it.
Third, even adaptive conjoint forces limitations on the survey designer that don't exist in the real world. This is particularly true when it comes to products that can be customized...an increasingly important issue as more and more products allow customers to create their own. Products that range from health insurance to cable/communications services often don't lend themselves to a conjoint design or force so many restrictions that the results bear no relationship with reality.
Configurators alleviate these issues. Results are easy to understand and can be used to support the simulator output in a way that conjoint results can't. Configurators don't assume people look at a product in full...instead , they build the product as they go. Finally, Configurators provide massive flexibility to ask about even the most complex of products. So, it would seem like an easy decision to push conjoint aside.
As with the McNabb trade, however, the decision isn't that easy. While advancements have eliminated many weaknesses of Configurators, there are still places where conjoint either does or is perceived to do a better job. For example, if your stakeholders are strong believers in conjoint, it might be hard to get them to embrace configurators and as noted above, results that are not embraced are of little value.
One of the biggest challengers with configurators is likely to never go away. While conjoint may require pricing of the various attribute bundles shown to respondents, it doesn't require that each level of each attribute be priced the way that configurator does. This can be a daunting taskt and it isn't always possible.
So for now, I am in the camp that says that conjoint is still the right tool for many situations. I've also argued that it was probably time for McNabb to go, though I wish he'd gone to the AFC. Of course, my opinions on both could well change by the time of the next Super Bowl.
P.S. You can actually learn more about our unique configurator product TeXo here.
Rich brings a passion for quantitative data and the use of choice to understand consumer behavior to his blog entries. His unique perspective has allowed him to muse on subjects as far afield as Dinosaurs and advanced technology with insight into what each can teach us about doing better research.