Behavioral Conjoint: Measuring Impact of Conscious and Subconscious Factors on Choice

Imagine that you are a researcher for a coffee brand and you would like to know which of two ad treatments would be more effective for your brand. The simplest way to test this would be to use a monadic experiment where the two treatments are shown to two different groups of people, and the relevant metrics for each cell are measured. But is this the best and most practical way to test the treatments?

Consider this complication:

In a monadic experiment, everything except the tested treatment is held constant. That is, in both cells the description of the coffee (its brand name, price, etc.) will need to be exactly the same, so that the impact of the background picture can be measured. Any change in the description of the coffee will nullify the experiment (or alternatively, require many more cells, and money and time, to test). But in reality, we know that the actual features of the coffee have a strong impact on choice, and that the background picture may have an additional impact. So neglecting the impact of the coffee features will exaggerate the impact of the background picture. How do we get around this problem?

The key is to think about the kinds of effects the various factors have on choice.

The features of the coffee are usually objectively defined and consumers are thought to evaluate them rationally and consciously. But other factors (such as the background picture) may have a subconscious and/or emotional impact on choice, which may also be smaller than the impact of the features. Research in the area of Behavioral Economics has explored the ways in which such environmental stimuli can impact choice, and perhaps lead to less than rational decisions on the part of consumers. So, how can we properly test if the background picture, in fact, has an impact on choice?

Combine the two streams of research in a single study.

Variations of features and their impact on choice can be best measured using conjoint analysis. Impact of subtle environmental factors on choice can best be understood through the prism of Behavioral Economics. Combining the two approaches, we can test if the background picture has an impact on choice, above and beyond the impact of features – an approach we call Behavioral Conjoint.

Behavioral Conjoint – How it Works

In a regular conjoint study, features of a product described on certain levels are combined to create variations, to which consumers respond. They respond to multiple screens of these products, often choosing between a few product variations, as in a discrete choice conjoint exercise. Complex statistical analysis on the back-end teases out the impact of each feature on the consumer’s choice. The nature of the exercise is such that the consumer is fully aware of every manipulation on the screen. So a conjoint study on coffee could manipulate brand name, type of roast, price, etc., and the respondent would see these variations on the screen, and choose the one she liked best based on a cost-benefit evaluation. There is really no room to test background pictures, other than to do it explicitly (in which case, there is no chance of testing the subconscious impact of the picture).

In contrast, experiments in Behavioral Economics focus on issues like the subconscious impact of the background picture, but get at it by using a monadic set-up. That is, respondents in one group see one picture and respondents in another group see another picture (or no picture) and the difference in take-rate is measured. But, there is no variation in the features of the product itself and hence, while we get at the subconscious impact, we neglect the conscious impact on choice.

In Behavioral Conjoint we combine the two approaches by overlaying two designs. Rather than conducting a single conjoint study (with one cell) we conduct a conjoint study with two cells that are identical in every respect, except for the background picture manipulation. That is, we overlay a monadic experiment on top of a conjoint study to exploit the advantages of both approaches. Jointly analyzing the data from both cells has some complexities and nuances, but can yield the exact information we are looking for. That is, it can tell us whether the background picture has an impact on choice above and beyond the impact of features. In effect, it is a way to measure the impact of subconscious features in their proper context – along with the features that have a conscious impact on choice, just as it works in real purchases.

The Coffee Example

In our study, we had four features for coffee – brand, roast, packaging and price. Two discrete choice conjoint cells were used, with one cell showing a plain text description of the coffee features and the other showing a colorful background picture. Obviously, the respondents in the study are not aware of the picture manipulation as they see no pictures at all, or see the same picture in every product (and it thus appears like scenery).

Once the data were collected, we combined the two cells and analyzed the conjoint data using Hierarchical Bayesian (HB) estimation to obtain individual-level utility scores. The most relevant question is whether we can show an effect for the background picture manipulation. Keep in mind that this effect has to show up above and beyond the impact of features on choice. That is, along with the utility scores for other features included in the conjoint design (such as brand, price, etc.) we needed to get a score for the treatment variable that is statistically significant. And indeed that is what we found.

But what does a statistically significant effect really mean? Practically speaking, we want to know if there is an impact on the number of people who would select a coffee when a background picture is included. To get at this, we first set up a baseline scenario in the conjoint simulator where we create two identical products (with both having a plain text background). They will have exactly the same shares as shown below.

Now we toggle the experimental variable so that one product has the background picture, while the other still has the plain text background. The results are shown below.

As is obvious, the background picture has a clear and strong impact on the number of people choosing the coffee. That is, compared to using a plain text background, a picture background increases share by 15 percentage points (which in this case is a 60% increase). This is quite a significant impact. It is very likely that the difference will not be so pronounced if the test is run between two background pictures, but we should expect to see enough of a difference to quantify the subconscious impact of one treatment over another.

Since price is included as one of the variables in the design, we can translate monetarily the impact of the inclusion of the picture as shown in the next table.

With coffee prices tested at approximately $7-$9 per pound, the background picture has an impact of about $0.64 on choice. Stated alternatively, the picture is worth a premium of about $0.64 compared to using just a plain text background.

Implications

The impact of the background picture on choice is subconscious and is measured above and beyond the impact of the features of the product. That is, we are measuring the impact of subtle environmental stimuli on choice, a hallmark of Behavioral Economics, using the tested robust approach of conjoint analysis. There is no other research approach that gets at both of these effects, and certainly not in a straightforward survey context, as we are doing here.

How else can the Behavioral Conjoint approach be used? In any choice situation where we suspect that both conscious and subconscious factors are in play, this approach will test and show the impact of each factor. Background (scenery, outdoors, indoors, etc.), faces (attractive, smiling, male, female, etc.), pictures, colors, fonts, music and anything else that is suspected to subconsciously affect choice can be cleanly tested with this approach.

Research has shown that pictures often tap into the emotional side of decision making (in contrast to words that play more to the rational side). Using the Behavioral Conjoint approach we can test the impact of the emotional factors, not just in isolation, but in addition to the impact of rational factors. And this can be done not with expensive and cumbersome approaches (such as fMRI studies), but in a simple, easy-to-use and robust survey context.