I was on a call recently working through the details of a complex discrete-choice task. Specifically we were debating how best to apply price prohibitions - restrictions on the design that would prevent certain "monthly" and "one-time" prices from ever appearing together.
Rest assured our thinking was all very logical. We needed to put controls in place because who in their right mind would ever choose an option where a low monthly rate, coupled with a contract, quickly added up to more out of pocket costs than would accrue by skipping the contract and paying a (slightly) higher up-front fee. That's when the client's client chimed in:
"People don't do the math," said the man who spends little to none of his time conducting surveys.And of course he was right.
We’re all consumers, and so we’ve known for some time now (perhaps forever?), that we don’t always add things up and make the logical choice. Those of us stubborn enough to ignore their intuitions can thank our friends in Behavioral Economics for demonstrating this reality experimentally.
And yet… on that call and in the heat of the moment… the quantitative researcher’s instinct to control things cropped up. Now it’s useful and often necessary to control things in our studies, and contemporary survey software packages make such controls almost effortless. But filters and other restrictions can bring false comfort, and can lead researchers to act more like trial attorneys – never asking any questions they don’t already know the answers to.
The client’s client wasn’t thinking like a researcher. He was thinking like someone who sells things to people. And he had enough perspective to know that companies can (and often do) make their whole year off of market inefficiencies and illogical consumer behavior. Noise can be noise, but it can sometimes also be valuable information too easily dismissed as noise.
So the prohibitions came off the design (making for a better and more balanced overall choice task), and our clients are looking forward to examining consumer choices through a wider lens.
Myself, I came away with another lesson in being a thoughtful researcher, and a renewed determination to invite the illogical into my work sometimes. Just don't tell my lawyer.