Welcome visitor you can log in or create an account

800.275.2827

Consumer Insights. Market Innovation.

blog-page
Recent blog posts

Shane Frederick (Associate Professor at Yale University’s School of Management) did a talk on Behavioral Economics at our recent research conference that got me thinking. But before we tap into the scary place that is my brain, let’s consider what behavioral economics is. Most of us with a formal business education have taken at least one if not several economics classes, during which we were exposed to market theories based on assumptions that sounded reasonable in principle but that really didn’t represent how things worked in real life. Behavioral economics, Shane started, is the study of economics when those assumptions are relaxed, and the relaxation of one of these assumptions, that people act rationally, is what got my attention.

One of the examples Shane used to make his point involved a pivotal point late in a 2009 football game between the New England Patriots and the Indianapolis Colts. Bill Belichick, the coach of the Patriots, decided to go for it on 4th and 2 deep in his own territory. The attempt failed, the Colts scored after the ensuring change of possession and won the game, and nearly everyone in the sports world pointed to Belichicks' seemingly insane decision.  But was it really insane? 

Like any research, market research has always recognized that to be certain results of research can be projected to an entire population; you need to eliminate any bias. We worried about things like:

  • Representativeness Effects – Needed to not only make sure we selected a random representative sample, but then do everything possible to maximize the percentage of people who completed the survey.
  • Interviewer Effects – Surveys needed to be done identically.   If one was done by mail, all should be with identical forms. If done by phone interviewers needed to be careful not to lead respondents and to keep pacing at consistent rate.
  • Framing Effects– If responses from one question are going to potentially bias a future response then the order should be changed to reflect it. In cases where changing the order merely changes which question biases which, use rotation or split samples so that bias effects can be measured and softened.

I know this is a simplified view of things, but the above three do get at the major forms of bias that we seek to eliminate in market research. In this blog, I'll focus on representativeness and at some point in the future I'll cover the other two.

what increases attention paid to adsAdvertisers and researchers do a lot of testing to determine how effective their advertising is prior to launching a campaign or message. We look for ways to get inside consumers’ heads, and as technology improves, we are afforded interesting glimpses into how consumers process information and make decisions. As my colleague Rajan pointed out in his blog different areas of the brain lead to different types of decision-making. Nobel Prize winner Daniel Kahneman posits that human thinking can be classified into two forms, System 1, which operates automatically, and System 2, which requires mental effort (I paraphrase). Jonah Lehrer, author of How We Decide asserts in his blog “Our best decisions are a finely tuned blend of both feeling and reason and the precise mix depends on the situation. When buying a house, for example, it’s best to let our unconscious mull over the many variables. But when we’re picking a stock, intuition often leads us astray. The trick is to determine when to use the different parts of the brain, and to do this, we need to think harder (and smarter) about how we think.”

With all of this exciting work being done in the field of neuroscience and behavioral economics, I wondered what kinds of answers we would get if we simply asked consumers directly what they think motivates them in considering advertising. Do they believe they respond to characters like the Geico gecko? Or is it really just a function of what they need at the time?

In Thinking, Fast & Slow, Nobel winner Daniel Kahneman (click here previous post about Thinking, Fast & Slow) talks about the two selves people have: the experiencing self and the remembering self. The terms are self-explanatory and vacations are a good way to think about them. The part of us that is enjoying the vacation is the experiencing self, while the part that is reliving it later (sometimes years later) is the remembering self. Neither one may be more important, but the emphasis we place on one or the other could determine our behavior. So, for example, you can enjoy the vacation or take plenty of pictures to relive it later, depending on the self that is more important. A way of finding out which self is more important is to ask ourselves whether we would go on a certain vacation if we could only enjoy it, but not take any pictures (or video, etc).

low response rateA recent discussionon Linkedin pondered whether MR is having its own global warming crisis in the form of an ever dwindling respondent pool. As always, this brought on arguments that response rates need to be improved, quality enforced and of course talk about how much we have slipped as an industry since the good old days. Some blame clients for this (they demand speed and lower cost without concern for quality!) and some blame researchers for not holding clients’ feet to the fire.   It struck me that this is yet another case of researchers not viewing things from a client perspective.

market research conference 2012Well, another conference is over, perhaps our best ever. A great roster of speakers, a room full of engaged attendees and a great location was a terrific formula for a memorable conference. Some highlights from the various sessions:

Lenny Murphy, Editor-in-Chief of the Greenbook blog opened with a wide sweep discussing the waves of changes rocking the market research world. Pulling from the GRIT survey, his discussion with emerging and established players, as well as his itinerant investigation, he was able to convincingly make the case that change in the MR industry is happening. Now. He talked about emerging technologies such as mobile, social media and text analytics and how academic expertise was a key to unlocking a future of new ideas. It was a perfect set-up for the group of academic presentations that were to follow.

Segmenting Movie Goers

Posted by on in New Product Research

A few months ago I posted that we researched 18 factors in deciding which movie to see and where to see it. We reported that “It’s in 3D” was at the bottom of the list, and concluded that 3-D was unlikely to save the American movie box office.  

What made the top of the list was “I like the plot or story,” followed by “It is in my favorite movie genre” and “It has my favorite stars.”  

But surely the plot isn’t the critical decision-maker for every movie-goer; there must be groups of viewers whose decisions revolve around some of the other items on that list. We took their ratings and ran a segmentation analysis. While this type of analysis is done on a much grander scale by researchers in the movie industry, we thought it would be interesting to do some analysis of our own.

market research conferenceOver the past year I’ve blogged about the things that I think will drive the future of Market Research and I’m pleased to announce that for our Frontiers of Research annual conference (May 8th, in NYC, view full agenda or register) we have assembled speakers who will drive that conversation forward. The conference will cover the full spectrum of buzz-worthy topics (Behavioral Economics, Neuroscience, Gamification, Predictive Analytics). And the focus, as always, will be on ideas presented in an easy to understand way (no math!). With speakers from four Ivy League schools, and presentations that range from poker to motion picture box office, this should be an informative and enjoyable day.

Leonard Murphy will set the table by calling on his extensive knowledge of the industry to illuminate how academia can and is driving us forward. Anyone who follows his blog knows that he is not only one of the most knowledgeable industry leaders around, but that he has a provocative view of where we are heading.

new years resolution market researchWe had a notion here at TRC that by the middle of March most New Year’s Resolutions would have been tossed by the wayside, either in favor of giving up something meaningful for Lent, or the simple acknowledgement that this just isn’t the year to lose 25 pounds. Would folks who made a resolution at the beginning of the year still be keeping that resolution 3 months later?

We kicked around a few hypotheses, and then went about testing them using our online panel of consumers:

  • Younger consumers would be more likely to make resolutions than older ones (we figured they hadn’t become jaded by their resolutions not working out over time)
  •  People would be more focused on issues relating to their health (losing weight, exercising more) than other types of resolutions.
  • Most folks who made a resolution would have dropped it by the 3-month mark

So how did our predictions fare?

The Outside View that Daniel Kahneman talks about in his book Thinking, Fast & Slow, is a specific remedy to a problem known as the planning fallacy (i.e.) the inability of people to make predictions. The planning fallacy is part of a larger problem of optimism bias. What is optimism bias? Simply put, people are generally more optimistic than they should be. For example, it is well known that most people think they are better than average drivers, an impossibility. It stems from a general dose of overconfidence not warranted by the situation on hand.

The best example of overconfidence is a study that Kahneman cites of CFOs of large corporations. They were asked to estimate the returns of the S&P Index over the following year. The data were collected over a number of years and hence there was ample opportunity to correlate it with the actual performance of the Index in the following year. Any guesses as to this correlation, given that the respondents should have been expected to have special insight in this matter? It was almost exactly zero, slightly less, in fact! And they seemed to have no idea their forecast was that bad.

Tagged in: Psychology

daniel-kahneman-thinking-fast-slowIn his opus Thinking, Fast & Slow, Nobel winner Daniel Kahneman (click here for previous post) relates a story from early in his career when he was leading a team to develop a curriculum and write a textbook on judgment and decision-making in high schools. He had assembled a group of experts and after working diligently for a year they had completed an outline of the syllabus and written two chapters. One fine day when discussing procedures for estimating uncertain quantities, it occurred to him that he should get an estimate from everyone on how long he thought this whole project would take. Being the clever psychologist that he was, rather than ask the group to guess publicly, he asked each person to make a confidential prediction. The mean was about two years and the range was about half a year on either side. In other words, the group was very consistent in its prediction.  

Then Kahneman had the idea of asking the curriculum expert in the group, Seymour Fox, for his specific opinion. Only this time he asked Seymour to think about other teams like theirs and asked how long it had taken them to finish. After a long silence the astonishing answer came out. Nearly half the groups never even finished the project. Among those who did the average time taken was about seven years! Seymour Fox also estimated that this group was slightly below average in terms of the skill set it possessed compared to the other groups. The killer, of course, was how long it actually took Kahneman’s group to complete their project. Eight years!

Effectively what had happened was that a group of experts in judgment and decision-making had somehow fooled themselves into thinking way too optimistically about the future and had made predictions based on it. This included the expert who in spite of having the best information somehow ignored that in favor of an optimism bias. As Kahneman graciously adds, it also included a leader who did not pull the plug on a project that would likely take another six years and was a coin toss as to whether it would even be completed.  

The biggest lesson Kahneman draws from this episode is that there are two approaches to forecasting which he labels the inside view and the outside view. The inside view is when we focus on the specifics of our own situation, try to form a coherent story and somehow convince ourselves that given the “special” nature of our situation success is just around the corner. In some ways this probably explains the enormously high failure rates of new products and the only slightly lower failure rates of new small businesses. The outside view is one that takes into account the general failure rate of the reference class of objects. Assuming the reference class is properly chosen, the outside view should provide a nice ballpark of where the estimate is going to be. In practice it is better to start there and adjust it using the special knowledge of the inside view and thus avoid embarrassing predictions. Not following this kind of procedure is why we routinely read about say, large transportation projects often running over by years and into several times the original projected cost. It is also why kitchen renovations routinely cost twice the initial estimate for the average household.

So are there specific lessons for market researchers? Of course. One is with the likelihood of success of any kind of new technological advance (mobile, neuro, text analytics, social media monitoring, whatever). Without understanding the reference information for how such new technologies can ultimately fare, we can too easily get caught up in the fanciful nature of a specific technology and make prognostications not just about success, but also about time frames within which such things can come true. On the flip side the death of older technologies can be too gleefully forecast (“Surveys will die in a year!”) because of the glamour of newer techniques if the reference cases are not carefully analyzed.

...

Even Economists Are Gamifying

Posted by on in New Product Research

Gamification as a means to understand consumer choice is a relatively new idea for research (and controversial in many circles), but it is not new everywhere. For example, one sociologist, Dmitri Williams, has been studying economic behavior using gamfication for four years. His experiments were based on the online fantasy game EverQuest II, which involves thousands of players selling millions of virtual items every month. In essence it is a fantasy economy that works like a real economy.  

Professor Williams theorized that this provided an opportunity to observe the choices players made without fear of the Hawthorne effect (some people give different answers when they know they are being watched).   It also allowed him to set up test and control groups and observe what happens when, to take a simple example, prices go up (if you guessed “people buy less” you win) and to look at gender roles. He saw applications in many fields, not the least of which being testing the impact of various government intervention options before implementing them in the real world.

buffet sign panel studyOn a trip to Las Vegas in November 2011 I was twice presented with an option to move to the head of the line – for a price. I could take advantage of “early check-in” by paying $25. And I could get my buffet breakfast right away without waiting in line, again for a small fee. The buffet sign struck me as peculiar, since the 4 people ahead of me didn’t really constitute much of a “line”. I snapped a photo.

The concept of express fees is nothing new – Universal Florida, for example, has offered its ExpressSM Plus Pass for years, affording visitors to skip the regular lines, and as a result experience more attractions during their visit. But the express fee is spreading beyond the domain of the theme park.  You can even pay to bypass the long security lines at the airport now, if you’re so inclined.

This got me thinking...who’s in such a rush?  And, even more important, who’s willing to fork over some cash so they won’t waste any more time waiting? We put that question to the test with a small web survey among members of TRC’s online panel.

Among the general population of adults, paying for speedy service is a somewhat polarizing notion. While about half of our survey takers are neutral on the concept, 1/3 are pro and 1/5 are anti. We asked about specific situations as well. Paying for early hotel check-in has nearly twice as many fans (23%) as paying for premium seating at a movie (12%) or paying to jump the line at a warehouse store (13%).

Discrete Choice in a Police Lineup

Posted by on in New Research Methods

police lineup discrete choiceThe Economist reviewed a study by Dr. Neil Brewer about effective police lineups which I think had implications for Market Research. Like researchers, police typically like to encourage witnesses to take their time to ensure they are making the correct choice. This makes logical sense, more time, means more thinking which naturally should lead to better results. Sadly, Dr. Brewer found otherwise.

He had volunteers view short films which detailed mundane scenes of everyday life and a crime (shoplifting, car theft, etc). Later (some minutes later, some a week), they were asked to identify the criminal from a group of 12 pictures of “suspects”. Half were given 3 seconds to evaluate each picture and asked how confident they were of their choice. The other half were given as much time as they wanted. The results showed that the group that had the limited time was correct 67% of the time. The group with more time was only correct 49% of the time.  

A former colleague of mine used to tell us to “torture the data until it confessed”. In other words, don’t just stop your investigation at the first finding. But rather, keep poking, prodding, flipping and coercing until you feel you’ve uncovered all the data has to give. Ah…images of Jack Bauer doing his thing flash through my mind just thinking about our own data “torture” sessions.

All kidding aside, what my colleague was really trying to say was spot on. I’m sure we’ve all known researchers who habitually stop at the first find. They rarely take the time to consider different ways of looking at data, of considering the message within.

The Nobel Prize winner and the intellectual godfather of behavioral economics, Daniel Kahneman, has summarized a lifetime of research in his recent book Thinking, Fast & Slow. In the next few blog posts I will be drawing upon some concepts that he espouses and link them up to research to see what practitioners can take away from his four decades of work.

This post goes directly to the title of the work; fast and slow thinking. This is the foundation of his work. He and his great collaborator Amos Tversky, (who passed away and therefore could not receive the Nobel) see human thinking in two forms that they call System 1 and System 2. More aptly they could be called “automatic” and “effortful” systems, but Fast and Slow is a good shorthand description. According to Kahneman’s description,

System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control

System 2 allocates attention to the effortful mental activities that demand it, including complex computations”

In my last blog, we learned that the answer to the question as to whether 3D can save the American Movie Box Office is Probably Not. The adult consumers we surveyed do not view 3D as important to them in selecting a movie to see.

But what is important?

We polled 829 US consumers age 18+ who are part of TRC's online panel. They were part of a broader test in which we experimented with various methods to determine the best way to differentiate importance factors in the decision-making process. We chose movie decision-making as our topic, and participants evaluated 18 factors in their decision which movie to see and where to see it.

movieticket_3dglasses3D is all the rage in Hollywood and is coming to a TV set near you if it isn't there already. 3D@Home Consortium lists no fewer than 20 movies planned for theatrical release in 2012 that will be offered up in 3D. These include Men in Black 3, Star Trek 2 and The Ring 3D.

But is Hollywood's push toward 3D the result of consumer demand? Holly McKay reporting for FoxNews.com says that less than 50% of the box office earnings for Kung Fu Panda 2, Pirates of the Caribbean, Green Lantern and Cars 2 in 2011 were from 3D showings.

But how does 3D fit in as a draw relative to the other decisions a potential movie-goer makes? Does 3D motivate an American adult to select a movie to see on a given day?

Apparently not.

You Think Researchers Have It Tough?

Posted by on in Healthcare

For the past few years MR blog posts have been dominated by posts questioning the future of Market Research or talking about just how tough it is to be a researcher in the new millennium. A recent discussion on Linkedin about the threat from DIY is a good example. If you read my blog frequently you know that I see the industry evolving, not going extinct. In any case, at TRC we do a great deal of research about Health Insurance and so I know that as challenging as research is, it is nothing compared to what the health insurance industry is going through.

First off, I'll ignore issues that have been with the industry for decades. More often than not they don't sell to the folks who use their products (most insurance comes through employers) and they often don't sell to the folks who pay the bills (a majority of insurance is sold through independent brokers). While some research clients don't expose us to their internal clients, we are nowhere near as separated from the folks who use our work as health insurance firms are.

Tagged in: Brand Market Research

blackswanThe Black Swan is a book that was published a few years ago and generated much publicity and at least some controversy. It occurred to me that there are lessons market researchers can learn from that book, particularly about the relationship between qualitative and quantitative data obtained from a survey format. The idea is that the framework used to analyze such data is different from that used for directly obtained qualitative data through methods such as IDIs and focus groups. Understanding the difference between quantitative and qualitative frameworks for data analysis (and in particular, the difference between statistical and managerial outliers) can help derive more value when the qualitative data are collected in a regular survey. But first, let's take a detour.

A Brief Tour of The Black Swan

In his informative (and entertaining) book, Nassim Nicholas Taleb argues that real data are either distributed normally (from "mediocristan") or not (from "extremistan"). The former are characterized by data that follow the traditional normal distribution (or bell curve). The majority of the distribution is near the middle surrounding the average and as we venture further out the number of observations becomes increasingly scarce. It is a distribution that defines many phenomena in the natural world. In fact, basic statistics shows that with a reasonable number of observations most distributions start approximating the normal.

Want to know more?

Give us a few details so we can discuss possible solutions.

Please provide your Name.
Please provide a valid Email.
Please provide your Phone.
Please provide your Comments.
Enter code below : Enter code below :
Please Enter Correct Captcha code
Our Phone Number is 1-800-275-2827
 Find TRC on facebook  Follow us on twitter  Find TRC on LinkedIn

Our Clients