Welcome visitor you can log in or create an account

800.275.2827

Consumer Insights. Market Innovation.

blog-page
Recent blog posts

new years resolution market researchWe had a notion here at TRC that by the middle of March most New Year’s Resolutions would have been tossed by the wayside, either in favor of giving up something meaningful for Lent, or the simple acknowledgement that this just isn’t the year to lose 25 pounds. Would folks who made a resolution at the beginning of the year still be keeping that resolution 3 months later?

We kicked around a few hypotheses, and then went about testing them using our online panel of consumers:

  • Younger consumers would be more likely to make resolutions than older ones (we figured they hadn’t become jaded by their resolutions not working out over time)
  •  People would be more focused on issues relating to their health (losing weight, exercising more) than other types of resolutions.
  • Most folks who made a resolution would have dropped it by the 3-month mark

So how did our predictions fare?

The Outside View that Daniel Kahneman talks about in his book Thinking, Fast & Slow, is a specific remedy to a problem known as the planning fallacy (i.e.) the inability of people to make predictions. The planning fallacy is part of a larger problem of optimism bias. What is optimism bias? Simply put, people are generally more optimistic than they should be. For example, it is well known that most people think they are better than average drivers, an impossibility. It stems from a general dose of overconfidence not warranted by the situation on hand.

The best example of overconfidence is a study that Kahneman cites of CFOs of large corporations. They were asked to estimate the returns of the S&P Index over the following year. The data were collected over a number of years and hence there was ample opportunity to correlate it with the actual performance of the Index in the following year. Any guesses as to this correlation, given that the respondents should have been expected to have special insight in this matter? It was almost exactly zero, slightly less, in fact! And they seemed to have no idea their forecast was that bad.

Tagged in: Psychology

daniel-kahneman-thinking-fast-slowIn his opus Thinking, Fast & Slow, Nobel winner Daniel Kahneman (click here for previous post) relates a story from early in his career when he was leading a team to develop a curriculum and write a textbook on judgment and decision-making in high schools. He had assembled a group of experts and after working diligently for a year they had completed an outline of the syllabus and written two chapters. One fine day when discussing procedures for estimating uncertain quantities, it occurred to him that he should get an estimate from everyone on how long he thought this whole project would take. Being the clever psychologist that he was, rather than ask the group to guess publicly, he asked each person to make a confidential prediction. The mean was about two years and the range was about half a year on either side. In other words, the group was very consistent in its prediction.  

Even Economists Are Gamifying

Posted by on in New Product Research

Gamification as a means to understand consumer choice is a relatively new idea for research (and controversial in many circles), but it is not new everywhere. For example, one sociologist, Dmitri Williams, has been studying economic behavior using gamfication for four years. His experiments were based on the online fantasy game EverQuest II, which involves thousands of players selling millions of virtual items every month. In essence it is a fantasy economy that works like a real economy.  

Professor Williams theorized that this provided an opportunity to observe the choices players made without fear of the Hawthorne effect (some people give different answers when they know they are being watched).   It also allowed him to set up test and control groups and observe what happens when, to take a simple example, prices go up (if you guessed “people buy less” you win) and to look at gender roles. He saw applications in many fields, not the least of which being testing the impact of various government intervention options before implementing them in the real world.

buffet sign panel studyOn a trip to Las Vegas in November 2011 I was twice presented with an option to move to the head of the line – for a price. I could take advantage of “early check-in” by paying $25. And I could get my buffet breakfast right away without waiting in line, again for a small fee. The buffet sign struck me as peculiar, since the 4 people ahead of me didn’t really constitute much of a “line”. I snapped a photo.

The concept of express fees is nothing new – Universal Florida, for example, has offered its ExpressSM Plus Pass for years, affording visitors to skip the regular lines, and as a result experience more attractions during their visit. But the express fee is spreading beyond the domain of the theme park.  You can even pay to bypass the long security lines at the airport now, if you’re so inclined.

This got me thinking...who’s in such a rush?  And, even more important, who’s willing to fork over some cash so they won’t waste any more time waiting? We put that question to the test with a small web survey among members of TRC’s online panel.

Among the general population of adults, paying for speedy service is a somewhat polarizing notion. While about half of our survey takers are neutral on the concept, 1/3 are pro and 1/5 are anti. We asked about specific situations as well. Paying for early hotel check-in has nearly twice as many fans (23%) as paying for premium seating at a movie (12%) or paying to jump the line at a warehouse store (13%).

Discrete Choice in a Police Lineup

Posted by on in New Research Methods

police lineup discrete choiceThe Economist reviewed a study by Dr. Neil Brewer about effective police lineups which I think had implications for Market Research. Like researchers, police typically like to encourage witnesses to take their time to ensure they are making the correct choice. This makes logical sense, more time, means more thinking which naturally should lead to better results. Sadly, Dr. Brewer found otherwise.

He had volunteers view short films which detailed mundane scenes of everyday life and a crime (shoplifting, car theft, etc). Later (some minutes later, some a week), they were asked to identify the criminal from a group of 12 pictures of “suspects”. Half were given 3 seconds to evaluate each picture and asked how confident they were of their choice. The other half were given as much time as they wanted. The results showed that the group that had the limited time was correct 67% of the time. The group with more time was only correct 49% of the time.  

A former colleague of mine used to tell us to “torture the data until it confessed”. In other words, don’t just stop your investigation at the first finding. But rather, keep poking, prodding, flipping and coercing until you feel you’ve uncovered all the data has to give. Ah…images of Jack Bauer doing his thing flash through my mind just thinking about our own data “torture” sessions.

All kidding aside, what my colleague was really trying to say was spot on. I’m sure we’ve all known researchers who habitually stop at the first find. They rarely take the time to consider different ways of looking at data, of considering the message within.

The Nobel Prize winner and the intellectual godfather of behavioral economics, Daniel Kahneman, has summarized a lifetime of research in his recent book Thinking, Fast & Slow. In the next few blog posts I will be drawing upon some concepts that he espouses and link them up to research to see what practitioners can take away from his four decades of work.

This post goes directly to the title of the work; fast and slow thinking. This is the foundation of his work. He and his great collaborator Amos Tversky, (who passed away and therefore could not receive the Nobel) see human thinking in two forms that they call System 1 and System 2. More aptly they could be called “automatic” and “effortful” systems, but Fast and Slow is a good shorthand description. According to Kahneman’s description,

System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control

System 2 allocates attention to the effortful mental activities that demand it, including complex computations”

In my last blog, we learned that the answer to the question as to whether 3D can save the American Movie Box Office is Probably Not. The adult consumers we surveyed do not view 3D as important to them in selecting a movie to see.

But what is important?

We polled 829 US consumers age 18+ who are part of TRC's online panel. They were part of a broader test in which we experimented with various methods to determine the best way to differentiate importance factors in the decision-making process. Our white paper can fill in the details. We chose movie decision-making as our topic, and participants evaluated 18 factors in their decision which movie to see and where to see it.

movieticket_3dglasses3D is all the rage in Hollywood and is coming to a TV set near you if it isn't there already. 3D@Home Consortium lists no fewer than 20 movies planned for theatrical release in 2012 that will be offered up in 3D. These include Men in Black 3, Star Trek 2 and The Ring 3D.

But is Hollywood's push toward 3D the result of consumer demand? Holly McKay reporting for FoxNews.com says that less than 50% of the box office earnings for Kung Fu Panda 2, Pirates of the Caribbean, Green Lantern and Cars 2 in 2011 were from 3D showings.

But how does 3D fit in as a draw relative to the other decisions a potential movie-goer makes? Does 3D motivate an American adult to select a movie to see on a given day?

Apparently not.

You Think Researchers Have It Tough?

Posted by on in Healthcare

For the past few years MR blog posts have been dominated by posts questioning the future of Market Research or talking about just how tough it is to be a researcher in the new millennium. A recent discussion on Linkedin about the threat from DIY is a good example. If you read my blog frequently you know that I see the industry evolving, not going extinct. In any case, at TRC we do a great deal of research about Health Insurance and so I know that as challenging as research is, it is nothing compared to what the health insurance industry is going through.

First off, I'll ignore issues that have been with the industry for decades. More often than not they don't sell to the folks who use their products (most insurance comes through employers) and they often don't sell to the folks who pay the bills (a majority of insurance is sold through independent brokers). While some research clients don't expose us to their internal clients, we are nowhere near as separated from the folks who use our work as health insurance firms are.

Tagged in: Brand Market Research

blackswanThe Black Swan is a book that was published a few years ago and generated much publicity and at least some controversy. It occurred to me that there are lessons market researchers can learn from that book, particularly about the relationship between qualitative and quantitative data obtained from a survey format. The idea is that the framework used to analyze such data is different from that used for directly obtained qualitative data through methods such as IDIs and focus groups. Understanding the difference between quantitative and qualitative frameworks for data analysis (and in particular, the difference between statistical and managerial outliers) can help derive more value when the qualitative data are collected in a regular survey. But first, let's take a detour.

A Brief Tour of The Black Swan

In his informative (and entertaining) book, Nassim Nicholas Taleb argues that real data are either distributed normally (from "mediocristan") or not (from "extremistan"). The former are characterized by data that follow the traditional normal distribution (or bell curve). The majority of the distribution is near the middle surrounding the average and as we venture further out the number of observations becomes increasingly scarce. It is a distribution that defines many phenomena in the natural world. In fact, basic statistics shows that with a reasonable number of observations most distributions start approximating the normal.

what am i supposed to doYes, it is a rather important issue and can be approached in a variety of ways. My purpose with this post is not to provide a comprehensive answer, but look at one specific solution based on what I recently read. The book is Thinking, Fast and Slow, the Nobel Prize winner Daniel Kahneman's excellent summary of a lifetime of research. He is perhaps the most accomplished psychologist around and could (among other things) justifiably be called the intellectual godfather of behavioral economics. It is always worth listening to what he says and in this particular case, it seems to me there is a nugget that applies to making quantitative research more actionable.

How to Be Happy by Spending Money Wisely

Posted by on in Consumer Behavior

It is that time of year when many people's thoughts turn towards buying gifts for loved ones. More generally it is a time when thoughts related to money and happiness occupy our attention. When thinking of ways to spend money either on oneself, for loved ones or even for complete strangers wouldn't it be nice if there was some actual research to provide data-based guidance on the topic? As it happens, there is. Researchers Elizabeth Dunn of the University of British Columbia, Daniel Gilbert of Harvard and Timothy Wilson of the University of Virginia have identified, through their research, eight principles designed to help consumers get more happiness for their money. Follow them as you will to enhance your life.

Tagged in: Psychology

webcamAs researchers it is critical that we ensure our data accurately reflect the thinking of the market....in other words, getting to the truth. This is complicated by several factors including limitations of a questionnaire, respondent's lack of attention and the fact that people don't always know what they really want or need. While careful design and methodology can help to minimize these issues (at TRC we believe in using choice questions and shorter surveys) and the use of other data (which can establish the facts), it is impossible to eliminate them.

Technology such as eye tracking, bio metrics and facial recognition software can be applied to neuroscience to help us understand more about what respondents are thinking. The trouble is they are often expensive (sometimes getting the whole truth isn't worth the price) and slow down the research process (sometimes a faster less complete answer is better than a slow one). The limited data available also make it difficult to draw good conclusions. An outstanding presentation at the ARF's 75th Annual Conference showed this quite well.

Is Cybercrime a Huge Problem?

Posted by on in A Day in a (MR) Life

cybercrimeCybercrime is a fear for just about everyone, from individuals fearing identity theft to large corporation guarding sensitive data. The question is, how valid is this fear?   It is a question that was raised recently in an Economist article and it makes it clear that politicians are not the only ones who misuse and abuse numbers.

Claims have been made that cybercrime is bigger than the drug trade and that it costs a trillion dollars annually. Most of these figures come from firms who specialize in preventing cybercrime...in other words the same folks who will benefit if people feel the need to protect themselves from cybercrime. These figures are generally not questioned, either out of numerical ignorance or the belief (probably correct) that big numbers scare people and help to sell newspapers (or in today's world web hits).  

Market researchers are fighting each day for a seat at the decision-making table. More and more "research professionals" are being bypassed by smart people with access to good tools, a hotly-debated topic within our community and perhaps a harbinger of what's to come in terms of when and how client- and vendor-side researchers get to contribute advice and ideas.

And yet too many researchers believe the value of market research is self-evident, and that the challenge facing our industry is really more of an obstacle caused by "everyone else." I see this train of thought emerge frequently on Twitter, or within any number of blogs and MRX-related posts.  It typically gets expressed along these lines:

Netflix screwed up. McDonald's screwed up. Coca-Cola screwed up (multiple times). If only they had done research!  A (name any large dollar amount) disaster that could have been averted with a $100,000 investment in listening to the customer. Silly companies.

Folks, it's hard to get better without humility.

Tagged in: Market Research
Recent comment in this post - Show all comments
  • Ed Olesky
    Ed Olesky says #
    Great post. Research needs to have impact on decision making - and demonstrably so. For suppliers, it's difficult to get all the a

Give Thanks for the Unknown

Posted by on in Market Research

1620 mayflower rockThis month here in the States we will be celebrating our biggest secular holiday, Thanksgiving.   Traditionally, the holiday is thought to have started when early settlers to the "new" world, the Pilgrims, sat down to have a meal to celebrate the harvest with the Native American's who had befriended them. As we begin to close out 2011 in an industry facing an uncertain future, I was struck by the similarities between those early settlers and market researchers today.

On the surface the story of adventurers seeking a better life is a bit different than the story of boring market researchers seeking to survive, but I disagree.

thanksgiving turkey dinnerLater this month, those of us in the United States celebrate one of my favorite holidays, Thanksgiving. Officially, Thanksgiving is a post-harvest celebration that was brought to the Americas by European settlers in the 16th or 17th century (depending on which historian you believe). Unofficially, it's the day where families and friends gather to feast, take naps and watch football. Oh my, even as I type this my mouth is watering...turkey, potatoes, stuffing, cranberry sauce, peas and the like, with chasers of pumpkin, apple and other assorted pies. All delicious, but I particularly love eating turkey on Thanksgiving.

The REAL Mobile Opportunity

Posted by on in A Day in a (MR) Life

Sometimes it seems like the future of quantitiative mobile research has already been determined.

onlinemobilesurvey- Real-short surveys, 5 to 10 questions long.
- Simple-response controls like big radio buttons.
- Small screens = small tasks = limited data sets.

At a time when clients, budgets and timelines are demanding that we do more with less, mobile quant would seem to do a pretty good job with the "less" part of things. If we're being honest that makes us primary researchers a little nervous, and prone to think of mobile as an interesting but ultimately niche methodology.

 

The change is a comin'

But I'd wager that the current definition of "short" and "simple" will change over time as more consumers come to live fully mobile lives, and mobile devices become an increasingly "best" way to reach people for feedback. Conventional wisdom says ask only 5 to 10 questions and use the simplest of instructions, but how can that be the end of the story when people - right now - are browsing, shopping, and buying on their Smartphones?

Want to know more?

Give us a few details so we can discuss possible solutions.

Please provide your Name.
Please provide a valid Email.
Please provide your Phone.
Please provide your Comments.
Enter code below : Enter code below :
Please Enter Correct Captcha code
Our Phone Number is 1-800-275-2827
 Find TRC on facebook  Follow us on twitter  Find TRC on LinkedIn

Our Clients