Welcome visitor you can log in or create an account

800.275.2827

Consumer Insights. Market Innovation.

blog-page
Recent blog posts

Our utilities clients have raised the issue of infrastructure improvements on more than one occasion. These improvements are often expensive to implement, but the average customer sees no tangible benefit -- the water ran yesterday and it’s running again today.

Yet maintaining the pipes, lines and wires are critical to keeping water and power flowing to our homes and businesses. When it comes time to invest in these improvements, it’s hard to rally support when communities face other issues that can produce more visible outcomes when addressed.

Just how far apart are community leaders and residents about the importance of improving their communities’ infrastructure?

We decided to find out.

Asymmetry and the Lottery

Posted by on in Market Research

If the lottery can accurately be called a “tax on the stupid”, does my playing it make me stupid? To understand (or perhaps rationalize) the answer, you need to understand the principles of Asymmetry

As usually happens when the jackpot on PowerBall goes into the stratosphere (in this case it reached nearly $600 Million), someone here at TRC started a collection to play as a group. A pretty high percentage of our staff decided to play, even those with the most advanced degrees in statistics. So given the chances of winning are something like 1:175 million per ticket, why did we do it?

It certainly wasn’t that by buying so many tickets (nearly 50), the odds became anything near a slam dunk. In fact, they were easy enough to calculate (1:3,650,489.79) so there was no doubt in my mind that I wouldn’t win when I played and yet I still did.

The reason was simple. I had to choose to play or not to play and consider the likely outcome if we won or didn’t win:

  • I play and lose (A small $6 loss and an outcome that my brain expected all along)
  • I play and win (A massive win with my share being $10Million…despite expecting to lose, my brain is now elated)
  • I don’t play and they lose (I have some very minor bragging rights, but ultimately I missed out on the fun and only saved $6)
  • I don’t play and they win (Even as I console myself that the odds were with me, I feel like a complete idiot)

In other words, playing offered only upside and not playing only downside. That is exactly why we consider Asymmetric effects whenever we do analysis.   Otherwise we may miss what really drives consumer decision making.

Hits: 3785 0 Comments

electoral map 2012 nov 7tWas the election outcome a surprise for you? It wasn’t for me.

In some ways election night was quite boring. And I blame Nate Silver, Sam Wang and others who predicted the outcome with such stunning accuracy that (at least for me) the drama was completely missing. While conventional pundits and partisans were making all kinds of predictions ranging from “Toss-up” to “Romney landslide”, a group of analysts (nerds, if you choose) were quietly predicting that Obama had a small but consistent and predictable lead. Turns out they were spot-on in their predictions (and were predictably smeared by vested interests).

In my last post I talked about Nate Silver and the approach he uses. This time I want to draw your attention to another analyst, Sam Wang of the Princeton Election Consortium. He is a neuroscientist who has been forecasting for the last three presidential election cycles and has been doing a remarkably good job of it. He nailed the Electoral College vote in 2004 and missed by just one in 2008. How did he do this time? Well, he had two predictions. One of them (based on his median estimator) was 303 for Obama, which is where the tally currently stands, subject to Florida being officially called. The second one (based on his modal estimator) was 332 for Obama which is where the tally is likely to end up if/when Obama wins Florida. Excellent calls whichever way you look at it, given the extremely close race in Florida.

A friend of mine posted on Facebook that she’d taken a web quiz to tell her which presidential candidate best lined up with her stand on the issues. She was outraged that the web site thought she would vote the way it did. I’m not surprised (by the outrage, not her choice)…it is a case of a badly applied choice technique.

Basically the quiz worked by asking a series of questions to see where she stood on the issues. It then aligns her choices against the stand taken by the candidate (if you want to try one, here is one from the GOP Primaries this year). In essence it is a Configurator. Instead of building the perfect product for you (as you would with a Configurator) you build the perfect candidate. There are a couple of problems with this application.

First, Configurators allow you to build the ideal but generally don’t give a clear idea of what choices you might make if that ideal were not available (our proprietary Texo™ helps overcome that issue). In politics it is not unusual for voting decisions to hinge on a single issue and unlike products you can’t decide to add or subtract an important feature.  

I’ll give you the simple answer. Surveys!

No, I don’t mean looking at whatever survey happens to catch your eye or tickles your (or your favorite network or blog’s) ideological fancy. I mean, using a system that is powered by old fashioned surveys and making very, very good explanations and predictions based off that. There is someone who has been doing exactly that for several years now and it makes sense for anyone interested in surveys to understand how he is doing that. I’m talking, of course, about Nate Silver at fivethirtyeight.com.

Interestingly, Silver does not actually do a single survey himself. Instead what he has done is build a database of surveys (that contains thousands) and used some simple and clear rules to analyze them. Based on these rules and the statistical models he has built, he is able to provide the best, unbiased view of the race. All this from survey data. How does he do it? Let’s take a look at some (and by no means all) of his rules.

When we dropped my daughter off for her first year of college a few weeks back my parting words were “Be true to yourself”. I thought this reflected both my accepting that my influence on her was now very limited and my hope that whatever good I’ve done should be put into practice. It strikes me that researchers too should heed the advice.

Our industry has changed and continues to change. Many of the old rules either no longer work or can’t be easily applied to the new tools at our disposal. So how can we apply what we know? A philosophy like “be true to yourself” allows us to do just that.

Personally it has allowed me to accept that representative sampling is no longer the most critical rule (it can’t be in a world where truly representative sampling is too slow and costly). It doesn’t mean I take any respondents I can get…care in trying to get as representative a sample as we can remains important. It just isn’t a stone cold requirement of quantitative research.  

Recent comment in this post - Show all comments
  • Ed Olesky
    Ed Olesky says #
    Nice article, thanks for the information.

The Olympics of Statistics

Posted by on in A Day in a (MR) Life

Watching sports provides a lot of great entertainment. The thrill of victory, agony of defeat and all that. It also provides many great opportunities for never ending arguments about just how great various sports achievements are. Often these arguments are bolstered by the misuse of statistics. One such example was the constant references to Michael Phelps as “the Greatest Olympian Ever” which was based on the fact that he’d won more medals than any other athlete in history.  

To be clear, I’m sure an argument can be made that he is the greatest ever, but the use of one number, medal count, to determine that really bothers me. As often happens in the media, the number is looked at in only one context (compared to the number of medals other athletes have won) rather than considering a great number of other factors:

environmentProtecting the environment is in our collective best interest. Certainly, that’s a given, but people individually don’t always act in their long-term best interests (as behavioral economists posit) so why do we think companies would do so?

Turns out, employers are doing a lot to conserve and protect the environment and natural resources – at least according to TRC’s online panelists we surveyed this spring.

Nearly three-quarters of our panelists who are employed full or part-time told us their employer was actively doing at least one of five activities related to conservation and energy preservation. The larger the employer, the greater the participation. While we can't project our findings to corporate America as a whole, this is certainly encouraging news for our planet.

imaginelehrerOn vacation I read a number of books (love my Kindle) including Why Nations Fail by Daron Acenoglu and James Robinson and Imagine by Jonah Lehrer. While clearly quite different, one on what has allowed some nations to grow and endure while others fail and the other one about unlocking the creative processes of the brain; I took away lessons for my work from both.

“How Nations Fail” isn’t a business book. It is more of a history book than anything, but I saw parallels with what we are facing. The book details a long string of historical examples of nations that either failed outright or that saw some success but then reversed course. The central core is that nations that succeed over time always feature the same factors which feature truly inclusive systems. Meaning, everyone has a chance to succeed on an equal footing. 

Healthcare ReformsWith the recent Supreme Court ruling, it appears that HealthCare Reform is here.  Regardless of which side of the fence consumers fall on, there is important information that they should understand about HCR in order to make critical choices for their care and coverage.  We were interested in finding out how well informed they are now, to see how far we need to go in educating them about their healthcare choices in the coming years. Just under half consider themselves to be slightly knowledgeable, which is about where we’d expect consumers to be at this stage.  One quarter considers themselves knowledgeable and a third report that they are not knowledgeable.

Over the past couple years there have been few topics as hot as “bank fees”. The financial collapse of 2008 started a chain reaction that included lots of consumer outcry and intense regulatory scrutiny. As a result, banks got squeezed…hard. Whether they deserved it or not is a debate for others who are smarter and better informed than I am, but what even I can figure out is that when a business starts to lose money and has its revenue streams cut, it has to identify ways to stop the bleeding. In bank-speak, that means raising fees.

As a consumer, I don't like fees any more than anybody else does, but I also recognize that a business is in business to make money. Rather than curse the fates, or fees in this case, I did what I do best...I researched the issue.

Recent comment in this post - Show all comments
  • Ed Olesky
    Ed Olesky says #
    This was an excellent read Bob. I feel that the consumer should be heard and research is the way to make change happen. Reading yo

mra market research conference 2012Spent a good bit of last week at the MRA conference in San Diego. The weather was overcast and cloudy for the first couple days, a perfect metaphor for the general mood of the industry and uncertain outlook the future holds for us. But as always, I saw a lot to be optimistic about. In particular the first and second to last presentation I watched featured experience researchers who are enthusiastically embracing the opportunities that exist today.

Hal Bloom of Sage Software talked about their satisfaction research using a standard likelihood to recommend approach. They attempt to survey every customer every year and succeed in getting 20% of them to respond. This means tens of thousands of surveys with a multiple of that in terms of open ended responses. Sage makes extensive use of text recognition software to determine sentiment and help sort out who their most vocal promoters and detractors are. A great use of new technology, but what struck me even more was what they do next.

Shane Frederick (Associate Professor at Yale University’s School of Management) did a talk on Behavioral Economics at our recent research conference that got me thinking. But before we tap into the scary place that is my brain, let’s consider what behavioral economics is. Most of us with a formal business education have taken at least one if not several economics classes, during which we were exposed to market theories based on assumptions that sounded reasonable in principle but that really didn’t represent how things worked in real life. Behavioral economics, Shane started, is the study of economics when those assumptions are relaxed, and the relaxation of one of these assumptions, that people act rationally, is what got my attention.

One of the examples Shane used to make his point involved a pivotal point late in a 2009 football game between the New England Patriots and the Indianapolis Colts. Bill Belichick, the coach of the Patriots, decided to go for it on 4th and 2 deep in his own territory. The attempt failed, the Colts scored after the ensuring change of possession and won the game, and nearly everyone in the sports world pointed to Belichicks' seemingly insane decision.  But was it really insane? 

Like any research, market research has always recognized that to be certain results of research can be projected to an entire population; you need to eliminate any bias. We worried about things like:

  • Representativeness Effects – Needed to not only make sure we selected a random representative sample, but then do everything possible to maximize the percentage of people who completed the survey.
  • Interviewer Effects – Surveys needed to be done identically.   If one was done by mail, all should be with identical forms. If done by phone interviewers needed to be careful not to lead respondents and to keep pacing at consistent rate.
  • Framing Effects– If responses from one question are going to potentially bias a future response then the order should be changed to reflect it. In cases where changing the order merely changes which question biases which, use rotation or split samples so that bias effects can be measured and softened.

I know this is a simplified view of things, but the above three do get at the major forms of bias that we seek to eliminate in market research. In this blog, I'll focus on representativeness and at some point in the future I'll cover the other two.

what increases attention paid to adsAdvertisers and researchers do a lot of testing to determine how effective their advertising is prior to launching a campaign or message. We look for ways to get inside consumers’ heads, and as technology improves, we are afforded interesting glimpses into how consumers process information and make decisions. As my colleague Rajan pointed out in his blog different areas of the brain lead to different types of decision-making. Nobel Prize winner Daniel Kahneman posits that human thinking can be classified into two forms, System 1, which operates automatically, and System 2, which requires mental effort (I paraphrase). Jonah Lehrer, author of How We Decide asserts in his blog “Our best decisions are a finely tuned blend of both feeling and reason and the precise mix depends on the situation. When buying a house, for example, it’s best to let our unconscious mull over the many variables. But when we’re picking a stock, intuition often leads us astray. The trick is to determine when to use the different parts of the brain, and to do this, we need to think harder (and smarter) about how we think.”

With all of this exciting work being done in the field of neuroscience and behavioral economics, I wondered what kinds of answers we would get if we simply asked consumers directly what they think motivates them in considering advertising. Do they believe they respond to characters like the Geico gecko? Or is it really just a function of what they need at the time?

In Thinking, Fast & Slow, Nobel winner Daniel Kahneman (click here previous post about Thinking, Fast & Slow) talks about the two selves people have: the experiencing self and the remembering self. The terms are self-explanatory and vacations are a good way to think about them. The part of us that is enjoying the vacation is the experiencing self, while the part that is reliving it later (sometimes years later) is the remembering self. Neither one may be more important, but the emphasis we place on one or the other could determine our behavior. So, for example, you can enjoy the vacation or take plenty of pictures to relive it later, depending on the self that is more important. A way of finding out which self is more important is to ask ourselves whether we would go on a certain vacation if we could only enjoy it, but not take any pictures (or video, etc).

low response rateA recent discussionon Linkedin pondered whether MR is having its own global warming crisis in the form of an ever dwindling respondent pool. As always, this brought on arguments that response rates need to be improved, quality enforced and of course talk about how much we have slipped as an industry since the good old days. Some blame clients for this (they demand speed and lower cost without concern for quality!) and some blame researchers for not holding clients’ feet to the fire.   It struck me that this is yet another case of researchers not viewing things from a client perspective.

market research conference 2012Well, another conference is over, perhaps our best ever. A great roster of speakers, a room full of engaged attendees and a great location was a terrific formula for a memorable conference. Some highlights from the various sessions:

Lenny Murphy, Editor-in-Chief of the Greenbook blog opened with a wide sweep discussing the waves of changes rocking the market research world. Pulling from the GRIT survey, his discussion with emerging and established players, as well as his itinerant investigation, he was able to convincingly make the case that change in the MR industry is happening. Now. He talked about emerging technologies such as mobile, social media and text analytics and how academic expertise was a key to unlocking a future of new ideas. It was a perfect set-up for the group of academic presentations that were to follow.

Segmenting Movie Goers

Posted by on in New Product Research

A few months ago I posted that we researched 18 factors in deciding which movie to see and where to see it. We reported that “It’s in 3D” was at the bottom of the list, and concluded that 3-D was unlikely to save the American movie box office.  

What made the top of the list was “I like the plot or story,” followed by “It is in my favorite movie genre” and “It has my favorite stars.”  

But surely the plot isn’t the critical decision-maker for every movie-goer; there must be groups of viewers whose decisions revolve around some of the other items on that list. We took their ratings and ran a segmentation analysis. While this type of analysis is done on a much grander scale by researchers in the movie industry, we thought it would be interesting to do some analysis of our own.

market research conferenceOver the past year I’ve blogged about the things that I think will drive the future of Market Research and I’m pleased to announce that for our Frontiers of Research annual conference (May 8th, in NYC, view full agenda or register) we have assembled speakers who will drive that conversation forward. The conference will cover the full spectrum of buzz-worthy topics (Behavioral Economics, Neuroscience, Gamification, Predictive Analytics). And the focus, as always, will be on ideas presented in an easy to understand way (no math!). With speakers from four Ivy League schools, and presentations that range from poker to motion picture box office, this should be an informative and enjoyable day.

Leonard Murphy will set the table by calling on his extensive knowledge of the industry to illuminate how academia can and is driving us forward. Anyone who follows his blog knows that he is not only one of the most knowledgeable industry leaders around, but that he has a provocative view of where we are heading.

Want to know more?

Give us a few details so we can discuss possible solutions.

Please provide your Name.
Please provide a valid Email.
Please provide your Phone.
Please provide your Comments.
Enter code below : Enter code below :
Please Enter Correct Captcha code
Our Phone Number is 1-800-275-2827
 Find TRC on facebook  Follow us on twitter  Find TRC on LinkedIn

Our Clients