Welcome visitor you can log in or create an account

800.275.2827

Consumer Insights. Market Innovation.

blog-page
Subscribe to this list via RSS Blog posts tagged in misleading statistics
statistics-in-market-research-
Last week I was watching the CBS morning news and they had a story about a new study that indicated that your attitude toward gym class as a child shaped your attitude toward exercise your entire life. After watching the story I am convinced that it is another case of causation being confused for correlation.
 
The basics were that kids who reported loving gym class were far more active decades later than kids who reported finding it stressful (“I was always picked last”).  I don’t doubt this correlation. My problem is they spoke of a need to make gym class more inclusive so that all kids grow up to exercise more. In other words, if we can take the stress out of gym for kids who are not good at sports, we can get them to love exercise more. I’m not sure achieving the first part of this is possible and I’m even more certain that even if we do it will not alter future behavior. 
 
I don’t see how you can eliminate the anxiety about gym class without eliminating the physical activity. Sure you could eliminate picking teams and save that humiliation, but once the games begin the kids who are poor at sports will continue to feel anxiety. Even if you simply make it an exercise class, the kids who are out of shape will stand out. Short of one-on-one classes I don’t see how you can fix the problem.
 
Doing so is also not likely to make us more active as adults. The kids who were not good at gym were not good for a variety of reasons but likely they either lacked the natural talent OR more likely the interest in sports that the athletes had. Gym class wasn’t the cause of this! If there had been no gym class I would bet that the kids who didn’t like gym class would still be less active than those that did (those who loved it and/or were good at it). I’d point the “causation arrow” backwards….if you love sports as an adult you probably liked gym class.  
 
It is easy to forget that we have to play a role in explaining statistical principles in our reporting and not just when doing sophisticated work like Discrete Choice Conjoint, Max-Diff, Segmentations and Regressions. Our direct clients likely understand causation and correlation issues, but it is important to know that their internal clients may not. Clear justification for pointing the “causation arrow” must be provided in reports and presentations. Just as important is knocking down attempts to point the arrow based solely on correlation. Otherwise they may walk away with a completely false assumption and not double back with researchers to validate it.  
 
This study is also useful in highlighting another common mistake made by internal clients. I was telling an old friend about the study and he said “that can’t be right, you hated gym class and you are far more active now than when we were kids”. Imagine me in a focus group telling my story of how much I hated waiting to be picked for a team and how my memory of that humiliation caused me to exercise more and more as an adult. The internal client stands up and says “That’s how we make people healthier…more humiliation in gym class!” In that case, someone will be in the room to point out that one person’s story is not projectable to the population…but that’s another blog. 
 
Hits: 599 0 Comments

The Olympics of Statistics

Posted by on in A Day in a (MR) Life

Watching sports provides a lot of great entertainment. The thrill of victory, agony of defeat and all that. It also provides many great opportunities for never ending arguments about just how great various sports achievements are. Often these arguments are bolstered by the misuse of statistics. One such example was the constant references to Michael Phelps as “the Greatest Olympian Ever” which was based on the fact that he’d won more medals than any other athlete in history.  

To be clear, I’m sure an argument can be made that he is the greatest ever, but the use of one number, medal count, to determine that really bothers me. As often happens in the media, the number is looked at in only one context (compared to the number of medals other athletes have won) rather than considering a great number of other factors:

Is Cybercrime a Huge Problem?

Posted by on in A Day in a (MR) Life

cybercrimeCybercrime is a fear for just about everyone, from individuals fearing identity theft to large corporation guarding sensitive data. The question is, how valid is this fear?   It is a question that was raised recently in an Economist article and it makes it clear that politicians are not the only ones who misuse and abuse numbers.

Claims have been made that cybercrime is bigger than the drug trade and that it costs a trillion dollars annually. Most of these figures come from firms who specialize in preventing cybercrime...in other words the same folks who will benefit if people feel the need to protect themselves from cybercrime. These figures are generally not questioned, either out of numerical ignorance or the belief (probably correct) that big numbers scare people and help to sell newspapers (or in today's world web hits).  

Numbers That Don't Add Up

Posted by on in Better Graphics

In my last blog I talked about a simple chart on Morning Joe, which was presented by Steven Rattner. I submitted that when we see data presented in the media or especially by politicians, we should judge it in terms of how a researcher would have presented the same data (because of course researchers are free of bias...well let's leave that for another blog). I gave Mr. Rattner a pass last time, but his presentation of a chart on infrastructure was misleading and would only have pleased a client who wanted misleading data to prove a point.

In this case he presented a chart showing infrastructure spending as a percentage of GDP . It showed a massive drop from the high in the 1950's to the low of today. The chart had a y axis that went from 0% to 1.5% which made the drop easier to see. Nothing wrong with that (assuming those viewing the chart understood that it was not based on 0-100%).

A  few blogs back I talked about how the political season would bring on a rash of misuse and abuse of numbers. I've had my ears open for examples and a couple that came up recently got me to realize that a more nuanced view is necessary here. The real rule should be that pundits and politicians should be held to the same standards as we are by our clients. Namely, the numbers should help in the decision making process...not mislead or confuse the facts.

In the next two blogs I'll use some charts presented by Steven Rattner on the Morning Joe television program. For those of you who don't know, Mr. Rattner was the President's Car Czar. While this probably means he comes with his own bias, I have generally found that when he presents data he does so in a pretty fair way.

Was at Il Tartufo* in Manayunk the other night, waiting near the bar after dinner when a waiter - who was not my waiter - surprised me. "How'd you like the Fettuccine Cinghiale?" she asked.

I had liked it just fine, but was curious to know how she knew what I'd eaten. I hadn't seen her near my table all evening.

"I just saw the bill for your table," she replied. "Guys always get the wild boar pasta."

Part waiter and part analyst - new competition for us market researchers?

I Look at Data From Both Sides Now

Posted by on in A Day in a (MR) Life

caddie and golferI was watching the final round of the Bridgestone Invitational and my 14 year old son came in to the room.  I told him the established narrative. After a difficult two years Tiger Woods had returned to golf, but not before firing his long time and very loyal caddie.   Most saw this as just plain nasty on Tiger's part.  

I then told him how another golfer, Adam Scott, hired the caddie and was now on the verge of winning the tournament. I summed it up by saying that justice had prevailed.

He didn't even miss a beat before asking me, "Did Adam Scott fire his caddie so that he could hire the caddie Tiger fired?"

I don't follow competitive golf closely enough to know the answer. Worse, I had not even considered that the narrative "Tiger mean/Adam good" might be a bit off.  

A good lesson for any analyst to learn.

Hits: 338674 0 Comments

istock_000000237809xsmallThe recent New MR Virtual Festival on presenting data had a number of really useful and interesting presentations. Mike Sherman’s presentation, “Less is More: Getting Value (Not Just Reams of Data) From Your Research” led to an interesting exchange that I think highlights the change in thinking that Market Research must make.

Mike reiterated the point that many have been making…we need to focus our reporting on the key things we learned and not waste executives’ time with a lot of superfluous information. In addition, the report should not just summarize the data, but rather it should synthesize it. He gave an example of a data set with these facts:

  • · Jim broke his knee
  • · A burglar broke Jim’s car window
  • · Jim got a speeding ticket.

A summary of these data might be “Jim’s knee and car window were damaged and he got a speeding ticket”.

A synthesis of that data would be “Jim has been living dangerously”.

The 2012 Presidential Election season is upon us. I don't know about you, but other than the barrage of commercials, the thing I like least about political campaigns is the terrible abuse of numbers. Combined with the current debate on the debt limit and we have the makings of a tsunami of misleading or outright incorrect statistics.

A  few weeks ago, Megan Holstine started a discussion about a Senator using a totally made up statistic. Sadly for him, he quoted a number that was far from accurate, but also one that was easily verified. His defense was that he didn't intend the statistic to be taken "literally".

Makes me wonder if perhaps we've got it wrong.  Think of the possibilities for us if we stopped taking numbers literally!

Want to know more?

Give us a few details so we can discuss possible solutions.

Please provide your Name.
Please provide a valid Email.
Please provide your Phone.
Please provide your Comments.
Enter code below : Enter code below :
Please Enter Correct Captcha code
Our Phone Number is 1-800-275-2827
 Find TRC on facebook  Follow us on twitter  Find TRC on LinkedIn

Our Clients