TRC is celebrating 30 years in business…a milestone to be sure.
Being a numbers guy, I did a quick search to see how likely it is for a business to survive 30 years. Only about 1 in 5 make it to 15 years, but there isn’t much data beyond that. Extrapolation beyond the available data range is dangerous, but it seems likely that less than 10% of businesses ever get to where we are. To what do I owe this success then?
It goes without saying that building strong client relationships and having great employees are critical. But I think there are three things that are key to having both those things:
I’ve always felt that researchers need to be curious and I’d say the same for Entrepreneurs. Obviously being curious about your industry will bring value, but even curiosity about subjects that have no obvious tie in can lead to innovation. For example, by learning more about telemarketing I discovered digital recording technology and applied it to our business to improve quality....
So much has been written about conducting research for new product development. Not surprisingly, as this is an area of research almost every organization, new or old, has to face day in and day out. As market research consultants, we deal with it all the time and thought it would be beneficial to provide our audience with our own recommendations for some useful sources that explain conjoint analysis – a method most often used when researching new products and conducting pricing research.
This is a relatively brief article from Sawtooth Software, the makers of software used for conjoint, that provides an explanation of the basics of conjoint. The paper uses a specific example of golf balls to make it easy to understand.
I recently heard an old John Oliver comedy routine in which he talked about a product he'd stumbled upon...a floating barbeque grille. He hilariously makes the case that it is nearly impossible to find a rationale for such a product and I have to agree with him. Things like that can make one wonder if in fact we've pretty well invented everything that can be invented.
A famous quote attributed to Charles Holland Duell makes the same case: "Everything that can be invented has been invented". He headed up the Patent Office from 1898 to 1901 so it's not hard to see why he might have felt that way. It was an era of incredible invention which took the world that was largely driven by human and animal power into one in which engines and motors completely changed everything.
It is easy for us to laugh at such stupidity, but I suspect marketers of the future might laugh at the notion that we live in a particularly hard era for new product innovation. In fact, we have many advantages over our ancestors 100+ years ago. First, the range of possibilities is far broader. Not only do we have fields that didn't exist then (such as information technology), but we also have new challenges that they couldn't anticipate. For example, coming up with greener ways to deliver the same or better standard of living.
Second, we have tools at our disposal that they didn't have. Vast data streams provide insight into the consumer mind that Edison couldn't dream of. Of course I'd selfishly point out that tools like conjoint analysis or consumer driven innovation (using tools like our own Idea Mill) further make innovation easier.
The key is to use these tools to drive true innovation. Don't just settle for slight improvements to what already exists....great ideas are out there....
We are now officially two months into 2017, which means it’s time to keep up with those New Year’s resolution goals. Resolutions can be difficult to attain in both personal and professional life settings. Recently, I stumbled upon an article by Crawford Hollingworth, an interesting read about behavioral science and its effect on New Year’s resolution goal attainment. As I was reading the article, I realized the suggestions for preparing resolution goals provided in the article also relate to the process of preparing a market research study. The four steps for developing a New Year’s resolution recommended in the article are: Make a plan, Substitute old behavior for new behavior, Make it easy, and Make only one New Year’s resolution. My view on how these strategies relate to market research is as follows:
The first step of the market research journey is to make an action plan. Figure out what the objective of your research is going to be – what do you want to know and from who do you want insight? Next, consider the methods through which you will obtain the most meaningful and useful results for your research objective. Finally, put together a schedule that includes every aspect of the research, including questionnaire design, fielding the survey, data delivery and reporting the research findings.
In the grand scheme of market research methodologies, there are plenty of approaches to choose from that will provide the results needed to make powerful decisions about your product or service. Of course, it is normal human behavior to have the desire to stick to what you know, and market research isn’t much different. However, methodologies are continuing to evolve and can provide findings in various ways. For example, TRC has developed methodologies such as Message Test Express™, Idea Mill™ and Bracket™, along with other solutions that are increasingly popular among the research we conduct. This is an opportunity to be creative and try methodologies that have been tested and offer proven results, which will allow you to view research findings from an alternative perspective.
In order to get reliable results from your research, it is best to start with consideration of the questionnaire design. Plan the design with the end in mind first, then work your way to the front; if you consider what you want to know first, the questions themselves will come together easily. This will allow you to easily interpret and analyze data during the final reporting stages. On the other hand, in terms of the actual survey, you want to avoid developing questions that are overly complicated or time consuming for respondents. Make sure the questions asked make sense and the instructions are clear and concise so that respondents can quickly grasp the idea of what you are asking of them.
A colleague of mine, Rajan Sambandam, provided insight during a recent meeting about the scope of market research studies being “Broad and Shallow” versus “Narrow and Deep” that I found to be interesting. A take-away from his statement is that you should either have a broad and shallow scope through which you will have less informative findings about a larger group of topics, or a narrow and deep scope through which you will have an abundance of detailed findings about one topic. Instead of striving to accomplish both “broad and shallow” and “narrow and deep” research in one initiative, focusing on one or the other will provide the most meaningful and useful information to be applied to your product or service....
Over the years our clients have increasingly looked to us to condense results. Their internal stakeholders often only read the executive summary and even then they might only focus on headlines and bold print. Where in the past they might have had time to review hundreds of splits of Max-Diff data or simulations in a conjoint, they now want us to focus our market research reporting on their business question and to answer it as concisely as possible. All of that makes perfect sense. For example, wouldn’t you rather read a headline like “the Eight Richest People in the World Have More Wealth than Half the World’s Population” than endless data tables that lay out all the ways that wealth is unfairly distributed? I know I would…if it were true.
The Economist Magazine did an analysis of the analysis that went into that headline-grabbing statement from Oxfam (a charity). The results indicate a number of flaws that are well worth understanding.
• They included negative wealth. Some 400 million people have negative wealth (they owe more than they own). So it requires lots of people with very low positive net worth to match the negative wealth of these 400 million people…thus making the overall group much larger than it might have been.
• For example, there are 21 million Americans with a net worth of over $350 Billion. Most of them would not be people you might associate with being very poor…rather they have borrowed money to make their lives better now with the plan to pay it off later.
• They were looking at only material wealth…meaning hard assets like property and cash. Even ignoring wealth like that of George Baily (“The richest man in town!”), each of us possesses wealth in terms of future earning potential. Bill Gates will still have more wealth than a farmer in sub-Saharan Africa, but collectively half the world’s population has a lot of earnings potential....
Now we turn to the real question: why aren't consumers recycling on a more consistent basis? Again we turned to our online consumer research panel and asked those with curbside recycling access who don't recycle regularly a simple question: Why not? What behaviors and attitudes can Recyclers act upon to educate their customers and encourage more recycling?
Well, like any complex problem, there's no one single answer. Lack of knowledge of what's recyclable and being unsure how to get questions answered play a big part (28%). Recyclers can raise awareness through careful and consistent messaging.
But just as significant as knowledge is overcoming basic laziness (29%). Sorting your recycling from your trash takes effort, and not everyone is willing to expend energy to do so. Recyclers may not be able to motivate them, but another concern is addressable, and that's scheduling – having trash and recycling pick-up on different days can de-motivate consumers to recycle (15%).
Another challenge is forgetfulness. Some folks are willing to recycle, but it slips their mind to do so (25%).
Education could help promote a feeling of responsibility and elevate recycling's importance:
• I don't feel that whether or not I recycle makes a difference (14%)
• Recycling isn't important to me (10%)
• I'm not convinced recycling helps the environment (8%)
In a recent survey we conducted among pet owners, we asked about microchip identification. We found that cat owners and dog owners are equally likely to say that having their pet microchipped is a necessary component of pet ownership. That’s the good news.
The bad news is that when it comes time to doing it, the majority haven’t taken that precaution. 69% of the cat owners and 64% of the dog owners we surveyed say they haven’t microchipped their companion.
Why is microchipping so important? Petfinder reports that The American Humane Association estimates over 10 million dogs and cats are lost or stolen in the US every year, and that 1 in 3 pets will become lost at some point during their lifetime. ID tags and collars can get lost or removed, which makes microchip identification the best tool shelters and vets use to reunite pets with their owners.
One barrier to microchipping is cost – it runs in the $25 to $50 dollar range for dogs and cats. Not a staggering amount, but pet ownership can get expensive – with all the “stuff” you need for your new friend, this can be a cost some people aren’t willing to bear. Vets, shelters and rescue groups sometimes discount their pricing when the animal is receiving other services, such as vaccines. Which begs the question, if vets want their patients to be microchipped, what’s the best way for them to price their services to make this important service more likely to be included?
It seems that pet microchipping would benefit from some pricing research. Beyond simply lowering the price, bundle offers may hold more appeal than a la carte. Then again, a single package price may be so high that it dissuades action altogether. Perhaps financing or staggered payments would help. And of course, discounts on other services, or on the service itself, may influence their decision. All of these possibilities could be addressed in a comprehensive pricing survey. We could use one of our pricing research tools, such as conjoint, to achieve a solid answer....
Is the Mini Cooper seen as an environmentally friendly car? What about Tesla as a luxury car? The traditional approach to understanding these questions is to conduct a survey among Mini and Tesla buyers (and perhaps non-buyers too, if budget allows). Such studies have been conducted for decades and often involve ratings of multiple attributes and brands. While certainly feasible, they can be expensive, time consuming and can get outdated over time. Is there a better way to get at attribute perceptions of brands that can be fast, economical and automated?
Aron Culotta and Jennifer Cutler describe such an approach in a recent issue of the INFORMS journal Marketing Science, and it involves the use of social media data – Twitter, in this case. Their method is novel because it does not use conventional (if one can use that term here) approaches to mining textual data, such as sentiment analysis or associative analysis. Sentiment analysis (social media monitoring) provides reports on positive and negative sentiments expressed online about a brand. In associative analysis, clustering and semantic networks are used to discover how product features or brands are perceptually clustered by consumers, often using data from online forums.
Breaking away from these approaches the authors use an innovative method to understand brand perceptions from online data. The key insight (drawn from well-established social science findings) is that proximity in a social network can be indicative of similarity. That is, understanding how closely brands are connected to exemplar organizations of certain attributes, it is possible to devise an affinity score that shows how highly a brand scores on a specific attribute. For example, when a Twitter user follows both Smart Car and Greenpeace, it likely indicates that Smart Car is seen as eco-friendly by that person. This does not have to be true for every such user, but at “big data” levels there is likely to be a strong enough association to extract signal from the noise.
What is unique about this approach to using social media data, is that it does not really depend on what people say online (as other approaches do). It only relies on who is following a brand while also following another (exemplar) organization. The strength of the social connection becomes a signal of the brand’s strength on a specific attribute. “Using social connections rather than text allows marketers to capture information from the silent majority of brand fans, who consume rather than create content,” says Jennifer Cutler, who teaches marketing at the Kellogg School of Management in Northwestern University.
Sounds great in theory, right? But how can we be sure that it produces meaningful results? By validating it with the trusted survey data that has been used for decades. When tested across 200+ brands in four sectors (Apparel, Cars, Food & Beverage, Personal Care) and three perceptual attributes (Eco-friendliness, Luxury, Nutrition), an average correlation of 0.72 shows that social connections can provide very good information on how brands are perceived. Unlike with survey data, this approach can be run continuously, at low cost with results being spit out in real time. And there is another advantage. “The use of social networks rather than text opens the door to measuring dimensions of brand image that are rarely discussed by consumers in online spaces,” says Professor Cutler....
The surprising result of the election has lots of people questioning the validity of polls…how could they have so consistently predicted a Clinton victory? Further, if the polls were wrong, how can we trust survey research to answer business questions? Ultimately even sophisticated techniques like discrete choice conjoint or max-diff rely upon these data so this is not an insignificant question.
As someone whose firm conducts thousands and thousands of surveys annually, I thought it made sense to offer my perspective. So here are five reasons that I think the polls were “wrong” and how I think that problem could impact our work.
1) People Don’t Know How to Read Results
Most polls had the race in the 2-5% range and the final tally had it nearly dead even (Secretary Clinton winning the popular vote by a slight margin). At the low end, this range is within the margin of error. At the high end, it is not far outside of it. Thus, even if everything else were perfect, we would expect that the election might well have been very close.
I always dread the inevitable "What do you do?" question. When you tell someone you are in market research you can typically expect a blank stare or a polite nod; so you must be prepared to offer further explanation. Oh, to be a doctor, lawyer or auto mechanic – no explanation necessary!
Of course, as researchers, we grapple with this issue daily, but it is not often we get to hear it played out on major news networks. After one of the debates, I heard Wolf Blitzer on CNN arguing (yes arguing) with one of the campaign strategists about why the online polls being quoted were not "real" scientific polls. Wolf's point was that because the Internet polls being referenced were from a self-selected sample their results were not representative of the population in question (likely voters). Of course, Wolf was correct, and it made me smile to hear this debated on national TV.
A week or so later I heard an even more, in-depth consideration of the same issue. The story was about how the race was breaking down in key swing states. The poll representative went through the results for key states one-by-one. When she discussed Nevada she raised a red flag as to interpreting the poll (which has one candidate ahead by 2 - % points). She further explained it is difficult to obtain a representative sample in Nevada due to a number of factors (odd work hours, transient population, large Spanish speaking population). Her point was that they try to mitigate these issues, but any results must be viewed with a caveat.
Aside from my personal delight that my day-to-day market research concerns are newsworthy, what is the take-away here? For me, it reinforces how important it is to do everything in our power to ensure that for each study our sample is representative. The advent of online data collection, the proliferation of cell phone use and do-it-yourself survey tools may have made the task more difficult, but no less important. When doing sophisticated conjoint, segmentation or max-diff studies, we need to keep in mind that they are only as good as the sample that feeds them.
In my previous blog, we determined that people with access to recycling services don’t necessarily recycle. And men were far less likely to recycle regularly than women.
One problem potential recyclers face is there is no federal standard for what is collected and how. Services vary from one contractor to the next. Items deemed recyclable in one municipality may not be the next town over. As a general rule, bottles, cans, and newspapers are curbside-recyclable. Also as a general rule, prescription drugs, electronic devices, CFL bulbs and batteries are not – they shouldn’t go in the trash either - they require special handling. But does the average consumer know this? We asked our online panelists who have access to recycling services how they believe their trash/recycling haulers would like them to handle certain items. And here’s what we learned:
Focusing solely on those who say they recycle, women are more likely than men to know what goes where…
Ladies, you may want to re-think having your gents handle the trash and recycling - or give them a quick lesson on what you've learned!
According to Economist.com, Americans aren’t doing a good job of recycling. There is actually a shortage of materials from recycling facilities that could be used to produce new products. The author posits that there are a variety of reasons for this, including simple access: “a quarter of Americans lack access to proper bins for collecting recyclable material, and another quarter go without any curbside recycling at all.” But I think it goes beyond access, and I surveyed our intrepid online panel of adult consumers to find out.
A little over a quarter (28%) of TRC’s panelists say they do not have residential recycling service. Increasing awareness and access for rental properties would certainly make a dent: renters are more likely to say they don’t have it (44%) than homeowners (19%).
But what if you are aware and have access? Does that mean you’re recycling? Not necessarily. Only 75% of those who could be recycling are doing so on a regular basis (usually or always). There’s no difference between renters and homeowners with recycling access as far as how often they recycle. But there is one key difference between those who regularly do and those who don’t: gender. Women are far more likely to say that they recycle regularly (84%) than men (62%). I’m not sure why there is a gender disparity, but we’ll explore how knowledgeable men and women are about recycling in my next blog.
I’ve become a huge fan of podcasts, downloading dozens every week and listening to them on the drive to and from work. The quantity and quality of material available is incredible. This week another podcast turned me on to eBay’s podcast “Open for Business”. Specifically the title of episode three “Price is Right” caught my ear.
While the episode was of more use to someone selling a consumer product than to someone selling professional services, I got a lot out of it.
First off, they highlighted their “Terapeak” product which offers free information culled from the massive data set of eBay buyers and sellers. For this episode they featured how you can use this to figure out how the market values products like yours. They used this to demonstrate the idea that you should not be pricing on a “cost plus” basis but rather on a “value” basis.
From there they talked about how positioning matters and gave a glimpse of a couple market research techniques for pricing. In one case, it seemed like they were using the Van Westendorp. The results indicated a range of prices that was far below where they wanted to price things. This led to a discussion of positioning (in this case, the product was an electronic picture frame which they hoped to be positioned not as a consumer electronic product but as home décor). The researchers here didn’t do anything to position the product and so consumers compared it to an iPad which led to the unfavorable view of pricing.
Finally, they talked to another researcher who indicated that she uses a simple “yes/no” technique…essentially “would you buy it for $XYZ?” She said that this matched the marketplace better than asking people to “name their price”.
Of the two methods cited I tend to go with the latter. Any reader of this blog knows that I favor questions that mimic the market place vs. asking strange questions that you wouldn’t consider in real life (what’s the most you would pay for this?”). Of course, there are a ton of choices that were not covered including conjoint analysis which I think is often the most effective means to set prices (see our White Paper - How to Conduct Pricing Research for more).
Still there was much that we as researchers can take from this. As noted, it is important to frame things properly. If the product will be sold in the home décor department, it is important to set the table along those lines and not allow the respondent to see it as something else. I have little doubt if the Van Westendorp questions were preceded by proper framing and messaging the results would have been different.
I also think the use of big data tools like Terapeak and Google analytics are something we should make more use of. Secondary research has never been easier! In the case of pricing research, knowing the range of prices being paid now can provide a good guide on what range of prices to include in, say, a Discrete Choice exercise. This is true even if the product has a new feature not currently available. Terapeak allows you to view prices over time so you can see the impact of the last big innovation, for example.
Overall, I commend eBay for their podcast. It is quite entertaining and provides a lot of useful information…especially for someone starting a new business.
Many researchers are by nature math geeks. We are comfortable with numbers and statistical methods like regression or max-diff. Some find the inclusion of fancy graphics as just being a distraction...just wasted space on the page that could be used to show more numbers! I've even heard infographics defined as "information lite". Surely top academics think differently!
No doubt if you asked top academics they might well tell you that they prefer to see the formulas and the numbers and not graphics. This is no different than respondents who tend to tell us that things like celebrity endorsements don't matter until we use an advanced method like discrete choice conjoint to prove otherwise.
Bill Howe and his colleagues at the University of Washington in Seattle, figured out a way to test the power of graphics without asking. They built an algorithm that could distinguish, with a high degree of success, between diagrams, equations, photographs and plots (bar charts for example) and tables. They then exposed the algorithm to 650,000 papers with over 10 Million figures in them.
For each paper they also calculated an Eigenfactor score (similar to what Google uses for search) to rate the importance of each paper (by looking at how often the paper is cited).
On average papers had 1 diagram for every three pages and 1.67 citations. Papers with more diagrams per page tended to get 2 extra citations for every additional diagram per page. So clearly, even among academics, diagrams seemed to increase the chances that the papers were read and the information was used.
Now we can of course say that this is "correlation" and not "causation" and that would be correct. It will take further research to truly validate the notion that graphics increase interest AND comprehension.
I'm not waiting for more research. These findings validate where the industry has been going. Clients are busy and their stakeholders are not as engaged as they might have been in the past. They don't care about the numbers or the formulas (by the way, formulas in academic papers reduced the frequency with which they were cited)...they care about what the data are telling them. If we can deliver those results in a clear graphical manner it saves them time, helps them internalize the results and because of that increases the likelihood that the results will be used.
So while graphics might not make us feel smart...they actually should.
We at TRC conduct a lot of marketing research projects using Conjoint Analysis. Conjoint is a very powerful tool for determining preferences for the various components that make up a product or service. The power of Conjoint comes from having consumers make mental trade-offs in evaluating products against each other. Do they prefer a lower cost product that contains few features, or a higher priced product that provides many benefits? How willing are they to choose a product that meets 2 or 3 of their criteria, but not all? Conjoint forces consumers to make these decisions, and the results can then be simulated to determine purchase preferences in a variety of scenarios.
But not all product development problems can be solved with Conjoint. Conjoint requires certain steps in the development cycle to have already been taken (defined features, some idea of pricing – see my previous blog on the topic.) In some cases, though, you may be at a stage in which Conjoint is feasible, but a different approach may be more appropriate, such as a Configurator. In a Configurator, otherwise known as a "Build-Your-Own" approach, you would use the same product features as in a Conjoint, but instead of pitting potential products against one another, the consumer "builds their own" ideal product.
So why choose one technique over the other? There are many reasons, but here are a few:
1. If determining overall product price sensitivity is the goal – Choose Conjoint. Conjoint will produce scores that assess both the importance of price overall as well as price tolerance for the product as features are included or excluded.
2. If you just want to know which features are the most popular, or which ones are selected when choosing or not choosing other features – Choose Configurator. In an a la carte scenario, respondents can choose which items to throw in their shopping cart and which ones to leave on the shelf. Getting simple counts on which features are popular and which ones are not – and in what combinations – can be very useful information, and it's an easier task for respondents. Keep in mind though that the Configurator works best if each feature is pre-assigned a price (to keep respondents from piling on).
3. If understanding competitive advantage/disadvantage is paramount – Choose Conjoint. Conjoint allows you to include "Brand" as a feature, and the results will link brand to the product price to see if respondents are willing to pay more (or less) for your product vs. that of a key competitor. You can also simulate competitive market scenarios. While you can include Brand in a Configurator, modeling the trade-off between brand and product price is far less robust.
4. If you have a lot of features, or complex relationships between the features - Choose Configurator. It's much easier for a respondent to sift through a long list of features and build their ideal product just once than to choose between products with a gigantic feature list multiple times. Conjoint works best when the features are not dependent on one another; a long list of restrictions on the features can disqualify Conjoint as a viable solution from a design perspective.
There are plenty of times when a technique may present itself as an obvious choice, and other times when the choice may be more subtle. And in those cases, we turn to our senior analysts who use their expertise and understanding of the research objectives to make sound recommendations.
At the beginning of my research career I grew accustomed to clients asking us for proposals using a methodology that they had pre-selected. In many cases, the client would send us the specs of the entire job, (this many completes, that length of survey) and just ask us for pricing. While this is certainly an efficient way for a client to compare bids across vendors, it didn’t allow for any discussion as to the appropriateness of the method being proposed.
Today most research clients are looking for their research suppliers to be more actively involved in formulating the research plan. That said, we are often asked to bid on a “conjoint study.” Our clients who’ve commissioned conjoint work in the past are usually knowledgeable about when a conjoint is appropriate, but sometimes there is a better method out there. And sometimes the product simply isn’t at the right place in the development “chain” to warrant conjoint.
Conjoint, for the uninitiated, is a useful research tool in product development. It is a choice-based method that allows participants to make choices between different products based on the product’s make-up. Each product comprises various features and levels within those features. What keeps respondents from choosing only products made up of the “best” features and levels is some type of constraint – usually price.
We look to conjoint to help determine an optimal or ideal product scenario, to help price a product given its features, or to suggest whether a client could charge a premium or require a discount. It has a wide range of uses, but it isn’t always a good fit:
When working with clients on parameters for a conjoint design, there is often an assumption that the design includes a current product configuration, or base case. This base case provides a benchmark against which new configurations can be compared.
Having a benchmark can be both useful and comforting when analyzing the conjoint results. Replicating a base case allows us to reference important metrics that are known for that product (for example, market share, CPU, revenue, etc.). As we configure new products and compare their appeal to our base case, we can gain insight into how these key metrics might be impacted.
Aside from establishing a benchmark, having a base case is also critical if there is concern about cannibalization. If the expectation for the new product is that it will compete in the market with a current configuration it is critical to understand what impact the new product will have on the current landscape.
However, allowing for a base case in the conjoint design is not always warranted. As products become more dissimilar from current offerings it can become difficult to include a base case. Trying to integrate the components of a current and new product that don’t share many characteristics can lead to conjoint parameters that are too complex to administer, or create apples to oranges comparisons. It is not wrong to leave out a base case as long as it is understood there will be no benchmark comparison.
One hybrid solution to consider is to allow for a set choice that reads something like “None of these, prefer the PRODUCTS currently available”. This is similar to a typical “none” option in the conjoint but provides a bit more information; specifically, that they would not leave the category but are not interested in the new, very different product configuration. Of course this solution would not be appropriate in all instances but does provide a good compromise.
Ultimately, the extent to which “real products” are modeled with a conjoint study’s parameters is a function of the specific information needs and the complexity of the design. Most of the time we want to include that dose of “reality” in our design but don’t be afraid to leave it behind if warranted.