Some months ago, Lily Allen mistakenly received an email containing harsh test group feedback regarding her new album. Select audience members believed the singer to be retired and threw in some comments that I won’t quote. If you are curious, the link to her Popjustice interview will let you see them in a more raw form. Allen returned the favor with some criticism on market research itself:
“The thing is, people who take part in market research: are they really representative of the marketplace? Probably not.” –Lily Allen
The singer brings up a valid concern. One of the many questions I pondered five months ago when I first took my current researcher-in-training position with TRC. Researchers are responsible for engaging a representative sample and delivering insights. How do we uphold those standards to ensure quality? Now that I have put in some time and have a few projects under my belt, I have assembled a starter list to address those concerns:
In order to complete any research project, there needs to be a clear objective. What are we measuring? Are we using one of our streamlined products, such a Message Test Express™, or will there be a conjoint involved? This may seem obvious, but it is also critical. A team of people is behind each project at TRC; including account executives, research managers, project directors, and various data experts. More importantly, the client should also be on the same page and kept in the loop. Was the artist the main client for the research done? My best guess is no, the feedback given was not meant to be a tool to rework the album.
Was the research done on Lily Allen’s album even meant to be representative? Qualitative interviews can produce deep insights among a small, non-representative, group of people. This can be done as a starting point or a follow-up to a project, or even stand alone, depending on the project objectives.
In a recent project, select call center employees were interviewed individually. The in-depth responses were used to write a quantitative questionnaire that went out to a larger population. Qualitative interviews helped mold a future survey and gave the project direction.
Allen received the email containing test group feedback by accident. She may have lacked the full picture. Was she able to look through all results? Did they largely have a positive or negative connation? TRC provides context. Statistical differences are noted, base numbers are displayed and conclusions help guide clients through the data.
None of this is able to soften the blow of negative criticism. After some mixed reviews on her album, Lily Allen has gone back on her original sentiment. “Maybe the songs aren’t good enough this time, who knows?”
Ellen is interested in everything research-related, from exploring the benefits (or drawbacks) of various methods, to smart survey design and reporting results to different stakeholders. Outside of work she can be found cooking delicious (ahem experimental) dishes, pouring over the latest Game of Thrones fan theories and exploring her Philadelphia home.