You thought you did everything right. You followed all the steps: you designed your questionnaire with care, successfully filled your study, and didn’t even have to replace any respondents. But somehow, your resulting consumer insights just aren’t all that, well, insightful. Maybe your questions remain unanswered, or all the data just seems totally irrelevant. It can be frustrating and demoralizing, and we want to make sure it never happens again! Below are a few of the most common reasons online market research goes wrong, including what you can do to fix it in the future.
1. Your sampling plan did not enable you to reach your target audience.
Yes, just about everyone is online these days. But no, that does not mean that you don’t need to have a well-defined target audience and an appropriate sampling plan. For instance, if your target audience is moms and you want to capture both working and stay-at home moms, you would need to allow the study to fill during the day and during the night to hopefully capture both types of moms. If you only field the study during the day, you will most likely underrepresent working moms, and skew your sample.
2. You focused on speed more than quality.
One of the largest benefits of online research is the speed at which you can reach respondents and conduct studies: what once took weeks to complete with phone interviews can now be completed in hours online. But when we execute studies at online speed, we need to ensure the right measures are in place to lead to quality results. Proper time needs to be taken to draft questions that fit an online format. The sampling plan needs to be thoughtfully considered and set up. There need to be quality measures put in place (such as captcha, straight-lining, and speeder terminates). And most importantly, though often forgotten, the analysis plan needs to be addressed prior to fielding the study. Just because online research is fast does not mean that it does not require the same careful planning and rigor that we have come to expect from traditional research.
3. You did not alter the research design.
Online research is not the same as in-person research. Both have their strengths and weaknesses, and knowing those will allow you to execute both in-person and online research better. If we took a discussion guide from an in-person focus group and used it for an online group, the resulting answers would be very different. For one thing, it is hard to change the conversation online in the same way that you would in person. If a respondent says something interesting online, you can follow-up; but most often, it’s not done in real time and you won’t get the same kind of conversation you would in person. Double or triple-barreled questions are even more dangerous online than they are in person, as online attention spans and communication preferences are shorter. Keep your questions focused, engaging, and non-repetitive, and design your study with the user in mind.
4. You are not analyzing the data correctly to account for the study being conducted online.
Words become even more important in online research than in-person. In-person research allows you to pay attention to tone and body language. But unless you have a video component*, those elements are missing from the analysis of online research—and even then, respondents have the chance to rehearse. Instead, you must pay closer attention to the words that are being said and the interaction between respondents. Reading that someone “dislikes” a product is different from someone saying they “hate” it. Keeping those nuances in mind will lead you to a more accurate, complete analysis.
Online market research is one of the handiest tools in an agile researcher’s toolbox, but only if they know how to properly design, execute, and interpret the studies. Hopefully these corrections will help resolve common research roadblocks, but if you’re ever in doubt, consider partnering with a research vendor who can understand and address your needs. To learn how the forward-thinking medical resource company Healthgrades incorporated online research into their latest mobile app development to ensure optimal adoption and usage, check out the case study below.
*Note: Online video uploads are a tool you can use to capture tone and body language, but there are best practices and things to consider for that methodology as well.