As a consultant, your clients are looking for you to deliver timely and useful insights that will help them make informed business decisions. Surveys that are well written, designed, and programmed will help you develop strong research-driven recommendations.
Good surveys can be challenging to create, and even experts face issues that can lead to poor survey data. Whether you’re putting together your first survey or your hundredth, here are some of the common pitfalls to watch out for – and what to do to avoid them.
Faulty Questions
Believe it or not, it’s possible for survey respondents to provide wrong answers, which can be disastrous when you analyze the data to develop conclusions and recommendations. This issue often happens when a question is ambiguous, confusing, or misleading.
Here are types of faulty questions to avoid:
- Poorly defined scales: If you ask, “How likely are you to recommend this product to a friend?” and then provide a scale from one to five, the respondent won’t know what each number in the scale represents. This confusion can be avoided by using a scale such as “1 = not at all likely” and “5 = very likely.”
- Packing two questions into one: Asking “Was it easy to find the website and place an order?” should be separated into two questions. Some respondents might not know how to answer correctly if they did find the website but did not place an order.
- Loaded questions: Biased questions, such as “How easy was it to place your order?” lead to skewed responses. A better question would be: “How would you rate your experience in placing your order?”
- Jargon: Avoid terms that could be unfamiliar to the survey respondent. The respondent might answer incorrectly because they didn’t know the terminology or believed it meant something different.
Lack of Representation
When the people you survey don’t accurately represent the entire group of people you’re trying to study, your responses will be skewed and may hinder your ability to gain meaningful information from the data collected. Consider, for example, that you need to survey what drugs medical professionals prescribe for certain conditions. If your sample pool only reaches doctors and omits nurse practitioners, the resulting data will likely be incomplete.
To avoid a lack of representation, make sure you understand what population you need to reach for your survey and make this clear to your survey partner and panel providers. Also, use screener questions to ensure that only people with the correct demographic and firmographic characteristics are taking the survey. (Download our Consultant's Guide to Diligence Surveys for a comprehensive guide to surveys, including screener questions.)
Nonresponses
Like lack of representation, nonresponses can prevent you from obtaining useful data because qualified, experienced candidates are declining to take it or abandoning it – typically because they are frustrated because there are no clear answer choices that apply to them. (This frustration can also lead to respondents rushing through the survey to reach the end, which would count as a response but introduces noise into the data.)
You can reduce the possibility of nonresponses as you design and program the survey by using survey logic to customize for different respondent types, which can cut abandonment rates by helping reduce survey length and improving respondent engagement. Again, you should discuss these issues with your survey partner and panel providers.
Your survey partner (or panel providers) can also help you design a strategy to minimize nonresponses. They may recommend increasing the population size (called the N) based on several factors, such as confidence levels and margins of error.
