Respondents define how your research will go. If you pick the wrong people or approach very few of them, you risk getting irrelevant outcomes. If you come to hasty conclusions, chances are that you’ll make a bad decision, lose time and money on a feature that no one wants.
You can’t just double check your research results. This won’t rule out mistakes. You need to learn to find respondents and develop a trusting relationship. That’s the only way to hear the truth, not just what you want to hear.
We made a survey to find out the most frequent research challenges before even writing this article 🙂 Finding respondents was our top priority. Of all product teams that we surveyed, 61% experienced difficulties in it.
During interviews, we asked the guys about the greatest challenges they had. Most of the time, it was difficult for them to find respondents that would help with a particular task. We think the opposite way: you should look for people with a goal that you can meet.
Sampling for quantitative research
The term speaks for itself. You may think that the more respondents, the better for surveys or product experiments. Not exactly.
You need to evaluate the universe before defining the required sample. How many people fit the parameters you set?
Let’s say the universe consists of male owners of IT companies. Derive a segment, for example, owners of B2B services, and use it as a sample for your research.
They are based on statistical theory and tell whether your results will be significant.
You can gradually increase the sample. For example, do research, release and improve the feature in several iterations, including MVP development, testing it on loyal users, and then expanding the sample depending on the feature objective.
Sampling for qualitative research
You may think qualitative research is easier because you need fewer respondents. However, you can’t calculate exactly how many of them you need.
In qualitative usability testing, there’s the classical “rule of five” by Jakob Nielsen. In a nutshell, it says that five respondents find 85% of interface flaws. The rule usually works, but if you apply it everywhere, it may affect your outcomes.
If you rarely do qualitative research, engage at least 10 respondents each time. This sample will help you identify more problems per one research iteration.
You need more respondents for customer development interviews — sometimes 20, sometimes 40, or even more. Segment audience and find two or three people with different experiences in each segment.
There’s the “saturation” concept that works for qualitative research. Your sample should be enough both to identify as many experience and perception configurations as possible and not to be too big (it’s counterproductive). If your respondents start repeating what you already heard, you should probably stop looking for new ones.
It’s the user’s experience that helps you test hypotheses, not the research type. If you wonder why a user bounced during checkout, you won’t find your answers among those who ordered. They managed to handle your interface flaws. You need the ones who see your website for the first time to meet your objectives.
You need to decide on your ideal respondent before inviting them to participate. These two questions can help:
- What are you expecting to find out in your research?
- What tasks do users perform with the existing product?
That’s how you define your selection criteria.
We hypothesized we could grow Dashly via EdTech. One of our jobs is the lead qualification, so we wanted to know:
• what makes EdTech businesses certain that their new lead matches their idea of a perfect customer;
• what challenges companies faced during lead qualification;
• how we can improve our product to cure their pains the best way.
Then, we needed to talk to people that fit. We made up a list of respondent traits not to waste time on pointless insights. We needed the people that knew everything about lead generation and qualification.
Role: СМО, marketing director, marketing manager. Sales: over three people in a team.
What makes the business special: active lead qualification.
Run a screening survey to filter out unfit people. Add questions that will help you find appropriate respondents.
Dashly’s experience: These are qualification questions we used to determine if people fit our EdTech research or not:
For a qualification survey, five to ten questions are enough. If you can’t limit your survey to ten questions, add the progress sidebar to make it easier for respondents to complete it.
If a survey is your research method, separate qualification questions and only show the remaining questions if a respondent answered questions from this section in an appropriate way.
And one more thing: don’t ask for too much sensitive and contact data. Respondents are not very generous about them. If you can’t do without it, ask for it at the very end not to scare your respondents. Make sure your questions are worded properly.
Download the lead qualification guide
Learn how to convert your website visitors into qualified leads
Thanks! Your guide is already in your inbox
Questions for respondent qualification
How should questions better be worded?
Checklist for respondent scouting
Experiment, test hypotheses, and enhance your product! We’ll be happy to accompany you along the way.