Surveys are a powerful tool which outputs qualitative and quantitative data at scale. They are fairly easy to execute, but at the same time one of the most difficult methods to execute correctly.
Surveys are not unique to UX; they are used across many disciplines. For UX Research, we use surveys to collect data from users (or future users) on their demographics, attitudes, opinions, needs, and behaviors.
We can classify the types of survey questions into two groups:
It’s sometimes useful to include both quantitative and qualitative survey questions within a single research effort. For example, you can start with qualitative surveys to establish an understanding of the scope of the topic and inform survey question choices to use within a quantitative survey. Or, you can use a quantitative survey question and then follow up with an open-ended qualitative survey question to understand why they selected their answer.
When determining which research method to use, you should start with your research questions and envision how those can be answered, given the available research methods.
Perhaps the most important item to consider when determining if you want to use a survey: if your research questions require open-ended questions to answer them, then using a survey probably isn’t the best method for you. That’s because too many open-ended questions lead to a variation in responses and often a longer survey. Sometimes respondents don’t give enough depth or breadth and get fatigued with open-ended questions. Also, this structure of survey can be time intensive to analyze. If your research questions don’t require open-ended questions, then a survey could be a good method for you.
| Quantitative research questions | Qualitative open-ended research questions |
|---|---|
| How satisfied are GitLab users with a given feature? | Why are users either satisfied or unsatisfied with a given feature? |
| Which DevOps workflow is the most common? | Why are different DevOps workflows challenging? |
| What feature is the most popular? | How will different features impact user workflows? Why? |
Additionally, a you might use a survey if:
If you’re unsure if a survey is right for your needs, it’s suggested that you get crisp on your research questions to understand what kind of data is needed to answer those questions. That exercise will often make it easier to reveal the best research method to use.
A survey comes with pros and cons, which you should consider before committing to the method.
Pros:
Cons:
Surveys rely on self-reported data, which is important to note. As a guideline, if we already have that information, we shouldn’t ask it; as it’s a tax on the respondent to answer those survey questions and can be less accurate. Examples of information we may already have are: license type, user tenure, or company size. Additionally, respondents can also make mistakes or possibly lie, making the responses less reliable. In cases when we have that information, we should utilize it. Or, if we don’t have that information, and want to avoid self-reported data, we should be using a different research method to obtain it (example: a user interview where you’re observing behaviors).
If you’ve decided that a survey is the right method to use, you can follow the steps below to conduct a survey:
