In a perfect world, we’d sample every single user within the population. However, we can’t do that. Why? It’s inefficient, expensive, time-consuming, and probably impossible to pull off. Instead, we try to attain a representative sample of the population from the population we want to focus on. To do this, we take a slice of that population, referred to as our ‘sample’.
Furthermore, we compute measures from the sample and use them as estimates of the measures for the entire population. For example, we can’t measure every users’ SUS score, even though that true SUS score does exist. Instead, we compute the SUS score from a sample of users. While it’s not going to be the exact SUS score for the population, it’s close within some degree of error (or margin of error).
One of the most common questions around sampling is ‘how many people should we sample?’ The correct answer is: ‘It depends’. There are several factors to consider when determining how large your sample should be. The main factors we look at are: confidence/accuracy, population, and the type of analysis. Further, not all studies need a high level of accuracy, but the information is still very valuable/useful, allowing you to make informed decisions. Here’s what we mean:
To help make it easier to determine sample sizes, here are some guidelines:
These are general guidelines, but the size will vary depending on your study and the level of accuracy needed. You can use a sample size calculator, but you will need to know your population size.
Not every result needs to be statistically significant. You can sometimes gather enough responses to have meaningful inferences while not achieving statistical significance. What’s more important is to have a representative sample of the population of interest.