Sample size calculator
What proportion of the target population do you need to interview to get results representative of the target population with the level of confidence that you are willing to accept?
Introduction to sample size
How can you calculate sample size, reduce the margin of error and produce surveys with statistically significant results? In this short guide, we explain how you can improve your surveys and showcase some of the tools and resources you can leverage in the process.
But first, when it comes to market research, how many people do you need to interview to get results representative of the target population with the level of confidence that you are willing to accept?
However, if all of this sounds new to you, let's start with what sample size is.
Free eBook: The complete guide to determining sample size
What is sample size?
Sample size is a term used in market research to define the number of subjects included in a survey, study, or experiment. In surveys with large populations, sample size is incredibly important. The reason for this is because it's unrealistic to get answers or results from everyone - instead, you can take a random sample of individuals that represent the population as a whole.
For example, we might want to compare the performance of long-distance runners that eat Weetabix for breakfast versus those who don't. Since it's impossible to track the dietary habits of every long-distance runner across the globe, we would have to focus on a segment of the survey population. This might mean selecting 1,000 runners for the study.
How can sample size influence results?
That said, no matter how diligent we are with our selection, there will always be some margin of error (also referred to as confidence interval) in the study results, that's because we can't speak to every long-distance runner or be confident of how Weetabix influences (in every possible scenario), the performance of long-distance runners. This is known as a "sampling error."
Larger sample sizes will help to mitigate the margin of error, helping to provide more statistically significant and meaningful results. In other words, a more accurate picture of how eating Weetabix can influence the performance of long-distance runners.
So what do you need to know when calculating the minimum sample size needed for a research project?
What you need to know to calculate survey sample size
Confidence interval (or margin of error)
The confidence interval is the plus-or-minus figure that represents the accuracy of the reported. Consider the following example:
A Canadian national sample showed "Who Canadians spend their money on for Mother's Day." Eighty-two percent of Canadians expect to buy gifts for their mom, compared to 20 percent for their wife and 15 percent for their mother-in-law. In terms of spending, Canadians expect to spend $93 on their wife this Mother's Day versus $58 on their mother. The national findings are accurate, plus or minus 2.75 percent, 19 times out of 20.
For example, if you use a confidence interval of 2.75 and 82% percent of your sample indicates they will "buy a gift for mom" you can be "confident (95% or 99%)" that if you had asked the question to ALL CANADIANS, somewhere between 79.25% (82%-2.75%) and 84.75% (82%+2.75%) would have picked that answer.
Confidence interval is also called the "margin of error." Are you needing to understand how the two calculations correlate?
Confidence level
The confidence level tells you how confident you are of this result. It is expressed as a percentage of times that different samples (if repeated samples were drawn) would produce this result. The 95% confidence level means that 19 times out of twenty that results would fall in this - + interval confidence interval. The 95% confidence level is the most commonly used.
When you put the confidence level and the confidence interval together, you can say that you are 95% (19 out of 20) sure that the true percentage of the population that will "buy a gift for mom" is between 79.25% and 84.75%.
Wider confidence intervals increase the certainty that the true answer is within the range specified. These wider confidence intervals come from smaller sample sizes. When the costs of an error is extremely high (a multi-million dollar decision is at stake) the confidence interval should be kept small. This can be done by increasing the sample size.
Population size
Population size is the total amount of people in the group you're trying to study. If you were taking a random sample of people across the U.K., then your population size would be just over 68 million (as of 09 August 2021).
Standard deviation
This refers to how much individual responses will vary between each other and the mean. If there's a low standard deviation, scores will be clustered near the mean with minimal variation. A higher standard deviation means that when plotted on a graph, responses will be more spread out.
Standard deviation is expressed as a decimal, and 0.5 is considered a "good" standard deviation to set to ensure a sample size that represents the population.
How can you calculate sample size?
After you've considered the four above variables, you should have everything required to calculate your sample size.
However, if you don't know your population size, you can still calculate your sample size. To do this, you need two pieces of information: a z-score and the sample size formula.
What is a z-score?
A z-score is simply the numerical representation of your desired confidence level. It tells you how many standard deviations from the mean your score is.
The most common percentages are 90%, 95%, and 99%.
z = (x – μ) / σ
As the formula shows, the z-score is simply the raw score minus the population mean and divided by the population's standard deviation.
Using a sample size calculation
Once you have your z-score, you can fill out your sample size formula, which is:
Is there an easier way to calculate sample size?
If you want an easier option, Qualtrics offers an online sample size calculator that can help you determine your ideal survey sample size in seconds. Just put in the confidence level, population size, margin of error, and the perfect sample size is calculated for you.
Best-practice tips for sample size
There are lots of variables to consider when it comes to generating a specific sample size. That said, there are a few best-practice tips (or rules) to ensure you get the best possible results:
1) Balance cost and confidence level
To increase confidence level or reduce the margin of error, you have to increase your sample size. Larger sizes almost invariably lead to higher costs. Take the time to consider what results you want from your surveys and what role it plays in your overall campaigns.
2) You don't always need statistically significant results
Depending on your target audience, you may not be able to get enough responses (or a large enough sample size) to achieve "statistically significant" results.
If it's for your own research and not a wider study, it might not be that much of a problem, but remember that any feedback you get from your surveys is important. It might not be statistically significant, but it can aid your activities going forward.
Ultimately, you should treat this on a case-by-case basis. Survey samples can still give you valuable answers without having sample sizes that represent the general population.
But more on this in the section below.
3) Ask open-ended questions
Yes and no questions provide certainty, but open-ended questions provide insights you would have otherwise not received. To get the best results, it's worth having a mix of closed and open-ended questions.
For a deeper dive into survey question types, check out our handbook.
Different types of surveys
From market research to customer satisfaction, there are plenty of different surveys that you can carry out to get the information you need, corresponding with your sample size.
The great thing about what we do at Qualtrics is that we offer a comprehensive collection of pre-made, customer, product, employee, and brand survey templates. This includes Net Promoter Score (NPS) surveys, manager feedback surveys, customer service surveys, and more.
The best part? You can access all of these templates for free. Each one is designed by our specialist team of subject matter experts and researchers so you can be sure that our best-practice question choices and clear designs will get more engagement and better quality data.
As well as offering free survey templates, you can check out our free survey builder. Trusted by over 11,000 brands and 99 of the top 100 business schools, our tool allows you to create, distribute and analyze surveys to find customer, employee, brand, product, and market research insights.
Drag-and-drop functionality means anyone can use it, and wherever you need to gather and analyze data, our platform can help.
What next?
Once you have determined your sample size, you’re ready for the next step in the research journey. market research.
Market research is the process of gathering information about consumers' needs and preferences, and it can provide incredible insights that help elevate your business (or your customers') to the next level.
If you want to learn more, we've got you covered. Just download our free guide and find out how you can:
- Identify use cases for market research
- Create and deliver effective market research campaigns
- Take action on research findings
