How many should I survey?
When you perform a survey you want the result to provide enough precision for your requirements and to be able to make statements or inferences from the results.
If you want to make inferences about a population then the design of the sampling process is important so that the results represent your population the best you can. Typically this is based on a random selection process. (Non-random sampling processes will have implications on how the results are interpreted.)
Sample size and precision of your results are intricately linked. The precision is dependent on the type of question and the results themselves. A useful and easy way to obtain measure of precision is based on confidence intervals of a percentage (for example the percentage of yes for a question).
Confidence intervals provide a measure of error or precision around a result. Typically 95% confidence intervals are used, meaning that we are 95% sure that the population value lies in this range (assuming the results are unbiased).
Here are some sample sizes and the corresponding 95% confidence interval that could be expected for a percentage result.
| Number people surveyed | 95% confidence interval |
|---|---|
| 5 | ±41% |
| 11 | ±30% |
| 17 | ±25% |
| 26 | ±20% |
| 33 | ±18% |
| 41 | ±16% |
| 54 | ±14% |
| 73 | ±12% |
| 100 | ±10% |
| 160 | ±8% |
| 281 | ±6% |
| 623 | ±4% |
| 1097 | ±3% |
| 2448 | ±2% |
The result assumes a very large population and is based on getting a 50% ‘Yes’ result, which gives the broadest confidence band for the range of potential results. Note that with smaller, known populations, the same sample size should give you a greater precision.
This means simply that if you randomly sampled 100 people within your population and you found that 50% of the respondents said ‘Yes’ to a Yes/No question , you could be 95% sure that the result for the whole population lies between 40% saying ‘Yes’ and 60% saying ‘Yes’ – ie +/- 10% from the sampled result.
As you sample more people the range reduces – but with a diminishing rate of increased precision. So if you sample 160 people in this example, your range for the population moves to between 42% and 58% – or +/- 8%. To reduce the range by another +/- 2% you need to sample a total of 281.
The decision you need to make is what level of precision you need – or are prepared to pay for – given the purpose of your survey and statements or decisions you wish to make based on the results. We would be happy to discuss this further with you.
Further resources on sample size can be found here.