Calculation of a Sample Size

"If you don't believe in random sampling, next time you go to the doctor for a blood test, have him take it all.
And realize that the doctor will use the same test tube size for both a human and an elephant."

• A brief introduction
• Calculation of the sample size
• Further considerations

Introduction

Sampling is at the heart of marketing research. But it comes at a price. You can never be 100% sure that your sample statistics show the exact values of the population parameters. Actually, you can be quite sure that your results will be a little off from the true population value you want to estimate. The important question is: How far off is your sample result?
The two key concepts in this context are reliability and accuracy.

Reliability deals with how confident we are that the conclusions based on our sample are correct.
A reliable statement: Tomorrow the maximum temperature in Arnhem will be between -15 ºC and 45 ºC.
I am very confident that this statement is true. But it is of little value since the margins are huge.

Accuracy deals with the distance between the minimum and the maximum value that I report in my estimate.
An accurate statement: Tomorrow the maximum temperature in Arnhem will be between 11 ºC and 13 ºC. 
This statement predicts the temperature with a small margin of error.

In general there is a trade-off between accuracy and reliability. If you want to improve both or one of them while the other remains the same, you have to pay a price. That is, you have to increase your sample size.

Calculation of the sample size

In our research design one of the key questions is how many respondents we need in order for our results to be both reliable and accurate.
The calculations to answer this question in case our requirements are referring to confidence intervals for population proportions can be done using an Excel sheet that we use in our statistics classes at Arnhem Business School: estimating population proportions (Excel, 24 KB).

This spreadsheet has four sheets. Two of them calculate confidence intervals for given values of the sample proportion, sample size and level of confidence. One deals with the large population situation, the other with the small population situation.

The other two sheets calculate the number of respondents you need such that for the proportions found by your research a certain level of confidence and a certain maximum margin of error are guaranteed. Again we make a distinction between the large population and small population situation.

Further considerations

• In calculating the sample size you have to choose which accuracy and reliability you want.
  In marketing research a confidence level of 95% or 90% is often used. As margin of error for proportions a lot of people aim at confidence intervals with p ± 5%.

• The resulting value of n is the number of respondents you need. Of course that is not equal to the number of surveys you have to plan and send out. There is always nonresponse. If the nonresponse is expected to be considerable then the number of questionnaires to send out will be a lot higher than the value of n given by the spreadsheet calculations.

• Research with a target population that has a relatively close bond to the research topic or to the company performing the survey can expect a response rate of around 25%.
  In marketing research response rates higher than 25% are quite rare.

• If the target population is less well defined and has little affection for the survey topcis response rates around 15% are already quite nice.

• When you are dealing with a small population (let us say N = 1000) and you want a confidence level of 95% plus a margin of error of ±5%, you need 278 respondents.
  However, with a response rate of 25% you can't expect more than 250 respondents.
  This means that in such a situation you will choose a census (send the questionnaire to everyone) and you pay special attention to the correctness of your sampling frame. And then you hope for the best.