What makes a statistically valid survey

Half of those responded. Conversely, if we have an accuracy goal for the survey project, we can use this chart to determine the number of responses needed.

Find those coordinates on the chart. By applying an estimate of our response rate, we can then determine the number of survey invitations we must send out, which is our sample size. When we actually conduct our survey and analyze the results, we will then know something about the variance in the responses. The confidence statistic incorporates the variance found in each survey question and can be calculated for each survey question.

Technically, the interval tells us where the mean of repeated survey samples would fall. Look at the above chart. It shows the calculation of the confidence statistic using Excel. The standard deviation is square root of the variance , and can be calculated using Excel.

Size is the number of responses. In this example, the mean for the survey question was 3. Our accuracy is 0. As a very rough rule of thumb, responses will provide fairly good survey accuracy under most assumptions and parameters of a survey project.

Everyone conducting a survey is concerned about response rates and the accuracy for their survey results. This is one part of a survey project that does require some fundamental understanding of statistics. In our Survey Design Workshop , we spend considerable time on this topic. Request Chapter 1 of the new edition of our Survey Guidebook for key points in a more effective survey program.

We assist clients with their survey projects ranging from a self-help guidebook, to targeted assistance, to full service solutions. Is that right? I groaned… If only life — and statistics — were so simple. Fill out my online form. The bigger the population, the smaller the response rate percentage needs to be — but it not linear.

Segmentation analysis desired. Typically, we analyze the data along some demographic segmentations, for example, region, annual sales volume, or support representative. If the critical decisions will focus on the analysis of these segments, then the statistical accuracy needs to be calculated for these smaller segments and not just for the population.

Degree of variance in responses. Polco Basics. Using the Polco Platform. Building Great Questions. Engagement and Outreach Success. Best Practices. Deep Dives. Staff and Council resources. New Features and Notes. Polco Pulse. If the variance is large, you would continue to survey for more statistical significance.

Final Thoughts Now that you know what makes your survey statistically significant, you want to know how many people to invite to your survey. If you know your expected response rate, you can decide how big of a population to survey. Image: Mauro Mora. This article was tagged survey , survey questions , survey response , survey tips.

Related Articles. We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it. Ok Read more.