As a part of the planning process, we recently conducted a Statistically-Valid Community Preference Survey. This allows us to get an accurate sense of resident's vision for the future of Mapleton. Only approximately 400 responses were required for statistical-validity in Mapleton, therefore, not all residents were asked to take it. The statistically-valid survey was followed by an Open-Access Survey which was open to all residents to take.
Further details regarding the Statistically-Valid Survey follow:
Sample Design: Mapleton’s estimated adult population according to the US Census Bureau’s American Community Survey is approximately 6,560. The Utah registered voter file for the City of Mapleton includes 5,985 voters. Based on the high rate of voter registration in Mapleton, our sampling pool for this survey will be the state’s registered voter file. Because each resident on file has an equal probability of being selected into the survey sample, we can avoid self-selection, geographic, and other biases and gather a sample that reflects, within the margin of error, the population of the City as a whole. If any of the sampled residents occupied the same household, one of the duplicate household representatives is randomly selected to be kept in the records. All of this is done mechanically, that is to say with minimal human interaction with our data processing software, to avoid introducing any preventable bias. If we receive lower than anticipated response rates from some geographic or demographic subgroup of residents (e.g., females under age 50), we can send targeted reminder emails or additional survey invitations to help ensure that the makeup of our eventual set of survey responses reflects the makeup of the City at large–albeit on a smaller scale.
Sample Size and Margin of Error: In total, we anticipate collecting 400 completed surveys from Mapleton residents. The margin of error for a random sample size of 400 in a city the size of Mapleton is about + or – 4.75 percentage points with a 95% confidence interval. This means that in 95 out of every 100 samples of the same size and type, the results we obtain would vary by no more than plus or minus 4.75 percentage points from the result we would get if we could interview every member of the Mapleton population. Thus, the chances are very high (95 out of 100) that any sample we draw will be within 5 points of the true Mapleton population value. Since the average response rate is around ten-percent, we anticipate sampling approximately 4,000 Mapleton residents in order to receive 400 completed surveys.
Data Weighting: The survey data we collect will be statistically adjusted using post-stratification weighting to correct for potential nonresponse error and potential coverage error. These types of sampling error result from collecting data from some, rather than all, members of the population. Nonresponse error would occur if the non-respondents from our sampling pool (those registered voters whose names matched a valid email address or who received a printed & mailed survey invitation) differed significantly from our actual respondents on their responses to the survey items. Coverage error would occur if the group that accepted the invitation to take the online survey differed significantly from the group that we are really trying to survey and draw inferences about (all Mapleton residents). Utah has very high computer ownership and internet access rates among the general population. These rates are likely to be even higher among voters, increasing the likelihood that our sample avoids coverage error as much as possible. It is impossible to be absolutely certain about the existence of nonresponse and coverage error but weighting is assumed to minimize the possibility of such error.
Our weighting procedures take the population distributions for age, gender, home ownership, and income from the American Community Survey and statistically adjusted the data for over- or under-represented groups. Weighting is an iterative process by which the distributions for all of the variables of interest are adjusted until they closely reflect their respective targets. When the weights are applied, the categories for age, gender, home ownership, and income are within five percentage points of their actual population values. At the end of the weighting process, the weights are trimmed at the 1st and 99th percentiles to prevent outliers from skewing the results.