A Well-Written Survey Can Help a Rec-Center Project Where it Matters Most - at the Ballot Box
Have you ever experienced the misfortune of having the voters of your community soundly defeat your proposal for a new community or family aquatic center. Has your recreation-center planning committee failed in its attempts to build consensus and get elected officials to support the project. Have you been frustrated in your attempts to estimate the market for potential users of a new facility.
The solution, in all three cases, is a reliably written, statistically valid survey. Surveys are a highly accurate tool that decision-makers such as mayors, city council members, city managers, nonprofit boards and others can use to gauge support for their vision and thus bring the project a step closer to being supported by other government officials and the electorate.
Statistically valid surveys are market research instruments that allow you to gain highly accurate information regarding the attitudes, opinions and priorities of your community by gathering responses from a much smaller random sampling of residents. A sampling of only 400 households can provide responses on a wide range of feasibility-study issues with a confidence of 95 percent and a margin of error of only plus or minus 5 percent.
Generally, you want to receive at least 300 completed surveys as part of a feasibility study or master-planning effort. Between 300 and 400 is the minimum for a community with a population of fewer than 40,000. For mid-size communities (up to 100,000 residents), 500 to 600 surveys is a good target, while communities larger than 100,000 residents should aim for between 700 and 1,000 (or more) surveys. Statistically valid surveys can be conducted by mail or phone - or by mail and phone. Phone surveys can be completed in as little as several weeks, with mail or combination surveys taking longer.
The main drawback of conducting surveys is that, to the uninitiated, surveys seem like too simple a process. Without keen attention paid to a variety of details, however, the data you collect can be irretrievably tainted.
The following are 15 details that will ensure your survey is read and understood, and that the results you get are statistically valid:
1. KEEP SURVEYS SHORT. Well-written short surveys will normally result in higher response rates than longer surveys. Ideally, a respondent should be able to complete a survey in no more than 10 to 12 minutes, regardless of whether it is a mail or phone survey. If you are doing a mail survey, this equates to five or six pages - or between 22 and 30 questions, depending on the length of each question. This is more than enough room to ask and have answered questions on a wide range of issues impacting the feasibility of your project.
2. CHOOSE QUALITY GRAPHICS. While it is important that your survey run no longer than six pages, this should not be accomplished by stuffing too many questions onto each page. An unattractive survey can dramatically reduce your response rate.
Some keys to making your survey graphically attractive include: • Print size. You should always use 12point type, with 1 inch of character spacing between lines of type. Many respondents, especially seniors, have difficulty reading small print. Be sure to CAPITALIZE instructions that are particularly important. • Paper color. Surveys on lightly colored paper such as yellow are more pleasing to the eye and should be used whenever possible. Avoid using dark colors to conduct surveys, as they are harder to read and print. • Format. Avoid using staples to hold pages together. Instead, have the survey professionally printed so that it is in booklet format. This looks more professional to the reader and will help increase your response rate.
3. INCLUDE A COVER LETTER. Always include a cover letter with mail surveys, briefly explaining the survey issues and emphasizing the importance of receiving the respondents' opinions and priorities. The letter should be no longer than three to four paragraphs and issued on agency stationary. Have the letter signed by a highly visible member of your community or organization, such as the mayor or president of your board. This will lend an air of importance to the project and increase your response rate.
4. IDENTIFY THE TOP ISSUES PRIOR TO WRITING THE SURVEY. It sounds overly obvious, but a survey can only provide answers to questions that are asked. It is vital that your survey ask questions for the full range of issues that will impact the success of the project. For example, key issues for a community center feasibility study would normally include: • Programming spaces: The types of spaces to be included in the facility - gymnasiums, fitness space, indoor pool, meeting rooms, steam and sauna rooms, teen areas, senior citizen areas and so on. • Locations: The potential sites that the community would support for building the community or family aquatic center. • Programming: The types of activities in which users would participate, such as recreational and/or competitive swimming, exercise programs, lessons and others. • Usage: How often respondents would use the facility being planned. • Fees: The types of fees users would support and what their price tolerance would be for paying fees. • Voter support: The support your project would receive from voters in a property-tax, sales-tax or other type of election.
5. AVOID YES OR NO QUESTIONS. With very few exceptions, questions that allow the respondent only two choices of response can seriously impact the quality of information obtained from a survey. This is particularly true when you are asking people for their opinions, since respondents often have not made up their minds about certain issues.
For most questions, you should give the respondent three to five options. At a minimum, you should always allow a "don't know" option for opinion questions. A better range of options for an opinion question is "very supportive," "somewhat supportive," "not supportive" and "not sure." Often, those who are "not sure" will mark "no" if they are forced to make a yes-or-no decision.
As Figure 1 (p. 80) indicates, yes/no only questions limit the information you can learn in comparison with questions that allow the respondent a broader range of options. Based only on the graph on the left, the parks and recreation agency would conclude that 48 percent of the residents do not think there is a need for a new community center. However, the graph on the right shows that only 22 percent of respondents believe there is not a need, while 26 percent are "not sure." This graph clearly shows that nearly twice as many respondents would definitely support the community center issue than would not support it; the data also gives you the opportunity to show that a verifiable number of "not sures" might be converted into supporters by the time of the election.
6. USE WORDS THAT ARE EASY TO READ AND UNDERSTAND. A survey is not a master's thesis. Quite the opposite: A good rule of thumb is to write the survey using language at about a sixth-grade reading level. Stay away from longer words like assessments or adolescents when shorter words like tax or teens mean the same thing. Most word-processing systems include dictionaries that can help find the best words to use.
If you are using a term that may not be understood in your community, explain it in words that are more familiar. For example, many citizens have never heard the term family aquatic center and may not know the difference between this type of water attraction and a traditional pool. If you ask their opinion regarding how much they would support development of "a family aquatic center," their responses may not give you the level of detail you require.
Instead, use the question to educate the respondents on potential features that could be included in the design of an aquatic center, and gain an understanding of which features they support the most.
7. ORDER YOUR SURVEY QUESTIONS PROPERLY. A survey needs to gather information from respondents, but it is often necessary to educate respondents in order to gather the most informed and reliable answers. Proper ordering can help achieve this. For example, at the beginning, draw up a list of activities (youth sports, arts and crafts, dance classes, exercise classes) and ask respondents to check the ones they would support. This way, respondents have an opportunity to learn the types of programming spaces they might use before considering what these spaces might cost them. Questions regarding the types of usage fees respondents would most prefer to pay and the amounts they would be willing to pay should be asked in the middle of a survey after programming questions. The results of these types of questions (shown in Figure 2) can be used by economists to help project operating costs for the new facility. Finally, questions regarding voter support should always be asked at the end of a survey, after the respondent has a better understanding of the facility and its costs.
8. WRITE SHORT AND CLEAR QUESTION EXPLANATIONS. Giving explanations is important, but it needs to be done in as few words as possible. An explanation should concisely tell what the question is about and then tell respondents what they need to do to accurately respond. Many survey questions can be effectively written in five to 10 words and seldom should any run more than 30 words. Here is an example from a telephone survey: "I am going to read some programs that could be held in a new community center. Please tell me whether you think each program is very important, somewhat important, not important, or whether you don't know."
9. AVOID POTENTIALLY BIASED QUESTION EXPLANATIONS. There will always be community members who are strongly opposed to or in favor of a project before a feasibility study has been conducted. If those who are opposed to the project believe that your survey is biased, they can discount the responses you receive and put the project in jeopardy. Here's an example of language that would be considered biased: "There is a tremendous shortage of space for youngsters to play basketball in our community. In light of this fact, please tell me your level of support for building new gymnasiums for public recreation in our community."
This question is biased for two reasons. First, it provides an opinion - "there is a tremendous shortage of space" - without giving any backup information. Second, it states that the opinion is a "fact." In addition, the introduction is not clear about how the respondent is to answer the question.
Experienced market-research firms are skilled at writing non-biased questions and can be very useful in this regard. If you are doing the survey yourself, a good way to test your survey is to share an early draft of your survey with higher-up decision-makers, such as the city manager or university president. Sharing your questions with them will provide good feedback and also instill confidence in them that the survey is highly objective and fair.
10. WRITE QUESTIONS THAT MEASURE PRIORITIES. Surveys are excellent vehicles for helping give shape to the new facility. By understanding issues of priority importance, designers and economists can work with clients to develop facilities that best meet the needs of the community, which is vital to gaining support.(See Figure 3.)
11. SECURE USEFUL DEMOGRAPHIC INFORMATION. A statistically valid survey is of special value because the attitudes, opinions and priorities expressed by the respondents represent the opinions of your entire market. Demographic information can only be gleaned from the survey data if care is taken at the outset to categorize the survey respondents (and if a sufficient quantity of responses are returned). Important categories include: • Gender (male or female) • Household size (one to more than five persons) • Location (zip code) • Years living in community (less than five to more than 30) • Age of respondent (18 years or younger to 50 years or older) • Annual income (less than $15,000 to more than $100,000)
It is essential that the market-research component of your survey be conducted properly, or the results of the survey will not be statistically reliable. If you are doing the survey in-house, you need to be prepared to show decision-makers and voters that the information is reliable and accurate.
12. SURVEY THOSE WHO WILL FUND THE PROJECT. If a project hopes to be funded through a city- or county-wide tax, it is absolutely essential to accurately survey a representative sample of all households within the community who will be voting on the project. You may also want to survey residents of neighboring communities if their usage is important to the success of your project.
Surveying those who will be asked to fund the project gives you the best information regarding what specific programming spaces, locations and so on will be supported at the polls. Additionally, elected officials place more weight and confidence on survey results that are representative of all citizens. A useful question to include within the survey is, "If a bond issue were proposed at a future election, and the funds from the tax were used to develop a new community recreation center, which of the following best describes how you would most likely vote." Answer choices should include "would vote in favor," "might vote in favor," "would vote against" and "not sure."
13. USE CROSS-TABS AND BENCHMARKING FOR ADDITIONAL INFORMATION. While your survey may only have 25 to 30 questions, you can gain a vast amount of additional information to support your project through the use of cross-tabs and national benchmarking.
Cross-tabbing is the comparison of answers to two or more questions to gain further insight into the respondents' attitudes, opinions and priorities. Crosstabbing can be conducted to compare the opinions of men and women, households of different sizes, and preferences for features by different potential voter groups. It's important to have a software program that allows you to do cross-tabbing in a fast and affordable manner, and a staff person or researcher who is proficient in interpreting it.
Figure 4 (p. 84) is an example of how cross-tabbing your voters can be of great assistance in your feasibility-study efforts. In this instance, a cross-tab was performed to understand which feature of a proposed family aquatic center was most important to those who either "would" or "might" vote in favor of a new facility. As you can see, lanes for lap swimming and a zero-depth-entry pool were the first choices for those who would vote in favor, while water slides and an area for swim lessons were the most important swimming features for those who might vote in favor. Clearly, including all four features in the center (representing the wishes of 74 percent of these respondents) would help bring some of the "might" voters into the "would vote in favor" category.
Benchmarking is a measurement tool that allows you to compare the survey results from your community with findings from other communities. Benchmarking questions can be written for a wide range of issues, including program features, usage, fees and voter support.
14. USE FIRST-CLASS MAIL AND STAMPED RETURN ENVELOPES. If you're using a written survey, first-class mail will give it a far greater chance to be read. The vast majority of other mail is thrown out by residents who assume it's junk mail. Be sure your survey is in an official city or agency envelope. You should always include a self-addressed return envelope with prepaid postage. Use a postage meter for your return mail to save the cost of putting stamps on all the mailed surveys.
15. PRETEST YOUR SURVEY. Whenever possible, pretest your survey before distributing it to your target audience. If you are doing a phone survey, you can call 10 households to be sure that the survey instrument accomplishes its objectives. Pretesting can identify problems with your questionnaire, such as excessive length or questions that are poorly written or ambiguous.
Do not use agency staff members or those very close to the project as pretesters, as their knowledge of the subject matter will generally be far greater than your target audience. If you have hired a company to conduct your survey, check on how they pretest surveys before initiating the entire process. Failure to conduct pretesting can result in a lot of time and money spent on a survey that does not work.
The techniques outlined here have helped dozens of communities tailor their project proposals to the needs and wants of the populace. There are two important points to remember: First, meeting citizens' needs spells victory at the polls. Second, bad data is worse than no data at all. Your ability to demonstrate that your finger is on the collective pulse of the community, along with the unassailability of your survey results, will help sell your project to the voters.