Rec Departments Embracing Web as Survey Tool

It is part of every recreation department administrator's job description: the evaluation of programs. But according to Barbara Heller, an administrator at multiple municipal recreation agencies before joining Indianapolis-based PROS Consulting, it is a part of the job that is often overlooked in favor of what she terms the "political goofiness" that accompanies work in the public sector. The result, according to Heller, is that rec departments become insular. "Directors of recreation agencies have to spend half their time running around doing government-official business," she says. "I ask them, 'What are your priorities? What are you doing that's more important than gauging how your customers feel about you?' "

While the typical rec director may offer multiple answers to Heller's questions, many others do make time to regularly solicit feedback from program participants. Historically, that feedback comes via a paper form, though some more adventurous departments are finding they can save paper, time and money through the use of widely available web-based survey and evaluation tools.

"With less staff and less money, you have to look at smarter, cost-effective ways of collecting feedback," says Jane Broida, professor of recreation studies at Metropolitan State College of Denver. "You also have to look at people's commitment level — if it's not fast, if it takes more than three minutes, people are not willing to do it."

A shortage of staff time is what inspired Mahomet (Ill.) Parks and Recreation director Dan Waldinger to drop his department's paper volunteer coach performance evaluation form in favor of an online version. He soon learned that the online form — which asks users to identify their sport and coach before entering into a series of multiple-choice questions followed by an open text field in which participants can inject their own thoughts — is helpful to his department in numerous ways.

Not least, it saves a little cash. "The cost of paper and ink, though it may be insignificant for larger departments, is significant to us," says Waldinger. "We are fee based, so every piece of paper we use ultimately gets tacked onto those fees." Waldinger has also found that the online system — which provides anonymity to users who previously had to hand a paper form directly to the coach they were appraising — has spawned more honest answers. "One coach called me up and said, 'It is great that we're doing this because I do know of colleagues who, before this format, would actually hand-pick the paper evaluations that they would turn back in.' " Similarly, Waldinger notes, "If somebody feels they have a very bad coach or there is a problem with the program, they want that opinion to be known, but they don't want to be the one to say it."

Perhaps most important for Waldinger, the paperless data is providing the framework for a rating system that gives each volunteer coach a score based on how well he or she is following through on the objectives of Mahomet's rec department as a whole. Those objectives — such as promoting good sportsmanship over winning — are addressed in, and can be quantified through, the online evaluation.

Though the return rate for Mahomet's evaluations is down somewhat since the web-only transition (only about 15 percent of participants have responded online through two seasons), that rate is climbing as Waldinger and his staff are promoting the tool through an electronic newsletter and global e-mails sent to program participants. A popular method involves keeping a database of e-mail addresses gleaned from registrations, and then targeting users by sending them links to the evaluation page once their particular class or program has ended.

Such specificity regarding the timing of solicitation and the content of the online evaluation form is required for even moderate return rates, says Heller. "If I go to Nordstrom and buy a pair of shoes, then a month later I get a survey in the mail asking about Nordstrom as a corporation, I'm not going to have any emotional attachment," she says. "But send me questions about my particular shoe-buying experience, and I may be able to give you some comments."

Online survey providers can personalize a program participant's actual evaluation experience through techniques such as "branching," in which users are provided specific questions based on their previous answers. The digital branching is immediate, and the data can be immediately sorted, saving the survey administrator from time-consuming data entry.

For many departments, an online evaluation or survey form represents a broad effort to reach out to more residents, particularly the younger set. In Glastonbury, Conn., 80 percent of recreation program registrations are done online, sparing loads of paper and staff time. The department has recently devised an online program-evaluation form, though it has yet to market the method and returns have been minimal. "We realize that the people who are participating in our programs are very technologically competent," says Anna Park, a recreation supervisor with the town's parks and recreation department. "We're trying to do as many things as possible in ways that they can relate to."

In Oak Park, Ill., recreation officials have likewise treaded — and likewise lightly — into the domain of web-based evaluation tools. The reason, they say, is simply to reflect the department's goal to promote open lines of communication with the residents it serves. "People like to be asked their opinion," Park District of Oak Park director Gary Balling says. "We hear it time and time again that they want more chances to express themselves." Despite that knowledge, Balling says his department has been leery of soliciting too much general user feedback, which is why it has yet to fully promote the online evaluations. "We want to make sure that when our residents do get a chance to express themselves, we are able to take the next steps — processing that information and putting it into action," he says.

Heller suggests that collecting data, whether online or otherwise, should only represent about 20 percent of a comprehensive evaluation system, which also requires analyzing feedback, responding to it within the financial limits of the department and verifying that any changes made are satisfying residents. "There are some departments that are so survey-happy that the customer will say, 'I refuse to respond to one more blooming survey from this group,' " Heller says. "The customer never sees anything back from the recreation agency that says, 'Thanks so much for the latest round of surveys. These are the kinds of things that we've been able to improve upon.' "

Heller is quick to point out that the statistical validity of data collected through online tools is, like any survey-based data, highly questionable. The reasons range from the inability to verify the identity or even the claims of a participant to the idea that only people reacting to negative experiences may be willing to provide feedback. Also, online evaluation tools should represent only a portion of a department's overall efforts — some of them equally as high-tech — to amass the collective thoughts of residents. "The online survey is but one way to do it," says Broida. "You can have touch-screen survey instruments in your facilities. You can go into parks, use a PDA to survey people and send the data back immediately."

While Oak Park's online evaluation form has yet to flourish, Balling notes that not a day goes by during which he doesn't receive at least one unsolicited e-mail from a resident offering feedback on some aspect of his department's programs. Such direct feedback, Balling says, is as valuable as anything he or his staff may find in a more formal evaluation form. "Everybody in our department is listed based on their specialty on our web site," he says. "We make our e-mail addresses readily available."

The prevalence of such informal e-valuations means department heads must be diligent in tracking feedback, including quantifying the numbers from and content of paper evaluations, online evaluations and even e-mails. "You want to be able to do a year-end data analysis," Heller says, "so you can say, 'I received 300 e-mails or X number of online surveys stating this position.' " Without such record keeping, rec departments may find themselves swamped with evaluations, and with seemingly no action plan. Says Heller, "The point of all of this is to put improvements into place that are going to respond to customer needs."

parks and recreation