Survey Says...

Over the last few weeks, District 113 conducted a survey that solicits community input about the future of our high schools. 

The survey questions covered roughly two broad areas: a post-mortem on the , and areas of need for the 113 schools. While I expect the results of the survey to start appearing in district communications soon, I am concerned about the way it was conducted and also the content of the survey, worries shared by many other District 113 residents.

The District 113 survey was widely publicized. District residents received a postcard soliciting survey responses, news stories provided links, and emails went home to booster club parents. My wife first heard about the survey from a city commissioner; within hours, both of us had taken the survey. That's when my spidey sense went off. 

In my day job, I run a product management organization and am quite familiar with market research survey design and output. Several things about the District 113 survey immediately worried me.

First, the survey has no validation. Anyone can take the survey unlimited times, which can lead to sample bias. The lack of validation also means survey responses could be completed by people who are not stakeholders in District 113, or students who aren't voting age. Few of the questions in the survey required answers, so you could go through and selectively provide input. Thus, there simply is no way to know how many times the ballot box has been stuffed, and whether the stuffers were eligible to do so at all. I asked District 113 about this concern. Their reply was that "Incorporating additional methodologies [for validation] would be very costly and in the opinion of the market research study group not worth the additional expense."

My second concern was the way in which some of the questions were asked. Who wouldn't indicate that they are in favor of "trustworthy elected officials" or "safe facilities" or "Adequate air ventilation"? Who would want to say that "enough quality lighting" is unimportant? Question seven asks readers to trade off these obviously-necessary components of any school. It seems like most answers will say that every possible aspect of a "quality high school" is needed; there is no weighting or reason for anyone filling out the survey to do anything other than indicate that every desire is important.

Third, I was anxious about the tone of some of the questions, and of District 113's response to criticism of the survey. The survey asked respondents to agree or disagree with statements like "I did not trust the intentions/expertise of the administrators who developed the plan"and "I was skeptical as to how well thought out the plan was." These questions wouldn't be asked in typical market research studies, because they assign a human, emotional element tied to people rather than process. Relating to these statements will not help District 113 design the next referendum, they only serve to assuage or impact the egos of those involved.

District 113, as is true in the survey itself, by asserting the qualifications of the volunteers assisting in this process. As with some of the survey questions, the tone of their response is "we know what we're doing, trust us."

This topic is too important to go with that approach. Simple techniques like IP address capture and cross-checking, use of authenticated Internet services for age verification, or serial number assignment are inexpensive and would have avoided any questions of propriety. 

I've personally come a long way over the last year on the issue of the referendum and the need to invest in both District 113 and District 112 schools. Like many voters, I am anxiously awaiting a new plan from District 113 on what the real needs are for our two area high schools. I've been persuaded by current and former families that the time is now to embark on some of the key improvements already identified. 

What I don't appreciate is a "lies, damn lies, and statistics" approach to decision-making. The district can assert that they will normalize the data from the survey, but without proper upfront controls, it is potentially garbage in, garbage out. Once District 113 starts using these statistics to influence the debate, how the numbers were arrived at will no longer be part of the discussion. It will be important for the District to prove that somehow the survey hasn't been unduly influenced by repeat voters, students, or partisans, or the survey was pointless.

Read District 113 Superintendant George Fornero's response to survey criticism .

For more news and updates from Highland Park Patch, "like" us on Facebook or follow us on Twitter.