Survey

BYTE Learns a Lesson in Web Surveys

What a difference a medium makes. Over the past several years at trade shows, such as Comdex, BYTE has polled attendees who visited the BYTE booth on current and future OSes and processor use. The surveys, which we occasionally published in BYTE, typically sampled 1000 or more attendees who happened to visit the BYTE booth at a trade show.

We wanted to do that on the Web to get even more respondents, but it didn't work as we'd hoped. Why? Because at Comdex, a busy trade show where people usually have too much work to do to try to stuff a ballot box, the poll surveyed a more-or-less random sample of the computing population. But on the Intern et, one posting in an Internet advocacy newsgroup can quickly rally an entire population segment to vote in a Web survey. And that's what happened to us.

The information below is a perfect example of that. You might think that AIX and OS/2 are surprisingly popular if you were to take the information below at face value. The problem is that our data was biased by OS/2 and AIX supporters who wanted to ensure their favorites did well: Survey responses are 12 times more likely to come from ibm.com than are typical BYTE Site visits.

Lesson learned: The Web doesn't change fundamental survey maxims; you can't let the studied population select itself. And, to the individual who voted over 80 times in the survey for Unix (your IP address is 198.182.4.224): Get a life.

---

Tangled Up in Blue

illustration_link (16 Kbytes)