Top-10 Web Survey Issues and How to Address Them

Written by Scott W. Spain

Having conducted Web-based surveys since 1996, I've seen Web-based interviewing evolve into what it is today. Providers of this service have come a long way, but there is still much work to be done. In this article I will examine 10 Web survey issues that can negatively impact your projects' response rates and stastical significance if not handled properly. A few are somewhat technical in nature (just pass them on to your programmer), but most are geared toward research professionals.

10. Avoid Heavy Desserts
A handful of Web survey survey applications and custom programmers require the use of "cookies" to carry data from page-to-page on multi-page Web surveys. What cookies are isn't important in the context of this article. What is important is that you avoid the use of Cookies at all costs. The "cookie scare" has not gone away. Thousands of Internet users have their browsers set to disable cookies. Cookies are not required to perform basic survey functions.

9. Caffeine Makes me Nervous
A few years ago Java was going to be the programming language that revolutionized the Web. So far, it hasn't lived up to its potential. When Java doesn't work on a respondent's system, it doesn't just not work. It often completely crashes or freezes the user's system. Additionally, some of today's most popular browsers no longer offern native Java support. Even some versions of Netscape, once Java's strogest supporter, requires users to download a separate plugin to run Java applications. Using this language in your surveys may leave 10%+ of your target market unable to access your surveys.

8. Examine Page Formats
There are two common formats for Web-based surveys: Single-page, and one-question-per-page. Be very careful when deciding which to use. The major downfall of a single-page survey is that you lose the ability to incorporate "true" skip patterns and piped responses. If the survey is long, respondents using dialup ISP's may use their Internet connection while answering. There are also problems with the one-question-per-page format. This method requires more server calls, which increases the amount of waiting time the respondents will experience. My recommendation? If your software allows it, try going with 2-4 questions per page, and watch your dropout rates fall.

7. Wider is NOT Better
Placing questions in tables is becoming increasingly popular. It should be… It's a great formatting option! Be sure that the software (or custom programmer) you use does not make tables too wide for everyone to view properly. At least 5% of the online population is viewing the Web at a low resolution setting of 640x480. Depending on your target audience, you may also have WebTV's users attempting to completed your survey. These folks are sufing with a resolution that is only 544 pixels wide. In addition to width, the height of your tables is also important. Have a look at the ficticious response table below, in which the respondent is asked to rate colors on a scale of excellent to poor. Unless your computer is set to a high resolution, you'll notice that the response choices (excellent-poor) scroll off the top of the screen while you're viewing the bottom half of the grid.

6. We Don't Need no Stinkin' Data Entry!
If you are using a system that sends survey responses by email and requires you to keypunch the data, think again! The respondent has already entered the data electronically, it should not need to be done twice. Find a programmer in your area. Creating a script that writes form data to an ASCII file is child's play for any CGI/Perl programmer.

5. Don't Force the Issue
If response rates are a concern (and they should be), be careful which questions you choose to validate (force respondents to answer). Over the telephone an interviewer can smooth out the process of forcing responses. On the Web, respondents get frustrated by error messages and often drop out when faced with too many of them. My recommendation? Only force respondents to answer questions that are critical to the success of your study.

4. Screening for Vengeance
Be careful when terminating respondents that do not meet your screening criteria. Never tell them they "don't qualify". It's too easy for them to use their browser's "back" button to change their answer in hopes of getting the incentive. Give them a shorter mock survey to complete, and clean their responses out later. Needless to say, you will need to address the incentive issue with these people. After all, they did show up.

3. Troubleshoot Poor Response Rates
There are two contributors to low response rates: 1. People not showing up to participate in the survey. 2. People dropping out after starting (but before completing) the interview. Before you can effectively improve a poor response rate, it's important to know what is causing it. The way you address people not showing up is completely different from how you handle people dropping out midway through the survey.

2. Hold the "Spam" Please!
Do not, under any circumstances, recruit for a survey through unsolicited email. Not only does it give our industry a bad name, but there are also too many potential problems on an individual level. You only have to send to the "wrong" person once. Send UCE (Unsolicited Commercial Email) to somebody with the knowledge and equipment to send an effective "email bomb," and your server could be incapacitated for weeks! Chances are, spamming is against the agreement you have with your ISP and/or hosting company anyway. Very few actually allow this practice.

1. Don't Recruit from "Home"
Building custom panels for Internet research is the latest online rage. Why not? It's a great idea! But, for the love of Waksberg, please do not include a "join our panel" link on your corporate home page! Do you really want the people who visit your corporate home page (prospects, customers, vendors, family members, competitors) joining your panel? Of course not! When building an online database of potential respondents, it is critical that you set up a completely separate site to do so.

©Copyright 2008 by DigitalBiz Corporation. All Rights Reserved.