Some Straight Q&A about Surveys

So you want to write a questionnaire and get some feedback from your customers.  Sounds easy, right?  When was the last time you generously agreed to take a survey only to discover it was a painful experience that you wish you could forget?  Those are the surveys you promised yourself you would never inflict on your own customers.  Why is everything easier said than done?  With these helpful tips, your customer Q&A sessions can now be very helpful to you and much less painful to everyone involved.

Decide on what you want to know before you begin.  It is extremely difficult to form a questionnaire, much less properly administer one if you don’t even know what you want to address.  The primary reason for surveys in the first place is to research a problem or what is perceived to be a potential problem rather than to ask a bunch of random annoying questions that have no value to the customer or the researcher.

Determine the method of administration.  There are four main ways of sending out questionnaires: mail, e-mail, telephone surveys, and good old-fashioned face-to-face interviews (sometimes abbreviated to the less personal version of “please fill out this survey and drop it in the box” method – useful for extremely short surveys).  There are pros and cons to each method, so consider them carefully against your personal situation to ensure you choose the best fit for you and the problem you are researching.

Determine the questions carefully.  This cannot be stressed enough.  There is a difference between asking questions about what you need to know to solve your problem and what you would like to know but doesn’t apply to your problem.  The more questions you ask, the longer your survey will be and the crankier your customer will likely become.  Ask only the questions that will address the problem you are researching – the rest is nice but irrelevant.  You will also need to pay very strict attention to how you phrase the questions.

  • It is possible to bias your survey if you ask the questions in the wrong order. Of course, the customer has heard about fabulous Product X if you just spent the last three questions asking them about it – you just told them about it!
  • Each question may only ask one question at a time, don’t try to confuse your customers (and destroy your results) by trying to multitask a question. Ask about quality of service or ask about quality of product, but not at the same time.
  • If your questions will not apply to the entire population it reaches, consider using filtering questions or redefining the population used for your research.
  • If a question can be answered from different perspectives, carefully phrase the question to provide only the perspective you need.
  • Consider whether the customers will be able to remember the details about the issue that the question is dealing with.
  • Make sure the time period is appropriate for the topic. Sometimes a week or two will suffice; sometimes a month or two would be more appropriate.
  • Avoid questions that require excessive effort to answer or deal with sensitive, embarrassing, or potentially threatening issues. This could kill your response rates if you don’t handle it carefully.  Just a hint, some people tend to get touchy about demographic questions, so only ask them if you actually need them.
  • If sensitive questions must be asked:
    • Guarantee respondent anonymity or confidentiality.
    • Use counter-biasing statements.
    • Phrase the question in terms of others and how they might feel or act.
    • Use categories or ranges rather than specific numbers.
    • Use randomized-response models whenever possible.

Determine the answers carefully.  After putting so much thought into the questions, it would be a shame to flub it now.

  • It is equally possible to bias your survey if your answers are in the same order for every survey you submit. Remember how it was the widely believed urban legend in grade school that if you didn’t know the answer on the test you should choose “C” because it was more likely to be correct?  Still happens.  The best way to avoid this is to randomize the answers whenever possible.
  • Some answer types provide the data that meet the information needs of the projects. Open-ended questions, where the customer writes whatever they want, will get you closer to honest answers about why something happens.  Close-ended questions provide (relatively) concrete answers about how many, how often, etc.  However, not every survey-taker wants to write a novelette just to satisfy your curiosity.  Save the open-ended answers for the most important questions and make those questions as specific as possible to reduce the randomness of the answers.
  • Try to avoid using absolutes and vague references when giving multiple choice answers. What might be mostly to one person might only qualify as sometimes to another even when it is the exact frequency they are talking about.  That doesn’t do you any good, now does it?  As for always and never, they usually mean almost always and almost never, but people don’t usually want to own up to socially unacceptable behavior and will idealize their goals of more acceptable behavior instead.  Dishonesty isn’t what you’re after, is it?
  • You might only be asking one question, but sometimes more than one answer is possible. “Check all that apply” is one way to find multiple answers when you give more than one option.
  • Be sure to provide “Don’t know” or “No opinion” or “Doesn’t apply” options if these are likely to apply to a significant portion of the population. The actual responses in and of itself will tell you something about the population you are reaching.
  • When using fixed-alternative questions, be sure the choices are exhaustive and mutually exclusive. Have you ever encountered the demographic question where a person, if they were the correct age, could fit into two categories but could only pick one?  Or how about the question where you wished they had an “Other” and then a space where you could write what that other was because none of the options they provided were even remotely accurate?

Determine your wording carefully.  Wait, didn’t we just do that?  Yes, now do it again – with the wary eye of seasoned survey-taker.  Is anything potentially confusing?  Is it asking something vague that can be misunderstood?  Is it asking something personal that you don’t feel comfortable telling a complete stranger (identity theft, anyone)?  Are the questions in a logical enough order that you can identify the point of the survey or does it jump around so much that you don’t know what it wants?  Is the language simple enough for a five-year-old to understand or do you require a Ph.D. in linguistics?  Do you feel railroaded by any of the questions? Do you feel like it is begging for a pat on the head or are you able to give a realistic view of service? If it is a feedback survey, is there a space for honest, open opinion or is it all check this box?  Does it ask how they can do better? (Half of the feedback surveys I’ve seen don’t actually include that question but they should – those are usually for the places that need it the most.)

Determine what it looks like.  Always make sure it looks professional and easy to answer (people are more likely to respond that way).  If it is printed, use quality printing and good paper.  Aim for it to be as short as possible while avoiding a crowded appearance.  Easy on the internet, but on paper this means one page, using the back if necessary.  If it must be longer than that, use a booklet format to prevent lost pages.  List the name of the organization conducting the survey on the first page.  Use clear, concise instructions.  Number the questions.  Use appropriate graphics if necessary to improve the appearance of the questionnaire.

Hey you!  Now that you have a nice, shiny new survey you need some people to take it.  You need to develop a recruiting script to get it out there!  Keep the message as brief as possible, but include the following at a minimum: who you are, why you are contacting them, your request for his or her help in providing information, approximately how long it will take to participate, responses will be anonymous or confidential (only if true), and any incentives the participant will be given (only if true).  For mail and e-mail surveys this can be polished for hours before you hit send or lick a stamp.  For telephone and in-person surveys, this can be polished for hours in a mirror before you go public.  The point is, polish this so it comes across as a natural conversation rather than sounding forced.

Test before you invest.  To check to see if your survey actually works, issue it to a small sampling and see if anything comes up that all of your careful planning failed to catch.  Is something worded a little off that didn’t confuse you, but is confusing others?  Did they draw in an “Other” box and give you what was missing from the predetermined choices?  Are the results of the test looking about like what you would expect or is there something… off?  Do the buttons on the e-mail survey actually work or do they just look pretty?  This is a great time to go back and give your survey a fine-tuning if it needs one before releasing it for real.

Insert your isolated DNA intoa suitable vector

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s