Become a Fan

Other Pages

My Photo

Disclosure

  • Disclosure
    Nobody pays me to write any of the copy on my blog, and should I ever have the good fortune that they do, I will declare it. My main employment is as the owner and principal of The Future Place consultancy. The Future Place provides two key services 1) training and services to industry and academic bodies and 2) consultancy services to companies. The details of the companies I work with are a private matter, but if I blog about any company who has paid The Future Place more than expenses recently (approx. two years) I will mention that they are a client. I hold equity in Virtual Surveys and provide consulting services to them from time to time. I am paid to run courses for a number of trade bodies and over the last few years clients have included ESOMAR, AMSRS, MRS, and MRIA.

My daughters eShop

« Update on what ASAP means to people | Main | The economy 'will' be weak in 2008 »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Ray

Hi Karin
Many thanks for your 'book'. In terms of long surveys, there is scope for good, interesting, long surveys conducted with online panel members, provided they have been properly warned and rewarded.

But, most people in today's society do not have 20-30 minutes to spend on a survey, many people do not have the attention span, so most surveys should not be long.

However, I think there is great scope to build a set of answers from people over time, raher than try to collect it all in one go.

Ray

Karin

Ray,

I'm really enjoying your site and polls. Is length really the biggest issue? I wonder if it would be an issue if surveys were better designed and had higher intrinsic value (topics of interest, results available so respondents feel they learned and were part of a bigger something).

I ended up voting for having respondents able to add answers - purely as a reaction to what I have experienced as the biggest failing of surveys, which is that they've been badly written. They are rife with spelling and grammatical mistakes, contain jargon and language that only marketing or industry people use, violate many of the basic questionnaire-writing principles that are (were?) taught in high school and university, and have answer sets that seem to have been created without adequate reference to actual behaviour, emotions and thoughts. I believe Max had some similar observations.

I cannot remember the last time I took part in a survey without feeling that at least one of the questions required me to compromise what I wanted to say to such a degree that my actual response was meaningless (I guess that's why I'm more a quallie). I know badly-written surveys are not limited to the online arena, but with phone or paper-based surveys, at least frustrated respondents could skip questions that didn't work for them and/or attempt to convey their thoughts on the topic or on the survey itself by speaking or writing. Many, if not most, online surveys allow no outlet for comment and frequently do not allow skipping questions. One survey I took last month I finally abandoned in frustration after a series of moderately unsatisfactory questions because I got to one that was so badly written that I refused to commit to one of the four totally wrong responses (there was no 'other' or 'don't know' response). I would have been happy to skip and continue with other questions if I'd had the option.

On the other side of the desk, I've seen some truly dreadful surveys go out and I'm afraid the majority of them were only adequate. Very rarely have I seen one that was great. I know that it's easy for me to criticize (surveys weren't really part of my job as a qual researcher) and that clients can sometimes make it very difficult to create a tightly-designed survey that has all the information they want in the timeframes required, but honestly, we should be doing better.

So, anyway, length is not the main issue for me, if the survey is well-designed and respondents feel they've genuinely communicated their views and receive some feedback about it.

Sorry for writing a book, but I'm keenly interested in survey design! I love surveys and am sad that the quality of many of them undermines their value (to the public and their utility due to quality/accuracy of data).

cheers
Karin

Courtney Kuehn

My vote is getting the least amount of agreement. I suppose it depends on what the goal of the survey is. I always find myself wondering what the results are after I take a survey.

I like it when a survey isn't horribly designed but how attractive should surveys be? Would it be distracting if it was too 'pretty'?

Ray Poynter

Max, I agree with most of your points. However, as I said in my commnet to John, this poll (poll is this case being used in the Internet sense of a survey with only one question) is part of the dialogue not the whole dialogue.

For example I wanted to see where accessibility came in people's minds. In the UK it is a legal requirement for surveys to be accessible, and a growing number of clients are beginning to request AA or AAA compliance. But most software vendors and agencies seem to be ignoring this area, without any impact on their business so far.

I two have a problem with the word survey, as it is a one-way process. I survey, you get surveyed. The whole research 2.0 approach is about collaboration and co-creation. And, I am sure the discussion will turn to that point shortly.
Ray

Ray Poynter

John, I completely accept the poll is limited, that in part is due to the format, but also because the poll is only part of the discourse I am trying to encourage. I would like to see more discussion on FaceBook of the sorts of issues you raise.

The issue about when mobile interviewing will take-off (and the issue is when not if) is a good one. I think the take off is more likely to be 2009 or 2010, because I think we need more phones like the iPhone and a development in the way we approach research, i.e. not simply another rollout of traditional research via a new medium.
Ray

Max Kalehoff

Ray,

In addition to length, efficiency of language and simple language is needed...BADLY. Too many surveys are written by academic geeks versus good writers who really know their respondents. Think about the copy in an issue of USA Today. That's the sort of ease of use in comprehension that surveys should take.

Secondly, packaging matters. Most surveys arrive at your inbox looking just as bad or worse than spam. Why can't market researchers be inspired by Apple, BMW or Nike when packaging their surveys? Why isn't just as much investment allocated to User Experience as questionnaire design, analysis and execution?

Thirdly, the first question researchers MUST ask themselves in any survey design is: "What value will this survey provide for our participants?" No, I'm not talking about artificial incentives like sweepstakes or payment. I'm talking about intrinsic value that the survey would/should provide. Entertainment, interest, community, insights, TRUE input into product development are some examples of intrinsic value. The vast majority of surveys really offer NOTHING in return, not even the courtesy of final survey results at the end of the project.

In my mind, survey borders on being a dirty word. Though I admit I'm still using the term. I hope to put it behind me by the end of 2008.

Regards,
Max

Jon Erickson

Ray, I was excite to see your Facebook note about asking for input on on-line surveys. However I believe your poll is too broad or vague to gather good input. There are so many ways to use on-line surveys ranging from creating customer preference centers to lead acquisition to customer satisfaction to conversion of off line to on line participants.

I believe 2008 is a year that marketers will significant expand the use of on-line surveys (yes I do work for a company that provides these solutions so I am biased) but also is a year that marketers start embracing the mobile devices to get real time feed back on various aspects of the customers experience.

Thanks for hearing me out on this.

The comments to this entry are closed.