AMSRS Summer School Points the Way to the Future

I spent Sunday and Monday of this week in a rain-drenched Bondi (Sydney, Australia) as a contributor to the AMSRS Summer School. In many ways this event, especially when taken in conjunction with the NetGain4 event in Toronto held about ten days earlier, shone a spotlight on the future.

One of the best things about Australian market researchers is the way they maintain their professional standing. Australian researchers need to undertake personal development work every year to gain a certain number of points. This means all the accredited researchers in Australia are looking for chances to improve themselves (or at least to earn their points) every year. One way of gathering a large number of points is to attend the annual Summer School, which this year ran over a Sunday and Monday and focused on online research. Because of the focus of the industry on personal development, learning opportunities tend to be seized by a wide range of people, including senior industry players, something which tends not to happen in other countries.

So, what was so good about this Summer School? For me the key things were:

1.    A universal agreement that online access panels cannot approximate to random probability sampling and they we need to make sure that clients understand the limitations of online research, as well as it advantages (e.g. it speed and low cost). This is not a reason not to use online panels, but simply a reason to develop a better understanding of when they work and when they don’t.

2.    Almost complete agreement that telephone research also cannot provide random probability sampling. But, a recognition that for some purposes telephone is better than online.

3.    Agreement that long surveys result in bad data and unhappy respondents. Although there was some disagreement about what ‘too long’ means. For some people anything over 15 minutes is too long, for others it is anything over 20 minutes or over 25 minutes.

4.    Recognition that surveys need to be more engaging, but equally that making long and boring surveys more attractive is like, in Reg Baker’s words, putting lipstick on a pig. There was also agreement that not all attempts to make surveys more fun improve the results.

5.    Acceptance that listening and ethnography are going to be a bigger part of the research picture, but equally that in the short term online surveys are going to be where the bulk of market research spending goes.

There were several very strong presenters and speakers, but I will single out just four for special praise.

• Pete Cape, of Survey Sampling, gave a good presentation on the way that online questions should be constructed with some really good examples and sound advice. For example, Pete suggested picking an academic, such as Jon Krosnick, and simply going with what they suggested, rather than letting every researcher in an organisation create every scale from scratch and according to their own preferences or ignorance. Pete was not specifically endorsing Krosnick over other relevant academics, but simply asserting that a company should adopt a house style and that the house style should be based on some formal reasoning.
• Jonathan Jephcott, gave a very thorough exposition on the state of current knowledge about online access panels, drawing heavily on ARF’s work. Jonathan is about as well qualified in the formal theory of statistical research as anybody in the industry and there is little scope for people to try to weasel round his comments, for example about the need to ensure clients understand what they are buying when they are buying research based on internet samples.
• Kris Hartvigson of Vision Critical gave a great presentation illustrating community or client owned panels. I truly believe that these are going to be massive over the next few years.
• Jason Bucahanan, the ex-Managing Director of Research Now Asia Pacific gave a breathtakingly honest picture of what can go wrong with access panel research when one or more of the end client, the researcher, or the panel do not properly think through what they are doing. Unless he is very expensive, I would think it is worth any agency hiring Jason for a morning to come in and to expand on this presentation for them. The savings they would make on avoiding mistakes would easily cover his costs.

My other take-outs from the two days were:

1. People seem to think that online research panels will always be here. Which I find odd because it is so new. When times are changing so fast (ten years ago online research with access panels was about 0% in Australia now it is 50% of survey research) it seems strange to assert that things won’t change much in the near future.
2. The research industry is currently facing the challenge of either exploring new boundaries or adopting a laager mentality *.

In terms of the event, a Twitter search for #amsrs shows the buzz it created with many people saying it was the best ever. Other than the need to achieve a better gender balance amongst the speakers, AMSRS seem to have hit on a winning formula with this year’s format, perhaps ‘listening and observing’ next year?

* twice over the last weekend the term laager mentality was used, once by me and once by Jonathan Jephcott, but only a handful of people seemed familiar with it (with a couple thinking it might even relate to a lager mentality!) The term laager refers back to 19th Century South Africa. When the voortrekkers were expanding into Africa they would circle their wagons into a circle at night or when they needed to rest. The laager is a defensive structure, with the people inside looking inwards, and with everybody outside considered hostile.

Given that the term laager mentality is becoming less common, perhaps I should simply stop using it, but that would be a shame since it precisely defines a meaning that no other term manages to articulate so clearly.

Comments

[buy sildenafil citrate]

hi guys
is there any good loop based music creation software that you can download from the web
i have allready tried dance ejay,storm,cubase and mixcraft
anyone know any other good software for creating music(dance and trance mainly)
cheers guys !

[CrisisMaven]

I see you are interested in statistical research. I have put one of the most comprehensive link lists for hundreds of thousands of statistical sources and indicators on my blog: Statistics Reference List (http://crisismaven.wordpress.com/references/). And what I find most fascinating is how data can be visualised nowadays with the graphical computing power of modern PCs, as in manyof the dozens of examples in these Data Visualisation References (http://crisismaven.wordpress.com/references/references-subjects-covered/data-structuring/data-visualisation-references/). If you miss anything that I might be able to find for you or you yourself want to share a resource, please leave a comment.

[Sensorpro]

Great summary Ray. Thanks for the laager reference, will try it out!

Chris

[twitter.com/JHenning]

I've often wondered why the Australians, as a group, seem disproportionately involved in the wider market-research community. Thanks for one possible explanation.

As for 'laager mentality', here in the States we call it 'bunker mentality': "An attitude of extreme defensiveness and self-justification based on an often exaggerated sense of being under persistent attack from others." (http://www.thefreedictionary.com/bunker+mentality)

[Praz]

Thanks Ray, I shared this post on my blog too (referenced to The Futue Place ofcourse)

[Ray Poynter]

I am happy for anybody to quote or use material from the blog provide they do credit the source. Except for stuff from the book which is no longer mine to give away.
Ray

[Praz]

Hi Ray,

You beat me to it. Awesome set of thoughts and totally agree on most counts (specially points on Community/client-owned panels). Was great seeing you in action as well. Till next time....
Praz

[Researchnewsmag]

Thanks for the great summary Ray. I presume you don't mind if I quote you in the story I am writing about SS10 for Research News?