The Future Place Blog

Looking back at the Research 2010 Conference

Well it has been a crazy week and a bit, but now things are settling down I have had a chance to think about the Research Conference.

My overall feeling was that it was well worth being there, both enjoyable and useful. It was certainly not a complete success, but most of the things that were problems were either things that can easily be addressed or were steps on the road to something better.

The worst thing about the conference was the food, closely followed by the layout of the rooms being used by the conference (3 levels, with a narrow set of stairs connecting them).

Key thoughts triggered by the Conference were:

The Number of Delegates

This year’s conference was much smaller than the MRS Conference used to be, which was initially a disappointment, but how many people do we really need? Once the number of attendees passes about 300, it becomes very difficult to meet more than about 80 people. More people is good for the people on the platform, it is good for the finances, and for anybody exhibiting, but it is not a major issue for attendees. Whilst it would be good for the numbers to increase, to generate a sense of momentum, it is not a key issue.

Timekeeping and Short Sessions 

One problem with this year’s conference was the very poor timekeeping; nearly all the speakers appeared to over-run their time slots. Some of the people with 12 minutes over-ran by more than 50%. This resulted in very little question time. Conference organisers and session chairs need to remember the people on the platform have not paid to be there, they can be bullied, but the audience have paid, when they are prevented from asking questions they are being short changed.

This year the Conference had too many short presentations, which resulted in a lack of depth in too many of the presentations. It is great to have some short sessions, such as the Ideas Rush, but there needs to be a mixture, with 5s, 10s, 20s, and 30s.

Highlights

So, what were the highlights? Well the first one for me were the eight speakers in the Ideas Rush session. They were some of the best prepared and thought through presentations of the conference. 

I thought some of the topics which cropped up were interesting such as the use of bots in research and the need to get young people more involved were good. I thought the amount of prominence given to semiotics was good, and was a direct rebuttal to the notion that something has to be social media or mobile to be fashionable.

I particularly liked the theatre of the Big Thnkers session, we need to push these boundaries further, and create more contrast between the theatrical and the pedagogic, and facilitate both.

The Less Successful Parts (IMHO)

The second keynote speaker was shockingly poor in my opinion. Perhaps because I disagreed with so much of what he had to say, but I felt it was poorly grounded and came across as a very articulate version of a pub opinionst.  He appeared to criticise the Government for being alarmist over fears about swine flu but went on to make a prediction of a fire and numerous deaths on the Central Line during the next 18 months.

The Question Time format did not work, IMHO. I think the reason was the choice of chairman, which before the session I had thought was a good choice. The chair was Stephen Sackur, journalist from BBC World. However, on the TV Sackur is more knowledgeable than the audience, but less knowledgeable than the experts. His job is to make the experts explain themselves in ways the viewers can understand. However, in this debate Sackur knew less about the topics that almost anybody in the room. The audience did not need him to dumb the panel down, they needed him to press the panel on issues like how will we make money in a social media future? What are the limits to self-regulation? Have the recent cuts in recruiting to MR created a problem for 2011?

Main Inspirations

The best thing at the conference, was a side session on Empowering Young People, led by four people aged 17 to 21 from the Edge Learning Forum – more sessions like this please.

Armando Iannucci was great, reminding us that we need to be more engaging, better tellers of stories, and that we need to get out of our box more.

Daren Cornish was massively motivating with his description of what he has been able to do with Social Media within his organisation.

Mark Earls' session and Rachel Lawes' session on semiotics were both great as showing that there is new thinking going on that is not related to technology, new thinking that is focusing on people and how we communicate.

Final Thoughts

I will be there next year, the Conference seems to be on a journey and is getting less like all the other conferences. I attend about 15 event s a year and this is now firmly in my top three.

Why a good presentation is not a good leave behind/report

I am involved in a fairly long LinkedIn discussion about presentations and presenting (). The drift of the conversation has recently shifted to whether on document can be both a good presentation and good record of the study/report/leave behind/ standalone presentation. My view is emphatically NO!

Below I have reproduced and modified my post to share it more widely.

My view of what a client debrief should include on the screen includes:

1. Only those parts of the questionnaire that contribute to the answer (i.e. don’t display all the questions).
2. Only significant differences get displayed (on quant studies) .
3. Min font size for labels and numbers on chart 16 point .
4. Min font for text 24 point, with major points going up to 48 point (some caveats or warnings might occasionally be as small as 16 point).
5. No bullets, or very few bullets in those rare cases where a list is actually useful 
6. Only show bases sizes where they help interpretation (which is rare). For example, I will say that the data are based on 750 interviews, and I will warn you when I am working with smaller numbers.
7. No detailed methodology/sample section.
8. In most cases I should be able to talk to the slides, introduce my thinking, tell the story, and provide conclusions/recommendations in less than 30 minutes, ideally less than 25 minutes. 

Even when I am presenting heavy quant data, for example choice based modelling, I show very few numbers. Numbers are the tools I use, as an analyst, to find the answers. What the client (in most cases) wants are answers, not the chance to work with the numbers. If I have an insight manager who loves the numbers, I try to have two debriefs; 1) a methodology/analysis version with the insight manager, perhaps lasting an hour (often using remote techniques such as GoToMeeting), and 2) answers version for the insight manager’s internal clients.

When I follow these guidelines (remembering that all clients are unique and consistency is the hobgoblin of small minds) I produce a PowerPoint deck that is great during the presentation, but is not great as a leave behind.

When I am a receiver of presentations, and when I have researched client needs, the sorts of things that I want in the leave behind are:

1. Methodology, sample, and fieldwork dates (99% of my presentations do not put the fieldwork date on the screen, I will simply say things like "and the fieldwork was in June" or "the fieldwork was in the week after the new product launched", but in the leave behind I want to know when the fieldwork was done, whether it was F2F, CATI, etc, who the sample were etcetera. For regular clients, the description of the sample might be as brief as “we interviewed you normal selection of heavy and medium buyers, in May”, on the screen this might be represented by one word, e.g. Who?
2. Data from all of the questions I have paid for. I want these in the leave behind so I can help use the data to answer future questions. 
3. The text of the questions, in case there is ambiguity what a presentation really means. 
4. An executive summary of 500 to 1000 words. 
5. Warnings and caveats. In the presentation I will say those warnings and caveats that are relevant to the audience. In the leave behind, I do not know the audience, so I have to include all the relevant warnings and caveats.
6. More detailed conclusions and recommendations. 
7. Information about my company and how further contact can be initiated (almost every client I have spoken to hates it when a debrief contains several pages of company background, they are fine in a pitch, fine in a leave behind, but 99% of the time they have no place in a debrief) 

 There are many ways of creating a suitable leave behind, for example a revised PowerPoint (particularly one that uses the notes page well), a report, an online hyperlinked document, an interactive model in Excel (with help/documentation features). But a good presentation is not a good leave behind.

Yes, I accept that a good presenter can work with a slide deck that has too much information on the screen, which has text and labels that are too small, but why do it? Why not use the presentation to support the speaker, not the speaker to support the materials? 

What is good for society is good for market research

In a recent article ResearchLive [http://www.research-live.com/news/government/obamas-healthcare-law-brings-good-news-for-survey-researchers/4002351.article] the view was put forward that it was good news that the a change to the US Sunshine legislation was a good thing. The original version made it likely that doctors would have to declare any income received for taking part in surveys. This, it was feared, would result in fewer doctors being willing to take part in research. The general background for the legislation was concern over whetehr the payments doctors received changes their attitude or behaviour towards specific prodcuts and servcies.

Here is the comment I posted:

"My starting point with legislation is the assumption that if it is good for society then it is good for research (even if the immediate impact on research is not helpful).

In his Wealth of Nations Adam Smith pointed out that healthcare cannot operate as a free market because of the imbalance of power and knowledge between the buyer and the seller.

I think that in that context, requiring doctors to file a return declaring all payments has to be a good thing. Buyers of doctor's services would be better served if they knew of all other payments.

If such declarations became the norm, I would expect the long term effect on co-operation to be minimal (but it might drive incentives even higher). But, in my opinion we ought NOT to put our short-term interests before those of society in general and buyers of doctors' services in particular.

Indeed, I think we are moving towards a society were openness and transparency will be the norm in most sectors."

My Slide from Research 2010

In my presentation yesterday I only really used one slide the one below, a taxonomy of NewMR.

 

Note, Red means researcher driven (ME research), Green is collaborative (WE research).

What else would you put on the map? 

What would you move?

No Surveys in Twenty Years?

Yesterday, at the MRS Conference in London, I caused a bit of a stir with my suggestion that in twenty years we won’t be conducting commercial market research surveys (see #res10). Clearly, in a question from a roving mike there is a limit to how much thinking one can get across, and anyway my thoughts were still forming. So, here is the slightly more nuanced, thought through, caveated version of the same proposition.

Firstly, I am not talking about social research. I think it is possible, perhaps likely, that core social research will need its surveys.

Secondly, I am talking about markets where market research has a developed to the point of having access panels (be they online, mobile, or some other new variety that might happen over the next twenty years).

Thirdly, when I say no surveys I mean instruments that last ten minutes or more and where the respondent can tell it is a questionnaire.

OK, so now I have got my caveats in place, what am I saying will change?

We can see a trend at the moment which will result in most, maybe almost all, commercial market research being conducted via access panels. Or, to give them their new name ‘convenience samples’ of people who are doing 100s of studies per year. Note, we have given up on random probability samples and we are giving up on the idea of interviewing people who see fewer questionnaires than we do.

Where will all this take us? IMHO, I expect to see?

1. More social media listening.
   
2. More discussions in social media (MROCs are just the start).
   
3. More new ways of researching ideas, think online gambling, think Iowa Electronic Markets, think about an eBay of idea information, think life blogging, think web analytics, think CRM, think new models such as Threadless.com.
   
4. Open-ended research. Instead of surveys we are getting to the point where we can ask and process open-ended questions. I expect this to be in place in 20 years. This will replace many surveys.
   
5. Given that most survey data come panel members, and that all their previous responses are stored in data warehouses, we only need to ask genuinely ‘new’ questions – not whole surveys. Given a survey that at the moment might last 20 minutes, we will only need to ask a panel member, say, 4 scale questions (Reichheld say just one question), plus conduct an automated, open-ended discussion.

Remember, clients do not need surveys, they need insight and guidance. Whilst surveys are the best or fastest, or cheapest solution they will persist, but they have no ‘right’ to exist. Just because we have been using them for quite a few years, does not mean we will be using them in the future.

So, Ray’s predition, tell me afterwards if I am wrong, in twenty years we will not be conducting commercial market research surveys.

Deriving insight from online discussions - the next book?

At the moment I am going through the editing process for the Handbook of Online and Social Media Research (publish date July this year), but I am also gathering ideas and materials for another book.

Being basically a research sort of person I have set up a poll asking people's help in deciding what I should try to write next. So far a generous 123 people have cast their vote, and just 4 have voted for No Book!

If you would like to cast a vote click here.

At the moment 'Deriving insight from online discussions' is ahead of the field with 41% of all votes cast.

In the runner-up position is 'Analysing Market Research Data from cross-tabs to conjoint', with 33% of the votes.

My initial favourite 'The busy exec's guide to social media' is lying third with 20%.

I had to smile when I saw the results (or rather the results so far) as I feel that the 'Deriving' results book is by far the greatest challenge, spanning large scale blog and buzz mining through discourse analysis, via MROCs, all the way to the sort of Netnography that Robert Kozinets talks about in his new book (Netnography: Doing ethnographic research online), which I highly recommend.

However, it probably makes sense. I feel that at the moment we are only scratching the surface of what is possible in terms of working with online discussions.

Twitter and the ESOMAR APAC Conference in Bangkok

The next ESOMAR APAC Conference is going to be held in Bangkok from April 25 to 27. The conference is already shaping up to be a great event, the programme looks good, lots of key people have already signed up to attend, and there is an interesting social network for people planning to attend.

One event at this year’s Conference is a Twitter event, which is being organised by myself and a group of key co-authors.

The description of the event in the programme is:

“And Now For Something Completely Different!

This Interactive event will illustrate how Twitter is changing the face of research for some. Join in and lean about Tweetference, TwitterWalls, and TweetUps. Decide for yourself whether it is fad, fetish, or the future! Listen out for more news on how you too can be involved in this project, follow #ESOMAR”

The team running the project, and who have written the background paper are:

Amber Coulter (The Research Agency, New Zealand), Angela Gao (SSI, China), Bill Zuo (SSI, China), Daniel Alexander-Head (Colmar Brunton, Australia),  Prashant Hari (Colmar Brunton, Australia), Shizue Vieira (Fem Marketing House, Japan), and myself (Ray Poynter, The Future Place, UK).

Some of us will be at the Conference, others will be chipping in from our home countries, and on the Tuesday morning we will presenting what we have learned during the run up to the conference and during the first day.

One of the challenges for this project is that not everybody follows Twitter and in some countries Twitter is not available, for example China (although China does have other microblogging services).

To help focus attention we have created a group on MRSpace which amongst other things reports the latest Twitter posts about #ESOMAR. You do not have to join MRSpace to check out the group, but you would have to join if you want to post your thoughts and views.

Our ambition is make this ESOMAR APAC the most externally viewed and tracked ESOMAR ever, even more than the Online Conference in Chicago last year. In effect we are creating a Tweetference to expand the impact and reach of the conference.

There will be Twitter reports, a Twitter wall, Tweetups, and events. If you have any suggestions about the best way of involving more people, please post a comment here, or on MRSpace, or simply Tweet it with a tag of #ESOMAR.

SMRD – Much to teach other conferences

Last week I spent two days at the Swedish Market Research Day (SMRD), and annual event organised by Swedish researchers and spearheaded by the indefatigable Henrik Hall. Whilst no event is, or can be, perfect, the SMARD has much about it that others can learn from.

Amongst its key strengths are the following:

1. The conference lasts just one day. Many events last 1.5 days, but in these cases many delegates tend to leave before the end of the event.
2. The SMRD finishes with a really good social event, which is good for networking but also cuts down on the risk of people leaving early.
3. The conference has a mixture of papers presented in English and Swedish, and this year used a mixture of one and two stream sessions to be able to provide an English option throughout the day, making it more attractive to international attendees.
4. The papers were a mixture of local papers, invited international guests, commercial presentations, and a great keynote speaker (Teo Härén).

One consequence of the strength of the programme was that there were about 250 delegates and this in turn meant that there were a really large number of exhibitors.

One innovation at this year’s conference was to hold a workshop on the day before the conference (a Master Class in Multivariate Techniques run by me). This seemed a good idea and I hope that SMRD will make more use of this in the future as it can help with the costs of speakers.

There is a report of the conference on Research –Live. One of the features of the conference that this report picks up on is the atmosphere of the conference, which is much friendlier and informal than many others. SMRD takes research seriously, but it does not take itself too seriously.

The SMRD format would not suit all conferences or all countries (for example in a country which had many conferences, such as the UK things are different, in a very large country, like the US, things are different), but I think that the SMRD model is one that many other organisers should look at and decide which elements they should borrow.

How big is online Qual?

This is a question I have been asked a few times this year and at the moment all I have got is a rough guess. The ESOMAR 2009 Global Report suggests that Qual was 14% of all research (by revenue) in 2008, with quant being 80% and ‘other’ being 6%. But it does not break the qual down by online versus face-to-face versus telephone (e.g. tele-depths). The total value of research in 2008 was estimated as US $32 billion, which would put qual at $4.5 billion.

My guess is that globally online qual is about 4%, and I would be surprised if it were actually less than 2% or more than 6%. I formed my estimate by looking at the total value of qual research (from the ESOMAR research), the number of groups run by some of the key suppliers, and then making two big guesses a) what proportion of the total these suppliers constituted and b) what the end charges to clients were.

About 30% of all research spend is in North America, but my feeling is that over 50% and perhaps over two-thirds of all online qual is in North America.

I would be really interested to hear other people’s estimates?

What about MROCs?

At the moment I do not think that most MROCs are counted as qualitative research, never mind as online qual. However, I would argue that the vast majority of MROC-based research is qual, it is after all based on ongoing conversations. From what I see, MROCs are already bigger in revenue terms than ‘conventional’ online qual. If MROCs and other new developments such as much of netnography were added to qual, then qual would be bigger than 14% and online would be a larger part of that larger pie.

The need for a change

Counting values and working out which technique go where matters because influence and innovation often follow the dollars. At the ESOMAR Online Conference in Chicago last year, Bill Blyth of TNS made the point that many of the NewMR techniques (such as MROC) were not currently scalable in a way that could support the industry as we know it (i.e. 50% of the industry is the largest 10 companies, and 80% of it quant). If the new techniques take-off in popularity but do not become more scalable the industry in total could be under pressure, even if some researchers find their companies doing better.

So?

Do people feel 4% is a reasonable estimate for online qual?

Do people think that MROCs should be counted as qual?

Do people think it matters as much as I seem to think it does?

MRSpace getting ready for its re-launch

MRSpace was one of the first experiments in creating a social network for market researchers and, whilst it was interesting, it has been over-taken by a number of other developments. 

For example, the best place for market research discussions seems to be LinkedIn, the best place to hear the buzz is Twitter, and great news services are provided by sites such MRWeb and ResearchLive.

So, we have had a re-think about MRSpace and we think we have identified a number of roles it can usefully fulfil and it is about to be re-launched.

The focus of MRSpace is going to be as a resource for market researchers, with elements for researchers new to social media and elements for researchers looking for signposts to further information.

One of the key changes is that my colleague Sue York has taken over as the editor of MRSpace, which will avoid the lapses that have occurred when I was looking after it, and add the benefit of new views and experiences.

Amongst the features currently available are:

• Resources, for example a new page serving as an introduction to Twitter.
• Events, a list of upcoming events.
• Profile polls, which provides a series of polls to allow researchers, in a fun way, to benchmark themselves against others.
• Topical polls, a series of polls looking at topical issues facing market researchers.
• Blogs, MRSpace is a great place to start blogging, if you are not already a regular blogger, or an additional place to express your thoughts if you are already a blogger.

In addition, MRSpace will remain a social network for market researchers, a place to find old friends and to 'high five' your contacts, but the social networking side is expected to be much less central to MRSpace's role than the usefulness of its resources.

Upcoming projects include: a set of resources for conjoint analysis, book reviews, and a series of surveys and reports looking at issues of relevance to market researchers.

So, over the next few weeks I would certainly suggest that you visit MRSpace and check it out.