The Future Place Blog

Why a good presentation is not a good leave behind/report

I am involved in a fairly long LinkedIn discussion about presentations and presenting (). The drift of the conversation has recently shifted to whether on document can be both a good presentation and good record of the study/report/leave behind/ standalone presentation. My view is emphatically NO!

Below I have reproduced and modified my post to share it more widely.

My view of what a client debrief should include on the screen includes:

1. Only those parts of the questionnaire that contribute to the answer (i.e. don’t display all the questions).
2. Only significant differences get displayed (on quant studies) .
3. Min font size for labels and numbers on chart 16 point .
4. Min font for text 24 point, with major points going up to 48 point (some caveats or warnings might occasionally be as small as 16 point).
5. No bullets, or very few bullets in those rare cases where a list is actually useful 
6. Only show bases sizes where they help interpretation (which is rare). For example, I will say that the data are based on 750 interviews, and I will warn you when I am working with smaller numbers.
7. No detailed methodology/sample section.
8. In most cases I should be able to talk to the slides, introduce my thinking, tell the story, and provide conclusions/recommendations in less than 30 minutes, ideally less than 25 minutes. 

Even when I am presenting heavy quant data, for example choice based modelling, I show very few numbers. Numbers are the tools I use, as an analyst, to find the answers. What the client (in most cases) wants are answers, not the chance to work with the numbers. If I have an insight manager who loves the numbers, I try to have two debriefs; 1) a methodology/analysis version with the insight manager, perhaps lasting an hour (often using remote techniques such as GoToMeeting), and 2) answers version for the insight manager’s internal clients.

When I follow these guidelines (remembering that all clients are unique and consistency is the hobgoblin of small minds) I produce a PowerPoint deck that is great during the presentation, but is not great as a leave behind.

When I am a receiver of presentations, and when I have researched client needs, the sorts of things that I want in the leave behind are:

1. Methodology, sample, and fieldwork dates (99% of my presentations do not put the fieldwork date on the screen, I will simply say things like "and the fieldwork was in June" or "the fieldwork was in the week after the new product launched", but in the leave behind I want to know when the fieldwork was done, whether it was F2F, CATI, etc, who the sample were etcetera. For regular clients, the description of the sample might be as brief as “we interviewed you normal selection of heavy and medium buyers, in May”, on the screen this might be represented by one word, e.g. Who?
2. Data from all of the questions I have paid for. I want these in the leave behind so I can help use the data to answer future questions. 
3. The text of the questions, in case there is ambiguity what a presentation really means. 
4. An executive summary of 500 to 1000 words. 
5. Warnings and caveats. In the presentation I will say those warnings and caveats that are relevant to the audience. In the leave behind, I do not know the audience, so I have to include all the relevant warnings and caveats.
6. More detailed conclusions and recommendations. 
7. Information about my company and how further contact can be initiated (almost every client I have spoken to hates it when a debrief contains several pages of company background, they are fine in a pitch, fine in a leave behind, but 99% of the time they have no place in a debrief) 

 There are many ways of creating a suitable leave behind, for example a revised PowerPoint (particularly one that uses the notes page well), a report, an online hyperlinked document, an interactive model in Excel (with help/documentation features). But a good presentation is not a good leave behind.

Yes, I accept that a good presenter can work with a slide deck that has too much information on the screen, which has text and labels that are too small, but why do it? Why not use the presentation to support the speaker, not the speaker to support the materials? 

SMRD – Much to teach other conferences

Last week I spent two days at the Swedish Market Research Day (SMRD), and annual event organised by Swedish researchers and spearheaded by the indefatigable Henrik Hall. Whilst no event is, or can be, perfect, the SMARD has much about it that others can learn from.

Amongst its key strengths are the following:

1. The conference lasts just one day. Many events last 1.5 days, but in these cases many delegates tend to leave before the end of the event.
2. The SMRD finishes with a really good social event, which is good for networking but also cuts down on the risk of people leaving early.
3. The conference has a mixture of papers presented in English and Swedish, and this year used a mixture of one and two stream sessions to be able to provide an English option throughout the day, making it more attractive to international attendees.
4. The papers were a mixture of local papers, invited international guests, commercial presentations, and a great keynote speaker (Teo Härén).

One consequence of the strength of the programme was that there were about 250 delegates and this in turn meant that there were a really large number of exhibitors.

One innovation at this year’s conference was to hold a workshop on the day before the conference (a Master Class in Multivariate Techniques run by me). This seemed a good idea and I hope that SMRD will make more use of this in the future as it can help with the costs of speakers.

There is a report of the conference on Research –Live. One of the features of the conference that this report picks up on is the atmosphere of the conference, which is much friendlier and informal than many others. SMRD takes research seriously, but it does not take itself too seriously.

The SMRD format would not suit all conferences or all countries (for example in a country which had many conferences, such as the UK things are different, in a very large country, like the US, things are different), but I think that the SMRD model is one that many other organisers should look at and decide which elements they should borrow.

When a Tweetference goes wrong

This post by Danah Boyd is a painfully honest story of how a Tweetference turned into a virtual mugging of a speaker.

I will not tell too much of the story, because I cannot do Danah's story justice, and I strongly recommend you read her post.

Danah was a speaker at the Web2.0 Expo. She was not told in advance that there was going to be a screen visible to the audience showing the tweets that people were making whilst she was speaking, and which she could not see. She got off to a poor start and the crowd starting abusing her on the Tweet Wall, and started laughing at their own comments. The level of abuse got worse and worse as the audience turned into the same sort of mob who get involved in happy slapping.

The big lessons here. I feel. is that Twitter Walls should not be visible to an audience during a presentation. The alternative is to present a pre-recorded presentation and let the presenter make their case during the session - remember Danah could not see the Twitter Wall, nor answer of the jibes.

The people who behaved so badly really ought to take a long look at themselves.

Many thanks to Alison for the tip

Tweetferences - the shape of future conferences?

Are Tweetferences the shape of future conferences. Will we see the attendees Tweeting more and more about what is happening, and non-attendees following the Tweets and perhaps also contributing their thoughts, and access to other resources.

A Tweetference is the wider experience that is created around a Conference where there is a critical mass of attendees Tweeting, and sufficient non-attendees joining in.

I attended the recent EOSAR Online Conference in Chicago and the delegates and followers used the hashtag #esoc to make the event even better, especially the Tweetup on the first evening, which included local researchers who were unable to attend the actual conference.

I was not at the ESOMAR Qual Conference, but it was great to follow it via the hashtag #esoq.

One concern for organisers is probably going to be whether a Tweetference will result in fewer paying visitors. I don't think this will happen. The biggest problem for most conferences is that most of them do not know it is going to happen, a Tweetference will spread the word much wider.

PowerPoint is not the problem, it's the people at the keyboard!

LoveStats has an interesting post and set of comments about the use of PowerPoint for market research debriefs. I have posted my views as a comment, but I thought I would share them here too.

The problem is not PowerPoint, despite what Tufte tends to say, it is the conflation of tasks, resulting in what Garr Reynolds called ‘sliduments’.

Tufte often talks about presenting data in ways that increases the recipient’s ability to process the data. He makes this point very strongly in the case of the Challenger Shuttle Disaster.

However, we need to think about three distinct situations.

1)      1) To analyse the data.
PowerPoint is poor for this. This process is best achieved interactively, using tools that link to the data, even SPSS will do. The recipient needs to be able to ask for alternative views and cuts as part of the process. This process should not happen in a debrief (unless it is a workshop debrief), a debrief is about communicating solutions to problems, not problems.

2)      2) To present the results.
PowerPoint can be really useful here (but I would discard the logos, the footers, and most 3D formats). The presentation is what the presenter and what they are saying – it is not the slides. The PowerPoint simply amplifies what the presenter is saying. The PowerPoint without the presenter is useless. This is often the opposite of what Tufte wants from a chart, he wants to be able to analyse the information in the chart, but that is not what a market research debrief is about. A market research debrief is where we give our findings and recommendations to the client.

3)      3) To document the process.
The report is (should be) a separate document. Thirty years ago I used to write lots of reports, ten years ago I did not write any reports, now I am writing reports again. A report stands on its own and has value without the researcher being there. The report has the questionnaire, the methodology, the key tables, a description of how it was analysed, along with what was found and what was ruled out.

One argument I often hear against providing reports is that it takes too long. But I think this is just a lack or practice. With modern word processing tools, cut and paste, exports, etc I am finding it probably adds 3 to 4 hours to the total process. Just look at how quickly Jeffrey Henning blogs!

 

ps don’t get me wrong, Tufte is right in general, he is very brilliant at what he does, but he does not really understand market research debriefs, he thinks we are trying to convey the data for discussion, we aren’t, we are conveying answers and advice.

The reporting of conferences

I have just finished typing up a report for ESOMAR on the Online Conference that was held last week in Chicago. ESOMAR will publish it shortly and hopefully it will be of value to people who were unable to attend and possibly to some who were there.

However, I wonder if this type of reporting may soon be a thing of the past, because there is now so much dynamic reporting, through blogs and Twitter.

For example, go to Twitter and search #esoc and you will see hundreds of tweets about the conference, many with links to photos and blog posts.

Similarly, go to Google, type in "ESOMAR Chicago Online Conference" and you will see a mass of information. If you use advanced search to restrict it to say the last week, you will tend to avoid the pre-Conference material.

In addition many people will have set their alerts and feeds to pick up information about the Conference, or about some of the speakers.

So, will there be a place for the more traditional write-up in the future? I suspect not, I think we will see a competition for readers amongst a hard core of bloggers and tweeters.

As an example of what is out there check these specific examples:

Jeffrey Henning comment on the MRS site

Jeffrey Henning's blog - this one is amazingly detailed!

Tom Ewing comment on MRS site

More Tom Ewing on the MRS site

LoveStats blog

LoveStats comment on the MRS site

There are many, many more, so please check them out.

ESOMAR's Research Idol on YouTube

Well done ESOMAR, at their recent Congress one of the best sessions was Research Idol, you can see that session on their website, and the videos of the whole Research Idol team are available on YouTube.

What makes a great conference? The Report

In June to August 2009, Brian Singh of Zinc Research, Rick Frank of Dufferin Research, and myself ran a survey looking into what makes a great conference. My take on the results can be downloaded here Download What Makes a Great Conference 2009 .

Brian is just finishing his report and it will be available shortly.

I would love to hear your views about the report and what you think future conferences should be like.

The 'What makes a great conference?' Survey

Following on from the survey about presenting, I have teamed up with Brian Singh of Zinc Research and Rick Frank of Dufferin Research to run a survey on what makes a great conference. So far we have 75 responses from 16 countries. It looks as though the majority of market researchers are a tad conservative in their tastes.

If you would like to take part in the survey click here.

The interim results are available here.

We will be keeping the survey open for a while yet. When the survey is closed we will be preparing and circulating a report.

Update, the sample size is up to 137 with 25 countries covered. A number of interesting findings are beginning to emerge.

The latest news is that there are 167 interviews from 29 countries. The survey is likely to stay open until the end of July, with the report following a couple of weeks later.

We will be closing the survey shortly and them producing the reports. At the moment there are 81 completes from 30 countries. One early indication is that the UK is more likely than other countries to favour a one day conference, and more likely to favour 15 minute presentations. The one day preference may be due to geography (most UK researchers are within easy reach of London), but the 15 minutes may be down to attention deficit?

What makes a great conference?

Following the recent success of our  study about market research presentations, download a copy here or the Research article here, I have teamed up with Zinc Research Dufferin Research to conduct a larger project looking into what makes a great conference.

As with earlier project the first step is a survey, which will link to live updates of the data for people taking part, and then to a report that will be published and made freely available.

You can take the survey inside Facebook by using this link

Or you can take the free standing survey by using this link

We have also created a discussion space on MRSpace to allow the results to be discussed further. The discussion will include a link to the results shortly.