The Future Place Blog

PowerPoint is not the problem, it's the people at the keyboard!

LoveStats has an interesting post and set of comments about the use of PowerPoint for market research debriefs. I have posted my views as a comment, but I thought I would share them here too.

The problem is not PowerPoint, despite what Tufte tends to say, it is the conflation of tasks, resulting in what Garr Reynolds called ‘sliduments’.

Tufte often talks about presenting data in ways that increases the recipient’s ability to process the data. He makes this point very strongly in the case of the Challenger Shuttle Disaster.

However, we need to think about three distinct situations.

1)      1) To analyse the data.
PowerPoint is poor for this. This process is best achieved interactively, using tools that link to the data, even SPSS will do. The recipient needs to be able to ask for alternative views and cuts as part of the process. This process should not happen in a debrief (unless it is a workshop debrief), a debrief is about communicating solutions to problems, not problems.

2)      2) To present the results.
PowerPoint can be really useful here (but I would discard the logos, the footers, and most 3D formats). The presentation is what the presenter and what they are saying – it is not the slides. The PowerPoint simply amplifies what the presenter is saying. The PowerPoint without the presenter is useless. This is often the opposite of what Tufte wants from a chart, he wants to be able to analyse the information in the chart, but that is not what a market research debrief is about. A market research debrief is where we give our findings and recommendations to the client.

3)      3) To document the process.
The report is (should be) a separate document. Thirty years ago I used to write lots of reports, ten years ago I did not write any reports, now I am writing reports again. A report stands on its own and has value without the researcher being there. The report has the questionnaire, the methodology, the key tables, a description of how it was analysed, along with what was found and what was ruled out.

One argument I often hear against providing reports is that it takes too long. But I think this is just a lack or practice. With modern word processing tools, cut and paste, exports, etc I am finding it probably adds 3 to 4 hours to the total process. Just look at how quickly Jeffrey Henning blogs!

 

ps don’t get me wrong, Tufte is right in general, he is very brilliant at what he does, but he does not really understand market research debriefs, he thinks we are trying to convey the data for discussion, we aren’t, we are conveying answers and advice.

The reporting of conferences

I have just finished typing up a report for ESOMAR on the Online Conference that was held last week in Chicago. ESOMAR will publish it shortly and hopefully it will be of value to people who were unable to attend and possibly to some who were there.

However, I wonder if this type of reporting may soon be a thing of the past, because there is now so much dynamic reporting, through blogs and Twitter.

For example, go to Twitter and search #esoc and you will see hundreds of tweets about the conference, many with links to photos and blog posts.

Similarly, go to Google, type in "ESOMAR Chicago Online Conference" and you will see a mass of information. If you use advanced search to restrict it to say the last week, you will tend to avoid the pre-Conference material.

In addition many people will have set their alerts and feeds to pick up information about the Conference, or about some of the speakers.

So, will there be a place for the more traditional write-up in the future? I suspect not, I think we will see a competition for readers amongst a hard core of bloggers and tweeters.

As an example of what is out there check these specific examples:

Jeffrey Henning comment on the MRS site

Jeffrey Henning's blog - this one is amazingly detailed!

Tom Ewing comment on MRS site

More Tom Ewing on the MRS site

LoveStats blog

LoveStats comment on the MRS site

There are many, many more, so please check them out.

ESOMAR's Research Idol on YouTube

Well done ESOMAR, at their recent Congress one of the best sessions was Research Idol, you can see that session on their website, and the videos of the whole Research Idol team are available on YouTube.

What makes a great conference? The Report

In June to August 2009, Brian Singh of Zinc Research, Rick Frank of Dufferin Research, and myself ran a survey looking into what makes a great conference. My take on the results can be downloaded here Download What Makes a Great Conference 2009 .

Brian is just finishing his report and it will be available shortly.

I would love to hear your views about the report and what you think future conferences should be like.

The 'What makes a great conference?' Survey

Following on from the survey about presenting, I have teamed up with Brian Singh of Zinc Research and Rick Frank of Dufferin Research to run a survey on what makes a great conference. So far we have 75 responses from 16 countries. It looks as though the majority of market researchers are a tad conservative in their tastes.

If you would like to take part in the survey click here.

The interim results are available here.

We will be keeping the survey open for a while yet. When the survey is closed we will be preparing and circulating a report.

Update, the sample size is up to 137 with 25 countries covered. A number of interesting findings are beginning to emerge.

The latest news is that there are 167 interviews from 29 countries. The survey is likely to stay open until the end of July, with the report following a couple of weeks later.

We will be closing the survey shortly and them producing the reports. At the moment there are 81 completes from 30 countries. One early indication is that the UK is more likely than other countries to favour a one day conference, and more likely to favour 15 minute presentations. The one day preference may be due to geography (most UK researchers are within easy reach of London), but the 15 minutes may be down to attention deficit?

What makes a great conference?

Following the recent success of our  study about market research presentations, download a copy here or the Research article here, I have teamed up with Zinc Research Dufferin Research to conduct a larger project looking into what makes a great conference.

As with earlier project the first step is a survey, which will link to live updates of the data for people taking part, and then to a report that will be published and made freely available.

You can take the survey inside Facebook by using this link

Or you can take the free standing survey by using this link

We have also created a discussion space on MRSpace to allow the results to be discussed further. The discussion will include a link to the results shortly.

Presenting article published by Research magazine

UK research magazine, Research, published my article on Presenting in their last issue and it is now available on their website here.

My next project will be looking at what makes a geat conference and the survey should be launched shortly.

Pie Charts are usually a bad idea

Pie charts are usually a bad idea. Although they superficially appear to be a way of expressing data visually, in reality they depend heavily on the labels and values to make any sense. Once a pie chart moves past about 3 elements it becomes very difficult to judge the different values, by the size of the sectors, especially if two sectors of interest are not adjacent. And, don't let me begin to start ranting about using two or more pie charts and then challenging the reader to draw an inference!

I suspect that the popularity of pie charts has something to do with the enthusiastic way they are taught in our schools. For teachers pie charts allow simple data to be combined with practice using angles and protractors. This makes for a great bit of teaching, but leads us to the impression that pie charts are useful.

A recent post on Junk Charts has a great example of the problems that real data create when people try to illustrate them with pie charts.

Spuriously accurate pessimism

Today, the BBC report that media agency Zentih Otpimedia have forecast that media revenues could fall by 6.9%. Whilst that is bad news, I was even more struck by the accuracy of the prediction. The way the article is written it would appear that there is a good chance of a 6.9% fall, but not much risk of a 7% fall.

The article also reported that TV’s share of advertising would increase to 38.1%, but that this would be a fall in absolute terms of 5.5%.

Strip the accuracy away and it all seems very plausible. For example, ad revenues might fall by 5% to 10%, TV’s share of the total could increase, but this still might  result in TV revenues in absolute terms.
As part of my ongoing project, looking at presentations, I am struck by how rarely decimal places are of value, and that it is even less common for them to help the reader make intuitive sense of the message. Even when the decimal places are accurate, they are usually useless, but when they represent spurious accuracy they are positively harmful.

Don't try these charts at home

Sometimes we can get guidance about what to do by looking at examples of what not to do. That simple idea is the basis of the JunkCharts site.

This link takes you to a slideshare presentation that has been very brave in its use of innovative angles and perspectives in its charts, unfortunately most of them do not work, and some are seriously hard to read.

Although there are some big problems with the charts, there is also a simple, easy to fix problem. The pie and bar charts show value labels, and display them to two decimal places. These labels would be much clearer without any decimal places, and the labels could then be in a larger font.