The Future Place Blog

Running the Saints Way for Coppafeel

Every year I like to set myself a physical challenge and I like to utilise that challenge to raise money for charity. This year I am going to run the Saints Ways in Cornwall, from Padstow on the North coast of Cornwall to Fowey on the South Coast. We are still tweaking the route but it is likely to be just under 30 miles (about 48 KM).

The charity I will be running for is CoppaFeel, a breast cancer awareness charity organised by two friends of my daughter. Even if you do not follow my progress, even if you do not dontate, please read Kris's story, read how she developed breast cance at 23 and the problems she had getting her GP to believe her. Kris and her sister Maren created CoppaFeel to raise awareness of breast cancer amongst younger people.

I am going to be running the Saints Way on 17 August, and I will be blogging and Tweeting about my progress, and hoping to persuade my longsuffering family to support me, again.

You can make a donation via my page on JustGiving 

Could SlideRocket replace PowerPoint as 'the standard'?

Following my earlier post about why PowerPoint will remain the delivery standard for the next few years I was asked about SlideRocket.

Well, I still need to evaluate SlideRocket, but from an early view of it I would say it looks interesting, suitable for some needs, but certainly not a replacement deliverable for most clients.

Why do I say that? The main reason is that SlideRocket is flexible, dynamic, and web-enabled. All of these sound like good things, and they are good things for some purposes, but not for delivered insights.

One of the most important things about research findings is that they need to be static. If I give you a presentation and a set of recommendations on Monday, you may write a summary for your boss on Tuesday, and forward them to her on Wednesday. You do not then want to find out that we change/improve the report on Thursday, making your summary completely nonsense. In the way that most companies work at the moment, the insight/findings presentation is a handing over of the baton, the agency has committed to these as the findings and the ownership of the thinking has moved to the client. The client may reduce, expand, or amend the findings as they see fit. However, if they do change the findings they are no longer the delivered findings of the vendor.

SlideRocket has a couple of other features that will be an advantage in some situations, but a problem in terms of delivering results to a client. The first is that SlideRocket is web-linked. Many, perhaps most, clients' IT and compliance departments do not want results that link to sources that are on the web, for security reasons. One interesting feature of SlideRocket is that you can track who sees your slides. I can see this as being really useful in some situations, but most clients are not going to want you to know who is seeing your materials.

Another potential weakness in SlideRocket is the metaphor it is using, namely one of co-creation and collaboration. In a growing number of research suppliers the process of creating deliverables is shifting away from the researcher towards design professionals, and most of the design professional seem to have very strong views about not letting researchers modify or edit the slides. By contrast, the metaphor SlideRocket extols is one where groups of people work together to produce the output.

The pricing of SlideRocket is a potential problem. The most expensive way to buy PowerPoint is as a standalone fully-featured package, which costs about $140. Most people will buy PowerPoint as part of a package, so its cost is much less. The standard SlideRocket package costs $24 per month, which means that over three years the cost would be about $864 - although I imagine the enterprise version would work out cheaper per user. This extra cost may be worthwhile for users who take full advantage of the web/networked facilities of SlideRocket, but not otherwise.

As I said, I have not tried SlideRocket yet, and I look forward to giving it a try. If the tour and video on its website are indicative of what to expect then I can see several situations where it might be useful. The first and most obvious thing is that because it is not PowerPoint it has the chance to stand out, to show that somebody is thinking and trying something new. The second use I can see is for marketing or promotional presentations, for example a presentation showing how a software package might be used. Because SlideRocket is dynamic the version of the presentation that people are looking at would always be the most up-to-date, and the tracking facility would help the creator check which slides were reviewed the most, and the questioning facility would allow the sales presentation to be more of a conversation as less of a lecture (to quote SlideRocket).

Another area where SlideRocket might be useful is in the process of creation of presentations, however, this will probably require a change in the current trend, i.e. towards collaboration and away from the trend of using specialists only.

It is probably also worth pointing out that all of the above relates to a presentation that has been given to the client as a deliverable. The rules for a presentation used as part of a face-to-face presentation are even more in PowerPoint's favour. Good debriefs and presentations are rehearsed, and you can't reliably rehearse a web-enabled, dynamic presentation - I am sure that for a presentation SlideRocket can be frozen, but then it is much more like PowerPoint anyway.

Will anything replace PowerPoint soon?

In the ASC LinkedIn Group AJ has posed the question "Complete this sentence. 'Over the next 3 years, powerpoint will be replaced as the default method of delivering results and insight to clients by....?)"

I have posted my reply in the LinkedIn group, but I thought I would also share my thinking here.

It is hard to imagine anything replacing PowerPoint in the foreseeable future.

There are some key elements that a system for delivering results and insights has to have to be a standard, including:

1) Available technology, the system needs to be able to produce PDF files and a file format that can be widely read by clients (e.g. Excel, Word, or PowerPoint 1997-2003), or WMV

2) The standard deliverable must not need to access the Internet, the intranet, or supporting files

3) The standard deliverable needs to be static, so that different clients see the same view, this rules out deliverables like What-if models, GapMinder, interactive tables.

4) The standard deliverable needs to be supported by tools and skills in the supplier side, to ensure that the deliverables are cost effective.

In the foreseeable future I do not see anything coming along that does what PowerPoint does better, cheaper, or faster.

I think there will be some great developments in reporting tools, but most of these will be for analysts, not end clients. I think that interactive infographics will facilitate analysts gaining better insights. But when that insight needs to be shared with 40 non-specialist clients spread across 20 offices and perhaps 8 countries, it needs to be turned into something like a photo, or a book, or a newspaper, or a report, i.e. something static, reliable, and which requires relatively few skills to read and interpret.

Most end-clients want results and findings, they do not want a method of finding the results for themselves. Most interactive systems are a method of finding results, they are not the findings themselves. 

One of the problems for vendors is that many clients say they want a system that is really easy to use and is at the same time flexible. But what they mean by flexible is inescapably linked to harder to use. As an example, SPSS is a relatively powerful way of storing quant data, it is relatively flexible in its analysis, and it is pretty easy to use. However, most end clients would not dream of using SPSS to access their results.

BTW, when I say end-clients I do not mean insight managers inside client companies, many of whom are perfectly willing and capable of working with interactive analysis tools. I mean the people whose full-time job does not include analysis, for example: brand managers, finance teams, creatives, designers, i.e. people who would like to be informed by the outcomes of research, but who do not have the time to learn how to do or analyse research.

In the longer term I think we will produce a reporting paradigm that reflects the fact that different people learn in different ways, some people like pictures, some like numbers, some like written text, some like to hear it, some like videos, and some like interactive learning. The results product would include the raw and processed data (but out of site to 99% of users), the analyst's findings would be linked to the data and would be capable of being presented in text, infographics, in colour or black and white (remember colour sucks for colour blind people), in sound rather than images (images suck for people with visual impairment), and with differing degrees of abstraction (a senior exec might say - show me it in 100 words, a doubtful exec might say about a particular finding "substantiate that"). Such a system would also seek to have a natural language processing system for queries, perhaps a bit like WolframAlpha, so the exec could simply ask, "Which concept performs best amongst our younger customers?" and the system would report back the answer.

Is the social survey an endangered species?

A couple of weeks ago I was invited to London to take part in a debate organised by NatCen, the UK's leading independent social research organisaiton. The event was chaired by Radio 4's Mark Eaton, the case for the social survey was presented by Penny Young, and I presented the contrary case. Below are three recordings from YouTube, Mark's introduction, Penny's presentation, and my response.

Mark Eaton - Chair

 

Penny Young, CEO NatCen

 

Ray Poynter, Author The Handbook of Online and Social Media Research

Why do most new products fail? An Olympic analogy.

There has been a long-running discussion in the Market Research Bulletin LinkedIn group about why 80% of products fail. There have been 291 comments so far in the 3 months the discussion has been running. Some of the comments put forward good ideas, some challenge/defend the estimate of 80%, and a few are what US Senator John McCain might call ‘bizarro’.

Whilst there are clearly many reasons why products fail, I think there is a key one that we need to keep in mind. There can be only so many winners. If the number of competitors keeps increasing faster than the chances to win, then the failure rate grows.

As an example, consider the 1896 and 2004 Athens Olympics. In 1896 there were about 43 gold medals issued (* in 1896 first place was a silver medal, but let’s ignore that). In 1896 about one-in-six competitors got a gold medal (slightly fewer since some competitors got more than one). Being very cruel, we could call this a failure rate of about just over 80%.

In 2004 there were about 301 medals on offer, but only about 1-in-35 competitors got a gold medal. Although the ‘market’ had grown by about 700%, the number of competitors had grown by over 4000%.

Think about success from the point of view of the consumer. How many new types of jam, bread, cheese, sport, television channel etc are you going to become a fan of in any given year. There are a limited number of ways to win. The number of ways to ‘win’ typically grows slower than the number of new products being launched.

Market research, like good training, is a process where the chance of winning is increased, but it can never ensure success.

 

Radio NewMR strides on

Following the success of last week’s show the Radio NewMR pilot will broadcast its second show at 10am New York, 3pm London time, with special guest Paul Child and Sean Copeland. If you’d like to listen to the show find out more/register at the NewMR Website.

If you’d like to appear on the show please email me at [email protected]

 

Should the social survey be an endangered species?

On Wednesday I am scheduled to emulate Daniel going into the lion’s den when I debate the topic “Is the social survey an endangered species” with Penny Young the CEO NatCen (Britain’s leading centre for independent social research). Or, perhaps since I will be the bad guy, perhaps I am a lion entering a den of Daniels?

I am not going to propose that all social surveys should end, nor will I suggest that ALL of the coalition Government’s cuts in social surveys are justifiable. But, I do believe that in the future there ‘should’ be fewer surveys. I will keep most of my powder dry until Wednesday, but I will share some of my concerns in the context of a recent study by NatCen. The study was “Estimating the prevalence of young people absent from school due to bullying”

Now, let me start by acknowledging that bullying is a major problem, at its worst it leads to suicides, self-harming, and psychological damage, and there is evidence (from the TellUs 2009 survey – a study which the current Government has scrapped BTW) that about half of children experience some form of bullying, and about one-in eight have experienced bullying in school in the last six months. I have spent 28 years as a councillor and had a large number of people contact me about bullying, as a parent I have had a good level of exposure to what happens in schools, and in writing this article I have refreshed my memory by speaking to somebody who acted as a peer group councillor to young people who had been bullied.

So, bullying is a problem. We need to understand it if we are to diminish it. But, is a survey a sensible way of understanding bullying? And, even if it is, was this the right sort of survey?

If you click on the link for the survey you should be impressed by the level of documentation, it is exemplary. It contains the objectives, the research sponsor, the questionnaire, the methodology (which included a good piloting stage), and summaries of the data as well as the findings.

However, as a study I think there are several problems with it:

1. The first problem is epistemological. The study seeks to estimate bullying in terms of numbers, and within that it seeks to translate it to the number of children losing out on schooling. It is a survey of adults about missed schooling by children. There are no definitions of bullying in the questionnaire (for good methodological reasons) which means the interpretations will be vague.
2. The second issue is that the study appears to make a value judgement that being electively home schooled is a bad thing. It adds together the number of children who have missed school through bullying to those who are being home schooled where the reason (or a reason) quoted is bullying. I am not a big fan of home schooling, but society and the law have both defined it as a right, whereas being absent from school is clearly a problem. Bullying may be interesting in both cases, but the outcomes should not, IMO, be simply added together.
3. The next problem relates to what Mark Earls talks about in his book Herd. We are all unreliable witnesses to our own motivations. Asking people what was the reason they did something is frankly flaky, asking that which of those reasons was the main one is flaky, and asking them to do it on behalf of one of their children, thinking back for a considerable amount of time is really flaky.
4. The study is statistically unreliable, despite its claim to offer error limits of which it is 95% confident. The survey had a response rate of 17%. The study quotes 95% confidence of the population, but does not make clear that the population is a hypothetical population comprising that part of the total population willing to take part in this sort of study. None response bias could mean that the 82% who did not reply had very different experiences. The study did do better than most commercial studies, they looked at the basic demographics and they did not indicate any great differences between the responders and the population, but the issue is not demographics, it is the experience of bullying.
5. The responses in the study indicate a large gap between the parents’ perceptions and the sampling frame. The survey was sent to a sample of parents whose children had been absent for 28 half days (which might mean 14 days, but it could also be a mixture of full days and half days). However, when the respondents answered how many days their child had been absent from school nearly half said less than 14 days (interpolating the 11 to 15 day break). This either means the frame was wrong, or that parents memories/knowledge is very hazy, or some combination of the two. If the scoring of a simple variable like number of days absent was wrong in 50% of cases, then surely the description of motivation is likely to be dodgy?
6. The final issue is that the study was sponsored by the Red Balloon Learner Centres and a link to their website was included in the letter sent to parents. Red Balloon is a great organisation; it works to achieve the recovery of bullied children. One of the statements on its website should remind all of us how bad bullying can be “We provide an 'intensive care' full-time education for children aged between 9 and 18 who are unable to go to school because they have been severely bullied. At least half of the students we take have attempted or seriously considered suicide.” However, having it as the declared sponsor of the project could be a source of bias.

The net result of the survey is that Red Balloon can issue a press release stating that over 16,000 children are absent from school due to severe bullying and it has a credible source of information to go to funding agencies, administrators, and politicians with.

But, have we really leaned very much about bullying from this study? Some children are bullied and never miss school, some miss school for a range of reasons, one of which is bullying, but the other reasons may also be priorities.

The implication in the headline from Red Balloon is that this bullying happens at school and happens from other pupils. The TellUs study suggests that a large proportion of bullying happens outside of school, and my caseload indicates that a small but noticeable amount of perceived bullying is from the teacher to the pupil. Red Balloon already know about the damage that bullying can do and that it can lead to children being absent from schools. Does an estimate (even if it were accurate) really tell us as much as case studies? Does it really matter if 10,000, 16,000, 30,000, 78,000 or even 5,000 are absent? Doesn’t it need dealing with irrespective of the fine detail of the numebr?

The press release from Red Balloon describes the bullying as severe, but this was not explored by the study. At one level, one could say that any bullying that led to a child missing school is severe, but that is simply looking at the outcomes. In my time as a councillor I have encountered two cases of serious sexual assault, several of physical harm, and one of a child being imprisoned in a shed whilst youths threw lit matches into the shed – by any stretch of the imagination these were severe. However, I have also met parents whose child felt bullied because another child had stopped being their friend or had not invited them to a sleep over. This sort of behaviour can seem like bullying to the recipient, it can cause actual damage, but it may not be perceived that way by the other parties in the event.

My general thesis is that I think we resort too easily and too readily to surveys.

There used to be a mantra that if you can’t measure it you can’t manage it, but more recently we have begun to understand that if you can measure it then it probably isn’t important (i.e. it is the unmeasurable things that really matter - check out Elizabeth Barrett Browning when she counts the ways she loves).

A study which produces a clear but only vaguely defensible number is not, in my opinion, likely to be as helpful as a range of other solutions. I am sure that Red Balloon will have done some of the following, but I will list them all the same:

1. Case studies of bullied children
2. Discussion with schools about bullying, getting them to assess the form and the prevalence and its consequences
3. Discussions with peer counsellors, asking them to assess the form and prevalence of bullying and its consequences
4. Discussions with children who appear not to be bullied, asking them to assess the form and prevalence of bullying and its consequences – I suspect that in many cases they will have direct experience of bullying
5. Looking at the discourse and semiotics of bullying to understand it better, to understand how it is described and experienced.
6. Social media monitoring research, looking for specific phrases, terms, and issues.

Instead of a survey, an estimate could have been formed from talking to schools and pupils, probably less accurate than the survey commissioned, but it would have indicated that thousands of children miss school through bullying, it could have contained case study material to bring the issue alive, and perhas a bit of social media monitoring to show levels of concern. For example entering Bullying into something as simple as Google Insights suggest that more peope are searching for information on Bullying in the Autumn, that the lead term is "anti bullying", that the regions with with highest searches are Northern Ireland and Wales, and that the top rising search term is cyber bullying.

If you would like to join Penny Young and me for the debate about the future of the social survey please check out the NatCen website.

Ps and BTW, my gut feel is that 16,000 seems too small a number. There are 2.8 million children aged 11 to 15 in secondary education. This would imply about one child in 175 was missing schooling because of bullying. In the main school in my area this would equate to  4 or 5 children (if my school was typical). But, thinking about my caseload, talking to a fellow councillor, and referring the question to my peer group counsellor contact, our feeling was that the numbers would be more like 20 - 40 (and possibly quite a bit more).  Interestingly, if the study had based its main estimate on the impact of bullying on whether bullying was mentioned as A reason, as opposed to being the main reason, the estimate nationally would have been 78,000, which would imply 23 from my local school, which triangulates better with our local experience.

 

Market Research Presentations are Special - #PresentingMR

For the last few years I have been trying to significantly improve my presentations and have as part of that process been running a growing number of workshops on presenting. This post, and several of the posts that will follow it address issues that I think market researchers need to tackle to improve their presentations.

When market researchers talk about presentations, and in particular how to improve their presentations, four names crop up with regularity:

• Edward Tufte, the originator of the term death by PowerPoint
• Seth Godin, “No more than six words on a slide, EVER”
• David McCandless, the Infograprachics man behind InformationIsBeautiful
• Garr Reynolds, author of PresentationZen

All of these people are great presenters/communicators and have taken the art forwards. But there is a problem for people trying to learn from them, they disagree massively with each other in terms of what they recommend. They disagree because they are doing different things, and to make life harder for market researchers, none of them is doing what market researchers do in a debrief.

To quickly review what these four experts are saying:

Edward Tufte is trying to convey the maximum amount of data to the audience, so the audience can process it and come to a conclusion. Indeed he recommends that before a meeting, hand out the data as pages of text and numbers on A4, ask everybody to read it, and then have the meeting. In most market research debriefs the client is paying the researcher to present their findings, not to hand over the data for the client to discover the findings.

Seth Godin says “The purpose of a presentation is to change minds.” Godin can restrict himself to less than 6 words per slide, no numbers, and stark images because he is 99% of the message. Godin says XYZ and people believe it (or not) because ‘Seth says so’, not because of some research, chart, or data. Market researchers are there to convey insight to help a better business decision be made, not to make the decision. The basis for the recommendation from the researcher is the research, so it needs to play more than 1% of the role in the debrief (I will discuss how much of a role in later posts).

David McCandless produces complex and engaging infographics, the best of which convey more information than a traditional graphic and do so in ways that are stimulating and engaging. However, most of these infographics are not tools for a debrief. They often work better for a solitary reader, for somebody who has time to look, to think, and mull over the information being shown.

PresentationZen. I have to admit, I am a major fan of Garr Reynolds and I find his material easier to use in a market research context than that from say Tufte and Godin. However, Reynolds is used to an environment where there is time and budget to get the presentation right. At one point in his book, PresentationZen, he makes the comment about spending several extra days on the slides to get them just right. However, market researchers often have only one or two days for the analysis, the story finding, and the creation of the presentation – this requires the researcher to cut corners.

Why are market research debriefs special?

There are several feature of a classic market research debrief that make it quite unlike the sorts of situation that the likes of Tufte, Godin, McCandless and even Reynolds are talking about. Market researchers can still learn from these sources, but they need to adapt the techniques to the needs of the situation and the audiences.

The key differences, in the context of a typical, 30-60 minute, end of project debrief are:

1. There will be different groups within the audience, ranging from an insight manager who is interested in the detail to a senior person who just wants to know the key findings and action points.
2. The researcher will only have a limited amount of time to prepare the analysis, the story, and the presentation – ranging from one day to perhaps ten days.
3. Most presentations will only be delivered once, reducing the ability of the presenter to fine tune the material.
4. Some clients want the presentation to double as the deliverable.
5. Some clients have a strong sense of what a debrief should look like, e.g. background, objectives, findings, and then conclusions.
6. A large part of the legitimacy of the findings is based on the research, so the research will have to play a non-negligible part in the debrief.
7. Because of the lack of time and because of the need to include the research (some of it) in the presentation it is normally not possible for the presenter to memorise ALL the key points and facts, which results in some of them having to be spelled out on the slides.

In the next few posts on presenting I will cover how market researchers can address these special requirements. By borrowing and blending from the experts and by adopting some sensible tricks of the trade market researchers can massively improve their presentations.

 

Radio NewMR – an experiment in collaboration

Over the next five weeks NewMR is going to be trialling a new idea, a weekly 25 minute broadcast exploring the news, views, comments, and opinion - which we have dubbed Radio NewMR. The sessions will be hosted by a combination of Ray Poynter, Leonard Murphy, and Andrew Jeavons. Each broadcast will involve several guests addressing key topics of interest, start-ups, and in some cases just something guests want to get off their chest.

The five broadcasts are going to be at 3pm London time, 10am New York time, on the next five Tuesdays, starting with Tuesday 19^th July. For this pilot we will be using the GoToWebinar platform, in the future we want to look at platforms designed for radio shows.

To find out more about Radio NewMR, to see the upcoming programme, to sign-up to attend, or to suggest yourself as a guest on the show visit the NewMR website.

 

A chance to brush up your quant

On Thursday NewMR has its final topic specific event before the Festival, an introduction to a range of Advanced Quant techniques, including pricing research, predictive markets, choice modelling, structural equation modelling, and more.

The three sessions are outlined below, you can register to attend at NewMR.org

Asia/Australia Session - Sydney 3 pm to 5.15 pm (Auckland 5pm, Singapore 1pm, Mumbai 10.30 am)

• Sue York, An Overview of Advanced Quant Techniques
• Ray Poynter, The Future Place, The Benefits of Conjoint Analysis and Choice Modelling
• Brian Fine & Con Menictas - The Superiority of Panel Research - A Fast Food Choice Modelling Example
• Scott MacLean, An introduction to Reflective and Formative Structural Equation Models

 

Europe/Africa Session – London 11 am to 1.15 pm (Mumbai 3.30 pm, Paris 12 Noon, New York 6 am)

• Ray Poynter, The Future Place, Improved BPTO
• Dirk Huisman, SKIM, The Challenges and Benefits of menu based choice modelling
• Stefan Binner, Adaptive Choice Based Conjoint
• Will Goodhand, BrainJuicer, 9,000 concepts later: how the wisdom of the crowd compares with monadic screening

 

Americas Session – New York 2 pm to 4.15 pm (London 7 pm, Seattle 11 am)

• Ray Poynter, The Future Place, An Introduction to Hierarchical Bayes
• Juan Tello, SKIM, Product Optimisation
• Jeffrey Henning, Affinnova, Survey-based pricing research techniques
• Will Goodhand, BrainJuicer, The pursuit of Happiness: can emotions predict in-market performance?

 

Event Sponsor – Affinnova

The Advanced Qaunt event has been sponsored by Affinnova and you can see an interview with Tamara Barber from Affinnova below.