The Future Place Blog

The Pull/Push Dichotomy - or why most reportals fail

One of the biggest differences between reporting methodologies is whether they are push or pull. A push technology is one where the vendor pushes information to the client, an approach which tends to result in more of the information being read. The classic push approach is email, where the vendor sends the results, or a link to the results, to the client, with an expectation that it will be acted on.

A pull technology is one where the client collects the information when they wish to. Examples of a pull technology include reportals, FTP sites, and the online reporting built into many Internet based data collection systems. Systems which deliver the power to search and trawl the data, such as Quanvert, are also examples of pull techniques.

At first glance, the pull approach seems more client friendly, and likely to be of more utility. The client knows what he or she needs, they can collect exactly the data or analysis they want, and they are not burdened with reports, tables, or data streams which are not relevant to their current needs.

However, most pull systems have been fairly unsuccessful. One problem is that many clients do not have the time to visit a number of different provider sites to access the various elements they might need. Clients are often unsure of exactly which bits of large data systems they need to access, they do not want to learn several different procedures (nor store a raft of passwords) in order to answer an urgent question. Off the record, many vendors have commented that when they have provided their information in a pull environment, some client never access it, rendering it very poor value, and making the project less likely to be funded in the future.

Push systems, such as emailing results to clients, are based on vendors wanting clients to access information. The vendor tends to know what is interesting in the information, and has a financial interest in ensuring that the client is aware of, and reads, interesting things in the data/information.

Pull systems seem to be most effective when they are delivering access to data rather than insight, and where the information is necessary to the client (as opposed to desirable), and where the amount of information is so large that the client might be over-burdened if the information was sent straight to them. Examples that work well with pull systems are audit data (such as that provided by AC Nielsen), news services (although even here people are starting to use things such as RSS to simulate a push service), and process information (such as accounts, sales, and transactional data such as number of visitors, number of page views etc.).

Push systems appear to have a distinct advantage over pull systems when the material being delivered is more insight focused, rather than data driven, and in cases where the information is ‘nice to have’ rather than essential – something which is often the case with market research data.

Some services provide a hybrid of pull and push. The underlying data might be made available via a reportal (i.e. a pull modality) but key findings are automatically emailed to the client, perhaps as dashboard results.

Using Excel to model survey data, in Munich

The SKIM / Sawtooth Conference is underway in Munich, Germany and today saw the first outing for a new workshop I am offering looking at techniques for using Excel to model survey data. The workshop starts with a simple model constructed from ranking data, and builds up to handle full-blown conjoint and choice models.

A copy of the materials will appear on this blog over the next few weeks, just as soon as all the things I have learned from running the course twice today have been implemented. One unwelcome surprise was the discovery that on German machines the names of functions are changed. So, when I was asking people to type in “=Countif(“, the Germans in the room had to type “=ZÄHLENWENN(”. If anybody from Microsoft is listening, I can tell them that 100% of the Germans I spoke to do NOT like the fact that the function names have been translated. To make matters more confusing the names of functions in VBA (Visual Basic for Applications) have not been translated.

Thursday is another day of workshops, whilst Friday will see the presentations. I am presenting a paper on TURF on Friday, and I am particularly looking forward to a paper by Stefan Binner on hit rates.

Munich is an interesting place to have a marketing sciences type conference at this time year. In the evenings the town centre is empty because of the Oktoberfest, but the hotels are both full and expensive.

Apple need to ‘get with the programme’

Since the late 1970s, one of Apple’s strengths has been its appeal to creative people, and since the 1980s it added to that strength by not being IBM. However, it could be putting all of that at risk with its predatory behaviour over the term pod.

Engadget report that Apple has sent a ‘cease and desist’ letter to a company called Podcast Ready who have a product called myPodder, which provides an automated way of downloading, so called, podcasts onto portable devices.

If Apple really do intend to run around trying to ‘own’ the use of all words beginning with pod, then they are likely to create an immense amount ill-will. Indeed, there could even be a backlash in terms of using an alternative term, perhaps Creative would be happy if everybody started referring to Zencasts?

What Apple appear to be forgetting is that on the web everything is visible, and everything gets discussed. The lawyers and bean counters at Apple have to realise that they are part of the brand management and brand building every bit as much as the designers and the marketers.

The Internet in 2020

The BBC have reported a study by the Pew Center looking into what the Internet will be like in 2020. The survey interviewed 742 experts (funny they did not contact me!) and collected their views on scenarios and priorities.

The key points were

• 56% thought that by 2020 the Internet would be a low-cost thriving network, with cheap, mobile access available almost everywhere on the inhabited planet.
• 42% thought English would displace other languages.
• 42% thought the use of automated bots for things like surveillance and security would create dangers that would not be easily stopped or remedied.
• 46% thought that most aspects of life would be transparent to all, and that this would make life better.
• 56% though virtual reality would create new tele-working opportunities for some, but dependency and addition for others.
• 52% thought we would see a “flat world” where non-geographic networks of people were more important that coutries and conventional geo-political entities.
• 58% thought that some refusniks and/or luddites would engage in acts of terrorism, whilst others would choose to live outside “the grid”.

For me, the strangest thing about the report is that it does not mention China. Soon China will have more Internet users than any other country; it is possible that within a few years a majority of Internet users in the world will be Chinese (and India plus China will certainly pass 50% of all users). The Chinese economy is on track to be one of the dominant economies, and its consumption of commodities is already restructuring markets.

In my view the issues raised by the ‘experts’ largely relate to the next 5 years, beyond that the impact of China will probably outstrip any of the scenarios they review.

A Wiki for Market Research

Mike MacLeod has started what he describes as a colossal project, and I he think he is right! EverythingMR is intended to be a wiki for all aspects of Market Research. Initial topics include: Organizations, Suppliers, Opinion, Legislation, and Methodologies.

Several people have been calling for just such a resource and there is no doubt that a wiki is needed. I will certainly be adding some copy and I encourage you all to check it out.

Ethics and Word of Mouth

Womma reports that the US Government’s FTC (Federal Trade Commission) is to host a seminar on protecting customers in the WOM world. The seminar, which will take place on 6 November, is intended to open a dialogue between Government, business, and technology players on emerging viral marketing techniques, as well as the proper role of advertising in social networks and user-generated media.

For more information about the seminar click here.

Now that WOM is becoming a larger part of market research we need to start a debate about the ethics. Is it alright to pose as somebody else in order to enter a social network? Is it alright to lurk in discussion rooms? What about posting false topics for discussion? In the early days of the Internet as a medium for research organisations such as ESOMAR were quick off the mark to provide guidance on what can and should be done, this needs to happen now, for WOM related research.

How Valid is the Net Promoters Score

Nigel Holis of Millward Brown has a great blog looking at a range of issues. It is good to see something so interesting from one of the major agencies.

One recent article has really caught my eye, one on the measurement of the mometn, the Net Promoters Score. Spurred on by doubts by others he has re-read the literature and noticed that the although people say NPS predicts sales growth, the validation studies tend to use sales data from earlier years than the NPS scores - which leaves the possiblity that it is sales growth that is driving the NPS score.

As well as reading Nigel's article check out the comments and replies at the bottom of his article. Nigel's net position, NPS is an interesting metric, but it may not be 'the' metric in the way some would appear to believe.

So, how is the Internet changing research?

This short note was written after the ESOMAR Congress discussion session led by myself (Ray Poynter) and is an attempt to reflect the many strands that emerged during the debate. If you would like a copy of the PowerPoint I used as the stimulus for the discussion then they can be downloaded at the bottom of this post.

The first question raised in the debate took the following observations:

• Before the Internet we assumed people only rarely completed surveys, were selected by the research process, and were assumed to fit a random sampling model.
• Since the adoption of online access panels, people are doing 50+ surveys a year, choosing to be members of panels, and can’t be assumed to be a random sample

Is this an important difference?

The consensus view was that this change was more apparent then real. Conventional research had in actuality been re-sampling the same people, ignoring non-response bias, and turning a blind-eye to a variety of professional respondents. At least with the rise of online panels the issue is being aired truthfully and investigated.

The second question was based on the following two observations:

• Turnarounds are getting faster, thinking time is shrinking, more research is becoming commodity research, more ‘so called’ insight is simply the reporting of the low hanging fruit, where the first plausible story is the only story reported.
• Research prices are falling, largely driven by the online access panel price wars, creating less scope for experimentation and more risk avoidance.

Again we asked, is this an important difference? Again, the consensus was that this change was one of degree rather than a transformation. Time lines have been getting shorted since the invention of the telephone, not to mention the fax (if you are too young to remember the fax, ask one of the old-timers). Research needs to adapt to the new realities are the new cost and time frames.

The third set of observations looked at online panels and the growth in outsourcing:

• Most online research conducted via third-party panels, who are busy merging and combining.
• Systems, software, scripting, & tabulating increasingly outsourced

One of the changes that the growth of panels and outsourced solutions has created is that a small consultancy can use the same sample sources as one of the large multinational agencies, and use the same systems and potentially even the same Indian outsourcer for tables. There was a consensus that commodity research is going to become more similar, and that more and more tasks will be outsourced.

The fourth set of observations built on the previous points looked at the issue of lower barriers to entry:

• Anybody can use a panel company, small agencies, clients, non-researchers.
• Data collection systems can be very cheap and can include a wide variety of additional functionality, e.g. tables.
• Quant research can be conducted from anywhere
• Small companies and non-researchers are perfectly viable (economically)

There was more disagreement about whether the Internet was raising or lowering the barriers to entry. Whilst some speakers pointed out how easy it was for a small agency to buy top quality sample, scripting and hosting, others pointed out to the way they the large agencies continue to increase their share. There was however agreement that the ‘worth’ of people who knew how to conduct good research is rising, and that not having access to these good people can be a major barrier to entry for new companies.

About half the debate focused on Web 2.0 and the implications for market research. As an example of the difference between Web 1.0 and Web 2.0 we started with the following table:

Web 1.0                                                 Web 2.0
Encyclopaedia Britannica                   Wikipedia
News & Editorials                             Blogs
Downloadable Movies                        YouTube
Photo Albums                                  Flickr
Newspapers & TV                             User Generated Media
Amazon                                          Price and review sites
WalMart                                         eBay
Online chat                                     MySpace

Most speakers, but not all, accepted that Web 2.0 was something different and that the rules were being remade. One speaker from  China wanted to know what to call the next revolution after Web 2.0, the one driven by the engine of  China as it increasingly becomes the leader in more areas of world trade and jumps existing technologies, perhaps by focusing on mobile approaches, Web 3.0 he wondered?

The existing research paradigm was highlighted with the chart below, raising the question, what will Research 2.0 be like?

Web 1.0                                                         Web 2.0
We select when to do surveys                 ?
We select the respondents                     ?
We pick the questions                            ?
We pick the answers                             ?
We keep the process secret                    ?
We keep the results secret                     ?
We treat customers as lab rats                ?

Several speakers talked about issues such as co-creation, user-generated media, allowing respondents to self-profile (as they do in MySpace), and the need to create communities.

A key conclusion of the session was that Research 2.0 will need new skills and a greater understanding of how people are taking over the Internet. It was also clearly understood that any researcher who wishes to be competent at Research 2.0 will need to master conventional research.

The final point, made by myself as moderator was that we have experienced about two years of rapid change on the Internet and in the way it impacts market research, we are probably going to have another 20 years of similarly fast, large, and transformational change.
Download is_research_changing_v2.pps

Ray’s personal highlights from ESOMAR Congress

Well the 2006 ESOMAR Congress has finished and delegates are heading back to the four corners of the world. There were many good papers and interesting incidents, but here are a few that I will remember in particular.

The paper about starting research from scratch in Afghanistan Matthew Warshaw, Rafiq Kakar, Torpekay Habibzei, and Zemarak Mohsini left me amazed at just how difficult it was, and how brave the local researchers were. Amongst the problems they highlighted was what to do when a local militia leader brings his Kalashnikov to a focus group.

John Jennick from Egg and Gary Schwartz from the Firm gave a great presentation, demonstrating how the gap between market research and CRM has to be blurred when companies want to offer a holistic and responsive service to their customers.

Alistair Millar from Unilever gave a presentation about the Axe (Lynx in the UK and Australia) click campaign, and pretty much confirmed our worst suspicions about men and their fantasies.

Perhaps the images and thoughts that will stay with me the longest came from the presentation by Graeme Chrystal and Neil McPhee. Graeme and Neil were reporting on an ethnography project looking into the everyday lives of osteoarthritis sufferers. The researchers spent two 12 hour days with each of the respondents, following and videoing them all day. The imagery and narrative about the pain and the way it limited what people could do was visceral. As a result of the research the end-clients, a drug company have a much better understanding of conditions that they are familiar with in medical terms, but less so in human terms.

Web 2.0, the buzz of ESOMAR Congress

The ESOMAR Congress is reaching the end of it second day and one of the themes of the Congress is Web 2.0. The keynote speaker, Charles Leadbeatter – author of The Weightless Society, kicked off the conference with an exploration of topics such as co-creation, user-generated media, and the growth of Wikis. Charles contrasted today to the days when he was a journalist with the print media, editors and journalists decided what we read, and readers simply had to take it or leave it, or try to get a letter published (which they would usually edit to their own amusement). Now everybody can be a journalist if they want to.

My session on the Impact of the Internet was over-subscribed (perhaps ESOMAR will give me a bigger room next year) and elicited several lively responses. The consensus was that we need to respond to Web 2.0, whether we call it Research 2.0 or simply use the old names. The other message was that new researchers need to learn both the old skills and the new skills, just having the new skills is not enough!

On Tuesday the session with Marwan Khammash and Katherine Allan focused on Electronic Word of Mouth. The presentation concentrated on why people read online reviews and comments, but eventually spilled out into the need to merge old and new research to understand the dynamic world. Around the stands and in the bars much of the buzz was about Web 2.0, well in the early part of the evening anyway.