Some thoughts on NewMR

As well as the book I am bashing away at several conference papers. Here is a short section below relating a taxonomy of NewMR. Any thoughts or suggestions?

 

List of Techniques

This section briefly lists the NewMR techniques that are currently important and describes them on three key dimensions, namely their epistemological position, whether the research is active or passive, and what the relationship is between the researcher and the respondent (from Researcher->Respondent for the conventional model of research done to respondents, to Researcher<->Respondent for research done with respondents, and Researcher<-Respondent for those cases where respondents are running the show). Below the table there is a quick reminder of what the techniques are.

Technique	Epistemological Position	Active/Passive	Researcher Respondent Model
Prediction Markets	Post-Positivist	Active	Researcher -> Respondent
WE-Research	Constructionist	Active	Researcher <- Respondent
Respondent Blogs	Constructionist	Active	Researcher <- Respondent
Online Research Communities	Constructionist	Active	Researcher <-> Respondent
Community Panels	Post-Positivist	Active	Researcher -> Respondent
Blog and  Buzz Mining	Post-Positivist	Passive	Researcher <- Respondent
E-ethnography	Constructionist	Active and Passive	Researcher <- Respondent
Other Non-Random Probability Sampling Quant	Post-Positivist	Active	Researcher -> Respondent

 

One interesting thing about a table such as the one above is that they highlight areas where something might be missing. For example there is not an entry in the table which is Constructionist, Passive, Researcher<-Respondent. However, this does not mean the technique does not exist, it may simply be that it is not considered part of market research. The increase use of feedback systems, such as the rating system in eBay fits the description, but is not considered to fall within the domain of market research.

The Techniques Outlined

This section briefly describes the techniques set out in the table above.

Prediction Markets

Prediction markets are a research tool where the predictions are formed by allowing the participants to buy and sell ‘positions’ on what they think will happen in the future (in a similar way to futures markets). The most famous example prediction markets are the Iowa Electronic Markets which have been successfully predicting the results of US elections for many years.

WE-Research

A term coined by Mark Earls and John Kearon to describe a phenomenon that others have described as citizen research, where the respondent is recruited to be the researcher, providing a combination of crowdsourcing and co-creation. The new ethnography iPhone app from EveryDayLives is a great example of how this field is developing.

Respondent Blogs

Respondent blogs are based on recruiting respondents to work as participants, capturing some aspect of their lives. In the early days of Research 2.0 this tended to involve using standard blogging tools, more recently there have a number of developments such as the mobile phone adaptations used by MESH Planning and new platforms such as Revelation. Although researchers set the context and may ask some questions, the process is mostly respondent driven and at its best is a good example of WE-Research.

Online Research Communities

Online research communities, also known as MROCs, are communities created to provide a research to companies. In a typical case these communities have 50 to 500 members and are essentially a qualitative approach, even though surveys and polls may be part of the engagement. In 2009 online research communities were one of the ‘hottest’ topics, both in terms of market growth and conference papers/presentations.

Community Panels

Community panels tend to be brand orientated and to be the natural successor to in-house panels, albeit with more of a community feel. The research that uses these panels tends to be quantitative in nature which is why they tend to be larger than MROCs, typically with 5,000 to 20,000 members. However, in line with recent trends and thinking they do not make any pretence to be a random probability sample.

Blog and Buzz Mining

Blog and buzz mining is a passive research approach that recognises that if people are talking about your brand then you had better be listening to them. Blog and buzz mining are developing techniques, from web scraping, to bots, to RSS feeds, that search the internet for conversations from blogs to Twitter, from social networks to comments on news stories and glean information about brands and social issues. The main limitation is that most of the online conversation appear to be largely restricted to the top 20% of brands, with very little being said about the 50% of brands with the lowest salience.

e-Ethnography

e-Ethnography, along with netnography and virtual ethnography, embraces both active and passive techniques. For example, e-ethnography can refer to the passive process of following conversations and interactions on the net, and the individual level (as opposed to the macro level of blog and buzz mining). Or, e-ethnography can involve engaging with people on the net and asking questions, or becoming a participant observer in a community.

Other Non-Random Probability Sampling Quant

Because market research has not had access to random probability samples for many years, and because respondents can’t answer many of the types of questions they are asked, market research has developed a range of techniques to provide useful answers. However, in most cases market researchers have not really made the point that these techniques are not based on the traditional research model. Perhaps the development if NewMR will allow many of these approaches to ‘come out of their closet’?

Amongst the non-random probability sampling approaches that researchers commonly use are:

Complex Weighting. This method is based on using external data to weight a panel to make it behave more like the population One method, made popular by Harris Interactive is propensity scoring (Terhanian 2008), and another is to use CHAID to assign weights (Fine et al 2009). However,these routes require time and high quality information about the population.

Quotas. The most common way of working with panels and other non-probability samples is to match a small number of key demographics to the target population, for example age, sex, and income. For example, if a new breakfast cereal is being tested via a panel then assumption is that if young, wealthy females on the panel like it, then young wealthy females who are not on the panel will also like it.

Benchmarking. One approach is to keep the sample specification constant and then to assume that changes over time amongst people who meet the specification will be matched by changes in the target population. This is the basis of most ad and brand tracking. The absolute values produced by the tracking may or may not reflect the wider population, but it is the changes in the tracking lines that are monitored and reported.

Modelling. Modelling is a more sophisticated variation of benchmarking and seeks to link the output from a panel survey to the real world. For example, a series of product tests might be conducted with samples from an online panel, with the specification of the sample held constant. Over time the company will acquire market feedback and can attempt to model what a survey score of X means in terms of market outcomes. When this system works it deals with both sampling problem and the face that people can’t really answer the questions in a way that is meaningful to their own personal outcomes.

Comments

[Tamara Barber]

Hi Ray,
Nice list. Are you also interested in covering things like fMRI and eye tracking? Or does that fall outside of market research for you?
Tamara