The Future Place Blog

How many members should an MROC have?

One of the questions I get asked from time to time is how many members should an online research community have. Clearly the answer depends on what the community is there to achieve and what the budget is. But I still get pressed for an answer. After asking around recently I have come up with my best estimates, and I would love to hear other people’s views.

In order to produce an estimate I have broken my thoughts into four answers, the last of which is not what I would call an MROC.

1. A short-term community, lasting two weeks, engaged in a qualitative exploration of a service feature, conducted with regular service user. 30-90 community members.
2. A short-term community, lasting four weeks, using qual and quant techniques to explore new menu options for a fast food restaurant, with sub-groups of personal and family users, 100-250 members.
3. An ongoing community, looking at different aspects of alcohol consumption at home, using qual and quant techniques, 300- 1200 members.
4. A brand panel utilising community techniques to increase engagement and panel retention, representing users of a specific brand of personal care product, the panel to provide a source for ad hoc quantitative projects and some qualitative projects during the year, 5000 – 50,000.

What I am aiming for with my upper and lower bands? I guess am aiming, intuitively, to capture the middle 80%. Perhaps 10% will be larger than my range, and perhaps 10% will be smaller than my range.

What do you think of my estimates?

What would you change?

Networks change power structures

For a couple of years I have been pointing out to some of the larger companies that Web 2.0 and social media are changing power structures in companies. In the past, the people in the top few positions in most research companies were the most knowledgeable, and often the most skilled, a combination of experience and darwinian evolution (other people may not like this point, but I am pretty sure it was true).

However, Web 2.0 and all that has changed the picture. Facebook came into most organisations because the new graduate intake brought it with them. Social media is much easier for the so called 'net naturals' to get their heads around than it is for some of the gurus of past glory.

However, until a few minutes ago, I had not realised how much the networks I am in has changed the people I connect with and the way I learn.

Because of Twitter I am much more in touch with people such as Nigel Legg, Susan Sweet, and Mike Gray than I would otherwise be.

Because of Facebook I learn about all sorts of insight from people like Andrew Jeavons and Peter Harris.

Because of LinkedIn I get advice and suggestions from people like Pete Cape, Paul Child, and Reg Baker.

The people who I see and who communicate with me on my key networks are undoubtedly shaping my views far more than the books I read and the conferences I attend (and I attend plenty of conferences) - and more than the people I tend to work with.

It may be fanciful, but if we look back in history there were many schools of thought that grew out of a physical/social network (e.g. the Bloomsbury set in Lindon or the Vienna Circle based at Vienna University). Will the changes of the future grow out of these new social networks? And, if so, I am in the right groups???

Will MROCs replace BBGs?

To a considerable extent the difference between BBGs (Bulletin Board Groups) and online research communities (aka MROCs) is simply a generational difference. When BBGs were developed they reflected current practice in terms of how the Internet was being used to create discussions, i.e. bulletin boards. Although some bulletin boards and forums still exist, they are increasingly being replaced by online communities, a phenomenon of the world of social media.

It is likely that BBGs, as a research tool, will be replaced with online research communities (also known as MROCs) over the next few years. The BBGs have the advantage of being quicker to create, of currently having more research functions, and of being more suitable for IDIs (parallel In-Depth Interviews). By contrast online research communities are more intuitive for the participants and more attractive to clients.

---

The question and text above have arisen out of the work I am doing for my book. I would be really glad to hear other people's thoughts.

Interesting article on the climate emails debacle

The Economist has a good article that sums up the latest picture about the recent release of emails from the Climate Research Unit. This article and a similar article in the New Scientist seem to net out to:

1) The data have not been fabricated by the scientists

2) Some of the scientists have sought to stop climate change sceptics from getting their papers published in peer reviewed journals, by unacceptable methods.

3) Some of the scientists have sought to avoid putting their data in the public domain, which is bad for science.

4) Some of the publicly funded scientists in the UK seem to have tried to avoid their legal responsibilities to answer Freedom of Information requests.

I am glad about item 1 (this is also confirmed by the two other big data sets which have similar results).

Item 2 is a major worry and needs to be investigated, and any academic guilty of it should be disciplined or sacked (depending on the individual circumstances).

Item 3 is a worry, and really the UN, given the importance it places on climate change should recommend that all Governments pass legislation obliging all data to be put in the public domain.

Item 4 is frankly shocking, if it happened (the emails asked for messages to be removed and threatened to delete data - which is wrong, but we do not yet know if anything was deleted). On the face of it it looks like conspiracy to break the law and I would like to see the police as well as academics investigate (FoI is not a criminal issues, but conspiracy to break a law is criminal).

So, what should voters and politicians do? For me the picture is still very straightforward.

A) We have the majority (the overwhelming majority) of the relevant scientists saying that the muck we are putting in the air is warming the planet and that the consequences if we don't do something about it are catastrophic. I don't buy the climate conspiracy, nobody could organise that many scientists to all tell the same story, have you met many scientists, getting them to buy the right brand of coffee is just about impossible!

B) The alternative view is broadly, a) the link between climate change and the much we throw into the atmosphere is not proven and b) we may only get as warm as it was 1000 years ago, and we managed OK then (parts of Greenland had a mild climate from about 800-1300).

If we believe the minority and do nothing, and the majority are right we will have a catastrophe.

If we believe the majority and reduce our use of fossil fuels, provide contraception and education to evey woman who wants it, and cut back on meat eating, and the minority are right, we will have improved our lives and cleaned up our planet for no good reason!

(BTW, meat eating uses masses of water, fossil fuels, produces clouds of methane, and consumes a large amount of food to produce it, and too much of it probably shortens our life - so I am trying to eat less of it)

Publicly funded scientists, politicians, and bankers need to realise the world has changed, they can't fool us for long now, they will be found out. So they all need to start behaving better.

ps when we talk about the data have not been fabricated, I am not talking about things like the hockey stick results. The data are what goes into the hockey stick calculation, not what comes out. The process of calculating the hockey stick chart is a theory, the output is simply a pretty chart expressing the data and the theory - and I have signficant doubts about the hockey stick exercise

Evidence based decision making

Earlier today I was reading an Open University study note on research methodology and I came across the sentence "The report is a presentation of argument based on evidence, not a history of a research project".

I wish the market research industry would consider instigating a set of basic tenets that we should all then repeat out loud at the beginning of each week. If we had such a collection of basic ideas, then this phrase, in the context of market research debriefs and reports, should be one of them.

A presentation of an argument, based on the evidence. Yes, that is what we want!

A history of who we spoke to, what we asked them, and what they said? Yes, that is what we too often see.

BTW, just two pages on there was another important, but more subtle point. The text had the sentence "It may be an interesting question - but is it a researchable question?". Are market researchers always brave enough to create a demarcation that enables us to describe some questions as not researchable? For example, when we are asked what was going through the mind of a shopping when they impulse bought some low engagement, low value product, do we propose a solution, as a triumph of hope over experience?

Using Participant Blogs in Market Research

As part of the book I am looking at how market researchers have used blogs as research tools, specifically where they have recruited respondents and enabled them to create a blog as part of a research topic.

Adding together my experience, colleagues’ experiences, and conference papers my views are that blog projects tend to have the following in common:

• Each participant is given their own blog to maintain.
• Usually qualitative. The participants may be asked to fill in a survey as part of the exercise, but this tends to be for background and segmentation purposes.
• The participants recruited in the same sorts of ways that are used for other qualitative projects and for online research communities.
• If incentives are used they tend to be quite high (compared with quantitative and online research communities).
• Tend to last more than three days, but less than three months.
• The number of participants tends to be modest, compared with quantitative studies and online research communities. Typically ranging from about ten to about 60.

What is your experience? Do you often run them longer/shorter? With fewer or more participants?
In terms of naming the people doing the blogging, what other names are you using? (I have heard collaborators, and academics sometimes say subjects).

Whys has online qual not taken off?

OK, I accept that there is lots of online qual in absolute terms! But, if we look back to 1999, online quant was about 2% of global quant revenues, now, according to the latest ESOMAR Global Report, online quant is the single largest modality by revenue for quant research.

By contrast, online qualitative has a longer history than online quant, but it is stil just a few percent of all qual.

My feeling is that online qual has not offered the same cost savings as quant (online quant is about one-third the cost of face-to-face quant). Also, although online quant has caused compromises to be made on quality (for example the acceptance of using panels), I think that mostly the compromises have been accepted or ignored.

By contrast, I think that many vendors and buyers of qualitative have a view that the quality compromise is much bigger with online qual.

What do you think about how we gor to where we are right now?

My take on qual versus quant

This post started life as part of a comment I added to a post by Surinder on his Research Talk site, but has now taken a life of its own!

The growth of interest in online research communities, buzz and blog mining, Twitter, e-ethnography, and semiotics has re-opened the discussion about what is qual and what is quant. In particular people have been asking questions along the line “If I have 200,000 posts, is the analysis of that qual or quant?”

My current, informal, definition of qual and quant is roughly:

a) If we look at an analysis process that if two trained practitioners use the same technique on the same data, with the same tools and with the same assumptions SHOULD produce the same answers, then it is quant - i.e. it derives its value entirely from the method and the data

b) If a) is not true, but the analysis is based on a proper methodology and approach, it is qual – i.e. the results are based on the method, the data and the interpretation of the researcher

c) if neither a) nor b) are true then the analysis is simply meta physics and should be put to one side.

Using the definitions above, where does that place using a tool like Tangerine or Leximancer to analyse very large amounts of text? My feeling is that they are unambiguously qualitative at the moment. In order for these tools to be useful, judgements have to be made about how text is grouped and coded. When those decisions are not trivial they rely on the users own sense of narrative and their social constructs.

What are the key issues about Web survey systems?

As part of the book I am writing for Wiley, a handbook of online and social media research I have been working this week on the systems that companies use to conduct online surveys, i.e. Web surveys.

I would really appreciate hearing people’s thoughts on the structure I am thinking about using.

The first level of the chapter looks at the categories that anybody choosing a an approach towards a web survey system should consider. The headings at this level are:

1. Whether to have a system  - or just buy completed surveys from third party suppliers
2. Buying or developing a system
3. Software or SaaS (Software as a Service)
4. Cost
5. System features
6. Capacity

The ‘System features’ category clearly needs subdividing and my thoughts at the moment are the following subdivisions:

1. Questionnaire creation tools
2. Questionnaire testing facilities
3. Project management and monitoring
4. Mulitmedia options
5. Advanced graphical interface
6. Panel management
7. Project reporting
8. Analysis options
9. International options
10. Mobile research
11. Hybrid survey options
12. Security options
13. Advanced marketing science modules
14. Programmable script or language

I would appreciate any suggestions or thoughts. Do you think I have missed an important category? Have I missed an important subdivision of Systems features?

Tweet whilst you present!

Here is a link to a story about a really interesting new app that takes the Tweetference concept one step further, giving presenters the chance to auto-tweet whilst they present!

The method is that you download the app and then embed tweets on the notes part of the page. As you present that page the tweet gets sent.

Lot's of potenital for bother, but sounds interesting!

Thanks to Nigel Legg for the tip.