The Future Place Blog

Should we conduct commercial market research with children?

Research-Live today reports that there is a new collaboration in the UK between advertisers, marketers, and market researchers to draw up a code of conduct for marketing to children - a response to the Government threatening a 'crackdown' on irresponsible activities.

However, I believe that the new plans and the new MRS proposed code of conduct do not go far enough. Here is a copy of the comments I have added to the Research-Live article. 

"Speaking as a parent I find the proposed/draft MRS code much too soft, too much in favour of clients and too little in favour of families or specifically children.

Amongst my beefs are:

a) it permits research in school. This is not why we send our children to school. The rules say parental permission 'may' be asked, but teacher or head is allowed. No way! (I am happy for some social research to be conducted in schools, but not commercial research).

b) it allows a researcher who discovers a child in harm (for example being molested) to report it under some circumstances. No way! It should be a duty to ensure that either the police or social services are alerted, not an option that the researcher may choose if they wish.

c) I believe that any research that facilitates the marketing of products at children should be ruled by the code as unethical. Do we really want MR to be associated with trying to get children to eat more junk, drink more junk, or to pester parents into buying things that might not be affordable?

Let us hope that it will not be too much longer before the government bans the marketing of products at children altogether, but until then we should recognise that helping this kind of marketing is not ethical. I think we need to think like parents, not like businesspeople on this occasion."

What do you think?

Public Sector Facebook Sites - some examples

Here is a small selection of UK public sector sites on Facebook that seem to be getting at least some things right. Nobody seems to be getting everything right (which is not surprising at this stage), but we can all learn from each other by picking the best from different places.

Love My Lancashire, created by Safer Lancashire, a Community Safety Partnership, a group that combines local government, police, fire, youth agencies, and many more.

Launceston Police, Launceston is a town in Cornwall and this site is maintained by the Launceston Policing Team and allows a wide range of views to be expressed and has some very robust conversations between local citizens.

Coventry City Council, Coventry has a well-developed social media presence which includes 7 Facebook pages. The social media page of their main website is at http://www.coventry.gov.uk/ccm/content/web-admin/online-communities.en , and their main Facebook page is at http://www.facebook.com/coventrycc.

West Midlands Police, a lively Facebook page with plenty of interaction with the public.

Nottingham City Transport, encourages people to subscribe via SMS so they can get the latest bus news (very useful in snowy weather!)

Belfast City Council, this Facebook page does not have many people ‘liking’ it (just over 1600), but it does have a lively set of posts and comments from the public.

Hampshire Fire and Rescue Service, this page has a large number of people liking it and keeps providing engagement.

 If you'd like to highlight other public sector Facebook pages, please add them in the comments to this post.

Quick update

Since my scare last week I have felt fine, but the process of trying to find out what caused it continues. I have been deeply touched by the very large number of people who have sent best wishes. To all of you, many thanks.

I have had to cancel my trip to Australia, but I will be visiting in November/December instead. Many apologies to all of those who were expecting to see me at meetings and conferences, last week, this week, and over the next couple of weeks.

For the time being I am mostly not working, I am trying to keep on top of emails, I will start tweeting again, and I will moving things along in terms of the Festival of NewMR. Note, if you want to submit a synopsis for the Main Stage webinar you need to do it before October 5th. But, overall I am not doing very much work at the moment.

If there is any time left over, I will try and squeeze in a couple of blog posts, including at least one new TARSK.

 

Thank you Nottinghamshire NHS

The last 24 hours have been a bit scary but seem to have worked out as well as they could have done. The day started with chest pains, an ambulance, blue flashing lights, and being hooked up to a range of devices in the care unit at Nottingham’s QMC Hospital.

During that time I was in the hands of a large team of very wonderful people, from the paramedics in the ambulance, through to the orderlies, nurses, porters, receptionists, and doctors at the hospital.

The great news, one day later, is that it seems it was not a heart attack as such and I probably won’t need a stent or anything like that. However, they are not sure of what caused it and, although I am back home today, I need to have more tests to make sure I am fine.

This has meant that I have already missed two events, so I need to apologise to all the people who were expecting me to deliver a webinar yesterday for the Rod Laird Organisation on using Facebook in the public sector to engage and consult, and the people attending the MRS’s Social Media Conference today who were expecting to see me as one of the speakers.

Unfortunately, I will need to miss some more events over the next few weeks whilst I sort out some health issues, and I offer my apologies. I will try to re-schedule these for later in the year or early next year.

I do not expect to be checking email until next week, so I may be a tad late in replying to messages sent yesterday, today, or this week. 

One of the strange things is that I feel fine, and indeed I may be fine, but I am prioritising the tests and medical advice over everything else for the moment, after giving my family and myself such a fright yesterday.

Jon Krosnick’s page – a key resource for all market researchers

In a post by Jeffrey Henning on Acquiescence bias in Likert Scales (and therefore why market researchers should not use them) I noticed a link to Jon Krosnick. I have met Jon before and read some of his work and have been thoroughly impressed. I feel that, along with Mick Couper, Krosnick is at the forefront of understanding what works and what does not work in surveys.

However, I was not aware of Jon Krosnick’s page on the Stanford University website until I saw a link to it in Henning’s post. Having now visited the page I can say that Krosnick’s page is a key resource for any market researcher, it has snippets from his research and links to some of his papers.

An example of the useful snippets is his report, with LinChiat Chang, of the predictive accuracy of asking people what they did last week as opposed to what they do in a typical week. As well as the snippet containing the key finding (that typical week was better in the test they conducted than last week) the page has a link to download an 80 page PDF of the study.

Great Festival opportunity for students and junior researchers

The Festival of NewMR is gathering speed and will be held (virtually) 6-10 December.

One of the features of the Festival, which may be of particular interest to students of market research or junior researchers is the Poster competition. The Poster competition is free to enter and has a prize of US$ 1000. More important than the prize is the chance to catch the eye of people you might want to work with and potentially to influence the shape of market research.

The challenge is to create a Poster that best answers the question “What is the next BIG idea in understanding consumers' needs and wants?"

So, if you work with students or junior researchers, please forward this article to them.

The Likert Scale – TARSK 14 (Things All Researchers Should Know)

The Likert scale is the single most common rating scale in quantitative market research. In its normal form it attempts to record a respondent’s level of agreement with statements. For example a statement might be “I am completely satisfied with the service provided by ACME” and the levels might be:

1. Strongly disagree
2. Disagree
3. Neither agree nor disagree
4. Agree
5. Strongly agree

In formal terms there is a distinction between a Likert scale and a Likert item. The question in the example above is an example of a Likert item. If several Likert items are used and the results summed across those items, then the result is a Likert scale. 

When the words are used correctly a Likert scale relates to the sum of results across several Likert items – but this is almost never done in commercial market research. Many/most people, in commercial market research, say Likert scale when they mean Likert item.

The Likert scale was developed by US educator and psychologist Rensis Likert (1903 – 1981) in 1932.

Amongst methodologists the big debate about Likert items is whether they can be treated as providing interval data (e.g. 1, 2, 3) or whether the numbers produced are simply ordinal (e.g. 1st, 2nd, and 3rd).

The case for ordinal data
The case for treating data from a Likert item as ordinal (i.e. 1st, 2nd, 3rd as opposed to 1, 2, 3) is based on the fact that there is no reason to believe that the difference between, say, ‘Strongly disagree’ and ‘Disagree’ is numerically the same as the difference between ‘Agree’ and ‘Neither agree nor disagree’. Similarly, if the scale is an interval scale the researcher needs to believe that the difference between ‘Agree’ and ‘Disagree’ is exactly twice as big as the difference between ‘Strongly disagree’ and ‘Disagree’.

If the data is not interval data, then it is ordinal data. If it is ordinal data then we can report the median, the mode, the percentage picking each level, the percentage picking the top two boxes, the bottom two boxes, the percentage agreeing or disagreeing, etc. However, we cannot and should not calculate a mean or standard deviation.

The case for interval data
Some researchers argue that to treat Likert numbers as ordinal loses information. They argue that the symmetry of the language of most Likert questions means that respondents understand they are supposed to respond in an interval scale fashion. To supplement this many researchers layout their questions graphically to help emphasise the interval nature of the data, and include placing numbers next to the text to help reinforce the interval scaling.

If a researcher makes the wild and unjustified decision (IMHO) to treat Likert numbers as an interval scale they are rewarded by being able to use means, standard deviations, and by being able to use the regular forms of techniques such as Factor Analysis and Regression (as opposed to the less familiar non-parametric alternatives).

In a commercial environment a researcher who refuses to pretend that Likert items produce interval data will probably face commercial difficulties (either from their clients or their employer).

The number of levels
Researchers disagree about the best number of levels to use. Probably the most common Likert item is the one with 5 levels, but 4, 7, 8, 9 and 10 are all fairly common. When different levels have been tested they have been shown to produce different results, for example different means and standard deviations. The most frequent results seem to be that more levels results in higher means and more dispersion (in the 5 point scale some researchers have reported that respondents are reluctant to use the 1 and 5, resulting in very limited variation). Most researchers have found that going beyond 8 points changes the results very little.

My own feeling is that with a larger number of levels a question changes from being a Likert item to more of a numerical scale, where there may be a stronger argument for treating it as an interval scale rather than just an ordinal scale.

ESOMAR Congress, ISOs and Standards

For me the most contentious presentation at the recent ESOMAR Congress was one by Bill Blyth and Judith Passingham which addressed the issue of standardisation. Bill is a committed and articulate proponent of standardisation and initiatives such as ISO. I hope I am not doing Bill and Judith a disservice when I paraphrase their argument (I disagree with them fairly substantially, so I need to be careful not to create straw men).

Their case seemed to be:

a)    Most market research we do now is based on principles and thoughts that date back 40 to 70 years and that this adherence to traditional techniques and values has facilitated the development of a $30 billion global industry.

b)    In a modern global market MR needs to provide the sort of service that modern businesses demand, including standardised services that can delivered in a reliable way in, for example across 30 countries, and outputs that can be interfaced with other business intelligence systems within companies.

c)    Initiatives, such as research ISOs, are an essential part of creating a product that will separate ‘real’ market research companies from other suppliers of superficially similar information, in the context of large and or global clients (which is something like 80% of the MR business).

I think that opponents of this view need to realise that its proponents are concentrating on something like 80% of the current market research business, i.e. the top few hundred buyers, almost all of the research being international, and almost all of it quantitative or similar (e.g. automated data collection).

My view has moved on slightly as a result of hearing the presentation. My key thoughts are:

1)    The reasons market research is very similar to 40 and 70 years ago is because we have wasted chances to grow the business in the past. If MR had taken a bigger share of website testing, credit checking, analytics, benchmarking, and direct marketing when we had the chance we would today be a larger industry with a wider range of tools.

2)    If we narrow our definition of what is and is not market research, we will continue to lose opportunities. For a while this might enable MR to grow in absolute terms (in line with global economic growth) but MR will continue to shrink as a proportion of Business Intelligence.

3)    Standardisation is at its most effective when dealing with data collection and data processing, especially in connection with quantitative data. If a client buys a 30 country survey they want to be sure that the same standards are applied in each country, that the same checking has been applied, that the same processing methods have been used.

4)    It seems to me that the ISO and standardisation debate is about dividing the research process into the ‘mechanics’ and the insight and then concentrating the process heavily on the mechanics. This may suit the large agencies, for whom nearly all their money comes from the mechanics. But it is alien to research providers who concentrate on interpretation, insight, advice, and consultancy.

5)    It seems to me that in a research process standardisation runs the risk of not answering those problems that are amenable to standardisation. There are two views about measurement. A) If you can’t measure it then you can’t manage it. B) If you can measure it then it does not matter very much. I believe the truth is somewhere in between the two, but closer to B.

I think some elements of the push towards standardisation are fine, such as not cheating, accurate tabulation, ESOMAR’s questions for panel owners, codes of conduct etc. However, I think that beyond these relatively obvious areas they are at best a waste of effort and at worst they will limit the growth and change of market research.

I think that if the push for ISOs continues and is successful, most market research will be done by companies who are not part of the MR community and who will not be signing up to MR ethics, codes, or ISOs.

Thoughts from ESOMAR Congress 2010

I am just back from ESOMAR’s Congress in Athens and I feel thoroughly enthused about research and very pleased about the improvements since last year’s Congress. This is not to say everything was better this year, but the trend seems, to me, to be heading in the right direction.

The Positives
The most obvious improvement between 2009 and 2010 was the improvement in the broader economic outlook for market research. In 2009 MR was coming towards the end of 12 months of recession and pain. The number of people attending the Congress in Montreux, in 2009, was fewer, many were being careful with their money (leading to people staying a variety of hotels, some quite a distance away), and the contrast between the expensive picturesque nature of Swiss Montreux and the state of the MR industry was unfortunate.

In 2010 the number of delegates was on the way back up, many familiar faces had returned, and there were plenty of new faces too. Athens is a very desirable destination, but it does not have the sort of opulence that Montreux has, Athens does not seem exclusive, and many of the delegates were reporting pleasure at how friendly the locals were (e.g. in the hotel, in the local restaurants, and in the taxis).

Before the Congress proper started there were two days of workshops and on the day before the main Congress there were rehearsals for the speakers. This meant that there was a buzz developing before the Congress even started, a real positive.

The exhibition was positioned right outside the main hall, which meant that delegates kept walking through the exhibition, which increased the contact between delegates and the best placed exhibitors, and made for a good buzz.

The best new sessions from 2009 were repeated and developed, in particular the Research Rockstar and Young Researcher of the Year. Some new formats were added, such as Philip De Wulf’s goldfish bowl which focused attention on young researchers and the need to attract and keep new talent.

The written papers (at least the ones I have had a chance to read so far) seemed very good, and most of the presentations were at least adequate, with some being very good.

Key themes that emerged from the Congress were:

Social Media is here, it is useful, but there are cases where it underperforms, there are cases where it needs to be combined with traditional techniques, and there are cases where traditional techniques are more relevant.

There are a growing number of pressures on MR from outside, from DIY, from data protection and privacy legislation, business intelligence companies, and companies that do not necessarily fit our model of a market research company (such as Dunnhumby).

The growing importance of Asia was again apparent (BTW my view is that over the next 10 years Asia will grow from its current 15% of market research global revenues to something like 30%-40%). 

The impact of qualitative research is growing and this is reflected by the breadth of related topics being presented and discussed, including: ethnography, semiotics, online discussions, some neuroscience, the role of video, diaries, online immersion, and even, from Neil McPhee, a spirited forwarding of the case for ‘real qualitative’ research, using trained, skilled, and experienced people to spend significant amounts of time with people, observing, talking, and interpreting.

The Negatives
Of course there were negatives, at least in my opinion. The first negative was the rushed nature of the programme. The main Congress was just 2 days, but the number of papers seemed to be the same as for a longer Congress. This meant the pacing seemed wrong, it made it more likely that attendees would miss presentations they wanted to see, and it meant that the coffee breaks became very valuable and therefore less useful for the exhibitors and for networking. If ESOMAR stays with a 2 day format then I think they need to reduce the number of papers. But ideally more space will be found, either by going back to a longer format or in some other innovative way.

Most of the presentations were 20 minutes, or thereabouts, which made the pacing of the Congress seem a bit predictable, some shorter, and definitely some longer sessions would have broken things up more. However, with a 2 day schedule longer sessions are a bigger gamble.

Those exhibitors who were not on the main path probably found they were too much out of the main flow and the Internet in the Hilton was limited and expensive.

With one exception about ISOs and standardised research, which I will mention in a subsequent post, the presentations all seemed very safe and there was a shortage of arguments and passion. I think that amongst the reasons for the lack of argument was the 20 minute format, the growing professional standard of presentations, and perhaps the temptation to seek to please too many people.

Social Media and Ethics

Whilst at the recent ESOMAR Congress in Athens I created a vignette in my mind to help explain the many issues that can arise. Consider the following hypothetical example:

In Facebook I discover the “ACME TV and Video” page which has a wide variety of posts on it from people who use or like or are interested in ACME TVs and ACME videos. Because I do not want to keep visiting the site I create a script in my favourite web scraper to log in every day and to collect all new posts and to put them in a database for me. 

For my client, a major electronics company based in the US, I produce a report, based on social media, looking at the different social contexts of the consumers home electronics. I note within this ACME page a 200 word poem written by a member of the page who writes about how her neighbour’s 8 year old daughter is often left at home on her own and uses the TV to feel less alone and lonely.

In my report I include a copy of the poem and I mention that it is taken from the ACME TV page. However, in order to protect the 'respondent anonymity', I change her name to some fictitious name.

So, what’s wrong with any of that? Well let’s count some of the ways that come readily to mind as potential problems.

1. By using a web scraper to collect data from Facebook I have broken the terms and conditions of my use of Facebook.
2. By using material for Facebook for my commercial research I may have further broken the terms and condition that I signed up to when I joined Facebook.
3. If the person I collected the data from was under 16 (according to the UK MRS rules) then I have collected data from a child without prior parental permission.
4. By not quoting the author of the post I have broken many conventions about properly citing and crediting sources.
5. By quoting the poem in its entirety, without permission and without citing I may have breached copyright issues.
6. By using a literal quote I may have made it possible for the author to be identified. Because I have named the group the client could use a web scraper to load all recent posts and then search for the text in the report.
7. Not only may I have compromised the anonymity of the author, but if they post photos, if they have not protected their profile, I may have provided access to personal information such as mobile phone number.
8. Because I may have identified the author, I may also have identified the existence and identity a vulnerable child left alone.
9. Because the data is personally identifiable it falls under (in the UK) under the Data Protection Act, which means I need to hold it securely, and I can’t simply send it overseas to the US which has a different set of legal protections.

I am sure there are other problems that people can highlight, but I hope that the vignette will help illustrate that when we are working inside the area of social media we need to be very careful and we need to think about what we are doing.

In thinking about social media we need to think about: laws, terms and conditions, market research ethics, and other relevant codes, practices, and expectations. The general concept of informed consent will still provide a good guide, but we need to be clear about whose consent and how we ensure that it is informed consent.