At both the ESOMAR meeting in London on Wednesday and at the ASC conference on Thursday the related topics of quality and ethics were discussed several times. The growth in the use of online access panels (or convenience samples as I prefer to call them) and the events in the UK over the last couple of days relating to opinion polling and the General Election (check http://search.twitter.com/search?q=yougov) have made this a hot topic. (Last night 3 polls were published which had very different results, this has resulted in public ridicule for MR).
This leads me to want to define my view of what I mean by both quality and ethics.
The first point I would make is that quality is really just a sub-set of the ethics issue. What we mean by quality is, IMHO, defined by our position on ethics. Ethics says what we should do, quality is doing it well and at least doing it as well as we have said or implied we would do.
The second point is that we have to define market research in order to be able to define where and when the ethics apply and what they cover.
My definition of market research is that it is a business service providing information and advice to organisations about market related activities based on human beliefs, attitudes, actions, and intentions.
In my view this means market research does not include:
• Academic research
• Social research
• Medical research
• Geological and mineral surveys
• Analysis of industrial processes
Many market researchers will also conduct academic, medical, and social research, but the ethics of those disciplines are different. For example, academic and social research can be conducted in ways that bring harm to the subjects, if the public good is large enough. Surprised? Check the US Federal law on Protection of Human Subjects, or think about the number of times that drugs trial cause bad effects, where the ethics simply require that the risk is minimised, that the subjects are properly informed, and that the potential benefit is big enough.
By contrast, in market research it is never acceptable for the design to knowingly risk harm to the subjects. The ‘good’ that market research generates is commercially for the client, hopefully our new collaborative research will also generate some general good as well, but it is hard to imagine the public good of a new brand of cheese is ever going to worth knowingly putting people at the risk of harm
So, what would I define as ethics? Well I have a pretty simple set of 3 points:
- Say what you are doing. For example, if you are using a convenience sample and using modelling techniques to forecast the market, say so.
- Do what you say you are doing. If you say you are using a technique based on the science of random probability sampling, then make sure you are.
- The method you are using should be based on a recognised body of knowledge or thought and should be capable of being refuted or verified. For example, astrology and phrenology are not acceptable market research methods.
To give a real world example of what I think should happen is that when an online access panel is used the researcher should not use sampling error statistics as an implied measure of validity. They can be used as an indicator of reliability, that is fine, but not validity. I would like researchers to adopt a more engineering frame of mind and talk about experience and tolerances. For example, rather than saying science shows we can only be wrong 5% of the time, say “historically, based on X number of tests, we have only been wrong Y% of the time, so we feel confident is suggesting ….”.
So, those are my thoughts and ethics, what is your view?