The Pros and Cons of ISOs for Research – Ray’s view

I have to admit that I am not a fan of ISOs for market research. 

I am a big fan of ISOs for products that can be defined clearly and especially in cases where the buyer is not in a position to make an informed choice. For example, I want to know that the plug on my washing machine won’t catch fire, will switch on and off, is made of the right material, and won’t electrocute me.

In terms of business processes, and I do not mean just market research, the case for ISOs is harder to make. In many cases the ISO simply relates to process control, but process control is not sufficient to make a good product, and in some cases it is not even necessary. 

If we take an extreme example let’s consider two ethnographers. The two ethnographers may spend two months 'hanging out’ with similar families, they may take the same number of hours of video, they may take the same amount of field notes, they may both conduct their analysis using grounded theory, they might both present their results using the same client specified templates.  Their process has been the same, but we would be surprised and probably shocked if the results were the same. If one of the two were part of an ISO organisation, they would probably be more expensive, but it is not more likely that their explication of the phenomena observed would be ‘better’.

Process control is important, and in larger organisations it is vital. Most quality management processes are, IMHO, designed to help large companies be as good as small companies. In a small company, perhaps a proprietor and ten employees, everything can be seen, the owner takes personal responsibility, and clients can soon work out whether what they are getting is good and consistent. However, if the company employs 1000 people, across many countries, there have to be process controls to create consistency; otherwise the customer has no idea about what they are buying.

In markets where customers are informed and knowledgeable, all the buyer needs is consistency from a supplier. The buyer can try a service and if it is suitable, they then buy more, if it is not suitable they buy less. Even before buying they can assess suitability by talking to people who have used the supplier before. However, if suppliers are not consistent, the buyer, even the informed buyer can’t make rational decisions.

So, consistency is vital, at all levels, and for large companies process control is an essential element of consistency. 

When ISOs are well drafted they do not necessarily represent an additional cost for large companies. Large companies need process control, so the ISO restrictions and recommendations can help them achieve better process control and can actually save them money.

For many (perhaps most) small companies ISO are a major cost. They are a major cost in consultancy fees, they are a major cost in adopting processes that may be prejudicial to efficiency and quality (what works best in a large company is not necessarily even good in a small company), and ISOs can create a major cost in auditing procedures.

If most market research buyers were to require ISO accreditation then this would confer a business advantage to the larger agencies, create a major problem for small agencies, and create barriers to entry for new companies. There have been researchers in small companies who believe that the reason that some of the larger companies want systems like ISO is to obtain this competitive advantage against new entrants and against the smaller research agencies.

Should the research industry favour ISOs?

That is a harder question. I own a small agency and ISOs do not help me or my clients, indeed they are a threat to me and to the quality of research my clients buy.

But, over half the market research industry is made up of the top ten agencies, the top 25 account for the vast bulk of market research, and ISOs may be good for all of them.

Should the trade bodies be promoting ISOs?

Again, this a hard question. If standards are going to be enforced, for example by governments or clients, it is clearly better if market researchers are involved in drafting them. If the trade bodies feel that the standards help deter ‘cowboys’ from other industries competing with market research and providing an inferior service, then that might be a good thing. If ISOs were to help, say, 80% of the industry at some risk to the other 20%, that might be acceptable (although as part of the 20%, I would not be too pleased about it).

What would I recommend to clients?

I think clients have to do a cost benefit analysis. If two suppliers are equal in every other way, but one is ISO accredited and the other isn’t, then pick the ISO one. However, if the ISO one is more expensive, or if you prefer the quality of the non-ISO, this needs to be taken into account. If clients adopted procedures that reduce the number of suppliers they can consider they will weaken their bargaining position and they will find costs escalating and quality dropping – clients need to maintain a degree of competition amongst their suppliers.

Where do I stand?

Well speaking for myself (and not on behalf of any of the bodies I am a member of, any clients I work with, or any political parties or lobby groups I am a member of), I think ISOs are the wrong solution to the problem.

I would like to see things like the ESOMAR 26 questions for online panels developed further, more codes of conduct to be agreed to, and improved training and possibly the introduction of professional personal accreditation for researchers (for example I really like the QPMR system used in Australia).

What do I think will happen?

I think most clients will ignore the the new standards whenever they want to buy from a non-accredited supplier. I think that most people like me will not even consider being ISO accredited, and I don’t expect it to cause me a problem to me or other small agencies.

Comments

[Kvistbo]

I share your views Ray and Alastair's comments are spot-on. ISO is not a one-size-fits... and for that reason, I'm personally not sure that it is a good idea for trade bodies to blindly promote these standards as such. In Denmark, the national MR association originally went as far as to demand that the members implemented ISO. Last year, the association was dissolved as members had fled - much due to disagreement on this issue.

The challenge however, is that ISO certification is often perceived by clients as a quality seal of approval. Although well documented processes and product consistency may be important elements of 'quality', it clearly doesn't mean that you're necessarily getting the better MR product opting for a certified vendor.

That said, like Alastair, I do believe that ISO can be really helpful (for mid-tier MR companies) in not only documenting existing processes, but also in improving these. It provides a systematic approach and I'd be surprised if the very certification process itself hasn't made most certified companies reconsider (and ultimately change) "one or two" processes, even if this is not the intended purpose of the standards. Thus, while ISO is not fit for all, I would certainly encourage any MR company to familiarize themselves with the standards to make an informed decision - and subsequently communicate to clients why that have/have not opted for certification. There are plenty of objective and legitimate reasons for the latter.

Finally, anyone is free to take whatever they find suitable from the standards and act accordingly, without necessarily going for the full package. Either way, I think that most companies would find it to be a healthy and learning process after all.

[Mr_roi]

Ray, Wonderful to see a rational analysis on this. A disclaimer: I'm biased as my day job is helping research agencies improve their business processes/profitability and I wouldn't want to be replaced by an ISO specialist! But what I've observed is that:
-ISO, done properly with the help of a decent consultant and taken seriously by all involved can be really useful for mid-tier MR companies to get their processes and systems documented. (I also think it's wasted on small companies, and insufficient for very large ones). Many such companies have "evolved" on the back of clever founders and much knowledge is often held in the heads of a few individuals so they can end up with lots of things being done for reasons that are lost in the mists of time. This is highly dangerous and systematically documenting what's going on is a good first step in preparing mid-size companies to grow.
- BUT ISO doesn't really tell you whether your processes are best practice for MR, what to prioritise or how your processes impact profitability or overall research quality. Importantly it does not really deal with how the "hard" bits of the research process (software systems, statistical controls etc.) interact with the "soft bits" (how researchers work, quality of deliverables etc.), and it's the "soft bits" that make the big difference in research transformation.

As an example, I've just completed a review in a company that is also doing ISO - I found it really helpful to see what had been done by the ISO guy, and it saved me time -- but the kinds of recommendations he gave, while useful as baseline measures of what was going on, were not going to transform the company in a significant manner or really improve outputs to the client.(Nor, to be fair, were they intended to).

So, while ISO can be useful to some MR companies, I don't think ISO accreditation really provides clients with the sort of assurance they need, and like you I think the industry has to start to develop it's own forms of certification.

[TomHCAnderson]

Well almost agree anyway

[TomHCAnderson]

Look at that Ray, I think we finally agree on something ;)

Tom