As part of the book on online and social media research I have been exploring the differences between countires, to help my understand what lies behnd the differential take up of online qual in North America.
I have come up with a set of observations that I would love to share with people and to see if they make sense, so here goes.
This section is based on generalisations, that is to say, there is enough truth in them to make them of value and enough exceptions to make them dangerous.
Most of these observations relate to traditional qualitative as much as they do to online, but they tend to be more important online as there may not be a local partner ‘educating’ the international researcher. The context of this section relates to what an online researcher needs to be aware of, especially as the iInternet creates the illusion that research can be conducted internationally without the involvement of local researchers.
Issues that are tackled in this section include:
• Europe versus US.
• Recruiting participants.
• Online versus offline honesty.
• Size of groups.
• Counting and voting in groups.
Europe Versus North America
Although concentrating on Europe and North America is a simplification, it can be justified in terms of the proportion of market research currently conducted there. The data collected by ESOMAR in the 2009 Global Market Research report indicated that 49% of the global research total (in terms of value) is conducted in Europe, with a further 30% being conducted in North America. The next highest total is Asia Pacific with 14%. Not only is the total in Asia Pacific much smaller, but the countries and cultures are much more differentiated in Asia Pacific than they are in Europe or North America.
Therefore the notes below focus on differences in conducting online qualitative between Europe and North America, with references to other markets where available.
Recruiting Participants
The big demarcation in the recruitment of participants (online respondents in qualitative projects) is whether or not they tend to be recruited from panels (either online access panels or specialist qualitative panels). The alternative is to recruit participants via recruiters (assuming the participants are not coming from a client list).
As a generalisation, North America is more willing to accept recruitment from panels and Europe is less willing.
The willingness to recruit qualitative participants from panels has a direct impact on the attractiveness of online qualitative research. Online qualitative, when combined with recruitment from online panels, is more likely to generate speed and cost advantages. Online qualitative when combined with traditional recruiters, free finding sample, tends not to show significant cost and speed advantages.
Online Versus Offline Honesty
Although there are plenty of researchers who have wondered aloud about whether respondents online are more inclined to fib, there is little support for their concerns from the studies that have been conducted. There are studies that suggest for some topics people may be willing to be more honest online.
A 2009 paper investigated the views of online community participants in Australia, New Zealand, China, and Japan, and in both China and Japan the view from the participants was that it was easier to be honest online because they were less subject to social pressures (Poynter, Cierpicki, Cape, Lewis, and Vieira 2009).
Size of Groups
The is no agreed consensus for how big an offline or online focus group should be. The span seems to run from about four (sometimes called mini-depths) through to about fourteen.
As a generalisation, offline focus groups in North America tend to be larger than those in Europe. Wendy Gordon refers to ten to twelve participants as being common in the US, with six to eight being more common in UK (Gordon 1999). Gordon also highlights that the US model is more likely to be round a table, whereas the in the UK it is more typical to see soft chairs or sofas.
Hy Mariampolski refers to US offline groups as likely to comprise eight to ten participants, and refers to European groups as being more likely to have six to eight members (Mariampolski 2001). Mariampolski also draws attention to differences in the typical length of focus groups, with a typical offline group in the USA being two hours, whereas in Europe the groups tend to be longer, perhaps two to three hours. Gordon highlighted that the UK also had shorter (two hour groups) and that running two groups in one evening was not uncommon.
However, despite there being little agreement about the size of offline groups, there is a general view that online groups should be smaller than offline groups, both in Europe and North America.
Conformity to a Plan
Qualitative researchers differ in terms of the extent that they tend to stick to a specific plan. For some researchers the discussion guide is a short list of topic headings and perhaps some prepared stimuli. For other researchers the discussion guide can be a several page document, with a number of specific questions, potentially including the wording of specific questions.
As a generalisation, qualitative research in North America tends to stay closer to the plan than it does in Europe (but there are plenty of counter examples).
Online qualitative research, especially online focus groups, is closer in form to the practice of conforming to the plan. The words on the screen are all that the participants see, the text has to do all the work that the face-to-face moderator would aim to achieve with voice, body, intonation, and situation. Similarly, when the analysis of an online group is conducted, it is done so in terms of the specific words used by the researcher/moderator.
Counting and Voting in Groups
One phenomenon that is very rare in European face-to-face qualitative, but which not uncommon in North America, is the practice of asking a question to the group and asking them to vote (for example by raising their hands), and then recording the answers.
Fully-featured OLFG and BBG software have built in features that allow participants to vote and answer surveys, and this reflects that some researchers want to include this voting as part of their research method.
It is perhaps interesting to note that although the issue of voting inside online focus groups and bulletin board groups is still considered divisive, there is broad agreement that using polls and surveys within online research communities is a useful feature to have available.