One of my favourite blogs is the Bad Science blog written by journalist Ben Goldacre. One of his latest posts is about a claim, repeated in many newspapers, that drinking seven cups of coffee of day can make you see dead people. Goldacre dissects the story, showing how the press have misused the information, but also exposes the way that researchers appear to have taken spurious correlations and made a case from them.
Goldacre’s example of machine gunning a wall and looking for cases where three bullet holes end up close together is very reminiscent of some market research, and very similar to the research published a couple of years ago which a superficially appeared to link certain diseases to certain star signs.
When market researchers cross-tab everything by everything and set of a fishing expedition looking for ‘significant’ findings, they are doing exactly the same thing. It is only significant if you stated it was a possible source of difference before running the stats.