| Lies, Damned Lies and Statistics |
| Saturday, 01 October 2005 | |
|
Tom Walters puts risk and rationality under the spotlight The London bombings of 7 July 2005 and the attempted attacks two weeks later caused many people to abandon travelling by bus and tube. Those who chose to stay away from the tube network did so because they perceived the risk of travelling on public transport as too great. But what governs our perception of risk and how easy is it to be misled by the statistics we hear?
We rely on our intuition in almost all situations in daily life; if you walk into a crowded pub and feel threatened for some reason, it’s not because you’ve performed an analysis of all the possible dangers and mitigating factors. If you had stopped to calculate the exact probability that the bunch of guys with lots of empty pint glasses and a certain team’s football shirt on were just up for a quiet night, and weren’t in fact going to take offence at your choice of wardrobe today, it might well have been too late. We need to be able to inform our intuition by understanding and interpreting data on the dangers that face us, data that may come in many different forms. One factor in judging risks using our intuition is the availability of mental images related to an event about which we are concerned. Images which are particularly dramatic or disturbing, or which we are exposed to frequently, will be recalled more easily than other images and may make an event seem much more likely than it really is. The shocking nature and extensive media coverage of the London bombings made the images of them much more accessible in people’s minds, and thus made it seem more likely to them that the events would happen again. Conversely, hazards which are hard to visualise are often perceived as being less dangerous. Control is also another key factor in the perception of risk. If a hazard seems to be outside of your control it can appear more serious. Psychologists have suggested that qualitative aspects of risk such as these can be roughly split into two categories: ‘dread’ and ‘unknown’. Dread is typically associated with risks which appear uncontrollable or which have the potential for large-scale destruction, even though they may be far in the future. Hazards like nuclear power, radiation and climate change are seen as being high in both dread and unknown, whereas smoking and driving too fast are ranked much lower in both categories. This may explain why people are much more likely to voluntarily expose themselves to risks of the latter type.
While the experimental system is a simple and robust method for assessing risk, there is the danger that some risks can be perceived as different from what they actually are — sometimes wildly so. The analytic system can be seen as a kind of ‘reality check’ for our intuition. If some real data are available against which we can check our ideas, then it would be a good idea to use them. Hard facts are useful, but again, there are dangers in assessing the statistics with which we are presented. A lot of the confusion that is encountered when assessing statistics occurs simply because they are presented in a form that makes them difficult to comprehend. Percentages generally tell us very little. If there is a 10% rise in muggings per year, what does that mean; one more, making the total 11, or 1000 more, bringing the total to 11,000? Furthermore, what is the overall population in which these muggings are occurring: is it your street? An entire city? The country? One story which made front-page news in June 2005 was a four-year study that suggested that taking ibuprofen increases your risk of heart attack. The figure that many newspapers reported was that taking ibuprofen increases your risk of a heart attack by 24%. Pretty scary stuff by the sound of it. Statistics, especially on health-related stories, tend to get reported in this vague fashion. But is there a better way? Humans like dealing with real-life examples using numbers in a way which are easily comprehensible. Natural frequencies, which use actual numbers rather than percentages, are a good way of expressing risks because they allow us to put the information in a realistic context.The actual data from the study stated that there would be one extra heart attack per 1,000 or so people on ibuprofen. All of a sudden the statistic isn’t quite so shocking. Legal evidence is another area where there are numerous statistical pitfalls. Although juries do not assess risk per se, they do have to deal with a great deal of evidence, some of it backed up by statistics or probabilities and some not. In the course of assessing probabilities it can be easy to fall for logical traps. The recent notorious case of Roy Meadow, the paediatrician called as an expert witness in several cases where mothers were accused of killing their babies, is a prime example. In each case, the defendant claimed that the children had fallen victim to cot death or Sudden Infant Death Syndrome (SIDS).The testimony of Meadow as an expert witness was instrumental in the conviction of three mothers. However, this testimony was later called into question. Meadow was found to have misled the jury, causing him to be discredited as an expert witness, investigated by the General Medical Council and eventually struck off the medical register. In the case of Sally Clark, the mother’s two children had both died in similar circumstances. Meadow claimed that the probability that the defendant was innocent in this case was about 1 in 73 million. It appears that he reasoned that since the incidence of SIDS is around 1 in 8,500, the probability of two cases occurring in the same family was that value squared, leading to the value of 1 in 73 million. He then inferred this to be the probability that the defendant was innocent — damning testimony from an expert witness. There were, however, two errors in Meadow’s reasoning. The first was the assumption that the two deaths were independent; that the fact that one death had already occurred had no bearing on the probability of another death occurring. The probability that the second death occurred due to natural causes should be calculated as the probability of a case of SIDS occurring given that a case has already occurred in the same family. Due to the possibility of either a genetic predisposition to SIDS or shared environmental factors, it is reasonable to believe that the probability of this would be considerably greater than 1 in 8,500. The Royal Statistical Society later criticized Meadow’s claim of 73 million to 1, saying that it had “no statistical basis”. The second error of reasoning that helped the jury to reach a guilty verdict in this case is more subtle, but has misled juries in many cases. Many errors of reasoning (fallacies) such as this are so common that they have their own name. It seems that the trap that Meadow, unwittingly or not, led the jury into was the ‘prosecutor’s fallacy’: mixing up his conditional probabilities. (See ‘The Prosecutor’s Fallacy’, below.) The figure that the prosecution quoted was the probability of a double cot death occurring given that the defendant was innocent: very small indeed. The quantity that we are actually interested in, however, is the probability that the defendant is innocent, given that a double cot death has occurred. These two quantities are not the same. That isn’t, of course, to say that the defendant was definitely innocent, but the crucial thing to remember when assessing statistics is that it must be done in context. In the case of a legal proceeding, that means evaluating the statistical and the non-statistical evidence simultaneously, quantifying what you can, and then leaving the jury to come to a reasoned conclusion. So, the moral of the story seems to be: trust your intuition, but don’t forget to give yourself a reality check every so often. And, when dealing with statistics, remember to look beyond the numbers to check that the fancy figures mean what you think they do. Tom Walters is a PhD student in the Centre for the Neural Basis of Hearing References: Maule, A. J. Translating risk management knowledge: The lessons to be learned from research on the perception and communication of risk. Risk Management: An International Journal (In Press) |
Analysis of the risks that surround us is something that we have to do every day. Cognitive psychologists suggest that there are two mechanisms by which humans can judge risk: the ‘experimental system’ and the ‘analytic system’.The former gives rise to our intuitive understanding of risk and relies on images and associations formed in our minds. This process occurs with little conscious control and it is the system which gives us a ‘gut feeling’ that something is wrong — that we shouldn’t eat some strangesmelling food, or walk down a dark alley. By contrast, the analytic system requires much more conscious thought. This is the system by which we logically analyse evidence and statistics in order to reach reasoned conclusions. 
| < Prev | Next > |
|---|
News Archives
| News Archives |
