Comparison of treatment of renal calculi by open surgery, percutaneous nephrolithotomy, and extracorporeal shockwave lithotripsy. In applied maths we deal not only with integrals and random processes, but also with ideas proposed by biologists, meteorologists, economists or medical doctors, as in this case. The trick is to pick out the essence of their research and forget about mathematically irrelevant parts. And not to get scared by the words we’ve never heard before.
The first mysterious sentence is just a title of a paper published in 1986. It contains results of a medical study comparing different treatments of kidney stones (renal calculi – sounds a bit like calculus, doesn’t it?). Why am I interested in a pretty outdated research into a nasty disease that I, luckily, don’t struggle with? Because its results are a beautiful illustration of the Simpson’s paradox, one of commonly misinterpreted statistical phenomena.
Disclaimer: I’m not a medical doctor (actually, I’m not a doctor at all – yet!), so if you suffer from renal calculi, please consult with a specialist before you decide on a particular treatment.
Researchers studied two types of treatments: open surgery and minimally-invasive procedures. They also looked at two groups of patients: with small and large kidney stones. As potential patients we care about success rates of both treatments: 78% for surgeries and 83% for minimally-invasive procedures. Clearly you should choose the second option, right? Well, it’s not that simple. Let’s take a closer look at the results presented in Table 1.
|Small stones||Group 1: 93% (81/87)||Group 2: 87% (234/270)|
|Large stones||Group 3: 73% (192/263)||Group 4: 69% (55/80)|
|Both||78% (273/350)||83% (289/350)|
Table 1: Success rates of kidney stones treatments; adapted from https://en.wikipedia.org/wiki/Simpson%27s_paradox.
If you happen to be a (un)lucky owner of small stones, according to this study you should decide on a surgery, as the success rate in 93% compared to 87% for minimally-invasive procedures. If your stones are large, the surgery remains your best option – 87% vs. the measly 69% for the other treatment.
What’s going on here: surgeries are better for patients with small as well as with large stones, but in some magical way when we combine these two groups, minimally-invasive procedures win? That’s exactly the Simpson’s paradox, a statistical phenomenon in which the relation between some variables disappears or reverses when we group or de-group the data. In English: the surgery seems to be better for both studied groups of patients, but worse for all of them combined. How is that possible?
Let’s look at the numbers instead of percentages. Note that patients with small stones usually avoid surgery – only 87 were operated, the remaining 270 had only a minimally-invasive procedure. However, patients with larger stones usually ended up on the operating table instead undergoing the less serious procedure: 270 vs. 80. So the severity of the disease is a key variable here! And when we look only at the last row of the table, so all patients combined, we forget about it. So it’s the severity of the case that has the biggest effect on the success of the treatment, not the treatment itself. It means that the more often doctors choose the less effective minimally-invasive procedures while treating less severe cases, the more effective this kind of treatment seems to be. If you missed the point, read the last sentence again!
This is just one example of how statistical results can be misinterpreted. Lies, damned lies and statistics? No, just statistics or ignorance. Stats can be a magical tool, but the Harry Potter generation knows very well how evil magic can be when misused. Soon I’ll describe other common statistical misconceptions, so stay tuned! And avoid too much salt, kidney stones don’t seem to be fun.