Why is statistics useful when many things that matter are one shot things?

I don’t know if it’s just me, but I am very skeptical of statistics in general. I can understand it in dice games, poker games, etc. Very small, simple, mostly self-contained repeated games are fine. For example, a coin landing on its edge is small enough to accept the probability that landing heads or tails is ~50%.

Playing a $10 game of poker aiming for a 95% win is fine. But what if your entire life savings + more is dependent on you hitting a win or not? How would knowing that you’d win in 95% of the time in that situation will help me at all? Expected value doesn’t help much there.

Other examples include a life-threatening surgery. How does that help knowing that it is 51% survival rate versus 99% survival rate given existing data? In both cases, I don’t think it will matter to me what the doctor tells me, and I would go for it. If actual data is 75%, he might as well tell me (barring ethics and law), that there is a 99.99999% chance of survival so I’d feel better. In other words, existing data doesn’t matter except binomially. Even then, it doesn’t matter if there is a 99.99999% survival rate, if I end up dying from it.

Also, earthquake probability. It doesn’t matter if a strong earthquake happened every x (where x > 100) years on average. I have no idea if an earthquake will happen ever in my lifetime. So why is it even useful information?

A less serious example, say, 100% of the places I’ve been to that I love are in the Americas, indifferent to 100% of the places I’ve been to in Europe, and hate 100% of the places that I have been to in Asia. Now, that by no means mean that I wouldn’t find a place that I love in Asia on my next trip or hate in Europe or indifferent in America, just by the very nature that the statistics doesn’t capture all of the information I need, and I probably can never capture all of the information I need, even if I have traveled to over x% of all of those continents. Just because there are unknowns in the 1-x% of those continents that I haven’t been to. (Feel free to replace the 100% with any other percentage).

I understand that there is no way to brute force everything and that you have to rely on statistics in many situations, but how can we believe that statistics are helpful in our one shot situation, especially when statistics basically do not extrapolate to outlier events?

Any insights to get over my skepticism of statistics?

Answer

First I think that you may be confusing “statistics” meaning a collection of numbers or other facts describing a group or situation, and “statistics” meaning the science of using data and information to understand the world in the face of variation (others may be able to improve on my definitions). Statisticians use both senses of the word, so it is not surprising when people mix them up.

Statistics (the science) is a lot about choosing strategies and choosing the best strategy even if we only get to apply it once. Some times when I (and others) teach probability we use the classic Monty Hall problem (3 doors, 2 goats, 1 car) to motivate it and we show how we can estimate probabilities by playing the game a bunch of times (not for prizes) and we can see that the “switch” strategy wins 2/3 of the time and the “stay” strategy only wins 1/3 of the time. Now if we had the opportunity to play the game a single time we would know some things about which strategy gives a better chance of winning.

The surgery example is similar, you will only have the surgery (or not have the surgery) once, but don’t you want to know which strategy benifits more people? If your choices are surgery with some chance greater than 0% of survival or no surgery and 0% of survival, then yes there is little difference between the surgery having 51% survival and 99.9% survival. But what if there are other options as well, you can choose between surgery, doing nothing (which has 25% survival) or a change of diet and exercise which has 75% survival (but requires effort on your part), now wouldn’t you care about if the surgery option has 51% vs. 99% survival?

Also consider the doctor, he will be doing more than just your surgery. If surgery has 99.9% survival then he has no reason to consider alternatives, but if it only has 51% survival then while it may be the best choice today, he should be looking for other alternatives that increase that survival. Yes even with 90% survival he will loose some patients, but which strategy gives him the best chance of saving the most patients?

This morning I wore my seat belt while driving (my usual strategy), but did not get in any accidents, so was my strategy a waste of time? If I knew when I would get in an accident then I could save time by only putting on the seat belt on those occasions and not on others. But I don’t know when I will be in an accident so I will stick with my wear the seat belt strategy because I believe it will give me the best chance if I ever am in an accident even if that means wasting a bit of time and effort in the high percentage (hopefully 100%) of times that there is no accident.

Attribution
Source : Link , Question Author : statskeptic , Answer Author : Greg Snow

Leave a Comment