Call Us: US - +1 845 478 5244 | UK - +44 20 7193 7850 | AUS - +61 2 8005 4826

Experimental and observational studies

Statistics is the science of changing your mind.

Making decisions based on facts (parameters) is hard enough as it is, but -curses!- sometimes we don’t even have the facts we need. Instead, what we know (our sample) is different from what we wish we knew (our population). That’s what it means to have uncertainty.

Statistics is the science of changing your mind under uncertainty. What might your mind be set to? A default action or a prior belief. What if your mind’s a blank slate? Read this instead.

Bayesians change their mind about beliefs.

Bayesian statistics is the school of thought that deals with incorporating data to update your beliefs. Bayesians like to report results using credible intervals (two numbers which are interpreted as, “I believe the answer lives between here and here”).

Frequentists change their mind about actions.

Frequentist statistics deals with changing your mind about actions. You don’t need to have a belief to have a default action, it’s simply what you’re committed to doing if you don’t analyze any data. Frequentist (a.k.a. classical) statistics is the one you’re more likely to encounter in the wild and in your STAT101 class, so let’s keep it classical for the rest of this article.

Hypotheses are descriptions of what the world might look like.

The null hypothesis describes all worlds where doing the default action is a happy choice; the alternative hypothesis is all other worlds. If I convince you -with data!- that you don’t live in the null hypothesis world, then you had better change your mind and take the alternative action.

For example: “We can walk to class together (default action) if you usually take under 15 minutes to get ready (null hypothesis), but if the evidence (data) suggests it’s longer (alternative hypothesis), you can walk by yourself because I’m outta here (alternative action).”

Testing in a nutshell: “Does our evidence make the null hypothesis look ridiculous?”

All of hypothesis testing is all about asking: does our evidence make the null hypothesis look ridiculous? Rejecting the null hypothesis means we learned something and we should change our minds. Not rejecting the null means we learned nothing interesting, just like going for a hike in the woods and seeing no humans doesn’t prove that there are no humans on the planet. It just means we didn’t learn anything interesting about humans existing. Does it make you sad to learn nothing? It shouldn’t, because you have a lovely insurance policy: you know exactly what action to take. If you learned nothing, you have no reason to change your mind, so keep doing the default action.