A frequentist approach says that the probability of something happening is the value something would approach if you did the same experiment an infinite number of times. If you flipped a coin an infinite number of times, the chance of heads will be 0.5. But since it’s impossible to do something infinite times, you are likely to get a result that isn’t exactly right. It runs into problems when you talk about the probability of things that can’t happen anywhere close to infinite times. What is the probability it will rain tomorrow? You can’t repeat tomorrow more than once, let alone infinite times. Bayesian approach says that the underlying probability already exists, without infinite repeating. But you update your belief/knowledge of what that probability actually is with each bit of evidence. This means you can actually talk about probabilities for rare occurrences, or still get some meaning out of small samples. But it creates problems with defining what the original probability was before your evidence. e.g. can you be totally sure a coin has a 0.5 chance of coming up heads before you flip it?
Frequentist: “The only things that can have probabilities assigned to them are ‘events’ – repeatable things that actually happen out there in the world, that we can observe or not, and see what fraction of the time they occur.”
Bayesian: “No, for anything that we’re not sure about – including things like ‘whether general relativity is true’ we can assign probabilities to them as a way of quantifying our uncertainty. And then update those probabilities when data comes in.”
I mean, that’s kind of the philosophical “let’s have a heated debate” way of putting it, but you can also take a totally pragmatic view of the distinction too, where Frequentist basically just means “statistical methods that don’t involve assigning probabilities to parameter values” and Bayesian just means the ones where we do.
Comments
A frequentist approach says that the probability of something happening is the value something would approach if you did the same experiment an infinite number of times. If you flipped a coin an infinite number of times, the chance of heads will be 0.5. But since it’s impossible to do something infinite times, you are likely to get a result that isn’t exactly right. It runs into problems when you talk about the probability of things that can’t happen anywhere close to infinite times. What is the probability it will rain tomorrow? You can’t repeat tomorrow more than once, let alone infinite times. Bayesian approach says that the underlying probability already exists, without infinite repeating. But you update your belief/knowledge of what that probability actually is with each bit of evidence. This means you can actually talk about probabilities for rare occurrences, or still get some meaning out of small samples. But it creates problems with defining what the original probability was before your evidence. e.g. can you be totally sure a coin has a 0.5 chance of coming up heads before you flip it?
Frequentist: “The only things that can have probabilities assigned to them are ‘events’ – repeatable things that actually happen out there in the world, that we can observe or not, and see what fraction of the time they occur.”
Bayesian: “No, for anything that we’re not sure about – including things like ‘whether general relativity is true’ we can assign probabilities to them as a way of quantifying our uncertainty. And then update those probabilities when data comes in.”
I mean, that’s kind of the philosophical “let’s have a heated debate” way of putting it, but you can also take a totally pragmatic view of the distinction too, where Frequentist basically just means “statistical methods that don’t involve assigning probabilities to parameter values” and Bayesian just means the ones where we do.