Pascal's Wager and Prior Probabilities
Assigning a prior probability of 0
A Simple Game
You are playing a game, and if you flip and it ends up heads you get $1. You flip the coin again, and if it ends up heads again you get $1 again. And so on. Let’s call the amount of money you earn X. What’s the expected payout from this game?
On average, $1!
More generally though, the probability distribution is as follows:
P(X=0)=1/2
P(X=1)=1/4
P(X=2)=1/8
P(X=3)=1/16
…
P(X=n)=1/2^(n+1)
The point here is just that the probability for successively larger and larger amounts of money gets smaller and smaller. Why is this? Because each coin flip is independent, and when you look at the probability of multiple independent events occuring, the overall probability keeps getting smaller.
A Mugging
A person corners you in a dark alley. They walk up to you, but they have no weapon. Instead, they say to you, “I am a God and have the power to change your life. Give me $10 right now and I will double it for you and give it back to you. There are no downsides to this offer.”
Do you believe them? No. You say to yourself that the probability they are lying is exceedlingly high, let’s say there is a 10^(-10) chance they are telling the truth. It obviously does not make sense to take this person up on their offer.
Another Mugging
A day after this you are walking in a similar dark alley and a similar person walks up to you and says, “I am a God and have the power to change your life. Give me $10 right now and I will increase its value* for you by 10^(11) and give it back to you. There are no downsides to this offer.”
(*I say “value” here to not worry about inflation. The offer is to change the $10 into something worth $10^(12) of real wealth.)
Do you take this person up on their offer? In a similar situation you said that there was a 10^(-10) chance that the person was telling the truth. According to that math, the offer is a good one. I propose, however, the anti-mugging strategy: assign your prior probabilities so that as the amount of utility being offered increases, the probability that this is a realistic offer decreases commensurately.
Life In Number Land #1
You live in number land. A random number is drawn from the NumberMachine, which uses a uniform sample of the interval [0,1]. It is possible that the exact number drawn will be 1/2. Nonetheless, you assign the correct probability to this possible event happening a probability of p=0. “Wait!” your friend objects. “You are forgetting the classic result from Bayesian probability theory: if your prior probability for an event is zero, then no finite amount of evidence can make your posterior probability nonzero! You are being irrational!”
Are you?
Suppose the number is drawn, but you can only look at one decimal at a time. The number is drawn, and you look at the first decimal: 0.5…. . Should your probability that the number is 1/2 change? You look at the next decimal: 0.50…. And so on. Should your probability ever change? In this scenario, you correctly assigned a prior probability of p=0, and you are correctly never updating away from that prior probability no matter how many observations you make.
Life In Number Land #2
The scenario is the same, but now you can view the whole number at once. The machine tells you the number drawn was exactly 1/2. What went wrong?
Let’s take a look at that “classic result from Bayesian probability theory.”
1. Baye’s Theorem
The posterior probability of a hypothesis H given data D is:
P(H | D) = ( P(D | H) * P(H) ) / P(D)
where the total probability of the data is:
P(D) = P(D | H) * P(H) + P(D | not H) * P(not H)
2. Suppose your prior is zero
Set P(H) = 0. Then the numerator becomes:
P(D | H) * P(H) = P(D | H) * 0 = 0
The denominator becomes:
P(D) = P(D | H) * P(H) + P(D | not H) * P(not H)
= P(D | H) * 0 + P(D | not H) * 1
= P(D | not H)
So the posterior is:
P(H | D) = 0 / P(D | not H) = 0
But what if the denominator P(D) is also 0?
Life On Earth
Christianity claims that you can get an infinite reward. I submit that the correct prior probability for this claim is 0. So should no evidence change your mind? Yes, because we’re in the equivalent of Number Land #1.
You have been in heaven for 1 year. But it turns out this was all a prank by an entity who calls itself PrankDemon1. PrankDemon1 thought it would be funny to pretend that Christianity is true and give believers 1 year of heaven, but then flip the tables on them exactly at the 1 year mark. Crucially, you never had any evidence for this until the 1 year mark. What would have been the correct prior probability for this happening?
Different scenario: you made it past the 1 year mark. You have been in heaven for 2 year. But it turns out this was all a prank by an entity who calls itself PrankDemon2. PrankDemon2 thought it would be funny to pretend that Christianity is true and give believers 2 year of heaven, but then flip the tables on them exactly at the 2 year mark. Crucially, you never had any evidence for this until the 2 year mark. What would have been the correct prior probability for this happening?
Different scenario: you made it past the 2 year mark. You have been in heaven for 3 year. But it turns out this was all a prank by an entity who calls itself PrankDemon3. PrankDemon3 thought it would be funny to pretend that Christianity is true and give believers 3 year of heaven, but then flip the tables on them exactly at the 3 year mark. Crucially, you never had any evidence for this until the 3 year mark. What would have been the correct prior probability for this happening?
Okay, you get the idea. What’s the prior probability assignment that we live in a universe where PrankDemonN is in charge, for all N? Given your priors, how do you update?
Christianity+X
Suppose that Christianity was mostly true, but there was one small tweak, call it X. If you carefully assign prior probabilities to Christianity+X, then you might run into a contradiction. For there are an infinite number of such X, and your total probability mass won’t add up to 1. You can’t say that the probabilities drop off fast enough that the total probability approaches a finite amount, because P(Christianity+X)~P(Christianity) by design. If by “Christianity” you mean some Borel set that includes a lot of possible events, then please define your sample space!