Early exercise on a dice game

(post from old site)

Rules:
you get to throw a fair die up to 3 times
the number of dollars you're gonna get is determined by the number you get on you last throw
How much would you pay to play this game?

=====================================================
First of all, if the game is free, what would your strategy be?

1st throw -----> 2nd throw -------------------> 3rd throw
              |-> early exercise     |-> early exercise

The above will be the decisions you'd have to make
after first throw, do you 
go for a 2nd throw, or
early exercise (ie taking the $ shown as on your first throw)?
Similarly, do you make the 3rd throw if you have done the 2nd one?


Let's determine the expected value (EV) for only one toss.  easy enough => 3.5.

part A
If my 2nd throw gets me 1, 2, or 3, I'll go for the 3rd throw.  Take the $ (exercise) otherwise.
  
After my 1st throw, it's a bit trickier.  Obviously, I'll go for the 2nd throw if my first throw is 1, 2, or 3, applying the same logic above.  If my first throw is 6, I will not go for the 2nd throw as I would have already achieved the highest payoff possible.

How about if you get 5 on your first toss? intuitively, I'd see my chance of getting a 6 on either 2nd throw or 3rd throw => 1 - p(1 to 5)*p(1 to 5), which essentially gives the chance of NOT (having first throw resulting in 1 TO 5 AND 2nd throw resulting in 1 TO 5)
you only have 11/36 to beat getting 5.  So, if your first toss gives 5, you'd early exercise

How about if you get 4 on your first toss?
similar calculation, you'd have 20/36 to beat getting 4.  In this case, you'd NOT exercise and go for the 2nd throw.

part b
So, that's the strategy.  Now, how much would you pay to play this game?

Again, working backward:
right before 2nd throw, 
50% chance to go for 3rd throw, which will result in EV of 3.5
50% of getting 4, 5, or 6, giving an EV of 5
=> before 2nd throw, the game is worth 4.25 

right before first throw,
66% chance to go for 2nd throw, which gives an EV of 4.25, as shown above
33% chance of getting 5 or 6, giving an EV of 5.5 
=> the game is worth $4.66666666...

So, if the game is offered to you below that price, go for it.  Or you can offer that game above that price (casino!).

One more note, we could have gotten the strategy by calculating the EV alone.  ie, part b alone will be sufficient.  since we get the EV of 2nd throw = $4.25.  If our first throw is above that number, we exercise.  If not, we play on.