I don’t really think the comparison with sports/business/weather forecasters really holds up, for a prosaic reason — in particle physics, the timescale for experiments is years and decades, not days. There is no way to efficiently grade/reward people on the accuracy of their predictions, and correspondingly no real incentive for anyone to make very quantitative predictions.

On the other hand, it’s not as if there is no incentive to be right. If you devote your life to working out the ramifications of low-energy supersymmetry and it’s not there, you won’t get fired (if you have tenure), but on the other hand your life’s work will be useless. Which is a pretty big incentive.

Posted by: Sean Carroll [from Cosmic Variance] | August 11, 2008 at 12:25 PM

—

Sean, I don’t understand the relevance of the timescale to the efficient grading of predictions. Given enough forecasts we can see a signal of accuracy above the noise of luck in individual forecasts. I agree that the longer the timescale the weaker are incentives from any given reward tied to scoring. But I’m not really focused on incentives in this post – I’m focused on whether it is reasonable for folks to crow about being vindicated when they weren’t willing to make scoreable forecasts.

Posted by: Robin Hanson | August 11, 2008 at 12:35 PM

-

Scientists don’t want to make scoreable forecasts.

Hence, it is impossible to collect track records.

Period.

Robin Hanson’s idea has no application —over than vanity blogging.

Let’s go back to our prediction markets (where traders work, for free, as info collectors).

Let’s not waste our precious time on fruitless ideas.

-

(In the wake of Milton Friedmann’s death, folks have been re-arguing his contention that successful predictions from an economic model are more important than correct assumptions underlying it. I would hope that both are important.)

Any comment on that, folks?

SHORT TERM: “Successful predictions” is the important stuff.

LONG TERM: “Correct assumptions” is the important stuff.

Am I right or wrong? (Will someone bring in the chicken-and-egg argument?)