FAMOUS SCIENTIST TO ROBIN HIGH IQ HANSON: Science, which is a very long-term endeavor, does not need your stickin idea about scoreable predictions and track records. Please, go back to minding economic issues in your Ivory Tower, and let us run science our way, on our timing. Thanks. Appreciated.

No Gravatar

Overcoming Whatever:

I don&#8217-t really think the comparison with sports/business/weather forecasters really holds up, for a prosaic reason &#8212- in particle physics, the timescale for experiments is years and decades, not days. There is no way to efficiently grade/reward people on the accuracy of their predictions, and correspondingly no real incentive for anyone to make very quantitative predictions.

On the other hand, it&#8217-s not as if there is no incentive to be right. If you devote your life to working out the ramifications of low-energy supersymmetry and it&#8217-s not there, you won&#8217-t get fired (if you have tenure), but on the other hand your life&#8217-s work will be useless. Which is a pretty big incentive.

Posted by: Sean Carroll [from Cosmic Variance] | August 11, 2008 at 12:25 PM

&#8212-

Sean, I don&#8217-t understand the relevance of the timescale to the efficient grading of predictions. Given enough forecasts we can see a signal of accuracy above the noise of luck in individual forecasts. I agree that the longer the timescale the weaker are incentives from any given reward tied to scoring. But I&#8217-m not really focused on incentives in this post – I&#8217-m focused on whether it is reasonable for folks to crow about being vindicated when they weren&#8217-t willing to make scoreable forecasts.

Posted by: Robin Hanson | August 11, 2008 at 12:35 PM

Scientists don&#8217-t want to make scoreable forecasts.

Hence, it is impossible to collect track records.

Period.

Robin Hanson&#8217-s idea has no application &#8212-over than vanity blogging.

Let&#8217-s go back to our prediction markets (where traders work, for free, as info collectors).

Let&#8217-s not waste our precious time on fruitless ideas.

Track Record Collecting vs. Prediction Markets

No Gravatar

Robin Hanson&#8217-s false good idea: collecting track records.

But his post is the living proof that he is wrong:

  • Prediction markets incentivize traders in researching issues (reading the experts&#8217- works), making probability bets, and delivering a collective verdict-
  • Experts don&#8217-t like to state publicly their home-made probabilistic predictions &#8212-as his post shows.

And if experts are not used to express scoreable forecasts, then, by essence, you can&#8217-t collect anything. Hence, the superiority of the prediction market method.

Another false good idea from Robin Hanson.