With all due respect, that testing was done on markets 30 days before they closed. As I noted, time had almost run out on each of the markets. Consequently, one might expect an almost perfect accuracy and calibration.
The problem with these long-term markets is that they are usually very inaccurate for most of their durations. It is only when the revelation is near that the market becomes accurate (and calibrated). Unfortunately, it is of little use to have an accurate prediction just before the outcome becomes known. In order to act upon the market’s prediction, we need to know that it is accurate sufficiently far in advance to do something about it. On this basis, these markets are not particularly useful.
I think we can all agree that each market is unique, in terms of the issue to be predicted, available information set, trader pool, etc… I don’t think you can take a sample of prediction markets from on marketplace (ideosphere), find they are calibrated, and conclude that *all* of the marketplace’s PMs are calibrated.
I don’t have the answers, here, but it would seem to me that you need to control for either the information set or the participant pool, to be able to make a general statement of calibration accuracy for any particular type of PM.
