Predictor Accuracy: the Hedgehog vs. the Fox

No Gravatar

&#8212-

Ellen Goodman:

[…] The closed-minded hedgehogs are those who know &#8221-one big thing&#8221- and relate everything to that single, central vision. The open-minded foxes &#8221-know many little things&#8221- and accept ambiguity and contradictions. […] It&#8217-s no surprise that foxes are better at forecasting than hedgehogs. […] How then do we cultivate good judgment? Most Americans are probably hybrid creatures. In a fox-like moment, Tetlock advises that we listen to our own ambivalence as &#8221-we struggle to strike the right balance between preserving our existing worldview and rethinking core assumptions.&#8221- […]

Hal Finney:

[…] One of the psychological measures or metrics which Tetlock found was well correlated with expert accuracy goes back to a distinction introduced by Isaiah Berlin in his book, The Hedgehog and the Fox. I haven&#8217-t read that book, but based on Tetlock&#8217-s presentation, Berlin distinguished between two cognitive styles to which he gave these colorful names. The hedgehog is said to know one thing and know it well. He sees events and trends in terms of his big idea, and aggressively extends it into new realms. Hedgehogs tend to be confident in the applicability of their fundamental concepts and impatient with those who &#8220-do not get it&#8221-. Foxes in contrast know many small things which they bring to bear in their analyses in a dynamical and flexible way. They tend to be uncertain and flexible, &#8220-on the other hand&#8221- types who are skeptical about their own predictive ability and in fact about the whole enterprise of making predictions in such an intractable realm. […]

Daniel Drezner:

A hedgehog is a person who sees international affairs to be ultimately determined by a single bottom-line force: balance-of-power considerations, or the clash of civilizations, or globalization and the spread of free markets. A hedgehog is the kind of person who holds a great-man theory of history, according to which the Cold War does not end if there is no Ronald Reagan. Or he or she might adhere to the “actor-dispensability thesis,” according to which Soviet Communism was doomed no matter what. Whatever it is, the big idea, and that idea alone, dictates the probable outcome of events. For the hedgehog, therefore, predictions that fail are only “off on timing,” or are “almost right,” derailed by an unforeseeable accident. There are always little swerves in the short run, but the long run irons them out.

Foxes, on the other hand, don’t see a single determining explanation in history. They tend, Tetlock says, “to see the world as a shifting mixture of self-fulfilling and self-negating prophecies: self-fulfilling ones in which success breeds success, and failure, failure but only up to a point, and then self-negating prophecies kick in as people recognize that things have gone too far.”

I wanted to go back to this 2005 book from Philip Tetlock (Expert Political Judgment: How Good Is It? How Can We Know?) because I have been thinking of the big issue of last Tuesday (Should a betting exchange be a content provider, too?). My answer is: A betting exchange has no business being a content provider&#8230- except if the betting exchange can and will develop some very special content that the traditional media can’t or won’t provide, and that is of high strategic interest. You&#8217-ll spot that that&#8217-s the kind of twisted answers that the foxes would give. And I am wondering whether one could say that the hedgehog thinking (read it, the bad thinking) is in fact represented in the two other camps:

  1. those who think that the betting exchanges have no business being content providers-
  2. those who think that the betting exchanges should also be content providers.

The answer to this dilemma is, of course, &#8230- and if Mike Linksvayer is reading this blog post, he has already divined where I want to lead my readers with all this &#8230- the answer is, of course, innovation.

&#8212-

InTrade-TradeSports should have expired the Larry Craig event derivative today, on October 4, 2007, and not on September 5, 2007.

No Gravatar

US Senator Larry Craig - Mugshot

InTrade-TradeSports should have expired the Larry Craig event derivative today, on October 4, 2007, and not on September 5, 2007. That&#8217-s their main error. Last September, they expired this event derivative on the basis on Larry Craig stating his &#8220-intent to resign&#8221- &#8212-which is different than to announce an upcoming or effective resignation. Larry Craig later changed his &#8220-intent&#8221- &#8212-he then intended to stay in the US Senate (providing he would be able to withdraw his guilty plea). Unfortunately, today, a judge ruled against his attempt to dismiss his guilty plea. So, US Senator Larry Craig is announcing today that:

  1. He is not resigning from the US Senate-
  2. He will not seek re-election for his US Senate seat.

The condition #2 is sufficient to expire the December 2007 Larry Craig contract on the &#8220-yes&#8221- side. (&#8221-Yes&#8221-, he is announcing his &#8220-intention not to run in 2008&#8243-.)

&#8212-

ADDENDUM: The original InTrade contract

Sen. Larry Craig to announce resignation or intention not to run in 2008 on/before 31 Dec 2007 – This event derivative has been expired by InTrade on September 5, 2007.

Larry Craig Resignation - InTrade Dec 2007

The rules (= the event derivative contract statement):

This contract will settle (expire) at 100 ($10.00) if Senator Larry Craig announces his resignation from the Senate or announces he will not run for re-election in 2008 on or before 11:59:59pm ET on the date specified in the contract.

The contract will settle (expire) at 0 ($0.00) if this does not happen on or before 11:59:59pm ET on the date specified in the contract.

A resignation does not have to result in the actual departure by the contract expiry date but rather the announcement of the resignation must be made before the date and time specified in the contract.

Expiry will be based on official, public announcements made by Larry Craig as reported by three independent and reliable media sources.

Due to the nature of this contract please also see Contract Rule 1.7 Unforeseen Circumstances.

The Exchange reserves the right to invoke Contract Rule 1.8 (Time Protection) if deemed appropriate.

Any changes to the result after the contract has expired will not be taken into account – Exchange Rule 1.4

Please contact the exchange by emailing [email protected] if you have any questions regarding this contract before you place a trade.

Important:
Please contact the Exchange if you have any query or uncertainty (including how it may be settled) about this Contract, the Rule above or the Contract Rules before you trade.

Prediction markets and the flow of information inside organizations

No Gravatar

Discussions of internal corporate prediction markets have sometimes pointed out that open, anonymous participation can lead to better information flow within the corporation. See, for example, Jed Christiansen’s write-up of the recent Consensus Point conference, particularly his notes on the presentations of Dave Perry and Fortune Elkins.

A lot of business research has been devoted to the topic of communication within an organization. An article from HBS Working Knowledge tries to sum up the current state of research into roadblocks to communication within an organization.

This propensity to maintain silence, a flaw at once personal and organizational, is &#8220-widespread and problematic&#8221- in both the public and the private sectors, says HBS professor Amy Edmondson, who chairs the Doctoral Programs and teaches in the Technology and Operations Management unit.

&#8220-To cite one example, former HBS doctoral student Jim Detert and I interviewed some 200 people of all ranks and functions in a high-tech multinational. We found to a very significant degree that people did not speak up about things they deemed important. Most of those were not &#8216-bad news&#8217- things- to our surprise we found that people were reluctant to voice what they perceived to be good ideas, unless they were extraordinarily confident they would be well-received. And this in a firm that lives and dies by its ideas.&#8221-

Edmondson says this reluctance to speak up stems variously from fears that superiors will not like the idea or that it may appear to criticize the status quo, which most people find reassuringly familiar or dangerous to challenge. Edmondson sums up the mental calculation this way: &#8220-The potential costs to me for speaking out seem reasonably certain and somewhat immediate- the potential benefit to me for speaking out seems rather uncertain and definitely long-range.&#8221-

The article cites HBS professor Max Bazerman to the effect that “within organizations, candor should be rewarded and incentives designed to encourage [it],” though the article does little to elaborate on the idea.

Instead, the article urges business executives to “develop disagreement” and suggests installing “a team at the top where high contention is demanded and rewarded.” During decision-making processes, according to one professor quoted, executives should ask probing questions and insist that managers “present each situation in objective terms, rather than with a positive spin.” The article sums up with, “What&#8217-s required in an organization is honest, thorough, and ongoing self-criticism, which, after all, is at the heart of continuous improvement.”

Part of the problem with the article is that it is exactly the old-style managerial approach that apparently doesn’t work. For how many decades have business executives been told to encourage honest disagreement, to reward challenges to the status quo? Saying “challenge the status quo” is the status quo. If it worked, then information would already always flow smoothly within corporations.

When the thing that stops an employee from speaking up is a mental decision calculus that compares relatively certain and immediate costs to uncertain future gains, as described by Edmonson, “demanding and rewarding high contention” just seems to raise the stakes. Raising the stakes seems like it would be counter-productive.

Maybe contentious meetings aren’t a good approach for eliciting honest revelations. So if what is said about prediction markets is true – if they can help information be gathered and processed in cases in which traditional ways of gathering and processing information fail – then they can be wonderful things. What was the example that Christiansen cited (from Perry’s presentation), a “forecasting team was trading against their official forecast”? That example shows information flowing more smoothly through prediction markets than through traditional channels.

The article hints at some of the reasons that prediction markets can improve information flow. With private, anonymous trading, the choice to disclose information via the market dramatically revises the mental decision calculus involved. The trader is rewarded if right and penalized if wrong, but in either case the disclosure and net reward is a private matter rather than social event. (At least until you brag about it around the water cooler.)

As the article said, candor should be rewarded and incentives designed to encourage it. Prediction markets provide incentives for candor. Not only that, but over time the traders with useful candor are encouraged by accumulated gains, while blowhards find their accounts diminished.

It is true that prediction market prices present relatively limited signals. Prices may go up or down, but they never say why. But with a signal, at least someone knows to start asking “why” and that is better than not knowing.

[Cross posted at Knowledge Problem.]

Political Strategy and Prediction Markets

No Gravatar

Strategists who want to know when a strategy is effective should pay attention to the market. Here is one losing strategy Edwards should drop:

According to Intrade.com, every time the Edwards presidential campaign (via Elizabeth Edwards) tries to engage Coulter in a catfight, his price plummets. His lowest point was after Elizabeth Edwards&#8217- &#8220-surprise&#8221- call-in to Hardball with Chris Matthews in June 2007.

Mr. Giuliani might want to check out intrade before he takes anymore calls mid-speech.

Cross posted from NastyBrutishAndTall

Robin Hansons annoying insistence on distinguishing prediction markets from betting markets

No Gravatar

Jed Christiansen has more on the 2007 Consensus Point conference on prediction markets&#8230-

Robin Hanson

Robin gave a quick addendum to his earlier talk, where he focused on the cost-value space of a prediction market. He described an evolution from betting markets, which have negative cost (aka profit) though little value to an organisation, to future prediction markets. Fully evolved prediction markets will certainly have a cost to operate, but the output could have tremendous value to a company.

Intro to Prediction Markets – (PPT file) – by Robin Hanson – 2007-09-24

prediction markets vs. betting markets

Is it the end for US-based betting exchange HedgeStreet?

No Gravatar

The End

Is HedgeStreet closing shop?

Yesterday, I re-published on Midas Oracle the e-mail sent to the HedgeStreet traders and prospects. (Curiously, this e-mail was not sent to me, even though I am in their listing.) They wrote that their overhauling would lead them to propose soon &#8220-new and exciting products&#8221-. Well, I found that interesting.

However, today, one of my Deep Throats ( :-D ) is telling me that it is all bullshit, actually.

  1. They are packaging HedgeStreet for a closing/fire-sale. Everyone is leaving, and only a skeleton crew is left.
  2. The CFTC has minimum capital requirements, which HedgeStreet is not able to sustain. They were burning $1m a month on 5k-10k of revenue, then tightening and tightening&#8230- There&#8217-s just no money left.
  3. UPDATE: Russell Andersson was laid off about 3 months ago. VP of finance leaving, VP of compliance leaving, etc. It is falling apart.

We will see.

Should a betting exchange be a content provider, too?

No Gravatar

Should BetFair have acquired TimeForm (the publisher of horseracing ratings, form guides and analysis)? Niall O&#8217-Connor (a betting market expert) thinks that there is a &#8220-strategic mismatch&#8221- between the two. I&#8217-m republishing his hatchet job, below. (I hope he won&#8217-t mind that I quote him in full, this time.)

At the end of November 2006 Betfair acquired Portway Press Limited, the owner of Timeform, the world-renowned brand involved in the publication of horseracing ratings, form guides and analysis. It was suggested at the time that the acquisition had cost Betfair somewhere in the region of ?15m.

At the time of the acquisition, Betfair announced that Timeform would retain full editorial independence, but that it would bring capital investment, technology, marketing and communication skills to the table.

On publishing its results for the year ended 30 April 2007, Betfair said that Timeform had &#8220-made a small loss in its first five months of trading after acquisition&#8221- but that it was confident that it would generate positive returns in the near future.

One can only speculate as to what the strategic thinking behind the acquisition was. Presumably it was believed that an association with the prestigious Timeform brand would make Betfair&#8217-s horse racing services more creditable- whilst at the same time affording cross-marketing opportunities.

Flawed logic at best&#8230-.

First, it is widely acknowledged that Betfair has given rise to, and nurtured, a culture of trading. Betfair traders do not typically run to form books, but rather, they let the Betfair market guide them, in the belief that it reflects all known information. (Betfair, with its lower transactions and information costs, provides its traders with a more realistic assessment as to the chance of longshots and the true probabilities of runners in a horse race- the favourite longshot bias is typically diminished, if not eroded).

Second, there is a strong case to be made that the incorporation of Timeform experts into Betfair&#8217-s horse racing radio service, has merely served to undermine the exclusiveness of the Timeform brand. Indeed, in situations where the Timeform expert&#8217-s selection fails to win, the brand is exposed to ridicule.


It is debatable whether the rather highbrow and serious research provided by Timeform, has any strategic fit with the Betfair trading culture.
And one is therefore left to conclude that Betfair has not only misjudged the market for Timeform products, but also, demonstrated a serious lack of awareness of its own customers.

The hard truth of a failed plan is that the core values of the Timeform brand have been diluted through its association with a commercial betting exchange.

Hummm&#8230- I won&#8217-t make any comment on TimeForm (which I&#8217-m not familiar with). However, I will say this, generally speaking.

  1. A betting exchange has no business being a content provider.
  2. Except if the betting exchange can and will develop some very special content that the traditional media can&#8217-t or won&#8217-t provide, and that is of high strategic interest.
  3. The difficulty, though, is that exchange executives and managers don&#8217-t have the first clue about how to set up and run a credible and popular media.
  4. That said, it could well be that some exchange-sponsored or -run media projects will succeed &#8212-due to some highly talented project managers (the exception, not the norm).

&#8212-

NEXT: Predictor Accuracy: the Hedgehog vs. the Fox

Two more wrap-up reviews of the 2007 Consensus Point conference on prediction markets

No Gravatar

&#8212-

Which way does Robin Hanson blow? When he was not involved, he had negative comments on internal prediction markets. Now that he is more involved (keynoting at a vendor conference), he has positive comments. Could it be that The Professor is biased? :-D

Weather Wane

&#8212-

#1. Justin Wolfers on the 2007 Consensus Point conference

Robin Hanson tells me that he is now (back to) bullish on prediction markets – he saw real evidence of real firms implementing prediction markets and taking them seriously.

#2. Jed Christiansen on the 2007 Consensus Point conference

Robin [Hanson] gave a fairly standard introduction to prediction markets lecture that some may have seen at other events or downloaded from his website. It was a good overview of the topic.

The question and answer period was the most interesting part with Robin. He was asked about manipulation, and provided some fairly convincing answers that manipulation shouldn’t be a worry (at least with the correct incentives.) Robin described the situation in terms of sheep and wolves. Sheep aren’t that knowledgeable- they are trading for any number of reasons, and are the “noise” in the marketplace. Wolves take advantage of that, and consequently they look for markets with lots of sheep. With better information, the wolves will easily have plenty to “eat.” The net result is that those noisy markets are accurate markets.

Another concept he talked about was creating a “fudge” account. Let’s say you want to weight one set of traders more than another, or simply want to “move” the forecast in one direction or another- create a “fudge” account to conduct those transactions. If after the account has been running for a while and it’s positive, you’ll know you’ve done a good job fudging. But if the fudge account is negative, you don’t know more than the market so just stop fudging and leave the market to itself. It’s a great idea, and fairly easy to implement.

UPDATE: Jed has more&#8230-

Robin Hanson

Robin gave a quick addendum to his earlier talk, where he focused on the cost-value space of a prediction market. He described an evolution from betting markets, which have negative cost (aka profit) though little value to an organisation, to future prediction markets. Fully evolved prediction markets will certainly have a cost to operate, but the output could have tremendous value to a company.

Encouraging participation in long-term information markets

No GravatarOne question that often comes up is how to encourage participation in long-term, multi-year event markets. This issue is closely linked to cost of capital, and even play-money markets have opportunity costs and discount rates.

If the outcome is revealed gradually, as with questions about the global climate, offering a series of short-term levered contracts should help. This is not entirely satisfying though because it could be viewed as just increasing the noise in traders&#8217- p&amp-l &#8212- although the traders with correct long-term views will do well, and the greater magnitude of the p&amp-l will attract more traders and keep them interested in the market.

There is just no way to settle a contract today for an event that will happen tomorrow. The most important thing you can do is to minimize the capital required to be committed to the distant event. For instance, traders may be compelled to only post or freeze a fraction of their worst-case loss, although this gets tricky if the contract will likely be resolved in a sudden, drastic jump. In this respect, questions about the climate are more tractable than ones concerning technological breakthroughs, for example. In many cases it may be better to use index futures instead of a binary options.

Most importantly perhaps, remember that most of what &#8220-prediction&#8221- markets do is just aggregating and discounting current information. Therefore ask yourself if anyone feasibly possesses information pertinent to the distant event. There may be nothing to aggregate. It may still be worthwhile to set-up a market because you&#8217-re not sure when someone will learn something relevant, but it is probably undesirable to try to force participation, even for very important questions.

Read the previous blog posts by Jason Ruspini:

  • 2009 tax futures yielding 1.5%
  • Intrade, with carry
  • Talking tax futures on BNN, Canada’s business channel
  • Tax Futures, “In Real Life”
  • How to sell art short
  • YooNew, fears and hopes
  • Policy Event Derivatives