Text Mining and Prediction Markets
Bo Cowgill December 9th, 2007
The recent work on the intersection between text mining and prices counts as some of the most interesting work being done in the prediction market field. A few thoughts:
- The concept has clear applications outside of political markets: It could be used as a trading strategy (for example, see how traders in real markets are already using similar text mining strategies) or as a market maker for illiquid markets.
- There are applications for corporate prediction markets as well: Every day, volumes of easily accessible text are produced about companies. This comes in the firm of news articles and analysis by third parties, traffic on internal company email lists, internal and external blog posts, wikis and other available firm documents. As I wrote in July, a shrewed text mining algorithm could turn these writings into liquidity for the internal prediction markets. It’d be a bit of a problem for the current generation of prediction market vendors, who might have trouble gaining access to all these documents.
- In either case, traders or market makers using text-mining for algorithmic trading need to deal with potential spammers who could write articles to trick the algorithm into submitting dumb bids.
On that note, the authors should consider text-mining political blogs in search of a better algorithm for predicting price movements. Firms such as Collective Intellect or Neilsen’s BlogPulse already have a large store of blog-mining data. Note that Collective Intellect is already designed to service the financial industry with data from text-mining blogs.
For text-mining junkies, Text Mining, Visualization and Social Media is a great blog.
- All Best Posts Ever , All Guest Authors's Posts , Analysis (Data) , Analysis (Meta) , Primary Indicators
- Comments(6)
Cross-posted? Don’t you know about Google’s duplicate content penalty
Heh well I think this is how Chris wants it to be … Chris?
I could not agree more.
Actually, automatic trading (possible backed by text mining algorithms) can improve substantially the performance of current prediction markets, in the same way that automatic trading in the 1980’s improved the efficiency of the stock market. I have heard stories from old-timers that in the 1980’s most of the algorithmic trading approaches were really “printing money”. Over time, though, the markets became more efficient and predictions harder. It would be ideal if we could make prediction markets hard to predict
Something similar can be done with prediction markets. Build a platform that allows automatic trading and then let people build automatic traders for playing these markets (even play money). If not anything else, it would be a great educational tool for students interested in learning about trading and about prediction markets. Is Google interested in funding such an academic effort?
We are also considering using BlogScope for mining political blogs to improve the predictions. But let’s proceed one step at a time.
About Cross-Posting:
#1. Most people who read Midas Oracle don’t read BoCowgill.com, and conversely. So it’s great, when there is an important post on either blog, that one blog excerpts and links to the other one.
#2. Most people won’t follow links. If there is something important, it’s better to excerpt it in all —that’s called “cross-posting”.
#3. If we want to cross-post, Google Webmaster Central advises us this way: “If you syndicate your content on other sites, make sure they include a link back to the original article on each syndicated article.”
http://googlewebmastercentral......ntent.html
–> Bo Cowgill did fine, except that he should have deep-linked (as opposed to root-link) to his blog post at BoCowgill.com —and the link back to Midas Oracle is totally necessary, by the way.
–> The idea here is to tell the search engines (and the readers) that BoCowgill.com is the owner of the content. That’s why a deep link from Midas Oracle to BoCowgill.com is necessary.
–> How To Publish
http://www.midasoracle.org/authors/how-to-publish/
I think that it is important that text mining is put into its proper historical context; lest people should get the impression that it is some sort of new fangled cutting edge application. A decent historical analysis and summary of the pitfalls may be found here;
http://www.ie.iwi.unibe.ch/pub.....WP-184.pdf
Political Prediction Markets: Some Thoughts
http://behind-the-enemy-lines......-some.html