With the ubiquitous use of web-based and wireless Social Networks, people are increasingly using the term “Collective Intelligence“. However, I do have serious doubts they really understand what they meant. Some call it the wisdom of crowds or collective wisdom, others smart mobs, while others wealth of knowledge, world brain and so on. Moreover, turning things worse, there are those also, which tend to see it, or confound it with crowd-sourcing as well as prediction markets. Even if there are some loosely conceptual bridges between all them, it will be probably useful to know that the term was instead been born over the Artificial Intelligence research area, while exploiting stigmergic phenomena (see also Swarm Intelligence) among ensembles of cooperative agents. So what follows is a recent definition provided by Univ. of Alberta, Canada. This entry was added last month (Nov. 2009) at the Dictionary of Cognitive Science (Michael R.W. Dawson, David A. Medler Eds.):

Collective intelligence – is a term that refers to the computational abilities of a group of agents. With collective intelligence, a group is capable of accomplishing a task, or of solving an information processing problem, that is beyond the capabilities of an individual agent.

Collective intelligence depends on more than mere numbers of agents.  For a collective to be considered intelligent, the whole must be greater than the sum of its parts.  This idea has been used to identify the presence of collective intelligence by relating the amount of work done by a collective to the number of agents in the collection (Beni & Wang, 1991). If there is a linear increase in amount of work done as a function of the number of agents, then collective intelligence is not evident. However, if there is a nonlinear increase (e.g., an exponential increase) in the amount of work done as a function of the number of agents, then Beni and Wang argue that this is evidence that the collective is intelligent.

Collective intelligence is of interest in cognitive science because many colonies of social insects appear to exhibit this kind of intelligence, and this has inspired researchers to explore “porting” such processing to robot collectives. As far as robots are concerned, collective intelligence is exciting because it offers the possiblity of developing systems that are scalable (they don’t get disrupted when more agents are added) and flexible (they don’t get disrupted when some agents are damaged or fail) (Sharkey, 2006).

References:

1. Beni, G., & Wang, J. (1991, April 9-11). Theoretical problems for the realization of distributed robotic systems. Paper presented at the IEEE International Conference on Robotics and Automation, Sacramento, CA.
2. Sharkey, A. J. C. (2006). Robots, insects and swarm intelligence. Artificial Intelligence Review, 26(4), 255-268.

I came across this post: On Prediction Markets for Climate Change by Rajiv Sethi, an economics professor at Columbia University.  In his post, he makes a very interesting point that I have yet to see in any research paper about prediction markets.  He was commenting on the recent debate between Matt Yglesias and Nate Silver, regarding the use of prediction markets to help guide policy about climate change.  By way of a very brief summary, Matt believes that big business (coal and oil) will manipulate the market to influence the setting (or not) of policies that would be detrimental to their interests.  Nate thinks this is rubbish.  If the markets are broad-based and have sufficient liquidity, attempts to manipulate the market price will not succeed.  Nate thinks the markets would be “efficient”, providing market prices that accurately aggregate available public information.

Compelling Logic?

Here is where it starts to get interesting.  Rajiv comments that the logic of Nate Silver’s position is so compelling, it simply must be true.  That is, broader participation and more liquidity makes for efficient markets that generate more accurate prices.  To his credit (and I might add that he seems to be the only one), Rajiv set out to see whether this holds up in the real world.  He used Intrade and IEM markets about the 2008 election.  The hypothesis was that the IEM markets, with a more limited base and lower trade volumes, should have been less efficient than the Intrade markets.  Instead, he found the opposite!  Compelling, indeed.

How Do You Measure Efficiency?

“First of all, let’s think for a minute about how one might determine which of two markets is aggregating information more efficiently. We can’t just look at events that occurred and examine which of the two markets assigned such events greater probability, because low probability events do indeed sometimes occur.   If we had a very large number of events (as in weather forecasting) then one could construct calibration curves to compare markets, but the number of contracts on IEM is very small and this option is not available. So what do we do?”

This paragraph from Rajiv’s post, summarizes the problem of determining whether a market is “accurate”.  We believe that if a market is well-calibrated, the distribution of its market prices will be “accurate”, reflecting all market information about the outcome.  Consequently, it will be described as “efficient”.  He points out the difficulty (in most cases the impossibility) of measuring the calibration of a market and asks “what do we do?”

Essentially, he comes to the conclusion that it is impossible to measure the efficiency of a market.  However, it is possible to say which market is more efficient.  In other words, we can determine relative efficiency of two markets.  He outlines a cross-market arbitrage mechanism that could be used to eliminate price differentials for identical contracts in different markets.  You can read the approach in his post, cited above.  While he did not actually run the arbitrage experiment, he did perform an informal test to determine which of two markets was more efficient. 

The market with the smaller change in price is the more efficient of the two markets.  Effectively, then, the more efficient market’s price will be a better predictor of the future market price in the other market.  This was how he determined that the IEM markets were more efficient than those in Intrade, despite there having a limited participant pool and lesser liquidity.

So far, we have been able to determine which of two markets is the more efficient, but we don’t know how much more efficient.  Also, we don’t know whether either market is  sufficiently “efficient” for the purpose of determining its accuracy.  Both markets may be “inefficient”, yielding inaccurate or misleading market prices. 

How did IEM do it?

Rajiv gives two possible explanations as to why the IEM markets were more efficient than the Intrade ones.  Neither explanation is good news for Nate Silver’s position.

One explanation has manipulative traders moving into the Intrade markets, in order to influence the prices (odds) quoted in the media and in political blogs.  The argument is that Intrade prices were much more widely cited than those of the IEM markets.  The reasoning goes that temporary dips in market prices can be eliminated through manipulative trading.  A political party may wish to see this done, so as not to upset campaign contributions or to minimize the impact of negative information.  The author argues that the benefit of such manipulative trading could be far in excess of the cost.  Since IEM’s markets were not as widely cited in the media or blogosphere, there was a lesser incentive to manipulate prices there.

Even if we believe the research (limited) on manipulation in prediction markets, it is more than likely that a short term (maybe even a very short term) manipulation could persist long enough to achieve the intended objective.   For example, the price could be manipulated just prior to when news stories are being finalized for the following day’s paper.  Once the paper hits the streets, the manipulated price may have been corrected, but the damage has already been done.  And this is the “best case” scenario regarding prediction market manipulation.  In the worst case, the manipulation is successful as the market is unable to correct the inaccurate price.

I’m not an expert on US campaign finance, but I wonder whether an Intrade market manipulator would need to declare the amount of funds used to implement the price manipulation scheme (or whether such a person or corporation would be considered a donor at all).  If the answer is no, it would provide an additional incentive for parties or candidates to manipulate the markets for political purposes (without having to account for the funds used).  We all know what happens when incentives are strengthened.

The other explanation is that inefficient markets attract higher participation rates and market liquidity, as traders seek to profit from inaccurate prices.  Efficient markets have fewer profit opportunities and less trading is required to keep prices accurate.  As Rajiv explains, Nate Silver is caught in a paradox.  Nate’s attempt to design a market with high participation and strong liquidity, in order to achieve efficiency (and hence, accurate prices), conflicts with Rajiv’s finding that it is the market inefficiency that generates the high participation and liquidity.

The Road Ahead

Despite all of these arguments, Rajiv Sethi believes that prediction markets on climate change topics should be tried.  He suggests that corresponding markets be offered in other marketplaces, such as the IEM, so that market efficiency comparisons can be performed and studied.  I’m sure useful information could be gleaned from this effort. 

We need to keep in mind that some (or most) prediction markets may not work, however.  The objective of prediction markets is to accurately aggregate information held by the market participants.  If those participants do not have the information (or are unable to get it and profit from it), the market will be unable to generate an accurate prediction or there will be too much uncertainty about the prediction, rendering it useless for decision-making.

Personally, I like the idea of decision markets, but I think we will find that our efforts to use these markets to help guide climate change policy will ultimately fail.  There is simply too much information that is needed to accurately predict the important metrics.  It is hopeless to think that, not only will there be “informed” traders, they will be able to counteract the trading of the uninformed traders and the manipulators.  Any useful standard of “informed” traders might result in a mere handful of individuals spread throughout the world.  The impact of manipulators would swamp any efforts of the informed to set the “right” price in the market.  That said, there may be metrics that can be predicted (with reasonable accuracy) by a large number of traders.  Such predictions could be used as inputs into public policy decision models.  As with all prediction markets, the predictions must be accurate and consistently so.

Information Aggregation and Manipulation in an Experimental Market – Robin Hanson, Ryan Oprea, David Porter

This study looks at price accuracy in experimental (laboratory) markets, where there are price manipulators.  The overall finding is that non-manipulative traders compensate for the bias inherent in the offers from manipulators, by setting a different threshold for trading.  The authors acknowledge that the “identification of manipulation in the field is difficult” and empirical evidence is scarce and tenuous.  Hence the need for a controlled, laboratory experiment.  For background on the experiments, please refer to the original paper.

There were two parts to this experiment.  In the Replication Treatment, there were no manipulators present, and in the Manipulation Treatment, one-half of the participants were given an incentive to increase the median price at the close of the market.  All participants knew that half of their number had this incentive to manipulate, and they knew the direction that the manipulation would take (upward). Where the non-manipulative traders knew that the manipulative traders would attempt to bid up the price in the market, they lowered their threshold for accepting offers, effectively counteracting the manipulative influence in the market. This makes intuitive sense, but only in the case where the non-manipulative traders know the direction of the manipulation.

In my previous post, I indicated that it would be necessary for the non-manipulative (“informed”) traders to know which direction the manipulators would try to move the market.  Robin Hanson commented that this is not necessary.  I think he is wrong, now, but he was right when this paper was written!   I think the authors are saying that it is required.  In fact, in the paper, they go a step further and allow all participants to know the strength of the incentive to manipulate.  We should keep in mind that, while this experiment demonstrates the concept of market manipulation and whether it can have a persistent effect on market prices, it is a pretty simple, controlled example.  The real question is whether it can be generalized to more complex, real-world situations.

Can Manipulators Mislead Market Observers? – Ryan Oprea, David Porter, Chris Hibbert, Robin Hanson and Dorina Tila.

This study showed that uninformed third parties (observers) are able to make significantly better forecasts of asset values based on market prices (of those values) in an experimental market.  Even when half of the traders attempted to manipulate the market, the observers’ forecasts were no less accurate.

It appears that the observers are able to adjust the market price to remove most, or all, of the effects of manipulation.  To me, this means the observers were using some other form of decision model to arrive at their forecast.  Such a model used the market price along with other trade data, enabling the observer to alter the forecast from that determined by the market price alone.  The authors note that the observers were able to do this, despite the fact that the non-manipulative traders and the observers did not know which direction the incentives for manipulation ran.

This is quite a remarkable result.  It would have been nice to know how they were able to make these accurate forecasts with market price data that had been manipulated.  One of the findings was that upward price manipulation resulted in about a 7% increase in the market price (though there was no similar effect for downward manipulation).  The authors note that further study is required along with robustness tests.  I agree that it might yield very useful insight into the process of making a forecast based on prediction market prices.

In a sense, the observer should be considered a decision-maker.  If decision-makers are able to filter out the effects of manipulation in a real public policy prediction market and make an accurate forecast of the underlying metric, perhaps there is a role for such markets.  I would feel a lot more comfortable if we knew how the decision-maker (observer) is able to accomplish this feat.  Finally, we need to know if this was only possible, because it was a fairly simple experimental model.  Will the same decision-maker’s  ability exist in extremely complex public policy markets?

In their paper, A Manipulator Can Aid Prediction Market Accuracy, Robin Hanson and Ryan Oprea use a theoretical model to show that a market can become more accurate when manipulators are present, by increasing the returns to informed trading, which provides incentives for traders to become informed.  However, given the number of assumptions made in the model, the authors caution that the “findings may not be robust” and that “since this is not a fully general model, it cannot by itself support strong general claims about the price effects of manipulation.”  So far, so good, we are in complete agreement, at least for some markets!

There are quite a few assumptions in the model, including:  “risk-neutrality, normally distributed values and signal errors, interior choices of information quantity, no transaction costs of trading, no budget constraints, and a single rational manipulator with quadratic manipulation preferences and a commonly known strength of desire to manipulate.”  The authors do examine the potential effects on their conclusions, if some of the assumptions don’t hold true in practice.   Let’s look at some of them.

A Manipulative Conspiracy

For example, if there were a conspiracy among most (maybe “many” would be enough?) traders to pursue a common manipulation objective, the supremacy of the informed over the manipulative traders could be upset.  This isn’t as far-fetched as it may sound.  Large, politically affiliated groups, unions, and industry association groups of members could be inspired to conspire either directly or indirectly (propaganda).

Uninformed by Choice or Constraint

The authors assume that providing an inducement for traders to become better informed, they will actually become better informed. 

What if it is not possible for traders to become sufficiently “informed”?  This could be the result of the issue being too complex or uncertain, or it could be that the cost of becoming sufficiently informed outweighs the benefits of using that information. 

For example, is it even possible for the average bettor to “read up” on climate change research to the extent necessary to determine that the market has been manipulated or that the market reflects too much uninformed, noise trading?  I highly doubt it. 

It could very well be that some issues (like this one) are so complex and so uncertain as to be unpredictable, until very soon before the market closes.  It is only the march of time that whittles away the uncertainty.

Relatively Speaking

The authors state that “when potentially informed traders have deep pockets relative to the volume of noise trading, increases in trading noise do not directly effect price accuracy.”  This assumes that traders can be “informed” and have a sufficient volume relative to noise trade volume (including that of manipulators).  I would argue that a market (such as a climate change PM) does not meet this condition. 

Creating a Prediction out of Nothing

The authors state that “historical, field, and laboratory data, however, have usually failed to find substantial effects of such manipulation on average price accuracy.”  Though this may be true, what happens in a market that has no clear average price (i.e. has a flat distribution)?  Couldn’t a manipulator create a misleadingly “accurate” market price?  The existence of a flat distribution (before manipulation) indicates the market does not have sufficient information to make a prediction.  The traders are uninformed.  Such a market would be ripe for a manipulation, and the market would not have enough informed traders to know what was happening or to do anything about it. 

Conclusion

On balance, I think the authors realize that there is a potential for markets to eliminate the effects of manipulation in some markets, if the necessary conditions and assumptions hold true.  In highly complex, uncertain situations, some of the key assumptions are unlikely to be met, calling into question the conclusion of the paper.  This is what I was trying to get across in my previous post.  Perhaps the single most important condition or assumption in their model is that the informed traders have relatively more trading volume than the noise traders (manipulators).  I explained why I didn’t think this would hold true for all markets.  In the authors’ paper discussed here, they simply state this condition as a fact.   They did warn us, however, that the findings may not be robust or generally applicable to all markets.

One down, three more to go (papers that is).

Robin Hanson and others have suggested that prediction markets be used to help shape the direction of public policy.  The current hot issue is how to combat global warming and its effects on the environment. 

Matt Yglesias has argued that big money can manipulate markets.  So, we should not use prediction markets for this purpose.  On paper, prediction markets provide monetary or ego-related rewards for truthfully revealing private information by trading.  In this sense, prediction markets are said to “incentivize accuracy”.  When the incentives for manipulating the market price are greater than the incentives for not doing so, it is obvious how traders will act.  Matt argues that prediction markets that are prone to manipulation, such as climate change futures, will make inaccurate predictions, and any policy that is based on these will be inappropriate.  I agree.

Robin Hanson, on the other hand, believes that big money manipulators can only improve the accuracy of prediction markets.  He goes so far as to say that prediction markets are “especially incorruptible”.  I need to read all of his papers on this subject in their entirety, however, based on his own summary of the findings, I will make a few comments, now.  [I promise I will read them, fully, and update this post if necessary]

Robin (and others) argue that prediction market accuracy improves “as more big money powers are known to want to manipulate them.”  Manipulators are in essence noise traders.  Markets with more noise traders are more accurate, because informed traders are attracted to the possibility of profiting by trading with the noise traders. 

He qualifies his conclusion by stating that “this isn’t an absolute guarantee.”  Then, he suggests that we try it before we condemn it.  However, before we do so, I suggest we look at the theory more closely.  We may find that it works as well as the neoclassical economic framework in economics.  It works fine in a hypothetical, assumption-simplified world, but fails miserably in practice. 

Let’s look at some of the simplifying assumptions in Robin Hanson’s application of prediction market theory.  One, the informed traders are more powerful than the manipulators, or noise traders.  In Hanson’s experiments, the manipulators are able to affect the market price, but the informed traders quickly bring prices back to an accurate level. 

What if informed traders aren’t wealthier (than the manipulators)?

In a typical prediction market, greater trader wealth is accumulated by being better informed than other traders and making trades that payoff more frequently.  By virtue of their greater wealth, informed traders have more power to influence the market than uninformed traders.  This is a necessary condition to mitigate against manipulative behaviour.

In a public, real money market, trader wealth may have nothing at all to do with knowledge about that, or any other, outcome.  Manipulative traders can simply bring wealth to the market.  Furthermore, if such wealth is known to other traders, it may send a false signal to all traders about the manipulator’s “expert” status.  That is, rather than being viewed as a manipulator, the trader may be seen as an expert.  This is especially likely in markets where it is difficult (or impossible) for any individual to have enough knowledge to make an “informed” trade.  Even if you place restrictions on wealth that may be traded, so as to prevent a small group of traders from manipulating the market, if the stakes are high enough, the big money manipulator will simply finance a large number of other traders to carry out the manipulation.  I think many big money players would find the incentives large enough in the global warming debate.

What if most (or all) of the traders are uninformed? 

I would argue that as long as the collective information set is sufficiently complete, the market could obtain a reasonably accurate prediction.  If this is not the case, we will likely see a very flat distribution of predictions, reflecting the high degree of uncertainty.  Such a result would be practically useless for policy decision-making, other than to indicate that we need much more information about the subject.  Unfortunately, for an extremely complex issue, like climate change, it is highly unlikely that the market participants will have a “complete” set of information.  It is doubtful whether any of the participants would be able to properly weigh and assess all of the information, in order to make a truly accurate prediction of any climate change metric.  There are simply no, known frameworks for making such assessments, which leads us to…

Another possibility is that if traders have very little personal information about the subject, they will instinctively look to the others (the market) for guidance.  The prediction market principle of independence begins to break down.  If the market price has been manipulated, there is a good chance that the non-manipulative traders (notice I didn’t say “informed”) may “read” information in the price that isn’t true and place their trades accordingly. 

Public vs. Enterprise Prediction Market Manipulation

One of the reasons I haven’t looked into the issue of market manipulation is that it isn’t much of a problem in enterprise prediction markets.  Generally, we expect EPMs to have a sufficient number of informed traders, who tend to be “wealthier” than manipulators.  There are some noise traders, but not too many.  I agree with Robin Hanson’s assessment that manipulation will be overcome in enterprise markets.  Consequently, I’ve had little interest in looking at this issue.

However, prediction markets on public policy issues are different.  Apart from the market participants, there are many groups that have vested interests in the implications that might flow from a public policy prediction market outcome, and they will seek to influence the market prediction, by trading or by other means.  For example, big business may try to influence the information available to all traders to achieve the desired prediction.  This may take the form of advertising, public announcements, privately funded research, and all forms of lobbying activity.  Governments issue their own propaganda.  This information may be corroborated with price changes in the prediction market, lending credibility to inaccurate information.  Unless these prediction markets can be insulated from the manipulative influence of non-trading interest groups, they will not be able to prevent or eliminate manipulation of the market predictions.

How Manipulation is Nullified (or not)

Robin Hanson states that the informed traders must know that the noise traders want to manipulate the market.  In order to profit from this knowledge, they also need to know which way they wish to manipulate the market price. 

In a global warming market, big business, carbon emitters would likely exert downward pressure on any metric that shows adverse effects from their activities, so that legislators would be less likely to impose costly laws to prevent such activities or to compensate others for the effects.  On the other hand, “tree-hugging” organizations may wish to increase the market price, so that such legislation is more likely to be enacted.  In both cases, the truly informed trader must know who the trader is and the trader’s motive for trading.  Since there is no way to prevent a trader from diguising his identity, it is impossible to properly match the motive with the trader.  It also begs the following question.

How might the informed trader distinguish between a manipulative trader and a misinformed honest trader?  I don’t have that answer, but unless it can be answered,  it may be impossible to ensure that attempts at manipulation will lead to more accurate predictions, at least in complex, public policy prediction markets. 

Conclusion

In theory, it is a nice idea to try and accurately aggregate as much information as possible in order to determine the best course of action in public policy decisions.  Most public policy decisions are remarkably complex with numerous tradeoffs among competing interests.  All decision-making benefits from more information that is more accurate and more timely.  Unfortunately, simply inserting a prediction market framework into the decision-making process does not eliminate the political biases that have been, and will always be, there. 

While it may be possible to operate public policy prediction markets for some issues, their use in the climate change or global warming debate is questionable.  Not only can there be no guarantee of manipulation-free markets, we wouldn’t even know if market predictions had been manipulated.  If actual public policy were to depend on false readings from such markets, the potential for significant misallocation of resources is immense.  It is simply too great a risk to consider at this time, in my opinion.

Recently, McKinsey released an interactive summary of their Global Survey of Enterprise 2.0 applications.  About 2,000 companies take part in the survey each of the last three years.  One of the categories surveyed is Prediction Markets.  Apparently, in 2007 the level of adoption among the responders was less than 1%.  In 2008, the adoption rate jumped to 9% and slipped slightly to 8% in 2009.  A pretty remarkable achievement, don’t you think?

Let me ask you, based on published reports over the last two or three years, does it seem that prediction market adoption has jumped by this much?  Certainly, I don’t see it.  Maybe there are other reasons for the results. 

1. The survey sample may not have been representative of the population in any of the years. 
2. The definition of “adoption” may include very limited trials and pilot projects involving prediction markets. 
3. The definition of “prediction markets” may include some collective intelligence applications that aren’t really true prediction markets.

I happen to think that the third reason is the most likely culprit.  Most, if not all, of the prediction market software vendors include an idea pageant type of “prediction market” in their offerings.  I’m willing to bet that McKinsey includes these types of markets in the definition of prediction markets.  I’d also be willing to bet that these are the types of markets that are growing in adoption over the last couple of years.

Idea Pageant Growth

Relative to true prediction markets (I’ll get into the distinction below), idea pageant markets are a pretty easy sell to senior management.  There is very little downside, if any, to trying them out.  There are no political “feathers” to ruffle in the process.  There is tremendous upside potential in identifying previously undisclosed new ideas.  Senior management doesn’t have to rely on the crowd to filter out the weaker ideas (but it should).  Essentially, it is a high tech electronic suggestion box with a built-in feasibility filter.  What’s not to like?

But is it a Prediction Market?

Typically, idea pageants are set up to solicit new ideas from the participants (usually employees).  The same individuals place investments on the ideas they think are most likely to be adopted or receive funding.  This investment aspect is carried out in a market that resembles a prediction market.  It also looks a lot like a horse race market (without actually running the race).

In a true prediction market, the participants make investments in shares that represent potential outcomes.  In effect, the shares are derivatives of the actual future outcomes.  When the outcome is revealed, the share that represents the actual outcome is paid off and all other shares are worthless on expiry.  Note that the outcome is determined or occurs independently of the results of the prediction market.  The market attempts to predict the outcome, it does not determine the outcome.  The “horse race” must be run to determine the outcome.

In an idea pageant (think of a beauty pageant for ideas), the mechanism is similar to that of a prediction market.  Participants place investments on binary share contracts.  If a company is trying to find the best idea to pursue, the idea pageant becomes a poll of the opinions of the participants.  It is a weighted poll, because those who are more adept at guessing the ideas with the best potential will have more wealth to invest (vote).  The market determines the idea with the best chance of success, as determined by the weighted “votes” of the participants.  A true prediction market predicts a future outcome, which is determined independently of the operation of the prediction market.  In an idea market, the future outcome is the “prediction” of the market.

Alternatively, some idea pageants are set up to “predict” the idea that will be pursued (or receive venture capital).  This means that someone (or a panel) will make a decision about which potential outcome will be “true”.  In this case, the market is really being asked to predict the idea that the judge will select, not the idea that is ”best”.  There is a big difference. We are constantly finding examples of prediction market failures in these types of markets.  Olympic site selection, Nobel Prize in Economics, etc…

Conclusion

Oddly enough, I think there is a place for idea pageants in the corporate world.  I just don’t think there’s a place for them in the definition of a prediction market.  If we were to remove all of the idea pageants in the McKinsey survey, I’m willing to bet that the true prediction market adoption rate is still around 1%.

As a followup to my previous post, this one covers Public prediction markets.  Up front, I have to admit that my interest in public prediction markets is minimal, mainly because I see very little potential for these types of markets to improve decision-making (public or private).  If they are unable to do this, what good are they?  I started writing this post in May, just after I completed my post on the future of enterprise prediction markets.  Instead of completing this post, I published posts on noteworthy failures of public prediction markets and about market calibration.

Recently, Chris Masse, on his Midas Oracle site, documented the very public failures of prediction markets to forecast the IOC’s eventual decision to hold the 2016 Olympics in Rio and to forecast the winner of the Nobel prize in Economics (or any of the other Nobel prizes, for that matter).  I made several comments on Midas Oracle about these failed markets, and the process has renewed my interest (ever so slightly) in public prediction markets.  Here is the result.

Is there a future for Public Prediction Markets?

Bet on it.  In fact, you may have to.  Exchanges, such as Betfair and InTrade, may be the only sustainable, profitable applications of prediction markets that are available to the public.  Let’s face it, people love to bet on uncertain outcomes.  Even when the odds are against them, people will try to beat the house.  In casinos, the odds are always against the bettor, yet there is no shortage of gamblers and the casinos become glitzier each year.  It’s no mystery where the money is coming from.

Internet-based prediction markets offer the public the convenience of betting at home.  They have the potential to greatly expand the variety and types of things on which wagers may be placed, from political races to trivial events, such as who might win the latest “star search” or who is the best dancer”.   By adding to the variety of betting options, it expands the potential market for bettors.

Take away the real money component, and these prediction markets become nothing more than trivial pursuits.  Hubdub is a good example of a play money marketplace.  While it appears to be well-run, its use for anything other than “entertainment” is questionable.  Eventually, public prediction markets like these will fade away as newer fads invade the consciousness of the play money, esteem-seeking, public bettors.

There is some potential for real (serious) money prediction markets that might provide investors with a hedging mechanism against future events for which there may not be any form of insurance.  For example, a company could hedge against the risk of a particular piece of legislation becoming law (and having adverse effects on the company).

While there is a glimmer of hope that the U.S. anti-gambling laws may be relaxed in the future to allow real money prediction markets, the amounts that may be wagered are likely to be too small to attract any investors who wish to hedge against an uncertain event.  The betting limits will, however, provide a sufficient opening to allow betting exchanges to reach a vast new market in predictions.

Is there any real value in Public Prediction Markets?

Since public prediction markets operate in the same manner as enterprise markets, we can learn more about how these markets work and what makes them work well, by analysing the much more prevalent public prediction markets.  We can learn which types of markets tend to work well and which do not.  This may be useful in identifying appropriate uses for Enterprise Prediction Markets.  We could test public prediction markets to determine their consistency (or lack thereof).  We could make incremental changes to the markets to assess the effects on accuracy, consistency and the potential length of forecasting ability.

We could learn much about the role of information completeness by monitoring the information sets of market participants and comparing markets with similar participants but having differing information sets.  This may lead to insights about using prediction markets to replace some of the costly components of enterprise forecasting processes.  For example, if a public prediction market is able to more accurately (and consistently) forecast key components of an enterprise’s annual budget than the internal corporate methods, it may be possible to improve the efficiency of the planning process.  There may be additional benefits from engaging the enterprise’s customer base in the decision-making process, too. 

Apart from the knowledge gained from operating public prediction markets, one is hard pressed to find any significant benefit of these markets.  Do they help allocate resources to their best uses?  This may be a possible benefit, if the results of certain prediction markets are used to help shape public policy.  But, prediction markets are unproven in their abilities to consistently and accurately forecast or predict future outcomes and events.  Until they overcome these substantial limitations, their use for anything other than trivial pursuits will be rare.

OMB yesterday released its latest guidance on federal acquisition.  This new guidance, Increasing Competition and Structuring Contracts for the Best Results, provides guidelines for agencies to move to more competitive and lower risk types of contracts, along with semi-annual reports on progress.

But it is worth standing back and looking at some of the one-the-ground trends that are evolving in the acquisition community.  A very good article by Federal Computer Week’s Matthew Wiegart, “2.0 Takes Hold in the Acquisition Community,” provides just such a perspective.

In his article, he describes several examples of what is being called “Acquisition 2.0.” 

The first is the creation of self-organizing discussion groups.  The best known is the on-line forum hosted by GovLoop.  The informal champion is GSA’s assistant commissioner Mary Davie. This forum discusses trends in hiring acquisition professionals, ways of responding to bid protests, and ways to meet government goals for “greening” procurement.

The second is the Better Buy Project. Also championed by Davie, this project is a discussion group co-sponsored by several organizations with the goal of identifying ways of improving the first phase of the acquisition process, known as the “pre-award phase.”  Here, the goal is to draw on the broader acquisition community, both inside and outside the government, to identify best practices and innovative ideas.

The third is the use of on-line training and virtual mentoring. Agencies are increasing their hiring of acquisition specialists (OMB’s goal is a 5 percent increase by 2014) and the demand for training and mentoring is increasing as well.  Agencies are beginning to use on-line training and videos of retiring acquisition experts to bring new staff up to speed.  This follows a similar pattern in commercial industry.

In addition, there seems to be a fourth trend – trying innovative ways of doing things.  A great example is offered by former administrator of the governmentwide Office of Federal Procurement Policy (OFPP), Steve Kelman.  He describes how some researchers are piloting the use of prediction markets to see if they can improve cost and schedule forecasts in federal acquisition programs.  This is reminiscent of how the District of Columbia managed its portfolio of technology projects using a shadow “stock market” of these projects.

However, even with these examples of new openness and innovation, another former administrator of OFPP, Dee Lee, says that, more broadly, acquisition employees are scared and in a defensive crouch because of the general climate of distrust and criticism from Congress, watchdog groups, and the inspectors general.  She says that the administration has to provide leadership if it wants to overcome this climate and expand the use of Acquisition 2.0 approaches.  Maybe this new champion will be the recently nominated head of OFPP, Dan Gordon.  Kelman seems to think so, based on an effusive blog he wrote about Gordon’s nomination!

Recently, there have been several very glaring public prediction market failures, including the IOC site selection and the Economics Nobel Prize markets.  Some followers of prediction markets are a bit shocked and concerned, but most, like Chris Masse (Midas Oracle), me, and others are not.  These particular types of prediction markets never had a chance to be accurate.  Had any of these markets actually managed to “pick” the right outcome, it would have been nothing more than a fluke.  Why we continue to waste our time on these types of markets, I’ll never understand.

Jed Christiansen (Mercury’s Blog) is an occasional commenter on Midas Oracle.  I may not always agree with him, but I respect his positions in a number of areas.  However, in response to these very public failures of prediction markets, Jed provided a number of factors that influence the accuracy of prediction markets.  It appears that his comments apply only to outcomes that are determined by a group.  Essentially, he means outcomes that are determined through some form of voting or polling, including elections, IOC site selection, Academy Awards, Nobel Prizes, etc…  While I applaud his efforts to identify the factors affecting prediction market accuracy, I find some of his comments confusing.

For example, Jed mentions that “more members/voters will be better than fewer” (in terms of improving the accuracy of prediction markets).  In these types of markets, the members/voters are determining the actual outcome.  This is entirely independent of a prediction market attempting to predict that same outcome.  Consequently, having more members involved in determining the actual outcome will have no effect, whatsoever, on the accuracy of any related prediction market.  Jed’s comment makes no sense.

Jed is absolutely correct to say that “more objective criteria will be better than less.”  However, all this means is that the more objective the determinants of the outcome, the more likely the market participants will be able to figure them out and predict the outcome.  The fewer the factors and the less uncertainty surrounding their roles in determining the outcome, the easier it will be to predict the actual outcome.  In the extreme, if a condition arises that determines (or causes) the future outcome with a high degree of certainty, the market will be able to predict with uncanny precision.  However,  if the outcome is this easily predicted, perhaps a simple decision model (If… Then…) would have provided the same “prediction”, without the bother of setting up a prediction market.

Generally, I would agree with Jed that “constrained choices will be better than unconstrained choices.”  In keeping with this statement, the fewer the choices, the more likely it is that the outcome will be predictable (only because there are fewer incorrect options)!  However, the IOC markets showed that, even with only four choices, the markets failed.  The real problem is that these markets did not have the necessary information to choose among even a very small number of alternatives.

Again, I agree with Jed that “voters signalling choices before a vote is better than if they don’t.”  Where the outcome is determined by a vote, any prior information about how some or all of the group intends to vote will be important information to be assessed by the market participants.  This merely supports the information completeness principle.  We see many examples of this type of information being accessed by participants (in the IOWA political prediction markets) where political polling influences the market prices.

Finally, Jed made a curious statement about “secretive and less secretive” committees that make decisions and that “neither will likely be as accurate as traditional open prediction markets.”  I have no idea what he means, here!  The committees (secretive or not) are the ones determining (creating) the actual outcome.  The committee has nothing to do with being “accurate” or predicting the outcome.  Traditional markets are expected to predict actual outcomes.  Jed is simply wrong to try and compare these two concepts!

Panos Ipeirotis asked if there is a more principled method of capturing the determinants of prediction market accuracy.  In response, I would suggest that we look to the first principles of prediction markets.  Perhaps the most important of which is that the market possess a sufficient degree of information completeness.  In the examples noted, the prediction market participants did not have an adequate level of information completeness to be able to arrive at accurate predictions, because the method of determining the outcome was far too complex and subjective, even when the choices were limited to four.

The only way, to provide the necessary information to the prediction market, in order for it to accurately determine the otucome, would have been to make all (or many) of the outcome voters (committee members) participants in the prediction market.  Of course, this would be a needless redundancy.  Note that in most of the enterprise prediction markets, many of the participants also take part in the internal forecasting process, effectively including the body of corporate information in the prediction markets.  If internal forecasting processes were to be replaced by prediction markets, it is highly doubtful that the markets would be able to provide accurate predictions.  The required information to make those accurate predictions would be missing.

These types of markets suffer from a fatal flaw, as well.  They are trying to predict a discrete (non-continuous variable) outcome.  “Coming close” means being completely wrong.  These types of markets are only suitable for betting purposes, and even then, only if they are proven to be “well-calibrated”.  It is questionable whether these particular markets were well-calibrated.

I have written fairly extensively on the determinants of prediction market usefulness.  I am especially concerned with their accuracy and consistency, for without these, their use in decision-making is not warranted.  I draw your attention to the following posts:

The Forgotten Principle Behind Prediction Markets

Calibration = Prediction Market Accuracy?

To answer Panos, we do have a general, principled model for assessing prediction market accuracy.  Now, we need to fill in the details.

As far as I can tell, no one has actually accepted the Nobel Peace Prize while they were prosecuting a war, much less two.  Lê Ðức Thọ was offered one while involved in the invasion of / civil war with South Vietnam, but refused to accept the award.

I see three logical possibilities for this year’s prize.   Obama could accept the award while dodging accusations of hypocrisy.  Or he could pull a Lê Ðức Thọ/ Jean-Paul Sartre and refuse it.

But the most intriguing possibility is that Obama could earn his award, and end the wars in Iraq and Afghanistan in time for the ceremony on Dec 10.  While he’s at it, he could sign a pact with the other nuclear powers to destroy all of the world’s weapons.  And veto any renewal of the PATRIOT ACT.

Don’t say it’s politically impossible, Obama has the power to everything except the arms treaty unilaterally, with approval from no one.  But if anyone out there believes he will really end the wars soon, all I ask is that you put your money where your mouth is.  If the wars will be over in two months, you could make a fortune shorting the stock of every publicly traded U.S. defense contractor.  Let me know how that goes!

In my quest to write hypothetical case studies on companies & organizations that need Prediction Markets, I’ve admittedly missed many easy opportunities. Just a quick mental scan of this summer’s headlines produced several ripe candidates: The Yahoo!-Microsoft Deal, post-bankruptcy General Motors, the “closing” of Guantanamo Bay, the Cash for Clunkers program, etc. All are business or political endeavors whose fate is (or was) unknown, but whose future could have been predicted, potentially, by the proverbial crowd. If only the crowd had a Prediction Market, that is.

A Prediction Market is a crowd-sourcing tool that organizations can use for improved decision intelligence. Employees play the market … executives get better information. These internal markets can be (and are being) used for forecasting, new product development, capital investments, and increasingly, project management.

But instead of jumping on these summer headlines and blogging up a storm, I got caught up with my own project management woes. Too many things to do, not enough time, yadda yadda.

In my case, there’s no real cost to delaying my next blog post. But in the case of major initiatives in Corporate America or our government, there is plenty at stake. The costs of missed deadlines, inaccurate sales forecasts, budget shortfalls, marketing flops, rejected legislation, or failed mergers are significant, and scary. Which is why the relatively simple and inexpensive prediction market solution is so compelling.

As I’ve written before, Prediction Markets can’t solve everything. But they can provide information that decision makers just can’t get anywhere else. They can ask Yahoo! and MSFT employees for individual prognoses on the partnership, then aggregate it into actionable data. They can ask GM dealers across the country how many customers will return cars under the 60-day guarantee, ensuring the benefits exceed the costs. They can ask government officials from disparate branches the likelihood of Gitmo closing on schedule, uncovering hurdles and loopholes.

When the answers to these big questions can’t be found among the usual experts or with the usual tools … wise companies are beginning to trust the wisdom of the crowd. They’re experimenting with prediction markets, wiki’s, open innovation models, and the like. They’re trusting the insights and ideas of their various stakeholders, not just their executives.

But many companies aren’t. Many companies look at Prediction Markets and similar cutting-edge tools as unnecessary costs without guaranteed returns. This sounds interesting, but what’s the ROI? That, the quintessential business question, can’t be shunned. But it only takes a brief contemplation of the costs associated with big business blunders to entertain a new form of the question:

What is the cost of not doing?

Michael Giberson

The IOC recently selected Rio de Janerio over three competing bids to host the 2016 summer Olympic games.  The Chicago bid was favored in public prediction markets, with prices at Intrade between 50 and 60 at the time of decision and prices at Betfair implying about a 50 percent chance.  Did the prediction markets fail to predict well?

At Midas Oracle, Chris Masse has been asserting that prediction markets for IOC selections are fundamentally flawed, saying that the IOC is a small, secretive committee that doesn’t leak information and therefore no information is “out there” available to be aggregated by a prediction market. He was saying this before the IOC vote, too; this is not just after-the-fact speculation, it was his before-the-fact speculation. (Also posts here, here, here, here, here, here and, from April 2007, this post.)

I think the “small, secretive committee” explanation is weak, so I’ve been poking back a little in the comments. Chris, as is his style, has been elevating my comments into new posts in order to re-assert his views.

But a more fundamental question is whether or not it can be said that the prediction markets got it wrong.  At Sabernomics, J.C. Bradbury reports watching Intrade closely the morning of the IOC decision:

Around 9 AM … the odds show Chicago to be the favorite with a 53% chance of winning, closely followed by Rio at 46%, Tokyo at 3%, and Madrid at 2%. Like all the pundits following the selection were saying, it was a race between Chicago and Rio, but was very close to call. These odds also show something else, Chicago was trending down and Rio was trending up. The trend would continue for the next few hours.

… Looks like useful information was leaking out from knowledgeable parties just before the vote. This is evidence for, not against, the strong-form of efficient markets hypothesis.

Bradbury does an excellent job sifting through the shifting coalitions revealed in the three rounds of IOC voting.  Neither Madrid nor Toyko showed any significant ability to attract votes as the rounds proceeded.  It was going to be Rio or Chicago all along, but Chicago was weakest in the four-way vote and lost early, leaving the games to go to Brazil.

Based on Bradbury’s analyis, I’m convinced that the decision was pretty much a toss up between Chicago and Rio.  That conclusion was also implied in the prediction market prices just before the decision.  Sure, the prediction markets favored Chicago, slightly, over Rio; I don’t think you can call it a miss given the closeness of the decision.

[Related: Market Design and Marginal Revolution both have brief notes; Infectious Greed provides related discussion.]

UPDATE: Chris Masse doesn’t like my analysis: Who has the best analysis for Chicago’s failed bid for the Olympics?; neither does Paul Hewitt: “Michael Giberson is wrong to imply that the prediction was accurate on the basis that Chicago and Rio were fairly close.” See also here.

Two hours before the International Olympic Committee votes on the host of the 2016 Olympics, prediction markets are acting rather unusual. For those who are not familiar with prediction markets, they are websites that allow users to make small bets (most allow money bets, while others just use points) in order to predict the probability of an event happening. The biggest one in the US is www.intrade.com. These markets are based on the concept of the “wisdom of the crowds,” which means that large groups of people are more capable of gathering all the information available out there to make an educated guess about the future than individuals are.

For the past week I have been following Intrade.com, the Brazilian Mercadodeprevisoes.com.br and the British Betfair.com (which is more a betting site then a prediction market) in order to see how close their predictions were. Intrade had been constantly putting Chicago at a high 50s price (around 58% probability) and Rio at the low 30s. Betfair had been a little more generous towards Rio putting it on the mid 30s, but the difference between the markets was small. Mercado de Previsões, on the other hand, had considered it a toss up, practically 50/50 for most of the week.

However, this morning the markets have diverged big time. Intrade and Betfair have lost much of their confidence in Chicago. Rio has climbed to the mid 40s in Intrade (with Chicago in the low 50s) while Betfair is an has an even tighter spread with Chicago in the high 40s and Rio in the low to mid 40s. Mercado de Previsões is the really weird case though. Volume of trading is way up (as expected, that also happened in the other markets) but now the toss up has turned against Rio. It now predicts that Rio has a 20% chance of winning, therefore giving Chicago a whopping 80% chance.

This has just made me even more curious about the IOC’s decision. Since mercadodeprevisoes.com is a point based website and has such a huge difference in prediction, I decided to place a large hedged position there. I will let you know what happens. In the meant time, it will be very interesting to see how these markets are behaving in the last hours of trading.

A new study of prediction markets in the corporate world was released, recently.  It’s called Forecasting Consumer Products Using Prediction Markets, by Kai Trepte and Rajaram Narayanaswamy.  Lo and behold, the prediction markets failed to provide any significant improvement in accuracy over that of the traditional corporate forecasting process.  The authors submitted their paper as part of their masters program requirements.  They don’t appear to have been beholden to any software vendor, though they did use the services of Consensus Point.  Today’s entry will focus on the accuracy and usefulness of the prediction markets that were part of the study.  A subsequent entry will cover other aspects of prediction markets that were discussed by the authors.

The good news is that the authors planned the operation of the markets well, and they used more participants than most studies we have seen.  There appears to have been a conscious effort to maximize the diversity of the participants, but, like most of these studies, many of the prediction market participants also had involvement in the corporate forecasting process.  Consequently, we could pretty much expect that the predictions would be fairly well correlated with the corporate forecasts, and they were.  So, how did they compare? 

The prediction markets weren’t failures, but they weren’t able to do any better than the established corporate forecasting process at General Mills, where 20 prediction markets were put in play.  Despite the efforts of many academics, researchers, vendors and corporations,  the breakthrough success story about enterprise prediction markets remains as elusive as ever. 

FINDINGS & COMMENTARY

Correlation of Predictions and Forecasts

The Mean Absolute Percentage Error (MAPE) of the prediction market and the operations forecast (internal process) were highly correlated.  As mentioned above, this is not surprising, given that many of those involved with the internal forecasting process were also involved with the prediction markets.  Furthermore, the initial probability distribution for the potential outcomes were based on normal distributions around the internally forecasted mean.  That is, the starting point for the prediction market was the corporate forecast.  There were good reasons for doing this, but still, it may have introduced some bias toward the internal forecast.

The authors of the study found that the prediction market forecasts were virtually identical to those of the internal operations forecasting process, as evidenced by their means falling within one standard deviation of each other.  Consequently, we could say that both processes/methods were good aggregators of available information, and any information that was generated internally was also available to the market participants. 

Some Predictions are Better Than Others

The authors included three types of markets:  Volume, Product Category and Promotional markets.  The Volumemarkets were characterized by products that might be considered staples, with fairly stable consumption patterns.  Internal forecasts and market predictions were both able to accurately gauge the future outcome.  Product Category markets were a bit more difficult to predict or forecast, due to the nature of the products and strategies used.  Finally, the Promotional markets, which were characterized by products that had very significant promotions planned, were the most difficult to forecast.  Not even the corporate marketing people were very good at forecasting the effectiveness of the promotional activities.  Again, both the internal forecasts and the market predictions were even less accurate, but still they were basically the same.

It appears that, if it is difficult to analyse data to come up with an accurate forecast, as was the case with the promotional markets, the use of a prediction market will not magically generate the information necessary to make a better prediction.  We have seen this in other studies and examples, where there is a significant amount of uncertainty about the outcome.  This is the information completeness principle that I’ve discussed previously.

Very Short Term Markets

I should note that the prediction markets were in operation for no longer than 10 weeks.  The authors described some of their prediction markets as being “long term”, but in reality, there were anything but.  In our quest for a useful enterprise prediction market, it must be able to generate consistently accurate predictions, sufficiently in advance, so that decision-makers are able to change their tactics, based on the predictions.  In the study’s “longer term” markets, none were able to generate accurate predictions until very near the time when the actual outcome would have been set.  In these cases, management would not have had time to change their tactics or decisions, once the market prediction had become known.  Therefore, even if the prediction had been perfectly accurate, it is completely useless for any decision-making purposes.

Costs vs. Benefits

The authors did not discuss the issue of costs and benefits of prediction markets, but perhaps we should.  Given that both the traditional forecasting process and the prediction markets provided equivalent forecasts, should General Mills’ management scrap their costly forecasting process and adopt these neat new tools?  We can’t know for sure, right now, but if they were to discontinue the internal forecasting process, most of the useful information that needs to be aggregated in the prediction markets would not have been available to the participants.  Accordingly, we would expect the predictions to become very inaccurate. 

It would appear that the accuracy of the prediction markets depends upon the information created by the forecasting process.  If you can’t have prediction markets without the internal forecasting, why would General Mills add prediction markets to the process?  One reason might be to verify the accuracy of the internal forecast, but I’ll bet they already know that, historically, their forecasts are reasonably accurate for their decision-making purposes.  They might consider eliminating the internal aggregation function, while continuing to generate forecasting information.  Prediction markets would be relied upon to perform the aggregation of the information more efficiently.  Finally, prediction markets generate distributions of possible outcomes along with the mean prediction or forecast.  This information can be used to assess the risk and uncertainty surrounding the forecast, enabling management to make better contingency plans.

Filtering Bias

One of the benefits of prediction markets is their ability to filter out bias during the aggregation process.  Consequently, I (and the authors) expected the prediction markets to provide significantly more accurate forecasts than those generated from the internal forecasting process.  The fact that they were not more accurate means, to me, that General Mills’ internal forecasting process performs its function in a reasonably unbiased fashion.  We should be studying why they have been able to minimize bias in their planning!   Another possibility, which I find too scary to contemplate, is that prediction markets aren’t as good at filtering out the bias as we have been led to believe!

Calculating Accuracy

The authors don’t discuss the method of calculating the forecast or prediction error, other than to note that General Mills uses the MAPE (see above) to calculate their own internal forecast errors.  I have a couple of issues with this approach (which was also used in the HP study).  Using the absolute value of the error provides only the magnitude and no information about whether the prediction was an over or under-estimation.  Accordingly, the actual error could be as much as twice the amount of the absolute error quoted.  Also, the authors (and others) use the actual outcome as the denominator in the calculation of the average.  This is incorrect, because it is the forecast (or prediction) value that is being evaluated, rather than the actual outcome.  Management relies upon the prediction in order to make decisions.  They don’t rely on the actual outcome (which isn’t known), when they are making decisions.  Accordingly, the prediction value should be used in the denominator and not the actual outcome.

My next blog entry will cover the authors’ comments about the operation of these prediction markets and how well they appear to aggregate available information.

The Journal’s Real Time Economics blog reports that the Philadelphia Fed has released the results of a survey it conducted of professional economic forecasters–and that 75% now expect the United States to emerge from recession in Q3 2009 (up from 54.5% three months ago). Well I checked in over at Intrade, and it turns out the general public has seen an even more dramatic swing of faith in the U.S. economy’s prospects for the quarter. Whereas three months ago, Intrade’s contract priced in a 40% chance of positive U.S. growth in Q3, now the market is pricing in a 90% chance. See the chart below.

Michael Giberson

Miriam Cherry and Robert Rogers explore the interaction of “Prediction Markets and the First Amendment.”  If prediction markets are “expressive,” does that mean that U.S. government actions that constrain prediction market development potentially raise First Amendment issues on free speech grounds? The authors propose a way forward in which courts, at least until the legislative and administrative branches clarify policy, apply a Miller v. California-like test for identifying permissible prediction markets.

Miller v. California established a three-part test for regulating obscenity, and it is the third part of that test is key for Cherry and Rogers: “whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.”  On these grounds Cherry and Rogers suggest the first amendment may offer protection for prediction markets directed at political, economic, cultural, or scientific topics, but perhaps not for prediction markets for sporting or entertainment events.

I’m not sure First Amendment law is the best protection for prediction markets, but it might be better than nothing.

HT to Chris Masse at Midas Oracle.

Intrade’s prediction market, which a month ago put Ben Bernanke’s probability of holding onto his Fed chairmanship in 2010 at less than 50%, now puts it over 80%. See chart below:

Intrade’s market guesses that the most likely candidates to replace Bernanke, were he not to keep his post, are Janet Yellen and Larry Summers.

I’m willing to bet a small sum (say $50) that the next version (to be released in 2010) of the popular soccer videogame FIFA will not include jerseys bearing the logo of American International Group (AIG).

Edit: Insurance company AON buys logo space on Manchester United jerseys to replace AIG.

I really like the story-telling genre of books, typified by Malcolm Gladwell (Blink, Tipping Point, Outliers). Sway (Ori Brafman & Rom Brafman) is another book in the same category, and a good one. The stories are helpful to bring the insights to life, and the insights are powerful to bring life… to life.

I wanted to pull out some key points for myself, and then I thought I would share them and some of my additional insights. It’s rationale to share. So my key take-aways:

Behaviors and decision making are influenced by an array of irrational, psychological forces. And like streams, these various forces combine and become very powerful.

Key ‘hidden’ forces are:

-       Loss Aversion – tendancy to go to great lengths to avoid possible loss

• also has notion of commitment – once started down a path, one feels obligated to continue, despite evidence

-       Value Attribution – inclination to ascribe to a person or thing certain qualities based on initial perceived value, rather than objective data

• this can be helpful to us as a mental shortcut to determine what is worthy of attention, but in some cases, very misleading
• value attribution can be arbitrary, with no link to any actual value of any sort

-       Diagnosis Bias – propensity to label people, ideas or things based on our initial opinions, blindness to evidence that contradicts our initial assessment of a person or situation

-       concept of fairness in terms of irrationality is mostly driven by the process rather than the outcome

• cultural bias of fairness is interesting (Americans & Russians think differently!)

-       Money & compensation use a special part of the brain… that is the opposite of the altruistic brain

• The nucleus accumbens enjoys the thrill (first date, gambling), loves highs, and deals with the notion of gains and losses for money (the ‘pleasure center’
• The posterior superior temporal sulcus looks after social interactions, how we perceive and relate with others (the ‘altruistic center’)
• The key driver is the anticipation of reward in driving addictive behavior and surpressing altruism

-       Group dynamics form an important part of irrational behavior, particularly allowing and handling dissent

• Four types of person: initiator (starts things, excites people), blocker (questions things, critic), supporter (takes a side), and observer (neutral)

Some implications for Innovation and Idea Management:

1)    Danger of Financial Rewards & Likely Reasons for Practical Failures of Prediction Markets

As we know, financial rewards are very damaging to Idea Management and innovation initiatives. Most people contribute out of a desire to help and engage with others. As soon as you add a financial component to the process, people’s brains think differently.

Sometimes it is better to get nothing than something

This is especially the case in the emergent area of prediction markets, typified by Spigit and NewsFutures. Individuals are encouraged to spend their time helping others (the ‘…sulcus’ part of the brain) by reading through ideas and making judgments. The ‘market’ component of the Prediction Market then triggers the more powerful ‘pleasure center’ part of the brain, and then surpresses one’s desire to help others. This is made even more damaging when the company highlights the potential benefits in scoring or spending time doing evaluations. Why spend two hours’ worth of time for someone else’s benefit… and all I get is a lousy T-shirt?

2)    Personality Styles

The work on dissent matches closely to our work on the Orchid Model and the instrument we use to identify personality styles regarding handling new things. One can intuitively see the connection between:

-       initiator = creator

-       blocker = inquisitor

-       supporter = helper

-       observer = doer (stays out of the issue, just wants to know what to do)

I like the work on irrational behavior a lot – it makes a lot more sense than the hopeful expectation that people are mostly rational. I do believe that even this body of work is missing in a trick in not encompassing the work on complex systems, emergent behavior, and particularly the importance of feedback loops. For instance, the heuristic ‘rule’ of “follow what your peers are doing” would explain why one person could do an irrational thing, and then other people follow. It does not immediately imply that everyone is irrational, more than the feedback loop is taking hold.

Sway is a great book and I recommend you take a look, whether rationally (it’s good for you) or irrationally ( I ask you to .

I love prediction markets, not because I think they are accurate but because I’m so often surprised at what they reveal about how people are interpreting the news.  One indicator I’ve been tracking on Intrade is the market predicting whom the GOP candidate will be in 2012. Today I noticed that Sarah Palin’s “stock” has been behaving interestingly since her resignation earlier this month. See the chart, below:

She announced her resignation on July 3. At that point, Intrade’s market put the percentage probability of Palin becoming the 2012 GOP presidential nominee at 13%. The news prompted a huge selloff, on high volume, halving that percentage to 6.5% in one day. But since, her stock has steadily risen, and the market now puts her chances of emerging as the nominee at 16%–higher than before she announced her plans to step down.

The stock market rebounded nicely this week due in part to several banks beating analysts’ 2Q earnings estimates. However, the financial sector, and the overall market haven’t completely escaped the recession wreckage. If you are sick of betting on the same companies that broke your heart this past year and your searching for an alternative and more entertaining way to lose your money, prediction markets may qualify. Sites such as Intrade and IEM allow you to bet on the outcome of future events. For example, you can bet on Sarah Palin being the 2012 presidential nominee, or you can bet on Harry Potter and the Half Blood Prince grossing over $90M in its opening weekend. There’s something for everyone. Even if you lose money, so long as its not too much money, prediction markets are a relatively affordable and entertaining coversation piece. However, prediction markets are more than unsophisticated gambling platforms. In fact, prediction markets highlight the difference between gambling and investing albeit a difference that is vague and difficult to discern. Peter Lynch has said that “An investment is simply a gamble in which you’ve managed to tilt the odds in your favor.” I would argue that the difference between investing and gambling is much more than tilting odds in your favor. The following is one of my favorite definitions of investing:

Any activity in which money is put at risk for the purpose of making a profit, and which is characterized by some or most of the following (in approximately descending order of importance): sufficient research has been conducted; the odds are favorable; the behavior is risk-averse; a systematic approach is being taken; emotions such as greed and fear play no role; the activity is ongoing and done as part of a long-term plan; the activity is not motivated solely by entertainment or compulsion; ownership of something tangible is involved; a net positive economic effect results.

This definition emphasizes the important elements that one can use to distinguish oneself from an ordinary gambler, and thus maximize returns. These tactics should be used when using prediction markets, the stock market, or any other form of markets (unless, of course, the stakes are low and your betting simply for the thrill). When used incorrectly Intrade and other prediction markets are a recession substitute for your annual trip to Las Vegas. When used correcly they may just pay for your vacation to Las Vegas. For example, if you can find similar items trading on different prediction market websites at different prices you can use arbitrage strategies to take advantage of the frequent mispricing that occurs in these inefficient markets. Of course, these arbitrage opportunities are few and far betweeen. However, during the recent presidential election, I made some money using prediction markets. For a while, Barack Obama was trading at favorable odds on Intrade.com while John McCain was trading at favorable odds on the University of Iowa Electronic Prediction Markets. By taking opposites positions on the 2009 presidential election you could have made a few dollars. Investing and gambling are not the same thing. Invest, don’t gamble.

One side note: Online betting in the U.S. is illegal. However, some sites escape government restrictions. Notable exceptions are Intrade, which avoids U.S. legal restrictions by operating from Dublin, Ireland, where gambling is legal and regulated, and the Iowa Electronic Markets, which operates from the University of Iowa under the cover of a no-action letter from the Commodity Futures Trading Commission and allows bets up to $500.

I pointed out a few weeks back that the USDCAD was forming a weekly flag.  The price action over the last few days seems to show this as being accurate.  It hasn’t confirmed yet, but the USDCAD has had a very large drop.

The AUDUSD looks like a flag as well on a weekly chart.  We will see how this executes over the next month, but given that our fundamental indicators show these currencies should dramatically move against the dollar, we are looking for them to be fullfilled.

www.generatefx.com

Equity markets prefer Republicans to Democrats. This was shown convincingly in “Partisan Impacts on the Economy: Evidence from Prediction Markets and Close Elections” by Snowberg, Wolfers and Zitzewitz. The paper uses the prediction market price of a contract for a Republican win on election day and estimates the price’s effect on equities. Estimates from the paper suggest that markets value a Republican victory 2 to 3 percent more than a Democrat victory. Even though the methodology of the paper is not bulletproof there is plenty of other evidence in support of this finding.

That said, it remains a puzzle as to why markets do this. A simple historical comparison of returns suggests that equities actually do better under Democrat presidents. The above paper provides three possible explanations for this fact.

1. Past Democratic presidents have pursued policies that are better for equity markets but investors haven’t noticed.
2. Past Democratic presidents have pursued better policies but investors don’t believe they will in the future.
3. The variance of equity returns trumps any simple comparison of partisan effects.

To this, I will add a fourth possibility. Republicans are better for equities but due to the nature of the business cycle Democrats get elected at market lows and thus experience greater equity returns during their tenure. If Republicans were in power during that time, then returns would be even higher.

I think that these possibilities miss the true implication of the market’s preference for Republicans. There is nothing close to conclusive evidence on which party is better for markets. The more relevant fact is that most traders and financiers have historically been Republicans. As a part of their ideology, they believe that Republicans are better for the economy. It is then ideology, rather than rational expectations, that causes markets to prefer Republicans. This ideology like many other beliefs, does not get selected out of the market. National elections happen only once every four years so traders who have false beliefs about the effects of elections do not face  a big penalty for being wrong.

The product

Firstly, this is totally incomplete. We’re not even close to polished idea of the product. There are, however, several things we know for sure. As most of the things created today, it will be a synthesis of several things. We are not about to invent new technologies, just new application of existing ones. Basically we want to give our users economic reality to play with.

Of the games, namely MMOGs, we want our product to have a need to think strategically and be demanding intellectually. Want a shooter? Go someplace else.

The simulation of the process, the excitement, limitless possibilities – in other words, not being constrained by reality. Its fun, it’s cool. Tick.

Competitive environment and sense of achievement will also be there, as well as the ability to gain respect by others.

The main difference? In all strategic games you are not the boss. You are GOD. Allocation of resources, military decisions, building of cities – you consult no one, you report to no one. You get fired by no one after screw-ups. This is not how it works in real life. The tools for collaboration needed for decision making, as well as some sort of hierarchy will be there.

Of social networks we want to give users a platform for a community. Users will be known by their real names. If you’re good – don’t hide yourself behind a nickname. If you suck – get lost. This will not be one of the “upload your photo, share your music interests and buy your avatar a pair of pants” type of thing, though. There are already many of those. Far too many to catch attention of the masses and people that are still developing new one are lunatics. Period.

Elements of prediction markets will also be included. Despite the failure of early entrants, this area still looks promising. We are especially eager to extract the wisdom of the crowds, as well as put it in action.

The thing we build will be designed for young professionals or Gen Y as they are called. We’ll expand on our customer in the upcoming posts.

The more time you spend on a website, the better. For the owner (=more ads) that is. Not true for our thing, though. Quality of decisions taken and not time spent will be most important. Addicts are not welcome here.

The thing will be playable on any computer in the world connected to the internet. These days folks either do want to have ultra-cool hardware with 20 processors or they don’t. There’s no middle path anymore. We cater for the later.

So is it clear now what sort of thing we are building here?

No, because things mentioned above are disjointed at best. So why can’t we just say “we’re going to be X for Y”? Because neither X nor Y exists. If it does, we’re not interested anymore. So I guess we’ll struggle explaining it for the foreseeable future. There goes the elevator-speech…