November 2017: I am an Associate Investigator on a Royal Society of NZ Marsden Fund grant along with Thomas Pfeiffer (Massey University, PI), Arkadii Slinko (Auckland Mathematics, AI), Anna Dreber (Stockholm School of Economics, AI) and Yiling Chen (Harvard, AI).
Background: Imagine a researcher who has the choice between several experimental designs to investigate a proposed effect. To find the design that has the highest chance to succeed, the researcher would like to harness the expertise of the research community. Moreover, the researcher would like to provide incentives to those experts providing the most accurate advice. How can such incentivized and “crowd-sourced” decision-making be implemented? Decision markets have been put forward as promising mechanisms for such decision-making problems. To select among several mutually exclusive actions, first, forecasts about the expected future consequences of each action are elicited from the participating experts. This step works similar to forecasting in incentivized prediction markets. Second, a decision rule is used to select an action based on the forecasted consequences. Once the consequences of the selected action are observed, they are used to provide incentives for the forecasts elicited in the first step. Because only one among the possible actions can be taken, it is not possible to evaluate all the forecasts made in the first step. To nevertheless maintain proper incentives in the forecasting step, the decision rule is stochastic, and the payoffs are adjusted based on the probabilities used in the decision rule. Since proper decision markets have been described only recently, there is – despite a fascinating range of potential practical implementations – only very little empirical knowledge on this topic.