The Good Judgment Project has roots in Phil Tetlock’s study of Expert Political Judgment, which may be best known for its conclusion that the “expert” forecasters he studied were often hard-pressed to do better than the proverbial dart-throwing chimp. This conclusion sometimes has been mis-interpreted as a nihilistic rejection of all forms of subject matter expertise. But, that’s not so. Indeed, our research team for the IARPA tournament relies on a wide range of subject matter experts, including political scientists who devote much of their time and research effort to improving geopolitical forecasts.
They are no ordinary pundits, though. Consider, for example, Jay Ulfelder, former research director of the Political Instability Task Force (PITF) who now blogs as the Dart-Throwing Chimp. Jay’s forecasting prowess was evident during Season 2 of the Good Judgment Project—he participated as a “regular forecaster” and would have qualified for “superforecaster” status in Season 3 had he not joined our research team. Jay’s performance far exceeded that achievable through random guessing.
Now, Jay faces the far more challenging task of helping to frame the forecasting questions for Season 3. With his assistance, we’ve made progress in developing questions that are both rigorous (with all terms defined specifically enough to allow unambiguous scoring of forecasts as right or wrong) and relevant (in that knowing the correct outcome would meaningfully inform policymakers’ decisions). But, much work remains to be done before we become as skilled at generating questions as the GJP forecasters have become at producing accurate answers to those questions.
Also new to our Season 3 research team is Mike Ward, whose research lab at Duke University “creates conflict predictions using Bayesian modeling and network analysis.” In 2012, Mike and Jay engaged in a spirited debate in Foreign Policy about whether there could be a “Nate Silver” in world politics (that is, a forecaster of geopolitical events with a track record comparable to Silver’s notable successes in forecasting US elections). Mike doesn’t claim to have achieved Silver status yet, but we expect that his sophisticated modeling techniques will add to the GJP’s statistical arsenal.
We’re also fortunate to have Phil Schrodt join our Season 3 research team. Like Jay, both Phil and Mike have been involved in prior government projects to develop accurate statistical forecasts of geopolitical events, in their cases through both the PITF and DARPA’s Integrated Crisis Early Warning Project (ICEWS). Phil’s expertise with “event data” (e.g., with GDELT, Global Data on Events, Location and Tone) complements GJP’s strength in crowd-sourced forecasts. We’re hoping that the whole will be greater than the sum of the parts. And, readers of this blog should hope that Phil and Jay, both bloggers extraordinaire, will find time to write one or two guest posts for us. Until then, you can sample Phil’s thoughts on political forecasting courtesy of slides for his recent lectures at the University of Konstanz (Germany) and at European Network for Conflict Research (ENCoRe), University of Essex (United Kingdom).
Jay, Mike and Phil round out a GJP political-scientist roster that already included Penn’s own Mike Horowitz, an expert on international conflict and security issues, and Rick Herrmann, Chair of Ohio State’s Political Science Department and former Director of the Mershon Center for International Security Studies. The breadth and depth of their experience has been invaluable in guiding GJP’s efforts to improve geopolitical forecasting and shed light on important debates in international relations.