Juntao WANG

Juntao Wang is a fifth-year PhD student at the School of Engineering and Applied Science at Harvard University. He received his BS and MS degrees in computer science from Shanghai Jiao Tong University. His research interests lie in developing more practical algorithms and mechanisms that leverage the collective intelligence of humans to solve challenging tasks. His algorithm has been adopted by replicationmarkets.com, which generates forecasts for the replicability of open social science studies using collective intelligence.

ABSTRACT

Forecast Aggregation via Peer Prediction

Crowdsourcing enables the solicitation of forecasts on a variety of prediction tasks from distributed groups of people. How to aggregate the solicited forecasts, which may vary in quality, into an accurate final prediction remains a challenging yet critical problem. Studies have found that weighing expert forecasts more in aggregation can improve the accuracy of the aggregated prediction. However, this approach usually requires access to the historical performance data of the forecasters, which are often not available. In this talk, he will introduce our work on aggregating forecasts without using historical performance data. We propose using peer prediction methods, a family of mechanisms initially designed to truthfully elicit private information in the absence of ground truth verification, to assess the expertise of forecasters, and then using this assessment to improve forecast aggregation. Our aggregators significantly and consistently improve the aggregation accuracy w.r.t. a variety of existing aggregators over a diverse collection of human forecast datasets.