文章基本信息

标题：Using cognitive models to combine probability estimates
本地全文：下载
作者：Lee, Michael D. ; Danileiko, Irina
期刊名称：Judgment and Decision Making
印刷版ISSN：1930-2975
出版年度：2014
卷号：9
期号：3
页码：259-273
出版社：Society for Judgment and Decision Making
摘要：We demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people's estimates of probabilities for general knowledge questions such as ``What percentage of the world's population speaks English as a first language?'' The second experiment involves people's estimates of probabilities in football (soccer) games, such as ``What is the probability a team leading 1--0 at half time will win the game?'', with ground truths based on analysis of large corpus of games played in the past decade. In both experiments, we collect people's probability estimates, and develop a cognitive model of the estimation process, including assumptions about the calibration of probabilities and individual differences. We show that the cognitive model approach outperforms standard statistical aggregation methods like the mean and the median for both experiments and, unlike most previous related work, is able to make good predictions in a fully unsupervised setting. We also show that the parameters inferred as part of the cognitive modeling, involving calibration and expertise, provide useful measures of the cognitive characteristics of individuals. We argue that the cognitive approach has the advantage of aggregating over latent human knowledge rather than observed estimates, and emphasize that it can be applied in predictive settings where answers are not yet available.
关键词：probability estimation; wisdom of the crowd; cognitive modeling; graphical models; individual differences.