Evaluating Prediction Mechanisms: A Profitability Test
Name
3643562.3672612.pdf
Size
1.97 MB
Format
Adobe PDF
Checksum (MD5)
792fd71e35764f5ab7d24f45b15115c0
Author(s)
Sethi, Rajiv
Seager, Julie
Cai, Emily
Benjamin, Daniel
Morstatter, Fred
Bobrownicki, Olivia
Cheng, Yuqi
Kumar, Anushka
Wanganoo, Anusha
Date Issued
June 27, 2024
Publisher
ACM|Collective Intelligence Conference
Citation
Sethi, Rajiv, Seager, Julie, Cai, Emily, Benjamin, Daniel, Morstatter, Fred et al. 2024. "Evaluating Prediction Mechanisms: A Profitability Test."
Version
Final published version
Abstract
Any forecasting model can be represented by a virtual trader in a prediction market, endowed with a budget, risk preferences, and beliefs inherited from the model. We propose and implement a profitability test for the evaluation of forecasting models based on this idea. The virtual trader enters a position and adjusts its portfolio over time in response to changes in the model forecast and market prices, and its profitability can be used as a measure of model accuracy. We implement this test using probabilistic forecasts for competitive states in the 2020 US presidential election and congressional elections in 2020 and 2022, using data from three sources: model-based forecasts published by The Economist and FiveThirtyEight, and prices from the PredictIt exchange. The proposed approach can be applied more generally to any forecasting activity as long as models and markets referencing the same events exist.
Description
CI ’24, June 27–28, 2024, Boston, MA, USA
MIT Department
Sloan School of Management
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3643562.3672612
