Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions
Philip Boeken, Onno Zoeter, Joris Mooij
In performative prediction, forecasts influence the outcomes they aim to predict, undermining the existence of correct forecasts and standard methods of elicitation and estimation. We show that conditioning forecasts on covariates that separate them from the outcome renders the target distribution forecast-invariant, restoring well-posedness of the forecasting problem. However, even under this condition, classical proper scoring rules fail to elicit correct forecasts. We prove a general impossibility result and identify two solutions: (i) in decision-theoretic settings, truthful and outcome-optimising scoring is possible if forecasts are separating; (ii) batch-scoring based on empirical divergence or inverse probability weights can yield correct forecasts. Applying these insights to parameter estimation, conditional forecasts and proper scoring rules enable performatively stable estimation of performatively correct parameters, resolving the issues raised by Perdomo et al. (2020). Our results expose fundamental limits of classical forecast evaluation and offer new tools for reliable and accurate forecasting in performative settings.