Ordinal relationships between measures of the “accuracy and “value” of probability forecasts: preliminary results

The preliminary results of an investigation of the relationships, on an ordinal scale, between particular measures of the “accuracy” (the validity measure) and “value” (the utility and expected-utility measures) of probability forecasts are described. For individual, comparable forecasts the validity measure represents an ordinal measure on utility and expected utility, i.e., the validity measure is able to order, or rank, forecasters and/or forecast procedures according to the utility or expected utility of their forecasts. Although forecasts, in general, are not often comparable, all forecasts are comparable in two-state, i.e., “weather–no weather”, situations and, in addition, these relationships can be expected to hold in most situations in which forecasts are nearly comparable. For collections of individually comparable forecasts similar relationships hold between average validity and average utility and expected utility in certain special situations. The nature of the relationships and scope of the results in both general and special situations are indicated by means of sample collections of forecasts.