Forecast Verification

High-resolution numerical models forecast weather with great detail and we might find them useful because observed features are better reproduced. However, the value of these forecasts is difficult to prove using traditional gridpoint-based verification statistics. The classical ‘‘double penalty problem’’ illustrates the limitations of the gridpoint-based error measures: a forecast of a precipitation feature that is correct in terms of intensity, size, and timing, but incorrect concerning location, results in very poor categorical error scores (many misses and false alarms), and large root-mean-square errors. To address this problem, spatial verification techniques have been developed that do not require the forecasts to exactly match the observations at fine scales. One such measure represents the Displacement and Amplitude Score (DAS) belonging to the field verification methods.

Measuring distance between observed and forecast features is not only an intuitive error measure for many users, but also avoids the double-counting penalty where a feature displaced in space is scored worse than either a complete miss or a false alarm since it is penalized as both at once. The Displacement and Amplitude Score DAS is based on an optical flow algorithm that defines a vector field that deforms, or morphs, one image to match another. In DAS distance and amplitude errors are combined to produce a single measure. The optical flow method does not require identification and matching of discrete objects, which is often subjective and sensitive to many parameters in the algorithms.

Observed (top left), forecast (top right), forecast superimposed with displacement vector fiel matching the forecast onto the observation (bottom left), and morphed forecast (bottom right) The two components of DAS in forecast space comprise the mean displacement vector lengt and the root mean square error of morphed forecast and observation (bottom right – top left).

Keil, C. and G. C. Craig (2009): A displacement and amplitude score employing an optical flow technique. Weather and Forecasting, 24, 1297-1308.