Tuesday, February 23, 2021

Winter '20 / '21 - Forecaster Statistics

Lincoln Memorial Reflecting Pool
and Washington Monument (c.1930)

For each contest-worthy snow storm ... the forecasts are verified against the observations of storm-total snowfall.

Statistics are calculated to determine how well each forecast captured the magnitude and distribution of the storm's snowfall.

Individual forecaster statistics for the first four snowstorms here.

---
Statistics include:

The average normalized ‘SUMSQ error’ is the Contest/s primary measure of forecaster performance.

This metric measures how well the forecaster/s expected snowfall 'distribution and magnitude' for _all_ forecast stations captured the 'distribution and magnitude' of _all_ observed snowfall amounts.

A forecaster with a lower average SUMSQ Z Score has made more skillful forecasts than a forecaster with higher average SUMSQ Z Score.

---
The 'Storm Total Precipitation error’ (STP) statistic is the absolute arithmetic difference between a forecaster/s sum-total snowfall for all stations and the observed sum-total snowfall.  This metric … by itself …is not a meaningful measure of skill …but can provide additional insight of forecaster bias.
 
---
The 'Total Absolute error' (TAE) statistic is the average of your forecast errors regardless of whether you over-forecast or under-forecast.  This metric measures the magnitude of a forecast’s errors.
 
----
The 'Average Absolute Error' (AAE) is the forecaster/s ‘Total Absolute Error’ divided by the number of stations where snow was forecast or observed.
 
---
The ‘RSQ error’ (R-Squared - coefficient of determination) statistic is a measure of the how well the forecast captured the variability of the observed snowfall.

Combined with the SUMSQ error statistic … RSQ provides added information about how strong the forecaster/s ‘model’ performed.

No comments: