Validation of wind speed forecasts from the AROME model against mast observations

When short-range wind speed forecasts obtained with the AROME model were compared against wind speed observations made over open sea, the results were found to be quite good. For five representative stations, the mean error (bias) was +0.1 m/s. The comparison of model wind against Helsinki Testbed mast observations revealed that the bias (overestimations) was slightly larger. The mean error for observations between 60 and 100 metres was +0.2 - +0.3 m/s. The overestimation of wind speed seems to be greater near the coastline than over open sea stations or inland stations.

The difference between model wind and observed wind differed very much from station to station. For some Helsinki Testbed area stations, the model wind was on average too strong or too weak. When wind speeds were the highest, the model underestimated the speed. Mast observation did not reflect the low-level jet stream because only one measurement level was available.

When comparing observed wind speed and model wind, one has to remember that model wind represents the mean value in a grid box with a homogenous landscape that excludes all the details of the actual landscape itself. The measurements may also include errors.

How was the comparison made?

The validation period was from the beginning of July 2008 to the end of July 2009. The measurement height for sea and coastal areas was about 30 metres, for radio masts between 220 and 270 metres and for Helsinki Testbed masts between 60 and 100 metres. Comparison was also made between model wind and measurements from a mast over a hill in Lapland.

Station No 3 4 5 6 7 8
Hanko 1 73 2535 7.76 6.86 0.85 2.36
Vänö 4 80 2496 8.35 8.29 0.06 1.80
Bågaskär 5 83 1377 8.63 8.83 -0.25 2.22
Padva 7 79 1935 7.38 7.09 0.26 2.30
Degerö 8 81 2128 7.25 6.17 1.09 1.84
Suvisaaristo 10 61 1854 6.83 5.91 0.92 1.91
Pirttisaari 12 61 988 9.25 8.43 0.77 1.75
Roihuvuori 15 100 2030 7.01 6.39 0.61 2.01
Oulunkylä 16 92 1499 6.63 6.56 -0.01 2.86
Essvik 19 86 2089 6.91 6.34 0.56 1.90
Boxby 20 80 1047 6.34 5.65 0.70 1.79
Luhtaanmäki 23 61 2203 4.72 5.17 -0.47 1.54
Monnikylä 27 80 1481 5.41 5.70 -0.30 1.56
Viirilä 28 81 2125 5.59 5.75 -0.17 1.52
Isosaari 36 83 1905 7.60 7.49 0.11 1.94
Isosaari 36 62 1905 7.35 7.11 0.24 1.93
Isosaari 36 42 1905 7.10 6.68 0.42 1.92
Mean value           0.24  

3= height (m)
4= number of observations
5= AROME mean wind speed
6= observed mean wind speed
7= Mean error (ME)
8= Standard Deviation of Error (STDE)

The above table is a summary of the comparison of model winds and mast observations over the period 1 July 2008 - 31 July 2009. A boldface figure in the column for observations indicates that the number represents less than 50% of all possible observations and that the station was therefore excluded from calculation of the mean error. The boldface figure in the STDE column also indicates that the station was excluded from calculation of the mean value or error over stations.

Station 2 3 4 5 6 7
Helsinki lighthouse 32 3033 7.75 7.38 0.37 1.93
Kalbådagrund 32 3036 7.73 7.74 -0.01 1.86
Utö 15/31 3028 7.90 7.12 0.78 1.90
Hanko 27/40 3022 7.78 8.05 -0.27 2.00
Kemi I 26 3014 7.07 6.97 0.11 1.86

2= height (m)
3= number of observations
4= AROME mean wind speed E
5= observed mean wind speed
6= Mean error (ME)
7= Standard Deviation of Error (STDE)

The summary table showing the comparison of wind speed between FMI sea weather stations and the AROME model over the period 1 July 2008 - 31 July 2009.

The purpose of the validation was to assess how well the model wind speed at a grid point corresponds to the wind speed observed at the mast. The wind speed for a grid point at a certain level corresponds to a mean value of a 2.5 km x 2.5 km square. Furthermore, the model topography corresponds to a mean topography of the same area. On flat areas the difference is not great, whereas in hilly areas the difference can be substantial. The interpolation of wind speed from model levels to constant height levels may also introduce error.

All observations have also errors as well as the model forecasts and the structure of the model error has been studied. The observations obtained with Helsinki Testbed masts also have many peak type errors and discontinuities. A quality control was performed for all the data and indistinct observations were excluded from error calculations. However, the calculations may still contain some erroneous observations. The graphics show all the observations.

The accuracy and reliability were determined from model forecasts. Reliability is more important in this examination. The validation of wind speed was made separately for every available station in order to determine if there are differences in error structure between masts in inland, near the coastline and in sea areas. The mean error was calculated for all stations. It is not easy to compare the results for different masts because the observation data differ from station to station. Only 10% of all possible observations were available for some masts while the corresponding figure for some Testbed stations may be more than 75%. The whole validation period was 13 months and observations from over 20 stations were available. However, only stations having more than 50% of all possible observations were included in the comparison. The outliers were excluded.

The results of the validation indicate that the error differed from station to station. The day/night difference was great as well. The mean errors were calculated for all representative stations.



 

When the above results are considered, one has to take into account that in the AROME model (or any model), wind speed increases (between levels of 50 - 100 metres) 0.13 m/s over an open sea area and 0.23 m/s over a land area for every 10 metre layer. Even a slight difference in topographical height between the model representation and the real environment can be partly responsible for the errors shown in tables.

The figures below show the results for Viirilä mast as an example of a typical error distribution for model wind speed and observed wind speed. The first figure shows the mean AROME wind speed profile and the mean observation. The second figure shows the scatter diagram for the AROME model wind speed and the wind speed observations.



Mean error (ME) of model wind speed at Viirilä mast for different model levels over the period 1 July 2008 - 31 July 2009. The measurement level at Viirilä mast is 81 metres.



Scatter diagram of the model wind speed and the observed wind speed at Viirilä mast over the period 1 July 2008 - 31 July 2009. The measurement level at Viirilä mast is 81 metres.



Marginal distributions of model wind speed and the observations for all stations. The height of the model winds and observations ranges from 60 to 100 metres.

Some stations overestimate the wind speed while others underestimate it. At stations near the coastline, the model seems to underestimate the wind speed more than in open sea areas or inland areas.