~~~~~~~~~~~~~~~~~~~~~~~~~
US forecast models have been pretty terrible during Hurricane Irma
NOAA's best weather model seems to be getting worse with hurricanes, not better.
Eric Berger - 9/8/2017

We have written a fair amount at Ars recently about the superiority of the European forecast model, suggesting to readers that they focus on the ensemble runs of this system to get a good handle on track forecasts for Hurricane Irma. Then we checked out some of the preliminary data on model performance during this major hurricane, and it was truly eye-opening.

Brian Tang, an atmospheric scientist at the University of Albany, tabulates data on "mean absolute error" for the location of a storm's center at a given time and where it was forecast to be at that time. Hurricane Irma has been a thing for about a week now, so we have started to get a decent sample size—at least 10 model runs—to assess performance.
The model data

The chart below is extremely busy, but when you understand how to read it, the data is striking. It shows the average position error (in kilometers) at forecast lead times of 12, 24, 48, 72, 96, and 120 hours (so, out to five days). It compares several different classes of models, including global models that forecast conditions around the planet, nested models focused on hurricanes, and consensus forecasts. Specifically, the models are referenced as follows:

AVNO: US Global Forecast System, or GFS. The premiere US global model

CMC: Canadian global model

UKM: UK Met Office global model
Further Reading
At times during Harvey, the European model outperformed humans

ECMWF: European global model

NGX: US Navy global model

HMON: New, experiment US hurricane model

HWRF: Operational US hurricane model

TVCN: Consensus model, essentially an average of the global models

OFCL: Official National Hurricane Center forecast

~~~~~ continue at ~~~~~

https://arstechnica.com/science/2017/09/us-forecast-models-have-been-pretty-terrible-during-hurricane-irma/

In the chart below, the bars are error bars, so smaller is better. Note that the orange ECMWF bars show the least error (click to enlarge):