Friday, January 29, 2016

Communicating Uncertainty

It's been a week since the heavy snows began. On Sunday, in a story about the Pennsylvania Turnpike reopening, a Turnpike official was quoted as saying that the situation worsened last Friday evening with the arrival of a storm that started earlier and farther west than initially anticipated. (Westbound motorists became stranded on the east side of the Turnpike's Allegheny Tunnel. This link to a wiki article about the tunnel includes a view from the final curve on what I remember as a grim uphill climb to the tunnel's eastern entrance.)
After reading the quote from the turnpike official, I googled "Louis Uccellini" because I remembered seeing a brief television clip showing the director of the National Weather Service cautioning uncertainty about the northern edge of the snow. Would that the local forecast offices in Pennsylvania had been echoing the caution of their Director. Instead, until the last minute, their public forecasts reflected a confident dismissal of the few models that were suggesting the possibility of a rapid northward advance of the heavy snow.
On Monday a Turnpike official clarified that, despite initial anticipations, their crews had reacted and had been doing everything they could to keep their road plowed and treated. On Tuesday the Turnpike officials and the Governor received new rounds of (undeserved, I think) criticism. Last Friday they were caught between a rock and a hard place. Even if an omniscient private forecaster had successfully urged closing the Bedford-New Stanton section of the Turnpike late Friday afternoon or early Friday evening, Turnpike officials would have been criticized by the high volume of travelers who had chosen to escape DC at the last minute, having timed their escape based on the increasingly confident and specific public forecasts, and so not expecting the possibility of being blockaded at Bedford or Breezewood. If I had been traveling west from Washington last Friday, I would not have cut it that close. But then, I'm a meteorologist.

Some people who were driving home last Friday evening, expecting to beat the storm, might have canceled traveling to Washington in the first place if they had understood what most meteorologists were anticipating already at least four days before the storm. (Among the results of the google search for Lou Uccellini is a story about meteorologists marveling about seeing the storm coming several days ahead of time.) But the public zone forecasts did not convey such visions. In at least one of those forecasts on Monday evening, for an area not far from the Allegheny Tunnel, the outlook for Friday night was a bland chance of snow.
Without continuing to dwell on specific examples, from this or other recent major events (the problem is not limited to Northeast snow storms), I will state my gripe in general with how the National Weather Service local forecasts communicate uncertainty/confidence. My gripe is double-ended. On one end, it's often the case these days that major events can be seen coming several days ahead of time (for example, the likelihood of another hard freeze in Tucson next week). Yet the public forecasts 4 or 5 days out often downplay the confidence, as if the uncertainty was what it had been 20 or 30 years ago. There is an odd product, apparently intended as a heads-up for emergency management agencies, in which the local offices sometimes predict what they might later be predicting. This prediction of a possible prediction is accompanied by a confident statement that confidence will be increasing. Although it stands to reason that forecast confidence increases the closer you get to an event, forecasts several days in advance are often very good, and new data sometimes mostly confirms that the models were already on track. Nevertheless, thanks to the initial lowball estimate, communicated forecast confidence steadily increases, and becomes a stampede of increasing confidence. So at the other end, in the last 24 hours, communicated confidence reaches unrealistic levels of absolute perfection. Even if 10 models all agree with each other within 20 miles during the last 24 hours, the question should be whether they might all be off by 40 miles.
About twenty or thirty years ago, I would keep a copy of the Old Farmers Almanac handy. If someone asked about a forecast for 1-2 weeks ahead, I would start leafing through the Almanac. But these days I just check the Climate Prediction Center's 6 to 10 and 8 to 14 day outlooks. Those 1-2 week outlooks are concluded with a rating of the forecast confidence on a scale of 1 to 5, with 3 being average. Over the last few days I noticed that within the NWS itself the Milwaukee office currently includes similar ratings within their forecast reasoning discussions. Although the Milwaukee confidence levels are not explicitly stated in the actual zone forecasts, it looks like the reasoning process does give their zone forecasts more heft (compared to other offices). I think it would be straightforward for the NWS to make it a standard practive to state numerical confidence levels with their zone forecasts. The subjective confidence levels could supplement probability of precipitation, and would be easier to explain and better understood than probabilities. Perhaps the statements of confidence would only be required when they were above or below average, and there would be a brief explanation of the reason for departure from average. The tendency would be toward an average level of confidence appropriate for a particular lead time, without a stampede toward unrealistically far above average at shorter lead times, and that would be a good thing.

No comments:

Post a Comment