It was interesting to see AEP’s witnesses at the Wednesday PSC hearing trying to defend their whizzbang computer models from Strategist® and Aurora(whatever).
First, we need to step back an look the real purpose of these forecasts in a PSC case. The only thing that matters for AEP is that their scheme to transfer their obsolete plants to WV rate payers is approved by the WV PSC some time this fall. So their forecasts only have to hold up until the PSC issues its final order.
As I learned on Wednesday from AEP VP Torpey, AEP’s Aurora(whatever) program provides the forecasts of various things like the price of power plant capacity on PJM’s markets, the price of natural gas, the price of various kinds of coal, prices of electricity that AEP can sell electricity for in various markets, electricity demand, etc. Aurora(whatever) spits out numbers which Mr. Torpey or his minions plug into Strategist® which contains a model of all of AEP’s systems.
These computer models then allow AEP to run different scenarios through their systems to see how different options will impact the costs and benefits of their coal plant scheme against other options. Except, and this is a big one for me, their scenarios (probably because of the limitations of their computer programs) involve committing to only one alternative for a long period of time. As far as I can tell, none of their modeling has involved using a mix of alternatives at different times over the same period of time.
But, remember, all of these scenarios are focused on convincing two of the three WV PSC Commissioners to approve the AEP scheme, and that convincing only has to last until this fall.
AEP’s computer models are all based on data generated in 2011, because they created their models in 2012. As intervenor expert David Schlissel pointed out in his testimony on Tuesday, many of AEP’s forecasts of PJM capacity prices and natural gas fuel prices used in their model have been proven to be wildly high by actual results in 2012 and 2013, especially in the May 2013 PJM capacity auction which affects power plant capacity prices in 2016 and 2017.
Power company witnesses on Wednesday rejected Schlissel’s criticisms, saying that they never expect one isolated year to hit right on their forecasts for that year. In fact, it would be unusual if any of the actual forecasts hit exactly on the forecast trend line. While this is true in a general sense, most practicing forecasters recognize the importance of updating that same trend line as real time data indicates the accuracy of previous forecasts. This is vitally important for forecasters whose predictions will be used years into the future.
But in AEP’s PSC case, all their forecasters have to do is stiffarm any criticism of their modeling until the PSC issues its final decision. So what did their witnesses say on Wednesday? The realities of 2012 and 2013 don’t matter at all. Besides, the power plant scheme won’t actually happen until 2014, and the only numbers that matter are their predictions for after 2014.
I have taken (and passed with good grades) graduate level statistics and forecasting classes. There are certain basic rules about forecasting that were drilled into my head in those classes. One is that current real data is the best predictor of the near term future. Why? Because most of the conditions of today, which produced the real results, are likely to remain in effect for the near future. The other rule I learned was that the farther into the future you make your forecast, the less reliable your results are bound to be, because over time conditions can change radically from the current situation. These are basic rules of thumb, but they make sense.
Now, apply them to what AEP’s witnesses said on Wednesday: the current real numbers won’t affect AEP’s forecasts and models at all and only their future predictions are accurate. Doesn’t make any sense at all.
PJM Interconnection’s single more important forecasting task involves predicting demand for electricity the region. Getting it right is vitally important to managing their system. They make forecasts of electricity demand in the PJM region every year. Are their predictions very good? They haven’t been for the past few years. See my past posts here, here and here. Their forecasts aren’t very good, but at least they update them every year to reflect reality as it happens.
I found it very odd on Wednesday that PSC Commissioner Palmer asked (ordered) AEP to rerun their computer models with alternative prices for carbon, but he didn’t ask (order) those same officials to rerun the models taking into account the actual numbers what have actually happened since the models were generated in 2012. Mr. Torpey eagerly accepted the carbon price task, because he knew it would make no difference. I doubt that he would have been so enthusiastic about using updated numbers for PJM capacity prices and fuel costs.
Of course, the whole forecasting dance reveals an important aspect of the whole PSC process. The PSC Commissioners aren’t really looking for the best solutions. They are looking for justification for their own decision, regardless whether that decision is a good one or not. So once the Commissioners are moving toward a particular decision, they are not looking for any new facts in the case that would create new uncertainty or force them to change direction.
Commissioner Palmer wanted to make sure the PSC would look like the Commission had considered different carbon price scenarios, because he knew they might get some criticism if they didn’t. But when much more fundamental issues concerning AEP’s forecasting appeared in the case, the Commissioners largely ignored it, letting AEP’s claims stand.
It was very interesting to watch this all play out in person on Wednesday. Body language and visual connections in the room tell you a lot about what is really going on. Chairman Albert’s solicitous, almost unctuous, comments to an AEP witness, who was attempting to stiffarm the Commission on the forecasting issue, stood in direct contrast to his castigation of citizens in the FirstEnergy case less than two months ago.