Published on May 15th, 2016 | by Zachary Shahan32
US EIA Responds To CleanTechnica Letter/Criticism On Energy Forecasts
May 15th, 2016 by Zachary Shahan
More than 2 years ago, some CleanTechnica readers got together and wrote an open letter to the US Energy Information Administration (EIA) regarding its routinely tragic and off-the-mark energy forecasts. Here’s one highlight from that:
It was forecast [in 2010] that we would reach 0.45 GW of Solar PV on the grid by 2035, in November 2013 we reached 7.11 GW according to the FERC.
Surely, in making new predictions it would be appropriate for the EIA to address how their models could produce a 25 year forecast which has already been surpassed 16 times over in less than 3 years.
Spending a couple of years to develop a response (I guess), and pressure by later articles along the same lines and tackling related matters (cost projections, for example), the EIA published this press release approximately one month ago:
EIA’s data for renewable electricity, in particular wind and solar generators, are one of the largest areas of interest among EIA data users, as well as one of the more frequent targets of criticism. Although particular details vary from source to source, several critiques have involved claims along the following lines:
- EIA data do not accurately track wind and solar generation or capacity, particularly distributed solar photovoltaics (PV)
- EIA projections “consistently” and “significantly” underestimate additions of wind and solar capacity
- EIA estimates for the cost of renewable capacity such as wind and solar are out-of-date and not representative of current market costs
In an effort to improve EIA’s approach to providing accurate, comprehensive data, and useful projections for policy analysis, EIA has conducted a review of its historical data and projections of capacity, generation, and cost projections for wind and solar technologies. While EIA’s internal processes and engagement with stakeholders are both continuing, this paper shares some early findings of EIA’s current review of our wind and solar data and projections, focusing in part on some of the issues that have been publicly raised by EIA’s critics.
The report addresses historical data and capacity and generation projections in the context of changing renewable energy markets, and addresses how EIA is responding to these evolving markets. A more detailed review of past EIA capacity and generation projections, and of actual and projected technology costs for both wind and solar is provided in the Appendix.
I saw this response and passed it on to Bob Wallace, who passed it on to other co-authors of the letter.
One of the co-authors, Robert Smithers, wrote: “Very defensive and self affirming but with a few key admissions of flaws…. I do agree with them that predicting congressional policy actions anything more than about 5 mins out from the expiry of previous bills is nigh on impossible. However, they don’t seem to respond to the criticism of their forecast that, without governmental support, not a single MW of solar would be installed in the next decade.”
Indeed, the report is quite “defensive and self affirming,” and doesn’t seem keen to highlight all the reasons why discrepancies between EIA forecasts and reality (like the one noted at the top) occurred. Much of the response is along the lines of, “actually, we were right and the critics were wrong … if we look at this very specific statement and the data from this report … not this one” (not a direct quote, of course). The main response in the “Introduction” section is as follows:
Given the recent growth rates in wind and solar capacity and generation, EIA follows these technologies closely and is committed to assuring the quality of its data and projections for these sources through regular internal and external reviews. While EIA’s internal processes and engagement with stakeholders are both continuing, this paper shares some early findings of EIA’s current review of our wind and solar data and projections, focusing in part on some of the issues that have been publicly raised by EIA’s critics.
Great that the review is ongoing, but it would have been nice if the EIA had clarified some of the core faults underlying its previous work in the introduction, so we’d have a better sense of whether or not we can trust its forecasts going forward.
But let’s dig into the precise responses to certain critiques in a bit more detail.
One interesting segment for me was this one:
Some critics suggest that EIA does not account for distributed PV systems (also known as “roof-top” or “end-use” PV systems) in its statistics and projections, thus missing as much as one-third of the current solar generation in the United States, along with associated future growth potential from this market segment. In fact, EIA has long produced current estimates and projections for both distributed and utility-scale PV installations and generation, and has recently enhanced its data on small-scale distributed PV.
… combined with this one a little further on in the report:
EIA recognizes that a more integrated presentation of generation data for utility-scale solar and small-scale (less than 1 megawatt) distributed PV would be beneficial. Since late 2014, EIA has been working to develop monthly state-level estimates of distributed small-scale PV generation, matching the periodicity and geographic breakdown used in presenting utility-scale generation data on EIA’s website. Starting with the publication of the November 2015 Electric Power Monthly (EPM) EIA is reporting these estimates together with utility-scale generation data in the EPM and including them in electricity-related tables and browsers associated with that publication.
Indeed, as I wrote in my December US electricity generation article, the EIA finally started including rooftop solar estimates in its monthly electricity reports (note that this doesn’t relate to anything in our initial letter to the EIA, but I had been producing rooftop solar estimates for many months since the EIA didn’t). One big problem, before the change, is that the terminology in the tables in the monthly reports made it seem like EIA was reporting all solar when, in fact, it wasn’t. Again, in its response here, the initial statement on the matter makes it sound like EIA has always done this: “has long produced current estimates and projections for both distributed and utility-scale PV installations and generation.” It’s not until the next page (following another section and chart) that EIA points out that it didn’t previously include rooftop solar estimates in its monthly electricity reports.
In any case, I was very happy to see EIA make the switch at the end of 2015 to include rooftop solar estimates in its monthly reports.
Regarding the main criticism of our readers, the report goes on at length to explain that the forecasts are based on assumptions regarding policy, and the reference scenarios have historically assumed that any policies with an expiration date will simply expire and nothing will replace them. That’s clear if you are sure to read the assumptions in the forecast reports (but not necessarily to a lay person running across a chart or statement from it), but if you haven’t yet read the 2014 letter from our readers or simply forget it, here’s a section of it that presented what still seems like an absurdly pessimistic picture (from the EIA) of the future of renewables without any policy changes:
PV solar installations stop in 2016 and do not resume for 12 years and even then at a rate significantly below current rates.
No thermal solar is constructed past 2014.
Construction of wind farms ceases in 2016 and does not resume for almost 20 years.
Municipal waste generation capacity additions grind to a halt in two years, never to occur again.
Wood and biomass suffer a similar fate.
We find it highly unusual that 5 statistical models using 5 data samples regarding 5 different energy technologies at varying levels of economic and technological maturity could produce such strikingly similar results. The implication is that the EIA’s official position is that all the major non-hydro grid-scale renewables will see 1-2 further years of capacity growth followed by 15-27 years of no further installations.
Yes, without the Clean Power Plan, fewer coal plants would be retired; without PTC and ITC extensions, less renewable energy would be installed; but a nearly complete flatlining for ~20 years?
Nonetheless, it is good to see and nice that EIA highlights that, with assumptions in place for the effect of the Clean Power Plan and renewable incentives, the story is quite different:
- When policies are assumed that restrict existing generation that competes with new renewables, EIA’s projections show significantly higher penetration rates for both wind and solar than when such policies are not considered. This effect is shown by comparing renewable capacity additions in EIA’s AEO2015 Reference case with its subsequent analysis of the Clean Power Plan (CPP)10, 11 as proposed by the U.S. Environmental Protection Agency (EPA) in June 2014. With the proposed CPP in place, wind capacity reaches 192 GW by 2030 and 205 GW by 2040, well above the projected levels of 87 GW and 110 GW for the same years in the Reference case. Solar PV capacity also grows much more rapidly with the CPP in place, reaching 74 GW in 2030 and 135 GW in 2040, compared to 37 GW and 59 GW for the same years in the Reference case.
- EIA’s AEO and CPP analyses reflect the view that with very slow growth in U.S. electricity demand, electricity from new wind and solar plants will largely be competing in the near term to displace generation from existing coal- and natural gas-fired power plants. In the Reference cases of recent AEO editions, new coal-fired generation plants are not economically competitive with renewables and other generation sources, as reflected by the near absence of new coal capacity additions over the 2015-40 period, even without an assumed extension of tax credits for renewables or promulgation of the CPP rules for new and existing sources. However, under the then-current laws reflected in those Reference cases, displacing incumbent coal generation is much less economically viable than under the recently analyzed CPP proposal, which essentially forces existing coal-fired units to reduce their generation.
But… again, does anyone really think it was sensible to forecast almost no solar or wind power growth without the CPP, ITC, and PTC?
Jigar Shah published a quick response to the new EIA report over on LinkedIn, which stimulated a nice train of interesting responses. Alex Gilbert writes:
Ideally, they would have presented an analysis of most of their past projections, heir assumptions underlying those projections, and identification of potential ways to correct model issues. Comparably, this analysis focused on selecting data that is most favorable to EIA, explaining away a few points, and ignoring the many, many projections from the EIA that were demonstrably way lower than reality.
A more thorough analysis of EIA’s projections, like the one Benjamin Sovacool and I published back in December, is much more helpful in understanding why these projections were so bad. As EIA has noted, policy changes are near impossible to predict and have certainly played a factor. However, the model underlying their projections does not account for many of the advantages of renewable energy: stable long term contracts, price hedging, reduced carbon regulatory risk, modularity, and environmental attributes, to name a few. These are critical factors underlying renewable energy growth and a model that does not account for them is concerning.
Adam Siegel adds:
Lowell Feld highlights how some of the solar forecasts have been in the range of 20,000% wrong: ‘EIA’s 2008 AEO forecast for solar power capacity in 2015, which was off by a whopping 19,471% (0.14 GW forecast vs. 27.4 GW actual solar capacity in 2015).’
The just released EIA paper is bureaucratic a__-covering, selectively citing limited critiques rather than whole-hearted ones and is not a serious analysis of why the forecasts are wrong and what can/should be done about this.
But let’s be really nice to the EIA and even assume that they were right all along and if there wasn’t the Clean Power Plan, if there wasn’t an extension of the ITC, if there wasn’t an extension of the PTC, and if other support for renewables was killed at expiration, the forecasts would have been right. Even if that would have been true (a lofty assumption in an illogical fantasy world), a key point is that energy experts and journalists at Scientific American, Greentech Media, POLITICO, CleanTechnica, and elsewhere were so confused by the EIA’s reports that they took the forecasts as genuine forecasts about the real future of our country — not building blocks to layer policy assumptions onto.
To me, that clearly indicates that, aside from underlying problems with the forecast assumptions that would somehow have the solar and wind industries flatlining for decades, the EIA’s presentation of these hypothetical forecasts is in need of improvement. It should be absolutely obvious with such unrealistic forecasts that they are not predictions, but are guesses about what would happen in an extreme world where fossil fuels kept their baked-in subsidies but renewables lost essentially all of their support. I hope we see a dramatic shift in Annual Energy Outlook 2016.
Robert Smithers adds: “Despite the defensive nature of the response, it seems clear our letter has put the EIA on notice that poorly performing or clearly nonsense renewable forecasts can not simply be buried in the detail of a report, they are being watched by people more than willing to publicly call them out if they fail to improve their performance.”