Published on May 15th, 2016 | by Zachary Shahan


US EIA Responds To CleanTechnica Letter/Criticism On Energy Forecasts

May 15th, 2016 by  

More than 2 years ago, some CleanTechnica readers got together and wrote an open letter to the US Energy Information Administration (EIA) regarding its routinely tragic and off-the-mark energy forecasts. Here’s one highlight from that:

It was forecast [in 2010] that we would reach 0.45 GW of Solar PV on the grid by 2035, in November 2013 we reached 7.11 GW according to the FERC.

Surely, in making new predictions it would be appropriate for the EIA to address how their models could produce a 25 year forecast which has already been surpassed 16 times over in less than 3 years.

Spending a couple of years to develop a response (I guess), and pressure by later articles along the same lines and tackling related matters (cost projections, for example), the EIA published this press release approximately one month ago:

EIA’s data for renewable electricity, in particular wind and solar generators, are one of the largest areas of interest among EIA data users, as well as one of the more frequent targets of criticism. Although particular details vary from source to source, several critiques have involved claims along the following lines:

  • EIA data do not accurately track wind and solar generation or capacity, particularly distributed solar photovoltaics (PV)
  • EIA projections “consistently” and “significantly” underestimate additions of wind and solar capacity
  • EIA estimates for the cost of renewable capacity such as wind and solar are out-of-date and not representative of current market costs

In an effort to improve EIA’s approach to providing accurate, comprehensive data, and useful projections for policy analysis, EIA has conducted a review of its historical data and projections of capacity, generation, and cost projections for wind and solar technologies. While EIA’s internal processes and engagement with stakeholders are both continuing, this paper shares some early findings of EIA’s current review of our wind and solar data and projections, focusing in part on some of the issues that have been publicly raised by EIA’s critics.

The report addresses historical data and capacity and generation projections in the context of changing renewable energy markets, and addresses how EIA is responding to these evolving markets. A more detailed review of past EIA capacity and generation projections, and of actual and projected technology costs for both wind and solar is provided in the Appendix.

See complete report

I saw this response and passed it on to Bob Wallace, who passed it on to other co-authors of the letter.

One of the co-authors, Robert Smithers, wrote: “Very defensive and self affirming but with a few key admissions of flaws…. I do agree with them that predicting congressional policy actions anything more than about 5 mins out from the expiry of previous bills is nigh on impossible. However, they don’t seem to respond to the criticism of their forecast that, without governmental support, not a single MW of solar would be installed in the next decade.”

Indeed, the report is quite “defensive and self affirming,” and doesn’t seem keen to highlight all the reasons why discrepancies between EIA forecasts and reality (like the one noted at the top) occurred. Much of the response is along the lines of, “actually, we were right and the critics were wrong … if we look at this very specific statement and the data from this report … not this one” (not a direct quote, of course). The main response in the “Introduction” section is as follows:

Given the recent growth rates in wind and solar capacity and generation, EIA follows these technologies closely and is committed to assuring the quality of its data and projections for these sources through regular internal and external reviews. While EIA’s internal processes and engagement with stakeholders are both continuing, this paper shares some early findings of EIA’s current review of our wind and solar data and projections, focusing in part on some of the issues that have been publicly raised by EIA’s critics.

Great that the review is ongoing, but it would have been nice if the EIA had clarified some of the core faults underlying its previous work in the introduction, so we’d have a better sense of whether or not we can trust its forecasts going forward.

But let’s dig into the precise responses to certain critiques in a bit more detail.

One interesting segment for me was this one:

Some critics suggest that EIA does not account for distributed PV systems (also known as “roof-top” or “end-use” PV systems) in its statistics and projections, thus missing as much as one-third of the current solar generation in the United States, along with associated future growth potential from this market segment. In fact, EIA has long produced current estimates and projections for both distributed and utility-scale PV installations and generation, and has recently enhanced its data on small-scale distributed PV.

… combined with this one a little further on in the report:

EIA recognizes that a more integrated presentation of generation data for utility-scale solar and small-scale (less than 1 megawatt) distributed PV would be beneficial. Since late 2014, EIA has been working to develop monthly state-level estimates of distributed small-scale PV generation, matching the periodicity and geographic breakdown used in presenting utility-scale generation data on EIA’s website. Starting with the publication of the November 2015 Electric Power Monthly (EPM) EIA is reporting these estimates together with utility-scale generation data in the EPM and including them in electricity-related tables and browsers associated with that publication.

Indeed, as I wrote in my December US electricity generation article, the EIA finally started including rooftop solar estimates in its monthly electricity reports (note that this doesn’t relate to anything in our initial letter to the EIA, but I had been producing rooftop solar estimates for many months since the EIA didn’t). One big problem, before the change, is that the terminology in the tables in the monthly reports made it seem like EIA was reporting all solar when, in fact, it wasn’t. Again, in its response here, the initial statement on the matter makes it sound like EIA has always done this: “has long produced current estimates and projections for both distributed and utility-scale PV installations and generation.” It’s not until the next page (following another section and chart) that EIA points out that it didn’t previously include rooftop solar estimates in its monthly electricity reports.

In any case, I was very happy to see EIA make the switch at the end of 2015 to include rooftop solar estimates in its monthly reports.

Regarding the main criticism of our readers, the report goes on at length to explain that the forecasts are based on assumptions regarding policy, and the reference scenarios have historically assumed that any policies with an expiration date will simply expire and nothing will replace them. That’s clear if you are sure to read the assumptions in the forecast reports (but not necessarily to a lay person running across a chart or statement from it), but if you haven’t yet read the 2014 letter from our readers or simply forget it, here’s a section of it that presented what still seems like an absurdly pessimistic picture (from the EIA) of the future of renewables without any policy changes:

PV solar installations stop in 2016 and do not resume for 12 years and even then at a rate significantly below current rates.

No thermal solar is constructed past 2014.

Construction of wind farms ceases in 2016 and does not resume for almost 20 years.

Municipal waste generation capacity additions grind to a halt in two years, never to occur again.

Wood and biomass suffer a similar fate.

We find it highly unusual that 5 statistical models using 5 data samples regarding 5 different energy technologies at varying levels of economic and technological maturity could produce such strikingly similar results. The implication is that the EIA’s official position is that all the major non-hydro grid-scale renewables will see 1-2 further years of capacity growth followed by 15-27 years of no further installations.

Yes, without the Clean Power Plan, fewer coal plants would be retired; without PTC and ITC extensions, less renewable energy would be installed; but a nearly complete flatlining for ~20 years?

Nonetheless, it is good to see and nice that EIA highlights that, with assumptions in place for the effect of the Clean Power Plan and renewable incentives, the story is quite different:

  • When policies are assumed that restrict existing generation that competes with new renewables, EIA’s projections show significantly higher penetration rates for both wind and solar than when such policies are not considered. This effect is shown by comparing renewable capacity additions in EIA’s AEO2015 Reference case with its subsequent analysis of the Clean Power Plan (CPP)10, 11 as proposed by the U.S. Environmental Protection Agency (EPA) in June 2014. With the proposed CPP in place, wind capacity reaches 192 GW by 2030 and 205 GW by 2040, well above the projected levels of 87 GW and 110 GW for the same years in the Reference case. Solar PV capacity also grows much more rapidly with the CPP in place, reaching 74 GW in 2030 and 135 GW in 2040, compared to 37 GW and 59 GW for the same years in the Reference case.
  • EIA’s AEO and CPP analyses reflect the view that with very slow growth in U.S. electricity demand, electricity from new wind and solar plants will largely be competing in the near term to displace generation from existing coal- and natural gas-fired power plants. In the Reference cases of recent AEO editions, new coal-fired generation plants are not economically competitive with renewables and other generation sources, as reflected by the near absence of new coal capacity additions over the 2015-40 period, even without an assumed extension of tax credits for renewables or promulgation of the CPP rules for new and existing sources. However, under the then-current laws reflected in those Reference cases, displacing incumbent coal generation is much less economically viable than under the recently analyzed CPP proposal, which essentially forces existing coal-fired units to reduce their generation.

But… again, does anyone really think it was sensible to forecast almost no solar or wind power growth without the CPP, ITC, and PTC?

Jigar Shah published a quick response to the new EIA report over on LinkedIn, which stimulated a nice train of interesting responses. Alex Gilbert writes:

Ideally, they would have presented an analysis of most of their past projections, heir assumptions underlying those projections, and identification of potential ways to correct model issues. Comparably, this analysis focused on selecting data that is most favorable to EIA, explaining away a few points, and ignoring the many, many projections from the EIA that were demonstrably way lower than reality.

A more thorough analysis of EIA’s projections, like the one Benjamin Sovacool and I published back in December, is much more helpful in understanding why these projections were so bad. As EIA has noted, policy changes are near impossible to predict and have certainly played a factor. However, the model underlying their projections does not account for many of the advantages of renewable energy: stable long term contracts, price hedging, reduced carbon regulatory risk, modularity, and environmental attributes, to name a few. These are critical factors underlying renewable energy growth and a model that does not account for them is concerning.

Adam Siegel adds:

Lowell Feld highlights how some of the solar forecasts have been in the range of 20,000% wrong: ‘EIA’s 2008 AEO forecast for solar power capacity in 2015, which was off by a whopping 19,471% (0.14 GW forecast vs. 27.4 GW actual solar capacity in 2015).’

The just released EIA paper is bureaucratic a__-covering, selectively citing limited critiques rather than whole-hearted ones and is not a serious analysis of why the forecasts are wrong and what can/should be done about this.

But let’s be really nice to the EIA and even assume that they were right all along and if there wasn’t the Clean Power Plan, if there wasn’t an extension of the ITC, if there wasn’t an extension of the PTC, and if other support for renewables was killed at expiration, the forecasts would have been right. Even if that would have been true (a lofty assumption in an illogical fantasy world), a key point is that energy experts and journalists at Scientific American, Greentech Media, POLITICO, CleanTechnica, and elsewhere were so confused by the EIA’s reports that they took the forecasts as genuine forecasts about the real future of our country — not building blocks to layer policy assumptions onto.

To me, that clearly indicates that, aside from underlying problems with the forecast assumptions that would somehow have the solar and wind industries flatlining for decades, the EIA’s presentation of these hypothetical forecasts is in need of improvement. It should be absolutely obvious with such unrealistic forecasts that they are not predictions, but are guesses about what would happen in an extreme world where fossil fuels kept their baked-in subsidies but renewables lost essentially all of their support. I hope we see a dramatic shift in Annual Energy Outlook 2016.

Robert Smithers adds: “Despite the defensive nature of the response, it seems clear our letter has put the EIA on notice that poorly performing or clearly nonsense renewable forecasts can not simply be buried in the detail of a report, they are being watched by people more than willing to publicly call them out if they fail to improve their performance.”

Check out our new 93-page EV report, based on over 2,000 surveys collected from EV drivers in 49 of 50 US states, 26 European countries, and 9 Canadian provinces.

Tags: , , , , , ,

About the Author

is tryin' to help society help itself (and other species) with the power of the typed word. He spends most of his time here on CleanTechnica as its director and chief editor, but he's also the president of Important Media and the director/founder of EV Obsession, Solar Love, and Bikocity. Zach is recognized globally as a solar energy, electric car, and energy storage expert. Zach has long-term investments in TSLA, FSLR, SPWR, SEDG, & ABB — after years of covering solar and EVs, he simply has a lot of faith in these particular companies and feels like they are good cleantech companies to invest in.

  • dRanger

    You can call me Captain Obvious but I am compelled to spell this out. Given the way that exponential growth curves work, if the EIA had ever produced an accurate prediction in the last 20 years, it would have forecast the death of the fossil fuel industry by 2030. As stock markets are foreward-looking, any such prediction would have been a self-fulfilling prophecy. The EIA has had the power of life and death over the FF industry for a long time and their preference has now become so painfully obvious that it just can’t be hidden any more. But their efforts extended the profitability of those companies by many years, so mission accomplished, I guess. I’m not sure that calling this “regulatory capture” really does the situation justice.

    • Bob_Wallace

      That strongly suggests the EIA may be under political pressure to mis-forecast the future in order to keep the fossil fuel industry from destroying their budget.

      • dRanger

        Yes, if you assume that those in the EIA who were responsible for this actually care about the future welfare of the agency.

        • Bob_Wallace

          They probably care about their paychecks.

          And their bosses probably care about the agency budget. Because, their paychecks….

  • NRG4All

    Because history seems to repeat itself, why doesn’t the EIA come out with a forecast for 2015 and next year come out with a forecast for 2016. I’ll bet they’d be a lot more accurate. (;oD

    • Bob_Wallace

      That’s not a bad idea. Have the EIA start with something that’s within their ability. Then, after a few years of success, they could attempt to predict a month into the future and stick with that until they get it right before moving on to two months.

      I just noticed that someone has updated the Wiki page on US electricity costs and pointed out how the EIA wind/solar prices are bogus.

      “Since 2010, the US Energy Information Administration (EIA) has published theAnnual Energy Outlook (AEO), with yearly LCOE-projections for future utility-scale facilities to be commissioned in about five years’ time. In 2015, EIA has been criticized by the Advanced Energy Economy (AEE) Institute after its release of the AEO 2015-report to “consistently underestimate the growth rate of renewable energy, leading to ‘misperceptions’ about the performance of these resources in the marketplace”. AEE points out that the average power purchase agreement (PPA) for wind power was already at $24/MWh in 2013. Likewise, PPA agreements for utility-scale solar PV are seen at current levels of $50–$75/MWh.[52] These
      figures contrast strongly with EIA’s estimated LCOE of $125/MWh (or $114/MWh including subsidies) for solar PV in 2020.[53] ”

      • RobS

        Yes the absurd situation that has existed for some time now where the EIA has forecast renewable costs at 5 years that are several times higher than the current price of these technologies is particularly absurd and is becoming more widely known so hopefully they take serious action to tackle those irregularities too.

        • eveee

          Sort of like the weather man looking outside to see if it’s raining before declaring its a sunny day

          • RobS

            It’s worse than that because they’re not even right about current prices, so it’s more like a weatherman who looks outside and sees that it’s sunny but declares its raining anyway

    • neroden

      Investment professionals don’t even read the EIA’s long term projections any more. We go straight to Lazard. That says something, doesn’t it?

      • Calamity_Jean

        How accurate has Lazard historically been?

  • Thomas Digby

    Fantastic article Zach. My biofuel startup is relying (in part) on EIA’s biofuel projections. Now I’m screwed.

  • eveee

    Particularly agree that the heat is now properly on EIA to perform . Now is the time to make formal requests. Here are my suggestions.

    1. Include logarithmic plots of past cost progress as a guide to future cost declines.

    2. Include cost decline versus volume estimates as Swansons Law predicts.

    3. Discuss possible future cost decline estimates and causes.

    4. Figure out tipping points versus existing conventional sources.

    5. Include storage advances and do likewise.

    6. Include Conventional source cost increases from historical trends.

    7. Discuss and estimate future and recent conventional source increases due to fuel resource depletion and emission and regulatory needs.

    8. Discuss and estimate the effects of technology substitution at tipping points. Note the accelerated adoption S curve effects.

    OK guys. Add to or improve this list. Who is willing to take this to the EIA and DOE or wherever necessary to get things moving?

  • Steven F

    Why doesn’t the EIA focus on Renewable Portfolio Standard that most states have today? If they did they would have to show renewables climbing every year. California’s law requires a a lot or renewables added to the grid all the way out to 2040. California alone blows out this forecast out of the water. Texas like California would do the same. The laws would not specify the renewables technologies to use. However it would not take a rocket scientist to guess that much of the new addition in texas would be wind while California would be mostly solar.

    So even if we make the worst possible assumptions on the directions of US energy law, renewables will still grow because Washington DC cannot control State RPS laws. Clean Technica should ask the EIA why RPS laws are not reflected in the EIA report and ask why.

  • OneHundredbyFifty

    Thank you for publishing this and continuing to move the ball forward. Nice that they grudgingly gave a response. Unfortunately it was designed to keep their jobs. If I recall they correctly pointed out that they were a few percentage points better than SEIA at doing a projection 1 year out. However they did not address, AT ALL, their horrific record for longer term projections – 5 years, 10 years. These are the projections on which policy is based. So congress has been able to point to the, supposedly, objective forcasts and say that ‘for the forseable future, renewables will have no role’ and therefore they do little to nothing to lay the groundwork for renewables. EIA is not alone however, the International Energy Agency is also doing an awful job. –

    I think the really sad thing is that these accelerating trends are going to crush the fossil fuel industry much faster than anyone expects. By pulling the wool over the eyes of policy makers the FF industry is (and I believe they do so knowingly) creating a situation where we will not be able to develop retraining policies quickly enough and a huge number of people are going to get economically crushed because there will not be the retraining infrastructure in place. Coal is done. Natural gas will be in serious trouble if even a few of the HVDC lines get built bringing high CF night wind to the coats from the Great Plains states. And finally, the oil industry is already seeing pain. However their prices are relatively inelastic. In the 2020 – 2025 timeframe EVs will put pressure on the price of gasoline. For a short time people will buy bigger cars and stave off the inevitable collapse. But battery prices will continue to drop and they will transition to selling gasoline at or below cost and rapidly collapse.

    The Tesla announcement of moving their 500,000 car target to 2018 will likely go down as a historical marker. It will be recognized as the point at which we went from market shift to market disruption.

  • Freddy D

    Glad to hear that there has been some semblance of a response. If direct engagement with the EIA does not result in cleaning up their sloppy work, perhaps the message needs to be taken above their heads to the executive branch and Congress. And keep up the great coverage in the press!

    I would encourage the press to have concise, poignant headlines and articles to call them on the carpet for their failed work.

  • Marion Meads

    They’re still lying and will continue to lie about renewables based on their response. I don’t know who they really wirk for. A massive campaign to discredit these sthink stunks should be spread and go viral especially directed to those they supposedly serve. Their no longer credible.

    • Mike Dill

      Unfortunately the massaged numbers from the department of labor statistics and the bureau of economic analysis are just as far off. The cost of living numbers have been ‘adjusted’ for years. They ‘say’ that health care costs are not a problem, and then your premiums go up.

      • JamesWimberley

        You need to provide a lot more evidence for the proposition that the BLS rigs its core statistics on output, inflation and unemployment for me not to dismiss it as an ignorant calumny.

        The topic is forecasts anyway, not historical data, which everybody agrees the EIA does a good job on. Major economic forecasting teams at the Fed, the Treasury, the IMF, the OECD, and private firms like Macro Advisers have a pretty good record. They don’t and can’t predict panics like the 2007 crisis.

        • neroden

          GDP numbers have been garbage for decades because nobody really knows how to measure it, and everyone wants to manipulate it. A particular problem area is the “financial sector contribution to GDP”, which is essentially fictional and can be manipulated directly by banks and insurance companies with internal trades. When I want to look at output, I look at steel production, railcar movements, etc.

          Inflation and unemployment are MUCH easier to measure and are pretty accurate.

          FWIW I predicted the 2007 crisis. Though nobody can predict the exact timing of such a crisis, it’s easy enough to spot an epidemic of fraud being committed at banks. Watch _The Big Short_ for evidence of this. I didn’t spot even a fraction of what those guys spotted and I still knew there was going to be a crash due to a fraud epidemic at banks.

  • JamesWimberley

    Don’t count on real improvements. The first step is always honesty: I am an alcoholic, the article is flawed, we screwed up. If the Treasury’s economic forecasters had a similar track record, they would be running the Juneau office and a different team would be in place. Why does Moniz put up with this?

    • Mike Dill

      Moniz and most of the DOE work for the power companies, which has been mostly coal and nuclear interests for the past fifty years. Hard to get the culture to change when the people paying do not want it to change. Same problem in TV ‘journalism’.

      • Ross

        I was under the impression Moniz was an academic and administrator and not working for fossil fuel vested interests.

        • Rick Thurman

          EIA’s Board of Directors is the US Congress. Not the Congress you might wish was in place, but the actual Congress with it’s combination of ideologically internalized willful ignorance and cowed deference to the power structures of the last century (ie, fossil fuels). We’ve already heard elsewhere that the US Navy has been told to not mention sea level rise in budget requests to raise the docks at the Hampton Roads facilities in Virginia (perhaps some brave discordant soul can schedule budget hearings at these docks at the next really high tide). Some state agencies in neighboring Southern coastal states have been told mentioning SLR in official reports could be bad for their collective careers. Has the EIA been “mentored ” to find a conservative methodology, if they know what’s good for them? How eager would you be to write up an accurate forecast if you knew its very accuracy would guarantee it would be you and your bosses’ last?
          Maybe the best that should be expected is just this: unassailable historical data to allow outsiders to competitively generate a variety of forecasts free from such political pressure, while applying a forecast methodology so blatantly mindless as to scream “to those with the eyes to see, let them see! We refuse to even pretend to offer guidance here. The truth may or may not be elsewhere, but it sure ain’t here… so start looking.”

        • TedKidd

          Regulatory capture? Sycophant?

          Is there a reason to be impressed by Moniz? Is he a doer, or a bureaucrat?

    • Omega Centauri

      If I were appointed as p[resident, my first action would be to completely abolish the EIA. Start over from scratch, retaining exactly zero of the current employees.

      • Bob_Wallace

        Their historical data reporting seems to be pretty solid. I’ve never seen a complaint about their numbers.

      • Rather than toss out the bureaucrats, why not task them with collecting accurate data and then paying for private forecasts that prove to accurately track EIA as-is data over time?

        Research companies could build a portfolio of accurate models that become a revenue stream, or build politically correct models and starve.

        Thus the EIA would foster an industry with the right profit motivations – accurately forecast or don’t get paid. This is similar to NASA’s (partially implemented thus far) approach to privatizing space – launch successfully or you lose future contracts.

        It’s all about the incentives.

      • RobS

        As I said in reply to a similar comment on the original letters article most of the EIA’s work is of high quality. Even their short term forecasting team which produce the Short Term Energy Outlooks (STEO) do a pretty good job, indeed many times the STEO has forecast renewable energy installations three years out that are higher than the long term forecasters have predicted at 25 years. This seems to be a highly contained issue within the office that produces the Annuals Energy Outlooks long range forecasts.

      • neroden

        Well, my first action would be to shut down the money-losing national-security-endangering US Military, but I’m a radical.

    • eveee

      Exoect change by making requests and being firm.

Back to Top ↑