Critique of ASHRAE Journal article by Torcellini, Judkoff, and Crawley

by Dr. John H. Scofield, Chair and Professor of Physics & Astronomy, Oberlin College

In their recent article, “Lessons Learned: High-Performance Buildings,” Torcellini, Judkoff, and Crawley (TJ&C) describe the energy performance of six “green” buildings [1].  I know very little about five of these buildings but I know a great deal about the sixth building, Oberlin College’s Lewis Environmental Center.  With regard to that building I find their article to have greatly distorted building performance by presenting “half-truths.”  Their paper is but a tax-payer funded advertisement for the DOE's High Performance Building Program, no more, no less. It does not meet the usual standards for a peer-reviewed, professional publication.

TJ&C claim that Oberlin College’s Lewis Environmental Center’s measured [site] energy consumption is

“less than half the average Midwest educational building use”

and its net source energy is

“... 77% less than [what they simulate for] a baseline, code-compliant building.” 

Most readers will, of course, conclude that the Lewis Center is a very efficient building.  But engineers and building scientists should not be taken in by these meaningless sound bites. 

Below I examine the details behind such claims and show that, without its $420,000 PV Array – something that could be added to any building to reduce the energy it imports – the building's "steady-state" annual source energy consumption is only 3-17% better (depending on your yardstick) than that for a comparable, conventional building, despite its cost of $500/sf, more than double that for a conventional building. Since most of the comparable buildings predate the 1973 oil crisis, one would expect such modest gains for any new, code-compliant building.

1. Failure to cite earlier work and key sources

The paper by TJ&C does not meet accepted standards for referencing earlier work and primary sources.  They make no mention of my 2002 ASHRAE Transactions paper, "Early energy-performance for a green academic building," in which I carefully document the "as-built" design of the Lewis Environmental Center, model its energy performance, and report data from its first two years of operation [2]. While I welcome TJC's confirmation that this building is not able to become a net-energy exporter within its footprint I believe they have a professional obligation to acknowledge that my study reached this conclusion two years earlier.

It would appear that TJ&C would prefer that readers (and potential reviewers) remain ignorant of the earlier study despite the many “lessons learned” discussed in that paper.  Rather than a commitment to open inquiry, the hallmark of scientific investigation, TJ&C appear to be interested in carefully controlling the information they make available about this building -- the hallmark of advertising.  Despite their use of 21 references, TJ&C fail even to cite the 120-page internal NREL document upon which they base their own claims [3].  My repeated requests to obtain a copy of this document have been denied by Torcellini on the basis that “it is being held up by internal NREL review so is not yet released to the public.”  Yet copies of this document have been distributed for more than a year to Oberlin College Environmental Studies students, and TJ&C draw upon this “unreleased” document for their “Lessons learned” paper [1].  Could it be they don't want their readers to know of the existence of this document for fear that widespread distribution might lead to too many questions? Are similar internal NREL and other documents also being suppressed describing details of the other five building studies?

2. Presentation of Selective Data

TJ&C omit important information that readers should have in evaluating the generality of their conclusions.  For instance, they do not tell their readers the period for which the building energy is being reported nor circumstances that surround building operation during this period.  The building has been occupied for nearly 5 years.  Readers would reasonably assume that TJ&C’s data correspond either to a long term average or to the most recent (and thus most relevant) 12-mos. period.  Both assumptions would be incorrect. Their data come from a selected period in which the building's energy consumption was lower than it was before or has been since.

Month by month energy performance for this entire period may be found at  Running 12-mos. energy consumption (red) and generation (green) are shown in the graph below.

Figure 1. Graph showing measured energy consumption (red) and PV energy generation (green) for the preceeding 12-month period. The purple dashed line represents the architect's (William McDonough & Partners) energy-consumption projections. The blue rectangle shows the annual energy consumption figures that correspond to the period of hte NREL study.

As these data show, annual energy consumption reached a minimum during the two years corresponding to the NREL study, Mar. 1, 2001 through Feb. 28, 2003.  This period includes the warmest winter in 50 years!  NREL's own simulations suggest that energy consumption for the first year was reduced 20% due to the unusually warm climate [4]. The architect and his consultants apparently are referring to this when they claim "... simulations done at NREL show that even allowing for "normal" winter, the data change only modestly." [5]

In the middle of the monitoring period (April 2002) the College completed an estimated $100,000 upgrade of the building’s HVAC system by replacing one of its two “as-built” electric boilers with ground-source heat pumps.  Despite the investment, annual energy consumption through the next winter held steady due to the return of normal, colder winter weather.

With the NREL study completed, energy consumption rose (see explanation below).  For the most recent 12-mos period the building consumed 151,000 kWh (143,000 kWh if, like NREL, you neglect building transformer losses), by either measure, 19% higher than it was during the period of the NREL study [6].  TJ&C continue to monitor building energy consumption and are aware that energy consumption has increased during the last 18 months.  Yet they chose not to share this information with their readers.

For completeness the running 12-month heating degree day (HDD) and cooling degree day (CDD) figures for Cleveland are shown below. Each bar represents the HDD (or CDD) for the 12-month period that ends with that month. The graph clearly shows the warm weather during the first year of NREL monitoring. It also shows that the latest winter, 2003-4 was also milder than usual.

Figure 2. Graph showing the cumulative heating degree days (red) and cooling degree days (blue) for the preceeding 12-month period, based on Cleveland National Weather Service data. The solid lines represent 30-year HDD (purple) and CDD (aqua) averages.

3. Energy Saved by Reduced Heating and Ventilation

The reason for the increased energy consumption for the last 18 months is simple.  Energy use for this building is dominated by winter heating and ventilation.  (Cooling energy is not significant due to the low density of electric equipment and occupants, climate, reduced summer use, and the fact that much of the building space is not air-conditioned at all.)

As Paul Torcellini has said, you can lower the energy used by any building by turning down the thermostat in the winter (or up in the summer). Or you can simply reduce space usage or cease to condition space at all [7]. It is not remarkable to save energy through any of these methods.

Yet, whether by concisous decision or inept facilities management, this is precisely what was done during winters of 2001-2 and 2002-3. Students and staff regularly complained of the cold and wore coats to class.  Daytime temperatures in non-classroom spaces (restrooms, atrium, hallways, etc.) were usually in the lower 60's.  During the first winter the living machine set temperature was 45ºF.  Tempered water heat pumps delivered insufficient heat to classrooms as the (ground) water supply temperature dropped below heat pumps operating specifications because the auxiliary electric boiler was not used.  Classrooms were frequently used for scheduled classes without ventilation.  Scheduled classes were held in the auditorium on many occasions with neither heating nor ventilation. (On more than 20 occasions auditorium CO2 levels rose above 1200 ppm as classes were held without ventilation.)

These claims are not mere conjecture -- they are backed up by logs of 400 points from the HVAC control computer on 15-minute intervals. For detailed discussion and graphs of selected HVAC points see Lewis Environmental Center HVAC Operations and Logs.

Energy consumption rose last winter (2003-4) following completion of the NREL study. One important reason is that Oberlin College HVAC technicians finally activated the auxiliary electric boiler (EB-1) when ground water temperature dropped below the operating specifications for the tempered water heat pumps. This was not done during the NREL study. This resulted in more comfortable classroom temperatures than those experienced in the previous two winters at the cost of increased energy consumption. Only two of the building's 25 heat pumps (those added in the April 2002 HVAC renovation) are ground source heat pumps designed to operate with supply water temperatures down to freezing. The rest are tempered water heat pumps, unable to handle normal heating loads when the ground water temperature drops below 40ºF, or in one case, 60ºF [2]. And it appears that supply water flow rates to many of the heat pumps is insufficient for them to supply the required heat even with 40ºF supply water.

I should also mention that there were operational problems that also led to excessive energy consumption during the second year of the NREL study. I have written about these in my criticism of the inability of Oberlin College's facilities managers, Aramark, to properly operate campus buildings, including the Lewis Center. One can find numerous isolated decisions that led to more or less building energy use. But the same is true for other buildings with which the Lewis Center performance is being compared (see below). Hence I am not claiming that NREL deliberately lowered set temperatures or turned off ventilation to reduce energy use. I am simply stating the undeniable fact that these conditions existed, lowering energy consumption and making occupants uncomfortable.

I speculate, but do not have data to confirm this, that classroom usage decreased during the NREL monitoring period as folks scheduled classes and symposia in more comfortable facilities.

My point is that the main difference between building operation during the second year of the NREL study and the subsequent year (2003-4, when consumption rose by 20%) is the decision following completion of the NREL study to keep the building warmer, in part, by using the auxiliary electric boiler.

4. Site Energy Comparison Meaningless for Electric Building

TJ&C’s claim that site energy was less than half that for a typical, old, existing Midwestern education building is true but meaningless.  All-electric buildings always consume less on-site energy than equivalent buildings heated directly from fossil fuels.  But this does not make them cheaper to operate or better for the environment because of the off-site losses associated with the production of electricity.  Proper comparison involves source energy, which accounts for the off-site energy cost of producing and delivering the energy used by the building.  In many cases, all-electric buildings, while having lower site energy, have higher source energy, operating costs, and pollution associated with them.  So the site energy claim may sound good, and is a wonderful sound bite for architects to share with uninformed consumers, but should hold no sway with an audience that understands the 2nd law of thermodynamics and the difference between heat and work.  The relevant comparison involves source or primary energy consumption.

5. On-site energy generation (PV) distorts “net-energy” figure

Torcellini and his coauthors define a metric they call “net-source energy” which credits the building for the electric energy generated by the PV Array.  (The "net-energy" was first used by Asst. Professor John Petersen, in his defense of the building in the Oberlin College Alumni Magazine [8]). The benefits of a PV Array are, of course important.  But in this case they totally mask the benefits, if any, of the many other High-Performance Building design features that dominate the building costs.  As I pointed out in an article to in the Oberlin Alumni Magazine, the “net-source energy” of any building can be arbitrarily lowered by adding to it a sufficiently large PV Array [9]! Surely the High-Performance Building Program and this $500/sf building design have more to offer with regard to energy savings than a plan to add PV arrays to conventional buildings!

Evaluation of the High Performance Building design is best achieved by looking at the annual source energy consumption apart from any energy generated by its PV array.  For its most recent year of operation the Lewis Center’s source energy consumption was 119,000 Btu/sf if you include the transformer losses, or 108,000 Btu/sf if you do not.

Note that none of the other five buildings included in Torcellini et al’s study include signficant on-site energy generation.  You could add the $420,000 PV array to any of these five buildings and instantly drop their net-site and net-source energy consumption below those of the Lewis Center.  (In the case of the much larger buildings you might have to add several such PV Arrays.)  What, if anything, would this demonstrate?

If, on the other hand, you already have a cost-effective building whose source energy consumption is significantly lower than a comparable, conventional building, then it is significant to go one step further and add on-site renewable energy generation (say, wind or PV) to cover the source energy requirements of the building.  As noted green building advocate and designer of the Lewis Center’s PV Array, Steven Strong, has said, you don’t make a building green by mounting a PV Array on its roof.  A PV Array is a “green badge” that a building must earn the right to wear [10].

6. Code-compliant Baseline Case contrived to yield high energy use

The final missing information from TJ&C's article is the description of their “baseline code-compliant building.”  Most readers will reasonably assume that the baseline case represents a conventional, reasonable alternative.  That is not the case.  The “baseline case” used by TJ&C was an all-electric building employing air-to-air electric heat pumps (with resistive electric auxiliary heat) with a window-to-wall ratio of 43% to match the actual building.  Yet, having incorporated the heat loss associated with the glazing, they did

“...not take advantage of daylighting for reducing lighting load”

nor did they use overhangs [11].  No self-respecting engineer would design a building in NE Ohio using air-to-air heat pumps! And it is mind-boggling to consider a conventional lighting design for a building with so many windows! TJ&C further inflate baseline enegy consumption by adding the energy for the Lewis Center's Living Machine, a system that is grossly oversized for waste generated by building occupants [2]. (Most readers would guess that a baseline case would employ the city sewer.) There is no guessing what temperature setpoints and schedule they adopted to further inflate energy use for their baseline case.

To no surprise, the 171,000 Btu/sf source energy for this all-electric baseline case is considerably higher than any reasonable conventional building.

7. Steady-state source energy consumption compared to other buildings

What is a reasonable baseline comparison?  In my ASHRAE Transactions paper I suggest two [2].  The first is the average source energy from the 1995 CBCES database for a comparable, conventional, educational building equal to 123,000 Btu/sf.  The second is the average source energy for 33 Oberlin College academic buildings, many of them more than 50 years old (based on 1995-6 data).  That number is 130,000 Btu/sf and, and it does include building transformer losses.

How does the source energy consumption for the last 12 months of building operation compare to these reasonable measures?  The measured source energy ranges from 108,000 to 119,000 Btu/sf depending upon whether or not you include the building transformer losses.  The two baseline cases range from 123,000 to 130,000 Btu/sf.  So, depending on the combinations you use, the Lewis Center’s annual source energy consumption is from 3 to 17% better than the baseline case.

The Lewis Center design and construction cost more than $500/sf, double the cost of a conventional building.  The $420,000 PV Array on its roof only adds $31/sf to this figure – an amount usually judged to be cost ineffective.  It is a good thing to use less energy than a comparable, conventional building.  ASHRAE members should judge whether this is a sensible building design or whether it might have been better to build a conventional building and use the cost savings to pay for several PV Arrays!

8. General Comments

The “Lessons learned” paper by TJ&C doesn’t come close to being a serious, unbiased case study of this building.  It is nothing more and nothing less than marketing literature for the DOE’s High Performance Building Program in which selective data are carefully packaged to yield the most flattering conclusions about these buildings, justifying the hundreds of thousands of tax payer dollars spent on promoting and monitoring these building designs.

In most cases (i.e., the other five buildings in TJ&C’s study) NREL and the building owner have sole access to performance data, design documents, and operational details.  The building owner and NREL actually enter into a contract that includes restrictions on disclosure. There is no opportunity for outsiders to independently evaluate the data or conclusions of the NREL report.  The NREL High Performance Building group typically operates in a monopolistic environment in which their word on building performance stands unchallenged.

In the case of Oberlin College’s Lewis Environmental Center we have, for the first time, the ability to obtain independent energy performance data, design and construction documents, and detailed information regarding building operation and usage.  As an Oberlin College professor I have been given the opportunity to independently monitor building energy performance and HVAC settings for the Lewis Center.  Tenure provides me the protection to make unpopular statements without fear of losing my job.  I doubt very much that employees at the other five buildings in TJ&C’s study enjoy the same protection and independence.  My study of this building has resulted in two, peer-reviewed papers regarding the design and early performance of this building [2] and its photovoltaic array [12].

With this independent ability to study the building we are able to evaluate the claims of Torcellini, Judkoff, and Crawley.  We find the facts they present to be accurate but highly selective.  By carefully choosing which facts to present, choosing not to disclose other information, and constructing a flattering (yet unspecified) comparison case they present conclusions that grossly skew the results to suit their political and economic purposes.  I can only assume these are their usual methods for preparing building energy studies, except that, in other cases, readers have no way to cross-check their results. Let me suggest that the manner in which DOE/NREL management respond to these charges will indicate whether this represents an isolated case or general operating protocol. These are the kinds of half-truths we have come to expect from tobacco and oil company scientists, but I was not aware that these were the methods of DOE/NREL building scientists who feed at the public trough.

I suggest that it is misguided energy policy to be promoting this kind of building design.  The DOE should be focusing on cost-effective technologies that work and can be widely adopted.  The High-Performance Building Program appears to be captivated by all-electric buildings and ground-source heat pumps.  If they focus on source energy rather than site energy they will see these are simply alternate approaches which may or may not be better for society – the answer is in the details.

The justification for an all-electric building is flawed from the beginning.  The architect chose this so that he could provide all the energy needs with a roof-top photovoltaic array.  But since the building has no energy storage, this array must export its excess energy to the grid and the building must import energy when its demand exceeds production.  In other words, at best, the building can become a net-energy exporter on paper – through use of energy debits and credits.  (The building must remain connected to the grid to survive – you cannot cut the supply line.)  Given this, there is no reason not to extend this further and bring in a natural gas supply line.  Such a building can import energy via the natural gas line and export even more of its PV-generated energy to the grid, in this case gaining the factor of 3 site-to-source energy conversion.  A building that imports both electricity and natural gas (or some other form of energy) can become a net-energy exporter just as easily as an all electric building.  This is one of the goals of the Leslie Shao-ming Sun Field Station at Stanford University's Jasper Ridge Biological Preserve.

Fixation on all-electric buildings is bad energy policy and just plain foolish.


[1.] Paul A. Torcellini, Ron Judkoff, and Drury B. Crawley, “Lessons Learned: High-Performance Buildings,” ASHRAE Journal, September. 2004, pp. S4-11.

[2.] J. H. Scofield, "Early energy-performance for a green academic building," ASHRAE Transaction, Vol. 108 Part2, 1214-1230 (2002).

[3.] Shanti Pless and Paul Torcellini, “Energy Performance Evaluation of a Low-Energy Academic Building – Adam Joseph Lewis Center for Environmental Studies, Oberlin College, Oberlin, Ohio,” NREL TP-550-33180, March 2003.

[4.] See Ref. 3, pg. 56.

[5.] K. Burke, D. W. Orr, R. Perkins, A. Tuluca, "Response to 'Early performance of an academic green building,'" ASHRAE Transaction, Vol. 108 Part2, 1227-1229 (2002). Also see J. H. Scofield, "Rebuttal to comments by Burke, Orr, Perkins, and Tuluca", ASHRAE Transaction, Vol. 108 Part2, 1229-1230 (2002).

[6.] Note that my energy figures come from the Oberlin Municiple Power and Light's bi-directional electric meter installed on the primary (utility) side of the building transformer. This meter location allows the utility to charge transformer losses to Oberlin College. The transformer is owned by the College, was included in the building project, and has no other purpose than to serve the building and its associated landscape. NREL's data come from meters installed on the secondary (building) side of the transformer and do not include these transformer losses. Data from the two monitoring systems have been compared in some detail and, after accounting for transformer losses, are in excellent agreement.

[7.] Paul Torcellini, private communication at a meeting to discuss Lewis Center energy monitoring, attended by J. Petersen, L. Evans, and J. Petersen, D. Crawley, and J. Scofield, Oberlin, OH, May 6, 2000.

[8.] John Petersen, "Judging the success of the Environmental Studies Center," Oberlin Alumni Magazine, Summer 2002.

[9.] John H. Scofield, "One scientist's perspective on the Lewis Center," Oberlin Alumni Magazine, Summer 2002.

[10.] Steven Strong, “Solar Electric Buildings,” presented at the NESEA Building Energy 2001 Conference, Tufts University, Mar. 21-24, 2001.

[11.] See Ref. 3, pg. 31-32.

[12.] J. H. Scofield and D. Kaufman, "First year performance for the roof-mounted, 45-kW PV-array on Oberlin College’s Adam Joseph Lewis Center," Proc. of the 29th  IEEE Photovoltaic Specialists Conference, May 20-24, 2002, New Orleans (IEEE, New York, 2002), pp. 1691-1694.