Why is Commissioning a Data Center Different from Other Facilities?

The American Society of Heating, Refrigeration and Air Conditioning Engineers' (ASHRAE), Guideline 1-1996 defines commissioning as, "...the process of ensuring that systems are designed, installed, functionally tested, and capable of being operated and maintained to perform in conformity with the design intent... Commissioning begins with planning and includes design, construction, start-up, acceptance and training, and can be applied throughout the life of the building."

In addition to ASHRAE, there are myriad other definitions of the commissioning process (Cx). However, they all basically reflect the same thing:

  • Commissioning is a process
  • Commissioning begins in the design phase with the documentation of the Owner's Project Requirements (OPR) and the basis of design
  • Commissioning verifies proper installation
  • Commissioning functionally tests equipment and systems to assure they perform within specifications and meet the design intent
  • Commissioning assures proper training of operational personnel

So then in principle there are no fundamental differences commissioning different types of facilities such as administrative building, manufacturing facilities or so-called mission critical facilities. But as a practical matter there are many different approaches to the commissioning process. In real practice the commissioning process is applied differently to different types of facilities. An office or school's commissioning might primarily be focused on the Building Management System (BMS), mechanical system equipment, and on energy efficiency. A data processing center's commissioning process focuses on the uninterruptible power supply (UPS), redundancy of the electrical and mechanical systems, cooling of IT equipment and system interactions. Some differences result from conflicting commissioning standards. While others are due to the Commissioning Authority's (CxA) individual approach. Sometimes the Cx is driven by the owner's expectations.

An added complication is that in electrically intensive facilities there is the overlap of electrical acceptance testing requirements, as defined by the InterNational Electrical Testing Association (NETA), with those of the commissioning process. NETA's Acceptance Testing Specifications for Electric Power Distribution Equipment and Systems says, "The purpose of these specifications is to assure that all tested electrical equipment and systems supplied by either contractor or owner are operational and within applicable standards and manufacturer's tolerances and that equipment and systems are installed in accordance with design specifications. As you can see there is an overlap with the Commissioning Authority's scope of work. These conflicts must be resolved during the construction planning process.

The most compelling concern for the owners of mission critical facilities is reliability and availability. These apply equally to the mechanical and electrical systems. High levels of reliability and availability are achieved with redundant equipment and systems. Over the past five years the data processing world has seen the advent of the "high density" data center. The high density data center is the result of the continuing compaction of computing equipment. One effect of this compaction is the production of high heat loads in confined spaces. As a result, much of the engineering focus has moved from electrical systems to mechanical. During the design process most data processing facilities perform computational fluid dynamic (CFD) modeling of air pressure, flow and temperature within the computing space.

It is the job of the Commissioning Authority to know and understand the designed reliability of systems and how the IT equipment heat is removed in order to develop a series of tests that will verify these systems meet the project requirements.

Most publications say a technically complex facility should command a commissioning cost which is ± 1.5% the total cost of construction. Experience tells us that owners are not willing to spend that much. The net result is that the Commissioning Authority must focus of the most critical items in the facility and cannot test all features and interactions. Our specialty is data process centers. Consequently our commissioning process focuses disproportionately on reliability and cooling, with a lesser emphasis on energy efficiency.

Three areas we focus on are:

  1. The ability of the electrical system to provide power to the computer loads in all normal and emergency modes within the 20 ms criterion established by the Information Technology Industry Council ITI (CBEMA) Curve which describes an AC input voltage envelope which typically can be tolerated (no interruption in function) by most Information Technology Equipment (ITE).
  2. The ability of electrical and mechanical systems to support the computing functions while sustaining faults and/or failures.
  3. The ability to effectively remove the heat generated by the computing equipment. Effectiveness is measured by the ability to remove the total heat load generated by the computing equipment with an absence of hot spotting or recirculation.

The ability of the electrical systems is maintain power within the 20 ms window is accomplished by the application of uninterruptable power supplies (UPS's), automatic static transfer switches (ASTS's), engine-generators and other sophisticated electrical and electronic equipment.

The Commissioning Authority must be aware of the performance specifications for electrical specified. The CxA then develops tests to verify performance pursuant to equipment specifications. This requires sensitive instrumentation capable of capturing sub-cycle electrical events, as shown in below Figure.

Second, we will look at heat removal from the data center floor. Testing the heat removal capabilities of a data center requires performing a "heat run" test. Usually the heat run test is a subset of integrated systems testing.

A "heat run" test is the application of the full design load on the raised floor by using resistive load banks, as represented in below Figures. The load banks are connected to the uniterruptible power supply (UPS) system and placed to approximate the computer loads, as close as possible. Temperature and humidity recorders are placed at representative locations throughout the data center. The first task is to determine if the room temperature and humidity stablizes. If so, where? Do they meet design requirments?

Next, air conditoning units are shut down, one-at-a-time, to refect redundancy levels. For example, if the redudancy of computer room air conditioning units (CRAC) is one out of four, then one out of four units is shut down until every combination of on/off units is tested. While recording temperature and humidity. See representative temperature/humidity chart in Figure below.

The integrated systems test (IST); tests the entire facility under design conditions including normal, emergency and fault modes. The IST is the final area of focus in commissioning data processing centers. The CxA directs the interruption of normal electrical service to the facility. The following actions take place:

  • UPS continues to power the load banks
  • Mechanical systems shut down on loss of power
  • Standby engine/generators start
  • Power is restored to the facility
  • Mechanical systems restart on generator power

It is critical that the electrical system supplies uninterrupted power to the IT equipment and that mechanical systems have the ability to return temperature and humidity to design setpoints after this transition. This is fundamental to the successful operation of a data center.

Temperature and humidity readings are compared to the results of the computational fluid dynamic model to verify if the computing area meets the design criteria.

  • Then the normal electrical service is restored, and the reverse transition takes place.

Additionally, all operational sequences, such as lead-lag or sequencing, are tested at this time. While these are not a comprehensive list of tests that are performed during the IST, they are representative of the issues at hand.

In summary the commissioning process of a data center is similar, yet different from other facilities. The similarities are that the process is intended to verify that the installed systems meet the design intent and basis of design. The differences are in the focus. The data center focus is on proving the electrical systems operate within the electrical window defined for computer power supplies and that the cooling requirements of the computing equipment can be meet. The commissioning agent must know and understand all of the reliability and redundancy requirements and must be able to fashion tests specific to those elements to verify they are being met. All normal operation modes, first-order-of-magnitude failures, and maintenance modes must be proven. He must fashion tests that represent the computer cooling load and test all operational modes and redundancies to assure the successful operation of the facility.

The unique requirements make the mission critical commissioning agent's job both extremely challenging and extremely rewarding.

REFERENCES:

  • ASHRAE Guideline 1-1996, the HVAC Commissioning Process.
  • InterNational Electrical Testing Association 2003, Acceptance Testing Specifications for Electrical Power Distribution Equipment and Systems.
Our Clients
CDCDG was a pleasure to work with on the evaluation of our 131,000 square foot data center.
CDCDG was a tremendous help to us in figuring out our long term data center requirements.
CDCDG has been a real asset to us in helping us choose the location for our backup data center.
The commissioning process was effortless thanks to CDCDG.
From concept to commissioning CDCDG was an invaluable partner to us.
CDCDG's thorough evaluations of our statewide data center initiative has helped us to be a success.
It's been a pleasure working with you on another successful data center project.
Your support of our worldwide data center initiative has helped us to our success.
Your design oversight saved us thousands of dollars on our data center build.
We could not have done it without you.
 
News & Updates
  • April 2012: Ron Hughes, Founder of CDCDG steps down to take a position appointed by Governor... » More
  • March 2012: State of CA /CDCDG are nominated for GEIT Award for... » More
  • March 2012: KIO Networks / CDCDG are nominated for GEIT Award for Cutting Edge Data Center Design Category... » More
  • Feb 2012: CDCDG completes commissioning Porto Segura's primary data center... » More
» More