Reliability Studies - What should a client expect?
December 2001, Power Quality Assurance Magazine
Your critical facility, be it a data center, central office or production control center fails! Management, suddenly aware of the cost associated with such a failure, decides to bring in a consultant to review the facility, identify areas of concern and make recommendation to fix the problems. As the owner/client, what should you expect?
Reliability studies, single point failure studies, site reviews, whatever you choose to call them are invaluable in identifying and solving problems with facilities before they cause an outage. A quality study will document your existing systems, review their maintainability and expandability, identify areas of concern, assign some sort of priority to the problems found, suggest appropriate alternatives, make recommendations for solutions, define the risk and the level of downtime associated with each recommendation, provide a rough order of magnitude cost estimate and a life cycle cost analysis for the solutions identified and summarize all of this in a short and easily understood executive summary.
The documentation of your existing systems should be the starting point for any reliability study. Up to date as-built drawings should either be available for the consultants review or should be recreated. Many times, the problems being experienced by a facility become pretty obvious when reviewing the drawings. The drawings should be CAD based so that changes can be documented and the drawings updated.
The scope of the work is always an issue in a reliability study. Do you need a review of the facility as a whole, or just certain systems. Many companies limit their review to electrical and mechanical systems. Again, it depends upon the goals of your study. If the goal is no downtime for any system, at any time, then the scope of the study must be more global. If, the goal of the study is limited to one or two problem systems, then a more limited study might be more appropriate.
The maintainability and expandability of the systems being reviewed should be addressed. All critical components of a facility should be designed for concurrent operations and maintenance. Anything that can't be maintained without a shutdown is a serious vulnerability in a 24/7 facility. A modular approach to designing critical systems is standard for most modern facilities. However that wasn't always true of designs completed just a few years ago. The ability to add a chiller or a cooling tower without shutting down the facility is a critical part of a 24 hours a day/7 days a week operation.
One of the big differences in the studies I've read is the level of detail. Some studies document every aspect of a facility, whether they are pertinent to the study or not. They go into great detail about items that have nothing to do with your areas of concern. For example, I've seen reports that contained thermographic profiles of underfloor airflows for data centers that weren't having a problem with cooling. As the owner, you want the best bang for your buck. Make certain that the level of detail provided is commensurate with the goals of the study and your areas of concern.
Another decision that must be made up front is how far into the future do you wish the study to address. If you plan to operate the facility indefinitely, then it makes sense to address issues that might not crop up for 8 to 10 years. If however, you plan to replace the facility in the next 3 to 5 years, then such a long term approach might not be appropriate.
Life cycle costing should be included for any large capitol improvements that are recommended. For example, I've seen studies that recommended the installation of numerous variable frequency drives without any sort cost benefit analysis. While I don't doubt that in many cases, they will improve the efficiency of a system, as an owner, I would like to know if the payback is two years or twenty.
A well written executive summary should explain in non-technical terms the findings of the study. Too many executive summaries are written in terms that might be easily understood by an engineer, but are unfathomable by a non-engineer. The executive summary should be the only part of the report that anyone should have to read to understand the findings and the recommendations of the report.
The recommendations should be as specific as possible and should include whatever drawings are necessary to explain them fully. All of the recommendations should be achievable. Solutions that can't be implemented are more appropriately presented as an alternative. The executive summary should also include a cost estimate that gives a rough order of magnitude cost estimate for the recommended changes.
I'm often asked what a reliability study should cost. Obviously, the cost is dependent upon the scope of the study and the size and complexity of the facility. It is also dependent upon the type of documentation that is available for the facility. Recreating as-builts is time consuming and expensive. Another variable is the involvement of the consultant after the study is completed. Some clients end the relationship once the report is completed. Others involve the consultant in the design and implementation phase. Those decisions can have a significant impact on the cost of the study.
I've looked at reliability studies ranging in cost from $10,000 to $200,000. Cost is not necessarily an indication of quality. In fact, some of the best studies I've seen have not been the most costly. However, you are paying for very specialized expertise and it doesn't come cheaply. In comparison to the mutli-million dollar cost of the facilities that are being reviewed and the cost of an outage, reliability studies are a bargain.
I hope that I've helped to clarify what you can expect from a reliability study. A level of detail commensurate with your needs, a global versus a specific review, a long term versus a short term approach to the study, a clear and concise executive summary and the cost associated with each of the recommended solutions. There are many quality firms out there that can provide you with a study that will meet these expectations. It is up to you however, to see that the recommendations are implemented. Too many reliability studies are filed away without any action, only to be reopened when the next failure occurs.