Server Inlet Temperature and Humidity Adjustments

Description

Temperature and Humidity Issues

• In 2008, American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) expanded the recommended temperature range at the inlet of the server from 68°F to 77°F (the 2004 level) to 65°F to 80°F. However, many data centers traditionally have set their temperatures as low as 55°F. As a result, many data centers can save energy simply by raising the thermostat.
• Until very recently, most data center managers believed it was important to tightly control humidity in data centers. Where humidity is too low, it can lead to electrostatic discharge (ESD) failures. Where humidity is too high, water can condense inside servers, causing electrical shorts that trip circuit breakers, damage equipment, or harm electrical circuits within the data center. High humidity and condensation can also cause rust and corrosion in servers, leading to machine failures.
• Because data center CRAC units typically use a great deal of energy to maintain required humidity levels, relaxing those requirements (expanding allowable humidity limits) can save energy. In 2008, ASHRAE expanded the recommended humidity ranges from the 2004 levels. (See table below.)

• A typical data center uses a great deal of energy maintaining humidity through multiple CRAC units. Each CRAC unit has the ability to humidify, de-humidify, heat and cool. When humidity gets too low, a typical CRAC unit will raise the humidity by using inefficient infrared or steam canister humidifiers.²⁸ These humidifiers can use an order of magnitude more energy than adiabatic humidifiers (e.g., an ultrasonic humidifier) and can also raise the temperature of the data center because they produce heat to generate steam. When the humidity gets too high, the CRAC unit will subcool the air to remove moisture and then reheat (typically using electric resistance heat) to maintain space temperature. Additionally, each CRAC unit typically bases its temperature and humidity settings on what it measures in the (warm) return air. Given the proximity of the CRAC units' exhausts and intakes, it is common for one CRAC to attempt to cool or humidify the air while another CRAC tries to dehumidify and/or reheat the air.
• There is a growing understanding that even 2008 ASHRAE recommended temperature and humidity ranges are too restrictive:
  • A recent ASHRAE journal article found that high humidity is rarely an issue in most data centers and concluded that "it is difficult to make a case for actively controlling humidity in data centers."²⁹ Typically, the temperature of IT equipment is significantly higher than that of the cooling coils' operating dew-point. Furthermore, most IT equipment is rated for operation up to 80% RH. The study states that humidification appears to be unnecessary if you follow best practices for electrostatic discharge: use IT equipment rated and tested for ESD conformance with IEC61000-4-2, and, where personnel handle electronic circuit boards and components, use personnel grounding procedures.
  • In August 2008, Intel conducted a 10-month study to assess the effectiveness of using only outside air to cool a data center. The temperature range was 64°F to 92°F. Humidity varied from 4% to over 90% and changed rapidly at times. No increase in server failure was observed³⁰.
• Data center equipment will operate safely at temperatures and humidity levels beyond the recommended ASHRAE ranges (see Figure 12). Consider these operating ranges for the following server models:
  • Sun Blade ( 4°F to 90°F)
  • Dell Blade (50°F to 95°F)
  • IBM Blade (50°F to 95°F)
  • NetAppStorage (50°F to 104°F)³¹

How to Adjust Temperature and Humidity

• Add a centralized control system to reduce or stop dueling CRAC units, and
• Change the humidity set points to a range of 30% to 70% relative humidity that conforms to actual equipment requirements and eliminates excessive humidification and/or dehumidification. Often, only one CRAC is needed to control humidity, while the others can be used as a backup, and set up with a much wider range of high-low humidity set points.
• Ensure that the HVAC unit uses the server intake temperature, not the return air temperature, as the set point temperature. (Return air refers to warm air entering the CRAC for cooling.)
• Raise the server intake temperature so that it is around 70°F at the lowest server in the rack and no more than 77°F at the highest server. Inlet temperature increase should be performed gradually; the first step should be to broaden the allowable thermostat temperature ranges. If this is successful, temperature should be gradually raised with careful metering of potential or existing hot spots.
• Consider using an efficient adiabatic humidification system that not only humidifies the air but also cools it at the same time.
• Install a series of temperature, humidity, power and pressure sensors throughout the data center. Use them for monitoring while adjusting temperature and humidity. Typical locations and measurements include:
   
  • Racks: equipment power usage, cold aisle temperature, hot aisle temperature, humidity, bypass airflow, recirculation airflow percentage,
  • Subfloor: pressure differentials within plenums,
  • CRAC/CRAH: supply temperature, return temperature, humidity (RH and dew point), and
  • Chilled water flow: ultrasonic flow meters measure water temperatures and flow rates.
  Kaiser Permanente set up a sophisticated temperature and humidity monitoring network to optimize their cooling efficiency in their ENERGY STAR certified data center.
• Utilize communication gateway devices to pull data points into a building management system software interface. If remote infrastructure is not metered, in most cases communication cards can be installed to facilitate system integration.
• Note that certain types of equipment can supply this information directly. For example, ENERGY STAR-qualified servers are required to provide data on input power consumption in watts and inlet air temperature. Rack-mounted power distribution units can provide real-time load monitoring of connected equipment to help maximize energy efficiency; this is especially important in a virtualized environment when workloads shift from location to location.

Savings and Costs

• Data centers can save 4% to 5% in energy costs for every 1°F increase in server inlet temperature.³²
• A CIO magazine article claims that data centers can save up to 50% of energy costs by turning off humidification controls on CRAC units and investing in an ultrasonic humidification unit.³³
• According to one vendor, ultrasonic humidifiers use only 25 watts of electricity for each pound of mist generated versus 380 watts for electric steam generators/boilers. Annual energy savings for a 100-pound-per-hour system would be $7,100 (based on 2,500 operating hours per year, at 8 cents per kWh).³⁴
• BNY Mellon, an ENERGY STAR certified data center, raised the temperature of their supply air from 72 to 78 degrees F, which allowed them to increase their chilled water temperature from 44 to 47 degrees F, lowering their cooling costs. In addition, BNY Mellon changed their humidification set points from relative humidity (which varies by temperature) to dew point (an absolute humidity value) to reduce humidification run-time from 80% to 20% of the time.
• RagingWire, an ENERGY STAR certified data center, raised their chilled water temperature from 50 to 60 degrees F after hot aisle containment. An extensive wireless monitoring network was used to ensure appropriate server inlet temperatures were maintained. The project paid for itself in a year.
• EBay recently completed a  case study (PDF, 578 KB) with EPA ENERGY STAR that showed the following paybacks for ultrasonic humidification and variable speed drives retrofits at their Phoenix data center:

  EBay Data Center Retrofit Measure Paybacks

  Measure	Payback without APS Incentive (years)	Payback with APS Incentive (years)
  Ultrasonic Humidification	1.9	0.5
  Variable Speed Drives	2.6	1.6

• Although adjusting inlet temperature and humidity is relatively inexpensive, metering temperature, humidity and power changes across the data center can be expensive. Contact a firm specializing in data center metering to get a cost estimate.
• According to one study, an ultrasonic humidification unit has a payback of two years.³⁵
• The Green Grid recently completed a  comprehensive analysis (PDF, 2 KB) of return on investment and power usage effectiveness for data center efficiency upgrades. Payback for adding baffles and blanking panels, repositioning temperature/humidity sensors, and adjusting temperature setpoints was 29 months on capital costs alone.
• Google’s Green Data Centers: Network POP Case Study (PDF, 4 KB) examines a small data center’s efforts at efficiency with optimized air vent tiles, temperature and humidity adjustments, cold aisle containment, and CRAC air return extensions that had an ROI of less than one year.

Considerations

• Raising inlet temperature and humidity may lead to uncomfortable working conditions. For example, at the higher end of ASHRAE's operating range in the cold aisle (80.6°F), hot aisle temperatures would be roughly 105°F to 110°F.
• Higher inlet temperatures can cause the internal fans in servers to increase their speed in order to cool the servers—thus raising energy costs. One source states that this happens at temperatures as low as 77°F.
• Data centers with air-side economizers and/or variable speed fan drives in their CRAC units stand to benefit the most from raising set point temperatures. That's because higher set point temperatures increase the amount of time that 1) outdoor air can be used for cooling, and 2) CRAC units can run on reduced fan speeds.
• Ultrasonic humidifiers should only be used with de-ionized water. One manufacturer mentions a de-ionizer system with resin cartridges that need to be replaced every two months.
• Adjusting temperatures may lead to data center hot spots.