Google may have made headlines when it stated energy costs outweigh server costs in its data centres, but a sobering thought, according to Rob Potts at APC, is only a third of datacentre energy is actually used for computing – up to 70% may be taken up by power, cooling and inefficiency losses
It is estimated that worldwide, datacentres consume some 40,000,000,000 kW/ hours of electricity annually•. Because of the need to provide high levels of redundancy in order to maximise uptime and reduce downtime – the goal of most facility operators – a degree of electrical inefficiency in the sector seems to be an acceptable fact of life. However, by increasing electrical efficiency there is also an opportunity to reduce energy use and therefore operating expenses.
How efficient is your physical layer?
For any device or system, efficiency is simply defined as the fraction of its input (ie. the fuel that makes it ‘go’) converted into the desired useful result, in this case computing. If all datacentres were 100% efficient, then all power supplied to the data centre would be utilised by IT equipment. However, energy is consumed by devices other than the IT load because of the practical requirements of keeping it properly housed, powered, cooled and protected. The devices that comprise network-critical physical infrastructure (NCPI) include those in series with the IT load (such as UPS and transformers) and those in parallel with the load (such as lighting and fans).
In simplistic terms, the more energy can be expended on computing and reduced on non-IT devices, the more efficient the facility.
Can data centre efficiency be improved?
Virtually all of the electricity feeding a datacentre will end up as heat emission. From a facilities point of view, efficiency can be improved in a number of ways including:
• Improve the design of NCPI devices so that they consume less power
• Rightsize NCPI components to the IT load
• Develop new technologies which reduce the power consumed by non-IT devices
On the face of it, option two provides the most immediate solution to meet current data centre challenges. At the same time, better power efficiency of servers is being achieved through the introduction of multi core processor architectures, and improved utilisation of IT layer is being brought about through virtualisation.
Real world options
Before setting out to realise available power savings, some common misconceptions need to be corrected:
• Firstly, the efficiency of a facility is not a constant, for example air conditioning units and UPS are far less efficient at low loads (and, conversely, far more efficient at higher loads)
• Secondly, the typical IT load tends to be significantly less than the design capacity of the NCPI components (due, in part, to conservative ‘nameplate’ rating by IT manufacturers)
• Thirdly, the heat output of the power and cooling NCPI components themselves creates a significant energy burden for the whole system and should be included when analysing overall facility efficiency.
An additional factor affecting the efficiency of facilities is that the IT load itself is not constant but dynamic, both operationally and through inventory changes. For instance, as computing throughput increases, electrical consumption is also increased. Also, over the lifetime of facilities, IT inventory is in a constant state of flux as new generations of equipment replace old. Until recently, every increase in server performance has come complete with an increase in electrical demand.
Efficiency is dynamic
Finding an improved model for data centre efficiency depends on how accurately individual components are modelled. However, the use of a single efficiency value is inadequate for real data centres as the efficiency of components such as the UPS are a function of the IT load.
Therefore when the UPS operates with a light load, efficiency drops off substantially. The losses that occur along this curve fall under three categories: no-load loss, proportional loss, and square-law loss.
No-load losses can represent more than 40% of all losses in a UPS and are by far the largest opportunity for improving UPS efficiency. These losses are independent of load and result from the need to power components like transformers, capacitors, and communication cards.
Proportional losses increase as load increases, as a larger amount of power must be ‘processed’ by various components in its power path. As the load increases on the UPS, the electrical current running through its components increases. This causes losses in the UPS with the square of the current sometimes referred to as ‘I-squared R’ losses or square-law losses. Square-law losses become significant (1 to 4%) at higher UPS loads.
The efficiency of a device can be effectively modelled using these three parameters, and a graphical output of efficiency can then be created for any component, as a function of load – understanding that typical datacentres operate well below their design capacity.
Effects of under-loading
If the efficiency of NCPI components such as UPS and cooling equipment decreases significantly at low loads, any analysis of data centre efficiency must properly represent load as a fraction of design capacity. It is a fact that in the average data centre, power and cooling equipment is routinely operated below rated capacity. There are four reasons for this:
• The data centre load is simply less than the system design capacity, in fact, research shows the average facility operates at 65% below its design value.
• Components have been purposely oversized to provide a safety margin – in order to provide high availability, ‘derating’ components by 10% - 20% is common design practice.
• Components operate with other similar components in an N+1 or 2N configuration to improve reliability or facilitate concurrent maintenance of hot components. However, such configurations have an impact on physical layer components, for example in a 2N system the loading on any single component is at best half of its design capacity.
• Components are oversized to handle load diversity, for example PDUs are routinely oversized between 30% and 100% in order to utilise capacity and overcome issues caused by imbalance between PDU loads.
Effects of power and cooling equipment
Heat generated by power and cooling equipment in the data centre is no different to heat generated by the IT load, and must also be removed by the cooling system. This creates additional work for the cooling system, causing it to be over sized, which in turn creates additional efficiency losses.
An improved model for datacentre efficiency
Armed with this knowledge it is possible to create an improved model and therefore make improved estimates of data centre efficiency. Using typical values for equipment losses, derating, load diversity, oversizing and redundancy, an efficiency curve can be developed.
Efficiency is dramatically decreased at lower loads where many data centres operate, e.g., if a facility only reaches 10% of its design capacity, only 10% of the power delivered to the data centre reaches the IT load. A staggering 90% is lost through inefficiencies in the NCPI layer.
Another way to look at this analysis is to consider its financial implications: at 30% capacity utilisation over 70% of the total electricity cost is caused by NCPI inefficiencies in power and cooling equipment. The primary contributor to data centre electrical costs are no-load losses of infrastructure components, which typically exceed IT load power consumption. Many of the losses are avoidable and analysis using the model can help identify and prioritise opportunities for increasing efficiencies. Based on this and the need to gain a quick return, the best solution is to right size facilities using an adaptable and modular architecture.
•Figures quoted from White Paper #113 “Electrical Efficiency Modelling for Data Centres”.