From free cooling to liquid cooling and heat reuse

• Cloud
• CSR

Share on Twitter : From free cooling to liquid cooling and heat reuse (New window) Share on Linkedin : From free cooling to liquid cooling and heat reuse (New window)

The challenge: dealing with equipment’s heat

During their operation, telecom and information technology equipment convert most of the electric energy received into heat. This heat must be controlled to avoid excessive temperature in the technical room, and to allow a safe and permanent operation of the equipment.

Currently, equipments are not optimized as energy consumption remains constant, whatever the network traffic observed. Servers CPUs are improving with standby energy consumption lowered. Selecting energy efficient ITN equipment remains the first option to lower the whole network’s energy consumption. Therefore, targets must be set in RFP.

Further energy reduction can be achieved by optimizing Virtual Machine’s distributions, within data centers (too often, servers are underused).

Energy hungry cooling systems are used to displace this heat from inside the building to outside. They represent an important part of the whole energy consumption, up to 60% in some data centres. Thus, reducing cooling’s energy consumption constitutes a main lever of improvement.

Orange research focused on two approaches. The first approach consists in designing equipments that can operate at higher temperatures to reduce the need for cooling, thereby lowering energy consumption and improving reliability. It is also a way to cope with the higher temperature resulting from climate change.

The second approach explores innovative concept for cooling systems. As classical air conditioning systems exhibit huge energy consumptions and use refrigerants with ozone depletion potentials, their suppression or at least the limitation of their use is essential.

20 years of research and standardization works on cooling systems

Orange started 20 years ago exploring the concept of free cooling, which consists in using external fresh air instead of air conditioning systems to cool telecommunication equipment rooms, when its temperature and humidity are within ranges defined by European standards.

Orange researchers raised the issue of heat control through the whole European ecosystem from scientific area and research programs to standardization and implementation. They were strongly active in relevant standardization bodies, notably to require that servers and telecom equipment are working properly in the same range of temperature and humidity, to reduce energy consumption. In parallel, Orange energy researchers also contributed to various research projects with other European stakeholders, including vendors, and public entities on energy efficiency, like Opera net 1 and 2 and SooGreen, that led to efficient cooling solutions for central offices and mobile networks.

Concretely speaking, the following tasks were performed:

• Laboratory experimentations alongside with scientific predictive calculations – published in specialized papers – leading to scientific conclusions regarding the whole design of the cooling solution, and to ways of optimizing their settings;
• Secured proof of concepts made on the field, leading to settings adaptation and improvement where needed, in partnership with the manufacturer of the solution;
• Collaboration with a French start-up that developed a new equipment to implement Orange’s improved ventilation patent;
• Setting up a complete monitoring system internally, gathering key data to gradually optimize the solution;
• A meticulous assessment of the best location for the new Orange site as free cooling system savings are directly linked to the external temperature, while taking into account its connection with Orange backhaul and with energy network, and avoiding area with seismic risks.

Achieving 50% reduction of cooling energy through Orange innovative solution for its datacenter

Orange research effort resulted in both

• widening equipment climatic ranges, from [20-24°C] temperature ranges and [45-65%] relative humidity ranges, to [24-28°C] and [30-75%], which leads to 33% decrease of the energy consumption;
• and replacing the air conditioning systems by intelligent free cooling systems which brings a 90% reduction in cooling energy consumption.

Concretely, Orange implemented the free cooling system in 2013 in its new big data center, 5000m², in Normandie, which reached a Power Usage Effectiveness of 1.3, to be compared with 1.8 for Orange’s other data centers in France. It represents more than a 50% reduction of cooling energy consumption. Orange has also implemented that technology in other data centers in France, Belgium and in MEA, like Cameroun.

Immersion Cooling and liquid cooling, alternatives to free cooling

As thermal properties of liquid are more interesting compared with air and provide better heat exchanges, immersion cooling using a dielectric liquid could be another solution. In these solutions servers or electronic devices are submerged in a liquid into direct contact with hot components, liquid being refreshed by outside air temperature. Outside temperature could be higher than 40°C, that means that immersion cooling could be used in most of countries where Orange is present, without energy hungry chiller. Experimental works have been carried out by Orange researchers with two fluids, single (Oil) or two phases. Single phase fluids are now well known, and some suppliers can provide materials. Two phases liquid cooling shows better performances, but some issues are remaining, this technology is still in research works.

Using immersion cooling solutions allows the reuse of heat more efficiently than using air due to thermal properties of fluid.

Direct to chip liquid cooling

Direct to chip liquid cooling consists in using liquid cooled plate to control temperatures of chips, directly on electronic boards.

Overall energy efficiency can be compared with immersion cooling. It also provides the same heat reuse opportunities.

Wasted heat recovery

Currently, heat dissipated by equipments is lost. Air cooling doesn’t allow an efficient and cheap recovery: Air temperature are too low, heat transfer coefficients inefficient, and an extra air to water heat exchanger is needed.

On the contrary, liquid cooling brings naturally heat reuse capabilities as liquid temperature is compliant with offices heating, greenhouses heating, etc.

AI to control cooling

Current complex cooling installations consists in several actuators with a few sensors, and basic control laws. It can be demonstrated that existing cooling installations are still far to reach optimal control on real installations.

Adding more sensors (or using IT equipment embedded sensors) and developing relevant AI algorithms could lead to huge energy savings.

Indeed, deep reinforcement learning techniques give the opportunity to add a double constraint on systems:

• ensuring that climatic ranges are compliant
• as well as minimizing energy consumption

Researchers have been developing and validating such algorithms in Orange labs.

How was this successful innovation made possible?

Orange long term R&D perspective has been supported by strong and long-lasting cooperation within different Orange entities (R&D, operational units) throughout the whole process, notably to evaluate risks, and to adopt and implement a bold move.

A test and learn approach allowed a progressive field implementation.

• Strong involvement in various European projects and ecosystem (academic, standardization bodies, R&D EU programs, etc.), essential in terms of scale;
• Public authorities’ support, allowing enhanced cooperation between various actors and stable financing;
• Strong alignment of Orange commitment in terms of green transition with EU and national objectives.

Next steps

The free cooling solution is now well known and can be easily used where external climatic conditions and equipment are compatible.

However, it is not a ‘one size fits all’ answer as it is dependent on the external temperature. Orange is therefore working on other solutions like liquid or immersion cooling. They can be tested, promoted, and co-developed with other stakeholders in Orange experimental Data Center, named Padus Lab.

All these explorations are also onboarded in Orange Project Portfolio for the Important Project of Common European Interest (IPCEI) on Cloud Infrastructure and Services that is expected to start soon.

To support the greening of the digital sector and the implementation of innovative solutions, European private and public entities have to work together to achieve sufficient scale in this global issue. Further works are needed, especially in terms of standardization to improve common metrics, and through very large European projects such as IPCEI to speed up research and implementation of innovative solutions.