ANALYSIS OF COOLING SYSTEM EFFICIENCY Текст научной статьи по специальности «Медицинские технологии»

DOI: 10.23968/2500-0055-2022-7-4-49-59

ANALYSIS OF COOLING SYSTEM EFFICIENCY

Yuri Dmitriev

Saint Petersburg State University of Architecture and Civil Engineering Vtoraja Krasnoarmeyskaya st., 4, 190005, Saint Petersburg, Russia

E-mail: [email protected] Abstract

Introduction: The main task of cooling systems in the machine room of a data center is to maintain optimal air parameters at the inlet to IT equipment in accordance with the applicable regulatory documents. If the value of air temperature at the inlet to IT equipment exceeds the optimal value, then so-called hot spots occur — the operation of IT equipment in such places can lead to its overheating and failure. The main reason for the occurrence of hot spots is the violation of air circulation between the hot and cold aisles. Purpose of the study: We aimed to analyze the influence of different methods of hot aisle isolation on the efficiency of IT equipment cooling. Methods: We performed modeling of different methods of hot aisle isolation at different capacity values of IT equipment in the STAR-CCM+ program. Result: As a result of studying temperature and air conditions in a data center, we found the most rational way of isolating the hot aisle — top horizontal containments with vertical blind panels in the free rack space.

Keywords

data center, ambient temperature, cold aisle, hot aisle, computational fluid dynamics (CFD).

Introduction

In recent decades, row arrangement of equipment racks in data centers have been widely used. In such an arrangement, rows of racks are divided into so-called hot and cold aisles. IT equipment is installed in racks in such a way that heat is removed to the hot aisle, and cooled air is supplied from the cold aisle.

IT equipment in a data center is cooled using the following systems:

1) surface air coolers and cooling units as well as monoblock air conditioners with a remote condenser;

2) evaporative cooling;

3) with ventilation air.

Cooling with ambient ventilation air makes it possible to significantly save on power consumption of the data center cooling system. However, this system can be used at outdoor temperature tout < 27°C. In cases when outdoor temperature is higher than this limit value (tout > 27°C) set by the effective standard (ASHRAE, 2016), additional air cooling is required.

The efficiency of cooling with ventilation air can be improved by using evaporative cooling. This method is based on adiabatic cooling of ambient air due to mist spraying or the use of wetted pads made of various materials. It should be noted that in winter an evaporative cooling system can serve as a humidifier to increase relative humidity to the optimal values (ASHRAE, 2016) without the need to use steam humidifiers characterized by high power consumption.

Air cooling in surface air coolers is carried out with the use of freon coolant or cooling media such as water or non-freezing liquid. When cooling

media are used, chilled water or non-freezing liquid from a steam compression (chiller) or absorption refrigerating machines is fed to an air cooler. Air cooling with freon coolant takes place in an air cooler located in a monoblock air conditioner.

Depending on arrangement in a data center, cooling system equipment can be classified into central and local. The first group includes central air conditioners using ambient air and evaporative cooling (Figs. 1 and 2). The second group includes surface air coolers that can be installed:

1) in monoblock air conditioners located either around the perimeter of a machine room (Fig. 3) or in rows of racks (Fig. 4);

2) in the machine room space (Fig. 5);

3) in each rack (Fig. 6).

The choice of a particular data center cooling solution depends on the feasibility study. During that study, the following should be taken into account: the climatic conditions of the region where the data center is constructed, the cost of available energy resources, the impact on the environment, and the capacity of IT equipment placed in racks. Today, racks with a capacity from 5 to 15 kW are widespread (Timonin, 2018). Systems using surface air coolers installed in monoblock air conditioners have become very popular in cooling IT equipment in such racks.

Many researchers performed comparative analysis of monoblock air conditioners located around the perimeter of the machine room and monoblock air conditioners located in rows of racks in a data center. As a result of comparative analysis performed by some researchers (Abbas

1 2 3 4 5

3

2

5

Fig. 1. Diagram of a data center air cooling system using

central air conditioners: 1 — racks with IT equipment; 2 — cold aisle; 3 — hot aisle; 4 — central air conditioner; 5 — drop separator; 6 — air inlet duct; 7 — suspended ceiling; CAC 1, CAC 2 — central air conditioner

et al., 2021; Priyadumkol and Kittichaikarn, 2014), it was revealed that when air conditioners are located around the perimeter of the machine room, an uneven distribution of air throughout the height of the racks is observed. But when air conditioners are located in rows of racks, air is distributed evenly. As a result of another comparative analysis (Cho and Woo, 2020), it was revealed that the intensity

Fig. 2. Diagram of a data center air cooling system using a central air conditioner: 1 — air inlet duct; 2 — coarse and fine filters; 3 — evaporative cooling system;

4 — drop separator; 5 — fans (axial or radial)

of air circulation between the aisles is higher when air conditioners are located around the perimeter of the machine room. It was also revealed that as a result of air circulation between the aisles, the heating of supply air in the cold aisle is AT = 4.6°C when air conditioners are located in rows of racks and AT = 8.3°C when air conditioners are located around the perimeter of the machine room. Thus, the arrangement of air conditioners in rows of racks is at least 50% more efficient than the arrangement around the perimeter of the machine room.

To increase the efficiency of cooling systems in a data center, hot and cold aisles are isolated using containment (Fig. 7) and blind panels in the rack space free from IT equipment (Fig. 8). Such researchers as Nada et al. (2016) and Zhang et al. (2017) addressed the issue of aisle isolation with the use of containments. They established that the use of containments to isolate the cold aisle reduces air circulation between the aisles and decreases the temperature of air supplied to the racks by 5°C compared to the option without containments. Cho et al. (2021) as well as Niemann et al. (2008) found out that hot aisle isolation allows for 24.9% improvement in the temperature conditions in the machine room and 43% reduction in the annual power consumption of the data center compared to cold aisle isolation. Other researchers studied isolation with blind panels in the rack space free of IT equipment (Rasmussen, 2012; Tatchell-Evans et al., 2017). They established that the use of blind panels for isolation reduces air circulation in the rack and decreases the temperature of air supplied to cool IT equipment by 12°C.

In continuation of the above studies, we performed comparative analysis for the efficiency of a cooling

7 4 6

S

lO

csi

A "A N » ■

•

i CAC 1 4

6

1

Fig. 3. Diagram of an air cooling system using air conditioners located around the perimeter of the data center machine room. The blue arrows indicate the air flow entering the rack, the red arrows indicate the air flow leaving the rack with IT equipment

hot aisle

Fig. 4. Diagram of a data center air cooling system using air conditioners located in rows of racks with IT equipment. The blue arrows indicate the air flow entering the rack, the red arrows indicate the air flow leaving the rack with IT equipment

Fig. 5. Diagram of a data center air cooling system using a surface air cooler located in the machine room space. The blue arrows indicate the air flow entering the rack, the red arrows indicate the air flow leaving the rack with IT equipment

system in a data center with different options of hot aisle isolation (using containments and blind panels in the free rack space) at different values of IT equipment capacity. Air conditioners located inside the rows of racks were used as cooling system equipment. To perform analysis, we conducted numerical studies of cooling with regard to IT equipment with a capacity of 4.7 and 9.5 kW in each rack for the following methods of hot aisle isolation:

A) without top containment (hereinafter — containment) and without blind panels in the free rack space (hereinafter — panels);

B) without containment but with panels;

C) with containment but without panels;

D) with containment and panels.

Rack

Fans

m

Air cooler

Fans

Cold aisle

>Server

Server

.Server

Server

.Server

Server

Server

Server

Server

Server

Fans

Server

Server

Server

Hot aisle

Air cooler

Fig. 6. Diagram of an air cooling system using air coolers located in rows of racks with IT equipment. The blue arrows indicate the air flow entering the rack, the red arrows indicate the air flow leaving the rack with IT equipment

a)

racks

L

J L

t_l U J

b) aisle containment

1

Fig. 7. An example of using containments to isolate the aisles between the racks: a — without containments, b — with containments

Materials and methods

We chose a real data center machine room in St. Petersburg (Fig. 9) using row arrangement or racks as an object for analysis in the STAR-CCM+ program. In each rack of the machine room, one IT equipment is placed (chassis with blade servers) (Almoli, 2013). The air conditioners are located between the rows of racks and are equipped with an

Fig. 8. An example of using blind panels in the racks with IT equipment. The blue arrows indicate the air flow entering the rack, the red arrows indicate the air flow leaving the rack with IT equipment: a — without blind panels in the racks b — with blind panels in the racks

Fig. 9. Layout of the machine room in the data center in St. Petersburg. Designations: R1-R24 — racks with IT equipment; C — air conditioners

inverter compressor and fans with EC motors (Vertiv, 2022).

Tables 1 and 2 present the initial data for model construction and calculation in the STAR-CCM+ program.

The values of IT equipment capacity (4.7 kW) and fan capacity (853 m3/h) in each rack are taken in accordance with experimental data that correspond to 10% of the computational load of this IT equipment in the data center machine room (Ponomarev et al., 2021). The maximum estimated capacity of this IT

Table 1. Initial data for model construction in the STAR-CCM+ program

Machine room dimensions (LxWxH), m 10.5 x 7.25 x 3.3

Number of racks with IT equipment, pcs. 24

Brand of IT equipment Huawei E9000 chassis with Huawei CH242 V3 blade servers

Number of air conditioners, pcs. 12

Brand of air conditioners Liebert CRV CR021RA

Supply air temperature Tsup, °C (ASHRAE, 2016) 20

equipment in each rack is 9.5 kW (Ponomarev et al., 2021). The capacity of fans of IT equipment in each rack for the capacity of IT equipment = 9.5 kW can be obtained by the following equation:

L-p-c-AT

ß = ■

3.6

W,

(1)

where L — capacity of fans of IT equipment, m3/h; p — air density, kg/m3; c — specific heat capacity of air, kJ/(kgK); AT — maximum temperature diffe rence of incoming and outgoing air in IT equipment, K AT = 20 K (Huawei, 2019).

The values of air capacity and cooling capacity of the air conditioners are taken to be equal to the needs of IT equipment in the racks.

To evaluate the modeling results, dimensionless indices RCL^U and KT are used.

HIGH 1

The RCL

index was first introduced

'HIGH ...wi by

Herrlin (2007) and is used to estimate the average air temperature at the inlet to IT equipment in racks, comparing it with the maximum permissible T = 32 °C and recommended T = 27 °C

max-perm max-rec

temperatures (ASHRAE, 2016). If the values of the average air temperature at the inlet to IT equipment in the racks exceed the maximum recommended then so-called hot

value,

spots occur —

Table 2. Initial data for model calculation in the STAR-CCM+ program

Capacity of IT equipment in each rack, kW Capacity of fans of IT equipment in each rack, m3/h Cooling capacity of one air conditioner, kW Air capacity of one air conditioner, m3/h

4.7 853 9.4 1706

9.5 1424 19 2848

the operation of IT equipment in such places can lead to its overheating and failure.

RCI

1 —

^¿=1 (nc i Tm

HIGH k

n X(^max-perm ^max-rec j

x100 %, (2)

where:

T . — average temperature of incoming air

inc I or d

entering IT equipment in the rack, °C;

T — maximum recommended temperature

max-rec ~

(T = 27 °C) of incoming air entering IT equipment

max-rec

in the racks, °C;

Tmax-perm — maximum permissible temperature (T = 32 °C) of incoming air entering IT

max-perm

equipment in the racks, °C;

n — the number of racks with IT equipment. If RC/high = 100%, it means that the average air temperature at the inlet to IT equipment in all the racks of the machine room does not exceed the recommended value T =27 °C. If

max-rec

RCL^U < 100%, that indicates that the recommended

HIGH

value T = 27 °C in one or more IT equipment

max-rec

in the racks is exceeded (the lower the value of RC/high < 100%, the more IT equipment is there in the racks where the average air temperature at the inlet exceeds the recommended value).

Such researchers as Capozzoli et al. (2014) as well as Norouzi-Khangah et al. (2016) studied the efficiency of cooling systems in data centers using this index.

The KT index (Huang et al., 2017) characterizes the deviation of the average temperature values of air entering IT equipment relative to the temperature Tsup = 20 °C of air supplied by air conditioners. The KT index can be calculated by the following equation:

Kt — "

n 2

Y(t ■-t r

Z^y^mc i ^sup J i—1

*sup

n -1

(3)

Tsup — supply air temperature, °C; T . — average temperature of incoming air

inc I or d

entering IT equipment in the rack, °C;

n — the number of racks with IT equipment. The lower the KT index, the more efficient is air distribution in the data center machine room. Results and discussion To present the results of the study, we made sections in the x-y and y-z planes in the model (Fig. 10). The results of the study are represented by air temperature fields for different methods of hot aisle isolation (Fig. 11):

A) without containment and panels;

B) without containment but with panels;

C) with containment but without panels;

D) with containment and panels.

Besides, each method is represented by air temperature fields for the capacity of IT equipment in each rack of 4.7 and 9.5 kW.

To evaluate the modeling results, we constructed graphs of the average values of air temperature at the inlet to IT equipment in the racks Tnc . (Figs. 12 and 13) and calculated the dimensionless indices RC/H|GH and KT (Figs. 14 and 15).

Thus, as a result of the analysis of the RC/HIGH index values for different isolation methods and capacity of IT equipment, the following was established:

1) The maximum values (100%) of the RC/HIGH index were obtained for methods B) and D), which assume the arrangement of panels in the racks. This means that the average air temperature at the inlet to IT equipment in the racks for these methods does not exceed the maximum recommended value T = 27 °C. Besides, the value of the RCL^U

mav-ren ' H RH

a)

b)

Fig. 10. Location of the sections for the model of the

data center machine room in St. Petersburg. a) section of the model in the x-y plane at el. +0.300 m from the floor level (level of IT equipment location in the racks); b) section of the model in the y-z plane perpendicular to the center of the rows of the racks

1

Bl|ll|ll|ll|ll| llllllllllllllllll

llllllllllllllllfl ll|ll|ll|ll|ll|ll

b) without containment but with panels:

a) without containment and panels:

LTITJ

with containment but without panels

Fig. 11. Air temperature fields (°C) for the machine room for different methods of hot aisle isolation and capacity of IT equipment in each rack in the y-z and x-y section planes

O

35 33

h 31

CD

3 29 to

£ 27 E

(D

^25

Ol <21

19

o

35

33

I— 31 a)

3 29 ta

&27 E

ID

^25

CD

ro 23

0) " >

<

21 19

J A A A t

L_J A L J H L K A o i A _J 1

m V A I V i A fy J * > * A A /

J * » w » m r ■f M. i » Mr * m m r\ i_I U i . : 1 y

* * * * * £ * » * »

1 2 3 4 5 6

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Numbers of racks with IT equipment

Fig. 12. Average air temperature Inci(°C) at the inlet to IT equipment in the racks. Capacity of IT equipment in each rack = 4.7 kW

1 2 3 4 5 6

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Numbers of racks with IT equipment

Fig. 13. Average air temperature Inci (°C) at the inlet to IT equipment in the racks. Capacity of IT equipment in each rack = 9.5 kW

-A) ■B) •C) -D)

•A) -B) •C) -D)

O X.

100 90 80 70 60 50 40 30 20 10 0

4,7 9,5

Capacity of IT equipment in each rack (kW)

0,35 0,30 0,25 0,20 0,15 0,10 0,05

0,00

4,7 9,5

Load on IT equipment in each rack (kW)

■A) BB) HC) "D)

Fig. 14. Values of the dimensionless RC/HIGH index for different methods of hot aisle isolation and capacity of IT equipment in each rack

■ A) *B) BC) bD)

Fig. 15. Values ofthe dimensionless Krindex for different methods of hot aisle isolation and capacity of IT equipment in each rack

index for these methods does not change with an increase in the capacity of IT equipment in the racks.

2) The minimum values of the RCL^U index

' HIGH

were obtained for methods A) and C), which do not assume the arrangement of blind panels in the racks. It was also revealed that the value of the RCL^U

HIGH

index for isolation method C) (with containment but without panels) is lower than its value for method A) (without containment and panels). This is due to the fact that top containment increases the rate of hot air flow through the racks that are not equipped with panels.

3) With an increase in the capacity of IT equipment in the racks for methods A) and C), the RC/HIGH index becomes lower, i.e., the number of hot spots increases.

As a result of the analysis of the KT index values for different isolation methods and capacity of IT equipment, the following was established:

1) The isolation method D) is the most effective in terms of air distribution in the data center machine room.

2) The highest value of the KT index is typical for methods A) and C), which do not assume the arrangement of blind panels in the racks. This means that the efficiency of air distribution for these methods is the lowest among those considered. Besides, the efficiency of air distribution for these methods decreases with an increase in the capacity of IT equipment in the racks.

Conclusions

The comparative analysis of different methods of hot aisle isolation at different capacity values of IT equipment in the data center machine room allows us to draw the following conclusions:

1) The arrangement of top containment for the hot aisle without blind panels in the free rack space as well as the absence of any hot aisle isolation result in the occurrence of hot spots — places where air temperature at the inlet to IT equipment exceeds the maximum recommended value T = 27 C. The

max-rec

arrangement of blind panels in the free rack space without top containment for the hot aisle makes it possible to ensure optimal air temperature values at the inlet to IT equipment in the racks. The most effective method to isolate the hot aisle in terms of air distribution in the data center machine room is the one that takes into account the joint arrangement of blind panels in the free rack space and top containment for the hot aisle.

2) For the methods of hot aisle isolation, taking into account the arrangement of blind panels in the free rack space, an increase in the capacity of IT equipment in the racks does not result in the occurrence of hot spots. For the methods of hot aisle isolation without blind panels in the free rack space, an increase in the capacity of IT equipment leads to an increase in the number of hot spots, a deterioration in the circulation of air flows, and a decrease in the efficiency of IT equipment cooling in the data center machine room.

References

Abbas, A. M., Huzayyin, A. S., Mouneer, T. A., and Nada, S. A. (2021). Effect of data center servers' power density on the decision of using in-row cooling or perimeter cooling. Alexandria Engineering Journal, Vol. 60, Issue 4, pp. 3855-3867. DOI: 10.1016/j.aej.2021.02.051.

Almoli, A. M. (2013). Air flow management inside data centres. PhD Thesis in Mechanical Engineering.

ASHRAE (2016). ASHRAE TC9.9 Data Center Power Equipment Thermal Guidelines and Best Practices. [online] Available at: https://www.ashrae.org/File%20Library/Technical%20Resources/Bookstore/ASHRAE_TC0909_Power_ White_Paper_22_June_2016_REVISED.pdf [Date accessed September 12, 2022].

Capozzoli, A., Serale, G., Liuzzo, L., and Chinnici, M. (2014). Thermal metrics for data centers: a critical review. Energy Procedia, Vol. 62, pp. 391-400. DOI: 10.1016/j.egypro.2014.12.401.

Cho, J., Park, C., and Choi, W. (2021). Numerical and experimental study of air containment systems in legacy data centers focusing on thermal performance and air leakage. Case Studies in Thermal Engineering, Vol. 26, 101084. DOI: 10.1016/j.csite.2021.101084.

Cho, J. and Woo, J. (2020). Development and experimental study of an independent row-based cooling system for improving thermal performance of a data center. Applied Thermal Engineering, Vol. 169, 114857. DOI: 10.1016/j. applthermaleng.2019.114857.

Herrlin, M. K. (2007). Improved data center energy efficiency and thermal performance by advanced airflow analysis. Digital Power Forum, San Francisco, USA, pp. 10-12.

Huang, Z., Dong, K., Sun, Q., Su, L., and Liu, T. (2017). Numerical simulation and comparative analysis of different airflow distributions in data centers. Procedia Engineering, Vol. 205, pp. 2378-2385. DOI: 10.1016/j.proeng.2017.09.854.

Huawei (2019). E9000 Server V100R001 User Guide. [online]Available at: https://www.manualslib.com/manual/1668286/ Huawei-E9000.html [Date accessed September 12, 2022].

Nada, S. A., Said, M. A., and Rady, M. A. (2016). CFD investigations of data centers' thermal performance for different configurations of CRACs units and aisles separation. Alexandria Engineering Journal, Vol. 55, Issue 2, pp. 959-971. DOI: 10.1016/j.aej.2016.02.025.

Niemann, J., Brown, K., and Avelar, V. (2008). Hot-aisle vs. cold-aisle containment for data centers. White Paper 135. [online] Available at: https://www.missioncriticalmagazine.com/ext/resources/MC/Home/Files/PDFs/WP-APC-Hot_vs_ Cold_Aisle.pdf [Date accessed September 12, 2022].

Norouzi-Khangah, B., Mohammadsadeghi-Azad, M. B., Hoseyni, S. M., and Hoseyni, S. M. (2016). Performance assessment of cooling systems in data centers; Methodology and application of a new thermal metric. Case Studies in Thermal Engineering, Vol. 8, pp. 152-163. DOI: 10.1016/j.csite.2016.06.004.

Ponomarev, N. S., Martianova, A. Yu., and Dmitriev, Yu. A. (2021). Assessment of heat gain from server equipment. Bulletin of Civil Engineers, Vol. 2 (85), pp. 166-172. DOI: 10.23968/1999-5571-2021-18-2-166-172.

Priyadumkol, J. and Kittichaikarn, C. (2014). Application of the combined air-conditioning systems for energy conservation in data center. Energy and Buildings, Vol. 68, Part A, pp. 580-586. DOI: 10.1016/j.enbuild.2013.07.082.

Rasmussen, N. (2012). Improving rack cooling performance using airflow management blanking panels. White Paper 44. [online] Available at: https://www.apc.com/us/en/download/document/SPD_SADE-5TPLKQ_EN/ [Date accessed September 12, 2022].

Tatchell-Evans, M., Kapur, N., Summers, J., Thompson, H., and Oldham, D. (2017). An experimental and theoretical investigation of the extent of bypass air within data centres employing aisle containment, and its impact on power consumption. Applied Energy, Vol. 186, Part 3, pp. 457-469. DOI: 10.1016/j.apenergy.2016.03.076.

Timonin, Yu. (2018). Concepts of data center cooling arrangement: in search of maximum efficiency. STA, No. 1. pp. 84-90.

Vertiv (2022). Liebert CRV, Row-based Cooling Unit. [online] Available at: https://www.vertiv.com/ru-emea/products-catalog/thermal-management/in-row-cooling/liebert-crv-row-based-cooling-unit-/ [Date accessed September 12, 2022].

Zhang, M., An, Q., Long, Z., Pan, W., Zhang, H., and Cheng, X. (2017). Optimization of airflow organization for a small-scale date center based on the cold aisle closure. Procedia Engineering, Vol. 205, pp. 1893-1900. DOI: 10.1016/j. proeng.2017.10.279.

АНАЛИЗ ЭФФЕКТИВНОСТИ СИСТЕМЫ ОХЛАЖДЕНИЯ

Юрий Александрович Дмитриев

Санкт-Петербургский государственный архитектурно-строительный университет 2-я Красноармейская ул., д. 4, 190005, Санкт-Петербург, Россия

E-mail: [email protected] Аннотация

Введение: Основной задачей систем охлаждения в помещении машинного зала центра обработки данных является поддержание оптимальных параметров воздуха на входе в IT-оборудование в соответствии с действующими нормативными документами. Если значение температуры воздуха на входе в IT-оборудование превышает оптимальное значение, то возникают так называемые «горячие точки» - работа IT-оборудования в таких местах может привести к его перегреву и выходу из строя. Основной причиной возникновения «горячих точек» является нарушение циркуляции воздуха между «горячим» и «холодным» коридорами. Целью данного исследования является анализ влияния различных способов изоляции «горячего» коридора на эффективность системы охлаждения IT-оборудования. Использован следующий метод: Моделирование в программном комплексе STAR-CCM+ различных способов изоляции «горячего» коридора при различных значениях мощности IT-оборудования. В результате исследования теплового и воздушного режимов в ЦОД выявлен наиболее рациональный способ устройства изоляции «горячего» коридора - горизонтальные ограждения над коридором с вертикальными глухими панелями в свободном пространстве стоек.

Ключевые слова: центр обработки данных, температура воздуха, холодный коридор, горячий коридор, вычислительная гидродинамика (CFD).