CC BY-NC-ND
36
ADVANCED DATA CENTER ECONOMY
R.R. Sukhov,
Finance manager of INO Uptime Technology E-mail: r.sukhov@uptimetechnology.ru Address: 1-2. M.Vasilevskogo st, Moscow, Russia
M.B. Amzarakov,
Director of INO Uptime Technology E-mail: m.amzarakov@uptimetechnology.ru Address: 1-2. M.Vasilevskogo st., Moscow, Russia
E.A. Isaev,
Candidate of technical science, Professor, Head of Department for Education Stack Group, National Research University Higher School of Economics E-mail: eisaev@hse.ru Address: 33 Kirpichnaya st., Moscow, Russia
Key words: energy efficiency, total cost of ownership, data center.
1. Introduction
Today to use information technologies one has to understand how to operate IT equipment in a correct way. IT-equipment is mostly maintained in data processing centers (DPC, data center etc.).
Data center is closely connected with the process of IT-equipment operation and therefore DC quality and safety play a significant role as the basic element of IT operation.
Professional environment is very much involved in creation of DCs. The reason is rapid development of corporate IT sphere, which, in its turn, is triggered by increasing
number of automated business processes, projects focused on retail sector, and increased competition in many sectors of the economy. Companies face the pressing issues whether to build a data-center on their own (and further be involved in none-core activity on its operation) or to engage task-oriented provider.
The reason for significant difference in estimates of capital costs for these projects is first of all that Russian companies are hardly familiar with the principles of DC construction and operation. Let’s try to figure it out.
The following key parameters are initial for planning a DC: net area for placement of computer equipment
(computer room), the minimum allowable parameters for system reliability, «power-to» data-center. Their values are subjected to the business requirements. In case of net area assessment everything is more or less clear. And to correctly define the reliability level and power consumption of a DC is not that easy.
Even minimum theoretical knowledge and common sense suggest that any complex system should be safe enough for each component and provide redundancy. Only this will ensure the highest rate of overall IT system reliability required to meet business needs.
This approach is described in many documents, for example - the American standard TIA 942, which clearly specifies the reliability level ensured by implementation of various technical solutions [1].
TIA 942 describes four levels of reliability, from Tier I to Tier IV.
2. Brief description of DC safety categories
As it was mentioned the most popular is DC separation for types according to operation safety level (see TIA 942).
DC OF TIER 1 CATEGORY «The simplest infrastructure».
Commonly such DC has no reservation of equipment, current-carrying and cold-supplying paths. Level of service availability is about 99.67% that assumes service interruption for no less than 28 hours annually. Most operations with DC infrastructure assume either partial or full cut-off of server equipment. Risk for unplanned equipment cut-off is extremely high due to technological design and a number of works on engineering infrastructure maintenance [2].
DC OF TIER 2 CATEGORY «Infrastructure with separate component reservation».
Engineering infrastructure equipment of such DC has reservation of power and cold supply system components. Though pathways (cable lines, bus wires, cold supply pipelines, copper/optical lines) aren’t reserved for supply of required capacity. Basic advantage of Tier II infrastructure is the possibility of equipment item cut-off for maintenance without service delivery violation (power supply, cold supply etc.). Such DC includes requirements for security system and guard personnel. Level of service availability is about 99.75% or 22 hours of planned and unplanned break annually [2].
DC OF TIER 3 CATEGORY «Infrastructure with competitive maintenance».
It means that each engineering infrastructure component of such DC might be taken for maintenance without cut-off of server equipment. This is the key possibility for DCs of this category. All components have N+1 reservation; pathways are designed with the possibility of «hot» maintenance using reserve lines. All server equipment shall have reservation of supply units to maintain power supply circuit by one of the lines. Level of service availability is about 99.98% or 1.6 hour of unplanned break annually [2].
DC OF TIER 4 CATEGORY «Defect-tolerant architecture».
Architecture and technological design, rules for maintenance of such facility assume that violation of server equipment performance is possible only in case of intended act, fire, and intersection of a number of technical failures. The requirement of this DC category is duplication of all DC systems without cut-off of working load. Level of service availability is about 99.99% or 0.8 hours of unplanned break annually [2].
3. Preliminary evaluation of investment costs
Knowing the standard requirements for data centers, we realize that the Tier 1 capital costs for DC construction cannot be linearly extrapolated to the Tier IV level. Nonlinearity cost increase is triggered by increased number of reserve infrastructure components and additional engineering systems subjected to redundancy while transition to the next reliability level of DC.
Practice of DC construction shows that the project cost per 1 sq. m. is about 15 thousand dollars for Tier II Class and 26 thousand dollars - for Tier III [3]. The costs are greatly affected by «power-to» of the future data center, which refers to a planned power consumption of a server room and the maximum amount of heat removal from the rack. For example, the costs for two data centers that have the same Tier III reliability level and the same area of the server rooms, but different power-to», may vary significantly.
Data centers constructed and operated by our team have specific power 1.5-4 kW per 1 sq. m. and structurally inherent possibility for energy consumption increase by at least 50%.
Estimated distribution of practice-based costs according to major expenditures, are listed in Table 1 [3].
Table 1.
Estimated distribution of costs while DC construction, % [3]
Expense item Tier I Tier II Tier III
Construction 15 10 7
Power supply systems 25 35 38
Cooling system 20 25 27
Supporting system 10 7 4
Telecommunication 10 8 5
External electricity connection 20 15 17
As already noted, estimated electrical power, referred to 1 sq.m. of net area, may seriously adjust the basic proportions. The data in the table are well correlated with indicators specified by The Uptime Institute [4]. According to one of its documents: «The analysis of DC construction costs shows that electro-mechanical (engineering) systems account for more than 70% of the total construction costs depending on the density and functionality of the equipment.»
For approximate estimation of DC construction costs Uptime Institute suggests the model with only two components. First, the «energy» cost component is normalized to 1 kW of power consumed by a server room, and is highly dependent on the reliability level of a data center. While Tier I facility construction the «power» unit cost of a project accounts for 11,5 thousand dollars per kW. Tier II facilities account for 12,5 thousand dollars. On the next level it grows up to 23 thousand dollars, and for Tier IV data centers the value is 25 thousand dollars for 1 kW of output UPS. The second component of costs is always the same: about 2.880 dollars per sq. m. area of a server room [5].
Focusing on the proposed model, we will try to calculate the cost of a Tier III data center with 2000 sq.m. net area at 1 kW «power-to» per sq.m. The cost of the data center accounts for 51.76 million, or 25.88 thousand dollars per 1 square meter of a server room. And with a double increase of «power-to» the total cost of the solution will increase up to 97.76 million, or 48.88 thousand dollars per unit area of a machine room.
Commercial operation of a data center, which includes several engineering systems, also requires attention in terms of operating cost planning. There are four of the most important articles of operating costs: power supply, maintenance of equipment (including staff salaries), utilities (including rent), consumables (Table 2) [3].
Estimated distribution of costs while operation of data center, % [3] Table 2.
Expense item 1 %
Power supply 40
Equipment operation (inc. salary of operation personal) 25
Public utilities (inc. rent of facilities) 25
Consumables 10
Effective technical solutions adopted at the stage of DC designing will help to reduce operating costs, and costs of upgrading and scaling. Practical experience of DC operating shows that such work should be carried out at least every four years.
4. DC energy efficiency impact on operational expenses
The main reason for close attention of engineers, who design equipment for DCs, and managers, who operate this equipment, is reduction of operation costs while equipment performance.
If to take a precise look at this article we’ll see that power supply costs include two basic components: electrical power costs for IT-equipment power supply and electrical power costs for cooling of this equipment.
Most often cooling costs might exceed power supply costs for IT-equipment.
According to the statistics on average DC spends about 40% of its operating expenses for payment of electricity bills.
APPROXIMATE DISTRIBUTION OF OPERATION COSTS
□ DС maintenace
□ Electricity
□ Staff costs
□ Rent of premies
Fig. 1. Budget distribution in DC (APC Company source 2005) [6]
In its turn electricity is consumed for:
♦ Power supply of IT-equipment;
♦ Power supply of cooling equipment;
♦ Power supply of supporting systems (lightning, ACS, fire alarm etc.);
♦ Losses in transmission lines and equipment;
However in most cases costs for energy supply of cooling systems and losses account for up to 80% of all electricity consumption.
Academic community has a common view how to integrally evaluate the efficiency of DC operation. And today it is provided by PUE factor.
PUE (Power Usage Effectiveness) index is commonly used for evaluation of energy efficiency [7].
PUE measures how effectively a DC uses electric -ity, specifically how much electricity is consumed by IT-equipment versus full electricity used by DC that includes IT-load, cooling, lightning and other consumers. Ideal PUE = 1, which means that all capacity, supplied to DC, is only used by IT-load. PUE could be calculated according to the formula:
_ Total facility power
IT equipment power
The calculations show how important to reduce electricity consumption of system elements that do not refer to IT-equipment. For most of DC’s this is the only way to increase their economic efficiency.
Let us give an example (Business Case) of how energy efficiency impacts on the economy of a DC:
Racks installed 80
Average consumption capacity for 1 rack 5 KW
Capacity consumed by IT 400 KW
Capacity consumed by cooling 550 KW
Losses in power lines and equipment 50 KW
Total capacity of DC1 1000 KW
Calculating PUE we shall obtain:
PUE = 1000/400 = 2,5
If 1 KWh equals 3.81 rub, total consumption equals 2 743 200 per month or 32 918 400 rub per year. Costs for cooling equal 1 508 760 rub per month or 18 105 120 rub per year and it accounts for 55% of all electricity costs.
The best practices in the field of DC construction allow to reach PUE=1.15-1.2 during one year evaluation.
Let us recalculate our case with PUE=1.2 and we shall obtain:
Racks installed 80
Average consumption capacity for 1 rack 5 KW
Capacity consumed by IT 400 KW
Capacity consumed by cooling 60 KW
Losses in power lines and equipment 20 KW
Total capacity of DC2 480 KW
Total consumption equals 1 316 736 rub per month or
15 800 832 rub per year. Costs for cooling equal 164 592 rub per month or 1 975 104 rub per year and it accounts for only 12.5% of all electricity costs.
It is worth noticing that leaving the level of IT-load at 400 KW we obtain the sum of economic effect that equals 17 117 568 rub. given annual operating costs.
It is difficult to reach the same level of efficiency at an operating DC without deep upgrading. Though if to conduct an expert study and analysis of current process solutions, in most cases it is possible to reach PUE=1.5 + 1.7 with moderate investments involved (with payback period less than or equal to one year). It may give an economic effect up to 12 million rub (compared to our DC 1 in Business Case) that is also a very good result, which allows to increase DC’s economic efficiency and obtain additional competitive advantage in terms of greater safety factor in service cost value [8].
5. Mathematic model of energy transfer
For operating efficiency evaluation of any DPC we should first of all draw attention to efficiency of cold supply system, as cold supply costs is the highest article of DC energy expenditures. Average costs for DC cooling according to world statistics equal to 50% of all energy costs for DC operation.
Generally air transferred capacity is defined by the formula:
Q(kW) = L(m3/sec) AT(°K)- p(kg/m2)- C(kJ/oKkg)
where: p = 1,2 kg/m2, C = 1 (kJ/°K-kg),
and L(m3/sec) = S(m2) • V(m/sec)
substituting the formula of finding the air volume per time unit into initial expression we shall obtain:
Q(kW) = S(m2) V(m/sec) • AT(oK)- p(kg/m2) •
• C(kJ/oKkg)
Having initial data for air transfer section areas we can define heating («cooling») capacities.
The following calculation allows to evaluate both required cooling system performance and performance losses due to air leakage and flows through unsealed sections.
Here is an example of such evaluation:
Q(kW) = S(m2)- V(m/sec) • AT(oK)- p(kg/m2) •
• C(kJ/°Rkg)
where, p=1.2, C=1, V=4
S(1U) = 19”*1 U = 0.483^0.045 = 0.022 m2 Q(1U) = 0.022*4*12*1.2*1 = 1.267 kW
The calculation shows that one opened unit (section with 0.483 m and 0.045 m dimensions) reduces cooling system performance by 1.267 kW.
Within this example the figure doesn’t seem huge, though we have to consider that number of units at an average DC (about 250 cabinets, each cabinet has 47 units) is 11 750 units.
The practice shows that on average such DC has about 5% of opened (unused) units. That for out example means 585 units or 744 kWof «lost» cooling system performance.
6. Energy efficiency influence on general energy costs
Above we considered the importance of DC cooling systems. Though we should not forget about IT-equip-ment, as IT-equipment energy efficiency influences all energy costs of the rest systems.
The reason of the influence is that while DC creation the basic factor to define systems like cold supply, ensured power supply, energy distribution is IT-load,
i.e. server consumed electric power. Calculation and design of the rest systems are performed according to this parameter.
Respectively if we use IT-equipment with high energy efficiency characteristics (i.e. lower power consumption per unit of computing effort done), the rest consumers together might have lower energy consumption parameters.
The influence of IT-equipment consumption on general DC consumption is shown in fig. 2.
As we can see each Watt of consumed by IT-equip-ment electric power on average requires additional 1.84 Watt.
Therefore we shall not forget about all joint components of DC infrastructure.
Server (IT) Load
hi
1 W energy save here
-1.0 W | -1.18 W |
DC-DC AC-DC
Additional savings are and 0,31 W more 0,18 W
\7
-1.49 W
Distributing
and 0,04 W here
-1.53 W
UPS
and 0,14 W here
-1.67 W
Cooling
CASCADE EFFECT
1 watt saved at the server hardware, results in a savings of 2.84 watts in total energy consumption
and 1,07 W here and 0,07 W here
-2.74 W
Tootal distributing
-2,84 W
Fig. 2. Energy efficiency influence to total expenses of energy [9]
7. Conclusion
The article includes data on basic infrastructure elements that influence economic parameters of DC operation, i.e. IT-equipment and cooling system.
While DC designing, as an infrastructure facility, we should consider evaluation of all costs both at the stage of construction and operation.
Combined cost evaluation allows to perform complete evaluation of DC total cost of ownership.
This approach allows minimizing events connected with reduction of DC operation profitability and increasing the probability of planned targets achievement upon return of investments in the projects at the stage of designing. ■
References
1. Sukhov R.R. How calculate the costs of data center // Network World. — 2008 — No. 4.
2. SP-3-0092 document: (TIA-942 standard, 7.0 edition). — February 2005.
3. Internal statistic data of JSC «Stack Telecom» (http://www.stacktelecom.ru). — 2008.
4. The Uptime institute: http://www.uptimeinstitute.com.
5. W. Pitt Turner IV, P.E. with Kenneth G. Brill, Cost Model: Dollars per kW plus Dollars per Square Foot of Computer Floor. — July 2010.
6. APC Company: http://www.apc.com (Date of request: 07/05/2013).
7. The Green Grid Consortium: http://www.thegreengrid.org/Global/Content/white-papers/The-Green-Grid-Data-Center-Power-Efficiency-Metrics-PUE-and-DCiE (Date of request: 07/05/2013).
8. Uptime Technology Russia: http://www.uptimetechnology.ru/evaluation_pue.html (Date of request: 07/05/2013).
9. Emerson Network Power Company: http://www.emersonnetworkpower.com/en-US/Latest-Thinking/EDC/ Resources/Pages/Cascading-Savings-Calculator.aspx (Date of request: 07/05/2013).
10.Sukhov R.R., Amzarakov M.B. How much will my business the energy efficiency // Bankovskie technologii. — 2011. - No. 6.
fr
ОСНОВЫ
ИНФОРМАЦИОННЫХ
ТЕХНОЛОГИЙ
СТРАТЕГИЧЕСКОЕ
УПРАВЛЕНИЕ
ИНФОРМАЦИОННЫМИ
СИСТЕМАМИ
_ Ч? 1-
СТРАТЕГИЧЕСКОЕ УПРАВЛЕНИЕ ИНФОРМАЦИОННЫМИ СИСТЕМАМИ Учебник.
Р. Б. Васильев, Г.Н. Каляное,
Г. А. Левочкина, О. В. Лукиноеа Под ред. Г.Н. Калянова
Москва: Интернет-уннверснтет Информационных технологий: Бином. Лаборатория знаний, 2010.
Учебник дает читателю представление о методологической базе и современных подходах и методах стратегического управления информационными системами, обеспечивающего целостный, процессноориентированный подход к принятию управленческих решений, направленных на повышение эффективности владения и развития информационных систем для достижения бизнес-целей предприятий и создания новых конкурентных преимуществ.
