CC BY
19POWER MANAGEMENT FOR SERVER CLUSTERS HARDWARE
12 3
Babkin O.V. , Varlamov A.A. , Gorshunov R.A. , Dos E.V.4, Kropachev A.V.5, Zuev D.O.6
1Babkin Oleg Vyacheslavovich - Strategy Consultant, IBM;
2VarlamovAleksandrAleksandrovich - CTO, SHARXDC LLC, MOSCOW;
3Gorshunov Roman Aleksandrovich - Solution Architect, AT&T, BRATISLAVA, SLOVAKIA;
4Dos Evgenii Vladimirovich - Lead DevOps Architect, EPAM, MINSK, REPUBLIC OF BELARUS;
5Kropachev Artemii Vasilyevich - Principal Architect, LI9 TECHNOLOGY SOLUTIONS, NORTH CAROLINA;
6Zuev Denis Olegovich - Independent Consultant, NEW JERSEY, USA
Abstract: power management for server clusters physical resources was analyzed. It was shown that low performance of data center infrastructure work refers to disproportion of servers' utilization. Overconsumption problem could be solved by minimization of the active servers' number within the bounds of the server consolidation procedure. In order to provide server consolidation implementation it is necessary to maintain acceptable performance level of the servers room infrastructure work. Server consolidation may cause performance degradation due to the conflict of using shared resources by virtual machines. It was demonstrated that threshold level of utilization regime analysis should be based in order to get a compromise between stable work of data center and opportunity for power savings which is associated with skipping of rare cases of servers' peak load. Basic scheme of clustering-based correlation-aware virtualization includes trace data center servers' physical resources utilization level, transform$ation of utilization traces into binary sequence up to the utilization threshold value, clustering of virtual machines up to the binary sequence in order to maintain not overlapping of different clusters and virtual machines allocation at physical servers in order to minimize the possibility of the service performance degradation at peak period. It was develop power management procedure which consists from user-interactive and fast changing service, maintaining of the minimal performance degradation caused by physical resources sharing conflict and high correlation level of virtual machines. Thereby it is important to estimate proper measure to quantify the correlation coefficient between virtual machines to overcome the inefficiency of the conventional correlation metric. Pearson's correlation coefficient was proved to be optimal instrument of the correlation of used data center virtual machines physical resources utilization quantifying. Developed model allows storing all samples and evenly distributing computational utilization as well as correlation between the events in the bounds of certain time period.
Keywords: data center, power consumption, virtual machine, clustering, Pearson's correlation coefficient, performance degradation, peak load.
1. Introduction
Low performance of data center infrastructure work is usually associated with
disproportion of servers' utilization. Statistically less than 30% of data servers are under -utilized more than 90 % of the total time while other servers normally cause idle power
consumption which leads to the 50% of power loss and inefficient work of data center
during peak period [1-5]. Overconsumption problem could be solved by minimization of the
active servers' number within the bounds of the server consolidation procedure. Server
consolidation is server virtualization technique, which allows workloads encapsulating as virtual machines (VMs) and, thus, run multiple VMs at single server with the aid of hypervisor block (Figure 1).
Main task of data center server consolidation implementation is maintenance of the prior performance level of the servers room infrastructure work. It leads to necessity of the data center peak utilization regime analysis, usually at the 90%, 95% and 99% of maximal recorded value threshold level. Threshold level should be based on the recorded sample in order to get a compromise between stable data center work and opportunity for power savings which is associated with skipping of rare cases of servers' peak load [6, 7]. Thereby, server consolidation may cause performance degradation due to the conflict of using shared resources by VMs [8, 9], specifically last level cache (LLC). The results of cache co-located VMs usage analysis show that sharing LLC between two copies of VMs leads to 20%-30% performance decreasing. The amount of interference could be characterized with a set of parameters, such as effective number of used sets [10]. Allocation of VMs, thus, can be realized by accounting for the amount of the interference and its minimization through the estimation of the required performance requirement.
Fig. 1. Data center server consolidation scheme
To develop data center server consolidation methodology there were analyzed recent studies and publications. Statistics of modern data servers' physical resources utilization rate and proportions was considered [1-5]. In order to overcome restrictions and optimize power consumption models that based on the resources utilization threshold value rather than the peak value were discussed [6-7]. Sharing of the servers' resources among co-located VMs, especially LLC problem were studied [8, 9]. To develop the methodology co-located VMs interference with a set of parameters, particularly effective number of used sets was analyzed [10], as well as correlations among VMs' workload [11-14]. Finally it was studied power management solution for data centers scale-out application and targeting distinctive workload characteristics of scale-out applications [15, 16].
2. Key aspects of the correlation-aware power management
As it was mentioned above, server consolidation could be achieved by considering correlation among workload variation. Basic scheme of clustering-based correlation-aware VM development [11, 16] solution includes:
• trace data center servers' physical resources utilization level;
• transform utilization traces into binary sequence up to the utilization threshold value;
• clustering of VMs up to the binary sequence in order to maintain not overlapping of different clusters;
• VMs allocation at physical servers in order to minimize the possibility of the service performance degradation at peak period.
Typical engineering solution is pairing of two uncorrelated VMs into super-VM. Maintaining of the super-VMs can be done by predicting of the aggregated workloads. Once two uncorrelated VMs are paired correlations of them within a same super-VM have not be considered, and possibility of further power consumption decrease will be lost. If servers' utilization is perfectly known this scheme could be extended by utilization of multiple VMs workload of such that for VM placement. To overcome those drawbacks it was proposed [15, 16] to develop power management procedure for cloud services that includes:
• user-interactive and fast changing service;
• maintaining of the minimal performance degradation caused by physical resources sharing conflict;
• high correlation level of VMs.
While the scale-out applications usually operate as highly parallel processes, it is advisable to assign the right number of CPU cores for each VM. At Figure 2 are demonstrated generalized results of recent studies [6, 17] of response time of a websearch cluster with respect to the number of queries for 90% threshold value. The number of allocated cores varied from 4 to 16. It should be noticed that resource utilization level depends on time and usually is lower than the available amount of resources, though dynamic power gating cannot be applicable to this type of applications due to the performance degradation caused by the unapropriable transition delay of power modes switching. Thereby it should be noted necessity of allocating the right number of cores for each VM according to its peak and off-peak resources utilization demands. This procedure has to be implemented at the stage of scaling voltage/frequency level (V/F level).
Fig. 2. Response time of cluster with respect to the number of queries for 90% threshold value
Dynamics of the websearch cluster's CPU utilization level is shown at Figure 3. CPU utilization level was traced for 2 VMs with respects to the number of clients' queries. CPU
utilizations of both VMs are synchronized with the variation of the number of queries and it could be seen that loads between VMs are not perfectly balanced. Therefore it should be mentioned that resource utilization efficiency has to be improved by sharing cores among multiple VM.
Fig. 3. CPU utilization level of two VMs with respect the number of clients queries
This procedure will provide more flexible use of the core cores up to the resource demands as a real-time function.
3. Proposed server consolidation method
As it was mentioned before clients queries are distributed between multiple VMs and of every cluster and workloads of VM within a same cluster are highly correlated in comparison of correlation of different clusters VMs. In Figures 4-6 is shown intra-cluster correlation of 2 VMs. It is can be seen that VMs resources utilization are strongly synchronized. Proposed method includes analysis of VMs pervasive correlation within a cluster and among clusters. The Figures demonstrates the effectiveness of the correlation-aware VM maintaining of 2 servers which possess 2x8 cores. Servers virtualization produces 4 VMs: VM-1, VM-2, VM-3, and VM-4 where VM-1 is co-located with VM-2 and VM-3 is co-located with VM-4 (Figure 4).
Fig. 4. Data center servers ' virtualization procedure simulation
Fig. 5. Drawbacks of VMs allocations without considering correlation value
VMs have the same tail distribution of CPU computational resource utilization and co-located ones are highly correlated. If one will not take into account the correlation (Figure 5), services 1 will allocate sets of VM-1 and VM-2, while service 2 will allocate sets of VM-3 and VM-4. In this case, extremum value of CPU utilization will attain 8x100% of core of each server (active state of all cores). In other hand, if one will pair services [VM-1; VM-3] and [VM-2; VM-4], extremum value of CPU utilization for each server cores may be lowered down to 6x100% (Figure 6), which allow to lower v/f level without services performance degradation.
VM1 N VM2 N VM3 VM4 I
N
Service 1
<>
Service 2
Ut ligation level, % Ut ligation level. %
T, s T, s
2 10 10
Fig. 6. Correlation-aware VMs allocations procedure
To develop an efficient mathematical model it is proposed to use Pearson's correlation [13] to quantify the correlation coefficient of used data center VMs CPU utilization. It can be calculated as the ratio of covariance of the two variables to the product of their standard deviations. However, Pearson's correlation could be inefficient for the task because this value refers to correlation throughout the corresponding time interval while only correlation at peak or threshold VMs utilization is required. Thereby, it was important to estimate proper measure to quantify the correlation coefficient between VMs that is able to overcome the inefficiency of the conventional correlation metric:
Cij = { 1 - 1 ucpJ ) ■ 1 0 0 %, (1)
where is correlation measure of and , is CPU utilization level of , of is CPU utilization level of and is aggregated actual peak utilization of co-located and . refers to complete correlation, while refers to
no correlation.
It is important to note that values of each recorded period of utilization have to be updated. Correlation coefficients between all VMs have to be modeled by matrix
where each element corresponds to the measuring function. This model will allow storing all samples and evenly distributing computational utilization as well as correlation between the events in the bounds of certain time period.
3. Conclusions
It was shown that low performance of data center infrastructure work refers to disproportion of servers' utilization. Overconsumption problem could be solved by minimization of the active servers' number. In order to provide server consolidation implementation it is necessary to maintain acceptable performance level of the servers room infrastructure work. Server consolidation may cause performance degradation due to the conflict of using shared resources by virtual machines. Basic scheme of correlation-aware virtualization includes: tracing data center servers' physical resources utilization level, transforming of utilization traces into binary sequence up to the utilization threshold value,
clustering of virtual machines up to the binary sequence and virtual machines allocation at physical servers. Power management procedure which consists from user-interactive and fast changing service, maintaining of the minimal performance degradation caused by physical resources sharing conflict, high correlation level of virtual machines was developed. Pearson's correlation coefficient was proved to be optimal instrument of the correlation of used data center virtual machines physical resources utilization quantifying. Developed model model allows storing all samples and evenly distributing computational utilization as well as correlation between the events in the bounds of certain time period.
References
1. Geng H., 2015. Data center handbook. Hoboken, NJ: John Wiley & Sons. Energy Star Program. "EDA Report to Congress on Server and Data Center Energy Efficiency", 2007.
2. Barroso L.A. and Holzle U. "The datacenter as a computer: An introduction to the design of warehouse-scale machines" Synthesis Lectures on Computer Architecture4. № 1, 2009: 1-108.
3. Ferdman М., Adileh А., Kocberber О., Volos S., Alisafaee М., Jevdjic D., Kaynak С., Popescu A.D., Ailamaki А. and Falsafi В. "Clearing the clouds: a study of emerging scale-out workloads on modern hardware," in ACM SIGARCH Computer Architecture News. Vol. 40. № 1. Pp. 37-48. ACM, 2012.
4. Harris M., 2014. Data Center Infrastructure Management. Data Center Handbook, 601618. doi:10.1002/9781118937563.ch33.
5. Meisner D., Sadler СМ., Barroso L.A., Weber W.-D. and Wenisch T.F. "Power management of online data-intensive services," in Computer Architecture (ISCA), 2011. 38th Annual International Symposium on, pp. 319-330. IEEE, 2011.
6. Bobroff N. et al. "Dynamic placement of virtual machines for managing sla violations," in Proc. IM2007.
7. Jaramillo D., Furht B. & Agarwal A., 2014. Mobile Virtualization Technologies. Virtualization Techniques for Mobile Systems, 5-20. doi:10.1007/978-3-319-05741-5_2.
8. Tickoo R. Iyer, R. Illikkal and Newell D. "Modeling virtual machine performance: challenges and approaches," in ACM SIGMETRICS Performance Evaluation Review 37, 2010.
9. Govindan S., Liu J., Kansal А. and Sivasubramaniam А. "Cuanta: quantifying effects of shared on-chip resource interference for consolidated virtual machines," in Proceedings of the 2nd ACM Symposium on Cloud Computing. P. 22. ACM, 2011.
10. Wang L. & Lu Y., 2010. Power-efficient workload distribution for virtualized server clusters. 2010 International Conference on High Performance Computing. doi: 10.1109/hipc.2010.5713178.
11. Meng Х. et al. "Efficient resource provisioning in compute clouds via VM multiplexign," in Proc. ICAC, 2010.
12. Chen М. et al. "Effective VM sizing in virtualized data centers," inProc. IM, 2011.
13. Halder ^ et al. "Risk aware provisioning and resource aggregation based consolidation of virtual machines," inProc. Cloud, 2012.
14. Santos J.R., Turner Y. Virtual Machine Management, 2011. Mastering. 255-326. doi:10.1002/9781118257432.ch7.
15. Kim J., Ruggiero М., Atienza D. andLederberger М. "Correlation-aware virtual machine allocation for energy-efficient datacenters," in Proc.Conference on Design, automation and Test in Europe (DATE). Pp. 1345-1350, 2013. № 5 (2003): 164-177.
16. Khan S.U., 2015. Handbook on data centers. Place of publication not identified: Springer.
