CC BY
8
DOI: 10.24412/2413-2527-2022-432-46-51
Russian version of the article © V. A. Lokhvitsky, V. A. Goncharenko, E. S. Levchik is published in Intelligent Technologies in Transport, 2022, No. 1 (29), Pp. 39-44. DOI: 10.24412/2413-2527-2022-129-39-44.
A Scalable Microservice Model Based on a Queuing System with «Cooling»
Grand PhD V. A. Lokhvitsky, PhD V. A. Goncharenko, E. S. Levchik Mozhaisky Military Space Academy
Saint Petersburg, Russia lokhv_va@mail.ru, vlango@mail.ru
Abstract. The formalization of a scalable service functioning is proposed, taking into account the time spent on configuring computing nodes based on a multichannel non-exponential queuing system with «heating» and «cooling» of channels. The complexity of the model is reduced by representing the durations of «heating» and «cooling» in the form of a generalized random delay before servicing. The main stages of the method of calculating the probability-time characteristics of the QS with «cooling» are described. The markovization of the non-Markov components of the QS under consideration was carried out using the H2-distribution. The probability distribution of the states of the QS under consideration is obtained. The results obtained by the numerical method are compared with the results of simulation.
Keywords: microservice modelling, non-exponential queuing system, queuing system with «cooling», distribution approximation, numerical queuing theory.
Introduction Modern information systems are developed and operate using a wide range of technologies. Among them, as one of the most promising, it is necessary to single out a technology based on a micro-service architecture with container virtualization [1].
The infrastructure of a microservice information system with container virtualization is characterized by a dynamic change in the computing structure and node parameters. In particular, when considering the scaling processes of services implemented on the basis of containerized applications, it is necessary to take into account the time spent on launching the required number of containers, checking their operability, as well as stopping containers that are not currently expected to be used [2].
The general scheme of the application processing process by a scalable service is shown in Figure 1.
The durations of individual operations of starting, stopping containers, and checking the operability (taking into account the type of service application running in the container) are random variables that must be taken into account in the corresponding models.
Formalization of the scalable service functioning Traditionally, queue theory models are used to model the processes of processing a random flow of tasks. In particular, the task processing process we are considering can be represented as a multichannel queuing system with additional consideration of the time spent on configuring the service.
Fig. 1. The scheme of a scalable service functioning, taking into account the costs of allocating and releasing resources
The specified configuration can be performed in several cases:
- when starting the service (instances of the service), if at the time of arrival of the next service request, the service was stopped;
- when the number of running instances of the service changes (scaling up or down the number of instances).
Depending on the types of distributions that characterize the random processes of the arrival of applications into the system, their maintenance, scaling of the service with a health check, models based on different types of queuing system (QS) can be used. Analysis of the works [3-6] showed that modeling of such processes is possible based on the use of QS with «heating» or «cooling» of service channels. A significant limitation of these works is the impossibility of simultaneous accounting in one model of the processes of «heating» and «cooling» channels. The schematic diagram of the desired process is shown in Figure 2.
In the stationary mode of operation of the service system, the location on the time axis of the moment of occurrence of the specified delay (before or after maintenance) does not matter in principle. It is important to take into account the total random duration of the delay, which directly affects the time the application stays in the system. Here it is necessary to take into account the presence of random delays of both types, therefore it is possible to present the process under consideration using a
Fig. 2. Representation of the process in the form of a multichannel QS with «heating» and/or «cooling»
general-type multichannel QS model with «cooling», proposed in [6]. To do this, at the initial stage, it is necessary to find a generalized random delay before (or after) maintenance based on two distributions of random variables — the duration of «heating» and the duration of «cooling».
Representation of heating and cooling processes
in the form of a random delay before queries servicing To calculate the desired delay, first we will build a time diagram of the specified process (Figure 3).
When constructing the diagram, we define the following conditions:
1. The «warm-up» mode is initiated only by the event of receipt of an application in an empty system.
2. The «cooling» mode is initiated only by the event of the end of processing of the last application in the absence of applications in the queue.
3. The beginning of processing of an application entering an empty system is determined by the duration of the «warm-up» stage and the remainder of the «cooling» time, provided that the system was «cooling» at the moment.
The conditions listed above require clarification. Firstly, the «heating» mode is activated if the system was empty (there was not a single request for maintenance and in the queue) and the next request arrived. In this case, the service of the arrived application will begin after a random period of «heating» of the system. In addition, all applications arriving during the «heating» of the system fall into the queue and are also not discussed until the end of this period.
secondly, the «cooling» mode is activated after the end of servicing the last application (that is, the maintenance of the next application has ended and there is not a single application in the queue). Applications arriving during the «cooling» of the system also fall into the queue, and their maintenance can begin no earlier than the «cooling» period ends, and then the «heat-ing» mode.
Thus, the desired duration of the random delay before maintenance is found by convolving the first initial distribution with the residual second (for example, «heating» with the residual distribution of «cooling»).
JL
Y TT
J,
—2_
J—L.
> t
I_c
m
rn:
MM
■> t
-> t
_M
t
| d - delay before servicing I - moments of the query arrival in empty system i - moments of the query arrival in non empty system
- servicing
- «cooling» | h - «heating» | e - «empty system»
Fig. 3. Time diagram of QS operation with «heating» and «cooling» of service channels
Since the Laplace-Stieltjes transform (LST) of the convolution is equal to the product of the LST components [7], the desired distribution of the «delay» time before service in terms of LST is defined as
d(s) = h(s)f(s)
where h(s) — is the LST of the distribution of the duration of «heating», f(s) — is the LST of the residual distribution of the duration of «cooling».
According to [7], the initial moments of the residual distribution based on the initial moments of the distribution of the duration of «cooling» can be found by the formula:
ck + 1
fk =
(k + 1)C-
-, k = 1,2,...,
where k is the order of the initial moment of the distribution.
Further, the desired random delay before or after servicing can be found based on the convolution of distributions directly at the moments [7]. To do this, we will use symbolic decomposition
dk = (h + f)k,
in which, after the decomposition of the binomial, the exponents of the degree are translated into the indices of the corresponding moments. In particular, the first three initial moments are according to
di = hi + fi, d2 = h2 + 2hifi + /2,
d3 = h3 + 3h2fi + 3hj2 + f3.
The found approximation of random durations makes it possible to reduce the complexity of modeling based on the use of the QS model only with «heating» or only with «cooling» of service channels instead of using models with simultaneous consideration of both the «heating» and «cooling» process.
Model description
Let's consider a model of a multichannel QS with a «cooling» type M/M/n —H^. Recall that the second-order hyperex-ponential distribution refers to phase-type distributions and assumes the choice of one of two alternative phases by a random process [8]. With probability y1, the «cooling» process enters the first phase and a random time is delayed in it, distributed exponentially with the parameter ^. With probability y2 = 1 - y1, the process enters the second phase, where the exponential delay has a parameter ^2. The diagram of transitions between the states of the Markov process describing the system is shown in Figure 4.
The leftmost column in the diagram shows the current number of applications in the system and indicates the number of the chart tier. on each tier, the system is in one of three states: maintenance and cooling of one of two types. The cooling condition characterizes some work that is performed by the system after the end of servicing the last application. In this model, the cooling duration of the system is characterized by a two-phase hyperexponential distribution. After servicing the last application, the system switches to the cooling mode (phase) of the 1st type with probability yi and to the cooling mode of the 2nd type with probability y2. Each of the cooling phases is characterized
ro i—
O)
.2 «
£
o
(ft a>
Q)
(ft >
(ft <D
(ft
a3 n-1
n —
0
1_
ffi n
■Q 11
Cooling mode Type 1
Service mode
Cooling mode Type 2
I 2j.il K V
3//( Ml x V
{
Ml
nfi( \X V
Ml Ml
tlLli \JL u
, Ml W Ml ,
n+1
Fig. 4. Transition diagram for the system M/M/n — H£
by its intensity — and \i2 accordingly. The service of applications in the system is carried out with intensity ^ multiplied by the number of busy service channels. At full employment (on tiers with more n), the chart stabilizes on all the underlying tiers.
Based on the transition diagram of the markovized QS, transition intensity matrices are constructed and vector-matrix equations of transition balance between microstates are solved.
Method of probable-time characteristics calculation According [7] we denote by Sj the set of all possible microstates of the system in which exactly j applications are being serviced, and by Oj the number of elements in Sj. Next, in accordance with the transition diagram, we construct the intensity matrices of infinitesimal transitions:
4/[cjxcj+i] — in Sj+i (by arrival of queries), Bj [cjxcj-i] — in Sj-i (by completion of servicing), Cj[c/xcj] — in Sj (by completion of cooling), D/Ig/Xg/] — departure from the microstates of the j-th tier (diagonal matrix).
The size of the matrices is indicated in square brackets here and further. The element (i, k) of any of these matrices represents the intensity of the transition from the i-th state of the j-th tier to the k-th state of the adjacent (by transitions of the type under consideration) tier.
For QS M/M/n — H{>, the transition matrices will have the following form:
Aj =
X 0 0
0 X 0 , j = 0, N - 1
0 0 X.
0 0 0 0 0 0
i= M i 0 My 2 , B2 = 0 2m 0
0 0 0 0 0 0
Bj =
CJ =
0 0 0 L0
0 0 0
0
&
j = n, N ;
Dj =
A + ni 0 0
0
A + min( n, j)^ 0
0 0
A + &
j = 0, N .
Obviously, matrices A and C have the same appearance for all tiers of the diagram, and matrices B and D depend on the number of the tier, but at j > n they stabilize.
We introduce string vectors Yj = [Yj,1, Yj,2, Yj,3] to describe the location of QS in j-level microstates. Now let's write down the vector-matrix equations of the balance of transitions between the states [7] indicated in the diagram
Y0DO = Y0co + Y1^i-YjDJ = YJ-A-1 + YjCj + YJ+iBJ+i, i = ^ ,
where N is the number of calculated tiers of the diagram.
To solve this system of equations, we will use the iterative method proposed in [8] and modified in [7]. The direction of the run during the calculation will be determined «from top to bottom» - in the direction of increasing the number of the tier. As a result of the calculation, we obtain the ratio of adjacent probabilities of the number of applications in the system
Xj = Pj + i/Pj
j = 1, N.
After the end of the iterations, using the values of x, we will make the transition to the probabilities of the states of the system according to the following algorithm:
- put the probability of a free state p0 equal to 1;
- calculate pj+1 = pjXj, j = 1, N;
- calculate the sum S = Y.%0 Pj;
- normalize the received values: pj = Pj/S, j = 0, N.
calculation is also possible for systems without buffer limitation. In this case, it is assumed that the «tail» of the distribution of the number of applications is an infinitely decreasing geometric progression, the denominator of which is equal to the ratio of the last two calculated probabilities.
The waiting time for a newly arrived application is determined by the state of the system immediately before its arrival. In accordance with the PASTA (Poison Arrival Sea Time Average) theorem [9], the distribution of the number of applications before the arrival of the next one coincides with the stationary one. Before the arrival of the next application, the system is in one of three microstates - type 1 or type 2 cooling or servicing.
Let's say there were already Several queries in the system before the arrival of the next query. The probabilities of the corresponding microstates are presented by the string vector
Yj = [Yj,1, YjA Yj,3].
If the system was in the first microstate of the j-th tier (see diagram), the newly arrived application will wait for the completion of cooling of the first type plus (j - n + 1) queue promotions. LST of the appropriate time is
,j-n+1
œ,,i(X) =
Mi
( nM Y Vnu + s)
M? + 5 VnM + 5
In the third microstate, the second type of cooling occurs. LST of the corresponding expectation is
®i,3(s) =
M2
{ nM ) VnM + s)
j-n+i
M2 + s VnM + s
The second microstate corresponds to the maintenance mode. The LST of the waiting time for the newly arrived application will be
/ nM \
ra-2(s) = Wtx)
j-n+i
We introduce a column vector
js) = [Oj;1(s), ffj-,2(s), ffj',3(s)]T.
Then the LST of the desired waiting time for the start of service
m(s ) = X Y jaj(s) .
(1)
J =71
To obtain the initial moments of the waiting time, you can perform differentiation at the zero of expression (1), or replace all the LST included in it with sets of initial moments, and their products with convolutions in moments [7].
Results of the numerical experiment
Let's calculate the system on a simulation model and numerically using the ^-approximation for the following set of initial data: the number of service channels n = 3, the average intensity of the incoming flow 1 = 2.5, the average service time of applications b\ = 1, the average cooling time of the system C1 = 2. The parameters of the ^-distribution were selected according to three moments of different initial distributions of the cooling duration:
- degenerate D (coefficient of variation u = 0);
- uniform U on the interval [0; 4] (u ~ 0.577);
- exponential M (u = 1);
- gamma r with a shape parameter of 0.5 (u ~ 1.41);
- Weibull W with a shape parameter of 0.46 (u = 2.5).
The results of calculating the distribution of the number of
queries in the system for D, M and W distributions are shown in Figure 5. Shading shows graphs obtained using a simulation model, solid lines — based on the numerical calculation method.
The graphs show the agreement of the results even in the field of complex and paradoxical parameters of hyperexpo-nents. The Kolmogorov distance between the results obtained by the simulation method and by approximating the D, U, M, r and W distributions of the cooling time was {0.006; 0.005; 0.006; 0.004; 0.043} accordingly, this indicates an acceptable approximation accuracy. Note that for D, U distributions, the hyperexponent parameters took complex values, which, however, did not affect the correctness of the final result.
10'1
102
10 0 5 10 15 20 25 30
j
Fig. 5. Distribution of the queries number in the system M/M/n-
The increase in the Kolmogorov distance for a W-distribu-tion with a shape parameter of 0.46 may be due to the fact that it belongs to the class of distributions with a «thick tail» and, consequently, the random variable described by it has a very large spread. In this regard, the requirements for the quality of the software random number sensors used in the simulation model are increasing.
Table 1 presents the results of calculating the initial moments of the waiting time distribution, obtained numerically (Num) and using a simulation model (Sim).
Table 1
Initial moments of waiting time
It follows from the presented results that with an increase in the coefficient of variation of the distribution of the «cooling» time, the average waiting time increases. The similarity of the results obtained numerically and with the help of simulation modeling allows us to conclude that the proposed method of numerical calculation of the probabilistic-temporal characteristics of the QS with «cooling» is correct.
Conclusions
Based on the conducted research, it can be concluded that the process of functioning of a scalable microservice with container virtualization can be represented by a QS model with «cooling» of service channels.
When modeling, the time delays for performing microservice scaling operations can be represented as a generalized delay before the start of servicing applications. The calculation of the probabilistic-temporal characteristics of the process under consideration is possible based on the use of an iterative
method, the accuracy of which is confirmed by the results of simulation modeling.
The presented results can be used to substantiate the requirements for the characteristics of the infrastructure of a scalable service that can function stably under dynamic load changes.
References
1. Newman S. Sozdanie mikroservisov [Building Microservices]. Saint Petersburg, Piter Publishing House, 2016, 304 p. (In Russian)
2. Khazaei H. Perfomance Modeling of Cloud Computing Centers: A Thesis of the Degree of Doctor of Philosophy. Winnipeg, The University of Manitoba, 2012, 218 p.
3. Gindin S. I., Khomonenko A. D., Adadurov S. E. Chislen-nyy raschet mnogokanalnoy sistemy massovogo obsluzhivaniya s rekurrentnym vkhodyashchim potokom i «razogrevom» [Numerical Calculations of Multichannel Queuing System with Recurrent Input and «Warm Up»], Izvestiya Peterburgskogo universiteta putey soobshcheniya [Proceedings of Petersburg Transport University], 2013, No. 4 (37), Pp. 92-101. (In Russian)
4. Khalil M. M., Andruk A. A. Testing of Software for Calculating a Multichannel Queuing System with «Cooling» and E2-approximation, Intellektualnye tekhnologii na transporte [Intellectual Technologies on Transport], 2016, No. 4 (8), Pp. 22-28.
5. Khomonenko A. D., Gindin S. I., Khalil M. M. A Cloud Computing Model Using Multi-Channel Queuing System with Cooling, Proceedings of the XIX International Conference on Soft Computing and Measurements (SCMS2016), Saint Petersburg, Russia, May 25-27, 2016. Institute of Electrical and Electronics Engineers, 2016, Pp. 103-106.
DOI: 10.1109/SCM.2016.7519697.
6. Lokhvitsky V. A., Ulanov A. V. Chislennyy analiz sis-temy massovogo obsluzhivaniya s gipereksponentsialnym «okhlazhdeniem» [The Numerical Analyses of Queuing System with Hyperexponential Distribution of Cooling Time], Vestnik Tomskogo gosudarstvennogo universiteta. Upravlenie, vychislitelnaya tekhnika i informatika [Tomsk State University Journal of Control and Computer Science], 2016, No. 4 (37), Pp. 36-43. DOI: 10.17223/19988605/37/4. (In Russian)
7. Ryzhikov Yu. I. Algoritmicheskiy podkhod k zadacham massovogo obsluzhivaniya: Monografiya [Algorithmic approach to queuing tasks: Monograph]. Saint Petersburg, Mozhaisky Military Space Academy, 2013, 496 p. (In Russian)
8. Takahashi Y., Takami Y. A Numerical Method for the Steady-State Probabilities of a G1/G/C Queuing System in a General Class, Journal of the Operations Reseach Society of Japan, 1976, Vol. 19, No. 2, Pp. 147-157.
DOI: 10.15807/jorsj.19.147.
9. Ryzhikov Yu. I., Khomonenko A. D. Iteratsionnyy metod rascheta mnogokanalnykh sistem s proizvolnym zakonom ob-sluzhivaniya [Iterative Method for Analysis of Multichannel Queueing Systems with General Service Time Distribution], Problemy upravleniya i teoriya informatsii [Problems of Control and Information Theory], 1980, Vol. 9, No. 3, Pp. 203-213. (In Russian).
B(t) W1 W2 W3
Num Sim Num Sim Num Sim
D 1.788 1.709 6.674 6.495 37.190 34.160
U 1.971 1.870 7.864 7.230 40.460 37.220
M 2.272 2.225 1.035 1.035 62.880 64.020
To,5 2.582 2.494 1.406 1.356 108.000 103.900
W 2.600 2.802 2.586 2.476 767.000 687.700
Б01: 10.24412/2413-2527-2022-432-46-51
Русскоязычная версия статьи © В. А. Лохвицкий, В. А. Гончаренко, Э. С. Левчик опубликована в журнале «Интеллектуальные технологии на транспорте». 2022. № 1 (29). С. 39-44. Б01: 10.24412/2413-2527-2022-129-39-44.
Модель масштабируемого микросервиса на основе системы массового обслуживания с
«охлаждением»
д.т.н. В. А. Лохвицкий, к.т.н. В. А. Гончаренко, Э. С. Левчик Военно-космическая академия имени А. Ф. Можайского Санкт-Петербург, Россия lokhv_va@mail.ru, vlango@mail.ru
Аннотация. Предложена формализация процесса функционирования масштабируемого сервиса с учетом временных затрат на конфигурирование вычислительных узлов на основе многоканальной неэкспоненциальной системы массового обслуживания с «разогревом» и «охлаждением» каналов. Выполнено снижение сложности модели путем представления длительностей «разогрева» и «охлаждения» в виде обобщенной случайной задержки перед обслуживанием. Описаны основные этапы метода расчета вероятностно-временных характеристик системы массового обслуживания c «охлаждением». Марковизация немарковских компонент рассматриваемой СМО осуществлена с использованием Ш-распределе-ния. Получено распределение вероятностей состояний рассматриваемой СМО. Выполнено сопоставление результатов, полученных численным методом, с результатами имитационного моделирования.
Ключевые слова: микросервис, система массового обслуживания, «охлаждение» каналов, марковизация, обобщенная задержка обслуживания.
Литература
1. Ньюмен, С. Создание микросервисов = Building Microservices / Пер. с англ. Н. Вильчинского. — Санкт-Петербург: Питер, 2016. — 304 с. — (Бестселлеры O'Reilly).
2. Khazaei, H. Perfomance Modeling of Cloud Computing Centers: A Thesis of the Degree of Doctor of Philosophy. — Winnipeg: The University of Manitoba, 2012. — 218 p.
3. Гиндин, С. И. Численный расчет многоканальной системы массового обслуживания с рекуррентным входящим потоком и «разогревом» / С. И. Гиндин, А. Д. Хомоненко, С. Е. Ададуров // Известия Петербургского университета путей сообщения. 2013. № 4 (37). С. 92-101.
4. Khalil, M. M. Testing of Software for Calculating a Multichannel Queuing System with «Cooling» and E2-approxima-tion / M. M. Khalil, A. A. Andruk // Интеллектуальные технологии на транспорте. 2016. № 4 (8). С. 22-28.
5. Khomonenko, A. D. A Cloud Computing Model Using Multi-Channel Queuing System with Cooling / A. D. Khomonenko, S. I. Gindin, M. M. Khalil // Proceedings of the XIX International Conference on Soft Computing and Measurements (SCM'2016) (Saint Petersburg, Russia, 25-27 May 2016). — Institute of Electrical and Electronics Engineers, 2016. — Pp. 103-106. DOI: 10.1109/SCM.2016.7519697.
6. Лохвицкий, В. А. Численный анализ системы массового обслуживания с гиперэкспоненциальным «охлаждением» / В. А. Лохвицкий, А. В. Уланов // Вестник Томского государственного университета. Управление, вычислительная техника и информатика. 2016. № 4 (37). С. 36-43. DOI: 10.17223/19988605/37/4.
7. Рыжиков, Ю. И. Алгоритмический подход к задачам массового обслуживания: Монография. — Санкт-Петербург: ВКА им. А. Ф. Можайского, 2013. — 496 с.
8. Takahashi, Y. A Numerical Method for the Steady-State Probabilities of a G1/G/C Queuing System in a General Class / Y. Takanashi, Y. Takami // Journal of the Operations Reseach Society of Japan. 1976. Vol. 19, No. 2. Pp. 147-157.
DOI: 10.15807/jorsj.19.147.
9. Рыжиков, Ю. И. Итерационный метод расчета многоканальных систем с произвольным законом обслуживания / Ю. И. Рыжиков, А. Д. Хомоненко // Проблемы управления и теория информации. 1980. Т. 9, № 3. С. 203-213.
