CC BY
47
"RT&rA Mn 1 fRA^
PROBABILISTIC COMPARISONS OF SYSTEMS OPERATION QUALITY ^ .TA No, ^^
FOR UNCERTAINTY CONDITIONS Volume 15' March 2020
Probabilistic Comparisons of Systems Operation Quality
for Uncertainty Conditions
Andrey Kostogryzov1, Petr Kanygin2, Andrey Nistratov2
1 Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, Akostogr@gmail.com
2 Scientific and Engineering Centre for Energy Safety, pskanygin@gmail.com, andrey.nistratov@gmail.com
Abstract
For systems, including systems of intelligent manufacturing, information and intellectual systems, an achievement of operation quality at admissible expenses is very important and needs prognostic comparisons. Here the probabilistic approach to compare an operation quality of functionally similar systems for uncertainty conditions is proposed. To be compared there may be: different systems for an one operation time period or for different time periods with identical duration; or the same system for different time periods on time line. The system operation outputs are considered in the forms of material products, information products and products, combined from material and information products. For the given time of prediction the main results of the approach application are: a relative part of functions executed with admissible quality, estimations of expenses considering inadmissible system operation quality, a relative part of system operation satisfaction connected with quality and cost. The approach is demonstrated by examples.
Keywords: analysis, model, estimation, operation, prediction, probability, quality, system
I. Introduction
Different complex systems needs to be compared in life cycle by covering many special aspects. For example, the output results of many system operation can be both material and information products. For compared systems the important problem is to estimate their operation quality for uncertainty conditions during given long time in future considering information quality and costs. Many standards recommend to solve this problem by system analysis methods (for example, see ISO/IEC/IEEE 15288 "Systems and software engineering — System life cycle processes", ISO/IEC/IEEE 15939 "Systems and software engineering—Measurement process", ISO 31000 "Risk management - Principles and guidelines", IEC 61508 "Functional safety of electrical / electronic / programmable electronic safety-related systems, etc.). But as a rule for the system with hypothetical or expected conditions in future these methods in details are the matter for creation. Considering the practical needs here the analytical approach to compare complex systems operation quality for uncertainty conditions is proposed. The systems operation with material, information and combined outputs (for example, robotics) are researched.
The approach develops the existing approaches [1-14]. It can be useful for analysis and comparisons of systems operation quality, system optimization, to rationale of quantitative system requirements and engineering solutions. Another probabilistic models which can predict probability of success or risks on a level of probability distribution functions (PDF) [1-7, 11-14] may be also applicable through this approach.
II. The assumptions and ideas for the methodological approach
The proposed methodological approach is developed by the assumptions that:
• for the compared systems an expected quality for system operation may be achieved;
• in general case all outputs of compared systems may be divided into material products, information products and products, combined from material and information products;
• for analyzed period of comparison in life cycle the possible expenses of satisfaction from systems operation quality are comprehended or approximately estimated.
The methods and models are proposed by the use of next 4 main ideas.
Idea 1. For the system and if needs, for each system elements, intended for producing material, information and combined outputs it is necessary to be able to estimate quantitatively the achieved quality level on time line:
a) concerning material outputs - depending on the frequency of significant changes in quality, the frequency of control measures for recovering the admissible system operation quality and the mean time of recovering;
b) concerning information outputs - depending on the system possibilities for reliable and timely producing complete, valid and, if needed, confidential information;
c) concerning system complexity - depending on the given set of functions executed.
Idea 2. From practice for different conditions of uncertainties there may be compared different systems for an one operation time period or for different time periods with identical duration, or the same system for different time periods on time line. The functions may be executed with different material and/or information outputs.
Idea 3. For the defined set of system functions it is essential to estimate systems operation on the level of relative parts of functions executing with admissible quality for hypothetical or expected conditions. Thus it is understood that taking into account conditions of uncertainties the level 100% is never achievable.
Idea 4. For different conditions of uncertainties the systems operation quality may be compared on the level of relative parts of satisfaction connected with quality and costs.
III. Proposed model for probabilistic estimations
According to the assumptions and the ideas every function execution can be described and researched by the next general models concerning material, information and combined outputs. The models allow to estimate probability of function execution with admissible quality.
A probabilistic space (Q, B, P) is created [1-7], where: Q - is a limited space of elementary events; B - a class of all subspace of Q-space, satisfied to the properties of a-algebra; P - a probability measure on a space of elementary events Q. It is intended for probabilistic estimation of achieved quality level in function execution.
I. Model concerning material outputs
The next elementary events for function execution analysis are defined: "function execution with admissible system operation quality" and "function execution with inadmissible system operation quality". From the point of elementary event view at the moment "t" users need system ability to satisfy real requirements with the admissible quality. These requirements also are changed on time line. An elementary state of system operation can be changed on state «function execution with inadmissible system operation quality» instead of "function execution with admissible system operation quality" because of earlier significant changes in quality (at the moment t-x). Also a time for recovering inadmissible system operation quality requires. It means system ability not to satisfy real requirements with admissible quality at the moment t.
The essence of "Black box" model is described by Figure 1.
Figure 1: Some events describing elementary events connected with function execution quality (abstraction)
The next expression is proposed for estimation the probability of function execution with admissible system operation quality [1, 3, 5]:
[i—»(-!)]. (i)
Here
| is mean time between significant changes in items concerning admissible system operation quality, I-1 - is the frequency of significant changes in quality (considering changes of user needs);
q is mean time between control measures for recovering admissible system operation quality, q -1 - is the frequency of control measures for recovering admissible system operation quality;
b is mean time for recovering admissible system operation quality.
The proof see in [1, 3, 5].
For the practical use this means that the achieved quality level for material outputs can be estimated by the probability of function execution with admissible system operation quality (P) - it is calculated by (1).
II. Model concerning information outputs
The same space (Q, B, P) is built [1, 3, 5] and proposed for using because system operation output quality may be considered as a quality of special information system, which reliable and timely produces complete, valid and, if needed, confidential information requested - see Figure 2.
Figure 2: An information output quality (abstraction)
There are proposed for using the next "Black box" models and estimated measures [1, 3, 5]:
"The model of system operation by a complex system in conditions of unreliability of its components", the measures: Tmtbf - the mean time between failures; Prel.(Treq.)- the probability of reliable operation of system during the given prognostic period Treq.; Pman(Treq.) - the probability of providing faultless man's actions during the given prognostic period Treq.;
"The models complex of calls processing (for the different dispatcher technologies - for unpriority calls processing in a consecutive order for singletasking processing mode, in a timesharing order for multitasking processing mode; for priority technologies of consecutive calls processing with relative and absolute priorities; for batch calls processing; for combination of technologies above), the measures: Ptim - the probability of well-timed processing during the given prognostic period; the relative portion of all well-timed processed calls; the relative portion of well-timed processed calls of those types for which the customer requirements are met Ctim;
"The model of entering into system current data concerning new objects of application domain", the measure: Pcompl - the probability that system contains complete current information about states of all objects and events;
"The model of information gathering", the measure: Pactual. - the probability of information actuality on the moment of its use;
Note. This model is similar mathematically to the model III.I for material outputs.
"The model of information analysis", the measures: Pcheck is the probability of errors absence after checking; the fraction of errors in information after checking; Pprocess - the probability of correct analysis results obtaining; the fraction of unaccounted essential information;
"The models complex of dangerous influences on a protected system", the measures: Pinfl.(Treq.) - the probability of required counteraction to dangerous influences from threats during the given prognostic period Treq.;
"The models complex of an authorized access to system resources", the measures: Pprot - the probability of providing system protection from an unauthorized access by means of barriers; Pconf.(Treq.) - the probability of providing information confidentiality by means of all barriers during the given prognostic period Treq..
III. Model concerning outputs which are combination of material and information outputs
For this case the models from subsections III.I and III.II are proposed to estimate every function execution quality - see Figure 3 and section IV. All the proposed models may be applied and improved for solving the problem to estimate and compare prognostic system operation quality for uncertainty conditions during given long time considering information quality and costs.
or this model, /' or this model,
(for materialproducts) \ (for information products)
A set of functions performed '■
1 . . . M
— . . . - . . ' '
or combination of these models
(for combinedpjoducts)
Figure 3: Variants for a choice of the model for every function
IV. Estimation of relative parts of functions executed with admissible quality
The next formula is proposed for calculation a relative part of functions executed with admissible system operation quality during the given prognostic period (here hypothetical conditions also may be considered):
m / M
S = VP ■ a /Va (2)
quality / , m hyp. m / , hyp. m ^ '
m=1 I m=1
where auyp. m is frequency of m-th type function execution during the given prognostic period; Pm is the probability of m-th function execution with admissible system operation quality; M is full set of essential functions which are executed by the system and are considered in
"RT/Ê À Mn 1
PROBABILISTIC COMPARISONS OF SYSTEMS OPERATION QUALITY . .TA , FOR UNCERTAINTY CONDITIONS_Volume 15, March 2020
comparisons.
In general case for every m (1<m<M) the probability Pm is calculated by the next variants:
a) concerning material outputs Pm is calculated by the model (1);
b) concerning information outputs Pm is calculated by formula:
Pm(Treq.) =Prel.m(Treq.) •Ctim m'Pcompl..m 'Pactual..m •Pcheck m 'Pprocess.m •Pinfl. (Treq.) •P man m(Treq.) 'Pprot.m 'Pconf.m (Treq.),
(3)
where all measures are calculated by the models, proposed in subsection III.II;
c) for a combination of material and information outputs
Pm(Treq.) = Pcombined.m • Prel
.m(Treq.) • Ctim m• Pcompl..m • Pactual..m • Pcheck m • Pprocess.m • Pinfl. m (Treq.) •
•Pman m(Treq.) • Pprot.m • Pconf.m (Treq.), (4)
where Pcombined.m is calculated by the model (1), and all the others measures are calculated by the models, proposed in subsection III.II.
For material outputs the result of calculation Pm by (1) means probability that m-th function is executed with admissible quality. For information outputs the results of calculation Pm by (3) means probability that m-th function is executed with admissible quality, i.e. requested information outputs are reliable and timely produced, are complete, valid and, if needed, confidential for the purpose use. For outputs, combined from material and information products, the results of calculation Pm by (4) means probability that the m-th function is executed with admissible quality according to material outputs and the requested information outputs are reliable and timely produced, are complete, valid and, if needed, confidential for the purpose use.
For calculation a relative part of functions executed with admissible operation quality for the past compared conditions during the given prognostic period the next formula may be used:
M I M
S quality y Ureal m a real ml ^ a real m ( )
m=1 l m=1
where areai m is real frequency of m-th type function execution (or considered as real according to assumption) during the given prognostic period;
Ureal m is real part of functions executed with admissible operation quality (it is measured from 0 to 1).
If the definition of Um in (5) is a problem, the formula (2) may be used for the conditions (different or identical to past conditions) during the same given time, for this case Um
V. Estimation of expenses in life cycle
If inadmissible system operation quality is not considered the next formula is proposed for the estimations of expected expenses [13]:
M
C (t) = C. + C (t) + ty a, C. for t < TVf,
exp / instal. mamA / / ; hyp. m hyp. m J life
m=1
M
and Cexp.(t) = Cinstal. + Cmain.(t) + ^y ahyp. mChyp. m + Cdisposal t > Tlife (6)
m=1
where Ohyp.m is a frequency of m-th function execution; the expected or real costs are indicated:
"RT/Ê A Mr» 1
PROBABILISTIC COMPARISONS OF SYSTEMS OPERATION QUALITY . .TA , ™,6,) FOR UNCERTAINTY CONDITIONS_Volume 15, March 2020
Cimtal - for system development and installation;
Cmain (t) - for system maintenance during time t;
CA m - for system operation in time unit for m-th function execution (for example, in a year
if time t is expressed in years);
Cdisposal - for system disposal; Tiife - system life time.
Considering inadmissible system operation quality for the moment t (i.e. mathematical expectation of expenses) the next formula is proposed for the estimations of expenses:
Cmat,(t) = CimM+ CmaJt) + £ (dhypmChyPm "t ' Pm + Dm ' Nm(t ) i1 - Pm )) ° t < T^ m=1
Cmath(t) = CmstaL + Cmam.(t) (fa^yp mChyp m ' Tlife ' Pm + Dm ' Nm(TlifeJ' (l - Pm ))_
m=1
+ Cdisposal for t > Tlife, (7)
where the probability of m-th function execution with admissible system operation quality (Pm) is calculated by (1), (3)-(4) in dependence on the chosen model for m-th function;
- a possible or real damage for inadmissible system operation quality of system for one loss of quality;
Nm (t) - a prognostic number of damages from installation to moment t.
VI. Estimation of the relative parts of system operation satisfaction connected with quality and costs
Let two systems are compared.
The next formula is proposed for calculation a relative part of system operation satisfaction connected with quality:
Squality = (Squality l/ Squality 2)'100%, (8)
where the relative part of functions executed with admissible operation quality for l-st system Squality i and for the 2-nd compared system during the given prognostic period Squality 2 are calculated by (2) or (5).
For calculation a relative part of system operation satisfaction connected with costs for one system the next formula is proposed:
Scost (t) = [Cmath. (t) / Cexp.(t)] "100%, (9)
where the expected expenses for satisfying quality requirements, if inadmissible system operation quality is not considered, is calculated by (6). The mathematical expectation of expenses for satisfying quality requirements, considering inadmissible system operation quality, for the moment t is calculated by (7).
For a preferability of the l-st system in comparison with the 2-nd system the relative part of system operation satisfaction connected with quality Squality should be more 100%. And for system operation satisfaction connected with costs a relative part Scost (t) should be less than 100%.
Some parts from the described methodological approach for probabilistic estimations are supported by the different versions of software Complex for Evaluation of Information Systems Operation Quality (CEISOQ+, registered by Rospatent №2000610272) and the software tools "Mathematical modeling of system life cycle processes" - "know how" (registered by Rospatent №2004610858) [1, 3, 5].
VII. Examples
Example 1. This example summarizes the numerous calculation results in applications the approach to intellectual systems of government agencies, manufacturing structures (including power generation, oil-and-gas systems), emergency services etc. [1, 3, 5-6, 8-15]. The typical estimations of measures for information outputs are presented by Table 1.
Table 1: Knowledge from the best practice
The typical estimations of measures for information outputs quality
Model tittle
Limits for measure value
The model of system operation by a complex system in conditions of unreliability of its components
The models complex of requests processing for the different dispatcher technologies
The model of entering into system current data concerning new objects of application domain The model of information gathering
The model of information analysis
The models complex of an authorized access to system resources
The models complex of dangerous influences on a protected system
Pa - no less than 0.99 Pman - no less than 0.95
Ctim - no less than 0.95 Pcompl. - no less than 0.9
Pactuai. - no less than 0.9
Pco- no less than 0.9 7 aPnpraocleyss.s-i snroelseuslst st hoabnta0i.n9i5n
Pprot. - no less th5n 0.99 Pconf. - no less than 0.999
Pinfi. - no less than 0.95
These estimations are confirmed also by statistical measuring data of the operation quality of the real monitored objects of dangerous manufacturing [8, 13].
Example 2. Suppose a special intellectual control system (SICS) is planning to create for monitoring intelligent manufacturing. Considering results of example 1 there is estimating a SICS operation during its life cycle. For simplification two types of function are to be executed. An output of each function is combination of material and information products. It means example is focused on using basic formula (4). Assumption is: to simplify this example we use only calculation measure for material output and identical "The model of information gathering", which is similar mathematically to the model III.I for information output. Another measures are constant on the level of Table 1 values.
So, whole set of functions is divided into 2 types - with more urgent (m=1) and less urgent (m=2, M=2) execution:
- for 1st type of functions (m=1) significant changes concerning information outputs produced for users occur once a month (|=1 month). The gathering, preparation and checking of data for entry into SICS b=2 hours, the system update of data after checking occurs once a day (q=1 day);
- for 2nd type of functions (m=2) - significant changes concerning material and information outputs relating to any of served users, also occur once a month (|=1month), the gathering, preparation and checking of outputs b=3 days, system update after checking occurs once a week (q=1 week).
At the stage of development this SICS was constructed on the assumptions that each year is
RT^rA Mr» 1 i'Ri^
PROBABILISTIC COMPARISONS OF SYSTEMS OPERATION QUALITY . .fA , FOR UNCERTAINTY CONDITIONS_Volume 15, March 2020
about 20 million requests for execution of the 1st type function (ahyp. 1) and 80 million type 2 requests (ahyp. 2). An expense about 705 million of cost conditional units (c.u.) during development and 5 years of operation is considered for satisfying quality requirements.
However, according to the real results of the 1st year of operation, the number of requests of the 1st type (areal 1) amounted to 10 million, while the number of requests of the 2nd type (areal 2) is 190 million. At the same time, the frequency of significant changes relating to any of served users doubled to 2 times a month.
The comparisons in advance are needed.
At the development stage a prognostic degree of satisfaction of quality is due to be estimated. It can allow to develop rational technical solutions. At the beginning of SICS operation, as data are gathered, a prediction of prognostic satisfaction needs to take a reasonable improvement of the SICS operation. What about the results of the probabilistic estimations?
The next additional data are used for input definition: Cinstal =200 m c.u.; Cmain. (1 year) =1 m c.u.; C m = 1 c.u. for 1 request; N1(1 year)=0, N2(1 year)=0.01%, D2=10.000 c.u.
The results of modeling by formulas (1), (3),(4) have presented by Figure 4 - the probabilities of function execution with admissible quality are:
- on development stage P1=0.98, P2=0.81;
- on operation stage P1=0.96, P2=0.67.
Evaluations of the relative hypothetical and the real parts of system operation connected with quality and costs satisfaction are the next - a relative prognostic part of functions executed with admissible quality:
on development stage is Squality = 0.844. It means 84.4% requests are satisfied;
after 1-st year of SICS operation Sreal = 0.685, it means 68.5% requests are satisfied considering changes in requests flows.
On development stage, when inadmissible system operation quality is not considered, an prognostic expenses for satisfying quality Cexp. (5 years) = 705 m c.u., on operation stage considering inadmissible system operation quality Cmath. (5 years) = 998 m c.u.
A relative part of system operation satisfaction connected with quality Squality =81.2%, with costs Scost (t)=141.6%.
Figure 4: Probability of 1-st and 2-nd type function execution with admissible system operation quality in dependence on frequency of significant changes in quality (E1, times in a month)
If on development stage the level of 84.4% for a relative prognostic part of executed functions is acceptable, the level 68.5% (of requests executed with admissible quality) on operation stage means only 81.2% from accepted level. Moreover this result is achieved by the cost 41.6% over the admissible level. Such efficiency can't be estimated as satisfied for analyzed SICS.
Example 3. What about the comparable pragmatic effects? Authors of this article took part in creation of the Complex of supporting technogenic safety on the systems of oil&gas transportation
and distribution. This Complex has been awarded by the Government of the Russian Federation in the field of a science and technics for 2014. Here peripheral posts are equipped additionally by means of monitoring operator actions to feel vibration, a fire, the flooding, unauthorized access, hurricane, and also intellectual means of the reaction in time, capable to recognize, identify and predict a development of extreme situations - see engineering decisions on Figure 5. Applications of Complex for 200 systems in several regions of Russia during the period 2009-2014 have already provided no accidents and economy about 8,5 Billions of Roubles. The economy is reached at the expense of effective implementation of adequate probabilistic modelling, risks prediction, justification of preventive measures against risks, processes optimization [10].
Figure 5: Some elements of the Complex of technogenic safety on the systems of oil&gas transportation and distribution
Conclusion
The proposed probabilistic approach allows to compare different systems for an one operation time period or for different time periods with identical duration or the same system for different time periods on time line. The approach can be useful for analysis and comparisons of systems operation quality, system optimization, to rationale of quantitative system requirements and engineering solutions for user satisfaction. The efficiency from implementation in life cycle of complex system is commensurable with expenses for system creation.
References
[1] Kostogryzov, A. and Nistratov, G. Standardization, mathematical modelling, rational management and certification in the field of system and software engineering. Armament.Policy.Conversion, Moscow, 2004.
[2] Zio, En. An Introduction to the Basics of Reliability and Risk Analysis, World Scientific Publishing Co.Pte.Ltd., 2006.
[3] Kostogryzov, A.I. and Stepanov, P.V. Innovative management of quality and risks in systems life cycle. Moscow, Armament.Policy.Conversion, Moscow, 2008.
[4] Kolowrocki, K. and Soszynska-Budny, J. Reliability and Safety of Complex Technical Systems and Processes, Springer-Verlag London Ltd., 2011.
[5] Kostogryzov, A., Nistratov, G. and Nistratov, A. Some Applicable Methods to Analyze and Optimize System Processes in Quality Management. Total Quality Management and Six Sigma, InTech, 2012: 127-196.
[6] Akimov, V., Kostogryzov, A., Mahutov N. at al. Security of Russia. Legal, Manufacturing&Economic and Scientific&Engineering Aspects. The Scientific Foundations of Technogenic Safety. Under the editorship of Mahutov N.A. Znanie, Moscow, 2015.
[7] Eid, M. and Rosato, V. Critical Infrastructure Disruption Scenarios Analyses via Simulation. Managing the Complexity of Critical Infrastructures. A Modelling and Simulation Approach, SpringerOpen, 2016: 43-62.
[8] Artemyev, V., Kostogryzov, A., Rudenko, Ju., Kurpatov, O., Nistratov, G., Nistratov, A. (2017). Probabilistic methods of estimating the mean residual time before the next parameters abnormalities for monitored critical systems. In: Proceedings of the 2nd International Conference on System Reliability and Safety (ICSRS), Milan, Italy, 2017: 368-373.
[9] Kostogryzov, A., Atakishchev, O., Stepanov, P., Nistratov, A., Nistratov, G., Grigoriev, L. (2017) Probabilistic modelling processes of mutual monitoring operators actions for transport systems. In: Proceedings of the 4th International Conference on Transportation Information and Safety (ICTIS), Canada, Banff, 2017: 865-871.
[10] Kostogryzov, A., Grigoriev, L., Golovin, S., Nistratov, A., Nistratov, G., Klimov, S. (2018) Probabilistic Modeling of Robotic and Automated Systems Operating in Cosmic Space. In: Proceedings of the International Conference on Communication, Network and Artificial Intelligence (CNAI), Beijing, China. DEStech Publications, Inc., 2018: 298-303.
[11] Artemyev, V., Rudenko, Ju., Nistratov, G. Probabilistic modeling in system engineering. Probabilistic methods and technologies of risks prediction and rationale of preventive measures by using "smart systems". Applications to coal branch for increasing Industrial safety of enterprises, IntechOpen, 2018: 23-51.
[12] Kershenbaum, V., Grigoriev, L., P. Kanygin, Nistratov, A. Probabilistic modeling in system engineering. Probabilistic modeling processes for oil and gas systems, IntechOpen, 2018: 5579.
[13] Kostogryzov, A., Panov, V., Grigoriev, L., Zubarev, I., Nistratov, A., Nistratov, G. (2019) Estimation of stakeholders satisfaction in application to socially significant systems. In: Proceedings of the Proceedings of the 6th International Conference Actual Problems of System and Software Engineering, Moscow, Russia, 12-14 November, 2019: 10-16.
[14] Kostogryzov, A., Korolev, V. Probabilistic Methods for Cognitive Solving of Some Problems in Artificial Intelligence Systems, IntechOpen, 2019. Doi: http://dx.doi.org/10.5772/intechopen.89168
