CC BY
60
ACCURACY OF RELIABILITY CALCULATED BY THE MONTE CARLO SIMULATION METHOD
Dušan Đ. Ostojić, Technical Test Center, Belgrade,
Slavko J. Pokorni, College of Professional Studies Information Technology, Belgrade Predrag I. Rakonjac, Technical Test Center, Belgrade, Dragoljub M. Brkić
DOI: 10.5937/vojtehg1204047O
FIELD: Electronics, Information Technology ARTICLE TYPE: Original Scientific Paper
Sumary:
Reliability is the main indicator of the quality of special purpose equipment/systems, which are in their service life exposed to extreme operating modes and environmental conditions. Reliability of electronic equipment/systems is difficult to determine analytically, specially for repairabile systems with a large number of elements because of a large number of possible system states, which requires setting up and solving a system of a large number of equations. Therefore, simulation methods are applied to determine the reliability of electronic equipment/systems. This paper examines the accuracy of the point estimate of reliability calculated by the Monte Carlo simulation method depending on the sample size n, and the number of iterations (repetitions of calculation), Np.
Key words: electronic system, reliability calculation, accuracy, Monte Carlo method, simulation.
Introduction
Reliability is one of the important indicators of quality of special purpose equipment/systems which are in their service life exposed to extreme operating modes and environmental conditions. Hence, it is important for the users of such a system to know what its reliability is.
Ostojić, D., et al., Accuracy of Reliability Calculated by the Monte Carlo Simulation Method, pp. 47-58
VOJNOTEHNIČKI GLASNIK/MILITARY TECHNICAL COURIER, 2012., Vol. LX, No. 4
In this paper, the reliability of electronic equipment/system is defined as a probability, with a certain level of confidence, that it will successfully perform the function for which it was intended, in the time interval from 0 to t, under the prescribed operating regimes and environmental conditions.
In general, the reliability of electronic equipment/systems is difficult to determine by analytical methods as they have a large number of elements (usualy in states operational or faulty), and according to that, a large number of possible states of the system, which requires setting up and solving a system of a large number of equations. Simulation methods are, therefore, more often applied to determine the reliability of electronic systems.
For this purpose, the authors of this paper have developed an appropriate mathematical and physical simulation model and a computer program that support the calculation of the reliability of electronic equipment/systems based on the application of the Monte Carlo simulation method. The simulation software package for calculating reliability is first checked on examples of simple electronic systems, for which it was possible to determine the analytical solutions. The comparative review of the obtained results by the simulation and analytical methods for the same electronic systems, subject to an exponential distribution, showed very close proximity, which was a confirmation of the correctness of the application of the developed simulation software package for the calculation of reliability.
In the reliability calculation using the Monte Carlo simulation method it was presumed that the equipment/system is built entirely of electronic elements and the period of normal operation, when failure rates are constant, because failures occur mostly by accident. Then we can use an exponential distribution to approximate the distribution of failures.
Because of the importance to know the reliability of modern military equipment, this paper analyzes the accuracy of the point reliability assessment, obtained using Monte Carlo simulation methods [1], depending on the sample size n and the number of iterations (repeating the calculation), Np.
Basic assumptions in the Monte Carlo reliability calculation method
It is very difficult to give a precise and complete definition of the Monte Carlo method. However, we can say that this is the numerical method for solving complex mathematical, statistical, physical, telecommunication and other problems with the random selection of samples. The basic idea of the Monte Carlo method is to build a stochastic model that is consistent with the real problem or is the direct simulation of the problem. In both cases, the
element of chance is introduced, making a large number of computer "experiments" - iterations and eventually implement their statistical analysis. In other words, the idea of the Monte Carlo method is: instead of describing a random phenomenon by analytical relations, we perform the simulation of this phenomenon in order to obtain its realization. The realization is performed by a simulation of random numbers. As a result of each repetition of such a procedure - iteration, one realization of the studied phenomenon is obtained.
The advantage of the application of simulation methods is that very complicated mathematical problems, which usually have no analytical solution, can easily be solved. The disadvantage of the method is a large number of iterations required to achieve the required accuracy, which means long computing time. However, with the rapid development of computer technology, this problem is becoming less significant and the Monte Carlo method is increasingly used in many fields of science and technology.
The Monte Carlo method is based on the stohastic nature of processes whose states are determined by the laws of probability. Theoretical ideas about the phenomena being treated in the probability distributions of random sizes are the base of the application of the Monte Carlo method for numerical iteration. The Monte Carlo simulation method of the stohastic system leads to solutions to problems which are impossible to obtain analytically. The idea is that taking into account the characteristics of a system, random numbers are assigned to system variables, and by series of simulations and appropriate statistical analysis of the expected results we can draw conclusions (e.g. the expected average value of a variable, reliability of a system, etc.).
For constant failure intensity, reliability of electronic equipment is determined by the following formula [2] (so-called exponential law of reliability):
m=Hf- m) (1)
where t - is the required operating time without failure, and m-mean time to/between failures of equipment.
The reliability parameter m is in practice (for example laboratory or field tests) determined by its point estimation by the following relation [3]:
1
m = — n
n
z >> ■
i=1
(2)
where t, is the time to the i-th failure, and n is the total number of failures.
(7T>
Ostojić, D., et al., Accuracy of Reliability Calculated by the Monte Carlo Simulation Method, pp. 47-58
VOJNOTEHNIČKI GLASNIK/MILITARY TECHNICAL COURIER, 2012., Vol. LX, No. 4
When in the reliability relation m is replaced with m, we get the estimated reliability or the point estimation of reliability:
For non-repairable/repairable systems, m or m represents the mean time to/between failures (MTTF/MTBF). The accuracy of the estimated reliability depends on the accuracy of the estimated mean time to/between failures, and the accuracy of this depends on the number of failures. Therefore, it is necessary to have a large number of times to/between failures to accurately determine the mean time to/between failures. When applying the Monte Carlo simulation method, times to/between failures are not coming from laboratory or field tests of equipment, but are software generated as pseudorandom variables using the relation:
ti = -m • ln r, (4)
where
m - adopted theoretical values of the mean time to/between failures and r, - pseudorandom number whose value is between zero and one,
r, e(0,1).
In the computer program, made for this purpose, pseudorandom numbers ri are obtained with the computer random generator.
The basic input data for calculating the reliability of electronic equipment/system is the mean time between failures which is given by the formula:
where X is the failure intensity rate of the element.
The mean time to/between failures was chosen as the input data because, in practice, this parameter for the electronic elements can be obtained in several ways. One, quite frequent, is the calculation based on standardized manuals for reliability, such as, for example, a military handbook (standard) MIL-HDBK-217 F. Next, manufacturers of electronic equipment/systems with special purpose have to provide information on the mean time between failures for these systems on request.
(3)
MTBF = m =
X
(5)
Dependence of the range of values of the point estimation of reliability of the sample size n
For a given operating time without failure of the equipment (t = Tz), adopted theoretical value for m, the number of random variables t,- or the sample size n, and the number of repetitions of calculations Np, using the computer program developed by the authors, the range of values R of the estimation of the reliability is defined by the following relation:
) = Rmax — Rmin- (6)
The 19 different values of the sample size, provided in this computer program, are obtained using the following relation:
ni =kma*/20J-i ; i = 1.2,... 19 (7)
where nmax - is the maximum sample size which is preset (for example nmax
n
= 400, n1 = 20 etc.), the integer value of expression Q = ~Jmax is taken into
20
account denoted by LQ J. The ranges of values R depending on the sample size n, for t = Tz = 24 h (hours), m = 2000 h, Np = 5000 and nmax = 400, determined by this program, are shown in Figure 1.
Figure 1 - Dependence of W(R,) of n Slika 1 - Zavisnost W(R) od n
Ostojić, D., et al., Accuracy of Reliability Calculated by the Monte Carlo Simulation Method, pp. 47-58
VOJNOTEHNIČKI GLASNIK/MILITARY TECHNICAL COURIER, 2012., Vol. LX, No. 4
As shown in Figure 1, when the sample size increases, the range of values for reliability narrows, at first rapidly, and then more and more slowly, as a result of the nature of statistical convergence. The points on the diagram are mean values of the reliability calculated in these ranges.
Probability density function of the point estimation of reliability
Based on the adopted values for the mean time to/between failures m, the required operating time without failure t = Tz and the sample size n, the n pseudo-random time to/between failures is generated. Based on this, the estimated value m is calculated, and then a point estimate of the reliability
R(t) is calculated based on m . This procedure is repeated Np times (Np should be as large as possible in order to obtain higher accuracy). For m = 2000 h, t = Tz = 24 h (estimated value of the reliability by relation (1) is R(t) = 0.9880717), n = 50 and Np = 5000, using the developed computer program we got results, shown in Figure 2 in the form of a histogram and a graph of the estimated probability density function of the point estimation of reliability.
Since the values of the point estimation of the reliability R(t) are between 0 and 1, we adopted beta distribution for the probability density function in the interval A-B, where A is the begining and B is the end of the distribution with shape parameters: a and b.
The probability density function of the point estimation of reliability is given by the following relation:
f (x)
1
(B - A) • B(a, b)
x - A
B-A
a-1
•11
x - A
b-1
B-A
(8)
where is A < x < B, a, b > 0 and B(a, b) is the beta function which can be determined using complete gamma functions by the following formula [4]:
B(a, b)
Г(а) -r(b) Г (a + b)
(9)
In the function f (x), in the formula (8), xshould be replaced with R . The distribution parameters A, B, a and b can be determined using different methods, such as: maximum likelihood method, method of moments, quantila methods, etc. In this computer program, the point
estimates of these parameters were determined using the Monte Carlo simulation by the following relations:
A = Rmin - 0.01-h + 0.01 • h • r
Bi = Rmax - 0.01 • h + 0.01 • h • r
cii = 0.05 + 29.95 • r b = 0.05 + 29.95 • r
where r - the pseudo-code ri G C0-1), h - the class histogram width.
Figure 2- Histogram and graph of the probability density function f (R) Slika 2 - Histogram i grafik funkcje gustine f (R )
These point estimations of the parameters are put in the relation for the probability density function and then the values of this function in the center of each class histogram can be compared with the height of the
($>
Ostojić, D., et al., Accuracy of Reliability Calculated by the Monte Carlo Simulation Method, pp. 47-58
VOJNOTEHNIČKI GLASNIK/MILITARY TECHNICAL COURIER, 2012., Vol. LX, No. 4
histogram class. In other words, the square differences between the value of the function and the class height are found and then these square differences are summarized for all classes of the histogram. Then we repeat the calculation of the parameters and adopt those with the smaller difference. The procedure is repeated for a very large number of times and we adopt the set of parameters which is obtained with the smallest sum of squares of the observed differences. This procedure is mathematically simple, but must be repeated many times, so it requires the use of a computer. The results are satisfactory, as it can be seen in Figure 2, where the estimated probability density function is suited well with the histogram, as well as the corresponding empirical distribution density function.
For m = 2000 h , t = Tz = 24 h (estimated value of the reliability R(t) = 0.988071713), n = 50 and Np = 5000, using the developed computer program, we obtained the following values of the point estimates of these parameters:
A = 0.977795
B = 0.992203
a = 9.652830
b = 4.205322.
The mean value is R = 0.987830808, the standard deviation is s = 0.020303677, and the confidence limits are R1 = 0.984149310 and R2 = 0.990742220 with the risks a = a> = 0.025.
Relative error in determining the reliability
By definition, reliability is the probability that the operating time without failure of the equipment is greater than a required time Tz.
R(t) = P(A) = P(t > Tz), (11)
where the event A hapens, under defined conditions, when the operating time without failure is greater than the required time, ie t > Tz. If the true (unknown) reliability is very high, then the event A happens more often, and the probability P(A) will be more accurately determined with the same number of data n (sample size or the number of times to/between failures) than when the reliability is low.
For 20 different theoretical (considered as real) values, the point estimation of reliability was carried out, according to the procedure described in the foregoing paragraphs of this work, and the relative errors RG were determined using the formula [5]:
RG = 100 (R - R)/R [%]. (12)
where R is true, and R the corresponding point estimation of reliability.
For, t = Tz = 24 h, n = 2000000, and the theoretical value R = : 0.050, 0.100, 0.150, ... 0.950, 0.99, R= {0.050, 0.100, 0.150, ... 0.950, 0.99}, using the computer program the relative errors in estimating the reliability were determined, which is graphically shown in Figure 3. We used a sample n = 2000000 purposely high, in order to get less deviation of the estimated reliability from the true reliability. As shown in the diagram (Figure 3), the relative error is much smaller for the high values of the true reliability than for the lower values, what was our assumption at the beginning of the paper.
Figure 3 - Graphical display of the relative error as a function of R(t)
Ostojić, D., et al., Accuracy of Reliability Calculated by the Monte Carlo Simulation Method, pp. 47-58
VOJNOTEHNIČKI GLASNIK/MILITARY TECHNICAL COURIER, 2012., Vol. LX, No. 4
Conclusion
This paper examined the accuracy of the point estimation of reliability depending on the sample size n, and the number of iterations (repetitions of calculation), Np.
It is shown that the sample size n has a greater effect on accuracy than the number of iterations Np, because if a sample is too small, we cannot obtain greater improvement of accuracy with a larger number of iterations. However, the determination of the parameters of the distribution of points estimation requires a very large number of iterations.
The application of the Monte Carlo simulation method [6] is proved to be very suitable in the estimation of distribution parameters, especially when the number of these parameters is greater than 3, because in this case the application of known methods, as the maximum likelihood method and other methods, is very complex. The accuracy of the estimation of the parameters of the distribution using this simulation method is satisfactory, and the application itself is very simple.
It should be noted that the Monte Carlo simulation method requires the application of a computer.
Literature
[1] Nikolić, V. N., Accuracy control in Monte Carlo simulations, Military Technical Courier/Vojnotehnički glasnik, Vol. 58, No. 2, pp. 90-107, 2010.
[2] Chapouille P., De Pazzis, R., Fiabilite des Systemes, Masson et Cie, Paris, 1968.
[3] Fisz, M., Mathematische Statistik und Wahrscheinlichkeitsrechnung, VEB Deutscher Verlag der Wissenschaften, Berlin, 1962.
[4] Mitrinovic, D., Djokovic, D., Special Functions, Building book, Belgrade, 1964.
[5] Van Der Waerden, L., Mathematische Statistik, Springer-Verlag, Berlin, 1965.
[6] Ostojic, D., Communication network reliability determination by simulation method and optimal path selection, Ph.D. thesis, Military Academy, Belgrade, 2010.
PROVERA TAČNOSTI PRORAČUNA POUZDANOSTI ODREĐENE PRIMENOM SIMULACIONE METODE MONTE CARLO
OBLAST: elektronika, informacione tehnologje VRSTA ČLANKA: originalni naučni članak
Sažetak:
Pouzdanost je najvažnji logistički parametar uređaja/sistema speci-jalne namene obzirom da su isti u svom radnom veku izloženi maksimal-nim radnim režimima i ekstremnim klimomehaničkim ulovima okoline.
Otuda je za korisnika sistema specjalne namene važno da zna sa kakvim performansama sistema raspolaže.
Kako je u ovom radu reč o elektronskim uređajima, za koje se mo-že smatrati da u svom radnom veku imaju konstantnu funkciju intenzite-ta otkaza, to je za proračun pouzdanosti primenjena eksponencjalna raspodela.
Pouzdanost elektronskih uređaja/sistema teško je odrediti anali-tičkim putem jer obično imaju veliki broj elemenata, a to znači veliki broj mogućih stanja sistema, što zahteva postavljanje i rešavanje sistema sa velikim brojem jednačina. To je razlog da se za određivanje pouzdanosti elektronskih uređaja/sistema sve više primenjuju simulacio-ne metode.
Autori ovog rada razvili su odgovarajući matematičko-fizički simu-lacioni model, kao i računarski program koji ga podržava, za proračun pouzdanosti elektronskih uređaja/sistema zasnovan na primeni simula-cione metode Monte Carlo. Softverski paket proveren je najpre na pri-merima jednostavnjih elektronskih uređaja/sistema, za koje je bilo mo-guće odrediti analitička rešenja za pouzdanost. Poređenjem rezultata za pouzdanost dobjenih simulacionom metodom i analitičkim relacja-ma, za iste elektronske uređaje/sisteme, utvrđena je velika bliskost dobjenih rezultata, što je bila potvrda ispravnosti primene razvjenog si-mulacionog softverskog paketa za proračun pouzdanosti.
Obzirom na značaj pouzdanosti savremenih vojnih uređaja, u ovom radu analizirana je tačnost proračuna pouzdanosti, dobjene pri-menom simulacione metode Monte Carlo, u zavisnosti od veličine uzorka n (broja vremena bezotkaznog rada do/između otkaza) i broja iteracja (ponavjanja izračunavanja), Np.
Na početku rada polazi se od osnovnog ulaznog podatka za pro-račun pouzdanosti, m - srednje vreme rada do/između otkaza uređaja, za koji u praksi postoji realan problem da se odredi eksperimentalnim putem. Zato se on zamenjuje sa tačkastom vrednošću parametra m koja se generiše kao pseudoslučajno promenljiva po odgovarajućem izrazu. Na osnovu parametra m proračunava se ocenjena ili tačkasta ocena pouzdanosti R(t)
Tačnost ocenjene vrednosti pouzdanosti R(t) zavisi od tačnosti ocenjene vrednosti za srednje vreme rada do/između otkaza m , a tač-nost ovoga zavisi od broja otkaza. Potrebno je imati što veći broj vremena do/između otkaza da bi se tačnje odredilo srednje vreme do/iz-među otkaza.
Promenom veličine uzorka elemenata n, za koji se vrši proračun pouzdanosti, menja se opseg vrednosti ocenjene pouzdanosti Ri i na taj način proverava se tačnost rezultata proračuna pouzdanosti odre-đena primenom simulacione metode Monte Carlo.
U nastavku rada ispituje se relativna greška u određivanju pouzdanosti simulacionom metodom u zavisnosti od stvarne vrednostipo-uzdanosti.
Ostojić, D., et al., Accuracy of Reliability Calculated by the Monte Carlo Simulation Method, pp. 47-58
VOJNOTEHNIČKI GLASNIK/MILITARY TECHNICAL COURIER, 2012., Vol. LX, No. 4
Na kraju ovog rada o ispitivanju zavisnosti tačnosti tačkaste oce-ne pouzdanosti od veličine uzorka n (broja vremena bezotkaznog rada do/između otkaza) i broja iteracja (ponavjanja izračunavanja), Np po-kazano je da veličina uzorka n ima veći uticaj na tačnost nego broj iteracja Np, jer kada je uzorak isuviše mali, velikim brojem iteracja ne može se dobiti veće pobojšanje tačnosti.
Međutim, kod određivanja tačkastih ocena parametara raspodele potreban je vrlo veliki broj iteracja. Primenjena simulaciona metoda Monte Carlo pokazala se kao veoma pogodna pri ocenjivanju parametara raspodele, naročito kada je broj ovih parametara veći od 3, jer u ovom slučaju primena poznatih metoda, kao metoda maksimalne verodostoj-nosti i dr. veoma je složena. Tačnost ocena parametara raspodele pri-menom ove simulacione metode je zadovoijavajuća, a sama primena je veoma jednostavna. Za simulacionu metodu Monte Carlo neophodna je primena računara.
Kjučne reči: elektronski sistem, proračun pouzdanosti, tačnost, Monte Carlo metod, simulacja.
Datum prijema članka/Paper received on: 02. 06. 2011.
Datum dostavjanja ispravki rukopisa/Manuscript corrections submitted on: 22. 06. 2012.
Datum konačnog prihvatanja članka za objavjivanje/ Paper accepted for publishing on: 23. 06. 2012.
