CC BY
39
Ramgopal Dhaka, Bhoopendra Pachauri, Anamika Jain RT&A, Special Issue No 2(64), TWO-DIMENSIONAL SRGM_Volume 16, November 2021
Two-Dimensional SRGM with Delay in Debugging by Considering the Uncertainty Factor and Predictive Analysis
Ramgopal Dhaka, Bhoopendra Pachauri*, Anamika Jain
Department of Mathematics & Statistics, Manipal University Jaipur, Jaipur, Rajasthan, India
[email protected] *[email protected] anamikaj ain_02@rediffmail. com
Abstract
Software is a set or group of programs and instructions, which is designed to perform a well-defined function. A software fault/error can be the cause of major productivity or economic loss. The need to determine the reliability of software led to development of the software reliability growth models (SRGMs). In research literature, exist various SRGMs which have been developed without considering fault removal time and only a few models have incorporated operating environment uncertainty. However, Most of the SRGMs are characterized under the condition that the software reliability enhancement process depends only on the testing time, which is called one-dimensional SRGM. In this article, two-dimensional SRGMs have been proposed using delay in debugging, with uncertainty in the operating environment. To represent the combined effect of used resources and testing time, the Cobb -Douglas production function has been used, which converts the one-dimensional model to two-dimensional. For better understanding the results predictive analysis is performed and to validate the proposed model results are compared with existing SRGMs using five statistical comparison criteria.
Keywords: Software, SRGMs, Predictive analysis, Reliability, Cobb-Douglas production function.
I. Introduction
Software lifecycle management through development, usage, operation and maintenance of systems, requires a systematic approach. Planning, design, coding, testing and analysis are the five phases of —any software lifecycle. Software testing is the critical phase to reliably measure the functionality of the software. Any error in the software may cause considerable economic and operational damage. For successful functioning of any software, the important factor is to minimize the possibility of software failure with high reliability. "Reliability is the probability that the system will execute its indeed function under definite limit" [1]. To develop— reliable software, many SRGMs have been studied but most of them are insufficient for today's complex software. The present SRGMs can be specified in two types, one-dimensional and two-dimensional models. One-dimensional SRGMs consider only testing time, whereas two-dimensional SRGMs depend on the effect of testing time as well as used resources. Kapur et al. [2] proposed a two-dimensional SRGM in perfect debugging environment for multi-release software. Singh et al. [3] considered the effect of the parameters like testing-time and testing-coverage or testing-time and testing- effort, simultaneously on fault detection process with change point. Anniprincy and Sridhar [4] discussed
an S-shaped model with joint effect of testing-time and testing-coverage in imperfect debugging. SRGM modeling based on a CES (constant elasticity of substitution) type time function was introduced by Minamino et al. [5]. Pachauri et al. [6] studied an SRGM in imperfect debugging environment that considers the delay in fault correction.
In traditional SRGMs, it has been studied under the common belief that when a fault occurs in the software, it can be immediately removed. These models assume that the testing and operating environment of the software are the same. The first NHPP based traditional SRGM (the NHPP exponential model) was introduced by Goel and Okumoto [7]. The inflection S-shaped SRGM, depend on the failure rate of each detectable fault was discussed in Ohba [8]. Inflection S-shaped function as a fault reduction factor was given by Pachauri et al. [9].
A few models were also developed using time lag in the fault removal process. Yamada et al. [10] considered the time delay effect between the fault removal and fault detection process. An SRGM with fault dependency and various time lags, to predict the reliability, was introduced by Pradhan et al. [11]. Huang and Lin [12] proposed that in the fault removal process the correction time cannot be neglected.
In some recent publications, authors have included uncertainty of environment in the SRGMs [13 -19]. Song et al. [14] used inflection factor of error detection with vagueness in operating environment. Pham [15] introduced a new SRGM with Vtub-shaped fault detection rate. Song et al. [17] studied an SRGM with an ideal software release time and have done sensitivity analysis. Pachauri et al. [20] proposed new SRGM under the fuzzy paradigm with optimum release time and calculated total software cost.
Predictive analysis can provide advance warning of any failure, which may prove invaluable for system operators. The predictive analysis depends on historical data to make predictions about any possible future failures and errors. Recently, Song et al. [14] have introduced predictive analysis method in their SRGM.
From the literature, it is found that the factors like environmental factor, delay in debugging, effect of testing-time and used-resources, etc. are not included in an SRGM simultaneously. Therefore, motivated by Kapur et al. [2], Pachauri et al. [6] and Song et al. [14], a perfect debugging SRGM by considering uncertainty of operating environment with delay has been proposed. The predictive analysis also has been done for a better understanding of prediction. The proposed study may be useful for any software development company to check the reliability of the software based on the testing data before launching it in the market and may be used to estimate the optimum release time of software.
In this paper, two-dimensional perfect debugging SRGMs have been proposed by considering delay in fault correction and uncertainty of environment. Calculation of Mean value function (MVF) is given in Section 2 with mathematical derivation. To validate the models, the used comparison criteria, numerical results are in Section 3 and predictive analysis is shown in section 4. Finally, the conclusion is presented in Section 5.
II. Proposed model
The following are the common assumptions for these models [6, 18]:
• The software failures phenomena follow NHPP.
• Fault detection rate and residual system faults are proportional.
• The operating environment uncertainty is the multiplication of the unit failure detection rate
b(t) with a random variable
• The software system may fail during execution, due to remaining faults in the system.
Ramgopal Dhaka, Bhoopendra Pachauri, Anamika Jain RT&A, Special Issue No 2(64), TWO-DIMENSIONAL SRGM_Volume 16, November 2021
• The compound effect of the used resources and testing effort is represented using Cobb-
Douglas production function.
Based on these assumptions, the rate of change in MVF with uncertainty factor is given as [15]:
!£f> = nmMN-mm (1)
where t] is a random variable (r.v.) that represents the uncertainty of operating environments with the probability density function g, b(t) is the fault-finding rate function and N is the total number of faults initially [15]. The solution of the above differential equation with the initial condition m(0) = 0 is given as:
- (2)
m(t) = [^(l- e-«tib(x)dx) dg(n),
J-n
where g has the parameters a > 0, p >0 and t] follows the gamma distribution. After applying the r.v. T] in equation (2) we get:
^ ( H V (3)
m(t) = N(l 1
(S + ^b&dS;
when detection rate function b(t) is given as [14],
b
1 + ae
b(t) = , ____bt,a,b>0,
(4)
then the cumulative number of faults at time t,
P
m(t) = N( 1
'+*№*)) (5)
where b is the constant fault detection rate and a is the inflection factor.
After considering the testing-time and used-resources simultaneously, the modified MVF is given as [2, 6];
m(s,u) = N
( V
p
- (a + eb(sYu^-y)\
(6)
where s is the testing time and u is used resources. By using the delay factor which is a function of time, the new MVF m(s, u) is given as;
m(s, u) = m(s — ty(s), u). (7)
m(s, u) = N
/ V
p
1 —
e + -lTa-))
(8)
Here, two types of delay functions are considered, delayed S-shaped and inflection S-shaped curve.
a
Ramgopal Dhaka, Bhoopendra Pachauri, Anamika Jain RT&A, Special Issue No 2(64), TWO-DIMENSIONAL SRGM_Volume 16, November 2021
Model-1
In the fault detection exercise, the delay function is modeled as an S-shaped curve. It considers the learning progress because the skills of the examiners or testers are directly proportional to time [6]. A good explanation of delayed S-shaped curve is given in [1] and mathematically it is defined as;
1
(p(s) = - log(l + bs),
(9)
after using the delay function as defined in equation (9), then the MVF is,
a
_P__(10)
m(s, u) = N 1 r
P + ln\-rnr
Model-2
When the inflection S-shaped curve is used as the delay function which is defined as,
<p(s)=hog( 1 + (11)
where ^ is the inflection factor of Inflected S-shaped curve [6]. Then, the MVF with delay factor is,
a
ß
m(s,u) = N 1--------—y
a + e\ Hï^))ul-r\ (12)
P + ln(-T+a
III. Numerical results and discussion
For the performance validation of the proposed models, four historical data sets have been used, which are summarized in Table-1. The estimated parameter values have been obtained using the curve fitting tool in MATLAB. The performance has been obtained in terms of mean square error (MSE), sum of squared error (SSE), root mean squared error (RMSE), R-square (R2), and adjusted R-square (Adj R2).
Table 1: Reference data sets.
Data sets Time (t) Total testing hours Faults Description References
DS1 21 (weeks) 7476 26 Data set of telecommunication system test [1]
DS2 22(days) 93 CPU hours 86 The pattern of discovery of error [23]
DS3 19 10,272 CPU 120 Tandem computers [24]
(weeks) hours
DS4 12 5053 CPU 61 Tandem computers [24]
(weeks) hours
For DS1, the estimated parameters values are given in Table-2 with comparison criteria (MSE, SSE, RMSE, R2, Adj R2). From Table-2, it can be seen that the obtained values of the proposed model 1 for all five criteria are, 0.537, 8.056, 0.7328, 0.9950 and 0.9933. Similarly, for the model 2, values for comparison criteria are 0.573, 8.019, 0.7568, 0.9950 and 0.9929, respectively. From the results of DS1, both the proposed models give-better results compared to the existing literature and in between model 1 & 2, model 1 gives better performance. For more clarity, the graphical representation of total number of faults with time is shown in figure-1.
Table 2: Parameter Estimated values and Comparison results for DS1.
No. Model Estimated value MSE SSE RMSE R2 Adj R2
1 GO [7] a = 3923854.73, b = 3.2 x 10-7 3.8672 73.477 1.9665 0.9582 0.9535
2 Y-DS [10] a = 3,9.82198, b = 0.1104 1.4938 28.382 1.2222 0.9838 0.9820
3 O-IS [8] a = 26.6845, b = 0.2918, P = 21.6856 0.6745 12.141 0.8213 0.9931 0.9919
4 K-SRGM 3 [21] == -- 0.1385, b = 0.1385, = 0.1012,A = 24.989, 1.2295 20.902 1.1088 0.9881 0.9851
5 R-M-D NHPP [22] a = P- 40.2018,a = 0.9319, = 0.1402, b = 0.1152 2.0059 34.101 1.4163 0.9806 0.9757
6 C-TC b = -- 2.234,P = 15.2504, 1.0939 17.502 1.0459 0.9900 0.9867
[16] a = 0.0043, N = 26.833,
a = 9959.1698,
7 P-Vtub [15] -- 1.5176, P = 11.3848, 1.2978,N = 25.7412 a = 1.0985, 0.7178 11.485 0.8472 0.9935 0.9913
8 S-3PFD [13] a = P 0.038, c = 1488.598, = 0.002, b = 0.292, N = 26.889 0.7590 12.144 0.8712 0.9931 0.9908
9 SONG -P [14] a = P = -- 0.2176, b = 1.0047, 155.501,N = 47.797 a = 108232.819, 0.5864 9.3824 0.7658 0.9947 0.9929
10 Model 1 a> - = 371.1, a = 0.8046, = 1.344, f = 0.4027, = 0.2355,N = 27.59 0.537 8.056 0.7328 0.9950 0.9933
11 Model 2 & -P N = 8.214, a = 0.6583 = 1.47, b = 0.2948, = 26.73,xp = 243.6, y = 0.6273 0.573 8.019 0.7568 0.9950 0.9929
Figure 1: Mean value function for various SRGMs for DS1.
From Table-3, the values of the comparison criteria for DS2 are 3.3219, 53.15, 1.8226, 0.9969, 0.9959 and 3.9006, 58.51, 1.975, 0.9966, 0.9952 for model 1 & 2, respectively. These values are less than to the other existing models for the criteria MSE, SSE, RMSE and greater than for R2, Adj R2. Which shows the better performance for the proposed models for DS2. Again, the results of DS2 are shown in figure-2 for better understanding.
Table 3: Parameter Estimated values and Comparison results for DS2.
No. Models Estimated value MSE SSE RMSE R2 Adj R2
1 GO [7] a = 153,b = 0.0414 25.190 503.818 5.019 0.9706 0.9691
2 Y-DS [10] a = 94.25, b = 0.1929 7.645 152.883 2.765 0.9911 0.9906
3 O-IS [8] a = 87.21,(3 = 6.899, H = 0.2631 5.909 112.317 2.431 0.9934 0.9928
4 K-SRGM 3 [21] A = 10.97, p = 1.152, b = 1.026, a = 0.890 9.865 177.557 3.141 0.9896 0.9879
5 R-M-D NHPP [22] a = 1.6, b = 0.1192 a = 6X13$ = 0.203, 18.084 325.809 4.255 0.9810 0.9778
6 C-TC [16] a = 0.449,a = 90.05, fi = 1190, b = 1.89, N = 88.34 7.6209 129.556 2.7606 0.9924 0.9907
7 P-Vtub [15] a = 15.78,b = 1.291, fi = 7.08, N = 60.13, a = 0.0367, 292.20 4967.6 17.094 0.7099 0.6417
8 S-3PFD [13] a = 1.117, c = 136.9, P = 0.251,N = 89.31 b = 0.2656 6.602 112.231 2.5694 0.9934 0.9919
9 SONG-P [14] a = 58.4,a = 0.8222, P = 0.292,N = 92.21 H = 0.3154 6.506 110.605 2.551 0.9935 0.9920
10 Model 1 a = 6.074,N = 99.37, a = 1.4E - 07, P = 1.23,y = 0.7093, b = 0.07321 3.3219 53.15 1.8226 0.9969 0.9959
11 Model 2 a = 4.022, a = 0.066, P = 0.919, b = 0.066, N = 100.9? = 0.8298 $ = 0.07275 3.9006 58.51 1.975 0.9966 0.9952
Figure 2: Mean value function for various SRGMs for DS2.
The results of proposed models 1, 2 for DS3 are shown in Table-4, which are 1.5055, 19.57, 1.227, 0.9992, 0.9989 and 1.5625, 18.74, 1.25, 0.9993, 0.9989, respectively. Comparison with other models in terms of MVF and time is given in figure-3. Similarly, from Table-5 the results of DS4 for both the models are 5.6644, 34.00, 2.380, 0.9932, 0.9875 and 8.8209, 44.09, 2.97, 0.9912, 0.9805, respectively. The comparative study of DS4 is shown in figure-4.
Table 4: Parameter Estimated values and Comparison results for DS3.
No. Models Estimated value MSE SSE RMSE R2 R2
1 GO [7] a = 183, 6 = 0.0615 26.002 442.0 5.0992 0.9824 0.9813
2 Y-DS [10] a = 127.4,0 = 0.2417 14.688 249.691 3.8325 0.9900 0.9895
3 O-IS [8] a = 124.4,0 = 0.2535 /? = 3.779 7.1268 114.0261 2.6696 0.9955 0.9895
4 K-SRGM 3 [21] i = : 1.858, p = 3.791 6 = 2.413, a = 0.9873 18.550 278.2 4.307 0.9889 0.9867
5 R-M-D NHPP [22] a = 98.13,â = 1.353, ô = 0.1835, = 0.215 13.432 201.5 3.665 0.9920 0.9904
6 C-TC [16] a = /? = 46.24,0 = 1.474, 26.63,iV = 125.7 a = 0.0787, 13.265 185.7 3.6422 0.9926 0.9905
7 P-Vtub [15] a = N = 60.55,/? = 8.282, 118.7,0 = 1.157, a = 0.0246 65.594 918.3 8.099 0.9634 0.9529
8 S-3PFD [13] a = /? = = 0.673,c = 39.4, 0.326,iV = 130.6 ô = 0.2555, 8.2599 115.7 2.874 0.9954 0.9941
9 SONG-P N = = 176, â = 3181, 2.143 30.006 1.464 0.9988 0.9984
[14] /? = a = 0.2208,
58.16,0 = 1.096,
10 Model 1 a = a = 77.35,/? = 45.28, 0.659,iV = 180.9 j? = 0.2777, ô = 0.1647, 1.5055 19.57 1.227 0.9992 0.9989
11 Model 2 a = 35.52, â = 0.533, = 16.66, iV = 141, ô = 0.2278, f = 0.3356, 1.5625 18.74 1.25 0.9993 0.9989
4> = 12.11
Figure 3: Mean value function for various SRGMs for DS3.
Table 5: Parameter Estimated values and Comparison results for DS4.
No. Models Estimated value MSE SSE RMSE R2 Adj R2
1 GO [7] a = 244.3, b = 0.0265 20.894 209 4.571 0.9581 0.9539
2 Y-DS b = = 0.2141,a = 16.25, 9.897 98.94 3.146 0.9802 0.9782
[10]
3 O-IS [8] a = 64.4,(1 = 11.36, 6.345 57.09 2.519 0.9885 0.9860
b = 0.483
4 K- a = 0.966,p = 1.942, 15.296 122.4 3.911 0.9755 0.9663
SRGM 3 A = 3.051, b = 2.091,
[21]
5 R-M-D a ■ = 59.05,b = 0.1996, 20.512 145.1 4.259 0.9709 0.9600
NHPP a = 1.353,(3 = 0.2526,
[22]
6 C-TC P = 94.38, a = 1010, 10.74 75.16 3.2768 0.9849 0.9763
[16] b = 1.921, a = 0.0439
N = 64.69
7 P-Vtub P = 852.4, a = 59.81, 6.6616 46.63 2.581 0.9906 0.9853
[15] b = 0.182, a = 1.923,
N = 61.01
8 S-3PFD a ■ = 29.31, $ = 2.851, 9.4741 66.32 3.078 0.9867 0.9791
[13] c = = 228.1, b = 0.4653,
N = 64.48
9 SONG-P a = = 2262, a = 0.5059, 6.287 44.01 2.507 0.9912 0.9861
[14] = 0.645, b = 0.1314,
N = 62.56
10 Model 1 a = = 48.34, a = 0.5042, 5.6644 34.00 2.380 0.9932 0.9875
P = = 0.1441, y = 0.4634
b = = 0.0629,№ = 66.51
11 Model 2 a = 92.11, a = 0.135 8.8209 44.09 2.97 0.9912 0.9805
V = 0.125, b = 0.0940
N = 66.83,?= 0.8239,
ip = 0.1244
-
• DATA
-e-fiO
—— Y-IXS
—•—Y-Exp
K-SRGM3
—H-M-I)
C-TC
P-V tub
S-3PFD
>~SGNG-P -¿—NEW MODEL-1 -■»■•' NEW MODEL-2
1 1 1 1 1 III
Time (Weeks)
Figure 4: Mean value function for various SRGMs for DS4.
The relative error with time is also calculated for all the data sets, which confirms the ability to provide better accuracy. The comparison of the proposed models with other models in terms of relative error are shown in figures 5-8. Calculated values of MSE, SSE, and RMSE for proposed models are less than the existing models for all four data sets. The values of R-square and Adjusted R-square for proposed models are greater than the others. Based on these results, it can be said that these proposed models give improved results and have considerably better goodness of fit. A4se, model-1 gives Abetter results compared to model-2.
2 * ■ & 12 1* 15 14 №
Time (Weeks)
Figure 5: Relative error curve for various SRGMs for DS1.
2 i 6 a 10 12 M ie is V) 22
Time (Days)
Figure 6: Relative error curve for various SRGMs for DS2.
Figure 7: Relative error curve for various SRGMs for DS3.
Figure 8: Relative error curve for various SRGMs for DS4.
IV. Predictive Analysis
In this paper, DS1 is used for the comparison of predicted values and how these differ for each model. 75% of the data set is used to estimate the parameters and the remaining 25% of the data set is used to predict the model performance. The sum of squared error of predicted value is denoted by PreSSE and it is calculated for the last 25% of the data.
The result of predictive analysis based on the six criteria (MSE, SSE, RMSE, R2, Adj R2 and PreSSE) is given in Table-6. From the Table-6, the values of the six criteria are 0.64, 6.417, 0.8010, 0.9924, 0.9873, 1.8684 and 0.72, 6.488, 0.8491, 0.9923, 0.9856, 1.8811 for both the models, respectively. Graphical representation of estimated MVF and predicted MVF is shown in figure-9. Based on the obtained results as shown in Table-6 and figure-9, it can be said that the proposed models give better prediction compared to the other existing models, because the SSE and PreSSE are minimum for both the proposed models.
Table 6: Parameter Estimated values, criteria and predictions from DS1
No Model Estimated value MSE SSE RMSE R2 Adj R2 PreSSE
1 GO [7] a = 2439.196, b = 0.00052 4.85 8.012 2.2041 0.9199 0.9076 5.756
2 Y-DS [10] a = 81.516,b = 0.0657 0.85 11.84 0.9195 0.9861 0.9839 128.71
3 O-IS [8] a = 32.236b = 0.2378, fi = 16.9353 0.67 8.690 0.8176 0.9898 0.9872 36.131
4 K- SRGM 3 [21] A = 43.931, p = 0.0265 b = 1.3062, a = 2.2423 1.02 12.24 1.0099 0.9856 0.9804 264.331
5 R-M-D a = 153.5748,(2 = 1.056, 0.91 10.91 0.9535 0.9872 0.9825 188.6321 NHPP /? = 0.046,6 = 0.0327
[22]
6 C-TC [16] a = /? = 0.0288,(2 = 193.585, 167.613,£ = 1.6825, N = 86.0872 1.02 11.24 1.0107 0.9868 0.9802 160.1096
7 P-Vtub [15] a = /? = 1.2816,(2 = 1.1473, = 0.1982, £ = 0.9844, N = 31.4297 0.78 8.613 0.8849 0.9899 0.9848 32.5458
8 S- 3PFD [13] a = /? = 0.1496,c = 62.4407, = 0.1982,6 = 0.2372, N = 36.9478 0.82 8.989 0.9040 0.9894 0.9841 45.9267
9 SONG -P [14] a = 6 = a = 10453.17 : 0.4174,/? = 0.1175, 0.53348, JV = 24.292 0.59 6.506 0.7691 0.9923 0.9885 2.6780
10 Model 1 a = /? = N = 26.45,(2 = 0.4766, 0.4243, £ = 0.03818, 26.45,f = 0.481, 0.64 6.417 0.8010 0.9924 0.9873 1.8684
11 Model 2 a = /? = N = 11.34,(2 = 0.7453, = 0.403, £ = 0.2607, 26.18, z/i = 86.91, f = 0.7413, 0.72 6.488 0.8491 0.9923 0.9856 1.8811
Time (Weeks)
Figure 9: Prediction of MVF for various models for DS1.
V. Conclusion
Since-software development and testing process takes place in a controlled environment which is far away from operating environment. Therefore, uncertainty in the operating environment has been considered. To reflect the joint effect of testing time and used resources Cobb-Douglas production function has been used. Fault (if any) correction time is also affected the software costs as well as launching time. Delay in debugging has been included in two different ways, delayed S-shaped and Inflection S-shaped. In this article, two SRGMs have been proposed and these have been validated on the four real data sets. For validation, a curve fitting tool in MATLAB has been used for estimation and five statistical comparison criteria (MSE, SSE, RMSE fl2, and ^dy fl2) have used to compare the results with existing literature. After validation, predictive analysis has been done to check the predictability of the proposed models using SSE and PreSSE. All the results with relative error have been figure out for better understating. On the basis of the results, it can be concluded that proposed
Ramgopal Dhaka, Bhoopendra Pachauri, Anamika Jain RT&A, Special Issue No 2(64), TWO-DIMENSIONAL SRGM_Volume 16, November 2021
models give significantly better performance compared to the others. In between these two models,
it is found that results are improved when delayed S-shaped function has been used as delay
function. In future, proposed work may include improvement of the proposed models with soft-
computing techniques or introduction of additional reliability factors, i.e., change point, testing
coverage, etc.
References
[1] Pham, H. System Software Reliability, Springer, London, UK, 2006.
[2] Kapur, P.K., Aggarwal, A.G. and Kaur, G. (2012). Two-dimensional multi-release software reliability modeling and optimal release planning. IEEE Trans. Reliab., 61: 111.
[3] Singh, O., Garmabaki, A.H.S. and Kapur, P.K., (2011). Unified framework for developing two-dimensional SRGMs with change point. IEEE International Conference on Quality and Reliability (ICQR), 570-574.
[4] Anniprincy, B. and Sridhar, S. (2014). Two-dimensional software reliability growth models using Cobb-Douglas production function and Yamada S-shaped model. J. Softw. Eng. Simul. 2(2): 1-11.
[5] Minamino, Y., Inoue S. and Yamada, S. (2017). Two-dimensional software reliability growth modeling based on a CES type time function, Proc. 2017 Infocom Technologies and Unmanned Systems (ICTUS'2017), 120-125.
[6] Pachauri, B., Kumar, A. and Raja, S. (2019). Imperfect software reliability growth model using delay in fault correction. In: Deep, K., Jain, M., Salhi, S. (eds.) Performance prediction and analytics of fuzzy, reliability and queuing models. Asset analytics (Performance and safety management). Springer, Singapore. https://doi.org/10.1007/978-981-13-0857-4_8.
[7] Goel, A.L. and Okumoto, K. (1979). Time-dependent error detection rate model for software reliability and other performance measures. IEEE Trans. Reliab., 28: 206-211.
[8] Ohba, M. (1984). Inflexion S-shaped software reliability growth models. In: Osaki, S., Hatoyama, Y. (eds.) Stochastic models in reliability theory, pp. 144-162. Springer, Berlin, Germany.
[9] Pachauri, B., Kumar, A. and Dhar, J. (2015). Incorporating inflection S-shaped fault reduction factor to enhance software reliability growth. Appl. Math. Model., 39(5): 14631469.
[10] Yamada, S., Ohba, M. and Osaki, S. (1983). S-shaped reliability growth modeling for software fault detection. IEEE Trans. Reliab., 32: 475-484.
[11] Pradhan, V., Kumar, A., Dhar, J. and Gupta, M. (2018). A software reliability model incorporating fault dependency considering time delay. Int. J. Pure Appl. Math., 22: 1527-1535.
[12] Huang, C.Y. and Lin, C.T. (2006). Software reliability analysis by considering fault dependency and debugging time lag. IEEE Trans. Reliab., 55(3): 436-450.
[13] Song, K.Y., Chang, I.H and, Pham, H. (2017). A Three-parameter fault-detection software reliability model with the uncertainty of operating environments. J. Syst. Sci. Syst. Eng., 26: 121-132.
[14] Song, K.Y., Chang, I.H. and Pham, H. (2019). NHPP software reliability model with inflection factor of the fault detection rate considering the uncertainty of software operating environments and predictive analysis. Symmetry, 11: 521. doi:10.3390./sym11040521.
[15] Pham, H. (2014). A new software reliability model with Vtub-Shaped fault detection rate and the uncertainty of operating environments. Optimization 63: 1481-1490.
[16] Chang, I.H., Pham, H., Lee, S.W. and Song, K.Y. (2014). A testing-coverage software reliability model with the uncertainty of operation environments. Int. J. Syst. Sci. Oper. Logist., 1: 220-227.
[17] Song, K.Y., Chang, I.H. and Pham, H. (2018). Optimal release time and sensitivity analysis using a new NHPP software reliability model with probability of fault removal subject to operating environments. Appl. Sci., 8: 714. doi: 10.3390/app8050714.
[18] Song, K.Y., Chang, I.H. and Pham, H. (2017). A software reliability model with a Weibull fault detection rate function subject to operating environments. Appl. Sci., 7: 983. doi:10.3390/app7100983.
[19] Lee, H.H., Chang, I.H., Pham, H. and Song, K.Y. (2018). A software reliability model considering the syntax error in uncertainty environments, optimal release time and sensitivity analysis. Appl. Sci., 8: 1483. doi:10.3390/app8091483.
[20] Pachauri, B., Kumar, A. and Dhar, J. (2013). Modeling optimal release policy under fuzzy paradigm in imperfect debugging environment. Inf. Softw. Technol., 55(11): 19741980.
[21] Kapur, P.K., Pham, H., Anand, S. and Yadav, K. (2011). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Trans. Reliab.: 60(1): 331-340.
[22] Roy, P., Mahapatra, G.S. and Dey, K.N. (2014). An NHPP software reliability growth model with imperfect debugging and error generation. Int. J. Reliab. Qual. Saf. Eng., 21(02) 1450008 (32 pages).
[23] Tohma, Y., Jacoby, R., Murata, Y. and Yamamoto, M. (1989). Hyper-Geometric distribution model to estimate the number of residual software fault. Proceedings of the Thirteenth Annual International Computer Software & Applications Conference, Orlando, FL, USA, pp. 610-617. doi: 10.1109/CMPSAC.1989.65155.
[24] Wood, A. Predicting Software Reliability, IEEE Comp. 29(11), 69-77, 1996.
