CC BY
55
An Optimal Resource Allocation Model Considering Two-Phase Software Reliability Growth Model with Testing Effort and Imperfect Debugging
Sujit K. Pradhan1, Anil Kumar2, Vijay Kumar3*
1,2Department of Mathematics, BITS Pilani-KK Birla Goa Campus, Zuarinagar, Goa-403726, India. E-mail: 1pradhansujit1994@gmail.com, 2anilpundir@goa.bits-pilani.ac.in 3Department of Mathematics, Amity Institute of Applied Sciences, Amity University
This study aims at investigating an optimal resource plan in order to minimize the software costs in the debugging and testing phases. We have proposed a resource allocation model to address testing efforts, imperfect debugging and change-point. We have considered two cases: In the first case, the software debugging cost is kept constant, and in this case, the optimal policy follows a bang-bang structure, which means investing entirely in the testing phase, followed by investing fully in the debugging stage. In the second case, we have taken the debugging cost in quadratic form. We have validated our model with the experimental data, and the results reveal that the presented model is reasonably accurate. We have also discussed the optimal resource allocation problems under certain conditions and examine the parameters' behavior in the model and obtain the variations in the total cost. This study provides a detailed optimal control theory-based testing resource allocation policy, which is supported by numerical examples.
Keywords: Software reliability growth model (SRGM), Change point, Testing effort, Optimal
control.
Computers are used in diverse areas, and with recent advances in computation, software related issues have emerged as one of the primary areas of concern. Hence, reliable software product demand has increased in the market. Software reliability models can be effectively utilized to generate quantified measures during the software development phase. SRGM attempts to tally between defect detection data and estimated residual defects with time. Therefore, software reliability is crucial in developing software and its quality. Specification, design, programming, testing-and-debugging are the four stages involved in any software development process. During the software development process, the testing-and-debugging phase is a key and expensive phase of the software development life cycle (SDLC).
The probability when the software will not result in system failure for a specific time and under specific conditions is known as software reliability (see, e.g., [1]). Further, software reliability and software cost must be up to the expectation level of users satisfaction. During the testing phase of SDLC, testing and debugging are the two main activities to be performed by the testing team, and there is always a trade-off between cost and reliability. The software testing team has to understand the variance of the software reliability and the instantaneous testing costs. Thus, software reliability, cost, and release time are important aspects of software development.
Uttar Pradesh, Noida-201313, India. E-mail: 3vijay_parashar@yahoo.com Corresponding Author
*
Abstract
1. Introduction
Research activities on software reliability growth models have been conducted over the past five decades. In general, the SRGMs have been proposed under consideration with the non-homogeneous Poisson process (NHPP) since software faults are associated with discrete time scales. Various SRGMs [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] have been proposed on different assumptions and applied to different situations. Many researchers have proposed SRGMs considering different scenarios; some researchers have proposed SRGM considering perfect debugging [2, 3, 5]; some researchers considered imperfect debugging [4, 6, 7]. Huang et al. [18] have discussed an SRGM, which incorporates debugging time-lag and fault dependency.
Many researchers proposed SRGMs incorporating testing efforts. Yamada et al. [19], and Musa et al. [20] proposed a novel SRGM. The model discussed in [19] describes the relationship between the amount of testing effort and the number of software errors detected. Kapur et al. [21], and Chang et al. [22] have proposed SRGMs, which incorporate the concept of testing-effort function. Generally, CPU hours are considered as testing efforts. Huang et al. [23], and Kapur et al. [24] represented a model in which the consumption rate of testing resource expenditures with the different testing-effort functions was discussed. Jin and Jin [25] used an S-shaped curve to describe software testing efforts. Huang et al. [26, 27] proposed SRGMs with logistic testing-effort function.
In software reliability studies, the initial number of faults contained are not known. One has to carry out the program in a particular environment. This helps to improve the quality of the program by correcting the faults. SRGMs proposed by Huang et al. [28] assumed that each failure caused by a fault is independent and random in time and occurs with the same distribution during the fault detection process. In SRGMs, the testing environment, testing strategy and resources are not necessarily the same throughout the development process. The testing environment is subjected to change with the learning process. Hence, the change-point problems play a vital role in software reliability growth modeling.
The change point is a point where the software testing team changes their testing strategy from one to another during the software development process due to the complexity of the program, testing facilities, and other random factors. The fault detection process is affected by different factors. The fault detection rate may change due to the increasing knowledge of the program and the testing strategies. Chang et al. [22] and Kapur et al. [29] discussed an SRGM with change point in fault detection rate. Shyur et al. [30] explained a stochastic software reliability growth model which incorporates imperfect debugging and change points.
Kapur et al. [31, 32] have discussed a model to assign the resources and minimize the total cost during the development period of SRGM under dynamic conditions. Kumar et al. [33, 34, 35] explained a resource allocation model for fault detection and fault correction process. They assumed detection and correction efforts to be independent. However, activities such as detection and correction may have budgetary constraints. Yonghua et al. [36, 37] proposed a model which incorporates a resource allocation plan to minimize the testing cost of the software. Kumar et al. [38] discussed resource allocation model for a multi-release SRGM to minimize the testing cost under dynamic conditions.
In the present work, we incorporate testing effort, imperfect debugging, and change points. These features are concomitant with the general SRGM. Yamada et al. [39] proposed an SRGM to minimize the total expenditure under static conditions. A problem emerges when the development process is carried out under dynamic conditions. The fault detection and fault correction process relies upon the operating environment and the quality of resources utilized. Experimental data analysis is carried out to evaluate the change point. We have used the failure increasing rate function to find the change point. We also study the optimization problem for optimal resource allocation for different conditions by examining the behavior of the model parameters and obtain variations in total cost. To do so, we have taken the quadratic form of the debugging function.
The remaining manuscript is structured as follows: We concluded Section 1 by providing notations used in the manuscript. In Section 2, we briefly discussed the model developed by Zhu et al. [40]. Section 3 deals with the model development, and we introduced an optimal control problem. In Section 4, the optimal policies are developed, and optimal solutions are given. Some
theoretical results are shown in Section 5, and the optimal policies are discussed for two special cases. In Section 6, the change point is calculated with the help of the data, and the behavior of model parameters in the variation of the total cost is discussed. In Section 7, we conclude the paper with some possible research on this topic.
Notations
[0, T] The complete life cycle of the software.
t1 The change-point.
m1(t) The cumulative number of faults detected in phase-I by time t lies between 0 to t1.
m2(t) The cumulative number of faults detected in phase-II by time t lies between t1 and T.
x1(t) The number of faults detected at any point of time t in phase-I.
x2(t) The number of faults detected at any point of time t in phase-II.
a1 (t) The total fault content function in phase-I.
a The initial number of fault.
a2(t) The total fault content function in phase-II.
a The fault introduction rate per detected fault.
b1 The fault detection rate in phase-I.
b2 The fault detection rate in phase-II.
c1 The non-removable fault rate in phase-I.
c2 The non-removable fault rate in phase-II.
w11(t) The testing effort in phase-I.
w21(t) The testing effort in phase-II.
w12(t) The debugging effort in phase-I.
w22(t) The debugging effort in phase-II.
c11 (t) The debugging cost per unit at time t associated with the debugging effort w12 (t) in phase-I.
c21 (t) The debugging cost per unit at time t associated with the debugging effort w22 (t) in phase-II.
c11 The base cost of debugging in phase-I.
c21 The base cost of debugging in phase-II.
c12 The cost of testing per unit testing effort at time t in phase-I.
c22 The cost of testing per unit testing effort at time t in phase-II.
y(t) The observed cumulative number of failures by time t.
y'(t) The failure increasing rate during time interval (t, t + At).
2. Two-pHAse sofTwARe ReLIAblLITY gRowTH MoDeL
An administer SRGM is tested by software test personnel to detect and correct software faults during the development process. In realistic conditions, different types of software faults are found in SRGMs. Software test personnel do not remove all faults during the debugging process by applying the same testing effort. So different testing efforts are applied by software test personnel to remove different types of faults. Thus, to remove two types of fault, software test personnel used a two-phase debugging process. Zhu et al. [40] proposed a two-phase software reliability model under the following assumptions, which involved software fault dependency and imperfect debugging.
1. The error detection in SRGMs follows the non-homogeneous Poisson process.
2. The model considered imperfect debugging, and new faults are introduced into the program at each time.
3. Type-I and type-II software faults are defined. The type-I and type-II faults are detected and corrected during phase-I and phase-II, respectively.
4. The software development team cannot remove all the faults experienced in both phases.
5. Due to different software fault types, the fault detection and non-removable fault rates are different in phase I and II.
6. The time to debug a fault is negligible.
Using the above assumptions, the following two-phase SRGM is considered for the study.
Phase-I. The type-I faults, which are independent and easily detected, are detected and corrected in phase-I. The total number of software faults that cause failure in phase-I is proportional to the difference between the total number of detected faults and the total number of non-removable faults. The total number of detected faults that cause failure is the product of fault detection rate and the number of remaining type-I faults which are represented by the following differential equation
dm1r = b1 (0M0 - m1 (t)] - C1 (two, 0 < t < t1. (1)
With a constant fault introduction rate a (> 0), new faults are introduced during the debugging phase due to imperfect debugging. Therefore, the fault present in phase-I at time t is given by
ai (t) = a(1 + at). (2)
In phase-I, let the non-removable fault rate is
C1(t) = C1, C1 > 0, (3)
with the initial condition m1 = 0 at t = 0 for phase-I.
Phase-II. During this phase, type-II faults are detected after the completion of type-I faults detection and correction. No new and residue type-I faults are introduced in phase-II. The total number of software faults that cause failure is proportional to the difference between the total number of detected faults and non-removable faults. The total number of detected faults that cause failure is the product of fault detection rate, the ratio of the total number of reduced faults to the total number of failures experienced, and the number of remaining type-II faults. Therefore, we have the following differential equation
^ = b2(t)mm2|) [a2(t) - m2(t)] - c2(t)m2(t), t1 < t < T. (4)
The cumulative number of fault present in phase-II is
ai(t) = a1 (t1) - m1(t1). (5)
In phase-II, let the non-removable fault rate is
C2(t) = C2, C2 > 0, (6)
with the continuity condition between the two phases
m2(t1 ) = m1 (t1). (7)
3. Model development
The testing and debugging phase aims to detect and correct faults and make the software more reliable during the development process of the software. Software reliability is affected by the testing resources spent in the testing phase. We have modified the SRGM given in Section-2 without increasing the complexity. We have developed a model, which incorporates time-dependent testing effort with change point under imperfect debugging environments. Here, a model is constructed with concurrent detection and correction activities. Resources should
be allocated optimally during software testing. Thus, a tactical plan is required for allocating resources optimally. The total resource in each phase is divided into two portions, i.e. testing effort and debugging effort. The mathematical expression for the resource allocation model is written as
wii (t) + wi2 (t) w21 (t) + w22 (t)
1, 0 < t < t1, 1, t1 < t < T,
(8) (9)
which is shown in Figure 1.
Figure 1: Optimal resource allocation in Phase-I and Phase-II.
Phase-I. We have assumed that the software development team detects the faults causing failure and corrects those faults during the testing phase to develop a new model. The total number of software faults that cause failure in phase-I per unit testing effort expenditure is proportional to the difference between the total number of detected faults and the total number of non-removable faults. The total number of detected faults that cause failure is the product of fault detection rate and the number of remaining type-I faults. All the detected faults are not possible to remove from software during the testing phase. Here, we have taken the testing effort in the fault detection process. We know that we should remove more faults if much testing effort is utilized in the testing phase. Some faults can't be removed. Based on the above assumptions and compatible with the idea of Zhu et al. [40], we have proposed the phase-I software reliability growth model as
dm1 (t)
x1 (t)
dt
wn(t) [&1 (a1 (t) - m1 (t)) - c1 m1 (t)],
0 < t< ti.
(10)
Phase-H. In phase-II, the total number of software faults that cause failure per unit testing effort expenditure is proportional to the difference between the total number of detected faults and the total number of non-removable faults. The total number of detected faults that cause failure is the product of fault detection rate, the ratio of the total number of reduced faults to the total number of failures experienced and the number of remaining type-II faults. The following differential equation gives the phase-II model:
x2 (t) = w21 (t)
b2m2!^(t) - m2(t)) - c2m2(t)
, t1 t T.
(11)
Optimal control problem. We aim to create a resource allocation plan to minimize the total testing expenditure. For the simplicity of the model, we have neglected all other costs except fault detection cost and correction cost over the finite planning period T. The proposed model incorporates testing effort as a control parameter. Then the problem can be described over the interval [0, T] as follows:
min
^ | Cii(i}xi(i}+ C12 Wii(f)j dt + £ |c21 (t)X2 (t) + C22 W21 (t)} dt
(12)
subject to
xi(t) = dmdr) = wii(t)[bi(«i(t) - mi(t)) - cimi(t)], 0 < t < ti, (13)
X2(t) = = W2i(t)
b2m^T^t) - m2(t)) - C2m2(t)
t1 < t < T, (14)
with the conditions m1(0) = 0, and m2(t1 ) = m1(t1 ).
4. Optimal policies and solution
To solve dynamic optimal control problem defined by equations (12)-(14), we shall use Pontryagin minimum principle [41]. Therefore, we first define the Hamiltonian function, which is given by
= (H1 ('), 0 < ' < (15)
v ; \h2(t), t1 < t < T
where
H1 (m1 (t), A1 (t), w11 (t), t) = c11 (t)x1 (t) + c12w11 (t) + A1 (t)x1 (t), 0 < t < t1, H2(m2(t), A2(t), w21 (t), t) = c21 (t)x2(t) + c22w21 (t) + A2(t)x2(t), t1 < t < T.
The necessary conditions within each time interval for an optimal solution are defined similar to [42]. The co-state variables (or adjoint variables) A1 (t) and A2(t) are given by
d a (t) = - 3Hi (mi(t) MO wii(t),t), 0 < t < ti, (16)
dt iW 3m1 (t) < < v '
d A (t) = - 3H2(m2(t), A2((t)), w21(t),t) , ti < t < T, (17)
dt 2W 3m2(t) 1 < < v '
with terminal conditions A2(T) = 0. Also, the following matching conditions are satisfied at t = t1
Ai(ti ) = A2(ti ), (18)
Hi (mi (ti), Ai(ti), wii (ti), ti) = H2(m2(ti), A2(ti), W21 (ti), ti), (19)
and we have
Hi (mi (ti ), Ai (ti), wii (ti), ti) < H2(m2(ti ), A2(ti), W21 (ti), ti), if 0 = ti < T, (20) Hi (mi (ti ), Ai (ti), wii (ti), ti) > H2(m2(ti ), A2(ti), W21 (ti), ti), if 0 < ti = T. (21) The control variables w11 (t) and w21(t) are given by 3 H1 (m1 (t), A1 (t), w11 (t), t)
dwn (t) d H2 (m2 (t), A2 (t), w21 (t), t) dwn (t)
which will give from the equations (22) and (23)
= 0, 0 < t < ti, (22) = 0, ti < t < T, (23)
Wi(t) = (cimi(t) (t) -mi(t))))((Ci1 (t) +Ai(t))- Ci2, 0 < t < ti, (24)
11 w ciiwii (t)bi ((«i(t) - mi (t)) - ci (t)mi(t)) < < ^ v '
# , (C2m2(t) - b2(mmi))(«2(t) - m2(t)))(C2i(t)+ A2(t)) - C22
w21 (t) = -a2(t) (t)-, ti < t < T, (25)
C21W21 (t)(b2 ( |f)(«2(t) - m2 (t)) - C2 m2 (t))
and from equations (8) and (9), we obtain
w*u(t) = 1 - (cimi(t) -b1 [a]((t) -mi(t)))(c11 (t) +Al- c12, 0 < t < ti, (26) 12W ciiwii (t)bi((ai(t) - mi(t)) - ci(t)mi(t)) ' < < 1 y '
(c2 m2(t) - b2 ( m§)(a2 (t) - m2 (t)))(c2l(t) + A2 (t)) - c22
w\2(t) = 1--a2( ) (t)-, ti < t < T. (27)
c2iw2i (t)(b2(m§)(a2(t) - m2(t)) - c2m2(t))
5. Theoretical results
In this section, we have discussed optimal criteria for two-phase, continuous-time optimal control problems where the integration of these problems depends upon the change point. The qualitative results developed here are demonstrated in the following theorem.
Theorem 1. Let the change point t1 occurs in the software development life cycle [0, T]. Then the following holds
(i) At t1 € (0, T), the total cost of fault detection and correction in phase-I is equal to the total
cost of fault detection and correction in phase-II.
(ii) At t1 = 0, the total cost of fault detection and correction in phase-I is less than the total cost
of fault detection and correction in phase-II.
(iii) At t1 = T, the total cost of fault detection and correction in phase-I is greater than the total cost of fault detection and correction in phase-II.
Proof. Let [0, T] is the planning period of SRGM and the change point t1 € (0, T). Since, the Hamiltonian H1 and H2 are equal at the change point t1 which is given by equation (19)
Hi (mi (ti), Ai(ti), wn(ti), ti) = H2(m2(ti), A2(ti), w2i (ti), ti).
We obtain
cii (t)xi (t) + ci2 wii (t) + Ai(t)xi(t) = c21 (t)x2 (t) + c22 w21 (t) + fa(t)x2(t), and hence, we get
(cii (t) + Ai(t))xi (t) + ci2 wii(t) = (c 21 (t) + A2 (t))x2 (t) + c22w21 (t). (28)
Therefore, in the equation (28), the total cost of fault detection and correction in phase-I is equal to the total cost of fault detection and correction in phase-II.
To prove (ii), let the change-point be t1 = 0, i.e. t1 / (0, T). Then we should skip directly to phase-II for fault detection and correction i.e. in phase-I, no-fault detection and correction are done. Hence, the total cost of fault detection and correction in phase-I is less than the total cost of fault detection and correction in phase-II.
Let the change-point be t1 = T i.e t1 / (0, T). Then we stick entirely in phase-I for fault detection and correction, i.e. in phase-II, no-fault detection and correction is done. Hence, the total cost of fault detection and correction in phase-I is greater than the total cost of fault detection and correction in phase-II. This complete the proof of part (iii). ■
Special cases. In general, at the initial time of the testing phase of SRGM, the debugging/fault correction cost is much higher as the uncertain nature of the errors. Later on, the fault intensity
gradually decreases, and most of the expenditure is invested in the testing phase. Therefore the debugging cost gradually decreases with time. Here, we shall discuss two special cases to show how debugging/fault correction costs impact the optimal policies.
Case-I. Let the debugging/fault correction cost per unit for cumulative fault removed at time t before and after change point are constant, i.e.
cii (t) = cii and C2i(t) = C21 (29)
Then for constant debugging cost, the objective function (i2) will take the following form
rT
mm
^ | Cil X1 (t) + C12 Wn(t)} dt + £ |c21 X2 (t) + C22 W21(t)} dt
subject to
x1 (t) = dm1(t) = w11(i)[b1(fl1 (t) - m1(t)) - c1m1(i)j, 0 < t < t1 (30)
dt
X2 (t) = = W21(t)
b2^T^MO - m2(t)) - C2m2(t) a2 (t)
t1 < t < T (31)
with the conditions m1(0) = 0,m2(t1 ) = m1 (t1), w11 (t) + w12(t) = 1 and w21(t) + w22(t) = 1. Then the Hamiltonian for the optimal control problem is
H1(m1(t), A1(t), w11(t), t) = c11x1(t) + c12w11 (t) + A1 (t)x1 (t), 0 < t < t1 H2(m2(t), A2(t), w21(t), t) = c21 x2(t) + c22w21 (t) + A2(t)x2(t), t1 < t < T,
The adjoint variable A1 (t) and A2 (t) are given by
dt;A1 (t) = ^1(t) = W11 (t)(cn + A1(t))(b1 + C1), 0 < t < t1 (32)
ddtA2(t) = A2(t) = W21 (t){C21(t)+ A2(t)}{C2 - - }, t1 < t < T (33)
with the terminal condition A2 (T) = 0, which on simplifying will give
A1 (t1) = A1 (0) + /f1 wn(t)(c11 + A1 (t))(b1 + C1)dt (34)
Jo
A2 (t1) = ^TW21 (t){C21 + A2(t)}{b2^1 - - C2 }dt. (35)
The necessary conditions for optimality are
a H1
3w11 a H2
0, 0 < t < t1 (36)
0, t1 < t < T (37)
3w2i which will give
= (cii + Ai(t))(bi(ai(t) - mi(t)) - cimi(t)) + ci2, 0 < t < ti,
and
H2w2i = (C2i + A2(t))(b2mm2(y(fl2(t) - m2(t)) - C2m2(t)) + C22, ti < t < T.
where Hiwn = Hi and ^ = g-.
The Hamiltonian in equation (i5) is linear in wii (t) and w2i (t), respectively. So we have the following bang-bang solution for wii(t) and W2i (t) to minimize the Hamiltonian. Therefore, we obtain wii(t) and w2i (t) as
(i) If 0 < t < ti,
(1, if Hiwii > 0
wl^t) = I undefined, if H1wi1 = 0 (38)
[0, if Hiwii < 0.
If the marginal value of the testing effort Hiwii is positive, then maximum possible testing effort is wii(t) = i because Hamiltonian Hi is linear in control variable wii(t). If Hiwii is negative, then minimum testing effort is wii(t) = 0 i.e. all effort should be allocated to debugging. If Hiwii = 0, then the testing effort wii(t) need not be determined from the condition Hiwii = 0.
(ii) If ti < t < T,
( 1, if H2w21 > 0
< un
w*2i(t) = < undefined, if H2w2i = 0 (39)
[0, if H2w2i < 0.
Similarly, if the marginal value of the testing effort H2w2i is positive, then maximum possible testing effort is w2i (t) = i because Hamiltonian H2 is linear in control variable w2i (t). If H2w2i is negative, then minimum testing effort is w2i(t) = 0 i.e. all effort should be allocated to debugging. If H2w2i = 0, then the testing effort w2i(t) need not be determined from the condition H2w2i = 0.
Case-II. Researchers have been proposed the effect of the experience curve in SRGM. Kapur et al. [3i] used Pegels' form [43] as debugging cost function. In this case, we consider that the total cost per unit fault removed is a quadratic function of the debugging efforts, i.e.
cii(t) = cii(wi2(t))2 and c2i(t) = c2i(w22(t))2. (40)
Then the Hamiltonian (i5) will reduce to
Hi(mi(t), Ai (t), wii(t), t) = cii (t)xi (t) + ci2wii (t) + Ai(t)xi(t), 0 < t < ti H2(m2(t), A2(t), w2i(t), t) = c2i (t)x2(t) + c22w2i (t) + A2(t)x2(t), ti < t < T.
From the necessary condition of optimality described by (22) and (23), we obtain
wi2 (t) = JAi(t)(T](t]- bl(ai (W- mi(t);- c12,0 < t < ti, (41)
V cii(bi (ai(t) - mi (t)) - ci mi (t)) ' < < 1 v ;
w22 (t)
\
A2 (t)(c2 m2 (t) - b2 ( m§)(a2(t) - m2 (t))) - c22
-^-—-, t1 < t < T. (42)
Î21 (b2(m§)(a2(t) - m2(t)) - c2m2(t))
Therefore, the optimal policy for wii(t) and w2i (t) using equations (8) and (9), can be expressed as
wii (t) = 1 -J Ai (t)(c}mi(t] - bi (ai (I- m(t))\- ci2 ,0 < t < ti, (43)
w V cii(bi (ai(t) - mi (t)) - ci mi (t)) ' < < 1 y '
w2i (t) = 1 -
A2(t)(c2m2 (t) - b2( m§)(a2 (t) - m2(t))) - c22
-77-,-—-, ti < t < T. (44)
c~2i(b2(m§)(a2(t) - m2(t)) - C2m2(t))
6. Numerical Analysis
In this section, different optimal policies have been discussed on the proposed model using a numerical example. This study aims to get some view into the result and study the impact of change in efforts on the model's total cost.
Our first objective is to find the change point. We have taken PL/I database application software data, given in Table 1, which was studied by Ohba [13] to get the change point of the model. Specifically, testing for the PL/I database application software ranges from week 1 to week 19, and a total of 328 errors are found. The discussed model in section-3 comprises two phases. Generally, when the testing is carried out, the software developers team knows the change point t1. But here, we don't know the value of t1. Therefore, we first need to determine the value of t1 in the model validation.
Weeks Cumulative execution time (CPU hours) Cumulative number of detected faults Weeks Cumulative execution time (CPU hours) Cumulative number of detected faults
1 2.45 15 11 26.23 233
2 4.9 44 12 27.67 255
3 6.86 66 13 30.93 276
4 7.84 103 14 34.77 298
5 9.52 105 15 38.61 304
6 12.89 110 16 40.91 311
7 17.1 146 17 42.67 320
8 20.47 175 18 44.66 325
9 21.43 179 19 47.65 328
10 23.35 206
Table 1: Data set (Failure data from PL/I database application [13])
To determine the value of ti, we shall use the software failure increasing rate concept which is given by
y'(>) = Jim M^+M-M. (45)
At-r 0
At
For different value of At, the pattern of y'(t) is shown in Figure 2. We know that y'(t) increases until it reached its optimum due to the tester's growing fault correction experience. Afterwards, it starts to decline to stabilize the rate. A similar trend will be showing again for fault correction of another type of faults. From Figure 2, we conclude that the optimum value for failure increasing rate in phase-I is obtained at t = 6 and phase-II at t = 9. Therefore, we get the change point at ti = 7.
Figure 2: Software failure rate with respect to time t.
From the above analysis, we have concluded that the change point ti occurs at 7. For ti = 7, the software failure in phase-I and phase-II is defined when t € [0,7] and t € [7, i9] respectively.
We discuss briefly the comparative analysis of the proposed SRGM with different testing efforts. The common criteria used to compare the model are the mean-squared error (MSE), the predictive-ratio risk (PRR) and the predictive power (PP), and the results are shown in Table 2.
The MSE measures the deviation between the predicted values with the actual data and is defined as:
1
n
MSE = - £ (mit, ) - yi )2,
n ■ ,
I=i
where n is the number of observations in the model. The PRR measures the distance of model estimates from the actual data against the model estimate which is given by
PRR = t ( m(ti} -ti\ m(ti)
whereas the PP measures the distance of model estimates from the actual data against the actual data and is defined as
PP = Y(m(U) - yi "
ti V yi
MSE PRR PP
Wii(t) W2l(t) 758.15 0.5001 0.4588
0.5,
0.6
wii (t) W21 (t) 428.54 0.3408 0.3881
0.6, 0.7
wn (t) W21 (t) 292.29 0.5849 0.3934
0.7,
0.8
w11(t) = 0.8, w21(t) = 0.9 46.57 0.1982 0.4087
Table 2: Model parameters comparison criteria
We have estimated different model parameters using Excel 20i9. The estimated parameters (a, a, bi, ci, b2 and c2) and the hypothetical value of the other parameters are presented in Table 3.
Parameter W11 (t) = 0.5, wn(t) = 0.6, wn(t) = 0.7, W11 (t) = 0.8,
W21 (t) = 0.6 W21(t) = 0.7 W21(t) = 0.8 W21 (t) = 0.9
a 380 440 450 454
a 0.02 0.016 0.018 0.015
b1 0.109 0.085 0.075 0.072
C1 0.0573 0.0496 0.046 0.032
b2 0.4889 0.4718 0.465 0.3354
C2 0.0135 0.0644 0.0644 0.0185
C11 500 950 1950 2200
C21 1100 3800 15000 220000
C12 8500 21000 25000 30000
C22 5000 12000 13000 15000
Table 3: Value of parameters for different wn (t) and W21 (t).
In this analysis, we have discussed the significance of allocation of testing efforts wii(t) and w2i(t). Figure 3 describe the relationship between the cumulative number of faults removed versus time. It shows that the increase in the testing efforts wii(t) and w2i(t) will increase the number of faults removed. It has been seen that when the value of wii (t) and w2i(t) gradually increases, the rate of fault removal increases.
The analysis is also done to show how the debugging cost impacts the future cost to remove one fault and also indicates that decreasing the debugging efforts w12(t) and w22(t) will lead to a depletion in the total debugging cost. The pattern is shown in Figure 4, which says that the co-state variable decreases with time and approaches zero.
—e—yt)(w21=0.6)
— A^xw^o.s)
—<•— A2(t)(w21=0.7)
X,(t)(w21=0.8)
—A1(t)(wt)=0.7}
—4— yt)(w21=0.9)
_A (t)(w =0.8)
10 Time
Figure 4: Shadow cost vs time
In the analysis, a reduction in the total debugging cost is observed when the debugging effort decreases and hence the rate of increase of the total expenditure is reduced. Also, it shows how the Hamiltonian (H) that is the sum of testing and debugging cost, starts decreasing after some time. In short, Hamiltonian represents the instantaneous total cost of the model at time t. The results are shown in Table 4, and the total developmental cost of the SRGM with different testing efforts are 93469, 247329, 357569 and 734065, respectively.
Time (Weeks) Total Cost Total Cost Total Cost Total Cost
(Wu (t) = 0.5, (W11 (t) = 0.6, (W11 (t) = 0.7, (W11 (t) = 0.8,
W21(t) = 0.6) W21 (t) = 0.7) W21(t) 0.8) W21 (t) 0.9)
1 4886 13516 18628 25407
2 4900 13532 18651 25413
3 4915 13550 18675 25418
4 4932 13569 18702 25425
5 4950 13590 18731 25431
6 4970 13612 18763 25437
7 4991 13637 18798 25444
8 5171 14009 23142 62883
9 5371 14329 23702 63850
10 5488 14419 23594 62797
11 5507 14270 22849 59895
12 5429 13910 21611 55575
13 5270 13393 20086 50399
14 5054 12786 18477 44925
15 4806 12150 16942 39606
16 4552 11536 15577 34744
17 4307 10974 14422 30501
18 4083 10482 13482 26925
19 3887 10065 12737 23990
Table 4: Total Cost (Testing cost and debugging cost)
7. Conclusions and future work
In this article, we have presented a resource allocation technique considering a two-phase software reliability growth model with testing effort, change-point, and imperfect debugging. Numerical simulations are done which support the accuracy of our proposed model. We have given an insight into how to find the change point. Generally, the software developers team knows the change point of real software testing. This paper aims to calculate the total cost of the software using two-phase SRGM. We have proposed the theoretical results using optimal control theory, and a different approach is used to allocate testing resources optimally. We can control the testing efforts when the company switches from one testing strategy to another. We observed from the graph of the future cost of detection that it eventually reaches zero with time. Moreover, from the graph of shadow cost of correction, we observed that, as time increases, the shadow cost decreases and tends to zero. The variation in cost with the change in various model parameters has also been depicted.
In contrast with the other studies, we proposed the theoretical results to find the change point from one testing strategy to another using optimal control theory. But in this paper, experimental data analysis is utilized to determine the change point. We have used the failure increasing rate function to find the change point. To propose a more realistic SRGM, more information is needed by software managers. In this direction, the stochastic model for fault detection and correction can be used for future work. We can also extend the proposed model by incorporating different fault content functions. All these issues may be part of further work.
References
[1] Musa, J. D. (1980). The measurement and management of software reliability. Proceedings of the IEEE, 68(9), pp. 1131-1143.
Goel, A. L., & Okumoto, K. (1979). Time-dependent error-detection rate model for software reliability and other performance measures. IEEE transactions on Reliability, 28(3), pp. 206-211. Ohba, M., & Yamada, S. (1984). S-shaped software reliability growth models. In International Colloquium on Reliability and Maintainability, 4 th, Tregastel, France, pp. 430-436. Yamada, S., Tokuno, K., & Osaki, S. (1992). Imperfect debugging models with fault introduction rate for software reliability assessment. International Journal of Systems Science, 23(12), pp. 2241-2252.
Ohba, M. (1984). Software reliability analysis models. IBM Journal of research and Development, 28(4), pp. 428-443.
Li, Q., & Pham, H. (2017). NHPP software reliability model considering the uncertainty of operating environments with imperfect debugging and testing coverage. Applied Mathematical Modelling, 51, pp. 68-85.
Pham, H. (1996). A software cost model with imperfect debugging, random life cycle and penalty cost. International Journal of Systems Science, 27(5), pp. 455-463. Yamada, S., & Osaki, S. (1985). Software reliability growth modeling: Models and applications. IEEE Transactions on Software Engineering, (12), pp. 1431-1437.
Kareer, N., Kapur, P. K., & Grover, P. S. (1990). An S-shaped software reliability growth model with two types of errors. Microelectronics Reliability, 30(6), pp. 1085-1090. Pham, H. (2016). A generalized fault-detection software reliability model subject to random operating environments. Vietnam Journal of Computer Science, 3(3), pp. 145-150. Kapur, P. K., Pham, H., Anand, S., & Yadav, K. (2011). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Transactions on Reliability, 60(1), pp. 331-340.
Kapur, P. K., Pham, H., Aggarwal, A. G.,& Kaur, G. (2012). Two dimensional multi-release software reliability modeling and optimal release planning. IEEE Transactions on Reliability, 61(3), pp. 758-768.
Lo, J. H., & Huang, C. Y. (2006). An integration of fault detection and correction processes in software reliability analysis. Journal of Systems and Software, 79(9), pp. 1312-1323. Pham, H., & Zhang, X. (1997). An NHPP software reliability model and its comparison. International Journal of Reliability, Quality and Safety Engineering, 4(3), pp. 269-282. Pham, H. (2007). An imperfect-debugging fault-detection dependent-parameter software. International Journal of Automation and Computing, 4(4), pp. 325.
Kumar, V., Singh, V. B., Dhamija, A., & Srivastav, S. (2018). Cost-reliability-optimal release time of software with patching considered. International Journal of Reliability, Quality and Safety Engineering, 25(04), pp. 1850018.
Kumar, V., Sahni, R., & Shrivastava, A. K. (2016). Two-dimensional multi-release software modelling with testing effort, time and two types of imperfect debugging. International Journal of Reliability and Safety, 10(4), pp. 368-388.
Huang, C. Y., & Lin, C. T. (2006). Software reliability analysis by considering fault dependency and debugging time lag. IEEE Transactions on reliability, 55(3), pp. 436-450. Yamada, S., Ohtera, H., & Narihisa, H. (1986). Software reliability growth models with testing-effort. IEEE Transactions on Reliability, 35(1), pp. 19-23.
Musa, J. D., Iannino, A., & Okumoto, K. (1990). Software reliability. Advances in computers, 30, pp. 85-170.
Kapur, P. K., Goswami, D. N., Bardhan, A., & Singh, O. (2008). Flexible software reliability
growth model with testing effort dependent learning process. Applied Mathematical Modelling, 32(7), pp. 1298-1307.
Chang, Y. P. (2001). Estimation of parameters for nonhomogeneous Poisson process: Software reliability with change-point model. Communications in Statistics-Simulation and Computation, 30(3), pp. 623-635.
Huang, C. Y., & Kuo, S. Y. (2002). Analysis of incorporating logistic testing-effort function into software reliability modeling. IEEE Transactions on reliability, 51(3), pp. 261-270.
[24] Kapur, P. K., Goswami, D. N., & Gupta, A. (2004). A software reliability growth model with testing effort dependent learning function for distributed systems. International Journal of Reliability, Quality and Safety Engineering, 11(04), pp. 365-377.
[25] Jin, C., & Jin, S. W. (2016). Parameter optimization of software reliability growth model with S-shaped testing-effort function using improved swarm intelligent optimization. Applied Soft Computing, 40, pp. 283-291.
[26] Huang, C. Y., Lyu, M. R., & Kuo, S. Y. (2003). A unified scheme of some nonhomogenous poisson process models for software reliability estimation. IEEE transactions on Software Engineering, 29(3), pp. 261-269.
[27] Huang, C. Y., Kuo, S. Y., & Lyu, M. R. (2007). An assessment of testing-effort dependent software reliability growth models. IEEE transactions on Reliability, 56(2), pp. 198-211.
[28] Huang, C. Y., Lin, C. T., Kuo, S. Y., Lyu, M. R., & Sue, C. C. (2004, September). Software reliability growth models incorporating fault dependency with various debugging time lags. In Proceedings of the 28th Annual International Computer Software and Applications Conference, COMPSAC 2004, IEEE, pp. 186-191.
[29] Kapur, P. K., Gupta, A., Shatnawi, O., & Yadavalli, V. S. S. (2006). Testing effort control using flexible software reliability growth model with change point. International Journal of Performability Engineering, 2(3), pp. 245.
[30] Shyur, H. J. (2003). A stochastic software reliability model with imperfect-debugging and change-point. Journal of Systems and Software, 66(2), pp. 135-141.
[31] Kapur, P. K., Pham, H., Chanda, U., & Kumar, V. (2013). Optimal allocation of testing effort during testing and debugging phases: a control theoretic approach. International Journal of Systems Science, 44(9), pp. 1639-1650.
[32] Kapur, P. K., Pham, H., Kumar, V., & Anand, A. (2012). Dynamic optimal control model for profit maximization of software product under the influence of promotional effort. The Journal of High Technology Management Research, 23(2), pp. 122-129.
[33] Kumar, V., & Sahni, R. (2016). An effort allocation model considering different budgetary constraint on fault detection process and fault correction process. Decision Science Letters, 5(1), pp. 143-156.
[34] Kumar, V., Kapur, P. K., Taneja, N., & Sahni, R. (2017). On allocation of resources during testing phase incorporating flexible software reliability growth model with testing effort under dynamic environment. International Journal of Operational Research, 30(4), pp. 523-539.
[35] Kumar, V., Khatri, S. K., Dua, H., Sharma, M., & Mathur, P. (2014). An assessment of testing cost with effort-dependent fdp and fcp under learning effect: a genetic algorithm approach. International Journal of Reliability, Quality and Safety Engineering, 21(06), pp. 1450027.
[36] Ji, Y., Mookerjee, V. S., & Sethi, S. P. (2005). Optimal software development: A control theoretic approach. Information Systems Research, 16(3), pp. 292-306.
[37] Ji, Y., Kumar, S., Mookerjee, V. S., Sethi, S. P., & Yeh, D. (2011). Optimal enhancement and lifetime of software systems: A control theoretic analysis. Production and Operations Management, 20(6), pp. 889-904.
[38] Kumar, V., & Sahni, R. (2020). Dynamic testing resource allocation modeling for multi-release software using optimal control theory and genetic algorithm. International Journal of Quality & Reliability Management, 37(6/7), pp. 1049-1069.
[39] Yamada, S., & Osaki, S. (1987). Optimal software release policies with simultaneous cost and reliability requirements. European Journal of Operational Research, 31(1), pp. 46-51.
[40] Zhu, M., & Pham, H. (2018). A two-phase software reliability modeling involving with software fault dependency and imperfect fault removal. Computer Languages, Systems & Structures, 53, pp. 27-42.
[41] Naidu, D. S. (2002). Optimal control systems. CRC press.
[42] Kamien, M. I., & Schwartz, N. L. (2012). Dynamic optimization: the calculus of variations and optimal control in economics and management. Courier Corporation.
[43] Pegels, C. C. (1969). On startup or learning curves: An expanded view. AIIE Transactions, 1(3), pp. 216-222.
