Research Article  Open Access
Fan Li, ZeLong Yi, "A New Software Reliability Growth Model: Multigeneration Faults and a PowerLaw TestingEffort Function", Mathematical Problems in Engineering, vol. 2016, Article ID 9276093, 13 pages, 2016. https://doi.org/10.1155/2016/9276093
A New Software Reliability Growth Model: Multigeneration Faults and a PowerLaw TestingEffort Function
Abstract
Software reliability growth models (SRGMs) based on a nonhomogeneous Poisson process (NHPP) are widely used to describe the stochastic failure behavior and assess the reliability of software systems. For these models, the testingeffort effect and the fault interdependency play significant roles. Considering a powerlaw function of testing effort and the interdependency of multigeneration faults, we propose a modified SRGM to reconsider the reliability of open source software (OSS) systems and then to validate the model’s performance using several realworld data. Our empirical experiments show that the model well fits the failure data and presents a highlevel prediction capability. We also formally examine the optimal policy of software release, considering both the testing cost and the reliability requirement. By conducting sensitivity analysis, we find that if the testingeffort effect or the fault interdependency was ignored, the best time to release software would be seriously delayed and more resources would be misplaced in testing the software.
1. Introduction
Software systems are thought to run successfully when they perform without any failure during a certain period of time. However, in practice, failures are frequently observed and a perfect operational process cannot always be guaranteed due to a variety of reasons. Software reliability, which is defined as the probability of failurefree operation of a software program for a specified time interval, is an essential consideration in the processes of software development and a critical factor in measurement of software quality. In fact, accurately modeling the software reliability growth process and predicting its possible trend are of high values for software developers. For these reasons, a bunch of software reliability growth models (SRGMs), most of which are based on a nonhomogeneous Poisson process (NHPP), have been constructed [1–5]. Unlike closedsource software, open source software’s (OSS) knowledge base is not bounded to a particular organization. As a result, reliability measurement is important and complicated for OSS, such as Linux operating systems, Mozilla browser, and Apache web server. Due to their unique properties, a parallel of SRGMs has been purposefully developed to quantify the stochastic failure behavior of OSS [6, 7].
One key factor of software reliability growth modeling is the testingeffort effect [8–14]. Huang and Kuo [15] incorporated a logistic testingeffort function into software reliability modeling and showed that it provides powerful prediction capability based on real failure data. Huang [16] considered a generalized logistic testingeffort function and changepoint parameters in software reliability modeling and some practical data were employed to fit the new models. Ahmad et al. [17] incorporated the exponentiated Weibull testingeffort function into inflection Sshaped software reliability growth models. Peng et al. [18] established a flexible and general software reliability model considering both testingeffort function and imperfect debugging.
In these models, assuming independence among successive software runs may not be appropriate [19–21]. For instance, GoševaPopstojanova and Trivedi [22] introduced a software reliability model based on Markov renewal processes that covers interdependence among successive software runs. The CompoundPoisson software reliability model presented by Sahinoglu [23] considered multiple failures that occur simultaneously. Singh et al. [24] proposed new SRGMs that assume the presence of two types of faults in the software: leading and dependent. Leading faults can be removed when a failure is observed, but dependent faults are masked by leading faults and can be removed only after the corresponding leading faults are removed.
We also notice that a bundle of prior papers have examined the problems of optimal release time under different criteria. One stream studies this issue by minimizing the total cost subject to a constraint on software reliability. For example, Huang and Lin [25] provided a series of theoretical findings regarding when a software version should be released and when fault dependency and debugging time lag are considered. In addition, Ahmad et al. [26] analyzed the optimal release timing in an SRGM with Burrtype X testingeffort functions. Another stream adopts the multiattribute utility theory (MAUT) to decide the release time. For instance, Singh et al. [27] built an SRGM based on the Yamada twostage model and focused on two different attributes to investigate the release time. Pachauri et al. [28] studied an SRGM incorporating a generalized modified Weibull testingeffort function in an imperfect debugging environment and the optimal release policy is examined using both genetic algorithm (GA) and MAUT.
In this work, we propose a new model that incorporates a powerlaw testingeffort function and three generations of interdependent errors. Specifically, the detection process for the firstgeneration errors is independent, and the detection rate is proportional to the instantaneous testingeffort expenditure and the mean of remaining firstgeneration errors in the system. By contrast, the detection process of the secondgeneration errors depends on the first generation and the detection rate is proportional to not only the instantaneous testingeffort expenditure and the mean of remaining secondgeneration errors but also the ratio of removed firstgeneration errors. Similarly, the detection process of the thirdgeneration errors relies on the secondgeneration errors and the detection rate is proportional to the instantaneous testingeffort expenditure, the mean of remaining thirdgeneration errors, and the ratio of removed secondgeneration errors.
Model parameters are estimated by the nonlinear least square estimation method, which is realized by minimizing the sum of squares of the deviations between estimated values and true observations. Numerical experiments are carried out based on three versions of Apache. The estimation results show that the proposed model well fits the real observations and performs better than the traditional GoelOkumoto model and the Yamada delayed Sshaped model. Comprehensive comparisons in terms of prediction capabilities among various models are conducted to reveal that our model can predict the error occurrence process more accurately than the benchmarks. We also analyze the optimal release policy and the minimal cost for the software development managers, based on which insightful suggestions are provided. Besides, we conduct sensitivity analysis and find that resources would be misplaced if either the testingeffort effect or the error interdependency is overlooked by the software testing team. Unlike the benchmarks, our results suggest that the strategy of an earlier release time can substantially reduce the cost.
2. Software Reliability Growth Modeling
2.1. Basic Assumptions
To describe the stochastic failure behavior of software systems, we define as a counting process that represents the cumulative number of failures by time . We use the mean value function (MVF) to denote the expected cumulative number of detected failures in the time period . That is,where means the expectation operator and is called failure intensity function which indicates the instantaneous faultdetection rate at time . Specifically, for an NHPPbased SRGM, it is assumed that follows a Poisson distribution with a mean value function . That is, the model can be mathematically characterized aswhere is the Poisson probability mass function (pmf) with mean [29, 30]. Generally, one can obtain different NHPP models by taking different mean value functions.
In this paper, to characterize the stochastic behavior of the software failure process and obtain the expression for the mean value function, we make the following assumptions:(i)The software system is subject to failures at random times caused by errors remaining in the system.(ii)The error detection phenomenon in software testing is modeled by a nonhomogeneous Poisson process.(iii)Each time a failure occurs, the error causing it is immediately removed and no new errors are introduced.(iv)The faults existing in the software are of three generations and each generation is modeled by a different growth curve.(v)The mean number of errors detected in the time interval is proportional to the instantaneous testingeffort expenditure and the mean number of remaining errors in the system.(vi)The mean number of errors detected in the time interval is proportional to the instantaneous testingeffort expenditure, the mean number of remaining errors in the system, and the ratio of removed errors.(vii)The mean number of errors detected in the time interval is proportional to the instantaneous testingeffort expenditure, the mean number of remaining errors in the system, and the ratio of removed errors.(viii)Faultdetection/removal rate is a powerlaw function of testing time for all three generations of errors.
The first four assumptions in the above list are the building blocks in most prior software reliability growth models and the others are specific in our model. To facilitate our later analysis, we summarize the notations used throughout this paper in Notations section.
2.2. Proposed Software Reliability Growth Model
In general, an implemented software system is tested to detect and correct software errors in the development process. During the testing phase, software errors remaining in the system are discovered and corrected by test personnel. Based on the aforementioned assumptions, the detection rate must be proportional to the instantaneous testingeffort expenditure. Particularly, unlike traditional closedsource software, in open source software, instructions are executed by various concurrent users and each release of open source software can attract an increasing number of volunteers in the software development and/or code modification procedure. Hence, it is reasonable to assume the detection/correction rate as a powerlaw function of testing time [19, 24]. That is, the instantaneous testingeffort expenditure at time , , takes the following form:where and are parameters to be estimated by realworld data sets. As such, we can quickly obtain the cumulative amount of testingeffort expenditures by time ; that is,
As stated above, it is essential to consider fault interdependency when modeling software reliability growth processes. Specifically, in this work we propose three generations of interdependent errors. That is, the detection of the secondgeneration errors relies on the first generation, and the detection of the thirdgeneration errors depends on the second generation. Moreover, the secondgeneration errors are detectable only if the firstgeneration errors have been removed, and the thirdgeneration errors are detectable only if the secondgeneration errors have been removed. The flow chart is shown in Figure 1 to illustrate the error detecting process in a software system.
Next, we sequentially examine the mean value function for each generation of errors:
(i) For the firstgeneration errors in the software system, the number of detected errors is proportional to the number of remaining errors and the instantaneous testingeffort expenditure. Thus, the mean value function for errors satisfies the following differential equation:where is the initial content of errors in the system and represents the expected number of errors detected in time . Specifically, is assumed to be a bounded nondecreasing function of with the initial condition . Solving the above differential equation yields
(ii) The mean value function for errors satisfies the following differential equation:where is the initial content of errors in the system and stands for the expected number of errors detected in time . Specifically, is also assumed to be a bounded nondecreasing function of with the initial condition . Solving the above differential equation yields
(iii) Similarly, the mean value function for errors satisfies the following differential equation:where is the initial content of errors in the system and means the expected number of errors detected in time . Specifically, is assumed to be a bounded nondecreasing function of with the initial condition . Solving the above differential equation leads to
After this, we formulate the overall mean value function that consists of mean value functions for three generations of errors. Assume that is the expected total number of faults that will be eventually detected and , , and are corresponding ratios for each generation of errors at the beginning of testing. By noting that , , , and , we haveAs such, is a bounded nondecreasing function of , with and .
Then, based on the aforesaid mean value function, the failure intensity function for the proposed model, with testingeffort, can be explicitly given as
We further note that when , , and , the original GoelOkumoto model can be realized [31]; that is,which is one of our benchmarks for the comparative analysis. In this case, errors are assumed to be independent and the testingeffort function keeps constant, that is, . Besides, the mean value function is a concave function with respect to .
Moreover, when , we have ; namely, the instantaneous testingeffort expenditure at time reduces to a constant. In this case, we haveand we call it No model for brevity.
In addition, when and , the error interdependency is ignored and only one generation of errors exists in the software system. As a result, we haveWe call it Only model for exposition.
As another benchmark, hereby we introduce an Sshaped model, which incorporates an increasing detection rate function and well explains “learning effect” of the testing team. Essentially, the mean value function is stated as follows:It is also called Yamada delayed Sshaped model [32, 33] in literature. For this model, the detection rate first increases and then decreases with .
3. Estimation and Analysis
3.1. Data Set
To compare the proposed model against traditional models for reliability assessment, numerical examples are provided based on realworld data. Specifically, the wellknown Apache project, the most used web server software that is developed and maintained by an open community of developers, is selected in this paper for illustration. This project has large and wellorganized communities: a great number of developers have the right to update and change files freely. In particular, testing data about three versions of Apache, 2.0.35/2.0.36/2.0.39, are used. The failure data is presented in Table 1 [8], including the time and the number of errors detected for each version. Specifically, testing for Apache 2.0.35 ranges from day 1 to day 43 and in total 74 errors are found; failure data for Apache 2.0.36 are collected from day 1 to day 103 and 50 errors are detected; Apache 2.0.39 is tested over a 164day period and in the end 58 errors are discovered by the testing team.

3.2. Estimation Method
The nonlinear least square estimation (NLSE) method is a widely used estimation technique, especially for small/medium sized data sets [34]. We will use this technique to estimate parameters in models raised in this paper. More specifically, estimation is done by minimizing the sum of squared residuals, which are the squares of the differences between estimated values and true observations. The objective function for the minimization problem is stated as follows:where is the th observation, is the total number of observations, is the time in which the th error occurs, and is the corresponding estimated/theoretical value. Specifically, in this paper, can be from (11), (13), (14), (15), or (16). They correspond to our proposed model, the GoelOkumoto model, the No model, the Only model, and the Yamada delayed Sshaped, respectively.
Differentiating with respect to each unknown parameter and setting the partial derivatives to zero, we can obtain the estimated values [34, 35]. Specifically, for our proposed model, we haveUsing numerical methods to solve the set of simultaneous equations above yields the estimation for unknown parameters in our proposed model.
3.3. Metrics for Model Comparison
With estimated parameters, we can conduct comparative analysis among models based on the following metrics:
() Goodness of Fit (): the value of determines how well the curve fits the real data [19, 36]. It is defined as follows:It measures the percentage of the total variance about the mean accounted for the fitted curve. It ranges from to . The larger is, the better the model fits data. In this work, the model is considered to provide a good fit if is above 0.95.
() Mean of Square Error (MSE): the value of MSE measures the average of squares of the differences between the predicted values and the actual values [9]. It is defined as follows:Generally, the closer to zero the MSE value is, the better the model fits the data.
() Theil Statistic (TS): the value of TS indicates the average percentage deviation with respect to the actual failure data during the selected time period [37]. It is defined as follows:A smaller TS value means a smaller deviation and a better prediction capability. In this work, the model is considered to provide a highlevel prediction if the TS value is below 10%.
() Relative Error (RE): the value of RE measures the relative difference between the predicted value and the real observations [25, 38]. The definition can be stated as follows:Specifically, assuming that we have observed failures by the end of test at time , the failure data up to time () will be used to estimate parameters in the models. Substituting the estimated parameters in the mean value function yields the estimation of the number of failures by . The estimation is then compared with the actual number . Such procedure is repeated for various values of . Positive values of error indicate overestimation and negative values indicate underestimation.
3.4. Estimation Results and Model Comparison
The parameter estimation can help characterize properties of theoretical models, determine the number of errors that have already been detected/corrected in the testing process, and predict the number of errors that will be eventually encountered by users. Specifically, by minimizing the objective function in (17), we can obtain the estimated results for each model.
(i) For Apache 2.0.35, the estimated results are presented in Table 2, with , , , , and for our proposed model. Then, the ratio of errors to all types of errors is approximately 59%, the ratio of errors is around 28%, and the leftover are errors. Note that the errors consist of not only independent errors (), although and account much less than . Through incorporating such interdependency among generations of errors, we are able to strengthen the understanding towards composition of software system. Besides, we find that the estimated value of is obviously larger than 0. Hence, not considering the effect of testing effort in the analysis of the reliability growth process may lead to significantly biased results.
 
Calculated by . 
Illustrations for the theoretical values versus their observations are shown in Figure 2. We find that the theoretical values are highly consistent with the realworld observations. We can further examine the failure intensity function . Figure 3 shows that the failure intensity function is a singlepeaked curve; that is, the error detection rate first increases and then decreases with time. It in turn implies that our proposed model falls into the Sshaped category for the SRGMs [34].
We observe that our proposed model well fits the realworld software failure data. This can be seen from the high value of , that is, 0.9954, which is very close to . By comparing with the GoelOkumoto model, the Yamada delayed Sshaped model, the No model, and the Only model, values are the highest in our proposed model. This conclusion can also be drawn by comparing MSE values among the three models; our proposed model provides the lowest MSE values (2.0979). From this aspect, the model in our paper best fits the observed data about Apache 2.0.35.
Furthermore, the prediction capability in our model is very high. The low TS value for Apache 2.0.35, 0.0252, is much lower than 10% and well confirms a high prediction capability for our model. Compared with the benchmarks, our model can provide significantly lower TS values and higher predication capacity to the failure of Apache system. Lastly, the relative error in prediction using this data set is computed and results are plotted in Figure 4. We observe that the relative error approaches 0 as approaches and the relative error curve is usually within .
(ii) For Apache 2.0.36, the estimated results are presented in Table 3, with , , , , and for our proposed model. Three generations of errors account for 40%, 29%, and 31%, respectively. Obviously, and significantly contribute to the error composition, which counts for 60% in total. We also find that the estimated value of is much larger than implying that the testingeffort effect is necessary.
 
Calculated by . 
Then, we can obtain theoretical values in our proposed model. Illustrations for the theoretical values versus their observations are presented in Figure 5, which shows that the theoretical values are consistent with the realworld data. In Figure 6, we present the failure intensity function based on the estimated values. Similar to Apache 2.3.35, the humpshaped curve shows that this function possesses an increasingthendecreasing trend.
We find that our proposed model also well fits the realworld software failure data for Apache 2.0.36. This can be seen from the high value of , 0.9806. It is also much higher than those in the GoelOkumoto model, the Yamada delayed Sshaped model, the No model, and the Only model. By comparing MSE values among the models, our proposed model provides the lowest MSE value (3.6163). From this aspect, the proposed model best fits the observed data about Apache 2.0.36.
Furthermore, the TS value for Apache 2.0.36 is as low as 0.0452, which is much lower than 10% and well confirms a high prediction capability of our model. Compared with the benchmarks, our model shows a lower value of TS and a higher level of predication capability. Lastly, the relative error in prediction for this data set is computed, as shown in Figure 7. We observe that the relative error approaches 0 as approaches and the curve is usually within .
(iii) For Apache 2.0.39, the estimated results are presented in Table 4, with , , , , and for our proposed model. The ratios of three generations of errors become about 53%, 38%, and 9%, respectively. Then, the ratio for errors is approximately 59%, the ratio for errors is around 28%, and the leftover are errors. We also find that the estimated value of is significantly greater than . Thus, ignoring the effect of testing effort in the analysis of the software reliability growth curve would lead to significant bias.
 
Calculated by . 
Comparison between the fitted curve and the observed failure data is illustrated in Figure 8, which indicates that the theoretical values and the realworld observations are highly consistent. In Figure 9, we present the failure intensity function based on the estimated values. Obviously, it is also a singlepeaked curve: the error detection rate first increases and then decreases with time.
The high value of , 0.9937, suggests that our proposed model well fits the realworld software failure data for this version. By comparing with the GoelOkumoto model, the Yamada delayed Sshaped model, the No model, and the Only model, values are the highest in our proposed model. This conclusion can also be drawn by comparing MSE values among the models; our proposed model provides the lowest MSE value (0.5314). Therefore, our proposed model best fits the observed data about Apache 2.0.39.
Furthermore, the TS value for Apache 2.0.39 is 0.0144. It is much lower than 10% and well confirms a high prediction capability for our model. Compared with the benchmarks, TS values in our model are significantly lower, and prediction capacity of the failure occurrence is also much greater. Lastly, the relative error in prediction for this data set is computed and results are plotted in Figure 10. We observe that the relative error approaches 0 as approaches and the error curve is usually within .
In summary, all of three tests using different versions of Apache failure data sets suggest that our proposed model well fits the real data and presents better performance than the benchmarks in error occurrence prediction. Besides, both the testing effort and the error interdependency effects play an important role in the parameter estimation.
4. Software Release Policy
The theoretical modeling can also determine whether the software is ready to release to users or how much more effort should be spent in further testing. In this section, we formally examine the software developer’s best release time and deliver sensitivity analysis regarding how the test effort effect and error interdependency affect the optimal time and the cost.
4.1. Optimal Release Time
When the failure process is modeled by a nonhomogeneous Poisson process with a mean value function, , a popular cost structure [15, 25, 27, 39] is applied aswhere is the release time of the software to be determined. In addition, is the expected cost of removing a fault during the testing phase, is the expected cost of removing a fault during the operation phase, and is the expected cost per unit time of testing. Specifically, the expected cost of removing a fault during the operation phase is higher than that during the testing phase; that is, . Overall, the above cost function is the summation of the testing cost, including the failure cost during testing and actual cost of testing, and the cost of failure in the field.
Proposition 1. If only the cost is considered, the optimal release time satisfies .
Proof of Proposition 1. Note that the objective function is as follows:The firstorder condition yields which indicates .
It is also reasonable to quit testing as soon as the number of remaining faults in the system is less than a prescribed portion of total faults. Hence, a simple criterion for software release is given [8, 25]:which is the typeI reliability definition.
Alternatively, given that the testing or the operation has been conducted up to time , the probability that a software failure will not occur in the time interval , with , can be given by the following conditional probability [3, 40]:which is the typeII reliability definition.
In most situations, the project managers need to strike a balance between the cost and the reliability level. That is, on one hand, the reliability level should be at least as high as a desired level set beforehand; on the other hand, given the target level of software reliability , the cost minimization problem over the release time should be imposed. Considering this tradeoff yieldsfor the typeI reliability definition or for the typeII reliability definition.
As an illustrative example, Figure 11 presents how the software reliability and the total cost change with time. In this example, we use the data set with Apache 2.0.36 and take , $, $, and $. Suppose that the reliability must be higher than 0.95; that is, . Then, the reliability constraint can be satisfied only if for the typeI reliability definition (or for the typeII reliability definition). When only the total cost is considered, we find that the cost is minimized at . Furthermore, when both criteria (reliability and cost) are taken into account, we find that for the typeI reliability definition (or for the typeII reliability definition).
In addition, for Apache 2.0.35, the reliability level is above 0.95 only if if the typeI reliability definition is taken (or for the typeII reliability definition). When only the cost is considered, it is minimized at . Furthermore, when both criteria (reliability and cost) are taken into account, we find that for the typeI reliability definition (or for the typeII reliability definition).
Similarly, for Apache 2.0.39, the reliability level is higher than 0.95 only if if the typeI reliability definition is taken (or for the typeII reliability definition). When only the cost is considered, the minimization is attained at . When both reliability and cost are considered, we find that for the typeI reliability definition (or for the typeII reliability definition).
4.2. Sensitivity Analysis
As mentioned above, we have incorporated both the testingeffort and error interdependency effects into our proposed model. In this part, we intend to examine what if the testing effort, the error interdependency, or both of them are ignored in decisionmaking of software release. The first case corresponds to the No model (14), the second case corresponds to the Only model (15), and the third case is just the GoelOkumoto model (13).
For each model, we shall examine the optimal release time and the corresponding minimal cost when a desired reliability level is set beforehand. The data set with Apache 2.0.36 is used to illustrate our findings. Under the typeI reliability definition for reliability measurement, the optimal release time and the minimal cost as a function of are presented in Figures 12 and 13, respectively. Similarly, under the typeII reliability definition, the optimal release time and the minimal cost as a function of are shown in Figures 14 and 15, respectively. Other parameters take the same values as analysis in Section 4.1; that is, , $, $, and $.
Under either typeI or typeII reliability definition, both the optimal release time and the total cost would be mistakenly estimated if either testing effort or fault interdependency is ignored. Unlike these two cases, our proposed model always suggests an earlier release time and a lower testing cost.
As an example, we compare the GoelOkumoto model and our proposed model when the typeI reliability definition is adopted and is set as 95%. The GoelOkumoto model indicates the best release time to be 76.56 while that for our proposed model is 54.3. Hence, the testing lasts much longer if the GoelOkumoto model is used to decide the stopping time. Similarly, using the GoelOkumoto model yields the minimal cost as 2529$ while that for our proposed model is just 1896$, implying that a decision based on our model can save the software developers as much as 25% in cost.
Alternatively, when the typeII reliability definition is adopted, the GoelOkumoto model predicts the best release time as 132.2 while that for our proposed model is 66.75, which suggests that the testing duration in our model is doubled compared with traditional GoelOkumoto model. Similarly, according to the GoelOkumoto model, the minimal cost is 3083$ while that for our proposed model is merely 2000$. Thus, a decision based on our model can save the software developers as much as 35% in terms of the cost.
5. Concluding Remarks
In the development process, a variety of tests need to be conducted to ensure the reliability of the software system. In this work, we propose a framework for software reliability modeling that simultaneously captures effects of testing effort and interdependence between error generations. In order to evaluate our proposed model, we compare its performance with other existing models by doing experiments with actual software failure data, that is, three versions of Apache. The nonlinear least square estimation technique is applied for parameter estimation, and values of , mean of square error, Theil Statistic, and relative error are used for model comparison in terms of fitting quality and prediction capability. The numerical analysis from all three Apache versions demonstrates that the proposed model fits the realworld data very well and exhibits a high level of prediction capability.
Our theoretical results can help software project managers assess the most costeffective time to stop software testing and to release the product. Specifically, we have formally examined the release timing and provided useful managerial insights based on the Apache data sets. Moreover, our proposed model recommends an earlier release time and a reduced cost compared with the GoelOkumoto model and the Yamada delayed Sshaped model. Therefore, our modeling essentially enhances accuracy of the understanding towards the software reliability growth process and the best time to release a software version.
Notations
The th generation of fault  
Testing time  
Mean value function of the th generation faults  
Mean value function for all three generations of faults  
Theth observation  
Failure intensity function  
Cumulative number of software errors detected by testing time  
Initial content of all three generations of faults  
Proportion of generation fault  
Initial content of generation fault  
Coefficient for the testingeffort function  
Power of the testingeffort function  
Testingeffort expenditure at time  
Cumulative testingeffort expenditure by time  
Metric for goodness of fit  
Mean of square error  
Theil Statistic  
Relative error  
Total testing cost  
TypeI reliability definition of software systems  
TypeII reliability definition of software systems  
Target level of reliability  
Testing cost coefficient. 
Competing Interests
The authors declare that they have no competing interests.
References
 P. K. Kapur, S. Anand, S. Yamada, and V. S. S. Yadavalli, “Stochastic differential equationbased flexible software reliability growth model,” Mathematical Problems in Engineering, vol. 2009, Article ID 581383, 15 pages, 2009. View at: Publisher Site  Google Scholar
 P. K. Kapur, A. Gupta, and V. S. S. Yadavalli, “Software reliability growth modeling using power function of testing time,” International Journal of Operations and Quantitative Management, vol. 12, no. 2, pp. 127–140, 2006. View at: Google Scholar
 C.Y. Huang, M. R. Lyu, and S.Y. Kuo, “A unified scheme of some nonhomogenous poisson process models for software reliability estimation,” IEEE Transactions on Software Engineering, vol. 29, no. 3, pp. 261–269, 2003. View at: Publisher Site  Google Scholar
 C.J. Hsu and C.Y. Huang, “Optimal weighted combinational models for software reliability estimation and analysis,” IEEE Transactions on Reliability, vol. 63, no. 3, pp. 731–749, 2014. View at: Publisher Site  Google Scholar
 H. Okamura, T. Dohi, and S. Osaki, “Software reliability growth models with normal failure time distributions,” Reliability Engineering and System Safety, vol. 116, pp. 135–141, 2013. View at: Publisher Site  Google Scholar
 Y. Tamura and S. Yamada, “A componentoriented reliability assessment method for open source software,” International Journal of Reliability, Quality and Safety Engineering, vol. 15, no. 1, pp. 33–53, 2008. View at: Publisher Site  Google Scholar
 Y. Tamura and S. Yamada, “Comparison of software reliability assessment methods for open source software,” in Proceedings of the 11th International Conference on Parallel and Distributed Systems, vol. 2, pp. 488–492, IEEE, Fukuoka, Japan, July 2005. View at: Publisher Site  Google Scholar
 X. Li, Y. F. Li, M. Xie, and S. H. Ng, “Reliability analysis and optimal versionupdating for open source software,” Information and Software Technology, vol. 53, no. 9, pp. 929–936, 2011. View at: Publisher Site  Google Scholar
 X. Li, M. Xie, and S. H. Ng, “Sensitivity analysis of release time of software reliability models incorporating testing effort with multiple changepoints,” Applied Mathematical Modelling, vol. 34, no. 11, pp. 3560–3570, 2010. View at: Publisher Site  Google Scholar  MathSciNet
 S. M. Rafi, K. N. Rao, and S. Akthar, “Incorporating generalized modified Weibull TEF in to software reliability growth model and analysis of optimal release policy,” Computer and Information Science, vol. 3, no. 2, article 145, 2010. View at: Google Scholar
 C. Jin and S.W. Jin, “Parameter optimization of software reliability growth model with Sshaped testingeffort function using improved swarm intelligent optimization,” Applied Soft Computing, vol. 40, pp. 283–291, 2016. View at: Publisher Site  Google Scholar
 J. Wang, Z. Wu, Y. Shu, and Z. Zhang, “An imperfect software debugging model considering loglogistic distribution fault content function,” Journal of Systems and Software, vol. 100, pp. 167–181, 2015. View at: Publisher Site  Google Scholar
 S. Yamada, H. Ohtera, and H. Narihisa, “A testingeffort dependent software reliability model and its application,” Microelectronics Reliability, vol. 27, no. 3, pp. 507–522, 1987. View at: Publisher Site  Google Scholar
 S. Yamada and H. Ohtera, “Software reliability growth models for testingeffort control,” European Journal of Operational Research, vol. 46, no. 3, pp. 343–349, 1990. View at: Publisher Site  Google Scholar
 C.Y. Huang and S.Y. Kuo, “Analysis of incorporating logistic testingeffort function into software reliability modeling,” IEEE Transactions on Reliability, vol. 51, no. 3, pp. 261–270, 2002. View at: Publisher Site  Google Scholar
 C.Y. Huang, “Performance analysis of software reliability growth models with testingeffort and changepoint,” Journal of Systems and Software, vol. 76, no. 2, pp. 181–194, 2005. View at: Publisher Site  Google Scholar
 N. Ahmad, M. G. M. Khan, and L. S. Rafi, “A study of testingeffort dependent inflection Sshaped software reliability growth models with imperfect debugging,” International Journal of Quality and Reliability Management, vol. 27, no. 1, pp. 89–110, 2010. View at: Publisher Site  Google Scholar
 R. Peng, Y. F. Li, W. J. Zhang, and Q. P. Hu, “Testing effort dependent software reliability model for imperfect debugging process considering both detection and correction,” Reliability Engineering and System Safety, vol. 126, pp. 37–43, 2014. View at: Publisher Site  Google Scholar
 V. B. Singh, G. P. Singh, R. Kumar, and P. K. Kapur, “A generalized reliability growth model for open source software,” in Proceedings of the 2nd International Conference on Reliability, Safety and Hazard (ICRESH '10), pp. 523–528, IEEE, Mumbai, India, December 2010. View at: Publisher Site  Google Scholar
 A. Pievatolo, F. Ruggeri, and R. Soyer, “A Bayesian hidden Markov model for imperfect debugging,” Reliability Engineering and System Safety, vol. 103, pp. 11–21, 2012. View at: Publisher Site  Google Scholar
 T. Aktekin and T. Caglar, “Imperfect debugging in software reliability: a Bayesian approach,” European Journal of Operational Research, vol. 227, no. 1, pp. 112–121, 2013. View at: Publisher Site  Google Scholar
 K. GoševaPopstojanova and K. Trivedi, “Failure correlation in software reliability models,” in Proceedings of the 10th International Symposium on Software Reliability Engineering (ISSRE '99), pp. 232–241, November 1999. View at: Google Scholar
 M. Sahinoglu, “CompoundPoisson software reliability model,” IEEE Transactions on Software Engineering, vol. 18, no. 7, pp. 624–630, 1992. View at: Publisher Site  Google Scholar
 V. B. Singh, K. Yadav, R. Kapur, and V. S. S. Yadavalli, “Considering the fault dependency concept with debugging time lag in software reliability growth modeling using a power function of testing time,” International Journal of Automation and Computing, vol. 4, no. 4, pp. 359–368, 2007. View at: Publisher Site  Google Scholar
 C.Y. Huang and C.T. Lin, “Software reliability analysis by considering fault dependency and debugging time lag,” IEEE Transactions on Reliability, vol. 55, no. 3, pp. 436–450, 2006. View at: Publisher Site  Google Scholar
 N. Ahmad, M. G. M. Khan, S. M. K. Quadri, and M. Kumar, “Modelling and analysis of software reliability with Burr type X testingeffort and releasetime determination,” Journal of Modelling in Management, vol. 4, no. 1, pp. 28–54, 2009. View at: Publisher Site  Google Scholar
 O. Singh, P. K. Kapur, and A. Anand, “A multiattribute approach for release time and reliability trend analysis of a Software,” International Journal of Systems Assurance Engineering and Management, vol. 3, no. 3, pp. 246–254, 2012. View at: Publisher Site  Google Scholar
 B. Pachauri, A. Kumar, and J. Dhar, “Software reliability growth modeling with dynamic faults and release time optimization using GA and MAUT,” Applied Mathematics and Computation, vol. 242, pp. 500–509, 2014. View at: Publisher Site  Google Scholar
 M. Xie, Software Reliability Modelling, vol. 1, World Scientific, River Edge, NJ, USA, 1991.
 M. R. Lyu, Handbook of Software Reliability Engineering, vol. 222, IEEE Computer Society Press, Los Alamitos, Calif, USA, 1996.
 A. L. Goel and K. Okumoto, “Timedependent errordetection rate model for software reliability and other performance measures,” IEEE Transactions on Reliability, vol. 28, no. 3, pp. 206–211, 1979. View at: Publisher Site  Google Scholar
 S. Yamada, M. Ohba, and S. Osaki, “Sshaped reliability growth modeling for software error detection,” IEEE Transactions on Reliability, vol. 32, no. 5, pp. 475–484, 1983. View at: Publisher Site  Google Scholar
 S. Yamada, M. Ohba, and S. Osaki, “Sshaped software reliability growth models and their applications,” IEEE Transactions on Reliability, vol. 33, no. 4, pp. 289–292, 1984. View at: Publisher Site  Google Scholar
 A. Wood, “Predicting software reliability,” Computer, vol. 29, no. 11, pp. 69–77, 1996. View at: Publisher Site  Google Scholar
 T. F. Coleman and Y. Li, “On the convergence of interiorreflective Newton methods for nonlinear minimization subject to bounds,” Mathematical Programming, vol. 67, no. 2, pp. 189–224, 1994. View at: Publisher Site  Google Scholar  MathSciNet
 K. Pillai and V. S. S. Nair, “A model for software development effort and cost estimation,” IEEE Transactions on Software Engineering, vol. 23, no. 8, pp. 485–497, 1997. View at: Publisher Site  Google Scholar
 K.C. Chiu, Y.S. Huang, and T.Z. Lee, “A study of software reliability growth from the perspective of learning effects,” Reliability Engineering and System Safety, vol. 93, no. 10, pp. 1410–1421, 2008. View at: Publisher Site  Google Scholar
 Y. P. Wu, Q. P. Hu, M. Xie, and S. H. Ng, “Modeling and analysis of software fault detection and correction process by considering time dependency,” IEEE Transactions on Reliability, vol. 56, no. 4, pp. 629–642, 2007. View at: Publisher Site  Google Scholar
 M. Xie and B. Yang, “A study of the effect of imperfect debugging on software development cost,” IEEE Transactions on Software Engineering, vol. 29, no. 5, pp. 471–473, 2003. View at: Publisher Site  Google Scholar
 S. Yamada, Software Reliability Modeling: Fundamentals and Applications, Springer Briefs in Statistics, Springer, Tokyo, Japan, 2014. View at: Publisher Site  MathSciNet
Copyright
Copyright © 2016 Fan Li and ZeLong Yi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.