Mathematical Problems in Engineering

Volume 2019, Article ID 9381728, 15 pages

https://doi.org/10.1155/2019/9381728

## An Enhanced Adaptive Random Testing by Dividing Dimensions Independently

State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou, China

Correspondence should be addressed to Zhibo Li; moc.361@9101obihzil

Received 11 April 2019; Revised 10 August 2019; Accepted 27 August 2019; Published 13 October 2019

Academic Editor: A. M. Bastos Pereira

Copyright © 2019 Zhibo Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Random testing (RT) is widely applied in the area of software testing due to its advantages such as simplicity, unbiasedness, and easy implementation. Adaptive random testing (ART) enhances RT. It improves the effectiveness of RT by distributing test cases as evenly as possible. Fixed Size Candidate Set (FSCS) is one of the most well-known ART algorithms. Its high failure-detection effectiveness only shows at low failure rates in low-dimensional spaces. In order to solve this problem, the boundary effect of the test case distribution is analyzed, and the FSCS algorithm of a limited candidate set (LCS-FSCS) is proposed. By utilizing the information gathered from success test cases (no failure-causing test inputs), a tabu generation domain of candidate test case is produced. This tabu generation domain is eliminated from the current candidate test case generation domain. Finally, the number of test cases at the boundary is reduced by constraining the candidate test case generation domain. The boundary effect is effectively relieved, and the distribution of test cases is more even. The results of the simulation experiment show that the failure-detection effectiveness of LCS-FSCS is significantly improved in high-dimensional spaces. Meanwhile, the failure-detection effectiveness is also improved for high failure rates and the gap of failure-detection effectiveness between different failure rates is narrowed. The results of an experiment conducted on some real-life programs show that LCS-FSCS is less effective than FSCS only when the failure distribution is concentrated on the boundary. In general, the effectiveness of LCS-FSCS is higher than that of FSCS.

#### 1. Introduction

While software is increasing in scale and complexity, also the quality of software has attracted more and more attention. As an important task of software quality assurance, software testing is becoming increasingly important in software development [1].

The Software Under Test (SUT) usually has a large input domain space. Therefore, it is important to select test inputs that can effectively identify software failure as test cases. Test case generation technology, such as combinatorial testing [2], symbolic execution [3], random testing (RT) [4, 5], partition testing [6], test case generation technology based on finite state machine [7], or test case generation technology based on search [8], guides the generation of effective test cases. RT is a simple and easy-to-implement test method. It does not need complex software requirements or structural information of programs. It only requires selecting test cases randomly in the input domain. Since RT does not utilize any information of the SUT, it has the disadvantages of high redundancy, low coverage, and blindness in the test case generation. RT is even considered the worst testing method by Myers [9]. However, RT has the advantages of simplicity, easy implementation, low costs, unbiasedness, and fast execution. It is usually used in combination with other testing methods in software testing and in reliability evaluation field [10, 11]. At the same time, in theory all test cases that can be generated by any testing method can be generated by RT as well. Thus, RT has the potential to detect all failures [12].

Experimental studies [13] have found that failure-causing inputs tend to cluster in continuous areas. Based on this conclusion, Chen et al. [14] proposed adaptive random testing (ART). Compared with RT that does not use any information to generate test cases randomly, ART achieves evenly distributed test cases by using the information of success test cases.

Experiments [14] show that in failure detection, ART performs better than RT, which means that the number of test cases that are needed to trigger the first failure is lower in ART. Various algorithms based on ART have been proposed, for example, distance ART algorithm (D-ART) [14], restricted ART algorithm (RRT) [15], partitioning adaptive random testing [16], and quasi-random testing [17].

FSCS [14] is one of the most well-known ART algorithms, but it does not perform well in high-dimensional spaces and at low failure rates [1, 12]. It is pointed out in the literature [18] that the best effectiveness is reached for of RT. In order to improve the effectiveness of FSCS, the boundary effect of the test case distribution is analyzed, and a novel algorithm, the FSCS algorithm of a limited candidate set (LCS-FSCS), is proposed in this paper. LCS-FSCS effectively relieves this boundary effect and distributes test cases more evenly.

The rest of this paper is organized as follows: The distribution of test cases and the effectiveness of FSCS are analyzed in Section 2. Section 3 presents the LCS-FSCS approach. In Section 4, LCS-FSCS is compared with FSCS through simulation experiments. Settings and results of empirical studies are reported in Section 5. Threats to validity are discussed in Section 6. Finally, the conclusion and future work are presented in Section 7.

#### 2. Analysis of FSCS

FSCS uses a distance-based selection criterion to evaluate a fixed set of randomly generated test case candidates. An initial test case is selected randomly. For each subsequent test case, this test case is selected from a candidate test case that has a maximum-minimum distance to any other existing test case. Let be the executed set and be the candidate set such that . The best test case can be selected by the following formula, , where dist is defined as the Euclidean distance.

##### 2.1. Analysis of Test Case Spatial Distribution of FSCS

For a 2-dimensional input domain, we assume that each dimension has a range of [1, 1000]. Suppose that FSCS generates 100 test cases continuously without failure and runs a total of 1000 times. Finally, the distribution of test cases is analyzed in each dimension, as shown in Figures 1 and 2.