Automated Search-Based Robustness Testing for Autonomous Vehicle Software

Table 5

Results of statistical hypothesis testing for mean of 50 most challenging test cases.

Trial	1	2	3	4	5	6
Simulation executions	50	100	200	500	1000	2000
H0: μ ≥ μ; H1: μ < μ	Fail to reject H0; p value = 0.82	Fail to reject H0; p value = 0.99	Reject H0; p value < 0.001	Reject H0; p value < 0.001	Reject H0; p value < 0.001	Reject H0; p value < 0.001
H0: μ ≥ μ; H1: μ < μ	Reject H0; p value = 0.008	Reject H0; p value < 0.001	Reject H0; p value < 0.001	Reject H0; p value < 0.001	Reject H0; p value < 0.001	Reject H0; p value < 0.001