Table 4: Moving target defense effectiveness evaluation method classification.

MethodClassic literaturesAdvantagesDisadvantages

Empirical evaluation based on offensive and defensive experimentsEvans et al. 2011
Leeuwen et al. 2016
It can be used to test defense mechanism responding to different attacks.It is limited to the number of samples. Thus difficulty in evaluating MTD effectiveness in wide range.

Empirical evaluation based on simulated experimentsZhuang et al. 2012
Zheng et al. 2016
It can analyze MTD effectiveness in different application scenarios and compares in vertical.It evaluates MTD effectiveness based on prior knowledge, which is not capable of assessing MTD to unknown attacks.

Abstract evaluation based on mathematical modelClark et al. 2013
Han et al. 2014
Carroll et al. 2014
It has the advantage of universality, thus being able to compare different MTD mechanisms.Since it is difficult to abstract complicated environment and tasks in parallel, it may cause the deviation between constructed model and the actual one.

Effectiveness evaluation based on mixed analysisOkhravi et al. 2014
Zaffarano et al. 2015
Eskridge et al. 2015
Lei et al. 2016
It combines the advantages of the above analysis methods.It needs to ensure the consistency of data process and transformation using different methods.