Research Article

Learning to Discriminate Adversarial Examples by Sensitivity Inconsistency in IoHT Systems

Table 4

Generalization evaluation of different attacks. Rec is recall, F1 is F1-score, R is the variation of the current adversarial recall rate relative to the default effect, and F is the variation of the current weighted average F1-score relative to the default effect. denotes the baseline, which is the experimental setup for training the detector, followed by testing the detector against other attack methods with the same dataset and model.

AttackCNNLSTMBERT
Rec (%)&R(%)F1 (%)&F(%)Rec (%)&R(%)F1 (%)&F(%)Rec (%)&R(%)F1 (%)&F(%)

TextFooler99.296.397.895.49797
PWWS98.2+0.196.0+0.891.7−0.791.8−0.495.4−2.096.8+0.3
BAE99.2+196.3+0.596.5−0.195.0−0.893.2−2.894.6−0.9
Deepwordbug97.4−0.894.8−0.492.0091.8+0.493.2−3.094.9−1.0

TextFooler99.0−0.295.5−0.897.4−0.495.8+0.497.5+0.596.2−0.8
PWWS98.195.292.492.297.496.5
BAE98.8+0.694.7−1.197.2+0.694.6−1.295−1.095.3−0.2
Deepwordbug97.8−0.494.8−0.492.2+0.291.2−0.294.6−1.694.3−1.6

TextFooler98.8−0.495.9−0.496.9−0.995.5+0.197.4+0.496.3−0.7
PWWS97.6−0.696.4+1.294.6+2.493.9+1.795.6−1.895.7−0.8
BAE98.295.896.695.89695.5
Deepwordbug97.0−1.294.7−1.191.0−1.091.9+0.595.6−0.695.3−0.6

TextFooler99.4+0.296.3095.8−2.095.1−0.397.8+0.896.6−0.4
PWWS97.6−0.695.6+0.493.6+1.291.7−0.595.0−2.495.7−0.8
BAE98.6+0.495.3−0.595.1−1.594.3−1.593.8−2.293.9−1.6
Deepwordbug98.295.292.091.496.295.9