Automatic Coding of Facial Expressions of Pain: Are We There Yet?
Table 1
Congruency indices between manual and automatic AU coding.
Action unit
Present
Recall/sensitivity
Precision
F1
GSL
FaceReader
1—inner brow raiser
98
94
0.47
0.39
0.43
2—outer brow raiser
92
30
0.32
0.48
0.38
4—brow lowerer
393
300
0.63
0.67
0.65
5—upper lid raiser
16
23
0.32
0.12
0.17
6—cheek raiser
174
31
0.15
0.67
0.25
7—lid tightener
252
154
0.36
0.44
0.40
9—nose wrinkler
74
44
0.32
0.45
0.38
10—upper lip raiser
151
40
0.27
0.54
0.36
12—lip corner puller
50
47
0.40
0.28
0.34
14—dimpler
151
38
0.15
0.59
0.25
15—lip corner depressor
33
69
0.55
0.19
0.29
17—chin raiser
102
138
0.54
0.29
0.38
18—lip pucker
66
22
0.15
0.46
0.23
20—lip pucker
18
4
0.11
0.50
0.18
23—lip tightener
83
75
0.34
0.28
0.30
24—lip pressor
39
61
0.33
0.16
0.21
25—lips part
195
99
0.46
0.74
0.57
26—jaw drop
161
50
0.27
0.73
0.40
27—mouth stretch
1
1
0.00
0.00
0.00
43—closing of the eyes
56
136
0.71
0.19
0.30
Average
110
73
0.36
0.43
0.32
GSL: gold standard labeling (manual FACS coded data); Present: the number of times an AU was manually FACS coded or coded by FaceReader, respectively; Recall/sensitivity: ratio of manually coded AUs that were correctly detected by FaceReader; Precision: ratio of how often FaceReader is correct when classifying an AU as present; F1: trade-off between recall/sensitivity and precision via the formula: . Note: higher values are marked in bold.
