Multitask Interactive Attention Learning Model Based on Hand Images for Assisting Chinese Medicine in Predicting Myocardial Infarction

Figure 1

The framework of the proposed MTIALM. The input is the processed palm image, and then, the feature is extracted by the shared network (the shared network in the figure is ResNet-18). The shared network is connected to task-specific attention branch networks. There are two task-specific attention branches: the light blue branch (task M) is the task of detecting metacarpophalangeal joint swelling and the dark blue branch (task P) is the task of detecting palmar thenar hypertrophy. They each have their own set of attention modules. Its structure is shown in the second attention module. We connect the IIM module (grey) between the task-specific attention branches.