Deep Hash with Optimal Transport-Based Domain Adaptation for Multisite MRI Retrieval

Figure 2

Illustration of the proposed multisite fMRI retrieval (MSFR) method for resting-state functional MRI (rs-fMRI) retrieval. It includes two major components: (a) generating functional connectivity features and (b) retrieval model learning. The raw rs-fMRI was first preprocessed using C-PAC, then the brain was divided into 116 brain regions using the AAL brain template, then the mean time series of each brain region were extracted, the Pearson correlation coefficient between the time series of pairwise brain regions was calculated to obtain the functional connectivity matrix, and finally, the upper triangular elements were retained to obtain the functional connectivity features. The proposed retrieval framework consists of submodels corresponding to the source domains, each of which includes domain adaptation based on optimal transport and hash learning based on a central similarity metric. Blue indicates the first source domain, yellow indicates the -th source domain, and orange indicates the target domain. The inputs to each submodel are source and target domain data, where the neural network assigned to each source domain is represented twice as a way of distinguishing between the source and target domain inputs. The labeled source domain data is used to learn the hash function, and the unlabeled target domain data is used only in the domain adaptation phase. The dotted line in the figure indicates that the target domain data only uses the hash coding layer in the testing stage. The retrieval database consists of a hash code obtained from the hash representation of the source domain in its corresponding hash coding layer, followed by the function. The query samples are obtained from the hash representation of the target domain data in each submodel and then processed by the aggregate module before being obtained by the function. The aggregate block is responsible for summing up the hash representations generated by each target domain sample in each hash coding layer to obtain the final hash representation of the sample.

(a)

(b)