Abstract

This research proposes a noncontact heart rate measurement method using medical radar and artificial intelligence techniques. This technique has a significant role in the design and development of a wireless system that monitors the body’s vital signs. Firstly, based on a signal model describing chest surface movement, we propose a method to create a dataset for the training process using the long-short-term memory model. Secondly, a novel method to extract chest motion from the received radar signal is proposed. Finally, the heart rate will be estimated by using the obtained model and the received motion signal. The performance of the proposed method is evaluated through the root mean square error parameter as well as compared with other methods. Experimental results evaluated according to Bland–Altman achieved an accuracy of 96.67%.

1. Introduction

The evolution of wireless technologies has promoted new applications in the medical field, especially, in health monitoring. Recent studies have mainly explored the application of radar in noncontact monitoring of human vital signs such as heart rate and breathing rate [112]. The radar used in monitoring the body’s vital signs can be ultrawideband (UWB) radar [2, 3], continuous wave (CW) radar [48], and frequency-modulated continuous wave radar (FMCW) [9].

With the development of artificial intelligence, machine learning models are being studied and applied to the field of signal processing to improve the accuracy of systems. Some recent studies refer to the use of artificial intelligence in health monitoring using radar [7, 8]. The application of artificial intelligence in heart rate extraction from radar signals guarantees simple system operation while improving accuracy. However, the shortage of training and testing data sets is a major challenge in the machine learning field that significantly affects the accuracy and quality of the machine learning models.

From the above challenges, this paper proposed a novel heart rate measurement system by using a medical radar system and artificial intelligence. To address the shortage of training and testing data for the artificial intelligence model, we proposed a technique to model the heartbeat signal obtained from the radar based on the theories of radar transceivers. Then, we generated data with the corresponding heart rate labels as input to the deep learning model. CW radar is used to ensure simplicity in radio structure, low energy consumption, and large detection range. However, the raw received signal from the radar includes information on many factors such as the vibration of the chest surface caused by heartbeat, respiration activity, and noise. Therefore, in order to solve this problem, the chest motion extraction method is proposed. Based on the generated data, the deep learning algorithm long-short-term memory (LSTM) is used to develop the estimation model which will calculate the heart rate from the fine-tuned chest motion signal. Heart rate classification results are referenced with the signal from the photoplethysmography (PPG) device. In this study, to evaluate the performance, the acquired machine learning model will be tested on 2 data sets: the data set published on [13] and the data set collected from the proposed system.

The article layout is organized into five sections. Section 2 presents the materials and methods used in this research. The proposed system is described in Section 3. Section 4 shows the experiments in practical and received results. And the conclusion is in Section 5.

2. Materials and Methods

2.1. Medical Radar System

In this work, a CW radar, NJR4262 J, is used with an operating frequency of 24 GHz, a MMIC chip for radio frequency processing, a built-in antenna, and an I/Q output with high accuracy and long-term frequency stability.

The NJR4262 J radar transmits a continuous wave to the target chest surface, which is moving because of respiration and cardiac activity, and then receives the reflected signal. The frequencies of these two signals differ due to the Doppler effect caused by the movement of the target’s chest surface. A built-in mixer compares the transmitted and received signals. Information on the chest motion is extracted by calculating the difference between the transmitted signal and the received signal. In the output, there are the in-phase signal and the quadrature signal .

The acquired signal from the radar is digitalized via a converter named ADC NI USB-6008. The outputs from the radar module are connected in turn to the analog input ports of the ADC. To collect the digital data, LabVIEW software is used to connect the analog-digital converter to computers. The data is sampled with a sampling rate of 100 Hz and recorded as csv-extension files. Figure 1 visualizes the connections of the system.

2.2. Model Architecture

Long-short-term memory is a recurrent neural network architecture-based version that addresses the problem of long-term dependence for sequence signals in the time domain. Each node in the network contains a memory cell that can store or forget previous information if necessary [14]. LSTM networks are widely used for prediction problems involving sequence signal data.

In this study, the LSTM network is used, which acts as the feature extraction layer of the signal. The output of this layer is linked to a classifier with a fully connected 3-layer architecture. The network architecture diagram is shown in Figure 2. The filtered motion signal will be fed into the network and normalized to a vector with a length of 2000. The inputs to the first and second fully connected layers are 1024 and 512, respectively, with the nonlinear activation function ReLU, and produce an output of size 256. At the last fully connected layer, the soft-max function is used to give the label with the highest probability. The outputs are labels corresponding to a heart rate range of 50 to 140 beats per minute. The labels are encoded as one-hot encoding and assigned to each signal data. Training processing uses the categorical loss entropy function and the optimization function in this work is ADAM.

To evaluate the performance of the trained model, root mean square error is applied with the formula as follows:where is the estimated heart rate. In this work, heart rate values are determined through the peak detection algorithm and the proposed method. is reference heart rate value obtained from the PPG device. represents the number of test signal samples.

3. Proposed System

Figure 3 presents the general picture of the proposed system. The system is divided into the following 5 functional blocks: data generator, training model, data acquisition, motion extraction, and heart rate estimation. In the data generator block, the amplitude of motion of the chest surface is modeled by using mathematical formulas. These signals are generated based on amplitude, frequency, and phase characteristics. This signal set is then divided into datasets corresponding to heart rate, forming a database that acts as the input to the training model block. After the model training, the trained model will be used to estimate the heart rate.

The data acquisition and motion extraction blocks perform the acquisition of radar data from a noncontact medical radar system. The baseband signals are collected, normalized, and corrected. From the processed baseband signals, the target motion is extracted and passed through a bandpass filter. Motion signals will be fed through a machine learning model that has been trained to estimate heart rate. This process is executed in the heart rate estimation block.

3.1. Signal Modeling
3.1.1. Signal Modeling of CW Radar

The transmitter transmits a signal expressed by the following formula:

Radiofrequency waves are emitted from the radar, directed towards the target surface of the chest of the individual being measured. In which, is the amplitude of the transmitted signal, , is the carrier frequency, and equals 24 GHz, is the phase of noise. According to the Doppler principle, the reflected wave will have a difference in frequency due to the movement of the chest surface caused by respiration and the impact of the heart on the chest wall. Thus, the reflected signal is modeled as follows:where is the amplitude of the received signal, is the wavelength and equals , is the speed of light, is the initial distance between the radar and the body surface, and represents the motion of the body surface.

When there is a received signal, the received and transmitted signals will be put through a frequency mixer. Its output will give two baseband signals. The signal is an in-phase signal, denoted by . The other signal is a quadrature-phase signal , denoted by . The two signals are out of phase by an angle of .

3.1.2. Chest Surface Movement Modeling

The amplitude of chest surface motion is affected by three components: respiration, cardiac effects on the chest wall, and noise, which is assumed to be white noise:

Based on [6], the authors demonstrated that for the motion of the chest surface due to cardiac action, the amplitude could be modeled as a sinusoidal pulse, a half-sinusoidal pulse, a Gaussian pulse, or a combination of two different pulses per heartbeat. The sinusoidal pulse can be chosen to represent the action of the heart on the chest wall.where is the amplitude of the chest vibration that is affected by the heart, is the heartbeat frequency, and is the initial phase.

For the vibration component due to respiration , the motion consists of two phases such as inhalation and exhalation. The authors in [15] proved that the vibration of the chest surface caused by the respiration component is modeled as the following formula:where is the amplitude at which respiration causes thoracic surface movement, T is the duration of one respiratory cycle, and are the inhalation and exhalation duration in one respiratory cycle, respectively, is the time constant.

3.2. Data Generation

Based on the analysis of radar signal structure along with the above mathematical analysis of chest surface movement, this study proposes to develop a database of chest surface motion signals corresponding to possible human heart rate values ranging from 50 ÷ 140 beats per minute (bpm). The signal carrying heart rate information was determined by using (7). Heart rate varies between 50 and 140 bpm, corresponding to a frequency of 0.83 to 2.33 . The signal amplitude carrying heart rate information ranges in magnitude from 0.2 to 0.5 mm, which was reported in [16, 17].

The effect of respiration on the chest wall was presented in [15] and expressed in (8). Therefore, the respiratory rate varies between 10 and 22 beats per minute, corresponding to the period from 2.73 to 6 seconds. The amplitude of chest wall change caused by respiration is shown in [18, 19], with a magnitude ranging from 4 to 12 mm.

The goal of the proposed method is to model the signal and extract the signal carrying heart rate information from the chest wall motion. As shown in (6), the signal contains heartbeat information along with the respiratory signal and noise that are components of the chest surface motion signal. The chest surface movement is the major cause of the Doppler effect on the receiver of the radar. Since the frequency components of the heartbeat signal are mainly located in the signal, the modeling heartbeat signal was obtained using a bandpass filter to extract the heartbeat signal from the modeling signal instead of the and signals from (4) and (5). Figure 4 presents the modeled chest surface motion signal. The signal consisted of heartbeat, respiration, and noise components. The chest surface motion signal in the frequency domain is shown in Figure 5. The chest surface motion signal contains the frequencies of the respiratory, heartbeat signal, respiratory harmonics, and white noise. The effects of respiratory harmonics were presented in [11].

Figure 6 shows a modeling signal carrying heart rate information which is extracted using a order Butterworth filter with the chest surface motion modeling signal. This signal would be the input of the training process. The dataset generated for training includes 91 labels of heart rate data ranging from 50 to 140 beats per minute. Each label includes 4000 samples of heart rate data. The dataset is then divided into two sets of training and validation according to the data ratio of 80 : 20.

3.3. Motion Extraction Method

Data collected from the radar is obtained simultaneously with the reference device PPG. The signal from the PPG device is an analog signal, digitized and sampled at a frequency of 100 Hz via the NI USB-6008 ADC.

3.3.1. Correcting Baseband Signals

As analyzed above, the motion of the target surface influences the baseband signals. In fact, there are many factors that affect the amplitude and frequency, causing the received signal to be distorted, and affecting the amplitude and phase components of this baseband signal. This causes the received signal to deviate from the signal models obtained at the database creation step. Therefore, the signals received at the radar need to be processed and restored. In this study, we propose to apply the Eclipse Fit method to process and restore the distortion-affected signals mentioned above [20]. In practice, baseband signals are represented by the following formulas:

Under ideal conditions,  = . However, the signal distortion factor makes the above components unequal and creates imbalance factors,  =  and . Based on [20], (10) and (11) are varied to get the following equation:

Consider as values on the horizontal axis and as values on the vertical axis in the Cartesian coordinate system, the normalization equation of an ellipse can be written as:

According to (11), the imbalance factors can be estimated as follow:

For the whole data with N points (N >> 5), these data points will satisfy the formula:

The coefficient matrix can be written as:

Finally, the parameters A, B, C, D, and E can be determined using the following equation:

From there, the imbalance factors values can be estimated through (13) and (14).

After the baseband signals are corrected for amplitude, based on [21] the 1-dimensional components will be found by optimizing the following function based on the gradient descent algorithm:

3.3.2. Extracting Motion Signals

After determining the DC components, the baseband signal data will be corrected using the Gram–Smith procedure.where are the corrected baseband signals.

Arctangent modulation is applied to extract motion signals from and baseband signals as follows:

4. Experiments and Results

4.1. Evaluation with Published Data

First, the proposed method will be implemented with the dataset collected by the authors in [13]. This dataset includes 30 volunteers. Each participant was asked to perform sampling for 5 different scenarios: normal breathing, sleep apnea, Valsalva, and tilt up and tilt down. The collected radar data was synchronized with the electrocardiogram (ECG) signal. Details of the sampling time for each context are presented in [13]. In this study, we only used the data of volunteers in the case of normal breathing.

Data for each volunteer is approximately 10–14 minutes in length for normal breathing. The data samples are split with signal windows of length 20 s. The target motion signal is then extracted through arctangent modulation and filtered using a Butterworth filter in the range of 0.83 to 2.33 Hz. These filtered signal samples are fed into the model for estimation. This result is then averaged to find the number of beats per minute for each volunteer over the measured periods. The estimated results of the proposed method will be compared with the arctangent method with the referenced standard value from the ECG signal published in [13].

The comparison results between the method applying only arctangent modulation and the proposed method are presented in Table 1. The results indicate that there is a huge improvement when applying the trained machine learning model to the heart rate estimation. The RMSE of the proposed method is 2.35 bpm, significantly lower than 8.74 bpm when only the arctangent modulation method is applied.

4.2. Evaluation with Experimental Data

This section presents the results of evaluating the effectiveness of the proposed system with the actual experiment. The actual measured data is also tested with the arctangent method and peak counting method and then compared with the results of the proposed system. The reference data for this comparison is the PPG data collected simultaneously during the measurement.

4.2.1. System Establishment

Figure 7 illustrates a volunteer participating in the proposed system in practice. The volunteer sat at an initial distance of approximately 30 cm away from the radar. The radar altitude was adjusted to be equal to chest height. His right hand was fitted with the PPG device to obtain reference data. The radar and reference data were fed into the ADC, which transferred the data from the equipment to the computer.

4.2.2. Data Acquisition

The chest wall motion signal obtained from the radar, after being processed into the baseband, will be passed through a 5th-order Butterworth bandpass filter to obtain the heartbeat signal. The signal is then amplitude normalized as an input to the classification model. Figure 8 depicts the motion signal extracted from the radar. To obtain the heartbeat signal, a bandpass filter with a frequency range of 0.83 Hz to 2.33 Hz is used to obtain a heart rate with a range of 50 to 140 bpm. The filtered heartbeat signal is presented in Figure 9.

The Fourier transform of the signal was performed as shown in Figure 10. The signal was strongly influenced by two major frequency components. The frequency component of 1.55 Hz was the frequency of the heartbeat, and the frequency component of 0.95 Hz was caused by the harmonics of respiration. The other noise components were also significant in magnitude relative to the desired frequency. High-intensity noise components still existed in the frequency band of the heartbeat signal because the system did not apply harmonic removal and noise reduction methods. This is also an important basis to evaluate the classification ability of the model in noisy conditions. To evaluate the results of heart rate classification, a PPG reference signal was used as shown in Figure 11.

4.2.3. Heart Rate Estimation Results

The process of testing the trained model for real data is presented in this section. The study participants are students from Hanoi University of Science and Technology, who are between the ages of 18 and 23. The system evaluated the results on 30 samples. After collecting data and using the proposed method, combined with a comparison with the peak counting method, the arctangent method, and the PPG reference device, the obtained results are shown in Table 2.

From the data shown in Table 2, the proposed method gave quite good results with the RMSE value of 3.18 bpm. This value is better than that obtained by the original arctangent modulation algorithm (RMSE = 5.74 bpm) and is clearly superior to the peak counting method (RMSE = 10.51 bpm). Obviously, it can be said that the proposed method has better performance than the previously published methods in the context of implementation in real-time applications.

The correlation between the estimated value from the proposed method and the reference value was investigated. The correlation between two datasets from two measurements, A and B, is expressed by the Pearson correlation, , expressed by the following formula:where is the number of samples; and are data points from two measurements A and B, respectively; and are the averages of two measurements A and B, respectively.

The coefficient indicates the strength of the correlation between two data sets. Based on the data in Table 3, the correlation coefficients between the peak counting algorithm, the arctangent modulation algorithm, and the proposed method with reference data are calculated. The arctangent technique and the reference data had a correlation coefficient of 89.57%, the algorithm to the peak had a correlation value of 50.27%, and the suggested approach had a correlation coefficient of 96.64%. The correlation coefficient between the proposed method and the reference data shows a large correlation between the two measurements. Thus, the application of a machine learning model can improve the correlation and accuracy of the heart rate estimation problem.

The B and A plots are shown in Figure 12. The standard deviation (SD) of the difference between the two measurements was 3.40 bpm. The mean difference between the two measurements was 0.967 bpm and is shown as a solid line. The space between the two dashed lines above and below the mean difference line (±1.96∙SD) represents the acceptable limit. The results from the above graph show that 29/30 data points are within the allowable difference range (−1.96∙SD, +1.96∙SD), reaching a rate of 96.67%.

The results obtained by the proposed method are compared with the results of previous studies as presented in Table 3. Studies [22] and [23] have achieved positive results in heart rate monitoring. But the heart rate monitoring results are still fluctuating due to the impact of the environment. LSTM helps to minimize the effects of noise because this extraction layer is capable of learning and extracting the most characteristic information. Therefore, based on the results from the study in [7], the application of the LSTM network has improved the results better than the application of ANN. Obviously, compared with previous studies, the proposed method has improved the accuracy as well as reduced the system complexity.

5. Conclusion

This study proposed a new high-precision heart rate measurement method with uncomplicated implementation. In this study, we achieved the following positive results. The first is modeling datasets describing chest surface movement caused by respiratory and heartbeat components. The signal generated from the modeling formulas serves as the training data for the LSTM model. The second is applying artificial intelligence to estimate heart rate from noncontact medical radar signals. The LSTM model is trained from the modeling dataset. The collected radar data will be processed and passed through the LSTM model to estimate the heart rate. The results of this study can be applied to develop a system in practice to assist doctors in diagnosis.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Ethical Approval

This study was approved by the Hanoi University of Science and Technology (Vietnam).

Conflicts of Interest

The authors declare that they have no conflicts of interest.