Abstract
Cardiopulmonary diseases, including cardiovascular disease (CVD) and chronic obstructive pulmonary disorder (COPD), are prevalent in the elderly population. Early identification, long-term health monitoring, and health management of cardiopulmonary disease are crucial for reversing organ damage and preventing further injury. However, most home health monitoring (HHM) devices need to be paired with a specific smartphone application, which leads to the complexity of monitoring multiple health indicators and hinders the feasibility of comprehensive multi-indicator analysis. Therefore, this paper designed a human cardiopulmonary health monitoring system based on an intelligent gateway to reduce the dependence of HHM on smartphones, achieve synchronous monitoring of multiple health indicators, and improve usability to better serve the elderly population. The proposed system can simultaneously monitor electrocardiogram (ECG), pulmonary function, blood pressure, and blood oxygen level (SpO2); process and analyze the data in real-time through the intelligent gateway’s edge computing power; and display the cardiopulmonary health status in real-time. The intelligent gateway embedded a specially designed CNN-LSTM artificial intelligence model on the STM32F429 microcontroller to realize real-time identification of ECG signals at the edge. The accuracy of the pretrained CNN-LSTM model for ECG signal identification is 99.49%, and the model has good performance in terms of complexity and RAM space occupied. According to the evaluation test, the system can achieve the purpose of monitoring human cardiopulmonary health, has a wide range of application scenarios, and has great value in promotion and application.
1. Introduction
With the miniaturization of microcontroller, the application of big data, cloud computing, artificial intelligence (AI) technology, and the upgrading and improvement of the Internet of Things (IoT) in the health field, intelligent home health monitoring has become a reality. Home health monitoring (HHM) enables users to monitor their vital signs, such as heart rate (HR), blood pressure, and blood oxygen level (SpO2), from home, get initial health risk assessment results, and share this information electronically with their healthcare professionals. In recent years, many HHM systems have been proposed [1–5] and are widely used in monitoring and managing various diseases [6, 7], enabling people to monitor their health status at home. HHM systems collect and evaluate multiple health indicators of patients noninvasively and continuously through wearable devices or sensors, which is also the trend of the most promising monitoring method.
Our study presents a novel cardiopulmonary health monitoring system that utilizes an intelligent gateway to accurately gather and analyze various health indicators, such as ECG, pulmonary function, blood oxygen level, and blood pressure. The edge computing capabilities of the intelligent gateway enable real-time analysis of these health metrics, and an embedded pretrained AI model facilitates real-time ECG signal classification. With ECG monitoring requiring long-term continuous data transmission, our system employs an improved CNN-LSTM model in the gateway to preclassify ECG signals and minimize transmission load by downsampling normal ECG signals. This not only reduces the data transmission load but also enhances data the availability and the efficiency of data analysis. Our proposed system enables effective human cardiopulmonary health monitoring without the need for smartphones and achieves real-time analysis of multiple health indicators at the edge.
2. Related Work
2.1. Health Monitoring System
The development of the health monitoring industry started early, and the origin of human health monitoring system service can be traced back to the 1970s. In 1967, doctors at Massachusetts General Hospital affiliated with Harvard Medical School established a remote system that could allow patients and doctors to interact, which became the prototype of the remote monitoring system [8]. Since then, numerous health monitoring systems have been studied and proposed. For example, the RTWPMS [1] monitors temperature, blood pressure, and heart rate in real-time for elderly patients. The AMON system [6] is a wearable medical monitoring device that collects and evaluates vital signs through wrist sensors. Sardini et al. [7] created a wearable belt for wireless health monitoring at home. The PlaIMoS [9] is a wearable device that monitors vital signs related to respiratory health, transmitting data via Bluetooth to a gateway and then wirelessly to the cloud for storage and analysis. Guan et al. [10] proposed a remote health monitoring system for the elderly using smart clothing, a home gateway, and a health care server. These proposed systems typically transmit the collected data to servers in remote data centers for analysis and storage.
With the support of artificial intelligence (AI) [11], health monitoring systems are becoming increasingly intelligent. Health data can be uploaded to remote servers and analyzed by trained AI models, which provide doctors with precise judgments on any abnormalities present in the data. For example, Lin et al. [12] proposed an ECG monitoring system that utilizes AI and IoT technologies, achieving a 98.22% accuracy rate through pretrained decision tree models. Zhang et al. [13] used deep learning to identify toilet users with an accuracy of 90% in their AI-toilet health monitoring system. Queralta et al. [14] proposed a fall detection system with an LSTM recurrent neural network that achieves a precision of over 90%. However, the AI models in these systems are typically deployed on remote servers. Therefore, the analysis of health data and early warning of health abnormality often rely on the stability of data uploads and the computing power of the remote server.
In health monitoring systems, monitoring multiple health indicators often means a large amount of data to be uploaded from wearable or portable devices to remote servers, which require high data transmission bandwidth and result in high energy consumption, particularly for health indicators like ECG that need continuous measurement. Therefore, instead of uploading all data to remote services for processing and analysis, edge intelligence [15–17] has attracted our attention. By using edge intelligence, data can be processed at the gateway, offering a potential solution that can save on communication costs, reduce latency, enhance real-time analysis capabilities, and significantly improve the quality of user service [18, 19]. Edge intelligence is particularly important in health monitoring, where timely and accurate diagnosis is essential.
2.2. Artificial Intelligence (AI) in Arrhythmia Classification
In the application scenario of home health monitoring (HHM), monitoring cardiopulmonary diseases is the most common application. Cardiopulmonary diseases mainly include cardiovascular diseases (CVDs) and chronic obstructive pulmonary disorder (COPD), which are chronic diseases with a long course, complex aetiology, and severe health damage. In the screening of CVDs, the ECG signal has become an essential indicator for monitoring physiological status because it can reflect the heart’s information. The noninvasive detection method identifies abnormal heartbeat types from ECG signals and can help to diagnose whether the human body suffers from CVDs intuitively and effectively. Although CVDs are the leading cause of death in humans, most CVDs are accompanied by arrhythmia, and 90% of cardiovascular diseases can be prevented by long-term monitoring of the ECG signals [12].
Arrhythmia classification algorithms are mainly realized through machine learning and deep learning models. Traditional machine learning algorithms require extensive feature engineering support to design and extract features based on ECG knowledge from the medical field. Osowski et al. [20] used two feature extraction methods of high-order statistics (HOS) and Hermit function, which, respectively, combined with SVM to generate two neural classifiers. Gao et al. [21] used RF combined with 150 features for classification. Myrovali et al. [22] proposed a random forest-based method to identify PAF patients using ECG recordings during sinus rhythm. The approach achieved a 93.45% accuracy, with descriptive statistics of P-wave amplitudes, slopes, and integrals being effective features for discerning PAF patients from healthy individuals. Compared with traditional machine learning, deep learning algorithms are more scalable and can automatically extract data features, including hidden features that cannot be found manually. Nguyen et al. [15] used deep learning to detect AF from 8528 single-lead ECG recordings. The researchers employed windowing and a two-layer prediction model (using CNN and SVM) for accurate classification, achieving an F1 score of 84.19% under fivefold cross-validation. Jun et al. [16] proposed a deep 2D CNN method for ECG arrhythmia classification, using ECG beats converted to grayscale images as input. The classifier achieved high accuracy and sensitivity on the MIT-BIH arrhythmia database. Ullah et al. [17] used Fourier transform to convert 1D ECG to 2D and classified eight-class data in the BIH-MIT using a 2D-CNN model, achieving 99.11% accuracy. Wang et al. [18] propose an end-to-end AF recognition method using a DPRNN from single-lead ECG, which can effectively recognize AF without human expertise and outperforms state-of-the-art baseline methods. The model uses a mix-up operation for data augmentation and can be generalized for other medical sequence signals. Andersen et al. [19] proposed an end-to-end model combining CNN and RNN to classify AF or NSR from RRI fragments. The model achieved high accuracy of 98.98% on three different databases. Yildirim [23] improved the detection of abnormal ECG signals by combining wavelet transform and LSTM. Wavelet decomposition enhanced the performance of the LSTM model, while the proposed wavelet sequence achieved an impressive 99.39% accuracy for ECG classification. Cui and Xia [24] proposed a hybrid CNN and BiLSTM algorithm for ECG abnormality detection, utilizing CNN’s automatic feature extraction and BiLSTM’s efficient time series data processing. The model achieved high accuracy of 98.56% on the arrhythmia dataset in the MIT-BIH database.
3. System Design
The human cardiopulmonary health monitoring system comprises the human cardiopulmonary health monitoring devices, the intelligent gateway, and the cardiopulmonary health management cloud platform. The system structure block diagram is shown in Figure 1.
As shown in Figure 1, four types of human cardiopulmonary health monitoring devices are used in the proposed system: a wearable ECG monitor, a portable spirometer, a portable oximeter, and a wearable blood pressure monitor. These devices are connected to the system via specially opened Bluetooth communication protocols. These wearable/portable devices measure physiological signals and indicators and transmit the collected data to a gateway via Bluetooth. The gateway preprocesses the data and displays the results on the LCD screen. The ECG classification model embedded in the gateway can identify abnormal ECG signals in real-time and provide an early warning to the user. Then, the gateway transmits the data through WiFi to the cardiopulmonary health management cloud platform, which analyzes, manages, and stores the data while providing health warnings to the users.
3.1. Collection of the Cardiopulmonary Health Indicators
The proposed system uses wearable/portable health monitoring devices as the data collection end of the system to collect cardiopulmonary health indicators, including long-term electrocardiogram (ECG), forced vital capacity (FVC), forced expiratory volume in 1 second (FEV1), FEV1/FVC, peak expiratory flow (PEF), blood oxygen level (SpO2), and blood pressure. (1)Long-Term ECG. Long-term ECG refers to continuous long-time (>24 hours) ECG measurement, during which the user’s daily actions are not affected, which increases the likelihood of detecting arrhythmia, myocardial ischemia, and other conditions that are difficult to be detected in rest ECG examination, which plays an important role in the prediction and detection of cardiovascular diseases. In this design, long-term ECG is measured by the LIFEON® ECG patch monitor, a small, lightweight wearable medical-grade ECG recorder with a size of and a weight of 30 g. In our design, the ECG patch automatically establishes a Bluetooth connection with our gateway when it is turned on, starts long-term ECG monitoring, and sends ECG data to the gateway in real-time upon receiving the instruction from the gateway. The gateway calculates the heart rate based on the ECG data, displays the ECG waveform on the screen, and identifies abnormal ECG signals through an embedded AI model to provide early warning to users(2)Pulmonary Function Metrics. Pulmonary function tests are a series of noninvasive breathing tests that are the most direct way to assess how well the lung is functioning. Pulmonary function tests are usually employed for diagnosing and monitoring chronic lung diseases in people undergoing lung or airway symptoms such as cough, shortness of breath, chest tightness, or difficulty breathing. Spirometry is the most common type of pulmonary function test, which measures how much air can be inhaled and exhaled as well as how easily and fast air is exhaled. In this design, pulmonary function metrics, including forced vital capacity (FVC), forced expiratory volume in 1 second (FEV1), and peak expiratory flow (PEF), are measured with a portable spirometer MIR Spirobank®. The portable spirometer automatically establishes a Bluetooth connection with our gateway when it is turned on and sends the gas flow rate when the user exhales into the spirometer after receiving the instruction from the gateway. The gateway calculates the results of FVC, FEV1, PEF, and FEV1/FVC through gas flow rate and sampling rate and displays two curves, including the flow-volume curve and the volume-time curve(3)Blood Oxygen Level. Blood oxygen level reflects the concentration of oxygen in the blood. The higher the oxygen content in the blood, the better the metabolism of the person. Pulse oximetry is the most common test used to measure blood oxygen level, with normal levels ranging from 95% to 100%. Levels below this range may indicate conditions such as hypoventilation, lung disease, cardiovascular disease, or carbon monoxide poisoning. In our system, Etcomm® Pulse Oximeter is used as the blood oxygen measuring device, which is small in size, easy to carry, supports one-key measurement, and provides a real-time display of pulse and blood oxygen level. The oximeter automatically establishes a Bluetooth connection with our gateway when turned on and sends the SpO2 to the gateway in real-time after receiving the instruction from the gateway(4)Ambulatory Blood Pressure. 24-hour ambulatory blood pressure (ABPM) levels are closely associated with the risk of cardiovascular and cerebrovascular diseases, and it also demonstrates the efficacy of antihypertensive drugs within 24 hours [25]. Blood pressure is divided into systolic blood pressure and diastolic blood pressure. The normal value of systolic blood pressure is 90 to 140 mmHg, and diastolic blood pressure is 60 to 90 mmHg. In our system, the Jinyidi® wrist blood pressure watch was adopted to measure ambulatory blood pressure. The wrist blood pressure watch automatically establishes a Bluetooth connection with our gateway when turned on and sends blood pressure to the gateway in real-time after receiving the instruction from the gateway
The cardiopulmonary health indicators monitored by our system are shown in Table 1.
3.2. Hardware Structure of the Intelligent Gateway
A gateway is a network node used in wireless communications that connects two networks with different transmission protocols together. In our design, the gateway collects data from different wearable devices via Bluetooth, analyzes the data in real-time, transmits the data to the cloud platform via WiFi, visualizes the data, and displays the results.
The hardware of the intelligent gateway consists of an STM32F429I-Discovery single-chip microcomputer (MCU), four WH-BLE106 BLE modules, and a USR-WIFI232-A2 WiFi module. The STM32F429I-Discovery MCU is a high-performance MCU for embedded development. STM32F429ZIT6 microcontroller featuring 180 MHz CPU, 2 M bytes of Flash memory, 256 K bytes of RAM, and 64-Mbit SDRAM. In addition to the above high performance, there are four main reasons for choosing the MCU of this system as shown in Figure 2. First, the STM32F429I-Discovery is equipped with eight UART serial ports to support communication with multiple health monitoring devices, which fully meets our system requirements. Second, the STM32F429I-Discovery has a touchable LCD screen, which enables us to provide a simplified graphical user interface (GUI) to facilitate the elderly and to display the monitoring results. Third, the MCU supports embedded AI models, which mean that the gateway can perform large-scale computing locally, achieve edge intelligence of Internet of Thing (IoT) devices, and reduce the cloud load. Finally, the MCU also supports portable real-time operating system (RTOS) which is very important for the development of embedded systems with real-time requirements [26].
Four Bluetooth modules and one WiFi module are connected to the MCU through five UART serial ports, respectively, which the MCU controls for data transmission, as shown in Figure 3. The Bluetooth module WH-BLE106 designed by Wenheng Technology® is a small-size, low-power, Bluetooth 5.2 module with a high-performance, on-chip antenna. It supports up to 1 Mbps data rate and up to a 170-meter wireless transmission range, fully meeting our requirements for data transmission. The WH-BLE106 also supports four working modes: master mode, slave mode, broadcast mode, and master-slave all-in-one mode, and can be configured through AT instructions. It is a simple, reliable, and ideal solution for IoT applications. The WiFi module USR-WIFI232-A2 is a small, cost-effective, high-performance stand-alone WiFi module that supports the IEEE 802.11 b/g/n standards, enabling simple, secure, and reliable wireless connectivity. The WiFi module has four working modes: transparent transmission mode, serial port command mode, GPIO mode, and HTTPD client mode, and supports two configuration modes: AT command and web page setting. Since both the WiFi module and Bluetooth module come from the same manufacturer, the instructions of different modules are similar, providing convenience for our firmware development.
3.3. Firmware Design of the Intelligent Gateway
Firmware is a piece of software code written into the nonvolatile memory, such as ROM, EPROM, or flash memory, of the MCU that controls the device’s hardware. In the design of the intelligent gateway, the STM32CubeMX visualization tool is used, which has the functions of initializing code, porting the AI model, porting RTOS, and building the MDK project. The FreeRTOS [27] real-time operating system is ported into the MCU and used for image output, data receiving, processing, uploading, and other time-sensitive tasks. The programming idea is that the MCU sends control instructions to the wearable/portable health monitoring devices according to the status change of the operation button on the touchable LCD screen of the gateway, as shown in Figure 4. After receiving data from devices, the gateway processes it, displays it on the screen, and packages the data to upload to the cloud platform. The flowchart of the programmed firmware is shown in Figure 5.
Data transmission is the core function of the gateway. The communication technology of the four wearable/portable health monitoring devices used in this system is Bluetooth BLE technology. In the system, the BLE module works in Bluetooth master mode, while the wearable/portable health monitoring devices work in Bluetooth slave mode. Communication between master and slave devices is realized by adding the MAC address of the slave device bound to the master device and modifying the UUID of the master device. To provide a sufficient data rate for wireless transmission, the baud rate of the serial port of the BLE module is set to 115 Kbps. Similar to the BLE module, the WiFi module is the medium connecting the gateway and the cloud platform wirelessly when working in station mode. Moreover, the baud rate of the serial port of the WiFi module is set to 115 Kbps to provide a sufficient data rate for wireless transmission. The BLE and WiFi module configurations are done by putting these modules into command mode and sending ASCII commands through serial connections. The settings of the serial ports are shown in Table 2.
The gateway processes the data collected by the health monitoring equipment, retains only the valid data, packages the data, and adds various identification codes matching with the health management cloud platform. The format of data uploaded to the cloud platform is shown in Table 3.
4. Intelligence of the Gateway
In our cardiopulmonary health monitoring system, intelligent gateway refers to the gateway with an embedded deep learning model and edge computing capability, which can realize real-time data processing and intelligent analysis locally. This technique aims to meet the efficiency requirements of health data analysis and reduce the uploading of invalid data to ease the load on the transmission and the cloud. Among the four types of health indicators of ECG, pulmonary function metrics, blood oxygen level, and blood pressure, long-term ECG requires continuous and high-bit-rate data transmission support, while other indicators are measured at intervals. Therefore, we embedded a specially designed AI model to classify the ECG signal into five arrhythmia classes (normal beat, left bundle branch block beat, right bundle branch block beat, premature ventricular contraction, and other abnormal beats) at the gateway so that our gateway can provide ECG abnormal warning in real-time. At the same time, for ECG data that is classified as normal (normal ECG accounts for more than 90% of the total ECG data) by the embedded AI model, the gateway reduces the data rate from 2 kbit/s to 1 kbit/s through downsampling, thus reducing the load on the WiFi module and cloud platform.
4.1. CNN-LSTM Model
In this work, we aimed to embed the trained model into the MCU to work in real-time in the home environment so we need to consider not only the accuracy of the model but also the size of the model in the MCU and the effect of the model in practical application. In our system, since the ECG is a 1D time series signal in which adjacent ECG waveforms have a certain correlation, we directly use 1D-CNN to process 1D ECG data and combine the 1D-CNN with the LSTM to design a CNN-LSTM model for the ECG arrhythmia classification. The CNN-LSTM network structure is shown in Figure 6.
The designed CNN-LSTM comprises two parts: the first part is the CNN section and the second part is the LSTM layers. The input is fed directly into a convolutional layer, followed by three consecutive pooling layers and convolutional layers. The convolutional layers extract temporal-dependent features from the signal, and the pooling layer reduces the dimension of the output to half the input, reducing the complexity. The first convolutional layer has a kernel size of and output features. The second convolutional layer has a kernel size of and output features. The third convolutional layer has a kernel size of and output features. The fourth convolutional layer has a kernel size of and output features. The three pooling layers perform the max-pooling operation using a kernel size of with strides of two. The ECG signals become a characteristic sequence of after processing by the convolution layers and pooling layers of the CNN part. The output from the CNN section is fed into a bidirectional LSTM layer with units of 10 hidden units. In the LSTM, the information of these features is stretched to 64 pieces and passes through a dropout layer, which can reduce invalid features, ensure the recognition speed of ECG signals, and prevent overfitting. The features processed by the LSTM section pass through 3 dense layers, which can perform nonlinear processing on the previously extracted features and extract the correlation between these features. Finally, the feature information of the output is sent to the Softmax layer to classify the data for 5.
4.2. Materials and Methods
4.2.1. Datasets
We used data from the MIT-BIH arrhythmia database [28] as the data source to train the proposed CNN-LSTM model for ECG arrhythmia classification. The database contains 48 ambulatory ECG recordings collected from 47 patients at a sampling rate of 360 Hz. Each recording contains a beat annotation file that shows the R-wave position and the label of the beat type. In our work, we only selected ECG records from lead II, and we segmented the ECG signal according to the position of the R-wave, taking 100 points before the R point and 199 points after the R point to obtain a relatively complete ECG beat signal. The MIT-BIH arrhythmia database labels ECG beats as more than a dozen types, we selected the top ten categories of data on the quantity of these ECG beats, the number of beats under each type is imbalanced: 71% of ECG beats are normal beat, 8% of ECG beats are left bundle branch block beat, 7% of ECG beats are right bundle branch block beat, 7% of ECG beats are premature ventricular contraction, 3% of ECG beats are paced beat, 2% of ECG beats are atrial premature beat, 2% of ECG beats are fusion of ventricular and normal beat, and almost 0% of ECG beats are ventricular flutter wave, fusion of paced and normal beat, and nodal (junctional) escape beat.
Since an imbalanced dataset will cause the AI model biased towards the majority class and underperform in predicting minority classes, we reduced the number of normal beats and grouped the original 10 classes of heartbeats into five bigger classes, namely, normal beat (NOR), left bundle branch block beat (LBB), right bundle branch block beat (RBB), premature ventricular contraction (PVC), and others (O). The class of others contains paced beat, atrial premature beat, fusion of ventricular and normal beat, ventricular flutter wave, fusion of paced and normal beat, and junctional escape beat. The number of beats under each class is shown in Table 4. In addition, we downsampled the ECG waveform from 300 points to 100 points to match the sampling rate of the LIFEON® ECG Patch Monitor that we use in the cardiopulmonary health monitoring system.
4.2.2. Experimental Setup
The proposed CNN-LSTM model was deployed in Python with TensorFlow [29]. We divided the dataset into two parts: the training set and the test set in a 4 to 1 ratio; 20% of the training data is randomly selected as the validation set. The training set is used to train the model on an NVIDIA GPU, and then the performance of the model in the gateway is evaluated by the test set after the pretrained model is ported to the MCU of the gateway.
We chose support vector machine (SVM) and random forest in traditional machine learning, convolutional neural networks (CNN) in deep learning, and our improved CNN-LSTM for this system for comparative experiments. In the random forest algorithm, the number of decision trees is 1000. The SVM selects the radial basis function and the gamma parameter set to 1. In both CNN and CNN-LSTM, the number of training is set to 100, and the ReLU activation function is selected because it is convenient for hardware implementation.
4.2.3. Evaluation Parameters
The evaluation parameters used in this experiment include accuracy, precision, recall, and F1-score.
Accuracy is a crucial metric for neural network models, which measures the degree of classification accuracy on a testing dataset. During the training process, we hope that the accuracy of the model can continuously improve, indicating that the model can better recognize and classify data. The importance of accuracy lies in its direct impact on the reliability and practicality of the model. If the accuracy of the model is low, it cannot effectively classify and predict input data, thus losing its application value. On the other hand, if the accuracy of the model is high, it can be used to solve many real-world problems, such as image recognition, speech recognition, and natural language processing. Accuracy is defined as follows:
means true positive, means true negative, means false negative, and means false positive.
Precision is crucial for neural network models as it measures the accuracy of the model’s predictions. A high precision score means that the model has a low false positive rate, indicating that the predicted results are reliable and trustworthy. In many real-world applications, such as medical diagnosis or financial forecasting, even small errors can have significant consequences. Therefore, achieving high precision can be critical for ensuring the effectiveness and safety of a neural network model in these contexts. Precision is defined as follows:
Recall is an important metric for evaluating the performance of a neural network model. It measures the percentage of relevant instances that the model can correctly identify from the total number of relevant instances in the dataset. In other words, recall measures how well the model is able to detect all actual positives. Recall is particularly important in applications where missing a positive instance can have serious consequences, such as detecting fraud or diagnosing a disease. In these scenarios, a high recall value ensures that the model is able to detect the majority of positive instances, reducing the chances of false negatives and increasing the accuracy of the model’s predictions. Recall is defined as follows:
The F1-score is a critical metric for evaluating the effectiveness of a neural network model. It combines both precision and recall into a single score, providing a balanced assessment of the model’s performance. A high F1-score suggests that the model achieves both high precision and high recall, indicating that it can effectively identify all relevant instances while minimizing false positives and false negatives. This is particularly important in applications where there is a trade-off between precision and recall, such as medical diagnosis or spam detection. By optimizing for the highest F1-score, neural network models can strike a balance between minimizing false positives and false negatives, resulting in more accurate and reliable predictions. Therefore, F1-score is an important metric to consider when evaluating the overall effectiveness of a neural network model. The F1-score is defined as follows:
4.3. Experimental Results
The comparison of the results of the four models is shown in Table 5. Although machine learning algorithms such as random forest and SVM have a good performance in ECG classification, they need to extract the features of ECG signals before classification, which requires the computing power and computing time of the gateway, resulting in increased gateway load and decreased real-time performance. Deep learning algorithms, by contrast, are better suited to applications on the gateway. The accuracy curve and the loss curve of the CNN and the CNN-LSTM are shown in Figures 7 and 8. It can be seen that the stability of the CNN-LSTM is much higher than that of the CNN in the process of training and validation.
(a)
(b)
(a)
(b)
Since the pretrained classification model is designed for our gateway, the complexity of the model running and the space it occupies in the MCU also need to be considered. Through the evaluation of the CNN model and the CNN-LSTM model in STM32CubeMX, the complexity and space of these two models running in the MCU are shown in Figure 9. According to the results, the complexity of the CNN-LSTM model is 514326 MACC, which is much smaller than the complexity of CNN model 1245116 MACC. Moreover, the flash occupation of the CNN-LSTM model is smaller than that of the CNN model. Therefore, we apply the proposed CNN-LSTM model in the gateway for ECG signal classification considering the high stability and low complexity of the CNN-LSTM.
(a)
(b)
4.4. Implementation in the Gateway
In order to run the pretrained CNN-LSTM model in the MCU, we use X-CUBE-AI to generate optimized AI code and integrate it into our application. X-CUBE-AI is a STM32Cube expansion package that automatically converts pretrained AI algorithms into C code that can be run in the MCU. After integrating the pretrained CNN-LSTM model into the MCU, the gateway realizes the intellectualization. With the embedded CNN-LSTM model, our intelligent gateway is capable of processing and classifying the ECG data in real-time. Figure 10 shows the processing flow of ECG data after the gateway receives the ECG signal.
As shown in Figure 11, the gateway preprocesses incoming raw ECG signals include wavelet denoising, R peak detection, ECG beat signal slice, and heart rate calculation. The gateway screen displays the waveform, the heart rate, and a number of abnormal ECGs detected by the AI model. Every 5 seconds, the gateway sends ECG data, including downsampled normal ECG data and all abnormal ECG data to the cloud platform. Since incoming ECG signals are often accompanied by different types of noise, such as power line interference, muscle noise, and motion artefacts, it is necessary to denoise the ECG signals before formal analysis. The selection of a suitable wavelet is crucial for achieving optimal denoising performance in signal processing using the discrete wavelet transform (DWT). Commonly used wavelets include the Haar wavelet, Daubechies wavelet, and Symlet wavelet. In our previous work [30], we studied and compared the performance of different noise reduction methods and found that the db8 wavelet is very suitable for noise reduction of motion artefact in ECG signals. So in our intelligent gateway, we use db8 wavelet to decompose the noisy ECG signal into six levels, set the 1st detail coefficients and the 6th approximation coefficient to zero, and reconstruct the ECG signal with the rest coefficients to obtain the denoised ECG signal. The performance of the implemented wavelet denoising on noisy ECG data is demonstrated in Figure 11.
After denoising, the intelligent gateway uses the classical Pan-Tompkins [31] algorithm to detect the position of the R peak in the ECG waveform. The Pan-Tompkins algorithm detects QRS complexes based on digital analyses of slope, amplitude, and width. The performance of the Pan-Tompkins algorithm implemented in the intelligent gateway is demonstrated in Figure 12. According to the position of the R peak, we then slice the ECG signal by cutting out 33 points before the R peak and 67 points after the R peak (including the R peak point). Finally, the sliced ECG beat signal is ingested into the embedded CNN-LSTM model to classify the ECG beat signal into five classes.
To evaluate the performance of the pretrained CNN-LSTM model after embedding it into the MCU, we directly feed the preserved test set into the gateway through the UART port. The results of the ECG classification model are output by the serial port assistant, as shown in Figure 13. The out_data array stores the probability that the ECG signal is classified into each category in order. The detailed performance of the embedded ECG classification model is presented using confusion matrices for all five classes, as shown in Figure 14.
5. System Test
To test the intelligent gateway based cardiopulmonary health monitoring system, we performed a 24-hour home health monitoring trial, which included 24-hour long-term ECG monitoring, interval pulmonary function tests, interval blood oxygen measurements, and 24-hour ambulatory blood pressure monitoring.
The cardiopulmonary health monitoring system is designed to facilitate the use of the elderly population, so the intelligent gateway is equipped with a touchable LCD screen that provides a simplified graphical user interface to display monitoring results and give messages on abnormal results. To start the ECG monitoring, the user clicks the “ECG” button on the main control interface, and the intelligent gateway starts to receive ECG data from the ECG patch monitor and conduct real-time analysis and upload of the ECG data. The screen displays the ECG waveform, heart rate, and the number of abnormal ECGs in real-time, as demonstrated in Figure 15.
Since our system supports simultaneous measurement of multiple health indicators, users can schedule pulmonary function tests as needed. To start the pulmonary function test, the user only needs to go to the main control interface, click the “Pulmonary Function” button, and then start the test using the portable spirometer. The intelligent gateway receives data, processes and displays the results on the screen, and uploads the pulmonary function data to the cloud platform. The pulmonary function test results, including the flow-volume curve, volume-time curve, FVC, FEV1, FEV1/FVC, and PEF indicators, are displayed on the screen, as shown in Figure 16.
To perform a blood oxygen level measurement, the user only needs to click the “Blood Oxygen” button on the main control interface and measure the blood oxygen level using the portable pulse oximeter. The intelligent gateway acquires data, presents the results, and keeps track of the number of abnormal blood oxygen levels detected based on the established range of 95-100% for normal blood oxygen saturation. Additionally, it uploads the gathered data to the cloud platform. The display of the blood oxygen level is shown in Figure 17; the interface shows the results of each measurement.
In our system, the wrist blood pressure monitor supports ambulatory blood pressure measurement, which automatically measures the blood pressure every half hour. The user only needs to click the “Blood Pressure” button on the main control interface, and the gateway starts receiving data from the blood pressure watch. The intelligent gateway exhibits the measurements obtained and tracks the count of abnormal blood pressure levels based on the standard range of systolic blood pressure between 90 and 140 mmHg and diastolic blood pressure between 60 and 90 mmHg. It also uploads the collected data onto the cloud platform. The blood pressure display interface is shown in Figure 18.
6. Summary
In this paper, an intelligent gateway based human cardiopulmonary health monitoring system is designed. This system realizes synchronous monitoring of multiple health indicators, such as long-term ECG, pulmonary function metrics, blood oxygen level, and ambulatory blood pressure. The designed intelligent gateway is based on the MCU STM32F429I, with four built-in Bluetooth modules for data collection, one WiFi module for data transmission, and one touchable screen for user operation and display of monitoring results. The intelligent gateway embedded a specially trained CNN-LSTM model to realize real-time ECG signal classification at the edge. The accuracy of the pretrained CNN-LSTM model for ECG signal classification is 99.49%, and the model has good performance in terms of complexity and RAM space occupied. According to the evaluation test, the system has achieved the expected design goal.
The proposed human cardiopulmonary health monitoring system has four advantages:
(1) Data Accuracy Assurance. Our system establishes a partnership with mature technology enterprises and adopts widely recognized medical-grade health monitoring equipment as the data acquisition equipment of this design
(2) Improved Usability. Our system is independent of smartphones and complex applications, making it easy for the elderly population to use
(3) Improved Timeliness of Data Analysis. The intelligent gateway realizes real-time analysis through edge computing and an embedded AI model, ensuring timely warning of health abnormalities
(4) Support Comprehensive Data Analysis. Our system implements multi-indicator data collection and upload
Users can specify monitoring plans based on their health conditions, and the cloud platform can use multi-indicator data for comprehensive health management of users. In summary, our proposed intelligent gateway based cardiopulmonary health monitoring system has a wide range of application scenarios and great value in promotion and application. In our future work, we will use our system to conduct comprehensive monitoring and analysis of health indicators and study the prediction and management of cardiovascular and chronic respiratory diseases using comprehensive health indicators.
Data Availability
The data presented in this study is openly available in https://physionet.org/content/mitdb/1.0.0/, accessed on 24 Feb. 2005.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was also supported by the fund of the Beijing Municipal Education Commission, China, under grant number 22019821001. This work was supported by the Climbing Program Foundation from Beijing Institute of Petrochemical Technology (Project Nos. BIPTAAI-2021-001 and BIPTAAI-2021-002). This work was supported by the fund of Cross-Disciplinary Science Foundation from Beijing Institute of Petrochemical Technology (Project No. BIPTCSF-023).