Abstract
An electrocardiogram (ECG) uses electrodes to monitor the heart rhythm and identify minute electrical changes that occur with each beat. It is employed to investigate particular varieties of aberrant heart activity, such as arrhythmias and conduction problems. One of the most essential tools for detecting heart problems is the electrocardiogram (ECG). The majority of ECG records are still on paper. Manual ECG paper record analysis can be difficult and time-consuming. It is possible to digitally digitize these paper ECG recordings for automated analysis and diagnosis. In this paper, we proposed a system to digitize the ECG paper, automatically detecting R peaks, calculating the average heart rate, and sending SMS to the doctor via cloud in the event of detection of abnormality. The method of the system is uploading an ECG image, then dimensionality reduction, feature extraction in the form of digital signals, and saving it in a CSV file format using the MATLAB programming language. After that, the system retrieves the signals for further processing of the raw signals. We used the fast Fourier transform (FFT) algorithm to calculate R peaks and calculate the heart rate. If the heart rate is abnormal, the system sends SMS messages to doctors via a technology platform (Twilio) using the Python programming language.
1. Introduction
Heart disease is the major reason of the high number of deaths in the world [1]. According to the WHO, heart disease remains the major reason of death globally, accounting for 16% of all deaths from all causes. It kills more people today than ever before, with the number of deaths from heart disease rising by more than two million since 2000, reaching nearly nine million in 2019 [2]. One of the standards for detecting cardiac problems is electrocardiogram (ECG) analysis. ECG is a capture of electrical action in the heart of the muscle of the heart over the course of one cardiac cycle [3, 4]. The rhythmic depolarization and repolarization of the myocardium associated with the contractions of the atria and ventricles during each cardiac cycle are represented by a recurring sequence of P, QRS, T, and a conditional U wave [5] (Figure 1).
ECG is a semiperiodic, rhythmically, and concurrent signal with a cardiac task that is obtained using a passive sensory instrument that functions as a generator of bioelectric signals, simulating the heart’s function [6]. The ECG signals are intrinsically low and noisy signals made up of numerous changeable components due to a variety of environmental conditions such as variations in body temperature, body movement, and line frequency (50/60 Hz), among others. Because the ECG signal cannot be directly conditioned, amplified, or replicated, digital filtering techniques with configurable windows are utilized [7].
Typically, clinicians examine the ECG signal visually, looking at its form, rhythm, and voltage. Digital signal processing, in combination with classical or advanced machine learning, is currently playing a major part in medical diagnosis, particularly in ECG diagnosis. However, most commercial ECG devices do not allow raw data, so it is often limited. One way to approach this challenge is to use a classification system based on ECG image processing. The ECG monitoring system’s main source of power usage is wireless transmission of ECG data. As a result, a solution that decreases the size while maintaining signal quality integrity is required. An efficient compression approach can lower the size of the transmitted ECG data. On the other hand, the majority of high-performance compression algorithms, on the other hand, are sophisticated and energy-intensive, making them unsuitable for wearable ECG monitoring systems [8, 9].
The contributions of this work are as follows:(a)Proposed a system that automatically extracts the features from ECG paper (image) and saves these features in CSV.(b)Improved R peaks detection and calculation of average of heart rate.(c)Classification normal or abnormal of ECG paper.(d)Collection dataset of ECG signals.
The remaining sections of the paper are structured as follows: Section 2 is related work on ECG digitization and R peaks detection. Section 3 introduces the proposed system. Section 4 deals with results and discussion, and Section 5 includes conclusion.
2. Related Work
In order to denoise a noisy ECG signal, Kumar et al. [5] employed an FFT-based bandpass filter. They used multistage adaptive peak detection to identify the R-top in the ECG signal’s QRS compound. The proposed technique was evaluated using the MIT/BIH Arrhythmia Database. Their method yields 99.98 percent recall and 99.96 percent positive productivity (PP).
Almalchy et al. [7] investigated a number of models of finite impulse response (FIR) low-pass and high-pass filters, as well as their characteristics in terms of reply time, earning, consistent deformation, and repudiation, to define best bandpass nomination sample to produce an ECG signal that exceedingly like a patient’s actual heart function. A hybrid nomination technique is also presented and tested. Signal quality is calculated using the mean square error (MSE). In the experimental section of the investigation, the signals were simulated using the MATLAB platform. Sensors, wireless connection modules and linkages, surveillance and processing units, medical shields, wearable materials, and enhanced decision-making and data extraction algorithms are all part of the bigger project.
Hadiyoso et al. [10] used a deep learning methodology to offer an image-based ECG classification algorithm. The ECG signals examined in this work were based on normal sinus rhythm (NSR) and premature ventricular contraction (PVC). They used a CNN with VGG16 to feature extraction and classification. The simulation findings demonstrate that ECG abnormalities can be detected with up to 95% accuracy. Vijayakumar et al. [11] used ECG signal points and additional image processing characteristics to recreate human perception of visible cues. The method created a model with two new characteristics, utilizing characteristic extraction and the ECG signals that were related to success in detecting accuracy, based on human eye methodologies. The results of this study were a 99 percent average sensitivity, a 100 percent specificity, and a 100 percent overall accuracy.
Reklewski et al. [12] proposed and validated an extremum sampling-based ECG R-peak that adapts in real time to the events identification method in this study. The suggestion technique has three primary steps: event-driven QRS window identification, detection of ECG signal extremum, and waiting for the current QRS complex to finish. The algorithm’s performance was tested on recordings from the MIT-BIH Arrhythmia Dataset. The results were as follows: recall and rates of positive prediction of 99.55% and 99.88%, respectively, and a total false detection rate of 0.58%. In this paper, Park et al. [13] proposed a wavelet transform Shannon energy envelope (WTSEE) algorithm to develop an R-peak revelation approach for rapid ECG testing. The suggested WTSEE algorithm has the following steps: executing a WT to minimize the volume and noise of ECG data, using Shannon energy envelope (SEE) to calculate the peak energy envelope (PEE), and estimating R peaks using PEE. The results of the algorithm are as follows: sensitivity, positive predictability, detection error rate, and accuracy are 99.93%, 99.91%, 0.16%, and 99.84%, respectively.
Abdullah Al et al. [14] proposed the dynamic mode selected energy-adaptive window sizing (DMSE-AWS) technique for the detection of R tops. The DMSE method is used to remove the QRS ingredients and all nonthematic, ingredients of the ECG signal, and the AWS approach identifies the zone of concern (ZOC) of the QRS compound. The suggested method has a detection error rate of 0.06 percent and an accuracy of 99.94 percent, recall of 99.98 percent, and positive foreseeability of 99.96 percent. Uzair Zahid et al. [15] suggested a novel strategy to close this gap by using shoaly and consolidated 1-D Self-Organized Operational Neural Networks to provide a real-time solution (Self-ONNs). They used 1-D Self-ONN layers, which allowed us to capture the shape of the ECG waveform surrounding the R tops and inject temporary data depending on the RR period. The MIT-BIH Arrhythmia Dataset was used. The procedure produced the best results: for normal (N) beats (precision, recall, and F1 score), the results were 99.21%, 99.10%, and 99.15%, respectively, while for supraventricular ectopic beats (precision, recall, and F1 score), the results were 94.41%, 96.10%, and 95.2%.
Madan et al. [16] offered a model based on a hybrid deep learning called 2-D-CNN–LSTM for the disclosure and rating process. The methodology of this paper includes two stages: automated noise reduction and characteristic extraction. The collected findings reveal that the suggested method correctly diagnoses cardiac arrhythmias (ARR), congestive heart failure (CHF), and normal sinus rhythm (NSR) with 98.7%, 99.9%, and 99.9% accuracy, respectively. Furthermore, the suggested model has an average sensitivity of 98.33 percent and a specificity of 98.35 percent for all three arrhythmias.
Robinson and Kabari [17] created a model that can disclose and analyze the properties of an ECG signal, allowing them to identify heart nonlinearity. Before using the signals for rating and calculating the rate of heart using peak values and intervals, they employed the FFT to nominee the ECG readability dataset and eliminate undesirable signals. About 218,000 ECG readings were included in the dataset, along with the patients’ gender and age grade. This method was based on the Object-Oriented Analysis and Design Methodology (OOADM). MATLAB software was used to build the system. The model’s overall efficiency is 95%, which exceeds other available models. Kouekeu et al. [18] presented a novel algorithm for QRS compound discover depending on the power and direction of the ECG signal. The algorithm depends on the condenser threshold that catches the signal power of two other sills, which depend on the signal direction to detect the placement of the R top. The terminus of the r peak and the ECG detour are utilized to detect the start of the QRS compound. F1 score measure for MIT-BIH arrhythmia, MIT-BIH sinus, and MIT-BIH noise stress are 99.75%, 99.84%, 97.39%, and 86.63%, respectively.
3. Proposed System
The proposed approach aims to produce a precise diagnosis of heart-related problems by transforming paper ECG recordings into a 1-D or 2-D signal and employing FFT techniques. The recommended approach makes use of ECG sensor-captured images or collected ECG paper records from hospitals or the Internet. Using effective processing methods, the shadow in the photographs is removed. The FFT approach is used to establish the threshold value that separates the ECG signal from its background after applying a number of image processing techniques. The threshold ECG image is then converted into a digital ECG. As we can see in Figure 2, the suggested method consists of the following five stages: (1) ECG paper, (2) ECG digitization, (3) signal analysis (FFT), (4) average of heart beats, and (5) cloud communication.
3.1. ECG Gathering
An electrocardiogram (ECG) represents the electrical activity of the heart and can be obtained by attaching electrodes to the body’s surface. The heart’s electrical activity is captured in a recording that shows the cardiac cycle. Right now, there are two ways that we can get an ECG image: directly (in real time) and indirectly. In real-time mode, the ECG picture is taken directly from the ECG devices connected to the patient’s body and scanned, as opposed to indirect mode, where the ECG image is collected from the hospitals as an ECG paper. About 1000 ECG papers (pictures) were gathered from hospitals and the Internet.
3.2. ECG Digitization
One of the most vital methods for determining heart abnormalities is the electrocardiogram (ECG). ECG records are still primarily only available on paper. The paper ECG records may be challenging and time-consuming to manually analyze. We can therefore digitize these paper ECG recordings and automate diagnosis and analysis. Thus, at this stage, the ECG paper is transformed into a 1-D and a 2-D signal. This stage consists of four steps: crop original ECG image, masking ECG image, feature extraction, and save ECG digits.
Algorithm steps of digitization are as follows:(i)Step 1: START.(ii)Step 2: read ECG paper.(iii)Step 3: crop ECG image (resize (height and width).(iv)Step 4: create mask (remove the grid from paper.)(v)Step 5: R peaks detection and feature extractions.(vi)Step 6: save 1-D-ECG to CSV file.
3.3. Read ECG Image
The system reads the ECG image in this step and shows it to the user. To assess the patient’s heart rate, the ECG paper has 12 or 3 leads. Each lead captures the pulse from the area of the body where it is attached, and the device displays that pulse as graph signals, as shown in the ECG paper sample in Figure 3.
3.3.1. Crop ECG Image
Cropping an image involves reducing its size or altering its aspect ratio. It can be dropped by removing or modifying its outside border. We used this stage to crop the ECG image because, as we can see in the ECG paper, which contains the several lines of signal, we needed to do so before our system could digitize it. Any location in the graph can be chosen and cropped. The portion where we cropped it will show up in the little window as seen in Figure 4:
3.3.2. Masking ECG Image
A mask image is any image in which some pixel intensity values are zero while others are not. Wherever the pixel intensity value in the mask image is zero, the pixel intensity of the resulting masked picture will be set to the background value (normally zero). The backdrop can be removed from images of objects with fuzzier edges using an image masking technique. As seen in Figure 5, this process will erase the grid from the cropped image.
3.3.3. R Peaks and Feature Extraction of ECG
It is possible to identify the R peaks and QRS complexes in an ECG signal, which provide details about the heart rate and conduction velocity. It also includes information about the tissue of the heart and a variety of abnormalities. It offers the evidence required to make a heart disease diagnosis. Due to this, it has drawn a lot of attention in the field of ECG signal processing. However, the noise and time-varying morphology make detection difficult. Figure 6 displays the peak/valley detection from the initial ECG data. The peaks and valleys (especially the Q, R, and S points) become more distinct after this phase. After extracting the QRS complex detection feature, we can further investigate the feature using additional methods. We can do heart rate variability (HRV) analysis on the R-R interval signal to show the state of the heart and neurological system. However, there is still an issue, as shown in Figure 6, where the T wave is also clearly visible. We will see how to address this issue throughout the preprocessing phase.
3.3.4. Convert ECG Image to 1-D ECG Signals and 2-D ECG Signals
This is the last stage in the ECG digitization part. It is quite simple to extract the digital signals from the ECG image and store them in text files (.CSV, .mat, and .dat) for both 1-D and 2-D signals after cropping, masking, removing the grid from the ECG image, and QRS detection. The amount of the crop will determine the amount of data that can be extracted through digitalization. Figure 7 displays the data gathered at this stage.
3.4. Signals Analysis (FFT)
For signal analysis, this stage used a fast Fourier transform (FFT). FFT is a method for generating a sequence’s discrete Fourier transform (DFT). The discrete Fourier transform (DFT) is a technique for translating specific types of function sequences into various forms of representation. This stage consists of five steps, which we shall go over in detail in this section.
3.4.1. Compute Frequency Domain
Signals or mathematical functions are evaluated in the frequency domain rather than the time domain. Time-domain analysis is a method of studying physical signals, mathematical functions, or time series of economic or environmental data in terms of time as in Figure 8.
3.4.2. Filtering Signal
A filter is a signal processing technique that eliminates unwanted ingredients or characteristics. Filtering is a signal processing technique in which one or more sides of the signal are muted fully or partially. Compute filtering coefficient to eliminate 50 Hz bum noise. There are many methods to remove noise from ECG signals; in this paper, we used Kaiser Window: it’s a one-parameter family of window functions used in spectrum analysis and finite impulse response filter design, as shown in Figure 9.
3.4.3. Peak Detection
In this step, the system automatically determines the R peaks and marks it as we can see in Figure 10.
3.5. Heart Rate Average
The heart rate, also referred to as the pulse rate, is the frequency with which a person’s heart beats per minute. The average adult heartbeat is between 60 and 100 beats per minute, yet everyone’s heartbeats at a different rate. In order to obtain the average of the heart beat equation (1) and the number of peaks, we must first calculate the R-R interval (the space between beats), convert sample distances to ms distances (1 minute = 60000 ms), and then divide 60000 by the mean of the R-R interval calculation.
3.6. Twilio Cloud
Twilio is a cloud communication platform that includes APIs for audio, video, messaging, authentication, and other features. By Twilio, the system automatically contacts the doctor when it detects the danger case of the patient’s heart disease. We can add this part if we make a wearable device to monitor the patient. The wearable device consists of the AD8232 ECG sensor module with an Arduino board (Arduino Uno) and a Jetson Nano mini-computer to receive data from the Arduino via the serial port. Also, if we apply this system in hospitals to monitoring patients who are suffering from heart diseases, it offers a great service in hospitals and reduces the stress on the doctors and nurses. If the average heart rate is abnormal, the system can send an SMS message to the doctor using the Twilio cloud, as shown in Figure 11.
4. Result and Discussion
The algorithm performance has been examined on a set of raw ECG paper (image). The performance measures utilized to calculate the performance of the proposed R-peak detection method are recall or sensitivity (SE).
where TP stands for True Positive (real R peaks correctly identified as peaks) and FN stands for False Negative (actual peaks not detected as peaks).
We tested our system on an ECG paper (raw image) as we mentioned in Table 1, and the results are presented in Table 2.
5. Conclusion
The majority of cardiovascular problems and diseases can be avoided, yet death rates rise as a result of insufficient treatment and misdiagnosis. Consequently, early detection of arrhythmia can save a patient’s life. Tachycardia is a disturbance of the heart rate (pulse) or heart rhythm. When it beats too quickly, it is referred to as tachycardia. When it beats too slowly, it is referred to as bradycardia. The detection is carried out by analyzing and extracting some features from the electrocardiogram (ECG) signals.
This paper proposes a system that converts ECG paper from image to digits for easy treatment. This system uses fast Fourier transform (FFT) to diagnose and detect R peaks and calculate heart rate by calculating average intervals between R peaks. The system can also send notifications to caregivers or doctors in the case of abnormal situations of heart disease. This system can be used as a wearable device to monitor the patient or in the operating room of a hospital where it can take ECG images from ECG devices and convert them to digits and detect R peaks and calculate the average heart rate easily without the intervention of doctors. It can also send SMS messages to doctors via the Twilio cloud. This point reduces the stress on the doctors with an increase in patients suffering from diseases and also a small number of doctors.
Data Availability
The data are ECG images, we added samples ECG imges in “Table 1: Test results for some normal & abnormal ECG paper”.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Supplementary Materials
The supplementary material includes MATLAB code to convert ECG images to digits and save these digits in a CSV file, which is then passed to Python code for further processing, such as R peaks detection, calculating the average of heart beats, detecting abnormalities, and sending an alert to the doctor via Twilio cloud in a danger case. (Supplementary Materials)