International Journal of Telemedicine and Applications

International Journal of Telemedicine and Applications / 2011 / Article

Research Article | Open Access

Volume 2011 |Article ID 631848 | 6 pages | https://doi.org/10.1155/2011/631848

Scientific Comparison of Different Online Heart Rate Monitoring Systems

Academic Editor: Y. L. Hsu
Received22 Feb 2011
Accepted12 May 2011
Published07 Jul 2011

Abstract

Recent technical development focused on real-time heart rate monitoring instead of postexercise evaluation of recorded data. There are several systems on the market that allow direct and real-time monitoring of several individuals at the same time. The present study compared the systems of Polar, Acentas, Activio, and Suunto in a field test with twelve subjects regarding failure quota, operating distance, and ECG validity. Moreover, the installation and use of software and hardware were evaluated with a quality rating system. Chest belts were evaluated with a questionnaire, too. Overall the system of Acentas reached the best mark of all systems, but detailed results showed that every system has its advantages and disadvantages depending on using purpose, location, and weather. So this evaluation cannot recommend a single system but rather shows strength and weakness of all systems and additionally can be used for further system improvements.

1. Introduction

Over the last 25 years, heart rate monitors have been widely used in sports and sports science. Since the development of handy heart rate monitors (HRMs) with the size of a watch and appropriate flexible chest belts with electrodes to transmit signals wirelessly to the watch, measurement of heart rate (HR) has become an easily accessible and valuable tool for training and coaching purposes and conducting laboratory or field studies [1]. Validity and reliability have been shown to be high for these tools compared to ECG measurements for both HR and heart rate variability (HRV) [16]. Owed to its widespread use in physical activity not only for individuals but also for team sports, recent technical development focused on real-time monitoring instead of post-exercise evaluation of recorded data. This led to both software and hardware development of wireless online HR monitoring systems. Currently, several different systems are available on the market, of which four were tested in the present study: POLAR Team2 Pro, Acentas team monitoring system, Suunto Pro Team Pack and Activio Sport System. All systems allow direct and real-time monitoring of several individuals at the same time. Data are transmitted wirelessly to a receiver which is connected with a standard laptop with relevant software. However, a direct comparison of these systems and validity of recorded data are missing in literature. Therefore the main purpose of this study was to compare all systems in a field test against each other. Operating distance and validity against a commercially available ECG were evaluated as well as the installation and use of software and hardware of the above-mentioned systems.

2. Methods

2.1. Subjects

A total of 12 active people (11 men and 1 woman; age:  yr; height:  cm; mass:  kg) participated in the study. Ten men of an amateur soccer team conducted an online field test during a real football match. One male subject participated for validity of HR monitoring systems against ECG whereas the female subject was only involved in operating distance measurement. Because of the low case number of the last two tests, data can only be considered as a pilot study for ECG validation and distance measurement.

2.2. Heart Rate Monitoring Systems

The following four commercially available systems were used for all tests: POLAR Team2 Pro (Polar Electro Oy, Kempele, Finland), Acentas team monitoring system (Acentas GmbH, Hörgertshausen, Germany), Suunto Pro Team Pack (Suunto Oy, Vantaa, Finland), and Activio Sport System (Activio AB, Stockholm, Sweden).

2.3. Quality Rating System

A quality rating system similar to rating systems used in trade journals was used to test electronic equipment and software. A valuable design was found in the journal “Connect” (http://www.connect.de/) which is specialised on the testing of mobile and electronic communication tools. For this purpose, the authors decide to subdivide the rating system into three main categories: measurement, software, and hardware. At this, the core area of the study focused on the adequate online measurement of the heart rate. Therefore, the rating system was set in favour of the measurement which was represented by 200 scores of maximal 500. The latter 300 were equally divided on quality of the hardware and the usability of the software. Detailed distribution into the several subcategories is depicted in Table 1. Subcategories were weighted subjectively by the authors in consideration of heart rate monitoring of team sports including the monitoring of 10 or more subjects in parallel. In addition, the following subcategories were rated once by an independent observer who was not familiar with any of the monitor systems to get a first impression: instruction manual, packing, chest belts (quality), and initial software handling. The remaining categories were judged once by an experienced person. With exception of wearing comfort and failing quota in field, all subcategories were judged and quantified by using a ten-stepped visual analogue scale which is known for pain assessment [7]. Visual analogue scaling was weighted by multiplying itself with factor : ( = maximal score of subcategory/10). Parameters leading the judgement of the subcategories are described in Tables 2 and 4.


Main categorySubcategoryItemScores

HardwareInstruction manualunderstandability10
PackingQuality10
Handling15
Chest beltStorage10
Battery10
Quality20
Wearing comfort25
ReceiverPower consumption5
Quality25
Handling15
Connectivity5

Subtotal150

Installation10
SoftwareMenue navigation40
Features100

Subtotal150

ECG deviation30
MeasurementsDistance70
Failure quota in field test100

Subtotal200

Maximum overall score500


SystemAcentasActivioPolarSuunto

Instruction manual
(language, comprehensibility, volume, usefulness)
(i) Short and compact but sufficient (+)(i) Very detailed IM in English (+)
(ii) Easily comprehensible (+)
(i) Short instruction in six languages (+)
(ii) Laminated (+)
(iii) Personal instruction by member of staff (+)
(iv) Useful (+)
(v) Only short hardware & installation instruction (−)
(i) No instruction manual (−)

Packing
(quality, handling)
(i) Compact backpack (<1 kg) (+)
(ii) Good accessibility & separation of hardware (+)
(iii) Poor quality of packing system (−)
(i) Good arrangement of hardware (+)
(ii) Heavy (+2 kg) (−)
(iii) Hardware & software not fixed in packing (−)
(iv) Poor quality of packing system (−)
(i) Carrying case (+)
(ii) Good quality of packing system (+)
(iii) Good arrangement of single system parts (+)
(iv) Heavy (~3 kg) (−)
(i) Carrying case (+)
(ii) Lightweight (~1kg) (+)
(iii) Poor arrangement of hardware (−)

Chest belt
(storage, battery, quality)
(i) Simple snap closing (+)
(ii) Water resistant (+)
(iii) Very good battery (+)
(iv) No internal storage (−)
(i) Simple snap closing (+)
(ii) Water resistant (+)
(iii) Very good battery (+)
(iv) No internal storage (no rating) (−)
(i) Bluetooth USB stick for storage unit (+)
(ii) High storage capacity (+)
(iii) Small, bulky snap closing (−)
(iv) Poor battery (−)
(i) USB reader for storage unit (+)
(ii) Two types of chest belt (±)
(iii) Small snap closing (−)
(iv) No driver for USB (& not found on homepage) (−)
(v) Low storage capacity (−)

Receiver
(power consumption, quality, handling, connectivity)
(i) Lightweight (~150 g) (+)
(ii) Connection only via 150 cm USB cable (±)
(iii) Water repellent (+)
(iv) Power connection via USB from computer (+)
(i) Similar to Acentas system (±)
(ii) Satisfactory quality (±)
(iii) Power connection via USB from computer (+)
(iv) Heavy weight antenna (~500 g) (−)
(i) Best quality (+)
(ii) Power independent from computer (±)
(iii) LAN and WLAN connection (+)
(iv) Very heavy (~3 kg) (−)
(v) WLAN connection only <50 m (−)
(vi) WLAN energy consuming (−)
(vii) Plug and power cable necessary (−)
(i) Lightweight (+)
(ii) Good quality (+)
(iii) Power connection via USB from computer (+)
(iv) USB cable cannot be unplugged (−)
(v) Control LED missing (−)

Wearing comfort of the chest belts was evaluated via a short anonymous questionnaire which all 12 participants (see Section 2.1) had to complete directly after wearing. The questionnaire contained five questions about the handling and individual fitting of the chest belts. The questions one and two are positively formulated whereas questions three to five are negative (see Table 3). All questions had to be answered by an even scaled rating system including six categories. The six different categories range from “totally agree” = 5 points to “absolutely disagree” = 0 points. To compare positively and negatively formulated questions, the negative one was transformed before mean value was calculated. Maximum score for the whole questionnaire was 25; hence, each question was scored with maximal 5 points.


StatementAcentasActivioPolarSuunto

“The belt is easily fixed and locked”3.83.42.82.9
“The belt is precisely and simply adapted to my chest girth”3.13.23.22.6
“I consider the belt disturbing right after attaching it to the body”0.91.00.91.1
“The belt hindered me during match play”2.62.71.62.9
“The belt caused pain”3.53.52.03.6


SystemAcentasActivioPolarSuunto

Installation(i) On USB stick (+)
(ii) All drivers included (+)
(i) Easy installation (+)
(ii) Driver problems (Win XP) (−)
(i) Installation by Polar personnel (+)
(ii) Complicated installation process (−)
(iii) Connectivity of LAN not existing (−)
(iv) Permanent update function (−)
(i) No software included, download only on homepage (−)
(ii) Same problem with drivers (−)
(iii) Nevertheless, easily installed (+)

Menu navigation
(intuitional, unproblematic)
(i) Automatic recognition of chest belts (+)
(ii) Well-arranged & simple screen (+)
(i) Very good arrangement of functions & team function (+)
(ii) Long-loading software (−)
(iii) Manual recognition of chest belts (−)
(i) Long-loading software (−)
(ii) Long recognition of each receiver unit (−)
(iii) Complex software (−)
(iv) Overloaded screen (−)
(v) No automatically recording of data (−)
(i) Well-arranged & simple screen (+)

Features
(user friendliness, help function, additional function, graphical representation of data, use and treatment of data)
(i) Simple personalisation of a subject (+)
(ii) Complete pdf instruction manual (+)
(iii) Good and easy graphical representation of data (+)
(iv) Simple data recording & saving (+)
(v) Good data export (+)
(vi) No import of different systems’ data possible (−)
(i) Simple and fast recognition of active chest belts (+)
(ii) Good team function (+)
(iii) Clear separation of single and team players (+)
(iv) Help function did not work (−)
(v) Good and easy graphical representation of data (+)
(vi) Good data comparison & treatment (+)
(vii) Simple data print, pdf creation and export (+)
(viii) No import of different systems’ data possible (−)
(i) Multifaced and user-friendly data creation (+)
(ii) Comprehensive features (+)
(iii) Extensive help function (+)
(iv) Initial problems (−)
(v) Good and easy graphical representation of data and saved sessions (+)
(vi) Overloaded graphical presentation for several individuals (−)
(vii) Email export (+)
(viii) No import of different systems’ data possible (−)
(i) Unlabelled belts, boxes (−)
(ii) Time-consuming data input (−)
(iii) Missing “team” function of monitor software (−)
(iv) Poor search function (−)
(v) Good help function (+)
(vi) Good and easy graphical representation of data (+)
(vii) Separate connection of every single player (−)
(viii) Simple data recording & saving (+)
(ix) No additional features (−)
(x) No import of different systems’ data possible (−)

2.4. Measurements

The main field test was measurement of failure ratio during the half time of a real soccer match. All players apart from goal keeper were equipped with chest belts of the relevant system and measured during first half (= 45 min) of a league soccer game during four home matches in summer 2009. Temperature range was within 9°C, and weather conditions were similar (no precipitation) for all measurements. The receiver of the relevant system was always positioned three meters in projection of midline of the soccer ground. In order to improve transmission distance, receiver was positioned one meter above ground level. As data loss due to injury could not be excluded a priori, a mean failure ratio was determined only with transmitters sending valid data. Each deviation/loss of one percent of whole duration measurement of 45 min (= 2700 s) was subtracted with 10 scores, and a maximal score of 100 could be obtained. If, for example, a total loss time of all ten transmitters of a system was 270 s, a total score of 90 would be credited to this system. A failure ratio of 10 percent of whole duration measurement would be credited with a zero score.

Distance measurement was conducted on a uphill street with a constant slope and in far distance to high-voltage line or radio tower to reduce possible interference. Receiver antenna was placed on highest elevation of the hill one meter above ground, and unit was connected to a laptop (Toshiba Satellite Series; Toshiba Inc, NY, USA). The participants wearing the chest belts were equipped with a GPS device (Nokia N79, Nokia Group, Espoo, Finland) to measure distance. By moving backwards it was guaranteed that the chest belt transmitters were permanently faced to the receiver. Measurements were terminated when transmission signal was lost. Additionally, the participants were instructed to move arms to keep HR variable with time. Attempts were only started when both valid signals of GPS and HR were available and stopped when HR signal was lost. Tests were conducted twice, and mean value of both attempts was calculated and taken as maximal range. Maximal range distance was credited with 70 scores whereas the system with the absolute maximal covered range was taken as reference system and credited maximum score. Percentage deviation of distance of other systems was analogously deducted as percentage of scores. If, for example, reference system had 100 m range, it was credited 70 scores. If a second system had only 50 m range, it would be credited 50% of maximum score, hence 35.

Comparison against ECG was conducted in a laboratory by using ECG tool Vicardio ECP-6 standard (Energy-Lab Technologies, Hamburg, Germany). Measurement was conducted two minutes at rest with single-polar extremity drains. Mean HRs of both ECG and relevant system were compared. A maximum score of 30 could be obtained at this test. A deviation of 1 beat per minute (bpm) was deducted with two scores. Only complete accordance of both values was credited with maximum score.

3. Results

3.1. Comparison of Hardware

Results for hardware are summarized in Table 2. A plus (+) is a positive feature; a minus (−) is a negative one. If all systems were similar or identical, there is no separate classification. A plus/minus (±) indicates aspects which are both positive and negative. In context of the supplied hardware Acentas and Polar lead the field of monitor systems, both reaching 100 and 106 scores in sum for the hardware. The leading position of Polar depends on the storage capacity of the chest belt increasing data safety but also decreasing wearing performance which is indicated by the chest-belt questionnaire presented in Table 3. In case of wearing comfort, it has to be noted that both chest belts of Activio and Acentas were supplied by the identical manufacture. Both Polar and Acentas delivered high-quality receivers.

3.2. Comparison of Software

Results of software evaluation are summarized in Table 4. Again, similar to hardware evaluation, a plus (+) is a positive feature, and a minus (−) is a negative one. If all systems were similar or identical, there is no separate classification. A plus/minus (±) indicates aspects which are both positive and negative. Here, Activio offered the harmonious concept which led to the highest ranking. The installation procedure was easiest with the Acentas software, but Acentas did not focus on extensive statistical analysis. In this case, Polar as well as Activia provide a more developed software tool.

3.3. Measurement Results

As data of ECG comparison were only obtained with a single subject, they have to be regarded with caution. Further evaluation would be necessary to confirm or reject results. Data are represented against reference system. Acentas had 86 bpm average HR versus 88 of reference system. Suunto’s deviation was 80 to 85 bpm, Activio’s deviation was 81 to 85 bpm, and Polar’s deviation was 82 to 85 bpm, respectively. Score distribution is depicted in Table 5. Concerning distance measurement, same caution has to be taken as only one subject’s data were included in analysis. Nevertheless, Acentas gave reference with 349 m, followed by Activio with 141 m and Suunto with 107.5 m. It has to be noted that Activio’s system was supposed to reach 300 m. Last but not least, Polar reached 102 m which corresponded to the manufacturer information for this system. Nevertheless, all other systems had a wider range of data acquisition, and hence score distribution was credited as follows: Acentas 70, Activio 28, Suunto 21, and Polar 20.


CategoryMaximum attainable scoreAcentasActivioPolarSuunto

Hardware150

Instruction manual1091050
Packing25
Quality1075107
Handling1514556
Chest belt65
Storage1000108
Battery109968
Quality2017171815
Wearing comfort2514141113
Receiver50
Power consumption55535
Quality2519192317
Handling151381013
Connectivity53353

Software150

Installation1010544
Menu navigation4035383035
Features10075859050

Measurements200

ECG-deviation3026222420
Distance7070282021
Failure quota1009480900

Overall score500420353364225

Relative score (in %)84707345

Price (in EUR)ca. 3.500ca. 6.000ca. 3.500ca. 3.000

Regarding failure quota, average downtime was 148 s (0,6%) for Acentas, 286 s (1,0%) for Polar, and 552 s (2,0%) for Activio. The Suunto system stood out negatively, as a complete data loss was seen. Single belts were recognised from time to time, but no consistent data acquisition could be accomplished. A receiver problem might have caused this failure. Therefore, Suunto was not scored at all in this test. Similarly, five out of ten Polar chest belts had signal loss and could not be reactivated whereas Activio managed 10 out of 10 signals, and for the Acentas system 9 out of 10 were used for data evaluation as one person was injured during match play. The summarized scores and overall results are represented in Table 5.

4. Discussion

The purpose of the present study was to compare four commercially available online heart rate monitoring systems and especially their use in everyday situation in team sports. The system of Polar could score especially with a clear design, useable features of the software, and a convincing product quality. Only Polar uses WLAN for data transmission and also adds a mobile terminal in pocket format which can easily be used outdoor and with bad weather conditions. But the high energy consumption of WLAN transmission would terminate the battery capacities and additionally the duration of measurement. The system of Activio has a functional and useful software and the instruction manual was very detailed and easily comprehensible. The price, however, was very high compared with the other systems. Acentas could score with a huge operating distance, a valid heart rate signal compared with ECG, a low failure quota, and an easy handling. In contrast, the system of Suunto had a total breakdown during competition measurements and could not be reactivated. So this system was evaluated as the worst. The evaluation of the several chest belts revealed no important advantage of one manufacturer. Only the chest belt of Polar showed weaknesses regarding wearing comfort. Altogether, all systems cannot be used for online heart rate monitoring. Prospectively, it would be a great challenge to realise online monitoring for swimmers, possibly leading to new opportunities for the training regime. In this context the system of Polar and Suunto had the advantage that data are also recorded on the chest belt memory during water immersion and could be evaluated after exercise.

Meanwhile, some of the tested heart rate monitoring systems are updated and upgraded with new functions, respectively. For example, Acentas included an offline memory in the chest belts and added a wrist monitor for direct heart rate analysis from up to twenty athletes by the coach.

In summary, Acentas system reached the best mark of all systems, but every system has its advantages and disadvantages depending on the using purpose, location, and weather. So this present comparative study cannot recommend a single system but rather shows strength and weakness of all systems and additionally can be used for further system improvements.

Disclosures

The investigators were responsible for the study design, data collection, data analysis, data interpretation, and submission of the paper for publication, independently of all funding sources. The authors declare that they have no financial conflict of interests to disclose with any of the companies named in this paper. There is no preferential order of companies in the paper.

Acknowledgment

The authors thank all manufactures for the delivery of evaluation systems with no charge.

References

  1. J. Achten and A. E. Jeukendrup, “Heart rate monitoring: applications and limitations,” Sports Medicine, vol. 33, no. 7, pp. 517–538, 2003. View at: Publisher Site | Google Scholar
  2. F. X. Gamelin, S. Berthoin, and L. Bosquet, “Validity of the polar S810 Heart rate monitor to measure R-R intervals at rest,” Medicine and Science in Sports and Exercise, vol. 38, no. 5, pp. 887–893, 2006. View at: Publisher Site | Google Scholar
  3. M. Kingsley, M. J. Lewis, and R. E. Marson, “Comparison of Polar 810s and an ambulatory ECG system for RR interval measurement during progressive exercise,” International Journal of Sports Medicine, vol. 26, no. 1, pp. 39–44, 2005. View at: Publisher Site | Google Scholar
  4. H. Kinnunen and I. Heikkila, “The timing accuracy of the Polar Vantage NV heart rate monitor,” Journal of sports sciences, vol. 16, p. 107, 1998. View at: Google Scholar
  5. L. Leger and M. Thivierge, “Heart rate monitors: validity, stability, and functionality,” Physician and Sportsmedicine, vol. 16, no. 5, pp. 143–148, 1988. View at: Google Scholar
  6. M. Thivierge and L. Leger, “The reliability of heart rate monitors,” Science and Sports, vol. 3, no. 3, pp. 211–221, 1988. View at: Google Scholar
  7. B. A. Lord and B. Parsell, “Measurement of pain in the prehospital setting using a visual analogue scale,” Prehospital and Disaster Medicine, vol. 18, no. 4, pp. 353–358, 2003. View at: Google Scholar

Copyright © 2011 Martin Schönfelder et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

7398 Views | 1420 Downloads | 14 Citations
 PDF  Download Citation  Citation
 Download other formatsMore
 Order printed copiesOrder

Related articles

We are committed to sharing findings related to COVID-19 as quickly and safely as possible. Any author submitting a COVID-19 paper should notify us at help@hindawi.com to ensure their research is fast-tracked and made available on a preprint server as soon as possible. We will be providing unlimited waivers of publication charges for accepted articles related to COVID-19. Sign up here as a reviewer to help fast-track new submissions.