Research Article | Open Access
Design and Development of a Flexible, Plug-and-Play, Cost-Effective Tool for on-Field Evaluation of Gas Sensors
Atmospheric pollution is one of the biggest concerns for public health. Air quality monitoring is currently performed by expensive and cumbersome monitoring stations. For this reason, they are sparse, and therefore, inadequate to provide enough accurate information on the personal exposure to pollutant gases. The current worldwide trend to address this issue consists in the use of low-cost small gas sensors, already available on the market, with a wide range of costs and performances. However, the performance of these sensors is heavily affected by the environmental conditions of the specific location used for their deployment. For this reason, it is of fundamental importance to test them in real-world scenarios. Field evaluation of sensor performance could be a challenging task because, on the one hand, they have heterogeneous output signals, and on the other hand, there is no widely shared evaluation protocol. The SentinAir system has been designed and developed to facilitate this task. It can carry out performance evaluations for any type of sensor thanks to its configurable and adaptable sensing capability, multiple wireless sensor network compatibility, flexibility, and usability. In order to evaluate SentinAir capabilities and functionalities, the performances of CO2, NO2, and O3 sensors were tested in real-world scenarios against reference instruments. To the best of our knowledge, there is no previous study providing information about the performance of SP-61 (O3 sensor), IRC-A1 (CO2 sensor), and TDS5008 (CO2 sensor) achieved during on-field tests. On the contrary, results obtained by OXB431 (O3 sensor) and NO2B43F (NO2 sensor) are consistent with the ones shown in previous studies carried out in similar conditions. During validation tests, we have found for the best performing NO2 sensor, and for the best O3 sensor. Concerning the indoor experiment, the best CO2 sensor performance showed an excellent . In conclusion, the effectiveness of this tool in evaluating the performance of heterogeneous gas sensors in different real-world scenarios has been demonstrated. Therefore, we anticipate that the use of SentinAir will facilitate researchers to carry out these challenging tasks.
In the last decades, air pollution has become one of the biggest concerns in the world. It was proved that poor air quality has negative effects on public health [1–3], not just in outdoor environments [4, 5], but also in indoor ones [6–8]. Air pollution is mainly generated by anthropic activities, which have a severe impact both on human health and on global earth warming via the greenhouse effect [9, 10]. Several studies have proved that Particulate Matter (PM), NO2, O3, CO, SO2, and Volatile Organic Compounds (VOCs) are among the most relevant air pollutants which directly or indirectly threaten public health and affect air quality [1, 3–8].
Currently, air quality monitoring is mostly performed by fixed stations based on chemical analyzers. These reference instruments are based on traditional technologies and give accurate measurements. However, they are quite expensive, and their prices range between €5000 and €30000 per device . Moreover, significant additional resources are required for their maintenance, and their operation needs for significant infrastructure (secure and large enclosures, mains power, etc.) . The consequence is that, because of their high costs, households can hardly afford indoor air quality monitoring performed by these devices. Additionally, outdoor air pollution levels are detected by sparse fixed station networks that provide accurate data only in few locations, but do not provide information about localized gradients of potential importance to health protection . Clearly, personal exposure to air pollutants is hard to achieve by expensive fixed stations deployed by public authorities, so alternative means to supplement their data are beneficial [11–14].
An alternative method for air quality monitoring is based on passive samplers. They are easy to use and do not require electricity, but their limitation is that they only allow the quantification of cumulative air pollutant levels after laboratory analysis. Moreover, they cannot identify short-term pollutant episodes in real-time [12, 15].
Air quality models might be additional useful tools to supplement air quality data, but they require highly specialized knowledge, and they suffer from systematic errors depending on the model parameter choices [12, 16].
The current worldwide trend to improve spatio-temporal resolution in monitoring air pollutants, consists in using Low-Cost Small commercial gas Sensors (LCSSs) [11–14, 17–23], even though some researchers have found some limitations, or caveats, in their use [12, 24–26]. More precisely, some studies [12, 25] have shown that LCSS performance can be significantly affected by weather conditions, and in general, by the location where they are deployed. These devices are commonly called low-cost sensors because their prices, ranging from few Euros to few hundreds of Euros, might be ten times lower compared to the reference instrument prices . Almost the totality of the LCSS types belong to five different technologies: resistive sensors, electrochemical sensors, Non-Dispersive Infrared Radiation absorption (NDIR) sensors, laser light scattering sensors for PM detection, and photoionization detector sensors . In the last years, the commercial availability of LCSSs has significantly increased; as a result, there is a huge variety of LCSSs on the market with a wide range of performances .
Among the factors affecting LCSS performance, the calibration method [25, 28, 29] and also the surrounding environment where the sensors are collocated or evaluated are both relevant [12, 25]. Data quality provided by these devices depends on the test environment: laboratory test chamber, indoor, or outdoor. Moreover, as reported in some studies, the performance of LCSSs calibrated in the laboratory can significantly get worse when they are deployed on the field, or in different environments . It also might occur that calibration models obtained from calibration processes carried out in real-world scenarios can lead to different results if the sensors are used in sites or locations other than the one where they have been calibrated . In addition to this, currently, there is no widely accepted protocol to test LCSS performance, meaning further sensor data quality variability . For all these reasons, it is crucial to test or calibrate LCSSs in the field, or more precisely, in the specific location or site where they are going to be deployed.
From the end-user point of view, LCSSs cannot be usually purchased ready-to-use, but they need suitable electronic circuitry for their operation, which means extra costs. Alternatively, the electronic board required for their use might be designed and developed on purpose by the end-user.
Another option is given by Sensor Systems (SSs) based on LCSSs . They are composed of the sensors produced by the original equipment manufacturers, a protective box, a sampling system, and also by electronic hardware and software for data acquisition and treatment . In contrast to the single LCSS, SSs are ready-to-use systems, although their prices can be substantially higher than the single LCSS . Furthermore, most of the SSs could appear to be like a “black box”, therefore, it might not be so simple to integrate and operate any type of LCSS different from those provided by the SS manufacturer for evaluation purposes. Setting up a complete system having all the functionalities mentioned above could be challenging due to heterogeneous output signals of LCSSs, e.g., analog, SPI, I2C, RS232 on TTL levels, and USB.
All the factors mentioned above could make the field evaluation of LCSSs or SSs a challenging task.
To address these issues and facilitate LCSS evaluation in real-world scenarios, we designed and developed a system called SentinAir. It is intended to be a tool for facilitating LCSS or SS evaluation and also to be a cost-effective, portable, flexible, readily expandable unit for air quality monitoring. The system proposed in this paper can operate in a “stand-alone” mode, but it also can be integrated into Wireless Sensor Networks (WSNs), or it can be operated remotely through the internet or Wi-Fi connections. Moreover, thanks to its flexibility, SentinAir can be configured to work as an Internet of Things (IoT) object.
In several previous studies, systems based on LCSSs providing real-time data of air pollution were designed and implemented. Monitoring units proposed in [23, 30–33] are devoted to indoor air quality monitoring. The system illustrated in  was designed to compute an Indoor Environment Quality index (IEQ). It has no wireless link capability, while data are stored locally in a micro SD card. The study discussed in  proposes a low-cost wireless sensor network composed of sensor nodes based on Arduino architecture and XBee modules. Data gathered from nodes are uploaded on a central database, while sensor selection for each node is limited by the electronic circuitry provided by the Arduino platform. An indoor monitoring unit called “iAirCO2” based on IoT architecture is proposed in . This system provides data on CO2 concentrations in indoor environments, and it does not appear to be expandable to other types of gas sensors. The device presented in  is another IoT air quality monitoring unit that relies on an IoT platform. It provides an Android “app” for measurement readouts and data storage service. The work discussed in the paper  presents “Smart-Air”, a device based on the IoT technology and designed to monitor indoor parameters such as total VOCs, CO, CO2, temperature, and humidity. Devices presented in [30–33] cannot operate in a “stand-alone” mode, but they need an IoT platform to store and display data; therefore, the IoT platform on which they rely is an essential part of the system. Conversely, the system illustrated in  can operate only in a “stand-alone” mode, and it is not provided for any wireless link.
Concerning the pollutant gas monitoring units for outdoor/indoor purposes, a portable gas sensor system called Nasus is presented in [17, 20]. It was used to measure concentrations of pollutant gases such as SO2, CO, NO2, and BTEX coming from a landfill, while in , the same device was used to monitor CH4. Nasus IV, presented in , is a handheld device designed to monitor CO, NO2, SO2, and H2S. This device was colocated with reference instruments in  to compute Air Quality Index (AQI) in outdoor environments. Sun et al.  developed a sensor network to monitor the air quality in the urban area of Hong Kong. In conclusion, a Modular plug-and-play Sensor System (MSS) for urban air pollution monitoring was presented in . It has configurable and adaptable sensing capability provided by its Universal Sensor Interface (USI), and it can be integrated into WSNs.
2. System Design and Implementation
2.1. Design Goals and System Overview
The architecture of the Portable Monitoring Unit (PMU) presented in this study has been designed to achieve two main goals. The first one is to provide a user-friendly, cost-effective, flexible, real-time PMU for air quality to be used by low skilled end users. The second goal is essentially to provide a very flexible and easy to use tool to perform evaluations of any LCSS or SS in real-world scenarios. The general architecture of the system designed to achieve the mentioned goals is depicted in Figure 1.
SentinAir is enclosed in an inexpensive plastic, surface mounting, watertight, junction box. This cheap container is IP56 (International Protection code 56) rated, and, as it is shown in Figure 2(a), plastic bends have been attached to shelter from the rain all the sensor surfaces, the stop button, and the check lights mounted on the front panel of the box. In order to limit power consumption, system costs, and space occupancy in the SentinAir box, no air pump has been used (as it can be seen in Figure 2(b)). Sensing surfaces or openings of sensors are directly exposed to the environment air by mounting them just below the surface of the front panel device. This way, they sense the gases through holes on the front panel of the SentinAir container.
In order to ensure ease of use for low skilled end users, system interfaces have been designed in the simplest way possible. There is a stop button, which allows us to stop and shut down properly the whole system, and three LED check lights, which give indications on the SentinAir possible states: system powered and in standby, system powered and in measuring mode, system fault. Besides, complete information about the system, including current sensor measurements, sensor reading rate, system status, log files, sensor data files, and their plots, can be read on website pages (see Figure 3). They are served by the local HTTP server running on the device and accessible by connecting to SentinAir through the direct Wi-Fi link, or optionally through the internet, if the device is placed remotely from the end-user.
High skilled end users can access the SentinAir system through the direct Wi-Fi link, or optionally through the internet, to configure, update, or modify PMU software for changing the system behavior. This operation is easily feasible, thanks to the system motherboard, which is an embedded minicomputer called “Raspberry 3 B+”  with a Linux based Operative System (OS) called “Raspbian Stretch Lite”  running on. Using an embedded minicomputer like the Raspberry 3 B+, instead of microcontroller boards, offers several advantages: more computational power, multitasking capabilities, services, and programs capable of efficiently managing complex tasks for a relatively high skilled end-user. Moreover, these features enable the system to operate without external supports, such as on purpose apps to be installed on smart-phones for retrieving and visualizing data (see for example [31, 35]), or remote cloud-computing platforms to process or store measurements (see for example [30, 31, 35]).
SentinAir system can be connected to LCSSs, SSs, and Reference Instruments (RIs) such as chemical analyzers. This operation can be quickly done by plugging them into the Raspberry USB or Ethernet ports. The system, thanks to its OS routines and thanks to the software modules developed in our laboratory, automatically recognizes the LCSS, the SS, or the RI connected, providing the plug-and-play feature, which in turn, gives more ease of use and flexibility.
The dual wireless communication channel also ensures portability and flexibility: the user can reach SentinAir through the Wi-Fi LAN created by the device, or he can access it through internet connections from the remote. This is possible thanks to the Dataplicity service : it makes reachable SentinAir IP private address from the internet through the “IP tunneling” technology [39, 40]. SentinAir users can choose among several “IP tunneling” service companies available on the web, each of them offering various pricing options: from free pricing plans to few Euros per month [41, 42].
The device is powered and gets started by plugging the power cable into standard power sockets providing electrical alternated current at 220 Volts. An inexpensive and lightweight embedded ac/dc switching converter provides the electrical power to the whole system. Obviously, power consumption strictly depends on the sensor array, which acts as the system payload, but typically it seldom goes beyond 6 Watts.
The system has been designed trying to limit as much as possible its building costs, which are summarized in Table 1. The total system cost is approximately 162 €, excluding the sensor array payload. As for power consumption, also the device weight strictly depends on the sensor array, but it never exceeds 1.5 Kg.
2.2. System Motherboard and Its Relevant Software Modules
The “brain” of the system is on its motherboard, which is an inexpensive Raspberry 3 B+ board with a Cortex-A53 (ARMv8) 64-bit microprocessor running at 1.4 GHz, and with a 1 GB LPDDR2 SDRAM . It is an embedded minicomputer which dimensions are . Like every computer, the motherboard has an Operative System (OS) running on it. The OS, all of the programs or applications, and their data files are stored in the mini SD card. The Raspberry board has a built-in 802.11n Wireless LAN adapter, which we have configured to act as a Wireless Access Point (WAP) for creating a Wi-Fi LAN without the support of external LAN routers. SentinAir motherboard has four USB 2.0 ports, an SPI port, an I2C port, a TTL serial port, and an Ethernet port, which are used in the SentinAir system to connect the USB stick modem, the LCSS USB adapter, the SSs, the LCSS boards, or the RIs. Moreover, the Raspberry board is provided with General Purpose IO pins (GPIOs), which are used in the SentinAir system to drive the LED check lights and to detect the stop button pressure. A 5 Volts direct current powers Raspberry board, and its typical power consumption is nearly 2 Watts.
The system motherboard is in charge of reading sensor data, making data elaborations, and storing them in the SD card. Moreover, it is in charge of creating a Wireless Access Point (WAP) for getting direct Wi-Fi connections, managing IMAP/SMTP connections, and running the HTTP server, which acts as an end-user interface. All of these tasks are carried out by both the OS and the software modules developed in our laboratory. The software modules are written in Python , which is an open-source, high-level programming language supported by the Raspbian OS, and freely distributed together with the OS itself. Figure 4 shows a schematic representing the software architecture running on the motherboard system.
The software modules developed in our laboratory are the core of the system. As it is shown in Figure 4, they are the Monitoring Process Manager (MPM), the IMAP/SMTP Communication Module (ISCM), the User Interface Routines (UIRs), and the LCSS/SS or RI Drivers (LRDs). User commands can come either from SSH connections through the built-in 802.11n Wireless LAN adapter, or from SSH connections through the optional internet channel provided by the Huawey E303 USB stick modem plugged into one of the USB motherboard port. The main wireless communication channel is the Wi-Fi LAN set up by the system. Extra software modules to be installed on the motherboard are needed to its operation: they are “hostapd” and “dsnmasq”, both open-source software supported by the Raspbian OS and freely downloadable from the internet. The optional wireless communication channel is given by the chain composed by the USB stick modem hosting the Internet Service Provider (ISP) SIM card, and the “IP tunneling” service provider. User commands from high skilled users can be given through the Command Line Interface (CLI) provided by the UIRs. Figure 5 shows the appearance of the CLI, while in Table 2, there is a list of the user commands.
The system receives all the HTTP requests coming from the user web browser via Wi-Fi or internet, as a particular type of user command. For this reason, they are received by the HTTP server running on the motherboard and subsequently sent to the UIRs, which, in turn, send back the requested data. This operation is done by the Common Gateway Interfaces technology [44, 45] supported by the HTTP server. The HTTP server used for the SentinAir system is called “Lighttpd” ; it is an open-source software fully supported by the Raspbian OS, and freely downloadable from the internet. Finally, any kind of user command constitutes the input for the UIRs, which in turn, transmit it to the MPM, which responds by giving back the requested data to the UIRs. Subsequently, they are turned toward the channel where they are from. ISCM is the other software module in charge of managing communications between the MPM, the OS, and IMAP/SMTP servers. This module is necessary to operate the extra communication channel we designed, based on sending e-mails containing user commands to the system and on receiving e-mails containing responses from it. Obviously, to use this channel, an e-mail account is needed. We designed this particular communication protocol thinking to all those cases or situations in which SentinAir is placed in areas where the radio signal is weak or unstable. Basically, every 30 seconds, ISCM tries to connect to the IMAP server to check for e-mails containing command lines from the user. If the connection fails because of the weak radio signal, it keeps trying to connect until it achieves success. If an e-mail for SentinAir is found, the included command line is automatically read and executed; therefore, ISCM tries to connect to the SMTP server to deliver the response. The MPM is the engine of the system, and it is structured in a multitasking way, as Figure 6 shows. Essentially, it is in charge of detecting possible faults, gathering sensor data through LRDs at the sampling rate selected by the user, computing hourly and daily averages for each of the measurements, plotting their graphs, making data elaborations, and storing data in files on the SD card of the motherboard. Figure 6 shows the simplified flow chart of the MPM.
One of the most relevant features of the SentinAir system is the automatic identification and management of the sensor boards or devices connected to the motherboard through USB ports, Ethernet ports, SPI, I2C, or serial ports. This functionality is implemented by the software drivers (LRDs) written in Python language inserted in the software system. Each LRD is specific for just one sensor, or sensor board, or device pluggable into SentinAir, giving to the system the plug-and-play feature. MPM can deal with any LRD thanks to the common software function interfaces that any LRD compatible with the SentinAir system must-have. A list of software drivers currently developed and tested for SentinAir is shown in Table 3.
Considering that it is possible to use USB hub devices, USB expander, or Ethernet hubs to connect the devices to the system, practically the maximum number of devices that SentinAir can manage at the same time is only limited by the room available in the enclosure of SentinAir.
2.3. LCSS USB Adapter
A relevant part of LCSSs available on the market gives their output data as an analog signal, which usually is a direct current or tension. Therefore, it is necessary to convert those signals in digital ones using Analog-to-Digital Converters (ADCs) to allow their treatment and storage. Raspberry 3 B+ board has not a built-in ADC; for this reason, the LCSS USB adapter has been designed and developed in our laboratory. It was preferred to design and develop this device in-house instead of using electronic boards already available on the market and then fit them for the purpose, to put the maximum care in managing the signals coming from the sensors. In fact, it is imperative to avoid that during the analog-to-digital conversion, additional and unwanted electronic noise affects the sensor measures. This is very important for preserving LCSS performance that could be heavily degraded by unintentional electronic noise, especially in the case of the electrochemical sensors. The custom PCB (Printed Circuit Board) of the LCSS adapter was created by using ORCAD 10.0 CAD tool; its dimensions are . In Figures 7 and 8 are shown a copy of the LCSS USB adapter.
LCSS adapter typical power consumption is 0.06 Watts, and it is powered through the Raspberry USB port. Raspberry USB ports are suitable for the purpose because they give a maximum power of 0.5 Watts at 5 Volts. The LCSS adapter board is designed to be also powered through Li-Ion 3.3 Volts batteries. In order to charge the batteries, the board is provided with all the necessary electronic circuitry. The LCSS adapter board was designed to be also ready for being equipped with a Bluetooth adapter. These features are not used in the SentinAir system, but they were designed and implemented for future works. Sensors or devices having an analogical output signal, which could be either a current or a voltage signal, have to be wired to one of the ten channels available on the adapter. The signals are amplified by the low noise AD8609 operational amplifiers, while the gain of each amplifier can be set by a resistive trimmer placed on the PCB. Moreover, electrical noise is filtered out by filter blocks on the LCSS USB adapter board. The output of the amplifiers is converted by the PIC18F4685 microcontroller manufactured by Microchip. The software running on the microcontroller, written in C language, performs further filtering on data coming from the sensors using the rolling average technique. Sensor data are given through the USB port when they are requested by the motherboard that acts as the “master” device, while LCSS USB adapter board acts as the “slave”. The FT232RL microprocessor is the interface between the USB socket and the PIC18F4685 serial port. Figure 9 shows a schematic of the LCSS adapter and its main blocks.
3. System Evaluation
SentinAir system functionalities were evaluated in two different experiments where the PMU was used as a tool for on-field LCSS evaluation. The first experiment was performed in an indoor environment, while the second one was performed outdoor. In the second experiment, LCSSs mounted inside the SentinAir device were calibrated and then validated against RIs connected through the Ethernet port of the SentinAir system. During the tests, measurements were taken at five minutes sampling rate. Considering that the ENEA laboratories are placed out of the SentinAir wireless LAN range, we planned to download the data and control the device using the internet link through both the IMAP/SMTP communication channel and the web page interface provided by the HTTP server to evaluate SentinAir communication system. Just before we started the experiment, ENEA laboratories were temporarily shut down due to the COVID-19 pandemic. The laboratory staff was forced to work at home, so we decided to go on with the experiment and to remotely control the SentinAir device from home instead of controlling it from our laboratories like it was planned at first. On the contrary, during the indoor experiment, connections with the device were made by the wireless LAN automatically set up by SentinAir. During both the experiments, the system automatically computed in real-time the hourly and the daily average of all the measurements, producing two additional datasets.
3.1. System Evaluation in an Indoor Environment
In the first experiment, the SentinAir device was placed in an office of the ENEA laboratory center of Brindisi. The room is 4 m long, 3,5 m large, and 2,5 m high. In the daytime, it typically hosts an office worker and occasionally the other two employees; in that room, there are also some plants to decorate the office. This experiment was designed to evaluate the SentinAir device as a tool for assessing CO2 LCSS performance in an indoor environment. The RI was the CARBOCAP Carbon Dioxide Module GMM112 by Vaisala  mounted in the 106L GO3 PRO package by 2BTechnologies. It was previously calibrated in our laboratory by certified CO2 gas cylinders. The sensors under test were two copies of IRC-A1 sensors by Alphasense and two TDS0058 by Dynament . At the same time, temperature and relative humidity were, respectively, measured by the TC1047 sensor by Microchip and the HIH5031 by Honeywell. Both the IRC-A1 and the TDS0058 are NDIR sensors with a measurement range from 0 ppm to 5000 ppm. The IRC-A1 was purchased with the support circuit provided by Alphasense, which gives IRC-A1 output through the USB port. At the same time, the TDS0058 sensor was mounted on the evaluation board called “Multisensor” developed and distributed by Tecnosens . Both the CO2 sensor support circuit boards give CO2 concentrations calculated by the onboard processor on the base of calibrations carried out by the sensor manufacturer or distributor. For this reason, to evaluate these sensors, we compared their responses with the RI measurements by calculating the squared correlation coefficient (R2), as well as the Mean Absolute Error (MAE), and the Standard Deviation (SD) defined in equations (1) and (2): where is the reference value corresponding to the sensor value at the time , is the number of records in the dataset, and is the gas concentration value calculated by the sensor data at the time . According to , NDIR CO2 sensors generally have good sensitivity and selectivity, so the effects of interfering gases were not expected, and therefore, not measured.
3.2. System Evaluation in an Outdoor Environment
In the second experiment, SentinAir was placed in the car park of the ENEA laboratory center of Brindisi (40°35’23.9”N, 17°50’59.5”E) nearby the railway line in a semirural area (see Figure 10). It was arranged very close (about 2 meters) to the little room where the RIs were placed.
The RIs were connected to the SentinAir system through the Ethernet cable to calibrate and then validate LCSSs. Data flowing from RIs were used together with LCSS output signal readouts to make measurement records. This way, it was possible to have data from both RIs and LCSSs in a unique CSV (Comma Separated Values) file, made with records featured by the timestamp together with the RI and LCSS measurements. RIs and LCSSs connected to the SentinAir system acted as slave devices, while SentinAir acted as the master; this way, time synchronization of every measure was ensured. The LCSSs used to test SentinAir usability and functionalities in this experiment were chosen from the ones designed to monitor O3 and NO2. They were two NO2B43F by Alphasense (NO2 sensors), two OXB431 by Alphasense (O3 sensors), and two SP-61 by Nissha FIS (O3 sensors). The RI used for calibrating and validating NO2 sensors was the AC32M by Environnement, which is a chemiluminescent chemical analyzer. Moreover, to calibrate and validate O3 sensors, an UV absorption chemical analyzer, the O342M by Environnement, was used. The sensors that we chose to measure temperature and relative humidity were the HIH5031 by Honeywell and the TC1047 by Microchip. All of the sensors above mentioned were mounted in the SentinAir device enclosure at the same time in this experiment (see Figure 2(b)). LCSSs chosen for the test and supplied by Alphasense are four-electrode electrochemical sensors designed for low ppb gas levels. As well as the normal Working, Reference, and Counter electrodes, these sensors include a 4th auxiliary electrode, which is used to correct for zero current changes. Each sensor was mounted on test boards (called Individual Sensors Boards, or ISB) provided by Alphasense. The two SP-61 sensors installed in the SentinAir system are resistive semiconductor type sensors designed for low ppb ozone detection. They were purchased with their evaluation modules provided by the manufacturer . LCSSs used in this experiment provide direct tensions or currents as output signals through their test boards or evaluation modules. They were connected to the LCSS adapters, which in turn, were plugged into the SentinAir USB ports.
To assess the performance of the LCSSs, we considered the Linear Regression (LR) as a calibration model. The calibration function is described for each sensor in Table 4. As shown in [11, 12, 25–28], the performance of both electrochemical and resistive sensors is affected by temperature, humidity, and interfering gases. In order to take into account the effects of these interfering parameters, it was also considered, as calibration function, the Multivariate Linear Regression (MLR) model, which gave significant improvements as shown in . In our study, several MLR models were tested, but just the ones listed in Table 4 are presented; both the LR and MLR were computed using the ordinary least square method. As can be noted in Table 4, the MLR model takes into account the interfering effects of relative humidity, temperature, and also the ones given by the interfering gases.
Regarding the NO2 sensors, the most relevant interfering gas is the O3 and vice versa. We chose to take into account these effects by inserting in the MLR models the responses of the relative sensor instead of the RI measurements. This way, we meant to investigate if it is possible to achieve meaningful improvements in LCSS performance without the RI support in real-world scenarios. For example, in the case of ozone sensors, we considered in the MLR, the best NO2B43F response obtained by the LR models, instead of the NO2 measurement given by the AC32M reference analyzer. This way, we intended to assess the predictive capacity of the SentinAir system without relying on RIs.
This experiment was composed of two periods: after calculating the calibration functions in the first period, we performed a second test to validate them. In this latter period, the tests were carried out following the same conditions as the first one. To assess the performance of each model, a comparison between RI measurements, and the responses given by the sensor data elaborated by the LR and MLR models, was done. To evaluate how much RI and LCSS measurements are correlated, and therefore the quality of the data, we have calculated the squared correlation coefficient (R2), as well as the Mean Absolute Error (MAE), and the Standard Deviation (SD) defined in the equations (1) and (2).
4. Experiment Results
In this section, the results of indoor and outdoor experiments are shown. For each test, the results obtained by the datasets having measurements made at the sampling rate of five minutes are presented. SentinAir communication channels and the whole system worked without any fail throughout both the experiments. We could verify that by remotely monitoring the SentinAir device, and by downloading the log files stored in the SD card of the system, which gave us information about every significant event that occurred.
4.1. Indoor Experiment Results
Indoor tests were carried out in two periods: the first one was from the 3rd of February 2020 to the 9th of February 2020, while the second one was from the 10th of February 2020 to the 15th of February 2020. The dataset related to the first period is therefore composed of the CO2 concentrations provided by the reference, the CO2 measurements given by the first copy of the IRC-A1 sensor (here denoted as IRCA1(1)), the CO2 concentration levels provided by the first copy of theTDS0058 (indicated as TDS0058(1)), the temperature, and the relative humidity. The dataset concerning the second period has a composition similar to the first one; the difference is that the other two CO2 sensors (called here IRCA1(2) and TDS0058(2)) were involved. Figures 11 and 12 summarize the statistics concerning the CO2 concentration measured by the RI, the temperature, and the relative humidity related to the two periods of the experiment.
CO2 trend shown in these figures was affected by the people present in the office room, by how long the office window was kept open, and therefore, by the ventilation rate. The temperature and humidity levels were influenced by the use of the heater systems. Anyway, Figures 11 and 12 show that CO2 concentration levels during the experiment were featured by a remarkable heterogeneity, which is useful to facilitate the sensor evaluations. Table 5 shows the results of both the test periods. The performance indicators were computed by comparing the CO2 sensor outputs with the RI measurements. The dataset related to the first period is composed of 1732 records, while the second one has 1731 records. INTERCEPT, MAE, and SD are expressed in ppm.
Table 5 indicates that each of the sensors under test achieved excellent results, especially in terms of R2 and slope, both very close to the unit value. In particular, the TDS0058(2) has given the best results respect to all the indicators chosen to assess the performance of the sensors. The quality of the results provided by the CO2 sensors is also highlighted in Figure 13.
In this figure are shown the scatter plots representing the CO2 concentrations provided by the RI and the related data given by the sensors. The overall good performances of the CO2 sensors are also highlighted in the time series shown in Figure 14. In this figure, concentrations given by the RI at a specific time are compared with the measurements of the sensors.
4.2. Outdoor Experiment Results
As mentioned earlier, this experiment was split into two parts: the calibration period, that went on from the 17th of March 2020 to the 22nd of March 2020, and the validation period that went on from the 27th of March 2020 to the 9th of April 2020. The overall experiment duration was less than a month; therefore, sensor drift effects were considered negligible. The datasets obtained through this experiment are composed of O3 and NO2 concentrations expressed in ppb and provided by the RIs, temperature in Celsius degrees, relative humidity, tensions given by the two copies of SP-61 sensors, and electrode tensions given by the working and auxiliary electrode belonging to the two copies of NO2B43F and OXB431 sensors. Ozone and nitrogen dioxide concentration levels recorded in the datasets are typical of the semirural areas in this period of the year: ozone concentration levels are modest, while the nitrogen dioxide ones are quite low. Whisker box plots shown in Figure 15 summarize, respectively, temperature, relative humidity, NO2, and O3 statistics.
As for all of the sensors involved in this experiment, we first performed LR calibrations, and then we used the O3 sensor that gave the best results in the LR calibration to compute the MLR function relative to the other NO2 sensors. Similarly, we used the best NO2 sensor in the LR calibration to calculate the MLR function for all the O3 sensors. A summary of the results for the sensors involved in the outdoor experiment is shown in Table 6.
The data presented in Table 6 show the results achieved by the calibration dataset. Therefore, they are compared with the validation dataset to give an idea about the reliability of the calibration models. The total number of the calibration dataset records is 1482, while the validation dataset is composed of 3192 records; INTERCEPT, MAE, and SD are expressed in ppb. It can be noted by examining the Table 6, that R2 values are globally slightly better in the calibration period than in the validation one. At the same time, MAE and SD parameters show a slight increase. These trends are expected and, in general, they show that there is no relevant difference in terms of model performance, considering the calibration and the validation period. R2 values range from 0.357 to 0.61 for NO2 sensors, while in the case of O3 sensors, this parameter goes from 0.282 to 0.668. To get a complete picture of the experiment results, it is also essential to examine the scatter plots concerning the calibration model responses related to the true NO2 or O3 concentrations provided by the RIs. Figures 16 and 17 show this data for each sensor, comparing LR and MLR models computed by the calibration dataset and then applied to the validation dataset. The spread of the points in these plots is related to the R2 values of the calibration functions, the higher is this value, the narrower is the cloud composed of the points in the scatter plot. Finally, to complete the picture of the calibration function performances, it is useful to assess their responses in comparison with the RI measurements in the time series shown in Figures 18 and 19 as well. These last figures are only related to the validation period because it gives significant indications about the accuracy of the calibration functions computed by considering the calibration dataset as their input.
In the case of NO2 sensors, the best results are given by the NO2B43F(2). The performance of this sensor improves of the 8.7% considering the LR model () and the MLR model (). This trend is expected because MLR allows us to quantify the interfering effect of temperature, relative humidity, and ozone gas presence. The low NO2 concentrations and the almost threefold levels of ozone shown in Figure 15 did not facilitate the sensor performance; this could explain the modest extent of R2 values for both the NO2 sensors. In Figure 17 are shown the scatter plots of both the NO2 sensors. It can be noted a slightly better performance of the second sensor under test not just in terms of R2 values, but also in terms of MAE and SD as indicated in Table 6 as well. Moreover, by examining the time series shown in Figure 18, it should be noted a slight underestimation in the NO2B43F(1) calibration models. This fact also explains the general lower R2 values of this sensor compared with the NO2B43F(2). The general performance of NO2 sensors could be improved by considering in the MLR computation the interfering effect of the temperature that affects the sensor sensitivity, or by considering neural network calibration models . This aspect has not been investigated in this study, which is mainly focused on SentinAir tool presentation and evaluation, but it will be considered in future works. However, it should be noted that the R2 values related to the NO2 sensors we have found in this study are consistent (or, in some cases, slightly better) to previous works concerning similar investigations and carried out in comparable conditions [12, 28, 35].
In the case of ozone sensors, the highest coefficients of correlation are achieved by the electrochemical ones. In particular, the OXB431(2) calibrated by the MLR shows an R2 value of 0.666 for the calibration dataset and a similar R2 value () achieved in the validation period. By examining Table 6 and the scatter plots in Figure 17, we can see that, in general, ozone electrochemical sensor responses are slightly better correlated with the reference measures. We think that this is also due to the low concentrations of NO2, which is the main interfering gas for ozone sensors, in conjunction with the O3 levels that are threefold respect the NO2 ones, as it is shown in Figure 15. The semiconductor type sensors show modest R2 values in the calibration period and low correlation in the validation tests. We suppose that this is mainly caused by the effect of the interfering gases such as the CO. In fact, especially during the calibration period, we noted a drop of the sensor signals in correspondence of CO peaks measured by the CO chemical analyzer collocated in the room together with the other RIs we used. We have just indicative observations of this effect; unfortunately, in this study, we could not investigate or confirm this, although it could be a matter for future works. Anyway, as already noted for electrochemical NO2 sensors, we suppose that for ozone sensors as well as the correlation with the RI could be improved by considering the effects of temperature or humidity in the sensor sensitivity to be calculated in the MLR, or by using neural network calibration models. Also for the electrochemical ozone sensors, if we consider the same test conditions, which are mainly given by the same calibration functions, or the same environmental scenario, the results found in this study are consistent with similar previous studies [12, 28, 56]. Regarding the performance of the SP-61 sensors, we did not find investigations concerning on-field tests of this sensor. Spinelle et al.  tested it in a laboratory test chamber; therefore, we could not make comparisons between our results with this previous study.
Concerning the indoor experiment, we can surely say that the already known good selectivity and sensitivity of NDIR CO2 sensors  were confirmed. In fact, thanks to these features, it has been possible for all the sensors evaluated in this study to achieve R2 values above 0.9 units. This parameter, in conjunction with the SLOPE very close to the unit value, gave us indications that these sensors do not significantly suffer from interfering effects of temperature or humidity. The best performance is achieved by the TDS5008(2), which has shown remarkable results in terms of the correlation coefficient, MAE, and SD (, , ). The excellent performance of this sensor is also confirmed by the SLOPE very close to the unit value (0.985). However, we did not find previous studies that provide quantitative information about the performance of these sensors in terms of R2, MAE, and SD.
In this study, we presented the SentinAir system architecture to address the fixed hardware configuration issue of the existing air quality monitoring units. The system proposed in this paper represents a new idea of PMU. It can be used both as a portable gas monitoring unit and as a tool for LCSS evaluation in real-world scenarios. Its main features are configurable and adaptable sensing capability, compatibility with multiple wireless sensor networks, flexibility, and usability. To the best of our knowledge, SentinAir is the first air monitoring tool with plug-and-play features based on an embedded minicomputer. The software driver system developed for LCSSs, SSs, and also for RIs enables us to use any device in a plug-and-play way. The design and the development of a SentinAir prototype were illustrated in this paper; moreover, its capability to easily fit in different real scenarios was proved as well. Evaluation of the system functionalities and capabilities was carried out by performing two separate experiments. The first one was carried out in an indoor environment, where four LCSSs were evaluated. The second one was an outdoor experiment, where the performance of six LCSSs was assessed.
The datasets made by the outdoor experiment were composed of measurements of RIs plugged on purpose into the SentinAir device. The NO2 and O3 sensors were calibrated using LR and MLR functions through the SentinAir LCSS USB adapter. Subsequently, their performance was evaluated by validation tests carried out by comparing sensor responses with RI measurements. The best result during the validation tests was achieved by the ozone sensor OXB431(2) () calibrated by the MLR function. Concerning the evaluation of the NO2 sensors, the best performance was given by the NO2B43F(2) () calibrated by the MLR as well. Results of the electrochemical sensors show that their performance in terms of squared correlation coefficient (R2) is consistent, and in some cases, better, to previous studies carried out in similar conditions. Concerning the SP-61 sensors, to the best of our knowledge, this is the first study that investigates their performance by on-field tests.
In the indoor experiment, four NDIR sensors for CO2 monitoring were evaluated. They were previously calibrated by their manufacturers or distributors and enabled to give the CO2 concentrations through USB port connections. Data provided by the sensors were gathered by the SentinAir system and compared with measurements given by the RI plugged into it. Performances of all the CO2 sensors were excellent: R2 values for each of the sensors were beyond the 0.9 units. The best result for CO2 sensors was obtained by the TDS5008(2) (). Although previous works indicated that NDIR CO2 sensors are generally featured by excellent performance, we did not find any study providing quantitative indications about their performance in terms of R2, MAE, and SD.
In conclusion, the advantages of using the SentinAir tool to evaluate various LCSSs or SSs in a plug-and-play way was proven. Moreover, this study provided for the first time quantitative information in terms of R2, MAE, and SD about on-field tests concerning TDS5008, IRC-A1, and SP-61 sensor performance. Finally, the implementation of more SentinAir devices, the development of more software drivers to enable the use of more LCSSs, SSs, or RIs, the use of SentinAir tool to investigate alternative calibration functions or models for further improving LCSS performance are considered as future works.
Data are available on request. Please contact the author Domenico Suriano: email@example.com
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
- B. Brunekreef and S. T. Holgate, “Air pollution and health,” The Lancet, vol. 360, no. 9341, pp. 1233–1242, 2002.
- WHO, “7 million premature deaths annually linked to air pollution,” 2014, February 2020, https://www.who.int/mediacentre/news/releases/2014/air-pollution/en/.
- J. A. Bernstein, N. Alexis, C. Barnes et al., “Health effects of air pollution,” Journal of Allergy and Clinical Immunology, vol. 114, no. 5, pp. 1116–1123, 2004.
- R. McConnell, K. Berhane, F. Gilliland et al., “Asthma in exercising children exposed to ozone: a cohort study,” The Lancet, vol. 359, no. 9304, pp. 386–391, 2002.
- M. Kampa and E. Castanas, “Human health effects of air pollution,” Environmental Pollution, vol. 151, no. 2, pp. 362–367, 2008.
- WHO, Who guidelines for indoor air quality: selected pollutants, World Health Organization, Copenhagen, Denmark, 2010.
- WHO, “Household air pollution and health,” (February 2020), 2018, https://www.who.int/en/news-room/fact-sheets/detail/household-air-pollution-and-health.
- J. H. Lee, H. S. Lee, M. R. Park et al., “Relationship between indoor air pollutant levels and residential environment in children with atopic dermatitis,” Allergy, Asthma & Immunology Research, vol. 6, no. 6, pp. 517–524, 2014.
- T. R. Anderson, E. Hawkins, and P. D. Jones, “CO2, the greenhouse effect and global warming: from the pioneering work of Arrhenius and Callendar to today's Earth System Models,” Endeavour, vol. 40, no. 3, pp. 178–187, 2016.
- IPCC, “Intergovernmental panel on climate change 5th assessment report. Climate change: the physical science basis,” (February 2020), 2013, https://www.ipcc.ch/site/assets/uploads/2018/03/WG1AR5_SummaryVolume_FINAL.pdf.
- M. I. Mead, O. A. M. Popoola, G. B. Stewart et al., “The use of electrochemical sensors for monitoring urban air quality in low-cost, high-density networks,” Atmospheric Environment, vol. 70, pp. 186–203, 2013.
- N. Castell, F. R. Dauge, P. Schneider et al., “Can commercial low-cost sensor platforms contribute to air quality monitoring and exposure estimates?” Environment International, vol. 99, pp. 293–302, 2017.
- P. Kumar, L. Morawska, C. Martani et al., “The rise of low-cost sensing for managing air pollution in cities,” Environment International, vol. 75, pp. 199–205, 2015.
- E. G. Snyder, T. H. Watkins, P. A. Solomon et al., “The changing paradigm of air pollution monitoring,” Environmental Science and Technology, vol. 47, no. 20, pp. 11369–11377, 2013.
- S. V. Krupa and A. H. Legge, “Passive sampling of ambient, gaseous air pollutants: an assessment from an ecological perspective,” Environmental Pollution, vol. 107, no. 1, pp. 31–45, 2000.
- F. Pannullo, D. Lee, E. Waclawski, and A. H. Leyland, “How robust are the estimated effects of air pollution on health? Accounting for model uncertainty using Bayesian model averaging,” Spatial and Spatio-temporal Epidemiology, vol. 18, pp. 53–62, 2016.
- D. Suriano, R. Rossi, M. Alvisi et al., “A portable sensor system for air pollution monitoring and malodours olfactometric control,” Lecture Notes in Electrical Engineering, vol. 109, pp. 87–92, 2012.
- D. Suriano, G. Cassano, and M. Penza, “A portable gas sensor system for air quality monitoring,” Lecture Notes in Electrical Engineering, vol. 268, pp. 155–158, 2014.
- M. Penza, D. Suriano, M. G. Villani, L. Spinelle, and M. Gerboles, “Towards air quality indices in smart cities by calibrated low-cost sensors applied to networks,” in IEEE SENSORS 2014 Proceedings, vol. 2014, pp. 2012–2017, Valencia, Spain, 2014.
- M. Penza, D. Suriano, G. Cassano et al., “A case-study of microsensors for landfill air-pollution monitoring applications,” Urban Climate, vol. 14, pp. 351–369, 2015.
- S. Moltchanov, I. Levy, Y. Etzion, U. Lerner, D. M. Broday, and B. Fishbain, “On the feasibility of measuring urban air pollution by wireless distributed sensor networks,” Science of The Total Environment, vol. 502, pp. 537–547, 2015.
- M. G. Villani, F. Cignini, F. Ortenzi, D. Suriano, and M. Prato, “The smart ring experience in L'Aquila (Italy): integrating smart mobility public services with air quality indexes,” Chemosensors, vol. 4, no. 4, p. 24, 2016.
- A. Tiele, S. Esfahani, and J. Covington, “Design and development of a low-cost, portable monitoring device for indoor environment quality,” Journal of Sensors, vol. 2018, Article ID 5353816, 14 pages, 2018.
- A. Lewis and P. Edwards, “Validate personal air-pollution sensors,” Nature, vol. 535, no. 7610, pp. 29–31, 2016.
- F. Karagulian, M. Barbiere, A. Kotsev et al., “Review of the performance of low-cost sensors for air quality monitoring,” Atmosphere, vol. 10, no. 9, pp. 506–547, 2019.
- C. Borrego, A. M. Costa, J. Ginja et al., “Assessment of air quality microsensors versus reference methods: the EuNetAir joint exercise,” Atmospheric Environment, vol. 147, pp. 246–263, 2016.
- M. Aleixandre and M. Gerboles, “Review of small commercial sensors for indicative monitoring of ambient gas,” Chemical Engineering Transactions, vol. 30, pp. 169–174, 2012.
- L. Spinelle, M. Gerboles, M. G. Villani, M. Aleixandre, and F. Bonavitacola, “Field calibration of a cluster of low-cost available sensors for air quality monitoring. Part A: ozone and nitrogen dioxide,” Sensors and Actuators B: Chemical, vol. 215, pp. 249–257, 2015.
- L. Spinelle, M. Gerboles, M. G. Villani, M. Aleixandre, and F. Bonavitacola, “Field calibration of a cluster of low-cost commercially available sensors for air quality monitoring. Part B: NO, CO and CO2,” Sensors and Actuators B: Chemical, vol. 238, pp. 706–715, 2017.
- S. Abraham and X. Li, “A cost-effective wireless sensor network system for indoor air quality monitoring applications,” Procedia Computer Science, vol. 34, pp. 165–171, 2014.
- G. Marques, C. R. Ferreira, and R. Pitarma, “Indoor air quality assessment using a CO2 monitoring system based on internet of things,” Journal of Medical Systems, vol. 43, no. 3, 2019.
- M. Tastan and H. Gokozan, “Real-time monitoring of indoor air quality with internet of things-based E-nose,” Applied Sciences, vol. 9, no. 16, 2019.
- J. Jo, B. Jo, J. Kim, S. J. Kim, and W. Y. Han, “Development of an IoT-based indoor air quality monitoring platform,” Journal of Sensors, vol. 2020, Article ID 8749764, 14 pages, 2020.
- L. Sun, K. Wong, P. Wei et al., “Development and application of a Next Generation air sensor network for the Hong Kong marathon 2015 air quality monitoring,” Sensors, vol. 16, no. 2, 2016.
- W.-Y. Yi, K.-S. Leung, and Y. Leung, “A modular plug-and-play sensor system for urban air pollution monitoring: design, implementation and evaluation,” Sensors, vol. 18, no. 2, p. 7, 2018.
- “Raspberry,” February 2020, https://www.raspberrypi.org/products/raspberry-pi-3-model-b-plus/.
- “Raspbian,” February 2020, https://www.raspberrypi.org/downloads/raspbian/.
- “Dataplicity,” February 2020, https://www.dataplicity.com.
- S. Raab and M. W. Chandra, Mobile Ip Technology and Applications, Cisco Press, 2005.
- A. Shneyderman and A. Casati, Mobile VPN, Delivering Advanced Services in Next-Generation Wireless Systems, John Wiley & Sons, 2003.
- “Pitunnel,” February 2020, https://www.pitunnel.com.
- “Remote.it,” February 2020, https://remote.it.
- “Python,” February 2020, https://www.python.org.
- D. Robinson and K. Coar, “Rfc3875: the common gateway interface, version 1.1,” 2004, February 2020, https://tools.ietf.org/html/rfc3875.
- D. Connoly and L. Quin, “CGI: Common Gateway Interface,” 2011, February 2020, https://www.w3.org/CGI/.
- “Lighttpd,” February 2020, https://www.lighttpd.net/.
- “Alphasense,” February 2020, https://www.alphasense.com.
- “Plantower,” February 2020, https://www.plantower.com.
- “Tecnosens,” February 2020, https://www.tecnosens.it.
- “2B Technologies,” February 2020, https://www.twobtech.com.
- “Environnement,” February 2020, https://www.environnement-sa.com.
- “Nissha FIS ozone evaluation module,” February 2020, http://www.fisinc.co.jp/en/common/pdf/A1320301-SP61%20seriesE_P.pdf.
- “Vaisala GMM112,” February 2020, https://www.vaisala.com/sites/default/files/documents/GMM112-Datasheet-B210567EN-E-LoRes.pdf.
- “TDS0058,” February 2020, https://www.dynament.com/_webedit/uploaded-files/All%20Files/Data/tds0058.pdf.
- L. Spinelle, M. Gerboles, M. Aleixandre, and F. Bonavitacola, “Evaluation of metal oxides sensors for the monitoring of O3 in ambient air at ppb level,” Chemical Engineering Transaction, vol. 54, pp. 319–324, 2016.
- E. S. Cross, L. R. Williams, D. K. Lewis et al., “Use of electrochemical sensors for measurement of air pollution: correcting interference response and validating measurements,” Atmosphere Measurements Techniques, vol. 10, no. 9, pp. 3575–3588, 2017.
Copyright © 2020 Domenico Suriano et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.