Abstract
This paper introduces the Smart Home Simulator, one of the main outcomes of the D4All project. This application takes into account the variety of issues involved in the development of Ambient Assisted Living (AAL) solutions, such as the peculiarity of each end-users, appliances, and technologies with their deployment and data-sharing issues. The Smart Home Simulator—a mixed reality application able to support the configuration and customization of domestic environments in AAL systems—leverages on integration capabilities of Semantic Web technologies and the possibility to model relevant knowledge (about both the dwellers and the domestic environment) into formal models. It also exploits Virtual Reality technologies as an efficient means to simplify the configuration of customized AAL environments. The application and the underlying framework will be validated through two different use cases, each one foreseeing the customized configuration of a domestic environment for specific segments of users.
1. Introduction
Ambient Assisted Living (AAL) is a research field emerged in the 1990s and acquiring growing importance. It proposes residential tools and solutions devoted to the improvement of everyday life, focusing on the person and his/her interactions with both technologies and domestic environment. In this context, AAL solutions aim at enhancing the dwellers’ quality of life, comfort, and well-being and can be customized to address specific problems for particular segments of population, such as elderlies or people affected by disabilities. However, the development of AAL systems is not an easy task, since it has to take into account some fundamental features. The first regards the end-users for whom the systems are being developed: each solution, in fact, must be tailored on the dwellers and their needs. It is therefore imperative to adopt a proper paradigm able to be accountable for the users’ real needs and capabilities. Another relevant issue gathering the attention of many researchers concerns the possibility to get full interoperability among the devices deployed in the house. Furthermore, a third issue arises when it comes to validating the abovementioned interoperability: the costs and time to set up a domestic environment are relevant and may result dissuasive.
In order to tackle these three major issues, the D4All project relied on a consolidated framework (Section 3.5), which leverages the integration capabilities of Semantic Web technologies and the possibility to model relevant knowledge (about both the dwellers and the domestic environment) into formal models. Furthermore, the project framework took into account the possibilities offered by Mixed and Virtual Reality technologies to develop an application dedicated to the configuration and customization of domestic environments—the Smart Home Simulator.
The remainder of the paper is structured as follows: Section 2 offers a review of the use of Semantic Web technologies and Virtual Reality for the configuration of AAL systems. Section 3 delves into the aims, issues, solutions, and architecture addressed by the D4All project. Section 4 describes in detail the semantic knowledge base, backbone of the D4All project, while Section 5 depicts the role of the Virtual Reality. Section 6 proposes real use cases developed to test the efficiency of the Smart Home Simulator and its framework. Finally, the conclusions summarize the main outcomes and sketch the future works that will be pursued.
2. Related Works
This section highlights the most relevant works in the field of semantic model for the description of dwellers and users of AAL solutions, interoperability among appliances, AAL systems, and the use of Virtual Reality technologies for the simulation and configuration of living environments and smart homes.
Semantic Web technologies have proved to be an efficient way to represent the knowledge related to the domain of persons and houses, and they have been exploited in a variety of works. In particular, the semantic modelling of the smart home’s dwellers and their activities has covered an increasing importance in the last decade. In [1], the authors focused on the detection of human activity inside a home, representing them into a set of ontologies covering the contexts of the smart home and activities of daily living. This model allows to infer, among other features, activity-concerned information, thanks to reasoning processes over raw data and contextual information. Razmerita et al. [2] developed OntobUM, an ontology-based architecture providing information on the user’s identity, contacts, preferences, and competencies; this model is partially generated by the user and several intelligent services, which automatically update the information about the user taking into account his/her behaviour. In [3], an ontology for the modelling of AAL services is presented; this ontology considers concepts necessary to describe the environments, the users involved, and their activities in the environment, as well as their habits and abilities. A focus on the modelling of the user’s static information, such as physical characteristics, living condition, profession, interests, education, and user experience, is provided in [4]. This work provided a methodology for an ontology-based activity of user profiling able to avoid omissions and errors. Skillen et al. [5] proposed an ontology for the personalization of context-aware applications, which considered both static and dynamic aspects of the users. The model takes into account users’ health, interests, preferences, and abilities. In the field of context-aware adaptive systems, in [6], it is described a user-profile ontology able to represent situation-dependent subprofiles; the ontology allows to automatically trigger the personalization of services according to the users.
Also in the field of interoperability among devices, the possibility to represent domain knowledge offers several advantages, described in various works. Welge et al. [7] addressed the issue of interoperability of different devices and distributed systems identifying in the exploitation of knowledge management and ontologies the key to overcome this problem. Semantic Web technologies are presented in [8] as an effective approach to allow the management of heterogeneous information in Ambient Intelligence solutions; in this work, the FLERSA tool [9] is enhanced in its interoperability features, allowing the deployment of Ambient Intelligence applications making use of different technologies and platforms. In [10], the authors described an upper ontology for the semantic support of AAL platforms, capturing the semantics of the AAL domain and depicting the services provided by a system and including the orchestration of data coming from sensors and other services. In [11], a result of the European COMANCHE project, an ontology-enabled knowledge base allows the modelling of a domestic environment with detail on the services provided to its inhabitants. The knowledge base allows the conflict-free and up-to-date functioning of the home network while providing a description of the appliances and the relationships among them.
Dealing with the configuration and the validation of Smart Homes, in the last decade, researchers have developed several systems with the aim of testing their solutions before implementing them in the real world. Anticipating the validation of the designed systems, comprising different smart objects and sensors exchanging data in real time, in fact, reduces time and costs of a solution otherwise potentially very onerous. In [12, 13], the authors developed two context-aware simulators (ISS and CAAS) that are able to handle concurrently data coming from the sensors and home inhabitants to respond to specific user’s needs; they were both able to respond to specific sensors’ measurement also detecting and handling potential conflicts of operating rules, which may arise when conflictual actions are triggered by multiple input data. Although these two solutions were provided with a graphical-user interface and a simplistic 2D representation of the house, they do not make use of virtual reality. Instead, Sernani et al. developed a VR-based system to validate their expert system, named, Virtual Carer, which manages a distributed network of smart objects inside a smart home, where each component is modelled as an autonomous agent [14]. To recreate a realistic scenario, they also simulated the human behaviour taking into account a model based on human basic needs to trigger specific activities inside the smart home. Human behaviour was simulated also in [15] with the final aim of developing a low-cost system capable of synthesizing a dataset for activity recognition research in smart environments. In [16], the authors demonstrated that virtual environments can be a promising tool in the design phase, since they provide the stakeholders with a demonstration of the system functionalities, thus improving the final design of the solution—a nursing home, in this case—through the users’ feedback and suggestions. In [17], the authors presented a Simulation Control Panel (SCP) to be integrated in the authoring tool in order to ease the validation of the designed solution by creating a direct link between the modelling and the testing environments. Finally, VR-based simulations were also employed to test the feasibility and the reliability of brain-computer interfaces in controlling smart home appliances and services with the final aim of restoring environmental control for subjects with severe disabilities [18, 19].
3. The D4All Project: Aims and Solutions
3.1. The D4All Project: Applying Universal Design to Domestic Environments
Traditional design of products and solutions is usually oriented toward standard individuals, which are abstraction of real men and women. This approach fails to consider the many variables regarding real end-users, such as their skills, knowledge, social interactions, and requirements. To overcome this limitation, the paradigm of “Universal Design” [20]—also known as “Design For All”—has started to stand out: it aims at taking into proper consideration the different features characterizing the real human users, focusing on proposing solutions able to adjust to the specific needs of users. The main principles of this paradigm can be found also in the field of AAL, a discipline addressing the issue of increasing the quality of life of people in all stages of the lifecycle and providing them with assistive systems for an independent life, according to their abilities [21, 22]. Thus, AAL systems aim at finding efficient solutions to help elderly or impaired people to maintain an independent and autonomous living. Although many efforts have been made toward the development of AAL solutions, most of the systems and devices created are not able to take into consideration the real needs of their users and neglect the side of human interaction and real capabilities [23].
In this context, Design For All (D4All) project [24] aims at applying the guidelines of Universal Design into domotic and inclusive domestic environments. These environments, grouped under the term “Smart Home” [25], should be able to anticipate and respond to the needs of their dwellers, to promote their comfort and well-being during many activities of daily living (ADL) [26].
The design of a Smart Home for normally endowed people, families, elderlies, and people with impairments or disabilities requires a set of heterogeneous tools during its whole lifecycle process (e.g., concept, design, implementation, and test [17]). Furthermore, it requires paying great attention to the specific users who interact with the services and technologies made available, as well as they interact with these functionalities. Consequently, it is fundamental to manage in a coherent and efficient way the vast amount of diverse data that these interactions can generate, making the data available to both the end-users (the inhabitants) and the remote users (such as caregivers or clinical personnel). The following subsections delve into the specific issues belonging to these two branches, respectively, the proper identification of users’ needs, with particular attention toward being physically active, and requirements and the possibility to make the appliances and tools of a domestic environment fully interoperable among each other.
3.2. Users’ Needs and Requirements
Benefits of being physically active are widely known for people of all ages; different studies have demonstrated how physical activity can prevent the onset of several chronic illnesses such as hypertension, obesity, diabetes, osteoporosis, some forms of cancer, and cardiovascular diseases [27]. This is particularly true for elderlies, since ageing leads to the structural and functional deterioration of many physiological systems, even in absence of a specific pathology. This clinical condition is defined as frailty, a term that indicates a status of increased vulnerability, in which not only the risk of an adverse event (e.g., minor infection and falls) is increased but also the body response to a small insult results in a disproportionate change in the health condition (from independent to dependent or from lucid to cognitive impaired) [28]. In this context, it is indeed true that no intervention can stop the physical and cognitive decline related with age. However, there is strong evidence that a minimization of risks of chronic illnesses and disabilities can be obtained through regular exercise. Both the World Health Organization and the American College of Sport and Medicine provided recommendations for regular physical activity in elderlies. They underline its importance as preventive measure against physical and cognitive decline [29, 30], the loss of autonomy in daily living, and thus, the economic burden that the ageing of the world population has risen in recent years [31].
This concept holds also for healthy and people suffering from different chronic pathologies, although it is clear that for this last category of home dwellers, the provision of a supervised physical exercise may not be enough to guarantee their autonomy during ADL. In all these cases, the Smart Home must be enhanced with tailored solutions, addressing the coping with specific issues. For instance, for visually impaired users—if not completely blind—there is the need to light up efficiently every part of the house, especially dangerous areas such as steps. Deaf people need visual or haptic alarm to replace normal alerts [32]. Motor-impaired users who use a wheelchair require an accessible house, where spaces are designed to allow an easy manoeuvring of the wheelchair and where appliances and controls are reachable while staying seated [33].
3.3. Interoperability and Cooperation among the Appliances in the Smart Home
A Smart Home is expected to be a domestic residence equipped with a set of appliances—often called Smart Objects; as mentioned above, the Smart Home and its appliances work to ensure the dwellers’ comfort and personalized living conditions. In traditional houses, the various appliances deployed are able to perform their tasks in a separate and isolated way. On the contrary, inside a Smart Home, the appliances are required to work together in a reliable and predictable behaviour [34] and to acquire, handle, communicate, and share the knowledge about the home inhabitants in order to meet the goal of achieving their comfort and well-being [35]. Therefore, appliances must be able to provide tailored services to improve the dwellers’ assistance for a better, healthier, and safer life in their everyday living environments and must cooperate with several other appliances—possibly specialized and multivendor.
Nevertheless, this kind of synergy among distributed appliances is nowadays guaranteed only in domestic environments where appliances with the same communication pattern and protocol stack [36] have been deployed. The desired interoperability among appliances should comprise not only mere communication interoperability but also data and information models, as well as services provided and discovery mechanisms. Reaching this level of interoperability is not an easy task: it takes a lot of time and requires many design decisions to be made to accommodate the constrained nature of specific devices in a certain usage scenario.
3.4. The Project’s Approach and the Smart Home Simulator
D4All faces the above challenges relying on a framework that encompasses the description of the users’ physiological status, the appliances, and the services they can provide. The goal of the framework (and of the application derived from it) is to provide users characterized by frailty or impairment with appliances and services able to cope with their impairments, thus helping them in performing activities that would otherwise be unattainable or ponderous. The solution is not limited to specific categories of users but extends its services to the whole spectrum of house dwellers. This goal is achieved through the configuration of the dwellers’ house that allows to analyse both whether the specific user’s physiological requirements are satisfied and the behaviour of the interoperable appliances before their hard deployment in the real house.
The need to properly assess the user’s health condition in a qualitative and quantitative way was addressed by recurring to the International Classification of Functioning, Disability and Health (ICF), a holistic World Health Organization’s framework providing a unified and standard language for the description of health-related components. ICF conceptualizes the functioning of an individual as a “dynamic interaction between a person’s health condition, environmental factors, and personal factors” [37], acting as a tool able to ease the communication among the health stakeholders (caregivers and clinicians) and providing a standard and worldwide comparable description of the functional experiences of the individuals. Due to its vocabulary, which is easily interpretable also by nonclinical personnel, the classification can also be used in various health-related domains, such as rehabilitation [38] or reintegration of injured workers in workplaces [39]. The classification is organized in two main parts: the first, “Functioning and Disability,” provides a description of the components Body functions, Body structures, and Activities and participation; while the second, “Contextual Factors,” provides the means to describe the impact of the components Environmental factors and Personal factors. Each component is further deepened into chapters, which identify the addressed domain. The functioning of a person is then described through the interaction between his/her health condition and the context where he/she acts. Each component is identified by a letter (b for Body functions, s for Body structures, e for Environmental factors, and d for Activities and participation) and can be detailed by adding digits (Figure 1).
According to the number of digits following the letter, it is possible to get a code, whose length indicates the level of granularity—up to five digits. The functioning or disability of an individual can be assessed selecting the suitable category and its corresponding code and then adding a qualifier (from 0, meaning “absence of impairment,” to 4, indicating a “complete impairment”).
In addition to the characterization of the users performed using ICF, the knowledge about their physical status was deepened taking into account the cardiorespiratory fitness (CRF). As mentioned in Section 3.2, in fact, the general physical condition plays an important role in preventing the occurrence of several pathologies and, thus, must be taken in consideration not only when describing the user but also when designing the Smart Home, which should be built to take care of the users’ health and well-being too [40]. CRF expresses the physical fitness of a person, specifying his/her ability to carry out a dynamic, moderate-to-high intensity exercise over a prolonged period of time [41]. In addition, it has been demonstrated to be inversely proportional to the risk of mortality and of cardiovascular, pulmonary, and coronary diseases [42]. Therefore, CRF represents a suitable indicator of a subject general health status and can be used to monitor the progress or the regression of the health status of the Smart Home inhabitants through time [41].
Semantic Web technologies [43], in particular the modelling of domain knowledge into ontologies [44], were chosen to provide a set of formal and sharable descriptions of the concepts and their relationships composing the domains of the domestic environment, the user, and its appliances. Semantic Web technologies can indeed provide a description of the functioning of the appliances and their services, thus enhancing the semantic interoperability among them. Furthermore, with the adoption of Semantic Web Rule Language (SWRL) rules, it is possible to trigger specific inferences, such as the deployment of specific appliances and services to address the user’s particular needs (as further illustrated in Section 3). However, the interaction of the appliances distributed in a cooperative domestic environment cannot always be validated in a real environment—mainly because of the high costs and time to set the environment up. In order to ease an a-priori evaluation, as well as to validate the design of integrated appliance solutions providing AAL services to users and their services, the Smart Home Simulator (SHS) has been developed. The aim of this application is to allow designers of domestic environments to simulate and configure a house by taking advantage of a virtual representation of the house and the appliances. SHS also allows the designer to tailor the services offered by each appliance, leveraging the descriptions provided by Semantic Web technologies; in this way, the SHS allows the construction of a complete and clear picture of the status of the assisted users and their living environment before effectively hard-deploying the customized AAL solution. The result is a realistic simulation of a home where the designer can set up the appliances according to the changes occurring to the dwellers and their environment. In the SHS, the combination of Virtual Reality and Semantic Web generates an integrated and aggregated view on relevant knowledge related to the domestic environment. This coupling allows physical and virtual objects to coexist and interact in real time, thus enhancing appliances’ configuration operations also in real scenarios. Moreover, Semantic Web technologies can provide a formal method to represent and model the digital counterpart of the real physical world corresponding to the domestic environment.
3.5. The System’s Architecture
The SHS is an application relying on an integrated service-oriented platform described in [45]. The main purposes of this platform are to manage the knowledge about the home and user’s domains, while allowing the various appliances to exchange data among them. This framework is called Virtual Home Framework and is based on four main pillars:(i)The first is the semantic layer, named “Knowledge-Base Home” (KBHome), which is the set of ontologies describing the relevant knowledge of the abovementioned domains (the KBHome, further described in Section 3).(ii)The second pillar consists in the virtual representation of the domestic environment: this feature allows to virtually represent appliances and sensors (as described in Sections 4 and 5).(iii)Integration Services guarantee data integration synchronization between physical and virtual components of the system. This approach is delved in [46], and in this context, applications can easily interoperate while the data are integrated, shared, dispatched, and aggregated through mechanism transparent to their clients. Therefore, the Integration Services are able to promote the semantic integration among the data provided by the various domestic appliances and sensors (both real and virtual) and to contribute to enhance their near real-time synchronization:(1)They enable the acquisition of information from any device (appliance or sensor and real or virtual).(2)They allow to store, interpret, and manage the information received.(3)They share and dispatch information when a device asks for them or when a needed information becomes available.(iv)The Real Home, with its real appliances, represents the deployment of the solutions identified, thanks to the cooperation among the three pillars of the framework.
The SHS acts like a user-friendly, semantic, and virtual reality-based interface to allow designers to configure and test (with either virtual or real appliances) AAL solutions customized on specific users (Figure 2).
4. The Home Knowledge Base
The knowledge base containing the information of the user and his/her health and physical status and the living environments composing the house and the appliances deployed or deployable in it are modelled in a set of ontologies named “KBHome.” The ontologies are modelled following the NeOn Methodology [47], an ontology engineering methodology allowing to focus on the reuse of already existing resources—both ontological or not. The knowledge base has been developed using the software Protégé [48], while Resource Description Framework (RDF) [49] and Web Ontology Language (OWL) [50], with the use of Semantic Web Rule Language (SWRL) [51], were the selected implementation languages; Pellet [52] was the reasoner used to perform reasoning activities on the ontologies. KBHome is composed of four main ontologies, each addressing a specific domain.
4.1. The User’s Health Condition and Cardiorespiratory Fitness Ontology
This domain ontology describes concepts regarding the domain of the user, starting with his registry records (date and place of birth, current address, gender, and phone numbers). The most relevant feature of this ontology is the description of the user’s health condition using the codes and qualifiers included in the ICF. To this purpose, the ontology of ICF—publicly available on the BioPortal—was partially reengineered: the specific ICF codes, originally modelled as individuals, were converted into datatype properties, in order to make possible to model several health conditions using the same ICF code. Each user is linked to his/her health condition (modelled as an individual), which is described via the datatype properties composing the ICF (Figure 3).
Each health condition can then be classified as “Vision Impaired Health Condition,” “Motor Impaired Health Condition,” “Hearing Impaired Health Condition,” and “Cognitive Impaired Health Condition.” For each of these classes, there is a further subclassification regarding the grade of impairment associated, according to the qualifier assigned to each datatype property; for instance, the class “Vision Impaired Health Condition” can be deepened into “Mild Visual Impairment,” “Moderate Visual Impairment,” “Severe Visual Impairment,” and “Complete Visual Impairment.” According to the typology of impairment associated with his/her health condition, a user can then be inferred to belong to a specific class of users (“Vision Impaired User,” “Motor Impaired User,” “Cognitive Impaired User,” or “Hearing Impaired User”).
Following the same ontology design pattern, a user is linked to his/her CRF (indicated as ), which is assessed during a test under the supervision of clinical personnel. According to the value detected and stored in this ontology, the user’s CRF can be classified, thanks to reasoning processes into subclasses (“Seriously limited CRF,” “Reduced CRF,” “Adequate CRF,” “Good CRF,” “Excellent CRF,” and “More than excellent CRF,” according to the percentile [53] the user’s value fits).
User’s value can be used to assess the customized workload of an exercise (in this case performed on a cycle-ergometer), as described in the following equation [54]:
The angular coefficient m and the intercept q are user-dependent and are calculated from the interpolation of the data obtained during the test execution. A training intensity of 65% is chosen in agreement with ACMS, which identifies it as the minimum intensity capable of producing an increase in the user’s CRF [55]. Equation (1) can be easily translated into SWRL, thus determining the exercise load to be applied to the cycle-ergometer: Device(?cycle), User(Jane),TrainingIntensity (?trInt), hasTrIntValue (?trInt, ?TrIntValue), hasCardiorespFitness(Jane, ?cf), isCharacterizedByLinearRelationship(?cf, ?lr), hasMValue(?lr, ?mvalue), hasQValue(?lr, ?qvalue), hasVO2maxValue(?cf, ?VO2max), multiply(?step1, ?VO2max, ?TrIntValue), subtract(?step2, ?step1, ?qvalue), divide(?WLcycle, ?step2, ?mvalue), → setsWLonCycleErg(?cycle, ?WLcycle)
In this way, it is possible to represent relevant knowledge about the user and some features of his/her physiological status in a simple way, as summarized in Figure 4.
4.2. Appliances and Domestic Environment Ontology
This ontology aims at providing a description of the appliances and linking them to the room where they are deployed. Moreover, it allows to provide a list of the measurements each appliance can perform. The ontology is composed of three main modules: the first provides a simple representation of a generic domestic environment, modelled with classes and individuals. Each individual represents a room of the user’s house and is described with datatype properties illustrating the dimensions of each room.
The second module is the appliances module, which collects a list of appliances (both white and brown goods) and sensors and provides a description for each of them; this description is achieved taking advantage of the HicMO “grammar” [56], a set of XML properties able to describe the features of any appliance. According to the guidelines provided by NeOn methodology, HicMo properties were converted into datatype and object properties to create a semantic model of the XML descriptors. In this way, it is possible to provide a sort of “ID card” for each appliance deployed or deployable in the house. The description of a Smart Object is integrated with a submodule describing the list of programs available for each appliance; each program is described as an individual and is associated with one or more appliances.
Since each appliance can produce one or more measurements, the third module allows to provide a description of these measurements. The latter can be classified into “Environmental measurements,” “Vital sign,” and “Appliance measurements”; for instance, a digital thermometer located in the living room can detect the temperature of the room, while a thermometer located on the user measures his/her internal temperature. Measurements describing the user’s physiological status exploit a partially reengineered version of the Vital Sign Ontology [57], while measurements regarding appliances and domestic environments exploit the Units Ontology [58] to provide a sound description. Figure 5 depicts an example of a sensor performing a measurement of the user’s temperature.
4.3. Comfort Metrics and Domestic Environment Model
This ontology describes comfort dimensions inside the domestic environment, taking into account measurements modelled in classes such as “CO2Concentration,” “DomesticHumidityRate,” “DomesticLuminance,” and “DomesticTemperature.” In particular, the class “DomesticTemperature” is divided into “WinterDomesticTemperature” and “SummerDomesticTemperature.” Each of the classes is split into “Acceptable” and its complement “NotAcceptable.” Each individual “Environmental measurement” is then classified according to these classes. For instance, a measurement detecting a CO2 concentration equal to 1147 ppm is inferred to be a “NotAcceptableCO2Concentration,” since the “AcceptableCO2Concentration” requires environmental measurements with a value less than or equal to 1000 ppm.
The classes for the description of comfort dimensions convert the limits described into several standards (such as ASHRAE [59] for the thermal and humidity rate comfort, UNI-2004 [60] for the luminance recommendations, and UNI-2008 [61] for the air quality in domestic environments).
4.4. Orchestration of Services in the House
This ontology describes the events triggered by one or more of the conditions occurring in the environment or to the user. With the use of SWRL rules, it is possible to describe the conditions under which a specific action is activated. For instance, to set the proper air-conditioning system’s program during summer, the following conditions must hold: EnvironmentalMeasurement (?m), NotAcceptableSummerDomesticTemperature (?m), hasMeasurementValue (?m, ?value), greaterThanOrEqual (?value, 27), EnvironmentalMeasurement(?n), NotAcceptableDomesticHumidityRate (?n), hasMeasurementValue (?n, ?value2), greaterThanOrEqual (?value2, 60), Appliance (air_conditioning_system), AirConditioningSystemProgram (SummerBreeze) -> hasProgram (air_conditioning_system, SummerBreeze)
Several situations, similar to the one described above, were described in this ontology, providing the conditions under which the appliances are able to respond with the proper services. More complex situation involves customized services to be deployed for particular categories of impaired users.
4.5. Configuration of Living Environments
This ontology allows to classify the appliances (described as noted in Section 3.2) into other classes; according to their characteristics and the programs they have, appliances can provide useful services to the users, to help them cope with their impairments while performing activities of daily living. Therefore, the appliances of the house are classified also according to their suitability for specific categories of impaired users, using the classes “Cognitive Impairment Appliance,” “Hearing Impairment Appliance,” “Motor Impairment Appliance,” and “Visual Impairment Appliance.”
Taking advantage of these features, it is possible to model a configuration project. Each project is modelled as an individual with an ID number and allows to select appliances together with their programs. Furthermore, it is possible to specify the user for which the project is designed (Figure 6). This ontology also allows to automatically infer if an appliance is suitable for a particular category of impaired users according to the programs available for the application. In this way, a designer can choose from a list of appliances inferred to be adequate to cope with a user’s impairment, and the outputs coming from the reasoning process of this ontology are exploitable as a decision support system [62] for the design phase of the living environment.
5. Customization of the Smart Home Services
In order to design a Smart Home able to provide its dwellers with the right instruments and services that enhance their well-being and autonomy in ADL, the customization of the offered solution is necessary. This step passes through the collection of the subjects’ needs—through the evaluation of their health status—and the design of a customized solution able to help the users in coping with their impairments or to improve in some way their quality of life.
5.1. User Characterization
For the assessment of the health condition of the home dweller(s), the intervention of clinicians is required to fill in the ICF-based module. The clinical personnel have to assess all the user’s impairments through standard tests and clinical scales specifically dedicated to investigate a precise domain (vision, hearing [63], motor or cognitive deficits) or the general health status in elderlies [64, 65].
The evaluation of the CRF occurs in a second phase and can be performed in different ways. The gold standard methodology consists in the measurement of oxygen and carbon dioxide through spirometry during a physical exercise of incremental effort. This leads to the determination of the maximum oxygen uptakedefined as — which represents the ability of an individual to take up and use the inspired oxygen [22]. Alternative techniques foresee the indirect estimation of the CRF starting from the measurement of the subject’s heart rate (HR). These techniques are based on the assumption of a linear relationship between the user’s HR and the workload; moreover, they assume that all individuals of the same age have indeed the same maximal cardiac frequency. Though these hypotheses are usually true for healthy subjects, these assumptions often do not stand in the case of elderlies or people with disability [23]. Thus, the first methodology—based on direct measurements of expired gases—should always be preferred. Another important aspect to be taken into account is the safety of the tested individual during the CRF evaluation test. The American Thoracic Society identified the cycle ergometer as the equipment to be preferred with respect to the treadmill in case of patients, because it does not introduce risk of falls and allows the concurrent measurement of gases and work rate, in exchange of a reduced capability of reaching high levels of , which should not be an issue in case of frail individuals.
Having defined the equipment and other supplementary tools—a blood pulse oximeter, an electrocardiograph (ECG), and a blood pressure (BP) monitor at least—needed to ensure the safety of the subject during the test, the CRF of the Smart Home dweller can be identified following a multistage exercise protocol [22]. The chosen protocol foresees a preliminary warm-up phase with low and constant workload followed by an increase of 20 W in the workload after a steady state of 3 minutes. All the conditions triggering the increase in workload are reported in details in Table 1.
During the test, expert clinical personnel must continuously monitor the conditions of the individual performing the cardiopulmonary exercise: this is necessary since the final aim is to identify the subject’s maximal effort reaching his/her physiological limits. Therefore, if any of the conditions listed in Table 2 occurs, the exercise must be immediately terminated. Moreover, in case of exercise interruption, the subject must be assisted by a physician until normal values and a stable condition are recovered.
Having completed these two phases of evaluation, each subject is provided with a complete record of his/her health status—according to the worldwide standard of ICF—accurate information about his/her CRF, and thus his/her physical capabilities. From the latter, the customization of the daily physical exercise for each individual could be addressed (as described in Section 6.3). The cycle ergometer is maintained as the training equipment also in the Smart Home environment configuration because of its higher safety with respect to the treadmill, which is the only other fitness equipment able to provide a direct control on the workload. Of course, the use of a cycle ergometer is compatible with individuals who do not suffer from severe motor limitations or have impairments in the postural control. These severe situations are identified during the first clinical assessment, formalized through the ICF completion and should be addressed with customized solutions, as for instance, an arm ergometer.
5.2. Configuration and Test of the User of the Smart Home
Starting from the pieces of information gathered during the user’s evaluation phase, the Smart Home designer, being able to interpret these data, can set up a mixed reality environment able to respond to the user’s needs with the most appropriate solutions. In this context, using either mixed or virtual reality leads to several advantages; the first, as already mentioned, is the possibility to test the communication among the different appliances in real time. The second consists in the possibility for the final user to directly experience the VEs, with the double aim of becoming familiar with his/her newly configured Smart Home and of giving suggestions to the designer who can improve the final home design. Finally, the employment of virtual reality, coupled with the semantic models described in Section 3, allows the Smart Home designer/the final user to test the functioning of a sensor or an appliance without the need of owing it in real world, thus saving time and costs.
5.2.1. Customized Configuration of the Smart Home
In this phase, the designer receives the blueprint of the final users’ current house; he/she is thus able to model it using an authoring tool, reconstructing a digital model of the dwelling rooms. This model can then be imported inside the Smart Home Simulator, the PC-based application developed in the D4All project using Unity 3D. Within this application, the designer can add devices and appliances to the house digital model, choosing them among the ones modelled in the semantic “catalogue” described in Section 3.2. The catalogue is updated in real time inside the VE using SPARQL to query the semantic repository, in which the user’s needs and peculiarities were stored.
The designer has also the possibility to integrate real devices or sensors in the digital representation of the Smart Home, thanks to the architecture described in Section 3.5, which permits the data exchange between the real and the virtual world. When the design of the customized environments is completed and the communication among the devices, the sensors, and the human users is validated, the designer can save the project and store it for further modifications.
5.2.2. Testing the Designed Solution
At the end of the design phase, the Smart Home Simulator offers the possibility for the final user to test the solution specifically developed according to his/her characteristics and needs. The solution can be deployed on different platforms, depending either on the characteristics of the target user or on the type (virtual or mixed reality) of environment. Of course, immersive (e.g., head-mounted displays and CAVE) or semi-immersive (e.g., semicylindrical projected screens) experiences constitute the most promising means to validate different scenarios and to learn how the Smart Home services work, because of the higher sense of presence they elicit and the more natural interaction they often provide [66]. However, when choosing the VR technologies, particular attention must be paid on the target user: for frail elderlies or severe cognitive impaired subjects, the risks of adverse events and sickness while using head-mounted displays are not deniable [65]. Therefore, other solutions should be preferred, even in exchange of a reduced sense of presence. The use of nonimmersive environments should also be preferred in case of real devices or appliances included in the final solution, because—if not properly reconstructed in the virtual world—the interaction may become very complicated or even dangerous (i.e., the user can trip).
After the validation in the mixed reality environment, the designer is able to determine if the implemented solution is able to help the user in coping with his/her limitations during activities of daily living, having observed his/her behaviour during the simulation. Moreover, he/she can retrieve direct information from the final user, interviewing him/her and asking about the changes they would make in their future house. These modifications can be then implemented in the configurator and retested till the reaching of the optimal solution that the final user will implement at his/her own place.
6. Validation with Real Use Cases
The SHS of the adopted framework were validated through two real use cases. The first foresees the configuration of a kitchen for a visually impaired user, while the second addresses the problem of active ageing and foresees an elderly user performing domestic physical activity.
6.1. Configuration of a Real Environment: The Kitchen
This use case consists of a kitchen designer who has to design a kitchen using the SHS selecting the most suitable appliances for the final user: a person afflicted by a moderate visual impairment (specified with ICF codes: b21022.2—“moderate impairment in the contrast sensitivity”) and hyposmia (b255.2—“moderate impairment in the smell function”). The designer acquires the user’s kitchen blueprints and elaborates a virtual model of the kitchen. In this model, he/she places the most suitable appliances and sensors to cope with the user’s impairment(s) choosing them from a list, provided by the KBHome (as seen in Section 3.5).
For this user, the KBHome retrieves(i)two induction cooktops: one able to produce a high contrast on its surface (with a black panel) with textured button surfaces; the other providing a black surface with high-contrast controls and digital and backlit display;(ii)two models of convection ovens with digital and backlit display and a set of control lights;(iii)four models of dishwashers with high contrast and backlit digital display;(iv)two models of refrigerators with digital and backlit interface, illustrating the current inside temperature and internal light.
According to the user’s preferences, the designer chooses the appliances and sets them into the virtual model of the kitchen, as illustrated in Figures 7–10.
6.2. Validation of the Application
The validation of the configuration part is performed following the formative evaluation methodology, as described in [67, 68]. This type of validation foresees an observation and empirical evaluation of representative users’ interaction with the VEs in a task-based scenario: in this case, the design of the kitchen described in Section 6.1. The aims of this type of evaluation are identifying usability problems and assessing the user’s learning curve and his/her task performance, both with qualitative and quantitative results. To do this, five designers will be enrolled in the study; they will be presented the scenario and the Smart Home Simulator application during a first training phase of about 15 minutes. To ease and speed up their work, the digital model of the empty kitchen will also be provided as the starting point. During the configuration phase, tasks timing, errors, potential software bugs, and users’ comments will be registered. At the end of the trial, the System Usability Score (SUS) questionnaire [69] will be administered to each participant. Free comments will also be collected and taken into account to improve the software according to the final users’ (the designers) needs.
6.3. Configuration of Living Environment for Physical Activities
The first use case depicts a common situation for elderly people, where a 72-year-old frail woman has to perform daily the physical exercise on a cycle-ergometer placed in her bedroom. Her health condition is described by the following ICF codes: b7353.2 (“moderate impairment in the tone of muscle of lower half of the body”), b4550.2 (“moderate impairment in the general physical endurance”), and s75002.181 (“mild impairment to the muscle of the right tight (nature of the impairment not specified)), while her CRF was assessed at 18.3. Therefore, she has been prescribed an adequate physical activity, to be performed monitoring her results and conditions. The data deriving from her exercise sessions must be made available to caregivers or clinical personnel, in order to periodically assess enhancements or deteriorations of her abilities and conditions. The use case foresees that the user’s smart home is equipped with the cycle ergometer, a heart rate monitor, a breath rate monitor, and a blood pressure monitor. The user receives information about the performance by means of a tablet. To accomplish her daily physical activity, she is requested to enter the bedroom, where a presence sensor detects the user’s presence. She is then required to prepare the bedroom in order to perform the exercise, by ventilating the room and waiting until it reaches the proper temperature (which is set at 20°C). Once the air quality and the temperature reach the proper values, the user can wear the sensors and begin the exercise, whose workload is set basing on her specific health condition. While performing the activity, her physiological status is monitored. At the end of the physical exercise, the data regarding the exercise session and the physiological measurements detected are stored and remain available for the caregivers. All the instructions are given to the user via a virtual tablet, through which she can also check her performance in real time and potentially receive different types of alerts based on the sensors’ measurements.
The designer, using the SHS features, equips the environment with real sensors (heart rate monitor, breath rate monitor, and blood pressure meter) and provides the virtual representation of the living environment with an actuator to enable the automatic opening and closing of the window. The virtual environment also includes a presence sensor, able to detect the user’s presence in the room, an environmental thermometer, and an air-quality sensor, able to measure the CO2 concentration in the room. Data regarding the user’s health condition and CRF are stored in the KBHome.
By navigating the virtual scene, the user taps the virtual tablet on the exercise icon and receives the instructions about how to set the environment for her activity. At first, she is asked to enter the bedroom, where the cycle ergometer is located. The presence sensor is able to detect her presence and triggers the air quality sensor installed in the room. It measures the CO2 concentration in the room and stores the acquired data into the KBHome, where this piece of information is processed. Hence, having registered an unacceptable CO2 concentration in the room, the user receives an alert on the tablet warning her that, due to the current CO2 concentration, she is not allowed to perform the exercise: the user receives the suggestion to open the window. Then, she opens the window by tapping the proper option on the virtual tablet. When the air quality and the temperature reach a suitable level to perform the exercise, the user receives a notification on the tablet. Finally, she can begin the physical exercise, whose duration and level of difficulty are automatically selected by the KBHome based on her health condition and CRF assessment (as shown in Sections 4.1 and 5.1). In this case, the workload is calculated according to the equation presented in Section 4.1, and the duration of the exercise is set at 20 minutes (or user request). The user receives instructions on how and where to wear the sensors. While performing the exercise, the user’s blood pressure, breath rate, and pulse rate are monitored by these wearable devices, and, if any physiological anomaly arises, the user is warned to immediately modify or stop the exercise via a tablet alert. At the end of the exercise, physical and physiological data regarding the performance (blood pressure, pulse rate, and duration of the physical activity) are stored in the Semantic Repository, where they are at the caregivers’ disposal.
6.4. Development of the Validation on Elderly Subjects for the Configured Environment for Physical Activities
The validation on the elderlies follows the same methodology described for the kitchen design scenario. A small sample (n = 5) of target users will be enrolled in a first-pilot trial dedicated to the assessment of potential software or methodological issues in the developed scenario.
Enrolled subjects will have to fulfil the following inclusion criteria: age ≥ 65 years old, they should have a mild-to-moderate impairment in their general physical endurance, and they should be judged by a clinician as subjects who would benefit from a light daily physical exercise. Exclusion criteria are the presence of severe cognitive deficits or vision problems and the inability to express the informed consent. Each of the subjects, once enrolled, should undergo a CRF assessment in the clinical setting, as described in Section 5.1. After the completion of the test, his/her CRF value will be used to assess the target workload to be set at the beginning of the exercise.
The setup used for the validation of the scenario dedicated to elderlies tries to mimic as much as possible the situation described in Section 6.3. Thus, an entire room is completely dedicated to the recreation of the described scenario; it will be provided with a cycle ergometer (real), a pulse oximeter (real), a blood pressure meter (real), a tablet (real), a presence sensor (real), an automated window (virtual), an air quality sensor (virtual), and a thermometer (virtual). All the virtual objects will be presented to the target user using a wall projector.
Expert personnel will instruct each subject about the aim and the functionalities of the system, providing an overview of each component. As soon as they are confident with the setup, the subjects will be left free—under constant supervision by clinical and technical personnel—to interact with the system following the instructions given through the tablet. Each subject will perform the 20 minutes of cycling, after having opened and closed the (virtual) window, with the workload set according to his/her CRF. In order to guarantee, at each instant, the safety of the training, the monitored parameters must not exceed the values reported in Table 3.
The validation of this scenario will pass through the collection of objective quantitative data (errors, task timing, need of suggestions, and general performance) and the interview of the subjects using an ad hoc developed questionnaire. Due to the characteristics of the population, in fact, semistructured interviews were chosen as the preferred methodology to gather qualitative information assessing the acceptability and usability of the designed system [72].
7. Conclusion and Further Works
This paper presents the Smart Home Simulator, an AAL application that takes advantage of both Virtual Reality and Semantic Web technologies to tackle the configuration of domestic environments. The dwellers’ health conditions, periodically assessed by clinical personnel, are modelled into a semantic knowledge base (KBHome) that allows to automatically infer a set of appliances able to help the dwellers in performing several daily life activities autonomously. The ontologies also provide a formal description of the users’ health conditions and their cardiorespiratory fitness, environmental comfort metrics, and appliances and their behaviours enabling the possibility to provide tailored services to the dwellers. In fact, the results of reasoning process allow, from the one hand, to identify a set of appliances that can support the dwellers in daily life activities, thus helping them to cope with their impairments and to live in an autonomous way; on the other hand, KBHome can provide the dwellers with parameters to set up a daily physical activity. One of the most relevant aspects of the Smart Home Simulator relies on the exploitation of the International Classification of Functioning, Disability and Health, an international standard acting as a common language in health-related fields among clinical personnel and nonclinical professionals.
The results coming from the reasoning process are fed to a Virtual Reality-based application, able to provide a reproduction of a real domestic environment in which smart home designers or architects can select and deploy the appliances to study their behaviours and how they can cope with users’ impairments. This virtual configuration process has the advantages of being completely organized around user’s real needs and limits (modelled into the ontology with ICF) and to considerably reduce the costs and time to set up a smart home by testing its appliances (and their behaviours) in a virtual environment.
Two use cases were deployed to validate the Smart Home Simulator functionalities and usability and to test the possibility to configure a specific environment for performing physical exercise according to particular criteria.
In the next months, the validation phase described in Section 6 will take place involving different medical and industrial partners. The validation process includes experimental campaigns with patients (characterized by mild and moderate impairments), who will be asked to dive into the Virtual Environments and to perform the tasks described in the use case provided in Section 6.3. In this way, it will be possible to validate the methodological approach and the chosen technologies.
Another goal of the validation phase is to optimize the framework in order to ensure comfort levels to elderly people inside their homes and to extend the approach here described to the measurement and the assessment of domestic comfort indicators.
Further works foresee the development of other scenarios involving different kinds of users, characterized by diverse impairments and different domestic environments. The KBHome ontologies will be enriched to comprise a larger number of comfort metrics and to model rules to provide tailored comfort metrics to specific categories of dwellers. These developments are expected to be eased by the scalar architecture already implemented and will allow to provide suitable answers to various users’ needs, as foreseen in the “D4All” project.
Another improvement of the presented approach will also be addressed in the future. An attempt to partially automatize the process of information retrieval from the semantic models and to subsequently exploit those data with Machine Learning (ML) techniques will be performed. The use of ML techniques should be integrated into the semantic approach, allowing the modification of the knowledge base. In this case, some applications should be developed to ensure the proper modelling of the data acquired via ML techniques; otherwise, there exists the possibility to corrupt the original ontologies. Furthermore, using the ML approach, the supervision of an expert (e.g., a designer or a clinician), that today is strongly necessary, may be limited only to crucial tasks (e.g., decision-making on the users’ health). Indeed, the complete automation of the process still represents an open challenge, because of the sensitive domains that could be affected by wrong decisions taken by the ML algorithms. When dealing with (frail) human users and their health, in fact, no mistakes could be tolerated, and, since nowadays there is no guarantee that automatic learning always works perfectly, the supervision of a human expert is required for both ethics and legal reasons.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors would like to acknowledge the Progetto MIUR no. CTN01_00128_297089 “Design for All—Sw integration and advanced Human Machine Interfaces in design for Ambient Assisted Living.”