Journal of Control Science and Engineering

Volume 2017 (2017), Article ID 3035892, 18 pages

https://doi.org/10.1155/2017/3035892

## Online Model Learning of Buildings Using Stochastic Hybrid Systems Based on Gaussian Processes

Department of Electrical Engineering and Computer Science, Institute for Software Integrated Systems, Vanderbilt University, Nashville, TN 37235, USA

Correspondence should be addressed to Hamzah Abdel-Aziz

Received 17 March 2017; Revised 12 June 2017; Accepted 2 July 2017; Published 23 August 2017

Academic Editor: Tushar Jain

Copyright © 2017 Hamzah Abdel-Aziz and Xenofon Koutsoukos. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Dynamical models are essential for model-based control methodologies which allow smart buildings to operate autonomously in an energy and cost efficient manner. However, buildings have complex thermal dynamics which are affected externally by the environment and internally by thermal loads such as equipment and occupancy. Moreover, the physical parameters of buildings may change over time as the buildings age or due to changes in the buildings’ configuration or structure. In this paper, we introduce an online model learning methodology to identify a nonparametric dynamical model for buildings when the thermal load is latent (i.e., the thermal load cannot be measured). The proposed model is based on stochastic hybrid systems, where the discrete state describes the level of the thermal load and the continuous dynamics represented by Gaussian processes describe the thermal dynamics of the air temperature. We demonstrate the evaluation of the proposed model using two-zone and five-zone buildings. The data for both experiments are generated using the EnergyPlus software. Experimental results show that the proposed model estimates the thermal load level correctly and predicts the thermal behavior with good performance.

#### 1. Introduction

Heating, ventilation, and air conditioning (HVAC) in buildings is a major source of energy consumption. Annual reports show that it is the highest cause of energy consumption in residential buildings. Moreover, HVAC along with miscellaneous electric loads accounts for the highest two energy consumption sources in commercial buildings [1]. Therefore, there is a high demand for developing advanced HVAC control methodologies that can reduce the energy consumption of HVAC units without compromising users’ comfort. Many of these advanced control methodologies are model-based and rely on accurate thermal models [2].

Developing accurate dynamical models is a challenging task because of the complex behavior of buildings. Thermal dynamics of buildings are stochastic nonlinear dynamics which are affected externally by the environment (e.g., ambient temperature) and internally by interactions between adjacent thermal zones. Additionally, the thermal dynamics of buildings behave differently based on the thermal load (e.g., occupancy) which may not be measured. Developing a unified parametric model for buildings is usually not practical because thermal dynamics differ from one building to another since each building has different construction materials, size, layout, and location. Moreover, the model parameters may change over time as the buildings age or due to changes in the configuration or structure. Such challenges can be addressed using nonparametric modeling approaches based on online model learning.

In this paper, we introduce a novel modeling framework to learn multizone data-driven dynamical models for buildings in an online fashion. The nonparametric nature of the model supports creating unified data-driven models for buildings. Furthermore, we do not assume that the thermal load can be measured, and we estimate it as a latent discrete variable. The proposed approach incorporates the effects of the thermal load, and as a result it increases the model accuracy. Additionally, the online learning adapts the model to the time variability during the system operation. We also include the ambient temperature prediction to incorporate the environment effects in the model.

The main contributions of this work are as follows:(1)We introduce a nonparametric SHS model based on Gaussian processes which can be used to model complex coupled discrete and continuous dynamics required to develop data-driven models of smart buildings.(2)We develop an online clustering-based learning methodology for SHS when the discrete dynamics cannot be measured. In the proposed methodology, we use the -means clustering algorithm to identify the discrete states of the system and to segment the data into each corresponding discrete state. Then, each segment is used to build a distinct GP model to abstract the continuous dynamics. Learning these distinct models is more efficient and mitigates the computation limitations of using a single GP model with every variable as inputs.(3)We utilize the data-driven modeling approach to provide an accurate thermal model for multizone buildings when the buildings’ thermal load is latent. We evaluate the performance, and the efficiency of the proposed model is evaluated using datasets of two-zone and five-zone buildings. In both experiments, we show the model accuracy of estimating the level of the thermal load and predicting the zone air temperature.

The organization of this paper is as follows. Section 2 presents the related work of modeling thermal dynamics and model learning of dynamical systems using GPs. Section 3 illustrates a brief theoretical background of GPs. The proposed model is defined in Section 4 along with the model learning problem. The proposed model learning approach is presented in Section 5. Finally, the experimental results are discussed in Section 6 with the evaluation.

#### 2. Related Work

##### 2.1. Thermal Modeling of Buildings

Developing thermal models for buildings has been investigated extensively in the literature. Many studies proposed linear models for single-zone buildings [3, 4], while other studies use lumped thermal models to approximate multizone buildings as a single-zone model [5, 6]. Although these methods are useful, they typically result in very simple models. On the other hand, multizone models have been investigated to identify the thermal dynamics of each zone in multizone buildings [7–9]. These models are more accurate and they can be used for advanced control design (e.g., model predictive control [8]).

Most of the above-mentioned work relies on parametric models to represent the thermal behavior of buildings. However, parametric models require the building’s detailed structure and/or the thermal equations to be known a priori. Also, they typically use linear dynamics and are not very accurate. Recently, nonparametric models have drawn attention to construct accurate nonlinear thermal models. A nonparametric thermal model based on recurrent neural network (RNN) architecture has been developed to learn a compact thermal model for a single thermal zone [10]. Models based on RNNs are very useful to represent the nonlinearity of thermal dynamics; however, they typically do not consider the stochasticity in the thermal dynamics. Moreover, these models require a large set of the training data to learn the model with the desired quality. To mitigate these limitations, a nonparametric probabilistic approach based on Gaussian processes (GPs) has been developed to learn thermal models in an online adaptive learning framework [11]. Both the RNN and the GP models assume that the thermal load of buildings (e.g., occupancy, heating rate from light and equipment) can be measured and they used it as a model input. However, this assumption may not be valid in many real scenarios. Another modeling approach is developed to learn the effect of the thermal load when it cannot be measured [12]. This model is based on using a gray box parametric model for the thermal dynamics and combining it with a latent force model based on Gaussian processes. The latent force model improves the parametric model accuracy by learning the dynamical effects of the latent thermal load.

Existing nonparametric modeling approaches for buildings assume that the thermal load can be measured and used as a model input. This assumption may limit the use of these approaches in many systems; therefore, we propose a nonparametric SHS model for multizone buildings when the thermal load is latent. The proposed model estimates the level of the thermal load and learns a distinct model for each level in order to improve the model efficiency and accuracy.

##### 2.2. Model Learning of Dynamical Systems Using GPs

In this paper, the proposed modeling approach is based on Gaussian processes [13]. Gaussian processes have shown a great success in modeling many modern systems because of their attractive features [14]. GPs are nonparametric probabilistic models which can express the uncertainty in the system dynamics and the model confidence through the model predictive variance. Furthermore, models based on GPs are very simple since they have few hyperparameters and relatively require small datasets to learn the model. These features allow GPs to build efficient, robust, and adaptive dynamical models. GPs have been used recently to develop many time-series models for short-term and multistep forecasting [15, 16]. For instance, GPs are used to build forecast models for the ambient temperature and carbon intensity. These models are used to develop an adaptive heating control algorithm that minimizes the energy cost and carbon emissions [4]. Also, GPs are used to develop a load forecasting model for power systems in [17] and a short-term traffic volume prediction model with a high performance [18]. In addition to time-series models, GPs are used to identify state-space models with a control input in order to support robust filtering, smoothing, and prediction of the system state [19]. For instance, in robotics application, GPs have been used to build a model-based data-efficient learning framework for control policy search, known as PILCO. In this framework, the robot can learn the system dynamics and the control policy efficiently in an online fashion [20].

Gaussian processes have been used widely to model stochastic continuous systems. However, they typically do not consider systems with coupled discrete/continuous dynamics (e.g., SHS). In this paper, we utilize GP models to represent the continuous dynamics of SHS. Furthermore, we consider systems with latent discrete dynamics. This allows us to efficiently integrate the state-space modeling techniques for stochastic continuous systems into the proposed data-driven SHS modeling framework.

##### 2.3. System Identification of SHS

Stochastic hybrid systems (SHS) are dynamical systems that integrate continuous and discrete dynamics. Moreover, the continuous and/or the discrete dynamics exhibit a probabilistic behavior [21]. System identification of hybrid systems (HS) has been investigated in the literature substantially to develop system identification methods for various classes of HS. Typically, these methods aim to estimate the system model parameters for a given model complexity [22]. For HS with unknown model complexity, a kernel-based approach is developed to identify a popular class of HS, known as piecewise affine systems, where GPs are used to model the impulse response of each submodel of the HS [23].

System identification of SHS has an additional level of complexity because of the presence of uncertainty in the model behavior along with the coupled continuous/discrete dynamics. For a given model structure, many methods have been developed to learn the model parameters (i.e., parameter identification) such as simulation-based approaches. Simulation-based approaches use the simulated trajectories to identify the model parameters based on randomized optimization techniques such as Markov chain Monte Carlo (MCMC) [24]. Most of these approaches assume a known model structure with unknown parameters. However, finding the model structure is very crucial in many complex systems. Alternatively, online model learning with reachability analysis framework based on Gaussian processes is developed to learn nonparametric models for SHS with deterministic and known discrete dynamics [25].

Recent nonparametric models for SHS are either limited to deterministic discrete dynamics or developed for offline learning. In this paper, we propose a nonparametric clustering-based modeling framework based on GPs which utilizes sensory data to learn SHS in an online fashion.

#### 3. Background

##### 3.1. Gaussian Process Model

Gaussian processes (GPs) are nonparametric probabilistic models that require only high-level knowledge about the system behavior and use the observed data to model the behavior of the underlying system [13]. Generally, GPs build a Gaussian distribution over functions, by which a function index variable is mapped to an infinite-dimensional function space. GPs are identified by mean and covariance functions. The mean function represents the expected value before observing any data and the covariance function (also called kernel) identifies the expected correlation between the observed data. For instance, the mean function and the covariance function are defined as The function modeled by the GP can be written as

We typically use a zero mean function for simplicity and squared exponential (SE) covariance kernel for its expressiveness combined with a noise kernel. Therefore, the mean and covariance functions can be expressed aswhere is the kernel signal variance, is the characteristic length-scales matrix, is the Kronecker delta, and is the noise variance. The above GP model builds a probability distribution over the functions by mapping -samples of a continuous variable to a vector of random variable with a Gaussian joint distribution, such that where is nD-by-nD covariance matrix generated by (3).

We are interested in the GP posterior distribution given some test inputs and observations (training data). We define the set of test inputs where we want to predict the function value as . After observing data and according to (4), the joint distribution of the known and the unknown function values is Therefore, the posterior distribution is also a conditional Gaussian distribution with a mean and a covariance given by where , , , and .

The hyperparameters of the above GP model are defined as . The training data () are used to identify the model hyperparameters which best represent the training data. The learning process can be expressed as an optimization problem, where the optimal hyperparameters () maximize the marginal likelihood such that where the marginal likelihood is given by Optimization algorithms based on conjugate gradients have been developed to optimize the hyperparameters [13, 26]. Also, the popular quasi-Newton optimization method for nonlinear functions has been used to learn the GPs’ hyperparameters effectively.

#### 4. Stochastic Hybrid Systems

We introduce in this section a nonparametric stochastic hybrid systems (SHS) model based on GPs. To formalize the SHS model, we define as the set of discrete states and denote the continuous state space by for each discrete state with dimension . The hybrid state space is defined as . For each discrete mode , its corresponding continuous dynamics evolve according to a stochastic process modeled by a GP model. The discrete state may also change based on a stochastic process. Furthermore, we consider systems with two types of inputs: () control input and () external uncontrolled input (disturbance) from the environment. The control input usually affects the system dynamics based on a control policy () which maps the hybrid state space () into the control input space (). On the other hand, the external uncontrolled input () affects the system dynamics and represents the interaction with the environment. Therefore, we propose to model the external input as a time-series disturbance model (). The model is formalized by the following definition.

*Definition 1 (nonparametric SHS). *A nonparametric SHS is defined as a tuple :(i), for some , represents the discrete state space.(ii) is a set of continuous variables in the Euclidean space .(iii)Init: is an initial probability measure on the Borel space where .(iv), for some , represents the control input space.(v), for some , represents the external uncontrolled input space.(vi) assigns to each discrete state a random function modeled by a GP which represents the evolution of the continuous state given the predecessor continuous state , a control input , and an external uncontrolled input .

A graphical representation of the model is shown in Figure 1.