A Probabilistic Approach to Control of Complex Systems and Its Application to Real-Time Pricing
Control of complex systems is one of the fundamental problems in control theory. In this paper, a control method for complex systems modeled by a probabilistic Boolean network (PBN) is studied. A PBN is widely used as a model of complex systems such as gene regulatory networks. For a PBN, the structural control problem is newly formulated. In this problem, a discrete probability distribution appeared in a PBN is controlled by the continuous-valued input. For this problem, an approximate solution method using a matrix-based representation for a PBN is proposed. Then, the problem is approximated by a linear programming problem. Furthermore, the proposed method is applied to design of real-time pricing systems of electricity. Electricity conservation is achieved by appropriately determining the electricity price over time. The effectiveness of the proposed method is presented by a numerical example on real-time pricing systems.
Analysis and control of complex systems such as power systems and gene regulatory networks are one of the fundamental problems in control theory of large-scale systems. In order to deal with such complex systems, it is one of the appropriate methods to approximate a complex system by a discrete abstract model (see, e.g., ). On the other hand, human decision making is also complex and is modeled by a discrete model (see, e.g., ). Thus, in analysis and control of complex systems and those with human decision making, a discrete model plays an important role.
Several discrete models such as Petri nets, Bayesian networks, automata-based models, and Boolean networks have been proposed so far (see, e.g., ). In this paper, we focus on a Boolean network (BN) . In a BN, the state is given by a binary value ( or ), and the dynamics are expressed by a set of Boolean functions. Since Boolean functions are used, it is easy to understand the interaction between states. In the field of theoretical biology, there is a criticism that a BN is too simple as a model of gene regulatory networks (see, e.g., ), but a BN can be relatively applied to large-scale systems. In addition, since the behavior of complex systems is frequently stochastic by the effects of noise, it is appropriate that a Boolean function is randomly decided at each time among the candidates of Boolean functions. Thus, a probabilistic BN (PBN) has been proposed in . Furthermore, a context-sensitive PBN (CS-PBN) in which the deciding time is randomly selected has been proposed as a general form of PBNs [7, 8]. In this paper, we adopt a probabilistic Boolean network (PBN) as a mathematical model of complex systems.
For a given PBN, we consider the structural control problem (see, e.g., [9–11]). In this problem, a discrete probability distribution is controlled. For example, in , a discrete probability distribution at each time is selected among a given set. In this paper, we consider fine control of a discrete probability distribution by using the continuous-valued input. For a newly formulated problem, we propose an approximate solution method. First, a matrix-based representation of BNs proposed in  is extended to that of PBNs. Next, using the obtained representation, the original problem is approximated by a linear programming (LP) problem.
Furthermore, as one of the applications, we consider a design method of real-time pricing systems (see, e.g., [13–16]). A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In general, a real-time pricing system consists of one controller deciding the price at each time and multiple electric customers such as commercial facilities and homes. If electricity conservation is needed, then the price is set to a high value. Since the economic load becomes high, customers conserve electricity. Thus, electricity conservation is achieved. In the existing methods, the price at each time is given by a simple function with respect to power consumptions and voltage deviations and so on (see, e.g., ). To the best of our knowledge, decision making of customers has not been explicitly considered so far. In order to realize more precisely pricing, it is necessary to use a mathematical model of customers. Thus, decision making of customers is modeled by a PBN, and the problem of finding the price at each time is formulated as a structural control problem. The price corresponds to the continuous-valued input. By a numerical example, the effectiveness of the proposed method is presented.
The proposed framework provides us a basic method for control of complex systems using PBNs.
Notation. For the -dimensional vector and the index set , define . For two matrices and , let denote the Kronecker product of and . In addition, for vectors and the index set , define . For example, for two-dimensional vectors and , we can obtain where is the th element of . Finally, let denote the matrix whose elements are all one.
2. Probabilistic Boolean Network
First, we explain a (deterministic) Boolean network (BN). A BN is defined by where is the state, and is the discrete time. The set is a given index set, and the function is a given Boolean function consisting of logical operators such as AND (), OR (), and NOT (). If holds, then is uniquely determined as or .
Next, we explain a probabilistic Boolean network (PBN) (see  for further details). In a PBN, the candidates of are given, and for each , selecting one Boolean function is probabilistically independent at each time. Let denote the candidates of . The probability that is selected is defined by Then, the following relation must be satisfied. Probabilistic distributions are derived from experimental results. Finally, , are defined by We present a simple example.
Example 1. Consider the PBN in which Boolean functions and probabilities are given by where , , and hold, , , and hold, and we see that the relation (5) is satisfied. Next, consider the state trajectory. Then, for , we obtain In this example, the cardinality of the finite state set is given by , and we obtain the state transition diagram of Figure 1 by computing the transition from each state. In Figure 1, the number assigned to each node denotes , , (elements of the state), and the number assigned to each arc denotes the transition probability from some state to other state. Note here that, for simplicity, the state transition from only is illustrated in Figure 1.
3. Problem Formulation
In this section, we formulate the control problem studied in this paper. In the conventional control problem, the control input is added to a given Boolean function. For example, the control input is added as follows: , . In general, we assume that the value of the control input can be arbitrarily given. However, there is a possibility that there exists no control input satisfying this assumption. In control of gene regulatory networks, a structural control (or structural intervention) method for PBNs has been proposed so far (see, e.g., [9, 10]). For example, in , the discrete probabilistic distribution is switched at each time. In other words, the discrete probabilistic distribution is selected from the set of candidates. On the other hand, in complex systems such as gene regulatory networks, power systems, and social systems, it will be desirable to consider a weaker control method. Thus, in this paper, we consider fine control of probabilities in a discrete probabilistic distribution. This control method can be regarded as a kind of structural control methods.
In the structural control problem formulated here, we assume that the probability in (4) is given by where is the control input. The set expresses the input constraint, and are given in advance. The parameter expresses elasticity of the probability to the control input. Finding and is the important problem, and will be focused on in future efforts. Of course, we must find such that satisfies (5) and In addition, the dimension of the control input may be less than the dimension of the state.
Under the above preparation, we consider the following problem.
Problem 2. Suppose that for the PBN with (9), the lower and upper bounds of input constraints , , and the initial state are given. Then, find a control input sequence minimizing the cost function under the constraints (5) and (10), where , are weighting vectors whose element is a nonnegative real number, and denotes a conditional expected value.
The linear cost function (11) is appropriate from the following reason. For a binary variable , the relation holds. That is, in the cost function, the quadratic term such as is not necessary.
According to the result in , Problem 2 can be rewritten as a polynomial optimization problem. However, in the case of large-scale PBNs, it will be difficult to solve a polynomial optimization problem. In this paper, an approximate solution method for Problem 2 is proposed.
Hereafter, the condition in the conditional expected value is omitted.
4. Solution Method
In this section, we derive an approximate solution method for Problem 2. First, a matrix-based representation for PBNs is derived. The obtained representation is an extension of a matrix-based representation for BNs proposed in . Next, using the matrix-based representation, an approximate solution method for Problem 2 is derived.
4.1. Matrix-Based Representation for PBNs
As a preparation, the notation is defined. Binary variables and are introduced. If holds, then holds; otherwise holds. If holds, then holds; otherwise holds. Then, the equality is satisfied. Using and , consider transforming the BN (2) into a matrix-based representation.
First, we explain the outline of a matrix-based representation by using a simple example.
Example 3. Consider the following BN: where , , and . Then, we can obtain the truth table for each . See Tables 1 and 2. From these truth tables, we can obtain the following matrix-based representation: where each element of , is given by a binary value ( or ), and a sum of all elements in each column of is equal to .
Such a matrix-based representation has been proposed in also [18, 19]. However, in the representation proposed in [18, 19], matrices with the size of must be manipulated ( is the dimension of the state). In the matrix-based representation proposed in , matrices with the size of are manipulated for each . Thus, the proposed representation enables us to model a BN using matrices with the smaller size.
Consider a general case. Define Then, the matrix-based representation for is given by where and . The matrix can be derived from the following procedure.
Procedure for Deriving in (15)
Step 1. Derive a truth table for .
Step 2. Based on the obtained truth table, assign or for each element of .
Step 3. Express the assignment obtained in Step 2 by a row vector. Denote the obtained row vector by .
Step 4. Derive as Next, consider extending the matrix-based representation of BNs to that of PBNs. First, using a simple example, we explain the outline.
Example 4. Consider the PBN in Example 1. Using the matrix-based representation, the expected value of can be obtained as where the condition is omitted. In this representation, the matrices and correspond to the Boolean functions and , respectively. In a similar way, , , and correspond to the Boolean functions , , and , respectively.
In general, using the matrix-based representation, the expected value of can be obtained as where and . The matrix can be derived from the above procedure.
4.2. Reduction to a Linear Programming Problem
In a similar way to Problem 2, Problem 5 is rewritten as a polynomial optimization problem. In this paper, we focus on the structure of and derive the relaxed problem for Problem 5. The relaxed problem is reduced to a linear programming (LP) problem, which can be solved faster than a polynomial optimization problem.
First, we present an example.
Example 6. Consider the matrix-based representation obtained in Example 4. We remark that the discrete probabilistic distribution for each is independent. Then, in (17), we can obtain In a similar way, we can obtain In addition, holds. The obtained equalities are linear with respect to , , , and , and , , , and . Hence, these can be used as constraints in the relaxed problem.
Next, consider a general case. Define Then, (18) can be rewritten as where . The relation between and is given by For , , matrices , can be derived as Let and denote the th element of and , respectively. Then, we can obtain From , we can obtain In addition, we introduce the following constraints: Thus, we can obtain the following problem as a relaxed problem of Problem 5.
By a simple calculation, Problem 7 can be equivalently rewritten as the LP problem with , , and as decision variables. By solving Problem 7, we can evaluate the lower bound of the optimal value of the cost function in Problem 7. In this paper, only an approximate solution method is provided. However, since the control input can be obtained by solving an LP problem, Problem 7 can be solved fast.
Finally, according to the receding horizon policy (see, e.g., ), we present the procedure of model predictive control (MPC).
Procedure of MPC
Step 1. Set .
Step 2. Measure the state .
Step 3. Derive by solving Problem 7.
Step 4. Apply only to the system.
Step 5. Set , and return to Step 2.
5. Application to Design of Real-Time Pricing Systems
In this section, we consider a design method of real-time pricing systems as an application of structural control of PBNs. First, the outline of real-time pricing systems of electricity is explained. Next, the PBN-based model of real-time pricing systems is derived. Finally, a numerical example is presented.
Figure 2 shows an illustration of real-time pricing systems studied in this paper. This system consists of one controller and multiple electric customers such as commercial facilities and homes. For an electric customer, we suppose that each customer can monitor the status of electricity conservation of other customers. In other words, the status of some customer affects that of other customers. For example, in commercial facilities, we suppose that the status of rival commercial facilities can be checked by lighting, Blog, Twitter, and so on. Depending on power consumption, that is, the status of electricity conservation, the controller determines the price. If electricity conservation is needed, then the price is set to a high value. Since the economic load becomes high, customers conserve electricity. Thus, electricity conservation is achieved. The price does not depend on each customer and is uniquely determined.
Consider modeling the set of customers as a PBN. The number of customers is given by . We assume that the state of customer is binary and is denoted by . The state implies The binary value of is determined by power consumption of customer . Let , denote the set of customers, which affect customer . In addition, we assume that there exists one leader in the local area. The state of a leader is given by . Then, for customer , we consider the following PBN as one of the situations: where is the control input corresponding to the price. The Boolean functions and imply that customer forcibly conserves (or does not conserve) electricity. In these cases, time evolution of the state does not depend on the past state. The Boolean function implies that the state is not changed. The Boolean function implies that the state of customer is changed depending on the other customers. The Boolean function implies that the state of customer is changed depending on the leader. Thus, decision making of customers can be modeled by a PBN. The above Boolean functions are an example of models for decision making. Depending on real situations, we may use other Boolean functions.
For the PBN-based model obtained, we consider the following problem: find a time sequence of the price such that customers conserve electricity as much as possible. However, it is not desirable that the price is too high.The condition that customers conserve electricity as much as possible can be characterized by . In other words, power consumption is expressed by . Hence, this problem can be formulated as Problem 2 by appropriately setting the weights and .
5.3. Numerical Example
We present a numerical example. Parameters in the system are given as follows: , , , , , , , , , , , , , , , , and . We remark that under the input constraint , (5), and (10) hold. The Boolean function is given by Parameters in Problem 2 are given as follows: , , , and .
Next, we present the computation result. Here, Problem 7 was solved once. Figure 3 shows trajectories of . Figure 4 shows trajectories of the control input (the price). From these figures, we see that becomes small by fine adjustment of the control input. In this example, the expected value of each state converges to .
In addition, when the obtained control input is applied to the system, the value of the cost function in Problem 2 was . In the case of (i.e., the constant input), the value of the cost function in Problem 2 was . In the case of , the value of the cost function in Problem 2 was . From these values, we see that the obtained control input is more effective than trivial control inputs. Furthermore, in order to verify the optimality, consider the case of . The optimal control input was derived as and by solving the polynomial programming problem, which is equivalent to Problem 2. On the other hand, the control input obtained by solving Problem 7 was and . Thus, there is a possibility that the control input obtained by solving Problem 7 is not optimal for Problem 2.
Finally, we discuss the computation time for solving Problem 7. The computation time was 0.6 sec for and 0.03 sec for , where we used IBM ILOG CPLEX 11.0 as the LP solver. The computation time for solving the polynomial optimization problem for was 232.2 sec, where we used SparsePOP  and MATLAB 32-bit version. In the case of , owing to memory warning, the polynomial optimization problem cannot be solved. Thus, although Problem 7 is an approximation of the original problem, Problem 7 can be solved fast.
In this paper, we studied control of complex systems modeled by a probabilistic Boolean network (PBN). First, the structural control problem for a PBN was newly formulated. Next, an approximate solution method was proposed based on a matrix-based representation of Boolean functions. Finally, as an application, we considered design of real-time pricing systems of electricity. The proposed method provides us a new control method for complex systems.
There are several open problems. It is significant to consider a method for evaluating the accuracy of an approximation from the theoretical viewpoint. It is also significant to develop an identification method of Boolean functions and parameters , in (9). Finally, it will be one of the interesting topics to apply the proposed method to several classes of PBNs, for example, a large-scale PBN with scale-free structure.
Conflict of Interests
The authors declare that they have no conflict of interests.
This research was partly supported by JST, CREST.
M. Adomi, Y. Shikauchi, and S. Ishii, “Hidden Markov model for human decision process in a partially observable environment,” in Proceedings of the 20th International Conference on Artificial Neural Networks, vol. 6353 of Lecture Notes in Computer Science, pp. 94–103, 2010.View at: Google Scholar
K. Kobayashi and K. Hiraishi, “Design of boolean networks based on prescribed singleton attractors,” in Proceedings of the European Control Conference, pp. 1504–1509, 2014.View at: Google Scholar
S. Borenstein, M. Jaske, and A. Rosenfeld, Dynamic Pricing, Advanced Metering, and Demand Response in Electricity Markets, Center for the Study of Energy Markets, University of California, Berkeley, Calif, USA, 2002.
A.-H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, and R. Schober, “Optimal and autonomous incentive-based energy consumption scheduling algorithm for smart grid,” in Proceedings of the Innovative Smart Grid Technologies (ISGT '10), pp. 1–10, Gaithersburg, Md, USA, January 2010.View at: Publisher Site | Google Scholar
C. Vivekananthan, Y. Mishra, and G. Ledwich, “A novel real time pricing scheme for demand response in residential distribution systems,” in Proceedings of the 39th Annual Conference of the IEEE Industrial Electronics Society (IECON '13), pp. 1956–1961, Vienna, Austria, November 2013.View at: Publisher Site | Google Scholar
E. F. Camacho and C. B. Alba, Model Predictive Control, Springer, 2nd edition, 2007.