Abstract
The model checking of Infinite-State Continuous Time Markov Chains will inevitably encounter the state explosion problem when constructing the CTMCs model; our method is to get a truncated model of the infinite one; to get a sufficient truncated model to meet the model checking of Continuous Stochastic Logic based system properties, we propose a multistep extending advanced truncation method towards model construction of CTMCs and implement it in the INFAMY model checker; the experiment results show that our method is effective.
1. Introduction
Continuous Time Markov Chains (CTMCs) have been used in various areas of research as a formalism; so far, the model checking of CTMCs has been a hot research topic in computer science research communities. Some algorithms and implementations have been shown in several papers and tools [1–7]. However, our research aims to the model checking problem of the Infinite-State CTMCs, which means that the states of the CTMCs in our interest can be infinite. The papers [8–11] and the tool INFAMY [7] are with the same interest, which are also our research basis. Due to the explosion of states of CTMCs, our approach is based on a truncated CTMC model, which is determined by the exploration on the fly [9, 10], which means that the depth of model is computed dynamically by the exploration of states.
The truncation process is involved with reachability analysis [9, 10]; that is, the transient probability is computed with the exploration of the model. The transient probability is carried out by uniformization method [10]. Together with the transient analysis when constructing the model, computation is very heavy, so to get a sufficient truncated model to meet the requirement of related Continuous Stochastic Logic (CSL) property or certain precision as fast as possible is our destination. We introduce a multistep extending advanced truncation method to meet this end, and the experimental results show our method is effective.
The main contents of the paper are organized as follows. Section 2 introduces the truncation based reachability analysis; Section 3 introduces an advanced truncation algorithm and do experiments. Section 4 proposes some multistep extending solutions and experiments. Section 5 talks about our result and next job.
2. Truncation Based Reachability Analysis
2.1. Finite Truncations
Let be a CTMC, where is a countable set of states, is the rate matrix, and is a labeling function.
First, we introduce some paths and probabilistic measures from [9]. A (timed) infinite path is an infinite sequence satisfying , and for all . For the path and , let denote the th state, and let denote the time spent in . For , let denote such that is the smallest index with . For , let denote the set of all paths, and let denote the set of all paths starting from . For state , a probability measure, denoted by , on the set can be defined. A finite path is a finite sequence for satisfying and for . Let denote the length of the path, denote the first state, and denote the last state of the path. Let denote the set of all finite paths. We omit the superscript if it is clear from context.
Next, we introduce the notion of depth. Let be a finite subset of with depth ; that is, for . For now, one may think of as being equal to the support of the initial distribution . However, can be an arbitrary finite set. This will allow us not only to deal with the initial distribution, but also to compute truncation depths for nested CSL formulas. The depth of state corresponds to the minimal distance from the set .
Definition 1. For and , the depth function is defined by .
Observe that for all . The subscript is omitted if is clear from the context. Intuitively, corresponds to the minimal length of any finite path starting from and ending in .
We consider a partition of the state space , where is the set of states with depth . We say that the set is the layer with depth and call its elements layer- states. Assume that is a special state not in and furthermore is also an atomic proposition not in . For , let denote the set of states with depth greater than ; that is, .
Further, we write . We define the -truncated CTMC as follows.
Definition 2. Let be a CTMC and let be the layer-zero states. For , we define the -truncated CTMC of by , where the states are . The labeling function is defined by if , and . The rate matrix is defined by the following: equals if , equals if = , and equals otherwise.
The -truncation of an infinite CTMC is illustrated in Figure 1. Intuitively, the transition matrix is restricted to the truncated state space , and is the distinguished absorbing state, which, by construction, is only reachable from states with depth . In state only the atomic proposition holds, which indicates that the system is in state . Since we consider finitely branching CTMCs, not surprisingly, the -truncated CTMC is always finite. The absorbing state has been introduced to abstract . We assume that for any state formula . We consider the probability of reaching the absorbing state in the -truncated CTMC , that is, . For mere notational convenience, we extend to states of with depth higher than for all with . For a fixed , we define the forward rate of a state within . For , it is the sum of the rates that go into the next layer , and, for , it is the sum of the rates entering states in .
2.2. The Logic CSL
The logic we consider is CSL without steady-state operator and unbounded until operator [10]. Let be an interval with and . Let and . The syntax of state formula and path formulas is
The semantics of the state formulas and path formula are precisely defined in [10]. In this paper, the state formula like the kind of will be more focused on; it means that starting some state, the probability of the system that satisfies a path formula would meet the rational relation or not, so the first solution to this problem is to compute this probability, so this is reduced to transient analysis, which is our main interest, to improve the efficiency of transient analysis for the whole model while extending the border states layer by layer.
2.3. Transient Analysis
Transient analysis means that, starting at state , the transient probability vector at time , it can be denoted as . If , we have if and 0 otherwise. The uniformization method based solution of was given at [10]. Let and , where is the uniformization rate satisfying and denotes the th Poisson probability with parameter and vector is transient probability of the uniformized DTMC at step .
The truncated model in our method is given by dynamically exploring from the initial state(s); once we want to add a layer on it, we need to compute the reachability probability from the initial(s) to the current layer states which we want to add. As shown in Figure 1, is the initial states set, and each of is explored dynamically, so we get a -depth truncated model of the infinite one. is the absorbing state, which is the border states set, for which the sum of the reachability probabilities from the initial state(s) is less than ; is the precision of the result, which can be , , or , and so forth, which can be set under INFAMY model checker as a circumstance variable for some certain need.
The reachability probability is carried out by the uniformization method to the CTMC, which is a relatively fast method to get the transient probability at a certain bound time at some state, which is denoted as . However, if the state explosion situation is very serious, the time to construct the model layer by layer would be of much cost. So we introduce an advanced truncation algorithm to explore the states, this method can improve the efficiency of the model constructing and model checking, and then at Section 4, we further propose some multistep extending solutions, which are implemented based on INFAMY model checker, the experiment results show that these solutions can help to improve the efficiency.
3. An Advanced Truncation Algorithm and Experiments
The truncation process is implemented by extending the states layer by layer from the initial states, and all the new states need to be transient analyzed and then to be extended further no matter how small the probability is. For that the precision of the result is under some certain value, so some states with relatively nearly no contribution to the result can be omitted when extending; thus, an advanced truncation algorithm is introduced. It is different from the finite state projection (FSP) [9, 10] and layered chain and uniform chain method [9, 10]. The algorithm is shown in Algorithm 1.
| ||||||||||||||||||||||||||||||||||
The algorithm aims to stop the extending of less important (small probability) states and proceed to the extending of much important (large probability) states; the less or more is determined by the state reduction policy; as shown in Figure 2, line 7, and . The policy means that the states in the border states set, which has been sorted upward, , will be excluded from the extending states set, for the sum of them is just exactly less than the precision. So with this policy, the number of states to be transient analyzed will be smaller to the FSP and forward-layered based model.
We consider the dependability of a fault-tolerant workstation cluster which is directly taken from case studies of [7]. Figure 2 depicts a dependable cluster of workstations. The cluster consists of two subclusters, which, in turn, contain workstations connected via a central switch. The two switches are connected via a backbone. Each component of the system can break down and is then fixed by a single repair unit responsible for the entire system. Hereby, the quality of service (QoS) constraint minimum requires at least workstations to be operational, where . Workstations have to be connected via switches. If in each subcluster the number of operational workstations is smaller than , the backbone is required to be operational to provide the required service. We consider the property.
. This probability means that the QoS drops below minimum quality within time unit.
For the property, we compare PRISM [6], FSP method and layered method of INFAMY. The results are given in Tables 1, 2, and 3. Because the resulting probabilities are very small in some cases, we use a precision of here, for the computation of the truncation point. Results for INFAMY are given for the layered chain, FSP, and advanced configurations, respectively. The uniform chain configuration is omitted, as it is always dominated by the layered chain configuration. PRISM implements three different engines: a sparse-matrix and two symbolic engines. We used the sparse-matrix engine as it was the fastest one. The results are shown in Tables 1, 2, and 3.
The experiment conditions are shown as follows.
Host Machine: it includes Mac Book, OS: Mac OS X 10.6.8, Processor: 2.2 GHz Intel Core 2 Duo T7500, and Storage: 2 GB 667 MHz DDR3 SDRAM.
Guest Machine: it includes Virtual Machine Software: VirtualBox for mac; Virtual OS: Linux ubuntu12.04 LTS 32Bit; Processor: 2.2 GHz Intel Core 2 Duo T7500; Storage: 512 MB
From Tables 1, 2, and 3, we can see that, for , FSP based INFAMY is faster, but for , , INFAMY model checker needs more time; this is because the transient analysis when constructing the model needs more computing. This is also the result of [11]. For the advanced method of the context, as in Tables 1, 2, and 3, comparing advanced with FSP, we can get that, under advanced based method, the depth of model is much deeper, and the states number is smaller; for the time costing, for , , the costing is less reduced, but for , the costing is greater; this result is reasonable, for the exploration policy is essentially undeterministically efficient for different models; for the current case, when , the current states number is very large, even with the reduction policy, with no contribution for it any more. We need to take other techniques to tackle this situation. Thus, we propose a multistep extending solution. See Section 4.
4. Multistep Extending Solutions and Experiments
The mutistep extending solution aims to reduce the extending of less important (small probability) states and enhance the extending of more important (large probability) states; for the latter states, we can, for example, extend two or more steps per extending, and for the former states, we can, for example, extend one step per extending, as we know that transient probabilities will be computed once again before the states were added to the border states, so, if we extend two or more steps, transient analysis at the intermediate states will be omitted; thus, time on the model construction will be reduced; then we can make the model much faster to converge to the absorbing state.
We continue with the upper case study. As shown in Algorithm 1, Line 11, is to extend the model from set and get the border states set . This means that the advanced method in the upper section is extending one step per extending. Now, we design some multistep extending solutions:(1)two-step extending solution, that is, ;(2)three-step extending solution, that is, ;(3)synthesis solution 1: a synthesis extending solution separates set to three parts, as set is a sorted set, which is sorted upward by the probabilities, so we can separate as , , and , and the sizes of each part are the same. Then we can get(i);(ii);(iii);(iv);(4)synthesis solution 2 separates set to two parts, so we can separate to and , and the sizes of the two parts are the same. Then we can get(i);(ii);(iii).
The experiments data are shown in Table 4. From the data in Table 4, we can see that under different extending solutions, the times needed are different, synthesis solution 2 performs better, when , and the time (including model constructing and model checking) has been reduced to 1000 s. This means that for this case, the step per extending should be relatively small; for synthesis solution 2, half small states extend one step per-extending, and other half large states extend two steps per extending. And for this, we continue to propose a solution to revise synthesis solution 2, named Solution D the policy is as follows: For the states in set , the gap between the largest and smallest probabilities: . We can separate set to two parts and , such that.(i) and ;(ii) . And then we can get that(i) ;(ii) ;(iii) .
The experiment results are as shown in Table 4. From Table 4, we can see that current solution D performs better on this case.
5. Conclusion
The multistep extending advanced truncation method can improve the efficiency of model construction of Infinite-State CTMCs; this is because the transient probabilities of states which have been jumped have not been computed, so to some extent this method is effective; however, which solution performs better needs to be experimented; there is no general solution that fits well for all cases. The efficiency is determined by the iterations when computing the transient probability. Less iteration is more efficient. However, this approach is essentially a linear approach to improve the efficiency; when the outsider state gets explosion, this approach will be less effective; just as in our case study, this approach can be used effectively to improve the model checking efficiency at a relatively small time bound . For future work, we need to consider other techniques to tackle the state explosion problem on model checking of CTMCs. And other works like [12–15] can also be considered.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to thank Lijun Zhang and Ernst Moritz Hahn for their original work on INFAMY model checker and the free contribution of the tool codes; Definitions 1 and 2 are directly taken from [10]. This paper is partially funded by the Communication University of China, 2013 Planned Research Program, and Research Funding Foundation of School of Computer Science, and the scholarship from China Scholarship Council for the Excellent University Teachers Study Abroad Program and supported by the National Natural Science Foundation of China (61103199), Beijing Municipal Natural Science Foundation (4112052); Supported by Engineering Program Project of CUC (3132013XNG1326, 3132013XNG1321), and the National Key Science and Technology Pillar Programs of China (2012BAH51F02, 2012BAH38F05, and 2013BAH66F02).