Abstract
Obstacle avoidance is a key performance of mobile robots. However, its experimental verification is rather difficult, due to the probabilistic behaviors of both the robots and the obstacles. This paper presents the Markov Decision Process based probabilistic formal models for three obstacle-avoidance strategies of a mobile robot in an uncertain dynamic environment. The models are employed to make analyses in PRISM, and the correctness of the analysis results is verified by MATLAB simulations. Finally, the minimum time and the energy consumption are determined by further analyses in PRISM, which prove to be useful in finding the optimal strategy. The present work provides a foundation for the probabilistic formal verification of more complicated obstacle-avoidance strategies.
1. Introduction
Obstacle avoidance is a key technique for the design and applications of mobile robots, because static barriers and even dynamic obstacles frequently exist in their paths. When several robots move in the same region, they act in fact as obstacles to one another themselves [1, 2]. Therefore, studies on obstacle avoidance have become an active topic in the field of mobile robots, and a series of strategies have been proposed to realize the avoidance of static and/or dynamic obstacles [2–5]. In the above references, the dynamic environments of the mobile robots are known in advance, because the obstacles are assumed to have predefined moving behaviors. However, mobile robots occasionally have to work in uncertain circumstances, where the movements of the obstacles cannot be predicted accurately [6, 7]. Besides, the sensor and actuator noises, the time delay of network, and other factors may still lead to inaccurate operations of the robots themselves [8]. Due to these reasons, probability-based obstacle-avoidance algorithms have been put forward in recent years [9–11], which models both the movements of the obstacles and the operations of the robots as probabilistic events.
The correctness of obstacle-avoidance algorithm is very important. Testing has been the most frequently used method in the performance validation of robots, but it is by no means the best way [12]. Its deficiencies mainly lie in two aspects: the testing results are incomplete, because only a subset of all the possible cases can be examined by physical experiments or software simulations; the testing results are generally small-sample data that are unsuitable for probabilistic analysis.
Formal verification [13] is a useful alternative method to physical experiments and software simulations, because it is not only complete in logic but also adaptable for the description and analysis of probabilistic events [14–16]. For example, probabilistic model checking (PMC) is a probability-based formal verification method widely used in the performance validation of software and hardware [17, 18]. There are three main steps involved in PMC. First, a mathematical model is built to incorporate all the possible probabilistic behaviors; second, Probabilistic Computation Tree Logic (PCTL) formulae [19] are derived to describe the key logical requirements; third, a software tool such as PRISM [20] is employed to check whether the mathematical model satisfies the logical requirements or not. If all the requirements are fully satisfied, the probabilistic behaviors are validated. Otherwise, it implies that some errors may exist in the original model. In recent years, formal verification has been widely used to verify the path planning of mobile robots. However, PMC has been scarcely applied in validating the obstacle-avoidance strategies.
The purpose of the present paper is to verify three avoidance algorithms that involve a mobile robot and a single dynamic obstacle [9]. The paths of the robot and obstacle cross each other, and their movements have the properties of Markov Decision Processes (MDPs) [21, 22]. The probabilistic models and the PCTL formulae are first built, and then the PRISM software is employed to make analyses on the three avoidance strategies. Both the time and the energy requirements for the robot to reach the destination have been obtained, and the time or energy based optimal condition is thereby applied to help find the optimal obstacle-avoidance strategy under diverse circumstances. In addition, simulations are also made in MATLAB to verify the correctness of the results obtained by PRISM. To this end, the time of obstacle-avoidance output by PRISM is taken as the input of MATLAB.
The rest of the paper is organized as follows. Section 2 introduces the necessary preliminaries. In Section 3, we formulate the obstacle-avoidance algorithms. Then, the MDPs are constructed in Section 4, and the verification on the algorithm is made in Section 5. Finally, conclusions are presented in Section 6.
2. Preliminaries
2.1. Markov Decision Process
A Markov Decision Process (MDP) is generally defined as a multituple [18]. In the present work, a five-tuple, (), is adopted to describe the probabilistic model, where(1) represents a group of states,(2) is the initial state and it belongs to ,(3) stand for a set of actions, each of which has its own probability,(4) are the state transition functions,(5) is a reward function that describes quantitative information of the model.
Figure 1 shows the schematic diagram of a simple MDP, where , , , , and are the probabilities of the corresponding actions, , , , , , , and the four state transition functions are
In addition, the reward function “” only describes quantitative information that can be calculated somehow but is generally difficult to be shown in the schematic diagram.
2.2. Probabilistic Computation Tree Logic
Probabilistic Computation Tree Logic (PCTL) is an extension of the nonprobabilistic Computation Tree Logic (CTL), which expresses the qualitative properties of the probabilistic models [19]. Generally, a PCTL is expressed as a formula that connects the properties of the MDP using the Boolean operators (such as ¬ (negation) and (conjunction)), the temporal operators (such as (next), (until), (future), and (globally)), the probabilistic operator , and a reward (or cost) operator .
A PCTL formula can be used to calculate a certain quantity of the MDP such as the maximum probability and the minimum time (or energy) consumption. For example, using the following formula one can compute the maximum probability of a state that satisfies both the two specifications spec1 and spec2.
Again, the following formula can be used to find the minimum cost of energy when the specification spec3 is satisfied:Finally, consider another PCTL formula
Obviously, it can be used to calculate the maximum probability of a state, which meets the two requirements spec4 and spec6 but does not satisfy spec5.
3. Obstacle-Avoidance Algorithms
3.1. Probabilistic Model of a Dynamic Obstacle
Divide the whole moving process of the dynamic obstacle into steps, each of which has the same time interval . Assume that the dynamic obstacle has a stepwise uniform motion; that is, it has different velocity at different time step. In the whole process of movement, its minimum and maximum velocities are denoted by and . Then, the probability density function for the obstacle to reach the position after steps can be expressed as [9] where
Here, , , and are the initial position, variance, and average velocity, respectively. Integrating along the practical path of the obstacle, one can obtain the probability of reaching the position .
3.2. Obstacle-Avoidance Strategies of a Mobile Robot
Here, three obstacle-avoidance strategies are given for a mobile robot with only a single dynamic obstacle. As shown in Figure 2, the former and the latter are represented by a circle and a square, respectively.
Assume that the paths of the robot and the obstacle intersect at region with angle (see Figure 2). and represent their locations when they begin to enter region , while and stand for those when they just leave such a region completely. is a reference position of the robot, which can be specified at an arbitrary point not over . and are two reference positions of the obstacle to be determined in subsequent derivation. is an undetermined position of the robot.
Under this condition, we mainly consider three avoidance strategies: Acceleration, Deceleration, and Detour. In the Acceleration mode, the robot goes across region earlier than the obstacle does by enhancing its velocity; in the Deceleration one, it passes region later than the obstacle does by decreasing its velocity; in the Detour one, it realizes obstacle avoidance by changing its moving direction as shown in Figure 3, where is the rotation angle and is the rotation radius. When and are given, the robot may go along either or , due to its probabilistic behavior. Here, represents the expected Detour path, while denotes the unexpected one. In Figure 3, represents the position where the obstacle begins to enter the path intersection region in the expected Detour mode.
However, how to choose one of these three strategies under a specific circumstance is still a problem that needs to be solved. Here, the concept of Dangerous Region is introduced to help explain the decision-making process of the robot. If the obstacle stays in such a region at the initial time, the robot has to choose a certain strategy to avoid collision.
To describe such a concept, one needs to make the following three parameters clear at first.(a)Reference position : when the two paths are given, the position of is fixed (see Figure 2). Therefore, the point can be determined by the distance , which is defined as [9] where is the initial velocity of the robot and is the distance between and .(b) Time : .(c) Reference position : it is defined in such a way that the obstacle can take less time than to go from it to with the velocity .
Then, we name the region between and as the Dangerous Region. Note: the distance between and is less than that between and .
4. Construction of MDPs
This section gives the MDP models for both the obstacle and the robot. The model symbols are shown in Symbol Description.
4.1. Formal Model of the Dynamic Obstacle
Assume that the dynamic obstacle moves along its path from the initial position to the terminal one . In this process, it passes the locations , , , , and in turn. Under this condition, its movement is divided into seven states, and then its formal model is constructed as shown in Figure 4. According to the obstacle-avoidance strategies, is the Dangerous Region. Therefore, if the obstacle is in the state or , the robot should choose a strategy to avoid collision.
To help understand the model, three typical state transitions are explained as follows:(1).
When the timer is less than , the obstacle is in the state ; that is, the obstacle is moving in the region .(2).
When becomes equal to , the obstacle switches to the state from the state ; that is, the obstacle moves into the region from the region , and then the timer is reset to .(3).
The obstacle stays in the state and will not switch to other states; that is, the obstacle reaches its terminal position.
Dividing each region in Figure 4 into some equal steps in time, one can compute the probability for the obstacle to reach the position by integrating the probability density function in (5).
4.2. Formal Model of the Mobile Robot
This section provides the formal representation for the state transitions of the robot (see Figures 5–7). In the description, we assume that each avoidance strategy can be executed by the robot with a certain probability. Specifically, when the robot takes the “Detour” operation, it can rotate to the expected angle with probability p1, as a result of the wheel slippage and other factors; if the robot selects the action “Acceleration” or “Deceleration,” the probability of changing its velocity to the expected value is or , because of the actuator error and other uncertainties.
4.2.1. Detour Strategy
The mobile robot moves from its initial position to the reference one and then takes the detour strategy to avoid the obstacle. If the robot rotates to the expected angle, it can move into the desired region successfully (see Figure 3). Otherwise, it may go into another arbitrary region such as (see Figure 3). As stated above, it rotates to the expected angle with probability . Therefore, the robot can go into with probability , while it will deviate into with probability . In the former case, the collision will be effectively avoided and the robot can go to its terminal safely, because the detour operation is successful. However, in the latter case, whether the robot will collide with the obstacle or not is still uncertain. If the collision is avoided, the robot will also go to successfully. Otherwise, the collision will make the detour operation fail.
To help understand the model, four typical state transitions are explained as follows:(1).
When the mobile robot is in the state and the value of Safe is equal to 0 (i.e., the obstacle is in the Dangerous Region), the robot will take the detour strategy to switch to the state . The detour operation can be successfully made with probability .(2) .
When the robot is in the state and the value of Safe is equal to 0, the robot will switch to the state with probability ().(3).
When the robot is in the state and the value of Conflict is equal to 1 (i.e., the robot collides with the obstacle), the robot will switch to the Collision state.(4).
The robot will stay in the Collision state and stop moving.
4.2.2. Acceleration Strategy
If the robot locating at takes the Acceleration strategy to avoid collision, it will move to by enhancing its velocity. As stated above, the probability of increasing its velocity to the expected value is . Therefore, it can reach with probability and then go to the terminal safely. In this strategy, it is easy to infer that the change of velocity will fail with probability . When the desired velocity is not obtained, whether the robot will collide with the obstacle or not is uncertain. If the collision is avoided, the robot will also go to successfully.
Two typical state transitions are explained as follows:(1).
When the robot is in the state and the value of Safe is equal to 0, the robot chooses the Acceleration action. Its state switches from to with probability .(2).
When the robot is in the state and the value of Safe is equal to , the robot chooses the Acceleration action. Its state will change from to with probability (), because the Acceleration action will fail with probability ().
4.2.3. Deceleration Strategy
If the robot locating at takes the Deceleration strategy to avoid collision, it will move to a certain position between and by reducing its velocity. As stated above, the probability of decreasing its velocity to the expected value is . Therefore, it can reach with probability and then go to the terminal safely. Obviously, the change of velocity will fail with probability . When the desired velocity is not obtained, whether the robot will collide with the obstacle or not is uncertain. If the collision is avoided, the robot will also go to successfully.
Two typical state transitions are as follows:(1).
When the robot is in the state and the value of Safe is equal to 0, the robot chooses the Deceleration action. Its state switches from to with probability .(2) .
When the robot is in the state and the value of Safe is equal to 0, the robot chooses the Deceleration action. Its state will change from to with probability (), because the Deceleration action will fail with probability ().
5. Probabilistic Analysis and Verification
In this section, we take an example to make probabilistic analysis and verification. The velocities and geometrical parameters of the example are shown in Figure 8, where and represent the diameter of the circle and the side length of the square, respectively.
Next, probabilistic analysis is performed in the PRISM software to get the probability-time curves of the three obstacle-avoidance strategies. Then, the corresponding computation is also carried out in MATLAB to verify the effectiveness of the analysis in PRISM. On this basis, the time and energy consumption of the three strategies are calculated in PRISM to help find the optimal strategy.
5.1. Mobile Robot Transiting through the Expected States
Here, we consider the cases of the mobile robot moving through the states in Detour, in Acceleration, and in Deceleration. It is assumed that .
5.1.1. Probability-Time Curves Given by PRISM
To get the probability-time curves by analysis in PRISM, one has to construct the following PCTL formula at first:where the value 1 of means that the robot succeeds in avoiding the obstacle.
The analysis results corresponding to the three avoidance strategies are shown in Figure 9.
(1) Detour Strategy. (The rotation radius of the robot is prescribed as 1.0 cm.) If the time for the robot to pass the region is less than or equal to 7 s, the obstacle will be successfully avoided.
(2) Acceleration Strategy. If the time to go across the region does not exceed 4 s, the robot is able to avoid collision.
(3) Deceleration Strategy. If the time to pass the region is more than or equal to 21 seconds, the collision can be prevented.
5.1.2. Simulation Results by MATLAB
In this subsection, MATLAB is used to make simulation to verify the correctness of the results obtained by PRISM. To this end, the time of obstacle-avoidance output by PRISM is taken as the input of MATLAB.
Nevertheless, how to determine the input time is still a problem needing consideration. In the strategy of Detour or Acceleration, if the robot takes longer time to pass the region or , the probability for it to avoid collision will become smaller. However, shorter time to pass will give rise to smaller obstacle-avoidance probability in the Deceleration mode. Therefore, we choose the maximum time of obstacle avoidance in Detour (i.e., 7 s) or Acceleration (i.e., 4 s) but the minimum one in Deceleration (i.e., 21 s) to make verification in MATLAB.
The visualized results of the simulation are provided in Figure 10. It is seen that the collision is effectively avoided in each case, which verifies the analysis results in PRISM.
(a) Detour
(b) Acceleration
(c) Deceleration
5.2. Mobile Robot Transiting through the Unexpected States
In this subsection, we consider the cases of the mobile robot moving through the states in Detour, in Acceleration, and in Deceleration.
5.2.1. Verification Results by PRISM
The PCTL formula iswhere the value 1 of means that the robot succeeds in avoiding the obstacle finally, even if it goes through the unexpected state.
The probability-time curves of the three obstacle-avoidance strategies are shown in Figure 11.
(a) Detour with
(b) Acceleration
(c) Deceleration
(1) Detour Strategy. Assume that the expected rotation angle is , but the actual one is only or . Under this condition, the robot will adjust its motion to avoid collisions. Figure 11(a) shows that when the actual rotation angle is or the maximum time to successfully avoid collision is 5 s or 6 s, respectively.
(2) Acceleration. Like in case , if the time for the robot to reach is less than or equal to 4 s, the obstacle will be avoided (see Figure 11(b)).
(3) Deceleration Strategy. If the time to arrive at is longer than 7 s, the robot can avoid the obstacle (see Figure 11(c)).
5.2.2. Simulation Results by MATLAB
In this subsection, simulations are carried out in MATLAB to verify the analysis results shown in Figure 11.
(1) Detour Strategy. Here, the time of obstacle-avoidance output by PRISM is taken as the input of MATLAB. Therefore, when the actual rotation angle is or , we take 5 s or 6 s to perform simulation. Figure 12 shows the visualized results, which obviously verify the results in Figure 11(a).
(a) Detour with
(b) Detour with
(2) Acceleration or Deceleration Strategy. For Acceleration or Deceleration, we take six different values of time and four different values of or to make simulations in MATLAB. In each case, 100 times of simulations are performed to calculate the probability of obstacle avoidance. The results are given in Tables 1 and 2, which are consistent with those shown in Figures 11(b) and 11(c). From this, we come to the conclusion that probabilistic model checking is an effective method to verify the obstacle-avoidance strategies of mobile robots.
5.3. Minimum Time and Energy Consumption
Here, the minimum time and energy consumption for the robot to go from its initial position to the destination are used to find the optimal obstacle-avoidance strategy. For this purpose, analysis is further performed in PRISM to determine these two parameters for different obstacle-avoidance paths.
The PCTL formula corresponding to the minimum time is where the value 1 of Done means the robot reaches the destination safely.
The PCTL formula associated with the minimum energy is
In addition, the kinetic energy of the robot can be calculated as follows: where is the mass, is the rotational inertia, is the linear velocity, and is the angular velocity.
The analysis results of the minimum time and energy consumption are shown in Table 3. In the Detour mode, if actual rotation angle is , the robot arrives at its destination by passing the expected state; otherwise, it reaches the terminal by going through the unexpected state. It is seen that the expected path costs the lowest “minimum energy” but the longest “minimum time,” while the unexpected trajectory () consumes the shortest “minimum time” but the highest “minimum energy.” In the acceleration mode, the expected path () requires the shortest “minimum time” but the highest “minimum energy,” while the unexpected one () needs the lowest “minimum energy” but the longest “minimum time.” In the Deceleration mode, the expected path () spends the longest “minimum time,” while all the paths expend the same “minimum energy.” Further comparison shows that the Acceleration strategy () is the least time-costing, while the Deceleration one is the least energy-consuming, among all the three kinds of strategies.
Finally, it deserves noting that the data in Table 3 can help us to find the optimal obstacle-avoidance strategy. For example, if the movement time of the robot is limited, we can choose the Acceleration strategy (). If the power supply is not so sufficient, the Deceleration strategy can be selected instead.
6. Conclusions
The MDP-based probabilistic formal models are constructed for three different kinds of obstacle-avoidance strategies for a mobile robot in an uncertain dynamic environment. The PCTL formulae are then built to perform analysis in the PRISM software, which yields the probability-time curves for the obstacle-avoidance paths. In addition, the time of obstacle-avoidance output by PRISM is taken as input to conduct simulations in MATLAB, which verify the correctness of the results obtained by PRISM. Further, both the minimum time and the energy consumption for the robot to reach its destination under each obstacle-avoidance strategy are obtained by analyses in PRISM. Finally, the time or energy based optimal condition is thereby applied to determine the optimal obstacle-avoidance strategy under diverse circumstances.
It deserves noting that the present paper only verifies the probabilistic obstacle-avoidance strategies that involve a mobile robot and a single dynamic obstacle. In practice, there may be more complex obstacle-avoidance conditions. For example, a single mobile robot may encounter several dynamic obstacles, or, even worse, a swarm of mobile robots may have to go across a terrain with multiple dynamic obstacles. These complex cases require more complicated probabilistic formal verifications. The present paper lays a good foundation for these future works.
Symbols
| , : | Two timers |
| : | It represents the region between and , where and |
| : | A state index of the dynamic obstacle, which means the obstacle locates at region |
| : | The time for the obstacle to pass region |
| : | The security index. If the value of Safe is , it means that an obstacle exists in the Dangerous Region and the robot needs to avoid collisions |
| : | The collision index. If the value of Conflict is , it means the robot collides with an obstacle |
| : | It represents the region between and , where and |
| : | A state index of the robot, which means the robot locates at region |
| : | A state index of the robot. It means the robot locates at region with operation errors |
| : | The time for the robot to pass region |
| The time for the robot to pass region with operation errors. |
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the International Cooperation Program on Science and Technology (2010DFB10930 and 2011DFG13000), NSFC (61070049, 61170304, 61104035, 61373034, 61303014, and 61472468), NSFCB (4122017), TJSHG (201310028014), KM (201510028015), KZ (201210028036), and Scientific Research Base Development Program of the Beijing Municipal Commission of Education.