A Shortest-Path Lyapunov Approach for Forward Decision Processes

Abstract

In previous work, attention was restricted to tracking the net using a backward method that knows the target point beforehand (Bellmans's equation), this work tracks the state-space in a forward direction, and a natural form of termination is ensured by an equilibrium point 𝑝^∗(𝑀(𝑝^∗)=𝑆<∞and𝑝^∗•=∅). We consider dynamical systems governed by ordinary difference equations described by Petri nets. The trajectory over the net is calculated forward using a discrete Lyapunov-like function, considered as a distance function. Because a Lyapunov-like function is a solution to a difference equation, it is constructed to respect the constraints imposed by the system (a Euclidean metric does not consider these factors). As a result, we prove natural generalizations of the standard outcomes for the deterministic shortest-path problem and shortest-path game theory.

1. Introduction

The shortest-path problem (see [1–3]) plays a fundamental role in Petri nets theory, since it can be used to model processes. The analysis of these models can show useful information about the process. For example, deadlocks, equilibrium points, and so forth can be identified by computational analysis.

While it is possible to analyze such processes using the existing classical theory through the Bellman's equation with the cost criterion ([4–15]), much of this theory has few disadvantages. Bellman's equation is expressed as a sum over the state of a trajectory needs to be solved backwards in time from the equilibrium point (target point). It results in an optimal function when it is governed by Bellman's principle, producing the shortest path needed to reach a known equilibrium point. Notice that the necessity to know the equilibrium point beforehand when applying the equation is a significant constrain, given that, in many practical situations, the state space of a Petri net is too large for an easy identification of the equilibrium point.

Moreover, algorithms using Bellman's equation usually solve the problem in two phases [16]: preprocessing and search. In the preprocessing phase, the distance is usually calculated between each state and the equilibrium points (final states) of the problem, in a backward direction. Then, in the search phase, these results are employed to calculate the distance between each state and the equilibrium points, leading the search process to a forward search.

Tracking the state space in a forward direction allows the decision maker to avoid invalid states that occur in the space generated by a backward search. In most cases, the forward search gives the impression to be more useful than the backward search. The explanation is that in the backward direction, when the case of incomplete final states arises, invalid states appear causing problems.

Shortest-path problem [17, 18] can be classified by two key categories [19]: (a) the single-source shortest-path problem where the goal is to find the shortest path from a given node to a target node (e.g., the algorithms of Dijkstra and Bellman-Ford); and (b) the all-pairs shortest-path problem is a similar problem in which the objective is to determine the shortest path between every pair of nodes in the net (e.g., the algorithms of Floyd-Warshall and Johnson).

We are concerned about the first case. However, we consider dynamical systems governed by difference equations described by Petri nets. The trajectory over the net is calculated using a discrete Lyapunov-like function. A Lyapunov-like function is considered as a distance function denoting the length from the source place to the equilibrium point. This work is concerned with the analysis of the decision process where a natural form of termination is ensured by an equilibrium point.

Lyapunov-like functions can be used as forward trajectory-tracking functions. Each applied optimal action produces a monotonic progress towards an equilibrium point. Because it is a solution to the difference equation, naturally it will lead the system from the source place to the equilibrium point.

It is important to note that there exist areas of research using Petri nets as modeling tool where the use of a Lyapunov-like function is inherent. For instance, the “Entropy” function is a specific Lyapunov-like function used in Information Theory as a measure of the information disorder. The “free Gibbs energy function” is a Lyapunov-like function used in molecular biology for calculating the energy change in a metabolic network.

This paper introduces a modeling paradigm for shortest-path decision process representation in Petri nets theory. The main point of this paper is its ability to represent the characteristics related only with the global system behavior, and those characteristics related with the trajectory-tracking behavior.

Within the global system behavior properties, we show notions of stability. In this sense, we call equilibrium point to the place in a Petri net that its marking is bounded and it is the last place in the net (sink).

In the trajectory-tracking behavior properties framework, we define the trajectory function as a Lyapunov-like function. By an appropriate selection of the Lyapunov-like function, it is possible to optimize the trajectory. By optimizing the trajectory, we understand that it is the minimum trajectory-tracking value (in a certain sense). In addition, we use the notions of stability in the sense of Lyapunov to characterize the stability properties of the Petri net. The core idea of our approach uses a nonnegative trajectory function that converges in decreasing form to a (set of) final decision states. It is important to point out that the value of the trajectory function associated with the Petri net implicitly determines a set of policies, not just a single policy (in case of having several decisions states that could be reached). We call “optimum point” the best choice selected from a number of possible final decision places that may be reached (to select the optimum point, the decision process chooses the strategy that optimizes the trajectory-tracking value).

As a result, we show that the global system behavior properties and the trajectory-tracking behavior properties of equilibrium, stability, and optimum-point conditions meet under certain restrictions: if the Petri net is finite, then we have that a final decision place is an equilibrium point.

The paper is structured in the following manner. The next section discusses the motivation of the work. Section 3 presents the formulation of the decision model, and all the structural assumptions are introduced there, giving a detailed analysis of the equilibrium, stability, and optimum-point conditions for the global system behavior properties and the trajectory tracking behavior parts of the Petri net. Section 4 presents the properties of the model. Finally, in Section 5 some concluding remarks are outlined.

2. Motivation

In this paper, we consider dynamical systems in which the time variable changes discretely, and the system is governed by ordinary difference equations. Let us consider systems of first-order difference equations given by𝑠_𝑛+1=𝑓(𝑠_𝑛,𝑎_𝑛),𝑠_𝑛0=𝑠₀,𝑛∈ℕ^𝑛0₊, (1) where 𝑠_𝑖 with 𝑖∈ℕ are the state variable of the system, 𝑠₀ is the initial state, 𝑎_𝑖 and 𝑖∈ℕ are the action of the system, ℕ^𝑛0₊={𝑛₀,𝑛₀+1,…,𝑛₀+𝑘,…},𝑛₀≥0. The system is specified by the state transition function 𝑓, which is always assumed as a one-to-one function for any fixed 𝑎 and 𝑛∈ℕ, continuous in all its arguments.

Lyapunov defined a scalar function 𝐿, called a Lyapunov-like function, inspired by a classical energy function, which has four important properties that are sufficient for establishing the domain of attraction of a stable equilibrium point: (a) ∃𝑠^∗ such that 𝐿(𝑠^∗)=0; (b) 𝐿(𝑠)>0 for all 𝑠≠𝑠^∗; (c) 𝐿(𝑠)→∞ when 𝑠→∞; and (d) Δ𝐿=𝐿(𝑠_𝑖+1)−𝐿(𝑠_𝑖)<0 for all 𝑖,𝑠_𝑖≠𝑠^∗. The condition (a) requires the equilibrium point to have zero potential by means of a translation to the origin, (b) means that the Lyapunov-like function to be semipositive defined, (c) means that there is no 𝑠^∗ reachable from some 𝑠, and (d) means that the Lyapunov-like function has a minimum at the equilibrium point.

The main idea of Lyapunov is attained in the following interpretation: given an isolated physical system, if the change of the energy 𝐸 for every possible state 𝑠 is negative, with the exception of the equilibrium point 𝑠^∗, then the energy will decrease until it finally reaches the minimum at 𝑠^∗. Intuitively, this concept of stability means that a system perturbed from its equilibrium point will always return to it.

A system is stable [20, 21] if for a given set of initial states the state of the system ensures (i) to reach a given set of states and stay there perpetually or, (ii) to go to a given set of states infinitely often. The conventional notions of stability in the sense of Lyapunov and asymptotic stability can be used to characterize the stability properties of discrete event systems. An important advantage of the Lyapunov approach is that it does not require high-computational complexity but the difficulty lies in specifying the Lyapunov-like function for a given problem.

At this point, it is important to note that the Lyapunov-like function 𝐿 is not unique, however the energy function of a system is only one of its kind. A system whose energy 𝐸 decreases on the average, but not necessarily at each instance, is stable but 𝐸 is not a Lyapunov-like function.

Lyapunov-like functions [22] can be used as trajectory-tracking functions and optimal cost-to-target functions. As a result of calculating a Lyapunov-like function, a discrete vector field can be built for tracking the actions over the net. Each applied optimal action produces a monotonic progress (of the optimal cost-to-target value) toward an equilibrium point. In this sense, if the function decreases with each action taken, then it approaches an infimum/minimum (that converges asymptotically or reaches a constant).

From what we have stated before, we can deduce the following geometric interpretation of distance [22]: (a) 𝐿(𝑠) is a measure of the distance from the starting state 𝑠₀ to any state 𝑠 in the state space (this is straightforward from the fact that ∃𝑠^∗ such that 𝐿(𝑠^∗)=0 and 𝐿(𝑠)>0 for all 𝑠≠𝑠^∗); and (b) the distance from the stating state 𝑠₀ to any state 𝑠_𝑛 in the state space decreases, when 𝑛→∞. It is because 𝐿(𝑠_𝑖+1)−𝐿(𝑠_𝑖)<0 for all 𝑖,𝑠_𝑖≠𝑠^∗.

A Lyapunov-like function can be considered as a distance function denoting the length from the initial state to the equilibrium point. It is important to note that the Lyapunov-like function is constructed to respect the constraints imposed by the difference equation of the system. In contrast, a Euclidean metric does not take into account these factors. For that reason, the Lyapunov-like function offers a better understanding of the concept of the distance required to converge to an equilibrium point in a discrete dynamical system.

By applying the computed actions, a kind of discrete vector field can be imagined over the search graph. Each applied optimal action yields a reduction in the optimal cost-to-target value, until the equilibrium point is reached. Then, the cost-to-target values can be considered as a discrete Lyapunov function.

In our case, an optimal discrete problem, the cost-to-target values are calculated using a discrete Lyapunov-like function. Every time a discrete vector field of possible actions is calculated over the decision process. Each applied optimal action (selected via some “criteria”) decreases the optimal value, ensuring that the optimal course of action is followed and establishing a preference relation. In this sense, the criteria change the asymptotic behavior of the Lyapunov-like function by an optimal trajectory-tracking value.

Usually, the criterion in optimization problems is related with the choice of whether to minimize or maximize the optimal action. If the problem is related with energy transformations, as is classically the case in control theory, then the criterion of minimization is applied. However, if the dilemma involves a reward, typical in game theory, then maximization is considered. In this work, we will arbitrary consider the criterion of minimization.

The Lyapunov-like function can be employed as a trajectory-tracking function through the use of an operator, which represents the criterion that selects the optimal action that forces the function to decrease and approaches an infimum/minimum. It forces the function to make a monotonic progress toward the equilibrium point. The Lyapunov-like function can be defined, for example, as𝐿^∗(𝑠_𝑛+1)=min_{𝑎∗∈𝐴}𝐿(𝑓(𝑠_𝑛,𝑎^∗_𝑛)) (2) which means that the optimal action is chosen to reach the infimum/minimum. The function 𝐿^∗ works as a guide leading the system optimally from its initial state to the equilibrium point.

Example 1. To illustrate the shortest-path problem, let us consider a grid world (see Figure 1). At each time step, an agent is able to select an action among a finite set 𝐴 of actions, for example, 𝐴={Up,Down,Left,Right}. A transition model specifies how the world changes when an action is executed. An “equilibrium point” 𝑠^∗ is a natural final state of the system. Therefore, the shortest-path problem is a search through the state space for an optimal path to the equilibrium point 𝑠^∗, using a deterministic transition model. The value of a state 𝑠 is a number 𝑉(𝑠) that intuitively speaking expresses the desirability of state 𝑠. For instance, let us consider the state-value function 𝑉 being equal to the min function [23] as a specific Lyapunov-like function able to lead an agent to an equilibrium point in a grid world.Example 2. The relative entropy or Kullback-Leibler [24, 25] distance between two probability distributions 𝑞¹_{𝑖𝑗|𝑘} and 𝑞²_{𝑖𝑗|𝑘} is defined as𝒱(𝑞¹,𝑞²)=^𝑁𝑁_𝑖=1𝑗=1𝑞¹_{𝑖𝑗|𝑘}𝑞log^{1𝑖𝑗|𝑘}𝑞²_{𝑖𝑗|𝑘}. (3)In the above definition, we use the convention (based on continuity arguments) that 0log(0/𝑞²_{𝑖𝑗|𝑘})=0 and 𝑞¹_{𝑖𝑗|𝑘}log(𝑞¹_{𝑖𝑗|𝑘}/0)=∞. The relative entropy is always nonnegative and is zero if and only if 𝑞¹_{𝑖𝑗|𝑘}=𝑞²_{𝑖𝑗|𝑘}. 𝒱(𝑞¹,𝑞²) is a distance-like function between distributions since it is not symmetric and does not satisfy the triangle inequality.

Figure 1 

An illustrative example of finding the shortest path in a grid world.

Example 3. Glycolysis pathway (see Figure 2) is well known and described [11, 26, 27]. It is a ten-step catabolic pathway that makes use of eleven different enzymes. The outcome are the conversion of glucose in two molecules of pyruvate with concurrent net production of 2 ATPs. Glycolysis process can be divided in two stages: (1) the conversion of glucose to glyceraldehyde 3-phosphate with a required input of 2 ATPs, (2) the conversion of glyceraldehyde 3-phosphate to pyruvate with a net output of 4 ATPs.
Glycolysis can be informally explained from an energetic perspective as follows. The initial amount of glucose may be represented as a ball at the top of an irregular hill. Every time the ball bounces, the hill represents a reaction state in the breakdown of the sugar process. Each bounce of the ball corresponds to a change in free energy level. This energy change is modeled by the Gibbs energy function which is a Lyapunov-like function. It is important to note that bounces are irregular (reaching lower and higher energy levels) and determined by the environment conditions. The final state (pyruvate) is represented by the bottom of the hill where the ball reaches a steady state (not bounces).
Let us explain the Petri net dynamics of the system model as follows. Continuing with the ball and hill explanation, let us suppose that the ball, representing the product pyruvate, is at the bottom of the hill. And let us suppose that there is no net force able to move the ball either up or down the hill. That means that the reactions (forward and backward) are evenly balanced. Therefore, the substances and products are in equilibrium, and no net dynamics will take place. That is, “the metabolic network system is in equilibrium.”

Figure 2 

Glycolysis and pentose-phosphate pathways model.

3. Formulation

We introduce the concept of decision process Petri nets (DPPNs) by locally randomizing the possible choices, for each individual place of the Petri net [23, 28]. Definition 1. A decision process Petri net is a 7-tuple DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈}, where
(i)𝑃={𝑝₀,𝑝₁,𝑝₂,…,𝑝_𝑚} is a finite set of places,(ii)𝑄={𝑞₁,𝑞₂,…,𝑞_𝑛} is a finite set of transitions,(iii)𝐹⊆𝐼∪𝑂 is a flow relation, where 𝐼⊆(𝑃×𝑄) and 𝑂⊆(𝑄×𝑃) such that 𝑃∩𝑄=∅ and 𝑃∪𝑄≠∅,(iv)W: 𝐹→ℕ¹₊ is a weight function,(v)𝑀₀: 𝑃→ℕ is the initial marking,(vi)𝜋: 𝐼→ℝ₊ is a routing policy representing the probability of choosing a particular transition, such that for each 𝑝∈𝑃,∑_{𝑞𝑗∶(𝑝,𝑞𝑗)∈𝐼}𝜋((𝑝,𝑞_𝑗))=1,(vii)𝑈: 𝑃→ℝ₊ is a trajectory-tracking function.

We adopt the standard rules about representing nets as directed graphs, namely, places are represented as circles, transitions as rectangles, the flow relation by arcs, and markings are shown by placing tokens within circles [29]. As usual, we will denote 𝑧•={𝑦|(𝑧,𝑦)∈𝐹} and •𝑧={𝑦|(𝑦,𝑧)∈𝐹}, for all 𝑧∈𝐼∪𝑂. A source place is a place 𝑝₀∈𝑃 such that •𝑝₀=∅ (there are no incoming arcs into place 𝑝₀). A sink place is a place 𝑝_𝑓∈𝑃 such 𝑝_𝑓•=∅ (there are no outgoing arcs from 𝑝_𝑓). A net system is a pair Σ=(𝑁,𝑀₀) comprising a finite net 𝑁=(𝑃,𝑄,𝐹) and an initial marking 𝑀₀. A transition 𝑞∈𝑄 is enabled at a marking 𝑀, denoted by 𝑀[𝑞⟩, if for every 𝑝∈•𝑞, we have that 𝑀(𝑝)≥1. Such a transition can be executed, leading to a marking 𝑀^′ defined by 𝑀^′=𝑀−•𝑞+𝑞•. We denote this by 𝑀[𝑞⟩𝑀^′ or 𝑀[⟩𝑀^′. The set of reachable markings of Σ is the smallest (with respect to set inclusion) set [𝑀₀⟩ containing 𝑀₀ and such that if 𝑀∈[𝑀₀⟩ and 𝑀[⟩𝑀^′, then 𝑀^′∈[𝑀₀⟩.

The previous behavior of the DPPN is described as follows. When a token reach a place, it is reserved for the firing of a given transition according to the routing policy determined by 𝑈. A transition 𝑞 must fire as soon as all the places 𝑝₁∈𝑃 contain enough tokens reserved for transition 𝑞. Once the transition fires, it consumes the corresponding tokens and immediately produces an amount of tokens in each subsequent place 𝑝₂∈𝑃. When 𝜋(𝜄)=0 for 𝜄∈𝐼 means that there are no outgoing arcs in the place-transitions Petri net (i.e., 𝑝∈𝜄 is a sink).

In Figure 2, we have represented partial routing policies 𝜋 that generate a transition from state 𝑝₁ to state 𝑝₂, where 𝑝₁,𝑝₂∈𝑃 as follows.

Case 1. The probability that 𝑞₁ generates a transition from state 𝑝₁ to 𝑝₂ is 1/3. But, because 𝑞₁ transition to state 𝑝₂ has two arcs, the probability to generate a transition from state 𝑝₁ to 𝑝₂ is increased to 2/3.

Case 2. We set by convention for the probability that 𝑞₁ generates a transition from state 𝑝₁ to 𝑝₂ is 1/3 (1/6 plus 1/6). However, because 𝑞₁ transition to state 𝑝₂ has only one arc, the probability to generate a transition from state 𝑝₁ to 𝑝₂ is decreased to 1/6.

Case 3. Finally, we have the trivial case when there exists only one arc from 𝑝₁ to 𝑞₁ and from 𝑞₁ to 𝑝₂.

It is important to note that, by definition, the trajectory-tracking function 𝑈 is employed only for establishing a trajectory tracking, working in a different execution level of that of the place-transitions Petri net. The trajectory-tracking function 𝑈 in no way change the place-transitions Petri net evolution or performance.

𝑈_𝑘(⋅) denotes the trajectory-tracking value at place 𝑝_𝑖∈𝑃 at time 𝑘 and let 𝑈_𝑘=[𝑈_𝑘(⋅),…,𝑈_𝑘(⋅)]^𝑇 denote the trajectory-tracking state of DDPN at time 𝑘.𝐹𝑁∶𝐹→ℝ₊ is the number of arcs from place 𝑝 to transition 𝑞 (the number of arcs from transition 𝑞 to place 𝑝).

Consider an arbitrary 𝑝_𝑖∈𝑃 and for each fixed transition 𝑞_𝑗∈𝑄 that forms an output arc (𝑞_𝑗,𝑝_𝑖)∈𝑂, we look at all theprevious places 𝑝_ℎ of the place 𝑝_𝑖 denoted by the list (set) •𝑝_𝜂𝑖𝑗={𝑝_ℎ∶ℎ∈𝜂_𝑖𝑗}, where 𝜂_𝑖𝑗={ℎ∶(𝑝_ℎ,𝑞_𝑗)∈𝐼,(𝑞_𝑗,𝑝_𝑖)∈𝑂}, that materializes all the input arcs (𝑝_ℎ,𝑞_𝑗)∈𝐼 and forms the sum_{ℎ∈𝜂𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗,𝑝_𝑖)𝑈_𝑘(𝑝_ℎ), (4) where Ψ(𝑝_ℎ,𝑞_𝑗,𝑝_𝑖)=𝜋(𝑝_ℎ,𝑞_𝑗)∗(𝐹𝑁(𝑞_𝑗,𝑝_𝑖)/𝐹𝑁(𝑝_ℎ,𝑞_𝑗)) and the index sequence 𝑗 is the set 𝜅={𝑗∶𝑞_𝑗∈(𝑝_ℎ,𝑞_𝑗)∩(𝑞_𝑗,𝑝_𝑖) & 𝑝_ℎ running over the set •𝑝_𝜂𝑖𝑗}. Remark 1. •𝑝_𝜂𝑖𝑗 denote the previous places to 𝑝_𝑖 for a fixed transition 𝑞_𝑗∈𝑄.

Continuing with all the 𝑞_𝑗's, we form the vector indexed by the sequence 𝑗 identified by (𝑗₀,𝑗₁,…,𝑗_𝑓) as follows:_{ℎ∈𝜂0𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗0,𝑝_𝑖)𝑈_𝑘(𝑝_ℎ),_{ℎ∈𝜂1𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗1,𝑝_𝑖)𝑈_𝑘(𝑝_ℎ),…,_{ℎ∈𝜂𝑓𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗𝑓,𝑝_𝑖)𝑈_𝑘(𝑝_ℎ). (5)Intuitively, vector (5) represents all the possible trajectories through the transitions 𝑞_𝑗s to a place 𝑝_𝑖 for a fixed 𝑖, where 𝑗 is represented by the sequence (𝑗₁,𝑗₂,…,𝑗_𝑓) and 𝑓=#(𝜅).

Then, formally we define the trajectory-tracking function 𝑈 as follows. Definition 2. The trajectory-tracking function 𝑈 with respect a decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is represented by the following equation𝑈^𝑞𝑗_𝑘(𝑝_𝑖𝑈)=^𝑘(𝑝₀)if𝑖=0,𝑘=0,𝐿(𝛼)if𝑖>0,𝑘=0,𝑖≥0,𝑘>0, (6)where𝛼=^{ℎ∈𝜂0𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗0,𝑝_𝑖)𝑈^𝑞𝑗0_𝑘(𝑝_ℎ),_{ℎ∈𝜂1𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗1,𝑝_𝑖)𝑈^𝑞𝑗1_𝑘(𝑝_ℎ),…,_{ℎ∈𝜂𝑓𝑖𝑗}Ψ(𝑝_ℎ,𝑞_𝑗𝑓,𝑝_𝑖)𝑈^𝑞𝑗𝑓_𝑘(𝑝_ℎ), (7)the function 𝐿∶𝐷⊆ℝ^𝑛₊→ℝ₊ is a Lyapunov-like function which optimizes the trajectory-tracking value through all possible transitions (i.e., through all the possible trajectories defined by the different 𝑞_𝑗𝑠),𝐷 is the decision set formed by the 𝑗's; 0≤𝑗≤𝑓, of all those possible transitions (𝑞_𝑗𝑝_𝑖)∈𝑂, Ψ(𝑝_ℎ,𝑞_𝑗,𝑝_𝑖)=𝜋(𝑝_ℎ,𝑞_𝑗)∗(𝐹𝑁(𝑞_𝑗,𝑝_𝑖)/𝐹𝑁(𝑝_ℎ,𝑞_𝑗)),𝜂_𝑖𝑗 is the index sequence of the list of previous places to 𝑝_𝑖 through transition 𝑞_𝑗,𝑝_ℎ(ℎ∈𝜂_𝑖𝑗) is a specific previous place of 𝑝_𝑖 through transition 𝑞_𝑗.

Example 4. OR-Path (see Figure 3). Define the Lyapunov-like function 𝐿 in terms of the Entropy 𝐻(𝑝_𝑖)=−𝑝_𝑖ln𝑝_𝑖 as 𝐿=min_{𝑖=1,…,|𝛼|}(−𝛼_𝑖ln𝛼_𝑖):
(i)𝑈_𝑘=0(𝑝₀)=0.2,(ii)𝑈^𝑞𝑎_𝑘=0(𝑝₁)=𝐿[Ψ(𝑝₀,𝑞_𝑎,𝑝₁)𝑈^𝑞𝑎_𝑘=0(𝑝₀)]=min𝐻[1∗0.2]=𝐻[0.2]=0.321,(iii)𝑈^𝑞𝑏_𝑘=0(𝑝₂)=𝐿[Ψ(𝑝₁,𝑞_𝑏,𝑝₂)𝑈^𝑞𝑏_𝑘=0(𝑝₁),Ψ(𝑝₁,𝑞_𝑐,𝑝₂)𝑈^𝑞𝑏_𝑘=0(𝑝₁)]=min𝐻[1/3∗0.321,2/3∗0.321]=min[0.239,0.329]=0.239.

Figure 3 

(Left): routing policy case 1. (Right): routing policy case 2.

Example 5. AND-Path (see Figure 4). Define the Lyapunov-like function 𝐿 in terms of the Entropy 𝐻(𝑝_𝑖)=−𝑝_𝑖ln𝑝_𝑖 as 𝐿=min_{𝑖=1,…,|𝛼|}(−𝛼_𝑖ln𝛼_𝑖):
(i)𝑈_𝑘=0(𝑝₀)=0.2,(ii)𝑈^𝑞𝑎_𝑘=0(𝑝₁)=𝐿[Ψ(𝑝₀,𝑞_𝑎,𝑝₁)𝑈^𝑞𝑎_𝑘=0(𝑝₀)]=min𝐻[1∗0.2]=𝐻[0.2]=0.321,(iii)𝑈^𝑞𝑏_𝑘=0(𝑝₂)=𝐿[Ψ(𝑝₀,𝑞_𝑏,𝑝₂)𝑈^𝑞𝑏_𝑘=0(𝑝₀)]=min𝐻[1∗0.2]=𝐻[0.2]=0.321,(iv)𝑈^𝑞𝑐_𝑘=0(𝑝₃)=𝐿[Ψ(𝑝₁,𝑞_𝑐,𝑝₃)𝑈^𝑞𝑐_𝑘=0(𝑝₁)+Ψ(𝑝₂,𝑞_𝑐,𝑝₃)𝑈^𝑞𝑐_𝑘=0(𝑝₂)]=min𝐻[1∗0.321+1∗0.321]=𝐻[0.642]=0.284.

Figure 4 

OR-Path Example.

Figure 5 

AND-path example.

From the previous definition, we have the following remark. Remark 2. (i) Note that the Lyapunov-like function 𝐿 guarantees that the optimal course of action is followed (taking into account all the possible paths defined). In addition, the function 𝐿 establishes a preference relation because, by definition, 𝐿 is asymptotic; this condition gives to the decision maker the opportunity to select a path that optimizes the trajectory-tracking value.
(ii) The iteration over 𝑘 for 𝑈 is as follows: (1)for 𝑖=0 and 𝑘=0 the trajectory-tracking value is 𝑈₀(𝑝₀) at place 𝑝₀ and for the rest of the places 𝑝_𝑖 the trajectory-tracking value is 0;(2)for 𝑖≥0 and 𝑘>0 the trajectory-tracking value is 𝑈^𝑞𝑗_𝑘(𝑝_𝑖) at each place 𝑝_𝑖, and is computed by taking into account the trajectory-tracking value of the previous places 𝑝_ℎ for 𝑘 and 𝑘−1 (when needed).

Property 1. The continues function 𝑈(⋅) satisfies the following properties:
(1)∃𝑝^Δ∈𝑃 such that (a)if there exists an infinite sequence {𝑝_𝑖}^∞_𝑖=1∈𝑃 with 𝑝_𝑛→_𝑛→∞𝑝^Δ such that 0≤⋯<𝑈(𝑝_𝑛)<𝑈(𝑝_𝑛−1)⋯<𝑈(𝑝₁), then 𝑈(𝑝^Δ) is the infimum, that is, 𝑈(𝑝^Δ)=0;(b)if there exists a finite sequence 𝑝₁,…,𝑝_𝑛∈𝑃 with 𝑝₁,…,𝑝_𝑛→𝑝^Δ such that 𝐶=𝑈(𝑝_𝑛)<𝑈(𝑝_𝑛−1)⋯<𝑈(𝑝₁), then 𝑈(𝑝^Δ) is the min imum, that is, 𝑈(𝑝^Δ)=𝐶, where 𝐶∈ℝ, (𝑝^Δ=𝑝_𝑛);(2)𝑈(𝑝)>0 or 𝑈(𝑝)>𝐶, where 𝐶∈ℝ, for all 𝑝∈𝑃 such that 𝑝≠𝑝^Δ;(3)for all 𝑝_𝑖,𝑝_𝑖−1∈𝑃 such that 𝑝_𝑖−1≤_𝑈𝑝_𝑖 then Δ𝑈=𝑈(𝑝_𝑖)−𝑈(𝑝_𝑖−1)<0.

From the previous property, we have the following remark. Remark 3. In property 1 point 3, we state that Δ𝑈=𝑈(𝑝_𝑖)−𝑈(𝑝_𝑖−1)<0 for determining the asymptotic condition of the Lyapunov-like function. However, it is easy to show that such property is convenient for deterministic systems. In Markov decision process, systems are necessary to include probabilistic decreasing asymptotic conditions to guarantee the asymptotic condition of the Lyapunov-like function.

Property 2. The trajectory-tracking function 𝑈∶𝑃→ℝ₊ is a Lyapunov-like function.

Proof. Proof comes straightforward from the previous definitions.

Remark 4. From Properties 1 and 2, we have the following:
(i)𝑈(𝑝^Δ)=0 or 𝑈(𝑝^Δ)=𝐶 means that a final state is reached. Without lost generality, we can say that 𝑈(𝑝^Δ)=0 by means of a translation to the origin.(ii) In Property 1, we determine that the Lyapunov-like function 𝑈(𝑝) approaches to a infimum/minimum when 𝑝 is large thanks to property (d) of the definition the Lyapunov-like function (see motivation).(iii) Property 1, point 3 is equivalent to the following statement: ∃{𝜀_𝑖},𝜀_𝑖>0 such that |𝑈(𝑝_𝑖)−𝑈(𝑝_𝑖−1)|>𝜀_𝑖, for all 𝑝_𝑖,𝑝_𝑖−1∈𝑃 such that 𝑝_𝑖−1≤_𝑈𝑝_𝑖.

Explanation.

Intuitively, a Lyapunov-like function can be considered as trajectory-tracking function and optimal cost function. In our case, an optimal discrete problem, the cost-to-target values are calculated using a discrete Lyapunov-like function. Every time a discrete vector field of possible transitions is calculated over the decision process. Each applied optimal transition (selected via some “criterion,” e.g., min(⋅)) decreases the optimal value, ensuring that the optimal course of action is followed and establishing a preference relation. In this sense, the criterion changes the asymptotic behavior of the Lyapunov-like function by an optimal trajectory-tracking value. It is important to note that the process finished when the equilibrium point is reached. This point determines a significant difference with Bellman's equation.

Example 6 (Conc-Path (see Figure 2)). Biochemical pathway of the free energy profile of the glycolysis and pentose-phosphate. The following was adapted from Biochemistry Lehninger et al. [26] and Campbell and Farrel [30]. The free energy changes were calculated using the steady-state metabolite concentrations in RBC's and the equation 𝑈=𝑅𝑇ln([Products]/[Reactants]). 𝑈=0 was set arbitrarily at the end of the pathway after the pyruvate kinase step. The overall reaction for the pathway is shown in Figure 1. Because 𝐿∶𝐷⊆ℝ^𝑛→ℝ₊, we will use the function min_{𝑖=1,…,|𝛼|}(𝛼_𝑖∈𝐷) to select the proper element of the vector 𝛼∈𝐷:
(i)𝑈_𝑘=0(Glucose)=17.17 kcal/mol;(ii)𝑈^𝑞𝑎_𝑘=0(𝐺6𝑃)=𝐿[Ψ(Glucose,𝑞_𝑎,𝐺6𝑃)∗𝑈^𝑞𝑎_𝑘=0(Glucose)]=𝐺[Ψ(𝑝₀,𝑞_𝑎,𝑝₁)𝑈^𝑞𝑎_𝑘=0(𝑝₀)]=9.17 kcal/mol.
A decision is taken and 𝑞_𝑏 is selected instead of 𝑞_𝑘 based in the environment condition modeled via the routing policy (1/3, 2/3).
(i)𝑈^𝑞𝑏_𝑘=0(𝐹6𝑃)=𝐿[Ψ(𝐺6𝑃,𝑞_𝑏,𝐹6𝑃)∗𝑈^𝑞𝑏_𝑘=0(𝐺6𝑃)]=𝐺[2/3∗𝑈^𝑞𝑏_𝑘=0(𝑝₁)]=8.98 kcal/mol.(ii)𝑈^𝑞𝑐_𝑘=0(𝐹𝐵𝑃)=𝐿[Ψ(𝐹6𝑃,𝑞_𝑐,𝐹𝐵𝑃)∗𝑈^𝑞𝑐_𝑘=0(𝐹6𝑃)]=𝐺[Ψ(𝑝₂,𝑞_𝑐,𝑝₃)∗𝑈^𝑞𝑐_𝑘=0(𝑝₂)]=3.90 kcal/mol.(iii)𝑈^𝑞𝑑_𝑘=0(𝐷𝐻𝑃)=𝐿[Ψ(𝐹𝐵𝑃,𝑞_𝑑,𝐷𝐻𝑃)∗𝑈^𝑞𝑑_𝑘=0(𝐹𝐵𝑃)]=𝐺[Ψ(𝑝₃,𝑞_𝑑,𝑝₄)∗𝑈^𝑞𝑑_𝑘=0(𝑝₃)]=3.71 kcal/mol.
The Conc-Path is calculated at 𝑝₅.
(i)𝑈^𝑞𝑑_𝑘=0(𝐺𝐴𝑃)=𝐿[Ψ(𝐹𝐵𝑃,𝑞_𝑑,𝐺𝐴𝑃)∗𝑈^𝑞𝑑_𝑘=0(𝐹𝐵𝑃)]𝐿[Ψ(𝐷𝐻𝑃,𝑞_𝑒,𝐺𝐴𝑃)∗𝑈^𝑞𝑒_𝑘=0(𝐷𝐻𝑃)]=𝐺[Ψ(𝑝₃,𝑞_𝑑,𝑝₅)∗𝑈^𝑞𝑑_𝑘=0(𝑝₃)]𝐺[Ψ(𝑝₄,𝑞_𝑒,𝑝₅)∗𝑈^𝑞𝑒_𝑘=0(𝑝₄)]=4.00kcal/mole.(ii)𝑈^𝑞𝑓_𝑘=0(𝐵𝑃𝐺)=𝐿[Ψ(𝐺𝐴𝑃,𝑞_𝑓,𝐵𝑃𝐺)∗𝑈^𝑞𝑓_𝑘=0(𝐺𝐴𝑃)]=𝐺[Ψ(𝑝₅,𝑞_𝑓,𝑝₆)∗𝑈^𝑞𝑓_𝑘=0(𝑝₅)]=3.71 kcal/mol.(iii)𝑈^𝑞𝑔_𝑘=0(3𝑃𝐺)=𝐿[Ψ(𝐵𝑃𝐺,𝑞_𝑔,3𝑃𝐺)∗𝑈^𝑞𝑔_𝑘=0(𝐵𝑃𝐺)]=𝐺[Ψ(𝑝₆,𝑞_𝑔,𝑝₇)∗𝑈^𝑞𝑔_𝑘=0(𝑝₆)]=4.10 kcal/mol.(iv)𝑈^𝑞ℎ_𝑘=0(2𝑃𝐺)=𝐿[Ψ(3𝑃𝐺,𝑞_ℎ,2𝑃𝐺)∗𝑈^𝑞ℎ_𝑘=0(3𝑃𝐺)]=𝐺[Ψ(𝑝₇,𝑞_ℎ,𝑝₈)∗𝑈^𝑞ℎ_𝑘=0(𝑝₇)]=4.20 kcal/mol.(v)𝑈^𝑞𝑖_𝑘=0(𝑃𝐸𝑃)=𝐿[Ψ(2𝑃𝐺,𝑞_𝑖,𝑃𝐸𝑃)∗𝑈^𝑞𝑖_𝑘=0(2𝑃𝐺)]=𝐺[Ψ(𝑝₈,𝑞_𝑖,𝑝₉)∗𝑈^𝑞𝑖_𝑘=0(𝑝₈)]=4.00 kcal/mol.(vi)𝑈^𝑞𝑗_𝑘=0(Pyruvate)=𝐿[Ψ(𝑃𝐸𝑃,𝑞_𝑗,Pyruvate)∗𝑈^𝑞𝑗_𝑘=0(𝑃𝐸𝑃)]=𝐺[Ψ(𝑝₉,𝑞_𝑗,𝑝₁₀)∗𝑈^𝑞𝑗_𝑘=0(𝑝₉)]=0 kcal/mol.(vii)𝑈^𝑞𝑗_𝑘=0(Pyruvate)=0was set arbitrarily at the end of the pathway, that is, after the pyruvate kinase step.

Remark 5. We are using [] to denote the OR-Path, ∑ to denote the AND-Path, and {| to denote the Conc-Path.

4. Properties of the Model

We will identify the global system properties of the DPPN as those properties related with the PN. Theorem 1. The decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is bounded by a place 𝑝^∗ of the system.Proof. Let us suppose that the DPPN is not finite. Then 𝑝^∗ is never reached. Therefore, it is possible to evolve in time 𝑛 and to reduce the trajectory function value over 𝑝^∗. However, the Lyapunov-like trajectory function converges to zero when 𝑛→∞ (or reached a minimum), that is, 𝑈_𝑛=0 or 𝑈_𝑛=𝐶.Theorem 2. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a decision process Petri net bounded by a place 𝑝^∗. Then, a Lyapunov-like trajectory function can be constructed if and only if 𝑝^∗ is reachable from 𝑠₀.Proof. (⇒) If 𝑈 is a Lyapunov-like function then by the previous theorem 𝑝^∗ is reachable.
(⇐) By induction, let us construct the optimal inverse path from 𝑝^∗ to 𝑝₀. At each discrete time 𝑛∈ℕ in descending order (𝑛 is the maximum place index) the place of a system 𝑝_𝑛 is observed and a transition 𝑞_𝑘∈𝑄 leading to 𝑝_𝑛−1 is chosen. We choose the trajectory function 𝑈 as the best choice set of states. We continue this process until 𝑝₀ is reached. Then, the trajectory function 𝑈 is a Lyapunov-like function.

Notation.

Let ℕ={0,1,2,…}, ℕ^𝑛0₊={𝑛₀,𝑛₀+1,…,𝑛₀+𝑘,…}, 𝑛₀≥0, ℝ=(−∞,∞) and ℝ₊=[0,∞).

Let us consider systems of first ordinary difference equations given by𝑥(𝑛+1)=𝜓[𝑛,𝑥(𝑛)]𝑥(𝑛₀)=𝑥₀for𝑛∈ℕ^𝑛0₊, (8) where 𝑥(𝑛)∈ℝ^𝑑 and 𝜓∶ℕ^𝑛0₊×ℝ^𝑑→ℝ^𝑑 is continuous in 𝑥(𝑛). Definition 3. The 𝑛-vector valued function 𝜙(𝑛,𝑛₀,𝑥₀) is a solution of (8) if 𝜙(𝑛₀,𝑛₀,𝑥₀)=𝑥₀ and 𝜙(𝑛+1,𝑛₀,𝑥₀)=𝜓(𝑛,𝜙(𝑛,𝑛₀,𝑥₀)) for all 𝑛∈ℕ^𝑛0₊.Definition 4. The system (8) is said to be (see [20, 21]) practically stable if, given (𝜆,𝐴) with 0<𝜆<𝐴, it holds that|𝑥₀|<𝜆⇒|𝑥(𝑛,𝑛₀,𝑥₀)|<𝐴,∀𝑛∈ℕ^𝑛0₊,𝑛₀≥0. (9)Definition 5. The system (8) is said to be (see [20, 21]) uniformly practically stable, if it is practically stable for every 𝑛₀≥0.Definition 6. A continuous function 𝛼∶[0,∞)→[0,∞) belongs to class 𝒦 if it is strictly increasing and 𝛼(0)=0.

Let us consider [21] the vector function 𝑣(𝑛,𝑥(𝑛)), 𝑣∶ℕ^𝑛0₊×ℝ^𝑑→ℝ^𝑝₊ and let us define the variation of 𝑣 relative to (8) byΔ𝑣=𝑣(𝑛+1,𝑥(𝑛+1))−𝑣(𝑛,𝑥(𝑛)). (10)

Then, we have the following results [20, 21, 31, 32]. Theorem 3. Let 𝑣∶ℕ^𝑛0₊×ℝ^𝑛→ℝ₊ be a continuous function in 𝑥, such that for 𝛽,𝛼∈𝒦, it holds that 𝛽(|𝑥|)≤𝑣(𝑛,𝑥(𝑛))≤𝛼(|𝑥|) and Δ𝑣(𝑛,𝑥(𝑛))≤𝑤(𝑛,𝑣(𝑛,𝑥(𝑛))) holds for 𝑛∈ℕ^𝑛0₊, 𝑥(𝑛)∈ℝ^𝑛, where 𝑤∶ℕ^𝑛0₊×ℝ₊→ℝ is a continuous function in the second argument. Suppose that 𝛾(𝑛,𝑢)≡𝑢+𝑤(𝑛,𝑢) is nondecreasing in 𝑢, 0<𝜆<𝐴 are given and finally that 𝛼(𝜆)<𝛽(𝐴) is satisfied. Then, the stability properties of𝑢(𝑛+1)=𝛾(𝑛,𝑢(𝑛)),𝑢(𝑛₀)=𝑢₀≥0 (11)imply the corresponding stability properties of the system (8).Proof. The stability properties are preserved for the following.
(1) Practically stable. Let us suppose that 𝑢(𝑛+1) is practically stable for (𝛼(𝜆),𝛽(𝐴)) then, we have that 𝑢₀<𝛼(𝜆)⇒|𝑢(𝑛,𝑛₀,𝑢₀)|<𝛽(𝐴) for 𝑛≥𝑛₀, where 𝑢(𝑛,𝑛₀,𝑢₀) is the solution of (11). Let |𝑥₀|<𝜆, we claim that |𝑥(𝑛,𝑛₀,𝑥₀)|<𝐴 for 𝑛≥𝑛₀. If not, there would exist 𝑛₁≥𝑛₀ and a solution 𝑥(𝑛,𝑛₀,𝑥₀) such that |𝑥(𝑛₁)|≥𝐴 and |𝑥(𝑛)|<𝐴 for 𝑛₀≤𝑛<𝑛₁. Choose 𝑢₀=𝑣(𝑛₀,𝑥₀), then 𝑣(𝑛,𝑥(𝑛))≤𝑢(𝑛) for all 𝑛≥𝑛₀. (If not 𝑣(𝑛,𝑥(𝑛))≤𝑢(𝑛) and 𝑣(𝑛+1,𝑥(𝑛+1))>𝑢(𝑛+1)⇒𝛾(𝑛,𝑢(𝑛))=𝑢(𝑛+1)<𝑣(𝑛+1,𝑥(𝑛+1))=Δ𝑣(𝑛,𝑥₀)+𝑣(𝑛,𝑥(𝑛))≤𝑤(𝑛,𝑣(𝑛))+𝑣(𝑛,𝑥(𝑛))=𝛾(𝑛,𝑣(𝑛))−𝑣(𝑛,𝑥(𝑛))+𝑣(𝑛,𝑥(𝑛))≤𝛾(𝑛,𝑢(𝑛)) which is a contradiction). Hence we get that 𝛽(𝐴)≤𝛽(|𝑥(𝑛₁)|)≤𝑣(𝑛₁,𝑥(𝑛₁))≤𝑢(𝑛₁,𝑛₀,𝑢₀)<𝛽(𝐴) (where the last inequality is because the condition |𝑥₀|<𝜆⇒𝑣(𝑛₀,𝑥₀)<𝛼(𝜆)), which cannot hold therefore, system (8) is practically stable.
(2) Stable. Suppose that system (11) is stable, that is, for all 𝜀>0∃𝜉=𝜉(𝜀,𝑛₀)>0 such that if 𝑢₀<𝜉⇒|𝑢(𝑛,𝑛₀,𝑢₀)|<𝛽(𝜀) for 𝑛≥𝑛₀. Now, since 𝑣 is a continuous function in 𝑥, there exists a 𝛿=𝛿(𝜀,𝑛₀)>0 such that if |𝑥₀|<𝛿⇒|𝑣(𝑛₀,𝑥₀)|<𝜉 then setting 𝑣(𝑛₀,𝑥₀) equal to 𝑢₀ by the comparison principle (which was implicitly proved in point 1) implies that 𝑣(𝑛,𝑥(𝑛))≤𝑢(𝑛) for all 𝑛≥𝑛₀. Taking 𝛿 equal to the one given from the continuity of 𝑣, |𝑥(𝑛,𝑛₀,𝑥₀)|<𝜀 for 𝑛≥𝑛₀. If not, there would exist 𝑛₁≥𝑛₀ such that |𝑥(𝑛₁)|≥𝜀 and |𝑥(𝑛)|<𝜀 for 𝑛₀≤𝑛<𝑛₁ but then𝛽(𝜀)≤𝛽(𝑥(𝑛₁))≤𝑣(𝑛₁,𝑥(𝑛₁))≤𝑢(𝑛₁,𝑛₀,𝑢₀)<𝛽(𝜀) (12)which cannot hold therefore, we must have that |𝑥(𝑛,𝑛₀,𝑥₀)|<𝜀 for 𝑛≥𝑛₀ as desired.
(3) Asymptotically stable. We know that system (8) is stable, the fact that it is asymptotically stable follows thanks to0≤lim_𝑛→∞𝛽(|𝑥_𝑛|)≤lim_𝑛→∞𝑣(𝑛,𝑥(𝑛))≤lim_𝑛→∞𝑢(𝑛)=0⇒lim_𝑛→∞|𝑥_𝑛|=0. (13)
(4) Uniformly stable. Assume that the comparison system is uniformly stable, meaning that ∃𝜉=𝜉(𝜀)>0 (independent of 𝑛) such that 𝑢₀<𝜉⇒|𝑢(𝑛,𝑛₀,𝑢₀)|<𝛽(𝜀) for 𝑛≥𝑛₀ and let 𝛿>0 independent of 𝑛 such that |𝑥₀|<𝛿⇒|𝑥(𝑛,𝑛₀,𝑥₀)|<𝜀, for 𝑛≥𝑛₀. Since 𝑣 is a decreasing function there exists 𝑎𝛼𝜖𝒦 such that 𝑣(𝑛,𝑥(𝑛))≤𝛼(|𝑥_𝑛|). Then, choosing 𝛿=𝛼⁻¹(𝜉) works (if |𝑥₀|<𝛿=𝛼⁻¹(𝜉)⇒𝑣(𝑛₀,𝑥₀)=|𝑣(𝑛₀,𝑥₀)|≤𝛼(|𝑥₀|)<𝜉) and choosing 𝑢₀=𝑣(𝑛₀,𝑥₀) we arrive to the inequality𝛽(|𝑥(𝑛)|)≤𝑣(𝑛,𝑥(𝑛))≤𝑢(𝑛,𝑛₀,𝑢₀)<𝛽(𝜀). (14)But 𝛿 is independent of 𝑛. Therefore, the system (8) is uniformly stable.

We will extend the last theorem to the case of several Lyapunov functions. Let us consider a vector Lyapunov function 𝑣(𝑛,𝑥(𝑛)), 𝑣∶ℕ^𝑛0₊×ℝ^𝑑→ℝ^𝑝₊ and let us define the variation of 𝑣 relative to (8). Then, we have the following theorem. Theorem 4. Let 𝑣∶ℕ^𝑛0₊×ℝ^𝑑→ℝ^𝑝₊ be a continuous function in 𝑥, define the function 𝑣₀(𝑛,𝑥(𝑛))=∑^𝑝_𝑖=1𝑣_𝑖(𝑛,𝑥(𝑛)) such that it satisfies the estimates:𝛽(|𝑥|)≤𝑣₀(𝑛,𝑥(𝑛))≤𝛼(|𝑥|)for𝛼,𝛽∈𝒦,Δ𝑣(𝑛,𝑥(𝑛))≤𝑤(𝑛,𝑣(𝑛,𝑥(𝑛))) (15)for 𝑛∈ℕ^𝑛0₊, 𝑥(𝑛)∈ℝ^𝑑, where 𝑤∶ℕ^𝑛0₊×ℝ^𝑝₊→ℝ^𝑝 is a continuous function in the second argument. Assume that 𝛾(𝑛,𝑢)≜𝑢+𝑤(𝑛,𝑢) is nondecreasing in 𝑢, 0<𝜆<𝐽 are given and 𝛼(𝜆)<𝛽(𝐴) is satisfied. Then, the practical stability properties of𝑢(𝑛+1)=𝛾(𝑛,𝑢(𝑛)),𝑢(𝑛₀)=𝑢₀≥0 (16)implies the corresponding practical stability properties of system (8).Proof. (1) Let us suppose that 𝑢(𝑛+1) is practically stable for (𝛼(𝜆),𝛽(𝐴)). Then we have that ∑^𝑝_𝑖=1𝑢_0𝑖<𝛼(𝜆)⇒∑^𝑝_𝑖=1𝑢_𝑖(𝑛,𝑛₀,𝑢₀)<𝛽(𝐴), for 𝑛≥𝑛₀, where 𝑢_𝑖(𝑛,𝑛₀,𝑢₀) is the vector solution of (16). Let |𝑥₀|<𝜆, we claim that |𝑥(𝑛,𝑛₀,𝑥₀)|<𝐴 for 𝑛≥𝑛₀. If not, there would exist 𝑛₁≥𝑛₀ and a solution 𝑥(𝑛,𝑛₀,𝑥₀) such that |𝑥(𝑛₁)|≥𝐴 and |𝑥(𝑛)|<𝐴 for 𝑛₀≤𝑛<𝑛₁. Choose 𝑢₀=𝑣(𝑛₀,𝑥₀), then 𝑣(𝑛,𝑥(𝑛))≤𝑢(𝑛,𝑛₀,𝑢₀) for all 𝑛≥𝑛₀. Therefore we have that 𝛽(𝐴)≤𝛽(|𝑥(𝑛₁)|)≤𝑣₀(𝑛₁,𝑥(𝑛₁))≤∑^𝑝_𝑖=1𝑢_𝑖(𝑛₁,𝑛₀,𝑢₀)<𝛽(𝐴) which cannot hold. As a result, system (8) is practically stable.
(2) From the continuity of 𝑣 with respect to the second argument, it is always possible to make 𝑣₀(𝑛₀,𝑥₀)<∑^𝑝_𝑖=1𝑢_0𝑖<𝛼(𝜆)⇒𝑣₀(𝑛,𝑥(𝑛))≤∑^𝑝_𝑖=1𝑢_𝑖(𝑛,𝑛₀,𝑢₀)<𝛽(𝐴). We want to prove that |𝑥(𝑛,𝑛₀,𝑥₀)|<𝐴 for 𝑛≥𝑛₀. If it is not true, there exists an 𝑛₁≥𝑛₀ and a solution 𝑥(𝑛,𝑛₀,𝑥₀) such that |𝑥(𝑛₁)|≥𝐴 and |𝑥(𝑛)|<𝐴 for 𝑛₀≤𝑛<𝑛₁. Then, we have that 𝛽(𝐴)≤𝛽(|𝑥(𝑛₁)|)≤𝑣₀(𝑛₁,𝑥(𝑛₁))≤∑^𝑝_𝑖=1𝑢_𝑖(𝑛₁,𝑛₀,𝑢₀)<𝛽(𝐴)!, which proves our claim.Remark 6. If in the point 1 of the proof it is not true that 𝑣(𝑛,𝑥(𝑛))≤𝑒(𝑛,𝑛₀,𝑒₀) and 𝑣(𝑛+1,𝑥(𝑛+1))>𝑒(𝑛+1,𝑛₀,𝑒₀), then we have that 𝛾(𝑛,𝑒(𝑛))=𝑒(𝑛+1,𝑛₀,𝑒₀)<𝑣(𝑛+1,𝑥(𝑛+1))=Δ𝑣(𝑛,𝑥₀)+𝑣(𝑛,𝑥(𝑛))≤𝑤(𝑛,𝑣(𝑛))+𝑣(𝑛,𝑥(𝑛))=𝛾(𝑛,𝑣(𝑛))−𝑣(𝑛,𝑥(𝑛))+𝑣(𝑛,𝑥(𝑛))=𝛾(𝑛,𝑣(𝑛))≤𝛾(𝑛,𝑒(𝑛)) which is a contradiction.

Then, we have the following result [21]. Corollary 1. From Theorem 5, the following hold.
(1) If 𝑤(𝑛,𝑒)≡0, the uniform practical stability of (8) which implies structural stability [21, 33] is obtained.(2) If 𝑤(𝑛,𝑒)=−𝑐(𝑒), for 𝑐∈𝒦, the uniform practical asymptotic stability of (8) [21] is obtained.Example 7. The diamond is the stable form of carbon at extremely high pressures while the graphite is the stable form at normal atmospheric pressures. Regardless of that, diamonds appear stable at normal temperatures and pressures, but, in fact, are very slowly converting to graphite. Heat increases the rate of this transformation, but at normal temperatures the diamond is uniformly practically stable.

For Petri nets, we have the following results of stability [31]. Proposition 1. Let 𝑃𝑁 be a Petri net. Therefore, 𝑃𝑁 is uniform practical stable if there exists a Φ strictly positive 𝑚 vector such thatΔ𝑣=𝑢^𝑇𝐴Φ≤0. (17)Moreover, 𝑁 is uniform practical asymptotic stability if the following equation holds:Δ𝑣=𝑢^𝑇𝐴Φ≤−𝑐(𝑒),for𝑐∈𝒦. (18)Proof. Let us choose as our candidate Lyapunov function 𝑣(𝑀)=𝑀^𝑇Φ with Φ and 𝑚 vector to be chosen. It is simple to verify that 𝑣 satisfies all the conditions of Theorem 3. Therefore, the uniform practical asymptotic stability is obtained if there exists a strictly positive vector Φ such that equation (17) holds.Proposition 2. Let 𝑃𝑁 be a Petri net. Therefore, 𝑃𝑁 is uniformly practically stable if there exists a Φ strictly positive 𝑚 vector such thatΔ𝑣=𝑢^𝑇𝐴Φ≤0⟺𝐴Φ≤0. (19)Proof. ⇒) Since 𝑢^𝑇𝐴Φ≤0 holds, therefore for every 𝑢 we have that 𝐴Φ≤0.
⇐) This came from the fact that 𝑢 is positive.Remark 7. The if-and-only-if relationship of (19) exists from the fact that 𝑢 is positive.Example 8. The biochemical pathway of the glycolysis (Figure 1). The incidence matrix is as follows:[−110000000000−110000000000−110000000000−111000000000−110000000000−110000000000−110000000000−110000000000−110000000000−11]. (20) Choosing Φ=[1,1,1,1,1/2,1/2,1/2,1/2,1/2,1/2,1/2], Φ>0, we obtain that 𝐴Φ=[0,0,0,−1/2,0,0,0,0,0,0] concluding stability. Definition 7. An equilibrium point with respect to a decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is a place 𝑝^∗∈𝑃 such that 𝑀_𝑙(𝑝^∗)=𝑆<∞, for all 𝑙≥𝑘, and 𝑝^∗ is a sink.Theorem 5. The decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is uniformly practically stable iff there exists a Φ strictly positive 𝑚 vector such that Δ𝑣=𝑢^𝑇𝐴Φ≤0.Proof. ⇒) It follows directly from Propositions 1 and 2.
⇐) Let us suppose by contradiction that 𝑢^𝑇𝐴Φ>0 with Φ fixed. From 𝑀^′=𝑀+𝑢^𝑇𝐴 we have that 𝑀^′Φ=𝑀Φ+𝑢^𝑇𝐴Φ>𝑀Φ. Then, it is possible to construct an increasing sequence 𝑀Φ<𝑀^′Φ<⋯<𝑀^𝑛Φ<⋯ which grows up without bound. Therefore, the DDPN is not uniformly practically stable.Remark 8. It is important to underline that the only places where the DPPN will be allowed to get blocked are those which correspond to equilibrium points.

We will identify the trajectory-tracking properties of the DPPN as those properties related with the trajectory-tracking value at each place of the PN. In this sense, we will relate an optimum point the best possible performance choice. Formally we will introduce the following definition [23]. Definition 8. A final decision point 𝑝_𝑓∈𝑃 with respect to a decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is a place 𝑝∈𝑃 where the infimum is asymptotically approached (or the minimum is attained), that is, 𝑈(𝑝)=0 or 𝑈(𝑝)=𝐶.Definition 9. An optimum point 𝑝^Δ∈𝑃 with respect to a decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is a final decision point 𝑝_𝑓∈𝑃 where the best choice is selected “according to some criteria.”

Property 3. Every decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} has a final decision point.

Remark 9. In case that ∃𝑝₁,…,𝑝_𝑛∈𝑃, such that 𝑈(𝑝₁)=⋯=𝑈(𝑝_𝑛)=0, then 𝑝₁,…,𝑝_𝑛 are optimum points.Remark 10. The monotonicity of 𝑈 guarantees that it is possible to make the search starting from the decision points.

Then, we can conclude the following theorem. Theorem 6. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a finite decision process Petri net and let (𝑝₀,𝑝₁,…,𝑝_𝑛) be a realized trajectory which converges to 𝑝^Δ such that ∃𝜖_𝑖∶|𝑈_𝑖+1−𝑈_𝑖|>𝜖_𝑖(with𝜖_𝑖>0). Let 𝜖=min{𝜖_𝑖}, then the optimum decision point 𝑝^Δ is reached in a time step bounded by 𝑂(𝑈₀/𝜖).Proof. Let us suppose that 𝑝^Δ is never reached, then, 𝑝^Δ is not a sink (the last place) in the decision process Petri net. So, it is possible to find some output transition to 𝑝^Δ. Therefore, it is possible to reduce the trajectory function value over 𝑝^Δ by at least 𝜖. As a result, it is possible to obtain a lower value than 𝐶 (that is a contradiction).Theorem 7. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a decision process Petri net. Then, 𝑈 converges to an optimum (final) decision point 𝑝^Δ(𝑝_𝑓).Proof. We have to show that 𝑈 converges to an optimum (final) decision point 𝑝^Δ(𝑝_𝑓). By the previous theorem, the optimum decision point 𝑝^Δ is reached in a time step bounded by 𝑂(𝑈₀/𝜖), therefore 𝑈 converges to 𝑝^Δ.Proposition 3. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a decision process Petri net and let 𝑝^Δ∈𝑃 be an optimum point. Then 𝑈(𝑝^Δ)≤𝑈(𝑝), for all 𝑝∈𝑃 such that 𝑝≤_𝑈𝑝^Δ.Proof. We have that 𝑈(𝑝^Δ) is equal to the minimum or the infimum. Therefore, 𝑈(𝑝^Δ)≤𝑈(𝑝) for all 𝑝∈𝑃 such that 𝑝≤_𝑈𝑝^Δ.Theorem 8. The decision process Petri net DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} is uniformly practically stable iff 𝑈(𝑝_𝑖+1)−𝑈(𝑝_𝑖)≤0.Proof. (⇒) Let us choose 𝑣=𝐼𝑑(𝑈(𝑝_𝑖)), then Δ𝑣=𝑈(𝑝_𝑖+1)−𝑈(𝑝_𝑖)≤0. Then by the autonomous version of Theorem 4 and Corollary 1 the DPPN is stable.
(⇐) We want to show that the DPPN is practically stable, that is, given 0<𝜆<𝐴, we must show that |𝑈(𝑝_𝑖)|<𝐴. We know that 𝑈(𝑝₀)<𝜆 and since 𝑈 is nondecreasing, we have that |𝑈(𝑝_𝑖)|<|𝑈(𝑝₀)|<𝜆<𝐴.Theorem 9. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a decision process Petri net. If 𝑝^∗∈𝑃 is an equilibrium point, then it is a final decision point. Proof. Let us suppose that 𝑝^∗ is an equilibrium point, we want to show that its trajectory-tracking value has asymptotically approached an infimum (or reached a minimum). Since 𝑝^∗ is an equilibrium point, by definition, it is bounded and it is a sink, for example, its marking can not be modified. But, this implies that the routing policy attached to the transition(s) that follows 𝑝^∗ is 0 (in case there is such a transition(s), i.e., worst case). Therefore, its trajectory-tracking value can not be modified and since the value is a decreasing function of 𝑝_𝑖, an infimum or a minimum is attained. Then, 𝑝^∗ is a final decision point.Theorem 10. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a finite and nonblocking decision process Petri net (unless 𝑝∈𝑃 is an equilibrium point). If 𝑝_𝑓∈𝑃 is a final decision point, then it is an equilibrium point. Proof. If 𝑝_𝑓 is a final decision point, since the DDPN is finite, there exists a 𝑘 such that 𝑈_𝑘(𝑝_𝑓)=𝐶. Let us suppose that 𝑝_𝑓 is not an equilibrium point.
Case 1. Then, it is not bounded. So, it is possible to increment the marks of 𝑝_𝑓 in the net. Therefore, it is possible to modify its trajectory-tracking value. As a result, it is possible to obtain a lower value than 𝐶.
Case 2. Then, it is not bounded and it is not a sink. So, it is possible to fire some output transition to 𝑝_𝑓 in such a way that its marking is modified. Therefore, it is possible to modify the trajectory-tracking value over 𝑝_𝑓. As a result, it is possible to obtain a lower trajectory-tracking value than 𝐶.Corollary 2. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a finite and nonblocking decision process Petri net (unless 𝑝∈𝑃 is an equilibrium point). Then, an optimum point 𝑝^Δ∈𝑃 is an equilibrium point.Proof. From the previous theorem, we know that a final decision point is an equilibrium point and since in particular 𝑝^Δ is final decision point, then it is an equilibrium point.

5. Completeness

Theorem 11. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a decision process Petri net and let (𝑝₀,𝑝₁,…,𝑝_𝑛) be a realized trajectory which converges to 𝑝^∗ such that ∃𝜖_𝑖∶|𝑈_𝑖+1−𝑈_𝑖|>𝜖_𝑖(with𝜖_𝑖>0). Let 𝜖=min{𝜖_𝑖}, then an optimum point 𝑝^∗ is reached in a time step bounded by 𝑂(𝑈₀/𝜖).Proof. Let us suppose that 𝑝^∗ is never reached, then 𝑝^∗ is not the last place in the decision process Petri net. So, it is possible to find some output transition to 𝑝^∗. Therefore, it is possible to reduce the trajectory function value over 𝑝^∗ by at least 𝜖. As a result, it is possible to obtain a lower value than 𝐶 (that is a contradiction).Remark 11. The complexity time 𝑂(𝑈₀/𝜖) differs with that of the Dijkstra's algorithm.Remark 12. Each path in DDPN corresponds to a trajectory of/in a given system. The trajectory-tracking function value of 𝑈 at the source place (𝑈₀^𝑚𝑜𝑛) divided by 𝜖=min{𝜖_𝑖} equals the length of the shortest-path. Then, the infimum is equivalent to the infimum length over all paths in DDPN.Theorem 12. Let DDPN={𝑃,𝑄,𝐹,𝑊,𝑀₀,𝜋,𝑈} be a decision process Petri net. Then, 𝑈 converges to a point 𝑝^∗.Proof. We have to show that 𝑈 converges to a point 𝑝^∗. By the previous theorem, the optimum point 𝑝^∗ is reached in a time step bounded by 𝑂(𝑈₀/𝜖), therefore 𝑈 converges to 𝑝^∗.Proposition 4. The finite and nonblocking (unless 𝑝∈𝑃 is an equilibrium point) condition over the DDPN can not be relaxed.Proof. (1) Let us suppose that the DDPN is not finite, that is, 𝑝 is in a cycle, then the Lyapunov-like function converges when 𝑘→∞, to zero, that is, 𝐿(𝑝)=0 but the DPPN has no final place therefore, it is not an equilibrium point.
(2) Let us suppose that the DDPN blocks at some place (not an equilibrium point) 𝑝∈𝑃. Then, the Lyapunov-like function has a minimum at place 𝑝, lets say 𝐿(𝑝)=𝐶 but 𝑝 is not an equilibrium point, because it is not necessary to have a sink in the net.

6. Conclusions

In this work, a formal framework for shortest-path decision process problem representation has been presented. Whereas in previous work, attention was restricted to tracking the net using a utility function Bellman's equation, this work uses a Lyapunov-like function. In this sense, we are changing the traditional cost function by a trajectory-tracking function which is also an optimal cost-to-target function for tracking the net. This makes a significant difference in the conceptualization of the problem domain. The Lyapunov method introduces a new equilibrium and stability concept in decision process.