#### Abstract

In a quantum Markov chain, the temporal succession of states is modeled by the repeated action of a “bistochastic quantum operation” on the density matrix of a quantum system. Based on this conceptual framework, we derive some new results concerning the evolution of a quantum system, including its long-term behavior. Among our findings is the fact that the Cesàro limit of any quantum Markov chain always exists and equals the orthogonal projection of the initial state upon the eigenspace of the unit eigenvalue of the bistochastic quantum operation. Moreover, if the unit eigenvalue is the only eigenvalue on the unit circle, then the quantum Markov chain converges in the conventional sense to the said orthogonal projection. As a corollary, we offer a new derivation of the classic result describing limiting distributions of unitary quantum walks on finite graphs (Aharonov et al., 2001).

#### 1. Introduction

The theory of Markov chains, when appropriately generalized, provides a potent paradigm for analyzing the stochastic evolution of quantum systems. Over the past decade, motivated largely by the prospect of superefficient algorithms, the theory of so-called *quantum Markov chains*, especially in the guise of quantum walks, has generated a huge volume of research, including many discoveries of fundamental importance, such as those in [1–10] and numerous recent advances, such as those in [11–15].

In the context of quantum walks, the itinerary of the walker is confined to a particular topological network. The walker's every move, from node to adjacent node, is governed by a set of local rules. When applied repeatedly to a given initial state of the system (represented by a superposition of basis states), these rules yield a succession of new states, reflecting, ad infinitum, the evolution of the system. A transition rule can be either unitary (closed) or nonunitary (open), depending, respectively, on whether it is intrinsic to the system or exposes the system to external influences such as decoherence, noise, or measurement.

In this paper, we adopt the formalism of “quantum operations,” whereby both unitary and nonunitary rules of state transition, as well as various combinations thereof, are treated under a unified mathematical model. In this framework, the “transition matrix” of a classical Markov chain is replaced by a “bistochastic quantum operation,” and the “state distribution vector” of the classical Markov chain is replaced by a “density matrix.” The resulting description of quantum state evolution, known as a *quantum Markov chain* [1, 16], turns out to resemble very closely the evolution of a classical Markov chain. In particular, the same kind of questions pertain, including questions about the long-term evolution of the process and the possible existence of a limiting state.

Among our findings is the fact that the Cesàro limit of any quantum Markov chain converges always to a stationary “state,” regardless of the initial state. As a noteworthy special case of this result, we remark that for any unitary quantum walk on a graph, as in [1], the limit of the time-averaged probability distribution always exists.

To complete the picture, we specify conditions for the existence of a limiting state in the strict (non-Cesàro) sense of the word “limiting.” In the strict sense, it turns out that the limiting behavior depends only on the deployment on the unit disc of the eigenvalues of the bistochastic quantum operation. Specifically, if is the only eigenvalue on the unit circle, then, for any given initial state, the associated quantum Markov chain converges to a stationary state. Moreover, if the eigenspace of is one-dimensional or contains only a single density operator, then the associated Markov chain converges to the maximally mixed state, irrespective of the initial state. Otherwise, if the bistochastic quantum operation possesses any eigenvalue on the unit circle other than , then a limiting state is not guaranteed to exist except in the generalized sense of Cesàro. These findings are seen to be analogous, in a very natural way, to the fundamental properties of classical Markov chains (e.g., in [17, Chapter 8]).

Our results may represent substantial progress toward answering the first of a set of “open questions” posed by Ambainis [16]. We would be remiss not to acknowledge our indebtedness to [18], which treats the special case of a quantum Markov chain generated by a *random unitary operation*.

In what follows, after a brief review of some preliminaries (Section 2), we proceed (Section 3) to present our main results on the question of the limiting behavior of a quantum Markov chain. The proofs of the results given in Section 3 are deferred to Section 4. In Section 5, we present a comprehensive classification of quantum operations according to their limiting behavior. Finally, in Section 6, we offer some concluding remarks, including some relevant questions for further investigation.

#### 2. Bistochastic Quantum Operations

Given a Hilbert space of finite dimension , let denote the set of all linear operators on , with inner product defined by

The corresponding norm, called *Frobenius norm* or *Schatten* 2-* norm*, is defined by
This choice of norm on will remain in effect throughout this paper.

Let denote the set of * positive* operators with Tr. The operators are the so-called “density operators.” They serve to model, as faithfully as do the “state vectors” themselves, the possible states of a quantum system whose state vectors reside in .

By a *superoperator * on , we mean a linear mapping , with norm defined by

Note that , where . Thus, any superoperator on can be represented, relative to a given basis for , by an matrix. In the sequel, this matrix will be denoted by the symbol . In particular, relative to a special basis consisting of eigenvectors and generalized eigenvectors of , the shape of the matrix conforms to a special quasidiagonal lay-out called the Jordan canonical form. The details can be found in any one of a number of sources, including [18].

Among the set of superoperators, we distinguish a special subset called “quantum operations”. By definition, to qualify as a quantum operation, the superoperator must be *completely positive*, meaning that the extended map is positive for all .

The formalism of quantum operations is flexible enough to handle both unitary (closed) and nonunitary (open), or a mixture thereof, of discrete transitions of state of a quantum system. For a good introductory exposition of this subject, see [19, 20].

By Choi's Theorem [19–22], any completely positive linear operator, including any quantum operation , can be represented in terms of a set of “Kraus operators” as follows: In this expression, which we call the “Choi expansion” of , the symbol denotes (transpose of the complex conjugate of ).

In terms of the Choi expansion, the condition of being *trace preserving*, meaning that for all , is equivalent to the condition

On the other hand, if the Kraus operators of satisfy the dual condition
then is said to be* unital*. Note that (2.6) is equivalent to the simple statement that .

A quantum operation which is both unital and trace preserving is called *bistochastic*. The term *doubly stochastic quantum channel* also has been used to refer to quantum operations of this sort [23]. Note that a bistochastic quantum operation transforms elements of into elements of . In other words, since the elements of represent the states of a quantum system, a bistochastic quantum operation transforms states of a quantum system into other legitimate states of that system.

#### 3. Limit Theorems for Bistochastic Quantum Operations

In the sequel, the proofs of all theorems, corollaries and supporting lemmas are deferred to Section 4.

By [24], a bistochastic quantum operation must satisfy the condition . Thus the spectrum of is confined to the unit disk. Also, since , we see that is an eigenvalue of and . For future reference, we record these observations in the form of a lemma.

Lemma 3.1. *Let be a bistochastic quantum operation on the Hilbert space , then *(1)*; *(2)*if is an eigenvalue of , then ; *(3)*the value is an eigenvalue of . *

The observations recorded in Lemma 3.1 are new by no means. See, for instance, [25].

For an eigenvalue of , let and denote, respectively, the kernel and range of the operator on the Hilbert space .

Lemma 3.2. *Let be a bistochastic quantum operation on , and let be an eigenvalue of with . Then .*

From the preceding lemma, we can derive an important inference concerning the algebraic and geometric multiplicities of the eigenvalues of . For an eigenvalue of , let its algebraic multiplicity be denoted by , and let its geometric multiplicity be denoted by . Recall that . In addition, let the spectrum of be denoted by , and let denote the subset of consisting of with .

Lemma 3.3. *If , where is a bistochastic quantum operation on , then .*

As in the proof of Lemma 3.3, let denote the Jordan canonical matrix representation of . To conserve type-set space, and without undue risk of confusion to the reader, we prefer to display the matrix in the following self-explanatory format: where and , , denote the Jordan blocks corresponding to eigenvalues whose norms are strictly less than unity.

For an eigenvalue of , let denote the eigenspace of . In particular, for , it has been established in [26–28] that

The following lemma articulates the special status of relative to the other eigenspaces of .

Lemma 3.4. *Let be an eigenvalue of such that and . Let be an eigenvalue of with . Let denote the generalized eigenvectors belonging to . Then *(1)*;
*(2)*. *

In other words, by Lemma 3.4, the eigenspace is orthogonal to every eigenvector (including every generalized eigenvector) of belonging to eigenvalues other than .

Theorem 3.5. *Let be a bistochastic quantum operation on the Hilbert space , and let denote the density matrix representing the initial state of a quantum system. Let denote the geometric multiplicity of the eigenvalue , and let denote an orthonormal basis for . If is the only eigenvalue of on the unit circle, then the iterated succession of quantum states converges to . In particular, if , then , independently of the initial state . *

In previous publications, such as [29], the special case of a bistochastic quantum operation whereby is known as a *primitive quantum channel*. Reference [29] provides an excellent analysis of this line of investigation.

In the literature, numerous examples can be found of bistochastic quantum operations (a.k.a. quantum channels) for which the limiting behavior of the associated quantum Markov chain is governed by Theorem 3.5. Among these examples are (1) the *quantum walk* on a cycle, subject to decoherence on the degree freedom of coin [30] and (2) the generalized *depolarizing channel* on a multiple-dimensional quantum system [19, 20]. These examples can be treated as special cases of the following corollary to Theorem 3.5.

Corollary 3.6. *If the bistochastic quantum operation is given by
**
where and , then, as in Theorem 3.5, . *

In the context of quantum channels, the Kraus operators defining the generalized *depolarizing channel* can be scalar multiples of the discrete Weyl operators (or the generalized Pauli operators). When such is the case, as in, for instance, [23], the eigenspace of the eigenvalue 1 must be one-dimensional. Consequently the associated quantum Markov chain must converge to the maximally mixed state, regardless of the initial state.

The main assertion of Theorem 3.5 pertains only to bistochastic quantum operations possessing no eigenvalues on the unit circle other than . In general, when this condition is relaxed to admit other eigenvalues on the unit circle, the expression is compelled no longer to converge to a stationary state. Many examples can be found in the literature, including classic cases of unitary quantum walks on finite graphs [1].

However, even if a limiting state fails to exist in the usual sense, we still might want to probe the possibility of a “limiting state” in the sense of Cesàro:

In terms of this generalized sense of “limiting state,” it turns out that every quantum Markov chain converges.

Theorem 3.7. *Let be a bistochastic quantum operation on the Hilbert space , and let denote the density matrix representing the initial state of a quantum system. Let denote the geometric multiplicity of the eigenvalue , and let denote an orthonormal basis for . Then, for every ,
*

According to this theorem, the sequence of Cesàro means is guaranteed always to converge to a limit which coincides with the orthogonal projection of the initial state upon the eigenspace of the unit eigenvalue of the bistochastic quantum operation. It is only fair to point out that alternate versions of this result have appeared in the literature. For instance, in [26, Theorem 2.4(a)] it is shown that there exists a subsequence of the full sequence of Cesàro means which converges to a limit within the space of fixed points of the quantum operation. A closer match to Theorem 3.7, derived by different means and framed in different language, can be found in [31, Chapter 6].

As an immediate corollary of Theorem 3.7, it follows, as in [1], that the time-averaged probability distribution for any unitary quantum walk on a finite graph must converge. For a unitary quantum walk on a finite graph starting with an initial state , let denote the probability of finding the walker at node at time , and let denote the time-averaged probability.

Corollary 3.8 (see [1, Theorem 3.4]). * Let denote the unit eigenvectors and corresponding eigenvalues of the unitary operator associated with a unitary quantum walk on a finite graph, and let the bistochastic quantum operation be defined by . Then, for any initial state , one has
**
where the sum is restricted to pairs such that .*

#### 4. Proofs of the Theorems

This section is reserved for the proofs of theorems, lemmas, and corollaries given in Section 3.

*Proof of Lemma 3.2. *Suppose . Then and for some . Applying the linearity of , we infer that for all . Consequently , which implies that . But since (see Lemma 3.1), we have . Since this inequality must hold for all , we conclude that , whence .

In the above proof, we have borrowed liberally from the reasoning employed in [18], which treats the special case of a *random unitary operation*.

*Proof of Lemma 3.3. *We proceed by contradiction. Suppose . Let denote the matrix representation of in Jordan canonical form. On the one hand, must contain a Jordan block belonging to of size >1. On the other hand, by standard matrix theory, there must exist a generalized eigenvector, say , of such that is itself an eigenvector of . This implies that , which contradicts Lemma 3.2.

*Proof of Lemma 3.4. *The adjoint operator of is given by
Thus, by (3.2), we have .

To prove (1) in Lemma 3.4, let and . Then . Since , it follows that . Hence .

We proceed to justify (2) in Lemma 3.4. For an eigenvalue with , we may assume, without loss of generality, that the generalized eigenvectors belonging to are arranged in a sequence such that
where, by definition, . It follows that . If , then , which implies that . Similarly, , from which it follows that . Likewise, by the same reasoning, applied repeatedly, we deduce that for any . Thus .

*Proof of Theorem 3.5. *Let denote the eigenvalues of whose absolute values are less than 1. Since, by Lemma 3.3, is the only eigenvalue on the unit circle, a basis for might be assembled as follows. Starting with an orthonormal basis for eigenspace , append a maximal set of linearly independent generalized eigenvectors belonging to the eigenvalues , . Then, relative to this basis, the Jordan canonical matrix representation of is given by
where is the Jordan block corresponding to the eigenvalue , .

Consider what becomes of the Jordan blocks of the powers as . Since each of the Jordan blocks is an upper triangular matrix whose diagonal is populated by a single eigenvalue of modulus strictly less than unity, it is a simple exercise in elementary algebra to show that (zero matrix of same size as ). Thus, if we define
then .

Next, we consider the effect upon on the initial state of as .

By Lemma 3.4, we may choose for a basis consisting of a basis for together with an orthogonally complement basis consisting of generalized eigenvectors belonging to all other eigenvalues. In terms of such a basis we have , where for all . By simple linear algebra, it follows that for . Therefore, by (4.4), we have .

Finally, if , then, to serve as a normalized basis for , we may take , from which it follows that , for any initial state .

*Proof of Corollary 3.6. *It suffices to verify that is the only eigenvalue of on the unit circle. By [24], whenever , then for any operator . Now suppose is an eigenvalue of on the unit circle and is an eigenvector belonging to . Then , from which it follows that . Thus, all of the preceding inequalities actually are equalities. In particular, we have . Thus , which implies that . It follows that .

*Proof of Theorem 3.7. *To evaluate the limit in (3.4), we have only to reexamine (3.1). For every eigenvalue of , if , then . Similarly, for every Jordan block in (3.1), we have . Based on these two observations, we deduce that converges to the diagonal matrix
The expression for follows by a path of reasoning identical to that employed in the proof of Theorem 3.5 to obtain the analogous expression for the limiting state in that situation.

*Proof of Corollary 3.8. *As an orthonormal basis for the eigenspace , we may take the set of vectors , where the union is restricted to pairs such that . Accordingly, the orthogonal projection of the initial state into the eigenspace is
where, as above, the sum is restricted to pairs such that .

Since
and since is a continuous function of the argument, it follows, by Theorem 3.7, that converges to
where the sum is restricted to pairs such that .

#### 5. Classification of Bistochastic Quantum Operations

Let be a bistochastic quantum operation, and let denote the density matrix representing the initial state of a quantum system. Depending on the limiting behavior, as , of the corresponding quantum Markov process, must belong to one of the following four mutually exclusive categories:(1) converges to the maximally mixed state , independently of the initial state ;(2) converges, but the limit depends upon the initial state ;(3) fails to converge, but the Cesàro average exists and equals the maximally mixed state , independently of the initial state ;(4) fails to converge, but the Cesàro average exists and depends upon the initial state .

To elucidate each of the above categories, we proceed to offer some comments and examples.

In category (1), the quantum operation possesses no eigenvalue on the unit circle other than . Moreover, the eigenspace of is one-dimensional, spanned only by the density operator , or contains only a single density operator .

As an example, consider the quantum operation [16] defined by where is the unitary transformation given by

According to [18], this is an example of a so-called* random unitary operation*. The eigenspace where is equal to the set . Evidently, . A simple calculation shows that the eigenspace of the eigenvalue is , which is one-dimensional. Therefore for any initial state .

It can be verified that the *bit-phase flip* channel [19, page 377] also belongs to category (1). A less trivial example of this sort can be found in [32].

In category (2), possesses as its only eigenvalue on the unit circle the value and is at least two-dimensional.

As an example, consider the quantum operation associated with a so-called * phase flip* channel on single qubits [19], given by
where is the *Pauli matrix*:

By a pattern of reasoning similar to that employed in the previous example, we infer that , and the eigenspace is spanned by the two linearly independent density operators:

In this case, , where and are given by
It can be verified without much difficulty that the *bit flip* channel [19, page 376] belongs also to category (2). Additional examples of this sort can be found in [20, page 108].

In category (3), the quantum operation possesses at least two distinct eigenvalues (including ) on the unit circle. The eigenspace of , namely, , is one-dimensional and spanned by . An example of this type of quantum operation is provided by *quantum walks* on the -cycle. In this scenario, is assumed even, the Hadamard transform serves as the coin operator, and the evolution of the system is subject to decoherence on both the position and the coin degrees freedom. A detailed treatment of this model is planned for our forthcoming paper [33].

In category (4), has at least two distinct eigenvalues (including ) on the unit circle and contains at least two linearly independent density operators. As an example of this type of quantum operation, we cite [18] which studies the properties of the so-called *two-qubit controlled-not* operators.

#### 6. Conclusion and Related Questions

We speculate that the eigenspaces of eigenvalues on the unit circle, might conform always to a formulation in terms of a set of “Kraus operators”, and this formulation might provide an efficient means for identifying the eigenspaces of all eigenvalues of absolute 1. For *random unitary operations,* as in [18], and so-called *generalized random unitary operations, *as in [33], the structures of the aforementioned eigenspaces have been fully elaborated.

#### Acknowledgment

C. Liu was partially supported by NSF Grant CCF-1005564.