#### Abstract

Let be a complex Hilbert space; denote by Alg and the atomic nest algebra associated with the atomic nest on and the space of Schatten- class operators on, respectively. Let be the space of Schatten- class operators in Alg . When and , we give a complete characterization of nonlinear surjective isometries on . If , we also prove that a nonlinear surjective isometry on is the translation of an orthogonality preserving map.

#### 1. Introduction

Let and be normed spaces and let be a map from to . We say that is a nonlinear isometry (or a distance preserving map) if for every pair in . In particular, is an isometry if is linear and distance preserving. The question of characterizing isometries between operator algebras is very important in studying geometric structure of operator algebras. Many authors pay their attention to such a problem (see [1–11] and their references). One of well-known results is due to Kadison, who states that every isometry for the operator norm from a unital -algebra onto another unital -algebra is a -isomorphism followed by left multiplication by a fixed unitary element (see [7]). Besides, of the operator norm, isometries for the Schatten- norm also are studied extensively (see [1, 2, 5, 6, 8–11] and their references). Early in 1975, Arazy in [2] gave a characterization of isometries on the Schatten- class (). In [1], Anoussis and Katavolos characterized isometries on the Schatten- class in nest algebras and obtained the following theorem.

Theorem 1. *Let and be two nests of projections on a Hilbert space and a fixed involution on . Assume that , . The surjective isometries have one of the following forms:
**
where is a unitary operator, and is an order isomorphism of onto ( is an order isomorphism of onto ).*

More generally, in recent years, many authors are devoted to characterizing distance preserving maps on operator algebras (see [3, 4, 12–15] and their references). In [14], Chan et al. showed that a nonlinear surjective isometry for the unitarily invariant norm on complex matrix algebras has one of the following forms.(a)There are unitary matrices and a matrix such that or for each matrix .(b)If and has the form, there are unitary matrices and a matrix such that or for each matrix .(c)If the unitarily invariant norm is a multiple of the Frobenius norm, that is, for some , then the map is a real orthogonal transformation with respect to the inner product for each matrix .

Recall that a norm of operators is a unitary invariant norm if for any unitary operators . In [3, 4, 12, 13, 15], distance preserving maps on several kinds of operator algebras in the infinite dimensional case were characterized. Bai and Hou in [12] give a characterization of nonlinear numerical radius isometries on . Cui and Hou in [13] characterize nonlinear numerical radius isometries on atomic nest algebras and diagonal algebras. Hou and He in [15] give a characterization of nonlinear isometries on the Schatten- class. A nature problem is how to characterize nonlinear isometries on the Schatten- class in nest algebras. The main purpose of this paper is to give a complete characterization of nonlinear surjective isometries on the Schatten- class () in atomic nest algebras acting on Hilbert spaces (Theorem 2). Such a result generalizes the linear map assumption in Theorem 1 to the nonlinear case. Also, the problem on orthogonality of nonlinear surjective isometries for Hilbert-Schmidt norms is discussed (Theorem 3).

By the classical Mazur-Ulam theorem (see [10]) which states that every distance preserving surjective map sending 0 to 0 between normed spaces is real linear, we essentially deal with the real linear isometries (i.e., the distance preserving real linear maps). One can not expect that each real isometry has the same structure as the complex isometry. Applicable examples are found in [3] (Example 0.2, 0.3 in [3]).

Following the idea of [3], a key step in our approach is to show that the distance preserving maps on the Schatten- class () in nest algebras also preserve rank-one operators in both directions. This leads to a demand for characterizing rank-1 preserving additive maps between nest algebras. Related results had been obtained in [16].

Before embarking upon our results, it is convenient here to introduce some notations. Denote by or the real or complex field. For an operator on , we denote the range of by and the adjoint of by . Let be an automorphism (or homomorphism) of or . If a map on satisfies and for every and , then we say that is -linear. If is a ring homomorphism, then we say that is semilinear and in the case that and , we say is conjugate linear. If is a conjugate linear operator between Hilbert spaces and , is called conjugate unitary or antiunitary, where is the Hilbert space conjugate operator of . Denote by and the space of all compact operators and the space of all finite rank operators on the Hilbert space . For any , the trace of , , where is a normal orthogonal base in the Hilbert space . Let ; the Schatten- norm of is as follows: The Schatten- class is the set of all Schatten- class operators, that is, all compact operators with the finite Schatten- norm. If , the set is called the trace class. If , . Recall that a nest on is a chain of closed (under norm topology) subspaces of containing and , which is closed under the formation of arbitrary closed linear span (denoted by ) and intersection (denoted by ). Alg denotes the associated nest algebra, which is the set of all operators in such that for every element . If is a nest, is a nest. If , we say that is nontrivial. We denote . For any , let , , and . , . If , we say is an atom of . A nest on is said to be atomic if is spanned by its atoms and to be maximal if is atomic and all its atoms are one-dimensional. The rank-one operator Alg if and only if there is an such that and . For each , and , . If is a nest and , for , ; for , . Assume that , , , and , , and . If the nest is fixed, they are written briefly as , , , , , , respectively.

#### 2. Main Results

In the following theorem, we give a characterization of nonlinear surjective isometries on , where is an atomic nest.

Theorem 2. *Let be a complex Hilbert space, an atomic nest on . Assume that , , is a surjective map. Then satisfies for all if and only if one of the following holds true. *(1)*There exist an operator , a dimension preserving order isomorphism , and unitary operators satisfying and for every , such that
*(2)*There exist an operator , a dimension preserving order isomorphism , and conjugate unitary operators satisfying and for every , such that
*(3)*There exist an operator , a dimension preserving order isomorphism , and unitary operators satisfying and for every , such that
*(4)*There exist an operator , a dimension preserving order isomorphism , and conjugate unitary operators satisfying and for every , such that
*

The problem on orthogonality of nonlinear surjective isometries for Hilbert-Schmidt norms is discussed in the following theorem.

Theorem 3. *Let be a complex Hilbert space, an atomic nest on . Assume that is a surjective map. Then satisfies for all and then the map is a real linear and an orthogonal transformation on with respect to the real inner product .*

#### 3. Proof of Main Results

To prove our main results, we need the following lemmas.

Lemma 4 (see [17]). *For arbitrary and , , if and only if
*

In the following lemmas, we give a characterization of rank-oneness of operators by the relation of orthogonality between operators. Let for arbitrary . The set is maximal, if for arbitrary operator , .

Lemma 5 (see Lemma 3 in [1]). *For , then *(1)* unless , in which case ;*(2)* unless , in which case .*

Lemma 6. *For any nonzero operator with the atomic nest , if the set is maximal and nonempty, then . Conversely, if , and either or , then the set is maximal and nonempty.*

*Proof. *If is maximal and nonempty, we show that rank . If not, rank , then there are two nonzero vectors and such that . Since the nest is atomic, let and one can find a vector such that (if necessary, interchanging for ). Now for any , it follows from the definition of that . So and ; it follows that . So we have . One can check but . That is, but is not in . It is a contradiction to the maximum of . So rank .

If rank , let , and either or , and by Lemma 5, one of the following three cases happens.*Case 1. * and .*Case 2. * and .*Case 3. * and .

If for , , then either
or
It follows that either or . It implies that rank and are linearly dependent. By computation, then . So is maximal and nonempty.

In the following lemma that is taken from [16], let be the dual of a Banach space . Let be a nest on over real or complex field . If dim , , and if dim , .

Lemma 7 (see [16]). *Let and be two nests on Banach spaces and over real or complex field , respectively. Let be a continuous surjective additive map. Then preserves rank-1 operators in both directions if and only if one of the following is true.*(1)*There are linear or conjugate linear bounded bijective operators , , a dimension preserving order isomorphism , and vectors , such that , for every , and for each rank-1 operator ,
*(2)*There are linear or conjugate linear bounded bijective operators , , a dimension preserving order isomorphism , and vectors , such that , for every , and for each rank-1 operator ,
**Moreover, in this case, and are reflexive.*

Lemma 8. *For any , the following are equivalent:*(I)*;*(II)* for any real number .*

*Proof. * If , for any real number ,

Without loss of generality, assume that , and by , we have, for any real number ,
that is, . So . It follows from arbitrariness of that . We complete the proof.

*Proof of Theorem 2. *Checking the “if” part is straightforward, so we will only deal with the “only if” part.

Let for any ; then , and for any . By the Mazur-Ulam theorem (see [10]), we have that is an additive map. Furthermore, we have that and for any . By Lemma 4, satisfies that for all .

Next we show that preserves rank-one operators in both directions. For any rank-one operator , by the above discussion, . By Lemma 6, if either or , then has rank one. has the same property as , and preserves rank-one operators. So preserves rank-one operators in both directions. If both and , take rank-one operator ; then . So we have . Since has rank one, then has rank one. As has the same property as , so preserves rank-one operators in both directions.

preserves rank-one operators in both directions; then has the form in Lemma 7. In the case of complex Hilbert space, we have that one of the following is true.(1)There are linear or conjugate linear bounded bijective operators , vectors , and a dimension preserving order isomorphism such that , for every , such that
(2)There are linear or conjugate linear bounded bijective operators , vectors , and a dimension preserving order isomorphism such that , for every , such that

One can note that for all rank-one operator , so is the same to the proof of Lemma 4.11 in [3], can be chosen as unitary or conjugate unitary operators; denote by .

If case occurs, next we claim , and . For case , similarly, we can show that and . Assume that occurs and has the second form, in fact dim, . Just like the discussion in Lemma 4.11 in [3], we have , . Assume on the contrary that . Let , ; then . By a computation, . So , a contradiction. So . If has the third form, in this case dim , . Just like the discussion in Lemma 4.11 in [3] again, . Assume on the contrary that ; let , similar to the above discussion, and we get a contradiction again.

Every finite rank operator can be written as a sum of rank-one operators in the nest algebra, and the set of finite rank operators is dense in , so by addition of , we have that Theorem 2 holds true by replacing by .

*Proof of Theorem 3. *Let for any ; then , and for . By the Mazur-Ulam theorem (see [10]), is real linear.

By real linearity of , we have that for any real number . By Lemma 8, is real linear and the map preserves orthogonality with respect to . We complete the proof.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

#### Acknowledgment

This work was supported partially by National Science Foundation of China (11201329, 11171249).