A Subspace Embedding Method in 𝐿₂ Norm via Fast Cauchy Transform

Abstract

We propose a subspace embedding method via Fast Cauchy Transform (FCT) in 𝐿₂ norm. It is motivated by and complements the work of the subspace embedding method in 𝐿_𝑝 norm, for all 𝑝∈[1,∞] except 𝑝=2, by K. L. Clarkson (ACM-SIAM, 2013). Unlike the traditionally used orthogonal basis in Johnson-Lindenstrauss (JL) embedding, we employ the well-conditioned basis in 𝐿₂ norm to obtain concentration property of FCT in 𝐿₂ norm.

1. Introduction

Subspace embedding methods have recently proven useful in modern science and engineering. A well-known example is the famous Johnson-Lindenstrauss (JL) lemma [1, 2] which provides low-distortion subspace embeddings in the 𝐿₂ norm and has obtained lots of important applications in dimension reduction.

A vital mathematical problem in subspace embedding is to devise a basis 𝑈 for the embedded subspace such that ∀𝑥∈ℝ^𝑑,‖𝑥‖_𝑝≤‖𝑈𝑥‖_𝑝≤poly(𝑑)²‖𝑥‖_𝑝, (1) where poly(𝑑) stands for the distortion; poly(𝑑) is a polynomial about 𝑑. Obviously, this can be done by taking an orthonormal basis for 𝐿₂ norm due to the invariant property under an orthonormal transformation. For general 𝐿_𝑝 norm with 𝑝≠2, the invariant property under an orthonormal transformation disappears. However, works [3, 4] pointed out that it can be overcome by constructing well-conditioned bases for (1). Actually, well-conditioned bases are more practical than orthonormal basis. Recently, Sohler and Wooddruff [5] have proved that, using random Cauchy variables, over linear mapping 𝑅, with arbitrary large constant probability, for any fixed 𝑑-dimensional subspace, for all 𝑥∈𝐿,‖𝑥‖₁≤‖𝑅𝑥‖₁=𝑂(𝑑log𝑑)‖𝑥‖₁ holds. Clarkson et al. [6] extended this to 𝐿_𝑝 norm linear regression, for all 𝑝∈[1,∞] except 𝑝=2. The result of FCT in 𝐿₁ norm could be formulated as follows.

Theorem 1 (Fast Cauchy Transform (FCT) in 𝐿₁ norm, [6]). There is a distribution, over matrices Π₁∈ℝ^𝑟1×𝑛, with 𝑟₁=𝑂(𝑑log(𝑑/𝛿)) and 𝑠=𝑟⁶₁, such that for an arbitrary 𝐴∈ℝ^𝑛×𝑑 and for all 𝑥∈ℝ^𝑑, the inequality ‖𝐴𝑥‖₁≤Π₁𝐴𝑥₁≤𝜅‖𝐴𝑥‖₁ (2) holds with probability 1−𝛿, where 𝑑√𝜅=𝑂𝑠𝛿log𝑟₁𝑑. (3)

That is, with large probability, simultaneously for all 𝑥, ‖𝐴𝑥‖₁≈‖Π₁𝐴𝑥‖₁. Setting 𝑠=𝑟⁶₁, we obtain the distortion 𝑂(𝑑^2+𝜂) for an arbitrarily small constant 𝜂>0. Further, for any 𝑦∈ℝ^𝑛, the product Π₁𝑦 can be computed in 𝑂(𝑛log𝑟₁) time.

In this paper, we study the Fast Cauchy Transform in the case of 𝐿₂ norm and prove the following.

Theorem 2 (FCT in 𝐿₂ norm). There is a distribution, over matrices Π₂∈ℝ^𝑟1×𝑛, with 𝑟₁=𝑂(𝑑log(𝑑/𝛿)), such that for an arbitrary 𝐴∈ℝ^𝑛×𝑑 and for all 𝑥∈ℝ^𝑑, the inequality ‖𝐴𝑥‖₂≤Π₂𝐴𝑥₂≤𝜅‖𝐴𝑥‖₂ (4) holds with probability 1−𝛿, where 𝜅=𝑂𝑟₁𝑑³𝛿. (5)

For any 𝑦∈ℝ^𝑛, the product Π₂𝑦 can be computed in 𝑂(𝑛log𝑟₁) time. Setting 𝛿 to a small constant, since √𝑠=𝑟³₁ and 𝑟₁=𝑂(𝑑log𝑑), it follows that 𝜅=𝑂(𝑑²log^1/2𝑑). To get concentration, we need to independently bounded 𝛾²_𝑗 in our proof which required 𝑠=𝑟⁶₁ to obtain the high probability result, and we to obtain the distortion 𝑂(𝑑^1+𝜂). From 𝐿₁ norm to 𝐿₂ norm, in the embedding matrix, Cauchy variable 𝐶_𝑖 becomes 𝐶²_𝑖 and the Cauchy distribution has density 𝑝(𝑥)=(1/𝜋)(1/(1+𝑥²)). Aiming to obtain 𝐿₂ embedding properties, we take 𝐶²_𝑖 in the tail inequality and obtain the FCT in 𝐿₂ norm which is our main result.

2. Preliminaries

Let 𝐴∈ℝ^𝑛×𝑑 be an 𝑛×𝑑 input matrix, where we assume 𝑛≫𝑑 and 𝐴 has full column rank. For 𝑝∈[1,∞], the 𝐿_𝑝 norm of a vector 𝑥 is ‖𝑥‖_𝑝=(∑_𝑖|𝑥_𝑖|^𝑝)^1/𝑝 and max_𝑖|𝑥_𝑖| for 𝑝=∞. Let [𝑛] denote the set {1,2,…,𝑛}. For matrices, we use the 𝐿₂ operator norm ‖𝐴‖₂=sup_{‖𝑥‖2=1}‖𝐴𝑥‖₂ and the entrywise 𝐿_𝑝 norm ‖𝑋‖_𝑝=(∑_𝑖,𝑗|𝑋_𝑖𝑗|^𝑝)^1/𝑝. 𝐼_𝑛 denotes the 𝑛×𝑛 identity matrix and 𝑐 refers to a generic constant. The Cauchy distribution having density 𝑝(𝑥)=(1/𝜋)(1/(1+𝑥²)). The Cauchy distribution will factor heavily in our discussion and bounds for sums of Cauchy random variables' square will be used throughout. First, we give Maurer's inequality as follows.

Lemma 3 (see [7]). Let 𝑋_𝑖≥0 be independent random variables, ∑_𝑖𝐄[𝑋²_𝑖]<∞, and define 𝑋=∑_𝑖𝑋_𝑖. Then, for any 𝑡>0, 𝐏𝐫[𝑋≤𝐄[𝑋]−𝑡]≤exp−𝑡²2∑_𝑖𝐄𝑋²_𝑖. (6)

Here, we take 𝑋 to be 𝑋=∑_{𝑖∈[𝑛]}𝜉_𝑖(1−𝑥_𝑖).

Lemma 4 (see [6]). Let 𝑥_𝑖 be i.i.d. (0,1) random variables with probability 𝑝 and let 𝑋=∑_{𝑖∈[𝑛]}𝜉_𝑖𝑥_𝑖, where 𝜉_𝑖≥0, with ∑_{𝑖∈[𝑛]}𝜉_𝑖=𝜉 and ∑_{𝑖∈[𝑛]}𝜉²_𝑖≤𝜉²/𝛽². Then, for any 𝑡>0, −𝛽𝐏𝐫𝑋≥𝜉𝑝+𝑡≤exp²𝑡²2.1−𝑝 (7)

3. Main Technical Results: The Fast Cauchy Transform of 𝐿₂ Norm

3.1. Two Tail Inequalities

In the section, we present two tail inequalities and the definition of well-conditioned basis in 𝐿₂ norm. Then we present the Fast Cauchy Transform in 𝐿₂ norm, which is analog of the fast Johnson-Lindenstrauss transform, and prove our main result Theorem 2 in Section 3.2.

Lemma 5 (upper-tail inequality). For 𝑖∈[𝑚], let 𝐶_𝑖 be 𝑚 i.i.d. Cauchy random variables and 𝛾≥0 with 𝛾=∑_{𝑖∈[𝑚]}𝛾_𝑖. Let 𝑋=∑_{𝑖∈[𝑚]}𝛾_𝑖𝐶²_𝑖. Then, for any 𝑡>0, 𝐏𝐫𝑋>𝛾𝑡≤𝑂𝑚𝑡. (8)

Proof. For fixed 𝑀>0 and defined 𝐹_𝑖={𝐶²_𝑖≤𝑀} and 𝐹=⋂_{𝑖∈[𝑚]}𝐹_𝑖, then we have 𝐹_𝑖={𝐶_𝑖≤√𝑀}; note that 𝐹⋂𝐹_𝑖=𝐹. Because of tan⁻¹𝑥≤𝑥, we have that 𝐏𝐫𝐹_𝑖=2𝜋tan⁻¹√𝑀2=1−𝜋tan⁻¹1√𝑀2≥1−𝜋√𝑀. (9) By a union bound, 𝐏𝐫[𝐹]≥1−(2𝑚/𝜋𝑀). Further, 𝐏𝐫[𝐹∣𝐹_𝑖]𝐏𝐫[𝐹_𝑖]=𝐏𝐫[𝐹⋂𝐹_𝑖]=𝐏𝐫[𝐹_𝑖]; hence, 𝐏𝐫[𝐹∣𝐹_𝑖]=𝐏𝐫[𝐹]/𝐏𝐫[𝐹_𝑖]. First, we need to bound 𝐄[𝐶²_𝑖∣𝐹], 𝐄𝐶²_𝑖∣𝐹_𝑖=𝐄𝐶²_𝑖∣𝐹_𝑖𝐹𝐏𝐫𝐹∣𝐹_𝑖+𝐄𝐶²_𝑖∣𝐹_𝑖𝐹𝐏𝐫𝐹∣𝐹_𝑖≥𝐄𝐶²_𝑖∣𝐹_𝑖𝐹𝐏𝐫𝐹∣𝐹_𝑖. (10) Then, 𝐄𝐶²_𝑖≤𝐄𝐶∣𝐹^2𝑖∣𝐹_𝑖𝐏𝐫𝐹∣𝐹_𝑖=𝐄𝐶²_𝑖∣𝐹_𝑖𝐏𝐫𝐹_𝑖[𝐹].𝐏𝐫 (11) By using the pdf of a Cauchy variable, 𝐄𝐶²_𝑖∣𝐹_𝑖=2√𝑀/𝜋−(1/𝜋)tan⁻¹√𝑀𝐏𝐫𝐹_𝑖, (12) so 𝐄𝐶²_𝑖≤2√∣𝐹𝑀/𝜋−(1/𝜋)tan⁻¹√𝑀[𝐹]≤2√𝐏𝐫𝑀/𝜋−(1/𝜋)tan⁻¹√𝑀1−2𝑚/𝜋√𝑀. (13) We conclude that 𝐄[𝑋∣𝐹]=_{𝑖∈[𝑚]}𝛾_𝑖𝐄𝐶²_𝑖≤√∣𝐹2𝛾𝑀/𝜋−(1/𝜋)tan⁻¹√𝑀1−2𝑚/𝜋√𝑀. (14) Because of Markov's inequality and 𝐏𝐫[𝑋≥𝛾𝑡∣𝐹]≤1, we have [𝐹]𝐏𝐫𝑋≥𝛾𝑡=𝐏𝐫𝑋≥𝛾𝑡∣𝐹𝐏𝐫+𝐏𝐫𝑋≥𝛾𝑡∣𝐹[𝐹≤()√(1−𝐏𝐫])2/𝑡𝑀/𝜋−(1/𝜋)tan⁻¹√𝑀1−2𝑚/𝜋√𝑀+2𝑚𝜋√𝑀. (15) We set 𝑀=𝑚𝑡; then 𝐏𝐫𝑋≥𝛾𝑡≤𝑂𝑚𝑡. (16)

Lemma 6 (lower-tail inequality). For 𝑖∈[𝑟], let 𝐶_𝑖 be independent Cauchy random variables and with 𝛾=∑_{𝑖∈[𝑚]}𝛾²_𝑖 and ∑_{𝑖∈[𝑚]}𝛾⁴_𝑖≤𝛾²/𝛽². Let 𝑋=∑_{𝑖∈[𝑟]}𝛾²_𝑖𝐶²_𝑖. Then, for any 𝑡>0, (−𝛽𝐏𝐫𝑋≤𝛾1−𝑡)≤exp²𝑡²3. (17)

Proof. To bound the lower tail, we use Lemma 3. By homogeneity, it suffices to prove the result for 𝛾=1. Let 𝑍_𝑖=𝛾²_𝑖min(𝐶²_𝑖,𝑀). Clearly 𝑍_𝑖≤𝛾²_𝑖𝐶²_𝑖 and defining 𝑍=∑_𝑖𝑍_𝑖, we have that 𝑍≤𝑋 and 𝐏𝐫[𝑋≤1−𝑡]≤𝐏𝐫[𝑍≤1−𝑡]. Thus, we have that [][[𝑍]−(𝐄[𝑍]−(𝐄[𝑍])𝐏𝐫𝑍≤1−𝑡=𝐏𝐫𝑍≤𝐄−1+𝑡)]≤exp−1+𝑡²2∑_𝑖𝐄𝑍²_𝑖, (18) where the last inequality holds by Lemma 3 for 1−𝑡<𝐄[𝑍]. Using the distribution of half-Cauchy, we can verify that by choosing a fixed 𝑀 to make sure 2∫^√𝑀₀𝑥²(1/𝜋)(1/(1+𝑥²))𝑑𝑥=1, since 1<𝑀<2, 𝐄[𝑍_𝑖]=𝛾²_𝑖 and 𝐄[𝑍²_𝑖]≤(3/2)𝛾⁴_𝑖, so ∑_𝑖𝐄[𝑍_𝑖]=1 and ∑_𝑖𝐄[𝑍²_𝑖]≤(3/2)∑_𝑖𝛾⁴_𝑖≤(3/2𝛽²). It follows that 𝐏𝐫[𝑍≤1−𝑡]≤exp(−𝑡²𝛽²/3) and the result follows.

3.2. Definition of 𝐿₂ Well-Conditioned Basis and Construction of FCT in 𝐿₂ Norm

Clarkson et al. [6] constructed Π₁ as Π₁≡4𝐵𝐶𝐻; let 𝛿∈(0,1] be a parameter governing the failure probability of our algorithm; we modified 𝐵 and obtain Π₂, where: 𝐵∈ℝ^𝑟1×2𝑛 has each column chosen independently and uniformly from the 𝑟₁ orthogonal standard basis vector for ℝ^𝑟1 and 𝐵_𝑖𝑗 is chosen independently from a (0,1)-distribution; for 𝛼 sufficiently large, we will set the parameter 𝑟₁=𝛼𝑑log𝑑/𝛿; 𝐶∈ℝ^2𝑛×2𝑛 is a diagonal matrix with diagonal entries chosen independently from a Cauchy distribution; and 𝐻∈ℝ^2𝑛×𝑛 is a block-diagonal matrix comprised of 𝑛/𝑠 blocks along the diagonal. Each block is the 2𝑠×𝑠 matrix 𝐺_𝑠=[𝐻_𝑠;𝐼_𝑠], where 𝐼_𝑠 is the 𝑠×𝑠 identity matrix and 𝐻_𝑠 is the normalized Hadamard matrix. We will set 𝑠=𝑟⁶₁. The effects of 𝐻 here are to spread the weight of a vector, so that 𝐻𝑦 has most entries that are not too small.

We describe 𝐿₂ norm well-conditioned basis and our main aim for constructing an 𝐿₂ well-conditioned basis is to make sure that we can tolerate the distortion of subspace embedding. To form an approximate basis for the range of 𝐴, we are led to faster algorithms for a range of related problems including low-rank matrix approximation [8, 9].

Definition 7 (𝐿₂ well-conditioned basis, [3, 4]). A basis 𝑈 for the range of 𝐴 is 𝐿₂-(𝛼,𝛽)-conditioned if ‖𝑈‖₂≤𝛼 and for all 𝑋∈ℝ^𝑑, ‖𝑥‖₂≤𝛽‖𝑈𝑥‖₂.

Next we show the construction of 𝐿₂ well-conditioned basis 𝑈, which consists two steps: (a) let Π₂ be a matrix satisfying (4) and compute Π₂𝐴 and its QR-factorization Π₂𝐴=𝑄𝑅, where 𝑄 is an orthogonal matrix; (b) output 𝑈=𝐴𝑅⁻¹=𝐴(𝑄^𝑇Π₂𝐴)⁻¹. This structure is similar to the algorithm of [10] for computing an 𝐿₂ well-conditioned basis.

To obtain Theorem 2, we divide it into two propositions. To prove the upper bound, we use the existence of a (𝑑,1)-conditioned basis 𝑈 and apply this basis to show that ‖Π₂𝑈𝑥‖₂ cannot expand too much. To prove the lower bound, we show that the inequality holds with high probability for a particular 𝑦; then we use a suitable 𝛾-net to obtain the result for all 𝑦.

Proposition 8. With probability at least 1−𝛿, for all 𝑥∈ℝ^𝑑, ‖Π₂𝐴𝑥‖₂≤𝜅‖𝐴𝑥‖₂, where 𝜅=𝑂(𝑟₁𝑑³/𝛿).

Proof. Let 𝑈∈ℝ^𝑛×𝑑 be a 𝐿₂-(𝑑,1)-conditioned basis space of 𝐴, which implies that for some 𝑍∈ℝ^𝑑×𝑑 we have 𝐴=𝑈𝑍. By the construction of 𝑈, for any 𝑥∈ℝ^𝑑, ‖𝑥‖₂≤‖𝑈𝑥‖₂ and so Π₂𝑈𝑥₂≤Π₂𝑈₂‖𝑥‖₂≤Π₂𝑈₂‖𝑈𝑥‖₂. (19) Thus our main aim is to show that ‖Π₂𝑈‖₂≤𝜅. We have Π₂𝑈₂=4𝐵𝐶𝐻𝑈₂=4_𝑗∈𝑑𝐵𝐶𝐻𝑈^(𝑗)₂=4_𝑗∈𝑑𝐵𝐶𝑈^(𝑗)₂, (20) where 𝑈=𝐻𝑈. We need to bound ‖𝑈‖²₂. For any vector 𝑦∈ℝ^𝑛, we represent 𝑦 by 𝑛/𝑠 block of size 𝑠, so 𝑧_𝑖∈ℝ^𝑠 and 𝑦^𝑇=[𝑧^𝑇_𝑖,𝑧^𝑇₂,…,𝑧^𝑇_𝑛/𝑠]. Recall that 𝐺_𝑠=[𝐻_𝑠;𝐼_𝑠] and ‖𝐺_𝑠‖₂=√2. Then, 𝐻𝑦²₂=_{𝑖∈𝑛/𝑠}𝐺_𝑠𝑧_𝑖²₂. (21) It follows that 𝐻𝑦²₂≤_{𝑖∈𝑛/𝑠}𝐺_𝑠²₂𝑧_𝑖²₂=2𝑦²₂. (22) Applying this to 𝑦∈𝑈^(𝑗) for 𝑗∈[𝑑], 𝑈²₂=_{𝑗∈[𝑑]}𝑈^(𝑗)2₂≤2‖𝑈‖²₂≤2𝑑². (23) The (𝑖,𝑗) entry of 𝐵𝐶𝑈 is ∑_𝑘𝐵_𝑖𝑘𝐶_𝑘𝑘𝑈_𝑘𝑗; here we take 𝛾_𝑖𝑗=∑_𝑘𝐵_𝑖𝑘𝑈_𝑘𝑗. So, 𝐵𝐶𝑈²₂=_{𝑖,𝑗𝑘}𝐵_𝑖𝑘𝐶_𝑘𝑘𝑈_𝑘𝑗2=_𝑖,𝑗𝛾_𝑖𝑗𝐶_𝑖𝑗²=_𝑖,𝑗𝛾²_𝑖𝑗𝐶²_𝑖𝑗. (24) Here 𝐶_𝑖,𝑗 are dependent Cauchy random variables and 𝛾=∑_𝑖,𝑗𝛾²_𝑖𝑗. Using 𝐵 as a standard orthogonal basis, we obtain 𝛾=_𝑖,𝑗𝛾²_𝑖𝑗=_𝑖𝑗𝑘𝐵_𝑖𝑘𝑈_𝑘𝑗²=𝑈²₂. (25) Hence, we can apply Lemma 5, and 𝑚=𝑟₁𝑑 to obtain, 𝐏𝐫𝐵𝐶𝑈²₂≥𝑡𝛾≤𝑂𝑚𝑡. (26) Setting 𝑚/𝑡 to 𝛿, then we can get 𝑡=𝑂((1/𝛿²)𝑟₁𝑑). Thus, with probability at least 1−𝛿, 𝐵𝐶𝐻₂=𝑂𝑟₁𝑑³𝛿. (27) To prove the second section, first we will show a result for fixed 𝑦∈ℝ^𝑛; we will describe in the next lemma.

Lemma 9. Let 𝐏𝐫[‖Π₂𝑦‖₂<2‖𝑦‖₂]≤exp(−𝑟³₁/64)+exp(−𝑠^1/2/16𝑟²₁+log𝑟₁).

Proof. We represent any vector 𝑦∈ℝ^𝑛 by its 𝑛/𝑠 blocks of size 𝑠, so 𝑧_𝑖∈ℝ^𝑠 and 𝑦^𝑇=[𝑧^𝑇₁,𝑧^𝑇₂,…,𝑧^𝑇_𝑛/𝑠]. Let 𝑔=𝐻𝑦,𝑔=[𝐺_𝑠𝑧₁;𝐺_𝑠𝑧₂,…,𝐺_𝑠𝑧_𝑛/𝑠]. We have that ‖𝑔‖²₂=∑_𝑖‖𝐺_𝑠𝑧_𝑖‖²₂=2∑_𝑖‖𝑧_𝑖‖²₂=2‖𝑦‖²₂ and 𝑔₁=_𝑖𝐺_𝑠𝑧_𝑖𝑖≥12𝑠^1/4_𝑖𝑧_𝑖2≥12𝑠^1/4_𝑖𝑧_𝑖²₂^1/2=12𝑠^1/4𝑦₂. (28)
We can conclude that ‖𝑔‖₁≥(1/2√2)𝑠^1/4‖𝑔‖₂, so ‖𝑔‖²₂≥(1/8)𝑠^1/4‖𝑔‖²₄. To analyze 4‖𝐵𝐶𝑔‖²₂(Π₂𝑦=𝐵𝐶𝑔, 𝑔=𝐻𝑦), we could have (𝐵𝐶𝑔)_𝑗=∑_𝑖𝐵_𝑗𝑖𝐶_𝑖𝑖𝑔_𝑖. Here we take 𝛾_𝑗=∑^𝑛_𝑖=1𝐵_𝑗𝑖𝑔_𝑖; then (𝐵𝐶𝑔)_𝑗=𝛾_𝑗𝐶_𝑗(𝐶_𝑗 is an independent Cauchy random variable), so 𝐵𝐶𝑔²₂=_𝑗𝑖𝐵_𝑗𝑖𝐶_𝑖𝑖𝑔_𝑖²=_𝑗𝛾²_𝑗𝐶²_𝑗. (29) To apply Lemma 6, we need to bound ∑_𝑗𝛾²_𝑗 and ∑_𝑗𝛾⁴_𝑗. First, because 𝐵 is standard orthogonal, then _𝑗𝛾⁴_𝑗=_𝑗𝑖𝐵_𝑗𝑖𝑔_𝑖²=_𝑗𝑖𝐵²_𝑗𝑖𝑔²_𝑖=𝑔²₂. (30)
To bound ∑_𝑗𝛾⁴_𝑗, we will show that 𝛾²_𝑗 is nearly uniform. Because 𝐵_𝑗𝑖 and 𝐵_𝑗𝑘 are independent for 𝑖≠𝑘, we can use Lemma 4 with 1−𝑝=1−1/𝑟₁, with 𝜉_𝑖=𝑔²_𝑖 and ∑_𝑖𝜉_𝑖=‖𝑔‖²₂; setting 𝑡=1/𝑟₁ in Lemma 4, 𝐏𝐫𝛾²_𝑗≥𝑝+𝑡⋅_𝑖𝜉_𝑖−𝑡≤exp²2⋅𝑔⁴₂𝑔⁴₄−𝑠≤exp^1/216𝑟²₁. (31) By a union bound, none of the 𝛾²_𝑗 exceed 2‖𝑔‖²₂/𝑟²₁ with probability at most 𝑟₁exp(−𝑠^1/2/16𝑟²₁). We assume this high probability event, so we get ∑_𝑗𝛾⁴_𝑗≤4‖𝑔‖⁴₂/𝑟³₁. We apply Lemma 4 with 𝑡=3/4 and 𝛽²=𝑟³₁/4 to obtain 𝐏𝐫[‖𝐵𝐶𝑔‖₂≤‖𝑔‖₂⋅(1/2)]≤exp(−𝑟³₁/64). By a union bound, with probability at least 1−exp(−𝑟³₁/64)−exp((−𝑠^1/2/16𝑟²₁)+log𝑟₁).

Proposition 10. Assume Proposition 8 holds. Then, for all 𝑥∈ℝ_𝑑, ‖Π₂𝐴𝑥‖₂≥‖𝐴𝑥‖₂ holds with probability at least ()×−𝑟1−exp𝑑log2𝑑𝜅exp³¹−𝑠64+exp^1/216𝑟²₁+log𝑟₁. (32)

Proof. The proposition follows by putting a 𝛾-net Γ on the range of 𝐴 (observe that the range of 𝐴 has dimension at most 𝑑). Specifically, let 𝐿 be any fixed 𝑑 dimensional subspace of ℝ^𝑛 (𝐿 is the range of 𝐴). Consider the 𝛾-net on 𝐿 with cubes of side 𝛾/𝑑; there are (2𝛾/𝑑)^𝑑 such cubes required to cover the hypercube ‖𝑦‖_∞; and for any two points 𝑦₁,𝑦₂ inside the same cube, ‖𝑦₁−𝑦₂‖₂≤𝛾/√𝑑. From each of the 𝛾/𝑑-cubes, select a fixed generic point 𝑦^∗ and ‖𝑦^∗‖₂=1. In order to make sure that 𝑦^∗ belongs to (2𝛾/𝑑)^𝑑, by a union bound and Lemma 9, 𝐏𝐫min_𝑦∗Π₂𝑦^∗₂𝑦^∗₂<12≤2𝑑𝛾^𝑑−𝑟exp³¹−𝑠64+exp^1/216𝑟²₁+log𝑟₁. (33)
We will condition on the high probability event that ‖Π₂𝑦‖₂≥‖𝑦^∗‖₂/2 for all 𝑦^∗. For any 𝑦∈𝐿 with ‖𝑦‖₂=1, let 𝑦^∗ denote the representative point which is in the same cube of 𝑦(‖𝑦^∗‖₂=1). Then ‖𝑦−𝑦^∗‖₂≤𝛾/√𝑑. Considering Π₂𝑦₂=Π₂𝑦^∗+Π₂(𝑦−𝑦^∗)₂≥Π₂𝑦^∗₂−Π₂(𝑦−𝑦^∗)₂≥2𝑦^∗₂−𝜅𝑦−𝑦^∗₂, (34) by choosing 𝛾=1/𝜅, we have ‖Π₂𝑦‖₂≥‖𝑦‖₂, with probability at least ()×−𝑟1−exp𝑑log2𝜅𝑑exp³¹−𝑠64+exp^1/216𝑟²₁+log𝑟₁. (35)

Since 𝑠^1/2=𝑟³₁, by choosing 𝑟₁=𝛼𝑑log(𝑑/𝛿) for large enough 𝛼, the final probability of failure is at most 2𝛿. And also since 𝜅=𝑂(𝑟₁𝑑³/𝛿) and 𝑟₁=𝛼𝑑log(𝑑/𝛿), by the inequality (1), we could get the distortion 𝑂(𝑑^1+𝜂) for an arbitrarily small constant 𝜂>0.

4. Conclusion

In the paper, we extend selections of the embedding matrix and apply the well-conditioned basis in 𝐿₂ norm. Based on Maurer's inequality, we obtain the tail inequality about Cauchy variables. By the specified matrix Π₂, we present Fast Cauchy Transform in 𝐿₂ norm.

Acknowledgments

The work is supported in part by the National Science Foundation of China under Grants no. 61271014, no. 61072118, and no. 61170159; by Science Project of National University of Defense Technology JC120201, also by National Natural Science Foundation of Hunan Province (China) 13JJ2001. The authors would like to thank anonymous referees for helpful suggestions and they would also like to thank Hui Zhang and Tao Sun for many thoughtful discussions.