An Efficient Collision Detection Method for Computing Discrete Logarithms with Pollard's Rho
Pollard's rho method and its parallelized variant are at present known as the best generic algorithms for computing discrete logarithms. However, when we compute discrete logarithms in cyclic groups of large orders using Pollard's rho method, collision detection is always a high time and space consumer. In this paper, we present a new efficient collision detection algorithm for Pollard's rho method. The new algorithm is more efficient than the previous distinguished point method and can be easily adapted to other applications. However, the new algorithm does not work with the parallelized rho method, but it can be parallelized with Pollard's lambda method. Besides the theoretical analysis, we also compare the performances of the new algorithm with the distinguished point method in experiments with elliptic curve groups. The experiments show that the new algorithm can reduce the expected number of iterations before reaching a match from 1.309 to 1.295 under the same space requirements for the single rho method.
One of the most important assumptions in modern cryptography is the hardness of the discrete logarithm problem (DLP). Many popular cryptosystems base their security on DLP. Such cryptosystems are, for example, the Diffie-Hellman key agreement protocol , the ElGamal signature and encryption schemes , the US Government Digital Signature Algorithm (DSA) , and the Schnorr signature scheme . Originally, they worked with multiplicative groups of finite prime fields. Once elliptic curve cryptosystems were proposed by Koblitz  and Miller , analogous practical systems based on the DLP in groups of points of elliptic curves over finite fields were designed . Recall the following two definitions.
Definition 1.1 (discrete logarithm problem, DLP). Let be a cyclic group of prime order , and let be generator of . Given , determine the integer such that .
Definition 1.2 (elliptic curve discrete logarithm problem, ECDLP). Let be an elliptic curve defined over finite field . Let be a point of prime order , and let be the subgroup of generated by . Given , determine the integer such that .
For DLP on a multiplicative subgroup of prime order of finite field , the index calculus method determines the size of , which is a subexponential time algorithm, while the size of is set by Pollard's rho method .
Furthermore, for ECDLP, Pollard's rho method and its modifications by Gallant et al.  and Wiener and Zuccherato  are to date known as the most efficient general algorithms. van Oorschot and Wiener  showed that the modified Pollard's rho method can be parallelized with linear speedup.
Pollard's rho method is a randomized algorithm for computing discrete logarithms. Generally, an iteration function is used to define a pseudorandom sequence by for , with some initial value . The sequence represents a walk in the group . The basic assumption is that the walk behaves as a random walk. Because the order of the group is finite, the sequence will ultimately reach an element that has occurred before. This is called a collision or a match. The advantage of this method is that the space requirements are small if one uses a clever method of detecting a collision. The problem of efficient collision detection of a pseudo-random walk in Pollard's rho method is the central topic of this paper.
There are several collision detection algorithms for a random walk in the group . These algorithms in general do not exploit the group structure of . As a result, the algorithms discussed in this paper in fact apply to any set on which an iterated function is used to make random walks, and their utilization goes beyond discrete logarithm computation.
A simple approach to detecting a collision with Pollard's rho method is to use Floyd's cycle-finding algorithm , which shows that it suffices to compare and for all to find a collision. Floyd's algorithm uses only a small constant amount of storage, but needs roughly three times more work than is necessary. Brent  improved this approach by using an auxiliary variable. Nivasch designed an algorithm  for detecting periodicity in sequences using a single pointer and a small stack. This stack algorithm halts at a uniformly random point in the second loop through the sequence's cycle.
In finding DES collisions [14, 15], Quisquater and Delescaille took a different approach based on storing distinguished points, an idea noted earlier by Rivest to reduce the search time in Hellman time-memory tradeoff . A distinguished point is one that has some easily checked property such as having a fixed number of leading zero bits. During the pseudo-random walk, points that satisfy the distinguishing property are stored. Collision can be detected when a distinguished point is encountered a second time. This technique can be efficiently applied to find collisions among multiple processors .
In this paper, we describe a new efficient collision detection algorithm for computing discrete logarithm with Pollard's rho method. It is a probabilistic algorithm and more efficient than the previous methods. With this algorithm, we can significantly reduce the space requirements and provide a better time-space trade-off approach. We also compare their performances in experiments with elliptic curve groups, and our experimental results confirmed the theoretical analysis.
The remainder of this paper is organized as follows. In Section 2, we recall Pollard's rho method for discrete logarithm computation and discuss several previous methods for collision detection. We describe and analyze the new algorithm in Section 3 and discuss its applications in Section 4. We present our experiments in Section 5 and conclude the paper in Section 6.
In this section, we describe Pollard's rho method for discrete logarithm computation and then discuss several collision detection algorithms and their performances.
2.1. Pollard's Rho Method
Pollard  proposed an elegant algorithm for the discrete logarithms based on a Monte Carlo idea and called it the rho method. The rho method works by first defining a sequence of elements that will be periodically recurrent, then looking for a match in the sequence. The match will lead to a solution of the discrete logarithm problem with high probability. The two key ideas involved are the iteration function for generating the sequence and the cycle-finding algorithm for detecting a match.
If is any finite set and is a mapping and the sequence in is defined by the rule: this sequence is ultimately periodic. Hence, there exist unique integers and such that are all distinct, but for all . A pair of two elements of the sequence is called a match if where, . For the expected values of and , we have the following theorem.
Theorem 2.1 (see ). Under the assumption that an iteration function behaves like a truly random mapping and the initial value is a randomly chosen group element, the expected values for and are . The expected number of evaluations before a match appears is .
Now we explain how the rho method for computing discrete logarithms works. Let be a cyclic group of prime order , and let be generator of and . The discrete logarithm problem is to compute satisfying . Pollard defined the iteration function as follows:
Let the initial value . In each iteration of , the function uses one of three rules depending on the value of . The group is partitioned into three subsets , , of roughly equal size. Each has the form . The sequence (and similarly for ) can be computed as follows:
As soon as we have a match , we have the equation .
Since , this gives Now, if , we get that . Due to the method of Pohlig and Hellman , in practice applications the group order is prime, so that it is very unlikely that if is large.
Theorem 2.1 makes the assumption of true randomness. However, it has been shown empirically that this assumption does not hold exactly for Pollard's iteration function . The actual performance is worse than the expected value given in Theorem 2.1.
Teske  proposed better iteration functions by applying more arbitrary multipliers. Assume that we are using partitions (multipliers). We generate random numbers, Then we precompute multipliers , where , for . Define a hash function, Then the iteration function defined as The indices are update by
The difference in performance between Pollard's original walk and Teske’s -adding walk has been studied in [19, 20]. We summarize the results as follows. In prime order subgroups of , the value of for Pollard's original walk and Teske's -adding walk is and , while in groups of points of elliptic curves over finite fields, the value is and , respectively.
2.2. Previous Methods for Collision Detection
To find the collision in the pseudo-random walk, it always needs much storage. In order to minimize the storage requirements, a collision detection algorithm can be applied with a small penalty in the running time.
Floyd's Cycle-Finding Algorithm
In Pollard's paper, Floyd's algorithm is applied. To find , the algorithm calculates until . For each iteration, we compute and , which means that this algorithm requires negligible storage. Floyd's algorithm is based on the following idea.
Theorem 2.2 (see ). For a periodic sequence , there exists an such that and the smallest such lies in the range .
The best running time requires iterations and the worst takes iterations. Under the assumption that behaves like a truly random mapping, the expected number of iterations before reaching a match is . The key point for this algorithm is that we need three group operations and one comparison for each iteration, which makes it inefficient.
Brent proposed an algorithm  which is generally 25% faster than Floyd's method. It uses an auxiliary variable, say , which at each stage of the algorithm holds , where . is compared with for each iteration and is updated by when for . The correctness of this algorithm depends on the following fact.
Theorem 2.3 (see ). For a periodic sequence , there exists an such that and . The smallest such is .
Under the assumption that is a random mapping, with Brent's algorithm the first match is expected to occur after iterations . The algorithm needs one group operation and one comparison for each iteration, which makes it 25%–30% faster than Floyd's algorithm. Variations of Brent's algorithm requiring slightly more storage and comparisons but less iterations can be found in [21, 22].
In 2004, Nivasch proposed an interesting algorithm that uses logarithmic storage space and can be adapted with tradeoff for storage space versus speed. This algorithm requires that a total ordering < be defined on the set , and it works as follows. Keep a stack of pairs , where, at all times, both the 's and the 's in the stack form strictly increasing sequences. The stack is initially empty. At each step , pop from the stack all entries , where . If a match is found with an element in the stack, the algorithm terminates successfully. Otherwise, push on top of the stack and continue. The stack algorithm depends on the following fact.
Theorem 2.4 (see ). The stack algorithm always halts on the smallest value of the sequence's cycle, at some time in .
Under the assumption that is a random mapping, the expected number of iterations before finding a match is . The algorithm needs a little bit more than one group operation and one comparison for each iteration. Under the same assumption, Nivasch proves also that the expected size of the stack is ln . Therefore, the algorithm only requires a logarithmic amount of memory.
The idea of the distinguished point method is to search for a match not among all terms of the sequence, but only among a small subset of terms that satisfy a certain distinguishing property. It works as follows. One defines a set , a subset of , that consists of all group elements that satisfy a certain distinguishing property. During the pseudo-random walk, points that satisfy the distinguishing property are stored. Collision can be detected when a distinguished point is encountered a second time.
Currently, the distinguished point method is the most efficient algorithm to detect collisions in pseudo-random walk when is large. A popular way of defining is to fix an integer and to define that if and only if the least significant bits in the representation of as a binary string are zero. To break ECC2K-130, it  defines the distinguishing property as the Hamming weight of normal-basis representation of -coordinate of the point less than or equal to 34. Notice that this kind of definitions allows a fast check for the distinguishing property to hold, and the size of can be easily monitored as well. Obviously, we have the following theorem.
Theorem 2.5 (see ). Let be the proportion of points in which satisfy the distinguishing property, that is, . Under the assumption that is a random mapping and is a uniform distribution in , the expected number of iterations before finding a match is .
3. The New Algorithm
We are motivated by the fact that, in distinguished point method, the distinguished points may be not uniformly distributed in the pseudo-random walk, also the points in subset may be not uniformly distributed in , which always results in more iteration requirements. We are trying to design an algorithm which leads to a uniform distribution and also to provide a better way for time-space tradeoff rather than distinguishing property.
3.1. The Basic Algorithm
To find a collision in pseudo-random walk, which is produced by the iteration function , assuming is a random mapping on , our basic algorithm works as follows. We fix an integer and use an auxiliary variable, say , which at each iterations keep the minimum value of successive values produced by the iteration function . Once getting the minimum value from successive values, we check whether this value has occurred before in the stored sequence, if so, we find the match and we are done. Otherwise, store this value to the sequence. Then continue to compute the next new values and repeat the previous procedures. Choose the integer properly, we will find the match among the newly generated minimum value and stored minimum values.
More precisely, to find a collision in pseudo-random sequence , which is produced by the iteration function , we have Algorithm 1.
It is obvious that the algorithm can be considered as two parts. In the first part, that is from step (3) to step (11), we seek the minimum value from successive values in the pseudo-random walk. The operation is very simple; if the current value is smaller than , then just update with the current value and continue the next iteration. Notice that steps (7), (8), and (9) can be omitted, since it is unlikely that there is a match within iterations. Even if it happened, the algorithm ensures that we can find a match within the next iterations. As a result, there is only one group operation and one comparison in the first part, which consist the main operations of the algorithm.
In the second part, that is from step (13) to step (19), once we get a minimum value , we check whether has appeared before in the previous stored values , which is empty at the beginning. If this is the case, the algorithm will return the corresponding indices and we are done. Otherwise, save the value to the sequence of the minimum values, and the second part is finished, continue the next iterations. It is clear that the second part can be speeded up by using a hash table. And, more important, the second part can be independent to the first part, which means the stored sequence of minimum values can be off line; that is, the first part is response for generating minimum values along the random walk, while the second part searches the collision among stored minimum values independently.
For further analysis of the algorithm, we assume that the iteration function behaves like a truly random mapping. According to Theorem 2.1, the expected number of iterations before reaching a match is . Let us look at some simple cases for the new algorithm. For , which means we store all the values in the sequence before reaching a match, and the match can be found once it appears. For , we need to store half of the values in the sequence before reaching a match, and always the match can be found once it appears. Obviously, the bigger the integer , the less values we need to store. As a result, with the integer increasing, there is a probability that we cannot detect the collision immediately when it happens. So, the new algorithm is a probabilistic algorithm. However, with high probability, the algorithm will halt close to the beginning of the second loop. More precisely, we have the following theorem.
Theorem 3.1. Under the assumption that an iteration function behaves like a truly random mapping and the initial value is a randomly chosen group element, for Algorithm 1, the expected number of iterations before finding a match is with probability , where .
Proof. Let be the set that consists of the th successive values generated by iteration function ; that is,
For finite group , the sequence produced by is eventually period; that is, for any fixed and , there exist certain integers and , such that
To prove the theorem, we divide it into two cases, that is, and . For , let and be the minimum values of and , then that is, the probability of successfully detecting the collision within the first two intersection sets is .
For each of , we notice that, for two (intersected) sets and , let and be the minimum values of and , respectively, we have Therefore, we have Under the assumption that the iteration function is a random mapping, according to Theorem 2.1, the expected number of evaluations before a match appears is . Combining the above two cases, using Algorithm 1, the expected number of iterations before reaching a match among minimum values is with probability , where .
Remark 3.2. According to the above theorem, we need to store terms to find the match with the probability , where can be ; that is, by setting parameter , we can balance the expected number of iterations and the expected space requirements. Therefore, Algorithm 1 is a time-space trade-off algorithm.
Remark 3.3. Algorithm 1 is a probabilistic algorithm. There is a probability that we cannot detect the collision immediately when it happens. However, with high probability, the algorithm will halt close to the beginning of the second loop. For example, the probability of successfully detecting the collision is 0.90 with .
Notice that, compared to the distinguished point method, the new algorithm has two advantages. First, the distinguished point method depends on the assumption that the distinguished points are uniformly distributed in the pseudo-random walk, and also the points in subset are uniformly distributed in . However, in practice this may not be the case, which generally results in more iterations requirement, while, for the new algorithm, each stored minimum value represents successive values and the performance of the new algorithm independent of such assumption. Because the distinguished point method is currently the most efficient algorithm, we compare the actual performances of the new algorithm with the distinguished point method under the same expected storage requirement in experiments with elliptic curve groups in Section 5.
Second, using distinguished point method, it is possible for a random walk to fall into a loop which contains no distinguished point . Then, the processor cannot find new distinguished point any more on the path. Left undetected, the processor would cease to contribute to the collision search. In this case, we can restart the random walk by choosing a different initial point. However, those points calculated by previous walk do not help for the collision search, while the new algorithm can avoid such problem, because it can always find the collision among minimum values whenever it falls into a loop.
The new algorithm can be combined with other algorithms, such as Pollard's lambda method, and can be adapted to a wide range of problems which can be reduced to finding collisions, such as in Pollard's rho method for factorization  and in studying the behavior of random number generators . In this section, we will address some of these issues.
4.1. Pollard's Lambda Method
It is clear that the new algorithm can be applied to Pollard's lambda method (also called the kangaroo method). The lambda method computes a discrete logarithm in an arbitrary cyclic group, given that the value is known to lie in a certain interval, that is, , where but unknown.
Generally, we have two kangaroos, one tame and one wild. Their positions are represented by group elements, the tame kangaroo with starting point and the wild kangaroo with starting point , and they travel in the cyclic group . In terms of the exponents of , starts at the middle of the interval , while starts at . Since we do not know , we do not know the exact location of the wild kangaroo, and that is why it is called wild. The two kangaroos produce two different pseudo-random walks with the same walking rules. It is obvious that, at all times, the point of tame kangaroo has the form and the point of wild kangaroo has the form for some known integers and . The purpose is to provoke a collision between the tame and the wild kangaroos, from which we can deduce the wild kangaroo's starting point, that is, .
Similar to the case of Pollard's rho method, the new algorithm can be applied in this case to efficiently detect the collision. The advantage of the new algorithm is that we can achieve uniform distributions of minimum values in the pseudo-random walks both for the tame kangaroos and the wild kangaroos. The different performances of the new algorithm and distinguished point method in this case can refer to the case of Pollard's rho method.
As we have mentioned above, during the random walk, finding the minimum value from iterations and comparing the minimum value to all previously stored values can be separated. This feature makes the new algorithm suit for distributed computation.
However, Pollard's rho method is inherently serial in nature; one must wait for the current iteration of the function to complete before the next can begin. Each value in the sequence totally depends on the previous value and the iteration rules. In discussing the rho method for factorization, Brent considered running many processors in parallel each producing an independent sequence and noted that “Unfortunately, parallel implementation of the “rho” method does not give linear speedup” . Analogous comments apply to the rho method for computing logarithms and the generalized rho method for collision search. Notice that here each parallel processor is producing its own sequence of points independently of the others and each particular processor does not increase the probability of success of any other processor. For the corresponding picture, with high probability, each processor draws a different “rho” that never intersect with each other. There is a little chance that different processors may intersect with each other.
van Oorschot and Wiener  showed that the expected speedup of the direct parallelization of Pollard's rho method, using processors, is only a factor of . This is a very inefficient use of parallelization. They provided a modified version of Pollard's rho method and claimed that it can be linearly parallelized with the distinguished point method; that is, the expected running time of the modified version, using processors, is roughly group operations.
In the modified version, to perform a parallel collision search each processor proceeds as follows. Select a random starting point , and produce the trail of points , for , until a distinguished point is reached based on some easily testable distinguished property. Store distinguished point, and start producing a new trail from a new random starting point. Unfortunately, the new algorithm is not efficient for the parallelized modified version of Pollard's rho method. The key point is that there is a probability that the new algorithm fails to detect the collision while it actually happened, which cannot be efficiently solved like the serial version.
However, for Pollard's lambda method, the new algorithm can be efficiently parallelized with linear speedup. We present here a modified version of parallelized Pollard's lambda method from . Assume we have processors with even. Then, instead of one tame and one wild kangaroo, we work with two herds of kangaroos, one herd of tame kangaroos and one herd of wild kangaroos, with one kangaroo on each processor. Each kangaroo starts from a different point, stores a minimum value every iterations, just like the serial version. A center server collects all the minimums, and tries to find a collision between the tame kangaroo minimums and the wild kangaroo minimums By choosing a reasonable integer , the new algorithm provides an optimal time-space trade-off method for collision detection.
We implemented Pollard's rho method with elliptic curve groups over prime fields using SAGE , which is an open source computer algebra software. Obviously, such experiments can also be done for DLP on a multiplicative subgroup of finite field . We compared the different performances between distinguished point method and the new algorithm. In this section, we describe these experiments and analyse the results.
For our experiments, we briefly introduce the elliptic curve groups over prime fields and the notation we use in the following. Let be a prime, and let denote the field of integers modulo . Let such that . Then the elliptic curve over is defined through the equation The set of all solutions of this equation, together with the element called the “point at infinity,” forms a finite abelian group which we denote by . Usually, this group is written additively. Let be a point of prime order , and let denote the subgroup of generated by . Given , determine the integer such that .
For the iteration function, we use Teske's -adding walk and set ; that is, we divide the group into 20 subsets: . Define the iteration function as follows: where and randomly chosen from and can be precomputed for . This means it only needs one group operation for each iteration.
Let be any point of ; we define the partition of into subsets as follows. First we compute a rational approximation of the golden ratio , with a precision of decimal places. Let where is the nonnegative fraction part of . Then let This method is originally from Knuth's multiplicative hash function  and suggested by Teske . From the theory of multiplicative hash functions, we know that, among all numbers between 0 and 1, choosing as a rational approximation of with a sufficiently large precision leads to the most uniformly distributed hash values, even for nonrandom inputs.
The purpose of our experiments is to evaluate the expected numbers of steps until a match is found with different collision detection methods, that is, distinguished point method and the new algorithm, under the same expected space requirement. Generally, we randomly choose a big prime number , where is in certain range. Then we randomly choose the parameters and , where , which determine the unique elliptic curve over . We will check whether the order of group has large prime factor in certain range. If not, repeat the above procedures until we get a prime order subgroup of . Then we set the generator of and choose a random point of . When using Pollard's rho method to compute this discrete logarithm, we count the number of steps we performed until a match is found with different collision detection methods on the same case. Then we determine the ratio of the number of steps and . We repeat it a couple of times with the same but several randomly chosen ’s. Furthermore, for practical reasons, we do the above procedures with a couple of groups, where the group order is between 231 and 236. We have Algorithm 2.
More precisely, for each , we generate 20 elliptic curves, where each of them has a subgroup of prime order , such that . Then for, each group , we generate 100 to 3200 DLPs with the same generator but randomly generated . The number of elliptic curves and instances of DLPs computed is given in Table 1. For each DLP, we use Teske's -adding walk for iteration function and find the match using distinguished point method and the new algorithm simultaneously. Once reaching a match, we compute the ratio as (the number of steps until match is found)/. Then we compute the average ratio of all DLPs over the same elliptic curve. Finally, we count the average ratio of all DLPs with the same , where and .
Now, let us explain the parameters for distinguishing property and the new algorithm in more detail. In our experiments, we compute the average ratio of under the same space requirement. To do this, generally we first define the distinguishing property and then compute the expected storage requirements. With the same storage requirements, we can deduce the parameter for the new algorithm.
For example, if , which means , the order of is a 36-bit prime number. According to , we are expected to take iterations before reaching a match. We define the distinguishing property as the Hamming weight of normal-basis representation of -coordinate of the point less than or equal to 9. Each point has probability almost exactly of being a distinguished point, that is, ; that is, to find a collision it is expected to compute distinguished points. To keep the same storage requirements, we set for the new algorithm.
The experimental results are given in Table 2. It shows that on average using the new algorithm for collision detection can reduce the number of iterations until a match is found from to under the same space requirements for the single rho method.
Under the same expected storage requirements, the main reason for the different performances of the distinguished point method and the new algorithm is that the distinguished points may be not uniformly distributed in the pseudo-random walk, also the points in subset may be not uniformly distributed in , which always results in more iterations requirement. while, for the new algorithm, each stored minimum value represents successive values, which leads to an equal-interval distribution.
In this paper, we proposed an optimal time-space trade-off method for collision detection in the pseudo-random walk when computing discrete logarithms with Pollard's rho method. We discussed the new algorithm both in theoretical analysis and in practical experiments. By comparison to other methods, it shows that the new algorithm is more efficient than previous methods. Unfortunately, the only practical application of the new idea is with the parallelized lambda method and it does not work with the parallelized rho method. As a further work, we would like to explore the performances of the new algorithm in other applications.
This work is partially supported by the National Natural Science Foundation of China (no. 61070168). The authors would like to thank the reviewers for their helpful comments and suggestions.
FIPS 186-2, “Digital signature standard,” Tech. Rep. 186-2, Federal Information Processing Standards Publication, 2000.View at: Google Scholar
V. Miller, “Use of elliptic curves in cryptography,” in Advances in Cryptology: Proceedings of Crypto'85, vol. 218 of LNCS, pp. 417–426, Springer, New York, NY, USA, 1986.View at: Google Scholar
A. Menezes, P. van Oorschot, and S. A. Vanstone, Handbook of Applied Cryptography, CRC Press, Boca Raton, Fla, USA, 1996.
J. J. Quisquater and J. P. Delescaille, “How easy is collision search? Application to DES,” in Proceedings of the Advances in Cryptology—Eurocrypt, vol. 434 of Lecture Notes in Computer Science, pp. 429–434, Springer, New York, NY, USA, 1989.View at: Google Scholar
J. J. Quisquater and J. P. Delescaille, “How easy is collision search. New results and applications to DES,” in Proceedings of the Advances in Cryptology—Crypto, vol. 435 of Lecture Notes in Computer Science, pp. 408–413, Springer, New York, NY, USA, 1989.View at: Google Scholar
S. Bai and R. P. Brent, “On the efficiency of Pollard’s rho method for discrete logarithms,” in CATS 2008, J. Harland and P. Manyem, Eds., pp. 125–131, Australian Computer Society, 2008.View at: Google Scholar
D. V. Bailey, L. Batina, D. J. Bernstein et al., “Breaking ECC2K-130,” Tech. Rep. 2009/541, Cryptology ePrint Archive, 2009.View at: Google Scholar
D. E. Knuth, The Art of Computer Programming, vol. 2, Addison-Wesley, Reading, Mass, USA, 3rd edition, 1997.
D. E. Knuth, The Art of Computer Programming, vol. 3, Addison-Wesley, Reading, Mass, USA, 2nd edition, 1981.