Abstract
The laser chaos decision maker has been demonstrated to enable ultra-high-speed solutions of multiarmed bandit problems or decision-making in the GHz order. However, the underlying mechanisms are not well understood. In this paper, we analyze the chaotic dynamics inherent in experimentally observed laser chaos time series via surrogate data and further accelerate the decision-making performance via parameter optimization. We first evaluate the negative autocorrelation in a chaotic time series and its impact on decision-making detail. Then, we analyze the decision-making ability using three different surrogate chaos time series to examine the underlying mechanism. We clarify that the negative autocorrelation of laser chaos improves decision-making and that the amplitude distribution of the original laser chaos time series is not optimal. Hence, we introduce a new parameter for adjusting the amplitude distribution of the laser chaos to enhance the decision-making performance. This study provides a new insight into exploiting the supremacy of chaotic dynamics in artificially constructed intelligent systems.
1. Introduction
In recent years, photons and laser physics for high-speed information processing have been studied in various applications including photonic reservoir computing and neuromorphic photonic computing, among others [1–7]. A high-speed decision maker using ultrafast laser chaos has been demonstrated for applications in solving multiarmed bandit (MAB) problems [4, 5]. MAB problems accompany a difficult tradeoff called exploration-exploitation dilemma, which is fundamental to reinforcement learning [8,9]. It has been shown that laser chaos enables ultra-high-speed decision-making in the GHz order for MAB problems with two arms [4]. Naruse et al. [5] proposed a scalable laser chaos decision maker applicable to MABs with up to 64 arms. These ultrafast decision makers can be applied to various problems, which require fast decision-making in dynamically changing uncertain environments [6, 7]. Takeuchi et al. applied laser chaos decision-making to channel selection problems in wireless communication systems, in which communication channels suffer from dynamically changing disturbances owing to traffic, interference, fading, etc. [6]. Furthermore, Kanno et al. utilized laser chaos for adaptive model selection with photonic reservoir computing [7].
The effectiveness of the laser chaos time series was studied by demonstrating its performance superiority to pseudorandom numbers, colored noise, and random shuffle surrogate data of the original chaotic time series [4]. It was suggested that the negative autocorrelation property of the laser chaos may provide positive effects for decision-making [4]. Similarly, it was found that the negative autocorrelation property improves the performance of MAB algorithms [10].
Indeed, the effectiveness of chaotic time series with negative autocorrelation has been demonstrated in previous studies, for example, on code division multiple access (CDMA) in wireless communications by Kohda and Fujisaki [11] and Hasegawa [12]. It has been theoretically shown that the negative autocorrelation in each chaotic spreading sequence minimizes cross-correlation among those sequences. Applications to solution search in combinatorial optimization problems have also been studied utilizing chaotic dynamics [13–15]. Hasegawa et al. [15] demonstrated the effectiveness of the method of surrogate data by showing a specific negative autocorrelation of the chaotic time series accelerating solution search in combinatorial optimization algorithms.
The method of surrogate data has been proposed to analyze the properties of nonlinear dynamics [16]. An important property characterizing the given time series can be extracted by comparing the original and surrogate data, which are generated by a stochastic process, while preserving some statistical property of the original time series. The preserved statistical property is determined based on a null hypothesis that the original data can be characterized by the preserved property. Three types of surrogate data and their generating algorithms have been proposed: (1) random shuffle (RS), (2) Fourier transform (FT), and (3) amplitude-adjusted Fourier transform (AAFT) [16]. RS surrogates preserve the empirical histogram, FT surrogates preserve the autocorrelation and power spectrum, and AAFT surrogates preserve the empirical histogram, autocorrelation, and power spectrum. Further extensions of these surrogate algorithms have been proposed [16–18]. Note that the surrogate data analysis on decision-making in [4] regards only RS surrogate data.
In this paper, the effectiveness of negative autocorrelations inherent laser chaos will be clarified using the methods of surrogate data. We analyze the effective properties of the laser chaos that affect the performance of the decision maker. First, we evaluate the relationship between the decision-making performance and autocorrelation coefficient of the laser chaos time series. Next, the three types of surrogate data are adapted to investigate the impact of statistical properties of the laser chaos time series on decision-making. Based on these analyses, we propose a method to further enhance the performance of the laser chaos decision maker by adding a new control parameter, which optimizes the amplitude of the chaotic time series.
2. Laser Chaos Decision Maker
Figure 1(a) shows the architecture of the laser chaos decision maker for a two-arm MAB problem [4], of which performance analysis and optimization is the primal concerns of the present study. In this section, we review the principle of laser chaos-based decision-making. The details are found in [4].
(a)
(b)
High-frequency laser chaos was generated by the delayed optical feedback of the reflector to the semiconductor laser [19, 20]. Another output from the polarization-maintaining (PM) coupler was detected by the photodetector (PD) through the optical isolator (ISO) and the variable attenuator (ATT) and was sampled at a rate of 100 G samples per second. Figure 1(b) shows a snapshot of a waveform example of the laser chaos time series.
The decisions at time are made by comparing the sampled values of the laser chaos time series with the threshold for decision-making. When the laser chaotic time series at time , that is, , is larger than the threshold , slot machine A is selected. Otherwise, the slot machine B is selected. The threshold is updated according to the result of carrying out the selected slot machine after each decision-making. For example, if slot machine A is selected and the result is “win,” the threshold decreases. This leads to the increase of the probability that the threshold becomes smaller than the sampling value. Hence, the probability of selecting slot machine A is increased. Contrarily, when the result is “fail,” the threshold value increases. Hence, the probability of selecting slot machine A is decreased.
The values of threshold are determined as follows:where is a threshold adjuster, which is determined by the results of carrying out current decisions. k is the width of the threshold adjuster step. The number of the thresholds is given by where N is a natural number. The range of is limited between and . A large value means a small change in the value of the threshold. Therefore, intuitively, the resultant decision-making takes long time to converge, but the decision accuracy improves thanks to sufficient exploration. is defined aswhere is set as a constant increment (i.e., = 1). α is the forgetting parameter for weighting previous threshold adjuster variables, which ranges from 0 to 1, i.e., . If , is determined without considering the previous threshold . is the increment parameter, which is given bywhere is the estimated reward probability of slot machine i at time t. is given bywhere is the number of times the slot machine i was selected until time t. is the number of “win” times of slot machine i until time t. is large when the estimated reward probability obtained from the slot machine is large.
Multiple two-arm laser chaos bandits were cascaded in a hierarchical structure to form a scalable decision maker applicable to MAB problems with more than two arms [5]. Each two-arm bandit uses laser chaos samples and updates the threshold based on decisions, in the same manner as the two-arm laser chaos decision maker described above. Solution of a 64-arm MAB problem has been confirmed [5].
3. Effectiveness of Negative Autocorrelation for Ultrafast Decision Makers
3.1. Autocorrelation of the Laser Chaos
The laser chaos time series signal has different properties for different sampling intervals. A time series signal has a positive or negative autocorrelation when the sampling interval is changed, as shown in Figure 2. When the sampling interval is 10 ps, the time series signal has a positive autocorrelation. The value of the autocorrelation decreases as the sampling interval increases up to 50 ps. From Figure 2, we can see that the negative autocorrelation is maximum when the sampling interval is 50 ps. At that time, the value of the autocorrelation coefficient is approximately .
3.2. Performance Evaluation of Autocorrelation
In this subsection, we analyze the effectiveness of the negative correlation of the laser chaos time series in decision-making. We used a chaotically oscillating laser chaos time series generated by a semiconductor laser sampled at 10 ps intervals for a total of 107 samples. Such 107-long time series were obtained 120 times. The experimental details have been presented in [4].
We evaluate the performance using the correct decision rate (CDR), which is defined as the ratio of the number of runs in selecting the arm with the highest hit probability to the total number of runs. The definition of CDR can be expressed aswhere is the number of runs with different laser chaos and different random initial conditions. , if the slot machine with the highest reward probability is selected at the decision of the cycle; otherwise, . Table 1 lists the simulation parameters. In the following simulations,
Figure 3 shows the performance evaluated by CDRs as a function of the autocorrelation coefficient of the laser chaos. We evaluated the results for two types of MAB problems (i.e., ), which indicate that the negative autocorrelation improves the performance of the laser chaos decision maker. The effectiveness of the negative autocorrelation is larger when and are higher, as shown in Figure 3(b).
(a)
(b)
4. Analysis of Characteristics of Effective Laser Chaos Using the Method of Surrogate Data
Surrogate data have been deployed to analyze the nonlinearity of time series [16–18] and the effectiveness of chaos in combinatorial optimization problems [15], e.g., in the literature. In this section, we analyze the properties of laser chaos via the surrogate data method to examine the decision-making performance. Surrogate data methods are introduced in Section 4.1, while the effectiveness analysis of laser chaos is described in Section 4.2.
4.1. The Method of Surrogate Data
The method of surrogate data can be used to find important properties characterizing the given data. Surrogate data are generated by a stochastic process that preserves some statistical properties of the original time series. The preserved statistical properties are determined based on the null hypothesis that the results of the original data can be characterized by these preserved properties. By comparing the results of the original and surrogate data, the importance of the preserved property can be evaluated.
Various surrogate data generation algorithms have been proposed [16–18]. Herein, we used three kinds of basic surrogate data, i.e., RS, FT, and AAFT. Table 2 shows the properties (i.e., histogram, power spectrum, and autocorrelation) of surrogate data that destroy or store the original data. The power spectrum represents the frequency component of a time series signal. The notations in Table 2, i.e., “good,” “bad,” and “medium,” reflect the ability of the corresponding property to preserve the statistical properties completely, hardly, and approximately, respectively.
4.1.1. RS Surrogate Data
RS surrogate data can be generated by randomly replacing the original time series signal. The original time series signal histogram is preserved; however, the autocorrelation and power spectrum are destroyed. RS surrogate data, , are generated by randomly swapping each value of the original time series signal, , where ,..., .
4.1.2. FT Surrogate Data
FT surrogate data are a time series signal that preserves the power spectrum and autocorrelation of the original data; however, the histogram is destroyed. The process of generating FT surrogate data is as follows:(1)The original time series signal, , where ,..., , is transformed to via discrete Fourier transformation(2)The phase of is randomized to generate (3)The phase of is symmetrized to generate (4)An inverse discrete Fourier transform, , is performed to generate the time series FT surrogate data,
4.1.3. AAFT Surrogate Data
AAFT surrogate data preserve the original time series signal histogram and the near-power spectrum. The process of generating AAFT surrogate data is as follows:(1)A Gaussian random number, {,...,}, is generated(2)The Gaussian random number, , and original time series signal, , are arranged so that the magnitudes of the values are in the same order; then, the time series signal, , is generated(3)The FT surrogate data, , for are generated(4)The original time series signal and are arranged so that the magnitudes of the values are in the same order, and the AAFT surrogate data, , are generated
Figure 4 illustrates snapshots of the original laser chaos time series (Figure 4(a)) and its RS (Figure 4(b)), FT (Figure 4(c)), and AAFT (Figure 4(d)) surrogate data, respectively. Because the laser chaos time series signal used in this study has an 8 bit output, its value ranges from −127 to 128. Here, we examine the laser chaos time series with a sampling rate of 50 ps that exhibits a negative maximum autocorrelation, as shown in Figure 2.
(a)
(b)
(c)
(d)
Figure 5 shows the signal level histogram and autocorrelation coefficient of the original and surrogate data. Note that in Figure 5(a), the histograms of the RS and AAFT surrogate and the original laser chaos time series are equivalent and overlap (purple line), whereas the FT surrogate exhibits a different histogram with a Gaussian distribution. Conversely, in Figure 5(b), the autocorrelation of the original, FT, and AAFT surrogate data exhibits nearly equivalent traces, whereas the FT surrogate does not show any correlation, except for a time lag of zero.
(a)
(b)
4.2. Performance Analysis Using the Method of Surrogate Data
We analyze the properties of the laser chaos that affect the performance of decision-making via surrogate data methods. Here, we consider the same two-armed bandit problem detailed in Section 3. The simulation conditions are the same as those listed in Table 1. Furthermore, we evaluated the performance for different numbers of threshold levels defined by N = 4, 8, 16, and 32.
Figure 6 summarizes the CDR values at a time cycle of 1,000 with respect to the number of thresholds for the original laser chaos time series, RS, FT, and AAFT surrogate data. First, we observe that the original, FT, and AAFT surrogate data are more effective than the RS surrogate data for any N. Note that the RS surrogate destroys autocorrelation. Therefore, we can conclude that the time-domain correlation inherent in the original laser chaos improves the performance regardless of the number of thresholds. Second, we observe that the CDR of the FT surrogate data is superior to that of the original data regardless of N. Note that the FT surrogate data follow a Gaussian distribution, whereas the histogram of the original laser chaos time series deviates from the Gaussian profile and contains a minor asymmetry. Therefore, the histogram of the original laser chaos is not the best for this decision-making problem. Furthermore, this suggests that the performance of the laser chaos decision maker can be improved by tuning the distribution of the laser chaos time series, which is examined in the next section.
(a)
(b)
5. Improving Performance of the Laser Chaos Decision Maker
In Section 4, we found that the performance can be enhanced by changing the distribution of the laser chaos because FT surrogates exhibit the best performance. However, technologically speaking, employing a surrogate data algorithm is unfeasible, especially in real-time applications because it loses the ultrafast aspects of the physical phenomena of the laser chaos. Meanwhile, the signal level of the experimentally observed chaotic time series could be technologically reasonably configured in the optical and/or electrical domains. Moreover, such approaches preserve the time-domain correlation, which is another key factor of supremacy.
Therefore, here, we simply introduce a parameter for scaling the incoming signal level. Specifically, the time series signal is adjusted to as
In this way, the width of the distribution of the time series signal can be adjusted to that of , which can further improve the performance of the decision maker while preserving the time-domain correlations.
Figure 7 summarizes the performances of the CDR at a time cycle of 1,000 as a function of . Different curves indicate different numbers of threshold levels: N = 2, 4, 7, 16, 32, 64, and 128. The other simulation parameters are the same as those in Table 1. The reward probabilities assumed in Figures 7(a) and 7(b) are also the same as previously summarized in Figures 6(a) and 6(b), respectively. The original CDR corresponds to the cases when the scaling parameter is unity.
(a)
(b)
From the results, we confirm that a higher CDR value is achievable by turning depending on the number of threshold levels. For example, in the case of N = 2 shown in Figure 7(a), the CDR increases by approximately 0.88 at = 5, which corresponds to a performance enhancement of over 20%. Such performance gains are more clearly observed in Figure 7(b), where the reward probabilities of both machines are high for all numbers of threshold levels. Furthermore, in the cases of N = 2, 4, and 8, the scaling parameter = 5 provides optimal performance in both Figures 7(a) and 7(b). Conversely, the optimal may differ, for instance, in the case of N = 128, depending on the reward probability setting. The optimal values in Figures 7(a) and 7(b) are 1 and approximately 2.5, respectively. Seeking the best and N will be the basis of our future studies along with exploration of additional performance factors, such as the quickness toward environmental change.
In addition to future research on simple acceleration mechanisms, the generation of optimally conditioned chaotic time series constitutes another exciting topic; that is, Gaussian distribution temporally correlated ultrafast sequences at the physical layer level. Other important associated topics include applications of the analysis and proposals to the scalable laser chaos decision maker for applications associated with large-scale decision-making problems. As mentioned in the introduction, the effectiveness of the laser chaos decision maker has been demonstrated in wireless communication systems and reservoir computing. As there are many kinds of decision-making problems in the real world, application of the proposed principles to real-world problems is also important.
6. Conclusions
We analyzed the effectiveness of the laser chaos decision maker using surrogate data methods. We first verified that laser chaos with negative correlation enables high performance for decision-making problems. Then, we analyzed the decision-making performance using random shuffle, Fourier transform, and amplitude-adjusted Fourier transform surrogate methods. We clearly confirmed that the time-domain correlation inherent in the laser chaos provides the performance enhancement of the decision-making problem. In addition, the Fourier transform surrogate indicated that a Gaussian distribution of signal levels yields high performance; that is, the original experimental data of the laser chaos time series are not the most effective sequence for decision-making. To solve this problem, we proposed deployment of a scaling parameter for the laser chaos signals, which is simply multiplying a constant value by the observed signal, and confirmed the existence of the optimal scaling parameter that maximizes the decision-making performance. Note that the time-domain correlations of the original laser chaotic sequences were completely preserved. This study paves a new way of exploiting ultrafast chaotic supremacy in artificial intelligence.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon reasonable request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the CREST project (JPMJCR17N2) funded by the Japan Science and Technology Agency, the Core-to-Core Program, and Advanced Research Networks and Grants-in-Aid for Scientific Research (JP17H01277 and JP20H00233) funded by the Japan Society for the Promotion of Science.