Research Article  Open Access
Ivo Bukovsky, Noriyasu Homma, Kei Ichiji, Matous Cejnek, Matous Slama, Peter M. Benes, Jiri Bila, "A Fast Neural Network Approach to Predict Lung Tumor Motion during Respiration for Radiation Therapy Applications", BioMed Research International, vol. 2015, Article ID 489679, 13 pages, 2015. https://doi.org/10.1155/2015/489679
A Fast Neural Network Approach to Predict Lung Tumor Motion during Respiration for Radiation Therapy Applications
Abstract
During radiotherapy treatment for thoracic and abdomen cancers, for example, lung cancers, respiratory motion moves the target tumor and thus badly affects the accuracy of radiation dose delivery into the target. A realtime imageguided technique can be used to monitor such lung tumor motion for accurate dose delivery, but the system latency up to several hundred milliseconds for repositioning the radiation beam also affects the accuracy. In order to compensate the latency, neural network prediction technique with realtime retraining can be used. We have investigated realtime prediction of 3D time series of lung tumor motion on a classical linear model, perceptron model, and on a class of higherorder neural network model that has more attractive attributes regarding its optimization convergence and computational efficiency. The implemented static feedforward neural architectures are compared when using gradient descent adaptation and primarily the LevenbergMarquardt batch algorithm as the ones of the most common and most comprehensible learning algorithms. The proposed technique resulted in fast realtime retraining, so the total computational time on a PC platform was equal to or even less than the real treatment time. For onesecond prediction horizon, the proposed techniques achieved accuracy less than one millimeter of 3D mean absolute error in one hundred seconds of total treatment time.
1. Introduction
In radiation therapy, accurate and sufficient amount of dose delivery only to the target tumor is required to not only maximize the therapeutic effects, but also minimize inaccurate delivery of doses to healthy tissues surrounding the tumor. Such accurate irradiation is, however, a nontrivial task due to the body motion. For example, the respiratory motion complicates the targeting of external radiation to tumors in lungs, pancreas, and other thoracic and abdominal sites. The tumor motion can be associated with the internal movements caused by respiration and cardiac cycles and also with systematic drifts and patient's stochastic movements [1, 2]. Among them, respiration is dominant and thus the respiratory motion has been widely analyzed. In lung tumor motion, it is well known to have amplitude between 0.5 and 2.5 cm, even some times 5 cm [3]. As a consequence, the dose distribution may be delivered significantly different from the prescribed one and increase the radiation toxicity dramatically [4–9]. The time series of the lung respiration has a quasiperiodic nature and the behavior may vary in time [2, 5, 10]. The respiration motion becomes a complex nonstationary process; that is, it changes amplitude and period over time. Some breathing is highly irregular in patients whose pulmonary functions are affected by disease [11–14]. Several methods have been developed for the respiratory motion gated radiation therapy or realtime tumor tracking, but their use is still questioned [2, 10]. Three general approaches have been achieved to predict respiration behavior [10].
In Isaksson et al. [5] it is shown that adaptive signal processing filters can provide more accurate tumor position estimates than stationary filters when presented with nonstationary breathing motion. Murphy and Dieterich [10] analyzed linear versus nonlinear neural network filter for predicting tumor motion when breathing behavior is moderately to extremely irregular. In Homma et al. [15] the authors developed a time series prediction based on a seasonal ARIMA model by using the realtime compensation of the time variant nature involved in the cyclic dynamics of the respiration motion. Their evaluation by using a clinical dataset showed that the proposed system can achieve a clinically useful high accuracy and longterm prediction of the average error mm at 1second prediction ahead. Riaz et al. [16] proposed a linear adaptive filter with gradient descent and support vector regression approach to predict tumor movement up to 1 sec into the future. They used data from 14 treatment sessions and a root mean square error (RMSE) was used as a metric. Their support vector regression gave them the best prediction accuracy for 400 ms and 1 sec, with RMSE less than 2 mm at 1 s. In Ichiji et al. [17] the authors proposed a tumor motion prediction using time variant seasonal ARIMA (TVSARIMA) model; they took attention in estimating the time variant periodical nature of lung tumor motion. In order to obtain better prediction accuracy, Ichiji et al. [17] combined TVSARIMA with three more methods: unweighted average, multiple regression, and multilayer perceptrons (MLPs) type of neural network (NN). The authors reached the highest prediction accuracy by using combination of TVSARIMA and MLP with 10 neurons in a hidden layer and the mean absolute error was mm at 0.5 s ahead and mm for 1second prediction horizon. Yan et al. [7] presents an adaptive approach to infer internal target position by external marker positions. For both internal and external marker motions, two networks with the same type were used. During a simulation, a patient was immobilized and positioned as if it were in a treatment room. The authors indicated that their technique was capable of predicting target position for shortterm response time (less than 10 ms). They achieved prediction error 23% on average of internal target positions based on the clinical data observed between external marker and internal marker motions. In Ma et al. [3] a tumor position was detected by an electronic portal imaging device. The methods used are adaptive filtering and nonlinear method based on Takens theorem. The adaptive filtering algorithm is fast whilst the strategy based on nonlinear time series analysis approaches better precision with the price of higher computational effort. In Murphy [18], neural networks are analyzed to correlate surrogate and tumor motion temporal prediction to compensate the lag time of tracking system; when the correlation changes rapidly with time, the simple stationary linear filter was unable to make a useful prediction, while the neural network provided the best prediction of the data with time changing correlation.
From the above reviewed achievements, it is apparent that feedforward NNs or MLPs have promising capabilities for implementation to lung motion time series prediction, and lung motion prediction with NN is a subject of great interest in medicine due to the possibility of capturing dynamics and structural aspects [4, 10]. Some authors are convinced that deep analysis is still needed [4, 10, 16, 19].
From the theoretical point of view, we shall also recall the publication of Hornik et al., 1989 [20], where it is presented that MLP can approximate a function to an arbitrary degree of accuracy that has become often cited in publications on NNs by many authors up to nowadays; however, it is not usually mentioned explicitly that the statement about arbitrary degree of accuracy of MLPs is limited only to training data because the very precise training does not necessarily imply correct functionality of the trained NN for new data, that is, for testing. Then we talk about the well known issues such as generalization capability, overfitting (overtraining) issue, or about the local minima issue of MLPs that makes proper training of NNs, especially for nonstationary data such as lung motion, a nontrivial issue.
Regarding the above mentioned issues of MLPs and considering our experience with higherorder nonlinear neural architectures we also extend our study with focus on a secondorder nonlinear neural unit which is the so called quadratic neural unit (QNU) [21–25]. QNU can be considered a standalone secondorder neural unit of higherorder NNs (HONN) or a class of polynomial NNs [26–28]. For fundamental works on higherorder NNs we can refer to works of [29–34]. We may recall that polynomial neural networks (including QNU) are attractive due to the reliable theoretical results for their universal approximation abilities according to the Weierstrass theorem and for their generalization power measured by the VapnikChervonenkis (VC) dimension [27].
For the fact we study implementation of static NNs, we use the most popular learning algorithm; that is, the LevenbergMarquardt (LM) algorithm [35, 36] that is a powerful optimization algorithm and it is easy to be implemented. LM technique is used for nonlinear leastsquares problems. We also briefly compare the performance of a classical gradient descent (GD) adaptation algorithm with the best performing predictor in our experiments. Also, because of the nonstationary nature of lung tumor motion in time, we implemented sliding window retraining (e.g., [37, 38]) to capture temporal variations in time series validity of the neural model at every sample of prediction.
In this paper, we propose and study prediction method of lung tumor motion, first, with the use of conventional static MLP with a single hidden perceptron layer and, second, with the static QNU, that is, a class of polynomial neural network (or a higherorder neural unit). We also demonstrate that QNU can be trained in a very efficient and fast way for realtime retraining. The objective of our study was to achieve the prediction accuracy within 1 mm for prediction horizon second by using NN approaches and to study capabilities of the simplest yet powerful NN models. That is, we adopt static MLPs and QNUs to achieve better prediction accuracy than in published and comparable works that are referenced above. The QNU was chosen for its high quality of nonlinear approximation and its excellent convergence due to its inparameter linearity that implies a linear optimization problem while the predictor is nonlinear [23].
Section 2 describes 3D lung tumor motion data used for the experimental study. Section 3 describes the NN models, that is, the MLP and QNU, the realtime retraining technique, the used LM and GD learning algorithms, and the modifications of the LM learning algorithm (later as MLM) to increase efficiency and the speed of the retraining for realtime computation. Section 4 presents results with real lung tumor 3D motion data, and these are discussed in Section 5 with directions for further research on unexpected move processing, on increasing the accuracy via online estimation accuracy with connotations to intensity modulated approach. At the very end, more results are shown also on additional artificial time series featuring respiration nonlinear dynamics and unexpected move in Appendix, where also the evidence of lower accuracy of linear predictor is shown for both artificial and real data.
2. Data Description
The threedimensional time series of lung tumor motion data (Figure 1) with uncontrolled respiration of a patient were obtained courtesy of Hokkaido University Hospital. To measure the threedimensional coordinates of the tumor motion as shown in Figure 1, a fiducial gold marker was implanted into the lung tumor or its neighbour, and the motion was measured in the directions of the lateral, cephalocaudal, and anteroposterior axes, respectively [15, 17]. The original sampling frequency was 30 Hz, and the spatial resolution was 0.01 mm; the time series were preprocessed by applying Kalman filter and statistical filters in order to reduce the noise and avoid abnormal data included in rough data of the time series [15, 17, 39].
The elements of vector are The dominant periods of the time series are varying around 3 seconds.
3. Prediction Methods
This section describes the neural network models used in this study. Section 3.1 gives the necessary details on the sliding window retraining technique that increased prediction accuracy (as discussed in Section 5 later). Section 3.2 gives details on the implemented classical perceptrontype static feedforward NN with a single hidden layer of neurons with sigmoidal output function (Figure 3) and the LM algorithm used for batch training of this neural architecture is recalled. Section 3.3 presents weightbyweight modification of LM that accelerates realtime computation by avoiding inverse matrix computation. Section 3.4 describes the implemented static QNU (Figure 4, equations (11) and (13)) that performs nonlinear inputoutput mapping yet its linear optimization nature suppresses the issue of local minima for convergence of neural weights. Also modification of the LM algorithm (9)–(14) for enhanced computational speed of QNU is described as the inverse matrix computation is avoided and the Jacobian is constant for static QNU.
3.1. Sliding Window Retraining
Because the respiration time series are naturally nonstationary and thus quasiperiodic with time varying frequency, mean, and amplitudes, it is impossible to obtain a generally valid model from a single training data set. Therefore, we investigated the effect of realtime retraining of the above described predictive models (Figures 3 and 4) to their prediction accuracy. By retraining with the most recent history of measured values, we capture the contemporary valid governing laws of a nonstationary system. We retrained the models at every new measured sample, that is, before each new sample prediction. This approach can be referred to as a sliding window approach (e.g., [37, 40]). Before NN retrainings, every sliding window was normalized by subtracting the mean and divided by standard deviation, respectively, for each signal (). The retraining (sliding) window for the predictive models (Figures 3 and 4) is shown in Figure 2, where stands for the mapping function of the NN model (MLP or QNU).
After the current window training is performed, the NN predicts the unknown samples ahead from the new measured value and then the data normalization, retraining, and prediction repeat when a new sample is available.
3.2. Perceptron Neural Network with LevenbergMarquardt Adaptation
The static MLP NN with discrete time notation and with a single hidden layer is given by the following equation: where is the output of the network calculated at time as an samples ahead predicted value. is a weight matrix whose rows correspond to weights of neurons in a hidden layer, is a weight vector for the output neuron, and the input vector is given for the static model (2), that is, for a directly predicting (static) model as where the length of input vector is , and the sigmoidal output function of neurons in the hidden layer is given as follows:
This network architecture (2)–(4) with neurons in its hidden layer is sketched in Figure 3, and this model was studied as a classical NN model for direct prediction of time series in Figure 1 with sliding window retraining was described in Section 3.1.
The common formula for LM algorithm for weight increments of the th hidden neuron at every epoch of training is then given as follows:where elements of are weight increments of the th neuron, is identity matrix, is a learning rate that is optionally adjustable (see below), is a vector of errors between real values and neural outputs (7), stands for matrix transposition, and is the Jacobian matrix that contains the first derivatives of the network outputs with respect to weights of the th neuron as follows: where is the length of training data (the number of samples). A training performance is for training samples given as the sum of square errorsThe LM algorithm for the perceptrontype network as in (2)–(4) (Figure 3) requires computation of the Jacobian matrix as in (6) at each epoch, so the matrix inverse has to be always calculated according to the basic LM formula (5) epoch times. The inverse matrix calculation as in (5) for the network (3) results in slowing down the realtime computation. Modified Levenberqmarquard algorithm is able to avoid that and the retraining and prediction run faster; this is presented in Section 3.3.
3.3. Perceptron Neural Network with Modified LevenbergMarquardt Adaptation
The resulting formula for modified LM algorithm for the th weight increment of the th hidden neuron at every epoch of training is then given as follows:where is a th weight increment of the th neuron, is identity matrix, is a learning rate, e is a vector of errors between real values and neural outputs (7), stands for matrix transposition, and is the Jacobian vector: that contains the first derivatives of the network outputs with respect to th weight of the th neuron as follows. Perceptron neural network predictor with this modification of LevenbergMarquardt learning algorithm is further denoted as MLP predictor with MLM learning.
In next subsection we show QNU and its linear nature of optimization (by LM algorithm) that, in principle, prevents QNU from local minima issue for a given training data set, so the weight convergence of QNU is superior to conventional perceptrontype neural networks [23, 41].
3.4. Quadratic Neural Unit with LevenbergMarquardt Adaptation
QNU may be considered as a special case of higherorder neural unit or as a case of polynomial NN. The static QNU is sketched in Figure 4.
The output of QNU from Figure 4 can be written in a vector multiplication form that can be decomposed into a long vector representation as follows:where (as shown in Figure 4), is a predicted value, are external neural inputs at a sample time , are neural weights of QNU, is a longvector representation of weight matrix of QNU, and colx is a long column vector of polynomial terms of neural inputs defined as follows:Notice that for weight optimization of polynomial static model (11), all and are substituted with measured training data, so (11) yields a linear combination of neural weights that has, in principle, a unique solution for a given training data. Thus, contrary to MLP networks, the linear optimization nature of QNU implies that QNU avoids the local minima issue for a given training data while the neural model maintains high quality of nonlinear approximation that we have observed so far [23, 41].
Another advantage of QNU against MLP is the fact that the Jacobian matrix of QNU, that is, , derived accordingly to (6), becomes merely function of inputs; thus the of QNU becomes a constant for its all training epochs. Then, of QNU is given for its all weights as follows:and this matrix (13) is evaluated only once, so the weight updates by LM formula (5) can be evaluated with only varying error that is recalculated at each epoch of training, so the the matrix multiplications and inversion with are calculated only once for each retraining of QNU. However, the natural disadvantage of QNU is the exponentially increasing number of weights with number of inputs (e.g., QNU with external inputs has weights), so the inverse matrix operation in the LM formula significantly slows down the retraining even if it is calculated once for all epochs. Also, choice of a proper technique for computation of precise inverse matrix may be an issue itself that can negatively influence the training technique. Therefore we may implement a weightbyweight calculation approach (modified LevenbergMarquardt adaptation) that avoids matrix inversion to calculate all neural weight updates by the LM algorithm (as indicated in previous section and also used for MLP in that same subsection). The approach is shown in next subsection, where we show that Jacobian matrix (6) of static QNU can be calculated only once and that also the matrix inversion (5) can be avoided for QNU (Section 3.5). Thus the QNU becomes computationally fast enough for realtime calculation even on a PC (Ubuntu 12.04, Intel i5).
3.5. Quadratic Neural Unit with Modified LevenbergMarquardt Adaptation
In this subsection, we present how we modified LM algorithm to accelerate training of QNU by avoiding the inverse matrix computation. A general column of Jacobian matrix of QNU (10) that corresponds to a general weight is vector denoted as and it is written as follows:
Then a singleweight increment is formally calculated according to original LM formula (5) and because the term results in a scalar, we can use formula (9).
It is much faster to calculate individual vectors correspondingly to individual weights in a for loop using merely division (14) rather than to calculate all weight updates once by the original LM formula with the inverse of a large matrix, that is, for QNU with too many inputs. Notice that all are calculated only once before the training in epochs starts and then we also calculate the weight updates only with varying that is the only vector that is recalculated at every epoch in the modified LM formula (14). As a result of the above modification of the LM algorithm for QNU, the computation speed of QNU with retraining and prediction at every sample increased significantly (Figure 6). In other words, we are capable to implement the realtime prediction with retraining on a commonly available computational hardware without the need for more powerful one and the prototype of the software can be typically implemented either in Python or Matlab. The technique is further denoted as QNU predictor with MLM learning.
3.6. Quadratic Neural Unit with Normalised Gradient Descent Adaptation
In this subsection we present the normalized gradient descent algorithm [42, 43] for QNU with adaptation for prediction of lung tumor motion. This method of adaptation recalculates weights for every new sample. The weight update formula could be presented as follows: where is the normalised learning rate (16), is prediction error (7), and the colx is vector of inputs, obtained from vector (3) as shown in (12).
To improve stability of weight update system (15) during GD adaptation, the learning rate is normalized at every sample time as follows:where stands for learning rate defined before the start of the simulation.
4. Experimental Analysis
4.1. Evaluation Criteria
Experimental analysis was performed on real respiration data of lung motion as described in Section 2 and using the two predictive models and the techniques described in Section 3. The objective of the analysis is to investigate the potentials for the prediction accuracy of 1 mm for prediction horizon of 1 s. We also present a more exhaustive study and comparison of static NN performance for prediction of lung motion using the realtime retraining technique. To evaluate the performance under the longterm condition required for clinical use, we highlight the results for prediction of the prediction horizons of 0.5 s and 1 s.
As the lung motion is measured in three axes, we analysed the predicting accuracy for various configurations by a 3D mean absolute error (MAE) as follows:where , , and are the predicting errors of corresponding axes, respectively. From the 3D error we can get the MAE with formula as follows:where is the number of testing samples.
4.2. Experiments Setups
Also, the effect of various input configurations of the length of neural inputs for the NN architectures (MLP in Figure 3, QNU in Figure 4) was studied. The optimum number of inputoutput training patterns and number of neurons in the hidden layer (in the case of MLP) were estimated after experiments. For MLP, we run each setup for the number of neurons in hidden layer as (Figure 3) and this each instance of MLP predictor was repeated 10× from allays different random initial conditions. We highlight the results for prediction horizon s as summarized in Table 1.

4.3. Results
As it was specified in previous subsection, we ran 356 simulations for QNU and 2475 for MLP for real lung tumor motion time series shown in Figure 1. The results for all settings are shown in pivot chart of MAE in Figure 5. As we can see in that chart, results vary according to all parameters of simulation.
We concluded to setup 5 to 8 epochs for the sliding window retraining for MLP and 8 epochs for QNU as we could notice that the mean absolute error was not improved with more number of training epochs into the window especially for longterm prediction (up to 1 s). For pretraining before actual prediction we concluded to use 800 epochs for MLP and QNU using LevenbergMarquardt adaptation and 400 epochs for QNU using gradient descent adaptation.
We have to highlight that results of simulations with MLP depended more on random selection of initial weights than with QNU. The standard deviation of MAE of QNU was superior to MLPs as it is shown also in Table 2. This is most naturaly due to the known local minima issue of MLP with LM algorithm while QNU is linear in its parameters, so QNU features a single minimum optimization problem. The lowest MAE was achieved by QNU with MLM adaptation and sampling 15 Hz as it is shown in Table 3. And in general, it is possible to say that simulations with smaller size of , that is, covering the range of about two respiration cycles, have better results. Difference of the MAE between using LM and MLM can also be caused by initial random weights. However, the initial random weights were important to verify the general validity of this prediction approach. Also in general we can see from Figure 5 that QNU was more accurate than MLP. Figure 6 shows the pivot chart of computational speeds of all simulations. The higher the value on axis means the higher the computational speed. The fastest prediction was achieved with GD and MLM in combination with QNU for sampling 15 Hz, but the accuracy of GD with QNU was much worse than on with MLM. For almost of all used settings, the MLM was the fastest learning algorithm as it avoids inverse Jacobian matrix calculation. For the MLP predictor, the difference in computational speed of MLM and LM was not that high as the Jacobian matrices of MLP were not so large here. The computation speed of QNU with MLM is significantly fastest as QNU calculates Jacobian only once and MLM avoids its inversion. The most accurate predictions and also the second fastest ones were obtained for QNU with the slower sampling 15 Hz, faster sampling 30 Hz, and with MLM learning algorithm as it is shown in Table 2. The prediction including retraining with QNU performed (PC, Ubuntu 12.04, Intel i5) in average 16 samples per second for one time series that is faster than real time. QNU performed also statistically better for all setups than MLP as regards the mean and standard deviation of MAE as it is shown in Table 3.


5. Discussion
As shown in Section 4.3, the results vary according to big amount of simulation settings. So far, we found the best algorithm for lung tumor motion prediction to be the QNU in combination with MLM. This prediction model achieved better accuracy than MLP and also it was fast in comparison with MLP model. Another advantage of QNU was better independence from initial weights for LevenbergMarquardt algorithm. The choice of initial weights affected the prediction precision of MLP models (because of local minima issue mentioned earlier) and that accuracy issue can be a crucial problem for realtime usage of LM algorithm with MLP predictors. According to our research that is presented in this paper, we can recommend the QNU predictor with MLM algorithm as a more suitable method than MLP or other reviewed approaches for fast respiratory time series prediction. However, the MLP predictors shall be further investigated as they are, in principle, capable of very high prediction accuracy and other suitable learning algorithm shall be investigated. For real implementations in a nearfuture, the computational speed (of MLP and of other approaches) might be significantly improved by nowadays spreading chipset on board and FPGA technologies. Our proposed prediction method is based on realtime retraining that can capture varying dynamics of a patient respiration. It shall be highlighted that our method was applied to lung tumor motion without any control of patient respiration. This implies that the dynamics of the patient respiration was varying unexpectedly. As regards instant variations of respiration dynamics and unexpected moves of a patient, we focus our research toward adaptive novelty detection for estimation of actual prediction accuracy [44]. Such approach seems to be promising for further improvement of prediction accuracy and for instant detection of unexpected moves with prospects to intensity modulated radiation tracking therapy.
6. Conclusions
In this paper, we have proposed and investigated realtime series predictive models of lung tumor 3D time series. A MLP with one hidden layer and quadratic neural unit were proposed and studied as predictive models. The studied learning rules for the models were the gradient descent adaptation (GD) and LevenbergMarquardt batch optimization implemented as realtime retraining technique. We further modified LM algorithm for faster realtime calculation. We demonstrated and compared the predictive capability of the models and algorithms on respiratory 3D lung tumor motion time series for realtime prediction. For the GD and LM algorithm, we can conclude the superiority of QNU over MLP as regards the accuracy and realtime computational efficiency and reproducibility of the results. The inparameter linearity of QNU avoided local minima issue during optimization while the initial weight setup of MLP importantly affects retraining accuracy for these comprehensible learning algorithms. The prediction results obtained by the predictive models satisfied the goals of our work for the prediction accuracy of 3D MAE of 1 mm for 1second prediction horizon while the computational time was well shorter than the real treatment time.
Appendix
Artificial Data Experiment
Up to date, we unfortunately have not obtained another real lung tumor 3D motion data with uncontrolled patient respiration. Thus we generated three artificial time series shown in Figure 7 to validate the proposed approach. We created first time series 1 by a nonlinear function (A.1) as followsand the second time series was generated by (A.1) with randomly varying main frequency every 3 samples. The third time series was generated using the famous chaotic MackeyGlass equation as follows:where denotes continuous time, and the chaotic behavior was generated by the setup of , , and the lag .
The results on 3D MAE and speed of computation are shown in Figures 8, 9, 10, and 11. These results confirm our achievements on real data sets, in particularly, the QNU appears accurate and efficient predictor in comparison to conventional MLP networks when GD, LM, or MLM learning are used.
For completeness of this study, we show results achieved with (with realtime retraining) linear predictor LNU (linear neural unit) that demonstrates the need for nonlinear predictive models because of prediction accuracy.
For validation of result reproducibility, we also performed the computations on artifical data with another HW (PC, Windows 7, i7), so the computational speeds can differ from the ones achieved for real data. The results on artifical data confirm our achievements with real tumor motion data with uncontrolled respiration.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the Grant Agency of the Czech Technical University in Prague, Grant no. SGS12/177/OHK2/3T/12 and by the Kayamori Foundation of Informational Science Advancement and JSPS Kakenhi Grant no. 25293258. The authors are grateful to the Japan Society for the Promotion of Science (JSPS) and Japan Science and Technology Agency (JST) for their research funding and also to the Matsumae International Foundation that gave them the opportunity of international cooperation since 2009. They specially thank Dr. Sharito and his colleagues from Hokkaido University Hospital for sharing the tumor motion data with them.
References
 J. D. P. Hoisak, K. E. Sixel, R. Tirona, P. C. F. Cheung, and J.P. Pignol, “Prediction of lung tumour position based on spirometry and on abdominal displacement: accuracy and reproducibility,” Radiotherapy & Oncology, vol. 78, no. 3, pp. 339–346, 2006. View at: Publisher Site  Google Scholar
 G. C. Sharp, S. B. Jiang, S. Shimizu, and H. Shirato, “Prediction of respiratory tumour motion for realtime imageguided radiotherapy,” Physics in Medicine and Biology, vol. 49, no. 3, pp. 425–440, 2004. View at: Publisher Site  Google Scholar
 L. Ma, C. Herrmann, and K. Schilling, “Modeling and prediction of lung tumor motion for robotic assisted radiotherapy,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '07), pp. 189–194, San Diego, Calif, USA, November 2007. View at: Publisher Site  Google Scholar
 X. A. Li, P. J. Keall, and C. G. Orton, “Respiratory gating for radiation therapy is not ready for prime time,” Medical Physics, vol. 34, no. 3, Article ID 86787, 2007. View at: Google Scholar
 M. Isaksson, J. Jalden, and M. J. Murphy, “On using an adaptive neural network to predict lung tumor motion during respiration for radiotherapy applications,” Medical Physics, vol. 32, no. 12, pp. 3801–3809, 2005. View at: Publisher Site  Google Scholar
 K. Demachi, H. Zhu, M. Ishikawa, and H. Shirato, “Predictive Simulation of tumor movement for chasing radiotherapy,” Journal of the Japan Society of Applied Electromagnetics and Mechanics, vol. 17, pp. 222–226, 2009. View at: Google Scholar
 H. Yan, F.F. Yin, G.P. Zhu, M. Ajlouni, and J. H. Kim, “Adaptive prediction of internal target motion using external marker motion: a technical study,” Physics in Medicine and Biology, vol. 51, no. 1, pp. 31–44, 2006. View at: Publisher Site  Google Scholar
 D. Ruan, “Kernel density estimationbased realtime prediction for respiratory motion,” Physics in Medicine and Biology, vol. 55, no. 5, pp. 1311–1326, 2010. View at: Publisher Site  Google Scholar
 D. Ruan, “Prospective detection of large prediction errors: a hypothesis testing approach,” Physics in Medicine and Biology, vol. 55, no. 13, pp. 3885–3904, 2010. View at: Publisher Site  Google Scholar
 M. J. Murphy and S. Dieterich, “Comparative performance of linear and nonlinear neural networks to predict irregular breathing,” Physics in Medicine and Biology, vol. 51, pp. 5903–5914, 2006. View at: Publisher Site  Google Scholar
 I. Buzurovic, T. K. Podder, K. Huang, and Y. Yu, “Tumor motion prediction and tracking in adaptive radiotherapy,” in Proceedings of the IEEE International Conference on Bioinformatics and Bioengineering (BIBE '10), pp. 273–278, Philadelphia, Pa, USA, June 2010. View at: Publisher Site  Google Scholar
 G. Benchetrit, “Breathing pattern in humans: diversity and individuality,” Respiration Physiology, vol. 122, no. 23, pp. 123–129, 2000. View at: Publisher Site  Google Scholar
 S. Dieterich, J. Tang, J. Rodgers, and K. Cleary, “Sking respiratory motion tracking for stereotactic radiosurgery using the CyberKnife,” International Congress Series, vol. 1256, pp. 130–136, 2003. View at: Publisher Site  Google Scholar
 A. Sahih, O. C. L. Haas, J. H. Goodband, D. Putra, J. A. Mills, and K. J. Burnham, “Respiratory motion prediction for adaptive radiotherapy,” in Proceedings of the DVDROM IAR & ACD Conference, Nancy, France, 2006. View at: Google Scholar
 N. Homma, M. Sakai, H. Endo, M. Mitsuya, Y. Takai, and M. Yoshizawa, “A new motion management method for lung tumor tracking radiation therapy,” WSEAS Transactions on Systems, vol. 8, no. 4, pp. 471–480, 2009. View at: Google Scholar
 N. Riaz, P. Shanker, R. Wiersma et al., “Predicting respiratory tumor motion with multidimensional adaptive filters and support vector regression,” Physics in Medicine and Biology, vol. 54, no. 19, pp. 5735–5748, 2009. View at: Publisher Site  Google Scholar
 K. Ichiji, N. Homma, I. Bukovsky, and M. Yoshizawa, “Intelligent sensing of biomedical signals: lung tumor motion prediction for accurate radiotherapy,” in Proceeding of the IEEE Merging Fields Of Computational Intelligence And Sensor Technology (CompSens '11), pp. 35–41, Paris, France, April 2011. View at: Publisher Site  Google Scholar
 M. J. Murphy, “Using neural networks to predict breathing motion,” in Proceedings of the 7th International Conference on Machine Learning and Applications (ICMLA '08), pp. 528–532, San Diego, Calif, USA, December 2008. View at: Publisher Site  Google Scholar
 M. Kakar, H. Nyström, L. R. Aarup, T. J. Nøttrup, and D. R. Olsen, “Respiratory motion prediction by using the adaptive neuro fuzzy inference system (ANFIS),” Physics in Medicine and Biology, vol. 50, no. 19, pp. 4721–4728, 2005. View at: Publisher Site  Google Scholar
 K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359–366, 1989. View at: Publisher Site  Google Scholar
 M. M. Gupta, J. Liang, and N. Homma, Static and Dynamic Neural Networks: From Fundamentals to Advanced Theory, John Wiley & Sons, New Jersey, NJ, USA, 2003.
 M. M. Gupta, N. Homma, Z.G. Hou, M. G. Solo, and I. Bukovsky, “Higher order neural networks: fundamental theory and applications,” in Artificial Higher Order Neural Networks for Computer Science and Engineering: Trends for Emerging Applications, M. Zhang, Ed., pp. 397–422, IGI Global, 2010. View at: Google Scholar
 M. Gupta, M. Bukovsky, I. Homma, M. G. A. Solo, and Z.G. Hou, “Fundamentals of higher order neural networks for modeling and simulation,” in Artificial Higher Order Neural Networks for Modeling and Simulation, M. Zhang, Ed., chapter 6, pp. 103–133, IGI Global, 2012. View at: Publisher Site  Google Scholar
 I. Bukovsky, J. Bila, M. M. Gupta, Z.G. Hou, and N. Homma, “Foundation and classification of nonconventional neural units and paradigm of nonsynaptic neural interaction,” in Discoveries and Breakthroughs in Cognitive Informatics and Natural Intelligence, Y. Wang, Ed., University of Calgary, Calgary, Canada; IGI Publishing, Hershey, Pa, USA, 2009. View at: Google Scholar
 I. Bukovsky and J. Bila, “Adaptive evaluation of complex dynamical systems using lowdimensional neural architectures,” in Advances in Cognitive Informatics and Cognitive Computing, D. Zhang, Y. Wang, and W. Kinsner, Eds., vol. 323 of Studies in Computational Intelligence, pp. 33–57, Springer, 2010. View at: Publisher Site  Google Scholar
 A. G. Ivakhnenko, “Polynomial theory of complex systems,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 1, no. 4, pp. 364–378, 1971. View at: Google Scholar  MathSciNet
 N. Y. Nikolaev and H. Iba, “Learning polynomial feedforward neural networks by genetic programming and backpropagation,” IEEE Transactions on Neural Networks, vol. 14, no. 2, pp. 337–350, 2003. View at: Publisher Site  Google Scholar
 N. Y. Nikolaev and H. Iba, Adaptive Learning of Polynomial Networks: Genetic Programming, Backpropagation and Bayesian Methods, vol. 14 of Genetic and Evolutionary Computation, Springer, New York, NY, USA, 2006. View at: MathSciNet
 Y. Shin and J. Ghosh, “The pisigma network: an efficient higherorder neural network for pattern classification and function approximation,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '91), pp. 13–18, Seattle, Wash, USA, July 1991. View at: Google Scholar
 R. W. Softky and D. M. Kammen, “Correlations in high dimensional or asymmetric data sets: Hebbian neuronal processing,” Neural Networks, vol. 4, no. 3, pp. 337–347, 1991. View at: Publisher Site  Google Scholar
 J. G. Taylor and S. Coombes, “Learning higher order correlations,” Neural Networks, vol. 6, no. 3, pp. 423–427, 1993. View at: Publisher Site  Google Scholar
 W. Schmidt and J. Davis, “Pattern recognition properties of various feature spaces for higher order neural networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 8, pp. 795–801, 1993. View at: Publisher Site  Google Scholar
 E. B. Kosmatopoulos, M. M. Polycarpou, M. A. Christodoulou, and P. A. Ioannou, “Highorder neural network structures for identification of dynamical systems,” IEEE Transactions on Neural Networks, vol. 6, no. 2, pp. 422–431, 1995. View at: Publisher Site  Google Scholar
 M. Heywood and P. Noakes, “Framework for improved training of SigmaPi networks,” IEEE Transactions on Neural Networks, vol. 6, no. 4, pp. 893–903, 1995. View at: Publisher Site  Google Scholar
 D. W. Marquardt, “An algorithm for leastsquares estimation of nonlinear parameters,” SIAM Journal of Applied Mathematics, vol. 11, no. 2, pp. 431–441, 1963. View at: Google Scholar  MathSciNet
 J. J. Moré, “The LevenbergMarquardt algorithm: implementation and theory,” in Numerical Analysis, vol. 630 of Lecture Notes in Mathematics, pp. 105–116, Springer, Berlin, Germany, 1978. View at: Publisher Site  Google Scholar
 D. F. Morgado, A. Antunes, J. Vieira, and A. Mota, “Implementing the LevenbergMarquardt algorithm online: a sliding window approach with early stopping,” in Proceedings of the 2nd IFAC Workshop on Advanced Fuzzy/Neural Control, 2004. View at: Google Scholar
 F. M. Dias, A. Antunes, J. Vieira, and A. M. Mota, “Online training of neural networks: a sliding window approach for the LevenbergMarquardt algorithm,” in Proceedings of the 1st International WorkConference on the Interplay Between Natural and Artificial Computation (IWINAC '05), pp. 577–585, Las Palmas, Spain, June 2005. View at: Publisher Site  Google Scholar
 K. Ichiji, M. Sakai, N. Homma, Y. Takai, and M. Yoshizawa, “SUHHBRB10: adaptive seasonal autoregressive model based intrafractional lung tumor motion prediction for continuously irradiation,” Medical Physics, vol. 37, pp. 3331–3332, 2010. View at: Google Scholar
 F. M. Dias, A. Antunes, J. Vieira, and A. M. Mota, “Online training of neural networks: a sliding window approach for the LevenbergMarquardt algorithm,” in Proceedings of the 1st International WorkConference on the Interplay Between Natural and Artificial Computation (IWINAC 2005), vol. 3562 of Lecture Notes in Computer Science, pp. 577–585, Las Palmas, Spain, June 2005. View at: Publisher Site  Google Scholar
 I. Bukovsky, K. Ichiji, N. Homma, M. Yoshizawa, and R. Rodriguez, “Testing potentials of dynamic quadratic neural unit for prediction of lung motion during respiration for tracking radiation therapy,” in Proceedings of the IEEE WCCI International Joint Conference on Neural Networks, Barcelona, Spain, July 2010. View at: Google Scholar
 D. P. Mandic, “A generalized normalized gradient descent algorithm,” IEEE Signal Processing Letters, vol. 11, no. 2, pp. 115–118, 2004. View at: Publisher Site  Google Scholar
 B. Widrow and S. D. Stearns, Adaptive Signal Processing, PrenticeHall, Englewood Cliffs, NJ, USA, 1985.
 I. Bukovsky, N. Homma, M. Cejnek, and K. Ichiji, “Study of learning entropy for novelty detection in lung tumor motion prediction for target tracking radiation therapy,” in Proceedings of the International Joint Conference on Neural Networks, Beijing, China, 2014. View at: Google Scholar
Copyright
Copyright © 2015 Ivo Bukovsky et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.