Table of Contents Author Guidelines Submit a Manuscript
Computational Intelligence and Neuroscience
Volume 2016, Article ID 3917892, 14 pages
http://dx.doi.org/10.1155/2016/3917892
Research Article

FPGA-Based Stochastic Echo State Networks for Time-Series Forecasting

Physics Department, University of the Balearic Islands, 07122 Palma de Mallorca, Spain

Received 2 August 2015; Revised 8 October 2015; Accepted 15 October 2015

Academic Editor: Mikhail A. Lebedev

Copyright © 2016 Miquel L. Alomar et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. A. M. Schäfer and H.-G. Zimmermann, “Recurrent neural networks are universal approximators,” International Journal of Neural Systems, vol. 17, no. 4, pp. 253–263, 2007. View at Publisher · View at Google Scholar · View at Scopus
  2. M. Rabinovich, R. Huerta, and G. Laurent, “NEUROSCIENCE: transient dynamics for neural processing,” Science, vol. 321, no. 5885, pp. 48–50, 2008. View at Publisher · View at Google Scholar · View at Scopus
  3. M. Lukoševičius and H. Jaeger, “Reservoir computing approaches to recurrent neural network training,” Computer Science Review, vol. 3, no. 3, pp. 127–149, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus
  4. H. Jaeger, “The ‘echo state’ approach to analysing and training recurrent neural networks—with an Erratum note,” GMD Technical Report, German National Research Center for Information Technology, 2010. View at Google Scholar
  5. M. Lukoševičius, H. Jaeger, and B. Schrauwen, “Reservoir computing trends,” KI—Künstliche Intelligenz, vol. 26, no. 4, pp. 365–371, 2012. View at Publisher · View at Google Scholar
  6. W. Maass, T. Natschläger, and H. Markram, “Real-time computing without stable states: a new framework for neural computation based on perturbations,” Neural Computation, vol. 14, no. 11, pp. 2531–2560, 2002. View at Publisher · View at Google Scholar · View at Scopus
  7. H. Jaeger and H. Haas, “Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication,” Science, vol. 304, no. 5667, pp. 78–80, 2004. View at Publisher · View at Google Scholar · View at Scopus
  8. D. Verstraeten, B. Schrauwen, M. D'Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks, vol. 20, no. 3, pp. 391–403, 2007. View at Publisher · View at Google Scholar · View at Scopus
  9. J. B. Butcher, D. Verstraeten, B. Schrauwen, C. R. Day, and P. W. Haycock, “Reservoir computing and extreme learning machines for non-linear time-series data analysis,” Neural Networks, vol. 38, pp. 76–89, 2013. View at Publisher · View at Google Scholar · View at Scopus
  10. M. C. Soriano, D. Brunner, M. Escalona-Morán, C. R. Mirasso, and I. Fischer, “Minimal approach to neuro-inspired information processing,” Frontiers in Computational Neuroscience, vol. 9, article 68, 2015. View at Publisher · View at Google Scholar
  11. Y. Li, J. Zhang, D. Pan, and D. Hu, “A study of speech recognition based on RNN-RBM language model,” Journal of Computer Research and Development, vol. 51, no. 9, pp. 1936–1944, 2014. View at Publisher · View at Google Scholar
  12. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015. View at Publisher · View at Google Scholar
  13. Ł. Brocki and K. Marasek, “Deep belief neural networks and bidirectional long-short term memory hybrid for speech recognition,” Archives of Acoustics, vol. 40, no. 2, pp. 191–195, 2015. View at Publisher · View at Google Scholar
  14. X. Ma, H. Yu, Y. Wang, Y. Wang, and J. Gomez-Gardenes, “Large-scale transportation network congestion evolution prediction using deep learning theory,” PLoS ONE, vol. 10, no. 3, Article ID e0119044, 2015. View at Publisher · View at Google Scholar
  15. G. A. Abandah, A. Graves, B. Al-Shagoor, A. Arabiyat, F. Jamour, and M. Al-Taee, “Automatic diacritization of Arabic text using recurrent neural networks,” International Journal on Document Analysis and Recognition, vol. 18, no. 2, pp. 183–197, 2015. View at Publisher · View at Google Scholar
  16. A. Rodan and P. Tiňo, “Minimum complexity echo state network,” IEEE Transactions on Neural Networks, vol. 22, no. 1, pp. 131–144, 2011. View at Publisher · View at Google Scholar · View at Scopus
  17. J. Misra and I. Saha, “Artificial neural networks in hardware: a survey of two decades of progress,” Neurocomputing, vol. 74, no. 1–3, pp. 239–255, 2010. View at Publisher · View at Google Scholar · View at Scopus
  18. L. Appeltant, M. C. Soriano, G. Van der Sande et al., “Information processing using a single dynamical node as complex system,” Nature Communications, vol. 2, article 468, 2011. View at Publisher · View at Google Scholar · View at Scopus
  19. L. Larger, M. C. Soriano, D. Brunner et al., “Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing,” Optics Express, vol. 20, no. 3, pp. 3241–3249, 2012. View at Publisher · View at Google Scholar · View at Scopus
  20. Y. Paquot, F. Duport, A. Smerieri et al., “Optoelectronic reservoir computing,” Scientific Reports, vol. 2, article 287, 6 pages, 2012. View at Publisher · View at Google Scholar · View at Scopus
  21. K. Vandoorne, W. Dierckx, B. Schrauwen et al., “Toward optical signal processing using photonic reservoir computing,” Optics Express, vol. 16, no. 15, pp. 11182–11192, 2008. View at Publisher · View at Google Scholar · View at Scopus
  22. B. Schrauwen, M. D'Haene, D. Verstraeten, and J. Van Campenhout, “Compact hardware liquid state machines on FPGA for real-time speech recognition,” Neural Networks, vol. 21, no. 2-3, pp. 511–523, 2008. View at Publisher · View at Google Scholar · View at Scopus
  23. A. Muthuramalingam, S. Himavathi, and E. Srinivasan, “Neural network implementation using FPGA: issues and application,” International Journal of Information Technology, vol. 4, no. 2, pp. 86–92, 2008. View at Google Scholar
  24. B. R. Gaines, “R68-18 random pulse machines,” IEEE Transactions on Computers, vol. 17, no. 4, p. 410, 1968. View at Publisher · View at Google Scholar
  25. S. L. Toral, J. M. Quero, and L. G. Franquelo, “Stochastic pulse coded arithmetic,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '00), vol. 1, pp. 599–602, IEEE, Geneva, Switzerland, May 2000. View at Publisher · View at Google Scholar
  26. Y. Kondo and Y. Sawada, “Functional abilities of a stochastic logic neural network,” IEEE Transactions on Neural Networks, vol. 3, no. 3, pp. 434–443, 1992. View at Publisher · View at Google Scholar · View at Scopus
  27. S. L. Bade and B. L. Hutchings, “FPGA-based stochastic neural networks-implementation,” in Proceedings of the IEEE Workshop on FPGAs for Custom Computing Machines, pp. 189–198, IEEE, Napa Valley, Calif, USA, April 1994. View at Publisher · View at Google Scholar
  28. B. D. Brown and H. C. Card, “Stochastic neural computation. I. Computational elements,” IEEE Transactions on Computers, vol. 50, no. 9, pp. 891–905, 2001. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  29. J. L. Rosselló, V. Canals, and A. Morro, “Hardware implementation of stochastic-based neural networks,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '10), Barcelona, Spain, July 2010.
  30. D. Verstraeten, “Stochastic bitstream-based reservoir computing with feedback,” in Proceedings of the 5th FirW PhD Symposium, 2005.
  31. D. Verstraeten, B. Schrauwen, and D. Stroobandt, “Reservoir computing with stochastic bitstream neurons,” in Proceedings of the 16th Annual Workshop on Circuits, Systems and Signal Processing (ProRisc '05), pp. 454–459, Veldhoven, The Netherlands, November 2005.
  32. A. Morro, V. Canals, A. Oliver, M. L. Alomar, and J. L. Rossello, “Ultra-fast data-mining hardware architecture based on stochastic computing,” PLoS One, vol. 10, no. 5, Article ID e0124176, 2015. View at Publisher · View at Google Scholar
  33. A. Alaghi and J. P. Hayes, “Survey of stochastic computing,” ACM Transactions on Embedded Computing Systems, vol. 12, no. 2, article 92, 19 pages, 2013. View at Publisher · View at Google Scholar · View at Scopus
  34. R. V. Kulkarni, A. Förster, and G. K. Venayagamoorthy, “Computational intelligence in wireless sensor networks: a survey,” IEEE Communications Surveys and Tutorials, vol. 13, no. 1, pp. 68–96, 2011. View at Publisher · View at Google Scholar · View at Scopus
  35. M. A. Escalona-Morán, M. C. Soriano, I. Fischer, and C. R. Mirasso, “Electrocardiogram classification using reservoir computing with logistic regression,” IEEE Journal of Biomedical and Health Informatics, vol. 19, no. 3, pp. 892–898, 2015. View at Publisher · View at Google Scholar
  36. D. Zhang and H. Li, “A stochastic-based FPGA controller for an induction motor drive with integrated neural network algorithms,” IEEE Transactions on Industrial Electronics, vol. 55, no. 2, pp. 551–561, 2008. View at Publisher · View at Google Scholar · View at Scopus
  37. M. L. Alomar, V. Canals, V. Martínez-Moll, and J. L. Rosselló, “Low-cost hardware implementation of reservoir computers,” in Proceedings of the 24th International Workshop on Power and Timing Modeling, Optimization and Simulation (PATMOS '14), pp. 1–5, Palma de Mallorca, Spain, October 2014. View at Publisher · View at Google Scholar · View at Scopus
  38. J. Hayes, “Introduction to stochastic computing and its challenges,” in Proceedings of the 52nd Annual Design Automation Conference, pp. 1–3, San Francisco, Calif, USA, June 2015.
  39. M. T. Tommiska, “Efficient digital implementation of the sigmoid function for reprogrammable logic,” IEE Proceedings: Computers and Digital Techniques, vol. 150, no. 6, pp. 403–411, 2003. View at Publisher · View at Google Scholar · View at Scopus
  40. W. Qian, X. Li, M. D. Riedel, K. Bazargan, and D. J. Lilja, “An architecture for fault-tolerant computation with stochastic logic,” IEEE Transactions on Computers, vol. 60, no. 1, pp. 93–105, 2011. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  41. K. Basterretxea, J. M. Tarela, and I. Del Campo, “Digital design of sigmoid approximator for artificial neural networks,” Electronics Letters, vol. 38, no. 1, pp. 35–37, 2002. View at Publisher · View at Google Scholar · View at Scopus
  42. A. S. Weigend and N. A. Gershenfeld, Time Series Prediction: Forecasting the Future and Understanding the Past, Addison-Wesley, Reading, Mass, USA, 1994.
  43. Altera FPGAs, https://www.altera.com/products/fpga/cyclone-series.html.