Table of Contents Author Guidelines Submit a Manuscript
Journal of Robotics
Volume 2015, Article ID 643869, 10 pages
http://dx.doi.org/10.1155/2015/643869
Research Article

Action Selection and Operant Conditioning: A Neurorobotic Implementation

Département d’Informatique, Université du Québec à Montréal (UQAM), Succursale Centre-Ville, Case Postale 8888, Montreal, QC, Canada H3C 3P8

Received 23 January 2015; Revised 7 April 2015; Accepted 14 May 2015

Academic Editor: Maki K. Habib

Copyright © 2015 André Cyr and Frédéric Thériault. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. A. K. Seth, T. J. Prescott, and J. J. Bryson, Modelling Natural Action Selection, Cambridge University Press, Cambridge, UK, 2011.
  2. M. D. Humphries, K. Gurney, and T. J. Prescott, “Is there a brainstem substrate for action selection?” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 362, no. 1485, pp. 1627–1639, 2007. View at Publisher · View at Google Scholar · View at Scopus
  3. J. I. Gold and M. N. Shadlen, “The neural basis of decision making,” Annual Review of Neuroscience, vol. 30, pp. 535–574, 2007. View at Publisher · View at Google Scholar · View at Scopus
  4. M. Khamassi, L. Lachèze, B. Girard, A. Berthoz, and A. Guillot, “Actor-critic models of reinforcement learning in the basal ganglia: from natural to artificial rats,” Adaptive Behavior, vol. 13, no. 2, pp. 131–148, 2005. View at Publisher · View at Google Scholar · View at Scopus
  5. N. F. Lepora, C. W. Fox, M. H. Evans, M. E. Diamond, K. Gurney, and T. J. Prescott, “Optimal decision-making in mammals: insights from a robot study of rodent texture discrimination,” Journal of the Royal Society Interface, vol. 9, no. 72, pp. 1517–1528, 2012. View at Publisher · View at Google Scholar · View at Scopus
  6. T. J. Prescott, F. M. M. González, K. Gurney, M. D. Humphries, and P. Redgrave, “A robot model of the basal ganglia: behavior and intrinsic processing,” Neural Networks, vol. 19, no. 1, pp. 31–61, 2006. View at Publisher · View at Google Scholar · View at Scopus
  7. W. Maass, “Networks of spiking neurons: the third generation of neural network models,” Neural Networks, vol. 10, no. 9, pp. 1659–1671, 1997. View at Publisher · View at Google Scholar · View at Scopus
  8. T. C. Stewart, T. Bekolay, and C. Eliasmith, “Learning to select actions with spiking neurons in the basal ganglia,” Frontiers in Neuroscience, vol. 6, article 2, 2012. View at Publisher · View at Google Scholar · View at Scopus
  9. S. Skorheim, P. Lonjers, and M. Bazhenov, “A spiking network model of decision making employing rewarded STDP,” PLoS ONE, vol. 9, no. 3, Article ID e90821, 2014. View at Publisher · View at Google Scholar · View at Scopus
  10. E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Transactions on Neural Networks, vol. 14, no. 6, pp. 1569–1572, 2003. View at Publisher · View at Google Scholar · View at Scopus
  11. S. Faumont, T. H. Lindsay, and S. R. Lockery, “Neuronal microcircuits for decision making in C. elegans,” Current Opinion in Neurobiology, vol. 22, no. 4, pp. 580–591, 2012. View at Publisher · View at Google Scholar · View at Scopus
  12. M. Zhang, W. R. Schafer, and R. Breitling, “A circuit model of the temporal pattern generator of Caenorhabditis egg-laying behavior,” BMC Systems Biology, vol. 4, no. 1, article 81, 2010. View at Publisher · View at Google Scholar · View at Scopus
  13. A. L. Stöckl, R. Petie, and D.-E. Nilsson, “Setting the pace: new insights into central pattern generator interactions in box jellyfish swimming,” PLoS ONE, vol. 6, no. 11, Article ID e27201, 2011. View at Publisher · View at Google Scholar · View at Scopus
  14. C. R. von Reyn, P. Breads, M. Y. Peek et al., “A spike-timing mechanism for action selection,” Nature Neuroscience, 2014. View at Publisher · View at Google Scholar · View at Scopus
  15. S. Grillner, “Neurobiological bases of rhythmic motor acts in vertebrates,” Science, vol. 228, no. 4696, pp. 143–149, 1985. View at Publisher · View at Google Scholar · View at Scopus
  16. K. Matsuoka, “Mechanisms of frequency and pattern control in the neural rhythm generators,” Biological Cybernetics, vol. 56, no. 5-6, pp. 345–353, 1987. View at Publisher · View at Google Scholar · View at Scopus
  17. A. J. Ijspeert, “Central pattern generators for locomotion control in animals and robots: a review,” Neural Networks, vol. 21, no. 4, pp. 642–653, 2008. View at Publisher · View at Google Scholar · View at Scopus
  18. M. Heisenberg, “Action selection,” in Invertebrate Learning and Memory, R. Menzel and P. Benjamin, Eds., vol. 22, Academic Press, 2013. View at Google Scholar
  19. P. Redgrave, T. J. Prescott, and K. Gurney, “The basal ganglia: a vertebrate solution to the selection problem?” Neuroscience, vol. 89, no. 4, pp. 1009–1023, 1999. View at Publisher · View at Google Scholar · View at Scopus
  20. P. Cisek, “Making decisions through a distributed consensus.,” Current opinion in neurobiology, vol. 22, no. 6, pp. 927–936, 2012. View at Publisher · View at Google Scholar · View at Scopus
  21. P. Cisek, G. A. Puskas, and S. El-Murr, “Decisions in changing conditions: the urgency-gating model,” The Journal of Neuroscience, vol. 29, no. 37, pp. 11560–11571, 2009. View at Publisher · View at Google Scholar · View at Scopus
  22. S. D. Brown and A. Heathcote, “The simplest complete model of choice response time: linear ballistic accumulation,” Cognitive Psychology, vol. 57, no. 3, pp. 153–178, 2008. View at Publisher · View at Google Scholar · View at Scopus
  23. M. Usher and J. L. McClelland, “The time course of perceptual choice: the leaky, competing accumulator model,” Psychological Review, vol. 108, no. 3, pp. 550–592, 2001. View at Publisher · View at Google Scholar · View at Scopus
  24. M. Kassim, N. Zainal, and M. Arshad, “Central pattern generator in bio-inspired simulation using matlab,” in Proceedings of the MEDINFO, vol. 98, 1998.
  25. D. Standage, D. Wang, and G. Blohm, “Neural dynamics implement a flexible decision bound with a fixed firing rate for choice: a model-based hypothesis,” Frontiers in Neuroscience, vol. 8, article 318, 2014. View at Publisher · View at Google Scholar
  26. B. Brembs, F. D. Lorenzetti, F. D. Reyes, D. A. Baxter, and J. H. Byrne, “Operant reward learning in Aplysia: neuronal correlates and mechanisms,” Science, vol. 296, no. 5573, pp. 1706–1709, 2002. View at Publisher · View at Google Scholar · View at Scopus
  27. K. M. Crisp and K. A. Mesce, “Beyond the central pattern generator: amine modulation of decision-making neural pathways descending from the brain of the medicinal leech,” Journal of Experimental Biology, vol. 209, no. 9, pp. 1746–1756, 2006. View at Publisher · View at Google Scholar · View at Scopus
  28. T. M. Wright Jr. and R. L. Calabrese, “Patterns of presynaptic activity and synaptic strength interact to produce motor output,” The Journal of Neuroscience, vol. 31, no. 48, pp. 17555–17571, 2011. View at Publisher · View at Google Scholar · View at Scopus
  29. A. Cyr, M. Boukadoum, and F. Thériault, “Operant conditioning: a minimal components requirement in artificial spiking neurons designed for bio-inspired robot’s controller,” Frontiers in Neurorobotics, vol. 8, article 21, 2014. View at Google Scholar
  30. N. Caporale and Y. Dan, “Spike timing-dependent plasticity: a Hebbian learning rule,” Annual Review of Neuroscience, vol. 31, pp. 25–46, 2008. View at Publisher · View at Google Scholar · View at Scopus
  31. A. Cyr and M. Boukadoum, “Classical conditioning in different temporal constraints: an STDP learning rule for robots controlled by spiking neural networks,” Adaptive Behavior, vol. 20, no. 4, pp. 257–272, 2012. View at Publisher · View at Google Scholar · View at Scopus
  32. H. Markram, W. Gerstner, and P. J. Sjöström, “A history of spike-timing-dependent plasticity,” Frontiers in Synaptic Neuroscience, vol. 3, article 4, 2011. View at Publisher · View at Google Scholar · View at Scopus
  33. A. I. Selverston, “Invertebrate central pattern generator circuits,” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 365, no. 1551, pp. 2329–2345, 2010. View at Publisher · View at Google Scholar · View at Scopus
  34. P. Cassey, A. Heathcote, S. D. Brown, and O. Sporns, “Brain and behavior in decision-making,” PLoS Computational Biology, vol. 10, no. 7, Article ID e1003700, 2014. View at Publisher · View at Google Scholar
  35. R. Bogacz, E.-J. Wagenmakers, B. U. Forstmann, and S. Nieuwenhuis, “The neural basis of the speed-accuracy tradeoff,” Trends in Neurosciences, vol. 33, no. 1, pp. 10–16, 2010. View at Publisher · View at Google Scholar · View at Scopus
  36. R. P. Heitz and J. D. Schall, “Neural mechanisms of speed-accuracy tradeoff,” Neuron, vol. 76, no. 3, pp. 616–628, 2012. View at Publisher · View at Google Scholar · View at Scopus