Table of Contents Author Guidelines Submit a Manuscript
Computational Intelligence and Neuroscience
Volume 2015, Article ID 481375, 17 pages
Research Article

Kernel Temporal Differences for Neural Decoding

1Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA
2Department of Biomedical Engineering, University of Miami, Coral Gables, FL 33146, USA
3Department of Physiology and Pharmacology, Robert F. Furchgott Center for Neural & Behavioral Science, SUNY Downstate Medical Center, Brooklyn, NY 11203, USA

Received 8 September 2014; Revised 28 January 2015; Accepted 3 February 2015

Academic Editor: Daoqiang Zhang

Copyright © 2015 Jihye Bae et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


We study the feasibility and capability of the kernel temporal difference (KTD)() algorithm for neural decoding. KTD() is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm’s convergence can be guaranteed for policy evaluation. The algorithm’s nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey’s neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm’s capabilities in reinforcement learning brain machine interfaces.