Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2013 (2013), Article ID 926267, 8 pages
Research Article

Multiagent Reinforcement Learning with Regret Matching for Robot Soccer

Qiang Liu,1,2 Jiachen Ma,1,2 and Wei Xie1,2

1School of Astronautics, Harbin Institute of Technology, Harbin 150001, China
2School of Information and Electrical Engineering, Harbin Institute of Technology (Weihai), Weihai 264209, China

Received 4 April 2013; Revised 19 July 2013; Accepted 20 July 2013

Academic Editor: Yudong Zhang

Copyright © 2013 Qiang Liu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


This paper proposes a novel multiagent reinforcement learning (MARL) algorithm Nash- learning with regret matching, in which regret matching is used to speed up the well-known MARL algorithm Nash- learning. It is critical that choosing a suitable strategy for action selection to harmonize the relation between exploration and exploitation to enhance the ability of online learning for Nash- learning. In Markov Game the joint action of agents adopting regret matching algorithm can converge to a group of points of no-regret that can be viewed as coarse correlated equilibrium which includes Nash equilibrium in essence. It is can be inferred that regret matching can guide exploration of the state-action space so that the rate of convergence of Nash- learning algorithm can be increased. Simulation results on robot soccer validate that compared to original Nash- learning algorithm, the use of regret matching during the learning phase of Nash- learning has excellent ability of online learning and results in significant performance in terms of scores, average reward and policy convergence.