Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015, Article ID 896943, 16 pages
Research Article

Intelligent Ramp Control for Incident Response Using Dyna- Architecture

1School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China
2Institute for Transport Studies, University of Leeds, Leeds LS2 9JT, UK

Received 18 June 2015; Revised 22 September 2015; Accepted 28 September 2015

Academic Editor: Dongsuk Kum

Copyright © 2015 Chao Lu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


Reinforcement learning (RL) has shown great potential for motorway ramp control, especially under the congestion caused by incidents. However, existing applications limited to single-agent tasks and based on -learning have inherent drawbacks for dealing with coordinated ramp control problems. For solving these problems, a Dyna- based multiagent reinforcement learning (MARL) system named Dyna-MARL has been developed in this paper. Dyna- is an extension of -learning, which combines model-free and model-based methods to obtain benefits from both sides. The performance of Dyna-MARL is tested in a simulated motorway segment in the UK with the real traffic data collected from AM peak hours. The test results compared with Isolated RL and noncontrolled situations show that Dyna-MARL can achieve a superior performance on improving the traffic operation with respect to increasing total throughput, reducing total travel time and CO2 emission. Moreover, with a suitable coordination strategy, Dyna-MARL can maintain a highly equitable motorway system by balancing the travel time of road users from different on-ramps.