Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015, Article ID 896943, 16 pages
http://dx.doi.org/10.1155/2015/896943
Research Article

Intelligent Ramp Control for Incident Response Using Dyna- Architecture

1School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China
2Institute for Transport Studies, University of Leeds, Leeds LS2 9JT, UK

Received 18 June 2015; Revised 22 September 2015; Accepted 28 September 2015

Academic Editor: Dongsuk Kum

Copyright © 2015 Chao Lu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. M. Papageorgiou and A. Kotsialos, “Freeway ramp metering: an overview,” IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 4, pp. 271–281, 2002. View at Publisher · View at Google Scholar · View at Scopus
  2. A. Skabardonis, P. Varaiya, and K. F. Petty, “Measuring recurrent and nonrecurrent traffic congestion,” Transportation Research Record, vol. 1856, pp. 118–124, 2003. View at Google Scholar · View at Scopus
  3. M. Papageorgiou, H. Hadj-Salem, and J.-M. Blosseville, “ALINEA: a local feedback control law for on-ramp metering,” Journal of the Transportation Research Board, vol. 1320, pp. 58–64, 1991. View at Google Scholar
  4. E. Smaragdis and M. Papageorgiou, “Series of new local ramp metering strategies,” Transportation Research Record, vol. 1856, pp. 74–86, 2003. View at Google Scholar · View at Scopus
  5. L. N. Jacobson, K. C. Henry, and O. Mehyar, “Real-time metering algorithm for centralized control,” Transportation Research Record, vol. 1232, pp. 17–26, 1989. View at Google Scholar
  6. G. Paesani, J. Kerr, P. Perovich, and F. Khosravi, “System wide adaptive ramp metering (SWARM),” in Proceedings of the 7th ITS America Annual Meeting and Exposition: Merging the Transportation and Communications Revolutions, Washington, DC, USA, June 1997.
  7. R. Lau, Ramp Metering by Zone—The Minnesota Algorithm, Minnesota Department of Transportation, 1997.
  8. H. M. Zhang and W. W. Recker, “On optimal freeway ramp control policies for congested traffic corridors,” Transportation Research Part B: Methodological, vol. 33, no. 6, pp. 417–436, 1999. View at Publisher · View at Google Scholar · View at Scopus
  9. A. Kotsialos, M. Papageorgiou, and F. Middelham, “Optimal coordinated ramp metering with advanced motorway optimal control,” Transportation Research Record, no. 1748, pp. 55–65, 2001. View at Google Scholar · View at Scopus
  10. A. Hegyi, B. De Schutter, and H. Hellendoorn, “Model predictive control for optimal coordination of ramp metering and variable speed limits,” Transportation Research C: Emerging Technologies, vol. 13, no. 3, pp. 185–209, 2005. View at Publisher · View at Google Scholar · View at Scopus
  11. G. Gomes and R. Horowitz, “Optimal freeway ramp metering using the asymmetric cell transmission model,” Transportation Research Part C: Emerging Technologies, vol. 14, no. 4, pp. 244–262, 2006. View at Publisher · View at Google Scholar · View at Scopus
  12. A. H. F. Chow and Y. Li, “Robust optimization of dynamic motorway traffic via ramp metering,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 3, pp. 1374–1380, 2014. View at Publisher · View at Google Scholar · View at Scopus
  13. R. W. Hall, “Non-recurrent congestion: how big is the problem? Are traveler information systems the solution?” Transportation Research Part C, vol. 1, no. 1, pp. 89–103, 1993. View at Publisher · View at Google Scholar · View at Scopus
  14. P. Prevedouros, B. Halkias, K. Papandreou, and P. Kopelias, “Freeway incidents in the United States, United Kingdom, and Attica Tollway, Greece: characteristics, available capacity, and models,” Transportation Research Record, vol. 2047, pp. 57–65, 2008. View at Publisher · View at Google Scholar · View at Scopus
  15. T. L. Greenlee and H. J. Payne, “Freeway ramp metering strategies for responding to incidents,” in Proceedings of the IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and a Special Symposium on Fuzzy Set Theory and Applications, pp. 987–992, New Orleans, LA, USA, December 1977. View at Publisher · View at Google Scholar
  16. M. H. Wang, Optimal ramp metering policies for nonrecurring congestion with uncertain incident duration [Ph.D. thesis], Purdue University, West Lafayette, Ind, USA, 1994.
  17. J.-B. Sheu, “Stochastic modeling of the dynamics of incident-induced lane traffic states for incident-responsive local ramp control,” Physica A: Statistical Mechanics and its Applications, vol. 386, no. 1, pp. 365–380, 2007. View at Publisher · View at Google Scholar · View at Scopus
  18. J.-B. Sheu and M.-S. Chang, “Stochastic optimal-control approach to automatic incident-responsive coordinated ramp control,” IEEE Transactions on Intelligent Transportation Systems, vol. 8, no. 2, pp. 359–367, 2007. View at Publisher · View at Google Scholar · View at Scopus
  19. C. Jacob and B. Abdulhai, “Machine learning for multi-jurisdictional optimal traffic corridor control,” Transportation Research Part A: Policy and Practice, vol. 44, no. 2, pp. 53–64, 2010. View at Publisher · View at Google Scholar · View at Scopus
  20. M. Davarynejad, A. Hegyi, J. Vrancken, and J. van den Berg, “Motorway ramp-metering control with queuing consideration using Q-learning,” in Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems (ITSC '11), pp. 1652–1658, IEEE, Washington, DC, USA, October 2011. View at Publisher · View at Google Scholar · View at Scopus
  21. K. Rezaee, B. Abdulhai, and H. Abdelgawad, “Application of reinforcement learning with continuous state space to ramp metering in real-world conditions,” in Proceedings of the 15th International IEEE Conference on Intelligent Transportation Systems (ITSC '12), pp. 1590–1595, IEEE, Anchorage, Alaska, USA, September 2012. View at Publisher · View at Google Scholar · View at Scopus
  22. C. Jacob and B. Abdulhai, “Automated adaptive traffic corridor control using reinforcement learning: approach and case studies,” Transportation Research Record, vol. 1959, pp. 1–8, 2006. View at Google Scholar · View at Scopus
  23. K. Rezaee, B. Abdulhai, and H. Abdelgawad, “Self-Learning adaptive ramp metering: analysis of design parameters on a test case in Toronto, Canada,” Transportation Research Record, vol. 2396, pp. 10–18, 2013. View at Publisher · View at Google Scholar · View at Scopus
  24. X.-J. Wang, X.-M. Xi, and G.-F. Gao, “Reinforcement learning ramp metering without complete information,” Journal of Control Science and Engineering, vol. 2012, Article ID 208456, 8 pages, 2012. View at Publisher · View at Google Scholar · View at Scopus
  25. A. Fares and W. Gomaa, “Multi-agent reinforcement learning control for ramp metering,” in Progress in Systems Engineering, vol. 330 of Advances in Intelligent Systems and Computing, pp. 167–173, Springer, Basel, Switzerland, 2015. View at Publisher · View at Google Scholar
  26. K. Veljanovska, K. M. Bombol, and T. Maher, “Reinforcement learning technique in multiple motorway access control strategy design,” PROMET-Traffic & Transportation, vol. 22, no. 2, pp. 117–123, 2010. View at Google Scholar · View at Scopus
  27. C. Lu, H. Chen, and S. Grant-Muller, “An indirect reinforcement learning approach for ramp control under incident-induced congestion,” in Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems (ITSC '13), pp. 979–984, IEEE, The Hague, The Netherlands, October 2013. View at Publisher · View at Google Scholar · View at Scopus
  28. L. Buşoniu, R. Babuška, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 38, no. 2, pp. 156–172, 2008. View at Publisher · View at Google Scholar · View at Scopus
  29. C. Lu, H. Chen, and S. Grant-Muller, “Indirect Reinforcement Learning for Incident-responsive ramp control,” Procedia—Social and Behavioral Sciences, vol. 111, pp. 1112–1122, 2014. View at Publisher · View at Google Scholar
  30. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998.
  31. C. C. H. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, no. 3-4, pp. 279–292, 1992. View at Publisher · View at Google Scholar · View at Scopus
  32. J. R. Kok and N. Vlassis, “Collaborative multiagent reinforcement learning by payoff propagation,” Journal of Machine Learning Research, vol. 7, pp. 1789–1828, 2006. View at Google Scholar · View at Scopus
  33. C. Guestrin, M. G. Lagoudakis, and R. Parr, “Coordinated reinforcement learning,” in Proceedings of the 19th International Conference on Machine Learning, pp. 227–234, Sydney, Australia, July 2002.
  34. S. El-Tantawy, B. Abdulhai, and H. Abdelgawad, “Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (marlin-atsc): methodology and large-scale application on downtown toronto,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 3, pp. 1140–1150, 2013. View at Publisher · View at Google Scholar · View at Scopus
  35. L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: a survey,” Journal of Artificial Intelligence Research, vol. 4, pp. 237–285, 1996. View at Google Scholar · View at Scopus
  36. C. F. Daganzo, “The cell transmission model: a dynamic representation of highway traffic consistent with the hydrodynamic theory,” Transportation Research Part B: Methodological, vol. 28, no. 4, pp. 269–287, 1994. View at Publisher · View at Google Scholar · View at Scopus
  37. J. Haddad, M. Ramezani, and N. Geroliminis, “Cooperative traffic control of a mixed network with two urban regions and a freeway,” Transportation Research Part B: Methodological, vol. 54, pp. 17–36, 2013. View at Publisher · View at Google Scholar · View at Scopus
  38. H. Mongeot and J.-B. Lesort, “Analytical expressions of incident-induced flow dynamics perturbations: using macroscopic theory and extension of Lighthill-Whitham theory,” Transportation Research Record, vol. 1710, pp. 58–68, 2000. View at Google Scholar · View at Scopus
  39. Transport Simulation Systems, Aimsun User's Manual 6.1, TTS, Barcelona, Spain, 2010.
  40. Highways England, “Hatris Homepage,” 2013, https://www.hatris.co.uk/.
  41. E. Cascetta, “Estimation of trip matrices from traffic counts and survey data: a generalized least squares estimator,” Transportation Research B, vol. 18, no. 4-5, pp. 289–299, 1984. View at Publisher · View at Google Scholar · View at Scopus
  42. L. Zhang and D. Levinson, “Balancing efficiency and equity of ramp meters,” Journal of Transportation Engineering, vol. 131, no. 6, pp. 477–481, 2005. View at Publisher · View at Google Scholar · View at Scopus
  43. A. Kotsialos and M. Papageorgiou, “Efficiency and equity properties of freeway network-wide ramp metering with AMOC,” Transportation Research Part C: Emerging Technologies, vol. 12, no. 6, pp. 401–420, 2004. View at Publisher · View at Google Scholar · View at Scopus