Machine Learning for Distributed Traffic Management

Traditional systems for traffic management attempt to influence vehicle flow by controlling traffic signals or ramp meters. However, these approaches ignore an important component of traffic management: coordination of the vehicles themselves.

This project explores a novel approach to traffic management that relies on a distributed scheme in which vehicles themselves select lanes, speeds, and routes. A simulated highway network populated by thousands of vehicles provides the research testbed. A reinforcement learning system acquires control strategies based on experience with this simulator, and a separate module learns to predict the expected loads along alternative routes. The reward function involves not individual vehicles but overall traffic behavior, as measured by the average squared difference between vehicles' actual and desired speeds.

Experimental studies have demonstrated that the strategies learned by this system let drivers more closely match their desired speeds than do handcrafted strategies, while also reducing the number of lane changes. They also suggest that the learned behaviors generalize to different traffic densities, different numbers of lanes, situations involving blocked lanes, and even the presence of selfish drivers.

This research was funded by DaimlerChrysler Research and Technology.

Contributors to the Project

  • Dr. Simon Handley

  • Professor Pat Langley

  • Dr. David Moriarty
  • Dr. Mark Pendrith

    Related Papers

    Moriarty, D., & Langley, P. (1998). Learning cooperative lane selection strategies for highways. Proceedings of the Fifeenth National Conference on Artificial Intelligence (pp. 684-691). Madison, WI: AAAI Press.

    Moriarty, D. E., Handley, S., & Langley, P. (1998). Learning distributed strategies for traffic control. Proceedings of the Fifth International Conference of the Society for Adaptive Behavior (pp. 437-446). Zurich, Switzerland.

    Moriarty, D., & Langley, P. (1998). Distributed learning of lane-selection strategies for traffic management (Technical Report 98-2). Daimler-Benz Research & Technology Center, Palo Alto, CA.

    For more information, please send email to .