Modeling of traffic flow using cellular automata and traffic signal control by Q-learning

Recently, the flow of traffic has increased in the cities, and it has caused problems because of CO₂ emissions due to traffic jams. The traffic signal control is a typical counter measures for the congestion easing. The traffic signal control method includes the point control, the series control, and the wide area control, and the cycle time, the split, and the offset are used as the control parameters of the traffic signal. The offset is the difference of the start for the green signal between adjoining crossroads. The existing researches to generate the offset automatically are the cycle-less control technique, the real-time simulation using GA, and the optimization technique by the inclination method. First, the traffic flow is modeled to reproduce the movement of the vehicle on the road in this paper. There are two models of the traffic flow being developed now: one is to model the traffic flow as a continuous style, and the other is to regard the vehicle as the individual movement and to form the whole flow. The traffic flow is modeled using the cellular automata as the latter case here. A traffic signal is consisted as an agent and the agent learns the control parameters of the traffic signal, which are the split and the offset under the fixed cycle length, using Q-learning method. In this paper, the offset of the signal agent is deduced using Q-learning method considering the adaptation for the dynamic change of the traffic flow.