Transit signal priority control using reinforcement learning based on cell transmission model

Abstract

This dissertation is aimed at developing a new framework to control traffic signal light for the road network with recently introduced bus rapid transit (BRT) system. By applying the automated goal-directed learning and decision making called reinforcement learning (RL), the best possible traffic signal actions can be sought upon changes of network states as modelled by the signalised cell transmission model (CTM). There are three main original contributions in this dissertation. Firstly, the model combining CTM to capture the system dynamics together with the implementation of RL approach called Q learning has been introduced for an isolated intersection. Despite of such isolation constraint, a new external delay function has been proposed at the system boundary condition to capture the effects on the neighbourhood of that isolated intersection system. With the proper setting of red light delay as the RL reward function, reported results show that our proposed framework using RL and CTM in the macroscopic level can efficiently find the proper control solution that is close to the brute-forcely searched best periodic signal solution (BPSS). Secondly, the performance comparison of optimal traffic signal controls based on the derivation of theoretical M/M/1 and D/D/1 models and based on the RL approach has been evaluated. In particular, based on M/M/1 and D/D/1 queuing, the optimal split has been derived to minimise the mean waiting time of an intersection with two conflicting flows. The results confirm the validity in adopting the RL approach to control the traffic signal. Finally, an extension to a network of cascading interactions with BRT system has been proposed with simple uni-directional flows without turning movements. Motivated by the BRT system in Bangkok, the conventional signalised CTM has been generalised to cope with the preplanned space-usage priority of BRT over other non-priority vehicles by modelling explicitly the existence of BRT physical lane separator as well as the location of BRT stations. The delay function of both carried passengers on BRT and on other non-priority vehicles as well as waiting passengers at stations has been introduced. Based on the investigated scenarios, the deployment of BRT system with one lane deducted by the lane separator cannot reduce the total passenger delay in comparison with the comparable road and traffic condition before the BRT installation. However, with BRT, the passenger throughput can be greatly increased by up to 9-15\\% in the jamming conditions when at least 40\\% from the overall passengers choose the BRT for their journey. Moreover, our proposed method outperforms the conventional preemptive and differential priority control methods because of the improved awareness of signal switching cost. Based on all findings, the outstanding merit will entirely contribute towards to support the development of sustainable transportation systems.