Resilience Analysis of Urban Road Networks Based on Adaptive Signal Controls: Day-to-Day Traffic Dynamics with Deep Reinforcement Learning

Table 2

Procedures for the proposed doubly dynamic learning framework.

Step 1	On day , start with the equilibrium state of the URN under DTD dynamics, and add different levels of capacity reduction on links
Step 2	Based on the perceived route cost and actual route cost on day , update the perceived route travel cost on day by performing DTD dynamic learning process (1)
Step 3	Based on route choice probability formula (2) and route assignment equation (3), determine the flow on all routes on day
Step 4	Obtain link flow on day by using formula (4); based on the current state (link flow and capacity), utilize the trained DQN to output the action (red time split); following this, integrate the action into the DTD model by using equation (10), and then achieve the actual route cost on day by performing network loading (9)
Step 5	If convergence condition (22) is satisfied, stop; otherwise, return to Step 2