Appropriate traffic regulations for a crowd of people, e.g. planned road closure, are important in congested event spaces and urban areas as regards improving safety and convenience. Without regulations, overcrowded traffic might result in fatal accidents caused by people falling over each other as well as a reduction in passenger satisfaction due to a prolonged latency time. Crowd simulation based on multi-agent systems  has been successfully used for finding appropriate regulations [15, 1, 19]. A crowd simulation under a regulation scenario enables us to calculate the effect of the regulation, such as the arrival time at the destination and the degree of congestion. By calculating the effect of various regulation scenarios, we can find an appropriate regulation. However, this approach requires to conduct many simulations under different regulation scenarios, which prevents us from finding appropriate regulations to avoid accidents in the current situation.
In this paper, we propose a method to learn a function that outputs appropriate regulations given the current traffic situation as inputs. If the function is learned using the training data of many simulation runs in advance, we can obtain an appropriate regulation by bypassing simulations for the current situation. For example, we would like to select a road to close to avoid congestion. Closing a road can reduce congestion by directing the traffic along different roads 
. This phenomenon is known as the Braess’s paradox. Also, closing a road in one direction can allow for a better flow in the opposite direction. In this case, the input is the number of people on each road, which is represented by a vector, and the output is the road to be closed. We can use standard classifiers, such as logistic regression, support vector machines and feed-forward neural networks, to learn the function if we fix the number of roads, or fix the area that we wish to regulate. However, the learned function cannot be used for other areas since the input and output size changes depending on the area and these classifiers require fixed-size inputs and outputs. Therefore, these standard classifiers are inapplicable to unseen areas. In addition, learning functions for various areas in advance requires huge computational time and memory.
To find regulations for various areas with a single function, we use graph convolutional networks (GCNs) . The GCN takes variable size feature vectors and graphs as inputs, and outputs values for each node. With the proposed method, we construct a graph for each area, where a node represents a road, and an edge represents the road connection. Then, the GCN can employ traffic information for any areas with different numbers of roads in the form of a graph as inputs. In addition, the GCN can output the road to conduct a regulation for any areas. By using the GCN with multiple layers, we can use the information on roads with the multi-hop distance, which is transformed so as to be beneficial for finding regulations.
Ii Proposed method
The proposed method is composed of the following three steps:
Generating training data by simulations,
Learning a function using the generated data,
Finding regulations by the learned function.
The proposed method is applicable to pedestrians, cars and bikes if we use simulators for them. In the following discussion, we assume for simplicity that the regulation procedure is to select a road to close. However, the proposed method is applicable to other types of regulations if they can be represented by a graph as discussed in Section II-D.
Ii-a Data generation
|(a) Junction graph||(b) Road graph||(c) Calculating traffic situations||(d) Calculating regulation effect|
First, the proposed method generates training data to learn a function that outputs a way to regulate given road graph and traffic situation. The training data consist of sets of road graphs, traffic situations and regulation effects. The data generation procedure is organized in the following four steps.
Obtaining junction graphs
The junction graph is a directed graph, where each node represents a junction or a dead end, and each edge represents a road. The junction graphs are used for simulations as well as for constructing road graphs in the next step. Figure 1 (a) shows an example junction graph.
Constructing road graphs
A road graph is a directed graph, where each node represents a road, and each edge represents a connection between roads. A road graph is constructed from a junction graph. When the end of a road and the start of another road share a junction, the corresponding road nodes are connected by a directed edge. If two roads are the same road running in opposite directions, the nodes are not connected, which indicates that roads are not connected by a U-turn. Figure 1(b) shows an example of a road graph constructed from the junction graph in Figure 1(a). The road graphs are used for the inputs of the function. Suppose we construct road graphs, , from junction graphs, where is the th road graph with nodes.
Calculating traffic situations
The traffic situation represents the circumstances of each road, such as the population and the average moving speed. We generate a traffic situation for each road graph using a crowd simulator. The crowd simulator calculates the number of pedestrian for each road over time given a junction graph, origin, destination and start time for each pedestrian. These inputs for the simulator can be set randomly. For example, when we use the population on each road as the traffic situation, it is represented by an -dimensional vector, , where is the number of pedestrians on road in graph . Figure 1(c) shows the simulation on the junction graph.
Calculating regulation effects
The regulation effects for each traffic situation are calculated with a simulator. We save the simulation result when we obtain the traffic situation in the previous step. Next, we conduct a regulation operation, i.e. closing a road in the road graph, and run simulation from the saved point. Then, we calculate the regulation effect, such as the arrival time at the destination, the degree of congestion, the population that arrived at their destination, and the number of congested roads. Figure 1(d) shows the simulation result when road ‘D’ is closed. The regulation effects on graph is represented by a vector , where is the regulation effect value when road is closed. We assume that a higher indicates better regulation.
Ii-B Function learning
Given sets of triplets of road graph, traffic situation and regulation effect, , we learn a function that outputs the regulation effects given road graph and traffic situation . We use the graph convolutional networks (GCNs) for the function.
With the GCN, each node has -layer hidden states. Let be the th-layer hidden state of the node of graph . The th layer corresponds to the input traffic situation itself, . The hidden state of the next layer is calculated using the hidden states of its connected nodes and the node itself at the current layer as follows,
where is a set of nodes connected to node , is a set of nodes connected from node , which are defined by the road graph , , and are the
th layer linear transformation matrices,is the hidden state size at the th layer, and
. The parameters to be estimated are, , . The last (th) layer hidden state corresponds to the outputs, , where represents the GCN. Figure 2 shows the calculation of the hidden state of the next layer. By using different transformations for incoming and outcoming nodes with and , we can handle directed flow information on the graph, which is important for traffic modeling . A graph convolutional layer (1) can collect information from nodes that are directly connected. By using layers, information on nodes within the -hop distance can be used for the output, which is transformed so as to be beneficial for selecting a road to conduct the regulation.
By minimizing this loss function, the regulations that have relatively high effects are likely to be high output values, even if the regulation is not the best, which leads to robust learning of the function.
Ii-C Regulation output
Given road graph and traffic situation , a regulation is selected by using the learned function as follows, , which indicates road is the road to close. Here, is the estimated parameters of the GCN. In some applications, we would like to rank the regulations according to their effects. The ranking is obtained by . If we have a simulator for the target situation, by running simulations under a regulation in the ranked order, we would find the best regulation with the smaller number of simulation runs.
We considered closing a road for regulation. However, the proposed method is applicable to other types of regulations if they can be represented by a graph. We can conduct multiple regulation operations, e.g. closing several roads, by applying the proposed method recursively; 1) finding a road to close, 2) modifying the road graph by closing the road, and 3) finding another road to close. If a simulator is available, the traffic situation can also be updated by using the simulator when applying the selected regulation.
We considered the population on each road in terms of traffic situation. Other features on each road, such as populations over time, average moving speed and road width, can be used by including them in the input vector of the GCN.
We evaluated the proposed method using real-world road graphs in Japan for finding a road to close to reduce the time taken to arrive at the destination. We collected junction graphs around train stations with an area of 1,450 meters square from the website of the Geographical Information Authority of Japan 111https://maps.gsi.go.jp/development/ichiran.html. Then, we constructed the road graphs from the junction graphs, where the average number of nodes (roads) was 1,227, and the average number of edges (road connections) was 2,075.
We ran a 30-minute simulation with 100,000 pedestrians to obtain the traffic situation for each graph. With each pedestrian, the train station was set as either the origin or the destination, and these destinations or origins were randomly selected from junctions. Then, we closed a road and ran an additional 30-minute simulation to calculate the regulation effect. For the regulation effect, we used the average reduction in the time taken to arrive at the destination. When there was no route to the destination as the result of closing a road, we set the regulation effect as the worst value in that situation. With 19% of the roads the arrival time was sooner with closure than that without the regulation. With 48% of the roads the arrival time was later, and there was no effect for 33% of the roads. The average arrival time was 704.9 seconds, and the average maximum time reduction was 76.3 seconds. The number of road graphs for training, validation and test were 1,500, 100, and 100, respectively. Note that road graphs in the test data were not included in the training and validation data, and we needed to find a regulation for unseen areas.
To obtain traffic situations and regulation effects, we used an inhouse crowd simulator based on a multi-agent system. With the simulator, each agent moves to the destination along the shortest path given the origin, destination, start time and road graph. The maximum speed
is drawn from a log normal distribution with a median of 1.16 meter per second and a standard deviation of 0.12 for each agent. Each agent moves at the maximum speed if there is no congestion, and the speed decreases with the degree of congestion. Specifically, the speedis determined by
where is the population density in front of the agent.
Iii-C Evaluation measurements
We evaluated the proposed method using the following three evaluation measurements: time reduction, top-10 accuracy, and maximum time reduction through simulations. The time reduction is the average reduction in the time taken to arrive at the destination compared with the arrival time without regulation. The time reduction is positive when the arrival time is reduced by the regulation, and it is zero when the arrival time is the same with that without regulation. The top-10 accuracy is a rate when the best regulation is included in the top-10 list. The best regulation is that with the maximum time reduction of all regulations. With the maximum time reduction through simulations, we assume that we can use simulations to check suggested regulations. By conducting simulations with different regulations in a ranked order, we can find the best regulation according to the maximum time reduction. Since we would like to find a better regulation in a shorter time, a larger time reduction with fewer simulations is preferred.
Iii-D Comparing methods
We compared the proposed method with the Population, Closeness, Betweenness, Bayesian optimization, and Random methods. When ranking the roads to close, the Population method uses the road information, the Closeness and Betweenness methods use the road graph structures, and the Bayesian optimization method uses simulation results. The Random method randomly ranks the roads.
The Population method ranks roads to close in descending order of population of each road. It is intended to disperse people in a congested road.
The Closeness method ranks roads to close in descending order of closeness centrality . A road that can access other roads with short path lengths has high closeness centrality. The closeness centrality is calculated as the inverse of the sum of the shortest path length between the node (road) and all the other nodes in the graph as follows, , where is the shortest path length between nodes and .
The Betweenness method ranks roads to close in descending order of betweenness centrality . A road that is often used by traffic between all pairs of nodes has high betweenness centrality. The betweenness centrality is calculated as the sum of the fraction of all-pair shortest paths that pass through the node as follows, , where is the number of shortest paths between nodes and , and is the number of those paths passing through node other than and .
The Bayesian optimization (BO) method optimizes a black-box function with as few function evaluations as possible . Since BO is applicable when function evaluations, in this case simulation runs, are possible, we included BO as a comparative method for the maximum time reduction through simulations, but not for the time reduction and top-10 accuracy. With BO, a Gaussian process is used to approximate the time reduction with the following two kernels: the RBF kernel on the population and the diffusion kernel on the road graph. The RBF kernel is defined by , where , and
are the hyperparameters,is the Kronecker delta, if and otherwise, and is the population on road . The diffusion kernel is defined by , where is the hyperparameter, and is the graph Laplacian of the road graph. We selected the hyperparameters by employing a grid search.
The proposed method used the GCN with layers of
hidden states on each layer. The activation function was ReLU, and it was followed by batch normalization. The best model was selected by using validation data based on the time reduction. The GCN was implemented by using a deep learning framework, Chainer.
The time reduction and top-10 accuracy shown in Table I were averaged over ten experiments with different training, validation and test data sets. The proposed method successfully reduced the average arrival time, and achieved the maximum time reduction. On the other hand, the time reduction with the other methods were negative; they increased the average arrival time. The top-10 accuracy with the proposed method was higher than that with the other methods. These results indicates that the proposed method can determine a regulation operation by learning a function using GCNs.
|Time reduction||Top-10 accuracy|
Figure 3 shows the maximum time reduction through simulations. The proposed method achieved the highest maximum time reduction over every number of simulation runs. With the proposed method, the maximum time reduction quickly increased in a small number of simulation runs. This result indicates that we can obtain better regulations by running simulations in the ranked order by the proposed method. The Population method gave the lowest maximum time reduction without simulations, but the time reduction increased by running simulations. The maximum time reduction with the Bayesian optimization methods, BO(diffusion) and BO(RBF), was small. This is because the time reduction over the road graph was difficult to approximate by Gaussian processes.
|(a) Population of each road||(b) True regulation effects||(c) Estimated regulation effects|
|(a) Without regulation||(b) With regulation by the proposed method|
Figure 4 shows the population of each road, true regulation effects and estimated regulation effects obtained with the proposed method for two example areas. Truly high regulation effect roads were successfully included in the roads with high estimated regulation effects estimated by the proposed method.
Figure 5 shows the population for a one-hour simulation time without regulation (a), and the population for a one-hour simulation time with regulation with a 30-minute simulation time (b), where the proposed method selected road to close as the regulation. The population on road was large. Without regulation, people who go to from also use the road, and the road was heavily congested. By closing road , people who went to used a different route , Then, the congestion on road was alleviated, and the average arrival time was reduced.
Iv Related work
, which is a method for optimizing of black-box functions, can reduce the number of simulation runs, data assimilation and multiple simulation runs are still needed. A number of methods have been proposed for adaptive crowd regulations, especially signal control, using reinforcement learning[5, 20, 10, 9, 8]. However, these methods consider a specific area, and learned models cannot be used for other areas. The proposed method can find regulations on any areas by using road graph information without data assimilation or simulation runs. Regulation methods for individuals, i.e. cars and pedestrians, have been extensively studied [21, 13, 14, 2]. These methods find a route with the minimum travel time for each individual given the origin and destination. The proposed method finds a regulation for a crowd, and does not use origin and destination information. Graph convolutional networks have been successfully used for a wide variety of applications, such as chemicals 11] and traffic forecasting [23, 12], but not for finding traffic regulations.
In this paper, we proposed a method for finding appropriate regulations. With the proposed method, a function that outputs regulation effects given traffic situations and road graphs is modeled with graph convolutional networks. With experiments using real-world road graphs and a crowd simulator, we confirmed that the proposed method can find appropriate regulations efficiently. Although our results have been encouraging, our framework can be further improved upon in a number of ways. Firstly, we plan to use time information by incorporating recurrent neural networks. Secondly, we would like to extend our framework to adaptive regulation by using reinforcement learning.
-  J. E. Almeida, R. J. Rosseti, and A. L. Coelho. Crowd simulation modeling applied to emergency and evacuation simulations using multi-agent systems. arXiv preprint arXiv:1303.4692, 2013.
-  M. Arikawa, S. Konomi, and K. Ohnishi. Navitime: Supporting pedestrian navigation in the real world. IEEE Pervasive Computing, 6(3), 2007.
-  A. Bavelas. Communication patterns in task-oriented groups. The Journal of the Acoustical Society of America, 22(6):725–730, 1950.
-  A. Braun, S. R. Musse, L. P. L. de Oliveira, and B. E. Bodmann. Modeling individual behaviors in crowd simulation. In Computer Animation and Social Agents, 2003. 16th International Conference on, pages 143–148. IEEE, 2003.
-  S. P. Choi and D.-Y. Yeung. Predictive Q-routing: A memory-based reinforcement learning approach to adaptive traffic control. In Advances in Neural Information Processing Systems, pages 945–951, 1996.
-  D. K. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in Neural Information Processing Systems, pages 2224–2232, 2015.
-  L. C. Freeman. A set of measures of centrality based on betweenness. Sociometry, pages 35–41, 1977.
-  J. Gao, Y. Shen, J. Liu, M. Ito, and N. Shiratori. Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network. arXiv preprint arXiv:1705.02755, 2017.
-  W. Genders and S. Razavi. Using a deep reinforcement learning agent for traffic signal control. arXiv preprint arXiv:1611.01142, 2016.
M. A. Khamis and W. Gomaa.
Adaptive multi-objective reinforcement learning with hybrid
exploration for traffic signal control based on cooperative multi-agent
Engineering Applications of Artificial Intelligence, 29:134–151, 2014.
-  T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
-  Y. Li, R. Yu, C. Shahabi, and Y. Liu. Diffusion graph convolutional recurrent neural network: Data-driven traffic forecasting. CoRR, abs/1707.01926, 2017.
-  J. Lin, W. Yu, X. Yang, Q. Yang, X. Fu, and W. Zhao. A real-time en-route route guidance decision scheme for transportation-based cyberphysical systems. IEEE Transactions on Vehicular Technology, 66(3):2551–2566, 2017.
-  A. J. May, T. Ross, S. H. Bayer, and M. J. Tarkiainen. Pedestrian navigation aids: information requirements and design implications. Personal and Ubiquitous Computing, 7(6):331–338, 2003.
-  Y. Murakami, K. Minami, T. Kawasoe, and T. Ishida. Multi-agent simulation for crisis management. In Knowledge Media Networking, 2002. Proceedings. IEEE Workshop on, pages 135–139. IEEE, 2002.
M. Pelikan, D. E. Goldberg, and E. Cantú-Paz.
BOA: The Bayesian optimization algorithm.
Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation-Volume 1, pages 525–532. Morgan Kaufmann Publishers Inc., 1999.
-  M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling. Modeling relational data with graph convolutional networks. arXiv preprint arXiv:1703.06103, 2017.
S. Tokui, K. Oono, S. Hido, and J. Clayton.
Chainer: a next-generation open source framework for deep learning.
Proceedings of Workshop on Machine Learning Systems in the Annual Conference on Neural Information Processing Systems, volume 5, 2015.
-  N. Ueda, F. Naya, H. Shimizu, T. Iwata, M. Okawa, and H. Sawada. Real-time and proactive navigation via spatio-temporal prediction. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, pages 1559–1566, 2015.
-  M. Wiering. Multi-agent reinforcement learning for traffic light control. In Machine Learning: Proceedings of the Seventeenth International Conference (ICML’2000), pages 1151–1158, 2000.
-  T. Yamashita, K. Izumi, K. Kurumatani, and H. Nakashima. Smooth traffic flow with a cooperative car navigation system. In Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, pages 478–485. ACM, 2005.
-  H. Youn, M. T. Gastner, and H. Jeong. Price of anarchy in transportation networks: efficiency and optimality control. Physical review letters, 101(12):128701, 2008.
-  B. Yu, H. Yin, and Z. Zhu. Spatio-temporal graph convolutional neural network: A deep learning framework for traffic forecasting. CoRR, abs/1709.04875, 2017.