Autonomous vehicles are safety-critical systems, and they should be tested thoroughly before they are deployed on public roads. Although the testing in real traffic environments is not fully replaceable, simulation-based testing provides many advantages such as fully controllable environments, ground-truth information, ability to try a massive number of scenarios, and creating risky scenarios without risking human life or the vehicle under test.
Optimization-guided falsification techniques utilize optimization engines to generate challenging scenarios for an Autonomous Vehicle (AV) under test with the ultimate goal of finding a scenario in which the Vehicle Under Test (VUT), also called as Ego vehicle, fails to satisfy its requirements [9, 20, 19, 6]. In our previous work , we have used an optimization-guided falsification tool, , to search for maneuvers of agent actors (other road occupants) with the aim of identifying the boundary between the safe and unsafe operation of VUT.
In this paper, we propose an alternative approach to the falsification-based approach that is described in . We consider the test generation problem as a path planning problem for agent actors with the aim of creating interesting collisions with vehicles under test by utilizing Rapidly-exploring Random Trees (RRTs). Because of the nature of the driving environment, an automated vehicle cannot avoid all collisions. For instance, a collision with a vehicle that loses control and drives into the automated vehicle may not be avoidable. 1 illustrates an unavoidable collision example. However, identifying the boundaries between the collisions that are barely avoided or the collisions that could have been avoided with minor changes in the control or environment would be valuable for the engineering teams to improve the safety-related capabilities of the system. We focus on finding test cases that lead to behaviors at the proximity of the boundary between collisions and near-collisions. For this purpose, we propose a cost function that can guide the test generation toward that boundary.
RRTs have been first developed for robot path/motion planning problems [14, 11, 12]. However, thanks to their ability to efficiently search over high-dimensional spaces, RRT-based approaches also deliver promising results in the test generation domain [8, 13, 2, 4, 16, 7].
Since their first introduction, many variants of RRTs have been proposed. In , a method called Transition-based RRT (T-RRT) was introduced. Transition-based RRT method extends the classical RRT by incorporating additional cost criteria to the explored paths rather than only aiming to reach a target configuration. T-RRT borrows the notion of transitions tests from stochastic optimization approaches. Hence, it can be considered as a method that is merging RRTs with stochastic optimization. Furthermore, T-RRT controls exploration versus refinement using a method called minimal expansion control which helps to promote the expansion of a tree to the unexplored areas of the search space. Efficiency of T-RRT on continuous cost spaces is studied in . An optimal RRT approach, RRT*, was proposed, and the optimality was analyzed by Karaman and Frazzoli in .
2 Overview of the Approach
In our approach, we utilize notions from RRT*  and T-RRT  with a custom cost function that we propose to find boundary-case collisions. We implement our version of minimal expansion control using the notion of sparseness
from evolutionary algorithms that perform novelty search[17, 15].
With this approach, which we will refer to as “RRT-based approach”, we address the following limitations of our falsification-based approach :
In , we use a limited number of control points over the longitudinal position axis as the specific points where the lateral axis of the vehicle trajectories are sampled. As the number of control points decreases, possible variations in shapes of the trajectories are limited. On the other hand, increasing the number of control points also increases the dimension of the search space, which makes the problem more challenging.
In , the duration of the simulations is fixed. With the RRT-based approach, although there is an inherent limit on the maximum simulation duration that is dictated by the maximum number of RRT nodes, there is flexibility in the duration of the simulations. So, non-promising simulations are stopped earlier while more promising ones can be executed for longer times.
With the RRT-based approach, we can minimize the need for hand-designing a test scenario in detail and allow more freedom in the exploration of the space compared to the falsification-based approach.
RRT-based approach is promising to avoid local minima that can be challenging to the falsification-based approach proposed in .
The flowchart of our RRT-based approach is shown in 2. In the rest of this section, we will describe the key components of our approach.
2.1 Initializing the Search
A scenario, , is described by the sets of Ego vehicles , agent actors and environment as . The model of the driving scenario is given as where is the set of system states, is the set of system inputs, is the set of system parameters, and is the system simulation function, respectively, for the overall scenario. Given , is the state reached starting from the system state after time
under the input vectorand the parameter vector . Hence, the system simulation function is a mapping where is a discrete set of time samples , with .
In this approach, the set of inputs, , can simply be a set of target paths for the agent actors, as well as inputs to the other entities in the simulation environment such as models of road and weather conditions. After general outlines of the driving scenario are described, the sampling space for the initial states, , is used to sample initial configurations of the simulation entities, including Ego vehicles and agent actors.
2.2 Information Stored on RRT Tree Nodes
A tree grows while seeking to discover interesting behaviors. While growing the tree, instead of executing simulation traces starting from the initial configuration, only a partial simulation is executed starting from an existing node in the tree. For that purpose, the state of the system, the state of the controllers, and the simulation time are stored on the tree nodes. 1 provides more details on the data stored on the tree nodes after each partial simulation. The history-related fields will be empty for the root nodes of trees.
|State||The state of the system at the end of the corresponding part of the simulation is stored. It serves as the initial state for a new partial simulation that is starting from the configuration represented in the current node.|
|State History||(optional) The state history over the corresponding part of the simulation is used to (i) reproduce agent behaviors after the search is over (ii) to simulate any sensor delays for the next simulation step.|
|Input History||(optional) The history of inputs to the system is used as the past input data for the next simulation step, which may be useful for the systems that need to remember past inputs. This data is also valuable for analysis purposes when the search is over.|
|Controller State||The final state of controllers, if available, are stored so that the next step of the simulation can be started from the current node without needing to run the whole simulation from the starting node to the current node.|
|Simulation Time||The time at the end of the corresponding part of the simulation is stored.|
2.3 Sampling a Target Path Segment
A sample target path segment is simply a set of waypoints which is used as an immediate target for the corresponding vehicle. A waypoint is denoted as where are the -coordinate, -coordinate, target driving direction and the target speed at the waypoint. The sampling space for the waypoints is defined by a corresponding parameter space that describe the limits on the coordinates, driving direction, and target speed where . An example waypoint sampled on a straight road is shown in 3. A coordinate transformation can be applied for sampling from curved roads. Although the example waypoint in 3 is sampled from a road, the sample space of the waypoint doesn’t have to be the same as the area of a road in the simulation. It may be defined to go beyond the road limits, it may be limited to only a part of a road, or it may be completely irrelevant to a road in the simulation environment.
Once a waypoint is sampled, the next step in sampling a target path segment is to add an endpoint at a predefined distance from the waypoint, along the direction of the waypoint. 4 shows a target path segment formed using this approach. The sampled target path segment for this example can be denoted by .
If the endpoint of a target path segment is outside the sampling space of the waypoint, we simply break the segment at the boundary of the sampling space and add a second leg along the boundary in the direction closest to the waypoint direction. 5 shows an example target path section for a longer compared to the one in 4. The sampled target path segment for this example can be denoted by .
After the shape of the target segment is decided, we also sample a target speed, , for the target path segment.
This approach is applied to each agent vehicle for sampling their target path segments. Note that the framework we propose allows using a different algorithm for selecting a target path and/or any other type of input to the agent vehicles.
2.4 Selecting the Best Node from the Search Tree
Once target path segments for the agent vehicles are sampled, we pick one node from the existing tree, as the initial configuration for the simulation that will be executed with the sampled target path segments. There is no single correct approach to decide which node of the tree would be the best choice.
In our study, the notion of selecting the optimal node is adopted from the RRT* method . In , when adding a new node to the tree, existing nodes in a neighborhood of the new node are all checked and the one which minimizes the cost is selected as the previous node. Our approach has similarities to the RRT* one. We compute the sum of distances from each vehicle to the starting point of their target path segment with the constraint that the configurations of all vehicles are behind the start position of the initial target waypoint with respect to the driving direction of that waypoint. Then, we execute partial simulations from the best candidate previous nodes ( for our case studies) and pick the one which gives the minimum cost. We believe that this approach is promising to create relatively natural-looking vehicle trajectories while still allowing enough randomness in the maneuvers.
In , after adding the new node to the tree, there is a rewiring step which modifies the connections to the other existing nodes in the neighborhood of the new node. The rewiring step checks whether reaching to an existing node in that neighborhood from the newly added node would result in a reduction in the cost without violating any constraints such as obstacles. If so, it replaces the existing edge incoming to that node with an edge originating from the newly added node. Application of the rewiring step is straightforward for path planning problems on Euclidean spaces where the tree nodes represent planned waypoints on a path instead of the simulated vehicle configurations. In our approach, the tree nodes are the resulting configurations reached by the simulated vehicles with a controller and an input target path. Hence, the rewiring step requires the execution of partial simulations starting from the newly added node to the other nodes in the neighborhood of the newly added node. Since the resulting configuration will most likely be different at the end of such a partial simulation, the configurations on the target nodes will change. This creates the necessity to execute simulations from the updated nodes to all of the remaining tree nodes that can be reached from the modified node. Hence, the rewiring step can be computationally costly in our approach, and so, we do not apply the rewiring step and leave it as a future work for which the applicability should be analyzed.
We would like to emphasize that the function used for selecting the best previous node is user-configurable in our framework and depending on how much randomness is plausible in the generated driving paths, a different algorithm, simply selecting the closest node, can be utilized.
2.5 Simulating the System
After obtaining a set of target path segments for agent vehicles and deciding the initial configuration for the simulation, we create the simulation scene in the simulation environment using the data stored in the selected node of the search tree. That is, we set the initial states of the simulation entities and initialize the Ego vehicle controllers with the previous inputs and the saved controller states. We also pass the sampled target path segments to the agent vehicles as inputs. Finally, we simulate the system for time and collect state and input histories at each time step of the simulation. For our setup, we use simulations, however, this is not mandatory and another simulator can be used. Note that if the simulator and the Ego vehicle controllers allow saving the state and continuing simulation from a saved state, which is the case in our setup, the time spent in the simulations can be radically reduced because it would be enough to simulate only the new part of the simulation. Otherwise, the simulation should always start from the root node of the tree and run until the current target time.
2.6 Cost Function
After a simulation is executed, a cost function is used to compute how close the simulation trace is to an interesting behavior. The approach we describe here can be utilized to discover other types of interesting/failing behavior; however, our target in this work is to explore the behaviors that are on the boundary between safe and unsafe operation. Hence, an interesting behavior for our purposes would be (i) a collision between an Ego vehicle and an agent that could have been avoided with a minor change in the control applied or agent trajectories, (ii) an almost-collision (near-collision) which could have ended with a collision with a minor change in the control applied or agent trajectories.
The properties of a good cost function that would guide the search toward an interesting behavior for our purposes can be listed as follows:
Among two similar collisions between an Ego vehicle and an agent vehicle, the one which has the smaller magnitude in the relative speed between the vehicles should have a smaller cost, as a smaller change in the speed of the Ego vehicle would be enough to avoid the collision.
Among two similar collisions, the one which has the smaller impact area, the area of the collision surface, should have a smaller cost, as a smaller change in the steering maneuver of the Ego vehicle would be enough to avoid the collision.
For vehicle paths without a collision, a smaller time-to-collision at any point of the path, and a smaller corresponding collision speed and a smaller area for that collision-to-be should lead to a smaller cost.
We propose the following cost function:
where is the ratio of the collision surface to the overall surface on the collision side of the vehicle,
is the relative speed of the vehicles at the moment of collision, andis the mimimum time-to-collision encountered during the simulation output trace . For the simulations with a collision, is . For the simulations without a collision, and are computed at the instance of smallest time-to-collision with the assumption that the vehicles continue their motion without changing their speeds and orientations. When the simulation output trace contains collision(s) with Ego vehicle, we only consider the first collision of an Ego vehicle with any object for computing 1. 6 shows the function with respect to the minimum time-to-collision and collision speed variables for a fixed collision surface. The effect of the collision surface to the cost is linear.
2.7 Transition Check Function
The function we use for accepting a new configuration based on the cost is similar to the one proposed in . 1 repeats it for convenience. We denote the newly simulated configuration as the candidate and the initial configuration selected from the search tree as previous configuration. In the algorithm, is a constant parameter normalizing the change in the cost, is the temperature parameter that is governing the likelihood of acceptance. The temperature parameter is adaptively tuned by multiplying with or dividing by with respect to the ratio of rejections to acceptances.
2.8 Novelty Function
To get better coverage of the state space and to avoid local minima, we reward novelty in our search. For an Ego vehicle and an agent , we define as the vector of relative states and change in relative states at discrete simulation time , .
We compute the novelty of as follows:
where is the nearest neighbor of in the set which contains vectors for all ego-agent pairs for all times before . The function computes the Mahalanobis distance between and the elements of its -nearest neighbors set. We choose to use the Mahalanobis distance because of its ability to provide a dissimilarity measure between two observations by utilizing the sample covariance matrix .
As each new configuration has a corresponding partial simulation of length starting from a previous configuration, we compute the novelty for the trace of that partial simulation using 2.
2.9 Termination Condition
Our algorithm checks a set of termination conditions to stop the search and returns the configuration which has the minimum cost associated with it. One of the termination conditions we use is a threshold for the minimum interesting cost. Another termination condition is a preset maximum overall time spent. Alternative termination conditions can be used, a maximum number of nodes in the search tree.
3 Case Studies
Here, we present 2 case studies and compare our RRT-based approach with our falsification-based approach .
3.1 Case Study 1
3.1.1 Scenario Setup
In this case study, we have 2 agent vehicles and 1 Ego vehicle on a 3-lane straight road, and . 7 gives an high-level overview of our simulation setup. The initial position of agent vehicle 1 on the axis is randomly sampled between and , the initial position of agent vehicle 2 is randomly sampled between and , and the initial position of Ego vehicle is randomly sampled between and . The initial positions of agent vehicles on the axis are sampled between the centers of lane 1 and lane 3, between and , and the initial position of Ego vehicle is randomly sampled between and , that is the lane markings separating Lane 3 and Lane 1 from Lane 2, respectively. The initial orientation of the Ego vehicle with respect to the axis is randomly sampled between and . The initial speed of Ego vehicle is sampled between and while the target speed is fixed to . The initial speeds of the agent vehicles are sampled between and , and their target speeds at each waypoint are sampled between and .
Ego vehicle has 5 sensors. 8 visualizes the sensor placement and ranges of the sensors. A long-range sensor with a field of view and range is placed at the front of the vehicle. Two -range sensors with field-of-view are placed on the sides, facing left and right. Two -range sensors with field-of-view are placed at the rear-left and rear-right corners with an angle to scan the area behind the rear corners of the vehicle.
Agent vehicles are controlled by the move-to-pose controller described in 
. Ego vehicle controller has multiple modes based on the collision risk. When there is no estimated collision risk, a proportional control is applied to track target driving speed. If a collision is estimated in front, emergency brakes are applied and at the same time, depending on the occupancy of rear-left and rear-right areas, left or right steering is applied, respectively. If a collision is estimated in front-left (right), emergency brakes are applied and if rear-right (left) area is empty, also steering is applied to the right (left). If a collision is estimated in rear-left (right), if front and front-right (left) areas are empty, vehicle is accelerated with a right (left) steering, if only front area is empty and the vehicle on the front-right (left) is not imposing a risk, vehicle is accelerated without steering, otherwise emergency brakes are applied and right (left) steering is applied if the rear-right (left) area is empty. If the area to which a maneuver is being done gets occupied during the maneuver, emergency brakes are applied. Control switches back to normal mode if there is no more collision risk and a predefined time has passed since the last collision estimation. The collision avoidance algorithm presented here is very simple and it is not comparable to a controller that could be found in a real ADS. However, since our target in this work to study test generation approaches, rather than proposing a controller, we argue that this naive control approach is satisfactory for the purpose of this work. For the lateral control, we use the Stanford Racing Team’s controller that was used in the DARPA Grand challenge.
3.1.2 Experiment Results
We have run experiments with the falsification approach and with the RRT-based approach. The minimum, mean and maximum costs achieved by the falsification approach were , and , respectively. The minimum, mean and maximum costs achieved by the RRT-based approach were , and , respectively. 9 and 10 visualize the minimum-cost trajectories returned by the RRT-based and falsification-based approach, respectively. Histories of the vehicles are numbered to show their evolution over time. 11 provides box and whisker diagrams of the minimum costs achieved by the two approaches among the experiments we have carried. The black diamonds plotted on top of the box plots show the mean values for the returned minimum costs. In this case study, the falsification-based approach achieved smaller mean cost values, as well as the smaller minimum cost compared to the RRT-based approach. One reason for this is that, since the space between agent vehicles and Ego vehicle is open, there are not many local minimums that would make the exploration capabilities to achieve better than falsification-based approach. As it is easy to find a trajectory that is in the neighborhood of an interesting case, falsification approach can focus on that neighborhood and minimize the cost as much as possible while RRT-based approach keeps looking for novel trajectories.
3.2 Case Study 2
3.2.1 Scenario Setup
In this case study, we have 4 agent vehicles and 1 Ego vehicle on a multiple-lane straight road, and . 12 gives an high-level overview of our simulation setup. The initial position of agent on the axis is randomly sampled between and , the initial positions of agent vehicles are randomly sampled between and . The initial speed of is randomly sampled between and , and its target speed at each waypoint is sampled between and . The initial and target speeds of all other vehicles are set to . All other initial states of the vehicles are fixed.
Ego vehicle has 5 sensors. 13 visualizes the sensor placement and ranges of the sensors. A long-range sensor with a field of view and range is placed at the front of the vehicle. Two -range sensors with field-of-view are placed on the sides, facing left and right. Two -range sensors with field-of-view are placed at the rear-left and rear-right corners with an angle to scan the area behind the rear corners of the vehicle.
Agent vehicle is controlled by the move-to-pose controller described in . Agent vehicles , and are driven with a constant speed on a straight line. Ego vehicle controller is the same as the one described in Case Study 1.
3.2.2 Experiment Results
For this case study, we only search for an optimal trajectory for with the target of minimizing the cost function described in 1. The existence of agent vehicles , and between Ego vehicle and agent vehicle creates many local minima for the selection of trajectories.
In this case study, we have executed experiments with both the RRT-based approach and the falsification-based approach in . Each run of both approaches had a time-out duration of . Out of runs, only of the minimum-cost trajectories returned by the falsification-based approach was able to make to move into the lane of Ego vehicle, and only trajectories were able to cause a collision ( high-speed collision and boundary-case collision) and, other than the collision cases, only trajectory was able to challenge Ego vehicle by activating its collision avoidance system. All other trajectories returned by the falsification approach were stuck in local-minima where tries to get closer to Ego vehicle and ends up colliding with one of the other agent vehicles. On the other hand in of the minimum-cost trajectories returned by the RRT-based approach, agent was able to get into the lane of Ego vehicle and it was able to cause Ego vehicle to collide in of those cases.
14 shows one of the interesting collision cases discovered by the RRT-based approach. Agent first forces Ego to move to the right to avoid a collision and then to the left where it ends up colliding with Agent . Histories of (red) and Ego (yellow) vehicles are numbered to show their evolution over time. 15 shows the only small-speed collision case discovered by the falsification-based approach. Agent moves into the Ego vehicle’s lane, accelerates and rear-ends with Ego vehicle even though Ego vehicle tries to avoid the collision by accelerating and steering away. 16 shows a typical trajectory that is stuck in a local minimum. Agent tries to move closer to Ego vehicle and reduces the time-to-collision but collides with one of the other agents, which is in this figure. Although both approaches can get stuck in a local minimum, this case is significantly more common for the falsification-based approach as discussed above.
As a numerical comparison of the minimum costs discovered by the two approaches, the minimum, mean and maximum costs achieved by the falsification approach were , and , respectively. The minimum, mean and maximum costs achieved by the RRT-based approach were . 17 provides box and whisker diagrams of the minimum costs achieved by the two approaches among the experiments we have carried. The black diamonds plotted on top of the box plots show the mean values for the returned minimum costs. While out of RRT-based approach experiments were able to avoid local minima occurring around the cost , only of the falsification approach experiments were able to achieve this.
An observation we would like to share is that in the single case where the falsification approach was able to cause a small speed collision, the achieved minimum cost was significantly smaller than any of the collision cases discovered by the RRT-based approach. This observation supports our conclusion in Case Study 1, when the falsification-based approach can get into the neighborhood of a local or global minimum, it is more able to get closer to the minimum point than the RRT-based approach. The advantage of the RRT-based approach in avoiding local minima and the advantage of the falsification-based in reaching minima suggests that an approach combining these two has the potential to improve the overall test generation performance. In such an approach, firstly, RRT-based approach would discover neighborhoods of minima and then the falsification-based approach would guide the test toward the minima starting from the results of the RRT-based approach.
4 Conclusion and Future Work
We proposed an approach that explores maneuvers for road occupants using rapidly-exploring random trees with the target of minimizing safety/performance cost functions defined for the automated driving system under test. In our approach, we have adopted notions from transition-based RRT  and RRT*  methods. Our RRT-based approach delivered more promising results compared to the stochastic optimization-based falsification approach which is described in  for the problems which contain many local minima that should be avoided for reaching the global minimum.
When formulating the agent trajectory generation problem as an optimization problem, the trajectories are presented with a finite number of parameters, waypoint parameters for a fixed number of waypoints, which are provided by the test designer. One of the advantages of utilizing the RRT-based exploration is the ability to abandon the finite parameterization of the trajectories. This creates the opportunity to more freely explore the space, and to minimize the need for manually designing the general structure of the trajectory shapes. In an optimization-based approach, choosing a small number of parameters would limit the flexibility in generating critical trajectories, while choosing a large number would increase the dimensionality of the search space. For instance, in our Case Study 2, the number of parameters that the optimization method would need to create the minimum-cost trajectories that are discovered by the RRT-based approach varies from to based on the number of parameters at each waypoint and the number of waypoints that are used to create these tajectories.
Future work will include:
Exploring new methods for computing the novelty of a trajectory instead of a single state
Computing a cost on the shape of the trajectory instead of computing the minimum of the instantaneous costs of the points on the trajectory. This may especially be useful for rewarding some types of vehicle paths such as the ones which are closer to real-world driving behaviors.
Exploring new methods for using multi-objective optimization can be studied for different objectives like time-to-collision, collision speed, collision impact area etc.
Combining RRT-based approach with the falsification-based approach such that the exploration starts with RRT-based approach and falsification-based approach further minimizes the cost starting from the best cases discovered by the RRT-based approach.
-  Michael S Branicky, Michael M Curtiss, Joshua Levine & Stuart Morgan (2006): Sampling-based planning, control and verification of hybrid systems. IEE Proceedings-Control Theory and Applications 153(5), pp. 575–590.
-  Peter Corke (2017): Robotics, Vision and Control: Fundamental Algorithms In MATLAB® Second, Completely Revised. 118, Springer.
-  Thao Dang, Alexandre Donzé, Oded Maler & Noa Shalev (2008): Sensitive state-space exploration. In: Decision and Control, 2008. CDC 2008. 47th IEEE Conference on, IEEE, pp. 4049–4054.
-  Roy De Maesschalck, Delphine Jouan-Rimbaud & Désiré L Massart (2000): The mahalanobis distance. Chemometrics and intelligent laboratory systems 50(1), pp. 1–18.
A. Sangiovanni-Vincentelli &
S. A. Seshia
Systematic Testing of Convolutional Neural Networks for Autonomous Driving. In:
Reliable Machine Learning in the Wild (RMLW).
-  Tommaso Dreossi, Thao Dang, Alexandre Donzé, James Kapinski, Xiaoqing Jin & Jyotirmoy V. Deshmukh (2015): Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems. In: 7th International Symposium NASA Formal Methods (NFM), LNCS 9058, Springer, pp. 127–142.
-  Joel M Esposito, Jongwoo Kim & Vijay Kumar (2004): Adaptive RRTs for validating hybrid robotic control systems. In: Algorithmic Foundations of Robotics VI, Springer, pp. 107–121.
-  Georgios Fainekos, Sriram Sankaranarayanan, Koichi Ueda & Hakan Yazarel (2012): Verification of Automotive Control Applications using S-TaLiRo. In: Proceedings of the American Control Conference.
-  Gabriel M Hoffmann, Claire J Tomlin, Michael Montemerlo & Sebastian Thrun (2007): Autonomous automobile trajectory tracking for off-road driving: Controller design, experimental validation and racing. In: American Control Conference, pp. 2296–2301.
-  Léonard Jaillet, Juan Cortés & Thierry Siméon (2008): Transition-based RRT for path planning in continuous cost spaces. In: Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, IEEE, pp. 2145–2150.
-  Sertac Karaman & Emilio Frazzoli (2011): Sampling-based algorithms for optimal motion planning. The international journal of robotics research 30(7), pp. 846–894.
-  Jongwoo Kim, Joel M Esposito & Vijay Kumar (2005): An RRT-based algorithm for testing and validating multi-robot controllers. Technical Report, Moore Schoolf Of Electrical Engineering Philadelphi PA Grasp Lab.
-  Steven M LaValle (1998): Rapidly-exploring random trees: A new tool for path planning. Technical Report.
-  Joel Lehman & Kenneth O Stanley (2008): Exploiting open-endedness to solve problems through the search for novelty. In: ALIFE, pp. 329–336.
-  Erion Plaku, Lydia E Kavraki & Moshe Y Vardi (2009): Hybrid systems: from verification to falsification by combining motion planning and discrete search. Formal Methods in System Design 34(2), pp. 157–182.
-  Kenneth O Stanley & Risto Miikkulainen (2002): Evolving neural networks through augmenting topologies. Evolutionary computation 10(2), pp. 99–127.
-  Cumhur Erkan Tuncali (2019): Search-based Test Generation for Automated Driving Systems: From Perception to Control Logic. Ph.D. thesis, Arizona State University.
-  Cumhur Erkan Tuncali, Georgios Fainekos, Hisahiro Ito & James Kapinski (2018): Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components. In: IEEE Intelligent Vehicles Symposium (IV).
-  Cumhur Erkan Tuncali, Theodore P. Pavlic & Georgios Fainekos (2016): Utilizing S-TaLiRo as an Automatic Test Generation Framework for Autonomous Vehicles. In: IEEE Intelligent Transportation Systems Conference.