Given a source location and a target location in a graph, a shortest path query (e.g.,see [ouyang2020efficient, abraham2011hub]) returns the path from to with the minimum total weight (e.g., travel time, distance). The shortest path query is one of the most fundamental queries in graphs and has applications in a wide variety of domains such as in road networks where a user may issue the shortest path query to find a route to travel from one location to the other. Since the shortest path may not always match a user’s traveling choices, modern map-based systems often provide several alternative routes so that the user can choose a path that they find most suitable. It is critical for the effectiveness of the alternative routes that these routes are significantly different from each other and are meaningful (e.g., without unnecessary detours).
Inspired by the importance of alternative routes, in the past several years, a large body of research has focused on computing alternative routes [jones2012method, abraham2013alternative, chen2007reliable, akgun2000finding, chondrogiannis2020finding, chondrogiannis2018finding, liu2017finding]. Intuitively, the alternative routes reported to the users must be meaningful/natural and significantly different from each other. However, there is no agreed definition of what constitutes a set of “good” alternative routes. This is because the “goodness” of the alternative routes is mostly subjective and it is not trivial to define quantitative measures to evaluate the quality of routes. This necessitates a user study to compare the perceived quality of the alternative routes generated by the existing techniques. Surprisingly, there does not exist any such systematic study that is concerns how users perceive the quality of the routes provided by different techniques.
To fill this gap, in this paper, we present the first user study that compares four popular techniques including Google Maps which is among the most widely used commercial solutions providing alternative routes. Specifically, we create a web-based demo system that asks users to select source and target locations within the Melbourne Metropolitan area. It then displays up to routes generated by the following four techniques: Google Maps, Plateaus [jones2012method, abraham2013alternative], Penalty [chen2007reliable, akgun2000finding] and Dissimilaritys [chondrogiannis2020finding, chondrogiannis2018finding, liu2017finding]. The users are asked to provide a rating from 1-5 (higher the better) for each of the four approaches.
In total, we received responses ( from Melbourne residents and
from non-residents). We show the mean rating and standard deviation for each of the four approaches for different groups of respondents. Also, we show the mean rating based on the lengths of the routes (small routes, medium routes and long routes). The overall mean ratings for Google Maps, Plateau, Dissimilarity and Penalty are, , and , respectively. However, a one-way ANOVA test shows that the results are not statistically significant. We remark that the data used by Google Maps111Google Maps uses real-time and/or historical traffic data to compute the routes. This data is not made publicly available. Therefore, we were unable to use Google Maps data for all four approaches. Also, it is not possible to enforce Google Maps to generate alternative routes using the OpenStreetMap data. to compute the alternative routes is different from the OpenStreetMap (OSM) data used by the other three approaches. We provide the details of how this may have impacted the participants’ ratings. We also list some other limitations of this user study that are beyond our control.
The rest of the paper is organized as follows. In Section 2, we describe some of the most popular existing techniques to compute alternative paths including the three approaches we use in the user study. The details of our web-based demo system are presented in Section 3. The details of our user study and its results are presented in Section 4. Some limitations of the user study are also discussed in Section 4. Section 5 concludes this paper.
2 Related Work
Computing alternative routes has received significant research in the past decade or so [jones2012method, abraham2013alternative, chen2007reliable, akgun2000finding, chondrogiannis2020finding, chondrogiannis2018finding, liu2017finding]. In this section, we briefly describe some of the most popular techniques and focus on presenting the details of the three techniques that we compare in this user study: Penalty, Plateaus and Dissimilarity.
The basic idea behind this technique [akgun2000finding, chen2007reliable] is to iteratively compute shortest paths and, after each iteration, apply a penalty on each edge of the shortest path found in the previous iteration (by increasing its weight by a certain factor). Since the edge weights of the previous shortest paths are increased, it is likely that the new shortest path found on the graph will be different from the previous path(s). The algorithm stops when shortest paths are retrieved.
This approach does not guarantee that the paths are “signficiantly” different from each other or are meaningful (e.g., without small detours). However, we observe that, in practice, the routes generated by this approach turn out to be pretty good in most of the cases. This is mainly because, on typical road networks, there exists several different paths from to with very similar traveling time. Thus, when the algorithm applies penalty to one of these paths, it naturally tends to select the other significantly different paths. Furthermore, after retrieving each path, the algorithm can specifically apply additional filtering criteria to remove the paths that fail to meet certain requirements, e.g., the paths that are too similar to existing paths or have detours.
The technique to generate alternative paths using plateaus was developed [jones2012method] by Cotares Limited for their routing engine Choice Routing. We use Fig. 1 to illustrate how alternative paths from Cambridge (source ) to Manchester (target ) are generated using plateaus. First, two shortest path trees are generated: a forward shortest path tree rooted at (see Fig. 1(a)); and a backward shortest path tree rooted at (Fig. 1(b)). Then, the two trees and are joined to obtain the branches common in both trees. These common branches are called plateaus. Fig. 1(c) shows some of the most prominent plateaus. It was noted [jones2012method] that longer plateaus result in more meaningful alternative paths. Therefore, top- plateaus are selected based on their lengths. Let and be two ends of a plateau where is the end closer to the source and is the end closer to the target. Each plateau is used to generate an alternative path by appending the shortest paths from to and to to the plateau. Fig. 1(d) shows five alternative paths generated using the five longest plateaus from Fig. 1(c).
It was shown that the alternative paths generated using plateaus are local optimal [abraham2013alternative]. Furthermore, the plateaus do not intersect each other. Thus, two paths generated using longer plateaus are expected to have a smaller overlap (i.e., lower similarity). Thus, the paths generated using the longest plateaus are likely to be more dissimilar to each other. The computational cost to compute alternative paths consists of generating two shortest path trees (e.g., using Dijkstra’s algorithm) and joining the two trees. The latter can be done in time linear to the size of the tree [jones2012method]. Thus, the total cost is dominated by the two Dijkstra searches.
Some existing works [chondrogiannis2020finding, chondrogiannis2018finding, liu2017finding] specifically define a dissimilarity function to compute the dissimilarity of a candidate path to a set of paths . The aim is to iteratively add paths to the result set in ascending order of their lengths as long as they are sufficiently dissimilar to the previously selected paths. Specifically, a path is added to only if where is a user-defined dissimilarity threshold. As a result, the paths reported to the user are significantly different to each other and are short.
The advantage of this approach is that it guarantees that the alternative paths are sufficiently dissimilar to each other (as defined by the parameter ). However, this approach does not guarantee that the generated alternative paths are free of small unnecessary detours. This can be addressed by having additional filtering criteria to prune the paths that do not meet certain criteria. A major disadvantage of this approach is that the problem is NP-hard [chondrogiannis2020finding, liu2017finding]. The existing studies have proposed several approximate algorithms. However, many of these techniques still appear to be too slow taking tens of seconds to report alternative paths on a city-scale road network.
In this study, we use SSVP-D+ [chondrogiannis2018finding] which has been shown to generate good quality alternative routes and has reasonable computation time. The basic idea is to use via-nodes to generate paths. A path (called via-path) generated using a via-node is the concatenation of and where denotes the shortest path from to . To efficiently compute and , similar to the plateaus-based approach, two shortest path trees are constructed rooted at and , respectively. The algorithm iteratively selects via-nodes in an ascending order of their via-paths lengths. A via-path is added to the result set only if its dissimilarity to the existing paths in is greater than the threshold .
2.4 Other techniques
Yen’s algorithm [yen1971finding] can be used to compute shortest paths from to . However, these shortest paths are all expected to be very similar to each other. Thus, Yen’s algorithm is not suitable for generating alternative paths if applied trivially. However, some existing techniques (e.g., see [ChondrogiannisB15]) use Yen’s algorithm to incrementally generate shortest paths and apply filtering techniques to prune the paths that do not meet certain criteria. Pareto optimal [barth2019alternative, BarthFS19] paths (i.e., skyline paths) report the paths that are not dominated by any other path according to given criteria (e.g., distance, travel time and ease of traveling). Many techniques use via-nodes to generate alternative paths (e.g., the SSVP-D+ techniques discussed in Section 2.3). Such techniques identify interesting via-nodes in the road network and then apply different filtering/ranking criteria to generate the top- alternative paths.
3 Demonstration System
Our web-based demonstration system222http://aamircheema.com/routing/demo.html consists of the following major components: 1) road network constructor; 2) user-interface; 3) query processor. We briefly describe each component below.
Road Network Constructor: The road network constructor takes a rectangular area as input and extracts the road network data from OpenStreetMap (OSM) that lies within the input rectangle. We design our demo system to allow routing queries only in Melbourne Metropolitan area, i.e., the input rectangle covers the Melbourne Metropolitan area. First, we export the raw OSM data using Geofabrik333http://download.geofabrik.de/
. Then, we filter the data that lies in the input rectangle. Finally, we parse the content of this raw OSM data to generate the road network data needed for the approaches. Specifically, we extract tuples where each tuple represents an edge of the road network along with its end vertices and edge weight (travel time). The travel time is obtained by dividing the length of the edge with the maximum speed along the edge. In real-world scenarios, the vehicles may need to stop at intersections, wait at traffic lights or slow down while turning even when there is no congestion on the roads. Thus, estimating the travel time using maximum speed is not realistic. To better simulate real-world scenarios, for each road segment that is not a freeway/motorway, we multiply the edge weight (travel time) by 1.3. Our trials showed that this results in a reasonably good estimate of actual travel time when the roads have no congestion (e.g., compared with the travel time estimated by Google Maps at 3:00 am).
The user can rate each approach by clicking on the “Submit Rating” button which opens a form as shown in Fig. 3. Specifically, the form requires the user to rate each approach from 1-5 (higher is better). We also ask the users whether they are currently living (or have lived) in Melbourne. This enable us to analyse the results based on the ratings received by Melbourne residents vs non-residents.
Query Processor: The input to the query processor is a pair of source and target locations each represented by longitude and latitude. First, the query processor performs geo-coordinate matching and selects the closest vertices from the OSM data to the source and target locations, respectively. Then, the query processor computes the alternative routes from the source location to the target location using the three techniques we implemented (Penalty, Plateaus and Dissimilarity). It also calls Google Maps API to get the alternative routes generated by Google Maps. For each of the routes generated by these four approaches, the query processor computes its travel time by using the OSM data. Each travel time is rounded to display time in minutes. Finally, the routes generated by each approach are passed to Google Maps API to display these routes using different colors so that they are easily distinguishable.
|Overall||3.37 (1.33)||3.63 (1.25)||3.58 (1.29)||3.56 (1.17)||237|
|Melbourne residents||3.55 (1.28)||3.69 (1.17)||3.70 (1.22)||3.66 (1.12)||156|
|Non-residents||3.04 (1.37)||3.51 (1.38)||3.34 (1.37)||3.37 (1.25)||81|
|Small Routes (0, 10] (mins)||3.53 (1.17)||3.48 (1.27)||3.69 (1.18)||3.81 (1.08)||66|
|Medium Routes (10, 25] (mins)||3.44 (1.39)||3.51 (1.27)||3.58 (1.26)||3.42 (1.23)||109|
|Long Routes (25, 80] (mins)||3.11 (1.36)||3.98 (1.13)||3.45 (1.44)||3.54 (1.14)||62|
|Melbourne residents||3.55 (1.28)||3.69 (1.17)||3.70 (1.22)||3.66 (1.12)||156|
|Small Routes (0, 10] (mins)||3.50 (1.16)||3.42 (1.27)||3.68 (1.25)||3.97 (0.99)||38|
|Medium Routes (10, 25] (mins)||3.64 (1.28)||3.70 (1.14)||3.78 (1.13)||3.55 (1.17)||83|
|Long Routes (25, 80] (mins)||3.40 (1.42)||3.97 (1.10)||3.54 (1.44)||3.60 (1.09)||35|
|Non-residents||3.04 (1.37)||3.51 (1.38)||3.34 (1.37)||3.37 (1.25)||81|
|Small Routes (0, 10] (mins)||3.57 (1.20)||3.57 (1.29)||3.71 (1.08)||3.61 (1.17)||28|
|Medium Routes (10, 25] (mins)||2.81 (1.55)||2.92 (1.47)||2.96 (1.48)||3.00 (1.33)||26|
|Long Routes (25, 80] (mins)||2.74 (1.23)||4.00 (1.21)||3.33 (1.47)||3.48 (1.22)||27|
Parameter Details: The three approaches that we implemented (Penalty, Plateaus and Dissimilarity) use some parameters. As suggested in [bader2011alternative], for the Penalty approach, the penalty that we apply to each edge is , i.e., the edge weight is multiplied by . For the Plateaus and Dissimilarity approaches, we use the upper bound [abraham2013alternative] to be which ensures that the travel time of any alternative path reported by these approaches is not higher than times the travel time of the fastest path. The dissimilarity threshold for the Dissimilarity approach is set to as suggested by the relevant studies [chondrogiannis2018finding, chondrogiannis2020finding]. We tried several other values for each of the above mentioned parameters to confirm that the chosen values are appropriate.
4 User Study
We created a webpage555http://aamircheema.com/routing explaining the purpose and background of this research and providing instructions to the participants. The participants were able to provide feedback using any internet enabled device (e.g., laptop, tablets or smartphones). We sent the requests for participation in the study via emails and personal messages. Most of the requests were sent to the people living in Melbourne who were likely to be familiar with the roads. For the sake of this study, we say a participant is Melbourne resident if he/she was living (or had lived) in Melbourne at the time of participation. We also sent requests to non-residents to get their ratings for each approach based on their perceived route quality. The Melbourne residents were requested to select the source and target locations for the routes familiar to them (although we cannot guarantee if they always did so). We made sure that none of the approaches tries to avoid toll roads and we told the participants to ignore toll charges on the roads. In total, we received responses ( from Melbourne residents and from non-residents).
We show the results (mean rating and standard deviation) for each approach for different groups of participants (Melbourne residents vs non-residents). We also group the responses based on the lengths of the shortest routes from source to target locations. Specifically, small routes correspond to all the responses where the fastest travel time from to was at most minutes. Medium and long routes correspond to the responses with fastest travel time from to within ranges and minutes, respectively. The highest mean rating for for each group is shown using bold font.
The results in Table 1 show the results for all respondents. The Plateaus achieves the highest mean rating and Google Maps the lowest considering all responses. It is also interesting to note that Penalty has the highest mean rating for the small routes whereas Dissimilarity and Plateaus perform better for the medium and long routes, respectively. Also note that the difference between the mean ratings of Google Maps and the other approaches shrinks, considering responses only from Melbourne residents. This indicates that the routes provided by Google Maps may be perceived to be of poor quality by non-residents when in fact they may not be necessarily so. Next, we provide detailed results considering responses only from Melbourne residents (Table 2) and non-residents (Table 3).
Table 2 shows the results considering responses only from the Melbourne residents. Dissimilarity achieves the highest overall mean rating whereas Plateaus and Penalty have marginally lower mean ratings. As earlier, Penalty performs the best for the small routes whereas Dissimilarity and Plateaus have the highest mean ratings for medium and long routes, respectively. The results considering only the non-residents are shown in Table 3. Plateaus achieves the highest mean rating whereas the Google Maps the lowest. Interestingly, the mean rating for each approach is smaller compared to the mean rating received by Melbourne residents (significantly smaller for medium routes).
We also conducted one-way ANOVA tests for different categories of respondents. Given a null hypothesis of no statistically significant difference in mean ratings of the four approaches, the p-value is relatively high (p-value=0.16 considering all respondents, p-value=0.68 for Melbourne residents and p-value=0.18 for non-residents) suggesting that there is no evidence that the null hypothesis is false, i.e., there is no credible evidence that the four approaches receive different ratings on average.
4.2 Limitations of the study
While we have tried our best to be as fair as possible to all the approaches, certain factors may have affected the participants’ perceived quality of the routes. Below we list some of these.
Different data used by Google Maps and other approaches: One of the major factors (beyond our control) potentially affecting the participants’ ratings is that the data used by Google Maps and the OpenStreetMap (OSM) data used by the other three approaches are different. To estimate the travel time, Google Maps uses real-time traffic data (or historical data for queries issued at future dates/times). To minimize the impact of real-time traffic or historical traffic data, we call Google Maps API to retrieve the routes at 3:00 am on the next day (assuming minimal traffic on roads at that time). However, the travel time estimation for the same route may still be different for Google Maps and OpenStreetMap data. We observe that this difference in the underlying data affects the routes generated by Google Maps and the other approaches.
Consider the example given in Fig. 4 that shows the alternative routes generated by Google Maps and Plateaus for the same pair of source and target. While the blue and green routes returned by both approaches are the same, the purple route returned by Google Maps looks significantly more complicated and appears to have detours. We carefully looked into the purple routes provided by the two approaches. It turns out that if the data from OpenStreetMap is used, the travel time of the purple route by Google Maps is a few minutes higher than that of the purple route by Plateaus. However, when Google Maps data is used to obtain the travel times (by forcing Google Maps to generate a path taken by Plateaus), the travel time of the purple route taken by Plateaus is a few minutes higher than that of the purple route by Google Maps. A user looking at the routes in Fig. 4 is likely to give a higher rating to Plateaus but this may be unfair because the two approaches are essentially using different underlying data.
Apparent detours that are not: Due to the complex structure of road networks, a participant may incorrectly assume that a route provided by an approach has a detour. In the example of Fig. 4(a), the purple route by Google Maps appears to have a detour, e.g., one may assume that the route should have turned left at around “Shrine of Remembrance” instead of the detour. However, this is not a detour because the route goes through a tunnel and there is no left turn available near “Shrine of Remembrance”, i.e., the path returned by Google Maps is a reasonable path. Unless a user is familiar with these roads and/or carefully looks at the road structure (e.g., by zooming in), he/she may perceive it as a detour and give a lower rating. We remark that this does not only negatively affect the ratings received by Google Maps. Other approaches may also be negatively affected by such scenarios where a user may incorrectly assume a reasonable route to have detours.
Biases to favorite routes/navigation systems: A participant’s rating may be biased by his/her favorite routes (or his/her favorite navigation system). E.g., a person who regularly uses Waze to travel from home to office may perceive the routes that are similar to those provided by Waze to have a better quality, although this may not be necessarily true. For example, one participant submitted the following comment: “no route using Blackburn rd”. We believe that the route via Blackburn road is his/her favorite route which was not returned by any of the approaches. Consequently, the maximum rating the user gave to any approach in this case was .
Additional filtering/ranking criteria are not considered: For the approaches that we implemented (Plateaus, Dissimilarity and Penalty), we could choose to use some additional filtering/ranking criteria to refine the top- alternative routes. For example, we could improve the routes generated by Penalty and Plateaus by pruning the alternative routes that have very high similarity to the other routes. Similarly, we could filter the routes in Penalty and Dissimilarity approaches that did not satisfy local optimality [abraham2013alternative]. Some comments from the participants also point out that, at least some, users consider certain factors to be important which we did not consider in our implementation but can be easily included.
For example, some comments from the participants were: “Approach C provides paths with less turns”; “less zig-zag is better”; and “highest rated path follows wide roads”
. This indicates that these users perceive the routes to be better if they have fewer turns or are using wider roads (which probably means roads with more lanes). Since Google Maps is a widely used commercial product, we believe that they would have spent significant time and resources to identify such potentially important factors and implement additional filtering/ranking criteria to report alternative routes. However, most of these filtering criteria can also be easily included for the other approaches but are not considered in this study.
Considering the factors mentioned above (and the other unforeseen factors), we recommend the readers to make any conclusions with caution. However, we believe that our study shows evidence that the three approaches that we implemented (Plateaus, Dissimilarity and Penalty) are at least comparable to Google Maps. We also received some comments from the participants indicating that they find the routes to be of similar quality. For example a participant commented: “I don’t see these approaches as very distinct from each other.”. Another participant sent a personal message to the authors stating that he is finding it hard to rank the approaches since they all seem to be of similar quality.
This is the first detailed user study that compares some of the most popular approaches to generate alternative routes, including Google Maps which is one of the most widely used navigation applications. We develop a web-based demo system to conduct the user study. Our results show that the three approaches published in the literature provide alternative routes of similar quality to each other. Although the mean ratings received by these approaches is higher than that of Google Maps, one-way ANOVA test shows that the results are not statistically significant. We also identify the limitations of this study that might have potentially affected the ratings submitted by the participants. Despite the limitations of the study, we believe that it is fair to conclude that the three approaches (Plateaus, Dissimilarity and Penalty) are promising and the quality of routes generated by them is at least comparable to the routes provided by Google Maps.