DeepAI AI Chat
Log In Sign Up

Towards Reliable Evaluation of Road Network Reconstructions

by   Leonardo Citraro, et al.

Existing performance measures rank delineation algorithms inconsistently, which makes it difficult to decide which one is best in any given situation. We show that these inconsistencies stem from design flaws that make the metrics insensitive to whole classes of errors. To provide more reliable evaluation, we design three new metrics that are far more consistent even though they use very different approaches to comparing ground-truth and reconstructed road networks. We use both synthetic and real data to demonstrate this and advocate the use of these corrected metrics as a tool to gauge future progress.


The Effect of Ground Truth Accuracy on the Evaluation of Localization Systems

The ability to accurately evaluate the performance of location determina...

A metric for evaluating 3D reconstruction and mapping performance with no ground truthing

It is not easy when evaluating 3D mapping performance because existing m...

Data-driven Natural Language Generation: Paving the Road to Success

We argue that there are currently two major bottlenecks to the commercia...

Towards Reliable Item Sampling for Recommendation Evaluation

Since Rendle and Krichene argued that commonly used sampling-based evalu...

Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth

The accurate measurement of security metrics is a critical research prob...

Communication in Complex Networks

The investigation of properties of networks has many applications and is...