RoadAtlas: Intelligent Platform for Automated Road Defect Detection and Asset Management

09/08/2021
by   Zhuoxiao Chen, et al.
The University of Queensland
0

With the rapid development of intelligent detection algorithms based on deep learning, much progress has been made in automatic road defect recognition and road marking parsing. This can effectively address the issue of an expensive and time-consuming process for professional inspectors to review the street manually. Towards this goal, we present RoadAtlas, a novel end-to-end integrated system that can support 1) road defect detection, 2) road marking parsing, 3) a web-based dashboard for presenting and inputting data by users, and 4) a backend containing a well-structured database and developed APIs.

READ FULL TEXT VIEW PDF

Authors

page 2

06/19/2018

A Decision Support System Web-Application for the Management of Forest Road Network

The present study contributes to the development of an online FRMP (Fore...
12/12/2017

Review. Machine learning techniques for traffic sign detection

An automatic road sign detection system localizes road signs from within...
07/20/2014

Optimized Method for Iranian Road Signs Detection and recognition system

Road sign recognition is one of the core technologies in Intelligent Tra...
05/05/2019

A Methodological Review of Visual Road Recognition Procedures for Autonomous Driving Applications

The current research interest in autonomous driving is growing at a rapi...
05/17/2021

Learning to Automatically Catch Potholes in Worldwide Road Scene Images

Among several road hazards that are present in any paved way in the worl...
01/17/2018

User assisted and automatic inverse procedural road modelling at the city scale

Cities are structured by roads. Having up to date and detailed maps of t...
08/06/2018

Sunlight Enabled Vehicle Detection by LED Street Lights

We propose and demonstrate a preliminary traffic sensing system based on...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Assessing the condition of the road network is vital for ensuring its continued functionality. The majority of road agencies in Australia conduct road network condition surveys through employing specialised machines such as laser profiler meters to obtain performance and condition data of the road network. There are also a number of road agencies that rely on experience inspectors via walking or driving the road network to collect defects data. Being a costly and time-consuming procedure (around $350K to $400K per survey), it prevents the local government from conducting frequent surveys to keep the data up to date. The cracks and destroyed road markings not only affect the aesthetic value of the city, but also increase the risks of injury and fatality faced by pedestrians. Hence, there is an urgent need for automation in road asset management by adopting data science for predictive analytics, visual recognition, and interactive visualisation.

The framework’s purpose is to forge a mutually beneficial program by forming a novel AI-powered road defects and assets management platform, namely RoadAtlas. Our vision is to deliver the intelligent system for jointly road defect segmentation and road marking parsing, being able to grow and adapt to the new data or novel tasks (e.g., detecting a new crack type). The RoadAtlas will afford an unprecedented capacity to automatically demonstrate and report types, severity, location and evidence of road anomalies in real-time for planning the follow-up repairs. The RoadAtlas also enhances local government employees’ data analytic skillsets and help road asset managers monitor the functional and structural performance of road networks efficiently and effectively.

2. System Overview

Figure 1. The overall framework of the RoadAtlas.

The system consists of three modules: the dashboard, deep visual models and the backend, as shown in Figure 1. The dashboard is a user interaction interface that supports various front-end functions. The deep models are designed for road defect and road marking recognition. The backend is built with Python Flask to exchange data with the front-end and save data in the MongoDB database.

Figure 2. Dashboard (top) and Road Defects View (bottom).

2.1. Front-end and Back-end

The Dashboard is the primary user interface designed as a web application, illustrated in Figure 2. It features four primary functions: 1) Road Defect, 2) Road Marking, 3) Database Update and 4) Report Export. Road Defect and Road Marking allow users to inspect and validate labelled images from the database. The Road Defect example shown in Figure 2 displays an interactive map with map pins representing individual defects. When the user selects a map pin, a validation panel displays on the right, providing defect details such as the labelled images and coordinates. Next, the user can validate the correctness of defects and mark them as checked. The Database Update function is a gateway for users to upload unprocessed images to the deep model for defect and marking detection. Additional Report Export function supports saving validated images data as CSV or JSON-formatted files. The backend aims to provide functional APIs to exchange data with the dashboard side. Our developers set up multiple APIs according to the need of the dashboard via the Python Flask framework. Also, we construct the MongoDB database for managing data. We further provide 20TB hard disk storage on our server for storing huge amounts of images.

2.2. Road Defect Detection and Segmentation

When users utilise the database update function by inserting new road images, the proposed deep models can perform recognition for road defects.

Dataset Annotation.

To train models for road defect detection following supervised learning principles, labels are required for training images. However, the 10,000 raw images provided by the local government are unlabelled. These images represent the foreground of the car, taken in Logan City, Australia. We arranged five annotators to label all images with various classes of road defects, including

background, Kerb_Cracking, Road_Crocodile, Road_Longitudinal, Road_Transverse, Road_Block and Sealed_Crack. Each of the road defects is labelled in an irregular polygon.

Figure 3. Examples of road defect detection and segmentation results.

Crack Detection and Segmentation. We developed a pair of deep models to effectively capture road defects consisting of a crack detector and a crack segmenter. There are four steps for crack recognition. First, we utilise the anonymiser (understand-ai, 2019) to protect the privacy of car owners via automatically hiding the license plate numbers. Second, we convert labelled polygons to the minimised bounding boxes that cover the polygon. Third, we proposed a crack detector inspired by (Jocher, 2020), fully trained with converted labelled bounding boxes. Thus, the crack detector can accurately produce bounding boxes for different defects. The last step is to feed predicted bounding boxes into the crack segmenter to produce pixel-level predictions for each road defect. We follow (Ronneberger et al., 2015) to construct the crack segmenter, which is specially trained for segmenting road defects. Figure 3 shows one detection result on the top, and two segmentation pixel-level results (shown in two rows) at the bottom. In the bottom segmentation result, the left image is the detected bounding boxes by the crack detector. The image at the mid is the prediction feature map by the crack segmenter. The right image maps predictions back to the original image for visualisation.

2.3. Road Marking Parsing

We develop a road marking segmenter to detect different categories of road markings at pixel level accurately. The road marking segmenter follows (Zhao et al., 2017) and is thoroughly trained with the existing benchmark dataset (Huang et al., 2020) containing various labelled road markings. After prediction, we performed an extra refinement process to fit the results back to the original image. The refinement process can be summarised into three steps. The first step is to detect the region of interest (ROI), which is the road in our case. Then we project both original images and prediction maps from street view into bird eye view to obtain parallel or perpendicular road markings. The second step is to implement low-level contour detection, which focuses on generating the boundaries of objects. The low-level contour of road markings is regular. The predicted pixels are irregular but cover most areas of the road markings. Thus, we compare the prediction map with low-level contours. If there is a high overlap between them, the low-level contours are kept and mapped back to the original image.

3. Conclusion

In this paper, we present a fully developed system - RoadAtlas, that comprehensively provides functions of road defect recognition, road marking parsing integrated into an end-to-end platform. It can achieve faster automated recognition of road defects than manually reviewing images by well-trained inspectors, allowing local governments to repair roads in time. Besides, a great portion of budgets can be saved by full automation. The local governments can effortlessly grasp and manage road conditions with RoadAtlas.

References

  • X. Huang, P. Wang, X. Cheng, D. Zhou, Q. Geng, and R. Yang (2020) The apolloscape open dataset for autonomous driving and its application. IEEE Trans. Pattern Anal. Mach. Intell. 42 (10), pp. 2702–2719. Cited by: §2.3.
  • G. Jocher (2020) ultralytics/yolov5: v3.1 - Bug Fixes and Performance Improvements Note: https://github.com/ultralytics/yolov5 External Links: Document, Link Cited by: §2.2.
  • O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III, N. Navab, J. Hornegger, W. M. W. III, and A. F. Frangi (Eds.), Lecture Notes in Computer Science, Vol. 9351, pp. 234–241. External Links: Link, Document Cited by: §2.2.
  • understand-ai (2019) Understand.ai anonymizer. GitHub. Note: https://github.com/understand-ai/anonymizer Cited by: §2.2.
  • H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia (2017) Pyramid scene parsing network. In

    2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017

    ,
    pp. 6230–6239. Cited by: §2.3.