AI-DRIVEN ROAD CRACK RECOGNITION USING IMAGE DATA

Abstract

Road infrastructure is a crucial component having a direct impact on the growth and development of any country. These infrastructures require regular upkeep, maintenance, and failure zone identification to ensure proper working. However, maintaining a colossal road network of millions of kilometers is a matter of concern. Apropos, substantial financial investments are provided every year for maintenance and repairs of roads throughout the world. Early detection of pavement defects is imperative to minimize these costs, demanding timely road assessment for existing and upcoming faults. The conventional approach to monitoring road conditions involves manual inspection. It fails to meet the current requirements due to the extensive coverage of the road network to be inspected in a limited time. Recently, some methods to automate the process have been developed in several countries. Some municipalities have started monitoring road conditions by collecting road images from fast-moving vehicles. Later, the images are used to find out the road damage. This approach improves safety but still uses manual distress identification, which is time-consuming. Some fully automated methods have also been introduced to monitor road conditions on time. These methods are primarily based on using specialized or dedicated vehicles equipped with multiple sensors such as laser scanners, road profilers, cameras, etc. These methods save time; however, the assets required are costly. As a result, most municipalities fail to afford this approach, particularly in developing countries. This research addresses the limitations mentioned above, identifies the challenges associated with low-cost automatic monitoring, and fills the gaps for probing the suitability of solutions developed for one country to monitor road conditions in other countries. The proposed solution targets municipalities and local governments seeking collaboration with citizens/residents to improve the road infrastructure. Initially, it is designed considering Indian scenarios; additionally, recommendations are provided to scale the proposed solution for other countries with minimal effort. Recognizing the potential of data science, computer vision, and deep learning techniques for automation, the study explores data-driven approaches to automate road condition monitoring. First of all, the datasets available from existing studies are identified. The identified datasets are then evaluated for their suitability to support low-cost monitoring and applicability in the Indian context. The investigation unfolds the requirement of a new dataset to train India-specific low-cost road damage detection models. To meet the aforementioned requirement, the study proposes a new large-scale heterogeneous road damage dataset, RDD-2020. The dataset comprises 26620 Smartphone images captured from India, Japan, and the Czech Republic. The images are annotated for four types of road damage - longitudinal cracks (26.30%), transverse cracks (17.48%), alligator cracks (33.86%), and potholes (22.36%). The dataset is used to train several deep learning models considering different scenarios for efficient road crack recognition in India, Japan, and the Czech Republic. Two aspects, namely Data Augmentation and Network Architecture improvement, are considered to optimize the trained models. Data Augmentation involves augmenting the data for both pure modeling and mixed modeling. Data from the same country is utilized in pure modeling, and several models are trained with varying amounts of trained data. Similarly, for mixed modeling, data from other countries are mixed with the local data (of India), and new models are introduced. The performance of the models is assessed using F1-score and mean Average Precision (mAP), and the model trained using a combination of Indian and Japanese data is identified as the best performing model for both countries. The analysis in the study demonstrates the usefulness of data augmentation as a powerful tool to address the domain adaptation for low-cost road condition monitoring across multiple countries. Concerning damage detection, the investigation shows that utilizing foreign data for data augmentation while training deep neural network models improves the models' accuracy and generalizing ability. For damage classification, it is discovered that some damage categories may be better represented while training the models by using foreign data. After deciding whether the models should be trained using the data from a single country (India, in this case) or utilizing data from foreign countries along with the local one, Network Architecture improvement is explored. To this end, Global Road Damage Detection Challenge (GRDDC) is organized as an IEEE Big Data Cup in 2020. GRDDC involved releasing the data, problem statement, and preliminary analysis of the research. Out of the several solutions proposed through GRDDC, the top 12 solutions are shortlisted and considered in the current work. These solutions are analyzed, and recommendations are provided to researchers and practitioners from multiple countries to use the road damage detection and classification models considered in the study asper their requirements. The highest accuracy (F1-score: 0.67) in detecting and classifying road damage in three countries, India, Japan, and Czech Republic, is achieved by the YOLOv5-based ensemble model. The analysis so far considers detection and classification of road damage at the object level, using boundary boxes. The final objective of the study contemplates crack recognition at the pixel level. First of all, the conventional edge-detection operators are analyzed. Two main factors are considered to conduct the study: the effect of convolution kernel size on the performance and the impact of using the directional component of the operators. Based on the analysis, a new algorithm, GEOM, is proposed to effectively highlight crack pixels in the image. GEOM combines low-pass and high-pass filters with optimal kernel size, Otsu thresholding, and morphological transformations and outperforms the conventional edge detection operators. The performance improvement is validated by comparing the performance of four operators: Sobel, Scharr, Prewitt, and Roberts, when used stand-alone vs. when used as per GEOM. Overall, the study provides new insights on monitoring road conditions in multiple countries in a systematic and holistic manner. The study contributes to the literature by offering new data, deep learning-based models for object-level road damage detection and classification, a computer-vision-based approach for pixel-level crack recognition, and analysis to transfer the solution from one country to another. The findings shall be helpful for transportation planners, government authorities, road agencies, policymakers, and researchers in proposing guidelines and formulating new strategies that encourage Smartphone-based AI-driven road condition monitoring in India and several other countries. As an outcome of the study, a smartphone application and a web-based interface have been proposed enabling road managers to perform quick road surveys and road users to report the road damage in their vicinity for swift repairing. Keywords: Artificial Intelligence, Automation, Deep Learning, Damage Detection and Classification, Intelligent Transport Systems, Multinational Solutions, Road Cracks, Road Imaging, Road Damage, Road Condition Inspection, Pavement Maintenance, Smartphone-based Monitoring.