Volume 21, Issue 3 (JIAEEE Vol.21 No.3 2024)
Journal of Iranian Association of Electrical and Electronics Engineers 2024, 21(3): 139-154 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Aghayan Mashhady N, Amirkhani A. Improving the Accuracy of Road Damage Detectors Using Traditional Augmentations and Object Bounding Box Augmentation. Journal of Iranian Association of Electrical and Electronics Engineers 2024; 21 (3) :139-154
URL: http://jiaeee.com/article-1-1561-en.html
URL: http://jiaeee.com/article-1-1561-en.html
Department of Electrical Engineering, School of Automotive Engineering, Iran University of Science and Technology
Abstract: (1283 Views)
Automatic detection of road damage, speeds up the process of their maintenance and prevents many traffic accidents. In this research paper, ten road damage detectors were developed in the RDD2020 dataset to check the performance of road damage detectors using YOLOv5. By simulating environmental conditions such as intense sunlight and the shadow on the road surface it was determined that road damage detectors perform poorly in these conditions. Further, by using traditional data augmentation techniques, the robustness of road damage detectors was improved in different environmental conditions. Using these techniques, a validation dataset was created to check the performance of detectors in different environmental conditions in the RDD2020 dataset. In order to solve the problem of data scarcity in this dataset, the bounding box augmentation technique and modified Poisson blending were combined. In this method, the number of instances of the reference dataset was increased in pothole and horizontal crack class. The results of this research show that training detectors with new data has improved their performance in F1-Score and mAP by 33% and 50%, respectively.
Type of Article: Research |
Subject:
Electronic
Received: 2023/01/24 | Accepted: 2023/10/1 | Published: 2024/11/2
Received: 2023/01/24 | Accepted: 2023/10/1 | Published: 2024/11/2
References
1. [1] Z. Pei, R. Lin, X. Zhang, H. Shen, J. Tang, and Y. Yang, "Cfm: A consistency filtering mechanism for road damage detection", in IEEE International Conference on Big Data (Big Data), 2020 pp. 5584-5591. [DOI:10.1109/BigData50022.2020.9377911]
2. [2[ V. Mandal, A. R. Mussah, and Y. Adu-Gyamfi, "Deep learning frameworks for pavement distress classification: A comparative analysis", in IEEE International Conference on Big Data (Big Data), 2020pp. 5577-5583. [DOI:10.1109/BigData50022.2020.9378047]
3. [3] Y. Li, P. Che, C. Liu, D. Wu, and Y. Du, "Cross‐scene pavement distress detection by a novel transfer learning framework", Computer‐Aided Civil and Infrastructure Engineering, vol. 36, no. 11, pp. 1398-1415, 2021. [DOI:10.1111/mice.12674]
4. [4] D. Arya et al., "Deep learning-based road damage detection and classification for multiple countries", Automation in Construction, vol. 132, p. 103935, 2021. [DOI:10.1016/j.autcon.2021.103935]
5. [5] A. Ragnoli, M. R. De Blasiis, and A. Di Benedetto, "Pavement distress detection methods: A review", Infrastructures, vol. 3, no. 4, p. 58, 2018. [DOI:10.3390/infrastructures3040058]
6. [6] T. B. Coenen and A. Golroo, "A review on automated pavement distress detection methods", Cogent Engineering, vol. 4, no. 1, p. 1374822, 2017. [DOI:10.1080/23311916.2017.1374822]
7. [7] Z. Du, J. Yuan, F. Xiao, and C. Hettiarachchi, "Application of image technology on pavement distress detection: A review", Measurement, vol. 184, pp 109900. 2021. [DOI:10.1016/j.measurement.2021.109900]
8. [8] V. Pham, C. Pham, and T. Dang, "Road damage detection and classification with detectron2 and faster r-cnn", in IEEE International Conference on Big Data (Big Data), 2020, pp. 5592-5601. [DOI:10.1109/BigData50022.2020.9378027]
9. [9] H. Oliveira and P. L. Correia, "CrackIT-An image processing toolbox for crack detection and characterization", in IEEE International Conference on Image Processing (ICIP), 2014, pp. 798-802. [DOI:10.1109/ICIP.2014.7025160] [PMID] []
10. [10] A. Zhang et al., "Automated pixel‐level pavement crack detection on 3D asphalt surfaces using a deep‐learning network", Computer‐Aided Civil and Infrastructure Engineering, vol. 32, no. 10, pp. 805-819, 2017. [DOI:10.1111/mice.12297]
11. [11] S. Li, Y. Cao, and H. Cai, "Automatic pavement-crack detection and segmentation based on steerable matched filtering and an active contour model", Journal of Computing in Civil Engineering, vol. 31, no. 5, p. 04017045, 2017. [DOI:10.1061/(ASCE)CP.1943-5487.0000695]
12. [12] N. Kanopoulos, N. Vasanthavada, and R. L. Baker, "Design of an image edge detection filter using the Sobel operator", IEEE Journal of Solid-State Circuits, vol. 23, no. 2, pp. 358-367, 1988. [DOI:10.1109/4.996]
13. [13] L. Yang, X. Wu, D. Zhao, H. Li, and J. Zhai, "An improved Prewitt algorithm for edge detection based on noised image", in 4th International Congress on Image and Signal Processing, 2011, vol. 3, pp. 1197-1200. [DOI:10.1109/CISP.2011.6100495] []
14. [14] H. Maeda, Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata, "Road damage detection and classification using deep neural networks with smartphone images", Computer‐Aided Civil and Infrastructure Engineering, vol. 33, no. 12, pp. 1127-1141, 2018. [DOI:10.1111/mice.12387]
15. [15] D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, and Y. Sekimoto, "RDD2020: An annotated image dataset for automatic road damage detection using deep learning", Data in Brief, vol. 36, p. 107133, 2021. [DOI:10.1016/j.dib.2021.107133] [PMID] []
16. [16] M. S. Sohrabi and M. Moazzami, "A Hybrid Approach for Probabilistic mid-term Electricity Price Forecasting using deep learning", Journal of Iranian Association of Electrical and Electronics Engineers, Research vol. 20, no. 4, pp. 123-132, 2023. [DOI:10.61186/jiaeee.20.4.412]
17. [17] N. Aghaei, G. Akbarizadeh, and A. Kosarian, "Using ShuffleNet to design a deep semantic segmentation model for oil spill detection in synthetic aperture radar images", Journal of Iranian Association of Electrical and Electronics Engineers, Research vol. 19, no. 3, pp. 131-144, 2022. [DOI:10.52547/jiaeee.19.3.131]
18. [18] M. Molaei and A. Amirkhani, "Policy-based Auto-Driving in Highway based on Distributional Reinforcement Learning Methods", Journal of Iranian Association of Electrical and Electronics Engineers, Research vol. 19, no. 2, pp. 209-222, 2022. [DOI:10.52547/jiaeee.19.2.207]
19. [19] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 580-587. [DOI:10.1109/CVPR.2014.81]
20. [20] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 779-788. [DOI:10.1109/CVPR.2016.91]
21. [21] S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks", Advances in Neural Information Processing Systems, vol. 28, 2015.
22. [22] R. Girshick, "Fast r-cnn", in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440-1448. [DOI:10.1109/ICCV.2015.169]
23. [23] Glenn Jocher, "ultralytics/YOLOv5: v6.0 - YOLOv5n 'Nano' models, Roboflow integration, TensorFlow export, OpenCV DNN support", Zenodo, Oct. 12, 2021. Doi: 10.5281/zenodo.5563715.
24. [24] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: Optimal speed and accuracy of object detection", arXiv preprint arXiv:2004.10934, 2020.
25. [25] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement", arXiv preprint arXiv:1804.02767, 2018.
26. [26] U. Nepal and H. Eslamiat, "Comparing YOLOv3, YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs", Sensors, vol. 22, no. 2, p. 464, 2022. [DOI:10.3390/s22020464] [PMID] []
27. [27] M. Li, Z. Zhang, L. Lei, X. Wang, and X. Guo, "Agricultural greenhouses detection in high-resolution satellite images based on convolutional neural networks: Comparison of faster R-CNN, YOLO v3 and SSD", Sensors, vol. 20, no. 17, p. 4938, 2020. [DOI:10.3390/s20174938] [PMID] []
28. [28] Y. J. Wang, M. Ding, S. Kan, S. Zhang, and C. Lu, "Deep proposal and detection networks for road damage detection and classification", in IEEE International Conference on Big Data (Big Data), 2018, pp. 5224-5227. [DOI:10.1109/BigData.2018.8622599]
29. [29] W. Liu et al., "Ssd: Single shot multibox detector", in European conference on computer vision, 2016: Springer, pp. 21-37. [DOI:10.1007/978-3-319-46448-0_2]
30. [30] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition", in Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, 2016, pp. 770-778. [DOI:10.1109/CVPR.2016.90] [PMID]
31. [31] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", arXiv preprint arXiv:1409.1556, 2014.
32. [32] W. Wang, B. Wu, S. Yang, and Z. Wang, "Road damage detection and classification with faster R-CNN", in 2018 IEEE International Conference on Big Data (Big data), 2018, pp. 5220-5223. [DOI:10.1109/BigData.2018.8622354] []
33. [33] F. Kluger et al., "Region-based cycle-consistent data augmentation for object detection", in 2018 IEEE International Conference on Big Data (Big Data), 2018: IEEE, pp. 5205-5211. [DOI:10.1109/BigData.2018.8622318]
34. [34] H. Maeda, T. Kashiyama, Y. Sekimoto, T. Seto, and H. Omata, "Generative adversarial network for road damage detection", Computer-Aided Civil and Infrastructure Engineering, vol. 36, no. 1, pp. 47-60, 2021. [DOI:10.1111/mice.12561]
35. [35] I. Goodfellow et al., "Generative adversarial nets", Advances in Neural Information Processing Systems, vol. 27, 2014.
36. [36] T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation", arXiv preprint arXiv:1710.10196, 2017.
37. [37] P. Pérez, M. Gangnet, and A. Blake, "Poisson image editing", in ACM SIGGRAPH 2003 Papers, 2003, pp. 313-318. [DOI:10.1145/1201775.882269] [PMID] []
38. [38] M.-T. Cao, Q.-V. Tran, N.-M. Nguyen, and K.-T. Chang, "Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources", Advanced Engineering Informatics, vol. 46, p. 101182, 2020. [DOI:10.1016/j.aei.2020.101182]
39. [39] S. R. Bose and V. S. Kumar, "Efficient inception V2 based deep convolutional neural network for real‐time hand action recognition", IET Image Processing, vol. 14, no. 4, pp. 688-696, 2020. [DOI:10.1049/iet-ipr.2019.0985]
40. [40] A. Alfarrarjeh, D. Trivedi, S. H. Kim, and C. Shahabi, "A deep learning approach for road damage detection from smartphone images", in 2018 IEEE International Conference on Big Data (Big Data), 2018: IEEE, pp. 5201-5204. [DOI:10.1109/BigData.2018.8621899]
41. [41] P.-j. Chun, K. HASHIMOTO, N. KATAOKA, N. KURAMOTO, and M. OHGA, "Asphalt pavement crack detection using image processing and naive bayes based machine learning approach", Journal of Japan Society of Civil Engineers, Ser. E1 (Pavement Engineering), vol. 70, no. 3, 2015. [DOI:10.2208/jscejpe.70.I_1]
42. [42] E. Zalama, J. Gómez‐García‐Bermejo, R. Medina, and J. Llamas, "Road crack detection using visual features extracted by Gabor filters", Computer‐Aided Civil and Infrastructure Engineering, vol. 29, no. 5, pp. 342-358, 2014. [DOI:10.1111/mice.12042]
43. [43] W. R. L. d. Silva and D. S. d. Lucena, "Concrete cracks detection based on deep learning image classification", in Proceedings, 2018, vol. 2, no. 8: MDPI AG, p. 489. [DOI:10.3390/ICEM18-05387] [PMID]
44. [44] A. Angulo, J. A. Vega-Fernández, L. M. Aguilar-Lobo, S. Natraj, and G. Ochoa-Ruiz, "Road damage detection acquisition system based on deep neural networks for physical asset management", in Mexican International Conference on Artificial Intelligence, 2019: Springer, pp. 3-14. [DOI:10.1007/978-3-030-33749-0_1]
45. [45] H. Maeda, T. Kashiyama, Y. Sekimoto, T. Seto, and H. Omata, "Generative adversarial network for road damage detection", Computer‐Aided Civil and Infrastructure Engineering, vol. 36, no. 1, pp. 47-60, 2021. [DOI:10.1111/mice.12561]
46. [46] L. Liu et al., "Deep learning for generic object detection: A survey", International Journal of Computer Vision, vol. 128, no. 2, pp. 261-318, 2020. [DOI:10.1007/s11263-019-01247-4]
47. [47] C.-Y. Wang, H.-Y. M. Liao, Y.-H. Wu, P.-Y. Chen, J.-W. Hsieh, and I.-H. Yeh, "CSPNet: A new backbone that can enhance learning capability of CNN", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 390-391. [DOI:10.1109/CVPRW50498.2020.00203]
48. [48] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, "Path aggregation network for instance segmentation", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8759-8768. [DOI:10.1109/CVPR.2018.00913]
49. [49] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2117-2125. [DOI:10.1109/CVPR.2017.106] []
50. [50] S. Elfwing, E. Uchibe, and K. Doya, "Sigmoid-weighted linear units for neural network function approximation in reinforcement learning", Neural Networks, vol. 107, pp. 3-11, 2018. [DOI:10.1016/j.neunet.2017.12.012] [PMID]
51. [51] V. Hegde, D. Trivedi, A. Alfarrarjeh, A. Deepak, S. H. Kim, and C. Shahabi, "Yet another deep learning approach for road damage detection using ensemble learning", in IEEE International Conference on Big Data (Big Data), 2020, pp. 5553-5558. [DOI:10.1109/BigData50022.2020.9377833]
52. [52] T.-Y. Lin et al., "Microsoft coco: Common objects in context", in European Conference on Computer Vision, 2014: Springer, pp. 740-755. [DOI:10.1007/978-3-319-10602-1_48]
53. [53] H. Zhang and J. Wang, "Towards adversarially robust object detection", in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 421-430. [DOI:10.1109/ICCV.2019.00051]
54. [54] M. Khodabandeh, A. Vahdat, M. Ranjbar, and W. G. Macready, "A robust learning approach to domain adaptive object detection", in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 480-490. [DOI:10.1109/ICCV.2019.00057] [PMID]
55. [55] R. Volpi, H. Namkoong, O. Sener, J. C. Duchi, V. Murino, and S. Savarese, "Generalizing to unseen domains via adversarial data augmentation", Advances in Neural Information Processing Systems, vol. 31, 2018.
56. [56] A. Buslaev, V. I. Iglovikov, E. Khvedchenya, A. Parinov, M. Druzhinin, and A. A. Kalinin, "Albumentations: fast and flexible image augmentations", Information, vol. 11, no. 2, p. 125, 2020. [DOI:10.3390/info11020125]
57. [57] B. Zoph, E. D. Cubuk, G. Ghiasi, T.-Y. Lin, J. Shlens, and Q. V. Le, "Learning data augmentation strategies for object detection", in European Conference on Computer Vision, 2020: Springer, pp. 566-583. [DOI:10.1007/978-3-030-58583-9_34]
58. [58] M. Afifi and K. F. Hussain, "MPB: A modified poisson blending technique", Computational Visual Media, vol. 1, no. 4, pp. 331-341, 2015. [DOI:10.1007/s41095-015-0027-z]
Send email to the article author
| Rights and permissions | |
 |
This Journal is an open access Journal Licensed under the Creative Commons Attribution-NonCommercial 4.0 International License. (CC BY NC 4.0) |
