Abstract
The paper outlines a pipeline for the removal of transient objects from orthophotos to enhance the clarity and utility for orthophotos in military and civilian geo-databases generation as the main application. The presented deep-learning-based pipeline includes detecting the objects of interest, masking them out, and using the image and an enhanced inpainting mask to fill in these areas seamlessly. The approach combines semantic segmentation, utilizing an adapted DeepLabv3+ model, with shadow detection using Particle Swarm Optimization, and concludes with a generative inpainting process using a three-stage Generative Adversarial Network (3GAN) system for edge, segmentation, and texture inpainting. This method is applied to a well-known remote sensing dataset for detailed analysis, highlighting the integrated approach’s effectiveness in creating realistic, cleaned-up orthophotos.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ammour, N., Alhichri, H., Bazi, Y., Benjdira, B., Alajlan, N., Zuair, M.: Deep learning approach for car detection in UAV imagery. Remote Sensing 9(4/312), 1–15 (2017)
Bulatov, D., Häufel, G., Meidow, J., Pohl, M., Solbrig, P., Wernerus, P.: Context-based automatic reconstruction and texturing of 3D urban terrain for quick-response tasks. ISPRS J. Photogramm. Remote. Sens. 93, 157–170 (2014)
Chen, H., Luo, Y., Cao, L., Zhang, B., Guo, G., Wang, C., Li, J., Ji, R.: Generalized zero-shot vehicle detection in remote sensing imagery via coarse-to-fine framework. In: International Joint Conference on Artificial Intelligence. pp. 687–693 (2019)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European Conference on Computer Vision. pp. 801–818 (2018)
Chen, X., Xiang, S., Liu, C.L., Pan, C.H.: Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 11(10), 1797–1801 (2014)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)
Elharrouss, O., Almaadeed, N., Al-Maadeed, S., Akbari, Y.: Image inpainting: A review. Neural Process. Lett. 51, 2007–2028 (2019)
Frommholz, D., Kuijper, F., Bulatov, D., Cheung, D.: Geospecific terrain databases for military simulation environments. In: Electro-Optical Remote Sensing XVI. vol. 12272, pp. 46–59. SPIE (2022)
Guo, S., Xiong, X., Liu, Z., Bai, X., Zhou, F.: Infrared simulation of large-scale urban scene through LOD. Opt. Express 26(18), 23980–24002 (2018)
He, Z., Zhang, Z., Guo, M., Wu, L., Huang, Y.: Adaptive unsupervised-shadow-detection approach for remote-sensing image based on multichannel features. Remote Sensing 14(12), 2756 (2022)
Huang, Z., Qin, C., Liu, R., Weng, Z., Zhu, Y.: Semantic-aware context aggregation for image inpainting. In: Proc. IEEE International Conference on Acoustics, Speech and Signal Processing. pp. 2465–2469. IEEE (2021)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Transactions on Graphics (TOG) 36(4), 1–14 (2017)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1125–1134 (2017)
Ji, H., Gao, Z., Mei, T., Ramesh, B.: Vehicle detection in remote sensing images leveraging on simultaneous super-resolution. IEEE Geosci. Remote Sens. Lett. 17(4), 676–680 (2019)
Kottler, B., List, L., Bulatov, D., Weinmann, M.: 3GAN: A three-gan-based approach for image inpainting applied to the reconstruction of occluded parts of building walls. In: VISIGRAPP (4: VISAPP). pp. 427–435 (2022)
Leberl, F., Bischof, H., Grabner, H., Kluckner, S.: Recognizing cars in aerial imagery to improve orthophotos. In: Proc. ACM International Symposium on Advances in Geographic Information Systems. p. 2. ACM (2007)
Li, J., Wang, N., Zhang, L., Du, B., Tao, D.: Recurrent feature reasoning for image inpainting. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7760–7768 (2020)
Liao, L., Xiao, J., Wang, Z., Lin, C.W., Satoh, S.: Guidance and evaluation: Semantic-aware image inpainting for mixed scenes. In: Proc. 16th European Conference on Computer Vision, Part XXVII 16. pp. 683–700. Springer (2020)
Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 85–100 (2018)
Liu, Y., Piramanayagam, S., Monteiro, S.T., Saber, E.: Dense semantic labeling of very-high-resolution aerial imagery and lidar with fully-convolutional neural networks and higher-order CRFs. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp. 76–85 (2017)
Mo, N., Yan, L.: Improved faster RCNN based on feature amplification and oversampling data augmentation for oriented vehicle detection in aerial images. Remote Sensing 12(16/2558), 1–21 (2020)
Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., Ebrahimi, M.: EdgeConnect: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212 (2019)
Park, J., Cho, Y.K., Kim, S.: Deep learning-based UAV image segmentation and inpainting for generating vehicle-free orthomosaic. Int. J. Appl. Earth Obs. Geoinf. 115, 103111 (2022)
Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: Feature learning by inpainting. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition. pp. 2536–2544 (2016)
Pyo, J., Rocha, Y.G., Ghosh, A., Lee, K., In, G., Kuc, T.: Object removal and inpainting from image using combined GANs. In: Proc. 20th International Conference on Control, Automation and Systems (ICCAS). pp. 1116–1119 (2020)
Qiu, K., Bulatov, D., Lucks, L.: Improving car detection from aerial footage with elevation information and markov random fields. In: Proceedings of the 19th International Conference on Signal Processing and Multimedia Applications, SIGMAP 2022. pp. 112–119. SCITEPRESS (2022)
Rottensteiner, F., Sohn, G., Gerke, M., Wegner, J.D., Breitkopf, U., Jung, J.: Results of the ISPRS benchmark on urban object detection and 3D building reconstruction. ISPRS J. Photogramm. Remote. Sens. 93, 256–271 (2014)
Schilling, H., Bulatov, D., Niessner, R., Middelmann, W., Soergel, U.: Detection of vehicles in multisensor data via multibranch convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 11(11), 4299–4316 (2018)
Schlagenhauf, T., Xia, Y., Fleischer, J.: Context-based image segment labeling (cbisl). arXiv preprint arXiv:2011.00784 (2020)
Shalunts, G., Haxhimusa, Y., Sablatnig, R.: Architectural style classification of building facade windows. In: International Symposium on Visual Computing. pp. 280–289. Springer (2011)
Shao, H., Wang, Y., Fu, Y., Yin, Z.: Generative image inpainting via edge structure and color aware fusion. Signal Processing: Image Communication 87–115929, 1–9 (2020)
Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., Kuo, C.C.J.: Contextual-based image inpainting: Infer, match, and translate. In: Proc. IEEE European Conference on Computer Vision (ECCV). pp. 3–19 (2018)
Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., Kuo, C.C.J.: Spg-net: Segmentation prediction and guidance network for image inpainting. In: Proc. British Machine Vision Conference. vol. 97, pp. 1–14 (2018)
Tang, T., Zhou, S., Deng, Z., Zou, H., Lei, L.: Vehicle detection in aerial images based on region convolutional neural networks and hard negative example mining. Sensors 17(2), 336 (2017)
Tayara, H., Soo, K.G., Chong, K.T.: Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network. IEEE Access 6, 2220–2230 (2017)
Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., Li, H.: High-resolution image inpainting using multi-scale neural patch synthesis. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition. pp. 6721–6729 (2017)
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Free-form image inpainting with gated convolution. In: Proc. IEEE/CVF International Conference on Computer Vision. pp. 4471–4480 (2019)
Zhang, J., Fukuda, T., Yabuki, N.: Automatic object removal with obstructed façades completion using semantic segmentation and generative adversarial inpainting. IEEE Access 9, 117486–117495 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kottler, B., Qiu, K., Häufel, G., Bulatov, D. (2025). A Stratified Pipeline for Vehicle Inpainting in Orthophotos. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15322. Springer, Cham. https://doi.org/10.1007/978-3-031-78312-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-78312-8_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78311-1
Online ISBN: 978-3-031-78312-8
eBook Packages: Computer ScienceComputer Science (R0)