Skip to main content
SpringerLink
Log in

Holistically-Nested Structure-Aware Graph Neural Network for Road Extraction

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13017))

Included in the following conference series:

  • 1285 Accesses

Abstract

Convolutional neural networks (CNN) have made significant advances in detecting roads from satellite images. However, existing CNN approaches are generally repurposed semantic segmentation architectures and suffer from the poor delineation of long and curved regions. Lack of overall road topology and structure information further deteriorates their performance on challenging remote sensing images. This paper presents a novel multi-task graph neural network (GNN) which simultaneously detects both road regions and road borders; the inter-play between these two tasks unlocks superior performance from two perspectives: (1) the hierarchically detected road borders enable the network to capture and encode holistic road structure to enhance road connectivity (2) identifying the intrinsic correlation of semantic landcover regions mitigates the difficulty in recognizing roads cluttered by regions with similar appearance. Experiments on challenging dataset demonstrate that the proposed architecture can improve the road border delineation and road extraction accuracy compared with the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alshehhi, R., Marpu, P.R.: Hierarchical graph-based segmentation for extracting road networks from high-resolution satellite images. ISPRS J. Photogram. Remote Sens. 126, 245–260 (2017)

    Article  Google Scholar 

  2. Chen, L., Zhu, Q., Xie, X., Hu, H., Zeng, H.: Road extraction from VHR remote-sensing imagery via object segmentation constrained by Gabor features. ISPRS Int. J. Geo-Information 7(9), 362 (2018)

    Article  Google Scholar 

  3. Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)

  4. Das, S., Mirnalinee, T.T., Varghese, K.: Use of salient features for the design of a multistage framework to extract roads from high-resolution multispectral satellite images. IEEE TGRS 49(10), 3906–3931 (2011). https://doi.org/10.1109/TGRS.2011.2136381

    Article  Google Scholar 

  5. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59(2), 167–181 (2004). https://doi.org/10.1023/B:VISI.0000022288.19776.77

    Article  Google Scholar 

  6. Gao, L., Song, W., Dai, J., Chen, Y.: Road extraction from high-resolution remote sensing imagery using refined deep residual convolutional neural network. Remote Sens. 11(5), 552 (2019)

    Article  Google Scholar 

  7. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  8. Jia, J., et al.: Tradeoffs in the spatial and spectral resolution of airborne hyperspectral imaging systems crop identification case study. IEEE TGRS 1–18 (2021)

    Google Scholar 

  9. Jia, J., et al.: Road extraction technology based on multi-source remote sensing data: Review and prospects. Opt. Precis. Eng. 29

    Google Scholar 

  10. Kestur, R., Farooq, S., Abdal, R., Mehraj, E., Narasipura, O.S., Mudigere, M.: UFCN: a fully convolutional neural network for road extraction in RGB imagery acquired by remote sensing from an unmanned aerial vehicle. J. Appl. Remote Sens. 12(1), 016020 (2018)

    Article  Google Scholar 

  11. Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI (2018)

    Google Scholar 

  12. Liu, Y., Yao, J., Lu, X., Xia, M., Wang, X., Liu, Y.: RoadNet: learning to comprehensively analyze road networks in complex urban scenes from high-resolution remotely sensed images. IEEE TGRS 57(4), 2043–2056 (2018)

    Google Scholar 

  13. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015)

    Google Scholar 

  14. Lu, X., et al.: Multi-scale and multi-task deep learning framework for automatic road extraction. IEEE TGRS 57(11), 9362–9377 (2019)

    Google Scholar 

  15. Mnih, V.: Machine learning for aerial image labeling. University of Toronto, Canada (2013)

    Google Scholar 

  16. Mnih, V., Hinton, G.E.: Learning to detect roads in high-resolution aerial images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 210–223. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15567-3_16

    Chapter  Google Scholar 

  17. Oner, D., Koziński, M., Citraro, L., Dadap, N.C., Konings, A.G., Fua, P.: Promoting connectivity of network-like structures by enforcing region separation. arXiv preprint arXiv:2009.07011 (2020)

  18. Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., Sorkine-Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: CVPR, pp. 724–732 (2016)

    Google Scholar 

  19. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  20. Saito, S., Yamashita, T., Aoki, Y.: Multiple object extraction from aerial imagery with convolutional neural networks. Electron. Imaging 2016(10), 1–9 (2016)

    Article  Google Scholar 

  21. Shao, Z., Zhou, Z., Huang, X., Zhang, Y.: MRENet: simultaneous extraction of road surface and road centerline in complex urban scenes from very high-resolution images. Remote Sens. 13(2), 239 (2021)

    Article  Google Scholar 

  22. Song, M., Civco, D.: Road extraction using SVM and image segmentation. Photogram. Eng. Remote Sens. 70(12), 1365–1371 (2004)

    Article  Google Scholar 

  23. Wang, H.: Spectral graph reasoning network for hyperspectral image classification. In: Farkaš, I., Masulli, P., Wermter, S. (eds.) ICANN 2020, Part I. LNCS, vol. 12396, pp. 711–723. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61609-0_56

    Chapter  Google Scholar 

  24. Wang, H., Raiko, T., Lensu, L., Wang, T., Karhunen, J.: Semi-supervised domain adaptation for weakly labeled semantic video object segmentation. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016, Part I. LNCS, vol. 10111, pp. 163–179. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54181-5_11

    Chapter  Google Scholar 

  25. Wang, H., Wang, T.: Boosting objectness: semi-supervised learning for object detection and segmentation in multi-view images. In: ICASSP, pp. 1796–1800 (2016)

    Google Scholar 

  26. Wang, H., Wang, T.: Primary object discovery and segmentation in videos via graph-based transductive inference. Comput. Vis. Image Underst. 143, 159–172 (2016)

    Article  Google Scholar 

  27. Wang, H., Wang, T., Chen, K., Kämäräinen, J.K.: Cross-granularity graph inference for semantic video object segmentation. In: IJCAI, pp. 4544–4550 (2017)

    Google Scholar 

  28. Wang, S., Yang, H., Wu, Q., Zheng, Z., Wu, Y., Li, J.: An improved method for road extraction from high-resolution remote-sensing images that enhances boundary information. Sensors 20(7), 2064 (2020)

    Article  Google Scholar 

  29. Wang, T., Wang, G., Tan, K.E., Tan, D., et al.: Hyperspectral image classification via pyramid graph reasoning. In: Bebis, G. (ed.) ISVC 2020, Part I. LNCS, vol. 12509, pp. 707–718. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64556-4_55

    Chapter  Google Scholar 

  30. Wang, T., Wang, H.: Graph transduction learning of object proposals for video object segmentation. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014, Part IV. LNCS, vol. 9006, pp. 553–568. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16817-3_36

    Chapter  Google Scholar 

  31. Waswani, A., et al.: Attention is all you need. In: NIPS (2017)

    Google Scholar 

  32. Xie, S., Tu, Z.: Holistically-nested edge detection. In: ICCV,p. 1395–1403 (2015)

    Google Scholar 

  33. Xu, Y., Xie, Z., Feng, Y., Chen, Z.: Road extraction from high-resolution remote sensing imagery using deep learning. Remote Sens. 10(9), 1461 (2018)

    Article  Google Scholar 

  34. Yang, X., Li, X., Ye, Y., Lau, R.Y., Zhang, X., Huang, X.: Road detection and centerline extraction via deep recurrent convolutional neural network u-net. IEEE TGRS 57(9), 7209–7220 (2019)

    Google Scholar 

  35. Zhang, Z., Liu, Q., Wang, Y.: Road extraction by deep residual U-Net. IEEE Geosci. Remote Sens. Lett. 15(5), 749–753 (2018)

    Article  Google Scholar 

  36. Zhou, L., Zhang, C., Wu, M.: D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. In: CVPR Workshops, pp. 182–186 (2018)

    Google Scholar 

  37. Zhu, L., Wang, T., Aksu, E., Kamarainen, J.: Portrait instance segmentation for mobile devices. In: ICME, pp. 1630–1635 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Huawei Helsinki Research Center, Helsinki, Finland

    Tinghuai Wang, Guangming Wang & Kuan Eeik Tan

Authors
  1. Tinghuai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kuan Eeik Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tinghuai Wang .

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, NV, USA

    George Bebis

  2. University of Texas at Arlington, Arlington, TX, USA

    Vassilis Athitsos

  3. University of South Carolina, Columbia, SC, USA

    Tong Yan

  4. City University of Hong Kong, Kowloon, Hong Kong

    Manfred Lau

  5. School of Engineering and Computing, University of Durham, Durham, Durham, UK

    Frederick Li

  6. Airbnb, New York, NY, USA

    Conglei Shi

  7. Peking University, Beijing, China

    Xiaoru Yuan

  8. Purdue University, West Lafayette, IN, USA

    Christos Mousas

  9. IST, School of Modeling, Simulation, and Training, Orlando, FL, USA

    Gerd Bruder

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, T., Wang, G., Tan, K.E. (2021). Holistically-Nested Structure-Aware Graph Neural Network for Road Extraction. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2021. Lecture Notes in Computer Science(), vol 13017. Springer, Cham. https://doi.org/10.1007/978-3-030-90439-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90439-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90438-8

  • Online ISBN: 978-3-030-90439-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation