Skip to main content
SpringerLink
Log in

DustNet: Attention to Dust

  • Conference paper
  • First Online:
Pattern Recognition (DAGM GCPR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14264))

Included in the following conference series:

  • 95 Accesses

Abstract

Detecting airborne dust in common RGB images is hard. Nevertheless, monitoring airborne dust can greatly contribute to climate protection, environmentally friendly construction, research, and numerous other domains. In order to develop an efficient and robust airborne dust monitoring algorithm, various challenges have to be overcome. Airborne dust may be opaque as well translucent, can vary heavily in density, and its boundaries are fuzzy. Also, dust may be hard to distinguish from other atmospheric phenomena such as fog or clouds. To cover the demand for a performant and reliable approach for monitoring airborne dust, we propose DustNet, a dust density estimation neural network. DustNet exploits attention and convolutional-based feature pyramid structures to combine features from multiple resolution and semantic levels. Furthermore, DustNet utilizes highly aggregated global information features as an adaptive kernel to enrich high-resolution features. In addition to the fusion of local and global features, we also present multiple approaches for the fusion of temporal features from consecutive images. In order to validate our approach, we compare results achieved by our DustNet with those results achieved by methods originating from the crowd-counting and the monocular depth estimation domains on an airborne dust density dataset. Our DustNet outperforms the other approaches and achieves a 2.5% higher accuracy in localizing dust and a 14.4% lower mean absolute error than the second-best approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agarwal, A., Arora, C.: Attention attention everywhere: monocular depth prediction with skip attention. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5861–5870 (2023)

    Google Scholar 

  2. Avvenuti, M., Bongiovanni, M., Ciampi, L., Falchi, F., Gennaro, C., Messina, N.: A spatio-temporal attentive network for video-based crowd counting. In: Proceedings of the 2022 IEEE Symposium on Computers and Communications, pp. 1–6. IEEE (2022)

    Google Scholar 

  3. Bhat, S.F., Alhashim, I., Wonka, P.: AdaBins: depth estimation using adaptive bins. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4009–4018 (2021)

    Google Scholar 

  4. Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accuracy and its posterior distribution. In: Proceedings of the 2010 20th International Conference on Pattern Recognition, pp. 3121–3124. IEEE (2010)

    Google Scholar 

  5. Cheng, B., Choudhuri, A., Misra, I., Kirillov, A., Girdhar, R., Schwing, A.G.: Mask2former for video instance segmentation. arXiv preprint arXiv:2112.10764 (2021)

  6. Cheng, Z.Q., Dai, Q., Li, H., Song, J., Wu, X., Hauptmann, A.G.: Rethinking spatial invariance of convolutional networks for object counting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19638–19648 (2022)

    Google Scholar 

  7. De Silva, A., Ranasinghe, R., Sounthararajah, A., Haghighi, H., Kodikara, J.: A benchmark dataset for binary segmentation and quantification of dust emissions from unsealed roads. Sci. Data 10(1), 14 (2023)

    Article  Google Scholar 

  8. Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transattentions for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  9. Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: Advances in Neural Information Processing Systems, vol. 27 (2014)

    Google Scholar 

  10. Elfwing, S., Uchibe, E., Doya, K.: Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Netw. 107, 3–11 (2018)

    Article  Google Scholar 

  11. Fu, H., Gong, M., Wang, C., Batmanghelich, K., Tao, D.: Deep ordinal regression network for monocular depth estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2002–2011 (2018)

    Google Scholar 

  12. gabort@AdobeStock: (2023). https://www.stock.adobe.com

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  14. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the International Conference on Machine Learning, pp. 448–456 (2015)

    Google Scholar 

  15. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  16. Lee, J., et al.: Machine learning based algorithms for global dust aerosol detection from satellite images: inter-comparisons and evaluation. Remote Sens. 13(3) (2021)

    Google Scholar 

  17. Lee, M., Hwang, S., Park, C., Lee, S.: EdgeConv with attention module for monocular depth estimation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2858–2867 (2022)

    Google Scholar 

  18. Li, X., Chen, S., Hu, X., Yang, J.: Understanding the disharmony between dropout and batch normalization by variance shift. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2682–2690 (2019)

    Google Scholar 

  19. Li, Y., Zhang, X., Chen, D.: CSRNet: dilated convolutional neural networks for understanding the highly congested scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1091–1100 (2018)

    Google Scholar 

  20. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

    Google Scholar 

  21. Liu, W., Salzmann, M., Fua, P.: Context-aware crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5099–5108 (2019)

    Google Scholar 

  22. Liu, Z., et al.: Swin transattention attention v2: scaling up capacity and resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12009–12019 (2022)

    Google Scholar 

  23. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transattention attention: hierarchical vision transattention attention using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

    Google Scholar 

  24. Liu, Z., et al.: Video swin transattention attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3202–3211 (2022)

    Google Scholar 

  25. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

  26. Luo, A., et al.: Hybrid graph neural networks for crowd counting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11693–11700 (2020)

    Google Scholar 

  27. Michel, A., Weinmann, M., et al.: Terrestrial visual dust density estimation based on deep learning. In: Proceedings of the 2023 IEEE International Geoscience and Remote Sensing Symposium (2023)

    Google Scholar 

  28. Patil, V., Sakaridis, C., Liniger, A., Van Gool, L.: P3Depth: monocular depth estimation with a piecewise planarity prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1610–1621 (2022)

    Google Scholar 

  29. Ren, J., Zhang, M., Yu, C., Liu, Z.: Balanced MSE for imbalanced visual regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7926–7935 (2022)

    Google Scholar 

  30. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  31. Sam, D.B., Surya, S., Babu, R.V.: Switching convolutional neural network for crowd counting. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 4031–4039. IEEE (2017)

    Google Scholar 

  32. Song, M., Lim, S., Kim, W.: Monocular depth estimation using laplacian pyramid-based depth residuals. IEEE Trans. Circuits Syst. Video Technol. 31(11), 4381–4393 (2021)

    Article  Google Scholar 

  33. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  Google Scholar 

  34. Su, H., Jampani, V., Sun, D., Gallo, O., Learned-Miller, E., Kautz, J.: Pixel-adaptive convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11166–11175 (2019)

    Google Scholar 

  35. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  36. Wang, L., Zhang, J., Wang, Y., Lu, H., Ruan, X.: CLIFFNet for monocular depth estimation with hierarchical embedding loss. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part V. LNCS, vol. 12350, pp. 316–331. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58558-7_19

    Chapter  Google Scholar 

  37. Yuan, F., Zhang, L., Xia, X., Huang, Q., Li, X.: A wave-shaped deep neural network for smoke density estimation. IEEE Trans. Image Process. 29, 2301–2313 (2020)

    Article  Google Scholar 

  38. Yuan, W., Gu, X., Dai, Z., Zhu, S., Tan, P.: Neural window fully-connected CRFs for monocular depth estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3916–3925 (2022)

    Google Scholar 

  39. Zhang, Y., Zhou, D., Chen, S., Gao, S., Ma, Y.: Single-image crowd counting via multi-column convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 589–597 (2016)

    Google Scholar 

  40. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)

    Google Scholar 

  41. Zhu, X.X., et al.: Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geosci. Remote Sens. Mag. 5(4), 8–36 (2017)

    Article  Google Scholar 

Download references

Acknowledgment

The images in the presented figures and those used for creating the Meteodata dust dataset are from the pit of Minera Los Pelambres, which collaborates with Meteodata in the advanced use of cameras for emission control strategies. The permission to use the images in this publication is kindly appreciated.

Author information

Authors and Affiliations

  1. Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB, Karlsruhe, Germany

    Andreas Michel, Fabian Schenkel & Wolfgang Middelmann

  2. Institute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, Karlsruhe, Germany

    Andreas Michel, Martin Weinmann, Fabian Schenkel & Stefan Hinz

  3. Meteodata, Santiago, Chile

    Tomas Gomez, Mark Falvey & Rainer Schmitz

Authors
  1. Andreas Michel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Weinmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabian Schenkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tomas Gomez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark Falvey
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rainer Schmitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wolfgang Middelmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stefan Hinz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Michel .

Editor information

Editors and Affiliations

  1. IWR, Heidelberg University, Heidelberg, Germany

    Ullrich Köthe

  2. IWR, Heidelberg University, Heidelberg, Germany

    Carsten Rother

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Michel, A. et al. (2024). DustNet: Attention to Dust. In: Köthe, U., Rother, C. (eds) Pattern Recognition. DAGM GCPR 2023. Lecture Notes in Computer Science, vol 14264. Springer, Cham. https://doi.org/10.1007/978-3-031-54605-1_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54605-1_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54604-4

  • Online ISBN: 978-3-031-54605-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation