Skip to main content
Log in

End-to-end learning for image-based air quality level estimation

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Air quality estimation is an important and fundamental problem in environmental protection. Several efforts have been made in the past decades using expensive sensor-based or indirect methods like based on social networks; however, image-based air pollution estimation is still far from solved. This paper devises an effective convolutional neural network (CNN) to estimate air quality based on images. Our method is comprised of three ingredients: We first design an ensemble CNN for air quality estimation which is expected to obtain more accurate and stable results than a single classifier. Second, three ordinal classifiers, namely negative log–log ordinal classifier, cauchit ordinal classifier and complementary log–log ordinal classifier, are devised in the last layer of each CNN, to improve the ordinal discriminative ability of the model. Third, as a variant of the rectified linear units, an adjusted activation function is introduced. We collect open air images with corresponding air quality levels from an official agency as the ground truth. Experimental results demonstrate the effectiveness of our method on the real-world dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. The preliminary version of this paper has appeared in the conference paper [62]. We make several extensions including: (1) an updated introduction and related work review on the recent development for image-based air quality estimation; (2) a new ensemble-based model as well as new variants of the baseline devised in [62] is proposed for air quality estimation; (3) updated and more comprehensive experimental results are reported with various ablation tests. Concurrent to [62], CNN-based model for quality.

  2. FNReLU-CNN-Negative in fact is the method presented in the conference version of this paper [62], whereby it is titled as PAPLE for shot.

References

  1. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  2. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MATH  Google Scholar 

  3. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  4. Cai, X., Song, B.: Image-based pencil drawing synthesized using convolutional neural network feature maps. Mach. Vision Appl. 299, 1–10 (2018)

    Google Scholar 

  5. Chen, J., Chen, H., Zheng, G., Pan, J.Z., Wu, H., Zhang, N.: Big smog meets web science: smog disaster analysis based on social media and device data on the web. In: Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pp. 505–510. International World Wide Web Conferences Steering Committee (2014)

  6. Chung, Y.S.: Air pollution detection by satellites: the transport and deposition of air pollutants over oceans. Atmos. Environ. (1967) 20(4), 617–630 (1986)

    Article  Google Scholar 

  7. Clench-Aas, J., Bartnova, A., Bøhler, T., Grønskei, K.E., Sivertson, B., Larssen, S.: Air pollution exposure monitoring and estimating. Part I. Integrated air quality monitoring system. J. Environ. Monit. 1(4), 313–319 (1999)

    Article  Google Scholar 

  8. Djork-Arné C., Thomas U., Sepp H.: Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 (2015)

  9. Delle Monache, L., Stull, R.B.: An ensemble air-quality forecast over western Europe during an ozone episode. Atmos. Environ. 37(25), 3469–3474 (2003)

    Article  Google Scholar 

  10. Elhoseiny, M., Huang, S., Elgammal, A.: Weather classification with deep convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 3349–3353 (2015)

  11. Freund, Y., Schapire, R.E., et al.: Experiments with a new boosting algorithm. ICML 96, 148–156 (1996)

    Google Scholar 

  12. Gandhi I., Pandey, M.: Hybrid ensemble of classifiers using voting. In: 2015 International Conference on Green Computing and Internet of Things (ICGCIoT), IEEE, pp. 399–404 (2015)

  13. Greenland, S.: Alternative models for ordinal logistic regression. Stat. Med. 13(16), 1665–1677 (1994)

    Article  Google Scholar 

  14. Han, J., Kamber, M., Pei, J.: Data Mining Concepts and Techniques, 3rd edn. Morgan Kaufmann Publishers, Waltham (2012)

    MATH  Google Scholar 

  15. Hauck, H., Berner, A., Gomiscek, B., Stopper, S., Puxbaum, H., Kundi, M., Preining, O.: On the equivalence of gravimetric PM data with teom and beta-attenuation measurements. J. Aerosol Sci. 35(9), 1135–1149 (2004)

    Article  Google Scholar 

  16. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 2341–2353 (2011)

    Article  Google Scholar 

  17. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)

    Article  Google Scholar 

  18. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)

  19. Hinton, G.E., Salakhutdinov, R.R.: Replicated Softmax: an undirected topic model. In: Advances in Neural Information Processing Systems, pp. 1607–1614 (2009)

  20. Hodgeson, J.A., McClenny, W.A., Hanst, P.L.: Air pollution monitoring by advanced spectroscopic techniques a variety of spectroscopic methods are being used to detect air pollutants in the gas phase. Science 182(4109), 248–258 (1973)

    Article  Google Scholar 

  21. http://210.72.1.216:8080/gzaqi/Document/gjzlbz.pdf. Accessed 19 April 2017

  22. http://zx.bjmemc.com.cn/. Accessed 19 April 2017

  23. Jarvelin, K., Jaana, K.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20, 2002 (2002)

    Article  Google Scholar 

  24. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)

  25. Jiang, Y.-G., Zuxuan, W., Wang, J., Xue, X., Chang, S.-F.: Exploiting feature and class relationships in video categorization with regularized deep neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 352–364 (2018)

    Article  Google Scholar 

  26. Jurek, A., Bi, Y., Wu, S., Nugent, C.D.: Clustering-based ensembles as an alternative to stacking. TKDE 26(9), 2120–2137 (2014)

    Google Scholar 

  27. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)

  28. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  29. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  30. Li, C., Liu, Q., Liu, J., Lu, H.: Learning ordinal discriminative features for age estimation. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2570–2577 (2012)

  31. Li, C., Liu, Q., Liu, J., Hanqing, L.: Ordinal distance metric learning for image ranking. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1551–1559 (2015)

    Article  MathSciNet  Google Scholar 

  32. Li, Y., Zhou, Y., Yan, J., Yang, J., He, X.: Tensor error correction for corrupted values in visual data. In: 2010 IEEE International Conference on Image Processing, IEEE, pp. 2321–2324 (2010)

  33. Li, Y., Huang, J., Luo, J.: Using user generated online photos to estimate and monitor air pollution in major cities. In: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service, ACM, p. 79 (2015)

  34. Liu, C., Tsow, F., Zou, Y., Tao, N.: Particle pollution estimation based on image analysis. PLoS ONE 11(2), e0145955 (2016)

    Article  Google Scholar 

  35. Liu, F., Shen, C., Lin, G., Reid, I.: Learning depth from single monocular images using deep convolutional neural fields. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2024–2039 (2016)

    Article  Google Scholar 

  36. Liu, Y., Racah, E., Correa, J., Khosrowshahi, A., Lavers, D., Kunkel, K., Wehner, M., Collins, W. et al.: Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv preprint arXiv:1605.01156 (2016)

  37. Lu, C., Lin, D., Jia, J., Tang, C.-K.: Two-class weather classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3718–3725 (2014)

  38. Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (2015)

  39. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, p. 1 (2013)

  40. Mao, J., Phommasak, U., Watanabe, S., Shioya, H.: Detecting foggy images and estimating the haze degree factor. J. Comput. Sci. Syst. Biol. 7, 1 (2014)

    Article  Google Scholar 

  41. Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artif. Intell. Rev. 42(2), 275–293 (2014)

    Article  Google Scholar 

  42. Mei, S., Li, H., Fan, J., Zhu, X., Dyer, C.R.: Inferring air pollution by sniffing social media. In: 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), IEEE, pp. 534–539 (2014)

  43. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)

  44. Narasimhan, S., Nayar, S.: Vision and the atmosphere. Int. J. Comput. Vision 48, 233 (2001)

    Article  MATH  Google Scholar 

  45. Nelder, J.A., Baker, R.J.: Generalized Linear Models. Encyclopedia of Statistical Sciences. Wiley, New York (1972)

    Google Scholar 

  46. Pope III, C.A., Dockery, D.W.: Health effects of fine particulate air pollution: lines that connect. J. Air Waste Manag. Assoc. 56(6), 709–742 (2006)

    Article  Google Scholar 

  47. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS (2015)

  48. Ren, Y., Zhang, L., Suganthan, P.N.: Ensemble classification and regression-recent developments, applications and future directions. IEEE Comput. Int. Mag. 11(1), 41–53 (2016)

    Article  Google Scholar 

  49. Ren, Z., Yan, J., Ni, B., Liu, B., Yang, X., Zha, H.: Unsupervised deep learning for optical flow estimation. In: AAAI, pp. 1495–1501 (2017)

  50. Smith, J.D., Atkinson, D.B.: A portable pulsed cavity ring-down transmissometer for measurement of the optical extinction of the atmospheric aerosol. Analyst 126(8), 1216–1220 (2001)

    Article  Google Scholar 

  51. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  52. Stein, O., Flemming, J., Inness, A., Kaiser, J.W., Schultz, M.G.: Global reactive gases forecasts and reanalysis in the MACC project. J. Integr. Environ. Sci. 9(sup1), 57–70 (2012)

    Article  Google Scholar 

  53. Tarel, J.-P., Hautiere, N., Caraffa, L., Cord, A., Halmaoui, H., Gruyer, D.: Vision enhancement in homogeneous and heterogeneous fog. IEEE Intell. Transp. Syst. Mag. 4(2), 6–20 (2012)

    Article  Google Scholar 

  54. Vautard, R., Schaap, M., Bergström, R., Bessagnet, B., Brandt, J., Builtjes, P.J.H., Christensen, J.H., Cuvelier, C., Foltescu, V., Graff, A., et al.: Skill and uncertainty of a regional air quality model ensemble. Atmos. Environ. 43(31), 4822–4832 (2009)

    Article  Google Scholar 

  55. Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992)

    Article  Google Scholar 

  56. Wu, H., Miao, Z., Wang, Y., Chen, J., Ma, C., Zhou, T.: Image completion with multi-image based on entropy reduction. Neurocomputing 159, 157–171 (2015)

    Article  Google Scholar 

  57. Wu, Z., Jiang, Y.-G., Wang, J., Pu, J., Xue, X.: Exploring inter-feature and inter-class relationships with deep neural networks for video classification. In: Proceedings of the 22nd ACM International Conference on Multimedia, ACM, pp. 167–176 (2014)

  58. Wu, Z., Jiang, Y.-G., Wang, X., Ye, H., Xue, X.: Multi-stream multi-class fusion of deep networks for video classification. In: Proceedings of the 2016 ACM on Multimedia Conference, ACM, pp. 791–800 (2016)

  59. Yan, J., Zhu, M., Liu, H., Liu, Y.: Visual saliency detection via sparsity pursuit. IEEE Signal Process. Lett. 17(8), 739–742 (2010)

    Article  Google Scholar 

  60. Yu, D., Ning, L., Zou, Y., Jiguo, Y., Cheng, X., Lau, F.: Distributed spanner construction with physical interference: constant stretch and linear sparseness. IEEE/ACM Trans. Netw. (TON) 25(4), 2138–2151 (2017)

    Article  Google Scholar 

  61. Yu, J., Huang, B., Cheng, X., Atiquzzaman, M.: Shortest link scheduling algorithms in wireless networks under the SINR model. IEEE Trans. Veh. Technol. 66(3), 2643–2657 (2017)

    Article  Google Scholar 

  62. Zhang, C., Yan, J., Li, C., Rui, X., Liu, L., Bie, R.: On estimating air pollution from photos using convolutional neural network. In: Proceedings of the 2016 ACM on Multimedia Conference, ACM, pp. 297–301 (2016)

  63. Zhao, R.-W., Wu, Z., Li, J., Jiang, Y.-G.: Learning semantic feature map for visual content recognition. In: Proceedings of the 2017 ACM on Multimedia Conference, ACM, pp. 1291–1299 (2017)

Download references

Acknowledgements

This research is partially sponsored by National Natural Science Foundation of China (Nos. 61571049, 61601033, 61401029, 11401028, 61472044, 61472403) and the Fundamental Research Funds for the Central Universities (No. 2016NT14). The authors are thankful to the anonymous reviewers for valuable discussion and feedback.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hao Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Yan, J., Li, C. et al. End-to-end learning for image-based air quality level estimation. Machine Vision and Applications 29, 601–615 (2018). https://doi.org/10.1007/s00138-018-0919-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-018-0919-x

Keywords

Navigation