Abstract
Air quality estimation is an important and fundamental problem in environmental protection. Several efforts have been made in the past decades using expensive sensor-based or indirect methods like based on social networks; however, image-based air pollution estimation is still far from solved. This paper devises an effective convolutional neural network (CNN) to estimate air quality based on images. Our method is comprised of three ingredients: We first design an ensemble CNN for air quality estimation which is expected to obtain more accurate and stable results than a single classifier. Second, three ordinal classifiers, namely negative log–log ordinal classifier, cauchit ordinal classifier and complementary log–log ordinal classifier, are devised in the last layer of each CNN, to improve the ordinal discriminative ability of the model. Third, as a variant of the rectified linear units, an adjusted activation function is introduced. We collect open air images with corresponding air quality levels from an official agency as the ground truth. Experimental results demonstrate the effectiveness of our method on the real-world dataset.










Similar content being viewed by others
Notes
The preliminary version of this paper has appeared in the conference paper [62]. We make several extensions including: (1) an updated introduction and related work review on the recent development for image-based air quality estimation; (2) a new ensemble-based model as well as new variants of the baseline devised in [62] is proposed for air quality estimation; (3) updated and more comprehensive experimental results are reported with various ablation tests. Concurrent to [62], CNN-based model for quality.
FNReLU-CNN-Negative in fact is the method presented in the conference version of this paper [62], whereby it is titled as PAPLE for shot.
References
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Cai, X., Song, B.: Image-based pencil drawing synthesized using convolutional neural network feature maps. Mach. Vision Appl. 299, 1–10 (2018)
Chen, J., Chen, H., Zheng, G., Pan, J.Z., Wu, H., Zhang, N.: Big smog meets web science: smog disaster analysis based on social media and device data on the web. In: Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pp. 505–510. International World Wide Web Conferences Steering Committee (2014)
Chung, Y.S.: Air pollution detection by satellites: the transport and deposition of air pollutants over oceans. Atmos. Environ. (1967) 20(4), 617–630 (1986)
Clench-Aas, J., Bartnova, A., Bøhler, T., Grønskei, K.E., Sivertson, B., Larssen, S.: Air pollution exposure monitoring and estimating. Part I. Integrated air quality monitoring system. J. Environ. Monit. 1(4), 313–319 (1999)
Djork-Arné C., Thomas U., Sepp H.: Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 (2015)
Delle Monache, L., Stull, R.B.: An ensemble air-quality forecast over western Europe during an ozone episode. Atmos. Environ. 37(25), 3469–3474 (2003)
Elhoseiny, M., Huang, S., Elgammal, A.: Weather classification with deep convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 3349–3353 (2015)
Freund, Y., Schapire, R.E., et al.: Experiments with a new boosting algorithm. ICML 96, 148–156 (1996)
Gandhi I., Pandey, M.: Hybrid ensemble of classifiers using voting. In: 2015 International Conference on Green Computing and Internet of Things (ICGCIoT), IEEE, pp. 399–404 (2015)
Greenland, S.: Alternative models for ordinal logistic regression. Stat. Med. 13(16), 1665–1677 (1994)
Han, J., Kamber, M., Pei, J.: Data Mining Concepts and Techniques, 3rd edn. Morgan Kaufmann Publishers, Waltham (2012)
Hauck, H., Berner, A., Gomiscek, B., Stopper, S., Puxbaum, H., Kundi, M., Preining, O.: On the equivalence of gravimetric PM data with teom and beta-attenuation measurements. J. Aerosol Sci. 35(9), 1135–1149 (2004)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 2341–2353 (2011)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)
Hinton, G.E., Salakhutdinov, R.R.: Replicated Softmax: an undirected topic model. In: Advances in Neural Information Processing Systems, pp. 1607–1614 (2009)
Hodgeson, J.A., McClenny, W.A., Hanst, P.L.: Air pollution monitoring by advanced spectroscopic techniques a variety of spectroscopic methods are being used to detect air pollutants in the gas phase. Science 182(4109), 248–258 (1973)
http://210.72.1.216:8080/gzaqi/Document/gjzlbz.pdf. Accessed 19 April 2017
http://zx.bjmemc.com.cn/. Accessed 19 April 2017
Jarvelin, K., Jaana, K.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20, 2002 (2002)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)
Jiang, Y.-G., Zuxuan, W., Wang, J., Xue, X., Chang, S.-F.: Exploiting feature and class relationships in video categorization with regularized deep neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 352–364 (2018)
Jurek, A., Bi, Y., Wu, S., Nugent, C.D.: Clustering-based ensembles as an alternative to stacking. TKDE 26(9), 2120–2137 (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Li, C., Liu, Q., Liu, J., Lu, H.: Learning ordinal discriminative features for age estimation. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2570–2577 (2012)
Li, C., Liu, Q., Liu, J., Hanqing, L.: Ordinal distance metric learning for image ranking. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1551–1559 (2015)
Li, Y., Zhou, Y., Yan, J., Yang, J., He, X.: Tensor error correction for corrupted values in visual data. In: 2010 IEEE International Conference on Image Processing, IEEE, pp. 2321–2324 (2010)
Li, Y., Huang, J., Luo, J.: Using user generated online photos to estimate and monitor air pollution in major cities. In: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service, ACM, p. 79 (2015)
Liu, C., Tsow, F., Zou, Y., Tao, N.: Particle pollution estimation based on image analysis. PLoS ONE 11(2), e0145955 (2016)
Liu, F., Shen, C., Lin, G., Reid, I.: Learning depth from single monocular images using deep convolutional neural fields. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2024–2039 (2016)
Liu, Y., Racah, E., Correa, J., Khosrowshahi, A., Lavers, D., Kunkel, K., Wehner, M., Collins, W. et al.: Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv preprint arXiv:1605.01156 (2016)
Lu, C., Lin, D., Jia, J., Tang, C.-K.: Two-class weather classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3718–3725 (2014)
Ma, C., Huang, J.-B., Yang, X., Yang, M.-H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (2015)
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, p. 1 (2013)
Mao, J., Phommasak, U., Watanabe, S., Shioya, H.: Detecting foggy images and estimating the haze degree factor. J. Comput. Sci. Syst. Biol. 7, 1 (2014)
Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artif. Intell. Rev. 42(2), 275–293 (2014)
Mei, S., Li, H., Fan, J., Zhu, X., Dyer, C.R.: Inferring air pollution by sniffing social media. In: 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), IEEE, pp. 534–539 (2014)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)
Narasimhan, S., Nayar, S.: Vision and the atmosphere. Int. J. Comput. Vision 48, 233 (2001)
Nelder, J.A., Baker, R.J.: Generalized Linear Models. Encyclopedia of Statistical Sciences. Wiley, New York (1972)
Pope III, C.A., Dockery, D.W.: Health effects of fine particulate air pollution: lines that connect. J. Air Waste Manag. Assoc. 56(6), 709–742 (2006)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS (2015)
Ren, Y., Zhang, L., Suganthan, P.N.: Ensemble classification and regression-recent developments, applications and future directions. IEEE Comput. Int. Mag. 11(1), 41–53 (2016)
Ren, Z., Yan, J., Ni, B., Liu, B., Yang, X., Zha, H.: Unsupervised deep learning for optical flow estimation. In: AAAI, pp. 1495–1501 (2017)
Smith, J.D., Atkinson, D.B.: A portable pulsed cavity ring-down transmissometer for measurement of the optical extinction of the atmospheric aerosol. Analyst 126(8), 1216–1220 (2001)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Stein, O., Flemming, J., Inness, A., Kaiser, J.W., Schultz, M.G.: Global reactive gases forecasts and reanalysis in the MACC project. J. Integr. Environ. Sci. 9(sup1), 57–70 (2012)
Tarel, J.-P., Hautiere, N., Caraffa, L., Cord, A., Halmaoui, H., Gruyer, D.: Vision enhancement in homogeneous and heterogeneous fog. IEEE Intell. Transp. Syst. Mag. 4(2), 6–20 (2012)
Vautard, R., Schaap, M., Bergström, R., Bessagnet, B., Brandt, J., Builtjes, P.J.H., Christensen, J.H., Cuvelier, C., Foltescu, V., Graff, A., et al.: Skill and uncertainty of a regional air quality model ensemble. Atmos. Environ. 43(31), 4822–4832 (2009)
Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992)
Wu, H., Miao, Z., Wang, Y., Chen, J., Ma, C., Zhou, T.: Image completion with multi-image based on entropy reduction. Neurocomputing 159, 157–171 (2015)
Wu, Z., Jiang, Y.-G., Wang, J., Pu, J., Xue, X.: Exploring inter-feature and inter-class relationships with deep neural networks for video classification. In: Proceedings of the 22nd ACM International Conference on Multimedia, ACM, pp. 167–176 (2014)
Wu, Z., Jiang, Y.-G., Wang, X., Ye, H., Xue, X.: Multi-stream multi-class fusion of deep networks for video classification. In: Proceedings of the 2016 ACM on Multimedia Conference, ACM, pp. 791–800 (2016)
Yan, J., Zhu, M., Liu, H., Liu, Y.: Visual saliency detection via sparsity pursuit. IEEE Signal Process. Lett. 17(8), 739–742 (2010)
Yu, D., Ning, L., Zou, Y., Jiguo, Y., Cheng, X., Lau, F.: Distributed spanner construction with physical interference: constant stretch and linear sparseness. IEEE/ACM Trans. Netw. (TON) 25(4), 2138–2151 (2017)
Yu, J., Huang, B., Cheng, X., Atiquzzaman, M.: Shortest link scheduling algorithms in wireless networks under the SINR model. IEEE Trans. Veh. Technol. 66(3), 2643–2657 (2017)
Zhang, C., Yan, J., Li, C., Rui, X., Liu, L., Bie, R.: On estimating air pollution from photos using convolutional neural network. In: Proceedings of the 2016 ACM on Multimedia Conference, ACM, pp. 297–301 (2016)
Zhao, R.-W., Wu, Z., Li, J., Jiang, Y.-G.: Learning semantic feature map for visual content recognition. In: Proceedings of the 2017 ACM on Multimedia Conference, ACM, pp. 1291–1299 (2017)
Acknowledgements
This research is partially sponsored by National Natural Science Foundation of China (Nos. 61571049, 61601033, 61401029, 11401028, 61472044, 61472403) and the Fundamental Research Funds for the Central Universities (No. 2016NT14). The authors are thankful to the anonymous reviewers for valuable discussion and feedback.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, C., Yan, J., Li, C. et al. End-to-end learning for image-based air quality level estimation. Machine Vision and Applications 29, 601–615 (2018). https://doi.org/10.1007/s00138-018-0919-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-018-0919-x