Abstract
Fine particulate matter (PM\(_{2.5}\)) poses a significant public health risk due to its association with respiratory and cardiovascular diseases. Given the limited availability of PM\(_{2.5}\) monitoring stations, there is a pressing need for reliable real-time forecasting models. This study introduces TSPPM25, a novel Transformer-based model designed for spatiotemporal prediction of PM\(_{2.5}\) levels. TSPPM25 leverages multiple data embedding techniques and various attention layers to effectively capture the intricate spatiotemporal relationships in multivariate data. The model’s performance is evaluated using a California Aerosol Optical Depth PM\(_{2.5}\) dataset and compared with several baseline models, including LSTM, Bi-LSTM, Linear Regression, and basic heuristics models. The results demonstrate that TSPPM25 exhibits superior spatiotemporal learning capabilities, robustness, and stability, outperforming other models across MSE, MAE, and SMAPE metrics. Furthermore, the study explores the underlying patterns in PM\(_{2.5}\) data through harmonic analysis, revealing that TSPPM25 performs exceptionally well even in complex scenarios characterized by mixed wave patterns. Conclusively, TSPPM25 emerges as a valuable tool for predicting PM\(_{2.5}\) levels demonstrating its efficacy in the California region, and thereby contributing significantly to the field of air quality forecasting.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The datasets generated during and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Code availibility
The code generated during and/or analysed during the current study is not publicly available but is available from the corresponding author on reasonable request.
References
Brokamp C, Jandarov R, Rao M et al (2017) Exposure assessment models for elemental components of particulate matter in an urban environment: a comparison of regression and random forest approaches. Atmos Environ 151:1–11. https://doi.org/10.1016/j.atmosenv.2016.11.066
Chu Y, Liu Y, Li X et al (2016) A review on predicting ground pm2.5 concentration using satellite aerosol optical depth. Atmosphere 7(10):129. https://doi.org/10.3390/atmos7100129
Chylek P, Henderson BG, Lesins G (2005) Aerosol optical depth retrieval over the nasa stennis space center: Mti, modis, and aeronet. IEEE Trans Geosci Remote Sens 43(9):1978–1983. https://doi.org/10.1109/TGRS.2005.851773
Devlin J, Chang MW, Lee K et al (2018) Bert: pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.48550/arXiv.1810.04805. arXiv:1810.04805
Di Q, Kloog I, Koutrakis P et al (2016) Assessing pm2.5 exposures with high spatiotemporal resolution across the continental united states. Environ Sci Technol 50(9):4712–4721. https://doi.org/10.1021/acs.est.5b06121
Dockery DW, Pope CA, Xu X et al (1993) An association between air pollution and mortality in six us cities. New England J Med 329(24):1753–1759. https://doi.org/10.1056/NEJM199312093292401
EPA (2006) Air quality guidelines: global update 2005: particulate matter, ozone, nitrogen dioxide, and sulfur dioxide. World Health Organization
Fan J, Li S, Fan C et al (2016) The impact of pm2.5 on asthma emergency department visits: a systematic review and meta-analysis. Environ Sci Pollut Res 23(1):843–850. https://doi.org/10.1007/s11356-015-5321-x
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Holben BN, Eck TF, Ia Slutsker et al (1998) Aeronet-a federated instrument network and data archive for aerosol characterization. Remote Sens Environ 66(1):1–16. https://doi.org/10.1016/S0034-4257(98)00031-5
Holben BN, Tanre D, Smirnov A et al (2001) An emerging ground-based aerosol climatology: aerosol optical depth from aeronet. J Geophys Res: Atmos 106(D11):12,067–12,097. https://doi.org/10.1029/2001JD900014
Hu X, Belle JH, Meng X et al (2017) Estimating pm2.5 concentrations in the conterminous united states using the random forest approach. Environ Sci Technol 51(12):6936–6944. https://doi.org/10.1021/acs.est.7b01210
Huang F, Cao Z, Guo J et al (2020) Comparisons of heuristic, general statistical and machine learning models for landslide susceptibility prediction and mapping. Catena 191(104):580. https://doi.org/10.1016/j.catena.2020.104580
Kazemi SM, Goel R, Eghbali S et al (2019) Time2vec: learning a vector representation of time. https://doi.org/10.48550/arXiv.1907.05321. arXiv:1907.05321
Lee KH, Chen X, Hua G et al (2018) Stacked cross attention for image-text matching. In: Proceedings of the european conference on computer vision (ECCV), pp 201–216, https://doi.org/10.1007/978-3-030-01225-0_13
Limperis J, Tong W, Hamza-Lup F et al (2023) Pm2.5 forecasting based on transformer neural network and data embedding. Earth Sci Inf 1–14. https://doi.org/10.1007/s12145-023-01002-x
Martin RV, Brauer M, van Donkelaar A et al (2019) No one knows which city has the highest concentration of fine particulate matter. Atmospheric Environ: X 3(100):040. https://doi.org/10.1016/j.aeaoa.2019.100040
Meng X, Hand JL, Schichtel BA et al (2018) Space-time trends of pm2.5 constituents in the conterminous united states estimated by a machine learning approach, 2005–2015. Environ Int 121:1137–1147. https://doi.org/10.1016/j.envint.2018.10.029
Morain SA, Budge AM (2012) Environmental tracking for public health surveillance. CRC Press. https://doi.org/10.1201/b12680
Nussbaumer HJ (1981) The fast fourier transform. In: Fast fourier transform and convolution algorithms. Springer, pp 80–111. https://doi.org/10.1007/978-3-662-00551-4_4
Oord Avd, Dieleman S, Zen H et al (2016) Wavenet: a generative model for raw audio. arXiv:1609.03499. https://doi.org/10.48550/arXiv.1609.03499
Qi Y, Li Q, Karimian H et al (2019) A hybrid model for spatiotemporal forecasting of pm2. 5 based on graph convolutional neural network and long short-term memory. Sci Total Environ 664:1–10. https://doi.org/10.1016/j.scitotenv.2019.01.333
Samet JM, Dominici F, Curriero FC et al (2000) Fine particulate air pollution and mortality in 20 us cities, 1987–1994. New England J Med 343(24):1742–1749. https://doi.org/10.1056/NEJM200012143432401
Savtchenko A, Ouzounov D, Ahmad S et al (2004) Terra and aqua modis products available from nasa ges daac. Adv Space Res 34(4):710–714. https://doi.org/10.1016/j.asr.2004.03.012
Schliep E, Gelfand A, Holland D (2015) Autoregressive spatially varying coefficients model for predicting daily pm2.5 using viirs satellite aot. Adv Stat Climatol Meteorol Oceanograph 1(1):59–74. https://doi.org/10.5194/ascmo-1-59-2015
Shen S, Yao Z, Gholami A et al (2020) Powernorm: rethinking batch normalization in transformers. In: International conference on machine learning, pp 8741–8751. https://doi.org/10.48550/arXiv.2003.07845
Stein EM, Shakarchi R (2011) Fourier analysis: an introduction, vol 1. Princeton University Press
Streets DG, Canty T, Carmichael GR et al (2013) Emissions estimation from satellite retrievals: a review of current capability. Atmosph Environ 77:1011–1042. https://doi.org/10.1016/j.atmosenv.2013.05.051
Sun W, Zhang H, Palazoglu A et al (2013) Prediction of 24-hour-average pm2.5 concentrations using a hidden markov model with different emission distributions in northern California. Sci Total Environ 443:93–103. https://doi.org/10.1016/j.scitotenv.2012.10.070
Tobler WR (1970) A computer movie simulating urban growth in the Detroit region. Economic Geograph 46:234–240. https://doi.org/10.2307/143141
Tong W, Li L, Zhou X et al (2019) Efficient spatiotemporal interpolation with spark machine learning. Earth Sci Inf 12:87–96. https://doi.org/10.1007/s12145-018-0364-4
Tong W, Li L, Zhou X et al (2019) Deep learning pm2.5 concentrations with bidirectional lstm rnn. Air Q Atmos Health 12:411–423. https://doi.org/10.1007/s11869-018-0647-4
Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008. https://doi.org/10.5555/3295222.3295349
Xi X, Wei Z, Xiaoguang R et al (2015) A comprehensive evaluation of air pollution prediction improvement by a machine learning method. In: 2015 IEEE international conference on service operations and logistics, and informatics (SOLI), IEEE, pp 176–181. https://doi.org/10.1109/SOLI.2015.7367615
Xing YF, Xu YH, Shi MH et al (2016) The impact of pm2.5 on the human respiratory system. J Thoracic Disease 8(1):E69. https://doi.org/10.3978/j.issn.2072-1439.2016.01.19
Zerveas G, Jayaraman S, Patel D et al (2021) A transformer-based framework for multivariate time series representation learning. In: The 27th ACM SIGKDD conference on knowledge discovery & data mining, pp 2114–2124. https://doi.org/10.1145/3447548.3467401
Zhou X, Tong W, Li L (2020) Deep learning spatiotemporal air pollution data in china using data fusion. Earth Sci Inf 13:859–868. https://doi.org/10.1007/s12145-020-00470-9
Acknowledgements
All authors are supported by the Office of Research, Georgia Southern University.
Funding
All authors are supported by the Office of Research, Georgia Southern University.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Data collection, experiments, and analysis were performed by Jordan Limperis and Weitian Tong. The first draft of the manuscript was written by Jordan Limperis and Weitian Tong. All authors (i.e., Jordan Limperis, Weitian Tong, Felix Hamza-Lup, Yao Xu, and Lixin Li) commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval
Not applicable
Consent to participate
Not applicable
Consent for publication
Not applicable
Competing interests
The authors declare no competing interests.
Additional information
Communicated by: H. Babaie.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Tong, W., Limperis, J., Hamza-Lup, F. et al. Robust Transformer-based model for spatiotemporal PM\(_{2.5}\) prediction in California. Earth Sci Inform 17, 315–328 (2024). https://doi.org/10.1007/s12145-023-01138-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-023-01138-w