Skip to main content

Transformer Model for Multivariate Time Series Classification: A Case Study of Solar Flare Prediction

  • Conference paper
  • First Online:
Pattern Recognition (ICPR 2024)

Abstract

Classifying solar flares is essential for understanding their impact on space weather forecasting. We propose a novel approach using a multi-head attention and transformer mechanism to classify multivariate time series (MVTS) instances of photospheric magnetic field parameters of the flaring events in the solar active regions. Attention mechanisms and transformer architectures capture complex temporal dependencies and interactions among features in multivariate time series data. Our model simultaneously attends to relevant features and learns their dependencies, enabling accurate classification of solar flare events. We evaluated our approach on SWAN-SF, the largest MVTS dataset for predicting solar flares, and compared its performance against several state-of-the-art methods. The experimental results demonstrate that our approach achieves superior classification performance, even when dealing with a highly imbalanced dataset characterized by the scarcity of major flaring events. These findings highlight the effectiveness of attention mechanisms and transformer models in learning discriminatory features from MVTS-based space weather data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/Kalshammari/Transformer-Model.git.

References

  1. Ahmadzadeh, A., Aydin, B., Georgoulis, M.K., Kempton, D.J., Mahajan, S.S., Angryk, R.A.: How to train your flare prediction model: Revisiting robust sampling of rare events. The Astrophysical Journal Supplement Series 254(2), 23 (may 2021) https://doi.org/10.3847/1538-4365/abec88,https://doi.org/10.3847/1538-4365/abec88

  2. Ahmadzadeh, A., Aydin, B., Georgoulis, M.K., Kempton, D.J., Mahajan, S.S., Angryk, R.A.: How to train your flare prediction model: Revisiting robust sampling of rare events. Astrophys. J. Suppl. Ser. 254(2), 23 (2021). https://doi.org/10.3847/1538-4365/abec88

    Article  Google Scholar 

  3. Allouche, O., Tsoar, A., Kadmon, R.: Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (tss). J. Appl. Ecol. 43(6), 1223–1232 (2006). https://doi.org/10.1111/j.1365-2664.2006.01214.x

    Article  Google Scholar 

  4. Alshammari, K., Hamdi, S.M., Boubrahimi, S.F.: Feature selection from multivariate time series data: A case study of solar flare prediction. In: IEEE International Conference on Big Data, Big Data 2022, Osaka, Japan, December 17-20, 2022. pp. 4796–4801. IEEE (2022).https://doi.org/10.1109/BIGDATA55660.2022.10020669,https://doi.org/10.1109/BigData55660.2022.10020669

  5. Alshammari, K., Hamdi, S.M., Boubrahimi, S.F.: Identifying flare-indicative photospheric magnetic field parameters from multivariate time-series data of solar active regions. Astrophys. J. Suppl. Ser. 271(2), 39 (2024)

    Article  Google Scholar 

  6. Alshammari, K., Hamdi, S.M., Muzaheed, A.A.M., Boubrahimi, S.F.: Forecasting multivariate time series of the magnetic field parameters of the solar events. CIKM workshop for Applied Machine Learning Methods for Time Series Forecasting (AMLTS) (2022)

    Google Scholar 

  7. Angryk, R.A., Martens, P.C., Aydin, B., Kempton, D., Mahajan, S.S., Basodi, S., Ahmadzadeh, A., Cai, X., Filali Boubrahimi, S., Hamdi, S.M., et al.: Multivariate time series dataset for space weather data analytics. Scientific data 7(1), 1–13 (2020)

    Article  Google Scholar 

  8. Bloomfield, D.S., Higgins, P.A., McAteer, R.T.J., Gallagher, P.T.: Toward reliable benchmarking of solar flare forecasting methods. The Astrophysical Journal Letters 747(2), L41 (feb 2012).https://doi.org/10.1088/2041-8205/747/2/L41,https://doi.org/10.1088/2041-8205/747/2/L41

  9. Bobra, M.G., Couvidat, S.: Solar flare prediction using sdo/hmi vector magnetic field data with a machine-learning algorithm. Astrophys J 798(2), 135 (2015)

    Article  Google Scholar 

  10. Cortes, C., Vapnik, V.: Support-vector networks. Machine learning 20(3), 273–297 (1995)

    Google Scholar 

  11. Cui, Y., Li, R., Zhang, L., He, Y., Wang, H.: Correlation between solar flare productivity and photospheric magnetic field properties. Sol. Phys. 237(1), 45–59 (2006)

    Article  Google Scholar 

  12. Dempster, A., Schmidt, D.F., Webb, G.I.: MiniRocket: A very fast (almost) deterministic transform for time series classification. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 248–257. ACM, New York (2021)

    Google Scholar 

  13. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding (2019)

    Google Scholar 

  14. Eastwood, J., Biffis, E., Hapgood, M., Green, L., Bisi, M., Bentley, R., Wicks, R., McKinnell, L.A., Gibbs, M., Burnett, C.: The economic impact of space weather: Where do we stand?: The economic impact of space weather. Risk Analysis 37 (02 2017) https://doi.org/10.1111/risa.12765

  15. Gao, J., Han, Y., Mao, Y.: A novel evaluation metric for imbalanced classification based on gini coefficient and tss. IEEE Access 8, 80268–80280 (2020) https://doi.org/10.1109/ACCESS.2020.2996775

  16. Hamdi, S.M., Aydin, B., Boubrahimi, S.F., Angryk, R., Krishnamurthy, L.C., Morris, R.: Biomarker detection from fmri-based complete functional connectivity networks. In: 2018 IEEE First International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). pp. 17–24. IEEE (2018)

    Google Scholar 

  17. Hamdi, S.M., Kempton, D., Ma, R., Boubrahimi, S.F., Angryk, R.A.: A time series classification-based approach for solar flare prediction. In: 2017 IEEE International Conference on Big Data (Big Data). pp. 2543–2551 (2017).https://doi.org/10.1109/BigData.2017.8258213

  18. Hosseinzadeh, P., Boubrahimi, S.F., Hamdi, S.M.: Improving solar energetic particle event prediction through multivariate time series data augmentation. Astrophys. J. Suppl. Ser. 270(2), 31 (2024)

    Article  Google Scholar 

  19. Hosseinzadeh, P., Filali Boubrahimi, S., Hamdi, S.M.: Toward enhanced prediction of high-impact solar energetic particle events using multimodal time series data fusion models. Space Weather 22(6), e2024SW003982 (2024) https://doi.org/10.1029/2024SW003982, https://doi.org/10.1029/2024SW003982, e2024SW003982 2024SW003982

  20. Leka, K., Barnes, G.: Photospheric magnetic field properties of flaring versus flare-quiet active regions. ii. discriminant analysis. The Astrophysical Journal 595(2), 1296 (2003)

    Google Scholar 

  21. Li, X., Zheng, Y., Wang, X., Wang, L.: Predicting solar flares using a novel deep convolutional neural network. Astrophys J 891(1), 10 (2020)

    Article  Google Scholar 

  22. McIntosh, P.S.: The classification of sunspot groups. Sol. Phys. 125(2), 251–267 (1990)

    Article  Google Scholar 

  23. Middlehurst, M., Large, J., Bagnall, A.J.: The canonical interval forest (CIF) classifier for time series classification. CoRR abs/2008.09172 (2020), https://arxiv.org/abs/2008.09172

  24. Muzaheed, A.A.M., Hamdi, S.M., Boubrahimi, S.F.: Sequence model-based end-to-end solar flare classification from multivariate time series data. In: 20th IEEE International Conference on Machine Learning and Applications, ICMLA 2021, Pasadena, CA, USA, December 13-16, 2021. pp. 435–440. IEEE (2021)https://doi.org/10.1109/ICMLA52953.2021.00074,https://doi.org/10.1109/ICMLA52953.2021.00074

  25. Nguyen, T.L., Gsponer, S., Ilie, I., O’Reilly, M., Ifrim, G.: Interpretable time series classification using linear models and multi-resolution multi-domain symbolic representations. CoRR abs/2006.01667 (2020), https://arxiv.org/abs/2006.01667

  26. Nishizuka, N., Sugiura, K., Kubo, Y., Den, M., Watari, S., Ishii, M.: Solar flare prediction model with three machine-learning algorithms using ultraviolet brightening and vector magnetograms. Astrophys. J. 835(2), 156 (2017)

    Article  Google Scholar 

  27. Saini, K., Alshammari, K., Hamdi, S.M., Filali Boubrahimi, S.: Classification of major solar flares from extremely imbalanced multivariate time series data using minimally random convolutional kernel transform. Universe 10(6) (2024) https://doi.org/10.3390/universe10060234, https://www.mdpi.com/2218-1997/10/6/234

  28. Song, H., Tan, C., Jing, J., Wang, H., Yurchyshyn, V., Abramenko, V.: Statistical assessment of photospheric magnetic features in imminent solar flare predictions. Sol. Phys. 254(1), 101–125 (2009)

    Article  Google Scholar 

  29. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need (2017)

    Google Scholar 

  30. Velivckovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: Proceedings of the 5th International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

  31. Yu, D., Huang, X., Wang, H., Cui, Y.: Short-term solar flare prediction using a sequential supervised learning method. Sol. Phys. 255(1), 91–105 (2009)

    Article  Google Scholar 

  32. Zheng, X., Zhang, C., Woodland, P.C.: Adapting gpt, gpt-2 and bert language models for speech recognition (2021)

    Google Scholar 

  33. Zheng, Y., Li, X., Wang, X.: Solar flare prediction with the hybrid deep convolutional neural network. Astrophys J 885(1), 73 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This project has been supported in part by funding from the Division of Atmospheric and Geospace Sciences within the Directorate for Geosciences, under NSF awards #2301397, #2204363, and #2240022, and by funding from the Office of Advanced Cyberinfrastructure within the Directorate for Computer and Information Science and Engineering, under NSF award #2305781. The authors acknowledge the use of ChatGPT (GPT-3.5) to rephrase sentences and improve the writing style of the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Khaznah Alshammari .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alshammari, K., Hamdi, S.M., Boubrahimi, S.F. (2025). Transformer Model for Multivariate Time Series Classification: A Case Study of Solar Flare Prediction. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15324. Springer, Cham. https://doi.org/10.1007/978-3-031-78383-8_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-78383-8_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-78382-1

  • Online ISBN: 978-3-031-78383-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics