Skip to main content
SpringerLink
Log in

1-D CNNs with lock-free asynchronous adaptive stochastic gradient descent algorithm for classification of astronomical spectra

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

At present, large-scale sky surveys have obtained a large volume of stellar spectra. An efficient classification algorithm is of great importance to the practice of astronomical research. In this paper, we propose a novel parallel optimization algorithm based on a lock-free and shared-memory environment to solve the model for astronomical spectra class. Firstly, the SMOTE-TOMEK and RobustScaler are introduced to use for class balancing and data normalization. Secondly, 1-Dimensional Convolutional Neural Networks (1-D CNN) with L2-norm loss function is utilized as a classifier. Finally, LFA-SGD, LFA-Adagrad, LFA-RMSprop and LFA-Adam algorithms are proposed and applied to the classifier solution. The Lock-Free and shared-memory parallel Asynchronous environment (LFA) relies on GPU multiprocessing, allowing the algorithm to fully utilize the multi-core resources of the computer. Due to its sparsity, the convergence speed is significantly faster. The experimental results show that LFA-SGD algorithm and its variants achieved state-of-the-art accuracy and efficiency for astronomical spectra class.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Siegmund OHW, Tremsin AS, Vallerga JV (2007) High performance cross-strip detector technologies for space astrophysics. In: 2007 IEEE nuclear science symposium conference record, vol. 3, pp 2246–2251. https://doi.org/10.1109/NSSMIC.2007.4436595

  2. Ding-Qiang S, Peng J, Genrong L (2012) The atmospheric dispersion corrector for the large sky area multi-object fibre spectroscopic telescope (lamost). Mon Not R Astron Soc 4:3406–3413

    Google Scholar 

  3. Xiang-ru L, Zhong-tian L, Zhan-yi H (2007) Celestial spectrum flux standardization for classification. Acta Astronomica Sinica 48(3):9

    Google Scholar 

  4. Thara DK, Premasudha BG, Xiong F (2019) Auto-detection of epileptic seizure events using deep neural network with different feature scaling techniques. Pattern Recognit Lett 128(Dec.):544–550

    Google Scholar 

  5. Wang Qixun ZG, Zhou F (2020) Classification of astronomical spectra based on densenet. Astron Technol Res 17(1):85–95

    Google Scholar 

  6. Das R, Biswas SK, Devi D, Sarma B (2020) An oversampling technique by integrating reverse nearest neighbor in smote: reverse-smote. In: 2020 international conference on smart electronics and communication (ICOSEC), pp 1239–1244. https://doi.org/10.1109/ICOSEC49089.2020.9215387

  7. Jonathan B, Putra PH, Ruldeviyani Y (2020) Observation imbalanced data text to predict users selling products on female daily with smote, tomek, and smote-tomek. In: 2020 IEEE international conference on industry 4.0, artificial intelligence, and communications technology (IAICT), pp 81–85. https://doi.org/10.1109/IAICT50021.2020.9172033

  8. Liu C, Cui W-Y, Zhang B, Wan J-C, Deng L-C, Hou Y-H, Wang Y-F, Yang M, Zhang Y (2015) Spectral classification of stars based on lamost spectra. Res Astron Astrophys 15(8):1137. https://doi.org/10.1088/1674-4527/15/8/004

    Article  ADS  CAS  Google Scholar 

  9. Li X-R, Lin Y-T, Qiu K-B (2019) Stellar spectral classification and feature evaluation based on a random forest. Res Astron Astrophys 19(8):111. https://doi.org/10.1088/1674-4527/19/8/111

    Article  ADS  Google Scholar 

  10. Schierscher F, Paunzen E (2011) An artificial neural network approach to classify sdss stellar spectra. Astron Nachr 332(6):597–601

    Article  ADS  Google Scholar 

  11. XU Ting-ting ZJ-M, Chenye MA, Wei-hong Z (2019) Classification of lamost spectra based on deep learning. Acta Astron Sinica 060(002):13–22

    Google Scholar 

  12. Swati ZNK, Zhao Q, Kabir M, Ali F, Lu J (2019) Brain tumor classification for mr images using transfer learning and fine-tuning. Comput Med Imaging Graphics 75:34–46

    Article  Google Scholar 

  13. Zhao SY, Xie YP, Wu-Jun LI (2021) On the convergence and improvement of stochastic normalized gradient descent. Sci China Inf Sci 64:1–13

    Article  MathSciNet  Google Scholar 

  14. Zinkevich M, Weimer M, Smola AJ, Li L (2011) Parallelized stochastic gradient descent. In: advances in neural information processing systems 23: conference on neural information processing systems a meeting held December

  15. Xing E, P Q, Dai W, Kim JK, (2015) Petuum: a new platform for distributed machine learning on big data. IEEE Trans Big Data 1:1335–1344

  16. Sra S, Yu AW, Li M, Smola AJ (2016) Adadelay: delay adaptive distributed stochastic convex optimization. Artif Intell Stat 957–965. PMLR

  17. Niu F, Recht B, Re C, Wright SJ (2011) Hogwild!: a lock-free approach to parallelizing stochastic gradient descent. Adv Neural Inf Process Syst 24:693–701

    Google Scholar 

  18. Mania H, Pan X, Papailiopoulos D, Recht B, Ramchandran K, Jordan MI (2015) Perturbed iterate analysis for asynchronous stochastic optimization. Mathematics 18(1):51–52

    Google Scholar 

  19. Luo W, Zhong L, Zhang B (2008) Null subspace analysis for spectral unmixing in hyperspectral remote sensing. In: 2008 Congress on Image and Signal Processing, vol. 4, pp 763–767. IEEE

  20. Kheirdastan S, Bazarghan M (2016) Sdss-dr12 bulk stellar spectral classification: artificial neural networks approach. Astrophys Space Sci 361(9):304

    Article  ADS  Google Scholar 

  21. Xiaona F, Chengwu L, Xianyong B (2018) A detection method for sunspots based on convolutional neural network lenet-5. Astron Res Technol 15(3):340–346

    Google Scholar 

  22. Li C, Zhang Y, Cui C, Fan D, Zhao Y, Wu X-B, He B, Xu Y, Li S, Han J, Tao Y, Mi L, Yang H, Yang S (2021) Identification of bass dr3 sources as stars, galaxies, and quasars by xgboost. Mon Not R Astron Soc 506(2):1651–1664. https://doi.org/10.1093/mnras/stab1650

    Article  ADS  CAS  Google Scholar 

  23. Jing-min Z, Chen-ye M, Lu W, Li-ting D (2020) A new stellar spectral feature extraction method based on two-dimensional fourier spectrum image and its application in the stellar spectral classification based on deep network. Acta Astron Sinica 61(2):93–101

    Google Scholar 

  24. Abdeljaber O, Avci O, Kiranyaz MS, Boashash B, Sodano H, Inman DJ (2018) 1-d cnns for structural damage detection: verification on a structural health monitoring benchmark data. Neurocomputing 275:1308–1317. https://doi.org/10.1016/j.neucom.2017.09.069

    Article  Google Scholar 

  25. Wu S, Li G, Deng L, Liu L, Wu D, Xie Y, Shi L (2019) \(l1\) -norm batch normalization for efficient training of deep neural networks. IEEE Trans Neural Netw Learn Syst 30(7):2043–2051. https://doi.org/10.1109/TNNLS.2018.2876179

    Article  PubMed  Google Scholar 

  26. Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev 60(2):223–311

    Article  MathSciNet  Google Scholar 

  27. Konen J, Jie L, Richtárik P, Taká M (2014) ms2gd: Mini-batch semi-stochastic gradient descent in the proximal setting. IEEE J Sel Top Signal Process 10(2):242–255

    ADS  Google Scholar 

  28. Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization, pp 257–269

  29. Graves A (2013) Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850

  30. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  31. Nesterov Yu (2013) Gradient methods for minimizing composite functions. Math Program 140(1):125–161

    Article  MathSciNet  Google Scholar 

  32. Li M (2014) Efficient mini-batch training for stochastic optimization. In: proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 661–670

  33. Dekel O, Gilad-Bachrach R, Shamir O, Xiao L (2010) Optimal distributed online prediction using mini-batches. J Mach Learn Res 13

  34. Kale RS, Shitole S (2022) Deep learning optimizer performance analysis for pomegranate fruit quality gradation. In: 2022 IEEE bombay section signature conference (IBSSC), pp 1–5. https://doi.org/10.1109/IBSSC56953.2022.10037429

  35. Devikanniga D, Vetrivel K, Badrinath N (2019) Review of meta-heuristic optimization based artificial neural networks and its applications. In: IOP Publishing Ltd, p 012074

  36. Qin C, Li B, Han B (2023) Fast brain tumor detection using adaptive stochastic gradient descent on shared-memory parallel environment. Eng Appl Artif Intell 120:105816. https://doi.org/10.1016/j.engappai.2022.105816

    Article  Google Scholar 

  37. Eiras-Franco C, Bolón-Canedo V, Ramos S, González-Domínguez J, Alonso-Betanzos A, Touriño J (2016) Multithreaded and spark parallelization of feature selection filters. J Comput Sci 17:609–619. https://doi.org/10.1016/j.jocs.2016.07.002

    Article  Google Scholar 

  38. Hector EC, Luo L, Song PX-K (2023) Parallel-and-stream accelerator for computationally fast supervised learning. Comput Stat Data Anal 177:107587. https://doi.org/10.1016/j.csda.2022.107587

    Article  MathSciNet  Google Scholar 

  39. Leblond R, Pedregosa F, Lacoste-Julien S (2018) Improved asynchronous parallel optimization analysis for stochastic incremental methods. J Mach Learn Res 19:1–68

    MathSciNet  Google Scholar 

  40. Shang F, Huang H, Fan J, Liu Y, Liu H, Liu J (2022) Asynchronous parallel, sparse approximated svrg for high-dimensional machine learning. IEEE Trans Knowl Data Eng 34(12):5636–5648. https://doi.org/10.1109/TKDE.2021.3070539

    Article  Google Scholar 

  41. Wang D, Lei Y, Zhou J (2021) Hybrid mpi/openmp parallel asynchronous distributed alternating direction method of multipliers. Computing 103:2737–2762

    Article  MathSciNet  Google Scholar 

  42. Ghosh S, Aquino B, Gupta V (2022) Eventgrad: event-triggered communication in parallel machine learning. Neurocomputing 483:474–487. https://doi.org/10.1016/j.neucom.2021.08.143

    Article  Google Scholar 

  43. Zaccone G (2019) Python Parallel Programming cookbook - Second Edition, pp 144–178

  44. Kusakabe R, Ichimura T, Fujita K, Hori M, Wijerathne L (2022) Scalable large-scale multi-physics earthquake simulation on multiple gpus with stabilization. J Comput Sci 61:101619. https://doi.org/10.1016/j.jocs.2022.101619

    Article  Google Scholar 

  45. Barrachina S, Castelló A, Catalán M, Dolz MF, Mestre JI (2023) Using machine learning to model the training scalability of convolutional neural networks on clusters of gpus. Computing 105(5):915–934

    Article  MathSciNet  Google Scholar 

  46. Meng Q, Chen W, Wang Y, Ma Z-M, Liu T-Y (2019) Convergence analysis of distributed stochastic gradient descent with shuffling. Neurocomputing 337:46–57. https://doi.org/10.1016/j.neucom.2019.01.037

    Article  Google Scholar 

  47. Sardy S, Diaz-Rodriguez J, Giacobino C (2022) Thresholding tests based on affine lasso to achieve non-asymptotic nominal level and high power under sparse and dense alternatives in high dimension. Comput Stat Data Anal 173:107507. https://doi.org/10.1016/j.csda.2022.107507

    Article  MathSciNet  Google Scholar 

  48. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  Google Scholar 

  49. Tianchi (2018) Celestial spectral classification dataset. https://tianchi.aliyun.com/dataset/dataDetail?dataId=1077

  50. Chen T, Sun Y, Li T-H (2021) A semi-parametric estimation method for the quantile spectrum with an application to earthquake classification using convolutional neural network. Comput Stat Data Anal 154:107069. https://doi.org/10.1016/j.csda.2020.107069

    Article  MathSciNet  Google Scholar 

  51. Ali H, Salleh MNM, Saedudin R, Hussain K, Mushtaq MF (2019) Imbalance class problems in data mining: a review. Indones J Electr Eng Comput Sci 14(3):1560–1571

    Google Scholar 

  52. Gore S, Chibber Y, Bhasin M, Mehta S, Suchitra S (2023) Customer churn prediction using neural networks and smote-enn for data sampling. In: 2023 3rd international conference on artificial intelligence and signal processing (AISP), pp 1–5. https://doi.org/10.1109/AISP57993.2023.10134827

  53. Zhang J, Wang T, Ng WWY, Pedrycz W (2022) Ensembling perturbation-based oversamplers for imbalanced datasets. Neurocomputing 479:1–11. https://doi.org/10.1016/j.neucom.2022.01.049

    Article  Google Scholar 

  54. Ning Q, Zhao X, Ma Z (2022) A novel method for identification of glutarylation sites combining borderline-smote with tomek links technique in imbalanced data. IEEE/ACM Trans Comput Biol Bioinform 19(5):2632–2641. https://doi.org/10.1109/TCBB.2021.3095482

    Article  CAS  PubMed  Google Scholar 

  55. Park S, Park H (2021) Combined oversampling and undersampling method based on slow-start algorithm for imbalanced network traffic. Computing 103(1):1–24

    MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China (No.62066001), Natural Science Foundation of Ningxia Province (No.2021AAC03230). The authors would like to thank the anonymous reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. School of Mathematics and Information Science, North Minzu University, Yinchuan, 750021, China

    Chuandong Qin & Yu Cao

  2. Ningxia Key Laboratory of Intelligent Information and Big data processing, Yinchuan, 750021, China

    Chuandong Qin

Authors
  1. Chuandong Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Cao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, C., Cao, Y. 1-D CNNs with lock-free asynchronous adaptive stochastic gradient descent algorithm for classification of astronomical spectra. Computing 106, 713–739 (2024). https://doi.org/10.1007/s00607-023-01240-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-023-01240-3

Keywords

Mathematics Subject Classification

Search

Navigation