skip to main content
10.1145/3449639.3459395acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Coevolution of remaining useful lifetime estimation pipelines for automated predictive maintenance

Published:26 June 2021Publication History

ABSTRACT

Automated machine learning (AutoML) strives for automatically constructing and configuring compositions of machine learning algorithms, called pipelines, with the goal to optimize a suitable performance measure on a concrete learning task. So far, most AutoML tools are focused on standard problem classes, such as classification and regression. In the field of predictive maintenance, especially the estimation of remaining useful lifetime (RUL), the task of AutoML becomes more complex. In particular, a good feature representation for multivariate sensor data is essential to achieve good performance. Due to the need for methods generating feature representations, the search space of candidate pipelines enlarges. Moreover, the runtime of a single pipeline increases substantially. In this paper, we tackle these problems by partitioning the search space into two sub-spaces, one for feature extraction methods and one for regression methods, and employ cooperative coevolution for searching a good combination. Thereby, we benefit from the fact that the generated feature representations can be cached, whence the evaluation of multiple regressors based on the same feature representation speeds up, allowing the evaluation of more candidate pipelines. Experimentally, we show that our coevolutionary strategy performs superior to the baselines.

References

  1. Anthony J. Bagnall, Jason Lines, Aaron Bostrom, James Large, and Eamonn J. Keogh. 2017. The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 31, 3 (2017), 606--660. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, and Hod Lipson. 2018. Autostacker: a compositional evolutionary learning system. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2018, Kyoto, Japan, July 15-19, 2018. 402--409. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Weiwei Cheng, Gjergji Kasneci, Thore Graepel, David H. Stern, and Ralf Herbrich. 2011. Automated feature generation from structured knowledge. In Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011, Glasgow, United Kingdom, October 24-28, 2011. 1395--1404. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Maximilian Christ, Nils Braun, Julius Neuffer, and Andreas W. Kempa-Liehr. 2018. Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package). Neurocomputing 307 (2018), 72--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Alex Guimarães Cardoso de Sá, Alex Alves Freitas, and Gisele L. Pappa. 2018. Automated Selection and Configuration of Multi-Label Classification Algorithms with Grammar-Based Genetic Programming. In Parallel Problem Solving from Nature - PPSN XV - 15th International Conference, Coimbra, Portugal, September 8-12, 2018, Proceedings, Part II. 308--320. Google ScholarGoogle ScholarCross RefCross Ref
  6. Kalyanmoy Deb and Ram Bhushan Agrawal. 1995. Simulated Binary Crossover for Continuous Search Space. Complex Syst. 9, 2 (1995). http://www.complex-systems.com/abstracts/v09_i02_a02.htmlGoogle ScholarGoogle Scholar
  7. Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6, 2 (2002), 182--197. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Kalyanmoy Deb and Mayank Goyal. 1996. A Combined Genetic Adaptive Search (GeneAS) for Engineering Design. Computer Science and Informatics 26 (1996), 30--45.Google ScholarGoogle Scholar
  9. Angus Dempster, François Petitjean, and Geoffrey I. Webb. 2020. ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels. Data Min. Knowl. Discov. 34, 5 (2020), 1454--1495. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural Architecture Search: A Survey. J. Mach. Learn. Res. 20 (2019), 55:1--55:21. http://jmlr.org/papers/v20/18-598.htmlGoogle ScholarGoogle Scholar
  11. Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and Efficient Hyperparameter Optimization at Scale. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018. 1436--1445. http://proceedings.mlr.press/v80/falkner18a.htmlGoogle ScholarGoogle Scholar
  12. Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. 2962--2970. https://proceedings.neurips.cc/paper/2015/hash/11d0e6287202fced83f79975ec59a3a6-Abstract.htmlGoogle ScholarGoogle Scholar
  13. Peter I. Frazier. 2018. A Tutorial on Bayesian Optimization. CoRR abs/1807.02811 (2018). arXiv:1807.02811 http://arxiv.org/abs/1807.02811Google ScholarGoogle Scholar
  14. Martin W. Hoffmann, Stephan Wildermuth, Ralf Gitzel, Aydin Boyaci, Jörg Gebhardt, Holger Kaul, Ido Amihai, Bodo Forg, Michael Suriyah, Thomas Leibfried, Volker Stich, Jan Hicking, Martin Bremer, Lars Kaminski, Daniel Beverungen, Philipp zur Heiden, and Tanja Tornede. 2020. Integration of Novel Sensors and Machine Learning for Predictive Maintenance in Medium Voltage Switchgear to Enable the Energy and Mobility Revolutions. Sensors 20, 7 (2020), 2099. Google ScholarGoogle ScholarCross RefCross Ref
  15. Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren (Eds.). 2019. Automated Machine Learning - Methods, Systems, Challenges. Springer. Google ScholarGoogle ScholarCross RefCross Ref
  16. Celio H. N. Larcher Jr. and Helio J. C. Barbosa. 2019. Auto-CVE: a coevolutionary approach to evolve ensembles in automated machine learning. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, July 13-17, 2019. 392--400. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Ambika Kaul, Saket Maheshwary, and Vikram Pudi. 2017. AutoLearn - Automated Feature Generation and Selection. In 2017 IEEE International Conference on Data Mining, ICDM 2017, New Orleans, LA, USA, November 18-21, 2017. 217--226. Google ScholarGoogle ScholarCross RefCross Ref
  18. Udayan Khurana,Deepak S. Turaga, Horst Samulowitz, and Srinivasan Parthasrathy. 2016. Cognito: Automated Feature Engineering for Supervised Learning. In IEEE International Conference on Data Mining Workshops, ICDM Workshops 2016, December 12-15, 2016, Barcelona, Spain. 1304--1307. Google ScholarGoogle ScholarCross RefCross Ref
  19. Lisha Li, Kevin G. Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2017. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. J. Mach. Learn. Res. 18 (2017), 185:1--185:52. http://jmlr.org/papers/v18/16-558.htmlGoogle ScholarGoogle Scholar
  20. Jason Lines, Luke M. Davis, Jon Hills, and Anthony J. Bagnall. 2012. A shapelet transform for time series classification. In The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '12, Beijing, China, August 12-16, 2012. 289--297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Felix Mohr, Marcel Wever, and Eyke Hüllermeier. 2018. ML-Plan: Automated machine learning via hierarchical planning. Mach. Learn. 107, 8-10 (2018), 1495--1515. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Fatemeh Nargesian, Horst Samulowitz, Udayan Khurana, Elias B. Khalil, and Deepak S. Turaga. 2017. Learning Feature Engineering for Classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017. 2529--2535. Google ScholarGoogle ScholarCross RefCross Ref
  23. Patrick Nectoux, Rafael Gouriveau, Kamal Medjaher, Emmanuel Ramasso, Brigitte Chebel-Morello, Noureddine Zerhouni, and Christophe Varnier. 2012. PRONOS-TIA: An experimental platform for bearings accelerated degradation tests.Google ScholarGoogle Scholar
  24. Randal S. Olson and Jason H. Moore. 2016. TPOT: A Tree-based Pipeline Optimization Tool for Automating Machine Learning. In Proceedings of the 2016 Workshop on Automatic Machine Learning, AutoML 2016, co-located with 33rd International Conference on Machine Learning (ICML 2016), New York City, NY, USA, June 24, 2016. 66--74. http://proceedings.mlr.press/v64/olson_tpot_2016.htmlGoogle ScholarGoogle Scholar
  25. Fernando E. B. Otero, Tom Castle, and Colin G. Johnson. 2012. EpochX: genetic programming in java with statistics and event monitoring. In Genetic and Evolutionary Computation Conference, GECCO '12, Philadelphia, PA, USA, July 7-11, 2012, Companion Material Proceedings. 93--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay. 2011. Scikit-learn: Machine Learning in Python. J. Mack. Learn. Res. 12 (2011), 2825--2830. http://dl.acm.org/citation.cfm?id=2078195Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Hojjat Rakhshani, Hassan Ismail Fawaz, Lhassane Idoumghar, Germain Forestier, Julien Lepagnot, Jonathan Weber, Mathieu Brévilliers, and Pierre-Alain Muller. 2020. Neural Architecture Search for Time Series Classification. In 2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19-24, 2020. 1--8. Google ScholarGoogle ScholarCross RefCross Ref
  28. Yongyi Ran, Xin Zhou, Pengfeng Lin, Yonggang Wen, and Ruilong Deng. 2019. A Survey of Predictive Maintenance: Systems, Purposes and Approaches. CoRR abs/1912.07383 (2019). arXiv:1912.07383 http://arxiv.org/abs/1912.07383Google ScholarGoogle Scholar
  29. Abhinav Saxena, Kai Goebel, Don Simon, and Neil Eklund. 2008. Damage propagation modeling for aircraft engine run-to-failure simulation. In International Conference on Prognostics and Health Management. IEEE, 1--9.Google ScholarGoogle ScholarCross RefCross Ref
  30. A. Saxena and K. Goebelt. 2008. PHM08 Challenge Data Set, Vol. NASA Ames Prognostics Data Repository. NASA Ames Research Center, Moffett Field, CA. http://ti.arc.nasa.gov/project/prognostic-data-repository (accessed May 20, 2020).Google ScholarGoogle Scholar
  31. A. Saxena and K. Goebelt. 2008. Turbofan Engine Degradation Simulation Data Set, Vol. NASA Ames Prognostics Data Repository. NASA Ames Research Center, Moffett Field, CA. http://ti.arc.nasa.gov/project/prognostic-data-repository (accessed May 20, 2020).Google ScholarGoogle Scholar
  32. Patrick Schäfer. 2015. The BOSS is concerned with time series classification in the presence of noise. Data Min. Knowl. Discov. 29, 6 (2015), 1505--1530. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013, Chicago, IL, USA, August 11-14, 2013. 847--855. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Tanja Tornede, Alexander Tornede, Marcel Wever, Felix Mohr, and Eyke Hüllermeier. 2020. AutoML for Predictive Maintenance: One Tool to RUL them all. IoTStream ECMLPKDD 2020 (2020).Google ScholarGoogle ScholarCross RefCross Ref
  35. Marcel Wever, Felix Mohr, Alexander Tornede, and Eyke Hüllermeier. 2019. Automating multi-label classification extending ML-Plan. In Proceedings of the AutoML Workshop at ICML, Vol. 2020.Google ScholarGoogle Scholar
  36. Marcel Wever, Alexander Tornede, Felix Mohr, and Eyke Hüllermeier. 2021. AutoML for Multi-Label Classification: Overview and Empirical Evaluation. IEEE Transactions on Pattern Analysis & Machine Intelligence 01 (2021), 1--1.Google ScholarGoogle ScholarCross RefCross Ref
  37. Peter A Whigham et al. 1995. Grammatically-based genetic programming. In Proceedings of the workshop on genetic programming: from theory to real-world applications, Vol. 16. 33--41.Google ScholarGoogle Scholar
  38. Martin Wistuba, Josif Grabocka, and Lars Schmidt-Thieme. 2015. Ultra-Fast Shapelets for Time Series Classification. CoRR abs/1503.05018 (2015). arXiv:1503.05018 http://arxiv.org/abs/1503.05018Google ScholarGoogle Scholar
  39. Yexu Zhou, Yuting Gao, Yiran Huang, Michael Hefenbrock, Till Riedel, and Michael Beigl. 2020. Automatic Remaining Useful Life Estimation Framework with Embedded Convolutional LSTM as the Backbone. CoRR abs/2008.03961 (2020). arXiv:2008.03961 https://arxiv.org/abs/2008.03961Google ScholarGoogle Scholar

Index Terms

  1. Coevolution of remaining useful lifetime estimation pipelines for automated predictive maintenance

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference
          June 2021
          1219 pages
          ISBN:9781450383509
          DOI:10.1145/3449639

          Copyright © 2021 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 26 June 2021

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          Overall Acceptance Rate1,669of4,410submissions,38%

          Upcoming Conference

          GECCO '24
          Genetic and Evolutionary Computation Conference
          July 14 - 18, 2024
          Melbourne , VIC , Australia

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader