ABSTRACT
Automated machine learning (AutoML) strives for automatically constructing and configuring compositions of machine learning algorithms, called pipelines, with the goal to optimize a suitable performance measure on a concrete learning task. So far, most AutoML tools are focused on standard problem classes, such as classification and regression. In the field of predictive maintenance, especially the estimation of remaining useful lifetime (RUL), the task of AutoML becomes more complex. In particular, a good feature representation for multivariate sensor data is essential to achieve good performance. Due to the need for methods generating feature representations, the search space of candidate pipelines enlarges. Moreover, the runtime of a single pipeline increases substantially. In this paper, we tackle these problems by partitioning the search space into two sub-spaces, one for feature extraction methods and one for regression methods, and employ cooperative coevolution for searching a good combination. Thereby, we benefit from the fact that the generated feature representations can be cached, whence the evaluation of multiple regressors based on the same feature representation speeds up, allowing the evaluation of more candidate pipelines. Experimentally, we show that our coevolutionary strategy performs superior to the baselines.
- Anthony J. Bagnall, Jason Lines, Aaron Bostrom, James Large, and Eamonn J. Keogh. 2017. The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 31, 3 (2017), 606--660. Google ScholarDigital Library
- Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, and Hod Lipson. 2018. Autostacker: a compositional evolutionary learning system. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2018, Kyoto, Japan, July 15-19, 2018. 402--409. Google ScholarDigital Library
- Weiwei Cheng, Gjergji Kasneci, Thore Graepel, David H. Stern, and Ralf Herbrich. 2011. Automated feature generation from structured knowledge. In Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011, Glasgow, United Kingdom, October 24-28, 2011. 1395--1404. Google ScholarDigital Library
- Maximilian Christ, Nils Braun, Julius Neuffer, and Andreas W. Kempa-Liehr. 2018. Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package). Neurocomputing 307 (2018), 72--77. Google ScholarDigital Library
- Alex Guimarães Cardoso de Sá, Alex Alves Freitas, and Gisele L. Pappa. 2018. Automated Selection and Configuration of Multi-Label Classification Algorithms with Grammar-Based Genetic Programming. In Parallel Problem Solving from Nature - PPSN XV - 15th International Conference, Coimbra, Portugal, September 8-12, 2018, Proceedings, Part II. 308--320. Google ScholarCross Ref
- Kalyanmoy Deb and Ram Bhushan Agrawal. 1995. Simulated Binary Crossover for Continuous Search Space. Complex Syst. 9, 2 (1995). http://www.complex-systems.com/abstracts/v09_i02_a02.htmlGoogle Scholar
- Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6, 2 (2002), 182--197. Google ScholarDigital Library
- Kalyanmoy Deb and Mayank Goyal. 1996. A Combined Genetic Adaptive Search (GeneAS) for Engineering Design. Computer Science and Informatics 26 (1996), 30--45.Google Scholar
- Angus Dempster, François Petitjean, and Geoffrey I. Webb. 2020. ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels. Data Min. Knowl. Discov. 34, 5 (2020), 1454--1495. Google ScholarDigital Library
- Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural Architecture Search: A Survey. J. Mach. Learn. Res. 20 (2019), 55:1--55:21. http://jmlr.org/papers/v20/18-598.htmlGoogle Scholar
- Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and Efficient Hyperparameter Optimization at Scale. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018. 1436--1445. http://proceedings.mlr.press/v80/falkner18a.htmlGoogle Scholar
- Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. 2962--2970. https://proceedings.neurips.cc/paper/2015/hash/11d0e6287202fced83f79975ec59a3a6-Abstract.htmlGoogle Scholar
- Peter I. Frazier. 2018. A Tutorial on Bayesian Optimization. CoRR abs/1807.02811 (2018). arXiv:1807.02811 http://arxiv.org/abs/1807.02811Google Scholar
- Martin W. Hoffmann, Stephan Wildermuth, Ralf Gitzel, Aydin Boyaci, Jörg Gebhardt, Holger Kaul, Ido Amihai, Bodo Forg, Michael Suriyah, Thomas Leibfried, Volker Stich, Jan Hicking, Martin Bremer, Lars Kaminski, Daniel Beverungen, Philipp zur Heiden, and Tanja Tornede. 2020. Integration of Novel Sensors and Machine Learning for Predictive Maintenance in Medium Voltage Switchgear to Enable the Energy and Mobility Revolutions. Sensors 20, 7 (2020), 2099. Google ScholarCross Ref
- Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren (Eds.). 2019. Automated Machine Learning - Methods, Systems, Challenges. Springer. Google ScholarCross Ref
- Celio H. N. Larcher Jr. and Helio J. C. Barbosa. 2019. Auto-CVE: a coevolutionary approach to evolve ensembles in automated machine learning. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, July 13-17, 2019. 392--400. Google ScholarDigital Library
- Ambika Kaul, Saket Maheshwary, and Vikram Pudi. 2017. AutoLearn - Automated Feature Generation and Selection. In 2017 IEEE International Conference on Data Mining, ICDM 2017, New Orleans, LA, USA, November 18-21, 2017. 217--226. Google ScholarCross Ref
- Udayan Khurana,Deepak S. Turaga, Horst Samulowitz, and Srinivasan Parthasrathy. 2016. Cognito: Automated Feature Engineering for Supervised Learning. In IEEE International Conference on Data Mining Workshops, ICDM Workshops 2016, December 12-15, 2016, Barcelona, Spain. 1304--1307. Google ScholarCross Ref
- Lisha Li, Kevin G. Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2017. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. J. Mach. Learn. Res. 18 (2017), 185:1--185:52. http://jmlr.org/papers/v18/16-558.htmlGoogle Scholar
- Jason Lines, Luke M. Davis, Jon Hills, and Anthony J. Bagnall. 2012. A shapelet transform for time series classification. In The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '12, Beijing, China, August 12-16, 2012. 289--297. Google ScholarDigital Library
- Felix Mohr, Marcel Wever, and Eyke Hüllermeier. 2018. ML-Plan: Automated machine learning via hierarchical planning. Mach. Learn. 107, 8-10 (2018), 1495--1515. Google ScholarDigital Library
- Fatemeh Nargesian, Horst Samulowitz, Udayan Khurana, Elias B. Khalil, and Deepak S. Turaga. 2017. Learning Feature Engineering for Classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017. 2529--2535. Google ScholarCross Ref
- Patrick Nectoux, Rafael Gouriveau, Kamal Medjaher, Emmanuel Ramasso, Brigitte Chebel-Morello, Noureddine Zerhouni, and Christophe Varnier. 2012. PRONOS-TIA: An experimental platform for bearings accelerated degradation tests.Google Scholar
- Randal S. Olson and Jason H. Moore. 2016. TPOT: A Tree-based Pipeline Optimization Tool for Automating Machine Learning. In Proceedings of the 2016 Workshop on Automatic Machine Learning, AutoML 2016, co-located with 33rd International Conference on Machine Learning (ICML 2016), New York City, NY, USA, June 24, 2016. 66--74. http://proceedings.mlr.press/v64/olson_tpot_2016.htmlGoogle Scholar
- Fernando E. B. Otero, Tom Castle, and Colin G. Johnson. 2012. EpochX: genetic programming in java with statistics and event monitoring. In Genetic and Evolutionary Computation Conference, GECCO '12, Philadelphia, PA, USA, July 7-11, 2012, Companion Material Proceedings. 93--100. Google ScholarDigital Library
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay. 2011. Scikit-learn: Machine Learning in Python. J. Mack. Learn. Res. 12 (2011), 2825--2830. http://dl.acm.org/citation.cfm?id=2078195Google ScholarDigital Library
- Hojjat Rakhshani, Hassan Ismail Fawaz, Lhassane Idoumghar, Germain Forestier, Julien Lepagnot, Jonathan Weber, Mathieu Brévilliers, and Pierre-Alain Muller. 2020. Neural Architecture Search for Time Series Classification. In 2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19-24, 2020. 1--8. Google ScholarCross Ref
- Yongyi Ran, Xin Zhou, Pengfeng Lin, Yonggang Wen, and Ruilong Deng. 2019. A Survey of Predictive Maintenance: Systems, Purposes and Approaches. CoRR abs/1912.07383 (2019). arXiv:1912.07383 http://arxiv.org/abs/1912.07383Google Scholar
- Abhinav Saxena, Kai Goebel, Don Simon, and Neil Eklund. 2008. Damage propagation modeling for aircraft engine run-to-failure simulation. In International Conference on Prognostics and Health Management. IEEE, 1--9.Google ScholarCross Ref
- A. Saxena and K. Goebelt. 2008. PHM08 Challenge Data Set, Vol. NASA Ames Prognostics Data Repository. NASA Ames Research Center, Moffett Field, CA. http://ti.arc.nasa.gov/project/prognostic-data-repository (accessed May 20, 2020).Google Scholar
- A. Saxena and K. Goebelt. 2008. Turbofan Engine Degradation Simulation Data Set, Vol. NASA Ames Prognostics Data Repository. NASA Ames Research Center, Moffett Field, CA. http://ti.arc.nasa.gov/project/prognostic-data-repository (accessed May 20, 2020).Google Scholar
- Patrick Schäfer. 2015. The BOSS is concerned with time series classification in the presence of noise. Data Min. Knowl. Discov. 29, 6 (2015), 1505--1530. Google ScholarDigital Library
- Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013, Chicago, IL, USA, August 11-14, 2013. 847--855. Google ScholarDigital Library
- Tanja Tornede, Alexander Tornede, Marcel Wever, Felix Mohr, and Eyke Hüllermeier. 2020. AutoML for Predictive Maintenance: One Tool to RUL them all. IoTStream ECMLPKDD 2020 (2020).Google ScholarCross Ref
- Marcel Wever, Felix Mohr, Alexander Tornede, and Eyke Hüllermeier. 2019. Automating multi-label classification extending ML-Plan. In Proceedings of the AutoML Workshop at ICML, Vol. 2020.Google Scholar
- Marcel Wever, Alexander Tornede, Felix Mohr, and Eyke Hüllermeier. 2021. AutoML for Multi-Label Classification: Overview and Empirical Evaluation. IEEE Transactions on Pattern Analysis & Machine Intelligence 01 (2021), 1--1.Google ScholarCross Ref
- Peter A Whigham et al. 1995. Grammatically-based genetic programming. In Proceedings of the workshop on genetic programming: from theory to real-world applications, Vol. 16. 33--41.Google Scholar
- Martin Wistuba, Josif Grabocka, and Lars Schmidt-Thieme. 2015. Ultra-Fast Shapelets for Time Series Classification. CoRR abs/1503.05018 (2015). arXiv:1503.05018 http://arxiv.org/abs/1503.05018Google Scholar
- Yexu Zhou, Yuting Gao, Yiran Huang, Michael Hefenbrock, Till Riedel, and Michael Beigl. 2020. Automatic Remaining Useful Life Estimation Framework with Embedded Convolutional LSTM as the Backbone. CoRR abs/2008.03961 (2020). arXiv:2008.03961 https://arxiv.org/abs/2008.03961Google Scholar
Index Terms
- Coevolution of remaining useful lifetime estimation pipelines for automated predictive maintenance
Recommendations
Remaining useful lifetime prediction for predictive maintenance in manufacturing
AbstractTraditional maintenance approaches often result in either premature replacement of machine parts or downtime in production lines due to malfunctions. Consequently, these lead to significant amount waste in material, time and, ultimately, money. ...
Highlights- The study proposes a machine learning-based predictive maintenance approach in manufacturing.
- ML Models are developed using real-world big data from IoT sensors on assembly lines.
- Hybrid solutions were applied by combining ...
Using supervised and one-class automated machine learning for predictive maintenance
AbstractPredictive Maintenance (PdM) is a critical area that is benefiting from the Industry 4.0 advent. Recently, several attempts have been made to apply Machine Learning (ML) to PdM, with the majority of the research studies assuming an ...
Highlights- Automated Machine Learning (AutoML) is rarely used for Predictive Maintenance (PdM).
Prediction of Maintenance Equipment Failures Using Automated Machine Learning
Intelligent Data Engineering and Automated Learning – IDEAL 2021AbstractPredictive maintenance is a key area that is benefiting from the Industry 4.0 advent. Recently, there have been several attempts to use Machine Learning (ML) in order to optimize the maintenance of equipments and their repairs, with most of these ...
Comments