Abstract
Industrial applications put special demands on machine learning algorithms. Noisy data, outliers, and sensor faults present an immense challenge for learners. A considerable part of machine learning research focuses on the selection of relevant, non-redundant features. This contribution details an approach to group and fuse redundant features prior to learning and classification. Features are grouped relying on a correlation-based redundancy measure. The fusion of features is guided by determining the majority observation based on possibility distributions. Furthermore, this paper studies the effects of feature fusion on the robustness and performance of classification with a focus on industrial applications. The approach is statistically evaluated on public datasets in comparison to classification on selected features only.
Zusammenfassung
Industrielle Anwendungen stellen besondere Anforderungen an maschinelle Lernalgorithmen. Verrauschte Daten, Ausreißer und Sensorfehler sind eine große Herausforderung für Lerner. Ein erheblicher Teil der Forschung im Bereich maschinelles Lernen konzentriert sich auf die Auswahl relevanter, nicht redundanter Merkmale. Dieser Beitrag beschreibt einen Ansatz zur Gruppierung und Fusion redundanter Merkmale, die vor dem Lernen und Klassifizieren ausgeführt werden. Die Gruppierung der Merkmale basiert auf einer korrelationsbasierten Redundanzmessung, während die Fusionierung von Merkmalen auf der Bestimmung einer über Möglichkeitsverteilungen ermittelten Mehrheitsbeobachtung basiert. Darüber hinaus untersucht dieser Beitrag die Auswirkungen der Merkmalsfusion auf die Robustheit und Leistungsfähigkeit der Klassifizierung. Der Ansatz wird anhand öffentlicher Datensätze im Vergleich zur Klassifizierung auf ausgewählten Merkmalen statistisch ausgewertet.
Funding source: Bundesministerium für Bildung und Forschung
Award Identifier / Grant number: 01IS18041D
Funding statement: This work was partly funded by the German Federal Ministry of Education and Research within the project ITS.ML, Grant number 01IS18041D.
About the authors
Christoph-Alexander Holst received his master’s degree in information technology from the Technische Hochschule Ostwestfalen-Lippe, Germany. He is working towards his doctoral degree in cooperation with the Computer Engineering Group at the Brandenburg University of Technology Cottbus-Senftenberg. His main research topics are information fusion, fusion system design, and sensor orchestration.
Volker Lohweg is director of the inIT – Institute Industrial IT and head of the research group Discrete Systems. The research group’s working area is dedicated to cognitive systems in automation especially information fusion in the context of intelligent technical systems. He is active in SPIE and IEEE as a reviewer in the field of image processing and data analysis. His actual interests are sensory conflict modelling and multi-scale signal analysis.
References
1. M. A. Aizerman, E. M. Braverman and L. I. Rozonoer. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25:821–837, 1964.Search in Google Scholar
2. E. Alpaydın. Introduction to Machine Learning. The MIT Press, Cambridge, 2nd edition, 2010.Search in Google Scholar
3. B. M. Ayyub and G. J. Klir. Uncertainty Modeling and Analysis in Engineering and the Sciences. Chapman & Hall/CRC, Boca Raton, FL, 2006.10.1201/9781420011456Search in Google Scholar
4. J. Beyerer, J. Jasperneite and O. Sauer. Industrie 4.0. at – Automatisierungstechnik, 63(10), 2015.10.1515/auto-2015-0068Search in Google Scholar
5. F. Bocklisch and D. Hausmann. Multidimensional fuzzy pattern classifier sequences for medical diagnostic reasoning. Applied Soft Computing, 66:297–310, 2018.10.1016/j.asoc.2018.02.041Search in Google Scholar
6. S. F. Bocklisch. Prozeßanalyse mit unscharfen Verfahren. Verlag Technik, Berlin, 1st edition, 1987.Search in Google Scholar
7. L. Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996.10.1007/BF00058655Search in Google Scholar
8. L. Breiman. Classification and Regression Trees. Routledge, New York, 2017.10.1201/9781315139470Search in Google Scholar
9. D. Dheeru and E. Karra Taniskidou. UCI Machine Learning Repository, 2017.Search in Google Scholar
10. T. G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895–1923, 1998.10.1162/089976698300017197Search in Google Scholar PubMed
11. A. Diez-Olivan, J. Del Ser, D. Galar and B. Sierra. Data fusion and machine learning for industrial prognosis: Trends and perspectives towards Industry 4.0. Information Fusion, 50:92–111, 2019.10.1016/j.inffus.2018.10.005Search in Google Scholar
12. H. Dörksen and V. Lohweg. Combinatorial refinement of feature weighting for linear classification. In Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA), pages 1–7, 2014.10.1109/ETFA.2014.7005106Search in Google Scholar
13. D. Dubois, L. Foulloy, G. Mauris and H. Prade. Probability-possibility transformations, triangular fuzzy sets, and probabilistic inequalities. Reliable Computing, 10(4):273–297, 2004.10.1023/B:REOM.0000032115.22510.b5Search in Google Scholar
14. J.-F. Ehlenbröker, U. Mönks and V. Lohweg. Sensor defect detection in multisensor information fusion. Journal of Sensors and Sensor Systems, 5(2):337–353, 2016.10.5194/jsss-5-337-2016Search in Google Scholar
15. W. Elmenreich. An Introduction to Sensor Fusion, 2002.Search in Google Scholar
16. A. Fritze, U. Mönks, C.-A. Holst and V. Lohweg. An approach to automated fusion system design and adaptation. Sensors, 17(3):601, 2017.10.3390/s17030601Search in Google Scholar PubMed PubMed Central
17. S. Glock, K. Voth, J. Schaede and V. Lohweg. A framework for possibilistic multi-source data fusion with monitoring of sensor reliability. In World Conference on Soft Computing, 2011.Search in Google Scholar
18. I. Guyon, S. R. Gunn, M. Nikravesh and L. A. Zadeh. Feature extraction: Foundations and applications, volume 207 of Studies in Fuzziness and Soft Computing. Springer, Berlin Heidelberg, 2006.10.1007/978-3-540-35488-8Search in Google Scholar
19. D. L. Hall, J. Llinas and M. E. Liggins, editors. Handbook of Multisensor Data Fusion: Theory and Practice. The Electrical Engineering and Applied Signal Processing Series. CRC Press, Boca Raton, FL, 2nd edition, 2009.Search in Google Scholar
20. T. Hastie, R. Tibshirani, D. Botstein and P. Brown. Supervised harvesting of expression trees. Genome Biology, 2(1):research0003.1, 2001.10.1186/gb-2001-2-1-research0003Search in Google Scholar PubMed PubMed Central
21. N. Helwig, E. Pignanelli and A. Schütze. Detecting and compensating sensor faults in a hydraulic condition monitoring system. In SENSOR 2015, pages 641–646, Nürnberg, 2015. AMA Service GmbH.10.5162/sensor2015/D8.1Search in Google Scholar
22. A.-J. Hempel. Netzorientierte Fuzzy-Pattern-Klassifikation nichtkonvexer Objektmengenmorphologien. Doctoral thesis, Technische Universität Chemnitz, Chemnitz, 2011.Search in Google Scholar
23. C.-A. Holst and V. Lohweg. Improving majority-guided fuzzy information fusion for Industry 4.0 condition monitoring. In 2019 22nd International Conference on Information Fusion (FUSION). IEEE, 2019.10.23919/FUSION43075.2019.9011347Search in Google Scholar
24. Z. Hu and S. Mahadevan. Uncertainty quantification in prediction of material properties during additive manufacturing. Scripta Materialia, 135:135–140, 2017.10.1016/j.scriptamat.2016.10.014Search in Google Scholar
25. E. Hüllermeier. Fuzzy methods in machine learning and data mining: Status and prospects. Fuzzy Sets and Systems, 156(3):387–406, 2005.10.1016/j.fss.2005.05.036Search in Google Scholar
26. W. Jiang, C. Xie, M. Zhuang, Y. Shou and Y. Tang. Sensor data fusion with Z-numbers and its application in fault diagnosis. Sensors, 16(9), 2016.10.3390/s16091509Search in Google Scholar PubMed PubMed Central
27. W. Jiang, M. Zhuang and C. Xie. A reliability-based method to sensor data fusion. Sensors, 17(7), 2017.10.3390/s17071575Search in Google Scholar PubMed PubMed Central
28. M. Krüger. Gradual vs. binary conflicts in Bayesian networks applied to sensor failure detection. In 2015 18th International Conference on Information Fusion (Fusion), pages 66–73, 2015.Search in Google Scholar
29. P. Larrañaga, A. Ogbechie, J. Diaz-Rozo, D. Atienza Alonso, C. Bielza and C. Puerto-Santana. Industrial Applications of Machine Learning. Data Mining and Knowledge Series. CRC Press, Boca Raton, Florida, 2019.10.1201/9781351128384Search in Google Scholar
30. H. Li, H.-Z. Huang, Y.-F. Li, J. Zhou and J. Mi. Physics of failure-based reliability prediction of turbine blades using multi-source information fusion. Applied Soft Computing, 72:624–635, 2018.10.1016/j.asoc.2018.05.015Search in Google Scholar
31. J. Li, K. Cheng, S. Wang, F. Morstatter, R. P. Trevino, J. Tang and H. Liu. Feature selection: A data perspective. ACM Computing Surveys, 50(6):1–45, 2018.10.1145/3136625Search in Google Scholar
32. V. Lohweg, C. Diederichs and D. Müller. Algorithms for hardware-based pattern recognition. EURASIP Journal on Applied Signal Processing, 2004(12):1912–1920, 2004.10.1155/S1110865704404247Search in Google Scholar
33. R. C. Luo and M. G. Kay. Data fusion and sensor integration: State-of-the-art 1990s. In M. A. Abidi and R. C. Gonzalez, editors, Data Fusion in Robotics and Machine Intelligence, pages 7–136. Acad. Press, San Francisco, CA, USA, 1992.Search in Google Scholar
34. R. Maclin and D. Opitz. An empirical evaluation of bagging and boosting. In Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Conference on Innovative Applications of Artificial Intelligence, AAAI’97/IAAI’97, pages 546–551. AAAI Press, 1997.Search in Google Scholar
35. G. Mauris, V. Lasserre and L. Foulloy. Fuzzy modeling of measurement data acquired from physical sensors. IEEE Transactions on Instrumentation and Measurement, 49(6):1201–1205, 2000.10.1109/19.893256Search in Google Scholar
36. U. Mönks. Information Fusion Under Consideration of Conflicting Input Signals. Technologies for Intelligent Automation. Springer, Berlin, Heidelberg, 2017.10.1007/978-3-662-53752-7Search in Google Scholar
37. U. Mönks, D. Petker and V. Lohweg. Fuzzy-Pattern-Classifier training with small data sets. In E. Hüllermeier, R. Kruse, and F. Hoffmann, editors, Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Methods, pages 426–435, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg.10.1007/978-3-642-14055-6_44Search in Google Scholar
38. M. Y. Park, T. Hastie and R. Tibshirani. Averaged gene expressions for regression. Biostatistics, 8(2):212–227, 2007.10.1093/biostatistics/kxl002Search in Google Scholar PubMed
39. H. Peng, F. Long and C. Ding. Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226–1238, 2005.10.1109/TPAMI.2005.159Search in Google Scholar PubMed
40. V. Ricquebourg, L. Delahoche, B. Marhic, M. Delafosse, A.-M. Jolly-Desodt and D. Menga. Anomalies recognition in a context aware architecture based on TBM approach. In 2008 11th International Conference on Information Fusion, pages 1–8, 2008.Search in Google Scholar
41. H. Rinne. Taschenbuch der Statistik. Wissenschaftlicher Verlag Harri Deutsch GmbH, Frankfurt am Main, 4th edition, 2008.Search in Google Scholar
42. F. Shi, X. Su, H. Qian, N. Yang and W. Han. Research on the fusion of dependent evidence based on rank correlation coefficient. Sensors, 17(10), 2017.10.3390/s17102362Search in Google Scholar PubMed PubMed Central
43. M. Sokolova and G. Lapalme. A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4):427–437, 2009.10.1016/j.ipm.2009.03.002Search in Google Scholar
44. A. Vergara, S. Vembu, T. Ayhan, M. A. Ryan, M. L. Homer and R. Huerta. Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical, 166–167:320–329, 2012.10.1016/j.snb.2012.01.074Search in Google Scholar
45. K. Voth, S. Glock, U. Mönks, V. Lohweg and T. Türke. Multi-sensory machine diagnosis on security printing machines with two-layer conflict solving. In SENSOR+TEST Conference 2011, pages 686–691, Wunstorf, 2011. AMA Service GmbH.10.5162/sensor11/sp2.1Search in Google Scholar
46. L. A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965.10.1016/S0019-9958(65)90241-XSearch in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
