Abstract
Classification by means of machine learning models constitutes one relevant technology in process automation and predictive maintenance. However, common techniques such as deep networks or random forests suffer from their black box characteristics and possible adversarial examples. In this contribution, we give an overview about a popular alternative technology from machine learning, namely modern variants of learning vector quantization, which, due to their combined discriminative and generative nature, incorporate interpretability and the possibility of explicit reject options for irregular samples. We give an explicit bound on minimum changes required for a change of the classification in case of LVQ networks with reject option, and we demonstrate the efficiency of reject options in two examples.
Zusammenfassung
Verfahren des maschinellen Lernens werden etwa zur automatisierten Klassifikation in der Prozessautomation oder vorhersagenden Wartung eingesetzt. Prominente Techiken wie tiefes Lernen oder Random Forests leiden allerdings an ihrer Black Box-Charakteristik, die sie gegen Attacken angreifbar macht. Im vorliegenden Beitrag betrachten wir eine populäre Alternative, nämlich lernende Vektorquantisierung. Diese liefert eine interpretierbare Klassifikation und eine einfache Erweiterung zu expliziter Rückweisung unsicherer Klassifikationen aufgrund ihrer Charakteristik als sowohl diskriminatives als auch daten-repräsentierendes Verfahren. Wir leiten explizite Schranken her für notwendige Änderungen eines Beispiels, um die Klassifikation zu ändern, und wir demonstrieren das Verhalten für zwei Benchmarkanwendungen.
About the authors
Johannes Brinkrolf received his master’s degree from Bielefeld University in 2016. From April 2016 to September 2016 he was a research assistant at the South Westphalia University of Applied Science. Since October 2016 he has been a PhD student at the Cognitive Interaction Technology Center of Excellence at Bielefeld University.
Barbara Hammer received her Ph.D. in Computer Science in 1999 and her venia legendi in Computer Science in 2003, both from the University of Osnabrueck, Germany. From 2000-2004, she was chair of the junior research group ‘Learning with Neural Methods on Structured Data’ at the University of Osnabrueck before accepting an offer as professor for Theoretical Computer Science at Clausthal University of Technology, Germany, in 2004. Since 2010, she holds a professorship for Theoretical Computer Science for Cognitive Systems at the CITEC cluster of excellence at Bielefeld University, Germany. Several research stays have taken her to Italy, the U.K., India, France, the Netherlands, and the U.S.A. Her areas of expertise include hybrid systems, self-organizing maps, clustering, and recurrent networks as well as applications in bioinformatics, industrial process monitoring, or cognitive science. She chaired the IEEE CIS Technical Committee on Data Mining in 2013/2014, and she is chair of the Fachgruppe Neural Networks of the GI and vice-chair of the GNNs. She has published more than 200 contributions to international conferences / journals, and she is coauthor/editor of four books.
References
1. Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz, A Public Domain Dataset for Human Activity Recognition using Smartphones, in: 21st European Symposium on Artificial Neural Networks, ESANN 2013, Bruges, Belgium, April 24–26, 2013, 2013.Search in Google Scholar
2. Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok, Synthesizing Robust Adversarial Examples, CoRR abs/1707.07397 (2017).Search in Google Scholar
3. Michael Biehl, Barbara Hammer, Frank-Michael Schleif, Petra Schneider, and Thomas Villmann, Stationarity of Matrix Relevance LVQ, in: IJCNN, 2015.10.1109/IJCNN.2015.7280441Search in Google Scholar
4. Michael Biehl, Barbara Hammer, and Thomas Villmann, Prototype-based models in machine learning, WIREs Cognitive Science 7(2) (2016), 92–111.10.1002/wcs.1378Search in Google Scholar PubMed
5. Kerstin Bunte, Petra Schneider, Barbara Hammer, Frank-Michael Schleif, Thomas Villmann, and Michael Biehl, Limited Rank Matrix Learning, discriminative dimension reduction and visualization, Neural Networks 26 (2012), 159–173.10.1016/j.neunet.2011.10.001Search in Google Scholar PubMed
6. Davide Castelvecchi, Can we open the black box of AI?, Nature 538 (2016), 20–23.10.1038/538020aSearch in Google Scholar PubMed
7. A. Chandiok and D. K. Chaturvedi, Machine learning techniques for cognitive decision making, in: 2015 IEEE Workshop on Computational Intelligence: Theories, Applications and Future Directions (WCI), pp. 1–6, Dec 2015.10.1109/WCI.2015.7495529Search in Google Scholar
8. C. Chow, On Optimum Recognition Error and Reject Tradeoff, IEEE Trans. Inf. Theor. 16(1) (2006), 41–46.10.1109/TIT.1970.1054406Search in Google Scholar
9. B. Cline, R. S. Niculescu, D. Huffman, and B. Deckel, Predictive maintenance applications for machine learning, in: 2017 Annual Reliability and Maintainability Symposium (RAMS), pp. 1–7, Jan 2017.10.1109/RAM.2017.7889679Search in Google Scholar
10. G. Ditzler, M. Roveri, C. Alippi, and R. Polikar, Learning in Nonstationary Environments: A Survey, IEEE Computational Intelligence Magazine 10(4) (2015), 12–25.10.1109/MCI.2015.2471196Search in Google Scholar
11. L. Fischer, B. Hammer, and H. Wersing, Efficient rejection strategies for prototype-based classification, Neurocomputing 169 (2015), 334–342.10.1016/j.neucom.2014.10.092Search in Google Scholar
12. Lydia Fischer, Barbara Hammer, and Heiko Wersing, Optimal local rejection for classifiers, Neurocomputing 214 (2016), 445–457.10.1016/j.neucom.2016.06.038Search in Google Scholar
13. Alexander Geppert and Barbara Hammer, Incremental learning algorithms and applications, in: ESANN, 2016.Search in Google Scholar
14. Javier González-Jiménez, Javier G. Monroy, and José-Luis Blanco, The Multi-Chamber Electronic Nose–An Improved Olfaction Sensor for Mobile Robotics, Sensors 11(6) (2011), 6145–6164.10.3390/s110606145Search in Google Scholar PubMed PubMed Central
15. I. J. Goodfellow, J. Shlens, and C. Szegedy, Explaining and Harnessing Adversarial Examples, ArXiv e-prints (2014).Search in Google Scholar
16. Radu Herbei and Marten H. Wegkamp, Classification with Reject Option, The Canadian Journal of Statistics / La Revue Canadienne de Statistique 34(4) (2006), 709–721.10.1002/cjs.5550340410Search in Google Scholar
17. Joel Janai, Fatma Güney, Aseem Behl, and Andreas Geiger, Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art, CoRR abs/1704.05519 (2017).10.1561/9781680836899Search in Google Scholar
18. T. Kohonen, M. R. Schroeder, and T. S. Huang (eds.), Self-Organizing Maps, 3rd ed, Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2001.Search in Google Scholar
19. A. L’Heureux, K. Grolinger, H. F. Elyamany, and M. A. M. Capretz, Machine Learning With Big Data: Challenges and Approaches, IEEE Access 5 (2017), 7776–7797.10.1109/ACCESS.2017.2696365Search in Google Scholar
20. Zachary Chase Lipton, The Mythos of Model Interpretability, CoRR abs/1606.03490 (2016).Search in Google Scholar
21. Viktor Losing, Barbara Hammer, and Heiko Wersing, Interactive Online Learning for Obstacle Classification on a Mobile Robot, in: IJCNN, 2015.10.1109/IJCNN.2015.7280610Search in Google Scholar
22. Viktor Losing, Barbara Hammer, and Heiko Wersing, Choosing the best algorithm for an incremental on-line learning task, in: ESANN, 2016.Search in Google Scholar
23. Viktor Losing, Barbara Hammer, and Heiko Wersing, Tackling heterogeneous concept drift with the Self-Adjusting Memory (SAM), Knowl. Inf. Syst. 54(1) (2018), 171–201.10.1007/s10115-017-1137-ySearch in Google Scholar
24. Anh Mai Nguyen, Jason Yosinski, and Jeff Clune, Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images, CoRR abs/1412.1897 (2014).10.1109/CVPR.2015.7298640Search in Google Scholar
25. Matthias Richter, Thomas Längle, and Jürgen Beyerer, Knowing when you don’t: Bag of visual words with reject option for automatic visual inspection of bulk materials, in: 23rd International Conference on Pattern Recognition, ICPR 2016, Cancún, Mexico, December 4–8, 2016, pp. 3079–3084, 2016.10.1109/ICPR.2016.7900107Search in Google Scholar
26. P. Schneider, M. Biehl, and B. Hammer, Adaptive relevance matrices in learning vector quantization, Neural Computation 21 (2009), 3532–3561.10.1162/neco.2009.11-08-908Search in Google Scholar PubMed
27. Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, New York, NY, USA, 2014.10.1017/CBO9781107298019Search in Google Scholar
28. Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Mung Chiang, and Prateek Mittal, DARTS: Deceiving Autonomous Cars with Toxic Signs, Cornell University Library, Report no. arXiv:1802.06430, 2017.Search in Google Scholar
29. J. Su, D. Vasconcellos Vargas, and S. Kouichi, One pixel attack for fooling deep neural networks, ArXiv e-prints (2017).10.1109/TEVC.2019.2890858Search in Google Scholar
30. Lucia Ureche, Keisuke Umezawa, Yoshihiko Nakamura, and Aude Billard, Task Parameterization Using Continuous Constraints Extracted From Human Demonstrations, IEEE Transactions on Robotics 31(6) (2015), 1458–1471.10.1109/TRO.2015.2495003Search in Google Scholar
31. Kush R. Varshney and Homa Alemzadeh, On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products, CoRR abs/1610.01256 (2016).10.1089/big.2016.0051Search in Google Scholar
32. Marten Wegkamp and Ming Yuan, Support vector machines with a reject option, Bernoulli 17(4) (2011), 1368–1385.10.3150/10-BEJ320Search in Google Scholar
© 2018 Walter de Gruyter GmbH, Berlin/Boston
