Niclas Feldkamp
Soeren Bergmann
Steffen Strassburger
Skip Abstract Section
Abstract
This article provides a comprehensive and in-depth overview of our work on knowledge discovery in simulations. Application-wise, we focus on manufacturing simulations. Specifically, we propose and discuss a methodology for designing, executing, and analyzing large-scale simulation experiments with a broad coverage of possible system behavior targeted at generating knowledge about the system. Based on the concept of data farming, we suggest a two-phase process which starts with a data generation phase, in which a smart experiment design is used to set up and efficiently execute a large number of simulation experiments. In the second phase, the knowledge discovery phase, data mining and visually aided analysis methods are applied on the gathered simulation input and output data. This article gives insights into this knowledge discovery phase by discussing different machine learning approaches and their suitability for different manufacturing simulation problems. With this, we provide guidelines on how to conduct knowledge discovery studies within the manufacturing simulation context. We also introduce different case studies, both academic and applied, and use them to validate our methodology.
- R. Arias-Hernandez, L. T. Kaastra, T. M. Green, and B. Fisher. 2011. Pair analytics: Capturing reasoning processes in collaborative visual analytics. In Proceedings of the 44th Hawaii International Conference on System Sciences. IEEE, Washington DC, 1–10. DOI:https://doi.org/10.1109/HICSS.2011.339Google Scholar
- Michael Behrisch, Dirk Streeb, Florian Stoffel, Daniel Seebacher, Brian Matejek, Stefan H. Weber, Sebastian Mittelstaedt, Hanspeter Pfister, and Daniel Keim. 2018. Commercial visual analytics systems - Advances in the big data analytics field. IEEE Transactions on Visualization and Computer Graphics. DOI:https://doi.org/10.1109/TVCG.2018.2859973Google Scholar
- Michael R. Berthold, Nicolas Cebron, Fabian Dill, Thomas R. Gabriel, Tobias Kötter, Thorsten Meinl, Peter Ohl, Kilian Thiel, and Bernd Wiswedel. 2009. KNIME - The Konstanz information miner. ACM SIGKDD Explorations Newsletter 11, 1 (2009), 26. DOI:https://doi.org/10.1145/1656274.1656280Google ScholarDigital Library
- Christopher M. Bishop. 2009. Pattern recognition and machine learning (8th). Information Science and Statistics. Springer, New York.Google Scholar
- Thomas M. Cioppa and Thomas W. Lucas. 2007. Efficient nearly orthogonal and space-filling Latin hypercubes. Technometrics 49, 1 (2007), 45--55. DOI:https://doi.org/10.1198/004017006000000453Google ScholarCross Ref
- Çağatay Demiralp, Peter J. Haas, Srinivasan Parthasarathy, and Tejaswini Pedapati. 2017. Foresight: Rapid data exploration through guideposts. In Proceedings of the DSIA Workshop at IEEE VIS.Google Scholar
- Filip K. Dosilovic, Mario Brcic, and Nikica Hlupic. 2018. Explainable artificial intelligence: A survey. In Proceedings of the 41st International Convention on Information and Communication Technology, Electronics and Microelectronics. IEEE, 210--215. DOI:https://doi.org/10.23919/MIPRO.2018.8400040Google ScholarCross Ref
- Geoffry Ellis and Alan Dix. 2007. A taxonomy of clutter reduction for information visualisation. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1216--1223. DOI:https://doi.org/10.1109/TVCG.2007.70535Google ScholarDigital Library
- Usama M. Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth. 1996. From data mining to knowledge discovery in databases. AI Magazine 17, 37--54.Google ScholarDigital Library
- Niclas Feldkamp, Soeren Bergmann, and Steffen Strassburger. 2015. Knowledge discovery in manufacturing simulations. In Proceedings of the 3rd ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (SIGSIM PADS’15). ACM, New York, 3--12. DOI:https://doi.org/10.1145/2769458.2769468Google ScholarDigital Library
- Niclas Feldkamp, Soeren Bergmann, and Steffen Strassburger. 2015. Visual analytics of manufacturing simulation data. In Proceedings of the 2015 Winter Simulation Conference. IEEE, Piscataway, N.J., 779--790.Google ScholarDigital Library
- Niclas Feldkamp, Soeren Bergmann, and Steffen Strassburger. 2017. Online analysis of simulation data with stream-based data mining. In Proceedings of the 2017 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (SIGSIM-PADS'17). ACM, New York, 241--248. DOI:https://doi.org/10.1145/3064911.3064915Google ScholarDigital Library
- Niclas Feldkamp, Soeren Bergmann, Steffen Strassburger, Erik Borsch, Magnus Richter, and Rainer Souren. 2018. Combining data farming and data envelopment analysis for measuring productive efficiency in Manufacturing Simulations. In Proceedings of the 2018 Winter Simulation Conference. IEEE, Piscataway, N.J.Google ScholarDigital Library
- Niclas Feldkamp, Soeren Bergmann, Steffen Strassburger, and Thomas Schulze. 2017. Knowledge discovery and robustness analysis in manufacturing simulations. In Proceedings of the 2017 Winter Simulation Conference. IEEE.Google ScholarDigital Library
- Niclas Feldkamp, Soeren Bergmann, Steffen Strassburger, Thomas Schulze, Praneeth Akondi, and Marco Lemessi. 2017. Knowledge discovery in simulation data – A case study for a backhoe assembly line. In Proceedings of the 2017 Winter Simulation Conference. IEEE, 4456--4458.Google ScholarCross Ref
- Niclas Feldkamp, Sören Bergmann, Steffen Strassburger, and Thomas Schulze. 2016. Knowledge discovery in simulation data: A case study of a gold mining facility. In Proceedings of the 2016 Winter Simulation Conference. IEEE, Piscataway, N. J. 1607--1618. DOI:https://doi.org/10.1109/WSC.2016.7822210Google ScholarCross Ref
- Leonardo Feltrin. 2015. KNIME an open source solution for predictive analytics in the geosciences [software and data sets]. IEEE Geoscience and Remote Sensing Magazine 3, 4 (2015), 28--38. DOI:https://doi.org/10.1109/MGRS.2015.2496160Google ScholarCross Ref
- Philippe J. Giabbanelli. 2010. Impact of complex network properties on routing in backbone networks. In Proceedings of the 2010 IEEE Globecom Workshops // 2010 IEEE Globecom Workshops (GC'10). Workshops: Dec. 5, 2010 to Dec. 10, 2010 in Miami, Florida. IEEE, Piscataway, N.J., 389--393. DOI:https://doi.org/10.1109/GLOCOMW.2010.5700347Google Scholar
- R. R. Hill, J. O. Miller, and G. A. McIntyre. 2001. Applications of discrete event simulation modeling to military problems. In Proceedings of the 2001 Winter Simulation Conference. IEEE, Piscataway, N.J., 780--788. DOI:https://doi.org/10.1109/WSC.2001.977367Google Scholar
- Gary Horne, Bernt Åkesson, Ted Meyer. 2014. Data farming in support of NATO. Final Report of Task Group MSG-088. STO technical report, TR-MSG-088. North Atlantic Treaty Organisation, Neuilly-sur-Seine Cedex.Google Scholar
- Gary E. Horne and Ted E. Meyer. 2005. Data farming: Discovering surprise. In Proceedings of the 2005 Winter Simulation Conference. IEEE, Piscataway, N.J., 1082--1087.Google Scholar
- D. Kallfass and T. Schlaak. 2012. NATO MSG-088 case study results to demonstrate the benefit of using data farming for military decision support. In Proceedings of the 2012 Winter Simulation Conference (WSC 2012). IEEE, Piscataway, N. J. 1--12. DOI:https://doi.org/10.1109/WSC.2012.6465132Google Scholar
- Daniel A. Keim. 2002. Information visualization and visual data mining. IEEE Transactions on Visualization and Computer Graphics 8, 1 (2002), 1--8. DOI:https://doi.org/10.1109/2945.981847Google ScholarDigital Library
- Daniel A. Keim, Florian Mansmann, Jörn Schneidewind, Jim Thomas, and Hartmut Ziegler. 2008. Visual analytics: Scope and challenges. In Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, Simeon Simoff, Michael H. Boehlen and Arturas Mazeika, Eds. Springer, Berlin.Google Scholar
- Jack P. C. Kleijnen, Susan M. Sanchez, Thomas W. Lucas, and Thomas M. Cioppa. 2005. State-of-the-art review: A user's guide to the brave new world of designing simulation experiments. INFORMS Journal on Computing 17, 3 (2005), 263--289.Google ScholarDigital Library
- Bum C. Kwon, Janu Verma, Peter J. Haas, and Cagatay Demiralp. 2017. Sampling for scalable visual analytics. IEEE Computer Graphics and Applications 37, 1 (2017), 100--108. DOI:https://doi.org/10.1109/MCG.2017.6Google ScholarDigital Library
- Averill M. Law. 2003. How to conduct a successful simulation study. In Proceedings of the 2003 Winter Simulation Conference. IEEE, Piscataway, N. J. 66--70. DOI:https://doi.org/10.1109/WSC.2003.1261409Google ScholarCross Ref
- Martin Luboschik, Stefan Rybacki, Roland Ewald, Benjamin Schwarze, Heidrun Schumann, and Adelinde M. Uhrmacher. 2012. Interactive visual exploration of simulator accuracy: A case study for stochastic simulation algorithms. In Proceedings of the 2012 Winter Simulation Conference. WSC ’12. IEEE., Piscataway, N.J.Google Scholar
- Martin Luboschik, Christian Tominski, Arne Bittig, Adelinde Uhrmacher, and H. Schumann. 2012. Towards interactive visual analysis of microscopic-level simulation data. In Proceedings of SIGRAD 2012. Interactive Visual Analysis of Data, 81. Linköping University Electronic Press, Linköpings, 91--94.Google Scholar
- Kresimir Matkovic, Denis Gračanin, Mario Jelović, and Helwig Hauser. 2015. Interactive visual analysis of large simulation ensembles. In Proceedings of the 2015 Winter Simulation Conference. IEEE., Piscataway, N.J., 517--528. DOI:https://doi.org/10.1109/WSC.2015.7408192Google ScholarDigital Library
- Ana Minanovic, Hrvoje Gabelica, and Zivko Krstic. 2014. Big data and sentiment analysis using KNIME: Online reviews vs. Social Media. In Proceedings of the 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE, 1464--1468. DOI:https://doi.org/10.1109/MIPRO.2014.6859797Google Scholar
- Brian L. Morgan, Harrison C. Schramm, Jerry R. Smith, Thomas W. Lucas, Mary L. McDonald, Paul J. Sánchez, Susan M. Sanchez, and Stephen C. Upton. 2018. Improving U.S. Navy campaign analyses with big data. Interfaces 48, 2 (2018), 130--146. DOI:https://doi.org/10.1287/inte.2017.0900Google ScholarDigital Library
- Christopher J. Nannini, Jeffrey A. Appleget, and Alejandro S. Hernandez. 2013. Game for peace: Progressive education in peace operations. The Journal of Defense Modeling and Simulation 10, 3 (2013), 283--296. DOI:https://doi.org/10.1177/1548512913477258Google ScholarCross Ref
- Susan M. Sanchez. 2014. Simulation experiments: Better data, not just big data. In Proceedings of the 2014 Winter Simulation Conference. IEEE, Piscataway, N. J. 805--816.Google ScholarCross Ref
- Susan M. Sanchez, Thomas W. Lucas, Paul J. Sanchez, Christopher J. Nannini, and Hong Wan. 2012. Designs for large‐scale simulation experiments, with applications to defense and homeland security. In Design and Analysis of Experiments: Special Designs and Applications. Klaus Hinkelmann, Ed. John Wiley 8 Sons, Inc., Hoboken, N.J., 413--442.Google Scholar
- Susan M. Sanchez and Hong Wan. 2009. Better than a petaflop: The power of efficient experimental design. In Proceedings of the 2009 Winter Simulation Conference (WSC 2009). (Austin, Texas: 13–16 December 2009). IEEE, Piscataway, N.J., 60--74. DOI:https://doi.org/10.1109/WSC.2009.5429316Google Scholar
- Johan Schubert, Ronnie Johansson, and Pontus Hörling. 2015. Skewed distribution analysis in simulation-based operation planning. In Proceedings of the 9th Operations Research and Analysis Conference.Google Scholar
- Danielle Soban, David Thornhill, Santosh Salunkhe, and Alastair Long. 2016. Visual analytics as an enabler for manufacturing process decision-making. Procedia CIRP 56, 209--214. DOI:https://doi.org/10.1016/j.procir.2016.10.056Google ScholarCross Ref
- Zhiwu Tang, Qing Xue, Meng Zhao, and Yang Wei. 2009. Decision tree algorithm for tank damage analysis in combat simulation tests. In Proceedings of the 9th International Conference on Electronic Measurement 8 Instruments (ICEMI 2009), 3-830-3–835. DOI:https://doi.org/10.1109/ICEMI.2009.5274185Google ScholarCross Ref
- Simon J. E. Taylor, Tamas Kiss, Anastasia Anagnostou, Gabor Terstyanszky, Peter Kacsuk, Joris Costes, and Nicola Fantini. 2018. The CloudSME simulation platform and its applications. A Generic Multi-cloud Platform for Developing and Executing Commercial Cloud-based Simulations. Future Generation Computer Systems 88, 524--539. DOI:https://doi.org/10.1016/j.future.2018.06.006Google ScholarCross Ref
- Hasan Tercan, Toufik Al Khawli, Urs Eppelt, Christian Büscher, Tobias Meisen, and Sabina Jeschke. 2016. Use of classification techniques to design laser cutting processes. Procedia 5CIRP6 52, 292--297. DOI:https://doi.org/10.1016/j.procir.2016.08.001Google Scholar
- J. J. Thomas and Kristin A. Cook. 2005. Illuminating the Path. Research and Development Agenda for Visual Analytics (1st). IEEE Computer Society, Los Alamitos, California.Google Scholar
- Luxmi Verma, S. Srinivasan, and Varun Sapra. 2014. Integration of rule based and case based reasoning system to support decision making. In Proceedings of the International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT). IEEE, 106--108. DOI:https://doi.org/10.1109/ICICICT.2014.6781260Google ScholarCross Ref
- Sigrid Wenzel, Jochen Bernhard, and Ulrich Jessen. 2003. Visualization for modeling and simulation: A taxonomy of visualization techniques for simulation in production and logistics. In Proceedings of the 2003 Winter Simulation Conference. IEEE., Piscataway, N.J.Google Scholar
- Mike Wu, Michael C. Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, and Finale Doshi Velez. 2018. Beyond Sparsity: Tree Regularization of Deep Models for Interpretability. AAAI.Google Scholar
- Zheguang Zhao, Lorenzo de Stefani, Emanuel Zgraggen, Carsten Binnig, Eli Upfal, and Tim Kraska. 2017. Controlling false discoveries during interactive data exploration. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD'17). ACM, New York, 527--540. DOI:https://doi.org/10.1145/3035918.3064019Google ScholarDigital Library
Index Terms
- Knowledge Discovery in Simulation Data
Recommendations
Knowledge Discovery in Manufacturing Simulations
SIGSIM PADS '15: Proceedings of the 3rd ACM SIGSIM Conference on Principles of Advanced Discrete SimulationDiscrete event simulation studies in a manufacturing context are a powerful instrument when modeling and evaluating processes of various industries. Usually simulation experts conduct simulation experiments for a predetermined system specification by ...
Online Analysis of Simulation Data with Stream-based Data Mining
SIGSIM-PADS '17: Proceedings of the 2017 ACM SIGSIM Conference on Principles of Advanced Discrete SimulationDiscrete event simulation is an accepted instrument for investigating the dynamic behavior of complex systems and evaluating processes. Usually simulation experts conduct simulation experiments for a predetermined system specification by manually ...
Knowledge Discovery and Data Visualization: Theories and Perspectives
This article reviews the literature in the search for the theories and perspectives of knowledge discovery and data visualization. The literature review highlights the overview of knowledge discovery; Knowledge Discovery in Databases KDD; Knowledge ...
Comments