Hostname: page-component-8448b6f56d-xtgtn Total loading time: 0 Render date: 2024-04-24T10:33:41.935Z Has data issue: false hasContentIssue false
  • English
  • Français

Segmentation of design protocol using EEG

Published online by Cambridge University Press:  03 April 2018

Philon Nguyen ,
Thanh An Nguyen  and
Yong Zeng
Philon Nguyen
Affiliation:
Concordia Institute for Information Systems Engineering, Concordia University, Montreal, Quebec, Canada
Thanh An Nguyen
Affiliation:
Department of Electrical Engineering, Concordia University, Montreal, Quebec, Canada
Yong Zeng*
Affiliation:
Concordia Institute for Information Systems Engineering, Concordia University, Montreal, Quebec, Canada
*
Author for correspondence: Yong Zeng, E-mail: yong.zeng@concordia.ca
Get access Rights & Permissions [Opens in a new window]

Abstract

Design protocol data analysis methods form a well-known set of techniques used by design researchers to further understand the conceptual design process. Verbal protocols are a popular technique used to analyze design activities. However, verbal protocols are known to have some limitations. A recurring problem in design protocol analysis is to segment and code protocol data into logical and semantic units. This is usually a manual step and little work has been done on fully automated segmentation techniques. Physiological signals such as electroencephalograms (EEG) can provide assistance in solving this problem. Such problems are typical inverse problems that occur in the line of research. A thought process needs to be reconstructed from its output, an EEG signal. We propose an EEG-based method for design protocol coding and segmentation. We provide experimental validation of our methods and compare manual segmentation by domain experts to algorithmic segmentation using EEG. The best performing automated segmentation method (when manual segmentation is the baseline) is found to have an average deviation from manual segmentations of 2 s. Furthermore, EEG-based segmentation can identify cognitive structures that simple observation of design protocols cannot. EEG-based segmentation does not replace complex domain expert segmentation but rather complements it. Techniques such as verbal protocols are known to fail in some circumstances. EEG-based segmentation has the added feature that it is fully automated and can be readily integrated in engineering systems and subsystems. It is effectively a window into the mind.

Keywords

Design cognitiondesign protocolelectroencephalography (EEG)physiological methodssegmentation
Type
Research Article
Information
AI EDAM , Volume 33 , Issue 1 , February 2019 , pp. 11 - 23
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akin, Ö (2009) Variants and invariants of design cognition. In McDonnell, J and Lloyd, P (eds). About: Designing-Analysing Design Meetings. London: CRC Press/Balkema, pp. 119–133.Google Scholar
Alexiou, K, Zamenopoulos, T, Johnson, J and Gilbert, S (2009) Exploring the neurological basis of design cognition using brain imaging: some preliminary results. Design Studies 30(6), 623–647.Google Scholar
Ball, L and Christensen, B (2009) Analogical reasoning and mental simulation in design: two strategies linked to uncertainty resolution. In McDonnell, J and Lloyd, P (eds). About: Designing – Analysing Design Meetings. London: CRC Press/Balkema, pp. 137–152.Google Scholar
Blankertz, B, Tangermann, M, Vidaurre, C, Dickhaus, T, Sannelli, C, Popescu, F, Fazli, S, Danczy, M, Curio, G and Müller, K (2010) Detecting mental states by machine learning techniques: the Berlin braincomputer interface. In Graimann, B, Allison, BZ and Pfurtscheller, G (eds). Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction. London: Springer, pp. 113–136.Google Scholar
Chiu, I and Shu, LH (2010) Potential limitations of verbal protocols in design experiments. In Proceedings of ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (DTM).Google Scholar
Cross, N, Christiaans, H and Dorst, K (1996) Introduction: the Delft protocols workshop. In Cross, N, Christiaans, H and Dorst, K (eds). Analyzing Design Activity. Chichester, NY: Wiley, pp. 1–14.Google Scholar
Dantec, CL and Do, E (2009) The mechanisms of value transfer in design meetings. In McDonnell, J and Lloyd, P (eds). About: Designing – Analyzing Design Meetings. London: CRC Press/Balkema, pp. 101–118.Google Scholar
Delorme, A and Makeig, S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics. Journal of Neuroscience Methods 134, 9–21.Google Scholar
Dong, A, Kleinsmann, M and Valkenburg, R (2009) Affect-in-cognition through the language of appraisal. In McDonnell, J and Lloyd, P (eds). About: Designing – Analysing Design Meetings. London: CRC Press/Balkema, pp. 119–133.Google Scholar
Enis, C and Gyeszly, S (1987) Protocol analysis of the engineering systems design process. Research in Engineering Design 3(1), 15–22.Google Scholar
Ericsson, KA and Simon, HA (1984) Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press.Google Scholar
Fallshore, M and Schooler, JW (1993) Verbal vulnerability of perceptual expertise. Journal of Experimental Psychology: Learning, Memory and Cognition 21(6), 1608–1623.Google Scholar
Freudiger, J (2003) Brain States Analysis for Direct Brain-computer Communication. Zürich, Switzerland: Swiss Federal Institute of Technology.Google Scholar
Gero, J and McNeill, T (1998) An approach to the analysis of design protocols. Design Studies 19(1), 21–61.Google Scholar
Gero, JS (1990) Design prototypes: a knowledge representation schema for design. AI Magazine 11(4), 26–36.Google Scholar
Goel, V (2014) Creative brains: designing in the real world. Frontiers in Human Neuroscience 8, 241.Google Scholar
Goel, V, Eimontaite, I, Goel, A and Schindler, I (2015) Differential modulation of performance in insight and divergent thinking tasks with tdcs. Journal of Problem Solving 8(1), 2.Google Scholar
Gordon, SE (1992) Implications of Cognitive Theory for Knowledge Acquisition. New York, NY: Springer Verlag.Google Scholar
Irmscher, WF (1987) Finding a Comfortable Identity. College Composition and Communication 38(1), 81–87.Google Scholar
Jaarsveld, S, Fink, A, Rinner, M, Schwab, D, Benedek, M and Lachmann, T (2015) Intelligence in creative processes: an EEG study. Intelligence 49, 171–178.Google Scholar
Jiang, H and Yen, C (2009) Protocol analysis in design research: a review. In IASDR (International Association of Societies of Design Research) 2009, October 18–22, 2009, Seoul, South Korea.Google Scholar
Kahneman, D (1973) Attention and Effort. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Kan, J and Gero, J (2008) Acquiring information from linkography in protocol studies of designing. Design Studies 29(4), 315–337.Google Scholar
Kan, J and Gero, J (2009) Using FBS ontology to capture semantic design information in design protocol studies. In McDonnell, J and Lloyd, P (eds). About: Designing – Analysing Design Meetings. London: CRC Press/Balkema, pp. 213–229.Google Scholar
Koenig, T, Lehmann, D, Merlo, M, Kochi, K, Hell, D and Koukkou, M (1999) A deviant EEG brain microstate in acute neuroleptic-naive schizophrenics at rest. European Archives of Psychiatry and Clinical Neuroscience 249, 205–211.Google Scholar
Koenig, T, Prischep, L, Lehmann, D, Sosa, P, Braeker, E, Kleinlogel, H, Isenhart, R and John, E (2002) Millisecond by millisecond, year by year: normative EEG microstates and developmental stages. NeuroImage 16, 41–48.Google Scholar
Kuusela, H and Paul, P (2000) A comparison of concurrent and retrospective verbal protocol analysis. American Journal of Psychology 113(3), 387–404.Google Scholar
Lehmann, D (1990) Brain electric microstates and cognition: the atoms of thought. In John, ER (ed.). Machinery of the Mind. Boston: Birkhaüser, pp. 209–224.Google Scholar
Lehmann, D, Ozaki, H and Pal, I (1987) EEG alpha map series: brain micro-states by space-oriented adaptive segmentation. Electroencephalography and Clinical Neurophysiology 67, 271–288.Google Scholar
Matthews, B (2009) Intersections of brainstorming rules and social order. In McDonnell, J and Lloyd, P (eds). About: Designing – Analysing Design Meetings. London: CRC Press/Balkema, pp. 137–152.Google Scholar
McNeill, T, Gero, J and Warren, J (1998) Understanding conceptual electronic design using protocol analysis. Research in Engineering Design 10(3), 129–140.Google Scholar
Metcalfe, J (1986) Premonitions of insight predict impending error. Journal of Experimental Psychology: Learning, Memory, and Cognition 12, 623–634.Google Scholar
Michel, C, Koenig, T, Brandeis, D, Gianotti, L and Wackermann, J (2009) Electrical Neuroimaging. Medecine. Boston, MA: Cambridge University Press.Google Scholar
Moriguchi, A, Otsuka, A, Kohara, K, Mikami, H, Katahira, K, Tsunetoshi, T, Higashimori, K, Ohishi, M, Yo, Y and Ogihara, T (1992) Spectral changes in heart rate variability in response to mental arithmetic before and after beta-adrenoceptor blocker. Clinical Autonomic Research 2(4), 267–270.Google Scholar
NASA (1986) Nasa Task Load Index (TLX) v. 1.0 Manual.Google Scholar
Nguyen, P, Nguyen, TA and Zeng, Y (2015) Physiologically based segmentation of design protocol. In Proceedings of the International Conference on Engineering Design (ICED).Google Scholar
Nguyen, TA and Zeng, Y (2010) Analysis of design activities using EEG signals. In Proceedings of the ASME 2010 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Montreal, 15–18 August 2010.Google Scholar
Nguyen, TA and Zeng, Y (2012) Clustering designers mental activities based on EEG power. In Proceedings of Tools and Methods of Competitive Engineering (TMCE) 2012, May 711, 2012, Karlsruhe, Germany.Google Scholar
Nguyen, TA and Zeng, Y (2014) A physiological study of relations between designers mental effort and mental stress during conceptual design. Computer Aided Design 54, 3–18.Google Scholar
Nguyen, TA and Zeng, Y (2016) Effects of stress and effort on self-rated reports in experimental study of design activities. Journal of Intelligent Manufacturing 28(7), 1609–1622.Google Scholar
Pascual-Marqui, R, Michel, CM and Lehmann, D (1995) Segmentation of brain electrical activity into microstates: model estimation and validation. IEEE Transactions on Biomedical Engineering 42(7), 658–665.Google Scholar
Rasmussen, J and Jensen, A (1991) Mental procedures in real life tasks. A case study in electronics trouble shooting. Ergonomics 17, 293–330.Google Scholar
Rosé, C, Wang, Y, Cui, Y, Arguello, J, Stegmann, K, Weinberger, A and Fisher, F (2008) Analyzing collaborative learning processes automatically: exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning 3(3), 237–271.Google Scholar
Schooler, JW and Engstler-Schooler, TY (1993) Verbal overshadowing of visual memories: some things are better left unsaid. Cognitive Psychology 17, 31–71.Google Scholar
Schooler, JW, Ohlsson, S and Brooks, K (1993) Thoughts beyond words: when language overshadows insight. Journal of Experimental psychology: General 122, 166–183.Google Scholar
Schooler, JW, Ryan, R and Reder, LM (1991) Better the second time around: Representation reverses verbalization's impairment of face recognition. In International Conference on Memory.Google Scholar
Smagorinsky, P (1989) The reliability and validity of protocol analysis. Written Communication 6(4), 463–479.Google Scholar
Stacey, M, Eckert, C and Earl, C (2009) From Ronchamp by sledge: on the pragmatics of object references. In McDonnell, J and Lloyd, P (eds). About: Designing – Analysing Design Meetings. London: CRC Press/Balkema, pp. 361–380.Google Scholar
Suwa, M and Twersky, B (1997) What do architects and students perceive in their design sketches? A protocol analysis. Design Studies 18(4), 385–403.Google Scholar
Tang, Y and Zeng, Y (2009) Quantifying designer's mental stress in the conceptual design process using kinesics study. In Proceedings of the 17th international conference on engineering design.Google Scholar
Ullman, D, Stauffer, L and Dietterich, T (1987) Preliminary results of and experimental study of the mechanical design process. In Proceedings of the NSF Workshop on Design Theory and Methodology.Google Scholar
Visser, W (2008) The function of gesture in an architectural design meeting. In McDonnell, J and Lloyd, P (eds). About: Designing – Analysing Design Meetings. London: CRC Press/Balkema.Google Scholar
Völker, J, Hitzler, P and Cimiano, P (2012) Acquisition of owl dl axioms from lexical resources. In Proceedings of the 4th European conference on The Semantic Web, pp. 670–685.Google Scholar
Wilson, TD (1984) The proper protocol: validity and completeness of verbal reports. Psychological Science 5, 249–252.Google Scholar
Wilson, TD, Lisle, DJ, Schooler, JW, Hodges, SD, Klaaren, KJ and Lafleur, SJ (2013) Introspecting about reasons can reduce post-choice satisfaction. Personality and Social Psychology Bulletin 19(3), 331–339.Google Scholar
Wilson, TD and Schooler, JW (1991) Thinking too much: introspection can reduce the quality of preferences and decisions? Journal of Personality and Social Psychology 60, 181–192.Google Scholar
Wong, W, Lieu, W and Bennamoun, M (2012) Ontology learning from text: a look back and into the future. ACM Computing Surveys 44(4), 20–36.Google Scholar