Abstract
Visualization and visual analytic tools amplify one’s perception of data, facilitating deeper and faster insights that can improve decision making. For multidimensional data sets, one of the most common approaches of visualization methods is to map the data into lower dimensions. Scatterplot matrices (SPLOM) are often used to visualize bivariate relationships between combinations of variables in a multidimensional dataset. However, the number of scatterplots increases quadratically with respect to the number of variables. For high dimensional data, the corresponding enormous number of scatterplots makes data exploration overwhelmingly complex, thereby hindering the usefulness of SPLOM in human decision making processes. One approach to address this difficulty utilizes Graph-theoretic Scatterplot Diagnostic (Scagnostics) to automatically extract a subset of scatterplots with salient features and of manageable size with the hope that the data will be sufficient for improving human decisions. In this paper, we use Electroencephalogram (EEG) to observe brain activity while participants make decisions informed by scatterplots created using different visual measures. We focused on 4 categories of Scagnostics measures: Clumpy, Monotonic, Striated, and Stringy. Our findings demonstrate that by adjusting the level of difficulty in discriminating between data sets based on the Scagnostics measures, different parts of the brain are activated: easier visual discrimination choices involve brain activity mostly in visual sensory cortices located in the occipital lobe, while more difficult discrimination choices tend to recruit more parietal and frontal regions as they are known to be involved in resolving ambiguities. Our results imply that patterns of neural activity are predictive markers of which specific Scagnostics measures most assist human decision making based on visual stimuli such as ours.
Similar content being viewed by others
References
Albuquerque G, Eisemann M, Magnor M (2011) Perception-based visual quality measures. In: 2011 IEEE conference on visual analytics science and technology (VAST). IEEE, pp 13–20
Sedlmair M, Tatu A, Munzner T, Tory M (2012) A taxonomy of visual cluster separation factors. In: Computer graphics forum, vol 31. Wiley Online Library, pp 1335–1344
Sips M, Neubert B, Lewis JP, Hanrahan P (2009) Selecting good views of high-dimensional data using class consistency. In: Computer graphics Forum, vol 28. Wiley Online Library, pp 831–838
Lehmann DJ, Kemmler F, Zhyhalava T, Kirschke M, Theisel H (2015) Visualnostics: visual guidance pictograms for analyzing projections of high-dimensional data. In: Computer graphics forum, vol 34. Wiley Online Library, pp 291–300
Wilkinson L, Anand A, Grossman R (2005) Graph-theoretic scagnostics. In: IEEE Symposium on information visualization. INFOVIS 2005. IEEE, pp 157–164
Tatu A, Bak P, Bertini E, Keim D, Schneidewind J (2010) Visual quality metrics and human perception: an initial study on 2D projections of large multidimensional data. In: Proceedings of the international conference on advanced visual interfaces, pp 49–56
Harrison L, Yang F, Franconeri S, Chang R (2014) Ranking visualizations of correlation using Weber’s law. IEEE Trans Visual Comput Graphics 20(12):1943–1952
Rensink R, Baldridge G (2010) The visual perception of correlation in scatterplots. J Vis 10(7):975–975
Etemadpour R, Olk B, Linsen L (2014) Eye-tracking investigation during visual analysis of projected multidimensional data with 2D scatterplots. In: 2014 international conference on information visualization theory and applications (IVAPP). IEEE, pp 233–246
Koffka K (2013) Principles of Gestalt psychology, vol 44. Routledge
Rensink RA (2017) The nature of correlation perception in scatterplots. Psychon Bull Rev 24(3):776–797
Matute J, Telea AC, Linsen L (2017) Skeleton-based scagnostics. IEEE Trans Visual Comput Graphics 24(1):542–552
Kay M, Heer J (2015) Beyond Weber’s law: a second look at ranking visualizations of correlation. IEEE Trans Visual Comput Graphics 22(1):469–478
Sedlmair M, Aupetit M (2015) Data-driven evaluation of visual quality measures. In: Computer graphics forum, vol 34. Wiley Online Library, pp 201–210
Etemadpour R, Motta R, de Souza Paiva JG, Minghim R, De Oliveira MCF, Linsen L (2014) Perception-based evaluation of projection methods for multidimensional data visualization. IEEE Trans Visual Comput Graphics 21(1):81–94
Deco G, Romo R (2008) The role of fluctuations in perception. Trends Neurosci 31(11):591–598
Bode S, Sewell DK, Lilburn S, Forte JD, Smith PL, Stahl J (2012) Predicting perceptual decision biases from early brain activity. J Neurosci 32(36):12488–12498
Bode S, Bogler C, Haynes JD (2013) Similar neural mechanisms for perceptual guesses and free decisions. Neuroimage 65:456–465
Parsons L, Haque E, Liu H (2004) Subspace clustering for high dimensional data: a review. ACM SIGKDD Explorations Newsl 6(1):90–105
Assent I, Krieger R, Müller E, Seidl T (2007) VISA: visual subspace clustering analysis. ACM SIGKDD Explorations Newsl 9(2):5–12
Maimon, O Rokach L (2010) Introduction to knowledge discovery and data mining, data mining and knowledge discovery handbook. Springer, pp 1–15
Dang TN, Wilkinson L (2014a) Scagexplorer: exploring scatterplots by their scagnostics. In: 2014 IEEE pacific visualization symposium. IEEE, pp 73–80
Dang TN, Wilkinson L (2014) Transforming scagnostics to reveal hidden features. IEEE Trans Visual Comput Graphics 20(12):1624–1632
Shao L, Mahajan A, Schreck T, Lehmann DJ (2017) Interactive regression lens for exploring scatter plots. In: Computer graphics forum, vol 36. Wiley Online Library, pp 157–166
Wang Y, Wang Z, Liu T, Correll M, Cheng Z, Deussen O, Sedlmair M (2019) Improving the robustness of scagnostics. IEEE Trans Visual Comput Graphics 26(1):759–769
Pandey AV, Krause J, Felix C, Boy J, Bertini E (2016) Towards understanding human similarity perception in the analysis of large sets of scatter plots. In: Proceedings of the 2016 CHI conference on human factors in computing systems, pp 3659–3669
Aupetit M, Sedlmair M (2016) SepMe: 2002 new visual separation measures. In: 2016 IEEE pacific visualization symposium (PacificVis). IEEE, pp 1–8
Shao L, Schleicher T, Behrisch M, Schreck T, Sipiran I, Keim DA (2016) Guiding the exploration of scatter plot data using motif-based interest measures. J Vis Lang Comput 36:1–12
Lehmann DJ, Hundt S, Theisel H (2015) A study on quality metrics vs. human perception: can visual measures help us to filter visualizations of interest? It Inf Technol 57(1):11–21
Wang M, Arteaga D, He BJ (2013) Brain mechanisms for simple perception and bistable perception. Proc Natl Acad Sci 110(35):E3350–E3359
Shankar S, Kayser AS (2017) Perceptual and categorical decision making: goal-relevant representation of two domains at different levels of abstraction. J Neurophysiol 117(6):2088–2103
Bode S, He AH, Soon CS, Trampel R, Turner R, Haynes JD (2011) Tracking the unconscious generation of free decisions using uitra-high field fMRI. PLoS ONE 6(6):e21612
Meijer GT, Mertens PE, Pennartz CM, Olcese U, Lansink CS (2019) The circuit architecture of cortical multisensory processing: distinct functions jointly operating within a common anatomical network. Prog Neurobiol 174:1–15
Xing J, Andersen RA (2000) Models of the posterior parietal cortex which perform multimodal integration and represent space in several coordinate frames. J Cogn Neurosci 12(4):601–614
Roitman JD, Brannon EM, Platt ML (2012) Representation of numerosity in posterior parietal cortex. Front Integr Neurosci 6:25
Walsh V (2003) A theory of magnitude: common cortical metrics of time, space and quantity. Trends Cogn Sci 7(11):483–488
Lipton JS, Spelke ES (2003) Origins of number sense: large-number discrimination in human infants. Psychol Sci 14(5):396–401
Sato T, Murthy A, Thompson KG, Schall JD (2001) Search efficiency but not response interference affects visual selection in frontal eye field. Neuron 30(2):583–591
Zhang H, Eppes A, Beatty-Martínez A, Navarro-Torres C, Diaz MT (2018) Task difficulty modulates brain-behavior correlations in language production and cognitive control: behavioral and fMRI evidence from a phonological go/no-go picture-naming paradigm. Cogn Affect Behav Neurosci 18(5):964–981
Wilkinson L, Anand A, Urbanek MS (2018) Package ‘scagnostics’
Netzel R, Vuong J, Engelke U, O’Donoghue S, Weiskopf D, Heinrich J (2017) Comparative eye-tracking evaluation of scatterplots and parallel coordinates. Vis Inform 1(2):118–131
Eilbert J, Peters Z, Eliott FM, Stassun KG, Kunda M (2018) Shapes in scatterplots: comparing human visual impressions and computational metrics. In: CogSci
Edelsbrunner H, Kirkpatrick D, Seidel R (1983) On the shape of a set of points in the plane. IEEE Trans Inf Theory 29(4):551–559
Skiena SS (1998) The algorithm design manual, vol 2. Springer
Wilkinson L, Anand A, Grossman R (2006) High-dimensional visual analytics: interactive exploration guided by pairwise views of point distributions. IEEE Trans Visual Comput Graphics 12(6):1363–1372
Sedlmair M, Munzner T, Tory M (2013) Empirical guidance on scatterplot and dimension reduction technique choices. IEEE Trans Visual Comput Graphics 19(12):2634–2643
Sasane S, Schwabe L (2012) Decoding of EEG activity from object views: active detection vs. passive visual tasks. In: International conference on brain informatics. Springer, pp 277–287
Kay KN, Naselaris T, Prenger RJ, Gallant JL (2008) Identifying natural images from human brain activity. Nature 452(7185):352–355
Esch L, Dinh C, Larson E, Engemann D, Jas M, Khan S, Gramfort A, Hämäläinen MS (2019) MNE: software for acquiring, processing, and visualizing MEG/EEG data. From signals to dynamic cortical networks, Magnetoencephalography, pp 355–371
Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430
Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52(3/4):591–611
Tatler BW, Hayhoe MM, Land MF, Ballard DH (2011) Eye guidance in natural vision: reinterpreting salience. J Vis 11(5):5–5
Shomstein S, Behrmann M (2008) Object-based attention: strength of object representation and attentional guidance. Percept Psychophys 70(1):132–144
Patel SH, Azzam PN (2005) Characterization of N200 and P300: selected studies of the event-related potential. Int J Med Sci 2(4):147
Maguire MJ, Magnon G, Ogiela DA, Egbert R, Sides L (2013) The N300 ERP component reveals developmental changes in object and action identification. Dev Cogn Neurosci 5:1–9
Ellmore TM, Ng K, Reichert CP (2017) Early and late components of EEG delay activity correlate differently with scene working memory performance. PLoS ONE 12(10):e0186072
Mazerolle EL, D’Arcy RC, Marchand Y, Bolster RB (2007) ERP assessment of functional status in the temporal lobe: examining spatiotemporal correlates of object recognition. Int J Psychophysiol 66(1):81–92
Bledowski C, Prvulovic D, Hoechstetter K, Scherg M, Wibral M, Goebel R, Linden DE (2004) Localizing P300 generators in visual target and distractor processing: a combined event-related potential and functional magnetic resonance imaging study. J Neurosci 24(42):9353–9360
Liu T, Slotnick SD, Serences JT, Yantis S (2003) Cortical mechanisms of feature-based attentional control. Cereb Cortex 13(12):1334–1343
Scolari M, Seidl-Rathkopf KN, Kastner S (2015) Functions of the human frontoparietal attention network: evidence from neuroimaging. Curr Opin Behav Sci 1:32–39
Wang L, Li C, Wu J (2017) The status of research into intention recognition. Improving the quality of life for dementia patients through progressive detection, treatment, and care. IGI Global, pp 201–221
Read GL, Innis IJ (2017) Electroencephalography (EEG). Int Encyclopedia Commun Res Meth 1–18
Radhakrishnan R, Addy PH, Sewell RA, Skosnik PD, Ranganathan M, D’Souza DC (2014) Cannabis, cannabinoids, and the association with psychosis. The effects of drug abuse on the human nervous system. Elsevier, pp 423–474
Escera C, Yago E, Alho K (2001) Electrical responses reveal the temporal dynamics of brain events during involuntary attention switching. Eur J Neurosci 14(5):877–883
Picton TW, Stuss DT, Champagne SC, Nelson RF (1984) The effects of age on human event-related potentials. Psychophysiology 21(3):312–326
Mueller V, Brehmer Y, Von Oertzen T, Li SC, Lindenberger U (2008) Electrophysiological correlates of selective attention: a lifespan comparison. BMC Neurosci 9(1):18
Di Russo F, Martínez A, Sereno MI, Pitzalis S, Hillyard SA (2002) Cortical sources of the early components of the visual evoked potential. Hum Brain Mapp 15(2):95–111
Kutas M, Federmeier KD (2011) Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annu Rev Psychol 62:621–647
Menon V, Ford JM, Lim KO, Glover GH, Pfefferbaum A (1997) Combined event-related fMRI and EEG evidence for temporal-parietal cortex activation during target detection. NeuroReport 8(14):3029–3037
Krigolson OE, Cheng D, Binsted G (2015) The role of visual processing in motor learning and control: insights from electroencephalography. Vision Res 110:277–285
Kirino E, Belger A, Goldman-Rakic P, McCarthy G (2000) Prefrontal activation evoked by infrequent target and novel stimuli in a visual target detection task: an event-related functional magnetic resonance imaging study. J Neurosci 20(17):6612–6618
Friedman D, Cycowicz YM, Gaeta H (2001) The novelty P3: an event-related brain potential (ERP) sign of the brain’s evaluation of novelty. Neurosci Biobehav Rev 25(4):355–373
Comerchero MD, Polich J (1999) P3a and P3b from typical auditory and visual stimuli. Clin Neurophysiol 110(1):24–30
Nieuwenhuis S, De Geus EJ, Aston-Jones G (2011) The anatomical and functional relationship between the P3 and autonomic components of the orienting response. Psychophysiology 48(2):162–175
Hagen GF, Gatherwright JR, Lopez BA, Polich J (2006) P3a from visual stimuli: task difficulty effects. Int J Psychophysiol 59(1):8–14
Katayama J, Polich J (1998) Stimulus context determines P3a and P3b. Psychophysiology 35(1):23–33
Noppeney U, Price CJ, Penny WD, Friston KJ (2006) Two distinct neural mechanisms for category-selective responses. Cereb Cortex 16(3):437–445
Haxby JV, Gobbini MI, Montgomery K (2004) Spatial and temporal distribution of face and object representations in the human brain
Lafer-Sousa R, Conway BR (2013) Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex. Nat Neurosci 16(12):1870
Lüdtke J, Friedrich CK, De Filippis M, Kaup B (2008) Event-related potential correlates of negation in a sentence-picture verification paradigm. J Cogn Neurosci 20(8):1355–1370
Potts GF, Patel SH, Azzam PN (2004) Impact of instructed relevance on the visual ERP. Int J Psychophysiol 52(2):197–209
Bourisly AK, Shuaib A (2018) Neurophysiological effects of aging: a P200 ERP study. Transl Neurosci 9(1):61–66
Potts GF (2004) An ERP index of task relevance evaluation of visual stimuli. Brain Cogn 56(1):5–13
Madden DJ, Parks EL, Tallman CW, Boylan MA, Hoagey DA, Cocjin SB, Johnson MA, Yh Chou GG, Potter Nk Chen et al (2017) Frontoparietal activation during visual conjunction search: effects of bottom-up guidance and adult age. Hum Brain Mapp 38(4):2128–2149
Corbetta M, Patel G, Shulman GL (2008) The reorienting system of the human brain: from environment to theory of mind. Neuron 58(3):306–324
Makin AD, Wright D, Rampone G, Palumbo L, Guest M, Sheehan R, Cleaver H, Bertamini M (2016) An electrophysiological index of perceptual goodness. Cereb Cortex 26(12):4416–4434
Wright D, Makin AD, Bertamini M (2015) Right-lateralized alpha desynchronization during regularity discrimination: hemispheric specialization or directed spatial attention? Psychophysiology 52(5):638–647
Bertamini M, Rampone G, Oulton J, Tatlidil S, Makin AD (2019) Sustained response to symmetry in extrastriate areas after stimulus offset: an EEG study. Sci Rep 9(1):1–11
Jacobsen T, Klein S, Löw A (2018) The posterior sustained negativity revisited-an SPN reanalysis of Jacobsen and Höfel (2003). Symmetry 10(1):27
Kahneman D (1973) Attention and effort, vol 1063. Citeseer
Xu G, Wu Y, Li M (2020) The study of influence of sound on visual ERP-based brain computer interface. Sensors 20(4):1203
Polich J, Kok A (1995) Cognitive and biological determinants of P300: an integrative review. Biol Psychol 41(2):103–146
Saliasi E, Geerligs L, Lorist MM, Maurits NM (2013) The relationship between P3 amplitude and working memory performance differs in young and older adults. PLoS ONE 8(5):e63701
Friederici AD, Hahne A, Saddy D (2002) Distinct neurophysiological patterns reflecting aspects of syntactic complexity and syntactic repair. J Psycholinguist Res 31(1):45–63
Roldan SM (2017) Object recognition in mental representations: directions for exploring diagnostic features through visual mental imagery. Front Psychol 8:833
Vance J, Stokes D (2017) Noise, uncertainty, and interest: predictive coding and cognitive penetration. Conscious Cogn 47:86–98
Ransom M, Fazelpour S, Mole C (2017) Attention in the predictive mind. Conscious Cogn 47:99–112
Yip HMK, Cheung LY, Wong YK, Wong ACN (2020) The effect of task on object processing revealed by EEG decoding. bioRxiv
Voss JL, Federmeier KD (2011) FN400 potentials are functionally identical to N400 potentials and reflect semantic processing during recognition testing. Psychophysiology 48(4):532–546
Paller KA, Lucas HD, Voss JL (2012) Assuming too much from ‘familiar’ brain potentials. Trends Cogn Sci 16(6):313–315
Leynes PA, Batterman A, Abrimian A (2019) Expectations alter recognition and event-related potentials (ERPS). Brain Cogn 135:103573
Yang J, Long C (2020) Common and distinctive cognitive processes between categorization and category-based induction: evidence from event-related potentials. Brain Res 1749:147134
Kutas M, Federmeier KD (2009) N400. Scholarpedia 4(10):7790
Foxe JJ, Simpson GV (2002) Flow of activation from V1 to frontal cortex in humans. Exp Brain Res 142(1):139–150
Albuquerque G, Eisemann M, Lehmann DJ, Theisel H, Magnor M (2010 Improving the visual analysis of high-dimensional datasets using quality measures. In: 2010 IEEE symposium on visual analytics science and technology. IEEE, pp 19–26
Acknowledgements
The authors would like to thank Prof. Chris Weaver at University of Oklahoma for his initial helpful guidance and encouragement in designing this study about Scagnostic measures. We would like to also thank Dr. Chelsea P Reichert for her support and guides in analyzing the brain data. Finally, we thank students Arezoo Bybordi, Marino Echavarria, Vincent Filardi, Kenneth Ng, Chhewang Sherpa and Yu Xuan Huang at The City College of New York who helped us in implementation, conducting and facilitating the user study.
Funding
This work was supported by a grant from the PSC CUNY 61256-00 49.
Author information
Authors and Affiliations
Contributions
Ronak Etemadpour as the first author contributed in conceptualization, designing the study, gaining fund for the project, conducting the study, performing data analysis, data curation, administrating the project, data interpretation and drafting the manuscript. SS performed the data analysis, coding and programming, and partially drafting the manuscript. AS supervised experimental design, training sessions of neuroimaging protocols, quality assurance, and critical review and drafting of manuscript.
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no competing interests.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Etemadpour, R., Shintree, S. & Shereen, A.D. Brain Activity is Influenced by How High Dimensional Data are Represented: An EEG Study of Scatterplot Diagnostic (Scagnostics) Measures. J Healthc Inform Res 8, 19–49 (2024). https://doi.org/10.1007/s41666-023-00145-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41666-023-00145-2