Skip to main content

Advertisement

Springer Nature Link
Log in

Toward tangibles and display-rich interfaces for co-located and distributed genomics collaborations

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

We discuss and present design probes investigating how pervasive displays could offer unique opportunities for enhancing discovery and learning with “big data.” Our collaboration across three universities undertook a series of design exercises investigating approaches for collaborative, interactive, tangibles, and multitouch-engaged visualizations of genomic and related scientific datasets. These exercises led to several envisionments of tangible interfaces that employ active tokens and interactive surfaces to facilitate co-located and distributed engagement with large datasets. We describe some of the motivation and background for these envisioned interfaces; consider key aspects linking and distinguishing the designs; and relate these to the present and near-future state of the art for tangible and multitouch engagement with pervasive displays toward collaborative science.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. This manuscript draws in significant part from a same-authored ACM Pervasive Displays 2019 conference proceedings

References

  1. The Genomes Project, C., et al., An integrated map of genetic variation from 1,092 human genomes. Nature, 2012. 491: p. 56

  2. Garvey WD (2014) Communication: the essence of science: facilitating information exchange among librarians, scientists, engineers and students. Elsevier. ISBN 9781483182070

  3. Due Credit (2013) Nature 496(270)

  4. Collaboration, A et al (2015) Combined measurement of the Higgs boson mass in pp collisions at sqrt(s)=7 and 8 TeV with the ATLAS and CMS experiments. Phys Rev Lett 114(19):191803

    Article  Google Scholar 

  5. Collaboration, L.S et al (2016) Observation of gravitational waves from a binary black hole merger. Phys Rev Lett 116(6):061102

    Article  MathSciNet  Google Scholar 

  6. Genomes Consortium (2010) A map of human genome variation from population-scale sequencing. Nature 467(7319):1061–1073

    Article  Google Scholar 

  7. Konkel MK, Ullmer B, Shaer O, Mazalek A (2019) Envisioning tangibles and display-rich interfaces for co-located and distributed genomics collaborations. In: Proc. of Pervasive Displays, vol 2019

    Google Scholar 

  8. Marshall E (2001) Bermuda rules: community spirit, with teeth. Science 291(5507):1192–1192

    Article  Google Scholar 

  9. Sharing data from large-scale biological research projects: a system of tripartite responsibility. (Wellcome Trust, 2003); available at http://www.wellcome.ac.uk/stellent/groups/corporatesite/@policy_communications/documents/web_document/wtd003207.pdf

  10. Toronto International Data Release Workshop, A (2009) Prepublication data sharing. Nature 461:168

    Article  Google Scholar 

  11. Nanda S, Kowalczuk MK (2014) Unpublished genomic data–how to share? BMC Genomics 15(1):5

    Article  Google Scholar 

  12. Koboldt D (2013) Data sharing, embargo, and big science. MassGenomics Available from: http://massgenomics.org/2013/06/data-sharing-embargo.html

  13. Siegel, V. The logic of journal embargoes: why we have to wait for scientific news. The Conversation 2016; Available from: http://theconversation.com/the-logic-of-journal-embargoes-why-we-have-to-wait-for-scientific-news-53677

  14. Shaer O et al (2013) From big data to insights: opportunities and challenges for TEI in genomics. In: Proc. of TEI’13, pp 109–116

    Google Scholar 

  15. Baldi P et al (2011) Countering GATTACA: efficient and secure testing of fully-sequenced human genomes. In: Proceedings of the 18th ACM conference on Computer and communications security. ACM, Chicago, pp 691–702

    Chapter  Google Scholar 

  16. Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. D. . 3D tangibles facilitate joint visual attention in dyads. in Proc. of CSCL 2015. 2015

  17. Schneider B et al (2011) Benefits of a tangible interface for collaborative learning and interaction. IEEE Trans Learn Technol 4(3):222–232

    Article  Google Scholar 

  18. Klemmer SR et al (2001) The designers’ outpost: a tangible interface for collaborative web site. In: Proc. of UIST’01, pp 1–10

    Google Scholar 

  19. Shaer, O. and E. Hornecker, Tangible user interfaces: past, present, and future directions. Found. Trends Hum.-Comput. Interact., 2010. 3(1--2): p. 1--137

  20. Everitt KM et al (2003) Two worlds apart: bridging the gap between physical and virtual media for distributed design collaboration. In: Proc. of CHI ‘03, pp 553–560

    Google Scholar 

  21. Ullmer, B., Tangible interfaces for manipulating aggregates of digital information. 2002, Massachusetts Institute of Technology

  22. Watson, J.D., The double helix: a personal account of the discovery of the structure of DNA. 1968: Antheneum

  23. Iwasa J (2010) Animating the model figure. Trends Cell Biol 20(12):699–704

    Article  Google Scholar 

  24. Myers N (2008) Molecular embodiments and the body-work of modeling in protein crystallography. Soc Stud Sci 38(2):163–199

    Article  Google Scholar 

  25. Höst GE, Larsson C, Olson A, Tibell LA (2013) Student learning about biomolecular self-assembly using two different external representations. CBE—Life Sciences Education 12(3):471–482

    Article  Google Scholar 

  26. Dori YJ, Miri AB (2001) Virtual and physical molecular modeling: fostering model perception and spatial understanding. J Educ Technol Soc 4(1):61–74

    Google Scholar 

  27. Valdes C et al (2014) Exploring the design space of gestural interaction with active tokens through user-defined gestures. In: Proc. of CHI’14, pp 4107–4116

    Google Scholar 

  28. Grote C et al (2015) Eugenie: multi-touch and tangible interaction for bio-design. In: Proc. of TEI’15

    Google Scholar 

  29. Bartindale T, Harrison C (2009) Stacks on the surface: resolving physical order using fiducial markers with structured transparency. In: Proc. of ITS’09, pp 57–60

    Google Scholar 

  30. Agarawala A, Balakrishnan R (2006) Keepin’it real: pushing the desktop metaphor with physics, piles and the pen. In: Proc. of CHI’06, pp 1283–1292

    Google Scholar 

  31. Hornecker E et al (2008) TEI goes on: tangible and embedded interaction. IEEE Pervasive Computing 7(2):91–96

    Google Scholar 

  32. Mazalek A, Van den Hoven E (2009) Framing tangible interaction frameworks. AI EDAM 23(3):225–235

    Google Scholar 

  33. Ishii, H. and B. Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, in Proc. of CHI’97. 1997. p. 234–241

  34. Ullmer B et al (2019) Tangible and embodied interaction. ACM Books

  35. Drucker SM et al (2013) TouchViz: a case study comparing two interfaces for data analytics on tablets. In: Proc. of CHI’13, pp 2301–2310

    Google Scholar 

  36. Vogel, D. and P. Baudisch, Shift: a technique for operating pen-based interfaces using touch, in Proc. of CHI’07. 2007 p. 657–666

  37. Voida, S., et al., Getting practical with interactive tabletop displays: designing for dense data, fat fingers, diverse interactions, and face-to-face collaboration, in Proc. of ITS’09. 2009. p. 109–116

  38. Block F et al (2012) The DeepTree exhibit: visualizing the tree of life to facilitate informal learning. IEEE Transactions on Visualization and Computer Graphics 18(12):2789–2798

    Article  Google Scholar 

  39. Isenberg P et al (2013) Data visualization on interactive surfaces: a research agenda. IEEE Computer Graphics and Applications 33(2):16–24

    Article  Google Scholar 

  40. Zigelbaum, J., et al., The tangible video editor: collaborative video editing with active tokens, in Proc. of TEI’07. 2007. p. 43--46

  41. Mazalek A et al (2014) Tangible meets gestural: gesture based interactions with active tokens, in Gesture-based interaction design: communication and cognition, CHI. Workshop, p 2014

  42. Ullmer B, Ishii H (2000) Emerging frameworks for tangible user interfaces. IBM Syst J 39(3):915–931

    Article  Google Scholar 

  43. Ishii, H., Tangible bits: beyond pixels, in Proc. of TEI’08. 2008. p. xv--xxv

  44. Weiser M (1991) The computer for the 21st century. Sci Am 272(3)

  45. Rekimoto, J. and M. Saitoh, Augmented surfaces: a spatially continuous work space for hybrid computing environments, in Proc. of CHI ‘99. 1999. p. 378--385

  46. Chin L et al (2011) Making sense of cancer genomic data. Genes & Development 25(6):534–555

    Article  Google Scholar 

  47. Chen R, Mias GI, Li-Pook-Than J, Jiang L, Lam HY, Chen R, Miriami E, Karczewski KJ, Hariharan M, Dewey FE, Cheng Y, Clark MJ, Im H, Habegger L, Balasubramanian S, O’Huallachain M, Dudley JT, Hillenmeyer S, Haraksingh R, Sharon D, Euskirchen G, Lacroute P, Bettinger K, Boyle AP, Kasowski M, Grubert F, Seki S, Garcia M, Whirl-Carrillo M, Gallardo M, Blasco MA, Greenberg PL, Snyder P, Klein TE, Altman RB, Butte AJ, Ashley EA, Gerstein M, Nadeau KC, Tang H, Snyder M (2012) Personal omics profiling reveals dynamic molecular and medical phenotypes. Cell 148(6):1293–1307

    Article  Google Scholar 

  48. Shaer, O., et al., Communicating personal genomic information to non-experts: a new frontier for human-computer interaction. Foundations and Trends® in Human-Computer Interaction, 2017. 11: p. 1–62

  49. Benioff, M.R., et al., Computational science: ensuring America’s competitiveness. President’s Information Technology Advisory Committee (PITAC), 2005

  50. Grand challenges to computational science. 1988

  51. Mardis ER (2008) The impact of next-generation sequencing technology on genetics. Elsevier Trends Journals, pp 133–141

  52. Ouh-young M et al (1988) Using a manipulator for force display in molecular docking. In: Proc. of Robotics and Automation, vol 3, pp 1824–1829

    Google Scholar 

  53. Gillet A et al (2005) Tangible augmented interfaces for structural molecular biology. IEEE Computer Graphics and Applications 25(2):13–17

    Article  Google Scholar 

  54. Schkolne S, Ishii H, Schroder P (2004) Immersive design of DNA molecules with a tangible interface, in Proceedings of the conference on Visualization ‘04. IEEE Computer Society, Washington, pp 227–234

    Google Scholar 

  55. Arnstein L et al (2002) Labscape: a smart environment for the cell biology laboratory. IEEE Pervasive Computing 1(3):13–21

    Article  Google Scholar 

  56. Yeh R et al (2006) ButterflyNet: a mobile capture and access system for field biology research. In: Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, New York, pp 571–580

    Chapter  Google Scholar 

  57. Mackay WE et al (2002) The missing link: augmenting biology laboratory notebooks. In: Proceedings of the 15th annual ACM symposium on User interface software and technology. ACM, New York, pp 41–50

    Chapter  Google Scholar 

  58. Tabard A et al (2011) The eLabBench: an interactive tabletop system for the biology laboratory. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. ACM, New York, pp 202–211

    Chapter  Google Scholar 

  59. Wigdor D et al (2009) WeSpace: the design development and deployment of a walk-up and share multi-surface visual collaboration system. In: Proceedings of the 27th international conference on Human factors in computing systems. ACM, New York, pp 1237–1246

    Google Scholar 

  60. Morris MR et al (2006) TeamTag: exploring centralized versus replicated controls for co-located tabletop groupware. In: Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, New York, pp 1273–1282

    Chapter  Google Scholar 

  61. Isenberg P et al (2010) An exploratory study of co-located collaborative visual analytics around a tabletop display. In: Symposium on Visual Analytics Science and Technology (VAST), pp 179–186

    Google Scholar 

  62. Kuznetsov S et al (2012) At the seams: DIYbio and opportunities for HCI. In: Proceedings of the Designing Interactive Systems Conference. ACM, New York, pp 258–267

    Chapter  Google Scholar 

  63. Scott SD, Grant KD, Mandryk RL (2003) System guidelines for co-located, collaborative work on a tabletop display. In: Proc. of ECSCW’03. Kluwer Academic Publishers, Norwell, pp 159–178

    Google Scholar 

  64. Morris MR et al (2004) Beyond “social protocols”: multi-user coordination policies for co-located groupware. In: Proc. of CSCW’04, pp 262–265

    Google Scholar 

  65. Shaer O et al (2012) The design, development, and deployment of a tabletop interface for collaborative exploration of genomic data. Int J Hum Comput Stud 70(10):746–764

    Article  Google Scholar 

  66. Kirsh, D. Methodologies for evaluating collaboration behavior in co-located environments. in CSCW 2004 Workshop: Methodologies for Evaluating Collaboration in Co-Located Environments. 2004

  67. Inkpen, K. “Just because:” the challenges of evaluating face-to-face collaboration. in CSCW 2004 Workshop: Methodologies for Evaluating Collaboration in Co-Located Environments 2004

  68. Westendorf L et al (2017) Understanding collaborative decision making around a large-scale interactive tabletop. Proceedings of the ACM on Human-Computer Interaction 1(CSCW):1–21

    Article  Google Scholar 

  69. Meulen HVD et al (2016) Towards understanding collaboration around interactive surfaces: exploring joint visual attention. In: In Proc. of UIST’16. ACM, Tokyo, pp 219–220

    Google Scholar 

  70. Fjeld, M., et al., Tangible user interface for chemistry education: comparative evaluation and re-design. Proc. of CHI’07. 2007. 805–808

  71. Horn, M.S., M. Tobiasz, and C. Shen, Visualizing biodiversity with voronoi treemaps, in In Proc. of ISVD ‘09. 2009. p. 265–270

  72. Schneider, B., et al., Phylo-Genie: engaging students in collaborative ‘tree-thinking’ through tabletop techniques, in Proc. of CHI’12. 2012. p. 3071--3080

  73. Loparev, A., et al., BacPack: exploring the role of tangibles in a museum exhibit for bio-design, in Proc. of TEI’17. 2017, ACM: Yokohama, Japan p 111-120

  74. Tetteroo, D., I. Soute, and P. Markopoulos, Five key challenges in end-user development for tangible and embodied interaction, in Proc. of ICMI’13. 2013, ACM: Sydney, Australia p 247-254

  75. Turchi, T. and A. Malizia, Pervasive displays in the wild: employing end user programming in adaption and re-purposing, in End-User Development: 5th International Symposium, IS-EUD 2015, Madrid, Spain, May 26–29, 2015. Proceedings, P. Díaz, et al., Editors. 2015, Springer International Publishing: Cham. p. 223–229

  76. Turchi T, Malizia A, Dix A (2017) TAPAS: a tangible end-user development tool supporting the repurposing of pervasive displays. J Vis Lang Comput 39:66–77

    Article  Google Scholar 

  77. Chi, E.H.-H., et al., A spreadsheet approach to information visualization, in Proceedings of Information Visualization’97. 1997. p. 17–24

  78. Ishii: Tangible bits: towards seamless interface - Google Scholar. 2009, \urlhttp://scholar.google.com/scholar?cites=16442430376593508398%5C&hl=en

  79. Konkel MK et al (2016) Discovery of a new repeat family in the Callithrix jacchus genome. Genome Res 26(5):649–659

    Article  Google Scholar 

  80. Carbone L, Harris RA, Mootnick AR, Milosavljevic A, Martin DI, Rocchi M, Capozzi O, Archidiacono N, Konkel MK, Walker JA, Batzer MA, de Jong PJ (2012) Centromere remodeling in Hoolock leuconedys (Hylobatidae) by a new transposable element unique to the gibbons. Genome Biol Evol 4(7):648–658

    Article  Google Scholar 

  81. Kirsh D, Maglio P (1994) On distinguishing epistemic from pragmatic action. Cogn Sci 18(4):513–549

    Article  Google Scholar 

  82. M. E. Scott, S. Perry, L. Staskawicz, et al. github : sifteo / thundercracker. 2011; Available from: https://github.com/sifteo/thundercracker

  83. Merrill, D., J. Kalanithi, and P. Maes, Siftables: towards sensor network user interfaces, in Proc. of TEI’07. 2007, ACM: New York, NY, USA. p. 75--78

  84. Merrill, D.J., Interaction with embodied media. 2009, MIT

  85. Merrill, D., E. Sun, and J. Kalanithi, Sifteo cubes, in CHI’12 Extended Abstracts. 2012. p. 1015--1018

  86. Pillias, C., R.E. Robert-Bouchard, and G. Levieux, Designing tangible video games: lessons learned from the sifteo cubes, in Proc. of CHI’14. 2014. p. 3163--3166

  87. Ullmer, B., et al., Cartouche: conventions for tangibles bridging diverse interactive systems, in Proc. of TEI’10. 2010. p. 93--100

  88. Ansoff I (1957) Strategies for diversification 35(5):113–124

    Google Scholar 

  89. Giardine B et al (2005) Galaxy: a platform for interactive large-scale genome analysis, pp 1451–1455

    Google Scholar 

  90. Goecks J, Nekrutenko A, Taylor J (2010) Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biol 11:1–13

    Article  Google Scholar 

  91. Grüning BA, Rasche E, Rebolledo-Jaramillo B, Eberhard C, Houwaart T, Chilton J, Coraor N, Backofen R, Taylor J, Nekrutenko A (2017) Jupyter and galaxy: easing entry barriers into complex data analyses for biomedical researchers. PLoS Comput Biol 13(5):e1005425

    Article  Google Scholar 

  92. Geurts L et al (2014) Playfully learning visual perspective taking skills with sifteo cubes. In: Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play. ACM, New York, pp 107–113

    Chapter  Google Scholar 

  93. Arif, A.S., et al., Sparse tangibles: collaborative exploration of gene networks using active tangibles and interactive tabletops, in Proc. of TEI’16. 2016, ACM: New York, NY, USA p 287--295

  94. Roozbeh Manshaei, S.D., Uzair Mayat, Dhrumil Patal, Matthew Kyan, and Ali Mazalek. Tangible BioNets: multi-surface and tangible interactions for exploring structural features of biological networks. in Proc. of EICS’19. 2019

  95. East B et al (2016) Actibles: open source active tangibles. In: Proceedings of theACM International Conference on Interactive Surfaces and Spaces. ACM, Niagara Falls, Ontario, pp 469–472

    Google Scholar 

  96. Roozbeh Manshaei, U.M., Aneesh Tarun, Sean DeLong, David Chiang, Justin Digregorio, Shahin Khayyer, Apurva Gupta, Matthew Kyan, and Ali Mazalek. Tangible tensors: an interactive system for grasping trends in biological systems modeling. in Proc. of C&C 2019. 2019

  97. DeLong, S., Ahmed Sabbir Arif, and Ali Mazalek. Design and evaluation of graphical feedback on tangible interactions in a low-resolution edge display. in Proc. of Pervasive Displays 2019. 2019

  98. Ben-Joseph E et al (2001) Urban simulation and the luminous planning table: bridging the gap between the digital and the tangible. J Plan Educ Res 21(2):196–203

    Google Scholar 

  99. Underkoffler, J. and H. Ishii, Urp: a luminous-tangible workbench for urban planning and design, in Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit. 1999. p. 386--393

  100. Brudy, F., et al., Cross-device taxonomy: survey, opportunities and challenges of interactions spanning across multiple devices, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019, ACM: Glasgow, Scotland Uk p 1–28

  101. Le Goc, M., et al. Zooids: building blocks for swarm user interfaces in Proceedings of the 29th Annual Symposium on User Interface Software and Technology

  102. Harley, D., et al. Tangible VR: diegetic tangible objects for virtual reality narratives in Proceedings of the 2017 Conference on Designing Interactive Systems

  103. TUIO: A protocol for table-top tangible user interfaces. 2005

    Google Scholar 

  104. Software implementing TUIO. http://www.tuio.org/?software

  105. Kaltenbrunner, M., reacTIVision and TUIO: a tangible tabletop toolkit, in Proc. of ITS’09. 2009. p. 9--16

  106. Aish R, Noakes P (1984) Architecture without numbers-CAAD based on a 3 D modelling system. Comput Aided Des 16(6):321–328

    Article  Google Scholar 

  107. Aish R (1979) 3D input for CAAD systems. Comput Aided Des 11:66–70

    Article  Google Scholar 

Download references

Acknowledgments

We thank David Merrill, Liam Staskawicz, Consuelo Valdes, Casey Grote, André Wiggins, and Michael Lynn for supporting this work.

Funding

We are appreciative of NSF grants CNS-1828611, CNS-1126739, IIS-1149530, and IIS-1320350; the NSERC Discovery Grant and Canada Research Chairs programs, and the Canada Foundation for Innovation (CFI) and Ontario Ministry of Research and Innovation (MRI) Innovation Fund for partial support of this work.

Author information

Authors and Affiliations

  1. Dept. of Genetics & Biochemistry, Clemson University, Clemson, SC, USA

    Miriam K. Konkel

  2. Human-Centered Computing Division, Clemson University, Clemson, SC, USA

    Brygg Ullmer

  3. Dept. of Computer Science, Wellesley College, Wellesley, MA, USA

    Orit Shaer

  4. Synaesthetic Media Lab, Ryerson University, Toronto, ON, Canada

    Ali Mazalek

  5. Dept. of Computer Science, Drury University, Springfield, MO, USA

    Chris Branton

Authors
  1. Miriam K. Konkel
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Brygg Ullmer
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Orit Shaer
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ali Mazalek
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Chris Branton
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Miriam K. Konkel or Brygg Ullmer.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Konkel, M.K., Ullmer, B., Shaer, O. et al. Toward tangibles and display-rich interfaces for co-located and distributed genomics collaborations. Pers Ubiquit Comput 26, 767–779 (2022). https://doi.org/10.1007/s00779-020-01376-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01376-5

Keywords