Abstract
Researchers have begun to explore tools that allow multiple users to collaborate across multiple devices in collocated environments. These tools often allow users to simultaneously place and interact with information on shared displays. Unfortunately, there is a lack of experimental tasks to evaluate the effectiveness of these tools for information coordination in such scenarios. In this article, we introduce job-shop scheduling as a task that could be used to evaluate systems and interactions within computer-supported collaboration environments. We describe properties that make the task useful, as well as evaluation measures that may be used. We also present two experiments as case studies to illustrate the breadth of scenarios in which this task may be applied. The first experiment shows the differences when users interact with different communicative gesturing schemes, while the second demonstrates the benefits of shared visual information on large displays. We close by discussing the general applicability of the tasks.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Mennecke BE, Wheeler BC (1993) An essay and resource guide for dyadic and group task selection and usage. Institute for research on the management of information systems working paper #9309, Indiana University, Bloomington
Heath CC, Knoblauch J, Luff P (2000) Technology and social interaction: the emergence of workplace studies. Br J Sociol 51(2):299–320
Dourish P (2006) Implications for design. Proceedings of CHI 2006 conference on human factors in computing systems, pp 541–550
Inkpen K, Mandryk R, DiMicco JM, Scott S (2004) Methodologies for evaluating collaboration in co-located environments. Workshop presented at the ACM conference on computer-supported cooperative work 2004
Pinelle D, Gutwin C, Greenberg S (2003) Task analysis for groupware usability evaluation: modeling shared-workspace tasks with the mechanics of collaboration. ACM Trans Hum Comput Interact 10(4):281–311
McGrath JE (1984) Groups: interaction and performance. Prentice Hall, New Jersey
Olson JS, Olson GM, Storrøsten M, Carter M (1993) Groupwork close up: a comparison of the group design process with and without a simple group editor. ACM Trans Hum Comput Interact 11:321–348
Scott SD, Carpendale MST, Inkpen, KM (2004) Territoriality in collaborative tabletop workspaces. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 294–303
Gutwin C (2002) Traces: visualizing the immediate past to support group interaction. Proceedings of graphics interface 2002, pp 43–50
Clark HH, Krych MA (2004) Speaking while monitoring addressees for understanding. J Mem Lang 50:62–81
Kraut RE, Gergle D, Fussell SR (2002) The use of visual information in shared visual spaces: informing the development of virtual co-presence. Proceedings of the ACM conference on computer-supported cooperative work 2002, pp 31–40
Rocco E (1998) Trust breaks down in electronic contexts but can be repaired by some initial face-to-face contact. Proceedings of CHI 1998 conference on human factors in computing systems, pp 496–502
Setlock LD, Fussell SR, Neuwirth C (2004). Taking it out of context: collaborating within and across cultures in face-to-face settings and via instant messaging. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 604–613
Karger D, Stein C, Wein J (1997) Scheduling algorithms. In: Atallah MJ (ed) Handbook of algorithms and theory of computation. CRC Press, Boca Raton
Gutwin C, Penner R (2002) Improving interpretation of remote gestures with telepointer traces. Proceedings of the ACM conference on computer-supported cooperative work 2002, pp 49–57
Khan A, Matejka J, Fitzmaurice G, Kurtenbach G (2005) Spotlight: directing users’ attention on large displays. Proceedings of CHI 2005 conference on human factors in computing systems, pp 791–798
Clark HH, Wilkes-Gibbs D (1986) Referring as a collaborative process. Cognition 22:1–39
Fisher H, Thompson G (1963) Probabilistic learning combinations of local job-shop scheduling rules. In: Muth J, Thompson G (eds) Industrial scheduling. Prentice Hall, Englewood Cliffs, pp 225–251
Beasley JE (1990) OR-Library: distributing test problems by electronic mail. J Oper Res Soc 41(11):1069–1072. http://www.brunel.ac.uk/depts/ma/research/jeb/info.html
Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Human Factors Special Issue Situation Awareness 37:32–64
Kraut RE, Fussell SR, Siegel J (2003) Visual information as a conversational resource in collaborative physical tasks. Hum Comput Interact 18:13–49
Gergle D, Kraut RE, Fussell, SR (2004) Action as language in a shared visual space. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 487–496
Tan DS, Meyers B, Czerwinski M (2004) Wincuts: manipulating arbitrary window regions for more effective use of screen space. Proceedings of CHI 2004 conference on human factors in computing systems, pp 1525–1528
Kenny DA, Mannetti L, Pierro A, Livi S, Kashy DA (2002) The statistical analysis of data from small groups. J Pers Soc Psychol 83:126–137
Hoenig JM, Heisey DM (2001) The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat 55:19–24
Ryall K, Forlines C, Shen C, Ringel-Morris M (2004) Tabletop design: exploring the effects of group size and table size on interactions with tabletops shared-display groupware. Proceedings of the ACM conference on computer-supported cooperative work 2004, pp 284–293
Scott SD (2005) Territoriality in collaborative tabletop workspaces. Doctoral Dissertation, Department of Computer Science, University of Calgary
Tan DS (2004) Exploiting the cognitive and social benefits of physically large displays. Doctoral Dissertation, School of Computer Science, Carnegie Mellon University
Johanson B, Fox A, Winograd T (2002) The interactive workspaces project: Experiences with ubiquitous computing rooms. IEEE Pervasive Comput Mag 1(2):67–74
Acknowledgments
We thank Patrick Baudisch, Andrew Faulring, James Fogarty, Jana Foster, Susan Fussell, Teresa Janz, Brian Meyers, Jeff Nichols, Judy Olson, Randy Pausch, Daniel Robbins, George Robertson, Greg Smith, and Ryder Ziola for their discussions of the tasks and assistance with the experiments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tan, D.S., Gergle, D., Mandryk, R. et al. Using job-shop scheduling tasks for evaluating collocated collaboration. Pers Ubiquit Comput 12, 255–267 (2008). https://doi.org/10.1007/s00779-007-0154-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00779-007-0154-3