Abstract
Evaluation of collaborative systems is necessary in several situations. However, evaluation is frequently done in an ad-hoc manner or not at all. This paper presents a survey of evaluation methods for groupware systems. The analysis, comparison and classification of these methods will help developers choose the appropriate methods for their situation. Furthermore, the survey allows identification of strengths and weaknesses of existing methods, opening opportunities for research in this area. The proposed comparison criteria represent a framework to evaluate and classify new evaluation methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Antunes, P., Costa, C.: Perceived value: A low-cost approach to evaluate meetingware. In: Favela, J., Decouchant, D. (eds.) CRIWG 2003. LNCS, vol. 2806, pp. 109–125. Springer, Heidelberg (2003)
Antunes, P., Ferreira, A., Pino, J.: Analyzing shared workspaces design with human-performance models. In: Dimitriadis, Y.A., Zigurs, I., Gómez-Sánchez, E. (eds.) CRIWG 2006. LNCS, vol. 4154, pp. 62–77. Springer, Heidelberg (2006)
Baeza-Yates, R., Pino, J.: Towards formal evaluation of collaborative work and its application to information retrieval. Information Research 11(4), 271 (2006)
Baker, K., Greenberg, S., Gutwin, C.: Empirical development of a heuristic evaluation methodology for shared workspace groupware. In: CSCW 2002, pp. 96–105 (2002)
Grudin, J.: Why CSCW applications fail: problems in the design and evaluation of organization of organizational interfaces. In: CSCW 1988, pp. 85–93. ACM Press, New York (1988)
Gutwin, C., Greenberg, S.: The mechanics of collaboration: Developing low cost usability evaluation methods for shared workspaces. In: WETICE 2000, pp. 98–103. IEEE Comp. Soc., Los Alamitos (2000)
Haynes, S., Purao, S., Skattebo, A.: Situating evaluation in scenarios of use. In: CSCW 2004, pp. 92–101. ACM Press, New York (2004)
Huang, J.: A conceptual framework for understanding collab. systems evaluation. In: WETICE 2005. 14th I. Workshop on Enabling Tech., pp. 215–220. IEEE Comp. Soc., Los Alamitos (2005)
Hughes, J., King, V., Rodden, T., Andersen, H.: Moving out from the control room: Ethnography in system design. In: CSCW 1994, pp. 429–439. ACM Press, New York (1994)
Huis in’t Veld, M., Andriessen, J., Verburg, R.: E-magine: The development of an evaluation method to assess groupware applications. In: WETICE 2003, vol. 153, IEEE, Los Alamitos (2003)
Inkpen, K., Mandryk, R., DiMicco, J., Scott, S.: Methodology for evaluating collaboration in co-located environments. Interactions 11(6) (2004)
Pinelle, D., Gutwin, C.: A review of groupware evaluations. In: WETICE 2000. 9th International Workshop on Enabling Technologies, pp. 86–91. IEEE Computer Society, Los Alamitos (2000)
Pinelle, D., Gutwin, C.: Groupware walkthrough: adding context to groupware usability evaluation. In: CHI 2002, pp. 455–462. ACM Press, New York (2002)
Pinelle, D., Gutwin, C., Greenberg, S.: Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration. ACM Transactions on Computer-Human Interaction 10(4), 281–311 (2003)
Plowman, L., Rogers, Y., Ramage, M.: What are workplace studies for? In: ECSCW 1995. Fourth European Conference on Computer-Supported Cooperative Work, pp. 309–324 (1995)
Ross, S., Ramage, M., Rogers, Y.: PETRA: Participatory evaluation through redesign and analysis. Interacting with Computers 7(4), 335–360 (1995)
Stiemerling, O., Cremers, A.: The use of cooperation scenarios in the design and evaluation of a CSCW system. IEEE Transact. on Software Engineering 25, 140 (1999)
Twidale, M., Randall, D., Bentley, R.: Situated evaluation for cooperative systems. In: CSCW 1994, pp. 441–452. ACM Press, New York (1994)
Vizcaíno, A., Martinez, M., Aranda, G., Piattini, M.: Evaluating collaborative applications from a knowledge management approach. In: WETICE 2005, pp. 221–225. IEEE Comp. Soc., Los Alamitos (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Herskovic, V., Pino, J.A., Ochoa, S.F., Antunes, P. (2007). Evaluation Methods for Groupware Systems. In: Haake, J.M., Ochoa, S.F., Cechich, A. (eds) Groupware: Design, Implementation, and Use. CRIWG 2007. Lecture Notes in Computer Science, vol 4715. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74812-0_26
Download citation
DOI: https://doi.org/10.1007/978-3-540-74812-0_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74811-3
Online ISBN: 978-3-540-74812-0
eBook Packages: Computer ScienceComputer Science (R0)