Xiangmin Zhang
ABSTRACT
This paper reports a usability evaluation of three operational digital libraries (DLs): the ACM DL, the IEEE Computer Society DL, and the IEEE Xplore DL. An experiment was conducted in a usability lab and 35 participants completed the assigned tasks. The results demonstrate that all three DLs have more or less usability problems by various measures. Searching in Xplore by inexperienced users was problematic, and browsing in IEEE CS was extremely difficult for all users. Interaction design features that caused these usability problems were identified and discussed. The study implies there is still large room for operational DLs to improve in order to provide more satisfactory services.
- Belkin, N.J., Marchetti, P.G., Cool, C. 1993. Braque: design of an interface to support user interaction in information retrieval. Information Processing&Management. 29(3), 325--344. Google Scholar
Digital Library
- Chang, S. L. 1995. Toward a multidimensional framework for understanding browsing. Unpublished doctoral dissertation, Rutgers, The State University of New Jersey, New Brunswick, NJ.Google Scholar
- Chen, S. Y., Magoulas, G. D., Dimakopoulos, D. 2005. A flexible interface design for Web directories to accommodate different cognitive styles. Journal of the American Society for Information Science and Technology. 56(1), 70--83. Google ScholarDigital Library
- Chimera, R., Shneiderman, B. 1994. An exploratory evaluation of three interfaces for browsing large hierarchical tables of contents. ACM Transactions on Information Systems (TOIS). 12(4), 383--406. Google ScholarDigital Library
- Chin, J., Diehl, V. A.,&Norman, K. L. 1988. Development of an instrument measuring user satisfaction of the human-computer interface. Proceedings of the SIGCHI conference on Human factors in computing systems (pp.213--218). Washington, D.C., May 15--19, 1988. Google ScholarDigital Library
- Chrzastowski, T. E.,&Scheeline, A. 2006. ASDL: The Analytical Sciences Digital Library taking the next steps. Science&Technology Libraries, 26(3/4), 79--94.Google Scholar
- Dakka, W., Ipeirotis, P. G., Wood, K. R. 2005. Automatic construction of multifaceted browsing interfaces, in: Chowdhury, A., Fuhr, N., Ronthaler, M., Schek, H.-J., Teiken, W. (Eds.), Proceedings of the 14th ACM international conference on Information and knowledge management CIKM '05. ACM Press, New York, pp. 768--775. Google ScholarDigital Library
- Davis, F.D. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 3, 319--340.Google ScholarDigital Library
- Doll, W.J.; Hendrickson, A.; and Deng, X. 1998. Using Davis's perceived usefulness and ease of use instruments for decision making: A confirmatory and multigroup invariance analysis. Decision Sciences, 29, 4, 839--869.Google ScholarCross Ref
- Egan, D.E., Remde, J.R., Landauer, T.K. 1989. Behavioral evaluation and analysis of a hypertext browser, in: Bice, K., Lewis, C. (Eds.), Proceedings of CHI'89 Conference on Human Factors in Computing Systems. ACM Press, New York, pp.205--210. Google ScholarDigital Library
- Hartson, H. R., Shivakumar, P.,&Perez-Quinones, M. A. 2004. Usability inspection of digital libraries: a case study. International Journal of Digital Library, 4, 108--123.Google ScholarDigital Library
- Hearst, M., Elliot, A., English, J., Sinha, R., Swearingen, K.,&Yee, K.P. 2002. Finding the flow in Website search. Communications of the ACM, 45(9):42--49. Google ScholarDigital Library
- Hertzum, M., Frøkjær, E. 1996. Browsing and querying in online documentation: a study of user interfaces and the interaction process. ACM Transactions on Computer-Human Interaction (TOCHI). 3(2), 136--161. Google ScholarDigital Library
- Hill, L.L., Carver, L., Larsgaard, M., Dolin, R., Smith, T. R., Frew, J.,&Rae, M.-A. 2000. Alexandria Digital Library: User Evaluation Studies and System Design. Journal of the American Society for Information Science. 51(3):246--259. Google ScholarDigital Library
- Hinton, P. R., Brownlow, C., McMurray, I.,&Cozens, B. 2004. SPSS Explained. New York, NY: Routledge, Taylor&Francis Group. Google ScholarDigital Library
- Hong, W., Thong, J. Y.L., Wong, W.-M.,&Tam, K.-Y. 2001--2002. Determinants of user acceptance of digital libraries: An empirical examination of individual differences and system characteristics. Journal of Management Information Systems, 18(3), 97--124. Google ScholarDigital Library
- Kani-Zabihi, E., Ghinea, G., Chen, S. Y. 2006. Digital libraries: what do users want? Online Information Review, 30(4), 395--412.Google ScholarCross Ref
- Kengeri, R., Seals, C. D., Harley, H. D., Reddy, H. R.,&Fox, E. A. 1999. Usability study of digital libraries: ACM, IEEE-CS, NCSTRL, NDLTD. International Journal of Digital Library, 2, 157--169.Google ScholarDigital Library
- Lesk, M. 2005. Understanding Digital Libraries, 2nd ed. San Francisco, CA: Morgan Kaufmann Publishers. Google ScholarDigital Library
- Marchionini, G. 1995. Information Seeking in Electronic Environments. Cambridge University Press, United Kingdom. Google ScholarDigital Library
- Marchionini, G., Norman, K.,&Morrell, K. 1992. Final project report: Computer interface design for intermediate results for Grateful Med. Report to the National Library of Medicine. Bethesda, MD: NLM.Google Scholar
- Notess, M., Kouper, I.,&Swan, M. B. 2005. Designing effective tasks for digital library user tests: lessons learned. OCLC Systems&Services, 21(4), 300--310.Google Scholar
- Paynter, G. W., Witten, I. H., Cunningham, S. J., Buchanan, G. 2000. Scalable browsing for large collections: a case study, in: Nürnberg, P. J., Hicks, D. L., Furuta, R. (Eds.), Proceedings of the fifth ACM Conference on Digital libraries. ACM Press, New York, pp.215--223. Google ScholarDigital Library
- Preece, J., Rogers, Y.,&Sharp, H. 2007. Interaction design: beyond human-computer interaction. New York: John&Wiley Sons, Inc. Google ScholarDigital Library
- Resnick, M.L.&Vaughan, M.W. 2006. Best practices and future visions for search user interfaces. Journal of the American Society for Information Science&Technology, 57(6), 781--787. Google ScholarDigital Library
- Saracevic, T. 2000. Digital library evaluation: towards an evolution of concepts. Library Trends, 49(3): 350--369.Google Scholar
- Shiri, A. and Molberg, K. 2005. Interfaces to knowledge organization systems in Canadian digital library collections. Online information Review, 29(6), 604--620.Google Scholar
- Shneiderman, B.,&Plaisant C. 2005. Designing the user interface. Boston, MA: Pearson Education, Inc. Google ScholarDigital Library
- Shneiderman, B., Byrd, D.,&Croft, B. 1997. Clarifying search: A user-interface framework for text searches. D-Lib Magazine, January 1997. Retrieved January 20, 2008, from http://www.dlib.org/dlib/january97/retrieval/01shneiderman.html.Google ScholarDigital Library
- Tague-Sutcliffe, J. 1992. The pragmatics of information retrieval experimentation, revisited. Information Processing&Management, 28(4), 467--490. Google ScholarDigital Library
- Theng, Y. L. 1999. "Lostness" and digital libraries. Proceedings of the fourth ACM conference on Digital libraries, 250--251. Google ScholarDigital Library
- Thong, J. Y. L., Hong, W.,&Tam, K.-Y. 2002. Understanding user acceptance of digital libraries: what are the roles of interface characteristics, organizational context, and individual differences. International Journal of Human-Computer Studies, 57, 215--242. Google ScholarDigital Library
- Tsakonas, G.,&Papatheodorou, C. 2006. Analysing and evaluating usefulness and usability in electronic information services. Journal of Information Science, 32 (5), 400--419. Google ScholarDigital Library
- Tsakonas, G.,&Papatheodorou, C. 2008. Exploring usefulness and usability in the evaluation of open access digital libraries. Information Processing and Management, 44, 1234--1250. Google ScholarDigital Library
- Yee, K.P., Swearingen, K., Li, K., Hearst, M. 2003. Searching and organizing: Faceted metadata for image search and browsing, in: Cockton, G., Korhonen, P. (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, New York, pp. 401--408. Google ScholarDigital Library
- Xie, H. 2006. Evaluation of digital libraries: Criteria and problems from users' perspectives. Library&Information Science Research, 28, 433--452.Google Scholar
- Zhang, J., Marchionini, G. 2005. User interfaces for integrating web-based information: Evaluation and evolution of a browse and search interface: relation browser, in: Delcambre, L. M. L., Giuliano, G. (Eds.), Proceedings of the 2005 National Conference on Digital Government Research. Digital Government Research Center, California, pp.179--188. Google ScholarDigital Library
- Zhang, X., Li, Y., Liu, J.,&Zhang, Y. 2008. Effects of interaction design in digital libraries on user interactions. Journal of Documentation, 64(3), 438--463.Google ScholarCross Ref
Index Terms
- How usable are operational digital libraries: a usability evaluation of system interactions
Recommendations
Digital competencies for developing and managing digital libraries
PurposeThe purpose of this study was to explore the essential digital competencies for developing and managing digital libraries. The study identified useful training programs for university librarians to acquire digital competencies. It examined their ...
Digital Preservation, Archival Science and Methodological Foundations for Digital Libraries
Digital libraries, whether commercial, public, or personal, lie at the heart of the information society. Yet, research into their long-term viability and the meaningful accessibility of their contents remains in its infancy. In general, as we have ...
Worksets Expand the Scholarly Utility of Digital Libraries
JCDL '18: Proceedings of the 18th ACM/IEEE on Joint Conference on Digital LibrariesScholars using digital libraries and archives routinely create worksets-aggregations of digital objects-as a way to segregate resources of interest for in-depth scrutiny. To illustrate how worksets can enhance the scholarly utility of digital library ...
Comments