Keywords

1 Introduction

Searching for information has become an increasingly important human activity in the modern world. As the explosion of information has become commonplace for knowledge workers, search has been widely used to ease the stress of information overload. Contemporary search happens in many different types of data collections: general Web search, image collections, ecommerce sites, government data, health and medical databases, and digital libraries and archives. While many see the search engine Google as the synonym of search, Amazon users would understand that it takes more than an omnibox to identify one suitable product from the enormous online store. Faceting, sorting, checking rankings, reading reviews, comparing product details are among the common techniques an online shopper engages in an ecommerce environment. General purpose search user interfaces (SUIs), therefore, do not satisfy all search needs [1]. In other words, specialized SUIs are necessary when the information seeking needs and contexts go beyond a general Web search.

Academic search SUIs include the search portals of academic and research libraries, digital data repositories, academic data aggregators, and publishers. The design of SUIs is important to information seeking activities and could influence information seeking behavior. For example, a simplistic design such as Google Search requires users to issue longer queries to compensate for the lack of a functional faceting mechanism; ineffective SUIs could waste the cognitive resources of the information seeker [2]. For academic information seekers, the use of Google Search is convenient but at the cost of search result quality [3].

Due to the intensive use of information in scientific activities, library and information science (LIS) researchers have long been interested in the information seeking and retrieval behavior of scholars. However, although scientific researchers spend much of their time in information seeking [4], how academic SUIs could better support scientific information seeking remains under-studied. As Bates [5] has pointed out, “[t]o optimize information search, … various design layers need to be recognized, understood, and designed for in an interface that nonetheless feels simple and natural to the end user.” Other researchers have also framed search as “data exploration for knowledge building” [6] to call for the design of future search interfaces.

As a core concept of human-computer interaction (HCI), usability has been used as an attribute for quality measurement of interactive systems. To achieve scientific understanding and better user experience of academic SUIs, usability evaluation has received attention among LIS researchers and professionals. According to a survey, 85% of academic libraries of the Association of Research Libraries (ARL) have conducted usability testing on their websites or OPAC [7]. However, usability is a fluid concept and the variation in perspective has led to different usability conceptualizations and evaluation approaches, which makes usability difficult to operationalize in practice. As a result, many usability measurement studies fall short in validity and reliability [8, 9].

The lack of consensus in usability conceptualization has also caused usability evaluation studies in digital libraries and discovery tools to be mostly ad hoc. In order to create academic SUIs with high usability, a clearly articulated conceptual framework of usability is critical for the design and evaluation of academic SUIs. Such a framework should bring the scientific information seeking knowledge together with contextualized usability conceptualization to further conceptual development and empirical examination of academic SUIs.

Conceptual models are critical for theory-informed design. Researchers have adopted models to guide the design of SUI. Jackson et al. [10] designed an exploratory search interface to support scholarly activities in searching an Internet archive (webarchive.ca) following Shneiderman’s [11] visual information seeking mantra as a design principle and the chess analogy of Hearst et al. [12] as a task model. Many recent prototyped experimental SUI features have been developed under the exploratory search paradigm [13,14,15]. The development of contextualized usability models for academic SUI would therefore provide guidance for the design and evaluation of SUIs for academic search.

A major difference between academic SUIs and general interactive systems is that academic SUIs take content, rather than the system’s functional features, as the purpose of the interaction. Content relevance hence becomes a key criterion of successful retrieval. In addition, the design of academic SUIs is further complicated by the huge volume and idiosyncratic essence of content used by users from various disciplines. Researchers [16] have therefore called for the study of the work needs, patterns, and workflow of researchers in order to integrate internal and external content with search services. As Shneiderman and Plaisant [17] point out, “[t]he conversion of information needs … to interface actions is a large cognitive step.” The objective of this study, therefore, is to explore the issue of contextualized academic SUI usability modeling through the perspectives of academic information seeking tasks and contextualized usability factors.

2 Literature Analysis

While LIS researchers have intensively studied the information seeking behavior of scientists, research issues related to HCI have received relatively little attention from information retrieval researchers [18]. Pettigrew and McKechnie [19], after a content analysis of six major information science journals, found that HCI represents only 2% of the published articles. Many usability evaluation studies on academic and research library websites have also placed less attention to usability concepts and specific SUI features. As Bates [5] pointed out, interface design specific to searching is an under-studied and promising research area in information seeking and HCI.

2.1 Usability

Usability as a quality attribute of interactive systems has critical implications for the implementation and evaluation of information systems. However, the lack of a clear definition of usability as a conceptual construct has led to problems in the principled design and measurement of interactive systems. This predicament is shown by the popularity of the term user experience (UX) as both a displacement and synonym of usability to denote the broader aspects of human experience with products and services. The conceptual evolution and overlapping are evidenced by the rebranding of the Usability Professional Association as User Experience Professional Experience in 2012 and the inclusion of the subtitle of “improving the user experience” in the U.S. government usability website (www.usability.gov) [20]. In many cases, the terms usability, user experience, and human-centered design have been used commonly without carrying specific meanings [21].

One reason for the lack of conceptual clarity in usability is that the evolvement of context has changed the nature of interaction that usability as an academic term once meant. For example, usability was discussed in a time when “[m]ost computer software in use today is unnecessarily difficult to understand, hard to learn, and complicated to use” [22]. Users, instead of systems, were once the target of improvement in that “[u]sability depends heavily on users’ abilities to map their goals onto a system’s capabilities” [23]. The commonly referenced ISO 9241-11:1998 was conceived when “evaluation of usability by user based measurement of effectiveness, efficiency, and satisfaction, as this was a convincing way of demonstrating the existence of usability problems to system developers” [24]. When these contexts no longer hold true, reconceptualization or creation of new concepts become necessary. That is the reason why the new ISO usability guidelines are incorporating UX perspectives under the satisfaction aspect in the coming new revision [24].

Usability Frameworks.

General usability frameworks such as ISO standard 9241-11 [25] and Nielsen’s heuristic evaluation [26] are commonly used in conducting empirical usability design and evaluation. It should be noted that these models are often defined with different factors. The ISO standard 9241-11 defines usability with three factors of (1) effectiveness, (2) efficiency, and (3) satisfaction; while Nielsen [27] defines usability with five quality components of (1) learnability, (2) efficiency, (3) memorability, (4) errors, and (5) satisfaction. Research reviews have thus pointed out the lack of consensus in the definition of usability [9, 28, 29].

Among the attempts to clarify the concept of usability, Alonso-Ríos et al. [30] proposed a usability taxonomy with six factors of (1) knowability, (2) operability, (3) efficiency, (4) robustness, (5) safety, and (6) subjective satisfaction with sub-attributes discussed under each factor. Similarly, Seffah et al. [31] reviewed various usability standards and models, and proposed a Quality in Use Integrated Measurement (QUIM) model with ten factors (efficiency, effectiveness, productivity, satisfaction, learnability, safety, trustfulness, accessibility, universality, and usefulness), 26 sub-factors, and measurement metrics.

The existence of multiple definitions with varied factors has evidenced usability as a multi-dimensional concept [32]. What troubles a unified definition of usability is that the included factors are related to each other [31, 33] and thus making analysis difficult. A solution for conceptual clarification is to specify the use context of the usability framework. In fact, this emphasis on context is addressed in the ISO 9241-11 standards by defining usability as the “[e]xtent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [25].

Contextualizing Usability.

Fidel [34] discussed the design of context-specific information systems and suggested that a context-specific system serves a particular community of users and its design may rely on the use of a context-general system. The efforts to address the issue of usability in context can be seen in the specialized models, metrics, and instruments developed for measuring usability in various fields. The Questionnaire for User Interface Satisfaction [35, 36] is a usability scale for interface measurement and has aspects of screen and terminology specifically as factors. To ease the difficulty in usability implementation and the dependency on evaluator expertise, Lin et al. [37] developed a comprehensive index of software interface usability based on human information processing theory. Eight factors (compatibility, consistency, flexibility, learnability, minimal action, minimal memory load, perceptual limitation, and user guidance) are indexed and the resulting Purdue Usability Testing Questionnaire (PUTQ) contains 100 questions.

Many contextualized usability models and measurement instruments are based on the ISO 2941-11 model’s factors of effectiveness, efficiency, and satisfaction. Some models are also informed by theories, while most are based on literature review and domain features. Evaluation tools are often based on the developed models containing metrics and/or questionnaires. Usability contextualization is emphasized by researchers [38, 39]. As Bevan and Macleod [22] state: “[t]he ideal way to specify and measure usability would be to specify the features and attributes required to make a product usable, and measure whether they are present in the implemented product” thus enabling quality to be designed into a product. They also suggest that usability can only be measured empirically “by assessing effectiveness, efficiency, and satisfaction with which representative users carry out representative tasks in representative environments” [22].

2.2 Academic Information Seeking

Among the academic SUIs, the library websites of academic and research universities are studied more than publishers, aggregators, and academic databases. Broadly speaking, the websites of academic and research libraries are academic SUIs since the central function of a library website is content discovery and delivery. Academic searchers engage in scientific problem-solving with browsing and search activities. The process of academic information seeking is therefore usually complicated, longitudinal, and exploratory in nature.

As Shneiderman and Plaisant [17] point out, the weaknesses of traditional search interfaces include “difficulty in repeating searches across multiple databases, weak methods for discovering where to narrow broad searches, poor integration with other tools.” Contemporary academic SUIs have made progress in the capacity of combining content sources through federated search and discovery tools, although tool integration has not been greatly improved and academic library websites are still often complicated and low in usability due to the issue of resource management [40, 41]. As Web search engines have gradually evolved from keyword matching and Boolean search to semantic search, the representation of search results has also progressed from “search for links” to “search for information;” yet the topic of academic search continues to be under-studied [42].

Nel and Fourie [4], for example, found that about one-third of veterinary researchers spend more than 50% of their research time on information seeking and that they rely on electronic journal articles, scientific databases, and internet search tools as their sources of information. These scientific information needs should cause researchers to primarily rely on academic SUIs. However, according to a large-scale survey on researcher information behavior, a new generation of researchers across disciplines have a strong preference for using Google/Google Scholar for information seeking [43]. While academic SUIs and Google Search should complement each other, academic SUI’s would satisfy the information needs of scientists more if they offered better usability.

Information Seeking Tasks.

LIS researchers have investigated the information behavior of scientists and developed influential descriptive models of the information seeking processes. Wilson [44] reviewed information seeking models and indicated that the models are at different levels of information behavior, information-seeking behavior, and information search behavior. Descriptive models such as Wilson’s [44] model of information behavior provide overarching descriptions of information behavior; Ingwersen’s [45] cognitive model of IR interaction is a high-level overview of information retrieval; whereas Saracevic’s [46] stratified model of IR interaction analyzes the information retrieval process from the interface perspective. Such models provide a basis for understanding academic searchers and their interaction with academic SUIs.

Due to the complexity in information needs and content, academic SUIs are less studied and most library website usability evaluation studies are ad hoc in nature. This complexity is evident in the question raised by Bates [47] of “how much and what type of activity the user should be able to direct the system to do at once.” Although Bates [48] has proposed the embedment of search tasks in SUI to support academic search, embedding tasks in SUI is difficult. As Shneiderman and Plaisant [17] explain, “[t]he conversion of information needs … to interface actions is a large cognitive step.”

Järvelin and Ingwersen [49] propose that information seeking and retrieval research should pay more attention to tasks and their contexts, which include information retrieval, information seeking, and work task contexts. Bates [48] emphasizes the importance of information seeking tasks in academic search by discussing the embedment of six scholarly browsing techniques that are commonly used by scientists in SUIs: footnote chasing, citation searching, journal run, area scanning, subject searching, and author searching. These browsing techniques are partially supported in existing SUIs, but not fully developed and integrated to support academic search.

Researchers have described the behavioral tasks of academic searchers from experience and empirical studies and formulated these tasks in information seeking models. These models offer components at the task level in addition to conceptual and procedural descriptions. For example, Kuhlthau [50] proposed the six-stage model of Information Search Process with specific tasks (task initiation, topic selection, prefocus exploration, focus formulation, information collection, and search closure). Ellis [51] identified six behavioral characteristics of scholarly information seeking: starting, chaining, browsing, differentiating, monitoring, and extracting. Bates [48] used the metaphor of berrypicking to describe the evolving nature of academic search and suggested six common browsing techniques to be supported by SUIs. Belkin’s [52] model of the standard view of information retrieval illustrates information seeking behavior as an interaction between information need and text with four tasks of comparison, retrieve texts, judgment, and modification. Shneiderman et al. [53] also propose an SUI model with four phases: formulation, action, review of results, and refinement. These models offer task descriptions grounded in the information seeking and retrieval practice of academic searchers and are thus critical to the conceptualization of academic SUI usability.

Prior studies in information seeking models and tasks have been used as lens for further understanding of information seeking behavior and tasks of scientists. Al-Suqri [54] integrated the work of Ellis [51], Kuhlthau [55], and Wilson [44] to propose an information seeking behavior model and verified the model elements through a qualitative study. Following the grounded theory approach of Ellis [56], Moral et al. [57] analyzed data collected from one focus group and eight interviews with computer science researchers in a modeling study. An information-seeking process model was developed from the 169 derived concepts with eight information seeking purposes: (1) obtain relevant information about a topic, (2) elaborate a state-of-the-art, (3) find again a forgotten reference, (4) update a bibliography, (5) find a specific information, (6) find a reference for a citation, (7) browse a document collection, and (8) incorporate a set of documents into an existing local collection. Also, in the information seeking task sub-model, five first-level tasks are identified: exploration, reading, filtering, single information-seeking, and chained information-seeking [57].

Exploratory Search.

Early research in search has characterized scientific information seeking as exploratory browsing and search. The exploratory nature of information seeking behavior is discussed by Bates [48] as “berrypicking,” depicting information seeking as evolving with iteration of browsing and retrieval tasks through feedback and query refining. The resurgence of exploratory search since the mid-2000’s [58, 59] has brought attention to the exploratory nature of search and its importance in the design of SUI features in support of academic search tasks.

Exploratory search is one approach to dealing with the complexity of search, which traditional search systems are not built to support [60]. As empirical study has shown, exploratory search can be characterized by the use of short queries, maximum scroll depth, and long task completion time [61]. Marchionini [62] differentiated lookup from browsing and proposed the two features of learning and investigation in exploratory search. The idea that exploratory search is closely related to learning and problem solving [62, 63] has enriched exploratory search as a field of study and demonstrated the uniqueness of academic search. Such extended definition of exploratory search is therefore widely used by contemporary exploratory search researchers. Many of the exploratory search feature prototyped therefore conceptually follow the processes of lookup, learn, and investigate.

With the evolution of technology and data science, exploratory search researchers are able to prototype SUI features to support learning and investigation. For example, temporal presentation of single search query results [15], temporal comparison of multiple entities [10], overview of searched area topic flows [14], comparative overviewing of query result documents [14, 15], and spatial presentation of query results [13, 64]. At task level, these developments echo Shneiderman’s [11] proposal of visual data types and information retrieval tasks (overview, zoom, filter, details-on-demand, relate, history, and extract). In addition to learning and investigation, the exploratory search prototypes have also given much management capacity to the academic searcher; for example, displaying documents collected in session for document management and search trail for query management [15]. Contemporary academic search, therefore, can contain the four processes of lookup, learn, investigate, and manage within one SUI (Fig. 1).

Fig. 1.
figure 1

Processes of contemporary exploratory search in an academic SUI

Full size image

Discovery Tools.

In recent years, discovery tools have become increasingly dominant in academic and research libraries [65] for the integration of information resources across institutional repositories, subscribed commercial publication databases, open access resources, aggregation services, and Web resources. Discovery tools offer a Google-like simplistic search interface and attempt to address the issue of a growing volume and complexity of data. As libraries gradually adopt discovery tools, research has shown that users favor the Google-type single search box feature although usability and content integration are still major problems [66]. Hanrath and Kottman [67] also point out that existing discovery tools lack sufficient integration between the discovery tool interface and content providers such as publishers and content aggregators.

Academic search tasks are in the middle ground of the search activity levels of move, tactic, stratagem, and strategy as Bates [47] proposed. These search tasks are seen in both research lines of information seeking modeling and exploratory search and could serve as the foundation for usability modeling of academic SUI. With the strength to search across information resources with a single search interface, discovery tools have the potential to serve as contextualized academic SUIs if the academic search tasks are embedded through implementation of exploratory search features.

2.3 Academic SUI Usability

While there is a lack of research in the conceptualization of academic SUI usability, usability studies of digital libraries may inform the design of academic SUIs. Usability has received attention from digital library researchers because content and usability are important for users when evaluating digital libraries [68]. In the context of digital library research, researchers have defined usability as “a system has visible working functionality familiar to its users, maximum readability, and useful content that is supported by its environment and aligned with context of use” [69]. Tsakonas et al. [70] proposed a contextualized digital library interaction model, in which usability and usefulness are placed at the same conceptual level (Fig. 2). Usability in this model is defined as the interaction between user and system, and usefulness is defined as the interaction between user and content. Such conceptualization reflects the unique importance of content in digital library context. Content is therefore unique in the usability conceptualization of academic SUI.

Fig. 2.
figure 2

Associations in the interaction triptych framework [70, 71]

Full size image

Jeng [72] reviewed usability definitions and proposed a digital library usability model to add learnability to the ISO factors of effectiveness, efficiency, and satisfaction. Similarly, also based on the ISO 2941-11 usability factors and Nielsen [73], Joo and Lee [74] developed a model for digital library usability evaluation to include the four dimensions of efficiency, effectiveness, satisfaction, and learnability. A study [69] to survey and contrast usability definitions between library researchers and practitioners found 11 attributes of usability in a library context. As researchers [29] indicated after reviewing the digital library evaluation models, the major challenge in usability evaluation in digital libraries remains to find consensus on the definition of usability. This definition issue is complicated by the finding that there is a correlation relationship among the digital library usability factors [71, 72] as general usability frameworks. This phenomenon of a lack of consensus in definition demonstrates the need for contextualized efforts to define and develop usability models for academic SUI.

3 A Formative Model

As Fidel [34] discussed, formative models describe the required context under which a desired behavior happens. A conceptualization of usability for academic SUI, therefore needs to describe the conditions under which information seeking tasks would be better supported before metrics can be specified. Information seeking tasks are grounded in information seeking and behavior models and can be integrated into the exploratory search processes to map with related usability constructs.

Given the complicated nature of usability, it is not surprising that researchers have not been able to reach consensus on the academic SUI usability factors. Some research has, however, approached the issue of user interface usability from a different perspective. Parush [75], in discussing conceptual models and design for interactive systems, proposed five human factors to assess the implications of conceptual models: (1) mental models and understanding, (2) location awareness, (3) visual search effectiveness, (4) operational load, and (5) working memory load. These factors seem to be appropriate in intermediating between the user tasks and usability factors given that they are human behavioral performance constructs and can also be interpreted as usability evidenced through interaction.

Due to the lack of clearly defined usability factors in academic SUI, this study takes the layered architect approach of Folmer and Bosch [28] by including levels of related usability factors and indicators to form a framework of usability. The layered elements are organized to demonstrate the rough relationships between the layers without specific matches among the elements. Based on the above analysis of literature, the proposed framework puts together a formative model of academic SUI by mapping the academic search tasks of users and the usability constructs to indicate the specific conditions to be achieved in order to effectively support the academic searcher. This mapping has led to the academic SUI usability framework depicted in Table 1.

Table 1. Conceptual mapping of task-usability for academic SUI
Full size table

The matching of the information seeking tasks involves identifying and interpreting the information seeking tasks from prior studies. For example, the “Details-on-demand” in Shneiderman [11], which focuses on selected items for details, is similar to Reading in the Prefocus stage of Kulthau [50], in which reading is “Reading to become informed,” and associated with the strategy of “Reading to learn about topic” (p. 238) after locating relevant information. They are therefore grouped under the task of Reading. The task Browsing is taken broadly to mean going through resources such as scanning topic or spatial areas, online collections, or lists of resources. This broad definition approach follows Ellis [51], although similar activities and behavior are sometimes termed exploration.

The information seeking tasks are generally grouped into the category of lookup, learn, explore, and manage. This category roughly matches the search activities of lookup, learn, and investigate of Marchionini [62] and Kuhlthau’s [55] description of actions in the Information Search Process model: seeking background information, seeking relevant information, and seeking relevant/focused information. The process of Manage is added due to the user capacity offered by contemporary search systems. Especially with the recent features in exploratory search SUI prototypes, the process of Manage is thereby added as an extension of information seeking processes.

4 Conclusion

The development of a context-specific usability framework through identification of user information seeking tasks and integration of usability constructs is the focus of this study. This study stipulates the relationship between the information-seeking tasks and the usability constructs in the academic SUI context to reveal how scientific information seeking behavior could be supported in the SUI to achieve high usability. As Shneiderman and Plaisant [17] have indicated, there is a huge gap between information needs and interface actions. Well-developed usability conceptual frameworks, however, would guide the implementation of academic SUIs and their empirical validation for validity and reliability.

Usability is a broad field of study significant to research and design in the industry. This study aims at the conceptualization of a contextualized usability framework grounded in the existing knowledge of information seeking tasks and usability. This framework may be further refined and developed into context-specific usability models of and evaluation tools for academic SUIs. The architect approach to model usability in the context of academic SUI is pragmatic in generally presenting the usability constructs. Future studies should further clarify usability factors and indicators according to context.

A number of SUI features unseen in the traditional information seeking models are supported in recent exploratory SUI prototypes and more innovative features could be developed in the future [6]. These tasks especially enhance the exploration and management processes in academic search and are mostly open-ended tasks at a higher cognitive level, which would trigger the next browsing and search moves. In contrast to general search, in which search is meant to satisfy the information needs of the searcher, future academic SUIs could include more personalization and collaborative academic information management features to facilitate the learning and investigation tasks. Development of strategic SUI features such as collaboration, recommendation, and adaptation and their corresponding usability constructs are also needed for future academic SUIs.

As Galitz [79] indicated, interface design must start with knowing the user. Ethnographic studies may provide new understanding of representative user tasks and information seeking scenarios when the academic search context has undergone changes with the adoption of discovery systems and development of exploratory search research. A desired development may be to integrate the SUI features developed in the exploratory search prototypes into existing discovery systems for empirical evaluation and improvement. Meanwhile, such implementations would be better if guided by clearly contextualized academic SUI usability models.