Competent information search in the World Wide Web: Development and evaluation of a web training for pupils

https://doi.org/10.1016/j.chb.2007.01.029Get rights and content

Abstract

This paper describes the development and empirical evaluation of a web training for pupils (CIS-WEB, Competent Information Search in the World Wide WEB) which aims to convey prerequisite knowledge and skills that are necessary for a competent search for information on the web. The web training focuses on competent information handling and is based on two theoretical analyses. First, a conceptual analysis of information search from the perspective of media literacy research and information retrieval research was conducted and yielded a set of five pivotal content aspects that need to be covered by a web training. Each of these content aspects is characterized by declarative and procedural knowledge components which are necessary for the pursuit of a competent search for information on the web. Second, we conducted a task analysis which conceptualizes the search for information on the web as a problem-solving process and which allows to systematically distinguish between different types of information problems. In the empirical part of the paper two classroom studies are reported. In Study 1, the widespread training concept of a technically oriented Internet training for pupils was evaluated and it was shown that no substantial improvement of web searching skills can be expected from this type of treatment. In Study 2, it was shown that the web training CIS-WEB improves pupils’ declarative knowledge of the web as well as their search performance, thereby outperforming the conventional Internet training used in Study 1.

Introduction

The Internet provides numerous services for the user (e.g., email, instant messaging, chat or groupware), however, undoubtedly one of its primary applications is the search for information in the World Wide Web (WWW). Although the birth of the WWW at the European laboratory for particle physics CERN in 1993 took place less than fifteen years ago, the technical development since then (particularly with regard to powerful web browsers, advanced multimedia plug-ins, user friendly authoring tools for web pages, and the physical network structure) enables individuals worldwide to comfortably publish and retrieve information on almost any topic and in almost any representation format imaginable (e.g., text, pictures, audio, video, animation). At the same time today’s information society creates an omnipresent demand for immediate access to information about diverse topics. Thus, the WWW seems to be the information environment of choice to meet these needs. In line with these developments, school children and young adults take the availability of Internet-based information resources for granted. According to the ARD/ZDF-Online-Study 2004 (Van Eimeren, Gerhard, & Frees, 2004) as many as 95% of the individuals between the age of 14 and 19 years regularly use the Internet in Germany. With regard to the Internet availability in school, it has been shown that, for instance, 99% of the US public schools in 2001 had access to the Internet (National Center for Educational Statistics, 2002).

Contrary to these promising introductory remarks with regard to the potential benefits and the ubiquitous availability of the WWW, however, the web is also characterized by several problematic features that impose additional cognitive processing demands and knowledge requirements onto information seekers. Due to these features, the web can be seen as “a unique searching environment that necessitates further and independent study” (Jansen & Pooch, 2001, p. 244). Characteristic features of the WWW are, for instance, its worldwide open access with regard to information retrieval and authorship, its heterogeneity of the user population, the distributed authorship of documents, a lack of a central agency responsible for structure and quality of document collections, a lack of temporal stability, an enormous amount of information available, the heterogeneity of contents, an unordered network structure, and the integration of interactive multimedia components (Blair, 2002, Hölscher, 2000, Jansen and Pooch, 2001, Jenkins et al., 2003, Kleinberg and Lawrence, 2001). As a result, web users need specific skills ranging from orientation and search planning to information selection and evaluation in order to benefit from the enormous information resources provided by the WWW.

Considering the abovementioned fact that school children quite early encounter the web as an information environment, it seems quite natural that web-related skills should be conveyed and trained in schools. Whereas it seems rather improbable that pupils may acquire these skills and knowledge prerequisites on the fly by engaging in web searches on their own it is a common assumption that attending one of the currently popular short term Internet trainings might provide a sufficient training for competent information search on the web. Nevertheless, it has to be critically noted that existing Internet trainings predominantly focus on technical aspects and neglect most of the abovementioned cognitive aspects of skilled information searches. In our own research, we therefore developed the user-oriented training program CIS-WEB (Competent Information Search in the World Wide WEB) that aims at improving cognitive and metacognitive aspects of pupils’ ability to competently search for information in the WWW.

The development of CIS-WEB is based on two theoretical analyses, which will be described in the subsequent sections in greater detail. First, a conceptual analysis of information search within the fields of media literacy research and information retrieval research was conducted to identify pivotal content aspects that need to be covered by a web training. Second, we conducted a task analysis which conceptualizes the search for information on the web as a problem-solving process in order to systematically distinguish between different types of information problems that need to be addressed by a web training.

Section snippets

Conceptual analysis of information search from the perspective of media literacy research and information retrieval research

In a first step we tried to identify important content aspects that need to be covered by a training program for competent information search in the WWW. We conducted a conceptual analysis within the fields of media literacy research and information retrieval research because in both disciplines users competencies to search for information are of focal interest. The dominant approach in media literacy research is to decompose the overarching concept of media literacy into a list of different

Task analysis: Information search as problem solving

Whereas the preceding conceptual analysis of information search from the perspective of media literacy research and information retrieval research allowed deriving a set of content aspects that need to be covered by the CIS-WEB training, an additional task analysis of information search tasks was conducted to come up with a systematic taxonomy of different information problems. One reason for why such a taxonomy is most helpful in designing a web training consists in the fact that a competent

Design of the web training CIS-WEB

In order to design the web training CIS-WEB, we focused on the five pivotal content aspects identified in the conceptual analysis as well as on the four different sub-goal structures of information problems described by the task analysis conducted. Based on these two analyses, six training modules were developed for the target group of pupils and for implementation in the classroom. CIS-WEB used a problem-based training approach according to Reinmann-Rothmeier and Mandl (2000). All six training

Study 1: Evaluation of a popular conventional web training (“Surfcheck-Online”)

Existing Internet trainings predominantly focus on technical aspects of web search and neglect most of the cognitive aspects of skilled information searches. In particular, they do neither rely on the five pivotal content aspects identified in our conceptual analysis nor on the four different sub-goal structures of information problems described by our task analysis. In order to test whether such a conventional and technology-focused approach is sufficient to improve pupils search skills on the

Study 2: Evaluation of the web training CIS-WEB

Study 2 aimed at exploring whether CIS-WEB – contrary to the conventional web training “Surfcheck-Online” evaluated in Study 1 – is more suitable to improve pupils’ search competencies as well as their declarative web-related knowledge. Probably due to floor effects, the sixth graders participating in Study 1displayed a very poor search performance. Therefore, participants in Study 2 were recruited from grade seven and grade eight, assuming that pupils search competence on the WWW increases

Summary and discussion

The empirical studies reported in this paper demonstrate the necessity of supporting pupils in acquiring skills for the competent search for information in the WWW. Both studies reveal that pupils had great difficulties in their information search despite their substantial Internet experience and their solid declarative basic knowledge. Additionally, these difficulties existed regardless of pupils’ age or grade; thus, even in a long-term perspective it cannot be expected that aging

References (30)

  • D.C. Blair

    The challenge of commercial document retrieval, Part II: A strategy for document searching based on identifiable document partitions

    Information Processing and Management

    (2002)
  • M.L. Gick et al.

    Schema induction and analogical transfer

    Cognitive Psychology

    (1983)
  • N.J. Belkin

    Information concepts for information science

    Journal of Documentation

    (1978)
  • S.K. Card et al.

    The psychology of human–computer interaction

    (1983)
  • A. Clark

    Global abductive inference and authoritative sources, or, how search engines can save cognitive science

    Cognitive Science Quarterly

    (2002)
  • D.D. Cummins

    Role of analogical reasoning in the induction of problem categories

    Journal of Experimental Psychology: Learning, Memory, and Cognition

    (1992)
  • C.S. Doyle

    Final report to National Forum on Information Literacy [Report]

    (1992)
  • M.B. Eisenberg et al.

    Helping with homework: A parent’s guide to information problem-solving

    (1996)
  • R. Fidel

    Research on information access

  • H. Gapski

    Medienkompetenz. Eine Bestandsaufnahme und Vorüberlegungen zu einem systemtheoretischen Rahmenkonzept [Media literacy: A survey and preliminary considerations for system theoretical framework]

    (2001)
  • E. Hargittai

    Beyond logs and surveys: in-depth measures of people’s web use skills

    Journal of the American Society for Information Science and Technology

    (2002)
  • Hölscher, C. (2000). Informationssuche im Internet: Web-Expertise und Wissenseinflüsse [Information search in the...
  • P. Ingwersen

    Information retrieval interaction

    (1992)
  • B.J. Jansen et al.

    Web user studies: a review of current and framework for future research

    Journal of the American Society for Information Science and Technology

    (2001)
  • C. Jenkins et al.

    Patterns of information seeking on the web: a qualitative study of domain expertise and web expertise

    IT & Society

    (2003)
  • Cited by (38)

    • Exploring students' affect and achievement goals in the context of an intervention to improve web searching skills

      2015, Computers in Human Behavior
      Citation Excerpt :

      The extent, however, to which any gender differences are disappearing permanently remains to be determined. Concerning age, we did not find differences between fifth and sixth grade students, supporting earlier research that reported similar search performance of students within a narrow age range (Gerjets & Hellenthal-Schorr, 2008). However, as students move through the grades, they accumulate more experience with web searching, acquire better general knowledge and improve their reading ability.

    • When adults without university education search the Internet for health information: The roles of Internet-specific epistemic beliefs and a source evaluation intervention

      2015, Computers in Human Behavior
      Citation Excerpt :

      Internet users’ information-seeking and source-evaluation strategies can also be fostered by interventions about how to critically evaluate and use different information sources on the Internet. In recent years several researchers have developed and tested such interventions for students in primary school (e.g. Kuiper, Volman, & Terwel, 2008), secondary school (e.g., Argelagós & Pifarré, 2012; Braasch et al., 2013; Gerjets & Hellenthal-Schorr, 2008; Mason et al., 2014; Walraven et al., 2010), or university (e.g., Graesser et al., 2007; Wiley et al., 2009). These interventions range from single, one-hour instructional units (Braasch et al., 2013; Graesser et al., 2007; Wiley et al., 2009) to a series of instructional units over several consecutive days (Gerjets & Hellenthal-Schorr, 2008; Kuiper et al., 2008), to comprehensive training programs embedded into regular classroom lessons lasting several months (Argelagós & Pifarré, 2012; Walraven et al., 2010).

    View all citing articles on Scopus
    View full text