Blended learning and pure e-learning concepts for information retrieval: experiences and future directions

Abstract

Today, teaching and learning are mostly supported by digital material and electronic communication ranging from the provision of slides or scripts in digital form to elaborate, interactive learning environments. This article describes the prospects and risks of blended learning and e-learning for information retrieval courses. It deals with adequate content presentation and representation, as well as with interaction concepts and didactic considerations concerning the cost-benefit ratio of animations, applets, and multimedia elements. We present lessons learnt from 6 years of teaching information retrieval in blended learning and pure e-learning scenarios, and derive graded concepts for basic and advanced topics based on a book-like content representation on the one side, and lecture-recordings on the other side. Each concept is complemented by a pragmatic and focussed use of auxiliary elements such as forums and self-tests. Examples for beneficial and misguided applets and animations are given, along with criteria for their differentiation. Finally, critical success factors for technology enhanced learning approaches in the information retrieval field are derived concerning the creation, utilisation, and maintenance of courses. In short, we will argue that taking into account the nature and stability of the presented content, as well as a thorough consideration of the affordable creation and maintenance effort, are crucial for the success of such concepts. In addition, the closer the concept is to pure e-learning, the more important a high digital presence of the lecturer becomes.

1 Introduction

The field of information retrieval (IR) has recently gained more and more attention, and the importance of and demand for corresponding academic programs such as bachelor’s or master’s courses is subsequently rising. The addressed areas of such courses reach from basic IR models and algorithms to advanced topics like cross-language IR, geographical IR or distributed and peer-to-peer (P2P) IR. Obviously, the basic content needed as a foundation for teaching IR remains more or less stable over the years, whereas the content of more advanced topics of this field changes rapidly. This is already a very important aspect affecting the teaching of IR.

In addition, the nature of teaching in general is changing. More and more electronic support is feasible, and is also generally demanded by the students or learners.¹ To demonstrate the various possible settings and to describe our understanding of technology enhanced learning we use the continuum of blended learning introduced by Jones (2006) and shown in Fig. 1.

Fig. 1

Continuum of blended learning (CELT 2008)

The range of digital support in different learning settings stretches from presentation slides provided in digital form as a supplement for a given face-to-face (F2F) lecture to pure e-learning scenarios where teachers and students never meet each other physically. Mature learning management and course management systems (LMS/CMS) exist to support those settings and learning processes such as Blackboard and WebCT² or Moodle³ among others. These systems provide communication means such as forums or chat-rooms and more advanced learning features, whereas the development and the preparation of the content remains the most important task of the lecturer or professor in charge of a course.

This variety of learning settings also implies the use of courses in multiple modes, for example as in traditional F2F-lectures (with complementary digital material) for students on-site, on the one hand, and on the other hand, as pure e-learning courses for employed students, and/or students in a wider geographical region.

This article describes our approach to the creation and preparation of IR courses in this context. This approach, in particular, reflects the following requirements:

• The existence of different content types: content dealing with more stable topics (with low update rates) and content dealing with dynamic and unsettled topics (with high update rates).

• The limited resources for the creation and preparation of the content in digital form.

• Quality (and especially up-to-dateness) requirements causing the need for easy maintenance.

• The requirements for platform independence and for low technical infrastructure needs on the student’s side.

• The demand for printable material, frequently stated by students.

• The ambition to exploit, in an adequate way, the potential of digital media with respect to interactivity and multimedia presentation.

These requirements led us to develop an approach which uses book-like, LaTeX-based PDF-documents enhanced with applets and self-test tasks for the more stable content of our basic IR course, and traditional F2F-lectures supported with PowerPoint ⁴ slides and recordings of the presentations for the more unsettled content of our advanced IR courses.

Obviously, this general concept is not specific to IR. However, it was developed for our IR courses and therefore reflects the needs and peculiarities of this area.

This article therefore outlines the derivation and evaluation of our concept for teaching IR. First, the Sects. 2 and 3 sketch the theoretical foundation by dealing with content and content representation in general, as well as possibilities for interaction in particular. In the following, the actual concept and its two already indicated variants are described in detail based on practical experience. In order to especially cover the field of interaction, the exemplary examination of beneficial and misguided applets as well as the derivation of factors for their differentiation are attached in Sect. 5. Finally, an evaluation of the presented concept and critical success factors for the implementation of this or similar concepts are summarised in Sect. 6. The article is completed with a conclusion and a brief perspective of teaching IR in the future.

2 Content—representation and types

Content is the essence of every learning environment. Of course various techniques can be employed to create and provide digitalised content. Some of the most common of these techniques will be introduced briefly in the following. Also, due to the vast number of possibilities, it seems proper to derive a classification of different kinds of content in order to allow a well-considered association of creation and maintenance techniques with different classes of content. Hence, a simple possible classification and the corresponding associations are presented as a second part of this section.

2.1 Technical possibilities for preparation of content

Looking at technical possibilities for the preparation of content for teaching environments, at least three different ways of creating content may be distinguished ad hoc: slides often used as basis for presentations, self-study environments developed with tools such as Authorware ⁵ or Flash,⁶ and elaborated book-like formats. Taking a closer look, this differentiation can be enhanced.

Table 1 gives a rough scheme and a basic rating of some alternatives for content preparation. The selected techniques for preparation, including reasons for their rating, will be described as representatives for an even broader scope of possibilities in the following:

Table 1 Alternatives for content preparation (−− and − indicate a very negative or negative rating, ○ is used for an average rating, + and ++ indicate a positive or very positive rating)

Text processors The “ease of use” argument as one of the important aspects for content authors suggests usual text processing systems such as Microsoft Word ⁷ or OpenOffice Writer. ⁸ Usually, the corresponding creation process is composed of writing the course content in a text (or word) processor, creating PDF-documents ⁹ and finally providing these documents to the students. In general, there is the impression that this is a good method for specific content types—the particular types will be discussed in more detail in the second part of this section. The effort for creating content is relatively low, and the maintenance effort is at an acceptable level. The support for features such as linking and interactivity are attractive, and the resulting PDF-documents can be used on-screen as well as in a printed version on paper. The material thus created is absolutely suitable for an application in a traditional lecturing scenario although there is the danger of separation for lectures and learning if the provided material is used for self-study. Additionally, this approach is also applicable for settings of blended learning and pure e-learning. One of the main drawbacks of normal text processors is the missing support for large technical documents (formulae, bibliography, etc.)—which, however, at the end seems to be a matter of taste but important for the area of IR.

LaTeX As a specific text processor LaTeX is well-adapted to scientific and technical documents. Although this might increase the creation effort somewhat, these options, in our perception, make the choice of LaTeX appropriate for the field of IR. The remaining characteristics are equal to other text processors and are therefore similar to those described above. At the end, a well-elaborated PDF-document is provided to the students, and potentially includes, for example, links, an index, a bibliography, and marginalia to improve the usability of the document.¹⁰

HTML For many early e-learning courses HTML has been the undoubted successor of text processors and the content has been written directly in HTML. To this end, either low-level or high-level HTML-editors can be used for content creation. In some sense, this led to more or less book-like content presentations usable via the Web. Unfortunately, the support for formulae or bibliographies is not very good. Furthermore, the lacking clear separation of content and layout, as well as the limited print quality are disadvantages of this approach. Nowadays, the use of CMS can only slightly attenuate these drawbacks. The provided material is suitable for print, although it might not be as easy as printing PDF-documents. The application of HTML-content is possible and appropriate for all three types of courses—traditional lecturing, blended and e-learning. However, there is again the danger of separation for the scenario of traditional lecturing.

XML Especially with respect to the missing separation of content and layout for HTML, XML seems to be a promising alternative. In addition, there is the vision of “single source publishing”—the preparation of a single source and the possibility of publishing in multiple modes. An approach considering single source publishing especially for e-learning is for example described in Walsh (2007). Since the publication in multiple modes is easily conceivable in the context of e-learning, this seems to be an attractive alternative. Content could be written once in XML and then published for online use or for printouts, and perhaps even for the use on small devices such as PDAs. This vision led us to give this alternative a serious try, but in short, three major weaknesses became obvious:

• The e-learning oriented standards for XML-schemes hardly cover aspects related to the content representation.¹¹ They are more concerned with the integration into learning management systems or interchangeability. XML-based markup languages for technical documentation, like DocBook,¹² do not really fill this gap.

• Although there are very powerful XML-editors available, the usability for the actual content creator of such an approach is much worse than with typical text processing systems. This aspect turned out to be a problem, especially for the maintenance process and for casual content authors.

• Last but not least, “single source publishing” is an ambitious claim. If one wants to exploit the target formats in an optimal way, one usually ends up with XML-documents containing lots of conditional elements which are hard to maintain.

Slides Slides are used in most courses anyway. This is why they also can be (and usually are) digitally provided to the students. Slides are easy to create and the maintenance effort is manageable. The origins show that slides are appropriate for traditional learning, but only a small add-on to traditional lecturing. At most, it can be seen as a first step towards blended learning, but it is by no means a sustainable basis for pure e-learning scenarios. Steps like the addition of notes included in the provided PDF-document cannot overcome these restrictions. Needless to say: slides, normally provided as PDF-documents, are of course also printable.

Slides & recordings To fill the gap mentioned above, it is conceivable to record the presentations in order to later provide them for different scenarios. Various tools for recording slide-based presentations on the fly (e.g. Camtasia,¹³ Lecturnity, ¹⁴ or Acrobat Connect ¹⁵) exist. Focus and functionality of these tools reach from simple screen grabbing to more elaborated lecture recording or even collaborative work. In our experience, these “rapid e-learning” tools allow a good compromise concerning creation effort and presentation quality. The recordings can be provided to the students in addition to the slides, and form an attractive and sustainable basis even for pure e-learning scenarios. This will be discussed in more detail in Sect. 2.2.2.

Rich media formats There are various authoring tools and different formats for so-called rich-media applications. Some of these tools or formats directly address the field of e-learning (such as Authorware ¹⁶ or ToolBook ¹⁷), while others are not especially dedicated to this field (such as Flash ¹⁸ or SVG¹⁹). These tools and formats are without any doubt very useful for building attractive multimedia presentations. However, in our experience the effort needed to create and maintain high quality e-learning content is nevertheless rather high, especially for casual users. In order to keep the threshold for new users as low as possible, our focus was and is more on the easy creation of scientific and technical documents which can be used online and in a printed version than on attractive rich media presentations. In our perception, such tools are nice if a small presentation corresponding to a 30-min talk is developed, but require enormous effort if the digital representation of a whole semester-long course is desired.

LMS content Most LMS or CMS include facilities for the creation and representation of courses and the course content. The functionality of these facilities differs from simple HTML-editors to elaborated course formats. If an LMS is already being used, the use of these embedded facilities might be attractive. However, there are some issues that have to be carefully considered: What happens if the LMS is replaced? Do the LMS intrinsic facilities support online use and printout? What about formulae or bibliographic data? Since the answers to all these questions can be considered as crucial success factors, these questions led us to the decision to use an LMS-independent strategy for content representation. Possible support for such a strategy might, for example, be SCORM, ²⁰ but in fact there are still many problems concerning the compliance and reliability of technical implementations.

To summarise, these considerations we refer to the three different groups of approaches mentioned at the beginning of this section:

• The first group comprises text processors, LaTeX, HTML, and XML. All of these alternatives finally lead to text-documents which can either be designed as supplements or as substitutes for traditional lectures. We personally think that the substitution is a logical consequence. This substitution involves a change of the character of the meetings between lecturers and students—the prior instructional meetings can proceed to a discussion of open questions and further topics based on the provided material worked over by students prior to a session. An approach falling into this group will be described in Sect. 2.2.1.

• The second group contains all representatives that are based on slides, and can potentially be enriched with corresponding lecture recordings. The combination of a traditional lecture supplemented by slides and the opportunity of more or less automatic creation of blended learning or pure e-learning scenarios makes this approach especially attractive for advanced courses. A possible usage scenario will be described in Sect. 2.2.2.

• In contrast, the third group is more multimedia-based and embraces rich media formats and LMS content. These formats basically target a higher degree of multimediality and interactivity. Although this is an appealing perspective, there are two reasons to not use representatives of this group: first, the created content is bound to a proprietary format and export facilities are usually not lossless. Second, the creation and maintenance effort is higher than with other alternatives. However, there are ways to compensate for the potential lack of multimedia and interactivity for the other approaches. We decided to use applets if reasonable in our scenario. More details will be given in Sects. 3 and 5.

2.2 Concepts for different kinds of content

In order to develop promising concepts for courses it is generally indispensable to take into account the specific characteristics of the represented content. If content is examined at a very general level, two polar forms of content emerge—stable content, on the one hand, and unsettled content on the other hand. Of course, it is not useful to completely assign units of content or courses to one of the forms, but it seems to be a tenable statement that fragments of the content can always clearly be assigned to one side—either stable or unsettled content.

Stable content or courses with stable content are characterised by a more static substance. Generally speaking this primarily applies to fundamental and mature issues, or courses, respectively. Lots of fields in general—and IR in particular—have basic issues that are well delved and constitute the basis of the subject. Often courses containing such issues are less volatile and need almost no changes over the years to remain up-to-date.

In contrast, unsettled content or courses with unsettled content feature a much more dynamic substance. This often applies to issues of special interest, or current research fields. It is obvious that courses dealing with this type of content have the need for frequent changes and updates. In an extreme case, this might result in courses hardly coming through a whole semester without being obsolete.

These considerations have so far been completely subject-independent. Taking a closer look at the characteristics and typical contents of IR, it becomes obvious that these assumptions are adaptable for IR without problems. There are basic issues that facilitate the essential understanding of IR and are indispensable for proceeding in this field. On the other hand, teaching IR is predestined to also cover current research topics like, for instance, XML or multimedia retrieval, where different annual conferences and ongoing research constantly supply new insights. A detailed differentiation for possible aspects of IR is abandoned at this point, because this would require the prior establishment of an all-embracing curriculum. An approach to developing such a curriculum that could be used as foundation for such a distinction is, for example, presented in Bawden et al. (2007), but at this level it is important to merely conclude the validity of this differentiation.

To talk about possible representations of content, we employ the introduced differentiation. Stable content, by assumption, allows for a more static format. The representation and a possible technical realisation for courses showing these features will be embraced by the term “text-based courses” in the following. To present unsettled content, a more dynamic format should be employed. The possibility for changes is the most significant requirement that needs to be fulfilled. The representation and a possible technical solution for courses belonging to this category will be embraced by the term “recording-based courses”. Both representations will now be described in more detail.

2.2.1 Text-based courses

In consequence of the described characteristics applying to more stable content, a promising realisation approach is to present the subject matter text-based, in a book-like fashion.

As stated in Sect. 2.1, there are of course different technical possibilities for implementing large text documents. In order to make an appropriate decision possible standards for publication need to be considered. When looking at the market for digital documents, regarding the academic background, PDF and PDF-documents clearly are the format of choice (Henrich and Morgenroth 2007). Besides benefits like platform-independence and the degree of prevalence, there are ambitions and developments to provide more interactivity in PDF-documents, for instance by including Flash movies or animation. Since interactivity is considered to be an important benefit (see Sect. 3) this fact also points to PDF. Another important aspect is the adaptability of PDF-documents for on-screen reading as well as for printing. That way one document provides different opportunities and can satisfy the needs of different learning styles.

Referring to the technical realisation, the question of an appropriate creation strategy arises. The tool chosen to create the PDF-documents has to be widespread, platform-independent and at best easy to use so that changes and updates are possible without any restrictions. Several commonly known tools provide assistance in creating PDF-documents. The tool fitting the requirements best in the context of IR proves to be LaTeX. It has to be admitted that at first glance it is possibly not as easy to use as for example Microsoft Office products, but considering the features and the relatively low effort needed to become familiar with it, these facts can be discounted.

While actually working out the content, focus should be on a special preparation of the content to provide well-elaborated and visually attractive material as an overall result. As a first step, additional LaTeX-features like footnotes for providing further explanation of terms and concepts, and the appending of a reasoned and helpful index as well as a wide-ranging bibliography or marginalia should be considered. Furthermore, the application of substantial linking concepts can provide additional interactive benefits—links of different scopes can enrich the material:

• Links as anchors to different parts of the learning material in order to illustrate the overall context and the essence of the course. These links allow, in part, the ability to overcome the restrictions of the linear presentation of the course content in a text document, and to support the non-linear exploration of the content by the user.

• Links to further material and external websites to round out and complete the concept, and to provide further starting points for particularly interested students.

Despite the relatively high degree of effort needed, this preparation is justifiable because the main effort occurs by the time the content is processed for the first time. Changes and updates are, firstly, not supposed to be necessary at a high frequency for the considered type of content, and secondly, they are nevertheless possible with reasonable effort.

Obviously, a variation of text-based courses might be the use of textual building blocks from various sources. Traditionally, this corresponds to text books edited by a group of experts, and comprising articles from multiple authors. A modification can be the use of online sources. It turned out, for example, that in the field of algorithms and data structures—and in particular pattern matching algorithms—meanwhile many high quality descriptions and presentations can be found, for example, in Wikipedia ²¹ among other sources. Following this approach, creating a text-based course can also imply referencing selected and quality assured external sources and providing the overall perspective. An example for an approach in the field of IR is presented in Jones (2007).

2.2.2 Recording-based courses

To frame a concept for the presentation of more unsettled content, the requirements are completely different. As already mentioned, the courses are exposed to frequent changes. This fact constitutes the most important demand—the demand for changes to be made as easily and as quickly as possible.

At this point, a variety of technical solutions is possible. Looking at the business sector, where such requirements are common, it becomes obvious that slides—or more precisely Microsoft PowerPoint slides—are the typically chosen way to handle those requirements. Slides with bullet points and figures allow the changes required. Possibilities to easily change the order of slides and insert new slides also support the desired procedure. Last but not least, the format enables an export to PDF-documents so that— without remarkable additional effort—the created slides can be provided in a platform-independent format that has the previously addressed advantages.

Due to the difference from text-based courses, the material has so far not provided any additional benefits—and in particular it cannot be used for pure e-learning scenarios. On account of this one possible solution is to start pursuing a rapid e-learning approach, since the trend and support for rapid e-learning is growing (Archibald 2008). This means that the presentation of the slides is recorded (Ottmann et al. 2003; Lauer et al. 2004) in order to bring off auxiliary convenience for students. A recording system is needed to realise this approach. There is a huge market for recording systems, and one example of a system that arranges sufficient functionality with reasonable financial resources is Lecturnity. ²²

The requirements needed to run the system are conceivably low, and the software can be used off the shelf. A tablet PC (or a notebook with an attached pen tablet) connected to a projector is needed to present the slides, and in addition, an ordinary web cam can be used to record video and audio impressions of the presentation. In contrast, the features provided by the system are remarkable. As extendible basis, PowerPoint slides can be imported. While actually presenting the slides, additional comments and assistance can be added directly to the slides, and additional slides can be created off the cuff if desired. After the presentation is completed, the recording software combines the source slides as well as additional slides created and all supplements on existing slides with the audio and video recordings of the presenter in a single file. The standard result is a proprietary format that can be provided. A free player for viewing the recordings is available. Exports to other standard formats such as Flash or SCORM ²³ exist as well. The recording itself has a split screen. This way the actual slide, the corresponding video recording and additional components for allowing navigation in the recording are available all at once while viewing the recording. Figure 2 shows one possible result of recording with Lecturnity.

Fig. 2

Screenshot of the lecturnity player

Detailed explanations of how to integrate the basically technical solutions presented in Sects. 2.2.1 and 2.2.2 into graded teaching concepts in IR are given in Sect. 4.

3 Interaction

Blended learning and pure e-learning settings provide the potential for new didactic concepts which exploit the technical opportunities with respect to interaction and multimedia. Considerations concerning the use of interactivity and possible multimedia realisations in this field can be found in various publications such as Chapman and Chapman (2004) and Niegemann et al. (2004).

In order to carefully select appropriate techniques, different types of interaction or interactivity must first be distinguished. Figure 3 shows four possible dimensions of interaction for F2F and distributed learning environments. In addition a brief overview differentiating possibilities of interaction with the system and interaction with other learners is given in Table 2.

Fig. 3

Four dimensions of interaction (Graham 2006)

Table 2 Different types of interaction separated by opposite

Below we will discuss the applicability of these different techniques in the context of teaching IR. Figure 4 summarises our estimation of the cost-benefit ratio of the different interaction types for the field of IR—while the x-axis depicts the usefulness and prevalence, the y-axis displays the presumable creation effort. A short description of possible scenarios will be given for each technique along with an explanation of this classification.

Fig. 4

Cost–benefit ratio of different techniques for interaction

Synchronous communication—chat First of all, standard tools like stand-alone instant messengers such as Skype ²⁴ or ICQ ²⁵ are simple possibilities, available without charges, for allowing synchronous communication. Furthermore, possibilities like, for example, chats are normally already integrated into common LMS. Although these possibilities are offered, most of them suffer from lack of usage due to time restrictions. Technical restrictions like the lacking maturity of integrated solutions are another problem which does not facilitate the usage of chats. Of course there are more and more good, albeit commercial systems available that provide support for shared white boards and application sharing, but the use of such systems is again a question of costs and benefits. According to our deliberations, the use of costly systems for synchronous communication is justifiable only for pure e-learning scenarios. For reasons of completeness, it has to be said that the development of stable solutions for synchronous communication is hardly possible without investing a great deal of time and money, and it therefore seems unsustainable. Furthermore, not all areas of Germany currently have high-speed internet access, which makes more advanced synchronous communication solutions a double-edged sword.

Asynchronous communication—forum In contrast to synchronous communication, the use of asynchronous communication—that is most often forums as a commonly known form—is indispensable. Forums are an established form of communication, and are available in abundant supply with desired features and maturity. The creation of own solutions is therefore not necessary. However, some technical aspects should be considered when choosing one system. The existence of possibilities for including formulae, graphs and other appendices in posts—or in other words an appropriate technical support of discussed content—generally fosters the use of forums. Even more important aspects concerning the usage of forums have a more informal character. In order to encourage the usage among all participants, the participation of lecturers and tutors is very important. A short response time also has very positive effects on the use of forums.

Hypertext At first glance, hypertext might not be interactive at all. However, the use of hypertext in general should not be questioned. Although the grade of interactivity could be considered relatively low, hypertext should definitely be employed to offer different kinds of hyperlinks like, for example, cross references to related course parts or links to continuative material (see Sect. 2.2.1). The use of an according tag to differ between obligatory and voluntary material here is very helpful for students.

Computer supported cooperative work (CSCW)/computer supported cooperative learning (CSCL) CSCW and CSCL are very attractive supplemental possibilities. That way students are encouraged to join in, and have the chance to contribute to the course. Different implementations can easily benefit the students, and an integration is therefore a possibility to be considered. Possible scenarios are strongly dependent on the actual content that is supposed to be supported. General and advanced considerations concerning CSCL, CSCW, and the integration into teaching as well as problems of this integration can, for example, be found in Ackerman (2000) or Miao et al. (2005). If an LMS is given, CSCW and CSCL could be based on a wiki or glossary-tools that are normally already provided by LMS. For this reason, the technical effort and knowledge needed is relatively low. Another technique which also falls into the category of CSCL are shared white board applications or tools for desktop sharing. These tools are extremely helpful in pure e-learning scenarios when programming or design tasks are being trained. In blended learning scenarios however the use of such techniques might be less important.

Quizzes and self-tests Quizzes and self-tests generally provide valuable support for students since they can be used for self assessment. Most of the common LMS already supply pre-configured modules for this purpose, such that quizzes and self-tests can easily be created without the need for further technical effort or knowledge. A problem that should not be neglected emerges when looking at educational theory. To be especially valuable, further directions and indications ought to be derived from the individual results (Alberts et al. 2007)—lots of LMS, for example, allow condition-based activation of learning material, such that material can be activated depending on the results of a test. Nevertheless, the cost–benefit ratio of quizzes and self-tests is good, especially in pure e-learning scenarios, since the direct feedback of co-learners is lacking here.

Video and audio clips Video or audio clips can be employed in different forms: shared video clips (for example videos of surgeries in the medical domain), screen recordings, lecture recordings and many others. Examining these different possibilities shared video clips can easily be used as appetisers—even for teaching IR—if they already exist. Special preparation and production of video clips is otherwise a costly process. At this point Google, ²⁶ YouTube, ²⁷ and other providers like Slidestar ²⁸ should be kept in mind because they can probably become an interesting pool for educational purposes in the future.

In contrast, screen and lecture recordings are in our opinion highly recommended. In our experience, the acceptance of this kind of support is very high. Furthermore, many existing recording systems provide a very good cost–benefit ratio since most of the recordings can be made with standard equipment. As a crucial success factor it is indispensable to a choose format and tools according to the desired purpose. For lecture recordings it is, for instance, conceivable to provide recordings combined with a text search on the slides and thumbnails as additional navigation, because simple video recording functionality like fast forward or rewind are not appropriate for lecture recording. In contrast, the manual creation of chapters in a recording requires too much time and effort. Therefore the automatic creation of new chapters in the video for each presented slide and the visualisation of these chapters by slide thumbnails are extremely helpful. The fact that the use of recordings benefits different learning models due to the integration of textual, visual and audio channels will be described in more detail in Sect. 4.

Group projects Group projects are of course a beneficial technique to facilitate understanding, and are related to techniques of CSCW and CSCL. However, implementations of group projects have shown that they are basically not part of the lecture, but rather an additional exercise. Due to the necessary complexity of (group) projects, the development of software (software engineering and software project management) and other aspects like group processes gain the attention in such projects. Therefore, the integration of group projects into (basic) IR courses must be carefully considered. In fact, group projects with an IR topic seem to be worthwhile for follow-up courses which round up students IR and software engineering abilities.

Group assignments For group assignments, basically the same considerations as with group projects are valid. However, because of the lower complexity, group assignments can be integrated more easily and they may be appropriate in advanced IR courses as a complementary element. One problem with group assignments however is the grading, depending on legal parameters it might be difficult to realise group assignments.

Applets and animations Applets and animations are normally employed to visualise important aspects of a particular subjects. Animations usually visualise a more or less predefined situation, whereas applets allow a parameterization. Animations can be implemented with Flash, SVG or others, while applets can for example be written in Java or other programming languages. There are of course many complex subjects in IR which seem to be worth a respective visualisation. However, one has to clearly point out the didactic aim of each applet or animation. Temporal progression can, for example, be displayed with a series of images as well as with an applet or animation. Furthermore, a series of images is much cheaper to produce and, in contrast to applets or animations, it is printable. In our opinion, this suggests the cautious use of applets and animations restricted to situations where the didactic aim and its implementation is clear. A related approach for the field of statistics is described in Darius et al. (2000). An approach for the field of IR will be considered in more depth in Sect. 5.

Games and simulations Games and simulations are of course useful in technical disciplines (for example flight simulators) or in economics (for instance the simulation of economic trade models). In the domain of IR applications for games or simulations are not that obvious. Furthermore, the creation effort of well-designed games and simulations is high—a well-founded concept needs to be derived in order to integrate the desired sophisticated concepts, and advanced technical knowledge is necessary. Especially the area of so-called “serious games” requires profound research. Therefore, the cost–benefit ratio usually prevents the development and integration of games and simulations in IR courses.

Multi player games and multi player simulations The scenario for multi player games and multi player simulations corresponds to the scenario of games and simulation. However, the technical design and implementation of multi player solutions is usually even more complex than for single player solutions. Also the didactic concept becomes even more important, since pedagogical aspects of designing collaborative scenarios as described in Manninen (2002) or Manninen and Korva (2005) need to be considered. Therefore the integration of such games or simulations is very improbable at the actual stage of technical possibilities and support.

This description is meant to give a rough overview of different possibilities. Some aspects of multimedia possibilities have already been included in the considerations above. Further possibilities like 3D-applications or visualisations seem to be valuable and useful for only very special issues, due to the relatively high production costs. For IR courses it seems hard to find useful applications—a possible exception could be constituted by user interface aspects of IR systems where the 3D or visualisation aspects are an intrinsic feature of the systems themselves.

4 Experiences and graded concepts

Now that we have considered different opportunities for the content representation and for the utilisation of interactivity, the focus can be turned to the integration of these building blocks into superordinated course concepts for different teaching scenarios.

Searching for concepts that can be employed to implement blended learning and e-learning, different approaches can be found. Derntl and Motsching-Pitrik (2004) for instance describe patterns for blended, person-centered learning, while others like Chun (2004) work on teaching and learning methodology as well as corresponding e-learning platforms. The following concepts are especially designed to take into account critical success factors like the creation and maintenance effort, and they are based on our experiences from more than 6 years of teaching IR in blended learning and pure e-learning scenarios.

Besides other courses in the field of media informatics, our curricula contains two different IR courses. There is one basic course (IR I) that contains all issues considered to be fundamental, such as IR models, evaluation of IR systems and so on. The second course we offer is an advanced course (IR II) which covers more complex issues, for example advanced pattern matching algorithms or actual aspects of research such as cross-language IR or P2P IR.

Both courses are offered as part of the F2F study paths Applied Computer Science and Information Systems at the University of Bamberg. ²⁹ In addition, IR I is offered as part of the Virtual Master Program Business Informatics (VAWI) ³⁰ and is available as course for students of universities joining the Virtual Bavarian University (VHB). ³¹ All of the courses are offered at least once a year for each program and study path, respectively.

The description above already suggests the following differentiation using the terms introduced in Sect. 2.2. The basic course is implemented as a text-based course, while the advanced course is realised as a recording-based course. Since the technical realisation of the different course types has already been described, the blended learning and e-learning concepts (Henrich and Sieber 2007) will now be presented in detail.

At this point it should be mentioned that all of the material is integrated into, and therefore provided via our university-wide LMS. ³² First of all, the material is collected in this way, and second, there are of course communication possibilities and many other tools—as outlined in Sect. 3—provided by the system that can be used to enhance learning and support for students, as well as work for lecturers and tutors.

4.1 Blended learning concepts

Figure 5 shows the blended learning concept for text- and recording-based courses. As you can see, there are three main parts that frame the concept as a whole. While building the concept we were trying to follow the concept of cognitive apprenticeship and the corresponding framework for designing learning environments described in Collins et al. (1989). References to different parts of this concept, especially the six methods modelling, coaching, scaffolding and fading, articulation, reflection, and exploration will be included in the following description, while the description of the whole underlying concept is skipped and instead referred to Collins et al. (1989).

Fig. 5

Blended learning concept for text- and recording-based courses

Lecture The lecture part is the first part which constitutes the basis for all other parts and differs for text- and recording-based courses. Nevertheless, the lecture part basically represents the modelling and also scaffolding part of the concept of cognitive apprenticeship which have the goal of describing a conceptual model of the processes and delivering support to the students.

The Lecture part for text-based courses has in turn two subparts. The first subpart corresponds to students’ self-organised learning. In contrast to traditional lectures, the subject matter is provided for students—as PDF-documents as described in Sect. 2.2.1—at the beginning of each semester. It is in the individual responsibility of the students to work through the subject matter—or of course in self-organised groups if preferred. The obvious advantages consist in a location- and in parts time-independence for students. The time-independence is only given partly because of a provided schedule that shows the suggested optimal allocation of learning. The schedule is also used as a rough guideline for the second subpart.

To monitor students’ progress and understanding there are, as a second subpart, weekly face-to-face meetings offered by the lecturer. These meetings especially correspond the method of scaffolding and coaching, since there is a permanent contact supervising and supporting the students. The meetings are generally used for discussions. Basically, the material scheduled for the last week is discussed, but of course the discussion is open to all kinds of problems and questions related to the subject. This concept is aimed more at individual support and supervision.

Supplemental material is added to support students’ learning. Two different concepts are used to provide additional possibilities and interaction. The first concept is represented by self-test tasks. These tasks especially refer to the method of articulation and reflection in the model of cognitive apprenticeship. In this way, students get the chance for critical self-monitoring (Fong et al. 2004). At the end of each chapter, self-test tasks addressing the basic concepts and their understanding are available. Normally, self-test tasks are realised as a sequence of multiple choice questions. The type of questions ranges from questions dealing with theoretical aspects to sophisticated computations. The second concept is represented by Java Web Start applications. Java Web Start applications are used to provide animations that are meant to bring the benefits already outlined in Sect. 3. Examples and further details will be given in Sect. 5.

The Lecture part for recording-based courses retains the traditional lecture concept, but completes it with a rapid e-learning approach as described in Sect. 2.2.2. The blended learning concept is therefore composed of traditional lectures, the recording of the lectures—in real time while the lecture is actually being delivered—and the provisioning of the recordings in addition to the existing learning material, directly after the recording is completed. In this way, students can use the recordings for post processing or exam preparation. In this case, reflection or at least possibilities for reflection are also already basically included in the lecture part. Additionally, the recordings support different types of learning, since different channels are involved—learners preferring audio information can be satisfied, as well as learners preferring textual or visual information. Besides, most of the learners normally do not have a clear preference but can be allocated to a hybrid type, which means that a student may learn best if textual as well as audio information is provided.

Exercise The second part is the exercise part. Weekly meetings for tutors and students are held to allow a practical application of the theoretical concepts learnt. In short, the exercise lessons are mainly used to gather experiences in programming concepts of IR, for example pattern-matching algorithms or parts of an IR system like the crawling or indexing component, but scrutinising difficult theoretical concepts is also part of the exercises. On the one hand, this is part of the concept of reflection, but on the other hand this also refers to the concept of exploration, since there are concrete problems that must be solved individually by students.

Additional inducement for students is given by the voluntary offer to collect bonus points for the exam in advance. The exam must first be passed, and afterwards the collected points are added. In this way, students can increase their final score by up to 13% of the potential score. Usually this is a very motivational factor for students. More precisely, there are five tests throughout the semester, each one providing the same possible score. The best three test results constitute the bonus points for the exam. Students are given 2 weeks to work through the tests on their own—a statement must be signed in order ensure that students really work on their own. Besides the bonus points, each student receives a detailed evaluation of the handed in solution. The purpose of this offer is the encouragement of students from the beginning. Often students start to work through the material intensely only shortly before the exam actually takes place, which is obviously to late to get a deeper understanding of the underlying concepts. It is therefore important to concentrate on this fact so that, in order to successfully complete the tasks, students need the desired understanding.

Supervision The third part of the overall concept is labeled supervision and support. Obviously this part aims to particularly seize on the concept of coaching. One of the most important insights of the last years lies in the fact that there are auxiliary but critical success factors for teaching concepts that should not be undervalued.

Especially the use of learning or course management systems—either to support blended learning or to allow pure e-learning—brings several new requirements. The most significant influence is exerted if the lecturer himself participates in and attends forum discussions. Evaluations show that students perceive this fact positively. It seems to encourage students to participate as well.

4.2 Pure e-learning concepts

Since there are virtual learning offers that are supported at the University of Bamberg, and this area is expected to increase in the future, we have been mapping and partly reworking the blended learning concepts to get graded pure e-learning concepts as well. Figure 6 shows the e-learning concept for text- as well as recording-based courses. As you can see, the three parts framing the concept remain, but are changed in detail. In general, the underlying parts of the concept of cognitive apprenticeship stay the same as for blended learning courses—specific changes are described additionally.

Fig. 6

e-Learning concept for text- and recording-based courses

Lecture The first part is still constituted by the lecture part. In this case, self-organised learning represents this part for both types of courses. Depending on the course—text- or recording-based—there are different materials and resources available to support self-organised learning. For text-based courses the learning material can be prepared as usual. PDF-documents following the previously described format and specification presenting the subject matter, and additional material like self-test tasks and Java Web Start applications are made available.

For recording-based courses, the lecturer has to, at first glance, invest additional time, because obviously the recordings have to be created before they can be provided. If blended learning and e-learning concepts are used together to deliver courses, a possible solution might be to offer a recording-based course F2F one semester and a virtual version in the same or the following semester. This way recordings can be re-used. Most parts are normally still up-to-date enough so that at least not all parts have to be recorded again. Independent of the course type, all discussions normally taking place during the lectures have to be moved to forums provided by the LMS. At this point the attendance of forum discussions by the lecturer becomes even more important. As a technical remark, it should be stated that the recording of a 90-min lecture has a size of 120–150 Mbytes. Depending on the students network connection, this can sometimes cause problems. Therefore we send the students a DVD with all recordings on demand. Obviously, this is only possible in a scenario where the pure e-learning course is given in a semester following the semester in which the recordings are made. Of course the scaffolding and coaching is reduced that way. Therefore, more attention needs to be paid to the third part of e-learning course—the supervision part.

Exercise As the second part, the exercises remain but have to be reworked. Concerning this part, it is hardly possible to replace the practical experiences that can be gathered in the lessons. A possible solution is the provisioning of tasks equal to the tasks that would be worked on in the F2F exercise lessons. The processing of the tasks by students can be done individually or in groups if there are students located in the same region. If the LMS supports shared white boards or application sharing, this option can be used to cooperate. The support and supervision—coaching referring to cognitive apprenticeship—that is normally given in exercise lessons has to be transferred to support via e-Mail or forum discussions in the LMS. Altogether, supervision and support requires much more time and additional effort, since this is actually the only possibility for students to get feedback. The possibility to collect bonus points can be offered as described, but to enhance supervision it is even more important to give detailed feedback for every solution.

Supervision The third part, supervision and support, constitutes the most important part for e-learning concepts. As already mentioned, due to spatial separation of the participants, all F2F parts need to be replaced, and therefore support via e-mail or via forum discussions has to be raised. This concept has already become an important part of the lecture and exercise parts, and cannot be separated like in the blended learning concepts.

5 Examples for applets

As described in Sect. 3, interactivity is a very important part of successful learning concepts. Regarding learning theory and technical possibilities, there are lots of different ways to realise these benefits. Applets are one common opportunity to visualise important concepts and foster a deeper understanding by providing interactivity for students. Technically speaking Java Web Start applications are a possible solution that provide some benefits. Due to their stand-alone nature, applications are independent from any system and can be used in various ways. They can, for example, be included in an LMS in order to present all resources in one place. The independent nature is also very important concerning creation and maintenance efforts, as well as aspects of re-use.

Most of the time there are several concepts in every course or content package for which such an application can provide useful visualisation and additional understanding. Depending on the characteristics of the content there is a widespread range of applications conceivable and possible. IR courses are particularly qualified to be supported by applications, or more specifically applets. Basic as well as advanced parts include, for example, numerous algorithms and concepts that can be implemented in this way with different goals. In this section, examples for beneficial and misguided applets are given and criteria for their differentiation are presented.

5.1 Beneficial applets

If applet design is carefully considered, applets can easily complement the entire content and concept, as well as the learning experiences of students. In the field of IR, lots of efficient applets that can be classified as beneficial are available. By way of example two of them will be described now.

Example: visualising clustering algorithms In Fig. 7 you can see an applet visualising clustering algorithms and their progress. The theoretical part of the corresponding IR course explains different possibilities for clustering unorganised items. Since the theoretical descriptions of the introduced concepts can easily sound similar or even equal without a profound understanding, it is useful to visualise the progress and results of the different algorithms.

Fig. 7

Progression of a clustering applet visualising a clustering process

The application allows variable settings in the beginning. First of all, point clouds representing the items to be clustered can be set. Therefore either predefined sets can be used, or the point clouds can be set manually, and a combination of both methods is also possible. As usual in practise the number of final clusters to be reached can be set too. Methods of how to examine the number of final clusters are skipped at this point because the focus of the application concerns the visualising of the algorithms themselves. Last but not least the algorithm which has to be applied can be chosen. All theoretically described algorithms of the IR course are available as a choice. While finally computing the chosen algorithm, the different clusters and, more importantly, the change and growth of different clusters is visualised by different and changing colours.

If the application is executed with different settings of point clouds, or more important different algorithms for equal point clouds, the diverging procedures and strategies of the algorithms easily become obvious. This way the theoretical concepts are expanded with practical aspects. Understanding and internalising the theoretical aspects is facilitated in this manner. The didactic aim of this applet is to boost the students’ intuitive understanding of the character of the emerging clusters for the different algorithms. Evaluations have shown that this aim can be achieved.

Example: explaining the implementation of pattern matching algorithms In Fig. 8 an applet designed to foster the deeper understanding of pattern matching algorithms is shown. The focus of this applet lies in a combination of understanding the algorithm on an abstract level, and the programming concepts. In this way the link between theoretical knowledge and practical experiences is created. A profound understanding of the algorithms and the programming skills of students ought to be enhanced simultaneously.

Fig. 8

Progression of an applet combining understanding and implementation of pattern matching algorithms

The applet consists of two main parts. At the top of the applet, configuration is possible. Since pattern matching algorithms are explained, the text to be searched in and the search pattern need to be specified. Analogous to the clustering applet, all algorithms described in the corresponding part of the IR course are available as selection for the actual computation.

While computing the chosen algorithm, the applet shows the actual state of computation in different levels of abstraction. At the top of the visualisation the text and the search pattern are continuously displayed. If part of the algorithm, the border- or sborder-table is displayed—first in its initial empty state, changing during the computation of the algorithm and finally calculated when the pattern matching actually starts. Figure 8 shows the actual match between text and search pattern, as well as the shifts executed while the comparison is actually performed.

To create the desired link in the students’ understanding, the Java code needed to implement the chosen algorithm is shown as a second main part in the lower area of the applet. To facilitate the understanding of the code, the position corresponding to the actual state of the algorithm is shown while computing the whole algorithm. Breakpoints can be set in order to work through the code in detail. In combination, the lower right area of the applet shows the actual values and the changing of the most important variables used in the Java code.

The visualisation of all changes—the calculation of the border- and sborder-table, the shift and comparison of text and search pattern, the change of variables as well as the progress and actual position in the corresponding Java code—at one point in time is seen as the most remarkable benefit of this applet. Students often declare a notable lack of knowledge of how to translate high level understanding of algorithms and concepts into concrete implementations when it comes to first professional experiences. This applet aims specifically at bridging this gap.

5.2 Misguided applets

On the other hand, lots of applets fail. Reasons for failure are widespread, and for this reason two misguided applets and their deficiencies will be described in the following examples.

Example: computing the Soundex algorithm In Fig. 9 an applet computing the Soundex algorithm is shown. The Soundex algorithm is a phonetic algorithm indexing words according to their sound. It aims at the mapping of similar sounding, but differently written words to the same structure, and is therefore part of the IR field.

Fig. 9

Applet computing the Soundex algorithm

The options and possibilities of this applet are limited. Any number of words can be entered and the computation can be performed. The visual result consists of two parts. First of all, the results of the computation are shown directly beneath the delivered words. Second, the detailed steps of the computation, as well as intermediate results are shown for every single word in the lower area of the applet.

The main reason for the failure of this applet lies in the simple concept that is implemented. The applet obviously brings no additional understanding or visualisation of the underlying concept. It can of course be used as a control tool for computations that were previously done manually, but interactivity encouraging the practical application or facilitating deeper understanding cannot be found at this stage.

Example: computation of string distances In Fig. 10 the result of an applet computing the string distance is shown. The computation of string distances is also covered in our basic IR course. Different concepts of how to compute the distance between two strings are considered. Two different methods—the Hamming distance, and the Levenshtein distance—are addressed in the corresponding applet.

Fig. 10

Result of an applet computing the string distance using two different methods

The options for students to interact are conceivably low here, too. Of course, the algorithm to compute the distance can be chosen, and the words whose string distance is to be computed can be entered. For computing the Hamming distance there is an additional restriction, the two words have to have exactly the same length. The result of performing the Hamming distance is presented after the computation is finished, and is confined to presenting the distance. No further explanation of the performed computation or single steps is given. When computing the string distance according to the algorithm of Levenshtein, no additional options are available, but in contrast to the Hamming distance, the result consists of a matrix that can be used to reproduce the solution.

Basically, the reasons for the failure of this applet are the same as the reasons which apply to the applet for computing the Soundex algorithm. A slight difference applies for the Levenshtein algorithm, because the computation is more sophisticated, and having a tool to control own computations can be more useful. Drawing an overall summary, there is no visualisation of the addressed concepts, and deeper understanding is only provided at a very low level.

5.3 Criteria for differentiation

Looking for criteria to differentiate beneficial and misguided applets or similar applications, the goal and underlying learning theory applied when designing the applet seem to turn the scale.

We discovered that the main factor for developing beneficial applets lies in an elaborated, underlying draft for each applet. Before starting to actually develop the applet, the aspects that should be clarified or visualised by the applet need to be pointed out in order to be able to implement a useful applet. While working out the draft, the attention should therefore be turned to aspects of learning theory and learning methods.

In this process—besides others—three important aspects should be considered:

• Complexity: Is the topic to be addressed complex enough to justify the effort? In our perception, string distance algorithms are not complex enough, whereas sophisticated pattern matching algorithms are. Obviously, this criterion is relative, but it turned out that students do not use applets addressing topics which can be understood easily from a textual description, an image, or a short presentation.

• Visualisability: Some topics might be very complex (for example a hard proof of a theorem) but they can hardly be visualised. If there is no appealing idea for the visualisation, an applet is not the weapon of choice. Furthermore, the visualisable aspects have to coincide with the aspects that should be clarified by the applet. Otherwise we will end up with a nice applet that—in the worst case—gives the student the impression that he or she has understood the important aspects without actually meeting the important points.

• Separability: An applet should concentrate on a certain aspect or the relation between two aspects. If a huge context is necessary, the focus of the applet may get lost.

The design of an applet has to clearly address the didactic aim: Which aspects are addressed? Is it, for example, the result of a process (e.g. the characteristics of a created clustering) or the process itself (e.g. the progress of an algorithm)? Does the applet aim at facilitating a basic understanding or implementation know-how?

Since the creation of well-elaborated applets is costly, the question is which applets among the potential applets should be realised. Here the results of exams or course evaluations are beneficial. Aspects which turn out to be marked as critical in those evaluations are potential candidates for applets. For those candidates, the aspects pointed out above (complexity, visualisability, separability) have to be checked, and in case a draft for a potential applet should be developed, one must keep in mind that applets and animations are a didactic means, useful in adequate situations.

If such a concept is developed in advance, most of the misguided applets can be avoided. Furthermore, the elaboration of the mentioned concept shows the cost-benefit ratio of the future applet. That way the development of expensive applets providing very low benefit, and therefore ultimately suffering from minor usage, should be prevented.

6 Conclusions

This section on the one hand delivers a brief evaluation of the presented concepts with respect to the students’ perspective, as well as a more general classification of the concepts using the “ten good practices in the design of online course material” proposed by Alberts et al. (2007) and the discourse of Chew et al. (2007). On the other hand, this section also derives critical success factors for the design and implementation of individual concepts.

6.1 Concept evaluation & classification

6.1.1 Evaluation

Since the concepts were derived taking into account the students’ perspective, but nevertheless were developed from lecturers, we were trying to integrate the perspective of students by evaluating courses following the two introduced concepts against different criteria at the end of each semester. We are presenting exemplary evaluation results of one text-based and one recording-based course, that took place in the last semester—the results are based on 25 evaluated questionnaires.

There are some general questions that are similar for text-based and recording-based blended learning courses. Students were asked general ratings for the lecture or the lecture-like part, and the exercises; the scale for the general questions reached from 1 (very good) to 6 (very bad). The grading for the lecture concept as a whole and the associated exercises ranked from 2.1 to 2.6 for text-based courses and from 1.6 to 2.0 for recording-based courses.

The special focus for text-based courses was on questions about organisation and structure of the learning material and the weekly F2F meetings. The scale available reached from 1 (disagreement) to 4 (agreement). The learning material was rated in different criteria, for example, structure and understandability, as well as a personal estimation on how good the learning material covered the underlying area, and how helpful the learning material was for understanding the important concepts of this area; the average values of ratings in this area reached from 3.1 to 3.8. Concerning the weekly F2F meetings, students basically had to rank the degree of help that was offered by these meetings; the average value was 3.1 in this area.

The focus for recording-based courses was naturally on rating the use of recordings. The plain result was, that all participants were using the recordings, some for equalising missed lectures, some for post processing difficult parts, and some for exam preparation.

Drawing an overall resume for courses of the blended learning concept, all values are positively above average, and students seem to be comfortable with the concept of text-based as well as recording-based courses.

Detailed evaluations for the concept of pure e-learning courses have shown, that students are generally pretty satisfied with this approach, too. Evaluations are also taking place at the end of each semester, and each proceeded course is evaluated properly. The evaluation results presented below refer to two text-based and one recording-based course of the last two semesters, that is about 40 fully completed and evaluated questionnaires. We restricted the results to the last two semesters in order to present the results as actual as possible. Evaluation criteria were, on the one hand, criteria concerning the quality of supervision, for example, supervision of content-related or technical problems as well as organisational support. On the other hand, criteria concerning the content of the courses, for example, the intermediation of knowledge, comprehensibility, or structure of the presented content were evaluated. The evaluation scale allowed grading every criteria on a scale from 1 (low) to 7 (high), additional space was available for comments on especially positive or negative features, as well as further comments.

Although the students admitted that the effort needed to pass the courses was above average by tendency, the responses were averagely very positive. All criteria concerning the quality of supervision were ranked very high, at least 6.0 or above, in all courses. There is no recognisable difference between recording-based and text-based courses. The criteria concerning the content of the courses are rated little worse, but still very high relating to the evaluation scale. Concrete values vary from 5.5 to 6.7 for text-based courses, and lie closely around 6.0 for recording-based courses. Analysing the spaces for free comments, it became obvious, that especially the provided recordings were appreciated when looking at the recording-based course. There were a few students, that had start-up difficulties, because the learning material for recording-based courses is organised as a structured collection of bullet items and not continuous text, but the recordings and possibilities available by forums smoothed these problems. Concerning the text-based courses, many students commended the systematic book-like structure of the learning material. Some students always go to the bother of suggesting concrete improvements, and as far as it is possible we are trying to consider the suggestions for the next semester.

The most salient overall result was, that a few simple but effective facts were mentioned very positive across all courses. Extremely positive responses gained the fact, that the lecturer itself was actively attending the forum discussions and answering questions of every kind. The short response time provided by lecturer and tutor, is also a circumstance that was appreciated a lot.

6.1.2 Classification

Alberts et al. (2007) postulate “ten good practices in the design of online course material”. These practices will now be used for further classification of the presented concepts. Table 3 describes each practise and an explanation on how these practises are considered and fulfilled.

Table 3 Ten good practices in the design of online course material regarding the concepts of text- and recording-based courses

In summary, among other aspects, the importance of a specification of the desired learning outcomes and the need for adequate feedback to the learner as well as for interaction in the e-learning environment is stressed. Additionally, the need for a clear structure and navigation and linking concept is emphasised.

It is important for the classification shown in Table 3 to be aware of the fact, that this table is trying to explicitly associate parts of the presented concepts with the 10 practises of Alberts et al. (2007). Different points of criticism can also be found. Chew et al. (2007), for example, argue that the central problem of blended learning is the missing focus on educational theory. We have to admit, that these thoughts could have been applied to our concepts as well. Due to the incremental development and the technical focus, the concepts suffered from the lack of underlying pedagogical theory in the beginning. Therefore, as part of the ongoing development, the concept was substantiated with mature pedagogic concepts as, for example, the concept of cognitive apprenticeship and subsequently slightly adopted as needed—the results and associations of single building blocks have already been described in Sect. 4.

6.2 Critical success factors

Besides specific evaluation results as described above, it turned out that a healthy balance between existing resources and the desired outcome—considering the view of lecturers as well as students—is one of the main determining factors. On a more pragmatic level, we have identified the following five critical success factors for efforts classed with the continuum of e-learning:

• Concept Obviously, each course offered needs a clear concept. This comprises the addressed content as well as the targeted audience. The audience may be on-site or not, and also mixed audiences are possible—which will become more and more standard. Furthermore, situations where the same content has to be presented to different target groups (on-site and remote) are becoming common. The configuration of a course has to take into account the concrete situation, and it has to cover the aspects of lecture, exercise, and supervision. It is not useful to overwhelm the students with material. Instead, a clear task assignment is needed for the components of the concept.

  Let us consider a blended learning scenario with lectures given, slides provided as PDF-files, and lecture recordings. In this case, the concept might be that the students participate in the lectures. Beforehand they should print out the slides and make remarks on the printed slides during the lecture—for this, the slides have to be provided early enough. The lecture recordings are provided for post-processing of the lectures and exam preparation. Obviously, this use of the lecture recordings causes the need for a sophisticated fine-grained navigation mechanism in the recordings. The students will usually use the recordings with questions in mind such as “Which explanation did the lecturer give for that figure on this slide?”

  These considerations point out that the requirements for the single components constituting the concept can only be defined with the concrete usage scenario in mind.

• Creation The creation and the creation effort are crucial for the success. As pointed out in Sect. 2.2 we designed two types in this respect—text-based and recording-based courses. Obviously, other types are conceivable as well. However, the technical setting used for the preparation of the content has to ensure a good cost-benefit ratio and an appropriate content presentation. For example, the audience of our pure e-learning course on advanced IR topics is not that large. Hence, the creation effort for a sophisticated multimedia presentation would not be justifiable. Here, our on-site course on the same topics together with the lecture recordings allow us to provide an appreciated offering to the students.

• Maintenance Apart from the course creation, the continuous update is important. Of course, this is more important for advanced topics than for stable, basic content, but a clear course maintenance concept is indispensable in any case. Many nice multimedia-presentations used in e-learning scenarios turned out to be hardly maintainable at all. After a few years they became obsolete. Therefore, a sustainable concept has to address this aspect, and the maintenance costs have to be justified, usually by the number of participants.

• Utilisation In some sense, the utilisation might be considered a sub-item to the overall concept. Nevertheless, its importance makes it worthwhile as an item on its own. Components of learning arrangements which are not utilised are in most cases an indicator for an unclear concept. For example, applets addressing topics which, without problems, become clear from the lecture will not be used, or a chat provided within the LMS will not be used if either the course participants meet physically, or they use a general purpose instant messaging program anyway.

• Participation Finally, the participation in the learning environment is an important point. This is based on group dynamics and other social aspects. Here the lecturer and the tutor(s) play an important role. They need to encourage participation and they should themselves participate in course forums and other interaction facilities.

Obviously, the above-mentioned aspects are worthy of discussion. Nevertheless, according to our experiences they can be seen as essential aspects for successful blended learning and pure e-learning courses.

Finally, it should be mentioned that the success factors may vary depending on the target audience. For example, Rosenberg (2006) describes a more company-oriented perspective on e-learning where e-learning is closely related to knowledge management. In such a setting, slightly different priorities may apply.

7 Future work and perspectives

If we keep the distinction drawn above, there are basic and advanced issues. Looking at future challenges and developments, different strategies seem to be obvious due to the special characteristics of content or concepts as described above.

Basic aspects For basic courses it is likely to suppose the development of a core tutorial similar to other mature subjects, like algorithms & data structures. There are already many different studies working on establishing such a structure, and in the end, a profound textbook for basic issues of IR. The range of studies is widespread. Bawden et al. (2007), for example, are working on establishing a broad curriculum in the general context of IR. Similar approaches work on examining optimal teaching strategies in the general field of IR. Mizarro (2007), for instance, examines a useful order for teaching Web IR. Steps towards establishing textbooks have already been taken by various authors. Baeza-Yates and Ribeiro-Neto (1999), for example, give a broad view of basic IR issues while other authors like Grossman and Frieder (2004) focus on special areas—in this case algorithms and heuristics.

Furthermore, an extended applet sharing should be deployed. As described, design as well as implementation of applets demand lots of time. Therefore, if applet sharing is established, different lecturers can benefit from each other. Efficient and useful applets get the chance to become widespread due to their positive usage statistics. The additional benefit, especially for students, is also obvious at this point.

Advanced aspects As already described, advanced issues, in contrast to basic aspects, have different characteristics, and therefore a different strategy is likely to be developed and used. Since the field of IR has lots of advanced issues like P2P IR, cross-language IR, XML IR, geographical IR and many others, the assumption of having an expert for every single special aspect in charge of teaching these special issues at a single university is unrealistic. Instead, it is likely to anticipate scenarios like the following:

The lecturer is an expert for one or two special aspects of IR. He or she is therefore able to develop well-elaborated learning material and outline the special characteristics and issues of these particular areas or aspects. In order to present advanced issues for a broad range of special areas of the widespread field of IR, content sharing needs to be established. Lecturers should be able to use experts for other special areas or the learning material that they have prepared for their teaching. That way the quality of the learning material can be brought to a higher level, since lecturers can simply use the expertise of other lecturers to complete and improve their teaching. In return, the lecturer shares the content prepared and provided by himself. An extended applet sharing, as described for basic topics, can of course also be part of the more advanced topics.

There are already researchers and teachers applying strategies like these. Jones (2007), for example, describes a concept using key texts from different sources in order to cover a wide area and teach students important parts of the widespread IR field.

In summary, e-learning scenarios seem to enable and demand more collaboration and communication between lecturers. Therefore, in the future concepts for teaching IR should not only consider the concepts of blended learning or e-learning, but also possible collaborations in order to improve the experience of teaching and learning IR for all participants.