Information extraction from syllabi for academic e-Advising

Abstract

Creating an academic e-Advisor to automate the process of transferring course credits between institutions and recommend courses for further study requires an extensive database of course information. This paper presents an application for creating such a database by automatically extracting relevant information from HTML course outlines stored on an institution’s website and storing it in machine-readable XML. The developed application, called CODE (course outline data extractor), parses a course outline based on its HTML tags and content to build a document object model then applies a combination of web mining, natural language processing, and pattern recognition techniques to automatically classify and extract content useful for the semi-automatic e-Advisor and store it as XML. The current implementation is restricted to HTML course outlines, but the concepts can be extended to other formats of learning objects or entirely different domains. The quality of extraction and classification is evaluated for a corpus of syllabi as proof of concept.

Introduction

The globalization of society, increasing migration between countries, and the popularization of international study, student exchanges, and adult learning has dramatically increased the work load of academic advisors and academic credential evaluation services who offer learners advice on furthering their education. Issues constantly dealt with by academic advisors include identification of equivalent courses between institutions and allocation of transfer credits, recommendation of courses for continued study based on a learner’s academic history, and international degree recognition based on program similarity. These tedious tasks involve intense analysis of learning objects (LOs) such as university programs, academic calendars, course outlines (syllabi), transcripts, and other academic credentials. While the advent of e-Learning and the use of the Internet as an information delivery tool has given academic advisors (and students) the facility to access many LOs on the Web, academic advising remains a time consuming and cumbersome undertaking; however, recent breakthroughs in knowledge engineering and the Semantic Web have uncovered exciting new prospects, making the concept of a semi-automatic academic e-Advisor a reality.

The proposed e-Advising system is a conceptual expert system for continued learning that is intended to automate the process of transferring course credits between institutions and to recommend courses for further study. Such a system would use a learner’s academic history (based on transcripts and other records) and university profiles (based on academic calendars and syllabi) to semi-automatically determine equivalent courses between institutions and suggest the best solution for continuation of study.

The initial idea of academic e-Advising and the drive to automate the process was proposed by Kamarthi, Valbuena, Velou, Kumara, and Enscore (1992), but the described expert system, ADVISOR, assumed the existence of an extensive internal course database containing course descriptions, schedules, prerequisites, corequisites, substitutions, credits, and weights for programs at different institutions. Presently, no such database exists and creating one with an entry for every course in every program offered by every institution would be a long, tedious process; however, the information needed to populate such a database is readily available on most institutions’ websites, in the form of an academic calendar and course syllabi. Therefore, if these existing materials could be used to automatically create a multi-institution course database, the prospect of making a semi-automatic academic e-Advisor could be realized.

This paper presents an approach to extracting information from LOs with the goal of automatically building a course database. More specifically, this paper describes the course outline data extractor (CODE) application, a tool capable of automatically transforming syllabi from semi-structured, human-readable HTML stored on an institution’s website to structured, machine-readable XML (Biletskiy & Scribner, 2005). Many of the information extraction (IE) and classification techniques described in the paper could easily be adapted to automatically extract from academic calendars, transcripts, other LOs, or documents from entirely different domains.

Section 2 presents an overview of the proposed e-Advising system and other fundamental background information needed for an understanding of the rest of the paper. Section 3 describes the work related to this paper, including potential applications of the CODE approach in other domains. Section 4 describes the CODE approach and methodologies. Section 5 presents details of the HTML to XML conversion. Section 6 evaluates the application and Section 7 concludes the paper and discusses potential future work.