Keywords

1 Introduction

Textbooks are considered to be the primary teaching instruments that are necessary to guarantee the quality of education. For students with print disabilities, it is important that various types of accessible textbooks should be available so that students can choose one as appropriate. In Japan, large print textbooks and DAISY textbooks have been available for students with print disabilities in compulsory education. Recently, textbook publishing companies started producing large print textbooks by themselves at their expense. The quality of large print textbooks is very improved. DAISY (Digital Audio Accessible Information System) is a world standard of accessible books for people with print disabilities [1]. A number of textbooks are translated into DAISY by the efforts of volunteers. Many students with print disabilities in special supported classrooms use either large print textbooks or DAISY textbooks. However, the number of students in ordinary classrooms who use these textbooks is very few.

As new types of accessible textbooks suitable for use in ordinary classrooms, the authors have developed three kinds of textbooks with audio support: Multimodal textbooks [2, 3], Sound-embedded PDF textbooks [4], and Audio textbooks with AR technology. Multimodal textbooks are paper-based textbooks with audio support utilizing invisible 2-dimensional codes and digital audio players with a 2-dimensional code scanner. Because invisible 2-dimensional codes are printed overlappingly on normal characters of regular textbooks, the look of multimodal textbooks is almost the same as regular textbooks. When a 2-dimensional code is scanned, the corresponding speech sound is reproduced. Sound-embedded PDF textbooks are digital interactive textbooks that run on PDF viewers such as Adobe Acrobat Reader. When a text is tapped on the screen, the text is highlighted and the corresponding speech sound is reproduced. Audio textbooks with AR technology are one of augmented reality applications on a regular textbook. When a page of a regular textbook is captured by a camera of a tablet, and some part of the textbook on the screen is pointed, the corresponding speech sound is reproduced.

In order to ease the production of various types of accessible textbooks for students with print disabilities, a unified production system was developed. The information of the structure of a document is essential for the production of each type of accessible textbooks. By Japanese “Barrier-free textbook law”, we can obtain PDF data of regular textbooks from textbook publishing companies. From a PDF file of a textbook, the unified production system automatically recognizes the layout of the textbook. Since the PDF file of a textbook only contains the information of characters, lines, and pictures, the structure of a document such as headings, paragraphs, and sentences has to be recognized.

The unified production system outputs production master data of the three kinds of textbooks with audio support. Since the unified production system was developed, the cost of production of accessible textbooks was much reduced. In addition, the number of mistakes was much decreased.

Fig. 1.
figure 1

Screenshot of the unified production system

Full size image

2 Unified Production System for Accessible Textbooks

The unified production system (Fig. 1) is an application software developed on Java with standard widget toolkit (SWT). It runs on multi-platforms: Windows and OS X. Apache PDFbox [5] is used to obtain character information and page images from a PDF file. The table 1 shows the system requirements.

Table 1. System requirements
Full size table

The system was originally developed for the production of multimodal textbooks. Since the production master data of multimodal textbooks contains enough information to produce accessible textbooks in other formats, the system was extended.

2.1 Usage of the Unified Production System

The system is designed so that average PC users can use it smoothly. It has the following features:

  • Major functions can be accessed by pushing icons on the tool bar (area 2 of Fig. 1).

  • At the center, a page image of a PDF file is shown (area 5 of Fig. 1). The scale is adjustable with the scalebar. Pages are selectable from the list (area 3 of Fig. 1).

  • Lines in a page are editable. Each line has the information of characters composing it. They can be combined and connected to form a “sentence”. Creation of a new line and resize of a line can be done with mouse operations (Fig. 2).

  • A set of sentences form a paragraph. Paragraphs are editable (area 7 of Fig. 1).

  • Speech sound files can be assigned to each sentence with drag and drop operations. Sound formats, wave and mp3 are supported. Speech sound files are combined automatically when more than 2 files are assigned to the same line. The assignment of speech sound files can be checked on the system.

Fig. 2.
figure 2

Resizing a line with a mouse operation

Full size image

2.2 Automatic Layout Analysis

The system automatically analyzes the layout of a PDF file. The analysis is done in the following steps:

  1. 1.

    The system uses Apache PDFbox to obtain character information from a PDF files. The information of font name, font size, position, size of bounding-box and order of characters are obtained.

  2. 2.

    A sequence of characters with the same font name and the same font size are combined and connected if they don’t contain a full stop character at the middle. The combined charters form sentence Full stop characters are chosen from ‘.’, ‘:’, ‘;’ , ‘,’, ‘?’, ‘!’, ‘’ and ‘’.

  3. 3.

    A class is automatically assigned to each sentence. A class is chosen from “HEAD”, “BODY”, “NOTE”, “LIST” and “AUXILIARY” by font size and position in a page.

  4. 4.

    A sequence of sentences in the same class are combined if an indentation dose not exist between lines. They form a “paragraph”.

2.3 Creating Production Master Data with the System

Production master data are created by the following directions.

  1. 1.

    Choose an input PDF file.

  2. 2.

    The system automatically analyzes the layout of the PDF file. The results of analysis are displayed on the center of the window.

  3. 3.

    Errors of the automatic analysis can be corrected by users with the editor of the system. It is easy because of the simple interface of the editor.

  4. 4.

    The system outputs a manuscript for reading the document. The script will be transferred to volunteers or a speech synthesizer.

  5. 5.

    Assignment of sound files to sentences can be done by drug and drop operations. Users can check the assigned sound.

  6. 6.

    By choosing a type of accessible textbooks, the system outputs a production master data.

3 New Types of Accessible Textbooks

As new types of accessible textbooks suitable for use in ordinary classrooms, three kinds of textbooks with audio support were developed.

It is desirable that each student with print disabilities can choose an appropriate one from large print textbooks, DAISY textbooks, and these 3 kinds of accessible textbooks.

Fig. 3.
figure 3

Multimodal textbooks

Full size image

3.1 Multimodal Textbooks

Multimodal textbooks (Fig. 3) are paper-based textbooks with audio support. Invisible 2-dimensional codes are printed overlappingly on normal characters of regular textbooks. When a 2-dimensional code is scanned, the corresponding speech sound is reproduced. We think multimodal textbooks are best in ordinary classrooms because the paper textbook looks the same as a regular textbook, the price of a digital audio player is reasonable (about 30 euro), and the digital audio player is small and can be used with a headphone.

Since sight and hearing help each other, students can read the textbooks easily and correctly.

3.2 Sound-Embedded PDF Textbooks

PDF is a standard file format used to present document independent of application software, hardware, and operating systems [6]. Sound-embedded PDF textbooks (Fig. 4) are PDF documents with embedded sounds that run on a PDF viewers such as Adobe Acrobat Reader [7]. Digital interactive functions of PDF such as highlight, reflow, black and white reversal are also available in sound-embedded PDF. A computer or tablet with PDF viewer is necessary to use sound-embedded PDF textbooks.

Fig. 4.
figure 4

Sound-embedded PDF textbooks

Full size image

3.3 Audio Textbook with AR Technology

Audio textbooks with AR technology (Fig. 5) are one of augmented reality applications on a regular textbook. The application was developed for Android platform. When images of a page of a regular textbook captured by the camera of a tablet device are converted into a feature vector, the page number of the images is identified by comparing the feature vector to vectors in the feature space of the page images of the textbook.

Some part of the textbook on the screen is pointed, the corresponding speech sound is reproduced. A tablet device with a camera or smartphone is necessary to use audio textbook with AR technology. A regular textbook is also necessary. This textbook is suitable for temporary use.

Fig. 5.
figure 5

Audio textbooks with AR technology

Full size image

4 Effectiveness of the Unified Production System

Before the development of the unified production system, the production of multimodal textbooks was done with Garyotensei (Apollo Japan Co., Ltd). Garyotensei is an image editing software for 2-dimensional code patterns. Its usage is like a bitmap editor such as Microsoft Paint, which is used with a lot of mouse operations.

In order to evaluate the unified production system, we compare the time needed to produce dot patters of 2-dimensional codes for a collection of pages from textbooks with our system to the time needed with Garyotensei. The collection consists of 6 pages; story, expository writing, work, Kanji table, discussion and essay from textbooks in Japanese language for elementary school published by Mitsumura Tosho publishing Co., Ltd. The number of pages, characters and sentences of the collections of pages are shown in Table 2.

Table 2. Number of pages, characters, sentences
Full size table
Table 3. Result of experiment
Full size table

For experimental subjects, 3 university students are employed. They are asked to create dot patters of collection of the pages by the different ways. The one way is using Garyotensei and the other is using the unified production system.

The result is shown in Table 3. The time needed with the unified production system is about one-third of the time needed with Garyotensei.

5 Conclusion

A unified production system was developed for efficient production of 3 new kinds of textbooks with audio support. The cost of preparation of production master data of these new textbooks was much reduced with the system. In addition, the number of mistakes was much decreased.

It is planned to extend the function of the unified production system so that it can output production master data in DAISY and accessible PDF [8].

The system will be released on the website of our laboratory [9].