Audio formatting—Making spoken text and math comprehensible

Abstract

A_ST_ER is an interactive computing system foraudio formatting electronic documents (presently, documents written in (LA)T_EX) to produce audio documents. A_ST_ER can speak both literary texts and highly technical documents that contain complex mathematics. In fact, the effective speaking of mathematics is a key goal of A_ST_ER. To this end, a listener can request that segments of text or mathematics be spoken using several different rendering styles, in an interactive fashion. Listeners can themselves construct rendering rules and styles, if they feel it necessary.

In this paper, we describe the rendering component of A_ST_ER—the system for writing rules for speaking various parts of text and mathematics—and discuss some of the principles that were used in developing rules for making spoken text, mathematics, and tables comprehensible.

Visual communication is characterized by the eye's ability to actively access parts of a two-dimensional display. The reader is active, while the display is passive. This active-passive role is reversed by the temporal nature of oral communication: information flows actively past a passive listener. This prohibits multiple views—it is impossible to first obtain a high-level view and then “look” at details. These shortcomings become severe when presenting complex mathematics orally.

Audio formatting, which renders information structure in a manner attuned to an auditory display, overcomes these problems. Audio layout, composed of fleeting and persistent cues, conveys complex structure without detracting from the content. A_ST_ER is interactive, and the ability to browse information structure and obtain multiple views enablesactive listening.