Flying Together with Audio and Video: Enhancing Communication for the Hearing-Impaired Through an Emerging Closed Captioning Standard

Abstract

As the text-based visual representation of a program’s audio elements, Closed Captioning primarily serves as a technology to enhance communication for the hearing impaired. Since text is much simpler than audio and video, Closed Captioning is traditionally transmitted as supplementary or auxiliary information as part of the image or in an extended or private data field of an encoded video bitstream called video elementary stream, usually accompanied by one or more audio elementary streams. Since Closed Captioning is extremely important for the accessibility of the audio content of a program to the hearing impaired, we propose to encode the closed caption into a bitstream called caption elementary stream, which can fly together with audio and video elementary streams. In other words, closed caption can be stored and transmitted in a manner similar to how audio and video are handled. We have drafted a national standard for Closed Captioning in China, which is now in its final stage of approval and publication. In this paper, the main technical content of the emerging Closed Captioning standard will be introduced. Specifically, the encoding, storage, and transmission of Closed Captioning will be described. Moreover, the decoding and presentation of Closed Captioning under the two scenarios of on demand streaming and live streaming will also be designed and discussed. The AI technology of Speech-to-Text enables Closed Captioning to be implemented efficiently with the help of manual proofreading. Positively, the emergence of the Closed Captioning standard will enhance accessibility to audio-visual programs on both the broadcasting network and the Internet for the hearing-impaired in China and worldwide.