The reader as subjective entropy: a novel analysis of multimodal readability

The purpose of this study was to explore the viability of transinformation analysis as a multimodal readability metric. A novel approach was called for, considering that existing and established readability metrics are strictly used to measure linguistic complexity. Yet, the corpus of multimodal literature continues to grow, along with the need to understand how non-linguistic modalities contribute to the complexity of the reading experience.

In this exploratory study, think aloud screen recordings of eighth-grade readers of the born-digital novel Inanimate Alice were analyzed for complexity, along with transcripts of post-oral retellings. Pixel-level entropy analysis served as both an objective measure of the document and a subjective measure of the amount of reader information attention. Post-oral retelling entropy was calculated at the unit level of the word, serving as an indication of complexity in recall.

Findings confirmed that transinformation analysis is a viable multimodal readability metric. Inanimate Alice is an objectively complex document, creating a subjectively complex reading experience for the participants. Readers largely attended to the linguistic mode of the story, effectively reducing the amount of information they processed. This was also evident in the brevity and below average complexity of their post-oral retellings, which relied on recall of the linguistic mode. There were no significant group differences among the readers.

This is the first study that uses entropy to analyze multimodal readability.