Keywords

1 Introduction

Sophie van der Linden [1] proposes categories for the analysis of picture book design, considering that narrative is built textually and visually, through the relationship between text and image. Although picture books and children’s appbooks are different artifacts, both convey children’s literature. Therefore, it is appropriate for an investigation into the children’s appbook to incorporate Linden’s reflection.

The nature of interactive media, however, leads us to some transformations and reframing of graphic media categories. For example: (a) text can be animated; (b) if the film language is suggested in the printed book by means of successive images in a sequence of pages, thanks to video it gets an empirical presence in interactive media etc.

But we think that digital visual narrative analysis can be more effective if we use also the categories proposed by Janet Murray [2]: immersion, transformation and agency. The author highlights the importance of user-reader actions and how the aesthetic experience achieves a different intensity in the new media environment.

Features, such as hypertextuality and the use of game procedures, bring new possibilities to the narrative. In both hypertext and game, users move in a virtual space, where navigation becomes essential for reliable and fluid storytelling experiences.

Based on these theoretical references, and aiming to investigate the role of navigation in children’s appbook interactive visual narrative, we selected four tablet appbooks that show different characteristics and have been recognized for the quality of their realization (great impact in social media, prizes at book fairs and specialized conferences on children’s ebooks). (1) The Very Cranky Bear [3], winner of Parent’s Choice Awards 2013 (Silver Honor Awards) is more traditional, close to its printed version; (2) The Monster’s Socks [4], Bologna Ragazzi Digital Award 2013 finalist for children’s fiction, that has no printed version, incorporates some game features; (3) Petting Zoo [5], winner of Parent’s Choice Awards 2014 (Golden Awards), is a book of animated images without text; (4) the classic Jack and the Beanstalk [6], honor mention award in Bologna Ragazzi Digital Award 2014, presents a game structure. The analysis aim to contribute to the understanding of design issues present in the production of appbooks for children. It has an exploratory nature, in order to support researches to come.

2 Narratives

Considering that a literary piece results from the writer’s work, the narrative “is the exposure of a text capable of evoking a world given as real or imagined through linked facts in time and determined space” [7]. In this context, the book design can bring comfortable reading and/or connotations due to typographic choices (e.g., a neutral font, or a “fancy” font etc.), but it may not modify the structure of the story. Picture books, however, call for a redefinition of traditional narrative. The participation of images on a visual narrative ranges from different degrees of visual emphasis to a totally visually structured storytelling. Although the term visual narrative could also be used for films or comics, we are dealing here with visual storytelling in books.

According to Sherline Pimenta [8], visual narrative is an essentially explicit way to narrate through images. The story displays a series of events connected by causality, temporality or order of occurrence; as narrative means the act of telling a story, or the story itself, or its order of presentation. Pimenta lists five characteristics of visual narratives, based on verbal narrative features: (1) the presence of a story is the most essential feature of the visual narrative; (2) the visual is constructed with the objective of communicating a story to the onlooker; (3) there is a presence of actors (participants), and an actor is a character in the story who performs an action; (4) the visual narrative has an ‘universe’ of its own, and the participants exist in this universe, that mimics the real or imagined world; (5) the visual narrative can be expressed on any medium, e.g. paper, stone, electronic device etc. [8].

Pimenta describes the visual narrative in digital media as an interactive visual narrative, characterized by three conditions: “(1) that it is essentially visual in nature; (2) has the narrative aspect to it; and (3) involves interaction from the viewer” [8]. The interactive visual narrative begins with the invention of virtual spaces and navigation systems to access them. The interaction can range from linear navigation to total immersion in the narrative mediated by characters and/or situations. The user-reader’s interaction and choices are directly linked to the development of the story and to the perception of its movement.

To Murray one of the essential characteristics of digital environments is the simulated spatiality, in which users can move by their own. Rather than creating imaginary spaces from reading, or watching situations assembled from the clipping of scenes, like in the movies, the computer allows the users to transit through spaces with some control over their footsteps. In these terms, the navigation can be thought of as part of the narrative structure.

In this research, we focus on children’s appbooks as interactive visual narrative, where the images match or exceed the text.

3 Categories Related to Structural Relations Between Text and Image According to Linden

Sophie van der Linden discusses the relations between text and image within the picture book in the chapter “Pages and book spaces” [1], from which we highlight the following structural features: types of diagramming; framing, field and extra-field; montage and junction.

3.1 Types of Diagramming

Dissociation. Dissociative diagramming is characterized by the alternation between text pages and image pages. In printed books the maximum separation between text and image is due to the book fold, which leads the reader to switch between viewing the image and reading the text. In the appbook, usually there is no such differentiation between pages. The most logical is the simple page display, but there are also cases in which the double-page spread can be also displayed as a flowable layout.

When the appbook reproduces the double-page spread, showing the fold mark and mimicking printed page flipping, the dissociative layout happens exactly as on printed books. However, beyond this kind of solution, an appbook with a dissociative layout, for instance, provides the possibility of enabling and disabling text by touching the screen, decreasing the reading rate and letting the users “read” the illustrations alone.

Association. Association diagramming combines at least one verbal statement to one visual statement on the same page. Text and images can be arranged in several ways on a single page or on a double-page spread. These different structures will have different implications, depending on the narrative intention. In the appbook, the associative diagramming can occur statically, as on a printed book; or it can happen through the fade-out feature (text color gradual darkening) of the image during screen transition.

Conjunction. Text and images are organized in an overall composition. It differs from associative diagramming because it presents several statements that show more contiguousness than continuity. Text and images are integrated, while in associative diagramming verbal and visual messages tend to be perceived separately.

3.2 Framing, Field and Extra-Field

The framing – from top to bottom (plongée), from bottom to top (contraplongée) or an oblique framing (that transmits a sense of instability), for instance – entails semantic effects to the storytelling. Besides that, the framing of a character (or any other element) tends to suggest an extra-field: framing in a picture book scene determine what is inside or outside of our field of vision. And the reader can infer the existence of another space beyond the frame boundaries.

These notions have arisen with the film, but can also be conveyed by the illustration. In the printed book these viewpoints are statically represented, and are developed page by page, whereas the appbook can actually simulate the camera movement.

3.3 Montage

The picture book can use the composition in double-page spread, referring to the film montage and making the page sequence more fluid. “If montage, in cinema, is the sequence of plans, in the picture book it is to organize the succession of double-page spreads” [1]. When transposed to an appbook, the double-page spread suffers adjustments: reductions, cuts, strains, reducing of the body font, etc. However, the camera motion feature may supply the function of the double-page spread in picture books, keeping the attention by means of a sequence plan.

3.4 Junction

“It is a sequence of articulated images connected by one or more links whether iconic, semantic, plastic, logical or only technical” [1]. The organization of narrative sequences has the function of joining, and these articulations may forecast fundamental changes in margins, vignettes and more. There is always an element that fulfills the role of making a transition.

4 Interactive Visual Media: Navigation and Orientation

Adapting Linden’s categories to the digital media means considering how multimedia objects, animation and interactivity features influence the design relation between text and image. Although it is not a concern for Linden’s research, it is our intent in this paper to discuss the navigation for the interactive visual media and its effects on narrative.

In hypermedia devices, navigation can be understood as a metaphor of the act of moving in space [9]. The printed page and the screen space are both two-dimensional planes. In the former, the sequence of different physical pages, bound by the book spine, establishes a physicality for the time dimension. In the latter, the narrative temporality is developed through the updating of a succession of images in the same visual plane.

According to Rosenfeld and Morville [10], we create navigation tools to avoid getting lost, either moving on land, on sea or in hypertext network. The authors summarize the utility of navigation tools on three points: “to define a path to follow, determine the position in space and find the way back” [10]. These three points are related to the notion of orientation: to know where you are, to know how to go ahead, and to know how to return to the beginning.

Tidwell [11] signs three “common-sense features” of navigation systems for web interfaces that we can use to understand digital narratives as well:

  1. 1.

    Good Signage: “Clear, unambiguous labels anticipate what you’re looking for and tell you where to go; signs are where you expect them to be, and you’re never left standing at a decision point without guidance”.

  2. 2.

    Environmental clues: Culturally determined conventions integrated on interface, e.g., you would look for an “X” close button at the top right of a modal dialog. In real life could be a rock in a trail that signs the way back.

  3. 3.

    Maps: Gives a whole picture of space, summarizing everything you need to know to move without feeling lost, e.g. roadmap, sitemap, etc.

5 Immersion, Agency and Transformation

Involved with cyberliterature projects at MIT, Janet Murray [2] seeks to expand the notion of narrative under the influence of computer, presenting the possible experiences that stories can provide the reader. The author focuses on the fact that, in addition to the simulation of other media, the digital medium is developed as a hypertext tissue, in which the user involvement is crucial to the course of the narrative.

5.1 Immersion

Immersion is one of three aesthetic categories, which the author calls the three narrative dimensions of pleasure. “Immersion is a metaphorical term derived from the physical experience of being submerged in water.” The expected sensation of “being surrounded by a completely other reality, as different as water is from air, that takes over all of our attention, our whole perceptual apparatus.” In “a participatory medium, immersion implies learning to swim, to do the things that the new environment makes possible” [2]. Learning to swim is learning to interact in the digital environment. Murray points out that the challenge of narrative immersion is to avoid breaking the enchantment of the story. That keeps users delighted by both stimulating narrative plot and establishing engagement with the story.

5.2 Agency

Agency is the satisfaction of taking meaningful actions and the perceivable results of decisions and choices displayed on interface. The sense of agency responsiveness is fundamental in the project of navigation tools. When the controls do not work or do not match with the indicated directions, they awake in users the feeling of being lost. The sense of agency is related to the user autonomy. The highest autonomy level, described by Murray as intentional navigation, appears in a game when the player denies the story to explore the virtual territory just for fun.

Murray describes two configurations of the digital space with its own narrative power and agency intensity: “the solvable maze and tangled rhizome” [2]. The maze always leads to the exit, even if on their way find countless doors to open. The rhizome is a nonlinear undefined and intertwined path, with many entrances and exits. There is no correct starting and ending point, as the path is created by the user’s moves.

5.3 Transformation

The third category, transformation, describes how the features of graphic interface offer users “countless ways of shape-shifting” [2]. The computer allows the storyteller to create stories with simultaneous realities, mosaics of multiple fragments and viewpoints, with not clearly determined closures. At the same time, it helps readers to be more than passive agents, allowing them to build their own ways on a narrative and making them feel as characters of the story.

6 Case Studies

6.1 The Very Cranky Bear

The Very Cranky Bear tells the story of three friends who, when getting shelter from the rain in a cave, find a very angry bear. The appbook was produced from the printed version, and maybe that’s the reason why its narrative is fairly linear. The appbook design keeps almost the same configuration of the physical book. One configuration in particular, the film language, is leveraged in the digital environment. We identified two of Linden’s categories related with film: (1) Framing, field and extra field (Fig. 1), performed by zoom and travelling effects simulated by framings and sequence of pages; (2) Montage (Fig. 2), in which camera zoom is activated by the user agency making the image sequence more fluid.

Fig. 1.
figure 1figure 1

Camera zoom (framing, field and extra-field)

Full size image
Fig. 2.
figure 2figure 2

Camera moving sequence (montage) and navigation controls

Full size image

Forest, rain and animals sounds are other features that emphasize film language, also suggesting that there is something beyond what is framed. They awaken in the reader the will to know what happens in this space that is not framed. Therefore, these film features work as an environmental clue for the navigation.

The camera view style inserts the reader in the story as a passive observer, who moves along the scenes. The navigation controls (Fig. 2) determine the pace and the direction (forwards or backwards) of the narrative. In these conditions agency doesn’t contribute very much to its development, as the autonomy of the user is restricted to few actions.

This appbook uses printed book language and uses, at the same time, multimedia to perform immersion through navigation. Although the appbook keeps diagramming static layout, the space environment is constructed with film shots, animated transitions and sound effects, substituting page flipping and double spread page.

6.2 Monster’s Socks

The book tells the story of a monster called Monster, who is looking for its socks. On its way, as it talks to other characters it can figure out where they are. This appbook displays two navigation modes: the linear regular course of the story through navigation controls; and hyperlinked thumbnails scenes, jumping directly to specific scenes (Fig. 3).

Fig. 3.
figure 3figure 3

Thumbnails navigation (top), navigation controls (bottom), Yellow circle text activator (centered bottom) (Color figure online).

Full size image

The arrow shaped buttons controls are placed in the lower corners, as in all other appbooks. But they work differently, acting as video game controls, in which users must keep the button pressed for moving the monster around the scene. This design stimulates handhold style of the joystick, keeping both hands always close to the buttons. When the monster crosses the entire scene, the buttons perform a new function, changing the scene.

The storytelling is directly related to the act of walking in space. The text turns visible only when Monster passes over yellow circles on the floor. The alternation between image and text changes the meaning of reading. When two paragraphs are alternated at the same scene, their meaning changes, making the story go on. Navigation leads to a change from dissociative to associative diagramming, unfolding the narrative. The user agency corresponds to the Monster movements, standing for printed book’s text and flipping pages.

6.3 Petting Zoo

The sense of going to the zoo is provided by the only text within the appbook, which is the title “Petting Zoo”. Besides a series of pets’ illustrations, there is no scenario representing a zoo. The narrative is not a sequence of related events, but a sequence of depicted animals. Despite the book structure in linear sequence of scenes, each scene could be understood as an independent narrative, and when put together they display the Zoo ride. The animated transitions establish the sense of contiguity between the scenes (Fig. 4). They can be understood from the perspective of Linden’s junction. Bellotti and Lessa describe the animated transition as “a fun metamorphosis of an animal into another, (…) a junction that becomes animated by using the features offered by the digital medium” [12].

Fig. 4.
figure 4figure 4

Scene transition

Full size image

The user-reader’s immersion happens by what Murray calls transformation, i.e. an imaginary part of the narrative not represented in the application. The more users imagine being inside the story, the more clearly they can interact in the space.

Another feature that enhance the feeling of immersion is the well-defined interaction model input, in contrast with the singular animated outputs for the interactive pets. The gesture inputs follow the same logic throughout the application: users swipe vertically and horizontally to “play” with the animals; to change a scene they must touch the lower right corner of the screen to go forwards, and the lower left corner to go backwards. There is an alternation between gestures repetition and the variety of animations, linking uniform conventions and surprising responses.

The narrative can be understood with the help of two navigation features. The first and most important is the linear displacement through the scenes, triggered by the navigation controls (arrow shaped buttons). The second, that is the summary (Fig. 5), features a mosaic of hyperlinks, allowing direct access to any scene. The orientation of the reader is provided by environmental clues, by animated transition between illustrations in response to the inputs gestures and by the background color shifting. The lack of scenario weakens the notion of a given place in time and space, encouraging the imaginary construction of the story. In a book where there is no text, the navigation features provide the means that take users for a zoo ride.

Fig. 5.
figure 5figure 5

Summary screen

Full size image

6.4 Jack and the Beanstalk

Based on a classic tale, this appbook presents a maze-like narrative, characterized by its multiple endings and by a sense of always going in search of the exit. The final location of the game – safely returning home – is always the same, but the actions and decisions taken along the giant’s castle determine the conditions in which the character ends the game. Differently from the traditional tale, in this application the narrative plot changes as it is performed by the user agency.

Of the four appbooks, this one presents the greater agency and, at the same time, more navigation features. The arrow shaped buttons on the bottom of the screen go forwards and backwards on the scenes, changing their color as soon as all tasks are accomplished (Fig. 6a). Jack’s movements are controlled by input gestures, according to the context of the scene. While he is climbing the beanstalk, the user swipes left or right to avoid obstacles on the way; or Jack walks along the castle halls when the user drags him with the finger.

Fig. 6.
figure 6figure 6

Navigation features (left to right image): a. navigation controls (bottom); b. blinking spot (on Jack’s shirt); c. conversation ballons.

Full size image

As the inputs depend on the scene, the orientation is given both by signage and by environmental clues. The former correspond to conversation balloons (Fig. 6b) indicating what Jack needs to do; the latter to the scenario depicted as a cartoon, providing affordance (e.g. the end of the corridor has stairs indicating another level up). In addition there is another kind of signage – that appears as a blinking spot (Fig. 6c) – if users can’t find out how to perform the task.

The summary (Fig. 7) is designed as a location map, so users can identify the chapters and all the tasks that must be completed for Jack’s happy end. The appbook also provides a map giving access to all parts, regardless the order of reading.

Fig. 7.
figure 7figure 7

Summary screen

Full size image

7 Considerations

In the four appbooks the main navigation controls are displayed as arrow shaped buttons placed on the bottom of the screen. The buttons are always visible, differently of textual ebooks, in which the navigation controls are hidden (flip gesture). At the same time, the controls of each appbook respond differently to inputs, according to design decisions, as could be specially seen on Monster’s Socks.

As a trend, the navigation structure changes from the page flipping metaphor to animated transitions and hipertext jumps. Even in an appboks close to the printed book, like The Very Cranky Bear, motion graphics marks scene transitions, reinforcing its film style.

On three of the appbooks the summary is composed of hyperlinked objects, despite their different designs due to the logic of each narrative. The summary of Jack and the Beanstalk is a map, showing the path of the maze-like world. On the contrary, on Petting Zoo the summary screen gathers all the pets as they appear in the scenes.

These books exemplify combinations of categories from different means, aiming to promote the development of interactive visual narrative and the user involvement with the story. Their analysis enabled us to understand how the compatibility of the traditional structure of the book, film language and new technological resources fit together to create the interactive visual narrative, guiding users in the story, through scenes, images and texts.

This approach to children’s appbooks, with the help of design and digital storytelling categories, shows that the designer of the interactive visual narrative must control both the story and the media features. As Janet Murray points out this “kind storyteller” ought to be half bard and half hacker.