Fractal image perception provides novel insights into hierarchical cognition

Highlights

• In this study we investigated the neural processing of fractals.

• Visual hierarchies are efficiently represented using self-embedding rules.

• Rule-based hierarchical processing recruits the visual dorsal stream.

• Self-embedding rules additionally activate the parieto-medial temporal pathway.

• This pathway integrates objects in contexts, and integrates “space” and “meaning”.

Abstract

Hierarchical structures play a central role in many aspects of human cognition, prominently including both language and music. In this study we addressed hierarchy in the visual domain, using a novel paradigm based on fractal images. Fractals are self-similar patterns generated by repeating the same simple rule at multiple hierarchical levels. Our hypothesis was that the brain uses different resources for processing hierarchies depending on whether it applies a “fractal” or a “non-fractal” cognitive strategy. We analyzed the neural circuits activated by these complex hierarchical patterns in an event-related fMRI study of 40 healthy subjects.

Brain activation was compared across three different tasks: a similarity task, and two hierarchical tasks in which subjects were asked to recognize the repetition of a rule operating transformations either within an existing hierarchical level, or generating new hierarchical levels. Similar hierarchical images were generated by both rules and target images were identical.

We found that when processing visual hierarchies, engagement in both hierarchical tasks activated the visual dorsal stream (occipito-parietal cortex, intraparietal sulcus and dorsolateral prefrontal cortex). In addition, the level-generating task specifically activated circuits related to the integration of spatial and categorical information, and with the integration of items in contexts (posterior cingulate cortex, retrosplenial cortex, and medial, ventral and anterior regions of temporal cortex). These findings provide interesting new clues about the cognitive mechanisms involved in the generation of new hierarchical levels as required for fractals.

Introduction

The ability to represent and generate complex hierarchical structures is one of the hallmarks of human cognition. In many domains, including language, music, problem solving, action-sequencing and spatial navigation, humans organize basic elements into higher-order groupings and structures (Badre, 2008, Chomsky, 1957, Hauser et al., 2002, Nardini et al., 2008, Unterrainer and Owen, 2006, Wohlschlager et al., 2003). This ability to encode the relationship between items (words, people, etc.) and the broader structures in which these items are embedded (sentences, corporations, etc) affords flexibility to human behavior. For example, in action sequencing, humans are able to change, add or adapt certain basic movements to particular contexts, while keeping the overall structure (and goals) of canonical motor procedures intact (Wohlschlager et al., 2003). Typical examples of these actions-in-context are ‘grinding the beans’ or ‘re-filling the water container’ in the process of making coffee (Jackendoff, 2002). Individuals can evaluate the need for these actions and omit them if they are unnecessary without impairing the overall procedure of making coffee (Badre and D'Esposito, 2009). This ability is different from simple action sequencing, and seems very limited in non-human animals (Conway and Christiansen, 2001).

A promising method to represent complex hierarchical structures – realized in nature and attractive for experimental research – is the use of recursive embedding processes (Fitch, 2010, Martins, 2012). Recursive embedding refers to the incorporation of a structure inside another structure of the same sort, and it allows the generation of hierarchies with infinite depth using very simple rules. We can add several new elements to a certain hierarchical level using within-level transformation rules (Fig. 1A), but it is only possible to generate multiple hierarchical levels with a single rule if this rule involves recursive embedding (Fig. 1B). When used in association with other rule-based processes, recursive embedding allows the generation of hierarchies that are deep, structurally rich and perceived as attractive. Some examples are the fractal Mandelbrot images or fractal structures in nature such as tree branches, algae, the flower of the Brassica oleracea, snail shells and coastlines. These structures can be extended or sub-divided indefinitely while visual and structural similarity is retained at all scales. These kinds of structures contrast with others with simpler modes of organization such as grass or crop fields, which like bead necklaces, are formed by adding several items to a group at fixed hierarchical levels.

Here we investigate the ability to recognize well-formed visuo-spatial hierarchical structures, based on the application of rules that either operate transformations within a hierarchical level, or rules which generate new self-similar hierarchical levels (Fig. 1). For simplicity, we simply use the expression ‘recursive’ or ‘recursion’ to refer to ‘recursive embedding’.

The processing of visuo-spatial stimuli is often described as occurring in parallel in two different systems – the ventral stream and the dorsal stream (de Haan and Cowey, 2011, Kravitz et al., 2011). The ventral stream, an occipito-temporal network, seems to process object quality or semantic information, with more abstract categories represented in more anterior portions of the temporal lobe (Kravitz et al., 2013). The dorsal stream, an occipito-parietal network, has classically been described as processing spatial information only. Recently, however, this classical view of the dorsal stream has been updated (Kravitz et al., 2011). While projections from the parietal cortex to the prefrontal cortex seem to be important for spatial working memory and visually-guided action, a third system, called the parietal-medial temporal pathway (PMT) appears to be necessary to integrate spatial and semantic information (Kravitz et al., 2011). The PMT pathway connects the dorsal stream with the medial temporal cortex (hippocampus and parahippocampus), through the posterior cingulate (PCC) and retrosplenial cortices (RSC) (Kravitz et al., 2011, Margulies et al., 2009). This pathway appears to be crucial for the retrieval of landmark information during spatial navigation and for the integration of objects in contextual frames (e.g. a mug in a date in a coffee shop) (Aguirre and D'Esposito, 1999, Buzsáki and Moser, 2013, Ino et al., 2007, Ranganath and Ritchey, 2012, Sato et al., 2010) (Fig. 2). We therefore hypothesize that the PMT may play a specific role in the representation of principles that allow the recognition and generation of well-formed hierarchical embeddings in the visuo-spatial domain.

Based on the principles depicted in Fig. 1, we developed two tasks to investigate the cognitive processes involved in the representation of visuo-spatial hierarchies: The Visual Recursion Task (VRT) and the Embedded Iteration Task (EIT). In both tasks participants are exposed to generative processes for a certain number of iterative steps and then asked to make inferences about further iterations. This means that in both tasks participants are asked to extract simple rules from the first iterations which can then be applied to predict further transformations. In VRT, each iterative step generates a new hierarchical level according to one particular spatial rule isomorphic to the rule displayed in previous levels of the hierarchy. The brain requires only one simple rule to be able to generate large self-similar structures (fractals) with an unlimited number of levels. In EIT, new elements are embedded iteratively within a fixed hierarchical level, according to a spatial rule but without generating new levels. It is important to clarify that both tasks are iterative (i.e. a certain rule is applied a given number of times) and both may generate hierarchies of similar complexity (see Fig. 1, Fig. 3).

Our previous research with these tasks (Martins, 2012, Martins and Fitch, 2012) suggests that, in comparison with EIT, performance in VRT is more strongly associated with abstract reasoning and less correlated with specific visuo-spatial cognitive abilities. In the current study, we investigated the neural bases involved in the representation of visuo-spatial hierarchies by comparing the brain circuits active during VRT and EIT. As a control task we introduced a ‘similarity task’ (Positional Similarity Visual Task — PSVT), in which participants were asked to match a target visuo-spatial hierarchy with a set of alternatives. The setup and images displayed were closely matched for all three tasks. As indicated above, our primary hypothesis was that the brain uses different resources for processing identical hierarchical structures depending on whether it applies a “fractal” or a “non-fractal” cognitive strategy.