Taxonomical Associative Memory

Abstract

Assigning categories to objects allows the mind to code experience by concepts, thus easing the burden in perceptual, storage, and reasoning processes. Moreover, maximal efficiency of cognitive resources is attained with categories that best mirror the structure of the perceived world. In this work, we will explore how taxonomies could be represented in the brain, and their application in learning and recall. In a recent work, Sacramento and Wichert (in Neural Netw 24(2):143–147, 2011) proposed a hierarchical arrangement of compressed associative networks, improving retrieval time by allowing irrelevant neurons to be pruned early. We present an extension to this model where superordinate concepts are encoded in these compressed networks. Memory traces are stored in an uncompressed network, and each additional network codes for a taxonomical rank. Retrieval is progressive, presenting increasingly specific superordinate concepts. The semantic and technical aspects of the model are investigated in two studies: wine classification and random correlated data.

Appendix: Cluster Membership Heuristic

After the preprocessing step where elements are clustered, learning of item–category membership associations is performed at every taxonomical rank by an associative net. To determine the cluster C _p that contains a certain item x at a certain taxonomical rank, we could simply refer to our cluster-tree.

Alternatively, drawing inspiration from the fixed-points of the oscillator model presented in [14], we propose a heuristic to test whether an element belongs to a cluster which warrants no false negatives (which would impair accuracy). This approach reduces lookup costs in the computational setting and provides a hint of how the model could generalize learning while leveraging a symbolic preprocessing step.

We use a convenient short-hand representation of a cluster, the Boolean sum pattern ${\fancyscript{U}(C_p),}$ previously presented in eq. 20, that describes the feature-set containing every feature which is present in at least one element of C _p . Once again we turn to our simple fruit taxonomy to illustrate. Table 9 shows these feature-sets ${\fancyscript{U}(C_p)}$ for each non-single-element cluster. For a single-element cluster, this feature-set is given by that very element.

Table 9 Fruit clusters: all features

Table 10 Table presenting item–category associations stored at every auxiliary network for the fruit dataset

These feature-sets are analogous to the extension of the Boolean OR aggregates in the prime model. Note, however, that unlike the prime model, the content space is not partitioned, that is, the divisions are possibly overlapping. Thus, in our model, two contrasting feature-sets (at the same taxonomical level) may have a nonzero intersection. For instance, in our illustrative taxonomy of fruits (refer to Table 9), we have $C_2 \; \cap \; C_3 =$ {sweet, round}.

Given the requirement that we produce no false negatives when pruning from one network to the next, we defined our pattern versus cluster matching heuristic to require a single shared feature between a pattern x and a cluster description ${\fancyscript{U}(C_p)}$ to associate a pattern with said cluster C _p .

We exploit the binary structure of the taxonomical tree, testing always cluster pairs. Alike a binary search procedure, when we are checking at which clusters an element x ^μ belongs, if we have determined that at a given level ${\bf x}^{\mu} \in C_a \wedge {\bf x}^{\mu} \notin C_b,$ we may at deeper levels disregard the descendants of C _b .

This approach carries a trade-off: it produces false positives. Consider, for instance, the definition of apple in our fruit taxonomy (refer to Table 1 and the feature-sets of C ₂ and C ₃ in Table 9). According to our heuristic, we cannot determine which of these clusters contain the pattern (both produce feature matches); however, at deeper levels, this uncertainty of “taxonomical location” decreases.

For an illustration of the side-effects of this method, consider Table 10 depicting item–category associations for the fruit taxonomy where cluster codes are approximated with the heuristic here described. Given the compact and dense feature space of this dataset, and the fact it produces a taxonomy that is not very deep, overlaps in feature-unions are plenty and reduction in uncertainty is minimal.