Modified balanced iterative reducing and clustering using hierarchies (m-BIRCH) for visual clustering

Abstract

In modern visual clustering applications where datasets are large and updates with new data may be ongoing, methods of online clustering are extremely important. Online clustering algorithms incrementally cluster the data points, use a fraction of the dataset memory, and update the clustering decisions when new data comes in. In this paper we adapt a classic online clustering algorithm called balanced iterative reducing and clustering using hierarchies (BIRCH) to incrementally cluster large datasets of features commonly used in visual clustering, e.g., 840 K color SIFT descriptors, 1.09 million color patches, 60 K outlier corrupted grayscale patches, and 700 K grayscale SIFT descriptors. We use the algorithm to cluster datasets consisting of non-convex clusters, e.g., Hopkins 155 3D motion segmentation dataset. We call the adapted version modified-BIRCH (m-BIRCH). BIRCH was originally developed by the database management community, but has not been used in computer vision. Modifications made in m-BIRCH enable data-driven parameter selection and effectively handle varying density regions in the feature space. Data-driven parameter selection automatically controls the level of coarseness of the data summarization. Effective handling of varying density regions is necessary to well represent the different density regions in the data summarization. Our implementation of the algorithm provides a useful clustering tool and is made publicly available.

Appendices

Insertion

Suppose we want to insert a new clustering-feature, ${\mathbf {CF}}_{\text{in}}$, in the CF-tree. In any node, ${\text{node}}_{i}$, of the tree the sum of all the clustering-features, ${\mathbf {CF}}_{\text{node}_{i}}=\sum _{j}{\mathbf {CF}}_{j}$, represents the set $\bigcup _{j}S_{j}$, where $S_{j}$ is the set of points represented by the clustering-feature ${\mathbf {CF}}_{j}$. ${\mathbf {CF}}_{\text{node}_{i}}$ is stored in the parent node. We call ${\text{node}}_{i}$ the child of ${\mathbf {CF}}_{\text{node}_{i}}$. The parent node is linked to the child node of each clustering-feature stored in it.

In order to insert ${\mathbf {CF}}_{\text{in}}$ we find the closest leaf node by traversing the tree starting at the root node; at each level we proceed to the child node of the closest clustering-feature. We step through the leaf node to check if ${\mathbf {CF}}_{\text{in}}$ can be absorbed in any clustering-feature without violating the threshold condition. ${\mathbf {CF}}_{\text{in}}$ can be absorbed in a leaf clustering-feature only if the size of the resulting clustering-feature is less than threshold T. If ${\mathbf {CF}}_{\text{in}}$ cannot be absorbed, we check whether the leaf has less than $B$ clustering-features. If the leaf node has less than $B$ clustering-features, then ${\mathbf {CF}}_{\text{in}}$ can be accommodated and stored in the leaf node. In the event of a successful absorption or accommodation, ${\mathbf {CF}}_{\text{in}}$ is added to each clustering-feature which was determined to be the closest when the tree was traversed from the root to the leaf node.

Figure 13 illustrates the tree insertion process, when ${\mathbf {CF}}_{\text{in}}$ is successfully absorbed in a leaf clustering-feature. Figure 13a shows the CF-tree before ${\mathbf {CF}}_{\text{in}}$ is inserted. The dotted arrows show the path taken when the tree was traversed from root to the leaf node. Figure 13b shows CF-tree after ${\mathbf {CF}}_{\text{in}}$ is inserted.

Fig. 13

a, b CF-tree before and after ${\mathbf {CF}}_{\text{in}}$ is inserted. ${\mathbf {CF}}_{\text{in}}$ gets absorbed by ${\mathbf {CF}}_{8}$. The gray rectangles denote sub-trees, which are not encountered while inserting ${\mathbf {CF}}_{\text{in}}$. The dotted arrows in (a) indicate the path taken during insertion of ${\mathbf {CF}}_{\text{in}}$ to reach the leaf node

Successful absorption of ${\mathbf {CF}}_{\text{in}}$ does not result in increase of memory used by the CF-tree. Successful accommodation of ${\mathbf {CF}}_{\text{in}}$ does result in an increase of memory used by the CF-tree. Node split occurs when ${\mathbf {CF}}_{\text{in}}$ can neither be absorbed nor accommodated in the leaf node. Node split results in the generation of two new nodes, ${\text{node}}A$ and ${\text{node}}B$. Clustering-features of the original leaf node and ${\mathbf {CF}}_{\text{in}}$ are distributed between the two new nodes. The furthest pair of clustering-features are found; one clustering-feature of the pair is stored in ${\text{node}}A$ and the other in ${\text{node}}B$. Remaining clustering-features are distributed depending on the distance between the clustering-feature and the current clustering-feature representation of ${\text{node}}A$ and ${\text{node}}B$. Figure 14 illustrates the distribution of clustering-features when a node split takes place.

Fig. 14

a Original leaf node. b, c ${\text{node}}A$ and ${\text{node}}B$ generated after the split, with clustering-features distributed between them

Figure 14a shows the original tree node, and Fig. 14b, c show ${\text{node}}A$ and ${\text{node}}B$ generated after the split. The $NULL$ in Fig. 14b and c indicates no clustering-feature is present. Suppose ${\mathbf {CF}}_{1}$ and ${\mathbf {CF}}_{2}$ are the furthest pair; thus ${\mathbf {CF}}_{1}$ is stored in ${\text{node}}A$ and ${\mathbf {CF}}_{2}$ is stored in ${\text{node}}B$. ${\mathbf {CF}}_{3}$ is closer to ${\mathbf {CF}}_{1}$ than to ${\mathbf {CF}}_{2}$; thus ${\mathbf {CF}}_{3}$ is stored in ${\text{node}}A$. ${\mathbf {CF}}_{\text{in}}$ is closer to ${\mathbf {CF}}_{2}$ than to ${\mathbf {CF}}_{1}+{\mathbf {CF}}_{3}$; thus ${\mathbf {CF}}_{1}$ is stored in ${\text{node}}B$. Once the clustering-features are distributed ${\text{node}}A$ replaces the original leaf node in the CF-tree, and the information along the path traversed to reach the leaf node is updated. Let ${\mathbf {CF}}_{\text{node}B}$ be the sum of all the clustering-features in ${\text{node}}B$. We attempt to accommodate ${\mathbf {CF}}_{\text{node}B}$ in the parent of the original leaf node. If ${\mathbf {CF}}_{\text{node}B}$ is successfully accommodated, then parent node is linked to ${\text{node}}B$, and information in the CF-tree is updated. If ${\mathbf {CF}}_{\text{node}B}$ cannot be accommodated in the parent node, then the parent node itself is split and the procedure is repeated. If the split propagates to the root and the root is split, then the tree height increases by one.

Rebuild

Suppose in the current tree, the leaf nodes are numbered from left to right starting with one, and the $i{\text{th}}$ leaf node in the original tree is denoted as ${\text{node}}_{i}$. Clustering-features of ${\text{node}}_{1}$ are inserted in the new tree by creating a path from root to leaf node identical to the path in the current tree. The first clustering-feature of ${\text{node}}_{1}$ is stored in exactly the same location as in the original tree. Remaining clustering-features in ${\text{node}}_{1}$ are stored in the leaf node of the new tree containing the first clustering-feature of ${\text{node}}_{1}$; at this point the new tree has only one leaf node. We step through the leaf node to check if the clustering-feature can be absorbed in any of the clustering-features currently stored, without the size of the resulting clustering-feature becoming greater than $T+ \Delta T$. If the absorption is unsuccessful, the clustering-feature is accommodated in the leaf node of the new tree.

Nodes in the current tree are deleted, when they are no longer required for the insertion of the remaining clustering-features. During the rebuild procedure at any point there are at most only $h$ extra nodes compared to the total number of nodes in the current tree, where $h$ is the height of the current tree.

Figure 15 illustrates the tree rebuild procedure.

Fig. 15

a Current CF-tree at the beginning of the rebuild procedure. b Insertion of the first clustering-feature of ${\text{node}}_{1}$. c Insertion of remaining clustering-features of ${\text{node}}_{1}$. d–f Insertion of clustering-features of ${\text{node}}_{2}$

Figure 15a shows the current CF-tree, at the beginning of the rebuild procedure. Figure 15b shows the insertion for the first clustering-feature of ${\text{node}}_{1}$ in the new tree and Fig. 15c shows the insertion of remaining clustering-features of ${\text{node}}_{1}$. Figure 15d–f show the insertion of the clustering-features of ${\text{node}}_{2}$. We find the closest leaf node to ${\mathbf {CF}}_{4}$ by traversing the tree shown in Fig. 15c. Figure 15d shows that ${\mathbf {CF}}_{4}$ could be absorbed by a clustering-feature in the closest leaf node. We find the closest leaf node to ${\mathbf {CF}}_{5}$ by traversing the tree shown in Fig. 15d. ${\mathbf {CF}}_{5}$ could not be absorbed in any clustering-feature stored in the closest leaf node. We accommodated ${\mathbf {CF}}_{5}$ in a new leaf node as shown in Fig. 15e; note that the path of the new leaf node is exactly same as the path of ${\text{node}}_{2}$ in the original tree. Figure 15f shows that the closest leaf node to ${\mathbf {CF}}_{6}$ in the new tree is the second leaf node. ${\mathbf {CF}}_{6}$ was successfully absorbed by a clustering-feature in the closest leaf node. Remaining clustering-features are inserted in a similar way.