Distorted High-Dimensional Binary Patterns Search by Scalar Neural Network Tree

Abstract

The paper offers an algorithm (SNN-tree) that extends the binary tree search algorithm so that it can deal with distorted input vectors. Perceptrons are the tree nodes. The algorithm features an iterative solution search and stopping criterion. Unlike the SNN-tree algorithm, popular methods (LSH, k-d tree, BBF-tree, spill-tree) stop working as the dimensionality of the space grows (N > 1000). In this paper we managed to obtain an estimate of the upper bound on the error probability for SNN-tree algorithm. The proposed algorithm works much faster than exhaustive search (26 times faster at N = 10000).

Appendices

Appendix A

It is necessary to calculate the following probability:

$$ P^{*} = 1 - \Pr \left[ {\bigcap\limits_{m = 1}^{M - 1} {\left| {{\mathbf{XX}}_{m}^{T} } \right| < (1 - 2b_{\hbox{max} } )N} } \right]. $$

(A1)

Let scalar products ${\mathbf{XX}}_{m}^{T}$ and ${\mathbf{XX}}_{\mu }^{T}$ be independent random quantities, $m \ne \mu$.

$$ P^{*} = 1 - \prod\limits_{m = 1}^{M - 1} {\Pr \left[ {\left| {{\mathbf{XX}}_{m}^{T} } \right| < (1 - 2b_{\hbox{max} } )N} \right]} . $$

(A2)

Now, it is necessary to calculate the probability that the product of each pattern by input vector is smaller than the threshold.

Scalar product ${\mathbf{XX}}_{m}^{T}$ is a discrete quantity, which values lie in $[ - N;N]$. Let k be the number of components with the opposite sign in vectors ${\mathbf{X}}$ and ${\mathbf{X}}_{m}$. Then its probability function is:

$$ \Pr \left[ {{\mathbf{XX}}_{m}^{T} = (N - 2k)} \right] = \frac{{C_{N}^{k} }}{{2^{N} }}. $$

(A3)

Random variable ${\mathbf{XX}}_{m}^{T}$ is symmetrically distributed with zero mean, so

$$ \Pr \left[ {\left| {{\mathbf{XX}}_{m}^{T} } \right| < (1 - 2b_\text{{max}} )N} \right] = 1 - 2\sum\limits_{k = 0}^{{b_\text{{max}} N}} {\frac{{C_{N}^{k} }}{{2^{N} }}} . $$

(A4)

From A2 and A4 we can conclude that

$$ P^{*} = 1 - \left\{ {1 - 2\sum\limits_{k = 0}^{{b_\text{{max}} N}} {\frac{{C_{N}^{k} }}{{2^{N} }}} } \right\}^{M - 1} . $$

(A5)

Appendix B

Scalar product

$$ \xi = {\mathbf{XX}}_{m}^{T} = \sum\limits_{i = 1}^{N} {x_{i} x_{mi} } $$

(B1)

consists of a large number of random quantities. Therefore, at big dimensions (N > 100) its distribution can be approximated by Gaussian law with the following probability moments:

$$ \bar{\xi } = 0\quad \text{b}\quad \sigma^{2} \left( \xi \right) = N. $$

(B2)

Therefore, probability (A1) can be described by integral expression:

$$ P^{*} \sim 1 - \left\{ {1 - \frac{2}{{\sqrt {2\pi N} }}\int\limits_{ - \infty }^{{ - (1 - 2b_{\hbox{max} } )N}} {e^{{ - \frac{{\xi^{2} }}{2N}}} d\xi } } \right\}^{M - 1} . $$

(B3)

Using the following approximation

$$ \int\limits_{x}^{\infty } {e^{{ - t^{2} }} dt} \approx \frac{{e^{{ - x^{2} }} }}{2x},x \gg 1, $$

(B4)

obtain the final estimation of probability (A1.1):

$$ P^{*} < \frac{2M}{{\sqrt {2\pi \tilde{N}} }}\exp \left( { - \frac{{\tilde{N}}}{2}} \right),\quad \tilde{N} = N(1 - 2b_{\hbox{max} } )^{2} . $$

(B5)