Resampling in an indefinite database to approximate functional dependencies

Abstract

Functional Dependency satisfaction, where the value of one attribute uniquely determines another, may be approximated by Numerical Dependencies (NDs), wherein an attribute set determines at most k attribute sets. Hence, we use NDs to “mine” a relation to see how well a given FD set is approximated. We motivate NDs by examining their use with indefinite information in relations. The family of all possible ND sets which may approximate an FD set forms a complete lattice. Using this, a proximity metric is presented and used to assess the distance of each resulting ND set to a given FD set.

Searching for a definite relation extracted from an indefinite relation which satisfies a given set of FDs, known as the consistency problem, has been shown to be NP-complete. We propose a novel application of the bootstrap, a computer intensive resampling technique, to determine a suitable number of definite relations upon which to apply a heuristic based hill-climbing algorithm which attempts to minimise the distance between the best ND set and the given FD set. The novelty is that we repeatedly apply the bootstrap to an indefinite relation with an increasing sample size until an approximate fixpoint is reached at which point we assume that the sample size is then representative of the indefinite relation. We compare the bootstrap with its predecessor, the jackknife, and conclude that both are applicable with the bootstrap providing additional flexibility. This work highlights the utility of computer intensive resampling within a dependency data mining context.