Data Mining and Scientific Knowledge: Some Cautions for Scholarly Researchers

Abstract

Recent years have seen the emergence of data analytic techniques requiring for their practical use previously unimaginable raw computational power. Such techniques include neural network analysis, genetic algorithms, classification and regression trees, v-fold cross-validation clustering and suchlike. Many of these methods are what could be called ‘learning’ algorithms, which can be used for prediction, classification, association, and clustering of data based on previously estimated features of a data set. In other words, they are ‘trained’ on a data set with both predictors and target variables, and the model estimated is then used on future data which does not contain measured values of the target variable. Or in clustering methods, an iterative algorithm looks to generate clusters which are as homogenous within and as heterogeneous between as possible.