A combined nonparametric approach to feature selection and binary decision tree design

doi:10.1016/0031-3203(80)90029-1

Pattern Recognition

Volume 12, Issue 5, 1980, Pages 313-317

https://doi.org/10.1016/0031-3203(80)90029-1 Get rights and content

Abstract

An efficient procedure which integrates feature selection and binary decision tree construction is presented. The nonparametric approach is based on the Kolmogorov-Smirnov criterion which yields an optimal classification decision at each node. By combining the feature selection with the design of the classifier, only the most informative features are retained for classification.

References (13)

B. Chandrasekaran et al.
Quantization complexity and independent measurements
IEEE Trans. Comput.
(1974)
E.G. Henrichon et al.
A nonparametric partitioning procedure for pattern classification
IEEE Trans. Comput.
(1969)
H. Hauska et al.
The decision tree classifier: design and potential
L.V. Kanal
Some current concepts and problems in pattern classification and feature extraction
J.H. Friedman
A recursive partitioning decision rule for nonparametric classification
IEEE Trans. Comput.
(1977)
H.J. Eghbali
K-S test for detecting changes from Landsat imagery data
IEEE Trans. Systems, Man Cybernet.
(1979)

There are more references available in the full text version of this article.

Cited by (74)

Towards improving decision tree induction by combining split evaluation measures
2023, Knowledge-Based Systems
Explainability is essential for users to effectively understand, trust, and manage powerful artificial intelligence solutions. Decision trees are one of the pioneer explanaible artificial intelligence algorithms widely used by experts in several contexts. C4.5 is a prominent decision tree that continues to serve as the base for subsequent improvements. One of main pillars on the decision tree induction is the split evaluation measure used for assesing the candidate splits. Usually, one single split evaluation measure is used during the induction procedure. However, combining multiple split evaluation measures into the induction procedure could improve the classification results. Our proposal aims to obtain better classification results by selecting the best candidate split taking advantage of the power of voting methods to aggregate the ranking issued by multiple split evaluation measures. Consequently, we performed a set of experiments to appraise the performance of our proposal on 114 databases. From these experiments, two novel split evaluation measures, proposed in this paper, were assessed against 11 individual and prominent state-of-the-art split evaluation measures. Our classification results show that our proposal outperforms the other state-of-the-art split evaluation measures.
Method selection in short-term eruption forecasting
2021, Journal of Volcanology and Geothermal Research
Citation Excerpt :
A decision tree is a collection of conditions (internal nodes) from which branches split; the decision is any branch without a condition at the end. The aim of a decision tree is to have as few branches as possible while classifying the data as accurately as possible (Rounds, 1980). Different types of trees include, for example, classification trees, which have decisions based on categorical data (e.g., eruption / no eruption), and regression trees which carry continuous values (e.g., probability of eruption).
For accurate and timely information on the evolving state of our volcanoes we need reliable short-term forecasts. These forecasts directly impact crisis management from evacuations, exclusion zones, and when it is safe to return. Eruption forecasting should not be viewed as an academic exercise or a theoretical discussion in a back room, nor is now the time for dramatic data interpretations or a set of ‘we-told-you-so’ hindcasting demonstrations. To produce a short-term eruption forecast, a systematic evaluation of options is required with a critical assessment of outstanding issues and assumption validity. We run this lens over a set of existing short-term eruption forecasting methods and provide a straightforward data-driven methodology for forecast selection. Six eruption forecasting methods are discussed here: (1) Expert interpretation, (2) Event trees, (3) Belief networks, (4) Failure forecasting, (5) Process / Source models, and (6) Machine-learning algorithms with a view to forecasting: (1) Eruption occurrence (onset time), (2) Eruptive vent location(s), (3) Eruption size, (4) Initial eruption style/phase, (5) Eruption phase duration, and (6) Eruption specific hazards.
This work constitutes a decision tool that can be directly applied to a volcanic system of interest to determine which eruption forecasting methods are possible, plausible, and with what implementation steps. Accompanying this is an extensive evaluation of assumption validity (and assumption avoidance options) to ensure the accurate and transparent application of any eruption forecasting method. Significant potential is identified in methods that are generally data-hungry (e.g., belief networks and machine-learning algorithms), and/or by the coupling of probabilistic methods to process/source models. However, as most volcanic systems are data-poor, expert interpretation and event trees remain the only currently available forecasting methods that can be readily and widely applied during volcanic crises.
Multivariate statistical analysis to investigate the subduction zone parameters favoring the occurrence of giant megathrust earthquakes
2018, Tectonophysics
Citation Excerpt :
Due to the limited amount of available data, we performed only the learning phase aiming at identifying any recurrent pattern that discriminates segments that have experienced GEqs from those that have not. The analysis was based on two different PR non-parametric algorithms: the Binary Decision Tree BDT (Mulargia et al., 1992; Rounds, 1980) and the Fisher discriminant analysis FIS (e.g., Duda and Hart, 1973). Both algorithms have been previously used on synthetic data with known patterns to test their ability of recognizing recurrent schemes and extracting relevant features on small datasets, with non-normal and discrete/categorical parameters (e.g., Sandri and Marzocchi, 2004).
The observed maximum magnitude of subduction megathrust earthquakes is highly variable worldwide. One key question is which conditions, if any, favor the occurrence of giant earthquakes (M_w ≥ 8.5). Here we carry out a multivariate statistical study in order to investigate the factors affecting the maximum magnitude of subduction megathrust earthquakes. We find that the trench-parallel extent of subduction zones and the thickness of trench sediments provide the largest discriminating capability between subduction zones that have experienced giant earthquakes and those having significantly lower maximum magnitude. Monte Carlo simulations show that the observed spatial distribution of giant earthquakes cannot be explained by pure chance to a statistically significant level. We suggest that the combination of a long subduction zone with thick trench sediments likely promotes a great lateral rupture propagation, characteristic of almost all giant earthquakes.
Climate-driven endemic cholera is modulated by human mobility in a megacity
2017, Advances in Water Resources
Citation Excerpt :
Following the methodology in Reiner et al. (2012), the optimal outbreak probability threshold retained for prediction was found using the Kolmogorov–Smirnov test. The threshold corresponds to the point of maximum distance between the Cumulative Mass Functions of the predicted outbreak probability in months where an outbreak did and did not occur (Rounds, 1980). Given the identified optimal threshold, the confusion matrix of observed and predicted outbreak was used a basis for the computation of predictive performance, including accuracy and false positive and negative rates.
Although a differential sensitivity of cholera dynamics to climate variability has been reported in the spatially heterogeneous megacity of Dhaka, Bangladesh, the specific patterns of spread of the resulting risk within the city remain unclear. We build on an established probabilistic spatial model to investigate the importance and role of human mobility in modulating spatial cholera transmission. Mobility fluxes were inferred using a straightforward and generalizable methodology that relies on mapping population density based on a high resolution urban footprint product, and a parameter-free human mobility model. In accordance with previous findings, we highlight the higher sensitivity to the El Niño Southern Oscillation (ENSO) in the highly populated urban center than in the more rural periphery. More significantly, our results show that cholera risk is largely transmitted from the climate-sensitive core to the periphery of the city, with implications for the planning of control efforts. In addition, including human mobility improves the outbreak prediction performance of the model with an 11 month lead. The interplay between climatic and human mobility factors in cholera transmission is discussed from the perspective of the rapid growth of megacities across the developing world.
Optimization of spectral indices and long-term separability analysis for classification of cereal crops using multi-spectral RapidEye imagery
2016, International Journal of Applied Earth Observation and Geoinformation
Citation Excerpt :
Values of dKS ≃ 1 indicate high separability while identical spectra are characterized by dKS close to 0. This measure has been frequently applied in remote sensing for feature selection, class separability assessment and multi-temporal change detection applications or geometric accuracy assessment (Rounds, 1980; Möller et al., 2012; Tang et al., 2011; Möller et al., 2013). A practical difference of the two calculated spectral separability indicators is that η2 can potentially be used for a multi-class classification scenario, while the Kolmogorov–Smirnov distance can only be used to evaluate spectral differences of two different classes.
Crop monitoring using remotely sensed image data provides valuable input for a large variety of applications in environmental and agricultural research. However, method development for discrimination between spectrally highly similar crop species remains a challenge in remote sensing. Calculation of vegetation indices is a frequently applied option to amplify the most distinctive parts of a spectrum. Since no vegetation index exist, that is universally best-performing, a method is presented that finds an index that is optimized for the classification of a specific satellite data set to separate two cereal crop types. The η² (eta-squared) measure of association – presented as novel spectral separability indicator – was used for the evaluation of the numerous tested indices. The approach is first applied on a RapidEye satellite image for the separation of winter wheat and winter barley in a Central German test site. The determined optimized index allows a more accurate classification (97%) than several well-established vegetation indices like NDVI and EVI (<87%). Furthermore, the approach was applied on a RapidEye multi-spectral image time series covering the years 2010–2014. The optimized index for the spectral separation of winter barley and winter wheat for each acquisition date was calculated and its ability to distinct the two classes was assessed. The results indicate that the calculated optimized indices perform better than the standard indices for most seasonal parts of the time series. The red edge spectral region proved to be of high significance for crop classification. Additionally, a time frame of best spectral separability of wheat and barley could be detected in early to mid-summer.
Enhancing the Accuracy of Land Cover Classification by Airborne LiDAR Data and WorldView-2 Satellite Imagery
2022, ISPRS International Journal of Geo-Information

View all citing articles on Scopus

^∗: Author now at the Aerospace Corporation, Mail Station A2-1213, P.O. Box 92957 Los Angeles, CA 90009, U.S.A.

View full text

A combined nonparametric approach to feature selection and binary decision tree design

Abstract

Quantization complexity and independent measurements

IEEE Trans. Comput.

A nonparametric partitioning procedure for pattern classification

IEEE Trans. Comput.

The decision tree classifier: design and potential

Some current concepts and problems in pattern classification and feature extraction

A recursive partitioning decision rule for nonparametric classification

IEEE Trans. Comput.

K-S test for detecting changes from Landsat imagery data

IEEE Trans. Systems, Man Cybernet.