Angshuman Paul
ABSTRACT
Reinforcement learning improves classification accuracy. But use of reinforcement learning is relatively unexplored in case of random forest classifier. We propose a reinforced random forest (RRF) classifier that exploits reinforcement learning to improve classification accuracy. Our algorithm is initialized with a forest. Then the entire training data is tested using the initial forest. In order to reinforce learning, we use mis-classified data points to grow certain number of new trees. A subset of the new trees is added to the existing forest using a novel graph-based approach. We show that addition of these trees ensures improvement in classification accuracy. This process is continued iteratively until classification accuracy saturates. The proposed RRF has low computational burden. We achieve at least 3% improvement in F-measure compared to random forest in three breast cancer datasets. Results on benchmark datasets show significant reduction in average classification error.
- http://ludo17.free.fr/mitos_2012/. Available as on 08.03.2016.Google Scholar
- http://amida13.isi.uu.nl/?q=node/62. Available as on 08.03.2016.Google Scholar
- http://mitos-atypia-14.grand-challenge.org/dataset/. Available as on 08.03.2016.Google Scholar
- A. Asuncion and D. Newman. Uci machine learning repository, 2007.Google Scholar
- L. Breiman. Random forests. Machine learning, 45(1):5--32, 2001. Google ScholarDigital Library
- O. Chapelle, B. Schölkopf, A. Zien, et al. Semi-supervised learning. 2006. Google ScholarDigital Library
- T. F. Cootes, M. C. Ionita, C. Lindner, and P. Sauer. Robust and accurate shape model fitting using random forest regression voting. In Computer Vision-ECCV 2012, pages 278--291. Springer, 2012. Google ScholarDigital Library
- A. Cuzzocrea, S. L. Francis, and M. M. Gaber. An information-theoretic approach for setting the optimal number of decision trees in random forests. In Systems, Man, and Cybernetics (SMC), 2013 IEEE International Conference on, pages 1013--1019. IEEE, 2013. Google ScholarDigital Library
- G. Fanelli, M. Dantone, J. Gall, A. Fossati, and L. Van Gool. Random forests for real time 3d face analysis. International Journal of Computer Vision, 101(3):437--458, 2013. Google ScholarDigital Library
- Y. Freund, R. E. Schapire, et al. Experiments with a new boosting algorithm. In ICML, volume 96, pages 148--156, 1996.Google ScholarDigital Library
- I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror. Result analysis of the nips 2003 feature selection challenge. In Advances in neural information processing systems, pages 545--552, 2004. Google ScholarDigital Library
- J. J. Hull. A database for handwritten text recognition research. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 16(5):550--554, 1994. Google ScholarDigital Library
- L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, pages 237--285, 1996. Google ScholarDigital Library
- T. M. Khoshgoftaar, M. Golawala, and J. V. Hulse. An empirical study of learning from imbalanced data using random forest. In Tools with Artificial Intelligence, 2007. ICTAI 2007. 19th IEEE International Conference on, volume 2, pages 310--317. IEEE, 2007. Google ScholarDigital Library
- P. Latinne, O. Debeir, and C. Decaestecker. Limiting the number of trees in random forests. In Multiple Classifier Systems, pages 178--187. Springer, 2001. Google ScholarDigital Library
- X.-Y. Liu, J. Wu, and Z.-H. Zhou. Exploratory undersampling for class-imbalance learning. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 39(2):539--550, 2009. Google ScholarDigital Library
- C. Luo, Z. Wang, S. Wang, J. Zhang, and J. Yu. Locating facial landmarks using probabilistic random forest. Signal Processing Letters, IEEE, 22(12):2324--2328, Dec 2015.Google ScholarCross Ref
- T. M. Oshiro, P. S. Perez, and J. A. Baranauskas. How many trees in a random forest? In MLDM, pages 154--168. Springer, 2012. Google ScholarDigital Library
- A. Paul, A. Dey, D. P. Mukherjee, J. Sivaswamy, and V. Tourani. Regenerative random forest with automatic feature selection to detect mitosis in histopathological breast cancer images. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, pages 94--102. Springer, 2015.Google ScholarCross Ref
- A. Paul and D. Mukherjee. Mitosis detection for invasive breast cancer grading in histopathological images. Image Processing, IEEE Transactions on, 24(11):4041--4054, Nov 2015.Google Scholar
- A. Paul and D. P. Mukherjee. Enhanced random forest for mitosis detection. In Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image Processing, page 85. ACM, 2014. Google ScholarDigital Library
- N. Quadrianto and Z. Ghahramani. A very simple safe-bayesian random forest. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 37(6):1297--1303, June 2015.Google Scholar
- S. Ren, X. Cao, Y. Wei, and J. Sun. Global refinement of random forest. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 723--730, 2015.Google Scholar
- M. R. Segal. Machine learning benchmarks and random forest regression. Center for Bioinformatics & Molecular Biostatistics, 2004.Google Scholar
- S. Skiena. Dijkstra's algorithm. Implementing Discrete Mathematics: Combinatorics and Graph Theory with Mathematica, Reading, MA: Addison-Wesley, pages 225--227, 1990. Google ScholarDigital Library
- V. Svetnik, A. Liaw, C. Tong, J. C. Culberson, R. P. Sheridan, and B. P. Feuston. Random forest: a classification and regression tool for compound classification and qsar modeling. Journal of chemical information and computer sciences, 43(6):1947--1958, 2003.Google Scholar
Index Terms
- Reinforced random forest
Recommendations
An improved random forest based on the classification accuracy and correlation measurement of decision trees
Highlights- Propose an improved random forest based on the improvement of decision trees.
- Improve the evaluation mechanism for the classification effect of decision trees.
- Propose a method for quantifying the diversity between decision trees.
AbstractRandom forest is one of the most widely used machine learning algorithms. Decision trees used to construct the random forest may have low classification accuracies or high correlations, which affects the comprehensive performance of the random ...
A fuzzy random forest
When individual classifiers are combined appropriately, a statistically significant increase in classification accuracy is usually obtained. Multiple classifier systems are the result of combining several individual classifiers. Following Breiman's ...
Rotation Forest: A New Classifier Ensemble Method
We propose a method for generating classifier ensembles based on feature extraction. To create the training data for a base classifier, the feature set is randomly split into K subsets (K is a parameter of the algorithm) and Principal Component Analysis ...
Comments