Abstract
The original Random Forest derives the final result with respect to the number of leaf nodes voted for the corresponding class. Each leaf node is treated equally and the class with the most number of votes wins. Certain leaf nodes in the topology have better classification accuracies and others often lead to a wrong decision. Also the performance of the forest for different classes differs due to uneven class proportions. In this work, a novel voting mechanism is introduced: each leaf node has an individual weight. The final decision is not determined by majority voting but rather by a linear combination of individual weights leading to a better and more robust decision. This method is inspired by the construction of a strong classifier using a linear combination of small rules of thumb (AdaBoost). Small fluctuations which are caused by the use of binary decision trees are better balanced. Experimental results on several datasets for object recognition and action recognition demonstrate that our method successfully improves the classification accuracy of the original Random Forest algorithm.
This work has been partially funded by the ERC within the starting grant Dynamic MinVIP.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156. IEEE (1996)
Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks (2012)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324 (1998)
Weinland, D., Özuysal, M., Fua, P.: Making action recognition robust to occlusions and viewpoint changes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 635–648. Springer, Heidelberg (2010)
Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. In: Computer Vision and Image Understanding (CVIU) (2006)
Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local svm approach. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR) (2004)
Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: 10th International Conference on Computer Vision (ICCV), pp. 1395–1402 (2005)
Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 29, 2247–2253 (2007)
Esuli, A., Fagni, T., Sebastiani, F.: TreeBoost.MH: A boosting algorithm for multi-label hierarchical text categorization. In: Crestani, F., Ferragina, P., Sanderson, M. (eds.) SPIRE 2006. LNCS, vol. 4209, pp. 13–24. Springer, Heidelberg (2006)
Grossmann, E.: Adatree 2: boosting to build decision trees or improving adatree with soft splitting rules. unpublished work done at the Center for Visualisation and Virtual Environments, University of Kentucky (2004)
Grossmann, E.: Adatree: Boosting a weak classifier into a decision tree. In: Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 2004), vol. 6, p. 105. IEEE Computer Society, Washington, DC (2004)
Roe, B.P., Yang, H.J., Zhu, J., Liu, Y., Stancu, I., McGregor, G.: Boosted decision trees as an alternative to artificial neural networks for particle identification. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 543, 577–584 (2005)
Zhang, Z., Xie, X.: Research on adaboost. m1 with random forest. In: 2010 2nd International Conference on Computer Engineering and Technology (ICCET), vol. 1, pp. V1–V647 (2010)
Zhou, S.K.: A binary decision tree implementation of a boosted strong classifier. In: Zhao, W., Gong, S., Tang, X. (eds.) AMFG 2005. LNCS, vol. 3723, pp. 198–212. Springer, Heidelberg (2005)
Seyedhosseini, M., Paiva, A.R., Tasdizen, T.: Fast adaboost training using weighted novelty selection. In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 1245–1250. IEEE (2011)
Schulter, S., Roth, P.M., Bischof, H.: Ordinal random forests for object detection. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 261–270. Springer, Heidelberg (2013)
Schulter, S., Wohlhart, P., Leistner, C., Saffari, A., Roth, P.M., Bischof, H.: Alternating decision forests. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
Bernard, S., Adam, S., Heutte, L.: Dynamic random forests. Pattern Recognition Letters 33, 1580–1586 (2012)
Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision 57, 137–154 (2004)
Breiman, L.: Bagging predictors. In: Machine Learning, vol. 24, pp. 123–140 (1996)
Ho, T.K.: Random decision forests. In: Proceedings of the Third International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282. IEEE (1995)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 832–844 (1998)
Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9, 1545–1588 (1997)
Bernard, S., Adam, S., Heutte, L.: Using random forests for handwritten digit recognition. In: Ninth International Conference on Document Analysis and Recognition, ICDAR 2007, vol. 2, pp. 1043–1047. IEEE (2007)
Bernard, S., Heutte, L., Adam, S.: On the selection of decision trees in random forests. In: International Joint Conference on Neural Networks, IJCNN 2009, pp. 302–307. IEEE (2009)
Fan, G., Wang, Z., Wang, J.: Cw-ssim kernel based random forest for image classification. In: Visual Communications and Image Processing 2010, pp. 774425–774425. International Society for Optics and Photonics (2010)
O’Hara, S., Draper, B.A.: Scalable action recognition with a subspace forest. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
Lui, Y.M., Beveridge, J., Kirby, M.: Action classification on product manifolds. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)
Schindler, K., Van Gool, L.: Action snippets: How many frames does human action recognition require? In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)
Li, W., Yu, Q., Sawhney, Vasconcelos, N.: Recognizing activities via bag of words for attribute dynamics. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2587–2594 (2013)
Tian, Y., Sukthankar, R., Shah, M.: Spatiotemporal deformable part models for action detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
Yu, T.H., Kim, T.K., Cipolla, R.: Unconstrained monocular 3d human pose estimation by action detection and cross-modality regression forest. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649 (2013)
Jhuang, H., Serre, T., Wolf, L., Poggio, T.: A biologically inspired system for action recognition. In: 11th International Conference on Computer Vision (ICCV), pp. 1–8. IEEE (2007)
Lin, Z., Jiang, Z., Davis, L.S.: Recognizing actions by shape-motion prototype trees. In: 12th International Conference on Computer Vision (ICCV), pp. 444–451 (2009)
Yeffet, L., Wolf, L.: Local trinary patterns for human action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)
Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)
Kihl, O., Picard, D., Gosselin, P.H., et al.: Local polynomial space-time descriptors for actions classification. In: International Conference on Machine Vision Applications (2013)
Baumann, F., Liao, J., Ehlers, A., Rosenhahn, B.: Motion binary patterns for action recognition. In: 3rd International Conference on Pattern Recognition Applications and Methods (ICPRAM) (2014)
Wang, Z., Wang, J., Xiao, J., Lin, K.-H., Huang, T.: Substructure and boundary modeling for continuous action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
Wu, X., Xu, D., Duan, L., Luo, J.: Action recognition using context and appearance distribution features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)
Li, R., Zickler, T.: Discriminative virtual views for cross-view action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Baumann, F., Li, F., Ehlers, A., Rosenhahn, B. (2014). Thresholding a Random Forest Classifier. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2014. Lecture Notes in Computer Science, vol 8888. Springer, Cham. https://doi.org/10.1007/978-3-319-14364-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-14364-4_10
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14363-7
Online ISBN: 978-3-319-14364-4
eBook Packages: Computer ScienceComputer Science (R0)