Abstract
Developing effective and efficient small-scale data classification methods is very challenging in the digital age. Recent researches have shown that deep forest achieves a considerable increase in classification accuracy compared with general methods, especially when the training set is small. However, the standard deep forest may experience over-fitting and feature vanishing in dealing with small sample size. In this paper, we tackle this problem by proposing a skip connection deep forest (SForest), which can be viewed as a modification of the standard deep forest model. It leverages multi-class-grained scanning method to train multiple binary forest from different training sub-dataset of classes to encourage the diversity of ensemble and solve the class-imbalance problem. To expand the diversity of each layer in cascade forest, five different classifiers are employed. Meanwhile, the fitting quality of each classifiers is taken into consideration in representation learning. In addition, we propose a skip connection strategy to augment the feature vector, and use Gradient Boosting Decision Tree (GBDT) as the final classifier to improve the overall performance. Experiments demonstrated the proposed model achieved superior performance than the-state-of-the-art deep forest methods with almost the same parameter.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29(6), 82–97 (2012)
Utkin, L.V., Ryabinin, M.A.: A siamese deep forest. Knowl. Based Syst. 139, 13–22 (2018)
Dong, M., Yao, L., Wang, X., Benatallah, B., Zhang, S.: GrCAN: gradient boost convolutional autoencoder with neural decision forest. arXiv preprint arXiv:1806.08079 (2018)
Hu, G., Peng, X., Yang, Y., Hospedales, T.M., Verbeek, J.: Frankenstein: learning deep face representations using small data. IEEE Trans. Image Process. 27(1), 293–303 (2018)
Chen, M., Shi, X., Zhang, Y., Wu, D., Guizani, M.: Deep features learning for medical image analysis with convolutional autoencoder neural network. IEEE Trans. Big Data (2017)
Zhou, Z.H., Jiang, Y.: NeC4.5 neural ensemble based C4.5. IEEE Trans. Knowl. Data Eng. 16(6), 770–773 (2004)
Zhou, Z.H., Feng, J.: Deep forest: towards an alternative to deep neural networks. arXiv preprint arXiv:1702.08835 (2017)
Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat., 1189–1232 (2001)
Guo, Y., Liu, S., Li, Z., Shang, X.: Towards the classification of cancer subtypes by using cascade deep forest model in gene expression data. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1664–1669. IEEE (2017)
Martínez-Muñoz, G., Suárez, A.: Out-of-bag estimation of the optimal sample size in bagging. Pattern Recogn. 43(1), 143–152 (2010)
Bylander, T.: Estimating generalization error on two-class datasets using out-of-bag estimates. Mach. Learn. 48(1–3), 287–297 (2002)
Mellor, A., Boukir, S.: Exploring diversity in ensemble classification: applications in large area land cover mapping. ISPRS J. Photogrammetry Remote Sens. 129, 151–161 (2017)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. ACM (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-volume 1, pp. 142–150. Association for Computational Linguistics (2011)
Díaz-Uriarte, R., De Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC Bioinf. 7(1), 3 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, M., Zhang, Z. (2019). Small-Scale Data Classification Based on Deep Forest. In: Douligeris, C., Karagiannis, D., Apostolou, D. (eds) Knowledge Science, Engineering and Management. KSEM 2019. Lecture Notes in Computer Science(), vol 11775. Springer, Cham. https://doi.org/10.1007/978-3-030-29551-6_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-29551-6_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29550-9
Online ISBN: 978-3-030-29551-6
eBook Packages: Computer ScienceComputer Science (R0)