Abstract
Ensemble learning is in favour of machine learning community due to its tolerance in handling divergence and biasness issues faced by a single learner. In this work, an ensemble temporal learner, namely temporal sampling forest (TS-F), is proposed. Building on the random forest, we consider its limitations in handling temporal classification tasks. Temporal data classification is an important area of machine learning and data mining, where it fills the gap of ordinary data classification when the observed datasets are temporally related across sequential and time domains. TS-F incorporated the temporal sampling (bagging) and temporal randomization procedures in the classical random forest, hence extending its ability to handle temporal data . TS-F was tested on 11 public sequential and temporal datasets from different domains . Experiments demonstrate that TS-F could provide promising results with average classification accuracy of 98 %, substantiating its ability to escalate the random forest performance in the application of temporal classification.




Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Altun K, Barshan B, Tunçel O (2010) Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognit 43(10):3605–3620. doi:10.1016/j.patcog.2010.04.019
Anacleto R, Figueiredo L, Almeida A, Novais P (2014) Localization system for pedestrians based on sensor and information fusion. IEEE 17th international conference on information fusion (FUSION), p 8. http://ieeexplore.ieee.org.ezproxy.auckland.ac.nz/stamp/stamp.jsp?tp=&arnumber=6916127&isnumber=6915967
Anacleto R, Figueiredo L, Almeida A, Novais P, Meireles A (2015) Step characterization using sensor information fusion and machine learning. Int J Interact Multimed Artif Intell 3(5):53–60. doi:10.9781/ijimai.2015.357
Bache K, Lichman M (2013) UCI machine learning repository. School of Information and Computer Science, University of California, Irvin. [http://archive.ics.uci.edu/ml]
Bernard S, Adam S, Heutte L (2012) Dynamic random forests. Pattern Recognit Lett 33(12):1580–1586. doi:10.1016/j.patrec.2012.04.003
Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140. doi:10.1007/BF00058655
Breiman L (2001) Random forest. Mach Learn 45(1):5–32
Casale P, Pujol O, Radeva P (2012) Personalization and user verification in wearable systems using biometric walking patterns. Pers Ubiquitous Comput 16(5):563–580. doi:10.1007/s00779-011-0415-z
Chen R, Deng Z, Song Z (2015) The prediction of malignant middle cerebral artery infarction: a predicting approach using random forest. J Stroke Cerebrovasc Dis 24(5):958–964. doi:10.1016/j.jstrokecerebrovasdis.2014.12.016
Cohen J (1960) A coefficient of agreement for nominal scale. Educ Psychol Meas 20(1):37–46. doi:10.1177/001316446002000104
Corcoran J, Frank W, Maloney M (1974) CORST. 1. pdf. J Symb Logic 39(4):625–637
Deng H, Runger G, Tuv E, Vladimir M (2013) A time series forest for classification and feature extraction. Inf Sci 239:142–153. doi:10.1016/j.ins.2013.02.030
Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874. doi:10.1016/j.patrec.2005.10.010
Firmino PRA, de Mattos Neto PSG, Ferreira TAE (2014) Correcting and combining time series forecasters. Neural Netw 50:1–11. doi:10.1016/j.neunet.2013.10.008
Freund Y, Schapire R (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139. doi:10.1006/jcss.1997.1504
Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Proceedings of the 13th international conference on machine learning, Bari, Italy, 3–6 July 1996, pp 148–156
Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232
García-Díaz V, Pascual-Espada J, Pelayo G-Bustelo C, Cueva-Lovelle JM (2015) Towards a standard-based domain-specific platform to solve machine learning-based problems. Int J Interact Multimed Artif Intell 3(5):6–12. doi:10.9781/ijimai.2015.351
Geurts P, Ernst D, Wehenkel L (2006) Extremely Randomized Trees. Mach Learn 63(1):3–42. doi:10.1007/s10994-006-6226-1
González Crespo R, Escobar RF, Joyanes Aguilar L, Velazco S, Castillo Sanz AG (2013) Use of ARIMA mathematical analysis to model the implementation of expert system courses by means of free software OpenSim and Sloodle platforms in virtual university campuses. Expert Syst Appl 40(18):7381–7390. doi:10.1016/j.eswa.2013.06.054
Heo J, Yang JY (2014) AdaBoost based bankruptcy forecasting of Korean construction companies. Appl Soft Comput 24:494–499. doi:10.1016/j.asoc.2014.08.009
Ho TK (1998) The random subspace method for constructing decision forest. IEEE Trans Pattern Anal Mach Intell 20(8):832–844. doi:10.1109/34.709601
Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial. Stat Sci 14(4):382–417. doi:10.2307/2676803
Hong S, Khim S, Rhee PK (2014) Efficient facial landmark localization using spatial-contextual AdaBoost algorithm. J Vis Commun Image Represent 25(6):1366–1377. doi:10.1016/j.jvcir.2014.05.001
Kim M-J, Kang D-K, Kim HB (2015) Geometric mean based boosting algorithm with over-sampling to resolve data imbalance problem for bankruptcy prediction. Expert Syst Appl 42(3):1074–1082. doi:10.1016/j.eswa.2014.08.025
Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint conference on artificial intelligence (IJCAI), vol 5. Morgan Kaufmann, San Mateo, pp 1137–1143
Lebedev AV, Westman E, Van Westen GJP, Kramberger MG, Lundervold A, Aarsland D, Simmons A (2014) Random forest ensembles for detection and prediction of Alzheimer’s disease with a good between-cohort robustness. NeuroImage Clin 6:115–125. doi:10.1016/j.nicl.2014.08.023
Liu B, Ma Y, Wong CK, Yu PS (2003) Scoring the data using association rules. Appl Intell 18(2):119–135
Liu S, Xu J, Zhao J, Xie X, Zhang W (2014) Efficiency enhancement of a process-based rainfall-runoff model using a new modified AdaBoost.RT technique. Appl Soft Comput 23:521–529. doi:10.1016/j.asoc.2014.05.033
Louzada F, Ara A (2012) Bagging k-dependence probabilistic networks: an alternative powerful fraud detection tool. Expert Syst Appl 39(14):11583–11592. doi:10.1016/j.eswa.2012.04.024
Mitsa T (2010) Temporal data mining, 1st edn. Chapman & Hall/CRC. http://dl.acm.org/citation.cfm?id=1809755
Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106
Radicioni DP, Esposito R (2010) BREVE?: an HMPerceptron-based chord recognition system. Adv Music Inf Retr Stud Comput Intell 274:143–164
Revesz P, Triplet T (2011) Temporal data classification using linear classifiers. Inf Syst 36(1):30–41. doi:10.1016/j.is.2010.06.006
Rodríguez JJ, Kuncheva LI, Alonso CJ (2006) Rotation forest: a new classifier ensemble method. IEEE Trans Pattern Anal Mach Intell 28(10):1619–1630. doi:10.1109/TPAMI.2006.211
Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227. doi:10.1007/BF00116037
Seewald A, Fürnkranz J (2001) An evaluation of grading classifiers. Advances in intelligent data analysis. Lecture notes in computer science, 2189, pp 115–124. http://link.springer.com/chapter/10.1007/3-540-44816-0_12
Ting KM, Witten IH (1997) Stacking bagged and dagged models. In: Proceedings of the fourteenth international conference on machine learning, pp 367–375
Tripoliti EE, Fotiadis DI, Manis G (2013) Modifications of the construction and voting mechanisms of the random forests algorithm. Data Knowl Eng 87:41–65. doi:10.1016/j.datak.2013.07.002
Tseng VS, Lee CH (2009) Effective temporal data classification by integrating sequential pattern mining and probabilistic induction. Expert Syst Appl 36(5):9524–9532. doi:10.1016/j.eswa.2008.10.077
Webb GI (2000) MultiBoosting: a technique for combining boosting and wagging. Mach Learn 40(2):159–196. doi:10.1023/A:1007659514849
Witten IH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques. Elsevier, Amsterdam. doi:10.1016/B978-0-12-374856-0.00014-6
Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241–259. doi:10.1016/S0893-6080(05)80023-1
Yang Y, Jiang J (2014) HMM-based hybrid meta-clustering ensemble for temporal data. Knowl Based Syst 56:299–310. doi:10.1016/j.knosys.2013.12.004
Zhang Y, Haghani A (2015) A gradient boosting method to improve travel time prediction. Trans Res Part C Emerg Technol. doi:10.1016/j.trc.2015.02.019
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Ooi, S.Y., Tan, S.C. & Cheah, W.P. Temporal sampling forest (\(\varvec{\textit{TS-F}}\)): an ensemble temporal learner. Soft Comput 21, 7039–7052 (2017). https://doi.org/10.1007/s00500-016-2242-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2242-7