Skip to main content
SpringerLink
Log in

Comparison of machine learning techniques for target detection

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

This paper focuses on machine learning techniques for real-time detection. Although many supervised learning techniques have been described in the literature, no technique always performs best. Several comparative studies are available, but have not always been performed carefully, leading to invalid conclusions. Since benchmarking all techniques is a tremendous task, literature has been used to limit the available options, selecting the two most promising techniques (AdaBoost and SVM), out of 11 different Machine Learning techniques. Based on a thorough comparison using 2 datasets and simulating noise in the feature set as well as in the labeling, AdaBoost is concluded to be the best machine learning technique for real-time target detection as its performance is comparable to SVM, its detection time is one or multiple orders of magnitude faster, its inherent feature selection eliminates this as a separate task, while it is more straightforward to use (only three coupled parameters to tune) and has a lower training time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Amasyali MF, Ersoy OK (2011) Comparison of single and ensemble classifiers in terms of accuracy and execution time. In: International symposium on innovation intelligent system applications. doi:10.1109/INISTA.2011.5946119

  • Aruna S, Rajagopalan SP, Nandakishore LV (2011) An empirical comparison of supervised learning algorithms in disease detection. Int J Inf Technol Converg Serv 1: 81–92

    Google Scholar 

  • Bartlett MS, Littlewort G, Lainscsek C, Fasel I, Movellan J (2004) Machine learning methods for fully automatic recognition of facial expressions and facial actions. In: Proceedings of the IEEE international conference systems, man and cybernetics, pp 592–597

  • Breiman L (1999) Prediction games and arcing algorithms. Neural Comput 11(7):1493–1517. doi:10.1162/089976699300016106

    Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45(1): 5–32

    Article  MATH  Google Scholar 

  • Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Statistics/ probability series. Wadsworth Publishing Company, Belmont

    Google Scholar 

  • Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:121–167. doi:10.1023/A:1009715923555. http://dl.acm.org/citation.cfm?id=593419.593463

  • Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on machine learning, pp 161–168. doi:10.1145/1143844.1143865

  • Caruana R, Karampatziakis N, Yessenalina A (2008) An empirical evaluation of supervised learning in high dimensions. In: Proceedings of the 25th international conference on machine learning, pp 96–103. doi:10.1145/1390156.1390169

  • Che D, Hockenbury C, Marmelstein R, Rasheed K (2010) Classification of genomic islands using decision trees and their ensemble algorithms. BMC Genomics 11(Suppl 2):S1. doi:10.1186/1471-2164-11-S2-S1

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20: 273–297

    MATH  Google Scholar 

  • Dettling M, Bühlmann P (2003) Boosting for tumor classification with gene expression data. Bioinforma 19(9):1061–1069. http://dblp.uni-trier.de/db/journals/bioinformatics/bioinformatics19.htmlDettlingB03

    Google Scholar 

  • Doshi F, Brunskill E, Shkolnik A, Kollar T, Rohanimanesh K, Tedrake R, Roy N (2007) Collision detection in legged locomotion using supervised learning. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems

  • Douglas PK, Harris S, Yuille A, Cohen MS (2011)Performance comparison of machine learning algorithms and number of independent components used in fMRI decoding of belief vs. disbelief. NeuroImage 56(2):544–553. doi:10.1016/j.neuroimage.2010.11.002

  • Enzweiler M, Gavrila DM (2009) Monocular pedestrian detection: survey and experiments. IEEE Trans Pattern Analysis Mach Intell 31(12):2179–2195. doi:10.1109/TPAMI.2008.260

    Google Scholar 

  • Flexer A (1994) Statistical evaluation of neural network experiments: minimum requirements and current practice. Aust Res Inst Artif Intell 2: 1005–1008

    Google Scholar 

  • Frank A, Asuncion A (2010) UCI machine learning repository. http://archive.ics.uci.edu/ml

  • Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55:119–139. doi:10.1006/jcss.1997.1504. http://portal.acm.org/citation.cfm?id=261540.261549

    Google Scholar 

  • Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200): 675–701. doi:10.2307/2279372

    Article  Google Scholar 

  • Friedman J, Hastie T, Tibshirani R (1998) Additive logistic regression: a statistical view of boosting. Ann Stat 28: 2000

    MathSciNet  Google Scholar 

  • García S, Herrera F (2008) An extension on “Statistical Comparisons of Classifiers over Multiple Data Sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694. http://www.jmlr.org/papers/volume9/garcia08a/garcia08a.pdf

    Google Scholar 

  • Graf ABA., Smola AJ., Borer S (2003) Classification in a normalized feature space using support vector machines. IEEE Trans Neural Netw 14(3): 597–605. doi:10.1109/TNN.2003.811708

    Article  Google Scholar 

  • Guyon I., Hur AB., Gunn S., Dror G (2004) Result analysis of the nips 2003 feature selection challenge. Adv Neural Inf Process Syst 17: 545–552

    Google Scholar 

  • Hall M., Frank E., Holmes G., Pfahringer B., Reutemann P., Witten IH (2009) The weka data mining software: an update. SIGKDD Explor Newsl 11(1): 10–18. doi:10.1145/1656274.1656278

    Article  Google Scholar 

  • Ho YC, Pepyne DL (2002) Simple explanation of the no free lunch theorem of optimization. Cybern Syst Anal 38(2):4409–4414. http://www.springerlink.com/index/T1Q45BRR3TNU1K1R.pdf

  • Hume D, Hendel C (1955) An Inquiry concerning human understanding: with a supplement, an abstract of a treatise of human nature. Library of liberal arts, Bobbs-Merrill. http://books.google.nl/books?id=-P4HAQAAIAAJ

  • Iman RL, Davenport JM (1980) Approximations of the critical region of the friedman statistic. Commun Stat 9: 571–595

    Article  Google Scholar 

  • Joachims T (1999) Making large-scale support vector machine learning practical. MIT Press, Cambridge, pp 169–184. http://dl.acm.org/citation.cfm?id=299094.299104

  • Khan R, Hanbury A, Stöttinger J (2010) Skin detection: a random forest approach. IEEE Int Conf Image Process 4613–4616

  • Kotsiantis SB (2007) Supervised machine learning: a review of classification techniques. Informatica 31(3):249–268. http://www.informatica.si/PDF/31-3/11_Kotsiantis

    Google Scholar 

  • Krishnaraj Y., Reddy CK (2008) Boosting methods for protein fold recognition: an empirical comparison. IEEE Int Conf Bioinform Biomed 8: 393–396. doi:10.1109/BIBM.2008.83

    Google Scholar 

  • Lin H, Li L (2005) Infinite ensemble learning with support vector machines. In: Proceedings of the 16th European conference on machine learning, pp 242–254

  • Lloyd SP (1982) Least squares quantization in pcm. IEEE Trans Inf Theory 28: 129–137

    Article  MathSciNet  MATH  Google Scholar 

  • Matthews BW (1975) Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Struct 405(2):442–451. doi:10.1016/0005-2795(75)90109-9. http://www.sciencedirect.com/science/article/pii/0005279575901099

  • McDonald RA, Hand DJ, Eckley IA (2003) An empirical comparison of three boosting algorithms on real data sets with artificial class noise. In: Proceedings of the 4th international conference multiple classification system, pp 35–44. http://dl.acm.org/citation.cfm?id=1764295.1764301

  • Mease D, Wyner A (2008) Evidence contrary to the statistical view of boosting. J Mach Learn Res 9:131–156. http://dl.acm.org/citation.cfm?id=1390681.1390687

    Google Scholar 

  • Mitchell TM (1980) The need for biases in learning generalizations. Technical Report CBM-TR-117, Department of Computer Science, Rutgers University

  • Miyamoto D, Hazeyama H, Kadobayashi Y (2008) An evaluation of machine learning-based methods for detection of phishing sites. Aus J Intell Inf Process Syst 10(2): 54–63

    Google Scholar 

  • Mooney RJ (1996) Comparative experiments on disambiguating word senses: an illustration of the role of bias in machine learning. In: Proceedings of the conference on Empire methods in national language processing, pp 82–91. http://www.cs.utexas.edu/users/ai-lab/pub-view.php?PubID=51464

  • Prechelt L (1996) A quantitative study of experimental evaluations of neural network learning algorithms: current research practice. Neural Netw 9(3):457–462. doi:10.1016/0893-6080(95)00123-9. http://www.sciencedirect.com/science/article/pii/0893608095001239

  • Quddus A, Fieguth P, Basir O (2005) Adaboost and support vector machines for white matter lesion segmentation in mr images. In: 27th Annual international conference on engineering in medicine and biology society, pp 463–466. doi:10.1109/IEMBS.2005.1616447

  • Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, San Francisco

    Google Scholar 

  • Ridgeway G (1999) The state of boosting. Comput Sci Stat 31: 172–181

    Google Scholar 

  • Rodríguez JJ, Kuncheva LI, Alonso CJ (2006) Rotation forest: a new classifier ensemble method. IEEE Trans Pattern Analysis Mach Intell 28(10): 1619–1630

    Article  Google Scholar 

  • Rojas-Bello RN, Lago-Fernández LF, Martínez-Muñoz G, Sánchez-Montañ és MA (2011) A comparison of techniques for robust gender recognition. IEEE Int Conf Image Process 569–572

  • Salzberg S (1997) On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min Knowl Discov 1: 317–327

    Article  Google Scholar 

  • Salzberg SL (1999) On comparing classifiers: a critique of current research and methods. Data Min Knowl Discov 1: 1–12

    Google Scholar 

  • Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human Pose Recognition in Parts from Single Depth Images. Comput Vis Pattern Recogn. doi:10.1109/CVPR.2011.5995316. http://research.microsoft.com/apps/pubs/default.aspx?id=145347

  • Statnikov A, Aliferis CF (2007) Are random forests better than support vector machines for microarray-based cancer classification? In: AMIA annual symposium proceedings, pp 686–690. http://view.ncbi.nlm.nih.gov/pubmed/18693924

  • Statnikov A, Tsamardinos I, Dosbayev Y, Aliferis CF (2005) GEMS: a system for automated cancer diagnosis and biomarker discovery from microarray gene expression data. Int J Med Inform 74(7–8):491–503. doi:10.1016/j.ijmedinf.2005.05.002

    Google Scholar 

  • Statnikov A, Wang L, Aliferis C (2008) A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification. BMC Bioinforma 9(1):319. doi:10.1186/1471-2105-9-319

    Google Scholar 

  • Tang Y, Krasser S, He Y, Yang W, Alperovitch D (2008) Support vector machines and random forests modeling for spam senders behavior analysis. GLOBECOM pp 2174–2178. http://dblp.uni-trier.de/db/conf/globecom/globecom2008.htmlTangKHYA08

  • Vink JP, de Haan G (2011) No-reference metric design with machine learning for local video compression artifact level. IEEE J Sel Top Signal Process 5(2): 297–308. doi:10.1109/JSTSP.2010.2055832

    Article  Google Scholar 

  • Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2): 137–154. doi:10.1023/B:VISI.0000013087.49260.fb

    Article  Google Scholar 

  • Wang Y, Han P, Lu X, Wu R, Huang J (2006) The performance comparison of adaboost and svm applied to sar atr. CIE international conference on radar, pp 1–4. doi:10.1109/ICR.2006.343515

  • Webb GI (2000) Multiboosting: a technique for combining boosting and wagging. Mach Learn 40(2):159–196

    Google Scholar 

  • Wolpert DH (1992) On the connection between in-sample testing and generalization error. Complex Syst 6: 47–94

    MathSciNet  MATH  Google Scholar 

  • Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82. doi:10.1109/4235.585893

    Google Scholar 

  • Wu X, Kumar V, Ross Quinlan J, Ghosh J, Yang Q, Motoda H, McLachlan GJ, Ng A, Liu B, Yu PS, Zhou ZH, Steinbach M, Hand DJ, Steinberg D (2007) Top 10 algorithms in data mining. Knowl Inf Syst 14:1–37. doi:10.1007/s10115-007-0114-2

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Video and Image Processing Group, Philips Research, Eindhoven, The Netherlands

    Jelte Peter Vink & Gerard de Haan

Authors
  1. Jelte Peter Vink
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerard de Haan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jelte Peter Vink.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vink, J.P., de Haan, G. Comparison of machine learning techniques for target detection. Artif Intell Rev 43, 125–139 (2015). https://doi.org/10.1007/s10462-012-9366-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-012-9366-7

Keywords

Search

Navigation