Skip to main content
SpringerLink
Log in

On Comparing Early and Late Fusion Methods

  • Conference paper
  • First Online:
Advances in Computational Intelligence (IWANN 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14134))

Included in the following conference series:

Abstract

This paper presents a theoretical comparison of early and late fusion methods. An initial discussion on the conditions to apply early or late (soft or hard) fusion is introduced. The analysis show that, if large training sets are available, early fusion will be the best option. If training sets are limited we must do late fusion, either soft or hard. In this latter case, the complications inherent in optimally estimating the fusion function could be avoided in exchange for lower performance. The paper also includes a comparative review of the fusion state of the art methods with the following divisions: early sensor-level fusion; early feature-level fusion; late score-level fusion (late soft fusion); and late decision-level fusion (late hard fusion). The main strengths and weaknesses of the methods are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Usa, H., Escamilla-Ambrosio, P.J., Mort, N.: Hybrid kalman filter-fuzzy logic adaptive multisensor data fusion architectures (2003)

    Google Scholar 

  2. Vergara, L., Soriano, A., Safont, G., Salazar, A.: On the fusion of non-independent detectors. Digit. Signal Process. 50, 24–33 (2016)

    Article  Google Scholar 

  3. Salazar, A., Safont, G., Vergara, L., Letters, E.V.-P.R.: Pattern recognition techniques for provenance classification of archaeological ceramics using ultrasounds. Pattern Recogn. Lett. 135, 441–450 (2020)

    Article  Google Scholar 

  4. Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374(2065), 20150202 (2016)

    Google Scholar 

  5. Soriano, A., Vergara, L., Ahmed, B., Salazar, A.: Fusion of scores in a detection context based on Alpha integration. Neural Comput. 27(9), 1983–2010 (2015)

    Article  Google Scholar 

  6. Safont, G., Salazar, A., Vergara, L.: Multiclass Alpha integration of scores from multiple classifiers. Neural Comput. 31(4), 806–825 (2019)

    Article  MathSciNet  Google Scholar 

  7. Safont, G., Salazar, A., Vergara, L.: Vector score alpha integration for classifier late fusion. Pattern Recogn. Lett. 136, 48–55 (2020)

    Article  Google Scholar 

  8. Salazar, A., Vergara, L., Vidal, E.: A proxy learning curve for the Bayes Classifier. Pattern Recogn. 136(109240), 1–14 (2023)

    Google Scholar 

  9. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  10. Kalman, R.E., Bucy, R.S.: New results in linear filtering and prediction theory. J. Basic Eng. 83(1), 95–108 (1961)

    Article  MathSciNet  Google Scholar 

  11. Moon, S., Park, Y., Ko, D.W., Suh, I.H.: Multiple kinect sensor fusion for human skeleton tracking using Kalman filtering (2016)

    Google Scholar 

  12. Yazdkhasti, S., Sasiadek, J.Z.: Multi sensor fusion based on adaptive Kalman filtering. In: Dołęga, B., Głębocki, R., Kordos, D., Żugaj, M. (eds.) Advances in Aerospace Guidance, Navigation and Control, pp. 317–333. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-65283-2_17

    Chapter  Google Scholar 

  13. Dempster, A.P.: A generalization of Bayesian inference. J. Roy. Stat. Soc.: Ser. B (Methodol.) 30(2), 205–232 (1968)

    MathSciNet  Google Scholar 

  14. Coninx, A., et al.: Bayesian sensor fusion with fast and low power stochastic circuits. In: 2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - November (2016)

    Google Scholar 

  15. Coué, C., Fraichard, T., Bessiere, P., Mazer, E.: Multi-sensor data fusion using Bayesian programming: an automotive application. In:IEEE International Conference on Intelligent Robots and Systems, vol. 1, pp. 141–146 (2002)

    Google Scholar 

  16. Amin, M., Akhoundi, A., Valavi, E.: Multi-sensor fuzzy data fusion using sensors with different characteristics (2010)

    Google Scholar 

  17. Stover, J.A., Hall, D.L., Gibson, R.E.: A fuzzy-logic architecture for autonomous multisensor data fusion. IEEE Trans. Industr. Electron. 43(3), 403–410 (1996)

    Article  Google Scholar 

  18. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)

    Article  MathSciNet  Google Scholar 

  19. Quan, Y., Zhou, M.C., Luo, Z.: On-line robust identification of tool-wear via multi-sensor neural-network fusion. Eng. Appl. Artif. Intell. 11(6), 717–722 (1998)

    Article  Google Scholar 

  20. Lee, J., Steele, C.M., Chau, T.: Swallow segmentation with artificial neural networks and multi-sensor fusion. Med. Eng. Phys. 31(9), 1049–1055 (2009)

    Article  Google Scholar 

  21. Kańtoch, E.: Human activity recognition for physical rehabilitation using wearable sensors fusion and artificial neural networks. Comput. Cardiol. 2010(44), 1–4 (2017)

    Google Scholar 

  22. Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417–441 (1933)

    Article  Google Scholar 

  23. Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2(4), 433–459 (2010)

    Article  Google Scholar 

  24. Hasan, M.M., Islam, N., Rahman, M.M.: Gastrointestinal polyp detection through a fusion of contourlet transform and neural features. J. King Saud Univ. – Comput. Inf. Sci. 34(3), 526–533 (2022)

    Google Scholar 

  25. Nasir, H., Stanković, V., Marshall, S.: Singular value decomposition based fusion for super-resolution image reconstruction. Signal Process Image Commun. 27(2), 180–191 (2012)

    Article  Google Scholar 

  26. Zhao, X., Ye, B.: Singular value decomposition packet and its application to extraction of weak fault feature. Mech. Syst. Signal Process 70–71, 73–86 (2016)

    Article  Google Scholar 

  27. Zhu, H., He, Z., Wei, J., Wang, J., Zhou, H.: Bearing fault feature extraction and fault diagnosis method based on feature fusion, vol. 21, no. 7, p. 2524 (2021)

    Google Scholar 

  28. Ye, X., Gao, W., Yan, Y., Osadciw, L.A.: Multiple tests for wind turbine fault detection and score fusion using two- level multidimensional scaling (MDS), vol. 7704, pp. 70–77 (2010)

    Google Scholar 

  29. Tian, G.Y., Taylor, D.: Colour image retrieval using virtual reality. In: Proceedings of the International Conference on Information Visualisation, vol. 2000-July, pp. 221–225 (2000)

    Google Scholar 

  30. Choo, J., Bohn, S., Nakamura, G.C., White, A.M., Park, H.: Heterogeneous data fusion via space alignment using nonmetric multidimensional scaling. In: Proceedings West Mark Ed Association Conference, pp. 177–188 (2012)

    Google Scholar 

  31. He, Q.-H., et al: Deep learning and radiomic feature-based blending ensemble classifier for malignancy risk prediction in cystic renal lesions (2023)

    Google Scholar 

  32. Wang, K., Xu, C., Li, G., Zhang, Y., Zheng, Y., Sun, C.: Combining convolutional neural networks and self-attention for fundus diseases identification (2023)

    Google Scholar 

  33. Jing, J., Wu, H., Sun, J., Fang, X., Zhang, H.: Multimodal fake news detection via progressive fusion networks. Inf. Process Manag. 60(1) (2023)

    Google Scholar 

  34. Salazar, A., Safont, G., Soriano, A., Vergara, L.: Automatic credit card fraud detection based on non-linear signal processing. In: ICCST, pp 207–2012 (2012). Article no. 6393560

    Google Scholar 

  35. Salazar, A., Safont, G., Vergara, L.: Surrogate techniques for testing fraud detection algorithms in credit card operations. ICCST 2014(6986987), 124–129 (2014)

    Google Scholar 

  36. Vergara, L., Salazar, A., Belda, J., Safont, G., Moral, S., Iglesias, S.: Signal processing on graphs for improving automatic credit card fraud detection. In: ICCST, pp. 1–6, Spain (2017)

    Google Scholar 

  37. Nanni, L., Lumini, A., Brahnam, S.: Likelihood ratio-based features for a trained biometric score fusion. Expert Syst. Appl. 38(1), 58–63 (2011)

    Article  Google Scholar 

  38. Zafar, R., et al.: Prediction of human brain activity using likelihood ratio based score fusion. IEEE Access 5, 13010–13019 (2017)

    Article  Google Scholar 

  39. Salazar, A., Safont, G., Rodriguez, A., Vergara, L.: Combination of multiple detectors for credit card fraud detection. In: ISSPIT, pp. 138–143, Limassol, Cyprus (2016)

    Google Scholar 

  40. Hube, J.P.: Neyman-Pearson biometric score fusion as an extension of the sum rule, vol. 6539, pp. 200–208 (2007)

    Google Scholar 

  41. Hammouche, R., Attia, A., Akhrouf, S.: Score level fusion of major and minor finger knuckle patterns based symmetric sum-based rules for person authentication. Evolving Syst. 13(3), 469–483 (2022)

    Google Scholar 

  42. Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)

    Article  Google Scholar 

  43. Eskandari, M., Toygar, Ö.: Fusion of face and iris biometrics using local and global feature extraction methods. Signal Image Video Process 8(6), 995–1006 (2014)

    Article  Google Scholar 

  44. Fierrez-Aguilar, J., et al.: A comparative evaluation of fusion strategies for multimodal biometric verification. Lect. Notes Comput. Sci. 2688, 830–837 (2003)

    Google Scholar 

  45. Arigbabu, O.A., et al.: Integration of multiple soft biometrics for human identification. Pattern Recogn. Lett. 68, 278–287 (2015)

    Google Scholar 

  46. Ruta, D., Gabrys, B.: Classifier selection for majority voting. Inf. Fusion 6(1), 63–81 (2005)

    Article  Google Scholar 

  47. Jimenez, L.O., Morales-Morell, A.: Classification of hyperdimensional data based on feature and decision fusion approaches using projection pursuit, majority voting, and neural networks. IEEE Geosci. Remote. Sens. 37(3 I), 1360–1366 (1999)

    Google Scholar 

  48. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    Article  Google Scholar 

  49. Ćwiklińska-Jurkowska, M.: Boosting, bagging and fixed fusion methods performance for aiding diagnosis. Biocybern. Biomed. Eng. 32(2), 17–31 (2012)

    Article  Google Scholar 

  50. Dietterich, T.G.: Experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach. Learn. 40(2), 139–157 (2000)

    Article  Google Scholar 

  51. Ferreira, A.J., Figueiredo, M.A.T.: Boosting algorithms: a review of methods, theory, and applications. Ensemble Mach. Learn. 35–85 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Telecommunications and Multimedia Applications, Universitat Politècnica de València, Valencia, Spain

    Luis Manuel Pereira, Addisson Salazar & Luis Vergara

Authors
  1. Luis Manuel Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Addisson Salazar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luis Vergara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Addisson Salazar .

Editor information

Editors and Affiliations

  1. University of Granada, Granada, Spain

    Ignacio Rojas

  2. University of Malaga, Málaga, Spain

    Gonzalo Joya

  3. Polytechnic University of Catalonia, Vilanova i la Geltrú, Spain

    Andreu Catala

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pereira, L.M., Salazar, A., Vergara, L. (2023). On Comparing Early and Late Fusion Methods. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14134. Springer, Cham. https://doi.org/10.1007/978-3-031-43085-5_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43085-5_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43084-8

  • Online ISBN: 978-3-031-43085-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation