Skip to main content

Advertisement

Springer Nature Link
Log in

Big data aggregation in the case of heterogeneity: a feasibility study for digital health

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In big data applications, an important factor that may affect the value of the acquired data is the missing data, which arises when data is lost either during acquisition or during storage. The former can be a result of faulty acquisition devices or non responsive sensors whereas the latter can occur as a result of hardware failures at the storage units. In this paper, we consider human activity recognition as a case study of a typical machine learning application on big datasets. We conduct a comprehensive feasibility study on the fusion of sensory data that is acquired from heterogeneous sources. We present insights on the aggregation of heterogeneous datasets with minimal missing data values for future use. Our experiments on the accuracy, F-1 score, and PPV of various key machine learning algorithms show that sensory data acquired by wearables are less vulnerable to missing data and smaller training sets whereas smart portable devices require larger training sets to reduce the impacts of possibly missing data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Suciu G, Suciu V, Halunga S, Fratu O (2015) Big data, internet of things and cloud convergence for e-Health applications. Adv Intell Syst Comput. https://doi.org/10.1007/978-3-319-16486-1fng15

    Article  Google Scholar 

  2. Paul A, Rho S (2016) Probabilistic model for M2M in IoT networking and communication. Telecommun Syst 62(1):59–66

    Article  Google Scholar 

  3. Liu W, Park EK (2014) Big data as an e-Health service. In: 2014 international conference on computing, networking and communications ICNC 2014. https://doi.org/10.1109/ICCNC.2014.6785471

  4. Wu J, Guo S, Huang H, Liu W, Xiang Y (2018) Information and communications technologies for sustainable development goals: state-of-the-art, needs and perspectives. IEEE Commun Surv Tutor 20:2389–2406

    Article  Google Scholar 

  5. Diaz M, Juan G, Lucas O, Ryuga A (2012) Big data on the internet of things: an example for the e-Health. In: Proceedings—6th international conference on innovative mobile and internet services in ubiquitous computing, IMIS 2012. https://doi.org/10.1109/IMIS.2012.198

  6. Naversnik K, Mrhar A (2013) Cost-effectiveness of a Novele-Health depression service. Telemed e-Health. https://doi.org/10.1089/tmj.2012.0081

    Article  Google Scholar 

  7. Thuemmler C, Bai C (eds) (2017) Health 4.0: how virtualization and big data are revolutionizing healthcare. Springer, New York, NY

    Google Scholar 

  8. Shin D, Sahama T, Gajanayake R (2013) Secured e-health data retrieval in DaaS and Big Data. In: 2013 IEEE 15th international conference on e-Health networking, applications and services, Healthcom 2013. https://doi.org/10.1109/HealthCom.2013.6720677

  9. Roy S, Conti M, Setia S, Jajodia S (2014) Secure data aggregation in wireless sensor networks: filtering out the attacker’s impact. IEEE Trans Inf Forensics Secur. https://doi.org/10.1109/TIFS.2014.2307197

    Article  Google Scholar 

  10. Daniel A, Subburathinam K, Paul A, Rajkumar N, Rho S (2017) Big autonomous vehicular data classifications: towards procuring intelligence in ITS. Vehic Commun 9:306–312

    Article  Google Scholar 

  11. Quoc Viet Hung N, Tam NT, Tran LN, Aberer K (2013) An evaluation of aggregation techniques in crowd sourcing. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-642-41154-0fng1

  12. Paul A (2014) Real-time power management for embedded M2M using intelligent learning methods. ACM Trans Embed Comput Syst 13(5 s):148

    Google Scholar 

  13. Chen MY, Chen BT (2014) Online fuzzy time series analysis based on entropy discretization and a Fast Fourier Transform. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2013.07.024

    Article  Google Scholar 

  14. Perkins NJ, Cole SR, Harel O, Tchetgen Tchetgen EJ, Sun B, Mitchell EM, Schisterman EF (2017) Principled approaches to missing data in epidemiologic studies. Am J Epidemiol. https://doi.org/10.1093/aje/kwx348

    Article  Google Scholar 

  15. Beaulieu-Jones BK, Moore JH, CONSORTIUM T.P.R.O.A.A.C.T. (2017) Missing data imputation in the electronic health record using deeply learned autoencoders. Pacific symposium on biocomputing. https://doi.org/10.1142/9789813207813$4ng0021

  16. Lara OD, Labrador MA (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor 15(3):1192–1209

    Article  Google Scholar 

  17. Su X, Tong H, Ji P (2014) Activity recognition with smartphone sensors. Tsinghua Sci Technol. https://doi.org/10.1109/TST.2014.6838194

    Article  Google Scholar 

  18. Davila J, Cretu AM, Zaremba M (2017) Wearable sensor data classification for human activity recognition based on an iterative learning framework. Sensors. https://doi.org/10.3390/s17061287

    Article  Google Scholar 

  19. Hassanalieragh M, Page A, Soyata T, Sharma G, Aktas M, Mateos G, Kantarci B, Andreescu S (2015) Health monitoring and management using Internet-of-Things (IoT) sensing with cloud-based processing: opportunities and challenges. In: Proceedings—2015 IEEE international conference on services computing, SCC 2015. https://doi.org/10.1109/SCC.2015.47

  20. Lupton D (2013) The commodification of patient opinion: the digital patient experience economy in the age of big data. Sociol Health Illness. https://doi.org/10.1111/1467-9566.12109

    Article  Google Scholar 

  21. Springman MK, Bermeo Y, Limper HM, Tothy AS (2016) Developing an analytic approach to understanding the patient care experience. J Patient Exp. https://doi.org/10.1177/2374373516685956

    Article  Google Scholar 

  22. Delen D, Fuller C (2013) An analytic approach to understanding and predicting healthcare coverage. Stud Health Technol Inf. https://doi.org/10.3233/978-1-61499-276-9-198

    Article  Google Scholar 

  23. Brownstein JS, Freifeld CC, Madoff LC (2009) Digital disease detection harnessing the web for public health surveillance. N Engl J Med. https://doi.org/10.1056/NEJMp0900702

    Article  Google Scholar 

  24. Barrett Ma, Humblet O, Hiatt RA, Adler NE (2013) Big data and disease prevention: from quantified self to quantified communities. Big Data. https://doi.org/10.1089/big.2013.0027

    Article  Google Scholar 

  25. Zhang M, Sawchuk AA (2013) Human daily activity recognition with sparse representation using wearable sensors. IEEE J Biomed Health Inform. https://doi.org/10.1109/JBHI.2013.2253613

    Article  Google Scholar 

  26. Din S, Paul A (2019) Smart health monitoring and management system: toward autonomous wearable sensing for internet of things using big data analytics. Future Gener Comput Syst 91:611–619

    Article  Google Scholar 

  27. Paul A, Ahmad A, Rathore MM, Jabbar S (2016) Smartbuddy: defining human behaviors using big data analytics in social internet of things. IEEE Wirel Commun 23(5):68–74

    Article  Google Scholar 

  28. Chernbumroong S, Cang S, Atkins A, Yu H (2013) Elderly activities recognition and classification for applications in assisted living. Expert Syst Appl 40(5):1662–1674

    Article  Google Scholar 

  29. Gjoreski H, Kozina S, Gams M, Lustrek M (2014) RAReFall—real-time activity recognition and fall detection system. In: Pervasive computing and communications workshops (PERCOM workshops), 2014 IEEE international conference on. IEEE, pp 145–147

  30. Zhou B, Sundholm M, Cheng J, Cruz H, Lukowicz P (2017) Measuring muscle activities during gym exercises with textile pressure mapping sensors. Pervasive Mob Comput 38:331–345

    Article  Google Scholar 

  31. O’Donovan T, O’Donoghue J, Sreenan C, Sammon D, O’Reilly P, O’Connor K (2009) A context aware wireless body area network (BAN). Pervasive computing technologies for healthcare (2009) PervasiveHealth 2009. 3rd international conference on

  32. Rutherford JJ (2010) Wearable technology. IEEE Eng Med Biol Mag. https://doi.org/10.1109/MEMB.2010.936550

    Article  Google Scholar 

  33. Piwek L, Ellis DA, Andrews S, Joinson A (2016) The rise of consumer health wearables: promises and barriers. PLoS Med. https://doi.org/10.1371/journal.pmed.1001953

    Article  Google Scholar 

  34. Cahyani NDW, Martini B, Choo KKR, Al-Azhar AMN (2017) Forensic data acquisition from cloud-of-things devices: windows smartphones as a case study. Concurr Comput. https://doi.org/10.1002/cpe.3855

    Article  Google Scholar 

  35. Rehman M, Liew C, Wah T, Shuja J, Daghighi B (2015) Mining personal data using smartphones and wearable devices: a survey. Sensors. https://doi.org/10.3390/s150204430

    Article  Google Scholar 

  36. Feng M, Fukuda Y, Mizuta M, Ozer E (2015) Citizen sensors for SHM: use of accelerometer data from smartphones. Sensors (Switzerland). https://doi.org/10.3390/s150202980

    Article  Google Scholar 

  37. Habibzadeh H, Qin Z, Soyata T, Kantarci B (2017) Largescale distributed dedicated- and non-dedicated smart city sensing systems. IEEE Sens J 17(23):7649–7658. https://doi.org/10.1109/JSEN.2017.2725638

    Article  Google Scholar 

  38. Pouryazdan M, Kantarci B, Soyata T, Foschini L, Song H (2017) Quantifying user reputation scores, data trustworthiness, and user incentives in mobile crowdsensing. IEEE Access 5:1382–1397. https://doi.org/10.1109/ACCESS.2017.2660461

    Article  Google Scholar 

  39. Yang D, Xue G, Fang X, Tang J (2016) Incentive mechanisms for crowdsensing: crowdsourcing with smartphones. IEEE ACM Trans Netw 24(3):1732–1744. https://doi.org/10.1109/TNET.2015.2421897

    Article  Google Scholar 

  40. Predic B, Zhixian Y, Eberle J, Stojanovic D, Aberer K (2013) ExposureSense: integrating daily activities with air quality using mobile participatory sensing. In: 2013 IEEE international conference on pervasive computing and workshops C (PERCOM Workshops). https://doi.org/10.1109/PerComW.2013.6529500

  41. Obinikpo AA, Zhang Y, Song H, Luan TH, Kantarcih B (2017) Queuing algorithm for effective target coverage in mobile crowd sensing. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2017.2688366

    Article  Google Scholar 

  42. Kantarci B, Mouftah HT (2014) Trustworthy sensing for public safety in cloud-centric internet of things. IEEE Internet Things J 1(4):360–368

    Article  Google Scholar 

  43. Hao T, Xing G, Zhou G (2013) iSleep: unobtrusive sleep quality monitoring using smartphones. In: Proceedings of the 11th ACM conference on embedded networked sensor systems. https://doi.org/10.1145/2517351.2517359

  44. Linkov I, Massey O, Keisler J, Rusyn I, Hartung T (2015) From “weight of evidence” to quantitative data integration using multicriteria decision analysis and Bayesian methods. Altex. https://doi.org/10.14573/altex.1412231

    Article  Google Scholar 

  45. Chen Y, Cook WD, Du J, Hu H, Zhu J (2015) Bounded and discrete data and Likert scales in data envelopment analysis: application to regional energy efficiency in China. Ann Oper Res. https://doi.org/10.1007/s10479-015-1827-3

    Article  Google Scholar 

  46. Pargett M, Umulis DM (2013) Quantitative model analysis with diverse biological data: applications in developmental pattern formation. Methods. https://doi.org/10.1016/j.ymeth.2013.03.024

    Article  Google Scholar 

  47. Vosloo J, Taylor-Powell E, Renner M, Research-part B, Reid S, Punch KF, O‘connor H, Gibson N, Miles MB, Huberman Ma, Saldana J, Mellish L, Morris S, Do M, Mcnair R, Taft A, Hegarty K, Lacey A, Luff D, Hunn A, Fox N, Hunn A, Free R, For D, Data Q, Miles A, Framework U, Framework U, Flick U, Data ACI (2014) Qualitative data analysis qualitative data. The SAGE handbook of qualitative data analysis. https://doi.org/10.1136/ebnurs.2011.100352

    Article  Google Scholar 

  48. Raghupathi W, Raghupathi V (2014) Big data analytics in healthcare: promise and potential. Health Inf Sci Syst. https://doi.org/10.1186/2047-2501-2-3

    Article  MATH  Google Scholar 

  49. Nakamura J (2005) Image sensors and signal processing for digital still cameras. https://doi.org/10.1201/9781420026856

  50. Bouveyron C, Brunet-Saumard C (2014) Model-based clustering of high-dimensional data: a review. Comput Stat Data Anal. https://doi.org/10.1016/j.csda.2012.12.008

    Article  MathSciNet  MATH  Google Scholar 

  51. Tomasev N, Radovanovic M, Mladenic D, Ivanovic M (2014) The role of hubness in clustering high-dimensional data. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2013.25

    Article  Google Scholar 

  52. Graham JW (2012) Analysis of missing data. Miss Data. https://doi.org/10.1007/978-1-4614-4018-5fng2

    Article  MATH  Google Scholar 

  53. Zhou P, Fan LW, Zhou DQ (2010) Data aggregation in constructing composite indicators: a perspective of information loss. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2009.05.039

    Article  Google Scholar 

  54. Ladra S, Torra V (2010) Information loss for synthetic data through fuzzy clustering. Int J Uncertain Fuzziness Knowl Based Syst. https://doi.org/10.1142/S0218488510006362

    Article  Google Scholar 

  55. Hsieh TS, Noyes D, Liu H, Fiondella L (2015) Quantifying the impact of data loss incidents on publicly-traded organizations. In: 2015 IEEE international symposium on technologies for homeland security, HST 2015. https://doi.org/10.1109/THS.2015.7225301

  56. Obinikpo AA, Kantarci B (2017) Big sensed data meets deep learning for smarter health care in smart cities. J Sens Actuator Netw. https://doi.org/10.3390/jsan6040026

    Article  Google Scholar 

  57. Neubeck L, Lowres N, Benjamin EJ, Freedman SB, Coorey G, Redfern J (2015) The mobile revolution using smartphone apps to prevent cardiovascular disease. https://doi.org/10.1038/nrcardio.2015.34

    Article  Google Scholar 

  58. Velasco E, Agheneza T, Denecke K, Kirchner G, Eckmanns T (2014) Social media and internet-based data in global systems for public health surveillance: a systematic review. https://doi.org/10.1111/1468-0009.12038

    Article  Google Scholar 

  59. Shwe HY, Jet TK, Chong PHJ (2016) An IoT-oriented data storage framework in smart city applications. In: 2016 international conference on information and communication technology convergence (ICTC), pp 106–108

  60. Witten IH, Frank E, Hall MA, Pal CJ (2016) Data mining: practical machine learning tools and techniques. Morgan Kaufmann, San Francisco, California

    Google Scholar 

  61. Wu J, Guo S, Li J, Zeng D (2016) Big data meet green challenges: big data toward green applications. IEEE Syst J 10(3):888–900

    Article  Google Scholar 

  62. Chen CP, Zhang CyY (2014) Data-intensive applications, challenges, techniques and technologies: a survey on Big Data. Inf Sci. https://doi.org/10.1016/j.ins.2014.01.015

    Article  Google Scholar 

  63. Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on machine learning ICML 06. https://doi.org/10.1145/1143844.1143865

  64. Mesnil G, Dauphin Y, Glorot X, Rifai S, Bengio Y, Goodfellow I, Lavoie E, Muller X, Desjardins G, Warde-Farley D, Vincent P (2011) Unsupervised and transfer learning challenge: a deep learning approach. In: Proceedings of the 2011 international conference on unsupervised and transfer learning workshop, Vol 27, pp 97–111, JMLR. org

  65. Hahne F, Huber W, Gentleman R, Falcon S (2008) Unsupervised machine learning. Bioconduct Case Stud. https://doi.org/10.1007/978-0-387-77240-0$4

    Article  Google Scholar 

  66. Libbrecht MW, Noble WS (2015) Machine learning applications in genetics and genomics. https://doi.org/10.1038/nrg3920

    Article  Google Scholar 

  67. Grys BT, Lo DS, Sahin N, Kraus OZ, Morris Q, Boone C, Andrews BJ (2017) Machine learning and computer vision approaches for phenotypic profiling. J Cell Biol. https://doi.org/10.1083/jcb.201610026

    Article  Google Scholar 

  68. Hijazi S, Page A, Kantarci B, Soyata T (2016) Machine learning in cardiac health monitoring and decision support. IEEE Comput Mag 49(11):38–48. https://doi.org/10.1109/MC.2016.339

    Article  Google Scholar 

  69. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn. https://doi.org/10.1023/A:1022627411411

    Article  MATH  Google Scholar 

  70. Squares L, Vector S (2010) 4 variants of support vector machines. Advances. https://doi.org/10.1007/978-1-84996-098-4

    Article  Google Scholar 

  71. Wang Z, Xue X (2014) Multi-class support vector machine. Support Vector Mach Appl. https://doi.org/10.1007/978-3-319-02300-7$4ng2

    Article  Google Scholar 

  72. Hamedani K, Liu L, Atat R, Wu J, Yi Y (2018) Reservoir computing meets smart grids: attack detection using delayed feedback networks. IEEE Trans Ind Inf 14(2):734–743

    Article  Google Scholar 

  73. Murty MN, Raghava R (2016) Linear support vector machines. In: Support vector machines and perceptrons. Springer, Cham. https://doi.org/10.1007/978-3-319-41063-0fng4

    Chapter  MATH  Google Scholar 

  74. Paul S, Boutsidis C, Magdon-Ismail M, Drineas P (2013) Random projections for support vector machines. In: Proceedings of the sixteenth international conference on artificial intelligence and statistics. https://doi.org/10.1145/2641760

    Article  Google Scholar 

  75. Raghavendra S, Deka PC (2014) Support vector machine applications in the field of hydrology: a review. https://doi.org/10.1016/j.asoc.2014.02.002

    Article  Google Scholar 

  76. Fischetti M (2016) Fast training of support vector machines with Gaussian kernel. Discret Optim. https://doi.org/10.1016/j.disopt.2015.03.002

    Article  MathSciNet  MATH  Google Scholar 

  77. Shinde A, Sahu A, Apley D, Runger G (2014) Preimages for variation patterns from kernel PCA and bagging. IIE Trans. https://doi.org/10.1080/0740817X.2013.849836

    Article  Google Scholar 

  78. Breiman L (1996) Bagging predictors. Mach Learn. https://doi.org/10.1007/BF00058655

    Article  MATH  Google Scholar 

  79. Kozak K, Kozak M, Stapor K (2006) Weighted k-nearest-neighbor techniques for high throughput screening data. Int J Biomed Sci 1:155–160

    Google Scholar 

  80. Xu Y, Zhu Q, Fan Z, Qiu M, Chen Y, Liu H (2013) Coarse to fine K nearest neighbor classifier. Pattern Recogn Lett. https://doi.org/10.1016/j.patrec.2013.01.028

    Article  Google Scholar 

  81. Yadav S, Kaur A, Bhauryal NS (2016) Resolving the celestial classification using fine k-NN classifier. In: 2016 4th international conference on parallel, distributed and grid computing, PDGC 2016. https://doi.org/10.1109/PDGC.2016.7913215

  82. Chavarriaga R, Sagha H, Calatroni A, Digumarti ST, Tröster G, Millan JDR, Roggen D (2013) The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recogn Lett. https://doi.org/10.1016/j.patrec.2012.12.014

    Article  Google Scholar 

  83. Stisen A, Blunck H, Bhattacharya S, Prentow TS, Kjaergaard MB, Dey A, Sonne T, Jensen MM (2015) Smart devices are different: assessing and mitigating-mobile sensing heterogeneities for activity recognition. In: Proceedings of the 13th ACM conference on embedded networked sensor systems—SenSys ’15. https://doi.org/10.1145/2809695.2809718

  84. Auria L, Moro RA (2008) Support vector machines (SVM) as a technique for solvency analysis. DIW Berlin German Institute for economic Research. https://doi.org/10.2139/ssrn.1424949

Download references

Acknowledgements

This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) under RGPIN/2017-04032.

Author information

Authors and Affiliations

  1. University of Ottawa, Ottawa, Canada

    Alex Adim Obinikpo & Burak Kantarci

Authors
  1. Alex Adim Obinikpo
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Burak Kantarci
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Burak Kantarci.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Obinikpo, A.A., Kantarci, B. Big data aggregation in the case of heterogeneity: a feasibility study for digital health. Int. J. Mach. Learn. & Cyber. 10, 2643–2655 (2019). https://doi.org/10.1007/s13042-018-00904-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-018-00904-3

Keywords