Skip to main content
SpringerLink
Log in

SeaCloudDM: a database cluster framework for managing and querying massive heterogeneous sensor sampling data

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Recent advances in sensor networks and communication technologies have made the Internet of Things (IoT) a hot research issue. An IoT system can sample and manage the historical and present states of various kinds of physical and virtual objects such as vehicles, lakes, mountains, dams, city traffic conditions, atmosphere qualities, and so forth. It is well acknowledged that IoT will greatly change the way how people live and work. However, IoT also brings about great challenges to the data management community. For instance, the data to be managed in IoT are highly dynamic and heterogeneous. Meanwhile, since the sensor sampling data are managed in a centralized manner, the data size can be huge. Moreover, sensor data are intrinsically spatial-temporal data which may involve complicated spatial-temporal computations in query processing. To meet these challenges, we propose a novel Sea-Cloud-based Data Management (SeaCloudDM) mechanism in this paper. The experimental results show that the SeaCloudDM mechanism provides satisfactory performances in managing and querying massive sensor sampling data, and is thus a viable solution for IoT data management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ye T, Huang X, Wang W, et al (2010) The annual blue book on china’s development of “Internet of things” industry. Published by CIT-CHINA

  2. Sarma S, Brock DL, Ashton K (2000) MIT auto ID WH-001: the networked physical world—proposals for engineering the next generation of computing, commerce & automatic-identification. MIT Press, Cambridge

    Google Scholar 

  3. Sundmaeker H, Guillemin P, Friess P, Woelfflé S (eds) (2010) Vision and challenges for realizing the Internet of Things. Publications Office of the European Union, Luxembourg

    Google Scholar 

  4. Ning H, Ning N, Qu S et al (2007) Layered structure and management in Internet of Things. In: Proc of the future generation communication and network (FGCN’07), Jeju Island, Korea. IEEE Comput Soc, Los Alamitos

    Google Scholar 

  5. Gurgen L, Roncancio C, Labbé C, Bottaro A, Olive V (2008) SStreaMWare: a service oriented middleware for heterogeneous senser data management. In: Proc of the 5th international conference on pervasive services. ACM, New York

    Google Scholar 

  6. Gao J, Liu F, Ning H, et al (2007) RFID coding, name and information service for Internet of Things. In: Proc of the wireless, mobile and sensor network, Shanghai, China. IEEE Press, New York

    Google Scholar 

  7. International telecommunication union (ITU) (2005) ITU internet reports 2005: the Internet of Things. Tunis: World Summit on the Information Society (WSIS)

  8. Commission of the European Communities, Internet of Things—an action plan for Europe (2009). http://ec.europa.eu/information_society/policy/rfid/documents/commiot2009.pdf

  9. Yan L, Zhang Y, Yang LT, Ning H (2008) The Internet of Things: from RFID to the next-generation pervasive network systems. Auerbach Publications, New York

    Book  Google Scholar 

  10. Haller S, Karnouskos S, Schroth C (2008) The Internet of Things in an enterprise context. In: The first future internet symposium (FIS 2008), Vienna, Austria. LNCS, vol 5468. Springer, Berlin

    Google Scholar 

  11. Giusto D, Iera A, Morabito G, Atzori L (eds) (2010) The Internet of Things—20th Tyrrhenian workshop on digital communications. Springer, Berlin

    Google Scholar 

  12. Atzori L, Iera A, Morabito G (2010) The Internet of Things: a survey. Comput Netw, 54(15)

  13. Weber RH (2010) Internet of Things—new security and privacy challenges. Comput Law Secur Rev 26(1)

  14. Luo Q, Wu H (2007) System design issues in sensor databases. In: SIGMOD, pp 1182–1185

    Google Scholar 

  15. Klan D, Hose K, Karnstedt M, Sattler K-U (2010) Power-aware data analysis in sensor networks. In: ICDE, pp 1125–1128

    Google Scholar 

  16. Buchmann E, Tatbul N, Nascimento MA (2011) Query processing in sensor networks. Distrib Parallel Databases 29(1–2):1–2

    Article  Google Scholar 

  17. Gonzalez H, Han J, Cheng H et al (2010) Modeling massive RFID data sets: a gateway-based movement graph approach. IEEE Trans Knowl Data Eng 22(1):90–104

    Article  Google Scholar 

  18. Rolewicz I, Catasta M, Jeung H, Miklós Z, Aberer K (2011) Building a front end for a sensor data cloud. In: ICCSA, pp 566–581

    Google Scholar 

  19. Sun N, Xu Z, Li G Sea-Computing: a novel computation model for the Internet of Things. Commun Chin Comput Found 2010(2)

  20. Güting RH, Böhlen MH, Erwig M, Jensen CS, et al (2000) A foundation for representing and querying moving objects. ACM Trans Database Syst 25(1)

  21. Güting RH, de Almeida VT, Ding Z (2006) Modeling and querying moving objects in networks. VLDB J 15(2)

  22. Kanth KVR, Ravada S, Abugov D (2002) Quadtree and R-tree indexes in oracle spatial: a comparison using GIS data. In: SIGMOD, pp 546–557

    Google Scholar 

  23. Kanth KVR, Ravada S, Xu W (2003) Spatial processing using oracle table functions. In: ICDE, pp 851–856

    Google Scholar 

  24. Strobl C (2008) PostGIS. In: Encyclopedia of GIS, pp 891–898

    Chapter  Google Scholar 

  25. Feature Compare of Oracle 11GR2 Spatial/Locator, PostGIS PostgreSQL, and SQL Server 2008 R2

  26. Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: Proc of the 6th symposium on operating system design and implementation (OSDI04), San Francisco, CA, USA

    Google Scholar 

  27. Chaiken R, Jenkins B, Larson P, et al (2008) SCOPE: easy and efficient parallel processing of massive data sets. In: Proc of the 34th VLDB, Auckland, New Zealand

    Google Scholar 

  28. Abadi DJ (2009) Data management in the cloud: limitations and opportunities. IEEE Data Eng Bull 32(1)

  29. Wu S, Jiang D, Ooi BC, Wu K-L (2010) Efficient B-tree based indexing for cloud data processing. Proc VLDB Endow 3(1):1207–1218

    Google Scholar 

  30. Chang F, Dean J, Ghemawat S, et al (2006) Bigtable: a distributed storage system for structured data. In: Proc of the 7th symposium on operating systems design and implementation (OSDI’06), Berkeley, CA, USA, November 2006

    Google Scholar 

  31. DeCandia G, Hastorun D, Jampani M, et al (2007) Dynamo: Amazon’s highly available key-value store. In: Proc of the 21st ACM symposium on operating systems principles (SOSP’2007), Stevenson, WA, USA, October 2007

    Google Scholar 

  32. Carstoiu D, Lepadatu E, Gaspar M (2010) Hbase—non-SQL database, performances evaluation. Int J Adv Comput Technol 2(5)

  33. Abouzeid A, Pawlikowski KB, Abadi DJ et al (2009) HadoopDB: an architectural hybrid of MapReduce and DBMS technologies for analytical workloads. In: Proc of the 35th VLDB, Lyon, France

    Google Scholar 

  34. Cooper BF, Ramakrishnan R, Srivastava U, et al (2008) PNUTS: Yahoo!’s hosted data serving platform. In: Proc of the 34th VLDB, Auckland, New Zealand, August 2008

    Google Scholar 

  35. Thusoo A, Sarma JS, Jain N, et al (2009) Hive—a warehousing solution over a map-reduced framework. In: Proc of the 35th VLDB, Lyon, France, August 2009

    Google Scholar 

  36. Campbell DG, Kakivaya G, Ellis N (2010) Extreme scale with full SQL language support in Microsoft SQL Azure. In: Proc of the ACM SIGMOD 2010, Indiana, USA, June 2010

    Google Scholar 

  37. del Cid PJ, Matthys N, Huygens C, et al (2011) Sensor middleware to support diverse data qualities. In: ITNG, pp 673–676

    Google Scholar 

  38. Domingues JPO, Damaso AVL, Rosa NS (2010) WISeMid: middleware for integrating wireless sensor networks and the Internet. In: DAIS, pp 70–83

    Google Scholar 

  39. Chandrasekaran S, Cooper O, Deshpande A, et al (2003) TelegraphCQ: continuous dataflow processing. In: SIGMOD, p 668

    Google Scholar 

  40. Ahmad Y, Berg B, Çetintemel U, et al (2005) Distributed operation in the Borealis stream processing engine. In: SIGMOD, pp 882–884

    Google Scholar 

  41. Poess M, Nambiar RO (2005) Large scale data warehouses on grid: oracle database 10g and HP ProLiant systems. In: VLDB, pp 1055–1066

    Google Scholar 

  42. Waas FM (2008) Beyond conventional data warehousing—massively parallel data processing with Greenplum database. In: BIRTE (informal proceedings)

    Google Scholar 

  43. Ding Z, Yang Q, Wu H (2011) Massive heterogeneous sensor data management in the Internet of Things. In: Proceedings of iThings 2011. IEEE Comput Soc, Los Alamitos

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by NSFC under Grants Nos. 91124001 and 60970030.

Author information

Authors and Affiliations

  1. Institute of Software, Chinese Academy of Sciences, South-Fourth-Street 4, Zhong-Guan-Cun, Beijing, 100190, P.R. China

    Zhiming Ding & Jiajie Xu

  2. National Center of ITS Engineering & Technology, Xitucheng-Road. 8, Haidian District, Beijing, 100088, P.R. China

    Qi Yang

Authors
  1. Zhiming Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiajie Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiming Ding.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ding, Z., Xu, J. & Yang, Q. SeaCloudDM: a database cluster framework for managing and querying massive heterogeneous sensor sampling data. J Supercomput 66, 1260–1284 (2013). https://doi.org/10.1007/s11227-012-0762-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-012-0762-1

Keywords

Search

Navigation