research-article

Performance comparisons of spatial data processing techniques for a large scale mobile phone dataset

Authors:

Apichon Witayangkurn,

Teerayut Horanont,

Ryosuke ShibasakiAuthors Info & Claims

COM.Geo '12: Proceedings of the 3rd International Conference on Computing for Geospatial Research and Applications

Article No.: 25, Pages 1 - 6

https://doi.org/10.1145/2345316.2345346

Published: 01 July 2012 Publication History

Abstract

Mobile technology, especially mobile phone, is very popular nowadays. Increasing number of mobile users and availability of GPS-embedded mobile phones generate large amount of GPS trajectories that can be used in various research areas such as people mobility and transportation planning. However, how to handle such a large-scale dataset is a significant issue particularly in spatial analysis domain. In this paper, we aimed to explore a suitable way for extracting geo-location of GPS coordinate that achieve large-scale support, fast processing, and easily scalable both in storage and calculation speed. Geo-locations are cities, zones, or any interesting points. Our dataset is GPS trajectories of 1.5 million individual mobile phone users in Japan accumulated for one year. The total number was approximately 9.2 billion records. Therefore, we conducted performance comparisons of various methods for processing spatial data, particularly for a huge dataset. In this work, we first processed data on PostgreSQL with PostGIS that is a traditional way for spatial data processing. Second, we used java application with spatial library called Java Topology suite (JTS). Third, we tried on Hadoop Cloud Computing Platform focusing on using Hive on top of Hadoop to allow SQL-like support. However, Hadoop/Hive did not support spatial query at the moment. Hence, we proposed a solution to enable spatial support on Hive. As the results, Hadoop/hive with spatial support performed best result in large-scale processing among evaluated methods and in addition, we recommended techniques in Hadoop/Hive for processing different types of spatial data.

References

[1]

Liao, L., et al. 2005. Building Personal Map from GPS Data. In proceedings of IJCAI MOO05, Springer Press(2005): 249--265

[2]

Ashbrook, D., and Starner, T. 2003. Using GPS to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing 7(5), 275--286

Digital Library

[3]

Zheng, Y., et al. 2008. Learning transportation mode from raw GPS data from geographic applications on the Web. In Proceedings of WWW 2008, (Beijing, China, April 2008), ACM Press: 247--256

Digital Library

[4]

Zheng, Y., et al. 2009. Mining interesting location and travel sequences from GPS trajectories. In Proceedings of WWW 2009, (Madrid, Spain, April 2009), ACM Press: 791--800

Digital Library

[5]

PostGIS: http://postgis.refractions.net/

[6]

Java Topology Suite: http://tsusiatsoftware.net/jts/main.html

[7]

Yang, J. and Wu, Su. 2010. Studies on Application of Cloud Computing Techniques in GIS. In Proceedings of IGASS 2010, (China, 2010), pp. 492--495.

[8]

Buyya, R., et al. 2008. Cloud Computing and Emerging IT Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility. Future Generation Computer Systems 25(6), pp. 599--616

Digital Library

[9]

Hadoop Project: http://hadoop.apache.org/

[10]

Thusoo, A., et al. 2010. Data warehousing and analytics infrastructure at Facebook. In Proceedings of ACM SIGMOD 2010, pp. 1013--1020.

Digital Library

[11]

Lam, C. 2011. Hadoop in Action. Connecticut, pp.-17--19.

Digital Library

[12]

Zhang, S., et al. 2009. Spatial queries evaluation with MapReduce. In Proceedings of International Conference on Grid and Cooperative Computing 2009, IEEE Computer Society (2009), pp. 287--292.

Digital Library

[13]

Zhang, S., et al. 2009. SJMR: Parallelizing spatial join with MapReduce on clusters. CLUSTER (2009), pp. 1--8.

[14]

Hive Project: http://hive.apache.org/

[15]

Thusoo, A., et al. 2010. Hive - a petabyte scale data warehouse using hadoop. In Proceedings of ICDE 2010, pp. 996--1005.

[16]

Wang, F., et al. 2011. Hadoop-GIS: A High Performance Query System for Analytical Medical Imaging with MapReduce. Technical Report. Center for Comprehensive Information, Emory University.

[17]

Katayama, N. and Satoh, S. 1997. The SR-tree: An Index Structure for High-Dimensional Nearest Neighbor Queries. In Proceedings of ACM SIGMOD 1997, (Arizona, USA, 1997), pp. 396--380.

Digital Library

Cited By

Witayangkurn AArai AShibasaki R(2022)Development of Big Data-Analysis Pipeline for Mobile Phone Data with Mobipack and Spatial EnhancementISPRS International Journal of Geo-Information10.3390/ijgi1103019611:3(196)Online publication date: 15-Mar-2022
https://doi.org/10.3390/ijgi11030196
Chaikhan SPhimoltares SLursinsap C(2021)Correct and stable sorting for overflow streaming data with a limited storage size and a uniprocessorPeerJ Computer Science10.7717/peerj-cs.3557(e355)Online publication date: 12-Feb-2021
https://doi.org/10.7717/peerj-cs.355
Lei B(2020)The Data Visualization of Large Scale AIS Trajectories Data on Hadoop2020 6th International Conference on Control, Automation and Robotics (ICCAR)10.1109/ICCAR49639.2020.9108055(758-761)Online publication date: Apr-2020
https://doi.org/10.1109/ICCAR49639.2020.9108055
Show More Cited By

Index Terms

Performance comparisons of spatial data processing techniques for a large scale mobile phone dataset
1. Human-centered computing
  1. Visualization
    1. Visualization application domains
      1. Geographic visualization
2. Information systems

Recommendations

A Performance Study of Big Spatial Data Systems
BigSpatial '18: Proceedings of the 7th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data

With the accelerated growth in spatial data volume, being generated from a wide variety of sources, the need for efficient storage, retrieval, processing and analyzing of spatial data is ever more important. Hence, spatial data processing system has ...
Towards a framework for large-scale multimedia data storage and processing on Hadoop platform

Cloud computing techniques take the form of distributed computing by utilizing multiple computers to execute computing simultaneously on the service side. To process the increasing quantity of multimedia data, numerous large-scale multimedia data ...
Efficient spatial query processing for big data
SIGSPATIAL '14: Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Spatial queries are widely used in many data mining and analytics applications. However, a huge and growing size of spatial data makes it challenging to process the spatial queries efficiently. In this paper we present a lightweight and scalable spatial ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

COM.Geo '12: Proceedings of the 3rd International Conference on Computing for Geospatial Research and Applications

July 2012

212 pages

ISBN:9781450311137

DOI:10.1145/2345316

General Chair:
Lindi Liao

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

COM.Geo '12

COM.Geo '12: 3rd International Conference on Computing for Geospatial Research and Application

July 1 - 3, 2012

D.C., Washington, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
376
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Witayangkurn AArai AShibasaki R(2022)Development of Big Data-Analysis Pipeline for Mobile Phone Data with Mobipack and Spatial EnhancementISPRS International Journal of Geo-Information10.3390/ijgi1103019611:3(196)Online publication date: 15-Mar-2022
https://doi.org/10.3390/ijgi11030196
Chaikhan SPhimoltares SLursinsap C(2021)Correct and stable sorting for overflow streaming data with a limited storage size and a uniprocessorPeerJ Computer Science10.7717/peerj-cs.3557(e355)Online publication date: 12-Feb-2021
https://doi.org/10.7717/peerj-cs.355
Lei B(2020)The Data Visualization of Large Scale AIS Trajectories Data on Hadoop2020 6th International Conference on Control, Automation and Robotics (ICCAR)10.1109/ICCAR49639.2020.9108055(758-761)Online publication date: Apr-2020
https://doi.org/10.1109/ICCAR49639.2020.9108055
Maso JTorres ABaumann P(2019)New Model for Geospatial Coverages in JSONEmerging Technologies and Applications in Data Processing and Management10.4018/978-1-5225-8446-9.ch015(316-357)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8446-9.ch015
Lei B(2019)A Hadoop-Based Spatial Computation Framework for Large-Scale AIS Data2019 IEEE 2nd International Conference on Electronics Technology (ICET)10.1109/ELTECH.2019.8839429(599-602)Online publication date: May-2019
https://doi.org/10.1109/ELTECH.2019.8839429
Bao LLe Y(2018)A Spatial Big Data Framework for Maritime Traffic Data2018 3rd International Conference on Computational Intelligence and Applications (ICCIA)10.1109/ICCIA.2018.00054(244-248)Online publication date: Jul-2018
https://doi.org/10.1109/ICCIA.2018.00054
Witayangkurn AHoranont TNagai MShibasaki R(2017)Large Scale Mobility Analysis: Extracting Significant Places Using Hadoop/Hive and Spatial ProcessingRecent Advances and Future Prospects in Knowledge, Information and Creativity Support Systems10.1007/978-3-319-70019-9_17(205-219)Online publication date: 2-Dec-2017
https://doi.org/10.1007/978-3-319-70019-9_17
Li MWu NTang XLu J(2017)Understanding Job-Housing Relationship from Cell Phone Data Based on HadoopBig Data Support of Urban Planning and Management10.1007/978-3-319-51929-6_19(359-387)Online publication date: 28-Sep-2017
https://doi.org/10.1007/978-3-319-51929-6_19
Mao BYu ZCao J(2016)Large scale spatial temporal data visualization based on spark and 3D volume rendering2016 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2016.7727428(1879-1882)Online publication date: Jul-2016
https://doi.org/10.1109/IJCNN.2016.7727428
Witayangkurn AHoranont TSekimoto YShibasaki RMattern FSantini SCanny JLangheinrich MRekimoto J(2013)Anomalous event detection on large-scale GPS data from mobile phones using hidden markov model and cloud platformProceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication10.1145/2494091.2497352(1219-1228)Online publication date: 8-Sep-2013
https://dl.acm.org/doi/10.1145/2494091.2497352

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten