research-article

Spatiotemporal Event Forecasting from Incomplete Hyper-local Price Data

Authors:

Arnold P. Boedihardjo,

Naren RamakrishnanAuthors Info & Claims

CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Pages 507 - 516

https://doi.org/10.1145/3132847.3132996

Published: 06 November 2017 Publication History

Abstract

Hyper-local pricing data, e.g., about foods and commodities, exhibit subtle spatiotemporal variations that can be useful as crucial precursors of future events. Three major challenges in modeling such pricing data include: i) temporal dependencies underlying features; ii) spatiotemporal missing values; and iii) constraints underlying economic phenomena. These challenges hinder traditional event forecasting models from being applied effectively. This paper proposes a novel spatiotemporal event forecasting model that concurrently addresses the above challenges. Specifically, given continuous price data, a new soft time-lagged model is designed to select temporally dependent features. To handle missing values, we propose a data tensor completion method based on price domain knowledge. The parameters of the new model are optimized using a novel algorithm based on the Alternative Direction Methods of Multipliers (ADMM). Extensive experimental evaluations on multiple datasets demonstrate the effectiveness of our proposed approach.

References

[1]

Henk-Jan Brinkman and Cullen S Hendrix. Food insecurity and violent conflict: Causes, consequences, and addressing the challenges. World Food Programme, 2011.

[2]

Joe Weinberg, Ryan Bakker, et al. Let them eat cake: Food prices, domestic policy and social unrest. Conflict Management and Peace Science, 32(3):309--326, 2015.

[3]

Oeindrila Dube and Juan Vargas. Commodity price shocks and civil conflict: Evidence from colombia*. The Review of Economic Studies, 2013.

[4]

World Bank Group. World Development Indicators 2012. World Bank Publications, 2012.

[5]

Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. Earthquake shakes twitter users: Real-time event detection by social sensors. In Proceedings of the 19th International Conference on World Wide Web, WWW '10, pages 851--860, New York, NY, USA, 2010. ACM.

Digital Library

[6]

Liang Zhao, Jiangzhuo Chen, Feng Chen, Wei Wang, Chang-Tien Lu, and Naren Ramakrishnan. Simnest: Social media nested epidemic simulation via online semi-supervised deep learning. In Proceedings of the 2015 IEEE International Conference on Data Mining (ICDM), ICDM '15, pages 639--648, Washington, DC, USA, 2015. IEEE Computer Society.

Digital Library

[7]

Johan Bollen, Huina Mao, and Xiaojun Zeng. Twitter mood predicts the stock market. Journal of Computational Science, 2(1):1--8, 2011.

[8]

Brendan O'Connor, Ramnath Balasubramanyan, Bryan R Routledge, and Noah A Smith. From tweets to polls: Linking text sentiment to public opinion time series. ICWSM, 11(122--129):1--2, 2010.

[9]

Liang Zhao, Qian Sun, Jieping Ye, Feng Chen, Chang-Tien Lu, and Naren Ramakrishnan. Multi-task learning for spatio-temporal event forecasting. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15, pages 1503--1512, New York, NY, USA, 2015. ACM.

Digital Library

[10]

Matthew S. Gerber. Predicting crime using twitter and kernel density estimation. Decision Support Systems, 61:115--125, 2014.

[11]

Susan E Hardy, Heather Allore, and Stephanie A Studenski. Missing data: a special challenge in aging research. Journal of the American Geriatrics Society, 57(4):722--729, 2009.

[12]

Sujuan Gao. A shared random effect parameter approach for longitudinal dementia data with non-ignorable missing data. Statistics in Medicine, 23(2):211--219, 2004.

[13]

Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. J. Mach. Learn. Res., 11:2287--2322, August 2010.

Digital Library

[14]

Jin Huang, Feiping Nie, Heng Huang, Yu Lei, and Chris Ding. Social trust prediction using rank-k matrix recovery. IJCAI '13, pages 2647--2653. AAAI Press, 2013.

Digital Library

[15]

Emmanuel J. Candès and Terence Tao. The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theor., 56(5):2053--2080, May 2010.

Digital Library

[16]

Ji Liu, Przemyslaw Musialski, Peter Wonka, and Jieping Ye. Tensor completion for estimating missing values in visual data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1):208--220, 2013.

Digital Library

[17]

M. Signoretto, R. Van de Plas, B. De Moor, and J. A. K. Suykens. Tensor versus matrix completion: A comparison with application to spectral data. IEEE Signal Processing Letters, 18(7):403--406, July 2011.

[18]

Hua Wang, Feiping Nie, and Heng Huang. Low-rank tensor completion with spatio-temporal consistency. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI'14, pages 2846--2852. AAAI Press, 2014.

Digital Library

[19]

Frank Harrell. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Springer, 2015.

[20]

Michael Lim and Trevor Hastie. Learning interactions via hierarchical group-lasso regularization. Journal of Computational and Graphical Statistics, 24(3):627--654, 2015. 26759522.

[21]

Heewon Park and Fumitake Sakaori. Lag weighted lasso for time series model. Computational Statistics, 28(2):493--504, 2013.

Digital Library

[22]

Robert Tibshirani and Xiaotong Suo. An ordered lasso and sparse time-lagged regression. Technometrics, 58(4):415--423, 2016.

[23]

L. Eldén. Matrix Methods in Data Mining and Pattern Recognition. Society for Industrial and Applied Mathematics, 2007.

[24]

Owen A Lamont and Richard H Thaler. Anomalies: The law of one price in financial markets. The Journal of Economic Perspectives, 17(4):191--202, 2003.

[25]

Ryan Compton, Craig Lee, Jiejun Xu, Luis Artieda-Moncada, Tsai-Ching Lu, Lalindra De Silva, and Michael Macy. Using publicly visible social media to build detailed forecasts of civil unrest. Security Informatics, 3(1):1--10, 2014.

[26]

Wang Chong, David Blei, and Fei-Fei Li. Simultaneous image classification and annotation. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 1903--1910. IEEE, 2009.

[27]

Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1):183--202, 2009.

Digital Library

[28]

MITRE. https://www.mitre.org/, September 2016.

[29]

Naren Ramakrishnan, Patrick Butler, Sathappan Muthiah, Nathan Self, Rupinder Khandpur, and Saraf et. al. 'beating the news' with embers: Forecasting civil unrest using open source indicators. KDD '14, pages 1799--1808, New York, NY, USA, 2014. ACM.

Digital Library

[30]

Joseph O Ogutu, Torben Schulz-Streeck, and Hans-Peter Piepho. Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. In BMC proceedings, volume 6, page 1. BioMed Central, 2012.

Cited By

R. Ramya Dr. S. Kannan (2022)Tracking Challenges in Online Social Environment using Deep Learning TechniquesInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT228121(162-170)Online publication date: 1-Feb-2022
https://doi.org/10.32628/CSEIT228121
Li ZYan HZhang CTsung F(2020)Long-Short Term Spatiotemporal Tensor Prediction for Passenger Flow ProfileIEEE Robotics and Automation Letters10.1109/LRA.2020.3004785(1-1)Online publication date: 2020
https://doi.org/10.1109/LRA.2020.3004785
Fu KJi TZhao LLu C(2019)TITANProceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/3347146.3359381(329-338)Online publication date: 5-Nov-2019
https://dl.acm.org/doi/10.1145/3347146.3359381
Show More Cited By

Index Terms

Spatiotemporal Event Forecasting from Incomplete Hyper-local Price Data
1. Information systems
  1. Information systems applications
    1. Spatial-temporal systems
      1. Data streaming
      2. Location based services

Recommendations

Hierarchical Incomplete Multi-source Feature Learning for Spatiotemporal Event Forecasting
KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Forecasting significant societal events is an interesting and challenging problem as it taking into consideration multiple aspects of a society, including its economics, politics, and culture. Traditional forecasting methods based on a single data source ...
Severe Hail Prediction within a Spatiotemporal Relational Data Mining Framework
ICDMW '13: Proceedings of the 2013 IEEE 13th International Conference on Data Mining Workshops

Severe hail, or spherical ice precipitation over 1 inch in diameter, has caused billions of dollars in damage to crops, buildings, automobiles, and aircraft. Accurate predictions of severe hail with enough lead time can allow people to mitigate some ...
Civil Unrest Event Forecasting Using Graphical and Sequential Neural Networks
Artificial Neural Networks and Machine Learning – ICANN 2021
Abstract
Having the ability to forecast civil unrest events, such as violent protests, is crucial because they can lead to severe violent conflict and social instabilities. Civil unrests are comprehensive consequences of multiple factors, which could be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

November 2017

2604 pages

ISBN:9781450349185

DOI:10.1145/3132847

General Chairs:
Ee-Peng Lim
Singapore Management University, Singapore
,
Marianne Winslett
University of Illinois at Urbana-Champaign, USA, and Advanced Digital Sciences Center, Singapore
,
Program Chairs:
Mark Sanderson
RMIT, Australia
,
Ada Fu
Chinese University of Hong Kong, Hong Kong
,
Jimeng Sun
Georgia Tech, USA
,
Shane Culpepper
RMIT, Australia
,
Eric Lo
Chinese University of Hong Kong, Hong Kong
,
Joyce Ho
Emory University, USA
,
Debora Donato
Mix Tech, Inc., USA
,
Rakesh Agrawal
Data Insights Laboratories, USA
,
Yu Zheng
Microsoft Research Asia, China
,
Carlos Castillo
Qatar Computing Research Institute, Qatar
,
Aixin Sun
Nanyang Technological University, Singapore
,
Vincent S. Tseng
National Cheng Kung University, Taiwan
,
Chenliang Li
Wuhan University, China

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Department of Interior National Business Center

Conference

CIKM '17

Sponsor:

CIKM '17: ACM Conference on Information and Knowledge Management

November 6 - 10, 2017

Singapore, Singapore

Acceptance Rates

CIKM '17 Paper Acceptance Rate 171 of 855 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
235
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

R. Ramya Dr. S. Kannan (2022)Tracking Challenges in Online Social Environment using Deep Learning TechniquesInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT228121(162-170)Online publication date: 1-Feb-2022
https://doi.org/10.32628/CSEIT228121
Li ZYan HZhang CTsung F(2020)Long-Short Term Spatiotemporal Tensor Prediction for Passenger Flow ProfileIEEE Robotics and Automation Letters10.1109/LRA.2020.3004785(1-1)Online publication date: 2020
https://doi.org/10.1109/LRA.2020.3004785
Fu KJi TZhao LLu C(2019)TITANProceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/3347146.3359381(329-338)Online publication date: 5-Nov-2019
https://dl.acm.org/doi/10.1145/3347146.3359381
Huang CZhang CZhao JWu XYin DChawla N(2019)MiST: A Multiview and Multimodal Spatial-Temporal Learning Framework for Citywide Abnormal Event ForecastingThe World Wide Web Conference10.1145/3308558.3313730(717-728)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313730
Wu XShi BDong YHuang CChawla NCulpepper JMoffat ABennett PLerman K(2019)Neural Tensor Factorization for Temporal Interaction LearningProceedings of the Twelfth ACM International Conference on Web Search and Data Mining10.1145/3289600.3290998(537-545)Online publication date: 30-Jan-2019
https://dl.acm.org/doi/10.1145/3289600.3290998
Li RJiang JJu CWang WCulpepper JMoffat ABennett PLerman K(2019)CORALSProceedings of the Twelfth ACM International Conference on Web Search and Data Mining10.1145/3289600.3290995(69-77)Online publication date: 30-Jan-2019
https://dl.acm.org/doi/10.1145/3289600.3290995
Zhang XZhao LChen ZLu C(2018)Distributed self-paced learning in alternating direction method of multipliersProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304889.3305098(3148-3154)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304889.3305098
Wu XShi BDong YHuang CFaust LChawla NCuzzocrea AAllan JPaton NSrivastava DAgrawal RBroder AZaki MCandan SLabrinidis ASchuster AWang H(2018)RESTFulProceedings of the 27th ACM International Conference on Information and Knowledge Management10.1145/3269206.3271794(1073-1082)Online publication date: 17-Oct-2018
https://dl.acm.org/doi/10.1145/3269206.3271794
Huang CZhang JZheng YChawla NCuzzocrea AAllan JPaton NSrivastava DAgrawal RBroder AZaki MCandan SLabrinidis ASchuster AWang H(2018)DeepCrimeProceedings of the 27th ACM International Conference on Information and Knowledge Management10.1145/3269206.3271793(1423-1432)Online publication date: 17-Oct-2018
https://dl.acm.org/doi/10.1145/3269206.3271793
Zhang XLei SZhao LBoedihardjo ALu C(2018)Robust Regression via Online Feature Selection Under Adversarial Data Corruption2018 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM.2018.00199(1440-1445)Online publication date: Nov-2018
https://doi.org/10.1109/ICDM.2018.00199
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten