skip to main content
research-article

Customer Volume Prediction Using Fusion of Shared-private Dynamic Weighting over Multiple Modalities

Published: 24 March 2023 Publication History

Abstract

Customer volume prediction is crucial for a variety of urban applications, such as store location selection. So far, the key challenge lies in how to fuse multiple modalities from different data sources, on account of the massive amount of data accessible, for example, spatio-temporal data and satellite images. In this article, we investigate three dynamic weighting ensemble learning models to fuse spatio-temporal features and visual features for predicting customer volume in the urban commercial district of interest. Specifically, we propose the shared-private dynamic weighting model by incorporating graph neural networks, which is proposed to capture geographic dependencies (i.e., competitiveness or dependencies) between urban commercial districts in an end-to-end manner. To the best of our knowledge, it is the first work to utilize graph neural networks to model such geographic relationships. We conduct a series of experiments to demonstrate the effectiveness of the proposed models based on two real datasets. Furthermore, an elaborated visualization method is performed for knowledge discovery.

References

[1]
Cătălina Cangea, Petar Veličković, Nikola Jovanović, Thomas Kipf, and Pietro Liò. 2018. Towards Sparse Hierarchical Graph Classifiers. (2018). arxiv:stat.ML/1811.01287.
[2]
Jingmin Chen, Su Yang, Wenshan Wang, and Minjie Wang. 2015. Social context awareness from taxi traces: Mining how human mobility patterns are shaped by bags of POI. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (UbiComp/ISWC’15 Adjunct). Association for Computing Machinery, New York, NY, 97–100.
[3]
Longbiao Chen, Xiaoliang Fan, Leye Wang, Daqing Zhang, Zhiyong Yu, Jonathan Li, Thi-Mai-Trang Nguyen, Gang Pan, and Cheng Wang. 2018. RADAR: Road obstacle identification for disaster response leveraging cross-domain urban data. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 130 (Jan. 2018), 23 pages.
[4]
Longbiao Chen, Daqing Zhang, Gang Pan, Xiaojuan Ma, Dingqi Yang, Kostadin Kushlev, Wangsheng Zhang, and Shijian Li. 2015. Bike sharing station placement leveraging heterogeneous urban open data. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’15). ACM, New York, NY, 571–575.
[5]
Marco De Nadai, Radu Laurentiu Vieriu, Gloria Zen, Stefan Dragicevic, Nikhil Naik, Michele Caraviello, Cesar Augusto Hidalgo, Nicu Sebe, and Bruno Lepri. 2016. Are safer looking neighborhoods more lively? A multimodal investigation into urban life. In Proceedings of the 24th ACM International Conference on Multimedia (MM’16). ACM, New York, NY, 1127–1135.
[6]
Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, and César A. Hidalgo. 2016. Deep learning the city: Quantifying urban perception at a global scale. In Computer Vision (ECCV’16), Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 196–212.
[7]
Shen Fang, Qi Zhang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. 2019. GSTNet: Global spatial-temporal network for traffic flow prediction. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). International Joint Conferences on Artificial Intelligence Organization, 2286–2293.
[8]
Jerome H. Friedman. 2000. Greedy function approximation: a gradient boosting machine. Annals of Statistics 29 (2000), 1189–1232.
[9]
Yanjie Fu, Hui Xiong, Yong Ge, Zijun Yao, Yu Zheng, and Zhi-Hua Zhou. 2014. Exploiting geographic dependencies for real estate appraisal: A mutual perspective of ranking and clustering. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’14). Association for Computing Machinery, New York, NY, 1047–1056.
[10]
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (ICML’17). JMLR.org, 1263–1272.
[11]
K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 770–778.
[12]
Zhiyuan He and Su Yang. 2018. Multi-view commercial hotness prediction using context-aware neural network ensemble. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 168 (Dec. 2018), 19 pages.
[13]
Danfeng Hong, Lianru Gao, Naoto Yokoya, Jing Yao, Jocelyn Chanussot, Qian Du, and Bing Zhang. 2021. More diverse means better: Multimodal deep learning meets remote-sensing imagery classification. IEEE Transactions on Geoscience and Remote Sensing 59, 5 (2021), 4340–4354.
[14]
Dmytro Karamshuk, Anastasios Noulas, Salvatore Scellato, Vincenzo Nicosia, and Cecilia Mascolo. 2013. Geo-spotting: Mining online location-based services for optimal retail store placement. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’13). Association for Computing Machinery, New York, NY, 793–801.
[15]
Dmytro Karamshuk, Anastasios Noulas, Salvatore Scellato, Vincenzo Nicosia, and Cecilia Mascolo. 2013. Geo-spotting: Mining online location-based services for optimal retail store placement. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’13). ACM, New York, NY, 793–801.
[16]
Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. CoRR abs/1609.02907 (2016). arxiv:1609.02907. http://arxiv.org/abs/1609.02907.
[17]
Huan Ma, Zongbo Han, Changqing Zhang, Huazhu Fu, Joey Tianyi Zhou, and Qinghua Hu. 2021. Trustworthy multimodal regression with mixture of normal-inverse gamma distributions. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 6881–6893. https://proceedings.neurips.cc/paper/2021/file/371bce7dc83817b7893bcdeed13799b5-Paper.pdf.
[18]
N. Naik, J. Philipoom, R. Raskar, and C. Hidalgo. 2014. Streetscore – Predicting the perceived safety of one million streetscapes. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops. 793–799.
[19]
Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. 2019. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery; Data Mining (KDD’19). Association for Computing Machinery, New York, NY, 1720–1730.
[20]
Yan Qu and Jun Zhang. 2013. Trade area analysis using user generated mobile location data. In Proceedings of the 22nd International Conference on World Wide Web (WWW’13). Association for Computing Machinery, New York, NY, 1053–1064.
[21]
Daniele Quercia, Neil Keith O’Hare, and Henriette Cramer. 2014. Aesthetic capital: What makes london look beautiful, quiet, and happy? In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work; Social Computing (CSCW’14). ACM, New York, NY, 945–955.
[22]
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (ICML’17). JMLR.org, 3319–3328. http://dl.acm.org/citation.cfm?id=3305890.3306024.
[23]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2017. Graph Attention Networks. (2017). arxiv:stat.ML/1710.10903.
[24]
Jingyuan Wang, Yating Lin, Junjie Wu, Zhong Wang, and Zhang Xiong. 2017. Coupling Implicit and Explicit Knowledge for Customer Volume Prediction. (2017). https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14440.
[25]
Wenshan Wang. 2019. Neural aesthetic image reviewer. IET Computer Vision 13, 8 (December 2019), 749–758(9).
[26]
Wenshan Wang, Su Yang, Zhiyuan He, Minjie Wang, Jiulong Zhang, and Weishan Zhang. 2018. Urban perception of commercial activeness from satellite images and streetscapes. In Companion Proceedings of the the Web Conference 2018 (WWW’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 647–654.
[27]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? CoRR abs/1810.00826 (2018). arxiv:1810.00826. http://arxiv.org/abs/1810.00826.
[28]
Mengwen Xu, Tianyi Wang, Zhengwei Wu, Jingbo Zhou, Jian Li, and Haishan Wu. 2016. Store location selection via mining search query logs of baidu maps. CoRR abs/1606.03662 (2016). arxiv:1606.03662. http://arxiv.org/abs/1606.03662.
[29]
Su Yang, Minjie Wang, Wenshan Wang, Yi Sun, Jun Gao, Weishan Zhang, and Jiulong Zhang. 2017. Predicting commercial activeness over urban big data. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 119 (Sept. 2017), 20 pages.
[30]
Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and Jure Leskovec. 2018. Hierarchical graph representation learning with differentiable pooling. CoRR abs/1806.08804 (2018). arxiv:1806.08804. http://arxiv.org/abs/1806.08804.
[31]
Junbo Zhang, Yu Zheng, Junkai Sun, and Dekang Qi. 2020. Flow prediction in spatio-temporal networks based on multitask deep learning. IEEE Transactions on Knowledge and Data Engineering 32, 3 (2020), 468–478.
[32]
Y. Zheng. 2015. Methodologies for cross-domain data fusion: An overview. IEEE Transactions on Big Data 1, 1 (March 2015), 16–34.

Cited By

View all
  • (2023)Basketball Footwork and Application Supported by Deep Learning Unsupervised Transfer MethodInternational Journal of Information Technology and Web Engineering10.4018/IJITWE.33436518:1(1-17)Online publication date: 1-Dec-2023

Index Terms

  1. Customer Volume Prediction Using Fusion of Shared-private Dynamic Weighting over Multiple Modalities

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Intelligent Systems and Technology
    ACM Transactions on Intelligent Systems and Technology  Volume 14, Issue 3
    June 2023
    451 pages
    ISSN:2157-6904
    EISSN:2157-6912
    DOI:10.1145/3587032
    • Editor:
    • Huan Liu
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 March 2023
    Online AM: 09 January 2023
    Accepted: 22 December 2022
    Revised: 01 October 2022
    Received: 07 November 2020
    Published in TIST Volume 14, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Multimodal fusion
    2. dynamic weighting
    3. ensemble learning

    Qualifiers

    • Research-article

    Funding Sources

    • State Grid Corporation of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)50
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 02 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Basketball Footwork and Application Supported by Deep Learning Unsupervised Transfer MethodInternational Journal of Information Technology and Web Engineering10.4018/IJITWE.33436518:1(1-17)Online publication date: 1-Dec-2023

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media