research-article

StreetNet: preference learning with convolutional neural network on urban crime perception

Authors:

Chang-Tien LuAuthors Info & Claims

SIGSPATIAL '18: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Pages 269 - 278

https://doi.org/10.1145/3274895.3274975

Published: 06 November 2018 Publication History

Abstract

One can infer from the broken window theory that the perception of a city street's safety level relies significantly on the visual appearance of the street. Previous works have addressed the feasibility of using computer vision algorithms to classify urban scenes. Most of the existing urban perception predictions focus on binary outcomes such as safe or dangerous, wealthy or poor. However, binary predictions are not representative and cannot provide informative inferences such as the potential crime types in certain areas. In this paper, we explore the connection between urban perception and crime inferences. We propose a convolutional neural network (CNN) - StreetNet to learn crime rankings from street view images. The learning process is formulated on the basis of preference learning and label ranking settings. We design a street view images retrieval algorithm to improve the representation of urban perception. A data-driven, spatiotemporal algorithm is proposed to find unbiased label mappings between the street view images and the crime ranking records. Extensive evaluations conducted on images from different cities and comparisons with baselines demonstrate the effectiveness of our proposed method.

References

[1]

Virginia O Andersson, Marco AF Birck, and Ricardo M Araujo. 2017. Investigating Crime Rate Prediction Using Street-Level Images and Siamese Convolutional Neural Networks. In Latin American Workshop on Computational Neuroscience. Springer, 81--93.

[2]

Sean M Arietta, Alexei A Efros, Ravi Ramamoorthi, and Maneesh Agrawala. 2014. City forensics: Using visual elements to predict non-visual city attributes. IEEE transactions on visualization and computer graphics 20, 12 (2014), 2624--2633.

[3]

Wim Bernasco. 2010. Modeling micro-level crime location choice: Application of the discrete choice framework to crime at places. Journal of Quantitative Criminology 26, 1 (2010), 113--138.

[4]

Wim Bernasco. 2010. A sentimental journey to crime: Effects of residential history on crime location choice. Criminology 48, 2 (2010), 389--416.

[5]

Anna Bosch, Andrew Zisserman, and Xavier Muñoz. 2006. Scene classification via pLSA. In European conference on computer vision. Springer, 517--530.

Digital Library

[6]

Gang Chen, Yangqiu Song, Fei Wang, and Changshui Zhang. 2008. Semi-supervised multi-label learning by solving a sylvester equation. In Proceedings of the 2008 SIAM International Conference on Data Mining. SIAM, 410--419.

[7]

Nico Cornelis, Bastian Leibe, Kurt Cornelis, and Luc Van Gool. 2008. 3d urban scene modeling integrating recognition and reconstruction. International Journal of Computer Vision 78, 2-3 (2008), 121--141.

Digital Library

[8]

Gabriella Csurka, Christopher Dance, Lixin Fan, Jutta Willamowski, and Cédric Bray. 2004. Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, Vol. 1. Prague, 1--2.

[9]

Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 1. IEEE, 886--893.

Digital Library

[10]

Marco De Nadai, Radu Laurentiu Vieriu, Gloria Zen, Stefan Dragicevic, Nikhil Naik, Michele Caraviello, Cesar Augusto Hidalgo, Nicu Sebe, and Bruno Lepri. 2016. Are safer looking neighborhoods more lively?: A multimodal investigation into urban life. In Proceedings of the 2016 ACM on Multimedia Conference. ACM, 1127--1135.

Digital Library

[11]

Thomas Deselaers, Lexi Pimenidis, and Hermann Ney. 2008. Bag-of-visual-words models for adult image classification and filtering. In Pattern Recognition, 2008. ICPR 2008. 19th International Conference on. IEEE, 1--4.

[12]

Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, and César A Hidalgo. 2016. Deep learning the city: Quantifying urban perception at a global scale. In European Conference on Computer Vision. Springer, 196--212.

[13]

Kaiqun Fu, Chang-Tien Lu, Rakesh Nune, and Jason Xianding Tao. 2015. Steds: Social Media Based Transportation Event Detection with Text Summarization. In ITSC. 1952--1957.

Digital Library

[14]

Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieberman Aiden, and Li Fei-Fei. 2017. Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences (2017), 201700035.

[15]

Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, and Li Fei-Fei. 2017. Fine-Grained Car Detection for Visual Census Estimation. In AAAI, Vol. 2. 6.

[16]

Edward L Glaeser, Scott Duke Kominers, Michael Luca, and Nikhil Naik. 2018. Big data and big cities: The promises and limitations of improved measures of urban life. Economic Inquiry 56, 1 (2018), 114--137.

[17]

Ian J Goodfellow, Yaroslav Bulatov, Julian Ibarz, Sacha Arnoud, and Vinay Shet. 2013. Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv:1312.6082 (2013).

[18]

Asaad Hakeem, Roberto Vezzani, Mubarak Shah, and Rita Cucchiara. 2006. Estimating geospatial trajectory of a moving camera. In 18th International Conference on Pattern Recognition (ICPR'06), Vol. 2. IEEE, 82--87.

Digital Library

[19]

Sariel Har-Peled, Dan Roth, and Dav Zimak. 2002. Constraint classification for multiclass classification and ranking. Urbana 51 (2002), 61801.

[20]

Taoran Ji, Kaiqun Fu, Nathan Self, Chang-Tien Lu, and Naren Ramakrishnan. 2018. Multi-task Learning for Transit Service Disruption Detection. In IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM'18).

[21]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.

Digital Library

[22]

Ickjai Lee and Peter Phillips. 2008. Urban crime analysis through areal categorized multivariate associations mining. Applied Artificial Intelligence 22, 5 (2008), 483-- 499.

Digital Library

[23]

Xirong Li, Cees GM Snoek, and Marcel Worring. 2010. Unsupervised multi-feature tag relevance learning for social image retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval. ACM, 10--17.

Digital Library

[24]

Dong Liu, Xian-Sheng Hua, Linjun Yang, Meng Wang, and Hong-Jiang Zhang. 2009. Tag ranking. In Proceedings of the 18th international conference on World wide web. ACM, 351--360.

Digital Library

[25]

Meiling Liu, Kaiqun Fu, Chang-Tien Lu, Guangsheng Chen, and Huiqiang Wang. 2014. A search and summary application for traffic events detection based on twitter data. In Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 549--552.

Digital Library

[26]

Xiaobai Liu, Qi Chen, Lei Zhu, Yuanlu Xu, and Liang Lin. 2017. Place-centric Visual Urban Perception with Deep Multi-instance Regression. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 19--27.

Digital Library

[27]

David G Lowe. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60, 2 (2004), 91--110.

Digital Library

[28]

Yong Luo, Dacheng Tao, Chang Xu, Dongchen Li, and Chao Xu. 2013. Vector-Valued Multi-View Semi-Supervsed Learning for Multi-Label Image Classification. In 27th AAAI Conference on Artificial Intelligence (AAAI). AAAI, 647--653.

Digital Library

[29]

Branislav Micusik and Jana Kosecka. 2009. Piecewise planar city 3D modeling from street view panoramic sequences. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2906--2912.

[30]

Jeffrey D Morenoff and Robert J Sampson. 1997. Violent crime and the spatial dynamics of neighborhood transition: Chicago, 1970--1990. Social forces 76, 1 (1997), 31--64.

[31]

Yair Movshovitz-Attias, Qian Yu, Martin C Stumpe, Vinay Shet, Sacha Arnoud, and Liron Yatziv. 2015. Ontological supervision for fine grained classification of street view storefronts. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1693--1702.

[32]

Nikhil Naik, Jade Philipoom, Ramesh Raskar, and César Hidalgo. 2014. Streetscore-Predicting the Perceived Safety of One Million Streetscapes. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 793--799.

Digital Library

[33]

Nikhil Naik, Ramesh Raskar, and César A Hidalgo. 2016. Cities are physical too: Using computer vision to measure the quality and impact of urban appearance. American Economic Review 106, 5 (2016), 128--32.

[34]

Vicente Ordonez and Tamara L Berg. 2014. Learning high-level judgments of urban perception. In European Conference on Computer Vision. Springer, 494--510.

[35]

Genevieve Patterson, Chen Xu, Hang Su, and James Hays. 2014. The sun attribute database: Beyond categories for deeper scene understanding. International Journal of Computer Vision 108, 1-2 (2014), 59--81.

Digital Library

[36]

Peter Phillips and Ickjai Lee. 2011. Crime analysis through spatial areal aggregated density patterns. Geoinformatica 15, 1 (2011), 49--74.

Digital Library

[37]

Peter Phillips and Ickjai Lee. 2012. Mining co-distribution patterns for large crime datasets. Expert Systems with Applications 39, 14 (2012), 11556--11563.

Digital Library

[38]

Lorenzo Porzi, Samuel Rota Bulò, Bruno Lepri, and Elisa Ricci. 2015. Predicting and understanding urban perception with convolutional neural networks. In Proceedings of the 23rd ACM international conference on Multimedia. ACM, 139-- 148.

Digital Library

[39]

Grant Schindler, Matthew Brown, and Richard Szeliski. 2007. City-scale location recognition. In 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1--7.

[40]

Herbert W Schroeder and LM Anderson. 1984. Perception of personal safety in urban recreation sites. Journal of Leisure Research 16, 2 (1984), 178.

[41]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[42]

James Q Wilson and George L Kelling. 1982. Broken windows. Critical issues in policing: Contemporary readings (1982), 395--407.

[43]

Jun Yang, Yu-Gang Jiang, Alexander G Hauptmann, and Chong-Wah Ngo. 2007. Evaluating bag-of-visual-words representations in scene classification. In Proceedings of the international workshop on Workshop on multimedia information retrieval. ACM, 197--206.

Digital Library

[44]

Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2016. Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark. In 30th AAAI Conference on Artificial Intelligence (AAAI). AAAI, 308--314.

Digital Library

[45]

Amir Roshan Zamir and Mubarak Shah. 2010. Accurate image localization based on google maps street view. In European Conference on Computer Vision. Springer, 255--268.

Digital Library

[46]

Amir Roshan Zamir and Mubarak Shah. 2014. Image geo-localization based on multiplenearest neighbor feature matching usinggeneralized graphs. IEEE transactions on pattern analysis and machine intelligence 36, 8 (2014), 1546--1558.

Digital Library

[47]

Matthew D Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012).

[48]

Man Zhang, Ran He, Dong Cao, Zhenan Sun, and Tieniu Tan. 2016. Simultaneous Feature and Sample Reduction for Image-Set Classification. In 30th AAAI Conference on Artificial Intelligence (AAAI). AAAI, 1401--1407.

Digital Library

[49]

Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2014. Learning deep features for scene recognition using places database. In Advances in neural information processing systems. 487--495.

Digital Library

Cited By

Ren QSun M(2025)Predicting the spatial demand for public charging stations for EVs using multi-source big data: an example from jinan city, chinaScientific Reports10.1038/s41598-025-91106-915:1Online publication date: 27-Feb-2025
https://doi.org/10.1038/s41598-025-91106-9
Taha K(2024)Empirical and Experimental Insights into Data Mining Techniques for Crime Prediction: A Comprehensive SurveyACM Transactions on Intelligent Systems and Technology10.1145/369951516:2(1-75)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3699515
Das DChakraborty SMitra B(2024)DriveR: Towards Generating a Dynamic Road Safety Map with Causal ContextsProceedings of the ACM on Human-Computer Interaction10.1145/36764988:MHCI(1-35)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676498
Show More Cited By

Index Terms

StreetNet: preference learning with convolutional neural network on urban crime perception
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Perception
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Learning to rank
2. Information systems
  1. Information systems applications
    1. Data mining
    2. Spatial-temporal systems
      1. Geographic information systems

Recommendations

Quantifying Urban Safety Perception on Street View Images
WI-IAT '21: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology

In the last 40 years, Urban perception has become an important research area covering several fields, such as criminology, psychology, urban planning, Broken windows theory. It aims to analyze and interpret the behavior of the perception in cities. ...
Predicting and Understanding Urban Perception with Convolutional Neural Networks
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Cities' visual appearance plays a central role in shaping human perception and response to the surrounding urban environment. For example, the visual qualities of urban spaces affect the psychological states of their inhabitants and can induce negative ...
Understanding Safety Based on Urban Perception
Intelligent Computing Theories and Application
Abstract
Currently, one important field on machine learning is Urban Perception Computing is to model the way in which humans can interact and understand the environment that surrounds them. This process is performed using convolutional models to learn and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '18: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2018

655 pages

ISBN:9781450358897

DOI:10.1145/3274895

General Chairs:
Farnoush Banaei-Kashani
University of Colorado, Denver
,
Erik Hoel
Esri
,
Program Chairs:
Ralf Hartmut Güting
FernUniversität in Hagen, Germany
,
Roberto Tamassia
Brown University
,
Li Xiong
Emory University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGSPATIAL '18

Sponsor:

SIGSPATIAL

SIGSPATIAL '18: 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 6 - 9, 2018

Washington, Seattle

Acceptance Rates

SIGSPATIAL '18 Paper Acceptance Rate 30 of 150 submissions, 20%;

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
467
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)12

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ren QSun M(2025)Predicting the spatial demand for public charging stations for EVs using multi-source big data: an example from jinan city, chinaScientific Reports10.1038/s41598-025-91106-915:1Online publication date: 27-Feb-2025
https://doi.org/10.1038/s41598-025-91106-9
Taha K(2024)Empirical and Experimental Insights into Data Mining Techniques for Crime Prediction: A Comprehensive SurveyACM Transactions on Intelligent Systems and Technology10.1145/369951516:2(1-75)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3699515
Das DChakraborty SMitra B(2024)DriveR: Towards Generating a Dynamic Road Safety Map with Causal ContextsProceedings of the ACM on Human-Computer Interaction10.1145/36764988:MHCI(1-35)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676498
Mokbel MSakr MXiong LZüfle AAlmeida JAnderson TAref WAndrienko GAndrienko NCao YChawla SCheng RChrysanthis PFei XGhinita GGraser AGunopulos DJensen CKim JKim KKröger PKrumm JLauer JMagdy ANascimento MRavada SRenz MSacharidis DSalim FSarwat MSchoemans MShahabi CSpeckmann BTanin ETeng XTheodoridis YTorp KTrajcevski Gvan Kreveld MWenk CWerner MWong RWu SXu JYoussef MZeinalipour DZhang MZimányi E(2024)Mobility Data Science: Perspectives and ChallengesACM Transactions on Spatial Algorithms and Systems10.1145/365215810:2(1-35)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3652158
Alpherts TGhebreab SHsu YVan Noord N(2024)Perceptive Visual Urban Analytics is Not (Yet) Suitable for MunicipalitiesProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658976(1341-1354)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658976
Park JPark YCheong ILee JHuh YJeon H(2024)What Makes Deviant Places?IEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.339340846:11(7405-7420)Online publication date: Nov-2024
https://doi.org/10.1109/TPAMI.2024.3393408
Yang SKrenz KQiu WLi W(2023)The Role of Subjective Perceptions and Objective Measurements of the Urban Environment in Explaining House Prices in Greater London: A Multi-Scale Urban Morphology AnalysisISPRS International Journal of Geo-Information10.3390/ijgi1206024912:6(249)Online publication date: 19-Jun-2023
https://doi.org/10.3390/ijgi12060249
Das DChakraborty SMitra B(2023)DriCon: On-device Just-in-Time Context Characterization for Unexpected Driving Events2023 IEEE International Conference on Pervasive Computing and Communications (PerCom)10.1109/PERCOM56429.2023.10099058(12-21)Online publication date: 13-Mar-2023
https://doi.org/10.1109/PERCOM56429.2023.10099058
Payandeh ABaghaei KFayyazsanavi PRamezani SChen ZRahimi S(2023)Deep Representation Learning: Fundamentals, Technologies, Applications, and Open ChallengesIEEE Access10.1109/ACCESS.2023.333519611(137621-137659)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3335196
Jenga KCatal CKar G(2023)Machine learning in crime predictionJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-023-04530-y14:3(2887-2913)Online publication date: 2-Feb-2023
https://doi.org/10.1007/s12652-023-04530-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten