Abstract
To protect privacy of individuals or companies that may be leaked in street view images, we present a system to automatically detect and remove cars as if they had never been there. Although street view service providers have made efforts on blurring human faces and license plates, we argue that remaining features, such as license numbers and phone numbers printed on car bodies, could cause privacy leak. Given a sequence of street view images, this system first detects cars by the deformable part model, and then determines foreground/background seeds for the GrabCut image segmentation module in order to facilitate automatic car segmentation. After removing cars, an exemplar-based inpainting method is developed with special designs on filling priority determination and road structure propagation. Hierarchical texture propagation and randomized texture propagation are integrated to implement the inpainting process, so that aesthetically pleasing inpainting results as well as privacy protection can be accomplished.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
“Front,” “Back,” “Left,” and “Right” mean street view images captured by the cameras facing forward, backward, leftward, and rightward, respectively.
References
Agarwal S, Awan A, Roth D (2004) Learning to detect objects in images via a sparse. Part-based representation. IEEE Trans PAMI 26(11):1475–1490
Barnes C, Shechtman E, Finkelestein A, Goldman DB (2009) PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graph 28(3)
Boykov YY, Jolly M-P (2001) Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proc. of ICCV, p 105–112
Criminisi A, Perez P, Toyama K (2004) Region filling and object removal by examplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proc. of CVPR, p 886–893
Di Zenzo S (1986) A note on the gradient of a multi-image. Comp Vision Graph Image Process 33(1):116–125
Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans PAMI 32(9):1627–1645
Felzenszwalb P, Girshick R, McAllester D (2010) Cascade object detection with deformable part models. In: Proc. of CVPR, p 2241–2248
Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181
Flores A, Belongie S (2010) Removing pedestrians from Google Street view images. In: Proc. of CVPR, p 53–58
Frome A, Cheung G, Adbulkader A, Zennaro M, Wu B, Bissacco A, Adam H, Neven H, Vincent L (2009) Large-scale privacy protection in Google Street View. In: Proc. of ICCV, p 2373–2380
Guy R, Truong K (2012) CrossingGuard: exploring information content in navigation aids for visually impaired pedestrians. In: Proc. of CHI, p 405–414
He K, Sun J (2012) Statistics of patch offsets for image completion. In: Proc. of ECCV, p 16–29
Hota RN, Jonna K, Krishna PR (2010) On-road vehicle detection by cascaded classifiers. In: Proc. of the Third Annual ACM Bangalore Conference
Huang J-B, King SB, Ahuja N, Kopf J (2014) Image completion using planar structure guidance. In: Proc. of ACM SIGGRAPH
Komodakis N, Tziritas G (2007) Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans Image Process 16(11):2649–2661
Kopf J, Chen B, Szeliski R, Cohen M (2010) Street slide: browsing street level imagery. ACM Trans Graph 29(4)
Kuo C-H, Nevatia R (2009) Robust multi-view car detection using unsupervised sub-categorization. In: Proc. of Workshop on Applications on Computer Vision
Le Meur O, Gautier J, Guillemot C (2011) Examplar-based inpainting based on local geometry. In: Proc. of ICIP, p 3401–3404
Li Y, Sun J, Tang C-K, Shum H-Y (2004) Lazy snapping. In: Proc. of ACM SIGGRAPH, p 303–308
Rother C, Kolmogorov V, Blake A (2004) Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans PAMI 22(8):888–905
Telea A (2004) An image inpainting techniques based on the fast matching method. J Graph Tools 9(1):25–36
Tschumperle D, Deriche R (2005) Vector-valued image regularization with PDEs: a common framework for different applications. IEEE Trans PAMI 27(4):206–517
Vincent L (2007) Taking online maps down to street level. Computer 40(12):118–120
Weir J, Yan W (2010) Resolution variant visual cryptography for street view of Google Maps. In: Proc. of ISCAS, p 1695–1698
Yoshimoto Y, Dang TH, Kimura A, Shibata F, Tamura H (2011) Interaction design of 2D/3D map navigation on wall and tabletop displays. In: Proc. of ACM International Conference on Interactive Tabletops and Surfaces, p 254–255
Zheng W, Liang L (2009) Fast car detection using image strip features. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, p 2703–2710
Acknowledgments
The work was partially supported by the National Science Council of Taiwan under the grants NSC101-2221-E-194-055-MY2.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chu, WT., Chao, YC. & Chang, YS. Street sweeper: detecting and removing cars in street view images. Multimed Tools Appl 74, 10965–10988 (2015). https://doi.org/10.1007/s11042-014-2213-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-014-2213-x