skip to main content
10.1145/3343031.3351077acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

See Through the Windshield from Surveillance Camera

Published: 15 October 2019 Publication History

Abstract

This paper attempts to address the challenging task of seeing through the windshield images captured by surveillance cameras in the wild. Such images usually have very low visibility due to heterogeneous degradations caused by blur, haze, reflection, noise etc., which makes existing image enhancing methods inapplicable. We propose a windshield image restoration generative adversarial network (WIRE-GAN) to restore and enhance the visibility of windshield images. We adopt the weakly supervised framework based on the generative model, which has effectively released the request of paired training data for a specific type of degradation. To generate more semantically consistent results even in extreme lighting conditions, we introduce a novel content-preserving strategy into the proposed weakly-supervised framework. To make the image restoration more reliable, the WIRE-GAN network constructs a sort of content-aware embedding space and enforces the constraint of the restored windshield images being closer to the original input in the embedding space. Moreover, we collect a large-scale windshield image dataset (WIRE dataset) to validate the advantage of our method in improving the image quality, and further evaluate the impact of windshield restoration on the vehicle ReID performance.

References

[1]
R. M. Anwer, F. S. Khan, J. van de Weijer, and J. Laaksonen. Combining holistic and part-based deep representations for computational painting categorization. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, pages 339--342. ACM, 2016.
[2]
R. S. Arora and A. Elgammal. Towards automated classification of fine-art painting style: A comparative study. In Pattern Recognition (ICPR), 2012 21st International Conference on, pages 3541--3544. IEEE, 2012.
[3]
W.-T. Chu and Y.-L. Wu. Deep correlation features for image style classification. In Proceedings of the 2016 ACM on Multimedia Conference, pages 402--406. ACM, 2016.
[4]
Y. Fu, L. Cao, G. Guo, and T. S. Huang. Multiple feature fusion by subspace learning. In Proceedings of the 2008 international conference on Content-based image and video retrieval, pages 127--134. ACM, 2008.
[5]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770--778, 2016.
[6]
K. A. Jangtjik, M.-C. Yeh, and K.-L. Hua. Artist-based classification via deep learning with multi-scale weighted pooling. In Proceedings of the 2016 ACM on Multimedia Conference, pages 635--639. ACM, 2016.
[7]
J. Li, L. Yao, E. Hendriks, and J. Z. Wang. Rhythmic brushstrokes distinguish van gogh from his contemporaries: findings via automated brushstroke extraction. IEEE transactions on pattern analysis and machine intelligence, 34(6):1159--1176, 2012.
[8]
W. Liu, Y. Wen, Z. Yu, and M. Yang. Large-margin softmax loss for convolutional neural networks. In Proceedings of The 33rd International Conference on Machine Learning, pages 507--516, 2016.
[9]
D. G. Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91--110, 2004.
[10]
L. v. d. Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579--2605, 2008.
[11]
T. Mensink and J. Van Gemert. The rijksmuseum challenge: Museum-centered visual recognition. In Proceedings of International Conference on Multimedia Retrieval, page 451. ACM, 2014.
[12]
A. Puthenputhussery, Q. Liu, and C. Liu. Color multi-fusion fisher vector feature for fine art painting categorization and influence analysis. In Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on, pages 1--9. IEEE, 2016.
[13]
A. Puthenputhussery, Q. Liu, and C. Liu. Sparse representation based complete kernel marginal fisher analysis framework for computational art painting categorization. In European Conference on Computer Vision, pages 612--627. Springer, 2016.
[14]
B. Saleh and A. Elgammal. A unified framework for painting classification. In Data Mining Workshop (ICDMW), 2015 IEEE International Conference on, pages 1254--1261. IEEE, 2015.
[15]
J. Sánchez, F. Perronnin, T. Mensink, and J. Verbeek. Image classification with the fisher vector: Theory and practice. International journal of computer vision, 105(3):222--245, 2013.
[16]
L. Shamir, T. Macura, N. Orlov, D. M. Eckley, and I. G. Goldberg. Impressionism, expressionism, surrealism: Automated recognition of painters and schools of art. ACM Transactions on Applied Perception (TAP), 7(2):8, 2010.
[17]
M. Simon, E. Rodner, and J. Denzler. Imagenet pre-trained models with batch normalization. arXiv preprint arXiv:1612.01452, 2016.
[18]
Q.-S. Sun, S.-G. Zeng, Y. Liu, P.-A. Heng, and D.-S. Xia. A new method of feature fusion and its application in image recognition. Pattern Recognition, 38(12):2437--2448, 2005.
[19]
W. R. Tan, C. S. Chan, H. E. Aguirre, and K. Tanaka. Ceci n'est pas une pipe: A deep convolutional network for fine-art paintings classification. In Image Processing (ICIP), 2016 IEEE International Conference on, pages 3703--3707. IEEE, 2016.
[20]
E. Tola, V. Lepetit, and P. Fua. Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE transactions on pattern analysis and machine intelligence, 32(5):815--830, 2010.
[21]
N. van Noord, E. Hendriks, and E. Postma. Toward discovery of the artist's style: Learning to recognize artists by their artworks. IEEE Signal Processing Magazine, 32(4):46--54, 2015.
[22]
N. van Noord and E. Postma. Learning scale-variant and scale-invariant features for deep image classification. Pattern Recognition, 61:583--592, 2017.
[23]
A. Vedaldi and B. Fulkerson. VLFeat: An open and portable library of computer vision algorithms, 2008.
[24]
M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818--833. Springer, 2014.

Cited By

View all
  • (2022)Single Image Reflection Removal Based on Knowledge-Distilling Content DisentanglementIEEE Signal Processing Letters10.1109/LSP.2022.314866829(568-572)Online publication date: 2022
  • (2021)Detecting Number of Passengers in a Moving Vehicle with Publicly Available DataIntelligent Systems and Applications10.1007/978-3-030-82196-8_39(536-548)Online publication date: 3-Aug-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '19: Proceedings of the 27th ACM International Conference on Multimedia
October 2019
2794 pages
ISBN:9781450368896
DOI:10.1145/3343031
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. generative adversarial network
  2. heterogeneous degradations
  3. windshield restoration

Qualifiers

  • Research-article

Funding Sources

Conference

MM '19
Sponsor:

Acceptance Rates

MM '19 Paper Acceptance Rate 252 of 936 submissions, 27%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Single Image Reflection Removal Based on Knowledge-Distilling Content DisentanglementIEEE Signal Processing Letters10.1109/LSP.2022.314866829(568-572)Online publication date: 2022
  • (2021)Detecting Number of Passengers in a Moving Vehicle with Publicly Available DataIntelligent Systems and Applications10.1007/978-3-030-82196-8_39(536-548)Online publication date: 3-Aug-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media