skip to main content
10.1145/3447548.3467116acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Automated Testing of Graphics Units by Deep-Learning Detection of Visual Anomalies

Published: 14 August 2021 Publication History

Abstract

We present a novel system for performing real-time detection of diverse visual corruptions in videos, for validating the quality of graphics units in our company. The system is used for several types of content, including movies and 3D graphics, with strict constraints on low false alert rates and real-time processing of millions of video frames per day. These constraints required novel solutions involving both hardware and software, including new supervised and weakly-supervised methods we developed. Our deployed system has enabled a ~20X reduction of human effort and discovering new corruptions missed by humans and existing approaches.

Supplementary Material

MP4 File (deep_learning_based_video_inspection-lev_faivishevsky-adi_szeskin-38958085-kRMn.mp4)
Presentation Video for the KDD 2021 paper "Automated Testing of Graphics Units by Deep-Learning Detection of Visual Anomalies"

References

[1]
Eran Avidan. 2018. Real-Time Deep Learning on Video Streams. https://www.youtube.com/watch?v=FvJNVFKZJO0 STRATA.
[2]
David Berthelot, Thomas Schumm, and Luke Metz. 2017. BEGAN: boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717.
[3]
Purnima Bholowalia and Arvind Kumar. 2014. EBK-means: A clustering technique based on elbow method and k-means in WSN. International Journal of Computer Applications, Vol. 105, 9 (2014).
[4]
Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018).
[5]
Tadeusz Cali'nski and Jerzy Harabasz. 1974. A dendrite method for cluster analysis. Communications in Statistics-theory and Methods, Vol. 3, 1 (1974), 1--27.
[6]
Damon M. Chandler and David J. Field. 2007. Estimates of the information content and dimensionality of natural scenes from proximity distributions. J. Opt. Soc. Am. A, Vol. 24, 4 (Apr 2007), 922--941. https://doi.org/10.1364/JOSAA.24.000922
[7]
Wikimedia Commons. 2017. A basic desktop environment of BOSS Linux 6.1 on a personal computer with menus above and a taskbar below. https://commons.wikimedia.org/wiki/File:Screenshot_of_BOSS_Linux_v6.1_Desktop_Environment.png
[8]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd, Vol. 96. 226--231.
[9]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. CoRR, Vol. abs/1512.03385.
[11]
Maximilian Ilse, Jakub Tomczak, and Max Welling. 2018. Attention-based Deep Multiple Instance Learning. In ICML. 2127--2136.
[12]
Ahmed Imran. 2016. Pre and Post Silicon Validation Automation with Graphics Behavior Checker. Int. J. of Hybrid Information Technology, Vol. 9, 6 (2016), 51--66.
[13]
Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-scale Video Classification with Convolutional Neural Networks. In CVPR.
[14]
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017).
[15]
Jongyoo Kim, Hui Zeng, Deepti Ghadiyaram, Sanghoon Lee, Lei Zhang, and Alan C. Bovik. 2017. Deep convolutional neural models for picture-quality prediction: Challenges and solutions to data-driven image quality assessment. IEEE Signal Processing Magazine, Vol. 34, 6, 130--141. https://doi.org/10.1109/MSP.2017.2736018
[16]
Diederik P Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. http://arxiv.org/abs/1312.6114 cite arxiv:1312.6114.
[17]
Dimitrios Kotzias, Misha Denil, Nando de Freitas, and Padhraic Smyth. 2015. From Group to Individual Labels Using Deep Features. In KDD. 597-606.
[18]
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation Forest. In ICDM. IEEE Computer Society, 413--422.
[19]
James MacQueen et al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Vol. 1. Oakland, CA, USA, 281--297.
[20]
T Soni Madhulatha. 2012. An overview on clustering methods. arXiv preprint arXiv:1205.1117 (2012).
[21]
Giuseppe Marbach, Markus Loepfe, and Thomas Brupbacher. 2006. An image processing technique for fire detection in video images. Fire Safety Journal, Vol. 41 (06 2006), 285--289. https://doi.org/10.1016/j.firesaf.2006.02.001
[22]
Ripal Patel, Shubha Pandey, Chirag I Patel, and Robinson Paul. 2017. Effective Flicker Detection Technique Using Artificial Neural Network For Video. International Conference on Research and Innovations in Science, Engineering And Technology (02 2017). https://doi.org/10.29007/xwg4
[23]
Max Pixel. [n.d.] a. Donut Frosting Food Sugar Bakery Eat Bake. https://www.maxpixel.net/Donut-Frosting-Food-Sugar-Bakery-Eat-Bake-4475458
[24]
Max Pixel. [n.d.] b. Sweet Waffle Delicious Food Enjoy Dessert Eat. https://www.maxpixel.net/Sweet-Waffle-Delicious-Food-Enjoy-Dessert-Eat-4477521
[25]
Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, Vol. 20 (1987), 53--65.
[26]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[27]
W. Sultani, C. Chen, and M. Shah. 2018. Real-World Anomaly Detection in Surveillance Videos. In CVPR. 6479--6488.
[28]
Du Tran, Lubomir D. Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning Spatiotemporal Features with 3D Convolutional Networks. In ICCV. IEEE Computer Society, 4489--4497.
[29]
M. T. Vega, D. C. Mocanu, J. Famaey, S. Stavrou, and A. Liotta. 2017. Deep Learning for Quality Assessment in Live Video Streaming. IEEE Signal Processing Letters, Vol. 24, 6 (June 2017), 736--740. https://doi.org/10.1109/LSP.2017.2691160
[30]
Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In ICML. 478--487.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
August 2021
4259 pages
ISBN:9781450383325
DOI:10.1145/3447548
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. anomaly detection
  2. computer vision
  3. deep learning
  4. graphics processors validation
  5. multiple instance learning

Qualifiers

  • Research-article

Conference

KDD '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 225
    Total Downloads
  • Downloads (Last 12 months)18
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media