research-article

Optasia: A Relational Platform for Efficient Large-Scale Video Analytics

Authors:

Aakanksha Chowdhery,

Srikanth KandulaAuthors Info & Claims

SoCC '16: Proceedings of the Seventh ACM Symposium on Cloud Computing

Pages 57 - 70

https://doi.org/10.1145/2987550.2987564

Published: 05 October 2016 Publication History

Abstract

Camera deployments are ubiquitous, but existing methods to analyze video feeds do not scale and are error-prone. We describe Optasia, a dataflow system that employs relational query optimization to efficiently process queries on video feeds from many cameras. Key gains of Optasia result from modularizing vision pipelines in such a manner that relational query optimization can be applied. Specifically, Optasia can (i) de-duplicate the work of common modules, (ii) auto-parallelize the query plans based on the video input size, number of cameras and operation complexity, (iii) offers chunk-level parallelism that allows multiple tasks to process the feed of a single camera. Evaluation on traffic videos from a large city on complex vision queries shows high accuracy with many fold improvements in query completion time and resource usage relative to existing systems.

References

[1]

Operation virtual shield: a homeland security grid established in chicago. http://bit.ly/2bb33F7.

[2]

OpenALPR: Open automatic license plate recognition library. https://github.com/openalpr/openalpr.

[3]

Seattle department of transportation live traffic videos. http://web6.seattle.gov/travelers/.

[4]

Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2/.

[5]

Defense advanced research projects agency information systems office's three-year program on video surveillance and monitoring technology. http://bit.ly/2bQIixs.

[6]

ImageNET. http://www.image-net.org.

[7]

Microsoft COCO: Common objects in context. http://mscoco.org.

[8]

Netflix tech blog: High quality video encoding at scale. http://nflx.it/2bdMceJ.

[9]

Earthcam live feeds from NYC. http://bit.ly/1SZgZQv.

[10]

OpenCV. http://opencv.org/.

[11]

Video surveillance storage: How much is enough? http://bit.ly/2bb3rDR.

[12]

TPC-DS Benchmark. http://bit.ly/1J6uDap.

[13]

Trafficland. http://www.trafficland.com.

[14]

Supply of video management software remains fragmented. http://bit.ly/1TiDnVr.

[15]

Vehicle counting based on blob detection. http://bit.ly/2bdjNWA.

[16]

Hadoop YARN Project. http://bit.ly/1iS8xvP.

[17]

Youtube: Video encoding settings. http://bit.ly/2bygQ8m.

[18]

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous systems. http://www.tensorflow.org.

[19]

ACLU. Police body-mounted cameras: With right policies in place, a win for all. http://bit.ly/1RBzI1i.

[20]

S. Agarwal, S. Kandula, N. Burno, M.-C. Wu, I. Stoica, and J. Zhou. Re-optimizing data parallel computing. In NSDI, 2012.

Digital Library

[21]

G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris. Reining in the outliers in map-reduce clusters using mantri. In OSDI, 2010.

Digital Library

[22]

M. Armbrust, R. S. Xin, C. Lian, Y. Huai, D. Liu, J. K. Bradley, X. Meng, T. Kaftan, M. J. Franklin, and A. Ghodsi. Spark sql: Relational data processing in spark. In SIGMOD, 2015.

Digital Library

[23]

Avigilon. Video surveillance solutions. http://bit.ly/21EIIr3.

[24]

D. Bradley and G. Roth. Adaptive thresholding using the integral image. Journal of graphics, gpu, and game tools, 12 (2):13--21, 2007.

[25]

L. Breiman. Random forests. Machine learning, 45(1):5--32, 2001.

Digital Library

[26]

N. Buch, S. Velastin, and J. Orwell. A review of computer vision techniques for the analysis of urban traffic. IEEE T. on Intelligent Transportation Systems, 12(3):920--939, 2011.

Digital Library

[27]

R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. Scope: easy and efficient parallel processing of massive data sets. VLDB Endowment, 1(2): 1265--1276, 2008.

Digital Library

[28]

C. Chambers, A. Raniwala, F. Perry, S. Adams, R. R. Henry, R. Bradshaw, and N. Weizenbaum. Flumejava: easy, efficient data-parallel pipelines. In PLDI, 2010.

Digital Library

[29]

B. Chandramouli, J. Goldstein, M. Barnett, R. DeLine, D. Fisher, J. C. Platt, J. F. Terwilliger, and J. Wernsing. Trill: A high-performance incremental query processor for diverse analytics. VLDB Endowment, 8(4):401--412, 2014.

Digital Library

[30]

Y. Cheng. Mean shift, mode seeking, and clustering. IEEE T. on Pattern Analysis and Machine Intelligence, 17(8):790--799, 1995.

Digital Library

[31]

T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. Mapreduce online. In NSDI, 2010.

Digital Library

[32]

N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. 2000.

Digital Library

[33]

N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR., 2005.

Digital Library

[34]

K. Davidson. A flow travel time relationship for use in transportation planning. In Australian Road Research Board Conference, 1966, Sydney, 1966.

[35]

J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, 2004.

Digital Library

[36]

A. Elgammal, D. Harwood, and L. Davis. Non-parametric model for background subtraction. In ECCV. 2000.

Digital Library

[37]

M. Everingham, L. Van Gool, C. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge. IJCV, 88(2):303--338, 2010.

Digital Library

[38]

R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. Liblinear: A library for large linear classification. JMLR, 9:1871--1874, 2008.

Digital Library

[39]

M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24 (6):381--395, 1981.

Digital Library

[40]

G. Graefe. The cascades framework for query optimization. IEEE Data Eng. Bull., 18(3):19--29, 1995.

[41]

R. M. Haralick and L. G. Shapiro. Image segmentation techniques. Computer vision, graphics, and image processing, 29 (1):100--132, 1985.

[42]

S. P. Hoogendoorn and P. H. Bovy. State-of-the-art of vehicular traffic flow modelling. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 215(4):283--303, 2001.

[43]

IIHS. Communities using red light cameras and speed cameras. http://bit.ly/2bb3OOA.

[44]

M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys, 2007.

Digital Library

[45]

D. Jiang, B. Ooi, L. Shi, and S. Wu. The performance of mapreduce: An in-depth study. VLDB Endow., 3(1), 2010.

Digital Library

[46]

J. Juang and Y.-C. Huang. Intelligent Technologies and Engineering Systems. 2013.

[47]

P. KaewTraKulPong and R. Bowden. An improved adaptive background mixture model for real-time tracking with shadow detection. In Video-based surveillance systems, pages 135--144. Springer, 2002.

[48]

S. Liao, Y. Hu, X. Zhu, and S. Z. Li. Person re-identification by local maximal occurrence representation and metric learning. In CVPR, 2015.

[49]

S. Liu, J. Pu, Q. Luo, H. Qu, L. Ni, and R. Krishnan. Vait: A visual analytics system for metropolitan transportation. IEEE T. on Intelligent Transportation Systems, 2013.

Digital Library

[50]

D. G. Lowe. Object recognition from local scale-invariant features. In ICCV, 1999.

Digital Library

[51]

B. D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In IJCAI, 1981.

Digital Library

[52]

T. Malisiewicz, A. Gupta, and A. A. Efros. Ensemble of exemplar-svms for object detection and beyond. In ICCV, 2011.

Digital Library

[53]

A. D. May. Traffic flow fundamentals. 1990.

[54]

P. McGowen and M. Sanderson. Accuracy of pneumatic road tube counters. In Proceedings of the 2011 Western District Annual Meeting, Anchorage, AK, USA, 2011.

[55]

Microsoft. An Overview of Windows Azure. http://bit.ly/1Qo6yUg.

[56]

L. E. Y. Mimbela and L. A. Klein. Summary of vehicle detection and surveillance technologies used in intelligent transportation systems. http://bit.ly/2bEZsgW.

[57]

C. Olston et al. Pig Latin: A Not-So-Foreign Language for Data Processing. In SIGMOD, 2008.

Digital Library

[58]

Omnicast. Video Management Software. http://bit.ly/2bIKnYK.

[59]

R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the data: Parallel analysis with sawzall. Scientific Prog., 2003.

Digital Library

[60]

Pro-Vigil. Video surveillance. http://pro-vigil.com/.

[61]

R. Ramakrishnan and J. Gehrke. Database management systems. 2000.

Digital Library

[62]

P. Remagnino. Video-Based Surveillance Systems: Computer Vision and Distributed Processing. 2001.

Digital Library

[63]

P. Roy, S. Seshadri, S. Sudarshan, and S. Bhobe. Efficient and extensible algorithms for multi query optimization. In SIGMOD, 2000.

Digital Library

[64]

M. Satyanarayanan. Mobile computing: The next decade. SIGMOBILE Mob. Comput. Commun. Rev., 2011.

Digital Library

[65]

J. Shi and C. Tomasi. Good features to track. In CVPR, 1994.

[66]

A. Thusoo et al. Hive: A Warehousing Solution Over A Map-Reduce Framework. VLDB Endow., 2009.

Digital Library

[67]

B. Tian, B. Morris, M. Tang, Y. Liu, Y. Yao, C. Gou, D. Shen, and S. Tang. Hierarchical and networked vehicle surveillance in its: A survey. IEEE T. on Intelligent Transportation Systems, 16(2):557--580, 2015.

[68]

Y.-l. Tian, L. Brown, A. Hampapur, M. Lu, A. Senior, and C.-f. Shu. Ibm smart surveillance system (s3): Event based video surveillance system with an open and extensible framework. Mach. Vision Appl., 19(5-6):315--327, 2008.

Digital Library

[69]

H. Vceraraghavan, O. Masoud, and N. Papanikolopoulos. Vision-based monitoring of intersections. In IEEE International Conference on Intelligent Transportation Systems, pages 7--12, 2002.

[70]

X. Wang, X. Ma, and W. E. L. Grimson. Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models. IEEE Transactions on PAMI, 31(3):539--555, 2009.

Digital Library

[71]

S. Yeung, O. Russakovsky, G. Mori, and L. Fei-Fei. End-to-end learning of action detection from frame glimpses in videos. In CVPR, 2016.

[72]

Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. K. Gunda, and J. Currey. Dryadlinq: a system for general-purpose distributed data-parallel computing using a high-level language. In OSDI, 2008.

Digital Library

[73]

G. Yuan, X. Zhang, Q. Yao, and K. Wang. Hierarchical and modular surveillance systems in its. IEEE Intelligent Systems, 26(5):10--15, 2011.

Digital Library

[74]

M. Zaharia, T. Das, H. Li, S. Shenker, and I. Stoica. Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. In HotCloud, 2012.

Digital Library

[75]

J. Zhou, N. Bruno, and W. Lin. Advanced partitioning techniques for massively distributed computation. In SIGMOD, 2012.

Digital Library

[76]

J. Zhou, N. Bruno, M.-C. Wu, P.-A. Larson, R. Chaiken, and D. Shakib. Scope: parallel databases meet mapreduce. VLDB Endowment, 21(5):611--636, 2012.

Digital Library

[77]

Z. Zivkovic. Improved adaptive gaussian mixture model for background subtraction. In ICPR, 2004.

Digital Library

[78]

Z. Zivkovic and F. van der Heijden. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern recognition letters, 27(7):773--780, 2006.

Digital Library

Cited By

Wong MRamanujam MBalakrishnan GNetravali RVanbever LZhang I(2024)MadEyeProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691856(549-568)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691856
Giagkos DTzenetopoulos AMasouros DXydis SCatthoor FSoudris D(2024)AI-Driven QoS-Aware Scheduling for Serverless Video Analytics at the EdgeInformation10.3390/info1508048015:8(480)Online publication date: 13-Aug-2024
https://doi.org/10.3390/info15080480
He WSabek ILou YCafarella M(2024)Optimizing Video Selection LIMIT Queries with Commonsense KnowledgeProceedings of the VLDB Endowment10.14778/3654621.365463917:7(1751-1764)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.14778/3654621.3654639
Show More Cited By

Index Terms

Optasia: A Relational Platform for Efficient Large-Scale Video Analytics
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization
2. Software and its engineering
  1. Software creation and management
    1. Designing software
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems

Recommendations

Equivalence and minimization of conjunctive queries under combined semantics
ICDT '12: Proceedings of the 15th International Conference on Database Theory

The problems of query containment, equivalence, and minimization are fundamental problems in the context of query processing and optimization. In their classic work [2] published in 1977, Chandra and Merlin solved the three problems for the language of ...
Optimizing large star-schema queries with snowflakes via heuristic-based query rewriting
CASCON '03: Proceedings of the 2003 conference of the Centre for Advanced Studies on Collaborative research

User queries have been becoming increasingly complex (e.g., involving a large number of joins) as database technology is applied to some application domains such as data warehouses and life sciences. Query optimizers in existing database management ...
Efficient Top-k Query Answering through its Top-N Rewritings Using Views
PIKM '15: Proceedings of the 8th Workshop on Ph.D. Workshop in Information and Knowledge Management

Recently, various algorithms were proposed to speed up top-k query answering by using multiple materialized query results. Nevertheless, for most of the proposed algorithms, a potentially costly view selection operation is required. In fact, the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SoCC '16: Proceedings of the Seventh ACM Symposium on Cloud Computing

October 2016

534 pages

ISBN:9781450345255

DOI:10.1145/2987550

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SoCC '16

Sponsor:

SoCC '16: ACM Symposium on Cloud Computing

October 5 - 7, 2016

CA, Santa Clara, USA

Acceptance Rates

SoCC '16 Paper Acceptance Rate 38 of 151 submissions, 25%;

Overall Acceptance Rate 169 of 722 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

60
Total Citations
View Citations
768
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wong MRamanujam MBalakrishnan GNetravali RVanbever LZhang I(2024)MadEyeProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691856(549-568)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691856
Giagkos DTzenetopoulos AMasouros DXydis SCatthoor FSoudris D(2024)AI-Driven QoS-Aware Scheduling for Serverless Video Analytics at the EdgeInformation10.3390/info1508048015:8(480)Online publication date: 13-Aug-2024
https://doi.org/10.3390/info15080480
He WSabek ILou YCafarella M(2024)Optimizing Video Selection LIMIT Queries with Commonsense KnowledgeProceedings of the VLDB Endowment10.14778/3654621.365463917:7(1751-1764)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.14778/3654621.3654639
Ma QYuan HHu ZWang XYang Z(2024)A Liquidity Analysis System for Large-scale Video Streams in the OilfieldACM Transactions on Sensor Networks10.1145/364922220:3(1-22)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3649222
Zhong TZhang ZLu GYuan YWang YWang G(2023)TVM: A Tile-based Video Management FrameworkProceedings of the VLDB Endowment10.14778/3636218.363622417:4(671-684)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.14778/3636218.3636224
Kossmann FWu ZLai ETatbul NCao LKraska TMadden S(2023)Extract-Transform-Load for Video StreamsProceedings of the VLDB Endowment10.14778/3598581.359860016:9(2302-2315)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.14778/3598581.3598600
Wu YLentz MZhuo DLu Y(2023)Serving and Optimizing Machine Learning Workflows on Heterogeneous InfrastructuresProceedings of the VLDB Endowment10.14778/3570690.357069216:3(406-419)Online publication date: 23-Jan-2023
https://dl.acm.org/doi/10.14778/3570690.3570692
Yang ZHuang YWang ZGao FLu YLi CWang X(2022)Demonstration of accelerating machine learning inference queries with correlative proxy modelsProceedings of the VLDB Endowment10.14778/3554821.355488715:12(3734-3737)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554887
Shaowang TZhao JSintos SKrishnan S(2022)Towards causal physical error discovery in video analytics systemsProceedings of the Workshop on Human-In-the-Loop Data Analytics10.1145/3546930.3547495(1-6)Online publication date: 12-Jun-2022
https://dl.acm.org/doi/10.1145/3546930.3547495
He WCafarella MIves ZBonifati AEl Abbadi A(2022)Controlled Intentional Degradation in Analytical Video SystemsProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517899(2105-2119)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517899
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten