Skip to main content

Advertisement

SpringerLink
Log in

SEMARTCam scheduler: semantics driven real-time data collection from indoor camera networks to maximize event detection

  • Special Issue
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

A distributed camera network allows for many compelling applications such as large-scale tracking or event detection. In most practical systems, resources are constrained. Although one would like to probe every camera at every time instant and store every frame, this is simply not feasible. Constraints arise from network bandwidth restrictions, I/O and disk usage from writing images, and CPU usage needed to extract features from the images. Assume that, due to resource constraints, only a subset of sensors can be probed at any given time unit. This paper examines the problem of selecting the “best” subset of sensors to probe under some user-specified objective—e.g., detecting as much motion as possible. With this objective, we would like to probe a camera when we expect motion, but would not like to waste resources on a non-active camera. The main idea behind our approach is the use of sensor semantics to guide the scheduling of resources. We learn a dynamic probabilistic model of motion correlations between cameras, and use the model to guide resource allocation for our sensor network. Although previous work has leveraged probabilistic models for sensor-scheduling, our work is distinct in its focus on real-time building-monitoring using a camera network. We validate our approach on a sensor network of a dozen cameras spread throughout a university building, recording measurements of unscripted human activity over a two week period. We automatically learnt a semantic model of typical behaviors, and show that one can significantly improve efficiency of resource allocation by exploiting this model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. For example, due to network delays, deployment has to be done in advance.

  2. More on this in Sect. 7.

  3. We chose second, but other scales were possible to discrete time.

  4. Each camera has one nearest neighbor.

  5. As most of the time there is no motion.

  6. which is n*n*M where n is the number of sensors and M is the time window to look back.

  7. The table size is 1262 after thresholding in the extended semantics case, as opposed to 11*11*15*4 = 7, 260. The last 4 represents the four states: event start, event ongoing, event end and motion.

References

  1. Anandathirtha, P., Ramakrishnan, K., Raja, S., Kankanhalli, M.: Experiential sampling for object detection in video. In: Multimedia Content Analysis: Theory and Applications, p. 175 (2009)

  2. Brown, R.G., Hwang, P.Y.C.: Introduction to Random Signals and Applied Kalman Filtering. Wiley, New York (1992)

  3. Chen, C., Roussopoulos, N.: Adaptive selectivity estimation using query feedback. In: Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data, pp. 161–172 (1994)

  4. Daeipour, E., Bar-Shalom, Y.: An interacting multiple model approach for target tracking withglint noise. IEEE Trans. Aerosp. Electr. Syst. 31(2), 706–715 (1995)

    Article  Google Scholar 

  5. Gupta, V., Chung, T., Hassibi, B., Murray, R.: On a stochastic sensor selection algorithm with applications in sensor scheduling and sensor coverage. Automatica 42(2), 251–260 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  6. Han, Q., Mehrotra, S., Venkatasubramanian, N.: Energy efficient data collection in distributed sensor environments. In: Proceedings of 24th International Conference on Distributed Computing Systems, vol. 39, pp. 590–597 (2004)

  7. Ihler, A., Hutchins, J., Smyth, P.: Adaptive event detection with time-varying poisson processes. In: KDD ’06: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 207–216. ACM Press, New York (2006). doi:10.1145/1150402.1150428

  8. Isler, V., Bajcsy, R.: The sensor selection problem for bounded uncertainty sensing models. In: IPSN 2005 Fourth International Symposium on Information Processing in Sensor Networks, 2005, pp. 151–158 (2005)

  9. Kao, B., Garcia-Molina, H.: Deadline assignment in a distributed soft real-time system. IEEE Trans. Parallel Distrib. Syst. 8(12), 1268–1274 (1997)

    Article  Google Scholar 

  10. Krishnamurthy, V.: Algorithms for optimal scheduling and management of hidden Markovmodel sensors. IEEE Trans. Signal Process. [see also IEEE Trans. Acoust. Speech Signal Process.] 50(6), 1382–1397 (2002)

    MathSciNet  Google Scholar 

  11. Lazaridis, I., Mehrotra, S.: Capturing sensor-generated time series with quality guarantees. In: International Conference on Data Engineering (ICDE) (2003)

  12. Makris, D., Ellis, T., Black, J.: Bridging the gaps between cameras. In: CVPR 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2 (2004)

  13. Murphy, K.P., Weiss, Y.: The factored frontier algorithm for approximate inference in dbns. In: UAI ’01: Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, pp. 378–385. Morgan Kaufmann Publishers Inc., San Francisco (2001)

  14. Nieh, J., Lam, M.: The design, implementation and evaluation of SMART: a scheduler for multimedia applications. In: Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles, pp. 184–197. ACM, New York (1997)

  15. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)

  16. Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  17. Schulz, D., Fox, D., Hightower, J.: People tracking with anonymous and id-sensors using rao-blackwellised particle filters. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2003)

  18. Scott, S.: Detecting network intrusion using a Markov modulated nonhomogeneous Poisson process. J. Am. Stat. Assoc. (2010)

  19. Song, B., Roy-Chowdhury, A.: Stochastic adaptive tracking in a camera network. In: IEEE 11th International Conference on Computer Vision, 2007 ICCV, pp. 1–8 (2007)

  20. Vaisenberg, R., Mehrotra, S., Ramanan, D.: Exploiting semantics for scheduling real-time data collection from sensors to maximize event detection. In: Proceedings of SPIE, vol. 7253, p. 72530B (2009)

  21. Williams, J., Fisher, J. III, Willsky, A.: An approximate dynamic programming approach to a communication constrained sensor management problem. In: Proceedings of Eighth International Conference of Information Fusion (2005)

  22. Yu, X., Niyogi, K., Mehrotra, S., Venkatasubramanian, N.: Adaptive target tracking in sensor networks. In: Communication Networks and Distributed Systems Modeling and Simulation Conference (CNDS04) (2004)

Download references

Acknowledgments

Support of this research by the National Science Foundation under Award Numbers 0331707 and 0331690 and the Department of Homeland Security under award number EMW-2007-FP-02535 is gratefully acknowledged.

Author information

Authors and Affiliations

  1. School of Information and Computer Sciences, University of California, Irvine, CA, 92697-3425, USA

    Ronen Vaisenberg, Sharad Mehrotra & Deva Ramanan

Authors
  1. Ronen Vaisenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sharad Mehrotra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deva Ramanan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ronen Vaisenberg.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vaisenberg, R., Mehrotra, S. & Ramanan, D. SEMARTCam scheduler: semantics driven real-time data collection from indoor camera networks to maximize event detection. J Real-Time Image Proc 5, 215–230 (2010). https://doi.org/10.1007/s11554-009-0147-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-009-0147-8

Keywords

Search

Navigation