Abstract
The pixel-wise code exposure (PCE) camera is a compressive sensing camera that has several advantages, such as low power consumption and high compression ratio. Moreover, one notable advantage is the capability to control individual pixel exposure time. Conventional approaches of using PCE cameras involve a time-consuming and lossy process to reconstruct the original frames and then use those frames for target tracking and classification. Otherwise, conventional approaches will fail if compressive measurements are used. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. Our approach has two parts: tracking and classification. The tracking has been done via detection using You Only Look Once (YOLO), and the classification is achieved using residual network (ResNet). Extensive simulations using short-wave infrared (SWIR) videos demonstrated the efficacy of our proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhao, Z., Chen, H., Chen, G., Kwan, C., Li, X. R.: Comparison of several ballistic target tracking filters. In: Proceedings of American Control Conference, pp. 2197–2202 (2006)
Zhao, Z., Chen, H., Chen, G., Kwan, C., Li, X. R.: IMM-LMMSE filtering algorithm for ballistic target tracking with unknown ballistic coefficient. In: Proceedings of SPIE, Volume 6236, Signal and Data Processing of Small Targets (2006)
Zhou, J., Kwan, C.: Anomaly detection in low quality traffic monitoring videos using optical flow. In: Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX (2018)
Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.: Staple: complementary learners for real-time tracking. In: Conference on Computer Vision and Pattern Recognition (2016)
Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2, 2246–2252 (1999)
Kwan, C., Zhou, J., Wang, Z., Li, B.: Efficient anomaly detection algorithms for summarizing low quality videos. In: Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, vol. 1064906 (2018)
Kwan, C., Yin, J., Zhou, J.: The development of a video browsing and video summary review tool. In: Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, vol. 1064907 (2018)
Kandylakis, Z., et al.: Multimodal data fusion for effective surveillance of critical infrastructure. In: ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 87–93 (2017)
Berg, A.: Detection and Tracking in Thermal Infrared Imagery. Linköping University Electronic Press, Diss (2016)
Zhou, J., Kwan, C.: Tracking of multiple pixel targets using multiple cameras. In: 15th International Symposium on Neural Networks (2018)
Kwan, C., Chou, B., Kwan, L. M.: A comparative study of conventional and deep learning target tracking algorithms for low quality videos. In: 15th International Symposium on Neural Networks (2018)
Candes, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Signal Proc. Mag. 25, 21–30 (2008)
Zhang, J., Xiong, T., Tran, T., Chin, S., Etienne-Cummings, R.: Compact all-CMOS spatio-temporal compressive sensing video camera with pixel-wise coded exposure. Opt. Express 24(8), 9013–9024 (2016)
Yang, J., Zhang, Y.: Alternating direction algorithms for l 1-problems in compressive sensing. SIAM J. Sci. Comput. 33, 250–278 (2011)
Tropp, J.A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)
Dao, M., Kwan, C., Koperski, K., Marchisio, G.: A joint sparsity approach to tunnel activity monitoring using high resolution satellite images. In: IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, pp. 322–328, New York City (2017)
Zhou, J., Ayhan, B., Kwan, C., Tran, T.: ATR performance improvement using images with corrupted or missing pixels. In: Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, vol. 106490E (2018)
Applied Research LLC, Phase 1 Final report, August 2016
Yang, M.H., Zhang, K., Zhang, L.: Real-time compressive tracking. In: European Conference on Computer Vision (2012)
Kwan, C., Chou, B., Echavarren, A., Budavari, B., Li, J., Tran, T.: Compressive vehicle tracking using deep learning. In: IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City (2018)
Redmon, J., Farhadi, A.: YOLOv3: An Incremental Improvement (2018). arXiv:1804.02767
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (2016)
Definition of mAP, https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173. Accessed 10 Jan 2019
Acknowledgements
This research was supported by the US Air Force under contract FA8651-17-C-0017. The views, opinions, and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the US Government. DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kwan, C., Chou, B., Yang, J. et al. Target tracking and classification using compressive sensing camera for SWIR videos. SIViP 13, 1629–1637 (2019). https://doi.org/10.1007/s11760-019-01506-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-019-01506-4