Abstract
Weakly-supervised temporal action localization (WTAL) has recently attracted attentions. Many of the state-of-the-art methods usually utilize temporal class activation map (T-CAM) to obtain target action temporal regions. However, class-specific T-CAM tends to cover only the most discriminative part of the actions, not the entire action. In this paper, we propose an erasing strategy for mining discriminative regions in weakly-supervised temporal action localization (DRES). DRES achieves better performance with action localization, which can be attribute to two aspects. First, we employ the salient detection module, which suppresses the background to obtain the most discriminative regions. Second, we design the eraser module to discover the missed action regions by the salient detection module, which complements action regions. Based on experiments, we demonstrate that DRES improve the state-of-the-art performance on THUMOS’14.
H. Zeng—Student as the first author.
This work was supported by the National Natural Science Foundation of China, with grant numbers 61902101 and 61806063 respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alwassel, H., Heilbron, C.F., Thabet, K.A., Ghanem, B.: RefineLoc: iterative refinement for weakly-supervised action localization. arXiv Computer Vision and Pattern Recognition (2019)
Carreira, J., Zisserman, A.: Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset (2017)
Chao, Y.W., Vijayanarasimhan, S., Seybold, B., Ross, D.A., Deng, J., Sukthankar, R.: Rethinking the Faster R-CNN Architecture for Temporal Action Localization (2018)
Deng, J., Dong, W., Socher, R., Li, L.j., Li, K., Li, F.f.: ImageNet: a large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)
Fang, Z., Zhu, S., Yu, J., Tian, Q.: PCPCAD: proposal complementary action detector. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 424–429, July 2019. https://doi.org/10.1109/ICME.2019.00080
Gao, J., Yang, Z., Nevatia, R.: Cascaded boundary regression for temporal action detection. In: BMVC (2017)
Jiang, Y.G., et al.: Thumos Challenge: Action Recognition with a Large Number of Classes (2014)
Lee, P., Uh, Y., Byun, H.: Background suppression network for weakly-supervised temporal action localization. In: AAAI (2020)
Lin, T., Zhao, X., Shou, Z.: Single shot temporal action detection. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 988–996. ACM (2017)
Lin, T., Zhao, X., Su, H., Wang, C., Yang, M.: BSN: Boundary sensitive network for temporal action proposal generation (2018)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Liu, Z., et al.: Weakly Supervised Temporal Action Localization Through Contrast Based Evaluation Networks, pp. 3899–3908 (2019)
Narayan, S., Cholakkal, H., Khan, S.F., Shao, L.: 3C-net - category count and center loss for weakly-supervised action localization. In: ICCV, pp. 8678–8686 (2019)
Nguyen, P., Han, B., Liu, T., Prasad, G.: Weakly supervised action localization by sparse temporal pooling network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6752–6761 (2018)
Paul, S., Roy, S., Roy-Chowdhury, A.K.: W-TALC: weakly-supervised temporal activity localization and classification. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 588–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_35
Shou, Z., Chan, J., Zareian, A., Miyazawa, K., Chang, S.F.: CDC: convolutional-de-convolutional networks for precise temporal action localization in untrimmed videos (2017)
Shou, Z., Gao, H., Zhang, L., Miyazawa, K., Chang, S.-F.: AutoLoc: weakly-supervised temporal action localization in untrimmed videos. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 162–179. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_10
Shou, Z., Wang, D., Chang, S.F.: Temporal action localization in untrimmed videos via multi-stage CNNs (2016)
Singh, K.K., Lee, Y.J.: Hide-and-seek: forcing a network to be meticulous for weakly-supervised object and action localization. In: International Conference on Computer Vision (ICCV) (2017)
Singh, K.K., Lee, J.Y.: Hide-and-seek: forcing a network to be meticulous for weakly-supervised object and action localization. In: ICCV, pp. 3544–3553 (2017)
Wang, L., Xiong, Y., Lin, D., Gool, V.L.: UntrimmedNets for weakly supervised action recognition and detection. In: CVPR (2017)
Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-L1 optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Statistical and Geometrical Approaches to Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03061-1_2
Xu, H., Das, A., Saenko, K.: R-C3D: region convolutional 3D network for temporal activity detection. In: Proceedings of the International Conference on Computer Vision (ICCV) (2017)
Xu, Y., et al.: Segregated temporal assembly recurrent networks for weakly supervised multiple action detection. In: National Conference on Artificial Intelligence (2019)
Yuan, Y., Lyu, Y., Shen, X., Tsang, I., Yeung, D.Y.: Marginalized average attentional network for weakly-supervised learning. In: International Conference on Learning Representations (2019)
Zhang, X., Wei, Y., Feng, J., Yang, Y., Huang, T.: Adversarial complementary learning for weakly supervised object localization. In: IEEE CVPR (2018)
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., Lin, D.: Temporal action detection with structured segment networks (2017)
Zhong, J.X., Li, N., Kong, W., Zhang, T., Li, H.T., Li, G.: Step-by-step erasion, one-by-one collection: a weakly supervised temporal action detector. In: MM 2018: ACM Multimedia Conference Seoul Republic of Korea October 2018, pp. 35–44 (2018)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Computer Vision and Pattern Recognition (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zeng, H., Zhu, S., Yu, J. (2020). Discriminative Regions Erasing Strategy for Weakly-Supervised Temporal Action Localization. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12306. Springer, Cham. https://doi.org/10.1007/978-3-030-60639-8_53
Download citation
DOI: https://doi.org/10.1007/978-3-030-60639-8_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60638-1
Online ISBN: 978-3-030-60639-8
eBook Packages: Computer ScienceComputer Science (R0)