Skip to main content
Springer Nature Link
Log in

DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognition

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15142))

Included in the following conference series:

  • 257 Accesses

Abstract

Neuromorphic sensors, specifically event cameras, revolutionize visual data acquisition by capturing pixel intensity changes with exceptional dynamic range, minimal latency, and energy efficiency, setting them apart from conventional frame-based cameras. The distinctive capabilities of event cameras have ignited significant interest in the domain of event-based action recognition, recognizing their vast potential for advancement. However, the development in this field is currently slowed by the lack of comprehensive, large-scale datasets, which are critical for developing robust recognition frameworks. To bridge this gap, we introduces DailyDVS-200, a meticulously curated benchmark dataset tailored for the event-based action recognition community. DailyDVS-200 is extensive, covering 200 action categories across real-world scenarios, recorded by 47 participants, and comprises more than 22,000 event sequences. This dataset is designed to reflect a broad spectrum of action types, scene complexities, and data acquisition diversity. Each sequence in the dataset is annotated with 14 attributes, ensuring a detailed characterization of the recorded actions. Moreover, DailyDVS-200 is structured to facilitate a wide range of research paths, offering a solid foundation for both validating existing approaches and inspiring novel methodologies. By setting a new benchmark in the field, we challenge the current limitations of neuromorphic data processing and invite a surge of new approaches in event-based action recognition techniques, which paves the way for future explorations in neuromorphic computing and beyond. The dataset and source code are available at https://github.com/QiWang233/DailyDVS-200.

Q. Wang and Z. Xu—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.prophesee.ai.

References

  1. Amir, A., et al.: A low power, fully event-based gesture recognition system. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7243–7252 (2017)

    Google Scholar 

  2. Baldwin, R.W., Liu, R., Almatrafi, M., Asari, V., Hirakawa, K.: Time-ordered recent event (TORE) volumes for event cameras. IEEE Trans. Pattern Anal. Mach. Intell. 45(2), 2519–2532 (2022)

    Article  Google Scholar 

  3. Bertasius, G., Wang, H., Torresani, L.: Is space-time attention all you need for video understanding? In: ICML, vol. 2, p. 4 (2021)

    Google Scholar 

  4. Bi, Y., Chadha, A., Abbas, A., Bourtsoulatze, E., Andreopoulos, Y.: Graph-based spatio-temporal feature learning for neuromorphic vision sensing. IEEE Trans. Image Process. 29, 9084–9098 (2020)

    Article  Google Scholar 

  5. de Blegiers, T., Dave, I.R., Yousaf, A., Shah, M.: EventTransAct: a video transformer-based framework for event-camera based action recognition. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–7. IEEE (2023)

    Google Scholar 

  6. Brandli, C., Berner, R., Yang, M., Liu, S.C., Delbruck, T.: A \(240 \times 180\) 130 db 3 \(\upmu \)s latency global shutter spatiotemporal vision sensor. IEEE J. Solid-State Circuits 49(10), 2333–2341 (2014). https://doi.org/10.1109/JSSC.2014.2342715

    Article  Google Scholar 

  7. Caba Heilbron, F., Escorcia, V., Ghanem, B., Carlos Niebles, J.: ActivityNet: a large-scale video benchmark for human activity understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–970 (2015)

    Google Scholar 

  8. Cannici, M., Ciccone, M., Romanoni, A., Matteucci, M.: A differentiable recurrent surface for asynchronous event-based data. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part XX. LNCS, vol. 12365, pp. 136–152. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58565-5_9

    Chapter  Google Scholar 

  9. Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)

    Google Scholar 

  10. Che, K., et al.: Differentiable hierarchical and surrogate gradient search for spiking neural networks. Adv. Neural. Inf. Process. Syst. 35, 24975–24990 (2022)

    Google Scholar 

  11. Chen, S., Guo, M.: Live demonstration: CeleX-V: a 1m pixel multi-mode event-based sensor. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1682–1683. IEEE (2019)

    Google Scholar 

  12. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  13. Dong, Y., Li, Y., Zhao, D., Shen, G., Zeng, Y.: Bullying10k: a large-scale neuromorphic dataset towards privacy-preserving bullying recognition. Adv. Neural Inf. Process. Syst. 36 (2024)

    Google Scholar 

  14. Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  15. Duarte, L., Neto, P.: Event-based dataset for the detection and classification of manufacturing assembly tasks. Data Brief 54, 110340 (2024)

    Article  Google Scholar 

  16. Feichtenhofer, C.: X3D: expanding architectures for efficient video recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 203–213 (2020)

    Google Scholar 

  17. Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6202–6211 (2019)

    Google Scholar 

  18. Gao, Y., et al.: Action recognition and benchmark using event cameras. IEEE Trans. Pattern Anal. Mach. Intell. (2023)

    Google Scholar 

  19. Gehrig, D., Loquercio, A., Derpanis, K.G., Scaramuzza, D.: End-to-end learning of representations for asynchronous event-based data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5633–5643 (2019)

    Google Scholar 

  20. Kay, W., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

  21. Kim, J., Bae, J., Park, G., Zhang, D., Kim, Y.M.: N-ImageNet: towards robust, fine-grained object recognition with event cameras. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2146–2156 (2021)

    Google Scholar 

  22. Kliper-Gross, O., Hassner, T., Wolf, L.: The action similarity labeling challenge. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 615–621 (2011)

    Article  Google Scholar 

  23. Kong, Y., Fu, Y.: Human action recognition and prediction: a survey. Int. J. Comput. Vision 130(5), 1366–1401 (2022)

    Article  Google Scholar 

  24. Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: HMDB: a large video database for human motion recognition. In: 2011 International Conference on Computer Vision, pp. 2556–2563. IEEE (2011)

    Google Scholar 

  25. Lagorce, X., Orchard, G., Galluppi, F., Shi, B.E., Benosman, R.B.: HOTS: a hierarchy of event-based time-surfaces for pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1346–1359 (2016)

    Article  Google Scholar 

  26. Laptev, I.: On space-time interest points. Int. J. Comput. Vision 64, 107–123 (2005)

    Article  Google Scholar 

  27. Li, H., Liu, H., Ji, X., Li, G., Shi, L.: CIFAR10-DVS: an event-stream dataset for object classification. Front. Neurosci. 11, 309 (2017)

    Article  Google Scholar 

  28. Li, J., Wang, X., Zhu, L., Li, J., Huang, T., Tian, Y.: Retinomorphic object detection in asynchronous visual streams. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 1332–1340 (2022)

    Google Scholar 

  29. Li, Y., Dong, Y., Zhao, D., Zeng, Y.: N-Omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning. Sci. Data 9(1), 746 (2022)

    Article  Google Scholar 

  30. Li, Y., et al.: Graph-based asynchronous event processing for rapid object recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 934–943 (2021)

    Google Scholar 

  31. Lin, J., Gan, C., Han, S.: TSM: temporal shift module for efficient video understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7083–7093 (2019)

    Google Scholar 

  32. Lin, Y., Ding, W., Qiang, S., Deng, L., Li, G.: ES-ImageNet: a million event-stream classification dataset for spiking neural networks. Front. Neurosci. 15, 1546 (2021)

    Article  Google Scholar 

  33. Liu, J., Kuipers, B., Savarese, S.: Recognizing human actions by attributes. In: CVPR 2011, pp. 3337–3344. IEEE (2011)

    Google Scholar 

  34. Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.Y., Kot, A.C.: NTU RGB+ D 120: a large-scale benchmark for 3D human activity understanding. IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2684–2701 (2019)

    Article  Google Scholar 

  35. Liu, Q., Xing, D., Tang, H., Ma, D., Pan, G.: Event-based action recognition using motion information and spiking neural networks. In: IJCAI, pp. 1743–1749 (2021)

    Google Scholar 

  36. Liu, Z., et al.: Video swin transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3202–3211 (2022)

    Google Scholar 

  37. Messikommer, N., Gehrig, D., Loquercio, A., Scaramuzza, D.: Event-based asynchronous sparse convolutional networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part VIII. LNCS, vol. 12353, pp. 415–431. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58598-3_25

    Chapter  Google Scholar 

  38. Miao, S., et al.: Neuromorphic vision datasets for pedestrian detection, action recognition, and fall detection. Front. Neurorobot. 13, 38 (2019)

    Article  Google Scholar 

  39. Moeys, D.P., et al.: Steering a predator robot using a mixed frame/event-driven convolutional neural network. In: 2016 Second International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), pp. 1–8. IEEE (2016)

    Google Scholar 

  40. Morency, L.P., Quattoni, A., Darrell, T.: Latent-dynamic discriminative models for continuous gesture recognition. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

    Google Scholar 

  41. Neftci, E.O., Mostafa, H., Zenke, F.: Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Process. Mag. 36(6), 51–63 (2019)

    Article  Google Scholar 

  42. Orchard, G., Jayawant, A., Cohen, G.K., Thakor, N.: Converting static image datasets to spiking neuromorphic datasets using saccades. Front. Neurosci. 9, 437 (2015)

    Article  Google Scholar 

  43. Peng, Y., Zhang, Y., Xiong, Z., Sun, X., Wu, F.: GET: group event transformer for event-based vision. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6038–6048 (2023)

    Google Scholar 

  44. Posch, C., Matolin, D., Wohlgenannt, R.: A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid-State Circuits 46(1), 259–275 (2010)

    Article  Google Scholar 

  45. Rebecq, H., Horstschaefer, T., Scaramuzza, D.: Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization (2017)

    Google Scholar 

  46. Sabater, A., Montesano, L., Murillo, A.C.: Event transformer. A sparse-aware solution for efficient event data processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2677–2686 (2022)

    Google Scholar 

  47. Schaefer, S., Gehrig, D., Scaramuzza, D.: AEGNN: asynchronous event-based graph neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12371–12381 (2022)

    Google Scholar 

  48. Scovanner, P., Ali, S., Shah, M.: A 3-dimensional sift descriptor and its application to action recognition. In: Proceedings of the 15th ACM International Conference on Multimedia, pp. 357–360 (2007)

    Google Scholar 

  49. Serrano-Gotarredona, T., Linares-Barranco, B.: Poker-DVS and MNIST-DVS. Their history, how they were made, and other details. Front. Neurosci. 9, 481 (2015)

    Article  Google Scholar 

  50. Shi, Q., Cheng, L., Wang, L., Smola, A.: Human action segmentation and recognition using discriminative semi-Markov models. Int. J. Comput. Vision 93, 22–32 (2011)

    Article  Google Scholar 

  51. Sironi, A., Brambilla, M., Bourdis, N., Lagorce, X., Benosman, R.: HATS: histograms of averaged time surfaces for robust event-based object classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1731–1740 (2018)

    Google Scholar 

  52. Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)

  53. Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)

    Google Scholar 

  54. Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6450–6459 (2018)

    Google Scholar 

  55. Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  56. Wang, H., Oneata, D., Verbeek, J., Schmid, C.: A robust and efficient video representation for action recognition. Int. J. Comput. Vision 119, 219–238 (2016)

    Article  MathSciNet  Google Scholar 

  57. Wang, X., et al.: Reliable object tracking via collaboration of frame and event flows. arXiv preprint arXiv:2108.05015 (2021)

  58. Wang, X., et al.: HARDVS: revisiting human activity recognition with dynamic vision sensors. arXiv preprint arXiv:2211.09648 (2022)

  59. Yao, M., et al.: Spike-driven transformer. Adv. Neural Inf. Process. Syst. 36 (2024)

    Google Scholar 

  60. Zeng, Y., et al.: BrainCog: a spiking neural network based, brain-inspired cognitive intelligence engine for brain-inspired AI and brain simulation. Patterns 4(8) (2023)

    Google Scholar 

  61. Zhou, Z., et al.: SpikFormer: when spiking neural network meets transformer. arXiv preprint arXiv:2209.15425 (2022)

  62. Zhu, A.Z., Yuan, L., Chaney, K., Daniilidis, K.: Unsupervised event-based learning of optical flow, depth, and egomotion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 989–997 (2019)

    Google Scholar 

  63. Zhu, L., Li, J., Wang, X., Huang, T., Tian, Y.: NeuSpike-net: high speed video reconstruction via bio-inspired neuromorphic cameras. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2400–2409 (2021)

    Google Scholar 

  64. Zhu, L., Wang, X., Chang, Y., Li, J., Huang, T., Tian, Y.: Event-based video reconstruction via potential-assisted spiking neural network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3594–3604 (2022)

    Google Scholar 

  65. Zhu, S., Yang, T., Mendieta, M., Chen, C.: A3D: adaptive 3D networks for video action recognition. arXiv preprint arXiv:2011.12384 (2020)

Download references

Acknowledgements

This work was partly supported by Chinese Defense Advanced Research Program (50912020105), NSFC of China under Grants Nos. 62073258 and 62072352, and Natural Science Foundation of Shaanxi Province (2024JC-JCQN-66).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xidian University, Xi’an, China

    Qi Wang, Zhou Xu, Yuming Lin, Jingtao Ye, Hongsheng Li, Guangming Zhu & Liang Zhang

  2. Edith Cowan University, Perth, Australia

    Syed Afaq Ali Shah

  3. University of Western Australia, Perth, Australia

    Mohammed Bennamoun

Authors
  1. Qi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhou Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuming Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jingtao Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongsheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guangming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Syed Afaq Ali Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mohammed Bennamoun
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Liang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liang Zhang .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Hessen, Germany

    Stefan Roth

  4. Princeton University, Palo Alto, CA, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 3755 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Q. et al. (2025). DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognition. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15142. Springer, Cham. https://doi.org/10.1007/978-3-031-72907-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72907-2_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72906-5

  • Online ISBN: 978-3-031-72907-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics